2024-06-10T00:33:35.368 INFO:root:teuthology version: 0.0.1.dev303+g861a8dc 2024-06-10T00:33:35.368 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/misc}} --archive /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747344 --owner scheduled_teuthology@teuthology --name teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi --verbose -- /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747344/orig.config.yaml 2024-06-10T00:33:35.423 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-10T00:33:35.511 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747344 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/misc}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7747344' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 7b7a5d375cf4e4afab96b9d4d229b699ae8304bb owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 178 sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba sleep_before_teardown: 0 subset: 130/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 7b7a5d375cf4e4afab96b9d4d229b699ae8304bb syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi037.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJGRT96cWEBMXjnmgOJIBzPKEmhMJVfeV3GamlkYABZoZlXhHb47D0jnGGkZ/Bv27zJgwf63VJArhKe3CuLBP8c= smithi094.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOshmxaNEF7iQRVnMnhCWe34Olx9Q1CYiJ6aFNopO7pYgqgI57CD1mb7P4L/NJ3s5ruqmlJXMHWfY9Mp6vutycY= smithi167.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOuXhOcGvmqQ72n12RInORNQbQTIQ+dQdwyYxduY4lHRysAbRmpEh5JMqbr4FvO9x1RuJHMISRe5hm3TG6UQ4DA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - fs/misc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 861a8dcf7aa816a26e13f039336f7ed0a9aec0fa timestamp: 2024-06-09_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-06-10T00:33:35.512 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa; will attempt to use it 2024-06-10T00:33:35.512 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks 2024-06-10T00:33:35.512 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-06-10T00:33:35.514 INFO:teuthology.task.internal:Checking packages... 2024-06-10T00:33:35.537 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3d89accfe3201286f000e065a1faea2a88c7a6ba' 2024-06-10T00:33:35.537 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-10T00:33:35.537 INFO:teuthology.packaging:ref: None 2024-06-10T00:33:35.537 INFO:teuthology.packaging:tag: None 2024-06-10T00:33:35.537 INFO:teuthology.packaging:branch: squid 2024-06-10T00:33:35.537 INFO:teuthology.packaging:sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:33:35.538 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-06-10T00:33:35.717 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2970.g7b7a5d37 2024-06-10T00:33:35.718 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-06-10T00:33:35.724 INFO:teuthology.task.internal:no buildpackages task found 2024-06-10T00:33:35.724 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-06-10T00:33:35.758 INFO:teuthology.task.internal:Saving configuration 2024-06-10T00:33:35.777 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-06-10T00:33:35.783 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-06-10T00:33:35.807 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747344', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-10 00:26:26.131730', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJGRT96cWEBMXjnmgOJIBzPKEmhMJVfeV3GamlkYABZoZlXhHb47D0jnGGkZ/Bv27zJgwf63VJArhKe3CuLBP8c='} 2024-06-10T00:33:35.834 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747344', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-10 00:26:26.134924', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOshmxaNEF7iQRVnMnhCWe34Olx9Q1CYiJ6aFNopO7pYgqgI57CD1mb7P4L/NJ3s5ruqmlJXMHWfY9Mp6vutycY='} 2024-06-10T00:33:35.858 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747344', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-10 00:26:26.133439', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOuXhOcGvmqQ72n12RInORNQbQTIQ+dQdwyYxduY4lHRysAbRmpEh5JMqbr4FvO9x1RuJHMISRe5hm3TG6UQ4DA='} 2024-06-10T00:33:35.858 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-06-10T00:33:35.864 INFO:teuthology.task.internal:roles: ubuntu@smithi037.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-06-10T00:33:35.864 INFO:teuthology.task.internal:roles: ubuntu@smithi094.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-06-10T00:33:35.864 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-06-10T00:33:35.864 INFO:teuthology.run_tasks:Running task console_log... 2024-06-10T00:33:35.973 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4d2ce03be0>, signals=[15]) 2024-06-10T00:33:35.973 INFO:teuthology.run_tasks:Running task internal.connect... 2024-06-10T00:33:35.978 INFO:teuthology.task.internal:Opening connections... 2024-06-10T00:33:35.978 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi037.front.sepia.ceph.com 2024-06-10T00:33:35.980 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T00:33:36.059 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi094.front.sepia.ceph.com 2024-06-10T00:33:36.060 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T00:33:36.135 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi167.front.sepia.ceph.com 2024-06-10T00:33:36.136 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T00:33:36.211 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-06-10T00:33:36.217 DEBUG:teuthology.orchestra.run.smithi037:> uname -m 2024-06-10T00:33:36.235 INFO:teuthology.orchestra.run.smithi037.stdout:x86_64 2024-06-10T00:33:36.236 DEBUG:teuthology.orchestra.run.smithi037:> cat /etc/os-release 2024-06-10T00:33:36.293 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="CentOS Stream" 2024-06-10T00:33:36.293 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="9" 2024-06-10T00:33:36.294 INFO:teuthology.orchestra.run.smithi037.stdout:ID="centos" 2024-06-10T00:33:36.294 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE="rhel fedora" 2024-06-10T00:33:36.294 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="9" 2024-06-10T00:33:36.294 INFO:teuthology.orchestra.run.smithi037.stdout:PLATFORM_ID="platform:el9" 2024-06-10T00:33:36.294 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-10T00:33:36.294 INFO:teuthology.orchestra.run.smithi037.stdout:ANSI_COLOR="0;31" 2024-06-10T00:33:36.294 INFO:teuthology.orchestra.run.smithi037.stdout:LOGO="fedora-logo-icon" 2024-06-10T00:33:36.294 INFO:teuthology.orchestra.run.smithi037.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-10T00:33:36.294 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://centos.org/" 2024-06-10T00:33:36.294 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-10T00:33:36.294 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-10T00:33:36.294 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-10T00:33:36.295 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2024-06-10T00:33:36.319 DEBUG:teuthology.orchestra.run.smithi094:> uname -m 2024-06-10T00:33:36.337 INFO:teuthology.orchestra.run.smithi094.stdout:x86_64 2024-06-10T00:33:36.337 DEBUG:teuthology.orchestra.run.smithi094:> cat /etc/os-release 2024-06-10T00:33:36.393 INFO:teuthology.orchestra.run.smithi094.stdout:NAME="CentOS Stream" 2024-06-10T00:33:36.394 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION="9" 2024-06-10T00:33:36.394 INFO:teuthology.orchestra.run.smithi094.stdout:ID="centos" 2024-06-10T00:33:36.394 INFO:teuthology.orchestra.run.smithi094.stdout:ID_LIKE="rhel fedora" 2024-06-10T00:33:36.394 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_ID="9" 2024-06-10T00:33:36.394 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM_ID="platform:el9" 2024-06-10T00:33:36.394 INFO:teuthology.orchestra.run.smithi094.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-10T00:33:36.394 INFO:teuthology.orchestra.run.smithi094.stdout:ANSI_COLOR="0;31" 2024-06-10T00:33:36.394 INFO:teuthology.orchestra.run.smithi094.stdout:LOGO="fedora-logo-icon" 2024-06-10T00:33:36.394 INFO:teuthology.orchestra.run.smithi094.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-10T00:33:36.394 INFO:teuthology.orchestra.run.smithi094.stdout:HOME_URL="https://centos.org/" 2024-06-10T00:33:36.394 INFO:teuthology.orchestra.run.smithi094.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-10T00:33:36.394 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-10T00:33:36.394 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-10T00:33:36.395 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server 2024-06-10T00:33:36.419 DEBUG:teuthology.orchestra.run.smithi167:> uname -m 2024-06-10T00:33:36.437 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2024-06-10T00:33:36.437 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2024-06-10T00:33:36.494 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Stream" 2024-06-10T00:33:36.494 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="9" 2024-06-10T00:33:36.494 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos" 2024-06-10T00:33:36.494 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora" 2024-06-10T00:33:36.495 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="9" 2024-06-10T00:33:36.495 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el9" 2024-06-10T00:33:36.495 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-10T00:33:36.495 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2024-06-10T00:33:36.495 INFO:teuthology.orchestra.run.smithi167.stdout:LOGO="fedora-logo-icon" 2024-06-10T00:33:36.495 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-10T00:33:36.495 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://centos.org/" 2024-06-10T00:33:36.495 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-10T00:33:36.495 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-10T00:33:36.495 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-10T00:33:36.495 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2024-06-10T00:33:36.519 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-06-10T00:33:36.529 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-06-10T00:33:36.563 INFO:teuthology.task.internal:Checking for old test directory... 2024-06-10T00:33:36.563 DEBUG:teuthology.orchestra.run.smithi037:> test '!' -e /home/ubuntu/cephtest 2024-06-10T00:33:36.566 DEBUG:teuthology.orchestra.run.smithi094:> test '!' -e /home/ubuntu/cephtest 2024-06-10T00:33:36.573 DEBUG:teuthology.orchestra.run.smithi167:> test '!' -e /home/ubuntu/cephtest 2024-06-10T00:33:36.590 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-06-10T00:33:36.595 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-06-10T00:33:36.596 DEBUG:teuthology.orchestra.run.smithi037:> test -z $(ls -A /var/lib/ceph) 2024-06-10T00:33:36.624 DEBUG:teuthology.orchestra.run.smithi094:> test -z $(ls -A /var/lib/ceph) 2024-06-10T00:33:36.626 DEBUG:teuthology.orchestra.run.smithi167:> test -z $(ls -A /var/lib/ceph) 2024-06-10T00:33:36.723 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-06-10T00:33:36.803 INFO:teuthology.run_tasks:Running task kernel... 2024-06-10T00:33:36.818 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-06-10T00:33:36.818 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-06-10T00:33:36.818 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-06-10T00:33:36.819 DEBUG:teuthology.orchestra.run.smithi037:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-10T00:33:36.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T00:33:36.836 DEBUG:teuthology.orchestra.run.smithi037:> uname -r 2024-06-10T00:33:36.894 INFO:teuthology.orchestra.run.smithi037.stdout:5.14.0-437.el9.x86_64 2024-06-10T00:33:36.895 INFO:teuthology.task.kernel:Running kernel on smithi037: 5.14.0-437.el9.x86_64 2024-06-10T00:33:36.895 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum install -y kernel 2024-06-10T00:33:40.435 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.1 MB 00:01 2024-06-10T00:33:47.674 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 3.2 MB/s | 19 MB 00:06 2024-06-10T00:33:52.325 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 7.2 MB/s | 6.2 MB 00:00 2024-06-10T00:33:53.882 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 51 kB/s | 17 kB 00:00 2024-06-10T00:33:54.835 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 26 MB/s | 21 MB 00:00 2024-06-10T00:34:00.216 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-06-10T00:34:02.272 INFO:teuthology.orchestra.run.smithi037.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-06-10T00:34:02.272 INFO:teuthology.orchestra.run.smithi037.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-06-10T00:34:02.372 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-06-10T00:34:02.390 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-06-10T00:34:02.390 INFO:teuthology.orchestra.run.smithi037.stdout: Package Architecture Version Repository Size 2024-06-10T00:34:02.390 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-06-10T00:34:02.390 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-06-10T00:34:02.390 INFO:teuthology.orchestra.run.smithi037.stdout: kernel x86_64 5.14.0-452.el9 baseos 893 k 2024-06-10T00:34:02.390 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-06-10T00:34:02.390 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-core x86_64 5.14.0-452.el9 baseos 16 M 2024-06-10T00:34:02.391 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-modules x86_64 5.14.0-452.el9 baseos 35 M 2024-06-10T00:34:02.391 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-modules-core x86_64 5.14.0-452.el9 baseos 29 M 2024-06-10T00:34:02.391 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:34:02.391 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-06-10T00:34:02.391 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-06-10T00:34:02.391 INFO:teuthology.orchestra.run.smithi037.stdout:Install 4 Packages 2024-06-10T00:34:02.391 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:34:02.391 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 80 M 2024-06-10T00:34:02.391 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 124 M 2024-06-10T00:34:02.391 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-06-10T00:34:02.748 INFO:teuthology.orchestra.run.smithi037.stdout:[MIRROR] kernel-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:34:02.753 INFO:teuthology.orchestra.run.smithi037.stdout:[MIRROR] kernel-core-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:34:02.753 INFO:teuthology.orchestra.run.smithi037.stdout:[MIRROR] kernel-modules-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:34:03.017 INFO:teuthology.orchestra.run.smithi037.stdout:(1/4): kernel-5.14.0-452.el9.x86_64.rpm 2.5 MB/s | 893 kB 00:00 2024-06-10T00:34:03.459 INFO:teuthology.orchestra.run.smithi037.stdout:(2/4): kernel-core-5.14.0-452.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-06-10T00:34:04.009 INFO:teuthology.orchestra.run.smithi037.stdout:(3/4): kernel-modules-core-5.14.0-452.el9.x86_6 29 MB/s | 29 MB 00:00 2024-06-10T00:34:04.468 INFO:teuthology.orchestra.run.smithi037.stdout:(4/4): kernel-modules-5.14.0-452.el9.x86_64.rpm 19 MB/s | 35 MB 00:01 2024-06-10T00:34:04.470 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-06-10T00:34:04.470 INFO:teuthology.orchestra.run.smithi037.stdout:Total 39 MB/s | 80 MB 00:02 2024-06-10T00:34:05.194 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-06-10T00:34:05.601 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-06-10T00:34:05.601 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-06-10T00:34:06.590 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-06-10T00:34:06.591 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-06-10T00:34:09.117 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-06-10T00:34:09.756 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : kernel-modules-core-5.14.0-452.el9.x86_64 1/4 2024-06-10T00:34:09.843 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-10T00:34:12.657 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-10T00:34:12.814 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-10T00:34:17.903 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-10T00:34:17.962 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : kernel-5.14.0-452.el9.x86_64 4/4 2024-06-10T00:34:23.069 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-10T00:34:54.387 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 4/4 2024-06-10T00:34:54.428 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 4/4 2024-06-10T00:34:54.728 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-5.14.0-452.el9.x86_64 4/4 2024-06-10T00:34:54.728 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-5.14.0-452.el9.x86_64 1/4 2024-06-10T00:34:54.728 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-10T00:34:54.728 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-10T00:34:55.513 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-10T00:34:55.513 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:34:55.513 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-06-10T00:34:55.513 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-5.14.0-452.el9.x86_64 2024-06-10T00:34:55.513 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-core-5.14.0-452.el9.x86_64 2024-06-10T00:34:55.513 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-modules-5.14.0-452.el9.x86_64 2024-06-10T00:34:55.513 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-modules-core-5.14.0-452.el9.x86_64 2024-06-10T00:34:55.513 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:34:55.514 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-06-10T00:34:55.869 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q kernel | sort -rV | head -n 1 2024-06-10T00:34:55.960 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-10T00:34:55.961 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-10T00:34:55.961 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-452.el9.x86_64 2024-06-10T00:34:55.961 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-10T00:34:55.961 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-10T00:34:55.961 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-06-10T00:34:55.962 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi037.front.sepia.ceph.com, path=None, version=distro) 2024-06-10T00:34:55.962 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q kernel | sort -rV | head -n 1 2024-06-10T00:34:56.017 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-10T00:34:56.017 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-10T00:34:56.017 DEBUG:teuthology.orchestra.run.smithi037:> sudo rpm -qi grub2-tools 2024-06-10T00:34:56.058 INFO:teuthology.orchestra.run.smithi037.stdout:Name : grub2-tools 2024-06-10T00:34:56.059 INFO:teuthology.orchestra.run.smithi037.stdout:Epoch : 1 2024-06-10T00:34:56.059 INFO:teuthology.orchestra.run.smithi037.stdout:Version : 2.06 2024-06-10T00:34:56.059 INFO:teuthology.orchestra.run.smithi037.stdout:Release : 68.el9 2024-06-10T00:34:56.059 INFO:teuthology.orchestra.run.smithi037.stdout:Architecture: x86_64 2024-06-10T00:34:56.059 INFO:teuthology.orchestra.run.smithi037.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-10T00:34:56.059 INFO:teuthology.orchestra.run.smithi037.stdout:Group : Unspecified 2024-06-10T00:34:56.059 INFO:teuthology.orchestra.run.smithi037.stdout:Size : 8264254 2024-06-10T00:34:56.059 INFO:teuthology.orchestra.run.smithi037.stdout:License : GPLv3+ 2024-06-10T00:34:56.059 INFO:teuthology.orchestra.run.smithi037.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-10T00:34:56.059 INFO:teuthology.orchestra.run.smithi037.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-10T00:34:56.059 INFO:teuthology.orchestra.run.smithi037.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-10T00:34:56.059 INFO:teuthology.orchestra.run.smithi037.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-10T00:34:56.060 INFO:teuthology.orchestra.run.smithi037.stdout:Packager : builder@centos.org 2024-06-10T00:34:56.060 INFO:teuthology.orchestra.run.smithi037.stdout:Vendor : CentOS 2024-06-10T00:34:56.060 INFO:teuthology.orchestra.run.smithi037.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-10T00:34:56.060 INFO:teuthology.orchestra.run.smithi037.stdout:Summary : Support tools for GRUB. 2024-06-10T00:34:56.060 INFO:teuthology.orchestra.run.smithi037.stdout:Description : 2024-06-10T00:34:56.060 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:34:56.060 INFO:teuthology.orchestra.run.smithi037.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-10T00:34:56.060 INFO:teuthology.orchestra.run.smithi037.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-10T00:34:56.060 INFO:teuthology.orchestra.run.smithi037.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-10T00:34:56.060 INFO:teuthology.orchestra.run.smithi037.stdout:hardware devices. 2024-06-10T00:34:56.060 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:34:56.060 INFO:teuthology.orchestra.run.smithi037.stdout:This subpackage provides tools for support of all platforms. 2024-06-10T00:34:56.061 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-10T00:34:56.061 INFO:teuthology.task.kernel:Updating grub on smithi037 to boot 5.14.0-452.el9.x86_64 2024-06-10T00:34:56.061 DEBUG:teuthology.orchestra.run.smithi037:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-10T00:34:56.717 INFO:teuthology.orchestra.run.smithi037.stderr:Generating grub configuration file ... 2024-06-10T00:34:58.094 INFO:teuthology.orchestra.run.smithi037.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-10T00:34:58.124 INFO:teuthology.orchestra.run.smithi037.stderr:done 2024-06-10T00:34:58.126 DEBUG:teuthology.orchestra.run.smithi037:> mktemp 2024-06-10T00:34:58.145 INFO:teuthology.orchestra.run.smithi037.stdout:/tmp/tmp.7G8rjiCxzv 2024-06-10T00:34:58.145 DEBUG:teuthology.orchestra.run.smithi037:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.7G8rjiCxzv 2024-06-10T00:34:58.212 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0666 /tmp/tmp.7G8rjiCxzv 2024-06-10T00:34:58.471 DEBUG:teuthology.orchestra.remote:smithi037:/tmp/tmp.7G8rjiCxzv is 6KB 2024-06-10T00:34:58.519 DEBUG:teuthology.orchestra.run.smithi037:> rm -fr /tmp/tmp.7G8rjiCxzv 2024-06-10T00:34:58.535 DEBUG:teuthology.orchestra.run.smithi037:> sudo /bin/ls /boot/loader/entries || true 2024-06-10T00:34:58.596 INFO:teuthology.orchestra.run.smithi037.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-10T00:34:58.597 INFO:teuthology.orchestra.run.smithi037.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-10T00:34:58.597 INFO:teuthology.orchestra.run.smithi037.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-10T00:34:58.597 INFO:teuthology.orchestra.run.smithi037.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64.conf 2024-06-10T00:34:58.598 DEBUG:teuthology.orchestra.run.smithi037:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64 2024-06-10T00:34:58.747 DEBUG:teuthology.orchestra.run.smithi037:> sudo shutdown -r now 2024-06-10T00:35:28.751 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-06-10T00:35:28.752 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-06-10T00:35:28.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T00:35:47.102 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.37 2024-06-10T00:35:56.104 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-06-10T00:35:56.105 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T00:36:14.494 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.37 2024-06-10T00:36:26.495 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-06-10T00:36:26.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T00:36:26.923 DEBUG:teuthology.orchestra.run.smithi037:> true 2024-06-10T00:36:27.356 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-06-10T00:36:27.356 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-452.el9.x86_64"... 2024-06-10T00:36:27.356 DEBUG:teuthology.orchestra.run.smithi037:> uname -r 2024-06-10T00:36:27.374 INFO:teuthology.orchestra.run.smithi037.stdout:5.14.0-452.el9.x86_64 2024-06-10T00:36:27.375 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-452.el9.x86_64 vs 5.14.0-452.el9.x86_64 2024-06-10T00:36:27.375 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-10T00:36:27.375 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-10T00:36:28.375 DEBUG:teuthology.parallel:result is None 2024-06-10T00:36:28.376 INFO:teuthology.run_tasks:Running task internal.base... 2024-06-10T00:36:28.384 INFO:teuthology.task.internal:Creating test directory... 2024-06-10T00:36:28.384 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-10T00:36:28.387 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-10T00:36:28.390 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-10T00:36:28.407 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-06-10T00:36:28.427 INFO:teuthology.run_tasks:Running task internal.archive... 2024-06-10T00:36:28.486 INFO:teuthology.task.internal:Creating archive directory... 2024-06-10T00:36:28.486 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-10T00:36:28.488 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-10T00:36:28.491 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-10T00:36:28.678 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-06-10T00:36:28.685 INFO:teuthology.task.internal:Enabling coredump saving... 2024-06-10T00:36:28.685 DEBUG:teuthology.orchestra.run.smithi094:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-10T00:36:28.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T00:36:28.702 DEBUG:teuthology.orchestra.run.smithi167:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-10T00:36:28.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T00:36:28.719 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-10T00:36:28.721 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-10T00:36:28.745 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-10T00:36:28.791 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-10T00:36:28.795 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-10T00:36:28.810 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-10T00:36:28.812 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-10T00:36:28.813 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-10T00:36:28.853 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-10T00:36:28.855 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-06-10T00:36:28.863 INFO:teuthology.task.internal:Configuring sudo... 2024-06-10T00:36:28.863 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-10T00:36:28.865 DEBUG:teuthology.orchestra.run.smithi094:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-10T00:36:28.868 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-10T00:36:28.894 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-06-10T00:36:28.904 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-06-10T00:36:28.904 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-10T00:36:28.932 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-10T00:36:28.937 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-10T00:36:28.952 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-10T00:36:29.031 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-10T00:36:29.070 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-10T00:36:29.153 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-10T00:36:29.192 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-06-10T00:36:29.192 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-10T00:36:29.262 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-10T00:36:29.306 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-10T00:36:29.375 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-10T00:36:29.418 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-10T00:36:29.459 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-10T00:36:29.459 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-10T00:36:29.524 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-10T00:36:29.570 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-10T00:36:29.640 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-10T00:36:29.682 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-10T00:36:29.725 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-10T00:36:29.725 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-10T00:36:29.790 DEBUG:teuthology.orchestra.run.smithi037:> sudo service rsyslog restart 2024-06-10T00:36:29.793 DEBUG:teuthology.orchestra.run.smithi094:> sudo service rsyslog restart 2024-06-10T00:36:29.796 DEBUG:teuthology.orchestra.run.smithi167:> sudo service rsyslog restart 2024-06-10T00:36:29.827 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-10T00:36:29.859 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-10T00:36:29.865 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-10T00:36:30.150 INFO:teuthology.run_tasks:Running task internal.timer... 2024-06-10T00:36:30.158 INFO:teuthology.task.internal:Starting timer... 2024-06-10T00:36:30.158 INFO:teuthology.run_tasks:Running task pcp... 2024-06-10T00:36:30.195 INFO:teuthology.run_tasks:Running task selinux... 2024-06-10T00:36:30.238 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-06-10T00:36:30.238 DEBUG:teuthology.orchestra.run.smithi037:> sudo service auditd rotate 2024-06-10T00:36:30.327 INFO:teuthology.orchestra.run.smithi037.stdout:Rotating logs: 2024-06-10T00:36:30.329 DEBUG:teuthology.orchestra.run.smithi094:> sudo service auditd rotate 2024-06-10T00:36:30.395 INFO:teuthology.orchestra.run.smithi094.stdout:Rotating logs: 2024-06-10T00:36:30.398 DEBUG:teuthology.orchestra.run.smithi167:> sudo service auditd rotate 2024-06-10T00:36:30.477 INFO:teuthology.orchestra.run.smithi167.stdout:Rotating logs: 2024-06-10T00:36:30.479 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-06-10T00:36:30.479 DEBUG:teuthology.orchestra.run.smithi037:> /usr/sbin/getenforce 2024-06-10T00:36:30.507 INFO:teuthology.orchestra.run.smithi037.stdout:Permissive 2024-06-10T00:36:30.507 DEBUG:teuthology.orchestra.run.smithi094:> /usr/sbin/getenforce 2024-06-10T00:36:30.583 INFO:teuthology.orchestra.run.smithi094.stdout:Permissive 2024-06-10T00:36:30.584 DEBUG:teuthology.orchestra.run.smithi167:> /usr/sbin/getenforce 2024-06-10T00:36:30.603 INFO:teuthology.orchestra.run.smithi167.stdout:Permissive 2024-06-10T00:36:30.604 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi037.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi094.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi167.front.sepia.ceph.com': 'permissive'} 2024-06-10T00:36:30.604 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-10T00:36:30.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T00:36:30.631 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-10T00:36:30.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T00:36:30.661 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-10T00:36:30.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T00:36:30.685 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-06-10T00:36:30.686 DEBUG:teuthology.orchestra.run.smithi037:> sudo /usr/sbin/setenforce permissive 2024-06-10T00:36:30.742 DEBUG:teuthology.orchestra.run.smithi094:> sudo /usr/sbin/setenforce permissive 2024-06-10T00:36:30.769 DEBUG:teuthology.orchestra.run.smithi167:> sudo /usr/sbin/setenforce permissive 2024-06-10T00:36:30.793 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-06-10T00:36:30.803 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-06-10T00:36:30.806 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-06-10T00:36:30.919 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-06-10T00:36:30.930 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-06-10T00:36:30.931 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi037.front.sepia.ceph.com,smithi094.front.sepia.ceph.com,smithi167.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-06-10T00:41:10.568 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi037.front.sepia.ceph.com'), Remote(name='ubuntu@smithi094.front.sepia.ceph.com'), Remote(name='ubuntu@smithi167.front.sepia.ceph.com')] 2024-06-10T00:41:10.569 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-06-10T00:41:10.570 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T00:41:10.647 DEBUG:teuthology.orchestra.run.smithi037:> true 2024-06-10T00:41:10.725 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-06-10T00:41:10.726 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-06-10T00:41:10.726 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T00:41:10.802 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-06-10T00:41:10.882 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-06-10T00:41:10.883 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-06-10T00:41:10.883 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-10T00:41:10.960 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-06-10T00:41:11.038 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-06-10T00:41:11.038 INFO:teuthology.run_tasks:Running task clock... 2024-06-10T00:41:11.049 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-06-10T00:41:11.050 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-10T00:41:11.050 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-10T00:41:11.052 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-10T00:41:11.053 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-10T00:41:11.055 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-10T00:41:11.055 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-10T00:41:11.081 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-10T00:41:11.082 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-10T00:41:11.096 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-10T00:41:11.096 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-10T00:41:11.108 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-10T00:41:11.120 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: ntpd: command not found 2024-06-10T00:41:11.121 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-10T00:41:11.122 INFO:teuthology.orchestra.run.smithi037.stderr:sudo: ntpd: command not found 2024-06-10T00:41:11.130 INFO:teuthology.orchestra.run.smithi094.stdout:506 Cannot talk to daemon 2024-06-10T00:41:11.133 INFO:teuthology.orchestra.run.smithi037.stdout:506 Cannot talk to daemon 2024-06-10T00:41:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-10T00:41:11.145 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: ntpd: command not found 2024-06-10T00:41:11.148 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-10T00:41:11.156 INFO:teuthology.orchestra.run.smithi167.stdout:506 Cannot talk to daemon 2024-06-10T00:41:11.157 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-10T00:41:11.163 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-10T00:41:11.170 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-10T00:41:11.183 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-10T00:41:11.211 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-06-10T00:41:11.214 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-10T00:41:11.214 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-06-10T00:41:11.214 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:41:11.214 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:41:11.214 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:41:11.214 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:41:11.222 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-06-10T00:41:11.226 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-10T00:41:11.226 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-06-10T00:41:11.226 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:41:11.226 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:41:11.226 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:41:11.226 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:41:11.241 INFO:teuthology.orchestra.run.smithi167.stderr:bash: line 1: ntpq: command not found 2024-06-10T00:41:11.244 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-10T00:41:11.244 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2024-06-10T00:41:11.245 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:41:11.245 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:41:11.245 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:41:11.245 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T00:41:11.245 INFO:teuthology.run_tasks:Running task install... 2024-06-10T00:41:11.253 DEBUG:teuthology.task.install:project ceph 2024-06-10T00:41:11.253 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'}} 2024-06-10T00:41:11.253 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'} 2024-06-10T00:41:11.253 INFO:teuthology.task.install:Using flavor: default 2024-06-10T00:41:11.258 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-06-10T00:41:11.259 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-06-10T00:41:11.259 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-10T00:41:11.260 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:41:11.262 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-10T00:41:11.262 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:41:11.263 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-10T00:41:11.263 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:41:11.401 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-10T00:41:11.401 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-10T00:41:11.411 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-10T00:41:11.411 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-10T00:41:11.480 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-10T00:41:11.480 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-10T00:41:11.611 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-10T00:41:11.611 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-06-10T00:41:11.611 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-10T00:41:11.616 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-10T00:41:11.616 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-10T00:41:11.616 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-10T00:41:11.639 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-10T00:41:11.639 DEBUG:teuthology.orchestra.run.smithi037:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-10T00:41:11.642 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-10T00:41:11.642 DEBUG:teuthology.orchestra.run.smithi094:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-10T00:41:11.706 DEBUG:teuthology.orchestra.run.smithi037:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-10T00:41:11.708 DEBUG:teuthology.orchestra.run.smithi094:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-10T00:41:11.738 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-10T00:41:11.738 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-10T00:41:11.738 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-10T00:41:11.763 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-10T00:41:11.763 DEBUG:teuthology.orchestra.run.smithi167:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-10T00:41:11.782 DEBUG:teuthology.orchestra.run.smithi094:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-10T00:41:11.783 DEBUG:teuthology.orchestra.run.smithi037:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-10T00:41:11.828 DEBUG:teuthology.orchestra.run.smithi167:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-10T00:41:11.846 INFO:teuthology.orchestra.run.smithi094.stdout:check_obsoletes = 1 2024-06-10T00:41:11.848 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum clean all 2024-06-10T00:41:11.849 INFO:teuthology.orchestra.run.smithi037.stdout:check_obsoletes = 1 2024-06-10T00:41:11.851 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum clean all 2024-06-10T00:41:11.900 DEBUG:teuthology.orchestra.run.smithi167:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-10T00:41:11.964 INFO:teuthology.orchestra.run.smithi167.stdout:check_obsoletes = 1 2024-06-10T00:41:11.965 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum clean all 2024-06-10T00:41:12.073 INFO:teuthology.orchestra.run.smithi094.stdout:45 files removed 2024-06-10T00:41:12.096 INFO:teuthology.orchestra.run.smithi037.stdout:45 files removed 2024-06-10T00:41:12.107 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-10T00:41:12.123 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-10T00:41:12.196 INFO:teuthology.orchestra.run.smithi167.stdout:45 files removed 2024-06-10T00:41:12.223 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-10T00:41:13.039 INFO:teuthology.orchestra.run.smithi094.stdout:ceph packages for x86_64 124 kB/s | 83 kB 00:00 2024-06-10T00:41:13.110 INFO:teuthology.orchestra.run.smithi037.stdout:ceph packages for x86_64 119 kB/s | 83 kB 00:00 2024-06-10T00:41:13.159 INFO:teuthology.orchestra.run.smithi167.stdout:ceph packages for x86_64 123 kB/s | 83 kB 00:00 2024-06-10T00:41:13.673 INFO:teuthology.orchestra.run.smithi094.stdout:ceph noarch packages 19 kB/s | 11 kB 00:00 2024-06-10T00:41:13.760 INFO:teuthology.orchestra.run.smithi037.stdout:ceph noarch packages 19 kB/s | 11 kB 00:00 2024-06-10T00:41:13.876 INFO:teuthology.orchestra.run.smithi167.stdout:ceph noarch packages 17 kB/s | 11 kB 00:00 2024-06-10T00:41:14.399 INFO:teuthology.orchestra.run.smithi094.stdout:ceph source packages 2.7 kB/s | 1.9 kB 00:00 2024-06-10T00:41:14.511 INFO:teuthology.orchestra.run.smithi037.stdout:ceph source packages 2.7 kB/s | 1.9 kB 00:00 2024-06-10T00:41:14.636 INFO:teuthology.orchestra.run.smithi167.stdout:ceph source packages 2.6 kB/s | 1.9 kB 00:00 2024-06-10T00:41:16.323 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.1 MB 00:01 2024-06-10T00:41:18.243 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - BaseOS 2.1 MB/s | 8.1 MB 00:03 2024-06-10T00:41:18.739 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 1.9 MB/s | 8.1 MB 00:04 2024-06-10T00:41:19.416 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - AppStream 9.6 MB/s | 19 MB 00:01 2024-06-10T00:41:20.881 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:00 2024-06-10T00:41:25.073 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 6.2 MB 00:02 2024-06-10T00:41:25.123 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-06-10T00:41:26.690 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 17 kB 00:00 2024-06-10T00:41:27.090 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 17 kB 00:00 2024-06-10T00:41:27.577 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 29 MB/s | 21 MB 00:00 2024-06-10T00:41:28.026 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 27 MB/s | 21 MB 00:00 2024-06-10T00:41:32.899 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-06-10T00:41:33.307 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-06-10T00:41:34.619 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:15 2024-06-10T00:41:35.277 INFO:teuthology.orchestra.run.smithi037.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-10T00:41:35.278 INFO:teuthology.orchestra.run.smithi037.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-10T00:41:35.289 INFO:teuthology.orchestra.run.smithi037.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-10T00:41:35.290 INFO:teuthology.orchestra.run.smithi037.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-10T00:41:35.291 INFO:teuthology.orchestra.run.smithi037.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-10T00:41:35.291 INFO:teuthology.orchestra.run.smithi037.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-10T00:41:35.293 INFO:teuthology.orchestra.run.smithi037.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-10T00:41:35.295 INFO:teuthology.orchestra.run.smithi037.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-10T00:41:35.296 INFO:teuthology.orchestra.run.smithi037.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-10T00:41:35.362 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-06-10T00:41:35.378 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-06-10T00:41:35.378 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-06-10T00:41:35.378 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-06-10T00:41:35.378 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-06-10T00:41:35.378 INFO:teuthology.orchestra.run.smithi037.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-10T00:41:35.378 INFO:teuthology.orchestra.run.smithi037.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-10T00:41:35.378 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-10T00:41:35.379 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-10T00:41:35.379 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-10T00:41:35.379 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-10T00:41:35.379 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-10T00:41:35.379 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-10T00:41:35.379 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-10T00:41:35.379 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-10T00:41:35.379 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-10T00:41:35.379 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-10T00:41:35.379 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-10T00:41:35.379 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-10T00:41:35.379 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-10T00:41:35.379 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-10T00:41:35.380 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-10T00:41:35.380 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-10T00:41:35.380 INFO:teuthology.orchestra.run.smithi037.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-10T00:41:35.380 INFO:teuthology.orchestra.run.smithi037.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-10T00:41:35.380 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-10T00:41:35.380 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-10T00:41:35.380 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-10T00:41:35.380 INFO:teuthology.orchestra.run.smithi037.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-10T00:41:35.380 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-10T00:41:35.380 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-10T00:41:35.380 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-10T00:41:35.380 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-10T00:41:35.381 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-10T00:41:35.381 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-10T00:41:35.381 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-10T00:41:35.381 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-10T00:41:35.381 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-10T00:41:35.381 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-10T00:41:35.381 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-10T00:41:35.381 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-10T00:41:35.381 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-10T00:41:35.381 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrading: 2024-06-10T00:41:35.381 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-10T00:41:35.381 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-10T00:41:35.382 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-10T00:41:35.382 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-10T00:41:35.382 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-10T00:41:35.382 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-10T00:41:35.382 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-10T00:41:35.382 INFO:teuthology.orchestra.run.smithi037.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-10T00:41:35.382 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-10T00:41:35.382 INFO:teuthology.orchestra.run.smithi037.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-10T00:41:35.382 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-10T00:41:35.382 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-06-10T00:41:35.382 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-10T00:41:35.382 INFO:teuthology.orchestra.run.smithi037.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-10T00:41:35.382 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-10T00:41:35.383 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-10T00:41:35.383 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-10T00:41:35.383 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-10T00:41:35.383 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-10T00:41:35.383 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-10T00:41:35.383 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-10T00:41:35.383 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-10T00:41:35.383 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-10T00:41:35.383 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-10T00:41:35.383 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-10T00:41:35.383 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-10T00:41:35.383 INFO:teuthology.orchestra.run.smithi037.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-10T00:41:35.383 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-10T00:41:35.384 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-10T00:41:35.384 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-10T00:41:35.384 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-10T00:41:35.384 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-10T00:41:35.384 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-10T00:41:35.384 INFO:teuthology.orchestra.run.smithi037.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-10T00:41:35.384 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-10T00:41:35.384 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-10T00:41:35.384 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-10T00:41:35.384 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-10T00:41:35.384 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-10T00:41:35.384 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-10T00:41:35.384 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-10T00:41:35.384 INFO:teuthology.orchestra.run.smithi037.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-10T00:41:35.385 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-10T00:41:35.385 INFO:teuthology.orchestra.run.smithi037.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-10T00:41:35.385 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-10T00:41:35.385 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-10T00:41:35.385 INFO:teuthology.orchestra.run.smithi037.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-10T00:41:35.385 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-10T00:41:35.385 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-10T00:41:35.385 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-10T00:41:35.385 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-10T00:41:35.385 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-10T00:41:35.386 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-10T00:41:35.386 INFO:teuthology.orchestra.run.smithi037.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-10T00:41:35.386 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-10T00:41:35.386 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-10T00:41:35.386 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-10T00:41:35.386 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-10T00:41:35.386 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-10T00:41:35.386 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-10T00:41:35.386 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-10T00:41:35.386 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-10T00:41:35.386 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-10T00:41:35.386 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-10T00:41:35.387 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-10T00:41:35.387 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-10T00:41:35.387 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-10T00:41:35.387 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth noarch 1:2.29.0-2.el9 epel 205 k 2024-06-10T00:41:35.387 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-10T00:41:35.387 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-10T00:41:35.387 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-10T00:41:35.387 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-10T00:41:35.387 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-10T00:41:35.387 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-10T00:41:35.387 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-10T00:41:35.387 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-10T00:41:35.387 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-10T00:41:35.388 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-10T00:41:35.388 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-10T00:41:35.388 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-10T00:41:35.388 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-10T00:41:35.388 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-10T00:41:35.388 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-10T00:41:35.388 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-10T00:41:35.388 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-10T00:41:35.388 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-10T00:41:35.388 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-10T00:41:35.388 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-10T00:41:35.388 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-10T00:41:35.388 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-10T00:41:35.389 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-10T00:41:35.389 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-10T00:41:35.389 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-10T00:41:35.389 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-10T00:41:35.389 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-10T00:41:35.389 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-10T00:41:35.389 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-10T00:41:35.389 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-10T00:41:35.389 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-10T00:41:35.389 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-10T00:41:35.389 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-10T00:41:35.389 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-10T00:41:35.389 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-10T00:41:35.390 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-10T00:41:35.390 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-10T00:41:35.390 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-10T00:41:35.390 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-10T00:41:35.390 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-10T00:41:35.390 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-10T00:41:35.390 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-10T00:41:35.390 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-10T00:41:35.390 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-10T00:41:35.391 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-10T00:41:35.391 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-10T00:41:35.391 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-10T00:41:35.391 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-10T00:41:35.391 INFO:teuthology.orchestra.run.smithi037.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-10T00:41:35.391 INFO:teuthology.orchestra.run.smithi037.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-10T00:41:35.391 INFO:teuthology.orchestra.run.smithi037.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-10T00:41:35.391 INFO:teuthology.orchestra.run.smithi037.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-10T00:41:35.391 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-10T00:41:35.391 INFO:teuthology.orchestra.run.smithi037.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-10T00:41:35.391 INFO:teuthology.orchestra.run.smithi037.stdout:Installing weak dependencies: 2024-06-10T00:41:35.392 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-10T00:41:35.392 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:41:35.392 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-06-10T00:41:35.392 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-06-10T00:41:35.392 INFO:teuthology.orchestra.run.smithi037.stdout:Install 148 Packages 2024-06-10T00:41:35.392 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrade 11 Packages 2024-06-10T00:41:35.392 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:41:35.393 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 225 M 2024-06-10T00:41:35.393 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-06-10T00:41:35.567 INFO:teuthology.orchestra.run.smithi167.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-10T00:41:35.568 INFO:teuthology.orchestra.run.smithi167.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-10T00:41:35.578 INFO:teuthology.orchestra.run.smithi167.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-10T00:41:35.579 INFO:teuthology.orchestra.run.smithi167.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-10T00:41:35.580 INFO:teuthology.orchestra.run.smithi167.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-10T00:41:35.581 INFO:teuthology.orchestra.run.smithi167.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-10T00:41:35.582 INFO:teuthology.orchestra.run.smithi167.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-10T00:41:35.584 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-10T00:41:35.585 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-10T00:41:35.648 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-06-10T00:41:35.673 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-06-10T00:41:35.673 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2024-06-10T00:41:35.673 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-06-10T00:41:35.674 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-06-10T00:41:35.674 INFO:teuthology.orchestra.run.smithi167.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-10T00:41:35.674 INFO:teuthology.orchestra.run.smithi167.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-10T00:41:35.674 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-10T00:41:35.674 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-10T00:41:35.674 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-10T00:41:35.674 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-10T00:41:35.674 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-10T00:41:35.674 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-10T00:41:35.675 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-10T00:41:35.675 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-10T00:41:35.675 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-10T00:41:35.675 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-10T00:41:35.675 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-10T00:41:35.675 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-10T00:41:35.675 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-10T00:41:35.675 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-10T00:41:35.675 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-10T00:41:35.675 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-10T00:41:35.676 INFO:teuthology.orchestra.run.smithi167.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-10T00:41:35.676 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-10T00:41:35.676 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-10T00:41:35.676 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-10T00:41:35.676 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-10T00:41:35.676 INFO:teuthology.orchestra.run.smithi167.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-10T00:41:35.676 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-10T00:41:35.676 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-10T00:41:35.676 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-10T00:41:35.676 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-10T00:41:35.677 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-10T00:41:35.677 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-10T00:41:35.677 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-10T00:41:35.677 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-10T00:41:35.677 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-10T00:41:35.677 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-10T00:41:35.677 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-10T00:41:35.677 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-10T00:41:35.677 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-10T00:41:35.678 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrading: 2024-06-10T00:41:35.678 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-10T00:41:35.678 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-10T00:41:35.678 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-10T00:41:35.678 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-10T00:41:35.678 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-10T00:41:35.678 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-10T00:41:35.678 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-10T00:41:35.678 INFO:teuthology.orchestra.run.smithi167.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-10T00:41:35.678 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-10T00:41:35.678 INFO:teuthology.orchestra.run.smithi167.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-10T00:41:35.679 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-10T00:41:35.679 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2024-06-10T00:41:35.679 INFO:teuthology.orchestra.run.smithi167.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-10T00:41:35.679 INFO:teuthology.orchestra.run.smithi167.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-10T00:41:35.679 INFO:teuthology.orchestra.run.smithi167.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-10T00:41:35.679 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-10T00:41:35.679 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-10T00:41:35.679 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-10T00:41:35.679 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-10T00:41:35.679 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-10T00:41:35.680 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-10T00:41:35.680 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-10T00:41:35.680 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-10T00:41:35.680 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-10T00:41:35.680 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-10T00:41:35.680 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-10T00:41:35.680 INFO:teuthology.orchestra.run.smithi167.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-10T00:41:35.680 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-10T00:41:35.680 INFO:teuthology.orchestra.run.smithi167.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-10T00:41:35.680 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-10T00:41:35.680 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-10T00:41:35.681 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-10T00:41:35.681 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-10T00:41:35.681 INFO:teuthology.orchestra.run.smithi167.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-10T00:41:35.681 INFO:teuthology.orchestra.run.smithi167.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-10T00:41:35.681 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-10T00:41:35.681 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-10T00:41:35.681 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-10T00:41:35.681 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-10T00:41:35.681 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-10T00:41:35.681 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-10T00:41:35.681 INFO:teuthology.orchestra.run.smithi167.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-10T00:41:35.682 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-10T00:41:35.682 INFO:teuthology.orchestra.run.smithi167.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-10T00:41:35.682 INFO:teuthology.orchestra.run.smithi167.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-10T00:41:35.682 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-10T00:41:35.682 INFO:teuthology.orchestra.run.smithi167.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-10T00:41:35.682 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-10T00:41:35.682 INFO:teuthology.orchestra.run.smithi167.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-10T00:41:35.682 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-10T00:41:35.682 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-10T00:41:35.682 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-10T00:41:35.683 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-10T00:41:35.683 INFO:teuthology.orchestra.run.smithi167.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-10T00:41:35.683 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-10T00:41:35.683 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-10T00:41:35.683 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-10T00:41:35.683 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-10T00:41:35.683 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-10T00:41:35.683 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-10T00:41:35.683 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-10T00:41:35.683 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-10T00:41:35.684 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-10T00:41:35.684 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-10T00:41:35.684 INFO:teuthology.orchestra.run.smithi167.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-10T00:41:35.684 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-10T00:41:35.684 INFO:teuthology.orchestra.run.smithi167.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-10T00:41:35.684 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth noarch 1:2.29.0-2.el9 epel 205 k 2024-06-10T00:41:35.684 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-10T00:41:35.684 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-10T00:41:35.684 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-10T00:41:35.684 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-10T00:41:35.685 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-10T00:41:35.685 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-10T00:41:35.685 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-10T00:41:35.685 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-10T00:41:35.685 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-10T00:41:35.685 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-10T00:41:35.685 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-10T00:41:35.685 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-10T00:41:35.685 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-10T00:41:35.685 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-10T00:41:35.686 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-10T00:41:35.686 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-10T00:41:35.686 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-10T00:41:35.686 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-10T00:41:35.686 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-10T00:41:35.686 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-10T00:41:35.686 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-10T00:41:35.686 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-10T00:41:35.686 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-10T00:41:35.686 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-10T00:41:35.686 INFO:teuthology.orchestra.run.smithi167.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-10T00:41:35.687 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-10T00:41:35.687 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-10T00:41:35.687 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-10T00:41:35.687 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-10T00:41:35.687 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-10T00:41:35.687 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-10T00:41:35.687 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-10T00:41:35.687 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-10T00:41:35.687 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-10T00:41:35.687 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-10T00:41:35.687 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-10T00:41:35.688 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-10T00:41:35.688 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-10T00:41:35.688 INFO:teuthology.orchestra.run.smithi167.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-10T00:41:35.688 INFO:teuthology.orchestra.run.smithi167.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-10T00:41:35.688 INFO:teuthology.orchestra.run.smithi167.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-10T00:41:35.688 INFO:teuthology.orchestra.run.smithi167.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-10T00:41:35.688 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-10T00:41:35.688 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-10T00:41:35.688 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-10T00:41:35.688 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-10T00:41:35.688 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-10T00:41:35.688 INFO:teuthology.orchestra.run.smithi167.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-10T00:41:35.689 INFO:teuthology.orchestra.run.smithi167.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-10T00:41:35.689 INFO:teuthology.orchestra.run.smithi167.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-10T00:41:35.689 INFO:teuthology.orchestra.run.smithi167.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-10T00:41:35.689 INFO:teuthology.orchestra.run.smithi167.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-10T00:41:35.689 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-10T00:41:35.689 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-10T00:41:35.689 INFO:teuthology.orchestra.run.smithi167.stdout:Installing weak dependencies: 2024-06-10T00:41:35.689 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-10T00:41:35.689 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-10T00:41:35.689 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-06-10T00:41:35.689 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-06-10T00:41:35.689 INFO:teuthology.orchestra.run.smithi167.stdout:Install 148 Packages 2024-06-10T00:41:35.689 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrade 11 Packages 2024-06-10T00:41:35.690 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-10T00:41:35.690 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 225 M 2024-06-10T00:41:35.690 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-06-10T00:41:36.358 INFO:teuthology.orchestra.run.smithi167.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-06-10T00:41:36.467 INFO:teuthology.orchestra.run.smithi037.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-06-10T00:41:37.759 INFO:teuthology.orchestra.run.smithi037.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 689 kB/s | 890 kB 00:01 2024-06-10T00:41:37.901 INFO:teuthology.orchestra.run.smithi037.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 1.0 MB/s | 145 kB 00:00 2024-06-10T00:41:38.058 INFO:teuthology.orchestra.run.smithi167.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 523 kB/s | 890 kB 00:01 2024-06-10T00:41:38.309 INFO:teuthology.orchestra.run.smithi167.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 580 kB/s | 145 kB 00:00 2024-06-10T00:41:39.335 INFO:teuthology.orchestra.run.smithi037.stdout:(4/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 1.6 MB/s | 2.4 MB 00:01 2024-06-10T00:41:39.601 INFO:teuthology.orchestra.run.smithi037.stdout:(5/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 1.5 MB/s | 5.4 MB 00:03 2024-06-10T00:41:40.185 INFO:teuthology.orchestra.run.smithi037.stdout:(6/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 1.8 MB/s | 1.5 MB 00:00 2024-06-10T00:41:40.567 INFO:teuthology.orchestra.run.smithi167.stdout:(4/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 1.0 MB/s | 2.4 MB 00:02 2024-06-10T00:41:40.627 INFO:teuthology.orchestra.run.smithi037.stdout:(7/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 4.6 MB/s | 4.7 MB 00:01 2024-06-10T00:41:41.053 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - CRB 2.2 MB/s | 6.2 MB 00:02 2024-06-10T00:41:41.619 INFO:teuthology.orchestra.run.smithi037.stdout:(8/159): ceph-common-19.0.0-2946.g3d89accf.el9. 3.8 MB/s | 21 MB 00:05 2024-06-10T00:41:41.686 INFO:teuthology.orchestra.run.smithi037.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 379 kB/s | 25 kB 00:00 2024-06-10T00:41:41.909 INFO:teuthology.orchestra.run.smithi167.stdout:(5/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 1.1 MB/s | 1.5 MB 00:01 2024-06-10T00:41:43.028 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - Extras packages 21 kB/s | 17 kB 00:00 2024-06-10T00:41:43.111 INFO:teuthology.orchestra.run.smithi037.stdout:(10/159): ceph-radosgw-19.0.0-2946.g3d89accf.el 4.1 MB/s | 10 MB 00:02 2024-06-10T00:41:43.195 INFO:teuthology.orchestra.run.smithi037.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 2.6 MB/s | 221 kB 00:00 2024-06-10T00:41:43.262 INFO:teuthology.orchestra.run.smithi037.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 405 kB/s | 27 kB 00:00 2024-06-10T00:41:43.352 INFO:teuthology.orchestra.run.smithi037.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 353 kB/s | 32 kB 00:00 2024-06-10T00:41:43.562 INFO:teuthology.orchestra.run.smithi037.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 3.5 MB/s | 747 kB 00:00 2024-06-10T00:41:43.646 INFO:teuthology.orchestra.run.smithi037.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 1.9 MB/s | 163 kB 00:00 2024-06-10T00:41:43.721 INFO:teuthology.orchestra.run.smithi037.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 1.7 MB/s | 127 kB 00:00 2024-06-10T00:41:43.871 INFO:teuthology.orchestra.run.smithi037.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 3.3 MB/s | 501 kB 00:00 2024-06-10T00:41:43.948 INFO:teuthology.orchestra.run.smithi094.stdout:Extra Packages for Enterprise Linux 27 MB/s | 21 MB 00:00 2024-06-10T00:41:44.913 INFO:teuthology.orchestra.run.smithi037.stdout:(18/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 5.1 MB/s | 5.3 MB 00:01 2024-06-10T00:41:44.997 INFO:teuthology.orchestra.run.smithi037.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 543 kB/s | 45 kB 00:00 2024-06-10T00:41:45.089 INFO:teuthology.orchestra.run.smithi037.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 1.4 MB/s | 135 kB 00:00 2024-06-10T00:41:45.142 INFO:teuthology.orchestra.run.smithi167.stdout:(6/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 1.5 MB/s | 4.7 MB 00:03 2024-06-10T00:41:45.172 INFO:teuthology.orchestra.run.smithi037.stdout:(21/159): python3-cephfs-19.0.0-2946.g3d89accf. 1.9 MB/s | 162 kB 00:00 2024-06-10T00:41:45.273 INFO:teuthology.orchestra.run.smithi037.stdout:(22/159): python3-rados-19.0.0-2946.g3d89accf.e 3.1 MB/s | 321 kB 00:00 2024-06-10T00:41:45.373 INFO:teuthology.orchestra.run.smithi037.stdout:(23/159): python3-rbd-19.0.0-2946.g3d89accf.el9 2.9 MB/s | 300 kB 00:00 2024-06-10T00:41:45.442 INFO:teuthology.orchestra.run.smithi167.stdout:(7/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 579 kB/s | 5.4 MB 00:09 2024-06-10T00:41:45.456 INFO:teuthology.orchestra.run.smithi037.stdout:(24/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.2 MB/s | 99 kB 00:00 2024-06-10T00:41:45.542 INFO:teuthology.orchestra.run.smithi037.stdout:(25/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.0 MB/s | 85 kB 00:00 2024-06-10T00:41:46.182 INFO:teuthology.orchestra.run.smithi037.stdout:(26/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 4.8 MB/s | 3.1 MB 00:00 2024-06-10T00:41:46.266 INFO:teuthology.orchestra.run.smithi037.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 2.0 MB/s | 171 kB 00:00 2024-06-10T00:41:46.349 INFO:teuthology.orchestra.run.smithi037.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 381 kB/s | 32 kB 00:00 2024-06-10T00:41:46.433 INFO:teuthology.orchestra.run.smithi037.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 1.7 MB/s | 142 kB 00:00 2024-06-10T00:41:47.175 INFO:teuthology.orchestra.run.smithi037.stdout:(30/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 8.8 MB/s | 48 MB 00:05 2024-06-10T00:41:47.242 INFO:teuthology.orchestra.run.smithi037.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 4.3 MB/s | 3.5 MB 00:00 2024-06-10T00:41:47.342 INFO:teuthology.orchestra.run.smithi037.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 2.4 MB/s | 249 kB 00:00 2024-06-10T00:41:47.434 INFO:teuthology.orchestra.run.smithi037.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 538 kB/s | 49 kB 00:00 2024-06-10T00:41:47.510 INFO:teuthology.orchestra.run.smithi037.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2946.g3 218 kB/s | 16 kB 00:00 2024-06-10T00:41:47.610 INFO:teuthology.orchestra.run.smithi037.stdout:(35/159): ceph-volume-19.0.0-2946.g3d89accf.el9 2.6 MB/s | 264 kB 00:00 2024-06-10T00:41:47.794 INFO:teuthology.orchestra.run.smithi037.stdout:(36/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 4.1 MB/s | 763 kB 00:00 2024-06-10T00:41:47.994 INFO:teuthology.orchestra.run.smithi037.stdout:(37/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 2.1 MB/s | 17 MB 00:07 2024-06-10T00:41:48.019 INFO:teuthology.orchestra.run.smithi037.stdout:(38/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 81 kB/s | 18 kB 00:00 2024-06-10T00:41:48.080 INFO:teuthology.orchestra.run.smithi037.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:41:48.086 INFO:teuthology.orchestra.run.smithi037.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:41:48.186 INFO:teuthology.orchestra.run.smithi037.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 419 kB/s | 80 kB 00:00 2024-06-10T00:41:48.187 INFO:teuthology.orchestra.run.smithi037.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:41:48.262 INFO:teuthology.orchestra.run.smithi037.stdout:(40/159): libgfortran-11.4.1-3.el9.x86_64.rpm 3.2 MB/s | 803 kB 00:00 2024-06-10T00:41:48.287 INFO:teuthology.orchestra.run.smithi037.stdout:(41/159): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-06-10T00:41:48.370 INFO:teuthology.orchestra.run.smithi037.stdout:(42/159): ceph-mgr-diskprediction-local-19.0.0- 6.2 MB/s | 7.4 MB 00:01 2024-06-10T00:41:48.396 INFO:teuthology.orchestra.run.smithi037.stdout:(43/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.8 MB/s | 253 kB 00:00 2024-06-10T00:41:48.438 INFO:teuthology.orchestra.run.smithi037.stdout:(44/159): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-06-10T00:41:48.471 INFO:teuthology.orchestra.run.smithi037.stdout:(45/159): python3-cryptography-36.0.1-4.el9.x86 6.7 MB/s | 1.2 MB 00:00 2024-06-10T00:41:48.496 INFO:teuthology.orchestra.run.smithi037.stdout:(46/159): python3-ply-3.11-14.el9.noarch.rpm 847 kB/s | 106 kB 00:00 2024-06-10T00:41:48.522 INFO:teuthology.orchestra.run.smithi037.stdout:(47/159): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-06-10T00:41:48.522 INFO:teuthology.orchestra.run.smithi037.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-10T00:41:48.523 INFO:teuthology.orchestra.run.smithi037.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:41:48.556 INFO:teuthology.orchestra.run.smithi037.stdout:(48/159): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-06-10T00:41:48.556 INFO:teuthology.orchestra.run.smithi037.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:41:49.213 INFO:teuthology.orchestra.run.smithi094.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-06-10T00:41:49.289 INFO:teuthology.orchestra.run.smithi037.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 146 kB/s | 107 kB 00:00 2024-06-10T00:41:49.481 INFO:teuthology.orchestra.run.smithi037.stdout:(50/159): automake-1.16.2-8.el9.noarch.rpm 700 kB/s | 689 kB 00:00 2024-06-10T00:41:49.548 INFO:teuthology.orchestra.run.smithi037.stdout:(51/159): bison-3.7.4-5.el9.x86_64.rpm 919 kB/s | 943 kB 00:01 2024-06-10T00:41:49.915 INFO:teuthology.orchestra.run.smithi037.stdout:(52/159): elfutils-libelf-devel-0.191-4.el9.x86 35 kB/s | 22 kB 00:00 2024-06-10T00:41:50.132 INFO:teuthology.orchestra.run.smithi037.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 56 kB/s | 33 kB 00:00 2024-06-10T00:41:50.307 INFO:teuthology.orchestra.run.smithi037.stdout:(54/159): flex-2.6.4-9.el9.x86_64.rpm 380 kB/s | 313 kB 00:00 2024-06-10T00:41:50.682 INFO:teuthology.orchestra.run.smithi037.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 33 kB/s | 18 kB 00:00 2024-06-10T00:41:50.816 INFO:teuthology.orchestra.run.smithi037.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 22 kB/s | 11 kB 00:00 2024-06-10T00:41:50.942 INFO:teuthology.orchestra.run.smithi167.stdout:(8/159): ceph-radosgw-19.0.0-2946.g3d89accf.el9 1.9 MB/s | 10 MB 00:05 2024-06-10T00:41:51.001 INFO:teuthology.orchestra.run.smithi167.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 433 kB/s | 25 kB 00:00 2024-06-10T00:41:51.008 INFO:teuthology.orchestra.run.smithi037.stdout:(57/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.7 MB/s | 3.0 MB 00:01 2024-06-10T00:41:51.385 INFO:teuthology.orchestra.run.smithi167.stdout:(10/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 2.7 MB/s | 17 MB 00:06 2024-06-10T00:41:51.483 INFO:teuthology.orchestra.run.smithi037.stdout:(58/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 200 kB/s | 160 kB 00:00 2024-06-10T00:41:51.485 INFO:teuthology.orchestra.run.smithi167.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 2.2 MB/s | 221 kB 00:00 2024-06-10T00:41:51.509 INFO:teuthology.orchestra.run.smithi037.stdout:(59/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 65 kB/s | 45 kB 00:00 2024-06-10T00:41:51.543 INFO:teuthology.orchestra.run.smithi167.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 467 kB/s | 27 kB 00:00 2024-06-10T00:41:51.556 INFO:teuthology.orchestra.run.smithi094.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-10T00:41:51.557 INFO:teuthology.orchestra.run.smithi094.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-10T00:41:51.568 INFO:teuthology.orchestra.run.smithi094.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-10T00:41:51.568 INFO:teuthology.orchestra.run.smithi094.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-10T00:41:51.569 INFO:teuthology.orchestra.run.smithi094.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-10T00:41:51.570 INFO:teuthology.orchestra.run.smithi094.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-10T00:41:51.572 INFO:teuthology.orchestra.run.smithi094.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-10T00:41:51.574 INFO:teuthology.orchestra.run.smithi094.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-10T00:41:51.575 INFO:teuthology.orchestra.run.smithi094.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-10T00:41:51.602 INFO:teuthology.orchestra.run.smithi167.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 543 kB/s | 32 kB 00:00 2024-06-10T00:41:51.638 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-06-10T00:41:51.657 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-06-10T00:41:51.657 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-06-10T00:41:51.657 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-06-10T00:41:51.657 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-06-10T00:41:51.657 INFO:teuthology.orchestra.run.smithi094.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-10T00:41:51.657 INFO:teuthology.orchestra.run.smithi094.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-10T00:41:51.657 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-10T00:41:51.657 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-10T00:41:51.657 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-10T00:41:51.657 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-10T00:41:51.658 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-10T00:41:51.658 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-10T00:41:51.658 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-10T00:41:51.658 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-10T00:41:51.658 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-10T00:41:51.658 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-10T00:41:51.658 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-10T00:41:51.658 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-10T00:41:51.658 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-10T00:41:51.658 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-10T00:41:51.658 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-10T00:41:51.659 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-10T00:41:51.659 INFO:teuthology.orchestra.run.smithi094.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-10T00:41:51.659 INFO:teuthology.orchestra.run.smithi094.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-10T00:41:51.659 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-10T00:41:51.659 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-10T00:41:51.659 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-10T00:41:51.659 INFO:teuthology.orchestra.run.smithi094.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-10T00:41:51.659 INFO:teuthology.orchestra.run.smithi094.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-10T00:41:51.659 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-10T00:41:51.659 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-10T00:41:51.659 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-10T00:41:51.659 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-10T00:41:51.659 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-10T00:41:51.660 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-10T00:41:51.660 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-10T00:41:51.660 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-10T00:41:51.660 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-10T00:41:51.660 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-10T00:41:51.660 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-10T00:41:51.660 INFO:teuthology.orchestra.run.smithi094.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-10T00:41:51.660 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrading: 2024-06-10T00:41:51.660 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-10T00:41:51.660 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-10T00:41:51.660 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-10T00:41:51.660 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-10T00:41:51.661 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-10T00:41:51.661 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-10T00:41:51.661 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-10T00:41:51.661 INFO:teuthology.orchestra.run.smithi094.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-10T00:41:51.661 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-10T00:41:51.661 INFO:teuthology.orchestra.run.smithi094.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-10T00:41:51.661 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-10T00:41:51.661 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-06-10T00:41:51.661 INFO:teuthology.orchestra.run.smithi094.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-10T00:41:51.661 INFO:teuthology.orchestra.run.smithi094.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-10T00:41:51.661 INFO:teuthology.orchestra.run.smithi094.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-10T00:41:51.661 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-10T00:41:51.661 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-10T00:41:51.661 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-10T00:41:51.662 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-10T00:41:51.662 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-10T00:41:51.662 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-10T00:41:51.662 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-10T00:41:51.662 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-10T00:41:51.662 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-10T00:41:51.662 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-10T00:41:51.662 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-10T00:41:51.662 INFO:teuthology.orchestra.run.smithi094.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-10T00:41:51.662 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-10T00:41:51.662 INFO:teuthology.orchestra.run.smithi094.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-10T00:41:51.663 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-10T00:41:51.663 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-10T00:41:51.663 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-10T00:41:51.663 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-10T00:41:51.663 INFO:teuthology.orchestra.run.smithi094.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-10T00:41:51.663 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-10T00:41:51.663 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-10T00:41:51.663 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-10T00:41:51.663 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-10T00:41:51.663 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-10T00:41:51.663 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-10T00:41:51.663 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-10T00:41:51.663 INFO:teuthology.orchestra.run.smithi094.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-10T00:41:51.664 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-10T00:41:51.664 INFO:teuthology.orchestra.run.smithi094.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-10T00:41:51.664 INFO:teuthology.orchestra.run.smithi094.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-10T00:41:51.664 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-10T00:41:51.664 INFO:teuthology.orchestra.run.smithi094.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-10T00:41:51.664 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-10T00:41:51.664 INFO:teuthology.orchestra.run.smithi094.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-10T00:41:51.664 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-10T00:41:51.664 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-10T00:41:51.664 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-10T00:41:51.664 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-10T00:41:51.664 INFO:teuthology.orchestra.run.smithi094.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-10T00:41:51.665 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-10T00:41:51.665 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-10T00:41:51.665 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-10T00:41:51.665 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-10T00:41:51.665 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-10T00:41:51.665 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-10T00:41:51.665 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-10T00:41:51.665 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-10T00:41:51.665 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-10T00:41:51.665 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-10T00:41:51.665 INFO:teuthology.orchestra.run.smithi094.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-10T00:41:51.665 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-10T00:41:51.665 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-10T00:41:51.666 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth noarch 1:2.29.0-2.el9 epel 205 k 2024-06-10T00:41:51.666 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-10T00:41:51.666 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-10T00:41:51.666 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-10T00:41:51.666 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-10T00:41:51.666 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-10T00:41:51.666 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-10T00:41:51.666 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-10T00:41:51.666 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-10T00:41:51.666 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-10T00:41:51.666 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-10T00:41:51.667 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-10T00:41:51.667 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-10T00:41:51.667 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-10T00:41:51.667 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-10T00:41:51.667 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-10T00:41:51.667 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-10T00:41:51.667 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-10T00:41:51.667 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-10T00:41:51.667 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-10T00:41:51.667 INFO:teuthology.orchestra.run.smithi094.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-10T00:41:51.667 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-10T00:41:51.667 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-10T00:41:51.667 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-10T00:41:51.667 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-10T00:41:51.668 INFO:teuthology.orchestra.run.smithi094.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-10T00:41:51.668 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-10T00:41:51.668 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-10T00:41:51.668 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-10T00:41:51.668 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-10T00:41:51.668 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-10T00:41:51.668 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-10T00:41:51.668 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-10T00:41:51.668 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-10T00:41:51.668 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-10T00:41:51.668 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-10T00:41:51.668 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-10T00:41:51.668 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-10T00:41:51.669 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-10T00:41:51.669 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-10T00:41:51.669 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-10T00:41:51.669 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-10T00:41:51.669 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-10T00:41:51.669 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-10T00:41:51.669 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-10T00:41:51.669 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-10T00:41:51.669 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-10T00:41:51.669 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-10T00:41:51.669 INFO:teuthology.orchestra.run.smithi094.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-10T00:41:51.669 INFO:teuthology.orchestra.run.smithi094.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-10T00:41:51.670 INFO:teuthology.orchestra.run.smithi094.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-10T00:41:51.670 INFO:teuthology.orchestra.run.smithi094.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-10T00:41:51.670 INFO:teuthology.orchestra.run.smithi094.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-10T00:41:51.670 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-10T00:41:51.670 INFO:teuthology.orchestra.run.smithi094.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-10T00:41:51.670 INFO:teuthology.orchestra.run.smithi094.stdout:Installing weak dependencies: 2024-06-10T00:41:51.670 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-10T00:41:51.670 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-10T00:41:51.670 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-06-10T00:41:51.670 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-06-10T00:41:51.670 INFO:teuthology.orchestra.run.smithi094.stdout:Install 148 Packages 2024-06-10T00:41:51.671 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrade 11 Packages 2024-06-10T00:41:51.671 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-10T00:41:51.671 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 225 M 2024-06-10T00:41:51.671 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-06-10T00:41:51.975 INFO:teuthology.orchestra.run.smithi037.stdout:(60/159): librdkafka-1.6.1-102.el9.x86_64.rpm 685 kB/s | 662 kB 00:00 2024-06-10T00:41:51.977 INFO:teuthology.orchestra.run.smithi167.stdout:(14/159): ceph-common-19.0.0-2946.g3d89accf.el9 1.3 MB/s | 21 MB 00:16 2024-06-10T00:41:52.011 INFO:teuthology.orchestra.run.smithi167.stdout:(15/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 1.8 MB/s | 747 kB 00:00 2024-06-10T00:41:52.078 INFO:teuthology.orchestra.run.smithi167.stdout:(16/159): libcephsqlite-19.0.0-2946.g3d89accf.e 1.6 MB/s | 163 kB 00:00 2024-06-10T00:41:52.094 INFO:teuthology.orchestra.run.smithi037.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 44 kB/s | 25 kB 00:00 2024-06-10T00:41:52.112 INFO:teuthology.orchestra.run.smithi167.stdout:(17/159): librados-devel-19.0.0-2946.g3d89accf. 1.2 MB/s | 127 kB 00:00 2024-06-10T00:41:52.296 INFO:teuthology.orchestra.run.smithi167.stdout:(18/159): libradosstriper1-19.0.0-2946.g3d89acc 2.3 MB/s | 501 kB 00:00 2024-06-10T00:41:52.354 INFO:teuthology.orchestra.run.smithi167.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 774 kB/s | 45 kB 00:00 2024-06-10T00:41:52.413 INFO:teuthology.orchestra.run.smithi167.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 2.3 MB/s | 135 kB 00:00 2024-06-10T00:41:52.431 INFO:teuthology.orchestra.run.smithi094.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-06-10T00:41:52.435 INFO:teuthology.orchestra.run.smithi037.stdout:(62/159): libtool-2.4.6-45.el9.x86_64.rpm 612 kB/s | 582 kB 00:00 2024-06-10T00:41:52.513 INFO:teuthology.orchestra.run.smithi167.stdout:(21/159): python3-cephfs-19.0.0-2946.g3d89accf. 1.6 MB/s | 162 kB 00:00 2024-06-10T00:41:52.613 INFO:teuthology.orchestra.run.smithi167.stdout:(22/159): python3-rados-19.0.0-2946.g3d89accf.e 3.1 MB/s | 321 kB 00:00 2024-06-10T00:41:52.685 INFO:teuthology.orchestra.run.smithi037.stdout:(63/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 67 kB/s | 47 kB 00:00 2024-06-10T00:41:52.722 INFO:teuthology.orchestra.run.smithi167.stdout:(23/159): python3-rbd-19.0.0-2946.g3d89accf.el9 2.7 MB/s | 300 kB 00:00 2024-06-10T00:41:52.789 INFO:teuthology.orchestra.run.smithi167.stdout:(24/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.5 MB/s | 99 kB 00:00 2024-06-10T00:41:52.856 INFO:teuthology.orchestra.run.smithi167.stdout:(25/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.3 MB/s | 85 kB 00:00 2024-06-10T00:41:52.873 INFO:teuthology.orchestra.run.smithi094.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 2.0 MB/s | 890 kB 00:00 2024-06-10T00:41:52.940 INFO:teuthology.orchestra.run.smithi094.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 2.1 MB/s | 145 kB 00:00 2024-06-10T00:41:53.010 INFO:teuthology.orchestra.run.smithi037.stdout:(64/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 319 kB/s | 292 kB 00:00 2024-06-10T00:41:53.235 INFO:teuthology.orchestra.run.smithi037.stdout:(65/159): lua-5.4.4-4.el9.x86_64.rpm 236 kB/s | 188 kB 00:00 2024-06-10T00:41:53.328 INFO:teuthology.orchestra.run.smithi037.stdout:(66/159): openblas-0.3.21-2.el9.x86_64.rpm 57 kB/s | 35 kB 00:00 2024-06-10T00:41:53.565 INFO:teuthology.orchestra.run.smithi167.stdout:(26/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 3.7 MB/s | 5.3 MB 00:01 2024-06-10T00:41:53.645 INFO:teuthology.orchestra.run.smithi094.stdout:(4/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 3.4 MB/s | 2.4 MB 00:00 2024-06-10T00:41:53.656 INFO:teuthology.orchestra.run.smithi167.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 1.8 MB/s | 171 kB 00:00 2024-06-10T00:41:53.757 INFO:teuthology.orchestra.run.smithi167.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 317 kB/s | 32 kB 00:00 2024-06-10T00:41:53.824 INFO:teuthology.orchestra.run.smithi094.stdout:(5/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 3.3 MB/s | 5.4 MB 00:01 2024-06-10T00:41:53.882 INFO:teuthology.orchestra.run.smithi167.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 1.1 MB/s | 142 kB 00:00 2024-06-10T00:41:54.016 INFO:teuthology.orchestra.run.smithi167.stdout:(30/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 2.7 MB/s | 3.1 MB 00:01 2024-06-10T00:41:54.091 INFO:teuthology.orchestra.run.smithi094.stdout:(6/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 3.4 MB/s | 1.5 MB 00:00 2024-06-10T00:41:54.194 INFO:teuthology.orchestra.run.smithi037.stdout:(67/159): openblas-openmp-0.3.21-2.el9.x86_64.r 4.0 MB/s | 4.7 MB 00:01 2024-06-10T00:41:54.369 INFO:teuthology.orchestra.run.smithi037.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-06-10T00:41:54.425 INFO:teuthology.orchestra.run.smithi094.stdout:(7/159): ceph-common-19.0.0-2946.g3d89accf.el9. 9.5 MB/s | 21 MB 00:02 2024-06-10T00:41:54.445 INFO:teuthology.orchestra.run.smithi037.stdout:(69/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.7 MB/s | 4.4 MB 00:01 2024-06-10T00:41:54.700 INFO:teuthology.orchestra.run.smithi094.stdout:(8/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 5.4 MB/s | 4.7 MB 00:00 2024-06-10T00:41:54.801 INFO:teuthology.orchestra.run.smithi094.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 252 kB/s | 25 kB 00:00 2024-06-10T00:41:55.049 INFO:teuthology.orchestra.run.smithi167.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 3.0 MB/s | 3.5 MB 00:01 2024-06-10T00:41:55.103 INFO:teuthology.orchestra.run.smithi037.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 182 kB/s | 133 kB 00:00 2024-06-10T00:41:55.212 INFO:teuthology.orchestra.run.smithi037.stdout:(71/159): postgresql-contrib-13.14-1.el9.x86_64 897 kB/s | 911 kB 00:01 2024-06-10T00:41:55.216 INFO:teuthology.orchestra.run.smithi167.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 1.5 MB/s | 249 kB 00:00 2024-06-10T00:41:55.316 INFO:teuthology.orchestra.run.smithi167.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 494 kB/s | 49 kB 00:00 2024-06-10T00:41:55.400 INFO:teuthology.orchestra.run.smithi167.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2946.g3 197 kB/s | 16 kB 00:00 2024-06-10T00:41:55.476 INFO:teuthology.orchestra.run.smithi094.stdout:(10/159): ceph-radosgw-19.0.0-2946.g3d89accf.el 9.7 MB/s | 10 MB 00:01 2024-06-10T00:41:55.501 INFO:teuthology.orchestra.run.smithi167.stdout:(35/159): ceph-volume-19.0.0-2946.g3d89accf.el9 2.6 MB/s | 264 kB 00:00 2024-06-10T00:41:55.543 INFO:teuthology.orchestra.run.smithi094.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 3.3 MB/s | 221 kB 00:00 2024-06-10T00:41:55.602 INFO:teuthology.orchestra.run.smithi094.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 462 kB/s | 27 kB 00:00 2024-06-10T00:41:55.645 INFO:teuthology.orchestra.run.smithi037.stdout:(72/159): postgresql-server-13.14-1.el9.x86_64. 4.8 MB/s | 5.8 MB 00:01 2024-06-10T00:41:55.660 INFO:teuthology.orchestra.run.smithi094.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 546 kB/s | 32 kB 00:00 2024-06-10T00:41:55.767 INFO:teuthology.orchestra.run.smithi167.stdout:(36/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 2.8 MB/s | 763 kB 00:00 2024-06-10T00:41:55.777 INFO:teuthology.orchestra.run.smithi094.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 6.2 MB/s | 747 kB 00:00 2024-06-10T00:41:55.834 INFO:teuthology.orchestra.run.smithi167.stdout:(37/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 273 kB/s | 18 kB 00:00 2024-06-10T00:41:55.844 INFO:teuthology.orchestra.run.smithi094.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 2.4 MB/s | 163 kB 00:00 2024-06-10T00:41:55.875 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:41:55.911 INFO:teuthology.orchestra.run.smithi094.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 1.8 MB/s | 127 kB 00:00 2024-06-10T00:41:55.946 INFO:teuthology.orchestra.run.smithi037.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 119 kB/s | 87 kB 00:00 2024-06-10T00:41:55.987 INFO:teuthology.orchestra.run.smithi094.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 6.5 MB/s | 501 kB 00:00 2024-06-10T00:41:56.129 INFO:teuthology.orchestra.run.smithi037.stdout:(74/159): protobuf-3.14.0-13.el9.x86_64.rpm 1.0 MB/s | 1.0 MB 00:01 2024-06-10T00:41:56.301 INFO:teuthology.orchestra.run.smithi167.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 172 kB/s | 80 kB 00:00 2024-06-10T00:41:56.302 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:41:56.494 INFO:teuthology.orchestra.run.smithi167.stdout:(39/159): ceph-mgr-diskprediction-local-19.0.0- 3.0 MB/s | 7.4 MB 00:02 2024-06-10T00:41:56.495 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:41:56.554 INFO:teuthology.orchestra.run.smithi094.stdout:(18/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 9.4 MB/s | 5.3 MB 00:00 2024-06-10T00:41:56.596 INFO:teuthology.orchestra.run.smithi094.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 1.1 MB/s | 45 kB 00:00 2024-06-10T00:41:56.646 INFO:teuthology.orchestra.run.smithi037.stdout:(75/159): python3-colorama-0.4.6-3.el9.noarch.r 87 kB/s | 61 kB 00:00 2024-06-10T00:41:56.805 INFO:teuthology.orchestra.run.smithi094.stdout:(20/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 6.2 MB/s | 17 MB 00:02 2024-06-10T00:41:56.830 INFO:teuthology.orchestra.run.smithi094.stdout:(21/159): python3-ceph-common-19.0.0-2946.g3d89 577 kB/s | 135 kB 00:00 2024-06-10T00:41:56.843 INFO:teuthology.orchestra.run.smithi167.stdout:(40/159): libgfortran-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 803 kB 00:00 2024-06-10T00:41:56.864 INFO:teuthology.orchestra.run.smithi094.stdout:(22/159): python3-cephfs-19.0.0-2946.g3d89accf. 2.7 MB/s | 162 kB 00:00 2024-06-10T00:41:56.871 INFO:teuthology.orchestra.run.smithi037.stdout:(76/159): python3-babel-2.9.1-2.el9.noarch.rpm 4.9 MB/s | 6.0 MB 00:01 2024-06-10T00:41:56.898 INFO:teuthology.orchestra.run.smithi094.stdout:(23/159): python3-rados-19.0.0-2946.g3d89accf.e 4.7 MB/s | 321 kB 00:00 2024-06-10T00:41:56.940 INFO:teuthology.orchestra.run.smithi094.stdout:(24/159): python3-rbd-19.0.0-2946.g3d89accf.el9 3.9 MB/s | 300 kB 00:00 2024-06-10T00:41:56.952 INFO:teuthology.orchestra.run.smithi167.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-06-10T00:41:56.963 INFO:teuthology.orchestra.run.smithi037.stdout:(77/159): python3-devel-3.9.18-3.el9.x86_64.rpm 293 kB/s | 244 kB 00:00 2024-06-10T00:41:56.965 INFO:teuthology.orchestra.run.smithi094.stdout:(25/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.4 MB/s | 99 kB 00:00 2024-06-10T00:41:56.994 INFO:teuthology.orchestra.run.smithi167.stdout:(42/159): libquadmath-11.4.1-3.el9.x86_64.rpm 390 kB/s | 195 kB 00:00 2024-06-10T00:41:57.015 INFO:teuthology.orchestra.run.smithi094.stdout:(26/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.1 MB/s | 85 kB 00:00 2024-06-10T00:41:57.099 INFO:teuthology.orchestra.run.smithi094.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 2.0 MB/s | 171 kB 00:00 2024-06-10T00:41:57.103 INFO:teuthology.orchestra.run.smithi167.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 985 kB/s | 106 kB 00:00 2024-06-10T00:41:57.191 INFO:teuthology.orchestra.run.smithi094.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 346 kB/s | 32 kB 00:00 2024-06-10T00:41:57.211 INFO:teuthology.orchestra.run.smithi167.stdout:(44/159): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-06-10T00:41:57.254 INFO:teuthology.orchestra.run.smithi167.stdout:(45/159): python3-cryptography-36.0.1-4.el9.x86 4.1 MB/s | 1.2 MB 00:00 2024-06-10T00:41:57.267 INFO:teuthology.orchestra.run.smithi094.stdout:(29/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 10 MB/s | 3.1 MB 00:00 2024-06-10T00:41:57.292 INFO:teuthology.orchestra.run.smithi094.stdout:(30/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 1.4 MB/s | 142 kB 00:00 2024-06-10T00:41:57.312 INFO:teuthology.orchestra.run.smithi167.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-06-10T00:41:57.313 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-10T00:41:57.363 INFO:teuthology.orchestra.run.smithi167.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-06-10T00:41:57.364 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:41:57.505 INFO:teuthology.orchestra.run.smithi037.stdout:(78/159): python3-jinja2-2.11.3-6.el9.noarch.rp 289 kB/s | 248 kB 00:00 2024-06-10T00:41:57.522 INFO:teuthology.orchestra.run.smithi037.stdout:(79/159): python3-jsonpatch-1.21-16.el9.noarch. 41 kB/s | 26 kB 00:00 2024-06-10T00:41:57.539 INFO:teuthology.orchestra.run.smithi037.stdout:(80/159): python3-jsonpointer-2.0-4.el9.noarch. 33 kB/s | 19 kB 00:00 2024-06-10T00:41:57.601 INFO:teuthology.orchestra.run.smithi094.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 11 MB/s | 3.5 MB 00:00 2024-06-10T00:41:57.668 INFO:teuthology.orchestra.run.smithi094.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 3.6 MB/s | 249 kB 00:00 2024-06-10T00:41:57.718 INFO:teuthology.orchestra.run.smithi094.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 988 kB/s | 49 kB 00:00 2024-06-10T00:41:57.777 INFO:teuthology.orchestra.run.smithi094.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2946.g3 282 kB/s | 16 kB 00:00 2024-06-10T00:41:57.847 INFO:teuthology.orchestra.run.smithi094.stdout:(35/159): ceph-volume-19.0.0-2946.g3d89accf.el9 3.9 MB/s | 264 kB 00:00 2024-06-10T00:41:57.953 INFO:teuthology.orchestra.run.smithi094.stdout:(36/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 6.9 MB/s | 763 kB 00:00 2024-06-10T00:41:58.019 INFO:teuthology.orchestra.run.smithi094.stdout:(37/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 274 kB/s | 18 kB 00:00 2024-06-10T00:41:58.059 INFO:teuthology.orchestra.run.smithi094.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:41:58.153 INFO:teuthology.orchestra.run.smithi094.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 603 kB/s | 80 kB 00:00 2024-06-10T00:41:58.156 INFO:teuthology.orchestra.run.smithi094.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:41:58.156 INFO:teuthology.orchestra.run.smithi037.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 55 kB/s | 35 kB 00:00 2024-06-10T00:41:58.238 INFO:teuthology.orchestra.run.smithi167.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 744 kB/s | 689 kB 00:00 2024-06-10T00:41:58.239 INFO:teuthology.orchestra.run.smithi094.stdout:(39/159): ceph-mgr-diskprediction-local-19.0.0- 7.8 MB/s | 7.4 MB 00:00 2024-06-10T00:41:58.241 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:41:58.242 INFO:teuthology.orchestra.run.smithi094.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:41:58.297 INFO:teuthology.orchestra.run.smithi167.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-06-10T00:41:58.306 INFO:teuthology.orchestra.run.smithi037.stdout:(82/159): python3-mako-1.1.4-6.el9.noarch.rpm 215 kB/s | 172 kB 00:00 2024-06-10T00:41:58.340 INFO:teuthology.orchestra.run.smithi094.stdout:(40/159): libgfortran-11.4.1-3.el9.x86_64.rpm 4.2 MB/s | 803 kB 00:00 2024-06-10T00:41:58.347 INFO:teuthology.orchestra.run.smithi167.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 989 kB/s | 107 kB 00:00 2024-06-10T00:41:58.365 INFO:teuthology.orchestra.run.smithi094.stdout:(41/159): libquadmath-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 195 kB 00:00 2024-06-10T00:41:58.390 INFO:teuthology.orchestra.run.smithi094.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-06-10T00:41:58.397 INFO:teuthology.orchestra.run.smithi167.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 216 kB/s | 22 kB 00:00 2024-06-10T00:41:58.432 INFO:teuthology.orchestra.run.smithi094.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-06-10T00:41:58.472 INFO:teuthology.orchestra.run.smithi167.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-06-10T00:41:58.474 INFO:teuthology.orchestra.run.smithi094.stdout:(44/159): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-06-10T00:41:58.498 INFO:teuthology.orchestra.run.smithi167.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 326 kB/s | 33 kB 00:00 2024-06-10T00:41:58.516 INFO:teuthology.orchestra.run.smithi094.stdout:(45/159): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-06-10T00:41:58.558 INFO:teuthology.orchestra.run.smithi094.stdout:(46/159): python3-urllib3-1.26.5-5.el9.noarch.r 5.0 MB/s | 215 kB 00:00 2024-06-10T00:41:58.560 INFO:teuthology.orchestra.run.smithi094.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-10T00:41:58.671 INFO:teuthology.orchestra.run.smithi094.stdout:(47/159): python3-cryptography-36.0.1-4.el9.x86 4.1 MB/s | 1.2 MB 00:00 2024-06-10T00:41:58.671 INFO:teuthology.orchestra.run.smithi094.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:41:58.781 INFO:teuthology.orchestra.run.smithi167.stdout:(54/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.7 MB/s | 3.0 MB 00:00 2024-06-10T00:41:58.807 INFO:teuthology.orchestra.run.smithi167.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 59 kB/s | 18 kB 00:00 2024-06-10T00:41:58.807 INFO:teuthology.orchestra.run.smithi037.stdout:(83/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.0 MB/s | 5.1 MB 00:01 2024-06-10T00:41:58.882 INFO:teuthology.orchestra.run.smithi167.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-06-10T00:41:58.900 INFO:teuthology.orchestra.run.smithi094.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 2.0 MB/s | 689 kB 00:00 2024-06-10T00:41:58.901 INFO:teuthology.orchestra.run.smithi094.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:41:58.907 INFO:teuthology.orchestra.run.smithi167.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-06-10T00:41:58.967 INFO:teuthology.orchestra.run.smithi094.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 3.1 MB/s | 943 kB 00:00 2024-06-10T00:41:58.982 INFO:teuthology.orchestra.run.smithi037.stdout:(84/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 304 kB/s | 250 kB 00:00 2024-06-10T00:41:58.982 INFO:teuthology.orchestra.run.smithi167.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 452 kB/s | 45 kB 00:00 2024-06-10T00:41:58.992 INFO:teuthology.orchestra.run.smithi094.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-06-10T00:41:59.009 INFO:teuthology.orchestra.run.smithi094.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 520 kB/s | 22 kB 00:00 2024-06-10T00:41:59.024 INFO:teuthology.orchestra.run.smithi167.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-06-10T00:41:59.060 INFO:teuthology.orchestra.run.smithi094.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 655 kB/s | 33 kB 00:00 2024-06-10T00:41:59.100 INFO:teuthology.orchestra.run.smithi167.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 4.9 MB/s | 582 kB 00:00 2024-06-10T00:41:59.118 INFO:teuthology.orchestra.run.smithi094.stdout:(53/159): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-06-10T00:41:59.133 INFO:teuthology.orchestra.run.smithi167.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 234 kB/s | 25 kB 00:00 2024-06-10T00:41:59.140 INFO:teuthology.orchestra.run.smithi037.stdout:(85/159): python3-oauthlib-3.1.1-5.el9.noarch.r 266 kB/s | 222 kB 00:00 2024-06-10T00:41:59.177 INFO:teuthology.orchestra.run.smithi094.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 313 kB/s | 18 kB 00:00 2024-06-10T00:41:59.200 INFO:teuthology.orchestra.run.smithi167.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-06-10T00:41:59.236 INFO:teuthology.orchestra.run.smithi094.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 191 kB/s | 11 kB 00:00 2024-06-10T00:41:59.242 INFO:teuthology.orchestra.run.smithi167.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-06-10T00:41:59.310 INFO:teuthology.orchestra.run.smithi167.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-06-10T00:41:59.328 INFO:teuthology.orchestra.run.smithi094.stdout:(56/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-06-10T00:41:59.343 INFO:teuthology.orchestra.run.smithi167.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 351 kB/s | 35 kB 00:00 2024-06-10T00:41:59.386 INFO:teuthology.orchestra.run.smithi094.stdout:(57/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 777 kB/s | 45 kB 00:00 2024-06-10T00:41:59.462 INFO:teuthology.orchestra.run.smithi094.stdout:(58/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.4 MB/s | 3.0 MB 00:00 2024-06-10T00:41:59.557 INFO:teuthology.orchestra.run.smithi037.stdout:(86/159): python3-packaging-20.9-5.el9.noarch.r 103 kB/s | 77 kB 00:00 2024-06-10T00:41:59.560 INFO:teuthology.orchestra.run.smithi167.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 19 MB/s | 4.7 MB 00:00 2024-06-10T00:41:59.570 INFO:teuthology.orchestra.run.smithi094.stdout:(59/159): libtool-2.4.6-45.el9.x86_64.rpm 5.3 MB/s | 582 kB 00:00 2024-06-10T00:41:59.604 INFO:teuthology.orchestra.run.smithi094.stdout:(60/159): librdkafka-1.6.1-102.el9.x86_64.rpm 3.0 MB/s | 662 kB 00:00 2024-06-10T00:41:59.629 INFO:teuthology.orchestra.run.smithi094.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-06-10T00:41:59.650 INFO:teuthology.orchestra.run.smithi037.stdout:(87/159): python3-prettytable-0.7.2-27.el9.noar 63 kB/s | 42 kB 00:00 2024-06-10T00:41:59.654 INFO:teuthology.orchestra.run.smithi094.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-06-10T00:41:59.688 INFO:teuthology.orchestra.run.smithi094.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-06-10T00:41:59.710 INFO:teuthology.orchestra.run.smithi167.stdout:(67/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-06-10T00:41:59.752 INFO:teuthology.orchestra.run.smithi167.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 8.4 MB/s | 1.6 MB 00:00 2024-06-10T00:41:59.755 INFO:teuthology.orchestra.run.smithi094.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-06-10T00:41:59.780 INFO:teuthology.orchestra.run.smithi094.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 384 kB/s | 35 kB 00:00 2024-06-10T00:41:59.828 INFO:teuthology.orchestra.run.smithi167.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 7.7 MB/s | 911 kB 00:00 2024-06-10T00:41:59.855 INFO:teuthology.orchestra.run.smithi167.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 133 kB 00:00 2024-06-10T00:42:00.008 INFO:teuthology.orchestra.run.smithi037.stdout:(88/159): python3-protobuf-3.14.0-13.el9.noarch 310 kB/s | 269 kB 00:00 2024-06-10T00:42:00.089 INFO:teuthology.orchestra.run.smithi167.stdout:(71/159): postgresql-server-13.14-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-06-10T00:42:00.123 INFO:teuthology.orchestra.run.smithi167.stdout:(72/159): protobuf-3.14.0-13.el9.x86_64.rpm 3.8 MB/s | 1.0 MB 00:00 2024-06-10T00:42:00.189 INFO:teuthology.orchestra.run.smithi167.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 888 kB/s | 87 kB 00:00 2024-06-10T00:42:00.297 INFO:teuthology.orchestra.run.smithi094.stdout:(66/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 8.8 MB/s | 48 MB 00:05 2024-06-10T00:42:00.306 INFO:teuthology.orchestra.run.smithi167.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 525 kB/s | 61 kB 00:00 2024-06-10T00:42:00.367 INFO:teuthology.orchestra.run.smithi037.stdout:(89/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 196 kB/s | 159 kB 00:00 2024-06-10T00:42:00.381 INFO:teuthology.orchestra.run.smithi167.stdout:(75/159): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 6.0 MB 00:00 2024-06-10T00:42:00.407 INFO:teuthology.orchestra.run.smithi167.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-06-10T00:42:00.482 INFO:teuthology.orchestra.run.smithi167.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-06-10T00:42:00.489 INFO:teuthology.orchestra.run.smithi094.stdout:(67/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 6.2 MB/s | 4.4 MB 00:00 2024-06-10T00:42:00.507 INFO:teuthology.orchestra.run.smithi167.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-06-10T00:42:00.518 INFO:teuthology.orchestra.run.smithi037.stdout:(90/159): python3-pyasn1-modules-0.4.8-6.el9.no 321 kB/s | 279 kB 00:00 2024-06-10T00:42:00.583 INFO:teuthology.orchestra.run.smithi167.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-06-10T00:42:00.598 INFO:teuthology.orchestra.run.smithi094.stdout:(68/159): postgresql-contrib-13.14-1.el9.x86_64 8.2 MB/s | 911 kB 00:00 2024-06-10T00:42:00.608 INFO:teuthology.orchestra.run.smithi167.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-06-10T00:42:00.659 INFO:teuthology.orchestra.run.smithi037.stdout:(91/159): python3-pytz-2021.1-5.el9.noarch.rpm 78 kB/s | 51 kB 00:00 2024-06-10T00:42:00.665 INFO:teuthology.orchestra.run.smithi094.stdout:(69/159): openblas-openmp-0.3.21-2.el9.x86_64.r 5.4 MB/s | 4.7 MB 00:00 2024-06-10T00:42:00.683 INFO:teuthology.orchestra.run.smithi167.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 346 kB/s | 35 kB 00:00 2024-06-10T00:42:00.690 INFO:teuthology.orchestra.run.smithi094.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-06-10T00:42:00.724 INFO:teuthology.orchestra.run.smithi094.stdout:(71/159): postgresql-13.14-1.el9.x86_64.rpm 3.7 MB/s | 1.6 MB 00:00 2024-06-10T00:42:00.818 INFO:teuthology.orchestra.run.smithi094.stdout:(72/159): protobuf-3.14.0-13.el9.x86_64.rpm 7.9 MB/s | 1.0 MB 00:00 2024-06-10T00:42:00.849 INFO:teuthology.orchestra.run.smithi094.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 698 kB/s | 87 kB 00:00 2024-06-10T00:42:00.858 INFO:teuthology.orchestra.run.smithi167.stdout:(82/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-06-10T00:42:00.884 INFO:teuthology.orchestra.run.smithi167.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 250 kB 00:00 2024-06-10T00:42:00.908 INFO:teuthology.orchestra.run.smithi094.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-06-10T00:42:00.959 INFO:teuthology.orchestra.run.smithi167.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-06-10T00:42:00.966 INFO:teuthology.orchestra.run.smithi094.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-06-10T00:42:00.984 INFO:teuthology.orchestra.run.smithi167.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 770 kB/s | 77 kB 00:00 2024-06-10T00:42:01.034 INFO:teuthology.orchestra.run.smithi094.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-06-10T00:42:01.042 INFO:teuthology.orchestra.run.smithi037.stdout:(92/159): python3-requests-oauthlib-1.3.0-12.el 79 kB/s | 54 kB 00:00 2024-06-10T00:42:01.059 INFO:teuthology.orchestra.run.smithi167.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 422 kB/s | 42 kB 00:00 2024-06-10T00:42:01.085 INFO:teuthology.orchestra.run.smithi167.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-06-10T00:42:01.092 INFO:teuthology.orchestra.run.smithi094.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 441 kB/s | 26 kB 00:00 2024-06-10T00:42:01.160 INFO:teuthology.orchestra.run.smithi167.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-06-10T00:42:01.176 INFO:teuthology.orchestra.run.smithi094.stdout:(78/159): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-06-10T00:42:01.186 INFO:teuthology.orchestra.run.smithi167.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-06-10T00:42:01.201 INFO:teuthology.orchestra.run.smithi094.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 176 kB/s | 19 kB 00:00 2024-06-10T00:42:01.261 INFO:teuthology.orchestra.run.smithi167.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 511 kB/s | 51 kB 00:00 2024-06-10T00:42:01.268 INFO:teuthology.orchestra.run.smithi037.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 68 kB/s | 42 kB 00:00 2024-06-10T00:42:01.276 INFO:teuthology.orchestra.run.smithi094.stdout:(80/159): postgresql-server-13.14-1.el9.x86_64. 9.5 MB/s | 5.8 MB 00:00 2024-06-10T00:42:01.286 INFO:teuthology.orchestra.run.smithi167.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 535 kB/s | 54 kB 00:00 2024-06-10T00:42:01.293 INFO:teuthology.orchestra.run.smithi094.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 378 kB/s | 35 kB 00:00 2024-06-10T00:42:01.327 INFO:teuthology.orchestra.run.smithi094.stdout:(82/159): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-06-10T00:42:01.352 INFO:teuthology.orchestra.run.smithi094.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-06-10T00:42:01.394 INFO:teuthology.orchestra.run.smithi094.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-06-10T00:42:01.419 INFO:teuthology.orchestra.run.smithi094.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-06-10T00:42:01.453 INFO:teuthology.orchestra.run.smithi094.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 722 kB/s | 42 kB 00:00 2024-06-10T00:42:01.487 INFO:teuthology.orchestra.run.smithi094.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-06-10T00:42:01.512 INFO:teuthology.orchestra.run.smithi094.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-06-10T00:42:01.546 INFO:teuthology.orchestra.run.smithi094.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-06-10T00:42:01.571 INFO:teuthology.orchestra.run.smithi094.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 869 kB/s | 51 kB 00:00 2024-06-10T00:42:01.646 INFO:teuthology.orchestra.run.smithi094.stdout:(91/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-06-10T00:42:01.671 INFO:teuthology.orchestra.run.smithi094.stdout:(92/159): python3-requests-oauthlib-1.3.0-12.el 428 kB/s | 54 kB 00:00 2024-06-10T00:42:01.684 INFO:teuthology.orchestra.run.smithi037.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 69 kB/s | 44 kB 00:00 2024-06-10T00:42:01.703 INFO:teuthology.orchestra.run.smithi167.stdout:(92/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 4.5 MB/s | 48 MB 00:10 2024-06-10T00:42:01.705 INFO:teuthology.orchestra.run.smithi094.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 714 kB/s | 42 kB 00:00 2024-06-10T00:42:01.731 INFO:teuthology.orchestra.run.smithi094.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 753 kB/s | 44 kB 00:00 2024-06-10T00:42:01.781 INFO:teuthology.orchestra.run.smithi094.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-06-10T00:42:01.806 INFO:teuthology.orchestra.run.smithi094.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 475 kB/s | 36 kB 00:00 2024-06-10T00:42:01.820 INFO:teuthology.orchestra.run.smithi167.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 78 kB/s | 42 kB 00:00 2024-06-10T00:42:01.865 INFO:teuthology.orchestra.run.smithi094.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 567 kB/s | 47 kB 00:00 2024-06-10T00:42:01.898 INFO:teuthology.orchestra.run.smithi094.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 3.2 MB/s | 305 kB 00:00 2024-06-10T00:42:01.924 INFO:teuthology.orchestra.run.smithi094.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 988 kB/s | 58 kB 00:00 2024-06-10T00:42:01.957 INFO:teuthology.orchestra.run.smithi094.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-06-10T00:42:01.970 INFO:teuthology.orchestra.run.smithi167.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 167 kB/s | 44 kB 00:00 2024-06-10T00:42:01.982 INFO:teuthology.orchestra.run.smithi094.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-06-10T00:42:01.983 INFO:teuthology.orchestra.run.smithi094.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:42:02.016 INFO:teuthology.orchestra.run.smithi094.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 767 kB/s | 45 kB 00:00 2024-06-10T00:42:02.017 INFO:teuthology.orchestra.run.smithi094.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:42:02.068 INFO:teuthology.orchestra.run.smithi037.stdout:(95/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-06-10T00:42:02.117 INFO:teuthology.orchestra.run.smithi094.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 167 kB/s | 22 kB 00:00 2024-06-10T00:42:02.169 INFO:teuthology.orchestra.run.smithi037.stdout:(96/159): qatlib-24.02.0-1.el9.x86_64.rpm 253 kB/s | 221 kB 00:00 2024-06-10T00:42:02.221 INFO:teuthology.orchestra.run.smithi167.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 144 kB/s | 36 kB 00:00 2024-06-10T00:42:02.284 INFO:teuthology.orchestra.run.smithi094.stdout:(104/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-06-10T00:42:02.334 INFO:teuthology.orchestra.run.smithi094.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 2.5 MB/s | 548 kB 00:00 2024-06-10T00:42:02.359 INFO:teuthology.orchestra.run.smithi094.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 857 kB/s | 19 kB 00:00 2024-06-10T00:42:02.362 INFO:teuthology.orchestra.run.smithi167.stdout:(96/159): qatlib-24.02.0-1.el9.x86_64.rpm 409 kB/s | 221 kB 00:00 2024-06-10T00:42:02.401 INFO:teuthology.orchestra.run.smithi094.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 2.6 MB/s | 309 kB 00:00 2024-06-10T00:42:02.426 INFO:teuthology.orchestra.run.smithi094.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-06-10T00:42:02.446 INFO:teuthology.orchestra.run.smithi167.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 212 kB/s | 47 kB 00:00 2024-06-10T00:42:02.460 INFO:teuthology.orchestra.run.smithi094.stdout:(109/159): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-06-10T00:42:02.485 INFO:teuthology.orchestra.run.smithi037.stdout:(97/159): qatlib-service-24.02.0-1.el9.x86_64.r 45 kB/s | 36 kB 00:00 2024-06-10T00:42:02.510 INFO:teuthology.orchestra.run.smithi094.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 582 kB/s | 49 kB 00:00 2024-06-10T00:42:02.544 INFO:teuthology.orchestra.run.smithi094.stdout:(111/159): luarocks-3.9.2-5.el9.noarch.rpm 4.8 MB/s | 151 kB 00:00 2024-06-10T00:42:02.569 INFO:teuthology.orchestra.run.smithi094.stdout:(112/159): libunwind-1.6.2-1.el9.x86_64.rpm 618 kB/s | 67 kB 00:00 2024-06-10T00:42:02.605 INFO:teuthology.orchestra.run.smithi167.stdout:(98/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 12 MB/s | 16 MB 00:01 2024-06-10T00:42:02.611 INFO:teuthology.orchestra.run.smithi094.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-06-10T00:42:02.630 INFO:teuthology.orchestra.run.smithi167.stdout:(99/159): socat-1.7.4.1-5.el9.x86_64.rpm 1.1 MB/s | 305 kB 00:00 2024-06-10T00:42:02.636 INFO:teuthology.orchestra.run.smithi094.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-06-10T00:42:02.656 INFO:teuthology.orchestra.run.smithi167.stdout:(100/159): uuid-1.6.2-55.el9.x86_64.rpm 277 kB/s | 58 kB 00:00 2024-06-10T00:42:02.671 INFO:teuthology.orchestra.run.smithi094.stdout:(115/159): python3-asyncssh-2.13.2-5.el9.noarch 5.3 MB/s | 548 kB 00:00 2024-06-10T00:42:02.695 INFO:teuthology.orchestra.run.smithi094.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 551 kB/s | 32 kB 00:00 2024-06-10T00:42:02.721 INFO:teuthology.orchestra.run.smithi094.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 297 kB/s | 15 kB 00:00 2024-06-10T00:42:02.722 INFO:teuthology.orchestra.run.smithi167.stdout:(101/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 806 kB/s | 94 kB 00:00 2024-06-10T00:42:02.726 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:42:02.746 INFO:teuthology.orchestra.run.smithi094.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-06-10T00:42:02.752 INFO:teuthology.orchestra.run.smithi037.stdout:(98/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 72 kB/s | 47 kB 00:00 2024-06-10T00:42:02.756 INFO:teuthology.orchestra.run.smithi167.stdout:(102/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 635 kB/s | 64 kB 00:00 2024-06-10T00:42:02.771 INFO:teuthology.orchestra.run.smithi094.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-06-10T00:42:02.781 INFO:teuthology.orchestra.run.smithi167.stdout:(103/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 357 kB/s | 45 kB 00:00 2024-06-10T00:42:02.782 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:42:02.796 INFO:teuthology.orchestra.run.smithi094.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-06-10T00:42:02.815 INFO:teuthology.orchestra.run.smithi167.stdout:(104/159): abseil-cpp-20211102.0-3.el9.x86_64.r 16 MB/s | 548 kB 00:00 2024-06-10T00:42:02.821 INFO:teuthology.orchestra.run.smithi094.stdout:(121/159): python3-google-auth-2.29.0-2.el9.noa 4.0 MB/s | 205 kB 00:00 2024-06-10T00:42:02.840 INFO:teuthology.orchestra.run.smithi167.stdout:(105/159): lua-devel-5.4.4-4.el9.x86_64.rpm 190 kB/s | 22 kB 00:00 2024-06-10T00:42:02.855 INFO:teuthology.orchestra.run.smithi094.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-06-10T00:42:02.880 INFO:teuthology.orchestra.run.smithi094.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-06-10T00:42:02.899 INFO:teuthology.orchestra.run.smithi167.stdout:(106/159): gperftools-libs-2.9.1-2.el9.x86_64.r 3.6 MB/s | 309 kB 00:00 2024-06-10T00:42:02.906 INFO:teuthology.orchestra.run.smithi094.stdout:(124/159): python3-jaraco-classes-3.2.1-5.el9.n 699 kB/s | 18 kB 00:00 2024-06-10T00:42:02.924 INFO:teuthology.orchestra.run.smithi167.stdout:(107/159): grpc-data-1.46.7-10.el9.noarch.rpm 233 kB/s | 19 kB 00:00 2024-06-10T00:42:02.931 INFO:teuthology.orchestra.run.smithi094.stdout:(125/159): python3-jaraco-collections-3.0.0-8.e 929 kB/s | 23 kB 00:00 2024-06-10T00:42:02.949 INFO:teuthology.orchestra.run.smithi167.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-06-10T00:42:02.956 INFO:teuthology.orchestra.run.smithi094.stdout:(126/159): python3-jaraco-functools-3.5.0-2.el9 783 kB/s | 19 kB 00:00 2024-06-10T00:42:02.983 INFO:teuthology.orchestra.run.smithi167.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-06-10T00:42:03.017 INFO:teuthology.orchestra.run.smithi167.stdout:(110/159): protobuf-compiler-3.14.0-13.el9.x86_ 3.2 MB/s | 863 kB 00:00 2024-06-10T00:42:03.067 INFO:teuthology.orchestra.run.smithi167.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 807 kB/s | 67 kB 00:00 2024-06-10T00:42:03.090 INFO:teuthology.orchestra.run.smithi094.stdout:(127/159): python3-jaraco-text-3.2.0-6.el9.noar 148 kB/s | 20 kB 00:00 2024-06-10T00:42:03.109 INFO:teuthology.orchestra.run.smithi167.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-06-10T00:42:03.119 INFO:teuthology.orchestra.run.smithi037.stdout:(99/159): socat-1.7.4.1-5.el9.x86_64.rpm 321 kB/s | 305 kB 00:00 2024-06-10T00:42:03.145 INFO:teuthology.orchestra.run.smithi037.stdout:(100/159): uuid-1.6.2-55.el9.x86_64.rpm 88 kB/s | 58 kB 00:00 2024-06-10T00:42:03.159 INFO:teuthology.orchestra.run.smithi167.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.1 MB/s | 839 kB 00:00 2024-06-10T00:42:03.176 INFO:teuthology.orchestra.run.smithi167.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-06-10T00:42:03.233 INFO:teuthology.orchestra.run.smithi094.stdout:(128/159): libarrow-9.0.0-11.el9.x86_64.rpm 5.1 MB/s | 4.4 MB 00:00 2024-06-10T00:42:03.258 INFO:teuthology.orchestra.run.smithi094.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-06-10T00:42:03.283 INFO:teuthology.orchestra.run.smithi094.stdout:(130/159): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-06-10T00:42:03.301 INFO:teuthology.orchestra.run.smithi167.stdout:(115/159): python3-asyncssh-2.13.2-5.el9.noarch 2.8 MB/s | 548 kB 00:00 2024-06-10T00:42:03.316 INFO:teuthology.orchestra.run.smithi094.stdout:(131/159): python3-natsort-7.1.1-5.el9.noarch.r 1.7 MB/s | 58 kB 00:00 2024-06-10T00:42:03.375 INFO:teuthology.orchestra.run.smithi094.stdout:(132/159): python3-grpcio-1.46.7-10.el9.x86_64. 3.5 MB/s | 2.0 MB 00:00 2024-06-10T00:42:03.400 INFO:teuthology.orchestra.run.smithi094.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-06-10T00:42:03.426 INFO:teuthology.orchestra.run.smithi094.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-06-10T00:42:03.459 INFO:teuthology.orchestra.run.smithi094.stdout:(135/159): python3-kubernetes-26.1.0-2.el9.noar 2.8 MB/s | 1.0 MB 00:00 2024-06-10T00:42:03.485 INFO:teuthology.orchestra.run.smithi094.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 482 kB/s | 28 kB 00:00 2024-06-10T00:42:03.485 INFO:teuthology.orchestra.run.smithi167.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 104 kB/s | 32 kB 00:00 2024-06-10T00:42:03.510 INFO:teuthology.orchestra.run.smithi094.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-06-10T00:42:03.510 INFO:teuthology.orchestra.run.smithi167.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 72 kB/s | 15 kB 00:00 2024-06-10T00:42:03.520 INFO:teuthology.orchestra.run.smithi037.stdout:(101/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 122 kB/s | 94 kB 00:00 2024-06-10T00:42:03.535 INFO:teuthology.orchestra.run.smithi094.stdout:(138/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 669 kB/s | 90 kB 00:00 2024-06-10T00:42:03.552 INFO:teuthology.orchestra.run.smithi094.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-06-10T00:42:03.559 INFO:teuthology.orchestra.run.smithi037.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:42:03.569 INFO:teuthology.orchestra.run.smithi094.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-06-10T00:42:03.569 INFO:teuthology.orchestra.run.smithi167.stdout:(118/159): libarrow-9.0.0-11.el9.x86_64.rpm 6.6 MB/s | 4.4 MB 00:00 2024-06-10T00:42:03.586 INFO:teuthology.orchestra.run.smithi094.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 1.5 MB/s | 51 kB 00:00 2024-06-10T00:42:03.594 INFO:teuthology.orchestra.run.smithi167.stdout:(119/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-06-10T00:42:03.611 INFO:teuthology.orchestra.run.smithi094.stdout:(142/159): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-06-10T00:42:03.620 INFO:teuthology.orchestra.run.smithi167.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-06-10T00:42:03.636 INFO:teuthology.orchestra.run.smithi094.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-06-10T00:42:03.645 INFO:teuthology.orchestra.run.smithi167.stdout:(121/159): python3-google-auth-2.29.0-2.el9.noa 4.0 MB/s | 205 kB 00:00 2024-06-10T00:42:03.670 INFO:teuthology.orchestra.run.smithi094.stdout:(144/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-06-10T00:42:03.670 INFO:teuthology.orchestra.run.smithi167.stdout:(122/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.9 MB/s | 303 kB 00:00 2024-06-10T00:42:03.695 INFO:teuthology.orchestra.run.smithi094.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-06-10T00:42:03.696 INFO:teuthology.orchestra.run.smithi167.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-06-10T00:42:03.729 INFO:teuthology.orchestra.run.smithi094.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-06-10T00:42:03.754 INFO:teuthology.orchestra.run.smithi094.stdout:(147/159): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-06-10T00:42:03.770 INFO:teuthology.orchestra.run.smithi037.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 72 kB/s | 45 kB 00:00 2024-06-10T00:42:03.772 INFO:teuthology.orchestra.run.smithi167.stdout:(124/159): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-06-10T00:42:03.795 INFO:teuthology.orchestra.run.smithi037.stdout:(103/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 94 kB/s | 64 kB 00:00 2024-06-10T00:42:03.796 INFO:teuthology.orchestra.run.smithi037.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-10T00:42:03.804 INFO:teuthology.orchestra.run.smithi094.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-06-10T00:42:03.805 INFO:teuthology.orchestra.run.smithi167.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 162 kB/s | 18 kB 00:00 2024-06-10T00:42:03.830 INFO:teuthology.orchestra.run.smithi167.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 403 kB/s | 23 kB 00:00 2024-06-10T00:42:03.838 INFO:teuthology.orchestra.run.smithi037.stdout:(104/159): abseil-cpp-20211102.0-3.el9.x86_64.r 13 MB/s | 548 kB 00:00 2024-06-10T00:42:03.862 INFO:teuthology.orchestra.run.smithi037.stdout:(105/159): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-06-10T00:42:03.864 INFO:teuthology.orchestra.run.smithi167.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 331 kB/s | 19 kB 00:00 2024-06-10T00:42:03.880 INFO:teuthology.orchestra.run.smithi037.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 1.1 MB/s | 19 kB 00:00 2024-06-10T00:42:03.889 INFO:teuthology.orchestra.run.smithi167.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 337 kB/s | 20 kB 00:00 2024-06-10T00:42:03.923 INFO:teuthology.orchestra.run.smithi167.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-06-10T00:42:03.930 INFO:teuthology.orchestra.run.smithi037.stdout:(107/159): lua-devel-5.4.4-4.el9.x86_64.rpm 54 kB/s | 22 kB 00:00 2024-06-10T00:42:03.940 INFO:teuthology.orchestra.run.smithi167.stdout:(130/159): python3-more-itertools-8.12.0-2.el9. 4.6 MB/s | 79 kB 00:00 2024-06-10T00:42:03.965 INFO:teuthology.orchestra.run.smithi167.stdout:(131/159): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-06-10T00:42:03.988 INFO:teuthology.orchestra.run.smithi037.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 447 kB/s | 26 kB 00:00 2024-06-10T00:42:04.007 INFO:teuthology.orchestra.run.smithi167.stdout:(132/159): python3-grpcio-1.46.7-10.el9.x86_64. 5.3 MB/s | 2.0 MB 00:00 2024-06-10T00:42:04.032 INFO:teuthology.orchestra.run.smithi167.stdout:(133/159): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-06-10T00:42:04.055 INFO:teuthology.orchestra.run.smithi037.stdout:(109/159): libarrow-9.0.0-11.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-06-10T00:42:04.058 INFO:teuthology.orchestra.run.smithi167.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-06-10T00:42:04.091 INFO:teuthology.orchestra.run.smithi167.stdout:(135/159): python3-kubernetes-26.1.0-2.el9.noar 4.5 MB/s | 1.0 MB 00:00 2024-06-10T00:42:04.105 INFO:teuthology.orchestra.run.smithi037.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 415 kB/s | 49 kB 00:00 2024-06-10T00:42:04.116 INFO:teuthology.orchestra.run.smithi167.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 483 kB/s | 28 kB 00:00 2024-06-10T00:42:04.131 INFO:teuthology.orchestra.run.smithi037.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 896 kB/s | 67 kB 00:00 2024-06-10T00:42:04.142 INFO:teuthology.orchestra.run.smithi167.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-06-10T00:42:04.156 INFO:teuthology.orchestra.run.smithi037.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-06-10T00:42:04.167 INFO:teuthology.orchestra.run.smithi167.stdout:(138/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 669 kB/s | 90 kB 00:00 2024-06-10T00:42:04.180 INFO:teuthology.orchestra.run.smithi094.stdout:(149/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 7.4 MB/s | 3.1 MB 00:00 2024-06-10T00:42:04.192 INFO:teuthology.orchestra.run.smithi167.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-06-10T00:42:04.198 INFO:teuthology.orchestra.run.smithi037.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-06-10T00:42:04.218 INFO:teuthology.orchestra.run.smithi167.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-06-10T00:42:04.223 INFO:teuthology.orchestra.run.smithi037.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-06-10T00:42:04.230 INFO:teuthology.orchestra.run.smithi094.stdout:(150/159): librados2-19.0.0-2946.g3d89accf.el9. 6.8 MB/s | 3.4 MB 00:00 2024-06-10T00:42:04.243 INFO:teuthology.orchestra.run.smithi167.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-06-10T00:42:04.249 INFO:teuthology.orchestra.run.smithi037.stdout:(115/159): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-06-10T00:42:04.274 INFO:teuthology.orchestra.run.smithi037.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 644 kB/s | 32 kB 00:00 2024-06-10T00:42:04.285 INFO:teuthology.orchestra.run.smithi167.stdout:(142/159): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2024-06-10T00:42:04.299 INFO:teuthology.orchestra.run.smithi037.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-06-10T00:42:04.310 INFO:teuthology.orchestra.run.smithi167.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-06-10T00:42:04.324 INFO:teuthology.orchestra.run.smithi037.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-06-10T00:42:04.330 INFO:teuthology.orchestra.run.smithi094.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 415 kB/s | 41 kB 00:00 2024-06-10T00:42:04.335 INFO:teuthology.orchestra.run.smithi167.stdout:(144/159): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-06-10T00:42:04.358 INFO:teuthology.orchestra.run.smithi037.stdout:(119/159): protobuf-compiler-3.14.0-13.el9.x86_ 1.4 MB/s | 863 kB 00:00 2024-06-10T00:42:04.360 INFO:teuthology.orchestra.run.smithi167.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-06-10T00:42:04.386 INFO:teuthology.orchestra.run.smithi167.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-06-10T00:42:04.413 INFO:teuthology.orchestra.run.smithi167.stdout:(147/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-06-10T00:42:04.417 INFO:teuthology.orchestra.run.smithi037.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-06-10T00:42:04.431 INFO:teuthology.orchestra.run.smithi094.stdout:(152/159): NetworkManager-1.47.91-1.el9.x86_64. 3.7 MB/s | 2.3 MB 00:00 2024-06-10T00:42:04.442 INFO:teuthology.orchestra.run.smithi037.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-06-10T00:42:04.461 INFO:teuthology.orchestra.run.smithi167.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-06-10T00:42:04.464 INFO:teuthology.orchestra.run.smithi094.stdout:(153/159): NetworkManager-tui-1.47.91-1.el9.x86 1.8 MB/s | 248 kB 00:00 2024-06-10T00:42:04.468 INFO:teuthology.orchestra.run.smithi037.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-06-10T00:42:04.490 INFO:teuthology.orchestra.run.smithi094.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 625 kB/s | 37 kB 00:00 2024-06-10T00:42:04.515 INFO:teuthology.orchestra.run.smithi094.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 4.0 MB/s | 207 kB 00:00 2024-06-10T00:42:04.518 INFO:teuthology.orchestra.run.smithi037.stdout:(123/159): python3-google-auth-2.29.0-2.el9.noa 1.3 MB/s | 205 kB 00:00 2024-06-10T00:42:04.543 INFO:teuthology.orchestra.run.smithi037.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-06-10T00:42:04.569 INFO:teuthology.orchestra.run.smithi037.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-06-10T00:42:04.594 INFO:teuthology.orchestra.run.smithi037.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-06-10T00:42:04.607 INFO:teuthology.orchestra.run.smithi094.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 2.2 MB/s | 260 kB 00:00 2024-06-10T00:42:04.619 INFO:teuthology.orchestra.run.smithi037.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-06-10T00:42:04.645 INFO:teuthology.orchestra.run.smithi037.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-06-10T00:42:04.649 INFO:teuthology.orchestra.run.smithi094.stdout:(157/159): NetworkManager-libnm-1.47.91-1.el9.x 3.8 MB/s | 1.8 MB 00:00 2024-06-10T00:42:04.670 INFO:teuthology.orchestra.run.smithi037.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-06-10T00:42:04.695 INFO:teuthology.orchestra.run.smithi037.stdout:(130/159): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-06-10T00:42:04.720 INFO:teuthology.orchestra.run.smithi037.stdout:(131/159): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-06-10T00:42:04.754 INFO:teuthology.orchestra.run.smithi037.stdout:(132/159): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-06-10T00:42:04.787 INFO:teuthology.orchestra.run.smithi037.stdout:(133/159): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-06-10T00:42:04.813 INFO:teuthology.orchestra.run.smithi037.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 281 kB/s | 16 kB 00:00 2024-06-10T00:42:04.838 INFO:teuthology.orchestra.run.smithi037.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-06-10T00:42:04.862 INFO:teuthology.orchestra.run.smithi167.stdout:(149/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 7.0 MB/s | 3.1 MB 00:00 2024-06-10T00:42:04.863 INFO:teuthology.orchestra.run.smithi037.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-06-10T00:42:04.874 INFO:teuthology.orchestra.run.smithi094.stdout:(158/159): openssl-3.2.1-1.el9.x86_64.rpm 3.8 MB/s | 1.3 MB 00:00 2024-06-10T00:42:04.889 INFO:teuthology.orchestra.run.smithi037.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-06-10T00:42:04.914 INFO:teuthology.orchestra.run.smithi037.stdout:(138/159): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-06-10T00:42:04.939 INFO:teuthology.orchestra.run.smithi037.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-06-10T00:42:04.965 INFO:teuthology.orchestra.run.smithi037.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 710 kB/s | 36 kB 00:00 2024-06-10T00:42:04.990 INFO:teuthology.orchestra.run.smithi037.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-06-10T00:42:05.015 INFO:teuthology.orchestra.run.smithi037.stdout:(142/159): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-06-10T00:42:05.057 INFO:teuthology.orchestra.run.smithi037.stdout:(143/159): python3-grpcio-1.46.7-10.el9.x86_64. 3.2 MB/s | 2.0 MB 00:00 2024-06-10T00:42:05.082 INFO:teuthology.orchestra.run.smithi037.stdout:(144/159): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-06-10T00:42:05.100 INFO:teuthology.orchestra.run.smithi037.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 475 kB/s | 20 kB 00:00 2024-06-10T00:42:05.125 INFO:teuthology.orchestra.run.smithi037.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 4.5 MB/s | 192 kB 00:00 2024-06-10T00:42:05.150 INFO:teuthology.orchestra.run.smithi037.stdout:(147/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-06-10T00:42:05.204 INFO:teuthology.orchestra.run.smithi167.stdout:(150/159): librados2-19.0.0-2946.g3d89accf.el9. 4.1 MB/s | 3.4 MB 00:00 2024-06-10T00:42:05.209 INFO:teuthology.orchestra.run.smithi037.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-06-10T00:42:05.349 INFO:teuthology.orchestra.run.smithi094.stdout:(159/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 3.2 MB/s | 2.4 MB 00:00 2024-06-10T00:42:05.355 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-06-10T00:42:05.355 INFO:teuthology.orchestra.run.smithi094.stdout:Total 16 MB/s | 225 MB 00:13 2024-06-10T00:42:05.384 INFO:teuthology.orchestra.run.smithi037.stdout:(149/159): NetworkManager-1.47.91-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-06-10T00:42:05.451 INFO:teuthology.orchestra.run.smithi037.stdout:(150/159): NetworkManager-libnm-1.47.91-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-06-10T00:42:05.505 INFO:teuthology.orchestra.run.smithi167.stdout:(151/159): NetworkManager-1.47.91-1.el9.x86_64. 2.2 MB/s | 2.3 MB 00:01 2024-06-10T00:42:05.509 INFO:teuthology.orchestra.run.smithi037.stdout:(151/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 8.7 MB/s | 3.1 MB 00:00 2024-06-10T00:42:05.535 INFO:teuthology.orchestra.run.smithi037.stdout:(152/159): NetworkManager-team-1.47.91-1.el9.x8 495 kB/s | 41 kB 00:00 2024-06-10T00:42:05.563 INFO:teuthology.orchestra.run.smithi167.stdout:(152/159): NetworkManager-team-1.47.91-1.el9.x8 115 kB/s | 41 kB 00:00 2024-06-10T00:42:05.577 INFO:teuthology.orchestra.run.smithi037.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 883 kB/s | 37 kB 00:00 2024-06-10T00:42:05.613 INFO:teuthology.orchestra.run.smithi167.stdout:(153/159): NetworkManager-tui-1.47.91-1.el9.x86 2.2 MB/s | 248 kB 00:00 2024-06-10T00:42:05.618 INFO:teuthology.orchestra.run.smithi037.stdout:(154/159): NetworkManager-tui-1.47.91-1.el9.x86 2.2 MB/s | 248 kB 00:00 2024-06-10T00:42:05.644 INFO:teuthology.orchestra.run.smithi037.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 3.0 MB/s | 207 kB 00:00 2024-06-10T00:42:05.678 INFO:teuthology.orchestra.run.smithi037.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 4.3 MB/s | 260 kB 00:00 2024-06-10T00:42:05.720 INFO:teuthology.orchestra.run.smithi037.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 18 MB/s | 1.3 MB 00:00 2024-06-10T00:42:05.721 INFO:teuthology.orchestra.run.smithi167.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 1.9 MB/s | 207 kB 00:00 2024-06-10T00:42:05.747 INFO:teuthology.orchestra.run.smithi167.stdout:(155/159): elfutils-debuginfod-client-0.191-4.e 199 kB/s | 37 kB 00:00 2024-06-10T00:42:05.803 INFO:teuthology.orchestra.run.smithi037.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-06-10T00:42:05.831 INFO:teuthology.orchestra.run.smithi167.stdout:(156/159): NetworkManager-libnm-1.47.91-1.el9.x 1.9 MB/s | 1.8 MB 00:00 2024-06-10T00:42:05.856 INFO:teuthology.orchestra.run.smithi167.stdout:(157/159): elfutils-libs-0.191-4.el9.x86_64.rpm 1.9 MB/s | 260 kB 00:00 2024-06-10T00:42:05.962 INFO:teuthology.orchestra.run.smithi037.stdout:(159/159): librados2-19.0.0-2946.g3d89accf.el9. 4.0 MB/s | 3.4 MB 00:00 2024-06-10T00:42:05.969 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-06-10T00:42:05.969 INFO:teuthology.orchestra.run.smithi037.stdout:Total 7.4 MB/s | 225 MB 00:30 2024-06-10T00:42:06.131 INFO:teuthology.orchestra.run.smithi167.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 8.0 MB/s | 2.4 MB 00:00 2024-06-10T00:42:06.514 INFO:teuthology.orchestra.run.smithi167.stdout:(159/159): openssl-3.2.1-1.el9.x86_64.rpm 1.8 MB/s | 1.3 MB 00:00 2024-06-10T00:42:06.521 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-06-10T00:42:06.521 INFO:teuthology.orchestra.run.smithi167.stdout:Total 7.3 MB/s | 225 MB 00:30 2024-06-10T00:42:06.561 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-06-10T00:42:06.717 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-06-10T00:42:06.717 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-06-10T00:42:07.198 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-06-10T00:42:07.378 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-06-10T00:42:07.378 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-06-10T00:42:07.766 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-06-10T00:42:07.924 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-06-10T00:42:07.925 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-06-10T00:42:08.516 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-06-10T00:42:08.516 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-06-10T00:42:09.631 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-06-10T00:42:09.632 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-06-10T00:42:09.732 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-06-10T00:42:09.732 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-06-10T00:42:14.723 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-06-10T00:42:14.815 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-10T00:42:14.853 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-10T00:42:14.915 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-10T00:42:14.949 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-10T00:42:14.974 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-10T00:42:15.016 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-10T00:42:15.350 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-10T00:42:15.484 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T00:42:15.597 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T00:42:15.627 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T00:42:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T00:42:15.739 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-10T00:42:15.771 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-10T00:42:15.903 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-06-10T00:42:15.953 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-10T00:42:15.994 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-10T00:42:16.004 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-10T00:42:16.032 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T00:42:16.041 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-10T00:42:16.074 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-06-10T00:42:16.094 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-10T00:42:16.128 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-10T00:42:16.135 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T00:42:16.154 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-10T00:42:16.158 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-10T00:42:16.158 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-10T00:42:16.196 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-10T00:42:16.197 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-10T00:42:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-10T00:42:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-10T00:42:16.258 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-10T00:42:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-10T00:42:16.292 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-10T00:42:16.318 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-10T00:42:16.366 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-10T00:42:16.459 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-10T00:42:16.529 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-10T00:42:16.585 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-10T00:42:16.655 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T00:42:16.711 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-10T00:42:16.739 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-10T00:42:16.793 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T00:42:16.793 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T00:42:16.810 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T00:42:16.819 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-10T00:42:16.881 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T00:42:16.910 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-10T00:42:16.918 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T00:42:16.930 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T00:42:16.943 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-10T00:42:17.026 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T00:42:17.070 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T00:42:17.131 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-10T00:42:17.146 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T00:42:17.175 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-10T00:42:17.183 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-10T00:42:17.211 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T00:42:17.217 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-10T00:42:17.237 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T00:42:17.315 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T00:42:17.337 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-10T00:42:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-10T00:42:17.400 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-10T00:42:17.416 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-10T00:42:17.422 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-10T00:42:17.441 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-10T00:42:17.444 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-10T00:42:17.465 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-10T00:42:17.481 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-10T00:42:17.492 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T00:42:17.524 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-10T00:42:17.581 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T00:42:17.588 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-10T00:42:17.597 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-10T00:42:17.609 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-10T00:42:17.623 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-10T00:42:17.656 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-10T00:42:17.675 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-10T00:42:17.677 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-10T00:42:17.697 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-10T00:42:17.708 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-10T00:42:17.727 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-10T00:42:17.736 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-10T00:42:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-10T00:42:17.857 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-10T00:42:17.869 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-10T00:42:17.889 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-10T00:42:17.913 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-10T00:42:17.929 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-10T00:42:17.932 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-10T00:42:18.016 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-10T00:42:18.031 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T00:42:18.051 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-10T00:42:18.087 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-10T00:42:18.094 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-10T00:42:18.105 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T00:42:18.123 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-10T00:42:18.192 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-10T00:42:18.233 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-10T00:42:18.237 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-10T00:42:18.293 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-10T00:42:18.319 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-10T00:42:18.321 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T00:42:18.399 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T00:42:18.429 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-10T00:42:18.460 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-10T00:42:18.461 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T00:42:18.515 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-10T00:42:18.524 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-10T00:42:18.561 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-10T00:42:18.604 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-10T00:42:18.669 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-10T00:42:18.678 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-10T00:42:18.712 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-10T00:42:18.755 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-10T00:42:18.779 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-10T00:42:18.788 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-10T00:42:18.816 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-10T00:42:18.875 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-10T00:42:18.885 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-10T00:42:18.917 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-10T00:42:18.938 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-10T00:42:18.952 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-10T00:42:18.968 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-10T00:42:19.004 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-10T00:42:19.027 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-10T00:42:19.028 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-10T00:42:19.068 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-10T00:42:19.080 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-10T00:42:19.111 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-10T00:42:19.111 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-10T00:42:19.114 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-10T00:42:19.140 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-10T00:42:19.144 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-10T00:42:19.158 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-10T00:42:19.171 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-10T00:42:19.176 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-10T00:42:19.186 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-10T00:42:19.225 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-10T00:42:19.243 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-10T00:42:19.258 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-10T00:42:19.263 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-10T00:42:19.294 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-10T00:42:19.301 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-10T00:42:19.304 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-10T00:42:19.324 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-10T00:42:19.369 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-10T00:42:19.377 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-10T00:42:19.446 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-10T00:42:19.458 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-10T00:42:19.479 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-10T00:42:19.508 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-10T00:42:19.513 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-10T00:42:19.523 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-10T00:42:19.551 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-10T00:42:19.562 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-10T00:42:19.563 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-10T00:42:19.623 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-10T00:42:19.642 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-10T00:42:19.666 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-10T00:42:19.683 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-10T00:42:19.720 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-10T00:42:19.743 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-10T00:42:19.766 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-10T00:42:19.802 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-10T00:42:19.845 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-10T00:42:19.879 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-10T00:42:19.896 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-10T00:42:19.936 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-10T00:42:20.039 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-10T00:42:20.090 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-10T00:42:20.103 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-10T00:42:20.130 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-10T00:42:20.185 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-10T00:42:20.220 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-10T00:42:20.224 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-10T00:42:20.256 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-10T00:42:20.264 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-google-auth-1:2.29.0-2.el9.noarch 63/170 2024-06-10T00:42:20.313 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-10T00:42:20.314 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-10T00:42:20.346 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-10T00:42:20.352 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-10T00:42:20.371 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-10T00:42:20.394 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-10T00:42:20.487 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-10T00:42:20.544 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-10T00:42:20.575 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-10T00:42:20.610 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-10T00:42:20.613 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-10T00:42:20.646 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-10T00:42:20.669 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-10T00:42:20.695 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-10T00:42:20.733 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-10T00:42:20.742 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-10T00:42:20.775 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-10T00:42:20.781 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-10T00:42:20.794 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-10T00:42:20.818 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-10T00:42:20.863 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-10T00:42:20.900 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-10T00:42:20.925 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-10T00:42:20.934 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-10T00:42:20.943 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-10T00:42:20.962 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-10T00:42:20.986 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-10T00:42:20.986 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-10T00:42:20.990 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-10T00:42:21.032 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-10T00:42:21.042 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-10T00:42:21.071 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-10T00:42:21.103 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-10T00:42:21.116 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-10T00:42:21.125 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-10T00:42:21.141 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-10T00:42:21.153 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-10T00:42:21.184 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-10T00:42:21.190 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-10T00:42:21.196 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-10T00:42:21.217 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-10T00:42:21.237 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-10T00:42:21.237 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-10T00:42:21.260 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-10T00:42:21.272 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-10T00:42:21.294 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-10T00:42:21.316 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-10T00:42:21.333 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-10T00:42:21.358 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-10T00:42:21.379 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-google-auth-1:2.29.0-2.el9.noarch 63/170 2024-06-10T00:42:21.380 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-10T00:42:21.419 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-10T00:42:21.440 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-10T00:42:21.444 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-10T00:42:21.469 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-10T00:42:21.480 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-10T00:42:21.517 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-10T00:42:21.527 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-10T00:42:21.545 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-10T00:42:21.562 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-10T00:42:21.604 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-10T00:42:21.624 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-10T00:42:21.673 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-10T00:42:21.674 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-10T00:42:21.701 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-10T00:42:21.710 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-10T00:42:21.738 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-10T00:42:21.745 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-google-auth-1:2.29.0-2.el9.noarch 63/170 2024-06-10T00:42:21.772 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-10T00:42:21.785 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-10T00:42:21.812 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-10T00:42:21.818 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-10T00:42:21.843 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-10T00:42:21.844 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-10T00:42:21.873 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-10T00:42:21.907 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-10T00:42:21.924 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-10T00:42:21.941 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-10T00:42:21.983 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-10T00:42:22.017 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-10T00:42:22.066 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-10T00:42:22.100 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-10T00:42:22.105 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-10T00:42:22.128 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-10T00:42:22.148 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-10T00:42:22.163 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-10T00:42:22.187 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-10T00:42:22.202 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-10T00:42:22.234 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-10T00:42:22.242 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-10T00:42:22.271 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-10T00:42:22.297 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-10T00:42:22.331 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-10T00:42:22.342 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-10T00:42:22.373 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-10T00:42:22.388 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-10T00:42:22.397 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-10T00:42:22.407 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-10T00:42:22.436 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-10T00:42:22.470 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-10T00:42:22.496 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-10T00:42:22.547 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-10T00:42:22.567 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-10T00:42:22.577 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-10T00:42:22.625 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-10T00:42:22.625 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-10T00:42:22.678 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-10T00:42:22.736 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-10T00:42:22.765 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-10T00:42:22.796 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-10T00:42:22.798 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-10T00:42:22.975 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-10T00:42:23.004 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-10T00:42:23.081 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-10T00:42:23.145 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-10T00:42:23.202 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-10T00:42:23.236 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-10T00:42:23.340 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-10T00:42:23.392 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-10T00:42:23.437 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-10T00:42:23.527 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-10T00:42:23.554 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-10T00:42:23.572 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-10T00:42:23.600 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-10T00:42:23.634 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-10T00:42:23.717 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-10T00:42:23.736 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-10T00:42:23.763 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-10T00:42:23.790 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-10T00:42:23.795 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-10T00:42:23.834 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-10T00:42:23.835 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-10T00:42:23.873 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-10T00:42:23.925 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-10T00:42:23.970 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-10T00:42:24.153 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-10T00:42:24.184 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-10T00:42:24.264 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-10T00:42:24.535 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-10T00:42:24.645 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-10T00:42:24.943 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-10T00:42:25.045 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-10T00:42:25.061 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-10T00:42:25.132 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-10T00:42:25.168 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-10T00:42:25.210 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-10T00:42:25.529 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-10T00:42:25.565 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-10T00:42:25.655 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-10T00:42:25.686 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-10T00:42:25.722 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-10T00:42:25.913 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-10T00:42:25.999 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-10T00:42:26.027 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-10T00:42:26.060 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-10T00:42:26.192 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-10T00:42:26.218 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-10T00:42:26.288 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-10T00:42:26.319 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-10T00:42:26.372 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-10T00:42:26.372 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-10T00:42:26.423 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-10T00:42:26.454 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-10T00:42:26.474 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-10T00:42:26.498 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-10T00:42:26.499 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-10T00:42:26.528 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-10T00:42:26.565 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-10T00:42:26.825 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-10T00:42:26.870 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-10T00:42:26.945 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-10T00:42:26.975 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-10T00:42:27.012 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-10T00:42:27.037 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-10T00:42:27.076 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-10T00:42:27.147 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-10T00:42:27.177 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-10T00:42:27.507 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-10T00:42:27.539 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-10T00:42:27.594 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-10T00:42:27.625 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-10T00:42:28.600 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T00:42:28.632 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T00:42:29.169 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T00:42:29.411 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-10T00:42:29.415 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T00:42:29.449 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T00:42:29.597 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-10T00:42:29.597 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-10T00:42:29.597 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-10T00:42:29.597 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-10T00:42:29.628 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-10T00:42:29.993 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T00:42:30.013 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T00:42:30.028 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-10T00:42:30.049 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T00:42:30.119 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-10T00:42:30.119 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-10T00:42:30.120 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-10T00:42:30.120 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-10T00:42:30.154 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-10T00:42:30.616 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-10T00:42:30.658 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-10T00:42:30.750 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-10T00:42:30.750 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-10T00:42:30.750 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-10T00:42:30.750 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:42:30.784 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-10T00:42:41.100 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-10T00:42:41.101 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /sys 2024-06-10T00:42:41.101 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /proc 2024-06-10T00:42:41.101 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /mnt 2024-06-10T00:42:41.101 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /var/tmp 2024-06-10T00:42:41.101 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /home 2024-06-10T00:42:41.101 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /root 2024-06-10T00:42:41.101 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /tmp 2024-06-10T00:42:41.101 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-10T00:42:41.272 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-10T00:42:41.287 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-10T00:42:41.288 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:42:41.288 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-10T00:42:41.288 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-10T00:42:41.812 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-10T00:42:41.812 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /sys 2024-06-10T00:42:41.812 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /proc 2024-06-10T00:42:41.813 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /mnt 2024-06-10T00:42:41.813 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /var/tmp 2024-06-10T00:42:41.813 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /home 2024-06-10T00:42:41.813 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /root 2024-06-10T00:42:41.813 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /tmp 2024-06-10T00:42:41.813 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-10T00:42:41.975 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-10T00:42:41.991 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-10T00:42:41.991 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:42:41.991 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-10T00:42:41.991 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-10T00:42:42.737 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-10T00:42:42.738 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /sys 2024-06-10T00:42:42.738 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /proc 2024-06-10T00:42:42.738 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /mnt 2024-06-10T00:42:42.738 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /var/tmp 2024-06-10T00:42:42.738 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /home 2024-06-10T00:42:42.738 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /root 2024-06-10T00:42:42.738 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /tmp 2024-06-10T00:42:42.738 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:42:42.767 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-10T00:42:42.792 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-10T00:42:42.792 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:42:42.793 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-10T00:42:42.793 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-10T00:42:42.793 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-10T00:42:42.793 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-10T00:42:42.906 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-10T00:42:42.922 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-10T00:42:42.923 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:42:42.923 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-10T00:42:42.923 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:42:43.026 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-10T00:42:43.048 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-10T00:42:43.048 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:42:43.048 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-10T00:42:43.048 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-10T00:42:43.048 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-10T00:42:43.048 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-10T00:42:43.461 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-10T00:42:43.485 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-10T00:42:43.488 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-10T00:42:43.488 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:42:43.488 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-10T00:42:43.488 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-10T00:42:43.488 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-10T00:42:43.488 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-10T00:42:43.509 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-10T00:42:43.509 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:42:43.509 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-10T00:42:43.509 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-10T00:42:43.509 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-10T00:42:43.509 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-10T00:42:43.607 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-10T00:42:43.664 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-10T00:42:43.712 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-10T00:42:43.727 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-10T00:42:43.736 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-10T00:42:43.736 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:42:43.736 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-10T00:42:43.736 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-10T00:42:43.736 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-10T00:42:43.736 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-10T00:42:43.913 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-10T00:42:43.970 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-10T00:42:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-10T00:42:44.152 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-10T00:42:44.207 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-10T00:42:44.230 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-10T00:42:44.230 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:42:44.230 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-10T00:42:44.230 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-10T00:42:44.230 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-10T00:42:44.231 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-10T00:42:44.291 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-10T00:42:44.337 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-10T00:42:44.338 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-10T00:42:44.369 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-10T00:42:44.395 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-10T00:42:44.405 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-10T00:42:44.451 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-10T00:42:44.451 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-10T00:42:44.478 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-10T00:42:44.479 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:42:44.479 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-10T00:42:44.479 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-10T00:42:44.479 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-10T00:42:44.479 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:42:44.649 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-10T00:42:44.691 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-10T00:42:44.718 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-10T00:42:44.742 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-10T00:42:44.743 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:42:44.743 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-10T00:42:44.743 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-10T00:42:44.743 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-10T00:42:44.743 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:42:44.771 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-10T00:42:44.841 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-10T00:42:44.859 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-10T00:42:44.871 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-10T00:42:44.902 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-10T00:42:44.965 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-10T00:42:44.994 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-10T00:42:45.010 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-10T00:42:45.050 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-10T00:42:45.082 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-10T00:42:45.110 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-10T00:42:45.219 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-10T00:42:45.243 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-10T00:42:45.243 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:42:45.243 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-10T00:42:45.243 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-10T00:42:45.244 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-10T00:42:45.244 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:42:45.358 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-10T00:42:45.418 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-10T00:42:45.480 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-10T00:42:45.562 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-10T00:42:45.584 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-10T00:42:45.615 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-10T00:42:45.673 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-10T00:42:45.678 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-10T00:42:45.707 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-10T00:42:45.720 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-10T00:42:45.800 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-10T00:42:45.897 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-10T00:42:46.033 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-10T00:42:46.041 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-10T00:42:46.079 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-10T00:42:46.086 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-10T00:42:46.118 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-10T00:42:46.145 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-10T00:42:46.568 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-10T00:42:46.594 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-10T00:42:46.626 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-10T00:42:46.690 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-10T00:42:46.717 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-10T00:42:46.753 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-10T00:42:46.784 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-10T00:42:47.048 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-10T00:42:47.080 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-10T00:42:47.182 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-10T00:42:47.343 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-10T00:42:47.374 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-10T00:42:47.396 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-10T00:42:47.397 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:42:47.397 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-10T00:42:47.397 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-10T00:42:47.397 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-10T00:42:47.397 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-10T00:42:47.441 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-10T00:42:47.452 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-10T00:42:47.567 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-10T00:42:47.615 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-10T00:42:47.756 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-10T00:42:47.784 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-10T00:42:47.801 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-10T00:42:47.844 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-10T00:42:47.889 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-10T00:42:48.048 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-10T00:42:48.070 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-10T00:42:48.093 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-10T00:42:48.094 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:42:48.094 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-10T00:42:48.094 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-10T00:42:48.094 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-10T00:42:48.094 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-10T00:42:48.146 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-10T00:42:48.157 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-10T00:42:48.254 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-10T00:42:48.312 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-10T00:42:48.512 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-10T00:42:48.544 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-10T00:42:48.567 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-10T00:42:48.567 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:42:48.567 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-10T00:42:48.567 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-10T00:42:48.567 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-10T00:42:48.568 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-10T00:42:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-10T00:42:48.633 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-10T00:42:48.633 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:42:48.633 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-10T00:42:48.633 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-10T00:42:48.679 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-10T00:42:48.702 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-10T00:42:48.702 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:42:48.702 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-10T00:42:48.702 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-10T00:42:48.702 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-10T00:42:48.702 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-10T00:42:48.856 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-10T00:42:48.886 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-10T00:42:48.994 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-10T00:42:49.013 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-10T00:42:49.036 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-10T00:42:49.036 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:42:49.036 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-10T00:42:49.036 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-10T00:42:49.036 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-10T00:42:49.036 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-10T00:42:49.189 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-10T00:42:49.212 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-10T00:42:49.221 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-10T00:42:49.240 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-10T00:42:49.246 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-10T00:42:49.246 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:42:49.246 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-10T00:42:49.247 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-10T00:42:49.247 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-10T00:42:49.247 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:42:49.263 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-10T00:42:49.264 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:42:49.264 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-10T00:42:49.264 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-10T00:42:49.264 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-10T00:42:49.264 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-10T00:42:49.293 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-10T00:42:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-10T00:42:49.307 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-10T00:42:49.330 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-10T00:42:49.330 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:42:49.330 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-10T00:42:49.330 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-10T00:42:49.383 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-10T00:42:49.406 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-10T00:42:49.406 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-10T00:42:49.406 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:42:49.407 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-10T00:42:49.407 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-10T00:42:49.407 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-10T00:42:49.407 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-10T00:42:49.473 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-10T00:42:49.717 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-10T00:42:49.740 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-10T00:42:49.740 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:42:49.740 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-10T00:42:49.740 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-10T00:42:49.740 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-10T00:42:49.740 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-10T00:42:50.425 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-10T00:42:50.454 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-10T00:42:50.478 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-10T00:42:50.479 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:42:50.479 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-10T00:42:50.479 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-10T00:42:50.479 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-10T00:42:50.479 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:42:50.530 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-10T00:42:50.553 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-10T00:42:50.553 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:42:50.553 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-10T00:42:50.553 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:42:50.597 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-10T00:42:50.621 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-10T00:42:50.621 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:42:50.621 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-10T00:42:50.621 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-10T00:42:50.622 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-10T00:42:50.622 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:42:50.939 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-10T00:42:50.964 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-10T00:42:50.964 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-10T00:42:50.964 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-10T00:42:50.964 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-10T00:42:50.964 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-10T00:42:50.964 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:42:53.558 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-10T00:42:53.599 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-10T00:42:53.640 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-10T00:42:53.673 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-10T00:42:53.708 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-10T00:42:53.897 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-10T00:42:53.938 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-10T00:42:53.971 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-10T00:42:54.011 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-10T00:42:54.041 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-10T00:42:54.087 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-10T00:42:54.263 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-10T00:42:54.320 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-10T00:42:54.362 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-10T00:42:54.394 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-10T00:42:54.421 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-10T00:42:54.613 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-10T00:42:54.651 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-10T00:42:54.683 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-10T00:42:54.702 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T00:42:54.724 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-10T00:42:54.744 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T00:42:54.753 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-10T00:42:54.764 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T00:42:54.791 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-10T00:42:55.364 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T00:42:55.407 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T00:42:55.427 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T00:42:55.606 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-10T00:42:55.700 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-10T00:42:55.734 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-10T00:42:55.741 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-10T00:42:55.776 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-10T00:42:55.823 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-10T00:42:55.825 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-10T00:42:55.854 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-10T00:42:55.857 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-10T00:42:55.882 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-10T00:42:55.884 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-10T00:42:55.908 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-10T00:42:55.936 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-10T00:42:55.936 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-10T00:42:56.029 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-10T00:42:56.062 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-10T00:42:56.062 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-10T00:42:56.098 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-10T00:42:56.146 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-10T00:42:56.146 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-10T00:42:56.171 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-10T00:42:56.171 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-10T00:42:56.176 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T00:42:56.176 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T00:42:56.203 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-10T00:42:56.232 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-10T00:42:56.270 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-10T00:42:56.312 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-10T00:42:56.364 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T00:42:56.365 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-10T00:42:56.404 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-10T00:42:56.446 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-10T00:42:56.449 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-10T00:42:56.474 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-10T00:42:56.531 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-10T00:42:56.538 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-10T00:42:56.566 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-10T00:42:56.591 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-10T00:42:56.616 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-10T00:42:56.643 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-10T00:42:56.643 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-10T00:42:56.711 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-10T00:42:56.744 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-10T00:42:56.744 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-10T00:42:56.837 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-10T00:42:56.862 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-10T00:42:56.867 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T00:42:56.867 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T00:42:56.908 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T00:42:56.943 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T00:42:56.963 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-10T00:42:57.047 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T00:42:57.047 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-10T00:42:57.131 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-10T00:42:57.164 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-10T00:42:57.238 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-10T00:42:57.862 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-10T00:42:57.964 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-10T00:42:58.006 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-10T00:42:58.099 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-10T00:42:58.142 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-10T00:42:58.166 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-10T00:42:58.192 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-10T00:42:58.218 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-10T00:42:58.218 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-10T00:42:58.303 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-10T00:42:58.335 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-10T00:42:58.336 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-10T00:42:58.412 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-10T00:42:58.437 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-10T00:42:58.443 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T00:42:58.443 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T00:42:58.639 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-10T00:42:58.639 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-10T00:42:58.757 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-10T00:42:58.781 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-10T00:42:58.851 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-10T00:43:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-10T00:43:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-10T00:43:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-10T00:43:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-10T00:43:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-10T00:43:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-10T00:43:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-10T00:43:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-10T00:43:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T00:43:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T00:43:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-10T00:43:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-10T00:43:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-10T00:43:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-10T00:43:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T00:43:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-10T00:43:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-10T00:43:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-10T00:43:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-10T00:43:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-10T00:43:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-10T00:43:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-10T00:43:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-10T00:43:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-10T00:43:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-10T00:43:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-10T00:43:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-10T00:43:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-10T00:43:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-10T00:43:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-10T00:43:02.484 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-10T00:43:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-10T00:43:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-10T00:43:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-10T00:43:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-10T00:43:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-10T00:43:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-10T00:43:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-10T00:43:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-10T00:43:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-10T00:43:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-10T00:43:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-10T00:43:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-10T00:43:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-10T00:43:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-10T00:43:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-10T00:43:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-10T00:43:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-10T00:43:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-10T00:43:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-10T00:43:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-10T00:43:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-10T00:43:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-10T00:43:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-10T00:43:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-10T00:43:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-10T00:43:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-10T00:43:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-10T00:43:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-10T00:43:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-10T00:43:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-10T00:43:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-10T00:43:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-10T00:43:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-10T00:43:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-10T00:43:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-10T00:43:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-10T00:43:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-10T00:43:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-10T00:43:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-10T00:43:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-10T00:43:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-10T00:43:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-10T00:43:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-10T00:43:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-10T00:43:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-10T00:43:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-10T00:43:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-10T00:43:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-10T00:43:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-10T00:43:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-10T00:43:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-10T00:43:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-10T00:43:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-10T00:43:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-10T00:43:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-10T00:43:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-10T00:43:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-10T00:43:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-10T00:43:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-10T00:43:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-10T00:43:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-10T00:43:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-10T00:43:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-10T00:43:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-10T00:43:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-10T00:43:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-10T00:43:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-10T00:43:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-10T00:43:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-10T00:43:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-10T00:43:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-10T00:43:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-10T00:43:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-10T00:43:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-10T00:43:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-10T00:43:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-10T00:43:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-10T00:43:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-10T00:43:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-10T00:43:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-10T00:43:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-10T00:43:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-10T00:43:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-10T00:43:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-10T00:43:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-10T00:43:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-10T00:43:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-10T00:43:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-10T00:43:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-10T00:43:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-10T00:43:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-10T00:43:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-google-auth-1:2.29.0-2.el9.noarch 122/170 2024-06-10T00:43:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-10T00:43:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-10T00:43:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-10T00:43:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-10T00:43:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-10T00:43:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-10T00:43:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-10T00:43:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-10T00:43:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-10T00:43:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-10T00:43:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-10T00:43:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-10T00:43:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-10T00:43:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-10T00:43:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-10T00:43:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-10T00:43:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-10T00:43:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-10T00:43:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-10T00:43:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-10T00:43:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-10T00:43:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-10T00:43:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-10T00:43:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-10T00:43:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-10T00:43:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-10T00:43:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-10T00:43:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-10T00:43:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-10T00:43:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-10T00:43:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-10T00:43:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-10T00:43:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-10T00:43:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-10T00:43:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-10T00:43:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-10T00:43:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-10T00:43:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-10T00:43:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-10T00:43:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-10T00:43:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-10T00:43:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-10T00:43:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-10T00:43:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-10T00:43:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-10T00:43:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-10T00:43:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-10T00:43:02.780 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-10T00:43:02.780 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-10T00:43:02.780 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-10T00:43:02.780 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-10T00:43:02.780 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-10T00:43:02.780 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-10T00:43:02.780 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-10T00:43:02.780 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-10T00:43:02.780 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T00:43:02.780 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T00:43:02.780 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-10T00:43:02.781 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-10T00:43:02.781 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-10T00:43:02.781 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-10T00:43:02.781 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T00:43:02.781 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-10T00:43:02.781 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-10T00:43:02.781 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-10T00:43:02.781 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-10T00:43:02.781 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-10T00:43:02.781 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-10T00:43:02.781 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-10T00:43:02.781 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-10T00:43:02.781 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-10T00:43:02.781 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-10T00:43:02.781 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-10T00:43:02.782 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-10T00:43:02.782 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-10T00:43:02.782 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-10T00:43:02.782 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-10T00:43:02.782 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-10T00:43:02.782 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-10T00:43:02.782 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-10T00:43:02.782 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-10T00:43:02.782 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-10T00:43:02.782 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-10T00:43:02.782 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-10T00:43:02.782 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-10T00:43:02.782 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-10T00:43:02.782 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-10T00:43:02.783 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-10T00:43:02.783 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-10T00:43:02.783 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-10T00:43:02.783 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-10T00:43:02.783 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-10T00:43:02.783 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-10T00:43:02.783 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-10T00:43:02.783 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-10T00:43:02.783 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-10T00:43:02.783 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-10T00:43:02.783 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-10T00:43:02.783 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-10T00:43:02.783 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-10T00:43:02.783 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-10T00:43:02.783 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-10T00:43:02.784 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-10T00:43:02.784 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-10T00:43:02.784 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-10T00:43:02.784 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-10T00:43:02.784 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-10T00:43:02.784 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-10T00:43:02.784 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-10T00:43:02.784 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-10T00:43:02.784 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-10T00:43:02.784 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-10T00:43:02.784 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-10T00:43:02.784 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-10T00:43:02.784 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-10T00:43:02.784 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-10T00:43:02.784 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-10T00:43:02.784 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-10T00:43:02.785 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-10T00:43:02.785 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-10T00:43:02.785 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-10T00:43:02.785 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-10T00:43:02.785 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-10T00:43:02.785 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-10T00:43:02.785 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-10T00:43:02.785 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-10T00:43:02.785 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-10T00:43:02.785 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-10T00:43:02.785 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-10T00:43:02.785 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-10T00:43:02.785 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-10T00:43:02.785 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-10T00:43:02.786 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-10T00:43:02.786 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-10T00:43:02.786 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-10T00:43:02.786 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-10T00:43:02.786 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-10T00:43:02.786 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-10T00:43:02.786 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-10T00:43:02.786 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-10T00:43:02.786 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-10T00:43:02.786 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-10T00:43:02.786 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-10T00:43:02.786 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-10T00:43:02.786 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-10T00:43:02.787 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-10T00:43:02.787 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-10T00:43:02.787 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-10T00:43:02.787 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-10T00:43:02.787 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-10T00:43:02.787 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-10T00:43:02.787 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-10T00:43:02.787 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-10T00:43:02.787 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-10T00:43:02.787 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-10T00:43:02.787 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-10T00:43:02.787 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-10T00:43:02.788 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-10T00:43:02.788 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-10T00:43:02.788 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-10T00:43:02.788 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-10T00:43:02.788 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-10T00:43:02.788 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-10T00:43:02.788 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-10T00:43:02.788 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-10T00:43:02.788 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-10T00:43:02.788 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-10T00:43:02.788 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-10T00:43:02.788 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-10T00:43:02.788 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-google-auth-1:2.29.0-2.el9.noarch 122/170 2024-06-10T00:43:02.788 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-10T00:43:02.789 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-10T00:43:02.789 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-10T00:43:02.789 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-10T00:43:02.789 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-10T00:43:02.789 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-10T00:43:02.789 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-10T00:43:02.789 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-10T00:43:02.789 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-10T00:43:02.789 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-10T00:43:02.789 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-10T00:43:02.789 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-10T00:43:02.789 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-10T00:43:02.790 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-10T00:43:02.790 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-10T00:43:02.790 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-10T00:43:02.790 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-10T00:43:02.790 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-10T00:43:02.790 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-10T00:43:02.790 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-10T00:43:02.790 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-10T00:43:02.790 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-10T00:43:02.790 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-10T00:43:02.791 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-10T00:43:02.791 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-10T00:43:02.791 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-10T00:43:02.791 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-10T00:43:02.791 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-10T00:43:02.791 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-10T00:43:02.791 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-10T00:43:02.791 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-10T00:43:02.791 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-10T00:43:02.791 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-10T00:43:02.791 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-10T00:43:02.791 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-10T00:43:02.792 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-10T00:43:02.792 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-10T00:43:02.792 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-10T00:43:02.792 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-10T00:43:02.792 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-10T00:43:02.792 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-10T00:43:02.792 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-10T00:43:02.792 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-10T00:43:02.792 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-10T00:43:02.792 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-10T00:43:02.792 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-10T00:43:02.793 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-10T00:43:04.360 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-10T00:43:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-10T00:43:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-10T00:43:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-10T00:43:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-10T00:43:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-10T00:43:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-10T00:43:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-10T00:43:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-10T00:43:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-10T00:43:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-10T00:43:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-10T00:43:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-10T00:43:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-10T00:43:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-10T00:43:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-10T00:43:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-10T00:43:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-10T00:43:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-10T00:43:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-10T00:43:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-10T00:43:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-10T00:43:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-10T00:43:04.363 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-10T00:43:04.363 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-10T00:43:04.363 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-10T00:43:04.363 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-10T00:43:04.363 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-10T00:43:04.363 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-10T00:43:04.375 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-10T00:43:04.375 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-10T00:43:04.375 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-10T00:43:04.375 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-10T00:43:04.375 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-10T00:43:04.375 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-10T00:43:04.375 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-10T00:43:04.375 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-10T00:43:04.375 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-10T00:43:04.375 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-10T00:43:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-10T00:43:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-10T00:43:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-10T00:43:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-10T00:43:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-10T00:43:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-10T00:43:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-10T00:43:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-10T00:43:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-10T00:43:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-10T00:43:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-10T00:43:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-10T00:43:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-10T00:43:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-10T00:43:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-10T00:43:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-10T00:43:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-10T00:43:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-10T00:43:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-10T00:43:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-10T00:43:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-10T00:43:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-10T00:43:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-10T00:43:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-10T00:43:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-10T00:43:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-10T00:43:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-10T00:43:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-10T00:43:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-10T00:43:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-10T00:43:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-10T00:43:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-10T00:43:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-10T00:43:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-10T00:43:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-10T00:43:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-10T00:43:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-10T00:43:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-10T00:43:04.379 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-10T00:43:04.379 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-10T00:43:04.379 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-10T00:43:04.379 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-10T00:43:04.379 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-10T00:43:04.379 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-10T00:43:04.379 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-10T00:43:04.379 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-10T00:43:04.379 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-10T00:43:04.379 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-10T00:43:04.379 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-10T00:43:04.379 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-10T00:43:04.379 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-10T00:43:04.379 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-10T00:43:04.380 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-10T00:43:04.380 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-10T00:43:04.380 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-10T00:43:04.380 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-10T00:43:04.380 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-10T00:43:04.380 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-10T00:43:04.380 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-10T00:43:04.380 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-10T00:43:04.380 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-10T00:43:04.380 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-10T00:43:04.380 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-10T00:43:04.380 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-10T00:43:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-10T00:43:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-10T00:43:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-10T00:43:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-10T00:43:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-10T00:43:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-10T00:43:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-10T00:43:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-10T00:43:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-10T00:43:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-10T00:43:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-10T00:43:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-10T00:43:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-10T00:43:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-10T00:43:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-10T00:43:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-10T00:43:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-10T00:43:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-10T00:43:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-10T00:43:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-google-auth-1:2.29.0-2.el9.noarch 122/170 2024-06-10T00:43:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-10T00:43:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-10T00:43:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-10T00:43:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-10T00:43:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-10T00:43:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-10T00:43:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-10T00:43:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-10T00:43:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-10T00:43:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-10T00:43:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-10T00:43:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-10T00:43:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-10T00:43:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-10T00:43:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-10T00:43:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-10T00:43:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-10T00:43:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-10T00:43:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-10T00:43:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-10T00:43:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-10T00:43:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-10T00:43:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-10T00:43:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-10T00:43:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-10T00:43:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-10T00:43:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-10T00:43:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-10T00:43:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-10T00:43:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-10T00:43:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-10T00:43:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-10T00:43:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-10T00:43:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-10T00:43:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-10T00:43:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-10T00:43:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-10T00:43:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-10T00:43:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-10T00:43:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-10T00:43:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-10T00:43:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-10T00:43:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-10T00:43:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-10T00:43:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-10T00:43:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-10T00:43:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-10T00:43:05.214 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-10T00:43:05.214 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-10T00:43:05.215 INFO:teuthology.orchestra.run.smithi094.stdout:Upgraded: 2024-06-10T00:43:05.215 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-10T00:43:05.215 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-10T00:43:05.215 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-10T00:43:05.215 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-10T00:43:05.215 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-10T00:43:05.215 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-10T00:43:05.215 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-10T00:43:05.215 INFO:teuthology.orchestra.run.smithi094.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.215 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.215 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-10T00:43:05.215 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-10T00:43:05.215 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-06-10T00:43:05.216 INFO:teuthology.orchestra.run.smithi094.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-10T00:43:05.216 INFO:teuthology.orchestra.run.smithi094.stdout: automake-1.16.2-8.el9.noarch 2024-06-10T00:43:05.216 INFO:teuthology.orchestra.run.smithi094.stdout: bison-3.7.4-5.el9.x86_64 2024-06-10T00:43:05.216 INFO:teuthology.orchestra.run.smithi094.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-10T00:43:05.216 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.216 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.216 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.216 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.216 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:05.216 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.216 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.216 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.217 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:05.217 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:05.217 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:05.217 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:05.217 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:05.217 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.217 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.217 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:05.217 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.217 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.217 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.217 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:05.217 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:05.217 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.218 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.218 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:05.218 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-10T00:43:05.218 INFO:teuthology.orchestra.run.smithi094.stdout: flex-2.6.4-9.el9.x86_64 2024-06-10T00:43:05.218 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-10T00:43:05.218 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-10T00:43:05.218 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-10T00:43:05.218 INFO:teuthology.orchestra.run.smithi094.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-10T00:43:05.218 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-10T00:43:05.218 INFO:teuthology.orchestra.run.smithi094.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-10T00:43:05.218 INFO:teuthology.orchestra.run.smithi094.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-10T00:43:05.218 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-10T00:43:05.219 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-10T00:43:05.219 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.219 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.219 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.219 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-10T00:43:05.219 INFO:teuthology.orchestra.run.smithi094.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-10T00:43:05.219 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-10T00:43:05.219 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-10T00:43:05.219 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-10T00:43:05.219 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.219 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.219 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-10T00:43:05.219 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.219 INFO:teuthology.orchestra.run.smithi094.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-10T00:43:05.220 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-10T00:43:05.220 INFO:teuthology.orchestra.run.smithi094.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-10T00:43:05.220 INFO:teuthology.orchestra.run.smithi094.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-10T00:43:05.220 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-10T00:43:05.220 INFO:teuthology.orchestra.run.smithi094.stdout: lua-5.4.4-4.el9.x86_64 2024-06-10T00:43:05.220 INFO:teuthology.orchestra.run.smithi094.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-10T00:43:05.220 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-10T00:43:05.220 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-10T00:43:05.220 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-10T00:43:05.220 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-10T00:43:05.220 INFO:teuthology.orchestra.run.smithi094.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-10T00:43:05.220 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-13.14-1.el9.x86_64 2024-06-10T00:43:05.220 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-10T00:43:05.221 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-10T00:43:05.221 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-10T00:43:05.221 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-10T00:43:05.221 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-10T00:43:05.221 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-10T00:43:05.221 INFO:teuthology.orchestra.run.smithi094.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-10T00:43:05.221 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-10T00:43:05.221 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-10T00:43:05.221 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-10T00:43:05.221 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.221 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.221 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.221 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-10T00:43:05.221 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-10T00:43:05.222 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-10T00:43:05.222 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-10T00:43:05.222 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-10T00:43:05.222 INFO:teuthology.orchestra.run.smithi094.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-10T00:43:05.222 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-10T00:43:05.222 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-10T00:43:05.222 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth-1:2.29.0-2.el9.noarch 2024-06-10T00:43:05.222 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-10T00:43:05.222 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-10T00:43:05.222 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-10T00:43:05.222 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-10T00:43:05.222 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-10T00:43:05.222 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-10T00:43:05.223 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-10T00:43:05.223 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-10T00:43:05.223 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-10T00:43:05.223 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-10T00:43:05.223 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-10T00:43:05.223 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-10T00:43:05.223 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-10T00:43:05.223 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-10T00:43:05.223 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-10T00:43:05.223 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-10T00:43:05.223 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-10T00:43:05.223 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-10T00:43:05.223 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-10T00:43:05.223 INFO:teuthology.orchestra.run.smithi094.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-10T00:43:05.224 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-10T00:43:05.224 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply-3.11-14.el9.noarch 2024-06-10T00:43:05.224 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-10T00:43:05.224 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-10T00:43:05.224 INFO:teuthology.orchestra.run.smithi094.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-10T00:43:05.224 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-10T00:43:05.224 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-10T00:43:05.224 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-10T00:43:05.224 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-10T00:43:05.224 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-10T00:43:05.224 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-10T00:43:05.224 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.224 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.224 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-10T00:43:05.224 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-10T00:43:05.225 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-10T00:43:05.225 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.225 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-10T00:43:05.225 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-10T00:43:05.225 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-10T00:43:05.225 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-10T00:43:05.225 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-10T00:43:05.225 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-10T00:43:05.225 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-10T00:43:05.225 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-10T00:43:05.225 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-10T00:43:05.225 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-10T00:43:05.225 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-10T00:43:05.225 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-10T00:43:05.226 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-10T00:43:05.226 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-10T00:43:05.226 INFO:teuthology.orchestra.run.smithi094.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-10T00:43:05.226 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.226 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.226 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.226 INFO:teuthology.orchestra.run.smithi094.stdout: re2-1:20211101-3.el9.x86_64 2024-06-10T00:43:05.226 INFO:teuthology.orchestra.run.smithi094.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-10T00:43:05.226 INFO:teuthology.orchestra.run.smithi094.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-10T00:43:05.226 INFO:teuthology.orchestra.run.smithi094.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-10T00:43:05.226 INFO:teuthology.orchestra.run.smithi094.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-10T00:43:05.226 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-10T00:43:05.227 INFO:teuthology.orchestra.run.smithi094.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-10T00:43:05.227 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-10T00:43:05.227 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-06-10T00:43:05.490 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-10T00:43:05.491 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-10T00:43:05.491 INFO:teuthology.orchestra.run.smithi167.stdout:Upgraded: 2024-06-10T00:43:05.491 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-10T00:43:05.491 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-10T00:43:05.491 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-10T00:43:05.491 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-10T00:43:05.491 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-10T00:43:05.491 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-10T00:43:05.491 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-10T00:43:05.491 INFO:teuthology.orchestra.run.smithi167.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.491 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.492 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-10T00:43:05.492 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-10T00:43:05.492 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-06-10T00:43:05.492 INFO:teuthology.orchestra.run.smithi167.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-10T00:43:05.492 INFO:teuthology.orchestra.run.smithi167.stdout: automake-1.16.2-8.el9.noarch 2024-06-10T00:43:05.492 INFO:teuthology.orchestra.run.smithi167.stdout: bison-3.7.4-5.el9.x86_64 2024-06-10T00:43:05.492 INFO:teuthology.orchestra.run.smithi167.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-10T00:43:05.492 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.492 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.492 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.492 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.492 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:05.493 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.493 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.493 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.493 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:05.493 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:05.493 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:05.493 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:05.493 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:05.493 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.493 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.493 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:05.493 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.493 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.494 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.494 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:05.494 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:05.494 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.494 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.494 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:05.494 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-10T00:43:05.494 INFO:teuthology.orchestra.run.smithi167.stdout: flex-2.6.4-9.el9.x86_64 2024-06-10T00:43:05.494 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-10T00:43:05.494 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-10T00:43:05.494 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-10T00:43:05.494 INFO:teuthology.orchestra.run.smithi167.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-10T00:43:05.495 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-10T00:43:05.495 INFO:teuthology.orchestra.run.smithi167.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-10T00:43:05.495 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-10T00:43:05.495 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-10T00:43:05.495 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-10T00:43:05.495 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.495 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.495 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.495 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-10T00:43:05.495 INFO:teuthology.orchestra.run.smithi167.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-10T00:43:05.495 INFO:teuthology.orchestra.run.smithi167.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-10T00:43:05.495 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-10T00:43:05.495 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-10T00:43:05.496 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.496 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.496 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-10T00:43:05.496 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.496 INFO:teuthology.orchestra.run.smithi167.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-10T00:43:05.496 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-10T00:43:05.496 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-10T00:43:05.496 INFO:teuthology.orchestra.run.smithi167.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-10T00:43:05.496 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-10T00:43:05.496 INFO:teuthology.orchestra.run.smithi167.stdout: lua-5.4.4-4.el9.x86_64 2024-06-10T00:43:05.496 INFO:teuthology.orchestra.run.smithi167.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-10T00:43:05.496 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-10T00:43:05.497 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-10T00:43:05.497 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-10T00:43:05.497 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-10T00:43:05.497 INFO:teuthology.orchestra.run.smithi167.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-10T00:43:05.497 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-13.14-1.el9.x86_64 2024-06-10T00:43:05.497 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-10T00:43:05.497 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-10T00:43:05.497 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-10T00:43:05.497 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-10T00:43:05.497 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-10T00:43:05.497 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-10T00:43:05.497 INFO:teuthology.orchestra.run.smithi167.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-10T00:43:05.497 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-10T00:43:05.498 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-10T00:43:05.498 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-10T00:43:05.498 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.498 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.498 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.498 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-10T00:43:05.498 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-10T00:43:05.498 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-10T00:43:05.498 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-10T00:43:05.498 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-10T00:43:05.499 INFO:teuthology.orchestra.run.smithi167.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-10T00:43:05.499 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-10T00:43:05.499 INFO:teuthology.orchestra.run.smithi167.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-10T00:43:05.499 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth-1:2.29.0-2.el9.noarch 2024-06-10T00:43:05.499 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-10T00:43:05.499 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-10T00:43:05.499 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-10T00:43:05.499 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-10T00:43:05.499 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-10T00:43:05.499 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-10T00:43:05.499 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-10T00:43:05.499 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-10T00:43:05.500 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-10T00:43:05.500 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-10T00:43:05.500 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-10T00:43:05.500 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-10T00:43:05.500 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-10T00:43:05.500 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-10T00:43:05.500 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-10T00:43:05.500 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-10T00:43:05.500 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-10T00:43:05.500 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-10T00:43:05.500 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-10T00:43:05.500 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-10T00:43:05.500 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-10T00:43:05.500 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ply-3.11-14.el9.noarch 2024-06-10T00:43:05.500 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-10T00:43:05.501 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-10T00:43:05.501 INFO:teuthology.orchestra.run.smithi167.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-10T00:43:05.501 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-10T00:43:05.501 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-10T00:43:05.501 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-10T00:43:05.501 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-10T00:43:05.501 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-10T00:43:05.501 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-10T00:43:05.501 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.501 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.501 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-10T00:43:05.501 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-10T00:43:05.501 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-10T00:43:05.501 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.502 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-10T00:43:05.502 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-10T00:43:05.502 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-10T00:43:05.502 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-10T00:43:05.502 INFO:teuthology.orchestra.run.smithi167.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-10T00:43:05.502 INFO:teuthology.orchestra.run.smithi167.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-10T00:43:05.502 INFO:teuthology.orchestra.run.smithi167.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-10T00:43:05.502 INFO:teuthology.orchestra.run.smithi167.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-10T00:43:05.502 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-10T00:43:05.502 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-10T00:43:05.502 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-10T00:43:05.502 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-10T00:43:05.502 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-10T00:43:05.503 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-10T00:43:05.503 INFO:teuthology.orchestra.run.smithi167.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-10T00:43:05.503 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.503 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.503 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:05.503 INFO:teuthology.orchestra.run.smithi167.stdout: re2-1:20211101-3.el9.x86_64 2024-06-10T00:43:05.503 INFO:teuthology.orchestra.run.smithi167.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-10T00:43:05.503 INFO:teuthology.orchestra.run.smithi167.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-10T00:43:05.503 INFO:teuthology.orchestra.run.smithi167.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-10T00:43:05.503 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-10T00:43:05.503 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-10T00:43:05.504 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-10T00:43:05.504 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-10T00:43:05.504 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-06-10T00:43:05.656 DEBUG:teuthology.parallel:result is None 2024-06-10T00:43:05.967 DEBUG:teuthology.parallel:result is None 2024-06-10T00:43:06.909 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-10T00:43:06.909 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:43:06.910 INFO:teuthology.orchestra.run.smithi037.stdout:Upgraded: 2024-06-10T00:43:06.910 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-10T00:43:06.910 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-10T00:43:06.910 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-10T00:43:06.910 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-10T00:43:06.910 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-10T00:43:06.910 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-10T00:43:06.910 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-10T00:43:06.910 INFO:teuthology.orchestra.run.smithi037.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.911 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.911 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-10T00:43:06.911 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-10T00:43:06.911 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-06-10T00:43:06.911 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-10T00:43:06.911 INFO:teuthology.orchestra.run.smithi037.stdout: automake-1.16.2-8.el9.noarch 2024-06-10T00:43:06.911 INFO:teuthology.orchestra.run.smithi037.stdout: bison-3.7.4-5.el9.x86_64 2024-06-10T00:43:06.911 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-10T00:43:06.911 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.911 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.911 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.912 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.912 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:06.912 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.912 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.912 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.912 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:06.912 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:06.912 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:06.912 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:06.912 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:06.912 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.913 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.913 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:06.913 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.913 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.913 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.913 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:06.913 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:06.913 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.913 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.913 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-10T00:43:06.913 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-10T00:43:06.914 INFO:teuthology.orchestra.run.smithi037.stdout: flex-2.6.4-9.el9.x86_64 2024-06-10T00:43:06.914 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-10T00:43:06.914 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-10T00:43:06.914 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-10T00:43:06.914 INFO:teuthology.orchestra.run.smithi037.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-10T00:43:06.914 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-10T00:43:06.914 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-10T00:43:06.914 INFO:teuthology.orchestra.run.smithi037.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-10T00:43:06.914 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-10T00:43:06.914 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-10T00:43:06.915 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.915 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.915 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.915 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-10T00:43:06.915 INFO:teuthology.orchestra.run.smithi037.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-10T00:43:06.915 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-10T00:43:06.915 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-10T00:43:06.915 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-10T00:43:06.915 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.915 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.915 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-10T00:43:06.916 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.916 INFO:teuthology.orchestra.run.smithi037.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-10T00:43:06.916 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-10T00:43:06.916 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-10T00:43:06.916 INFO:teuthology.orchestra.run.smithi037.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-10T00:43:06.916 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-10T00:43:06.916 INFO:teuthology.orchestra.run.smithi037.stdout: lua-5.4.4-4.el9.x86_64 2024-06-10T00:43:06.916 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-10T00:43:06.916 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-10T00:43:06.916 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-10T00:43:06.917 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-10T00:43:06.917 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-10T00:43:06.917 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-10T00:43:06.917 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-13.14-1.el9.x86_64 2024-06-10T00:43:06.917 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-10T00:43:06.917 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-10T00:43:06.917 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-10T00:43:06.917 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-10T00:43:06.917 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-10T00:43:06.917 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-10T00:43:06.917 INFO:teuthology.orchestra.run.smithi037.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-10T00:43:06.918 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-10T00:43:06.918 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-10T00:43:06.918 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-10T00:43:06.918 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.918 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.918 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.918 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-10T00:43:06.918 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-10T00:43:06.918 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-10T00:43:06.918 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-10T00:43:06.919 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-10T00:43:06.919 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-10T00:43:06.919 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-10T00:43:06.919 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-10T00:43:06.919 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth-1:2.29.0-2.el9.noarch 2024-06-10T00:43:06.919 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-10T00:43:06.919 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-10T00:43:06.919 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-10T00:43:06.919 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-10T00:43:06.919 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-10T00:43:06.919 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-10T00:43:06.919 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-10T00:43:06.920 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-10T00:43:06.920 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-10T00:43:06.920 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-10T00:43:06.920 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-10T00:43:06.920 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-10T00:43:06.920 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-10T00:43:06.920 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-10T00:43:06.920 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-10T00:43:06.920 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-10T00:43:06.920 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-10T00:43:06.920 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-10T00:43:06.920 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-10T00:43:06.921 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-10T00:43:06.921 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-10T00:43:06.921 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply-3.11-14.el9.noarch 2024-06-10T00:43:06.921 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-10T00:43:06.921 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-10T00:43:06.921 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-10T00:43:06.921 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-10T00:43:06.921 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-10T00:43:06.921 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-10T00:43:06.921 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-10T00:43:06.921 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-10T00:43:06.921 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-10T00:43:06.922 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.922 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.922 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-10T00:43:06.922 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-10T00:43:06.922 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-10T00:43:06.922 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.922 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-10T00:43:06.922 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-10T00:43:06.922 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-10T00:43:06.922 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-10T00:43:06.923 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-10T00:43:06.923 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-10T00:43:06.923 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-10T00:43:06.923 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-10T00:43:06.923 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-10T00:43:06.923 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-10T00:43:06.923 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-10T00:43:06.923 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-10T00:43:06.923 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-10T00:43:06.923 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-10T00:43:06.924 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-10T00:43:06.924 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.924 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.924 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-10T00:43:06.924 INFO:teuthology.orchestra.run.smithi037.stdout: re2-1:20211101-3.el9.x86_64 2024-06-10T00:43:06.924 INFO:teuthology.orchestra.run.smithi037.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-10T00:43:06.924 INFO:teuthology.orchestra.run.smithi037.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-10T00:43:06.924 INFO:teuthology.orchestra.run.smithi037.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-10T00:43:06.924 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-10T00:43:06.924 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-10T00:43:06.925 INFO:teuthology.orchestra.run.smithi037.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-10T00:43:06.925 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:43:06.925 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-06-10T00:43:07.336 DEBUG:teuthology.parallel:result is None 2024-06-10T00:43:07.337 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:43:07.565 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-10T00:43:07.589 INFO:teuthology.orchestra.run.smithi037.stdout:19.0.0-2946.g3d89accf.el9 2024-06-10T00:43:07.590 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-10T00:43:07.590 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-10T00:43:07.592 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:43:07.729 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-10T00:43:07.754 INFO:teuthology.orchestra.run.smithi094.stdout:19.0.0-2946.g3d89accf.el9 2024-06-10T00:43:07.754 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-10T00:43:07.754 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-10T00:43:07.756 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:43:07.904 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-10T00:43:07.930 INFO:teuthology.orchestra.run.smithi167.stdout:19.0.0-2946.g3d89accf.el9 2024-06-10T00:43:07.930 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-10T00:43:07.930 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-10T00:43:07.932 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-06-10T00:43:07.932 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-06-10T00:43:07.932 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-10T00:43:07.966 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-10T00:43:07.985 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-10T00:43:08.017 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-10T00:43:08.017 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-10T00:43:08.047 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-06-10T00:43:08.047 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-06-10T00:43:08.047 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/daemon-helper 2024-06-10T00:43:08.078 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-10T00:43:08.147 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-10T00:43:08.147 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/daemon-helper 2024-06-10T00:43:08.176 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-10T00:43:08.243 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-10T00:43:08.243 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/daemon-helper 2024-06-10T00:43:08.271 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-10T00:43:08.339 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-06-10T00:43:08.339 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-06-10T00:43:08.339 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-10T00:43:08.369 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-10T00:43:08.439 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-10T00:43:08.439 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-10T00:43:08.467 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-10T00:43:08.535 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-10T00:43:08.535 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-10T00:43:08.563 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-10T00:43:08.630 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-06-10T00:43:08.631 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-06-10T00:43:08.631 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/stdin-killer 2024-06-10T00:43:08.661 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-10T00:43:08.732 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-10T00:43:08.732 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/stdin-killer 2024-06-10T00:43:08.760 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-10T00:43:08.826 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-10T00:43:08.826 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/stdin-killer 2024-06-10T00:43:08.854 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-10T00:43:08.920 INFO:teuthology.run_tasks:Running task cephadm... 2024-06-10T00:43:09.005 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-06-10T00:43:09.005 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:43:09.006 INFO:tasks.cephadm:Cluster fsid is 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:43:09.006 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-06-10T00:43:09.006 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.37', 'mon.b': '172.21.15.94', 'mon.c': '172.21.15.167'} 2024-06-10T00:43:09.006 INFO:tasks.cephadm:First mon is mon.a on smithi037 2024-06-10T00:43:09.006 INFO:tasks.cephadm:First mgr is x 2024-06-10T00:43:09.006 INFO:tasks.cephadm:Normalizing hostnames... 2024-06-10T00:43:09.007 DEBUG:teuthology.orchestra.run.smithi037:> sudo hostname $(hostname -s) 2024-06-10T00:43:09.038 DEBUG:teuthology.orchestra.run.smithi094:> sudo hostname $(hostname -s) 2024-06-10T00:43:09.065 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname $(hostname -s) 2024-06-10T00:43:09.092 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-06-10T00:43:09.092 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:43:09.227 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55273/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2946-g3d89accf', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2946.g3d89accf'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-30 17:04:17.570799', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-06-10T00:43:09.363 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 3d89accfe3201286f000e065a1faea2a88c7a6ba from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:43:09.365 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm 2024-06-10T00:43:09.365 DEBUG:teuthology.orchestra.run.smithi037:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-10T00:43:09.786 INFO:teuthology.orchestra.run.smithi037.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 10 00:43 /home/ubuntu/cephtest/cephadm 2024-06-10T00:43:09.787 DEBUG:teuthology.orchestra.run.smithi094:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-10T00:43:10.382 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 10 00:43 /home/ubuntu/cephtest/cephadm 2024-06-10T00:43:10.382 DEBUG:teuthology.orchestra.run.smithi167:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-10T00:43:10.779 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 10 00:43 /home/ubuntu/cephtest/cephadm 2024-06-10T00:43:10.779 DEBUG:teuthology.orchestra.run.smithi037:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-10T00:43:10.800 DEBUG:teuthology.orchestra.run.smithi094:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-10T00:43:10.818 DEBUG:teuthology.orchestra.run.smithi167:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-10T00:43:10.852 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba on all hosts... 2024-06-10T00:43:10.853 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-10T00:43:10.855 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-10T00:43:10.861 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-10T00:43:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-10T00:43:11.788 INFO:teuthology.orchestra.run.smithi167.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-10T00:43:11.886 INFO:teuthology.orchestra.run.smithi094.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-10T00:43:41.121 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-06-10T00:43:41.122 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-10T00:43:41.122 INFO:teuthology.orchestra.run.smithi094.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-10T00:43:41.122 INFO:teuthology.orchestra.run.smithi094.stdout: "repo_digests": [ 2024-06-10T00:43:41.122 INFO:teuthology.orchestra.run.smithi094.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-10T00:43:41.122 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-06-10T00:43:41.122 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-06-10T00:44:06.161 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T00:44:06.161 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-10T00:44:06.161 INFO:teuthology.orchestra.run.smithi037.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-10T00:44:06.162 INFO:teuthology.orchestra.run.smithi037.stdout: "repo_digests": [ 2024-06-10T00:44:06.162 INFO:teuthology.orchestra.run.smithi037.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-10T00:44:06.162 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-06-10T00:44:06.162 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T00:44:14.954 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-06-10T00:44:14.954 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-10T00:44:14.954 INFO:teuthology.orchestra.run.smithi167.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-10T00:44:14.954 INFO:teuthology.orchestra.run.smithi167.stdout: "repo_digests": [ 2024-06-10T00:44:14.954 INFO:teuthology.orchestra.run.smithi167.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-10T00:44:14.955 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-06-10T00:44:14.955 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-06-10T00:44:14.979 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /etc/ceph 2024-06-10T00:44:15.013 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /etc/ceph 2024-06-10T00:44:15.046 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /etc/ceph 2024-06-10T00:44:15.073 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 777 /etc/ceph 2024-06-10T00:44:15.103 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 777 /etc/ceph 2024-06-10T00:44:15.129 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 777 /etc/ceph 2024-06-10T00:44:15.153 INFO:tasks.cephadm:Writing seed config... 2024-06-10T00:44:15.154 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-06-10T00:44:15.154 INFO:tasks.cephadm: override: [client] debug client = 20 2024-06-10T00:44:15.154 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-06-10T00:44:15.154 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-06-10T00:44:15.154 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-06-10T00:44:15.154 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-06-10T00:44:15.154 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-06-10T00:44:15.154 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-06-10T00:44:15.155 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-06-10T00:44:15.155 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-06-10T00:44:15.155 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-06-10T00:44:15.155 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-06-10T00:44:15.155 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-06-10T00:44:15.155 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-06-10T00:44:15.155 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-06-10T00:44:15.155 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-06-10T00:44:15.155 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-06-10T00:44:15.155 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-06-10T00:44:15.155 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-06-10T00:44:15.155 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-06-10T00:44:15.155 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-06-10T00:44:15.155 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-06-10T00:44:15.156 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-06-10T00:44:15.156 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-06-10T00:44:15.156 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-06-10T00:44:15.156 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-06-10T00:44:15.156 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-06-10T00:44:15.156 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-06-10T00:44:15.156 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-06-10T00:44:15.156 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-06-10T00:44:15.156 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-06-10T00:44:15.156 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-06-10T00:44:15.156 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-06-10T00:44:15.156 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-06-10T00:44:15.156 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-06-10T00:44:15.156 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-06-10T00:44:15.157 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-06-10T00:44:15.157 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-06-10T00:44:15.157 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-06-10T00:44:15.157 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-06-10T00:44:15.157 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-06-10T00:44:15.157 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-06-10T00:44:15.157 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-06-10T00:44:15.157 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-06-10T00:44:15.157 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-06-10T00:44:15.157 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-06-10T00:44:15.158 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-06-10T00:44:15.158 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-06-10T00:44:15.177 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 68793514-26c2-11ef-bc9b-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-06-10T00:44:15.177 DEBUG:teuthology.orchestra.run.smithi037:mon.a> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mon.a.service 2024-06-10T00:44:15.220 DEBUG:teuthology.orchestra.run.smithi037:mgr.x> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mgr.x.service 2024-06-10T00:44:15.263 INFO:tasks.cephadm:Bootstrapping... 2024-06-10T00:44:15.263 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba -v bootstrap --fsid 68793514-26c2-11ef-bc9b-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.37 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-06-10T00:44:15.468 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-06-10T00:44:15.468 INFO:teuthology.orchestra.run.smithi037.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba', '-v', 'bootstrap', '--fsid', '68793514-26c2-11ef-bc9b-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.37', '--skip-admin-label'] 2024-06-10T00:44:15.469 INFO:teuthology.orchestra.run.smithi037.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-06-10T00:44:15.469 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying podman|docker is present... 2024-06-10T00:44:15.495 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout 5.0.2 2024-06-10T00:44:15.495 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying lvm2 is present... 2024-06-10T00:44:15.495 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying time synchronization is in place... 2024-06-10T00:44:15.503 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-10T00:44:15.503 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-10T00:44:15.510 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-10T00:44:15.510 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-06-10T00:44:15.517 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout enabled 2024-06-10T00:44:15.523 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout active 2024-06-10T00:44:15.523 INFO:teuthology.orchestra.run.smithi037.stdout:Unit chronyd.service is enabled and running 2024-06-10T00:44:15.523 INFO:teuthology.orchestra.run.smithi037.stdout:Repeating the final host check... 2024-06-10T00:44:15.549 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout 5.0.2 2024-06-10T00:44:15.549 INFO:teuthology.orchestra.run.smithi037.stdout:podman (/bin/podman) version 5.0.2 is present 2024-06-10T00:44:15.549 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl is present 2024-06-10T00:44:15.549 INFO:teuthology.orchestra.run.smithi037.stdout:lvcreate is present 2024-06-10T00:44:15.556 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-10T00:44:15.556 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-10T00:44:15.562 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-10T00:44:15.562 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-06-10T00:44:15.569 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout enabled 2024-06-10T00:44:15.575 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout active 2024-06-10T00:44:15.575 INFO:teuthology.orchestra.run.smithi037.stdout:Unit chronyd.service is enabled and running 2024-06-10T00:44:15.575 INFO:teuthology.orchestra.run.smithi037.stdout:Host looks OK 2024-06-10T00:44:15.576 INFO:teuthology.orchestra.run.smithi037.stdout:Cluster fsid: 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:44:15.576 INFO:teuthology.orchestra.run.smithi037.stdout:Acquiring lock 140040234853760 on /run/cephadm/68793514-26c2-11ef-bc9b-c7b262605968.lock 2024-06-10T00:44:15.576 INFO:teuthology.orchestra.run.smithi037.stdout:Lock 140040234853760 acquired on /run/cephadm/68793514-26c2-11ef-bc9b-c7b262605968.lock 2024-06-10T00:44:15.576 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying IP 172.21.15.37 port 3300 ... 2024-06-10T00:44:15.576 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying IP 172.21.15.37 port 6789 ... 2024-06-10T00:44:15.577 INFO:teuthology.orchestra.run.smithi037.stdout:Base mon IP(s) is [172.21.15.37:3300, 172.21.15.37:6789], mon addrv is [v2:172.21.15.37:3300,v1:172.21.15.37:6789] 2024-06-10T00:44:15.580 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.37 metric 100 2024-06-10T00:44:15.580 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.37 metric 100 2024-06-10T00:44:15.582 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-06-10T00:44:15.583 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-06-10T00:44:15.583 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-06-10T00:44:15.585 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-06-10T00:44:15.585 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-06-10T00:44:15.585 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-10T00:44:15.585 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-06-10T00:44:15.585 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:7794/64 scope link noprefixroute 2024-06-10T00:44:15.585 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-10T00:44:15.586 INFO:teuthology.orchestra.run.smithi037.stdout:Mon IP `172.21.15.37` is in CIDR network `172.21.0.0/20` 2024-06-10T00:44:15.586 INFO:teuthology.orchestra.run.smithi037.stdout:Mon IP `172.21.15.37` is in CIDR network `172.21.0.0/20` 2024-06-10T00:44:15.586 INFO:teuthology.orchestra.run.smithi037.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-06-10T00:44:15.587 INFO:teuthology.orchestra.run.smithi037.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-06-10T00:44:15.587 INFO:teuthology.orchestra.run.smithi037.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-10T00:44:16.078 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-10T00:44:16.079 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-10T00:44:16.079 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Getting image source signatures 2024-06-10T00:44:16.079 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying blob sha256:53b2d4191269e9d20972bfa652565296fad6514cf6a3d99f19db644875f67876 2024-06-10T00:44:16.079 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying blob sha256:3ca969c7b6d79946c6f10490fffc00028babe7df67f50ed53c7bc209325e0612 2024-06-10T00:44:16.079 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying config sha256:b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-10T00:44:16.079 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Writing manifest to image destination 2024-06-10T00:44:17.050 INFO:teuthology.orchestra.run.smithi037.stdout:ceph: stdout ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-10T00:44:17.050 INFO:teuthology.orchestra.run.smithi037.stdout:Ceph version: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-10T00:44:17.051 INFO:teuthology.orchestra.run.smithi037.stdout:Extracting ceph user uid/gid from container image... 2024-06-10T00:44:17.998 INFO:teuthology.orchestra.run.smithi037.stdout:stat: stdout 167 167 2024-06-10T00:44:17.998 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial keys... 2024-06-10T00:44:18.862 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQBiTGZm9oFlExAANokKxv8OPvMiwdqoOrc5nw== 2024-06-10T00:44:19.775 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQBjTGZmyM9BDBAAKieTPartAa5NHAf7pyBQng== 2024-06-10T00:44:20.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQBkTGZmr7paBxAAPD1hvCZ0+2MRdP2hsZCDIg== 2024-06-10T00:44:20.765 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial monmap... 2024-06-10T00:44:21.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-10T00:44:21.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-06-10T00:44:21.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:44:21.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-10T00:44:21.662 INFO:teuthology.orchestra.run.smithi037.stdout:monmaptool for a [v2:172.21.15.37:3300,v1:172.21.15.37:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-10T00:44:21.663 INFO:teuthology.orchestra.run.smithi037.stdout:setting min_mon_release = quincy 2024-06-10T00:44:21.663 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: set fsid to 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:44:21.663 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-10T00:44:21.663 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:44:21.663 INFO:teuthology.orchestra.run.smithi037.stdout:Creating mon... 2024-06-10T00:44:22.685 INFO:teuthology.orchestra.run.smithi037.stdout:create mon.a on 2024-06-10T00:44:22.905 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-06-10T00:44:23.047 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-06-10T00:44:23.236 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-68793514-26c2-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-68793514-26c2-11ef-bc9b-c7b262605968.target. 2024-06-10T00:44:23.236 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-68793514-26c2-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-68793514-26c2-11ef-bc9b-c7b262605968.target. 2024-06-10T00:44:23.469 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-68793514-26c2-11ef-bc9b-c7b262605968@mon.a 2024-06-10T00:44:23.469 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to reset failed state of unit ceph-68793514-26c2-11ef-bc9b-c7b262605968@mon.a.service: Unit ceph-68793514-26c2-11ef-bc9b-c7b262605968@mon.a.service not loaded. 2024-06-10T00:44:23.640 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-68793514-26c2-11ef-bc9b-c7b262605968.target.wants/ceph-68793514-26c2-11ef-bc9b-c7b262605968@mon.a.service → /etc/systemd/system/ceph-68793514-26c2-11ef-bc9b-c7b262605968@.service. 2024-06-10T00:44:23.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:23 smithi037 systemd[1]: Starting Ceph mon.a for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T00:44:23.960 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:23 smithi037 podman[31797]: 2024-06-10 00:44:23.875489777 +0000 UTC m=+0.100826437 container create 497c9b53d88aba969f7f1c41ea18cd4ddea0b264c5a5c191ab737762f2912fd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-a, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True) 2024-06-10T00:44:23.960 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:23 smithi037 podman[31797]: 2024-06-10 00:44:23.792588452 +0000 UTC m=+0.017925105 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:44:24.152 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-10T00:44:24.152 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-06-10T00:44:24.160 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-10T00:44:24.160 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-06-10T00:44:24.160 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-06-10T00:44:24.160 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to enable service . firewalld.service is not available 2024-06-10T00:44:24.161 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mon to start... 2024-06-10T00:44:24.161 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mon... 2024-06-10T00:44:24.278 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:24 smithi037 podman[31797]: 2024-06-10 00:44:24.061258948 +0000 UTC m=+0.286595619 container init 497c9b53d88aba969f7f1c41ea18cd4ddea0b264c5a5c191ab737762f2912fd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc) 2024-06-10T00:44:24.278 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:24 smithi037 podman[31797]: 2024-06-10 00:44:24.065122471 +0000 UTC m=+0.290459133 container start 497c9b53d88aba969f7f1c41ea18cd4ddea0b264c5a5c191ab737762f2912fd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-06-10T00:44:24.279 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:24 smithi037 ceph-mon[31818]: mkfs 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:44:24.279 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:24 smithi037 ceph-mon[31818]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-10T00:44:24.279 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:24 smithi037 bash[31797]: 497c9b53d88aba969f7f1c41ea18cd4ddea0b264c5a5c191ab737762f2912fd7 2024-06-10T00:44:24.279 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:24 smithi037 systemd[1]: Started Ceph mon.a for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T00:44:25.184 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:25 smithi037 ceph-mon[31818]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-10T00:44:25.184 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:25 smithi037 ceph-mon[31818]: monmap epoch 1 2024-06-10T00:44:25.184 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:25 smithi037 ceph-mon[31818]: fsid 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:44:25.184 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:25 smithi037 ceph-mon[31818]: last_changed 2024-06-10T00:44:21.142496+0000 2024-06-10T00:44:25.185 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:25 smithi037 ceph-mon[31818]: created 2024-06-10T00:44:21.142496+0000 2024-06-10T00:44:25.185 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:25 smithi037 ceph-mon[31818]: min_mon_release 19 (squid) 2024-06-10T00:44:25.185 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:25 smithi037 ceph-mon[31818]: election_strategy: 1 2024-06-10T00:44:25.185 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:25 smithi037 ceph-mon[31818]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-06-10T00:44:25.185 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:25 smithi037 ceph-mon[31818]: fsmap 2024-06-10T00:44:25.185 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:25 smithi037 ceph-mon[31818]: osdmap e1: 0 total, 0 up, 0 in 2024-06-10T00:44:25.185 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:25 smithi037 ceph-mon[31818]: mgrmap e1: no daemons active 2024-06-10T00:44:25.185 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:25 smithi037 ceph-mon[31818]: from='client.? 172.21.15.37:0/136455089' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-06-10T00:44:25.186 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout cluster: 2024-06-10T00:44:25.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout id: 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:44:25.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-06-10T00:44:25.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-06-10T00:44:25.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout services: 2024-06-10T00:44:25.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.566324s) 2024-06-10T00:44:25.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-06-10T00:44:25.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-06-10T00:44:25.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-06-10T00:44:25.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout data: 2024-06-10T00:44:25.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-06-10T00:44:25.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-06-10T00:44:25.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-06-10T00:44:25.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout pgs: 2024-06-10T00:44:25.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-06-10T00:44:25.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.591+0000 7f5a0350b700 1 Processor -- start 2024-06-10T00:44:25.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.591+0000 7f5a0350b700 1 -- start start 2024-06-10T00:44:25.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.591+0000 7f5a0350b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59fc106830 0x7f59fc106c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:25.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.591+0000 7f5a0350b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc07c6c0 con 0x7f59fc106830 2024-06-10T00:44:25.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.592+0000 7f5a012a7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59fc106830 0x7f59fc106c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:25.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.592+0000 7f5a012a7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59fc106830 0x7f59fc106c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49272/0 (socket says 172.21.15.37:49272) 2024-06-10T00:44:25.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.592+0000 7f5a012a7700 1 -- 172.21.15.37:0/1404153420 learned_addr learned my addr 172.21.15.37:0/1404153420 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:44:25.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.592+0000 7f5a012a7700 1 -- 172.21.15.37:0/1404153420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59fc107190 con 0x7f59fc106830 2024-06-10T00:44:25.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.592+0000 7f5a012a7700 1 --2- 172.21.15.37:0/1404153420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59fc106830 0x7f59fc106c50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f59f80098a0 tx=0x7f59f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9602b3ce3ecd0a9c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:25.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.592+0000 7f59f3fff700 1 -- 172.21.15.37:0/1404153420 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59f8004160 con 0x7f59fc106830 2024-06-10T00:44:25.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.592+0000 7f59f3fff700 1 -- 172.21.15.37:0/1404153420 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59f8004300 con 0x7f59fc106830 2024-06-10T00:44:25.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.592+0000 7f59f3fff700 1 -- 172.21.15.37:0/1404153420 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59f80036a0 con 0x7f59fc106830 2024-06-10T00:44:25.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.593+0000 7f5a0350b700 1 -- 172.21.15.37:0/1404153420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59fc106830 msgr2=0x7f59fc106c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:25.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.593+0000 7f5a0350b700 1 --2- 172.21.15.37:0/1404153420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59fc106830 0x7f59fc106c50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f59f80098a0 tx=0x7f59f8009bb0 comp rx=0 tx=0).stop 2024-06-10T00:44:25.195 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.593+0000 7f5a0350b700 1 -- 172.21.15.37:0/1404153420 shutdown_connections 2024-06-10T00:44:25.196 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.593+0000 7f5a0350b700 1 --2- 172.21.15.37:0/1404153420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59fc106830 0x7f59fc106c50 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:25.196 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.593+0000 7f5a0350b700 1 -- 172.21.15.37:0/1404153420 >> 172.21.15.37:0/1404153420 conn(0x7f59fc1023d0 msgr2=0x7f59fc104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:25.196 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.593+0000 7f5a0350b700 1 -- 172.21.15.37:0/1404153420 shutdown_connections 2024-06-10T00:44:25.196 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.593+0000 7f5a0350b700 1 -- 172.21.15.37:0/1404153420 wait complete. 2024-06-10T00:44:25.196 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.594+0000 7f5a0350b700 1 Processor -- start 2024-06-10T00:44:25.196 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.594+0000 7f5a0350b700 1 -- start start 2024-06-10T00:44:25.196 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.594+0000 7f5a0350b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59fc106830 0x7f59fc19d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:25.196 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.594+0000 7f5a0350b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc1076c0 con 0x7f59fc106830 2024-06-10T00:44:25.196 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.594+0000 7f5a012a7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59fc106830 0x7f59fc19d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:25.196 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.594+0000 7f5a012a7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59fc106830 0x7f59fc19d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49284/0 (socket says 172.21.15.37:49284) 2024-06-10T00:44:25.196 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.594+0000 7f5a012a7700 1 -- 172.21.15.37:0/136455089 learned_addr learned my addr 172.21.15.37:0/136455089 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:44:25.196 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.594+0000 7f5a012a7700 1 -- 172.21.15.37:0/136455089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59fc19dd50 con 0x7f59fc106830 2024-06-10T00:44:25.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.594+0000 7f5a012a7700 1 --2- 172.21.15.37:0/136455089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59fc106830 0x7f59fc19d810 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f59f8000c00 tx=0x7f59f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:25.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.594+0000 7f59f27fc700 1 -- 172.21.15.37:0/136455089 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59f8003db0 con 0x7f59fc106830 2024-06-10T00:44:25.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.595+0000 7f5a0350b700 1 -- 172.21.15.37:0/136455089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59fc19dfe0 con 0x7f59fc106830 2024-06-10T00:44:25.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.595+0000 7f59f27fc700 1 -- 172.21.15.37:0/136455089 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59f8011630 con 0x7f59fc106830 2024-06-10T00:44:25.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.595+0000 7f59f27fc700 1 -- 172.21.15.37:0/136455089 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59f8011950 con 0x7f59fc106830 2024-06-10T00:44:25.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.595+0000 7f5a0350b700 1 -- 172.21.15.37:0/136455089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59fc1a0cd0 con 0x7f59fc106830 2024-06-10T00:44:25.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.595+0000 7f59f27fc700 1 -- 172.21.15.37:0/136455089 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f59f8011b80 con 0x7f59fc106830 2024-06-10T00:44:25.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.595+0000 7f59f27fc700 1 -- 172.21.15.37:0/136455089 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f59f801c450 con 0x7f59fc106830 2024-06-10T00:44:25.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.596+0000 7f5a0350b700 1 -- 172.21.15.37:0/136455089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59fc063af0 con 0x7f59fc106830 2024-06-10T00:44:25.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.598+0000 7f59f27fc700 1 -- 172.21.15.37:0/136455089 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f59f8032020 con 0x7f59fc106830 2024-06-10T00:44:25.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.671+0000 7f5a0350b700 1 -- 172.21.15.37:0/136455089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f59fc1a0ea0 con 0x7f59fc106830 2024-06-10T00:44:25.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.671+0000 7f59f27fc700 1 -- 172.21.15.37:0/136455089 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f59f8004260 con 0x7f59fc106830 2024-06-10T00:44:25.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.672+0000 7f5a0350b700 1 -- 172.21.15.37:0/136455089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59fc106830 msgr2=0x7f59fc19d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:25.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.672+0000 7f5a0350b700 1 --2- 172.21.15.37:0/136455089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59fc106830 0x7f59fc19d810 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f59f8000c00 tx=0x7f59f8004750 comp rx=0 tx=0).stop 2024-06-10T00:44:25.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.672+0000 7f5a0350b700 1 -- 172.21.15.37:0/136455089 shutdown_connections 2024-06-10T00:44:25.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.672+0000 7f5a0350b700 1 --2- 172.21.15.37:0/136455089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59fc106830 0x7f59fc19d810 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:25.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.672+0000 7f5a0350b700 1 -- 172.21.15.37:0/136455089 >> 172.21.15.37:0/136455089 conn(0x7f59fc1023d0 msgr2=0x7f59fc193e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:25.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.672+0000 7f5a0350b700 1 -- 172.21.15.37:0/136455089 shutdown_connections 2024-06-10T00:44:25.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:24.672+0000 7f5a0350b700 1 -- 172.21.15.37:0/136455089 wait complete. 2024-06-10T00:44:25.198 INFO:teuthology.orchestra.run.smithi037.stdout:mon is available 2024-06-10T00:44:25.198 INFO:teuthology.orchestra.run.smithi037.stdout:Assimilating anything we can from ceph.conf... 2024-06-10T00:44:26.166 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-06-10T00:44:26.166 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [global] 2024-06-10T00:44:26.166 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout fsid = 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:44:26.166 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.37:3300,v1:172.21.15.37:6789] 2024-06-10T00:44:26.166 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-10T00:44:26.166 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-10T00:44:26.166 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-10T00:44:26.166 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-10T00:44:26.166 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-06-10T00:44:26.166 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-10T00:44:26.167 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-10T00:44:26.167 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-06-10T00:44:26.167 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [osd] 2024-06-10T00:44:26.167 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-10T00:44:26.167 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-10T00:44:26.167 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-10T00:44:26.167 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-10T00:44:26.167 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-10T00:44:26.167 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.622+0000 7fbd4420e700 1 Processor -- start 2024-06-10T00:44:26.167 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.622+0000 7fbd4420e700 1 -- start start 2024-06-10T00:44:26.167 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.622+0000 7fbd4420e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd3c07e6e0 0x7fbd3c07cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:26.167 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.622+0000 7fbd4420e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c07d160 con 0x7fbd3c07e6e0 2024-06-10T00:44:26.167 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.623+0000 7fbd41faa700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd3c07e6e0 0x7fbd3c07cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:26.168 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.623+0000 7fbd41faa700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd3c07e6e0 0x7fbd3c07cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49292/0 (socket says 172.21.15.37:49292) 2024-06-10T00:44:26.168 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.623+0000 7fbd41faa700 1 -- 172.21.15.37:0/2832791704 learned_addr learned my addr 172.21.15.37:0/2832791704 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:44:26.168 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.623+0000 7fbd41faa700 1 -- 172.21.15.37:0/2832791704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3c07d2e0 con 0x7fbd3c07e6e0 2024-06-10T00:44:26.168 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.623+0000 7fbd41faa700 1 --2- 172.21.15.37:0/2832791704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd3c07e6e0 0x7fbd3c07cb90 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c009e80 tx=0x7fbd2c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d24be4bfa0dbc1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:26.168 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.623+0000 7fbd40fa8700 1 -- 172.21.15.37:0/2832791704 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd2c017070 con 0x7fbd3c07e6e0 2024-06-10T00:44:26.168 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.623+0000 7fbd40fa8700 1 -- 172.21.15.37:0/2832791704 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd2c003a20 con 0x7fbd3c07e6e0 2024-06-10T00:44:26.168 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.623+0000 7fbd40fa8700 1 -- 172.21.15.37:0/2832791704 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd2c003d20 con 0x7fbd3c07e6e0 2024-06-10T00:44:26.168 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.624+0000 7fbd4420e700 1 -- 172.21.15.37:0/2832791704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd3c07e6e0 msgr2=0x7fbd3c07cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:26.168 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.624+0000 7fbd4420e700 1 --2- 172.21.15.37:0/2832791704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd3c07e6e0 0x7fbd3c07cb90 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c009e80 tx=0x7fbd2c00b330 comp rx=0 tx=0).stop 2024-06-10T00:44:26.168 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.624+0000 7fbd4420e700 1 -- 172.21.15.37:0/2832791704 shutdown_connections 2024-06-10T00:44:26.168 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.624+0000 7fbd4420e700 1 --2- 172.21.15.37:0/2832791704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd3c07e6e0 0x7fbd3c07cb90 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:26.168 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.624+0000 7fbd4420e700 1 -- 172.21.15.37:0/2832791704 >> 172.21.15.37:0/2832791704 conn(0x7fbd3c1035a0 msgr2=0x7fbd3c1059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:26.168 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.624+0000 7fbd4420e700 1 -- 172.21.15.37:0/2832791704 shutdown_connections 2024-06-10T00:44:26.169 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.624+0000 7fbd4420e700 1 -- 172.21.15.37:0/2832791704 wait complete. 2024-06-10T00:44:26.169 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.625+0000 7fbd4420e700 1 Processor -- start 2024-06-10T00:44:26.169 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.625+0000 7fbd4420e700 1 -- start start 2024-06-10T00:44:26.169 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.625+0000 7fbd4420e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd3c07e6e0 0x7fbd3c19d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:26.169 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.625+0000 7fbd4420e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c07d810 con 0x7fbd3c07e6e0 2024-06-10T00:44:26.169 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.625+0000 7fbd41faa700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd3c07e6e0 0x7fbd3c19d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:26.169 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.625+0000 7fbd41faa700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd3c07e6e0 0x7fbd3c19d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49300/0 (socket says 172.21.15.37:49300) 2024-06-10T00:44:26.169 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.625+0000 7fbd41faa700 1 -- 172.21.15.37:0/4033526987 learned_addr learned my addr 172.21.15.37:0/4033526987 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:44:26.169 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.625+0000 7fbd41faa700 1 -- 172.21.15.37:0/4033526987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3c19ddd0 con 0x7fbd3c07e6e0 2024-06-10T00:44:26.169 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.625+0000 7fbd41faa700 1 --2- 172.21.15.37:0/4033526987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd3c07e6e0 0x7fbd3c19d890 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c07dc00 tx=0x7fbd2c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:26.169 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.625+0000 7fbd3affd700 1 -- 172.21.15.37:0/4033526987 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd2c022020 con 0x7fbd3c07e6e0 2024-06-10T00:44:26.170 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.626+0000 7fbd3affd700 1 -- 172.21.15.37:0/4033526987 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd2c018040 con 0x7fbd3c07e6e0 2024-06-10T00:44:26.170 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.626+0000 7fbd4420e700 1 -- 172.21.15.37:0/4033526987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd3c19e060 con 0x7fbd3c07e6e0 2024-06-10T00:44:26.170 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.626+0000 7fbd3affd700 1 -- 172.21.15.37:0/4033526987 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd2c017070 con 0x7fbd3c07e6e0 2024-06-10T00:44:26.170 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.626+0000 7fbd4420e700 1 -- 172.21.15.37:0/4033526987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd3c1a0d50 con 0x7fbd3c07e6e0 2024-06-10T00:44:26.170 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.626+0000 7fbd3affd700 1 -- 172.21.15.37:0/4033526987 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbd2c025020 con 0x7fbd3c07e6e0 2024-06-10T00:44:26.170 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.626+0000 7fbd3affd700 1 -- 172.21.15.37:0/4033526987 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbd2c00f050 con 0x7fbd3c07e6e0 2024-06-10T00:44:26.170 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.627+0000 7fbd4420e700 1 -- 172.21.15.37:0/4033526987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd3c063af0 con 0x7fbd3c07e6e0 2024-06-10T00:44:26.170 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.629+0000 7fbd3affd700 1 -- 172.21.15.37:0/4033526987 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fbd2c02c030 con 0x7fbd3c07e6e0 2024-06-10T00:44:26.170 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.700+0000 7fbd4420e700 1 -- 172.21.15.37:0/4033526987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fbd3c02dda0 con 0x7fbd3c07e6e0 2024-06-10T00:44:26.170 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.703+0000 7fbd3affd700 1 -- 172.21.15.37:0/4033526987 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7fbd2c01e480 con 0x7fbd3c07e6e0 2024-06-10T00:44:26.170 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.704+0000 7fbd3affd700 1 -- 172.21.15.37:0/4033526987 <== mon.0 v2:172.21.15.37:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbd2c003a20 con 0x7fbd3c07e6e0 2024-06-10T00:44:26.171 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.704+0000 7fbd4420e700 1 -- 172.21.15.37:0/4033526987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd3c07e6e0 msgr2=0x7fbd3c19d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:26.171 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.704+0000 7fbd4420e700 1 --2- 172.21.15.37:0/4033526987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd3c07e6e0 0x7fbd3c19d890 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c07dc00 tx=0x7fbd2c004ad0 comp rx=0 tx=0).stop 2024-06-10T00:44:26.171 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.704+0000 7fbd4420e700 1 -- 172.21.15.37:0/4033526987 shutdown_connections 2024-06-10T00:44:26.171 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.704+0000 7fbd4420e700 1 --2- 172.21.15.37:0/4033526987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd3c07e6e0 0x7fbd3c19d890 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:26.171 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.704+0000 7fbd4420e700 1 -- 172.21.15.37:0/4033526987 >> 172.21.15.37:0/4033526987 conn(0x7fbd3c1035a0 msgr2=0x7fbd3c105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:26.171 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.704+0000 7fbd4420e700 1 -- 172.21.15.37:0/4033526987 shutdown_connections 2024-06-10T00:44:26.171 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:25.704+0000 7fbd4420e700 1 -- 172.21.15.37:0/4033526987 wait complete. 2024-06-10T00:44:26.171 INFO:teuthology.orchestra.run.smithi037.stdout:Generating new minimal ceph.conf... 2024-06-10T00:44:26.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:26 smithi037 ceph-mon[31818]: from='client.? 172.21.15.37:0/4033526987' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-10T00:44:26.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:26 smithi037 ceph-mon[31818]: from='client.? 172.21.15.37:0/4033526987' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-06-10T00:44:27.162 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.571+0000 7fbabb176700 1 Processor -- start 2024-06-10T00:44:27.162 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.571+0000 7fbabb176700 1 -- start start 2024-06-10T00:44:27.162 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.571+0000 7fbabb176700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab4108a20 0x7fbab410ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:27.162 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.571+0000 7fbabb176700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbab407c6c0 con 0x7fbab4108a20 2024-06-10T00:44:27.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.571+0000 7fbab8f12700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab4108a20 0x7fbab410ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:27.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.571+0000 7fbab8f12700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab4108a20 0x7fbab410ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49302/0 (socket says 172.21.15.37:49302) 2024-06-10T00:44:27.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.571+0000 7fbab8f12700 1 -- 172.21.15.37:0/2739188378 learned_addr learned my addr 172.21.15.37:0/2739188378 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:44:27.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.572+0000 7fbab8f12700 1 -- 172.21.15.37:0/2739188378 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab410b350 con 0x7fbab4108a20 2024-06-10T00:44:27.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.572+0000 7fbab8f12700 1 --2- 172.21.15.37:0/2739188378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab4108a20 0x7fbab410ae10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbaa80098a0 tx=0x7fbaa8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15f58c87001bb1f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:27.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.572+0000 7fbab37fe700 1 -- 172.21.15.37:0/2739188378 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbaa8004160 con 0x7fbab4108a20 2024-06-10T00:44:27.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.572+0000 7fbab37fe700 1 -- 172.21.15.37:0/2739188378 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbaa8004300 con 0x7fbab4108a20 2024-06-10T00:44:27.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.572+0000 7fbab37fe700 1 -- 172.21.15.37:0/2739188378 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbaa8011b50 con 0x7fbab4108a20 2024-06-10T00:44:27.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.572+0000 7fbabb176700 1 -- 172.21.15.37:0/2739188378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab4108a20 msgr2=0x7fbab410ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:27.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.572+0000 7fbabb176700 1 --2- 172.21.15.37:0/2739188378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab4108a20 0x7fbab410ae10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbaa80098a0 tx=0x7fbaa8009bb0 comp rx=0 tx=0).stop 2024-06-10T00:44:27.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.573+0000 7fbabb176700 1 -- 172.21.15.37:0/2739188378 shutdown_connections 2024-06-10T00:44:27.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.573+0000 7fbabb176700 1 --2- 172.21.15.37:0/2739188378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab4108a20 0x7fbab410ae10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:27.163 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.573+0000 7fbabb176700 1 -- 172.21.15.37:0/2739188378 >> 172.21.15.37:0/2739188378 conn(0x7fbab4102410 msgr2=0x7fbab4104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:27.164 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.573+0000 7fbabb176700 1 -- 172.21.15.37:0/2739188378 shutdown_connections 2024-06-10T00:44:27.164 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.573+0000 7fbabb176700 1 -- 172.21.15.37:0/2739188378 wait complete. 2024-06-10T00:44:27.164 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.573+0000 7fbabb176700 1 Processor -- start 2024-06-10T00:44:27.164 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.573+0000 7fbabb176700 1 -- start start 2024-06-10T00:44:27.164 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.573+0000 7fbabb176700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab4108a20 0x7fbab419bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:27.164 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.573+0000 7fbabb176700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbab410c310 con 0x7fbab4108a20 2024-06-10T00:44:27.164 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.574+0000 7fbab8f12700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab4108a20 0x7fbab419bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:27.164 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.574+0000 7fbab8f12700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab4108a20 0x7fbab419bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49306/0 (socket says 172.21.15.37:49306) 2024-06-10T00:44:27.164 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.574+0000 7fbab8f12700 1 -- 172.21.15.37:0/1947922640 learned_addr learned my addr 172.21.15.37:0/1947922640 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:44:27.164 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.574+0000 7fbab8f12700 1 -- 172.21.15.37:0/1947922640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab419c400 con 0x7fbab4108a20 2024-06-10T00:44:27.164 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.574+0000 7fbab8f12700 1 --2- 172.21.15.37:0/1947922640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab4108a20 0x7fbab419bec0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8011b60 tx=0x7fbaa8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:27.165 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.574+0000 7fbab1ffb700 1 -- 172.21.15.37:0/1947922640 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbaa8003b20 con 0x7fbab4108a20 2024-06-10T00:44:27.165 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.574+0000 7fbab1ffb700 1 -- 172.21.15.37:0/1947922640 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbaa8003cc0 con 0x7fbab4108a20 2024-06-10T00:44:27.165 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.574+0000 7fbabb176700 1 -- 172.21.15.37:0/1947922640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab419c690 con 0x7fbab4108a20 2024-06-10T00:44:27.165 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.574+0000 7fbab1ffb700 1 -- 172.21.15.37:0/1947922640 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbaa8024840 con 0x7fbab4108a20 2024-06-10T00:44:27.165 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.574+0000 7fbabb176700 1 -- 172.21.15.37:0/1947922640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab419f380 con 0x7fbab4108a20 2024-06-10T00:44:27.165 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.575+0000 7fbab1ffb700 1 -- 172.21.15.37:0/1947922640 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbaa8022020 con 0x7fbab4108a20 2024-06-10T00:44:27.165 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.575+0000 7fbab1ffb700 1 -- 172.21.15.37:0/1947922640 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbaa8011e00 con 0x7fbab4108a20 2024-06-10T00:44:27.165 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.575+0000 7fbabb176700 1 -- 172.21.15.37:0/1947922640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbab4063af0 con 0x7fbab4108a20 2024-06-10T00:44:27.165 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.578+0000 7fbab1ffb700 1 -- 172.21.15.37:0/1947922640 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fbaa8043dd0 con 0x7fbab4108a20 2024-06-10T00:44:27.165 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.648+0000 7fbabb176700 1 -- 172.21.15.37:0/1947922640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fbab419f8f0 con 0x7fbab4108a20 2024-06-10T00:44:27.165 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.648+0000 7fbab1ffb700 1 -- 172.21.15.37:0/1947922640 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fbaa8032030 con 0x7fbab4108a20 2024-06-10T00:44:27.165 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.649+0000 7fbabb176700 1 -- 172.21.15.37:0/1947922640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab4108a20 msgr2=0x7fbab419bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:27.165 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.649+0000 7fbabb176700 1 --2- 172.21.15.37:0/1947922640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab4108a20 0x7fbab419bec0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8011b60 tx=0x7fbaa8011c40 comp rx=0 tx=0).stop 2024-06-10T00:44:27.166 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.649+0000 7fbabb176700 1 -- 172.21.15.37:0/1947922640 shutdown_connections 2024-06-10T00:44:27.166 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.649+0000 7fbabb176700 1 --2- 172.21.15.37:0/1947922640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab4108a20 0x7fbab419bec0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:27.166 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.649+0000 7fbabb176700 1 -- 172.21.15.37:0/1947922640 >> 172.21.15.37:0/1947922640 conn(0x7fbab4102410 msgr2=0x7fbab4102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:27.166 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.649+0000 7fbabb176700 1 -- 172.21.15.37:0/1947922640 shutdown_connections 2024-06-10T00:44:27.166 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:26.649+0000 7fbabb176700 1 -- 172.21.15.37:0/1947922640 wait complete. 2024-06-10T00:44:27.166 INFO:teuthology.orchestra.run.smithi037.stdout:Restarting the monitor... 2024-06-10T00:44:27.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:27 smithi037 ceph-mon[31818]: from='client.? 172.21.15.37:0/1947922640' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:44:27.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:27 smithi037 systemd[1]: Stopping Ceph mon.a for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T00:44:27.843 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:27 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-a[31814]: 2024-06-10T00:44:27.417+0000 7f796caf5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T00:44:27.843 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:27 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-a[31814]: 2024-06-10T00:44:27.417+0000 7f796caf5700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-06-10T00:44:27.843 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:27 smithi037 podman[32067]: 2024-06-10 00:44:27.632763428 +0000 UTC m=+0.294230873 container died 497c9b53d88aba969f7f1c41ea18cd4ddea0b264c5a5c191ab737762f2912fd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-a, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T00:44:28.134 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:27 smithi037 podman[32067]: 2024-06-10 00:44:27.925199696 +0000 UTC m=+0.586667134 container cleanup 497c9b53d88aba969f7f1c41ea18cd4ddea0b264c5a5c191ab737762f2912fd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-a, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-06-10T00:44:28.134 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:27 smithi037 bash[32067]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-a 2024-06-10T00:44:28.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 podman[32080]: 2024-06-10 00:44:28.134314585 +0000 UTC m=+0.500803451 container remove 497c9b53d88aba969f7f1c41ea18cd4ddea0b264c5a5c191ab737762f2912fd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3d89acc) 2024-06-10T00:44:28.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 systemd[1]: ceph-68793514-26c2-11ef-bc9b-c7b262605968@mon.a.service: Deactivated successfully. 2024-06-10T00:44:28.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 systemd[1]: Stopped Ceph mon.a for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T00:44:28.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 systemd[1]: Starting Ceph mon.a for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T00:44:28.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 podman[32173]: 2024-06-10 00:44:28.38408578 +0000 UTC m=+0.017993475 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:44:28.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 podman[32173]: 2024-06-10 00:44:28.493682906 +0000 UTC m=+0.127590608 container create b72c961cf1e0d476a1b01095215bfac013f9985e0afb1213b4b8989ac35eb135 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-a, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-10T00:44:28.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 podman[32173]: 2024-06-10 00:44:28.70414529 +0000 UTC m=+0.338052971 container init b72c961cf1e0d476a1b01095215bfac013f9985e0afb1213b4b8989ac35eb135 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0) 2024-06-10T00:44:28.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 podman[32173]: 2024-06-10 00:44:28.70781718 +0000 UTC m=+0.341724861 container start b72c961cf1e0d476a1b01095215bfac013f9985e0afb1213b4b8989ac35eb135 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-a, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T00:44:28.792 INFO:teuthology.orchestra.run.smithi037.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-06-10T00:44:29.037 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: set uid:gid to 167:167 (ceph:ceph) 2024-06-10T00:44:29.037 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-10T00:44:29.037 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: pidfile_write: ignore empty --pid-file 2024-06-10T00:44:29.037 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: load: jerasure load: lrc 2024-06-10T00:44:29.037 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: RocksDB version: 7.9.2 2024-06-10T00:44:29.037 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Git sha 0 2024-06-10T00:44:29.037 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-10T00:44:29.037 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: DB SUMMARY 2024-06-10T00:44:29.037 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: DB Session ID: PU11GAZ81P5VFO21D9D8 2024-06-10T00:44:29.038 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: CURRENT file: CURRENT 2024-06-10T00:44:29.038 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: IDENTITY file: IDENTITY 2024-06-10T00:44:29.038 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-06-10T00:44:29.038 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-06-10T00:44:29.038 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104890 ; 2024-06-10T00:44:29.038 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.error_if_exists: 0 2024-06-10T00:44:29.038 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.create_if_missing: 0 2024-06-10T00:44:29.038 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.paranoid_checks: 1 2024-06-10T00:44:29.038 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-10T00:44:29.038 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-10T00:44:29.039 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-10T00:44:29.039 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.env: 0x560156a0ac60 2024-06-10T00:44:29.039 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.fs: PosixFileSystem 2024-06-10T00:44:29.039 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.info_log: 0x5601590cdb20 2024-06-10T00:44:29.039 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.max_file_opening_threads: 16 2024-06-10T00:44:29.039 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.statistics: (nil) 2024-06-10T00:44:29.039 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.use_fsync: 0 2024-06-10T00:44:29.039 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.max_log_file_size: 0 2024-06-10T00:44:29.039 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-10T00:44:29.039 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-10T00:44:29.039 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.keep_log_file_num: 1000 2024-06-10T00:44:29.039 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.recycle_log_file_num: 0 2024-06-10T00:44:29.040 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.allow_fallocate: 1 2024-06-10T00:44:29.040 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.allow_mmap_reads: 0 2024-06-10T00:44:29.040 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.allow_mmap_writes: 0 2024-06-10T00:44:29.040 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.use_direct_reads: 0 2024-06-10T00:44:29.040 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-10T00:44:29.040 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.create_missing_column_families: 0 2024-06-10T00:44:29.040 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.db_log_dir: 2024-06-10T00:44:29.040 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.wal_dir: 2024-06-10T00:44:29.040 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-10T00:44:29.041 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-10T00:44:29.042 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-10T00:44:29.042 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-10T00:44:29.042 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-10T00:44:29.042 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-10T00:44:29.042 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.advise_random_on_open: 1 2024-06-10T00:44:29.042 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.db_write_buffer_size: 0 2024-06-10T00:44:29.042 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.write_buffer_manager: 0x56015909e1e0 2024-06-10T00:44:29.042 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-10T00:44:29.042 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-10T00:44:29.042 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-10T00:44:29.042 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.rate_limiter: (nil) 2024-06-10T00:44:29.042 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-10T00:44:29.043 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.wal_recovery_mode: 2 2024-06-10T00:44:29.043 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.enable_thread_tracking: 0 2024-06-10T00:44:29.043 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.enable_pipelined_write: 0 2024-06-10T00:44:29.043 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.unordered_write: 0 2024-06-10T00:44:29.043 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-10T00:44:29.043 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-10T00:44:29.043 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-10T00:44:29.043 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-10T00:44:29.043 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.row_cache: None 2024-06-10T00:44:29.043 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.wal_filter: None 2024-06-10T00:44:29.043 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-10T00:44:29.043 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.allow_ingest_behind: 0 2024-06-10T00:44:29.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.two_write_queues: 0 2024-06-10T00:44:29.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.manual_wal_flush: 0 2024-06-10T00:44:29.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.wal_compression: 0 2024-06-10T00:44:29.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.atomic_flush: 0 2024-06-10T00:44:29.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-10T00:44:29.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-10T00:44:29.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-10T00:44:29.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.log_readahead_size: 0 2024-06-10T00:44:29.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-10T00:44:29.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.best_efforts_recovery: 0 2024-06-10T00:44:29.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-10T00:44:29.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-10T00:44:29.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.allow_data_in_errors: 0 2024-06-10T00:44:29.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.db_host_id: __hostname__ 2024-06-10T00:44:29.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-10T00:44:29.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.max_background_jobs: 2 2024-06-10T00:44:29.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.max_background_compactions: -1 2024-06-10T00:44:29.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.max_subcompactions: 1 2024-06-10T00:44:29.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-10T00:44:29.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-10T00:44:29.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-10T00:44:29.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.max_total_wal_size: 0 2024-06-10T00:44:29.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-10T00:44:29.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-10T00:44:29.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-10T00:44:29.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-10T00:44:29.046 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.max_open_files: -1 2024-06-10T00:44:29.046 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.bytes_per_sync: 0 2024-06-10T00:44:29.046 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-10T00:44:29.046 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-10T00:44:29.046 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.compaction_readahead_size: 0 2024-06-10T00:44:29.046 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.max_background_flushes: -1 2024-06-10T00:44:29.046 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Compression algorithms supported: 2024-06-10T00:44:29.046 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-10T00:44:29.046 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: kZSTD supported: 0 2024-06-10T00:44:29.046 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: kXpressCompression supported: 0 2024-06-10T00:44:29.046 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: kLZ4HCCompression supported: 1 2024-06-10T00:44:29.046 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: kZlibCompression supported: 1 2024-06-10T00:44:29.046 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: kSnappyCompression supported: 1 2024-06-10T00:44:29.047 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: kLZ4Compression supported: 1 2024-06-10T00:44:29.047 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: kBZip2Compression supported: 0 2024-06-10T00:44:29.047 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-10T00:44:29.047 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-10T00:44:29.047 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-06-10T00:44:29.047 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-10T00:44:29.047 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-10T00:44:29.047 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.merge_operator: 2024-06-10T00:44:29.047 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.compaction_filter: None 2024-06-10T00:44:29.047 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.compaction_filter_factory: None 2024-06-10T00:44:29.047 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.sst_partitioner_factory: None 2024-06-10T00:44:29.047 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-10T00:44:29.047 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-10T00:44:29.047 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5601590cdc40) 2024-06-10T00:44:29.048 INFO:journalctl@ceph.mon.a.smithi037.stdout: cache_index_and_filter_blocks: 1 2024-06-10T00:44:29.048 INFO:journalctl@ceph.mon.a.smithi037.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-10T00:44:29.048 INFO:journalctl@ceph.mon.a.smithi037.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-10T00:44:29.048 INFO:journalctl@ceph.mon.a.smithi037.stdout: pin_top_level_index_and_filter: 1 2024-06-10T00:44:29.048 INFO:journalctl@ceph.mon.a.smithi037.stdout: index_type: 0 2024-06-10T00:44:29.048 INFO:journalctl@ceph.mon.a.smithi037.stdout: data_block_index_type: 0 2024-06-10T00:44:29.048 INFO:journalctl@ceph.mon.a.smithi037.stdout: index_shortening: 1 2024-06-10T00:44:29.048 INFO:journalctl@ceph.mon.a.smithi037.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-10T00:44:29.048 INFO:journalctl@ceph.mon.a.smithi037.stdout: checksum: 4 2024-06-10T00:44:29.048 INFO:journalctl@ceph.mon.a.smithi037.stdout: no_block_cache: 0 2024-06-10T00:44:29.048 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache: 0x5601590cf090 2024-06-10T00:44:29.048 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache_name: BinnedLRUCache 2024-06-10T00:44:29.048 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache_options: 2024-06-10T00:44:29.049 INFO:journalctl@ceph.mon.a.smithi037.stdout: capacity : 536870912 2024-06-10T00:44:29.049 INFO:journalctl@ceph.mon.a.smithi037.stdout: num_shard_bits : 4 2024-06-10T00:44:29.049 INFO:journalctl@ceph.mon.a.smithi037.stdout: strict_capacity_limit : 0 2024-06-10T00:44:29.049 INFO:journalctl@ceph.mon.a.smithi037.stdout: high_pri_pool_ratio: 0.000 2024-06-10T00:44:29.049 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache_compressed: (nil) 2024-06-10T00:44:29.049 INFO:journalctl@ceph.mon.a.smithi037.stdout: persistent_cache: (nil) 2024-06-10T00:44:29.049 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_size: 4096 2024-06-10T00:44:29.049 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_size_deviation: 10 2024-06-10T00:44:29.049 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_restart_interval: 16 2024-06-10T00:44:29.049 INFO:journalctl@ceph.mon.a.smithi037.stdout: index_block_restart_interval: 1 2024-06-10T00:44:29.049 INFO:journalctl@ceph.mon.a.smithi037.stdout: metadata_block_size: 4096 2024-06-10T00:44:29.049 INFO:journalctl@ceph.mon.a.smithi037.stdout: partition_filters: 0 2024-06-10T00:44:29.049 INFO:journalctl@ceph.mon.a.smithi037.stdout: use_delta_encoding: 1 2024-06-10T00:44:29.049 INFO:journalctl@ceph.mon.a.smithi037.stdout: filter_policy: bloomfilter 2024-06-10T00:44:29.049 INFO:journalctl@ceph.mon.a.smithi037.stdout: whole_key_filtering: 1 2024-06-10T00:44:29.050 INFO:journalctl@ceph.mon.a.smithi037.stdout: verify_compression: 0 2024-06-10T00:44:29.050 INFO:journalctl@ceph.mon.a.smithi037.stdout: read_amp_bytes_per_bit: 0 2024-06-10T00:44:29.050 INFO:journalctl@ceph.mon.a.smithi037.stdout: format_version: 5 2024-06-10T00:44:29.050 INFO:journalctl@ceph.mon.a.smithi037.stdout: enable_index_compression: 1 2024-06-10T00:44:29.050 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_align: 0 2024-06-10T00:44:29.050 INFO:journalctl@ceph.mon.a.smithi037.stdout: max_auto_readahead_size: 262144 2024-06-10T00:44:29.050 INFO:journalctl@ceph.mon.a.smithi037.stdout: prepopulate_block_cache: 0 2024-06-10T00:44:29.050 INFO:journalctl@ceph.mon.a.smithi037.stdout: initial_auto_readahead_size: 8192 2024-06-10T00:44:29.050 INFO:journalctl@ceph.mon.a.smithi037.stdout: num_file_reads_for_auto_readahead: 2 2024-06-10T00:44:29.050 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.write_buffer_size: 33554432 2024-06-10T00:44:29.053 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.max_write_buffer_number: 2 2024-06-10T00:44:29.053 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.compression: NoCompression 2024-06-10T00:44:29.053 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.bottommost_compression: Disabled 2024-06-10T00:44:29.053 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.prefix_extractor: nullptr 2024-06-10T00:44:29.053 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-10T00:44:29.053 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.num_levels: 7 2024-06-10T00:44:29.053 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-10T00:44:29.053 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-10T00:44:29.054 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-10T00:44:29.054 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-10T00:44:29.054 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-10T00:44:29.054 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-10T00:44:29.054 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-10T00:44:29.054 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-10T00:44:29.054 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-10T00:44:29.054 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-10T00:44:29.054 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-10T00:44:29.054 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-10T00:44:29.054 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-10T00:44:29.054 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.compression_opts.level: 32767 2024-06-10T00:44:29.055 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.compression_opts.strategy: 0 2024-06-10T00:44:29.055 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-10T00:44:29.055 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-10T00:44:29.055 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-10T00:44:29.055 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-10T00:44:29.055 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.compression_opts.enabled: false 2024-06-10T00:44:29.055 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-10T00:44:29.055 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-10T00:44:29.055 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-10T00:44:29.055 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-10T00:44:29.055 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.target_file_size_base: 67108864 2024-06-10T00:44:29.055 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-10T00:44:29.055 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-10T00:44:29.055 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-10T00:44:29.056 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-10T00:44:29.056 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-10T00:44:29.056 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-10T00:44:29.056 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-10T00:44:29.056 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-10T00:44:29.056 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-10T00:44:29.056 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-10T00:44:29.056 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-10T00:44:29.056 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-10T00:44:29.056 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-10T00:44:29.056 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-10T00:44:29.056 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.arena_block_size: 1048576 2024-06-10T00:44:29.056 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-10T00:44:29.057 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-10T00:44:29.057 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.disable_auto_compactions: 0 2024-06-10T00:44:29.057 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-10T00:44:29.057 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-10T00:44:29.057 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-10T00:44:29.057 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-10T00:44:29.057 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-10T00:44:29.057 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-10T00:44:29.057 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-10T00:44:29.057 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-10T00:44:29.057 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-10T00:44:29.057 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-10T00:44:29.058 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.table_properties_collectors: 2024-06-10T00:44:29.058 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.inplace_update_support: 0 2024-06-10T00:44:29.058 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-10T00:44:29.058 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-10T00:44:29.058 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-10T00:44:29.058 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-10T00:44:29.058 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.bloom_locality: 0 2024-06-10T00:44:29.058 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.max_successive_merges: 0 2024-06-10T00:44:29.058 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-10T00:44:29.058 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.paranoid_file_checks: 0 2024-06-10T00:44:29.058 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.force_consistency_checks: 1 2024-06-10T00:44:29.058 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.report_bg_io_stats: 0 2024-06-10T00:44:29.059 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.ttl: 2592000 2024-06-10T00:44:29.059 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-10T00:44:29.059 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-10T00:44:29.059 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-10T00:44:29.059 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.enable_blob_files: false 2024-06-10T00:44:29.059 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.min_blob_size: 0 2024-06-10T00:44:29.059 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.blob_file_size: 268435456 2024-06-10T00:44:29.059 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-10T00:44:29.059 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-10T00:44:29.059 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-10T00:44:29.059 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-10T00:44:29.060 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-10T00:44:29.060 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.blob_file_starting_level: 0 2024-06-10T00:44:29.060 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-10T00:44:29.060 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-06-10T00:44:29.060 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-06-10T00:44:29.060 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 88f623c4-2fb8-467c-93d6-a0c38629ac16 2024-06-10T00:44:29.060 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717980268738641, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-06-10T00:44:29.060 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-06-10T00:44:29.060 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717980268739716, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100002, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97913, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14195, "raw_average_key_size": 49, "raw_value_size": 90200, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717980268, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "88f623c4-2fb8-467c-93d6-a0c38629ac16", "db_session_id": "PU11GAZ81P5VFO21D9D8", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-06-10T00:44:29.060 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717980268739801, "job": 1, "event": "recovery_finished"} 2024-06-10T00:44:29.060 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-06-10T00:44:29.061 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-10T00:44:29.061 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5601591be000 2024-06-10T00:44:29.061 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: DB pointer 0x5601591a8000 2024-06-10T00:44:29.061 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-10T00:44:29.061 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-10T00:44:29.061 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** DB Stats ** 2024-06-10T00:44:29.061 INFO:journalctl@ceph.mon.a.smithi037.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-10T00:44:29.061 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-10T00:44:29.061 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-10T00:44:29.061 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-10T00:44:29.061 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-10T00:44:29.061 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-10T00:44:29.061 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-10T00:44:29.061 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-06-10T00:44:29.062 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** Compaction Stats [default] ** 2024-06-10T00:44:29.062 INFO:journalctl@ceph.mon.a.smithi037.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-10T00:44:29.062 INFO:journalctl@ceph.mon.a.smithi037.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-10T00:44:29.062 INFO:journalctl@ceph.mon.a.smithi037.stdout: L0 2/0 99.49 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 146.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-10T00:44:29.062 INFO:journalctl@ceph.mon.a.smithi037.stdout: Sum 2/0 99.49 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 146.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-10T00:44:29.062 INFO:journalctl@ceph.mon.a.smithi037.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 146.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-10T00:44:29.062 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-06-10T00:44:29.062 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** Compaction Stats [default] ** 2024-06-10T00:44:29.062 INFO:journalctl@ceph.mon.a.smithi037.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-10T00:44:29.062 INFO:journalctl@ceph.mon.a.smithi037.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-10T00:44:29.062 INFO:journalctl@ceph.mon.a.smithi037.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 146.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-10T00:44:29.062 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-06-10T00:44:29.063 INFO:journalctl@ceph.mon.a.smithi037.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-10T00:44:29.063 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-06-10T00:44:29.063 INFO:journalctl@ceph.mon.a.smithi037.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-10T00:44:29.063 INFO:journalctl@ceph.mon.a.smithi037.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-10T00:44:29.063 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-10T00:44:29.063 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-10T00:44:29.063 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-10T00:44:29.063 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-10T00:44:29.063 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative compaction: 0.00 GB write, 30.15 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-10T00:44:29.063 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval compaction: 0.00 GB write, 30.15 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-10T00:44:29.063 INFO:journalctl@ceph.mon.a.smithi037.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-10T00:44:29.063 INFO:journalctl@ceph.mon.a.smithi037.stdout: Block cache BinnedLRUCache@0x5601590cf090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-06-10T00:44:29.063 INFO:journalctl@ceph.mon.a.smithi037.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-06-10T00:44:29.063 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-06-10T00:44:29.064 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-10T00:44:29.064 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: starting mon.a rank 0 at public addrs [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] at bind addrs [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:44:29.064 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: mon.a@-1(???) e1 preinit fsid 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:44:29.064 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: mon.a@-1(???).mds e1 new map 2024-06-10T00:44:29.064 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: mon.a@-1(???).mds e1 print_map 2024-06-10T00:44:29.064 INFO:journalctl@ceph.mon.a.smithi037.stdout: e1 2024-06-10T00:44:29.064 INFO:journalctl@ceph.mon.a.smithi037.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-10T00:44:29.064 INFO:journalctl@ceph.mon.a.smithi037.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-10T00:44:29.064 INFO:journalctl@ceph.mon.a.smithi037.stdout: legacy client fscid: -1 2024-06-10T00:44:29.064 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-06-10T00:44:29.064 INFO:journalctl@ceph.mon.a.smithi037.stdout: No filesystems configured 2024-06-10T00:44:29.064 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-10T00:44:29.064 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-10T00:44:29.065 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-10T00:44:29.065 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-10T00:44:29.065 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-06-10T00:44:29.065 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 bash[32173]: b72c961cf1e0d476a1b01095215bfac013f9985e0afb1213b4b8989ac35eb135 2024-06-10T00:44:29.065 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 systemd[1]: Started Ceph mon.a for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T00:44:29.065 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-10T00:44:29.065 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: monmap epoch 1 2024-06-10T00:44:29.065 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: fsid 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:44:29.065 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: last_changed 2024-06-10T00:44:21.142496+0000 2024-06-10T00:44:29.065 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: created 2024-06-10T00:44:21.142496+0000 2024-06-10T00:44:29.065 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: min_mon_release 19 (squid) 2024-06-10T00:44:29.065 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: election_strategy: 1 2024-06-10T00:44:29.065 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-06-10T00:44:29.066 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: fsmap 2024-06-10T00:44:29.066 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: osdmap e1: 0 total, 0 up, 0 in 2024-06-10T00:44:29.066 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:28 smithi037 ceph-mon[32192]: mgrmap e1: no daemons active 2024-06-10T00:44:29.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.248+0000 7f443b0d2700 1 Processor -- start 2024-06-10T00:44:29.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.248+0000 7f443b0d2700 1 -- start start 2024-06-10T00:44:29.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.248+0000 7f443b0d2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4434108a10 0x7f443410ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:29.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.248+0000 7f443b0d2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f443407be70 con 0x7f4434108a10 2024-06-10T00:44:29.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.249+0000 7f4438e6e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4434108a10 0x7f443410ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:29.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.249+0000 7f4438e6e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4434108a10 0x7f443410ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49322/0 (socket says 172.21.15.37:49322) 2024-06-10T00:44:29.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.249+0000 7f4438e6e700 1 -- 172.21.15.37:0/185100905 learned_addr learned my addr 172.21.15.37:0/185100905 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:44:29.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.249+0000 7f4438e6e700 1 -- 172.21.15.37:0/185100905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443410b340 con 0x7f4434108a10 2024-06-10T00:44:29.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.249+0000 7f4438e6e700 1 --2- 172.21.15.37:0/185100905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4434108a10 0x7f443410ae00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4424009e80 tx=0x7f442400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a48dfb9838d46cb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:29.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.250+0000 7f44337fe700 1 -- 172.21.15.37:0/185100905 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4424017070 con 0x7f4434108a10 2024-06-10T00:44:29.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.250+0000 7f44337fe700 1 -- 172.21.15.37:0/185100905 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f442400b8c0 con 0x7f4434108a10 2024-06-10T00:44:29.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.250+0000 7f44337fe700 1 -- 172.21.15.37:0/185100905 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44240045c0 con 0x7f4434108a10 2024-06-10T00:44:29.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.250+0000 7f443b0d2700 1 -- 172.21.15.37:0/185100905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4434108a10 msgr2=0x7f443410ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:29.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.250+0000 7f443b0d2700 1 --2- 172.21.15.37:0/185100905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4434108a10 0x7f443410ae00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4424009e80 tx=0x7f442400b330 comp rx=0 tx=0).stop 2024-06-10T00:44:29.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.250+0000 7f443b0d2700 1 -- 172.21.15.37:0/185100905 shutdown_connections 2024-06-10T00:44:29.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.250+0000 7f443b0d2700 1 --2- 172.21.15.37:0/185100905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4434108a10 0x7f443410ae00 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:29.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.250+0000 7f443b0d2700 1 -- 172.21.15.37:0/185100905 >> 172.21.15.37:0/185100905 conn(0x7f4434102400 msgr2=0x7f4434104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:29.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.250+0000 7f443b0d2700 1 -- 172.21.15.37:0/185100905 shutdown_connections 2024-06-10T00:44:29.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.250+0000 7f443b0d2700 1 -- 172.21.15.37:0/185100905 wait complete. 2024-06-10T00:44:29.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.251+0000 7f443b0d2700 1 Processor -- start 2024-06-10T00:44:29.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.251+0000 7f443b0d2700 1 -- start start 2024-06-10T00:44:29.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.251+0000 7f443b0d2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4434108a10 0x7f443419be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:29.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.251+0000 7f443b0d2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f443410c360 con 0x7f4434108a10 2024-06-10T00:44:29.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.251+0000 7f4438e6e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4434108a10 0x7f443419be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:29.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.251+0000 7f4438e6e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4434108a10 0x7f443419be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49334/0 (socket says 172.21.15.37:49334) 2024-06-10T00:44:29.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.251+0000 7f4438e6e700 1 -- 172.21.15.37:0/3065501361 learned_addr learned my addr 172.21.15.37:0/3065501361 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:44:29.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.251+0000 7f4438e6e700 1 -- 172.21.15.37:0/3065501361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443419c3c0 con 0x7f4434108a10 2024-06-10T00:44:29.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.252+0000 7f4438e6e700 1 --2- 172.21.15.37:0/3065501361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4434108a10 0x7f443419be80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f44240072a0 tx=0x7f442400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:29.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.252+0000 7f4431ffb700 1 -- 172.21.15.37:0/3065501361 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4424020020 con 0x7f4434108a10 2024-06-10T00:44:29.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.252+0000 7f4431ffb700 1 -- 172.21.15.37:0/3065501361 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4424003b50 con 0x7f4434108a10 2024-06-10T00:44:29.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.252+0000 7f443b0d2700 1 -- 172.21.15.37:0/3065501361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f443419c650 con 0x7f4434108a10 2024-06-10T00:44:29.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.252+0000 7f4431ffb700 1 -- 172.21.15.37:0/3065501361 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4424017040 con 0x7f4434108a10 2024-06-10T00:44:29.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.252+0000 7f443b0d2700 1 -- 172.21.15.37:0/3065501361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f443419f340 con 0x7f4434108a10 2024-06-10T00:44:29.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.252+0000 7f4431ffb700 1 -- 172.21.15.37:0/3065501361 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4424023020 con 0x7f4434108a10 2024-06-10T00:44:29.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.252+0000 7f4431ffb700 1 -- 172.21.15.37:0/3065501361 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f442400f050 con 0x7f4434108a10 2024-06-10T00:44:29.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.253+0000 7f443b0d2700 1 -- 172.21.15.37:0/3065501361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4434063af0 con 0x7f4434108a10 2024-06-10T00:44:29.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.255+0000 7f4431ffb700 1 -- 172.21.15.37:0/3065501361 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f442402a020 con 0x7f4434108a10 2024-06-10T00:44:29.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.326+0000 7f443b0d2700 1 -- 172.21.15.37:0/3065501361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f443419f510 con 0x7f4434108a10 2024-06-10T00:44:29.905 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.327+0000 7f4431ffb700 1 -- 172.21.15.37:0/3065501361 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f4424003d00 con 0x7f4434108a10 2024-06-10T00:44:29.905 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.328+0000 7f443b0d2700 1 -- 172.21.15.37:0/3065501361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4434108a10 msgr2=0x7f443419be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:29.905 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.328+0000 7f443b0d2700 1 --2- 172.21.15.37:0/3065501361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4434108a10 0x7f443419be80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f44240072a0 tx=0x7f442400bd50 comp rx=0 tx=0).stop 2024-06-10T00:44:29.905 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.328+0000 7f443b0d2700 1 -- 172.21.15.37:0/3065501361 shutdown_connections 2024-06-10T00:44:29.905 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.328+0000 7f443b0d2700 1 --2- 172.21.15.37:0/3065501361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4434108a10 0x7f443419be80 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:29.905 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.328+0000 7f443b0d2700 1 -- 172.21.15.37:0/3065501361 >> 172.21.15.37:0/3065501361 conn(0x7f4434102400 msgr2=0x7f4434102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:29.905 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.329+0000 7f443b0d2700 1 -- 172.21.15.37:0/3065501361 shutdown_connections 2024-06-10T00:44:29.905 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:29.329+0000 7f443b0d2700 1 -- 172.21.15.37:0/3065501361 wait complete. 2024-06-10T00:44:29.905 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote config to /etc/ceph/ceph.conf 2024-06-10T00:44:29.926 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-06-10T00:44:29.926 INFO:teuthology.orchestra.run.smithi037.stdout:Creating mgr... 2024-06-10T00:44:29.927 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying port 0.0.0.0:9283 ... 2024-06-10T00:44:29.927 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying port 0.0.0.0:8765 ... 2024-06-10T00:44:30.177 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-68793514-26c2-11ef-bc9b-c7b262605968@mgr.x 2024-06-10T00:44:30.177 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to reset failed state of unit ceph-68793514-26c2-11ef-bc9b-c7b262605968@mgr.x.service: Unit ceph-68793514-26c2-11ef-bc9b-c7b262605968@mgr.x.service not loaded. 2024-06-10T00:44:30.334 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-68793514-26c2-11ef-bc9b-c7b262605968.target.wants/ceph-68793514-26c2-11ef-bc9b-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-68793514-26c2-11ef-bc9b-c7b262605968@.service. 2024-06-10T00:44:30.571 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:30 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3065501361' entity='client.admin' 2024-06-10T00:44:30.571 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:30 smithi037 systemd[1]: Starting Ceph mgr.x for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T00:44:30.895 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:30 smithi037 podman[32408]: 2024-06-10 00:44:30.473904304 +0000 UTC m=+0.018223627 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-10T00:44:31.896 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:31 smithi037 podman[32408]: 2024-06-10 00:44:31.630865654 +0000 UTC m=+1.175184948 container create 8038bdd9f3fd672ed68a5b12abe68d9eda1020817d5aaa89eb0e3d36c41062c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528) 2024-06-10T00:44:33.857 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:33 smithi037 podman[32408]: 2024-06-10 00:44:33.491127995 +0000 UTC m=+3.035447290 container init 8038bdd9f3fd672ed68a5b12abe68d9eda1020817d5aaa89eb0e3d36c41062c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_CLEAN=True) 2024-06-10T00:44:33.857 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:33 smithi037 podman[32408]: 2024-06-10 00:44:33.495209787 +0000 UTC m=+3.039529098 container start 8038bdd9f3fd672ed68a5b12abe68d9eda1020817d5aaa89eb0e3d36c41062c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-06-10T00:44:33.857 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:33 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:33.663+0000 7f0433d6c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-10T00:44:33.857 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:33 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:33.855+0000 7f0433d6c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:44:34.145 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:34 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:33.964+0000 7f0433d6c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-10T00:44:34.451 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-10T00:44:34.452 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-06-10T00:44:34.459 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-10T00:44:34.459 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-06-10T00:44:34.459 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-06-10T00:44:34.459 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to enable service . firewalld.service is not available 2024-06-10T00:44:34.468 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-10T00:44:34.468 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-06-10T00:44:34.476 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-10T00:44:34.476 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-06-10T00:44:34.476 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-06-10T00:44:34.476 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-06-10T00:44:34.476 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr to start... 2024-06-10T00:44:34.476 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr... 2024-06-10T00:44:34.581 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:34 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:34.288+0000 7f0433d6c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-10T00:44:34.581 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:34 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:34.378+0000 7f0433d6c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-10T00:44:34.581 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:34 smithi037 bash[32408]: 8038bdd9f3fd672ed68a5b12abe68d9eda1020817d5aaa89eb0e3d36c41062c9 2024-06-10T00:44:34.581 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:34 smithi037 systemd[1]: Started Ceph mgr.x for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T00:44:35.895 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:35 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:35.680+0000 7f0433d6c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-10T00:44:35.896 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:35 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:35.859+0000 7f0433d6c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-10T00:44:36.351 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:36 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:36.039+0000 7f0433d6c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-10T00:44:36.351 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:36 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:36.349+0000 7f0433d6c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-10T00:44:36.646 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:36 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:36.447+0000 7f0433d6c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-10T00:44:37.068 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:36 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:36.663+0000 7f0433d6c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-10T00:44:37.068 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:36 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:36.866+0000 7f0433d6c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:44:37.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:37 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1330542947' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-10T00:44:37.646 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:37 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:37.505+0000 7f0433d6c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-10T00:44:38.261 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:37 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:37.950+0000 7f0433d6c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-10T00:44:38.262 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:38 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:38.051+0000 7f0433d6c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-10T00:44:38.262 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:38 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:38.260+0000 7f0433d6c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-10T00:44:39.145 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:38 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:38.806+0000 7f0433d6c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-10T00:44:39.146 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:38 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:38.901+0000 7f0433d6c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-10T00:44:39.514 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:39 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:39.155+0000 7f0433d6c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-10T00:44:39.515 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:39 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:39.294+0000 7f0433d6c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-10T00:44:39.515 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:39 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:39.392+0000 7f0433d6c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-10T00:44:39.515 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:39 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:39.513+0000 7f0433d6c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-10T00:44:40.145 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:39 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:39.825+0000 7f0433d6c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-10T00:44:40.740 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:40 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:40.458+0000 7f0433d6c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-10T00:44:40.741 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:40 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:40.684+0000 7f0433d6c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-10T00:44:41.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:40 smithi037 ceph-mon[32192]: Activating manager daemon x 2024-06-10T00:44:41.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:40 smithi037 ceph-mon[32192]: mgrmap e2: x(active, starting, since 0.00303723s) 2024-06-10T00:44:41.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:40 smithi037 ceph-mon[32192]: from='mgr.14100 172.21.15.37:0/2620922503' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-10T00:44:41.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:40 smithi037 ceph-mon[32192]: from='mgr.14100 172.21.15.37:0/2620922503' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-10T00:44:41.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:40 smithi037 ceph-mon[32192]: from='mgr.14100 172.21.15.37:0/2620922503' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-10T00:44:41.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:40 smithi037 ceph-mon[32192]: from='mgr.14100 172.21.15.37:0/2620922503' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:44:41.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:40 smithi037 ceph-mon[32192]: from='mgr.14100 172.21.15.37:0/2620922503' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-10T00:44:41.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:40 smithi037 ceph-mon[32192]: Manager daemon x is now available 2024-06-10T00:44:41.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:40 smithi037 ceph-mon[32192]: from='mgr.14100 172.21.15.37:0/2620922503' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-10T00:44:41.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:40 smithi037 ceph-mon[32192]: from='mgr.14100 172.21.15.37:0/2620922503' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-10T00:44:41.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:40 smithi037 ceph-mon[32192]: from='mgr.14100 172.21.15.37:0/2620922503' entity='mgr.x' 2024-06-10T00:44:41.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:40 smithi037 ceph-mon[32192]: from='mgr.14100 172.21.15.37:0/2620922503' entity='mgr.x' 2024-06-10T00:44:41.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:40 smithi037 ceph-mon[32192]: from='mgr.14100 172.21.15.37:0/2620922503' entity='mgr.x' 2024-06-10T00:44:41.604 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-06-10T00:44:41.605 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-06-10T00:44:41.605 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "68793514-26c2-11ef-bc9b-c7b262605968", 2024-06-10T00:44:41.605 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-06-10T00:44:41.605 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-10T00:44:41.605 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-10T00:44:41.605 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-10T00:44:41.606 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:44:41.606 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-10T00:44:41.606 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-10T00:44:41.606 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-06-10T00:44:41.606 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:44:41.606 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-10T00:44:41.606 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-06-10T00:44:41.606 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:44:41.606 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-06-10T00:44:41.607 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-10T00:44:41.607 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:44:41.607 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-10T00:44:41.607 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-10T00:44:41.607 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:44:41.607 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-10T00:44:41.607 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:44:41.607 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-10T00:44:41.607 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-10T00:44:41.607 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-10T00:44:41.615 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-10T00:44:41.615 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-10T00:44:41.615 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-10T00:44:41.615 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:44:41.615 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-10T00:44:41.615 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-10T00:44:41.615 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-10T00:44:41.616 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-10T00:44:41.616 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-10T00:44:41.616 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-10T00:44:41.616 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-10T00:44:41.616 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-10T00:44:41.616 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-10T00:44:41.616 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:44:41.616 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-10T00:44:41.616 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:44:41.616 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-10T00:44:41.616 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-10T00:44:41.616 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:44:41.616 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-10T00:44:41.616 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-10T00:44:41.617 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-10T00:44:41.617 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-10T00:44:41.617 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-10T00:44:41.617 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-10T00:44:41.617 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-06-10T00:44:41.617 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:44:41.617 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-10T00:44:41.617 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:44:41.617 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-10T00:44:41.617 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:44:41.617 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-06-10T00:44:24.107195+0000", 2024-06-10T00:44:41.617 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-10T00:44:41.617 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:44:41.617 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-10T00:44:41.617 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-06-10T00:44:41.618 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.493+0000 7fa2ccee2700 1 Processor -- start 2024-06-10T00:44:41.618 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.493+0000 7fa2ccee2700 1 -- start start 2024-06-10T00:44:41.618 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.494+0000 7fa2ccee2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2c8106e50 0x7fa2c8107270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:41.618 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.494+0000 7fa2ccee2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c81077b0 con 0x7fa2c8106e50 2024-06-10T00:44:41.618 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.494+0000 7fa2c659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2c8106e50 0x7fa2c8107270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:41.618 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.494+0000 7fa2c659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2c8106e50 0x7fa2c8107270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56124/0 (socket says 172.21.15.37:56124) 2024-06-10T00:44:41.618 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.494+0000 7fa2c659c700 1 -- 172.21.15.37:0/3359819614 learned_addr learned my addr 172.21.15.37:0/3359819614 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:44:41.618 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.494+0000 7fa2c659c700 1 -- 172.21.15.37:0/3359819614 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c8107fb0 con 0x7fa2c8106e50 2024-06-10T00:44:41.618 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.495+0000 7fa2c659c700 1 --2- 172.21.15.37:0/3359819614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2c8106e50 0x7fa2c8107270 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80098a0 tx=0x7fa2b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db2a33edf2eca1a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:41.618 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.495+0000 7fa2c559a700 1 -- 172.21.15.37:0/3359819614 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2b8004160 con 0x7fa2c8106e50 2024-06-10T00:44:41.618 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.495+0000 7fa2c559a700 1 -- 172.21.15.37:0/3359819614 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa2b8004300 con 0x7fa2c8106e50 2024-06-10T00:44:41.618 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.495+0000 7fa2c559a700 1 -- 172.21.15.37:0/3359819614 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2b8011b50 con 0x7fa2c8106e50 2024-06-10T00:44:41.618 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.495+0000 7fa2ccee2700 1 -- 172.21.15.37:0/3359819614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2c8106e50 msgr2=0x7fa2c8107270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:41.618 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.495+0000 7fa2ccee2700 1 --2- 172.21.15.37:0/3359819614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2c8106e50 0x7fa2c8107270 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80098a0 tx=0x7fa2b8009bb0 comp rx=0 tx=0).stop 2024-06-10T00:44:41.619 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.496+0000 7fa2ccee2700 1 -- 172.21.15.37:0/3359819614 shutdown_connections 2024-06-10T00:44:41.619 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.496+0000 7fa2ccee2700 1 --2- 172.21.15.37:0/3359819614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2c8106e50 0x7fa2c8107270 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:41.619 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.496+0000 7fa2ccee2700 1 -- 172.21.15.37:0/3359819614 >> 172.21.15.37:0/3359819614 conn(0x7fa2c81023d0 msgr2=0x7fa2c8104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:41.619 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.496+0000 7fa2ccee2700 1 -- 172.21.15.37:0/3359819614 shutdown_connections 2024-06-10T00:44:41.619 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.496+0000 7fa2ccee2700 1 -- 172.21.15.37:0/3359819614 wait complete. 2024-06-10T00:44:41.619 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.496+0000 7fa2ccee2700 1 Processor -- start 2024-06-10T00:44:41.619 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.497+0000 7fa2ccee2700 1 -- start start 2024-06-10T00:44:41.619 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.497+0000 7fa2ccee2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2c8106e50 0x7fa2c819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:41.619 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.497+0000 7fa2ccee2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c810b3f0 con 0x7fa2c8106e50 2024-06-10T00:44:41.619 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.497+0000 7fa2c659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2c8106e50 0x7fa2c819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:41.619 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.497+0000 7fa2c659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2c8106e50 0x7fa2c819add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56140/0 (socket says 172.21.15.37:56140) 2024-06-10T00:44:41.619 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.497+0000 7fa2c659c700 1 -- 172.21.15.37:0/1330542947 learned_addr learned my addr 172.21.15.37:0/1330542947 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:44:41.619 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.497+0000 7fa2c659c700 1 -- 172.21.15.37:0/1330542947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c819b310 con 0x7fa2c8106e50 2024-06-10T00:44:41.619 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.497+0000 7fa2c659c700 1 --2- 172.21.15.37:0/1330542947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2c8106e50 0x7fa2c819add0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8011670 tx=0x7fa2b8003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:41.619 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.497+0000 7fa2bf7fe700 1 -- 172.21.15.37:0/1330542947 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2b8003c20 con 0x7fa2c8106e50 2024-06-10T00:44:41.620 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.497+0000 7fa2bf7fe700 1 -- 172.21.15.37:0/1330542947 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa2b8011e50 con 0x7fa2c8106e50 2024-06-10T00:44:41.620 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.497+0000 7fa2ccee2700 1 -- 172.21.15.37:0/1330542947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c819b5a0 con 0x7fa2c8106e50 2024-06-10T00:44:41.620 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.498+0000 7fa2bf7fe700 1 -- 172.21.15.37:0/1330542947 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2b801b820 con 0x7fa2c8106e50 2024-06-10T00:44:41.620 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.498+0000 7fa2ccee2700 1 -- 172.21.15.37:0/1330542947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c807b860 con 0x7fa2c8106e50 2024-06-10T00:44:41.620 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.498+0000 7fa2bf7fe700 1 -- 172.21.15.37:0/1330542947 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa2b801b9c0 con 0x7fa2c8106e50 2024-06-10T00:44:41.620 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.498+0000 7fa2bf7fe700 1 -- 172.21.15.37:0/1330542947 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa2b802e730 con 0x7fa2c8106e50 2024-06-10T00:44:41.620 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.499+0000 7fa2ccee2700 1 -- 172.21.15.37:0/1330542947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c8063af0 con 0x7fa2c8106e50 2024-06-10T00:44:41.620 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.501+0000 7fa2bf7fe700 1 -- 172.21.15.37:0/1330542947 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa2b8004230 con 0x7fa2c8106e50 2024-06-10T00:44:41.620 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.576+0000 7fa2ccee2700 1 -- 172.21.15.37:0/1330542947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa2c807bd30 con 0x7fa2c8106e50 2024-06-10T00:44:41.620 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.577+0000 7fa2bf7fe700 1 -- 172.21.15.37:0/1330542947 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fa2b801ccb0 con 0x7fa2c8106e50 2024-06-10T00:44:41.620 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.578+0000 7fa2ccee2700 1 -- 172.21.15.37:0/1330542947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2c8106e50 msgr2=0x7fa2c819add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:41.620 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.578+0000 7fa2ccee2700 1 --2- 172.21.15.37:0/1330542947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2c8106e50 0x7fa2c819add0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8011670 tx=0x7fa2b8003980 comp rx=0 tx=0).stop 2024-06-10T00:44:41.620 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.578+0000 7fa2ccee2700 1 -- 172.21.15.37:0/1330542947 shutdown_connections 2024-06-10T00:44:41.620 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.578+0000 7fa2ccee2700 1 --2- 172.21.15.37:0/1330542947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2c8106e50 0x7fa2c819add0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:41.621 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.578+0000 7fa2ccee2700 1 -- 172.21.15.37:0/1330542947 >> 172.21.15.37:0/1330542947 conn(0x7fa2c81023d0 msgr2=0x7fa2c8102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:41.621 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.578+0000 7fa2ccee2700 1 -- 172.21.15.37:0/1330542947 shutdown_connections 2024-06-10T00:44:41.621 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:37.578+0000 7fa2ccee2700 1 -- 172.21.15.37:0/1330542947 wait complete. 2024-06-10T00:44:41.621 INFO:teuthology.orchestra.run.smithi037.stdout:mgr not available, waiting (1/15)... 2024-06-10T00:44:43.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:42 smithi037 ceph-mon[32192]: mgrmap e3: x(active, since 1.00802s) 2024-06-10T00:44:43.992 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:43 smithi037 ceph-mon[32192]: mgrmap e4: x(active, since 2s) 2024-06-10T00:44:44.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-06-10T00:44:44.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-06-10T00:44:44.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "68793514-26c2-11ef-bc9b-c7b262605968", 2024-06-10T00:44:44.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-06-10T00:44:44.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-10T00:44:44.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-10T00:44:44.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-10T00:44:44.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:44:44.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-10T00:44:44.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-10T00:44:44.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-06-10T00:44:44.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:44:44.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-10T00:44:44.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-06-10T00:44:44.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:44:44.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-06-10T00:44:44.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-10T00:44:44.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:44:44.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-10T00:44:44.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-10T00:44:44.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:44:44.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-10T00:44:44.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:44:44.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-10T00:44:44.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-10T00:44:44.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-10T00:44:44.910 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-10T00:44:44.911 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-10T00:44:44.911 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-10T00:44:44.911 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:44:44.911 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-10T00:44:44.911 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-10T00:44:44.911 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-10T00:44:44.911 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-10T00:44:44.911 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-10T00:44:44.911 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-10T00:44:44.911 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-10T00:44:44.911 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-10T00:44:44.911 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-10T00:44:44.911 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:44:44.912 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-10T00:44:44.912 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:44:44.912 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-10T00:44:44.912 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-10T00:44:44.912 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:44:44.912 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-10T00:44:44.912 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-10T00:44:44.912 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-10T00:44:44.912 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-10T00:44:44.912 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-10T00:44:44.912 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-10T00:44:44.912 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-06-10T00:44:44.912 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-06-10T00:44:44.912 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-10T00:44:44.913 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:44:44.913 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-10T00:44:44.913 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-10T00:44:44.913 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-06-10T00:44:24.107195+0000", 2024-06-10T00:44:44.913 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-10T00:44:44.913 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-06-10T00:44:44.913 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-10T00:44:44.913 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-06-10T00:44:44.913 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.048+0000 7f91027ba700 1 Processor -- start 2024-06-10T00:44:44.913 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.048+0000 7f91027ba700 1 -- start start 2024-06-10T00:44:44.913 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.048+0000 7f91027ba700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90fc107f70 0x7f90fc108390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:44.913 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.048+0000 7f91027ba700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f90fc1088d0 con 0x7f90fc107f70 2024-06-10T00:44:44.913 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.049+0000 7f90fbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90fc107f70 0x7f90fc108390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:44.913 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.049+0000 7f90fbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90fc107f70 0x7f90fc108390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35884/0 (socket says 172.21.15.37:35884) 2024-06-10T00:44:44.913 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.049+0000 7f90fbfff700 1 -- 172.21.15.37:0/4178824489 learned_addr learned my addr 172.21.15.37:0/4178824489 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:44:44.914 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.049+0000 7f90fbfff700 1 -- 172.21.15.37:0/4178824489 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90fc109080 con 0x7f90fc107f70 2024-06-10T00:44:44.914 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.049+0000 7f90fbfff700 1 --2- 172.21.15.37:0/4178824489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90fc107f70 0x7f90fc108390 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f90ec0098a0 tx=0x7f90ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b9a0695ac238a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:44.914 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.049+0000 7f90faffd700 1 -- 172.21.15.37:0/4178824489 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90ec004160 con 0x7f90fc107f70 2024-06-10T00:44:44.914 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.049+0000 7f90faffd700 1 -- 172.21.15.37:0/4178824489 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f90ec004300 con 0x7f90fc107f70 2024-06-10T00:44:44.914 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.049+0000 7f90faffd700 1 -- 172.21.15.37:0/4178824489 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90ec011b50 con 0x7f90fc107f70 2024-06-10T00:44:44.914 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.050+0000 7f91027ba700 1 -- 172.21.15.37:0/4178824489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90fc107f70 msgr2=0x7f90fc108390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:44.914 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.050+0000 7f91027ba700 1 --2- 172.21.15.37:0/4178824489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90fc107f70 0x7f90fc108390 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f90ec0098a0 tx=0x7f90ec009bb0 comp rx=0 tx=0).stop 2024-06-10T00:44:44.914 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.050+0000 7f91027ba700 1 -- 172.21.15.37:0/4178824489 shutdown_connections 2024-06-10T00:44:44.914 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.050+0000 7f91027ba700 1 --2- 172.21.15.37:0/4178824489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90fc107f70 0x7f90fc108390 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:44.914 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.050+0000 7f91027ba700 1 -- 172.21.15.37:0/4178824489 >> 172.21.15.37:0/4178824489 conn(0x7f90fc103530 msgr2=0x7f90fc105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:44.914 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.050+0000 7f91027ba700 1 -- 172.21.15.37:0/4178824489 shutdown_connections 2024-06-10T00:44:44.914 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.050+0000 7f91027ba700 1 -- 172.21.15.37:0/4178824489 wait complete. 2024-06-10T00:44:44.914 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.051+0000 7f91027ba700 1 Processor -- start 2024-06-10T00:44:44.914 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.051+0000 7f91027ba700 1 -- start start 2024-06-10T00:44:44.915 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.051+0000 7f91027ba700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90fc107f70 0x7f90fc19bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:44.915 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.051+0000 7f91027ba700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f90fc10c4c0 con 0x7f90fc107f70 2024-06-10T00:44:44.915 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.051+0000 7f90fbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90fc107f70 0x7f90fc19bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:44.915 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.051+0000 7f90fbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90fc107f70 0x7f90fc19bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35890/0 (socket says 172.21.15.37:35890) 2024-06-10T00:44:44.915 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.051+0000 7f90fbfff700 1 -- 172.21.15.37:0/3810516714 learned_addr learned my addr 172.21.15.37:0/3810516714 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:44:44.915 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.052+0000 7f90fbfff700 1 -- 172.21.15.37:0/3810516714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90fc19c440 con 0x7f90fc107f70 2024-06-10T00:44:44.915 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.052+0000 7f90fbfff700 1 --2- 172.21.15.37:0/3810516714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90fc107f70 0x7f90fc19bf00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f90ec011b60 tx=0x7f90ec011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:44.915 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.052+0000 7f90f97fa700 1 -- 172.21.15.37:0/3810516714 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90ec0039e0 con 0x7f90fc107f70 2024-06-10T00:44:44.915 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.052+0000 7f90f97fa700 1 -- 172.21.15.37:0/3810516714 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f90ec003b80 con 0x7f90fc107f70 2024-06-10T00:44:44.915 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.052+0000 7f91027ba700 1 -- 172.21.15.37:0/3810516714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90fc19c6d0 con 0x7f90fc107f70 2024-06-10T00:44:44.915 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.052+0000 7f90f97fa700 1 -- 172.21.15.37:0/3810516714 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90ec024760 con 0x7f90fc107f70 2024-06-10T00:44:44.915 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.052+0000 7f91027ba700 1 -- 172.21.15.37:0/3810516714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90fc19f3c0 con 0x7f90fc107f70 2024-06-10T00:44:44.915 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.053+0000 7f90f97fa700 1 -- 172.21.15.37:0/3810516714 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f90ec022020 con 0x7f90fc107f70 2024-06-10T00:44:44.915 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.053+0000 7f90f97fa700 1 --2- 172.21.15.37:0/3810516714 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f90e4039000 0x7f90e403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:44.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.053+0000 7f90f97fa700 1 -- 172.21.15.37:0/3810516714 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f90ec011e00 con 0x7f90fc107f70 2024-06-10T00:44:44.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.053+0000 7f91027ba700 1 -- 172.21.15.37:0/3810516714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90fc063af0 con 0x7f90fc107f70 2024-06-10T00:44:44.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.053+0000 7f90fb7fe700 1 --2- 172.21.15.37:0/3810516714 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f90e4039000 0x7f90e403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:44.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.054+0000 7f90fb7fe700 1 --2- 172.21.15.37:0/3810516714 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f90e4039000 0x7f90e403b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f90f0006fd0 tx=0x7f90f0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:44.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.059+0000 7f90f97fa700 1 -- 172.21.15.37:0/3810516714 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f90ec04cb00 con 0x7f90fc107f70 2024-06-10T00:44:44.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.351+0000 7f91027ba700 1 -- 172.21.15.37:0/3810516714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f90fc19f880 con 0x7f90fc107f70 2024-06-10T00:44:44.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.352+0000 7f90f97fa700 1 -- 172.21.15.37:0/3810516714 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f90ec00b5a0 con 0x7f90fc107f70 2024-06-10T00:44:44.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.354+0000 7f91027ba700 1 -- 172.21.15.37:0/3810516714 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f90e4039000 msgr2=0x7f90e403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:44.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.354+0000 7f91027ba700 1 --2- 172.21.15.37:0/3810516714 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f90e4039000 0x7f90e403b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f90f0006fd0 tx=0x7f90f0006e40 comp rx=0 tx=0).stop 2024-06-10T00:44:44.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.354+0000 7f91027ba700 1 -- 172.21.15.37:0/3810516714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90fc107f70 msgr2=0x7f90fc19bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:44.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.354+0000 7f91027ba700 1 --2- 172.21.15.37:0/3810516714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90fc107f70 0x7f90fc19bf00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f90ec011b60 tx=0x7f90ec011c40 comp rx=0 tx=0).stop 2024-06-10T00:44:44.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.354+0000 7f91027ba700 1 -- 172.21.15.37:0/3810516714 shutdown_connections 2024-06-10T00:44:44.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.354+0000 7f91027ba700 1 --2- 172.21.15.37:0/3810516714 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f90e4039000 0x7f90e403b4c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:44.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.354+0000 7f91027ba700 1 --2- 172.21.15.37:0/3810516714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90fc107f70 0x7f90fc19bf00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:44.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.354+0000 7f91027ba700 1 -- 172.21.15.37:0/3810516714 >> 172.21.15.37:0/3810516714 conn(0x7f90fc103530 msgr2=0x7f90fc1040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:44.917 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.354+0000 7f91027ba700 1 -- 172.21.15.37:0/3810516714 shutdown_connections 2024-06-10T00:44:44.917 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:44.355+0000 7f91027ba700 1 -- 172.21.15.37:0/3810516714 wait complete. 2024-06-10T00:44:44.917 INFO:teuthology.orchestra.run.smithi037.stdout:mgr is available 2024-06-10T00:44:44.996 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:44 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3810516714' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-10T00:44:45.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:45 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1912075624' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-10T00:44:46.003 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-06-10T00:44:46.003 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [global] 2024-06-10T00:44:46.003 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout fsid = 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:44:46.004 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-10T00:44:46.004 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-10T00:44:46.004 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-10T00:44:46.004 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-10T00:44:46.004 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-06-10T00:44:46.004 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-10T00:44:46.004 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-10T00:44:46.004 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-06-10T00:44:46.004 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [osd] 2024-06-10T00:44:46.004 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-10T00:44:46.004 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-10T00:44:46.004 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-10T00:44:46.004 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-10T00:44:46.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-10T00:44:46.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.245+0000 7f8fcefc3700 1 Processor -- start 2024-06-10T00:44:46.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.246+0000 7f8fcefc3700 1 -- start start 2024-06-10T00:44:46.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.246+0000 7f8fcefc3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc8108a90 0x7f8fc810ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:46.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.246+0000 7f8fcefc3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc807c720 con 0x7f8fc8108a90 2024-06-10T00:44:46.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.246+0000 7f8fccd5f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc8108a90 0x7f8fc810ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:46.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.246+0000 7f8fccd5f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc8108a90 0x7f8fc810ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35896/0 (socket says 172.21.15.37:35896) 2024-06-10T00:44:46.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.246+0000 7f8fccd5f700 1 -- 172.21.15.37:0/1849810380 learned_addr learned my addr 172.21.15.37:0/1849810380 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:44:46.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.246+0000 7f8fccd5f700 1 -- 172.21.15.37:0/1849810380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc810b3c0 con 0x7f8fc8108a90 2024-06-10T00:44:46.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.246+0000 7f8fccd5f700 1 --2- 172.21.15.37:0/1849810380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc8108a90 0x7f8fc810ae80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc0098a0 tx=0x7f8fbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82e6e641fec72bb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:46.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.247+0000 7f8fc77fe700 1 -- 172.21.15.37:0/1849810380 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fbc004160 con 0x7f8fc8108a90 2024-06-10T00:44:46.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.247+0000 7f8fc77fe700 1 -- 172.21.15.37:0/1849810380 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8fbc004300 con 0x7f8fc8108a90 2024-06-10T00:44:46.005 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.247+0000 7f8fc77fe700 1 -- 172.21.15.37:0/1849810380 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fbc011b50 con 0x7f8fc8108a90 2024-06-10T00:44:46.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.247+0000 7f8fcefc3700 1 -- 172.21.15.37:0/1849810380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc8108a90 msgr2=0x7f8fc810ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:46.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.247+0000 7f8fcefc3700 1 --2- 172.21.15.37:0/1849810380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc8108a90 0x7f8fc810ae80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc0098a0 tx=0x7f8fbc009bb0 comp rx=0 tx=0).stop 2024-06-10T00:44:46.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.247+0000 7f8fcefc3700 1 -- 172.21.15.37:0/1849810380 shutdown_connections 2024-06-10T00:44:46.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.247+0000 7f8fcefc3700 1 --2- 172.21.15.37:0/1849810380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc8108a90 0x7f8fc810ae80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:46.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.247+0000 7f8fcefc3700 1 -- 172.21.15.37:0/1849810380 >> 172.21.15.37:0/1849810380 conn(0x7f8fc81024e0 msgr2=0x7f8fc8104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:46.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.247+0000 7f8fcefc3700 1 -- 172.21.15.37:0/1849810380 shutdown_connections 2024-06-10T00:44:46.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.247+0000 7f8fcefc3700 1 -- 172.21.15.37:0/1849810380 wait complete. 2024-06-10T00:44:46.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.248+0000 7f8fcefc3700 1 Processor -- start 2024-06-10T00:44:46.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.248+0000 7f8fcefc3700 1 -- start start 2024-06-10T00:44:46.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.248+0000 7f8fcefc3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc8108a90 0x7f8fc8108310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:46.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.248+0000 7f8fcefc3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc810c300 con 0x7f8fc8108a90 2024-06-10T00:44:46.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.249+0000 7f8fccd5f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc8108a90 0x7f8fc8108310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:46.006 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.249+0000 7f8fccd5f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc8108a90 0x7f8fc8108310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35902/0 (socket says 172.21.15.37:35902) 2024-06-10T00:44:46.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.249+0000 7f8fccd5f700 1 -- 172.21.15.37:0/1912075624 learned_addr learned my addr 172.21.15.37:0/1912075624 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:44:46.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.249+0000 7f8fccd5f700 1 -- 172.21.15.37:0/1912075624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc8108850 con 0x7f8fc8108a90 2024-06-10T00:44:46.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.249+0000 7f8fccd5f700 1 --2- 172.21.15.37:0/1912075624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc8108a90 0x7f8fc8108310 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc011670 tx=0x7f8fbc011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:46.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.250+0000 7f8fc5ffb700 1 -- 172.21.15.37:0/1912075624 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fbc0039c0 con 0x7f8fc8108a90 2024-06-10T00:44:46.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.250+0000 7f8fc5ffb700 1 -- 172.21.15.37:0/1912075624 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8fbc011e50 con 0x7f8fc8108a90 2024-06-10T00:44:46.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.250+0000 7f8fcefc3700 1 -- 172.21.15.37:0/1912075624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc8106a40 con 0x7f8fc8108a90 2024-06-10T00:44:46.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.250+0000 7f8fc5ffb700 1 -- 172.21.15.37:0/1912075624 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fbc01a5c0 con 0x7f8fc8108a90 2024-06-10T00:44:46.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.250+0000 7f8fcefc3700 1 -- 172.21.15.37:0/1912075624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc8106f80 con 0x7f8fc8108a90 2024-06-10T00:44:46.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.251+0000 7f8fc5ffb700 1 -- 172.21.15.37:0/1912075624 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f8fbc01a870 con 0x7f8fc8108a90 2024-06-10T00:44:46.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.251+0000 7f8fc5ffb700 1 --2- 172.21.15.37:0/1912075624 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f8fb8039000 0x7f8fb803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:46.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.251+0000 7f8fcefc3700 1 -- 172.21.15.37:0/1912075624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fc8063af0 con 0x7f8fc8108a90 2024-06-10T00:44:46.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.251+0000 7f8fc5ffb700 1 -- 172.21.15.37:0/1912075624 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8fbc04d1e0 con 0x7f8fc8108a90 2024-06-10T00:44:46.007 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.251+0000 7f8fc7fff700 1 --2- 172.21.15.37:0/1912075624 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f8fb8039000 0x7f8fb803b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:46.008 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.252+0000 7f8fc7fff700 1 --2- 172.21.15.37:0/1912075624 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f8fb8039000 0x7f8fb803b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4006fd0 tx=0x7f8fb4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:46.008 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.256+0000 7f8fc5ffb700 1 -- 172.21.15.37:0/1912075624 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8fbc019c70 con 0x7f8fc8108a90 2024-06-10T00:44:46.008 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.460+0000 7f8fcefc3700 1 -- 172.21.15.37:0/1912075624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f8fc8107260 con 0x7f8fc8108a90 2024-06-10T00:44:46.008 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.461+0000 7f8fc5ffb700 1 -- 172.21.15.37:0/1912075624 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f8fbc003b60 con 0x7f8fc8108a90 2024-06-10T00:44:46.008 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.463+0000 7f8fcefc3700 1 -- 172.21.15.37:0/1912075624 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f8fb8039000 msgr2=0x7f8fb803b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:46.008 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.463+0000 7f8fcefc3700 1 --2- 172.21.15.37:0/1912075624 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f8fb8039000 0x7f8fb803b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4006fd0 tx=0x7f8fb4006e40 comp rx=0 tx=0).stop 2024-06-10T00:44:46.008 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.463+0000 7f8fcefc3700 1 -- 172.21.15.37:0/1912075624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc8108a90 msgr2=0x7f8fc8108310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:46.008 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.463+0000 7f8fcefc3700 1 --2- 172.21.15.37:0/1912075624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc8108a90 0x7f8fc8108310 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc011670 tx=0x7f8fbc011b60 comp rx=0 tx=0).stop 2024-06-10T00:44:46.008 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.463+0000 7f8fcefc3700 1 -- 172.21.15.37:0/1912075624 shutdown_connections 2024-06-10T00:44:46.008 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.463+0000 7f8fcefc3700 1 --2- 172.21.15.37:0/1912075624 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f8fb8039000 0x7f8fb803b4c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:46.008 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.463+0000 7f8fcefc3700 1 --2- 172.21.15.37:0/1912075624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc8108a90 0x7f8fc8108310 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:46.008 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.463+0000 7f8fcefc3700 1 -- 172.21.15.37:0/1912075624 >> 172.21.15.37:0/1912075624 conn(0x7f8fc81024e0 msgr2=0x7f8fc8102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:46.008 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.463+0000 7f8fcefc3700 1 -- 172.21.15.37:0/1912075624 shutdown_connections 2024-06-10T00:44:46.009 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:45.463+0000 7f8fcefc3700 1 -- 172.21.15.37:0/1912075624 wait complete. 2024-06-10T00:44:46.009 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling cephadm module... 2024-06-10T00:44:47.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:46 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3369199360' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-06-10T00:44:47.103 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:46 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: ignoring --setuser ceph since I am not root 2024-06-10T00:44:47.104 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:46 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: ignoring --setgroup ceph since I am not root 2024-06-10T00:44:47.104 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:46 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:46.933+0000 7f7cab665200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-10T00:44:47.358 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.426+0000 7fa6748b2700 1 Processor -- start 2024-06-10T00:44:47.358 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.426+0000 7fa6748b2700 1 -- start start 2024-06-10T00:44:47.358 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.427+0000 7fa6748b2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa670107fc0 0x7fa6701083e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:47.358 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.427+0000 7fa6748b2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa6701089b0 con 0x7fa670107fc0 2024-06-10T00:44:47.359 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.427+0000 7fa66e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa670107fc0 0x7fa6701083e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:47.359 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.427+0000 7fa66e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa670107fc0 0x7fa6701083e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35904/0 (socket says 172.21.15.37:35904) 2024-06-10T00:44:47.359 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.427+0000 7fa66e59c700 1 -- 172.21.15.37:0/1754847413 learned_addr learned my addr 172.21.15.37:0/1754847413 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:44:47.359 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.427+0000 7fa66e59c700 1 -- 172.21.15.37:0/1754847413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa670109170 con 0x7fa670107fc0 2024-06-10T00:44:47.359 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.428+0000 7fa66e59c700 1 --2- 172.21.15.37:0/1754847413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa670107fc0 0x7fa6701083e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa6600098a0 tx=0x7fa660009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec57ac412eacad server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:47.359 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.428+0000 7fa66d59a700 1 -- 172.21.15.37:0/1754847413 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa660004160 con 0x7fa670107fc0 2024-06-10T00:44:47.359 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.428+0000 7fa66d59a700 1 -- 172.21.15.37:0/1754847413 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa660004300 con 0x7fa670107fc0 2024-06-10T00:44:47.359 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.428+0000 7fa66d59a700 1 -- 172.21.15.37:0/1754847413 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa660011b50 con 0x7fa670107fc0 2024-06-10T00:44:47.359 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.428+0000 7fa6748b2700 1 -- 172.21.15.37:0/1754847413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa670107fc0 msgr2=0x7fa6701083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:47.359 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.428+0000 7fa6748b2700 1 --2- 172.21.15.37:0/1754847413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa670107fc0 0x7fa6701083e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa6600098a0 tx=0x7fa660009bb0 comp rx=0 tx=0).stop 2024-06-10T00:44:47.359 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.428+0000 7fa6748b2700 1 -- 172.21.15.37:0/1754847413 shutdown_connections 2024-06-10T00:44:47.360 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.428+0000 7fa6748b2700 1 --2- 172.21.15.37:0/1754847413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa670107fc0 0x7fa6701083e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:47.360 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.428+0000 7fa6748b2700 1 -- 172.21.15.37:0/1754847413 >> 172.21.15.37:0/1754847413 conn(0x7fa6701035a0 msgr2=0x7fa6701059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:47.360 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.429+0000 7fa6748b2700 1 -- 172.21.15.37:0/1754847413 shutdown_connections 2024-06-10T00:44:47.360 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.429+0000 7fa6748b2700 1 -- 172.21.15.37:0/1754847413 wait complete. 2024-06-10T00:44:47.360 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.429+0000 7fa6748b2700 1 Processor -- start 2024-06-10T00:44:47.360 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.429+0000 7fa6748b2700 1 -- start start 2024-06-10T00:44:47.360 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.430+0000 7fa6748b2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa670107fc0 0x7fa67019bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:47.360 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.430+0000 7fa6748b2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa67010c520 con 0x7fa670107fc0 2024-06-10T00:44:47.360 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.430+0000 7fa66e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa670107fc0 0x7fa67019bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:47.360 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.430+0000 7fa66e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa670107fc0 0x7fa67019bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35918/0 (socket says 172.21.15.37:35918) 2024-06-10T00:44:47.360 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.430+0000 7fa66e59c700 1 -- 172.21.15.37:0/3369199360 learned_addr learned my addr 172.21.15.37:0/3369199360 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:44:47.360 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.430+0000 7fa66e59c700 1 -- 172.21.15.37:0/3369199360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa67019c460 con 0x7fa670107fc0 2024-06-10T00:44:47.360 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.430+0000 7fa66e59c700 1 --2- 172.21.15.37:0/3369199360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa670107fc0 0x7fa67019bf20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa660000c00 tx=0x7fa660011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:47.361 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.430+0000 7fa6677fe700 1 -- 172.21.15.37:0/3369199360 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa660003b20 con 0x7fa670107fc0 2024-06-10T00:44:47.361 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.430+0000 7fa6677fe700 1 -- 172.21.15.37:0/3369199360 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa660003cc0 con 0x7fa670107fc0 2024-06-10T00:44:47.361 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.430+0000 7fa6748b2700 1 -- 172.21.15.37:0/3369199360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa67019c6f0 con 0x7fa670107fc0 2024-06-10T00:44:47.361 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.430+0000 7fa6677fe700 1 -- 172.21.15.37:0/3369199360 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa660024910 con 0x7fa670107fc0 2024-06-10T00:44:47.361 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.431+0000 7fa6748b2700 1 -- 172.21.15.37:0/3369199360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa67019f3e0 con 0x7fa670107fc0 2024-06-10T00:44:47.361 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.431+0000 7fa6677fe700 1 -- 172.21.15.37:0/3369199360 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fa660022020 con 0x7fa670107fc0 2024-06-10T00:44:47.361 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.431+0000 7fa6748b2700 1 -- 172.21.15.37:0/3369199360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa670063af0 con 0x7fa670107fc0 2024-06-10T00:44:47.361 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.432+0000 7fa6677fe700 1 --2- 172.21.15.37:0/3369199360 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7fa65c039050 0x7fa65c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:47.361 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.432+0000 7fa6677fe700 1 -- 172.21.15.37:0/3369199360 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa660011e00 con 0x7fa670107fc0 2024-06-10T00:44:47.361 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.432+0000 7fa66dd9b700 1 --2- 172.21.15.37:0/3369199360 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7fa65c039050 0x7fa65c03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:47.361 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.432+0000 7fa66dd9b700 1 --2- 172.21.15.37:0/3369199360 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7fa65c039050 0x7fa65c03b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa658006fd0 tx=0x7fa658006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:47.361 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.437+0000 7fa6677fe700 1 -- 172.21.15.37:0/3369199360 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa66000b3e0 con 0x7fa670107fc0 2024-06-10T00:44:47.362 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.707+0000 7fa6748b2700 1 -- 172.21.15.37:0/3369199360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fa67019f5b0 con 0x7fa670107fc0 2024-06-10T00:44:47.362 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.759+0000 7fa6677fe700 1 -- 172.21.15.37:0/3369199360 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fa66002cda0 con 0x7fa670107fc0 2024-06-10T00:44:47.362 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.760+0000 7fa6677fe700 1 -- 172.21.15.37:0/3369199360 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fa66001acd0 con 0x7fa670107fc0 2024-06-10T00:44:47.362 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.762+0000 7fa6748b2700 1 -- 172.21.15.37:0/3369199360 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7fa65c039050 msgr2=0x7fa65c03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:47.362 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.762+0000 7fa6748b2700 1 --2- 172.21.15.37:0/3369199360 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7fa65c039050 0x7fa65c03b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa658006fd0 tx=0x7fa658006e40 comp rx=0 tx=0).stop 2024-06-10T00:44:47.362 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.762+0000 7fa6748b2700 1 -- 172.21.15.37:0/3369199360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa670107fc0 msgr2=0x7fa67019bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:47.362 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.762+0000 7fa6748b2700 1 --2- 172.21.15.37:0/3369199360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa670107fc0 0x7fa67019bf20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa660000c00 tx=0x7fa660011c10 comp rx=0 tx=0).stop 2024-06-10T00:44:47.362 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.762+0000 7fa6748b2700 1 -- 172.21.15.37:0/3369199360 shutdown_connections 2024-06-10T00:44:47.362 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.762+0000 7fa6748b2700 1 --2- 172.21.15.37:0/3369199360 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7fa65c039050 0x7fa65c03b510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:47.362 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.762+0000 7fa6748b2700 1 --2- 172.21.15.37:0/3369199360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa670107fc0 0x7fa67019bf20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:47.362 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.762+0000 7fa6748b2700 1 -- 172.21.15.37:0/3369199360 >> 172.21.15.37:0/3369199360 conn(0x7fa6701035a0 msgr2=0x7fa670104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:47.362 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.762+0000 7fa6748b2700 1 -- 172.21.15.37:0/3369199360 shutdown_connections 2024-06-10T00:44:47.363 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:46.762+0000 7fa6748b2700 1 -- 172.21.15.37:0/3369199360 wait complete. 2024-06-10T00:44:47.395 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:47 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:47.130+0000 7f7cab665200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:44:47.395 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:47 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:47.238+0000 7f7cab665200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-10T00:44:47.763 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:47 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:47.573+0000 7f7cab665200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-10T00:44:47.764 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:47 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:47.666+0000 7f7cab665200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-10T00:44:48.108 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:47 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3369199360' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-06-10T00:44:48.108 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:47 smithi037 ceph-mon[32192]: mgrmap e5: x(active, since 6s) 2024-06-10T00:44:48.639 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-06-10T00:44:48.640 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-06-10T00:44:48.640 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-10T00:44:48.640 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-10T00:44:48.640 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-10T00:44:48.640 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-06-10T00:44:48.640 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.797+0000 7f94fb9c8700 1 Processor -- start 2024-06-10T00:44:48.640 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.797+0000 7f94fb9c8700 1 -- start start 2024-06-10T00:44:48.640 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.798+0000 7f94fb9c8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94f4108430 0x7f94f4108850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:48.640 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.798+0000 7f94fb9c8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f94f4108e20 con 0x7f94f4108430 2024-06-10T00:44:48.640 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.798+0000 7f94f9764700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94f4108430 0x7f94f4108850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:48.640 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.798+0000 7f94f9764700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94f4108430 0x7f94f4108850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35936/0 (socket says 172.21.15.37:35936) 2024-06-10T00:44:48.640 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.798+0000 7f94f9764700 1 -- 172.21.15.37:0/3086891218 learned_addr learned my addr 172.21.15.37:0/3086891218 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:44:48.640 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.798+0000 7f94f9764700 1 -- 172.21.15.37:0/3086891218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f4107090 con 0x7f94f4108430 2024-06-10T00:44:48.641 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.798+0000 7f94f9764700 1 --2- 172.21.15.37:0/3086891218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94f4108430 0x7f94f4108850 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f94f00098a0 tx=0x7f94f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b6477d5b46980b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:48.641 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.798+0000 7f94e7fff700 1 -- 172.21.15.37:0/3086891218 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94f0004160 con 0x7f94f4108430 2024-06-10T00:44:48.641 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.798+0000 7f94e7fff700 1 -- 172.21.15.37:0/3086891218 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f94f0004300 con 0x7f94f4108430 2024-06-10T00:44:48.641 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.798+0000 7f94e7fff700 1 -- 172.21.15.37:0/3086891218 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94f0011b50 con 0x7f94f4108430 2024-06-10T00:44:48.641 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.799+0000 7f94fb9c8700 1 -- 172.21.15.37:0/3086891218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94f4108430 msgr2=0x7f94f4108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:48.641 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.799+0000 7f94fb9c8700 1 --2- 172.21.15.37:0/3086891218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94f4108430 0x7f94f4108850 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f94f00098a0 tx=0x7f94f0009bb0 comp rx=0 tx=0).stop 2024-06-10T00:44:48.641 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.799+0000 7f94fb9c8700 1 -- 172.21.15.37:0/3086891218 shutdown_connections 2024-06-10T00:44:48.641 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.799+0000 7f94fb9c8700 1 --2- 172.21.15.37:0/3086891218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94f4108430 0x7f94f4108850 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:48.641 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.799+0000 7f94fb9c8700 1 -- 172.21.15.37:0/3086891218 >> 172.21.15.37:0/3086891218 conn(0x7f94f4101b80 msgr2=0x7f94f4103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:48.641 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.799+0000 7f94fb9c8700 1 -- 172.21.15.37:0/3086891218 shutdown_connections 2024-06-10T00:44:48.641 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.799+0000 7f94fb9c8700 1 -- 172.21.15.37:0/3086891218 wait complete. 2024-06-10T00:44:48.641 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.800+0000 7f94fb9c8700 1 Processor -- start 2024-06-10T00:44:48.641 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.800+0000 7f94fb9c8700 1 -- start start 2024-06-10T00:44:48.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.800+0000 7f94fb9c8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94f4108430 0x7f94f419e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:48.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.800+0000 7f94fb9c8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f94f4105f80 con 0x7f94f4108430 2024-06-10T00:44:48.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.800+0000 7f94f9764700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94f4108430 0x7f94f419e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:48.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.800+0000 7f94f9764700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94f4108430 0x7f94f419e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35950/0 (socket says 172.21.15.37:35950) 2024-06-10T00:44:48.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.800+0000 7f94f9764700 1 -- 172.21.15.37:0/624154717 learned_addr learned my addr 172.21.15.37:0/624154717 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:44:48.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.800+0000 7f94f9764700 1 -- 172.21.15.37:0/624154717 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f419e6b0 con 0x7f94f4108430 2024-06-10T00:44:48.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.801+0000 7f94f9764700 1 --2- 172.21.15.37:0/624154717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94f4108430 0x7f94f419e170 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f94f0011670 tx=0x7f94f0011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:48.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.801+0000 7f94e67fc700 1 -- 172.21.15.37:0/624154717 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94f00039c0 con 0x7f94f4108430 2024-06-10T00:44:48.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.801+0000 7f94e67fc700 1 -- 172.21.15.37:0/624154717 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f94f0011e50 con 0x7f94f4108430 2024-06-10T00:44:48.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.801+0000 7f94fb9c8700 1 -- 172.21.15.37:0/624154717 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94f419e940 con 0x7f94f4108430 2024-06-10T00:44:48.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.801+0000 7f94e67fc700 1 -- 172.21.15.37:0/624154717 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94f001a5c0 con 0x7f94f4108430 2024-06-10T00:44:48.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.801+0000 7f94fb9c8700 1 -- 172.21.15.37:0/624154717 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94f41a15e0 con 0x7f94f4108430 2024-06-10T00:44:48.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.802+0000 7f94fb9c8700 1 -- 172.21.15.37:0/624154717 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94f4063af0 con 0x7f94f4108430 2024-06-10T00:44:48.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.802+0000 7f94e67fc700 1 -- 172.21.15.37:0/624154717 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f94f001a870 con 0x7f94f4108430 2024-06-10T00:44:48.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.802+0000 7f94e67fc700 1 --2- 172.21.15.37:0/624154717 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f94e0039050 0x7f94e003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:48.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.802+0000 7f94e67fc700 1 -- 172.21.15.37:0/624154717 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f94f004d410 con 0x7f94f4108430 2024-06-10T00:44:48.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.802+0000 7f94f8f63700 1 -- 172.21.15.37:0/624154717 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f94e0039050 msgr2=0x7f94e003b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2803174880 2024-06-10T00:44:48.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.802+0000 7f94f8f63700 1 --2- 172.21.15.37:0/624154717 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f94e0039050 0x7f94e003b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T00:44:48.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:47.807+0000 7f94e67fc700 1 -- 172.21.15.37:0/624154717 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f94f0019c70 con 0x7f94f4108430 2024-06-10T00:44:48.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:48.003+0000 7f94f8f63700 1 -- 172.21.15.37:0/624154717 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f94e0039050 msgr2=0x7f94e003b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2803174880 2024-06-10T00:44:48.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:48.003+0000 7f94f8f63700 1 --2- 172.21.15.37:0/624154717 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f94e0039050 0x7f94e003b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-10T00:44:48.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:48.107+0000 7f94fb9c8700 1 -- 172.21.15.37:0/624154717 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f94f41932c0 con 0x7f94f4108430 2024-06-10T00:44:48.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:48.108+0000 7f94e67fc700 1 -- 172.21.15.37:0/624154717 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f94f41932c0 con 0x7f94f4108430 2024-06-10T00:44:48.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:48.112+0000 7f94fb9c8700 1 -- 172.21.15.37:0/624154717 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f94e0039050 msgr2=0x7f94e003b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T00:44:48.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:48.112+0000 7f94fb9c8700 1 --2- 172.21.15.37:0/624154717 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f94e0039050 0x7f94e003b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:48.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:48.112+0000 7f94fb9c8700 1 -- 172.21.15.37:0/624154717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94f4108430 msgr2=0x7f94f419e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:48.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:48.112+0000 7f94fb9c8700 1 --2- 172.21.15.37:0/624154717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94f4108430 0x7f94f419e170 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f94f0011670 tx=0x7f94f0011b60 comp rx=0 tx=0).stop 2024-06-10T00:44:48.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:48.112+0000 7f94fb9c8700 1 -- 172.21.15.37:0/624154717 shutdown_connections 2024-06-10T00:44:48.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:48.112+0000 7f94fb9c8700 1 --2- 172.21.15.37:0/624154717 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f94e0039050 0x7f94e003b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:48.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:48.112+0000 7f94fb9c8700 1 --2- 172.21.15.37:0/624154717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94f4108430 0x7f94f419e170 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f94f0011670 tx=0x7f94f0011b60 comp rx=0 tx=0).stop 2024-06-10T00:44:48.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:48.112+0000 7f94fb9c8700 1 -- 172.21.15.37:0/624154717 >> 172.21.15.37:0/624154717 conn(0x7f94f4101b80 msgr2=0x7f94f4190380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:48.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:48.113+0000 7f94fb9c8700 1 -- 172.21.15.37:0/624154717 shutdown_connections 2024-06-10T00:44:48.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:48.113+0000 7f94fb9c8700 1 -- 172.21.15.37:0/624154717 wait complete. 2024-06-10T00:44:48.644 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for the mgr to restart... 2024-06-10T00:44:48.644 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr epoch 5... 2024-06-10T00:44:49.018 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:48 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/624154717' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-10T00:44:49.018 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:48 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:48.935+0000 7f7cab665200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-10T00:44:49.295 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:49 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:49.117+0000 7f7cab665200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-10T00:44:49.295 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:49 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:49.294+0000 7f7cab665200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-10T00:44:49.895 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:49 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:49.596+0000 7f7cab665200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-10T00:44:49.896 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:49 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:49.693+0000 7f7cab665200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-10T00:44:50.395 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:49 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:49.903+0000 7f7cab665200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-10T00:44:50.412 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:50 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:50.105+0000 7f7cab665200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:44:51.145 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:50 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:50.737+0000 7f7cab665200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-10T00:44:51.489 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:51 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:51.178+0000 7f7cab665200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-10T00:44:51.490 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:51 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:51.280+0000 7f7cab665200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-10T00:44:51.490 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:51 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:51.489+0000 7f7cab665200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-10T00:44:52.385 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:52 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:52.035+0000 7f7cab665200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-10T00:44:52.386 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:52 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:52.131+0000 7f7cab665200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-10T00:44:52.645 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:52 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:52.385+0000 7f7cab665200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-10T00:44:52.646 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:52 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:52.524+0000 7f7cab665200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-10T00:44:52.646 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:52 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:52.624+0000 7f7cab665200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-10T00:44:53.056 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:52 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:52.744+0000 7f7cab665200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-10T00:44:53.056 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:53 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:53.055+0000 7f7cab665200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-10T00:44:53.965 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:53 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:53.686+0000 7f7cab665200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-10T00:44:53.965 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:53 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:44:53.912+0000 7f7cab665200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-10T00:44:54.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:53 smithi037 ceph-mon[32192]: Active manager daemon x restarted 2024-06-10T00:44:54.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:53 smithi037 ceph-mon[32192]: Activating manager daemon x 2024-06-10T00:44:54.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:53 smithi037 ceph-mon[32192]: osdmap e2: 0 total, 0 up, 0 in 2024-06-10T00:44:54.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:53 smithi037 ceph-mon[32192]: mgrmap e6: x(active, starting, since 0.00349822s) 2024-06-10T00:44:54.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:53 smithi037 ceph-mon[32192]: from='mgr.14116 172.21.15.37:0/4158559188' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:44:54.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:53 smithi037 ceph-mon[32192]: from='mgr.14116 172.21.15.37:0/4158559188' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-10T00:44:54.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:53 smithi037 ceph-mon[32192]: from='mgr.14116 172.21.15.37:0/4158559188' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-10T00:44:54.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:53 smithi037 ceph-mon[32192]: from='mgr.14116 172.21.15.37:0/4158559188' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-10T00:44:54.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:53 smithi037 ceph-mon[32192]: from='mgr.14116 172.21.15.37:0/4158559188' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-10T00:44:54.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:53 smithi037 ceph-mon[32192]: Manager daemon x is now available 2024-06-10T00:44:54.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:53 smithi037 ceph-mon[32192]: from='mgr.14116 172.21.15.37:0/4158559188' entity='mgr.x' 2024-06-10T00:44:54.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:53 smithi037 ceph-mon[32192]: from='mgr.14116 172.21.15.37:0/4158559188' entity='mgr.x' 2024-06-10T00:44:54.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:53 smithi037 ceph-mon[32192]: from='mgr.14116 172.21.15.37:0/4158559188' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:44:54.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:53 smithi037 ceph-mon[32192]: from='mgr.14116 172.21.15.37:0/4158559188' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:44:54.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:53 smithi037 ceph-mon[32192]: from='mgr.14116 172.21.15.37:0/4158559188' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-10T00:44:55.220 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:54 smithi037 ceph-mon[32192]: Found migration_current of "None". Setting to last migration. 2024-06-10T00:44:55.220 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:54 smithi037 ceph-mon[32192]: from='mgr.14116 172.21.15.37:0/4158559188' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-10T00:44:55.220 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:54 smithi037 ceph-mon[32192]: mgrmap e7: x(active, since 1.00561s) 2024-06-10T00:44:55.505 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-06-10T00:44:55.505 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-06-10T00:44:55.505 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-10T00:44:55.506 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-06-10T00:44:55.506 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.046+0000 7f9623fc1700 1 Processor -- start 2024-06-10T00:44:55.506 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.046+0000 7f9623fc1700 1 -- start start 2024-06-10T00:44:55.506 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.046+0000 7f9623fc1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f961c106e50 0x7f961c107270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:55.506 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.046+0000 7f9623fc1700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f961c1077b0 con 0x7f961c106e50 2024-06-10T00:44:55.506 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.046+0000 7f9621d5d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f961c106e50 0x7f961c107270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:55.506 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.046+0000 7f9621d5d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f961c106e50 0x7f961c107270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35962/0 (socket says 172.21.15.37:35962) 2024-06-10T00:44:55.506 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.046+0000 7f9621d5d700 1 -- 172.21.15.37:0/2603509517 learned_addr learned my addr 172.21.15.37:0/2603509517 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:44:55.506 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.047+0000 7f9621d5d700 1 -- 172.21.15.37:0/2603509517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961c108040 con 0x7f961c106e50 2024-06-10T00:44:55.506 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.047+0000 7f9621d5d700 1 --2- 172.21.15.37:0/2603509517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f961c106e50 0x7f961c107270 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f961800bb40 tx=0x7f961800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3f2b1ba9559cec server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:55.506 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.047+0000 7f9620d5b700 1 -- 172.21.15.37:0/2603509517 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f961800dbe0 con 0x7f961c106e50 2024-06-10T00:44:55.507 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.047+0000 7f9620d5b700 1 -- 172.21.15.37:0/2603509517 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f96180044f0 con 0x7f961c106e50 2024-06-10T00:44:55.507 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.047+0000 7f9620d5b700 1 -- 172.21.15.37:0/2603509517 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9618010470 con 0x7f961c106e50 2024-06-10T00:44:55.507 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.048+0000 7f9623fc1700 1 -- 172.21.15.37:0/2603509517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f961c106e50 msgr2=0x7f961c107270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:55.507 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.048+0000 7f9623fc1700 1 --2- 172.21.15.37:0/2603509517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f961c106e50 0x7f961c107270 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f961800bb40 tx=0x7f961800be50 comp rx=0 tx=0).stop 2024-06-10T00:44:55.507 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.048+0000 7f9623fc1700 1 -- 172.21.15.37:0/2603509517 shutdown_connections 2024-06-10T00:44:55.507 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.048+0000 7f9623fc1700 1 --2- 172.21.15.37:0/2603509517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f961c106e50 0x7f961c107270 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:55.507 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.048+0000 7f9623fc1700 1 -- 172.21.15.37:0/2603509517 >> 172.21.15.37:0/2603509517 conn(0x7f961c1024e0 msgr2=0x7f961c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:55.507 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.048+0000 7f9623fc1700 1 -- 172.21.15.37:0/2603509517 shutdown_connections 2024-06-10T00:44:55.507 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.048+0000 7f9623fc1700 1 -- 172.21.15.37:0/2603509517 wait complete. 2024-06-10T00:44:55.507 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.049+0000 7f9623fc1700 1 Processor -- start 2024-06-10T00:44:55.507 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.049+0000 7f9623fc1700 1 -- start start 2024-06-10T00:44:55.507 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.049+0000 7f9623fc1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f961c106e50 0x7f961c199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:55.507 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.049+0000 7f9623fc1700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f961c108ed0 con 0x7f961c106e50 2024-06-10T00:44:55.508 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.049+0000 7f9621d5d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f961c106e50 0x7f961c199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:55.508 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.049+0000 7f9621d5d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f961c106e50 0x7f961c199d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35968/0 (socket says 172.21.15.37:35968) 2024-06-10T00:44:55.508 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.049+0000 7f9621d5d700 1 -- 172.21.15.37:0/2402952154 learned_addr learned my addr 172.21.15.37:0/2402952154 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:44:55.508 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.049+0000 7f9621d5d700 1 -- 172.21.15.37:0/2402952154 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961c19a2d0 con 0x7f961c106e50 2024-06-10T00:44:55.508 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.049+0000 7f9621d5d700 1 --2- 172.21.15.37:0/2402952154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f961c106e50 0x7f961c199d90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f961800b340 tx=0x7f96180040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:55.508 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.050+0000 7f9612ffd700 1 -- 172.21.15.37:0/2402952154 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9618027070 con 0x7f961c106e50 2024-06-10T00:44:55.508 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.050+0000 7f9612ffd700 1 -- 172.21.15.37:0/2402952154 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9618010be0 con 0x7f961c106e50 2024-06-10T00:44:55.508 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.050+0000 7f9623fc1700 1 -- 172.21.15.37:0/2402952154 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f961c19a560 con 0x7f961c106e50 2024-06-10T00:44:55.508 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.050+0000 7f9612ffd700 1 -- 172.21.15.37:0/2402952154 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9618022c40 con 0x7f961c106e50 2024-06-10T00:44:55.508 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.050+0000 7f9623fc1700 1 -- 172.21.15.37:0/2402952154 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f961c19d210 con 0x7f961c106e50 2024-06-10T00:44:55.508 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.051+0000 7f9612ffd700 1 -- 172.21.15.37:0/2402952154 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f961802a070 con 0x7f961c106e50 2024-06-10T00:44:55.509 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.051+0000 7f9612ffd700 1 --2- 172.21.15.37:0/2402952154 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f9608039050 0x7f960803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:55.509 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.051+0000 7f9612ffd700 1 -- 172.21.15.37:0/2402952154 --> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f960803bc20 con 0x7f9608039050 2024-06-10T00:44:55.509 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.051+0000 7f9612ffd700 1 -- 172.21.15.37:0/2402952154 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f961801f070 con 0x7f961c106e50 2024-06-10T00:44:55.509 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.051+0000 7f962155c700 1 -- 172.21.15.37:0/2402952154 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f9608039050 msgr2=0x7f960803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2803174880 2024-06-10T00:44:55.509 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.051+0000 7f962155c700 1 --2- 172.21.15.37:0/2402952154 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f9608039050 0x7f960803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T00:44:55.510 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.252+0000 7f962155c700 1 -- 172.21.15.37:0/2402952154 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f9608039050 msgr2=0x7f960803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2803174880 2024-06-10T00:44:55.510 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.252+0000 7f962155c700 1 --2- 172.21.15.37:0/2402952154 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f9608039050 0x7f960803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-10T00:44:55.510 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.653+0000 7f962155c700 1 -- 172.21.15.37:0/2402952154 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f9608039050 msgr2=0x7f960803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2803174880 2024-06-10T00:44:55.510 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:49.653+0000 7f962155c700 1 --2- 172.21.15.37:0/2402952154 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f9608039050 0x7f960803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-10T00:44:55.510 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:50.454+0000 7f962155c700 1 -- 172.21.15.37:0/2402952154 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f9608039050 msgr2=0x7f960803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2803174880 2024-06-10T00:44:55.510 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:50.454+0000 7f962155c700 1 --2- 172.21.15.37:0/2402952154 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f9608039050 0x7f960803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-10T00:44:55.511 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:52.055+0000 7f962155c700 1 -- 172.21.15.37:0/2402952154 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f9608039050 msgr2=0x7f960803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2803174880 2024-06-10T00:44:55.511 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:52.055+0000 7f962155c700 1 --2- 172.21.15.37:0/2402952154 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f9608039050 0x7f960803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-10T00:44:55.511 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:53.915+0000 7f9612ffd700 1 -- 172.21.15.37:0/2402952154 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f961802cce0 con 0x7f961c106e50 2024-06-10T00:44:55.511 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:53.915+0000 7f9612ffd700 1 -- 172.21.15.37:0/2402952154 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f9608039050 msgr2=0x7f960803b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T00:44:55.511 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:53.915+0000 7f9612ffd700 1 --2- 172.21.15.37:0/2402952154 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f9608039050 0x7f960803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:55.511 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:54.916+0000 7f9612ffd700 1 -- 172.21.15.37:0/2402952154 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f961800fd70 con 0x7f961c106e50 2024-06-10T00:44:55.511 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:54.916+0000 7f9612ffd700 1 --2- 172.21.15.37:0/2402952154 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f960803c980 0x7f960803ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:55.511 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:54.916+0000 7f9612ffd700 1 -- 172.21.15.37:0/2402952154 --> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f961804eff0 con 0x7f960803c980 2024-06-10T00:44:55.511 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:54.917+0000 7f962155c700 1 --2- 172.21.15.37:0/2402952154 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f960803c980 0x7f960803ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:55.511 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:54.917+0000 7f962155c700 1 --2- 172.21.15.37:0/2402952154 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f960803c980 0x7f960803ed70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f960c003a10 tx=0x7f960c0092b0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:55.511 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:54.918+0000 7f9612ffd700 1 -- 172.21.15.37:0/2402952154 <== mgr.14116 v2:172.21.15.37:6800/2766692263 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f961804eff0 con 0x7f960803c980 2024-06-10T00:44:55.511 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:54.925+0000 7f9623fc1700 1 -- 172.21.15.37:0/2402952154 --> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f961c19aa40 con 0x7f960803c980 2024-06-10T00:44:55.512 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:54.925+0000 7f9612ffd700 1 -- 172.21.15.37:0/2402952154 <== mgr.14116 v2:172.21.15.37:6800/2766692263 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f961c19aa40 con 0x7f960803c980 2024-06-10T00:44:55.512 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:54.925+0000 7f9623fc1700 1 -- 172.21.15.37:0/2402952154 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f960803c980 msgr2=0x7f960803ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:55.512 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:54.925+0000 7f9623fc1700 1 --2- 172.21.15.37:0/2402952154 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f960803c980 0x7f960803ed70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f960c003a10 tx=0x7f960c0092b0 comp rx=0 tx=0).stop 2024-06-10T00:44:55.512 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:54.925+0000 7f9623fc1700 1 -- 172.21.15.37:0/2402952154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f961c106e50 msgr2=0x7f961c199d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:55.512 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:54.925+0000 7f9623fc1700 1 --2- 172.21.15.37:0/2402952154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f961c106e50 0x7f961c199d90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f961800b340 tx=0x7f96180040b0 comp rx=0 tx=0).stop 2024-06-10T00:44:55.512 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:54.925+0000 7f9623fc1700 1 -- 172.21.15.37:0/2402952154 shutdown_connections 2024-06-10T00:44:55.512 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:54.925+0000 7f9623fc1700 1 --2- 172.21.15.37:0/2402952154 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f960803c980 0x7f960803ed70 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:55.512 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:54.925+0000 7f9623fc1700 1 --2- 172.21.15.37:0/2402952154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f961c106e50 0x7f961c199d90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:55.512 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:54.925+0000 7f9623fc1700 1 --2- 172.21.15.37:0/2402952154 >> [v2:172.21.15.37:6800/2803174880,v1:172.21.15.37:6801/2803174880] conn(0x7f9608039050 0x7f960803b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:55.512 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:54.925+0000 7f9623fc1700 1 -- 172.21.15.37:0/2402952154 >> 172.21.15.37:0/2402952154 conn(0x7f961c1024e0 msgr2=0x7f961c1902f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:55.512 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:54.925+0000 7f9623fc1700 1 -- 172.21.15.37:0/2402952154 shutdown_connections 2024-06-10T00:44:55.512 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:54.925+0000 7f9623fc1700 1 -- 172.21.15.37:0/2402952154 wait complete. 2024-06-10T00:44:55.512 INFO:teuthology.orchestra.run.smithi037.stdout:mgr epoch 5 is available 2024-06-10T00:44:55.513 INFO:teuthology.orchestra.run.smithi037.stdout:Setting orchestrator backend to cephadm... 2024-06-10T00:44:56.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:56 smithi037 ceph-mon[32192]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-10T00:44:56.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:56 smithi037 ceph-mon[32192]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-10T00:44:56.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:56 smithi037 ceph-mon[32192]: from='mgr.14116 172.21.15.37:0/4158559188' entity='mgr.x' 2024-06-10T00:44:56.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:56 smithi037 ceph-mon[32192]: from='mgr.14116 172.21.15.37:0/4158559188' entity='mgr.x' 2024-06-10T00:44:56.727 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.917+0000 7f3f95573700 1 Processor -- start 2024-06-10T00:44:56.727 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.917+0000 7f3f95573700 1 -- start start 2024-06-10T00:44:56.727 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.917+0000 7f3f95573700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f901062b0 0x7f3f901066d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:56.727 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.917+0000 7f3f95573700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3f90106ca0 con 0x7f3f901062b0 2024-06-10T00:44:56.727 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.917+0000 7f3f8effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f901062b0 0x7f3f901066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:56.728 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.918+0000 7f3f8effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f901062b0 0x7f3f901066d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43550/0 (socket says 172.21.15.37:43550) 2024-06-10T00:44:56.728 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.918+0000 7f3f8effd700 1 -- 172.21.15.37:0/4258255514 learned_addr learned my addr 172.21.15.37:0/4258255514 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:44:56.728 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.918+0000 7f3f8effd700 1 -- 172.21.15.37:0/4258255514 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f90109240 con 0x7f3f901062b0 2024-06-10T00:44:56.728 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.918+0000 7f3f8effd700 1 --2- 172.21.15.37:0/4258255514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f901062b0 0x7f3f901066d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3f80009e80 tx=0x7f3f8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51f5286024bc76c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:56.728 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.918+0000 7f3f8dffb700 1 -- 172.21.15.37:0/4258255514 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f80017070 con 0x7f3f901062b0 2024-06-10T00:44:56.728 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.918+0000 7f3f8dffb700 1 -- 172.21.15.37:0/4258255514 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3f8000b8c0 con 0x7f3f901062b0 2024-06-10T00:44:56.728 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.918+0000 7f3f8dffb700 1 -- 172.21.15.37:0/4258255514 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f800045c0 con 0x7f3f901062b0 2024-06-10T00:44:56.728 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.919+0000 7f3f95573700 1 -- 172.21.15.37:0/4258255514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f901062b0 msgr2=0x7f3f901066d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:56.728 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.919+0000 7f3f95573700 1 --2- 172.21.15.37:0/4258255514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f901062b0 0x7f3f901066d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3f80009e80 tx=0x7f3f8000b330 comp rx=0 tx=0).stop 2024-06-10T00:44:56.728 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.919+0000 7f3f95573700 1 -- 172.21.15.37:0/4258255514 shutdown_connections 2024-06-10T00:44:56.728 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.919+0000 7f3f95573700 1 --2- 172.21.15.37:0/4258255514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f901062b0 0x7f3f901066d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:56.729 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.919+0000 7f3f95573700 1 -- 172.21.15.37:0/4258255514 >> 172.21.15.37:0/4258255514 conn(0x7f3f90101c00 msgr2=0x7f3f90104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:56.729 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.919+0000 7f3f95573700 1 -- 172.21.15.37:0/4258255514 shutdown_connections 2024-06-10T00:44:56.729 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.919+0000 7f3f95573700 1 -- 172.21.15.37:0/4258255514 wait complete. 2024-06-10T00:44:56.729 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.919+0000 7f3f95573700 1 Processor -- start 2024-06-10T00:44:56.729 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.920+0000 7f3f95573700 1 -- start start 2024-06-10T00:44:56.729 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.920+0000 7f3f95573700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f901062b0 0x7f3f90108ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:56.729 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.920+0000 7f3f95573700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9010c5f0 con 0x7f3f901062b0 2024-06-10T00:44:56.729 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.920+0000 7f3f8effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f901062b0 0x7f3f90108ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:56.729 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.920+0000 7f3f8effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f901062b0 0x7f3f90108ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43558/0 (socket says 172.21.15.37:43558) 2024-06-10T00:44:56.729 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.920+0000 7f3f8effd700 1 -- 172.21.15.37:0/1288284559 learned_addr learned my addr 172.21.15.37:0/1288284559 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:44:56.729 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.920+0000 7f3f8effd700 1 -- 172.21.15.37:0/1288284559 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f90109000 con 0x7f3f901062b0 2024-06-10T00:44:56.729 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.920+0000 7f3f8effd700 1 --2- 172.21.15.37:0/1288284559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f901062b0 0x7f3f90108ac0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3f800072a0 tx=0x7f3f8000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:56.729 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.920+0000 7f3f87fff700 1 -- 172.21.15.37:0/1288284559 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f80020020 con 0x7f3f901062b0 2024-06-10T00:44:56.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.920+0000 7f3f87fff700 1 -- 172.21.15.37:0/1288284559 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3f80003b50 con 0x7f3f901062b0 2024-06-10T00:44:56.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.920+0000 7f3f87fff700 1 -- 172.21.15.37:0/1288284559 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f80017040 con 0x7f3f901062b0 2024-06-10T00:44:56.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.921+0000 7f3f95573700 1 -- 172.21.15.37:0/1288284559 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f901071f0 con 0x7f3f901062b0 2024-06-10T00:44:56.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.921+0000 7f3f95573700 1 -- 172.21.15.37:0/1288284559 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f90107730 con 0x7f3f901062b0 2024-06-10T00:44:56.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.921+0000 7f3f87fff700 1 -- 172.21.15.37:0/1288284559 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f3f80023020 con 0x7f3f901062b0 2024-06-10T00:44:56.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.922+0000 7f3f95573700 1 -- 172.21.15.37:0/1288284559 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f90063af0 con 0x7f3f901062b0 2024-06-10T00:44:56.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.922+0000 7f3f87fff700 1 --2- 172.21.15.37:0/1288284559 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f3f7c038ed0 0x7f3f7c03b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:56.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.922+0000 7f3f87fff700 1 -- 172.21.15.37:0/1288284559 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3f8000f050 con 0x7f3f901062b0 2024-06-10T00:44:56.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.922+0000 7f3f8e7fc700 1 --2- 172.21.15.37:0/1288284559 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f3f7c038ed0 0x7f3f7c03b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:56.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.922+0000 7f3f8e7fc700 1 --2- 172.21.15.37:0/1288284559 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f3f7c038ed0 0x7f3f7c03b390 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3f78006fd0 tx=0x7f3f78006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:56.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:55.927+0000 7f3f87fff700 1 -- 172.21.15.37:0/1288284559 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3f8002a020 con 0x7f3f901062b0 2024-06-10T00:44:56.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:56.024+0000 7f3f87fff700 1 -- 172.21.15.37:0/1288284559 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f3f80012980 con 0x7f3f901062b0 2024-06-10T00:44:56.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:56.168+0000 7f3f95573700 1 -- 172.21.15.37:0/1288284559 --> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f3f90102e00 con 0x7f3f7c038ed0 2024-06-10T00:44:56.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:56.175+0000 7f3f87fff700 1 -- 172.21.15.37:0/1288284559 <== mgr.14116 v2:172.21.15.37:6800/2766692263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3f90102e00 con 0x7f3f7c038ed0 2024-06-10T00:44:56.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:56.178+0000 7f3f95573700 1 -- 172.21.15.37:0/1288284559 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f3f7c038ed0 msgr2=0x7f3f7c03b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:56.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:56.178+0000 7f3f95573700 1 --2- 172.21.15.37:0/1288284559 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f3f7c038ed0 0x7f3f7c03b390 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3f78006fd0 tx=0x7f3f78006e40 comp rx=0 tx=0).stop 2024-06-10T00:44:56.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:56.178+0000 7f3f95573700 1 -- 172.21.15.37:0/1288284559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f901062b0 msgr2=0x7f3f90108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:56.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:56.178+0000 7f3f95573700 1 --2- 172.21.15.37:0/1288284559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f901062b0 0x7f3f90108ac0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3f800072a0 tx=0x7f3f8000bd50 comp rx=0 tx=0).stop 2024-06-10T00:44:56.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:56.178+0000 7f3f95573700 1 -- 172.21.15.37:0/1288284559 shutdown_connections 2024-06-10T00:44:56.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:56.178+0000 7f3f95573700 1 --2- 172.21.15.37:0/1288284559 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f3f7c038ed0 0x7f3f7c03b390 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:56.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:56.178+0000 7f3f95573700 1 --2- 172.21.15.37:0/1288284559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f901062b0 0x7f3f90108ac0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:56.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:56.178+0000 7f3f95573700 1 -- 172.21.15.37:0/1288284559 >> 172.21.15.37:0/1288284559 conn(0x7f3f90101c00 msgr2=0x7f3f901026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:56.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:56.178+0000 7f3f95573700 1 -- 172.21.15.37:0/1288284559 shutdown_connections 2024-06-10T00:44:56.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:56.178+0000 7f3f95573700 1 -- 172.21.15.37:0/1288284559 wait complete. 2024-06-10T00:44:57.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:57 smithi037 ceph-mon[32192]: mgrmap e8: x(active, since 2s) 2024-06-10T00:44:57.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:57 smithi037 ceph-mon[32192]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:44:57.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:57 smithi037 ceph-mon[32192]: from='mgr.14116 172.21.15.37:0/4158559188' entity='mgr.x' 2024-06-10T00:44:57.353 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:57 smithi037 ceph-mon[32192]: from='mgr.14116 172.21.15.37:0/4158559188' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:44:57.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout value unchanged 2024-06-10T00:44:57.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.133+0000 7f1a25f38700 1 Processor -- start 2024-06-10T00:44:57.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.133+0000 7f1a25f38700 1 -- start start 2024-06-10T00:44:57.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.133+0000 7f1a25f38700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a20106e70 0x7f1a20107290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:57.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.133+0000 7f1a25f38700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1a201077d0 con 0x7f1a20106e70 2024-06-10T00:44:57.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.133+0000 7f1a1f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a20106e70 0x7f1a20107290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:57.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.133+0000 7f1a1f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a20106e70 0x7f1a20107290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43562/0 (socket says 172.21.15.37:43562) 2024-06-10T00:44:57.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.133+0000 7f1a1f7fe700 1 -- 172.21.15.37:0/3610170071 learned_addr learned my addr 172.21.15.37:0/3610170071 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:44:57.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.134+0000 7f1a1f7fe700 1 -- 172.21.15.37:0/3610170071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a20107fd0 con 0x7f1a20106e70 2024-06-10T00:44:57.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.134+0000 7f1a1f7fe700 1 --2- 172.21.15.37:0/3610170071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a20106e70 0x7f1a20107290 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1a100098a0 tx=0x7f1a10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c305a5e9c38bca87 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:57.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.134+0000 7f1a1e7fc700 1 -- 172.21.15.37:0/3610170071 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a10004160 con 0x7f1a20106e70 2024-06-10T00:44:57.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.134+0000 7f1a1e7fc700 1 -- 172.21.15.37:0/3610170071 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1a10004300 con 0x7f1a20106e70 2024-06-10T00:44:57.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.134+0000 7f1a1e7fc700 1 -- 172.21.15.37:0/3610170071 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a10011b50 con 0x7f1a20106e70 2024-06-10T00:44:57.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.135+0000 7f1a25f38700 1 -- 172.21.15.37:0/3610170071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a20106e70 msgr2=0x7f1a20107290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:57.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.135+0000 7f1a25f38700 1 --2- 172.21.15.37:0/3610170071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a20106e70 0x7f1a20107290 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1a100098a0 tx=0x7f1a10009bb0 comp rx=0 tx=0).stop 2024-06-10T00:44:57.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.135+0000 7f1a25f38700 1 -- 172.21.15.37:0/3610170071 shutdown_connections 2024-06-10T00:44:57.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.135+0000 7f1a25f38700 1 --2- 172.21.15.37:0/3610170071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a20106e70 0x7f1a20107290 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:57.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.135+0000 7f1a25f38700 1 -- 172.21.15.37:0/3610170071 >> 172.21.15.37:0/3610170071 conn(0x7f1a20102470 msgr2=0x7f1a20104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:57.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.135+0000 7f1a25f38700 1 -- 172.21.15.37:0/3610170071 shutdown_connections 2024-06-10T00:44:57.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.135+0000 7f1a25f38700 1 -- 172.21.15.37:0/3610170071 wait complete. 2024-06-10T00:44:57.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.135+0000 7f1a25f38700 1 Processor -- start 2024-06-10T00:44:57.810 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.136+0000 7f1a25f38700 1 -- start start 2024-06-10T00:44:57.810 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.136+0000 7f1a25f38700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a20106e70 0x7f1a20192010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:57.810 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.136+0000 7f1a25f38700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2010b410 con 0x7f1a20106e70 2024-06-10T00:44:57.810 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.136+0000 7f1a1f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a20106e70 0x7f1a20192010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:57.810 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.136+0000 7f1a1f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a20106e70 0x7f1a20192010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43570/0 (socket says 172.21.15.37:43570) 2024-06-10T00:44:57.810 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.136+0000 7f1a1f7fe700 1 -- 172.21.15.37:0/1357163297 learned_addr learned my addr 172.21.15.37:0/1357163297 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:44:57.810 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.136+0000 7f1a1f7fe700 1 -- 172.21.15.37:0/1357163297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a20192550 con 0x7f1a20106e70 2024-06-10T00:44:57.810 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.136+0000 7f1a1f7fe700 1 --2- 172.21.15.37:0/1357163297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a20106e70 0x7f1a20192010 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1a10011b60 tx=0x7f1a10011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:57.810 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.136+0000 7f1a1cff9700 1 -- 172.21.15.37:0/1357163297 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a10003970 con 0x7f1a20106e70 2024-06-10T00:44:57.810 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.136+0000 7f1a1cff9700 1 -- 172.21.15.37:0/1357163297 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1a10003b10 con 0x7f1a20106e70 2024-06-10T00:44:57.810 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.136+0000 7f1a25f38700 1 -- 172.21.15.37:0/1357163297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a20190740 con 0x7f1a20106e70 2024-06-10T00:44:57.810 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.137+0000 7f1a1cff9700 1 -- 172.21.15.37:0/1357163297 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a10024610 con 0x7f1a20106e70 2024-06-10T00:44:57.810 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.137+0000 7f1a25f38700 1 -- 172.21.15.37:0/1357163297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a20190c80 con 0x7f1a20106e70 2024-06-10T00:44:57.810 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.137+0000 7f1a1cff9700 1 -- 172.21.15.37:0/1357163297 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f1a1001a430 con 0x7f1a20106e70 2024-06-10T00:44:57.811 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.138+0000 7f1a1cff9700 1 --2- 172.21.15.37:0/1357163297 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f1a0c039050 0x7f1a0c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:57.811 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.138+0000 7f1a25f38700 1 -- 172.21.15.37:0/1357163297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a200695b0 con 0x7f1a20106e70 2024-06-10T00:44:57.811 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.138+0000 7f1a1cff9700 1 -- 172.21.15.37:0/1357163297 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1a10011e00 con 0x7f1a20106e70 2024-06-10T00:44:57.811 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.138+0000 7f1a1effd700 1 --2- 172.21.15.37:0/1357163297 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f1a0c039050 0x7f1a0c03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:57.811 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.138+0000 7f1a1effd700 1 --2- 172.21.15.37:0/1357163297 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f1a0c039050 0x7f1a0c03b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1a08006fd0 tx=0x7f1a08006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:57.811 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.143+0000 7f1a1cff9700 1 -- 172.21.15.37:0/1357163297 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1a100212a0 con 0x7f1a20106e70 2024-06-10T00:44:57.811 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.351+0000 7f1a25f38700 1 -- 172.21.15.37:0/1357163297 --> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f1a20195df0 con 0x7f1a0c039050 2024-06-10T00:44:57.811 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.352+0000 7f1a1cff9700 1 -- 172.21.15.37:0/1357163297 <== mgr.14116 v2:172.21.15.37:6800/2766692263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f1a20195df0 con 0x7f1a0c039050 2024-06-10T00:44:57.811 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.355+0000 7f1a25f38700 1 -- 172.21.15.37:0/1357163297 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f1a0c039050 msgr2=0x7f1a0c03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:57.811 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.355+0000 7f1a25f38700 1 --2- 172.21.15.37:0/1357163297 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f1a0c039050 0x7f1a0c03b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1a08006fd0 tx=0x7f1a08006e40 comp rx=0 tx=0).stop 2024-06-10T00:44:57.811 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.355+0000 7f1a25f38700 1 -- 172.21.15.37:0/1357163297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a20106e70 msgr2=0x7f1a20192010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:57.811 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.355+0000 7f1a25f38700 1 --2- 172.21.15.37:0/1357163297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a20106e70 0x7f1a20192010 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1a10011b60 tx=0x7f1a10011c40 comp rx=0 tx=0).stop 2024-06-10T00:44:57.811 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.355+0000 7f1a25f38700 1 -- 172.21.15.37:0/1357163297 shutdown_connections 2024-06-10T00:44:57.811 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.355+0000 7f1a25f38700 1 --2- 172.21.15.37:0/1357163297 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f1a0c039050 0x7f1a0c03b510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:57.811 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.355+0000 7f1a25f38700 1 --2- 172.21.15.37:0/1357163297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a20106e70 0x7f1a20192010 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:57.812 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.355+0000 7f1a25f38700 1 -- 172.21.15.37:0/1357163297 >> 172.21.15.37:0/1357163297 conn(0x7f1a20102470 msgr2=0x7f1a20103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:57.812 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.355+0000 7f1a25f38700 1 -- 172.21.15.37:0/1357163297 shutdown_connections 2024-06-10T00:44:57.812 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:57.355+0000 7f1a25f38700 1 -- 172.21.15.37:0/1357163297 wait complete. 2024-06-10T00:44:57.812 INFO:teuthology.orchestra.run.smithi037.stdout:Generating ssh key... 2024-06-10T00:44:58.114 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:58 smithi037 ceph-mon[32192]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:44:58.114 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:58 smithi037 ceph-mon[32192]: [10/Jun/2024:00:44:57] ENGINE Bus STARTING 2024-06-10T00:44:58.114 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:58 smithi037 ceph-mon[32192]: [10/Jun/2024:00:44:57] ENGINE Serving on https://172.21.15.37:7150 2024-06-10T00:44:58.114 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:58 smithi037 ceph-mon[32192]: from='mgr.14116 172.21.15.37:0/4158559188' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:44:58.832 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:58 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: Generating public/private rsa key pair. 2024-06-10T00:44:58.832 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:58 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: Your identification has been saved in /tmp/tmp7cqpg2mo/key. 2024-06-10T00:44:58.832 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:58 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: Your public key has been saved in /tmp/tmp7cqpg2mo/key.pub. 2024-06-10T00:44:58.832 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:58 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: The key fingerprint is: 2024-06-10T00:44:58.833 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:58 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: SHA256:gEU55a5nj7a0ZQ3H+eSJwAEJEpxw//DB9DmJushy1tg ceph-68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:44:58.833 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:58 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: The key's randomart image is: 2024-06-10T00:44:58.833 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:58 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: +---[RSA 3072]----+ 2024-06-10T00:44:58.833 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:58 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: | .o+++++. | 2024-06-10T00:44:58.833 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:58 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: | .o=o+.+ o | 2024-06-10T00:44:58.833 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:58 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: | . +.= * | 2024-06-10T00:44:58.833 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:58 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: | B o + . | 2024-06-10T00:44:58.833 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:58 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: | . S + + . | 2024-06-10T00:44:58.833 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:58 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: | . = o = = . | 2024-06-10T00:44:58.833 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:58 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: | . * E + o o + | 2024-06-10T00:44:58.833 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:58 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: | + +.* | 2024-06-10T00:44:58.833 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:58 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: | .+.. | 2024-06-10T00:44:58.833 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:44:58 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: +----[SHA256]-----+ 2024-06-10T00:44:59.081 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.215+0000 7fc93c0ec700 1 Processor -- start 2024-06-10T00:44:59.081 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.215+0000 7fc93c0ec700 1 -- start start 2024-06-10T00:44:59.082 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.215+0000 7fc93c0ec700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc93407e710 0x7fc93407cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:59.082 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.215+0000 7fc93c0ec700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc93407d140 con 0x7fc93407e710 2024-06-10T00:44:59.082 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.215+0000 7fc939e88700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc93407e710 0x7fc93407cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:59.082 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.215+0000 7fc939e88700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc93407e710 0x7fc93407cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43580/0 (socket says 172.21.15.37:43580) 2024-06-10T00:44:59.082 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.215+0000 7fc939e88700 1 -- 172.21.15.37:0/4180999031 learned_addr learned my addr 172.21.15.37:0/4180999031 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:44:59.082 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.216+0000 7fc939e88700 1 -- 172.21.15.37:0/4180999031 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93407d2c0 con 0x7fc93407e710 2024-06-10T00:44:59.082 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.216+0000 7fc939e88700 1 --2- 172.21.15.37:0/4180999031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc93407e710 0x7fc93407cb70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc9300098a0 tx=0x7fc930009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bdcb00e9be341b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:59.082 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.216+0000 7fc938e86700 1 -- 172.21.15.37:0/4180999031 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc930004160 con 0x7fc93407e710 2024-06-10T00:44:59.082 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.216+0000 7fc938e86700 1 -- 172.21.15.37:0/4180999031 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc930004300 con 0x7fc93407e710 2024-06-10T00:44:59.082 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.216+0000 7fc938e86700 1 -- 172.21.15.37:0/4180999031 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc930011b50 con 0x7fc93407e710 2024-06-10T00:44:59.082 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.217+0000 7fc93c0ec700 1 -- 172.21.15.37:0/4180999031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc93407e710 msgr2=0x7fc93407cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:59.082 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.217+0000 7fc93c0ec700 1 --2- 172.21.15.37:0/4180999031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc93407e710 0x7fc93407cb70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc9300098a0 tx=0x7fc930009bb0 comp rx=0 tx=0).stop 2024-06-10T00:44:59.083 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.217+0000 7fc93c0ec700 1 -- 172.21.15.37:0/4180999031 shutdown_connections 2024-06-10T00:44:59.083 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.217+0000 7fc93c0ec700 1 --2- 172.21.15.37:0/4180999031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc93407e710 0x7fc93407cb70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:59.083 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.217+0000 7fc93c0ec700 1 -- 172.21.15.37:0/4180999031 >> 172.21.15.37:0/4180999031 conn(0x7fc934103560 msgr2=0x7fc934105980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:59.083 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.217+0000 7fc93c0ec700 1 -- 172.21.15.37:0/4180999031 shutdown_connections 2024-06-10T00:44:59.083 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.217+0000 7fc93c0ec700 1 -- 172.21.15.37:0/4180999031 wait complete. 2024-06-10T00:44:59.083 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.217+0000 7fc93c0ec700 1 Processor -- start 2024-06-10T00:44:59.083 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.217+0000 7fc93c0ec700 1 -- start start 2024-06-10T00:44:59.083 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.218+0000 7fc93c0ec700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc93407e710 0x7fc93419e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:59.083 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.218+0000 7fc93c0ec700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc93410e590 con 0x7fc93407e710 2024-06-10T00:44:59.083 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.218+0000 7fc939e88700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc93407e710 0x7fc93419e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:59.083 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.218+0000 7fc939e88700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc93407e710 0x7fc93419e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43594/0 (socket says 172.21.15.37:43594) 2024-06-10T00:44:59.083 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.218+0000 7fc939e88700 1 -- 172.21.15.37:0/1870531582 learned_addr learned my addr 172.21.15.37:0/1870531582 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:44:59.083 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.218+0000 7fc939e88700 1 -- 172.21.15.37:0/1870531582 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93419e600 con 0x7fc93407e710 2024-06-10T00:44:59.084 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.218+0000 7fc939e88700 1 --2- 172.21.15.37:0/1870531582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc93407e710 0x7fc93419e0c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc930011b60 tx=0x7fc930011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:59.084 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.218+0000 7fc92affd700 1 -- 172.21.15.37:0/1870531582 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc930003b20 con 0x7fc93407e710 2024-06-10T00:44:59.084 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.218+0000 7fc92affd700 1 -- 172.21.15.37:0/1870531582 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc930003cc0 con 0x7fc93407e710 2024-06-10T00:44:59.084 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.218+0000 7fc92affd700 1 -- 172.21.15.37:0/1870531582 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc930024840 con 0x7fc93407e710 2024-06-10T00:44:59.084 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.218+0000 7fc93c0ec700 1 -- 172.21.15.37:0/1870531582 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc93419e890 con 0x7fc93407e710 2024-06-10T00:44:59.084 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.218+0000 7fc93c0ec700 1 -- 172.21.15.37:0/1870531582 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9341a1580 con 0x7fc93407e710 2024-06-10T00:44:59.084 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.219+0000 7fc92affd700 1 -- 172.21.15.37:0/1870531582 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fc930022020 con 0x7fc93407e710 2024-06-10T00:44:59.084 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.219+0000 7fc93c0ec700 1 -- 172.21.15.37:0/1870531582 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc934063af0 con 0x7fc93407e710 2024-06-10T00:44:59.084 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.219+0000 7fc92affd700 1 --2- 172.21.15.37:0/1870531582 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7fc9200390a0 0x7fc92003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:44:59.084 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.219+0000 7fc92affd700 1 -- 172.21.15.37:0/1870531582 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc930011e00 con 0x7fc93407e710 2024-06-10T00:44:59.084 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.219+0000 7fc939687700 1 --2- 172.21.15.37:0/1870531582 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7fc9200390a0 0x7fc92003b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:44:59.084 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.220+0000 7fc939687700 1 --2- 172.21.15.37:0/1870531582 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7fc9200390a0 0x7fc92003b560 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc924006fd0 tx=0x7fc924006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:44:59.085 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.224+0000 7fc92affd700 1 -- 172.21.15.37:0/1870531582 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc93001a7b0 con 0x7fc93407e710 2024-06-10T00:44:59.085 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.432+0000 7fc93c0ec700 1 -- 172.21.15.37:0/1870531582 --> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fc934104800 con 0x7fc9200390a0 2024-06-10T00:44:59.085 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.527+0000 7fc92affd700 1 -- 172.21.15.37:0/1870531582 <== mgr.14116 v2:172.21.15.37:6800/2766692263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc934104800 con 0x7fc9200390a0 2024-06-10T00:44:59.085 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.529+0000 7fc93c0ec700 1 -- 172.21.15.37:0/1870531582 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7fc9200390a0 msgr2=0x7fc92003b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:59.085 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.529+0000 7fc93c0ec700 1 --2- 172.21.15.37:0/1870531582 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7fc9200390a0 0x7fc92003b560 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc924006fd0 tx=0x7fc924006e40 comp rx=0 tx=0).stop 2024-06-10T00:44:59.085 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.529+0000 7fc93c0ec700 1 -- 172.21.15.37:0/1870531582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc93407e710 msgr2=0x7fc93419e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:44:59.085 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.529+0000 7fc93c0ec700 1 --2- 172.21.15.37:0/1870531582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc93407e710 0x7fc93419e0c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc930011b60 tx=0x7fc930011c40 comp rx=0 tx=0).stop 2024-06-10T00:44:59.085 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.529+0000 7fc93c0ec700 1 -- 172.21.15.37:0/1870531582 shutdown_connections 2024-06-10T00:44:59.085 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.529+0000 7fc93c0ec700 1 --2- 172.21.15.37:0/1870531582 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7fc9200390a0 0x7fc92003b560 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:59.085 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.529+0000 7fc93c0ec700 1 --2- 172.21.15.37:0/1870531582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc93407e710 0x7fc93419e0c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:44:59.085 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.529+0000 7fc93c0ec700 1 -- 172.21.15.37:0/1870531582 >> 172.21.15.37:0/1870531582 conn(0x7fc934103560 msgr2=0x7fc9341040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:44:59.085 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.529+0000 7fc93c0ec700 1 -- 172.21.15.37:0/1870531582 shutdown_connections 2024-06-10T00:44:59.085 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:58.529+0000 7fc93c0ec700 1 -- 172.21.15.37:0/1870531582 wait complete. 2024-06-10T00:44:59.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:59 smithi037 ceph-mon[32192]: [10/Jun/2024:00:44:57] ENGINE Serving on http://172.21.15.37:8765 2024-06-10T00:44:59.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:59 smithi037 ceph-mon[32192]: [10/Jun/2024:00:44:57] ENGINE Bus STARTED 2024-06-10T00:44:59.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:59 smithi037 ceph-mon[32192]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:44:59.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:59 smithi037 ceph-mon[32192]: Generating ssh key... 2024-06-10T00:44:59.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:59 smithi037 ceph-mon[32192]: from='mgr.14116 172.21.15.37:0/4158559188' entity='mgr.x' 2024-06-10T00:44:59.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:44:59 smithi037 ceph-mon[32192]: from='mgr.14116 172.21.15.37:0/4158559188' entity='mgr.x' 2024-06-10T00:45:00.264 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7TIymYqklc98F6uYaAs39Pd6IcD+eA4WVkiibprOitzX0D/nKxT/UFFq7moiNcqgG84Zh+VSrK1tLGWfsYWkKfPMnSZqlj5yfpkCAJ25hN5BHmiexFOKOzKvgt71orDXmbvy2+qj7t0rXPJbpYVVRa4CmHVgNSbdnIoWyA9oIOQwmR8fG680Lh4rp3fqO4x0SQi1ZTrxQeNMqWkhnATRhfkI8RxB76x9eJOSlGnKUXYc4YNK5rXpEtynTkpPPXsatYGFmSlUGjIHYd+zDxfrLfo3qGrp8pkJBv+BkezHQKrNSfufPbT7670eLLZn6fWonMS4fbpSlHYOvBIx/S4C4aGD/EhoIf59Bcj6U66k21rRhPwcic5NQIYv/9CgkwDIwtGZFJQnJKtKMF9s4oIuxMuEGG99D2g3kJww3idm/NFdw1A8sFJg42GMyWsGYQJnCzf7XYxgXcACs+TUvLihrtoe59iMWIhQT08nXWLxmzWFuzt8g8tZqzhS+Mkj5Yxs= ceph-68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:45:00.264 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.455+0000 7f9ba9e93700 1 Processor -- start 2024-06-10T00:45:00.264 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.455+0000 7f9ba9e93700 1 -- start start 2024-06-10T00:45:00.264 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.455+0000 7f9ba9e93700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ba4108a40 0x7f9ba410ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:00.264 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.455+0000 7f9ba9e93700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba407be70 con 0x7f9ba4108a40 2024-06-10T00:45:00.264 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.456+0000 7f9ba37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ba4108a40 0x7f9ba410ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:00.265 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.456+0000 7f9ba37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ba4108a40 0x7f9ba410ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43608/0 (socket says 172.21.15.37:43608) 2024-06-10T00:45:00.265 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.456+0000 7f9ba37fe700 1 -- 172.21.15.37:0/805383157 learned_addr learned my addr 172.21.15.37:0/805383157 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:00.265 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.456+0000 7f9ba37fe700 1 -- 172.21.15.37:0/805383157 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba410b370 con 0x7f9ba4108a40 2024-06-10T00:45:00.265 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.456+0000 7f9ba37fe700 1 --2- 172.21.15.37:0/805383157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ba4108a40 0x7f9ba410ae30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9b940098a0 tx=0x7f9b94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff31763413f6eef server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:00.265 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.456+0000 7f9ba27fc700 1 -- 172.21.15.37:0/805383157 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b94004160 con 0x7f9ba4108a40 2024-06-10T00:45:00.265 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.456+0000 7f9ba27fc700 1 -- 172.21.15.37:0/805383157 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9b94004300 con 0x7f9ba4108a40 2024-06-10T00:45:00.265 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.456+0000 7f9ba27fc700 1 -- 172.21.15.37:0/805383157 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b94011b50 con 0x7f9ba4108a40 2024-06-10T00:45:00.265 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.457+0000 7f9ba9e93700 1 -- 172.21.15.37:0/805383157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ba4108a40 msgr2=0x7f9ba410ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:00.265 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.457+0000 7f9ba9e93700 1 --2- 172.21.15.37:0/805383157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ba4108a40 0x7f9ba410ae30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9b940098a0 tx=0x7f9b94009bb0 comp rx=0 tx=0).stop 2024-06-10T00:45:00.265 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.457+0000 7f9ba9e93700 1 -- 172.21.15.37:0/805383157 shutdown_connections 2024-06-10T00:45:00.265 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.457+0000 7f9ba9e93700 1 --2- 172.21.15.37:0/805383157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ba4108a40 0x7f9ba410ae30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:00.265 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.457+0000 7f9ba9e93700 1 -- 172.21.15.37:0/805383157 >> 172.21.15.37:0/805383157 conn(0x7f9ba4102450 msgr2=0x7f9ba4104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:00.266 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.457+0000 7f9ba9e93700 1 -- 172.21.15.37:0/805383157 shutdown_connections 2024-06-10T00:45:00.266 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.457+0000 7f9ba9e93700 1 -- 172.21.15.37:0/805383157 wait complete. 2024-06-10T00:45:00.266 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.458+0000 7f9ba9e93700 1 Processor -- start 2024-06-10T00:45:00.266 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.458+0000 7f9ba9e93700 1 -- start start 2024-06-10T00:45:00.266 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.458+0000 7f9ba9e93700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ba4108a40 0x7f9ba419bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:00.266 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.458+0000 7f9ba9e93700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba410c390 con 0x7f9ba4108a40 2024-06-10T00:45:00.266 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.458+0000 7f9ba37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ba4108a40 0x7f9ba419bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:00.266 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.458+0000 7f9ba37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ba4108a40 0x7f9ba419bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43624/0 (socket says 172.21.15.37:43624) 2024-06-10T00:45:00.266 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.458+0000 7f9ba37fe700 1 -- 172.21.15.37:0/1076658544 learned_addr learned my addr 172.21.15.37:0/1076658544 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:00.266 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.458+0000 7f9ba37fe700 1 -- 172.21.15.37:0/1076658544 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba419c410 con 0x7f9ba4108a40 2024-06-10T00:45:00.266 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.459+0000 7f9ba37fe700 1 --2- 172.21.15.37:0/1076658544 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ba4108a40 0x7f9ba419bed0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9b94011b60 tx=0x7f9b94011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:00.266 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.459+0000 7f9ba0ff9700 1 -- 172.21.15.37:0/1076658544 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b94003b20 con 0x7f9ba4108a40 2024-06-10T00:45:00.266 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.459+0000 7f9ba0ff9700 1 -- 172.21.15.37:0/1076658544 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9b94003cc0 con 0x7f9ba4108a40 2024-06-10T00:45:00.266 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.459+0000 7f9ba9e93700 1 -- 172.21.15.37:0/1076658544 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ba419c6a0 con 0x7f9ba4108a40 2024-06-10T00:45:00.267 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.459+0000 7f9ba0ff9700 1 -- 172.21.15.37:0/1076658544 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b94024840 con 0x7f9ba4108a40 2024-06-10T00:45:00.267 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.459+0000 7f9ba9e93700 1 -- 172.21.15.37:0/1076658544 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ba419f390 con 0x7f9ba4108a40 2024-06-10T00:45:00.267 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.460+0000 7f9ba0ff9700 1 -- 172.21.15.37:0/1076658544 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f9b94022020 con 0x7f9ba4108a40 2024-06-10T00:45:00.267 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.460+0000 7f9ba0ff9700 1 --2- 172.21.15.37:0/1076658544 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f9b8c039050 0x7f9b8c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:00.267 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.460+0000 7f9ba0ff9700 1 -- 172.21.15.37:0/1076658544 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9b94011e00 con 0x7f9ba4108a40 2024-06-10T00:45:00.267 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.460+0000 7f9ba2ffd700 1 --2- 172.21.15.37:0/1076658544 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f9b8c039050 0x7f9b8c03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:00.267 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.460+0000 7f9ba9e93700 1 -- 172.21.15.37:0/1076658544 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ba4063af0 con 0x7f9ba4108a40 2024-06-10T00:45:00.267 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.460+0000 7f9ba2ffd700 1 --2- 172.21.15.37:0/1076658544 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f9b8c039050 0x7f9b8c03b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9b98006fd0 tx=0x7f9b98006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:00.267 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.465+0000 7f9ba0ff9700 1 -- 172.21.15.37:0/1076658544 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9b94029de0 con 0x7f9ba4108a40 2024-06-10T00:45:00.267 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.673+0000 7f9ba9e93700 1 -- 172.21.15.37:0/1076658544 --> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f9ba4103650 con 0x7f9b8c039050 2024-06-10T00:45:00.267 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.674+0000 7f9ba0ff9700 1 -- 172.21.15.37:0/1076658544 <== mgr.14116 v2:172.21.15.37:6800/2766692263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f9ba4103650 con 0x7f9b8c039050 2024-06-10T00:45:00.267 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.676+0000 7f9ba9e93700 1 -- 172.21.15.37:0/1076658544 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f9b8c039050 msgr2=0x7f9b8c03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:00.267 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.676+0000 7f9ba9e93700 1 --2- 172.21.15.37:0/1076658544 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f9b8c039050 0x7f9b8c03b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9b98006fd0 tx=0x7f9b98006e40 comp rx=0 tx=0).stop 2024-06-10T00:45:00.267 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.676+0000 7f9ba9e93700 1 -- 172.21.15.37:0/1076658544 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ba4108a40 msgr2=0x7f9ba419bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:00.268 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.676+0000 7f9ba9e93700 1 --2- 172.21.15.37:0/1076658544 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ba4108a40 0x7f9ba419bed0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9b94011b60 tx=0x7f9b94011c40 comp rx=0 tx=0).stop 2024-06-10T00:45:00.268 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.676+0000 7f9ba9e93700 1 -- 172.21.15.37:0/1076658544 shutdown_connections 2024-06-10T00:45:00.268 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.677+0000 7f9ba9e93700 1 --2- 172.21.15.37:0/1076658544 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f9b8c039050 0x7f9b8c03b510 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:00.268 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.677+0000 7f9ba9e93700 1 --2- 172.21.15.37:0/1076658544 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ba4108a40 0x7f9ba419bed0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:00.268 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.677+0000 7f9ba9e93700 1 -- 172.21.15.37:0/1076658544 >> 172.21.15.37:0/1076658544 conn(0x7f9ba4102450 msgr2=0x7f9ba4102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:00.268 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.677+0000 7f9ba9e93700 1 -- 172.21.15.37:0/1076658544 shutdown_connections 2024-06-10T00:45:00.268 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:44:59.677+0000 7f9ba9e93700 1 -- 172.21.15.37:0/1076658544 wait complete. 2024-06-10T00:45:00.268 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-06-10T00:45:00.268 INFO:teuthology.orchestra.run.smithi037.stdout:Adding key to root@localhost authorized_keys... 2024-06-10T00:45:00.268 INFO:teuthology.orchestra.run.smithi037.stdout:Adding host smithi037... 2024-06-10T00:45:00.878 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:00 smithi037 ceph-mon[32192]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:45:02.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:01 smithi037 ceph-mon[32192]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi037", "addr": "172.21.15.37", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:45:02.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:01 smithi037 ceph-mon[32192]: Deploying cephadm binary to smithi037 2024-06-10T00:45:05.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Added host 'smithi037' with addr '172.21.15.37' 2024-06-10T00:45:05.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.655+0000 7fa50e12c700 1 Processor -- start 2024-06-10T00:45:05.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.655+0000 7fa50e12c700 1 -- start start 2024-06-10T00:45:05.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.655+0000 7fa50e12c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5081068b0 0x7fa508106cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:05.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.655+0000 7fa50e12c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa50807c670 con 0x7fa5081068b0 2024-06-10T00:45:05.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.656+0000 7fa5077fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5081068b0 0x7fa508106cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:05.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.656+0000 7fa5077fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5081068b0 0x7fa508106cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43638/0 (socket says 172.21.15.37:43638) 2024-06-10T00:45:05.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.656+0000 7fa5077fe700 1 -- 172.21.15.37:0/2946766779 learned_addr learned my addr 172.21.15.37:0/2946766779 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:05.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.656+0000 7fa5077fe700 1 -- 172.21.15.37:0/2946766779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa508107210 con 0x7fa5081068b0 2024-06-10T00:45:05.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.656+0000 7fa5077fe700 1 --2- 172.21.15.37:0/2946766779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5081068b0 0x7fa508106cd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa4f80098a0 tx=0x7fa4f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e865176ee84c88b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:05.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.656+0000 7fa5067fc700 1 -- 172.21.15.37:0/2946766779 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4f8004160 con 0x7fa5081068b0 2024-06-10T00:45:05.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.656+0000 7fa5067fc700 1 -- 172.21.15.37:0/2946766779 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa4f8004300 con 0x7fa5081068b0 2024-06-10T00:45:05.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.656+0000 7fa5067fc700 1 -- 172.21.15.37:0/2946766779 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4f8011b50 con 0x7fa5081068b0 2024-06-10T00:45:05.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.657+0000 7fa50e12c700 1 -- 172.21.15.37:0/2946766779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5081068b0 msgr2=0x7fa508106cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:05.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.657+0000 7fa50e12c700 1 --2- 172.21.15.37:0/2946766779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5081068b0 0x7fa508106cd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa4f80098a0 tx=0x7fa4f8009bb0 comp rx=0 tx=0).stop 2024-06-10T00:45:05.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.657+0000 7fa50e12c700 1 -- 172.21.15.37:0/2946766779 shutdown_connections 2024-06-10T00:45:05.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.657+0000 7fa50e12c700 1 --2- 172.21.15.37:0/2946766779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5081068b0 0x7fa508106cd0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:05.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.657+0000 7fa50e12c700 1 -- 172.21.15.37:0/2946766779 >> 172.21.15.37:0/2946766779 conn(0x7fa508102490 msgr2=0x7fa5081048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:05.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.657+0000 7fa50e12c700 1 -- 172.21.15.37:0/2946766779 shutdown_connections 2024-06-10T00:45:05.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.657+0000 7fa50e12c700 1 -- 172.21.15.37:0/2946766779 wait complete. 2024-06-10T00:45:05.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.658+0000 7fa50e12c700 1 Processor -- start 2024-06-10T00:45:05.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.658+0000 7fa50e12c700 1 -- start start 2024-06-10T00:45:05.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.658+0000 7fa50e12c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5081068b0 0x7fa50819e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:05.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.658+0000 7fa50e12c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa508108120 con 0x7fa5081068b0 2024-06-10T00:45:05.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.658+0000 7fa5077fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5081068b0 0x7fa50819e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:05.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.658+0000 7fa5077fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5081068b0 0x7fa50819e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43642/0 (socket says 172.21.15.37:43642) 2024-06-10T00:45:05.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.658+0000 7fa5077fe700 1 -- 172.21.15.37:0/4131462305 learned_addr learned my addr 172.21.15.37:0/4131462305 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:05.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.659+0000 7fa5077fe700 1 -- 172.21.15.37:0/4131462305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa50819e640 con 0x7fa5081068b0 2024-06-10T00:45:05.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.659+0000 7fa5077fe700 1 --2- 172.21.15.37:0/4131462305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5081068b0 0x7fa50819e100 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8011670 tx=0x7fa4f8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:05.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.659+0000 7fa504ff9700 1 -- 172.21.15.37:0/4131462305 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4f80039c0 con 0x7fa5081068b0 2024-06-10T00:45:05.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.659+0000 7fa50e12c700 1 -- 172.21.15.37:0/4131462305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa50819e8d0 con 0x7fa5081068b0 2024-06-10T00:45:05.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.659+0000 7fa504ff9700 1 -- 172.21.15.37:0/4131462305 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa4f8011e50 con 0x7fa5081068b0 2024-06-10T00:45:05.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.659+0000 7fa504ff9700 1 -- 172.21.15.37:0/4131462305 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4f801a5c0 con 0x7fa5081068b0 2024-06-10T00:45:05.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.659+0000 7fa50e12c700 1 -- 172.21.15.37:0/4131462305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5081a15c0 con 0x7fa5081068b0 2024-06-10T00:45:05.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.660+0000 7fa504ff9700 1 -- 172.21.15.37:0/4131462305 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa4f8024430 con 0x7fa5081068b0 2024-06-10T00:45:05.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.660+0000 7fa50e12c700 1 -- 172.21.15.37:0/4131462305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa508063af0 con 0x7fa5081068b0 2024-06-10T00:45:05.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.660+0000 7fa504ff9700 1 --2- 172.21.15.37:0/4131462305 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7fa4f0039050 0x7fa4f003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:05.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.660+0000 7fa504ff9700 1 -- 172.21.15.37:0/4131462305 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa4f804d380 con 0x7fa5081068b0 2024-06-10T00:45:05.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.660+0000 7fa506ffd700 1 --2- 172.21.15.37:0/4131462305 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7fa4f0039050 0x7fa4f003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:05.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.661+0000 7fa506ffd700 1 --2- 172.21.15.37:0/4131462305 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7fa4f0039050 0x7fa4f003b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc006fd0 tx=0x7fa4fc006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:05.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.665+0000 7fa504ff9700 1 -- 172.21.15.37:0/4131462305 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa4f802d910 con 0x7fa5081068b0 2024-06-10T00:45:05.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:00.878+0000 7fa50e12c700 1 -- 172.21.15.37:0/4131462305 --> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi037", "addr": "172.21.15.37", "target": ["mon-mgr", ""]}) -- 0x7fa50802df80 con 0x7fa4f0039050 2024-06-10T00:45:05.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:05.244+0000 7fa504ff9700 1 -- 172.21.15.37:0/4131462305 <== mgr.14116 v2:172.21.15.37:6800/2766692263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fa50802df80 con 0x7fa4f0039050 2024-06-10T00:45:05.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:05.247+0000 7fa50e12c700 1 -- 172.21.15.37:0/4131462305 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7fa4f0039050 msgr2=0x7fa4f003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:05.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:05.247+0000 7fa50e12c700 1 --2- 172.21.15.37:0/4131462305 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7fa4f0039050 0x7fa4f003b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc006fd0 tx=0x7fa4fc006e40 comp rx=0 tx=0).stop 2024-06-10T00:45:05.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:05.247+0000 7fa50e12c700 1 -- 172.21.15.37:0/4131462305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5081068b0 msgr2=0x7fa50819e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:05.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:05.247+0000 7fa50e12c700 1 --2- 172.21.15.37:0/4131462305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5081068b0 0x7fa50819e100 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8011670 tx=0x7fa4f8011b60 comp rx=0 tx=0).stop 2024-06-10T00:45:05.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:05.247+0000 7fa50e12c700 1 -- 172.21.15.37:0/4131462305 shutdown_connections 2024-06-10T00:45:05.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:05.247+0000 7fa50e12c700 1 --2- 172.21.15.37:0/4131462305 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7fa4f0039050 0x7fa4f003b510 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:05.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:05.247+0000 7fa50e12c700 1 --2- 172.21.15.37:0/4131462305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5081068b0 0x7fa50819e100 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:05.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:05.247+0000 7fa50e12c700 1 -- 172.21.15.37:0/4131462305 >> 172.21.15.37:0/4131462305 conn(0x7fa508102490 msgr2=0x7fa508194d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:05.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:05.247+0000 7fa50e12c700 1 -- 172.21.15.37:0/4131462305 shutdown_connections 2024-06-10T00:45:05.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:05.247+0000 7fa50e12c700 1 -- 172.21.15.37:0/4131462305 wait complete. 2024-06-10T00:45:05.834 INFO:teuthology.orchestra.run.smithi037.stdout:Deploying unmanaged mon service... 2024-06-10T00:45:06.550 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:06 smithi037 ceph-mon[32192]: from='mgr.14116 172.21.15.37:0/4158559188' entity='mgr.x' 2024-06-10T00:45:06.550 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:06 smithi037 ceph-mon[32192]: Added host smithi037 2024-06-10T00:45:06.550 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:06 smithi037 ceph-mon[32192]: from='mgr.14116 172.21.15.37:0/4158559188' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:45:07.375 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-06-10T00:45:07.375 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.307+0000 7f062e4e6700 1 Processor -- start 2024-06-10T00:45:07.375 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.307+0000 7f062e4e6700 1 -- start start 2024-06-10T00:45:07.375 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.307+0000 7f062e4e6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f062807e730 0x7f062807cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:07.375 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.307+0000 7f062e4e6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f062807d160 con 0x7f062807e730 2024-06-10T00:45:07.375 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.307+0000 7f0627fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f062807e730 0x7f062807cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:07.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.308+0000 7f0627fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f062807e730 0x7f062807cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38998/0 (socket says 172.21.15.37:38998) 2024-06-10T00:45:07.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.308+0000 7f0627fff700 1 -- 172.21.15.37:0/2194510954 learned_addr learned my addr 172.21.15.37:0/2194510954 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:07.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.308+0000 7f0627fff700 1 -- 172.21.15.37:0/2194510954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f062807d2e0 con 0x7f062807e730 2024-06-10T00:45:07.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.308+0000 7f0627fff700 1 --2- 172.21.15.37:0/2194510954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f062807e730 0x7f062807cb90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f06180098a0 tx=0x7f0618009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0eea3e722f6ab4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:07.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.308+0000 7f0626ffd700 1 -- 172.21.15.37:0/2194510954 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0618004160 con 0x7f062807e730 2024-06-10T00:45:07.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.308+0000 7f0626ffd700 1 -- 172.21.15.37:0/2194510954 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0618004300 con 0x7f062807e730 2024-06-10T00:45:07.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.308+0000 7f0626ffd700 1 -- 172.21.15.37:0/2194510954 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0618011b50 con 0x7f062807e730 2024-06-10T00:45:07.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.309+0000 7f062e4e6700 1 -- 172.21.15.37:0/2194510954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f062807e730 msgr2=0x7f062807cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:07.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.309+0000 7f062e4e6700 1 --2- 172.21.15.37:0/2194510954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f062807e730 0x7f062807cb90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f06180098a0 tx=0x7f0618009bb0 comp rx=0 tx=0).stop 2024-06-10T00:45:07.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.309+0000 7f062e4e6700 1 -- 172.21.15.37:0/2194510954 shutdown_connections 2024-06-10T00:45:07.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.309+0000 7f062e4e6700 1 --2- 172.21.15.37:0/2194510954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f062807e730 0x7f062807cb90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:07.377 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.309+0000 7f062e4e6700 1 -- 172.21.15.37:0/2194510954 >> 172.21.15.37:0/2194510954 conn(0x7f0628103560 msgr2=0x7f0628105980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:07.377 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.309+0000 7f062e4e6700 1 -- 172.21.15.37:0/2194510954 shutdown_connections 2024-06-10T00:45:07.377 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.309+0000 7f062e4e6700 1 -- 172.21.15.37:0/2194510954 wait complete. 2024-06-10T00:45:07.377 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.309+0000 7f062e4e6700 1 Processor -- start 2024-06-10T00:45:07.377 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.309+0000 7f062e4e6700 1 -- start start 2024-06-10T00:45:07.377 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.310+0000 7f062e4e6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f062807e730 0x7f062819e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:07.377 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.310+0000 7f062e4e6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f062810e570 con 0x7f062807e730 2024-06-10T00:45:07.377 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.310+0000 7f0627fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f062807e730 0x7f062819e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:07.377 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.310+0000 7f0627fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f062807e730 0x7f062819e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39002/0 (socket says 172.21.15.37:39002) 2024-06-10T00:45:07.377 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.310+0000 7f0627fff700 1 -- 172.21.15.37:0/1888121816 learned_addr learned my addr 172.21.15.37:0/1888121816 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:07.377 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.310+0000 7f0627fff700 1 -- 172.21.15.37:0/1888121816 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f062819e6a0 con 0x7f062807e730 2024-06-10T00:45:07.377 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.310+0000 7f0627fff700 1 --2- 172.21.15.37:0/1888121816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f062807e730 0x7f062819e160 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0618000c00 tx=0x7f0618011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:07.377 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.311+0000 7f06257fa700 1 -- 172.21.15.37:0/1888121816 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0618003b20 con 0x7f062807e730 2024-06-10T00:45:07.378 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.311+0000 7f06257fa700 1 -- 172.21.15.37:0/1888121816 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0618003cc0 con 0x7f062807e730 2024-06-10T00:45:07.378 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.311+0000 7f062e4e6700 1 -- 172.21.15.37:0/1888121816 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f062819e930 con 0x7f062807e730 2024-06-10T00:45:07.378 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.311+0000 7f06257fa700 1 -- 172.21.15.37:0/1888121816 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0618024910 con 0x7f062807e730 2024-06-10T00:45:07.378 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.311+0000 7f062e4e6700 1 -- 172.21.15.37:0/1888121816 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06281a1620 con 0x7f062807e730 2024-06-10T00:45:07.378 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.311+0000 7f06257fa700 1 -- 172.21.15.37:0/1888121816 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f0618022020 con 0x7f062807e730 2024-06-10T00:45:07.378 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.312+0000 7f062e4e6700 1 -- 172.21.15.37:0/1888121816 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0628063af0 con 0x7f062807e730 2024-06-10T00:45:07.378 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.312+0000 7f06257fa700 1 --2- 172.21.15.37:0/1888121816 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f06100390a0 0x7f061003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:07.378 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.312+0000 7f06257fa700 1 -- 172.21.15.37:0/1888121816 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0618011e00 con 0x7f062807e730 2024-06-10T00:45:07.378 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.312+0000 7f06277fe700 1 --2- 172.21.15.37:0/1888121816 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f06100390a0 0x7f061003b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:07.378 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.312+0000 7f06277fe700 1 --2- 172.21.15.37:0/1888121816 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f06100390a0 0x7f061003b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f061c006fd0 tx=0x7f061c006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:07.378 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.317+0000 7f06257fa700 1 -- 172.21.15.37:0/1888121816 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f061802cdf0 con 0x7f062807e730 2024-06-10T00:45:07.378 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.549+0000 7f062e4e6700 1 -- 172.21.15.37:0/1888121816 --> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f06281047f0 con 0x7f06100390a0 2024-06-10T00:45:07.378 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.554+0000 7f06257fa700 1 -- 172.21.15.37:0/1888121816 <== mgr.14116 v2:172.21.15.37:6800/2766692263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f06281047f0 con 0x7f06100390a0 2024-06-10T00:45:07.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.557+0000 7f062e4e6700 1 -- 172.21.15.37:0/1888121816 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f06100390a0 msgr2=0x7f061003b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:07.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.557+0000 7f062e4e6700 1 --2- 172.21.15.37:0/1888121816 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f06100390a0 0x7f061003b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f061c006fd0 tx=0x7f061c006e40 comp rx=0 tx=0).stop 2024-06-10T00:45:07.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.557+0000 7f062e4e6700 1 -- 172.21.15.37:0/1888121816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f062807e730 msgr2=0x7f062819e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:07.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.557+0000 7f062e4e6700 1 --2- 172.21.15.37:0/1888121816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f062807e730 0x7f062819e160 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0618000c00 tx=0x7f0618011c10 comp rx=0 tx=0).stop 2024-06-10T00:45:07.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.557+0000 7f062e4e6700 1 -- 172.21.15.37:0/1888121816 shutdown_connections 2024-06-10T00:45:07.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.557+0000 7f062e4e6700 1 --2- 172.21.15.37:0/1888121816 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f06100390a0 0x7f061003b560 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:07.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.557+0000 7f062e4e6700 1 --2- 172.21.15.37:0/1888121816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f062807e730 0x7f062819e160 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:07.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.557+0000 7f062e4e6700 1 -- 172.21.15.37:0/1888121816 >> 172.21.15.37:0/1888121816 conn(0x7f0628103560 msgr2=0x7f06281040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:07.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.557+0000 7f062e4e6700 1 -- 172.21.15.37:0/1888121816 shutdown_connections 2024-06-10T00:45:07.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:06.557+0000 7f062e4e6700 1 -- 172.21.15.37:0/1888121816 wait complete. 2024-06-10T00:45:07.379 INFO:teuthology.orchestra.run.smithi037.stdout:Deploying unmanaged mgr service... 2024-06-10T00:45:07.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:07 smithi037 ceph-mon[32192]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:45:07.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:07 smithi037 ceph-mon[32192]: Saving service mon spec with placement count:5 2024-06-10T00:45:07.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:07 smithi037 ceph-mon[32192]: from='mgr.14116 172.21.15.37:0/4158559188' entity='mgr.x' 2024-06-10T00:45:08.591 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-06-10T00:45:08.591 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.799+0000 7fc4ea074700 1 Processor -- start 2024-06-10T00:45:08.591 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.799+0000 7fc4ea074700 1 -- start start 2024-06-10T00:45:08.591 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.799+0000 7fc4ea074700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4e4108ac0 0x7fc4e410aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:08.591 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.799+0000 7fc4ea074700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e407bdf0 con 0x7fc4e4108ac0 2024-06-10T00:45:08.591 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.799+0000 7fc4e37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4e4108ac0 0x7fc4e410aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:08.591 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.800+0000 7fc4e37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4e4108ac0 0x7fc4e410aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39018/0 (socket says 172.21.15.37:39018) 2024-06-10T00:45:08.591 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.800+0000 7fc4e37fe700 1 -- 172.21.15.37:0/333702813 learned_addr learned my addr 172.21.15.37:0/333702813 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:08.591 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.800+0000 7fc4e37fe700 1 -- 172.21.15.37:0/333702813 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e410b3f0 con 0x7fc4e4108ac0 2024-06-10T00:45:08.591 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.800+0000 7fc4e37fe700 1 --2- 172.21.15.37:0/333702813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4e4108ac0 0x7fc4e410aeb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40098a0 tx=0x7fc4d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a7ff4286301cb8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:08.591 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.800+0000 7fc4e27fc700 1 -- 172.21.15.37:0/333702813 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4d4004160 con 0x7fc4e4108ac0 2024-06-10T00:45:08.592 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.800+0000 7fc4e27fc700 1 -- 172.21.15.37:0/333702813 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc4d4004300 con 0x7fc4e4108ac0 2024-06-10T00:45:08.592 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.800+0000 7fc4e27fc700 1 -- 172.21.15.37:0/333702813 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4d4011b50 con 0x7fc4e4108ac0 2024-06-10T00:45:08.592 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.801+0000 7fc4ea074700 1 -- 172.21.15.37:0/333702813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4e4108ac0 msgr2=0x7fc4e410aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:08.592 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.801+0000 7fc4ea074700 1 --2- 172.21.15.37:0/333702813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4e4108ac0 0x7fc4e410aeb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40098a0 tx=0x7fc4d4009bb0 comp rx=0 tx=0).stop 2024-06-10T00:45:08.592 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.801+0000 7fc4ea074700 1 -- 172.21.15.37:0/333702813 shutdown_connections 2024-06-10T00:45:08.592 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.801+0000 7fc4ea074700 1 --2- 172.21.15.37:0/333702813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4e4108ac0 0x7fc4e410aeb0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:08.592 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.801+0000 7fc4ea074700 1 -- 172.21.15.37:0/333702813 >> 172.21.15.37:0/333702813 conn(0x7fc4e4102450 msgr2=0x7fc4e4104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:08.592 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.801+0000 7fc4ea074700 1 -- 172.21.15.37:0/333702813 shutdown_connections 2024-06-10T00:45:08.592 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.801+0000 7fc4ea074700 1 -- 172.21.15.37:0/333702813 wait complete. 2024-06-10T00:45:08.592 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.802+0000 7fc4ea074700 1 Processor -- start 2024-06-10T00:45:08.592 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.802+0000 7fc4ea074700 1 -- start start 2024-06-10T00:45:08.593 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.802+0000 7fc4ea074700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4e4108ac0 0x7fc4e419bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:08.593 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.802+0000 7fc4ea074700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e410c360 con 0x7fc4e4108ac0 2024-06-10T00:45:08.593 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.802+0000 7fc4e37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4e4108ac0 0x7fc4e419bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:08.593 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.802+0000 7fc4e37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4e4108ac0 0x7fc4e419bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39032/0 (socket says 172.21.15.37:39032) 2024-06-10T00:45:08.593 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.802+0000 7fc4e37fe700 1 -- 172.21.15.37:0/1733386856 learned_addr learned my addr 172.21.15.37:0/1733386856 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:08.593 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.803+0000 7fc4e37fe700 1 -- 172.21.15.37:0/1733386856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e419c420 con 0x7fc4e4108ac0 2024-06-10T00:45:08.593 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.803+0000 7fc4e37fe700 1 --2- 172.21.15.37:0/1733386856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4e4108ac0 0x7fc4e419bee0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4011b60 tx=0x7fc4d4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:08.593 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.803+0000 7fc4e0ff9700 1 -- 172.21.15.37:0/1733386856 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4d4003b20 con 0x7fc4e4108ac0 2024-06-10T00:45:08.593 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.803+0000 7fc4e0ff9700 1 -- 172.21.15.37:0/1733386856 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc4d4003cc0 con 0x7fc4e4108ac0 2024-06-10T00:45:08.593 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.803+0000 7fc4ea074700 1 -- 172.21.15.37:0/1733386856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e419c6b0 con 0x7fc4e4108ac0 2024-06-10T00:45:08.593 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.803+0000 7fc4e0ff9700 1 -- 172.21.15.37:0/1733386856 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4d4024840 con 0x7fc4e4108ac0 2024-06-10T00:45:08.593 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.803+0000 7fc4ea074700 1 -- 172.21.15.37:0/1733386856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e419f3a0 con 0x7fc4e4108ac0 2024-06-10T00:45:08.593 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.804+0000 7fc4e0ff9700 1 -- 172.21.15.37:0/1733386856 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fc4d4022020 con 0x7fc4e4108ac0 2024-06-10T00:45:08.593 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.804+0000 7fc4e0ff9700 1 --2- 172.21.15.37:0/1733386856 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7fc4d0039050 0x7fc4d003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:08.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.804+0000 7fc4e0ff9700 1 -- 172.21.15.37:0/1733386856 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc4d4011e00 con 0x7fc4e4108ac0 2024-06-10T00:45:08.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.804+0000 7fc4ea074700 1 -- 172.21.15.37:0/1733386856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4e4063af0 con 0x7fc4e4108ac0 2024-06-10T00:45:08.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.805+0000 7fc4e2ffd700 1 --2- 172.21.15.37:0/1733386856 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7fc4d0039050 0x7fc4d003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:08.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.805+0000 7fc4e2ffd700 1 --2- 172.21.15.37:0/1733386856 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7fc4d0039050 0x7fc4d003b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc006fd0 tx=0x7fc4cc006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:08.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:07.810+0000 7fc4e0ff9700 1 -- 172.21.15.37:0/1733386856 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc4d400ba00 con 0x7fc4e4108ac0 2024-06-10T00:45:08.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:08.054+0000 7fc4ea074700 1 -- 172.21.15.37:0/1733386856 --> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fc4e4103620 con 0x7fc4d0039050 2024-06-10T00:45:08.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:08.059+0000 7fc4e0ff9700 1 -- 172.21.15.37:0/1733386856 <== mgr.14116 v2:172.21.15.37:6800/2766692263 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc4e4103620 con 0x7fc4d0039050 2024-06-10T00:45:08.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:08.061+0000 7fc4ea074700 1 -- 172.21.15.37:0/1733386856 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7fc4d0039050 msgr2=0x7fc4d003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:08.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:08.061+0000 7fc4ea074700 1 --2- 172.21.15.37:0/1733386856 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7fc4d0039050 0x7fc4d003b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc006fd0 tx=0x7fc4cc006e40 comp rx=0 tx=0).stop 2024-06-10T00:45:08.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:08.061+0000 7fc4ea074700 1 -- 172.21.15.37:0/1733386856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4e4108ac0 msgr2=0x7fc4e419bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:08.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:08.061+0000 7fc4ea074700 1 --2- 172.21.15.37:0/1733386856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4e4108ac0 0x7fc4e419bee0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4011b60 tx=0x7fc4d4011c40 comp rx=0 tx=0).stop 2024-06-10T00:45:08.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:08.061+0000 7fc4ea074700 1 -- 172.21.15.37:0/1733386856 shutdown_connections 2024-06-10T00:45:08.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:08.061+0000 7fc4ea074700 1 --2- 172.21.15.37:0/1733386856 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7fc4d0039050 0x7fc4d003b510 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:08.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:08.061+0000 7fc4ea074700 1 --2- 172.21.15.37:0/1733386856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4e4108ac0 0x7fc4e419bee0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:08.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:08.061+0000 7fc4ea074700 1 -- 172.21.15.37:0/1733386856 >> 172.21.15.37:0/1733386856 conn(0x7fc4e4102450 msgr2=0x7fc4e4102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:08.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:08.061+0000 7fc4ea074700 1 -- 172.21.15.37:0/1733386856 shutdown_connections 2024-06-10T00:45:08.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:08.061+0000 7fc4ea074700 1 -- 172.21.15.37:0/1733386856 wait complete. 2024-06-10T00:45:09.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:09 smithi037 ceph-mon[32192]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:45:09.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:09 smithi037 ceph-mon[32192]: Saving service mgr spec with placement count:2 2024-06-10T00:45:09.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:09 smithi037 ceph-mon[32192]: from='mgr.14116 172.21.15.37:0/4158559188' entity='mgr.x' 2024-06-10T00:45:09.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:09 smithi037 ceph-mon[32192]: from='mgr.14116 172.21.15.37:0/4158559188' entity='mgr.x' 2024-06-10T00:45:10.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.315+0000 7f945da7c700 1 Processor -- start 2024-06-10T00:45:10.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.315+0000 7f945da7c700 1 -- start start 2024-06-10T00:45:10.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.315+0000 7f945da7c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9458106800 0x7f9458106c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:10.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.316+0000 7f945da7c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f945807c6e0 con 0x7f9458106800 2024-06-10T00:45:10.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.316+0000 7f94577fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9458106800 0x7f9458106c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:10.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.316+0000 7f94577fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9458106800 0x7f9458106c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39034/0 (socket says 172.21.15.37:39034) 2024-06-10T00:45:10.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.316+0000 7f94577fe700 1 -- 172.21.15.37:0/3954858840 learned_addr learned my addr 172.21.15.37:0/3954858840 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:10.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.316+0000 7f94577fe700 1 -- 172.21.15.37:0/3954858840 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9458107160 con 0x7f9458106800 2024-06-10T00:45:10.058 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.316+0000 7f94577fe700 1 --2- 172.21.15.37:0/3954858840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9458106800 0x7f9458106c20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f94480098a0 tx=0x7f9448009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e098dd8da5932beb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:10.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.316+0000 7f94567fc700 1 -- 172.21.15.37:0/3954858840 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9448004160 con 0x7f9458106800 2024-06-10T00:45:10.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.316+0000 7f94567fc700 1 -- 172.21.15.37:0/3954858840 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9448004300 con 0x7f9458106800 2024-06-10T00:45:10.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.316+0000 7f94567fc700 1 -- 172.21.15.37:0/3954858840 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9448011b20 con 0x7f9458106800 2024-06-10T00:45:10.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.317+0000 7f945da7c700 1 -- 172.21.15.37:0/3954858840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9458106800 msgr2=0x7f9458106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:10.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.317+0000 7f945da7c700 1 --2- 172.21.15.37:0/3954858840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9458106800 0x7f9458106c20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f94480098a0 tx=0x7f9448009bb0 comp rx=0 tx=0).stop 2024-06-10T00:45:10.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.317+0000 7f945da7c700 1 -- 172.21.15.37:0/3954858840 shutdown_connections 2024-06-10T00:45:10.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.317+0000 7f945da7c700 1 --2- 172.21.15.37:0/3954858840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9458106800 0x7f9458106c20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:10.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.317+0000 7f945da7c700 1 -- 172.21.15.37:0/3954858840 >> 172.21.15.37:0/3954858840 conn(0x7f9458102420 msgr2=0x7f9458104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:10.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.317+0000 7f945da7c700 1 -- 172.21.15.37:0/3954858840 shutdown_connections 2024-06-10T00:45:10.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.317+0000 7f945da7c700 1 -- 172.21.15.37:0/3954858840 wait complete. 2024-06-10T00:45:10.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.318+0000 7f945da7c700 1 Processor -- start 2024-06-10T00:45:10.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.318+0000 7f945da7c700 1 -- start start 2024-06-10T00:45:10.059 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.318+0000 7f945da7c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9458106800 0x7f945819e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:10.060 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.318+0000 7f945da7c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9458108230 con 0x7f9458106800 2024-06-10T00:45:10.060 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.318+0000 7f94577fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9458106800 0x7f945819e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:10.060 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.318+0000 7f94577fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9458106800 0x7f945819e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39040/0 (socket says 172.21.15.37:39040) 2024-06-10T00:45:10.060 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.318+0000 7f94577fe700 1 -- 172.21.15.37:0/3887351713 learned_addr learned my addr 172.21.15.37:0/3887351713 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:10.060 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.319+0000 7f94577fe700 1 -- 172.21.15.37:0/3887351713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f945819e5c0 con 0x7f9458106800 2024-06-10T00:45:10.060 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.319+0000 7f94577fe700 1 --2- 172.21.15.37:0/3887351713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9458106800 0x7f945819e080 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f94480117e0 tx=0x7f9448011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:10.060 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.319+0000 7f9454ff9700 1 -- 172.21.15.37:0/3887351713 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9448003ba0 con 0x7f9458106800 2024-06-10T00:45:10.060 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.319+0000 7f9454ff9700 1 -- 172.21.15.37:0/3887351713 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f944801a910 con 0x7f9458106800 2024-06-10T00:45:10.060 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.319+0000 7f945da7c700 1 -- 172.21.15.37:0/3887351713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f945819e850 con 0x7f9458106800 2024-06-10T00:45:10.060 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.319+0000 7f9454ff9700 1 -- 172.21.15.37:0/3887351713 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94480248f0 con 0x7f9458106800 2024-06-10T00:45:10.060 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.319+0000 7f945da7c700 1 -- 172.21.15.37:0/3887351713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f945819ecb0 con 0x7f9458106800 2024-06-10T00:45:10.060 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.320+0000 7f9454ff9700 1 -- 172.21.15.37:0/3887351713 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f9448022020 con 0x7f9458106800 2024-06-10T00:45:10.060 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.320+0000 7f945da7c700 1 -- 172.21.15.37:0/3887351713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9458063af0 con 0x7f9458106800 2024-06-10T00:45:10.061 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.320+0000 7f9454ff9700 1 --2- 172.21.15.37:0/3887351713 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f94440390a0 0x7f944403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:10.061 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.320+0000 7f9454ff9700 1 -- 172.21.15.37:0/3887351713 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f944804d740 con 0x7f9458106800 2024-06-10T00:45:10.061 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.321+0000 7f9456ffd700 1 --2- 172.21.15.37:0/3887351713 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f94440390a0 0x7f944403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:10.061 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.321+0000 7f9456ffd700 1 --2- 172.21.15.37:0/3887351713 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f94440390a0 0x7f944403b560 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9440006fd0 tx=0x7f9440006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:10.061 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.326+0000 7f9454ff9700 1 -- 172.21.15.37:0/3887351713 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f944802cdf0 con 0x7f9458106800 2024-06-10T00:45:10.061 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.537+0000 7f945da7c700 1 -- 172.21.15.37:0/3887351713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f94580630a0 con 0x7f9458106800 2024-06-10T00:45:10.061 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.543+0000 7f9454ff9700 1 -- 172.21.15.37:0/3887351713 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f944801ab30 con 0x7f9458106800 2024-06-10T00:45:10.061 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.546+0000 7f945da7c700 1 -- 172.21.15.37:0/3887351713 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f94440390a0 msgr2=0x7f944403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:10.061 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.546+0000 7f945da7c700 1 --2- 172.21.15.37:0/3887351713 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f94440390a0 0x7f944403b560 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9440006fd0 tx=0x7f9440006e40 comp rx=0 tx=0).stop 2024-06-10T00:45:10.061 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.546+0000 7f945da7c700 1 -- 172.21.15.37:0/3887351713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9458106800 msgr2=0x7f945819e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:10.061 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.546+0000 7f945da7c700 1 --2- 172.21.15.37:0/3887351713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9458106800 0x7f945819e080 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f94480117e0 tx=0x7f9448011ba0 comp rx=0 tx=0).stop 2024-06-10T00:45:10.061 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.546+0000 7f945da7c700 1 -- 172.21.15.37:0/3887351713 shutdown_connections 2024-06-10T00:45:10.062 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.546+0000 7f945da7c700 1 --2- 172.21.15.37:0/3887351713 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f94440390a0 0x7f944403b560 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:10.062 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.546+0000 7f945da7c700 1 --2- 172.21.15.37:0/3887351713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9458106800 0x7f945819e080 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:10.062 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.546+0000 7f945da7c700 1 -- 172.21.15.37:0/3887351713 >> 172.21.15.37:0/3887351713 conn(0x7f9458102420 msgr2=0x7f9458194c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:10.062 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.546+0000 7f945da7c700 1 -- 172.21.15.37:0/3887351713 shutdown_connections 2024-06-10T00:45:10.062 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:09.546+0000 7f945da7c700 1 -- 172.21.15.37:0/3887351713 wait complete. 2024-06-10T00:45:10.710 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:10 smithi037 ceph-mon[32192]: from='mgr.14116 172.21.15.37:0/4158559188' entity='mgr.x' 2024-06-10T00:45:10.711 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:10 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3887351713' entity='client.admin' 2024-06-10T00:45:11.332 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.474+0000 7f14e559a700 1 Processor -- start 2024-06-10T00:45:11.332 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.474+0000 7f14e559a700 1 -- start start 2024-06-10T00:45:11.332 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.475+0000 7f14e559a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14e0108a40 0x7f14e010ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:11.332 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.475+0000 7f14e559a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f14e007be70 con 0x7f14e0108a40 2024-06-10T00:45:11.332 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.475+0000 7f14deffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14e0108a40 0x7f14e010ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:11.332 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.475+0000 7f14deffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14e0108a40 0x7f14e010ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39056/0 (socket says 172.21.15.37:39056) 2024-06-10T00:45:11.332 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.475+0000 7f14deffd700 1 -- 172.21.15.37:0/1019935463 learned_addr learned my addr 172.21.15.37:0/1019935463 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:11.332 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.475+0000 7f14deffd700 1 -- 172.21.15.37:0/1019935463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e010b370 con 0x7f14e0108a40 2024-06-10T00:45:11.332 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.475+0000 7f14deffd700 1 --2- 172.21.15.37:0/1019935463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14e0108a40 0x7f14e010ae30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f14d00098a0 tx=0x7f14d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c2af5920039edc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:11.333 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.476+0000 7f14ddffb700 1 -- 172.21.15.37:0/1019935463 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14d0004160 con 0x7f14e0108a40 2024-06-10T00:45:11.333 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.476+0000 7f14ddffb700 1 -- 172.21.15.37:0/1019935463 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d0004300 con 0x7f14e0108a40 2024-06-10T00:45:11.333 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.476+0000 7f14ddffb700 1 -- 172.21.15.37:0/1019935463 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14d0011b20 con 0x7f14e0108a40 2024-06-10T00:45:11.333 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.476+0000 7f14e559a700 1 -- 172.21.15.37:0/1019935463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14e0108a40 msgr2=0x7f14e010ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:11.333 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.476+0000 7f14e559a700 1 --2- 172.21.15.37:0/1019935463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14e0108a40 0x7f14e010ae30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f14d00098a0 tx=0x7f14d0009bb0 comp rx=0 tx=0).stop 2024-06-10T00:45:11.333 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.476+0000 7f14e559a700 1 -- 172.21.15.37:0/1019935463 shutdown_connections 2024-06-10T00:45:11.333 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.476+0000 7f14e559a700 1 --2- 172.21.15.37:0/1019935463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14e0108a40 0x7f14e010ae30 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:11.333 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.476+0000 7f14e559a700 1 -- 172.21.15.37:0/1019935463 >> 172.21.15.37:0/1019935463 conn(0x7f14e0102450 msgr2=0x7f14e0104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:11.333 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.476+0000 7f14e559a700 1 -- 172.21.15.37:0/1019935463 shutdown_connections 2024-06-10T00:45:11.333 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.476+0000 7f14e559a700 1 -- 172.21.15.37:0/1019935463 wait complete. 2024-06-10T00:45:11.333 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.477+0000 7f14e559a700 1 Processor -- start 2024-06-10T00:45:11.333 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.477+0000 7f14e559a700 1 -- start start 2024-06-10T00:45:11.333 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.477+0000 7f14e559a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14e0108a40 0x7f14e019be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:11.333 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.477+0000 7f14e559a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f14e010c3d0 con 0x7f14e0108a40 2024-06-10T00:45:11.334 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.478+0000 7f14deffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14e0108a40 0x7f14e019be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:11.334 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.478+0000 7f14deffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14e0108a40 0x7f14e019be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39070/0 (socket says 172.21.15.37:39070) 2024-06-10T00:45:11.334 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.478+0000 7f14deffd700 1 -- 172.21.15.37:0/3445902642 learned_addr learned my addr 172.21.15.37:0/3445902642 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:11.334 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.478+0000 7f14deffd700 1 -- 172.21.15.37:0/3445902642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e019c3a0 con 0x7f14e0108a40 2024-06-10T00:45:11.334 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.478+0000 7f14deffd700 1 --2- 172.21.15.37:0/3445902642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14e0108a40 0x7f14e019be60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f14d00117e0 tx=0x7f14d0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:11.334 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.478+0000 7f14c7fff700 1 -- 172.21.15.37:0/3445902642 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14d0003bf0 con 0x7f14e0108a40 2024-06-10T00:45:11.334 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.478+0000 7f14e559a700 1 -- 172.21.15.37:0/3445902642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e019c630 con 0x7f14e0108a40 2024-06-10T00:45:11.334 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.478+0000 7f14c7fff700 1 -- 172.21.15.37:0/3445902642 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d001a910 con 0x7f14e0108a40 2024-06-10T00:45:11.334 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.478+0000 7f14c7fff700 1 -- 172.21.15.37:0/3445902642 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14d0024970 con 0x7f14e0108a40 2024-06-10T00:45:11.334 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.478+0000 7f14e559a700 1 -- 172.21.15.37:0/3445902642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e019ca90 con 0x7f14e0108a40 2024-06-10T00:45:11.334 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.479+0000 7f14c7fff700 1 -- 172.21.15.37:0/3445902642 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f14d0022020 con 0x7f14e0108a40 2024-06-10T00:45:11.334 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.479+0000 7f14e559a700 1 -- 172.21.15.37:0/3445902642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14e0063af0 con 0x7f14e0108a40 2024-06-10T00:45:11.334 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.480+0000 7f14c7fff700 1 --2- 172.21.15.37:0/3445902642 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f14c8039000 0x7f14c803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:11.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.480+0000 7f14c7fff700 1 -- 172.21.15.37:0/3445902642 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f14d004d350 con 0x7f14e0108a40 2024-06-10T00:45:11.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.480+0000 7f14de7fc700 1 --2- 172.21.15.37:0/3445902642 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f14c8039000 0x7f14c803b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:11.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.480+0000 7f14de7fc700 1 --2- 172.21.15.37:0/3445902642 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f14c8039000 0x7f14c803b4c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f14d4006fd0 tx=0x7f14d4006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:11.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.485+0000 7f14c7fff700 1 -- 172.21.15.37:0/3445902642 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f14d0011e20 con 0x7f14e0108a40 2024-06-10T00:45:11.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.710+0000 7f14e559a700 1 -- 172.21.15.37:0/3445902642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f14e019f520 con 0x7f14e0108a40 2024-06-10T00:45:11.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.713+0000 7f14c7fff700 1 -- 172.21.15.37:0/3445902642 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f14d000b3e0 con 0x7f14e0108a40 2024-06-10T00:45:11.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.715+0000 7f14e559a700 1 -- 172.21.15.37:0/3445902642 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f14c8039000 msgr2=0x7f14c803b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:11.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.715+0000 7f14e559a700 1 --2- 172.21.15.37:0/3445902642 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f14c8039000 0x7f14c803b4c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f14d4006fd0 tx=0x7f14d4006e40 comp rx=0 tx=0).stop 2024-06-10T00:45:11.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.715+0000 7f14e559a700 1 -- 172.21.15.37:0/3445902642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14e0108a40 msgr2=0x7f14e019be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:11.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.715+0000 7f14e559a700 1 --2- 172.21.15.37:0/3445902642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14e0108a40 0x7f14e019be60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f14d00117e0 tx=0x7f14d0011c10 comp rx=0 tx=0).stop 2024-06-10T00:45:11.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.716+0000 7f14e559a700 1 -- 172.21.15.37:0/3445902642 shutdown_connections 2024-06-10T00:45:11.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.716+0000 7f14e559a700 1 --2- 172.21.15.37:0/3445902642 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f14c8039000 0x7f14c803b4c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:11.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.716+0000 7f14e559a700 1 --2- 172.21.15.37:0/3445902642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14e0108a40 0x7f14e019be60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:11.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.716+0000 7f14e559a700 1 -- 172.21.15.37:0/3445902642 >> 172.21.15.37:0/3445902642 conn(0x7f14e0102450 msgr2=0x7f14e0102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:11.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.716+0000 7f14e559a700 1 -- 172.21.15.37:0/3445902642 shutdown_connections 2024-06-10T00:45:11.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:10.716+0000 7f14e559a700 1 -- 172.21.15.37:0/3445902642 wait complete. 2024-06-10T00:45:11.336 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling the dashboard module... 2024-06-10T00:45:11.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:11 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3445902642' entity='client.admin' 2024-06-10T00:45:13.384 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:13 smithi037 ceph-mon[32192]: from='mgr.14116 172.21.15.37:0/4158559188' entity='mgr.x' 2024-06-10T00:45:13.384 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:13 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/4235006025' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-06-10T00:45:13.384 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:45:13 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: ignoring --setuser ceph since I am not root 2024-06-10T00:45:13.384 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:45:13 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: ignoring --setgroup ceph since I am not root 2024-06-10T00:45:13.384 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:45:13 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:45:13.256+0000 7fcbcf90d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-10T00:45:13.645 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:45:13 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:45:13.455+0000 7fcbcf90d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:45:13.646 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:45:13 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:45:13.561+0000 7fcbcf90d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-10T00:45:13.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.829+0000 7f07797dd700 1 Processor -- start 2024-06-10T00:45:13.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.829+0000 7f07797dd700 1 -- start start 2024-06-10T00:45:13.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.830+0000 7f07797dd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0774107fc0 0x7f07741083e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:13.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.830+0000 7f07797dd700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0774108920 con 0x7f0774107fc0 2024-06-10T00:45:13.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.830+0000 7f0772ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0774107fc0 0x7f07741083e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:13.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.830+0000 7f0772ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0774107fc0 0x7f07741083e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51808/0 (socket says 172.21.15.37:51808) 2024-06-10T00:45:13.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.830+0000 7f0772ffd700 1 -- 172.21.15.37:0/917890547 learned_addr learned my addr 172.21.15.37:0/917890547 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:13.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.830+0000 7f0772ffd700 1 -- 172.21.15.37:0/917890547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0774109080 con 0x7f0774107fc0 2024-06-10T00:45:13.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.830+0000 7f0772ffd700 1 --2- 172.21.15.37:0/917890547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0774107fc0 0x7f07741083e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f07640098a0 tx=0x7f0764009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4313688e524f6b0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:13.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.830+0000 7f0771ffb700 1 -- 172.21.15.37:0/917890547 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0764004160 con 0x7f0774107fc0 2024-06-10T00:45:13.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.831+0000 7f0771ffb700 1 -- 172.21.15.37:0/917890547 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0764004300 con 0x7f0774107fc0 2024-06-10T00:45:13.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.831+0000 7f0771ffb700 1 -- 172.21.15.37:0/917890547 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0764011b20 con 0x7f0774107fc0 2024-06-10T00:45:13.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.831+0000 7f07797dd700 1 -- 172.21.15.37:0/917890547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0774107fc0 msgr2=0x7f07741083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:13.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.831+0000 7f07797dd700 1 --2- 172.21.15.37:0/917890547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0774107fc0 0x7f07741083e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f07640098a0 tx=0x7f0764009bb0 comp rx=0 tx=0).stop 2024-06-10T00:45:13.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.831+0000 7f07797dd700 1 -- 172.21.15.37:0/917890547 shutdown_connections 2024-06-10T00:45:13.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.831+0000 7f07797dd700 1 --2- 172.21.15.37:0/917890547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0774107fc0 0x7f07741083e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:13.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.831+0000 7f07797dd700 1 -- 172.21.15.37:0/917890547 >> 172.21.15.37:0/917890547 conn(0x7f0774103560 msgr2=0x7f07741059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:13.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.831+0000 7f07797dd700 1 -- 172.21.15.37:0/917890547 shutdown_connections 2024-06-10T00:45:13.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.831+0000 7f07797dd700 1 -- 172.21.15.37:0/917890547 wait complete. 2024-06-10T00:45:13.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.832+0000 7f07797dd700 1 Processor -- start 2024-06-10T00:45:13.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.832+0000 7f07797dd700 1 -- start start 2024-06-10T00:45:13.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.832+0000 7f07797dd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0774107fc0 0x7f077419be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:13.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.832+0000 7f07797dd700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f077410c4c0 con 0x7f0774107fc0 2024-06-10T00:45:13.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.832+0000 7f0772ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0774107fc0 0x7f077419be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:13.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.832+0000 7f0772ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0774107fc0 0x7f077419be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51814/0 (socket says 172.21.15.37:51814) 2024-06-10T00:45:13.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.832+0000 7f0772ffd700 1 -- 172.21.15.37:0/4235006025 learned_addr learned my addr 172.21.15.37:0/4235006025 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:13.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.833+0000 7f0772ffd700 1 -- 172.21.15.37:0/4235006025 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077419c360 con 0x7f0774107fc0 2024-06-10T00:45:13.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.833+0000 7f0772ffd700 1 --2- 172.21.15.37:0/4235006025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0774107fc0 0x7f077419be20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f07640117e0 tx=0x7f0764011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:13.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.833+0000 7f076bfff700 1 -- 172.21.15.37:0/4235006025 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0764003bf0 con 0x7f0774107fc0 2024-06-10T00:45:13.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.833+0000 7f076bfff700 1 -- 172.21.15.37:0/4235006025 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f076401a910 con 0x7f0774107fc0 2024-06-10T00:45:13.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.833+0000 7f07797dd700 1 -- 172.21.15.37:0/4235006025 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f077419c5f0 con 0x7f0774107fc0 2024-06-10T00:45:13.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.833+0000 7f076bfff700 1 -- 172.21.15.37:0/4235006025 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0764024970 con 0x7f0774107fc0 2024-06-10T00:45:13.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.833+0000 7f07797dd700 1 -- 172.21.15.37:0/4235006025 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f077419ca50 con 0x7f0774107fc0 2024-06-10T00:45:13.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.834+0000 7f076bfff700 1 -- 172.21.15.37:0/4235006025 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f0764022020 con 0x7f0774107fc0 2024-06-10T00:45:13.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.834+0000 7f07797dd700 1 -- 172.21.15.37:0/4235006025 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0774063af0 con 0x7f0774107fc0 2024-06-10T00:45:13.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.834+0000 7f076bfff700 1 --2- 172.21.15.37:0/4235006025 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f0760039050 0x7f076003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:13.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.834+0000 7f076bfff700 1 -- 172.21.15.37:0/4235006025 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f076404d330 con 0x7f0774107fc0 2024-06-10T00:45:13.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.834+0000 7f07727fc700 1 --2- 172.21.15.37:0/4235006025 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f0760039050 0x7f076003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:13.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.835+0000 7f07727fc700 1 --2- 172.21.15.37:0/4235006025 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f0760039050 0x7f076003b510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f075c006fd0 tx=0x7f075c006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:13.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:11.839+0000 7f076bfff700 1 -- 172.21.15.37:0/4235006025 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0764011e20 con 0x7f0774107fc0 2024-06-10T00:45:13.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:12.121+0000 7f07797dd700 1 -- 172.21.15.37:0/4235006025 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f077419f420 con 0x7f0774107fc0 2024-06-10T00:45:13.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:13.067+0000 7f076bfff700 1 -- 172.21.15.37:0/4235006025 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f076402ddb0 con 0x7f0774107fc0 2024-06-10T00:45:13.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:13.069+0000 7f076bfff700 1 -- 172.21.15.37:0/4235006025 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f076401a430 con 0x7f0774107fc0 2024-06-10T00:45:13.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:13.070+0000 7f07797dd700 1 -- 172.21.15.37:0/4235006025 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f0760039050 msgr2=0x7f076003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:13.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:13.070+0000 7f07797dd700 1 --2- 172.21.15.37:0/4235006025 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f0760039050 0x7f076003b510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f075c006fd0 tx=0x7f075c006e40 comp rx=0 tx=0).stop 2024-06-10T00:45:13.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:13.070+0000 7f07797dd700 1 -- 172.21.15.37:0/4235006025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0774107fc0 msgr2=0x7f077419be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:13.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:13.070+0000 7f07797dd700 1 --2- 172.21.15.37:0/4235006025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0774107fc0 0x7f077419be20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f07640117e0 tx=0x7f0764011c10 comp rx=0 tx=0).stop 2024-06-10T00:45:13.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:13.070+0000 7f07797dd700 1 -- 172.21.15.37:0/4235006025 shutdown_connections 2024-06-10T00:45:13.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:13.070+0000 7f07797dd700 1 --2- 172.21.15.37:0/4235006025 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f0760039050 0x7f076003b510 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:13.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:13.070+0000 7f07797dd700 1 --2- 172.21.15.37:0/4235006025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0774107fc0 0x7f077419be20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:13.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:13.070+0000 7f07797dd700 1 -- 172.21.15.37:0/4235006025 >> 172.21.15.37:0/4235006025 conn(0x7f0774103560 msgr2=0x7f0774103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:13.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:13.071+0000 7f07797dd700 1 -- 172.21.15.37:0/4235006025 shutdown_connections 2024-06-10T00:45:13.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:13.071+0000 7f07797dd700 1 -- 172.21.15.37:0/4235006025 wait complete. 2024-06-10T00:45:13.909 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:45:13 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:45:13.894+0000 7fcbcf90d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-10T00:45:14.314 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:14 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/4235006025' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-06-10T00:45:14.315 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:14 smithi037 ceph-mon[32192]: mgrmap e9: x(active, since 19s) 2024-06-10T00:45:14.315 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:45:13 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:45:13.985+0000 7fcbcf90d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-10T00:45:14.964 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-06-10T00:45:14.964 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-06-10T00:45:14.964 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-10T00:45:14.964 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-10T00:45:14.964 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-10T00:45:14.964 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-06-10T00:45:14.965 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.108+0000 7f9ddf0bb700 1 Processor -- start 2024-06-10T00:45:14.965 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.108+0000 7f9ddf0bb700 1 -- start start 2024-06-10T00:45:14.965 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.108+0000 7f9ddf0bb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dd807e750 0x7f9dd807cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:14.965 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.108+0000 7f9ddf0bb700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd807d180 con 0x7f9dd807e750 2024-06-10T00:45:14.965 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.109+0000 7f9ddce57700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dd807e750 0x7f9dd807cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:14.965 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.109+0000 7f9ddce57700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dd807e750 0x7f9dd807cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51842/0 (socket says 172.21.15.37:51842) 2024-06-10T00:45:14.965 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.109+0000 7f9ddce57700 1 -- 172.21.15.37:0/428390076 learned_addr learned my addr 172.21.15.37:0/428390076 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:14.965 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.109+0000 7f9ddce57700 1 -- 172.21.15.37:0/428390076 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd807d300 con 0x7f9dd807e750 2024-06-10T00:45:14.965 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.109+0000 7f9ddce57700 1 --2- 172.21.15.37:0/428390076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dd807e750 0x7f9dd807cbb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0098a0 tx=0x7f9dcc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b3fe52ab2d5c87 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:14.965 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.109+0000 7f9dd77fe700 1 -- 172.21.15.37:0/428390076 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dcc004160 con 0x7f9dd807e750 2024-06-10T00:45:14.965 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.109+0000 7f9dd77fe700 1 -- 172.21.15.37:0/428390076 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dcc004300 con 0x7f9dd807e750 2024-06-10T00:45:14.965 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.109+0000 7f9dd77fe700 1 -- 172.21.15.37:0/428390076 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dcc011b20 con 0x7f9dd807e750 2024-06-10T00:45:14.965 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.110+0000 7f9ddf0bb700 1 -- 172.21.15.37:0/428390076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dd807e750 msgr2=0x7f9dd807cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:14.965 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.110+0000 7f9ddf0bb700 1 --2- 172.21.15.37:0/428390076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dd807e750 0x7f9dd807cbb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0098a0 tx=0x7f9dcc009bb0 comp rx=0 tx=0).stop 2024-06-10T00:45:14.966 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.110+0000 7f9ddf0bb700 1 -- 172.21.15.37:0/428390076 shutdown_connections 2024-06-10T00:45:14.966 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.110+0000 7f9ddf0bb700 1 --2- 172.21.15.37:0/428390076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dd807e750 0x7f9dd807cbb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:14.966 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.110+0000 7f9ddf0bb700 1 -- 172.21.15.37:0/428390076 >> 172.21.15.37:0/428390076 conn(0x7f9dd8103530 msgr2=0x7f9dd8105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:14.966 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.110+0000 7f9ddf0bb700 1 -- 172.21.15.37:0/428390076 shutdown_connections 2024-06-10T00:45:14.966 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.110+0000 7f9ddf0bb700 1 -- 172.21.15.37:0/428390076 wait complete. 2024-06-10T00:45:14.966 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.111+0000 7f9ddf0bb700 1 Processor -- start 2024-06-10T00:45:14.966 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.111+0000 7f9ddf0bb700 1 -- start start 2024-06-10T00:45:14.966 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.111+0000 7f9ddf0bb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dd807e750 0x7f9dd819e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:14.966 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.111+0000 7f9ddf0bb700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd810e570 con 0x7f9dd807e750 2024-06-10T00:45:14.966 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.111+0000 7f9ddce57700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dd807e750 0x7f9dd819e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:14.966 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.111+0000 7f9ddce57700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dd807e750 0x7f9dd819e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51846/0 (socket says 172.21.15.37:51846) 2024-06-10T00:45:14.966 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.111+0000 7f9ddce57700 1 -- 172.21.15.37:0/1814798810 learned_addr learned my addr 172.21.15.37:0/1814798810 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:14.966 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.111+0000 7f9ddce57700 1 -- 172.21.15.37:0/1814798810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd819e5b0 con 0x7f9dd807e750 2024-06-10T00:45:14.966 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.111+0000 7f9ddce57700 1 --2- 172.21.15.37:0/1814798810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dd807e750 0x7f9dd819e070 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0117e0 tx=0x7f9dcc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:14.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.112+0000 7f9dd5ffb700 1 -- 172.21.15.37:0/1814798810 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dcc003bf0 con 0x7f9dd807e750 2024-06-10T00:45:14.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.112+0000 7f9dd5ffb700 1 -- 172.21.15.37:0/1814798810 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dcc01a910 con 0x7f9dd807e750 2024-06-10T00:45:14.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.112+0000 7f9ddf0bb700 1 -- 172.21.15.37:0/1814798810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd819e840 con 0x7f9dd807e750 2024-06-10T00:45:14.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.112+0000 7f9dd5ffb700 1 -- 172.21.15.37:0/1814798810 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dcc0249a0 con 0x7f9dd807e750 2024-06-10T00:45:14.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.112+0000 7f9ddf0bb700 1 -- 172.21.15.37:0/1814798810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd819eca0 con 0x7f9dd807e750 2024-06-10T00:45:14.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.113+0000 7f9dd5ffb700 1 -- 172.21.15.37:0/1814798810 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f9dcc022020 con 0x7f9dd807e750 2024-06-10T00:45:14.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.113+0000 7f9ddf0bb700 1 -- 172.21.15.37:0/1814798810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dd8063af0 con 0x7f9dd807e750 2024-06-10T00:45:14.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.113+0000 7f9dd5ffb700 1 --2- 172.21.15.37:0/1814798810 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f9dc80390b0 0x7f9dc803b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:14.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.113+0000 7f9dd5ffb700 1 -- 172.21.15.37:0/1814798810 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9dcc04d5a0 con 0x7f9dd807e750 2024-06-10T00:45:14.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.113+0000 7f9dd7fff700 1 -- 172.21.15.37:0/1814798810 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f9dc80390b0 msgr2=0x7f9dc803b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2766692263 2024-06-10T00:45:14.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.113+0000 7f9dd7fff700 1 --2- 172.21.15.37:0/1814798810 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f9dc80390b0 0x7f9dc803b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T00:45:14.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.118+0000 7f9dd5ffb700 1 -- 172.21.15.37:0/1814798810 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9dcc011e20 con 0x7f9dd807e750 2024-06-10T00:45:14.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.314+0000 7f9dd7fff700 1 -- 172.21.15.37:0/1814798810 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f9dc80390b0 msgr2=0x7f9dc803b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2766692263 2024-06-10T00:45:14.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.314+0000 7f9dd7fff700 1 --2- 172.21.15.37:0/1814798810 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f9dc80390b0 0x7f9dc803b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-10T00:45:14.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.393+0000 7f9ddf0bb700 1 -- 172.21.15.37:0/1814798810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f9dd81a1710 con 0x7f9dd807e750 2024-06-10T00:45:14.968 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.394+0000 7f9dd5ffb700 1 -- 172.21.15.37:0/1814798810 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f9dcc00ba00 con 0x7f9dd807e750 2024-06-10T00:45:14.968 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.398+0000 7f9ddf0bb700 1 -- 172.21.15.37:0/1814798810 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f9dc80390b0 msgr2=0x7f9dc803b570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T00:45:14.968 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.398+0000 7f9ddf0bb700 1 --2- 172.21.15.37:0/1814798810 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f9dc80390b0 0x7f9dc803b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:14.968 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.398+0000 7f9ddf0bb700 1 -- 172.21.15.37:0/1814798810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dd807e750 msgr2=0x7f9dd819e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:14.968 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.398+0000 7f9ddf0bb700 1 --2- 172.21.15.37:0/1814798810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dd807e750 0x7f9dd819e070 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0117e0 tx=0x7f9dcc011c10 comp rx=0 tx=0).stop 2024-06-10T00:45:14.968 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.398+0000 7f9ddf0bb700 1 -- 172.21.15.37:0/1814798810 shutdown_connections 2024-06-10T00:45:14.968 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.398+0000 7f9ddf0bb700 1 --2- 172.21.15.37:0/1814798810 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f9dc80390b0 0x7f9dc803b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:14.968 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.398+0000 7f9ddf0bb700 1 --2- 172.21.15.37:0/1814798810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dd807e750 0x7f9dd819e070 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0117e0 tx=0x7f9dcc011c10 comp rx=0 tx=0).stop 2024-06-10T00:45:14.968 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.398+0000 7f9ddf0bb700 1 -- 172.21.15.37:0/1814798810 >> 172.21.15.37:0/1814798810 conn(0x7f9dd8103530 msgr2=0x7f9dd8103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:14.968 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.399+0000 7f9ddf0bb700 1 -- 172.21.15.37:0/1814798810 shutdown_connections 2024-06-10T00:45:14.968 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:14.399+0000 7f9ddf0bb700 1 -- 172.21.15.37:0/1814798810 wait complete. 2024-06-10T00:45:14.968 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for the mgr to restart... 2024-06-10T00:45:14.968 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr epoch 9... 2024-06-10T00:45:15.359 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:15 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1814798810' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-10T00:45:15.360 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:45:15 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:45:15.266+0000 7fcbcf90d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-10T00:45:15.626 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:45:15 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:45:15.448+0000 7fcbcf90d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-10T00:45:15.626 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:45:15 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:45:15.625+0000 7fcbcf90d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-10T00:45:16.234 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:45:15 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:45:15.927+0000 7fcbcf90d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-10T00:45:16.235 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:45:16 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:45:16.023+0000 7fcbcf90d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-10T00:45:16.235 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:45:16 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:45:16.234+0000 7fcbcf90d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-10T00:45:16.645 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:45:16 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:45:16.435+0000 7fcbcf90d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:45:17.395 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:45:17 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:45:17.065+0000 7fcbcf90d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-10T00:45:17.820 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:45:17 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:45:17.508+0000 7fcbcf90d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-10T00:45:17.820 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:45:17 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:45:17.609+0000 7fcbcf90d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-10T00:45:17.820 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:45:17 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:45:17.819+0000 7fcbcf90d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-10T00:45:18.645 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:45:18 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:45:18.367+0000 7fcbcf90d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-10T00:45:18.645 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:45:18 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:45:18.463+0000 7fcbcf90d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-10T00:45:19.084 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:45:18 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:45:18.717+0000 7fcbcf90d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-10T00:45:19.084 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:45:18 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:45:18.858+0000 7fcbcf90d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-10T00:45:19.084 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:45:18 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:45:18.962+0000 7fcbcf90d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-10T00:45:19.084 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:45:19 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:45:19.083+0000 7fcbcf90d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-10T00:45:19.645 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:45:19 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:45:19.394+0000 7fcbcf90d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-10T00:45:20.305 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:45:20 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:45:20.024+0000 7fcbcf90d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-10T00:45:20.305 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:45:20 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:45:20.252+0000 7fcbcf90d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-10T00:45:20.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:20 smithi037 ceph-mon[32192]: Active manager daemon x restarted 2024-06-10T00:45:20.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:20 smithi037 ceph-mon[32192]: Activating manager daemon x 2024-06-10T00:45:20.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:20 smithi037 ceph-mon[32192]: osdmap e3: 0 total, 0 up, 0 in 2024-06-10T00:45:20.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:20 smithi037 ceph-mon[32192]: mgrmap e10: x(active, starting, since 0.00347506s) 2024-06-10T00:45:20.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:20 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:45:20.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:20 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-10T00:45:20.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:20 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-10T00:45:20.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:20 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-10T00:45:20.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:20 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-10T00:45:20.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:20 smithi037 ceph-mon[32192]: Manager daemon x is now available 2024-06-10T00:45:20.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:20 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:45:21.475 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-10T00:45:21.476 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-10T00:45:21.476 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:21 smithi037 ceph-mon[32192]: mgrmap e11: x(active, since 1.00651s) 2024-06-10T00:45:22.183 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-06-10T00:45:22.184 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-06-10T00:45:22.184 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-10T00:45:22.184 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-06-10T00:45:22.184 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.371+0000 7f37e85f8700 1 Processor -- start 2024-06-10T00:45:22.184 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.371+0000 7f37e85f8700 1 -- start start 2024-06-10T00:45:22.184 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.371+0000 7f37e85f8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e01073a0 0x7f37e01077c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:22.184 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.371+0000 7f37e85f8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f37e0107d90 con 0x7f37e01073a0 2024-06-10T00:45:22.184 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.372+0000 7f37e6394700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e01073a0 0x7f37e01077c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:22.184 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.372+0000 7f37e6394700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e01073a0 0x7f37e01077c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51856/0 (socket says 172.21.15.37:51856) 2024-06-10T00:45:22.184 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.372+0000 7f37e6394700 1 -- 172.21.15.37:0/185550909 learned_addr learned my addr 172.21.15.37:0/185550909 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:22.185 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.372+0000 7f37e6394700 1 -- 172.21.15.37:0/185550909 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e01080c0 con 0x7f37e01073a0 2024-06-10T00:45:22.185 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.372+0000 7f37e6394700 1 --2- 172.21.15.37:0/185550909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e01073a0 0x7f37e01077c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0098a0 tx=0x7f37dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b84e63ed6791a7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:22.185 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.372+0000 7f37e5392700 1 -- 172.21.15.37:0/185550909 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37dc004160 con 0x7f37e01073a0 2024-06-10T00:45:22.185 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.372+0000 7f37e5392700 1 -- 172.21.15.37:0/185550909 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37dc004300 con 0x7f37e01073a0 2024-06-10T00:45:22.185 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.372+0000 7f37e5392700 1 -- 172.21.15.37:0/185550909 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37dc011b20 con 0x7f37e01073a0 2024-06-10T00:45:22.185 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.373+0000 7f37e85f8700 1 -- 172.21.15.37:0/185550909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e01073a0 msgr2=0x7f37e01077c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:22.185 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.373+0000 7f37e85f8700 1 --2- 172.21.15.37:0/185550909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e01073a0 0x7f37e01077c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0098a0 tx=0x7f37dc009bb0 comp rx=0 tx=0).stop 2024-06-10T00:45:22.185 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.373+0000 7f37e85f8700 1 -- 172.21.15.37:0/185550909 shutdown_connections 2024-06-10T00:45:22.185 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.373+0000 7f37e85f8700 1 --2- 172.21.15.37:0/185550909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e01073a0 0x7f37e01077c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:22.185 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.373+0000 7f37e85f8700 1 -- 172.21.15.37:0/185550909 >> 172.21.15.37:0/185550909 conn(0x7f37e0101c60 msgr2=0x7f37e0104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:22.185 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.373+0000 7f37e85f8700 1 -- 172.21.15.37:0/185550909 shutdown_connections 2024-06-10T00:45:22.185 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.373+0000 7f37e85f8700 1 -- 172.21.15.37:0/185550909 wait complete. 2024-06-10T00:45:22.185 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.374+0000 7f37e85f8700 1 Processor -- start 2024-06-10T00:45:22.186 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.374+0000 7f37e85f8700 1 -- start start 2024-06-10T00:45:22.186 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.374+0000 7f37e85f8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e01073a0 0x7f37e0192020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:22.186 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.374+0000 7f37e85f8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f37e010b470 con 0x7f37e01073a0 2024-06-10T00:45:22.186 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.374+0000 7f37e6394700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e01073a0 0x7f37e0192020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:22.186 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.374+0000 7f37e6394700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e01073a0 0x7f37e0192020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51862/0 (socket says 172.21.15.37:51862) 2024-06-10T00:45:22.186 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.374+0000 7f37e6394700 1 -- 172.21.15.37:0/3961880474 learned_addr learned my addr 172.21.15.37:0/3961880474 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:22.186 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.374+0000 7f37e6394700 1 -- 172.21.15.37:0/3961880474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e0192560 con 0x7f37e01073a0 2024-06-10T00:45:22.186 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.374+0000 7f37e6394700 1 --2- 172.21.15.37:0/3961880474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e01073a0 0x7f37e0192020 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f37dc011740 tx=0x7f37dc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:22.186 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.375+0000 7f37d37fe700 1 -- 172.21.15.37:0/3961880474 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37dc003a40 con 0x7f37e01073a0 2024-06-10T00:45:22.186 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.375+0000 7f37d37fe700 1 -- 172.21.15.37:0/3961880474 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37dc003be0 con 0x7f37e01073a0 2024-06-10T00:45:22.186 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.375+0000 7f37d37fe700 1 -- 172.21.15.37:0/3961880474 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37dc01a750 con 0x7f37e01073a0 2024-06-10T00:45:22.186 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.375+0000 7f37e85f8700 1 -- 172.21.15.37:0/3961880474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e01907a0 con 0x7f37e01073a0 2024-06-10T00:45:22.186 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.375+0000 7f37e85f8700 1 -- 172.21.15.37:0/3961880474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e0190c00 con 0x7f37e01073a0 2024-06-10T00:45:22.187 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.376+0000 7f37d37fe700 1 -- 172.21.15.37:0/3961880474 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f37dc024aa0 con 0x7f37e01073a0 2024-06-10T00:45:22.187 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.376+0000 7f37d37fe700 1 --2- 172.21.15.37:0/3961880474 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f37cc0390b0 0x7f37cc03b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:22.187 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.376+0000 7f37d37fe700 1 -- 172.21.15.37:0/3961880474 --> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f37cc03bc80 con 0x7f37cc0390b0 2024-06-10T00:45:22.187 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.376+0000 7f37d37fe700 1 -- 172.21.15.37:0/3961880474 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f37dc04d690 con 0x7f37e01073a0 2024-06-10T00:45:22.187 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.376+0000 7f37e5b93700 1 -- 172.21.15.37:0/3961880474 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f37cc0390b0 msgr2=0x7f37cc03b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2766692263 2024-06-10T00:45:22.187 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.376+0000 7f37e5b93700 1 --2- 172.21.15.37:0/3961880474 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f37cc0390b0 0x7f37cc03b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T00:45:22.187 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.577+0000 7f37e5b93700 1 -- 172.21.15.37:0/3961880474 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f37cc0390b0 msgr2=0x7f37cc03b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2766692263 2024-06-10T00:45:22.187 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.577+0000 7f37e5b93700 1 --2- 172.21.15.37:0/3961880474 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f37cc0390b0 0x7f37cc03b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-10T00:45:22.187 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.978+0000 7f37e5b93700 1 -- 172.21.15.37:0/3961880474 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f37cc0390b0 msgr2=0x7f37cc03b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2766692263 2024-06-10T00:45:22.187 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:15.978+0000 7f37e5b93700 1 --2- 172.21.15.37:0/3961880474 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f37cc0390b0 0x7f37cc03b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-10T00:45:22.187 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:16.779+0000 7f37e5b93700 1 -- 172.21.15.37:0/3961880474 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f37cc0390b0 msgr2=0x7f37cc03b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2766692263 2024-06-10T00:45:22.187 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:16.779+0000 7f37e5b93700 1 --2- 172.21.15.37:0/3961880474 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f37cc0390b0 0x7f37cc03b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-10T00:45:22.187 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:18.380+0000 7f37e5b93700 1 -- 172.21.15.37:0/3961880474 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f37cc0390b0 msgr2=0x7f37cc03b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2766692263 2024-06-10T00:45:22.188 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:18.380+0000 7f37e5b93700 1 --2- 172.21.15.37:0/3961880474 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f37cc0390b0 0x7f37cc03b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-10T00:45:22.188 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:20.255+0000 7f37d37fe700 1 -- 172.21.15.37:0/3961880474 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f37dc024430 con 0x7f37e01073a0 2024-06-10T00:45:22.188 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:20.255+0000 7f37d37fe700 1 -- 172.21.15.37:0/3961880474 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f37cc0390b0 msgr2=0x7f37cc03b570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T00:45:22.188 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:20.255+0000 7f37d37fe700 1 --2- 172.21.15.37:0/3961880474 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f37cc0390b0 0x7f37cc03b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:22.188 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:21.257+0000 7f37d37fe700 1 -- 172.21.15.37:0/3961880474 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f37dc04f340 con 0x7f37e01073a0 2024-06-10T00:45:22.188 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:21.257+0000 7f37d37fe700 1 --2- 172.21.15.37:0/3961880474 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f37cc03cb70 0x7f37cc03ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:22.188 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:21.257+0000 7f37d37fe700 1 -- 172.21.15.37:0/3961880474 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f37dc008ea0 con 0x7f37cc03cb70 2024-06-10T00:45:22.188 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:21.258+0000 7f37e5b93700 1 --2- 172.21.15.37:0/3961880474 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f37cc03cb70 0x7f37cc03ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:22.188 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:21.258+0000 7f37e5b93700 1 --2- 172.21.15.37:0/3961880474 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f37cc03cb70 0x7f37cc03ef60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f37d4003a10 tx=0x7f37d4009270 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:22.188 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:21.259+0000 7f37d37fe700 1 -- 172.21.15.37:0/3961880474 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f37dc008ea0 con 0x7f37cc03cb70 2024-06-10T00:45:22.188 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:21.272+0000 7f37e85f8700 1 -- 172.21.15.37:0/3961880474 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f37e00695b0 con 0x7f37cc03cb70 2024-06-10T00:45:22.188 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:21.272+0000 7f37d37fe700 1 -- 172.21.15.37:0/3961880474 <== mgr.14148 v2:172.21.15.37:6800/2387001309 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f37e00695b0 con 0x7f37cc03cb70 2024-06-10T00:45:22.188 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:21.272+0000 7f37e85f8700 1 -- 172.21.15.37:0/3961880474 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f37cc03cb70 msgr2=0x7f37cc03ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:22.189 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:21.272+0000 7f37e85f8700 1 --2- 172.21.15.37:0/3961880474 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f37cc03cb70 0x7f37cc03ef60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f37d4003a10 tx=0x7f37d4009270 comp rx=0 tx=0).stop 2024-06-10T00:45:22.189 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:21.272+0000 7f37e85f8700 1 -- 172.21.15.37:0/3961880474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e01073a0 msgr2=0x7f37e0192020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:22.189 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:21.272+0000 7f37e85f8700 1 --2- 172.21.15.37:0/3961880474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e01073a0 0x7f37e0192020 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f37dc011740 tx=0x7f37dc011ba0 comp rx=0 tx=0).stop 2024-06-10T00:45:22.189 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:21.272+0000 7f37e85f8700 1 -- 172.21.15.37:0/3961880474 shutdown_connections 2024-06-10T00:45:22.189 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:21.272+0000 7f37e85f8700 1 --2- 172.21.15.37:0/3961880474 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f37cc03cb70 0x7f37cc03ef60 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:22.189 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:21.272+0000 7f37e85f8700 1 --2- 172.21.15.37:0/3961880474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e01073a0 0x7f37e0192020 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:22.189 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:21.272+0000 7f37e85f8700 1 --2- 172.21.15.37:0/3961880474 >> [v2:172.21.15.37:6800/2766692263,v1:172.21.15.37:6801/2766692263] conn(0x7f37cc0390b0 0x7f37cc03b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:22.189 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:21.272+0000 7f37e85f8700 1 -- 172.21.15.37:0/3961880474 >> 172.21.15.37:0/3961880474 conn(0x7f37e0101c60 msgr2=0x7f37e0102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:22.189 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:21.273+0000 7f37e85f8700 1 -- 172.21.15.37:0/3961880474 shutdown_connections 2024-06-10T00:45:22.189 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:21.273+0000 7f37e85f8700 1 -- 172.21.15.37:0/3961880474 wait complete. 2024-06-10T00:45:22.189 INFO:teuthology.orchestra.run.smithi037.stdout:mgr epoch 9 is available 2024-06-10T00:45:22.189 INFO:teuthology.orchestra.run.smithi037.stdout:Generating a dashboard self-signed certificate... 2024-06-10T00:45:23.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:22 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:23.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:22 smithi037 ceph-mon[32192]: [10/Jun/2024:00:45:21] ENGINE Bus STARTING 2024-06-10T00:45:23.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:22 smithi037 ceph-mon[32192]: [10/Jun/2024:00:45:21] ENGINE Serving on https://172.21.15.37:7150 2024-06-10T00:45:23.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:22 smithi037 ceph-mon[32192]: [10/Jun/2024:00:45:22] ENGINE Serving on http://172.21.15.37:8765 2024-06-10T00:45:23.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:22 smithi037 ceph-mon[32192]: [10/Jun/2024:00:45:22] ENGINE Bus STARTED 2024-06-10T00:45:23.522 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-06-10T00:45:23.522 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.592+0000 7f5889755700 1 Processor -- start 2024-06-10T00:45:23.522 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.592+0000 7f5889755700 1 -- start start 2024-06-10T00:45:23.522 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.592+0000 7f5889755700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5884108a00 0x7f588410adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:23.522 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.592+0000 7f5889755700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f588407beb0 con 0x7f5884108a00 2024-06-10T00:45:23.523 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.593+0000 7f5882ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5884108a00 0x7f588410adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:23.523 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.593+0000 7f5882ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5884108a00 0x7f588410adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53966/0 (socket says 172.21.15.37:53966) 2024-06-10T00:45:23.523 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.593+0000 7f5882ffd700 1 -- 172.21.15.37:0/1499253831 learned_addr learned my addr 172.21.15.37:0/1499253831 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:23.523 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.593+0000 7f5882ffd700 1 -- 172.21.15.37:0/1499253831 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588410b330 con 0x7f5884108a00 2024-06-10T00:45:23.523 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.593+0000 7f5882ffd700 1 --2- 172.21.15.37:0/1499253831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5884108a00 0x7f588410adf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f58740098a0 tx=0x7f5874009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d758cd24b0e73c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:23.523 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.593+0000 7f58827fc700 1 -- 172.21.15.37:0/1499253831 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5874004160 con 0x7f5884108a00 2024-06-10T00:45:23.523 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.593+0000 7f58827fc700 1 -- 172.21.15.37:0/1499253831 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5874004300 con 0x7f5884108a00 2024-06-10T00:45:23.523 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.593+0000 7f58827fc700 1 -- 172.21.15.37:0/1499253831 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5874011b20 con 0x7f5884108a00 2024-06-10T00:45:23.523 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.594+0000 7f5889755700 1 -- 172.21.15.37:0/1499253831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5884108a00 msgr2=0x7f588410adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:23.523 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.594+0000 7f5889755700 1 --2- 172.21.15.37:0/1499253831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5884108a00 0x7f588410adf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f58740098a0 tx=0x7f5874009bb0 comp rx=0 tx=0).stop 2024-06-10T00:45:23.523 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.594+0000 7f5889755700 1 -- 172.21.15.37:0/1499253831 shutdown_connections 2024-06-10T00:45:23.523 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.594+0000 7f5889755700 1 --2- 172.21.15.37:0/1499253831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5884108a00 0x7f588410adf0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:23.523 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.594+0000 7f5889755700 1 -- 172.21.15.37:0/1499253831 >> 172.21.15.37:0/1499253831 conn(0x7f5884102450 msgr2=0x7f5884104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:23.523 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.594+0000 7f5889755700 1 -- 172.21.15.37:0/1499253831 shutdown_connections 2024-06-10T00:45:23.524 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.594+0000 7f5889755700 1 -- 172.21.15.37:0/1499253831 wait complete. 2024-06-10T00:45:23.524 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.595+0000 7f5889755700 1 Processor -- start 2024-06-10T00:45:23.524 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.595+0000 7f5889755700 1 -- start start 2024-06-10T00:45:23.524 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.595+0000 7f5889755700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5884108a00 0x7f588419be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:23.524 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.595+0000 7f5889755700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f588410c3a0 con 0x7f5884108a00 2024-06-10T00:45:23.524 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.595+0000 7f5882ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5884108a00 0x7f588419be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:23.524 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.595+0000 7f5882ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5884108a00 0x7f588419be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53974/0 (socket says 172.21.15.37:53974) 2024-06-10T00:45:23.524 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.595+0000 7f5882ffd700 1 -- 172.21.15.37:0/3106159842 learned_addr learned my addr 172.21.15.37:0/3106159842 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:23.524 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.595+0000 7f5882ffd700 1 -- 172.21.15.37:0/3106159842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588419c3b0 con 0x7f5884108a00 2024-06-10T00:45:23.524 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.596+0000 7f5882ffd700 1 --2- 172.21.15.37:0/3106159842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5884108a00 0x7f588419be70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f58740117e0 tx=0x7f5874011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:23.524 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.596+0000 7f5880ff9700 1 -- 172.21.15.37:0/3106159842 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5874003bf0 con 0x7f5884108a00 2024-06-10T00:45:23.524 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.596+0000 7f5880ff9700 1 -- 172.21.15.37:0/3106159842 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f587401a910 con 0x7f5884108a00 2024-06-10T00:45:23.524 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.596+0000 7f5889755700 1 -- 172.21.15.37:0/3106159842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f588419c640 con 0x7f5884108a00 2024-06-10T00:45:23.524 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.596+0000 7f5880ff9700 1 -- 172.21.15.37:0/3106159842 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5874024970 con 0x7f5884108a00 2024-06-10T00:45:23.525 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.596+0000 7f5889755700 1 -- 172.21.15.37:0/3106159842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f588419caa0 con 0x7f5884108a00 2024-06-10T00:45:23.525 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.597+0000 7f5889755700 1 -- 172.21.15.37:0/3106159842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5884063af0 con 0x7f5884108a00 2024-06-10T00:45:23.525 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.597+0000 7f5880ff9700 1 -- 172.21.15.37:0/3106159842 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f5874022020 con 0x7f5884108a00 2024-06-10T00:45:23.525 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.597+0000 7f5880ff9700 1 --2- 172.21.15.37:0/3106159842 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5870038f40 0x7f587003b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:23.525 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.597+0000 7f5880ff9700 1 -- 172.21.15.37:0/3106159842 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f587404d300 con 0x7f5884108a00 2024-06-10T00:45:23.525 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.598+0000 7f587bfff700 1 --2- 172.21.15.37:0/3106159842 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5870038f40 0x7f587003b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:23.525 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.598+0000 7f587bfff700 1 --2- 172.21.15.37:0/3106159842 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5870038f40 0x7f587003b400 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f586c006fd0 tx=0x7f586c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:23.525 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.602+0000 7f5880ff9700 1 -- 172.21.15.37:0/3106159842 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5874011e20 con 0x7f5884108a00 2024-06-10T00:45:23.525 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.815+0000 7f5889755700 1 -- 172.21.15.37:0/3106159842 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f5884103600 con 0x7f5870038f40 2024-06-10T00:45:23.525 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.863+0000 7f5880ff9700 1 -- 172.21.15.37:0/3106159842 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f587401a430 con 0x7f5884108a00 2024-06-10T00:45:23.525 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.978+0000 7f5880ff9700 1 -- 172.21.15.37:0/3106159842 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f5884103600 con 0x7f5870038f40 2024-06-10T00:45:23.525 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.980+0000 7f5889755700 1 -- 172.21.15.37:0/3106159842 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5870038f40 msgr2=0x7f587003b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:23.525 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.980+0000 7f5889755700 1 --2- 172.21.15.37:0/3106159842 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5870038f40 0x7f587003b400 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f586c006fd0 tx=0x7f586c006e40 comp rx=0 tx=0).stop 2024-06-10T00:45:23.526 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.980+0000 7f5889755700 1 -- 172.21.15.37:0/3106159842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5884108a00 msgr2=0x7f588419be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:23.526 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.980+0000 7f5889755700 1 --2- 172.21.15.37:0/3106159842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5884108a00 0x7f588419be70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f58740117e0 tx=0x7f5874011c10 comp rx=0 tx=0).stop 2024-06-10T00:45:23.526 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.981+0000 7f5889755700 1 -- 172.21.15.37:0/3106159842 shutdown_connections 2024-06-10T00:45:23.526 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.981+0000 7f5889755700 1 --2- 172.21.15.37:0/3106159842 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5870038f40 0x7f587003b400 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:23.526 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.981+0000 7f5889755700 1 --2- 172.21.15.37:0/3106159842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5884108a00 0x7f588419be70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:23.526 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.981+0000 7f5889755700 1 -- 172.21.15.37:0/3106159842 >> 172.21.15.37:0/3106159842 conn(0x7f5884102450 msgr2=0x7f5884102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:23.526 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.981+0000 7f5889755700 1 -- 172.21.15.37:0/3106159842 shutdown_connections 2024-06-10T00:45:23.526 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:22.981+0000 7f5889755700 1 -- 172.21.15.37:0/3106159842 wait complete. 2024-06-10T00:45:23.526 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial admin user... 2024-06-10T00:45:23.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:23 smithi037 ceph-mon[32192]: from='client.14160 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:45:23.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:23 smithi037 ceph-mon[32192]: mgrmap e12: x(active, since 2s) 2024-06-10T00:45:23.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:23 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:23.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:23 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:23.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:23 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:24.955 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$gFVMua7KR.cX9aXUoyH90eUywCMtj3x1OoM7oNCiOmiUYBqnbqSxC", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1717980324, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-06-10T00:45:24.956 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.944+0000 7f7415d0d700 1 Processor -- start 2024-06-10T00:45:24.956 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.944+0000 7f7415d0d700 1 -- start start 2024-06-10T00:45:24.956 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.944+0000 7f7415d0d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7410108a30 0x7f741010ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:24.956 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.944+0000 7f7415d0d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f741007c6c0 con 0x7f7410108a30 2024-06-10T00:45:24.956 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.944+0000 7f740f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7410108a30 0x7f741010ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:24.956 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.944+0000 7f740f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7410108a30 0x7f741010ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53988/0 (socket says 172.21.15.37:53988) 2024-06-10T00:45:24.956 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.944+0000 7f740f7fe700 1 -- 172.21.15.37:0/1031136806 learned_addr learned my addr 172.21.15.37:0/1031136806 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:24.956 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.945+0000 7f740f7fe700 1 -- 172.21.15.37:0/1031136806 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f741010b360 con 0x7f7410108a30 2024-06-10T00:45:24.956 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.945+0000 7f740f7fe700 1 --2- 172.21.15.37:0/1031136806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7410108a30 0x7f741010ae20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f74000098a0 tx=0x7f7400009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8127f380c9f37 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:24.956 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.945+0000 7f740e7fc700 1 -- 172.21.15.37:0/1031136806 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7400004160 con 0x7f7410108a30 2024-06-10T00:45:24.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.945+0000 7f740e7fc700 1 -- 172.21.15.37:0/1031136806 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7400004300 con 0x7f7410108a30 2024-06-10T00:45:24.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.945+0000 7f740e7fc700 1 -- 172.21.15.37:0/1031136806 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7400011b20 con 0x7f7410108a30 2024-06-10T00:45:24.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.946+0000 7f7415d0d700 1 -- 172.21.15.37:0/1031136806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7410108a30 msgr2=0x7f741010ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:24.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.946+0000 7f7415d0d700 1 --2- 172.21.15.37:0/1031136806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7410108a30 0x7f741010ae20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f74000098a0 tx=0x7f7400009bb0 comp rx=0 tx=0).stop 2024-06-10T00:45:24.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.946+0000 7f7415d0d700 1 -- 172.21.15.37:0/1031136806 shutdown_connections 2024-06-10T00:45:24.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.946+0000 7f7415d0d700 1 --2- 172.21.15.37:0/1031136806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7410108a30 0x7f741010ae20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:24.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.946+0000 7f7415d0d700 1 -- 172.21.15.37:0/1031136806 >> 172.21.15.37:0/1031136806 conn(0x7f7410102420 msgr2=0x7f7410104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:24.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.946+0000 7f7415d0d700 1 -- 172.21.15.37:0/1031136806 shutdown_connections 2024-06-10T00:45:24.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.946+0000 7f7415d0d700 1 -- 172.21.15.37:0/1031136806 wait complete. 2024-06-10T00:45:24.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.946+0000 7f7415d0d700 1 Processor -- start 2024-06-10T00:45:24.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.947+0000 7f7415d0d700 1 -- start start 2024-06-10T00:45:24.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.947+0000 7f7415d0d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7410108a30 0x7f741019be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:24.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.947+0000 7f7415d0d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f741010c360 con 0x7f7410108a30 2024-06-10T00:45:24.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.947+0000 7f740f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7410108a30 0x7f741019be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:24.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.947+0000 7f740f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7410108a30 0x7f741019be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53998/0 (socket says 172.21.15.37:53998) 2024-06-10T00:45:24.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.947+0000 7f740f7fe700 1 -- 172.21.15.37:0/4227824621 learned_addr learned my addr 172.21.15.37:0/4227824621 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:24.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.947+0000 7f740f7fe700 1 -- 172.21.15.37:0/4227824621 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f741019c3c0 con 0x7f7410108a30 2024-06-10T00:45:24.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.947+0000 7f740f7fe700 1 --2- 172.21.15.37:0/4227824621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7410108a30 0x7f741019be80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f74000117e0 tx=0x7f7400011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:24.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.948+0000 7f740cff9700 1 -- 172.21.15.37:0/4227824621 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7400003bf0 con 0x7f7410108a30 2024-06-10T00:45:24.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.948+0000 7f740cff9700 1 -- 172.21.15.37:0/4227824621 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f740001a910 con 0x7f7410108a30 2024-06-10T00:45:24.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.948+0000 7f7415d0d700 1 -- 172.21.15.37:0/4227824621 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f741019c650 con 0x7f7410108a30 2024-06-10T00:45:24.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.948+0000 7f740cff9700 1 -- 172.21.15.37:0/4227824621 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7400024970 con 0x7f7410108a30 2024-06-10T00:45:24.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.948+0000 7f7415d0d700 1 -- 172.21.15.37:0/4227824621 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f741019cab0 con 0x7f7410108a30 2024-06-10T00:45:24.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.949+0000 7f740cff9700 1 -- 172.21.15.37:0/4227824621 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f7400022020 con 0x7f7410108a30 2024-06-10T00:45:24.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.949+0000 7f7415d0d700 1 -- 172.21.15.37:0/4227824621 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7410063af0 con 0x7f7410108a30 2024-06-10T00:45:24.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.949+0000 7f740cff9700 1 --2- 172.21.15.37:0/4227824621 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f73f80390b0 0x7f73f803b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:24.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.949+0000 7f740cff9700 1 -- 172.21.15.37:0/4227824621 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f740001a430 con 0x7f7410108a30 2024-06-10T00:45:24.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.949+0000 7f740effd700 1 --2- 172.21.15.37:0/4227824621 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f73f80390b0 0x7f73f803b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:24.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.950+0000 7f740effd700 1 --2- 172.21.15.37:0/4227824621 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f73f80390b0 0x7f73f803b570 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7404006fd0 tx=0x7f7404006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:24.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:23.954+0000 7f740cff9700 1 -- 172.21.15.37:0/4227824621 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7400011e20 con 0x7f7410108a30 2024-06-10T00:45:24.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:24.181+0000 7f7415d0d700 1 -- 172.21.15.37:0/4227824621 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f741019f4c0 con 0x7f73f80390b0 2024-06-10T00:45:24.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:24.428+0000 7f740cff9700 1 -- 172.21.15.37:0/4227824621 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f741019f4c0 con 0x7f73f80390b0 2024-06-10T00:45:24.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:24.430+0000 7f7415d0d700 1 -- 172.21.15.37:0/4227824621 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f73f80390b0 msgr2=0x7f73f803b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:24.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:24.430+0000 7f7415d0d700 1 --2- 172.21.15.37:0/4227824621 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f73f80390b0 0x7f73f803b570 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7404006fd0 tx=0x7f7404006e40 comp rx=0 tx=0).stop 2024-06-10T00:45:24.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:24.430+0000 7f7415d0d700 1 -- 172.21.15.37:0/4227824621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7410108a30 msgr2=0x7f741019be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:24.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:24.430+0000 7f7415d0d700 1 --2- 172.21.15.37:0/4227824621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7410108a30 0x7f741019be80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f74000117e0 tx=0x7f7400011c10 comp rx=0 tx=0).stop 2024-06-10T00:45:24.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:24.430+0000 7f7415d0d700 1 -- 172.21.15.37:0/4227824621 shutdown_connections 2024-06-10T00:45:24.960 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:24.430+0000 7f7415d0d700 1 --2- 172.21.15.37:0/4227824621 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f73f80390b0 0x7f73f803b570 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:24.960 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:24.430+0000 7f7415d0d700 1 --2- 172.21.15.37:0/4227824621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7410108a30 0x7f741019be80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:24.960 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:24.430+0000 7f7415d0d700 1 -- 172.21.15.37:0/4227824621 >> 172.21.15.37:0/4227824621 conn(0x7f7410102420 msgr2=0x7f7410102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:24.960 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:24.431+0000 7f7415d0d700 1 -- 172.21.15.37:0/4227824621 shutdown_connections 2024-06-10T00:45:24.960 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:24.431+0000 7f7415d0d700 1 -- 172.21.15.37:0/4227824621 wait complete. 2024-06-10T00:45:24.960 INFO:teuthology.orchestra.run.smithi037.stdout:Fetching dashboard port number... 2024-06-10T00:45:25.605 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:25 smithi037 ceph-mon[32192]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:45:25.606 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:25 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:26.150 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 8443 2024-06-10T00:45:26.150 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.376+0000 7fcc5a7da700 1 Processor -- start 2024-06-10T00:45:26.150 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.376+0000 7fcc5a7da700 1 -- start start 2024-06-10T00:45:26.150 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.376+0000 7fcc5a7da700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc54107f10 0x7fcc54108330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:26.150 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.376+0000 7fcc5a7da700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcc54108870 con 0x7fcc54107f10 2024-06-10T00:45:26.150 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.377+0000 7fcc53fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc54107f10 0x7fcc54108330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:26.150 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.377+0000 7fcc53fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc54107f10 0x7fcc54108330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54014/0 (socket says 172.21.15.37:54014) 2024-06-10T00:45:26.150 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.377+0000 7fcc53fff700 1 -- 172.21.15.37:0/2861436076 learned_addr learned my addr 172.21.15.37:0/2861436076 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:26.150 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.377+0000 7fcc53fff700 1 -- 172.21.15.37:0/2861436076 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc54109070 con 0x7fcc54107f10 2024-06-10T00:45:26.150 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.377+0000 7fcc53fff700 1 --2- 172.21.15.37:0/2861436076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc54107f10 0x7fcc54108330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcc440098a0 tx=0x7fcc44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d856329ca88c4e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:26.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.377+0000 7fcc52ffd700 1 -- 172.21.15.37:0/2861436076 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc44004160 con 0x7fcc54107f10 2024-06-10T00:45:26.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.377+0000 7fcc52ffd700 1 -- 172.21.15.37:0/2861436076 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc44004300 con 0x7fcc54107f10 2024-06-10T00:45:26.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.377+0000 7fcc52ffd700 1 -- 172.21.15.37:0/2861436076 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc44011b20 con 0x7fcc54107f10 2024-06-10T00:45:26.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.378+0000 7fcc5a7da700 1 -- 172.21.15.37:0/2861436076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc54107f10 msgr2=0x7fcc54108330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:26.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.378+0000 7fcc5a7da700 1 --2- 172.21.15.37:0/2861436076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc54107f10 0x7fcc54108330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcc440098a0 tx=0x7fcc44009bb0 comp rx=0 tx=0).stop 2024-06-10T00:45:26.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.378+0000 7fcc5a7da700 1 -- 172.21.15.37:0/2861436076 shutdown_connections 2024-06-10T00:45:26.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.378+0000 7fcc5a7da700 1 --2- 172.21.15.37:0/2861436076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc54107f10 0x7fcc54108330 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:26.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.378+0000 7fcc5a7da700 1 -- 172.21.15.37:0/2861436076 >> 172.21.15.37:0/2861436076 conn(0x7fcc54103530 msgr2=0x7fcc54105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:26.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.378+0000 7fcc5a7da700 1 -- 172.21.15.37:0/2861436076 shutdown_connections 2024-06-10T00:45:26.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.378+0000 7fcc5a7da700 1 -- 172.21.15.37:0/2861436076 wait complete. 2024-06-10T00:45:26.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.379+0000 7fcc5a7da700 1 Processor -- start 2024-06-10T00:45:26.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.379+0000 7fcc5a7da700 1 -- start start 2024-06-10T00:45:26.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.379+0000 7fcc5a7da700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc54107f10 0x7fcc5407e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:26.151 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.379+0000 7fcc5a7da700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5410c4b0 con 0x7fcc54107f10 2024-06-10T00:45:26.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.379+0000 7fcc53fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc54107f10 0x7fcc5407e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:26.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.379+0000 7fcc53fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc54107f10 0x7fcc5407e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54026/0 (socket says 172.21.15.37:54026) 2024-06-10T00:45:26.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.379+0000 7fcc53fff700 1 -- 172.21.15.37:0/1840896718 learned_addr learned my addr 172.21.15.37:0/1840896718 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:26.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.380+0000 7fcc53fff700 1 -- 172.21.15.37:0/1840896718 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5407e940 con 0x7fcc54107f10 2024-06-10T00:45:26.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.380+0000 7fcc53fff700 1 --2- 172.21.15.37:0/1840896718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc54107f10 0x7fcc5407e400 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcc440117e0 tx=0x7fcc44011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:26.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.380+0000 7fcc517fa700 1 -- 172.21.15.37:0/1840896718 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc44003bf0 con 0x7fcc54107f10 2024-06-10T00:45:26.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.380+0000 7fcc517fa700 1 -- 172.21.15.37:0/1840896718 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc4401a910 con 0x7fcc54107f10 2024-06-10T00:45:26.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.380+0000 7fcc5a7da700 1 -- 172.21.15.37:0/1840896718 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc5407cb30 con 0x7fcc54107f10 2024-06-10T00:45:26.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.380+0000 7fcc517fa700 1 -- 172.21.15.37:0/1840896718 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc44024970 con 0x7fcc54107f10 2024-06-10T00:45:26.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.380+0000 7fcc5a7da700 1 -- 172.21.15.37:0/1840896718 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc5407cff0 con 0x7fcc54107f10 2024-06-10T00:45:26.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.381+0000 7fcc517fa700 1 -- 172.21.15.37:0/1840896718 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fcc44022020 con 0x7fcc54107f10 2024-06-10T00:45:26.152 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.381+0000 7fcc517fa700 1 --2- 172.21.15.37:0/1840896718 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fcc40039050 0x7fcc4003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:26.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.381+0000 7fcc517fa700 1 -- 172.21.15.37:0/1840896718 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcc4404d6a0 con 0x7fcc54107f10 2024-06-10T00:45:26.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.381+0000 7fcc537fe700 1 --2- 172.21.15.37:0/1840896718 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fcc40039050 0x7fcc4003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:26.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.381+0000 7fcc5a7da700 1 -- 172.21.15.37:0/1840896718 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc54063af0 con 0x7fcc54107f10 2024-06-10T00:45:26.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.382+0000 7fcc537fe700 1 --2- 172.21.15.37:0/1840896718 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fcc40039050 0x7fcc4003b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c006fd0 tx=0x7fcc3c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:26.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.387+0000 7fcc517fa700 1 -- 172.21.15.37:0/1840896718 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcc44011e20 con 0x7fcc54107f10 2024-06-10T00:45:26.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.605+0000 7fcc5a7da700 1 -- 172.21.15.37:0/1840896718 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fcc5407d5e0 con 0x7fcc54107f10 2024-06-10T00:45:26.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.606+0000 7fcc517fa700 1 -- 172.21.15.37:0/1840896718 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fcc44019c70 con 0x7fcc54107f10 2024-06-10T00:45:26.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.608+0000 7fcc5a7da700 1 -- 172.21.15.37:0/1840896718 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fcc40039050 msgr2=0x7fcc4003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:26.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.608+0000 7fcc5a7da700 1 --2- 172.21.15.37:0/1840896718 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fcc40039050 0x7fcc4003b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c006fd0 tx=0x7fcc3c006e40 comp rx=0 tx=0).stop 2024-06-10T00:45:26.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.608+0000 7fcc5a7da700 1 -- 172.21.15.37:0/1840896718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc54107f10 msgr2=0x7fcc5407e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:26.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.608+0000 7fcc5a7da700 1 --2- 172.21.15.37:0/1840896718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc54107f10 0x7fcc5407e400 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcc440117e0 tx=0x7fcc44011c10 comp rx=0 tx=0).stop 2024-06-10T00:45:26.153 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.608+0000 7fcc5a7da700 1 -- 172.21.15.37:0/1840896718 shutdown_connections 2024-06-10T00:45:26.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.608+0000 7fcc5a7da700 1 --2- 172.21.15.37:0/1840896718 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fcc40039050 0x7fcc4003b510 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:26.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.608+0000 7fcc5a7da700 1 --2- 172.21.15.37:0/1840896718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc54107f10 0x7fcc5407e400 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:26.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.608+0000 7fcc5a7da700 1 -- 172.21.15.37:0/1840896718 >> 172.21.15.37:0/1840896718 conn(0x7fcc54103530 msgr2=0x7fcc54103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:26.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.608+0000 7fcc5a7da700 1 -- 172.21.15.37:0/1840896718 shutdown_connections 2024-06-10T00:45:26.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:25.608+0000 7fcc5a7da700 1 -- 172.21.15.37:0/1840896718 wait complete. 2024-06-10T00:45:26.158 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-10T00:45:26.158 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-06-10T00:45:26.166 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-10T00:45:26.166 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-06-10T00:45:26.166 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-06-10T00:45:26.166 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-06-10T00:45:26.167 INFO:teuthology.orchestra.run.smithi037.stdout:Ceph Dashboard is now available at: 2024-06-10T00:45:26.168 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:45:26.168 INFO:teuthology.orchestra.run.smithi037.stdout: URL: https://smithi037.front.sepia.ceph.com:8443/ 2024-06-10T00:45:26.168 INFO:teuthology.orchestra.run.smithi037.stdout: User: admin 2024-06-10T00:45:26.168 INFO:teuthology.orchestra.run.smithi037.stdout: Password: jfwqg8817t 2024-06-10T00:45:26.168 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:45:26.168 INFO:teuthology.orchestra.run.smithi037.stdout:Saving cluster configuration to /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/config directory 2024-06-10T00:45:26.169 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling autotune for osd_memory_target 2024-06-10T00:45:26.561 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:26 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1840896718' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-06-10T00:45:27.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.602+0000 7fed2aaec700 1 Processor -- start 2024-06-10T00:45:27.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.602+0000 7fed2aaec700 1 -- start start 2024-06-10T00:45:27.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.602+0000 7fed2aaec700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed241089f0 0x7fed2410ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:27.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.602+0000 7fed2aaec700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fed2407c700 con 0x7fed241089f0 2024-06-10T00:45:27.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.603+0000 7fed28888700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed241089f0 0x7fed2410ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:27.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.603+0000 7fed28888700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed241089f0 0x7fed2410ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54034/0 (socket says 172.21.15.37:54034) 2024-06-10T00:45:27.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.603+0000 7fed28888700 1 -- 172.21.15.37:0/4176189766 learned_addr learned my addr 172.21.15.37:0/4176189766 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:27.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.603+0000 7fed28888700 1 -- 172.21.15.37:0/4176189766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2410b320 con 0x7fed241089f0 2024-06-10T00:45:27.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.603+0000 7fed28888700 1 --2- 172.21.15.37:0/4176189766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed241089f0 0x7fed2410ade0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fed14009e80 tx=0x7fed1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4edaed64d2ec239 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:27.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.603+0000 7fed237fe700 1 -- 172.21.15.37:0/4176189766 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed14017070 con 0x7fed241089f0 2024-06-10T00:45:27.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.603+0000 7fed237fe700 1 -- 172.21.15.37:0/4176189766 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed1400b8c0 con 0x7fed241089f0 2024-06-10T00:45:27.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.603+0000 7fed237fe700 1 -- 172.21.15.37:0/4176189766 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed140045c0 con 0x7fed241089f0 2024-06-10T00:45:27.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.604+0000 7fed2aaec700 1 -- 172.21.15.37:0/4176189766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed241089f0 msgr2=0x7fed2410ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:27.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.604+0000 7fed2aaec700 1 --2- 172.21.15.37:0/4176189766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed241089f0 0x7fed2410ade0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fed14009e80 tx=0x7fed1400b330 comp rx=0 tx=0).stop 2024-06-10T00:45:27.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.604+0000 7fed2aaec700 1 -- 172.21.15.37:0/4176189766 shutdown_connections 2024-06-10T00:45:27.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.604+0000 7fed2aaec700 1 --2- 172.21.15.37:0/4176189766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed241089f0 0x7fed2410ade0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:27.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.604+0000 7fed2aaec700 1 -- 172.21.15.37:0/4176189766 >> 172.21.15.37:0/4176189766 conn(0x7fed24102420 msgr2=0x7fed24104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:27.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.604+0000 7fed2aaec700 1 -- 172.21.15.37:0/4176189766 shutdown_connections 2024-06-10T00:45:27.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.604+0000 7fed2aaec700 1 -- 172.21.15.37:0/4176189766 wait complete. 2024-06-10T00:45:27.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.605+0000 7fed2aaec700 1 Processor -- start 2024-06-10T00:45:27.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.605+0000 7fed2aaec700 1 -- start start 2024-06-10T00:45:27.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.605+0000 7fed2aaec700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed241089f0 0x7fed2419be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:27.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.605+0000 7fed2aaec700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fed2410c380 con 0x7fed241089f0 2024-06-10T00:45:27.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.605+0000 7fed28888700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed241089f0 0x7fed2419be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:27.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.605+0000 7fed28888700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed241089f0 0x7fed2419be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54038/0 (socket says 172.21.15.37:54038) 2024-06-10T00:45:27.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.605+0000 7fed28888700 1 -- 172.21.15.37:0/900256189 learned_addr learned my addr 172.21.15.37:0/900256189 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:27.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.605+0000 7fed28888700 1 -- 172.21.15.37:0/900256189 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2419c3c0 con 0x7fed241089f0 2024-06-10T00:45:27.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.605+0000 7fed28888700 1 --2- 172.21.15.37:0/900256189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed241089f0 0x7fed2419be80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fed1400b7b0 tx=0x7fed1400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:27.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.605+0000 7fed21ffb700 1 -- 172.21.15.37:0/900256189 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed14020020 con 0x7fed241089f0 2024-06-10T00:45:27.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.606+0000 7fed21ffb700 1 -- 172.21.15.37:0/900256189 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed14003d90 con 0x7fed241089f0 2024-06-10T00:45:27.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.606+0000 7fed2aaec700 1 -- 172.21.15.37:0/900256189 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2419c650 con 0x7fed241089f0 2024-06-10T00:45:27.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.606+0000 7fed21ffb700 1 -- 172.21.15.37:0/900256189 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed14017040 con 0x7fed241089f0 2024-06-10T00:45:27.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.606+0000 7fed2aaec700 1 -- 172.21.15.37:0/900256189 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed2419f340 con 0x7fed241089f0 2024-06-10T00:45:27.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.607+0000 7fed21ffb700 1 -- 172.21.15.37:0/900256189 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fed14023020 con 0x7fed241089f0 2024-06-10T00:45:27.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.607+0000 7fed2aaec700 1 -- 172.21.15.37:0/900256189 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed24063af0 con 0x7fed241089f0 2024-06-10T00:45:27.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.607+0000 7fed21ffb700 1 --2- 172.21.15.37:0/900256189 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fed100390f0 0x7fed1003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:27.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.607+0000 7fed21ffb700 1 -- 172.21.15.37:0/900256189 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fed1400f050 con 0x7fed241089f0 2024-06-10T00:45:27.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.607+0000 7fed23fff700 1 --2- 172.21.15.37:0/900256189 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fed100390f0 0x7fed1003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:27.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.607+0000 7fed23fff700 1 --2- 172.21.15.37:0/900256189 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fed100390f0 0x7fed1003b5b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fed0c006fd0 tx=0x7fed0c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:27.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.612+0000 7fed21ffb700 1 -- 172.21.15.37:0/900256189 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fed1402a020 con 0x7fed241089f0 2024-06-10T00:45:27.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.822+0000 7fed2aaec700 1 -- 172.21.15.37:0/900256189 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fed2419f510 con 0x7fed241089f0 2024-06-10T00:45:27.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.822+0000 7fed21ffb700 1 -- 172.21.15.37:0/900256189 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fed14012a50 con 0x7fed241089f0 2024-06-10T00:45:27.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.825+0000 7fed2aaec700 1 -- 172.21.15.37:0/900256189 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fed100390f0 msgr2=0x7fed1003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:27.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.825+0000 7fed2aaec700 1 --2- 172.21.15.37:0/900256189 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fed100390f0 0x7fed1003b5b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fed0c006fd0 tx=0x7fed0c006e40 comp rx=0 tx=0).stop 2024-06-10T00:45:27.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.825+0000 7fed2aaec700 1 -- 172.21.15.37:0/900256189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed241089f0 msgr2=0x7fed2419be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:27.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.825+0000 7fed2aaec700 1 --2- 172.21.15.37:0/900256189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed241089f0 0x7fed2419be80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fed1400b7b0 tx=0x7fed1400bd50 comp rx=0 tx=0).stop 2024-06-10T00:45:27.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.825+0000 7fed2aaec700 1 -- 172.21.15.37:0/900256189 shutdown_connections 2024-06-10T00:45:27.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.825+0000 7fed2aaec700 1 --2- 172.21.15.37:0/900256189 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fed100390f0 0x7fed1003b5b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:27.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.825+0000 7fed2aaec700 1 --2- 172.21.15.37:0/900256189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed241089f0 0x7fed2419be80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:27.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.825+0000 7fed2aaec700 1 -- 172.21.15.37:0/900256189 >> 172.21.15.37:0/900256189 conn(0x7fed24102420 msgr2=0x7fed24102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:27.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.825+0000 7fed2aaec700 1 -- 172.21.15.37:0/900256189 shutdown_connections 2024-06-10T00:45:27.412 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:26.825+0000 7fed2aaec700 1 -- 172.21.15.37:0/900256189 wait complete. 2024-06-10T00:45:27.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:27 smithi037 ceph-mon[32192]: mgrmap e13: x(active, since 6s) 2024-06-10T00:45:28.704 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.823+0000 7ff413cf0700 1 Processor -- start 2024-06-10T00:45:28.704 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.823+0000 7ff413cf0700 1 -- start start 2024-06-10T00:45:28.704 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.823+0000 7ff413cf0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff40c107fc0 0x7ff40c1083e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:28.704 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.823+0000 7ff413cf0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff40c1089b0 con 0x7ff40c107fc0 2024-06-10T00:45:28.704 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.823+0000 7ff411a8c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff40c107fc0 0x7ff40c1083e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:28.704 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.823+0000 7ff411a8c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff40c107fc0 0x7ff40c1083e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54054/0 (socket says 172.21.15.37:54054) 2024-06-10T00:45:28.704 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.823+0000 7ff411a8c700 1 -- 172.21.15.37:0/2208595269 learned_addr learned my addr 172.21.15.37:0/2208595269 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:28.705 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.824+0000 7ff411a8c700 1 -- 172.21.15.37:0/2208595269 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40c109170 con 0x7ff40c107fc0 2024-06-10T00:45:28.705 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.824+0000 7ff411a8c700 1 --2- 172.21.15.37:0/2208595269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff40c107fc0 0x7ff40c1083e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff4080098a0 tx=0x7ff408009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3e11d78b13749c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:28.705 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.824+0000 7ff410a8a700 1 -- 172.21.15.37:0/2208595269 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff408004160 con 0x7ff40c107fc0 2024-06-10T00:45:28.705 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.824+0000 7ff410a8a700 1 -- 172.21.15.37:0/2208595269 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff408004300 con 0x7ff40c107fc0 2024-06-10T00:45:28.705 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.824+0000 7ff410a8a700 1 -- 172.21.15.37:0/2208595269 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff408011b20 con 0x7ff40c107fc0 2024-06-10T00:45:28.705 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.825+0000 7ff413cf0700 1 -- 172.21.15.37:0/2208595269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff40c107fc0 msgr2=0x7ff40c1083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:28.705 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.825+0000 7ff413cf0700 1 --2- 172.21.15.37:0/2208595269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff40c107fc0 0x7ff40c1083e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff4080098a0 tx=0x7ff408009bb0 comp rx=0 tx=0).stop 2024-06-10T00:45:28.705 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.825+0000 7ff413cf0700 1 -- 172.21.15.37:0/2208595269 shutdown_connections 2024-06-10T00:45:28.705 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.825+0000 7ff413cf0700 1 --2- 172.21.15.37:0/2208595269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff40c107fc0 0x7ff40c1083e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:28.705 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.825+0000 7ff413cf0700 1 -- 172.21.15.37:0/2208595269 >> 172.21.15.37:0/2208595269 conn(0x7ff40c1035a0 msgr2=0x7ff40c1059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:28.705 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.825+0000 7ff413cf0700 1 -- 172.21.15.37:0/2208595269 shutdown_connections 2024-06-10T00:45:28.705 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.825+0000 7ff413cf0700 1 -- 172.21.15.37:0/2208595269 wait complete. 2024-06-10T00:45:28.705 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.825+0000 7ff413cf0700 1 Processor -- start 2024-06-10T00:45:28.705 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.825+0000 7ff413cf0700 1 -- start start 2024-06-10T00:45:28.706 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.826+0000 7ff413cf0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff40c107fc0 0x7ff40c07e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:28.706 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.826+0000 7ff413cf0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff40c10c520 con 0x7ff40c107fc0 2024-06-10T00:45:28.706 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.826+0000 7ff411a8c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff40c107fc0 0x7ff40c07e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:28.706 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.826+0000 7ff411a8c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff40c107fc0 0x7ff40c07e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54070/0 (socket says 172.21.15.37:54070) 2024-06-10T00:45:28.706 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.826+0000 7ff411a8c700 1 -- 172.21.15.37:0/3263128327 learned_addr learned my addr 172.21.15.37:0/3263128327 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:28.706 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.826+0000 7ff411a8c700 1 -- 172.21.15.37:0/3263128327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40c07e940 con 0x7ff40c107fc0 2024-06-10T00:45:28.706 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.826+0000 7ff411a8c700 1 --2- 172.21.15.37:0/3263128327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff40c107fc0 0x7ff40c07e400 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff4080117e0 tx=0x7ff408011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:28.706 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.826+0000 7ff402ffd700 1 -- 172.21.15.37:0/3263128327 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff408003bf0 con 0x7ff40c107fc0 2024-06-10T00:45:28.706 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.826+0000 7ff402ffd700 1 -- 172.21.15.37:0/3263128327 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff40801a910 con 0x7ff40c107fc0 2024-06-10T00:45:28.706 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.826+0000 7ff413cf0700 1 -- 172.21.15.37:0/3263128327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff40c07cb30 con 0x7ff40c107fc0 2024-06-10T00:45:28.706 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.826+0000 7ff402ffd700 1 -- 172.21.15.37:0/3263128327 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4080249a0 con 0x7ff40c107fc0 2024-06-10T00:45:28.706 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.826+0000 7ff413cf0700 1 -- 172.21.15.37:0/3263128327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff40c07cff0 con 0x7ff40c107fc0 2024-06-10T00:45:28.707 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.827+0000 7ff402ffd700 1 -- 172.21.15.37:0/3263128327 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff408022020 con 0x7ff40c107fc0 2024-06-10T00:45:28.707 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.828+0000 7ff413cf0700 1 -- 172.21.15.37:0/3263128327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff40c063af0 con 0x7ff40c107fc0 2024-06-10T00:45:28.707 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.828+0000 7ff402ffd700 1 --2- 172.21.15.37:0/3263128327 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff3f80390f0 0x7ff3f803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:28.707 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.828+0000 7ff402ffd700 1 -- 172.21.15.37:0/3263128327 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff40804d5b0 con 0x7ff40c107fc0 2024-06-10T00:45:28.707 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.828+0000 7ff41128b700 1 --2- 172.21.15.37:0/3263128327 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff3f80390f0 0x7ff3f803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:28.707 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.828+0000 7ff41128b700 1 --2- 172.21.15.37:0/3263128327 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff3f80390f0 0x7ff3f803b5b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc006fd0 tx=0x7ff3fc006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:28.707 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:27.833+0000 7ff402ffd700 1 -- 172.21.15.37:0/3263128327 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff408011e20 con 0x7ff40c107fc0 2024-06-10T00:45:28.707 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:28.153+0000 7ff413cf0700 1 -- 172.21.15.37:0/3263128327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7ff40c07d270 con 0x7ff40c107fc0 2024-06-10T00:45:28.707 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:28.155+0000 7ff402ffd700 1 -- 172.21.15.37:0/3263128327 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7ff40800b3e0 con 0x7ff40c107fc0 2024-06-10T00:45:28.707 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:28.158+0000 7ff413cf0700 1 -- 172.21.15.37:0/3263128327 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff3f80390f0 msgr2=0x7ff3f803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:28.707 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:28.158+0000 7ff413cf0700 1 --2- 172.21.15.37:0/3263128327 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff3f80390f0 0x7ff3f803b5b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc006fd0 tx=0x7ff3fc006e40 comp rx=0 tx=0).stop 2024-06-10T00:45:28.707 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:28.158+0000 7ff413cf0700 1 -- 172.21.15.37:0/3263128327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff40c107fc0 msgr2=0x7ff40c07e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:28.707 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:28.158+0000 7ff413cf0700 1 --2- 172.21.15.37:0/3263128327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff40c107fc0 0x7ff40c07e400 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff4080117e0 tx=0x7ff408011c10 comp rx=0 tx=0).stop 2024-06-10T00:45:28.708 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:28.158+0000 7ff413cf0700 1 -- 172.21.15.37:0/3263128327 shutdown_connections 2024-06-10T00:45:28.708 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:28.158+0000 7ff413cf0700 1 --2- 172.21.15.37:0/3263128327 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff3f80390f0 0x7ff3f803b5b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:28.708 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:28.158+0000 7ff413cf0700 1 --2- 172.21.15.37:0/3263128327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff40c107fc0 0x7ff40c07e400 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:28.708 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:28.158+0000 7ff413cf0700 1 -- 172.21.15.37:0/3263128327 >> 172.21.15.37:0/3263128327 conn(0x7ff40c1035a0 msgr2=0x7ff40c103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:28.708 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:28.158+0000 7ff413cf0700 1 -- 172.21.15.37:0/3263128327 shutdown_connections 2024-06-10T00:45:28.708 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-06-10T00:45:28.158+0000 7ff413cf0700 1 -- 172.21.15.37:0/3263128327 wait complete. 2024-06-10T00:45:28.708 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-06-10T00:45:28.708 INFO:teuthology.orchestra.run.smithi037.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-06-10T00:45:28.708 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:45:28.708 INFO:teuthology.orchestra.run.smithi037.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-06-10T00:45:28.708 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:45:28.708 INFO:teuthology.orchestra.run.smithi037.stdout:Or, if you are only running a single cluster on this host: 2024-06-10T00:45:28.708 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:45:28.709 INFO:teuthology.orchestra.run.smithi037.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-06-10T00:45:28.709 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:45:28.709 INFO:teuthology.orchestra.run.smithi037.stdout:Please consider enabling telemetry to help improve Ceph: 2024-06-10T00:45:28.709 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:45:28.709 INFO:teuthology.orchestra.run.smithi037.stdout: ceph telemetry on 2024-06-10T00:45:28.709 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:45:28.709 INFO:teuthology.orchestra.run.smithi037.stdout:For more information see: 2024-06-10T00:45:28.709 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:45:28.709 INFO:teuthology.orchestra.run.smithi037.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-06-10T00:45:28.709 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:45:28.709 INFO:teuthology.orchestra.run.smithi037.stdout:Bootstrap complete. 2024-06-10T00:45:28.746 INFO:tasks.cephadm:Fetching config... 2024-06-10T00:45:28.747 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-06-10T00:45:28.747 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-06-10T00:45:28.764 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-06-10T00:45:28.764 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-06-10T00:45:28.764 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-06-10T00:45:28.820 INFO:tasks.cephadm:Fetching mon keyring... 2024-06-10T00:45:28.820 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-06-10T00:45:28.820 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/keyring of=/dev/stdout 2024-06-10T00:45:28.886 INFO:tasks.cephadm:Fetching pub ssh key... 2024-06-10T00:45:28.886 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-06-10T00:45:28.887 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-06-10T00:45:28.943 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-06-10T00:45:28.943 DEBUG:teuthology.orchestra.run.smithi037:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7TIymYqklc98F6uYaAs39Pd6IcD+eA4WVkiibprOitzX0D/nKxT/UFFq7moiNcqgG84Zh+VSrK1tLGWfsYWkKfPMnSZqlj5yfpkCAJ25hN5BHmiexFOKOzKvgt71orDXmbvy2+qj7t0rXPJbpYVVRa4CmHVgNSbdnIoWyA9oIOQwmR8fG680Lh4rp3fqO4x0SQi1ZTrxQeNMqWkhnATRhfkI8RxB76x9eJOSlGnKUXYc4YNK5rXpEtynTkpPPXsatYGFmSlUGjIHYd+zDxfrLfo3qGrp8pkJBv+BkezHQKrNSfufPbT7670eLLZn6fWonMS4fbpSlHYOvBIx/S4C4aGD/EhoIf59Bcj6U66k21rRhPwcic5NQIYv/9CgkwDIwtGZFJQnJKtKMF9s4oIuxMuEGG99D2g3kJww3idm/NFdw1A8sFJg42GMyWsGYQJnCzf7XYxgXcACs+TUvLihrtoe59iMWIhQT08nXWLxmzWFuzt8g8tZqzhS+Mkj5Yxs= ceph-68793514-26c2-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-10T00:45:29.017 INFO:teuthology.orchestra.run.smithi037.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7TIymYqklc98F6uYaAs39Pd6IcD+eA4WVkiibprOitzX0D/nKxT/UFFq7moiNcqgG84Zh+VSrK1tLGWfsYWkKfPMnSZqlj5yfpkCAJ25hN5BHmiexFOKOzKvgt71orDXmbvy2+qj7t0rXPJbpYVVRa4CmHVgNSbdnIoWyA9oIOQwmR8fG680Lh4rp3fqO4x0SQi1ZTrxQeNMqWkhnATRhfkI8RxB76x9eJOSlGnKUXYc4YNK5rXpEtynTkpPPXsatYGFmSlUGjIHYd+zDxfrLfo3qGrp8pkJBv+BkezHQKrNSfufPbT7670eLLZn6fWonMS4fbpSlHYOvBIx/S4C4aGD/EhoIf59Bcj6U66k21rRhPwcic5NQIYv/9CgkwDIwtGZFJQnJKtKMF9s4oIuxMuEGG99D2g3kJww3idm/NFdw1A8sFJg42GMyWsGYQJnCzf7XYxgXcACs+TUvLihrtoe59iMWIhQT08nXWLxmzWFuzt8g8tZqzhS+Mkj5Yxs= ceph-68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:45:29.031 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7TIymYqklc98F6uYaAs39Pd6IcD+eA4WVkiibprOitzX0D/nKxT/UFFq7moiNcqgG84Zh+VSrK1tLGWfsYWkKfPMnSZqlj5yfpkCAJ25hN5BHmiexFOKOzKvgt71orDXmbvy2+qj7t0rXPJbpYVVRa4CmHVgNSbdnIoWyA9oIOQwmR8fG680Lh4rp3fqO4x0SQi1ZTrxQeNMqWkhnATRhfkI8RxB76x9eJOSlGnKUXYc4YNK5rXpEtynTkpPPXsatYGFmSlUGjIHYd+zDxfrLfo3qGrp8pkJBv+BkezHQKrNSfufPbT7670eLLZn6fWonMS4fbpSlHYOvBIx/S4C4aGD/EhoIf59Bcj6U66k21rRhPwcic5NQIYv/9CgkwDIwtGZFJQnJKtKMF9s4oIuxMuEGG99D2g3kJww3idm/NFdw1A8sFJg42GMyWsGYQJnCzf7XYxgXcACs+TUvLihrtoe59iMWIhQT08nXWLxmzWFuzt8g8tZqzhS+Mkj5Yxs= ceph-68793514-26c2-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-10T00:45:29.070 INFO:teuthology.orchestra.run.smithi094.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7TIymYqklc98F6uYaAs39Pd6IcD+eA4WVkiibprOitzX0D/nKxT/UFFq7moiNcqgG84Zh+VSrK1tLGWfsYWkKfPMnSZqlj5yfpkCAJ25hN5BHmiexFOKOzKvgt71orDXmbvy2+qj7t0rXPJbpYVVRa4CmHVgNSbdnIoWyA9oIOQwmR8fG680Lh4rp3fqO4x0SQi1ZTrxQeNMqWkhnATRhfkI8RxB76x9eJOSlGnKUXYc4YNK5rXpEtynTkpPPXsatYGFmSlUGjIHYd+zDxfrLfo3qGrp8pkJBv+BkezHQKrNSfufPbT7670eLLZn6fWonMS4fbpSlHYOvBIx/S4C4aGD/EhoIf59Bcj6U66k21rRhPwcic5NQIYv/9CgkwDIwtGZFJQnJKtKMF9s4oIuxMuEGG99D2g3kJww3idm/NFdw1A8sFJg42GMyWsGYQJnCzf7XYxgXcACs+TUvLihrtoe59iMWIhQT08nXWLxmzWFuzt8g8tZqzhS+Mkj5Yxs= ceph-68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:45:29.081 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7TIymYqklc98F6uYaAs39Pd6IcD+eA4WVkiibprOitzX0D/nKxT/UFFq7moiNcqgG84Zh+VSrK1tLGWfsYWkKfPMnSZqlj5yfpkCAJ25hN5BHmiexFOKOzKvgt71orDXmbvy2+qj7t0rXPJbpYVVRa4CmHVgNSbdnIoWyA9oIOQwmR8fG680Lh4rp3fqO4x0SQi1ZTrxQeNMqWkhnATRhfkI8RxB76x9eJOSlGnKUXYc4YNK5rXpEtynTkpPPXsatYGFmSlUGjIHYd+zDxfrLfo3qGrp8pkJBv+BkezHQKrNSfufPbT7670eLLZn6fWonMS4fbpSlHYOvBIx/S4C4aGD/EhoIf59Bcj6U66k21rRhPwcic5NQIYv/9CgkwDIwtGZFJQnJKtKMF9s4oIuxMuEGG99D2g3kJww3idm/NFdw1A8sFJg42GMyWsGYQJnCzf7XYxgXcACs+TUvLihrtoe59iMWIhQT08nXWLxmzWFuzt8g8tZqzhS+Mkj5Yxs= ceph-68793514-26c2-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-10T00:45:29.120 INFO:teuthology.orchestra.run.smithi167.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7TIymYqklc98F6uYaAs39Pd6IcD+eA4WVkiibprOitzX0D/nKxT/UFFq7moiNcqgG84Zh+VSrK1tLGWfsYWkKfPMnSZqlj5yfpkCAJ25hN5BHmiexFOKOzKvgt71orDXmbvy2+qj7t0rXPJbpYVVRa4CmHVgNSbdnIoWyA9oIOQwmR8fG680Lh4rp3fqO4x0SQi1ZTrxQeNMqWkhnATRhfkI8RxB76x9eJOSlGnKUXYc4YNK5rXpEtynTkpPPXsatYGFmSlUGjIHYd+zDxfrLfo3qGrp8pkJBv+BkezHQKrNSfufPbT7670eLLZn6fWonMS4fbpSlHYOvBIx/S4C4aGD/EhoIf59Bcj6U66k21rRhPwcic5NQIYv/9CgkwDIwtGZFJQnJKtKMF9s4oIuxMuEGG99D2g3kJww3idm/NFdw1A8sFJg42GMyWsGYQJnCzf7XYxgXcACs+TUvLihrtoe59iMWIhQT08nXWLxmzWFuzt8g8tZqzhS+Mkj5Yxs= ceph-68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:45:29.132 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-06-10T00:45:29.266 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:29 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3263128327' entity='client.admin' 2024-06-10T00:45:29.384 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:45:31.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.078+0000 7f0374d58700 1 -- 172.21.15.37:0/1915560688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0370102a40 msgr2=0x7f0370104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:31.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.078+0000 7f0374d58700 1 --2- 172.21.15.37:0/1915560688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0370102a40 0x7f0370104e30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0360009910 tx=0x7f0360009c20 comp rx=0 tx=0).stop 2024-06-10T00:45:31.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.078+0000 7f0374d58700 1 -- 172.21.15.37:0/1915560688 shutdown_connections 2024-06-10T00:45:31.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.078+0000 7f0374d58700 1 --2- 172.21.15.37:0/1915560688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0370102a40 0x7f0370104e30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:31.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.078+0000 7f0374d58700 1 -- 172.21.15.37:0/1915560688 >> 172.21.15.37:0/1915560688 conn(0x7f03700fc390 msgr2=0x7f03700fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:31.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.078+0000 7f0374d58700 1 -- 172.21.15.37:0/1915560688 shutdown_connections 2024-06-10T00:45:31.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.078+0000 7f0374d58700 1 -- 172.21.15.37:0/1915560688 wait complete. 2024-06-10T00:45:31.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.079+0000 7f0374d58700 1 Processor -- start 2024-06-10T00:45:31.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.079+0000 7f0374d58700 1 -- start start 2024-06-10T00:45:31.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.079+0000 7f0374d58700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0370102a40 0x7f03701975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:31.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.079+0000 7f0374d58700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0370107f80 con 0x7f0370102a40 2024-06-10T00:45:31.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.080+0000 7f036e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0370102a40 0x7f03701975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:31.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.080+0000 7f036e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0370102a40 0x7f03701975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54092/0 (socket says 172.21.15.37:54092) 2024-06-10T00:45:31.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.080+0000 7f036e59c700 1 -- 172.21.15.37:0/2854368424 learned_addr learned my addr 172.21.15.37:0/2854368424 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:31.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.080+0000 7f036e59c700 1 -- 172.21.15.37:0/2854368424 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0370197b10 con 0x7f0370102a40 2024-06-10T00:45:31.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.080+0000 7f036e59c700 1 --2- 172.21.15.37:0/2854368424 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0370102a40 0x7f03701975d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0360004fa0 tx=0x7f0360005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:31.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.080+0000 7f035f7fe700 1 -- 172.21.15.37:0/2854368424 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0360027070 con 0x7f0370102a40 2024-06-10T00:45:31.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.080+0000 7f0374d58700 1 -- 172.21.15.37:0/2854368424 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0370197da0 con 0x7f0370102a40 2024-06-10T00:45:31.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.080+0000 7f035f7fe700 1 -- 172.21.15.37:0/2854368424 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f036000be40 con 0x7f0370102a40 2024-06-10T00:45:31.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.080+0000 7f035f7fe700 1 -- 172.21.15.37:0/2854368424 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f036000f460 con 0x7f0370102a40 2024-06-10T00:45:31.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.081+0000 7f0374d58700 1 -- 172.21.15.37:0/2854368424 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f037019aa40 con 0x7f0370102a40 2024-06-10T00:45:31.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.082+0000 7f035f7fe700 1 -- 172.21.15.37:0/2854368424 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f036002a070 con 0x7f0370102a40 2024-06-10T00:45:31.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.082+0000 7f0374d58700 1 -- 172.21.15.37:0/2854368424 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0370108700 con 0x7f0370102a40 2024-06-10T00:45:31.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.082+0000 7f035f7fe700 1 --2- 172.21.15.37:0/2854368424 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0358039150 0x7f035803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:31.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.082+0000 7f035f7fe700 1 -- 172.21.15.37:0/2854368424 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f036001e030 con 0x7f0370102a40 2024-06-10T00:45:31.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.082+0000 7f036dd9b700 1 --2- 172.21.15.37:0/2854368424 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0358039150 0x7f035803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:31.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.083+0000 7f036dd9b700 1 --2- 172.21.15.37:0/2854368424 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0358039150 0x7f035803b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0364006fd0 tx=0x7f0364006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:31.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.091+0000 7f035f7fe700 1 -- 172.21.15.37:0/2854368424 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0360013070 con 0x7f0370102a40 2024-06-10T00:45:31.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.296+0000 7f0374d58700 1 -- 172.21.15.37:0/2854368424 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f037002ddb0 con 0x7f0370102a40 2024-06-10T00:45:31.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.298+0000 7f035f7fe700 1 -- 172.21.15.37:0/2854368424 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f036002b3b0 con 0x7f0370102a40 2024-06-10T00:45:31.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.301+0000 7f0374d58700 1 -- 172.21.15.37:0/2854368424 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0358039150 msgr2=0x7f035803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:31.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.301+0000 7f0374d58700 1 --2- 172.21.15.37:0/2854368424 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0358039150 0x7f035803b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0364006fd0 tx=0x7f0364006e40 comp rx=0 tx=0).stop 2024-06-10T00:45:31.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.301+0000 7f0374d58700 1 -- 172.21.15.37:0/2854368424 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0370102a40 msgr2=0x7f03701975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:31.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.301+0000 7f0374d58700 1 --2- 172.21.15.37:0/2854368424 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0370102a40 0x7f03701975d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0360004fa0 tx=0x7f0360005080 comp rx=0 tx=0).stop 2024-06-10T00:45:31.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.301+0000 7f0374d58700 1 -- 172.21.15.37:0/2854368424 shutdown_connections 2024-06-10T00:45:31.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.301+0000 7f0374d58700 1 --2- 172.21.15.37:0/2854368424 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0358039150 0x7f035803b610 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:31.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.301+0000 7f0374d58700 1 --2- 172.21.15.37:0/2854368424 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0370102a40 0x7f03701975d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:31.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.301+0000 7f0374d58700 1 -- 172.21.15.37:0/2854368424 >> 172.21.15.37:0/2854368424 conn(0x7f03700fc390 msgr2=0x7f03700fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:31.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.301+0000 7f0374d58700 1 -- 172.21.15.37:0/2854368424 shutdown_connections 2024-06-10T00:45:31.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:31.301+0000 7f0374d58700 1 -- 172.21.15.37:0/2854368424 wait complete. 2024-06-10T00:45:32.028 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-06-10T00:45:32.029 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-06-10T00:45:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:45:32.553 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:32 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2854368424' entity='client.admin' 2024-06-10T00:45:32.553 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:32 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:32.553 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:32 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:32.553 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:32 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi037", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:45:32.553 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:32 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:32.553 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:32 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:45:32.553 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:32 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:33.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.651+0000 7f2cb37b6700 1 -- 172.21.15.37:0/2546479886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cac101f10 msgr2=0x7f2cac102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:33.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.651+0000 7f2cb37b6700 1 --2- 172.21.15.37:0/2546479886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cac101f10 0x7f2cac102330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c00b0a0 tx=0x7f2c9c00b3b0 comp rx=0 tx=0).stop 2024-06-10T00:45:33.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.651+0000 7f2cb37b6700 1 -- 172.21.15.37:0/2546479886 shutdown_connections 2024-06-10T00:45:33.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.651+0000 7f2cb37b6700 1 --2- 172.21.15.37:0/2546479886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cac101f10 0x7f2cac102330 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:33.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.651+0000 7f2cb37b6700 1 -- 172.21.15.37:0/2546479886 >> 172.21.15.37:0/2546479886 conn(0x7f2cac0fd4b0 msgr2=0x7f2cac0ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:33.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.651+0000 7f2cb37b6700 1 -- 172.21.15.37:0/2546479886 shutdown_connections 2024-06-10T00:45:33.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.651+0000 7f2cb37b6700 1 -- 172.21.15.37:0/2546479886 wait complete. 2024-06-10T00:45:33.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.652+0000 7f2cb37b6700 1 Processor -- start 2024-06-10T00:45:33.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.652+0000 7f2cb37b6700 1 -- start start 2024-06-10T00:45:33.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.652+0000 7f2cb37b6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cac101f10 0x7f2cac197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:33.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.652+0000 7f2cb37b6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac107f20 con 0x7f2cac101f10 2024-06-10T00:45:33.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.653+0000 7f2cb1552700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cac101f10 0x7f2cac197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:33.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.653+0000 7f2cb1552700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cac101f10 0x7f2cac197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55872/0 (socket says 172.21.15.37:55872) 2024-06-10T00:45:33.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.653+0000 7f2cb1552700 1 -- 172.21.15.37:0/2556728647 learned_addr learned my addr 172.21.15.37:0/2556728647 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:33.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.653+0000 7f2cb1552700 1 -- 172.21.15.37:0/2556728647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cac197ab0 con 0x7f2cac101f10 2024-06-10T00:45:33.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.653+0000 7f2cb1552700 1 --2- 172.21.15.37:0/2556728647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cac101f10 0x7f2cac197570 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c007890 tx=0x7f2c9c00bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:33.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.653+0000 7f2caa7fc700 1 -- 172.21.15.37:0/2556728647 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c9c017040 con 0x7f2cac101f10 2024-06-10T00:45:33.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.653+0000 7f2caa7fc700 1 -- 172.21.15.37:0/2556728647 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c9c022070 con 0x7f2cac101f10 2024-06-10T00:45:33.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.653+0000 7f2cb37b6700 1 -- 172.21.15.37:0/2556728647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cac197d40 con 0x7f2cac101f10 2024-06-10T00:45:33.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.653+0000 7f2caa7fc700 1 -- 172.21.15.37:0/2556728647 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c9c01d5b0 con 0x7f2cac101f10 2024-06-10T00:45:33.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.653+0000 7f2cb37b6700 1 -- 172.21.15.37:0/2556728647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cac19aa30 con 0x7f2cac101f10 2024-06-10T00:45:33.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.654+0000 7f2caa7fc700 1 -- 172.21.15.37:0/2556728647 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f2c9c01d860 con 0x7f2cac101f10 2024-06-10T00:45:33.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.654+0000 7f2cb37b6700 1 -- 172.21.15.37:0/2556728647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cac1086a0 con 0x7f2cac101f10 2024-06-10T00:45:33.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.654+0000 7f2caa7fc700 1 --2- 172.21.15.37:0/2556728647 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2c94039130 0x7f2c9403b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:33.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.655+0000 7f2caa7fc700 1 -- 172.21.15.37:0/2556728647 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2c9c00f050 con 0x7f2cac101f10 2024-06-10T00:45:33.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.655+0000 7f2cb0d51700 1 --2- 172.21.15.37:0/2556728647 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2c94039130 0x7f2c9403b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:33.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.655+0000 7f2cb0d51700 1 --2- 172.21.15.37:0/2556728647 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2c94039130 0x7f2c9403b5f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0006fd0 tx=0x7f2ca0006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:33.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.659+0000 7f2caa7fc700 1 -- 172.21.15.37:0/2556728647 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2c9c007e20 con 0x7f2cac101f10 2024-06-10T00:45:33.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.867+0000 7f2cb37b6700 1 -- 172.21.15.37:0/2556728647 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f2cac02ddb0 con 0x7f2c94039130 2024-06-10T00:45:33.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.871+0000 7f2caa7fc700 1 -- 172.21.15.37:0/2556728647 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2cac02ddb0 con 0x7f2c94039130 2024-06-10T00:45:33.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.873+0000 7f2cb37b6700 1 -- 172.21.15.37:0/2556728647 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2c94039130 msgr2=0x7f2c9403b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:33.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.873+0000 7f2cb37b6700 1 --2- 172.21.15.37:0/2556728647 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2c94039130 0x7f2c9403b5f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0006fd0 tx=0x7f2ca0006e40 comp rx=0 tx=0).stop 2024-06-10T00:45:33.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.873+0000 7f2cb37b6700 1 -- 172.21.15.37:0/2556728647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cac101f10 msgr2=0x7f2cac197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:33.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.873+0000 7f2cb37b6700 1 --2- 172.21.15.37:0/2556728647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cac101f10 0x7f2cac197570 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c007890 tx=0x7f2c9c00bd40 comp rx=0 tx=0).stop 2024-06-10T00:45:33.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.873+0000 7f2cb37b6700 1 -- 172.21.15.37:0/2556728647 shutdown_connections 2024-06-10T00:45:33.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.873+0000 7f2cb37b6700 1 --2- 172.21.15.37:0/2556728647 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2c94039130 0x7f2c9403b5f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:33.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.873+0000 7f2cb37b6700 1 --2- 172.21.15.37:0/2556728647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cac101f10 0x7f2cac197570 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:33.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.873+0000 7f2cb37b6700 1 -- 172.21.15.37:0/2556728647 >> 172.21.15.37:0/2556728647 conn(0x7f2cac0fd4b0 msgr2=0x7f2cac0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:33.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.873+0000 7f2cb37b6700 1 -- 172.21.15.37:0/2556728647 shutdown_connections 2024-06-10T00:45:33.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:33.873+0000 7f2cb37b6700 1 -- 172.21.15.37:0/2556728647 wait complete. 2024-06-10T00:45:34.587 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi094 2024-06-10T00:45:34.587 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-10T00:45:34.587 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/etc/ceph/ceph.conf 2024-06-10T00:45:34.605 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-10T00:45:34.605 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:45:34.663 INFO:tasks.cephadm:Adding host smithi094 to orchestrator... 2024-06-10T00:45:34.664 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph orch host add smithi094 2024-06-10T00:45:34.914 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:45:35.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:34 smithi037 ceph-mon[32192]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:45:35.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:34 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:35.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:34 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:45:35.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:34 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:45:35.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:34 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:45:35.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:34 smithi037 ceph-mon[32192]: Updating smithi037:/etc/ceph/ceph.conf 2024-06-10T00:45:36.267 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:35 smithi037 ceph-mon[32192]: Updating smithi037:/var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:45:36.267 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:35 smithi037 ceph-mon[32192]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:45:36.267 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:35 smithi037 ceph-mon[32192]: Updating smithi037:/var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-10T00:45:36.268 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:35 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:36.268 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:35 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:36.268 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:35 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:36.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.317+0000 7fe8d3268700 1 -- 172.21.15.37:0/3380250032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe8cc100c30 msgr2=0x7fe8cc101010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:36.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.317+0000 7fe8d3268700 1 --2- 172.21.15.37:0/3380250032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe8cc100c30 0x7fe8cc101010 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009910 tx=0x7fe8c8009c20 comp rx=0 tx=0).stop 2024-06-10T00:45:36.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.317+0000 7fe8d3268700 1 -- 172.21.15.37:0/3380250032 shutdown_connections 2024-06-10T00:45:36.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.317+0000 7fe8d3268700 1 --2- 172.21.15.37:0/3380250032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe8cc100c30 0x7fe8cc101010 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:36.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.317+0000 7fe8d3268700 1 -- 172.21.15.37:0/3380250032 >> 172.21.15.37:0/3380250032 conn(0x7fe8cc0fc8f0 msgr2=0x7fe8cc0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:36.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.317+0000 7fe8d3268700 1 -- 172.21.15.37:0/3380250032 shutdown_connections 2024-06-10T00:45:36.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.317+0000 7fe8d3268700 1 -- 172.21.15.37:0/3380250032 wait complete. 2024-06-10T00:45:36.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.318+0000 7fe8d3268700 1 Processor -- start 2024-06-10T00:45:36.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.318+0000 7fe8d3268700 1 -- start start 2024-06-10T00:45:36.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.318+0000 7fe8d3268700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe8cc100c30 0x7fe8cc19f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:36.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.318+0000 7fe8d3268700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe8cc10e770 con 0x7fe8cc100c30 2024-06-10T00:45:36.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.319+0000 7fe8d1004700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe8cc100c30 0x7fe8cc19f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:36.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.319+0000 7fe8d1004700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe8cc100c30 0x7fe8cc19f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55892/0 (socket says 172.21.15.37:55892) 2024-06-10T00:45:36.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.319+0000 7fe8d1004700 1 -- 172.21.15.37:0/2710801707 learned_addr learned my addr 172.21.15.37:0/2710801707 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:36.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.319+0000 7fe8d1004700 1 -- 172.21.15.37:0/2710801707 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8cc19fda0 con 0x7fe8cc100c30 2024-06-10T00:45:36.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.319+0000 7fe8d1004700 1 --2- 172.21.15.37:0/2710801707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe8cc100c30 0x7fe8cc19f860 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8005850 tx=0x7fe8c8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:36.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.319+0000 7fe8c27fc700 1 -- 172.21.15.37:0/2710801707 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8c8026070 con 0x7fe8cc100c30 2024-06-10T00:45:36.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.319+0000 7fe8c27fc700 1 -- 172.21.15.37:0/2710801707 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8c800bc30 con 0x7fe8cc100c30 2024-06-10T00:45:36.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.319+0000 7fe8d3268700 1 -- 172.21.15.37:0/2710801707 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8cc1a0030 con 0x7fe8cc100c30 2024-06-10T00:45:36.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.319+0000 7fe8d3268700 1 -- 172.21.15.37:0/2710801707 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8cc199a20 con 0x7fe8cc100c30 2024-06-10T00:45:36.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.319+0000 7fe8c27fc700 1 -- 172.21.15.37:0/2710801707 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8c8003a20 con 0x7fe8cc100c30 2024-06-10T00:45:36.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.320+0000 7fe8c27fc700 1 -- 172.21.15.37:0/2710801707 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe8c8029070 con 0x7fe8cc100c30 2024-06-10T00:45:36.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.320+0000 7fe8d3268700 1 -- 172.21.15.37:0/2710801707 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8cc06abe0 con 0x7fe8cc100c30 2024-06-10T00:45:36.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.320+0000 7fe8c27fc700 1 --2- 172.21.15.37:0/2710801707 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe8b8039150 0x7fe8b803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:36.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.320+0000 7fe8c27fc700 1 -- 172.21.15.37:0/2710801707 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe8c801e070 con 0x7fe8cc100c30 2024-06-10T00:45:36.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.321+0000 7fe8d0803700 1 --2- 172.21.15.37:0/2710801707 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe8b8039150 0x7fe8b803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:36.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.321+0000 7fe8d0803700 1 --2- 172.21.15.37:0/2710801707 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe8b8039150 0x7fe8b803b610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc006fd0 tx=0x7fe8bc006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:36.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.326+0000 7fe8c27fc700 1 -- 172.21.15.37:0/2710801707 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe8c8050030 con 0x7fe8cc100c30 2024-06-10T00:45:36.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:36.537+0000 7fe8d3268700 1 -- 172.21.15.37:0/2710801707 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi094", "target": ["mon-mgr", ""]}) -- 0x7fe8cc19af50 con 0x7fe8b8039150 2024-06-10T00:45:37.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:37 smithi037 ceph-mon[32192]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi094", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:45:38.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:38 smithi037 ceph-mon[32192]: Deploying cephadm binary to smithi094 2024-06-10T00:45:40.890 INFO:teuthology.orchestra.run.smithi037.stdout:Added host 'smithi094' with addr '172.21.15.94' 2024-06-10T00:45:40.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:40.889+0000 7fe8c27fc700 1 -- 172.21.15.37:0/2710801707 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fe8cc19af50 con 0x7fe8b8039150 2024-06-10T00:45:40.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:40.893+0000 7fe8d3268700 1 -- 172.21.15.37:0/2710801707 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe8b8039150 msgr2=0x7fe8b803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:40.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:40.893+0000 7fe8d3268700 1 --2- 172.21.15.37:0/2710801707 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe8b8039150 0x7fe8b803b610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc006fd0 tx=0x7fe8bc006e40 comp rx=0 tx=0).stop 2024-06-10T00:45:40.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:40.893+0000 7fe8d3268700 1 -- 172.21.15.37:0/2710801707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe8cc100c30 msgr2=0x7fe8cc19f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:40.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:40.893+0000 7fe8d3268700 1 --2- 172.21.15.37:0/2710801707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe8cc100c30 0x7fe8cc19f860 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8005850 tx=0x7fe8c8005030 comp rx=0 tx=0).stop 2024-06-10T00:45:40.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:40.893+0000 7fe8d3268700 1 -- 172.21.15.37:0/2710801707 shutdown_connections 2024-06-10T00:45:40.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:40.893+0000 7fe8d3268700 1 --2- 172.21.15.37:0/2710801707 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe8b8039150 0x7fe8b803b610 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:40.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:40.893+0000 7fe8d3268700 1 --2- 172.21.15.37:0/2710801707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe8cc100c30 0x7fe8cc19f860 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:40.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:40.893+0000 7fe8d3268700 1 -- 172.21.15.37:0/2710801707 >> 172.21.15.37:0/2710801707 conn(0x7fe8cc0fc8f0 msgr2=0x7fe8cc0fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:40.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:40.893+0000 7fe8d3268700 1 -- 172.21.15.37:0/2710801707 shutdown_connections 2024-06-10T00:45:40.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:40.893+0000 7fe8d3268700 1 -- 172.21.15.37:0/2710801707 wait complete. 2024-06-10T00:45:41.462 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-10T00:45:41.713 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:45:42.089 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:41 smithi037 ceph-mon[32192]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:45:42.090 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:42.090 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:41 smithi037 ceph-mon[32192]: Added host smithi094 2024-06-10T00:45:42.090 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:45:42.090 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:43.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.028+0000 7f09ea791700 1 -- 172.21.15.37:0/1138856938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09e4101040 msgr2=0x7f09e4101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:43.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.028+0000 7f09ea791700 1 --2- 172.21.15.37:0/1138856938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09e4101040 0x7f09e4101420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f09d4009910 tx=0x7f09d4009c20 comp rx=0 tx=0).stop 2024-06-10T00:45:43.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.028+0000 7f09ea791700 1 -- 172.21.15.37:0/1138856938 shutdown_connections 2024-06-10T00:45:43.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.028+0000 7f09ea791700 1 --2- 172.21.15.37:0/1138856938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09e4101040 0x7f09e4101420 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:43.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.028+0000 7f09ea791700 1 -- 172.21.15.37:0/1138856938 >> 172.21.15.37:0/1138856938 conn(0x7f09e40768f0 msgr2=0x7f09e4076d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:43.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.028+0000 7f09ea791700 1 -- 172.21.15.37:0/1138856938 shutdown_connections 2024-06-10T00:45:43.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.028+0000 7f09ea791700 1 -- 172.21.15.37:0/1138856938 wait complete. 2024-06-10T00:45:43.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.029+0000 7f09ea791700 1 Processor -- start 2024-06-10T00:45:43.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.029+0000 7f09ea791700 1 -- start start 2024-06-10T00:45:43.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.029+0000 7f09ea791700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09e4101040 0x7f09e419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:43.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.029+0000 7f09ea791700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f09e410b3c0 con 0x7f09e4101040 2024-06-10T00:45:43.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.029+0000 7f09e3fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09e4101040 0x7f09e419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:43.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.029+0000 7f09e3fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09e4101040 0x7f09e419c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55846/0 (socket says 172.21.15.37:55846) 2024-06-10T00:45:43.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.029+0000 7f09e3fff700 1 -- 172.21.15.37:0/33255426 learned_addr learned my addr 172.21.15.37:0/33255426 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:43.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.029+0000 7f09e3fff700 1 -- 172.21.15.37:0/33255426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e419cbd0 con 0x7f09e4101040 2024-06-10T00:45:43.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.029+0000 7f09e3fff700 1 --2- 172.21.15.37:0/33255426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09e4101040 0x7f09e419c690 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f09d4005850 tx=0x7f09d4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:43.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.030+0000 7f09e1ffb700 1 -- 172.21.15.37:0/33255426 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09d4026070 con 0x7f09e4101040 2024-06-10T00:45:43.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.030+0000 7f09ea791700 1 -- 172.21.15.37:0/33255426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e419cec0 con 0x7f09e4101040 2024-06-10T00:45:43.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.030+0000 7f09e1ffb700 1 -- 172.21.15.37:0/33255426 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09d400bbd0 con 0x7f09e4101040 2024-06-10T00:45:43.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.030+0000 7f09e1ffb700 1 -- 172.21.15.37:0/33255426 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09d4003970 con 0x7f09e4101040 2024-06-10T00:45:43.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.030+0000 7f09ea791700 1 -- 172.21.15.37:0/33255426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e4196930 con 0x7f09e4101040 2024-06-10T00:45:43.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.031+0000 7f09e1ffb700 1 -- 172.21.15.37:0/33255426 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f09d4029070 con 0x7f09e4101040 2024-06-10T00:45:43.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.031+0000 7f09e1ffb700 1 --2- 172.21.15.37:0/33255426 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f09d00390f0 0x7f09d003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:43.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.031+0000 7f09e1ffb700 1 -- 172.21.15.37:0/33255426 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f09d401e070 con 0x7f09e4101040 2024-06-10T00:45:43.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.031+0000 7f09ea791700 1 -- 172.21.15.37:0/33255426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e41024d0 con 0x7f09e4101040 2024-06-10T00:45:43.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.031+0000 7f09dbfff700 1 --2- 172.21.15.37:0/33255426 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f09d00390f0 0x7f09d003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:43.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.032+0000 7f09dbfff700 1 --2- 172.21.15.37:0/33255426 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f09d00390f0 0x7f09d003b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f09cc006fd0 tx=0x7f09cc006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:43.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.036+0000 7f09e1ffb700 1 -- 172.21.15.37:0/33255426 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f09d4030070 con 0x7f09e4101040 2024-06-10T00:45:43.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:42 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:43.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.249+0000 7f09ea791700 1 -- 172.21.15.37:0/33255426 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f09e4197c90 con 0x7f09d00390f0 2024-06-10T00:45:43.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.250+0000 7f09e1ffb700 1 -- 172.21.15.37:0/33255426 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f09e4197c90 con 0x7f09d00390f0 2024-06-10T00:45:43.250 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:45:43.251 INFO:teuthology.orchestra.run.smithi037.stdout:[{"addr": "172.21.15.37", "hostname": "smithi037", "labels": [], "status": ""}, {"addr": "172.21.15.94", "hostname": "smithi094", "labels": [], "status": ""}] 2024-06-10T00:45:43.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.252+0000 7f09ea791700 1 -- 172.21.15.37:0/33255426 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f09d00390f0 msgr2=0x7f09d003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:43.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.252+0000 7f09ea791700 1 --2- 172.21.15.37:0/33255426 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f09d00390f0 0x7f09d003b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f09cc006fd0 tx=0x7f09cc006e40 comp rx=0 tx=0).stop 2024-06-10T00:45:43.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.252+0000 7f09ea791700 1 -- 172.21.15.37:0/33255426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09e4101040 msgr2=0x7f09e419c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:43.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.252+0000 7f09ea791700 1 --2- 172.21.15.37:0/33255426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09e4101040 0x7f09e419c690 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f09d4005850 tx=0x7f09d4005030 comp rx=0 tx=0).stop 2024-06-10T00:45:43.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.252+0000 7f09ea791700 1 -- 172.21.15.37:0/33255426 shutdown_connections 2024-06-10T00:45:43.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.252+0000 7f09ea791700 1 --2- 172.21.15.37:0/33255426 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f09d00390f0 0x7f09d003b5b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:43.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.252+0000 7f09ea791700 1 --2- 172.21.15.37:0/33255426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09e4101040 0x7f09e419c690 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:43.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.252+0000 7f09ea791700 1 -- 172.21.15.37:0/33255426 >> 172.21.15.37:0/33255426 conn(0x7f09e40768f0 msgr2=0x7f09e4103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:43.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.252+0000 7f09ea791700 1 -- 172.21.15.37:0/33255426 shutdown_connections 2024-06-10T00:45:43.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:43.252+0000 7f09ea791700 1 -- 172.21.15.37:0/33255426 wait complete. 2024-06-10T00:45:43.793 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi167 2024-06-10T00:45:43.793 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-10T00:45:43.793 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/etc/ceph/ceph.conf 2024-06-10T00:45:43.813 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-10T00:45:43.813 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:45:43.870 INFO:tasks.cephadm:Adding host smithi167 to orchestrator... 2024-06-10T00:45:43.871 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph orch host add smithi167 2024-06-10T00:45:44.118 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:45:44.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:43 smithi037 ceph-mon[32192]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:45:44.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:43 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:45.264 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:44 smithi037 ceph-mon[32192]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:45:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.490+0000 7ff4be165700 1 -- 172.21.15.37:0/1585674229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4b8102100 msgr2=0x7ff4b81024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.490+0000 7ff4be165700 1 --2- 172.21.15.37:0/1585674229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4b8102100 0x7ff4b81024e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8009910 tx=0x7ff4a8009c20 comp rx=0 tx=0).stop 2024-06-10T00:45:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.490+0000 7ff4be165700 1 -- 172.21.15.37:0/1585674229 shutdown_connections 2024-06-10T00:45:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.490+0000 7ff4be165700 1 --2- 172.21.15.37:0/1585674229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4b8102100 0x7ff4b81024e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.490+0000 7ff4be165700 1 -- 172.21.15.37:0/1585674229 >> 172.21.15.37:0/1585674229 conn(0x7ff4b80fd9d0 msgr2=0x7ff4b80ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.490+0000 7ff4be165700 1 -- 172.21.15.37:0/1585674229 shutdown_connections 2024-06-10T00:45:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.491+0000 7ff4be165700 1 -- 172.21.15.37:0/1585674229 wait complete. 2024-06-10T00:45:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.491+0000 7ff4be165700 1 Processor -- start 2024-06-10T00:45:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.491+0000 7ff4be165700 1 -- start start 2024-06-10T00:45:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.491+0000 7ff4be165700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4b8102100 0x7ff4b8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.491+0000 7ff4be165700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b810c4d0 con 0x7ff4b8102100 2024-06-10T00:45:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.492+0000 7ff4b77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4b8102100 0x7ff4b8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.492+0000 7ff4b77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4b8102100 0x7ff4b8074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55860/0 (socket says 172.21.15.37:55860) 2024-06-10T00:45:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.492+0000 7ff4b77fe700 1 -- 172.21.15.37:0/1883422010 learned_addr learned my addr 172.21.15.37:0/1883422010 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.492+0000 7ff4b77fe700 1 -- 172.21.15.37:0/1883422010 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b8076680 con 0x7ff4b8102100 2024-06-10T00:45:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.492+0000 7ff4b77fe700 1 --2- 172.21.15.37:0/1883422010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4b8102100 0x7ff4b8074790 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8005850 tx=0x7ff4a8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:45.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.492+0000 7ff4b4ff9700 1 -- 172.21.15.37:0/1883422010 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4a8026070 con 0x7ff4b8102100 2024-06-10T00:45:45.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.492+0000 7ff4b4ff9700 1 -- 172.21.15.37:0/1883422010 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a800bc30 con 0x7ff4b8102100 2024-06-10T00:45:45.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.492+0000 7ff4be165700 1 -- 172.21.15.37:0/1883422010 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b8074d30 con 0x7ff4b8102100 2024-06-10T00:45:45.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.492+0000 7ff4b4ff9700 1 -- 172.21.15.37:0/1883422010 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4a8003a20 con 0x7ff4b8102100 2024-06-10T00:45:45.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.492+0000 7ff4be165700 1 -- 172.21.15.37:0/1883422010 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b80751f0 con 0x7ff4b8102100 2024-06-10T00:45:45.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.493+0000 7ff4b4ff9700 1 -- 172.21.15.37:0/1883422010 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff4a8029070 con 0x7ff4b8102100 2024-06-10T00:45:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.493+0000 7ff4be165700 1 -- 172.21.15.37:0/1883422010 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4b81035e0 con 0x7ff4b8102100 2024-06-10T00:45:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.493+0000 7ff4b4ff9700 1 --2- 172.21.15.37:0/1883422010 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff4a00390f0 0x7ff4a003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.493+0000 7ff4b4ff9700 1 -- 172.21.15.37:0/1883422010 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff4a801e070 con 0x7ff4b8102100 2024-06-10T00:45:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.494+0000 7ff4b6ffd700 1 --2- 172.21.15.37:0/1883422010 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff4a00390f0 0x7ff4a003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.494+0000 7ff4b6ffd700 1 --2- 172.21.15.37:0/1883422010 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff4a00390f0 0x7ff4a003b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac006fd0 tx=0x7ff4ac006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.499+0000 7ff4b4ff9700 1 -- 172.21.15.37:0/1883422010 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff4a8013070 con 0x7ff4b8102100 2024-06-10T00:45:45.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:45.710+0000 7ff4be165700 1 -- 172.21.15.37:0/1883422010 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}) -- 0x7ff4b8075cc0 con 0x7ff4a00390f0 2024-06-10T00:45:46.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:45 smithi037 ceph-mon[32192]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:45:47.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:46 smithi037 ceph-mon[32192]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:45:47.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:46 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:47.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:46 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:47.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:46 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:47.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:46 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:47.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:46 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi094", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:45:47.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:46 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:45:47.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:46 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:45:48.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:47 smithi037 ceph-mon[32192]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:45:48.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:47 smithi037 ceph-mon[32192]: Deploying cephadm binary to smithi167 2024-06-10T00:45:48.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:47 smithi037 ceph-mon[32192]: Updating smithi094:/etc/ceph/ceph.conf 2024-06-10T00:45:48.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:47 smithi037 ceph-mon[32192]: Updating smithi094:/var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:45:49.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:49.966+0000 7ff4b4ff9700 1 -- 172.21.15.37:0/1883422010 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7ff4b8075cc0 con 0x7ff4a00390f0 2024-06-10T00:45:49.968 INFO:teuthology.orchestra.run.smithi037.stdout:Added host 'smithi167' with addr '172.21.15.167' 2024-06-10T00:45:49.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:49.968+0000 7ff4be165700 1 -- 172.21.15.37:0/1883422010 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff4a00390f0 msgr2=0x7ff4a003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:49.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:49.968+0000 7ff4be165700 1 --2- 172.21.15.37:0/1883422010 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff4a00390f0 0x7ff4a003b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac006fd0 tx=0x7ff4ac006e40 comp rx=0 tx=0).stop 2024-06-10T00:45:49.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:49.968+0000 7ff4be165700 1 -- 172.21.15.37:0/1883422010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4b8102100 msgr2=0x7ff4b8074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:49.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:49.968+0000 7ff4be165700 1 --2- 172.21.15.37:0/1883422010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4b8102100 0x7ff4b8074790 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8005850 tx=0x7ff4a8005030 comp rx=0 tx=0).stop 2024-06-10T00:45:49.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:49.968+0000 7ff4be165700 1 -- 172.21.15.37:0/1883422010 shutdown_connections 2024-06-10T00:45:49.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:49.968+0000 7ff4be165700 1 --2- 172.21.15.37:0/1883422010 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff4a00390f0 0x7ff4a003b5b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:49.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:49.968+0000 7ff4be165700 1 --2- 172.21.15.37:0/1883422010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4b8102100 0x7ff4b8074790 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:49.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:49.968+0000 7ff4be165700 1 -- 172.21.15.37:0/1883422010 >> 172.21.15.37:0/1883422010 conn(0x7ff4b80fd9d0 msgr2=0x7ff4b8104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:49.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:49.968+0000 7ff4be165700 1 -- 172.21.15.37:0/1883422010 shutdown_connections 2024-06-10T00:45:49.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:49.968+0000 7ff4be165700 1 -- 172.21.15.37:0/1883422010 wait complete. 2024-06-10T00:45:50.294 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:49 smithi037 ceph-mon[32192]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:45:50.294 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:49 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:50.294 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:49 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:50.294 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:49 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:50.294 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:49 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:50.558 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-10T00:45:50.801 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:45:51.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:50 smithi037 ceph-mon[32192]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:45:51.215 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:50 smithi037 ceph-mon[32192]: Updating smithi094:/var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-10T00:45:51.215 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:50 smithi037 ceph-mon[32192]: Added host smithi167 2024-06-10T00:45:51.215 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:50 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:45:51.215 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:50 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:52.111 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:52 smithi037 ceph-mon[32192]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:45:52.112 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:52 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:52.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.145+0000 7fcded95e700 1 -- 172.21.15.37:0/3060014962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcde8102100 msgr2=0x7fcde81024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:52.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.145+0000 7fcded95e700 1 --2- 172.21.15.37:0/3060014962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcde8102100 0x7fcde81024e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8009910 tx=0x7fcdd8009c20 comp rx=0 tx=0).stop 2024-06-10T00:45:52.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.145+0000 7fcded95e700 1 -- 172.21.15.37:0/3060014962 shutdown_connections 2024-06-10T00:45:52.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.145+0000 7fcded95e700 1 --2- 172.21.15.37:0/3060014962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcde8102100 0x7fcde81024e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:52.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.145+0000 7fcded95e700 1 -- 172.21.15.37:0/3060014962 >> 172.21.15.37:0/3060014962 conn(0x7fcde80fd9d0 msgr2=0x7fcde80ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:52.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.145+0000 7fcded95e700 1 -- 172.21.15.37:0/3060014962 shutdown_connections 2024-06-10T00:45:52.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.145+0000 7fcded95e700 1 -- 172.21.15.37:0/3060014962 wait complete. 2024-06-10T00:45:52.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.146+0000 7fcded95e700 1 Processor -- start 2024-06-10T00:45:52.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.146+0000 7fcded95e700 1 -- start start 2024-06-10T00:45:52.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.146+0000 7fcded95e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcde8102100 0x7fcde819f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:52.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.146+0000 7fcded95e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcde810c4d0 con 0x7fcde8102100 2024-06-10T00:45:52.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.147+0000 7fcde6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcde8102100 0x7fcde819f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:52.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.147+0000 7fcde6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcde8102100 0x7fcde819f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40226/0 (socket says 172.21.15.37:40226) 2024-06-10T00:45:52.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.147+0000 7fcde6ffd700 1 -- 172.21.15.37:0/3758097300 learned_addr learned my addr 172.21.15.37:0/3758097300 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:52.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.147+0000 7fcde6ffd700 1 -- 172.21.15.37:0/3758097300 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde819fe20 con 0x7fcde8102100 2024-06-10T00:45:52.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.147+0000 7fcde6ffd700 1 --2- 172.21.15.37:0/3758097300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcde8102100 0x7fcde819f8e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8005850 tx=0x7fcdd8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:52.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.147+0000 7fcdec95c700 1 -- 172.21.15.37:0/3758097300 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcdd8026070 con 0x7fcde8102100 2024-06-10T00:45:52.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.147+0000 7fcdec95c700 1 -- 172.21.15.37:0/3758097300 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdd800bbd0 con 0x7fcde8102100 2024-06-10T00:45:52.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.147+0000 7fcdec95c700 1 -- 172.21.15.37:0/3758097300 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcdd8003970 con 0x7fcde8102100 2024-06-10T00:45:52.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.147+0000 7fcded95e700 1 -- 172.21.15.37:0/3758097300 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde81a0110 con 0x7fcde8102100 2024-06-10T00:45:52.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.147+0000 7fcded95e700 1 -- 172.21.15.37:0/3758097300 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde8199aa0 con 0x7fcde8102100 2024-06-10T00:45:52.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.148+0000 7fcdec95c700 1 -- 172.21.15.37:0/3758097300 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fcdd8029070 con 0x7fcde8102100 2024-06-10T00:45:52.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.149+0000 7fcdec95c700 1 --2- 172.21.15.37:0/3758097300 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fcdd0039150 0x7fcdd003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:52.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.148+0000 7fcded95e700 1 -- 172.21.15.37:0/3758097300 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcde81035e0 con 0x7fcde8102100 2024-06-10T00:45:52.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.149+0000 7fcdec95c700 1 -- 172.21.15.37:0/3758097300 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcdd801e070 con 0x7fcde8102100 2024-06-10T00:45:52.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.149+0000 7fcde67fc700 1 --2- 172.21.15.37:0/3758097300 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fcdd0039150 0x7fcdd003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:52.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.149+0000 7fcde67fc700 1 --2- 172.21.15.37:0/3758097300 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fcdd0039150 0x7fcdd003b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcddc006fd0 tx=0x7fcddc006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:52.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.154+0000 7fcdec95c700 1 -- 172.21.15.37:0/3758097300 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcdd8013070 con 0x7fcde8102100 2024-06-10T00:45:52.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.370+0000 7fcded95e700 1 -- 172.21.15.37:0/3758097300 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcde819afd0 con 0x7fcdd0039150 2024-06-10T00:45:52.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.371+0000 7fcdec95c700 1 -- 172.21.15.37:0/3758097300 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7fcde819afd0 con 0x7fcdd0039150 2024-06-10T00:45:52.372 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:45:52.372 INFO:teuthology.orchestra.run.smithi037.stdout:[{"addr": "172.21.15.37", "hostname": "smithi037", "labels": [], "status": ""}, {"addr": "172.21.15.94", "hostname": "smithi094", "labels": [], "status": ""}, {"addr": "172.21.15.167", "hostname": "smithi167", "labels": [], "status": ""}] 2024-06-10T00:45:52.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.373+0000 7fcded95e700 1 -- 172.21.15.37:0/3758097300 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fcdd0039150 msgr2=0x7fcdd003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:52.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.373+0000 7fcded95e700 1 --2- 172.21.15.37:0/3758097300 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fcdd0039150 0x7fcdd003b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcddc006fd0 tx=0x7fcddc006e40 comp rx=0 tx=0).stop 2024-06-10T00:45:52.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.373+0000 7fcded95e700 1 -- 172.21.15.37:0/3758097300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcde8102100 msgr2=0x7fcde819f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:52.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.373+0000 7fcded95e700 1 --2- 172.21.15.37:0/3758097300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcde8102100 0x7fcde819f8e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8005850 tx=0x7fcdd8005030 comp rx=0 tx=0).stop 2024-06-10T00:45:52.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.373+0000 7fcded95e700 1 -- 172.21.15.37:0/3758097300 shutdown_connections 2024-06-10T00:45:52.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.373+0000 7fcded95e700 1 --2- 172.21.15.37:0/3758097300 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fcdd0039150 0x7fcdd003b610 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:52.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.373+0000 7fcded95e700 1 --2- 172.21.15.37:0/3758097300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcde8102100 0x7fcde819f8e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:52.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.373+0000 7fcded95e700 1 -- 172.21.15.37:0/3758097300 >> 172.21.15.37:0/3758097300 conn(0x7fcde80fd9d0 msgr2=0x7fcde80746c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:52.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.373+0000 7fcded95e700 1 -- 172.21.15.37:0/3758097300 shutdown_connections 2024-06-10T00:45:52.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:52.373+0000 7fcded95e700 1 -- 172.21.15.37:0/3758097300 wait complete. 2024-06-10T00:45:52.931 INFO:tasks.cephadm:Setting crush tunables to default 2024-06-10T00:45:52.931 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd crush tunables default 2024-06-10T00:45:53.177 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:45:53.869 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:53 smithi037 ceph-mon[32192]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:45:53.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:53 smithi037 ceph-mon[32192]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:45:53.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:53 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:54.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.521+0000 7f125c47a700 1 -- 172.21.15.37:0/2729328608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12541020e0 msgr2=0x7f12541024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:54.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.521+0000 7f125c47a700 1 --2- 172.21.15.37:0/2729328608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12541020e0 0x7f12541024c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1250009910 tx=0x7f1250009c20 comp rx=0 tx=0).stop 2024-06-10T00:45:54.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.521+0000 7f125c47a700 1 -- 172.21.15.37:0/2729328608 shutdown_connections 2024-06-10T00:45:54.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.521+0000 7f125c47a700 1 --2- 172.21.15.37:0/2729328608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12541020e0 0x7f12541024c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:54.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.521+0000 7f125c47a700 1 -- 172.21.15.37:0/2729328608 >> 172.21.15.37:0/2729328608 conn(0x7f12540fd9d0 msgr2=0x7f12540ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:54.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.521+0000 7f125c47a700 1 -- 172.21.15.37:0/2729328608 shutdown_connections 2024-06-10T00:45:54.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.521+0000 7f125c47a700 1 -- 172.21.15.37:0/2729328608 wait complete. 2024-06-10T00:45:54.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.521+0000 7f125c47a700 1 Processor -- start 2024-06-10T00:45:54.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.522+0000 7f125c47a700 1 -- start start 2024-06-10T00:45:54.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.522+0000 7f125c47a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12541020e0 0x7f1254074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:54.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.522+0000 7f125c47a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f125410c4b0 con 0x7f12541020e0 2024-06-10T00:45:54.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.522+0000 7f125a216700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12541020e0 0x7f1254074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:54.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.522+0000 7f125a216700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12541020e0 0x7f1254074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40244/0 (socket says 172.21.15.37:40244) 2024-06-10T00:45:54.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.522+0000 7f125a216700 1 -- 172.21.15.37:0/1911006433 learned_addr learned my addr 172.21.15.37:0/1911006433 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:45:54.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.522+0000 7f125a216700 1 -- 172.21.15.37:0/1911006433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1254076680 con 0x7f12541020e0 2024-06-10T00:45:54.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.522+0000 7f125a216700 1 --2- 172.21.15.37:0/1911006433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12541020e0 0x7f1254074790 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1250005850 tx=0x7f1250005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:54.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.522+0000 7f124b7fe700 1 -- 172.21.15.37:0/1911006433 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1250026070 con 0x7f12541020e0 2024-06-10T00:45:54.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.522+0000 7f124b7fe700 1 -- 172.21.15.37:0/1911006433 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f125000bc30 con 0x7f12541020e0 2024-06-10T00:45:54.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.523+0000 7f125c47a700 1 -- 172.21.15.37:0/1911006433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1254074d30 con 0x7f12541020e0 2024-06-10T00:45:54.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.523+0000 7f124b7fe700 1 -- 172.21.15.37:0/1911006433 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1250003a20 con 0x7f12541020e0 2024-06-10T00:45:54.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.523+0000 7f125c47a700 1 -- 172.21.15.37:0/1911006433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12540751f0 con 0x7f12541020e0 2024-06-10T00:45:54.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.523+0000 7f124b7fe700 1 -- 172.21.15.37:0/1911006433 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1250029070 con 0x7f12541020e0 2024-06-10T00:45:54.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.523+0000 7f125c47a700 1 -- 172.21.15.37:0/1911006433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12541035c0 con 0x7f12541020e0 2024-06-10T00:45:54.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.524+0000 7f124b7fe700 1 --2- 172.21.15.37:0/1911006433 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f12400390f0 0x7f124003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:54.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.524+0000 7f124b7fe700 1 -- 172.21.15.37:0/1911006433 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f125001e070 con 0x7f12541020e0 2024-06-10T00:45:54.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.524+0000 7f1259a15700 1 --2- 172.21.15.37:0/1911006433 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f12400390f0 0x7f124003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:54.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.524+0000 7f1259a15700 1 --2- 172.21.15.37:0/1911006433 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f12400390f0 0x7f124003b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1244006fd0 tx=0x7f1244006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:54.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.529+0000 7f124b7fe700 1 -- 172.21.15.37:0/1911006433 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1250013070 con 0x7f12541020e0 2024-06-10T00:45:54.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:54.740+0000 7f125c47a700 1 -- 172.21.15.37:0/1911006433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f1254075d60 con 0x7f12541020e0 2024-06-10T00:45:55.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:55.573+0000 7f124b7fe700 1 -- 172.21.15.37:0/1911006433 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f1250030090 con 0x7f12541020e0 2024-06-10T00:45:55.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:55.576+0000 7f125c47a700 1 -- 172.21.15.37:0/1911006433 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f12400390f0 msgr2=0x7f124003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:55.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:55.576+0000 7f125c47a700 1 --2- 172.21.15.37:0/1911006433 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f12400390f0 0x7f124003b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1244006fd0 tx=0x7f1244006e40 comp rx=0 tx=0).stop 2024-06-10T00:45:55.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:55.576+0000 7f125c47a700 1 -- 172.21.15.37:0/1911006433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12541020e0 msgr2=0x7f1254074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:55.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:55.576+0000 7f125c47a700 1 --2- 172.21.15.37:0/1911006433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12541020e0 0x7f1254074790 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1250005850 tx=0x7f1250005030 comp rx=0 tx=0).stop 2024-06-10T00:45:55.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:55.577+0000 7f125c47a700 1 -- 172.21.15.37:0/1911006433 shutdown_connections 2024-06-10T00:45:55.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:55.577+0000 7f125c47a700 1 --2- 172.21.15.37:0/1911006433 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f12400390f0 0x7f124003b5b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:55.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:55.577+0000 7f125c47a700 1 --2- 172.21.15.37:0/1911006433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12541020e0 0x7f1254074790 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:55.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:55.577+0000 7f125c47a700 1 -- 172.21.15.37:0/1911006433 >> 172.21.15.37:0/1911006433 conn(0x7f12540fd9d0 msgr2=0x7f1254103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:55.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:55.577+0000 7f125c47a700 1 -- 172.21.15.37:0/1911006433 shutdown_connections 2024-06-10T00:45:55.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:45:55.577+0000 7f125c47a700 1 -- 172.21.15.37:0/1911006433 wait complete. 2024-06-10T00:45:55.579 INFO:teuthology.orchestra.run.smithi037.stderr:adjusted tunables profile to default 2024-06-10T00:45:55.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:55 smithi037 ceph-mon[32192]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:45:55.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:55 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1911006433' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-10T00:45:56.195 INFO:tasks.cephadm:Adding mon.a on smithi037 2024-06-10T00:45:56.195 INFO:tasks.cephadm:Adding mon.b on smithi094 2024-06-10T00:45:56.195 INFO:tasks.cephadm:Adding mon.c on smithi167 2024-06-10T00:45:56.195 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph orch apply mon '3;smithi037:172.21.15.37=a;smithi094:172.21.15.94=b;smithi167:172.21.15.167=c' 2024-06-10T00:45:56.429 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:45:56.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:56 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1911006433' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-10T00:45:56.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:56 smithi037 ceph-mon[32192]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T00:45:56.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:56 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:56.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:56 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:56.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:56 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:56.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:56 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:56.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:56 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:45:56.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:56 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:45:56.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:56 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:45:56.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:56 smithi037 ceph-mon[32192]: Updating smithi167:/etc/ceph/ceph.conf 2024-06-10T00:45:56.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:56 smithi037 ceph-mon[32192]: Updating smithi167:/var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:45:57.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:57.913+0000 7f1dab6cd700 1 -- 172.21.15.167:0/280748473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da4102100 msgr2=0x7f1da41024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:57.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:57.913+0000 7f1dab6cd700 1 --2- 172.21.15.167:0/280748473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da4102100 0x7f1da41024e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009910 tx=0x7f1d94009c20 comp rx=0 tx=0).stop 2024-06-10T00:45:57.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:57.913+0000 7f1dab6cd700 1 -- 172.21.15.167:0/280748473 shutdown_connections 2024-06-10T00:45:57.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:57.913+0000 7f1dab6cd700 1 --2- 172.21.15.167:0/280748473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da4102100 0x7f1da41024e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:57.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:57.913+0000 7f1dab6cd700 1 -- 172.21.15.167:0/280748473 >> 172.21.15.167:0/280748473 conn(0x7f1da40fd9d0 msgr2=0x7f1da40ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:57.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:57.913+0000 7f1dab6cd700 1 -- 172.21.15.167:0/280748473 shutdown_connections 2024-06-10T00:45:57.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:57.913+0000 7f1dab6cd700 1 -- 172.21.15.167:0/280748473 wait complete. 2024-06-10T00:45:57.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:57.914+0000 7f1dab6cd700 1 Processor -- start 2024-06-10T00:45:57.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:57.914+0000 7f1dab6cd700 1 -- start start 2024-06-10T00:45:57.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:57.914+0000 7f1dab6cd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da4102100 0x7f1da4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:57.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:57.914+0000 7f1dab6cd700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1da410c4d0 con 0x7f1da4102100 2024-06-10T00:45:57.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:57.914+0000 7f1da9469700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da4102100 0x7f1da4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:57.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:57.914+0000 7f1da9469700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da4102100 0x7f1da4074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.167:43068/0 (socket says 172.21.15.167:43068) 2024-06-10T00:45:57.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:57.914+0000 7f1da9469700 1 -- 172.21.15.167:0/4001399099 learned_addr learned my addr 172.21.15.167:0/4001399099 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-10T00:45:57.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:57.914+0000 7f1da9469700 1 -- 172.21.15.167:0/4001399099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da4076680 con 0x7f1da4102100 2024-06-10T00:45:57.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:57.915+0000 7f1da9469700 1 --2- 172.21.15.167:0/4001399099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da4102100 0x7f1da4074790 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1d94005850 tx=0x7f1d94005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:57.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:57.915+0000 7f1d9a7fc700 1 -- 172.21.15.167:0/4001399099 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d94026070 con 0x7f1da4102100 2024-06-10T00:45:57.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:57.915+0000 7f1d9a7fc700 1 -- 172.21.15.167:0/4001399099 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d9400bc30 con 0x7f1da4102100 2024-06-10T00:45:57.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:57.915+0000 7f1dab6cd700 1 -- 172.21.15.167:0/4001399099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da4074d30 con 0x7f1da4102100 2024-06-10T00:45:57.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:57.915+0000 7f1d9a7fc700 1 -- 172.21.15.167:0/4001399099 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d94003a20 con 0x7f1da4102100 2024-06-10T00:45:57.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:57.915+0000 7f1dab6cd700 1 -- 172.21.15.167:0/4001399099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da40751f0 con 0x7f1da4102100 2024-06-10T00:45:57.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:57.916+0000 7f1dab6cd700 1 -- 172.21.15.167:0/4001399099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1da41035e0 con 0x7f1da4102100 2024-06-10T00:45:57.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:57.916+0000 7f1d9a7fc700 1 -- 172.21.15.167:0/4001399099 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1d94029070 con 0x7f1da4102100 2024-06-10T00:45:57.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:57.916+0000 7f1d9a7fc700 1 --2- 172.21.15.167:0/4001399099 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1d900390f0 0x7f1d9003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:45:57.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:57.916+0000 7f1d9a7fc700 1 -- 172.21.15.167:0/4001399099 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1d9401e070 con 0x7f1da4102100 2024-06-10T00:45:57.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:57.916+0000 7f1da8c68700 1 --2- 172.21.15.167:0/4001399099 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1d900390f0 0x7f1d9003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:45:57.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:57.917+0000 7f1da8c68700 1 --2- 172.21.15.167:0/4001399099 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1d900390f0 0x7f1d9003b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1da0006fd0 tx=0x7f1da0006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:45:57.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:57.921+0000 7f1d9a7fc700 1 -- 172.21.15.167:0/4001399099 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1d94013070 con 0x7f1da4102100 2024-06-10T00:45:58.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:58.133+0000 7f1dab6cd700 1 -- 172.21.15.167:0/4001399099 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi094:172.21.15.94=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}) -- 0x7f1da4075cc0 con 0x7f1d900390f0 2024-06-10T00:45:58.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:58.139+0000 7f1d9a7fc700 1 -- 172.21.15.167:0/4001399099 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1da4075cc0 con 0x7f1d900390f0 2024-06-10T00:45:58.140 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mon update... 2024-06-10T00:45:58.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:58.141+0000 7f1dab6cd700 1 -- 172.21.15.167:0/4001399099 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1d900390f0 msgr2=0x7f1d9003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:58.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:58.141+0000 7f1dab6cd700 1 --2- 172.21.15.167:0/4001399099 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1d900390f0 0x7f1d9003b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1da0006fd0 tx=0x7f1da0006e40 comp rx=0 tx=0).stop 2024-06-10T00:45:58.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:58.141+0000 7f1dab6cd700 1 -- 172.21.15.167:0/4001399099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da4102100 msgr2=0x7f1da4074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:45:58.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:58.141+0000 7f1dab6cd700 1 --2- 172.21.15.167:0/4001399099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da4102100 0x7f1da4074790 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1d94005850 tx=0x7f1d94005030 comp rx=0 tx=0).stop 2024-06-10T00:45:58.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:58.141+0000 7f1dab6cd700 1 -- 172.21.15.167:0/4001399099 shutdown_connections 2024-06-10T00:45:58.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:58.141+0000 7f1dab6cd700 1 --2- 172.21.15.167:0/4001399099 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1d900390f0 0x7f1d9003b5b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:58.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:58.141+0000 7f1dab6cd700 1 --2- 172.21.15.167:0/4001399099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da4102100 0x7f1da4074790 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:45:58.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:58.141+0000 7f1dab6cd700 1 -- 172.21.15.167:0/4001399099 >> 172.21.15.167:0/4001399099 conn(0x7f1da40fd9d0 msgr2=0x7f1da4104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:45:58.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:58.142+0000 7f1dab6cd700 1 -- 172.21.15.167:0/4001399099 shutdown_connections 2024-06-10T00:45:58.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:45:58.142+0000 7f1dab6cd700 1 -- 172.21.15.167:0/4001399099 wait complete. 2024-06-10T00:45:58.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:57 smithi037 ceph-mon[32192]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:45:58.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:57 smithi037 ceph-mon[32192]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:45:58.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:57 smithi037 ceph-mon[32192]: Updating smithi167:/var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-10T00:45:58.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:57 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:58.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:57 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:58.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:57 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:58.672 DEBUG:teuthology.orchestra.run.smithi094:mon.b> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mon.b.service 2024-06-10T00:45:58.675 DEBUG:teuthology.orchestra.run.smithi167:mon.c> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mon.c.service 2024-06-10T00:45:58.677 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-10T00:45:58.677 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-10T00:45:59.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:58 smithi037 ceph-mon[32192]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi094:172.21.15.94=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:45:59.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:58 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:59.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:58 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:45:59.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:58 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:45:59.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:58 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:45:59.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:58 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:45:59.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:58 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:45:59.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:58 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:45:59.828 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:46:00.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:59 smithi037 ceph-mon[32192]: Saving service mon spec with placement smithi037:172.21.15.37=a;smithi094:172.21.15.94=b;smithi167:172.21.15.167=c;count:3 2024-06-10T00:46:00.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:59 smithi037 ceph-mon[32192]: Deploying daemon mon.c on smithi167 2024-06-10T00:46:00.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:45:59 smithi037 ceph-mon[32192]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:02.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:01 smithi037 ceph-mon[32192]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:02.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:02.707+0000 7f8c6d93e700 1 -- 172.21.15.167:0/4236771411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c68106620 msgr2=0x7f8c68106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:46:02.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:02.707+0000 7f8c6d93e700 1 --2- 172.21.15.167:0/4236771411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c68106620 0x7f8c68106a00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8c5800aa20 tx=0x7f8c5800ad30 comp rx=0 tx=0).stop 2024-06-10T00:46:02.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:02.707+0000 7f8c6d93e700 1 -- 172.21.15.167:0/4236771411 shutdown_connections 2024-06-10T00:46:02.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:02.707+0000 7f8c6d93e700 1 --2- 172.21.15.167:0/4236771411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c68106620 0x7f8c68106a00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:02.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:02.707+0000 7f8c6d93e700 1 -- 172.21.15.167:0/4236771411 >> 172.21.15.167:0/4236771411 conn(0x7f8c68077510 msgr2=0x7f8c68077920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:46:02.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:02.707+0000 7f8c6d93e700 1 -- 172.21.15.167:0/4236771411 shutdown_connections 2024-06-10T00:46:02.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:02.707+0000 7f8c6d93e700 1 -- 172.21.15.167:0/4236771411 wait complete. 2024-06-10T00:46:02.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:02.708+0000 7f8c6d93e700 1 Processor -- start 2024-06-10T00:46:02.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:02.708+0000 7f8c6d93e700 1 -- start start 2024-06-10T00:46:02.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:02.708+0000 7f8c6d93e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c68106620 0x7f8c6819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:46:02.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:02.708+0000 7f8c6d93e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6810a370 con 0x7f8c68106620 2024-06-10T00:46:02.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:02.708+0000 7f8c66ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c68106620 0x7f8c6819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:46:02.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:02.708+0000 7f8c66ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c68106620 0x7f8c6819b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.167:33626/0 (socket says 172.21.15.167:33626) 2024-06-10T00:46:02.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:02.708+0000 7f8c66ffd700 1 -- 172.21.15.167:0/3883819918 learned_addr learned my addr 172.21.15.167:0/3883819918 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-10T00:46:02.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:02.709+0000 7f8c66ffd700 1 -- 172.21.15.167:0/3883819918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c6819b920 con 0x7f8c68106620 2024-06-10T00:46:02.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:02.709+0000 7f8c66ffd700 1 --2- 172.21.15.167:0/3883819918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c68106620 0x7f8c6819b3e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8c58004c90 tx=0x7f8c58004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:46:02.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:02.709+0000 7f8c6c93c700 1 -- 172.21.15.167:0/3883819918 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c58026070 con 0x7f8c68106620 2024-06-10T00:46:02.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:02.709+0000 7f8c6d93e700 1 -- 172.21.15.167:0/3883819918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c6819bbb0 con 0x7f8c68106620 2024-06-10T00:46:02.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:02.709+0000 7f8c6c93c700 1 -- 172.21.15.167:0/3883819918 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c58016470 con 0x7f8c68106620 2024-06-10T00:46:02.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:02.709+0000 7f8c6c93c700 1 -- 172.21.15.167:0/3883819918 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c5800e5c0 con 0x7f8c68106620 2024-06-10T00:46:02.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:02.709+0000 7f8c6d93e700 1 -- 172.21.15.167:0/3883819918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c681955a0 con 0x7f8c68106620 2024-06-10T00:46:02.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:02.710+0000 7f8c6c93c700 1 -- 172.21.15.167:0/3883819918 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f8c58029070 con 0x7f8c68106620 2024-06-10T00:46:02.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:02.710+0000 7f8c6d93e700 1 -- 172.21.15.167:0/3883819918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c68101c90 con 0x7f8c68106620 2024-06-10T00:46:02.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:02.710+0000 7f8c6c93c700 1 --2- 172.21.15.167:0/3883819918 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8c500390f0 0x7f8c5003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:46:02.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:02.710+0000 7f8c6c93c700 1 -- 172.21.15.167:0/3883819918 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8c5804e2c0 con 0x7f8c68106620 2024-06-10T00:46:02.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:02.711+0000 7f8c667fc700 1 --2- 172.21.15.167:0/3883819918 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8c500390f0 0x7f8c5003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:46:02.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:02.711+0000 7f8c667fc700 1 --2- 172.21.15.167:0/3883819918 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8c500390f0 0x7f8c5003b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c006fd0 tx=0x7f8c5c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:46:02.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:02.716+0000 7f8c6c93c700 1 -- 172.21.15.167:0/3883819918 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8c5800e970 con 0x7f8c68106620 2024-06-10T00:46:03.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:03.009+0000 7f8c6d93e700 1 -- 172.21.15.167:0/3883819918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8c68196ad0 con 0x7f8c68106620 2024-06-10T00:46:03.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:03.010+0000 7f8c6c93c700 1 -- 172.21.15.167:0/3883819918 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f8c58030020 con 0x7f8c68106620 2024-06-10T00:46:03.010 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-10T00:46:03.011 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":1,"fsid":"68793514-26c2-11ef-bc9b-c7b262605968","modified":"2024-06-10T00:44:21.142496Z","created":"2024-06-10T00:44:21.142496Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-10T00:46:03.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:03.012+0000 7f8c6d93e700 1 -- 172.21.15.167:0/3883819918 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8c500390f0 msgr2=0x7f8c5003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:46:03.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:03.012+0000 7f8c6d93e700 1 --2- 172.21.15.167:0/3883819918 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8c500390f0 0x7f8c5003b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c006fd0 tx=0x7f8c5c006e40 comp rx=0 tx=0).stop 2024-06-10T00:46:03.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:03.012+0000 7f8c6d93e700 1 -- 172.21.15.167:0/3883819918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c68106620 msgr2=0x7f8c6819b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:46:03.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:03.012+0000 7f8c6d93e700 1 --2- 172.21.15.167:0/3883819918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c68106620 0x7f8c6819b3e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8c58004c90 tx=0x7f8c58004d70 comp rx=0 tx=0).stop 2024-06-10T00:46:03.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:03.012+0000 7f8c6d93e700 1 -- 172.21.15.167:0/3883819918 shutdown_connections 2024-06-10T00:46:03.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:03.012+0000 7f8c6d93e700 1 --2- 172.21.15.167:0/3883819918 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8c500390f0 0x7f8c5003b5b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:03.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:03.012+0000 7f8c6d93e700 1 --2- 172.21.15.167:0/3883819918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c68106620 0x7f8c6819b3e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:03.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:03.012+0000 7f8c6d93e700 1 -- 172.21.15.167:0/3883819918 >> 172.21.15.167:0/3883819918 conn(0x7f8c68077510 msgr2=0x7f8c680fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:46:03.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:03.013+0000 7f8c6d93e700 1 -- 172.21.15.167:0/3883819918 shutdown_connections 2024-06-10T00:46:03.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:03.013+0000 7f8c6d93e700 1 -- 172.21.15.167:0/3883819918 wait complete. 2024-06-10T00:46:03.014 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 1 2024-06-10T00:46:03.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:03 smithi167 systemd[1]: Started Ceph mon.c for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T00:46:04.768 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-10T00:46:04.768 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-10T00:46:05.012 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:46:08.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:08.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:46:08.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:46:08.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: mon.a calling monitor election 2024-06-10T00:46:08.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:46:08.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:08.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:46:08.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: mon.c calling monitor election 2024-06-10T00:46:08.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:46:08.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:08.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:46:08.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:46:08.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-10T00:46:08.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: monmap epoch 2 2024-06-10T00:46:08.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: fsid 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:46:08.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: last_changed 2024-06-10T00:46:03.026783+0000 2024-06-10T00:46:08.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: created 2024-06-10T00:44:21.142496+0000 2024-06-10T00:46:08.398 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: min_mon_release 19 (squid) 2024-06-10T00:46:08.398 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: election_strategy: 1 2024-06-10T00:46:08.398 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-06-10T00:46:08.398 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-06-10T00:46:08.398 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: fsmap 2024-06-10T00:46:08.398 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T00:46:08.398 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: mgrmap e13: x(active, since 47s) 2024-06-10T00:46:08.398 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: overall HEALTH_OK 2024-06-10T00:46:08.398 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:08.398 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:08.398 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:08.398 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:46:08.398 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:08 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:08.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:08.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:46:08.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:46:08.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: mon.a calling monitor election 2024-06-10T00:46:08.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:46:08.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:08.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:46:08.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: mon.c calling monitor election 2024-06-10T00:46:08.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:46:08.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:08.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:46:08.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:46:08.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-10T00:46:08.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: monmap epoch 2 2024-06-10T00:46:08.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: fsid 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:46:08.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: last_changed 2024-06-10T00:46:03.026783+0000 2024-06-10T00:46:08.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: created 2024-06-10T00:44:21.142496+0000 2024-06-10T00:46:08.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: min_mon_release 19 (squid) 2024-06-10T00:46:08.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: election_strategy: 1 2024-06-10T00:46:08.591 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-06-10T00:46:08.591 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-06-10T00:46:08.591 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: fsmap 2024-06-10T00:46:08.591 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T00:46:08.591 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: mgrmap e13: x(active, since 47s) 2024-06-10T00:46:08.591 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: overall HEALTH_OK 2024-06-10T00:46:08.591 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:08.591 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:08.592 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:08.592 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:46:08.593 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:08 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:09.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:09 smithi037 ceph-mon[32192]: Deploying daemon mon.b on smithi094 2024-06-10T00:46:09.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:09 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:46:09.439 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:09 smithi167 ceph-mon[36917]: Deploying daemon mon.b on smithi094 2024-06-10T00:46:09.439 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:09 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:46:09.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.438+0000 7f7fd06a9700 1 -- 172.21.15.167:0/2325040264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fb40035c0 msgr2=0x7f7fb4005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:46:09.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.438+0000 7f7fd06a9700 1 --2- 172.21.15.167:0/2325040264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fb40035c0 0x7f7fb4005a50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8069660 tx=0x7f7fc000fe60 comp rx=0 tx=0).stop 2024-06-10T00:46:09.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.438+0000 7f7fd06a9700 1 -- 172.21.15.167:0/2325040264 shutdown_connections 2024-06-10T00:46:09.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.438+0000 7f7fd06a9700 1 --2- 172.21.15.167:0/2325040264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fb40035c0 0x7f7fb4005a50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:09.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.438+0000 7f7fd06a9700 1 -- 172.21.15.167:0/2325040264 >> 172.21.15.167:0/2325040264 conn(0x7f7fc8077310 msgr2=0x7f7fc8077720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:46:09.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.438+0000 7f7fd06a9700 1 -- 172.21.15.167:0/2325040264 shutdown_connections 2024-06-10T00:46:09.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.438+0000 7f7fd06a9700 1 -- 172.21.15.167:0/2325040264 wait complete. 2024-06-10T00:46:09.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.439+0000 7f7fd06a9700 1 Processor -- start 2024-06-10T00:46:09.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.439+0000 7f7fd06a9700 1 -- start start 2024-06-10T00:46:09.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.440+0000 7f7fd06a9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fb40035c0 0x7f7fc8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:46:09.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.440+0000 7f7fd06a9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fc8198e70 0x7f7fc81992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:46:09.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.440+0000 7f7fd06a9700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc8108f50 con 0x7f7fc8198e70 2024-06-10T00:46:09.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.440+0000 7f7fd06a9700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc8108dd0 con 0x7f7fb40035c0 2024-06-10T00:46:09.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.440+0000 7f7fce445700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fb40035c0 0x7f7fc8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:46:09.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.440+0000 7f7fce445700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fb40035c0 0x7f7fc8198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:46780/0 (socket says 172.21.15.167:46780) 2024-06-10T00:46:09.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.440+0000 7f7fce445700 1 -- 172.21.15.167:0/3139761296 learned_addr learned my addr 172.21.15.167:0/3139761296 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-10T00:46:09.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.440+0000 7f7fce445700 1 -- 172.21.15.167:0/3139761296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fb40035c0 msgr2=0x7f7fc8198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-06-10T00:46:09.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.440+0000 7f7fce445700 1 -- 172.21.15.167:0/3139761296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fb40035c0 msgr2=0x7f7fc8198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-10T00:46:09.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.440+0000 7f7fce445700 1 --2- 172.21.15.167:0/3139761296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fb40035c0 0x7f7fc8198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-10T00:46:09.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.440+0000 7f7fcdc44700 1 --2- 172.21.15.167:0/3139761296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fc8198e70 0x7f7fc81992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:46:09.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.440+0000 7f7fce445700 1 --2- 172.21.15.167:0/3139761296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fb40035c0 0x7f7fc8198930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T00:46:09.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.440+0000 7f7fcdc44700 1 -- 172.21.15.167:0/3139761296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fb40035c0 msgr2=0x7f7fc8198930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T00:46:09.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.440+0000 7f7fcdc44700 1 --2- 172.21.15.167:0/3139761296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fb40035c0 0x7f7fc8198930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:09.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.440+0000 7f7fcdc44700 1 -- 172.21.15.167:0/3139761296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc81a19a0 con 0x7f7fc8198e70 2024-06-10T00:46:09.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.441+0000 7f7fcdc44700 1 --2- 172.21.15.167:0/3139761296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fc8198e70 0x7f7fc81992f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7fb800b4d0 tx=0x7f7fb800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:46:09.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.441+0000 7f7fbf7fe700 1 -- 172.21.15.167:0/3139761296 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f7fb8011630 con 0x7f7fc8198e70 2024-06-10T00:46:09.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.441+0000 7f7fbf7fe700 1 -- 172.21.15.167:0/3139761296 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fb8011cb0 con 0x7f7fc8198e70 2024-06-10T00:46:09.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.441+0000 7f7fd06a9700 1 -- 172.21.15.167:0/3139761296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc81a1bd0 con 0x7f7fc8198e70 2024-06-10T00:46:09.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.441+0000 7f7fbf7fe700 1 -- 172.21.15.167:0/3139761296 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f7fb8010e50 con 0x7f7fc8198e70 2024-06-10T00:46:09.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.441+0000 7f7fd06a9700 1 -- 172.21.15.167:0/3139761296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc81a2160 con 0x7f7fc8198e70 2024-06-10T00:46:09.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.442+0000 7f7fbf7fe700 1 -- 172.21.15.167:0/3139761296 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7fb8018020 con 0x7f7fc8198e70 2024-06-10T00:46:09.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.442+0000 7f7fbf7fe700 1 --2- 172.21.15.167:0/3139761296 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7fac038fd0 0x7f7fac03b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:46:09.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.442+0000 7f7fbf7fe700 1 -- 172.21.15.167:0/3139761296 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7fb8010510 con 0x7f7fc8198e70 2024-06-10T00:46:09.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.442+0000 7f7fd06a9700 1 -- 172.21.15.167:0/3139761296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fc8102780 con 0x7f7fc8198e70 2024-06-10T00:46:09.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.442+0000 7f7fce445700 1 --2- 172.21.15.167:0/3139761296 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7fac038fd0 0x7f7fac03b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:46:09.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.443+0000 7f7fce445700 1 --2- 172.21.15.167:0/3139761296 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7fac038fd0 0x7f7fac03b490 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0001eb0 tx=0x7f7fc0005e90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:46:09.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.448+0000 7f7fbf7fe700 1 -- 172.21.15.167:0/3139761296 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7fb801f020 con 0x7f7fc8198e70 2024-06-10T00:46:09.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.728+0000 7f7fd06a9700 1 -- 172.21.15.167:0/3139761296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f7fc802dca0 con 0x7f7fc8198e70 2024-06-10T00:46:09.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.728+0000 7f7fbf7fe700 1 -- 172.21.15.167:0/3139761296 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f7fb8018310 con 0x7f7fc8198e70 2024-06-10T00:46:09.729 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-10T00:46:09.729 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":2,"fsid":"68793514-26c2-11ef-bc9b-c7b262605968","modified":"2024-06-10T00:46:03.026783Z","created":"2024-06-10T00:44:21.142496Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-10T00:46:09.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.731+0000 7f7fd06a9700 1 -- 172.21.15.167:0/3139761296 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7fac038fd0 msgr2=0x7f7fac03b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:46:09.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.731+0000 7f7fd06a9700 1 --2- 172.21.15.167:0/3139761296 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7fac038fd0 0x7f7fac03b490 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0001eb0 tx=0x7f7fc0005e90 comp rx=0 tx=0).stop 2024-06-10T00:46:09.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.731+0000 7f7fd06a9700 1 -- 172.21.15.167:0/3139761296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fc8198e70 msgr2=0x7f7fc81992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:46:09.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.731+0000 7f7fd06a9700 1 --2- 172.21.15.167:0/3139761296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fc8198e70 0x7f7fc81992f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7fb800b4d0 tx=0x7f7fb800b890 comp rx=0 tx=0).stop 2024-06-10T00:46:09.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.731+0000 7f7fd06a9700 1 -- 172.21.15.167:0/3139761296 shutdown_connections 2024-06-10T00:46:09.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.731+0000 7f7fd06a9700 1 --2- 172.21.15.167:0/3139761296 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7fac038fd0 0x7f7fac03b490 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:09.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.731+0000 7f7fd06a9700 1 --2- 172.21.15.167:0/3139761296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fb40035c0 0x7f7fc8198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:09.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.731+0000 7f7fd06a9700 1 --2- 172.21.15.167:0/3139761296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fc8198e70 0x7f7fc81992f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:09.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.731+0000 7f7fd06a9700 1 -- 172.21.15.167:0/3139761296 >> 172.21.15.167:0/3139761296 conn(0x7f7fc8077310 msgr2=0x7f7fc80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:46:09.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.731+0000 7f7fd06a9700 1 -- 172.21.15.167:0/3139761296 shutdown_connections 2024-06-10T00:46:09.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:09.731+0000 7f7fd06a9700 1 -- 172.21.15.167:0/3139761296 wait complete. 2024-06-10T00:46:09.733 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 2 2024-06-10T00:46:10.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:10 smithi167 ceph-mon[36917]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:10.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:10 smithi167 ceph-mon[36917]: from='client.? 172.21.15.167:0/3139761296' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-10T00:46:10.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:10 smithi037 ceph-mon[32192]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:10.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:10 smithi037 ceph-mon[32192]: from='client.? 172.21.15.167:0/3139761296' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-10T00:46:11.309 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-10T00:46:11.310 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-10T00:46:11.539 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:46:11.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 podman[36689]: 2024-06-10 00:46:11.68031364 +0000 UTC m=+0.017009550 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:46:12.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 podman[36689]: 2024-06-10 00:46:11.953225977 +0000 UTC m=+0.289921858 container init 17bea145dd2ed513c1ed172bbd06da593f2a3f5d0fe606e50a2fb19989eabc7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-b, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True) 2024-06-10T00:46:12.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 podman[36689]: 2024-06-10 00:46:11.956793433 +0000 UTC m=+0.293489325 container start 17bea145dd2ed513c1ed172bbd06da593f2a3f5d0fe606e50a2fb19989eabc7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, ceph=True) 2024-06-10T00:46:12.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: set uid:gid to 167:167 (ceph:ceph) 2024-06-10T00:46:12.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-10T00:46:12.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: pidfile_write: ignore empty --pid-file 2024-06-10T00:46:12.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: load: jerasure load: lrc 2024-06-10T00:46:12.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: RocksDB version: 7.9.2 2024-06-10T00:46:12.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Git sha 0 2024-06-10T00:46:12.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-10T00:46:12.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: DB SUMMARY 2024-06-10T00:46:12.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: DB Session ID: WT1JTDDOURQ5SQE2G6WO 2024-06-10T00:46:12.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: CURRENT file: CURRENT 2024-06-10T00:46:12.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: IDENTITY file: IDENTITY 2024-06-10T00:46:12.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-06-10T00:46:12.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-06-10T00:46:12.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-06-10T00:46:12.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.error_if_exists: 0 2024-06-10T00:46:12.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.create_if_missing: 0 2024-06-10T00:46:12.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.paranoid_checks: 1 2024-06-10T00:46:12.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-10T00:46:12.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-10T00:46:12.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-10T00:46:12.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.env: 0x562bfd8f1c60 2024-06-10T00:46:12.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.fs: PosixFileSystem 2024-06-10T00:46:12.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.info_log: 0x562bfec1bb20 2024-06-10T00:46:12.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.max_file_opening_threads: 16 2024-06-10T00:46:12.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.statistics: (nil) 2024-06-10T00:46:12.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.use_fsync: 0 2024-06-10T00:46:12.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.max_log_file_size: 0 2024-06-10T00:46:12.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-10T00:46:12.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-10T00:46:12.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.keep_log_file_num: 1000 2024-06-10T00:46:12.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.recycle_log_file_num: 0 2024-06-10T00:46:12.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.allow_fallocate: 1 2024-06-10T00:46:12.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.allow_mmap_reads: 0 2024-06-10T00:46:12.368 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.allow_mmap_writes: 0 2024-06-10T00:46:12.368 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.use_direct_reads: 0 2024-06-10T00:46:12.368 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-10T00:46:12.368 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.create_missing_column_families: 0 2024-06-10T00:46:12.368 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.db_log_dir: 2024-06-10T00:46:12.368 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.wal_dir: 2024-06-10T00:46:12.368 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-10T00:46:12.368 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-10T00:46:12.368 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-10T00:46:12.368 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-10T00:46:12.368 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-10T00:46:12.368 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-10T00:46:12.368 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.advise_random_on_open: 1 2024-06-10T00:46:12.368 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.db_write_buffer_size: 0 2024-06-10T00:46:12.369 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.write_buffer_manager: 0x562bfebec1e0 2024-06-10T00:46:12.369 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-10T00:46:12.369 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-10T00:46:12.369 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-10T00:46:12.369 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.rate_limiter: (nil) 2024-06-10T00:46:12.369 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-10T00:46:12.369 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.wal_recovery_mode: 2 2024-06-10T00:46:12.369 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.enable_thread_tracking: 0 2024-06-10T00:46:12.369 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.enable_pipelined_write: 0 2024-06-10T00:46:12.369 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.unordered_write: 0 2024-06-10T00:46:12.369 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-10T00:46:12.369 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-10T00:46:12.369 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-10T00:46:12.369 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-10T00:46:12.369 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.row_cache: None 2024-06-10T00:46:12.370 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.wal_filter: None 2024-06-10T00:46:12.371 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-10T00:46:12.371 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.allow_ingest_behind: 0 2024-06-10T00:46:12.371 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.two_write_queues: 0 2024-06-10T00:46:12.371 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.manual_wal_flush: 0 2024-06-10T00:46:12.371 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.wal_compression: 0 2024-06-10T00:46:12.371 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.atomic_flush: 0 2024-06-10T00:46:12.371 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-10T00:46:12.372 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-10T00:46:12.372 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-10T00:46:12.372 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.log_readahead_size: 0 2024-06-10T00:46:12.372 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-10T00:46:12.372 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.best_efforts_recovery: 0 2024-06-10T00:46:12.372 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-10T00:46:12.372 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-10T00:46:12.372 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.allow_data_in_errors: 0 2024-06-10T00:46:12.372 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.db_host_id: __hostname__ 2024-06-10T00:46:12.372 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-10T00:46:12.372 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.max_background_jobs: 2 2024-06-10T00:46:12.372 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.max_background_compactions: -1 2024-06-10T00:46:12.372 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.max_subcompactions: 1 2024-06-10T00:46:12.372 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-10T00:46:12.373 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-10T00:46:12.373 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-10T00:46:12.373 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.max_total_wal_size: 0 2024-06-10T00:46:12.373 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-10T00:46:12.373 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-10T00:46:12.373 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-10T00:46:12.373 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-10T00:46:12.373 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.max_open_files: -1 2024-06-10T00:46:12.373 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.bytes_per_sync: 0 2024-06-10T00:46:12.373 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-10T00:46:12.373 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-10T00:46:12.373 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.compaction_readahead_size: 0 2024-06-10T00:46:12.373 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.max_background_flushes: -1 2024-06-10T00:46:12.373 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Compression algorithms supported: 2024-06-10T00:46:12.373 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-10T00:46:12.374 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: kZSTD supported: 0 2024-06-10T00:46:12.374 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: kXpressCompression supported: 0 2024-06-10T00:46:12.374 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: kLZ4HCCompression supported: 1 2024-06-10T00:46:12.374 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: kZlibCompression supported: 1 2024-06-10T00:46:12.374 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: kSnappyCompression supported: 1 2024-06-10T00:46:12.374 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: kLZ4Compression supported: 1 2024-06-10T00:46:12.374 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: kBZip2Compression supported: 0 2024-06-10T00:46:12.374 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-10T00:46:12.374 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-10T00:46:12.374 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-06-10T00:46:12.374 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-10T00:46:12.374 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-10T00:46:12.374 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.merge_operator: 2024-06-10T00:46:12.375 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.compaction_filter: None 2024-06-10T00:46:12.375 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.compaction_filter_factory: None 2024-06-10T00:46:12.375 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.sst_partitioner_factory: None 2024-06-10T00:46:12.375 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-10T00:46:12.375 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-10T00:46:12.375 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562bfec1bc40) 2024-06-10T00:46:12.375 INFO:journalctl@ceph.mon.b.smithi094.stdout: cache_index_and_filter_blocks: 1 2024-06-10T00:46:12.375 INFO:journalctl@ceph.mon.b.smithi094.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-10T00:46:12.375 INFO:journalctl@ceph.mon.b.smithi094.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-10T00:46:12.375 INFO:journalctl@ceph.mon.b.smithi094.stdout: pin_top_level_index_and_filter: 1 2024-06-10T00:46:12.375 INFO:journalctl@ceph.mon.b.smithi094.stdout: index_type: 0 2024-06-10T00:46:12.375 INFO:journalctl@ceph.mon.b.smithi094.stdout: data_block_index_type: 0 2024-06-10T00:46:12.375 INFO:journalctl@ceph.mon.b.smithi094.stdout: index_shortening: 1 2024-06-10T00:46:12.375 INFO:journalctl@ceph.mon.b.smithi094.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-10T00:46:12.375 INFO:journalctl@ceph.mon.b.smithi094.stdout: checksum: 4 2024-06-10T00:46:12.376 INFO:journalctl@ceph.mon.b.smithi094.stdout: no_block_cache: 0 2024-06-10T00:46:12.376 INFO:journalctl@ceph.mon.b.smithi094.stdout: block_cache: 0x562bfec1d090 2024-06-10T00:46:12.376 INFO:journalctl@ceph.mon.b.smithi094.stdout: block_cache_name: BinnedLRUCache 2024-06-10T00:46:12.376 INFO:journalctl@ceph.mon.b.smithi094.stdout: block_cache_options: 2024-06-10T00:46:12.376 INFO:journalctl@ceph.mon.b.smithi094.stdout: capacity : 536870912 2024-06-10T00:46:12.376 INFO:journalctl@ceph.mon.b.smithi094.stdout: num_shard_bits : 4 2024-06-10T00:46:12.376 INFO:journalctl@ceph.mon.b.smithi094.stdout: strict_capacity_limit : 0 2024-06-10T00:46:12.376 INFO:journalctl@ceph.mon.b.smithi094.stdout: high_pri_pool_ratio: 0.000 2024-06-10T00:46:12.376 INFO:journalctl@ceph.mon.b.smithi094.stdout: block_cache_compressed: (nil) 2024-06-10T00:46:12.376 INFO:journalctl@ceph.mon.b.smithi094.stdout: persistent_cache: (nil) 2024-06-10T00:46:12.376 INFO:journalctl@ceph.mon.b.smithi094.stdout: block_size: 4096 2024-06-10T00:46:12.376 INFO:journalctl@ceph.mon.b.smithi094.stdout: block_size_deviation: 10 2024-06-10T00:46:12.376 INFO:journalctl@ceph.mon.b.smithi094.stdout: block_restart_interval: 16 2024-06-10T00:46:12.376 INFO:journalctl@ceph.mon.b.smithi094.stdout: index_block_restart_interval: 1 2024-06-10T00:46:12.376 INFO:journalctl@ceph.mon.b.smithi094.stdout: metadata_block_size: 4096 2024-06-10T00:46:12.377 INFO:journalctl@ceph.mon.b.smithi094.stdout: partition_filters: 0 2024-06-10T00:46:12.377 INFO:journalctl@ceph.mon.b.smithi094.stdout: use_delta_encoding: 1 2024-06-10T00:46:12.377 INFO:journalctl@ceph.mon.b.smithi094.stdout: filter_policy: bloomfilter 2024-06-10T00:46:12.377 INFO:journalctl@ceph.mon.b.smithi094.stdout: whole_key_filtering: 1 2024-06-10T00:46:12.377 INFO:journalctl@ceph.mon.b.smithi094.stdout: verify_compression: 0 2024-06-10T00:46:12.377 INFO:journalctl@ceph.mon.b.smithi094.stdout: read_amp_bytes_per_bit: 0 2024-06-10T00:46:12.377 INFO:journalctl@ceph.mon.b.smithi094.stdout: format_version: 5 2024-06-10T00:46:12.377 INFO:journalctl@ceph.mon.b.smithi094.stdout: enable_index_compression: 1 2024-06-10T00:46:12.377 INFO:journalctl@ceph.mon.b.smithi094.stdout: block_align: 0 2024-06-10T00:46:12.377 INFO:journalctl@ceph.mon.b.smithi094.stdout: max_auto_readahead_size: 262144 2024-06-10T00:46:12.377 INFO:journalctl@ceph.mon.b.smithi094.stdout: prepopulate_block_cache: 0 2024-06-10T00:46:12.377 INFO:journalctl@ceph.mon.b.smithi094.stdout: initial_auto_readahead_size: 8192 2024-06-10T00:46:12.377 INFO:journalctl@ceph.mon.b.smithi094.stdout: num_file_reads_for_auto_readahead: 2 2024-06-10T00:46:12.377 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.write_buffer_size: 33554432 2024-06-10T00:46:12.377 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.max_write_buffer_number: 2 2024-06-10T00:46:12.378 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.compression: NoCompression 2024-06-10T00:46:12.378 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.bottommost_compression: Disabled 2024-06-10T00:46:12.378 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.prefix_extractor: nullptr 2024-06-10T00:46:12.378 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-10T00:46:12.378 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.num_levels: 7 2024-06-10T00:46:12.378 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-10T00:46:12.378 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-10T00:46:12.378 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-10T00:46:12.378 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-10T00:46:12.378 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-10T00:46:12.378 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-10T00:46:12.378 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-10T00:46:12.378 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-10T00:46:12.378 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-10T00:46:12.379 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-10T00:46:12.379 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-10T00:46:12.379 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-10T00:46:12.379 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-10T00:46:12.379 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.compression_opts.level: 32767 2024-06-10T00:46:12.379 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.compression_opts.strategy: 0 2024-06-10T00:46:12.379 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-10T00:46:12.379 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-10T00:46:12.379 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-10T00:46:12.379 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-10T00:46:12.379 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.compression_opts.enabled: false 2024-06-10T00:46:12.379 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-10T00:46:12.379 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-10T00:46:12.379 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-10T00:46:12.379 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-10T00:46:12.380 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.target_file_size_base: 67108864 2024-06-10T00:46:12.380 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-10T00:46:12.380 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-10T00:46:12.380 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-10T00:46:12.380 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-10T00:46:12.380 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-10T00:46:12.380 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-10T00:46:12.380 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-10T00:46:12.380 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-10T00:46:12.380 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-10T00:46:12.380 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-10T00:46:12.380 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-10T00:46:12.380 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-10T00:46:12.380 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-10T00:46:12.381 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-10T00:46:12.381 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.arena_block_size: 1048576 2024-06-10T00:46:12.381 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-10T00:46:12.381 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-10T00:46:12.381 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.disable_auto_compactions: 0 2024-06-10T00:46:12.381 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-10T00:46:12.381 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-10T00:46:12.381 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-10T00:46:12.381 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-10T00:46:12.381 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-10T00:46:12.381 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-10T00:46:12.381 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-10T00:46:12.381 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-10T00:46:12.381 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-10T00:46:12.381 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-10T00:46:12.381 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.table_properties_collectors: 2024-06-10T00:46:12.382 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.inplace_update_support: 0 2024-06-10T00:46:12.382 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-10T00:46:12.382 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-10T00:46:12.382 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-10T00:46:12.382 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-10T00:46:12.382 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.bloom_locality: 0 2024-06-10T00:46:12.382 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.max_successive_merges: 0 2024-06-10T00:46:12.382 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-10T00:46:12.382 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.paranoid_file_checks: 0 2024-06-10T00:46:12.382 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.force_consistency_checks: 1 2024-06-10T00:46:12.382 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.report_bg_io_stats: 0 2024-06-10T00:46:12.382 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.ttl: 2592000 2024-06-10T00:46:12.382 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-10T00:46:12.382 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-10T00:46:12.383 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-10T00:46:12.383 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.enable_blob_files: false 2024-06-10T00:46:12.383 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.min_blob_size: 0 2024-06-10T00:46:12.383 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.blob_file_size: 268435456 2024-06-10T00:46:12.383 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-10T00:46:12.383 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-10T00:46:12.383 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-10T00:46:12.383 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-10T00:46:12.383 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-10T00:46:12.383 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.blob_file_starting_level: 0 2024-06-10T00:46:12.383 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-10T00:46:12.383 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-06-10T00:46:12.383 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-06-10T00:46:12.383 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7208863b-9f4b-44d9-83e1-6a5411bb7e9a 2024-06-10T00:46:12.383 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717980371997641, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-06-10T00:46:12.384 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-06-10T00:46:12.384 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717980371998061, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717980371, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7208863b-9f4b-44d9-83e1-6a5411bb7e9a", "db_session_id": "WT1JTDDOURQ5SQE2G6WO", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-06-10T00:46:12.384 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717980371998142, "job": 1, "event": "recovery_finished"} 2024-06-10T00:46:12.384 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-06-10T00:46:12.384 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-10T00:46:12.384 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562bfed0c000 2024-06-10T00:46:12.384 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: DB pointer 0x562bfecf6000 2024-06-10T00:46:12.384 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-10T00:46:12.384 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-10T00:46:12.384 INFO:journalctl@ceph.mon.b.smithi094.stdout: ** DB Stats ** 2024-06-10T00:46:12.384 INFO:journalctl@ceph.mon.b.smithi094.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-10T00:46:12.384 INFO:journalctl@ceph.mon.b.smithi094.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-10T00:46:12.384 INFO:journalctl@ceph.mon.b.smithi094.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-10T00:46:12.384 INFO:journalctl@ceph.mon.b.smithi094.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-10T00:46:12.384 INFO:journalctl@ceph.mon.b.smithi094.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-10T00:46:12.385 INFO:journalctl@ceph.mon.b.smithi094.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-10T00:46:12.385 INFO:journalctl@ceph.mon.b.smithi094.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-10T00:46:12.385 INFO:journalctl@ceph.mon.b.smithi094.stdout: 2024-06-10T00:46:12.385 INFO:journalctl@ceph.mon.b.smithi094.stdout: ** Compaction Stats [default] ** 2024-06-10T00:46:12.385 INFO:journalctl@ceph.mon.b.smithi094.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-10T00:46:12.385 INFO:journalctl@ceph.mon.b.smithi094.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-10T00:46:12.385 INFO:journalctl@ceph.mon.b.smithi094.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-10T00:46:12.385 INFO:journalctl@ceph.mon.b.smithi094.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-10T00:46:12.385 INFO:journalctl@ceph.mon.b.smithi094.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-10T00:46:12.385 INFO:journalctl@ceph.mon.b.smithi094.stdout: 2024-06-10T00:46:12.385 INFO:journalctl@ceph.mon.b.smithi094.stdout: ** Compaction Stats [default] ** 2024-06-10T00:46:12.385 INFO:journalctl@ceph.mon.b.smithi094.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-10T00:46:12.385 INFO:journalctl@ceph.mon.b.smithi094.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-10T00:46:12.385 INFO:journalctl@ceph.mon.b.smithi094.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-10T00:46:12.385 INFO:journalctl@ceph.mon.b.smithi094.stdout: 2024-06-10T00:46:12.386 INFO:journalctl@ceph.mon.b.smithi094.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-10T00:46:12.386 INFO:journalctl@ceph.mon.b.smithi094.stdout: 2024-06-10T00:46:12.386 INFO:journalctl@ceph.mon.b.smithi094.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-10T00:46:12.386 INFO:journalctl@ceph.mon.b.smithi094.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-10T00:46:12.386 INFO:journalctl@ceph.mon.b.smithi094.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-10T00:46:12.386 INFO:journalctl@ceph.mon.b.smithi094.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-10T00:46:12.386 INFO:journalctl@ceph.mon.b.smithi094.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-10T00:46:12.386 INFO:journalctl@ceph.mon.b.smithi094.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-10T00:46:12.386 INFO:journalctl@ceph.mon.b.smithi094.stdout: Cumulative compaction: 0.00 GB write, 0.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-10T00:46:12.386 INFO:journalctl@ceph.mon.b.smithi094.stdout: Interval compaction: 0.00 GB write, 0.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-10T00:46:12.386 INFO:journalctl@ceph.mon.b.smithi094.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-10T00:46:12.386 INFO:journalctl@ceph.mon.b.smithi094.stdout: Block cache BinnedLRUCache@0x562bfec1d090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-06-10T00:46:12.386 INFO:journalctl@ceph.mon.b.smithi094.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-06-10T00:46:12.386 INFO:journalctl@ceph.mon.b.smithi094.stdout: 2024-06-10T00:46:12.387 INFO:journalctl@ceph.mon.b.smithi094.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-10T00:46:12.387 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-06-10T00:46:12.387 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:11 smithi094 ceph-mon[36708]: using public_addr v2:172.21.15.94:0/0 -> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] 2024-06-10T00:46:12.387 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: starting mon.b rank -1 at public addrs [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] at bind addrs [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:46:12.387 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: mon.b@-1(???) e0 preinit fsid 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:46:12.387 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: mon.b@-1(synchronizing).mds e1 new map 2024-06-10T00:46:12.387 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: mon.b@-1(synchronizing).mds e1 print_map 2024-06-10T00:46:12.387 INFO:journalctl@ceph.mon.b.smithi094.stdout: e1 2024-06-10T00:46:12.387 INFO:journalctl@ceph.mon.b.smithi094.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-10T00:46:12.387 INFO:journalctl@ceph.mon.b.smithi094.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-10T00:46:12.387 INFO:journalctl@ceph.mon.b.smithi094.stdout: legacy client fscid: -1 2024-06-10T00:46:12.387 INFO:journalctl@ceph.mon.b.smithi094.stdout: 2024-06-10T00:46:12.387 INFO:journalctl@ceph.mon.b.smithi094.stdout: No filesystems configured 2024-06-10T00:46:12.387 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-06-10T00:46:12.387 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-06-10T00:46:12.387 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-06-10T00:46:12.388 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-06-10T00:46:12.388 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-06-10T00:46:12.388 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-06-10T00:46:12.388 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-10T00:46:12.388 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-10T00:46:12.388 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-10T00:46:12.388 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-10T00:46:12.388 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:12.388 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: Deploying cephadm binary to smithi167 2024-06-10T00:46:12.388 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: Updating smithi094:/etc/ceph/ceph.conf 2024-06-10T00:46:12.388 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: Updating smithi094:/var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:46:12.388 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:12.388 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:12.388 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:12.389 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:12.389 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:12.389 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:46:12.389 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: Updating smithi094:/var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-10T00:46:12.389 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: Added host smithi167 2024-06-10T00:46:12.389 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:46:12.389 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:12.389 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:12.389 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:12.389 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:12.389 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:46:12.389 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:12.389 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:12.389 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1911006433' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-10T00:46:12.389 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1911006433' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-10T00:46:12.390 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T00:46:12.390 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:12.390 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:12.390 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:12.390 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:12.390 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:46:12.390 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:12.390 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:46:12.390 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: Updating smithi167:/etc/ceph/ceph.conf 2024-06-10T00:46:12.390 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: Updating smithi167:/var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:46:12.390 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:12.390 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:46:12.390 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: Updating smithi167:/var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-10T00:46:12.391 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:12.391 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:12.391 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:12.391 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi094:172.21.15.94=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:46:12.391 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:12.391 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:46:12.391 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:12.391 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:46:12.391 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:12.391 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:46:12.391 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:12.391 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: Saving service mon spec with placement smithi037:172.21.15.37=a;smithi094:172.21.15.94=b;smithi167:172.21.15.167=c;count:3 2024-06-10T00:46:12.391 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: Deploying daemon mon.c on smithi167 2024-06-10T00:46:12.391 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:12.391 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:12.392 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:12.392 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:46:12.392 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:46:12.392 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: mon.a calling monitor election 2024-06-10T00:46:12.392 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:46:12.392 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:12.392 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:46:12.392 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: mon.c calling monitor election 2024-06-10T00:46:12.392 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:46:12.392 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:12.392 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:46:12.392 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:46:12.392 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-10T00:46:12.392 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: monmap epoch 2 2024-06-10T00:46:12.392 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: fsid 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:46:12.392 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: last_changed 2024-06-10T00:46:03.026783+0000 2024-06-10T00:46:12.393 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: created 2024-06-10T00:44:21.142496+0000 2024-06-10T00:46:12.393 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: min_mon_release 19 (squid) 2024-06-10T00:46:12.393 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: election_strategy: 1 2024-06-10T00:46:12.393 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-06-10T00:46:12.393 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-06-10T00:46:12.393 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: fsmap 2024-06-10T00:46:12.393 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T00:46:12.393 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: mgrmap e13: x(active, since 47s) 2024-06-10T00:46:12.393 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: overall HEALTH_OK 2024-06-10T00:46:12.393 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:12.393 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:12.393 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:12.393 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:46:12.393 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:12.393 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: Deploying daemon mon.b on smithi094 2024-06-10T00:46:12.393 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:46:12.394 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:12.394 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: from='client.? 172.21.15.167:0/3139761296' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-10T00:46:12.394 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 ceph-mon[36708]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-06-10T00:46:12.394 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 bash[36689]: 17bea145dd2ed513c1ed172bbd06da593f2a3f5d0fe606e50a2fb19989eabc7c 2024-06-10T00:46:12.394 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:12 smithi094 systemd[1]: Started Ceph mon.b for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T00:46:17.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.321+0000 7f0b0c161700 1 -- 172.21.15.167:0/2480740941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0af00035c0 msgr2=0x7f0af0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:46:17.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.321+0000 7f0b0c161700 1 --2- 172.21.15.167:0/2480740941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0af00035c0 0x7f0af0005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0b04100db0 tx=0x7f0af400fe60 comp rx=0 tx=0).stop 2024-06-10T00:46:17.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.321+0000 7f0b0c161700 1 -- 172.21.15.167:0/2480740941 shutdown_connections 2024-06-10T00:46:17.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.321+0000 7f0b0c161700 1 --2- 172.21.15.167:0/2480740941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0af00035c0 0x7f0af0005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:17.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.321+0000 7f0b0c161700 1 -- 172.21.15.167:0/2480740941 >> 172.21.15.167:0/2480740941 conn(0x7f0b040fc970 msgr2=0x7f0b040fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:46:17.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.321+0000 7f0b0c161700 1 -- 172.21.15.167:0/2480740941 shutdown_connections 2024-06-10T00:46:17.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.321+0000 7f0b0c161700 1 -- 172.21.15.167:0/2480740941 wait complete. 2024-06-10T00:46:17.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.322+0000 7f0b0c161700 1 Processor -- start 2024-06-10T00:46:17.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.322+0000 7f0b0c161700 1 -- start start 2024-06-10T00:46:17.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.322+0000 7f0b0c161700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af00035c0 0x7f0b04198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:46:17.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.322+0000 7f0b0c161700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b04198fd0 0x7f0b04199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:46:17.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.322+0000 7f0b0c161700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b041a1b00 0x7f0b041a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:46:17.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.322+0000 7f0b0c161700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0b04108ec0 con 0x7f0b04198fd0 2024-06-10T00:46:17.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.322+0000 7f0b0c161700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0b04108d40 con 0x7f0af00035c0 2024-06-10T00:46:17.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.322+0000 7f0b0c161700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0b04109040 con 0x7f0b041a1b00 2024-06-10T00:46:17.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.322+0000 7f0b0a6fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b041a1b00 0x7f0b041a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:46:17.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.322+0000 7f0b096fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b04198fd0 0x7f0b04199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:46:17.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.322+0000 7f0b0a6fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b041a1b00 0x7f0b041a3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55768/0 (socket says 172.21.15.167:55768) 2024-06-10T00:46:17.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.322+0000 7f0b0a6fe700 1 -- 172.21.15.167:0/4037970447 learned_addr learned my addr 172.21.15.167:0/4037970447 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-10T00:46:17.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.322+0000 7f0b09efd700 1 --2- 172.21.15.167:0/4037970447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af00035c0 0x7f0b04198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:46:17.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.322+0000 7f0b0a6fe700 1 -- 172.21.15.167:0/4037970447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b041a1b00 msgr2=0x7f0b041a3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-06-10T00:46:17.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.322+0000 7f0b0a6fe700 1 -- 172.21.15.167:0/4037970447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b041a1b00 msgr2=0x7f0b041a3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-10T00:46:17.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.322+0000 7f0b0a6fe700 1 --2- 172.21.15.167:0/4037970447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b041a1b00 0x7f0b041a3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-10T00:46:17.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.322+0000 7f0b0a6fe700 1 --2- 172.21.15.167:0/4037970447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b041a1b00 0x7f0b041a3ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T00:46:17.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.323+0000 7f0b09efd700 1 -- 172.21.15.167:0/4037970447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af00035c0 msgr2=0x7f0b04198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-06-10T00:46:17.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.323+0000 7f0b09efd700 1 -- 172.21.15.167:0/4037970447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af00035c0 msgr2=0x7f0b04198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-10T00:46:17.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.323+0000 7f0b09efd700 1 --2- 172.21.15.167:0/4037970447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af00035c0 0x7f0b04198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-10T00:46:17.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.323+0000 7f0b096fc700 1 -- 172.21.15.167:0/4037970447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b041a1b00 msgr2=0x7f0b041a3ef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T00:46:17.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.323+0000 7f0b096fc700 1 --2- 172.21.15.167:0/4037970447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b041a1b00 0x7f0b041a3ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:17.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.323+0000 7f0b09efd700 1 --2- 172.21.15.167:0/4037970447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af00035c0 0x7f0b04198a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-10T00:46:17.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.323+0000 7f0b096fc700 1 -- 172.21.15.167:0/4037970447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af00035c0 msgr2=0x7f0b04198a90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T00:46:17.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.323+0000 7f0b096fc700 1 --2- 172.21.15.167:0/4037970447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af00035c0 0x7f0b04198a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:17.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.323+0000 7f0b096fc700 1 -- 172.21.15.167:0/4037970447 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b041a4700 con 0x7f0b04198fd0 2024-06-10T00:46:17.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.323+0000 7f0b096fc700 1 --2- 172.21.15.167:0/4037970447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b04198fd0 0x7f0b04199450 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0b0000b550 tx=0x7f0b0000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:46:17.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.323+0000 7f0afaffd700 1 -- 172.21.15.167:0/4037970447 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00004230 con 0x7f0b04198fd0 2024-06-10T00:46:17.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.323+0000 7f0afaffd700 1 -- 172.21.15.167:0/4037970447 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b000043d0 con 0x7f0b04198fd0 2024-06-10T00:46:17.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.323+0000 7f0b0c161700 1 -- 172.21.15.167:0/4037970447 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b041a49f0 con 0x7f0b04198fd0 2024-06-10T00:46:17.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.323+0000 7f0afaffd700 1 -- 172.21.15.167:0/4037970447 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00011c20 con 0x7f0b04198fd0 2024-06-10T00:46:17.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.323+0000 7f0b0c161700 1 -- 172.21.15.167:0/4037970447 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b041a4f80 con 0x7f0b04198fd0 2024-06-10T00:46:17.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.324+0000 7f0afaffd700 1 -- 172.21.15.167:0/4037970447 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0b0001a400 con 0x7f0b04198fd0 2024-06-10T00:46:17.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.324+0000 7f0afaffd700 1 --2- 172.21.15.167:0/4037970447 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0ae8039580 0x7f0ae803ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:46:17.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.324+0000 7f0b0c161700 1 -- 172.21.15.167:0/4037970447 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b04063af0 con 0x7f0b04198fd0 2024-06-10T00:46:17.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.325+0000 7f0afaffd700 1 -- 172.21.15.167:0/4037970447 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0b0004bdd0 con 0x7f0b04198fd0 2024-06-10T00:46:17.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.325+0000 7f0b09efd700 1 --2- 172.21.15.167:0/4037970447 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0ae8039580 0x7f0ae803ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:46:17.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.325+0000 7f0b09efd700 1 --2- 172.21.15.167:0/4037970447 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0ae8039580 0x7f0ae803ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0af4020040 tx=0x7f0af4001e70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:46:17.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.330+0000 7f0afaffd700 1 -- 172.21.15.167:0/4037970447 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0b000151c0 con 0x7f0b04198fd0 2024-06-10T00:46:17.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:46:17.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: mon.c calling monitor election 2024-06-10T00:46:17.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: mon.a calling monitor election 2024-06-10T00:46:17.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:46:17.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:46:17.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:17.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:46:17.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:46:17.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:17.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:46:17.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:46:17.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:46:17.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-10T00:46:17.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: monmap epoch 3 2024-06-10T00:46:17.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: fsid 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:46:17.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: last_changed 2024-06-10T00:46:12.020610+0000 2024-06-10T00:46:17.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: created 2024-06-10T00:44:21.142496+0000 2024-06-10T00:46:17.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: min_mon_release 19 (squid) 2024-06-10T00:46:17.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: election_strategy: 1 2024-06-10T00:46:17.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-06-10T00:46:17.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-06-10T00:46:17.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: 2: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.b 2024-06-10T00:46:17.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: fsmap 2024-06-10T00:46:17.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T00:46:17.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: mgrmap e13: x(active, since 56s) 2024-06-10T00:46:17.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: overall HEALTH_OK 2024-06-10T00:46:17.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:17.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:17.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:17.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:17 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:17.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:46:17.470 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: mon.c calling monitor election 2024-06-10T00:46:17.470 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: mon.a calling monitor election 2024-06-10T00:46:17.470 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:46:17.471 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:46:17.471 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:17.471 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:46:17.471 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:46:17.471 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:17.471 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:46:17.471 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:46:17.471 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:46:17.471 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-10T00:46:17.471 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: monmap epoch 3 2024-06-10T00:46:17.471 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: fsid 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:46:17.471 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: last_changed 2024-06-10T00:46:12.020610+0000 2024-06-10T00:46:17.471 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: created 2024-06-10T00:44:21.142496+0000 2024-06-10T00:46:17.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: min_mon_release 19 (squid) 2024-06-10T00:46:17.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: election_strategy: 1 2024-06-10T00:46:17.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-06-10T00:46:17.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-06-10T00:46:17.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: 2: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.b 2024-06-10T00:46:17.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: fsmap 2024-06-10T00:46:17.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T00:46:17.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: mgrmap e13: x(active, since 56s) 2024-06-10T00:46:17.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: overall HEALTH_OK 2024-06-10T00:46:17.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:17.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:17.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:17.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:17 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:17.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.622+0000 7f0b0c161700 1 -- 172.21.15.167:0/4037970447 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0b041a5310 con 0x7f0b04198fd0 2024-06-10T00:46:17.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.623+0000 7f0afaffd700 1 -- 172.21.15.167:0/4037970447 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f0b00003c80 con 0x7f0b04198fd0 2024-06-10T00:46:17.623 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-10T00:46:17.624 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":3,"fsid":"68793514-26c2-11ef-bc9b-c7b262605968","modified":"2024-06-10T00:46:12.020610Z","created":"2024-06-10T00:44:21.142496Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-10T00:46:17.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.625+0000 7f0b0c161700 1 -- 172.21.15.167:0/4037970447 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0ae8039580 msgr2=0x7f0ae803ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:46:17.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.625+0000 7f0b0c161700 1 --2- 172.21.15.167:0/4037970447 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0ae8039580 0x7f0ae803ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0af4020040 tx=0x7f0af4001e70 comp rx=0 tx=0).stop 2024-06-10T00:46:17.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.625+0000 7f0b0c161700 1 -- 172.21.15.167:0/4037970447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b04198fd0 msgr2=0x7f0b04199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:46:17.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.625+0000 7f0b0c161700 1 --2- 172.21.15.167:0/4037970447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b04198fd0 0x7f0b04199450 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0b0000b550 tx=0x7f0b0000b910 comp rx=0 tx=0).stop 2024-06-10T00:46:17.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.625+0000 7f0b0c161700 1 -- 172.21.15.167:0/4037970447 shutdown_connections 2024-06-10T00:46:17.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.625+0000 7f0b0c161700 1 --2- 172.21.15.167:0/4037970447 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0ae8039580 0x7f0ae803ba40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:17.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.625+0000 7f0b0c161700 1 --2- 172.21.15.167:0/4037970447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b041a1b00 0x7f0b041a3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:17.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.625+0000 7f0b0c161700 1 --2- 172.21.15.167:0/4037970447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af00035c0 0x7f0b04198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:17.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.625+0000 7f0b0c161700 1 --2- 172.21.15.167:0/4037970447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b04198fd0 0x7f0b04199450 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:17.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.625+0000 7f0b0c161700 1 -- 172.21.15.167:0/4037970447 >> 172.21.15.167:0/4037970447 conn(0x7f0b040fc970 msgr2=0x7f0b040fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:46:17.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.625+0000 7f0b0c161700 1 -- 172.21.15.167:0/4037970447 shutdown_connections 2024-06-10T00:46:17.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:17.625+0000 7f0b0c161700 1 -- 172.21.15.167:0/4037970447 wait complete. 2024-06-10T00:46:17.627 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 3 2024-06-10T00:46:18.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:18 smithi167 ceph-mon[36917]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:18.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:18 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:46:18.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:18 smithi167 ceph-mon[36917]: from='client.? 172.21.15.167:0/4037970447' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-10T00:46:18.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:18 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:46:18.253 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-06-10T00:46:18.253 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph config generate-minimal-conf 2024-06-10T00:46:18.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:18 smithi037 ceph-mon[32192]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:18.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:18 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:46:18.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:18 smithi037 ceph-mon[32192]: from='client.? 172.21.15.167:0/4037970447' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-10T00:46:18.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:18 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:46:18.511 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:46:19.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:46:19.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: mon.c calling monitor election 2024-06-10T00:46:19.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: mon.a calling monitor election 2024-06-10T00:46:19.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:46:19.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:46:19.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:19.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:46:19.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:46:19.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:19.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:46:19.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:46:19.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:46:19.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-10T00:46:19.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: monmap epoch 3 2024-06-10T00:46:19.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: fsid 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:46:19.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: last_changed 2024-06-10T00:46:12.020610+0000 2024-06-10T00:46:19.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: created 2024-06-10T00:44:21.142496+0000 2024-06-10T00:46:19.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: min_mon_release 19 (squid) 2024-06-10T00:46:19.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: election_strategy: 1 2024-06-10T00:46:19.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-06-10T00:46:19.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-06-10T00:46:19.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: 2: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.b 2024-06-10T00:46:19.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: fsmap 2024-06-10T00:46:19.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T00:46:19.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: mgrmap e13: x(active, since 56s) 2024-06-10T00:46:19.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: overall HEALTH_OK 2024-06-10T00:46:19.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:19.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:19.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:19.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:19.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:19.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:46:19.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='client.? 172.21.15.167:0/4037970447' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-10T00:46:19.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:46:20.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.020+0000 7fef9c25b700 1 -- 172.21.15.37:0/2467568616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef941020d0 msgr2=0x7fef941024b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:46:20.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.020+0000 7fef9c25b700 1 --2- 172.21.15.37:0/2467568616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef941020d0 0x7fef941024b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fef90009910 tx=0x7fef90009c20 comp rx=0 tx=0).stop 2024-06-10T00:46:20.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.021+0000 7fef9c25b700 1 -- 172.21.15.37:0/2467568616 shutdown_connections 2024-06-10T00:46:20.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.021+0000 7fef9c25b700 1 --2- 172.21.15.37:0/2467568616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef941020d0 0x7fef941024b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:20.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.021+0000 7fef9c25b700 1 -- 172.21.15.37:0/2467568616 >> 172.21.15.37:0/2467568616 conn(0x7fef940fd9a0 msgr2=0x7fef940ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:46:20.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.021+0000 7fef9c25b700 1 -- 172.21.15.37:0/2467568616 shutdown_connections 2024-06-10T00:46:20.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.021+0000 7fef9c25b700 1 -- 172.21.15.37:0/2467568616 wait complete. 2024-06-10T00:46:20.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.021+0000 7fef9c25b700 1 Processor -- start 2024-06-10T00:46:20.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.021+0000 7fef9c25b700 1 -- start start 2024-06-10T00:46:20.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.022+0000 7fef9c25b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef941020d0 0x7fef94074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:46:20.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.022+0000 7fef9c25b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef94074cd0 0x7fef94075150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:46:20.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.022+0000 7fef9c25b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef941a3e00 0x7fef941a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:46:20.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.022+0000 7fef9c25b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fef9410c460 con 0x7fef94074cd0 2024-06-10T00:46:20.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.022+0000 7fef9c25b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fef9410c2e0 con 0x7fef941a3e00 2024-06-10T00:46:20.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.022+0000 7fef9c25b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fef9410c5e0 con 0x7fef941020d0 2024-06-10T00:46:20.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.022+0000 7fef997f6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef94074cd0 0x7fef94075150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:46:20.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.022+0000 7fef997f6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef94074cd0 0x7fef94075150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49856/0 (socket says 172.21.15.37:49856) 2024-06-10T00:46:20.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.022+0000 7fef997f6700 1 -- 172.21.15.37:0/1711442235 learned_addr learned my addr 172.21.15.37:0/1711442235 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:46:20.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.022+0000 7fef99ff7700 1 --2- 172.21.15.37:0/1711442235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef941020d0 0x7fef94074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:46:20.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.022+0000 7fef9a7f8700 1 --2- 172.21.15.37:0/1711442235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef941a3e00 0x7fef941a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:46:20.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.022+0000 7fef997f6700 1 -- 172.21.15.37:0/1711442235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef941020d0 msgr2=0x7fef94074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:46:20.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.022+0000 7fef997f6700 1 --2- 172.21.15.37:0/1711442235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef941020d0 0x7fef94074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:20.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.022+0000 7fef997f6700 1 -- 172.21.15.37:0/1711442235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef941a3e00 msgr2=0x7fef941a61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:46:20.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.022+0000 7fef997f6700 1 --2- 172.21.15.37:0/1711442235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef941a3e00 0x7fef941a61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:20.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.022+0000 7fef997f6700 1 -- 172.21.15.37:0/1711442235 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef941a6920 con 0x7fef94074cd0 2024-06-10T00:46:20.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.022+0000 7fef99ff7700 1 --2- 172.21.15.37:0/1711442235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef941020d0 0x7fef94074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:46:20.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.022+0000 7fef997f6700 1 --2- 172.21.15.37:0/1711442235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef94074cd0 0x7fef94075150 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fef8400b780 tx=0x7fef8400bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:46:20.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.022+0000 7fef8affd700 1 -- 172.21.15.37:0/1711442235 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef84004230 con 0x7fef94074cd0 2024-06-10T00:46:20.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.022+0000 7fef9c25b700 1 -- 172.21.15.37:0/1711442235 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef941a6c10 con 0x7fef94074cd0 2024-06-10T00:46:20.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.022+0000 7fef8affd700 1 -- 172.21.15.37:0/1711442235 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef840043d0 con 0x7fef94074cd0 2024-06-10T00:46:20.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.022+0000 7fef8affd700 1 -- 172.21.15.37:0/1711442235 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef84011c60 con 0x7fef94074cd0 2024-06-10T00:46:20.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.023+0000 7fef9c25b700 1 -- 172.21.15.37:0/1711442235 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef941a71a0 con 0x7fef94074cd0 2024-06-10T00:46:20.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.023+0000 7fef8affd700 1 -- 172.21.15.37:0/1711442235 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fef8401a400 con 0x7fef94074cd0 2024-06-10T00:46:20.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.024+0000 7fef8affd700 1 --2- 172.21.15.37:0/1711442235 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fef80039580 0x7fef8003ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:46:20.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.024+0000 7fef8affd700 1 -- 172.21.15.37:0/1711442235 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fef8404cdd0 con 0x7fef94074cd0 2024-06-10T00:46:20.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.024+0000 7fef9c25b700 1 -- 172.21.15.37:0/1711442235 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef94063af0 con 0x7fef94074cd0 2024-06-10T00:46:20.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.024+0000 7fef99ff7700 1 --2- 172.21.15.37:0/1711442235 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fef80039580 0x7fef8003ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:46:20.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.024+0000 7fef99ff7700 1 --2- 172.21.15.37:0/1711442235 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fef80039580 0x7fef8003ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fef90009750 tx=0x7fef9000b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:46:20.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.029+0000 7fef8affd700 1 -- 172.21.15.37:0/1711442235 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fef84003c80 con 0x7fef94074cd0 2024-06-10T00:46:20.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: mon.b calling monitor election 2024-06-10T00:46:20.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:20.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: Updating smithi037:/etc/ceph/ceph.conf 2024-06-10T00:46:20.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: Updating smithi094:/etc/ceph/ceph.conf 2024-06-10T00:46:20.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: Updating smithi167:/etc/ceph/ceph.conf 2024-06-10T00:46:20.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: mon.b calling monitor election 2024-06-10T00:46:20.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: mon.c calling monitor election 2024-06-10T00:46:20.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: mon.a calling monitor election 2024-06-10T00:46:20.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-10T00:46:20.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: monmap epoch 3 2024-06-10T00:46:20.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: fsid 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:46:20.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: last_changed 2024-06-10T00:46:12.020610+0000 2024-06-10T00:46:20.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: created 2024-06-10T00:44:21.142496+0000 2024-06-10T00:46:20.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: min_mon_release 19 (squid) 2024-06-10T00:46:20.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: election_strategy: 1 2024-06-10T00:46:20.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-06-10T00:46:20.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-06-10T00:46:20.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: 2: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.b 2024-06-10T00:46:20.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: fsmap 2024-06-10T00:46:20.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T00:46:20.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: mgrmap e13: x(active, since 58s) 2024-06-10T00:46:20.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: overall HEALTH_OK 2024-06-10T00:46:20.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.149 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.149 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.149 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:46:20.149 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T00:46:20.149 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:19 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:20.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.265+0000 7fef9c25b700 1 -- 172.21.15.37:0/1711442235 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fef941a73d0 con 0x7fef94074cd0 2024-06-10T00:46:20.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.265+0000 7fef8affd700 1 -- 172.21.15.37:0/1711442235 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7fef8401a740 con 0x7fef94074cd0 2024-06-10T00:46:20.266 INFO:teuthology.orchestra.run.smithi037.stdout:# minimal ceph.conf for 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:46:20.266 INFO:teuthology.orchestra.run.smithi037.stdout:[global] 2024-06-10T00:46:20.266 INFO:teuthology.orchestra.run.smithi037.stdout: fsid = 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:46:20.266 INFO:teuthology.orchestra.run.smithi037.stdout: mon_host = [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] 2024-06-10T00:46:20.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.268+0000 7fef9c25b700 1 -- 172.21.15.37:0/1711442235 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fef80039580 msgr2=0x7fef8003ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:46:20.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.268+0000 7fef9c25b700 1 --2- 172.21.15.37:0/1711442235 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fef80039580 0x7fef8003ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fef90009750 tx=0x7fef9000b540 comp rx=0 tx=0).stop 2024-06-10T00:46:20.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.268+0000 7fef9c25b700 1 -- 172.21.15.37:0/1711442235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef94074cd0 msgr2=0x7fef94075150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:46:20.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.268+0000 7fef9c25b700 1 --2- 172.21.15.37:0/1711442235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef94074cd0 0x7fef94075150 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fef8400b780 tx=0x7fef8400bb40 comp rx=0 tx=0).stop 2024-06-10T00:46:20.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.268+0000 7fef9c25b700 1 -- 172.21.15.37:0/1711442235 shutdown_connections 2024-06-10T00:46:20.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.268+0000 7fef9c25b700 1 --2- 172.21.15.37:0/1711442235 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fef80039580 0x7fef8003ba40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:20.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.268+0000 7fef9c25b700 1 --2- 172.21.15.37:0/1711442235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef941a3e00 0x7fef941a61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:20.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.268+0000 7fef9c25b700 1 --2- 172.21.15.37:0/1711442235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef941020d0 0x7fef94074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:20.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.268+0000 7fef9c25b700 1 --2- 172.21.15.37:0/1711442235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef94074cd0 0x7fef94075150 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:20.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.268+0000 7fef9c25b700 1 -- 172.21.15.37:0/1711442235 >> 172.21.15.37:0/1711442235 conn(0x7fef940fd9a0 msgr2=0x7fef94103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:46:20.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.268+0000 7fef9c25b700 1 -- 172.21.15.37:0/1711442235 shutdown_connections 2024-06-10T00:46:20.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:20.268+0000 7fef9c25b700 1 -- 172.21.15.37:0/1711442235 wait complete. 2024-06-10T00:46:20.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: mon.b calling monitor election 2024-06-10T00:46:20.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:20.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: Updating smithi037:/etc/ceph/ceph.conf 2024-06-10T00:46:20.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: Updating smithi094:/etc/ceph/ceph.conf 2024-06-10T00:46:20.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: Updating smithi167:/etc/ceph/ceph.conf 2024-06-10T00:46:20.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: mon.b calling monitor election 2024-06-10T00:46:20.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: mon.c calling monitor election 2024-06-10T00:46:20.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: mon.a calling monitor election 2024-06-10T00:46:20.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-10T00:46:20.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: monmap epoch 3 2024-06-10T00:46:20.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: fsid 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:46:20.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: last_changed 2024-06-10T00:46:12.020610+0000 2024-06-10T00:46:20.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: created 2024-06-10T00:44:21.142496+0000 2024-06-10T00:46:20.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: min_mon_release 19 (squid) 2024-06-10T00:46:20.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: election_strategy: 1 2024-06-10T00:46:20.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-06-10T00:46:20.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-06-10T00:46:20.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: 2: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.b 2024-06-10T00:46:20.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: fsmap 2024-06-10T00:46:20.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T00:46:20.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: mgrmap e13: x(active, since 58s) 2024-06-10T00:46:20.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: overall HEALTH_OK 2024-06-10T00:46:20.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:46:20.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T00:46:20.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:19 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:20.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: mon.b calling monitor election 2024-06-10T00:46:20.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:20.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: Updating smithi037:/etc/ceph/ceph.conf 2024-06-10T00:46:20.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: Updating smithi094:/etc/ceph/ceph.conf 2024-06-10T00:46:20.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: Updating smithi167:/etc/ceph/ceph.conf 2024-06-10T00:46:20.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: mon.b calling monitor election 2024-06-10T00:46:20.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: mon.c calling monitor election 2024-06-10T00:46:20.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: mon.a calling monitor election 2024-06-10T00:46:20.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-10T00:46:20.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: monmap epoch 3 2024-06-10T00:46:20.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: fsid 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:46:20.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: last_changed 2024-06-10T00:46:12.020610+0000 2024-06-10T00:46:20.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: created 2024-06-10T00:44:21.142496+0000 2024-06-10T00:46:20.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: min_mon_release 19 (squid) 2024-06-10T00:46:20.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: election_strategy: 1 2024-06-10T00:46:20.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-06-10T00:46:20.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-06-10T00:46:20.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: 2: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.b 2024-06-10T00:46:20.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: fsmap 2024-06-10T00:46:20.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: osdmap e4: 0 total, 0 up, 0 in 2024-06-10T00:46:20.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: mgrmap e13: x(active, since 58s) 2024-06-10T00:46:20.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: overall HEALTH_OK 2024-06-10T00:46:20.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:20.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:46:20.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T00:46:20.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:21.302 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:20 smithi037 ceph-mon[32192]: Updating smithi167:/var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:46:21.303 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:20 smithi037 ceph-mon[32192]: Updating smithi037:/var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:46:21.303 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:20 smithi037 ceph-mon[32192]: Updating smithi094:/var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:46:21.303 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:20 smithi037 ceph-mon[32192]: Reconfiguring mon.a (unknown last config time)... 2024-06-10T00:46:21.303 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:20 smithi037 ceph-mon[32192]: Reconfiguring daemon mon.a on smithi037 2024-06-10T00:46:21.303 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:20 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:46:21.303 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:20 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1711442235' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:21.303 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:20 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:21.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:20 smithi167 ceph-mon[36917]: Updating smithi167:/var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:46:21.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:20 smithi167 ceph-mon[36917]: Updating smithi037:/var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:46:21.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:20 smithi167 ceph-mon[36917]: Updating smithi094:/var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:46:21.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:20 smithi167 ceph-mon[36917]: Reconfiguring mon.a (unknown last config time)... 2024-06-10T00:46:21.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:20 smithi167 ceph-mon[36917]: Reconfiguring daemon mon.a on smithi037 2024-06-10T00:46:21.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:20 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:46:21.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:20 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1711442235' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:21.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:20 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:21.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:20 smithi094 ceph-mon[36708]: Updating smithi167:/var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:46:21.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:20 smithi094 ceph-mon[36708]: Updating smithi037:/var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:46:21.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:20 smithi094 ceph-mon[36708]: Updating smithi094:/var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-10T00:46:21.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:20 smithi094 ceph-mon[36708]: Reconfiguring mon.a (unknown last config time)... 2024-06-10T00:46:21.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:20 smithi094 ceph-mon[36708]: Reconfiguring daemon mon.a on smithi037 2024-06-10T00:46:21.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:20 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:46:21.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:20 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1711442235' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:21.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:20 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:21.733 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-06-10T00:46:21.734 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-06-10T00:46:21.734 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.conf 2024-06-10T00:46:21.763 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-06-10T00:46:21.763 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:46:21.829 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-10T00:46:21.829 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.conf 2024-06-10T00:46:21.856 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-10T00:46:21.856 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:46:21.920 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-10T00:46:21.920 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.conf 2024-06-10T00:46:21.952 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-10T00:46:21.952 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-10T00:46:22.020 INFO:tasks.cephadm:Adding mgr.x on smithi037 2024-06-10T00:46:22.020 INFO:tasks.cephadm:Adding mgr.y on smithi094 2024-06-10T00:46:22.020 INFO:tasks.cephadm:Adding mgr.z on smithi167 2024-06-10T00:46:22.020 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph orch apply mgr '3;smithi037=x;smithi094=y;smithi167=z' 2024-06-10T00:46:22.259 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:21 smithi094 ceph-mon[36708]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:22.260 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:22.260 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:22.260 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:46:22.260 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T00:46:22.260 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:22.294 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:46:22.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:21 smithi167 ceph-mon[36917]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:22.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:22.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:22.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:46:22.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T00:46:22.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:22.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:21 smithi037 ceph-mon[32192]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:22.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:22.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:22.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:46:22.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T00:46:22.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:23.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:23 smithi167 ceph-mon[36917]: Reconfiguring mon.b (monmap changed)... 2024-06-10T00:46:23.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:23 smithi167 ceph-mon[36917]: Reconfiguring daemon mon.b on smithi094 2024-06-10T00:46:23.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:23 smithi094 ceph-mon[36708]: Reconfiguring mon.b (monmap changed)... 2024-06-10T00:46:23.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:23 smithi094 ceph-mon[36708]: Reconfiguring daemon mon.b on smithi094 2024-06-10T00:46:23.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:23 smithi037 ceph-mon[32192]: Reconfiguring mon.b (monmap changed)... 2024-06-10T00:46:23.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:23 smithi037 ceph-mon[32192]: Reconfiguring daemon mon.b on smithi094 2024-06-10T00:46:23.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.698+0000 7fe5f6c27700 1 -- 172.21.15.167:0/2733112707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f0100f30 msgr2=0x7fe5f0101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:46:23.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.698+0000 7fe5f6c27700 1 --2- 172.21.15.167:0/2733112707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f0100f30 0x7fe5f0101350 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4009910 tx=0x7fe5e4009c20 comp rx=0 tx=0).stop 2024-06-10T00:46:23.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.698+0000 7fe5f6c27700 1 -- 172.21.15.167:0/2733112707 shutdown_connections 2024-06-10T00:46:23.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.698+0000 7fe5f6c27700 1 --2- 172.21.15.167:0/2733112707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f0100f30 0x7fe5f0101350 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:23.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.698+0000 7fe5f6c27700 1 -- 172.21.15.167:0/2733112707 >> 172.21.15.167:0/2733112707 conn(0x7fe5f00fc4d0 msgr2=0x7fe5f00fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:46:23.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.698+0000 7fe5f6c27700 1 -- 172.21.15.167:0/2733112707 shutdown_connections 2024-06-10T00:46:23.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.698+0000 7fe5f6c27700 1 -- 172.21.15.167:0/2733112707 wait complete. 2024-06-10T00:46:23.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.698+0000 7fe5f6c27700 1 Processor -- start 2024-06-10T00:46:23.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.699+0000 7fe5f6c27700 1 -- start start 2024-06-10T00:46:23.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.699+0000 7fe5f6c27700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f0100f30 0x7fe5f006b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:46:23.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.699+0000 7fe5f6c27700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5f019a5c0 0x7fe5f019c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:46:23.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.699+0000 7fe5f6c27700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5f019cf90 0x7fe5f019f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:46:23.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.699+0000 7fe5f6c27700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f0106ef0 con 0x7fe5f0100f30 2024-06-10T00:46:23.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.699+0000 7fe5f6c27700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f0106d70 con 0x7fe5f019a5c0 2024-06-10T00:46:23.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.699+0000 7fe5f6c27700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f0107070 con 0x7fe5f019cf90 2024-06-10T00:46:23.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.699+0000 7fe5f51c4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5f019cf90 0x7fe5f019f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:46:23.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.699+0000 7fe5f51c4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5f019cf90 0x7fe5f019f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:46784/0 (socket says 172.21.15.167:46784) 2024-06-10T00:46:23.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.699+0000 7fe5f51c4700 1 -- 172.21.15.167:0/2746281963 learned_addr learned my addr 172.21.15.167:0/2746281963 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-10T00:46:23.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.699+0000 7fe5f49c3700 1 --2- 172.21.15.167:0/2746281963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f0100f30 0x7fe5f006b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:46:23.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.699+0000 7fe5effff700 1 --2- 172.21.15.167:0/2746281963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5f019a5c0 0x7fe5f019c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:46:23.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.699+0000 7fe5f51c4700 1 -- 172.21.15.167:0/2746281963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5f019a5c0 msgr2=0x7fe5f019c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:46:23.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.699+0000 7fe5f51c4700 1 --2- 172.21.15.167:0/2746281963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5f019a5c0 0x7fe5f019c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:23.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.699+0000 7fe5f51c4700 1 -- 172.21.15.167:0/2746281963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f0100f30 msgr2=0x7fe5f006b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:46:23.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.699+0000 7fe5f51c4700 1 --2- 172.21.15.167:0/2746281963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f0100f30 0x7fe5f006b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:23.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.699+0000 7fe5f51c4700 1 -- 172.21.15.167:0/2746281963 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f019fcf0 con 0x7fe5f019cf90 2024-06-10T00:46:23.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.699+0000 7fe5f49c3700 1 --2- 172.21.15.167:0/2746281963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f0100f30 0x7fe5f006b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:46:23.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.699+0000 7fe5f51c4700 1 --2- 172.21.15.167:0/2746281963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5f019cf90 0x7fe5f019f420 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800ba50 tx=0x7fe5e800bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:46:23.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.700+0000 7fe5edffb700 1 -- 172.21.15.167:0/2746281963 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8019070 con 0x7fe5f019cf90 2024-06-10T00:46:23.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.700+0000 7fe5edffb700 1 -- 172.21.15.167:0/2746281963 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5e8014440 con 0x7fe5f019cf90 2024-06-10T00:46:23.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.700+0000 7fe5f6c27700 1 -- 172.21.15.167:0/2746281963 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5f019ffe0 con 0x7fe5f019cf90 2024-06-10T00:46:23.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.700+0000 7fe5edffb700 1 -- 172.21.15.167:0/2746281963 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e80075b0 con 0x7fe5f019cf90 2024-06-10T00:46:23.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.700+0000 7fe5f6c27700 1 -- 172.21.15.167:0/2746281963 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5f01a04c0 con 0x7fe5f019cf90 2024-06-10T00:46:23.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.700+0000 7fe5edffb700 1 -- 172.21.15.167:0/2746281963 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe5e801c070 con 0x7fe5f019cf90 2024-06-10T00:46:23.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.701+0000 7fe5edffb700 1 --2- 172.21.15.167:0/2746281963 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe5e0039580 0x7fe5e003ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:46:23.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.701+0000 7fe5edffb700 1 -- 172.21.15.167:0/2746281963 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe5e804b840 con 0x7fe5f019cf90 2024-06-10T00:46:23.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.701+0000 7fe5f6c27700 1 -- 172.21.15.167:0/2746281963 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5f0063af0 con 0x7fe5f019cf90 2024-06-10T00:46:23.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.701+0000 7fe5f49c3700 1 --2- 172.21.15.167:0/2746281963 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe5e0039580 0x7fe5e003ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:46:23.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.702+0000 7fe5f49c3700 1 --2- 172.21.15.167:0/2746281963 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe5e0039580 0x7fe5e003ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4000c00 tx=0x7fe5e401a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:46:23.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.706+0000 7fe5edffb700 1 -- 172.21.15.167:0/2746281963 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe5e800e0a0 con 0x7fe5f019cf90 2024-06-10T00:46:23.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.919+0000 7fe5f6c27700 1 -- 172.21.15.167:0/2746281963 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi094=y;smithi167=z", "target": ["mon-mgr", ""]}) -- 0x7fe5f00fe3f0 con 0x7fe5e0039580 2024-06-10T00:46:23.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.926+0000 7fe5edffb700 1 -- 172.21.15.167:0/2746281963 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe5f00fe3f0 con 0x7fe5e0039580 2024-06-10T00:46:23.926 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mgr update... 2024-06-10T00:46:23.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.928+0000 7fe5f6c27700 1 -- 172.21.15.167:0/2746281963 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe5e0039580 msgr2=0x7fe5e003ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:46:23.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.928+0000 7fe5f6c27700 1 --2- 172.21.15.167:0/2746281963 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe5e0039580 0x7fe5e003ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4000c00 tx=0x7fe5e401a040 comp rx=0 tx=0).stop 2024-06-10T00:46:23.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.928+0000 7fe5f6c27700 1 -- 172.21.15.167:0/2746281963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5f019cf90 msgr2=0x7fe5f019f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:46:23.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.928+0000 7fe5f6c27700 1 --2- 172.21.15.167:0/2746281963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5f019cf90 0x7fe5f019f420 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800ba50 tx=0x7fe5e800bd60 comp rx=0 tx=0).stop 2024-06-10T00:46:23.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.928+0000 7fe5f6c27700 1 -- 172.21.15.167:0/2746281963 shutdown_connections 2024-06-10T00:46:23.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.928+0000 7fe5f6c27700 1 --2- 172.21.15.167:0/2746281963 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe5e0039580 0x7fe5e003ba40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:23.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.928+0000 7fe5f6c27700 1 --2- 172.21.15.167:0/2746281963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5f019cf90 0x7fe5f019f420 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:23.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.928+0000 7fe5f6c27700 1 --2- 172.21.15.167:0/2746281963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5f0100f30 0x7fe5f006b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:23.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.928+0000 7fe5f6c27700 1 --2- 172.21.15.167:0/2746281963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5f019a5c0 0x7fe5f019c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:23.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.928+0000 7fe5f6c27700 1 -- 172.21.15.167:0/2746281963 >> 172.21.15.167:0/2746281963 conn(0x7fe5f00fc4d0 msgr2=0x7fe5f00fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:46:23.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.928+0000 7fe5f6c27700 1 -- 172.21.15.167:0/2746281963 shutdown_connections 2024-06-10T00:46:23.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:46:23.928+0000 7fe5f6c27700 1 -- 172.21.15.167:0/2746281963 wait complete. 2024-06-10T00:46:24.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:24 smithi167 ceph-mon[36917]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:24.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:24 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:24.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:24 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:24.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:24 smithi167 ceph-mon[36917]: Reconfiguring mon.c (monmap changed)... 2024-06-10T00:46:24.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:24 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:46:24.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:24 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T00:46:24.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:24 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:24.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:24 smithi167 ceph-mon[36917]: Reconfiguring daemon mon.c on smithi167 2024-06-10T00:46:24.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:24 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:24.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:24 smithi094 ceph-mon[36708]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:24.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:24 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:24.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:24 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:24.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:24 smithi094 ceph-mon[36708]: Reconfiguring mon.c (monmap changed)... 2024-06-10T00:46:24.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:24 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:46:24.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:24 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T00:46:24.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:24 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:24.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:24 smithi094 ceph-mon[36708]: Reconfiguring daemon mon.c on smithi167 2024-06-10T00:46:24.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:24 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:24.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:24 smithi037 ceph-mon[32192]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:24.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:24 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:24.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:24 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:24.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:24 smithi037 ceph-mon[32192]: Reconfiguring mon.c (monmap changed)... 2024-06-10T00:46:24.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:24 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-10T00:46:24.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:24 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-10T00:46:24.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:24 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:24.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:24 smithi037 ceph-mon[32192]: Reconfiguring daemon mon.c on smithi167 2024-06-10T00:46:24.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:24 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:24.906 DEBUG:teuthology.orchestra.run.smithi094:mgr.y> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mgr.y.service 2024-06-10T00:46:24.909 DEBUG:teuthology.orchestra.run.smithi167:mgr.z> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mgr.z.service 2024-06-10T00:46:24.911 INFO:tasks.cephadm:Deploying OSDs... 2024-06-10T00:46:24.911 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-06-10T00:46:24.911 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/scratch_devs of=/dev/stdout 2024-06-10T00:46:24.931 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-10T00:46:24.932 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_1 2024-06-10T00:46:24.990 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-10T00:46:24.990 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:46:24.990 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-06-10T00:46:24.990 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:46:24.991 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:46:24.991 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-06-10 00:45:29.763228381 +0000 2024-06-10T00:46:24.991 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-06-10 00:40:18.302295727 +0000 2024-06-10T00:46:24.991 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-06-10 00:40:18.302295727 +0000 2024-06-10T00:46:24.991 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-06-10 00:40:18.302295727 +0000 2024-06-10T00:46:24.991 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-10T00:46:25.062 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-06-10T00:46:25.062 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-06-10T00:46:25.062 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.00021271 s, 2.4 MB/s 2024-06-10T00:46:25.064 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-10T00:46:25.124 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_2 2024-06-10T00:46:25.175 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:25 smithi167 ceph-mon[36917]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi094=y;smithi167=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:46:25.175 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:25 smithi167 ceph-mon[36917]: Saving service mgr spec with placement smithi037=x;smithi094=y;smithi167=z;count:3 2024-06-10T00:46:25.183 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-10T00:46:25.183 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:46:25.183 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-06-10T00:46:25.183 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:46:25.183 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:46:25.183 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-06-10 00:45:29.799227900 +0000 2024-06-10T00:46:25.183 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-06-10 00:40:18.639291179 +0000 2024-06-10T00:46:25.183 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-06-10 00:40:18.639291179 +0000 2024-06-10T00:46:25.183 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-06-10 00:40:18.639291179 +0000 2024-06-10T00:46:25.184 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-10T00:46:25.253 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-06-10T00:46:25.253 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-06-10T00:46:25.253 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000197828 s, 2.6 MB/s 2024-06-10T00:46:25.254 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-10T00:46:25.315 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_3 2024-06-10T00:46:25.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:25 smithi094 ceph-mon[36708]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi094=y;smithi167=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:46:25.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:25 smithi094 ceph-mon[36708]: Saving service mgr spec with placement smithi037=x;smithi094=y;smithi167=z;count:3 2024-06-10T00:46:25.371 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-10T00:46:25.371 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:46:25.371 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-06-10T00:46:25.371 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:46:25.372 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:46:25.372 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-06-10 00:45:29.840227352 +0000 2024-06-10T00:46:25.372 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-06-10 00:40:18.993286402 +0000 2024-06-10T00:46:25.372 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-06-10 00:40:18.993286402 +0000 2024-06-10T00:46:25.372 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-06-10 00:40:18.993286402 +0000 2024-06-10T00:46:25.372 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-10T00:46:25.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:25 smithi037 ceph-mon[32192]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi094=y;smithi167=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:46:25.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:25 smithi037 ceph-mon[32192]: Saving service mgr spec with placement smithi037=x;smithi094=y;smithi167=z;count:3 2024-06-10T00:46:25.419 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-06-10T00:46:25.420 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-06-10T00:46:25.420 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000191837 s, 2.7 MB/s 2024-06-10T00:46:25.421 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-10T00:46:25.477 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_4 2024-06-10T00:46:25.538 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-10T00:46:25.538 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:46:25.538 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-06-10T00:46:25.538 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:46:25.538 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:46:25.538 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-06-10 00:45:29.875226885 +0000 2024-06-10T00:46:25.539 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-06-10 00:40:19.335281787 +0000 2024-06-10T00:46:25.539 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-06-10 00:40:19.335281787 +0000 2024-06-10T00:46:25.539 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-06-10 00:40:19.335281787 +0000 2024-06-10T00:46:25.539 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-10T00:46:25.604 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-06-10T00:46:25.604 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-06-10T00:46:25.604 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000201866 s, 2.5 MB/s 2024-06-10T00:46:25.605 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-10T00:46:25.662 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-10T00:46:25.663 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/scratch_devs of=/dev/stdout 2024-06-10T00:46:25.681 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-10T00:46:25.681 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_1 2024-06-10T00:46:25.739 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-10T00:46:25.739 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:46:25.739 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-06-10T00:46:25.739 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:46:25.740 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:46:25.740 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-06-10 00:46:11.468359670 +0000 2024-06-10T00:46:25.740 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-06-10 00:40:25.113810675 +0000 2024-06-10T00:46:25.740 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-06-10 00:40:25.113810675 +0000 2024-06-10T00:46:25.740 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-06-10 00:40:25.113810675 +0000 2024-06-10T00:46:25.740 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-10T00:46:25.805 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-06-10T00:46:25.806 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-06-10T00:46:25.806 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000113777 s, 4.5 MB/s 2024-06-10T00:46:25.807 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-10T00:46:25.865 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_2 2024-06-10T00:46:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-10T00:46:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:46:25.923 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-06-10T00:46:25.923 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:46:25.923 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:46:25.923 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-06-10 00:46:11.468359670 +0000 2024-06-10T00:46:25.923 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-06-10 00:40:25.434804901 +0000 2024-06-10T00:46:25.923 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-06-10 00:40:25.434804901 +0000 2024-06-10T00:46:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-06-10 00:40:25.434804901 +0000 2024-06-10T00:46:25.923 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-10T00:46:25.987 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-06-10T00:46:25.987 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-06-10T00:46:25.987 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000120349 s, 4.3 MB/s 2024-06-10T00:46:25.989 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-10T00:46:26.045 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_3 2024-06-10T00:46:26.101 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-10T00:46:26.102 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:46:26.102 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-06-10T00:46:26.102 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:46:26.102 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:46:26.102 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-06-10 00:46:11.469359653 +0000 2024-06-10T00:46:26.102 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-06-10 00:40:25.743799342 +0000 2024-06-10T00:46:26.102 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-06-10 00:40:25.743799342 +0000 2024-06-10T00:46:26.102 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-06-10 00:40:25.743799342 +0000 2024-06-10T00:46:26.103 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-10T00:46:26.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:26 smithi094 ceph-mon[36708]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:26.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:26 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:26.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:26 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:26.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:26 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:46:26.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:26 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:26.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:26 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:46:26.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:26 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:26.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:26 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T00:46:26.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:26 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-10T00:46:26.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:26 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T00:46:26.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:26 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:26.137 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-06-10T00:46:26.137 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-06-10T00:46:26.137 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000116149 s, 4.4 MB/s 2024-06-10T00:46:26.139 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-10T00:46:26.195 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_4 2024-06-10T00:46:26.250 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-10T00:46:26.250 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:46:26.250 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-06-10T00:46:26.250 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:46:26.250 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:46:26.250 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-06-10 00:46:11.469359653 +0000 2024-06-10T00:46:26.250 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-06-10 00:40:26.074793389 +0000 2024-06-10T00:46:26.250 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-06-10 00:40:26.074793389 +0000 2024-06-10T00:46:26.251 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-06-10 00:40:26.074793389 +0000 2024-06-10T00:46:26.251 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-10T00:46:26.312 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-06-10T00:46:26.312 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-06-10T00:46:26.312 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 8.9529e-05 s, 5.7 MB/s 2024-06-10T00:46:26.313 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-10T00:46:26.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:26 smithi167 ceph-mon[36917]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:26.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:26 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:26.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:26 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:26.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:26 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:46:26.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:26 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:26.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:26 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:46:26.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:26 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:26.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:26 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T00:46:26.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:26 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-10T00:46:26.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:26 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T00:46:26.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:26 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:26.368 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-10T00:46:26.368 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/scratch_devs of=/dev/stdout 2024-06-10T00:46:26.383 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-10T00:46:26.384 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_1 2024-06-10T00:46:26.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:26 smithi037 ceph-mon[32192]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:26.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:26 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:26.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:26 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:26.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:26 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:46:26.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:26 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:26.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:26 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:46:26.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:26 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:26.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:26 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T00:46:26.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:26 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-10T00:46:26.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:26 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T00:46:26.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:26 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:26.438 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-10T00:46:26.438 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:46:26.438 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-06-10T00:46:26.438 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:46:26.438 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:46:26.438 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-06-10 00:46:01.741306762 +0000 2024-06-10T00:46:26.438 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-06-10 00:40:31.152339470 +0000 2024-06-10T00:46:26.438 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-06-10 00:40:31.152339470 +0000 2024-06-10T00:46:26.438 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-06-10 00:40:31.152339470 +0000 2024-06-10T00:46:26.438 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-10T00:46:26.500 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-06-10T00:46:26.500 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-06-10T00:46:26.500 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000286124 s, 1.8 MB/s 2024-06-10T00:46:26.501 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-10T00:46:26.556 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_2 2024-06-10T00:46:26.611 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-10T00:46:26.611 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:46:26.611 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-06-10T00:46:26.612 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:46:26.612 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:46:26.612 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-06-10 00:46:01.741306762 +0000 2024-06-10T00:46:26.612 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-06-10 00:40:31.484333599 +0000 2024-06-10T00:46:26.612 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-06-10 00:40:31.484333599 +0000 2024-06-10T00:46:26.612 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-06-10 00:40:31.484333599 +0000 2024-06-10T00:46:26.612 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-10T00:46:26.675 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-06-10T00:46:26.675 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-06-10T00:46:26.675 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000171426 s, 3.0 MB/s 2024-06-10T00:46:26.676 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-10T00:46:26.732 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_3 2024-06-10T00:46:26.787 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-10T00:46:26.787 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:46:26.787 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-06-10T00:46:26.787 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:46:26.787 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:46:26.787 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-06-10 00:46:01.741306762 +0000 2024-06-10T00:46:26.788 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-06-10 00:40:31.797328065 +0000 2024-06-10T00:46:26.788 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-06-10 00:40:31.797328065 +0000 2024-06-10T00:46:26.788 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-06-10 00:40:31.797328065 +0000 2024-06-10T00:46:26.788 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-10T00:46:26.850 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-06-10T00:46:26.850 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-06-10T00:46:26.850 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000192751 s, 2.7 MB/s 2024-06-10T00:46:26.852 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-10T00:46:26.907 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_4 2024-06-10T00:46:26.961 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-10T00:46:26.962 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-10T00:46:26.962 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-06-10T00:46:26.962 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:46:26.962 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-06-10T00:46:26.962 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-06-10 00:46:01.742306745 +0000 2024-06-10T00:46:26.962 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-06-10 00:40:32.127322231 +0000 2024-06-10T00:46:26.962 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-06-10 00:40:32.127322231 +0000 2024-06-10T00:46:26.962 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-06-10 00:40:32.127322231 +0000 2024-06-10T00:46:26.962 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-10T00:46:27.024 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-06-10T00:46:27.024 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-06-10T00:46:27.024 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000166613 s, 3.1 MB/s 2024-06-10T00:46:27.025 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-10T00:46:27.080 INFO:tasks.cephadm:Deploying osd.0 on smithi037 with /dev/vg_nvme/lv_4... 2024-06-10T00:46:27.081 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-10T00:46:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:46:27.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:27 smithi167 ceph-mon[36917]: Deploying daemon mgr.z on smithi167 2024-06-10T00:46:27.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:27 smithi094 ceph-mon[36708]: Deploying daemon mgr.z on smithi167 2024-06-10T00:46:27.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:27 smithi037 ceph-mon[32192]: Deploying daemon mgr.z on smithi167 2024-06-10T00:46:28.037 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:27 smithi167 systemd[1]: Starting Ceph mgr.z for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T00:46:28.289 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:28 smithi167 ceph-mon[36917]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:28.289 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:28 smithi167 podman[38751]: 2024-06-10 00:46:28.042758459 +0000 UTC m=+0.095545860 container create fbbed67eb8043fdc5144732a1a4db9b47c44ed66e7ce9ef1d0e5206406b0ab9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-06-10T00:46:28.289 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:28 smithi167 podman[38751]: 2024-06-10 00:46:27.963984728 +0000 UTC m=+0.016772134 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:46:28.290 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:28 smithi167 podman[38751]: 2024-06-10 00:46:28.262475999 +0000 UTC m=+0.315263396 container init fbbed67eb8043fdc5144732a1a4db9b47c44ed66e7ce9ef1d0e5206406b0ab9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-10T00:46:28.290 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:28 smithi167 podman[38751]: 2024-06-10 00:46:28.264978784 +0000 UTC m=+0.317766182 container start fbbed67eb8043fdc5144732a1a4db9b47c44ed66e7ce9ef1d0e5206406b0ab9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T00:46:28.290 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:28 smithi167 ceph-mgr[38770]: -- 172.21.15.167:0/658183283 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aa7e19e5a0 con 0x55aa7e173000 2024-06-10T00:46:28.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:28 smithi094 ceph-mon[36708]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:28.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:28 smithi037 ceph-mon[32192]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:28.588 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:28 smithi167 bash[38751]: fbbed67eb8043fdc5144732a1a4db9b47c44ed66e7ce9ef1d0e5206406b0ab9d 2024-06-10T00:46:28.588 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:28 smithi167 systemd[1]: Started Ceph mgr.z for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T00:46:28.589 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:28 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z[38766]: 2024-06-10T00:46:28.419+0000 7f2fd63d2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-10T00:46:29.021 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:28 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z[38766]: 2024-06-10T00:46:28.607+0000 7f2fd63d2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:46:29.021 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:28 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z[38766]: 2024-06-10T00:46:28.710+0000 7f2fd63d2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-10T00:46:29.022 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:29 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z[38766]: 2024-06-10T00:46:29.021+0000 7f2fd63d2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-10T00:46:29.140 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-10T00:46:29.140 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-10T00:46:29.140 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-06-10T00:46:29.140 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-06-10T00:46:29.140 INFO:teuthology.orchestra.run.smithi037.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0120787 s, 868 MB/s 2024-06-10T00:46:29.140 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-06-10T00:46:29.300 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:29 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z[38766]: 2024-06-10T00:46:29.112+0000 7f2fd63d2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-10T00:46:29.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:29 smithi167 ceph-mon[36917]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:29.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:29 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:29.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:29 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:29.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:29 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:29.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:29 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T00:46:29.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:29 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-10T00:46:29.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:29 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T00:46:29.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:29 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:29.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:29 smithi167 ceph-mon[36917]: Deploying daemon mgr.y on smithi094 2024-06-10T00:46:29.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:29 smithi094 ceph-mon[36708]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:29.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:29 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:29.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:29 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:29.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:29 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:29.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:29 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T00:46:29.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:29 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-10T00:46:29.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:29 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T00:46:29.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:29 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:29.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:29 smithi094 ceph-mon[36708]: Deploying daemon mgr.y on smithi094 2024-06-10T00:46:29.670 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:29 smithi037 ceph-mon[32192]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:29.670 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:29 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:29.670 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:29 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:29.670 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:29 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:29.670 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:29 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T00:46:29.670 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:29 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-10T00:46:29.670 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:29 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T00:46:29.670 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:29 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:29.670 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:29 smithi037 ceph-mon[32192]: Deploying daemon mgr.y on smithi094 2024-06-10T00:46:29.699 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_4 2024-06-10T00:46:29.946 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:46:30.588 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:30 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z[38766]: 2024-06-10T00:46:30.369+0000 7f2fd63d2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-10T00:46:30.589 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:30 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z[38766]: 2024-06-10T00:46:30.539+0000 7f2fd63d2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-10T00:46:30.997 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 00:46:30 smithi094 systemd[1]: Started Ceph mgr.y for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T00:46:31.000 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:30 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z[38766]: 2024-06-10T00:46:30.711+0000 7f2fd63d2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-10T00:46:31.295 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 00:46:30 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y[38087]: 2024-06-10T00:46:30.997+0000 7f85d6dc5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-10T00:46:31.295 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 00:46:31 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y[38087]: 2024-06-10T00:46:31.186+0000 7f85d6dc5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:46:31.295 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 00:46:31 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y[38087]: 2024-06-10T00:46:31.295+0000 7f85d6dc5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-10T00:46:31.306 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:31 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z[38766]: 2024-06-10T00:46:30.999+0000 7f2fd63d2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-10T00:46:31.306 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:31 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z[38766]: 2024-06-10T00:46:31.092+0000 7f2fd63d2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-10T00:46:31.306 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:31 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z[38766]: 2024-06-10T00:46:31.305+0000 7f2fd63d2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-10T00:46:31.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.419+0000 7f7647f3d700 1 -- 172.21.15.37:0/2203503171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76401025f0 msgr2=0x7f7640102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:46:31.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.419+0000 7f7647f3d700 1 --2- 172.21.15.37:0/2203503171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76401025f0 0x7f7640102a50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f76340099c0 tx=0x7f7634009cd0 comp rx=0 tx=0).stop 2024-06-10T00:46:31.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.419+0000 7f7647f3d700 1 -- 172.21.15.37:0/2203503171 shutdown_connections 2024-06-10T00:46:31.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.419+0000 7f7647f3d700 1 --2- 172.21.15.37:0/2203503171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7640102f90 0x7f7640109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:31.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.419+0000 7f7647f3d700 1 --2- 172.21.15.37:0/2203503171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76401013f0 0x7f7640101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:31.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.419+0000 7f7647f3d700 1 --2- 172.21.15.37:0/2203503171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76401025f0 0x7f7640102a50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:31.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.419+0000 7f7647f3d700 1 -- 172.21.15.37:0/2203503171 >> 172.21.15.37:0/2203503171 conn(0x7f76400fc990 msgr2=0x7f76400fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:46:31.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.419+0000 7f7647f3d700 1 -- 172.21.15.37:0/2203503171 shutdown_connections 2024-06-10T00:46:31.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.419+0000 7f7647f3d700 1 -- 172.21.15.37:0/2203503171 wait complete. 2024-06-10T00:46:31.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.420+0000 7f7647f3d700 1 Processor -- start 2024-06-10T00:46:31.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.420+0000 7f7647f3d700 1 -- start start 2024-06-10T00:46:31.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.420+0000 7f7647f3d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76401013f0 0x7f7640198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:46:31.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.420+0000 7f7647f3d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76401025f0 0x7f7640198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:46:31.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.420+0000 7f7647f3d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7640102f90 0x7f764019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:46:31.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.420+0000 7f7647f3d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f764010cfc0 con 0x7f76401025f0 2024-06-10T00:46:31.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.420+0000 7f7647f3d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f764010ce40 con 0x7f76401013f0 2024-06-10T00:46:31.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.420+0000 7f7647f3d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f764010d140 con 0x7f7640102f90 2024-06-10T00:46:31.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.420+0000 7f76454d8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76401025f0 0x7f7640198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:46:31.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.420+0000 7f7645cd9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76401013f0 0x7f7640198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:46:31.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.420+0000 7f7645cd9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76401013f0 0x7f7640198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:33386/0 (socket says 172.21.15.37:33386) 2024-06-10T00:46:31.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.420+0000 7f76454d8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76401025f0 0x7f7640198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33532/0 (socket says 172.21.15.37:33532) 2024-06-10T00:46:31.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.420+0000 7f7645cd9700 1 -- 172.21.15.37:0/1253880030 learned_addr learned my addr 172.21.15.37:0/1253880030 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:46:31.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.420+0000 7f76464da700 1 --2- 172.21.15.37:0/1253880030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7640102f90 0x7f764019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:46:31.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.421+0000 7f7645cd9700 1 -- 172.21.15.37:0/1253880030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7640102f90 msgr2=0x7f764019fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:46:31.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.421+0000 7f7645cd9700 1 --2- 172.21.15.37:0/1253880030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7640102f90 0x7f764019fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:31.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.421+0000 7f7645cd9700 1 -- 172.21.15.37:0/1253880030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76401025f0 msgr2=0x7f7640198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:46:31.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.421+0000 7f7645cd9700 1 --2- 172.21.15.37:0/1253880030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76401025f0 0x7f7640198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:31.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.421+0000 7f7645cd9700 1 -- 172.21.15.37:0/1253880030 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76401a0440 con 0x7f76401013f0 2024-06-10T00:46:31.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.421+0000 7f76454d8700 1 --2- 172.21.15.37:0/1253880030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76401025f0 0x7f7640198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:46:31.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.421+0000 7f7645cd9700 1 --2- 172.21.15.37:0/1253880030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76401013f0 0x7f7640198830 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7630007ee0 tx=0x7f763000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:46:31.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.421+0000 7f763effd700 1 -- 172.21.15.37:0/1253880030 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763001a070 con 0x7f76401013f0 2024-06-10T00:46:31.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.421+0000 7f763effd700 1 -- 172.21.15.37:0/1253880030 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f763000dd20 con 0x7f76401013f0 2024-06-10T00:46:31.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.421+0000 7f7647f3d700 1 -- 172.21.15.37:0/1253880030 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76401a0730 con 0x7f76401013f0 2024-06-10T00:46:31.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.421+0000 7f763effd700 1 -- 172.21.15.37:0/1253880030 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76300146b0 con 0x7f76401013f0 2024-06-10T00:46:31.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.421+0000 7f7647f3d700 1 -- 172.21.15.37:0/1253880030 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76401a0cc0 con 0x7f76401013f0 2024-06-10T00:46:31.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.422+0000 7f763effd700 1 -- 172.21.15.37:0/1253880030 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f763001d070 con 0x7f76401013f0 2024-06-10T00:46:31.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.422+0000 7f7647f3d700 1 -- 172.21.15.37:0/1253880030 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7640069180 con 0x7f76401013f0 2024-06-10T00:46:31.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.422+0000 7f763effd700 1 --2- 172.21.15.37:0/1253880030 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7628039330 0x7f762803b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:46:31.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.422+0000 7f763effd700 1 -- 172.21.15.37:0/1253880030 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f763004bd20 con 0x7f76401013f0 2024-06-10T00:46:31.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.422+0000 7f76454d8700 1 --2- 172.21.15.37:0/1253880030 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7628039330 0x7f762803b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:46:31.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.423+0000 7f76454d8700 1 --2- 172.21.15.37:0/1253880030 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7628039330 0x7f762803b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7634000c00 tx=0x7f7634005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:46:31.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.428+0000 7f763effd700 1 -- 172.21.15.37:0/1253880030 <== mon.2 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7630010070 con 0x7f76401013f0 2024-06-10T00:46:31.588 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:31 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z[38766]: 2024-06-10T00:46:31.507+0000 7f2fd63d2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:46:31.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:31.646+0000 7f7647f3d700 1 -- 172.21.15.37:0/1253880030 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f76400feb90 con 0x7f7628039330 2024-06-10T00:46:31.937 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 00:46:31 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y[38087]: 2024-06-10T00:46:31.622+0000 7f85d6dc5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-10T00:46:31.937 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 00:46:31 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y[38087]: 2024-06-10T00:46:31.719+0000 7f85d6dc5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-10T00:46:32.271 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:31 smithi037 ceph-mon[32192]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:32.272 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:31 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:32.272 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:31 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:32.272 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:31 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:32.272 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:31 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:32.272 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:31 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:46:32.272 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:31 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:46:32.272 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:31 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:46:32.272 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:31 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:32.338 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:32 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z[38766]: 2024-06-10T00:46:32.155+0000 7f2fd63d2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-10T00:46:32.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:31 smithi167 ceph-mon[36917]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:32.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:31 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:32.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:31 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:32.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:31 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:32.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:31 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:32.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:31 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:46:32.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:31 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:46:32.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:31 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:46:32.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:31 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:32.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:31 smithi094 ceph-mon[36708]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:32.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:31 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:32.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:31 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:32.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:31 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:32.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:31 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:32.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:31 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:46:32.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:31 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:46:32.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:31 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:46:32.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:31 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:32.898 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:32 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z[38766]: 2024-06-10T00:46:32.596+0000 7f2fd63d2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-10T00:46:32.898 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:32 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z[38766]: 2024-06-10T00:46:32.694+0000 7f2fd63d2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-10T00:46:32.898 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:32 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z[38766]: 2024-06-10T00:46:32.897+0000 7f2fd63d2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-10T00:46:33.287 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 00:46:32 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y[38087]: 2024-06-10T00:46:32.946+0000 7f85d6dc5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-10T00:46:33.288 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 00:46:33 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y[38087]: 2024-06-10T00:46:33.117+0000 7f85d6dc5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-10T00:46:33.288 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 00:46:33 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y[38087]: 2024-06-10T00:46:33.286+0000 7f85d6dc5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-10T00:46:33.288 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:32 smithi094 ceph-mon[36708]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:46:33.288 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:32 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:33.288 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:32 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:33.288 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:32 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:33.288 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:32 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:33.288 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:32 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:33.288 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:32 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:46:33.288 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:32 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:33.289 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:32 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T00:46:33.289 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:32 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T00:46:33.289 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:32 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:33.303 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:32 smithi037 ceph-mon[32192]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:46:33.303 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:32 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:33.303 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:32 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:33.303 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:32 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:33.303 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:32 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:33.303 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:32 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:33.303 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:32 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:46:33.304 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:32 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:33.304 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:32 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T00:46:33.304 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:32 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T00:46:33.304 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:32 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:33.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:32 smithi167 ceph-mon[36917]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:46:33.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:32 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:33.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:32 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:33.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:32 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:33.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:32 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:33.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:32 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:33.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:32 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:46:33.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:32 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:33.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:32 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-10T00:46:33.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:32 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-10T00:46:33.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:32 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:33.764 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:33 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z[38766]: 2024-06-10T00:46:33.428+0000 7f2fd63d2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-10T00:46:33.764 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:33 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z[38766]: 2024-06-10T00:46:33.519+0000 7f2fd63d2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-10T00:46:33.864 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 00:46:33 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y[38087]: 2024-06-10T00:46:33.570+0000 7f85d6dc5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-10T00:46:33.864 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 00:46:33 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y[38087]: 2024-06-10T00:46:33.661+0000 7f85d6dc5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-10T00:46:34.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:33 smithi167 ceph-mon[36917]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:34.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:33 smithi167 ceph-mon[36917]: Reconfiguring mgr.x (unknown last config time)... 2024-06-10T00:46:34.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:33 smithi167 ceph-mon[36917]: Reconfiguring daemon mgr.x on smithi037 2024-06-10T00:46:34.089 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:33 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z[38766]: 2024-06-10T00:46:33.763+0000 7f2fd63d2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-10T00:46:34.089 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:33 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z[38766]: 2024-06-10T00:46:33.899+0000 7f2fd63d2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-10T00:46:34.089 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:33 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z[38766]: 2024-06-10T00:46:33.995+0000 7f2fd63d2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-10T00:46:34.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:33 smithi094 ceph-mon[36708]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:34.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:33 smithi094 ceph-mon[36708]: Reconfiguring mgr.x (unknown last config time)... 2024-06-10T00:46:34.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:33 smithi094 ceph-mon[36708]: Reconfiguring daemon mgr.x on smithi037 2024-06-10T00:46:34.365 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 00:46:33 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y[38087]: 2024-06-10T00:46:33.867+0000 7f85d6dc5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-10T00:46:34.365 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 00:46:34 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y[38087]: 2024-06-10T00:46:34.058+0000 7f85d6dc5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-10T00:46:34.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:33 smithi037 ceph-mon[32192]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:34.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:33 smithi037 ceph-mon[32192]: Reconfiguring mgr.x (unknown last config time)... 2024-06-10T00:46:34.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:33 smithi037 ceph-mon[32192]: Reconfiguring daemon mgr.x on smithi037 2024-06-10T00:46:34.416 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:34 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z[38766]: 2024-06-10T00:46:34.113+0000 7f2fd63d2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-10T00:46:34.417 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:34 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z[38766]: 2024-06-10T00:46:34.416+0000 7f2fd63d2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-10T00:46:34.978 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 00:46:34 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y[38087]: 2024-06-10T00:46:34.655+0000 7f85d6dc5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-10T00:46:35.238 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:34 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2736575807' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1df6046a-6801-4950-9193-54303fed169b"}]: dispatch 2024-06-10T00:46:35.238 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:34 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2736575807' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1df6046a-6801-4950-9193-54303fed169b"}]': finished 2024-06-10T00:46:35.238 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:34 smithi167 ceph-mon[36917]: osdmap e5: 1 total, 0 up, 1 in 2024-06-10T00:46:35.238 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:34 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:46:35.238 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:34 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3861403291' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:46:35.238 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:35 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z[38766]: 2024-06-10T00:46:35.020+0000 7f2fd63d2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-10T00:46:35.239 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 00:46:35 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z[38766]: 2024-06-10T00:46:35.237+0000 7f2fd63d2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-10T00:46:35.364 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 00:46:35 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y[38087]: 2024-06-10T00:46:35.072+0000 7f85d6dc5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-10T00:46:35.364 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 00:46:35 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y[38087]: 2024-06-10T00:46:35.167+0000 7f85d6dc5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-10T00:46:35.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:34 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2736575807' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1df6046a-6801-4950-9193-54303fed169b"}]: dispatch 2024-06-10T00:46:35.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:34 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2736575807' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1df6046a-6801-4950-9193-54303fed169b"}]': finished 2024-06-10T00:46:35.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:34 smithi094 ceph-mon[36708]: osdmap e5: 1 total, 0 up, 1 in 2024-06-10T00:46:35.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:34 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:46:35.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:34 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3861403291' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:46:35.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:34 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2736575807' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1df6046a-6801-4950-9193-54303fed169b"}]: dispatch 2024-06-10T00:46:35.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:34 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2736575807' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1df6046a-6801-4950-9193-54303fed169b"}]': finished 2024-06-10T00:46:35.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:34 smithi037 ceph-mon[32192]: osdmap e5: 1 total, 0 up, 1 in 2024-06-10T00:46:35.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:34 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:46:35.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:34 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3861403291' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:46:35.864 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 00:46:35 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y[38087]: 2024-06-10T00:46:35.367+0000 7f85d6dc5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-10T00:46:35.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:35.984+0000 7f763effd700 1 -- 172.21.15.37:0/1253880030 <== mon.2 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f7630014850 con 0x7f76401013f0 2024-06-10T00:46:36.218 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 00:46:35 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y[38087]: 2024-06-10T00:46:35.884+0000 7f85d6dc5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-10T00:46:36.219 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 00:46:35 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y[38087]: 2024-06-10T00:46:35.973+0000 7f85d6dc5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-10T00:46:36.219 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:35 smithi094 ceph-mon[36708]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:36.219 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:35 smithi094 ceph-mon[36708]: Standby manager daemon z started 2024-06-10T00:46:36.219 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:35 smithi094 ceph-mon[36708]: from='mgr.? 172.21.15.167:0/1700228406' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-10T00:46:36.219 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:35 smithi094 ceph-mon[36708]: from='mgr.? 172.21.15.167:0/1700228406' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T00:46:36.219 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:35 smithi094 ceph-mon[36708]: from='mgr.? 172.21.15.167:0/1700228406' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-10T00:46:36.219 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:35 smithi094 ceph-mon[36708]: from='mgr.? 172.21.15.167:0/1700228406' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T00:46:36.219 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:35 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:36.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:35 smithi167 ceph-mon[36917]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:36.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:35 smithi167 ceph-mon[36917]: Standby manager daemon z started 2024-06-10T00:46:36.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:35 smithi167 ceph-mon[36917]: from='mgr.? 172.21.15.167:0/1700228406' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-10T00:46:36.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:35 smithi167 ceph-mon[36917]: from='mgr.? 172.21.15.167:0/1700228406' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T00:46:36.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:35 smithi167 ceph-mon[36917]: from='mgr.? 172.21.15.167:0/1700228406' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-10T00:46:36.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:35 smithi167 ceph-mon[36917]: from='mgr.? 172.21.15.167:0/1700228406' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T00:46:36.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:35 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:36.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:35 smithi037 ceph-mon[32192]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:36.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:35 smithi037 ceph-mon[32192]: Standby manager daemon z started 2024-06-10T00:46:36.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:35 smithi037 ceph-mon[32192]: from='mgr.? 172.21.15.167:0/1700228406' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-10T00:46:36.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:35 smithi037 ceph-mon[32192]: from='mgr.? 172.21.15.167:0/1700228406' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T00:46:36.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:35 smithi037 ceph-mon[32192]: from='mgr.? 172.21.15.167:0/1700228406' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-10T00:46:36.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:35 smithi037 ceph-mon[32192]: from='mgr.? 172.21.15.167:0/1700228406' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T00:46:36.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:35 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:36.559 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 00:46:36 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y[38087]: 2024-06-10T00:46:36.217+0000 7f85d6dc5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-10T00:46:36.559 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 00:46:36 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y[38087]: 2024-06-10T00:46:36.347+0000 7f85d6dc5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-10T00:46:36.559 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 00:46:36 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y[38087]: 2024-06-10T00:46:36.441+0000 7f85d6dc5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-10T00:46:36.559 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 00:46:36 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y[38087]: 2024-06-10T00:46:36.557+0000 7f85d6dc5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-10T00:46:37.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:36 smithi167 ceph-mon[36917]: mgrmap e14: x(active, since 75s), standbys: z 2024-06-10T00:46:37.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:36 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-10T00:46:37.364 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 00:46:36 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y[38087]: 2024-06-10T00:46:36.873+0000 7f85d6dc5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-10T00:46:37.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:36 smithi094 ceph-mon[36708]: mgrmap e14: x(active, since 75s), standbys: z 2024-06-10T00:46:37.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:36 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-10T00:46:37.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:36 smithi037 ceph-mon[32192]: mgrmap e14: x(active, since 75s), standbys: z 2024-06-10T00:46:37.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:36 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-10T00:46:37.864 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 00:46:37 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y[38087]: 2024-06-10T00:46:37.471+0000 7f85d6dc5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-10T00:46:37.864 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 00:46:37 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y[38087]: 2024-06-10T00:46:37.688+0000 7f85d6dc5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-10T00:46:38.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:38.003+0000 7f763effd700 1 -- 172.21.15.37:0/1253880030 <== mon.2 v2:172.21.15.94:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f7630022120 con 0x7f76401013f0 2024-06-10T00:46:38.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:37 smithi037 ceph-mon[32192]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:38.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:37 smithi037 ceph-mon[32192]: Standby manager daemon y started 2024-06-10T00:46:38.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:37 smithi037 ceph-mon[32192]: from='mgr.? 172.21.15.94:0/1807829091' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-10T00:46:38.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:37 smithi037 ceph-mon[32192]: from='mgr.? 172.21.15.94:0/1807829091' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T00:46:38.212 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:37 smithi037 ceph-mon[32192]: from='mgr.? 172.21.15.94:0/1807829091' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-10T00:46:38.212 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:37 smithi037 ceph-mon[32192]: from='mgr.? 172.21.15.94:0/1807829091' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T00:46:38.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:37 smithi167 ceph-mon[36917]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:38.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:37 smithi167 ceph-mon[36917]: Standby manager daemon y started 2024-06-10T00:46:38.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:37 smithi167 ceph-mon[36917]: from='mgr.? 172.21.15.94:0/1807829091' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-10T00:46:38.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:37 smithi167 ceph-mon[36917]: from='mgr.? 172.21.15.94:0/1807829091' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T00:46:38.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:37 smithi167 ceph-mon[36917]: from='mgr.? 172.21.15.94:0/1807829091' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-10T00:46:38.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:37 smithi167 ceph-mon[36917]: from='mgr.? 172.21.15.94:0/1807829091' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T00:46:38.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:37 smithi094 ceph-mon[36708]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:38.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:37 smithi094 ceph-mon[36708]: Standby manager daemon y started 2024-06-10T00:46:38.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:37 smithi094 ceph-mon[36708]: from='mgr.? 172.21.15.94:0/1807829091' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-10T00:46:38.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:37 smithi094 ceph-mon[36708]: from='mgr.? 172.21.15.94:0/1807829091' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-10T00:46:38.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:37 smithi094 ceph-mon[36708]: from='mgr.? 172.21.15.94:0/1807829091' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-10T00:46:38.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:37 smithi094 ceph-mon[36708]: from='mgr.? 172.21.15.94:0/1807829091' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-10T00:46:39.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:39 smithi037 ceph-mon[32192]: mgrmap e15: x(active, since 77s), standbys: y, z 2024-06-10T00:46:39.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:39 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-10T00:46:39.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:39 smithi167 ceph-mon[36917]: mgrmap e15: x(active, since 77s), standbys: y, z 2024-06-10T00:46:39.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:39 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-10T00:46:39.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:39 smithi094 ceph-mon[36708]: mgrmap e15: x(active, since 77s), standbys: y, z 2024-06-10T00:46:39.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:39 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-10T00:46:40.189 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:40 smithi037 ceph-mon[32192]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:40.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:40 smithi167 ceph-mon[36917]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:40.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:40 smithi094 ceph-mon[36708]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:41.572 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:41.572 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:41.572 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:41 smithi037 ceph-mon[32192]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:41.572 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:46:41.572 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:41.572 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:46:41.573 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:41.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:41 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:41.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:41 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:41.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:41 smithi167 ceph-mon[36917]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:41.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:41 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:46:41.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:41 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:41.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:41 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:46:41.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:41 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:41.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:41 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:41.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:41 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:41.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:41 smithi094 ceph-mon[36708]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:41.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:41 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:46:41.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:41 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:41.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:41 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:46:41.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:41 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:42.537 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:42 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-10T00:46:42.538 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:42 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:42.538 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:42 smithi037 ceph-mon[32192]: Deploying daemon osd.0 on smithi037 2024-06-10T00:46:42.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:42 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-10T00:46:42.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:42 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:42.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:42 smithi167 ceph-mon[36917]: Deploying daemon osd.0 on smithi037 2024-06-10T00:46:42.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:42 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-10T00:46:42.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:42 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:42.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:42 smithi094 ceph-mon[36708]: Deploying daemon osd.0 on smithi037 2024-06-10T00:46:43.519 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:43 smithi037 ceph-mon[32192]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:43.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:43 smithi167 ceph-mon[36917]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:43.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:43 smithi094 ceph-mon[36708]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:45.465 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:45 smithi037 ceph-mon[32192]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:45.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:45 smithi167 ceph-mon[36917]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:45.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:45 smithi094 ceph-mon[36708]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:48.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:47 smithi037 ceph-mon[32192]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:48.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:47 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:48.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:47 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:48.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:47 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:46:48.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:47 smithi167 ceph-mon[36917]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:48.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:47 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:48.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:47 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:48.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:47 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:46:48.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:47 smithi094 ceph-mon[36708]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:48.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:47 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:48.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:47 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:48.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:47 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:46:50.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:49 smithi037 ceph-mon[32192]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:50.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:49 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:50.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:49 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:50.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:49 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:50.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:49 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:46:50.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:49 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:50.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:49 smithi167 ceph-mon[36917]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:50.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:49 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:50.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:49 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:50.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:49 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:50.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:49 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:46:50.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:49 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:50.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:49 smithi094 ceph-mon[36708]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:50.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:49 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:50.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:49 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:50.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:49 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:50.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:49 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:46:50.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:49 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:50.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:50.505+0000 7f763effd700 1 -- 172.21.15.37:0/1253880030 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f76400feb90 con 0x7f7628039330 2024-06-10T00:46:50.506 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 0 on host 'smithi037' 2024-06-10T00:46:50.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:50.507+0000 7f7647f3d700 1 -- 172.21.15.37:0/1253880030 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7628039330 msgr2=0x7f762803b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:46:50.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:50.507+0000 7f7647f3d700 1 --2- 172.21.15.37:0/1253880030 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7628039330 0x7f762803b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7634000c00 tx=0x7f7634005fb0 comp rx=0 tx=0).stop 2024-06-10T00:46:50.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:50.507+0000 7f7647f3d700 1 -- 172.21.15.37:0/1253880030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76401013f0 msgr2=0x7f7640198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:46:50.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:50.507+0000 7f7647f3d700 1 --2- 172.21.15.37:0/1253880030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76401013f0 0x7f7640198830 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7630007ee0 tx=0x7f763000d3b0 comp rx=0 tx=0).stop 2024-06-10T00:46:50.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:50.508+0000 7f7647f3d700 1 -- 172.21.15.37:0/1253880030 shutdown_connections 2024-06-10T00:46:50.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:50.508+0000 7f7647f3d700 1 --2- 172.21.15.37:0/1253880030 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7628039330 0x7f762803b7f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:50.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:50.508+0000 7f7647f3d700 1 --2- 172.21.15.37:0/1253880030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7640102f90 0x7f764019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:50.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:50.508+0000 7f7647f3d700 1 --2- 172.21.15.37:0/1253880030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76401013f0 0x7f7640198830 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:50.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:50.508+0000 7f7647f3d700 1 --2- 172.21.15.37:0/1253880030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76401025f0 0x7f7640198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:50.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:50.508+0000 7f7647f3d700 1 -- 172.21.15.37:0/1253880030 >> 172.21.15.37:0/1253880030 conn(0x7f76400fc990 msgr2=0x7f76400fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:46:50.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:50.508+0000 7f7647f3d700 1 -- 172.21.15.37:0/1253880030 shutdown_connections 2024-06-10T00:46:50.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:46:50.508+0000 7f7647f3d700 1 -- 172.21.15.37:0/1253880030 wait complete. 2024-06-10T00:46:51.136 DEBUG:teuthology.orchestra.run.smithi037:osd.0> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.0.service 2024-06-10T00:46:51.138 INFO:tasks.cephadm:Deploying osd.1 on smithi094 with /dev/vg_nvme/lv_4... 2024-06-10T00:46:51.138 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-10T00:46:51.380 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T00:46:51.779 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:51 smithi094 ceph-mon[36708]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:51.779 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:51 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:51.779 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:51 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:51.780 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:51 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:46:51.780 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:51 smithi094 ceph-mon[36708]: from='osd.0 [v2:172.21.15.37:6802/3594704094,v1:172.21.15.37:6803/3594704094]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-10T00:46:51.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:51 smithi167 ceph-mon[36917]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:51.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:51 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:51.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:51 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:51.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:51 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:46:51.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:51 smithi167 ceph-mon[36917]: from='osd.0 [v2:172.21.15.37:6802/3594704094,v1:172.21.15.37:6803/3594704094]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-10T00:46:51.856 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:51 smithi037 ceph-mon[32192]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:51.857 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:51 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:51.857 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:51 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:51.857 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:51 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:46:51.857 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:51 smithi037 ceph-mon[32192]: from='osd.0 [v2:172.21.15.37:6802/3594704094,v1:172.21.15.37:6803/3594704094]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-10T00:46:52.671 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:52 smithi094 ceph-mon[36708]: from='osd.0 [v2:172.21.15.37:6802/3594704094,v1:172.21.15.37:6803/3594704094]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-10T00:46:52.672 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:52 smithi094 ceph-mon[36708]: osdmap e6: 1 total, 0 up, 1 in 2024-06-10T00:46:52.672 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:52 smithi094 ceph-mon[36708]: from='osd.0 [v2:172.21.15.37:6802/3594704094,v1:172.21.15.37:6803/3594704094]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-06-10T00:46:52.672 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:52 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:46:52.702 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:52 smithi037 ceph-mon[32192]: from='osd.0 [v2:172.21.15.37:6802/3594704094,v1:172.21.15.37:6803/3594704094]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-10T00:46:52.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:52 smithi037 ceph-mon[32192]: osdmap e6: 1 total, 0 up, 1 in 2024-06-10T00:46:52.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:52 smithi037 ceph-mon[32192]: from='osd.0 [v2:172.21.15.37:6802/3594704094,v1:172.21.15.37:6803/3594704094]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-06-10T00:46:52.703 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:52 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:46:52.703 INFO:journalctl@ceph.osd.0.smithi037.stdout:Jun 10 00:46:52 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-0[44549]: 2024-06-10T00:46:52.483+0000 7fdd76a3a700 -1 osd.0 0 waiting for initial osdmap 2024-06-10T00:46:52.703 INFO:journalctl@ceph.osd.0.smithi037.stdout:Jun 10 00:46:52 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-0[44549]: 2024-06-10T00:46:52.486+0000 7fdd7282d700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:46:52.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:52 smithi167 ceph-mon[36917]: from='osd.0 [v2:172.21.15.37:6802/3594704094,v1:172.21.15.37:6803/3594704094]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-10T00:46:52.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:52 smithi167 ceph-mon[36917]: osdmap e6: 1 total, 0 up, 1 in 2024-06-10T00:46:52.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:52 smithi167 ceph-mon[36917]: from='osd.0 [v2:172.21.15.37:6802/3594704094,v1:172.21.15.37:6803/3594704094]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-06-10T00:46:52.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:52 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:46:53.297 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-10T00:46:53.297 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-10T00:46:53.297 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-06-10T00:46:53.297 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-06-10T00:46:53.297 INFO:teuthology.orchestra.run.smithi094.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00868489 s, 1.2 GB/s 2024-06-10T00:46:53.297 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 2024-06-10T00:46:53.297 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-06-10T00:46:53.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:53 smithi037 ceph-mon[32192]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:53.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:53 smithi037 ceph-mon[32192]: from='osd.0 [v2:172.21.15.37:6802/3594704094,v1:172.21.15.37:6803/3594704094]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-06-10T00:46:53.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:53 smithi037 ceph-mon[32192]: osdmap e7: 1 total, 0 up, 1 in 2024-06-10T00:46:53.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:53 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:46:53.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:53 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:46:53.654 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:53 smithi094 ceph-mon[36708]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:53.654 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:53 smithi094 ceph-mon[36708]: from='osd.0 [v2:172.21.15.37:6802/3594704094,v1:172.21.15.37:6803/3594704094]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-06-10T00:46:53.654 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:53 smithi094 ceph-mon[36708]: osdmap e7: 1 total, 0 up, 1 in 2024-06-10T00:46:53.654 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:53 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:46:53.654 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:53 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:46:53.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:53 smithi167 ceph-mon[36917]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:53.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:53 smithi167 ceph-mon[36917]: from='osd.0 [v2:172.21.15.37:6802/3594704094,v1:172.21.15.37:6803/3594704094]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-06-10T00:46:53.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:53 smithi167 ceph-mon[36917]: osdmap e7: 1 total, 0 up, 1 in 2024-06-10T00:46:53.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:53 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:46:53.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:53 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:46:53.947 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_4 2024-06-10T00:46:54.186 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T00:46:54.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:54 smithi094 ceph-mon[36708]: purged_snaps scrub starts 2024-06-10T00:46:54.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:54 smithi094 ceph-mon[36708]: purged_snaps scrub ok 2024-06-10T00:46:54.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:54 smithi094 ceph-mon[36708]: osd.0 [v2:172.21.15.37:6802/3594704094,v1:172.21.15.37:6803/3594704094] boot 2024-06-10T00:46:54.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:54 smithi094 ceph-mon[36708]: osdmap e8: 1 total, 1 up, 1 in 2024-06-10T00:46:54.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:54 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:46:54.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:54 smithi094 ceph-mon[36708]: Detected new or changed devices on smithi037 2024-06-10T00:46:54.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:54 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:54.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:54 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:54.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:54 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:46:54.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:54 smithi094 ceph-mon[36708]: Adjusting osd_memory_target on smithi037 to 17092M 2024-06-10T00:46:54.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:54 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:54.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:54 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:54.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:54 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:46:54.616 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:54 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:54.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:54 smithi167 ceph-mon[36917]: purged_snaps scrub starts 2024-06-10T00:46:54.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:54 smithi167 ceph-mon[36917]: purged_snaps scrub ok 2024-06-10T00:46:54.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:54 smithi167 ceph-mon[36917]: osd.0 [v2:172.21.15.37:6802/3594704094,v1:172.21.15.37:6803/3594704094] boot 2024-06-10T00:46:54.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:54 smithi167 ceph-mon[36917]: osdmap e8: 1 total, 1 up, 1 in 2024-06-10T00:46:54.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:54 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:46:54.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:54 smithi167 ceph-mon[36917]: Detected new or changed devices on smithi037 2024-06-10T00:46:54.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:54 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:54.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:54 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:54.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:54 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:46:54.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:54 smithi167 ceph-mon[36917]: Adjusting osd_memory_target on smithi037 to 17092M 2024-06-10T00:46:54.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:54 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:54.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:54 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:54.840 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:54 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:46:54.840 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:54 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:54.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:54 smithi037 ceph-mon[32192]: purged_snaps scrub starts 2024-06-10T00:46:54.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:54 smithi037 ceph-mon[32192]: purged_snaps scrub ok 2024-06-10T00:46:54.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:54 smithi037 ceph-mon[32192]: osd.0 [v2:172.21.15.37:6802/3594704094,v1:172.21.15.37:6803/3594704094] boot 2024-06-10T00:46:54.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:54 smithi037 ceph-mon[32192]: osdmap e8: 1 total, 1 up, 1 in 2024-06-10T00:46:54.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:54 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-10T00:46:54.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:54 smithi037 ceph-mon[32192]: Detected new or changed devices on smithi037 2024-06-10T00:46:54.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:54 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:54.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:54 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:54.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:54 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:46:54.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:54 smithi037 ceph-mon[32192]: Adjusting osd_memory_target on smithi037 to 17092M 2024-06-10T00:46:54.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:54 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:54.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:54 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:54.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:54 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:46:54.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:54 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:46:55.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.699+0000 7f53971d4700 1 -- 172.21.15.94:0/793435602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5390102f90 msgr2=0x7f5390109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:46:55.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.699+0000 7f53971d4700 1 --2- 172.21.15.94:0/793435602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5390102f90 0x7f5390109840 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f538800b0d0 tx=0x7f538800b3e0 comp rx=0 tx=0).stop 2024-06-10T00:46:55.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.699+0000 7f53971d4700 1 -- 172.21.15.94:0/793435602 shutdown_connections 2024-06-10T00:46:55.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.699+0000 7f53971d4700 1 --2- 172.21.15.94:0/793435602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5390102f90 0x7f5390109840 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:55.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.699+0000 7f53971d4700 1 --2- 172.21.15.94:0/793435602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53901013f0 0x7f5390101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:55.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.699+0000 7f53971d4700 1 --2- 172.21.15.94:0/793435602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53901025f0 0x7f5390102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:55.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.699+0000 7f53971d4700 1 -- 172.21.15.94:0/793435602 >> 172.21.15.94:0/793435602 conn(0x7f53900fc9b0 msgr2=0x7f53900fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:46:55.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.699+0000 7f53971d4700 1 -- 172.21.15.94:0/793435602 shutdown_connections 2024-06-10T00:46:55.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.699+0000 7f53971d4700 1 -- 172.21.15.94:0/793435602 wait complete. 2024-06-10T00:46:55.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.699+0000 7f53971d4700 1 Processor -- start 2024-06-10T00:46:55.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.699+0000 7f53971d4700 1 -- start start 2024-06-10T00:46:55.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.699+0000 7f53971d4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53901013f0 0x7f539006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:46:55.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.699+0000 7f53971d4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53901025f0 0x7f539019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:46:55.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.699+0000 7f53971d4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5390102f90 0x7f539019fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:46:55.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.699+0000 7f53971d4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f539010d050 con 0x7f53901013f0 2024-06-10T00:46:55.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.699+0000 7f53971d4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f539010ced0 con 0x7f53901025f0 2024-06-10T00:46:55.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.699+0000 7f53971d4700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f539010d1d0 con 0x7f5390102f90 2024-06-10T00:46:55.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.699+0000 7f538ffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53901025f0 0x7f539019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:46:55.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.700+0000 7f538ffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53901025f0 0x7f539019d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52030/0 (socket says 172.21.15.94:52030) 2024-06-10T00:46:55.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.700+0000 7f538ffff700 1 -- 172.21.15.94:0/1958920723 learned_addr learned my addr 172.21.15.94:0/1958920723 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-10T00:46:55.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.700+0000 7f5394f70700 1 --2- 172.21.15.94:0/1958920723 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53901013f0 0x7f539006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:46:55.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.700+0000 7f5395771700 1 --2- 172.21.15.94:0/1958920723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5390102f90 0x7f539019fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:46:55.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.700+0000 7f538ffff700 1 -- 172.21.15.94:0/1958920723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5390102f90 msgr2=0x7f539019fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:46:55.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.700+0000 7f538ffff700 1 --2- 172.21.15.94:0/1958920723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5390102f90 0x7f539019fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:55.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.700+0000 7f538ffff700 1 -- 172.21.15.94:0/1958920723 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53901013f0 msgr2=0x7f539006b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:46:55.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.700+0000 7f538ffff700 1 --2- 172.21.15.94:0/1958920723 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53901013f0 0x7f539006b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:46:55.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.700+0000 7f538ffff700 1 -- 172.21.15.94:0/1958920723 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53901a0350 con 0x7f53901025f0 2024-06-10T00:46:55.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.700+0000 7f538ffff700 1 --2- 172.21.15.94:0/1958920723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53901025f0 0x7f539019d440 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f537c00b530 tx=0x7f537c00b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:46:55.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.700+0000 7f538dffb700 1 -- 172.21.15.94:0/1958920723 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537c015020 con 0x7f53901025f0 2024-06-10T00:46:55.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.700+0000 7f538dffb700 1 -- 172.21.15.94:0/1958920723 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f537c011d10 con 0x7f53901025f0 2024-06-10T00:46:55.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.700+0000 7f538dffb700 1 -- 172.21.15.94:0/1958920723 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537c00f2e0 con 0x7f53901025f0 2024-06-10T00:46:55.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.700+0000 7f53971d4700 1 -- 172.21.15.94:0/1958920723 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53901a0640 con 0x7f53901025f0 2024-06-10T00:46:55.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.700+0000 7f53971d4700 1 -- 172.21.15.94:0/1958920723 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53901a0c50 con 0x7f53901025f0 2024-06-10T00:46:55.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.701+0000 7f53971d4700 1 -- 172.21.15.94:0/1958920723 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5390069180 con 0x7f53901025f0 2024-06-10T00:46:55.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.702+0000 7f538dffb700 1 -- 172.21.15.94:0/1958920723 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f537c018020 con 0x7f53901025f0 2024-06-10T00:46:55.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.703+0000 7f538dffb700 1 --2- 172.21.15.94:0/1958920723 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f53800a2c10 0x7f53800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:46:55.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.703+0000 7f538dffb700 1 -- 172.21.15.94:0/1958920723 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f537c0ccdc0 con 0x7f53901025f0 2024-06-10T00:46:55.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.703+0000 7f5394f70700 1 --2- 172.21.15.94:0/1958920723 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f53800a2c10 0x7f53800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:46:55.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.704+0000 7f5394f70700 1 --2- 172.21.15.94:0/1958920723 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f53800a2c10 0x7f53800a50d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5390102450 tx=0x7f538400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:46:55.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.708+0000 7f538dffb700 1 -- 172.21.15.94:0/1958920723 <== mon.2 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f537c01f020 con 0x7f53901025f0 2024-06-10T00:46:55.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:55 smithi167 ceph-mon[36917]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:55.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:55 smithi167 ceph-mon[36917]: osdmap e9: 1 total, 1 up, 1 in 2024-06-10T00:46:55.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:55 smithi094 ceph-mon[36708]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:55.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:55 smithi094 ceph-mon[36708]: osdmap e9: 1 total, 1 up, 1 in 2024-06-10T00:46:55.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:55 smithi037 ceph-mon[32192]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-10T00:46:55.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:55 smithi037 ceph-mon[32192]: osdmap e9: 1 total, 1 up, 1 in 2024-06-10T00:46:55.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:46:55.911+0000 7f53971d4700 1 -- 172.21.15.94:0/1958920723 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f53900628d0 con 0x7f53800a2c10 2024-06-10T00:46:56.816 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:56 smithi094 ceph-mon[36708]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:46:56.817 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:56 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:46:56.817 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:56 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:46:56.817 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:56 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:56.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:56 smithi167 ceph-mon[36917]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:46:56.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:56 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:46:56.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:56 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:46:56.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:56 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:56.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:56 smithi037 ceph-mon[32192]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:46:56.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:56 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:46:56.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:56 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:46:56.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:56 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:46:57.687 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:57 smithi094 ceph-mon[36708]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:46:57.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:57 smithi167 ceph-mon[36917]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:46:57.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:57 smithi037 ceph-mon[32192]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:46:59.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:59 smithi167 ceph-mon[36917]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:46:59.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:59 smithi167 ceph-mon[36917]: from='client.? 172.21.15.94:0/2864565293' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e1c8944-94e4-4e46-9efe-56229fe6df1a"}]: dispatch 2024-06-10T00:46:59.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:59 smithi167 ceph-mon[36917]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e1c8944-94e4-4e46-9efe-56229fe6df1a"}]: dispatch 2024-06-10T00:46:59.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:59 smithi167 ceph-mon[36917]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e1c8944-94e4-4e46-9efe-56229fe6df1a"}]': finished 2024-06-10T00:46:59.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:59 smithi167 ceph-mon[36917]: osdmap e10: 2 total, 1 up, 2 in 2024-06-10T00:46:59.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:59 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:46:59.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:46:59 smithi167 ceph-mon[36917]: from='client.? 172.21.15.94:0/1472624720' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:46:59.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:59 smithi094 ceph-mon[36708]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:46:59.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:59 smithi094 ceph-mon[36708]: from='client.? 172.21.15.94:0/2864565293' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e1c8944-94e4-4e46-9efe-56229fe6df1a"}]: dispatch 2024-06-10T00:46:59.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:59 smithi094 ceph-mon[36708]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e1c8944-94e4-4e46-9efe-56229fe6df1a"}]: dispatch 2024-06-10T00:46:59.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:59 smithi094 ceph-mon[36708]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e1c8944-94e4-4e46-9efe-56229fe6df1a"}]': finished 2024-06-10T00:46:59.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:59 smithi094 ceph-mon[36708]: osdmap e10: 2 total, 1 up, 2 in 2024-06-10T00:46:59.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:59 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:46:59.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:46:59 smithi094 ceph-mon[36708]: from='client.? 172.21.15.94:0/1472624720' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:46:59.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:59 smithi037 ceph-mon[32192]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:46:59.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:59 smithi037 ceph-mon[32192]: from='client.? 172.21.15.94:0/2864565293' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e1c8944-94e4-4e46-9efe-56229fe6df1a"}]: dispatch 2024-06-10T00:46:59.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:59 smithi037 ceph-mon[32192]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e1c8944-94e4-4e46-9efe-56229fe6df1a"}]: dispatch 2024-06-10T00:46:59.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:59 smithi037 ceph-mon[32192]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e1c8944-94e4-4e46-9efe-56229fe6df1a"}]': finished 2024-06-10T00:46:59.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:59 smithi037 ceph-mon[32192]: osdmap e10: 2 total, 1 up, 2 in 2024-06-10T00:46:59.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:59 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:46:59.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:46:59 smithi037 ceph-mon[32192]: from='client.? 172.21.15.94:0/1472624720' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:47:01.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:01 smithi167 ceph-mon[36917]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:01.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:01 smithi094 ceph-mon[36708]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:01.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:01 smithi037 ceph-mon[32192]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:03.705 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:03 smithi094 ceph-mon[36708]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:03.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:03 smithi167 ceph-mon[36917]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:03.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:03 smithi037 ceph-mon[32192]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:05.656 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:05 smithi094 ceph-mon[36708]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:05.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:05 smithi167 ceph-mon[36917]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:05.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:05 smithi037 ceph-mon[32192]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:06.601 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:06 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-10T00:47:06.601 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:06 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:47:06.601 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:06 smithi094 ceph-mon[36708]: Deploying daemon osd.1 on smithi094 2024-06-10T00:47:06.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:06 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-10T00:47:06.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:06 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:47:06.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:06 smithi167 ceph-mon[36917]: Deploying daemon osd.1 on smithi094 2024-06-10T00:47:06.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:06 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-10T00:47:06.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:06 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:47:06.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:06 smithi037 ceph-mon[32192]: Deploying daemon osd.1 on smithi094 2024-06-10T00:47:07.775 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:07 smithi094 ceph-mon[36708]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:07.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:07 smithi167 ceph-mon[36917]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:07.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:07 smithi037 ceph-mon[32192]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:09.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:09 smithi094 ceph-mon[36708]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:09.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:09 smithi167 ceph-mon[36917]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:09.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:09 smithi037 ceph-mon[32192]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:12.219 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:11 smithi094 ceph-mon[36708]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:12.219 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:11 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:12.219 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:11 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:12.219 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:11 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:47:12.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:11 smithi167 ceph-mon[36917]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:12.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:11 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:12.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:11 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:12.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:11 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:47:12.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:11 smithi037 ceph-mon[32192]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:12.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:11 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:12.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:11 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:12.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:11 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:47:14.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:13 smithi094 ceph-mon[36708]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:14.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:13 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:14.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:13 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:14.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:13 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:47:14.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:13 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:47:14.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:13 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:14.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:13 smithi167 ceph-mon[36917]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:14.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:13 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:14.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:13 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:14.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:13 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:47:14.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:13 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:47:14.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:13 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:14.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:13 smithi037 ceph-mon[32192]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:14.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:13 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:14.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:13 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:14.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:13 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:47:14.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:13 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:47:14.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:13 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:14.481 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 1 on host 'smithi094' 2024-06-10T00:47:14.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:47:14.481+0000 7f538dffb700 1 -- 172.21.15.94:0/1958920723 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f53900628d0 con 0x7f53800a2c10 2024-06-10T00:47:14.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:47:14.483+0000 7f53971d4700 1 -- 172.21.15.94:0/1958920723 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f53800a2c10 msgr2=0x7f53800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:47:14.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:47:14.483+0000 7f53971d4700 1 --2- 172.21.15.94:0/1958920723 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f53800a2c10 0x7f53800a50d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5390102450 tx=0x7f538400b410 comp rx=0 tx=0).stop 2024-06-10T00:47:14.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:47:14.483+0000 7f53971d4700 1 -- 172.21.15.94:0/1958920723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53901025f0 msgr2=0x7f539019d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:47:14.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:47:14.483+0000 7f53971d4700 1 --2- 172.21.15.94:0/1958920723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53901025f0 0x7f539019d440 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f537c00b530 tx=0x7f537c00b840 comp rx=0 tx=0).stop 2024-06-10T00:47:14.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:47:14.483+0000 7f53971d4700 1 -- 172.21.15.94:0/1958920723 shutdown_connections 2024-06-10T00:47:14.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:47:14.483+0000 7f53971d4700 1 --2- 172.21.15.94:0/1958920723 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f53800a2c10 0x7f53800a50d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:47:14.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:47:14.483+0000 7f53971d4700 1 --2- 172.21.15.94:0/1958920723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5390102f90 0x7f539019fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:47:14.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:47:14.483+0000 7f53971d4700 1 --2- 172.21.15.94:0/1958920723 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53901013f0 0x7f539006b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:47:14.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:47:14.483+0000 7f53971d4700 1 --2- 172.21.15.94:0/1958920723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53901025f0 0x7f539019d440 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:47:14.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:47:14.483+0000 7f53971d4700 1 -- 172.21.15.94:0/1958920723 >> 172.21.15.94:0/1958920723 conn(0x7f53900fc9b0 msgr2=0x7f5390105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:47:14.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:47:14.484+0000 7f53971d4700 1 -- 172.21.15.94:0/1958920723 shutdown_connections 2024-06-10T00:47:14.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:47:14.484+0000 7f53971d4700 1 -- 172.21.15.94:0/1958920723 wait complete. 2024-06-10T00:47:15.192 DEBUG:teuthology.orchestra.run.smithi094:osd.1> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.1.service 2024-06-10T00:47:15.195 INFO:tasks.cephadm:Deploying osd.2 on smithi167 with /dev/vg_nvme/lv_4... 2024-06-10T00:47:15.196 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-10T00:47:15.442 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:47:15.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:15 smithi167 ceph-mon[36917]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:15.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:15 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:15.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:15 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:15.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:15 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:47:15.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:15 smithi167 ceph-mon[36917]: from='osd.1 [v2:172.21.15.94:6800/3427727155,v1:172.21.15.94:6801/3427727155]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-10T00:47:15.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:15 smithi167 ceph-mon[36917]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-10T00:47:15.743 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:15 smithi094 ceph-mon[36708]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:15.743 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:15 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:15.744 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:15 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:15.744 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:15 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:47:15.744 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:15 smithi094 ceph-mon[36708]: from='osd.1 [v2:172.21.15.94:6800/3427727155,v1:172.21.15.94:6801/3427727155]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-10T00:47:15.744 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:15 smithi094 ceph-mon[36708]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-10T00:47:15.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:15 smithi037 ceph-mon[32192]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:15.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:15 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:15.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:15 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:15.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:15 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:47:15.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:15 smithi037 ceph-mon[32192]: from='osd.1 [v2:172.21.15.94:6800/3427727155,v1:172.21.15.94:6801/3427727155]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-10T00:47:15.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:15 smithi037 ceph-mon[32192]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-10T00:47:16.722 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:16 smithi094 ceph-mon[36708]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-10T00:47:16.723 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:16 smithi094 ceph-mon[36708]: from='osd.1 [v2:172.21.15.94:6800/3427727155,v1:172.21.15.94:6801/3427727155]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-10T00:47:16.723 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:16 smithi094 ceph-mon[36708]: osdmap e11: 2 total, 1 up, 2 in 2024-06-10T00:47:16.723 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:16 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:47:16.723 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:16 smithi094 ceph-mon[36708]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-10T00:47:16.734 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:16 smithi167 ceph-mon[36917]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-10T00:47:16.734 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:16 smithi167 ceph-mon[36917]: from='osd.1 [v2:172.21.15.94:6800/3427727155,v1:172.21.15.94:6801/3427727155]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-10T00:47:16.735 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:16 smithi167 ceph-mon[36917]: osdmap e11: 2 total, 1 up, 2 in 2024-06-10T00:47:16.735 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:16 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:47:16.735 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:16 smithi167 ceph-mon[36917]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-10T00:47:16.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:16 smithi037 ceph-mon[32192]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-10T00:47:16.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:16 smithi037 ceph-mon[32192]: from='osd.1 [v2:172.21.15.94:6800/3427727155,v1:172.21.15.94:6801/3427727155]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-10T00:47:16.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:16 smithi037 ceph-mon[32192]: osdmap e11: 2 total, 1 up, 2 in 2024-06-10T00:47:16.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:16 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:47:16.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:16 smithi037 ceph-mon[32192]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-10T00:47:17.114 INFO:journalctl@ceph.osd.1.smithi094.stdout:Jun 10 00:47:16 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-1[41257]: 2024-06-10T00:47:16.924+0000 7f828ed15700 -1 osd.1 0 waiting for initial osdmap 2024-06-10T00:47:17.115 INFO:journalctl@ceph.osd.1.smithi094.stdout:Jun 10 00:47:16 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-1[41257]: 2024-06-10T00:47:16.927+0000 7f8286b00700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:47:17.338 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-10T00:47:17.338 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-10T00:47:17.338 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-06-10T00:47:17.338 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-06-10T00:47:17.338 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124855 s, 840 MB/s 2024-06-10T00:47:17.338 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-06-10T00:47:17.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:17 smithi167 ceph-mon[36917]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:17.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:17 smithi167 ceph-mon[36917]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-06-10T00:47:17.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:17 smithi167 ceph-mon[36917]: osdmap e12: 2 total, 1 up, 2 in 2024-06-10T00:47:17.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:17 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:47:17.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:17 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:47:17.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:17 smithi167 ceph-mon[36917]: from='osd.1 ' entity='osd.1' 2024-06-10T00:47:17.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:17 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:47:17.853 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:17 smithi094 ceph-mon[36708]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:17.853 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:17 smithi094 ceph-mon[36708]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-06-10T00:47:17.853 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:17 smithi094 ceph-mon[36708]: osdmap e12: 2 total, 1 up, 2 in 2024-06-10T00:47:17.854 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:17 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:47:17.854 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:17 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:47:17.854 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:17 smithi094 ceph-mon[36708]: from='osd.1 ' entity='osd.1' 2024-06-10T00:47:17.854 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:17 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:47:17.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:17 smithi037 ceph-mon[32192]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:17.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:17 smithi037 ceph-mon[32192]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-06-10T00:47:17.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:17 smithi037 ceph-mon[32192]: osdmap e12: 2 total, 1 up, 2 in 2024-06-10T00:47:17.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:17 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:47:17.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:17 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:47:17.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:17 smithi037 ceph-mon[32192]: from='osd.1 ' entity='osd.1' 2024-06-10T00:47:17.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:17 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:47:18.009 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_4 2024-06-10T00:47:18.247 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:47:18.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:18 smithi167 ceph-mon[36917]: purged_snaps scrub starts 2024-06-10T00:47:18.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:18 smithi167 ceph-mon[36917]: purged_snaps scrub ok 2024-06-10T00:47:18.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:18 smithi167 ceph-mon[36917]: osd.1 [v2:172.21.15.94:6800/3427727155,v1:172.21.15.94:6801/3427727155] boot 2024-06-10T00:47:18.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:18 smithi167 ceph-mon[36917]: osdmap e13: 2 total, 2 up, 2 in 2024-06-10T00:47:18.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:18 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:47:18.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:18 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:18.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:18 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:18.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:18 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:47:18.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:18 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:18.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:18 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:47:18.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:18 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:47:18.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:18 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:18.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:18 smithi094 ceph-mon[36708]: purged_snaps scrub starts 2024-06-10T00:47:18.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:18 smithi094 ceph-mon[36708]: purged_snaps scrub ok 2024-06-10T00:47:18.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:18 smithi094 ceph-mon[36708]: osd.1 [v2:172.21.15.94:6800/3427727155,v1:172.21.15.94:6801/3427727155] boot 2024-06-10T00:47:18.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:18 smithi094 ceph-mon[36708]: osdmap e13: 2 total, 2 up, 2 in 2024-06-10T00:47:18.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:18 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:47:18.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:18 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:18.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:18 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:18.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:18 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:47:18.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:18 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:18.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:18 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:47:18.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:18 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:47:18.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:18 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:18.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:18 smithi037 ceph-mon[32192]: purged_snaps scrub starts 2024-06-10T00:47:18.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:18 smithi037 ceph-mon[32192]: purged_snaps scrub ok 2024-06-10T00:47:18.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:18 smithi037 ceph-mon[32192]: osd.1 [v2:172.21.15.94:6800/3427727155,v1:172.21.15.94:6801/3427727155] boot 2024-06-10T00:47:18.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:18 smithi037 ceph-mon[32192]: osdmap e13: 2 total, 2 up, 2 in 2024-06-10T00:47:18.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:18 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-10T00:47:18.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:18 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:18.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:18 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:18.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:18 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:47:18.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:18 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:18.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:18 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:47:18.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:18 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:47:18.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:18 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:19.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:19 smithi167 ceph-mon[36917]: Detected new or changed devices on smithi094 2024-06-10T00:47:19.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:19 smithi167 ceph-mon[36917]: Adjusting osd_memory_target on smithi094 to 17088M 2024-06-10T00:47:19.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:19 smithi167 ceph-mon[36917]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:19.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.785+0000 7fe72520c700 1 -- 172.21.15.167:0/1198602187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe720101690 msgr2=0x7fe720101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:47:19.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.785+0000 7fe72520c700 1 --2- 172.21.15.167:0/1198602187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe720101690 0x7fe720101af0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe714009990 tx=0x7fe714009ca0 comp rx=0 tx=0).stop 2024-06-10T00:47:19.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.785+0000 7fe72520c700 1 -- 172.21.15.167:0/1198602187 shutdown_connections 2024-06-10T00:47:19.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.785+0000 7fe72520c700 1 --2- 172.21.15.167:0/1198602187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe720102030 0x7fe7201098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:47:19.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.785+0000 7fe72520c700 1 --2- 172.21.15.167:0/1198602187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7201029e0 0x7fe720102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:47:19.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.785+0000 7fe72520c700 1 --2- 172.21.15.167:0/1198602187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe720101690 0x7fe720101af0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:47:19.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.785+0000 7fe72520c700 1 -- 172.21.15.167:0/1198602187 >> 172.21.15.167:0/1198602187 conn(0x7fe7200fc100 msgr2=0x7fe7200fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:47:19.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.785+0000 7fe72520c700 1 -- 172.21.15.167:0/1198602187 shutdown_connections 2024-06-10T00:47:19.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.785+0000 7fe72520c700 1 -- 172.21.15.167:0/1198602187 wait complete. 2024-06-10T00:47:19.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.786+0000 7fe72520c700 1 Processor -- start 2024-06-10T00:47:19.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.786+0000 7fe72520c700 1 -- start start 2024-06-10T00:47:19.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.787+0000 7fe72520c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe720101690 0x7fe720101100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:47:19.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.787+0000 7fe72520c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe720102030 0x7fe72019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:47:19.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.787+0000 7fe72520c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7201029e0 0x7fe72019fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:47:19.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.787+0000 7fe72520c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe72010d030 con 0x7fe720102030 2024-06-10T00:47:19.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.787+0000 7fe72520c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe72010ceb0 con 0x7fe720101690 2024-06-10T00:47:19.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.787+0000 7fe72520c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe72010d1b0 con 0x7fe7201029e0 2024-06-10T00:47:19.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.787+0000 7fe71f59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7201029e0 0x7fe72019fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:47:19.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.787+0000 7fe71f59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7201029e0 0x7fe72019fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:40870/0 (socket says 172.21.15.167:40870) 2024-06-10T00:47:19.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.787+0000 7fe71f59e700 1 -- 172.21.15.167:0/2487160882 learned_addr learned my addr 172.21.15.167:0/2487160882 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-10T00:47:19.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.787+0000 7fe71ed9d700 1 --2- 172.21.15.167:0/2487160882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe720101690 0x7fe720101100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:47:19.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.787+0000 7fe71e59c700 1 --2- 172.21.15.167:0/2487160882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe720102030 0x7fe72019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:47:19.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.787+0000 7fe71f59e700 1 -- 172.21.15.167:0/2487160882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe720101690 msgr2=0x7fe720101100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:47:19.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.787+0000 7fe71f59e700 1 --2- 172.21.15.167:0/2487160882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe720101690 0x7fe720101100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:47:19.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.787+0000 7fe71f59e700 1 -- 172.21.15.167:0/2487160882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe720102030 msgr2=0x7fe72019d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:47:19.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.787+0000 7fe71f59e700 1 --2- 172.21.15.167:0/2487160882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe720102030 0x7fe72019d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:47:19.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.787+0000 7fe71f59e700 1 -- 172.21.15.167:0/2487160882 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7201a0380 con 0x7fe7201029e0 2024-06-10T00:47:19.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.787+0000 7fe71e59c700 1 --2- 172.21.15.167:0/2487160882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe720102030 0x7fe72019d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:47:19.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.787+0000 7fe71f59e700 1 --2- 172.21.15.167:0/2487160882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7201029e0 0x7fe72019fab0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe718007ee0 tx=0x7fe71800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:47:19.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.787+0000 7fe707fff700 1 -- 172.21.15.167:0/2487160882 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71800dbd0 con 0x7fe7201029e0 2024-06-10T00:47:19.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.788+0000 7fe707fff700 1 -- 172.21.15.167:0/2487160882 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe71800dd70 con 0x7fe7201029e0 2024-06-10T00:47:19.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.788+0000 7fe72520c700 1 -- 172.21.15.167:0/2487160882 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7201a0670 con 0x7fe7201029e0 2024-06-10T00:47:19.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.788+0000 7fe707fff700 1 -- 172.21.15.167:0/2487160882 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe718005a60 con 0x7fe7201029e0 2024-06-10T00:47:19.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.788+0000 7fe72520c700 1 -- 172.21.15.167:0/2487160882 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7201a0bb0 con 0x7fe7201029e0 2024-06-10T00:47:19.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.788+0000 7fe72520c700 1 -- 172.21.15.167:0/2487160882 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe720069180 con 0x7fe7201029e0 2024-06-10T00:47:19.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.790+0000 7fe707fff700 1 -- 172.21.15.167:0/2487160882 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fe71801a020 con 0x7fe7201029e0 2024-06-10T00:47:19.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.790+0000 7fe707fff700 1 --2- 172.21.15.167:0/2487160882 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe7080a2c10 0x7fe7080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:47:19.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.790+0000 7fe707fff700 1 -- 172.21.15.167:0/2487160882 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fe7180cd500 con 0x7fe7201029e0 2024-06-10T00:47:19.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.791+0000 7fe71ed9d700 1 --2- 172.21.15.167:0/2487160882 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe7080a2c10 0x7fe7080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:47:19.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.791+0000 7fe71ed9d700 1 --2- 172.21.15.167:0/2487160882 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe7080a2c10 0x7fe7080a50d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe72006aa00 tx=0x7fe71000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:47:19.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:19.794+0000 7fe707fff700 1 -- 172.21.15.167:0/2487160882 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe718010070 con 0x7fe7201029e0 2024-06-10T00:47:19.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:19 smithi094 ceph-mon[36708]: Detected new or changed devices on smithi094 2024-06-10T00:47:19.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:19 smithi094 ceph-mon[36708]: Adjusting osd_memory_target on smithi094 to 17088M 2024-06-10T00:47:19.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:19 smithi094 ceph-mon[36708]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:19.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:19 smithi037 ceph-mon[32192]: Detected new or changed devices on smithi094 2024-06-10T00:47:19.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:19 smithi037 ceph-mon[32192]: Adjusting osd_memory_target on smithi094 to 17088M 2024-06-10T00:47:19.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:19 smithi037 ceph-mon[32192]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-06-10T00:47:20.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:20.004+0000 7fe72520c700 1 -- 172.21.15.167:0/2487160882 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe7200628d0 con 0x7fe7080a2c10 2024-06-10T00:47:20.766 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:20 smithi167 ceph-mon[36917]: osdmap e14: 2 total, 2 up, 2 in 2024-06-10T00:47:20.766 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:20 smithi167 ceph-mon[36917]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:47:20.766 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:20 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:47:20.766 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:20 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:47:20.766 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:20 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:47:20.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:20 smithi094 ceph-mon[36708]: osdmap e14: 2 total, 2 up, 2 in 2024-06-10T00:47:20.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:20 smithi094 ceph-mon[36708]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:47:20.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:20 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:47:20.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:20 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:47:20.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:20 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:47:20.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:20 smithi037 ceph-mon[32192]: osdmap e14: 2 total, 2 up, 2 in 2024-06-10T00:47:20.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:20 smithi037 ceph-mon[32192]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:47:20.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:20 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:47:20.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:20 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:47:20.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:20 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:47:21.752 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:21 smithi167 ceph-mon[36917]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:21.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:21 smithi094 ceph-mon[36708]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:21.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:21 smithi037 ceph-mon[32192]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:23.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:23 smithi167 ceph-mon[36917]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:23.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:23 smithi167 ceph-mon[36917]: from='client.? 172.21.15.167:0/4012567296' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd2fcedc-4c88-478f-8025-13e5944a27ea"}]: dispatch 2024-06-10T00:47:23.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:23 smithi167 ceph-mon[36917]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd2fcedc-4c88-478f-8025-13e5944a27ea"}]: dispatch 2024-06-10T00:47:23.843 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:23 smithi167 ceph-mon[36917]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd2fcedc-4c88-478f-8025-13e5944a27ea"}]': finished 2024-06-10T00:47:23.843 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:23 smithi167 ceph-mon[36917]: osdmap e15: 3 total, 2 up, 3 in 2024-06-10T00:47:23.843 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:23 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:47:23.843 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:23 smithi167 ceph-mon[36917]: from='client.? 172.21.15.167:0/3428235932' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:47:23.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:23 smithi094 ceph-mon[36708]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:23.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:23 smithi094 ceph-mon[36708]: from='client.? 172.21.15.167:0/4012567296' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd2fcedc-4c88-478f-8025-13e5944a27ea"}]: dispatch 2024-06-10T00:47:23.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:23 smithi094 ceph-mon[36708]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd2fcedc-4c88-478f-8025-13e5944a27ea"}]: dispatch 2024-06-10T00:47:23.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:23 smithi094 ceph-mon[36708]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd2fcedc-4c88-478f-8025-13e5944a27ea"}]': finished 2024-06-10T00:47:23.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:23 smithi094 ceph-mon[36708]: osdmap e15: 3 total, 2 up, 3 in 2024-06-10T00:47:23.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:23 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:47:23.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:23 smithi094 ceph-mon[36708]: from='client.? 172.21.15.167:0/3428235932' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:47:23.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:23 smithi037 ceph-mon[32192]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:23.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:23 smithi037 ceph-mon[32192]: from='client.? 172.21.15.167:0/4012567296' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd2fcedc-4c88-478f-8025-13e5944a27ea"}]: dispatch 2024-06-10T00:47:23.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:23 smithi037 ceph-mon[32192]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd2fcedc-4c88-478f-8025-13e5944a27ea"}]: dispatch 2024-06-10T00:47:23.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:23 smithi037 ceph-mon[32192]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd2fcedc-4c88-478f-8025-13e5944a27ea"}]': finished 2024-06-10T00:47:23.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:23 smithi037 ceph-mon[32192]: osdmap e15: 3 total, 2 up, 3 in 2024-06-10T00:47:23.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:23 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:47:23.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:23 smithi037 ceph-mon[32192]: from='client.? 172.21.15.167:0/3428235932' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:47:25.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:25 smithi167 ceph-mon[36917]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:25.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:25 smithi094 ceph-mon[36708]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:25.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:25 smithi037 ceph-mon[32192]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:27.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:27 smithi167 ceph-mon[36917]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:27.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:27 smithi094 ceph-mon[36708]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:27.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:27 smithi037 ceph-mon[32192]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:29.703 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:29 smithi167 ceph-mon[36917]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:29.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:29 smithi094 ceph-mon[36708]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:29.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:29 smithi037 ceph-mon[32192]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:30.697 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:30 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-10T00:47:30.697 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:30 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:47:30.697 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:30 smithi167 ceph-mon[36917]: Deploying daemon osd.2 on smithi167 2024-06-10T00:47:30.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:30 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-10T00:47:30.860 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:30 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:47:30.860 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:30 smithi094 ceph-mon[36708]: Deploying daemon osd.2 on smithi167 2024-06-10T00:47:30.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:30 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-10T00:47:30.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:30 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:47:30.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:30 smithi037 ceph-mon[32192]: Deploying daemon osd.2 on smithi167 2024-06-10T00:47:31.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:31 smithi167 ceph-mon[36917]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:31.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:31 smithi094 ceph-mon[36708]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:31.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:31 smithi037 ceph-mon[32192]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:33.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:33 smithi167 ceph-mon[36917]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:33.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:33 smithi094 ceph-mon[36708]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:33.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:33 smithi037 ceph-mon[32192]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:35 smithi167 ceph-mon[36917]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:35 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:35 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:35 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:47:35.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:35 smithi094 ceph-mon[36708]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:35.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:35 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:35.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:35 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:35.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:35 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:47:35.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:35 smithi037 ceph-mon[32192]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:35.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:35 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:35.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:35 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:35.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:35 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:47:37.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:37 smithi167 ceph-mon[36917]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:37.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:37 smithi094 ceph-mon[36708]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:37.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:37 smithi037 ceph-mon[32192]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:38.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:38 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:38.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:38 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:38.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:38 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:47:38.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:38 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:47:38.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:38 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:38.909 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:38 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:38.909 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:38 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:38.909 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:38 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:47:38.909 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:38 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:47:38.909 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:38 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:38.974 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 2 on host 'smithi167' 2024-06-10T00:47:38.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:38.973+0000 7fe707fff700 1 -- 172.21.15.167:0/2487160882 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe7200628d0 con 0x7fe7080a2c10 2024-06-10T00:47:38.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:38.975+0000 7fe72520c700 1 -- 172.21.15.167:0/2487160882 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe7080a2c10 msgr2=0x7fe7080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:47:38.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:38.975+0000 7fe72520c700 1 --2- 172.21.15.167:0/2487160882 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe7080a2c10 0x7fe7080a50d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe72006aa00 tx=0x7fe71000b410 comp rx=0 tx=0).stop 2024-06-10T00:47:38.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:38.975+0000 7fe72520c700 1 -- 172.21.15.167:0/2487160882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7201029e0 msgr2=0x7fe72019fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:47:38.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:38.975+0000 7fe72520c700 1 --2- 172.21.15.167:0/2487160882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7201029e0 0x7fe72019fab0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe718007ee0 tx=0x7fe71800d3b0 comp rx=0 tx=0).stop 2024-06-10T00:47:38.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:38.975+0000 7fe72520c700 1 -- 172.21.15.167:0/2487160882 shutdown_connections 2024-06-10T00:47:38.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:38.975+0000 7fe72520c700 1 --2- 172.21.15.167:0/2487160882 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe7080a2c10 0x7fe7080a50d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:47:38.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:38.975+0000 7fe72520c700 1 --2- 172.21.15.167:0/2487160882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7201029e0 0x7fe72019fab0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:47:38.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:38.975+0000 7fe72520c700 1 --2- 172.21.15.167:0/2487160882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe720101690 0x7fe720101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:47:38.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:38.975+0000 7fe72520c700 1 --2- 172.21.15.167:0/2487160882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe720102030 0x7fe72019d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:47:38.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:38.975+0000 7fe72520c700 1 -- 172.21.15.167:0/2487160882 >> 172.21.15.167:0/2487160882 conn(0x7fe7200fc100 msgr2=0x7fe7201059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:47:38.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:38.976+0000 7fe72520c700 1 -- 172.21.15.167:0/2487160882 shutdown_connections 2024-06-10T00:47:38.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:47:38.976+0000 7fe72520c700 1 -- 172.21.15.167:0/2487160882 wait complete. 2024-06-10T00:47:39.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:38 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:39.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:38 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:39.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:38 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:47:39.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:38 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:47:39.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:38 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:39.689 DEBUG:teuthology.orchestra.run.smithi167:osd.2> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.2.service 2024-06-10T00:47:39.692 INFO:tasks.cephadm:Deploying osd.3 on smithi037 with /dev/vg_nvme/lv_3... 2024-06-10T00:47:39.692 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-10T00:47:39.950 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:47:40.073 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:39 smithi037 ceph-mon[32192]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:40.073 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:39 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:40.073 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:39 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:40.073 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:39 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:47:40.073 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:39 smithi037 ceph-mon[32192]: from='osd.2 [v2:172.21.15.167:6800/4002498292,v1:172.21.15.167:6801/4002498292]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-10T00:47:40.073 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:39 smithi037 ceph-mon[32192]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-10T00:47:40.192 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:39 smithi167 ceph-mon[36917]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:40.192 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:39 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:40.192 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:39 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:40.192 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:39 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:47:40.192 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:39 smithi167 ceph-mon[36917]: from='osd.2 [v2:172.21.15.167:6800/4002498292,v1:172.21.15.167:6801/4002498292]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-10T00:47:40.192 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:39 smithi167 ceph-mon[36917]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-10T00:47:40.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:39 smithi094 ceph-mon[36708]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:40.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:39 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:40.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:39 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:40.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:39 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:47:40.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:39 smithi094 ceph-mon[36708]: from='osd.2 [v2:172.21.15.167:6800/4002498292,v1:172.21.15.167:6801/4002498292]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-10T00:47:40.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:39 smithi094 ceph-mon[36708]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-10T00:47:41.250 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:40 smithi037 ceph-mon[32192]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-10T00:47:41.250 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:40 smithi037 ceph-mon[32192]: osdmap e16: 3 total, 2 up, 3 in 2024-06-10T00:47:41.251 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:40 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:47:41.251 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:40 smithi037 ceph-mon[32192]: from='osd.2 [v2:172.21.15.167:6800/4002498292,v1:172.21.15.167:6801/4002498292]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-10T00:47:41.251 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:40 smithi037 ceph-mon[32192]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-10T00:47:41.266 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:40 smithi167 ceph-mon[36917]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-10T00:47:41.266 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:40 smithi167 ceph-mon[36917]: osdmap e16: 3 total, 2 up, 3 in 2024-06-10T00:47:41.267 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:40 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:47:41.267 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:40 smithi167 ceph-mon[36917]: from='osd.2 [v2:172.21.15.167:6800/4002498292,v1:172.21.15.167:6801/4002498292]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-10T00:47:41.267 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:40 smithi167 ceph-mon[36917]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-10T00:47:41.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:40 smithi094 ceph-mon[36708]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-10T00:47:41.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:40 smithi094 ceph-mon[36708]: osdmap e16: 3 total, 2 up, 3 in 2024-06-10T00:47:41.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:40 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:47:41.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:40 smithi094 ceph-mon[36708]: from='osd.2 [v2:172.21.15.167:6800/4002498292,v1:172.21.15.167:6801/4002498292]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-10T00:47:41.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:40 smithi094 ceph-mon[36708]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-10T00:47:41.588 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 10 00:47:41 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-2[42020]: 2024-06-10T00:47:41.382+0000 7f648256a700 -1 osd.2 0 waiting for initial osdmap 2024-06-10T00:47:41.589 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 10 00:47:41 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-2[42020]: 2024-06-10T00:47:41.386+0000 7f6479b54700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:47:41.813 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-10T00:47:41.813 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-10T00:47:41.813 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-06-10T00:47:41.813 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-06-10T00:47:41.813 INFO:teuthology.orchestra.run.smithi037.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0167057 s, 628 MB/s 2024-06-10T00:47:41.813 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-06-10T00:47:42.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:41 smithi167 ceph-mon[36917]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:42.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:41 smithi167 ceph-mon[36917]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-10T00:47:42.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:41 smithi167 ceph-mon[36917]: osdmap e17: 3 total, 2 up, 3 in 2024-06-10T00:47:42.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:41 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:47:42.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:41 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:47:42.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:41 smithi167 ceph-mon[36917]: from='osd.2 ' entity='osd.2' 2024-06-10T00:47:42.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:41 smithi094 ceph-mon[36708]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:42.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:41 smithi094 ceph-mon[36708]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-10T00:47:42.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:41 smithi094 ceph-mon[36708]: osdmap e17: 3 total, 2 up, 3 in 2024-06-10T00:47:42.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:41 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:47:42.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:41 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:47:42.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:41 smithi094 ceph-mon[36708]: from='osd.2 ' entity='osd.2' 2024-06-10T00:47:42.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:41 smithi037 ceph-mon[32192]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:42.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:41 smithi037 ceph-mon[32192]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-10T00:47:42.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:41 smithi037 ceph-mon[32192]: osdmap e17: 3 total, 2 up, 3 in 2024-06-10T00:47:42.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:47:42.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:47:42.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:41 smithi037 ceph-mon[32192]: from='osd.2 ' entity='osd.2' 2024-06-10T00:47:42.425 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_3 2024-06-10T00:47:42.676 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:47:43.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:42 smithi167 ceph-mon[36917]: purged_snaps scrub starts 2024-06-10T00:47:43.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:42 smithi167 ceph-mon[36917]: purged_snaps scrub ok 2024-06-10T00:47:43.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:42 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:47:43.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:42 smithi167 ceph-mon[36917]: osd.2 [v2:172.21.15.167:6800/4002498292,v1:172.21.15.167:6801/4002498292] boot 2024-06-10T00:47:43.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:42 smithi167 ceph-mon[36917]: osdmap e18: 3 total, 3 up, 3 in 2024-06-10T00:47:43.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:42 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:47:43.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:42 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:43.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:42 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:43.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:42 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:47:43.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:42 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:43.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:42 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:47:43.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:42 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:47:43.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:42 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:43.113 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:42 smithi037 ceph-mon[32192]: purged_snaps scrub starts 2024-06-10T00:47:43.114 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:42 smithi037 ceph-mon[32192]: purged_snaps scrub ok 2024-06-10T00:47:43.114 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:42 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:47:43.114 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:42 smithi037 ceph-mon[32192]: osd.2 [v2:172.21.15.167:6800/4002498292,v1:172.21.15.167:6801/4002498292] boot 2024-06-10T00:47:43.114 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:42 smithi037 ceph-mon[32192]: osdmap e18: 3 total, 3 up, 3 in 2024-06-10T00:47:43.114 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:42 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:47:43.114 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:42 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:43.114 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:42 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:43.114 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:42 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:47:43.114 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:42 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:43.114 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:42 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:47:43.114 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:42 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:47:43.115 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:42 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:43.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:42 smithi094 ceph-mon[36708]: purged_snaps scrub starts 2024-06-10T00:47:43.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:42 smithi094 ceph-mon[36708]: purged_snaps scrub ok 2024-06-10T00:47:43.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:42 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:47:43.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:42 smithi094 ceph-mon[36708]: osd.2 [v2:172.21.15.167:6800/4002498292,v1:172.21.15.167:6801/4002498292] boot 2024-06-10T00:47:43.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:42 smithi094 ceph-mon[36708]: osdmap e18: 3 total, 3 up, 3 in 2024-06-10T00:47:43.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:42 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-10T00:47:43.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:42 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:43.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:42 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:43.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:42 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:47:43.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:42 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:43.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:42 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:47:43.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:42 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:47:43.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:42 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:44.050 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:43 smithi037 ceph-mon[32192]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:44.050 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:43 smithi037 ceph-mon[32192]: Detected new or changed devices on smithi167 2024-06-10T00:47:44.050 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:43 smithi037 ceph-mon[32192]: Adjusting osd_memory_target on smithi167 to 17088M 2024-06-10T00:47:44.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.251+0000 7f98bdb44700 1 -- 172.21.15.37:0/4281472702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98b8102510 msgr2=0x7f98b8102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:47:44.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.251+0000 7f98bdb44700 1 --2- 172.21.15.37:0/4281472702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98b8102510 0x7f98b8102930 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f98a80099c0 tx=0x7f98a8009cd0 comp rx=0 tx=0).stop 2024-06-10T00:47:44.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.251+0000 7f98bdb44700 1 -- 172.21.15.37:0/4281472702 shutdown_connections 2024-06-10T00:47:44.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.251+0000 7f98bdb44700 1 --2- 172.21.15.37:0/4281472702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98b81040b0 0x7f98b810a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:47:44.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.251+0000 7f98bdb44700 1 --2- 172.21.15.37:0/4281472702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98b8102510 0x7f98b8102930 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:47:44.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.251+0000 7f98bdb44700 1 --2- 172.21.15.37:0/4281472702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98b8103710 0x7f98b8103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:47:44.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.252+0000 7f98bdb44700 1 -- 172.21.15.37:0/4281472702 >> 172.21.15.37:0/4281472702 conn(0x7f98b80fdab0 msgr2=0x7f98b80ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:47:44.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.252+0000 7f98bdb44700 1 -- 172.21.15.37:0/4281472702 shutdown_connections 2024-06-10T00:47:44.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.252+0000 7f98bdb44700 1 -- 172.21.15.37:0/4281472702 wait complete. 2024-06-10T00:47:44.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.252+0000 7f98bdb44700 1 Processor -- start 2024-06-10T00:47:44.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.253+0000 7f98bdb44700 1 -- start start 2024-06-10T00:47:44.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.253+0000 7f98bdb44700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98b8102510 0x7f98b8198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:47:44.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.253+0000 7f98bdb44700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98b8103710 0x7f98b8198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:47:44.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.253+0000 7f98bdb44700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98b81040b0 0x7f98b819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:47:44.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.253+0000 7f98bdb44700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f98b810e160 con 0x7f98b8102510 2024-06-10T00:47:44.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.253+0000 7f98bdb44700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f98b810dfe0 con 0x7f98b81040b0 2024-06-10T00:47:44.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.253+0000 7f98bdb44700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f98b810e2e0 con 0x7f98b8103710 2024-06-10T00:47:44.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.253+0000 7f98b77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98b8102510 0x7f98b8198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:47:44.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.253+0000 7f98b77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98b8102510 0x7f98b8198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33274/0 (socket says 172.21.15.37:33274) 2024-06-10T00:47:44.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.253+0000 7f98b77fe700 1 -- 172.21.15.37:0/278892957 learned_addr learned my addr 172.21.15.37:0/278892957 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:47:44.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.253+0000 7f98b6ffd700 1 --2- 172.21.15.37:0/278892957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98b8103710 0x7f98b8198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:47:44.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.253+0000 7f98b7fff700 1 --2- 172.21.15.37:0/278892957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98b81040b0 0x7f98b819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:47:44.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.253+0000 7f98b77fe700 1 -- 172.21.15.37:0/278892957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98b8103710 msgr2=0x7f98b8198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:47:44.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.253+0000 7f98b77fe700 1 --2- 172.21.15.37:0/278892957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98b8103710 0x7f98b8198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:47:44.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.253+0000 7f98b77fe700 1 -- 172.21.15.37:0/278892957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98b81040b0 msgr2=0x7f98b819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:47:44.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.253+0000 7f98b77fe700 1 --2- 172.21.15.37:0/278892957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98b81040b0 0x7f98b819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:47:44.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.253+0000 7f98b77fe700 1 -- 172.21.15.37:0/278892957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b81a03a0 con 0x7f98b8102510 2024-06-10T00:47:44.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.254+0000 7f98b77fe700 1 --2- 172.21.15.37:0/278892957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98b8102510 0x7f98b8198740 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f98a8005e80 tx=0x7f98a8005eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:47:44.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.254+0000 7f98b4ff9700 1 -- 172.21.15.37:0/278892957 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8027070 con 0x7f98b8102510 2024-06-10T00:47:44.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.254+0000 7f98bdb44700 1 -- 172.21.15.37:0/278892957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98b81a0630 con 0x7f98b8102510 2024-06-10T00:47:44.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.254+0000 7f98b4ff9700 1 -- 172.21.15.37:0/278892957 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98a8022470 con 0x7f98b8102510 2024-06-10T00:47:44.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.254+0000 7f98b4ff9700 1 -- 172.21.15.37:0/278892957 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a800f6a0 con 0x7f98b8102510 2024-06-10T00:47:44.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.254+0000 7f98bdb44700 1 -- 172.21.15.37:0/278892957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98b81a0be0 con 0x7f98b8102510 2024-06-10T00:47:44.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.255+0000 7f98bdb44700 1 -- 172.21.15.37:0/278892957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98b8069180 con 0x7f98b8102510 2024-06-10T00:47:44.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.257+0000 7f98b4ff9700 1 -- 172.21.15.37:0/278892957 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f98a802a070 con 0x7f98b8102510 2024-06-10T00:47:44.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.257+0000 7f98b4ff9700 1 --2- 172.21.15.37:0/278892957 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f98a00a2c10 0x7f98a00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:47:44.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.258+0000 7f98b6ffd700 1 --2- 172.21.15.37:0/278892957 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f98a00a2c10 0x7f98a00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:47:44.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.258+0000 7f98b4ff9700 1 -- 172.21.15.37:0/278892957 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 2851+0+0 (secure 0 0 0) 0x7f98a80ced40 con 0x7f98b8102510 2024-06-10T00:47:44.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.258+0000 7f98b6ffd700 1 --2- 172.21.15.37:0/278892957 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f98a00a2c10 0x7f98a00a50d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009a10 tx=0x7f98ac009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:47:44.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.261+0000 7f98b4ff9700 1 -- 172.21.15.37:0/278892957 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f98a8013070 con 0x7f98b8102510 2024-06-10T00:47:44.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:43 smithi167 ceph-mon[36917]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:44.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:43 smithi167 ceph-mon[36917]: Detected new or changed devices on smithi167 2024-06-10T00:47:44.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:43 smithi167 ceph-mon[36917]: Adjusting osd_memory_target on smithi167 to 17088M 2024-06-10T00:47:44.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:43 smithi094 ceph-mon[36708]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-10T00:47:44.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:43 smithi094 ceph-mon[36708]: Detected new or changed devices on smithi167 2024-06-10T00:47:44.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:43 smithi094 ceph-mon[36708]: Adjusting osd_memory_target on smithi167 to 17088M 2024-06-10T00:47:44.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:44.479+0000 7f98bdb44700 1 -- 172.21.15.37:0/278892957 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f98b80628d0 con 0x7f98a00a2c10 2024-06-10T00:47:45.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:44 smithi167 ceph-mon[36917]: osdmap e19: 3 total, 3 up, 3 in 2024-06-10T00:47:45.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:44 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-10T00:47:45.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:44 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:47:45.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:44 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:47:45.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:44 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:47:45.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:44 smithi094 ceph-mon[36708]: osdmap e19: 3 total, 3 up, 3 in 2024-06-10T00:47:45.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:44 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-10T00:47:45.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:44 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:47:45.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:44 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:47:45.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:44 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:47:45.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:44 smithi037 ceph-mon[32192]: osdmap e19: 3 total, 3 up, 3 in 2024-06-10T00:47:45.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:44 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-10T00:47:45.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:44 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:47:45.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:44 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:47:45.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:44 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:47:46.250 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:45 smithi037 ceph-mon[32192]: pgmap v81: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:47:46.250 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:45 smithi037 ceph-mon[32192]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:47:46.250 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:45 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-10T00:47:46.250 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:45 smithi037 ceph-mon[32192]: osdmap e20: 3 total, 3 up, 3 in 2024-06-10T00:47:46.250 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:45 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-10T00:47:46.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:45 smithi167 ceph-mon[36917]: pgmap v81: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:47:46.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:45 smithi167 ceph-mon[36917]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:47:46.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:45 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-10T00:47:46.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:45 smithi167 ceph-mon[36917]: osdmap e20: 3 total, 3 up, 3 in 2024-06-10T00:47:46.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:45 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-10T00:47:46.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:45 smithi094 ceph-mon[36708]: pgmap v81: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:47:46.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:45 smithi094 ceph-mon[36708]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:47:46.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:45 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-10T00:47:46.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:45 smithi094 ceph-mon[36708]: osdmap e20: 3 total, 3 up, 3 in 2024-06-10T00:47:46.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:45 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-10T00:47:46.896 INFO:journalctl@ceph.osd.0.smithi037.stdout:Jun 10 00:47:46 smithi037 sudo[49220]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-10T00:47:46.896 INFO:journalctl@ceph.osd.0.smithi037.stdout:Jun 10 00:47:46 smithi037 sudo[49220]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:47:46.896 INFO:journalctl@ceph.osd.0.smithi037.stdout:Jun 10 00:47:46 smithi037 sudo[49220]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:47:46.896 INFO:journalctl@ceph.osd.0.smithi037.stdout:Jun 10 00:47:46 smithi037 sudo[49220]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:47:46.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:47:46.990+0000 7f98b4ff9700 1 -- 172.21.15.37:0/278892957 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f98a8022620 con 0x7f98b8102510 2024-06-10T00:47:47.259 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:46 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-10T00:47:47.259 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:46 smithi094 ceph-mon[36708]: osdmap e21: 3 total, 3 up, 3 in 2024-06-10T00:47:47.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:46 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-10T00:47:47.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:46 smithi167 ceph-mon[36917]: osdmap e21: 3 total, 3 up, 3 in 2024-06-10T00:47:47.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:46 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-10T00:47:47.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:46 smithi037 ceph-mon[32192]: osdmap e21: 3 total, 3 up, 3 in 2024-06-10T00:47:47.396 INFO:journalctl@ceph.osd.0.smithi037.stdout:Jun 10 00:47:46 smithi037 sudo[49240]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-10T00:47:47.396 INFO:journalctl@ceph.osd.0.smithi037.stdout:Jun 10 00:47:46 smithi037 sudo[49240]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:47:47.396 INFO:journalctl@ceph.osd.0.smithi037.stdout:Jun 10 00:47:46 smithi037 sudo[49240]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:47:47.396 INFO:journalctl@ceph.osd.0.smithi037.stdout:Jun 10 00:47:46 smithi037 sudo[49240]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:47:47.512 INFO:journalctl@ceph.osd.1.smithi094.stdout:Jun 10 00:47:47 smithi094 sudo[45221]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-10T00:47:47.512 INFO:journalctl@ceph.osd.1.smithi094.stdout:Jun 10 00:47:47 smithi094 sudo[45221]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:47:47.512 INFO:journalctl@ceph.osd.1.smithi094.stdout:Jun 10 00:47:47 smithi094 sudo[45221]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:47:47.512 INFO:journalctl@ceph.osd.1.smithi094.stdout:Jun 10 00:47:47 smithi094 sudo[45221]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:47:47.864 INFO:journalctl@ceph.osd.1.smithi094.stdout:Jun 10 00:47:47 smithi094 sudo[45224]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-10T00:47:47.864 INFO:journalctl@ceph.osd.1.smithi094.stdout:Jun 10 00:47:47 smithi094 sudo[45224]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:47:47.865 INFO:journalctl@ceph.osd.1.smithi094.stdout:Jun 10 00:47:47 smithi094 sudo[45224]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:47:47.865 INFO:journalctl@ceph.osd.1.smithi094.stdout:Jun 10 00:47:47 smithi094 sudo[45224]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:47:48.037 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 10 00:47:47 smithi167 sudo[46076]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-10T00:47:48.037 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 10 00:47:47 smithi167 sudo[46076]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:47:48.037 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 10 00:47:47 smithi167 sudo[46076]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:47:48.037 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 10 00:47:47 smithi167 sudo[46076]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:47:48.038 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:47 smithi167 ceph-mon[36917]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:47:48.038 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:47 smithi167 ceph-mon[36917]: osdmap e22: 3 total, 3 up, 3 in 2024-06-10T00:47:48.038 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:47 smithi167 ceph-mon[36917]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-06-10T00:47:48.038 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:47 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:48.038 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:47 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/283912704' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0153676-818c-4dcc-83b1-287435a0c8a8"}]: dispatch 2024-06-10T00:47:48.038 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:47 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/283912704' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0153676-818c-4dcc-83b1-287435a0c8a8"}]': finished 2024-06-10T00:47:48.038 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:47 smithi167 ceph-mon[36917]: osdmap e23: 4 total, 3 up, 4 in 2024-06-10T00:47:48.038 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:47 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:47:48.038 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:47 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:48.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:47 smithi037 ceph-mon[32192]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:47:48.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:47 smithi037 ceph-mon[32192]: osdmap e22: 3 total, 3 up, 3 in 2024-06-10T00:47:48.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:47 smithi037 ceph-mon[32192]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-06-10T00:47:48.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:47 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:48.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:47 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/283912704' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0153676-818c-4dcc-83b1-287435a0c8a8"}]: dispatch 2024-06-10T00:47:48.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:47 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/283912704' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0153676-818c-4dcc-83b1-287435a0c8a8"}]': finished 2024-06-10T00:47:48.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:47 smithi037 ceph-mon[32192]: osdmap e23: 4 total, 3 up, 4 in 2024-06-10T00:47:48.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:47 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:47:48.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:47 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:48.338 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 10 00:47:48 smithi167 sudo[46079]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-10T00:47:48.339 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 10 00:47:48 smithi167 sudo[46079]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:47:48.339 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 10 00:47:48 smithi167 sudo[46079]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:47:48.339 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 10 00:47:48 smithi167 sudo[46079]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:47:48.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:47 smithi094 ceph-mon[36708]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:47:48.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:47 smithi094 ceph-mon[36708]: osdmap e22: 3 total, 3 up, 3 in 2024-06-10T00:47:48.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:47 smithi094 ceph-mon[36708]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-06-10T00:47:48.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:47 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:48.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:47 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/283912704' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0153676-818c-4dcc-83b1-287435a0c8a8"}]: dispatch 2024-06-10T00:47:48.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:47 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/283912704' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0153676-818c-4dcc-83b1-287435a0c8a8"}]': finished 2024-06-10T00:47:48.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:47 smithi094 ceph-mon[36708]: osdmap e23: 4 total, 3 up, 4 in 2024-06-10T00:47:48.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:47 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:47:48.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:47 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:48.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:48 smithi037 sudo[49307]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-10T00:47:48.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:48 smithi037 sudo[49307]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:47:48.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:48 smithi037 sudo[49307]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:47:48.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:48 smithi037 sudo[49307]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:47:49.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:48 smithi037 sudo[49310]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-10T00:47:49.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:48 smithi037 sudo[49310]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:47:49.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:48 smithi037 sudo[49310]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:47:49.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:48 smithi037 sudo[49310]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:47:49.456 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:49 smithi167 sudo[46082]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-10T00:47:49.456 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:49 smithi167 sudo[46082]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:47:49.456 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:49 smithi167 sudo[46082]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:47:49.456 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:49 smithi167 sudo[46082]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:47:49.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:49 smithi167 sudo[46085]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-10T00:47:49.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:49 smithi167 sudo[46085]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:47:49.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:49 smithi167 sudo[46085]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:47:49.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:49 smithi167 sudo[46085]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:47:49.950 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:49 smithi094 sudo[45227]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-10T00:47:49.951 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:49 smithi094 sudo[45227]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:47:49.951 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:49 smithi094 sudo[45227]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:47:49.951 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:49 smithi094 sudo[45227]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:47:50.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:49 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:50.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:49 smithi167 ceph-mon[36917]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T00:47:50.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:49 smithi167 ceph-mon[36917]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T00:47:50.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:49 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:47:50.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:49 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3314934629' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:47:50.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:49 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:47:50.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:49 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:47:50.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:49 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:47:50.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:49 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:47:50.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:49 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:47:50.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:49 smithi094 sudo[45230]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-10T00:47:50.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:49 smithi094 sudo[45230]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-10T00:47:50.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:49 smithi094 sudo[45230]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-10T00:47:50.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:49 smithi094 sudo[45230]: pam_unix(sudo:session): session closed for user root 2024-06-10T00:47:50.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:49 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:50.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:49 smithi094 ceph-mon[36708]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T00:47:50.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:49 smithi094 ceph-mon[36708]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T00:47:50.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:49 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:47:50.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:49 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3314934629' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:47:50.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:49 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:47:50.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:49 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:47:50.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:49 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:47:50.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:49 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:47:50.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:49 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:47:50.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:49 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:47:50.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:49 smithi037 ceph-mon[32192]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T00:47:50.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:49 smithi037 ceph-mon[32192]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T00:47:50.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:49 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:47:50.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:49 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3314934629' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:47:50.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:49 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:47:50.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:49 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:47:50.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:49 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:47:50.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:49 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:47:50.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:49 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:47:51.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:50 smithi167 ceph-mon[36917]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:47:51.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:50 smithi167 ceph-mon[36917]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T00:47:51.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:50 smithi167 ceph-mon[36917]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T00:47:51.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:50 smithi167 ceph-mon[36917]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T00:47:51.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:50 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:47:51.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:50 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:47:51.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:50 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:47:51.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:50 smithi167 ceph-mon[36917]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T00:47:51.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:50 smithi094 ceph-mon[36708]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:47:51.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:50 smithi094 ceph-mon[36708]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T00:47:51.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:50 smithi094 ceph-mon[36708]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T00:47:51.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:50 smithi094 ceph-mon[36708]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T00:47:51.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:50 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:47:51.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:50 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:47:51.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:50 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:47:51.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:50 smithi094 ceph-mon[36708]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T00:47:51.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:50 smithi037 ceph-mon[32192]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:47:51.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:50 smithi037 ceph-mon[32192]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T00:47:51.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:50 smithi037 ceph-mon[32192]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T00:47:51.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:50 smithi037 ceph-mon[32192]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-10T00:47:51.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:50 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-10T00:47:51.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:50 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-10T00:47:51.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:50 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-10T00:47:51.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:50 smithi037 ceph-mon[32192]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-10T00:47:52.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:51 smithi167 ceph-mon[36917]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:47:52.342 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:51 smithi037 ceph-mon[32192]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:47:52.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:51 smithi094 ceph-mon[36708]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:47:54.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:53 smithi037 ceph-mon[32192]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:47:54.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:53 smithi167 ceph-mon[36917]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:47:54.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:53 smithi094 ceph-mon[36708]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:47:56.318 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:55 smithi037 ceph-mon[32192]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-06-10T00:47:56.318 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:55 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-10T00:47:56.318 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:55 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:47:56.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:55 smithi167 ceph-mon[36917]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-06-10T00:47:56.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:55 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-10T00:47:56.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:55 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:47:56.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:55 smithi094 ceph-mon[36708]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-06-10T00:47:56.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:55 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-10T00:47:56.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:55 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:47:57.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:56 smithi037 ceph-mon[32192]: Deploying daemon osd.3 on smithi037 2024-06-10T00:47:57.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:56 smithi167 ceph-mon[36917]: Deploying daemon osd.3 on smithi037 2024-06-10T00:47:57.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:56 smithi094 ceph-mon[36708]: Deploying daemon osd.3 on smithi037 2024-06-10T00:47:58.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:57 smithi037 ceph-mon[32192]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-06-10T00:47:58.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:57 smithi167 ceph-mon[36917]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-06-10T00:47:58.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:57 smithi094 ceph-mon[36708]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-06-10T00:48:00.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:47:59 smithi037 ceph-mon[32192]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-06-10T00:48:00.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:47:59 smithi167 ceph-mon[36917]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-06-10T00:48:00.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:47:59 smithi094 ceph-mon[36708]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-06-10T00:48:02.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:01 smithi037 ceph-mon[32192]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-06-10T00:48:02.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:01 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:02.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:01 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:02.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:01 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:48:02.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:01 smithi167 ceph-mon[36917]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-06-10T00:48:02.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:01 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:02.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:01 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:02.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:01 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:48:02.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:01 smithi094 ceph-mon[36708]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-06-10T00:48:02.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:01 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:02.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:01 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:02.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:01 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:48:04.272 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:03 smithi037 ceph-mon[32192]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-10T00:48:04.272 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:03 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:04.272 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:03 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:04.272 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:03 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:04.272 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:03 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:48:04.272 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:03 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:04.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:03 smithi167 ceph-mon[36917]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-10T00:48:04.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:03 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:04.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:03 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:04.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:03 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:04.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:03 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:48:04.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:03 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:04.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:03 smithi094 ceph-mon[36708]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-10T00:48:04.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:03 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:04.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:03 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:04.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:03 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:04.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:03 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:48:04.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:03 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:04.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:04.627+0000 7f98b4ff9700 1 -- 172.21.15.37:0/278892957 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f98b80628d0 con 0x7f98a00a2c10 2024-06-10T00:48:04.628 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 3 on host 'smithi037' 2024-06-10T00:48:04.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:04.629+0000 7f98bdb44700 1 -- 172.21.15.37:0/278892957 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f98a00a2c10 msgr2=0x7f98a00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:48:04.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:04.629+0000 7f98bdb44700 1 --2- 172.21.15.37:0/278892957 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f98a00a2c10 0x7f98a00a50d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009a10 tx=0x7f98ac009190 comp rx=0 tx=0).stop 2024-06-10T00:48:04.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:04.629+0000 7f98bdb44700 1 -- 172.21.15.37:0/278892957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98b8102510 msgr2=0x7f98b8198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:48:04.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:04.629+0000 7f98bdb44700 1 --2- 172.21.15.37:0/278892957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98b8102510 0x7f98b8198740 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f98a8005e80 tx=0x7f98a8005eb0 comp rx=0 tx=0).stop 2024-06-10T00:48:04.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:04.629+0000 7f98bdb44700 1 -- 172.21.15.37:0/278892957 shutdown_connections 2024-06-10T00:48:04.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:04.629+0000 7f98bdb44700 1 --2- 172.21.15.37:0/278892957 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f98a00a2c10 0x7f98a00a50d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:48:04.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:04.629+0000 7f98bdb44700 1 --2- 172.21.15.37:0/278892957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98b81040b0 0x7f98b819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:48:04.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:04.629+0000 7f98bdb44700 1 --2- 172.21.15.37:0/278892957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98b8102510 0x7f98b8198740 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:48:04.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:04.629+0000 7f98bdb44700 1 --2- 172.21.15.37:0/278892957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98b8103710 0x7f98b8198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:48:04.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:04.629+0000 7f98bdb44700 1 -- 172.21.15.37:0/278892957 >> 172.21.15.37:0/278892957 conn(0x7f98b80fdab0 msgr2=0x7f98b81069b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:48:04.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:04.630+0000 7f98bdb44700 1 -- 172.21.15.37:0/278892957 shutdown_connections 2024-06-10T00:48:04.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:04.630+0000 7f98bdb44700 1 -- 172.21.15.37:0/278892957 wait complete. 2024-06-10T00:48:05.218 DEBUG:teuthology.orchestra.run.smithi037:osd.3> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.3.service 2024-06-10T00:48:05.220 INFO:tasks.cephadm:Deploying osd.4 on smithi094 with /dev/vg_nvme/lv_3... 2024-06-10T00:48:05.220 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-10T00:48:05.463 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T00:48:05.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:05 smithi167 ceph-mon[36917]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-10T00:48:05.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:05 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:05.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:05 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:05.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:05 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:48:05.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:05 smithi167 ceph-mon[36917]: from='osd.3 [v2:172.21.15.37:6810/3425469324,v1:172.21.15.37:6811/3425469324]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-10T00:48:05.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:05 smithi094 ceph-mon[36708]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-10T00:48:05.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:05 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:05.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:05 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:05.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:05 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:48:05.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:05 smithi094 ceph-mon[36708]: from='osd.3 [v2:172.21.15.37:6810/3425469324,v1:172.21.15.37:6811/3425469324]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-10T00:48:05.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:05 smithi037 ceph-mon[32192]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-06-10T00:48:05.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:05 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:05.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:05 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:05.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:05 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:48:05.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:05 smithi037 ceph-mon[32192]: from='osd.3 [v2:172.21.15.37:6810/3425469324,v1:172.21.15.37:6811/3425469324]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-10T00:48:06.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:06 smithi037 ceph-mon[32192]: from='osd.3 [v2:172.21.15.37:6810/3425469324,v1:172.21.15.37:6811/3425469324]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-10T00:48:06.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:06 smithi037 ceph-mon[32192]: osdmap e24: 4 total, 3 up, 4 in 2024-06-10T00:48:06.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:06 smithi037 ceph-mon[32192]: from='osd.3 [v2:172.21.15.37:6810/3425469324,v1:172.21.15.37:6811/3425469324]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-06-10T00:48:06.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:06 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:48:06.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:06 smithi094 ceph-mon[36708]: from='osd.3 [v2:172.21.15.37:6810/3425469324,v1:172.21.15.37:6811/3425469324]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-10T00:48:06.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:06 smithi094 ceph-mon[36708]: osdmap e24: 4 total, 3 up, 4 in 2024-06-10T00:48:06.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:06 smithi094 ceph-mon[36708]: from='osd.3 [v2:172.21.15.37:6810/3425469324,v1:172.21.15.37:6811/3425469324]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-06-10T00:48:06.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:06 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:48:07.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:06 smithi167 ceph-mon[36917]: from='osd.3 [v2:172.21.15.37:6810/3425469324,v1:172.21.15.37:6811/3425469324]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-10T00:48:07.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:06 smithi167 ceph-mon[36917]: osdmap e24: 4 total, 3 up, 4 in 2024-06-10T00:48:07.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:06 smithi167 ceph-mon[36917]: from='osd.3 [v2:172.21.15.37:6810/3425469324,v1:172.21.15.37:6811/3425469324]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-06-10T00:48:07.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:06 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:48:07.353 INFO:journalctl@ceph.osd.3.smithi037.stdout:Jun 10 00:48:07 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-3[51301]: 2024-06-10T00:48:07.029+0000 7f33538a7700 -1 osd.3 0 waiting for initial osdmap 2024-06-10T00:48:07.353 INFO:journalctl@ceph.osd.3.smithi037.stdout:Jun 10 00:48:07 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-3[51301]: 2024-06-10T00:48:07.036+0000 7f334ae91700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:48:07.430 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-10T00:48:07.431 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-10T00:48:07.431 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-06-10T00:48:07.431 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-06-10T00:48:07.431 INFO:teuthology.orchestra.run.smithi094.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00845245 s, 1.2 GB/s 2024-06-10T00:48:07.431 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 2024-06-10T00:48:07.431 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-06-10T00:48:07.631 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:07 smithi094 ceph-mon[36708]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:48:07.631 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:07 smithi094 ceph-mon[36708]: from='osd.3 [v2:172.21.15.37:6810/3425469324,v1:172.21.15.37:6811/3425469324]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-06-10T00:48:07.632 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:07 smithi094 ceph-mon[36708]: osdmap e25: 4 total, 3 up, 4 in 2024-06-10T00:48:07.632 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:07 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:48:07.632 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:07 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:48:07.632 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:07 smithi094 ceph-mon[36708]: from='osd.3 [v2:172.21.15.37:6810/3425469324,v1:172.21.15.37:6811/3425469324]' entity='osd.3' 2024-06-10T00:48:07.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:07 smithi037 ceph-mon[32192]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:48:07.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:07 smithi037 ceph-mon[32192]: from='osd.3 [v2:172.21.15.37:6810/3425469324,v1:172.21.15.37:6811/3425469324]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-06-10T00:48:07.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:07 smithi037 ceph-mon[32192]: osdmap e25: 4 total, 3 up, 4 in 2024-06-10T00:48:07.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:07 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:48:07.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:07 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:48:07.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:07 smithi037 ceph-mon[32192]: from='osd.3 [v2:172.21.15.37:6810/3425469324,v1:172.21.15.37:6811/3425469324]' entity='osd.3' 2024-06-10T00:48:08.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:07 smithi167 ceph-mon[36917]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:48:08.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:07 smithi167 ceph-mon[36917]: from='osd.3 [v2:172.21.15.37:6810/3425469324,v1:172.21.15.37:6811/3425469324]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-06-10T00:48:08.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:07 smithi167 ceph-mon[36917]: osdmap e25: 4 total, 3 up, 4 in 2024-06-10T00:48:08.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:07 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:48:08.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:07 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:48:08.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:07 smithi167 ceph-mon[36917]: from='osd.3 [v2:172.21.15.37:6810/3425469324,v1:172.21.15.37:6811/3425469324]' entity='osd.3' 2024-06-10T00:48:08.137 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_3 2024-06-10T00:48:08.377 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T00:48:08.753 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:08 smithi094 ceph-mon[36708]: purged_snaps scrub starts 2024-06-10T00:48:08.753 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:08 smithi094 ceph-mon[36708]: purged_snaps scrub ok 2024-06-10T00:48:08.753 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:08 smithi094 ceph-mon[36708]: osd.3 [v2:172.21.15.37:6810/3425469324,v1:172.21.15.37:6811/3425469324] boot 2024-06-10T00:48:08.753 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:08 smithi094 ceph-mon[36708]: osdmap e26: 4 total, 4 up, 4 in 2024-06-10T00:48:08.753 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:08 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:48:08.754 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:08 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:08.754 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:08 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:08.754 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:08 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:48:08.754 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:08 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:48:08.754 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:08 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:08.754 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:08 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:08.754 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:08 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:48:08.754 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:08 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:08.754 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:08 smithi094 ceph-mon[36708]: osdmap e27: 4 total, 4 up, 4 in 2024-06-10T00:48:08.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:08 smithi037 ceph-mon[32192]: purged_snaps scrub starts 2024-06-10T00:48:08.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:08 smithi037 ceph-mon[32192]: purged_snaps scrub ok 2024-06-10T00:48:08.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:08 smithi037 ceph-mon[32192]: osd.3 [v2:172.21.15.37:6810/3425469324,v1:172.21.15.37:6811/3425469324] boot 2024-06-10T00:48:08.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:08 smithi037 ceph-mon[32192]: osdmap e26: 4 total, 4 up, 4 in 2024-06-10T00:48:08.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:08 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:48:08.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:08 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:08.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:08 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:08.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:08 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:48:08.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:08 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:48:08.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:08 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:08.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:08 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:08.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:08 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:48:08.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:08 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:08.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:08 smithi037 ceph-mon[32192]: osdmap e27: 4 total, 4 up, 4 in 2024-06-10T00:48:09.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:08 smithi167 ceph-mon[36917]: purged_snaps scrub starts 2024-06-10T00:48:09.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:08 smithi167 ceph-mon[36917]: purged_snaps scrub ok 2024-06-10T00:48:09.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:08 smithi167 ceph-mon[36917]: osd.3 [v2:172.21.15.37:6810/3425469324,v1:172.21.15.37:6811/3425469324] boot 2024-06-10T00:48:09.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:08 smithi167 ceph-mon[36917]: osdmap e26: 4 total, 4 up, 4 in 2024-06-10T00:48:09.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:08 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-10T00:48:09.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:08 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:09.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:08 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:09.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:08 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:48:09.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:08 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:48:09.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:08 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:09.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:08 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:09.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:08 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:48:09.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:08 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:09.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:08 smithi167 ceph-mon[36917]: osdmap e27: 4 total, 4 up, 4 in 2024-06-10T00:48:09.722 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:09 smithi094 ceph-mon[36708]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:48:09.722 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:09 smithi094 ceph-mon[36708]: Detected new or changed devices on smithi037 2024-06-10T00:48:09.723 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:09 smithi094 ceph-mon[36708]: Adjusting osd_memory_target on smithi037 to 8546M 2024-06-10T00:48:09.723 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:09 smithi094 ceph-mon[36708]: osdmap e28: 4 total, 4 up, 4 in 2024-06-10T00:48:09.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:09 smithi037 ceph-mon[32192]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:48:09.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:09 smithi037 ceph-mon[32192]: Detected new or changed devices on smithi037 2024-06-10T00:48:09.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:09 smithi037 ceph-mon[32192]: Adjusting osd_memory_target on smithi037 to 8546M 2024-06-10T00:48:09.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:09 smithi037 ceph-mon[32192]: osdmap e28: 4 total, 4 up, 4 in 2024-06-10T00:48:09.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.955+0000 7fc377130700 1 -- 172.21.15.94:0/1294060281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc370100f20 msgr2=0x7fc37010aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:48:09.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.955+0000 7fc377130700 1 --2- 172.21.15.94:0/1294060281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc370100f20 0x7fc37010aaf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc36800b0a0 tx=0x7fc36800b3b0 comp rx=0 tx=0).stop 2024-06-10T00:48:09.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.955+0000 7fc377130700 1 -- 172.21.15.94:0/1294060281 shutdown_connections 2024-06-10T00:48:09.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.955+0000 7fc377130700 1 --2- 172.21.15.94:0/1294060281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc370100f20 0x7fc37010aaf0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:48:09.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.955+0000 7fc377130700 1 --2- 172.21.15.94:0/1294060281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc370103b00 0x7fc370103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:48:09.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.955+0000 7fc377130700 1 --2- 172.21.15.94:0/1294060281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc370100580 0x7fc3701009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:48:09.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.955+0000 7fc377130700 1 -- 172.21.15.94:0/1294060281 >> 172.21.15.94:0/1294060281 conn(0x7fc3700fc140 msgr2=0x7fc3700fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:48:09.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.956+0000 7fc377130700 1 -- 172.21.15.94:0/1294060281 shutdown_connections 2024-06-10T00:48:09.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.956+0000 7fc377130700 1 -- 172.21.15.94:0/1294060281 wait complete. 2024-06-10T00:48:09.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.956+0000 7fc377130700 1 Processor -- start 2024-06-10T00:48:09.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.956+0000 7fc377130700 1 -- start start 2024-06-10T00:48:09.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.956+0000 7fc377130700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc370100580 0x7fc370103200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:48:09.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.956+0000 7fc377130700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc370100f20 0x7fc370101850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:48:09.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.956+0000 7fc377130700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc370103b00 0x7fc370101d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:48:09.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.956+0000 7fc377130700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc37010e160 con 0x7fc370100f20 2024-06-10T00:48:09.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.956+0000 7fc377130700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc37010dfe0 con 0x7fc370100580 2024-06-10T00:48:09.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.956+0000 7fc377130700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc37010e2e0 con 0x7fc370103b00 2024-06-10T00:48:09.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.956+0000 7fc374ecc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc370100580 0x7fc370103200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:48:09.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.956+0000 7fc374ecc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc370100580 0x7fc370103200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50060/0 (socket says 172.21.15.94:50060) 2024-06-10T00:48:09.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.956+0000 7fc374ecc700 1 -- 172.21.15.94:0/3837067149 learned_addr learned my addr 172.21.15.94:0/3837067149 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-10T00:48:09.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.956+0000 7fc36ffff700 1 --2- 172.21.15.94:0/3837067149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc370100f20 0x7fc370101850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:48:09.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.956+0000 7fc3756cd700 1 --2- 172.21.15.94:0/3837067149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc370103b00 0x7fc370101d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:48:09.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.957+0000 7fc374ecc700 1 -- 172.21.15.94:0/3837067149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc370103b00 msgr2=0x7fc370101d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:48:09.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.957+0000 7fc374ecc700 1 --2- 172.21.15.94:0/3837067149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc370103b00 0x7fc370101d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:48:09.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.957+0000 7fc374ecc700 1 -- 172.21.15.94:0/3837067149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc370100f20 msgr2=0x7fc370101850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:48:09.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.957+0000 7fc374ecc700 1 --2- 172.21.15.94:0/3837067149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc370100f20 0x7fc370101850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:48:09.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.957+0000 7fc374ecc700 1 -- 172.21.15.94:0/3837067149 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc370102640 con 0x7fc370100580 2024-06-10T00:48:09.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.957+0000 7fc3756cd700 1 --2- 172.21.15.94:0/3837067149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc370103b00 0x7fc370101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:48:09.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.957+0000 7fc374ecc700 1 --2- 172.21.15.94:0/3837067149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc370100580 0x7fc370103200 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc36400d980 tx=0x7fc36400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:48:09.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.957+0000 7fc36dffb700 1 -- 172.21.15.94:0/3837067149 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc364016070 con 0x7fc370100580 2024-06-10T00:48:09.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.957+0000 7fc36dffb700 1 -- 172.21.15.94:0/3837067149 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc36400be30 con 0x7fc370100580 2024-06-10T00:48:09.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.957+0000 7fc377130700 1 -- 172.21.15.94:0/3837067149 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3701a2850 con 0x7fc370100580 2024-06-10T00:48:09.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.957+0000 7fc36dffb700 1 -- 172.21.15.94:0/3837067149 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36400f3c0 con 0x7fc370100580 2024-06-10T00:48:09.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.957+0000 7fc377130700 1 -- 172.21.15.94:0/3837067149 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3701a2d90 con 0x7fc370100580 2024-06-10T00:48:09.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.958+0000 7fc377130700 1 -- 172.21.15.94:0/3837067149 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc370069180 con 0x7fc370100580 2024-06-10T00:48:09.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.959+0000 7fc36dffb700 1 -- 172.21.15.94:0/3837067149 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc364019070 con 0x7fc370100580 2024-06-10T00:48:09.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.960+0000 7fc36dffb700 1 --2- 172.21.15.94:0/3837067149 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3600a2ca0 0x7fc3600a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:48:09.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.960+0000 7fc36dffb700 1 -- 172.21.15.94:0/3837067149 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3698+0+0 (secure 0 0 0) 0x7fc3640cd9a0 con 0x7fc370100580 2024-06-10T00:48:09.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.960+0000 7fc36ffff700 1 --2- 172.21.15.94:0/3837067149 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3600a2ca0 0x7fc3600a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:48:09.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.960+0000 7fc36ffff700 1 --2- 172.21.15.94:0/3837067149 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3600a2ca0 0x7fc3600a5160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009a10 tx=0x7fc35c009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:48:09.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:09.965+0000 7fc36dffb700 1 -- 172.21.15.94:0/3837067149 <== mon.2 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc364020030 con 0x7fc370100580 2024-06-10T00:48:10.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:09 smithi167 ceph-mon[36917]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-10T00:48:10.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:09 smithi167 ceph-mon[36917]: Detected new or changed devices on smithi037 2024-06-10T00:48:10.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:09 smithi167 ceph-mon[36917]: Adjusting osd_memory_target on smithi037 to 8546M 2024-06-10T00:48:10.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:09 smithi167 ceph-mon[36917]: osdmap e28: 4 total, 4 up, 4 in 2024-06-10T00:48:10.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:10.169+0000 7fc377130700 1 -- 172.21.15.94:0/3837067149 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc3700628d0 con 0x7fc3600a2ca0 2024-06-10T00:48:10.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:10 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:48:10.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:10 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:48:10.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:10 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:10.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:10 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:48:10.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:10 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:48:10.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:10 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:11.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:10 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:48:11.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:10 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:48:11.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:10 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:11.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:11 smithi167 ceph-mon[36917]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:48:11.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:11 smithi167 ceph-mon[36917]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:48:11.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:11 smithi167 ceph-mon[36917]: osdmap e29: 4 total, 4 up, 4 in 2024-06-10T00:48:11.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:11 smithi037 ceph-mon[32192]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:48:11.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:11 smithi037 ceph-mon[32192]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:48:11.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:11 smithi037 ceph-mon[32192]: osdmap e29: 4 total, 4 up, 4 in 2024-06-10T00:48:11.925 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:11 smithi094 ceph-mon[36708]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:48:11.925 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:11 smithi094 ceph-mon[36708]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:48:11.925 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:11 smithi094 ceph-mon[36708]: osdmap e29: 4 total, 4 up, 4 in 2024-06-10T00:48:12.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:12 smithi037 ceph-mon[32192]: osdmap e30: 4 total, 4 up, 4 in 2024-06-10T00:48:13.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:12 smithi167 ceph-mon[36917]: osdmap e30: 4 total, 4 up, 4 in 2024-06-10T00:48:13.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:12 smithi094 ceph-mon[36708]: osdmap e30: 4 total, 4 up, 4 in 2024-06-10T00:48:13.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:13 smithi037 ceph-mon[32192]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:48:13.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:13 smithi037 ceph-mon[32192]: from='client.? 172.21.15.94:0/188039765' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "172c333c-00c0-4e94-8870-acb3b32b2b1b"}]: dispatch 2024-06-10T00:48:13.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:13 smithi037 ceph-mon[32192]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "172c333c-00c0-4e94-8870-acb3b32b2b1b"}]: dispatch 2024-06-10T00:48:13.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:13 smithi037 ceph-mon[32192]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "172c333c-00c0-4e94-8870-acb3b32b2b1b"}]': finished 2024-06-10T00:48:13.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:13 smithi037 ceph-mon[32192]: osdmap e31: 5 total, 4 up, 5 in 2024-06-10T00:48:13.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:13 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:48:13.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:13 smithi037 ceph-mon[32192]: from='client.? 172.21.15.94:0/3578018318' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:48:14.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:13 smithi167 ceph-mon[36917]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:48:14.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:13 smithi167 ceph-mon[36917]: from='client.? 172.21.15.94:0/188039765' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "172c333c-00c0-4e94-8870-acb3b32b2b1b"}]: dispatch 2024-06-10T00:48:14.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:13 smithi167 ceph-mon[36917]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "172c333c-00c0-4e94-8870-acb3b32b2b1b"}]: dispatch 2024-06-10T00:48:14.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:13 smithi167 ceph-mon[36917]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "172c333c-00c0-4e94-8870-acb3b32b2b1b"}]': finished 2024-06-10T00:48:14.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:13 smithi167 ceph-mon[36917]: osdmap e31: 5 total, 4 up, 5 in 2024-06-10T00:48:14.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:13 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:48:14.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:13 smithi167 ceph-mon[36917]: from='client.? 172.21.15.94:0/3578018318' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:48:14.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:13 smithi094 ceph-mon[36708]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:48:14.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:13 smithi094 ceph-mon[36708]: from='client.? 172.21.15.94:0/188039765' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "172c333c-00c0-4e94-8870-acb3b32b2b1b"}]: dispatch 2024-06-10T00:48:14.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:13 smithi094 ceph-mon[36708]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "172c333c-00c0-4e94-8870-acb3b32b2b1b"}]: dispatch 2024-06-10T00:48:14.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:13 smithi094 ceph-mon[36708]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "172c333c-00c0-4e94-8870-acb3b32b2b1b"}]': finished 2024-06-10T00:48:14.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:13 smithi094 ceph-mon[36708]: osdmap e31: 5 total, 4 up, 5 in 2024-06-10T00:48:14.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:13 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:48:14.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:13 smithi094 ceph-mon[36708]: from='client.? 172.21.15.94:0/3578018318' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:48:15.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:15 smithi037 ceph-mon[32192]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-06-10T00:48:16.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:15 smithi167 ceph-mon[36917]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-06-10T00:48:16.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:15 smithi094 ceph-mon[36708]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-06-10T00:48:17.884 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:17 smithi094 ceph-mon[36708]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-10T00:48:17.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:17 smithi037 ceph-mon[32192]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-10T00:48:18.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:17 smithi167 ceph-mon[36917]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-10T00:48:19.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:19 smithi094 ceph-mon[36708]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-06-10T00:48:19.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:19 smithi037 ceph-mon[32192]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-06-10T00:48:20.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:19 smithi167 ceph-mon[36917]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-06-10T00:48:20.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:20 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-10T00:48:20.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:20 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:20.938 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:20 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-10T00:48:20.938 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:20 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:21.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:20 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-10T00:48:21.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:20 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:21.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:21 smithi094 ceph-mon[36708]: Deploying daemon osd.4 on smithi094 2024-06-10T00:48:21.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:21 smithi094 ceph-mon[36708]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-06-10T00:48:21.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:21 smithi037 ceph-mon[32192]: Deploying daemon osd.4 on smithi094 2024-06-10T00:48:21.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:21 smithi037 ceph-mon[32192]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-06-10T00:48:22.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:21 smithi167 ceph-mon[36917]: Deploying daemon osd.4 on smithi094 2024-06-10T00:48:22.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:21 smithi167 ceph-mon[36917]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-06-10T00:48:23.894 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:23 smithi094 ceph-mon[36708]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T00:48:23.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:23 smithi037 ceph-mon[32192]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T00:48:24.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:23 smithi167 ceph-mon[36917]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T00:48:25.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:25 smithi094 ceph-mon[36708]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-06-10T00:48:25.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:25 smithi037 ceph-mon[32192]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-06-10T00:48:26.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:25 smithi167 ceph-mon[36917]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-06-10T00:48:26.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:26 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:48:26.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:26 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:26.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:26 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:26.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:26 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:48:26.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:26 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:26.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:26 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:27.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:26 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:48:27.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:26 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:27.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:26 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:27.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:27 smithi094 ceph-mon[36708]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:48:28.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:27 smithi167 ceph-mon[36917]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:48:28.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:27 smithi037 ceph-mon[32192]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:48:29.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:29.140+0000 7fc36dffb700 1 -- 172.21.15.94:0/3837067149 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc3700628d0 con 0x7fc3600a2ca0 2024-06-10T00:48:29.141 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 4 on host 'smithi094' 2024-06-10T00:48:29.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:29.144+0000 7fc377130700 1 -- 172.21.15.94:0/3837067149 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3600a2ca0 msgr2=0x7fc3600a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:48:29.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:29.144+0000 7fc377130700 1 --2- 172.21.15.94:0/3837067149 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3600a2ca0 0x7fc3600a5160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009a10 tx=0x7fc35c009190 comp rx=0 tx=0).stop 2024-06-10T00:48:29.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:29.144+0000 7fc377130700 1 -- 172.21.15.94:0/3837067149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc370100580 msgr2=0x7fc370103200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:48:29.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:29.144+0000 7fc377130700 1 --2- 172.21.15.94:0/3837067149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc370100580 0x7fc370103200 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc36400d980 tx=0x7fc36400dc90 comp rx=0 tx=0).stop 2024-06-10T00:48:29.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:29.144+0000 7fc377130700 1 -- 172.21.15.94:0/3837067149 shutdown_connections 2024-06-10T00:48:29.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:29.144+0000 7fc377130700 1 --2- 172.21.15.94:0/3837067149 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3600a2ca0 0x7fc3600a5160 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:48:29.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:29.144+0000 7fc377130700 1 --2- 172.21.15.94:0/3837067149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc370103b00 0x7fc370101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:48:29.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:29.144+0000 7fc377130700 1 --2- 172.21.15.94:0/3837067149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc370100580 0x7fc370103200 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:48:29.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:29.144+0000 7fc377130700 1 --2- 172.21.15.94:0/3837067149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc370100f20 0x7fc370101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:48:29.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:29.144+0000 7fc377130700 1 -- 172.21.15.94:0/3837067149 >> 172.21.15.94:0/3837067149 conn(0x7fc3700fc140 msgr2=0x7fc370106ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:48:29.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:29.144+0000 7fc377130700 1 -- 172.21.15.94:0/3837067149 shutdown_connections 2024-06-10T00:48:29.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:48:29.144+0000 7fc377130700 1 -- 172.21.15.94:0/3837067149 wait complete. 2024-06-10T00:48:29.774 DEBUG:teuthology.orchestra.run.smithi094:osd.4> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.4.service 2024-06-10T00:48:29.776 INFO:tasks.cephadm:Deploying osd.5 on smithi167 with /dev/vg_nvme/lv_3... 2024-06-10T00:48:29.777 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-10T00:48:30.023 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:48:30.107 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:30 smithi094 ceph-mon[36708]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:48:30.107 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:30 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:30.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:30 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:30.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:30 smithi094 ceph-mon[36708]: from='osd.4 [v2:172.21.15.94:6808/1740627431,v1:172.21.15.94:6809/1740627431]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-10T00:48:30.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:30 smithi094 ceph-mon[36708]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-10T00:48:30.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:30 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:30.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:30 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:30.394 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:30 smithi167 ceph-mon[36917]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:48:30.394 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:30 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:30.394 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:30 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:30.395 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:30 smithi167 ceph-mon[36917]: from='osd.4 [v2:172.21.15.94:6808/1740627431,v1:172.21.15.94:6809/1740627431]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-10T00:48:30.395 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:30 smithi167 ceph-mon[36917]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-10T00:48:30.395 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:30 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:30.395 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:30 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:30.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:30 smithi037 ceph-mon[32192]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:48:30.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:30 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:30.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:30 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:30.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:30 smithi037 ceph-mon[32192]: from='osd.4 [v2:172.21.15.94:6808/1740627431,v1:172.21.15.94:6809/1740627431]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-10T00:48:30.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:30 smithi037 ceph-mon[32192]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-10T00:48:30.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:30 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:30.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:30 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:31.373 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:31 smithi094 ceph-mon[36708]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-10T00:48:31.373 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:31 smithi094 ceph-mon[36708]: from='osd.4 [v2:172.21.15.94:6808/1740627431,v1:172.21.15.94:6809/1740627431]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-10T00:48:31.373 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:31 smithi094 ceph-mon[36708]: osdmap e32: 5 total, 4 up, 5 in 2024-06-10T00:48:31.373 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:31 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:48:31.373 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:31 smithi094 ceph-mon[36708]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-10T00:48:31.374 INFO:journalctl@ceph.osd.4.smithi094.stdout:Jun 10 00:48:31 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-4[48025]: 2024-06-10T00:48:31.119+0000 7f5f38f5c700 -1 osd.4 0 waiting for initial osdmap 2024-06-10T00:48:31.374 INFO:journalctl@ceph.osd.4.smithi094.stdout:Jun 10 00:48:31 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-4[48025]: 2024-06-10T00:48:31.127+0000 7f5f34d4f700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:48:31.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:31 smithi037 ceph-mon[32192]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-10T00:48:31.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:31 smithi037 ceph-mon[32192]: from='osd.4 [v2:172.21.15.94:6808/1740627431,v1:172.21.15.94:6809/1740627431]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-10T00:48:31.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:31 smithi037 ceph-mon[32192]: osdmap e32: 5 total, 4 up, 5 in 2024-06-10T00:48:31.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:31 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:48:31.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:31 smithi037 ceph-mon[32192]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-10T00:48:31.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:31 smithi167 ceph-mon[36917]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-10T00:48:31.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:31 smithi167 ceph-mon[36917]: from='osd.4 [v2:172.21.15.94:6808/1740627431,v1:172.21.15.94:6809/1740627431]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-10T00:48:31.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:31 smithi167 ceph-mon[36917]: osdmap e32: 5 total, 4 up, 5 in 2024-06-10T00:48:31.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:31 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:48:31.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:31 smithi167 ceph-mon[36917]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-10T00:48:32.059 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-10T00:48:32.059 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-10T00:48:32.059 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-06-10T00:48:32.059 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-06-10T00:48:32.060 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126379 s, 830 MB/s 2024-06-10T00:48:32.060 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-06-10T00:48:32.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:32 smithi094 ceph-mon[36708]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:48:32.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:32 smithi094 ceph-mon[36708]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-06-10T00:48:32.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:32 smithi094 ceph-mon[36708]: osdmap e33: 5 total, 4 up, 5 in 2024-06-10T00:48:32.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:32 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:48:32.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:32 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:48:32.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:32 smithi037 ceph-mon[32192]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:48:32.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:32 smithi037 ceph-mon[32192]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-06-10T00:48:32.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:32 smithi037 ceph-mon[32192]: osdmap e33: 5 total, 4 up, 5 in 2024-06-10T00:48:32.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:32 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:48:32.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:32 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:48:32.446 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:32 smithi167 ceph-mon[36917]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:48:32.447 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:32 smithi167 ceph-mon[36917]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-06-10T00:48:32.447 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:32 smithi167 ceph-mon[36917]: osdmap e33: 5 total, 4 up, 5 in 2024-06-10T00:48:32.447 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:32 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:48:32.447 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:32 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:48:32.737 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_3 2024-06-10T00:48:32.974 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:48:33.394 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:33 smithi167 ceph-mon[36917]: purged_snaps scrub starts 2024-06-10T00:48:33.394 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:33 smithi167 ceph-mon[36917]: purged_snaps scrub ok 2024-06-10T00:48:33.394 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:33 smithi167 ceph-mon[36917]: osd.4 [v2:172.21.15.94:6808/1740627431,v1:172.21.15.94:6809/1740627431] boot 2024-06-10T00:48:33.394 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:33 smithi167 ceph-mon[36917]: osdmap e34: 5 total, 5 up, 5 in 2024-06-10T00:48:33.394 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:33 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:48:33.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:33 smithi037 ceph-mon[32192]: purged_snaps scrub starts 2024-06-10T00:48:33.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:33 smithi037 ceph-mon[32192]: purged_snaps scrub ok 2024-06-10T00:48:33.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:33 smithi037 ceph-mon[32192]: osd.4 [v2:172.21.15.94:6808/1740627431,v1:172.21.15.94:6809/1740627431] boot 2024-06-10T00:48:33.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:33 smithi037 ceph-mon[32192]: osdmap e34: 5 total, 5 up, 5 in 2024-06-10T00:48:33.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:33 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:48:33.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:33 smithi094 ceph-mon[36708]: purged_snaps scrub starts 2024-06-10T00:48:33.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:33 smithi094 ceph-mon[36708]: purged_snaps scrub ok 2024-06-10T00:48:33.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:33 smithi094 ceph-mon[36708]: osd.4 [v2:172.21.15.94:6808/1740627431,v1:172.21.15.94:6809/1740627431] boot 2024-06-10T00:48:33.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:33 smithi094 ceph-mon[36708]: osdmap e34: 5 total, 5 up, 5 in 2024-06-10T00:48:33.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:33 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-10T00:48:34.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:34 smithi037 ceph-mon[32192]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:48:34.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:34 smithi037 ceph-mon[32192]: osdmap e35: 5 total, 5 up, 5 in 2024-06-10T00:48:34.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:34 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:34.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:34 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:34.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:34 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:48:34.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:34 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:48:34.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:34 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:34.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:34 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:34.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:34 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:48:34.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:34 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:34.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:34 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:48:34.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:34 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:34.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:34 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:48:34.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:34 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:34.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:34 smithi167 ceph-mon[36917]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:48:34.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:34 smithi167 ceph-mon[36917]: osdmap e35: 5 total, 5 up, 5 in 2024-06-10T00:48:34.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:34 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:34.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:34 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:34.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:34 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:48:34.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:34 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:48:34.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:34 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:34.501 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:34 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:34.501 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:34 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:48:34.501 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:34 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:34.501 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:34 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:48:34.501 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:34 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:34.501 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:34 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:48:34.501 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:34 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:34.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:34 smithi094 ceph-mon[36708]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-06-10T00:48:34.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:34 smithi094 ceph-mon[36708]: osdmap e35: 5 total, 5 up, 5 in 2024-06-10T00:48:34.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:34 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:34.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:34 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:34.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:34 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:48:34.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:34 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:48:34.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:34 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:34.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:34 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:34.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:34 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:48:34.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:34 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:34.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:34 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:48:34.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:34 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:34.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:34 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:48:34.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:34 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:34.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.721+0000 7fccf3ba8700 1 -- 172.21.15.167:0/360948511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccec1025e0 msgr2=0x7fccec102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:48:34.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.721+0000 7fccf3ba8700 1 --2- 172.21.15.167:0/360948511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccec1025e0 0x7fccec102a40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009990 tx=0x7fcce8009ca0 comp rx=0 tx=0).stop 2024-06-10T00:48:34.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.721+0000 7fccf3ba8700 1 -- 172.21.15.167:0/360948511 shutdown_connections 2024-06-10T00:48:34.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.721+0000 7fccf3ba8700 1 --2- 172.21.15.167:0/360948511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccec102f80 0x7fccec109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:48:34.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.721+0000 7fccf3ba8700 1 --2- 172.21.15.167:0/360948511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccec1013e0 0x7fccec101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:48:34.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.721+0000 7fccf3ba8700 1 --2- 172.21.15.167:0/360948511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccec1025e0 0x7fccec102a40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:48:34.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.721+0000 7fccf3ba8700 1 -- 172.21.15.167:0/360948511 >> 172.21.15.167:0/360948511 conn(0x7fccec0fc980 msgr2=0x7fccec0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:48:34.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.721+0000 7fccf3ba8700 1 -- 172.21.15.167:0/360948511 shutdown_connections 2024-06-10T00:48:34.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.721+0000 7fccf3ba8700 1 -- 172.21.15.167:0/360948511 wait complete. 2024-06-10T00:48:34.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.722+0000 7fccf3ba8700 1 Processor -- start 2024-06-10T00:48:34.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.722+0000 7fccf3ba8700 1 -- start start 2024-06-10T00:48:34.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.722+0000 7fccf3ba8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccec1013e0 0x7fccec1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:48:34.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.722+0000 7fccf3ba8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccec1025e0 0x7fccec198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:48:34.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.722+0000 7fccf3ba8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccec102f80 0x7fccec19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:48:34.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.722+0000 7fccf3ba8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fccec10cfc0 con 0x7fccec1025e0 2024-06-10T00:48:34.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.722+0000 7fccf3ba8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fccec10ce40 con 0x7fccec1013e0 2024-06-10T00:48:34.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.722+0000 7fccf3ba8700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fccec10d140 con 0x7fccec102f80 2024-06-10T00:48:34.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.722+0000 7fccf2145700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccec102f80 0x7fccec19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:48:34.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.722+0000 7fccf2145700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccec102f80 0x7fccec19fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:48812/0 (socket says 172.21.15.167:48812) 2024-06-10T00:48:34.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.722+0000 7fccf2145700 1 -- 172.21.15.167:0/2516734633 learned_addr learned my addr 172.21.15.167:0/2516734633 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-10T00:48:34.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.722+0000 7fccf1944700 1 --2- 172.21.15.167:0/2516734633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccec1013e0 0x7fccec1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:48:34.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.722+0000 7fccf1143700 1 --2- 172.21.15.167:0/2516734633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccec1025e0 0x7fccec198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:48:34.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.722+0000 7fccf2145700 1 -- 172.21.15.167:0/2516734633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccec1013e0 msgr2=0x7fccec1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:48:34.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.722+0000 7fccf2145700 1 --2- 172.21.15.167:0/2516734633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccec1013e0 0x7fccec1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:48:34.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.722+0000 7fccf2145700 1 -- 172.21.15.167:0/2516734633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccec1025e0 msgr2=0x7fccec198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:48:34.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.722+0000 7fccf2145700 1 --2- 172.21.15.167:0/2516734633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccec1025e0 0x7fccec198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:48:34.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.722+0000 7fccf2145700 1 -- 172.21.15.167:0/2516734633 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccec1a0350 con 0x7fccec102f80 2024-06-10T00:48:34.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.722+0000 7fccf1143700 1 --2- 172.21.15.167:0/2516734633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccec1025e0 0x7fccec198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:48:34.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.723+0000 7fccf2145700 1 --2- 172.21.15.167:0/2516734633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccec102f80 0x7fccec19fcb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcce4007f10 tx=0x7fcce400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:48:34.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.723+0000 7fcce2ffd700 1 -- 172.21.15.167:0/2516734633 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce4004bb0 con 0x7fccec102f80 2024-06-10T00:48:34.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.723+0000 7fcce2ffd700 1 -- 172.21.15.167:0/2516734633 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcce4004d50 con 0x7fccec102f80 2024-06-10T00:48:34.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.723+0000 7fcce2ffd700 1 -- 172.21.15.167:0/2516734633 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce4005a60 con 0x7fccec102f80 2024-06-10T00:48:34.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.723+0000 7fccf3ba8700 1 -- 172.21.15.167:0/2516734633 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccec1a0640 con 0x7fccec102f80 2024-06-10T00:48:34.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.723+0000 7fccf3ba8700 1 -- 172.21.15.167:0/2516734633 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccec1a0b80 con 0x7fccec102f80 2024-06-10T00:48:34.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.724+0000 7fccf3ba8700 1 -- 172.21.15.167:0/2516734633 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccec069180 con 0x7fccec102f80 2024-06-10T00:48:34.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.725+0000 7fcce2ffd700 1 -- 172.21.15.167:0/2516734633 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcce401a020 con 0x7fccec102f80 2024-06-10T00:48:34.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.725+0000 7fcce2ffd700 1 --2- 172.21.15.167:0/2516734633 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fccd80a2c20 0x7fccd80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:48:34.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.726+0000 7fcce2ffd700 1 -- 172.21.15.167:0/2516734633 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7fcce40cdda0 con 0x7fccec102f80 2024-06-10T00:48:34.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.726+0000 7fccf1944700 1 --2- 172.21.15.167:0/2516734633 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fccd80a2c20 0x7fccd80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:48:34.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.726+0000 7fccf1944700 1 --2- 172.21.15.167:0/2516734633 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fccd80a2c20 0x7fccd80a50e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fccec102440 tx=0x7fccdc00a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:48:34.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.729+0000 7fcce2ffd700 1 -- 172.21.15.167:0/2516734633 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcce4010070 con 0x7fccec102f80 2024-06-10T00:48:34.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:34.934+0000 7fccf3ba8700 1 -- 172.21.15.167:0/2516734633 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fccec06b110 con 0x7fccd80a2c20 2024-06-10T00:48:35.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:35 smithi167 ceph-mon[36917]: Detected new or changed devices on smithi094 2024-06-10T00:48:35.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:35 smithi167 ceph-mon[36917]: Adjusting osd_memory_target on smithi094 to 8544M 2024-06-10T00:48:35.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:35 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:48:35.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:35 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:48:35.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:35 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:35.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:35 smithi037 ceph-mon[32192]: Detected new or changed devices on smithi094 2024-06-10T00:48:35.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:35 smithi037 ceph-mon[32192]: Adjusting osd_memory_target on smithi094 to 8544M 2024-06-10T00:48:35.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:35 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:48:35.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:35 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:48:35.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:35 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:35.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:35 smithi094 ceph-mon[36708]: Detected new or changed devices on smithi094 2024-06-10T00:48:35.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:35 smithi094 ceph-mon[36708]: Adjusting osd_memory_target on smithi094 to 8544M 2024-06-10T00:48:35.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:35 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:48:35.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:35 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:48:35.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:35 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:36.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:36 smithi037 ceph-mon[32192]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:36.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:36 smithi037 ceph-mon[32192]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:48:36.517 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:36 smithi167 ceph-mon[36917]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:36.517 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:36 smithi167 ceph-mon[36917]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:48:36.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:36 smithi094 ceph-mon[36708]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:36.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:36 smithi094 ceph-mon[36708]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:48:38.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:38 smithi167 ceph-mon[36917]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:38.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:38 smithi167 ceph-mon[36917]: from='client.? 172.21.15.167:0/699789564' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbce656e-5f93-4bda-baf2-fc3cc03d8282"}]: dispatch 2024-06-10T00:48:38.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:38 smithi167 ceph-mon[36917]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbce656e-5f93-4bda-baf2-fc3cc03d8282"}]: dispatch 2024-06-10T00:48:38.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:38 smithi167 ceph-mon[36917]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dbce656e-5f93-4bda-baf2-fc3cc03d8282"}]': finished 2024-06-10T00:48:38.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:38 smithi167 ceph-mon[36917]: osdmap e36: 6 total, 5 up, 6 in 2024-06-10T00:48:38.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:38 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:48:38.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:38 smithi037 ceph-mon[32192]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:38.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:38 smithi037 ceph-mon[32192]: from='client.? 172.21.15.167:0/699789564' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbce656e-5f93-4bda-baf2-fc3cc03d8282"}]: dispatch 2024-06-10T00:48:38.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:38 smithi037 ceph-mon[32192]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbce656e-5f93-4bda-baf2-fc3cc03d8282"}]: dispatch 2024-06-10T00:48:38.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:38 smithi037 ceph-mon[32192]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dbce656e-5f93-4bda-baf2-fc3cc03d8282"}]': finished 2024-06-10T00:48:38.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:38 smithi037 ceph-mon[32192]: osdmap e36: 6 total, 5 up, 6 in 2024-06-10T00:48:38.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:38 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:48:38.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:38 smithi094 ceph-mon[36708]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:38.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:38 smithi094 ceph-mon[36708]: from='client.? 172.21.15.167:0/699789564' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbce656e-5f93-4bda-baf2-fc3cc03d8282"}]: dispatch 2024-06-10T00:48:38.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:38 smithi094 ceph-mon[36708]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbce656e-5f93-4bda-baf2-fc3cc03d8282"}]: dispatch 2024-06-10T00:48:38.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:38 smithi094 ceph-mon[36708]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dbce656e-5f93-4bda-baf2-fc3cc03d8282"}]': finished 2024-06-10T00:48:38.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:38 smithi094 ceph-mon[36708]: osdmap e36: 6 total, 5 up, 6 in 2024-06-10T00:48:38.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:38 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:48:39.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:39 smithi167 ceph-mon[36917]: from='client.? 172.21.15.167:0/1761717215' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:48:39.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:39 smithi094 ceph-mon[36708]: from='client.? 172.21.15.167:0/1761717215' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:48:39.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:39 smithi037 ceph-mon[32192]: from='client.? 172.21.15.167:0/1761717215' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:48:40.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:40 smithi167 ceph-mon[36917]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:40.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:40 smithi094 ceph-mon[36708]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:40.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:40 smithi037 ceph-mon[32192]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:42.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:42 smithi167 ceph-mon[36917]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:42.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:42 smithi094 ceph-mon[36708]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:42.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:42 smithi037 ceph-mon[32192]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:44.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:44 smithi167 ceph-mon[36917]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:44.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:44 smithi094 ceph-mon[36708]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:44.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:44 smithi037 ceph-mon[32192]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:45.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:45 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-10T00:48:45.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:45 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:45.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:45 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-10T00:48:45.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:45 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:45.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:45 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-10T00:48:45.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:45 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:46.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:46 smithi167 ceph-mon[36917]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:46.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:46 smithi167 ceph-mon[36917]: Deploying daemon osd.5 on smithi167 2024-06-10T00:48:46.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:46 smithi094 ceph-mon[36708]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:46.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:46 smithi094 ceph-mon[36708]: Deploying daemon osd.5 on smithi167 2024-06-10T00:48:46.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:46 smithi037 ceph-mon[32192]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:46.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:46 smithi037 ceph-mon[32192]: Deploying daemon osd.5 on smithi167 2024-06-10T00:48:47.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:47 smithi167 ceph-mon[36917]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:47.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:47 smithi094 ceph-mon[36708]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:47.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:47 smithi037 ceph-mon[32192]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:49.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:49 smithi094 ceph-mon[36708]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:49.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:49 smithi037 ceph-mon[32192]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:49.750 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:49 smithi167 ceph-mon[36917]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:51.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:51 smithi037 ceph-mon[32192]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:51.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:51 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:51.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:51 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:51.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:51 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:48:52.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:51 smithi167 ceph-mon[36917]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:52.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:51 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:52.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:51 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:52.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:51 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:48:52.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:51 smithi094 ceph-mon[36708]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:52.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:51 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:52.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:51 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:52.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:51 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:48:53.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:53 smithi037 ceph-mon[32192]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:54.020 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:53 smithi167 ceph-mon[36917]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:54.104 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 5 on host 'smithi167' 2024-06-10T00:48:54.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:54.103+0000 7fcce2ffd700 1 -- 172.21.15.167:0/2516734633 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fccec06b110 con 0x7fccd80a2c20 2024-06-10T00:48:54.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:54.105+0000 7fccf3ba8700 1 -- 172.21.15.167:0/2516734633 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fccd80a2c20 msgr2=0x7fccd80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:48:54.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:54.105+0000 7fccf3ba8700 1 --2- 172.21.15.167:0/2516734633 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fccd80a2c20 0x7fccd80a50e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fccec102440 tx=0x7fccdc00a380 comp rx=0 tx=0).stop 2024-06-10T00:48:54.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:54.106+0000 7fccf3ba8700 1 -- 172.21.15.167:0/2516734633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccec102f80 msgr2=0x7fccec19fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:48:54.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:54.106+0000 7fccf3ba8700 1 --2- 172.21.15.167:0/2516734633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccec102f80 0x7fccec19fcb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcce4007f10 tx=0x7fcce400d3b0 comp rx=0 tx=0).stop 2024-06-10T00:48:54.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:54.106+0000 7fccf3ba8700 1 -- 172.21.15.167:0/2516734633 shutdown_connections 2024-06-10T00:48:54.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:54.106+0000 7fccf3ba8700 1 --2- 172.21.15.167:0/2516734633 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fccd80a2c20 0x7fccd80a50e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:48:54.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:54.106+0000 7fccf3ba8700 1 --2- 172.21.15.167:0/2516734633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fccec102f80 0x7fccec19fcb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:48:54.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:54.106+0000 7fccf3ba8700 1 --2- 172.21.15.167:0/2516734633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccec1013e0 0x7fccec1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:48:54.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:54.106+0000 7fccf3ba8700 1 --2- 172.21.15.167:0/2516734633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccec1025e0 0x7fccec198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:48:54.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:54.106+0000 7fccf3ba8700 1 -- 172.21.15.167:0/2516734633 >> 172.21.15.167:0/2516734633 conn(0x7fccec0fc980 msgr2=0x7fccec06aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:48:54.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:54.106+0000 7fccf3ba8700 1 -- 172.21.15.167:0/2516734633 shutdown_connections 2024-06-10T00:48:54.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:48:54.106+0000 7fccf3ba8700 1 -- 172.21.15.167:0/2516734633 wait complete. 2024-06-10T00:48:54.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:53 smithi094 ceph-mon[36708]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:54.776 DEBUG:teuthology.orchestra.run.smithi167:osd.5> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.5.service 2024-06-10T00:48:54.779 INFO:tasks.cephadm:Deploying osd.6 on smithi037 with /dev/vg_nvme/lv_2... 2024-06-10T00:48:54.779 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-10T00:48:55.038 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:48:55.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:55 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:55.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:55 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:55.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:55 smithi037 ceph-mon[32192]: from='osd.5 [v2:172.21.15.167:6808/221805236,v1:172.21.15.167:6809/221805236]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-10T00:48:55.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:55 smithi037 ceph-mon[32192]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-10T00:48:55.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:55 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:55.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:55 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:55.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:55 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:55.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:55 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:55.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:55 smithi167 ceph-mon[36917]: from='osd.5 [v2:172.21.15.167:6808/221805236,v1:172.21.15.167:6809/221805236]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-10T00:48:55.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:55 smithi167 ceph-mon[36917]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-10T00:48:55.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:55 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:55.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:55 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:55.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:55 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:55.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:55 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:55.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:55 smithi094 ceph-mon[36708]: from='osd.5 [v2:172.21.15.167:6808/221805236,v1:172.21.15.167:6809/221805236]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-10T00:48:55.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:55 smithi094 ceph-mon[36708]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-10T00:48:55.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:55 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:55.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:55 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:56.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:56 smithi167 ceph-mon[36917]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:56.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:56 smithi167 ceph-mon[36917]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-10T00:48:56.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:56 smithi167 ceph-mon[36917]: osdmap e37: 6 total, 5 up, 6 in 2024-06-10T00:48:56.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:56 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:48:56.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:56 smithi167 ceph-mon[36917]: from='osd.5 [v2:172.21.15.167:6808/221805236,v1:172.21.15.167:6809/221805236]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-10T00:48:56.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:56 smithi167 ceph-mon[36917]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-10T00:48:56.340 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 00:48:56 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T00:48:56.085+0000 7f94b2551700 -1 osd.5 0 waiting for initial osdmap 2024-06-10T00:48:56.340 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 00:48:56 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T00:48:56.091+0000 7f94ae344700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:48:56.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:56 smithi094 ceph-mon[36708]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:56.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:56 smithi094 ceph-mon[36708]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-10T00:48:56.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:56 smithi094 ceph-mon[36708]: osdmap e37: 6 total, 5 up, 6 in 2024-06-10T00:48:56.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:56 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:48:56.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:56 smithi094 ceph-mon[36708]: from='osd.5 [v2:172.21.15.167:6808/221805236,v1:172.21.15.167:6809/221805236]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-10T00:48:56.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:56 smithi094 ceph-mon[36708]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-10T00:48:56.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:56 smithi037 ceph-mon[32192]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:56.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:56 smithi037 ceph-mon[32192]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-10T00:48:56.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:56 smithi037 ceph-mon[32192]: osdmap e37: 6 total, 5 up, 6 in 2024-06-10T00:48:56.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:56 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:48:56.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:56 smithi037 ceph-mon[32192]: from='osd.5 [v2:172.21.15.167:6808/221805236,v1:172.21.15.167:6809/221805236]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-10T00:48:56.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:56 smithi037 ceph-mon[32192]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-10T00:48:57.009 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-10T00:48:57.009 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-10T00:48:57.009 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-06-10T00:48:57.009 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-06-10T00:48:57.009 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113723 s, 922 MB/s 2024-06-10T00:48:57.009 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-06-10T00:48:57.323 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:57 smithi037 ceph-mon[32192]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-10T00:48:57.324 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:57 smithi037 ceph-mon[32192]: osdmap e38: 6 total, 5 up, 6 in 2024-06-10T00:48:57.324 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:57 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:48:57.324 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:57 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:48:57.324 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:57 smithi037 ceph-mon[32192]: osd.5 [v2:172.21.15.167:6808/221805236,v1:172.21.15.167:6809/221805236] boot 2024-06-10T00:48:57.324 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:57 smithi037 ceph-mon[32192]: osdmap e39: 6 total, 6 up, 6 in 2024-06-10T00:48:57.324 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:57 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:48:57.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:57 smithi167 ceph-mon[36917]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-10T00:48:57.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:57 smithi167 ceph-mon[36917]: osdmap e38: 6 total, 5 up, 6 in 2024-06-10T00:48:57.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:57 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:48:57.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:57 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:48:57.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:57 smithi167 ceph-mon[36917]: osd.5 [v2:172.21.15.167:6808/221805236,v1:172.21.15.167:6809/221805236] boot 2024-06-10T00:48:57.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:57 smithi167 ceph-mon[36917]: osdmap e39: 6 total, 6 up, 6 in 2024-06-10T00:48:57.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:57 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:48:57.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:57 smithi094 ceph-mon[36708]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-10T00:48:57.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:57 smithi094 ceph-mon[36708]: osdmap e38: 6 total, 5 up, 6 in 2024-06-10T00:48:57.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:57 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:48:57.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:57 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:48:57.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:57 smithi094 ceph-mon[36708]: osd.5 [v2:172.21.15.167:6808/221805236,v1:172.21.15.167:6809/221805236] boot 2024-06-10T00:48:57.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:57 smithi094 ceph-mon[36708]: osdmap e39: 6 total, 6 up, 6 in 2024-06-10T00:48:57.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:57 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-10T00:48:57.650 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_2 2024-06-10T00:48:57.906 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:48:58.224 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:58 smithi167 ceph-mon[36917]: purged_snaps scrub starts 2024-06-10T00:48:58.224 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:58 smithi167 ceph-mon[36917]: purged_snaps scrub ok 2024-06-10T00:48:58.224 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:58 smithi167 ceph-mon[36917]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:58.224 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:58 smithi167 ceph-mon[36917]: osdmap e40: 6 total, 6 up, 6 in 2024-06-10T00:48:58.330 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:58 smithi037 ceph-mon[32192]: purged_snaps scrub starts 2024-06-10T00:48:58.330 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:58 smithi037 ceph-mon[32192]: purged_snaps scrub ok 2024-06-10T00:48:58.330 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:58 smithi037 ceph-mon[32192]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:58.330 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:58 smithi037 ceph-mon[32192]: osdmap e40: 6 total, 6 up, 6 in 2024-06-10T00:48:58.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:58 smithi094 ceph-mon[36708]: purged_snaps scrub starts 2024-06-10T00:48:58.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:58 smithi094 ceph-mon[36708]: purged_snaps scrub ok 2024-06-10T00:48:58.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:58 smithi094 ceph-mon[36708]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-06-10T00:48:58.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:58 smithi094 ceph-mon[36708]: osdmap e40: 6 total, 6 up, 6 in 2024-06-10T00:48:59.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:59 smithi037 ceph-mon[32192]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:48:59.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:59 smithi037 ceph-mon[32192]: Detected new or changed devices on smithi167 2024-06-10T00:48:59.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:59 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:59.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:59 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:59.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:59 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:48:59.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:59 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:48:59.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:59 smithi037 ceph-mon[32192]: Adjusting osd_memory_target on smithi167 to 8544M 2024-06-10T00:48:59.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:59 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:59.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:59 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:59.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:59 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:48:59.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:59 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:59.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:59 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:48:59.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:59 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:59.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:59 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:48:59.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:59 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:59.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:48:59 smithi037 ceph-mon[32192]: osdmap e41: 6 total, 6 up, 6 in 2024-06-10T00:48:59.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.499+0000 7f1daa864700 1 -- 172.21.15.37:0/298246313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da41013e0 msgr2=0x7f1da4101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:48:59.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.499+0000 7f1daa864700 1 --2- 172.21.15.37:0/298246313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da41013e0 0x7f1da4101800 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1d940099c0 tx=0x7f1d94009cd0 comp rx=0 tx=0).stop 2024-06-10T00:48:59.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.499+0000 7f1daa864700 1 -- 172.21.15.37:0/298246313 shutdown_connections 2024-06-10T00:48:59.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.499+0000 7f1daa864700 1 --2- 172.21.15.37:0/298246313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1da4102f80 0x7f1da4109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:48:59.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.499+0000 7f1daa864700 1 --2- 172.21.15.37:0/298246313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da41013e0 0x7f1da4101800 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:48:59.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.499+0000 7f1daa864700 1 --2- 172.21.15.37:0/298246313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1da41025e0 0x7f1da4102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:48:59.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.499+0000 7f1daa864700 1 -- 172.21.15.37:0/298246313 >> 172.21.15.37:0/298246313 conn(0x7f1da40fc980 msgr2=0x7f1da40fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:48:59.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.499+0000 7f1daa864700 1 -- 172.21.15.37:0/298246313 shutdown_connections 2024-06-10T00:48:59.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.499+0000 7f1daa864700 1 -- 172.21.15.37:0/298246313 wait complete. 2024-06-10T00:48:59.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.500+0000 7f1daa864700 1 Processor -- start 2024-06-10T00:48:59.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.500+0000 7f1daa864700 1 -- start start 2024-06-10T00:48:59.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.500+0000 7f1daa864700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1da41013e0 0x7f1da41054c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:48:59.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.500+0000 7f1daa864700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1da41025e0 0x7f1da4103b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:48:59.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.500+0000 7f1daa864700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da4102f80 0x7f1da4104050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:48:59.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.500+0000 7f1daa864700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1da410d000 con 0x7f1da4102f80 2024-06-10T00:48:59.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.500+0000 7f1daa864700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1da410ce80 con 0x7f1da41025e0 2024-06-10T00:48:59.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.500+0000 7f1daa864700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1da410d180 con 0x7f1da41013e0 2024-06-10T00:48:59.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.500+0000 7f1da8e01700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da4102f80 0x7f1da4104050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:48:59.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.500+0000 7f1da8e01700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da4102f80 0x7f1da4104050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32796/0 (socket says 172.21.15.37:32796) 2024-06-10T00:48:59.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.500+0000 7f1da8e01700 1 -- 172.21.15.37:0/1824010807 learned_addr learned my addr 172.21.15.37:0/1824010807 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:48:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.500+0000 7f1da3fff700 1 --2- 172.21.15.37:0/1824010807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1da41013e0 0x7f1da41054c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:48:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.500+0000 7f1d9bfff700 1 --2- 172.21.15.37:0/1824010807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1da41025e0 0x7f1da4103b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:48:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.500+0000 7f1da8e01700 1 -- 172.21.15.37:0/1824010807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1da41013e0 msgr2=0x7f1da41054c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:48:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.500+0000 7f1da8e01700 1 --2- 172.21.15.37:0/1824010807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1da41013e0 0x7f1da41054c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:48:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.500+0000 7f1da8e01700 1 -- 172.21.15.37:0/1824010807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1da41025e0 msgr2=0x7f1da4103b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:48:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.500+0000 7f1da8e01700 1 --2- 172.21.15.37:0/1824010807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1da41025e0 0x7f1da4103b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:48:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.500+0000 7f1da8e01700 1 -- 172.21.15.37:0/1824010807 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da4104900 con 0x7f1da4102f80 2024-06-10T00:48:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.500+0000 7f1d9bfff700 1 --2- 172.21.15.37:0/1824010807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1da41025e0 0x7f1da4103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:48:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.501+0000 7f1da8e01700 1 --2- 172.21.15.37:0/1824010807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da4102f80 0x7f1da4104050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c007ee0 tx=0x7f1d9c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:48:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.501+0000 7f1da1ffb700 1 -- 172.21.15.37:0/1824010807 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c00dbd0 con 0x7f1da4102f80 2024-06-10T00:48:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.501+0000 7f1da1ffb700 1 -- 172.21.15.37:0/1824010807 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d9c00dd70 con 0x7f1da4102f80 2024-06-10T00:48:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.501+0000 7f1daa864700 1 -- 172.21.15.37:0/1824010807 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da41a28f0 con 0x7f1da4102f80 2024-06-10T00:48:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.501+0000 7f1da1ffb700 1 -- 172.21.15.37:0/1824010807 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c0133f0 con 0x7f1da4102f80 2024-06-10T00:48:59.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.501+0000 7f1daa864700 1 -- 172.21.15.37:0/1824010807 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da41a2e30 con 0x7f1da4102f80 2024-06-10T00:48:59.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.502+0000 7f1daa864700 1 -- 172.21.15.37:0/1824010807 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1da4069180 con 0x7f1da4102f80 2024-06-10T00:48:59.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.503+0000 7f1da1ffb700 1 -- 172.21.15.37:0/1824010807 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1d9c01a020 con 0x7f1da4102f80 2024-06-10T00:48:59.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.504+0000 7f1da1ffb700 1 --2- 172.21.15.37:0/1824010807 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1d900a2ca0 0x7f1d900a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:48:59.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.504+0000 7f1da3fff700 1 --2- 172.21.15.37:0/1824010807 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1d900a2ca0 0x7f1d900a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:48:59.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.504+0000 7f1da1ffb700 1 -- 172.21.15.37:0/1824010807 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4562+0+0 (secure 0 0 0) 0x7f1d9c0cdff0 con 0x7f1da4102f80 2024-06-10T00:48:59.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.504+0000 7f1da3fff700 1 --2- 172.21.15.37:0/1824010807 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1d900a2ca0 0x7f1d900a5160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400b580 tx=0x7f1d94005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:48:59.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.508+0000 7f1da1ffb700 1 -- 172.21.15.37:0/1824010807 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1d9c010070 con 0x7f1da4102f80 2024-06-10T00:48:59.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:59 smithi167 ceph-mon[36917]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:48:59.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:59 smithi167 ceph-mon[36917]: Detected new or changed devices on smithi167 2024-06-10T00:48:59.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:59 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:59.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:59 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:59.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:59 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:48:59.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:59 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:48:59.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:59 smithi167 ceph-mon[36917]: Adjusting osd_memory_target on smithi167 to 8544M 2024-06-10T00:48:59.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:59 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:59.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:59 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:59.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:59 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:48:59.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:59 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:59.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:59 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:48:59.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:59 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:59.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:59 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:48:59.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:59 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:59.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:48:59 smithi167 ceph-mon[36917]: osdmap e41: 6 total, 6 up, 6 in 2024-06-10T00:48:59.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:59 smithi094 ceph-mon[36708]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:48:59.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:59 smithi094 ceph-mon[36708]: Detected new or changed devices on smithi167 2024-06-10T00:48:59.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:59 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:59.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:59 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:59.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:59 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:48:59.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:59 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:48:59.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:59 smithi094 ceph-mon[36708]: Adjusting osd_memory_target on smithi167 to 8544M 2024-06-10T00:48:59.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:59 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:59.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:59 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:59.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:59 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:48:59.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:59 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:59.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:59 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:48:59.616 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:59 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:48:59.616 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:59 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:48:59.616 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:59 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:48:59.616 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:48:59 smithi094 ceph-mon[36708]: osdmap e41: 6 total, 6 up, 6 in 2024-06-10T00:48:59.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:48:59.725+0000 7f1daa864700 1 -- 172.21.15.37:0/1824010807 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f1da406b330 con 0x7f1d900a2ca0 2024-06-10T00:49:00.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:00 smithi167 ceph-mon[36917]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:49:00.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:00 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:49:00.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:00 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:49:00.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:00 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:49:00.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:00 smithi094 ceph-mon[36708]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:49:00.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:00 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:49:00.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:00 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:49:00.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:00 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:49:00.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:00 smithi037 ceph-mon[32192]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:49:00.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:00 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:49:00.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:00 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:49:00.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:00 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:49:01.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:01 smithi167 ceph-mon[36917]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:49:01.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:01 smithi167 ceph-mon[36917]: osdmap e42: 6 total, 6 up, 6 in 2024-06-10T00:49:01.594 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:01 smithi037 ceph-mon[32192]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:49:01.594 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:01 smithi037 ceph-mon[32192]: osdmap e42: 6 total, 6 up, 6 in 2024-06-10T00:49:01.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:01 smithi094 ceph-mon[36708]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:49:01.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:01 smithi094 ceph-mon[36708]: osdmap e42: 6 total, 6 up, 6 in 2024-06-10T00:49:02.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:02 smithi167 ceph-mon[36917]: osdmap e43: 6 total, 6 up, 6 in 2024-06-10T00:49:02.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:02 smithi094 ceph-mon[36708]: osdmap e43: 6 total, 6 up, 6 in 2024-06-10T00:49:02.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:02 smithi037 ceph-mon[32192]: osdmap e43: 6 total, 6 up, 6 in 2024-06-10T00:49:03.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:03 smithi167 ceph-mon[36917]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:49:03.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:03 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2766789959' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fcbda266-6d60-43b2-a51c-4e562476d1a9"}]: dispatch 2024-06-10T00:49:03.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:03 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2766789959' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fcbda266-6d60-43b2-a51c-4e562476d1a9"}]': finished 2024-06-10T00:49:03.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:03 smithi167 ceph-mon[36917]: osdmap e44: 7 total, 6 up, 7 in 2024-06-10T00:49:03.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:03 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:49:03.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:03 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1671946709' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:49:03.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:03 smithi094 ceph-mon[36708]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:49:03.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:03 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2766789959' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fcbda266-6d60-43b2-a51c-4e562476d1a9"}]: dispatch 2024-06-10T00:49:03.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:03 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2766789959' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fcbda266-6d60-43b2-a51c-4e562476d1a9"}]': finished 2024-06-10T00:49:03.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:03 smithi094 ceph-mon[36708]: osdmap e44: 7 total, 6 up, 7 in 2024-06-10T00:49:03.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:03 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:49:03.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:03 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1671946709' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:49:03.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:03 smithi037 ceph-mon[32192]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:49:03.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:03 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2766789959' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fcbda266-6d60-43b2-a51c-4e562476d1a9"}]: dispatch 2024-06-10T00:49:03.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:03 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2766789959' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fcbda266-6d60-43b2-a51c-4e562476d1a9"}]': finished 2024-06-10T00:49:03.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:03 smithi037 ceph-mon[32192]: osdmap e44: 7 total, 6 up, 7 in 2024-06-10T00:49:03.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:03 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:49:03.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:03 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1671946709' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:49:05.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:05 smithi094 ceph-mon[36708]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-06-10T00:49:05.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:05 smithi037 ceph-mon[32192]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-06-10T00:49:05.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:05 smithi167 ceph-mon[36917]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-06-10T00:49:07.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:07 smithi094 ceph-mon[36708]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-10T00:49:07.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:07 smithi037 ceph-mon[32192]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-10T00:49:07.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:07 smithi167 ceph-mon[36917]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-10T00:49:09.596 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:09 smithi037 ceph-mon[32192]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-10T00:49:09.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:09 smithi094 ceph-mon[36708]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-10T00:49:09.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:09 smithi167 ceph-mon[36917]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-10T00:49:10.600 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:10 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-10T00:49:10.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:10 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:49:10.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:10 smithi037 ceph-mon[32192]: Deploying daemon osd.6 on smithi037 2024-06-10T00:49:10.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:10 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-10T00:49:10.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:10 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:49:10.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:10 smithi094 ceph-mon[36708]: Deploying daemon osd.6 on smithi037 2024-06-10T00:49:10.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:10 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-10T00:49:10.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:10 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:49:10.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:10 smithi167 ceph-mon[36917]: Deploying daemon osd.6 on smithi037 2024-06-10T00:49:11.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:11 smithi167 ceph-mon[36917]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-06-10T00:49:11.616 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:11 smithi094 ceph-mon[36708]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-06-10T00:49:11.639 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:11 smithi037 ceph-mon[32192]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-06-10T00:49:13.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:13 smithi094 ceph-mon[36708]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T00:49:13.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:13 smithi037 ceph-mon[32192]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T00:49:13.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:13 smithi167 ceph-mon[36917]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T00:49:15.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:15 smithi094 ceph-mon[36708]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-06-10T00:49:15.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:15 smithi037 ceph-mon[32192]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-06-10T00:49:15.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:15 smithi167 ceph-mon[36917]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-06-10T00:49:16.529 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:16 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:49:16.529 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:16 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:16.530 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:16 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:16.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:16 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:49:16.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:16 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:16.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:16 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:16.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:16 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:49:16.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:16 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:16.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:16 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:17.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:17 smithi037 ceph-mon[32192]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:49:17.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:17 smithi167 ceph-mon[36917]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:49:17.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:17 smithi094 ceph-mon[36708]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:49:19.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:49:19.586+0000 7f1da1ffb700 1 -- 172.21.15.37:0/1824010807 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1da406b330 con 0x7f1d900a2ca0 2024-06-10T00:49:19.587 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 6 on host 'smithi037' 2024-06-10T00:49:19.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:49:19.588+0000 7f1daa864700 1 -- 172.21.15.37:0/1824010807 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1d900a2ca0 msgr2=0x7f1d900a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:49:19.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:49:19.588+0000 7f1daa864700 1 --2- 172.21.15.37:0/1824010807 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1d900a2ca0 0x7f1d900a5160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400b580 tx=0x7f1d94005fb0 comp rx=0 tx=0).stop 2024-06-10T00:49:19.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:49:19.588+0000 7f1daa864700 1 -- 172.21.15.37:0/1824010807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da4102f80 msgr2=0x7f1da4104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:49:19.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:49:19.588+0000 7f1daa864700 1 --2- 172.21.15.37:0/1824010807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da4102f80 0x7f1da4104050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c007ee0 tx=0x7f1d9c00d3b0 comp rx=0 tx=0).stop 2024-06-10T00:49:19.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:49:19.588+0000 7f1daa864700 1 -- 172.21.15.37:0/1824010807 shutdown_connections 2024-06-10T00:49:19.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:49:19.588+0000 7f1daa864700 1 --2- 172.21.15.37:0/1824010807 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1d900a2ca0 0x7f1d900a5160 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:49:19.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:49:19.589+0000 7f1daa864700 1 --2- 172.21.15.37:0/1824010807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da4102f80 0x7f1da4104050 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:49:19.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:49:19.589+0000 7f1daa864700 1 --2- 172.21.15.37:0/1824010807 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1da41013e0 0x7f1da41054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:49:19.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:49:19.589+0000 7f1daa864700 1 --2- 172.21.15.37:0/1824010807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1da41025e0 0x7f1da4103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:49:19.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:49:19.589+0000 7f1daa864700 1 -- 172.21.15.37:0/1824010807 >> 172.21.15.37:0/1824010807 conn(0x7f1da40fc980 msgr2=0x7f1da406ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:49:19.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:49:19.589+0000 7f1daa864700 1 -- 172.21.15.37:0/1824010807 shutdown_connections 2024-06-10T00:49:19.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:49:19.589+0000 7f1daa864700 1 -- 172.21.15.37:0/1824010807 wait complete. 2024-06-10T00:49:19.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:19 smithi094 ceph-mon[36708]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:49:19.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:19.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:19.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:49:19.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:49:19.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:19.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:19 smithi094 ceph-mon[36708]: from='osd.6 [v2:172.21.15.37:6818/472670875,v1:172.21.15.37:6819/472670875]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-10T00:49:19.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:19.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:19.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:49:19.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:19 smithi037 ceph-mon[32192]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:49:19.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:19 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:19.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:19 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:19.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:19 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:49:19.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:19 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:49:19.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:19 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:19.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:19 smithi037 ceph-mon[32192]: from='osd.6 [v2:172.21.15.37:6818/472670875,v1:172.21.15.37:6819/472670875]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-10T00:49:19.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:19 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:19.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:19 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:19.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:19 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:49:20.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:19 smithi167 ceph-mon[36917]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:49:20.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:19 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:20.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:19 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:20.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:19 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:49:20.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:19 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:49:20.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:19 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:20.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:19 smithi167 ceph-mon[36917]: from='osd.6 [v2:172.21.15.37:6818/472670875,v1:172.21.15.37:6819/472670875]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-10T00:49:20.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:19 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:20.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:19 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:20.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:19 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:49:20.371 DEBUG:teuthology.orchestra.run.smithi037:osd.6> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.6.service 2024-06-10T00:49:20.374 INFO:tasks.cephadm:Deploying osd.7 on smithi094 with /dev/vg_nvme/lv_2... 2024-06-10T00:49:20.374 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-10T00:49:20.619 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T00:49:20.741 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:20 smithi094 ceph-mon[36708]: from='osd.6 [v2:172.21.15.37:6818/472670875,v1:172.21.15.37:6819/472670875]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-10T00:49:20.741 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:20 smithi094 ceph-mon[36708]: osdmap e45: 7 total, 6 up, 7 in 2024-06-10T00:49:20.741 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:20 smithi094 ceph-mon[36708]: from='osd.6 [v2:172.21.15.37:6818/472670875,v1:172.21.15.37:6819/472670875]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-06-10T00:49:20.741 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:20 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:49:20.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:20 smithi037 ceph-mon[32192]: from='osd.6 [v2:172.21.15.37:6818/472670875,v1:172.21.15.37:6819/472670875]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-10T00:49:20.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:20 smithi037 ceph-mon[32192]: osdmap e45: 7 total, 6 up, 7 in 2024-06-10T00:49:20.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:20 smithi037 ceph-mon[32192]: from='osd.6 [v2:172.21.15.37:6818/472670875,v1:172.21.15.37:6819/472670875]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-06-10T00:49:20.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:20 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:49:20.897 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 00:49:20 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T00:49:20.621+0000 7fc77794a700 -1 osd.6 0 waiting for initial osdmap 2024-06-10T00:49:20.897 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 00:49:20 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T00:49:20.633+0000 7fc77373d700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:49:21.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:20 smithi167 ceph-mon[36917]: from='osd.6 [v2:172.21.15.37:6818/472670875,v1:172.21.15.37:6819/472670875]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-10T00:49:21.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:20 smithi167 ceph-mon[36917]: osdmap e45: 7 total, 6 up, 7 in 2024-06-10T00:49:21.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:20 smithi167 ceph-mon[36917]: from='osd.6 [v2:172.21.15.37:6818/472670875,v1:172.21.15.37:6819/472670875]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-06-10T00:49:21.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:20 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:49:21.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:21 smithi037 ceph-mon[32192]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:49:21.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:21 smithi037 ceph-mon[32192]: from='osd.6 [v2:172.21.15.37:6818/472670875,v1:172.21.15.37:6819/472670875]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-06-10T00:49:21.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:21 smithi037 ceph-mon[32192]: osdmap e46: 7 total, 6 up, 7 in 2024-06-10T00:49:21.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:49:21.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:49:22.005 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:21 smithi094 ceph-mon[36708]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:49:22.005 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:21 smithi094 ceph-mon[36708]: from='osd.6 [v2:172.21.15.37:6818/472670875,v1:172.21.15.37:6819/472670875]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-06-10T00:49:22.006 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:21 smithi094 ceph-mon[36708]: osdmap e46: 7 total, 6 up, 7 in 2024-06-10T00:49:22.006 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:49:22.006 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:49:22.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:21 smithi167 ceph-mon[36917]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:49:22.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:21 smithi167 ceph-mon[36917]: from='osd.6 [v2:172.21.15.37:6818/472670875,v1:172.21.15.37:6819/472670875]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-06-10T00:49:22.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:21 smithi167 ceph-mon[36917]: osdmap e46: 7 total, 6 up, 7 in 2024-06-10T00:49:22.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:49:22.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:49:22.688 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-10T00:49:22.688 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-10T00:49:22.688 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-06-10T00:49:22.688 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-06-10T00:49:22.688 INFO:teuthology.orchestra.run.smithi094.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00826013 s, 1.3 GB/s 2024-06-10T00:49:22.688 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-06-10T00:49:22.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:22 smithi037 ceph-mon[32192]: purged_snaps scrub starts 2024-06-10T00:49:22.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:22 smithi037 ceph-mon[32192]: purged_snaps scrub ok 2024-06-10T00:49:22.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:22 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:49:22.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:22 smithi037 ceph-mon[32192]: osd.6 [v2:172.21.15.37:6818/472670875,v1:172.21.15.37:6819/472670875] boot 2024-06-10T00:49:22.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:22 smithi037 ceph-mon[32192]: osdmap e47: 7 total, 7 up, 7 in 2024-06-10T00:49:22.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:22 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:49:22.988 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:22 smithi094 ceph-mon[36708]: purged_snaps scrub starts 2024-06-10T00:49:22.988 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:22 smithi094 ceph-mon[36708]: purged_snaps scrub ok 2024-06-10T00:49:22.988 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:22 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:49:22.989 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:22 smithi094 ceph-mon[36708]: osd.6 [v2:172.21.15.37:6818/472670875,v1:172.21.15.37:6819/472670875] boot 2024-06-10T00:49:22.989 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:22 smithi094 ceph-mon[36708]: osdmap e47: 7 total, 7 up, 7 in 2024-06-10T00:49:22.989 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:22 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:49:23.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:22 smithi167 ceph-mon[36917]: purged_snaps scrub starts 2024-06-10T00:49:23.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:22 smithi167 ceph-mon[36917]: purged_snaps scrub ok 2024-06-10T00:49:23.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:22 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:49:23.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:22 smithi167 ceph-mon[36917]: osd.6 [v2:172.21.15.37:6818/472670875,v1:172.21.15.37:6819/472670875] boot 2024-06-10T00:49:23.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:22 smithi167 ceph-mon[36917]: osdmap e47: 7 total, 7 up, 7 in 2024-06-10T00:49:23.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:22 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-10T00:49:23.402 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_2 2024-06-10T00:49:23.639 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T00:49:23.892 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:23 smithi094 ceph-mon[36708]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:49:23.892 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:23 smithi094 ceph-mon[36708]: osdmap e48: 7 total, 7 up, 7 in 2024-06-10T00:49:23.892 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:23 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:23.892 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:23 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:23.893 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:23 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:49:23.893 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:23 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:49:23.893 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:23 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:49:23.893 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:23 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:23.893 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:23 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:49:23.893 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:23 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:49:23.893 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:23 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:23.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:23 smithi037 ceph-mon[32192]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:49:23.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:23 smithi037 ceph-mon[32192]: osdmap e48: 7 total, 7 up, 7 in 2024-06-10T00:49:23.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:23 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:23.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:23 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:23.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:23 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:49:23.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:23 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:49:23.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:23 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:49:23.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:23 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:23.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:23 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:49:23.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:23 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:49:23.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:23 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:24.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:23 smithi167 ceph-mon[36917]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-06-10T00:49:24.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:23 smithi167 ceph-mon[36917]: osdmap e48: 7 total, 7 up, 7 in 2024-06-10T00:49:24.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:23 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:24.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:23 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:24.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:23 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:49:24.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:23 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:49:24.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:23 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:49:24.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:23 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:24.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:23 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:49:24.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:23 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:49:24.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:23 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:24.734 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:24 smithi094 ceph-mon[36708]: Detected new or changed devices on smithi037 2024-06-10T00:49:24.734 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:24 smithi094 ceph-mon[36708]: Adjusting osd_memory_target on smithi037 to 5697M 2024-06-10T00:49:24.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:24 smithi037 ceph-mon[32192]: Detected new or changed devices on smithi037 2024-06-10T00:49:24.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:24 smithi037 ceph-mon[32192]: Adjusting osd_memory_target on smithi037 to 5697M 2024-06-10T00:49:25.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:24 smithi167 ceph-mon[36917]: Detected new or changed devices on smithi037 2024-06-10T00:49:25.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:24 smithi167 ceph-mon[36917]: Adjusting osd_memory_target on smithi037 to 5697M 2024-06-10T00:49:25.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.300+0000 7f5b5ade4700 1 -- 172.21.15.94:0/3295758493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b54104060 msgr2=0x7f5b5410a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:49:25.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.300+0000 7f5b5ade4700 1 --2- 172.21.15.94:0/3295758493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b54104060 0x7f5b5410a910 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c00b0d0 tx=0x7f5b4c00b3e0 comp rx=0 tx=0).stop 2024-06-10T00:49:25.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.300+0000 7f5b5ade4700 1 -- 172.21.15.94:0/3295758493 shutdown_connections 2024-06-10T00:49:25.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.300+0000 7f5b5ade4700 1 --2- 172.21.15.94:0/3295758493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b54104060 0x7f5b5410a910 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:49:25.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.300+0000 7f5b5ade4700 1 --2- 172.21.15.94:0/3295758493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b541024c0 0x7f5b541028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:49:25.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.300+0000 7f5b5ade4700 1 --2- 172.21.15.94:0/3295758493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b541036c0 0x7f5b54103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:49:25.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.300+0000 7f5b5ade4700 1 -- 172.21.15.94:0/3295758493 >> 172.21.15.94:0/3295758493 conn(0x7f5b540fda60 msgr2=0x7f5b540ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:49:25.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.300+0000 7f5b5ade4700 1 -- 172.21.15.94:0/3295758493 shutdown_connections 2024-06-10T00:49:25.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.300+0000 7f5b5ade4700 1 -- 172.21.15.94:0/3295758493 wait complete. 2024-06-10T00:49:25.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.300+0000 7f5b5ade4700 1 Processor -- start 2024-06-10T00:49:25.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.301+0000 7f5b5ade4700 1 -- start start 2024-06-10T00:49:25.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.301+0000 7f5b5ade4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b541024c0 0x7f5b541986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:49:25.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.301+0000 7f5b5ade4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b541036c0 0x7f5b54198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:49:25.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.301+0000 7f5b5ade4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b54104060 0x7f5b5419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:49:25.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.301+0000 7f5b5ade4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5410e0e0 con 0x7f5b541036c0 2024-06-10T00:49:25.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.301+0000 7f5b5ade4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5410df60 con 0x7f5b541024c0 2024-06-10T00:49:25.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.301+0000 7f5b5ade4700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5410e260 con 0x7f5b54104060 2024-06-10T00:49:25.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.301+0000 7f5b58b80700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b541024c0 0x7f5b541986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:49:25.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.301+0000 7f5b58b80700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b541024c0 0x7f5b541986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58390/0 (socket says 172.21.15.94:58390) 2024-06-10T00:49:25.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.301+0000 7f5b58b80700 1 -- 172.21.15.94:0/1007323425 learned_addr learned my addr 172.21.15.94:0/1007323425 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-10T00:49:25.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.301+0000 7f5b53fff700 1 --2- 172.21.15.94:0/1007323425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b541036c0 0x7f5b54198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:49:25.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.301+0000 7f5b59381700 1 --2- 172.21.15.94:0/1007323425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b54104060 0x7f5b5419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:49:25.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.301+0000 7f5b58b80700 1 -- 172.21.15.94:0/1007323425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b54104060 msgr2=0x7f5b5419fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:49:25.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.301+0000 7f5b58b80700 1 --2- 172.21.15.94:0/1007323425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b54104060 0x7f5b5419fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:49:25.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.301+0000 7f5b58b80700 1 -- 172.21.15.94:0/1007323425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b541036c0 msgr2=0x7f5b54198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:49:25.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.301+0000 7f5b58b80700 1 --2- 172.21.15.94:0/1007323425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b541036c0 0x7f5b54198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:49:25.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.301+0000 7f5b58b80700 1 -- 172.21.15.94:0/1007323425 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b541a0300 con 0x7f5b541024c0 2024-06-10T00:49:25.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.301+0000 7f5b59381700 1 --2- 172.21.15.94:0/1007323425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b54104060 0x7f5b5419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:49:25.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.301+0000 7f5b58b80700 1 --2- 172.21.15.94:0/1007323425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b541024c0 0x7f5b541986a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800c970 tx=0x7f5b4800cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:49:25.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.301+0000 7f5b51ffb700 1 -- 172.21.15.94:0/1007323425 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48004220 con 0x7f5b541024c0 2024-06-10T00:49:25.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.302+0000 7f5b51ffb700 1 -- 172.21.15.94:0/1007323425 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b48007d80 con 0x7f5b541024c0 2024-06-10T00:49:25.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.302+0000 7f5b5ade4700 1 -- 172.21.15.94:0/1007323425 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b541a05f0 con 0x7f5b541024c0 2024-06-10T00:49:25.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.302+0000 7f5b51ffb700 1 -- 172.21.15.94:0/1007323425 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48003df0 con 0x7f5b541024c0 2024-06-10T00:49:25.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.302+0000 7f5b5ade4700 1 -- 172.21.15.94:0/1007323425 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b541a0b30 con 0x7f5b541024c0 2024-06-10T00:49:25.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.303+0000 7f5b5ade4700 1 -- 172.21.15.94:0/1007323425 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b54069180 con 0x7f5b541024c0 2024-06-10T00:49:25.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.304+0000 7f5b51ffb700 1 -- 172.21.15.94:0/1007323425 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5b480078a0 con 0x7f5b541024c0 2024-06-10T00:49:25.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.304+0000 7f5b51ffb700 1 --2- 172.21.15.94:0/1007323425 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5b440a2c20 0x7f5b440a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:49:25.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.305+0000 7f5b51ffb700 1 -- 172.21.15.94:0/1007323425 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f5b480cded0 con 0x7f5b541024c0 2024-06-10T00:49:25.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.305+0000 7f5b53fff700 1 --2- 172.21.15.94:0/1007323425 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5b440a2c20 0x7f5b440a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:49:25.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.305+0000 7f5b53fff700 1 --2- 172.21.15.94:0/1007323425 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5b440a2c20 0x7f5b440a50e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5b40009c60 tx=0x7f5b40009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:49:25.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.309+0000 7f5b51ffb700 1 -- 172.21.15.94:0/1007323425 <== mon.2 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5b48014030 con 0x7f5b541024c0 2024-06-10T00:49:25.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:25.512+0000 7f5b5ade4700 1 -- 172.21.15.94:0/1007323425 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5b540628d0 con 0x7f5b440a2c20 2024-06-10T00:49:25.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:25 smithi037 ceph-mon[32192]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:25.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:25 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:49:25.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:25 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:49:25.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:25 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:49:26.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:25 smithi167 ceph-mon[36917]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:26.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:25 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:49:26.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:25 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:49:26.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:25 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:49:26.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:25 smithi094 ceph-mon[36708]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:26.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:25 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:49:26.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:25 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:49:26.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:25 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:49:26.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:26 smithi094 ceph-mon[36708]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:49:27.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:26 smithi167 ceph-mon[36917]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:49:27.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:26 smithi037 ceph-mon[32192]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:49:27.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:27 smithi094 ceph-mon[36708]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:28.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:27 smithi167 ceph-mon[36917]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:28.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:27 smithi037 ceph-mon[32192]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:29.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:28 smithi167 ceph-mon[36917]: from='client.? 172.21.15.94:0/1983205137' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed829ed3-ba85-4e82-ac37-e44e9534d37c"}]: dispatch 2024-06-10T00:49:29.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:28 smithi167 ceph-mon[36917]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed829ed3-ba85-4e82-ac37-e44e9534d37c"}]: dispatch 2024-06-10T00:49:29.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:28 smithi167 ceph-mon[36917]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed829ed3-ba85-4e82-ac37-e44e9534d37c"}]': finished 2024-06-10T00:49:29.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:28 smithi167 ceph-mon[36917]: osdmap e49: 8 total, 7 up, 8 in 2024-06-10T00:49:29.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:28 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:49:29.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:28 smithi094 ceph-mon[36708]: from='client.? 172.21.15.94:0/1983205137' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed829ed3-ba85-4e82-ac37-e44e9534d37c"}]: dispatch 2024-06-10T00:49:29.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:28 smithi094 ceph-mon[36708]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed829ed3-ba85-4e82-ac37-e44e9534d37c"}]: dispatch 2024-06-10T00:49:29.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:28 smithi094 ceph-mon[36708]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed829ed3-ba85-4e82-ac37-e44e9534d37c"}]': finished 2024-06-10T00:49:29.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:28 smithi094 ceph-mon[36708]: osdmap e49: 8 total, 7 up, 8 in 2024-06-10T00:49:29.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:28 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:49:29.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:28 smithi037 ceph-mon[32192]: from='client.? 172.21.15.94:0/1983205137' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed829ed3-ba85-4e82-ac37-e44e9534d37c"}]: dispatch 2024-06-10T00:49:29.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:28 smithi037 ceph-mon[32192]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed829ed3-ba85-4e82-ac37-e44e9534d37c"}]: dispatch 2024-06-10T00:49:29.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:28 smithi037 ceph-mon[32192]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed829ed3-ba85-4e82-ac37-e44e9534d37c"}]': finished 2024-06-10T00:49:29.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:28 smithi037 ceph-mon[32192]: osdmap e49: 8 total, 7 up, 8 in 2024-06-10T00:49:29.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:28 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:49:30.015 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:29 smithi094 ceph-mon[36708]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:30.016 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:29 smithi094 ceph-mon[36708]: from='client.? 172.21.15.94:0/2277649222' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:49:30.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:29 smithi167 ceph-mon[36917]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:30.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:29 smithi167 ceph-mon[36917]: from='client.? 172.21.15.94:0/2277649222' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:49:30.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:29 smithi037 ceph-mon[32192]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:30.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:29 smithi037 ceph-mon[32192]: from='client.? 172.21.15.94:0/2277649222' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:49:31.942 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:31 smithi094 ceph-mon[36708]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:32.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:31 smithi167 ceph-mon[36917]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:32.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:31 smithi037 ceph-mon[32192]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:34.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:33 smithi167 ceph-mon[36917]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:34.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:33 smithi094 ceph-mon[36708]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:34.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:33 smithi037 ceph-mon[32192]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:36.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:35 smithi167 ceph-mon[36917]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:36.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:35 smithi094 ceph-mon[36708]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:36.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:35 smithi037 ceph-mon[32192]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:37.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:36 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-10T00:49:37.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:36 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:49:37.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:36 smithi094 ceph-mon[36708]: Deploying daemon osd.7 on smithi094 2024-06-10T00:49:37.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:36 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-10T00:49:37.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:36 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:49:37.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:36 smithi167 ceph-mon[36917]: Deploying daemon osd.7 on smithi094 2024-06-10T00:49:37.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:36 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-10T00:49:37.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:36 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:49:37.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:36 smithi037 ceph-mon[32192]: Deploying daemon osd.7 on smithi094 2024-06-10T00:49:37.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:37 smithi094 ceph-mon[36708]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:38.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:37 smithi167 ceph-mon[36917]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:38.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:37 smithi037 ceph-mon[32192]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:39.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:39 smithi037 ceph-mon[32192]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:39.985 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:39 smithi094 ceph-mon[36708]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:40.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:39 smithi167 ceph-mon[36917]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:41.967 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:41 smithi094 ceph-mon[36708]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:41.967 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:41 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:41.968 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:41 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:41.968 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:41 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:49:42.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:41 smithi167 ceph-mon[36917]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:42.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:41 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:42.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:41 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:42.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:41 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:49:42.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:41 smithi037 ceph-mon[32192]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:42.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:42.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:42.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:49:43.816 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:43 smithi094 ceph-mon[36708]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:44.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:43 smithi167 ceph-mon[36917]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:44.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:43 smithi037 ceph-mon[32192]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:45.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:45.078+0000 7f5b51ffb700 1 -- 172.21.15.94:0/1007323425 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5b540628d0 con 0x7f5b440a2c20 2024-06-10T00:49:45.080 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 7 on host 'smithi094' 2024-06-10T00:49:45.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:45.081+0000 7f5b5ade4700 1 -- 172.21.15.94:0/1007323425 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5b440a2c20 msgr2=0x7f5b440a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:49:45.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:45.081+0000 7f5b5ade4700 1 --2- 172.21.15.94:0/1007323425 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5b440a2c20 0x7f5b440a50e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5b40009c60 tx=0x7f5b40009260 comp rx=0 tx=0).stop 2024-06-10T00:49:45.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:45.081+0000 7f5b5ade4700 1 -- 172.21.15.94:0/1007323425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b541024c0 msgr2=0x7f5b541986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:49:45.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:45.081+0000 7f5b5ade4700 1 --2- 172.21.15.94:0/1007323425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b541024c0 0x7f5b541986a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800c970 tx=0x7f5b4800cc80 comp rx=0 tx=0).stop 2024-06-10T00:49:45.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:45.081+0000 7f5b5ade4700 1 -- 172.21.15.94:0/1007323425 shutdown_connections 2024-06-10T00:49:45.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:45.081+0000 7f5b5ade4700 1 --2- 172.21.15.94:0/1007323425 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5b440a2c20 0x7f5b440a50e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:49:45.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:45.081+0000 7f5b5ade4700 1 --2- 172.21.15.94:0/1007323425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b54104060 0x7f5b5419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:49:45.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:45.081+0000 7f5b5ade4700 1 --2- 172.21.15.94:0/1007323425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b541024c0 0x7f5b541986a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:49:45.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:45.081+0000 7f5b5ade4700 1 --2- 172.21.15.94:0/1007323425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b541036c0 0x7f5b54198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:49:45.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:45.081+0000 7f5b5ade4700 1 -- 172.21.15.94:0/1007323425 >> 172.21.15.94:0/1007323425 conn(0x7f5b540fda60 msgr2=0x7f5b541069c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:49:45.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:45.081+0000 7f5b5ade4700 1 -- 172.21.15.94:0/1007323425 shutdown_connections 2024-06-10T00:49:45.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:49:45.081+0000 7f5b5ade4700 1 -- 172.21.15.94:0/1007323425 wait complete. 2024-06-10T00:49:45.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:45 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:45.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:45 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:45.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:45 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:49:45.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:45 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:49:45.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:45 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:45.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:45 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:45.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:45 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:45.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:45 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:49:45.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:45 smithi094 ceph-mon[36708]: from='osd.7 [v2:172.21.15.94:6816/3932281552,v1:172.21.15.94:6817/3932281552]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-10T00:49:45.367 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:45 smithi094 ceph-mon[36708]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-10T00:49:45.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:45 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:45.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:45 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:45.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:45 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:49:45.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:45 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:49:45.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:45 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:45.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:45 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:45.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:45 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:45.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:45 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:49:45.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:45 smithi167 ceph-mon[36917]: from='osd.7 [v2:172.21.15.94:6816/3932281552,v1:172.21.15.94:6817/3932281552]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-10T00:49:45.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:45 smithi167 ceph-mon[36917]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-10T00:49:45.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:45 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:45.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:45 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:45.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:45 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:49:45.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:45 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:49:45.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:45 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:45.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:45 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:45.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:45 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:45.647 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:45 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:49:45.647 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:45 smithi037 ceph-mon[32192]: from='osd.7 [v2:172.21.15.94:6816/3932281552,v1:172.21.15.94:6817/3932281552]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-10T00:49:45.647 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:45 smithi037 ceph-mon[32192]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-10T00:49:45.766 DEBUG:teuthology.orchestra.run.smithi094:osd.7> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.7.service 2024-06-10T00:49:45.768 INFO:tasks.cephadm:Deploying osd.8 on smithi167 with /dev/vg_nvme/lv_2... 2024-06-10T00:49:45.768 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-10T00:49:46.015 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:49:46.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:46 smithi094 ceph-mon[36708]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:46.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:46 smithi094 ceph-mon[36708]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-10T00:49:46.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:46 smithi094 ceph-mon[36708]: from='osd.7 [v2:172.21.15.94:6816/3932281552,v1:172.21.15.94:6817/3932281552]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-10T00:49:46.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:46 smithi094 ceph-mon[36708]: osdmap e50: 8 total, 7 up, 8 in 2024-06-10T00:49:46.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:46 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:49:46.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:46 smithi094 ceph-mon[36708]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-10T00:49:46.449 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:46 smithi167 ceph-mon[36917]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:46.450 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:46 smithi167 ceph-mon[36917]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-10T00:49:46.450 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:46 smithi167 ceph-mon[36917]: from='osd.7 [v2:172.21.15.94:6816/3932281552,v1:172.21.15.94:6817/3932281552]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-10T00:49:46.450 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:46 smithi167 ceph-mon[36917]: osdmap e50: 8 total, 7 up, 8 in 2024-06-10T00:49:46.450 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:46 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:49:46.450 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:46 smithi167 ceph-mon[36917]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-10T00:49:46.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:46 smithi037 ceph-mon[32192]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:46.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:46 smithi037 ceph-mon[32192]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-10T00:49:46.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:46 smithi037 ceph-mon[32192]: from='osd.7 [v2:172.21.15.94:6816/3932281552,v1:172.21.15.94:6817/3932281552]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-10T00:49:46.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:46 smithi037 ceph-mon[32192]: osdmap e50: 8 total, 7 up, 8 in 2024-06-10T00:49:46.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:46 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:49:46.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:46 smithi037 ceph-mon[32192]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-10T00:49:47.865 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 00:49:47 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T00:49:47.530+0000 7f5d46ea2700 -1 osd.7 0 waiting for initial osdmap 2024-06-10T00:49:47.865 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 00:49:47 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T00:49:47.541+0000 7f5d3ec8d700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:49:48.077 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-10T00:49:48.077 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-10T00:49:48.077 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-06-10T00:49:48.077 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-06-10T00:49:48.077 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0141295 s, 742 MB/s 2024-06-10T00:49:48.078 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-06-10T00:49:48.102 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:48 smithi167 ceph-mon[36917]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:48.103 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:48 smithi167 ceph-mon[36917]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-06-10T00:49:48.103 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:48 smithi167 ceph-mon[36917]: osdmap e51: 8 total, 7 up, 8 in 2024-06-10T00:49:48.103 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:48 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:49:48.103 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:48 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:49:48.103 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:48 smithi167 ceph-mon[36917]: from='osd.7 ' entity='osd.7' 2024-06-10T00:49:48.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:48 smithi094 ceph-mon[36708]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:48.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:48 smithi094 ceph-mon[36708]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-06-10T00:49:48.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:48 smithi094 ceph-mon[36708]: osdmap e51: 8 total, 7 up, 8 in 2024-06-10T00:49:48.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:48 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:49:48.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:48 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:49:48.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:48 smithi094 ceph-mon[36708]: from='osd.7 ' entity='osd.7' 2024-06-10T00:49:48.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:48 smithi037 ceph-mon[32192]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:48.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:48 smithi037 ceph-mon[32192]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-06-10T00:49:48.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:48 smithi037 ceph-mon[32192]: osdmap e51: 8 total, 7 up, 8 in 2024-06-10T00:49:48.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:48 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:49:48.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:48 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:49:48.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:48 smithi037 ceph-mon[32192]: from='osd.7 ' entity='osd.7' 2024-06-10T00:49:48.806 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_2 2024-06-10T00:49:49.052 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:49:49.301 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:49 smithi094 ceph-mon[36708]: purged_snaps scrub starts 2024-06-10T00:49:49.301 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:49 smithi094 ceph-mon[36708]: purged_snaps scrub ok 2024-06-10T00:49:49.301 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:49 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:49:49.301 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:49 smithi094 ceph-mon[36708]: osd.7 [v2:172.21.15.94:6816/3932281552,v1:172.21.15.94:6817/3932281552] boot 2024-06-10T00:49:49.301 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:49 smithi094 ceph-mon[36708]: osdmap e52: 8 total, 8 up, 8 in 2024-06-10T00:49:49.301 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:49 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:49:49.301 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:49 smithi094 ceph-mon[36708]: osdmap e53: 8 total, 8 up, 8 in 2024-06-10T00:49:49.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:49 smithi167 ceph-mon[36917]: purged_snaps scrub starts 2024-06-10T00:49:49.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:49 smithi167 ceph-mon[36917]: purged_snaps scrub ok 2024-06-10T00:49:49.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:49 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:49:49.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:49 smithi167 ceph-mon[36917]: osd.7 [v2:172.21.15.94:6816/3932281552,v1:172.21.15.94:6817/3932281552] boot 2024-06-10T00:49:49.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:49 smithi167 ceph-mon[36917]: osdmap e52: 8 total, 8 up, 8 in 2024-06-10T00:49:49.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:49 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:49:49.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:49 smithi167 ceph-mon[36917]: osdmap e53: 8 total, 8 up, 8 in 2024-06-10T00:49:49.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:49 smithi037 ceph-mon[32192]: purged_snaps scrub starts 2024-06-10T00:49:49.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:49 smithi037 ceph-mon[32192]: purged_snaps scrub ok 2024-06-10T00:49:49.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:49 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:49:49.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:49 smithi037 ceph-mon[32192]: osd.7 [v2:172.21.15.94:6816/3932281552,v1:172.21.15.94:6817/3932281552] boot 2024-06-10T00:49:49.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:49 smithi037 ceph-mon[32192]: osdmap e52: 8 total, 8 up, 8 in 2024-06-10T00:49:49.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:49 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-10T00:49:49.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:49 smithi037 ceph-mon[32192]: osdmap e53: 8 total, 8 up, 8 in 2024-06-10T00:49:50.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:50 smithi167 ceph-mon[36917]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:50.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:50 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:50.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:50 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:50.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:50 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:49:50.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:50 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:49:50.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:50 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:49:50.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:50 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:50.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:50 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:49:50.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:50 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:49:50.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:50 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:50.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:50 smithi167 ceph-mon[36917]: osdmap e54: 8 total, 8 up, 8 in 2024-06-10T00:49:50.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:50 smithi094 ceph-mon[36708]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:50.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:50 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:50.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:50 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:50.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:50 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:49:50.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:50 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:49:50.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:50 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:49:50.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:50 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:50.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:50 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:49:50.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:50 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:49:50.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:50 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:50.366 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:50 smithi094 ceph-mon[36708]: osdmap e54: 8 total, 8 up, 8 in 2024-06-10T00:49:50.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:50 smithi037 ceph-mon[32192]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-06-10T00:49:50.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:50 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:50.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:50 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:50.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:50 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:49:50.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:50 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:49:50.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:50 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:49:50.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:50 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:50.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:50 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:49:50.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:50 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:49:50.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:50 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:49:50.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:50 smithi037 ceph-mon[32192]: osdmap e54: 8 total, 8 up, 8 in 2024-06-10T00:49:50.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.684+0000 7ff20f1a4700 1 -- 172.21.15.167:0/3114758051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2081025e0 msgr2=0x7ff208102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:49:50.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.684+0000 7ff20f1a4700 1 --2- 172.21.15.167:0/3114758051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2081025e0 0x7ff208102a40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009990 tx=0x7ff1f4009ca0 comp rx=0 tx=0).stop 2024-06-10T00:49:50.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.684+0000 7ff20f1a4700 1 -- 172.21.15.167:0/3114758051 shutdown_connections 2024-06-10T00:49:50.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.684+0000 7ff20f1a4700 1 --2- 172.21.15.167:0/3114758051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff208102f80 0x7ff208109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:49:50.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.684+0000 7ff20f1a4700 1 --2- 172.21.15.167:0/3114758051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2081013e0 0x7ff208101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:49:50.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.684+0000 7ff20f1a4700 1 --2- 172.21.15.167:0/3114758051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2081025e0 0x7ff208102a40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:49:50.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.684+0000 7ff20f1a4700 1 -- 172.21.15.167:0/3114758051 >> 172.21.15.167:0/3114758051 conn(0x7ff2080fc980 msgr2=0x7ff2080fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:49:50.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.684+0000 7ff20f1a4700 1 -- 172.21.15.167:0/3114758051 shutdown_connections 2024-06-10T00:49:50.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.684+0000 7ff20f1a4700 1 -- 172.21.15.167:0/3114758051 wait complete. 2024-06-10T00:49:50.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.685+0000 7ff20f1a4700 1 Processor -- start 2024-06-10T00:49:50.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.685+0000 7ff20f1a4700 1 -- start start 2024-06-10T00:49:50.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.685+0000 7ff20f1a4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2081013e0 0x7ff2081975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:49:50.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.685+0000 7ff20f1a4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2081025e0 0x7ff208197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:49:50.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.685+0000 7ff20f1a4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff208102f80 0x7ff20819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:49:50.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.685+0000 7ff20f1a4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff20810d020 con 0x7ff208102f80 2024-06-10T00:49:50.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.685+0000 7ff20f1a4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff20810cea0 con 0x7ff2081025e0 2024-06-10T00:49:50.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.685+0000 7ff20f1a4700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff20810d1a0 con 0x7ff2081013e0 2024-06-10T00:49:50.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.685+0000 7ff20cf40700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2081013e0 0x7ff2081975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:49:50.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.685+0000 7ff20cf40700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2081013e0 0x7ff2081975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:33668/0 (socket says 172.21.15.167:33668) 2024-06-10T00:49:50.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.685+0000 7ff20cf40700 1 -- 172.21.15.167:0/382558918 learned_addr learned my addr 172.21.15.167:0/382558918 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-10T00:49:50.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.685+0000 7ff207fff700 1 --2- 172.21.15.167:0/382558918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2081025e0 0x7ff208197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:49:50.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.685+0000 7ff20d741700 1 --2- 172.21.15.167:0/382558918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff208102f80 0x7ff20819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:49:50.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.685+0000 7ff20cf40700 1 -- 172.21.15.167:0/382558918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2081025e0 msgr2=0x7ff208197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:49:50.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.685+0000 7ff20cf40700 1 --2- 172.21.15.167:0/382558918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2081025e0 0x7ff208197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:49:50.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.685+0000 7ff20cf40700 1 -- 172.21.15.167:0/382558918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff208102f80 msgr2=0x7ff20819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:49:50.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.685+0000 7ff20cf40700 1 --2- 172.21.15.167:0/382558918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff208102f80 0x7ff20819eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:49:50.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.685+0000 7ff20cf40700 1 -- 172.21.15.167:0/382558918 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff20819f230 con 0x7ff2081013e0 2024-06-10T00:49:50.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.685+0000 7ff20d741700 1 --2- 172.21.15.167:0/382558918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff208102f80 0x7ff20819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:49:50.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.686+0000 7ff20cf40700 1 --2- 172.21.15.167:0/382558918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2081013e0 0x7ff2081975d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009920 tx=0x7ff1fc009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:49:50.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.686+0000 7ff205ffb700 1 -- 172.21.15.167:0/382558918 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc00d020 con 0x7ff2081013e0 2024-06-10T00:49:50.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.686+0000 7ff205ffb700 1 -- 172.21.15.167:0/382558918 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1fc00d1c0 con 0x7ff2081013e0 2024-06-10T00:49:50.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.686+0000 7ff205ffb700 1 -- 172.21.15.167:0/382558918 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc00ca40 con 0x7ff2081013e0 2024-06-10T00:49:50.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.686+0000 7ff20f1a4700 1 -- 172.21.15.167:0/382558918 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff20806aa60 con 0x7ff2081013e0 2024-06-10T00:49:50.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.686+0000 7ff20f1a4700 1 -- 172.21.15.167:0/382558918 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff20806af50 con 0x7ff2081013e0 2024-06-10T00:49:50.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.687+0000 7ff20f1a4700 1 -- 172.21.15.167:0/382558918 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff208069180 con 0x7ff2081013e0 2024-06-10T00:49:50.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.688+0000 7ff205ffb700 1 -- 172.21.15.167:0/382558918 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff1fc00cbe0 con 0x7ff2081013e0 2024-06-10T00:49:50.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.689+0000 7ff205ffb700 1 --2- 172.21.15.167:0/382558918 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff1f80a2ca0 0x7ff1f80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:49:50.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.689+0000 7ff205ffb700 1 -- 172.21.15.167:0/382558918 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7ff1fc0cfc50 con 0x7ff2081013e0 2024-06-10T00:49:50.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.689+0000 7ff207fff700 1 --2- 172.21.15.167:0/382558918 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff1f80a2ca0 0x7ff1f80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:49:50.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.689+0000 7ff207fff700 1 --2- 172.21.15.167:0/382558918 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff1f80a2ca0 0x7ff1f80a5160 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff1f400b5c0 tx=0x7ff1f4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:49:50.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.692+0000 7ff205ffb700 1 -- 172.21.15.167:0/382558918 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff1fc09d6f0 con 0x7ff2081013e0 2024-06-10T00:49:50.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:49:50.897+0000 7ff20f1a4700 1 -- 172.21.15.167:0/382558918 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff20806b1b0 con 0x7ff1f80a2ca0 2024-06-10T00:49:51.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:51 smithi167 ceph-mon[36917]: Detected new or changed devices on smithi094 2024-06-10T00:49:51.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:51 smithi167 ceph-mon[36917]: Adjusting osd_memory_target on smithi094 to 5696M 2024-06-10T00:49:51.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:51 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:49:51.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:51 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:49:51.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:51 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:49:51.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:51 smithi094 ceph-mon[36708]: Detected new or changed devices on smithi094 2024-06-10T00:49:51.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:51 smithi094 ceph-mon[36708]: Adjusting osd_memory_target on smithi094 to 5696M 2024-06-10T00:49:51.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:51 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:49:51.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:51 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:49:51.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:51 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:49:51.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:51 smithi037 ceph-mon[32192]: Detected new or changed devices on smithi094 2024-06-10T00:49:51.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:51 smithi037 ceph-mon[32192]: Adjusting osd_memory_target on smithi094 to 5696M 2024-06-10T00:49:51.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:51 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:49:51.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:51 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:49:51.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:51 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:49:52.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:52 smithi094 ceph-mon[36708]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:49:52.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:52 smithi094 ceph-mon[36708]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:49:52.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:52 smithi094 ceph-mon[36708]: osdmap e55: 8 total, 8 up, 8 in 2024-06-10T00:49:52.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:52 smithi037 ceph-mon[32192]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:49:52.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:52 smithi037 ceph-mon[32192]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:49:52.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:52 smithi037 ceph-mon[32192]: osdmap e55: 8 total, 8 up, 8 in 2024-06-10T00:49:52.489 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:52 smithi167 ceph-mon[36917]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:49:52.490 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:52 smithi167 ceph-mon[36917]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:49:52.490 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:52 smithi167 ceph-mon[36917]: osdmap e55: 8 total, 8 up, 8 in 2024-06-10T00:49:53.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:53 smithi094 ceph-mon[36708]: osdmap e56: 8 total, 8 up, 8 in 2024-06-10T00:49:53.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:53 smithi037 ceph-mon[32192]: osdmap e56: 8 total, 8 up, 8 in 2024-06-10T00:49:53.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:53 smithi167 ceph-mon[36917]: osdmap e56: 8 total, 8 up, 8 in 2024-06-10T00:49:54.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:54 smithi167 ceph-mon[36917]: pgmap v182: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:49:54.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:54 smithi167 ceph-mon[36917]: from='client.? 172.21.15.167:0/2715945338' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d6267d8-72f4-4704-b554-bafc52d11a11"}]: dispatch 2024-06-10T00:49:54.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:54 smithi167 ceph-mon[36917]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d6267d8-72f4-4704-b554-bafc52d11a11"}]: dispatch 2024-06-10T00:49:54.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:54 smithi167 ceph-mon[36917]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d6267d8-72f4-4704-b554-bafc52d11a11"}]': finished 2024-06-10T00:49:54.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:54 smithi167 ceph-mon[36917]: osdmap e57: 9 total, 8 up, 9 in 2024-06-10T00:49:54.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:54 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:49:54.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:54 smithi094 ceph-mon[36708]: pgmap v182: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:49:54.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:54 smithi094 ceph-mon[36708]: from='client.? 172.21.15.167:0/2715945338' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d6267d8-72f4-4704-b554-bafc52d11a11"}]: dispatch 2024-06-10T00:49:54.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:54 smithi094 ceph-mon[36708]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d6267d8-72f4-4704-b554-bafc52d11a11"}]: dispatch 2024-06-10T00:49:54.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:54 smithi094 ceph-mon[36708]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d6267d8-72f4-4704-b554-bafc52d11a11"}]': finished 2024-06-10T00:49:54.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:54 smithi094 ceph-mon[36708]: osdmap e57: 9 total, 8 up, 9 in 2024-06-10T00:49:54.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:54 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:49:54.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:54 smithi037 ceph-mon[32192]: pgmap v182: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:49:54.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:54 smithi037 ceph-mon[32192]: from='client.? 172.21.15.167:0/2715945338' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d6267d8-72f4-4704-b554-bafc52d11a11"}]: dispatch 2024-06-10T00:49:54.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:54 smithi037 ceph-mon[32192]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d6267d8-72f4-4704-b554-bafc52d11a11"}]: dispatch 2024-06-10T00:49:54.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:54 smithi037 ceph-mon[32192]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d6267d8-72f4-4704-b554-bafc52d11a11"}]': finished 2024-06-10T00:49:54.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:54 smithi037 ceph-mon[32192]: osdmap e57: 9 total, 8 up, 9 in 2024-06-10T00:49:54.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:54 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:49:55.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:55 smithi094 ceph-mon[36708]: from='client.? 172.21.15.167:0/3581861377' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:49:55.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:55 smithi037 ceph-mon[32192]: from='client.? 172.21.15.167:0/3581861377' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:49:55.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:55 smithi167 ceph-mon[36917]: from='client.? 172.21.15.167:0/3581861377' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:49:56.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:56 smithi094 ceph-mon[36708]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:49:56.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:56 smithi037 ceph-mon[32192]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:49:56.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:56 smithi167 ceph-mon[36917]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:49:58.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:49:58 smithi167 ceph-mon[36917]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:49:58.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:49:58 smithi094 ceph-mon[36708]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:49:58.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:49:58 smithi037 ceph-mon[32192]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:50:00.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:00 smithi167 ceph-mon[36917]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:50:00.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:00 smithi167 ceph-mon[36917]: overall HEALTH_OK 2024-06-10T00:50:00.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:00 smithi094 ceph-mon[36708]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:50:00.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:00 smithi094 ceph-mon[36708]: overall HEALTH_OK 2024-06-10T00:50:00.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:00 smithi037 ceph-mon[32192]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:50:00.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:00 smithi037 ceph-mon[32192]: overall HEALTH_OK 2024-06-10T00:50:01.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:01 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-10T00:50:01.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:01 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:50:01.372 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:01 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-10T00:50:01.372 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:01 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:50:01.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:01 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-10T00:50:01.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:01 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:50:02.289 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:02 smithi167 ceph-mon[36917]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:50:02.289 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:02 smithi167 ceph-mon[36917]: Deploying daemon osd.8 on smithi167 2024-06-10T00:50:02.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:02 smithi037 ceph-mon[32192]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:50:02.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:02 smithi037 ceph-mon[32192]: Deploying daemon osd.8 on smithi167 2024-06-10T00:50:02.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:02 smithi094 ceph-mon[36708]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:50:02.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:02 smithi094 ceph-mon[36708]: Deploying daemon osd.8 on smithi167 2024-06-10T00:50:04.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:04 smithi037 ceph-mon[32192]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:50:04.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:04 smithi167 ceph-mon[36917]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:50:04.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:04 smithi094 ceph-mon[36708]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:50:06.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:06 smithi167 ceph-mon[36917]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:50:06.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:06 smithi037 ceph-mon[32192]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:50:06.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:06 smithi094 ceph-mon[36708]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:50:08.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:07 smithi167 ceph-mon[36917]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:50:08.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:07 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:08.342 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:07 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:08.342 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:07 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:50:08.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:07 smithi094 ceph-mon[36708]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:50:08.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:07 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:08.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:07 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:08.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:07 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:50:08.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:07 smithi037 ceph-mon[32192]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:50:08.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:07 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:08.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:07 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:08.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:07 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:50:10.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:09 smithi037 ceph-mon[32192]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:50:10.233 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:09 smithi167 ceph-mon[36917]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:50:10.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:09 smithi094 ceph-mon[36708]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:50:10.567 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 8 on host 'smithi167' 2024-06-10T00:50:10.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:50:10.567+0000 7ff205ffb700 1 -- 172.21.15.167:0/382558918 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff20806b1b0 con 0x7ff1f80a2ca0 2024-06-10T00:50:10.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:50:10.569+0000 7ff20f1a4700 1 -- 172.21.15.167:0/382558918 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff1f80a2ca0 msgr2=0x7ff1f80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:50:10.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:50:10.569+0000 7ff20f1a4700 1 --2- 172.21.15.167:0/382558918 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff1f80a2ca0 0x7ff1f80a5160 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff1f400b5c0 tx=0x7ff1f4005fb0 comp rx=0 tx=0).stop 2024-06-10T00:50:10.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:50:10.569+0000 7ff20f1a4700 1 -- 172.21.15.167:0/382558918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2081013e0 msgr2=0x7ff2081975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:50:10.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:50:10.569+0000 7ff20f1a4700 1 --2- 172.21.15.167:0/382558918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2081013e0 0x7ff2081975d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009920 tx=0x7ff1fc009c30 comp rx=0 tx=0).stop 2024-06-10T00:50:10.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:50:10.569+0000 7ff20f1a4700 1 -- 172.21.15.167:0/382558918 shutdown_connections 2024-06-10T00:50:10.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:50:10.569+0000 7ff20f1a4700 1 --2- 172.21.15.167:0/382558918 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff1f80a2ca0 0x7ff1f80a5160 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:50:10.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:50:10.569+0000 7ff20f1a4700 1 --2- 172.21.15.167:0/382558918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff208102f80 0x7ff20819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:50:10.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:50:10.569+0000 7ff20f1a4700 1 --2- 172.21.15.167:0/382558918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2081013e0 0x7ff2081975d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:50:10.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:50:10.569+0000 7ff20f1a4700 1 --2- 172.21.15.167:0/382558918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2081025e0 0x7ff208197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:50:10.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:50:10.569+0000 7ff20f1a4700 1 -- 172.21.15.167:0/382558918 >> 172.21.15.167:0/382558918 conn(0x7ff2080fc980 msgr2=0x7ff2081058e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:50:10.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:50:10.569+0000 7ff20f1a4700 1 -- 172.21.15.167:0/382558918 shutdown_connections 2024-06-10T00:50:10.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:50:10.569+0000 7ff20f1a4700 1 -- 172.21.15.167:0/382558918 wait complete. 2024-06-10T00:50:11.312 DEBUG:teuthology.orchestra.run.smithi167:osd.8> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.8.service 2024-06-10T00:50:11.315 INFO:tasks.cephadm:Deploying osd.9 on smithi037 with /dev/vg_nvme/lv_1... 2024-06-10T00:50:11.315 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-10T00:50:11.578 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:50:11.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:11 smithi037 ceph-mon[32192]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:50:11.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:11 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:11.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:11 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:11.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:11 smithi037 ceph-mon[32192]: from='osd.8 [v2:172.21.15.167:6816/2290216047,v1:172.21.15.167:6817/2290216047]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-10T00:50:11.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:11 smithi037 ceph-mon[32192]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-10T00:50:11.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:11 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:11.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:11 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:11.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:11 smithi167 ceph-mon[36917]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:50:11.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:11 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:11.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:11 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:11.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:11 smithi167 ceph-mon[36917]: from='osd.8 [v2:172.21.15.167:6816/2290216047,v1:172.21.15.167:6817/2290216047]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-10T00:50:11.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:11 smithi167 ceph-mon[36917]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-10T00:50:11.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:11 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:11.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:11 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:11.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:11 smithi094 ceph-mon[36708]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:50:11.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:11 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:11.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:11 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:11.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:11 smithi094 ceph-mon[36708]: from='osd.8 [v2:172.21.15.167:6816/2290216047,v1:172.21.15.167:6817/2290216047]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-10T00:50:11.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:11 smithi094 ceph-mon[36708]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-10T00:50:11.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:11 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:11.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:11 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:12.836 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:12 smithi037 ceph-mon[32192]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-10T00:50:12.836 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:12 smithi037 ceph-mon[32192]: osdmap e58: 9 total, 8 up, 9 in 2024-06-10T00:50:12.836 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:12 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:50:12.836 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:12 smithi037 ceph-mon[32192]: from='osd.8 [v2:172.21.15.167:6816/2290216047,v1:172.21.15.167:6817/2290216047]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-10T00:50:12.836 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:12 smithi037 ceph-mon[32192]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-10T00:50:12.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:12 smithi167 ceph-mon[36917]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-10T00:50:12.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:12 smithi167 ceph-mon[36917]: osdmap e58: 9 total, 8 up, 9 in 2024-06-10T00:50:12.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:12 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:50:12.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:12 smithi167 ceph-mon[36917]: from='osd.8 [v2:172.21.15.167:6816/2290216047,v1:172.21.15.167:6817/2290216047]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-10T00:50:12.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:12 smithi167 ceph-mon[36917]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-10T00:50:12.839 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 00:50:12 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T00:50:12.549+0000 7f74af1ec700 -1 osd.8 0 waiting for initial osdmap 2024-06-10T00:50:12.839 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 00:50:12 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T00:50:12.558+0000 7f74aafdf700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:50:12.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:12 smithi094 ceph-mon[36708]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-10T00:50:12.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:12 smithi094 ceph-mon[36708]: osdmap e58: 9 total, 8 up, 9 in 2024-06-10T00:50:12.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:50:12.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:12 smithi094 ceph-mon[36708]: from='osd.8 [v2:172.21.15.167:6816/2290216047,v1:172.21.15.167:6817/2290216047]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-10T00:50:12.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:12 smithi094 ceph-mon[36708]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-10T00:50:13.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:13 smithi167 ceph-mon[36917]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:50:13.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:13 smithi167 ceph-mon[36917]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-10T00:50:13.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:13 smithi167 ceph-mon[36917]: osdmap e59: 9 total, 8 up, 9 in 2024-06-10T00:50:13.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:13 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:50:13.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:13 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:50:13.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:13 smithi167 ceph-mon[36917]: osd.8 [v2:172.21.15.167:6816/2290216047,v1:172.21.15.167:6817/2290216047] boot 2024-06-10T00:50:13.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:13 smithi167 ceph-mon[36917]: osdmap e60: 9 total, 9 up, 9 in 2024-06-10T00:50:13.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:13 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:50:13.678 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:13 smithi037 ceph-mon[32192]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:50:13.678 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:13 smithi037 ceph-mon[32192]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-10T00:50:13.679 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:13 smithi037 ceph-mon[32192]: osdmap e59: 9 total, 8 up, 9 in 2024-06-10T00:50:13.679 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:13 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:50:13.679 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:13 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:50:13.679 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:13 smithi037 ceph-mon[32192]: osd.8 [v2:172.21.15.167:6816/2290216047,v1:172.21.15.167:6817/2290216047] boot 2024-06-10T00:50:13.679 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:13 smithi037 ceph-mon[32192]: osdmap e60: 9 total, 9 up, 9 in 2024-06-10T00:50:13.679 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:13 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:50:13.693 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-10T00:50:13.693 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-10T00:50:13.693 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-06-10T00:50:13.693 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-06-10T00:50:13.693 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124306 s, 844 MB/s 2024-06-10T00:50:13.693 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-06-10T00:50:13.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:13 smithi094 ceph-mon[36708]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-06-10T00:50:13.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:13 smithi094 ceph-mon[36708]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-10T00:50:13.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:13 smithi094 ceph-mon[36708]: osdmap e59: 9 total, 8 up, 9 in 2024-06-10T00:50:13.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:13 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:50:13.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:13 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:50:13.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:13 smithi094 ceph-mon[36708]: osd.8 [v2:172.21.15.167:6816/2290216047,v1:172.21.15.167:6817/2290216047] boot 2024-06-10T00:50:13.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:13 smithi094 ceph-mon[36708]: osdmap e60: 9 total, 9 up, 9 in 2024-06-10T00:50:13.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:13 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-10T00:50:14.440 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_1 2024-06-10T00:50:14.678 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:14 smithi037 ceph-mon[32192]: purged_snaps scrub starts 2024-06-10T00:50:14.679 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:14 smithi037 ceph-mon[32192]: purged_snaps scrub ok 2024-06-10T00:50:14.679 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:14 smithi037 ceph-mon[32192]: osdmap e61: 9 total, 9 up, 9 in 2024-06-10T00:50:14.694 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:50:14.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:14 smithi167 ceph-mon[36917]: purged_snaps scrub starts 2024-06-10T00:50:14.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:14 smithi167 ceph-mon[36917]: purged_snaps scrub ok 2024-06-10T00:50:14.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:14 smithi167 ceph-mon[36917]: osdmap e61: 9 total, 9 up, 9 in 2024-06-10T00:50:14.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:14 smithi094 ceph-mon[36708]: purged_snaps scrub starts 2024-06-10T00:50:14.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:14 smithi094 ceph-mon[36708]: purged_snaps scrub ok 2024-06-10T00:50:14.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:14 smithi094 ceph-mon[36708]: osdmap e61: 9 total, 9 up, 9 in 2024-06-10T00:50:16.109 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:15 smithi037 ceph-mon[32192]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:50:16.109 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:15 smithi037 ceph-mon[32192]: Detected new or changed devices on smithi167 2024-06-10T00:50:16.109 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:15 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:16.109 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:15 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:16.109 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:15 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:50:16.110 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:15 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:50:16.110 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:15 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:50:16.110 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:15 smithi037 ceph-mon[32192]: Adjusting osd_memory_target on smithi167 to 5696M 2024-06-10T00:50:16.110 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:15 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:16.110 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:15 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:50:16.110 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:15 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:50:16.110 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:15 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:16.110 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:15 smithi037 ceph-mon[32192]: osdmap e62: 9 total, 9 up, 9 in 2024-06-10T00:50:16.110 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:15 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:50:16.110 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:15 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:50:16.110 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:15 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:50:16.110 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:15 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:16.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:15 smithi167 ceph-mon[36917]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:50:16.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:15 smithi167 ceph-mon[36917]: Detected new or changed devices on smithi167 2024-06-10T00:50:16.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:15 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:16.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:15 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:16.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:15 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:50:16.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:15 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:50:16.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:15 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:50:16.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:15 smithi167 ceph-mon[36917]: Adjusting osd_memory_target on smithi167 to 5696M 2024-06-10T00:50:16.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:15 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:16.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:15 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:50:16.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:15 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:50:16.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:15 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:16.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:15 smithi167 ceph-mon[36917]: osdmap e62: 9 total, 9 up, 9 in 2024-06-10T00:50:16.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:15 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:50:16.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:15 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:50:16.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:15 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:50:16.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:15 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:16.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:15 smithi094 ceph-mon[36708]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:50:16.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:15 smithi094 ceph-mon[36708]: Detected new or changed devices on smithi167 2024-06-10T00:50:16.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:15 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:16.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:15 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:16.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:15 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:50:16.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:15 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:50:16.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:15 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:50:16.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:15 smithi094 ceph-mon[36708]: Adjusting osd_memory_target on smithi167 to 5696M 2024-06-10T00:50:16.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:15 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:16.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:15 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:50:16.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:15 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:50:16.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:15 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:16.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:15 smithi094 ceph-mon[36708]: osdmap e62: 9 total, 9 up, 9 in 2024-06-10T00:50:16.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:15 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:50:16.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:15 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:50:16.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:15 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:50:16.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:15 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:16.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.365+0000 7efd6d456700 1 -- 172.21.15.37:0/2154396689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd681024f0 msgr2=0x7efd68102910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:50:16.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.365+0000 7efd6d456700 1 --2- 172.21.15.37:0/2154396689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd681024f0 0x7efd68102910 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efd580099c0 tx=0x7efd58009cd0 comp rx=0 tx=0).stop 2024-06-10T00:50:16.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.365+0000 7efd6d456700 1 -- 172.21.15.37:0/2154396689 shutdown_connections 2024-06-10T00:50:16.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.365+0000 7efd6d456700 1 --2- 172.21.15.37:0/2154396689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd68104090 0x7efd6810a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:50:16.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.365+0000 7efd6d456700 1 --2- 172.21.15.37:0/2154396689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd681024f0 0x7efd68102910 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:50:16.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.365+0000 7efd6d456700 1 --2- 172.21.15.37:0/2154396689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd681036f0 0x7efd68103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:50:16.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.365+0000 7efd6d456700 1 -- 172.21.15.37:0/2154396689 >> 172.21.15.37:0/2154396689 conn(0x7efd680fda90 msgr2=0x7efd680ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:50:16.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.365+0000 7efd6d456700 1 -- 172.21.15.37:0/2154396689 shutdown_connections 2024-06-10T00:50:16.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.365+0000 7efd6d456700 1 -- 172.21.15.37:0/2154396689 wait complete. 2024-06-10T00:50:16.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.366+0000 7efd6d456700 1 Processor -- start 2024-06-10T00:50:16.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.366+0000 7efd6d456700 1 -- start start 2024-06-10T00:50:16.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.366+0000 7efd6d456700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd681024f0 0x7efd68070e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:50:16.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.366+0000 7efd6d456700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd681036f0 0x7efd68071360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:50:16.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.366+0000 7efd6d456700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd68104090 0x7efd681a87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:50:16.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.366+0000 7efd6d456700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efd6810e0e0 con 0x7efd68104090 2024-06-10T00:50:16.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.366+0000 7efd6d456700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efd6810df60 con 0x7efd681036f0 2024-06-10T00:50:16.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.366+0000 7efd6d456700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efd6810e260 con 0x7efd681024f0 2024-06-10T00:50:16.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.366+0000 7efd677fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd68104090 0x7efd681a87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:50:16.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.366+0000 7efd667fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd681036f0 0x7efd68071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:50:16.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.366+0000 7efd677fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd68104090 0x7efd681a87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58278/0 (socket says 172.21.15.37:58278) 2024-06-10T00:50:16.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.366+0000 7efd677fe700 1 -- 172.21.15.37:0/82722235 learned_addr learned my addr 172.21.15.37:0/82722235 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:50:16.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.366+0000 7efd66ffd700 1 --2- 172.21.15.37:0/82722235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd681024f0 0x7efd68070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:50:16.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.367+0000 7efd667fc700 1 -- 172.21.15.37:0/82722235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd681024f0 msgr2=0x7efd68070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:50:16.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.367+0000 7efd667fc700 1 --2- 172.21.15.37:0/82722235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd681024f0 0x7efd68070e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:50:16.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.367+0000 7efd667fc700 1 -- 172.21.15.37:0/82722235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd68104090 msgr2=0x7efd681a87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:50:16.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.367+0000 7efd667fc700 1 --2- 172.21.15.37:0/82722235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd68104090 0x7efd681a87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:50:16.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.367+0000 7efd667fc700 1 -- 172.21.15.37:0/82722235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd681a8d30 con 0x7efd681036f0 2024-06-10T00:50:16.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.367+0000 7efd667fc700 1 --2- 172.21.15.37:0/82722235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd681036f0 0x7efd68071360 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efd5000d750 tx=0x7efd5000da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:50:16.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.367+0000 7efd5ffff700 1 -- 172.21.15.37:0/82722235 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd50014030 con 0x7efd681036f0 2024-06-10T00:50:16.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.367+0000 7efd5ffff700 1 -- 172.21.15.37:0/82722235 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd50004500 con 0x7efd681036f0 2024-06-10T00:50:16.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.367+0000 7efd6d456700 1 -- 172.21.15.37:0/82722235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd681a8f00 con 0x7efd681036f0 2024-06-10T00:50:16.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.367+0000 7efd5ffff700 1 -- 172.21.15.37:0/82722235 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd50005400 con 0x7efd681036f0 2024-06-10T00:50:16.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.367+0000 7efd6d456700 1 -- 172.21.15.37:0/82722235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd681a9350 con 0x7efd681036f0 2024-06-10T00:50:16.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.368+0000 7efd6d456700 1 -- 172.21.15.37:0/82722235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd68069180 con 0x7efd681036f0 2024-06-10T00:50:16.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.370+0000 7efd5ffff700 1 -- 172.21.15.37:0/82722235 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efd50004750 con 0x7efd681036f0 2024-06-10T00:50:16.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.370+0000 7efd5ffff700 1 --2- 172.21.15.37:0/82722235 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7efd540a2ca0 0x7efd540a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:50:16.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.371+0000 7efd66ffd700 1 --2- 172.21.15.37:0/82722235 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7efd540a2ca0 0x7efd540a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:50:16.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.371+0000 7efd5ffff700 1 -- 172.21.15.37:0/82722235 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7efd500ce480 con 0x7efd681036f0 2024-06-10T00:50:16.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.371+0000 7efd66ffd700 1 --2- 172.21.15.37:0/82722235 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7efd540a2ca0 0x7efd540a5160 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efd58009990 tx=0x7efd58005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:50:16.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.374+0000 7efd5ffff700 1 -- 172.21.15.37:0/82722235 <== mon.2 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efd5009bcf0 con 0x7efd681036f0 2024-06-10T00:50:16.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:16.591+0000 7efd6d456700 1 -- 172.21.15.37:0/82722235 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7efd680628d0 con 0x7efd540a2ca0 2024-06-10T00:50:17.446 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:17 smithi037 ceph-mon[32192]: osdmap e63: 9 total, 9 up, 9 in 2024-06-10T00:50:17.446 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:17 smithi037 ceph-mon[32192]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:50:17.446 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:17 smithi037 ceph-mon[32192]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:50:17.446 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:17 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:50:17.446 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:17 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:50:17.446 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:17 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:50:17.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:17 smithi167 ceph-mon[36917]: osdmap e63: 9 total, 9 up, 9 in 2024-06-10T00:50:17.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:17 smithi167 ceph-mon[36917]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:50:17.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:17 smithi167 ceph-mon[36917]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:50:17.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:17 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:50:17.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:17 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:50:17.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:17 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:50:17.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:17 smithi094 ceph-mon[36708]: osdmap e63: 9 total, 9 up, 9 in 2024-06-10T00:50:17.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:17 smithi094 ceph-mon[36708]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:50:17.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:17 smithi094 ceph-mon[36708]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:50:17.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:17 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:50:17.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:17 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:50:17.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:17 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:50:18.344 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:18 smithi037 ceph-mon[32192]: osdmap e64: 9 total, 9 up, 9 in 2024-06-10T00:50:18.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:18 smithi167 ceph-mon[36917]: osdmap e64: 9 total, 9 up, 9 in 2024-06-10T00:50:18.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:18 smithi094 ceph-mon[36708]: osdmap e64: 9 total, 9 up, 9 in 2024-06-10T00:50:19.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:19 smithi167 ceph-mon[36917]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:50:19.596 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:19 smithi037 ceph-mon[32192]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:50:19.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:19 smithi094 ceph-mon[36708]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:50:20.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:20 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/246972399' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "210fdc20-0a36-4eeb-bdf2-c0a7f130e19d"}]: dispatch 2024-06-10T00:50:20.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:20 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/246972399' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "210fdc20-0a36-4eeb-bdf2-c0a7f130e19d"}]': finished 2024-06-10T00:50:20.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:20 smithi167 ceph-mon[36917]: osdmap e65: 10 total, 9 up, 10 in 2024-06-10T00:50:20.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:20 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:50:20.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:20 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/762268885' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:50:20.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:20 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/246972399' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "210fdc20-0a36-4eeb-bdf2-c0a7f130e19d"}]: dispatch 2024-06-10T00:50:20.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:20 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/246972399' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "210fdc20-0a36-4eeb-bdf2-c0a7f130e19d"}]': finished 2024-06-10T00:50:20.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:20 smithi094 ceph-mon[36708]: osdmap e65: 10 total, 9 up, 10 in 2024-06-10T00:50:20.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:20 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:50:20.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:20 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/762268885' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:50:20.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:20 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/246972399' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "210fdc20-0a36-4eeb-bdf2-c0a7f130e19d"}]: dispatch 2024-06-10T00:50:20.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:20 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/246972399' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "210fdc20-0a36-4eeb-bdf2-c0a7f130e19d"}]': finished 2024-06-10T00:50:20.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:20 smithi037 ceph-mon[32192]: osdmap e65: 10 total, 9 up, 10 in 2024-06-10T00:50:20.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:20 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:50:20.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:20 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/762268885' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:50:21.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:21 smithi167 ceph-mon[36917]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-06-10T00:50:21.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:21 smithi094 ceph-mon[36708]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-06-10T00:50:21.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:21 smithi037 ceph-mon[32192]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-06-10T00:50:23.637 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:23 smithi037 ceph-mon[32192]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-06-10T00:50:23.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:23 smithi167 ceph-mon[36917]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-06-10T00:50:23.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:23 smithi094 ceph-mon[36708]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-06-10T00:50:25.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:25 smithi037 ceph-mon[32192]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-10T00:50:25.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:25 smithi167 ceph-mon[36917]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-10T00:50:25.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:25 smithi094 ceph-mon[36708]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-10T00:50:27.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:27 smithi037 ceph-mon[32192]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-06-10T00:50:27.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:27 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-10T00:50:27.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:27 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:50:27.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:27 smithi037 ceph-mon[32192]: Deploying daemon osd.9 on smithi037 2024-06-10T00:50:27.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:27 smithi167 ceph-mon[36917]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-06-10T00:50:27.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:27 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-10T00:50:27.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:27 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:50:27.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:27 smithi167 ceph-mon[36917]: Deploying daemon osd.9 on smithi037 2024-06-10T00:50:27.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:27 smithi094 ceph-mon[36708]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-06-10T00:50:27.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:27 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-10T00:50:27.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:27 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:50:27.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:27 smithi094 ceph-mon[36708]: Deploying daemon osd.9 on smithi037 2024-06-10T00:50:29.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:29 smithi037 ceph-mon[32192]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T00:50:29.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:29 smithi167 ceph-mon[36917]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T00:50:29.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:29 smithi094 ceph-mon[36708]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-10T00:50:31.641 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:31 smithi167 ceph-mon[36917]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:50:31.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:31 smithi037 ceph-mon[32192]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:50:31.666 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:31 smithi094 ceph-mon[36708]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:50:34.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:33 smithi167 ceph-mon[36917]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:50:34.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:33 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:34.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:33 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:34.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:33 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:50:34.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:33 smithi094 ceph-mon[36708]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:50:34.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:33 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:34.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:33 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:34.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:33 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:50:34.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:33 smithi037 ceph-mon[32192]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:50:34.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:33 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:34.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:33 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:34.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:33 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:50:35.980 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:35 smithi037 ceph-mon[32192]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:50:36.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:35 smithi167 ceph-mon[36917]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:50:36.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:35 smithi094 ceph-mon[36708]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:50:36.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:36.419+0000 7efd5ffff700 1 -- 172.21.15.37:0/82722235 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efd680628d0 con 0x7efd540a2ca0 2024-06-10T00:50:36.419 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 9 on host 'smithi037' 2024-06-10T00:50:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:36.421+0000 7efd6d456700 1 -- 172.21.15.37:0/82722235 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7efd540a2ca0 msgr2=0x7efd540a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:50:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:36.421+0000 7efd6d456700 1 --2- 172.21.15.37:0/82722235 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7efd540a2ca0 0x7efd540a5160 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efd58009990 tx=0x7efd58005fb0 comp rx=0 tx=0).stop 2024-06-10T00:50:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:36.421+0000 7efd6d456700 1 -- 172.21.15.37:0/82722235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd681036f0 msgr2=0x7efd68071360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:50:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:36.421+0000 7efd6d456700 1 --2- 172.21.15.37:0/82722235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd681036f0 0x7efd68071360 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efd5000d750 tx=0x7efd5000da60 comp rx=0 tx=0).stop 2024-06-10T00:50:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:36.421+0000 7efd6d456700 1 -- 172.21.15.37:0/82722235 shutdown_connections 2024-06-10T00:50:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:36.421+0000 7efd6d456700 1 --2- 172.21.15.37:0/82722235 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7efd540a2ca0 0x7efd540a5160 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:50:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:36.421+0000 7efd6d456700 1 --2- 172.21.15.37:0/82722235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd68104090 0x7efd681a87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:50:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:36.421+0000 7efd6d456700 1 --2- 172.21.15.37:0/82722235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd681024f0 0x7efd68070e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:50:36.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:36.421+0000 7efd6d456700 1 --2- 172.21.15.37:0/82722235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd681036f0 0x7efd68071360 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:50:36.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:36.421+0000 7efd6d456700 1 -- 172.21.15.37:0/82722235 >> 172.21.15.37:0/82722235 conn(0x7efd680fda90 msgr2=0x7efd68106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:50:36.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:36.422+0000 7efd6d456700 1 -- 172.21.15.37:0/82722235 shutdown_connections 2024-06-10T00:50:36.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:50:36.422+0000 7efd6d456700 1 -- 172.21.15.37:0/82722235 wait complete. 2024-06-10T00:50:37.055 DEBUG:teuthology.orchestra.run.smithi037:osd.9> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.9.service 2024-06-10T00:50:37.058 INFO:tasks.cephadm:Deploying osd.10 on smithi094 with /dev/vg_nvme/lv_1... 2024-06-10T00:50:37.058 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-10T00:50:37.300 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T00:50:37.588 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:37 smithi037 ceph-mon[32192]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:50:37.588 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:37 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:37.588 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:37 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:37.588 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:37 smithi037 ceph-mon[32192]: from='osd.9 [v2:172.21.15.37:6826/2063666428,v1:172.21.15.37:6827/2063666428]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-10T00:50:37.588 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:37 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:37.589 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:37 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:37.655 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:37 smithi094 ceph-mon[36708]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:50:37.655 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:37 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:37.655 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:37 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:37.655 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:37 smithi094 ceph-mon[36708]: from='osd.9 [v2:172.21.15.37:6826/2063666428,v1:172.21.15.37:6827/2063666428]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-10T00:50:37.655 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:37 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:37.655 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:37 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:37.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:37 smithi167 ceph-mon[36917]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:50:37.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:37 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:37.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:37 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:37.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:37 smithi167 ceph-mon[36917]: from='osd.9 [v2:172.21.15.37:6826/2063666428,v1:172.21.15.37:6827/2063666428]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-10T00:50:37.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:37 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:37.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:37 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:38.585 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:38 smithi037 ceph-mon[32192]: from='osd.9 [v2:172.21.15.37:6826/2063666428,v1:172.21.15.37:6827/2063666428]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-10T00:50:38.586 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:38 smithi037 ceph-mon[32192]: osdmap e66: 10 total, 9 up, 10 in 2024-06-10T00:50:38.586 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:38 smithi037 ceph-mon[32192]: from='osd.9 [v2:172.21.15.37:6826/2063666428,v1:172.21.15.37:6827/2063666428]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-06-10T00:50:38.586 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:38 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:50:38.720 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:38 smithi094 ceph-mon[36708]: from='osd.9 [v2:172.21.15.37:6826/2063666428,v1:172.21.15.37:6827/2063666428]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-10T00:50:38.720 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:38 smithi094 ceph-mon[36708]: osdmap e66: 10 total, 9 up, 10 in 2024-06-10T00:50:38.720 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:38 smithi094 ceph-mon[36708]: from='osd.9 [v2:172.21.15.37:6826/2063666428,v1:172.21.15.37:6827/2063666428]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-06-10T00:50:38.720 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:38 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:50:38.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:38 smithi167 ceph-mon[36917]: from='osd.9 [v2:172.21.15.37:6826/2063666428,v1:172.21.15.37:6827/2063666428]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-10T00:50:38.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:38 smithi167 ceph-mon[36917]: osdmap e66: 10 total, 9 up, 10 in 2024-06-10T00:50:38.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:38 smithi167 ceph-mon[36917]: from='osd.9 [v2:172.21.15.37:6826/2063666428,v1:172.21.15.37:6827/2063666428]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-06-10T00:50:38.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:38 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:50:39.146 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 00:50:38 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T00:50:38.853+0000 7f107ab52700 -1 osd.9 0 waiting for initial osdmap 2024-06-10T00:50:39.146 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 00:50:38 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T00:50:38.866+0000 7f107293d700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:50:39.420 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-10T00:50:39.420 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-10T00:50:39.421 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-06-10T00:50:39.421 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-06-10T00:50:39.421 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00864099 s, 1.2 GB/s 2024-06-10T00:50:39.421 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-06-10T00:50:39.719 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:39 smithi094 ceph-mon[36708]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:50:39.719 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:39 smithi094 ceph-mon[36708]: from='osd.9 [v2:172.21.15.37:6826/2063666428,v1:172.21.15.37:6827/2063666428]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-06-10T00:50:39.719 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:39 smithi094 ceph-mon[36708]: osdmap e67: 10 total, 9 up, 10 in 2024-06-10T00:50:39.719 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:39 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:50:39.719 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:39 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:50:39.720 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:39 smithi094 ceph-mon[36708]: from='osd.9 [v2:172.21.15.37:6826/2063666428,v1:172.21.15.37:6827/2063666428]' entity='osd.9' 2024-06-10T00:50:39.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:39 smithi037 ceph-mon[32192]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:50:39.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:39 smithi037 ceph-mon[32192]: from='osd.9 [v2:172.21.15.37:6826/2063666428,v1:172.21.15.37:6827/2063666428]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-06-10T00:50:39.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:39 smithi037 ceph-mon[32192]: osdmap e67: 10 total, 9 up, 10 in 2024-06-10T00:50:39.777 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:39 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:50:39.777 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:39 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:50:39.777 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:39 smithi037 ceph-mon[32192]: from='osd.9 [v2:172.21.15.37:6826/2063666428,v1:172.21.15.37:6827/2063666428]' entity='osd.9' 2024-06-10T00:50:39.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:39 smithi167 ceph-mon[36917]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:50:39.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:39 smithi167 ceph-mon[36917]: from='osd.9 [v2:172.21.15.37:6826/2063666428,v1:172.21.15.37:6827/2063666428]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-06-10T00:50:39.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:39 smithi167 ceph-mon[36917]: osdmap e67: 10 total, 9 up, 10 in 2024-06-10T00:50:39.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:39 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:50:39.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:39 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:50:39.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:39 smithi167 ceph-mon[36917]: from='osd.9 [v2:172.21.15.37:6826/2063666428,v1:172.21.15.37:6827/2063666428]' entity='osd.9' 2024-06-10T00:50:40.083 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_1 2024-06-10T00:50:40.327 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T00:50:40.678 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:40 smithi094 ceph-mon[36708]: purged_snaps scrub starts 2024-06-10T00:50:40.678 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:40 smithi094 ceph-mon[36708]: purged_snaps scrub ok 2024-06-10T00:50:40.678 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:40 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:50:40.678 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:40 smithi094 ceph-mon[36708]: osd.9 [v2:172.21.15.37:6826/2063666428,v1:172.21.15.37:6827/2063666428] boot 2024-06-10T00:50:40.678 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:40 smithi094 ceph-mon[36708]: osdmap e68: 10 total, 10 up, 10 in 2024-06-10T00:50:40.678 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:40 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:50:40.678 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:40 smithi094 ceph-mon[36708]: Detected new or changed devices on smithi037 2024-06-10T00:50:40.678 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:40 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:40.679 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:40 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:40.679 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:40 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:50:40.679 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:40 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:50:40.679 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:40 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:50:40.679 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:40 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:50:40.679 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:40 smithi094 ceph-mon[36708]: Adjusting osd_memory_target on smithi037 to 4273M 2024-06-10T00:50:40.679 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:40 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:40.679 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:40 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:50:40.679 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:40 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:50:40.679 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:40 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:40.679 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:40 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:50:40.680 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:40 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:50:40.680 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:40 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:50:40.680 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:40 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:40.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:40 smithi167 ceph-mon[36917]: purged_snaps scrub starts 2024-06-10T00:50:40.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:40 smithi167 ceph-mon[36917]: purged_snaps scrub ok 2024-06-10T00:50:40.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:40 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:50:40.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:40 smithi167 ceph-mon[36917]: osd.9 [v2:172.21.15.37:6826/2063666428,v1:172.21.15.37:6827/2063666428] boot 2024-06-10T00:50:40.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:40 smithi167 ceph-mon[36917]: osdmap e68: 10 total, 10 up, 10 in 2024-06-10T00:50:40.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:40 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:50:40.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:40 smithi167 ceph-mon[36917]: Detected new or changed devices on smithi037 2024-06-10T00:50:40.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:40 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:40.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:40 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:40.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:40 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:50:40.840 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:40 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:50:40.840 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:40 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:50:40.840 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:40 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:50:40.840 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:40 smithi167 ceph-mon[36917]: Adjusting osd_memory_target on smithi037 to 4273M 2024-06-10T00:50:40.840 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:40 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:40.840 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:40 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:50:40.840 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:40 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:50:40.840 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:40 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:40.840 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:40 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:50:40.840 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:40 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:50:40.840 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:40 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:50:40.840 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:40 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:40.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:40 smithi037 ceph-mon[32192]: purged_snaps scrub starts 2024-06-10T00:50:40.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:40 smithi037 ceph-mon[32192]: purged_snaps scrub ok 2024-06-10T00:50:40.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:40 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:50:40.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:40 smithi037 ceph-mon[32192]: osd.9 [v2:172.21.15.37:6826/2063666428,v1:172.21.15.37:6827/2063666428] boot 2024-06-10T00:50:40.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:40 smithi037 ceph-mon[32192]: osdmap e68: 10 total, 10 up, 10 in 2024-06-10T00:50:40.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:40 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-10T00:50:40.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:40 smithi037 ceph-mon[32192]: Detected new or changed devices on smithi037 2024-06-10T00:50:40.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:40 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:40.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:40 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:40.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:40 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:50:40.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:40 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:50:40.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:40 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:50:40.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:40 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:50:40.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:40 smithi037 ceph-mon[32192]: Adjusting osd_memory_target on smithi037 to 4273M 2024-06-10T00:50:40.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:40 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:40.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:40 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:50:40.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:40 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:50:40.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:40 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:40.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:40 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:50:40.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:40 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:50:40.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:40 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:50:40.898 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:40 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:41.487 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:41 smithi094 ceph-mon[36708]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:50:41.685 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:41 smithi037 ceph-mon[32192]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:50:41.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:41 smithi167 ceph-mon[36917]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-10T00:50:42.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.035+0000 7fe5dc186700 1 -- 172.21.15.94:0/581071400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5d4105900 msgr2=0x7fe5d4107cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:50:42.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.035+0000 7fe5dc186700 1 --2- 172.21.15.94:0/581071400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5d4105900 0x7fe5d4107cf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4009990 tx=0x7fe5c4009ca0 comp rx=0 tx=0).stop 2024-06-10T00:50:42.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.035+0000 7fe5dc186700 1 -- 172.21.15.94:0/581071400 shutdown_connections 2024-06-10T00:50:42.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.035+0000 7fe5dc186700 1 --2- 172.21.15.94:0/581071400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5d4108230 0x7fe5d410a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:50:42.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.035+0000 7fe5dc186700 1 --2- 172.21.15.94:0/581071400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5d4102fd0 0x7fe5d41053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:50:42.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.035+0000 7fe5dc186700 1 --2- 172.21.15.94:0/581071400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5d4105900 0x7fe5d4107cf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:50:42.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.035+0000 7fe5dc186700 1 -- 172.21.15.94:0/581071400 >> 172.21.15.94:0/581071400 conn(0x7fe5d40fc960 msgr2=0x7fe5d40feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:50:42.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.035+0000 7fe5dc186700 1 -- 172.21.15.94:0/581071400 shutdown_connections 2024-06-10T00:50:42.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.035+0000 7fe5dc186700 1 -- 172.21.15.94:0/581071400 wait complete. 2024-06-10T00:50:42.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.035+0000 7fe5dc186700 1 Processor -- start 2024-06-10T00:50:42.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.036+0000 7fe5dc186700 1 -- start start 2024-06-10T00:50:42.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.036+0000 7fe5dc186700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5d4102fd0 0x7fe5d41986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:50:42.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.036+0000 7fe5dc186700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5d4105900 0x7fe5d4198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:50:42.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.036+0000 7fe5dc186700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5d4108230 0x7fe5d419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:50:42.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.036+0000 7fe5dc186700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d410e0d0 con 0x7fe5d4108230 2024-06-10T00:50:42.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.036+0000 7fe5dc186700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d410df50 con 0x7fe5d4105900 2024-06-10T00:50:42.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.036+0000 7fe5dc186700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d410e250 con 0x7fe5d4102fd0 2024-06-10T00:50:42.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.036+0000 7fe5d9721700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5d4105900 0x7fe5d4198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:50:42.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.036+0000 7fe5d9721700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5d4105900 0x7fe5d4198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57216/0 (socket says 172.21.15.94:57216) 2024-06-10T00:50:42.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.036+0000 7fe5d9721700 1 -- 172.21.15.94:0/2370092848 learned_addr learned my addr 172.21.15.94:0/2370092848 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-10T00:50:42.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.036+0000 7fe5da723700 1 --2- 172.21.15.94:0/2370092848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5d4108230 0x7fe5d419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:50:42.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.036+0000 7fe5d9f22700 1 --2- 172.21.15.94:0/2370092848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5d4102fd0 0x7fe5d41986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:50:42.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.036+0000 7fe5d9721700 1 -- 172.21.15.94:0/2370092848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5d4102fd0 msgr2=0x7fe5d41986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:50:42.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.036+0000 7fe5d9721700 1 --2- 172.21.15.94:0/2370092848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5d4102fd0 0x7fe5d41986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:50:42.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.036+0000 7fe5d9721700 1 -- 172.21.15.94:0/2370092848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5d4108230 msgr2=0x7fe5d419fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:50:42.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.036+0000 7fe5d9721700 1 --2- 172.21.15.94:0/2370092848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5d4108230 0x7fe5d419fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:50:42.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.036+0000 7fe5d9721700 1 -- 172.21.15.94:0/2370092848 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d41a0350 con 0x7fe5d4105900 2024-06-10T00:50:42.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.036+0000 7fe5d9721700 1 --2- 172.21.15.94:0/2370092848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5d4105900 0x7fe5d4198c30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4005910 tx=0x7fe5c4009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:50:42.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.036+0000 7fe5caffd700 1 -- 172.21.15.94:0/2370092848 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4027070 con 0x7fe5d4105900 2024-06-10T00:50:42.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.036+0000 7fe5caffd700 1 -- 172.21.15.94:0/2370092848 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5c4004150 con 0x7fe5d4105900 2024-06-10T00:50:42.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.036+0000 7fe5dc186700 1 -- 172.21.15.94:0/2370092848 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5d41a05e0 con 0x7fe5d4105900 2024-06-10T00:50:42.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.036+0000 7fe5caffd700 1 -- 172.21.15.94:0/2370092848 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4022db0 con 0x7fe5d4105900 2024-06-10T00:50:42.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.037+0000 7fe5dc186700 1 -- 172.21.15.94:0/2370092848 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5d41a0a30 con 0x7fe5d4105900 2024-06-10T00:50:42.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.038+0000 7fe5dc186700 1 -- 172.21.15.94:0/2370092848 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5d4069180 con 0x7fe5d4105900 2024-06-10T00:50:42.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.038+0000 7fe5caffd700 1 -- 172.21.15.94:0/2370092848 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe5c402a070 con 0x7fe5d4105900 2024-06-10T00:50:42.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.039+0000 7fe5caffd700 1 --2- 172.21.15.94:0/2370092848 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe5c00a2c20 0x7fe5c00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:50:42.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.039+0000 7fe5caffd700 1 -- 172.21.15.94:0/2370092848 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7fe5c40cf5f0 con 0x7fe5d4105900 2024-06-10T00:50:42.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.039+0000 7fe5d9f22700 1 --2- 172.21.15.94:0/2370092848 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe5c00a2c20 0x7fe5c00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:50:42.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.040+0000 7fe5d9f22700 1 --2- 172.21.15.94:0/2370092848 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe5c00a2c20 0x7fe5c00a50e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0007c60 tx=0x7fe5d00073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:50:42.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.043+0000 7fe5caffd700 1 -- 172.21.15.94:0/2370092848 <== mon.2 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe5c405e850 con 0x7fe5d4105900 2024-06-10T00:50:42.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:50:42.250+0000 7fe5dc186700 1 -- 172.21.15.94:0/2370092848 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fe5d40628d0 con 0x7fe5c00a2c20 2024-06-10T00:50:42.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:42 smithi094 ceph-mon[36708]: osdmap e69: 10 total, 10 up, 10 in 2024-06-10T00:50:42.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:42 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:50:42.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:42 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:50:42.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:42 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:50:42.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:42 smithi037 ceph-mon[32192]: osdmap e69: 10 total, 10 up, 10 in 2024-06-10T00:50:42.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:42 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:50:42.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:42 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:50:42.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:42 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:50:42.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:42 smithi167 ceph-mon[36917]: osdmap e69: 10 total, 10 up, 10 in 2024-06-10T00:50:42.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:42 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:50:42.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:42 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:50:42.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:42 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:50:43.823 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:43 smithi094 ceph-mon[36708]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:50:43.824 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:43 smithi094 ceph-mon[36708]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:50:43.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:43 smithi167 ceph-mon[36917]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:50:43.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:43 smithi167 ceph-mon[36917]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:50:43.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:43 smithi037 ceph-mon[32192]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:50:43.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:43 smithi037 ceph-mon[32192]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:50:45.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:45 smithi094 ceph-mon[36708]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:50:45.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:45 smithi094 ceph-mon[36708]: from='client.? 172.21.15.94:0/3341431917' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e200b371-b3cf-4b92-a659-f155ca86f0a7"}]: dispatch 2024-06-10T00:50:45.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:45 smithi094 ceph-mon[36708]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e200b371-b3cf-4b92-a659-f155ca86f0a7"}]: dispatch 2024-06-10T00:50:45.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:45 smithi094 ceph-mon[36708]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e200b371-b3cf-4b92-a659-f155ca86f0a7"}]': finished 2024-06-10T00:50:45.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:45 smithi094 ceph-mon[36708]: osdmap e70: 11 total, 10 up, 11 in 2024-06-10T00:50:45.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:45 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:50:45.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:45 smithi167 ceph-mon[36917]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:50:45.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:45 smithi167 ceph-mon[36917]: from='client.? 172.21.15.94:0/3341431917' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e200b371-b3cf-4b92-a659-f155ca86f0a7"}]: dispatch 2024-06-10T00:50:45.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:45 smithi167 ceph-mon[36917]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e200b371-b3cf-4b92-a659-f155ca86f0a7"}]: dispatch 2024-06-10T00:50:45.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:45 smithi167 ceph-mon[36917]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e200b371-b3cf-4b92-a659-f155ca86f0a7"}]': finished 2024-06-10T00:50:45.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:45 smithi167 ceph-mon[36917]: osdmap e70: 11 total, 10 up, 11 in 2024-06-10T00:50:45.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:45 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:50:45.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:45 smithi037 ceph-mon[32192]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:50:45.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:45 smithi037 ceph-mon[32192]: from='client.? 172.21.15.94:0/3341431917' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e200b371-b3cf-4b92-a659-f155ca86f0a7"}]: dispatch 2024-06-10T00:50:45.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:45 smithi037 ceph-mon[32192]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e200b371-b3cf-4b92-a659-f155ca86f0a7"}]: dispatch 2024-06-10T00:50:45.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:45 smithi037 ceph-mon[32192]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e200b371-b3cf-4b92-a659-f155ca86f0a7"}]': finished 2024-06-10T00:50:45.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:45 smithi037 ceph-mon[32192]: osdmap e70: 11 total, 10 up, 11 in 2024-06-10T00:50:45.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:45 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:50:46.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:46 smithi167 ceph-mon[36917]: from='client.? 172.21.15.94:0/1070344403' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:50:46.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:46 smithi094 ceph-mon[36708]: from='client.? 172.21.15.94:0/1070344403' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:50:46.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:46 smithi037 ceph-mon[32192]: from='client.? 172.21.15.94:0/1070344403' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:50:47.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:47 smithi167 ceph-mon[36917]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:50:47.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:47 smithi094 ceph-mon[36708]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:50:47.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:47 smithi037 ceph-mon[32192]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:50:49.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:49 smithi094 ceph-mon[36708]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:50:49.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:49 smithi167 ceph-mon[36917]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:50:49.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:49 smithi037 ceph-mon[32192]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:50:51.542 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:51 smithi094 ceph-mon[36708]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:50:51.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:51 smithi167 ceph-mon[36917]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:50:51.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:51 smithi037 ceph-mon[32192]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:50:53.579 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:53 smithi094 ceph-mon[36708]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:50:53.579 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:53 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-10T00:50:53.579 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:53 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:50:53.579 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:53 smithi094 ceph-mon[36708]: Deploying daemon osd.10 on smithi094 2024-06-10T00:50:53.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:53 smithi167 ceph-mon[36917]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:50:53.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:53 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-10T00:50:53.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:53 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:50:53.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:53 smithi167 ceph-mon[36917]: Deploying daemon osd.10 on smithi094 2024-06-10T00:50:53.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:53 smithi037 ceph-mon[32192]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:50:53.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:53 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-10T00:50:53.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:53 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:50:53.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:53 smithi037 ceph-mon[32192]: Deploying daemon osd.10 on smithi094 2024-06-10T00:50:55.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:55 smithi094 ceph-mon[36708]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:50:55.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:55 smithi167 ceph-mon[36917]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:50:55.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:55 smithi037 ceph-mon[32192]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:50:57.698 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:57 smithi094 ceph-mon[36708]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:50:57.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:57 smithi167 ceph-mon[36917]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:50:57.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:57 smithi037 ceph-mon[32192]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:50:59.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:59 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:59.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:59 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:59.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:59 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:50:59.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:50:59 smithi167 ceph-mon[36917]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:50:59.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:59 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:59.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:59 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:59.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:59 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:50:59.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:50:59 smithi094 ceph-mon[36708]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:50:59.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:59 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:59.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:59 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:50:59.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:59 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:50:59.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:50:59 smithi037 ceph-mon[32192]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:51:01.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:01 smithi037 ceph-mon[32192]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:51:01.653 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:01 smithi094 ceph-mon[36708]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:51:01.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:01 smithi167 ceph-mon[36917]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:51:02.269 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 10 on host 'smithi094' 2024-06-10T00:51:02.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:51:02.268+0000 7fe5caffd700 1 -- 172.21.15.94:0/2370092848 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fe5d40628d0 con 0x7fe5c00a2c20 2024-06-10T00:51:02.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:51:02.270+0000 7fe5dc186700 1 -- 172.21.15.94:0/2370092848 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe5c00a2c20 msgr2=0x7fe5c00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:02.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:51:02.270+0000 7fe5dc186700 1 --2- 172.21.15.94:0/2370092848 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe5c00a2c20 0x7fe5c00a50e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0007c60 tx=0x7fe5d00073d0 comp rx=0 tx=0).stop 2024-06-10T00:51:02.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:51:02.271+0000 7fe5dc186700 1 -- 172.21.15.94:0/2370092848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5d4105900 msgr2=0x7fe5d4198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:02.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:51:02.271+0000 7fe5dc186700 1 --2- 172.21.15.94:0/2370092848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5d4105900 0x7fe5d4198c30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4005910 tx=0x7fe5c4009580 comp rx=0 tx=0).stop 2024-06-10T00:51:02.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:51:02.271+0000 7fe5dc186700 1 -- 172.21.15.94:0/2370092848 shutdown_connections 2024-06-10T00:51:02.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:51:02.271+0000 7fe5dc186700 1 --2- 172.21.15.94:0/2370092848 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe5c00a2c20 0x7fe5c00a50e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:02.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:51:02.271+0000 7fe5dc186700 1 --2- 172.21.15.94:0/2370092848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5d4108230 0x7fe5d419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:02.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:51:02.271+0000 7fe5dc186700 1 --2- 172.21.15.94:0/2370092848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5d4102fd0 0x7fe5d41986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:02.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:51:02.271+0000 7fe5dc186700 1 --2- 172.21.15.94:0/2370092848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5d4105900 0x7fe5d4198c30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:02.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:51:02.271+0000 7fe5dc186700 1 -- 172.21.15.94:0/2370092848 >> 172.21.15.94:0/2370092848 conn(0x7fe5d40fc960 msgr2=0x7fe5d4103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:51:02.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:51:02.271+0000 7fe5dc186700 1 -- 172.21.15.94:0/2370092848 shutdown_connections 2024-06-10T00:51:02.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-10T00:51:02.271+0000 7fe5dc186700 1 -- 172.21.15.94:0/2370092848 wait complete. 2024-06-10T00:51:02.595 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:02 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:02.595 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:02 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:02.595 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:02 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:02.595 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:02 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:51:02.595 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:02 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:02.595 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:02 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:02.596 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:02 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:02.596 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:02 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:51:02.596 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:02 smithi094 ceph-mon[36708]: from='osd.10 [v2:172.21.15.94:6824/1706618899,v1:172.21.15.94:6825/1706618899]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-10T00:51:02.596 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:02 smithi094 ceph-mon[36708]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-10T00:51:02.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:02 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:02.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:02 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:02.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:02 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:02.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:02 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:51:02.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:02 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:02.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:02 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:02.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:02 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:02.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:02 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:51:02.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:02 smithi167 ceph-mon[36917]: from='osd.10 [v2:172.21.15.94:6824/1706618899,v1:172.21.15.94:6825/1706618899]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-10T00:51:02.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:02 smithi167 ceph-mon[36917]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-10T00:51:02.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:02 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:02.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:02 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:02.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:02 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:02.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:02 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:51:02.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:02 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:02.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:02 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:02.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:02 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:02.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:02 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:51:02.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:02 smithi037 ceph-mon[32192]: from='osd.10 [v2:172.21.15.94:6824/1706618899,v1:172.21.15.94:6825/1706618899]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-10T00:51:02.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:02 smithi037 ceph-mon[32192]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-10T00:51:02.926 DEBUG:teuthology.orchestra.run.smithi094:osd.10> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.10.service 2024-06-10T00:51:02.928 INFO:tasks.cephadm:Deploying osd.11 on smithi167 with /dev/vg_nvme/lv_1... 2024-06-10T00:51:02.929 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-10T00:51:03.177 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:51:03.564 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:03 smithi167 ceph-mon[36917]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:51:03.564 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:03 smithi167 ceph-mon[36917]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-10T00:51:03.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:03 smithi167 ceph-mon[36917]: osdmap e71: 11 total, 10 up, 11 in 2024-06-10T00:51:03.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:03 smithi167 ceph-mon[36917]: from='osd.10 [v2:172.21.15.94:6824/1706618899,v1:172.21.15.94:6825/1706618899]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-10T00:51:03.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:03 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:51:03.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:03 smithi167 ceph-mon[36917]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-10T00:51:03.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:03 smithi094 ceph-mon[36708]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:51:03.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:03 smithi094 ceph-mon[36708]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-10T00:51:03.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:03 smithi094 ceph-mon[36708]: osdmap e71: 11 total, 10 up, 11 in 2024-06-10T00:51:03.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:03 smithi094 ceph-mon[36708]: from='osd.10 [v2:172.21.15.94:6824/1706618899,v1:172.21.15.94:6825/1706618899]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-10T00:51:03.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:03 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:51:03.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:03 smithi094 ceph-mon[36708]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-10T00:51:03.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:03 smithi037 ceph-mon[32192]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:51:03.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:03 smithi037 ceph-mon[32192]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-10T00:51:03.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:03 smithi037 ceph-mon[32192]: osdmap e71: 11 total, 10 up, 11 in 2024-06-10T00:51:03.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:03 smithi037 ceph-mon[32192]: from='osd.10 [v2:172.21.15.94:6824/1706618899,v1:172.21.15.94:6825/1706618899]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-10T00:51:03.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:03 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:51:03.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:03 smithi037 ceph-mon[32192]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-10T00:51:04.364 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 00:51:04 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T00:51:04.234+0000 7fb1e2b62700 -1 osd.10 0 waiting for initial osdmap 2024-06-10T00:51:04.365 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 00:51:04 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T00:51:04.245+0000 7fb1de955700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:51:05.355 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-10T00:51:05.355 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-10T00:51:05.355 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-06-10T00:51:05.355 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-06-10T00:51:05.355 INFO:teuthology.orchestra.run.smithi167.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113795 s, 921 MB/s 2024-06-10T00:51:05.355 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-06-10T00:51:05.460 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:05 smithi094 ceph-mon[36708]: purged_snaps scrub starts 2024-06-10T00:51:05.460 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:05 smithi094 ceph-mon[36708]: purged_snaps scrub ok 2024-06-10T00:51:05.460 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:05 smithi094 ceph-mon[36708]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-06-10T00:51:05.460 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:05 smithi094 ceph-mon[36708]: osdmap e72: 11 total, 10 up, 11 in 2024-06-10T00:51:05.460 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:05 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:51:05.460 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:05 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:51:05.461 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:05 smithi094 ceph-mon[36708]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:51:05.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:05 smithi167 ceph-mon[36917]: purged_snaps scrub starts 2024-06-10T00:51:05.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:05 smithi167 ceph-mon[36917]: purged_snaps scrub ok 2024-06-10T00:51:05.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:05 smithi167 ceph-mon[36917]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-06-10T00:51:05.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:05 smithi167 ceph-mon[36917]: osdmap e72: 11 total, 10 up, 11 in 2024-06-10T00:51:05.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:05 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:51:05.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:05 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:51:05.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:05 smithi167 ceph-mon[36917]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:51:05.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:05 smithi037 ceph-mon[32192]: purged_snaps scrub starts 2024-06-10T00:51:05.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:05 smithi037 ceph-mon[32192]: purged_snaps scrub ok 2024-06-10T00:51:05.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:05 smithi037 ceph-mon[32192]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-06-10T00:51:05.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:05 smithi037 ceph-mon[32192]: osdmap e72: 11 total, 10 up, 11 in 2024-06-10T00:51:05.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:05 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:51:05.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:05 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:51:05.647 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:05 smithi037 ceph-mon[32192]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-10T00:51:06.156 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_1 2024-06-10T00:51:06.402 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:51:06.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:06 smithi167 ceph-mon[36917]: osd.10 [v2:172.21.15.94:6824/1706618899,v1:172.21.15.94:6825/1706618899] boot 2024-06-10T00:51:06.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:06 smithi167 ceph-mon[36917]: osdmap e73: 11 total, 11 up, 11 in 2024-06-10T00:51:06.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:06 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:51:06.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:06 smithi167 ceph-mon[36917]: Detected new or changed devices on smithi094 2024-06-10T00:51:06.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:06 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:06.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:06 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:06.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:06 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:51:06.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:06 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:51:06.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:06 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:51:06.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:06 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:51:06.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:06 smithi167 ceph-mon[36917]: Adjusting osd_memory_target on smithi094 to 4272M 2024-06-10T00:51:06.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:06 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:06.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:06 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:06.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:06 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:51:06.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:06 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:06.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:06 smithi094 ceph-mon[36708]: osd.10 [v2:172.21.15.94:6824/1706618899,v1:172.21.15.94:6825/1706618899] boot 2024-06-10T00:51:06.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:06 smithi094 ceph-mon[36708]: osdmap e73: 11 total, 11 up, 11 in 2024-06-10T00:51:06.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:06 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:51:06.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:06 smithi094 ceph-mon[36708]: Detected new or changed devices on smithi094 2024-06-10T00:51:06.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:06 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:06.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:06 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:06.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:06 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:51:06.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:06 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:51:06.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:06 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:51:06.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:06 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:51:06.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:06 smithi094 ceph-mon[36708]: Adjusting osd_memory_target on smithi094 to 4272M 2024-06-10T00:51:06.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:06 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:06.616 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:06 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:06.616 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:06 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:51:06.616 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:06 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:06.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:06 smithi037 ceph-mon[32192]: osd.10 [v2:172.21.15.94:6824/1706618899,v1:172.21.15.94:6825/1706618899] boot 2024-06-10T00:51:06.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:06 smithi037 ceph-mon[32192]: osdmap e73: 11 total, 11 up, 11 in 2024-06-10T00:51:06.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:06 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-10T00:51:06.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:06 smithi037 ceph-mon[32192]: Detected new or changed devices on smithi094 2024-06-10T00:51:06.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:06 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:06.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:06 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:06.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:06 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:51:06.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:06 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:51:06.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:06 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:51:06.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:06 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:51:06.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:06 smithi037 ceph-mon[32192]: Adjusting osd_memory_target on smithi094 to 4272M 2024-06-10T00:51:06.647 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:06 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:06.647 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:06 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:06.647 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:06 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:51:06.647 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:06 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:07.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:07 smithi167 ceph-mon[36917]: osdmap e74: 11 total, 11 up, 11 in 2024-06-10T00:51:07.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:07 smithi167 ceph-mon[36917]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:51:07.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:07 smithi094 ceph-mon[36708]: osdmap e74: 11 total, 11 up, 11 in 2024-06-10T00:51:07.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:07 smithi094 ceph-mon[36708]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:51:07.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:07 smithi037 ceph-mon[32192]: osdmap e74: 11 total, 11 up, 11 in 2024-06-10T00:51:07.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:07 smithi037 ceph-mon[32192]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:51:08.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.134+0000 7f22ad33f700 1 -- 172.21.15.167:0/1497574825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22a8102fd0 msgr2=0x7f22a8109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:08.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.134+0000 7f22ad33f700 1 --2- 172.21.15.167:0/1497574825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22a8102fd0 0x7f22a8109880 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f22a0009990 tx=0x7f22a0009ca0 comp rx=0 tx=0).stop 2024-06-10T00:51:08.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.134+0000 7f22ad33f700 1 -- 172.21.15.167:0/1497574825 shutdown_connections 2024-06-10T00:51:08.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.134+0000 7f22ad33f700 1 --2- 172.21.15.167:0/1497574825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22a8102fd0 0x7f22a8109880 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:08.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.134+0000 7f22ad33f700 1 --2- 172.21.15.167:0/1497574825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22a8101430 0x7f22a8101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:08.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.134+0000 7f22ad33f700 1 --2- 172.21.15.167:0/1497574825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22a8102630 0x7f22a8102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:08.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.134+0000 7f22ad33f700 1 -- 172.21.15.167:0/1497574825 >> 172.21.15.167:0/1497574825 conn(0x7f22a80fc9b0 msgr2=0x7f22a80fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:51:08.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.134+0000 7f22ad33f700 1 -- 172.21.15.167:0/1497574825 shutdown_connections 2024-06-10T00:51:08.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.134+0000 7f22ad33f700 1 -- 172.21.15.167:0/1497574825 wait complete. 2024-06-10T00:51:08.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.134+0000 7f22ad33f700 1 Processor -- start 2024-06-10T00:51:08.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.134+0000 7f22ad33f700 1 -- start start 2024-06-10T00:51:08.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.135+0000 7f22ad33f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22a8101430 0x7f22a81975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:08.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.135+0000 7f22ad33f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22a8102630 0x7f22a8197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:08.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.135+0000 7f22ad33f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22a8102fd0 0x7f22a819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:08.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.135+0000 7f22ad33f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f22a810d090 con 0x7f22a8101430 2024-06-10T00:51:08.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.135+0000 7f22ad33f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f22a810cf10 con 0x7f22a8102fd0 2024-06-10T00:51:08.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.135+0000 7f22ad33f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f22a810d210 con 0x7f22a8102630 2024-06-10T00:51:08.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.135+0000 7f22a67fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22a8102630 0x7f22a8197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:08.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.135+0000 7f22a67fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22a8102630 0x7f22a8197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:38416/0 (socket says 172.21.15.167:38416) 2024-06-10T00:51:08.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.135+0000 7f22a67fc700 1 -- 172.21.15.167:0/3911030880 learned_addr learned my addr 172.21.15.167:0/3911030880 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-10T00:51:08.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.135+0000 7f22a6ffd700 1 --2- 172.21.15.167:0/3911030880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22a8101430 0x7f22a81975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:08.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.135+0000 7f22a67fc700 1 -- 172.21.15.167:0/3911030880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22a8102fd0 msgr2=0x7f22a819eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:08.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.135+0000 7f22a67fc700 1 --2- 172.21.15.167:0/3911030880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22a8102fd0 0x7f22a819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:08.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.135+0000 7f22a67fc700 1 -- 172.21.15.167:0/3911030880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22a8101430 msgr2=0x7f22a81975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:08.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.135+0000 7f22a67fc700 1 --2- 172.21.15.167:0/3911030880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22a8101430 0x7f22a81975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:08.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.135+0000 7f22a67fc700 1 -- 172.21.15.167:0/3911030880 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a819f240 con 0x7f22a8102630 2024-06-10T00:51:08.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.135+0000 7f22a6ffd700 1 --2- 172.21.15.167:0/3911030880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22a8101430 0x7f22a81975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:51:08.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.135+0000 7f22a67fc700 1 --2- 172.21.15.167:0/3911030880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22a8102630 0x7f22a8197b20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f229c00b560 tx=0x7f229c00b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:51:08.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.135+0000 7f228ffff700 1 -- 172.21.15.167:0/3911030880 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c015020 con 0x7f22a8102630 2024-06-10T00:51:08.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.135+0000 7f228ffff700 1 -- 172.21.15.167:0/3911030880 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f229c011d10 con 0x7f22a8102630 2024-06-10T00:51:08.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.135+0000 7f22ad33f700 1 -- 172.21.15.167:0/3911030880 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a819f530 con 0x7f22a8102630 2024-06-10T00:51:08.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.135+0000 7f228ffff700 1 -- 172.21.15.167:0/3911030880 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c00f2e0 con 0x7f22a8102630 2024-06-10T00:51:08.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.135+0000 7f22ad33f700 1 -- 172.21.15.167:0/3911030880 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a819faf0 con 0x7f22a8102630 2024-06-10T00:51:08.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.136+0000 7f22ad33f700 1 -- 172.21.15.167:0/3911030880 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22a8069180 con 0x7f22a8102630 2024-06-10T00:51:08.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.138+0000 7f228ffff700 1 -- 172.21.15.167:0/3911030880 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f229c018020 con 0x7f22a8102630 2024-06-10T00:51:08.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.138+0000 7f228ffff700 1 --2- 172.21.15.167:0/3911030880 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f22900a2c20 0x7f22900a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:08.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.138+0000 7f228ffff700 1 -- 172.21.15.167:0/3911030880 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6723+0+0 (secure 0 0 0) 0x7f229c0ce6e0 con 0x7f22a8102630 2024-06-10T00:51:08.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.139+0000 7f22a6ffd700 1 --2- 172.21.15.167:0/3911030880 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f22900a2c20 0x7f22900a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:08.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.139+0000 7f22a6ffd700 1 --2- 172.21.15.167:0/3911030880 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f22900a2c20 0x7f22900a50e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f22a8102490 tx=0x7f2298009040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:51:08.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.142+0000 7f228ffff700 1 -- 172.21.15.167:0/3911030880 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f229c01f020 con 0x7f22a8102630 2024-06-10T00:51:08.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:08.350+0000 7f22ad33f700 1 -- 172.21.15.167:0/3911030880 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f22a80628d0 con 0x7f22900a2c20 2024-06-10T00:51:08.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:08 smithi167 ceph-mon[36917]: osdmap e75: 11 total, 11 up, 11 in 2024-06-10T00:51:08.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:08 smithi094 ceph-mon[36708]: osdmap e75: 11 total, 11 up, 11 in 2024-06-10T00:51:08.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:08 smithi037 ceph-mon[32192]: osdmap e75: 11 total, 11 up, 11 in 2024-06-10T00:51:09.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:09 smithi167 ceph-mon[36917]: osdmap e76: 11 total, 11 up, 11 in 2024-06-10T00:51:09.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:09 smithi167 ceph-mon[36917]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:51:09.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:09 smithi167 ceph-mon[36917]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:51:09.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:09 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:51:09.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:09 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:51:09.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:09 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:09.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:09 smithi094 ceph-mon[36708]: osdmap e76: 11 total, 11 up, 11 in 2024-06-10T00:51:09.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:09 smithi094 ceph-mon[36708]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:51:09.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:09 smithi094 ceph-mon[36708]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:51:09.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:09 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:51:09.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:09 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:51:09.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:09 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:09.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:09 smithi037 ceph-mon[32192]: osdmap e76: 11 total, 11 up, 11 in 2024-06-10T00:51:09.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:09 smithi037 ceph-mon[32192]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:51:09.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:09 smithi037 ceph-mon[32192]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:51:09.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:09 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-10T00:51:09.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:09 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-10T00:51:09.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:09 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:10.547 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:10 smithi037 ceph-mon[32192]: osdmap e77: 11 total, 11 up, 11 in 2024-06-10T00:51:10.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:10 smithi167 ceph-mon[36917]: osdmap e77: 11 total, 11 up, 11 in 2024-06-10T00:51:10.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:10 smithi094 ceph-mon[36708]: osdmap e77: 11 total, 11 up, 11 in 2024-06-10T00:51:11.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:11 smithi167 ceph-mon[36917]: pgmap v242: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-06-10T00:51:11.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:11 smithi094 ceph-mon[36708]: pgmap v242: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-06-10T00:51:11.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:11 smithi037 ceph-mon[32192]: pgmap v242: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-06-10T00:51:12.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:12 smithi094 ceph-mon[36708]: from='client.? 172.21.15.167:0/3332386349' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cfa427df-37a5-4d28-ac2e-af777d607008"}]: dispatch 2024-06-10T00:51:12.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:12 smithi094 ceph-mon[36708]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cfa427df-37a5-4d28-ac2e-af777d607008"}]: dispatch 2024-06-10T00:51:12.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:12 smithi094 ceph-mon[36708]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cfa427df-37a5-4d28-ac2e-af777d607008"}]': finished 2024-06-10T00:51:12.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:12 smithi094 ceph-mon[36708]: osdmap e78: 12 total, 11 up, 12 in 2024-06-10T00:51:12.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:12 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:51:12.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:12 smithi094 ceph-mon[36708]: from='client.? 172.21.15.167:0/3250519166' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:51:12.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:12 smithi167 ceph-mon[36917]: from='client.? 172.21.15.167:0/3332386349' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cfa427df-37a5-4d28-ac2e-af777d607008"}]: dispatch 2024-06-10T00:51:12.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:12 smithi167 ceph-mon[36917]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cfa427df-37a5-4d28-ac2e-af777d607008"}]: dispatch 2024-06-10T00:51:12.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:12 smithi167 ceph-mon[36917]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cfa427df-37a5-4d28-ac2e-af777d607008"}]': finished 2024-06-10T00:51:12.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:12 smithi167 ceph-mon[36917]: osdmap e78: 12 total, 11 up, 12 in 2024-06-10T00:51:12.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:12 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:51:12.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:12 smithi167 ceph-mon[36917]: from='client.? 172.21.15.167:0/3250519166' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:51:12.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:12 smithi037 ceph-mon[32192]: from='client.? 172.21.15.167:0/3332386349' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cfa427df-37a5-4d28-ac2e-af777d607008"}]: dispatch 2024-06-10T00:51:12.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:12 smithi037 ceph-mon[32192]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cfa427df-37a5-4d28-ac2e-af777d607008"}]: dispatch 2024-06-10T00:51:12.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:12 smithi037 ceph-mon[32192]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cfa427df-37a5-4d28-ac2e-af777d607008"}]': finished 2024-06-10T00:51:12.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:12 smithi037 ceph-mon[32192]: osdmap e78: 12 total, 11 up, 12 in 2024-06-10T00:51:12.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:12 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:51:12.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:12 smithi037 ceph-mon[32192]: from='client.? 172.21.15.167:0/3250519166' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-10T00:51:13.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:13 smithi167 ceph-mon[36917]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-06-10T00:51:13.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:13 smithi094 ceph-mon[36708]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-06-10T00:51:13.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:13 smithi037 ceph-mon[32192]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-06-10T00:51:15.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:15 smithi167 ceph-mon[36917]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-06-10T00:51:15.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:15 smithi037 ceph-mon[32192]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-06-10T00:51:15.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:15 smithi094 ceph-mon[36708]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-06-10T00:51:17.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:17 smithi037 ceph-mon[32192]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-10T00:51:17.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:17 smithi167 ceph-mon[36917]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-10T00:51:17.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:17 smithi094 ceph-mon[36708]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-10T00:51:19.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:19 smithi037 ceph-mon[32192]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:51:19.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:19 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-10T00:51:19.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:19 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:19.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:19 smithi037 ceph-mon[32192]: Deploying daemon osd.11 on smithi167 2024-06-10T00:51:19.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:19 smithi167 ceph-mon[36917]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:51:19.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:19 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-10T00:51:19.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:19 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:19.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:19 smithi167 ceph-mon[36917]: Deploying daemon osd.11 on smithi167 2024-06-10T00:51:19.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:19 smithi094 ceph-mon[36708]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:51:19.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-10T00:51:19.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:19.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:19 smithi094 ceph-mon[36708]: Deploying daemon osd.11 on smithi167 2024-06-10T00:51:21.389 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:21 smithi167 ceph-mon[36917]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:51:21.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:21 smithi037 ceph-mon[32192]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:51:21.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:21 smithi094 ceph-mon[36708]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:51:23.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:23 smithi037 ceph-mon[32192]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:51:23.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:23 smithi167 ceph-mon[36917]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:51:23.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:23 smithi094 ceph-mon[36708]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:51:26.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:25 smithi167 ceph-mon[36917]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:51:26.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:25 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:26.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:25 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:26.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:25 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:51:26.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:25 smithi094 ceph-mon[36708]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:51:26.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:25 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:26.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:25 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:26.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:25 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:51:26.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:25 smithi037 ceph-mon[32192]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:51:26.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:25 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:26.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:25 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:26.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:25 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:51:28.235 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:27 smithi167 ceph-mon[36917]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:51:28.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:27 smithi094 ceph-mon[36708]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:51:28.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:27 smithi037 ceph-mon[32192]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:51:29.024 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 11 on host 'smithi167' 2024-06-10T00:51:29.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:29.023+0000 7f228ffff700 1 -- 172.21.15.167:0/3911030880 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f22a80628d0 con 0x7f22900a2c20 2024-06-10T00:51:29.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:29.026+0000 7f22ad33f700 1 -- 172.21.15.167:0/3911030880 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f22900a2c20 msgr2=0x7f22900a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:29.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:29.026+0000 7f22ad33f700 1 --2- 172.21.15.167:0/3911030880 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f22900a2c20 0x7f22900a50e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f22a8102490 tx=0x7f2298009040 comp rx=0 tx=0).stop 2024-06-10T00:51:29.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:29.026+0000 7f22ad33f700 1 -- 172.21.15.167:0/3911030880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22a8102630 msgr2=0x7f22a8197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:29.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:29.026+0000 7f22ad33f700 1 --2- 172.21.15.167:0/3911030880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22a8102630 0x7f22a8197b20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f229c00b560 tx=0x7f229c00b920 comp rx=0 tx=0).stop 2024-06-10T00:51:29.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:29.026+0000 7f22ad33f700 1 -- 172.21.15.167:0/3911030880 shutdown_connections 2024-06-10T00:51:29.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:29.026+0000 7f22ad33f700 1 --2- 172.21.15.167:0/3911030880 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f22900a2c20 0x7f22900a50e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:29.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:29.026+0000 7f22ad33f700 1 --2- 172.21.15.167:0/3911030880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22a8102fd0 0x7f22a819eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:29.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:29.026+0000 7f22ad33f700 1 --2- 172.21.15.167:0/3911030880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22a8101430 0x7f22a81975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:29.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:29.026+0000 7f22ad33f700 1 --2- 172.21.15.167:0/3911030880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22a8102630 0x7f22a8197b20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:29.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:29.026+0000 7f22ad33f700 1 -- 172.21.15.167:0/3911030880 >> 172.21.15.167:0/3911030880 conn(0x7f22a80fc9b0 msgr2=0x7f22a81058d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:51:29.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:29.026+0000 7f22ad33f700 1 -- 172.21.15.167:0/3911030880 shutdown_connections 2024-06-10T00:51:29.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:29.026+0000 7f22ad33f700 1 -- 172.21.15.167:0/3911030880 wait complete. 2024-06-10T00:51:29.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:29 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:29.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:29 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:29.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:29 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:29.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:29 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:51:29.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:29 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:29.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:29 smithi167 ceph-mon[36917]: from='osd.11 [v2:172.21.15.167:6824/1349892516,v1:172.21.15.167:6825/1349892516]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-10T00:51:29.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:29 smithi167 ceph-mon[36917]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-10T00:51:29.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:29 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:29.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:29 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:29.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:29 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:51:29.399 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:29 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:29.400 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:29 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:29.400 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:29 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:29.400 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:29 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:51:29.400 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:29 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:29.400 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:29 smithi037 ceph-mon[32192]: from='osd.11 [v2:172.21.15.167:6824/1349892516,v1:172.21.15.167:6825/1349892516]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-10T00:51:29.400 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:29 smithi037 ceph-mon[32192]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-10T00:51:29.400 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:29 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:29.400 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:29 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:29.400 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:29 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:51:29.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:29 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:29.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:29 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:29.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:29 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:29.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:29 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:51:29.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:29 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:29.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:29 smithi094 ceph-mon[36708]: from='osd.11 [v2:172.21.15.167:6824/1349892516,v1:172.21.15.167:6825/1349892516]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-10T00:51:29.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:29 smithi094 ceph-mon[36708]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-10T00:51:29.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:29 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:29.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:29 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:29.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:29 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:51:29.729 DEBUG:teuthology.orchestra.run.smithi167:osd.11> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.11.service 2024-06-10T00:51:29.732 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-06-10T00:51:29.732 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-06-10T00:51:29.993 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:51:30.231 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:30 smithi167 ceph-mon[36917]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:51:30.231 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:30 smithi167 ceph-mon[36917]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-10T00:51:30.231 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:30 smithi167 ceph-mon[36917]: osdmap e79: 12 total, 11 up, 12 in 2024-06-10T00:51:30.231 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:30 smithi167 ceph-mon[36917]: from='osd.11 [v2:172.21.15.167:6824/1349892516,v1:172.21.15.167:6825/1349892516]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-10T00:51:30.231 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:30 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:51:30.231 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:30 smithi167 ceph-mon[36917]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-10T00:51:30.368 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:30 smithi037 ceph-mon[32192]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:51:30.368 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:30 smithi037 ceph-mon[32192]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-10T00:51:30.368 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:30 smithi037 ceph-mon[32192]: osdmap e79: 12 total, 11 up, 12 in 2024-06-10T00:51:30.368 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:30 smithi037 ceph-mon[32192]: from='osd.11 [v2:172.21.15.167:6824/1349892516,v1:172.21.15.167:6825/1349892516]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-10T00:51:30.368 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:30 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:51:30.368 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:30 smithi037 ceph-mon[32192]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-10T00:51:30.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:30 smithi094 ceph-mon[36708]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:51:30.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:30 smithi094 ceph-mon[36708]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-10T00:51:30.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:30 smithi094 ceph-mon[36708]: osdmap e79: 12 total, 11 up, 12 in 2024-06-10T00:51:30.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:30 smithi094 ceph-mon[36708]: from='osd.11 [v2:172.21.15.167:6824/1349892516,v1:172.21.15.167:6825/1349892516]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-10T00:51:30.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:30 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:51:30.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:30 smithi094 ceph-mon[36708]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-10T00:51:30.839 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 00:51:30 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T00:51:30.559+0000 7fe574451700 -1 osd.11 0 waiting for initial osdmap 2024-06-10T00:51:30.839 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 00:51:30 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T00:51:30.571+0000 7fe56c23c700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-10T00:51:31.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:31 smithi167 ceph-mon[36917]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-10T00:51:31.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:31 smithi167 ceph-mon[36917]: osdmap e80: 12 total, 11 up, 12 in 2024-06-10T00:51:31.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:31 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:51:31.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:31 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:51:31.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:31 smithi167 ceph-mon[36917]: from='osd.11 ' entity='osd.11' 2024-06-10T00:51:31.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:31 smithi037 ceph-mon[32192]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-10T00:51:31.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:31 smithi037 ceph-mon[32192]: osdmap e80: 12 total, 11 up, 12 in 2024-06-10T00:51:31.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:31 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:51:31.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:31 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:51:31.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:31 smithi037 ceph-mon[32192]: from='osd.11 ' entity='osd.11' 2024-06-10T00:51:31.516 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:31 smithi094 ceph-mon[36708]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-10T00:51:31.516 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:31 smithi094 ceph-mon[36708]: osdmap e80: 12 total, 11 up, 12 in 2024-06-10T00:51:31.516 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:31 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:51:31.516 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:31 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:51:31.516 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:31 smithi094 ceph-mon[36708]: from='osd.11 ' entity='osd.11' 2024-06-10T00:51:31.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.866+0000 7f16035ec700 1 -- 172.21.15.37:0/593212968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15fc1013f0 msgr2=0x7f15fc101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:31.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.866+0000 7f16035ec700 1 --2- 172.21.15.37:0/593212968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15fc1013f0 0x7f15fc101810 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f15f80099c0 tx=0x7f15f8009cd0 comp rx=0 tx=0).stop 2024-06-10T00:51:31.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.866+0000 7f16035ec700 1 -- 172.21.15.37:0/593212968 shutdown_connections 2024-06-10T00:51:31.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.866+0000 7f16035ec700 1 --2- 172.21.15.37:0/593212968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15fc102f90 0x7f15fc109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:31.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.866+0000 7f16035ec700 1 --2- 172.21.15.37:0/593212968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15fc1013f0 0x7f15fc101810 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:31.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.866+0000 7f16035ec700 1 --2- 172.21.15.37:0/593212968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15fc1025f0 0x7f15fc102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:31.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.866+0000 7f16035ec700 1 -- 172.21.15.37:0/593212968 >> 172.21.15.37:0/593212968 conn(0x7f15fc0fc9b0 msgr2=0x7f15fc0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:51:31.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.866+0000 7f16035ec700 1 -- 172.21.15.37:0/593212968 shutdown_connections 2024-06-10T00:51:31.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.866+0000 7f16035ec700 1 -- 172.21.15.37:0/593212968 wait complete. 2024-06-10T00:51:31.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.867+0000 7f16035ec700 1 Processor -- start 2024-06-10T00:51:31.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.867+0000 7f16035ec700 1 -- start start 2024-06-10T00:51:31.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.867+0000 7f16035ec700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15fc1013f0 0x7f15fc06b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:31.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.867+0000 7f16035ec700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15fc1025f0 0x7f15fc19d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:31.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.867+0000 7f16035ec700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15fc102f90 0x7f15fc19fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:31.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.867+0000 7f16035ec700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc10cfb0 con 0x7f15fc1013f0 2024-06-10T00:51:31.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.867+0000 7f16035ec700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc10ce30 con 0x7f15fc1025f0 2024-06-10T00:51:31.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.867+0000 7f16035ec700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc10d130 con 0x7f15fc102f90 2024-06-10T00:51:31.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.867+0000 7f1601388700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15fc1013f0 0x7f15fc06b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:31.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.867+0000 7f1601388700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15fc1013f0 0x7f15fc06b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49604/0 (socket says 172.21.15.37:49604) 2024-06-10T00:51:31.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.867+0000 7f1601388700 1 -- 172.21.15.37:0/2657376646 learned_addr learned my addr 172.21.15.37:0/2657376646 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:51:31.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.867+0000 7f1601b89700 1 --2- 172.21.15.37:0/2657376646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15fc102f90 0x7f15fc19fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:31.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.867+0000 7f1600b87700 1 --2- 172.21.15.37:0/2657376646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15fc1025f0 0x7f15fc19d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:31.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.868+0000 7f1601388700 1 -- 172.21.15.37:0/2657376646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15fc102f90 msgr2=0x7f15fc19fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:31.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.868+0000 7f1601388700 1 --2- 172.21.15.37:0/2657376646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15fc102f90 0x7f15fc19fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:31.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.868+0000 7f1601388700 1 -- 172.21.15.37:0/2657376646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15fc1025f0 msgr2=0x7f15fc19d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:31.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.868+0000 7f1601388700 1 --2- 172.21.15.37:0/2657376646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15fc1025f0 0x7f15fc19d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:31.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.868+0000 7f1601388700 1 -- 172.21.15.37:0/2657376646 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15fc1a0330 con 0x7f15fc1013f0 2024-06-10T00:51:31.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.868+0000 7f1601b89700 1 --2- 172.21.15.37:0/2657376646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15fc102f90 0x7f15fc19fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:51:31.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.868+0000 7f1600b87700 1 --2- 172.21.15.37:0/2657376646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15fc1025f0 0x7f15fc19d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:51:31.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.868+0000 7f1601388700 1 --2- 172.21.15.37:0/2657376646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15fc1013f0 0x7f15fc06b6e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f15f800bf60 tx=0x7f15f8005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:51:31.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.868+0000 7f15f27fc700 1 -- 172.21.15.37:0/2657376646 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f8009e60 con 0x7f15fc1013f0 2024-06-10T00:51:31.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.868+0000 7f15f27fc700 1 -- 172.21.15.37:0/2657376646 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15f800f460 con 0x7f15fc1013f0 2024-06-10T00:51:31.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.868+0000 7f15f27fc700 1 -- 172.21.15.37:0/2657376646 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f8017650 con 0x7f15fc1013f0 2024-06-10T00:51:31.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.868+0000 7f16035ec700 1 -- 172.21.15.37:0/2657376646 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15fc1a05c0 con 0x7f15fc1013f0 2024-06-10T00:51:31.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.868+0000 7f16035ec700 1 -- 172.21.15.37:0/2657376646 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15fc1a0aa0 con 0x7f15fc1013f0 2024-06-10T00:51:31.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.869+0000 7f16035ec700 1 -- 172.21.15.37:0/2657376646 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15fc069180 con 0x7f15fc1013f0 2024-06-10T00:51:31.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.870+0000 7f15f27fc700 1 -- 172.21.15.37:0/2657376646 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f15f800f910 con 0x7f15fc1013f0 2024-06-10T00:51:31.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.871+0000 7f15f27fc700 1 --2- 172.21.15.37:0/2657376646 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f15e80a2ca0 0x7f15e80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:31.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.871+0000 7f1600b87700 1 --2- 172.21.15.37:0/2657376646 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f15e80a2ca0 0x7f15e80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:31.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.871+0000 7f15f27fc700 1 -- 172.21.15.37:0/2657376646 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f15f80cfbe0 con 0x7f15fc1013f0 2024-06-10T00:51:31.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.871+0000 7f1600b87700 1 --2- 172.21.15.37:0/2657376646 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f15e80a2ca0 0x7f15e80a5160 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f15ec005fd0 tx=0x7f15ec005f60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:51:31.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:31.874+0000 7f15f27fc700 1 -- 172.21.15.37:0/2657376646 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f15f8013070 con 0x7f15fc1013f0 2024-06-10T00:51:32.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:32.080+0000 7f16035ec700 1 -- 172.21.15.37:0/2657376646 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f15fc02dd70 con 0x7f15fc1013f0 2024-06-10T00:51:32.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:32.080+0000 7f15f27fc700 1 -- 172.21.15.37:0/2657376646 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f15fc02dd70 con 0x7f15fc1013f0 2024-06-10T00:51:32.081 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:51:32.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:32.083+0000 7f16035ec700 1 -- 172.21.15.37:0/2657376646 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f15e80a2ca0 msgr2=0x7f15e80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:32.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:32.083+0000 7f16035ec700 1 --2- 172.21.15.37:0/2657376646 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f15e80a2ca0 0x7f15e80a5160 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f15ec005fd0 tx=0x7f15ec005f60 comp rx=0 tx=0).stop 2024-06-10T00:51:32.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:32.083+0000 7f16035ec700 1 -- 172.21.15.37:0/2657376646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15fc1013f0 msgr2=0x7f15fc06b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:32.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:32.083+0000 7f16035ec700 1 --2- 172.21.15.37:0/2657376646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15fc1013f0 0x7f15fc06b6e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f15f800bf60 tx=0x7f15f8005ab0 comp rx=0 tx=0).stop 2024-06-10T00:51:32.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:32.083+0000 7f16035ec700 1 -- 172.21.15.37:0/2657376646 shutdown_connections 2024-06-10T00:51:32.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:32.083+0000 7f16035ec700 1 --2- 172.21.15.37:0/2657376646 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f15e80a2ca0 0x7f15e80a5160 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:32.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:32.083+0000 7f16035ec700 1 --2- 172.21.15.37:0/2657376646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15fc102f90 0x7f15fc19fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:32.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:32.083+0000 7f16035ec700 1 --2- 172.21.15.37:0/2657376646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15fc1013f0 0x7f15fc06b6e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:32.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:32.083+0000 7f16035ec700 1 --2- 172.21.15.37:0/2657376646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15fc1025f0 0x7f15fc19d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:32.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:32.083+0000 7f16035ec700 1 -- 172.21.15.37:0/2657376646 >> 172.21.15.37:0/2657376646 conn(0x7f15fc0fc9b0 msgr2=0x7f15fc1058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:51:32.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:32.083+0000 7f16035ec700 1 -- 172.21.15.37:0/2657376646 shutdown_connections 2024-06-10T00:51:32.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:32.083+0000 7f16035ec700 1 -- 172.21.15.37:0/2657376646 wait complete. 2024-06-10T00:51:32.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:32 smithi167 ceph-mon[36917]: purged_snaps scrub starts 2024-06-10T00:51:32.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:32 smithi167 ceph-mon[36917]: purged_snaps scrub ok 2024-06-10T00:51:32.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:32 smithi167 ceph-mon[36917]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:51:32.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:32 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:51:32.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:32 smithi167 ceph-mon[36917]: osd.11 [v2:172.21.15.167:6824/1349892516,v1:172.21.15.167:6825/1349892516] boot 2024-06-10T00:51:32.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:32 smithi167 ceph-mon[36917]: osdmap e81: 12 total, 12 up, 12 in 2024-06-10T00:51:32.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:32 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:51:32.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:32 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2657376646' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-10T00:51:32.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:32 smithi037 ceph-mon[32192]: purged_snaps scrub starts 2024-06-10T00:51:32.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:32 smithi037 ceph-mon[32192]: purged_snaps scrub ok 2024-06-10T00:51:32.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:32 smithi037 ceph-mon[32192]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:51:32.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:32 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:51:32.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:32 smithi037 ceph-mon[32192]: osd.11 [v2:172.21.15.167:6824/1349892516,v1:172.21.15.167:6825/1349892516] boot 2024-06-10T00:51:32.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:32 smithi037 ceph-mon[32192]: osdmap e81: 12 total, 12 up, 12 in 2024-06-10T00:51:32.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:32 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:51:32.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:32 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2657376646' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-10T00:51:32.539 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:32 smithi094 ceph-mon[36708]: purged_snaps scrub starts 2024-06-10T00:51:32.540 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:32 smithi094 ceph-mon[36708]: purged_snaps scrub ok 2024-06-10T00:51:32.540 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:32 smithi094 ceph-mon[36708]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-10T00:51:32.540 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:32 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:51:32.540 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:32 smithi094 ceph-mon[36708]: osd.11 [v2:172.21.15.167:6824/1349892516,v1:172.21.15.167:6825/1349892516] boot 2024-06-10T00:51:32.540 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:32 smithi094 ceph-mon[36708]: osdmap e81: 12 total, 12 up, 12 in 2024-06-10T00:51:32.540 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:32 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-10T00:51:32.540 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:32 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2657376646' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-10T00:51:32.907 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1717980691,"num_in_osds":12,"osd_in_since":1717980671,"num_remapped_pgs":0} 2024-06-10T00:51:32.908 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T00:51:33.160 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:51:33.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:33 smithi167 ceph-mon[36917]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:33.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:33 smithi167 ceph-mon[36917]: osdmap e82: 12 total, 12 up, 12 in 2024-06-10T00:51:33.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:33 smithi167 ceph-mon[36917]: Detected new or changed devices on smithi167 2024-06-10T00:51:33.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:33 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:33.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:33 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:33.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:33 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:51:33.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:33 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:51:33.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:33 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:51:33.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:33 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:51:33.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:33 smithi167 ceph-mon[36917]: Adjusting osd_memory_target on smithi167 to 4272M 2024-06-10T00:51:33.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:33 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:33.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:33 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:33.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:33 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:51:33.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:33 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:33.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:33 smithi094 ceph-mon[36708]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:33.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:33 smithi094 ceph-mon[36708]: osdmap e82: 12 total, 12 up, 12 in 2024-06-10T00:51:33.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:33 smithi094 ceph-mon[36708]: Detected new or changed devices on smithi167 2024-06-10T00:51:33.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:33 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:33.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:33 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:33.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:33 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:51:33.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:33 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:51:33.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:33 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:51:33.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:33 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:51:33.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:33 smithi094 ceph-mon[36708]: Adjusting osd_memory_target on smithi167 to 4272M 2024-06-10T00:51:33.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:33 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:33.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:33 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:33.866 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:33 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:51:33.866 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:33 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:33.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:33 smithi037 ceph-mon[32192]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:33.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:33 smithi037 ceph-mon[32192]: osdmap e82: 12 total, 12 up, 12 in 2024-06-10T00:51:33.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:33 smithi037 ceph-mon[32192]: Detected new or changed devices on smithi167 2024-06-10T00:51:33.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:33 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:33.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:33 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:33.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:33 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:51:33.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:33 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:51:33.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:33 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:51:33.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:33 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T00:51:33.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:33 smithi037 ceph-mon[32192]: Adjusting osd_memory_target on smithi167 to 4272M 2024-06-10T00:51:33.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:33 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:33.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:33 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:33.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:33 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:51:33.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:33 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:34.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.914+0000 7f0d8df7f700 1 -- 172.21.15.37:0/4086405973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d88101d50 msgr2=0x7f0d8810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:34.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.914+0000 7f0d8df7f700 1 --2- 172.21.15.37:0/4086405973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d88101d50 0x7f0d8810e240 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009990 tx=0x7f0d80009ca0 comp rx=0 tx=0).stop 2024-06-10T00:51:34.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.914+0000 7f0d8df7f700 1 -- 172.21.15.37:0/4086405973 shutdown_connections 2024-06-10T00:51:34.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.914+0000 7f0d8df7f700 1 --2- 172.21.15.37:0/4086405973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d88101d50 0x7f0d8810e240 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:34.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.914+0000 7f0d8df7f700 1 --2- 172.21.15.37:0/4086405973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d88107400 0x7f0d881077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:34.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.914+0000 7f0d8df7f700 1 --2- 172.21.15.37:0/4086405973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d88101390 0x7f0d88101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:34.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.914+0000 7f0d8df7f700 1 -- 172.21.15.37:0/4086405973 >> 172.21.15.37:0/4086405973 conn(0x7f0d88078340 msgr2=0x7f0d88078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:51:34.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.914+0000 7f0d8df7f700 1 -- 172.21.15.37:0/4086405973 shutdown_connections 2024-06-10T00:51:34.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.914+0000 7f0d8df7f700 1 -- 172.21.15.37:0/4086405973 wait complete. 2024-06-10T00:51:34.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.915+0000 7f0d8df7f700 1 Processor -- start 2024-06-10T00:51:34.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.915+0000 7f0d8df7f700 1 -- start start 2024-06-10T00:51:34.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.915+0000 7f0d8df7f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d88101390 0x7f0d8819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:34.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.915+0000 7f0d8df7f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d88101d50 0x7f0d8819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:34.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.915+0000 7f0d8df7f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d88107400 0x7f0d881989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:34.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.915+0000 7f0d8df7f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0d88111670 con 0x7f0d88101d50 2024-06-10T00:51:34.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.915+0000 7f0d8df7f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0d881114f0 con 0x7f0d88101390 2024-06-10T00:51:34.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.915+0000 7f0d8df7f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0d881117f0 con 0x7f0d88107400 2024-06-10T00:51:34.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.915+0000 7f0d86ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d88101d50 0x7f0d8819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:34.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.915+0000 7f0d86ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d88101d50 0x7f0d8819ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49630/0 (socket says 172.21.15.37:49630) 2024-06-10T00:51:34.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.915+0000 7f0d86ffd700 1 -- 172.21.15.37:0/675901842 learned_addr learned my addr 172.21.15.37:0/675901842 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:51:34.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.915+0000 7f0d877fe700 1 --2- 172.21.15.37:0/675901842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d88101390 0x7f0d8819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:34.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.915+0000 7f0d87fff700 1 --2- 172.21.15.37:0/675901842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d88107400 0x7f0d881989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:34.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.916+0000 7f0d86ffd700 1 -- 172.21.15.37:0/675901842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d88107400 msgr2=0x7f0d881989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:34.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.916+0000 7f0d86ffd700 1 --2- 172.21.15.37:0/675901842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d88107400 0x7f0d881989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:34.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.916+0000 7f0d86ffd700 1 -- 172.21.15.37:0/675901842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d88101390 msgr2=0x7f0d8819e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:34.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.916+0000 7f0d86ffd700 1 --2- 172.21.15.37:0/675901842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d88101390 0x7f0d8819e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:34.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.916+0000 7f0d86ffd700 1 -- 172.21.15.37:0/675901842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d88199140 con 0x7f0d88101d50 2024-06-10T00:51:34.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.916+0000 7f0d86ffd700 1 --2- 172.21.15.37:0/675901842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d88101d50 0x7f0d8819ed60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00c9b0 tx=0x7f0d7c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:51:34.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.916+0000 7f0d84ff9700 1 -- 172.21.15.37:0/675901842 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c007b00 con 0x7f0d88101d50 2024-06-10T00:51:34.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.916+0000 7f0d84ff9700 1 -- 172.21.15.37:0/675901842 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d7c00ce50 con 0x7f0d88101d50 2024-06-10T00:51:34.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.916+0000 7f0d84ff9700 1 -- 172.21.15.37:0/675901842 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c018660 con 0x7f0d88101d50 2024-06-10T00:51:34.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.916+0000 7f0d8df7f700 1 -- 172.21.15.37:0/675901842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d88199430 con 0x7f0d88101d50 2024-06-10T00:51:34.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.916+0000 7f0d8df7f700 1 -- 172.21.15.37:0/675901842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d881a56f0 con 0x7f0d88101d50 2024-06-10T00:51:34.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.917+0000 7f0d8df7f700 1 -- 172.21.15.37:0/675901842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d88069180 con 0x7f0d88101d50 2024-06-10T00:51:34.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.918+0000 7f0d84ff9700 1 -- 172.21.15.37:0/675901842 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0d7c00f450 con 0x7f0d88101d50 2024-06-10T00:51:34.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.919+0000 7f0d84ff9700 1 --2- 172.21.15.37:0/675901842 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0d700a2d20 0x7f0d700a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:34.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.919+0000 7f0d877fe700 1 --2- 172.21.15.37:0/675901842 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0d700a2d20 0x7f0d700a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:34.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.919+0000 7f0d84ff9700 1 -- 172.21.15.37:0/675901842 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0d7c0ced70 con 0x7f0d88101d50 2024-06-10T00:51:34.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.920+0000 7f0d877fe700 1 --2- 172.21.15.37:0/675901842 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0d700a2d20 0x7f0d700a51e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009550 tx=0x7f0d780091b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:51:34.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:34.923+0000 7f0d84ff9700 1 -- 172.21.15.37:0/675901842 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0d7c014030 con 0x7f0d88101d50 2024-06-10T00:51:35.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:35.128+0000 7f0d8df7f700 1 -- 172.21.15.37:0/675901842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d88199f10 con 0x7f0d88101d50 2024-06-10T00:51:35.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:35.129+0000 7f0d84ff9700 1 -- 172.21.15.37:0/675901842 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18421 (secure 0 0 0) 0x7f0d7c09c1e0 con 0x7f0d88101d50 2024-06-10T00:51:35.131 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:51:35.131 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":82,"fsid":"68793514-26c2-11ef-bc9b-c7b262605968","created":"2024-06-10T00:44:24.106713+0000","modified":"2024-06-10T00:51:32.563805+0000","last_up_change":"2024-06-10T00:51:31.562156+0000","last_in_change":"2024-06-10T00:51:11.294301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:47:44.773911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1df6046a-6801-4950-9193-54303fed169b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6803","nonce":3594704094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6805","nonce":3594704094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6809","nonce":3594704094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6807","nonce":3594704094}]},"public_addr":"172.21.15.37:6803/3594704094","cluster_addr":"172.21.15.37:6805/3594704094","heartbeat_back_addr":"172.21.15.37:6809/3594704094","heartbeat_front_addr":"172.21.15.37:6807/3594704094","state":["exists","up"]},{"osd":1,"uuid":"4e1c8944-94e4-4e46-9efe-56229fe6df1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6801","nonce":3427727155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6803","nonce":3427727155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6807","nonce":3427727155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6805","nonce":3427727155}]},"public_addr":"172.21.15.94:6801/3427727155","cluster_addr":"172.21.15.94:6803/3427727155","heartbeat_back_addr":"172.21.15.94:6807/3427727155","heartbeat_front_addr":"172.21.15.94:6805/3427727155","state":["exists","up"]},{"osd":2,"uuid":"fd2fcedc-4c88-478f-8025-13e5944a27ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6801","nonce":4002498292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6803","nonce":4002498292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6807","nonce":4002498292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6805","nonce":4002498292}]},"public_addr":"172.21.15.167:6801/4002498292","cluster_addr":"172.21.15.167:6803/4002498292","heartbeat_back_addr":"172.21.15.167:6807/4002498292","heartbeat_front_addr":"172.21.15.167:6805/4002498292","state":["exists","up"]},{"osd":3,"uuid":"d0153676-818c-4dcc-83b1-287435a0c8a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6811","nonce":3425469324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6813","nonce":3425469324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6817","nonce":3425469324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6815","nonce":3425469324}]},"public_addr":"172.21.15.37:6811/3425469324","cluster_addr":"172.21.15.37:6813/3425469324","heartbeat_back_addr":"172.21.15.37:6817/3425469324","heartbeat_front_addr":"172.21.15.37:6815/3425469324","state":["exists","up"]},{"osd":4,"uuid":"172c333c-00c0-4e94-8870-acb3b32b2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6809","nonce":1740627431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6811","nonce":1740627431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6815","nonce":1740627431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6813","nonce":1740627431}]},"public_addr":"172.21.15.94:6809/1740627431","cluster_addr":"172.21.15.94:6811/1740627431","heartbeat_back_addr":"172.21.15.94:6815/1740627431","heartbeat_front_addr":"172.21.15.94:6813/1740627431","state":["exists","up"]},{"osd":5,"uuid":"dbce656e-5f93-4bda-baf2-fc3cc03d8282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6809","nonce":221805236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6811","nonce":221805236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6815","nonce":221805236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6813","nonce":221805236}]},"public_addr":"172.21.15.167:6809/221805236","cluster_addr":"172.21.15.167:6811/221805236","heartbeat_back_addr":"172.21.15.167:6815/221805236","heartbeat_front_addr":"172.21.15.167:6813/221805236","state":["exists","up"]},{"osd":6,"uuid":"fcbda266-6d60-43b2-a51c-4e562476d1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6819","nonce":472670875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6821","nonce":472670875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6825","nonce":472670875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6823","nonce":472670875}]},"public_addr":"172.21.15.37:6819/472670875","cluster_addr":"172.21.15.37:6821/472670875","heartbeat_back_addr":"172.21.15.37:6825/472670875","heartbeat_front_addr":"172.21.15.37:6823/472670875","state":["exists","up"]},{"osd":7,"uuid":"ed829ed3-ba85-4e82-ac37-e44e9534d37c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6817","nonce":3932281552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6819","nonce":3932281552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6823","nonce":3932281552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6821","nonce":3932281552}]},"public_addr":"172.21.15.94:6817/3932281552","cluster_addr":"172.21.15.94:6819/3932281552","heartbeat_back_addr":"172.21.15.94:6823/3932281552","heartbeat_front_addr":"172.21.15.94:6821/3932281552","state":["exists","up"]},{"osd":8,"uuid":"7d6267d8-72f4-4704-b554-bafc52d11a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6817","nonce":2290216047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6819","nonce":2290216047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6823","nonce":2290216047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6821","nonce":2290216047}]},"public_addr":"172.21.15.167:6817/2290216047","cluster_addr":"172.21.15.167:6819/2290216047","heartbeat_back_addr":"172.21.15.167:6823/2290216047","heartbeat_front_addr":"172.21.15.167:6821/2290216047","state":["exists","up"]},{"osd":9,"uuid":"210fdc20-0a36-4eeb-bdf2-c0a7f130e19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6827","nonce":2063666428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6829","nonce":2063666428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6833","nonce":2063666428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6831","nonce":2063666428}]},"public_addr":"172.21.15.37:6827/2063666428","cluster_addr":"172.21.15.37:6829/2063666428","heartbeat_back_addr":"172.21.15.37:6833/2063666428","heartbeat_front_addr":"172.21.15.37:6831/2063666428","state":["exists","up"]},{"osd":10,"uuid":"e200b371-b3cf-4b92-a659-f155ca86f0a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6825","nonce":1706618899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6827","nonce":1706618899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6831","nonce":1706618899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6829","nonce":1706618899}]},"public_addr":"172.21.15.94:6825/1706618899","cluster_addr":"172.21.15.94:6827/1706618899","heartbeat_back_addr":"172.21.15.94:6831/1706618899","heartbeat_front_addr":"172.21.15.94:6829/1706618899","state":["exists","up"]},{"osd":11,"uuid":"cfa427df-37a5-4d28-ac2e-af777d607008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6825","nonce":1349892516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6827","nonce":1349892516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6831","nonce":1349892516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6829","nonce":1349892516}]},"public_addr":"172.21.15.167:6825/1349892516","cluster_addr":"172.21.15.167:6827/1349892516","heartbeat_back_addr":"172.21.15.167:6831/1349892516","heartbeat_front_addr":"172.21.15.167:6829/1349892516","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:46:51.846147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:15.763723+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:40.106800+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:05.923204+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:30.501883+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:55.565575+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:20.411360+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:46.071776+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:11.854599+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:37.608818+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:03.176767+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:29.768705+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/4084722397":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/3571803585":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/416587443":"2024-06-11T00:45:20.254342+0000","172.21.15.37:6801/2766692263":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/2026344134":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/2781321886":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6801/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/1110143997":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2766692263":"2024-06-11T00:45:20.254342+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:51:35.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:35.132+0000 7f0d8df7f700 1 -- 172.21.15.37:0/675901842 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0d700a2d20 msgr2=0x7f0d700a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:35.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:35.132+0000 7f0d8df7f700 1 --2- 172.21.15.37:0/675901842 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0d700a2d20 0x7f0d700a51e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009550 tx=0x7f0d780091b0 comp rx=0 tx=0).stop 2024-06-10T00:51:35.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:35.132+0000 7f0d8df7f700 1 -- 172.21.15.37:0/675901842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d88101d50 msgr2=0x7f0d8819ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:35.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:35.132+0000 7f0d8df7f700 1 --2- 172.21.15.37:0/675901842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d88101d50 0x7f0d8819ed60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00c9b0 tx=0x7f0d7c00ccc0 comp rx=0 tx=0).stop 2024-06-10T00:51:35.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:35.132+0000 7f0d8df7f700 1 -- 172.21.15.37:0/675901842 shutdown_connections 2024-06-10T00:51:35.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:35.132+0000 7f0d8df7f700 1 --2- 172.21.15.37:0/675901842 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0d700a2d20 0x7f0d700a51e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:35.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:35.132+0000 7f0d8df7f700 1 --2- 172.21.15.37:0/675901842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d88107400 0x7f0d881989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:35.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:35.132+0000 7f0d8df7f700 1 --2- 172.21.15.37:0/675901842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d88101390 0x7f0d8819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:35.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:35.132+0000 7f0d8df7f700 1 --2- 172.21.15.37:0/675901842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d88101d50 0x7f0d8819ed60 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:35.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:35.132+0000 7f0d8df7f700 1 -- 172.21.15.37:0/675901842 >> 172.21.15.37:0/675901842 conn(0x7f0d88078340 msgr2=0x7f0d881053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:51:35.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:35.132+0000 7f0d8df7f700 1 -- 172.21.15.37:0/675901842 shutdown_connections 2024-06-10T00:51:35.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:35.132+0000 7f0d8df7f700 1 -- 172.21.15.37:0/675901842 wait complete. 2024-06-10T00:51:35.757 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-10T00:47:44.773911+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-06-10T00:51:35.757 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-06-10T00:51:35.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:35 smithi167 ceph-mon[36917]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:35.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:35 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/675901842' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:51:35.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:35 smithi094 ceph-mon[36708]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:35.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:35 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/675901842' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:51:35.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:35 smithi037 ceph-mon[32192]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:35.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:35 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/675901842' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:51:36.006 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:51:37.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.536+0000 7f739e9a4700 1 -- 172.21.15.37:0/2212984652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7398101680 msgr2=0x7f7398101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:37.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.536+0000 7f739e9a4700 1 --2- 172.21.15.37:0/2212984652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7398101680 0x7f7398101a60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f73880099c0 tx=0x7f7388009cd0 comp rx=0 tx=0).stop 2024-06-10T00:51:37.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.536+0000 7f739e9a4700 1 -- 172.21.15.37:0/2212984652 shutdown_connections 2024-06-10T00:51:37.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.536+0000 7f739e9a4700 1 --2- 172.21.15.37:0/2212984652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73981066d0 0x7f739810e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:37.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.536+0000 7f739e9a4700 1 --2- 172.21.15.37:0/2212984652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7398101680 0x7f7398101a60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:37.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.536+0000 7f739e9a4700 1 --2- 172.21.15.37:0/2212984652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7398102030 0x7f7398105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:37.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.536+0000 7f739e9a4700 1 -- 172.21.15.37:0/2212984652 >> 172.21.15.37:0/2212984652 conn(0x7f73980773e0 msgr2=0x7f73980777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:51:37.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.536+0000 7f739e9a4700 1 -- 172.21.15.37:0/2212984652 shutdown_connections 2024-06-10T00:51:37.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.536+0000 7f739e9a4700 1 -- 172.21.15.37:0/2212984652 wait complete. 2024-06-10T00:51:37.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.536+0000 7f739e9a4700 1 Processor -- start 2024-06-10T00:51:37.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.537+0000 7f739e9a4700 1 -- start start 2024-06-10T00:51:37.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.537+0000 7f739e9a4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7398101680 0x7f739819d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:37.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.537+0000 7f739e9a4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7398102030 0x7f739819dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:37.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.537+0000 7f739e9a4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73981066d0 0x7f73981979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:37.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.537+0000 7f739e9a4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f73981116d0 con 0x7f7398102030 2024-06-10T00:51:37.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.537+0000 7f739e9a4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7398111550 con 0x7f73981066d0 2024-06-10T00:51:37.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.537+0000 7f739e9a4700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7398111850 con 0x7f7398101680 2024-06-10T00:51:37.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.537+0000 7f73977fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7398102030 0x7f739819dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:37.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.537+0000 7f73977fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7398102030 0x7f739819dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49652/0 (socket says 172.21.15.37:49652) 2024-06-10T00:51:37.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.537+0000 7f73977fe700 1 -- 172.21.15.37:0/1498699275 learned_addr learned my addr 172.21.15.37:0/1498699275 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:51:37.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.537+0000 7f739cf41700 1 --2- 172.21.15.37:0/1498699275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73981066d0 0x7f73981979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:37.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.537+0000 7f7397fff700 1 --2- 172.21.15.37:0/1498699275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7398101680 0x7f739819d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:37.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.537+0000 7f73977fe700 1 -- 172.21.15.37:0/1498699275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7398101680 msgr2=0x7f739819d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:37.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.537+0000 7f73977fe700 1 --2- 172.21.15.37:0/1498699275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7398101680 0x7f739819d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:37.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.538+0000 7f73977fe700 1 -- 172.21.15.37:0/1498699275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73981066d0 msgr2=0x7f73981979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:37.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.538+0000 7f73977fe700 1 --2- 172.21.15.37:0/1498699275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73981066d0 0x7f73981979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:37.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.538+0000 7f73977fe700 1 -- 172.21.15.37:0/1498699275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7398198130 con 0x7f7398102030 2024-06-10T00:51:37.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.538+0000 7f73977fe700 1 --2- 172.21.15.37:0/1498699275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7398102030 0x7f739819dd50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f738c00d950 tx=0x7f738c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:51:37.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.538+0000 7f73957fa700 1 -- 172.21.15.37:0/1498699275 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c017070 con 0x7f7398102030 2024-06-10T00:51:37.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.538+0000 7f739e9a4700 1 -- 172.21.15.37:0/1498699275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7398198420 con 0x7f7398102030 2024-06-10T00:51:37.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.538+0000 7f73957fa700 1 -- 172.21.15.37:0/1498699275 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f738c005020 con 0x7f7398102030 2024-06-10T00:51:37.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.538+0000 7f73957fa700 1 -- 172.21.15.37:0/1498699275 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c00fbc0 con 0x7f7398102030 2024-06-10T00:51:37.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.538+0000 7f739e9a4700 1 -- 172.21.15.37:0/1498699275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73981a46e0 con 0x7f7398102030 2024-06-10T00:51:37.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.539+0000 7f739e9a4700 1 -- 172.21.15.37:0/1498699275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7398069180 con 0x7f7398102030 2024-06-10T00:51:37.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.541+0000 7f73957fa700 1 -- 172.21.15.37:0/1498699275 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f738c00fd60 con 0x7f7398102030 2024-06-10T00:51:37.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.542+0000 7f73957fa700 1 --2- 172.21.15.37:0/1498699275 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f73800a2c20 0x7f73800a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:37.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.542+0000 7f7397fff700 1 --2- 172.21.15.37:0/1498699275 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f73800a2c20 0x7f73800a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:37.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.542+0000 7f73957fa700 1 -- 172.21.15.37:0/1498699275 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f738c0ce7e0 con 0x7f7398102030 2024-06-10T00:51:37.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.542+0000 7f7397fff700 1 --2- 172.21.15.37:0/1498699275 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f73800a2c20 0x7f73800a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7388000c00 tx=0x7f7388005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:51:37.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.545+0000 7f73957fa700 1 -- 172.21.15.37:0/1498699275 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f738c09bb40 con 0x7f7398102030 2024-06-10T00:51:37.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.751+0000 7f739e9a4700 1 -- 172.21.15.37:0/1498699275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f739802dca0 con 0x7f7398102030 2024-06-10T00:51:37.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.752+0000 7f73957fa700 1 -- 172.21.15.37:0/1498699275 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f738c09b440 con 0x7f7398102030 2024-06-10T00:51:37.752 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-06-10T00:51:37.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.754+0000 7f739e9a4700 1 -- 172.21.15.37:0/1498699275 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f73800a2c20 msgr2=0x7f73800a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:37.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.754+0000 7f739e9a4700 1 --2- 172.21.15.37:0/1498699275 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f73800a2c20 0x7f73800a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7388000c00 tx=0x7f7388005fb0 comp rx=0 tx=0).stop 2024-06-10T00:51:37.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.754+0000 7f739e9a4700 1 -- 172.21.15.37:0/1498699275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7398102030 msgr2=0x7f739819dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:37.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.754+0000 7f739e9a4700 1 --2- 172.21.15.37:0/1498699275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7398102030 0x7f739819dd50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f738c00d950 tx=0x7f738c00dc60 comp rx=0 tx=0).stop 2024-06-10T00:51:37.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.754+0000 7f739e9a4700 1 -- 172.21.15.37:0/1498699275 shutdown_connections 2024-06-10T00:51:37.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.754+0000 7f739e9a4700 1 --2- 172.21.15.37:0/1498699275 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f73800a2c20 0x7f73800a50e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:37.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.754+0000 7f739e9a4700 1 --2- 172.21.15.37:0/1498699275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73981066d0 0x7f73981979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:37.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.754+0000 7f739e9a4700 1 --2- 172.21.15.37:0/1498699275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7398101680 0x7f739819d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:37.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.754+0000 7f739e9a4700 1 --2- 172.21.15.37:0/1498699275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7398102030 0x7f739819dd50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:37.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.754+0000 7f739e9a4700 1 -- 172.21.15.37:0/1498699275 >> 172.21.15.37:0/1498699275 conn(0x7f73980773e0 msgr2=0x7f7398105860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:51:37.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.754+0000 7f739e9a4700 1 -- 172.21.15.37:0/1498699275 shutdown_connections 2024-06-10T00:51:37.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:37.754+0000 7f739e9a4700 1 -- 172.21.15.37:0/1498699275 wait complete. 2024-06-10T00:51:37.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:37 smithi167 ceph-mon[36917]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:37.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:37 smithi094 ceph-mon[36708]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:37.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:37 smithi037 ceph-mon[32192]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:38.547 INFO:tasks.cephadm:Adding mds.a on smithi037 2024-06-10T00:51:38.547 INFO:tasks.cephadm:Adding mds.d on smithi037 2024-06-10T00:51:38.548 INFO:tasks.cephadm:Adding mds.g on smithi037 2024-06-10T00:51:38.548 INFO:tasks.cephadm:Adding mds.j on smithi037 2024-06-10T00:51:38.548 INFO:tasks.cephadm:Adding mds.b on smithi094 2024-06-10T00:51:38.548 INFO:tasks.cephadm:Adding mds.e on smithi094 2024-06-10T00:51:38.548 INFO:tasks.cephadm:Adding mds.h on smithi094 2024-06-10T00:51:38.548 INFO:tasks.cephadm:Adding mds.k on smithi094 2024-06-10T00:51:38.548 INFO:tasks.cephadm:Adding mds.c on smithi167 2024-06-10T00:51:38.548 INFO:tasks.cephadm:Adding mds.f on smithi167 2024-06-10T00:51:38.548 INFO:tasks.cephadm:Adding mds.i on smithi167 2024-06-10T00:51:38.548 INFO:tasks.cephadm:Adding mds.l on smithi167 2024-06-10T00:51:38.548 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph orch apply mds all '12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi094=b;smithi094=e;smithi094=h;smithi094=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l' 2024-06-10T00:51:38.799 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T00:51:38.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:38 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1498699275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T00:51:38.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:38 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1498699275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T00:51:38.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:38 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1498699275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T00:51:39.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:39 smithi167 ceph-mon[36917]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:39.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:39 smithi094 ceph-mon[36708]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:39.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:39 smithi037 ceph-mon[32192]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:40.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.644+0000 7f67bf32e700 1 -- 172.21.15.167:0/4116828959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67b8102fc0 msgr2=0x7f67b81053b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:40.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.644+0000 7f67bf32e700 1 --2- 172.21.15.167:0/4116828959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67b8102fc0 0x7f67b81053b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f67b4009990 tx=0x7f67b4009ca0 comp rx=0 tx=0).stop 2024-06-10T00:51:40.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.644+0000 7f67bf32e700 1 -- 172.21.15.167:0/4116828959 shutdown_connections 2024-06-10T00:51:40.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.644+0000 7f67bf32e700 1 --2- 172.21.15.167:0/4116828959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67b8108220 0x7f67b810a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:40.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.644+0000 7f67bf32e700 1 --2- 172.21.15.167:0/4116828959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67b8102fc0 0x7f67b81053b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:40.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.644+0000 7f67bf32e700 1 --2- 172.21.15.167:0/4116828959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67b81058f0 0x7f67b8107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:40.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.644+0000 7f67bf32e700 1 -- 172.21.15.167:0/4116828959 >> 172.21.15.167:0/4116828959 conn(0x7f67b80fc9f0 msgr2=0x7f67b80fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:51:40.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.644+0000 7f67bf32e700 1 -- 172.21.15.167:0/4116828959 shutdown_connections 2024-06-10T00:51:40.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.644+0000 7f67bf32e700 1 -- 172.21.15.167:0/4116828959 wait complete. 2024-06-10T00:51:40.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.645+0000 7f67bf32e700 1 Processor -- start 2024-06-10T00:51:40.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.645+0000 7f67bf32e700 1 -- start start 2024-06-10T00:51:40.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.645+0000 7f67bf32e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67b8102fc0 0x7f67b8198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:40.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.645+0000 7f67bf32e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67b81058f0 0x7f67b8198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:40.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.645+0000 7f67bf32e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67b8108220 0x7f67b819fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:40.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.645+0000 7f67bf32e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f67b810e180 con 0x7f67b81058f0 2024-06-10T00:51:40.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.645+0000 7f67bf32e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f67b810e000 con 0x7f67b8102fc0 2024-06-10T00:51:40.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.645+0000 7f67bf32e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f67b810e300 con 0x7f67b8108220 2024-06-10T00:51:40.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.645+0000 7f67bd8cb700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67b8108220 0x7f67b819fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:40.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.645+0000 7f67bd8cb700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67b8108220 0x7f67b819fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:49750/0 (socket says 172.21.15.167:49750) 2024-06-10T00:51:40.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.645+0000 7f67bd8cb700 1 -- 172.21.15.167:0/3555107233 learned_addr learned my addr 172.21.15.167:0/3555107233 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-06-10T00:51:40.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.645+0000 7f67bc8c9700 1 --2- 172.21.15.167:0/3555107233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67b81058f0 0x7f67b8198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:40.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.645+0000 7f67bd0ca700 1 --2- 172.21.15.167:0/3555107233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67b8102fc0 0x7f67b8198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:40.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.645+0000 7f67bd8cb700 1 -- 172.21.15.167:0/3555107233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67b8102fc0 msgr2=0x7f67b8198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:40.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.645+0000 7f67bd8cb700 1 --2- 172.21.15.167:0/3555107233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67b8102fc0 0x7f67b8198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:40.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.645+0000 7f67bd8cb700 1 -- 172.21.15.167:0/3555107233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67b81058f0 msgr2=0x7f67b8198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:40.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.645+0000 7f67bd8cb700 1 --2- 172.21.15.167:0/3555107233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67b81058f0 0x7f67b8198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:40.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.645+0000 7f67bd8cb700 1 -- 172.21.15.167:0/3555107233 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b81a03f0 con 0x7f67b8108220 2024-06-10T00:51:40.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.645+0000 7f67bd0ca700 1 --2- 172.21.15.167:0/3555107233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67b8102fc0 0x7f67b8198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:51:40.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.646+0000 7f67bd8cb700 1 --2- 172.21.15.167:0/3555107233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67b8108220 0x7f67b819fd50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f67b0007ee0 tx=0x7f67b000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:51:40.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.646+0000 7f67ae7fc700 1 -- 172.21.15.167:0/3555107233 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b0004db0 con 0x7f67b8108220 2024-06-10T00:51:40.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.646+0000 7f67bf32e700 1 -- 172.21.15.167:0/3555107233 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b81a06e0 con 0x7f67b8108220 2024-06-10T00:51:40.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.646+0000 7f67bf32e700 1 -- 172.21.15.167:0/3555107233 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b81a0ca0 con 0x7f67b8108220 2024-06-10T00:51:40.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.646+0000 7f67ae7fc700 1 -- 172.21.15.167:0/3555107233 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67b000dbd0 con 0x7f67b8108220 2024-06-10T00:51:40.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.646+0000 7f67ae7fc700 1 -- 172.21.15.167:0/3555107233 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b0015400 con 0x7f67b8108220 2024-06-10T00:51:40.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.647+0000 7f67bf32e700 1 -- 172.21.15.167:0/3555107233 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f679c005360 con 0x7f67b8108220 2024-06-10T00:51:40.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.648+0000 7f67ae7fc700 1 -- 172.21.15.167:0/3555107233 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f67b0005020 con 0x7f67b8108220 2024-06-10T00:51:40.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.648+0000 7f67ae7fc700 1 --2- 172.21.15.167:0/3555107233 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f67a40a2ca0 0x7f67a40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:40.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.649+0000 7f67ae7fc700 1 -- 172.21.15.167:0/3555107233 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f67b00ce4d0 con 0x7f67b8108220 2024-06-10T00:51:40.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.649+0000 7f67bd0ca700 1 --2- 172.21.15.167:0/3555107233 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f67a40a2ca0 0x7f67a40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:40.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.649+0000 7f67bd0ca700 1 --2- 172.21.15.167:0/3555107233 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f67a40a2ca0 0x7f67a40a5160 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f67b400b5c0 tx=0x7f67b4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:51:40.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.653+0000 7f67ae7fc700 1 -- 172.21.15.167:0/3555107233 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f67b0010070 con 0x7f67b8108220 2024-06-10T00:51:40.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.857+0000 7f67bf32e700 1 -- 172.21.15.167:0/3555107233 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi094=b;smithi094=e;smithi094=h;smithi094=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}) -- 0x7f679c000cf0 con 0x7f67a40a2ca0 2024-06-10T00:51:40.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.866+0000 7f67ae7fc700 1 -- 172.21.15.167:0/3555107233 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f679c000cf0 con 0x7f67a40a2ca0 2024-06-10T00:51:40.866 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mds.all update... 2024-06-10T00:51:40.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.868+0000 7f67bf32e700 1 -- 172.21.15.167:0/3555107233 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f67a40a2ca0 msgr2=0x7f67a40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:40.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.868+0000 7f67bf32e700 1 --2- 172.21.15.167:0/3555107233 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f67a40a2ca0 0x7f67a40a5160 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f67b400b5c0 tx=0x7f67b4005fb0 comp rx=0 tx=0).stop 2024-06-10T00:51:40.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.868+0000 7f67bf32e700 1 -- 172.21.15.167:0/3555107233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67b8108220 msgr2=0x7f67b819fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:40.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.868+0000 7f67bf32e700 1 --2- 172.21.15.167:0/3555107233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67b8108220 0x7f67b819fd50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f67b0007ee0 tx=0x7f67b000d3b0 comp rx=0 tx=0).stop 2024-06-10T00:51:40.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.868+0000 7f67bf32e700 1 -- 172.21.15.167:0/3555107233 shutdown_connections 2024-06-10T00:51:40.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.868+0000 7f67bf32e700 1 --2- 172.21.15.167:0/3555107233 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f67a40a2ca0 0x7f67a40a5160 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:40.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.868+0000 7f67bf32e700 1 --2- 172.21.15.167:0/3555107233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67b8108220 0x7f67b819fd50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:40.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.868+0000 7f67bf32e700 1 --2- 172.21.15.167:0/3555107233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67b8102fc0 0x7f67b8198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:40.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.868+0000 7f67bf32e700 1 --2- 172.21.15.167:0/3555107233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67b81058f0 0x7f67b8198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:40.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.868+0000 7f67bf32e700 1 -- 172.21.15.167:0/3555107233 >> 172.21.15.167:0/3555107233 conn(0x7f67b80fc9f0 msgr2=0x7f67b8103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:51:40.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.868+0000 7f67bf32e700 1 -- 172.21.15.167:0/3555107233 shutdown_connections 2024-06-10T00:51:40.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-06-10T00:51:40.868+0000 7f67bf32e700 1 -- 172.21.15.167:0/3555107233 wait complete. 2024-06-10T00:51:41.609 DEBUG:teuthology.orchestra.run.smithi037:mds.a> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.a.service 2024-06-10T00:51:41.612 DEBUG:teuthology.orchestra.run.smithi037:mds.d> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.d.service 2024-06-10T00:51:41.656 DEBUG:teuthology.orchestra.run.smithi037:mds.g> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.g.service 2024-06-10T00:51:41.699 DEBUG:teuthology.orchestra.run.smithi037:mds.j> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.j.service 2024-06-10T00:51:41.742 DEBUG:teuthology.orchestra.run.smithi094:mds.b> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.b.service 2024-06-10T00:51:41.745 DEBUG:teuthology.orchestra.run.smithi094:mds.e> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.e.service 2024-06-10T00:51:41.788 DEBUG:teuthology.orchestra.run.smithi094:mds.h> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.h.service 2024-06-10T00:51:41.831 DEBUG:teuthology.orchestra.run.smithi094:mds.k> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.k.service 2024-06-10T00:51:41.874 DEBUG:teuthology.orchestra.run.smithi167:mds.c> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.c.service 2024-06-10T00:51:41.876 DEBUG:teuthology.orchestra.run.smithi167:mds.f> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.f.service 2024-06-10T00:51:41.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:41 smithi037 ceph-mon[32192]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:41.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:41 smithi037 ceph-mon[32192]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi094=b;smithi094=e;smithi094=h;smithi094=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:51:41.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:41 smithi037 ceph-mon[32192]: Saving service mds.all spec with placement smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi094=b;smithi094=e;smithi094=h;smithi094=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-06-10T00:51:41.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:41.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:51:41.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:41.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:51:41.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:41.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:51:41.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:51:41.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:41.920 DEBUG:teuthology.orchestra.run.smithi167:mds.i> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.i.service 2024-06-10T00:51:41.964 DEBUG:teuthology.orchestra.run.smithi167:mds.l> sudo journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.l.service 2024-06-10T00:51:42.007 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-06-10T00:51:42.007 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:51:42.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:41 smithi094 ceph-mon[36708]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:42.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:41 smithi094 ceph-mon[36708]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi094=b;smithi094=e;smithi094=h;smithi094=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:51:42.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:41 smithi094 ceph-mon[36708]: Saving service mds.all spec with placement smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi094=b;smithi094=e;smithi094=h;smithi094=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-06-10T00:51:42.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:41 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:42.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:41 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:51:42.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:41 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:42.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:41 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:51:42.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:41 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:42.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:41 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:51:42.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:41 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:51:42.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:41 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:42.211 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:41 smithi167 ceph-mon[36917]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:42.211 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:41 smithi167 ceph-mon[36917]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi094=b;smithi094=e;smithi094=h;smithi094=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:51:42.211 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:41 smithi167 ceph-mon[36917]: Saving service mds.all spec with placement smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi094=b;smithi094=e;smithi094=h;smithi094=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-06-10T00:51:42.211 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:41 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:42.211 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:41 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:51:42.211 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:41 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:42.211 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:41 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:51:42.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:41 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:42.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:41 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:51:42.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:41 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:51:42.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:41 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:42.259 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:51:43.053 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:42 smithi037 ceph-mon[32192]: Deploying daemon mds.l on smithi167 2024-06-10T00:51:43.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:42 smithi167 ceph-mon[36917]: Deploying daemon mds.l on smithi167 2024-06-10T00:51:43.222 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:42 smithi094 ceph-mon[36708]: Deploying daemon mds.l on smithi167 2024-06-10T00:51:43.813 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jun 10 00:51:43 smithi167 systemd[1]: Starting Ceph mds.l for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T00:51:43.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.983+0000 7f662ec28700 1 -- 172.21.15.37:0/1152195039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66281077b0 msgr2=0x7f662810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:43.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.983+0000 7f662ec28700 1 --2- 172.21.15.37:0/1152195039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66281077b0 0x7f662810f3c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f662000b0d0 tx=0x7f662000b3e0 comp rx=0 tx=0).stop 2024-06-10T00:51:43.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.984+0000 7f662ec28700 1 -- 172.21.15.37:0/1152195039 shutdown_connections 2024-06-10T00:51:43.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.984+0000 7f662ec28700 1 --2- 172.21.15.37:0/1152195039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66281077b0 0x7f662810f3c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:43.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.984+0000 7f662ec28700 1 --2- 172.21.15.37:0/1152195039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f662806ae60 0x7f662806b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:43.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.984+0000 7f662ec28700 1 --2- 172.21.15.37:0/1152195039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f662806b810 0x7f66281070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:43.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.984+0000 7f662ec28700 1 -- 172.21.15.37:0/1152195039 >> 172.21.15.37:0/1152195039 conn(0x7f66280773e0 msgr2=0x7f66280777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:51:43.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.984+0000 7f662ec28700 1 -- 172.21.15.37:0/1152195039 shutdown_connections 2024-06-10T00:51:43.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.984+0000 7f662ec28700 1 -- 172.21.15.37:0/1152195039 wait complete. 2024-06-10T00:51:43.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:43 smithi037 ceph-mon[32192]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.984+0000 7f662ec28700 1 Processor -- start 2024-06-10T00:51:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.984+0000 7f662ec28700 1 -- start start 2024-06-10T00:51:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.985+0000 7f662ec28700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f662806ae60 0x7f662819e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.985+0000 7f662ec28700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f662806b810 0x7f662819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:43.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.985+0000 7f662ec28700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66281077b0 0x7f6628198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:43.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.985+0000 7f662ec28700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6628112800 con 0x7f662806ae60 2024-06-10T00:51:43.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.985+0000 7f662ec28700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6628112680 con 0x7f66281077b0 2024-06-10T00:51:43.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.985+0000 7f662ec28700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6628112980 con 0x7f662806b810 2024-06-10T00:51:43.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.985+0000 7f662c9c4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f662806ae60 0x7f662819e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:43.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.985+0000 7f6627fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f662806b810 0x7f662819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:43.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.985+0000 7f662c9c4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f662806ae60 0x7f662819e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39540/0 (socket says 172.21.15.37:39540) 2024-06-10T00:51:43.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.985+0000 7f662c9c4700 1 -- 172.21.15.37:0/1830181534 learned_addr learned my addr 172.21.15.37:0/1830181534 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:51:43.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.985+0000 7f662d1c5700 1 --2- 172.21.15.37:0/1830181534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66281077b0 0x7f6628198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:43.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.985+0000 7f6627fff700 1 -- 172.21.15.37:0/1830181534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66281077b0 msgr2=0x7f6628198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:43.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.985+0000 7f6627fff700 1 --2- 172.21.15.37:0/1830181534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66281077b0 0x7f6628198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:43.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.985+0000 7f6627fff700 1 -- 172.21.15.37:0/1830181534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f662806ae60 msgr2=0x7f662819e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:43.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.985+0000 7f6627fff700 1 --2- 172.21.15.37:0/1830181534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f662806ae60 0x7f662819e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:43.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.985+0000 7f6627fff700 1 -- 172.21.15.37:0/1830181534 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66281991c0 con 0x7f662806b810 2024-06-10T00:51:43.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.986+0000 7f662c9c4700 1 --2- 172.21.15.37:0/1830181534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f662806ae60 0x7f662819e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:51:43.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.986+0000 7f6627fff700 1 --2- 172.21.15.37:0/1830181534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f662806b810 0x7f662819ede0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f661400e9c0 tx=0x7f661400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:51:43.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.986+0000 7f6625ffb700 1 -- 172.21.15.37:0/1830181534 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6614004220 con 0x7f662806b810 2024-06-10T00:51:43.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.986+0000 7f6625ffb700 1 -- 172.21.15.37:0/1830181534 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6614004d10 con 0x7f662806b810 2024-06-10T00:51:43.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.986+0000 7f662ec28700 1 -- 172.21.15.37:0/1830181534 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66281994b0 con 0x7f662806b810 2024-06-10T00:51:43.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.986+0000 7f6625ffb700 1 -- 172.21.15.37:0/1830181534 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6614005020 con 0x7f662806b810 2024-06-10T00:51:43.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.986+0000 7f662ec28700 1 -- 172.21.15.37:0/1830181534 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66281a5770 con 0x7f662806b810 2024-06-10T00:51:43.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.987+0000 7f662ec28700 1 -- 172.21.15.37:0/1830181534 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6628069180 con 0x7f662806b810 2024-06-10T00:51:43.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.988+0000 7f6625ffb700 1 -- 172.21.15.37:0/1830181534 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6614004750 con 0x7f662806b810 2024-06-10T00:51:43.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.989+0000 7f6625ffb700 1 --2- 172.21.15.37:0/1830181534 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f66180a2ca0 0x7f66180a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:43.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.989+0000 7f662c9c4700 1 --2- 172.21.15.37:0/1830181534 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f66180a2ca0 0x7f66180a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:43.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.989+0000 7f6625ffb700 1 -- 172.21.15.37:0/1830181534 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f66140ce980 con 0x7f662806b810 2024-06-10T00:51:43.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.989+0000 7f662c9c4700 1 --2- 172.21.15.37:0/1830181534 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f66180a2ca0 0x7f66180a5160 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6628199ba0 tx=0x7f661c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:51:43.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:43.993+0000 7f6625ffb700 1 -- 172.21.15.37:0/1830181534 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f661409bd60 con 0x7f662806b810 2024-06-10T00:51:44.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:43 smithi167 ceph-mon[36917]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:44.089 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jun 10 00:51:43 smithi167 podman[67050]: 2024-06-10 00:51:43.715540188 +0000 UTC m=+0.016780858 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:51:44.089 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jun 10 00:51:43 smithi167 podman[67050]: 2024-06-10 00:51:43.841836897 +0000 UTC m=+0.143077573 container create 035edf753fae6fde9869cd72f24fc7754dcb105de607f1de30c3ddffe6381785 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-l, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-06-10T00:51:44.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:43 smithi094 ceph-mon[36708]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:44.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:44.245+0000 7f662ec28700 1 -- 172.21.15.37:0/1830181534 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6628199f90 con 0x7f662806b810 2024-06-10T00:51:44.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:44.246+0000 7f6625ffb700 1 -- 172.21.15.37:0/1830181534 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f661409b660 con 0x7f662806b810 2024-06-10T00:51:44.246 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:51:44.247 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-06-10T00:51:44.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:44.248+0000 7f662ec28700 1 -- 172.21.15.37:0/1830181534 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f66180a2ca0 msgr2=0x7f66180a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:44.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:44.248+0000 7f662ec28700 1 --2- 172.21.15.37:0/1830181534 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f66180a2ca0 0x7f66180a5160 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6628199ba0 tx=0x7f661c009260 comp rx=0 tx=0).stop 2024-06-10T00:51:44.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:44.248+0000 7f662ec28700 1 -- 172.21.15.37:0/1830181534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f662806b810 msgr2=0x7f662819ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:44.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:44.248+0000 7f662ec28700 1 --2- 172.21.15.37:0/1830181534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f662806b810 0x7f662819ede0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f661400e9c0 tx=0x7f661400ecd0 comp rx=0 tx=0).stop 2024-06-10T00:51:44.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:44.248+0000 7f662ec28700 1 -- 172.21.15.37:0/1830181534 shutdown_connections 2024-06-10T00:51:44.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:44.248+0000 7f662ec28700 1 --2- 172.21.15.37:0/1830181534 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f66180a2ca0 0x7f66180a5160 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:44.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:44.248+0000 7f662ec28700 1 --2- 172.21.15.37:0/1830181534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66281077b0 0x7f6628198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:44.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:44.248+0000 7f662ec28700 1 --2- 172.21.15.37:0/1830181534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f662806ae60 0x7f662819e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:44.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:44.248+0000 7f662ec28700 1 --2- 172.21.15.37:0/1830181534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f662806b810 0x7f662819ede0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:44.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:44.248+0000 7f662ec28700 1 -- 172.21.15.37:0/1830181534 >> 172.21.15.37:0/1830181534 conn(0x7f66280773e0 msgr2=0x7f6628100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:51:44.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:44.248+0000 7f662ec28700 1 -- 172.21.15.37:0/1830181534 shutdown_connections 2024-06-10T00:51:44.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:44.249+0000 7f662ec28700 1 -- 172.21.15.37:0/1830181534 wait complete. 2024-06-10T00:51:44.250 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 2 2024-06-10T00:51:44.589 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jun 10 00:51:44 smithi167 podman[67050]: 2024-06-10 00:51:44.112785033 +0000 UTC m=+0.414025713 container init 035edf753fae6fde9869cd72f24fc7754dcb105de607f1de30c3ddffe6381785 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-l, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-10T00:51:44.589 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jun 10 00:51:44 smithi167 podman[67050]: 2024-06-10 00:51:44.115624351 +0000 UTC m=+0.416865024 container start 035edf753fae6fde9869cd72f24fc7754dcb105de607f1de30c3ddffe6381785 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-l, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-10T00:51:44.589 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jun 10 00:51:44 smithi167 ceph-mds[67069]: -- 172.21.15.167:0/4032400635 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564642e9da40 con 0x564642e71000 2024-06-10T00:51:44.589 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jun 10 00:51:44 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-l[67064]: starting mds.l at 2024-06-10T00:51:44.589 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jun 10 00:51:44 smithi167 bash[67050]: 035edf753fae6fde9869cd72f24fc7754dcb105de607f1de30c3ddffe6381785 2024-06-10T00:51:44.589 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jun 10 00:51:44 smithi167 systemd[1]: Started Ceph mds.l for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T00:51:45.256 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:45 smithi094 ceph-mon[36708]: mds.? [v2:172.21.15.167:6832/2828145715,v1:172.21.15.167:6833/2828145715] up:boot 2024-06-10T00:51:45.256 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:45 smithi094 ceph-mon[36708]: fsmap 1 up:standby 2024-06-10T00:51:45.257 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:45 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T00:51:45.257 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:45 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1830181534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:51:45.257 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:45 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:45.257 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:45 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:45.257 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:45 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:45.257 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:45 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:51:45.257 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:45 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:51:45.257 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:45 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:45.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:45 smithi167 ceph-mon[36917]: mds.? [v2:172.21.15.167:6832/2828145715,v1:172.21.15.167:6833/2828145715] up:boot 2024-06-10T00:51:45.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:45 smithi167 ceph-mon[36917]: fsmap 1 up:standby 2024-06-10T00:51:45.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:45 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T00:51:45.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:45 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1830181534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:51:45.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:45 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:45.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:45 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:45.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:45 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:45.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:45 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:51:45.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:45 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:51:45.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:45 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:45.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:45 smithi037 ceph-mon[32192]: mds.? [v2:172.21.15.167:6832/2828145715,v1:172.21.15.167:6833/2828145715] up:boot 2024-06-10T00:51:45.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:45 smithi037 ceph-mon[32192]: fsmap 1 up:standby 2024-06-10T00:51:45.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:45 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T00:51:45.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:45 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1830181534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:51:45.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:45 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:45.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:45 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:45.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:45 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:45.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:45 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:51:45.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:45 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:51:45.647 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:45 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:46.312 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:46 smithi094 ceph-mon[36708]: Deploying daemon mds.k on smithi094 2024-06-10T00:51:46.312 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:46 smithi094 ceph-mon[36708]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:46.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:46 smithi167 ceph-mon[36917]: Deploying daemon mds.k on smithi094 2024-06-10T00:51:46.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:46 smithi167 ceph-mon[36917]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:46.614 INFO:journalctl@ceph.mds.k.smithi094.stdout:Jun 10 00:51:46 smithi094 systemd[1]: Starting Ceph mds.k for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T00:51:46.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:46 smithi037 ceph-mon[32192]: Deploying daemon mds.k on smithi094 2024-06-10T00:51:46.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:46 smithi037 ceph-mon[32192]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:46.870 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:51:47.066 INFO:journalctl@ceph.mds.k.smithi094.stdout:Jun 10 00:51:46 smithi094 podman[66313]: 2024-06-10 00:51:46.616914601 +0000 UTC m=+0.017513521 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:51:47.067 INFO:journalctl@ceph.mds.k.smithi094.stdout:Jun 10 00:51:46 smithi094 podman[66313]: 2024-06-10 00:51:46.821834459 +0000 UTC m=+0.222433377 container create 2b9d0a4608f39d83ee5f09b073a3697b09aecd60be9cb391c3dba1ab4c7e6c68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-k, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-06-10T00:51:47.067 INFO:journalctl@ceph.mds.k.smithi094.stdout:Jun 10 00:51:47 smithi094 podman[66313]: 2024-06-10 00:51:47.066740235 +0000 UTC m=+0.467339151 container init 2b9d0a4608f39d83ee5f09b073a3697b09aecd60be9cb391c3dba1ab4c7e6c68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-k, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T00:51:47.131 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:51:47.364 INFO:journalctl@ceph.mds.k.smithi094.stdout:Jun 10 00:51:47 smithi094 podman[66313]: 2024-06-10 00:51:47.069539386 +0000 UTC m=+0.470138313 container start 2b9d0a4608f39d83ee5f09b073a3697b09aecd60be9cb391c3dba1ab4c7e6c68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-k, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc) 2024-06-10T00:51:47.365 INFO:journalctl@ceph.mds.k.smithi094.stdout:Jun 10 00:51:47 smithi094 ceph-mds[66332]: -- 172.21.15.94:0/337420188 <== mon.2 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561545b49a40 con 0x561545b1cc00 2024-06-10T00:51:47.365 INFO:journalctl@ceph.mds.k.smithi094.stdout:Jun 10 00:51:47 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-k[66328]: starting mds.k at 2024-06-10T00:51:47.365 INFO:journalctl@ceph.mds.k.smithi094.stdout:Jun 10 00:51:47 smithi094 bash[66313]: 2b9d0a4608f39d83ee5f09b073a3697b09aecd60be9cb391c3dba1ab4c7e6c68 2024-06-10T00:51:47.365 INFO:journalctl@ceph.mds.k.smithi094.stdout:Jun 10 00:51:47 smithi094 systemd[1]: Started Ceph mds.k for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T00:51:48.267 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:48 smithi037 ceph-mon[32192]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:48.268 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:48 smithi037 ceph-mon[32192]: mds.? [v2:172.21.15.94:6832/2915505496,v1:172.21.15.94:6833/2915505496] up:boot 2024-06-10T00:51:48.268 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:48 smithi037 ceph-mon[32192]: fsmap 2 up:standby 2024-06-10T00:51:48.268 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:48 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-10T00:51:48.268 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:48 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:48.268 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:48 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:48.268 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:48 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:48.268 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:48 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:51:48.268 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:48 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:51:48.269 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:48 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:48.508 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:48 smithi094 ceph-mon[36708]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:48.508 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:48 smithi094 ceph-mon[36708]: mds.? [v2:172.21.15.94:6832/2915505496,v1:172.21.15.94:6833/2915505496] up:boot 2024-06-10T00:51:48.508 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:48 smithi094 ceph-mon[36708]: fsmap 2 up:standby 2024-06-10T00:51:48.508 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:48 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-10T00:51:48.508 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:48 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:48.508 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:48 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:48.508 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:48 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:48.508 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:48 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:51:48.508 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:48 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:51:48.508 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:48 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:48.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:48 smithi167 ceph-mon[36917]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:48.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:48 smithi167 ceph-mon[36917]: mds.? [v2:172.21.15.94:6832/2915505496,v1:172.21.15.94:6833/2915505496] up:boot 2024-06-10T00:51:48.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:48 smithi167 ceph-mon[36917]: fsmap 2 up:standby 2024-06-10T00:51:48.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:48 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-10T00:51:48.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:48 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:48.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:48 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:48.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:48 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:48.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:48 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:51:48.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:48 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:51:48.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:48 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:49.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:49 smithi037 ceph-mon[32192]: Deploying daemon mds.j on smithi037 2024-06-10T00:51:49.435 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:49 smithi094 ceph-mon[36708]: Deploying daemon mds.j on smithi037 2024-06-10T00:51:49.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.491+0000 7fb7a82a9700 1 -- 172.21.15.37:0/4154535166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7a006ac60 msgr2=0x7fb7a006b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:49.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.491+0000 7fb7a82a9700 1 --2- 172.21.15.37:0/4154535166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7a006ac60 0x7fb7a006b0e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb7900099c0 tx=0x7fb790009cd0 comp rx=0 tx=0).stop 2024-06-10T00:51:49.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.491+0000 7fb7a82a9700 1 -- 172.21.15.37:0/4154535166 shutdown_connections 2024-06-10T00:51:49.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.491+0000 7fb7a82a9700 1 --2- 172.21.15.37:0/4154535166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7a006b620 0x7fb7a010f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:49.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.491+0000 7fb7a82a9700 1 --2- 172.21.15.37:0/4154535166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7a0108580 0x7fb7a0108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:49.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.491+0000 7fb7a82a9700 1 --2- 172.21.15.37:0/4154535166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7a006ac60 0x7fb7a006b0e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:49.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.491+0000 7fb7a82a9700 1 -- 172.21.15.37:0/4154535166 >> 172.21.15.37:0/4154535166 conn(0x7fb7a0078430 msgr2=0x7fb7a0078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:51:49.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.491+0000 7fb7a82a9700 1 -- 172.21.15.37:0/4154535166 shutdown_connections 2024-06-10T00:51:49.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.491+0000 7fb7a82a9700 1 -- 172.21.15.37:0/4154535166 wait complete. 2024-06-10T00:51:49.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.492+0000 7fb7a82a9700 1 Processor -- start 2024-06-10T00:51:49.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.492+0000 7fb7a82a9700 1 -- start start 2024-06-10T00:51:49.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.492+0000 7fb7a82a9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7a006ac60 0x7fb7a019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:49.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.492+0000 7fb7a82a9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7a006b620 0x7fb7a019edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:49.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.492+0000 7fb7a82a9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7a0108580 0x7fb7a0198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:49.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.492+0000 7fb7a82a9700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a0112840 con 0x7fb7a006ac60 2024-06-10T00:51:49.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.492+0000 7fb7a82a9700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a01126c0 con 0x7fb7a006b620 2024-06-10T00:51:49.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.492+0000 7fb7a82a9700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a01129c0 con 0x7fb7a0108580 2024-06-10T00:51:49.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.492+0000 7fb7a6045700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7a006ac60 0x7fb7a019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:49.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.492+0000 7fb7a6045700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7a006ac60 0x7fb7a019e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39554/0 (socket says 172.21.15.37:39554) 2024-06-10T00:51:49.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.492+0000 7fb7a6045700 1 -- 172.21.15.37:0/2426697982 learned_addr learned my addr 172.21.15.37:0/2426697982 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:51:49.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.492+0000 7fb7a5844700 1 --2- 172.21.15.37:0/2426697982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7a006b620 0x7fb7a019edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:49.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.492+0000 7fb7a6846700 1 --2- 172.21.15.37:0/2426697982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7a0108580 0x7fb7a0198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:49.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.492+0000 7fb7a6045700 1 -- 172.21.15.37:0/2426697982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7a0108580 msgr2=0x7fb7a0198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:49.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.492+0000 7fb7a6045700 1 --2- 172.21.15.37:0/2426697982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7a0108580 0x7fb7a0198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:49.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.492+0000 7fb7a6045700 1 -- 172.21.15.37:0/2426697982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7a006b620 msgr2=0x7fb7a019edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:49.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.492+0000 7fb7a6045700 1 --2- 172.21.15.37:0/2426697982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7a006b620 0x7fb7a019edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:49.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.493+0000 7fb7a6045700 1 -- 172.21.15.37:0/2426697982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a01991a0 con 0x7fb7a006ac60 2024-06-10T00:51:49.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.493+0000 7fb7a6045700 1 --2- 172.21.15.37:0/2426697982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7a006ac60 0x7fb7a019e880 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00ba50 tx=0x7fb79c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:51:49.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.493+0000 7fb7977fe700 1 -- 172.21.15.37:0/2426697982 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c019070 con 0x7fb7a006ac60 2024-06-10T00:51:49.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.493+0000 7fb7a82a9700 1 -- 172.21.15.37:0/2426697982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7a0199490 con 0x7fb7a006ac60 2024-06-10T00:51:49.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.493+0000 7fb7977fe700 1 -- 172.21.15.37:0/2426697982 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb79c014440 con 0x7fb7a006ac60 2024-06-10T00:51:49.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.493+0000 7fb7977fe700 1 -- 172.21.15.37:0/2426697982 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c0075d0 con 0x7fb7a006ac60 2024-06-10T00:51:49.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.493+0000 7fb7a82a9700 1 -- 172.21.15.37:0/2426697982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7a01a5750 con 0x7fb7a006ac60 2024-06-10T00:51:49.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.494+0000 7fb7a82a9700 1 -- 172.21.15.37:0/2426697982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7a010aa10 con 0x7fb7a006ac60 2024-06-10T00:51:49.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.495+0000 7fb7977fe700 1 -- 172.21.15.37:0/2426697982 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb79c01c070 con 0x7fb7a006ac60 2024-06-10T00:51:49.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.496+0000 7fb7977fe700 1 --2- 172.21.15.37:0/2426697982 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb78c0a2c20 0x7fb78c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:49.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.496+0000 7fb7a5844700 1 --2- 172.21.15.37:0/2426697982 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb78c0a2c20 0x7fb78c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:49.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.496+0000 7fb7977fe700 1 -- 172.21.15.37:0/2426697982 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb79c0cdf80 con 0x7fb7a006ac60 2024-06-10T00:51:49.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.496+0000 7fb7a5844700 1 --2- 172.21.15.37:0/2426697982 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb78c0a2c20 0x7fb78c0a50e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0078950 tx=0x7fb790005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:51:49.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.501+0000 7fb7977fe700 1 -- 172.21.15.37:0/2426697982 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb79c09b380 con 0x7fb7a006ac60 2024-06-10T00:51:49.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:49 smithi167 ceph-mon[36917]: Deploying daemon mds.j on smithi037 2024-06-10T00:51:49.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.751+0000 7fb7a82a9700 1 -- 172.21.15.37:0/2426697982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7a0069180 con 0x7fb7a006ac60 2024-06-10T00:51:49.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.752+0000 7fb7977fe700 1 -- 172.21.15.37:0/2426697982 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7fb79c09ac80 con 0x7fb7a006ac60 2024-06-10T00:51:49.753 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:51:49.753 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-06-10T00:51:49.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.754+0000 7fb7a82a9700 1 -- 172.21.15.37:0/2426697982 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb78c0a2c20 msgr2=0x7fb78c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.754+0000 7fb7a82a9700 1 --2- 172.21.15.37:0/2426697982 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb78c0a2c20 0x7fb78c0a50e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0078950 tx=0x7fb790005fb0 comp rx=0 tx=0).stop 2024-06-10T00:51:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.754+0000 7fb7a82a9700 1 -- 172.21.15.37:0/2426697982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7a006ac60 msgr2=0x7fb7a019e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.754+0000 7fb7a82a9700 1 --2- 172.21.15.37:0/2426697982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7a006ac60 0x7fb7a019e880 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00ba50 tx=0x7fb79c00bd60 comp rx=0 tx=0).stop 2024-06-10T00:51:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.754+0000 7fb7a82a9700 1 -- 172.21.15.37:0/2426697982 shutdown_connections 2024-06-10T00:51:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.754+0000 7fb7a82a9700 1 --2- 172.21.15.37:0/2426697982 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb78c0a2c20 0x7fb78c0a50e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.754+0000 7fb7a82a9700 1 --2- 172.21.15.37:0/2426697982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7a0108580 0x7fb7a0198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.754+0000 7fb7a82a9700 1 --2- 172.21.15.37:0/2426697982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7a006ac60 0x7fb7a019e880 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.754+0000 7fb7a82a9700 1 --2- 172.21.15.37:0/2426697982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7a006b620 0x7fb7a019edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.754+0000 7fb7a82a9700 1 -- 172.21.15.37:0/2426697982 >> 172.21.15.37:0/2426697982 conn(0x7fb7a0078430 msgr2=0x7fb7a00ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:51:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.754+0000 7fb7a82a9700 1 -- 172.21.15.37:0/2426697982 shutdown_connections 2024-06-10T00:51:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:49.754+0000 7fb7a82a9700 1 -- 172.21.15.37:0/2426697982 wait complete. 2024-06-10T00:51:49.756 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 3 2024-06-10T00:51:50.299 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:50 smithi037 ceph-mon[32192]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:50.299 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:50 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2426697982' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:51:50.550 INFO:journalctl@ceph.mds.j.smithi037.stdout:Jun 10 00:51:50 smithi037 systemd[1]: Starting Ceph mds.j for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T00:51:50.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:50 smithi167 ceph-mon[36917]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:50.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:50 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2426697982' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:51:50.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:50 smithi094 ceph-mon[36708]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:50.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:50 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2426697982' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:51:51.275 INFO:journalctl@ceph.mds.j.smithi037.stdout:Jun 10 00:51:51 smithi037 podman[70572]: 2024-06-10 00:51:51.051139461 +0000 UTC m=+0.018131308 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:51:51.276 INFO:journalctl@ceph.mds.j.smithi037.stdout:Jun 10 00:51:51 smithi037 podman[70572]: 2024-06-10 00:51:51.159529041 +0000 UTC m=+0.126520873 container create c1217c6e90fe832be6c35f29ca5b54d2962071d3c3e46a92aa965538b9c04ea9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-j, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-06-10T00:51:51.646 INFO:journalctl@ceph.mds.j.smithi037.stdout:Jun 10 00:51:51 smithi037 podman[70572]: 2024-06-10 00:51:51.39571547 +0000 UTC m=+0.362707301 container init c1217c6e90fe832be6c35f29ca5b54d2962071d3c3e46a92aa965538b9c04ea9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-j, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-06-10T00:51:51.646 INFO:journalctl@ceph.mds.j.smithi037.stdout:Jun 10 00:51:51 smithi037 podman[70572]: 2024-06-10 00:51:51.398760873 +0000 UTC m=+0.365752722 container start c1217c6e90fe832be6c35f29ca5b54d2962071d3c3e46a92aa965538b9c04ea9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-06-10T00:51:51.646 INFO:journalctl@ceph.mds.j.smithi037.stdout:Jun 10 00:51:51 smithi037 ceph-mds[70591]: -- 172.21.15.37:0/3324238741 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563c339a7a40 con 0x563c3397ac00 2024-06-10T00:51:51.646 INFO:journalctl@ceph.mds.j.smithi037.stdout:Jun 10 00:51:51 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-j[70587]: starting mds.j at 2024-06-10T00:51:51.646 INFO:journalctl@ceph.mds.j.smithi037.stdout:Jun 10 00:51:51 smithi037 bash[70572]: c1217c6e90fe832be6c35f29ca5b54d2962071d3c3e46a92aa965538b9c04ea9 2024-06-10T00:51:51.646 INFO:journalctl@ceph.mds.j.smithi037.stdout:Jun 10 00:51:51 smithi037 systemd[1]: Started Ceph mds.j for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T00:51:52.441 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:52 smithi037 ceph-mon[32192]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:52.441 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:52 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:52.441 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:52 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:52.442 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:52 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:52.442 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:52 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:51:52.442 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:52 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:51:52.442 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:52 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:52.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:52 smithi167 ceph-mon[36917]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:52.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:52 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:52.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:52 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:52.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:52 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:52.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:52 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:51:52.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:52 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:51:52.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:52 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:52.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:52 smithi094 ceph-mon[36708]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:52.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:52 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:52.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:52 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:52.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:52 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:52.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:52 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:51:52.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:52 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:51:52.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:52 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:52.974 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:51:53.280 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:53 smithi037 ceph-mon[32192]: Deploying daemon mds.d on smithi037 2024-06-10T00:51:53.280 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:53 smithi037 ceph-mon[32192]: mds.? [v2:172.21.15.37:6834/1021220889,v1:172.21.15.37:6835/1021220889] up:boot 2024-06-10T00:51:53.280 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:53 smithi037 ceph-mon[32192]: fsmap 3 up:standby 2024-06-10T00:51:53.280 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:53 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-10T00:51:53.356 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:51:53.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:53 smithi167 ceph-mon[36917]: Deploying daemon mds.d on smithi037 2024-06-10T00:51:53.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:53 smithi167 ceph-mon[36917]: mds.? [v2:172.21.15.37:6834/1021220889,v1:172.21.15.37:6835/1021220889] up:boot 2024-06-10T00:51:53.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:53 smithi167 ceph-mon[36917]: fsmap 3 up:standby 2024-06-10T00:51:53.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:53 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-10T00:51:53.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:53 smithi094 ceph-mon[36708]: Deploying daemon mds.d on smithi037 2024-06-10T00:51:53.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:53 smithi094 ceph-mon[36708]: mds.? [v2:172.21.15.37:6834/1021220889,v1:172.21.15.37:6835/1021220889] up:boot 2024-06-10T00:51:53.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:53 smithi094 ceph-mon[36708]: fsmap 3 up:standby 2024-06-10T00:51:53.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:53 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-10T00:51:54.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:54 smithi167 ceph-mon[36917]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:54.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:54 smithi094 ceph-mon[36708]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:54.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:54 smithi037 ceph-mon[32192]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:55.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.190+0000 7f372458c700 1 -- 172.21.15.37:0/1511320448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f371c1030f0 msgr2=0x7f371c107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:55.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.190+0000 7f372458c700 1 --2- 172.21.15.37:0/1511320448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f371c1030f0 0x7f371c107110 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f370c0099c0 tx=0x7f370c009cd0 comp rx=0 tx=0).stop 2024-06-10T00:51:55.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.190+0000 7f372458c700 1 -- 172.21.15.37:0/1511320448 shutdown_connections 2024-06-10T00:51:55.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.190+0000 7f372458c700 1 --2- 172.21.15.37:0/1511320448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f371c107840 0x7f371c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:55.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.190+0000 7f372458c700 1 --2- 172.21.15.37:0/1511320448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f371c102740 0x7f371c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:55.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.190+0000 7f372458c700 1 --2- 172.21.15.37:0/1511320448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f371c1030f0 0x7f371c107110 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:55.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.190+0000 7f372458c700 1 -- 172.21.15.37:0/1511320448 >> 172.21.15.37:0/1511320448 conn(0x7f371c0795f0 msgr2=0x7f371c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:51:55.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.191+0000 7f372458c700 1 -- 172.21.15.37:0/1511320448 shutdown_connections 2024-06-10T00:51:55.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.191+0000 7f372458c700 1 -- 172.21.15.37:0/1511320448 wait complete. 2024-06-10T00:51:55.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.191+0000 7f372458c700 1 Processor -- start 2024-06-10T00:51:55.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.191+0000 7f372458c700 1 -- start start 2024-06-10T00:51:55.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.191+0000 7f372458c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f371c102740 0x7f371c19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:55.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.191+0000 7f372458c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f371c1030f0 0x7f371c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:55.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.191+0000 7f372458c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f371c107840 0x7f371c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:55.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.191+0000 7f372458c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f371c112750 con 0x7f371c102740 2024-06-10T00:51:55.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.191+0000 7f372458c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f371c1125d0 con 0x7f371c1030f0 2024-06-10T00:51:55.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.191+0000 7f372458c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f371c1128d0 con 0x7f371c107840 2024-06-10T00:51:55.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.191+0000 7f3722328700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f371c102740 0x7f371c19e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:55.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.192+0000 7f3722328700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f371c102740 0x7f371c19e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48662/0 (socket says 172.21.15.37:48662) 2024-06-10T00:51:55.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.192+0000 7f3722328700 1 -- 172.21.15.37:0/1062851375 learned_addr learned my addr 172.21.15.37:0/1062851375 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:51:55.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.192+0000 7f3721b27700 1 --2- 172.21.15.37:0/1062851375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f371c1030f0 0x7f371c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:55.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.192+0000 7f3722b29700 1 --2- 172.21.15.37:0/1062851375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f371c107840 0x7f371c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:55.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.192+0000 7f3722328700 1 -- 172.21.15.37:0/1062851375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f371c107840 msgr2=0x7f371c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:55.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.192+0000 7f3722328700 1 --2- 172.21.15.37:0/1062851375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f371c107840 0x7f371c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:55.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.192+0000 7f3722328700 1 -- 172.21.15.37:0/1062851375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f371c1030f0 msgr2=0x7f371c19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:55.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.192+0000 7f3722328700 1 --2- 172.21.15.37:0/1062851375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f371c1030f0 0x7f371c19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:55.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.192+0000 7f3722328700 1 -- 172.21.15.37:0/1062851375 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371c199110 con 0x7f371c102740 2024-06-10T00:51:55.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.192+0000 7f3722b29700 1 --2- 172.21.15.37:0/1062851375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f371c107840 0x7f371c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:51:55.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.192+0000 7f3722328700 1 --2- 172.21.15.37:0/1062851375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f371c102740 0x7f371c19e7f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3718007e20 tx=0x7f371800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:51:55.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.192+0000 7f37137fe700 1 -- 172.21.15.37:0/1062851375 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3718016020 con 0x7f371c102740 2024-06-10T00:51:55.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.192+0000 7f37137fe700 1 -- 172.21.15.37:0/1062851375 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f371800ad40 con 0x7f371c102740 2024-06-10T00:51:55.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.192+0000 7f372458c700 1 -- 172.21.15.37:0/1062851375 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f371c199400 con 0x7f371c102740 2024-06-10T00:51:55.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.192+0000 7f37137fe700 1 -- 172.21.15.37:0/1062851375 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3718011700 con 0x7f371c102740 2024-06-10T00:51:55.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.192+0000 7f372458c700 1 -- 172.21.15.37:0/1062851375 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f371c1a56c0 con 0x7f371c102740 2024-06-10T00:51:55.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.193+0000 7f372458c700 1 -- 172.21.15.37:0/1062851375 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f371c069180 con 0x7f371c102740 2024-06-10T00:51:55.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.195+0000 7f37137fe700 1 -- 172.21.15.37:0/1062851375 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3718019020 con 0x7f371c102740 2024-06-10T00:51:55.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.195+0000 7f37137fe700 1 --2- 172.21.15.37:0/1062851375 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f37080a2c20 0x7f37080a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:51:55.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.195+0000 7f3721b27700 1 --2- 172.21.15.37:0/1062851375 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f37080a2c20 0x7f37080a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:51:55.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.196+0000 7f37137fe700 1 -- 172.21.15.37:0/1062851375 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f37180cecb0 con 0x7f371c102740 2024-06-10T00:51:55.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.196+0000 7f3721b27700 1 --2- 172.21.15.37:0/1062851375 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f37080a2c20 0x7f37080a50e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f370c000c00 tx=0x7f370c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:51:55.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.201+0000 7f37137fe700 1 -- 172.21.15.37:0/1062851375 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f37180182c0 con 0x7f371c102740 2024-06-10T00:51:55.455 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:51:55.456 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-06-10T00:51:55.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.454+0000 7f372458c700 1 -- 172.21.15.37:0/1062851375 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f371c199ee0 con 0x7f371c102740 2024-06-10T00:51:55.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.455+0000 7f37137fe700 1 -- 172.21.15.37:0/1062851375 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f371809c0b0 con 0x7f371c102740 2024-06-10T00:51:55.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.457+0000 7f372458c700 1 -- 172.21.15.37:0/1062851375 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f37080a2c20 msgr2=0x7f37080a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:55.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.457+0000 7f372458c700 1 --2- 172.21.15.37:0/1062851375 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f37080a2c20 0x7f37080a50e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f370c000c00 tx=0x7f370c005fb0 comp rx=0 tx=0).stop 2024-06-10T00:51:55.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.457+0000 7f372458c700 1 -- 172.21.15.37:0/1062851375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f371c102740 msgr2=0x7f371c19e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:51:55.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.457+0000 7f372458c700 1 --2- 172.21.15.37:0/1062851375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f371c102740 0x7f371c19e7f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3718007e20 tx=0x7f371800a340 comp rx=0 tx=0).stop 2024-06-10T00:51:55.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.457+0000 7f372458c700 1 -- 172.21.15.37:0/1062851375 shutdown_connections 2024-06-10T00:51:55.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.457+0000 7f372458c700 1 --2- 172.21.15.37:0/1062851375 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f37080a2c20 0x7f37080a50e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:55.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.457+0000 7f372458c700 1 --2- 172.21.15.37:0/1062851375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f371c107840 0x7f371c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:55.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.457+0000 7f372458c700 1 --2- 172.21.15.37:0/1062851375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f371c102740 0x7f371c19e7f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:55.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.457+0000 7f372458c700 1 --2- 172.21.15.37:0/1062851375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f371c1030f0 0x7f371c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:51:55.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.457+0000 7f372458c700 1 -- 172.21.15.37:0/1062851375 >> 172.21.15.37:0/1062851375 conn(0x7f371c0795f0 msgr2=0x7f371c100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:51:55.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.458+0000 7f372458c700 1 -- 172.21.15.37:0/1062851375 shutdown_connections 2024-06-10T00:51:55.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:51:55.458+0000 7f372458c700 1 -- 172.21.15.37:0/1062851375 wait complete. 2024-06-10T00:51:55.459 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 4 2024-06-10T00:51:55.849 INFO:journalctl@ceph.mds.d.smithi037.stdout:Jun 10 00:51:55 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-d[71025]: starting mds.d at 2024-06-10T00:51:56.145 INFO:journalctl@ceph.mds.d.smithi037.stdout:Jun 10 00:51:55 smithi037 bash[70980]: 2637085aa9ef443944fb215b4ff89b8d17f6867f4e34349ae6e19f4279eafcf5 2024-06-10T00:51:56.146 INFO:journalctl@ceph.mds.d.smithi037.stdout:Jun 10 00:51:55 smithi037 systemd[1]: Started Ceph mds.d for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T00:51:56.470 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:56 smithi037 ceph-mon[32192]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:56.470 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:56 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1062851375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:51:56.470 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:56 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:56.470 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:56 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:56.470 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:56 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:56.470 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:56 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:51:56.470 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:56 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:51:56.470 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:56 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:56.555 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:56 smithi094 ceph-mon[36708]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:56.555 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:56 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1062851375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:51:56.555 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:56 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:56.555 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:56 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:56.555 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:56 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:56.555 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:56 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:51:56.555 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:56 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:51:56.555 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:56 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:56.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:56 smithi167 ceph-mon[36917]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:56.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:56 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1062851375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:51:56.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:56 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:56.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:56 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:56.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:56 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:51:56.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:56 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:51:56.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:56 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:51:56.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:56 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:51:57.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:57 smithi167 ceph-mon[36917]: Deploying daemon mds.e on smithi094 2024-06-10T00:51:57.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:57 smithi167 ceph-mon[36917]: mds.? [v2:172.21.15.37:6836/4093234668,v1:172.21.15.37:6837/4093234668] up:boot 2024-06-10T00:51:57.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:57 smithi167 ceph-mon[36917]: fsmap 4 up:standby 2024-06-10T00:51:57.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:57 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-10T00:51:57.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:57 smithi094 ceph-mon[36708]: Deploying daemon mds.e on smithi094 2024-06-10T00:51:57.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:57 smithi094 ceph-mon[36708]: mds.? [v2:172.21.15.37:6836/4093234668,v1:172.21.15.37:6837/4093234668] up:boot 2024-06-10T00:51:57.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:57 smithi094 ceph-mon[36708]: fsmap 4 up:standby 2024-06-10T00:51:57.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:57 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-10T00:51:57.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:57 smithi037 ceph-mon[32192]: Deploying daemon mds.e on smithi094 2024-06-10T00:51:57.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:57 smithi037 ceph-mon[32192]: mds.? [v2:172.21.15.37:6836/4093234668,v1:172.21.15.37:6837/4093234668] up:boot 2024-06-10T00:51:57.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:57 smithi037 ceph-mon[32192]: fsmap 4 up:standby 2024-06-10T00:51:57.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:57 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-10T00:51:58.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:58 smithi094 ceph-mon[36708]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:58.365 INFO:journalctl@ceph.mds.e.smithi094.stdout:Jun 10 00:51:58 smithi094 systemd[1]: Starting Ceph mds.e for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T00:51:58.500 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:51:58.529 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:58 smithi037 ceph-mon[32192]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:58.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:58 smithi167 ceph-mon[36917]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:51:58.730 INFO:journalctl@ceph.mds.e.smithi094.stdout:Jun 10 00:51:58 smithi094 podman[66626]: 2024-06-10 00:51:58.371550744 +0000 UTC m=+0.016978638 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:51:58.730 INFO:journalctl@ceph.mds.e.smithi094.stdout:Jun 10 00:51:58 smithi094 podman[66626]: 2024-06-10 00:51:58.502505841 +0000 UTC m=+0.147933734 container create e7986ba36127b3a9a4e2d7cbbe74e4e866104e5c0e42c2963baac63e9f73f822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-e, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-06-10T00:51:58.730 INFO:journalctl@ceph.mds.e.smithi094.stdout:Jun 10 00:51:58 smithi094 podman[66626]: 2024-06-10 00:51:58.7304928 +0000 UTC m=+0.375920689 container init e7986ba36127b3a9a4e2d7cbbe74e4e866104e5c0e42c2963baac63e9f73f822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-e, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T00:51:58.764 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:51:59.114 INFO:journalctl@ceph.mds.e.smithi094.stdout:Jun 10 00:51:58 smithi094 podman[66626]: 2024-06-10 00:51:58.733285567 +0000 UTC m=+0.378713463 container start e7986ba36127b3a9a4e2d7cbbe74e4e866104e5c0e42c2963baac63e9f73f822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-e, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2) 2024-06-10T00:51:59.114 INFO:journalctl@ceph.mds.e.smithi094.stdout:Jun 10 00:51:58 smithi094 ceph-mds[66646]: -- 172.21.15.94:0/842270147 <== mon.2 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55988598ba40 con 0x55988595f000 2024-06-10T00:51:59.115 INFO:journalctl@ceph.mds.e.smithi094.stdout:Jun 10 00:51:58 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-e[66642]: starting mds.e at 2024-06-10T00:51:59.115 INFO:journalctl@ceph.mds.e.smithi094.stdout:Jun 10 00:51:58 smithi094 bash[66626]: e7986ba36127b3a9a4e2d7cbbe74e4e866104e5c0e42c2963baac63e9f73f822 2024-06-10T00:51:59.115 INFO:journalctl@ceph.mds.e.smithi094.stdout:Jun 10 00:51:58 smithi094 systemd[1]: Started Ceph mds.e for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T00:52:00.132 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:59 smithi037 ceph-mon[32192]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:00.133 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:59 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:00.133 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:59 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:00.133 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:59 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:00.133 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:59 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:52:00.133 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:59 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:52:00.133 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:59 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:52:00.133 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:59 smithi037 ceph-mon[32192]: Deploying daemon mds.f on smithi167 2024-06-10T00:52:00.133 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:59 smithi037 ceph-mon[32192]: mds.? [v2:172.21.15.94:6834/570288048,v1:172.21.15.94:6835/570288048] up:boot 2024-06-10T00:52:00.133 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:59 smithi037 ceph-mon[32192]: fsmap 5 up:standby 2024-06-10T00:52:00.133 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:51:59 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-10T00:52:00.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:59 smithi167 ceph-mon[36917]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:00.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:59 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:00.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:59 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:00.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:59 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:00.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:59 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:52:00.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:59 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:52:00.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:59 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:52:00.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:59 smithi167 ceph-mon[36917]: Deploying daemon mds.f on smithi167 2024-06-10T00:52:00.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:59 smithi167 ceph-mon[36917]: mds.? [v2:172.21.15.94:6834/570288048,v1:172.21.15.94:6835/570288048] up:boot 2024-06-10T00:52:00.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:59 smithi167 ceph-mon[36917]: fsmap 5 up:standby 2024-06-10T00:52:00.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:51:59 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-10T00:52:00.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:59 smithi094 ceph-mon[36708]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:00.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:59 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:00.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:59 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:00.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:59 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:00.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:59 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:52:00.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:59 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:52:00.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:59 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:52:00.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:59 smithi094 ceph-mon[36708]: Deploying daemon mds.f on smithi167 2024-06-10T00:52:00.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:59 smithi094 ceph-mon[36708]: mds.? [v2:172.21.15.94:6834/570288048,v1:172.21.15.94:6835/570288048] up:boot 2024-06-10T00:52:00.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:59 smithi094 ceph-mon[36708]: fsmap 5 up:standby 2024-06-10T00:52:00.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:51:59 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-10T00:52:00.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.372+0000 7f0a03189700 1 -- 172.21.15.37:0/2730476859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09fc108500 msgr2=0x7f09fc1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:00.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.372+0000 7f0a03189700 1 --2- 172.21.15.37:0/2730476859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09fc108500 0x7f09fc1088e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f09ec009990 tx=0x7f09ec009ca0 comp rx=0 tx=0).stop 2024-06-10T00:52:00.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.372+0000 7f0a03189700 1 -- 172.21.15.37:0/2730476859 shutdown_connections 2024-06-10T00:52:00.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.372+0000 7f0a03189700 1 --2- 172.21.15.37:0/2730476859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09fc06b580 0x7f09fc10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:00.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.372+0000 7f0a03189700 1 --2- 172.21.15.37:0/2730476859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09fc108500 0x7f09fc1088e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:00.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.372+0000 7f0a03189700 1 --2- 172.21.15.37:0/2730476859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09fc06abc0 0x7f09fc06b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:00.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.372+0000 7f0a03189700 1 -- 172.21.15.37:0/2730476859 >> 172.21.15.37:0/2730476859 conn(0x7f09fc078340 msgr2=0x7f09fc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:00.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.372+0000 7f0a03189700 1 -- 172.21.15.37:0/2730476859 shutdown_connections 2024-06-10T00:52:00.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.372+0000 7f0a03189700 1 -- 172.21.15.37:0/2730476859 wait complete. 2024-06-10T00:52:00.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.373+0000 7f0a03189700 1 Processor -- start 2024-06-10T00:52:00.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.373+0000 7f0a03189700 1 -- start start 2024-06-10T00:52:00.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.373+0000 7f0a03189700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09fc06abc0 0x7f09fc19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:00.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.373+0000 7f0a03189700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09fc06b580 0x7f09fc19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:00.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.373+0000 7f0a03189700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09fc108500 0x7f09fc1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:00.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.373+0000 7f0a03189700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc112760 con 0x7f09fc108500 2024-06-10T00:52:00.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.373+0000 7f0a03189700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc1125e0 con 0x7f09fc06b580 2024-06-10T00:52:00.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.373+0000 7f0a03189700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc1128e0 con 0x7f09fc06abc0 2024-06-10T00:52:00.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.373+0000 7f0a01726700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09fc108500 0x7f09fc1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:00.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.373+0000 7f0a01726700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09fc108500 0x7f09fc1989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48708/0 (socket says 172.21.15.37:48708) 2024-06-10T00:52:00.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.373+0000 7f0a01726700 1 -- 172.21.15.37:0/1077049079 learned_addr learned my addr 172.21.15.37:0/1077049079 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:52:00.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.373+0000 7f0a00f25700 1 --2- 172.21.15.37:0/1077049079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09fc06abc0 0x7f09fc19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:00.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.373+0000 7f09fbfff700 1 --2- 172.21.15.37:0/1077049079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09fc06b580 0x7f09fc19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:00.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.373+0000 7f0a01726700 1 -- 172.21.15.37:0/1077049079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09fc06abc0 msgr2=0x7f09fc19e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:00.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.373+0000 7f0a01726700 1 --2- 172.21.15.37:0/1077049079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09fc06abc0 0x7f09fc19e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:00.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.373+0000 7f0a01726700 1 -- 172.21.15.37:0/1077049079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09fc06b580 msgr2=0x7f09fc19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:00.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.373+0000 7f0a01726700 1 --2- 172.21.15.37:0/1077049079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09fc06b580 0x7f09fc19ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:00.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.373+0000 7f0a01726700 1 -- 172.21.15.37:0/1077049079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09fc199100 con 0x7f09fc108500 2024-06-10T00:52:00.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.373+0000 7f09fbfff700 1 --2- 172.21.15.37:0/1077049079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09fc06b580 0x7f09fc19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:52:00.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.374+0000 7f0a01726700 1 --2- 172.21.15.37:0/1077049079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09fc108500 0x7f09fc1989a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f09f4007d70 tx=0x7f09f400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:00.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.374+0000 7f09f9ffb700 1 -- 172.21.15.37:0/1077049079 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f4016020 con 0x7f09fc108500 2024-06-10T00:52:00.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.374+0000 7f09f9ffb700 1 -- 172.21.15.37:0/1077049079 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09f400ad40 con 0x7f09fc108500 2024-06-10T00:52:00.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.374+0000 7f0a03189700 1 -- 172.21.15.37:0/1077049079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09fc1993f0 con 0x7f09fc108500 2024-06-10T00:52:00.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.374+0000 7f09f9ffb700 1 -- 172.21.15.37:0/1077049079 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f4011600 con 0x7f09fc108500 2024-06-10T00:52:00.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.374+0000 7f0a03189700 1 -- 172.21.15.37:0/1077049079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09fc1a56b0 con 0x7f09fc108500 2024-06-10T00:52:00.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.375+0000 7f0a03189700 1 -- 172.21.15.37:0/1077049079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09fc10a9e0 con 0x7f09fc108500 2024-06-10T00:52:00.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.376+0000 7f09f9ffb700 1 -- 172.21.15.37:0/1077049079 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f09f4019020 con 0x7f09fc108500 2024-06-10T00:52:00.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.377+0000 7f09f9ffb700 1 --2- 172.21.15.37:0/1077049079 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f09e40a2c20 0x7f09e40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:00.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.377+0000 7f0a00f25700 1 --2- 172.21.15.37:0/1077049079 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f09e40a2c20 0x7f09e40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:00.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.377+0000 7f09f9ffb700 1 -- 172.21.15.37:0/1077049079 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f09f40ceba0 con 0x7f09fc108500 2024-06-10T00:52:00.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.377+0000 7f0a00f25700 1 --2- 172.21.15.37:0/1077049079 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f09e40a2c20 0x7f09e40a50e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00b5c0 tx=0x7f09ec005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:00.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.382+0000 7f09f9ffb700 1 -- 172.21.15.37:0/1077049079 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f09f4020020 con 0x7f09fc108500 2024-06-10T00:52:00.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.638+0000 7f0a03189700 1 -- 172.21.15.37:0/1077049079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09fc069180 con 0x7f09fc108500 2024-06-10T00:52:00.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.639+0000 7f09f9ffb700 1 -- 172.21.15.37:0/1077049079 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7f09f409bf00 con 0x7f09fc108500 2024-06-10T00:52:00.639 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:52:00.639 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-06-10T00:52:00.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.641+0000 7f0a03189700 1 -- 172.21.15.37:0/1077049079 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f09e40a2c20 msgr2=0x7f09e40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:00.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.641+0000 7f0a03189700 1 --2- 172.21.15.37:0/1077049079 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f09e40a2c20 0x7f09e40a50e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00b5c0 tx=0x7f09ec005960 comp rx=0 tx=0).stop 2024-06-10T00:52:00.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.641+0000 7f0a03189700 1 -- 172.21.15.37:0/1077049079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09fc108500 msgr2=0x7f09fc1989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:00.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.641+0000 7f0a03189700 1 --2- 172.21.15.37:0/1077049079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09fc108500 0x7f09fc1989a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f09f4007d70 tx=0x7f09f400a340 comp rx=0 tx=0).stop 2024-06-10T00:52:00.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.641+0000 7f0a03189700 1 -- 172.21.15.37:0/1077049079 shutdown_connections 2024-06-10T00:52:00.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.641+0000 7f0a03189700 1 --2- 172.21.15.37:0/1077049079 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f09e40a2c20 0x7f09e40a50e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:00.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.641+0000 7f0a03189700 1 --2- 172.21.15.37:0/1077049079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09fc108500 0x7f09fc1989a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:00.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.641+0000 7f0a03189700 1 --2- 172.21.15.37:0/1077049079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09fc06abc0 0x7f09fc19e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:00.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.641+0000 7f0a03189700 1 --2- 172.21.15.37:0/1077049079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09fc06b580 0x7f09fc19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:00.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.641+0000 7f0a03189700 1 -- 172.21.15.37:0/1077049079 >> 172.21.15.37:0/1077049079 conn(0x7f09fc078340 msgr2=0x7f09fc0ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:00.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.641+0000 7f0a03189700 1 -- 172.21.15.37:0/1077049079 shutdown_connections 2024-06-10T00:52:00.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:00.641+0000 7f0a03189700 1 -- 172.21.15.37:0/1077049079 wait complete. 2024-06-10T00:52:00.643 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 6 2024-06-10T00:52:00.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:00 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1077049079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:52:01.266 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:00 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1077049079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:52:01.283 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:00 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1077049079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:52:01.283 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 00:52:01 smithi167 systemd[1]: Starting Ceph mds.f for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T00:52:01.570 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 00:52:01 smithi167 podman[67353]: 2024-06-10 00:52:01.1866994 +0000 UTC m=+0.017280147 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:52:01.570 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 00:52:01 smithi167 podman[67353]: 2024-06-10 00:52:01.325778248 +0000 UTC m=+0.156358988 container create 3f963f7d3e44deceddeb70d919ef039e2a9515c333c36a02c7c319623b2c6b34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-f, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, ceph=True) 2024-06-10T00:52:01.570 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 00:52:01 smithi167 podman[67353]: 2024-06-10 00:52:01.570705438 +0000 UTC m=+0.401286190 container init 3f963f7d3e44deceddeb70d919ef039e2a9515c333c36a02c7c319623b2c6b34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-f, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-06-10T00:52:01.839 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 00:52:01 smithi167 podman[67353]: 2024-06-10 00:52:01.573696593 +0000 UTC m=+0.404277336 container start 3f963f7d3e44deceddeb70d919ef039e2a9515c333c36a02c7c319623b2c6b34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-f, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-06-10T00:52:01.839 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 00:52:01 smithi167 ceph-mds[67372]: -- 172.21.15.167:0/1722375704 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562c3741fa40 con 0x562c373f3000 2024-06-10T00:52:01.839 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 00:52:01 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-f[67368]: starting mds.f at 2024-06-10T00:52:01.839 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 00:52:01 smithi167 bash[67353]: 3f963f7d3e44deceddeb70d919ef039e2a9515c333c36a02c7c319623b2c6b34 2024-06-10T00:52:01.839 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 00:52:01 smithi167 systemd[1]: Started Ceph mds.f for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T00:52:02.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:01 smithi037 ceph-mon[32192]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:02.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:01 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:02.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:01 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:02.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:01 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:02.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:01 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:52:02.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:01 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:52:02.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:01 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:52:02.286 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:01 smithi094 ceph-mon[36708]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:02.286 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:01 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:02.287 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:01 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:02.287 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:01 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:02.287 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:01 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:52:02.287 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:01 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:52:02.287 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:01 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:52:02.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:01 smithi167 ceph-mon[36917]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:02.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:01 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:02.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:01 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:02.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:01 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:02.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:01 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:52:02.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:01 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:52:02.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:01 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:52:03.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:02 smithi037 ceph-mon[32192]: Deploying daemon mds.c on smithi167 2024-06-10T00:52:03.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:02 smithi037 ceph-mon[32192]: mds.? [v2:172.21.15.167:6834/3066042611,v1:172.21.15.167:6835/3066042611] up:boot 2024-06-10T00:52:03.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:02 smithi037 ceph-mon[32192]: fsmap 6 up:standby 2024-06-10T00:52:03.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:02 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T00:52:03.309 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:52:03.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:02 smithi167 ceph-mon[36917]: Deploying daemon mds.c on smithi167 2024-06-10T00:52:03.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:02 smithi167 ceph-mon[36917]: mds.? [v2:172.21.15.167:6834/3066042611,v1:172.21.15.167:6835/3066042611] up:boot 2024-06-10T00:52:03.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:02 smithi167 ceph-mon[36917]: fsmap 6 up:standby 2024-06-10T00:52:03.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:02 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T00:52:03.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:02 smithi094 ceph-mon[36708]: Deploying daemon mds.c on smithi167 2024-06-10T00:52:03.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:02 smithi094 ceph-mon[36708]: mds.? [v2:172.21.15.167:6834/3066042611,v1:172.21.15.167:6835/3066042611] up:boot 2024-06-10T00:52:03.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:02 smithi094 ceph-mon[36708]: fsmap 6 up:standby 2024-06-10T00:52:03.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:02 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T00:52:03.571 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:52:03.952 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:03 smithi167 ceph-mon[36917]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:03.952 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jun 10 00:52:03 smithi167 systemd[1]: Starting Ceph mds.c for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T00:52:04.006 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:03 smithi037 ceph-mon[32192]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:04.256 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jun 10 00:52:04 smithi167 podman[67659]: 2024-06-10 00:52:04.017390016 +0000 UTC m=+0.017223991 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:52:04.256 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jun 10 00:52:04 smithi167 podman[67659]: 2024-06-10 00:52:04.134579436 +0000 UTC m=+0.134413400 container create 73324939eab34dccc1ac6a85e525934bddc3f3e4cc136a24a2d8632ededab4c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-06-10T00:52:04.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:03 smithi094 ceph-mon[36708]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:04.841 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jun 10 00:52:04 smithi167 podman[67659]: 2024-06-10 00:52:04.471410796 +0000 UTC m=+0.471244762 container init 73324939eab34dccc1ac6a85e525934bddc3f3e4cc136a24a2d8632ededab4c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.license=GPLv2) 2024-06-10T00:52:04.842 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jun 10 00:52:04 smithi167 podman[67659]: 2024-06-10 00:52:04.474929368 +0000 UTC m=+0.474763342 container start 73324939eab34dccc1ac6a85e525934bddc3f3e4cc136a24a2d8632ededab4c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-06-10T00:52:04.842 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jun 10 00:52:04 smithi167 ceph-mds[67688]: -- 172.21.15.167:0/3137485988 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d02c833a40 con 0x55d02c807000 2024-06-10T00:52:04.842 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jun 10 00:52:04 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-c[67682]: starting mds.c at 2024-06-10T00:52:04.842 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jun 10 00:52:04 smithi167 bash[67659]: 73324939eab34dccc1ac6a85e525934bddc3f3e4cc136a24a2d8632ededab4c1 2024-06-10T00:52:04.842 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jun 10 00:52:04 smithi167 systemd[1]: Started Ceph mds.c for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T00:52:05.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.205+0000 7f4f32237700 1 -- 172.21.15.37:0/1770658420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f2c06ac60 msgr2=0x7f4f2c06b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:05.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.205+0000 7f4f32237700 1 --2- 172.21.15.37:0/1770658420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f2c06ac60 0x7f4f2c06b0e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4f200099c0 tx=0x7f4f20009cd0 comp rx=0 tx=0).stop 2024-06-10T00:52:05.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.205+0000 7f4f32237700 1 -- 172.21.15.37:0/1770658420 shutdown_connections 2024-06-10T00:52:05.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.205+0000 7f4f32237700 1 --2- 172.21.15.37:0/1770658420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f2c06b620 0x7f4f2c10f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:05.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.205+0000 7f4f32237700 1 --2- 172.21.15.37:0/1770658420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f2c108580 0x7f4f2c108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:05.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.205+0000 7f4f32237700 1 --2- 172.21.15.37:0/1770658420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f2c06ac60 0x7f4f2c06b0e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:05.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.205+0000 7f4f32237700 1 -- 172.21.15.37:0/1770658420 >> 172.21.15.37:0/1770658420 conn(0x7f4f2c078430 msgr2=0x7f4f2c078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:05.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.205+0000 7f4f32237700 1 -- 172.21.15.37:0/1770658420 shutdown_connections 2024-06-10T00:52:05.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.205+0000 7f4f32237700 1 -- 172.21.15.37:0/1770658420 wait complete. 2024-06-10T00:52:05.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.206+0000 7f4f32237700 1 Processor -- start 2024-06-10T00:52:05.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.206+0000 7f4f32237700 1 -- start start 2024-06-10T00:52:05.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.206+0000 7f4f32237700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f2c06ac60 0x7f4f2c101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:05.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.206+0000 7f4f32237700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f2c06b620 0x7f4f2c101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:05.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.206+0000 7f4f32237700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f2c108580 0x7f4f2c101e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:05.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.206+0000 7f4f32237700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c1127e0 con 0x7f4f2c108580 2024-06-10T00:52:05.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.206+0000 7f4f32237700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c112660 con 0x7f4f2c06ac60 2024-06-10T00:52:05.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.206+0000 7f4f32237700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c112960 con 0x7f4f2c06b620 2024-06-10T00:52:05.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.207+0000 7f4f2bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f2c108580 0x7f4f2c101e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:05.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.207+0000 7f4f2bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f2c108580 0x7f4f2c101e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44596/0 (socket says 172.21.15.37:44596) 2024-06-10T00:52:05.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.207+0000 7f4f2bfff700 1 -- 172.21.15.37:0/1400612911 learned_addr learned my addr 172.21.15.37:0/1400612911 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:52:05.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.207+0000 7f4f2affd700 1 --2- 172.21.15.37:0/1400612911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f2c06b620 0x7f4f2c101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:05.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.207+0000 7f4f2b7fe700 1 --2- 172.21.15.37:0/1400612911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f2c06ac60 0x7f4f2c101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:05.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.207+0000 7f4f2bfff700 1 -- 172.21.15.37:0/1400612911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f2c06b620 msgr2=0x7f4f2c101940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:05.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.207+0000 7f4f2bfff700 1 --2- 172.21.15.37:0/1400612911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f2c06b620 0x7f4f2c101940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:05.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.207+0000 7f4f2bfff700 1 -- 172.21.15.37:0/1400612911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f2c06ac60 msgr2=0x7f4f2c101400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:05.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.207+0000 7f4f2bfff700 1 --2- 172.21.15.37:0/1400612911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f2c06ac60 0x7f4f2c101400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:05.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.207+0000 7f4f2bfff700 1 -- 172.21.15.37:0/1400612911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f2c1a7150 con 0x7f4f2c108580 2024-06-10T00:52:05.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.207+0000 7f4f2bfff700 1 --2- 172.21.15.37:0/1400612911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f2c108580 0x7f4f2c101e80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4f24007ee0 tx=0x7f4f2400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:05.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.207+0000 7f4f28ff9700 1 -- 172.21.15.37:0/1400612911 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2400dd20 con 0x7f4f2c108580 2024-06-10T00:52:05.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.207+0000 7f4f32237700 1 -- 172.21.15.37:0/1400612911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f2c1a7440 con 0x7f4f2c108580 2024-06-10T00:52:05.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.207+0000 7f4f28ff9700 1 -- 172.21.15.37:0/1400612911 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f24004020 con 0x7f4f2c108580 2024-06-10T00:52:05.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.207+0000 7f4f28ff9700 1 -- 172.21.15.37:0/1400612911 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f24015660 con 0x7f4f2c108580 2024-06-10T00:52:05.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.207+0000 7f4f32237700 1 -- 172.21.15.37:0/1400612911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f2c1a7950 con 0x7f4f2c108580 2024-06-10T00:52:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.208+0000 7f4f32237700 1 -- 172.21.15.37:0/1400612911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f2c10aa10 con 0x7f4f2c108580 2024-06-10T00:52:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.210+0000 7f4f28ff9700 1 -- 172.21.15.37:0/1400612911 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4f240041d0 con 0x7f4f2c108580 2024-06-10T00:52:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.210+0000 7f4f28ff9700 1 --2- 172.21.15.37:0/1400612911 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4f140a2c20 0x7f4f140a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.210+0000 7f4f2b7fe700 1 --2- 172.21.15.37:0/1400612911 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4f140a2c20 0x7f4f140a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.210+0000 7f4f28ff9700 1 -- 172.21.15.37:0/1400612911 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4f240ce5f0 con 0x7f4f2c108580 2024-06-10T00:52:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.211+0000 7f4f2b7fe700 1 --2- 172.21.15.37:0/1400612911 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4f140a2c20 0x7f4f140a50e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0045b0 tx=0x7f4f1c00b3c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:05.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.215+0000 7f4f28ff9700 1 -- 172.21.15.37:0/1400612911 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4f24010070 con 0x7f4f2c108580 2024-06-10T00:52:05.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.467+0000 7f4f32237700 1 -- 172.21.15.37:0/1400612911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f2c10a3f0 con 0x7f4f2c108580 2024-06-10T00:52:05.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.468+0000 7f4f28ff9700 1 -- 172.21.15.37:0/1400612911 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7f4f2409b9f0 con 0x7f4f2c108580 2024-06-10T00:52:05.469 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:52:05.469 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-06-10T00:52:05.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.470+0000 7f4f32237700 1 -- 172.21.15.37:0/1400612911 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4f140a2c20 msgr2=0x7f4f140a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:05.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.470+0000 7f4f32237700 1 --2- 172.21.15.37:0/1400612911 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4f140a2c20 0x7f4f140a50e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0045b0 tx=0x7f4f1c00b3c0 comp rx=0 tx=0).stop 2024-06-10T00:52:05.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.470+0000 7f4f32237700 1 -- 172.21.15.37:0/1400612911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f2c108580 msgr2=0x7f4f2c101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:05.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.470+0000 7f4f32237700 1 --2- 172.21.15.37:0/1400612911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f2c108580 0x7f4f2c101e80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4f24007ee0 tx=0x7f4f2400d3b0 comp rx=0 tx=0).stop 2024-06-10T00:52:05.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.470+0000 7f4f32237700 1 -- 172.21.15.37:0/1400612911 shutdown_connections 2024-06-10T00:52:05.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.470+0000 7f4f32237700 1 --2- 172.21.15.37:0/1400612911 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4f140a2c20 0x7f4f140a50e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:05.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.470+0000 7f4f32237700 1 --2- 172.21.15.37:0/1400612911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f2c108580 0x7f4f2c101e80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:05.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.470+0000 7f4f32237700 1 --2- 172.21.15.37:0/1400612911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f2c06ac60 0x7f4f2c101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:05.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.470+0000 7f4f32237700 1 --2- 172.21.15.37:0/1400612911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f2c06b620 0x7f4f2c101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:05.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.470+0000 7f4f32237700 1 -- 172.21.15.37:0/1400612911 >> 172.21.15.37:0/1400612911 conn(0x7f4f2c078430 msgr2=0x7f4f2c0fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:05.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.470+0000 7f4f32237700 1 -- 172.21.15.37:0/1400612911 shutdown_connections 2024-06-10T00:52:05.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:05.470+0000 7f4f32237700 1 -- 172.21.15.37:0/1400612911 wait complete. 2024-06-10T00:52:05.473 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-06-10T00:52:05.769 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:05 smithi037 ceph-mon[32192]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:05.769 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:05 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:05.769 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:05 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:05.769 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:05 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:05.769 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:05 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:52:05.769 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:05 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:52:05.769 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:05 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:52:05.770 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:05 smithi037 ceph-mon[32192]: Deploying daemon mds.h on smithi094 2024-06-10T00:52:05.770 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:05 smithi037 ceph-mon[32192]: mds.? [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] up:boot 2024-06-10T00:52:05.770 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:05 smithi037 ceph-mon[32192]: fsmap 7 up:standby 2024-06-10T00:52:05.770 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:05 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-10T00:52:05.770 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:05 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1400612911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:52:05.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:05 smithi094 ceph-mon[36708]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:05.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:05 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:05.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:05 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:05.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:05 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:05.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:05 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:52:05.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:05 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:52:05.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:05 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:52:05.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:05 smithi094 ceph-mon[36708]: Deploying daemon mds.h on smithi094 2024-06-10T00:52:05.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:05 smithi094 ceph-mon[36708]: mds.? [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] up:boot 2024-06-10T00:52:05.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:05 smithi094 ceph-mon[36708]: fsmap 7 up:standby 2024-06-10T00:52:05.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:05 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-10T00:52:05.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:05 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1400612911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:52:06.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:05 smithi167 ceph-mon[36917]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:06.091 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:05 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:06.091 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:05 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:06.091 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:05 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:06.091 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:05 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:52:06.091 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:05 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:52:06.091 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:05 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:52:06.092 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:05 smithi167 ceph-mon[36917]: Deploying daemon mds.h on smithi094 2024-06-10T00:52:06.092 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:05 smithi167 ceph-mon[36917]: mds.? [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] up:boot 2024-06-10T00:52:06.092 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:05 smithi167 ceph-mon[36917]: fsmap 7 up:standby 2024-06-10T00:52:06.092 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:05 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-10T00:52:06.092 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:05 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1400612911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:52:07.094 INFO:journalctl@ceph.mds.h.smithi094.stdout:Jun 10 00:52:06 smithi094 systemd[1]: Starting Ceph mds.h for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T00:52:07.364 INFO:journalctl@ceph.mds.h.smithi094.stdout:Jun 10 00:52:07 smithi094 podman[66943]: 2024-06-10 00:52:06.997166288 +0000 UTC m=+0.016907128 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:52:07.365 INFO:journalctl@ceph.mds.h.smithi094.stdout:Jun 10 00:52:07 smithi094 podman[66943]: 2024-06-10 00:52:07.133393212 +0000 UTC m=+0.153134030 container create d3cac041c120c76eaf9883d1718c0b12e643bf82faaa4931a2d0ebd0f2acad0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-06-10T00:52:07.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:07 smithi094 ceph-mon[36708]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:07.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:07 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:07.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:07 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:07.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:07 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:07.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:07 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:52:07.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:07 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:52:07.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:07 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:52:07.865 INFO:journalctl@ceph.mds.h.smithi094.stdout:Jun 10 00:52:07 smithi094 podman[66943]: 2024-06-10 00:52:07.386506731 +0000 UTC m=+0.406247550 container init d3cac041c120c76eaf9883d1718c0b12e643bf82faaa4931a2d0ebd0f2acad0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS) 2024-06-10T00:52:07.865 INFO:journalctl@ceph.mds.h.smithi094.stdout:Jun 10 00:52:07 smithi094 podman[66943]: 2024-06-10 00:52:07.389868976 +0000 UTC m=+0.409609797 container start d3cac041c120c76eaf9883d1718c0b12e643bf82faaa4931a2d0ebd0f2acad0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-06-10T00:52:07.865 INFO:journalctl@ceph.mds.h.smithi094.stdout:Jun 10 00:52:07 smithi094 ceph-mds[66963]: -- 172.21.15.94:0/1030798356 <== mon.2 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a3f9043a40 con 0x55a3f9016c00 2024-06-10T00:52:07.865 INFO:journalctl@ceph.mds.h.smithi094.stdout:Jun 10 00:52:07 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-h[66959]: starting mds.h at 2024-06-10T00:52:07.866 INFO:journalctl@ceph.mds.h.smithi094.stdout:Jun 10 00:52:07 smithi094 bash[66943]: d3cac041c120c76eaf9883d1718c0b12e643bf82faaa4931a2d0ebd0f2acad0c 2024-06-10T00:52:07.866 INFO:journalctl@ceph.mds.h.smithi094.stdout:Jun 10 00:52:07 smithi094 systemd[1]: Started Ceph mds.h for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T00:52:07.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:07 smithi037 ceph-mon[32192]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:07.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:07 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:07.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:07 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:07.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:07 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:07.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:07 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:52:07.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:07 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:52:07.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:07 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:52:08.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:07 smithi167 ceph-mon[36917]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:08.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:07 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:08.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:07 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:08.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:07 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:08.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:07 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:52:08.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:07 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:52:08.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:07 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:52:08.157 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:52:08.421 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:52:08.798 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:08 smithi037 ceph-mon[32192]: Deploying daemon mds.i on smithi167 2024-06-10T00:52:08.798 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:08 smithi037 ceph-mon[32192]: mds.? [v2:172.21.15.94:6836/3801637791,v1:172.21.15.94:6837/3801637791] up:boot 2024-06-10T00:52:08.798 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:08 smithi037 ceph-mon[32192]: fsmap 8 up:standby 2024-06-10T00:52:08.798 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:08 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-10T00:52:08.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:08 smithi167 ceph-mon[36917]: Deploying daemon mds.i on smithi167 2024-06-10T00:52:08.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:08 smithi167 ceph-mon[36917]: mds.? [v2:172.21.15.94:6836/3801637791,v1:172.21.15.94:6837/3801637791] up:boot 2024-06-10T00:52:08.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:08 smithi167 ceph-mon[36917]: fsmap 8 up:standby 2024-06-10T00:52:08.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:08 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-10T00:52:09.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:08 smithi094 ceph-mon[36708]: Deploying daemon mds.i on smithi167 2024-06-10T00:52:09.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:08 smithi094 ceph-mon[36708]: mds.? [v2:172.21.15.94:6836/3801637791,v1:172.21.15.94:6837/3801637791] up:boot 2024-06-10T00:52:09.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:08 smithi094 ceph-mon[36708]: fsmap 8 up:standby 2024-06-10T00:52:09.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:08 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-10T00:52:09.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:09 smithi167 ceph-mon[36917]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:09.839 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jun 10 00:52:09 smithi167 systemd[1]: Starting Ceph mds.i for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T00:52:09.860 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:09 smithi037 ceph-mon[32192]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:10.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.080+0000 7fd3d995e700 1 -- 172.21.15.37:0/3728522216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3d4102f10 msgr2=0x7fd3d410f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:10.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.080+0000 7fd3d995e700 1 --2- 172.21.15.37:0/3728522216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3d4102f10 0x7fd3d410f2f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc00b0d0 tx=0x7fd3cc00b3e0 comp rx=0 tx=0).stop 2024-06-10T00:52:10.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.080+0000 7fd3d995e700 1 -- 172.21.15.37:0/3728522216 shutdown_connections 2024-06-10T00:52:10.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.080+0000 7fd3d995e700 1 --2- 172.21.15.37:0/3728522216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3d4102f10 0x7fd3d410f2f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:10.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.080+0000 7fd3d995e700 1 --2- 172.21.15.37:0/3728522216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3d41073a0 0x7fd3d4107780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:10.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.080+0000 7fd3d995e700 1 --2- 172.21.15.37:0/3728522216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3d4102550 0x7fd3d41029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:10.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.080+0000 7fd3d995e700 1 -- 172.21.15.37:0/3728522216 >> 172.21.15.37:0/3728522216 conn(0x7fd3d4079650 msgr2=0x7fd3d4079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:10.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.080+0000 7fd3d995e700 1 -- 172.21.15.37:0/3728522216 shutdown_connections 2024-06-10T00:52:10.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.080+0000 7fd3d995e700 1 -- 172.21.15.37:0/3728522216 wait complete. 2024-06-10T00:52:10.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.081+0000 7fd3d995e700 1 Processor -- start 2024-06-10T00:52:10.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.081+0000 7fd3d995e700 1 -- start start 2024-06-10T00:52:10.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.081+0000 7fd3d995e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3d4102550 0x7fd3d419e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:10.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.081+0000 7fd3d995e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3d4102f10 0x7fd3d419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:10.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.081+0000 7fd3d995e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3d41073a0 0x7fd3d4198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:10.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.081+0000 7fd3d995e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d41127b0 con 0x7fd3d4102550 2024-06-10T00:52:10.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.081+0000 7fd3d995e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d4112630 con 0x7fd3d4102f10 2024-06-10T00:52:10.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.081+0000 7fd3d995e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d4112930 con 0x7fd3d41073a0 2024-06-10T00:52:10.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.081+0000 7fd3d2ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3d4102550 0x7fd3d419e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:10.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.081+0000 7fd3d2ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3d4102550 0x7fd3d419e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44612/0 (socket says 172.21.15.37:44612) 2024-06-10T00:52:10.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.081+0000 7fd3d2ffd700 1 -- 172.21.15.37:0/4290542281 learned_addr learned my addr 172.21.15.37:0/4290542281 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:52:10.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.081+0000 7fd3d37fe700 1 --2- 172.21.15.37:0/4290542281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3d41073a0 0x7fd3d4198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:10.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.081+0000 7fd3d27fc700 1 --2- 172.21.15.37:0/4290542281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3d4102f10 0x7fd3d419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:10.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.082+0000 7fd3d2ffd700 1 -- 172.21.15.37:0/4290542281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3d41073a0 msgr2=0x7fd3d4198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:10.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.082+0000 7fd3d2ffd700 1 --2- 172.21.15.37:0/4290542281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3d41073a0 0x7fd3d4198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:10.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.082+0000 7fd3d2ffd700 1 -- 172.21.15.37:0/4290542281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3d4102f10 msgr2=0x7fd3d419ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:10.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.082+0000 7fd3d2ffd700 1 --2- 172.21.15.37:0/4290542281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3d4102f10 0x7fd3d419ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:10.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.082+0000 7fd3d2ffd700 1 -- 172.21.15.37:0/4290542281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d4199230 con 0x7fd3d4102550 2024-06-10T00:52:10.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.082+0000 7fd3d27fc700 1 --2- 172.21.15.37:0/4290542281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3d4102f10 0x7fd3d419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:52:10.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.082+0000 7fd3d2ffd700 1 --2- 172.21.15.37:0/4290542281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3d4102550 0x7fd3d419e8c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400d950 tx=0x7fd3c400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:10.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.082+0000 7fd3d895c700 1 -- 172.21.15.37:0/4290542281 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4016070 con 0x7fd3d4102550 2024-06-10T00:52:10.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.082+0000 7fd3d995e700 1 -- 172.21.15.37:0/4290542281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d4199520 con 0x7fd3d4102550 2024-06-10T00:52:10.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.082+0000 7fd3d895c700 1 -- 172.21.15.37:0/4290542281 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3c4009e20 con 0x7fd3d4102550 2024-06-10T00:52:10.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.082+0000 7fd3d895c700 1 -- 172.21.15.37:0/4290542281 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c400f620 con 0x7fd3d4102550 2024-06-10T00:52:10.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.082+0000 7fd3d995e700 1 -- 172.21.15.37:0/4290542281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d41a5790 con 0x7fd3d4102550 2024-06-10T00:52:10.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.083+0000 7fd3d995e700 1 -- 172.21.15.37:0/4290542281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3d4103b80 con 0x7fd3d4102550 2024-06-10T00:52:10.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.084+0000 7fd3d895c700 1 -- 172.21.15.37:0/4290542281 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd3c4019070 con 0x7fd3d4102550 2024-06-10T00:52:10.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.085+0000 7fd3d895c700 1 --2- 172.21.15.37:0/4290542281 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd3bc0a2c20 0x7fd3bc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:10.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.085+0000 7fd3d27fc700 1 --2- 172.21.15.37:0/4290542281 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd3bc0a2c20 0x7fd3bc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:10.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.085+0000 7fd3d895c700 1 -- 172.21.15.37:0/4290542281 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd3c40ce770 con 0x7fd3d4102550 2024-06-10T00:52:10.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.085+0000 7fd3d27fc700 1 --2- 172.21.15.37:0/4290542281 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd3bc0a2c20 0x7fd3bc0a50e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4079c70 tx=0x7fd3c800b5f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:10.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.089+0000 7fd3d895c700 1 -- 172.21.15.37:0/4290542281 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd3c409bad0 con 0x7fd3d4102550 2024-06-10T00:52:10.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:09 smithi094 ceph-mon[36708]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:10.275 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jun 10 00:52:09 smithi167 podman[67973]: 2024-06-10 00:52:09.881629281 +0000 UTC m=+0.017466871 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:52:10.275 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jun 10 00:52:10 smithi167 podman[67973]: 2024-06-10 00:52:10.018216282 +0000 UTC m=+0.154053851 container create 016d4410edc1fdb0573a1a1e7e6369c152269f74d2b1ac9a301a8728a02ed237 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-i, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T00:52:10.275 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jun 10 00:52:10 smithi167 podman[67973]: 2024-06-10 00:52:10.271404809 +0000 UTC m=+0.407242380 container init 016d4410edc1fdb0573a1a1e7e6369c152269f74d2b1ac9a301a8728a02ed237 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-06-10T00:52:10.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.341+0000 7fd3d995e700 1 -- 172.21.15.37:0/4290542281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3d402dca0 con 0x7fd3d4102550 2024-06-10T00:52:10.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.341+0000 7fd3d895c700 1 -- 172.21.15.37:0/4290542281 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7174 (secure 0 0 0) 0x7fd3c409b3d0 con 0x7fd3d4102550 2024-06-10T00:52:10.343 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:52:10.343 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24431,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-06-10T00:52:10.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.343+0000 7fd3d995e700 1 -- 172.21.15.37:0/4290542281 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd3bc0a2c20 msgr2=0x7fd3bc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:10.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.343+0000 7fd3d995e700 1 --2- 172.21.15.37:0/4290542281 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd3bc0a2c20 0x7fd3bc0a50e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4079c70 tx=0x7fd3c800b5f0 comp rx=0 tx=0).stop 2024-06-10T00:52:10.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.343+0000 7fd3d995e700 1 -- 172.21.15.37:0/4290542281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3d4102550 msgr2=0x7fd3d419e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:10.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.343+0000 7fd3d995e700 1 --2- 172.21.15.37:0/4290542281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3d4102550 0x7fd3d419e8c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400d950 tx=0x7fd3c400dc60 comp rx=0 tx=0).stop 2024-06-10T00:52:10.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.344+0000 7fd3d995e700 1 -- 172.21.15.37:0/4290542281 shutdown_connections 2024-06-10T00:52:10.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.344+0000 7fd3d995e700 1 --2- 172.21.15.37:0/4290542281 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd3bc0a2c20 0x7fd3bc0a50e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:10.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.344+0000 7fd3d995e700 1 --2- 172.21.15.37:0/4290542281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3d41073a0 0x7fd3d4198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:10.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.344+0000 7fd3d995e700 1 --2- 172.21.15.37:0/4290542281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3d4102550 0x7fd3d419e8c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:10.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.344+0000 7fd3d995e700 1 --2- 172.21.15.37:0/4290542281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3d4102f10 0x7fd3d419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:10.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.344+0000 7fd3d995e700 1 -- 172.21.15.37:0/4290542281 >> 172.21.15.37:0/4290542281 conn(0x7fd3d4079650 msgr2=0x7fd3d4100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:10.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.344+0000 7fd3d995e700 1 -- 172.21.15.37:0/4290542281 shutdown_connections 2024-06-10T00:52:10.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:10.344+0000 7fd3d995e700 1 -- 172.21.15.37:0/4290542281 wait complete. 2024-06-10T00:52:10.347 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 9 2024-06-10T00:52:10.589 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jun 10 00:52:10 smithi167 podman[67973]: 2024-06-10 00:52:10.274327337 +0000 UTC m=+0.410164923 container start 016d4410edc1fdb0573a1a1e7e6369c152269f74d2b1ac9a301a8728a02ed237 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-i, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-06-10T00:52:10.589 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jun 10 00:52:10 smithi167 ceph-mds[67992]: -- 172.21.15.167:0/1547071483 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5570293e7a40 con 0x5570293bac00 2024-06-10T00:52:10.589 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jun 10 00:52:10 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-i[67988]: starting mds.i at 2024-06-10T00:52:10.589 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jun 10 00:52:10 smithi167 bash[67973]: 016d4410edc1fdb0573a1a1e7e6369c152269f74d2b1ac9a301a8728a02ed237 2024-06-10T00:52:10.589 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jun 10 00:52:10 smithi167 systemd[1]: Started Ceph mds.i for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T00:52:10.960 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:10 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/4290542281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:52:10.961 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:10 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:10.961 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:10 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:10.961 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:10 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:10.961 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:10 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:52:10.961 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:10 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:52:10.961 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:10 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:52:11.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:10 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/4290542281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:52:11.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:10 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:11.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:10 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:11.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:10 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:11.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:10 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:52:11.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:10 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:52:11.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:10 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:52:11.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:10 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/4290542281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:52:11.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:10 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:11.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:10 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:11.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:10 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:11.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:10 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:52:11.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:10 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:52:11.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:10 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:52:11.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:11 smithi167 ceph-mon[36917]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:11.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:11 smithi167 ceph-mon[36917]: Deploying daemon mds.g on smithi037 2024-06-10T00:52:11.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:11 smithi167 ceph-mon[36917]: mds.? [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] up:boot 2024-06-10T00:52:11.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:11 smithi167 ceph-mon[36917]: fsmap 9 up:standby 2024-06-10T00:52:11.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:11 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-10T00:52:11.902 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:11 smithi037 ceph-mon[32192]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:11.902 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:11 smithi037 ceph-mon[32192]: Deploying daemon mds.g on smithi037 2024-06-10T00:52:11.902 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:11 smithi037 ceph-mon[32192]: mds.? [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] up:boot 2024-06-10T00:52:11.902 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:11 smithi037 ceph-mon[32192]: fsmap 9 up:standby 2024-06-10T00:52:11.902 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:11 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-10T00:52:12.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:11 smithi094 ceph-mon[36708]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:12.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:11 smithi094 ceph-mon[36708]: Deploying daemon mds.g on smithi037 2024-06-10T00:52:12.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:11 smithi094 ceph-mon[36708]: mds.? [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] up:boot 2024-06-10T00:52:12.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:11 smithi094 ceph-mon[36708]: fsmap 9 up:standby 2024-06-10T00:52:12.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:11 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-10T00:52:12.896 INFO:journalctl@ceph.mds.g.smithi037.stdout:Jun 10 00:52:12 smithi037 systemd[1]: Starting Ceph mds.g for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T00:52:12.992 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:52:13.175 INFO:journalctl@ceph.mds.g.smithi037.stdout:Jun 10 00:52:12 smithi037 podman[71816]: 2024-06-10 00:52:12.809697018 +0000 UTC m=+0.018193032 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:52:13.175 INFO:journalctl@ceph.mds.g.smithi037.stdout:Jun 10 00:52:12 smithi037 podman[71816]: 2024-06-10 00:52:12.939103681 +0000 UTC m=+0.147599666 container create fd0e2ed2a2e400d6f379c5c380b1f5056746853c05057f41b222af481410949c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-06-10T00:52:13.175 INFO:journalctl@ceph.mds.g.smithi037.stdout:Jun 10 00:52:13 smithi037 podman[71816]: 2024-06-10 00:52:13.175379154 +0000 UTC m=+0.383875150 container init fd0e2ed2a2e400d6f379c5c380b1f5056746853c05057f41b222af481410949c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-g, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-06-10T00:52:13.355 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:52:13.478 INFO:journalctl@ceph.mds.g.smithi037.stdout:Jun 10 00:52:13 smithi037 podman[71816]: 2024-06-10 00:52:13.17897835 +0000 UTC m=+0.387474350 container start fd0e2ed2a2e400d6f379c5c380b1f5056746853c05057f41b222af481410949c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-g, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T00:52:13.479 INFO:journalctl@ceph.mds.g.smithi037.stdout:Jun 10 00:52:13 smithi037 ceph-mds[71865]: -- 172.21.15.37:0/4128067071 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562c20319a40 con 0x562c202ec800 2024-06-10T00:52:13.479 INFO:journalctl@ceph.mds.g.smithi037.stdout:Jun 10 00:52:13 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-g[71848]: starting mds.g at 2024-06-10T00:52:13.479 INFO:journalctl@ceph.mds.g.smithi037.stdout:Jun 10 00:52:13 smithi037 bash[71816]: fd0e2ed2a2e400d6f379c5c380b1f5056746853c05057f41b222af481410949c 2024-06-10T00:52:13.479 INFO:journalctl@ceph.mds.g.smithi037.stdout:Jun 10 00:52:13 smithi037 systemd[1]: Started Ceph mds.g for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T00:52:13.805 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:13 smithi037 ceph-mon[32192]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:13.805 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:13 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:13.805 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:13 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:13.805 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:13 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:13.805 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:13 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:52:13.805 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:13 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:52:13.805 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:13 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:52:13.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:13 smithi094 ceph-mon[36708]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:13.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:13 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:13.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:13 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:13.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:13 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:13.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:13 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:52:13.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:13 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:52:13.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:13 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:52:14.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:13 smithi167 ceph-mon[36917]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:14.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:13 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:14.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:13 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:14.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:13 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:14.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:13 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:52:14.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:13 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:52:14.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:13 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:52:14.706 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:14 smithi037 ceph-mon[32192]: Deploying daemon mds.a on smithi037 2024-06-10T00:52:14.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:14 smithi037 ceph-mon[32192]: mds.? [v2:172.21.15.37:6838/595511083,v1:172.21.15.37:6839/595511083] up:boot 2024-06-10T00:52:14.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:14 smithi037 ceph-mon[32192]: fsmap 10 up:standby 2024-06-10T00:52:14.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:14 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-10T00:52:15.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:14 smithi167 ceph-mon[36917]: Deploying daemon mds.a on smithi037 2024-06-10T00:52:15.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:14 smithi167 ceph-mon[36917]: mds.? [v2:172.21.15.37:6838/595511083,v1:172.21.15.37:6839/595511083] up:boot 2024-06-10T00:52:15.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:14 smithi167 ceph-mon[36917]: fsmap 10 up:standby 2024-06-10T00:52:15.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:14 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-10T00:52:15.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:14 smithi094 ceph-mon[36708]: Deploying daemon mds.a on smithi037 2024-06-10T00:52:15.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:14 smithi094 ceph-mon[36708]: mds.? [v2:172.21.15.37:6838/595511083,v1:172.21.15.37:6839/595511083] up:boot 2024-06-10T00:52:15.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:14 smithi094 ceph-mon[36708]: fsmap 10 up:standby 2024-06-10T00:52:15.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:14 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-10T00:52:15.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.187+0000 7f7b2a047700 1 -- 172.21.15.37:0/3170008159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b241017f0 msgr2=0x7f7b2410f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:15.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.187+0000 7f7b2a047700 1 --2- 172.21.15.37:0/3170008159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b241017f0 0x7f7b2410f380 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c0099c0 tx=0x7f7b1c009cd0 comp rx=0 tx=0).stop 2024-06-10T00:52:15.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.187+0000 7f7b2a047700 1 -- 172.21.15.37:0/3170008159 shutdown_connections 2024-06-10T00:52:15.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.187+0000 7f7b2a047700 1 --2- 172.21.15.37:0/3170008159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b241017f0 0x7f7b2410f380 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:15.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.187+0000 7f7b2a047700 1 --2- 172.21.15.37:0/3170008159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b24107500 0x7f7b241078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:15.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.187+0000 7f7b2a047700 1 --2- 172.21.15.37:0/3170008159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b24100e30 0x7f7b241012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:15.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.187+0000 7f7b2a047700 1 -- 172.21.15.37:0/3170008159 >> 172.21.15.37:0/3170008159 conn(0x7f7b24076730 msgr2=0x7f7b24076b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:15.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.187+0000 7f7b2a047700 1 -- 172.21.15.37:0/3170008159 shutdown_connections 2024-06-10T00:52:15.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.187+0000 7f7b2a047700 1 -- 172.21.15.37:0/3170008159 wait complete. 2024-06-10T00:52:15.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.188+0000 7f7b2a047700 1 Processor -- start 2024-06-10T00:52:15.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.188+0000 7f7b2a047700 1 -- start start 2024-06-10T00:52:15.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.189+0000 7f7b2a047700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b24100e30 0x7f7b2419e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:15.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.189+0000 7f7b2a047700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b241017f0 0x7f7b2419ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:15.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.189+0000 7f7b2a047700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b24107500 0x7f7b24198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:15.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.189+0000 7f7b2a047700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7b241127f0 con 0x7f7b24100e30 2024-06-10T00:52:15.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.189+0000 7f7b2a047700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7b24112670 con 0x7f7b24107500 2024-06-10T00:52:15.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.189+0000 7f7b2a047700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7b24112970 con 0x7f7b241017f0 2024-06-10T00:52:15.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.189+0000 7f7b237fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b24100e30 0x7f7b2419e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:15.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.189+0000 7f7b237fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b24100e30 0x7f7b2419e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43264/0 (socket says 172.21.15.37:43264) 2024-06-10T00:52:15.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.189+0000 7f7b237fe700 1 -- 172.21.15.37:0/2309110047 learned_addr learned my addr 172.21.15.37:0/2309110047 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:52:15.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.189+0000 7f7b22ffd700 1 --2- 172.21.15.37:0/2309110047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b241017f0 0x7f7b2419ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:15.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.189+0000 7f7b23fff700 1 --2- 172.21.15.37:0/2309110047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b24107500 0x7f7b24198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:15.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.189+0000 7f7b237fe700 1 -- 172.21.15.37:0/2309110047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b241017f0 msgr2=0x7f7b2419ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:15.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.189+0000 7f7b237fe700 1 --2- 172.21.15.37:0/2309110047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b241017f0 0x7f7b2419ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:15.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.189+0000 7f7b237fe700 1 -- 172.21.15.37:0/2309110047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b24107500 msgr2=0x7f7b24198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:15.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.189+0000 7f7b237fe700 1 --2- 172.21.15.37:0/2309110047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b24107500 0x7f7b24198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:15.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.189+0000 7f7b237fe700 1 -- 172.21.15.37:0/2309110047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b24199070 con 0x7f7b24100e30 2024-06-10T00:52:15.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.189+0000 7f7b23fff700 1 --2- 172.21.15.37:0/2309110047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b24107500 0x7f7b24198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:52:15.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.189+0000 7f7b237fe700 1 --2- 172.21.15.37:0/2309110047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b24100e30 0x7f7b2419e750 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7b14007e20 tx=0x7f7b1400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:15.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.190+0000 7f7b20ff9700 1 -- 172.21.15.37:0/2309110047 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b14016020 con 0x7f7b24100e30 2024-06-10T00:52:15.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.190+0000 7f7b20ff9700 1 -- 172.21.15.37:0/2309110047 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b1400ad40 con 0x7f7b24100e30 2024-06-10T00:52:15.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.190+0000 7f7b2a047700 1 -- 172.21.15.37:0/2309110047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b24199360 con 0x7f7b24100e30 2024-06-10T00:52:15.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.190+0000 7f7b20ff9700 1 -- 172.21.15.37:0/2309110047 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b14011700 con 0x7f7b24100e30 2024-06-10T00:52:15.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.190+0000 7f7b2a047700 1 -- 172.21.15.37:0/2309110047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b241a5620 con 0x7f7b24100e30 2024-06-10T00:52:15.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.191+0000 7f7b2a047700 1 -- 172.21.15.37:0/2309110047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b24069180 con 0x7f7b24100e30 2024-06-10T00:52:15.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.192+0000 7f7b20ff9700 1 -- 172.21.15.37:0/2309110047 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7b14019020 con 0x7f7b24100e30 2024-06-10T00:52:15.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.192+0000 7f7b20ff9700 1 --2- 172.21.15.37:0/2309110047 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7b100a2c20 0x7f7b100a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:15.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.193+0000 7f7b22ffd700 1 --2- 172.21.15.37:0/2309110047 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7b100a2c20 0x7f7b100a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:15.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.193+0000 7f7b20ff9700 1 -- 172.21.15.37:0/2309110047 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f7b140cecf0 con 0x7f7b24100e30 2024-06-10T00:52:15.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.193+0000 7f7b22ffd700 1 --2- 172.21.15.37:0/2309110047 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7b100a2c20 0x7f7b100a50e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7b24076b50 tx=0x7f7b0c00a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:15.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.196+0000 7f7b20ff9700 1 -- 172.21.15.37:0/2309110047 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7b140182c0 con 0x7f7b24100e30 2024-06-10T00:52:15.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.451+0000 7f7b2a047700 1 -- 172.21.15.37:0/2309110047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b24199e40 con 0x7f7b24100e30 2024-06-10T00:52:15.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.452+0000 7f7b20ff9700 1 -- 172.21.15.37:0/2309110047 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8827 (secure 0 0 0) 0x7f7b1409c050 con 0x7f7b24100e30 2024-06-10T00:52:15.452 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:52:15.452 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24431,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-10T00:52:15.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.454+0000 7f7b2a047700 1 -- 172.21.15.37:0/2309110047 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7b100a2c20 msgr2=0x7f7b100a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:15.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.454+0000 7f7b2a047700 1 --2- 172.21.15.37:0/2309110047 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7b100a2c20 0x7f7b100a50e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7b24076b50 tx=0x7f7b0c00a380 comp rx=0 tx=0).stop 2024-06-10T00:52:15.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.454+0000 7f7b2a047700 1 -- 172.21.15.37:0/2309110047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b24100e30 msgr2=0x7f7b2419e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:15.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.454+0000 7f7b2a047700 1 --2- 172.21.15.37:0/2309110047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b24100e30 0x7f7b2419e750 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7b14007e20 tx=0x7f7b1400a340 comp rx=0 tx=0).stop 2024-06-10T00:52:15.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.454+0000 7f7b2a047700 1 -- 172.21.15.37:0/2309110047 shutdown_connections 2024-06-10T00:52:15.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.454+0000 7f7b2a047700 1 --2- 172.21.15.37:0/2309110047 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7b100a2c20 0x7f7b100a50e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:15.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.454+0000 7f7b2a047700 1 --2- 172.21.15.37:0/2309110047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b24107500 0x7f7b24198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:15.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.454+0000 7f7b2a047700 1 --2- 172.21.15.37:0/2309110047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b24100e30 0x7f7b2419e750 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:15.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.454+0000 7f7b2a047700 1 --2- 172.21.15.37:0/2309110047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b241017f0 0x7f7b2419ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:15.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.454+0000 7f7b2a047700 1 -- 172.21.15.37:0/2309110047 >> 172.21.15.37:0/2309110047 conn(0x7f7b24076730 msgr2=0x7f7b240fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:15.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.454+0000 7f7b2a047700 1 -- 172.21.15.37:0/2309110047 shutdown_connections 2024-06-10T00:52:15.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:15.454+0000 7f7b2a047700 1 -- 172.21.15.37:0/2309110047 wait complete. 2024-06-10T00:52:15.456 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 11 2024-06-10T00:52:15.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:15 smithi037 ceph-mon[32192]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:15.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:15 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2309110047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:52:16.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:15 smithi167 ceph-mon[36917]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:16.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:15 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2309110047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:52:16.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:15 smithi094 ceph-mon[36708]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:16.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:15 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2309110047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:52:17.389 INFO:journalctl@ceph.mds.a.smithi037.stdout:Jun 10 00:52:17 smithi037 systemd[1]: Starting Ceph mds.a for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T00:52:17.646 INFO:journalctl@ceph.mds.a.smithi037.stdout:Jun 10 00:52:17 smithi037 podman[72276]: 2024-06-10 00:52:17.291625299 +0000 UTC m=+0.018252286 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:52:17.646 INFO:journalctl@ceph.mds.a.smithi037.stdout:Jun 10 00:52:17 smithi037 podman[72276]: 2024-06-10 00:52:17.403774662 +0000 UTC m=+0.130401651 container create e6d28d7dca9d73c0ee0ab1821b13faae41b78c69d81c98ef267083b85944e69d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-a, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-10T00:52:18.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:17 smithi167 ceph-mon[36917]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:18.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:17 smithi094 ceph-mon[36708]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:18.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:17 smithi037 ceph-mon[32192]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:18.146 INFO:journalctl@ceph.mds.a.smithi037.stdout:Jun 10 00:52:17 smithi037 podman[72276]: 2024-06-10 00:52:17.665110905 +0000 UTC m=+0.391737873 container init e6d28d7dca9d73c0ee0ab1821b13faae41b78c69d81c98ef267083b85944e69d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-10T00:52:18.146 INFO:journalctl@ceph.mds.a.smithi037.stdout:Jun 10 00:52:17 smithi037 podman[72276]: 2024-06-10 00:52:17.668104372 +0000 UTC m=+0.394731340 container start e6d28d7dca9d73c0ee0ab1821b13faae41b78c69d81c98ef267083b85944e69d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-a, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-06-10T00:52:18.146 INFO:journalctl@ceph.mds.a.smithi037.stdout:Jun 10 00:52:17 smithi037 ceph-mds[72296]: -- 172.21.15.37:0/2454792579 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5603ac49fa40 con 0x5603ac472c00 2024-06-10T00:52:18.146 INFO:journalctl@ceph.mds.a.smithi037.stdout:Jun 10 00:52:17 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-a[72292]: starting mds.a at 2024-06-10T00:52:18.146 INFO:journalctl@ceph.mds.a.smithi037.stdout:Jun 10 00:52:17 smithi037 bash[72276]: e6d28d7dca9d73c0ee0ab1821b13faae41b78c69d81c98ef267083b85944e69d 2024-06-10T00:52:18.147 INFO:journalctl@ceph.mds.a.smithi037.stdout:Jun 10 00:52:17 smithi037 systemd[1]: Started Ceph mds.a for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T00:52:18.268 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:52:18.529 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:52:18.911 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:18 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:18.911 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:18 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:18.911 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:18 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:18.911 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:18 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:52:18.911 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:18 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:52:18.911 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:18 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:52:18.911 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:18 smithi037 ceph-mon[32192]: Deploying daemon mds.b on smithi094 2024-06-10T00:52:18.911 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:18 smithi037 ceph-mon[32192]: mds.? [v2:172.21.15.37:6840/2289348383,v1:172.21.15.37:6841/2289348383] up:boot 2024-06-10T00:52:18.911 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:18 smithi037 ceph-mon[32192]: fsmap 11 up:standby 2024-06-10T00:52:18.911 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:18 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-10T00:52:19.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:18 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:19.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:18 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:19.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:18 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:19.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:18 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:52:19.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:18 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:52:19.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:18 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:52:19.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:18 smithi167 ceph-mon[36917]: Deploying daemon mds.b on smithi094 2024-06-10T00:52:19.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:18 smithi167 ceph-mon[36917]: mds.? [v2:172.21.15.37:6840/2289348383,v1:172.21.15.37:6841/2289348383] up:boot 2024-06-10T00:52:19.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:18 smithi167 ceph-mon[36917]: fsmap 11 up:standby 2024-06-10T00:52:19.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:18 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-10T00:52:19.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:18 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:19.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:18 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:19.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:18 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:19.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:18 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-10T00:52:19.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:18 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-10T00:52:19.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:18 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:52:19.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:18 smithi094 ceph-mon[36708]: Deploying daemon mds.b on smithi094 2024-06-10T00:52:19.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:18 smithi094 ceph-mon[36708]: mds.? [v2:172.21.15.37:6840/2289348383,v1:172.21.15.37:6841/2289348383] up:boot 2024-06-10T00:52:19.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:18 smithi094 ceph-mon[36708]: fsmap 11 up:standby 2024-06-10T00:52:19.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:18 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-10T00:52:20.013 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:19 smithi037 ceph-mon[32192]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:20.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:19 smithi167 ceph-mon[36917]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:20.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:19 smithi094 ceph-mon[36708]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:20.114 INFO:journalctl@ceph.mds.b.smithi094.stdout:Jun 10 00:52:20 smithi094 systemd[1]: Starting Ceph mds.b for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T00:52:20.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.267+0000 7f2c556de700 1 -- 172.21.15.37:0/3410507090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c501030f0 msgr2=0x7f2c50107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:20.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.267+0000 7f2c556de700 1 --2- 172.21.15.37:0/3410507090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c501030f0 0x7f2c50107110 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2c440099c0 tx=0x7f2c44009cd0 comp rx=0 tx=0).stop 2024-06-10T00:52:20.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.268+0000 7f2c556de700 1 -- 172.21.15.37:0/3410507090 shutdown_connections 2024-06-10T00:52:20.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.268+0000 7f2c556de700 1 --2- 172.21.15.37:0/3410507090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c50107840 0x7f2c5010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:20.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.268+0000 7f2c556de700 1 --2- 172.21.15.37:0/3410507090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c50102740 0x7f2c50102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:20.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.268+0000 7f2c556de700 1 --2- 172.21.15.37:0/3410507090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c501030f0 0x7f2c50107110 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:20.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.268+0000 7f2c556de700 1 -- 172.21.15.37:0/3410507090 >> 172.21.15.37:0/3410507090 conn(0x7f2c500795f0 msgr2=0x7f2c50079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:20.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.268+0000 7f2c556de700 1 -- 172.21.15.37:0/3410507090 shutdown_connections 2024-06-10T00:52:20.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.268+0000 7f2c556de700 1 -- 172.21.15.37:0/3410507090 wait complete. 2024-06-10T00:52:20.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.269+0000 7f2c556de700 1 Processor -- start 2024-06-10T00:52:20.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.269+0000 7f2c556de700 1 -- start start 2024-06-10T00:52:20.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.269+0000 7f2c556de700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c50102740 0x7f2c5019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:20.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.269+0000 7f2c556de700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c501030f0 0x7f2c5019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:20.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.269+0000 7f2c556de700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c50107840 0x7f2c50198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:20.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.269+0000 7f2c556de700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2c50112750 con 0x7f2c50102740 2024-06-10T00:52:20.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.269+0000 7f2c556de700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2c501125d0 con 0x7f2c501030f0 2024-06-10T00:52:20.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.269+0000 7f2c556de700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2c501128d0 con 0x7f2c50107840 2024-06-10T00:52:20.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.269+0000 7f2c4effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c50102740 0x7f2c5019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:20.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.269+0000 7f2c4effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c50102740 0x7f2c5019e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43306/0 (socket says 172.21.15.37:43306) 2024-06-10T00:52:20.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.269+0000 7f2c4effd700 1 -- 172.21.15.37:0/164678800 learned_addr learned my addr 172.21.15.37:0/164678800 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:52:20.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.269+0000 7f2c4e7fc700 1 --2- 172.21.15.37:0/164678800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c501030f0 0x7f2c5019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:20.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.270+0000 7f2c4f7fe700 1 --2- 172.21.15.37:0/164678800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c50107840 0x7f2c50198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:20.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.270+0000 7f2c4effd700 1 -- 172.21.15.37:0/164678800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c50107840 msgr2=0x7f2c50198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:20.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.270+0000 7f2c4effd700 1 --2- 172.21.15.37:0/164678800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c50107840 0x7f2c50198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:20.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.270+0000 7f2c4effd700 1 -- 172.21.15.37:0/164678800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c501030f0 msgr2=0x7f2c5019ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:20.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.270+0000 7f2c4effd700 1 --2- 172.21.15.37:0/164678800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c501030f0 0x7f2c5019ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:20.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.270+0000 7f2c4effd700 1 -- 172.21.15.37:0/164678800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c501990b0 con 0x7f2c50102740 2024-06-10T00:52:20.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.270+0000 7f2c4effd700 1 --2- 172.21.15.37:0/164678800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c50102740 0x7f2c5019e790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000c9b0 tx=0x7f2c4000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:20.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.270+0000 7f2c37fff700 1 -- 172.21.15.37:0/164678800 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c40007b00 con 0x7f2c50102740 2024-06-10T00:52:20.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.270+0000 7f2c556de700 1 -- 172.21.15.37:0/164678800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c501993a0 con 0x7f2c50102740 2024-06-10T00:52:20.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.270+0000 7f2c37fff700 1 -- 172.21.15.37:0/164678800 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c40007ca0 con 0x7f2c50102740 2024-06-10T00:52:20.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.270+0000 7f2c37fff700 1 -- 172.21.15.37:0/164678800 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c40018730 con 0x7f2c50102740 2024-06-10T00:52:20.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.270+0000 7f2c556de700 1 -- 172.21.15.37:0/164678800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c501a5660 con 0x7f2c50102740 2024-06-10T00:52:20.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.271+0000 7f2c556de700 1 -- 172.21.15.37:0/164678800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c50069180 con 0x7f2c50102740 2024-06-10T00:52:20.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.272+0000 7f2c37fff700 1 -- 172.21.15.37:0/164678800 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2c400188d0 con 0x7f2c50102740 2024-06-10T00:52:20.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.273+0000 7f2c37fff700 1 --2- 172.21.15.37:0/164678800 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2c380a2c20 0x7f2c380a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:20.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.273+0000 7f2c4e7fc700 1 --2- 172.21.15.37:0/164678800 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2c380a2c20 0x7f2c380a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:20.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.273+0000 7f2c37fff700 1 -- 172.21.15.37:0/164678800 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2c400cee40 con 0x7f2c50102740 2024-06-10T00:52:20.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.274+0000 7f2c4e7fc700 1 --2- 172.21.15.37:0/164678800 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2c380a2c20 0x7f2c380a50e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2c50108c90 tx=0x7f2c44005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:20.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.277+0000 7f2c37fff700 1 -- 172.21.15.37:0/164678800 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2c40014030 con 0x7f2c50102740 2024-06-10T00:52:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.530+0000 7f2c556de700 1 -- 172.21.15.37:0/164678800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c50199eb0 con 0x7f2c50102740 2024-06-10T00:52:20.531 INFO:journalctl@ceph.mds.b.smithi094.stdout:Jun 10 00:52:20 smithi094 podman[67246]: 2024-06-10 00:52:20.146040968 +0000 UTC m=+0.017222234 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T00:52:20.531 INFO:journalctl@ceph.mds.b.smithi094.stdout:Jun 10 00:52:20 smithi094 podman[67246]: 2024-06-10 00:52:20.26936499 +0000 UTC m=+0.140546299 container create 6afed0933a54dae4adba4d24cbeae32f71fc6f3cb5ee0e604c077bd61815e212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-06-10T00:52:20.531 INFO:journalctl@ceph.mds.b.smithi094.stdout:Jun 10 00:52:20 smithi094 podman[67246]: 2024-06-10 00:52:20.530799933 +0000 UTC m=+0.401981208 container init 6afed0933a54dae4adba4d24cbeae32f71fc6f3cb5ee0e604c077bd61815e212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-b, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0) 2024-06-10T00:52:20.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.531+0000 7f2c37fff700 1 -- 172.21.15.37:0/164678800 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9653 (secure 0 0 0) 0x7f2c50199eb0 con 0x7f2c50102740 2024-06-10T00:52:20.532 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:52:20.532 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24431,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-10T00:52:20.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.533+0000 7f2c556de700 1 -- 172.21.15.37:0/164678800 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2c380a2c20 msgr2=0x7f2c380a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:20.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.533+0000 7f2c556de700 1 --2- 172.21.15.37:0/164678800 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2c380a2c20 0x7f2c380a50e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2c50108c90 tx=0x7f2c44005fb0 comp rx=0 tx=0).stop 2024-06-10T00:52:20.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.533+0000 7f2c556de700 1 -- 172.21.15.37:0/164678800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c50102740 msgr2=0x7f2c5019e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.533+0000 7f2c556de700 1 --2- 172.21.15.37:0/164678800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c50102740 0x7f2c5019e790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000c9b0 tx=0x7f2c4000ccc0 comp rx=0 tx=0).stop 2024-06-10T00:52:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.533+0000 7f2c556de700 1 -- 172.21.15.37:0/164678800 shutdown_connections 2024-06-10T00:52:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.533+0000 7f2c556de700 1 --2- 172.21.15.37:0/164678800 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2c380a2c20 0x7f2c380a50e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.533+0000 7f2c556de700 1 --2- 172.21.15.37:0/164678800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c50107840 0x7f2c50198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.533+0000 7f2c556de700 1 --2- 172.21.15.37:0/164678800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c50102740 0x7f2c5019e790 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.533+0000 7f2c556de700 1 --2- 172.21.15.37:0/164678800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c501030f0 0x7f2c5019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.533+0000 7f2c556de700 1 -- 172.21.15.37:0/164678800 >> 172.21.15.37:0/164678800 conn(0x7f2c500795f0 msgr2=0x7f2c501004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.533+0000 7f2c556de700 1 -- 172.21.15.37:0/164678800 shutdown_connections 2024-06-10T00:52:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:20.533+0000 7f2c556de700 1 -- 172.21.15.37:0/164678800 wait complete. 2024-06-10T00:52:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 12 2024-06-10T00:52:20.832 INFO:journalctl@ceph.mds.b.smithi094.stdout:Jun 10 00:52:20 smithi094 podman[67246]: 2024-06-10 00:52:20.533613087 +0000 UTC m=+0.404794362 container start 6afed0933a54dae4adba4d24cbeae32f71fc6f3cb5ee0e604c077bd61815e212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=) 2024-06-10T00:52:20.832 INFO:journalctl@ceph.mds.b.smithi094.stdout:Jun 10 00:52:20 smithi094 ceph-mds[67265]: -- 172.21.15.94:0/317314952 <== mon.2 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556385481a40 con 0x556385454c00 2024-06-10T00:52:20.832 INFO:journalctl@ceph.mds.b.smithi094.stdout:Jun 10 00:52:20 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-b[67261]: starting mds.b at 2024-06-10T00:52:20.832 INFO:journalctl@ceph.mds.b.smithi094.stdout:Jun 10 00:52:20 smithi094 bash[67246]: 6afed0933a54dae4adba4d24cbeae32f71fc6f3cb5ee0e604c077bd61815e212 2024-06-10T00:52:20.832 INFO:journalctl@ceph.mds.b.smithi094.stdout:Jun 10 00:52:20 smithi094 systemd[1]: Started Ceph mds.b for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T00:52:21.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:20 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/164678800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:52:21.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:20 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:21.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:20 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:21.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:20 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:21.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:20 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:21.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:20 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:21.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:20 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/164678800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:52:21.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:20 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:21.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:20 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:21.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:20 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:21.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:20 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:21.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:20 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:21.140 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:20 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/164678800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:52:21.140 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:20 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:21.140 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:20 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:21.141 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:20 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:21.141 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:20 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:21.141 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:20 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:22.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:21 smithi167 ceph-mon[36917]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:22.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:21 smithi167 ceph-mon[36917]: mds.? [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] up:boot 2024-06-10T00:52:22.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:21 smithi167 ceph-mon[36917]: fsmap 12 up:standby 2024-06-10T00:52:22.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-10T00:52:22.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:52:22.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:21 smithi094 ceph-mon[36708]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:22.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:21 smithi094 ceph-mon[36708]: mds.? [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] up:boot 2024-06-10T00:52:22.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:21 smithi094 ceph-mon[36708]: fsmap 12 up:standby 2024-06-10T00:52:22.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-10T00:52:22.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:52:22.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:21 smithi037 ceph-mon[32192]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:22.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:21 smithi037 ceph-mon[32192]: mds.? [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] up:boot 2024-06-10T00:52:22.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:21 smithi037 ceph-mon[32192]: fsmap 12 up:standby 2024-06-10T00:52:22.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-10T00:52:22.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:52:23.170 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:52:23.436 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:52:23.799 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:23 smithi037 ceph-mon[32192]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:23.799 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:23 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:23.799 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:23 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:23.799 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:23 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:23.799 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:23 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:23.799 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:23 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:23.799 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:23 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:23.799 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:23 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:52:23.799 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:23 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:52:23.799 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:23 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:23.799 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:23 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:52:23.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:23 smithi167 ceph-mon[36917]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:23.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:23 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:23.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:23 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:23.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:23 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:23.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:23 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:23.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:23 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:23.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:23 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:23.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:23 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:52:23.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:23 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:52:23.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:23 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:23.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:23 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:52:23.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:23 smithi094 ceph-mon[36708]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:23.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:23 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:23.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:23 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:23.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:23 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:23.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:23 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:23.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:23 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:23.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:23 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:23.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:23 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:52:23.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:23 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:52:23.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:23 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:23.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:23 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:52:25.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.070+0000 7f334141c700 1 -- 172.21.15.37:0/2146525036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f333c1084f0 msgr2=0x7f333c1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:25.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.070+0000 7f334141c700 1 --2- 172.21.15.37:0/2146525036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f333c1084f0 0x7f333c1088d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f332c009990 tx=0x7f332c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:52:25.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.070+0000 7f334141c700 1 -- 172.21.15.37:0/2146525036 shutdown_connections 2024-06-10T00:52:25.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.070+0000 7f334141c700 1 --2- 172.21.15.37:0/2146525036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f333c06b3c0 0x7f333c10f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:25.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.070+0000 7f334141c700 1 --2- 172.21.15.37:0/2146525036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f333c1084f0 0x7f333c1088d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:25.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.070+0000 7f334141c700 1 --2- 172.21.15.37:0/2146525036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333c06aa00 0x7f333c06ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:25.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.070+0000 7f334141c700 1 -- 172.21.15.37:0/2146525036 >> 172.21.15.37:0/2146525036 conn(0x7f333c078300 msgr2=0x7f333c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:25.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.071+0000 7f334141c700 1 -- 172.21.15.37:0/2146525036 shutdown_connections 2024-06-10T00:52:25.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.071+0000 7f334141c700 1 -- 172.21.15.37:0/2146525036 wait complete. 2024-06-10T00:52:25.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.071+0000 7f334141c700 1 Processor -- start 2024-06-10T00:52:25.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.072+0000 7f334141c700 1 -- start start 2024-06-10T00:52:25.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.072+0000 7f334141c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f333c06aa00 0x7f333c198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:25.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.072+0000 7f334141c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333c06b3c0 0x7f333c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:25.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.072+0000 7f334141c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f333c1084f0 0x7f333c19d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:25.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.072+0000 7f334141c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f333c1127a0 con 0x7f333c1084f0 2024-06-10T00:52:25.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.072+0000 7f334141c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f333c112620 con 0x7f333c06b3c0 2024-06-10T00:52:25.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.072+0000 7f334141c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f333c112920 con 0x7f333c06aa00 2024-06-10T00:52:25.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.072+0000 7f333b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f333c1084f0 0x7f333c19d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:25.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.072+0000 7f333b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f333c1084f0 0x7f333c19d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43292/0 (socket says 172.21.15.37:43292) 2024-06-10T00:52:25.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.072+0000 7f333b7fe700 1 -- 172.21.15.37:0/294737938 learned_addr learned my addr 172.21.15.37:0/294737938 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:52:25.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.072+0000 7f333affd700 1 --2- 172.21.15.37:0/294737938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f333c06aa00 0x7f333c198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:25.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.072+0000 7f333a7fc700 1 --2- 172.21.15.37:0/294737938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333c06b3c0 0x7f333c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:25.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.072+0000 7f333b7fe700 1 -- 172.21.15.37:0/294737938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f333c06aa00 msgr2=0x7f333c198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:25.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.072+0000 7f333b7fe700 1 --2- 172.21.15.37:0/294737938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f333c06aa00 0x7f333c198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:25.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.072+0000 7f333b7fe700 1 -- 172.21.15.37:0/294737938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333c06b3c0 msgr2=0x7f333c199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:25.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.072+0000 7f333b7fe700 1 --2- 172.21.15.37:0/294737938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333c06b3c0 0x7f333c199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:25.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.072+0000 7f333b7fe700 1 -- 172.21.15.37:0/294737938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f333c19dd40 con 0x7f333c1084f0 2024-06-10T00:52:25.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.072+0000 7f333affd700 1 --2- 172.21.15.37:0/294737938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f333c06aa00 0x7f333c198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:52:25.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.072+0000 7f333b7fe700 1 --2- 172.21.15.37:0/294737938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f333c1084f0 0x7f333c19d600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3334007ee0 tx=0x7f333400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:25.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.075+0000 7f3323fff700 1 -- 172.21.15.37:0/294737938 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333400ddd0 con 0x7f333c1084f0 2024-06-10T00:52:25.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.075+0000 7f334141c700 1 -- 172.21.15.37:0/294737938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f333c19e030 con 0x7f333c1084f0 2024-06-10T00:52:25.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.075+0000 7f3323fff700 1 -- 172.21.15.37:0/294737938 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3334004020 con 0x7f333c1084f0 2024-06-10T00:52:25.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.075+0000 7f3323fff700 1 -- 172.21.15.37:0/294737938 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3334015660 con 0x7f333c1084f0 2024-06-10T00:52:25.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.075+0000 7f334141c700 1 -- 172.21.15.37:0/294737938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f333c1a58b0 con 0x7f333c1084f0 2024-06-10T00:52:25.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.076+0000 7f334141c700 1 -- 172.21.15.37:0/294737938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f333c10a9d0 con 0x7f333c1084f0 2024-06-10T00:52:25.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.077+0000 7f3323fff700 1 -- 172.21.15.37:0/294737938 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f33340041d0 con 0x7f333c1084f0 2024-06-10T00:52:25.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.078+0000 7f3323fff700 1 --2- 172.21.15.37:0/294737938 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f33240a2c20 0x7f33240a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:25.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.078+0000 7f333affd700 1 --2- 172.21.15.37:0/294737938 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f33240a2c20 0x7f33240a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:25.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.078+0000 7f3323fff700 1 -- 172.21.15.37:0/294737938 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f33340cee50 con 0x7f333c1084f0 2024-06-10T00:52:25.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.079+0000 7f333affd700 1 --2- 172.21.15.37:0/294737938 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f33240a2c20 0x7f33240a50e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f332c006010 tx=0x7f332c00b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:25.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.082+0000 7f3323fff700 1 -- 172.21.15.37:0/294737938 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3334010070 con 0x7f333c1084f0 2024-06-10T00:52:25.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.349+0000 7f334141c700 1 -- 172.21.15.37:0/294737938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f333c19a090 con 0x7f333c1084f0 2024-06-10T00:52:25.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.350+0000 7f3323fff700 1 -- 172.21.15.37:0/294737938 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10476 (secure 0 0 0) 0x7f333409c1b0 con 0x7f333c1084f0 2024-06-10T00:52:25.351 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:52:25.351 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24431,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-10T00:52:25.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.352+0000 7f334141c700 1 -- 172.21.15.37:0/294737938 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f33240a2c20 msgr2=0x7f33240a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:25.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.352+0000 7f334141c700 1 --2- 172.21.15.37:0/294737938 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f33240a2c20 0x7f33240a50e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f332c006010 tx=0x7f332c00b260 comp rx=0 tx=0).stop 2024-06-10T00:52:25.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.352+0000 7f334141c700 1 -- 172.21.15.37:0/294737938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f333c1084f0 msgr2=0x7f333c19d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:25.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.352+0000 7f334141c700 1 --2- 172.21.15.37:0/294737938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f333c1084f0 0x7f333c19d600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3334007ee0 tx=0x7f333400d3b0 comp rx=0 tx=0).stop 2024-06-10T00:52:25.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.353+0000 7f334141c700 1 -- 172.21.15.37:0/294737938 shutdown_connections 2024-06-10T00:52:25.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.353+0000 7f334141c700 1 --2- 172.21.15.37:0/294737938 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f33240a2c20 0x7f33240a50e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:25.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.353+0000 7f334141c700 1 --2- 172.21.15.37:0/294737938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f333c1084f0 0x7f333c19d600 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:25.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.353+0000 7f334141c700 1 --2- 172.21.15.37:0/294737938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f333c06aa00 0x7f333c198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:25.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.353+0000 7f334141c700 1 --2- 172.21.15.37:0/294737938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f333c06b3c0 0x7f333c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:25.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.353+0000 7f334141c700 1 -- 172.21.15.37:0/294737938 >> 172.21.15.37:0/294737938 conn(0x7f333c078300 msgr2=0x7f333c100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:25.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.353+0000 7f334141c700 1 -- 172.21.15.37:0/294737938 shutdown_connections 2024-06-10T00:52:25.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:25.353+0000 7f334141c700 1 -- 172.21.15.37:0/294737938 wait complete. 2024-06-10T00:52:25.355 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 13 2024-06-10T00:52:26.260 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:26 smithi037 ceph-mon[32192]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:26.260 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:26 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:26.260 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:26 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:26.260 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:26 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:26.261 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:26 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:26.261 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:26 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:26.261 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:26 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/294737938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:52:26.294 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-06-10T00:52:26.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:26 smithi167 ceph-mon[36917]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:26.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:26 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:26.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:26 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:26.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:26 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:26.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:26 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:26.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:26 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:26.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:26 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/294737938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:52:26.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:26 smithi094 ceph-mon[36708]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:26.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:26 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:26.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:26 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:26.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:26 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:26.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:26 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:26.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:26 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:26.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:26 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/294737938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:52:26.564 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:52:28.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.267+0000 7f70733a4700 1 -- 172.21.15.37:0/3776935828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f706c06ab60 msgr2=0x7f706c06afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:28.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.267+0000 7f70733a4700 1 --2- 172.21.15.37:0/3776935828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f706c06ab60 0x7f706c06afe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7060009990 tx=0x7f7060009ca0 comp rx=0 tx=0).stop 2024-06-10T00:52:28.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.267+0000 7f70733a4700 1 -- 172.21.15.37:0/3776935828 shutdown_connections 2024-06-10T00:52:28.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.267+0000 7f70733a4700 1 --2- 172.21.15.37:0/3776935828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f706c06b520 0x7f706c10f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:28.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.267+0000 7f70733a4700 1 --2- 172.21.15.37:0/3776935828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f706c108530 0x7f706c108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:28.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.267+0000 7f70733a4700 1 --2- 172.21.15.37:0/3776935828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f706c06ab60 0x7f706c06afe0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:28.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.267+0000 7f70733a4700 1 -- 172.21.15.37:0/3776935828 >> 172.21.15.37:0/3776935828 conn(0x7f706c0783d0 msgr2=0x7f706c0787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:28.269 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:28 smithi037 ceph-mon[32192]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:28.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.268+0000 7f70733a4700 1 -- 172.21.15.37:0/3776935828 shutdown_connections 2024-06-10T00:52:28.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.268+0000 7f70733a4700 1 -- 172.21.15.37:0/3776935828 wait complete. 2024-06-10T00:52:28.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.268+0000 7f70733a4700 1 Processor -- start 2024-06-10T00:52:28.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.268+0000 7f70733a4700 1 -- start start 2024-06-10T00:52:28.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.269+0000 7f70733a4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f706c06ab60 0x7f706c19af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:28.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.269+0000 7f70733a4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f706c06b520 0x7f706c19b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:28.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.269+0000 7f70733a4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f706c108530 0x7f706c19f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:28.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.269+0000 7f70733a4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f706c112890 con 0x7f706c06b520 2024-06-10T00:52:28.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.269+0000 7f70733a4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f706c112710 con 0x7f706c06ab60 2024-06-10T00:52:28.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.269+0000 7f70733a4700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f706c112a10 con 0x7f706c108530 2024-06-10T00:52:28.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.269+0000 7f707093f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f706c06b520 0x7f706c19b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:28.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.269+0000 7f707093f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f706c06b520 0x7f706c19b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43302/0 (socket says 172.21.15.37:43302) 2024-06-10T00:52:28.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.269+0000 7f707093f700 1 -- 172.21.15.37:0/2011477818 learned_addr learned my addr 172.21.15.37:0/2011477818 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:52:28.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.269+0000 7f7071941700 1 --2- 172.21.15.37:0/2011477818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f706c108530 0x7f706c19f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:28.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.269+0000 7f7071140700 1 --2- 172.21.15.37:0/2011477818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f706c06ab60 0x7f706c19af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:28.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.269+0000 7f707093f700 1 -- 172.21.15.37:0/2011477818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f706c108530 msgr2=0x7f706c19f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:28.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.269+0000 7f707093f700 1 --2- 172.21.15.37:0/2011477818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f706c108530 0x7f706c19f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:28.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.269+0000 7f707093f700 1 -- 172.21.15.37:0/2011477818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f706c06ab60 msgr2=0x7f706c19af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:28.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.269+0000 7f707093f700 1 --2- 172.21.15.37:0/2011477818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f706c06ab60 0x7f706c19af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:28.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.269+0000 7f707093f700 1 -- 172.21.15.37:0/2011477818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f706c19ffa0 con 0x7f706c06b520 2024-06-10T00:52:28.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.269+0000 7f7071140700 1 --2- 172.21.15.37:0/2011477818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f706c06ab60 0x7f706c19af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:52:28.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.269+0000 7f707093f700 1 --2- 172.21.15.37:0/2011477818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f706c06b520 0x7f706c19b4d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7060000c00 tx=0x7f7060003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:28.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.269+0000 7f705e7fc700 1 -- 172.21.15.37:0/2011477818 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7060027070 con 0x7f706c06b520 2024-06-10T00:52:28.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.270+0000 7f70733a4700 1 -- 172.21.15.37:0/2011477818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f706c1a0230 con 0x7f706c06b520 2024-06-10T00:52:28.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.270+0000 7f705e7fc700 1 -- 172.21.15.37:0/2011477818 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7060004110 con 0x7f706c06b520 2024-06-10T00:52:28.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.270+0000 7f705e7fc700 1 -- 172.21.15.37:0/2011477818 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706000f460 con 0x7f706c06b520 2024-06-10T00:52:28.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.270+0000 7f70733a4700 1 -- 172.21.15.37:0/2011477818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f706c1a7b10 con 0x7f706c06b520 2024-06-10T00:52:28.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.271+0000 7f70733a4700 1 -- 172.21.15.37:0/2011477818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f706c103bf0 con 0x7f706c06b520 2024-06-10T00:52:28.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.272+0000 7f705e7fc700 1 -- 172.21.15.37:0/2011477818 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f706002a070 con 0x7f706c06b520 2024-06-10T00:52:28.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.273+0000 7f705e7fc700 1 --2- 172.21.15.37:0/2011477818 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f70580a2ca0 0x7f70580a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:28.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.273+0000 7f7071140700 1 --2- 172.21.15.37:0/2011477818 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f70580a2ca0 0x7f70580a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:28.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.273+0000 7f705e7fc700 1 -- 172.21.15.37:0/2011477818 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f70600cff40 con 0x7f706c06b520 2024-06-10T00:52:28.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.273+0000 7f7071140700 1 --2- 172.21.15.37:0/2011477818 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f70580a2ca0 0x7f70580a5160 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f70680099d0 tx=0x7f7068008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:28.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.276+0000 7f705e7fc700 1 -- 172.21.15.37:0/2011477818 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7060031070 con 0x7f706c06b520 2024-06-10T00:52:28.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:28 smithi167 ceph-mon[36917]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:28.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:28 smithi094 ceph-mon[36708]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:28.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.513+0000 7f70733a4700 1 -- 172.21.15.37:0/2011477818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f706c1a7ce0 con 0x7f706c06b520 2024-06-10T00:52:28.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.514+0000 7f705e7fc700 1 -- 172.21.15.37:0/2011477818 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f7060013070 con 0x7f706c06b520 2024-06-10T00:52:28.515 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:52:28.515 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-06-10T00:52:28.515 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T00:52:28.515 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-06-10T00:52:28.515 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-06-10T00:52:28.515 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T00:52:28.516 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-06-10T00:52:28.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.516+0000 7f70733a4700 1 -- 172.21.15.37:0/2011477818 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f70580a2ca0 msgr2=0x7f70580a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:28.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.516+0000 7f70733a4700 1 --2- 172.21.15.37:0/2011477818 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f70580a2ca0 0x7f70580a5160 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f70680099d0 tx=0x7f7068008040 comp rx=0 tx=0).stop 2024-06-10T00:52:28.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.516+0000 7f70733a4700 1 -- 172.21.15.37:0/2011477818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f706c06b520 msgr2=0x7f706c19b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:28.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.516+0000 7f70733a4700 1 --2- 172.21.15.37:0/2011477818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f706c06b520 0x7f706c19b4d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7060000c00 tx=0x7f7060003680 comp rx=0 tx=0).stop 2024-06-10T00:52:28.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.516+0000 7f70733a4700 1 -- 172.21.15.37:0/2011477818 shutdown_connections 2024-06-10T00:52:28.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.516+0000 7f70733a4700 1 --2- 172.21.15.37:0/2011477818 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f70580a2ca0 0x7f70580a5160 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:28.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.516+0000 7f70733a4700 1 --2- 172.21.15.37:0/2011477818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f706c108530 0x7f706c19f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:28.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.516+0000 7f70733a4700 1 --2- 172.21.15.37:0/2011477818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f706c06ab60 0x7f706c19af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:28.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.516+0000 7f70733a4700 1 --2- 172.21.15.37:0/2011477818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f706c06b520 0x7f706c19b4d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:28.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.516+0000 7f70733a4700 1 -- 172.21.15.37:0/2011477818 >> 172.21.15.37:0/2011477818 conn(0x7f706c0783d0 msgr2=0x7f706c10d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:28.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.517+0000 7f70733a4700 1 -- 172.21.15.37:0/2011477818 shutdown_connections 2024-06-10T00:52:28.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:28.517+0000 7f70733a4700 1 -- 172.21.15.37:0/2011477818 wait complete. 2024-06-10T00:52:29.216 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-06-10T00:52:29.217 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-06-10T00:52:29.469 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:52:29.592 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:29 smithi037 ceph-mon[32192]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:29.592 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:29 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:29.592 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:29 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:29.592 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:29 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2011477818' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-10T00:52:29.592 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:29 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:52:29.593 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:29 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:52:29.593 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:29 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:29.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:29 smithi167 ceph-mon[36917]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:29.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:29 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:29.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:29 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:29.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:29 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2011477818' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-10T00:52:29.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:29 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:52:29.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:29 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:52:29.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:29 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:29.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:29 smithi094 ceph-mon[36708]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:29.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:29 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:29.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:29 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:29.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:29 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2011477818' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-10T00:52:29.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:29 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:52:29.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:29 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:52:29.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:29 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:52:31.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.031+0000 7f604583d700 1 -- 172.21.15.37:0/3542705909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6040108580 msgr2=0x7f6040108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:31.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.031+0000 7f604583d700 1 --2- 172.21.15.37:0/3542705909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6040108580 0x7f6040108960 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f60300099c0 tx=0x7f6030009cd0 comp rx=0 tx=0).stop 2024-06-10T00:52:31.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.031+0000 7f604583d700 1 -- 172.21.15.37:0/3542705909 shutdown_connections 2024-06-10T00:52:31.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.031+0000 7f604583d700 1 --2- 172.21.15.37:0/3542705909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f604006b620 0x7f604010f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:31.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.031+0000 7f604583d700 1 --2- 172.21.15.37:0/3542705909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6040108580 0x7f6040108960 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:31.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.031+0000 7f604583d700 1 --2- 172.21.15.37:0/3542705909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f604006ac60 0x7f604006b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:31.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.031+0000 7f604583d700 1 -- 172.21.15.37:0/3542705909 >> 172.21.15.37:0/3542705909 conn(0x7f6040078430 msgr2=0x7f6040078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:31.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.031+0000 7f604583d700 1 -- 172.21.15.37:0/3542705909 shutdown_connections 2024-06-10T00:52:31.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.031+0000 7f604583d700 1 -- 172.21.15.37:0/3542705909 wait complete. 2024-06-10T00:52:31.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.032+0000 7f604583d700 1 Processor -- start 2024-06-10T00:52:31.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.032+0000 7f604583d700 1 -- start start 2024-06-10T00:52:31.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.032+0000 7f604583d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f604006ac60 0x7f6040101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:31.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.032+0000 7f604583d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f604006b620 0x7f6040101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:31.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.032+0000 7f604583d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6040108580 0x7f6040101e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:31.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.032+0000 7f604583d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f60401128b0 con 0x7f604006ac60 2024-06-10T00:52:31.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.032+0000 7f604583d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6040112730 con 0x7f604006b620 2024-06-10T00:52:31.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.032+0000 7f604583d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6040112a30 con 0x7f6040108580 2024-06-10T00:52:31.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.032+0000 7f603effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f604006ac60 0x7f6040101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:31.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.032+0000 7f603effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f604006ac60 0x7f6040101400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43318/0 (socket says 172.21.15.37:43318) 2024-06-10T00:52:31.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.032+0000 7f603effd700 1 -- 172.21.15.37:0/3318672053 learned_addr learned my addr 172.21.15.37:0/3318672053 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:52:31.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.032+0000 7f603e7fc700 1 --2- 172.21.15.37:0/3318672053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f604006b620 0x7f6040101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:31.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.033+0000 7f603f7fe700 1 --2- 172.21.15.37:0/3318672053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6040108580 0x7f6040101e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:31.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.033+0000 7f603effd700 1 -- 172.21.15.37:0/3318672053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6040108580 msgr2=0x7f6040101e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:31.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.033+0000 7f603effd700 1 --2- 172.21.15.37:0/3318672053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6040108580 0x7f6040101e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:31.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.033+0000 7f603effd700 1 -- 172.21.15.37:0/3318672053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f604006b620 msgr2=0x7f6040101940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:31.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.033+0000 7f603effd700 1 --2- 172.21.15.37:0/3318672053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f604006b620 0x7f6040101940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:31.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.033+0000 7f603effd700 1 -- 172.21.15.37:0/3318672053 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60401a7290 con 0x7f604006ac60 2024-06-10T00:52:31.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.033+0000 7f603f7fe700 1 --2- 172.21.15.37:0/3318672053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6040108580 0x7f6040101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:52:31.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.033+0000 7f603effd700 1 --2- 172.21.15.37:0/3318672053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f604006ac60 0x7f6040101400 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6030009fd0 tx=0x7f6030003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:31.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.033+0000 7f604483b700 1 -- 172.21.15.37:0/3318672053 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030027070 con 0x7f604006ac60 2024-06-10T00:52:31.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.033+0000 7f604583d700 1 -- 172.21.15.37:0/3318672053 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60401a7520 con 0x7f604006ac60 2024-06-10T00:52:31.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.033+0000 7f604483b700 1 -- 172.21.15.37:0/3318672053 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60300040d0 con 0x7f604006ac60 2024-06-10T00:52:31.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.033+0000 7f604483b700 1 -- 172.21.15.37:0/3318672053 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603000f460 con 0x7f604006ac60 2024-06-10T00:52:31.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.033+0000 7f604583d700 1 -- 172.21.15.37:0/3318672053 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60401a79d0 con 0x7f604006ac60 2024-06-10T00:52:31.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.034+0000 7f604583d700 1 -- 172.21.15.37:0/3318672053 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f604010aa10 con 0x7f604006ac60 2024-06-10T00:52:31.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.035+0000 7f604483b700 1 -- 172.21.15.37:0/3318672053 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f603002a070 con 0x7f604006ac60 2024-06-10T00:52:31.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.036+0000 7f604483b700 1 --2- 172.21.15.37:0/3318672053 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f60280a2c20 0x7f60280a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:31.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.036+0000 7f603e7fc700 1 --2- 172.21.15.37:0/3318672053 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f60280a2c20 0x7f60280a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:31.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.036+0000 7f604483b700 1 -- 172.21.15.37:0/3318672053 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f60300d02c0 con 0x7f604006ac60 2024-06-10T00:52:31.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.037+0000 7f603e7fc700 1 --2- 172.21.15.37:0/3318672053 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f60280a2c20 0x7f60280a50e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6040078a30 tx=0x7f6034009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:31.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.040+0000 7f604483b700 1 -- 172.21.15.37:0/3318672053 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6030031070 con 0x7f604006ac60 2024-06-10T00:52:31.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.247+0000 7f604583d700 1 -- 172.21.15.37:0/3318672053 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f604010a3f0 con 0x7f604006ac60 2024-06-10T00:52:31.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.737+0000 7f604483b700 1 -- 172.21.15.37:0/3318672053 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f6030013070 con 0x7f604006ac60 2024-06-10T00:52:31.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.740+0000 7f604583d700 1 -- 172.21.15.37:0/3318672053 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f60280a2c20 msgr2=0x7f60280a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:31.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.740+0000 7f604583d700 1 --2- 172.21.15.37:0/3318672053 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f60280a2c20 0x7f60280a50e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6040078a30 tx=0x7f6034009190 comp rx=0 tx=0).stop 2024-06-10T00:52:31.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.740+0000 7f604583d700 1 -- 172.21.15.37:0/3318672053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f604006ac60 msgr2=0x7f6040101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:31.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.740+0000 7f604583d700 1 --2- 172.21.15.37:0/3318672053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f604006ac60 0x7f6040101400 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6030009fd0 tx=0x7f6030003680 comp rx=0 tx=0).stop 2024-06-10T00:52:31.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.740+0000 7f604583d700 1 -- 172.21.15.37:0/3318672053 shutdown_connections 2024-06-10T00:52:31.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.740+0000 7f604583d700 1 --2- 172.21.15.37:0/3318672053 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f60280a2c20 0x7f60280a50e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:31.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.740+0000 7f604583d700 1 --2- 172.21.15.37:0/3318672053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6040108580 0x7f6040101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:31.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.740+0000 7f604583d700 1 --2- 172.21.15.37:0/3318672053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f604006ac60 0x7f6040101400 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:31.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.740+0000 7f604583d700 1 --2- 172.21.15.37:0/3318672053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f604006b620 0x7f6040101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:31.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.740+0000 7f604583d700 1 -- 172.21.15.37:0/3318672053 >> 172.21.15.37:0/3318672053 conn(0x7f6040078430 msgr2=0x7f60400fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:31.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.740+0000 7f604583d700 1 -- 172.21.15.37:0/3318672053 shutdown_connections 2024-06-10T00:52:31.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:31.740+0000 7f604583d700 1 -- 172.21.15.37:0/3318672053 wait complete. 2024-06-10T00:52:31.742 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-06-10T00:52:32.059 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:31 smithi037 ceph-mon[32192]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:32.060 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:31 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3318672053' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-10T00:52:32.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:31 smithi167 ceph-mon[36917]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:32.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:31 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3318672053' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-10T00:52:32.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:31 smithi094 ceph-mon[36708]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:32.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:31 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3318672053' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-10T00:52:32.373 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-06-10T00:52:32.626 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:52:33.056 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:32 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3318672053' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-10T00:52:33.057 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:32 smithi037 ceph-mon[32192]: osdmap e83: 12 total, 12 up, 12 in 2024-06-10T00:52:33.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:32 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3318672053' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-10T00:52:33.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:32 smithi167 ceph-mon[36917]: osdmap e83: 12 total, 12 up, 12 in 2024-06-10T00:52:33.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:32 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3318672053' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-10T00:52:33.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:32 smithi094 ceph-mon[36708]: osdmap e83: 12 total, 12 up, 12 in 2024-06-10T00:52:33.965 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:33 smithi037 ceph-mon[32192]: pgmap v289: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:33.965 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:33 smithi037 ceph-mon[32192]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-10T00:52:33.966 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:33 smithi037 ceph-mon[32192]: osdmap e84: 12 total, 12 up, 12 in 2024-06-10T00:52:34.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:33 smithi167 ceph-mon[36917]: pgmap v289: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:34.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:33 smithi167 ceph-mon[36917]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-10T00:52:34.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:33 smithi167 ceph-mon[36917]: osdmap e84: 12 total, 12 up, 12 in 2024-06-10T00:52:34.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:33 smithi094 ceph-mon[36708]: pgmap v289: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:34.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:33 smithi094 ceph-mon[36708]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-10T00:52:34.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:33 smithi094 ceph-mon[36708]: osdmap e84: 12 total, 12 up, 12 in 2024-06-10T00:52:34.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.236+0000 7fac5a6c9700 1 -- 172.21.15.37:0/698739392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac54101d30 msgr2=0x7fac5410e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:34.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.236+0000 7fac5a6c9700 1 --2- 172.21.15.37:0/698739392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac54101d30 0x7fac5410e210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fac4c00b0a0 tx=0x7fac4c00b3b0 comp rx=0 tx=0).stop 2024-06-10T00:52:34.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.236+0000 7fac5a6c9700 1 -- 172.21.15.37:0/698739392 shutdown_connections 2024-06-10T00:52:34.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.236+0000 7fac5a6c9700 1 --2- 172.21.15.37:0/698739392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac54101d30 0x7fac5410e210 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:34.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.236+0000 7fac5a6c9700 1 --2- 172.21.15.37:0/698739392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac541073d0 0x7fac541077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:34.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.236+0000 7fac5a6c9700 1 --2- 172.21.15.37:0/698739392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac54101390 0x7fac541017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:34.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.236+0000 7fac5a6c9700 1 -- 172.21.15.37:0/698739392 >> 172.21.15.37:0/698739392 conn(0x7fac540782c0 msgr2=0x7fac540786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:34.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.236+0000 7fac5a6c9700 1 -- 172.21.15.37:0/698739392 shutdown_connections 2024-06-10T00:52:34.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.236+0000 7fac5a6c9700 1 -- 172.21.15.37:0/698739392 wait complete. 2024-06-10T00:52:34.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.237+0000 7fac5a6c9700 1 Processor -- start 2024-06-10T00:52:34.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.237+0000 7fac5a6c9700 1 -- start start 2024-06-10T00:52:34.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.237+0000 7fac5a6c9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac54101390 0x7fac5419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:34.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.237+0000 7fac5a6c9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac54101d30 0x7fac5419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:34.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.237+0000 7fac5a6c9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac541073d0 0x7fac54198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:34.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.237+0000 7fac5a6c9700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fac54111650 con 0x7fac541073d0 2024-06-10T00:52:34.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.237+0000 7fac5a6c9700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fac541114d0 con 0x7fac54101d30 2024-06-10T00:52:34.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.237+0000 7fac5a6c9700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fac541117d0 con 0x7fac54101390 2024-06-10T00:52:34.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.237+0000 7fac58c66700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac541073d0 0x7fac54198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:34.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.237+0000 7fac58c66700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac541073d0 0x7fac54198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35756/0 (socket says 172.21.15.37:35756) 2024-06-10T00:52:34.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.237+0000 7fac58c66700 1 -- 172.21.15.37:0/4104509085 learned_addr learned my addr 172.21.15.37:0/4104509085 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:52:34.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.237+0000 7fac4bfff700 1 --2- 172.21.15.37:0/4104509085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac54101d30 0x7fac5419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:34.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.238+0000 7fac53fff700 1 --2- 172.21.15.37:0/4104509085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac54101390 0x7fac5419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:34.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.238+0000 7fac58c66700 1 -- 172.21.15.37:0/4104509085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac54101390 msgr2=0x7fac5419e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:34.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.238+0000 7fac58c66700 1 --2- 172.21.15.37:0/4104509085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac54101390 0x7fac5419e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:34.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.238+0000 7fac58c66700 1 -- 172.21.15.37:0/4104509085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac54101d30 msgr2=0x7fac5419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:34.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.238+0000 7fac58c66700 1 --2- 172.21.15.37:0/4104509085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac54101d30 0x7fac5419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:34.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.238+0000 7fac58c66700 1 -- 172.21.15.37:0/4104509085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac54199190 con 0x7fac541073d0 2024-06-10T00:52:34.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.238+0000 7fac53fff700 1 --2- 172.21.15.37:0/4104509085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac54101390 0x7fac5419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:52:34.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.238+0000 7fac58c66700 1 --2- 172.21.15.37:0/4104509085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac541073d0 0x7fac54198a30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0062a0 tx=0x7fac4c004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:34.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.238+0000 7fac51ffb700 1 -- 172.21.15.37:0/4104509085 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c021070 con 0x7fac541073d0 2024-06-10T00:52:34.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.238+0000 7fac51ffb700 1 -- 172.21.15.37:0/4104509085 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac4c007c00 con 0x7fac541073d0 2024-06-10T00:52:34.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.238+0000 7fac5a6c9700 1 -- 172.21.15.37:0/4104509085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac54199420 con 0x7fac541073d0 2024-06-10T00:52:34.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.238+0000 7fac51ffb700 1 -- 172.21.15.37:0/4104509085 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c01d450 con 0x7fac541073d0 2024-06-10T00:52:34.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.238+0000 7fac5a6c9700 1 -- 172.21.15.37:0/4104509085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac541a56f0 con 0x7fac541073d0 2024-06-10T00:52:34.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.239+0000 7fac5a6c9700 1 -- 172.21.15.37:0/4104509085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac54069180 con 0x7fac541073d0 2024-06-10T00:52:34.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.240+0000 7fac51ffb700 1 -- 172.21.15.37:0/4104509085 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fac4c004020 con 0x7fac541073d0 2024-06-10T00:52:34.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.241+0000 7fac51ffb700 1 --2- 172.21.15.37:0/4104509085 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fac400a2c20 0x7fac400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:34.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.241+0000 7fac53fff700 1 --2- 172.21.15.37:0/4104509085 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fac400a2c20 0x7fac400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:34.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.241+0000 7fac51ffb700 1 -- 172.21.15.37:0/4104509085 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7497+0+0 (secure 0 0 0) 0x7fac4c00f020 con 0x7fac541073d0 2024-06-10T00:52:34.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.241+0000 7fac53fff700 1 --2- 172.21.15.37:0/4104509085 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fac400a2c20 0x7fac400a50e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fac44005e20 tx=0x7fac44005db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:34.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.246+0000 7fac51ffb700 1 -- 172.21.15.37:0/4104509085 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fac4c02a020 con 0x7fac541073d0 2024-06-10T00:52:34.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.453+0000 7fac5a6c9700 1 -- 172.21.15.37:0/4104509085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fac541a58c0 con 0x7fac541073d0 2024-06-10T00:52:34.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.936+0000 7fac51ffb700 1 -- 172.21.15.37:0/4104509085 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v86) ==== 171+0+0 (secure 0 0 0) 0x7fac4c09d610 con 0x7fac541073d0 2024-06-10T00:52:34.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.938+0000 7fac5a6c9700 1 -- 172.21.15.37:0/4104509085 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fac400a2c20 msgr2=0x7fac400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:34.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.938+0000 7fac5a6c9700 1 --2- 172.21.15.37:0/4104509085 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fac400a2c20 0x7fac400a50e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fac44005e20 tx=0x7fac44005db0 comp rx=0 tx=0).stop 2024-06-10T00:52:34.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.938+0000 7fac5a6c9700 1 -- 172.21.15.37:0/4104509085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac541073d0 msgr2=0x7fac54198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:34.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.938+0000 7fac5a6c9700 1 --2- 172.21.15.37:0/4104509085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac541073d0 0x7fac54198a30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0062a0 tx=0x7fac4c004d70 comp rx=0 tx=0).stop 2024-06-10T00:52:34.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.939+0000 7fac5a6c9700 1 -- 172.21.15.37:0/4104509085 shutdown_connections 2024-06-10T00:52:34.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.939+0000 7fac5a6c9700 1 --2- 172.21.15.37:0/4104509085 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fac400a2c20 0x7fac400a50e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:34.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.939+0000 7fac5a6c9700 1 --2- 172.21.15.37:0/4104509085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac541073d0 0x7fac54198a30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:34.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.939+0000 7fac5a6c9700 1 --2- 172.21.15.37:0/4104509085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac54101390 0x7fac5419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:34.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.939+0000 7fac5a6c9700 1 --2- 172.21.15.37:0/4104509085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac54101d30 0x7fac5419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:34.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.939+0000 7fac5a6c9700 1 -- 172.21.15.37:0/4104509085 >> 172.21.15.37:0/4104509085 conn(0x7fac540782c0 msgr2=0x7fac541006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:34.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.939+0000 7fac5a6c9700 1 -- 172.21.15.37:0/4104509085 shutdown_connections 2024-06-10T00:52:34.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:34.939+0000 7fac5a6c9700 1 -- 172.21.15.37:0/4104509085 wait complete. 2024-06-10T00:52:34.940 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-06-10T00:52:35.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:34 smithi037 ceph-mon[32192]: osdmap e85: 12 total, 12 up, 12 in 2024-06-10T00:52:35.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:34 smithi167 ceph-mon[36917]: osdmap e85: 12 total, 12 up, 12 in 2024-06-10T00:52:35.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:34 smithi094 ceph-mon[36708]: osdmap e85: 12 total, 12 up, 12 in 2024-06-10T00:52:35.636 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-06-10T00:52:35.888 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:52:36.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:35 smithi037 ceph-mon[32192]: pgmap v292: 33 pgs: 8 creating+peering, 19 active+clean, 6 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:36.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:35 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/4104509085' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-10T00:52:36.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:35 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/4104509085' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-10T00:52:36.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:35 smithi037 ceph-mon[32192]: osdmap e86: 12 total, 12 up, 12 in 2024-06-10T00:52:36.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:35 smithi167 ceph-mon[36917]: pgmap v292: 33 pgs: 8 creating+peering, 19 active+clean, 6 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:36.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:35 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/4104509085' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-10T00:52:36.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:35 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/4104509085' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-10T00:52:36.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:35 smithi167 ceph-mon[36917]: osdmap e86: 12 total, 12 up, 12 in 2024-06-10T00:52:36.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:35 smithi094 ceph-mon[36708]: pgmap v292: 33 pgs: 8 creating+peering, 19 active+clean, 6 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:36.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:35 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/4104509085' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-10T00:52:36.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:35 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/4104509085' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-10T00:52:36.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:35 smithi094 ceph-mon[36708]: osdmap e86: 12 total, 12 up, 12 in 2024-06-10T00:52:37.078 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:36 smithi037 ceph-mon[32192]: osdmap e87: 12 total, 12 up, 12 in 2024-06-10T00:52:37.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:36 smithi167 ceph-mon[36917]: osdmap e87: 12 total, 12 up, 12 in 2024-06-10T00:52:37.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:36 smithi094 ceph-mon[36708]: osdmap e87: 12 total, 12 up, 12 in 2024-06-10T00:52:37.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.606+0000 7f185995a700 1 -- 172.21.15.37:0/855155367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1854107420 msgr2=0x7f1854107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:37.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.606+0000 7f185995a700 1 --2- 172.21.15.37:0/855155367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1854107420 0x7f1854107800 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f184400b0a0 tx=0x7f184400b3b0 comp rx=0 tx=0).stop 2024-06-10T00:52:37.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.606+0000 7f185995a700 1 -- 172.21.15.37:0/855155367 shutdown_connections 2024-06-10T00:52:37.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.606+0000 7f185995a700 1 --2- 172.21.15.37:0/855155367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1854101d30 0x7f185410e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:37.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.606+0000 7f185995a700 1 --2- 172.21.15.37:0/855155367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1854107420 0x7f1854107800 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:37.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.606+0000 7f185995a700 1 --2- 172.21.15.37:0/855155367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1854101370 0x7f18541017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:37.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.606+0000 7f185995a700 1 -- 172.21.15.37:0/855155367 >> 172.21.15.37:0/855155367 conn(0x7f18540783a0 msgr2=0x7f18540787b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:37.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.606+0000 7f185995a700 1 -- 172.21.15.37:0/855155367 shutdown_connections 2024-06-10T00:52:37.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.606+0000 7f185995a700 1 -- 172.21.15.37:0/855155367 wait complete. 2024-06-10T00:52:37.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.607+0000 7f185995a700 1 Processor -- start 2024-06-10T00:52:37.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.607+0000 7f185995a700 1 -- start start 2024-06-10T00:52:37.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.607+0000 7f185995a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1854101370 0x7f1854199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:37.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.607+0000 7f185995a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1854101d30 0x7f1854199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:37.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.607+0000 7f185995a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1854107420 0x7f18541a49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:37.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.607+0000 7f185995a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1854111630 con 0x7f1854107420 2024-06-10T00:52:37.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.607+0000 7f185995a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f18541114b0 con 0x7f1854101370 2024-06-10T00:52:37.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.607+0000 7f185995a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f18541117b0 con 0x7f1854101d30 2024-06-10T00:52:37.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.607+0000 7f18537fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1854107420 0x7f18541a49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:37.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.607+0000 7f18537fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1854107420 0x7f18541a49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35768/0 (socket says 172.21.15.37:35768) 2024-06-10T00:52:37.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.607+0000 7f18537fe700 1 -- 172.21.15.37:0/65224182 learned_addr learned my addr 172.21.15.37:0/65224182 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:52:37.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.607+0000 7f18527fc700 1 --2- 172.21.15.37:0/65224182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1854101d30 0x7f1854199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:37.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.608+0000 7f1852ffd700 1 --2- 172.21.15.37:0/65224182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1854101370 0x7f1854199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:37.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.608+0000 7f18537fe700 1 -- 172.21.15.37:0/65224182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1854101d30 msgr2=0x7f1854199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:37.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.608+0000 7f18537fe700 1 --2- 172.21.15.37:0/65224182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1854101d30 0x7f1854199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:37.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.608+0000 7f18537fe700 1 -- 172.21.15.37:0/65224182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1854101370 msgr2=0x7f1854199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:37.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.608+0000 7f18537fe700 1 --2- 172.21.15.37:0/65224182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1854101370 0x7f1854199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:37.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.608+0000 7f18537fe700 1 -- 172.21.15.37:0/65224182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18541a4ef0 con 0x7f1854107420 2024-06-10T00:52:37.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.608+0000 7f18537fe700 1 --2- 172.21.15.37:0/65224182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1854107420 0x7f18541a49b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f184c00c900 tx=0x7f184c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:37.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.608+0000 7f1858958700 1 -- 172.21.15.37:0/65224182 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184c0078d0 con 0x7f1854107420 2024-06-10T00:52:37.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.608+0000 7f185995a700 1 -- 172.21.15.37:0/65224182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18541a50c0 con 0x7f1854107420 2024-06-10T00:52:37.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.608+0000 7f1858958700 1 -- 172.21.15.37:0/65224182 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f184c00f450 con 0x7f1854107420 2024-06-10T00:52:37.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.608+0000 7f1858958700 1 -- 172.21.15.37:0/65224182 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184c018660 con 0x7f1854107420 2024-06-10T00:52:37.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.608+0000 7f185995a700 1 -- 172.21.15.37:0/65224182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18541a55d0 con 0x7f1854107420 2024-06-10T00:52:37.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.609+0000 7f185995a700 1 -- 172.21.15.37:0/65224182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1854069180 con 0x7f1854107420 2024-06-10T00:52:37.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.611+0000 7f1858958700 1 -- 172.21.15.37:0/65224182 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f184c007a70 con 0x7f1854107420 2024-06-10T00:52:37.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.612+0000 7f1858958700 1 --2- 172.21.15.37:0/65224182 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f183c0a2c20 0x7f183c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:37.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.612+0000 7f1852ffd700 1 --2- 172.21.15.37:0/65224182 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f183c0a2c20 0x7f183c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:37.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.612+0000 7f1858958700 1 -- 172.21.15.37:0/65224182 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7879+0+0 (secure 0 0 0) 0x7f184c0cee80 con 0x7f1854107420 2024-06-10T00:52:37.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.612+0000 7f1852ffd700 1 --2- 172.21.15.37:0/65224182 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f183c0a2c20 0x7f183c0a50e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f184400b070 tx=0x7f184400b7b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:37.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.616+0000 7f1858958700 1 -- 172.21.15.37:0/65224182 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f184c014030 con 0x7f1854107420 2024-06-10T00:52:37.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.867+0000 7f185995a700 1 -- 172.21.15.37:0/65224182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f185402dca0 con 0x7f1854107420 2024-06-10T00:52:37.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.946+0000 7f1858958700 1 -- 172.21.15.37:0/65224182 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f184c09bf90 con 0x7f1854107420 2024-06-10T00:52:37.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.949+0000 7f185995a700 1 -- 172.21.15.37:0/65224182 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f183c0a2c20 msgr2=0x7f183c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:37.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.949+0000 7f185995a700 1 --2- 172.21.15.37:0/65224182 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f183c0a2c20 0x7f183c0a50e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f184400b070 tx=0x7f184400b7b0 comp rx=0 tx=0).stop 2024-06-10T00:52:37.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.949+0000 7f185995a700 1 -- 172.21.15.37:0/65224182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1854107420 msgr2=0x7f18541a49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:37.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.949+0000 7f185995a700 1 --2- 172.21.15.37:0/65224182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1854107420 0x7f18541a49b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f184c00c900 tx=0x7f184c00cc10 comp rx=0 tx=0).stop 2024-06-10T00:52:37.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.949+0000 7f185995a700 1 -- 172.21.15.37:0/65224182 shutdown_connections 2024-06-10T00:52:37.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.949+0000 7f185995a700 1 --2- 172.21.15.37:0/65224182 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f183c0a2c20 0x7f183c0a50e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:37.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.949+0000 7f185995a700 1 --2- 172.21.15.37:0/65224182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1854107420 0x7f18541a49b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:37.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.949+0000 7f185995a700 1 --2- 172.21.15.37:0/65224182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1854101370 0x7f1854199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:37.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.949+0000 7f185995a700 1 --2- 172.21.15.37:0/65224182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1854101d30 0x7f1854199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:37.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.949+0000 7f185995a700 1 -- 172.21.15.37:0/65224182 >> 172.21.15.37:0/65224182 conn(0x7f18540783a0 msgr2=0x7f18540ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:37.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.949+0000 7f185995a700 1 -- 172.21.15.37:0/65224182 shutdown_connections 2024-06-10T00:52:37.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:37.949+0000 7f185995a700 1 -- 172.21.15.37:0/65224182 wait complete. 2024-06-10T00:52:37.951 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 2 and data pool 3 2024-06-10T00:52:38.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:37 smithi037 ceph-mon[32192]: pgmap v295: 97 pgs: 10 creating+peering, 29 active+clean, 58 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:38.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:37 smithi037 ceph-mon[32192]: osdmap e88: 12 total, 12 up, 12 in 2024-06-10T00:52:38.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:37 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/65224182' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-10T00:52:38.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:37 smithi037 ceph-mon[32192]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-06-10T00:52:38.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:37 smithi167 ceph-mon[36917]: pgmap v295: 97 pgs: 10 creating+peering, 29 active+clean, 58 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:38.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:37 smithi167 ceph-mon[36917]: osdmap e88: 12 total, 12 up, 12 in 2024-06-10T00:52:38.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:37 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/65224182' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-10T00:52:38.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:37 smithi167 ceph-mon[36917]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-06-10T00:52:38.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:37 smithi094 ceph-mon[36708]: pgmap v295: 97 pgs: 10 creating+peering, 29 active+clean, 58 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:38.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:37 smithi094 ceph-mon[36708]: osdmap e88: 12 total, 12 up, 12 in 2024-06-10T00:52:38.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:37 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/65224182' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-10T00:52:38.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:37 smithi094 ceph-mon[36708]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-06-10T00:52:38.627 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-06-10T00:52:38.627 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-06-10T00:52:38.885 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:52:39.266 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:38 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/65224182' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-10T00:52:39.266 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:38 smithi037 ceph-mon[32192]: osdmap e89: 12 total, 12 up, 12 in 2024-06-10T00:52:39.267 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:38 smithi037 ceph-mon[32192]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-06-10T00:52:39.267 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:38 smithi037 ceph-mon[32192]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-06-10T00:52:39.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:38 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/65224182' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-10T00:52:39.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:38 smithi167 ceph-mon[36917]: osdmap e89: 12 total, 12 up, 12 in 2024-06-10T00:52:39.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:38 smithi167 ceph-mon[36917]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-06-10T00:52:39.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:38 smithi167 ceph-mon[36917]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-06-10T00:52:39.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:38 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/65224182' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-10T00:52:39.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:38 smithi094 ceph-mon[36708]: osdmap e89: 12 total, 12 up, 12 in 2024-06-10T00:52:39.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:38 smithi094 ceph-mon[36708]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-06-10T00:52:39.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:38 smithi094 ceph-mon[36708]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-06-10T00:52:40.029 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:39 smithi037 ceph-mon[32192]: pgmap v298: 97 pgs: 2 creating+peering, 61 active+clean, 34 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 1 op/s 2024-06-10T00:52:40.030 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:39 smithi037 ceph-mon[32192]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-10T00:52:40.030 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:39 smithi037 ceph-mon[32192]: Cluster is now healthy 2024-06-10T00:52:40.030 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:39 smithi037 ceph-mon[32192]: mds.? [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] up:active 2024-06-10T00:52:40.030 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:39 smithi037 ceph-mon[32192]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T00:52:40.030 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:39 smithi037 ceph-mon[32192]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T00:52:40.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:39 smithi167 ceph-mon[36917]: pgmap v298: 97 pgs: 2 creating+peering, 61 active+clean, 34 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 1 op/s 2024-06-10T00:52:40.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:39 smithi167 ceph-mon[36917]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-10T00:52:40.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:39 smithi167 ceph-mon[36917]: Cluster is now healthy 2024-06-10T00:52:40.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:39 smithi167 ceph-mon[36917]: mds.? [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] up:active 2024-06-10T00:52:40.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:39 smithi167 ceph-mon[36917]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T00:52:40.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:39 smithi167 ceph-mon[36917]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T00:52:40.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:39 smithi094 ceph-mon[36708]: pgmap v298: 97 pgs: 2 creating+peering, 61 active+clean, 34 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 1 op/s 2024-06-10T00:52:40.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:39 smithi094 ceph-mon[36708]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-10T00:52:40.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:39 smithi094 ceph-mon[36708]: Cluster is now healthy 2024-06-10T00:52:40.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:39 smithi094 ceph-mon[36708]: mds.? [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] up:active 2024-06-10T00:52:40.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:39 smithi094 ceph-mon[36708]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T00:52:40.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:39 smithi094 ceph-mon[36708]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T00:52:40.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.578+0000 7f5ffd6b2700 1 -- 172.21.15.37:0/2596941884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff8107840 msgr2=0x7f5ff810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:40.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.578+0000 7f5ffd6b2700 1 --2- 172.21.15.37:0/2596941884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff8107840 0x7f5ff810f3c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000b0d0 tx=0x7f5ff000b3e0 comp rx=0 tx=0).stop 2024-06-10T00:52:40.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.578+0000 7f5ffd6b2700 1 -- 172.21.15.37:0/2596941884 shutdown_connections 2024-06-10T00:52:40.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.578+0000 7f5ffd6b2700 1 --2- 172.21.15.37:0/2596941884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff8107840 0x7f5ff810f3c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:40.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.578+0000 7f5ffd6b2700 1 --2- 172.21.15.37:0/2596941884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ff8102740 0x7f5ff8102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:40.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.578+0000 7f5ffd6b2700 1 --2- 172.21.15.37:0/2596941884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ff81030f0 0x7f5ff8107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:40.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.578+0000 7f5ffd6b2700 1 -- 172.21.15.37:0/2596941884 >> 172.21.15.37:0/2596941884 conn(0x7f5ff80795f0 msgr2=0x7f5ff8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:40.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.578+0000 7f5ffd6b2700 1 -- 172.21.15.37:0/2596941884 shutdown_connections 2024-06-10T00:52:40.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.578+0000 7f5ffd6b2700 1 -- 172.21.15.37:0/2596941884 wait complete. 2024-06-10T00:52:40.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.579+0000 7f5ffd6b2700 1 Processor -- start 2024-06-10T00:52:40.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.579+0000 7f5ffd6b2700 1 -- start start 2024-06-10T00:52:40.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.579+0000 7f5ffd6b2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ff8102740 0x7f5ff8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:40.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.579+0000 7f5ffd6b2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff81030f0 0x7f5ff8074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:40.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.579+0000 7f5ffd6b2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ff8107840 0x7f5ff8075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:40.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.579+0000 7f5ffd6b2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff81127d0 con 0x7f5ff81030f0 2024-06-10T00:52:40.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.579+0000 7f5ffd6b2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff8112650 con 0x7f5ff8107840 2024-06-10T00:52:40.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.579+0000 7f5ffd6b2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff8112950 con 0x7f5ff8102740 2024-06-10T00:52:40.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.579+0000 7f5ff67fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff81030f0 0x7f5ff8074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:40.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.579+0000 7f5ff67fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff81030f0 0x7f5ff8074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35782/0 (socket says 172.21.15.37:35782) 2024-06-10T00:52:40.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.579+0000 7f5ff67fc700 1 -- 172.21.15.37:0/913847461 learned_addr learned my addr 172.21.15.37:0/913847461 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:52:40.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.579+0000 7f5ff77fe700 1 --2- 172.21.15.37:0/913847461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ff8107840 0x7f5ff8075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:40.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.579+0000 7f5ff6ffd700 1 --2- 172.21.15.37:0/913847461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ff8102740 0x7f5ff8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:40.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.580+0000 7f5ff67fc700 1 -- 172.21.15.37:0/913847461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ff8102740 msgr2=0x7f5ff8074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:40.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.580+0000 7f5ff67fc700 1 --2- 172.21.15.37:0/913847461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ff8102740 0x7f5ff8074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:40.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.580+0000 7f5ff67fc700 1 -- 172.21.15.37:0/913847461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ff8107840 msgr2=0x7f5ff8075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:40.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.580+0000 7f5ff67fc700 1 --2- 172.21.15.37:0/913847461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ff8107840 0x7f5ff8075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:40.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.580+0000 7f5ff67fc700 1 -- 172.21.15.37:0/913847461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff81a7090 con 0x7f5ff81030f0 2024-06-10T00:52:40.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.580+0000 7f5ff6ffd700 1 --2- 172.21.15.37:0/913847461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ff8102740 0x7f5ff8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:52:40.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.580+0000 7f5ff67fc700 1 --2- 172.21.15.37:0/913847461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff81030f0 0x7f5ff8074cd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00b530 tx=0x7f5fec00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:40.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.580+0000 7f5fdffff700 1 -- 172.21.15.37:0/913847461 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec015020 con 0x7f5ff81030f0 2024-06-10T00:52:40.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.580+0000 7f5ffd6b2700 1 -- 172.21.15.37:0/913847461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff81a7380 con 0x7f5ff81030f0 2024-06-10T00:52:40.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.580+0000 7f5fdffff700 1 -- 172.21.15.37:0/913847461 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fec011d10 con 0x7f5ff81030f0 2024-06-10T00:52:40.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.580+0000 7f5fdffff700 1 -- 172.21.15.37:0/913847461 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec00f2e0 con 0x7f5ff81030f0 2024-06-10T00:52:40.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.580+0000 7f5ffd6b2700 1 -- 172.21.15.37:0/913847461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff81a78c0 con 0x7f5ff81030f0 2024-06-10T00:52:40.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.581+0000 7f5ffd6b2700 1 -- 172.21.15.37:0/913847461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ff8069180 con 0x7f5ff81030f0 2024-06-10T00:52:40.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.582+0000 7f5fdffff700 1 -- 172.21.15.37:0/913847461 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5fec018020 con 0x7f5ff81030f0 2024-06-10T00:52:40.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.583+0000 7f5fdffff700 1 --2- 172.21.15.37:0/913847461 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5fe00a2c20 0x7f5fe00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:40.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.583+0000 7f5ff6ffd700 1 --2- 172.21.15.37:0/913847461 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5fe00a2c20 0x7f5fe00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:40.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.583+0000 7f5fdffff700 1 -- 172.21.15.37:0/913847461 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f5fec0ced30 con 0x7f5ff81030f0 2024-06-10T00:52:40.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.583+0000 7f5ff6ffd700 1 --2- 172.21.15.37:0/913847461 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5fe00a2c20 0x7f5fe00a50e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8005950 tx=0x7f5fe80058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:40.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.587+0000 7f5fdffff700 1 -- 172.21.15.37:0/913847461 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5fec01f020 con 0x7f5ff81030f0 2024-06-10T00:52:40.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.806+0000 7f5ffd6b2700 1 -- 172.21.15.37:0/913847461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f5ff81a7c20 con 0x7f5ff81030f0 2024-06-10T00:52:40.963 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jun 10 00:52:40 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-i[67988]: 2024-06-10T00:52:40.657+0000 7fe3ceb57700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-06-10T00:52:40.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.967+0000 7f5fdffff700 1 -- 172.21.15.37:0/913847461 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v90) ==== 157+0+0 (secure 0 0 0) 0x7f5fec09bde0 con 0x7f5ff81030f0 2024-06-10T00:52:40.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.970+0000 7f5ffd6b2700 1 -- 172.21.15.37:0/913847461 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5fe00a2c20 msgr2=0x7f5fe00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:40.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.970+0000 7f5ffd6b2700 1 --2- 172.21.15.37:0/913847461 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5fe00a2c20 0x7f5fe00a50e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8005950 tx=0x7f5fe80058e0 comp rx=0 tx=0).stop 2024-06-10T00:52:40.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.970+0000 7f5ffd6b2700 1 -- 172.21.15.37:0/913847461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff81030f0 msgr2=0x7f5ff8074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:40.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.970+0000 7f5ffd6b2700 1 --2- 172.21.15.37:0/913847461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff81030f0 0x7f5ff8074cd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00b530 tx=0x7f5fec00b840 comp rx=0 tx=0).stop 2024-06-10T00:52:40.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.970+0000 7f5ffd6b2700 1 -- 172.21.15.37:0/913847461 shutdown_connections 2024-06-10T00:52:40.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.970+0000 7f5ffd6b2700 1 --2- 172.21.15.37:0/913847461 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5fe00a2c20 0x7f5fe00a50e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:40.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.970+0000 7f5ffd6b2700 1 --2- 172.21.15.37:0/913847461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ff8107840 0x7f5ff8075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:40.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.970+0000 7f5ffd6b2700 1 --2- 172.21.15.37:0/913847461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ff8102740 0x7f5ff8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:40.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.970+0000 7f5ffd6b2700 1 --2- 172.21.15.37:0/913847461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff81030f0 0x7f5ff8074cd0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:40.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.970+0000 7f5ffd6b2700 1 -- 172.21.15.37:0/913847461 >> 172.21.15.37:0/913847461 conn(0x7f5ff80795f0 msgr2=0x7f5ff8100410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:40.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.971+0000 7f5ffd6b2700 1 -- 172.21.15.37:0/913847461 shutdown_connections 2024-06-10T00:52:40.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:40.971+0000 7f5ffd6b2700 1 -- 172.21.15.37:0/913847461 wait complete. 2024-06-10T00:52:41.296 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:40 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/913847461' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-10T00:52:41.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:40 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/913847461' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-10T00:52:41.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:40 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/913847461' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-10T00:52:41.700 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-06-10T00:52:41.961 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:52:42.219 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:41 smithi037 ceph-mon[32192]: pgmap v299: 97 pgs: 2 creating+peering, 95 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-06-10T00:52:42.219 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:41 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/913847461' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-10T00:52:42.219 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:41 smithi037 ceph-mon[32192]: osdmap e90: 12 total, 12 up, 12 in 2024-06-10T00:52:42.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:41 smithi167 ceph-mon[36917]: pgmap v299: 97 pgs: 2 creating+peering, 95 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-06-10T00:52:42.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:41 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/913847461' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-10T00:52:42.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:41 smithi167 ceph-mon[36917]: osdmap e90: 12 total, 12 up, 12 in 2024-06-10T00:52:42.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:41 smithi094 ceph-mon[36708]: pgmap v299: 97 pgs: 2 creating+peering, 95 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-06-10T00:52:42.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:41 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/913847461' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-10T00:52:42.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:41 smithi094 ceph-mon[36708]: osdmap e90: 12 total, 12 up, 12 in 2024-06-10T00:52:43.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.534+0000 7f6372e23700 1 -- 172.21.15.37:0/3331237675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f636c107430 msgr2=0x7f636c107810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:43.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.534+0000 7f6372e23700 1 --2- 172.21.15.37:0/3331237675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f636c107430 0x7f636c107810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f635c009990 tx=0x7f635c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:52:43.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.534+0000 7f6372e23700 1 -- 172.21.15.37:0/3331237675 shutdown_connections 2024-06-10T00:52:43.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.534+0000 7f6372e23700 1 --2- 172.21.15.37:0/3331237675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f636c102f80 0x7f636c10f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:43.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.534+0000 7f6372e23700 1 --2- 172.21.15.37:0/3331237675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f636c107430 0x7f636c107810 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:43.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.534+0000 7f6372e23700 1 --2- 172.21.15.37:0/3331237675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f636c1025e0 0x7f636c102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:43.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.534+0000 7f6372e23700 1 -- 172.21.15.37:0/3331237675 >> 172.21.15.37:0/3331237675 conn(0x7f636c079680 msgr2=0x7f636c079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:43.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.534+0000 7f6372e23700 1 -- 172.21.15.37:0/3331237675 shutdown_connections 2024-06-10T00:52:43.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.534+0000 7f6372e23700 1 -- 172.21.15.37:0/3331237675 wait complete. 2024-06-10T00:52:43.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.535+0000 7f6372e23700 1 Processor -- start 2024-06-10T00:52:43.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.535+0000 7f6372e23700 1 -- start start 2024-06-10T00:52:43.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.535+0000 7f6372e23700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f636c1025e0 0x7f636c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:43.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.535+0000 7f6372e23700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f636c102f80 0x7f636c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:43.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.535+0000 7f6372e23700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f636c107430 0x7f636c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:43.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.535+0000 7f6372e23700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f636c112820 con 0x7f636c107430 2024-06-10T00:52:43.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.535+0000 7f6372e23700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f636c1126a0 con 0x7f636c102f80 2024-06-10T00:52:43.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.535+0000 7f6372e23700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f636c1129a0 con 0x7f636c1025e0 2024-06-10T00:52:43.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.535+0000 7f63713c0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f636c107430 0x7f636c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:43.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.535+0000 7f63713c0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f636c107430 0x7f636c075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50012/0 (socket says 172.21.15.37:50012) 2024-06-10T00:52:43.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.535+0000 7f63713c0700 1 -- 172.21.15.37:0/3134880355 learned_addr learned my addr 172.21.15.37:0/3134880355 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:52:43.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.535+0000 7f6370bbf700 1 --2- 172.21.15.37:0/3134880355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f636c1025e0 0x7f636c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:43.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.535+0000 7f636bfff700 1 --2- 172.21.15.37:0/3134880355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f636c102f80 0x7f636c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:43.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.536+0000 7f63713c0700 1 -- 172.21.15.37:0/3134880355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f636c1025e0 msgr2=0x7f636c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:43.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.536+0000 7f63713c0700 1 --2- 172.21.15.37:0/3134880355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f636c1025e0 0x7f636c074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:43.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.536+0000 7f63713c0700 1 -- 172.21.15.37:0/3134880355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f636c102f80 msgr2=0x7f636c074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:43.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.536+0000 7f63713c0700 1 --2- 172.21.15.37:0/3134880355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f636c102f80 0x7f636c074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:43.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.536+0000 7f63713c0700 1 -- 172.21.15.37:0/3134880355 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636c1a7120 con 0x7f636c107430 2024-06-10T00:52:43.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.536+0000 7f63713c0700 1 --2- 172.21.15.37:0/3134880355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f636c107430 0x7f636c075210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6364007ee0 tx=0x7f636400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:43.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.536+0000 7f6369ffb700 1 -- 172.21.15.37:0/3134880355 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636400dbd0 con 0x7f636c107430 2024-06-10T00:52:43.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.536+0000 7f6372e23700 1 -- 172.21.15.37:0/3134880355 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f636c1a7410 con 0x7f636c107430 2024-06-10T00:52:43.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.536+0000 7f6369ffb700 1 -- 172.21.15.37:0/3134880355 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f636400dd70 con 0x7f636c107430 2024-06-10T00:52:43.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.536+0000 7f6369ffb700 1 -- 172.21.15.37:0/3134880355 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63640133f0 con 0x7f636c107430 2024-06-10T00:52:43.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.536+0000 7f6372e23700 1 -- 172.21.15.37:0/3134880355 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f636c1a7950 con 0x7f636c107430 2024-06-10T00:52:43.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.537+0000 7f6372e23700 1 -- 172.21.15.37:0/3134880355 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f636c050a00 con 0x7f636c107430 2024-06-10T00:52:43.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.538+0000 7f6369ffb700 1 -- 172.21.15.37:0/3134880355 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f636401a020 con 0x7f636c107430 2024-06-10T00:52:43.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.539+0000 7f6369ffb700 1 --2- 172.21.15.37:0/3134880355 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f63540a2cc0 0x7f63540a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:43.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.539+0000 7f6370bbf700 1 --2- 172.21.15.37:0/3134880355 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f63540a2cc0 0x7f63540a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:43.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.539+0000 7f6369ffb700 1 -- 172.21.15.37:0/3134880355 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f63640ce8b0 con 0x7f636c107430 2024-06-10T00:52:43.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.540+0000 7f6370bbf700 1 --2- 172.21.15.37:0/3134880355 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f63540a2cc0 0x7f63540a5180 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f635c006010 tx=0x7f635c00b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:43.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.544+0000 7f6369ffb700 1 -- 172.21.15.37:0/3134880355 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6364010070 con 0x7f636c107430 2024-06-10T00:52:43.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:43.750+0000 7f6372e23700 1 -- 172.21.15.37:0/3134880355 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f636c075c20 con 0x7f636c107430 2024-06-10T00:52:44.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:44 smithi167 ceph-mon[36917]: pgmap v301: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-06-10T00:52:44.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:44 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3134880355' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-10T00:52:44.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:44 smithi094 ceph-mon[36708]: pgmap v301: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-06-10T00:52:44.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:44 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3134880355' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-10T00:52:44.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:44 smithi037 ceph-mon[32192]: pgmap v301: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-06-10T00:52:44.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:44 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3134880355' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-10T00:52:44.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:44.974+0000 7f6369ffb700 1 -- 172.21.15.37:0/3134880355 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v92) ==== 228+0+0 (secure 0 0 0) 0x7f636409b780 con 0x7f636c107430 2024-06-10T00:52:44.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:44.977+0000 7f6372e23700 1 -- 172.21.15.37:0/3134880355 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f63540a2cc0 msgr2=0x7f63540a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:44.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:44.977+0000 7f6372e23700 1 --2- 172.21.15.37:0/3134880355 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f63540a2cc0 0x7f63540a5180 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f635c006010 tx=0x7f635c00b540 comp rx=0 tx=0).stop 2024-06-10T00:52:44.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:44.977+0000 7f6372e23700 1 -- 172.21.15.37:0/3134880355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f636c107430 msgr2=0x7f636c075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:44.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:44.977+0000 7f6372e23700 1 --2- 172.21.15.37:0/3134880355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f636c107430 0x7f636c075210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6364007ee0 tx=0x7f636400d3b0 comp rx=0 tx=0).stop 2024-06-10T00:52:44.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:44.977+0000 7f6372e23700 1 -- 172.21.15.37:0/3134880355 shutdown_connections 2024-06-10T00:52:44.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:44.977+0000 7f6372e23700 1 --2- 172.21.15.37:0/3134880355 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f63540a2cc0 0x7f63540a5180 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:44.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:44.977+0000 7f6372e23700 1 --2- 172.21.15.37:0/3134880355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f636c107430 0x7f636c075210 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:44.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:44.977+0000 7f6372e23700 1 --2- 172.21.15.37:0/3134880355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f636c1025e0 0x7f636c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:44.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:44.977+0000 7f6372e23700 1 --2- 172.21.15.37:0/3134880355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f636c102f80 0x7f636c074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:44.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:44.977+0000 7f6372e23700 1 -- 172.21.15.37:0/3134880355 >> 172.21.15.37:0/3134880355 conn(0x7f636c079680 msgr2=0x7f636c100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:44.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:44.977+0000 7f6372e23700 1 -- 172.21.15.37:0/3134880355 shutdown_connections 2024-06-10T00:52:44.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:44.977+0000 7f6372e23700 1 -- 172.21.15.37:0/3134880355 wait complete. 2024-06-10T00:52:44.979 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data_ec' created 2024-06-10T00:52:45.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:45 smithi037 ceph-mon[32192]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-06-10T00:52:45.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:45 smithi037 ceph-mon[32192]: osdmap e91: 12 total, 12 up, 12 in 2024-06-10T00:52:45.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:45 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3134880355' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-10T00:52:45.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:45 smithi037 ceph-mon[32192]: osdmap e92: 12 total, 12 up, 12 in 2024-06-10T00:52:45.717 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-06-10T00:52:45.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:45 smithi167 ceph-mon[36917]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-06-10T00:52:45.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:45 smithi167 ceph-mon[36917]: osdmap e91: 12 total, 12 up, 12 in 2024-06-10T00:52:45.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:45 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3134880355' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-10T00:52:45.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:45 smithi167 ceph-mon[36917]: osdmap e92: 12 total, 12 up, 12 in 2024-06-10T00:52:45.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:45 smithi094 ceph-mon[36708]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-06-10T00:52:45.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:45 smithi094 ceph-mon[36708]: osdmap e91: 12 total, 12 up, 12 in 2024-06-10T00:52:45.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:45 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3134880355' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-10T00:52:45.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:45 smithi094 ceph-mon[36708]: osdmap e92: 12 total, 12 up, 12 in 2024-06-10T00:52:45.977 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:52:47.058 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:46 smithi037 ceph-mon[32192]: osdmap e93: 12 total, 12 up, 12 in 2024-06-10T00:52:47.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:46 smithi167 ceph-mon[36917]: osdmap e93: 12 total, 12 up, 12 in 2024-06-10T00:52:47.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:46 smithi094 ceph-mon[36708]: osdmap e93: 12 total, 12 up, 12 in 2024-06-10T00:52:47.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.616+0000 7f80eef7c700 1 -- 172.21.15.37:0/4292997477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80e8102770 msgr2=0x7f80e8102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:47.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.616+0000 7f80eef7c700 1 --2- 172.21.15.37:0/4292997477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80e8102770 0x7f80e8102b50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f80d800b0d0 tx=0x7f80d800b3e0 comp rx=0 tx=0).stop 2024-06-10T00:52:47.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.616+0000 7f80eef7c700 1 -- 172.21.15.37:0/4292997477 shutdown_connections 2024-06-10T00:52:47.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.616+0000 7f80eef7c700 1 --2- 172.21.15.37:0/4292997477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80e8107860 0x7f80e810f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:47.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.616+0000 7f80eef7c700 1 --2- 172.21.15.37:0/4292997477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80e8102770 0x7f80e8102b50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:47.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.616+0000 7f80eef7c700 1 --2- 172.21.15.37:0/4292997477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80e8103120 0x7f80e8107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:47.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.616+0000 7f80eef7c700 1 -- 172.21.15.37:0/4292997477 >> 172.21.15.37:0/4292997477 conn(0x7f80e8079600 msgr2=0x7f80e8079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:47.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.616+0000 7f80eef7c700 1 -- 172.21.15.37:0/4292997477 shutdown_connections 2024-06-10T00:52:47.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.616+0000 7f80eef7c700 1 -- 172.21.15.37:0/4292997477 wait complete. 2024-06-10T00:52:47.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.617+0000 7f80eef7c700 1 Processor -- start 2024-06-10T00:52:47.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.617+0000 7f80eef7c700 1 -- start start 2024-06-10T00:52:47.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.617+0000 7f80eef7c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80e8102770 0x7f80e819e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:47.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.617+0000 7f80eef7c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80e8103120 0x7f80e819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:47.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.617+0000 7f80eef7c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80e8107860 0x7f80e8198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:47.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.617+0000 7f80eef7c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f80e8112800 con 0x7f80e8107860 2024-06-10T00:52:47.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.617+0000 7f80eef7c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f80e8112680 con 0x7f80e8102770 2024-06-10T00:52:47.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.617+0000 7f80eef7c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f80e8112980 con 0x7f80e8103120 2024-06-10T00:52:47.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.617+0000 7f80ed519700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80e8107860 0x7f80e8198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:47.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.617+0000 7f80ed519700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80e8107860 0x7f80e8198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50026/0 (socket says 172.21.15.37:50026) 2024-06-10T00:52:47.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.617+0000 7f80ed519700 1 -- 172.21.15.37:0/3176968421 learned_addr learned my addr 172.21.15.37:0/3176968421 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:52:47.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.617+0000 7f80ecd18700 1 --2- 172.21.15.37:0/3176968421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80e8102770 0x7f80e819e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:47.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.617+0000 7f80e7fff700 1 --2- 172.21.15.37:0/3176968421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80e8103120 0x7f80e819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:47.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.617+0000 7f80ed519700 1 -- 172.21.15.37:0/3176968421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80e8103120 msgr2=0x7f80e819ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:47.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.617+0000 7f80ed519700 1 --2- 172.21.15.37:0/3176968421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80e8103120 0x7f80e819ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:47.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.617+0000 7f80ed519700 1 -- 172.21.15.37:0/3176968421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80e8102770 msgr2=0x7f80e819e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:47.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.617+0000 7f80ed519700 1 --2- 172.21.15.37:0/3176968421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80e8102770 0x7f80e819e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:47.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.617+0000 7f80ed519700 1 -- 172.21.15.37:0/3176968421 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e81991c0 con 0x7f80e8107860 2024-06-10T00:52:47.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.618+0000 7f80e7fff700 1 --2- 172.21.15.37:0/3176968421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80e8103120 0x7f80e819ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:52:47.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.618+0000 7f80ecd18700 1 --2- 172.21.15.37:0/3176968421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80e8102770 0x7f80e819e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:52:47.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.618+0000 7f80ed519700 1 --2- 172.21.15.37:0/3176968421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80e8107860 0x7f80e8198a60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f80e000b550 tx=0x7f80e000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:47.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.618+0000 7f80e5ffb700 1 -- 172.21.15.37:0/3176968421 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0015020 con 0x7f80e8107860 2024-06-10T00:52:47.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.618+0000 7f80eef7c700 1 -- 172.21.15.37:0/3176968421 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e81994b0 con 0x7f80e8107860 2024-06-10T00:52:47.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.618+0000 7f80e5ffb700 1 -- 172.21.15.37:0/3176968421 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80e0011d10 con 0x7f80e8107860 2024-06-10T00:52:47.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.618+0000 7f80e5ffb700 1 -- 172.21.15.37:0/3176968421 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e000f2e0 con 0x7f80e8107860 2024-06-10T00:52:47.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.618+0000 7f80eef7c700 1 -- 172.21.15.37:0/3176968421 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e81a5720 con 0x7f80e8107860 2024-06-10T00:52:47.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.619+0000 7f80eef7c700 1 -- 172.21.15.37:0/3176968421 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80e810aaa0 con 0x7f80e8107860 2024-06-10T00:52:47.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.620+0000 7f80e5ffb700 1 -- 172.21.15.37:0/3176968421 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f80e0018020 con 0x7f80e8107860 2024-06-10T00:52:47.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.621+0000 7f80e5ffb700 1 --2- 172.21.15.37:0/3176968421 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f80d40a2c20 0x7f80d40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:47.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.621+0000 7f80ecd18700 1 --2- 172.21.15.37:0/3176968421 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f80d40a2c20 0x7f80d40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:47.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.621+0000 7f80e5ffb700 1 -- 172.21.15.37:0/3176968421 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8770+0+0 (secure 0 0 0) 0x7f80e00ceac0 con 0x7f80e8107860 2024-06-10T00:52:47.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.622+0000 7f80ecd18700 1 --2- 172.21.15.37:0/3176968421 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f80d40a2c20 0x7f80d40a50e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f80d8006130 tx=0x7f80d8006000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:47.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.626+0000 7f80e5ffb700 1 -- 172.21.15.37:0/3176968421 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f80e001f020 con 0x7f80e8107860 2024-06-10T00:52:47.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.830+0000 7f80eef7c700 1 -- 172.21.15.37:0/3176968421 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f80e810a480 con 0x7f80e8107860 2024-06-10T00:52:47.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.991+0000 7f80e5ffb700 1 -- 172.21.15.37:0/3176968421 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v95) ==== 169+0+0 (secure 0 0 0) 0x7f80e009b7a0 con 0x7f80e8107860 2024-06-10T00:52:47.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.994+0000 7f80eef7c700 1 -- 172.21.15.37:0/3176968421 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f80d40a2c20 msgr2=0x7f80d40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:47.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.994+0000 7f80eef7c700 1 --2- 172.21.15.37:0/3176968421 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f80d40a2c20 0x7f80d40a50e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f80d8006130 tx=0x7f80d8006000 comp rx=0 tx=0).stop 2024-06-10T00:52:47.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.994+0000 7f80eef7c700 1 -- 172.21.15.37:0/3176968421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80e8107860 msgr2=0x7f80e8198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:47.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.994+0000 7f80eef7c700 1 --2- 172.21.15.37:0/3176968421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80e8107860 0x7f80e8198a60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f80e000b550 tx=0x7f80e000b910 comp rx=0 tx=0).stop 2024-06-10T00:52:47.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.994+0000 7f80eef7c700 1 -- 172.21.15.37:0/3176968421 shutdown_connections 2024-06-10T00:52:47.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.994+0000 7f80eef7c700 1 --2- 172.21.15.37:0/3176968421 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f80d40a2c20 0x7f80d40a50e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:47.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.994+0000 7f80eef7c700 1 --2- 172.21.15.37:0/3176968421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80e8107860 0x7f80e8198a60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:47.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.994+0000 7f80eef7c700 1 --2- 172.21.15.37:0/3176968421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80e8102770 0x7f80e819e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:47.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.994+0000 7f80eef7c700 1 --2- 172.21.15.37:0/3176968421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80e8103120 0x7f80e819ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:47.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.994+0000 7f80eef7c700 1 -- 172.21.15.37:0/3176968421 >> 172.21.15.37:0/3176968421 conn(0x7f80e8079600 msgr2=0x7f80e81006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:47.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.994+0000 7f80eef7c700 1 -- 172.21.15.37:0/3176968421 shutdown_connections 2024-06-10T00:52:47.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:47.994+0000 7f80eef7c700 1 -- 172.21.15.37:0/3176968421 wait complete. 2024-06-10T00:52:47.996 INFO:teuthology.orchestra.run.smithi037.stderr:set pool 4 allow_ec_overwrites to true 2024-06-10T00:52:48.307 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:47 smithi037 ceph-mon[32192]: pgmap v306: 129 pgs: 4 creating+peering, 28 unknown, 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 952 B/s wr, 2 op/s 2024-06-10T00:52:48.307 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:47 smithi037 ceph-mon[32192]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-10T00:52:48.307 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:47 smithi037 ceph-mon[32192]: osdmap e94: 12 total, 12 up, 12 in 2024-06-10T00:52:48.307 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:47 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3176968421' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-10T00:52:48.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:47 smithi167 ceph-mon[36917]: pgmap v306: 129 pgs: 4 creating+peering, 28 unknown, 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 952 B/s wr, 2 op/s 2024-06-10T00:52:48.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:47 smithi167 ceph-mon[36917]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-10T00:52:48.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:47 smithi167 ceph-mon[36917]: osdmap e94: 12 total, 12 up, 12 in 2024-06-10T00:52:48.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:47 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3176968421' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-10T00:52:48.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:47 smithi094 ceph-mon[36708]: pgmap v306: 129 pgs: 4 creating+peering, 28 unknown, 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 952 B/s wr, 2 op/s 2024-06-10T00:52:48.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:47 smithi094 ceph-mon[36708]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-10T00:52:48.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:47 smithi094 ceph-mon[36708]: osdmap e94: 12 total, 12 up, 12 in 2024-06-10T00:52:48.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:47 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3176968421' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-10T00:52:48.631 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-06-10T00:52:48.885 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:52:49.304 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:48 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3176968421' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-10T00:52:49.304 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:48 smithi037 ceph-mon[32192]: osdmap e95: 12 total, 12 up, 12 in 2024-06-10T00:52:49.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:48 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3176968421' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-10T00:52:49.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:48 smithi167 ceph-mon[36917]: osdmap e95: 12 total, 12 up, 12 in 2024-06-10T00:52:49.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:48 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3176968421' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-10T00:52:49.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:48 smithi094 ceph-mon[36708]: osdmap e95: 12 total, 12 up, 12 in 2024-06-10T00:52:50.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:50 smithi167 ceph-mon[36917]: pgmap v309: 129 pgs: 4 creating+peering, 7 unknown, 118 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:50.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:50 smithi094 ceph-mon[36708]: pgmap v309: 129 pgs: 4 creating+peering, 7 unknown, 118 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:50.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:50 smithi037 ceph-mon[32192]: pgmap v309: 129 pgs: 4 creating+peering, 7 unknown, 118 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:50.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.594+0000 7f25593c0700 1 -- 172.21.15.37:0/968553970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2554100a80 msgr2=0x7f2554100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:50.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.594+0000 7f25593c0700 1 --2- 172.21.15.37:0/968553970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2554100a80 0x7f2554100f00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f253c009990 tx=0x7f253c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:52:50.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.594+0000 7f25593c0700 1 -- 172.21.15.37:0/968553970 shutdown_connections 2024-06-10T00:52:50.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.594+0000 7f25593c0700 1 --2- 172.21.15.37:0/968553970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2554101440 0x7f255410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:50.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.594+0000 7f25593c0700 1 --2- 172.21.15.37:0/968553970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25541086f0 0x7f2554108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:50.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.594+0000 7f25593c0700 1 --2- 172.21.15.37:0/968553970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2554100a80 0x7f2554100f00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:50.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.594+0000 7f25593c0700 1 -- 172.21.15.37:0/968553970 >> 172.21.15.37:0/968553970 conn(0x7f2554077a00 msgr2=0x7f25540fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:50.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.594+0000 7f25593c0700 1 -- 172.21.15.37:0/968553970 shutdown_connections 2024-06-10T00:52:50.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.594+0000 7f25593c0700 1 -- 172.21.15.37:0/968553970 wait complete. 2024-06-10T00:52:50.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.594+0000 7f25593c0700 1 Processor -- start 2024-06-10T00:52:50.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.595+0000 7f25593c0700 1 -- start start 2024-06-10T00:52:50.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.595+0000 7f25593c0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2554100a80 0x7f2554198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:50.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.595+0000 7f25593c0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2554101440 0x7f2554199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:50.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.595+0000 7f25593c0700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25541086f0 0x7f255419d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:50.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.595+0000 7f25593c0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2554112a10 con 0x7f2554100a80 2024-06-10T00:52:50.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.595+0000 7f25593c0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2554112890 con 0x7f2554101440 2024-06-10T00:52:50.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.595+0000 7f25593c0700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2554112b90 con 0x7f25541086f0 2024-06-10T00:52:50.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.595+0000 7f2552ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2554100a80 0x7f2554198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:50.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.595+0000 7f2552ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2554100a80 0x7f2554198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50040/0 (socket says 172.21.15.37:50040) 2024-06-10T00:52:50.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.595+0000 7f2552ffd700 1 -- 172.21.15.37:0/2691806346 learned_addr learned my addr 172.21.15.37:0/2691806346 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:52:50.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.595+0000 7f25537fe700 1 --2- 172.21.15.37:0/2691806346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25541086f0 0x7f255419d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:50.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.595+0000 7f25527fc700 1 --2- 172.21.15.37:0/2691806346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2554101440 0x7f2554199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:50.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.595+0000 7f2552ffd700 1 -- 172.21.15.37:0/2691806346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25541086f0 msgr2=0x7f255419d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:50.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.595+0000 7f2552ffd700 1 --2- 172.21.15.37:0/2691806346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25541086f0 0x7f255419d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:50.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.595+0000 7f2552ffd700 1 -- 172.21.15.37:0/2691806346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2554101440 msgr2=0x7f2554199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:50.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.595+0000 7f2552ffd700 1 --2- 172.21.15.37:0/2691806346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2554101440 0x7f2554199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:50.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.595+0000 7f2552ffd700 1 -- 172.21.15.37:0/2691806346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f255419ded0 con 0x7f2554100a80 2024-06-10T00:52:50.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.595+0000 7f25537fe700 1 --2- 172.21.15.37:0/2691806346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25541086f0 0x7f255419d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:52:50.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.596+0000 7f2552ffd700 1 --2- 172.21.15.37:0/2691806346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2554100a80 0x7f2554198ec0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f254400b9c0 tx=0x7f254400bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:50.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.596+0000 7f254bfff700 1 -- 172.21.15.37:0/2691806346 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2544019070 con 0x7f2554100a80 2024-06-10T00:52:50.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.596+0000 7f254bfff700 1 -- 172.21.15.37:0/2691806346 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2544014440 con 0x7f2554100a80 2024-06-10T00:52:50.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.596+0000 7f25593c0700 1 -- 172.21.15.37:0/2691806346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f255419e1c0 con 0x7f2554100a80 2024-06-10T00:52:50.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.596+0000 7f254bfff700 1 -- 172.21.15.37:0/2691806346 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25440125a0 con 0x7f2554100a80 2024-06-10T00:52:50.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.596+0000 7f25593c0700 1 -- 172.21.15.37:0/2691806346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25541a5a40 con 0x7f2554100a80 2024-06-10T00:52:50.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.597+0000 7f25593c0700 1 -- 172.21.15.37:0/2691806346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2554069180 con 0x7f2554100a80 2024-06-10T00:52:50.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.599+0000 7f254bfff700 1 -- 172.21.15.37:0/2691806346 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f254401c070 con 0x7f2554100a80 2024-06-10T00:52:50.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.599+0000 7f254bfff700 1 --2- 172.21.15.37:0/2691806346 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f25400a2c20 0x7f25400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:50.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.600+0000 7f25527fc700 1 --2- 172.21.15.37:0/2691806346 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f25400a2c20 0x7f25400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:50.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.600+0000 7f254bfff700 1 -- 172.21.15.37:0/2691806346 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8770+0+0 (secure 0 0 0) 0x7f25440cebf0 con 0x7f2554100a80 2024-06-10T00:52:50.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.600+0000 7f25527fc700 1 --2- 172.21.15.37:0/2691806346 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f25400a2c20 0x7f25400a50e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f253c005cd0 tx=0x7f253c005c40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:50.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.603+0000 7f254bfff700 1 -- 172.21.15.37:0/2691806346 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f254409b8d0 con 0x7f2554100a80 2024-06-10T00:52:50.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:50.856+0000 7f25593c0700 1 -- 172.21.15.37:0/2691806346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f2554069f50 con 0x7f2554100a80 2024-06-10T00:52:51.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:51.010+0000 7f254bfff700 1 -- 172.21.15.37:0/2691806346 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f254409b1d0 con 0x7f2554100a80 2024-06-10T00:52:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:51.012+0000 7f25593c0700 1 -- 172.21.15.37:0/2691806346 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f25400a2c20 msgr2=0x7f25400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:51.012+0000 7f25593c0700 1 --2- 172.21.15.37:0/2691806346 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f25400a2c20 0x7f25400a50e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f253c005cd0 tx=0x7f253c005c40 comp rx=0 tx=0).stop 2024-06-10T00:52:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:51.012+0000 7f25593c0700 1 -- 172.21.15.37:0/2691806346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2554100a80 msgr2=0x7f2554198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:51.012+0000 7f25593c0700 1 --2- 172.21.15.37:0/2691806346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2554100a80 0x7f2554198ec0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f254400b9c0 tx=0x7f254400bd80 comp rx=0 tx=0).stop 2024-06-10T00:52:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:51.012+0000 7f25593c0700 1 -- 172.21.15.37:0/2691806346 shutdown_connections 2024-06-10T00:52:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:51.012+0000 7f25593c0700 1 --2- 172.21.15.37:0/2691806346 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f25400a2c20 0x7f25400a50e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:51.012+0000 7f25593c0700 1 --2- 172.21.15.37:0/2691806346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25541086f0 0x7f255419d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:51.012+0000 7f25593c0700 1 --2- 172.21.15.37:0/2691806346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2554100a80 0x7f2554198ec0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:51.012+0000 7f25593c0700 1 --2- 172.21.15.37:0/2691806346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2554101440 0x7f2554199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:51.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:51.012+0000 7f25593c0700 1 -- 172.21.15.37:0/2691806346 >> 172.21.15.37:0/2691806346 conn(0x7f2554077a00 msgr2=0x7f25540fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:51.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:51.012+0000 7f25593c0700 1 -- 172.21.15.37:0/2691806346 shutdown_connections 2024-06-10T00:52:51.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:51.012+0000 7f25593c0700 1 -- 172.21.15.37:0/2691806346 wait complete. 2024-06-10T00:52:51.014 INFO:teuthology.orchestra.run.smithi037.stderr:added data pool 4 to fsmap 2024-06-10T00:52:51.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:51 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2691806346' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-10T00:52:51.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:51 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2691806346' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-10T00:52:51.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:51 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2691806346' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-10T00:52:51.654 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:52:51.904 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:52:52.261 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:52 smithi037 ceph-mon[32192]: pgmap v310: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:52.261 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:52 smithi037 ceph-mon[32192]: osdmap e96: 12 total, 12 up, 12 in 2024-06-10T00:52:52.261 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:52 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2691806346' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-10T00:52:52.261 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:52 smithi037 ceph-mon[32192]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T00:52:52.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:52 smithi167 ceph-mon[36917]: pgmap v310: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:52.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:52 smithi167 ceph-mon[36917]: osdmap e96: 12 total, 12 up, 12 in 2024-06-10T00:52:52.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:52 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2691806346' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-10T00:52:52.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:52 smithi167 ceph-mon[36917]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T00:52:52.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:52 smithi094 ceph-mon[36708]: pgmap v310: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:52.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:52 smithi094 ceph-mon[36708]: osdmap e96: 12 total, 12 up, 12 in 2024-06-10T00:52:52.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:52 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2691806346' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-10T00:52:52.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:52 smithi094 ceph-mon[36708]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T00:52:53.306 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:53 smithi037 ceph-mon[32192]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-10T00:52:53.307 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:53 smithi037 ceph-mon[32192]: Cluster is now healthy 2024-06-10T00:52:53.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:53 smithi167 ceph-mon[36917]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-10T00:52:53.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:53 smithi167 ceph-mon[36917]: Cluster is now healthy 2024-06-10T00:52:53.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:53 smithi094 ceph-mon[36708]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-10T00:52:53.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:53 smithi094 ceph-mon[36708]: Cluster is now healthy 2024-06-10T00:52:53.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.543+0000 7fd0cb3b3700 1 -- 172.21.15.37:0/1859415350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0c4107840 msgr2=0x7fd0c410f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:53.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.543+0000 7fd0cb3b3700 1 --2- 172.21.15.37:0/1859415350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0c4107840 0x7fd0c410f3c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc00b0d0 tx=0x7fd0bc00b3e0 comp rx=0 tx=0).stop 2024-06-10T00:52:53.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.544+0000 7fd0cb3b3700 1 -- 172.21.15.37:0/1859415350 shutdown_connections 2024-06-10T00:52:53.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.544+0000 7fd0cb3b3700 1 --2- 172.21.15.37:0/1859415350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0c4107840 0x7fd0c410f3c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:53.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.544+0000 7fd0cb3b3700 1 --2- 172.21.15.37:0/1859415350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0c4102740 0x7fd0c4102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:53.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.544+0000 7fd0cb3b3700 1 --2- 172.21.15.37:0/1859415350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0c41030f0 0x7fd0c4107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:53.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.544+0000 7fd0cb3b3700 1 -- 172.21.15.37:0/1859415350 >> 172.21.15.37:0/1859415350 conn(0x7fd0c40795f0 msgr2=0x7fd0c4079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:53.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.544+0000 7fd0cb3b3700 1 -- 172.21.15.37:0/1859415350 shutdown_connections 2024-06-10T00:52:53.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.544+0000 7fd0cb3b3700 1 -- 172.21.15.37:0/1859415350 wait complete. 2024-06-10T00:52:53.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.544+0000 7fd0cb3b3700 1 Processor -- start 2024-06-10T00:52:53.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.544+0000 7fd0cb3b3700 1 -- start start 2024-06-10T00:52:53.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.545+0000 7fd0cb3b3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0c4102740 0x7fd0c4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:53.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.545+0000 7fd0cb3b3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0c41030f0 0x7fd0c4074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:53.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.545+0000 7fd0cb3b3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0c4107840 0x7fd0c4075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:53.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.545+0000 7fd0cb3b3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c4112780 con 0x7fd0c4107840 2024-06-10T00:52:53.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.545+0000 7fd0cb3b3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c4112600 con 0x7fd0c4102740 2024-06-10T00:52:53.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.545+0000 7fd0cb3b3700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c4112900 con 0x7fd0c41030f0 2024-06-10T00:52:53.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.545+0000 7fd0c9950700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0c4107840 0x7fd0c4075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:53.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.545+0000 7fd0c9950700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0c4107840 0x7fd0c4075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46516/0 (socket says 172.21.15.37:46516) 2024-06-10T00:52:53.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.545+0000 7fd0c9950700 1 -- 172.21.15.37:0/637932881 learned_addr learned my addr 172.21.15.37:0/637932881 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:52:53.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.545+0000 7fd0c914f700 1 --2- 172.21.15.37:0/637932881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0c4102740 0x7fd0c4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:53.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.545+0000 7fd0c894e700 1 --2- 172.21.15.37:0/637932881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0c41030f0 0x7fd0c4074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:53.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.545+0000 7fd0c9950700 1 -- 172.21.15.37:0/637932881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0c41030f0 msgr2=0x7fd0c4074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:53.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.545+0000 7fd0c9950700 1 --2- 172.21.15.37:0/637932881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0c41030f0 0x7fd0c4074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:53.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.545+0000 7fd0c9950700 1 -- 172.21.15.37:0/637932881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0c4102740 msgr2=0x7fd0c4074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:53.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.545+0000 7fd0c9950700 1 --2- 172.21.15.37:0/637932881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0c4102740 0x7fd0c4074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:53.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.545+0000 7fd0c9950700 1 -- 172.21.15.37:0/637932881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c41a7060 con 0x7fd0c4107840 2024-06-10T00:52:53.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.545+0000 7fd0c9950700 1 --2- 172.21.15.37:0/637932881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0c4107840 0x7fd0c4075210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc006070 tx=0x7fd0bc004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:53.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.545+0000 7fd0ba7fc700 1 -- 172.21.15.37:0/637932881 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc021070 con 0x7fd0c4107840 2024-06-10T00:52:53.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.545+0000 7fd0ba7fc700 1 -- 172.21.15.37:0/637932881 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0bc007b80 con 0x7fd0c4107840 2024-06-10T00:52:53.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.545+0000 7fd0cb3b3700 1 -- 172.21.15.37:0/637932881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0c41a72f0 con 0x7fd0c4107840 2024-06-10T00:52:53.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.545+0000 7fd0ba7fc700 1 -- 172.21.15.37:0/637932881 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc0052a0 con 0x7fd0c4107840 2024-06-10T00:52:53.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.546+0000 7fd0cb3b3700 1 -- 172.21.15.37:0/637932881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0c41a77f0 con 0x7fd0c4107840 2024-06-10T00:52:53.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.546+0000 7fd0cb3b3700 1 -- 172.21.15.37:0/637932881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0c4069180 con 0x7fd0c4107840 2024-06-10T00:52:53.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.548+0000 7fd0ba7fc700 1 -- 172.21.15.37:0/637932881 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd0bc024070 con 0x7fd0c4107840 2024-06-10T00:52:53.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.548+0000 7fd0ba7fc700 1 --2- 172.21.15.37:0/637932881 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd0b00a2ca0 0x7fd0b00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:53.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.549+0000 7fd0c914f700 1 --2- 172.21.15.37:0/637932881 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd0b00a2ca0 0x7fd0b00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:53.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.549+0000 7fd0ba7fc700 1 -- 172.21.15.37:0/637932881 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7fd0bc00f020 con 0x7fd0c4107840 2024-06-10T00:52:53.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.549+0000 7fd0c914f700 1 --2- 172.21.15.37:0/637932881 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd0b00a2ca0 0x7fd0b00a5160 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0005950 tx=0x7fd0c000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:53.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.552+0000 7fd0ba7fc700 1 -- 172.21.15.37:0/637932881 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd0bc02b020 con 0x7fd0c4107840 2024-06-10T00:52:53.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.807+0000 7fd0cb3b3700 1 -- 172.21.15.37:0/637932881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0c41a7b30 con 0x7fd0c4107840 2024-06-10T00:52:53.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.808+0000 7fd0ba7fc700 1 -- 172.21.15.37:0/637932881 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11794 (secure 0 0 0) 0x7fd0bc09dab0 con 0x7fd0c4107840 2024-06-10T00:52:53.808 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:52:53.809 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24431,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T00:52:50.857957+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24472,"qdb_cluster":[24472]},"id":1}]} 2024-06-10T00:52:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.810+0000 7fd0cb3b3700 1 -- 172.21.15.37:0/637932881 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd0b00a2ca0 msgr2=0x7fd0b00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.810+0000 7fd0cb3b3700 1 --2- 172.21.15.37:0/637932881 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd0b00a2ca0 0x7fd0b00a5160 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0005950 tx=0x7fd0c000b410 comp rx=0 tx=0).stop 2024-06-10T00:52:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.810+0000 7fd0cb3b3700 1 -- 172.21.15.37:0/637932881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0c4107840 msgr2=0x7fd0c4075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.810+0000 7fd0cb3b3700 1 --2- 172.21.15.37:0/637932881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0c4107840 0x7fd0c4075210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc006070 tx=0x7fd0bc004d00 comp rx=0 tx=0).stop 2024-06-10T00:52:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.810+0000 7fd0cb3b3700 1 -- 172.21.15.37:0/637932881 shutdown_connections 2024-06-10T00:52:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.810+0000 7fd0cb3b3700 1 --2- 172.21.15.37:0/637932881 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd0b00a2ca0 0x7fd0b00a5160 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.810+0000 7fd0cb3b3700 1 --2- 172.21.15.37:0/637932881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0c4107840 0x7fd0c4075210 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.810+0000 7fd0cb3b3700 1 --2- 172.21.15.37:0/637932881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0c4102740 0x7fd0c4074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.810+0000 7fd0cb3b3700 1 --2- 172.21.15.37:0/637932881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0c41030f0 0x7fd0c4074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.810+0000 7fd0cb3b3700 1 -- 172.21.15.37:0/637932881 >> 172.21.15.37:0/637932881 conn(0x7fd0c40795f0 msgr2=0x7fd0c4100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.810+0000 7fd0cb3b3700 1 -- 172.21.15.37:0/637932881 shutdown_connections 2024-06-10T00:52:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:53.810+0000 7fd0cb3b3700 1 -- 172.21.15.37:0/637932881 wait complete. 2024-06-10T00:52:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 17 2024-06-10T00:52:54.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:54 smithi167 ceph-mon[36917]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:54.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:54 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/637932881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:52:54.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:54 smithi094 ceph-mon[36708]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:54.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:54 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/637932881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:52:54.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:54 smithi037 ceph-mon[32192]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:54.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:54 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/637932881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:52:54.634 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T00:52:54.889 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:52:56.284 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:56 smithi037 ceph-mon[32192]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:56.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:56 smithi167 ceph-mon[36917]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:56.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:56 smithi094 ceph-mon[36708]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:56.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.662+0000 7ff9192c3700 1 -- 172.21.15.37:0/172691668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff914102510 msgr2=0x7ff914102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:56.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.662+0000 7ff9192c3700 1 --2- 172.21.15.37:0/172691668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff914102510 0x7ff914102990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff908009990 tx=0x7ff908009ca0 comp rx=0 tx=0).stop 2024-06-10T00:52:56.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.662+0000 7ff9192c3700 1 -- 172.21.15.37:0/172691668 shutdown_connections 2024-06-10T00:52:56.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.662+0000 7ff9192c3700 1 --2- 172.21.15.37:0/172691668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff914102ed0 0x7ff91410f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:56.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.662+0000 7ff9192c3700 1 --2- 172.21.15.37:0/172691668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff914108510 0x7ff9141088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:56.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.662+0000 7ff9192c3700 1 --2- 172.21.15.37:0/172691668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff914102510 0x7ff914102990 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:56.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.662+0000 7ff9192c3700 1 -- 172.21.15.37:0/172691668 >> 172.21.15.37:0/172691668 conn(0x7ff9140795b0 msgr2=0x7ff9140799c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:56.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.662+0000 7ff9192c3700 1 -- 172.21.15.37:0/172691668 shutdown_connections 2024-06-10T00:52:56.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.662+0000 7ff9192c3700 1 -- 172.21.15.37:0/172691668 wait complete. 2024-06-10T00:52:56.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.663+0000 7ff9192c3700 1 Processor -- start 2024-06-10T00:52:56.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.663+0000 7ff9192c3700 1 -- start start 2024-06-10T00:52:56.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.663+0000 7ff9192c3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff914102510 0x7ff91419e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:56.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.663+0000 7ff9192c3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff914102ed0 0x7ff91419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:56.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.663+0000 7ff9192c3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff914108510 0x7ff914198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:56.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.663+0000 7ff9192c3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff9141127e0 con 0x7ff914102ed0 2024-06-10T00:52:56.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.663+0000 7ff9192c3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff914112660 con 0x7ff914102510 2024-06-10T00:52:56.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.663+0000 7ff9192c3700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff914112960 con 0x7ff914108510 2024-06-10T00:52:56.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.663+0000 7ff9127fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff914102ed0 0x7ff91419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:56.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.663+0000 7ff9127fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff914102ed0 0x7ff91419ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46534/0 (socket says 172.21.15.37:46534) 2024-06-10T00:52:56.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.663+0000 7ff9127fc700 1 -- 172.21.15.37:0/3471769214 learned_addr learned my addr 172.21.15.37:0/3471769214 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:52:56.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.664+0000 7ff9137fe700 1 --2- 172.21.15.37:0/3471769214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff914108510 0x7ff914198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:56.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.664+0000 7ff912ffd700 1 --2- 172.21.15.37:0/3471769214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff914102510 0x7ff91419e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:56.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.664+0000 7ff9127fc700 1 -- 172.21.15.37:0/3471769214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff914108510 msgr2=0x7ff914198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:56.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.664+0000 7ff9127fc700 1 --2- 172.21.15.37:0/3471769214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff914108510 0x7ff914198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:56.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.664+0000 7ff9127fc700 1 -- 172.21.15.37:0/3471769214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff914102510 msgr2=0x7ff91419e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:56.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.664+0000 7ff9127fc700 1 --2- 172.21.15.37:0/3471769214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff914102510 0x7ff91419e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:56.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.664+0000 7ff9127fc700 1 -- 172.21.15.37:0/3471769214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff914199260 con 0x7ff914102ed0 2024-06-10T00:52:56.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.664+0000 7ff912ffd700 1 --2- 172.21.15.37:0/3471769214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff914102510 0x7ff91419e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:52:56.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.664+0000 7ff9137fe700 1 --2- 172.21.15.37:0/3471769214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff914108510 0x7ff914198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:52:56.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.664+0000 7ff9127fc700 1 --2- 172.21.15.37:0/3471769214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff914102ed0 0x7ff91419ee30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff9080092e0 tx=0x7ff9080037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:56.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.664+0000 7ff8fbfff700 1 -- 172.21.15.37:0/3471769214 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff908003d10 con 0x7ff914102ed0 2024-06-10T00:52:56.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.664+0000 7ff8fbfff700 1 -- 172.21.15.37:0/3471769214 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff908004830 con 0x7ff914102ed0 2024-06-10T00:52:56.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.664+0000 7ff9192c3700 1 -- 172.21.15.37:0/3471769214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9141994f0 con 0x7ff914102ed0 2024-06-10T00:52:56.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.664+0000 7ff8fbfff700 1 -- 172.21.15.37:0/3471769214 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff908005580 con 0x7ff914102ed0 2024-06-10T00:52:56.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.664+0000 7ff9192c3700 1 -- 172.21.15.37:0/3471769214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9141a57c0 con 0x7ff914102ed0 2024-06-10T00:52:56.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.665+0000 7ff9192c3700 1 -- 172.21.15.37:0/3471769214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff91410aa10 con 0x7ff914102ed0 2024-06-10T00:52:56.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.666+0000 7ff8fbfff700 1 -- 172.21.15.37:0/3471769214 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff908029060 con 0x7ff914102ed0 2024-06-10T00:52:56.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.667+0000 7ff8fbfff700 1 --2- 172.21.15.37:0/3471769214 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff8fc0a2c20 0x7ff8fc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:56.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.667+0000 7ff912ffd700 1 --2- 172.21.15.37:0/3471769214 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff8fc0a2c20 0x7ff8fc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:56.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.667+0000 7ff8fbfff700 1 -- 172.21.15.37:0/3471769214 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7ff9080d0030 con 0x7ff914102ed0 2024-06-10T00:52:56.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.667+0000 7ff912ffd700 1 --2- 172.21.15.37:0/3471769214 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff8fc0a2c20 0x7ff8fc0a50e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff914107d30 tx=0x7ff904008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:56.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.672+0000 7ff8fbfff700 1 -- 172.21.15.37:0/3471769214 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff908013070 con 0x7ff914102ed0 2024-06-10T00:52:56.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.882+0000 7ff9192c3700 1 -- 172.21.15.37:0/3471769214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff914199fe0 con 0x7ff914102ed0 2024-06-10T00:52:56.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.884+0000 7ff8fbfff700 1 -- 172.21.15.37:0/3471769214 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24093 (secure 0 0 0) 0x7ff90809cde0 con 0x7ff914102ed0 2024-06-10T00:52:56.885 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:52:56.885 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":96,"fsid":"68793514-26c2-11ef-bc9b-c7b262605968","created":"2024-06-10T00:44:24.106713+0000","modified":"2024-06-10T00:52:50.857814+0000","last_up_change":"2024-06-10T00:51:31.562156+0000","last_in_change":"2024-06-10T00:51:11.294301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:47:44.773911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:52:31.684780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:52:34.885503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:52:44.415437+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1df6046a-6801-4950-9193-54303fed169b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6803","nonce":3594704094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6805","nonce":3594704094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6809","nonce":3594704094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6807","nonce":3594704094}]},"public_addr":"172.21.15.37:6803/3594704094","cluster_addr":"172.21.15.37:6805/3594704094","heartbeat_back_addr":"172.21.15.37:6809/3594704094","heartbeat_front_addr":"172.21.15.37:6807/3594704094","state":["exists","up"]},{"osd":1,"uuid":"4e1c8944-94e4-4e46-9efe-56229fe6df1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6801","nonce":3427727155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6803","nonce":3427727155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6807","nonce":3427727155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6805","nonce":3427727155}]},"public_addr":"172.21.15.94:6801/3427727155","cluster_addr":"172.21.15.94:6803/3427727155","heartbeat_back_addr":"172.21.15.94:6807/3427727155","heartbeat_front_addr":"172.21.15.94:6805/3427727155","state":["exists","up"]},{"osd":2,"uuid":"fd2fcedc-4c88-478f-8025-13e5944a27ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6801","nonce":4002498292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6803","nonce":4002498292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6807","nonce":4002498292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6805","nonce":4002498292}]},"public_addr":"172.21.15.167:6801/4002498292","cluster_addr":"172.21.15.167:6803/4002498292","heartbeat_back_addr":"172.21.15.167:6807/4002498292","heartbeat_front_addr":"172.21.15.167:6805/4002498292","state":["exists","up"]},{"osd":3,"uuid":"d0153676-818c-4dcc-83b1-287435a0c8a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6811","nonce":3425469324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6813","nonce":3425469324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6817","nonce":3425469324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6815","nonce":3425469324}]},"public_addr":"172.21.15.37:6811/3425469324","cluster_addr":"172.21.15.37:6813/3425469324","heartbeat_back_addr":"172.21.15.37:6817/3425469324","heartbeat_front_addr":"172.21.15.37:6815/3425469324","state":["exists","up"]},{"osd":4,"uuid":"172c333c-00c0-4e94-8870-acb3b32b2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6809","nonce":1740627431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6811","nonce":1740627431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6815","nonce":1740627431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6813","nonce":1740627431}]},"public_addr":"172.21.15.94:6809/1740627431","cluster_addr":"172.21.15.94:6811/1740627431","heartbeat_back_addr":"172.21.15.94:6815/1740627431","heartbeat_front_addr":"172.21.15.94:6813/1740627431","state":["exists","up"]},{"osd":5,"uuid":"dbce656e-5f93-4bda-baf2-fc3cc03d8282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6809","nonce":221805236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6811","nonce":221805236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6815","nonce":221805236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6813","nonce":221805236}]},"public_addr":"172.21.15.167:6809/221805236","cluster_addr":"172.21.15.167:6811/221805236","heartbeat_back_addr":"172.21.15.167:6815/221805236","heartbeat_front_addr":"172.21.15.167:6813/221805236","state":["exists","up"]},{"osd":6,"uuid":"fcbda266-6d60-43b2-a51c-4e562476d1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6819","nonce":472670875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6821","nonce":472670875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6825","nonce":472670875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6823","nonce":472670875}]},"public_addr":"172.21.15.37:6819/472670875","cluster_addr":"172.21.15.37:6821/472670875","heartbeat_back_addr":"172.21.15.37:6825/472670875","heartbeat_front_addr":"172.21.15.37:6823/472670875","state":["exists","up"]},{"osd":7,"uuid":"ed829ed3-ba85-4e82-ac37-e44e9534d37c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6817","nonce":3932281552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6819","nonce":3932281552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6823","nonce":3932281552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6821","nonce":3932281552}]},"public_addr":"172.21.15.94:6817/3932281552","cluster_addr":"172.21.15.94:6819/3932281552","heartbeat_back_addr":"172.21.15.94:6823/3932281552","heartbeat_front_addr":"172.21.15.94:6821/3932281552","state":["exists","up"]},{"osd":8,"uuid":"7d6267d8-72f4-4704-b554-bafc52d11a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6817","nonce":2290216047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6819","nonce":2290216047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6823","nonce":2290216047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6821","nonce":2290216047}]},"public_addr":"172.21.15.167:6817/2290216047","cluster_addr":"172.21.15.167:6819/2290216047","heartbeat_back_addr":"172.21.15.167:6823/2290216047","heartbeat_front_addr":"172.21.15.167:6821/2290216047","state":["exists","up"]},{"osd":9,"uuid":"210fdc20-0a36-4eeb-bdf2-c0a7f130e19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6827","nonce":2063666428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6829","nonce":2063666428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6833","nonce":2063666428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6831","nonce":2063666428}]},"public_addr":"172.21.15.37:6827/2063666428","cluster_addr":"172.21.15.37:6829/2063666428","heartbeat_back_addr":"172.21.15.37:6833/2063666428","heartbeat_front_addr":"172.21.15.37:6831/2063666428","state":["exists","up"]},{"osd":10,"uuid":"e200b371-b3cf-4b92-a659-f155ca86f0a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6825","nonce":1706618899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6827","nonce":1706618899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6831","nonce":1706618899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6829","nonce":1706618899}]},"public_addr":"172.21.15.94:6825/1706618899","cluster_addr":"172.21.15.94:6827/1706618899","heartbeat_back_addr":"172.21.15.94:6831/1706618899","heartbeat_front_addr":"172.21.15.94:6829/1706618899","state":["exists","up"]},{"osd":11,"uuid":"cfa427df-37a5-4d28-ac2e-af777d607008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6825","nonce":1349892516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6827","nonce":1349892516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6831","nonce":1349892516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6829","nonce":1349892516}]},"public_addr":"172.21.15.167:6825/1349892516","cluster_addr":"172.21.15.167:6827/1349892516","heartbeat_back_addr":"172.21.15.167:6831/1349892516","heartbeat_front_addr":"172.21.15.167:6829/1349892516","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:46:51.846147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:15.763723+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:40.106800+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:05.923204+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:30.501883+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:55.565575+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:20.411360+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:46.071776+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:11.854599+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:37.608818+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:03.176767+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:29.768705+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/4084722397":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/3571803585":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/416587443":"2024-06-11T00:45:20.254342+0000","172.21.15.37:6801/2766692263":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/2026344134":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/2781321886":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6801/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/1110143997":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2766692263":"2024-06-11T00:45:20.254342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:52:56.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.887+0000 7ff9192c3700 1 -- 172.21.15.37:0/3471769214 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff8fc0a2c20 msgr2=0x7ff8fc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:56.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.887+0000 7ff9192c3700 1 --2- 172.21.15.37:0/3471769214 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff8fc0a2c20 0x7ff8fc0a50e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff914107d30 tx=0x7ff904008040 comp rx=0 tx=0).stop 2024-06-10T00:52:56.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.887+0000 7ff9192c3700 1 -- 172.21.15.37:0/3471769214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff914102ed0 msgr2=0x7ff91419ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:56.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.887+0000 7ff9192c3700 1 --2- 172.21.15.37:0/3471769214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff914102ed0 0x7ff91419ee30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff9080092e0 tx=0x7ff9080037c0 comp rx=0 tx=0).stop 2024-06-10T00:52:56.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.887+0000 7ff9192c3700 1 -- 172.21.15.37:0/3471769214 shutdown_connections 2024-06-10T00:52:56.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.887+0000 7ff9192c3700 1 --2- 172.21.15.37:0/3471769214 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff8fc0a2c20 0x7ff8fc0a50e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:56.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.887+0000 7ff9192c3700 1 --2- 172.21.15.37:0/3471769214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff914108510 0x7ff914198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:56.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.887+0000 7ff9192c3700 1 --2- 172.21.15.37:0/3471769214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff914102510 0x7ff91419e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:56.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.887+0000 7ff9192c3700 1 --2- 172.21.15.37:0/3471769214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff914102ed0 0x7ff91419ee30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:56.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.887+0000 7ff9192c3700 1 -- 172.21.15.37:0/3471769214 >> 172.21.15.37:0/3471769214 conn(0x7ff9140795b0 msgr2=0x7ff914101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:56.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.887+0000 7ff9192c3700 1 -- 172.21.15.37:0/3471769214 shutdown_connections 2024-06-10T00:52:56.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:56.887+0000 7ff9192c3700 1 -- 172.21.15.37:0/3471769214 wait complete. 2024-06-10T00:52:57.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:57 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3471769214' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:52:57.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:57 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3471769214' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:52:57.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:57 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3471769214' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:52:57.638 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T00:52:57.893 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:52:58.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:52:58 smithi167 ceph-mon[36917]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:58.358 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:52:58 smithi037 ceph-mon[32192]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:58.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:52:58 smithi094 ceph-mon[36708]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:52:59.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.634+0000 7fdf3487c700 1 -- 172.21.15.37:0/3141670420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf30107840 msgr2=0x7fdf3010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:59.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.634+0000 7fdf3487c700 1 --2- 172.21.15.37:0/3141670420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf30107840 0x7fdf3010f3c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdf2800b0d0 tx=0x7fdf2800b3e0 comp rx=0 tx=0).stop 2024-06-10T00:52:59.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.634+0000 7fdf3487c700 1 -- 172.21.15.37:0/3141670420 shutdown_connections 2024-06-10T00:52:59.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.634+0000 7fdf3487c700 1 --2- 172.21.15.37:0/3141670420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf30107840 0x7fdf3010f3c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.634+0000 7fdf3487c700 1 --2- 172.21.15.37:0/3141670420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf30102740 0x7fdf30102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.634+0000 7fdf3487c700 1 --2- 172.21.15.37:0/3141670420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf301030f0 0x7fdf30107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.634+0000 7fdf3487c700 1 -- 172.21.15.37:0/3141670420 >> 172.21.15.37:0/3141670420 conn(0x7fdf300795f0 msgr2=0x7fdf30079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.634+0000 7fdf3487c700 1 -- 172.21.15.37:0/3141670420 shutdown_connections 2024-06-10T00:52:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.634+0000 7fdf3487c700 1 -- 172.21.15.37:0/3141670420 wait complete. 2024-06-10T00:52:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.635+0000 7fdf3487c700 1 Processor -- start 2024-06-10T00:52:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.635+0000 7fdf3487c700 1 -- start start 2024-06-10T00:52:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.636+0000 7fdf3487c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf30102740 0x7fdf3019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.636+0000 7fdf3487c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf301030f0 0x7fdf3019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.636+0000 7fdf3487c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf30107840 0x7fdf301989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.636+0000 7fdf3487c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdf301127d0 con 0x7fdf301030f0 2024-06-10T00:52:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.636+0000 7fdf3487c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdf30112650 con 0x7fdf30107840 2024-06-10T00:52:59.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.636+0000 7fdf3487c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdf30112950 con 0x7fdf30102740 2024-06-10T00:52:59.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.636+0000 7fdf25d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf301030f0 0x7fdf3019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:59.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.636+0000 7fdf25d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf301030f0 0x7fdf3019ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46550/0 (socket says 172.21.15.37:46550) 2024-06-10T00:52:59.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.636+0000 7fdf25d9b700 1 -- 172.21.15.37:0/1337982853 learned_addr learned my addr 172.21.15.37:0/1337982853 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:52:59.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.636+0000 7fdf2e59c700 1 --2- 172.21.15.37:0/1337982853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf30102740 0x7fdf3019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:59.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.636+0000 7fdf2ed9d700 1 --2- 172.21.15.37:0/1337982853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf30107840 0x7fdf301989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:59.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.636+0000 7fdf25d9b700 1 -- 172.21.15.37:0/1337982853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf30102740 msgr2=0x7fdf3019e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:59.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.636+0000 7fdf25d9b700 1 --2- 172.21.15.37:0/1337982853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf30102740 0x7fdf3019e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:59.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.636+0000 7fdf25d9b700 1 -- 172.21.15.37:0/1337982853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf30107840 msgr2=0x7fdf301989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:59.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.636+0000 7fdf25d9b700 1 --2- 172.21.15.37:0/1337982853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf30107840 0x7fdf301989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:59.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.636+0000 7fdf25d9b700 1 -- 172.21.15.37:0/1337982853 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf30199130 con 0x7fdf301030f0 2024-06-10T00:52:59.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.636+0000 7fdf2ed9d700 1 --2- 172.21.15.37:0/1337982853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf30107840 0x7fdf301989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:52:59.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.636+0000 7fdf25d9b700 1 --2- 172.21.15.37:0/1337982853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf301030f0 0x7fdf3019ed50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800b530 tx=0x7fdf1800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:59.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.637+0000 7fdf27fff700 1 -- 172.21.15.37:0/1337982853 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf18015020 con 0x7fdf301030f0 2024-06-10T00:52:59.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.637+0000 7fdf27fff700 1 -- 172.21.15.37:0/1337982853 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf18011d10 con 0x7fdf301030f0 2024-06-10T00:52:59.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.637+0000 7fdf3487c700 1 -- 172.21.15.37:0/1337982853 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf30199420 con 0x7fdf301030f0 2024-06-10T00:52:59.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.637+0000 7fdf27fff700 1 -- 172.21.15.37:0/1337982853 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf18010e50 con 0x7fdf301030f0 2024-06-10T00:52:59.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.637+0000 7fdf3487c700 1 -- 172.21.15.37:0/1337982853 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf301a56e0 con 0x7fdf301030f0 2024-06-10T00:52:59.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.638+0000 7fdf3487c700 1 -- 172.21.15.37:0/1337982853 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf30069180 con 0x7fdf301030f0 2024-06-10T00:52:59.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.640+0000 7fdf27fff700 1 -- 172.21.15.37:0/1337982853 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdf18018020 con 0x7fdf301030f0 2024-06-10T00:52:59.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.640+0000 7fdf27fff700 1 --2- 172.21.15.37:0/1337982853 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdf1c0a2dc0 0x7fdf1c0a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:52:59.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.641+0000 7fdf2e59c700 1 --2- 172.21.15.37:0/1337982853 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdf1c0a2dc0 0x7fdf1c0a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:52:59.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.641+0000 7fdf27fff700 1 -- 172.21.15.37:0/1337982853 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7fdf180cf020 con 0x7fdf301030f0 2024-06-10T00:52:59.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.641+0000 7fdf2e59c700 1 --2- 172.21.15.37:0/1337982853 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdf1c0a2dc0 0x7fdf1c0a5280 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdf20005950 tx=0x7fdf20005ed0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:52:59.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.644+0000 7fdf27fff700 1 -- 172.21.15.37:0/1337982853 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdf1801f020 con 0x7fdf301030f0 2024-06-10T00:52:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.850+0000 7fdf3487c700 1 -- 172.21.15.37:0/1337982853 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf3002dca0 con 0x7fdf301030f0 2024-06-10T00:52:59.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.851+0000 7fdf27fff700 1 -- 172.21.15.37:0/1337982853 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24093 (secure 0 0 0) 0x7fdf18097070 con 0x7fdf301030f0 2024-06-10T00:52:59.852 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:52:59.852 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":96,"fsid":"68793514-26c2-11ef-bc9b-c7b262605968","created":"2024-06-10T00:44:24.106713+0000","modified":"2024-06-10T00:52:50.857814+0000","last_up_change":"2024-06-10T00:51:31.562156+0000","last_in_change":"2024-06-10T00:51:11.294301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:47:44.773911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:52:31.684780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:52:34.885503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:52:44.415437+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1df6046a-6801-4950-9193-54303fed169b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6803","nonce":3594704094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6805","nonce":3594704094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6809","nonce":3594704094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6807","nonce":3594704094}]},"public_addr":"172.21.15.37:6803/3594704094","cluster_addr":"172.21.15.37:6805/3594704094","heartbeat_back_addr":"172.21.15.37:6809/3594704094","heartbeat_front_addr":"172.21.15.37:6807/3594704094","state":["exists","up"]},{"osd":1,"uuid":"4e1c8944-94e4-4e46-9efe-56229fe6df1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6801","nonce":3427727155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6803","nonce":3427727155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6807","nonce":3427727155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6805","nonce":3427727155}]},"public_addr":"172.21.15.94:6801/3427727155","cluster_addr":"172.21.15.94:6803/3427727155","heartbeat_back_addr":"172.21.15.94:6807/3427727155","heartbeat_front_addr":"172.21.15.94:6805/3427727155","state":["exists","up"]},{"osd":2,"uuid":"fd2fcedc-4c88-478f-8025-13e5944a27ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6801","nonce":4002498292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6803","nonce":4002498292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6807","nonce":4002498292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6805","nonce":4002498292}]},"public_addr":"172.21.15.167:6801/4002498292","cluster_addr":"172.21.15.167:6803/4002498292","heartbeat_back_addr":"172.21.15.167:6807/4002498292","heartbeat_front_addr":"172.21.15.167:6805/4002498292","state":["exists","up"]},{"osd":3,"uuid":"d0153676-818c-4dcc-83b1-287435a0c8a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6811","nonce":3425469324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6813","nonce":3425469324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6817","nonce":3425469324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6815","nonce":3425469324}]},"public_addr":"172.21.15.37:6811/3425469324","cluster_addr":"172.21.15.37:6813/3425469324","heartbeat_back_addr":"172.21.15.37:6817/3425469324","heartbeat_front_addr":"172.21.15.37:6815/3425469324","state":["exists","up"]},{"osd":4,"uuid":"172c333c-00c0-4e94-8870-acb3b32b2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6809","nonce":1740627431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6811","nonce":1740627431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6815","nonce":1740627431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6813","nonce":1740627431}]},"public_addr":"172.21.15.94:6809/1740627431","cluster_addr":"172.21.15.94:6811/1740627431","heartbeat_back_addr":"172.21.15.94:6815/1740627431","heartbeat_front_addr":"172.21.15.94:6813/1740627431","state":["exists","up"]},{"osd":5,"uuid":"dbce656e-5f93-4bda-baf2-fc3cc03d8282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6809","nonce":221805236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6811","nonce":221805236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6815","nonce":221805236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6813","nonce":221805236}]},"public_addr":"172.21.15.167:6809/221805236","cluster_addr":"172.21.15.167:6811/221805236","heartbeat_back_addr":"172.21.15.167:6815/221805236","heartbeat_front_addr":"172.21.15.167:6813/221805236","state":["exists","up"]},{"osd":6,"uuid":"fcbda266-6d60-43b2-a51c-4e562476d1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6819","nonce":472670875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6821","nonce":472670875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6825","nonce":472670875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6823","nonce":472670875}]},"public_addr":"172.21.15.37:6819/472670875","cluster_addr":"172.21.15.37:6821/472670875","heartbeat_back_addr":"172.21.15.37:6825/472670875","heartbeat_front_addr":"172.21.15.37:6823/472670875","state":["exists","up"]},{"osd":7,"uuid":"ed829ed3-ba85-4e82-ac37-e44e9534d37c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6817","nonce":3932281552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6819","nonce":3932281552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6823","nonce":3932281552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6821","nonce":3932281552}]},"public_addr":"172.21.15.94:6817/3932281552","cluster_addr":"172.21.15.94:6819/3932281552","heartbeat_back_addr":"172.21.15.94:6823/3932281552","heartbeat_front_addr":"172.21.15.94:6821/3932281552","state":["exists","up"]},{"osd":8,"uuid":"7d6267d8-72f4-4704-b554-bafc52d11a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6817","nonce":2290216047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6819","nonce":2290216047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6823","nonce":2290216047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6821","nonce":2290216047}]},"public_addr":"172.21.15.167:6817/2290216047","cluster_addr":"172.21.15.167:6819/2290216047","heartbeat_back_addr":"172.21.15.167:6823/2290216047","heartbeat_front_addr":"172.21.15.167:6821/2290216047","state":["exists","up"]},{"osd":9,"uuid":"210fdc20-0a36-4eeb-bdf2-c0a7f130e19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6827","nonce":2063666428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6829","nonce":2063666428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6833","nonce":2063666428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6831","nonce":2063666428}]},"public_addr":"172.21.15.37:6827/2063666428","cluster_addr":"172.21.15.37:6829/2063666428","heartbeat_back_addr":"172.21.15.37:6833/2063666428","heartbeat_front_addr":"172.21.15.37:6831/2063666428","state":["exists","up"]},{"osd":10,"uuid":"e200b371-b3cf-4b92-a659-f155ca86f0a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6825","nonce":1706618899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6827","nonce":1706618899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6831","nonce":1706618899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6829","nonce":1706618899}]},"public_addr":"172.21.15.94:6825/1706618899","cluster_addr":"172.21.15.94:6827/1706618899","heartbeat_back_addr":"172.21.15.94:6831/1706618899","heartbeat_front_addr":"172.21.15.94:6829/1706618899","state":["exists","up"]},{"osd":11,"uuid":"cfa427df-37a5-4d28-ac2e-af777d607008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6825","nonce":1349892516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6827","nonce":1349892516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6831","nonce":1349892516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6829","nonce":1349892516}]},"public_addr":"172.21.15.167:6825/1349892516","cluster_addr":"172.21.15.167:6827/1349892516","heartbeat_back_addr":"172.21.15.167:6831/1349892516","heartbeat_front_addr":"172.21.15.167:6829/1349892516","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:46:51.846147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:15.763723+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:40.106800+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:05.923204+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:30.501883+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:55.565575+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:20.411360+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:46.071776+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:11.854599+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:37.608818+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:03.176767+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:29.768705+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/4084722397":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/3571803585":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/416587443":"2024-06-11T00:45:20.254342+0000","172.21.15.37:6801/2766692263":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/2026344134":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/2781321886":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6801/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/1110143997":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2766692263":"2024-06-11T00:45:20.254342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:52:59.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.854+0000 7fdf3487c700 1 -- 172.21.15.37:0/1337982853 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdf1c0a2dc0 msgr2=0x7fdf1c0a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:59.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.854+0000 7fdf3487c700 1 --2- 172.21.15.37:0/1337982853 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdf1c0a2dc0 0x7fdf1c0a5280 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdf20005950 tx=0x7fdf20005ed0 comp rx=0 tx=0).stop 2024-06-10T00:52:59.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.854+0000 7fdf3487c700 1 -- 172.21.15.37:0/1337982853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf301030f0 msgr2=0x7fdf3019ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:52:59.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.854+0000 7fdf3487c700 1 --2- 172.21.15.37:0/1337982853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf301030f0 0x7fdf3019ed50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800b530 tx=0x7fdf1800b8f0 comp rx=0 tx=0).stop 2024-06-10T00:52:59.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.854+0000 7fdf3487c700 1 -- 172.21.15.37:0/1337982853 shutdown_connections 2024-06-10T00:52:59.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.854+0000 7fdf3487c700 1 --2- 172.21.15.37:0/1337982853 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdf1c0a2dc0 0x7fdf1c0a5280 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:59.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.854+0000 7fdf3487c700 1 --2- 172.21.15.37:0/1337982853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf30107840 0x7fdf301989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:59.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.854+0000 7fdf3487c700 1 --2- 172.21.15.37:0/1337982853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf30102740 0x7fdf3019e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:59.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.854+0000 7fdf3487c700 1 --2- 172.21.15.37:0/1337982853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf301030f0 0x7fdf3019ed50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:52:59.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.854+0000 7fdf3487c700 1 -- 172.21.15.37:0/1337982853 >> 172.21.15.37:0/1337982853 conn(0x7fdf300795f0 msgr2=0x7fdf30100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:52:59.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.854+0000 7fdf3487c700 1 -- 172.21.15.37:0/1337982853 shutdown_connections 2024-06-10T00:52:59.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:52:59.854+0000 7fdf3487c700 1 -- 172.21.15.37:0/1337982853 wait complete. 2024-06-10T00:53:00.242 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:00 smithi037 ceph-mon[32192]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:00.242 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:00 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1337982853' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:53:00.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:00 smithi167 ceph-mon[36917]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:00.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:00 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1337982853' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:53:00.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:00 smithi094 ceph-mon[36708]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:00.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:00 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1337982853' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:53:00.638 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:53:00.891 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:53:02.269 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:02 smithi037 ceph-mon[32192]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:02.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:02 smithi167 ceph-mon[36917]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:02.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:02 smithi094 ceph-mon[36708]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:02.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.524+0000 7fa0171ee700 1 -- 172.21.15.37:0/397411396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa010102740 msgr2=0x7fa010102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:02.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.524+0000 7fa0171ee700 1 --2- 172.21.15.37:0/397411396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa010102740 0x7fa010102b20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa0040099c0 tx=0x7fa004009cd0 comp rx=0 tx=0).stop 2024-06-10T00:53:02.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.524+0000 7fa0171ee700 1 -- 172.21.15.37:0/397411396 shutdown_connections 2024-06-10T00:53:02.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.524+0000 7fa0171ee700 1 --2- 172.21.15.37:0/397411396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa010107840 0x7fa01010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:02.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.524+0000 7fa0171ee700 1 --2- 172.21.15.37:0/397411396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa010102740 0x7fa010102b20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:02.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.524+0000 7fa0171ee700 1 --2- 172.21.15.37:0/397411396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0101030f0 0x7fa010107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:02.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.524+0000 7fa0171ee700 1 -- 172.21.15.37:0/397411396 >> 172.21.15.37:0/397411396 conn(0x7fa0100795f0 msgr2=0x7fa010079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:02.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.524+0000 7fa0171ee700 1 -- 172.21.15.37:0/397411396 shutdown_connections 2024-06-10T00:53:02.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.524+0000 7fa0171ee700 1 -- 172.21.15.37:0/397411396 wait complete. 2024-06-10T00:53:02.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.525+0000 7fa0171ee700 1 Processor -- start 2024-06-10T00:53:02.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.525+0000 7fa0171ee700 1 -- start start 2024-06-10T00:53:02.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.525+0000 7fa0171ee700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa010102740 0x7fa01019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:02.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.525+0000 7fa0171ee700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0101030f0 0x7fa01019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:02.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.525+0000 7fa0171ee700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa010107840 0x7fa01019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:02.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.526+0000 7fa0171ee700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa010112780 con 0x7fa010107840 2024-06-10T00:53:02.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.526+0000 7fa0171ee700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa010112600 con 0x7fa010102740 2024-06-10T00:53:02.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.526+0000 7fa0171ee700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa010112900 con 0x7fa0101030f0 2024-06-10T00:53:02.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.526+0000 7fa01578b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa010107840 0x7fa01019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:02.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.526+0000 7fa01578b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa010107840 0x7fa01019f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56780/0 (socket says 172.21.15.37:56780) 2024-06-10T00:53:02.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.526+0000 7fa01578b700 1 -- 172.21.15.37:0/3498939536 learned_addr learned my addr 172.21.15.37:0/3498939536 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:53:02.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.526+0000 7fa014f8a700 1 --2- 172.21.15.37:0/3498939536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa010102740 0x7fa01019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:02.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.526+0000 7fa00ffff700 1 --2- 172.21.15.37:0/3498939536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0101030f0 0x7fa01019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:02.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.526+0000 7fa01578b700 1 -- 172.21.15.37:0/3498939536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0101030f0 msgr2=0x7fa01019b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:02.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.526+0000 7fa01578b700 1 --2- 172.21.15.37:0/3498939536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0101030f0 0x7fa01019b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:02.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.526+0000 7fa01578b700 1 -- 172.21.15.37:0/3498939536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa010102740 msgr2=0x7fa01019ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:02.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.526+0000 7fa01578b700 1 --2- 172.21.15.37:0/3498939536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa010102740 0x7fa01019ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:02.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.526+0000 7fa01578b700 1 -- 172.21.15.37:0/3498939536 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa01019fde0 con 0x7fa010107840 2024-06-10T00:53:02.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.526+0000 7fa01578b700 1 --2- 172.21.15.37:0/3498939536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa010107840 0x7fa01019f6a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa008007ee0 tx=0x7fa00800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:02.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.526+0000 7fa00dffb700 1 -- 172.21.15.37:0/3498939536 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa008004bb0 con 0x7fa010107840 2024-06-10T00:53:02.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.526+0000 7fa0171ee700 1 -- 172.21.15.37:0/3498939536 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0101a00d0 con 0x7fa010107840 2024-06-10T00:53:02.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.526+0000 7fa00dffb700 1 -- 172.21.15.37:0/3498939536 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa008004d50 con 0x7fa010107840 2024-06-10T00:53:02.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.527+0000 7fa00dffb700 1 -- 172.21.15.37:0/3498939536 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa008005960 con 0x7fa010107840 2024-06-10T00:53:02.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.527+0000 7fa0171ee700 1 -- 172.21.15.37:0/3498939536 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0101a7950 con 0x7fa010107840 2024-06-10T00:53:02.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.528+0000 7fa0171ee700 1 -- 172.21.15.37:0/3498939536 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa010069180 con 0x7fa010107840 2024-06-10T00:53:02.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.529+0000 7fa00dffb700 1 -- 172.21.15.37:0/3498939536 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa00801a020 con 0x7fa010107840 2024-06-10T00:53:02.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.529+0000 7fa00dffb700 1 --2- 172.21.15.37:0/3498939536 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa0000a2ca0 0x7fa0000a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:02.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.529+0000 7fa014f8a700 1 --2- 172.21.15.37:0/3498939536 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa0000a2ca0 0x7fa0000a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:02.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.530+0000 7fa00dffb700 1 -- 172.21.15.37:0/3498939536 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7fa0080ce910 con 0x7fa010107840 2024-06-10T00:53:02.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.530+0000 7fa014f8a700 1 --2- 172.21.15.37:0/3498939536 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa0000a2ca0 0x7fa0000a5160 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa004000c00 tx=0x7fa004005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:02.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.534+0000 7fa00dffb700 1 -- 172.21.15.37:0/3498939536 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa008010070 con 0x7fa010107840 2024-06-10T00:53:02.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.782+0000 7fa0171ee700 1 -- 172.21.15.37:0/3498939536 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0101a7b80 con 0x7fa010107840 2024-06-10T00:53:02.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.783+0000 7fa00dffb700 1 -- 172.21.15.37:0/3498939536 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11794 (secure 0 0 0) 0x7fa00809b630 con 0x7fa010107840 2024-06-10T00:53:02.784 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:53:02.784 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24431,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T00:52:50.857957+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24472,"qdb_cluster":[24472]},"id":1}]} 2024-06-10T00:53:02.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.785+0000 7fa0171ee700 1 -- 172.21.15.37:0/3498939536 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa0000a2ca0 msgr2=0x7fa0000a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:02.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.785+0000 7fa0171ee700 1 --2- 172.21.15.37:0/3498939536 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa0000a2ca0 0x7fa0000a5160 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa004000c00 tx=0x7fa004005fb0 comp rx=0 tx=0).stop 2024-06-10T00:53:02.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.785+0000 7fa0171ee700 1 -- 172.21.15.37:0/3498939536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa010107840 msgr2=0x7fa01019f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:02.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.785+0000 7fa0171ee700 1 --2- 172.21.15.37:0/3498939536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa010107840 0x7fa01019f6a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa008007ee0 tx=0x7fa00800d3b0 comp rx=0 tx=0).stop 2024-06-10T00:53:02.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.785+0000 7fa0171ee700 1 -- 172.21.15.37:0/3498939536 shutdown_connections 2024-06-10T00:53:02.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.785+0000 7fa0171ee700 1 --2- 172.21.15.37:0/3498939536 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa0000a2ca0 0x7fa0000a5160 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:02.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.785+0000 7fa0171ee700 1 --2- 172.21.15.37:0/3498939536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa010107840 0x7fa01019f6a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:02.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.785+0000 7fa0171ee700 1 --2- 172.21.15.37:0/3498939536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa010102740 0x7fa01019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:02.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.785+0000 7fa0171ee700 1 --2- 172.21.15.37:0/3498939536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0101030f0 0x7fa01019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:02.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.785+0000 7fa0171ee700 1 -- 172.21.15.37:0/3498939536 >> 172.21.15.37:0/3498939536 conn(0x7fa0100795f0 msgr2=0x7fa010100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:02.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.785+0000 7fa0171ee700 1 -- 172.21.15.37:0/3498939536 shutdown_connections 2024-06-10T00:53:02.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:02.785+0000 7fa0171ee700 1 -- 172.21.15.37:0/3498939536 wait complete. 2024-06-10T00:53:02.787 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 17 2024-06-10T00:53:03.334 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:03 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3498939536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:53:03.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:03 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3498939536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:53:03.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:03 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3498939536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:53:03.487 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T00:53:03.738 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:53:04.127 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:04 smithi037 ceph-mon[32192]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:04.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:04 smithi167 ceph-mon[36917]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:04.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:04 smithi094 ceph-mon[36708]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:05.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.562+0000 7fb2c9b9f700 1 -- 172.21.15.37:0/553487120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c4101520 msgr2=0x7fb2c4101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:05.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.562+0000 7fb2c9b9f700 1 --2- 172.21.15.37:0/553487120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c4101520 0x7fb2c4101900 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80099c0 tx=0x7fb2b8009cd0 comp rx=0 tx=0).stop 2024-06-10T00:53:05.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.562+0000 7fb2c9b9f700 1 -- 172.21.15.37:0/553487120 shutdown_connections 2024-06-10T00:53:05.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.562+0000 7fb2c9b9f700 1 --2- 172.21.15.37:0/553487120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2c4106600 0x7fb2c410e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:05.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.562+0000 7fb2c9b9f700 1 --2- 172.21.15.37:0/553487120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c4101520 0x7fb2c4101900 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:05.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.562+0000 7fb2c9b9f700 1 --2- 172.21.15.37:0/553487120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2c4101ed0 0x7fb2c4105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:05.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.562+0000 7fb2c9b9f700 1 -- 172.21.15.37:0/553487120 >> 172.21.15.37:0/553487120 conn(0x7fb2c40fcdb0 msgr2=0x7fb2c40ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:05.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.562+0000 7fb2c9b9f700 1 -- 172.21.15.37:0/553487120 shutdown_connections 2024-06-10T00:53:05.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.562+0000 7fb2c9b9f700 1 -- 172.21.15.37:0/553487120 wait complete. 2024-06-10T00:53:05.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.563+0000 7fb2c9b9f700 1 Processor -- start 2024-06-10T00:53:05.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.563+0000 7fb2c9b9f700 1 -- start start 2024-06-10T00:53:05.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.563+0000 7fb2c9b9f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c4101520 0x7fb2c419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:05.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.563+0000 7fb2c9b9f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2c4101ed0 0x7fb2c419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:05.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.563+0000 7fb2c9b9f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2c4106600 0x7fb2c4198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:05.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.563+0000 7fb2c9b9f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c41127a0 con 0x7fb2c4101520 2024-06-10T00:53:05.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.563+0000 7fb2c9b9f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c4112620 con 0x7fb2c4101ed0 2024-06-10T00:53:05.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.563+0000 7fb2c9b9f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c4112920 con 0x7fb2c4106600 2024-06-10T00:53:05.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.564+0000 7fb2c8b9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c4101520 0x7fb2c419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:05.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.564+0000 7fb2c8b9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c4101520 0x7fb2c419e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56804/0 (socket says 172.21.15.37:56804) 2024-06-10T00:53:05.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.564+0000 7fb2c8b9d700 1 -- 172.21.15.37:0/1799490635 learned_addr learned my addr 172.21.15.37:0/1799490635 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:53:05.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.564+0000 7fb2c3fff700 1 --2- 172.21.15.37:0/1799490635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2c4101ed0 0x7fb2c419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:05.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.564+0000 7fb2c939e700 1 --2- 172.21.15.37:0/1799490635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2c4106600 0x7fb2c4198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:05.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.564+0000 7fb2c8b9d700 1 -- 172.21.15.37:0/1799490635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2c4106600 msgr2=0x7fb2c4198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:05.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.564+0000 7fb2c8b9d700 1 --2- 172.21.15.37:0/1799490635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2c4106600 0x7fb2c4198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:05.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.564+0000 7fb2c8b9d700 1 -- 172.21.15.37:0/1799490635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2c4101ed0 msgr2=0x7fb2c419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:05.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.564+0000 7fb2c8b9d700 1 --2- 172.21.15.37:0/1799490635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2c4101ed0 0x7fb2c419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:05.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.564+0000 7fb2c8b9d700 1 -- 172.21.15.37:0/1799490635 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c4199160 con 0x7fb2c4101520 2024-06-10T00:53:05.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.564+0000 7fb2c3fff700 1 --2- 172.21.15.37:0/1799490635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2c4101ed0 0x7fb2c419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:53:05.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.564+0000 7fb2c8b9d700 1 --2- 172.21.15.37:0/1799490635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c4101520 0x7fb2c419e840 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8009990 tx=0x7fb2b8009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:05.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.564+0000 7fb2c1ffb700 1 -- 172.21.15.37:0/1799490635 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8027070 con 0x7fb2c4101520 2024-06-10T00:53:05.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.564+0000 7fb2c1ffb700 1 -- 172.21.15.37:0/1799490635 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2b8004050 con 0x7fb2c4101520 2024-06-10T00:53:05.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.564+0000 7fb2c9b9f700 1 -- 172.21.15.37:0/1799490635 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c41993f0 con 0x7fb2c4101520 2024-06-10T00:53:05.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.564+0000 7fb2c1ffb700 1 -- 172.21.15.37:0/1799490635 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8022d30 con 0x7fb2c4101520 2024-06-10T00:53:05.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.565+0000 7fb2c9b9f700 1 -- 172.21.15.37:0/1799490635 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c41a5710 con 0x7fb2c4101520 2024-06-10T00:53:05.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.565+0000 7fb2c9b9f700 1 -- 172.21.15.37:0/1799490635 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c4069180 con 0x7fb2c4101520 2024-06-10T00:53:05.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.567+0000 7fb2c1ffb700 1 -- 172.21.15.37:0/1799490635 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb2b802a070 con 0x7fb2c4101520 2024-06-10T00:53:05.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.567+0000 7fb2c1ffb700 1 --2- 172.21.15.37:0/1799490635 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb2b40a2c20 0x7fb2b40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:05.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.567+0000 7fb2c3fff700 1 --2- 172.21.15.37:0/1799490635 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb2b40a2c20 0x7fb2b40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:05.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.568+0000 7fb2c1ffb700 1 -- 172.21.15.37:0/1799490635 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7fb2b80d0650 con 0x7fb2c4101520 2024-06-10T00:53:05.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.568+0000 7fb2c3fff700 1 --2- 172.21.15.37:0/1799490635 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb2b40a2c20 0x7fb2b40a50e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4107bb0 tx=0x7fb2b0005f50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:05.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.571+0000 7fb2c1ffb700 1 -- 172.21.15.37:0/1799490635 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb2b8031070 con 0x7fb2c4101520 2024-06-10T00:53:05.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.778+0000 7fb2c9b9f700 1 -- 172.21.15.37:0/1799490635 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb2c402dca0 con 0x7fb2c4101520 2024-06-10T00:53:05.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.779+0000 7fb2c1ffb700 1 -- 172.21.15.37:0/1799490635 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24093 (secure 0 0 0) 0x7fb2b8013070 con 0x7fb2c4101520 2024-06-10T00:53:05.780 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:53:05.780 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":96,"fsid":"68793514-26c2-11ef-bc9b-c7b262605968","created":"2024-06-10T00:44:24.106713+0000","modified":"2024-06-10T00:52:50.857814+0000","last_up_change":"2024-06-10T00:51:31.562156+0000","last_in_change":"2024-06-10T00:51:11.294301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:47:44.773911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:52:31.684780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:52:34.885503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:52:44.415437+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1df6046a-6801-4950-9193-54303fed169b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6803","nonce":3594704094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6805","nonce":3594704094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6809","nonce":3594704094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6807","nonce":3594704094}]},"public_addr":"172.21.15.37:6803/3594704094","cluster_addr":"172.21.15.37:6805/3594704094","heartbeat_back_addr":"172.21.15.37:6809/3594704094","heartbeat_front_addr":"172.21.15.37:6807/3594704094","state":["exists","up"]},{"osd":1,"uuid":"4e1c8944-94e4-4e46-9efe-56229fe6df1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6801","nonce":3427727155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6803","nonce":3427727155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6807","nonce":3427727155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6805","nonce":3427727155}]},"public_addr":"172.21.15.94:6801/3427727155","cluster_addr":"172.21.15.94:6803/3427727155","heartbeat_back_addr":"172.21.15.94:6807/3427727155","heartbeat_front_addr":"172.21.15.94:6805/3427727155","state":["exists","up"]},{"osd":2,"uuid":"fd2fcedc-4c88-478f-8025-13e5944a27ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6801","nonce":4002498292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6803","nonce":4002498292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6807","nonce":4002498292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6805","nonce":4002498292}]},"public_addr":"172.21.15.167:6801/4002498292","cluster_addr":"172.21.15.167:6803/4002498292","heartbeat_back_addr":"172.21.15.167:6807/4002498292","heartbeat_front_addr":"172.21.15.167:6805/4002498292","state":["exists","up"]},{"osd":3,"uuid":"d0153676-818c-4dcc-83b1-287435a0c8a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6811","nonce":3425469324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6813","nonce":3425469324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6817","nonce":3425469324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6815","nonce":3425469324}]},"public_addr":"172.21.15.37:6811/3425469324","cluster_addr":"172.21.15.37:6813/3425469324","heartbeat_back_addr":"172.21.15.37:6817/3425469324","heartbeat_front_addr":"172.21.15.37:6815/3425469324","state":["exists","up"]},{"osd":4,"uuid":"172c333c-00c0-4e94-8870-acb3b32b2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6809","nonce":1740627431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6811","nonce":1740627431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6815","nonce":1740627431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6813","nonce":1740627431}]},"public_addr":"172.21.15.94:6809/1740627431","cluster_addr":"172.21.15.94:6811/1740627431","heartbeat_back_addr":"172.21.15.94:6815/1740627431","heartbeat_front_addr":"172.21.15.94:6813/1740627431","state":["exists","up"]},{"osd":5,"uuid":"dbce656e-5f93-4bda-baf2-fc3cc03d8282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6809","nonce":221805236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6811","nonce":221805236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6815","nonce":221805236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6813","nonce":221805236}]},"public_addr":"172.21.15.167:6809/221805236","cluster_addr":"172.21.15.167:6811/221805236","heartbeat_back_addr":"172.21.15.167:6815/221805236","heartbeat_front_addr":"172.21.15.167:6813/221805236","state":["exists","up"]},{"osd":6,"uuid":"fcbda266-6d60-43b2-a51c-4e562476d1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6819","nonce":472670875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6821","nonce":472670875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6825","nonce":472670875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6823","nonce":472670875}]},"public_addr":"172.21.15.37:6819/472670875","cluster_addr":"172.21.15.37:6821/472670875","heartbeat_back_addr":"172.21.15.37:6825/472670875","heartbeat_front_addr":"172.21.15.37:6823/472670875","state":["exists","up"]},{"osd":7,"uuid":"ed829ed3-ba85-4e82-ac37-e44e9534d37c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6817","nonce":3932281552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6819","nonce":3932281552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6823","nonce":3932281552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6821","nonce":3932281552}]},"public_addr":"172.21.15.94:6817/3932281552","cluster_addr":"172.21.15.94:6819/3932281552","heartbeat_back_addr":"172.21.15.94:6823/3932281552","heartbeat_front_addr":"172.21.15.94:6821/3932281552","state":["exists","up"]},{"osd":8,"uuid":"7d6267d8-72f4-4704-b554-bafc52d11a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6817","nonce":2290216047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6819","nonce":2290216047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6823","nonce":2290216047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6821","nonce":2290216047}]},"public_addr":"172.21.15.167:6817/2290216047","cluster_addr":"172.21.15.167:6819/2290216047","heartbeat_back_addr":"172.21.15.167:6823/2290216047","heartbeat_front_addr":"172.21.15.167:6821/2290216047","state":["exists","up"]},{"osd":9,"uuid":"210fdc20-0a36-4eeb-bdf2-c0a7f130e19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6827","nonce":2063666428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6829","nonce":2063666428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6833","nonce":2063666428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6831","nonce":2063666428}]},"public_addr":"172.21.15.37:6827/2063666428","cluster_addr":"172.21.15.37:6829/2063666428","heartbeat_back_addr":"172.21.15.37:6833/2063666428","heartbeat_front_addr":"172.21.15.37:6831/2063666428","state":["exists","up"]},{"osd":10,"uuid":"e200b371-b3cf-4b92-a659-f155ca86f0a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6825","nonce":1706618899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6827","nonce":1706618899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6831","nonce":1706618899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6829","nonce":1706618899}]},"public_addr":"172.21.15.94:6825/1706618899","cluster_addr":"172.21.15.94:6827/1706618899","heartbeat_back_addr":"172.21.15.94:6831/1706618899","heartbeat_front_addr":"172.21.15.94:6829/1706618899","state":["exists","up"]},{"osd":11,"uuid":"cfa427df-37a5-4d28-ac2e-af777d607008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6825","nonce":1349892516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6827","nonce":1349892516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6831","nonce":1349892516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6829","nonce":1349892516}]},"public_addr":"172.21.15.167:6825/1349892516","cluster_addr":"172.21.15.167:6827/1349892516","heartbeat_back_addr":"172.21.15.167:6831/1349892516","heartbeat_front_addr":"172.21.15.167:6829/1349892516","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:46:51.846147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:15.763723+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:40.106800+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:05.923204+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:30.501883+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:55.565575+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:20.411360+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:46.071776+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:11.854599+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:37.608818+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:03.176767+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:29.768705+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/4084722397":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/3571803585":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/416587443":"2024-06-11T00:45:20.254342+0000","172.21.15.37:6801/2766692263":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/2026344134":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/2781321886":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6801/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/1110143997":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2766692263":"2024-06-11T00:45:20.254342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:53:05.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.781+0000 7fb2c9b9f700 1 -- 172.21.15.37:0/1799490635 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb2b40a2c20 msgr2=0x7fb2b40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:05.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.781+0000 7fb2c9b9f700 1 --2- 172.21.15.37:0/1799490635 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb2b40a2c20 0x7fb2b40a50e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4107bb0 tx=0x7fb2b0005f50 comp rx=0 tx=0).stop 2024-06-10T00:53:05.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.781+0000 7fb2c9b9f700 1 -- 172.21.15.37:0/1799490635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c4101520 msgr2=0x7fb2c419e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:05.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.781+0000 7fb2c9b9f700 1 --2- 172.21.15.37:0/1799490635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c4101520 0x7fb2c419e840 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8009990 tx=0x7fb2b8009f80 comp rx=0 tx=0).stop 2024-06-10T00:53:05.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.782+0000 7fb2c9b9f700 1 -- 172.21.15.37:0/1799490635 shutdown_connections 2024-06-10T00:53:05.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.782+0000 7fb2c9b9f700 1 --2- 172.21.15.37:0/1799490635 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb2b40a2c20 0x7fb2b40a50e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:05.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.782+0000 7fb2c9b9f700 1 --2- 172.21.15.37:0/1799490635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2c4106600 0x7fb2c4198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:05.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.782+0000 7fb2c9b9f700 1 --2- 172.21.15.37:0/1799490635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c4101520 0x7fb2c419e840 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:05.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.782+0000 7fb2c9b9f700 1 --2- 172.21.15.37:0/1799490635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2c4101ed0 0x7fb2c419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:05.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.782+0000 7fb2c9b9f700 1 -- 172.21.15.37:0/1799490635 >> 172.21.15.37:0/1799490635 conn(0x7fb2c40fcdb0 msgr2=0x7fb2c40ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:05.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.782+0000 7fb2c9b9f700 1 -- 172.21.15.37:0/1799490635 shutdown_connections 2024-06-10T00:53:05.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:05.782+0000 7fb2c9b9f700 1 -- 172.21.15.37:0/1799490635 wait complete. 2024-06-10T00:53:06.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:06 smithi167 ceph-mon[36917]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:06.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:06 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1799490635' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:53:06.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:06 smithi094 ceph-mon[36708]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:06.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:06 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1799490635' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:53:06.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:06 smithi037 ceph-mon[32192]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:06.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:06 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1799490635' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:53:06.567 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-06-10T00:52:37.869277+0000', 'modified': '2024-06-10T00:52:50.857957+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24472}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24472': {'gid': 24472, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.167:6839/3945638144', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 3945638144}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 3945638144}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24472, 'qdb_cluster': [24472]} 2024-06-10T00:53:06.567 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-06-10T00:53:06.567 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-10T00:53:06.822 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:53:08.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:08 smithi094 ceph-mon[36708]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:08.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:08 smithi037 ceph-mon[32192]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:08.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:08 smithi167 ceph-mon[36917]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:08.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.615+0000 7f2d17eb7700 1 -- 172.21.15.37:0/1801832520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d10102520 msgr2=0x7f2d101029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:08.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.615+0000 7f2d17eb7700 1 --2- 172.21.15.37:0/1801832520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d10102520 0x7f2d101029a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2d000099c0 tx=0x7f2d00009cd0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.615+0000 7f2d17eb7700 1 -- 172.21.15.37:0/1801832520 shutdown_connections 2024-06-10T00:53:08.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.615+0000 7f2d17eb7700 1 --2- 172.21.15.37:0/1801832520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d10102ee0 0x7f2d1010f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.615+0000 7f2d17eb7700 1 --2- 172.21.15.37:0/1801832520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d101085c0 0x7f2d101089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.615+0000 7f2d17eb7700 1 --2- 172.21.15.37:0/1801832520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d10102520 0x7f2d101029a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.615+0000 7f2d17eb7700 1 -- 172.21.15.37:0/1801832520 >> 172.21.15.37:0/1801832520 conn(0x7f2d10079680 msgr2=0x7f2d10079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:08.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.615+0000 7f2d17eb7700 1 -- 172.21.15.37:0/1801832520 shutdown_connections 2024-06-10T00:53:08.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.615+0000 7f2d17eb7700 1 -- 172.21.15.37:0/1801832520 wait complete. 2024-06-10T00:53:08.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.616+0000 7f2d17eb7700 1 Processor -- start 2024-06-10T00:53:08.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.616+0000 7f2d17eb7700 1 -- start start 2024-06-10T00:53:08.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.616+0000 7f2d17eb7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d10102520 0x7f2d1019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:08.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.616+0000 7f2d17eb7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d10102ee0 0x7f2d1019edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:08.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.617+0000 7f2d15452700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d10102ee0 0x7f2d1019edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:08.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.617+0000 7f2d15c53700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d10102520 0x7f2d1019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:08.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.617+0000 7f2d15452700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d10102ee0 0x7f2d1019edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56828/0 (socket says 172.21.15.37:56828) 2024-06-10T00:53:08.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.617+0000 7f2d15452700 1 -- 172.21.15.37:0/2802204193 learned_addr learned my addr 172.21.15.37:0/2802204193 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:53:08.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.617+0000 7f2d17eb7700 1 --2- 172.21.15.37:0/2802204193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d101085c0 0x7f2d10198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:08.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.617+0000 7f2d17eb7700 1 -- 172.21.15.37:0/2802204193 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2d10112820 con 0x7f2d10102ee0 2024-06-10T00:53:08.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.617+0000 7f2d17eb7700 1 -- 172.21.15.37:0/2802204193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2d101126a0 con 0x7f2d10102520 2024-06-10T00:53:08.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.617+0000 7f2d17eb7700 1 -- 172.21.15.37:0/2802204193 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2d101129a0 con 0x7f2d101085c0 2024-06-10T00:53:08.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.617+0000 7f2d16454700 1 --2- 172.21.15.37:0/2802204193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d101085c0 0x7f2d10198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:08.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.617+0000 7f2d15452700 1 -- 172.21.15.37:0/2802204193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d101085c0 msgr2=0x7f2d10198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:08.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.617+0000 7f2d15452700 1 --2- 172.21.15.37:0/2802204193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d101085c0 0x7f2d10198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.617+0000 7f2d15452700 1 -- 172.21.15.37:0/2802204193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d10102520 msgr2=0x7f2d1019e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:08.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.617+0000 7f2d15452700 1 --2- 172.21.15.37:0/2802204193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d10102520 0x7f2d1019e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.617+0000 7f2d15452700 1 -- 172.21.15.37:0/2802204193 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d101991a0 con 0x7f2d10102ee0 2024-06-10T00:53:08.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.617+0000 7f2d15452700 1 --2- 172.21.15.37:0/2802204193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d10102ee0 0x7f2d1019edc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2d00006010 tx=0x7f2d00009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:08.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.618+0000 7f2d06ffd700 1 -- 172.21.15.37:0/2802204193 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00027070 con 0x7f2d10102ee0 2024-06-10T00:53:08.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.618+0000 7f2d06ffd700 1 -- 172.21.15.37:0/2802204193 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d00003780 con 0x7f2d10102ee0 2024-06-10T00:53:08.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.618+0000 7f2d17eb7700 1 -- 172.21.15.37:0/2802204193 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d10199430 con 0x7f2d10102ee0 2024-06-10T00:53:08.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.618+0000 7f2d06ffd700 1 -- 172.21.15.37:0/2802204193 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00022da0 con 0x7f2d10102ee0 2024-06-10T00:53:08.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.618+0000 7f2d17eb7700 1 -- 172.21.15.37:0/2802204193 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d101a5750 con 0x7f2d10102ee0 2024-06-10T00:53:08.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.620+0000 7f2d06ffd700 1 -- 172.21.15.37:0/2802204193 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2d0002a070 con 0x7f2d10102ee0 2024-06-10T00:53:08.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.621+0000 7f2d06ffd700 1 --2- 172.21.15.37:0/2802204193 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2cfc0a2ca0 0x7f2cfc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:08.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.621+0000 7f2d15c53700 1 --2- 172.21.15.37:0/2802204193 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2cfc0a2ca0 0x7f2cfc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:08.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.621+0000 7f2d06ffd700 1 -- 172.21.15.37:0/2802204193 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f2d000cff00 con 0x7f2d10102ee0 2024-06-10T00:53:08.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.621+0000 7f2d15c53700 1 --2- 172.21.15.37:0/2802204193 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2cfc0a2ca0 0x7f2cfc0a5160 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2d10199b80 tx=0x7f2d0c00b3c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:08.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.622+0000 7f2d2273ab80 1 Processor -- start 2024-06-10T00:53:08.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.622+0000 7f2d2273ab80 1 -- start start 2024-06-10T00:53:08.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.622+0000 7f2d2273ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e26b6b5f80 0x55e26b6b6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:08.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.622+0000 7f2d2273ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e26b6b68a0 0x55e26b6ba6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:08.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.622+0000 7f2d2273ab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e26b6babe0 0x55e26b6bb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:08.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.622+0000 7f2d2273ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e26b6037c0 con 0x55e26b6b5f80 2024-06-10T00:53:08.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.622+0000 7f2d2273ab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e26b5ab6d0 con 0x55e26b6babe0 2024-06-10T00:53:08.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.622+0000 7f2d2273ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e26b5ff780 con 0x55e26b6b68a0 2024-06-10T00:53:08.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.622+0000 7f2d15c53700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e26b6b5f80 0x55e26b6b6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:08.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.622+0000 7f2d15c53700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e26b6b5f80 0x55e26b6b6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56842/0 (socket says 172.21.15.37:56842) 2024-06-10T00:53:08.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.622+0000 7f2d15c53700 1 -- 172.21.15.37:0/1688503732 learned_addr learned my addr 172.21.15.37:0/1688503732 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:53:08.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.623+0000 7f2d15452700 1 --2- 172.21.15.37:0/1688503732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e26b6b68a0 0x55e26b6ba6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:08.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.623+0000 7f2d16454700 1 --2- 172.21.15.37:0/1688503732 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e26b6babe0 0x55e26b6bb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:08.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.623+0000 7f2d15c53700 1 -- 172.21.15.37:0/1688503732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e26b6b68a0 msgr2=0x55e26b6ba6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:53:08.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.623+0000 7f2d15c53700 1 --2- 172.21.15.37:0/1688503732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e26b6b68a0 0x55e26b6ba6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.623+0000 7f2d15c53700 1 -- 172.21.15.37:0/1688503732 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e26b6babe0 msgr2=0x55e26b6bb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:53:08.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.623+0000 7f2d15c53700 1 --2- 172.21.15.37:0/1688503732 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e26b6babe0 0x55e26b6bb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.623+0000 7f2d15c53700 1 -- 172.21.15.37:0/1688503732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e26b5a1f00 con 0x55e26b6b5f80 2024-06-10T00:53:08.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.623+0000 7f2d16454700 1 --2- 172.21.15.37:0/1688503732 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e26b6babe0 0x55e26b6bb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:53:08.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.623+0000 7f2d15c53700 1 --2- 172.21.15.37:0/1688503732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e26b6b5f80 0x55e26b6b6360 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c005d70 tx=0x7f2d0c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eadb36421025381e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:08.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.623+0000 7f2cfbfff700 1 -- 172.21.15.37:0/1688503732 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c011ad0 con 0x55e26b6b5f80 2024-06-10T00:53:08.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.623+0000 7f2cfbfff700 1 -- 172.21.15.37:0/1688503732 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d0c003e90 con 0x55e26b6b5f80 2024-06-10T00:53:08.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.623+0000 7f2cfbfff700 1 -- 172.21.15.37:0/1688503732 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c01c5f0 con 0x55e26b6b5f80 2024-06-10T00:53:08.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.624+0000 7f2d2273ab80 1 -- 172.21.15.37:0/1688503732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e26b6b5f80 msgr2=0x55e26b6b6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:08.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.624+0000 7f2d2273ab80 1 --2- 172.21.15.37:0/1688503732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e26b6b5f80 0x55e26b6b6360 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c005d70 tx=0x7f2d0c013000 comp rx=0 tx=0).stop 2024-06-10T00:53:08.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.624+0000 7f2d2273ab80 1 -- 172.21.15.37:0/1688503732 shutdown_connections 2024-06-10T00:53:08.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.624+0000 7f2d2273ab80 1 --2- 172.21.15.37:0/1688503732 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e26b6babe0 0x55e26b6bb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.624+0000 7f2d2273ab80 1 --2- 172.21.15.37:0/1688503732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e26b6b5f80 0x55e26b6b6360 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.624+0000 7f2d2273ab80 1 --2- 172.21.15.37:0/1688503732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e26b6b68a0 0x55e26b6ba6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.624+0000 7f2d2273ab80 1 -- 172.21.15.37:0/1688503732 >> 172.21.15.37:0/1688503732 conn(0x55e26b5dcc70 msgr2=0x55e26b6ac800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:08.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.624+0000 7f2d2273ab80 1 -- 172.21.15.37:0/1688503732 shutdown_connections 2024-06-10T00:53:08.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.624+0000 7f2d2273ab80 1 -- 172.21.15.37:0/1688503732 wait complete. 2024-06-10T00:53:08.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.624+0000 7f2d2273ab80 1 -- start start 2024-06-10T00:53:08.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.625+0000 7f2d2273ab80 1 Processor -- start 2024-06-10T00:53:08.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.625+0000 7f2d2273ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e26b6b5f80 0x55e26b6c2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:08.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.625+0000 7f2d2273ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e26b6b68a0 0x55e26b6bf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:08.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.625+0000 7f2d2273ab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e26b6babe0 0x55e26b6bf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:08.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.625+0000 7f2d2273ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e26b6c2d80 con 0x55e26b6b68a0 2024-06-10T00:53:08.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.625+0000 7f2d2273ab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e26b5a3ba0 con 0x55e26b6babe0 2024-06-10T00:53:08.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.625+0000 7f2d2273ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e26b6c2f00 con 0x55e26b6b5f80 2024-06-10T00:53:08.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.625+0000 7f2d2273ab80 10 client.0 _ll_register_callbacks cb 0x55e26b59d8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:53:08.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.625+0000 7f2d15452700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e26b6b68a0 0x55e26b6bf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:08.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.625+0000 7f2d15452700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e26b6b68a0 0x55e26b6bf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56844/0 (socket says 172.21.15.37:56844) 2024-06-10T00:53:08.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.625+0000 7f2d15452700 1 -- 172.21.15.37:0/3137847986 learned_addr learned my addr 172.21.15.37:0/3137847986 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:53:08.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.625+0000 7f2d15c53700 1 --2- 172.21.15.37:0/3137847986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e26b6b5f80 0x55e26b6c2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:08.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.625+0000 7f2d16454700 1 --2- 172.21.15.37:0/3137847986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e26b6babe0 0x55e26b6bf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:08.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.625+0000 7f2d15452700 1 -- 172.21.15.37:0/3137847986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e26b6b5f80 msgr2=0x55e26b6c2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:53:08.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.625+0000 7f2d15452700 1 --2- 172.21.15.37:0/3137847986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e26b6b5f80 0x55e26b6c2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.625+0000 7f2d15452700 1 -- 172.21.15.37:0/3137847986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e26b6babe0 msgr2=0x55e26b6bf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:53:08.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.625+0000 7f2d15452700 1 --2- 172.21.15.37:0/3137847986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e26b6babe0 0x55e26b6bf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.626+0000 7f2d15452700 1 -- 172.21.15.37:0/3137847986 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e26b552880 con 0x55e26b6b68a0 2024-06-10T00:53:08.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.626+0000 7f2d15452700 1 --2- 172.21.15.37:0/3137847986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e26b6b68a0 0x55e26b6bf470 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000c010 tx=0x7f2d0009c1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff017ed47a593ea2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:08.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.626+0000 7f2cf97fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:53:08.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.626+0000 7f2cf97fa700 1 -- 172.21.15.37:0/3137847986 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d000a4f00 con 0x55e26b6b68a0 2024-06-10T00:53:08.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.626+0000 7f2d2273ab80 1 -- 172.21.15.37:0/3137847986 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d0c003e90 con 0x55e26b6b68a0 2024-06-10T00:53:08.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.626+0000 7f2cf97fa700 1 -- 172.21.15.37:0/3137847986 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f2d00085b40 con 0x55e26b6b68a0 2024-06-10T00:53:08.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.626+0000 7f2cf97fa700 1 -- 172.21.15.37:0/3137847986 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f2d0009ccb0 con 0x55e26b6b68a0 2024-06-10T00:53:08.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.626+0000 7f2d2273ab80 10 client.14706 fetch_fsmap learned FSMap version 17 2024-06-10T00:53:08.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.626+0000 7f2d2273ab80 1 -- 172.21.15.37:0/3137847986 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55e26b6cd6d0 con 0x55e26b6b68a0 2024-06-10T00:53:08.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.626+0000 7f2cf97fa700 1 -- 172.21.15.37:0/3137847986 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d0c003e90 con 0x55e26b6b68a0 2024-06-10T00:53:08.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.626+0000 7f2cf97fa700 1 -- 172.21.15.37:0/3137847986 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f2d0009ba00 con 0x55e26b6b68a0 2024-06-10T00:53:08.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.626+0000 7f2d2273ab80 10 client.14706 fetch_fsmap finished waiting for FSMap version 17 2024-06-10T00:53:08.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.627+0000 7f2d2273ab80 10 client.14706 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T00:53:08.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.627+0000 7f2d2273ab80 20 client.14706 populate_metadata read hostname 'smithi037' 2024-06-10T00:53:08.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.627+0000 7f2d2273ab80 1 --2- 172.21.15.37:0/3137847986 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55e26b6cfc50 0x55e26b6d10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:08.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.627+0000 7f2d2273ab80 4 client.14706 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:53:08.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.627+0000 7f2d2273ab80 1 -- 172.21.15.37:0/3137847986 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e26b610cc0 con 0x55e26b6cfc50 2024-06-10T00:53:08.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.627+0000 7f2d15c53700 1 --2- 172.21.15.37:0/3137847986 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55e26b6cfc50 0x55e26b6d10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:08.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.627+0000 7f2cf97fa700 10 client.14706 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:53:08.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.627+0000 7f2d15c53700 1 --2- 172.21.15.37:0/3137847986 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55e26b6cfc50 0x55e26b6d10c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ccd656d8e470b68 server_cookie=246965ac757f5203 in_seq=0 out_seq=0 2024-06-10T00:53:08.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.628+0000 7f2cf97fa700 1 -- 172.21.15.37:0/3137847986 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e26b610cc0 con 0x55e26b6cfc50 2024-06-10T00:53:08.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.628+0000 7f2cf97fa700 10 client.14706 handle_command_reply: tid=0 2024-06-10T00:53:08.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.628+0000 7f2d2273ab80 1 client.14706 shutdown 2024-06-10T00:53:08.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.628+0000 7f2d2273ab80 1 -- 172.21.15.37:0/3137847986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e26b6b68a0 msgr2=0x55e26b6bf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:08.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.628+0000 7f2d2273ab80 1 --2- 172.21.15.37:0/3137847986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e26b6b68a0 0x55e26b6bf470 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000c010 tx=0x7f2d0009c1c0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.628+0000 7f2d2273ab80 1 -- 172.21.15.37:0/3137847986 shutdown_connections 2024-06-10T00:53:08.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.628+0000 7f2d2273ab80 1 --2- 172.21.15.37:0/3137847986 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55e26b6cfc50 0x55e26b6d10c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.628+0000 7f2d2273ab80 1 --2- 172.21.15.37:0/3137847986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e26b6babe0 0x55e26b6bf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.628+0000 7f2d2273ab80 1 --2- 172.21.15.37:0/3137847986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e26b6b5f80 0x55e26b6c2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.628+0000 7f2d2273ab80 1 --2- 172.21.15.37:0/3137847986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e26b6b68a0 0x55e26b6bf470 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.628+0000 7f2d2273ab80 1 -- 172.21.15.37:0/3137847986 >> 172.21.15.37:0/3137847986 conn(0x55e26b5dcc70 msgr2=0x55e26b6b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:08.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.628+0000 7f2cf97fa700 0 client.14706 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:53:08.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.628+0000 7f2d2273ab80 1 -- 172.21.15.37:0/3137847986 shutdown_connections 2024-06-10T00:53:08.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.628+0000 7f2d2273ab80 1 -- 172.21.15.37:0/3137847986 wait complete. 2024-06-10T00:53:08.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.629+0000 7f2d2273ab80 20 client.14706 trim_cache size 0 max 16384 2024-06-10T00:53:08.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.650+0000 7f2d2273ab80 1 Processor -- start 2024-06-10T00:53:08.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.650+0000 7f2d2273ab80 1 -- start start 2024-06-10T00:53:08.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.650+0000 7f2d2273ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e26b6b5f80 0x55e26b6bfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:08.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.650+0000 7f2d2273ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e26b6b68a0 0x55e26b6bdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:08.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.650+0000 7f2d2273ab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e26b6babe0 0x55e26b6be2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:08.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.650+0000 7f2d2273ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e26b6cf400 con 0x55e26b6b5f80 2024-06-10T00:53:08.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.650+0000 7f2d2273ab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e26b6cfad0 con 0x55e26b6babe0 2024-06-10T00:53:08.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.650+0000 7f2d2273ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e26b6d01a0 con 0x55e26b6b68a0 2024-06-10T00:53:08.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.650+0000 7f2d15c53700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e26b6b5f80 0x55e26b6bfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:08.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.650+0000 7f2d15c53700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e26b6b5f80 0x55e26b6bfa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56850/0 (socket says 172.21.15.37:56850) 2024-06-10T00:53:08.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.650+0000 7f2d15c53700 1 -- 172.21.15.37:0/1894277290 learned_addr learned my addr 172.21.15.37:0/1894277290 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:53:08.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.651+0000 7f2d15c53700 1 -- 172.21.15.37:0/1894277290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e26b6b68a0 msgr2=0x55e26b6bdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:53:08.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.651+0000 7f2d15452700 1 --2- 172.21.15.37:0/1894277290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e26b6b68a0 0x55e26b6bdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:08.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.651+0000 7f2d16454700 1 --2- 172.21.15.37:0/1894277290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e26b6babe0 0x55e26b6be2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:08.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.651+0000 7f2d15c53700 1 --2- 172.21.15.37:0/1894277290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e26b6b68a0 0x55e26b6bdd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.651+0000 7f2d15c53700 1 -- 172.21.15.37:0/1894277290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e26b6babe0 msgr2=0x55e26b6be2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:53:08.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.651+0000 7f2d15c53700 1 --2- 172.21.15.37:0/1894277290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e26b6babe0 0x55e26b6be2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.651+0000 7f2d15c53700 1 -- 172.21.15.37:0/1894277290 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e26b6c1db0 con 0x55e26b6b5f80 2024-06-10T00:53:08.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.651+0000 7f2d15c53700 1 --2- 172.21.15.37:0/1894277290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e26b6b5f80 0x55e26b6bfa00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c01ed40 tx=0x7f2d0c01ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93dabb17ab33731 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:08.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.651+0000 7f2cfb7fe700 1 -- 172.21.15.37:0/1894277290 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c011ad0 con 0x55e26b6b5f80 2024-06-10T00:53:08.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.651+0000 7f2cfb7fe700 1 -- 172.21.15.37:0/1894277290 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d0c01ea20 con 0x55e26b6b5f80 2024-06-10T00:53:08.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.651+0000 7f2cfb7fe700 1 -- 172.21.15.37:0/1894277290 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c031930 con 0x55e26b6b5f80 2024-06-10T00:53:08.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.652+0000 7f2d2273ab80 1 -- 172.21.15.37:0/1894277290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e26b6b5f80 msgr2=0x55e26b6bfa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:08.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.652+0000 7f2d2273ab80 1 --2- 172.21.15.37:0/1894277290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e26b6b5f80 0x55e26b6bfa00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c01ed40 tx=0x7f2d0c01ecd0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.652+0000 7f2d2273ab80 1 -- 172.21.15.37:0/1894277290 shutdown_connections 2024-06-10T00:53:08.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.652+0000 7f2d2273ab80 1 --2- 172.21.15.37:0/1894277290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e26b6babe0 0x55e26b6be2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.652+0000 7f2d2273ab80 1 --2- 172.21.15.37:0/1894277290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e26b6b5f80 0x55e26b6bfa00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.652+0000 7f2d2273ab80 1 --2- 172.21.15.37:0/1894277290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e26b6b68a0 0x55e26b6bdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.652+0000 7f2d2273ab80 1 -- 172.21.15.37:0/1894277290 >> 172.21.15.37:0/1894277290 conn(0x55e26b5dcc70 msgr2=0x55e26b6cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.652+0000 7f2d2273ab80 1 -- 172.21.15.37:0/1894277290 shutdown_connections 2024-06-10T00:53:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.652+0000 7f2d2273ab80 1 -- 172.21.15.37:0/1894277290 wait complete. 2024-06-10T00:53:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.652+0000 7f2d2273ab80 1 -- start start 2024-06-10T00:53:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.652+0000 7f2d2273ab80 1 Processor -- start 2024-06-10T00:53:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.653+0000 7f2d2273ab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e26b6b5f80 0x55e26b6dad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.653+0000 7f2d2273ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e26b6b68a0 0x55e26b6bedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.653+0000 7f2d2273ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e26b6babe0 0x55e26b6bf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.653+0000 7f2d2273ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e26b6d0f40 con 0x55e26b6b68a0 2024-06-10T00:53:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.653+0000 7f2d2273ab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e26b6d0870 con 0x55e26b6b5f80 2024-06-10T00:53:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.653+0000 7f2d2273ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e26b6d1610 con 0x55e26b6babe0 2024-06-10T00:53:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.653+0000 7f2d2273ab80 10 client.0 _ll_register_callbacks cb 0x55e26b59d8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:53:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.653+0000 7f2d15452700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e26b6b68a0 0x55e26b6bedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.653+0000 7f2d15452700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e26b6b68a0 0x55e26b6bedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56862/0 (socket says 172.21.15.37:56862) 2024-06-10T00:53:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.653+0000 7f2d15452700 1 -- 172.21.15.37:0/1654353601 learned_addr learned my addr 172.21.15.37:0/1654353601 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:53:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.653+0000 7f2d15c53700 1 --2- 172.21.15.37:0/1654353601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e26b6b5f80 0x55e26b6dad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:08.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.653+0000 7f2d16454700 1 --2- 172.21.15.37:0/1654353601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e26b6babe0 0x55e26b6bf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:08.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.653+0000 7f2d15452700 1 -- 172.21.15.37:0/1654353601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e26b6babe0 msgr2=0x55e26b6bf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:53:08.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.653+0000 7f2d15452700 1 --2- 172.21.15.37:0/1654353601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e26b6babe0 0x55e26b6bf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.653+0000 7f2d15452700 1 -- 172.21.15.37:0/1654353601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e26b6b5f80 msgr2=0x55e26b6dad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:53:08.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.653+0000 7f2d15452700 1 --2- 172.21.15.37:0/1654353601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e26b6b5f80 0x55e26b6dad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.653+0000 7f2d15452700 1 -- 172.21.15.37:0/1654353601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e26b6d88a0 con 0x55e26b6b68a0 2024-06-10T00:53:08.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.653+0000 7f2d15452700 1 --2- 172.21.15.37:0/1654353601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e26b6b68a0 0x55e26b6bedf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2d0009c540 tx=0x7f2d0009aac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=822b9fe9f374910 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:08.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.653+0000 7f2cf97fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:53:08.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.653+0000 7f2cf97fa700 1 -- 172.21.15.37:0/1654353601 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0009c5b0 con 0x55e26b6b68a0 2024-06-10T00:53:08.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.653+0000 7f2cf97fa700 1 -- 172.21.15.37:0/1654353601 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f2d00085b40 con 0x55e26b6b68a0 2024-06-10T00:53:08.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.653+0000 7f2d2273ab80 1 -- 172.21.15.37:0/1654353601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d0c01ea20 con 0x55e26b6b68a0 2024-06-10T00:53:08.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.654+0000 7f2cf97fa700 1 -- 172.21.15.37:0/1654353601 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f2d000a5190 con 0x55e26b6b68a0 2024-06-10T00:53:08.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.654+0000 7f2d2273ab80 10 client.14712 fetch_fsmap learned FSMap version 17 2024-06-10T00:53:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.654+0000 7f2d2273ab80 1 -- 172.21.15.37:0/1654353601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55e26b6d88a0 con 0x55e26b6b68a0 2024-06-10T00:53:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.654+0000 7f2cf97fa700 1 -- 172.21.15.37:0/1654353601 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d0c01ea20 con 0x55e26b6b68a0 2024-06-10T00:53:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.654+0000 7f2cf97fa700 1 -- 172.21.15.37:0/1654353601 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f2d0009b470 con 0x55e26b6b68a0 2024-06-10T00:53:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.654+0000 7f2d2273ab80 10 client.14712 fetch_fsmap finished waiting for FSMap version 17 2024-06-10T00:53:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.654+0000 7f2d2273ab80 10 client.14712 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T00:53:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.654+0000 7f2d2273ab80 20 client.14712 populate_metadata read hostname 'smithi037' 2024-06-10T00:53:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.654+0000 7f2d2273ab80 1 --2- 172.21.15.37:0/1654353601 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55e26b73fd40 0x55e26b73ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.654+0000 7f2d2273ab80 4 client.14712 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T00:53:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.654+0000 7f2d2273ab80 1 -- 172.21.15.37:0/1654353601 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "status"}) -- 0x55e26b60b240 con 0x55e26b73fd40 2024-06-10T00:53:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.654+0000 7f2d15c53700 1 --2- 172.21.15.37:0/1654353601 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55e26b73fd40 0x55e26b73ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.655+0000 7f2d15c53700 1 --2- 172.21.15.37:0/1654353601 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55e26b73fd40 0x55e26b73ea50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c94398764491d30b server_cookie=8318e2e3a14ed5a in_seq=0 out_seq=0 2024-06-10T00:53:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.655+0000 7f2cf97fa700 10 client.14712 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:53:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.655+0000 7f2cf97fa700 1 -- 172.21.15.37:0/1654353601 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55e26b60b240 con 0x55e26b73fd40 2024-06-10T00:53:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.655+0000 7f2cf97fa700 10 client.14712 handle_command_reply: tid=0 2024-06-10T00:53:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.655+0000 7f2d2273ab80 1 client.14712 shutdown 2024-06-10T00:53:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.655+0000 7f2d2273ab80 1 -- 172.21.15.37:0/1654353601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e26b6b68a0 msgr2=0x55e26b6bedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.655+0000 7f2d2273ab80 1 --2- 172.21.15.37:0/1654353601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e26b6b68a0 0x55e26b6bedf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2d0009c540 tx=0x7f2d0009aac0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.655+0000 7f2d2273ab80 1 -- 172.21.15.37:0/1654353601 shutdown_connections 2024-06-10T00:53:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.655+0000 7f2d2273ab80 1 --2- 172.21.15.37:0/1654353601 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55e26b73fd40 0x55e26b73ea50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.655+0000 7f2d2273ab80 1 --2- 172.21.15.37:0/1654353601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e26b6babe0 0x55e26b6bf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.655+0000 7f2d2273ab80 1 --2- 172.21.15.37:0/1654353601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e26b6b5f80 0x55e26b6dad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.655+0000 7f2cf97fa700 0 client.14712 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:53:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.655+0000 7f2d2273ab80 1 --2- 172.21.15.37:0/1654353601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e26b6b68a0 0x55e26b6bedf0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.655+0000 7f2d2273ab80 1 -- 172.21.15.37:0/1654353601 >> 172.21.15.37:0/1654353601 conn(0x55e26b5dcc70 msgr2=0x55e26b6bdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.655+0000 7f2d2273ab80 1 -- 172.21.15.37:0/1654353601 shutdown_connections 2024-06-10T00:53:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.655+0000 7f2d2273ab80 1 -- 172.21.15.37:0/1654353601 wait complete. 2024-06-10T00:53:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.656+0000 7f2d2273ab80 20 client.14712 trim_cache size 0 max 16384 2024-06-10T00:53:08.661 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T00:53:08.661 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "68793514-26c2-11ef-bc9b-c7b262605968", 2024-06-10T00:53:08.661 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-06-10T00:53:08.661 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24472, 2024-06-10T00:53:08.661 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-06-10T00:53:08.661 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-06-10T00:53:08.661 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-06-10T00:53:08.661 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 30.706777106000001, 2024-06-10T00:53:08.661 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 17, 2024-06-10T00:53:08.661 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 96, 2024-06-10T00:53:08.661 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 96, 2024-06-10T00:53:08.661 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 58.356160873 2024-06-10T00:53:08.662 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T00:53:08.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.656+0000 7f2d17eb7700 1 -- 172.21.15.37:0/2802204193 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2cfc0a2ca0 msgr2=0x7f2cfc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:08.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.656+0000 7f2d17eb7700 1 --2- 172.21.15.37:0/2802204193 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2cfc0a2ca0 0x7f2cfc0a5160 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2d10199b80 tx=0x7f2d0c00b3c0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.656+0000 7f2d17eb7700 1 -- 172.21.15.37:0/2802204193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d10102ee0 msgr2=0x7f2d1019edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:08.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.656+0000 7f2d17eb7700 1 --2- 172.21.15.37:0/2802204193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d10102ee0 0x7f2d1019edc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2d00006010 tx=0x7f2d00009f80 comp rx=0 tx=0).stop 2024-06-10T00:53:08.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.656+0000 7f2d17eb7700 1 -- 172.21.15.37:0/2802204193 shutdown_connections 2024-06-10T00:53:08.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.656+0000 7f2d17eb7700 1 --2- 172.21.15.37:0/2802204193 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2cfc0a2ca0 0x7f2cfc0a5160 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.656+0000 7f2d17eb7700 1 --2- 172.21.15.37:0/2802204193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d101085c0 0x7f2d10198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.656+0000 7f2d17eb7700 1 --2- 172.21.15.37:0/2802204193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d10102520 0x7f2d1019e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.656+0000 7f2d17eb7700 1 --2- 172.21.15.37:0/2802204193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d10102ee0 0x7f2d1019edc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.656+0000 7f2d17eb7700 1 -- 172.21.15.37:0/2802204193 >> 172.21.15.37:0/2802204193 conn(0x7f2d10079680 msgr2=0x7f2d10101090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.656+0000 7f2d17eb7700 1 -- 172.21.15.37:0/2802204193 shutdown_connections 2024-06-10T00:53:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:08.656+0000 7f2d17eb7700 1 -- 172.21.15.37:0/2802204193 wait complete. 2024-06-10T00:53:09.352 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-10T00:53:09.352 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-06-10T00:53:09.353 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-06-10T00:53:09.612 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:53:10.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:10 smithi094 ceph-mon[36708]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:10.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:10 smithi037 ceph-mon[32192]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:10.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:10 smithi167 ceph-mon[36917]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:11.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.256+0000 7f555bf8a700 1 -- 172.21.15.37:0/563312217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55541084f0 msgr2=0x7f55541088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:11.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.256+0000 7f555bf8a700 1 --2- 172.21.15.37:0/563312217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55541084f0 0x7f55541088d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f554c009990 tx=0x7f554c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:53:11.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.256+0000 7f555bf8a700 1 -- 172.21.15.37:0/563312217 shutdown_connections 2024-06-10T00:53:11.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.256+0000 7f555bf8a700 1 --2- 172.21.15.37:0/563312217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5554102e80 0x7f555410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:11.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.256+0000 7f555bf8a700 1 --2- 172.21.15.37:0/563312217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55541084f0 0x7f55541088d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:11.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.256+0000 7f555bf8a700 1 --2- 172.21.15.37:0/563312217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55541024e0 0x7f5554102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:11.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.256+0000 7f555bf8a700 1 -- 172.21.15.37:0/563312217 >> 172.21.15.37:0/563312217 conn(0x7f5554079530 msgr2=0x7f5554079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:11.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.256+0000 7f555bf8a700 1 -- 172.21.15.37:0/563312217 shutdown_connections 2024-06-10T00:53:11.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.256+0000 7f555bf8a700 1 -- 172.21.15.37:0/563312217 wait complete. 2024-06-10T00:53:11.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.257+0000 7f555bf8a700 1 Processor -- start 2024-06-10T00:53:11.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.257+0000 7f555bf8a700 1 -- start start 2024-06-10T00:53:11.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.257+0000 7f555bf8a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55541024e0 0x7f555419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:11.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.257+0000 7f555bf8a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5554102e80 0x7f555419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:11.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.258+0000 7f555bf8a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55541084f0 0x7f5554198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:11.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.258+0000 7f555bf8a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5554112750 con 0x7f5554102e80 2024-06-10T00:53:11.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.258+0000 7f555bf8a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f55541125d0 con 0x7f55541024e0 2024-06-10T00:53:11.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.258+0000 7f555bf8a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f55541128d0 con 0x7f55541084f0 2024-06-10T00:53:11.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.258+0000 7f5559525700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5554102e80 0x7f555419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:11.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.258+0000 7f5559525700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5554102e80 0x7f555419edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56884/0 (socket says 172.21.15.37:56884) 2024-06-10T00:53:11.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.258+0000 7f5559525700 1 -- 172.21.15.37:0/1527876269 learned_addr learned my addr 172.21.15.37:0/1527876269 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:53:11.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.258+0000 7f555a527700 1 --2- 172.21.15.37:0/1527876269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55541084f0 0x7f5554198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:11.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.258+0000 7f5559d26700 1 --2- 172.21.15.37:0/1527876269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55541024e0 0x7f555419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:11.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.258+0000 7f5559525700 1 -- 172.21.15.37:0/1527876269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55541084f0 msgr2=0x7f5554198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:11.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.258+0000 7f5559525700 1 --2- 172.21.15.37:0/1527876269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55541084f0 0x7f5554198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:11.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.258+0000 7f5559525700 1 -- 172.21.15.37:0/1527876269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55541024e0 msgr2=0x7f555419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:11.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.258+0000 7f5559525700 1 --2- 172.21.15.37:0/1527876269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55541024e0 0x7f555419e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:11.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.258+0000 7f5559525700 1 -- 172.21.15.37:0/1527876269 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55541991d0 con 0x7f5554102e80 2024-06-10T00:53:11.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.258+0000 7f5559d26700 1 --2- 172.21.15.37:0/1527876269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55541024e0 0x7f555419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:53:11.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.258+0000 7f5559525700 1 --2- 172.21.15.37:0/1527876269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5554102e80 0x7f555419edf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f554800c970 tx=0x7f554800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:11.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.258+0000 7f5552ffd700 1 -- 172.21.15.37:0/1527876269 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5548007aa0 con 0x7f5554102e80 2024-06-10T00:53:11.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.258+0000 7f5552ffd700 1 -- 172.21.15.37:0/1527876269 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5548004d10 con 0x7f5554102e80 2024-06-10T00:53:11.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.258+0000 7f555bf8a700 1 -- 172.21.15.37:0/1527876269 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55541994c0 con 0x7f5554102e80 2024-06-10T00:53:11.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.258+0000 7f5552ffd700 1 -- 172.21.15.37:0/1527876269 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5548005690 con 0x7f5554102e80 2024-06-10T00:53:11.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.258+0000 7f555bf8a700 1 -- 172.21.15.37:0/1527876269 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55541a5780 con 0x7f5554102e80 2024-06-10T00:53:11.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.259+0000 7f555bf8a700 1 -- 172.21.15.37:0/1527876269 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5554069180 con 0x7f5554102e80 2024-06-10T00:53:11.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.261+0000 7f5552ffd700 1 -- 172.21.15.37:0/1527876269 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5548004750 con 0x7f5554102e80 2024-06-10T00:53:11.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.261+0000 7f5552ffd700 1 --2- 172.21.15.37:0/1527876269 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f553c0a2ca0 0x7f553c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:11.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.261+0000 7f5559d26700 1 --2- 172.21.15.37:0/1527876269 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f553c0a2ca0 0x7f553c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:11.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.262+0000 7f5552ffd700 1 -- 172.21.15.37:0/1527876269 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f55480cf040 con 0x7f5554102e80 2024-06-10T00:53:11.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.262+0000 7f5559d26700 1 --2- 172.21.15.37:0/1527876269 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f553c0a2ca0 0x7f553c0a5160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f554c000c00 tx=0x7f554c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:11.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.266+0000 7f5552ffd700 1 -- 172.21.15.37:0/1527876269 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f554809bdf0 con 0x7f5554102e80 2024-06-10T00:53:11.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.520+0000 7f555bf8a700 1 -- 172.21.15.37:0/1527876269 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f555402dca0 con 0x7f5554102e80 2024-06-10T00:53:11.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.520+0000 7f5552ffd700 1 -- 172.21.15.37:0/1527876269 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f554809b6f0 con 0x7f5554102e80 2024-06-10T00:53:11.521 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-06-10T00:53:11.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.522+0000 7f555bf8a700 1 -- 172.21.15.37:0/1527876269 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f553c0a2ca0 msgr2=0x7f553c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:11.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.522+0000 7f555bf8a700 1 --2- 172.21.15.37:0/1527876269 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f553c0a2ca0 0x7f553c0a5160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f554c000c00 tx=0x7f554c005fb0 comp rx=0 tx=0).stop 2024-06-10T00:53:11.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.522+0000 7f555bf8a700 1 -- 172.21.15.37:0/1527876269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5554102e80 msgr2=0x7f555419edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:11.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.522+0000 7f555bf8a700 1 --2- 172.21.15.37:0/1527876269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5554102e80 0x7f555419edf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f554800c970 tx=0x7f554800cd30 comp rx=0 tx=0).stop 2024-06-10T00:53:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.522+0000 7f555bf8a700 1 -- 172.21.15.37:0/1527876269 shutdown_connections 2024-06-10T00:53:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.522+0000 7f555bf8a700 1 --2- 172.21.15.37:0/1527876269 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f553c0a2ca0 0x7f553c0a5160 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.522+0000 7f555bf8a700 1 --2- 172.21.15.37:0/1527876269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55541084f0 0x7f5554198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.522+0000 7f555bf8a700 1 --2- 172.21.15.37:0/1527876269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55541024e0 0x7f555419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.522+0000 7f555bf8a700 1 --2- 172.21.15.37:0/1527876269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5554102e80 0x7f555419edf0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.522+0000 7f555bf8a700 1 -- 172.21.15.37:0/1527876269 >> 172.21.15.37:0/1527876269 conn(0x7f5554079530 msgr2=0x7f5554100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.523+0000 7f555bf8a700 1 -- 172.21.15.37:0/1527876269 shutdown_connections 2024-06-10T00:53:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:11.523+0000 7f555bf8a700 1 -- 172.21.15.37:0/1527876269 wait complete. 2024-06-10T00:53:12.187 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-06-10T00:53:12.188 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-10T00:53:12.188 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-06-10T00:53:12.188 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-06-10T00:53:12.188 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-06-10T00:53:12.188 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-06-10T00:53:12.188 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-10T00:53:12.188 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-10T00:53:12.188 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-06-10T00:53:12.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:53:12.188 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-06-10T00:53:12.207 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-10T00:53:12.207 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-10T00:53:12.207 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-06-10T00:53:12.207 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-06-10T00:53:12.207 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-06-10T00:53:12.207 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-06-10T00:53:12.207 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-10T00:53:12.207 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-06-10T00:53:12.208 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-06-10T00:53:12.208 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-10T00:53:12.208 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-06-10T00:53:12.208 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-06-10T00:53:12.208 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-10T00:53:12.208 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-06-10T00:53:12.208 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-06-10T00:53:12.208 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-06-10T00:53:12.208 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42194sec preferred_lft 42194sec 2024-06-10T00:53:12.208 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link noprefixroute 2024-06-10T00:53:12.208 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-06-10T00:53:12.208 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-10T00:53:12.208 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-06-10T00:53:12.208 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-06-10T00:53:12.209 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-10T00:53:12.209 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T00:53:12.209 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-06-10T00:53:12.209 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add name ceph-brx type bridge 2024-06-10T00:53:12.209 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr flush dev ceph-brx 2024-06-10T00:53:12.209 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set ceph-brx up 2024-06-10T00:53:12.210 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-10T00:53:12.210 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-06-10T00:53:12.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T00:53:12.364 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:12 smithi037 ceph-mon[32192]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:12.364 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:12 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1527876269' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-10T00:53:12.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:12 smithi094 ceph-mon[36708]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:12.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:12 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1527876269' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-10T00:53:12.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:53:12.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:53:12.420 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-10T00:53:12.446 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-06-10T00:53:12.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:53:12.448 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-06-10T00:53:12.524 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-06-10T00:53:12.525 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-10T00:53:12.525 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-06-10T00:53:12.525 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-06-10T00:53:12.525 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-10T00:53:12.525 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T00:53:12.525 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-06-10T00:53:12.525 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-06-10T00:53:12.526 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-06-10T00:53:12.526 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-06-10T00:53:12.526 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-06-10T00:53:12.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:12 smithi167 ceph-mon[36917]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:12.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:12 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1527876269' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-10T00:53:12.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T00:53:12.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:53:12.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:53:12.821 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-06-10T00:53:12.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:53:12.837 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list-id 2024-06-10T00:53:12.893 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T00:53:12.893 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-06-10T00:53:12.894 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-10T00:53:12.894 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-06-10T00:53:12.894 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-06-10T00:53:12.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T00:53:13.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:53:13.014 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-06-10T00:53:13.015 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T00:53:13.015 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-06-10T00:53:13.015 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-06-10T00:53:13.015 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-10T00:53:13.015 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-06-10T00:53:13.015 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-06-10T00:53:13.015 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-06-10T00:53:13.015 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-06-10T00:53:13.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T00:53:13.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:53:13.147 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T00:53:13.147 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-06-10T00:53:13.147 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-06-10T00:53:13.147 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set dev brx.0 master ceph-brx 2024-06-10T00:53:13.147 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-06-10T00:53:13.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T00:53:13.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:53:13.229 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-06-10T00:53:13.229 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-10T00:53:13.229 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-06-10T00:53:13.245 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-06-10T00:53:13.245 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-10T00:53:13.245 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-06-10T00:53:13.301 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-06-10T00:53:13.368 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-06-10T00:53:13.425 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-06-10T00:53:13.425 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-06-10T00:53:13.425 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-06-10T00:53:13.425 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-06-10T00:53:13.425 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-06-10T00:53:13.425 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-06-10T00:53:13.425 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-06-10T00:53:13.426 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-06-10T00:53:13.426 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-06-10T00:53:13.426 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-06-10T00:53:13.426 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-06-10T00:53:13.426 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/selinux 2024-06-10T00:53:13.426 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-06-10T00:53:13.426 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-06-10T00:53:13.426 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-06-10T00:53:13.426 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-06-10T00:53:13.426 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-06-10T00:53:13.427 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-10T00:53:13.427 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-06-10T00:53:13.427 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysctl.service 2024-06-10T00:53:13.427 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-06-10T00:53:13.427 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-10T00:53:13.427 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/nfs/rpc_pipefs 2024-06-10T00:53:13.427 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-06-10T00:53:13.427 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-06-10T00:53:13.427 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay 2024-06-10T00:53:13.427 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/72917fc4fee6157b8b6a0eba1bea40127e1343f851b3aa47a0130217bf6d0133/merged 2024-06-10T00:53:13.427 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/b82c3a83afe005e7dcbc5a3602cc27fdbe2a32a1be1362e2bf0a61254a6da392/merged 2024-06-10T00:53:13.427 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/0 2024-06-10T00:53:13.427 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-06-10T00:53:13.428 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/aa501ada4d39d1720ec9dd405b503da8def30043a3ac4ae8418e40934a917bdd/merged 2024-06-10T00:53:13.428 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/78504ec97d528fb1ec24a1049be338f6803877526d40795823c55652465dacd5/merged 2024-06-10T00:53:13.428 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/9eb4b48744bd472142e43a97d10ba70b9d5f62ecb8c7bd6c68a0968f9156c735/merged 2024-06-10T00:53:13.428 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/9a576aba0a8fd3f77e135cbd271e5d92716de545f053cc548581ffbd189cb41e/merged 2024-06-10T00:53:13.428 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/aa368ccca5bf4ed7574503daef34fe807e89f2001244e2e14262f12196c33c7e/merged 2024-06-10T00:53:13.428 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/66d66b5fdf853912e3ea32ae06dad5909d6c06fe121e6e997bb6347eacd097c2/merged 2024-06-10T00:53:13.428 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/00e2e5150eb6e25db344ee6a8c032b156500eb01d194d65a47c30ba656fb5328/merged 2024-06-10T00:53:13.428 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/ee65ab3b1ea21a5b86cbd1f9ade498928da7c0ee71a7f7ba9d13f95142e95646/merged 2024-06-10T00:53:13.428 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-06-10T00:53:13.428 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-10T00:53:13.428 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-10T00:53:13.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:53:13.429 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-06-10T00:53:13.481 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-06-10T00:53:13.481 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-06-10T00:53:13.524 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-06-10T00:53:13.549 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-06-10T00:53:13.606 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-06-10T00:53:13.606 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-06-10T00:53:13.606 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-06-10T00:53:13.606 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-06-10T00:53:13.606 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-06-10T00:53:13.606 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-06-10T00:53:13.606 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-06-10T00:53:13.606 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-06-10T00:53:13.606 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-06-10T00:53:13.606 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-06-10T00:53:13.607 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-06-10T00:53:13.607 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/selinux 2024-06-10T00:53:13.607 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-06-10T00:53:13.607 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-06-10T00:53:13.607 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-06-10T00:53:13.607 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-06-10T00:53:13.607 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-06-10T00:53:13.607 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-10T00:53:13.607 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-06-10T00:53:13.607 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysctl.service 2024-06-10T00:53:13.607 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-06-10T00:53:13.607 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-10T00:53:13.607 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/nfs/rpc_pipefs 2024-06-10T00:53:13.608 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-06-10T00:53:13.608 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-06-10T00:53:13.608 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay 2024-06-10T00:53:13.608 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/72917fc4fee6157b8b6a0eba1bea40127e1343f851b3aa47a0130217bf6d0133/merged 2024-06-10T00:53:13.608 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/b82c3a83afe005e7dcbc5a3602cc27fdbe2a32a1be1362e2bf0a61254a6da392/merged 2024-06-10T00:53:13.608 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/0 2024-06-10T00:53:13.608 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-06-10T00:53:13.608 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/aa501ada4d39d1720ec9dd405b503da8def30043a3ac4ae8418e40934a917bdd/merged 2024-06-10T00:53:13.608 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/78504ec97d528fb1ec24a1049be338f6803877526d40795823c55652465dacd5/merged 2024-06-10T00:53:13.608 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/9eb4b48744bd472142e43a97d10ba70b9d5f62ecb8c7bd6c68a0968f9156c735/merged 2024-06-10T00:53:13.608 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/9a576aba0a8fd3f77e135cbd271e5d92716de545f053cc548581ffbd189cb41e/merged 2024-06-10T00:53:13.608 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/aa368ccca5bf4ed7574503daef34fe807e89f2001244e2e14262f12196c33c7e/merged 2024-06-10T00:53:13.608 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/66d66b5fdf853912e3ea32ae06dad5909d6c06fe121e6e997bb6347eacd097c2/merged 2024-06-10T00:53:13.609 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/00e2e5150eb6e25db344ee6a8c032b156500eb01d194d65a47c30ba656fb5328/merged 2024-06-10T00:53:13.609 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/containers/storage/overlay/ee65ab3b1ea21a5b86cbd1f9ade498928da7c0ee71a7f7ba9d13f95142e95646/merged 2024-06-10T00:53:13.609 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-06-10T00:53:13.609 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-10T00:53:13.609 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-10T00:53:13.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:53:13.610 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-06-10T00:53:13.630 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi037.stderr:2024-06-10T00:53:13.629+0000 7fe9c314e480 -1 init, newargv = 0x55cf1cfc3a90 newargc=15 2024-06-10T00:53:13.630 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi037.stderr:ceph-fuse[76315]: starting ceph client 2024-06-10T00:53:13.636 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi037.stderr:ceph-fuse[76315]: starting fuse 2024-06-10T00:53:13.646 INFO:teuthology.orchestra.run.smithi037.stdout:119 2024-06-10T00:53:13.647 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-06-10T00:53:13.647 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-06-10T00:53:13.647 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-06-10T00:53:13.647 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-06-10T00:53:13.647 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-06-10T00:53:13.647 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-06-10T00:53:13.647 DEBUG:teuthology.orchestra.run.smithi037:> 2024-06-10T00:53:13.647 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-06-10T00:53:13.647 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-06-10T00:53:13.647 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-06-10T00:53:13.647 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-06-10T00:53:13.648 DEBUG:teuthology.orchestra.run.smithi037:> 2024-06-10T00:53:13.648 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-06-10T00:53:13.648 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-06-10T00:53:13.648 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-06-10T00:53:13.648 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-06-10T00:53:13.648 DEBUG:teuthology.orchestra.run.smithi037:> 2024-06-10T00:53:13.648 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-06-10T00:53:13.648 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-06-10T00:53:13.648 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-06-10T00:53:13.648 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-06-10T00:53:13.648 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-06-10T00:53:13.648 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-06-10T00:53:13.648 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-06-10T00:53:13.648 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-06-10T00:53:13.648 DEBUG:teuthology.orchestra.run.smithi037:> 2024-06-10T00:53:13.648 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.admin")) 2024-06-10T00:53:13.649 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-06-10T00:53:13.768 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.admin.76315.asok 2024-06-10T00:53:13.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:53:13.778 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.76315.asok 2024-06-10T00:53:13.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:53:13.779 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.76315.asok status 2024-06-10T00:53:13.918 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T00:53:13.918 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-06-10T00:53:13.918 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-10T00:53:13.918 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-10T00:53:13.918 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "admin", 2024-06-10T00:53:13.918 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-06-10T00:53:13.918 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-06-10T00:53:13.918 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "76315", 2024-06-10T00:53:13.919 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-06-10T00:53:13.919 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T00:53:13.919 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-06-10T00:53:13.919 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-06-10T00:53:13.919 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 14724, 2024-06-10T00:53:13.919 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-06-10T00:53:13.919 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-06-10T00:53:13.919 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-06-10T00:53:13.919 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 14724 2024-06-10T00:53:13.919 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T00:53:13.919 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-06-10T00:53:13.919 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-06-10T00:53:13.919 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-06-10T00:53:13.920 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 605484967 2024-06-10T00:53:13.920 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T00:53:13.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T00:53:13.920 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-06-10T00:53:13.920 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-06-10T00:53:13.920 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-06-10T00:53:13.920 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 605484967 2024-06-10T00:53:13.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T00:53:13.920 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.14724 192.168.0.1:0/605484967", 2024-06-10T00:53:13.920 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.1:0/605484967", 2024-06-10T00:53:13.920 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-06-10T00:53:13.920 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 17, 2024-06-10T00:53:13.920 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 96, 2024-06-10T00:53:13.920 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-06-10T00:53:13.921 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-06-10T00:53:13.921 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs" 2024-06-10T00:53:13.921 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T00:53:13.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:53:13.926 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-06-10T00:53:13.926 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-06-10T00:53:13.943 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-06-10T00:53:13.943 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-06-10T00:53:13.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:53:13.943 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-06-10T00:53:14.010 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-06-10T00:53:14.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-10T00:53:14.133 INFO:teuthology.orchestra.run.smithi037.stdout:# file: . 2024-06-10T00:53:14.133 INFO:teuthology.orchestra.run.smithi037.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-06-10T00:53:14.133 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:53:14.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:53:14.139 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-06-10T00:53:14.139 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-10T00:53:14.156 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-10T00:53:14.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:14 smithi094 ceph-mon[36708]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:14.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:14 smithi037 ceph-mon[32192]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:14.446 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:53:14.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:14 smithi167 ceph-mon[36917]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.113+0000 7fd6d8caf700 1 -- 172.21.15.37:0/2536077554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d4103320 msgr2=0x7fd6d4103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.113+0000 7fd6d8caf700 1 --2- 172.21.15.37:0/2536077554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d4103320 0x7fd6d4103700 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4009990 tx=0x7fd6c4009ca0 comp rx=0 tx=0).stop 2024-06-10T00:53:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.113+0000 7fd6d8caf700 1 -- 172.21.15.37:0/2536077554 shutdown_connections 2024-06-10T00:53:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.113+0000 7fd6d8caf700 1 --2- 172.21.15.37:0/2536077554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6d4101d80 0x7fd6d410e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.113+0000 7fd6d8caf700 1 --2- 172.21.15.37:0/2536077554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d4103320 0x7fd6d4103700 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.113+0000 7fd6d8caf700 1 --2- 172.21.15.37:0/2536077554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6d41013c0 0x7fd6d4101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.113+0000 7fd6d8caf700 1 -- 172.21.15.37:0/2536077554 >> 172.21.15.37:0/2536077554 conn(0x7fd6d40773b0 msgr2=0x7fd6d40777c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.113+0000 7fd6d8caf700 1 -- 172.21.15.37:0/2536077554 shutdown_connections 2024-06-10T00:53:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.113+0000 7fd6d8caf700 1 -- 172.21.15.37:0/2536077554 wait complete. 2024-06-10T00:53:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.114+0000 7fd6d8caf700 1 Processor -- start 2024-06-10T00:53:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.114+0000 7fd6d8caf700 1 -- start start 2024-06-10T00:53:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.115+0000 7fd6d8caf700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6d41013c0 0x7fd6d4104180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.115+0000 7fd6d8caf700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d4101d80 0x7fd6d41046c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.115+0000 7fd6d8caf700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6d4103320 0x7fd6d4104c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.115+0000 7fd6d8caf700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d4111790 con 0x7fd6d4101d80 2024-06-10T00:53:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.115+0000 7fd6d8caf700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d4111610 con 0x7fd6d41013c0 2024-06-10T00:53:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.115+0000 7fd6d8caf700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d4111910 con 0x7fd6d4103320 2024-06-10T00:53:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.115+0000 7fd6c9d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d4101d80 0x7fd6d41046c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.115+0000 7fd6c9d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d4101d80 0x7fd6d41046c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43518/0 (socket says 172.21.15.37:43518) 2024-06-10T00:53:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.115+0000 7fd6c9d9b700 1 -- 172.21.15.37:0/3748880580 learned_addr learned my addr 172.21.15.37:0/3748880580 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:53:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.115+0000 7fd6d2d9d700 1 --2- 172.21.15.37:0/3748880580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6d4103320 0x7fd6d4104c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.115+0000 7fd6d259c700 1 --2- 172.21.15.37:0/3748880580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6d41013c0 0x7fd6d4104180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.115+0000 7fd6c9d9b700 1 -- 172.21.15.37:0/3748880580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6d4103320 msgr2=0x7fd6d4104c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.115+0000 7fd6c9d9b700 1 --2- 172.21.15.37:0/3748880580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6d4103320 0x7fd6d4104c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.115+0000 7fd6c9d9b700 1 -- 172.21.15.37:0/3748880580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6d41013c0 msgr2=0x7fd6d4104180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.115+0000 7fd6c9d9b700 1 --2- 172.21.15.37:0/3748880580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6d41013c0 0x7fd6d4104180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.115+0000 7fd6c9d9b700 1 -- 172.21.15.37:0/3748880580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d41a7240 con 0x7fd6d4101d80 2024-06-10T00:53:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.115+0000 7fd6d259c700 1 --2- 172.21.15.37:0/3748880580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6d41013c0 0x7fd6d4104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:53:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.115+0000 7fd6c9d9b700 1 --2- 172.21.15.37:0/3748880580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d4101d80 0x7fd6d41046c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00c920 tx=0x7fd6bc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.116+0000 7fd6cbfff700 1 -- 172.21.15.37:0/3748880580 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc007b00 con 0x7fd6d4101d80 2024-06-10T00:53:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.116+0000 7fd6d8caf700 1 -- 172.21.15.37:0/3748880580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d41a7530 con 0x7fd6d4101d80 2024-06-10T00:53:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.116+0000 7fd6cbfff700 1 -- 172.21.15.37:0/3748880580 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6bc004d10 con 0x7fd6d4101d80 2024-06-10T00:53:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.116+0000 7fd6cbfff700 1 -- 172.21.15.37:0/3748880580 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc005630 con 0x7fd6d4101d80 2024-06-10T00:53:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.116+0000 7fd6d8caf700 1 -- 172.21.15.37:0/3748880580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d41a7ac0 con 0x7fd6d4101d80 2024-06-10T00:53:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.117+0000 7fd6d8caf700 1 -- 172.21.15.37:0/3748880580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6d4069180 con 0x7fd6d4101d80 2024-06-10T00:53:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.118+0000 7fd6cbfff700 1 -- 172.21.15.37:0/3748880580 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd6bc004750 con 0x7fd6d4101d80 2024-06-10T00:53:16.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.119+0000 7fd6cbfff700 1 --2- 172.21.15.37:0/3748880580 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd6c00a2c20 0x7fd6c00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:16.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.119+0000 7fd6d259c700 1 --2- 172.21.15.37:0/3748880580 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd6c00a2c20 0x7fd6c00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:16.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.119+0000 7fd6cbfff700 1 -- 172.21.15.37:0/3748880580 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7fd6bc0cf2b0 con 0x7fd6d4101d80 2024-06-10T00:53:16.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.119+0000 7fd6d259c700 1 --2- 172.21.15.37:0/3748880580 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd6c00a2c20 0x7fd6c00a50e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4006010 tx=0x7fd6c400b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:16.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.122+0000 7fd6cbfff700 1 -- 172.21.15.37:0/3748880580 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd6bc014030 con 0x7fd6d4101d80 2024-06-10T00:53:16.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:16 smithi037 ceph-mon[32192]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:16.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.344+0000 7fd6d8caf700 1 -- 172.21.15.37:0/3748880580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd6d41a7d80 con 0x7fd6d4101d80 2024-06-10T00:53:16.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.344+0000 7fd6cbfff700 1 -- 172.21.15.37:0/3748880580 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v96) ==== 81+0+591 (secure 0 0 0) 0x7fd6bc09bf70 con 0x7fd6d4101d80 2024-06-10T00:53:16.345 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/4084722397 2024-06-11T00:45:20.254342+0000 2024-06-10T00:53:16.345 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3571803585 2024-06-11T00:45:20.254342+0000 2024-06-10T00:53:16.345 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/416587443 2024-06-11T00:45:20.254342+0000 2024-06-10T00:53:16.345 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/2766692263 2024-06-11T00:45:20.254342+0000 2024-06-10T00:53:16.345 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2026344134 2024-06-11T00:44:53.913969+0000 2024-06-10T00:53:16.345 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2781321886 2024-06-11T00:44:53.913969+0000 2024-06-10T00:53:16.345 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/2803174880 2024-06-11T00:44:53.913969+0000 2024-06-10T00:53:16.345 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/2803174880 2024-06-11T00:44:53.913969+0000 2024-06-10T00:53:16.345 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1110143997 2024-06-11T00:44:53.913969+0000 2024-06-10T00:53:16.346 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/2766692263 2024-06-11T00:45:20.254342+0000 2024-06-10T00:53:16.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.346+0000 7fd6d8caf700 1 -- 172.21.15.37:0/3748880580 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd6c00a2c20 msgr2=0x7fd6c00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:16.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.346+0000 7fd6d8caf700 1 --2- 172.21.15.37:0/3748880580 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd6c00a2c20 0x7fd6c00a50e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4006010 tx=0x7fd6c400b540 comp rx=0 tx=0).stop 2024-06-10T00:53:16.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.346+0000 7fd6d8caf700 1 -- 172.21.15.37:0/3748880580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d4101d80 msgr2=0x7fd6d41046c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:16.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.346+0000 7fd6d8caf700 1 --2- 172.21.15.37:0/3748880580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d4101d80 0x7fd6d41046c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00c920 tx=0x7fd6bc00cc30 comp rx=0 tx=0).stop 2024-06-10T00:53:16.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.346+0000 7fd6d8caf700 1 -- 172.21.15.37:0/3748880580 shutdown_connections 2024-06-10T00:53:16.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.346+0000 7fd6d8caf700 1 --2- 172.21.15.37:0/3748880580 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd6c00a2c20 0x7fd6c00a50e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:16.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.346+0000 7fd6d8caf700 1 --2- 172.21.15.37:0/3748880580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6d4103320 0x7fd6d4104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:16.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.346+0000 7fd6d8caf700 1 --2- 172.21.15.37:0/3748880580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6d41013c0 0x7fd6d4104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:16.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.346+0000 7fd6d8caf700 1 --2- 172.21.15.37:0/3748880580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d4101d80 0x7fd6d41046c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:16.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.346+0000 7fd6d8caf700 1 -- 172.21.15.37:0/3748880580 >> 172.21.15.37:0/3748880580 conn(0x7fd6d40773b0 msgr2=0x7fd6d406ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:16.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.347+0000 7fd6d8caf700 1 -- 172.21.15.37:0/3748880580 shutdown_connections 2024-06-10T00:53:16.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:16.347+0000 7fd6d8caf700 1 -- 172.21.15.37:0/3748880580 wait complete. 2024-06-10T00:53:16.348 INFO:teuthology.orchestra.run.smithi037.stderr:listed 10 entries 2024-06-10T00:53:16.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:16 smithi094 ceph-mon[36708]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:16.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:16 smithi167 ceph-mon[36917]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:16.996 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-06-10T00:53:16.997 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-10T00:53:17.015 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-10T00:53:17.306 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:53:17.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:17 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3748880580' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T00:53:17.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:17 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3748880580' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T00:53:17.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:17 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3748880580' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T00:53:18.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:18 smithi094 ceph-mon[36708]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-10T00:53:18.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:18 smithi037 ceph-mon[32192]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-10T00:53:18.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:18 smithi167 ceph-mon[36917]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-10T00:53:18.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.923+0000 7f8c39036700 1 -- 172.21.15.37:0/1204854364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c34107390 msgr2=0x7f8c34107770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:18.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.923+0000 7f8c39036700 1 --2- 172.21.15.37:0/1204854364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c34107390 0x7f8c34107770 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8c240099c0 tx=0x7f8c24009cd0 comp rx=0 tx=0).stop 2024-06-10T00:53:18.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.923+0000 7f8c39036700 1 -- 172.21.15.37:0/1204854364 shutdown_connections 2024-06-10T00:53:18.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.923+0000 7f8c39036700 1 --2- 172.21.15.37:0/1204854364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c3406b5b0 0x7f8c3410f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:18.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.923+0000 7f8c39036700 1 --2- 172.21.15.37:0/1204854364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c34107390 0x7f8c34107770 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:18.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.923+0000 7f8c39036700 1 --2- 172.21.15.37:0/1204854364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c3406abf0 0x7f8c3406b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:18.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.923+0000 7f8c39036700 1 -- 172.21.15.37:0/1204854364 >> 172.21.15.37:0/1204854364 conn(0x7f8c34078340 msgr2=0x7f8c34078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:18.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.923+0000 7f8c39036700 1 -- 172.21.15.37:0/1204854364 shutdown_connections 2024-06-10T00:53:18.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.923+0000 7f8c39036700 1 -- 172.21.15.37:0/1204854364 wait complete. 2024-06-10T00:53:18.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.924+0000 7f8c39036700 1 Processor -- start 2024-06-10T00:53:18.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.924+0000 7f8c39036700 1 -- start start 2024-06-10T00:53:18.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.924+0000 7f8c39036700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c3406abf0 0x7f8c3419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:18.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.924+0000 7f8c39036700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c3406b5b0 0x7f8c3419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:18.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.924+0000 7f8c39036700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c34107390 0x7f8c34198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:18.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.924+0000 7f8c39036700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8c34112740 con 0x7f8c3406abf0 2024-06-10T00:53:18.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.924+0000 7f8c39036700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8c341125c0 con 0x7f8c3406b5b0 2024-06-10T00:53:18.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.924+0000 7f8c39036700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8c341128c0 con 0x7f8c34107390 2024-06-10T00:53:18.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.924+0000 7f8c32d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c3406abf0 0x7f8c3419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:18.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.924+0000 7f8c32d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c3406abf0 0x7f8c3419e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43544/0 (socket says 172.21.15.37:43544) 2024-06-10T00:53:18.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.924+0000 7f8c32d9d700 1 -- 172.21.15.37:0/2903953282 learned_addr learned my addr 172.21.15.37:0/2903953282 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:53:18.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.924+0000 7f8c3259c700 1 --2- 172.21.15.37:0/2903953282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c3406b5b0 0x7f8c3419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:18.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.924+0000 7f8c3359e700 1 --2- 172.21.15.37:0/2903953282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c34107390 0x7f8c34198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:18.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.925+0000 7f8c32d9d700 1 -- 172.21.15.37:0/2903953282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c34107390 msgr2=0x7f8c34198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:18.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.925+0000 7f8c32d9d700 1 --2- 172.21.15.37:0/2903953282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c34107390 0x7f8c34198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:18.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.925+0000 7f8c32d9d700 1 -- 172.21.15.37:0/2903953282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c3406b5b0 msgr2=0x7f8c3419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:18.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.925+0000 7f8c32d9d700 1 --2- 172.21.15.37:0/2903953282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c3406b5b0 0x7f8c3419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:18.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.925+0000 7f8c32d9d700 1 -- 172.21.15.37:0/2903953282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c341991a0 con 0x7f8c3406abf0 2024-06-10T00:53:18.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.925+0000 7f8c32d9d700 1 --2- 172.21.15.37:0/2903953282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c3406abf0 0x7f8c3419e830 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8c24006010 tx=0x7f8c24009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:18.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.925+0000 7f8c2bfff700 1 -- 172.21.15.37:0/2903953282 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c24026070 con 0x7f8c3406abf0 2024-06-10T00:53:18.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.925+0000 7f8c39036700 1 -- 172.21.15.37:0/2903953282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c34199430 con 0x7f8c3406abf0 2024-06-10T00:53:18.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.925+0000 7f8c2bfff700 1 -- 172.21.15.37:0/2903953282 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c24004cd0 con 0x7f8c3406abf0 2024-06-10T00:53:18.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.925+0000 7f8c2bfff700 1 -- 172.21.15.37:0/2903953282 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c24003ba0 con 0x7f8c3406abf0 2024-06-10T00:53:18.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.925+0000 7f8c39036700 1 -- 172.21.15.37:0/2903953282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c341a5700 con 0x7f8c3406abf0 2024-06-10T00:53:18.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.926+0000 7f8c39036700 1 -- 172.21.15.37:0/2903953282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c34069180 con 0x7f8c3406abf0 2024-06-10T00:53:18.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.927+0000 7f8c2bfff700 1 -- 172.21.15.37:0/2903953282 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8c24029070 con 0x7f8c3406abf0 2024-06-10T00:53:18.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.928+0000 7f8c2bfff700 1 --2- 172.21.15.37:0/2903953282 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8c200a2c20 0x7f8c200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:18.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.928+0000 7f8c2bfff700 1 -- 172.21.15.37:0/2903953282 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f8c240cffb0 con 0x7f8c3406abf0 2024-06-10T00:53:18.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.928+0000 7f8c3259c700 1 --2- 172.21.15.37:0/2903953282 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8c200a2c20 0x7f8c200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:18.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.928+0000 7f8c3259c700 1 --2- 172.21.15.37:0/2903953282 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8c200a2c20 0x7f8c200a50e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8c340788b0 tx=0x7f8c1c00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:18.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:18.931+0000 7f8c2bfff700 1 -- 172.21.15.37:0/2903953282 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8c24030020 con 0x7f8c3406abf0 2024-06-10T00:53:19.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:19.155+0000 7f8c39036700 1 -- 172.21.15.37:0/2903953282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8c3402dca0 con 0x7f8c3406abf0 2024-06-10T00:53:19.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:19.156+0000 7f8c2bfff700 1 -- 172.21.15.37:0/2903953282 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v96) ==== 81+0+591 (secure 0 0 0) 0x7f8c24013070 con 0x7f8c3406abf0 2024-06-10T00:53:19.157 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/4084722397 2024-06-11T00:45:20.254342+0000 2024-06-10T00:53:19.157 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3571803585 2024-06-11T00:45:20.254342+0000 2024-06-10T00:53:19.158 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/416587443 2024-06-11T00:45:20.254342+0000 2024-06-10T00:53:19.158 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/2766692263 2024-06-11T00:45:20.254342+0000 2024-06-10T00:53:19.158 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2026344134 2024-06-11T00:44:53.913969+0000 2024-06-10T00:53:19.158 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2781321886 2024-06-11T00:44:53.913969+0000 2024-06-10T00:53:19.158 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/2803174880 2024-06-11T00:44:53.913969+0000 2024-06-10T00:53:19.158 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/2803174880 2024-06-11T00:44:53.913969+0000 2024-06-10T00:53:19.158 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1110143997 2024-06-11T00:44:53.913969+0000 2024-06-10T00:53:19.158 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/2766692263 2024-06-11T00:45:20.254342+0000 2024-06-10T00:53:19.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:19.158+0000 7f8c39036700 1 -- 172.21.15.37:0/2903953282 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8c200a2c20 msgr2=0x7f8c200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:19.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:19.158+0000 7f8c39036700 1 --2- 172.21.15.37:0/2903953282 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8c200a2c20 0x7f8c200a50e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8c340788b0 tx=0x7f8c1c00b410 comp rx=0 tx=0).stop 2024-06-10T00:53:19.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:19.158+0000 7f8c39036700 1 -- 172.21.15.37:0/2903953282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c3406abf0 msgr2=0x7f8c3419e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:19.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:19.158+0000 7f8c39036700 1 --2- 172.21.15.37:0/2903953282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c3406abf0 0x7f8c3419e830 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8c24006010 tx=0x7f8c24009f80 comp rx=0 tx=0).stop 2024-06-10T00:53:19.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:19.158+0000 7f8c39036700 1 -- 172.21.15.37:0/2903953282 shutdown_connections 2024-06-10T00:53:19.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:19.158+0000 7f8c39036700 1 --2- 172.21.15.37:0/2903953282 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8c200a2c20 0x7f8c200a50e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:19.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:19.158+0000 7f8c39036700 1 --2- 172.21.15.37:0/2903953282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c34107390 0x7f8c34198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:19.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:19.158+0000 7f8c39036700 1 --2- 172.21.15.37:0/2903953282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c3406abf0 0x7f8c3419e830 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:19.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:19.158+0000 7f8c39036700 1 --2- 172.21.15.37:0/2903953282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c3406b5b0 0x7f8c3419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:19.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:19.158+0000 7f8c39036700 1 -- 172.21.15.37:0/2903953282 >> 172.21.15.37:0/2903953282 conn(0x7f8c34078340 msgr2=0x7f8c340ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:19.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:19.158+0000 7f8c39036700 1 -- 172.21.15.37:0/2903953282 shutdown_connections 2024-06-10T00:53:19.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:19.158+0000 7f8c39036700 1 -- 172.21.15.37:0/2903953282 wait complete. 2024-06-10T00:53:19.161 INFO:teuthology.orchestra.run.smithi037.stderr:listed 10 entries 2024-06-10T00:53:19.801 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-06-10T00:53:19.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:53:19.801 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-06-10T00:53:19.829 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi037.stderr:ceph-fuse[76315]: fuse finished with error 0 and tester_r 0 2024-06-10T00:53:19.831 INFO:teuthology.orchestra.run:waiting for 300 2024-06-10T00:53:20.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:20 smithi094 ceph-mon[36708]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-10T00:53:20.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:20 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2903953282' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T00:53:20.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:20 smithi037 ceph-mon[32192]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-10T00:53:20.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:20 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2903953282' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T00:53:20.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:20 smithi167 ceph-mon[36917]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-10T00:53:20.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:20 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2903953282' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T00:53:21.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-06-10T00:53:21.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-06-10T00:53:21.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-06-10T00:53:21.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-06-10T00:53:21.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-06-10T00:53:21.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-06-10T00:53:21.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-06-10T00:53:21.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-06-10T00:53:21.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-06-10T00:53:21.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2024-06-10T00:53:21.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-06-10T00:53:21.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-06-10T00:53:21.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-06-10T00:53:21.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-06-10T00:53:21.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-06-10T00:53:21.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-06-10T00:53:21.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-06-10T00:53:21.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-06-10T00:53:21.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-06-10T00:53:21.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2024-06-10T00:53:21.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-06-10T00:53:21.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-06-10T00:53:21.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-06-10T00:53:21.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-06-10T00:53:21.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-06-10T00:53:21.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-06-10T00:53:21.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-06-10T00:53:21.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-06-10T00:53:21.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-06-10T00:53:21.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2024-06-10T00:53:22.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:22 smithi037 ceph-mon[32192]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-10T00:53:22.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:22 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-06-10T00:53:22.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:22 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-06-10T00:53:22.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:22 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-06-10T00:53:22.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:22 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-06-10T00:53:22.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:22 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-06-10T00:53:22.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:22 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-06-10T00:53:22.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:22 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-06-10T00:53:22.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:22 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-06-10T00:53:22.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:22 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-06-10T00:53:22.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:22 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-06-10T00:53:22.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:22 smithi037 ceph-mon[32192]: osdmap e97: 12 total, 12 up, 12 in 2024-06-10T00:53:22.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:22 smithi167 ceph-mon[36917]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-10T00:53:22.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:22 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-06-10T00:53:22.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:22 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-06-10T00:53:22.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:22 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-06-10T00:53:22.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:22 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-06-10T00:53:22.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:22 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-06-10T00:53:22.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:22 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-06-10T00:53:22.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:22 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-06-10T00:53:22.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:22 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-06-10T00:53:22.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:22 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-06-10T00:53:22.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:22 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-06-10T00:53:22.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:22 smithi167 ceph-mon[36917]: osdmap e97: 12 total, 12 up, 12 in 2024-06-10T00:53:22.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:22 smithi094 ceph-mon[36708]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-10T00:53:22.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:22 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-06-10T00:53:22.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:22 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-06-10T00:53:22.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:22 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-06-10T00:53:22.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:22 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-06-10T00:53:22.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:22 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-06-10T00:53:22.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:22 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-06-10T00:53:22.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:22 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-06-10T00:53:22.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:22 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-06-10T00:53:22.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:22 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-06-10T00:53:22.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:22 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-06-10T00:53:22.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:22 smithi094 ceph-mon[36708]: osdmap e97: 12 total, 12 up, 12 in 2024-06-10T00:53:23.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:23 smithi037 ceph-mon[32192]: osdmap e98: 12 total, 12 up, 12 in 2024-06-10T00:53:23.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:23 smithi167 ceph-mon[36917]: osdmap e98: 12 total, 12 up, 12 in 2024-06-10T00:53:23.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:23 smithi094 ceph-mon[36708]: osdmap e98: 12 total, 12 up, 12 in 2024-06-10T00:53:24.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:24 smithi037 ceph-mon[32192]: pgmap v329: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s 2024-06-10T00:53:24.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:24 smithi167 ceph-mon[36917]: pgmap v329: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s 2024-06-10T00:53:24.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:24 smithi094 ceph-mon[36708]: pgmap v329: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s 2024-06-10T00:53:25.932 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-06-10T00:53:25.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:53:25.933 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-06-10T00:53:25.964 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T00:53:26.266 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:53:26.389 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:26 smithi037 ceph-mon[32192]: pgmap v330: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:53:26.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:26 smithi167 ceph-mon[36917]: pgmap v330: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:53:26.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:26 smithi094 ceph-mon[36708]: pgmap v330: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:53:27.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.812+0000 7f1ced07e700 1 -- 172.21.15.37:0/3570348702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ce8101f40 msgr2=0x7f1ce81023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:27.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.812+0000 7f1ced07e700 1 --2- 172.21.15.37:0/3570348702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ce8101f40 0x7f1ce81023c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1cd00099c0 tx=0x7f1cd0009cd0 comp rx=0 tx=0).stop 2024-06-10T00:53:27.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.813+0000 7f1ced07e700 1 -- 172.21.15.37:0/3570348702 shutdown_connections 2024-06-10T00:53:27.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.813+0000 7f1ced07e700 1 --2- 172.21.15.37:0/3570348702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ce8102900 0x7f1ce810e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:27.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.813+0000 7f1ced07e700 1 --2- 172.21.15.37:0/3570348702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ce8107660 0x7f1ce8107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:27.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.813+0000 7f1ced07e700 1 --2- 172.21.15.37:0/3570348702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ce8101f40 0x7f1ce81023c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:27.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.813+0000 7f1ced07e700 1 -- 172.21.15.37:0/3570348702 >> 172.21.15.37:0/3570348702 conn(0x7f1ce80766b0 msgr2=0x7f1ce8076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:27.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.813+0000 7f1ced07e700 1 -- 172.21.15.37:0/3570348702 shutdown_connections 2024-06-10T00:53:27.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.813+0000 7f1ced07e700 1 -- 172.21.15.37:0/3570348702 wait complete. 2024-06-10T00:53:27.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.813+0000 7f1ced07e700 1 Processor -- start 2024-06-10T00:53:27.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.814+0000 7f1ced07e700 1 -- start start 2024-06-10T00:53:27.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.814+0000 7f1ced07e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ce8101f40 0x7f1ce81030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:27.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.814+0000 7f1ced07e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ce8102900 0x7f1ce8103600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:27.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.814+0000 7f1ced07e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ce8107660 0x7f1ce8103b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:27.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.814+0000 7f1ced07e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce8111830 con 0x7f1ce8102900 2024-06-10T00:53:27.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.814+0000 7f1ced07e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce81116b0 con 0x7f1ce8101f40 2024-06-10T00:53:27.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.814+0000 7f1ced07e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce81119b0 con 0x7f1ce8107660 2024-06-10T00:53:27.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.814+0000 7f1cde59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ce8102900 0x7f1ce8103600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:27.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.814+0000 7f1cde59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ce8102900 0x7f1ce8103600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35310/0 (socket says 172.21.15.37:35310) 2024-06-10T00:53:27.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.814+0000 7f1cde59c700 1 -- 172.21.15.37:0/646193693 learned_addr learned my addr 172.21.15.37:0/646193693 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:53:27.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.814+0000 7f1ce759e700 1 --2- 172.21.15.37:0/646193693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ce8107660 0x7f1ce8103b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:27.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.814+0000 7f1ce6d9d700 1 --2- 172.21.15.37:0/646193693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ce8101f40 0x7f1ce81030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:27.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.814+0000 7f1cde59c700 1 -- 172.21.15.37:0/646193693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ce8107660 msgr2=0x7f1ce8103b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:27.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.814+0000 7f1cde59c700 1 --2- 172.21.15.37:0/646193693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ce8107660 0x7f1ce8103b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:27.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.814+0000 7f1cde59c700 1 -- 172.21.15.37:0/646193693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ce8101f40 msgr2=0x7f1ce81030c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:27.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.814+0000 7f1cde59c700 1 --2- 172.21.15.37:0/646193693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ce8101f40 0x7f1ce81030c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:27.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.814+0000 7f1cde59c700 1 -- 172.21.15.37:0/646193693 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce81a7230 con 0x7f1ce8102900 2024-06-10T00:53:27.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.815+0000 7f1cde59c700 1 --2- 172.21.15.37:0/646193693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ce8102900 0x7f1ce8103600 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009fd0 tx=0x7f1cd0003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:27.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.815+0000 7f1ce4d99700 1 -- 172.21.15.37:0/646193693 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0027070 con 0x7f1ce8102900 2024-06-10T00:53:27.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.815+0000 7f1ced07e700 1 -- 172.21.15.37:0/646193693 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ce81a74c0 con 0x7f1ce8102900 2024-06-10T00:53:27.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.815+0000 7f1ce4d99700 1 -- 172.21.15.37:0/646193693 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cd0004150 con 0x7f1ce8102900 2024-06-10T00:53:27.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.815+0000 7f1ce4d99700 1 -- 172.21.15.37:0/646193693 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd000f460 con 0x7f1ce8102900 2024-06-10T00:53:27.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.815+0000 7f1ced07e700 1 -- 172.21.15.37:0/646193693 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce81a7970 con 0x7f1ce8102900 2024-06-10T00:53:27.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.816+0000 7f1ced07e700 1 -- 172.21.15.37:0/646193693 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ce8069180 con 0x7f1ce8102900 2024-06-10T00:53:27.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.817+0000 7f1ce4d99700 1 -- 172.21.15.37:0/646193693 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1cd002a070 con 0x7f1ce8102900 2024-06-10T00:53:27.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.818+0000 7f1ce4d99700 1 --2- 172.21.15.37:0/646193693 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1cd40a2ca0 0x7f1cd40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:27.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.818+0000 7f1ce6d9d700 1 --2- 172.21.15.37:0/646193693 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1cd40a2ca0 0x7f1cd40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:27.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.818+0000 7f1ce4d99700 1 -- 172.21.15.37:0/646193693 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f1cd00d02d0 con 0x7f1ce8102900 2024-06-10T00:53:27.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.818+0000 7f1ce6d9d700 1 --2- 172.21.15.37:0/646193693 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1cd40a2ca0 0x7f1cd40a5160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8006fd0 tx=0x7f1cd8008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:27.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:27.823+0000 7f1ce4d99700 1 -- 172.21.15.37:0/646193693 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1cd005e850 con 0x7f1ce8102900 2024-06-10T00:53:28.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:28.029+0000 7f1ced07e700 1 -- 172.21.15.37:0/646193693 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ce802dca0 con 0x7f1ce8102900 2024-06-10T00:53:28.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:28.030+0000 7f1ce4d99700 1 -- 172.21.15.37:0/646193693 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24562 (secure 0 0 0) 0x7f1cd0013070 con 0x7f1ce8102900 2024-06-10T00:53:28.032 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:53:28.033 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":98,"fsid":"68793514-26c2-11ef-bc9b-c7b262605968","created":"2024-06-10T00:44:24.106713+0000","modified":"2024-06-10T00:53:22.116215+0000","last_up_change":"2024-06-10T00:51:31.562156+0000","last_in_change":"2024-06-10T00:51:11.294301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:47:44.773911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:52:31.684780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:52:34.885503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:52:44.415437+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1df6046a-6801-4950-9193-54303fed169b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6803","nonce":3594704094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6805","nonce":3594704094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6809","nonce":3594704094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6807","nonce":3594704094}]},"public_addr":"172.21.15.37:6803/3594704094","cluster_addr":"172.21.15.37:6805/3594704094","heartbeat_back_addr":"172.21.15.37:6809/3594704094","heartbeat_front_addr":"172.21.15.37:6807/3594704094","state":["exists","up"]},{"osd":1,"uuid":"4e1c8944-94e4-4e46-9efe-56229fe6df1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6801","nonce":3427727155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6803","nonce":3427727155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6807","nonce":3427727155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6805","nonce":3427727155}]},"public_addr":"172.21.15.94:6801/3427727155","cluster_addr":"172.21.15.94:6803/3427727155","heartbeat_back_addr":"172.21.15.94:6807/3427727155","heartbeat_front_addr":"172.21.15.94:6805/3427727155","state":["exists","up"]},{"osd":2,"uuid":"fd2fcedc-4c88-478f-8025-13e5944a27ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6801","nonce":4002498292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6803","nonce":4002498292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6807","nonce":4002498292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6805","nonce":4002498292}]},"public_addr":"172.21.15.167:6801/4002498292","cluster_addr":"172.21.15.167:6803/4002498292","heartbeat_back_addr":"172.21.15.167:6807/4002498292","heartbeat_front_addr":"172.21.15.167:6805/4002498292","state":["exists","up"]},{"osd":3,"uuid":"d0153676-818c-4dcc-83b1-287435a0c8a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6811","nonce":3425469324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6813","nonce":3425469324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6817","nonce":3425469324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6815","nonce":3425469324}]},"public_addr":"172.21.15.37:6811/3425469324","cluster_addr":"172.21.15.37:6813/3425469324","heartbeat_back_addr":"172.21.15.37:6817/3425469324","heartbeat_front_addr":"172.21.15.37:6815/3425469324","state":["exists","up"]},{"osd":4,"uuid":"172c333c-00c0-4e94-8870-acb3b32b2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6809","nonce":1740627431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6811","nonce":1740627431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6815","nonce":1740627431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6813","nonce":1740627431}]},"public_addr":"172.21.15.94:6809/1740627431","cluster_addr":"172.21.15.94:6811/1740627431","heartbeat_back_addr":"172.21.15.94:6815/1740627431","heartbeat_front_addr":"172.21.15.94:6813/1740627431","state":["exists","up"]},{"osd":5,"uuid":"dbce656e-5f93-4bda-baf2-fc3cc03d8282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6809","nonce":221805236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6811","nonce":221805236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6815","nonce":221805236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6813","nonce":221805236}]},"public_addr":"172.21.15.167:6809/221805236","cluster_addr":"172.21.15.167:6811/221805236","heartbeat_back_addr":"172.21.15.167:6815/221805236","heartbeat_front_addr":"172.21.15.167:6813/221805236","state":["exists","up"]},{"osd":6,"uuid":"fcbda266-6d60-43b2-a51c-4e562476d1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6819","nonce":472670875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6821","nonce":472670875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6825","nonce":472670875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6823","nonce":472670875}]},"public_addr":"172.21.15.37:6819/472670875","cluster_addr":"172.21.15.37:6821/472670875","heartbeat_back_addr":"172.21.15.37:6825/472670875","heartbeat_front_addr":"172.21.15.37:6823/472670875","state":["exists","up"]},{"osd":7,"uuid":"ed829ed3-ba85-4e82-ac37-e44e9534d37c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6817","nonce":3932281552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6819","nonce":3932281552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6823","nonce":3932281552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6821","nonce":3932281552}]},"public_addr":"172.21.15.94:6817/3932281552","cluster_addr":"172.21.15.94:6819/3932281552","heartbeat_back_addr":"172.21.15.94:6823/3932281552","heartbeat_front_addr":"172.21.15.94:6821/3932281552","state":["exists","up"]},{"osd":8,"uuid":"7d6267d8-72f4-4704-b554-bafc52d11a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6817","nonce":2290216047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6819","nonce":2290216047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6823","nonce":2290216047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6821","nonce":2290216047}]},"public_addr":"172.21.15.167:6817/2290216047","cluster_addr":"172.21.15.167:6819/2290216047","heartbeat_back_addr":"172.21.15.167:6823/2290216047","heartbeat_front_addr":"172.21.15.167:6821/2290216047","state":["exists","up"]},{"osd":9,"uuid":"210fdc20-0a36-4eeb-bdf2-c0a7f130e19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6827","nonce":2063666428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6829","nonce":2063666428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6833","nonce":2063666428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6831","nonce":2063666428}]},"public_addr":"172.21.15.37:6827/2063666428","cluster_addr":"172.21.15.37:6829/2063666428","heartbeat_back_addr":"172.21.15.37:6833/2063666428","heartbeat_front_addr":"172.21.15.37:6831/2063666428","state":["exists","up"]},{"osd":10,"uuid":"e200b371-b3cf-4b92-a659-f155ca86f0a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6825","nonce":1706618899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6827","nonce":1706618899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6831","nonce":1706618899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6829","nonce":1706618899}]},"public_addr":"172.21.15.94:6825/1706618899","cluster_addr":"172.21.15.94:6827/1706618899","heartbeat_back_addr":"172.21.15.94:6831/1706618899","heartbeat_front_addr":"172.21.15.94:6829/1706618899","state":["exists","up"]},{"osd":11,"uuid":"cfa427df-37a5-4d28-ac2e-af777d607008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6825","nonce":1349892516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6827","nonce":1349892516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6831","nonce":1349892516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6829","nonce":1349892516}]},"public_addr":"172.21.15.167:6825/1349892516","cluster_addr":"172.21.15.167:6827/1349892516","heartbeat_back_addr":"172.21.15.167:6831/1349892516","heartbeat_front_addr":"172.21.15.167:6829/1349892516","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:46:51.846147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:15.763723+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:40.106800+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:05.923204+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:30.501883+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:55.565575+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:20.411360+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:46.071776+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:11.854599+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:37.608818+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:03.176767+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:29.768705+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/4084722397":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/3571803585":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/416587443":"2024-06-11T00:45:20.254342+0000","172.21.15.37:6801/2766692263":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/2026344134":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/2781321886":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6801/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/1110143997":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2766692263":"2024-06-11T00:45:20.254342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:53:28.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:28.033+0000 7f1ced07e700 1 -- 172.21.15.37:0/646193693 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1cd40a2ca0 msgr2=0x7f1cd40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:28.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:28.033+0000 7f1ced07e700 1 --2- 172.21.15.37:0/646193693 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1cd40a2ca0 0x7f1cd40a5160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8006fd0 tx=0x7f1cd8008040 comp rx=0 tx=0).stop 2024-06-10T00:53:28.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:28.033+0000 7f1ced07e700 1 -- 172.21.15.37:0/646193693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ce8102900 msgr2=0x7f1ce8103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:28.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:28.033+0000 7f1ced07e700 1 --2- 172.21.15.37:0/646193693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ce8102900 0x7f1ce8103600 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009fd0 tx=0x7f1cd0003730 comp rx=0 tx=0).stop 2024-06-10T00:53:28.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:28.033+0000 7f1ced07e700 1 -- 172.21.15.37:0/646193693 shutdown_connections 2024-06-10T00:53:28.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:28.033+0000 7f1ced07e700 1 --2- 172.21.15.37:0/646193693 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1cd40a2ca0 0x7f1cd40a5160 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:28.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:28.033+0000 7f1ced07e700 1 --2- 172.21.15.37:0/646193693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ce8107660 0x7f1ce8103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:28.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:28.033+0000 7f1ced07e700 1 --2- 172.21.15.37:0/646193693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ce8101f40 0x7f1ce81030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:28.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:28.033+0000 7f1ced07e700 1 --2- 172.21.15.37:0/646193693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ce8102900 0x7f1ce8103600 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:28.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:28.033+0000 7f1ced07e700 1 -- 172.21.15.37:0/646193693 >> 172.21.15.37:0/646193693 conn(0x7f1ce80766b0 msgr2=0x7f1ce8100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:28.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:28.033+0000 7f1ced07e700 1 -- 172.21.15.37:0/646193693 shutdown_connections 2024-06-10T00:53:28.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:28.033+0000 7f1ced07e700 1 -- 172.21.15.37:0/646193693 wait complete. 2024-06-10T00:53:28.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:28 smithi037 ceph-mon[32192]: pgmap v331: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:53:28.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:28 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/646193693' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:53:28.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:28 smithi167 ceph-mon[36917]: pgmap v331: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:53:28.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:28 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/646193693' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:53:28.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:28 smithi094 ceph-mon[36708]: pgmap v331: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:53:28.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:28 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/646193693' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:53:28.711 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T00:53:28.967 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:53:30.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:30 smithi037 ceph-mon[32192]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:53:30.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:30 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:53:30.432 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:30 smithi094 ceph-mon[36708]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:53:30.432 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:30 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:53:30.440 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:30 smithi167 ceph-mon[36917]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:53:30.440 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:30 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:53:30.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.706+0000 7ff569a53700 1 -- 172.21.15.37:0/3211186140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff564108500 msgr2=0x7ff5641088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:30.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.706+0000 7ff569a53700 1 --2- 172.21.15.37:0/3211186140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff564108500 0x7ff5641088e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff554009990 tx=0x7ff554009ca0 comp rx=0 tx=0).stop 2024-06-10T00:53:30.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.706+0000 7ff569a53700 1 -- 172.21.15.37:0/3211186140 shutdown_connections 2024-06-10T00:53:30.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.706+0000 7ff569a53700 1 --2- 172.21.15.37:0/3211186140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff56406ab30 0x7ff56410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:30.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.706+0000 7ff569a53700 1 --2- 172.21.15.37:0/3211186140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff564108500 0x7ff5641088e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:30.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.706+0000 7ff569a53700 1 --2- 172.21.15.37:0/3211186140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff56406a170 0x7ff56406a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:30.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.706+0000 7ff569a53700 1 -- 172.21.15.37:0/3211186140 >> 172.21.15.37:0/3211186140 conn(0x7ff564078340 msgr2=0x7ff564078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:30.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.706+0000 7ff569a53700 1 -- 172.21.15.37:0/3211186140 shutdown_connections 2024-06-10T00:53:30.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.706+0000 7ff569a53700 1 -- 172.21.15.37:0/3211186140 wait complete. 2024-06-10T00:53:30.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.707+0000 7ff569a53700 1 Processor -- start 2024-06-10T00:53:30.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.707+0000 7ff569a53700 1 -- start start 2024-06-10T00:53:30.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.707+0000 7ff569a53700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff56406a170 0x7ff56419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:30.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.707+0000 7ff569a53700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff56406ab30 0x7ff56419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:30.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.707+0000 7ff569a53700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff564108500 0x7ff564198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:30.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.707+0000 7ff569a53700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff5641127d0 con 0x7ff56406a170 2024-06-10T00:53:30.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.707+0000 7ff569a53700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff564112650 con 0x7ff564108500 2024-06-10T00:53:30.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.707+0000 7ff569a53700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff564112950 con 0x7ff56406ab30 2024-06-10T00:53:30.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.707+0000 7ff562ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff56406a170 0x7ff56419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:30.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.707+0000 7ff562ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff56406a170 0x7ff56419e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35320/0 (socket says 172.21.15.37:35320) 2024-06-10T00:53:30.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.707+0000 7ff562ffd700 1 -- 172.21.15.37:0/1307485210 learned_addr learned my addr 172.21.15.37:0/1307485210 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:53:30.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.708+0000 7ff5627fc700 1 --2- 172.21.15.37:0/1307485210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff56406ab30 0x7ff56419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:30.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.708+0000 7ff5637fe700 1 --2- 172.21.15.37:0/1307485210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff564108500 0x7ff564198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:30.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.708+0000 7ff562ffd700 1 -- 172.21.15.37:0/1307485210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff56406ab30 msgr2=0x7ff56419edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:30.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.708+0000 7ff562ffd700 1 --2- 172.21.15.37:0/1307485210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff56406ab30 0x7ff56419edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:30.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.708+0000 7ff562ffd700 1 -- 172.21.15.37:0/1307485210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff564108500 msgr2=0x7ff564198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:30.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.708+0000 7ff562ffd700 1 --2- 172.21.15.37:0/1307485210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff564108500 0x7ff564198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:30.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.708+0000 7ff562ffd700 1 -- 172.21.15.37:0/1307485210 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5641991b0 con 0x7ff56406a170 2024-06-10T00:53:30.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.708+0000 7ff5627fc700 1 --2- 172.21.15.37:0/1307485210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff56406ab30 0x7ff56419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:53:30.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.708+0000 7ff5637fe700 1 --2- 172.21.15.37:0/1307485210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff564108500 0x7ff564198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:53:30.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.708+0000 7ff562ffd700 1 --2- 172.21.15.37:0/1307485210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff56406a170 0x7ff56419e890 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff554000c00 tx=0x7ff554009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:30.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.708+0000 7ff568a51700 1 -- 172.21.15.37:0/1307485210 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554027070 con 0x7ff56406a170 2024-06-10T00:53:30.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.708+0000 7ff569a53700 1 -- 172.21.15.37:0/1307485210 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff564199440 con 0x7ff56406a170 2024-06-10T00:53:30.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.708+0000 7ff568a51700 1 -- 172.21.15.37:0/1307485210 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff554004050 con 0x7ff56406a170 2024-06-10T00:53:30.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.708+0000 7ff568a51700 1 -- 172.21.15.37:0/1307485210 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554022e00 con 0x7ff56406a170 2024-06-10T00:53:30.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.708+0000 7ff569a53700 1 -- 172.21.15.37:0/1307485210 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5641a5760 con 0x7ff56406a170 2024-06-10T00:53:30.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.709+0000 7ff569a53700 1 -- 172.21.15.37:0/1307485210 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff56410a9e0 con 0x7ff56406a170 2024-06-10T00:53:30.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.712+0000 7ff568a51700 1 -- 172.21.15.37:0/1307485210 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff55402a070 con 0x7ff56406a170 2024-06-10T00:53:30.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.712+0000 7ff568a51700 1 --2- 172.21.15.37:0/1307485210 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff54c0a2ca0 0x7ff54c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:30.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.712+0000 7ff5627fc700 1 --2- 172.21.15.37:0/1307485210 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff54c0a2ca0 0x7ff54c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:30.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.712+0000 7ff568a51700 1 -- 172.21.15.37:0/1307485210 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7ff5540d0b10 con 0x7ff56406a170 2024-06-10T00:53:30.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.713+0000 7ff5627fc700 1 --2- 172.21.15.37:0/1307485210 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff54c0a2ca0 0x7ff54c0a5160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff558005d90 tx=0x7ff558005d00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:30.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.715+0000 7ff568a51700 1 -- 172.21.15.37:0/1307485210 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff554031070 con 0x7ff56406a170 2024-06-10T00:53:30.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.927+0000 7ff569a53700 1 -- 172.21.15.37:0/1307485210 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff564199f80 con 0x7ff56406a170 2024-06-10T00:53:30.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.929+0000 7ff568a51700 1 -- 172.21.15.37:0/1307485210 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24562 (secure 0 0 0) 0x7ff554013070 con 0x7ff56406a170 2024-06-10T00:53:30.929 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:53:30.930 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":98,"fsid":"68793514-26c2-11ef-bc9b-c7b262605968","created":"2024-06-10T00:44:24.106713+0000","modified":"2024-06-10T00:53:22.116215+0000","last_up_change":"2024-06-10T00:51:31.562156+0000","last_in_change":"2024-06-10T00:51:11.294301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:47:44.773911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:52:31.684780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:52:34.885503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:52:44.415437+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1df6046a-6801-4950-9193-54303fed169b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6803","nonce":3594704094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6805","nonce":3594704094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6809","nonce":3594704094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6807","nonce":3594704094}]},"public_addr":"172.21.15.37:6803/3594704094","cluster_addr":"172.21.15.37:6805/3594704094","heartbeat_back_addr":"172.21.15.37:6809/3594704094","heartbeat_front_addr":"172.21.15.37:6807/3594704094","state":["exists","up"]},{"osd":1,"uuid":"4e1c8944-94e4-4e46-9efe-56229fe6df1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6801","nonce":3427727155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6803","nonce":3427727155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6807","nonce":3427727155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6805","nonce":3427727155}]},"public_addr":"172.21.15.94:6801/3427727155","cluster_addr":"172.21.15.94:6803/3427727155","heartbeat_back_addr":"172.21.15.94:6807/3427727155","heartbeat_front_addr":"172.21.15.94:6805/3427727155","state":["exists","up"]},{"osd":2,"uuid":"fd2fcedc-4c88-478f-8025-13e5944a27ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6801","nonce":4002498292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6803","nonce":4002498292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6807","nonce":4002498292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6805","nonce":4002498292}]},"public_addr":"172.21.15.167:6801/4002498292","cluster_addr":"172.21.15.167:6803/4002498292","heartbeat_back_addr":"172.21.15.167:6807/4002498292","heartbeat_front_addr":"172.21.15.167:6805/4002498292","state":["exists","up"]},{"osd":3,"uuid":"d0153676-818c-4dcc-83b1-287435a0c8a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6811","nonce":3425469324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6813","nonce":3425469324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6817","nonce":3425469324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6815","nonce":3425469324}]},"public_addr":"172.21.15.37:6811/3425469324","cluster_addr":"172.21.15.37:6813/3425469324","heartbeat_back_addr":"172.21.15.37:6817/3425469324","heartbeat_front_addr":"172.21.15.37:6815/3425469324","state":["exists","up"]},{"osd":4,"uuid":"172c333c-00c0-4e94-8870-acb3b32b2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6809","nonce":1740627431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6811","nonce":1740627431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6815","nonce":1740627431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6813","nonce":1740627431}]},"public_addr":"172.21.15.94:6809/1740627431","cluster_addr":"172.21.15.94:6811/1740627431","heartbeat_back_addr":"172.21.15.94:6815/1740627431","heartbeat_front_addr":"172.21.15.94:6813/1740627431","state":["exists","up"]},{"osd":5,"uuid":"dbce656e-5f93-4bda-baf2-fc3cc03d8282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6809","nonce":221805236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6811","nonce":221805236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6815","nonce":221805236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6813","nonce":221805236}]},"public_addr":"172.21.15.167:6809/221805236","cluster_addr":"172.21.15.167:6811/221805236","heartbeat_back_addr":"172.21.15.167:6815/221805236","heartbeat_front_addr":"172.21.15.167:6813/221805236","state":["exists","up"]},{"osd":6,"uuid":"fcbda266-6d60-43b2-a51c-4e562476d1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6819","nonce":472670875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6821","nonce":472670875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6825","nonce":472670875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6823","nonce":472670875}]},"public_addr":"172.21.15.37:6819/472670875","cluster_addr":"172.21.15.37:6821/472670875","heartbeat_back_addr":"172.21.15.37:6825/472670875","heartbeat_front_addr":"172.21.15.37:6823/472670875","state":["exists","up"]},{"osd":7,"uuid":"ed829ed3-ba85-4e82-ac37-e44e9534d37c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6817","nonce":3932281552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6819","nonce":3932281552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6823","nonce":3932281552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6821","nonce":3932281552}]},"public_addr":"172.21.15.94:6817/3932281552","cluster_addr":"172.21.15.94:6819/3932281552","heartbeat_back_addr":"172.21.15.94:6823/3932281552","heartbeat_front_addr":"172.21.15.94:6821/3932281552","state":["exists","up"]},{"osd":8,"uuid":"7d6267d8-72f4-4704-b554-bafc52d11a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6817","nonce":2290216047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6819","nonce":2290216047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6823","nonce":2290216047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6821","nonce":2290216047}]},"public_addr":"172.21.15.167:6817/2290216047","cluster_addr":"172.21.15.167:6819/2290216047","heartbeat_back_addr":"172.21.15.167:6823/2290216047","heartbeat_front_addr":"172.21.15.167:6821/2290216047","state":["exists","up"]},{"osd":9,"uuid":"210fdc20-0a36-4eeb-bdf2-c0a7f130e19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6827","nonce":2063666428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6829","nonce":2063666428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6833","nonce":2063666428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6831","nonce":2063666428}]},"public_addr":"172.21.15.37:6827/2063666428","cluster_addr":"172.21.15.37:6829/2063666428","heartbeat_back_addr":"172.21.15.37:6833/2063666428","heartbeat_front_addr":"172.21.15.37:6831/2063666428","state":["exists","up"]},{"osd":10,"uuid":"e200b371-b3cf-4b92-a659-f155ca86f0a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6825","nonce":1706618899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6827","nonce":1706618899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6831","nonce":1706618899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6829","nonce":1706618899}]},"public_addr":"172.21.15.94:6825/1706618899","cluster_addr":"172.21.15.94:6827/1706618899","heartbeat_back_addr":"172.21.15.94:6831/1706618899","heartbeat_front_addr":"172.21.15.94:6829/1706618899","state":["exists","up"]},{"osd":11,"uuid":"cfa427df-37a5-4d28-ac2e-af777d607008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6825","nonce":1349892516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6827","nonce":1349892516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6831","nonce":1349892516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6829","nonce":1349892516}]},"public_addr":"172.21.15.167:6825/1349892516","cluster_addr":"172.21.15.167:6827/1349892516","heartbeat_back_addr":"172.21.15.167:6831/1349892516","heartbeat_front_addr":"172.21.15.167:6829/1349892516","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:46:51.846147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:15.763723+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:40.106800+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:05.923204+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:30.501883+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:55.565575+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:20.411360+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:46.071776+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:11.854599+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:37.608818+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:03.176767+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:29.768705+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/4084722397":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/3571803585":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/416587443":"2024-06-11T00:45:20.254342+0000","172.21.15.37:6801/2766692263":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/2026344134":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/2781321886":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6801/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/1110143997":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2766692263":"2024-06-11T00:45:20.254342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:53:30.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.931+0000 7ff569a53700 1 -- 172.21.15.37:0/1307485210 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff54c0a2ca0 msgr2=0x7ff54c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:30.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.931+0000 7ff569a53700 1 --2- 172.21.15.37:0/1307485210 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff54c0a2ca0 0x7ff54c0a5160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff558005d90 tx=0x7ff558005d00 comp rx=0 tx=0).stop 2024-06-10T00:53:30.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.931+0000 7ff569a53700 1 -- 172.21.15.37:0/1307485210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff56406a170 msgr2=0x7ff56419e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:30.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.931+0000 7ff569a53700 1 --2- 172.21.15.37:0/1307485210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff56406a170 0x7ff56419e890 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff554000c00 tx=0x7ff554009f90 comp rx=0 tx=0).stop 2024-06-10T00:53:30.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.931+0000 7ff569a53700 1 -- 172.21.15.37:0/1307485210 shutdown_connections 2024-06-10T00:53:30.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.931+0000 7ff569a53700 1 --2- 172.21.15.37:0/1307485210 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff54c0a2ca0 0x7ff54c0a5160 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:30.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.931+0000 7ff569a53700 1 --2- 172.21.15.37:0/1307485210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff564108500 0x7ff564198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:30.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.931+0000 7ff569a53700 1 --2- 172.21.15.37:0/1307485210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff56406a170 0x7ff56419e890 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:30.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.931+0000 7ff569a53700 1 --2- 172.21.15.37:0/1307485210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff56406ab30 0x7ff56419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:30.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.931+0000 7ff569a53700 1 -- 172.21.15.37:0/1307485210 >> 172.21.15.37:0/1307485210 conn(0x7ff564078340 msgr2=0x7ff5640ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:30.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.931+0000 7ff569a53700 1 -- 172.21.15.37:0/1307485210 shutdown_connections 2024-06-10T00:53:30.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:30.931+0000 7ff569a53700 1 -- 172.21.15.37:0/1307485210 wait complete. 2024-06-10T00:53:31.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:31 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1307485210' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:53:31.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:31 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1307485210' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:53:31.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:31 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1307485210' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:53:32.038 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-06-10T00:53:32.313 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:53:32.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:32 smithi037 ceph-mon[32192]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 110 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:53:32.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:32 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:53:32.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:32 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:53:32.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:32 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:53:32.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:32 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:53:32.490 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:32 smithi167 ceph-mon[36917]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 110 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:53:32.490 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:32 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:53:32.490 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:32 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:53:32.490 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:32 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:53:32.490 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:32 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:53:32.516 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:32 smithi094 ceph-mon[36708]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 110 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:53:32.516 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:32 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:53:32.516 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:32 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:53:32.516 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:32 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:53:32.516 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:32 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:53:34.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.030+0000 7fc234faa700 1 -- 172.21.15.37:0/3112866489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2301073f0 msgr2=0x7fc2301077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:34.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.030+0000 7fc234faa700 1 --2- 172.21.15.37:0/3112866489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2301073f0 0x7fc2301077d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc220009990 tx=0x7fc220009ca0 comp rx=0 tx=0).stop 2024-06-10T00:53:34.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.030+0000 7fc234faa700 1 -- 172.21.15.37:0/3112866489 shutdown_connections 2024-06-10T00:53:34.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.030+0000 7fc234faa700 1 --2- 172.21.15.37:0/3112866489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc230101d40 0x7fc23010e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:34.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.030+0000 7fc234faa700 1 --2- 172.21.15.37:0/3112866489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2301073f0 0x7fc2301077d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:34.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.030+0000 7fc234faa700 1 --2- 172.21.15.37:0/3112866489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc230101380 0x7fc230101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:34.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.030+0000 7fc234faa700 1 -- 172.21.15.37:0/3112866489 >> 172.21.15.37:0/3112866489 conn(0x7fc230078300 msgr2=0x7fc230078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:34.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.030+0000 7fc234faa700 1 -- 172.21.15.37:0/3112866489 shutdown_connections 2024-06-10T00:53:34.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.030+0000 7fc234faa700 1 -- 172.21.15.37:0/3112866489 wait complete. 2024-06-10T00:53:34.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.031+0000 7fc234faa700 1 Processor -- start 2024-06-10T00:53:34.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.031+0000 7fc234faa700 1 -- start start 2024-06-10T00:53:34.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.032+0000 7fc234faa700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc230101380 0x7fc230199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:34.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.032+0000 7fc234faa700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc230101d40 0x7fc230199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:34.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.032+0000 7fc234faa700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2301073f0 0x7fc2301a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:34.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.032+0000 7fc234faa700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc230111660 con 0x7fc230101380 2024-06-10T00:53:34.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.032+0000 7fc234faa700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc2301114e0 con 0x7fc2301073f0 2024-06-10T00:53:34.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.032+0000 7fc234faa700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc2301117e0 con 0x7fc230101d40 2024-06-10T00:53:34.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.032+0000 7fc22e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc230101380 0x7fc230199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:34.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.032+0000 7fc22e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc230101380 0x7fc230199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49912/0 (socket says 172.21.15.37:49912) 2024-06-10T00:53:34.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.032+0000 7fc22e59c700 1 -- 172.21.15.37:0/2847830204 learned_addr learned my addr 172.21.15.37:0/2847830204 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:53:34.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.032+0000 7fc22dd9b700 1 --2- 172.21.15.37:0/2847830204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc230101d40 0x7fc230199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:34.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.032+0000 7fc22ed9d700 1 --2- 172.21.15.37:0/2847830204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2301073f0 0x7fc2301a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:34.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.032+0000 7fc22e59c700 1 -- 172.21.15.37:0/2847830204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc230101d40 msgr2=0x7fc230199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:34.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.032+0000 7fc22e59c700 1 --2- 172.21.15.37:0/2847830204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc230101d40 0x7fc230199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:34.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.032+0000 7fc22e59c700 1 -- 172.21.15.37:0/2847830204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2301073f0 msgr2=0x7fc2301a4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:34.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.032+0000 7fc22e59c700 1 --2- 172.21.15.37:0/2847830204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2301073f0 0x7fc2301a4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:34.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.032+0000 7fc22e59c700 1 -- 172.21.15.37:0/2847830204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2301a4fc0 con 0x7fc230101380 2024-06-10T00:53:34.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.033+0000 7fc22e59c700 1 --2- 172.21.15.37:0/2847830204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc230101380 0x7fc230199a00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc220000c00 tx=0x7fc22000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:34.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.033+0000 7fc21f7fe700 1 -- 172.21.15.37:0/2847830204 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc220027070 con 0x7fc230101380 2024-06-10T00:53:34.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.033+0000 7fc234faa700 1 -- 172.21.15.37:0/2847830204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2301a51f0 con 0x7fc230101380 2024-06-10T00:53:34.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.033+0000 7fc21f7fe700 1 -- 172.21.15.37:0/2847830204 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc220003d70 con 0x7fc230101380 2024-06-10T00:53:34.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.033+0000 7fc21f7fe700 1 -- 172.21.15.37:0/2847830204 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22000f460 con 0x7fc230101380 2024-06-10T00:53:34.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.033+0000 7fc234faa700 1 -- 172.21.15.37:0/2847830204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2301a56a0 con 0x7fc230101380 2024-06-10T00:53:34.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.034+0000 7fc234faa700 1 -- 172.21.15.37:0/2847830204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc230069180 con 0x7fc230101380 2024-06-10T00:53:34.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.035+0000 7fc21f7fe700 1 -- 172.21.15.37:0/2847830204 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc22002a070 con 0x7fc230101380 2024-06-10T00:53:34.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.036+0000 7fc21f7fe700 1 --2- 172.21.15.37:0/2847830204 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc2180a2ca0 0x7fc2180a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:34.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.036+0000 7fc22dd9b700 1 --2- 172.21.15.37:0/2847830204 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc2180a2ca0 0x7fc2180a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:34.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.036+0000 7fc21f7fe700 1 -- 172.21.15.37:0/2847830204 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fc2200d0120 con 0x7fc230101380 2024-06-10T00:53:34.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.036+0000 7fc22dd9b700 1 --2- 172.21.15.37:0/2847830204 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc2180a2ca0 0x7fc2180a5160 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc2300789e0 tx=0x7fc224009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:34.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.040+0000 7fc21f7fe700 1 -- 172.21.15.37:0/2847830204 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc220013070 con 0x7fc230101380 2024-06-10T00:53:34.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:34.294+0000 7fc234faa700 1 -- 172.21.15.37:0/2847830204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc23002dca0 con 0x7fc230101380 2024-06-10T00:53:34.295 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:34 smithi037 ceph-mon[32192]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-10T00:53:34.295 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:34 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:53:34.296 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:34 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:53:34.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:34 smithi167 ceph-mon[36917]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-10T00:53:34.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:34 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:53:34.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:34 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:53:34.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:34 smithi094 ceph-mon[36708]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-10T00:53:34.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:34 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:53:34.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:34 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:53:35.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:35.206+0000 7fc21f7fe700 1 -- 172.21.15.37:0/2847830204 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7fc22009cd50 con 0x7fc230101380 2024-06-10T00:53:35.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:35.209+0000 7fc234faa700 1 -- 172.21.15.37:0/2847830204 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc2180a2ca0 msgr2=0x7fc2180a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:35.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:35.209+0000 7fc234faa700 1 --2- 172.21.15.37:0/2847830204 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc2180a2ca0 0x7fc2180a5160 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc2300789e0 tx=0x7fc224009260 comp rx=0 tx=0).stop 2024-06-10T00:53:35.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:35.209+0000 7fc234faa700 1 -- 172.21.15.37:0/2847830204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc230101380 msgr2=0x7fc230199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:35.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:35.209+0000 7fc234faa700 1 --2- 172.21.15.37:0/2847830204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc230101380 0x7fc230199a00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc220000c00 tx=0x7fc22000bab0 comp rx=0 tx=0).stop 2024-06-10T00:53:35.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:35.209+0000 7fc234faa700 1 -- 172.21.15.37:0/2847830204 shutdown_connections 2024-06-10T00:53:35.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:35.209+0000 7fc234faa700 1 --2- 172.21.15.37:0/2847830204 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc2180a2ca0 0x7fc2180a5160 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:35.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:35.209+0000 7fc234faa700 1 --2- 172.21.15.37:0/2847830204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2301073f0 0x7fc2301a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:35.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:35.209+0000 7fc234faa700 1 --2- 172.21.15.37:0/2847830204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc230101380 0x7fc230199a00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:35.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:35.209+0000 7fc234faa700 1 --2- 172.21.15.37:0/2847830204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc230101d40 0x7fc230199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:35.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:35.209+0000 7fc234faa700 1 -- 172.21.15.37:0/2847830204 >> 172.21.15.37:0/2847830204 conn(0x7fc230078300 msgr2=0x7fc230100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:35.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:35.209+0000 7fc234faa700 1 -- 172.21.15.37:0/2847830204 shutdown_connections 2024-06-10T00:53:35.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:35.209+0000 7fc234faa700 1 -- 172.21.15.37:0/2847830204 wait complete. 2024-06-10T00:53:35.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:35 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2847830204' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-10T00:53:35.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:35 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:53:35.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:35 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:53:35.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:35 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:53:35.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:35 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2847830204' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-10T00:53:35.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:35 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:53:35.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:35 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:53:35.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:35 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:53:35.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:35 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2847830204' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-10T00:53:35.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:35 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:53:35.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:35 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:53:35.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:35 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:53:35.887 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-06-10T00:53:35.887 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-06-10T00:53:36.143 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:53:36.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:36 smithi037 ceph-mon[32192]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-10T00:53:36.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:36 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2847830204' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-10T00:53:36.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:36 smithi037 ceph-mon[32192]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T00:53:36.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:36 smithi167 ceph-mon[36917]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-10T00:53:36.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:36 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2847830204' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-10T00:53:36.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:36 smithi167 ceph-mon[36917]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T00:53:36.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:36 smithi094 ceph-mon[36708]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-10T00:53:36.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:36 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2847830204' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-10T00:53:36.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:36 smithi094 ceph-mon[36708]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T00:53:37.533 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:37 smithi037 ceph-mon[32192]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:37.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:37 smithi167 ceph-mon[36917]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:37.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:37 smithi094 ceph-mon[36708]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:37.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.786+0000 7f6662b48700 1 -- 172.21.15.37:0/124327373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f665c101390 msgr2=0x7f665c101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:37.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.786+0000 7f6662b48700 1 --2- 172.21.15.37:0/124327373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f665c101390 0x7f665c101810 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6650009990 tx=0x7f6650009ca0 comp rx=0 tx=0).stop 2024-06-10T00:53:37.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.786+0000 7f6662b48700 1 -- 172.21.15.37:0/124327373 shutdown_connections 2024-06-10T00:53:37.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.786+0000 7f6662b48700 1 --2- 172.21.15.37:0/124327373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f665c101d50 0x7f665c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:37.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.786+0000 7f6662b48700 1 --2- 172.21.15.37:0/124327373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f665c107400 0x7f665c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:37.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.786+0000 7f6662b48700 1 --2- 172.21.15.37:0/124327373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f665c101390 0x7f665c101810 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:37.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.786+0000 7f6662b48700 1 -- 172.21.15.37:0/124327373 >> 172.21.15.37:0/124327373 conn(0x7f665c078340 msgr2=0x7f665c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:37.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.786+0000 7f6662b48700 1 -- 172.21.15.37:0/124327373 shutdown_connections 2024-06-10T00:53:37.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.786+0000 7f6662b48700 1 -- 172.21.15.37:0/124327373 wait complete. 2024-06-10T00:53:37.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.787+0000 7f6662b48700 1 Processor -- start 2024-06-10T00:53:37.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.787+0000 7f6662b48700 1 -- start start 2024-06-10T00:53:37.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.787+0000 7f6662b48700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f665c101390 0x7f665c1a3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:37.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.787+0000 7f6662b48700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f665c101d50 0x7f665c1a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:37.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.787+0000 7f6662b48700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f665c107400 0x7f665c19f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:37.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.787+0000 7f6662b48700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f665c1116a0 con 0x7f665c101390 2024-06-10T00:53:37.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.787+0000 7f6662b48700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f665c111520 con 0x7f665c101d50 2024-06-10T00:53:37.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.787+0000 7f6662b48700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f665c111820 con 0x7f665c107400 2024-06-10T00:53:37.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.788+0000 7f66608e4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f665c101390 0x7f665c1a3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:37.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.788+0000 7f66608e4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f665c101390 0x7f665c1a3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49932/0 (socket says 172.21.15.37:49932) 2024-06-10T00:53:37.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.788+0000 7f66608e4700 1 -- 172.21.15.37:0/1955193428 learned_addr learned my addr 172.21.15.37:0/1955193428 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:53:37.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.788+0000 7f665bfff700 1 --2- 172.21.15.37:0/1955193428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f665c101d50 0x7f665c1a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:37.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.788+0000 7f66610e5700 1 --2- 172.21.15.37:0/1955193428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f665c107400 0x7f665c19f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:37.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.788+0000 7f66608e4700 1 -- 172.21.15.37:0/1955193428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f665c107400 msgr2=0x7f665c19f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:37.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.788+0000 7f66608e4700 1 --2- 172.21.15.37:0/1955193428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f665c107400 0x7f665c19f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:37.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.788+0000 7f66608e4700 1 -- 172.21.15.37:0/1955193428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f665c101d50 msgr2=0x7f665c1a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:37.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.788+0000 7f66608e4700 1 --2- 172.21.15.37:0/1955193428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f665c101d50 0x7f665c1a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:37.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.788+0000 7f66608e4700 1 -- 172.21.15.37:0/1955193428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665c19fb10 con 0x7f665c101390 2024-06-10T00:53:37.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.788+0000 7f66608e4700 1 --2- 172.21.15.37:0/1955193428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f665c101390 0x7f665c1a3df0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f664c00d950 tx=0x7f664c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:37.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.788+0000 7f6659ffb700 1 -- 172.21.15.37:0/1955193428 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c016070 con 0x7f665c101390 2024-06-10T00:53:37.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.788+0000 7f6662b48700 1 -- 172.21.15.37:0/1955193428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f665c19fe00 con 0x7f665c101390 2024-06-10T00:53:37.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.788+0000 7f6659ffb700 1 -- 172.21.15.37:0/1955193428 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f664c009e20 con 0x7f665c101390 2024-06-10T00:53:37.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.789+0000 7f6659ffb700 1 -- 172.21.15.37:0/1955193428 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c00f620 con 0x7f665c101390 2024-06-10T00:53:37.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.789+0000 7f6662b48700 1 -- 172.21.15.37:0/1955193428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f665c1abd20 con 0x7f665c101390 2024-06-10T00:53:37.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.790+0000 7f6662b48700 1 -- 172.21.15.37:0/1955193428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f665c069180 con 0x7f665c101390 2024-06-10T00:53:37.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.791+0000 7f6659ffb700 1 -- 172.21.15.37:0/1955193428 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f664c019070 con 0x7f665c101390 2024-06-10T00:53:37.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.791+0000 7f6659ffb700 1 --2- 172.21.15.37:0/1955193428 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f66440a2c20 0x7f66440a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:37.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.792+0000 7f665bfff700 1 --2- 172.21.15.37:0/1955193428 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f66440a2c20 0x7f66440a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:37.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.792+0000 7f6659ffb700 1 -- 172.21.15.37:0/1955193428 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f664c0cef30 con 0x7f665c101390 2024-06-10T00:53:37.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.792+0000 7f665bfff700 1 --2- 172.21.15.37:0/1955193428 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f66440a2c20 0x7f66440a50e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f665000b5c0 tx=0x7f6650005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:37.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:37.796+0000 7f6659ffb700 1 -- 172.21.15.37:0/1955193428 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f664c09bad0 con 0x7f665c101390 2024-06-10T00:53:38.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:38.050+0000 7f6662b48700 1 -- 172.21.15.37:0/1955193428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f665c1abef0 con 0x7f665c101390 2024-06-10T00:53:38.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:38.266+0000 7f6659ffb700 1 -- 172.21.15.37:0/1955193428 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f664c09b3d0 con 0x7f665c101390 2024-06-10T00:53:38.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:38.269+0000 7f6662b48700 1 -- 172.21.15.37:0/1955193428 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f66440a2c20 msgr2=0x7f66440a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:38.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:38.269+0000 7f6662b48700 1 --2- 172.21.15.37:0/1955193428 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f66440a2c20 0x7f66440a50e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f665000b5c0 tx=0x7f6650005960 comp rx=0 tx=0).stop 2024-06-10T00:53:38.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:38.269+0000 7f6662b48700 1 -- 172.21.15.37:0/1955193428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f665c101390 msgr2=0x7f665c1a3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:38.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:38.269+0000 7f6662b48700 1 --2- 172.21.15.37:0/1955193428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f665c101390 0x7f665c1a3df0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f664c00d950 tx=0x7f664c00dc60 comp rx=0 tx=0).stop 2024-06-10T00:53:38.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:38.269+0000 7f6662b48700 1 -- 172.21.15.37:0/1955193428 shutdown_connections 2024-06-10T00:53:38.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:38.269+0000 7f6662b48700 1 --2- 172.21.15.37:0/1955193428 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f66440a2c20 0x7f66440a50e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:38.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:38.269+0000 7f6662b48700 1 --2- 172.21.15.37:0/1955193428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f665c107400 0x7f665c19f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:38.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:38.269+0000 7f6662b48700 1 --2- 172.21.15.37:0/1955193428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f665c101390 0x7f665c1a3df0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:38.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:38.269+0000 7f6662b48700 1 --2- 172.21.15.37:0/1955193428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f665c101d50 0x7f665c1a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:38.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:38.269+0000 7f6662b48700 1 -- 172.21.15.37:0/1955193428 >> 172.21.15.37:0/1955193428 conn(0x7f665c078340 msgr2=0x7f665c06a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:38.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:38.269+0000 7f6662b48700 1 -- 172.21.15.37:0/1955193428 shutdown_connections 2024-06-10T00:53:38.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:38.269+0000 7f6662b48700 1 -- 172.21.15.37:0/1955193428 wait complete. 2024-06-10T00:53:38.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:38 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1955193428' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-10T00:53:38.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:38 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1955193428' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-10T00:53:38.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:38 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1955193428' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-10T00:53:38.924 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-06-10T00:53:39.175 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:53:39.516 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:39 smithi037 ceph-mon[32192]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-10T00:53:39.517 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:39 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1955193428' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-06-10T00:53:39.517 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:39 smithi037 ceph-mon[32192]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-10T00:53:39.517 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:39 smithi037 ceph-mon[32192]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T00:53:39.517 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:39 smithi037 ceph-mon[32192]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-06-10T00:53:39.517 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:39 smithi037 ceph-mon[32192]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-06-10T00:53:39.517 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:39 smithi037 ceph-mon[32192]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:39.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:39 smithi167 ceph-mon[36917]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-10T00:53:39.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:39 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1955193428' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-06-10T00:53:39.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:39 smithi167 ceph-mon[36917]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-10T00:53:39.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:39 smithi167 ceph-mon[36917]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T00:53:39.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:39 smithi167 ceph-mon[36917]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-06-10T00:53:39.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:39 smithi167 ceph-mon[36917]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-06-10T00:53:39.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:39 smithi167 ceph-mon[36917]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:39.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:39 smithi094 ceph-mon[36708]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-10T00:53:39.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:39 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1955193428' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-06-10T00:53:39.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:39 smithi094 ceph-mon[36708]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-10T00:53:39.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:39 smithi094 ceph-mon[36708]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-10T00:53:39.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:39 smithi094 ceph-mon[36708]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-06-10T00:53:39.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:39 smithi094 ceph-mon[36708]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-06-10T00:53:39.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:39 smithi094 ceph-mon[36708]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T00:53:40.570 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:40 smithi037 ceph-mon[32192]: mds.? [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] up:active 2024-06-10T00:53:40.571 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:40 smithi037 ceph-mon[32192]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-10T00:53:40.571 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:40 smithi037 ceph-mon[32192]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-10T00:53:40.571 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:40 smithi037 ceph-mon[32192]: Cluster is now healthy 2024-06-10T00:53:40.571 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:40 smithi037 ceph-mon[32192]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-06-10T00:53:40.571 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:40 smithi037 ceph-mon[32192]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-06-10T00:53:40.571 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:40 smithi037 ceph-mon[32192]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-06-10T00:53:40.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:40 smithi167 ceph-mon[36917]: mds.? [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] up:active 2024-06-10T00:53:40.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:40 smithi167 ceph-mon[36917]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-10T00:53:40.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:40 smithi167 ceph-mon[36917]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-10T00:53:40.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:40 smithi167 ceph-mon[36917]: Cluster is now healthy 2024-06-10T00:53:40.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:40 smithi167 ceph-mon[36917]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-06-10T00:53:40.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:40 smithi167 ceph-mon[36917]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-06-10T00:53:40.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:40 smithi167 ceph-mon[36917]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-06-10T00:53:40.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:40 smithi094 ceph-mon[36708]: mds.? [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] up:active 2024-06-10T00:53:40.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:40 smithi094 ceph-mon[36708]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-10T00:53:40.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:40 smithi094 ceph-mon[36708]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-10T00:53:40.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:40 smithi094 ceph-mon[36708]: Cluster is now healthy 2024-06-10T00:53:40.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:40 smithi094 ceph-mon[36708]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-06-10T00:53:40.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:40 smithi094 ceph-mon[36708]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-06-10T00:53:40.616 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:40 smithi094 ceph-mon[36708]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-06-10T00:53:40.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.817+0000 7f0f5f8ed700 1 -- 172.21.15.37:0/1931536513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f58101620 msgr2=0x7f0f58101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:40.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.817+0000 7f0f5f8ed700 1 --2- 172.21.15.37:0/1931536513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f58101620 0x7f0f58101a00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0f540099c0 tx=0x7f0f54009cd0 comp rx=0 tx=0).stop 2024-06-10T00:53:40.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.817+0000 7f0f5f8ed700 1 -- 172.21.15.37:0/1931536513 shutdown_connections 2024-06-10T00:53:40.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.817+0000 7f0f5f8ed700 1 --2- 172.21.15.37:0/1931536513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f581066b0 0x7f0f5810e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:40.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.817+0000 7f0f5f8ed700 1 --2- 172.21.15.37:0/1931536513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f58101620 0x7f0f58101a00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:40.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.817+0000 7f0f5f8ed700 1 --2- 172.21.15.37:0/1931536513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f58101fd0 0x7f0f58105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:40.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.817+0000 7f0f5f8ed700 1 -- 172.21.15.37:0/1931536513 >> 172.21.15.37:0/1931536513 conn(0x7f0f58078340 msgr2=0x7f0f58078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:40.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.817+0000 7f0f5f8ed700 1 -- 172.21.15.37:0/1931536513 shutdown_connections 2024-06-10T00:53:40.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.817+0000 7f0f5f8ed700 1 -- 172.21.15.37:0/1931536513 wait complete. 2024-06-10T00:53:40.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.818+0000 7f0f5f8ed700 1 Processor -- start 2024-06-10T00:53:40.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.818+0000 7f0f5f8ed700 1 -- start start 2024-06-10T00:53:40.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.818+0000 7f0f5f8ed700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f58101620 0x7f0f5819ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:40.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.818+0000 7f0f5f8ed700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f58101fd0 0x7f0f5819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:40.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.818+0000 7f0f5f8ed700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f581066b0 0x7f0f5819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:40.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.818+0000 7f0f5f8ed700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0f581116b0 con 0x7f0f58101620 2024-06-10T00:53:40.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.818+0000 7f0f5f8ed700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0f58111530 con 0x7f0f581066b0 2024-06-10T00:53:40.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.818+0000 7f0f5f8ed700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0f58111830 con 0x7f0f58101fd0 2024-06-10T00:53:40.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.818+0000 7f0f5d689700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f58101620 0x7f0f5819ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:40.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.818+0000 7f0f5d689700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f58101620 0x7f0f5819ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49946/0 (socket says 172.21.15.37:49946) 2024-06-10T00:53:40.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.818+0000 7f0f5d689700 1 -- 172.21.15.37:0/1413820504 learned_addr learned my addr 172.21.15.37:0/1413820504 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:53:40.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.818+0000 7f0f5de8a700 1 --2- 172.21.15.37:0/1413820504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f581066b0 0x7f0f5819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:40.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.818+0000 7f0f5ce88700 1 --2- 172.21.15.37:0/1413820504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f58101fd0 0x7f0f5819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:40.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.819+0000 7f0f5d689700 1 -- 172.21.15.37:0/1413820504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f58101fd0 msgr2=0x7f0f5819b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:40.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.819+0000 7f0f5d689700 1 --2- 172.21.15.37:0/1413820504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f58101fd0 0x7f0f5819b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:40.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.819+0000 7f0f5d689700 1 -- 172.21.15.37:0/1413820504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f581066b0 msgr2=0x7f0f5819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:40.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.819+0000 7f0f5d689700 1 --2- 172.21.15.37:0/1413820504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f581066b0 0x7f0f5819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:40.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.819+0000 7f0f5d689700 1 -- 172.21.15.37:0/1413820504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5819fe90 con 0x7f0f58101620 2024-06-10T00:53:40.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.819+0000 7f0f5d689700 1 --2- 172.21.15.37:0/1413820504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f58101620 0x7f0f5819ae80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0f540059d0 tx=0x7f0f540038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:40.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.819+0000 7f0f4e7fc700 1 -- 172.21.15.37:0/1413820504 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54027070 con 0x7f0f58101620 2024-06-10T00:53:40.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.819+0000 7f0f4e7fc700 1 -- 172.21.15.37:0/1413820504 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f54003e40 con 0x7f0f58101620 2024-06-10T00:53:40.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.819+0000 7f0f5f8ed700 1 -- 172.21.15.37:0/1413820504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f581a0120 con 0x7f0f58101620 2024-06-10T00:53:40.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.819+0000 7f0f4e7fc700 1 -- 172.21.15.37:0/1413820504 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54022d00 con 0x7f0f58101620 2024-06-10T00:53:40.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.819+0000 7f0f5f8ed700 1 -- 172.21.15.37:0/1413820504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f581a7a00 con 0x7f0f58101620 2024-06-10T00:53:40.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.820+0000 7f0f5f8ed700 1 -- 172.21.15.37:0/1413820504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f58069180 con 0x7f0f58101620 2024-06-10T00:53:40.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.821+0000 7f0f4e7fc700 1 -- 172.21.15.37:0/1413820504 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0f5402a070 con 0x7f0f58101620 2024-06-10T00:53:40.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.822+0000 7f0f4e7fc700 1 --2- 172.21.15.37:0/1413820504 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0f440a2c20 0x7f0f440a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:40.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.822+0000 7f0f5ce88700 1 --2- 172.21.15.37:0/1413820504 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0f440a2c20 0x7f0f440a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:40.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.822+0000 7f0f4e7fc700 1 -- 172.21.15.37:0/1413820504 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f0f540d0250 con 0x7f0f58101620 2024-06-10T00:53:40.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.823+0000 7f0f5ce88700 1 --2- 172.21.15.37:0/1413820504 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0f440a2c20 0x7f0f440a50e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0f581000a0 tx=0x7f0f4800b3c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:40.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:40.827+0000 7f0f4e7fc700 1 -- 172.21.15.37:0/1413820504 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0f5405e850 con 0x7f0f58101620 2024-06-10T00:53:41.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:41.078+0000 7f0f5f8ed700 1 -- 172.21.15.37:0/1413820504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f0f5802dca0 con 0x7f0f58101620 2024-06-10T00:53:41.088 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jun 10 00:53:40 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-i[67988]: 2024-06-10T00:53:40.661+0000 7fe3ceb57700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-06-10T00:53:41.088 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jun 10 00:53:40 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-i[67988]: 2024-06-10T00:53:40.661+0000 7fe3ceb57700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-06-10T00:53:41.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:41.287+0000 7f0f4e7fc700 1 -- 172.21.15.37:0/1413820504 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f0f54013070 con 0x7f0f58101620 2024-06-10T00:53:41.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:41.289+0000 7f0f5f8ed700 1 -- 172.21.15.37:0/1413820504 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0f440a2c20 msgr2=0x7f0f440a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:41.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:41.289+0000 7f0f5f8ed700 1 --2- 172.21.15.37:0/1413820504 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0f440a2c20 0x7f0f440a50e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0f581000a0 tx=0x7f0f4800b3c0 comp rx=0 tx=0).stop 2024-06-10T00:53:41.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:41.289+0000 7f0f5f8ed700 1 -- 172.21.15.37:0/1413820504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f58101620 msgr2=0x7f0f5819ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:41.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:41.289+0000 7f0f5f8ed700 1 --2- 172.21.15.37:0/1413820504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f58101620 0x7f0f5819ae80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0f540059d0 tx=0x7f0f540038b0 comp rx=0 tx=0).stop 2024-06-10T00:53:41.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:41.289+0000 7f0f5f8ed700 1 -- 172.21.15.37:0/1413820504 shutdown_connections 2024-06-10T00:53:41.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:41.289+0000 7f0f5f8ed700 1 --2- 172.21.15.37:0/1413820504 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0f440a2c20 0x7f0f440a50e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:41.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:41.289+0000 7f0f5f8ed700 1 --2- 172.21.15.37:0/1413820504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f581066b0 0x7f0f5819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:41.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:41.289+0000 7f0f5f8ed700 1 --2- 172.21.15.37:0/1413820504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f58101620 0x7f0f5819ae80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:41.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:41.289+0000 7f0f5f8ed700 1 --2- 172.21.15.37:0/1413820504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f58101fd0 0x7f0f5819b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:41.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:41.289+0000 7f0f5f8ed700 1 -- 172.21.15.37:0/1413820504 >> 172.21.15.37:0/1413820504 conn(0x7f0f58078340 msgr2=0x7f0f58100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:41.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:41.290+0000 7f0f5f8ed700 1 -- 172.21.15.37:0/1413820504 shutdown_connections 2024-06-10T00:53:41.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:41.290+0000 7f0f5f8ed700 1 -- 172.21.15.37:0/1413820504 wait complete. 2024-06-10T00:53:41.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:41 smithi037 ceph-mon[32192]: mds.? [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] up:active 2024-06-10T00:53:41.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:41 smithi037 ceph-mon[32192]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-06-10T00:53:41.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:41 smithi037 ceph-mon[32192]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 1 op/s 2024-06-10T00:53:41.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:41 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1413820504' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-10T00:53:41.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:41 smithi167 ceph-mon[36917]: mds.? [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] up:active 2024-06-10T00:53:41.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:41 smithi167 ceph-mon[36917]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-06-10T00:53:41.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:41 smithi167 ceph-mon[36917]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 1 op/s 2024-06-10T00:53:41.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:41 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1413820504' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-10T00:53:41.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:41 smithi094 ceph-mon[36708]: mds.? [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] up:active 2024-06-10T00:53:41.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:41 smithi094 ceph-mon[36708]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-06-10T00:53:41.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:41 smithi094 ceph-mon[36708]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 1 op/s 2024-06-10T00:53:41.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:41 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1413820504' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-10T00:53:41.925 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-06-10T00:53:42.181 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:53:42.581 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:42 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1413820504' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-10T00:53:42.581 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:42 smithi037 ceph-mon[32192]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-06-10T00:53:42.581 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:42 smithi037 ceph-mon[32192]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T00:53:42.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:42 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1413820504' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-10T00:53:42.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:42 smithi167 ceph-mon[36917]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-06-10T00:53:42.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:42 smithi167 ceph-mon[36917]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T00:53:42.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:42 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1413820504' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-10T00:53:42.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:42 smithi094 ceph-mon[36708]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-06-10T00:53:42.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:42 smithi094 ceph-mon[36708]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T00:53:43.331 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:43 smithi037 ceph-mon[32192]: pgmap v339: 129 pgs: 129 active+clean; 582 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 511 B/s wr, 4 op/s 2024-06-10T00:53:43.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:43 smithi167 ceph-mon[36917]: pgmap v339: 129 pgs: 129 active+clean; 582 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 511 B/s wr, 4 op/s 2024-06-10T00:53:43.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:43 smithi094 ceph-mon[36708]: pgmap v339: 129 pgs: 129 active+clean; 582 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 511 B/s wr, 4 op/s 2024-06-10T00:53:43.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.859+0000 7f47affb3700 1 -- 172.21.15.37:0/3825600509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47a8107860 msgr2=0x7f47a810f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:43.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.859+0000 7f47affb3700 1 --2- 172.21.15.37:0/3825600509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47a8107860 0x7f47a810f3e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f47a000b0d0 tx=0x7f47a000b3e0 comp rx=0 tx=0).stop 2024-06-10T00:53:43.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.859+0000 7f47affb3700 1 -- 172.21.15.37:0/3825600509 shutdown_connections 2024-06-10T00:53:43.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.859+0000 7f47affb3700 1 --2- 172.21.15.37:0/3825600509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47a8107860 0x7f47a810f3e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:43.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.859+0000 7f47affb3700 1 --2- 172.21.15.37:0/3825600509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47a8102770 0x7f47a8102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:43.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.859+0000 7f47affb3700 1 --2- 172.21.15.37:0/3825600509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47a8103120 0x7f47a8107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:43.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.859+0000 7f47affb3700 1 -- 172.21.15.37:0/3825600509 >> 172.21.15.37:0/3825600509 conn(0x7f47a8079600 msgr2=0x7f47a8079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:43.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.859+0000 7f47affb3700 1 -- 172.21.15.37:0/3825600509 shutdown_connections 2024-06-10T00:53:43.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.859+0000 7f47affb3700 1 -- 172.21.15.37:0/3825600509 wait complete. 2024-06-10T00:53:43.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.860+0000 7f47affb3700 1 Processor -- start 2024-06-10T00:53:43.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.860+0000 7f47affb3700 1 -- start start 2024-06-10T00:53:43.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.860+0000 7f47affb3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47a8102770 0x7f47a819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:43.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.860+0000 7f47affb3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47a8103120 0x7f47a819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:43.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.860+0000 7f47affb3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47a8107860 0x7f47a8198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:43.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.860+0000 7f47affb3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f47a8112800 con 0x7f47a8102770 2024-06-10T00:53:43.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.860+0000 7f47affb3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f47a8112680 con 0x7f47a8107860 2024-06-10T00:53:43.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.860+0000 7f47affb3700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f47a8112980 con 0x7f47a8103120 2024-06-10T00:53:43.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.860+0000 7f47add4f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47a8102770 0x7f47a819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:43.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.860+0000 7f47add4f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47a8102770 0x7f47a819e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35696/0 (socket says 172.21.15.37:35696) 2024-06-10T00:53:43.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.860+0000 7f47add4f700 1 -- 172.21.15.37:0/1918020197 learned_addr learned my addr 172.21.15.37:0/1918020197 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:53:43.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.860+0000 7f47ae550700 1 --2- 172.21.15.37:0/1918020197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47a8107860 0x7f47a8198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:43.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.860+0000 7f47ad54e700 1 --2- 172.21.15.37:0/1918020197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47a8103120 0x7f47a819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:43.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.860+0000 7f47add4f700 1 -- 172.21.15.37:0/1918020197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47a8103120 msgr2=0x7f47a819ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:43.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.860+0000 7f47add4f700 1 --2- 172.21.15.37:0/1918020197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47a8103120 0x7f47a819ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:43.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.860+0000 7f47add4f700 1 -- 172.21.15.37:0/1918020197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47a8107860 msgr2=0x7f47a8198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:43.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.860+0000 7f47add4f700 1 --2- 172.21.15.37:0/1918020197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47a8107860 0x7f47a8198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:43.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.860+0000 7f47add4f700 1 -- 172.21.15.37:0/1918020197 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a8199180 con 0x7f47a8102770 2024-06-10T00:53:43.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.860+0000 7f47ad54e700 1 --2- 172.21.15.37:0/1918020197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47a8103120 0x7f47a819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:53:43.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.861+0000 7f47add4f700 1 --2- 172.21.15.37:0/1918020197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47a8102770 0x7f47a819e810 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f479c00b530 tx=0x7f479c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:43.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.861+0000 7f479affd700 1 -- 172.21.15.37:0/1918020197 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c015020 con 0x7f47a8102770 2024-06-10T00:53:43.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.861+0000 7f479affd700 1 -- 172.21.15.37:0/1918020197 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f479c011d10 con 0x7f47a8102770 2024-06-10T00:53:43.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.861+0000 7f47affb3700 1 -- 172.21.15.37:0/1918020197 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a8199470 con 0x7f47a8102770 2024-06-10T00:53:43.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.861+0000 7f479affd700 1 -- 172.21.15.37:0/1918020197 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c00f2e0 con 0x7f47a8102770 2024-06-10T00:53:43.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.861+0000 7f47affb3700 1 -- 172.21.15.37:0/1918020197 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a81a56e0 con 0x7f47a8102770 2024-06-10T00:53:43.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.862+0000 7f47affb3700 1 -- 172.21.15.37:0/1918020197 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a810aaa0 con 0x7f47a8102770 2024-06-10T00:53:43.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.863+0000 7f479affd700 1 -- 172.21.15.37:0/1918020197 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f479c018020 con 0x7f47a8102770 2024-06-10T00:53:43.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.864+0000 7f479affd700 1 --2- 172.21.15.37:0/1918020197 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f47940a2ca0 0x7f47940a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:43.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.864+0000 7f47ad54e700 1 --2- 172.21.15.37:0/1918020197 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f47940a2ca0 0x7f47940a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:43.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.864+0000 7f479affd700 1 -- 172.21.15.37:0/1918020197 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f479c0cebc0 con 0x7f47a8102770 2024-06-10T00:53:43.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.864+0000 7f47ad54e700 1 --2- 172.21.15.37:0/1918020197 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f47940a2ca0 0x7f47940a5160 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f47a4005fd0 tx=0x7f47a4005f60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:43.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:43.868+0000 7f479affd700 1 -- 172.21.15.37:0/1918020197 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f479c01f020 con 0x7f47a8102770 2024-06-10T00:53:44.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:44.123+0000 7f47affb3700 1 -- 172.21.15.37:0/1918020197 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f47a8199f30 con 0x7f47a8102770 2024-06-10T00:53:44.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:44.300+0000 7f479affd700 1 -- 172.21.15.37:0/1918020197 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7f47a8199f30 con 0x7f47a8102770 2024-06-10T00:53:44.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:44.302+0000 7f47affb3700 1 -- 172.21.15.37:0/1918020197 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f47940a2ca0 msgr2=0x7f47940a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:44.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:44.302+0000 7f47affb3700 1 --2- 172.21.15.37:0/1918020197 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f47940a2ca0 0x7f47940a5160 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f47a4005fd0 tx=0x7f47a4005f60 comp rx=0 tx=0).stop 2024-06-10T00:53:44.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:44.302+0000 7f47affb3700 1 -- 172.21.15.37:0/1918020197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47a8102770 msgr2=0x7f47a819e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:44.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:44.302+0000 7f47affb3700 1 --2- 172.21.15.37:0/1918020197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47a8102770 0x7f47a819e810 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f479c00b530 tx=0x7f479c00b840 comp rx=0 tx=0).stop 2024-06-10T00:53:44.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:44.303+0000 7f47affb3700 1 -- 172.21.15.37:0/1918020197 shutdown_connections 2024-06-10T00:53:44.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:44.303+0000 7f47affb3700 1 --2- 172.21.15.37:0/1918020197 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f47940a2ca0 0x7f47940a5160 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:44.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:44.303+0000 7f47affb3700 1 --2- 172.21.15.37:0/1918020197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47a8107860 0x7f47a8198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:44.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:44.303+0000 7f47affb3700 1 --2- 172.21.15.37:0/1918020197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47a8102770 0x7f47a819e810 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:44.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:44.303+0000 7f47affb3700 1 --2- 172.21.15.37:0/1918020197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47a8103120 0x7f47a819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:44.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:44.303+0000 7f47affb3700 1 -- 172.21.15.37:0/1918020197 >> 172.21.15.37:0/1918020197 conn(0x7f47a8079600 msgr2=0x7f47a8100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:44.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:44.303+0000 7f47affb3700 1 -- 172.21.15.37:0/1918020197 shutdown_connections 2024-06-10T00:53:44.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:44.303+0000 7f47affb3700 1 -- 172.21.15.37:0/1918020197 wait complete. 2024-06-10T00:53:44.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:44 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1918020197' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-10T00:53:44.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:44 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1918020197' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-10T00:53:44.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:44 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1918020197' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-10T00:53:44.964 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-06-10T00:53:44.964 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-06-10T00:53:45.220 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:53:45.569 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:45 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1918020197' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-10T00:53:45.569 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:45 smithi037 ceph-mon[32192]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T00:53:45.569 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:45 smithi037 ceph-mon[32192]: pgmap v340: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-06-10T00:53:45.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:45 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1918020197' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-10T00:53:45.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:45 smithi167 ceph-mon[36917]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T00:53:45.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:45 smithi167 ceph-mon[36917]: pgmap v340: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-06-10T00:53:45.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:45 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1918020197' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-10T00:53:45.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:45 smithi094 ceph-mon[36708]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T00:53:45.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:45 smithi094 ceph-mon[36708]: pgmap v340: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-06-10T00:53:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.913+0000 7f9cc0754700 1 -- 172.21.15.37:0/1816431633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cb8102770 msgr2=0x7f9cb8102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.913+0000 7f9cc0754700 1 --2- 172.21.15.37:0/1816431633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cb8102770 0x7f9cb8102b50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9cb40099c0 tx=0x7f9cb4009cd0 comp rx=0 tx=0).stop 2024-06-10T00:53:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.913+0000 7f9cc0754700 1 -- 172.21.15.37:0/1816431633 shutdown_connections 2024-06-10T00:53:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.913+0000 7f9cc0754700 1 --2- 172.21.15.37:0/1816431633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb8107860 0x7f9cb810f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.913+0000 7f9cc0754700 1 --2- 172.21.15.37:0/1816431633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cb8102770 0x7f9cb8102b50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.913+0000 7f9cc0754700 1 --2- 172.21.15.37:0/1816431633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cb8103120 0x7f9cb8107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.913+0000 7f9cc0754700 1 -- 172.21.15.37:0/1816431633 >> 172.21.15.37:0/1816431633 conn(0x7f9cb8079600 msgr2=0x7f9cb8079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.913+0000 7f9cc0754700 1 -- 172.21.15.37:0/1816431633 shutdown_connections 2024-06-10T00:53:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.913+0000 7f9cc0754700 1 -- 172.21.15.37:0/1816431633 wait complete. 2024-06-10T00:53:46.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.913+0000 7f9cc0754700 1 Processor -- start 2024-06-10T00:53:46.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.914+0000 7f9cc0754700 1 -- start start 2024-06-10T00:53:46.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.914+0000 7f9cc0754700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb8102770 0x7f9cb819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:46.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.914+0000 7f9cc0754700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cb8103120 0x7f9cb819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:46.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.914+0000 7f9cc0754700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cb8107860 0x7f9cb8198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:46.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.914+0000 7f9cc0754700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb81127a0 con 0x7f9cb8103120 2024-06-10T00:53:46.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.914+0000 7f9cc0754700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb8112620 con 0x7f9cb8107860 2024-06-10T00:53:46.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.914+0000 7f9cc0754700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb8112920 con 0x7f9cb8102770 2024-06-10T00:53:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.914+0000 7f9cbdcef700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cb8103120 0x7f9cb819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.914+0000 7f9cbdcef700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cb8103120 0x7f9cb819ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35710/0 (socket says 172.21.15.37:35710) 2024-06-10T00:53:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.914+0000 7f9cbdcef700 1 -- 172.21.15.37:0/3441866348 learned_addr learned my addr 172.21.15.37:0/3441866348 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:53:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.914+0000 7f9cbecf1700 1 --2- 172.21.15.37:0/3441866348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cb8107860 0x7f9cb8198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.914+0000 7f9cbe4f0700 1 --2- 172.21.15.37:0/3441866348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb8102770 0x7f9cb819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.914+0000 7f9cbdcef700 1 -- 172.21.15.37:0/3441866348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb8102770 msgr2=0x7f9cb819e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.914+0000 7f9cbdcef700 1 --2- 172.21.15.37:0/3441866348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb8102770 0x7f9cb819e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.914+0000 7f9cbdcef700 1 -- 172.21.15.37:0/3441866348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cb8107860 msgr2=0x7f9cb8198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.914+0000 7f9cbdcef700 1 --2- 172.21.15.37:0/3441866348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cb8107860 0x7f9cb8198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.914+0000 7f9cbdcef700 1 -- 172.21.15.37:0/3441866348 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb8199160 con 0x7f9cb8103120 2024-06-10T00:53:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.915+0000 7f9cbdcef700 1 --2- 172.21.15.37:0/3441866348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cb8103120 0x7f9cb819ed80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800d980 tx=0x7f9ca800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.915+0000 7f9caf7fe700 1 -- 172.21.15.37:0/3441866348 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8016070 con 0x7f9cb8103120 2024-06-10T00:53:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.915+0000 7f9caf7fe700 1 -- 172.21.15.37:0/3441866348 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ca8004d10 con 0x7f9cb8103120 2024-06-10T00:53:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.915+0000 7f9cc0754700 1 -- 172.21.15.37:0/3441866348 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb8199450 con 0x7f9cb8103120 2024-06-10T00:53:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.915+0000 7f9caf7fe700 1 -- 172.21.15.37:0/3441866348 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca80055f0 con 0x7f9cb8103120 2024-06-10T00:53:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.915+0000 7f9cc0754700 1 -- 172.21.15.37:0/3441866348 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb81a5710 con 0x7f9cb8103120 2024-06-10T00:53:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.916+0000 7f9cc0754700 1 -- 172.21.15.37:0/3441866348 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cb810aaa0 con 0x7f9cb8103120 2024-06-10T00:53:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.917+0000 7f9caf7fe700 1 -- 172.21.15.37:0/3441866348 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9ca8004750 con 0x7f9cb8103120 2024-06-10T00:53:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.918+0000 7f9caf7fe700 1 --2- 172.21.15.37:0/3441866348 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9ca40a2c20 0x7f9ca40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.918+0000 7f9cbe4f0700 1 --2- 172.21.15.37:0/3441866348 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9ca40a2c20 0x7f9ca40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.918+0000 7f9caf7fe700 1 -- 172.21.15.37:0/3441866348 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f9ca80cf7d0 con 0x7f9cb8103120 2024-06-10T00:53:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.918+0000 7f9cbe4f0700 1 --2- 172.21.15.37:0/3441866348 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9ca40a2c20 0x7f9ca40a50e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4000c00 tx=0x7f9cb4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:46.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:46.921+0000 7f9caf7fe700 1 -- 172.21.15.37:0/3441866348 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9ca801f020 con 0x7f9cb8103120 2024-06-10T00:53:47.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:47.158+0000 7f9cc0754700 1 -- 172.21.15.37:0/3441866348 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9cb80628d0 con 0x7f9ca40a2c20 2024-06-10T00:53:47.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:47.187+0000 7f9caf7fe700 1 -- 172.21.15.37:0/3441866348 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9cb80628d0 con 0x7f9ca40a2c20 2024-06-10T00:53:47.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:47.189+0000 7f9cc0754700 1 -- 172.21.15.37:0/3441866348 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9ca40a2c20 msgr2=0x7f9ca40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:47.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:47.189+0000 7f9cc0754700 1 --2- 172.21.15.37:0/3441866348 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9ca40a2c20 0x7f9ca40a50e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4000c00 tx=0x7f9cb4005fb0 comp rx=0 tx=0).stop 2024-06-10T00:53:47.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:47.189+0000 7f9cc0754700 1 -- 172.21.15.37:0/3441866348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cb8103120 msgr2=0x7f9cb819ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:47.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:47.189+0000 7f9cc0754700 1 --2- 172.21.15.37:0/3441866348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cb8103120 0x7f9cb819ed80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800d980 tx=0x7f9ca800dc90 comp rx=0 tx=0).stop 2024-06-10T00:53:47.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:47.189+0000 7f9cc0754700 1 -- 172.21.15.37:0/3441866348 shutdown_connections 2024-06-10T00:53:47.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:47.189+0000 7f9cc0754700 1 --2- 172.21.15.37:0/3441866348 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9ca40a2c20 0x7f9ca40a50e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:47.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:47.189+0000 7f9cc0754700 1 --2- 172.21.15.37:0/3441866348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cb8107860 0x7f9cb8198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:47.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:47.189+0000 7f9cc0754700 1 --2- 172.21.15.37:0/3441866348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cb8102770 0x7f9cb819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:47.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:47.189+0000 7f9cc0754700 1 --2- 172.21.15.37:0/3441866348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cb8103120 0x7f9cb819ed80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:47.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:47.189+0000 7f9cc0754700 1 -- 172.21.15.37:0/3441866348 >> 172.21.15.37:0/3441866348 conn(0x7f9cb8079600 msgr2=0x7f9cb8100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:47.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:47.189+0000 7f9cc0754700 1 -- 172.21.15.37:0/3441866348 shutdown_connections 2024-06-10T00:53:47.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:47.190+0000 7f9cc0754700 1 -- 172.21.15.37:0/3441866348 wait complete. 2024-06-10T00:53:47.507 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:47 smithi037 ceph-mon[32192]: pgmap v341: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-10T00:53:47.507 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:47 smithi037 ceph-mon[32192]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:53:47.507 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:53:47 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:53:47.163+0000 7fcb31c8c700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:53:47.508 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:53:47 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:53:47.163+0000 7fcb31c8c700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:53:47.508 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:53:47 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:53:47.163+0000 7fcb31c8c700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:53:47.508 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:53:47 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:53:47.163+0000 7fcb31c8c700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:53:47.508 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:53:47 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:53:47.163+0000 7fcb31c8c700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:53:47.816 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-06-10T00:53:47.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:47 smithi167 ceph-mon[36917]: pgmap v341: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-10T00:53:47.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:47 smithi167 ceph-mon[36917]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:53:47.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:47 smithi094 ceph-mon[36708]: pgmap v341: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-10T00:53:47.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:47 smithi094 ceph-mon[36708]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:53:48.074 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:53:49.721 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:49 smithi037 ceph-mon[32192]: pgmap v342: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-06-10T00:53:49.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.748+0000 7febdcd13700 1 -- 172.21.15.37:0/2382889765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd8102770 msgr2=0x7febd8102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:49.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.748+0000 7febdcd13700 1 --2- 172.21.15.37:0/2382889765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd8102770 0x7febd8102b50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7febc80099c0 tx=0x7febc8009cd0 comp rx=0 tx=0).stop 2024-06-10T00:53:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.748+0000 7febdcd13700 1 -- 172.21.15.37:0/2382889765 shutdown_connections 2024-06-10T00:53:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.748+0000 7febdcd13700 1 --2- 172.21.15.37:0/2382889765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febd8107860 0x7febd810f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.748+0000 7febdcd13700 1 --2- 172.21.15.37:0/2382889765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd8102770 0x7febd8102b50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.748+0000 7febdcd13700 1 --2- 172.21.15.37:0/2382889765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febd8103120 0x7febd8107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.748+0000 7febdcd13700 1 -- 172.21.15.37:0/2382889765 >> 172.21.15.37:0/2382889765 conn(0x7febd8079600 msgr2=0x7febd8079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.748+0000 7febdcd13700 1 -- 172.21.15.37:0/2382889765 shutdown_connections 2024-06-10T00:53:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.748+0000 7febdcd13700 1 -- 172.21.15.37:0/2382889765 wait complete. 2024-06-10T00:53:49.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.749+0000 7febdcd13700 1 Processor -- start 2024-06-10T00:53:49.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.749+0000 7febdcd13700 1 -- start start 2024-06-10T00:53:49.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.749+0000 7febdcd13700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febd8102770 0x7febd819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:49.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.749+0000 7febdcd13700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febd8103120 0x7febd819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:49.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.749+0000 7febdcd13700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd8107860 0x7febd8198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:49.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.749+0000 7febdcd13700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7febd8112780 con 0x7febd8107860 2024-06-10T00:53:49.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.749+0000 7febdcd13700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7febd8112600 con 0x7febd8103120 2024-06-10T00:53:49.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.749+0000 7febdcd13700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7febd8112900 con 0x7febd8102770 2024-06-10T00:53:49.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.749+0000 7febd6d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd8107860 0x7febd8198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:49.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.749+0000 7febd6d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd8107860 0x7febd8198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35760/0 (socket says 172.21.15.37:35760) 2024-06-10T00:53:49.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.749+0000 7febd6d9d700 1 -- 172.21.15.37:0/797266286 learned_addr learned my addr 172.21.15.37:0/797266286 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:53:49.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.749+0000 7febd659c700 1 --2- 172.21.15.37:0/797266286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febd8102770 0x7febd819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:49.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.749+0000 7febd5d9b700 1 --2- 172.21.15.37:0/797266286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febd8103120 0x7febd819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:49.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.750+0000 7febd6d9d700 1 -- 172.21.15.37:0/797266286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febd8102770 msgr2=0x7febd819e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:49.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.750+0000 7febd6d9d700 1 --2- 172.21.15.37:0/797266286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febd8102770 0x7febd819e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:49.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.750+0000 7febd6d9d700 1 -- 172.21.15.37:0/797266286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febd8103120 msgr2=0x7febd819ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:49.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.750+0000 7febd6d9d700 1 --2- 172.21.15.37:0/797266286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febd8103120 0x7febd819ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:49.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.750+0000 7febd6d9d700 1 -- 172.21.15.37:0/797266286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd81990a0 con 0x7febd8107860 2024-06-10T00:53:49.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.750+0000 7febd5d9b700 1 --2- 172.21.15.37:0/797266286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febd8103120 0x7febd819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:53:49.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.750+0000 7febd6d9d700 1 --2- 172.21.15.37:0/797266286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd8107860 0x7febd8198940 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7febd0007ee0 tx=0x7febd000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:49.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.750+0000 7febc77fe700 1 -- 172.21.15.37:0/797266286 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd000dbd0 con 0x7febd8107860 2024-06-10T00:53:49.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.750+0000 7febdcd13700 1 -- 172.21.15.37:0/797266286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd8199390 con 0x7febd8107860 2024-06-10T00:53:49.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.750+0000 7febc77fe700 1 -- 172.21.15.37:0/797266286 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febd000dd70 con 0x7febd8107860 2024-06-10T00:53:49.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.750+0000 7febc77fe700 1 -- 172.21.15.37:0/797266286 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd0005a60 con 0x7febd8107860 2024-06-10T00:53:49.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.750+0000 7febdcd13700 1 -- 172.21.15.37:0/797266286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd81a5650 con 0x7febd8107860 2024-06-10T00:53:49.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.751+0000 7febdcd13700 1 -- 172.21.15.37:0/797266286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd8069180 con 0x7febd8107860 2024-06-10T00:53:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.752+0000 7febc77fe700 1 -- 172.21.15.37:0/797266286 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7febd001a020 con 0x7febd8107860 2024-06-10T00:53:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.753+0000 7febc77fe700 1 --2- 172.21.15.37:0/797266286 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7febc00a2cb0 0x7febc00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.753+0000 7febd659c700 1 --2- 172.21.15.37:0/797266286 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7febc00a2cb0 0x7febc00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.753+0000 7febc77fe700 1 -- 172.21.15.37:0/797266286 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7febd00cf330 con 0x7febd8107860 2024-06-10T00:53:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.753+0000 7febd659c700 1 --2- 172.21.15.37:0/797266286 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7febc00a2cb0 0x7febc00a5170 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7febc8000c00 tx=0x7febc8005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:49.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.756+0000 7febc77fe700 1 -- 172.21.15.37:0/797266286 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7febd0010070 con 0x7febd8107860 2024-06-10T00:53:49.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:49 smithi167 ceph-mon[36917]: pgmap v342: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-06-10T00:53:49.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:49 smithi094 ceph-mon[36708]: pgmap v342: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-06-10T00:53:49.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:49.997+0000 7febdcd13700 1 -- 172.21.15.37:0/797266286 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7febd8100b90 con 0x7febc00a2cb0 2024-06-10T00:53:50.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:50.028+0000 7febc77fe700 1 -- 172.21.15.37:0/797266286 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7febd8100b90 con 0x7febc00a2cb0 2024-06-10T00:53:50.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:50.030+0000 7febdcd13700 1 -- 172.21.15.37:0/797266286 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7febc00a2cb0 msgr2=0x7febc00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:50.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:50.030+0000 7febdcd13700 1 --2- 172.21.15.37:0/797266286 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7febc00a2cb0 0x7febc00a5170 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7febc8000c00 tx=0x7febc8005fb0 comp rx=0 tx=0).stop 2024-06-10T00:53:50.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:50.030+0000 7febdcd13700 1 -- 172.21.15.37:0/797266286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd8107860 msgr2=0x7febd8198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:50.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:50.030+0000 7febdcd13700 1 --2- 172.21.15.37:0/797266286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd8107860 0x7febd8198940 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7febd0007ee0 tx=0x7febd000d3b0 comp rx=0 tx=0).stop 2024-06-10T00:53:50.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:50.030+0000 7febdcd13700 1 -- 172.21.15.37:0/797266286 shutdown_connections 2024-06-10T00:53:50.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:50.030+0000 7febdcd13700 1 --2- 172.21.15.37:0/797266286 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7febc00a2cb0 0x7febc00a5170 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:50.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:50.030+0000 7febdcd13700 1 --2- 172.21.15.37:0/797266286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd8107860 0x7febd8198940 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:50.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:50.030+0000 7febdcd13700 1 --2- 172.21.15.37:0/797266286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febd8102770 0x7febd819e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:50.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:50.030+0000 7febdcd13700 1 --2- 172.21.15.37:0/797266286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febd8103120 0x7febd819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:50.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:50.030+0000 7febdcd13700 1 -- 172.21.15.37:0/797266286 >> 172.21.15.37:0/797266286 conn(0x7febd8079600 msgr2=0x7febd8100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:50.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:50.030+0000 7febdcd13700 1 -- 172.21.15.37:0/797266286 shutdown_connections 2024-06-10T00:53:50.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:50.030+0000 7febdcd13700 1 -- 172.21.15.37:0/797266286 wait complete. 2024-06-10T00:53:50.608 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:50 smithi037 ceph-mon[32192]: mgrmap e17: x(active, since 8m), standbys: y, z 2024-06-10T00:53:50.609 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:50 smithi037 ceph-mon[32192]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:53:50.638 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-06-10T00:53:50.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:50 smithi167 ceph-mon[36917]: mgrmap e17: x(active, since 8m), standbys: y, z 2024-06-10T00:53:50.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:50 smithi167 ceph-mon[36917]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:53:50.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:50 smithi094 ceph-mon[36708]: mgrmap e17: x(active, since 8m), standbys: y, z 2024-06-10T00:53:50.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:50 smithi094 ceph-mon[36708]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:53:50.893 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:53:51.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:51 smithi037 ceph-mon[32192]: pgmap v343: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-06-10T00:53:51.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:51 smithi167 ceph-mon[36917]: pgmap v343: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-06-10T00:53:51.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:51 smithi094 ceph-mon[36708]: pgmap v343: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-06-10T00:53:52.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.581+0000 7f20a88f9700 1 -- 172.21.15.37:0/834679314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20a4102740 msgr2=0x7f20a4102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:52.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.581+0000 7f20a88f9700 1 --2- 172.21.15.37:0/834679314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20a4102740 0x7f20a4102b20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f20940099c0 tx=0x7f2094009cd0 comp rx=0 tx=0).stop 2024-06-10T00:53:52.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.581+0000 7f20a88f9700 1 -- 172.21.15.37:0/834679314 shutdown_connections 2024-06-10T00:53:52.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.581+0000 7f20a88f9700 1 --2- 172.21.15.37:0/834679314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20a4107840 0x7f20a410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:52.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.581+0000 7f20a88f9700 1 --2- 172.21.15.37:0/834679314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20a4102740 0x7f20a4102b20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:52.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.581+0000 7f20a88f9700 1 --2- 172.21.15.37:0/834679314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20a41030f0 0x7f20a4107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:52.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.581+0000 7f20a88f9700 1 -- 172.21.15.37:0/834679314 >> 172.21.15.37:0/834679314 conn(0x7f20a40795f0 msgr2=0x7f20a4079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:52.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.581+0000 7f20a88f9700 1 -- 172.21.15.37:0/834679314 shutdown_connections 2024-06-10T00:53:52.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.581+0000 7f20a88f9700 1 -- 172.21.15.37:0/834679314 wait complete. 2024-06-10T00:53:52.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.582+0000 7f20a88f9700 1 Processor -- start 2024-06-10T00:53:52.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.582+0000 7f20a88f9700 1 -- start start 2024-06-10T00:53:52.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.582+0000 7f20a88f9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20a4102740 0x7f20a419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:52.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.582+0000 7f20a88f9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20a41030f0 0x7f20a419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:52.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.582+0000 7f20a88f9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20a4107840 0x7f20a4198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:52.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.582+0000 7f20a88f9700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f20a4112760 con 0x7f20a4102740 2024-06-10T00:53:52.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.582+0000 7f20a88f9700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f20a41125e0 con 0x7f20a41030f0 2024-06-10T00:53:52.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.582+0000 7f20a88f9700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f20a41128e0 con 0x7f20a4107840 2024-06-10T00:53:52.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.582+0000 7f20a259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20a4102740 0x7f20a419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:52.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.582+0000 7f20a259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20a4102740 0x7f20a419e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57294/0 (socket says 172.21.15.37:57294) 2024-06-10T00:53:52.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.582+0000 7f20a259c700 1 -- 172.21.15.37:0/2023479237 learned_addr learned my addr 172.21.15.37:0/2023479237 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:53:52.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.582+0000 7f209bfff700 1 --2- 172.21.15.37:0/2023479237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20a41030f0 0x7f20a419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:52.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.582+0000 7f20a2d9d700 1 --2- 172.21.15.37:0/2023479237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20a4107840 0x7f20a4198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:52.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.582+0000 7f20a259c700 1 -- 172.21.15.37:0/2023479237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20a4107840 msgr2=0x7f20a4198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:52.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.582+0000 7f20a259c700 1 --2- 172.21.15.37:0/2023479237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20a4107840 0x7f20a4198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:52.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.582+0000 7f20a259c700 1 -- 172.21.15.37:0/2023479237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20a41030f0 msgr2=0x7f20a419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:52.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.582+0000 7f20a259c700 1 --2- 172.21.15.37:0/2023479237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20a41030f0 0x7f20a419ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:52.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.582+0000 7f20a259c700 1 -- 172.21.15.37:0/2023479237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a41991c0 con 0x7f20a4102740 2024-06-10T00:53:52.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.583+0000 7f20a259c700 1 --2- 172.21.15.37:0/2023479237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20a4102740 0x7f20a419e850 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2094006010 tx=0x7f2094009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:52.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.583+0000 7f209b7fe700 1 -- 172.21.15.37:0/2023479237 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2094026070 con 0x7f20a4102740 2024-06-10T00:53:52.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.583+0000 7f209b7fe700 1 -- 172.21.15.37:0/2023479237 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2094004cd0 con 0x7f20a4102740 2024-06-10T00:53:52.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.583+0000 7f20a88f9700 1 -- 172.21.15.37:0/2023479237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a4199450 con 0x7f20a4102740 2024-06-10T00:53:52.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.583+0000 7f209b7fe700 1 -- 172.21.15.37:0/2023479237 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2094003ba0 con 0x7f20a4102740 2024-06-10T00:53:52.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.583+0000 7f20a88f9700 1 -- 172.21.15.37:0/2023479237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a41a5720 con 0x7f20a4102740 2024-06-10T00:53:52.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.584+0000 7f20a88f9700 1 -- 172.21.15.37:0/2023479237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20a4069180 con 0x7f20a4102740 2024-06-10T00:53:52.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.585+0000 7f209b7fe700 1 -- 172.21.15.37:0/2023479237 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2094029070 con 0x7f20a4102740 2024-06-10T00:53:52.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.586+0000 7f209b7fe700 1 --2- 172.21.15.37:0/2023479237 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f20900a2cb0 0x7f20900a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:52.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.586+0000 7f209bfff700 1 --2- 172.21.15.37:0/2023479237 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f20900a2cb0 0x7f20900a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:52.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.586+0000 7f209b7fe700 1 -- 172.21.15.37:0/2023479237 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f20940cfe10 con 0x7f20a4102740 2024-06-10T00:53:52.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.587+0000 7f209bfff700 1 --2- 172.21.15.37:0/2023479237 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f20900a2cb0 0x7f20900a5170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f20a4102040 tx=0x7f208c00a300 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:52.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.591+0000 7f209b7fe700 1 -- 172.21.15.37:0/2023479237 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f209405d850 con 0x7f20a4102740 2024-06-10T00:53:52.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.834+0000 7f20a88f9700 1 -- 172.21.15.37:0/2023479237 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f20a40628d0 con 0x7f20900a2cb0 2024-06-10T00:53:52.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.839+0000 7f209b7fe700 1 -- 172.21.15.37:0/2023479237 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f20a40628d0 con 0x7f20900a2cb0 2024-06-10T00:53:52.840 INFO:teuthology.orchestra.run.smithi037.stdout:/volumes/qa/sv_0/0b26a0b3-87f5-4b36-8bde-d6ebb2d7c000 2024-06-10T00:53:52.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.842+0000 7f20a88f9700 1 -- 172.21.15.37:0/2023479237 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f20900a2cb0 msgr2=0x7f20900a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:52.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.842+0000 7f20a88f9700 1 --2- 172.21.15.37:0/2023479237 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f20900a2cb0 0x7f20900a5170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f20a4102040 tx=0x7f208c00a300 comp rx=0 tx=0).stop 2024-06-10T00:53:52.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.842+0000 7f20a88f9700 1 -- 172.21.15.37:0/2023479237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20a4102740 msgr2=0x7f20a419e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:52.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.842+0000 7f20a88f9700 1 --2- 172.21.15.37:0/2023479237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20a4102740 0x7f20a419e850 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2094006010 tx=0x7f2094009f80 comp rx=0 tx=0).stop 2024-06-10T00:53:52.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.842+0000 7f20a88f9700 1 -- 172.21.15.37:0/2023479237 shutdown_connections 2024-06-10T00:53:52.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.842+0000 7f20a88f9700 1 --2- 172.21.15.37:0/2023479237 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f20900a2cb0 0x7f20900a5170 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:52.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.842+0000 7f20a88f9700 1 --2- 172.21.15.37:0/2023479237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20a4107840 0x7f20a4198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:52.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.842+0000 7f20a88f9700 1 --2- 172.21.15.37:0/2023479237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20a4102740 0x7f20a419e850 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:52.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.842+0000 7f20a88f9700 1 --2- 172.21.15.37:0/2023479237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20a41030f0 0x7f20a419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:52.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.842+0000 7f20a88f9700 1 -- 172.21.15.37:0/2023479237 >> 172.21.15.37:0/2023479237 conn(0x7f20a40795f0 msgr2=0x7f20a4108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:52.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.842+0000 7f20a88f9700 1 -- 172.21.15.37:0/2023479237 shutdown_connections 2024-06-10T00:53:52.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:52.842+0000 7f20a88f9700 1 -- 172.21.15.37:0/2023479237 wait complete. 2024-06-10T00:53:53.493 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:53 smithi037 ceph-mon[32192]: pgmap v344: 129 pgs: 129 active+clean; 609 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.4 KiB/s wr, 12 op/s 2024-06-10T00:53:53.493 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:53 smithi037 ceph-mon[32192]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:53:53.523 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-06-10T00:53:53.774 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:53:53.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:53 smithi167 ceph-mon[36917]: pgmap v344: 129 pgs: 129 active+clean; 609 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.4 KiB/s wr, 12 op/s 2024-06-10T00:53:53.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:53 smithi167 ceph-mon[36917]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:53:53.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:53 smithi094 ceph-mon[36708]: pgmap v344: 129 pgs: 129 active+clean; 609 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.4 KiB/s wr, 12 op/s 2024-06-10T00:53:53.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:53 smithi094 ceph-mon[36708]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:53:55.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.478+0000 7fc3db0a2700 1 -- 172.21.15.37:0/3094313352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d41024e0 msgr2=0x7fc3d4102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:55.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.478+0000 7fc3db0a2700 1 --2- 172.21.15.37:0/3094313352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d41024e0 0x7fc3d4102940 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0009990 tx=0x7fc3c0009ca0 comp rx=0 tx=0).stop 2024-06-10T00:53:55.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.478+0000 7fc3db0a2700 1 -- 172.21.15.37:0/3094313352 shutdown_connections 2024-06-10T00:53:55.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.478+0000 7fc3db0a2700 1 --2- 172.21.15.37:0/3094313352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3d4102e80 0x7fc3d410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:55.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.478+0000 7fc3db0a2700 1 --2- 172.21.15.37:0/3094313352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3d41084f0 0x7fc3d41088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:55.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.478+0000 7fc3db0a2700 1 --2- 172.21.15.37:0/3094313352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d41024e0 0x7fc3d4102940 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:55.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.478+0000 7fc3db0a2700 1 -- 172.21.15.37:0/3094313352 >> 172.21.15.37:0/3094313352 conn(0x7fc3d4079530 msgr2=0x7fc3d4079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:55.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.478+0000 7fc3db0a2700 1 -- 172.21.15.37:0/3094313352 shutdown_connections 2024-06-10T00:53:55.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.478+0000 7fc3db0a2700 1 -- 172.21.15.37:0/3094313352 wait complete. 2024-06-10T00:53:55.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.479+0000 7fc3db0a2700 1 Processor -- start 2024-06-10T00:53:55.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.479+0000 7fc3db0a2700 1 -- start start 2024-06-10T00:53:55.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.479+0000 7fc3db0a2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3d41024e0 0x7fc3d419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:55.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.479+0000 7fc3db0a2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3d4102e80 0x7fc3d419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:55.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.479+0000 7fc3db0a2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d41084f0 0x7fc3d41989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:55.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.479+0000 7fc3db0a2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d4112750 con 0x7fc3d41084f0 2024-06-10T00:53:55.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.479+0000 7fc3db0a2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d41125d0 con 0x7fc3d41024e0 2024-06-10T00:53:55.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.479+0000 7fc3db0a2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d41128d0 con 0x7fc3d4102e80 2024-06-10T00:53:55.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.479+0000 7fc3d963f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d41084f0 0x7fc3d41989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:55.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.479+0000 7fc3d963f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d41084f0 0x7fc3d41989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57316/0 (socket says 172.21.15.37:57316) 2024-06-10T00:53:55.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.479+0000 7fc3d963f700 1 -- 172.21.15.37:0/45346290 learned_addr learned my addr 172.21.15.37:0/45346290 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:53:55.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.479+0000 7fc3d3fff700 1 --2- 172.21.15.37:0/45346290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3d4102e80 0x7fc3d419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:55.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.479+0000 7fc3d8e3e700 1 --2- 172.21.15.37:0/45346290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3d41024e0 0x7fc3d419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:55.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.479+0000 7fc3d963f700 1 -- 172.21.15.37:0/45346290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3d4102e80 msgr2=0x7fc3d419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:55.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.479+0000 7fc3d963f700 1 --2- 172.21.15.37:0/45346290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3d4102e80 0x7fc3d419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:55.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.479+0000 7fc3d963f700 1 -- 172.21.15.37:0/45346290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3d41024e0 msgr2=0x7fc3d419e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:55.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.480+0000 7fc3d963f700 1 --2- 172.21.15.37:0/45346290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3d41024e0 0x7fc3d419e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:55.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.480+0000 7fc3d963f700 1 -- 172.21.15.37:0/45346290 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d4199130 con 0x7fc3d41084f0 2024-06-10T00:53:55.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.480+0000 7fc3d8e3e700 1 --2- 172.21.15.37:0/45346290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3d41024e0 0x7fc3d419e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:53:55.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.480+0000 7fc3d963f700 1 --2- 172.21.15.37:0/45346290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d41084f0 0x7fc3d41989d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc007ee0 tx=0x7fc3cc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:55.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.480+0000 7fc3d1ffb700 1 -- 172.21.15.37:0/45346290 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc00dbd0 con 0x7fc3d41084f0 2024-06-10T00:53:55.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.480+0000 7fc3d1ffb700 1 -- 172.21.15.37:0/45346290 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3cc00dd70 con 0x7fc3d41084f0 2024-06-10T00:53:55.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.480+0000 7fc3db0a2700 1 -- 172.21.15.37:0/45346290 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3d4199420 con 0x7fc3d41084f0 2024-06-10T00:53:55.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.480+0000 7fc3d1ffb700 1 -- 172.21.15.37:0/45346290 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc005a60 con 0x7fc3d41084f0 2024-06-10T00:53:55.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.480+0000 7fc3db0a2700 1 -- 172.21.15.37:0/45346290 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3d41a5690 con 0x7fc3d41084f0 2024-06-10T00:53:55.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.481+0000 7fc3db0a2700 1 -- 172.21.15.37:0/45346290 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3d4069180 con 0x7fc3d41084f0 2024-06-10T00:53:55.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.482+0000 7fc3d1ffb700 1 -- 172.21.15.37:0/45346290 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc3cc01a020 con 0x7fc3d41084f0 2024-06-10T00:53:55.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.483+0000 7fc3d1ffb700 1 --2- 172.21.15.37:0/45346290 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3c40a2d30 0x7fc3c40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:55.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.483+0000 7fc3d8e3e700 1 --2- 172.21.15.37:0/45346290 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3c40a2d30 0x7fc3c40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:55.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.483+0000 7fc3d1ffb700 1 -- 172.21.15.37:0/45346290 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fc3cc0cec70 con 0x7fc3d41084f0 2024-06-10T00:53:55.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.484+0000 7fc3d8e3e700 1 --2- 172.21.15.37:0/45346290 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3c40a2d30 0x7fc3c40a51f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8005ea0 tx=0x7fc3c8005e30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:55.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.487+0000 7fc3d1ffb700 1 -- 172.21.15.37:0/45346290 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc3cc010070 con 0x7fc3d41084f0 2024-06-10T00:53:55.488 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:55 smithi037 ceph-mon[32192]: pgmap v345: 129 pgs: 129 active+clean; 609 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-06-10T00:53:55.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.728+0000 7fc3db0a2700 1 -- 172.21.15.37:0/45346290 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc3d40628d0 con 0x7fc3c40a2d30 2024-06-10T00:53:55.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.753+0000 7fc3d1ffb700 1 -- 172.21.15.37:0/45346290 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc3d40628d0 con 0x7fc3c40a2d30 2024-06-10T00:53:55.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.755+0000 7fc3db0a2700 1 -- 172.21.15.37:0/45346290 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3c40a2d30 msgr2=0x7fc3c40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:55.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.755+0000 7fc3db0a2700 1 --2- 172.21.15.37:0/45346290 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3c40a2d30 0x7fc3c40a51f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8005ea0 tx=0x7fc3c8005e30 comp rx=0 tx=0).stop 2024-06-10T00:53:55.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.755+0000 7fc3db0a2700 1 -- 172.21.15.37:0/45346290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d41084f0 msgr2=0x7fc3d41989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:55.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.755+0000 7fc3db0a2700 1 --2- 172.21.15.37:0/45346290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d41084f0 0x7fc3d41989d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc007ee0 tx=0x7fc3cc00d3b0 comp rx=0 tx=0).stop 2024-06-10T00:53:55.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.755+0000 7fc3db0a2700 1 -- 172.21.15.37:0/45346290 shutdown_connections 2024-06-10T00:53:55.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.755+0000 7fc3db0a2700 1 --2- 172.21.15.37:0/45346290 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3c40a2d30 0x7fc3c40a51f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:55.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.755+0000 7fc3db0a2700 1 --2- 172.21.15.37:0/45346290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d41084f0 0x7fc3d41989d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:55.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.755+0000 7fc3db0a2700 1 --2- 172.21.15.37:0/45346290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3d41024e0 0x7fc3d419e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:55.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.755+0000 7fc3db0a2700 1 --2- 172.21.15.37:0/45346290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3d4102e80 0x7fc3d419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:55.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.755+0000 7fc3db0a2700 1 -- 172.21.15.37:0/45346290 >> 172.21.15.37:0/45346290 conn(0x7fc3d4079530 msgr2=0x7fc3d4100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:55.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.755+0000 7fc3db0a2700 1 -- 172.21.15.37:0/45346290 shutdown_connections 2024-06-10T00:53:55.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:55.755+0000 7fc3db0a2700 1 -- 172.21.15.37:0/45346290 wait complete. 2024-06-10T00:53:55.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:55 smithi167 ceph-mon[36917]: pgmap v345: 129 pgs: 129 active+clean; 609 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-06-10T00:53:55.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:55 smithi094 ceph-mon[36708]: pgmap v345: 129 pgs: 129 active+clean; 609 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-06-10T00:53:56.491 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-06-10T00:53:56.743 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:53:56.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:56 smithi167 ceph-mon[36917]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:53:56.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:56 smithi094 ceph-mon[36708]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:53:56.866 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:56 smithi037 ceph-mon[32192]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:53:57.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:57 smithi167 ceph-mon[36917]: pgmap v346: 129 pgs: 129 active+clean; 627 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-06-10T00:53:57.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:57 smithi094 ceph-mon[36708]: pgmap v346: 129 pgs: 129 active+clean; 627 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-06-10T00:53:57.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:57 smithi037 ceph-mon[32192]: pgmap v346: 129 pgs: 129 active+clean; 627 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-06-10T00:53:58.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.454+0000 7f266e871700 1 -- 172.21.15.37:0/1626164121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2668101390 msgr2=0x7f2668101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:58.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.454+0000 7f266e871700 1 --2- 172.21.15.37:0/1626164121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2668101390 0x7f2668101810 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f265c009990 tx=0x7f265c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:53:58.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.454+0000 7f266e871700 1 -- 172.21.15.37:0/1626164121 shutdown_connections 2024-06-10T00:53:58.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.454+0000 7f266e871700 1 --2- 172.21.15.37:0/1626164121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2668101d50 0x7f266810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:58.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.454+0000 7f266e871700 1 --2- 172.21.15.37:0/1626164121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2668107400 0x7f26681077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:58.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.454+0000 7f266e871700 1 --2- 172.21.15.37:0/1626164121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2668101390 0x7f2668101810 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:58.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.454+0000 7f266e871700 1 -- 172.21.15.37:0/1626164121 >> 172.21.15.37:0/1626164121 conn(0x7f2668078340 msgr2=0x7f2668078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:58.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.455+0000 7f266e871700 1 -- 172.21.15.37:0/1626164121 shutdown_connections 2024-06-10T00:53:58.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.455+0000 7f266e871700 1 -- 172.21.15.37:0/1626164121 wait complete. 2024-06-10T00:53:58.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.455+0000 7f266e871700 1 Processor -- start 2024-06-10T00:53:58.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.455+0000 7f266e871700 1 -- start start 2024-06-10T00:53:58.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.456+0000 7f266e871700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2668101390 0x7f266819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:58.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.456+0000 7f266e871700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2668101d50 0x7f266819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:58.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.456+0000 7f266e871700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2668107400 0x7f2668198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:58.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.456+0000 7f266e871700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f26681116a0 con 0x7f2668107400 2024-06-10T00:53:58.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.456+0000 7f266e871700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2668111520 con 0x7f2668101390 2024-06-10T00:53:58.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.456+0000 7f266e871700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2668111820 con 0x7f2668101d50 2024-06-10T00:53:58.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.456+0000 7f266ce0e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2668107400 0x7f2668198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:58.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.456+0000 7f266ce0e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2668107400 0x7f2668198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57330/0 (socket says 172.21.15.37:57330) 2024-06-10T00:53:58.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.456+0000 7f266ce0e700 1 -- 172.21.15.37:0/995399190 learned_addr learned my addr 172.21.15.37:0/995399190 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:53:58.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.456+0000 7f26677fe700 1 --2- 172.21.15.37:0/995399190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2668101d50 0x7f266819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:58.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.456+0000 7f2667fff700 1 --2- 172.21.15.37:0/995399190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2668101390 0x7f266819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:58.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.456+0000 7f266ce0e700 1 -- 172.21.15.37:0/995399190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2668101d50 msgr2=0x7f266819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:58.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.456+0000 7f266ce0e700 1 --2- 172.21.15.37:0/995399190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2668101d50 0x7f266819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:58.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.456+0000 7f266ce0e700 1 -- 172.21.15.37:0/995399190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2668101390 msgr2=0x7f266819e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:58.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.456+0000 7f266ce0e700 1 --2- 172.21.15.37:0/995399190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2668101390 0x7f266819e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:58.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.456+0000 7f266ce0e700 1 -- 172.21.15.37:0/995399190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2668199160 con 0x7f2668107400 2024-06-10T00:53:58.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.456+0000 7f26677fe700 1 --2- 172.21.15.37:0/995399190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2668101d50 0x7f266819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:53:58.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.456+0000 7f266ce0e700 1 --2- 172.21.15.37:0/995399190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2668107400 0x7f2668198a00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f266000ef30 tx=0x7f266000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:58.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.457+0000 7f26657fa700 1 -- 172.21.15.37:0/995399190 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26600092e0 con 0x7f2668107400 2024-06-10T00:53:58.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.457+0000 7f26657fa700 1 -- 172.21.15.37:0/995399190 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2660004bb0 con 0x7f2668107400 2024-06-10T00:53:58.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.457+0000 7f26657fa700 1 -- 172.21.15.37:0/995399190 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2660004280 con 0x7f2668107400 2024-06-10T00:53:58.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.457+0000 7f266e871700 1 -- 172.21.15.37:0/995399190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26681993f0 con 0x7f2668107400 2024-06-10T00:53:58.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.457+0000 7f266e871700 1 -- 172.21.15.37:0/995399190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26681a5710 con 0x7f2668107400 2024-06-10T00:53:58.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.458+0000 7f266e871700 1 -- 172.21.15.37:0/995399190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2668069180 con 0x7f2668107400 2024-06-10T00:53:58.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.460+0000 7f26657fa700 1 -- 172.21.15.37:0/995399190 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f266001b030 con 0x7f2668107400 2024-06-10T00:53:58.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.461+0000 7f26657fa700 1 --2- 172.21.15.37:0/995399190 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f26500a2cb0 0x7f26500a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:53:58.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.461+0000 7f2667fff700 1 --2- 172.21.15.37:0/995399190 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f26500a2cb0 0x7f26500a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:53:58.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.461+0000 7f26657fa700 1 -- 172.21.15.37:0/995399190 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f26600cfa30 con 0x7f2668107400 2024-06-10T00:53:58.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.462+0000 7f2667fff700 1 --2- 172.21.15.37:0/995399190 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f26500a2cb0 0x7f26500a5170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2668199b40 tx=0x7f2658009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:53:58.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.464+0000 7f26657fa700 1 -- 172.21.15.37:0/995399190 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2660022020 con 0x7f2668107400 2024-06-10T00:53:58.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.706+0000 7f266e871700 1 -- 172.21.15.37:0/995399190 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f26681051b0 con 0x7f26500a2cb0 2024-06-10T00:53:58.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.712+0000 7f26657fa700 1 -- 172.21.15.37:0/995399190 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f26681051b0 con 0x7f26500a2cb0 2024-06-10T00:53:58.712 INFO:teuthology.orchestra.run.smithi037.stdout:/volumes/qa/sv_1/f3e8e6e7-e4b5-44c3-b060-dff807fa8df4 2024-06-10T00:53:58.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.714+0000 7f266e871700 1 -- 172.21.15.37:0/995399190 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f26500a2cb0 msgr2=0x7f26500a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:58.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.714+0000 7f266e871700 1 --2- 172.21.15.37:0/995399190 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f26500a2cb0 0x7f26500a5170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2668199b40 tx=0x7f2658009260 comp rx=0 tx=0).stop 2024-06-10T00:53:58.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.714+0000 7f266e871700 1 -- 172.21.15.37:0/995399190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2668107400 msgr2=0x7f2668198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:53:58.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.714+0000 7f266e871700 1 --2- 172.21.15.37:0/995399190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2668107400 0x7f2668198a00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f266000ef30 tx=0x7f266000c3a0 comp rx=0 tx=0).stop 2024-06-10T00:53:58.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.714+0000 7f266e871700 1 -- 172.21.15.37:0/995399190 shutdown_connections 2024-06-10T00:53:58.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.714+0000 7f266e871700 1 --2- 172.21.15.37:0/995399190 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f26500a2cb0 0x7f26500a5170 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:58.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.714+0000 7f266e871700 1 --2- 172.21.15.37:0/995399190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2668107400 0x7f2668198a00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:58.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.714+0000 7f266e871700 1 --2- 172.21.15.37:0/995399190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2668101390 0x7f266819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:58.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.714+0000 7f266e871700 1 --2- 172.21.15.37:0/995399190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2668101d50 0x7f266819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:53:58.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.714+0000 7f266e871700 1 -- 172.21.15.37:0/995399190 >> 172.21.15.37:0/995399190 conn(0x7f2668078340 msgr2=0x7f2668100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:53:58.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.714+0000 7f266e871700 1 -- 172.21.15.37:0/995399190 shutdown_connections 2024-06-10T00:53:58.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:53:58.714+0000 7f266e871700 1 -- 172.21.15.37:0/995399190 wait complete. 2024-06-10T00:53:59.424 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:53:59.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:59 smithi037 ceph-mon[32192]: pgmap v347: 129 pgs: 129 active+clean; 627 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-06-10T00:53:59.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:53:59 smithi037 ceph-mon[32192]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:53:59.677 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:53:59.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:59 smithi167 ceph-mon[36917]: pgmap v347: 129 pgs: 129 active+clean; 627 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-06-10T00:53:59.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:53:59 smithi167 ceph-mon[36917]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:53:59.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:59 smithi094 ceph-mon[36708]: pgmap v347: 129 pgs: 129 active+clean; 627 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-06-10T00:53:59.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:53:59 smithi094 ceph-mon[36708]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:54:01.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.576+0000 7f1abc8a2700 1 -- 172.21.15.37:0/3019839998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ab8107400 msgr2=0x7f1ab81077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:01.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.576+0000 7f1abc8a2700 1 --2- 172.21.15.37:0/3019839998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ab8107400 0x7f1ab81077e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800b0a0 tx=0x7f1aa800b3b0 comp rx=0 tx=0).stop 2024-06-10T00:54:01.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.577+0000 7f1abc8a2700 1 -- 172.21.15.37:0/3019839998 shutdown_connections 2024-06-10T00:54:01.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.577+0000 7f1abc8a2700 1 --2- 172.21.15.37:0/3019839998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ab8101d50 0x7f1ab810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:01.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.577+0000 7f1abc8a2700 1 --2- 172.21.15.37:0/3019839998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ab8107400 0x7f1ab81077e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:01.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.577+0000 7f1abc8a2700 1 --2- 172.21.15.37:0/3019839998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ab8101390 0x7f1ab8101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:01.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.577+0000 7f1abc8a2700 1 -- 172.21.15.37:0/3019839998 >> 172.21.15.37:0/3019839998 conn(0x7f1ab8078340 msgr2=0x7f1ab8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:01.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.577+0000 7f1abc8a2700 1 -- 172.21.15.37:0/3019839998 shutdown_connections 2024-06-10T00:54:01.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.577+0000 7f1abc8a2700 1 -- 172.21.15.37:0/3019839998 wait complete. 2024-06-10T00:54:01.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.578+0000 7f1abc8a2700 1 Processor -- start 2024-06-10T00:54:01.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.578+0000 7f1abc8a2700 1 -- start start 2024-06-10T00:54:01.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.578+0000 7f1abc8a2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ab8101390 0x7f1ab8199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:01.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.578+0000 7f1abc8a2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ab8101d50 0x7f1ab819a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:01.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.578+0000 7f1abc8a2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ab8107400 0x7f1ab81a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:01.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.578+0000 7f1abc8a2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab81116a0 con 0x7f1ab8101d50 2024-06-10T00:54:01.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.578+0000 7f1abc8a2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab8111520 con 0x7f1ab8107400 2024-06-10T00:54:01.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.578+0000 7f1abc8a2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab8111820 con 0x7f1ab8101390 2024-06-10T00:54:01.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.578+0000 7f1ab5d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ab8101d50 0x7f1ab819a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:01.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.578+0000 7f1ab5d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ab8101d50 0x7f1ab819a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57340/0 (socket says 172.21.15.37:57340) 2024-06-10T00:54:01.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.578+0000 7f1ab5d9b700 1 -- 172.21.15.37:0/3005237013 learned_addr learned my addr 172.21.15.37:0/3005237013 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:54:01.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.578+0000 7f1ab659c700 1 --2- 172.21.15.37:0/3005237013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ab8101390 0x7f1ab8199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:01.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.578+0000 7f1ab6d9d700 1 --2- 172.21.15.37:0/3005237013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ab8107400 0x7f1ab81a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:01.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.578+0000 7f1ab5d9b700 1 -- 172.21.15.37:0/3005237013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ab8101390 msgr2=0x7f1ab8199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:01.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.578+0000 7f1ab5d9b700 1 --2- 172.21.15.37:0/3005237013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ab8101390 0x7f1ab8199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:01.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.578+0000 7f1ab5d9b700 1 -- 172.21.15.37:0/3005237013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ab8107400 msgr2=0x7f1ab81a4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:01.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.578+0000 7f1ab5d9b700 1 --2- 172.21.15.37:0/3005237013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ab8107400 0x7f1ab81a4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:01.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.578+0000 7f1ab5d9b700 1 -- 172.21.15.37:0/3005237013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab81a50c0 con 0x7f1ab8101d50 2024-06-10T00:54:01.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.579+0000 7f1ab5d9b700 1 --2- 172.21.15.37:0/3005237013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ab8101d50 0x7f1ab819a040 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00c970 tx=0x7f1aac00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:01.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.579+0000 7f1aa77fe700 1 -- 172.21.15.37:0/3005237013 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac007aa0 con 0x7f1ab8101d50 2024-06-10T00:54:01.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.579+0000 7f1aa77fe700 1 -- 172.21.15.37:0/3005237013 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aac004d10 con 0x7f1ab8101d50 2024-06-10T00:54:01.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.579+0000 7f1abc8a2700 1 -- 172.21.15.37:0/3005237013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab81a5290 con 0x7f1ab8101d50 2024-06-10T00:54:01.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.579+0000 7f1aa77fe700 1 -- 172.21.15.37:0/3005237013 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac00f450 con 0x7f1ab8101d50 2024-06-10T00:54:01.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.579+0000 7f1abc8a2700 1 -- 172.21.15.37:0/3005237013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab81a57a0 con 0x7f1ab8101d50 2024-06-10T00:54:01.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.580+0000 7f1abc8a2700 1 -- 172.21.15.37:0/3005237013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab8069180 con 0x7f1ab8101d50 2024-06-10T00:54:01.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.581+0000 7f1aa77fe700 1 -- 172.21.15.37:0/3005237013 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1aac004750 con 0x7f1ab8101d50 2024-06-10T00:54:01.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.582+0000 7f1aa77fe700 1 --2- 172.21.15.37:0/3005237013 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1aa00a2d30 0x7f1aa00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:01.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.582+0000 7f1ab659c700 1 --2- 172.21.15.37:0/3005237013 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1aa00a2d30 0x7f1aa00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:01.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.582+0000 7f1aa77fe700 1 -- 172.21.15.37:0/3005237013 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f1aac0cf2f0 con 0x7f1ab8101d50 2024-06-10T00:54:01.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.582+0000 7f1ab659c700 1 --2- 172.21.15.37:0/3005237013 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1aa00a2d30 0x7f1aa00a51f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800b070 tx=0x7f1aa800b7b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:01.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.586+0000 7f1aa77fe700 1 -- 172.21.15.37:0/3005237013 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1aac09bdc0 con 0x7f1ab8101d50 2024-06-10T00:54:01.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:01 smithi037 ceph-mon[32192]: pgmap v348: 129 pgs: 129 active+clean; 627 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.6 KiB/s wr, 7 op/s 2024-06-10T00:54:01.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.837+0000 7f1abc8a2700 1 -- 172.21.15.37:0/3005237013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ab802dca0 con 0x7f1ab8101d50 2024-06-10T00:54:01.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.838+0000 7f1aa77fe700 1 -- 172.21.15.37:0/3005237013 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11867 (secure 0 0 0) 0x7f1aac005700 con 0x7f1ab8101d50 2024-06-10T00:54:01.839 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:54:01.839 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T00:53:44.125049+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24458,24472,24451]},"id":1}]} 2024-06-10T00:54:01.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:01 smithi167 ceph-mon[36917]: pgmap v348: 129 pgs: 129 active+clean; 627 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.6 KiB/s wr, 7 op/s 2024-06-10T00:54:01.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.840+0000 7f1abc8a2700 1 -- 172.21.15.37:0/3005237013 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1aa00a2d30 msgr2=0x7f1aa00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:01.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.840+0000 7f1abc8a2700 1 --2- 172.21.15.37:0/3005237013 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1aa00a2d30 0x7f1aa00a51f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800b070 tx=0x7f1aa800b7b0 comp rx=0 tx=0).stop 2024-06-10T00:54:01.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.840+0000 7f1abc8a2700 1 -- 172.21.15.37:0/3005237013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ab8101d50 msgr2=0x7f1ab819a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:01.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.840+0000 7f1abc8a2700 1 --2- 172.21.15.37:0/3005237013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ab8101d50 0x7f1ab819a040 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00c970 tx=0x7f1aac00cc80 comp rx=0 tx=0).stop 2024-06-10T00:54:01.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.840+0000 7f1abc8a2700 1 -- 172.21.15.37:0/3005237013 shutdown_connections 2024-06-10T00:54:01.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.840+0000 7f1abc8a2700 1 --2- 172.21.15.37:0/3005237013 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1aa00a2d30 0x7f1aa00a51f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:01.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.840+0000 7f1abc8a2700 1 --2- 172.21.15.37:0/3005237013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ab8107400 0x7f1ab81a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:01.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.840+0000 7f1abc8a2700 1 --2- 172.21.15.37:0/3005237013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ab8101390 0x7f1ab8199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:01.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.840+0000 7f1abc8a2700 1 --2- 172.21.15.37:0/3005237013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ab8101d50 0x7f1ab819a040 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:01.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.840+0000 7f1abc8a2700 1 -- 172.21.15.37:0/3005237013 >> 172.21.15.37:0/3005237013 conn(0x7f1ab8078340 msgr2=0x7f1ab8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:01.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.840+0000 7f1abc8a2700 1 -- 172.21.15.37:0/3005237013 shutdown_connections 2024-06-10T00:54:01.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:01.840+0000 7f1abc8a2700 1 -- 172.21.15.37:0/3005237013 wait complete. 2024-06-10T00:54:01.842 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-06-10T00:54:01.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:01 smithi094 ceph-mon[36708]: pgmap v348: 129 pgs: 129 active+clean; 627 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.6 KiB/s wr, 7 op/s 2024-06-10T00:54:02.443 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T00:54:02.693 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:02.740 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:02 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3005237013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:54:02.766 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:02 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3005237013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:54:02.816 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:02 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3005237013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:54:03.815 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:03 smithi037 ceph-mon[32192]: pgmap v349: 129 pgs: 129 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.5 KiB/s wr, 9 op/s 2024-06-10T00:54:03.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:03 smithi167 ceph-mon[36917]: pgmap v349: 129 pgs: 129 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.5 KiB/s wr, 9 op/s 2024-06-10T00:54:03.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:03 smithi094 ceph-mon[36708]: pgmap v349: 129 pgs: 129 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.5 KiB/s wr, 9 op/s 2024-06-10T00:54:04.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.383+0000 7f37bad9f700 1 -- 172.21.15.37:0/3922181260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37b4102f10 msgr2=0x7f37b410f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:04.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.383+0000 7f37bad9f700 1 --2- 172.21.15.37:0/3922181260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37b4102f10 0x7f37b410f2f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00b0d0 tx=0x7f37ac00b3e0 comp rx=0 tx=0).stop 2024-06-10T00:54:04.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.383+0000 7f37bad9f700 1 -- 172.21.15.37:0/3922181260 shutdown_connections 2024-06-10T00:54:04.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.383+0000 7f37bad9f700 1 --2- 172.21.15.37:0/3922181260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37b4102f10 0x7f37b410f2f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:04.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.383+0000 7f37bad9f700 1 --2- 172.21.15.37:0/3922181260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37b41073a0 0x7f37b4107780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:04.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.383+0000 7f37bad9f700 1 --2- 172.21.15.37:0/3922181260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37b4102550 0x7f37b41029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:04.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.383+0000 7f37bad9f700 1 -- 172.21.15.37:0/3922181260 >> 172.21.15.37:0/3922181260 conn(0x7f37b4079650 msgr2=0x7f37b4079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:04.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.383+0000 7f37bad9f700 1 -- 172.21.15.37:0/3922181260 shutdown_connections 2024-06-10T00:54:04.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.383+0000 7f37bad9f700 1 -- 172.21.15.37:0/3922181260 wait complete. 2024-06-10T00:54:04.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.383+0000 7f37bad9f700 1 Processor -- start 2024-06-10T00:54:04.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.384+0000 7f37bad9f700 1 -- start start 2024-06-10T00:54:04.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.384+0000 7f37bad9f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37b4102550 0x7f37b419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:04.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.384+0000 7f37bad9f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37b4102f10 0x7f37b419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:04.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.384+0000 7f37bad9f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37b41073a0 0x7f37b4198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:04.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.384+0000 7f37bad9f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f37b4112780 con 0x7f37b4102550 2024-06-10T00:54:04.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.384+0000 7f37bad9f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f37b4112600 con 0x7f37b4102f10 2024-06-10T00:54:04.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.384+0000 7f37bad9f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f37b4112900 con 0x7f37b41073a0 2024-06-10T00:54:04.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.384+0000 7f37b8b3b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37b4102550 0x7f37b419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:04.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.384+0000 7f37b8b3b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37b4102550 0x7f37b419e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56096/0 (socket says 172.21.15.37:56096) 2024-06-10T00:54:04.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.384+0000 7f37b8b3b700 1 -- 172.21.15.37:0/2868053655 learned_addr learned my addr 172.21.15.37:0/2868053655 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:54:04.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.384+0000 7f37b3fff700 1 --2- 172.21.15.37:0/2868053655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37b4102f10 0x7f37b419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:04.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.384+0000 7f37b933c700 1 --2- 172.21.15.37:0/2868053655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37b41073a0 0x7f37b4198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:04.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.384+0000 7f37b8b3b700 1 -- 172.21.15.37:0/2868053655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37b41073a0 msgr2=0x7f37b4198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:04.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.384+0000 7f37b8b3b700 1 --2- 172.21.15.37:0/2868053655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37b41073a0 0x7f37b4198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:04.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.384+0000 7f37b8b3b700 1 -- 172.21.15.37:0/2868053655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37b4102f10 msgr2=0x7f37b419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:04.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.384+0000 7f37b8b3b700 1 --2- 172.21.15.37:0/2868053655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37b4102f10 0x7f37b419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:04.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.384+0000 7f37b8b3b700 1 -- 172.21.15.37:0/2868053655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b4199160 con 0x7f37b4102550 2024-06-10T00:54:04.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.384+0000 7f37b8b3b700 1 --2- 172.21.15.37:0/2868053655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37b4102550 0x7f37b419e7f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f37a800d950 tx=0x7f37a800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:04.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.385+0000 7f37b1ffb700 1 -- 172.21.15.37:0/2868053655 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a8016070 con 0x7f37b4102550 2024-06-10T00:54:04.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.385+0000 7f37bad9f700 1 -- 172.21.15.37:0/2868053655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37b4199450 con 0x7f37b4102550 2024-06-10T00:54:04.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.385+0000 7f37b1ffb700 1 -- 172.21.15.37:0/2868053655 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a800fd70 con 0x7f37b4102550 2024-06-10T00:54:04.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.385+0000 7f37b1ffb700 1 -- 172.21.15.37:0/2868053655 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a800e610 con 0x7f37b4102550 2024-06-10T00:54:04.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.385+0000 7f37bad9f700 1 -- 172.21.15.37:0/2868053655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37b41a56c0 con 0x7f37b4102550 2024-06-10T00:54:04.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.386+0000 7f37bad9f700 1 -- 172.21.15.37:0/2868053655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37b4103b80 con 0x7f37b4102550 2024-06-10T00:54:04.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.387+0000 7f37b1ffb700 1 -- 172.21.15.37:0/2868053655 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f37a8019070 con 0x7f37b4102550 2024-06-10T00:54:04.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.388+0000 7f37b1ffb700 1 --2- 172.21.15.37:0/2868053655 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f37a40a2cb0 0x7f37a40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:04.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.388+0000 7f37b1ffb700 1 -- 172.21.15.37:0/2868053655 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f37a80cef40 con 0x7f37b4102550 2024-06-10T00:54:04.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.388+0000 7f37b3fff700 1 --2- 172.21.15.37:0/2868053655 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f37a40a2cb0 0x7f37a40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:04.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.388+0000 7f37b3fff700 1 --2- 172.21.15.37:0/2868053655 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f37a40a2cb0 0x7f37a40a5170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f37b4079c30 tx=0x7f37a000a300 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:04.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.392+0000 7f37b1ffb700 1 -- 172.21.15.37:0/2868053655 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f37a809b9e0 con 0x7f37b4102550 2024-06-10T00:54:04.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.599+0000 7f37bad9f700 1 -- 172.21.15.37:0/2868053655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37b402dca0 con 0x7f37b4102550 2024-06-10T00:54:04.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.600+0000 7f37b1ffb700 1 -- 172.21.15.37:0/2868053655 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24562 (secure 0 0 0) 0x7f37a809b2e0 con 0x7f37b4102550 2024-06-10T00:54:04.601 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:54:04.601 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":98,"fsid":"68793514-26c2-11ef-bc9b-c7b262605968","created":"2024-06-10T00:44:24.106713+0000","modified":"2024-06-10T00:53:22.116215+0000","last_up_change":"2024-06-10T00:51:31.562156+0000","last_in_change":"2024-06-10T00:51:11.294301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:47:44.773911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:52:31.684780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:52:34.885503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:52:44.415437+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1df6046a-6801-4950-9193-54303fed169b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6803","nonce":3594704094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6805","nonce":3594704094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6809","nonce":3594704094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6807","nonce":3594704094}]},"public_addr":"172.21.15.37:6803/3594704094","cluster_addr":"172.21.15.37:6805/3594704094","heartbeat_back_addr":"172.21.15.37:6809/3594704094","heartbeat_front_addr":"172.21.15.37:6807/3594704094","state":["exists","up"]},{"osd":1,"uuid":"4e1c8944-94e4-4e46-9efe-56229fe6df1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6801","nonce":3427727155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6803","nonce":3427727155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6807","nonce":3427727155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6805","nonce":3427727155}]},"public_addr":"172.21.15.94:6801/3427727155","cluster_addr":"172.21.15.94:6803/3427727155","heartbeat_back_addr":"172.21.15.94:6807/3427727155","heartbeat_front_addr":"172.21.15.94:6805/3427727155","state":["exists","up"]},{"osd":2,"uuid":"fd2fcedc-4c88-478f-8025-13e5944a27ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6801","nonce":4002498292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6803","nonce":4002498292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6807","nonce":4002498292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6805","nonce":4002498292}]},"public_addr":"172.21.15.167:6801/4002498292","cluster_addr":"172.21.15.167:6803/4002498292","heartbeat_back_addr":"172.21.15.167:6807/4002498292","heartbeat_front_addr":"172.21.15.167:6805/4002498292","state":["exists","up"]},{"osd":3,"uuid":"d0153676-818c-4dcc-83b1-287435a0c8a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6811","nonce":3425469324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6813","nonce":3425469324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6817","nonce":3425469324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6815","nonce":3425469324}]},"public_addr":"172.21.15.37:6811/3425469324","cluster_addr":"172.21.15.37:6813/3425469324","heartbeat_back_addr":"172.21.15.37:6817/3425469324","heartbeat_front_addr":"172.21.15.37:6815/3425469324","state":["exists","up"]},{"osd":4,"uuid":"172c333c-00c0-4e94-8870-acb3b32b2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6809","nonce":1740627431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6811","nonce":1740627431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6815","nonce":1740627431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6813","nonce":1740627431}]},"public_addr":"172.21.15.94:6809/1740627431","cluster_addr":"172.21.15.94:6811/1740627431","heartbeat_back_addr":"172.21.15.94:6815/1740627431","heartbeat_front_addr":"172.21.15.94:6813/1740627431","state":["exists","up"]},{"osd":5,"uuid":"dbce656e-5f93-4bda-baf2-fc3cc03d8282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6809","nonce":221805236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6811","nonce":221805236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6815","nonce":221805236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6813","nonce":221805236}]},"public_addr":"172.21.15.167:6809/221805236","cluster_addr":"172.21.15.167:6811/221805236","heartbeat_back_addr":"172.21.15.167:6815/221805236","heartbeat_front_addr":"172.21.15.167:6813/221805236","state":["exists","up"]},{"osd":6,"uuid":"fcbda266-6d60-43b2-a51c-4e562476d1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6819","nonce":472670875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6821","nonce":472670875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6825","nonce":472670875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6823","nonce":472670875}]},"public_addr":"172.21.15.37:6819/472670875","cluster_addr":"172.21.15.37:6821/472670875","heartbeat_back_addr":"172.21.15.37:6825/472670875","heartbeat_front_addr":"172.21.15.37:6823/472670875","state":["exists","up"]},{"osd":7,"uuid":"ed829ed3-ba85-4e82-ac37-e44e9534d37c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6817","nonce":3932281552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6819","nonce":3932281552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6823","nonce":3932281552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6821","nonce":3932281552}]},"public_addr":"172.21.15.94:6817/3932281552","cluster_addr":"172.21.15.94:6819/3932281552","heartbeat_back_addr":"172.21.15.94:6823/3932281552","heartbeat_front_addr":"172.21.15.94:6821/3932281552","state":["exists","up"]},{"osd":8,"uuid":"7d6267d8-72f4-4704-b554-bafc52d11a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6817","nonce":2290216047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6819","nonce":2290216047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6823","nonce":2290216047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6821","nonce":2290216047}]},"public_addr":"172.21.15.167:6817/2290216047","cluster_addr":"172.21.15.167:6819/2290216047","heartbeat_back_addr":"172.21.15.167:6823/2290216047","heartbeat_front_addr":"172.21.15.167:6821/2290216047","state":["exists","up"]},{"osd":9,"uuid":"210fdc20-0a36-4eeb-bdf2-c0a7f130e19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6827","nonce":2063666428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6829","nonce":2063666428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6833","nonce":2063666428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6831","nonce":2063666428}]},"public_addr":"172.21.15.37:6827/2063666428","cluster_addr":"172.21.15.37:6829/2063666428","heartbeat_back_addr":"172.21.15.37:6833/2063666428","heartbeat_front_addr":"172.21.15.37:6831/2063666428","state":["exists","up"]},{"osd":10,"uuid":"e200b371-b3cf-4b92-a659-f155ca86f0a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6825","nonce":1706618899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6827","nonce":1706618899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6831","nonce":1706618899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6829","nonce":1706618899}]},"public_addr":"172.21.15.94:6825/1706618899","cluster_addr":"172.21.15.94:6827/1706618899","heartbeat_back_addr":"172.21.15.94:6831/1706618899","heartbeat_front_addr":"172.21.15.94:6829/1706618899","state":["exists","up"]},{"osd":11,"uuid":"cfa427df-37a5-4d28-ac2e-af777d607008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6825","nonce":1349892516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6827","nonce":1349892516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6831","nonce":1349892516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6829","nonce":1349892516}]},"public_addr":"172.21.15.167:6825/1349892516","cluster_addr":"172.21.15.167:6827/1349892516","heartbeat_back_addr":"172.21.15.167:6831/1349892516","heartbeat_front_addr":"172.21.15.167:6829/1349892516","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:46:51.846147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:15.763723+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:40.106800+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:05.923204+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:30.501883+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:55.565575+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:20.411360+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:46.071776+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:11.854599+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:37.608818+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:03.176767+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:29.768705+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/4084722397":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/3571803585":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/416587443":"2024-06-11T00:45:20.254342+0000","172.21.15.37:6801/2766692263":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/2026344134":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/2781321886":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6801/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/1110143997":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2766692263":"2024-06-11T00:45:20.254342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:54:04.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.602+0000 7f37bad9f700 1 -- 172.21.15.37:0/2868053655 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f37a40a2cb0 msgr2=0x7f37a40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:04.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.602+0000 7f37bad9f700 1 --2- 172.21.15.37:0/2868053655 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f37a40a2cb0 0x7f37a40a5170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f37b4079c30 tx=0x7f37a000a300 comp rx=0 tx=0).stop 2024-06-10T00:54:04.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.602+0000 7f37bad9f700 1 -- 172.21.15.37:0/2868053655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37b4102550 msgr2=0x7f37b419e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:04.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.602+0000 7f37bad9f700 1 --2- 172.21.15.37:0/2868053655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37b4102550 0x7f37b419e7f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f37a800d950 tx=0x7f37a800dc60 comp rx=0 tx=0).stop 2024-06-10T00:54:04.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.603+0000 7f37bad9f700 1 -- 172.21.15.37:0/2868053655 shutdown_connections 2024-06-10T00:54:04.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.603+0000 7f37bad9f700 1 --2- 172.21.15.37:0/2868053655 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f37a40a2cb0 0x7f37a40a5170 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:04.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.603+0000 7f37bad9f700 1 --2- 172.21.15.37:0/2868053655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37b41073a0 0x7f37b4198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:04.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.603+0000 7f37bad9f700 1 --2- 172.21.15.37:0/2868053655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37b4102550 0x7f37b419e7f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:04.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.603+0000 7f37bad9f700 1 --2- 172.21.15.37:0/2868053655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37b4102f10 0x7f37b419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:04.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.603+0000 7f37bad9f700 1 -- 172.21.15.37:0/2868053655 >> 172.21.15.37:0/2868053655 conn(0x7f37b4079650 msgr2=0x7f37b41005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:04.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.603+0000 7f37bad9f700 1 -- 172.21.15.37:0/2868053655 shutdown_connections 2024-06-10T00:54:04.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:04.603+0000 7f37bad9f700 1 -- 172.21.15.37:0/2868053655 wait complete. 2024-06-10T00:54:05.219 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-10T00:54:05.219 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-10T00:54:05.473 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:05.722 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:05 smithi037 ceph-mon[32192]: pgmap v350: 129 pgs: 129 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-06-10T00:54:05.722 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:05 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2868053655' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:54:05.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:05 smithi167 ceph-mon[36917]: pgmap v350: 129 pgs: 129 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-06-10T00:54:05.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:05 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2868053655' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:54:05.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:05 smithi094 ceph-mon[36708]: pgmap v350: 129 pgs: 129 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-06-10T00:54:05.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:05 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2868053655' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:54:07.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.268+0000 7fc7a51a1700 1 -- 172.21.15.37:0/3338868425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7a0102fb0 msgr2=0x7fc7a0107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:07.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.268+0000 7fc7a51a1700 1 --2- 172.21.15.37:0/3338868425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7a0102fb0 0x7fc7a0107000 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc7940099c0 tx=0x7fc794009cd0 comp rx=0 tx=0).stop 2024-06-10T00:54:07.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.268+0000 7fc7a51a1700 1 -- 172.21.15.37:0/3338868425 shutdown_connections 2024-06-10T00:54:07.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.268+0000 7fc7a51a1700 1 --2- 172.21.15.37:0/3338868425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7a01076e0 0x7fc7a010f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:07.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.268+0000 7fc7a51a1700 1 --2- 172.21.15.37:0/3338868425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7a0102600 0x7fc7a01029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:07.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.268+0000 7fc7a51a1700 1 --2- 172.21.15.37:0/3338868425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7a0102fb0 0x7fc7a0107000 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:07.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.268+0000 7fc7a51a1700 1 -- 172.21.15.37:0/3338868425 >> 172.21.15.37:0/3338868425 conn(0x7fc7a00fdeb0 msgr2=0x7fc7a01002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:07.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.268+0000 7fc7a51a1700 1 -- 172.21.15.37:0/3338868425 shutdown_connections 2024-06-10T00:54:07.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.268+0000 7fc7a51a1700 1 -- 172.21.15.37:0/3338868425 wait complete. 2024-06-10T00:54:07.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.269+0000 7fc7a51a1700 1 Processor -- start 2024-06-10T00:54:07.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.269+0000 7fc7a51a1700 1 -- start start 2024-06-10T00:54:07.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.269+0000 7fc7a51a1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7a0102600 0x7fc7a019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:07.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.269+0000 7fc7a51a1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7a0102fb0 0x7fc7a019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:07.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.269+0000 7fc7a51a1700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7a01076e0 0x7fc7a01989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:07.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.269+0000 7fc7a51a1700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a0112720 con 0x7fc7a0102600 2024-06-10T00:54:07.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.269+0000 7fc7a51a1700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a01125a0 con 0x7fc7a0102fb0 2024-06-10T00:54:07.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.269+0000 7fc7a51a1700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a01128a0 con 0x7fc7a01076e0 2024-06-10T00:54:07.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.269+0000 7fc79ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7a0102600 0x7fc7a019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:07.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.269+0000 7fc79ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7a0102600 0x7fc7a019e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56118/0 (socket says 172.21.15.37:56118) 2024-06-10T00:54:07.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.269+0000 7fc79ffff700 1 -- 172.21.15.37:0/2182974779 learned_addr learned my addr 172.21.15.37:0/2182974779 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:54:07.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.269+0000 7fc79f7fe700 1 --2- 172.21.15.37:0/2182974779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7a0102fb0 0x7fc7a019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:07.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.270+0000 7fc7a49a0700 1 --2- 172.21.15.37:0/2182974779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7a01076e0 0x7fc7a01989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:07.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.270+0000 7fc79ffff700 1 -- 172.21.15.37:0/2182974779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7a01076e0 msgr2=0x7fc7a01989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:07.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.270+0000 7fc79ffff700 1 --2- 172.21.15.37:0/2182974779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7a01076e0 0x7fc7a01989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:07.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.270+0000 7fc79ffff700 1 -- 172.21.15.37:0/2182974779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7a0102fb0 msgr2=0x7fc7a019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:07.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.270+0000 7fc79ffff700 1 --2- 172.21.15.37:0/2182974779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7a0102fb0 0x7fc7a019ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:07.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.270+0000 7fc79ffff700 1 -- 172.21.15.37:0/2182974779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a0199140 con 0x7fc7a0102600 2024-06-10T00:54:07.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.270+0000 7fc7a49a0700 1 --2- 172.21.15.37:0/2182974779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7a01076e0 0x7fc7a01989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:54:07.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.270+0000 7fc79f7fe700 1 --2- 172.21.15.37:0/2182974779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7a0102fb0 0x7fc7a019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:54:07.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.270+0000 7fc79ffff700 1 --2- 172.21.15.37:0/2182974779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7a0102600 0x7fc7a019e820 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc79000d950 tx=0x7fc79000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:07.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.270+0000 7fc79d7fa700 1 -- 172.21.15.37:0/2182974779 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc790016070 con 0x7fc7a0102600 2024-06-10T00:54:07.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.270+0000 7fc7a51a1700 1 -- 172.21.15.37:0/2182974779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a0199430 con 0x7fc7a0102600 2024-06-10T00:54:07.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.270+0000 7fc79d7fa700 1 -- 172.21.15.37:0/2182974779 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc790009b40 con 0x7fc7a0102600 2024-06-10T00:54:07.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.270+0000 7fc79d7fa700 1 -- 172.21.15.37:0/2182974779 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79000f620 con 0x7fc7a0102600 2024-06-10T00:54:07.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.270+0000 7fc7a51a1700 1 -- 172.21.15.37:0/2182974779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a01a56f0 con 0x7fc7a0102600 2024-06-10T00:54:07.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.271+0000 7fc7a51a1700 1 -- 172.21.15.37:0/2182974779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7a0069180 con 0x7fc7a0102600 2024-06-10T00:54:07.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.272+0000 7fc79d7fa700 1 -- 172.21.15.37:0/2182974779 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc790019070 con 0x7fc7a0102600 2024-06-10T00:54:07.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.273+0000 7fc79d7fa700 1 --2- 172.21.15.37:0/2182974779 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc7880a2cb0 0x7fc7880a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:07.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.274+0000 7fc79f7fe700 1 --2- 172.21.15.37:0/2182974779 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc7880a2cb0 0x7fc7880a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:07.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.274+0000 7fc79d7fa700 1 -- 172.21.15.37:0/2182974779 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fc7900cf570 con 0x7fc7a0102600 2024-06-10T00:54:07.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.274+0000 7fc79f7fe700 1 --2- 172.21.15.37:0/2182974779 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc7880a2cb0 0x7fc7880a5170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc794000c00 tx=0x7fc794005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:07.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.278+0000 7fc79d7fa700 1 -- 172.21.15.37:0/2182974779 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc79009c010 con 0x7fc7a0102600 2024-06-10T00:54:07.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.479+0000 7fc7a51a1700 1 -- 172.21.15.37:0/2182974779 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc7a0199f10 con 0x7fc7880a2cb0 2024-06-10T00:54:07.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.489+0000 7fc79d7fa700 1 -- 172.21.15.37:0/2182974779 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340633 (secure 0 0 0) 0x7fc7a0199f10 con 0x7fc7880a2cb0 2024-06-10T00:54:07.495 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:54:07.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.492+0000 7fc7a51a1700 1 -- 172.21.15.37:0/2182974779 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc7880a2cb0 msgr2=0x7fc7880a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:07.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.492+0000 7fc7a51a1700 1 --2- 172.21.15.37:0/2182974779 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc7880a2cb0 0x7fc7880a5170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc794000c00 tx=0x7fc794005fd0 comp rx=0 tx=0).stop 2024-06-10T00:54:07.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.492+0000 7fc7a51a1700 1 -- 172.21.15.37:0/2182974779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7a0102600 msgr2=0x7fc7a019e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:07.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.492+0000 7fc7a51a1700 1 --2- 172.21.15.37:0/2182974779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7a0102600 0x7fc7a019e820 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc79000d950 tx=0x7fc79000dd10 comp rx=0 tx=0).stop 2024-06-10T00:54:07.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.493+0000 7fc7a51a1700 1 -- 172.21.15.37:0/2182974779 shutdown_connections 2024-06-10T00:54:07.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.493+0000 7fc7a51a1700 1 --2- 172.21.15.37:0/2182974779 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc7880a2cb0 0x7fc7880a5170 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:07.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.493+0000 7fc7a51a1700 1 --2- 172.21.15.37:0/2182974779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7a01076e0 0x7fc7a01989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:07.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.493+0000 7fc7a51a1700 1 --2- 172.21.15.37:0/2182974779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7a0102600 0x7fc7a019e820 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:07.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.493+0000 7fc7a51a1700 1 --2- 172.21.15.37:0/2182974779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7a0102fb0 0x7fc7a019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:07.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.493+0000 7fc7a51a1700 1 -- 172.21.15.37:0/2182974779 >> 172.21.15.37:0/2182974779 conn(0x7fc7a00fdeb0 msgr2=0x7fc7a0100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:07.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.493+0000 7fc7a51a1700 1 -- 172.21.15.37:0/2182974779 shutdown_connections 2024-06-10T00:54:07.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:07.493+0000 7fc7a51a1700 1 -- 172.21.15.37:0/2182974779 wait complete. 2024-06-10T00:54:07.499 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-06-10T00:54:07.782 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:07 smithi037 ceph-mon[32192]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.9 KiB/s wr, 8 op/s 2024-06-10T00:54:07.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:07 smithi167 ceph-mon[36917]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.9 KiB/s wr, 8 op/s 2024-06-10T00:54:07.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:07 smithi094 ceph-mon[36708]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.9 KiB/s wr, 8 op/s 2024-06-10T00:54:08.239 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2024-06-10T00:54:06.366772+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659478,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":293,"num_read_kb":269,"num_write":350,"num_write_kb":3060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":282,"ondisk_log_size":282,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370580,"kb_used_data":13108,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571244,"statfs":{"total":1151940427776,"available":1151560953856,"internally_reserved":0,"allocated":13422592,"data_stored":8586652,"data_compressed":426333,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":35788,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":79,"num_read_kb":78,"num_write":27,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003303"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.123883+0000","last_change":"2024-06-10T00:52:35.971067+0000","last_active":"2024-06-10T00:53:22.123883+0000","last_peered":"2024-06-10T00:53:22.123883+0000","last_clean":"2024-06-10T00:53:22.123883+0000","last_became_active":"2024-06-10T00:52:35.970292+0000","last_became_peered":"2024-06-10T00:52:35.970292+0000","last_unstale":"2024-06-10T00:53:22.123883+0000","last_undegraded":"2024-06-10T00:53:22.123883+0000","last_fullsized":"2024-06-10T00:53:22.123883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:57:37.287238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.125927+0000","last_change":"2024-06-10T00:52:35.960421+0000","last_active":"2024-06-10T00:53:22.125927+0000","last_peered":"2024-06-10T00:53:22.125927+0000","last_clean":"2024-06-10T00:53:22.125927+0000","last_became_active":"2024-06-10T00:52:35.960070+0000","last_became_peered":"2024-06-10T00:52:35.960070+0000","last_unstale":"2024-06-10T00:53:22.125927+0000","last_undegraded":"2024-06-10T00:53:22.125927+0000","last_fullsized":"2024-06-10T00:53:22.125927+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:48:02.616434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.125113+0000","last_change":"2024-06-10T00:52:35.964778+0000","last_active":"2024-06-10T00:53:22.125113+0000","last_peered":"2024-06-10T00:53:22.125113+0000","last_clean":"2024-06-10T00:53:22.125113+0000","last_became_active":"2024-06-10T00:52:35.964463+0000","last_became_peered":"2024-06-10T00:52:35.964463+0000","last_unstale":"2024-06-10T00:53:22.125113+0000","last_undegraded":"2024-06-10T00:53:22.125113+0000","last_fullsized":"2024-06-10T00:53:22.125113+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:52:44.987363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.200376+0000","last_change":"2024-06-10T00:52:35.963226+0000","last_active":"2024-06-10T00:53:22.200376+0000","last_peered":"2024-06-10T00:53:22.200376+0000","last_clean":"2024-06-10T00:53:22.200376+0000","last_became_active":"2024-06-10T00:52:35.962862+0000","last_became_peered":"2024-06-10T00:52:35.962862+0000","last_unstale":"2024-06-10T00:53:22.200376+0000","last_undegraded":"2024-06-10T00:53:22.200376+0000","last_fullsized":"2024-06-10T00:53:22.200376+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:08:56.418071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124010+0000","last_change":"2024-06-10T00:52:35.957427+0000","last_active":"2024-06-10T00:53:22.124010+0000","last_peered":"2024-06-10T00:53:22.124010+0000","last_clean":"2024-06-10T00:53:22.124010+0000","last_became_active":"2024-06-10T00:52:35.957049+0000","last_became_peered":"2024-06-10T00:52:35.957049+0000","last_unstale":"2024-06-10T00:53:22.124010+0000","last_undegraded":"2024-06-10T00:53:22.124010+0000","last_fullsized":"2024-06-10T00:53:22.124010+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:16:04.257210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124433+0000","last_change":"2024-06-10T00:52:35.954251+0000","last_active":"2024-06-10T00:53:22.124433+0000","last_peered":"2024-06-10T00:53:22.124433+0000","last_clean":"2024-06-10T00:53:22.124433+0000","last_became_active":"2024-06-10T00:52:35.953908+0000","last_became_peered":"2024-06-10T00:52:35.953908+0000","last_unstale":"2024-06-10T00:53:22.124433+0000","last_undegraded":"2024-06-10T00:53:22.124433+0000","last_fullsized":"2024-06-10T00:53:22.124433+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:55:25.125836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120375+0000","last_change":"2024-06-10T00:52:35.956822+0000","last_active":"2024-06-10T00:53:22.120375+0000","last_peered":"2024-06-10T00:53:22.120375+0000","last_clean":"2024-06-10T00:53:22.120375+0000","last_became_active":"2024-06-10T00:52:35.956466+0000","last_became_peered":"2024-06-10T00:52:35.956466+0000","last_unstale":"2024-06-10T00:53:22.120375+0000","last_undegraded":"2024-06-10T00:53:22.120375+0000","last_fullsized":"2024-06-10T00:53:22.120375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:36:41.534392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120089+0000","last_change":"2024-06-10T00:52:35.962978+0000","last_active":"2024-06-10T00:53:22.120089+0000","last_peered":"2024-06-10T00:53:22.120089+0000","last_clean":"2024-06-10T00:53:22.120089+0000","last_became_active":"2024-06-10T00:52:35.962609+0000","last_became_peered":"2024-06-10T00:52:35.962609+0000","last_unstale":"2024-06-10T00:53:22.120089+0000","last_undegraded":"2024-06-10T00:53:22.120089+0000","last_fullsized":"2024-06-10T00:53:22.120089+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:20:57.469577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.199080+0000","last_change":"2024-06-10T00:52:35.963203+0000","last_active":"2024-06-10T00:53:22.199080+0000","last_peered":"2024-06-10T00:53:22.199080+0000","last_clean":"2024-06-10T00:53:22.199080+0000","last_became_active":"2024-06-10T00:52:35.962789+0000","last_became_peered":"2024-06-10T00:52:35.962789+0000","last_unstale":"2024-06-10T00:53:22.199080+0000","last_undegraded":"2024-06-10T00:53:22.199080+0000","last_fullsized":"2024-06-10T00:53:22.199080+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:00:17.819193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124909+0000","last_change":"2024-06-10T00:52:35.966940+0000","last_active":"2024-06-10T00:53:22.124909+0000","last_peered":"2024-06-10T00:53:22.124909+0000","last_clean":"2024-06-10T00:53:22.124909+0000","last_became_active":"2024-06-10T00:52:35.966554+0000","last_became_peered":"2024-06-10T00:52:35.966554+0000","last_unstale":"2024-06-10T00:53:22.124909+0000","last_undegraded":"2024-06-10T00:53:22.124909+0000","last_fullsized":"2024-06-10T00:53:22.124909+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:00.374437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120905+0000","last_change":"2024-06-10T00:52:35.960927+0000","last_active":"2024-06-10T00:53:22.120905+0000","last_peered":"2024-06-10T00:53:22.120905+0000","last_clean":"2024-06-10T00:53:22.120905+0000","last_became_active":"2024-06-10T00:52:35.960577+0000","last_became_peered":"2024-06-10T00:52:35.960577+0000","last_unstale":"2024-06-10T00:53:22.120905+0000","last_undegraded":"2024-06-10T00:53:22.120905+0000","last_fullsized":"2024-06-10T00:53:22.120905+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:50:02.366695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.200253+0000","last_change":"2024-06-10T00:52:35.964435+0000","last_active":"2024-06-10T00:53:22.200253+0000","last_peered":"2024-06-10T00:53:22.200253+0000","last_clean":"2024-06-10T00:53:22.200253+0000","last_became_active":"2024-06-10T00:52:35.964069+0000","last_became_peered":"2024-06-10T00:52:35.964069+0000","last_unstale":"2024-06-10T00:53:22.200253+0000","last_undegraded":"2024-06-10T00:53:22.200253+0000","last_fullsized":"2024-06-10T00:53:22.200253+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:06.708551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.128778+0000","last_change":"2024-06-10T00:52:35.966722+0000","last_active":"2024-06-10T00:53:22.128778+0000","last_peered":"2024-06-10T00:53:22.128778+0000","last_clean":"2024-06-10T00:53:22.128778+0000","last_became_active":"2024-06-10T00:52:35.966366+0000","last_became_peered":"2024-06-10T00:52:35.966366+0000","last_unstale":"2024-06-10T00:53:22.128778+0000","last_undegraded":"2024-06-10T00:53:22.128778+0000","last_fullsized":"2024-06-10T00:53:22.128778+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:57:46.021047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124780+0000","last_change":"2024-06-10T00:52:35.955307+0000","last_active":"2024-06-10T00:53:22.124780+0000","last_peered":"2024-06-10T00:53:22.124780+0000","last_clean":"2024-06-10T00:53:22.124780+0000","last_became_active":"2024-06-10T00:52:35.954140+0000","last_became_peered":"2024-06-10T00:52:35.954140+0000","last_unstale":"2024-06-10T00:53:22.124780+0000","last_undegraded":"2024-06-10T00:53:22.124780+0000","last_fullsized":"2024-06-10T00:53:22.124780+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:47:22.493598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.125049+0000","last_change":"2024-06-10T00:52:35.960592+0000","last_active":"2024-06-10T00:53:22.125049+0000","last_peered":"2024-06-10T00:53:22.125049+0000","last_clean":"2024-06-10T00:53:22.125049+0000","last_became_active":"2024-06-10T00:52:35.960263+0000","last_became_peered":"2024-06-10T00:52:35.960263+0000","last_unstale":"2024-06-10T00:53:22.125049+0000","last_undegraded":"2024-06-10T00:53:22.125049+0000","last_fullsized":"2024-06-10T00:53:22.125049+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:47:11.658666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.198195+0000","last_change":"2024-06-10T00:52:35.965796+0000","last_active":"2024-06-10T00:53:22.198195+0000","last_peered":"2024-06-10T00:53:22.198195+0000","last_clean":"2024-06-10T00:53:22.198195+0000","last_became_active":"2024-06-10T00:52:35.963965+0000","last_became_peered":"2024-06-10T00:52:35.963965+0000","last_unstale":"2024-06-10T00:53:22.198195+0000","last_undegraded":"2024-06-10T00:53:22.198195+0000","last_fullsized":"2024-06-10T00:53:22.198195+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:29:47.149852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.123532+0000","last_change":"2024-06-10T00:52:35.970714+0000","last_active":"2024-06-10T00:53:22.123532+0000","last_peered":"2024-06-10T00:53:22.123532+0000","last_clean":"2024-06-10T00:53:22.123532+0000","last_became_active":"2024-06-10T00:52:35.970173+0000","last_became_peered":"2024-06-10T00:52:35.970173+0000","last_unstale":"2024-06-10T00:53:22.123532+0000","last_undegraded":"2024-06-10T00:53:22.123532+0000","last_fullsized":"2024-06-10T00:53:22.123532+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:33:28.183156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124679+0000","last_change":"2024-06-10T00:52:35.963052+0000","last_active":"2024-06-10T00:53:22.124679+0000","last_peered":"2024-06-10T00:53:22.124679+0000","last_clean":"2024-06-10T00:53:22.124679+0000","last_became_active":"2024-06-10T00:52:35.962674+0000","last_became_peered":"2024-06-10T00:52:35.962674+0000","last_unstale":"2024-06-10T00:53:22.124679+0000","last_undegraded":"2024-06-10T00:53:22.124679+0000","last_fullsized":"2024-06-10T00:53:22.124679+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:56:38.836854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.130336+0000","last_change":"2024-06-10T00:53:22.130336+0000","last_active":"2024-06-10T00:53:22.130336+0000","last_peered":"2024-06-10T00:53:22.130336+0000","last_clean":"2024-06-10T00:53:22.130336+0000","last_became_active":"2024-06-10T00:53:22.129910+0000","last_became_peered":"2024-06-10T00:53:22.129910+0000","last_unstale":"2024-06-10T00:53:22.130336+0000","last_undegraded":"2024-06-10T00:53:22.130336+0000","last_fullsized":"2024-06-10T00:53:22.130336+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:23:50.600821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120406+0000","last_change":"2024-06-10T00:52:45.997276+0000","last_active":"2024-06-10T00:53:22.120406+0000","last_peered":"2024-06-10T00:53:22.120406+0000","last_clean":"2024-06-10T00:53:22.120406+0000","last_became_active":"2024-06-10T00:52:45.995638+0000","last_became_peered":"2024-06-10T00:52:45.995638+0000","last_unstale":"2024-06-10T00:53:22.120406+0000","last_undegraded":"2024-06-10T00:53:22.120406+0000","last_fullsized":"2024-06-10T00:53:22.120406+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:14:20.922348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:39.289847+0000","last_change":"2024-06-10T00:52:32.760888+0000","last_active":"2024-06-10T00:53:39.289847+0000","last_peered":"2024-06-10T00:53:39.289847+0000","last_clean":"2024-06-10T00:53:39.289847+0000","last_became_active":"2024-06-10T00:52:32.760531+0000","last_became_peered":"2024-06-10T00:52:32.760531+0000","last_unstale":"2024-06-10T00:53:39.289847+0000","last_undegraded":"2024-06-10T00:53:39.289847+0000","last_fullsized":"2024-06-10T00:53:39.289847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:05:59.052251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120410+0000","last_change":"2024-06-10T00:52:35.967626+0000","last_active":"2024-06-10T00:53:22.120410+0000","last_peered":"2024-06-10T00:53:22.120410+0000","last_clean":"2024-06-10T00:53:22.120410+0000","last_became_active":"2024-06-10T00:52:35.967295+0000","last_became_peered":"2024-06-10T00:52:35.967295+0000","last_unstale":"2024-06-10T00:53:22.120410+0000","last_undegraded":"2024-06-10T00:53:22.120410+0000","last_fullsized":"2024-06-10T00:53:22.120410+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:07:55.054327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.199614+0000","last_change":"2024-06-10T00:52:45.993974+0000","last_active":"2024-06-10T00:53:22.199614+0000","last_peered":"2024-06-10T00:53:22.199614+0000","last_clean":"2024-06-10T00:53:22.199614+0000","last_became_active":"2024-06-10T00:52:45.993242+0000","last_became_peered":"2024-06-10T00:52:45.993242+0000","last_unstale":"2024-06-10T00:53:22.199614+0000","last_undegraded":"2024-06-10T00:53:22.199614+0000","last_fullsized":"2024-06-10T00:53:22.199614+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:28:12.810368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120766+0000","last_change":"2024-06-10T00:52:32.959905+0000","last_active":"2024-06-10T00:53:22.120766+0000","last_peered":"2024-06-10T00:53:22.120766+0000","last_clean":"2024-06-10T00:53:22.120766+0000","last_became_active":"2024-06-10T00:52:32.959551+0000","last_became_peered":"2024-06-10T00:52:32.959551+0000","last_unstale":"2024-06-10T00:53:22.120766+0000","last_undegraded":"2024-06-10T00:53:22.120766+0000","last_fullsized":"2024-06-10T00:53:22.120766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:33:48.029129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.125237+0000","last_change":"2024-06-10T00:52:35.965790+0000","last_active":"2024-06-10T00:53:22.125237+0000","last_peered":"2024-06-10T00:53:22.125237+0000","last_clean":"2024-06-10T00:53:22.125237+0000","last_became_active":"2024-06-10T00:52:35.965475+0000","last_became_peered":"2024-06-10T00:52:35.965475+0000","last_unstale":"2024-06-10T00:53:22.125237+0000","last_undegraded":"2024-06-10T00:53:22.125237+0000","last_fullsized":"2024-06-10T00:53:22.125237+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:24:02.894411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.200194+0000","last_change":"2024-06-10T00:52:45.994464+0000","last_active":"2024-06-10T00:53:22.200194+0000","last_peered":"2024-06-10T00:53:22.200194+0000","last_clean":"2024-06-10T00:53:22.200194+0000","last_became_active":"2024-06-10T00:52:45.994022+0000","last_became_peered":"2024-06-10T00:52:45.994022+0000","last_unstale":"2024-06-10T00:53:22.200194+0000","last_undegraded":"2024-06-10T00:53:22.200194+0000","last_fullsized":"2024-06-10T00:53:22.200194+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:27:25.063074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:39.288755+0000","last_change":"2024-06-10T00:52:32.763885+0000","last_active":"2024-06-10T00:53:39.288755+0000","last_peered":"2024-06-10T00:53:39.288755+0000","last_clean":"2024-06-10T00:53:39.288755+0000","last_became_active":"2024-06-10T00:52:32.763606+0000","last_became_peered":"2024-06-10T00:52:32.763606+0000","last_unstale":"2024-06-10T00:53:39.288755+0000","last_undegraded":"2024-06-10T00:53:39.288755+0000","last_fullsized":"2024-06-10T00:53:39.288755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:01:58.161168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.198324+0000","last_change":"2024-06-10T00:52:35.966948+0000","last_active":"2024-06-10T00:53:22.198324+0000","last_peered":"2024-06-10T00:53:22.198324+0000","last_clean":"2024-06-10T00:53:22.198324+0000","last_became_active":"2024-06-10T00:52:35.966345+0000","last_became_peered":"2024-06-10T00:52:35.966345+0000","last_unstale":"2024-06-10T00:53:22.198324+0000","last_undegraded":"2024-06-10T00:53:22.198324+0000","last_fullsized":"2024-06-10T00:53:22.198324+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:58:27.169201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.199805+0000","last_change":"2024-06-10T00:52:45.993645+0000","last_active":"2024-06-10T00:53:22.199805+0000","last_peered":"2024-06-10T00:53:22.199805+0000","last_clean":"2024-06-10T00:53:22.199805+0000","last_became_active":"2024-06-10T00:52:45.993183+0000","last_became_peered":"2024-06-10T00:52:45.993183+0000","last_unstale":"2024-06-10T00:53:22.199805+0000","last_undegraded":"2024-06-10T00:53:22.199805+0000","last_fullsized":"2024-06-10T00:53:22.199805+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:47:00.541427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.122778+0000","last_change":"2024-06-10T00:52:32.761110+0000","last_active":"2024-06-10T00:53:22.122778+0000","last_peered":"2024-06-10T00:53:22.122778+0000","last_clean":"2024-06-10T00:53:22.122778+0000","last_became_active":"2024-06-10T00:52:32.760764+0000","last_became_peered":"2024-06-10T00:52:32.760764+0000","last_unstale":"2024-06-10T00:53:22.122778+0000","last_undegraded":"2024-06-10T00:53:22.122778+0000","last_fullsized":"2024-06-10T00:53:22.122778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:38:17.022460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":39,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.199668+0000","last_change":"2024-06-10T00:53:22.199668+0000","last_active":"2024-06-10T00:53:22.199668+0000","last_peered":"2024-06-10T00:53:22.199668+0000","last_clean":"2024-06-10T00:53:22.199668+0000","last_became_active":"2024-06-10T00:53:22.199329+0000","last_became_peered":"2024-06-10T00:53:22.199329+0000","last_unstale":"2024-06-10T00:53:22.199668+0000","last_undegraded":"2024-06-10T00:53:22.199668+0000","last_fullsized":"2024-06-10T00:53:22.199668+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:37.640478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120808+0000","last_change":"2024-06-10T00:52:45.995767+0000","last_active":"2024-06-10T00:53:22.120808+0000","last_peered":"2024-06-10T00:53:22.120808+0000","last_clean":"2024-06-10T00:53:22.120808+0000","last_became_active":"2024-06-10T00:52:45.994659+0000","last_became_peered":"2024-06-10T00:52:45.994659+0000","last_unstale":"2024-06-10T00:53:22.120808+0000","last_undegraded":"2024-06-10T00:53:22.120808+0000","last_fullsized":"2024-06-10T00:53:22.120808+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:03:45.652248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"98'6","reported_seq":68,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:54:00.328112+0000","last_change":"2024-06-10T00:52:32.761637+0000","last_active":"2024-06-10T00:54:00.328112+0000","last_peered":"2024-06-10T00:54:00.328112+0000","last_clean":"2024-06-10T00:54:00.328112+0000","last_became_active":"2024-06-10T00:52:32.761144+0000","last_became_peered":"2024-06-10T00:52:32.761144+0000","last_unstale":"2024-06-10T00:54:00.328112+0000","last_undegraded":"2024-06-10T00:54:00.328112+0000","last_fullsized":"2024-06-10T00:54:00.328112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:55:05.616564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124405+0000","last_change":"2024-06-10T00:52:35.962683+0000","last_active":"2024-06-10T00:53:22.124405+0000","last_peered":"2024-06-10T00:53:22.124405+0000","last_clean":"2024-06-10T00:53:22.124405+0000","last_became_active":"2024-06-10T00:52:35.962327+0000","last_became_peered":"2024-06-10T00:52:35.962327+0000","last_unstale":"2024-06-10T00:53:22.124405+0000","last_undegraded":"2024-06-10T00:53:22.124405+0000","last_fullsized":"2024-06-10T00:53:22.124405+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:38:19.063306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120676+0000","last_change":"2024-06-10T00:52:45.996882+0000","last_active":"2024-06-10T00:53:22.120676+0000","last_peered":"2024-06-10T00:53:22.120676+0000","last_clean":"2024-06-10T00:53:22.120676+0000","last_became_active":"2024-06-10T00:52:45.995767+0000","last_became_peered":"2024-06-10T00:52:45.995767+0000","last_unstale":"2024-06-10T00:53:22.120676+0000","last_undegraded":"2024-06-10T00:53:22.120676+0000","last_fullsized":"2024-06-10T00:53:22.120676+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:28:03.766944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":49,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:39.289504+0000","last_change":"2024-06-10T00:53:22.200546+0000","last_active":"2024-06-10T00:53:39.289504+0000","last_peered":"2024-06-10T00:53:39.289504+0000","last_clean":"2024-06-10T00:53:39.289504+0000","last_became_active":"2024-06-10T00:53:22.200059+0000","last_became_peered":"2024-06-10T00:53:22.200059+0000","last_unstale":"2024-06-10T00:53:39.289504+0000","last_undegraded":"2024-06-10T00:53:39.289504+0000","last_fullsized":"2024-06-10T00:53:39.289504+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:08:47.682594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.127463+0000","last_change":"2024-06-10T00:52:35.963105+0000","last_active":"2024-06-10T00:53:22.127463+0000","last_peered":"2024-06-10T00:53:22.127463+0000","last_clean":"2024-06-10T00:53:22.127463+0000","last_became_active":"2024-06-10T00:52:35.962627+0000","last_became_peered":"2024-06-10T00:52:35.962627+0000","last_unstale":"2024-06-10T00:53:22.127463+0000","last_undegraded":"2024-06-10T00:53:22.127463+0000","last_fullsized":"2024-06-10T00:53:22.127463+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:55:51.231795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.123318+0000","last_change":"2024-06-10T00:52:45.992557+0000","last_active":"2024-06-10T00:53:22.123318+0000","last_peered":"2024-06-10T00:53:22.123318+0000","last_clean":"2024-06-10T00:53:22.123318+0000","last_became_active":"2024-06-10T00:52:45.992166+0000","last_became_peered":"2024-06-10T00:52:45.992166+0000","last_unstale":"2024-06-10T00:53:22.123318+0000","last_undegraded":"2024-06-10T00:53:22.123318+0000","last_fullsized":"2024-06-10T00:53:22.123318+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:49:22.368409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:39.287388+0000","last_change":"2024-06-10T00:52:32.751867+0000","last_active":"2024-06-10T00:53:39.287388+0000","last_peered":"2024-06-10T00:53:39.287388+0000","last_clean":"2024-06-10T00:53:39.287388+0000","last_became_active":"2024-06-10T00:52:32.751516+0000","last_became_peered":"2024-06-10T00:52:32.751516+0000","last_unstale":"2024-06-10T00:53:39.287388+0000","last_undegraded":"2024-06-10T00:53:39.287388+0000","last_fullsized":"2024-06-10T00:53:39.287388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:37:55.920613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.119900+0000","last_change":"2024-06-10T00:52:35.957772+0000","last_active":"2024-06-10T00:53:22.119900+0000","last_peered":"2024-06-10T00:53:22.119900+0000","last_clean":"2024-06-10T00:53:22.119900+0000","last_became_active":"2024-06-10T00:52:35.957470+0000","last_became_peered":"2024-06-10T00:52:35.957470+0000","last_unstale":"2024-06-10T00:53:22.119900+0000","last_undegraded":"2024-06-10T00:53:22.119900+0000","last_fullsized":"2024-06-10T00:53:22.119900+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:16:43.645292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.125047+0000","last_change":"2024-06-10T00:52:45.996807+0000","last_active":"2024-06-10T00:53:22.125047+0000","last_peered":"2024-06-10T00:53:22.125047+0000","last_clean":"2024-06-10T00:53:22.125047+0000","last_became_active":"2024-06-10T00:52:45.996378+0000","last_became_peered":"2024-06-10T00:52:45.996378+0000","last_unstale":"2024-06-10T00:53:22.125047+0000","last_undegraded":"2024-06-10T00:53:22.125047+0000","last_fullsized":"2024-06-10T00:53:22.125047+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:40:54.251790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'1","reported_seq":39,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.127379+0000","last_change":"2024-06-10T00:52:32.759625+0000","last_active":"2024-06-10T00:53:22.127379+0000","last_peered":"2024-06-10T00:53:22.127379+0000","last_clean":"2024-06-10T00:53:22.127379+0000","last_became_active":"2024-06-10T00:52:32.759299+0000","last_became_peered":"2024-06-10T00:52:32.759299+0000","last_unstale":"2024-06-10T00:53:22.127379+0000","last_undegraded":"2024-06-10T00:53:22.127379+0000","last_fullsized":"2024-06-10T00:53:22.127379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:11:44.966453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.123037+0000","last_change":"2024-06-10T00:52:35.957672+0000","last_active":"2024-06-10T00:53:22.123037+0000","last_peered":"2024-06-10T00:53:22.123037+0000","last_clean":"2024-06-10T00:53:22.123037+0000","last_became_active":"2024-06-10T00:52:35.957328+0000","last_became_peered":"2024-06-10T00:52:35.957328+0000","last_unstale":"2024-06-10T00:53:22.123037+0000","last_undegraded":"2024-06-10T00:53:22.123037+0000","last_fullsized":"2024-06-10T00:53:22.123037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:43:59.658178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.119965+0000","last_change":"2024-06-10T00:52:45.996677+0000","last_active":"2024-06-10T00:53:22.119965+0000","last_peered":"2024-06-10T00:53:22.119965+0000","last_clean":"2024-06-10T00:53:22.119965+0000","last_became_active":"2024-06-10T00:52:45.994483+0000","last_became_peered":"2024-06-10T00:52:45.994483+0000","last_unstale":"2024-06-10T00:53:22.119965+0000","last_undegraded":"2024-06-10T00:53:22.119965+0000","last_fullsized":"2024-06-10T00:53:22.119965+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:03:28.712072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"98'3","reported_seq":68,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:54:04.333495+0000","last_change":"2024-06-10T00:52:32.754829+0000","last_active":"2024-06-10T00:54:04.333495+0000","last_peered":"2024-06-10T00:54:04.333495+0000","last_clean":"2024-06-10T00:54:04.333495+0000","last_became_active":"2024-06-10T00:52:32.754386+0000","last_became_peered":"2024-06-10T00:52:32.754386+0000","last_unstale":"2024-06-10T00:54:04.333495+0000","last_undegraded":"2024-06-10T00:54:04.333495+0000","last_fullsized":"2024-06-10T00:54:04.333495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:00.870324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.199229+0000","last_change":"2024-06-10T00:52:35.966871+0000","last_active":"2024-06-10T00:53:22.199229+0000","last_peered":"2024-06-10T00:53:22.199229+0000","last_clean":"2024-06-10T00:53:22.199229+0000","last_became_active":"2024-06-10T00:52:35.966492+0000","last_became_peered":"2024-06-10T00:52:35.966492+0000","last_unstale":"2024-06-10T00:53:22.199229+0000","last_undegraded":"2024-06-10T00:53:22.199229+0000","last_fullsized":"2024-06-10T00:53:22.199229+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:24:41.048935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124082+0000","last_change":"2024-06-10T00:52:45.998906+0000","last_active":"2024-06-10T00:53:22.124082+0000","last_peered":"2024-06-10T00:53:22.124082+0000","last_clean":"2024-06-10T00:53:22.124082+0000","last_became_active":"2024-06-10T00:52:45.998527+0000","last_became_peered":"2024-06-10T00:52:45.998527+0000","last_unstale":"2024-06-10T00:53:22.124082+0000","last_undegraded":"2024-06-10T00:53:22.124082+0000","last_fullsized":"2024-06-10T00:53:22.124082+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:46:15.986497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'2","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:38.283516+0000","last_change":"2024-06-10T00:52:32.765794+0000","last_active":"2024-06-10T00:53:38.283516+0000","last_peered":"2024-06-10T00:53:38.283516+0000","last_clean":"2024-06-10T00:53:38.283516+0000","last_became_active":"2024-06-10T00:52:32.765276+0000","last_became_peered":"2024-06-10T00:52:32.765276+0000","last_unstale":"2024-06-10T00:53:38.283516+0000","last_undegraded":"2024-06-10T00:53:38.283516+0000","last_fullsized":"2024-06-10T00:53:38.283516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:06:51.209012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.125460+0000","last_change":"2024-06-10T00:52:35.954897+0000","last_active":"2024-06-10T00:53:22.125460+0000","last_peered":"2024-06-10T00:53:22.125460+0000","last_clean":"2024-06-10T00:53:22.125460+0000","last_became_active":"2024-06-10T00:52:35.954565+0000","last_became_peered":"2024-06-10T00:52:35.954565+0000","last_unstale":"2024-06-10T00:53:22.125460+0000","last_undegraded":"2024-06-10T00:53:22.125460+0000","last_fullsized":"2024-06-10T00:53:22.125460+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:39:26.702804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.200616+0000","last_change":"2024-06-10T00:52:45.992194+0000","last_active":"2024-06-10T00:53:22.200616+0000","last_peered":"2024-06-10T00:53:22.200616+0000","last_clean":"2024-06-10T00:53:22.200616+0000","last_became_active":"2024-06-10T00:52:45.991453+0000","last_became_peered":"2024-06-10T00:52:45.991453+0000","last_unstale":"2024-06-10T00:53:22.200616+0000","last_undegraded":"2024-06-10T00:53:22.200616+0000","last_fullsized":"2024-06-10T00:53:22.200616+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:11:10.775610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'4","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:38.282608+0000","last_change":"2024-06-10T00:52:32.962214+0000","last_active":"2024-06-10T00:53:38.282608+0000","last_peered":"2024-06-10T00:53:38.282608+0000","last_clean":"2024-06-10T00:53:38.282608+0000","last_became_active":"2024-06-10T00:52:32.961835+0000","last_became_peered":"2024-06-10T00:52:32.961835+0000","last_unstale":"2024-06-10T00:53:38.282608+0000","last_undegraded":"2024-06-10T00:53:38.282608+0000","last_fullsized":"2024-06-10T00:53:38.282608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:36:44.724122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.198466+0000","last_change":"2024-06-10T00:52:35.963413+0000","last_active":"2024-06-10T00:53:22.198466+0000","last_peered":"2024-06-10T00:53:22.198466+0000","last_clean":"2024-06-10T00:53:22.198466+0000","last_became_active":"2024-06-10T00:52:35.962922+0000","last_became_peered":"2024-06-10T00:52:35.962922+0000","last_unstale":"2024-06-10T00:53:22.198466+0000","last_undegraded":"2024-06-10T00:53:22.198466+0000","last_fullsized":"2024-06-10T00:53:22.198466+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:05:32.164696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.123633+0000","last_change":"2024-06-10T00:52:45.997601+0000","last_active":"2024-06-10T00:53:22.123633+0000","last_peered":"2024-06-10T00:53:22.123633+0000","last_clean":"2024-06-10T00:53:22.123633+0000","last_became_active":"2024-06-10T00:52:45.997231+0000","last_became_peered":"2024-06-10T00:52:45.997231+0000","last_unstale":"2024-06-10T00:53:22.123633+0000","last_undegraded":"2024-06-10T00:53:22.123633+0000","last_fullsized":"2024-06-10T00:53:22.123633+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:52:30.994486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"98'34","reported_seq":132,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:54:05.340885+0000","last_change":"2024-06-10T00:52:32.761592+0000","last_active":"2024-06-10T00:54:05.340885+0000","last_peered":"2024-06-10T00:54:05.340885+0000","last_clean":"2024-06-10T00:54:05.340885+0000","last_became_active":"2024-06-10T00:52:32.761169+0000","last_became_peered":"2024-06-10T00:52:32.761169+0000","last_unstale":"2024-06-10T00:54:05.340885+0000","last_undegraded":"2024-06-10T00:54:05.340885+0000","last_fullsized":"2024-06-10T00:54:05.340885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:08:33.015035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64876,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":58,"num_read_kb":94,"num_write":35,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124609+0000","last_change":"2024-06-10T00:52:35.955304+0000","last_active":"2024-06-10T00:53:22.124609+0000","last_peered":"2024-06-10T00:53:22.124609+0000","last_clean":"2024-06-10T00:53:22.124609+0000","last_became_active":"2024-06-10T00:52:35.954949+0000","last_became_peered":"2024-06-10T00:52:35.954949+0000","last_unstale":"2024-06-10T00:53:22.124609+0000","last_undegraded":"2024-06-10T00:53:22.124609+0000","last_fullsized":"2024-06-10T00:53:22.124609+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:56:42.890022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.128098+0000","last_change":"2024-06-10T00:52:45.992974+0000","last_active":"2024-06-10T00:53:22.128098+0000","last_peered":"2024-06-10T00:53:22.128098+0000","last_clean":"2024-06-10T00:53:22.128098+0000","last_became_active":"2024-06-10T00:52:45.992490+0000","last_became_peered":"2024-06-10T00:52:45.992490+0000","last_unstale":"2024-06-10T00:53:22.128098+0000","last_undegraded":"2024-06-10T00:53:22.128098+0000","last_fullsized":"2024-06-10T00:53:22.128098+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:03:45.543476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:39.289638+0000","last_change":"2024-06-10T00:52:32.762347+0000","last_active":"2024-06-10T00:53:39.289638+0000","last_peered":"2024-06-10T00:53:39.289638+0000","last_clean":"2024-06-10T00:53:39.289638+0000","last_became_active":"2024-06-10T00:52:32.761049+0000","last_became_peered":"2024-06-10T00:52:32.761049+0000","last_unstale":"2024-06-10T00:53:39.289638+0000","last_undegraded":"2024-06-10T00:53:39.289638+0000","last_fullsized":"2024-06-10T00:53:39.289638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:29:38.384451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.128558+0000","last_change":"2024-06-10T00:52:35.961299+0000","last_active":"2024-06-10T00:53:22.128558+0000","last_peered":"2024-06-10T00:53:22.128558+0000","last_clean":"2024-06-10T00:53:22.128558+0000","last_became_active":"2024-06-10T00:52:35.960966+0000","last_became_peered":"2024-06-10T00:52:35.960966+0000","last_unstale":"2024-06-10T00:53:22.128558+0000","last_undegraded":"2024-06-10T00:53:22.128558+0000","last_fullsized":"2024-06-10T00:53:22.128558+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:32:21.269689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.122972+0000","last_change":"2024-06-10T00:52:46.002091+0000","last_active":"2024-06-10T00:53:22.122972+0000","last_peered":"2024-06-10T00:53:22.122972+0000","last_clean":"2024-06-10T00:53:22.122972+0000","last_became_active":"2024-06-10T00:52:46.001677+0000","last_became_peered":"2024-06-10T00:52:46.001677+0000","last_unstale":"2024-06-10T00:53:22.122972+0000","last_undegraded":"2024-06-10T00:53:22.122972+0000","last_fullsized":"2024-06-10T00:53:22.122972+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:16:56.405720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'3","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:38.284301+0000","last_change":"2024-06-10T00:52:32.756765+0000","last_active":"2024-06-10T00:53:38.284301+0000","last_peered":"2024-06-10T00:53:38.284301+0000","last_clean":"2024-06-10T00:53:38.284301+0000","last_became_active":"2024-06-10T00:52:32.756470+0000","last_became_peered":"2024-06-10T00:52:32.756470+0000","last_unstale":"2024-06-10T00:53:38.284301+0000","last_undegraded":"2024-06-10T00:53:38.284301+0000","last_fullsized":"2024-06-10T00:53:38.284301+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:29:24.854913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120568+0000","last_change":"2024-06-10T00:52:35.960693+0000","last_active":"2024-06-10T00:53:22.120568+0000","last_peered":"2024-06-10T00:53:22.120568+0000","last_clean":"2024-06-10T00:53:22.120568+0000","last_became_active":"2024-06-10T00:52:35.958397+0000","last_became_peered":"2024-06-10T00:52:35.958397+0000","last_unstale":"2024-06-10T00:53:22.120568+0000","last_undegraded":"2024-06-10T00:53:22.120568+0000","last_fullsized":"2024-06-10T00:53:22.120568+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:23:47.556462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.123769+0000","last_change":"2024-06-10T00:52:46.001475+0000","last_active":"2024-06-10T00:53:22.123769+0000","last_peered":"2024-06-10T00:53:22.123769+0000","last_clean":"2024-06-10T00:53:22.123769+0000","last_became_active":"2024-06-10T00:52:46.000983+0000","last_became_peered":"2024-06-10T00:52:46.000983+0000","last_unstale":"2024-06-10T00:53:22.123769+0000","last_undegraded":"2024-06-10T00:53:22.123769+0000","last_fullsized":"2024-06-10T00:53:22.123769+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:23:56.285356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"98'3","reported_seq":66,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:54:01.329121+0000","last_change":"2024-06-10T00:52:32.756665+0000","last_active":"2024-06-10T00:54:01.329121+0000","last_peered":"2024-06-10T00:54:01.329121+0000","last_clean":"2024-06-10T00:54:01.329121+0000","last_became_active":"2024-06-10T00:52:32.756221+0000","last_became_peered":"2024-06-10T00:52:32.756221+0000","last_unstale":"2024-06-10T00:54:01.329121+0000","last_undegraded":"2024-06-10T00:54:01.329121+0000","last_fullsized":"2024-06-10T00:54:01.329121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:50:34.789063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":39,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.125844+0000","last_change":"2024-06-10T00:53:22.125844+0000","last_active":"2024-06-10T00:53:22.125844+0000","last_peered":"2024-06-10T00:53:22.125844+0000","last_clean":"2024-06-10T00:53:22.125844+0000","last_became_active":"2024-06-10T00:53:22.125449+0000","last_became_peered":"2024-06-10T00:53:22.125449+0000","last_unstale":"2024-06-10T00:53:22.125844+0000","last_undegraded":"2024-06-10T00:53:22.125844+0000","last_fullsized":"2024-06-10T00:53:22.125844+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:08:42.936702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.128988+0000","last_change":"2024-06-10T00:52:45.999300+0000","last_active":"2024-06-10T00:53:22.128988+0000","last_peered":"2024-06-10T00:53:22.128988+0000","last_clean":"2024-06-10T00:53:22.128988+0000","last_became_active":"2024-06-10T00:52:45.998860+0000","last_became_peered":"2024-06-10T00:52:45.998860+0000","last_unstale":"2024-06-10T00:53:22.128988+0000","last_undegraded":"2024-06-10T00:53:22.128988+0000","last_fullsized":"2024-06-10T00:53:22.128988+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:14:24.936870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":70,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:39.289823+0000","last_change":"2024-06-10T00:53:22.210710+0000","last_active":"2024-06-10T00:53:39.289823+0000","last_peered":"2024-06-10T00:53:39.289823+0000","last_clean":"2024-06-10T00:53:39.289823+0000","last_became_active":"2024-06-10T00:53:22.202461+0000","last_became_peered":"2024-06-10T00:53:22.202461+0000","last_unstale":"2024-06-10T00:53:39.289823+0000","last_undegraded":"2024-06-10T00:53:39.289823+0000","last_fullsized":"2024-06-10T00:53:39.289823+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:28:14.671776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.127229+0000","last_change":"2024-06-10T00:52:35.966960+0000","last_active":"2024-06-10T00:53:22.127229+0000","last_peered":"2024-06-10T00:53:22.127229+0000","last_clean":"2024-06-10T00:53:22.127229+0000","last_became_active":"2024-06-10T00:52:35.966630+0000","last_became_peered":"2024-06-10T00:52:35.966630+0000","last_unstale":"2024-06-10T00:53:22.127229+0000","last_undegraded":"2024-06-10T00:53:22.127229+0000","last_fullsized":"2024-06-10T00:53:22.127229+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:04.117405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.127174+0000","last_change":"2024-06-10T00:52:45.992955+0000","last_active":"2024-06-10T00:53:22.127174+0000","last_peered":"2024-06-10T00:53:22.127174+0000","last_clean":"2024-06-10T00:53:22.127174+0000","last_became_active":"2024-06-10T00:52:45.992457+0000","last_became_peered":"2024-06-10T00:52:45.992457+0000","last_unstale":"2024-06-10T00:53:22.127174+0000","last_undegraded":"2024-06-10T00:53:22.127174+0000","last_fullsized":"2024-06-10T00:53:22.127174+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:13:49.491183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:41.285013+0000","last_change":"2024-06-10T00:52:32.763794+0000","last_active":"2024-06-10T00:53:41.285013+0000","last_peered":"2024-06-10T00:53:41.285013+0000","last_clean":"2024-06-10T00:53:41.285013+0000","last_became_active":"2024-06-10T00:52:32.763406+0000","last_became_peered":"2024-06-10T00:52:32.763406+0000","last_unstale":"2024-06-10T00:53:41.285013+0000","last_undegraded":"2024-06-10T00:53:41.285013+0000","last_fullsized":"2024-06-10T00:53:41.285013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:50:33.580163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.130891+0000","last_change":"2024-06-10T00:53:22.130891+0000","last_active":"2024-06-10T00:53:22.130891+0000","last_peered":"2024-06-10T00:53:22.130891+0000","last_clean":"2024-06-10T00:53:22.130891+0000","last_became_active":"2024-06-10T00:53:22.130508+0000","last_became_peered":"2024-06-10T00:53:22.130508+0000","last_unstale":"2024-06-10T00:53:22.130891+0000","last_undegraded":"2024-06-10T00:53:22.130891+0000","last_fullsized":"2024-06-10T00:53:22.130891+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:28:01.919825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124284+0000","last_change":"2024-06-10T00:52:46.241698+0000","last_active":"2024-06-10T00:53:22.124284+0000","last_peered":"2024-06-10T00:53:22.124284+0000","last_clean":"2024-06-10T00:53:22.124284+0000","last_became_active":"2024-06-10T00:52:46.241257+0000","last_became_peered":"2024-06-10T00:52:46.241257+0000","last_unstale":"2024-06-10T00:53:22.124284+0000","last_undegraded":"2024-06-10T00:53:22.124284+0000","last_fullsized":"2024-06-10T00:53:22.124284+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:17:34.137750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124115+0000","last_change":"2024-06-10T00:52:32.960515+0000","last_active":"2024-06-10T00:53:22.124115+0000","last_peered":"2024-06-10T00:53:22.124115+0000","last_clean":"2024-06-10T00:53:22.124115+0000","last_became_active":"2024-06-10T00:52:32.960163+0000","last_became_peered":"2024-06-10T00:52:32.960163+0000","last_unstale":"2024-06-10T00:53:22.124115+0000","last_undegraded":"2024-06-10T00:53:22.124115+0000","last_fullsized":"2024-06-10T00:53:22.124115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:39:51.374998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124440+0000","last_change":"2024-06-10T00:52:35.961884+0000","last_active":"2024-06-10T00:53:22.124440+0000","last_peered":"2024-06-10T00:53:22.124440+0000","last_clean":"2024-06-10T00:53:22.124440+0000","last_became_active":"2024-06-10T00:52:35.961497+0000","last_became_peered":"2024-06-10T00:52:35.961497+0000","last_unstale":"2024-06-10T00:53:22.124440+0000","last_undegraded":"2024-06-10T00:53:22.124440+0000","last_fullsized":"2024-06-10T00:53:22.124440+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:16:09.284184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.128831+0000","last_change":"2024-06-10T00:52:45.990846+0000","last_active":"2024-06-10T00:53:22.128831+0000","last_peered":"2024-06-10T00:53:22.128831+0000","last_clean":"2024-06-10T00:53:22.128831+0000","last_became_active":"2024-06-10T00:52:45.990441+0000","last_became_peered":"2024-06-10T00:52:45.990441+0000","last_unstale":"2024-06-10T00:53:22.128831+0000","last_undegraded":"2024-06-10T00:53:22.128831+0000","last_fullsized":"2024-06-10T00:53:22.128831+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:19:35.690378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120615+0000","last_change":"2024-06-10T00:52:32.753913+0000","last_active":"2024-06-10T00:53:22.120615+0000","last_peered":"2024-06-10T00:53:22.120615+0000","last_clean":"2024-06-10T00:53:22.120615+0000","last_became_active":"2024-06-10T00:52:32.753174+0000","last_became_peered":"2024-06-10T00:52:32.753174+0000","last_unstale":"2024-06-10T00:53:22.120615+0000","last_undegraded":"2024-06-10T00:53:22.120615+0000","last_fullsized":"2024-06-10T00:53:22.120615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:13:20.048635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124179+0000","last_change":"2024-06-10T00:52:35.965783+0000","last_active":"2024-06-10T00:53:22.124179+0000","last_peered":"2024-06-10T00:53:22.124179+0000","last_clean":"2024-06-10T00:53:22.124179+0000","last_became_active":"2024-06-10T00:52:35.965432+0000","last_became_peered":"2024-06-10T00:52:35.965432+0000","last_unstale":"2024-06-10T00:53:22.124179+0000","last_undegraded":"2024-06-10T00:53:22.124179+0000","last_fullsized":"2024-06-10T00:53:22.124179+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:52:01.240318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.128810+0000","last_change":"2024-06-10T00:52:46.241662+0000","last_active":"2024-06-10T00:53:22.128810+0000","last_peered":"2024-06-10T00:53:22.128810+0000","last_clean":"2024-06-10T00:53:22.128810+0000","last_became_active":"2024-06-10T00:52:46.241211+0000","last_became_peered":"2024-06-10T00:52:46.241211+0000","last_unstale":"2024-06-10T00:53:22.128810+0000","last_undegraded":"2024-06-10T00:53:22.128810+0000","last_fullsized":"2024-06-10T00:53:22.128810+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:12.041290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.201271+0000","last_change":"2024-06-10T00:53:22.200590+0000","last_active":"2024-06-10T00:53:22.201271+0000","last_peered":"2024-06-10T00:53:22.201271+0000","last_clean":"2024-06-10T00:53:22.201271+0000","last_became_active":"2024-06-10T00:53:22.200046+0000","last_became_peered":"2024-06-10T00:53:22.200046+0000","last_unstale":"2024-06-10T00:53:22.201271+0000","last_undegraded":"2024-06-10T00:53:22.201271+0000","last_fullsized":"2024-06-10T00:53:22.201271+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:43:22.378316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"98'4","reported_seq":65,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:54:01.328515+0000","last_change":"2024-06-10T00:52:32.959904+0000","last_active":"2024-06-10T00:54:01.328515+0000","last_peered":"2024-06-10T00:54:01.328515+0000","last_clean":"2024-06-10T00:54:01.328515+0000","last_became_active":"2024-06-10T00:52:32.959506+0000","last_became_peered":"2024-06-10T00:52:32.959506+0000","last_unstale":"2024-06-10T00:54:01.328515+0000","last_undegraded":"2024-06-10T00:54:01.328515+0000","last_fullsized":"2024-06-10T00:54:01.328515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:23:15.950345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":23,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120769+0000","last_change":"2024-06-10T00:52:45.993305+0000","last_active":"2024-06-10T00:53:22.120769+0000","last_peered":"2024-06-10T00:53:22.120769+0000","last_clean":"2024-06-10T00:53:22.120769+0000","last_became_active":"2024-06-10T00:52:45.992682+0000","last_became_peered":"2024-06-10T00:52:45.992682+0000","last_unstale":"2024-06-10T00:53:22.120769+0000","last_undegraded":"2024-06-10T00:53:22.120769+0000","last_fullsized":"2024-06-10T00:53:22.120769+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:49:41.817126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120099+0000","last_change":"2024-06-10T00:52:35.965893+0000","last_active":"2024-06-10T00:53:22.120099+0000","last_peered":"2024-06-10T00:53:22.120099+0000","last_clean":"2024-06-10T00:53:22.120099+0000","last_became_active":"2024-06-10T00:52:35.965426+0000","last_became_peered":"2024-06-10T00:52:35.965426+0000","last_unstale":"2024-06-10T00:53:22.120099+0000","last_undegraded":"2024-06-10T00:53:22.120099+0000","last_fullsized":"2024-06-10T00:53:22.120099+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:29:19.640836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:41.297842+0000","last_change":"2024-06-10T00:52:32.753631+0000","last_active":"2024-06-10T00:53:41.297842+0000","last_peered":"2024-06-10T00:53:41.297842+0000","last_clean":"2024-06-10T00:53:41.297842+0000","last_became_active":"2024-06-10T00:52:32.753260+0000","last_became_peered":"2024-06-10T00:52:32.753260+0000","last_unstale":"2024-06-10T00:53:41.297842+0000","last_undegraded":"2024-06-10T00:53:41.297842+0000","last_fullsized":"2024-06-10T00:53:41.297842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:12:55.932676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":364,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120324+0000","last_change":"2024-06-10T00:51:09.267750+0000","last_active":"2024-06-10T00:53:22.120324+0000","last_peered":"2024-06-10T00:53:22.120324+0000","last_clean":"2024-06-10T00:53:22.120324+0000","last_became_active":"2024-06-10T00:51:09.267246+0000","last_became_peered":"2024-06-10T00:51:09.267246+0000","last_unstale":"2024-06-10T00:53:22.120324+0000","last_undegraded":"2024-06-10T00:53:22.120324+0000","last_fullsized":"2024-06-10T00:53:22.120324+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:47:44.965452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:47:44.965452+0000","last_clean_scrub_stamp":"2024-06-10T00:47:44.965452+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:21.100090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.128372+0000","last_change":"2024-06-10T00:52:46.242152+0000","last_active":"2024-06-10T00:53:22.128372+0000","last_peered":"2024-06-10T00:53:22.128372+0000","last_clean":"2024-06-10T00:53:22.128372+0000","last_became_active":"2024-06-10T00:52:46.241542+0000","last_became_peered":"2024-06-10T00:52:46.241542+0000","last_unstale":"2024-06-10T00:53:22.128372+0000","last_undegraded":"2024-06-10T00:53:22.128372+0000","last_fullsized":"2024-06-10T00:53:22.128372+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:55:54.695465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.128241+0000","last_change":"2024-06-10T00:52:35.963911+0000","last_active":"2024-06-10T00:53:22.128241+0000","last_peered":"2024-06-10T00:53:22.128241+0000","last_clean":"2024-06-10T00:53:22.128241+0000","last_became_active":"2024-06-10T00:52:35.963571+0000","last_became_peered":"2024-06-10T00:52:35.963571+0000","last_unstale":"2024-06-10T00:53:22.128241+0000","last_undegraded":"2024-06-10T00:53:22.128241+0000","last_fullsized":"2024-06-10T00:53:22.128241+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:27:16.757428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:39.289046+0000","last_change":"2024-06-10T00:53:22.129035+0000","last_active":"2024-06-10T00:53:39.289046+0000","last_peered":"2024-06-10T00:53:39.289046+0000","last_clean":"2024-06-10T00:53:39.289046+0000","last_became_active":"2024-06-10T00:53:22.128561+0000","last_became_peered":"2024-06-10T00:53:22.128561+0000","last_unstale":"2024-06-10T00:53:39.289046+0000","last_undegraded":"2024-06-10T00:53:39.289046+0000","last_fullsized":"2024-06-10T00:53:39.289046+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:02:43.313118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.122925+0000","last_change":"2024-06-10T00:52:45.996907+0000","last_active":"2024-06-10T00:53:22.122925+0000","last_peered":"2024-06-10T00:53:22.122925+0000","last_clean":"2024-06-10T00:53:22.122925+0000","last_became_active":"2024-06-10T00:52:45.996554+0000","last_became_peered":"2024-06-10T00:52:45.996554+0000","last_unstale":"2024-06-10T00:53:22.122925+0000","last_undegraded":"2024-06-10T00:53:22.122925+0000","last_fullsized":"2024-06-10T00:53:22.122925+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:00:46.824071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'1","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:38.282338+0000","last_change":"2024-06-10T00:52:32.960223+0000","last_active":"2024-06-10T00:53:38.282338+0000","last_peered":"2024-06-10T00:53:38.282338+0000","last_clean":"2024-06-10T00:53:38.282338+0000","last_became_active":"2024-06-10T00:52:32.959871+0000","last_became_peered":"2024-06-10T00:52:32.959871+0000","last_unstale":"2024-06-10T00:53:38.282338+0000","last_undegraded":"2024-06-10T00:53:38.282338+0000","last_fullsized":"2024-06-10T00:53:38.282338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:18:17.099140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.129037+0000","last_change":"2024-06-10T00:52:35.962516+0000","last_active":"2024-06-10T00:53:22.129037+0000","last_peered":"2024-06-10T00:53:22.129037+0000","last_clean":"2024-06-10T00:53:22.129037+0000","last_became_active":"2024-06-10T00:52:35.962187+0000","last_became_peered":"2024-06-10T00:52:35.962187+0000","last_unstale":"2024-06-10T00:53:22.129037+0000","last_undegraded":"2024-06-10T00:53:22.129037+0000","last_fullsized":"2024-06-10T00:53:22.129037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:10:44.295192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.119897+0000","last_change":"2024-06-10T00:52:46.241554+0000","last_active":"2024-06-10T00:53:22.119897+0000","last_peered":"2024-06-10T00:53:22.119897+0000","last_clean":"2024-06-10T00:53:22.119897+0000","last_became_active":"2024-06-10T00:52:46.241142+0000","last_became_peered":"2024-06-10T00:52:46.241142+0000","last_unstale":"2024-06-10T00:53:22.119897+0000","last_undegraded":"2024-06-10T00:53:22.119897+0000","last_fullsized":"2024-06-10T00:53:22.119897+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:58:07.233080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"98'10","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:54:01.967107+0000","last_change":"2024-06-10T00:52:32.761854+0000","last_active":"2024-06-10T00:54:01.967107+0000","last_peered":"2024-06-10T00:54:01.967107+0000","last_clean":"2024-06-10T00:54:01.967107+0000","last_became_active":"2024-06-10T00:52:32.761522+0000","last_became_peered":"2024-06-10T00:52:32.761522+0000","last_unstale":"2024-06-10T00:54:01.967107+0000","last_undegraded":"2024-06-10T00:54:01.967107+0000","last_fullsized":"2024-06-10T00:54:01.967107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:06:43.269001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120684+0000","last_change":"2024-06-10T00:52:35.959250+0000","last_active":"2024-06-10T00:53:22.120684+0000","last_peered":"2024-06-10T00:53:22.120684+0000","last_clean":"2024-06-10T00:53:22.120684+0000","last_became_active":"2024-06-10T00:52:35.958925+0000","last_became_peered":"2024-06-10T00:52:35.958925+0000","last_unstale":"2024-06-10T00:53:22.120684+0000","last_undegraded":"2024-06-10T00:53:22.120684+0000","last_fullsized":"2024-06-10T00:53:22.120684+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:11:12.355505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120321+0000","last_change":"2024-06-10T00:52:45.991819+0000","last_active":"2024-06-10T00:53:22.120321+0000","last_peered":"2024-06-10T00:53:22.120321+0000","last_clean":"2024-06-10T00:53:22.120321+0000","last_became_active":"2024-06-10T00:52:45.991408+0000","last_became_peered":"2024-06-10T00:52:45.991408+0000","last_unstale":"2024-06-10T00:53:22.120321+0000","last_undegraded":"2024-06-10T00:53:22.120321+0000","last_fullsized":"2024-06-10T00:53:22.120321+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:16:34.912359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'4","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:38.280550+0000","last_change":"2024-06-10T00:52:32.869094+0000","last_active":"2024-06-10T00:53:38.280550+0000","last_peered":"2024-06-10T00:53:38.280550+0000","last_clean":"2024-06-10T00:53:38.280550+0000","last_became_active":"2024-06-10T00:52:32.868664+0000","last_became_peered":"2024-06-10T00:52:32.868664+0000","last_unstale":"2024-06-10T00:53:38.280550+0000","last_undegraded":"2024-06-10T00:53:38.280550+0000","last_fullsized":"2024-06-10T00:53:38.280550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:50:38.051090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.125667+0000","last_change":"2024-06-10T00:52:35.952356+0000","last_active":"2024-06-10T00:53:22.125667+0000","last_peered":"2024-06-10T00:53:22.125667+0000","last_clean":"2024-06-10T00:53:22.125667+0000","last_became_active":"2024-06-10T00:52:35.952002+0000","last_became_peered":"2024-06-10T00:52:35.952002+0000","last_unstale":"2024-06-10T00:53:22.125667+0000","last_undegraded":"2024-06-10T00:53:22.125667+0000","last_fullsized":"2024-06-10T00:53:22.125667+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:12:28.581522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.200356+0000","last_change":"2024-06-10T00:52:46.241773+0000","last_active":"2024-06-10T00:53:22.200356+0000","last_peered":"2024-06-10T00:53:22.200356+0000","last_clean":"2024-06-10T00:53:22.200356+0000","last_became_active":"2024-06-10T00:52:46.241272+0000","last_became_peered":"2024-06-10T00:52:46.241272+0000","last_unstale":"2024-06-10T00:53:22.200356+0000","last_undegraded":"2024-06-10T00:53:22.200356+0000","last_fullsized":"2024-06-10T00:53:22.200356+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:07:28.337536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:38.283362+0000","last_change":"2024-06-10T00:52:32.960100+0000","last_active":"2024-06-10T00:53:38.283362+0000","last_peered":"2024-06-10T00:53:38.283362+0000","last_clean":"2024-06-10T00:53:38.283362+0000","last_became_active":"2024-06-10T00:52:32.959779+0000","last_became_peered":"2024-06-10T00:52:32.959779+0000","last_unstale":"2024-06-10T00:53:38.283362+0000","last_undegraded":"2024-06-10T00:53:38.283362+0000","last_fullsized":"2024-06-10T00:53:38.283362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:09.555753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.122508+0000","last_change":"2024-06-10T00:52:35.955586+0000","last_active":"2024-06-10T00:53:22.122508+0000","last_peered":"2024-06-10T00:53:22.122508+0000","last_clean":"2024-06-10T00:53:22.122508+0000","last_became_active":"2024-06-10T00:52:35.955216+0000","last_became_peered":"2024-06-10T00:52:35.955216+0000","last_unstale":"2024-06-10T00:53:22.122508+0000","last_undegraded":"2024-06-10T00:53:22.122508+0000","last_fullsized":"2024-06-10T00:53:22.122508+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:23:12.197182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124090+0000","last_change":"2024-06-10T00:52:46.001381+0000","last_active":"2024-06-10T00:53:22.124090+0000","last_peered":"2024-06-10T00:53:22.124090+0000","last_clean":"2024-06-10T00:53:22.124090+0000","last_became_active":"2024-06-10T00:52:46.000348+0000","last_became_peered":"2024-06-10T00:52:46.000348+0000","last_unstale":"2024-06-10T00:53:22.124090+0000","last_undegraded":"2024-06-10T00:53:22.124090+0000","last_fullsized":"2024-06-10T00:53:22.124090+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:12:28.348852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'3","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:38.282471+0000","last_change":"2024-06-10T00:52:32.753444+0000","last_active":"2024-06-10T00:53:38.282471+0000","last_peered":"2024-06-10T00:53:38.282471+0000","last_clean":"2024-06-10T00:53:38.282471+0000","last_became_active":"2024-06-10T00:52:32.753149+0000","last_became_peered":"2024-06-10T00:52:32.753149+0000","last_unstale":"2024-06-10T00:53:38.282471+0000","last_undegraded":"2024-06-10T00:53:38.282471+0000","last_fullsized":"2024-06-10T00:53:38.282471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:28.241950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124153+0000","last_change":"2024-06-10T00:52:35.966718+0000","last_active":"2024-06-10T00:53:22.124153+0000","last_peered":"2024-06-10T00:53:22.124153+0000","last_clean":"2024-06-10T00:53:22.124153+0000","last_became_active":"2024-06-10T00:52:35.965907+0000","last_became_peered":"2024-06-10T00:52:35.965907+0000","last_unstale":"2024-06-10T00:53:22.124153+0000","last_undegraded":"2024-06-10T00:53:22.124153+0000","last_fullsized":"2024-06-10T00:53:22.124153+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:21:34.261306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":23,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.200347+0000","last_change":"2024-06-10T00:52:45.991034+0000","last_active":"2024-06-10T00:53:22.200347+0000","last_peered":"2024-06-10T00:53:22.200347+0000","last_clean":"2024-06-10T00:53:22.200347+0000","last_became_active":"2024-06-10T00:52:45.990546+0000","last_became_peered":"2024-06-10T00:52:45.990546+0000","last_unstale":"2024-06-10T00:53:22.200347+0000","last_undegraded":"2024-06-10T00:53:22.200347+0000","last_fullsized":"2024-06-10T00:53:22.200347+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:09:23.412922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'1","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120466+0000","last_change":"2024-06-10T00:52:32.764024+0000","last_active":"2024-06-10T00:53:22.120466+0000","last_peered":"2024-06-10T00:53:22.120466+0000","last_clean":"2024-06-10T00:53:22.120466+0000","last_became_active":"2024-06-10T00:52:32.763449+0000","last_became_peered":"2024-06-10T00:52:32.763449+0000","last_unstale":"2024-06-10T00:53:22.120466+0000","last_undegraded":"2024-06-10T00:53:22.120466+0000","last_fullsized":"2024-06-10T00:53:22.120466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:50:23.605873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120301+0000","last_change":"2024-06-10T00:52:35.955570+0000","last_active":"2024-06-10T00:53:22.120301+0000","last_peered":"2024-06-10T00:53:22.120301+0000","last_clean":"2024-06-10T00:53:22.120301+0000","last_became_active":"2024-06-10T00:52:35.955248+0000","last_became_peered":"2024-06-10T00:52:35.955248+0000","last_unstale":"2024-06-10T00:53:22.120301+0000","last_undegraded":"2024-06-10T00:53:22.120301+0000","last_fullsized":"2024-06-10T00:53:22.120301+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:26.124499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.199906+0000","last_change":"2024-06-10T00:52:46.241776+0000","last_active":"2024-06-10T00:53:22.199906+0000","last_peered":"2024-06-10T00:53:22.199906+0000","last_clean":"2024-06-10T00:53:22.199906+0000","last_became_active":"2024-06-10T00:52:46.241276+0000","last_became_peered":"2024-06-10T00:52:46.241276+0000","last_unstale":"2024-06-10T00:53:22.199906+0000","last_undegraded":"2024-06-10T00:53:22.199906+0000","last_fullsized":"2024-06-10T00:53:22.199906+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:08.027938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":53,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:38.284502+0000","last_change":"2024-06-10T00:53:22.204034+0000","last_active":"2024-06-10T00:53:38.284502+0000","last_peered":"2024-06-10T00:53:38.284502+0000","last_clean":"2024-06-10T00:53:38.284502+0000","last_became_active":"2024-06-10T00:53:22.199988+0000","last_became_peered":"2024-06-10T00:53:22.199988+0000","last_unstale":"2024-06-10T00:53:38.284502+0000","last_undegraded":"2024-06-10T00:53:38.284502+0000","last_fullsized":"2024-06-10T00:53:38.284502+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:39:54.012360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120778+0000","last_change":"2024-06-10T00:52:35.959182+0000","last_active":"2024-06-10T00:53:22.120778+0000","last_peered":"2024-06-10T00:53:22.120778+0000","last_clean":"2024-06-10T00:53:22.120778+0000","last_became_active":"2024-06-10T00:52:35.958828+0000","last_became_peered":"2024-06-10T00:52:35.958828+0000","last_unstale":"2024-06-10T00:53:22.120778+0000","last_undegraded":"2024-06-10T00:53:22.120778+0000","last_fullsized":"2024-06-10T00:53:22.120778+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:29:58.857659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.198410+0000","last_change":"2024-06-10T00:52:45.989739+0000","last_active":"2024-06-10T00:53:22.198410+0000","last_peered":"2024-06-10T00:53:22.198410+0000","last_clean":"2024-06-10T00:53:22.198410+0000","last_became_active":"2024-06-10T00:52:45.989279+0000","last_became_peered":"2024-06-10T00:52:45.989279+0000","last_unstale":"2024-06-10T00:53:22.198410+0000","last_undegraded":"2024-06-10T00:53:22.198410+0000","last_fullsized":"2024-06-10T00:53:22.198410+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:19:17.551947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:39.288814+0000","last_change":"2024-06-10T00:52:32.959662+0000","last_active":"2024-06-10T00:53:39.288814+0000","last_peered":"2024-06-10T00:53:39.288814+0000","last_clean":"2024-06-10T00:53:39.288814+0000","last_became_active":"2024-06-10T00:52:32.959286+0000","last_became_peered":"2024-06-10T00:52:32.959286+0000","last_unstale":"2024-06-10T00:53:39.288814+0000","last_undegraded":"2024-06-10T00:53:39.288814+0000","last_fullsized":"2024-06-10T00:53:39.288814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:26:04.361567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.121028+0000","last_change":"2024-06-10T00:52:35.960137+0000","last_active":"2024-06-10T00:53:22.121028+0000","last_peered":"2024-06-10T00:53:22.121028+0000","last_clean":"2024-06-10T00:53:22.121028+0000","last_became_active":"2024-06-10T00:52:35.959584+0000","last_became_peered":"2024-06-10T00:52:35.959584+0000","last_unstale":"2024-06-10T00:53:22.121028+0000","last_undegraded":"2024-06-10T00:53:22.121028+0000","last_fullsized":"2024-06-10T00:53:22.121028+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:23:38.702685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.125101+0000","last_change":"2024-06-10T00:52:46.002268+0000","last_active":"2024-06-10T00:53:22.125101+0000","last_peered":"2024-06-10T00:53:22.125101+0000","last_clean":"2024-06-10T00:53:22.125101+0000","last_became_active":"2024-06-10T00:52:46.001769+0000","last_became_peered":"2024-06-10T00:52:46.001769+0000","last_unstale":"2024-06-10T00:53:22.125101+0000","last_undegraded":"2024-06-10T00:53:22.125101+0000","last_fullsized":"2024-06-10T00:53:22.125101+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:33:51.365951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.127739+0000","last_change":"2024-06-10T00:52:32.757112+0000","last_active":"2024-06-10T00:53:22.127739+0000","last_peered":"2024-06-10T00:53:22.127739+0000","last_clean":"2024-06-10T00:53:22.127739+0000","last_became_active":"2024-06-10T00:52:32.755589+0000","last_became_peered":"2024-06-10T00:52:32.755589+0000","last_unstale":"2024-06-10T00:53:22.127739+0000","last_undegraded":"2024-06-10T00:53:22.127739+0000","last_fullsized":"2024-06-10T00:53:22.127739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:15:12.814489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120794+0000","last_change":"2024-06-10T00:52:35.955925+0000","last_active":"2024-06-10T00:53:22.120794+0000","last_peered":"2024-06-10T00:53:22.120794+0000","last_clean":"2024-06-10T00:53:22.120794+0000","last_became_active":"2024-06-10T00:52:35.955556+0000","last_became_peered":"2024-06-10T00:52:35.955556+0000","last_unstale":"2024-06-10T00:53:22.120794+0000","last_undegraded":"2024-06-10T00:53:22.120794+0000","last_fullsized":"2024-06-10T00:53:22.120794+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:42:18.324484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.126727+0000","last_change":"2024-06-10T00:52:45.988810+0000","last_active":"2024-06-10T00:53:22.126727+0000","last_peered":"2024-06-10T00:53:22.126727+0000","last_clean":"2024-06-10T00:53:22.126727+0000","last_became_active":"2024-06-10T00:52:45.988041+0000","last_became_peered":"2024-06-10T00:52:45.988041+0000","last_unstale":"2024-06-10T00:53:22.126727+0000","last_undegraded":"2024-06-10T00:53:22.126727+0000","last_fullsized":"2024-06-10T00:53:22.126727+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:14:08.979802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.129013+0000","last_change":"2024-06-10T00:52:32.765493+0000","last_active":"2024-06-10T00:53:22.129013+0000","last_peered":"2024-06-10T00:53:22.129013+0000","last_clean":"2024-06-10T00:53:22.129013+0000","last_became_active":"2024-06-10T00:52:32.765193+0000","last_became_peered":"2024-06-10T00:52:32.765193+0000","last_unstale":"2024-06-10T00:53:22.129013+0000","last_undegraded":"2024-06-10T00:53:22.129013+0000","last_fullsized":"2024-06-10T00:53:22.129013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:38:11.554673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.121112+0000","last_change":"2024-06-10T00:52:35.960216+0000","last_active":"2024-06-10T00:53:22.121112+0000","last_peered":"2024-06-10T00:53:22.121112+0000","last_clean":"2024-06-10T00:53:22.121112+0000","last_became_active":"2024-06-10T00:52:35.959664+0000","last_became_peered":"2024-06-10T00:52:35.959664+0000","last_unstale":"2024-06-10T00:53:22.121112+0000","last_undegraded":"2024-06-10T00:53:22.121112+0000","last_fullsized":"2024-06-10T00:53:22.121112+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:41:58.086330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124294+0000","last_change":"2024-06-10T00:52:35.967043+0000","last_active":"2024-06-10T00:53:22.124294+0000","last_peered":"2024-06-10T00:53:22.124294+0000","last_clean":"2024-06-10T00:53:22.124294+0000","last_became_active":"2024-06-10T00:52:35.966366+0000","last_became_peered":"2024-06-10T00:52:35.966366+0000","last_unstale":"2024-06-10T00:53:22.124294+0000","last_undegraded":"2024-06-10T00:53:22.124294+0000","last_fullsized":"2024-06-10T00:53:22.124294+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:27:06.047113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120608+0000","last_change":"2024-06-10T00:52:35.960517+0000","last_active":"2024-06-10T00:53:22.120608+0000","last_peered":"2024-06-10T00:53:22.120608+0000","last_clean":"2024-06-10T00:53:22.120608+0000","last_became_active":"2024-06-10T00:52:35.957895+0000","last_became_peered":"2024-06-10T00:52:35.957895+0000","last_unstale":"2024-06-10T00:53:22.120608+0000","last_undegraded":"2024-06-10T00:53:22.120608+0000","last_fullsized":"2024-06-10T00:53:22.120608+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:04:34.438070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120365+0000","last_change":"2024-06-10T00:52:35.962163+0000","last_active":"2024-06-10T00:53:22.120365+0000","last_peered":"2024-06-10T00:53:22.120365+0000","last_clean":"2024-06-10T00:53:22.120365+0000","last_became_active":"2024-06-10T00:52:35.961768+0000","last_became_peered":"2024-06-10T00:52:35.961768+0000","last_unstale":"2024-06-10T00:53:22.120365+0000","last_undegraded":"2024-06-10T00:53:22.120365+0000","last_fullsized":"2024-06-10T00:53:22.120365+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:35.793995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":39,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.130686+0000","last_change":"2024-06-10T00:53:22.130686+0000","last_active":"2024-06-10T00:53:22.130686+0000","last_peered":"2024-06-10T00:53:22.130686+0000","last_clean":"2024-06-10T00:53:22.130686+0000","last_became_active":"2024-06-10T00:53:22.130198+0000","last_became_peered":"2024-06-10T00:53:22.130198+0000","last_unstale":"2024-06-10T00:53:22.130686+0000","last_undegraded":"2024-06-10T00:53:22.130686+0000","last_fullsized":"2024-06-10T00:53:22.130686+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:50:26.773977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.129145+0000","last_change":"2024-06-10T00:52:35.957255+0000","last_active":"2024-06-10T00:53:22.129145+0000","last_peered":"2024-06-10T00:53:22.129145+0000","last_clean":"2024-06-10T00:53:22.129145+0000","last_became_active":"2024-06-10T00:52:35.956923+0000","last_became_peered":"2024-06-10T00:52:35.956923+0000","last_unstale":"2024-06-10T00:53:22.129145+0000","last_undegraded":"2024-06-10T00:53:22.129145+0000","last_fullsized":"2024-06-10T00:53:22.129145+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:33:25.743503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.123595+0000","last_change":"2024-06-10T00:52:35.964258+0000","last_active":"2024-06-10T00:53:22.123595+0000","last_peered":"2024-06-10T00:53:22.123595+0000","last_clean":"2024-06-10T00:53:22.123595+0000","last_became_active":"2024-06-10T00:52:35.963862+0000","last_became_peered":"2024-06-10T00:52:35.963862+0000","last_unstale":"2024-06-10T00:53:22.123595+0000","last_undegraded":"2024-06-10T00:53:22.123595+0000","last_fullsized":"2024-06-10T00:53:22.123595+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:48:58.554411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120966+0000","last_change":"2024-06-10T00:52:35.965046+0000","last_active":"2024-06-10T00:53:22.120966+0000","last_peered":"2024-06-10T00:53:22.120966+0000","last_clean":"2024-06-10T00:53:22.120966+0000","last_became_active":"2024-06-10T00:52:35.964496+0000","last_became_peered":"2024-06-10T00:52:35.964496+0000","last_unstale":"2024-06-10T00:53:22.120966+0000","last_undegraded":"2024-06-10T00:53:22.120966+0000","last_fullsized":"2024-06-10T00:53:22.120966+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:09:42.979731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120807+0000","last_change":"2024-06-10T00:52:35.964287+0000","last_active":"2024-06-10T00:53:22.120807+0000","last_peered":"2024-06-10T00:53:22.120807+0000","last_clean":"2024-06-10T00:53:22.120807+0000","last_became_active":"2024-06-10T00:52:35.963920+0000","last_became_peered":"2024-06-10T00:52:35.963920+0000","last_unstale":"2024-06-10T00:53:22.120807+0000","last_undegraded":"2024-06-10T00:53:22.120807+0000","last_fullsized":"2024-06-10T00:53:22.120807+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:41:54.593274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.127245+0000","last_change":"2024-06-10T00:52:35.961440+0000","last_active":"2024-06-10T00:53:22.127245+0000","last_peered":"2024-06-10T00:53:22.127245+0000","last_clean":"2024-06-10T00:53:22.127245+0000","last_became_active":"2024-06-10T00:52:35.960066+0000","last_became_peered":"2024-06-10T00:52:35.960066+0000","last_unstale":"2024-06-10T00:53:22.127245+0000","last_undegraded":"2024-06-10T00:53:22.127245+0000","last_fullsized":"2024-06-10T00:53:22.127245+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:01.024596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120129+0000","last_change":"2024-06-10T00:52:35.964464+0000","last_active":"2024-06-10T00:53:22.120129+0000","last_peered":"2024-06-10T00:53:22.120129+0000","last_clean":"2024-06-10T00:53:22.120129+0000","last_became_active":"2024-06-10T00:52:35.964159+0000","last_became_peered":"2024-06-10T00:52:35.964159+0000","last_unstale":"2024-06-10T00:53:22.120129+0000","last_undegraded":"2024-06-10T00:53:22.120129+0000","last_fullsized":"2024-06-10T00:53:22.120129+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:18.660859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.198928+0000","last_change":"2024-06-10T00:52:35.961539+0000","last_active":"2024-06-10T00:53:22.198928+0000","last_peered":"2024-06-10T00:53:22.198928+0000","last_clean":"2024-06-10T00:53:22.198928+0000","last_became_active":"2024-06-10T00:52:35.958738+0000","last_became_peered":"2024-06-10T00:52:35.958738+0000","last_unstale":"2024-06-10T00:53:22.198928+0000","last_undegraded":"2024-06-10T00:53:22.198928+0000","last_fullsized":"2024-06-10T00:53:22.198928+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:06:48.057551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.128823+0000","last_change":"2024-06-10T00:52:35.962563+0000","last_active":"2024-06-10T00:53:22.128823+0000","last_peered":"2024-06-10T00:53:22.128823+0000","last_clean":"2024-06-10T00:53:22.128823+0000","last_became_active":"2024-06-10T00:52:35.962255+0000","last_became_peered":"2024-06-10T00:52:35.962255+0000","last_unstale":"2024-06-10T00:53:22.128823+0000","last_undegraded":"2024-06-10T00:53:22.128823+0000","last_fullsized":"2024-06-10T00:53:22.128823+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:33:35.689446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.123198+0000","last_change":"2024-06-10T00:52:35.959616+0000","last_active":"2024-06-10T00:53:22.123198+0000","last_peered":"2024-06-10T00:53:22.123198+0000","last_clean":"2024-06-10T00:53:22.123198+0000","last_became_active":"2024-06-10T00:52:35.959251+0000","last_became_peered":"2024-06-10T00:52:35.959251+0000","last_unstale":"2024-06-10T00:53:22.123198+0000","last_undegraded":"2024-06-10T00:53:22.123198+0000","last_fullsized":"2024-06-10T00:53:22.123198+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:39:23.933279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284806,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68876,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":167,"num_read_kb":160,"num_write":167,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":190220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351009,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":554058,"data_compressed":26474,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612647,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28212,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716940,"statfs":{"total":95995035648,"available":95966146560,"internally_reserved":0,"allocated":1224704,"data_stored":1206202,"data_compressed":62689,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776172,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1097728,"data_stored":558390,"data_compressed":26474,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037808,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":554214,"data_compressed":26474,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299446,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36452,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708700,"statfs":{"total":95995035648,"available":95957708800,"internally_reserved":0,"allocated":1142784,"data_stored":1150122,"data_compressed":62689,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462970,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":549470,"data_compressed":26474,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1134592,"data_stored":606052,"data_compressed":26474,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":550858,"data_compressed":26474,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32300,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712852,"statfs":{"total":95995035648,"available":95961960448,"internally_reserved":0,"allocated":1150976,"data_stored":607680,"data_compressed":26474,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149769,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":549664,"data_compressed":26474,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888132,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":554232,"data_compressed":26474,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724607,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1150976,"data_stored":1145710,"data_compressed":62689,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":58300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T00:54:08.245 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-10T00:54:08.497 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:08.725 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:08 smithi037 ceph-mon[32192]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:54:08.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:08 smithi167 ceph-mon[36917]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:54:08.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:08 smithi094 ceph-mon[36708]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:54:09.729 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:09 smithi037 ceph-mon[32192]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-06-10T00:54:09.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:09 smithi167 ceph-mon[36917]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-06-10T00:54:09.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:09 smithi094 ceph-mon[36708]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-06-10T00:54:10.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.170+0000 7f77aa1e5700 1 -- 172.21.15.37:0/2792074620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a41030f0 msgr2=0x7f77a4107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:10.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.170+0000 7f77aa1e5700 1 --2- 172.21.15.37:0/2792074620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a41030f0 0x7f77a4107110 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f77980099c0 tx=0x7f7798009cd0 comp rx=0 tx=0).stop 2024-06-10T00:54:10.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.170+0000 7f77aa1e5700 1 -- 172.21.15.37:0/2792074620 shutdown_connections 2024-06-10T00:54:10.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.170+0000 7f77aa1e5700 1 --2- 172.21.15.37:0/2792074620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a4107840 0x7f77a410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:10.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.170+0000 7f77aa1e5700 1 --2- 172.21.15.37:0/2792074620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77a4102740 0x7f77a4102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:10.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.170+0000 7f77aa1e5700 1 --2- 172.21.15.37:0/2792074620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a41030f0 0x7f77a4107110 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:10.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.170+0000 7f77aa1e5700 1 -- 172.21.15.37:0/2792074620 >> 172.21.15.37:0/2792074620 conn(0x7f77a40795f0 msgr2=0x7f77a4079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:10.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.170+0000 7f77aa1e5700 1 -- 172.21.15.37:0/2792074620 shutdown_connections 2024-06-10T00:54:10.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.171+0000 7f77aa1e5700 1 -- 172.21.15.37:0/2792074620 wait complete. 2024-06-10T00:54:10.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.171+0000 7f77aa1e5700 1 Processor -- start 2024-06-10T00:54:10.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.171+0000 7f77aa1e5700 1 -- start start 2024-06-10T00:54:10.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.172+0000 7f77aa1e5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a4102740 0x7f77a4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:10.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.172+0000 7f77aa1e5700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77a41030f0 0x7f77a4074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:10.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.172+0000 7f77aa1e5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a4107840 0x7f77a4075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:10.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.172+0000 7f77aa1e5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41127d0 con 0x7f77a4102740 2024-06-10T00:54:10.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.172+0000 7f77aa1e5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f77a4112650 con 0x7f77a4107840 2024-06-10T00:54:10.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.172+0000 7f77aa1e5700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f77a4112950 con 0x7f77a41030f0 2024-06-10T00:54:10.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.172+0000 7f77a37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a4102740 0x7f77a4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:10.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.172+0000 7f77a37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a4102740 0x7f77a4074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56138/0 (socket says 172.21.15.37:56138) 2024-06-10T00:54:10.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.172+0000 7f77a37fe700 1 -- 172.21.15.37:0/2049823825 learned_addr learned my addr 172.21.15.37:0/2049823825 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:54:10.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.172+0000 7f77a2ffd700 1 --2- 172.21.15.37:0/2049823825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77a41030f0 0x7f77a4074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:10.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.172+0000 7f77a3fff700 1 --2- 172.21.15.37:0/2049823825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a4107840 0x7f77a4075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:10.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.172+0000 7f77a37fe700 1 -- 172.21.15.37:0/2049823825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77a41030f0 msgr2=0x7f77a4074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:10.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.172+0000 7f77a37fe700 1 --2- 172.21.15.37:0/2049823825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77a41030f0 0x7f77a4074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:10.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.172+0000 7f77a37fe700 1 -- 172.21.15.37:0/2049823825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a4107840 msgr2=0x7f77a4075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:10.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.172+0000 7f77a37fe700 1 --2- 172.21.15.37:0/2049823825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a4107840 0x7f77a4075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:10.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.172+0000 7f77a37fe700 1 -- 172.21.15.37:0/2049823825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a41a7120 con 0x7f77a4102740 2024-06-10T00:54:10.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.172+0000 7f77a37fe700 1 --2- 172.21.15.37:0/2049823825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a4102740 0x7f77a4074790 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f779400d950 tx=0x7f779400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:10.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.172+0000 7f77a0ff9700 1 -- 172.21.15.37:0/2049823825 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794016070 con 0x7f77a4102740 2024-06-10T00:54:10.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.173+0000 7f77a0ff9700 1 -- 172.21.15.37:0/2049823825 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f779400fd70 con 0x7f77a4102740 2024-06-10T00:54:10.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.173+0000 7f77aa1e5700 1 -- 172.21.15.37:0/2049823825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a41a7410 con 0x7f77a4102740 2024-06-10T00:54:10.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.173+0000 7f77a0ff9700 1 -- 172.21.15.37:0/2049823825 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779400e610 con 0x7f77a4102740 2024-06-10T00:54:10.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.173+0000 7f77aa1e5700 1 -- 172.21.15.37:0/2049823825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a41a7970 con 0x7f77a4102740 2024-06-10T00:54:10.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.174+0000 7f77aa1e5700 1 -- 172.21.15.37:0/2049823825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77a4069180 con 0x7f77a4102740 2024-06-10T00:54:10.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.175+0000 7f77a0ff9700 1 -- 172.21.15.37:0/2049823825 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7794019070 con 0x7f77a4102740 2024-06-10T00:54:10.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.175+0000 7f77a0ff9700 1 --2- 172.21.15.37:0/2049823825 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f778c0a2d30 0x7f778c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:10.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.175+0000 7f77a2ffd700 1 --2- 172.21.15.37:0/2049823825 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f778c0a2d30 0x7f778c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:10.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.176+0000 7f77a0ff9700 1 -- 172.21.15.37:0/2049823825 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f77940cef60 con 0x7f77a4102740 2024-06-10T00:54:10.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.176+0000 7f77a2ffd700 1 --2- 172.21.15.37:0/2049823825 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f778c0a2d30 0x7f778c0a51f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f77a40761d0 tx=0x7f7798005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:10.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.180+0000 7f77a0ff9700 1 -- 172.21.15.37:0/2049823825 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f779409ba00 con 0x7f77a4102740 2024-06-10T00:54:10.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.381+0000 7f77aa1e5700 1 -- 172.21.15.37:0/2049823825 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f77a40628d0 con 0x7f778c0a2d30 2024-06-10T00:54:10.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.391+0000 7f77a0ff9700 1 -- 172.21.15.37:0/2049823825 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340633 (secure 0 0 0) 0x7f77a40628d0 con 0x7f778c0a2d30 2024-06-10T00:54:10.396 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:54:10.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.395+0000 7f77aa1e5700 1 -- 172.21.15.37:0/2049823825 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f778c0a2d30 msgr2=0x7f778c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:10.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.395+0000 7f77aa1e5700 1 --2- 172.21.15.37:0/2049823825 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f778c0a2d30 0x7f778c0a51f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f77a40761d0 tx=0x7f7798005fb0 comp rx=0 tx=0).stop 2024-06-10T00:54:10.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.395+0000 7f77aa1e5700 1 -- 172.21.15.37:0/2049823825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a4102740 msgr2=0x7f77a4074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:10.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.395+0000 7f77aa1e5700 1 --2- 172.21.15.37:0/2049823825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a4102740 0x7f77a4074790 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f779400d950 tx=0x7f779400dc60 comp rx=0 tx=0).stop 2024-06-10T00:54:10.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.395+0000 7f77aa1e5700 1 -- 172.21.15.37:0/2049823825 shutdown_connections 2024-06-10T00:54:10.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.395+0000 7f77aa1e5700 1 --2- 172.21.15.37:0/2049823825 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f778c0a2d30 0x7f778c0a51f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:10.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.395+0000 7f77aa1e5700 1 --2- 172.21.15.37:0/2049823825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a4107840 0x7f77a4075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:10.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.395+0000 7f77aa1e5700 1 --2- 172.21.15.37:0/2049823825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a4102740 0x7f77a4074790 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:10.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.395+0000 7f77aa1e5700 1 --2- 172.21.15.37:0/2049823825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77a41030f0 0x7f77a4074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:10.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.395+0000 7f77aa1e5700 1 -- 172.21.15.37:0/2049823825 >> 172.21.15.37:0/2049823825 conn(0x7f77a40795f0 msgr2=0x7f77a4101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:10.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.395+0000 7f77aa1e5700 1 -- 172.21.15.37:0/2049823825 shutdown_connections 2024-06-10T00:54:10.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:10.395+0000 7f77aa1e5700 1 -- 172.21.15.37:0/2049823825 wait complete. 2024-06-10T00:54:10.399 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-06-10T00:54:11.166 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2024-06-10T00:54:10.367751+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659478,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":308,"num_read_kb":274,"num_write":350,"num_write_kb":3060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":282,"ondisk_log_size":282,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370596,"kb_used_data":13124,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571228,"statfs":{"total":1151940427776,"available":1151560937472,"internally_reserved":0,"allocated":13438976,"data_stored":8603060,"data_compressed":426333,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17669,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":62,"num_write":17,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003302"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.123883+0000","last_change":"2024-06-10T00:52:35.971067+0000","last_active":"2024-06-10T00:53:22.123883+0000","last_peered":"2024-06-10T00:53:22.123883+0000","last_clean":"2024-06-10T00:53:22.123883+0000","last_became_active":"2024-06-10T00:52:35.970292+0000","last_became_peered":"2024-06-10T00:52:35.970292+0000","last_unstale":"2024-06-10T00:53:22.123883+0000","last_undegraded":"2024-06-10T00:53:22.123883+0000","last_fullsized":"2024-06-10T00:53:22.123883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:57:37.287238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.125927+0000","last_change":"2024-06-10T00:52:35.960421+0000","last_active":"2024-06-10T00:53:22.125927+0000","last_peered":"2024-06-10T00:53:22.125927+0000","last_clean":"2024-06-10T00:53:22.125927+0000","last_became_active":"2024-06-10T00:52:35.960070+0000","last_became_peered":"2024-06-10T00:52:35.960070+0000","last_unstale":"2024-06-10T00:53:22.125927+0000","last_undegraded":"2024-06-10T00:53:22.125927+0000","last_fullsized":"2024-06-10T00:53:22.125927+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:48:02.616434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.125113+0000","last_change":"2024-06-10T00:52:35.964778+0000","last_active":"2024-06-10T00:53:22.125113+0000","last_peered":"2024-06-10T00:53:22.125113+0000","last_clean":"2024-06-10T00:53:22.125113+0000","last_became_active":"2024-06-10T00:52:35.964463+0000","last_became_peered":"2024-06-10T00:52:35.964463+0000","last_unstale":"2024-06-10T00:53:22.125113+0000","last_undegraded":"2024-06-10T00:53:22.125113+0000","last_fullsized":"2024-06-10T00:53:22.125113+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:52:44.987363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.200376+0000","last_change":"2024-06-10T00:52:35.963226+0000","last_active":"2024-06-10T00:53:22.200376+0000","last_peered":"2024-06-10T00:53:22.200376+0000","last_clean":"2024-06-10T00:53:22.200376+0000","last_became_active":"2024-06-10T00:52:35.962862+0000","last_became_peered":"2024-06-10T00:52:35.962862+0000","last_unstale":"2024-06-10T00:53:22.200376+0000","last_undegraded":"2024-06-10T00:53:22.200376+0000","last_fullsized":"2024-06-10T00:53:22.200376+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:08:56.418071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124010+0000","last_change":"2024-06-10T00:52:35.957427+0000","last_active":"2024-06-10T00:53:22.124010+0000","last_peered":"2024-06-10T00:53:22.124010+0000","last_clean":"2024-06-10T00:53:22.124010+0000","last_became_active":"2024-06-10T00:52:35.957049+0000","last_became_peered":"2024-06-10T00:52:35.957049+0000","last_unstale":"2024-06-10T00:53:22.124010+0000","last_undegraded":"2024-06-10T00:53:22.124010+0000","last_fullsized":"2024-06-10T00:53:22.124010+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:16:04.257210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124433+0000","last_change":"2024-06-10T00:52:35.954251+0000","last_active":"2024-06-10T00:53:22.124433+0000","last_peered":"2024-06-10T00:53:22.124433+0000","last_clean":"2024-06-10T00:53:22.124433+0000","last_became_active":"2024-06-10T00:52:35.953908+0000","last_became_peered":"2024-06-10T00:52:35.953908+0000","last_unstale":"2024-06-10T00:53:22.124433+0000","last_undegraded":"2024-06-10T00:53:22.124433+0000","last_fullsized":"2024-06-10T00:53:22.124433+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:55:25.125836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120375+0000","last_change":"2024-06-10T00:52:35.956822+0000","last_active":"2024-06-10T00:53:22.120375+0000","last_peered":"2024-06-10T00:53:22.120375+0000","last_clean":"2024-06-10T00:53:22.120375+0000","last_became_active":"2024-06-10T00:52:35.956466+0000","last_became_peered":"2024-06-10T00:52:35.956466+0000","last_unstale":"2024-06-10T00:53:22.120375+0000","last_undegraded":"2024-06-10T00:53:22.120375+0000","last_fullsized":"2024-06-10T00:53:22.120375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:36:41.534392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120089+0000","last_change":"2024-06-10T00:52:35.962978+0000","last_active":"2024-06-10T00:53:22.120089+0000","last_peered":"2024-06-10T00:53:22.120089+0000","last_clean":"2024-06-10T00:53:22.120089+0000","last_became_active":"2024-06-10T00:52:35.962609+0000","last_became_peered":"2024-06-10T00:52:35.962609+0000","last_unstale":"2024-06-10T00:53:22.120089+0000","last_undegraded":"2024-06-10T00:53:22.120089+0000","last_fullsized":"2024-06-10T00:53:22.120089+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:20:57.469577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.199080+0000","last_change":"2024-06-10T00:52:35.963203+0000","last_active":"2024-06-10T00:53:22.199080+0000","last_peered":"2024-06-10T00:53:22.199080+0000","last_clean":"2024-06-10T00:53:22.199080+0000","last_became_active":"2024-06-10T00:52:35.962789+0000","last_became_peered":"2024-06-10T00:52:35.962789+0000","last_unstale":"2024-06-10T00:53:22.199080+0000","last_undegraded":"2024-06-10T00:53:22.199080+0000","last_fullsized":"2024-06-10T00:53:22.199080+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:00:17.819193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124909+0000","last_change":"2024-06-10T00:52:35.966940+0000","last_active":"2024-06-10T00:53:22.124909+0000","last_peered":"2024-06-10T00:53:22.124909+0000","last_clean":"2024-06-10T00:53:22.124909+0000","last_became_active":"2024-06-10T00:52:35.966554+0000","last_became_peered":"2024-06-10T00:52:35.966554+0000","last_unstale":"2024-06-10T00:53:22.124909+0000","last_undegraded":"2024-06-10T00:53:22.124909+0000","last_fullsized":"2024-06-10T00:53:22.124909+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:00.374437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120905+0000","last_change":"2024-06-10T00:52:35.960927+0000","last_active":"2024-06-10T00:53:22.120905+0000","last_peered":"2024-06-10T00:53:22.120905+0000","last_clean":"2024-06-10T00:53:22.120905+0000","last_became_active":"2024-06-10T00:52:35.960577+0000","last_became_peered":"2024-06-10T00:52:35.960577+0000","last_unstale":"2024-06-10T00:53:22.120905+0000","last_undegraded":"2024-06-10T00:53:22.120905+0000","last_fullsized":"2024-06-10T00:53:22.120905+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:50:02.366695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.200253+0000","last_change":"2024-06-10T00:52:35.964435+0000","last_active":"2024-06-10T00:53:22.200253+0000","last_peered":"2024-06-10T00:53:22.200253+0000","last_clean":"2024-06-10T00:53:22.200253+0000","last_became_active":"2024-06-10T00:52:35.964069+0000","last_became_peered":"2024-06-10T00:52:35.964069+0000","last_unstale":"2024-06-10T00:53:22.200253+0000","last_undegraded":"2024-06-10T00:53:22.200253+0000","last_fullsized":"2024-06-10T00:53:22.200253+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:06.708551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.128778+0000","last_change":"2024-06-10T00:52:35.966722+0000","last_active":"2024-06-10T00:53:22.128778+0000","last_peered":"2024-06-10T00:53:22.128778+0000","last_clean":"2024-06-10T00:53:22.128778+0000","last_became_active":"2024-06-10T00:52:35.966366+0000","last_became_peered":"2024-06-10T00:52:35.966366+0000","last_unstale":"2024-06-10T00:53:22.128778+0000","last_undegraded":"2024-06-10T00:53:22.128778+0000","last_fullsized":"2024-06-10T00:53:22.128778+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:57:46.021047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124780+0000","last_change":"2024-06-10T00:52:35.955307+0000","last_active":"2024-06-10T00:53:22.124780+0000","last_peered":"2024-06-10T00:53:22.124780+0000","last_clean":"2024-06-10T00:53:22.124780+0000","last_became_active":"2024-06-10T00:52:35.954140+0000","last_became_peered":"2024-06-10T00:52:35.954140+0000","last_unstale":"2024-06-10T00:53:22.124780+0000","last_undegraded":"2024-06-10T00:53:22.124780+0000","last_fullsized":"2024-06-10T00:53:22.124780+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:47:22.493598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.125049+0000","last_change":"2024-06-10T00:52:35.960592+0000","last_active":"2024-06-10T00:53:22.125049+0000","last_peered":"2024-06-10T00:53:22.125049+0000","last_clean":"2024-06-10T00:53:22.125049+0000","last_became_active":"2024-06-10T00:52:35.960263+0000","last_became_peered":"2024-06-10T00:52:35.960263+0000","last_unstale":"2024-06-10T00:53:22.125049+0000","last_undegraded":"2024-06-10T00:53:22.125049+0000","last_fullsized":"2024-06-10T00:53:22.125049+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:47:11.658666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.198195+0000","last_change":"2024-06-10T00:52:35.965796+0000","last_active":"2024-06-10T00:53:22.198195+0000","last_peered":"2024-06-10T00:53:22.198195+0000","last_clean":"2024-06-10T00:53:22.198195+0000","last_became_active":"2024-06-10T00:52:35.963965+0000","last_became_peered":"2024-06-10T00:52:35.963965+0000","last_unstale":"2024-06-10T00:53:22.198195+0000","last_undegraded":"2024-06-10T00:53:22.198195+0000","last_fullsized":"2024-06-10T00:53:22.198195+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:29:47.149852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.123532+0000","last_change":"2024-06-10T00:52:35.970714+0000","last_active":"2024-06-10T00:53:22.123532+0000","last_peered":"2024-06-10T00:53:22.123532+0000","last_clean":"2024-06-10T00:53:22.123532+0000","last_became_active":"2024-06-10T00:52:35.970173+0000","last_became_peered":"2024-06-10T00:52:35.970173+0000","last_unstale":"2024-06-10T00:53:22.123532+0000","last_undegraded":"2024-06-10T00:53:22.123532+0000","last_fullsized":"2024-06-10T00:53:22.123532+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:33:28.183156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124679+0000","last_change":"2024-06-10T00:52:35.963052+0000","last_active":"2024-06-10T00:53:22.124679+0000","last_peered":"2024-06-10T00:53:22.124679+0000","last_clean":"2024-06-10T00:53:22.124679+0000","last_became_active":"2024-06-10T00:52:35.962674+0000","last_became_peered":"2024-06-10T00:52:35.962674+0000","last_unstale":"2024-06-10T00:53:22.124679+0000","last_undegraded":"2024-06-10T00:53:22.124679+0000","last_fullsized":"2024-06-10T00:53:22.124679+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:56:38.836854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.130336+0000","last_change":"2024-06-10T00:53:22.130336+0000","last_active":"2024-06-10T00:53:22.130336+0000","last_peered":"2024-06-10T00:53:22.130336+0000","last_clean":"2024-06-10T00:53:22.130336+0000","last_became_active":"2024-06-10T00:53:22.129910+0000","last_became_peered":"2024-06-10T00:53:22.129910+0000","last_unstale":"2024-06-10T00:53:22.130336+0000","last_undegraded":"2024-06-10T00:53:22.130336+0000","last_fullsized":"2024-06-10T00:53:22.130336+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:23:50.600821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120406+0000","last_change":"2024-06-10T00:52:45.997276+0000","last_active":"2024-06-10T00:53:22.120406+0000","last_peered":"2024-06-10T00:53:22.120406+0000","last_clean":"2024-06-10T00:53:22.120406+0000","last_became_active":"2024-06-10T00:52:45.995638+0000","last_became_peered":"2024-06-10T00:52:45.995638+0000","last_unstale":"2024-06-10T00:53:22.120406+0000","last_undegraded":"2024-06-10T00:53:22.120406+0000","last_fullsized":"2024-06-10T00:53:22.120406+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:14:20.922348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:39.289847+0000","last_change":"2024-06-10T00:52:32.760888+0000","last_active":"2024-06-10T00:53:39.289847+0000","last_peered":"2024-06-10T00:53:39.289847+0000","last_clean":"2024-06-10T00:53:39.289847+0000","last_became_active":"2024-06-10T00:52:32.760531+0000","last_became_peered":"2024-06-10T00:52:32.760531+0000","last_unstale":"2024-06-10T00:53:39.289847+0000","last_undegraded":"2024-06-10T00:53:39.289847+0000","last_fullsized":"2024-06-10T00:53:39.289847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:05:59.052251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120410+0000","last_change":"2024-06-10T00:52:35.967626+0000","last_active":"2024-06-10T00:53:22.120410+0000","last_peered":"2024-06-10T00:53:22.120410+0000","last_clean":"2024-06-10T00:53:22.120410+0000","last_became_active":"2024-06-10T00:52:35.967295+0000","last_became_peered":"2024-06-10T00:52:35.967295+0000","last_unstale":"2024-06-10T00:53:22.120410+0000","last_undegraded":"2024-06-10T00:53:22.120410+0000","last_fullsized":"2024-06-10T00:53:22.120410+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:07:55.054327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.199614+0000","last_change":"2024-06-10T00:52:45.993974+0000","last_active":"2024-06-10T00:53:22.199614+0000","last_peered":"2024-06-10T00:53:22.199614+0000","last_clean":"2024-06-10T00:53:22.199614+0000","last_became_active":"2024-06-10T00:52:45.993242+0000","last_became_peered":"2024-06-10T00:52:45.993242+0000","last_unstale":"2024-06-10T00:53:22.199614+0000","last_undegraded":"2024-06-10T00:53:22.199614+0000","last_fullsized":"2024-06-10T00:53:22.199614+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:28:12.810368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120766+0000","last_change":"2024-06-10T00:52:32.959905+0000","last_active":"2024-06-10T00:53:22.120766+0000","last_peered":"2024-06-10T00:53:22.120766+0000","last_clean":"2024-06-10T00:53:22.120766+0000","last_became_active":"2024-06-10T00:52:32.959551+0000","last_became_peered":"2024-06-10T00:52:32.959551+0000","last_unstale":"2024-06-10T00:53:22.120766+0000","last_undegraded":"2024-06-10T00:53:22.120766+0000","last_fullsized":"2024-06-10T00:53:22.120766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:33:48.029129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.125237+0000","last_change":"2024-06-10T00:52:35.965790+0000","last_active":"2024-06-10T00:53:22.125237+0000","last_peered":"2024-06-10T00:53:22.125237+0000","last_clean":"2024-06-10T00:53:22.125237+0000","last_became_active":"2024-06-10T00:52:35.965475+0000","last_became_peered":"2024-06-10T00:52:35.965475+0000","last_unstale":"2024-06-10T00:53:22.125237+0000","last_undegraded":"2024-06-10T00:53:22.125237+0000","last_fullsized":"2024-06-10T00:53:22.125237+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:24:02.894411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.200194+0000","last_change":"2024-06-10T00:52:45.994464+0000","last_active":"2024-06-10T00:53:22.200194+0000","last_peered":"2024-06-10T00:53:22.200194+0000","last_clean":"2024-06-10T00:53:22.200194+0000","last_became_active":"2024-06-10T00:52:45.994022+0000","last_became_peered":"2024-06-10T00:52:45.994022+0000","last_unstale":"2024-06-10T00:53:22.200194+0000","last_undegraded":"2024-06-10T00:53:22.200194+0000","last_fullsized":"2024-06-10T00:53:22.200194+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:27:25.063074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:39.288755+0000","last_change":"2024-06-10T00:52:32.763885+0000","last_active":"2024-06-10T00:53:39.288755+0000","last_peered":"2024-06-10T00:53:39.288755+0000","last_clean":"2024-06-10T00:53:39.288755+0000","last_became_active":"2024-06-10T00:52:32.763606+0000","last_became_peered":"2024-06-10T00:52:32.763606+0000","last_unstale":"2024-06-10T00:53:39.288755+0000","last_undegraded":"2024-06-10T00:53:39.288755+0000","last_fullsized":"2024-06-10T00:53:39.288755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:01:58.161168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.198324+0000","last_change":"2024-06-10T00:52:35.966948+0000","last_active":"2024-06-10T00:53:22.198324+0000","last_peered":"2024-06-10T00:53:22.198324+0000","last_clean":"2024-06-10T00:53:22.198324+0000","last_became_active":"2024-06-10T00:52:35.966345+0000","last_became_peered":"2024-06-10T00:52:35.966345+0000","last_unstale":"2024-06-10T00:53:22.198324+0000","last_undegraded":"2024-06-10T00:53:22.198324+0000","last_fullsized":"2024-06-10T00:53:22.198324+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:58:27.169201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.199805+0000","last_change":"2024-06-10T00:52:45.993645+0000","last_active":"2024-06-10T00:53:22.199805+0000","last_peered":"2024-06-10T00:53:22.199805+0000","last_clean":"2024-06-10T00:53:22.199805+0000","last_became_active":"2024-06-10T00:52:45.993183+0000","last_became_peered":"2024-06-10T00:52:45.993183+0000","last_unstale":"2024-06-10T00:53:22.199805+0000","last_undegraded":"2024-06-10T00:53:22.199805+0000","last_fullsized":"2024-06-10T00:53:22.199805+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:47:00.541427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.122778+0000","last_change":"2024-06-10T00:52:32.761110+0000","last_active":"2024-06-10T00:53:22.122778+0000","last_peered":"2024-06-10T00:53:22.122778+0000","last_clean":"2024-06-10T00:53:22.122778+0000","last_became_active":"2024-06-10T00:52:32.760764+0000","last_became_peered":"2024-06-10T00:52:32.760764+0000","last_unstale":"2024-06-10T00:53:22.122778+0000","last_undegraded":"2024-06-10T00:53:22.122778+0000","last_fullsized":"2024-06-10T00:53:22.122778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:38:17.022460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":39,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.199668+0000","last_change":"2024-06-10T00:53:22.199668+0000","last_active":"2024-06-10T00:53:22.199668+0000","last_peered":"2024-06-10T00:53:22.199668+0000","last_clean":"2024-06-10T00:53:22.199668+0000","last_became_active":"2024-06-10T00:53:22.199329+0000","last_became_peered":"2024-06-10T00:53:22.199329+0000","last_unstale":"2024-06-10T00:53:22.199668+0000","last_undegraded":"2024-06-10T00:53:22.199668+0000","last_fullsized":"2024-06-10T00:53:22.199668+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:37.640478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120808+0000","last_change":"2024-06-10T00:52:45.995767+0000","last_active":"2024-06-10T00:53:22.120808+0000","last_peered":"2024-06-10T00:53:22.120808+0000","last_clean":"2024-06-10T00:53:22.120808+0000","last_became_active":"2024-06-10T00:52:45.994659+0000","last_became_peered":"2024-06-10T00:52:45.994659+0000","last_unstale":"2024-06-10T00:53:22.120808+0000","last_undegraded":"2024-06-10T00:53:22.120808+0000","last_fullsized":"2024-06-10T00:53:22.120808+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:03:45.652248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"98'6","reported_seq":73,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:54:05.335631+0000","last_change":"2024-06-10T00:52:32.761637+0000","last_active":"2024-06-10T00:54:05.335631+0000","last_peered":"2024-06-10T00:54:05.335631+0000","last_clean":"2024-06-10T00:54:05.335631+0000","last_became_active":"2024-06-10T00:52:32.761144+0000","last_became_peered":"2024-06-10T00:52:32.761144+0000","last_unstale":"2024-06-10T00:54:05.335631+0000","last_undegraded":"2024-06-10T00:54:05.335631+0000","last_fullsized":"2024-06-10T00:54:05.335631+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:55:05.616564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124405+0000","last_change":"2024-06-10T00:52:35.962683+0000","last_active":"2024-06-10T00:53:22.124405+0000","last_peered":"2024-06-10T00:53:22.124405+0000","last_clean":"2024-06-10T00:53:22.124405+0000","last_became_active":"2024-06-10T00:52:35.962327+0000","last_became_peered":"2024-06-10T00:52:35.962327+0000","last_unstale":"2024-06-10T00:53:22.124405+0000","last_undegraded":"2024-06-10T00:53:22.124405+0000","last_fullsized":"2024-06-10T00:53:22.124405+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:38:19.063306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120676+0000","last_change":"2024-06-10T00:52:45.996882+0000","last_active":"2024-06-10T00:53:22.120676+0000","last_peered":"2024-06-10T00:53:22.120676+0000","last_clean":"2024-06-10T00:53:22.120676+0000","last_became_active":"2024-06-10T00:52:45.995767+0000","last_became_peered":"2024-06-10T00:52:45.995767+0000","last_unstale":"2024-06-10T00:53:22.120676+0000","last_undegraded":"2024-06-10T00:53:22.120676+0000","last_fullsized":"2024-06-10T00:53:22.120676+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:28:03.766944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":49,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:39.289504+0000","last_change":"2024-06-10T00:53:22.200546+0000","last_active":"2024-06-10T00:53:39.289504+0000","last_peered":"2024-06-10T00:53:39.289504+0000","last_clean":"2024-06-10T00:53:39.289504+0000","last_became_active":"2024-06-10T00:53:22.200059+0000","last_became_peered":"2024-06-10T00:53:22.200059+0000","last_unstale":"2024-06-10T00:53:39.289504+0000","last_undegraded":"2024-06-10T00:53:39.289504+0000","last_fullsized":"2024-06-10T00:53:39.289504+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:08:47.682594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.127463+0000","last_change":"2024-06-10T00:52:35.963105+0000","last_active":"2024-06-10T00:53:22.127463+0000","last_peered":"2024-06-10T00:53:22.127463+0000","last_clean":"2024-06-10T00:53:22.127463+0000","last_became_active":"2024-06-10T00:52:35.962627+0000","last_became_peered":"2024-06-10T00:52:35.962627+0000","last_unstale":"2024-06-10T00:53:22.127463+0000","last_undegraded":"2024-06-10T00:53:22.127463+0000","last_fullsized":"2024-06-10T00:53:22.127463+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:55:51.231795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.123318+0000","last_change":"2024-06-10T00:52:45.992557+0000","last_active":"2024-06-10T00:53:22.123318+0000","last_peered":"2024-06-10T00:53:22.123318+0000","last_clean":"2024-06-10T00:53:22.123318+0000","last_became_active":"2024-06-10T00:52:45.992166+0000","last_became_peered":"2024-06-10T00:52:45.992166+0000","last_unstale":"2024-06-10T00:53:22.123318+0000","last_undegraded":"2024-06-10T00:53:22.123318+0000","last_fullsized":"2024-06-10T00:53:22.123318+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:49:22.368409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:39.287388+0000","last_change":"2024-06-10T00:52:32.751867+0000","last_active":"2024-06-10T00:53:39.287388+0000","last_peered":"2024-06-10T00:53:39.287388+0000","last_clean":"2024-06-10T00:53:39.287388+0000","last_became_active":"2024-06-10T00:52:32.751516+0000","last_became_peered":"2024-06-10T00:52:32.751516+0000","last_unstale":"2024-06-10T00:53:39.287388+0000","last_undegraded":"2024-06-10T00:53:39.287388+0000","last_fullsized":"2024-06-10T00:53:39.287388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:37:55.920613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.119900+0000","last_change":"2024-06-10T00:52:35.957772+0000","last_active":"2024-06-10T00:53:22.119900+0000","last_peered":"2024-06-10T00:53:22.119900+0000","last_clean":"2024-06-10T00:53:22.119900+0000","last_became_active":"2024-06-10T00:52:35.957470+0000","last_became_peered":"2024-06-10T00:52:35.957470+0000","last_unstale":"2024-06-10T00:53:22.119900+0000","last_undegraded":"2024-06-10T00:53:22.119900+0000","last_fullsized":"2024-06-10T00:53:22.119900+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:16:43.645292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.125047+0000","last_change":"2024-06-10T00:52:45.996807+0000","last_active":"2024-06-10T00:53:22.125047+0000","last_peered":"2024-06-10T00:53:22.125047+0000","last_clean":"2024-06-10T00:53:22.125047+0000","last_became_active":"2024-06-10T00:52:45.996378+0000","last_became_peered":"2024-06-10T00:52:45.996378+0000","last_unstale":"2024-06-10T00:53:22.125047+0000","last_undegraded":"2024-06-10T00:53:22.125047+0000","last_fullsized":"2024-06-10T00:53:22.125047+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:40:54.251790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'1","reported_seq":39,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.127379+0000","last_change":"2024-06-10T00:52:32.759625+0000","last_active":"2024-06-10T00:53:22.127379+0000","last_peered":"2024-06-10T00:53:22.127379+0000","last_clean":"2024-06-10T00:53:22.127379+0000","last_became_active":"2024-06-10T00:52:32.759299+0000","last_became_peered":"2024-06-10T00:52:32.759299+0000","last_unstale":"2024-06-10T00:53:22.127379+0000","last_undegraded":"2024-06-10T00:53:22.127379+0000","last_fullsized":"2024-06-10T00:53:22.127379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:11:44.966453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.123037+0000","last_change":"2024-06-10T00:52:35.957672+0000","last_active":"2024-06-10T00:53:22.123037+0000","last_peered":"2024-06-10T00:53:22.123037+0000","last_clean":"2024-06-10T00:53:22.123037+0000","last_became_active":"2024-06-10T00:52:35.957328+0000","last_became_peered":"2024-06-10T00:52:35.957328+0000","last_unstale":"2024-06-10T00:53:22.123037+0000","last_undegraded":"2024-06-10T00:53:22.123037+0000","last_fullsized":"2024-06-10T00:53:22.123037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:43:59.658178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.119965+0000","last_change":"2024-06-10T00:52:45.996677+0000","last_active":"2024-06-10T00:53:22.119965+0000","last_peered":"2024-06-10T00:53:22.119965+0000","last_clean":"2024-06-10T00:53:22.119965+0000","last_became_active":"2024-06-10T00:52:45.994483+0000","last_became_peered":"2024-06-10T00:52:45.994483+0000","last_unstale":"2024-06-10T00:53:22.119965+0000","last_undegraded":"2024-06-10T00:53:22.119965+0000","last_fullsized":"2024-06-10T00:53:22.119965+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:03:28.712072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"98'3","reported_seq":68,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:54:04.333495+0000","last_change":"2024-06-10T00:52:32.754829+0000","last_active":"2024-06-10T00:54:04.333495+0000","last_peered":"2024-06-10T00:54:04.333495+0000","last_clean":"2024-06-10T00:54:04.333495+0000","last_became_active":"2024-06-10T00:52:32.754386+0000","last_became_peered":"2024-06-10T00:52:32.754386+0000","last_unstale":"2024-06-10T00:54:04.333495+0000","last_undegraded":"2024-06-10T00:54:04.333495+0000","last_fullsized":"2024-06-10T00:54:04.333495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:00.870324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.199229+0000","last_change":"2024-06-10T00:52:35.966871+0000","last_active":"2024-06-10T00:53:22.199229+0000","last_peered":"2024-06-10T00:53:22.199229+0000","last_clean":"2024-06-10T00:53:22.199229+0000","last_became_active":"2024-06-10T00:52:35.966492+0000","last_became_peered":"2024-06-10T00:52:35.966492+0000","last_unstale":"2024-06-10T00:53:22.199229+0000","last_undegraded":"2024-06-10T00:53:22.199229+0000","last_fullsized":"2024-06-10T00:53:22.199229+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:24:41.048935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124082+0000","last_change":"2024-06-10T00:52:45.998906+0000","last_active":"2024-06-10T00:53:22.124082+0000","last_peered":"2024-06-10T00:53:22.124082+0000","last_clean":"2024-06-10T00:53:22.124082+0000","last_became_active":"2024-06-10T00:52:45.998527+0000","last_became_peered":"2024-06-10T00:52:45.998527+0000","last_unstale":"2024-06-10T00:53:22.124082+0000","last_undegraded":"2024-06-10T00:53:22.124082+0000","last_fullsized":"2024-06-10T00:53:22.124082+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:46:15.986497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'2","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:38.283516+0000","last_change":"2024-06-10T00:52:32.765794+0000","last_active":"2024-06-10T00:53:38.283516+0000","last_peered":"2024-06-10T00:53:38.283516+0000","last_clean":"2024-06-10T00:53:38.283516+0000","last_became_active":"2024-06-10T00:52:32.765276+0000","last_became_peered":"2024-06-10T00:52:32.765276+0000","last_unstale":"2024-06-10T00:53:38.283516+0000","last_undegraded":"2024-06-10T00:53:38.283516+0000","last_fullsized":"2024-06-10T00:53:38.283516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:06:51.209012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.125460+0000","last_change":"2024-06-10T00:52:35.954897+0000","last_active":"2024-06-10T00:53:22.125460+0000","last_peered":"2024-06-10T00:53:22.125460+0000","last_clean":"2024-06-10T00:53:22.125460+0000","last_became_active":"2024-06-10T00:52:35.954565+0000","last_became_peered":"2024-06-10T00:52:35.954565+0000","last_unstale":"2024-06-10T00:53:22.125460+0000","last_undegraded":"2024-06-10T00:53:22.125460+0000","last_fullsized":"2024-06-10T00:53:22.125460+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:39:26.702804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.200616+0000","last_change":"2024-06-10T00:52:45.992194+0000","last_active":"2024-06-10T00:53:22.200616+0000","last_peered":"2024-06-10T00:53:22.200616+0000","last_clean":"2024-06-10T00:53:22.200616+0000","last_became_active":"2024-06-10T00:52:45.991453+0000","last_became_peered":"2024-06-10T00:52:45.991453+0000","last_unstale":"2024-06-10T00:53:22.200616+0000","last_undegraded":"2024-06-10T00:53:22.200616+0000","last_fullsized":"2024-06-10T00:53:22.200616+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:11:10.775610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'4","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:38.282608+0000","last_change":"2024-06-10T00:52:32.962214+0000","last_active":"2024-06-10T00:53:38.282608+0000","last_peered":"2024-06-10T00:53:38.282608+0000","last_clean":"2024-06-10T00:53:38.282608+0000","last_became_active":"2024-06-10T00:52:32.961835+0000","last_became_peered":"2024-06-10T00:52:32.961835+0000","last_unstale":"2024-06-10T00:53:38.282608+0000","last_undegraded":"2024-06-10T00:53:38.282608+0000","last_fullsized":"2024-06-10T00:53:38.282608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:36:44.724122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.198466+0000","last_change":"2024-06-10T00:52:35.963413+0000","last_active":"2024-06-10T00:53:22.198466+0000","last_peered":"2024-06-10T00:53:22.198466+0000","last_clean":"2024-06-10T00:53:22.198466+0000","last_became_active":"2024-06-10T00:52:35.962922+0000","last_became_peered":"2024-06-10T00:52:35.962922+0000","last_unstale":"2024-06-10T00:53:22.198466+0000","last_undegraded":"2024-06-10T00:53:22.198466+0000","last_fullsized":"2024-06-10T00:53:22.198466+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:05:32.164696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.123633+0000","last_change":"2024-06-10T00:52:45.997601+0000","last_active":"2024-06-10T00:53:22.123633+0000","last_peered":"2024-06-10T00:53:22.123633+0000","last_clean":"2024-06-10T00:53:22.123633+0000","last_became_active":"2024-06-10T00:52:45.997231+0000","last_became_peered":"2024-06-10T00:52:45.997231+0000","last_unstale":"2024-06-10T00:53:22.123633+0000","last_undegraded":"2024-06-10T00:53:22.123633+0000","last_fullsized":"2024-06-10T00:53:22.123633+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:52:30.994486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"98'34","reported_seq":132,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:54:05.340885+0000","last_change":"2024-06-10T00:52:32.761592+0000","last_active":"2024-06-10T00:54:05.340885+0000","last_peered":"2024-06-10T00:54:05.340885+0000","last_clean":"2024-06-10T00:54:05.340885+0000","last_became_active":"2024-06-10T00:52:32.761169+0000","last_became_peered":"2024-06-10T00:52:32.761169+0000","last_unstale":"2024-06-10T00:54:05.340885+0000","last_undegraded":"2024-06-10T00:54:05.340885+0000","last_fullsized":"2024-06-10T00:54:05.340885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:08:33.015035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64876,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":58,"num_read_kb":94,"num_write":35,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124609+0000","last_change":"2024-06-10T00:52:35.955304+0000","last_active":"2024-06-10T00:53:22.124609+0000","last_peered":"2024-06-10T00:53:22.124609+0000","last_clean":"2024-06-10T00:53:22.124609+0000","last_became_active":"2024-06-10T00:52:35.954949+0000","last_became_peered":"2024-06-10T00:52:35.954949+0000","last_unstale":"2024-06-10T00:53:22.124609+0000","last_undegraded":"2024-06-10T00:53:22.124609+0000","last_fullsized":"2024-06-10T00:53:22.124609+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:56:42.890022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.128098+0000","last_change":"2024-06-10T00:52:45.992974+0000","last_active":"2024-06-10T00:53:22.128098+0000","last_peered":"2024-06-10T00:53:22.128098+0000","last_clean":"2024-06-10T00:53:22.128098+0000","last_became_active":"2024-06-10T00:52:45.992490+0000","last_became_peered":"2024-06-10T00:52:45.992490+0000","last_unstale":"2024-06-10T00:53:22.128098+0000","last_undegraded":"2024-06-10T00:53:22.128098+0000","last_fullsized":"2024-06-10T00:53:22.128098+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:03:45.543476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:39.289638+0000","last_change":"2024-06-10T00:52:32.762347+0000","last_active":"2024-06-10T00:53:39.289638+0000","last_peered":"2024-06-10T00:53:39.289638+0000","last_clean":"2024-06-10T00:53:39.289638+0000","last_became_active":"2024-06-10T00:52:32.761049+0000","last_became_peered":"2024-06-10T00:52:32.761049+0000","last_unstale":"2024-06-10T00:53:39.289638+0000","last_undegraded":"2024-06-10T00:53:39.289638+0000","last_fullsized":"2024-06-10T00:53:39.289638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:29:38.384451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.128558+0000","last_change":"2024-06-10T00:52:35.961299+0000","last_active":"2024-06-10T00:53:22.128558+0000","last_peered":"2024-06-10T00:53:22.128558+0000","last_clean":"2024-06-10T00:53:22.128558+0000","last_became_active":"2024-06-10T00:52:35.960966+0000","last_became_peered":"2024-06-10T00:52:35.960966+0000","last_unstale":"2024-06-10T00:53:22.128558+0000","last_undegraded":"2024-06-10T00:53:22.128558+0000","last_fullsized":"2024-06-10T00:53:22.128558+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:32:21.269689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.122972+0000","last_change":"2024-06-10T00:52:46.002091+0000","last_active":"2024-06-10T00:53:22.122972+0000","last_peered":"2024-06-10T00:53:22.122972+0000","last_clean":"2024-06-10T00:53:22.122972+0000","last_became_active":"2024-06-10T00:52:46.001677+0000","last_became_peered":"2024-06-10T00:52:46.001677+0000","last_unstale":"2024-06-10T00:53:22.122972+0000","last_undegraded":"2024-06-10T00:53:22.122972+0000","last_fullsized":"2024-06-10T00:53:22.122972+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:16:56.405720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'3","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:38.284301+0000","last_change":"2024-06-10T00:52:32.756765+0000","last_active":"2024-06-10T00:53:38.284301+0000","last_peered":"2024-06-10T00:53:38.284301+0000","last_clean":"2024-06-10T00:53:38.284301+0000","last_became_active":"2024-06-10T00:52:32.756470+0000","last_became_peered":"2024-06-10T00:52:32.756470+0000","last_unstale":"2024-06-10T00:53:38.284301+0000","last_undegraded":"2024-06-10T00:53:38.284301+0000","last_fullsized":"2024-06-10T00:53:38.284301+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:29:24.854913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120568+0000","last_change":"2024-06-10T00:52:35.960693+0000","last_active":"2024-06-10T00:53:22.120568+0000","last_peered":"2024-06-10T00:53:22.120568+0000","last_clean":"2024-06-10T00:53:22.120568+0000","last_became_active":"2024-06-10T00:52:35.958397+0000","last_became_peered":"2024-06-10T00:52:35.958397+0000","last_unstale":"2024-06-10T00:53:22.120568+0000","last_undegraded":"2024-06-10T00:53:22.120568+0000","last_fullsized":"2024-06-10T00:53:22.120568+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:23:47.556462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.123769+0000","last_change":"2024-06-10T00:52:46.001475+0000","last_active":"2024-06-10T00:53:22.123769+0000","last_peered":"2024-06-10T00:53:22.123769+0000","last_clean":"2024-06-10T00:53:22.123769+0000","last_became_active":"2024-06-10T00:52:46.000983+0000","last_became_peered":"2024-06-10T00:52:46.000983+0000","last_unstale":"2024-06-10T00:53:22.123769+0000","last_undegraded":"2024-06-10T00:53:22.123769+0000","last_fullsized":"2024-06-10T00:53:22.123769+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:23:56.285356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"98'3","reported_seq":71,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:54:06.336699+0000","last_change":"2024-06-10T00:52:32.756665+0000","last_active":"2024-06-10T00:54:06.336699+0000","last_peered":"2024-06-10T00:54:06.336699+0000","last_clean":"2024-06-10T00:54:06.336699+0000","last_became_active":"2024-06-10T00:52:32.756221+0000","last_became_peered":"2024-06-10T00:52:32.756221+0000","last_unstale":"2024-06-10T00:54:06.336699+0000","last_undegraded":"2024-06-10T00:54:06.336699+0000","last_fullsized":"2024-06-10T00:54:06.336699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:50:34.789063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":39,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.125844+0000","last_change":"2024-06-10T00:53:22.125844+0000","last_active":"2024-06-10T00:53:22.125844+0000","last_peered":"2024-06-10T00:53:22.125844+0000","last_clean":"2024-06-10T00:53:22.125844+0000","last_became_active":"2024-06-10T00:53:22.125449+0000","last_became_peered":"2024-06-10T00:53:22.125449+0000","last_unstale":"2024-06-10T00:53:22.125844+0000","last_undegraded":"2024-06-10T00:53:22.125844+0000","last_fullsized":"2024-06-10T00:53:22.125844+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:08:42.936702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.128988+0000","last_change":"2024-06-10T00:52:45.999300+0000","last_active":"2024-06-10T00:53:22.128988+0000","last_peered":"2024-06-10T00:53:22.128988+0000","last_clean":"2024-06-10T00:53:22.128988+0000","last_became_active":"2024-06-10T00:52:45.998860+0000","last_became_peered":"2024-06-10T00:52:45.998860+0000","last_unstale":"2024-06-10T00:53:22.128988+0000","last_undegraded":"2024-06-10T00:53:22.128988+0000","last_fullsized":"2024-06-10T00:53:22.128988+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:14:24.936870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":70,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:39.289823+0000","last_change":"2024-06-10T00:53:22.210710+0000","last_active":"2024-06-10T00:53:39.289823+0000","last_peered":"2024-06-10T00:53:39.289823+0000","last_clean":"2024-06-10T00:53:39.289823+0000","last_became_active":"2024-06-10T00:53:22.202461+0000","last_became_peered":"2024-06-10T00:53:22.202461+0000","last_unstale":"2024-06-10T00:53:39.289823+0000","last_undegraded":"2024-06-10T00:53:39.289823+0000","last_fullsized":"2024-06-10T00:53:39.289823+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:28:14.671776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.127229+0000","last_change":"2024-06-10T00:52:35.966960+0000","last_active":"2024-06-10T00:53:22.127229+0000","last_peered":"2024-06-10T00:53:22.127229+0000","last_clean":"2024-06-10T00:53:22.127229+0000","last_became_active":"2024-06-10T00:52:35.966630+0000","last_became_peered":"2024-06-10T00:52:35.966630+0000","last_unstale":"2024-06-10T00:53:22.127229+0000","last_undegraded":"2024-06-10T00:53:22.127229+0000","last_fullsized":"2024-06-10T00:53:22.127229+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:04.117405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.127174+0000","last_change":"2024-06-10T00:52:45.992955+0000","last_active":"2024-06-10T00:53:22.127174+0000","last_peered":"2024-06-10T00:53:22.127174+0000","last_clean":"2024-06-10T00:53:22.127174+0000","last_became_active":"2024-06-10T00:52:45.992457+0000","last_became_peered":"2024-06-10T00:52:45.992457+0000","last_unstale":"2024-06-10T00:53:22.127174+0000","last_undegraded":"2024-06-10T00:53:22.127174+0000","last_fullsized":"2024-06-10T00:53:22.127174+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:13:49.491183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:41.285013+0000","last_change":"2024-06-10T00:52:32.763794+0000","last_active":"2024-06-10T00:53:41.285013+0000","last_peered":"2024-06-10T00:53:41.285013+0000","last_clean":"2024-06-10T00:53:41.285013+0000","last_became_active":"2024-06-10T00:52:32.763406+0000","last_became_peered":"2024-06-10T00:52:32.763406+0000","last_unstale":"2024-06-10T00:53:41.285013+0000","last_undegraded":"2024-06-10T00:53:41.285013+0000","last_fullsized":"2024-06-10T00:53:41.285013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:50:33.580163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.130891+0000","last_change":"2024-06-10T00:53:22.130891+0000","last_active":"2024-06-10T00:53:22.130891+0000","last_peered":"2024-06-10T00:53:22.130891+0000","last_clean":"2024-06-10T00:53:22.130891+0000","last_became_active":"2024-06-10T00:53:22.130508+0000","last_became_peered":"2024-06-10T00:53:22.130508+0000","last_unstale":"2024-06-10T00:53:22.130891+0000","last_undegraded":"2024-06-10T00:53:22.130891+0000","last_fullsized":"2024-06-10T00:53:22.130891+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:28:01.919825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124284+0000","last_change":"2024-06-10T00:52:46.241698+0000","last_active":"2024-06-10T00:53:22.124284+0000","last_peered":"2024-06-10T00:53:22.124284+0000","last_clean":"2024-06-10T00:53:22.124284+0000","last_became_active":"2024-06-10T00:52:46.241257+0000","last_became_peered":"2024-06-10T00:52:46.241257+0000","last_unstale":"2024-06-10T00:53:22.124284+0000","last_undegraded":"2024-06-10T00:53:22.124284+0000","last_fullsized":"2024-06-10T00:53:22.124284+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:17:34.137750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124115+0000","last_change":"2024-06-10T00:52:32.960515+0000","last_active":"2024-06-10T00:53:22.124115+0000","last_peered":"2024-06-10T00:53:22.124115+0000","last_clean":"2024-06-10T00:53:22.124115+0000","last_became_active":"2024-06-10T00:52:32.960163+0000","last_became_peered":"2024-06-10T00:52:32.960163+0000","last_unstale":"2024-06-10T00:53:22.124115+0000","last_undegraded":"2024-06-10T00:53:22.124115+0000","last_fullsized":"2024-06-10T00:53:22.124115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:39:51.374998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124440+0000","last_change":"2024-06-10T00:52:35.961884+0000","last_active":"2024-06-10T00:53:22.124440+0000","last_peered":"2024-06-10T00:53:22.124440+0000","last_clean":"2024-06-10T00:53:22.124440+0000","last_became_active":"2024-06-10T00:52:35.961497+0000","last_became_peered":"2024-06-10T00:52:35.961497+0000","last_unstale":"2024-06-10T00:53:22.124440+0000","last_undegraded":"2024-06-10T00:53:22.124440+0000","last_fullsized":"2024-06-10T00:53:22.124440+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:16:09.284184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.128831+0000","last_change":"2024-06-10T00:52:45.990846+0000","last_active":"2024-06-10T00:53:22.128831+0000","last_peered":"2024-06-10T00:53:22.128831+0000","last_clean":"2024-06-10T00:53:22.128831+0000","last_became_active":"2024-06-10T00:52:45.990441+0000","last_became_peered":"2024-06-10T00:52:45.990441+0000","last_unstale":"2024-06-10T00:53:22.128831+0000","last_undegraded":"2024-06-10T00:53:22.128831+0000","last_fullsized":"2024-06-10T00:53:22.128831+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:19:35.690378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120615+0000","last_change":"2024-06-10T00:52:32.753913+0000","last_active":"2024-06-10T00:53:22.120615+0000","last_peered":"2024-06-10T00:53:22.120615+0000","last_clean":"2024-06-10T00:53:22.120615+0000","last_became_active":"2024-06-10T00:52:32.753174+0000","last_became_peered":"2024-06-10T00:52:32.753174+0000","last_unstale":"2024-06-10T00:53:22.120615+0000","last_undegraded":"2024-06-10T00:53:22.120615+0000","last_fullsized":"2024-06-10T00:53:22.120615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:13:20.048635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124179+0000","last_change":"2024-06-10T00:52:35.965783+0000","last_active":"2024-06-10T00:53:22.124179+0000","last_peered":"2024-06-10T00:53:22.124179+0000","last_clean":"2024-06-10T00:53:22.124179+0000","last_became_active":"2024-06-10T00:52:35.965432+0000","last_became_peered":"2024-06-10T00:52:35.965432+0000","last_unstale":"2024-06-10T00:53:22.124179+0000","last_undegraded":"2024-06-10T00:53:22.124179+0000","last_fullsized":"2024-06-10T00:53:22.124179+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:52:01.240318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.128810+0000","last_change":"2024-06-10T00:52:46.241662+0000","last_active":"2024-06-10T00:53:22.128810+0000","last_peered":"2024-06-10T00:53:22.128810+0000","last_clean":"2024-06-10T00:53:22.128810+0000","last_became_active":"2024-06-10T00:52:46.241211+0000","last_became_peered":"2024-06-10T00:52:46.241211+0000","last_unstale":"2024-06-10T00:53:22.128810+0000","last_undegraded":"2024-06-10T00:53:22.128810+0000","last_fullsized":"2024-06-10T00:53:22.128810+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:12.041290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.201271+0000","last_change":"2024-06-10T00:53:22.200590+0000","last_active":"2024-06-10T00:53:22.201271+0000","last_peered":"2024-06-10T00:53:22.201271+0000","last_clean":"2024-06-10T00:53:22.201271+0000","last_became_active":"2024-06-10T00:53:22.200046+0000","last_became_peered":"2024-06-10T00:53:22.200046+0000","last_unstale":"2024-06-10T00:53:22.201271+0000","last_undegraded":"2024-06-10T00:53:22.201271+0000","last_fullsized":"2024-06-10T00:53:22.201271+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:43:22.378316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"98'4","reported_seq":70,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:54:06.336202+0000","last_change":"2024-06-10T00:52:32.959904+0000","last_active":"2024-06-10T00:54:06.336202+0000","last_peered":"2024-06-10T00:54:06.336202+0000","last_clean":"2024-06-10T00:54:06.336202+0000","last_became_active":"2024-06-10T00:52:32.959506+0000","last_became_peered":"2024-06-10T00:52:32.959506+0000","last_unstale":"2024-06-10T00:54:06.336202+0000","last_undegraded":"2024-06-10T00:54:06.336202+0000","last_fullsized":"2024-06-10T00:54:06.336202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:23:15.950345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":23,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120769+0000","last_change":"2024-06-10T00:52:45.993305+0000","last_active":"2024-06-10T00:53:22.120769+0000","last_peered":"2024-06-10T00:53:22.120769+0000","last_clean":"2024-06-10T00:53:22.120769+0000","last_became_active":"2024-06-10T00:52:45.992682+0000","last_became_peered":"2024-06-10T00:52:45.992682+0000","last_unstale":"2024-06-10T00:53:22.120769+0000","last_undegraded":"2024-06-10T00:53:22.120769+0000","last_fullsized":"2024-06-10T00:53:22.120769+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:49:41.817126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120099+0000","last_change":"2024-06-10T00:52:35.965893+0000","last_active":"2024-06-10T00:53:22.120099+0000","last_peered":"2024-06-10T00:53:22.120099+0000","last_clean":"2024-06-10T00:53:22.120099+0000","last_became_active":"2024-06-10T00:52:35.965426+0000","last_became_peered":"2024-06-10T00:52:35.965426+0000","last_unstale":"2024-06-10T00:53:22.120099+0000","last_undegraded":"2024-06-10T00:53:22.120099+0000","last_fullsized":"2024-06-10T00:53:22.120099+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:29:19.640836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:41.297842+0000","last_change":"2024-06-10T00:52:32.753631+0000","last_active":"2024-06-10T00:53:41.297842+0000","last_peered":"2024-06-10T00:53:41.297842+0000","last_clean":"2024-06-10T00:53:41.297842+0000","last_became_active":"2024-06-10T00:52:32.753260+0000","last_became_peered":"2024-06-10T00:52:32.753260+0000","last_unstale":"2024-06-10T00:53:41.297842+0000","last_undegraded":"2024-06-10T00:53:41.297842+0000","last_fullsized":"2024-06-10T00:53:41.297842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:12:55.932676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":364,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120324+0000","last_change":"2024-06-10T00:51:09.267750+0000","last_active":"2024-06-10T00:53:22.120324+0000","last_peered":"2024-06-10T00:53:22.120324+0000","last_clean":"2024-06-10T00:53:22.120324+0000","last_became_active":"2024-06-10T00:51:09.267246+0000","last_became_peered":"2024-06-10T00:51:09.267246+0000","last_unstale":"2024-06-10T00:53:22.120324+0000","last_undegraded":"2024-06-10T00:53:22.120324+0000","last_fullsized":"2024-06-10T00:53:22.120324+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:47:44.965452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:47:44.965452+0000","last_clean_scrub_stamp":"2024-06-10T00:47:44.965452+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:21.100090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.128372+0000","last_change":"2024-06-10T00:52:46.242152+0000","last_active":"2024-06-10T00:53:22.128372+0000","last_peered":"2024-06-10T00:53:22.128372+0000","last_clean":"2024-06-10T00:53:22.128372+0000","last_became_active":"2024-06-10T00:52:46.241542+0000","last_became_peered":"2024-06-10T00:52:46.241542+0000","last_unstale":"2024-06-10T00:53:22.128372+0000","last_undegraded":"2024-06-10T00:53:22.128372+0000","last_fullsized":"2024-06-10T00:53:22.128372+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:55:54.695465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.128241+0000","last_change":"2024-06-10T00:52:35.963911+0000","last_active":"2024-06-10T00:53:22.128241+0000","last_peered":"2024-06-10T00:53:22.128241+0000","last_clean":"2024-06-10T00:53:22.128241+0000","last_became_active":"2024-06-10T00:52:35.963571+0000","last_became_peered":"2024-06-10T00:52:35.963571+0000","last_unstale":"2024-06-10T00:53:22.128241+0000","last_undegraded":"2024-06-10T00:53:22.128241+0000","last_fullsized":"2024-06-10T00:53:22.128241+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:27:16.757428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:39.289046+0000","last_change":"2024-06-10T00:53:22.129035+0000","last_active":"2024-06-10T00:53:39.289046+0000","last_peered":"2024-06-10T00:53:39.289046+0000","last_clean":"2024-06-10T00:53:39.289046+0000","last_became_active":"2024-06-10T00:53:22.128561+0000","last_became_peered":"2024-06-10T00:53:22.128561+0000","last_unstale":"2024-06-10T00:53:39.289046+0000","last_undegraded":"2024-06-10T00:53:39.289046+0000","last_fullsized":"2024-06-10T00:53:39.289046+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:02:43.313118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.122925+0000","last_change":"2024-06-10T00:52:45.996907+0000","last_active":"2024-06-10T00:53:22.122925+0000","last_peered":"2024-06-10T00:53:22.122925+0000","last_clean":"2024-06-10T00:53:22.122925+0000","last_became_active":"2024-06-10T00:52:45.996554+0000","last_became_peered":"2024-06-10T00:52:45.996554+0000","last_unstale":"2024-06-10T00:53:22.122925+0000","last_undegraded":"2024-06-10T00:53:22.122925+0000","last_fullsized":"2024-06-10T00:53:22.122925+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:00:46.824071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'1","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:38.282338+0000","last_change":"2024-06-10T00:52:32.960223+0000","last_active":"2024-06-10T00:53:38.282338+0000","last_peered":"2024-06-10T00:53:38.282338+0000","last_clean":"2024-06-10T00:53:38.282338+0000","last_became_active":"2024-06-10T00:52:32.959871+0000","last_became_peered":"2024-06-10T00:52:32.959871+0000","last_unstale":"2024-06-10T00:53:38.282338+0000","last_undegraded":"2024-06-10T00:53:38.282338+0000","last_fullsized":"2024-06-10T00:53:38.282338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:18:17.099140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.129037+0000","last_change":"2024-06-10T00:52:35.962516+0000","last_active":"2024-06-10T00:53:22.129037+0000","last_peered":"2024-06-10T00:53:22.129037+0000","last_clean":"2024-06-10T00:53:22.129037+0000","last_became_active":"2024-06-10T00:52:35.962187+0000","last_became_peered":"2024-06-10T00:52:35.962187+0000","last_unstale":"2024-06-10T00:53:22.129037+0000","last_undegraded":"2024-06-10T00:53:22.129037+0000","last_fullsized":"2024-06-10T00:53:22.129037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:10:44.295192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.119897+0000","last_change":"2024-06-10T00:52:46.241554+0000","last_active":"2024-06-10T00:53:22.119897+0000","last_peered":"2024-06-10T00:53:22.119897+0000","last_clean":"2024-06-10T00:53:22.119897+0000","last_became_active":"2024-06-10T00:52:46.241142+0000","last_became_peered":"2024-06-10T00:52:46.241142+0000","last_unstale":"2024-06-10T00:53:22.119897+0000","last_undegraded":"2024-06-10T00:53:22.119897+0000","last_fullsized":"2024-06-10T00:53:22.119897+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:58:07.233080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"98'10","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:54:01.967107+0000","last_change":"2024-06-10T00:52:32.761854+0000","last_active":"2024-06-10T00:54:01.967107+0000","last_peered":"2024-06-10T00:54:01.967107+0000","last_clean":"2024-06-10T00:54:01.967107+0000","last_became_active":"2024-06-10T00:52:32.761522+0000","last_became_peered":"2024-06-10T00:52:32.761522+0000","last_unstale":"2024-06-10T00:54:01.967107+0000","last_undegraded":"2024-06-10T00:54:01.967107+0000","last_fullsized":"2024-06-10T00:54:01.967107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:06:43.269001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120684+0000","last_change":"2024-06-10T00:52:35.959250+0000","last_active":"2024-06-10T00:53:22.120684+0000","last_peered":"2024-06-10T00:53:22.120684+0000","last_clean":"2024-06-10T00:53:22.120684+0000","last_became_active":"2024-06-10T00:52:35.958925+0000","last_became_peered":"2024-06-10T00:52:35.958925+0000","last_unstale":"2024-06-10T00:53:22.120684+0000","last_undegraded":"2024-06-10T00:53:22.120684+0000","last_fullsized":"2024-06-10T00:53:22.120684+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:11:12.355505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120321+0000","last_change":"2024-06-10T00:52:45.991819+0000","last_active":"2024-06-10T00:53:22.120321+0000","last_peered":"2024-06-10T00:53:22.120321+0000","last_clean":"2024-06-10T00:53:22.120321+0000","last_became_active":"2024-06-10T00:52:45.991408+0000","last_became_peered":"2024-06-10T00:52:45.991408+0000","last_unstale":"2024-06-10T00:53:22.120321+0000","last_undegraded":"2024-06-10T00:53:22.120321+0000","last_fullsized":"2024-06-10T00:53:22.120321+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:16:34.912359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'4","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:38.280550+0000","last_change":"2024-06-10T00:52:32.869094+0000","last_active":"2024-06-10T00:53:38.280550+0000","last_peered":"2024-06-10T00:53:38.280550+0000","last_clean":"2024-06-10T00:53:38.280550+0000","last_became_active":"2024-06-10T00:52:32.868664+0000","last_became_peered":"2024-06-10T00:52:32.868664+0000","last_unstale":"2024-06-10T00:53:38.280550+0000","last_undegraded":"2024-06-10T00:53:38.280550+0000","last_fullsized":"2024-06-10T00:53:38.280550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:50:38.051090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.125667+0000","last_change":"2024-06-10T00:52:35.952356+0000","last_active":"2024-06-10T00:53:22.125667+0000","last_peered":"2024-06-10T00:53:22.125667+0000","last_clean":"2024-06-10T00:53:22.125667+0000","last_became_active":"2024-06-10T00:52:35.952002+0000","last_became_peered":"2024-06-10T00:52:35.952002+0000","last_unstale":"2024-06-10T00:53:22.125667+0000","last_undegraded":"2024-06-10T00:53:22.125667+0000","last_fullsized":"2024-06-10T00:53:22.125667+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:12:28.581522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.200356+0000","last_change":"2024-06-10T00:52:46.241773+0000","last_active":"2024-06-10T00:53:22.200356+0000","last_peered":"2024-06-10T00:53:22.200356+0000","last_clean":"2024-06-10T00:53:22.200356+0000","last_became_active":"2024-06-10T00:52:46.241272+0000","last_became_peered":"2024-06-10T00:52:46.241272+0000","last_unstale":"2024-06-10T00:53:22.200356+0000","last_undegraded":"2024-06-10T00:53:22.200356+0000","last_fullsized":"2024-06-10T00:53:22.200356+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:07:28.337536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:38.283362+0000","last_change":"2024-06-10T00:52:32.960100+0000","last_active":"2024-06-10T00:53:38.283362+0000","last_peered":"2024-06-10T00:53:38.283362+0000","last_clean":"2024-06-10T00:53:38.283362+0000","last_became_active":"2024-06-10T00:52:32.959779+0000","last_became_peered":"2024-06-10T00:52:32.959779+0000","last_unstale":"2024-06-10T00:53:38.283362+0000","last_undegraded":"2024-06-10T00:53:38.283362+0000","last_fullsized":"2024-06-10T00:53:38.283362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:09.555753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.122508+0000","last_change":"2024-06-10T00:52:35.955586+0000","last_active":"2024-06-10T00:53:22.122508+0000","last_peered":"2024-06-10T00:53:22.122508+0000","last_clean":"2024-06-10T00:53:22.122508+0000","last_became_active":"2024-06-10T00:52:35.955216+0000","last_became_peered":"2024-06-10T00:52:35.955216+0000","last_unstale":"2024-06-10T00:53:22.122508+0000","last_undegraded":"2024-06-10T00:53:22.122508+0000","last_fullsized":"2024-06-10T00:53:22.122508+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:23:12.197182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124090+0000","last_change":"2024-06-10T00:52:46.001381+0000","last_active":"2024-06-10T00:53:22.124090+0000","last_peered":"2024-06-10T00:53:22.124090+0000","last_clean":"2024-06-10T00:53:22.124090+0000","last_became_active":"2024-06-10T00:52:46.000348+0000","last_became_peered":"2024-06-10T00:52:46.000348+0000","last_unstale":"2024-06-10T00:53:22.124090+0000","last_undegraded":"2024-06-10T00:53:22.124090+0000","last_fullsized":"2024-06-10T00:53:22.124090+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:12:28.348852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'3","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:38.282471+0000","last_change":"2024-06-10T00:52:32.753444+0000","last_active":"2024-06-10T00:53:38.282471+0000","last_peered":"2024-06-10T00:53:38.282471+0000","last_clean":"2024-06-10T00:53:38.282471+0000","last_became_active":"2024-06-10T00:52:32.753149+0000","last_became_peered":"2024-06-10T00:52:32.753149+0000","last_unstale":"2024-06-10T00:53:38.282471+0000","last_undegraded":"2024-06-10T00:53:38.282471+0000","last_fullsized":"2024-06-10T00:53:38.282471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:28.241950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124153+0000","last_change":"2024-06-10T00:52:35.966718+0000","last_active":"2024-06-10T00:53:22.124153+0000","last_peered":"2024-06-10T00:53:22.124153+0000","last_clean":"2024-06-10T00:53:22.124153+0000","last_became_active":"2024-06-10T00:52:35.965907+0000","last_became_peered":"2024-06-10T00:52:35.965907+0000","last_unstale":"2024-06-10T00:53:22.124153+0000","last_undegraded":"2024-06-10T00:53:22.124153+0000","last_fullsized":"2024-06-10T00:53:22.124153+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:21:34.261306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":23,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.200347+0000","last_change":"2024-06-10T00:52:45.991034+0000","last_active":"2024-06-10T00:53:22.200347+0000","last_peered":"2024-06-10T00:53:22.200347+0000","last_clean":"2024-06-10T00:53:22.200347+0000","last_became_active":"2024-06-10T00:52:45.990546+0000","last_became_peered":"2024-06-10T00:52:45.990546+0000","last_unstale":"2024-06-10T00:53:22.200347+0000","last_undegraded":"2024-06-10T00:53:22.200347+0000","last_fullsized":"2024-06-10T00:53:22.200347+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:09:23.412922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'1","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120466+0000","last_change":"2024-06-10T00:52:32.764024+0000","last_active":"2024-06-10T00:53:22.120466+0000","last_peered":"2024-06-10T00:53:22.120466+0000","last_clean":"2024-06-10T00:53:22.120466+0000","last_became_active":"2024-06-10T00:52:32.763449+0000","last_became_peered":"2024-06-10T00:52:32.763449+0000","last_unstale":"2024-06-10T00:53:22.120466+0000","last_undegraded":"2024-06-10T00:53:22.120466+0000","last_fullsized":"2024-06-10T00:53:22.120466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:50:23.605873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120301+0000","last_change":"2024-06-10T00:52:35.955570+0000","last_active":"2024-06-10T00:53:22.120301+0000","last_peered":"2024-06-10T00:53:22.120301+0000","last_clean":"2024-06-10T00:53:22.120301+0000","last_became_active":"2024-06-10T00:52:35.955248+0000","last_became_peered":"2024-06-10T00:52:35.955248+0000","last_unstale":"2024-06-10T00:53:22.120301+0000","last_undegraded":"2024-06-10T00:53:22.120301+0000","last_fullsized":"2024-06-10T00:53:22.120301+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:26.124499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.199906+0000","last_change":"2024-06-10T00:52:46.241776+0000","last_active":"2024-06-10T00:53:22.199906+0000","last_peered":"2024-06-10T00:53:22.199906+0000","last_clean":"2024-06-10T00:53:22.199906+0000","last_became_active":"2024-06-10T00:52:46.241276+0000","last_became_peered":"2024-06-10T00:52:46.241276+0000","last_unstale":"2024-06-10T00:53:22.199906+0000","last_undegraded":"2024-06-10T00:53:22.199906+0000","last_fullsized":"2024-06-10T00:53:22.199906+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:08.027938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":53,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:38.284502+0000","last_change":"2024-06-10T00:53:22.204034+0000","last_active":"2024-06-10T00:53:38.284502+0000","last_peered":"2024-06-10T00:53:38.284502+0000","last_clean":"2024-06-10T00:53:38.284502+0000","last_became_active":"2024-06-10T00:53:22.199988+0000","last_became_peered":"2024-06-10T00:53:22.199988+0000","last_unstale":"2024-06-10T00:53:38.284502+0000","last_undegraded":"2024-06-10T00:53:38.284502+0000","last_fullsized":"2024-06-10T00:53:38.284502+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:39:54.012360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120778+0000","last_change":"2024-06-10T00:52:35.959182+0000","last_active":"2024-06-10T00:53:22.120778+0000","last_peered":"2024-06-10T00:53:22.120778+0000","last_clean":"2024-06-10T00:53:22.120778+0000","last_became_active":"2024-06-10T00:52:35.958828+0000","last_became_peered":"2024-06-10T00:52:35.958828+0000","last_unstale":"2024-06-10T00:53:22.120778+0000","last_undegraded":"2024-06-10T00:53:22.120778+0000","last_fullsized":"2024-06-10T00:53:22.120778+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:29:58.857659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.198410+0000","last_change":"2024-06-10T00:52:45.989739+0000","last_active":"2024-06-10T00:53:22.198410+0000","last_peered":"2024-06-10T00:53:22.198410+0000","last_clean":"2024-06-10T00:53:22.198410+0000","last_became_active":"2024-06-10T00:52:45.989279+0000","last_became_peered":"2024-06-10T00:52:45.989279+0000","last_unstale":"2024-06-10T00:53:22.198410+0000","last_undegraded":"2024-06-10T00:53:22.198410+0000","last_fullsized":"2024-06-10T00:53:22.198410+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:19:17.551947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":44,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:39.288814+0000","last_change":"2024-06-10T00:52:32.959662+0000","last_active":"2024-06-10T00:53:39.288814+0000","last_peered":"2024-06-10T00:53:39.288814+0000","last_clean":"2024-06-10T00:53:39.288814+0000","last_became_active":"2024-06-10T00:52:32.959286+0000","last_became_peered":"2024-06-10T00:52:32.959286+0000","last_unstale":"2024-06-10T00:53:39.288814+0000","last_undegraded":"2024-06-10T00:53:39.288814+0000","last_fullsized":"2024-06-10T00:53:39.288814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:26:04.361567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.121028+0000","last_change":"2024-06-10T00:52:35.960137+0000","last_active":"2024-06-10T00:53:22.121028+0000","last_peered":"2024-06-10T00:53:22.121028+0000","last_clean":"2024-06-10T00:53:22.121028+0000","last_became_active":"2024-06-10T00:52:35.959584+0000","last_became_peered":"2024-06-10T00:52:35.959584+0000","last_unstale":"2024-06-10T00:53:22.121028+0000","last_undegraded":"2024-06-10T00:53:22.121028+0000","last_fullsized":"2024-06-10T00:53:22.121028+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:23:38.702685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.125101+0000","last_change":"2024-06-10T00:52:46.002268+0000","last_active":"2024-06-10T00:53:22.125101+0000","last_peered":"2024-06-10T00:53:22.125101+0000","last_clean":"2024-06-10T00:53:22.125101+0000","last_became_active":"2024-06-10T00:52:46.001769+0000","last_became_peered":"2024-06-10T00:52:46.001769+0000","last_unstale":"2024-06-10T00:53:22.125101+0000","last_undegraded":"2024-06-10T00:53:22.125101+0000","last_fullsized":"2024-06-10T00:53:22.125101+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:33:51.365951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.127739+0000","last_change":"2024-06-10T00:52:32.757112+0000","last_active":"2024-06-10T00:53:22.127739+0000","last_peered":"2024-06-10T00:53:22.127739+0000","last_clean":"2024-06-10T00:53:22.127739+0000","last_became_active":"2024-06-10T00:52:32.755589+0000","last_became_peered":"2024-06-10T00:52:32.755589+0000","last_unstale":"2024-06-10T00:53:22.127739+0000","last_undegraded":"2024-06-10T00:53:22.127739+0000","last_fullsized":"2024-06-10T00:53:22.127739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:15:12.814489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120794+0000","last_change":"2024-06-10T00:52:35.955925+0000","last_active":"2024-06-10T00:53:22.120794+0000","last_peered":"2024-06-10T00:53:22.120794+0000","last_clean":"2024-06-10T00:53:22.120794+0000","last_became_active":"2024-06-10T00:52:35.955556+0000","last_became_peered":"2024-06-10T00:52:35.955556+0000","last_unstale":"2024-06-10T00:53:22.120794+0000","last_undegraded":"2024-06-10T00:53:22.120794+0000","last_fullsized":"2024-06-10T00:53:22.120794+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:42:18.324484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.126727+0000","last_change":"2024-06-10T00:52:45.988810+0000","last_active":"2024-06-10T00:53:22.126727+0000","last_peered":"2024-06-10T00:53:22.126727+0000","last_clean":"2024-06-10T00:53:22.126727+0000","last_became_active":"2024-06-10T00:52:45.988041+0000","last_became_peered":"2024-06-10T00:52:45.988041+0000","last_unstale":"2024-06-10T00:53:22.126727+0000","last_undegraded":"2024-06-10T00:53:22.126727+0000","last_fullsized":"2024-06-10T00:53:22.126727+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:14:08.979802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.129013+0000","last_change":"2024-06-10T00:52:32.765493+0000","last_active":"2024-06-10T00:53:22.129013+0000","last_peered":"2024-06-10T00:53:22.129013+0000","last_clean":"2024-06-10T00:53:22.129013+0000","last_became_active":"2024-06-10T00:52:32.765193+0000","last_became_peered":"2024-06-10T00:52:32.765193+0000","last_unstale":"2024-06-10T00:53:22.129013+0000","last_undegraded":"2024-06-10T00:53:22.129013+0000","last_fullsized":"2024-06-10T00:53:22.129013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:38:11.554673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.121112+0000","last_change":"2024-06-10T00:52:35.960216+0000","last_active":"2024-06-10T00:53:22.121112+0000","last_peered":"2024-06-10T00:53:22.121112+0000","last_clean":"2024-06-10T00:53:22.121112+0000","last_became_active":"2024-06-10T00:52:35.959664+0000","last_became_peered":"2024-06-10T00:52:35.959664+0000","last_unstale":"2024-06-10T00:53:22.121112+0000","last_undegraded":"2024-06-10T00:53:22.121112+0000","last_fullsized":"2024-06-10T00:53:22.121112+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:41:58.086330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.124294+0000","last_change":"2024-06-10T00:52:35.967043+0000","last_active":"2024-06-10T00:53:22.124294+0000","last_peered":"2024-06-10T00:53:22.124294+0000","last_clean":"2024-06-10T00:53:22.124294+0000","last_became_active":"2024-06-10T00:52:35.966366+0000","last_became_peered":"2024-06-10T00:52:35.966366+0000","last_unstale":"2024-06-10T00:53:22.124294+0000","last_undegraded":"2024-06-10T00:53:22.124294+0000","last_fullsized":"2024-06-10T00:53:22.124294+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:27:06.047113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120608+0000","last_change":"2024-06-10T00:52:35.960517+0000","last_active":"2024-06-10T00:53:22.120608+0000","last_peered":"2024-06-10T00:53:22.120608+0000","last_clean":"2024-06-10T00:53:22.120608+0000","last_became_active":"2024-06-10T00:52:35.957895+0000","last_became_peered":"2024-06-10T00:52:35.957895+0000","last_unstale":"2024-06-10T00:53:22.120608+0000","last_undegraded":"2024-06-10T00:53:22.120608+0000","last_fullsized":"2024-06-10T00:53:22.120608+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:04:34.438070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120365+0000","last_change":"2024-06-10T00:52:35.962163+0000","last_active":"2024-06-10T00:53:22.120365+0000","last_peered":"2024-06-10T00:53:22.120365+0000","last_clean":"2024-06-10T00:53:22.120365+0000","last_became_active":"2024-06-10T00:52:35.961768+0000","last_became_peered":"2024-06-10T00:52:35.961768+0000","last_unstale":"2024-06-10T00:53:22.120365+0000","last_undegraded":"2024-06-10T00:53:22.120365+0000","last_fullsized":"2024-06-10T00:53:22.120365+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:35.793995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":39,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.130686+0000","last_change":"2024-06-10T00:53:22.130686+0000","last_active":"2024-06-10T00:53:22.130686+0000","last_peered":"2024-06-10T00:53:22.130686+0000","last_clean":"2024-06-10T00:53:22.130686+0000","last_became_active":"2024-06-10T00:53:22.130198+0000","last_became_peered":"2024-06-10T00:53:22.130198+0000","last_unstale":"2024-06-10T00:53:22.130686+0000","last_undegraded":"2024-06-10T00:53:22.130686+0000","last_fullsized":"2024-06-10T00:53:22.130686+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:50:26.773977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.129145+0000","last_change":"2024-06-10T00:52:35.957255+0000","last_active":"2024-06-10T00:53:22.129145+0000","last_peered":"2024-06-10T00:53:22.129145+0000","last_clean":"2024-06-10T00:53:22.129145+0000","last_became_active":"2024-06-10T00:52:35.956923+0000","last_became_peered":"2024-06-10T00:52:35.956923+0000","last_unstale":"2024-06-10T00:53:22.129145+0000","last_undegraded":"2024-06-10T00:53:22.129145+0000","last_fullsized":"2024-06-10T00:53:22.129145+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:33:25.743503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.123595+0000","last_change":"2024-06-10T00:52:35.964258+0000","last_active":"2024-06-10T00:53:22.123595+0000","last_peered":"2024-06-10T00:53:22.123595+0000","last_clean":"2024-06-10T00:53:22.123595+0000","last_became_active":"2024-06-10T00:52:35.963862+0000","last_became_peered":"2024-06-10T00:52:35.963862+0000","last_unstale":"2024-06-10T00:53:22.123595+0000","last_undegraded":"2024-06-10T00:53:22.123595+0000","last_fullsized":"2024-06-10T00:53:22.123595+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:48:58.554411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120966+0000","last_change":"2024-06-10T00:52:35.965046+0000","last_active":"2024-06-10T00:53:22.120966+0000","last_peered":"2024-06-10T00:53:22.120966+0000","last_clean":"2024-06-10T00:53:22.120966+0000","last_became_active":"2024-06-10T00:52:35.964496+0000","last_became_peered":"2024-06-10T00:52:35.964496+0000","last_unstale":"2024-06-10T00:53:22.120966+0000","last_undegraded":"2024-06-10T00:53:22.120966+0000","last_fullsized":"2024-06-10T00:53:22.120966+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:09:42.979731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120807+0000","last_change":"2024-06-10T00:52:35.964287+0000","last_active":"2024-06-10T00:53:22.120807+0000","last_peered":"2024-06-10T00:53:22.120807+0000","last_clean":"2024-06-10T00:53:22.120807+0000","last_became_active":"2024-06-10T00:52:35.963920+0000","last_became_peered":"2024-06-10T00:52:35.963920+0000","last_unstale":"2024-06-10T00:53:22.120807+0000","last_undegraded":"2024-06-10T00:53:22.120807+0000","last_fullsized":"2024-06-10T00:53:22.120807+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:41:54.593274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.127245+0000","last_change":"2024-06-10T00:52:35.961440+0000","last_active":"2024-06-10T00:53:22.127245+0000","last_peered":"2024-06-10T00:53:22.127245+0000","last_clean":"2024-06-10T00:53:22.127245+0000","last_became_active":"2024-06-10T00:52:35.960066+0000","last_became_peered":"2024-06-10T00:52:35.960066+0000","last_unstale":"2024-06-10T00:53:22.127245+0000","last_undegraded":"2024-06-10T00:53:22.127245+0000","last_fullsized":"2024-06-10T00:53:22.127245+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:01.024596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.120129+0000","last_change":"2024-06-10T00:52:35.964464+0000","last_active":"2024-06-10T00:53:22.120129+0000","last_peered":"2024-06-10T00:53:22.120129+0000","last_clean":"2024-06-10T00:53:22.120129+0000","last_became_active":"2024-06-10T00:52:35.964159+0000","last_became_peered":"2024-06-10T00:52:35.964159+0000","last_unstale":"2024-06-10T00:53:22.120129+0000","last_undegraded":"2024-06-10T00:53:22.120129+0000","last_fullsized":"2024-06-10T00:53:22.120129+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:18.660859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.198928+0000","last_change":"2024-06-10T00:52:35.961539+0000","last_active":"2024-06-10T00:53:22.198928+0000","last_peered":"2024-06-10T00:53:22.198928+0000","last_clean":"2024-06-10T00:53:22.198928+0000","last_became_active":"2024-06-10T00:52:35.958738+0000","last_became_peered":"2024-06-10T00:52:35.958738+0000","last_unstale":"2024-06-10T00:53:22.198928+0000","last_undegraded":"2024-06-10T00:53:22.198928+0000","last_fullsized":"2024-06-10T00:53:22.198928+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:06:48.057551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.128823+0000","last_change":"2024-06-10T00:52:35.962563+0000","last_active":"2024-06-10T00:53:22.128823+0000","last_peered":"2024-06-10T00:53:22.128823+0000","last_clean":"2024-06-10T00:53:22.128823+0000","last_became_active":"2024-06-10T00:52:35.962255+0000","last_became_peered":"2024-06-10T00:52:35.962255+0000","last_unstale":"2024-06-10T00:53:22.128823+0000","last_undegraded":"2024-06-10T00:53:22.128823+0000","last_fullsized":"2024-06-10T00:53:22.128823+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:33:35.689446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":33,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-10T00:53:22.123198+0000","last_change":"2024-06-10T00:52:35.959616+0000","last_active":"2024-06-10T00:53:22.123198+0000","last_peered":"2024-06-10T00:53:22.123198+0000","last_clean":"2024-06-10T00:53:22.123198+0000","last_became_active":"2024-06-10T00:52:35.959251+0000","last_became_peered":"2024-06-10T00:52:35.959251+0000","last_unstale":"2024-06-10T00:53:22.123198+0000","last_undegraded":"2024-06-10T00:53:22.123198+0000","last_fullsized":"2024-06-10T00:53:22.123198+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:39:23.933279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284806,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68876,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":182,"num_read_kb":165,"num_write":167,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":206628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351010,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":554058,"data_compressed":26474,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612647,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28212,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716940,"statfs":{"total":95995035648,"available":95966146560,"internally_reserved":0,"allocated":1224704,"data_stored":1206202,"data_compressed":62689,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776172,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1097728,"data_stored":558390,"data_compressed":26474,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037809,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":554214,"data_compressed":26474,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299447,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36452,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708700,"statfs":{"total":95995035648,"available":95957708800,"internally_reserved":0,"allocated":1142784,"data_stored":1150122,"data_compressed":62689,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462971,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":549470,"data_compressed":26474,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36452,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708700,"statfs":{"total":95995035648,"available":95957708800,"internally_reserved":0,"allocated":1142784,"data_stored":614256,"data_compressed":26474,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":550858,"data_compressed":26474,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32308,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712844,"statfs":{"total":95995035648,"available":95961952256,"internally_reserved":0,"allocated":1159168,"data_stored":615884,"data_compressed":26474,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149770,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":549664,"data_compressed":26474,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888133,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":554232,"data_compressed":26474,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724608,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1150976,"data_stored":1145710,"data_compressed":62689,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":66504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T00:54:11.173 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-10T00:54:11.173 INFO:tasks.cephadm:Setting up client nodes... 2024-06-10T00:54:11.174 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-10T00:54:11.427 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:11.799 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:11 smithi037 ceph-mon[32192]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-06-10T00:54:11.799 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:11 smithi037 ceph-mon[32192]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:54:11.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:11 smithi167 ceph-mon[36917]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-06-10T00:54:11.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:11 smithi167 ceph-mon[36917]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:54:11.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:11 smithi094 ceph-mon[36708]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-06-10T00:54:11.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:11 smithi094 ceph-mon[36708]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:54:13.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.193+0000 7f850bae5700 1 -- 172.21.15.37:0/1490908183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8504103720 msgr2=0x7f8504103b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:13.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.193+0000 7f850bae5700 1 --2- 172.21.15.37:0/1490908183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8504103720 0x7f8504103b80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f84f40099c0 tx=0x7f84f4009cd0 comp rx=0 tx=0).stop 2024-06-10T00:54:13.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.193+0000 7f850bae5700 1 -- 172.21.15.37:0/1490908183 shutdown_connections 2024-06-10T00:54:13.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.193+0000 7f850bae5700 1 --2- 172.21.15.37:0/1490908183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85041040c0 0x7f850410a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:13.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.193+0000 7f850bae5700 1 --2- 172.21.15.37:0/1490908183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8504102520 0x7f8504102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:13.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.193+0000 7f850bae5700 1 --2- 172.21.15.37:0/1490908183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8504103720 0x7f8504103b80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:13.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.193+0000 7f850bae5700 1 -- 172.21.15.37:0/1490908183 >> 172.21.15.37:0/1490908183 conn(0x7f85040fdac0 msgr2=0x7f85040fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:13.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.193+0000 7f850bae5700 1 -- 172.21.15.37:0/1490908183 shutdown_connections 2024-06-10T00:54:13.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.193+0000 7f850bae5700 1 -- 172.21.15.37:0/1490908183 wait complete. 2024-06-10T00:54:13.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.194+0000 7f850bae5700 1 Processor -- start 2024-06-10T00:54:13.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.194+0000 7f850bae5700 1 -- start start 2024-06-10T00:54:13.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.194+0000 7f850bae5700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8504102520 0x7f8504198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:13.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.194+0000 7f850bae5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8504103720 0x7f8504198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:13.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.195+0000 7f850bae5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85041040c0 0x7f850419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:13.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.195+0000 7f850bae5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f850410e100 con 0x7f8504103720 2024-06-10T00:54:13.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.195+0000 7f850bae5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f850410df80 con 0x7f85041040c0 2024-06-10T00:54:13.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.195+0000 7f850bae5700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f850410e280 con 0x7f8504102520 2024-06-10T00:54:13.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.195+0000 7f8509080700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8504103720 0x7f8504198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:13.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.195+0000 7f8509080700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8504103720 0x7f8504198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42566/0 (socket says 172.21.15.37:42566) 2024-06-10T00:54:13.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.195+0000 7f8509080700 1 -- 172.21.15.37:0/4081503727 learned_addr learned my addr 172.21.15.37:0/4081503727 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:54:13.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.195+0000 7f8509881700 1 --2- 172.21.15.37:0/4081503727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8504102520 0x7f8504198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:13.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.195+0000 7f850a082700 1 --2- 172.21.15.37:0/4081503727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85041040c0 0x7f850419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:13.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.195+0000 7f8509080700 1 -- 172.21.15.37:0/4081503727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8504102520 msgr2=0x7f8504198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:13.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.195+0000 7f8509080700 1 --2- 172.21.15.37:0/4081503727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8504102520 0x7f8504198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:13.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.195+0000 7f8509080700 1 -- 172.21.15.37:0/4081503727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85041040c0 msgr2=0x7f850419fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:13.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.195+0000 7f8509080700 1 --2- 172.21.15.37:0/4081503727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85041040c0 0x7f850419fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:13.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.195+0000 7f8509080700 1 -- 172.21.15.37:0/4081503727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85041a0280 con 0x7f8504103720 2024-06-10T00:54:13.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.195+0000 7f850a082700 1 --2- 172.21.15.37:0/4081503727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85041040c0 0x7f850419fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:54:13.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.195+0000 7f8509080700 1 --2- 172.21.15.37:0/4081503727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8504103720 0x7f8504198b60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f84f4000c00 tx=0x7f84f400bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:13.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.196+0000 7f84faffd700 1 -- 172.21.15.37:0/4081503727 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4009e60 con 0x7f8504103720 2024-06-10T00:54:13.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.196+0000 7f850bae5700 1 -- 172.21.15.37:0/4081503727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85041a0510 con 0x7f8504103720 2024-06-10T00:54:13.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.196+0000 7f84faffd700 1 -- 172.21.15.37:0/4081503727 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84f400f460 con 0x7f8504103720 2024-06-10T00:54:13.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.196+0000 7f84faffd700 1 -- 172.21.15.37:0/4081503727 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4017650 con 0x7f8504103720 2024-06-10T00:54:13.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.196+0000 7f850bae5700 1 -- 172.21.15.37:0/4081503727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85041a0970 con 0x7f8504103720 2024-06-10T00:54:13.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.197+0000 7f850bae5700 1 -- 172.21.15.37:0/4081503727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8504050a00 con 0x7f8504103720 2024-06-10T00:54:13.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.199+0000 7f84faffd700 1 -- 172.21.15.37:0/4081503727 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f84f400f610 con 0x7f8504103720 2024-06-10T00:54:13.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.199+0000 7f84faffd700 1 --2- 172.21.15.37:0/4081503727 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f84f00a2d60 0x7f84f00a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:13.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.200+0000 7f8509881700 1 --2- 172.21.15.37:0/4081503727 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f84f00a2d60 0x7f84f00a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:13.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.200+0000 7f84faffd700 1 -- 172.21.15.37:0/4081503727 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f84f40d0470 con 0x7f8504103720 2024-06-10T00:54:13.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.200+0000 7f8509881700 1 --2- 172.21.15.37:0/4081503727 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f84f00a2d60 0x7f84f00a5220 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8504103580 tx=0x7f8500009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:13.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.203+0000 7f84faffd700 1 -- 172.21.15.37:0/4081503727 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f84f4013070 con 0x7f8504103720 2024-06-10T00:54:13.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.524+0000 7f850bae5700 1 -- 172.21.15.37:0/4081503727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f850402dd40 con 0x7f8504103720 2024-06-10T00:54:13.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.526+0000 7f84faffd700 1 -- 172.21.15.37:0/4081503727 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f850402dd40 con 0x7f8504103720 2024-06-10T00:54:13.527 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-06-10T00:54:13.527 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQC1TmZm34hSHxAAi9ov8BCrZtWr8qBvWKkgBA== 2024-06-10T00:54:13.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.529+0000 7f850bae5700 1 -- 172.21.15.37:0/4081503727 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f84f00a2d60 msgr2=0x7f84f00a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:13.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.529+0000 7f850bae5700 1 --2- 172.21.15.37:0/4081503727 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f84f00a2d60 0x7f84f00a5220 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8504103580 tx=0x7f8500009260 comp rx=0 tx=0).stop 2024-06-10T00:54:13.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.529+0000 7f850bae5700 1 -- 172.21.15.37:0/4081503727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8504103720 msgr2=0x7f8504198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:13.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.529+0000 7f850bae5700 1 --2- 172.21.15.37:0/4081503727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8504103720 0x7f8504198b60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f84f4000c00 tx=0x7f84f400bee0 comp rx=0 tx=0).stop 2024-06-10T00:54:13.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.529+0000 7f850bae5700 1 -- 172.21.15.37:0/4081503727 shutdown_connections 2024-06-10T00:54:13.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.529+0000 7f850bae5700 1 --2- 172.21.15.37:0/4081503727 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f84f00a2d60 0x7f84f00a5220 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:13.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.529+0000 7f850bae5700 1 --2- 172.21.15.37:0/4081503727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85041040c0 0x7f850419fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:13.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.529+0000 7f850bae5700 1 --2- 172.21.15.37:0/4081503727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8504102520 0x7f8504198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:13.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.529+0000 7f850bae5700 1 --2- 172.21.15.37:0/4081503727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8504103720 0x7f8504198b60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:13.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.529+0000 7f850bae5700 1 -- 172.21.15.37:0/4081503727 >> 172.21.15.37:0/4081503727 conn(0x7f85040fdac0 msgr2=0x7f8504106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:13.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.529+0000 7f850bae5700 1 -- 172.21.15.37:0/4081503727 shutdown_connections 2024-06-10T00:54:13.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:13.529+0000 7f850bae5700 1 -- 172.21.15.37:0/4081503727 wait complete. 2024-06-10T00:54:13.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:13 smithi167 ceph-mon[36917]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-06-10T00:54:13.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:13 smithi094 ceph-mon[36708]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-06-10T00:54:13.888 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:13 smithi037 ceph-mon[32192]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-06-10T00:54:14.182 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-06-10T00:54:14.182 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-06-10T00:54:14.182 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-06-10T00:54:14.221 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-06-10T00:54:14.221 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-06-10T00:54:14.221 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph mgr dump --format=json 2024-06-10T00:54:14.512 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:14.807 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:14 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/4081503727' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-10T00:54:14.807 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:14 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/4081503727' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-10T00:54:14.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:14 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/4081503727' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-10T00:54:14.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:14 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/4081503727' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-10T00:54:14.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:14 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/4081503727' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-10T00:54:14.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:14 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/4081503727' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-10T00:54:15.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:15 smithi037 ceph-mon[32192]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-06-10T00:54:15.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:15 smithi167 ceph-mon[36917]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-06-10T00:54:15.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:15 smithi094 ceph-mon[36708]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-06-10T00:54:16.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.259+0000 7f38bf8f3700 1 -- 172.21.15.37:0/1710199035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b8107400 msgr2=0x7f38b81077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:16.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.259+0000 7f38bf8f3700 1 --2- 172.21.15.37:0/1710199035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b8107400 0x7f38b81077e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f38b4009990 tx=0x7f38b4009ca0 comp rx=0 tx=0).stop 2024-06-10T00:54:16.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.259+0000 7f38bf8f3700 1 -- 172.21.15.37:0/1710199035 shutdown_connections 2024-06-10T00:54:16.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.259+0000 7f38bf8f3700 1 --2- 172.21.15.37:0/1710199035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38b8101d50 0x7f38b810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:16.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.259+0000 7f38bf8f3700 1 --2- 172.21.15.37:0/1710199035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b8107400 0x7f38b81077e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:16.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.259+0000 7f38bf8f3700 1 --2- 172.21.15.37:0/1710199035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38b8101390 0x7f38b8101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:16.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.259+0000 7f38bf8f3700 1 -- 172.21.15.37:0/1710199035 >> 172.21.15.37:0/1710199035 conn(0x7f38b8078340 msgr2=0x7f38b8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:16.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.259+0000 7f38bf8f3700 1 -- 172.21.15.37:0/1710199035 shutdown_connections 2024-06-10T00:54:16.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.259+0000 7f38bf8f3700 1 -- 172.21.15.37:0/1710199035 wait complete. 2024-06-10T00:54:16.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.260+0000 7f38bf8f3700 1 Processor -- start 2024-06-10T00:54:16.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.260+0000 7f38bf8f3700 1 -- start start 2024-06-10T00:54:16.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.260+0000 7f38bf8f3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38b8101390 0x7f38b819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:16.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.260+0000 7f38bf8f3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38b8101d50 0x7f38b819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:16.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.260+0000 7f38bf8f3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b8107400 0x7f38b8198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:16.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.260+0000 7f38bf8f3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f38b8111670 con 0x7f38b8107400 2024-06-10T00:54:16.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.260+0000 7f38bf8f3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f38b81114f0 con 0x7f38b8101390 2024-06-10T00:54:16.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.260+0000 7f38bf8f3700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f38b81117f0 con 0x7f38b8101d50 2024-06-10T00:54:16.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.260+0000 7f38bde90700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b8107400 0x7f38b8198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:16.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.260+0000 7f38bde90700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b8107400 0x7f38b8198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42580/0 (socket says 172.21.15.37:42580) 2024-06-10T00:54:16.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.261+0000 7f38bde90700 1 -- 172.21.15.37:0/1271297312 learned_addr learned my addr 172.21.15.37:0/1271297312 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:54:16.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.261+0000 7f38bd68f700 1 --2- 172.21.15.37:0/1271297312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38b8101390 0x7f38b819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:16.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.261+0000 7f38bce8e700 1 --2- 172.21.15.37:0/1271297312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38b8101d50 0x7f38b819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:16.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.261+0000 7f38bde90700 1 -- 172.21.15.37:0/1271297312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38b8101d50 msgr2=0x7f38b819edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:16.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.261+0000 7f38bde90700 1 --2- 172.21.15.37:0/1271297312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38b8101d50 0x7f38b819edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:16.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.261+0000 7f38bde90700 1 -- 172.21.15.37:0/1271297312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38b8101390 msgr2=0x7f38b819e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:16.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.261+0000 7f38bde90700 1 --2- 172.21.15.37:0/1271297312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38b8101390 0x7f38b819e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:16.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.261+0000 7f38bde90700 1 -- 172.21.15.37:0/1271297312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b81991b0 con 0x7f38b8107400 2024-06-10T00:54:16.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.261+0000 7f38bde90700 1 --2- 172.21.15.37:0/1271297312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b8107400 0x7f38b8198a50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f38b000ada0 tx=0x7f38b000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:16.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.261+0000 7f38ae7fc700 1 -- 172.21.15.37:0/1271297312 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b0017020 con 0x7f38b8107400 2024-06-10T00:54:16.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.261+0000 7f38bf8f3700 1 -- 172.21.15.37:0/1271297312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38b81994a0 con 0x7f38b8107400 2024-06-10T00:54:16.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.261+0000 7f38ae7fc700 1 -- 172.21.15.37:0/1271297312 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38b000a0d0 con 0x7f38b8107400 2024-06-10T00:54:16.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.261+0000 7f38ae7fc700 1 -- 172.21.15.37:0/1271297312 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b0005480 con 0x7f38b8107400 2024-06-10T00:54:16.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.261+0000 7f38bf8f3700 1 -- 172.21.15.37:0/1271297312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38b81a5760 con 0x7f38b8107400 2024-06-10T00:54:16.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.262+0000 7f38bf8f3700 1 -- 172.21.15.37:0/1271297312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38b8069180 con 0x7f38b8107400 2024-06-10T00:54:16.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.263+0000 7f38ae7fc700 1 -- 172.21.15.37:0/1271297312 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f38b0005620 con 0x7f38b8107400 2024-06-10T00:54:16.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.264+0000 7f38ae7fc700 1 --2- 172.21.15.37:0/1271297312 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f38a40a2cb0 0x7f38a40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:16.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.264+0000 7f38bd68f700 1 --2- 172.21.15.37:0/1271297312 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f38a40a2cb0 0x7f38a40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:16.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.264+0000 7f38ae7fc700 1 -- 172.21.15.37:0/1271297312 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f38b00cec30 con 0x7f38b8107400 2024-06-10T00:54:16.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.265+0000 7f38bd68f700 1 --2- 172.21.15.37:0/1271297312 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f38a40a2cb0 0x7f38a40a5170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f38b400b5c0 tx=0x7f38b4005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:16.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.269+0000 7f38ae7fc700 1 -- 172.21.15.37:0/1271297312 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f38b0021020 con 0x7f38b8107400 2024-06-10T00:54:16.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.535+0000 7f38bf8f3700 1 -- 172.21.15.37:0/1271297312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f38b81a5930 con 0x7f38b8107400 2024-06-10T00:54:16.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.540+0000 7f38ae7fc700 1 -- 172.21.15.37:0/1271297312 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263200 (secure 0 0 0) 0x7f38b009b700 con 0x7f38b8107400 2024-06-10T00:54:16.543 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:54:16.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.543+0000 7f38bf8f3700 1 -- 172.21.15.37:0/1271297312 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f38a40a2cb0 msgr2=0x7f38a40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:16.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.543+0000 7f38bf8f3700 1 --2- 172.21.15.37:0/1271297312 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f38a40a2cb0 0x7f38a40a5170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f38b400b5c0 tx=0x7f38b4005960 comp rx=0 tx=0).stop 2024-06-10T00:54:16.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.543+0000 7f38bf8f3700 1 -- 172.21.15.37:0/1271297312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b8107400 msgr2=0x7f38b8198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:16.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.543+0000 7f38bf8f3700 1 --2- 172.21.15.37:0/1271297312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b8107400 0x7f38b8198a50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f38b000ada0 tx=0x7f38b000c330 comp rx=0 tx=0).stop 2024-06-10T00:54:16.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.543+0000 7f38bf8f3700 1 -- 172.21.15.37:0/1271297312 shutdown_connections 2024-06-10T00:54:16.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.543+0000 7f38bf8f3700 1 --2- 172.21.15.37:0/1271297312 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f38a40a2cb0 0x7f38a40a5170 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:16.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.543+0000 7f38bf8f3700 1 --2- 172.21.15.37:0/1271297312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38b8107400 0x7f38b8198a50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:16.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.543+0000 7f38bf8f3700 1 --2- 172.21.15.37:0/1271297312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38b8101390 0x7f38b819e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:16.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.543+0000 7f38bf8f3700 1 --2- 172.21.15.37:0/1271297312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38b8101d50 0x7f38b819edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:16.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.543+0000 7f38bf8f3700 1 -- 172.21.15.37:0/1271297312 >> 172.21.15.37:0/1271297312 conn(0x7f38b8078340 msgr2=0x7f38b8100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:16.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.543+0000 7f38bf8f3700 1 -- 172.21.15.37:0/1271297312 shutdown_connections 2024-06-10T00:54:16.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:16.543+0000 7f38bf8f3700 1 -- 172.21.15.37:0/1271297312 wait complete. 2024-06-10T00:54:17.342 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"flags":0,"active_gid":14148,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2387001309},{"type":"v1","addr":"172.21.15.37:6801","nonce":2387001309}]},"active_addr":"172.21.15.37:6801/2387001309","active_change":"2024-06-10T00:45:20.254495+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.37:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":2549243596}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":2247424744}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":1093170904}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":880157111}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":3324630167}]}]} 2024-06-10T00:54:17.347 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-06-10T00:54:17.347 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-06-10T00:54:17.348 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T00:54:17.607 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:17.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:17 smithi037 ceph-mon[32192]: pgmap v356: 129 pgs: 129 active+clean; 648 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-06-10T00:54:17.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:17 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1271297312' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-10T00:54:17.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:17 smithi167 ceph-mon[36917]: pgmap v356: 129 pgs: 129 active+clean; 648 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-06-10T00:54:17.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:17 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1271297312' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-10T00:54:17.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:17 smithi094 ceph-mon[36708]: pgmap v356: 129 pgs: 129 active+clean; 648 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-06-10T00:54:17.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:17 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1271297312' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-10T00:54:19.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.370+0000 7f5668822700 1 -- 172.21.15.37:0/3784755076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5664101fd0 msgr2=0x7f5664105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:19.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.370+0000 7f5668822700 1 --2- 172.21.15.37:0/3784755076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5664101fd0 0x7f5664105fd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f56580099c0 tx=0x7f5658009cd0 comp rx=0 tx=0).stop 2024-06-10T00:54:19.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.370+0000 7f5668822700 1 -- 172.21.15.37:0/3784755076 shutdown_connections 2024-06-10T00:54:19.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.370+0000 7f5668822700 1 --2- 172.21.15.37:0/3784755076 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56641066b0 0x7f566410e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:19.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.370+0000 7f5668822700 1 --2- 172.21.15.37:0/3784755076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5664101620 0x7f5664101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:19.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.370+0000 7f5668822700 1 --2- 172.21.15.37:0/3784755076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5664101fd0 0x7f5664105fd0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:19.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.370+0000 7f5668822700 1 -- 172.21.15.37:0/3784755076 >> 172.21.15.37:0/3784755076 conn(0x7f5664078340 msgr2=0x7f5664078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:19.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.370+0000 7f5668822700 1 -- 172.21.15.37:0/3784755076 shutdown_connections 2024-06-10T00:54:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.370+0000 7f5668822700 1 -- 172.21.15.37:0/3784755076 wait complete. 2024-06-10T00:54:19.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.371+0000 7f5668822700 1 Processor -- start 2024-06-10T00:54:19.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.371+0000 7f5668822700 1 -- start start 2024-06-10T00:54:19.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.371+0000 7f5668822700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5664101620 0x7f566419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:19.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.371+0000 7f5668822700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5664101fd0 0x7f566419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:19.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.371+0000 7f5668822700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56641066b0 0x7f5664198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:19.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.371+0000 7f5668822700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f56641116f0 con 0x7f5664101fd0 2024-06-10T00:54:19.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.371+0000 7f5668822700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5664111570 con 0x7f56641066b0 2024-06-10T00:54:19.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.371+0000 7f5668822700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5664111870 con 0x7f5664101620 2024-06-10T00:54:19.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.371+0000 7f5661d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5664101fd0 0x7f566419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:19.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.371+0000 7f5661d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5664101fd0 0x7f566419edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42602/0 (socket says 172.21.15.37:42602) 2024-06-10T00:54:19.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.371+0000 7f5661d9b700 1 -- 172.21.15.37:0/111224543 learned_addr learned my addr 172.21.15.37:0/111224543 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:54:19.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.372+0000 7f5662d9d700 1 --2- 172.21.15.37:0/111224543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56641066b0 0x7f5664198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:19.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.372+0000 7f566259c700 1 --2- 172.21.15.37:0/111224543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5664101620 0x7f566419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:19.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.372+0000 7f5661d9b700 1 -- 172.21.15.37:0/111224543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5664101620 msgr2=0x7f566419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:19.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.372+0000 7f5661d9b700 1 --2- 172.21.15.37:0/111224543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5664101620 0x7f566419e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:19.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.372+0000 7f5661d9b700 1 -- 172.21.15.37:0/111224543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56641066b0 msgr2=0x7f5664198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:19.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.372+0000 7f5661d9b700 1 --2- 172.21.15.37:0/111224543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56641066b0 0x7f5664198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:19.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.372+0000 7f5661d9b700 1 -- 172.21.15.37:0/111224543 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56641991d0 con 0x7f5664101fd0 2024-06-10T00:54:19.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.372+0000 7f566259c700 1 --2- 172.21.15.37:0/111224543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5664101620 0x7f566419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:54:19.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.372+0000 7f5661d9b700 1 --2- 172.21.15.37:0/111224543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5664101fd0 0x7f566419edf0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f565800c010 tx=0x7f56580047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:19.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.372+0000 7f56537fe700 1 -- 172.21.15.37:0/111224543 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5658027070 con 0x7f5664101fd0 2024-06-10T00:54:19.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.372+0000 7f5668822700 1 -- 172.21.15.37:0/111224543 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5664199460 con 0x7f5664101fd0 2024-06-10T00:54:19.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.372+0000 7f56537fe700 1 -- 172.21.15.37:0/111224543 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5658003730 con 0x7f5664101fd0 2024-06-10T00:54:19.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.372+0000 7f56537fe700 1 -- 172.21.15.37:0/111224543 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5658022d40 con 0x7f5664101fd0 2024-06-10T00:54:19.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.372+0000 7f5668822700 1 -- 172.21.15.37:0/111224543 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56641a5780 con 0x7f5664101fd0 2024-06-10T00:54:19.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.373+0000 7f5668822700 1 -- 172.21.15.37:0/111224543 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5664069180 con 0x7f5664101fd0 2024-06-10T00:54:19.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.375+0000 7f56537fe700 1 -- 172.21.15.37:0/111224543 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f565802a070 con 0x7f5664101fd0 2024-06-10T00:54:19.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.375+0000 7f56537fe700 1 --2- 172.21.15.37:0/111224543 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f564c0a2d30 0x7f564c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:19.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.375+0000 7f56537fe700 1 -- 172.21.15.37:0/111224543 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f56580d0980 con 0x7f5664101fd0 2024-06-10T00:54:19.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.375+0000 7f566259c700 1 --2- 172.21.15.37:0/111224543 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f564c0a2d30 0x7f564c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:19.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.376+0000 7f566259c700 1 --2- 172.21.15.37:0/111224543 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f564c0a2d30 0x7f564c0a51f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5664199bb0 tx=0x7f5654008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:19.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.379+0000 7f56537fe700 1 -- 172.21.15.37:0/111224543 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5658031070 con 0x7f5664101fd0 2024-06-10T00:54:19.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.586+0000 7f5668822700 1 -- 172.21.15.37:0/111224543 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f566402dca0 con 0x7f5664101fd0 2024-06-10T00:54:19.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.588+0000 7f56537fe700 1 -- 172.21.15.37:0/111224543 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24562 (secure 0 0 0) 0x7f5658013070 con 0x7f5664101fd0 2024-06-10T00:54:19.588 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:54:19.589 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":98,"fsid":"68793514-26c2-11ef-bc9b-c7b262605968","created":"2024-06-10T00:44:24.106713+0000","modified":"2024-06-10T00:53:22.116215+0000","last_up_change":"2024-06-10T00:51:31.562156+0000","last_in_change":"2024-06-10T00:51:11.294301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:47:44.773911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:52:31.684780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:52:34.885503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:52:44.415437+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1df6046a-6801-4950-9193-54303fed169b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6803","nonce":3594704094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6805","nonce":3594704094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6809","nonce":3594704094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6807","nonce":3594704094}]},"public_addr":"172.21.15.37:6803/3594704094","cluster_addr":"172.21.15.37:6805/3594704094","heartbeat_back_addr":"172.21.15.37:6809/3594704094","heartbeat_front_addr":"172.21.15.37:6807/3594704094","state":["exists","up"]},{"osd":1,"uuid":"4e1c8944-94e4-4e46-9efe-56229fe6df1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6801","nonce":3427727155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6803","nonce":3427727155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6807","nonce":3427727155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6805","nonce":3427727155}]},"public_addr":"172.21.15.94:6801/3427727155","cluster_addr":"172.21.15.94:6803/3427727155","heartbeat_back_addr":"172.21.15.94:6807/3427727155","heartbeat_front_addr":"172.21.15.94:6805/3427727155","state":["exists","up"]},{"osd":2,"uuid":"fd2fcedc-4c88-478f-8025-13e5944a27ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6801","nonce":4002498292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6803","nonce":4002498292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6807","nonce":4002498292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6805","nonce":4002498292}]},"public_addr":"172.21.15.167:6801/4002498292","cluster_addr":"172.21.15.167:6803/4002498292","heartbeat_back_addr":"172.21.15.167:6807/4002498292","heartbeat_front_addr":"172.21.15.167:6805/4002498292","state":["exists","up"]},{"osd":3,"uuid":"d0153676-818c-4dcc-83b1-287435a0c8a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6811","nonce":3425469324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6813","nonce":3425469324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6817","nonce":3425469324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6815","nonce":3425469324}]},"public_addr":"172.21.15.37:6811/3425469324","cluster_addr":"172.21.15.37:6813/3425469324","heartbeat_back_addr":"172.21.15.37:6817/3425469324","heartbeat_front_addr":"172.21.15.37:6815/3425469324","state":["exists","up"]},{"osd":4,"uuid":"172c333c-00c0-4e94-8870-acb3b32b2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6809","nonce":1740627431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6811","nonce":1740627431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6815","nonce":1740627431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6813","nonce":1740627431}]},"public_addr":"172.21.15.94:6809/1740627431","cluster_addr":"172.21.15.94:6811/1740627431","heartbeat_back_addr":"172.21.15.94:6815/1740627431","heartbeat_front_addr":"172.21.15.94:6813/1740627431","state":["exists","up"]},{"osd":5,"uuid":"dbce656e-5f93-4bda-baf2-fc3cc03d8282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6809","nonce":221805236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6811","nonce":221805236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6815","nonce":221805236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6813","nonce":221805236}]},"public_addr":"172.21.15.167:6809/221805236","cluster_addr":"172.21.15.167:6811/221805236","heartbeat_back_addr":"172.21.15.167:6815/221805236","heartbeat_front_addr":"172.21.15.167:6813/221805236","state":["exists","up"]},{"osd":6,"uuid":"fcbda266-6d60-43b2-a51c-4e562476d1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6819","nonce":472670875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6821","nonce":472670875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6825","nonce":472670875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6823","nonce":472670875}]},"public_addr":"172.21.15.37:6819/472670875","cluster_addr":"172.21.15.37:6821/472670875","heartbeat_back_addr":"172.21.15.37:6825/472670875","heartbeat_front_addr":"172.21.15.37:6823/472670875","state":["exists","up"]},{"osd":7,"uuid":"ed829ed3-ba85-4e82-ac37-e44e9534d37c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6817","nonce":3932281552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6819","nonce":3932281552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6823","nonce":3932281552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6821","nonce":3932281552}]},"public_addr":"172.21.15.94:6817/3932281552","cluster_addr":"172.21.15.94:6819/3932281552","heartbeat_back_addr":"172.21.15.94:6823/3932281552","heartbeat_front_addr":"172.21.15.94:6821/3932281552","state":["exists","up"]},{"osd":8,"uuid":"7d6267d8-72f4-4704-b554-bafc52d11a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6817","nonce":2290216047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6819","nonce":2290216047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6823","nonce":2290216047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6821","nonce":2290216047}]},"public_addr":"172.21.15.167:6817/2290216047","cluster_addr":"172.21.15.167:6819/2290216047","heartbeat_back_addr":"172.21.15.167:6823/2290216047","heartbeat_front_addr":"172.21.15.167:6821/2290216047","state":["exists","up"]},{"osd":9,"uuid":"210fdc20-0a36-4eeb-bdf2-c0a7f130e19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6827","nonce":2063666428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6829","nonce":2063666428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6833","nonce":2063666428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6831","nonce":2063666428}]},"public_addr":"172.21.15.37:6827/2063666428","cluster_addr":"172.21.15.37:6829/2063666428","heartbeat_back_addr":"172.21.15.37:6833/2063666428","heartbeat_front_addr":"172.21.15.37:6831/2063666428","state":["exists","up"]},{"osd":10,"uuid":"e200b371-b3cf-4b92-a659-f155ca86f0a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6825","nonce":1706618899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6827","nonce":1706618899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6831","nonce":1706618899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6829","nonce":1706618899}]},"public_addr":"172.21.15.94:6825/1706618899","cluster_addr":"172.21.15.94:6827/1706618899","heartbeat_back_addr":"172.21.15.94:6831/1706618899","heartbeat_front_addr":"172.21.15.94:6829/1706618899","state":["exists","up"]},{"osd":11,"uuid":"cfa427df-37a5-4d28-ac2e-af777d607008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6825","nonce":1349892516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6827","nonce":1349892516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6831","nonce":1349892516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6829","nonce":1349892516}]},"public_addr":"172.21.15.167:6825/1349892516","cluster_addr":"172.21.15.167:6827/1349892516","heartbeat_back_addr":"172.21.15.167:6831/1349892516","heartbeat_front_addr":"172.21.15.167:6829/1349892516","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:46:51.846147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:15.763723+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:40.106800+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:05.923204+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:30.501883+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:55.565575+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:20.411360+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:46.071776+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:11.854599+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:37.608818+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:03.176767+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:29.768705+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/4084722397":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/3571803585":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/416587443":"2024-06-11T00:45:20.254342+0000","172.21.15.37:6801/2766692263":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/2026344134":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/2781321886":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6801/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/1110143997":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2766692263":"2024-06-11T00:45:20.254342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:54:19.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.590+0000 7f5668822700 1 -- 172.21.15.37:0/111224543 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f564c0a2d30 msgr2=0x7f564c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:19.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.590+0000 7f5668822700 1 --2- 172.21.15.37:0/111224543 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f564c0a2d30 0x7f564c0a51f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5664199bb0 tx=0x7f5654008040 comp rx=0 tx=0).stop 2024-06-10T00:54:19.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.590+0000 7f5668822700 1 -- 172.21.15.37:0/111224543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5664101fd0 msgr2=0x7f566419edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:19.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.590+0000 7f5668822700 1 --2- 172.21.15.37:0/111224543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5664101fd0 0x7f566419edf0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f565800c010 tx=0x7f56580047b0 comp rx=0 tx=0).stop 2024-06-10T00:54:19.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.590+0000 7f5668822700 1 -- 172.21.15.37:0/111224543 shutdown_connections 2024-06-10T00:54:19.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.590+0000 7f5668822700 1 --2- 172.21.15.37:0/111224543 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f564c0a2d30 0x7f564c0a51f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:19.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.590+0000 7f5668822700 1 --2- 172.21.15.37:0/111224543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56641066b0 0x7f5664198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:19.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.590+0000 7f5668822700 1 --2- 172.21.15.37:0/111224543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5664101620 0x7f566419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:19.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.590+0000 7f5668822700 1 --2- 172.21.15.37:0/111224543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5664101fd0 0x7f566419edf0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:19.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.590+0000 7f5668822700 1 -- 172.21.15.37:0/111224543 >> 172.21.15.37:0/111224543 conn(0x7f5664078340 msgr2=0x7f566406ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:19.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.590+0000 7f5668822700 1 -- 172.21.15.37:0/111224543 shutdown_connections 2024-06-10T00:54:19.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:19.590+0000 7f5668822700 1 -- 172.21.15.37:0/111224543 wait complete. 2024-06-10T00:54:19.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:19 smithi037 ceph-mon[32192]: pgmap v357: 129 pgs: 129 active+clean; 648 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-06-10T00:54:19.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:19 smithi167 ceph-mon[36917]: pgmap v357: 129 pgs: 129 active+clean; 648 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-06-10T00:54:19.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:19 smithi094 ceph-mon[36708]: pgmap v357: 129 pgs: 129 active+clean; 648 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-06-10T00:54:20.385 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-06-10T00:54:20.386 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T00:54:20.642 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:20.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:20 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/111224543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:54:20.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:20 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/111224543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:54:20.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:20 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/111224543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:54:21.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:21 smithi167 ceph-mon[36917]: pgmap v358: 129 pgs: 129 active+clean; 648 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-06-10T00:54:21.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-06-10T00:54:21.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-06-10T00:54:21.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-06-10T00:54:21.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2024-06-10T00:54:21.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]: dispatch 2024-06-10T00:54:21.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:21 smithi094 ceph-mon[36708]: pgmap v358: 129 pgs: 129 active+clean; 648 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-06-10T00:54:21.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-06-10T00:54:21.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-06-10T00:54:21.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-06-10T00:54:21.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2024-06-10T00:54:21.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]: dispatch 2024-06-10T00:54:21.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:21 smithi037 ceph-mon[32192]: pgmap v358: 129 pgs: 129 active+clean; 648 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-06-10T00:54:21.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-06-10T00:54:21.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-06-10T00:54:21.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-06-10T00:54:21.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2024-06-10T00:54:21.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]: dispatch 2024-06-10T00:54:22.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.288+0000 7ff20d6f1700 1 -- 172.21.15.37:0/3944790307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2081066b0 msgr2=0x7ff20810e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:22.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.288+0000 7ff20d6f1700 1 --2- 172.21.15.37:0/3944790307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2081066b0 0x7ff20810e280 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff20000b0d0 tx=0x7ff20000b3e0 comp rx=0 tx=0).stop 2024-06-10T00:54:22.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.288+0000 7ff20d6f1700 1 -- 172.21.15.37:0/3944790307 shutdown_connections 2024-06-10T00:54:22.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.288+0000 7ff20d6f1700 1 --2- 172.21.15.37:0/3944790307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2081066b0 0x7ff20810e280 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:22.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.288+0000 7ff20d6f1700 1 --2- 172.21.15.37:0/3944790307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff208101620 0x7ff208101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:22.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.288+0000 7ff20d6f1700 1 --2- 172.21.15.37:0/3944790307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff208101fd0 0x7ff208105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:22.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.288+0000 7ff20d6f1700 1 -- 172.21.15.37:0/3944790307 >> 172.21.15.37:0/3944790307 conn(0x7ff208078340 msgr2=0x7ff208078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:22.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.288+0000 7ff20d6f1700 1 -- 172.21.15.37:0/3944790307 shutdown_connections 2024-06-10T00:54:22.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.289+0000 7ff20d6f1700 1 -- 172.21.15.37:0/3944790307 wait complete. 2024-06-10T00:54:22.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.289+0000 7ff20d6f1700 1 Processor -- start 2024-06-10T00:54:22.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.290+0000 7ff20d6f1700 1 -- start start 2024-06-10T00:54:22.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.290+0000 7ff20d6f1700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff208101620 0x7ff208199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:22.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.290+0000 7ff20d6f1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff208101fd0 0x7ff208199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:22.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.290+0000 7ff20d6f1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2081066b0 0x7ff2081a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:22.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.290+0000 7ff20d6f1700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff2081116c0 con 0x7ff2081066b0 2024-06-10T00:54:22.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.290+0000 7ff20d6f1700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff208111540 con 0x7ff208101fd0 2024-06-10T00:54:22.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.290+0000 7ff20d6f1700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff208111840 con 0x7ff208101620 2024-06-10T00:54:22.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.290+0000 7ff2077fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2081066b0 0x7ff2081a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:22.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.290+0000 7ff2077fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2081066b0 0x7ff2081a4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43032/0 (socket says 172.21.15.37:43032) 2024-06-10T00:54:22.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.290+0000 7ff2077fe700 1 -- 172.21.15.37:0/1863449869 learned_addr learned my addr 172.21.15.37:0/1863449869 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:54:22.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.290+0000 7ff2067fc700 1 --2- 172.21.15.37:0/1863449869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff208101fd0 0x7ff208199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:22.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.290+0000 7ff206ffd700 1 --2- 172.21.15.37:0/1863449869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff208101620 0x7ff208199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:22.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.290+0000 7ff2077fe700 1 -- 172.21.15.37:0/1863449869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff208101620 msgr2=0x7ff208199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:22.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.290+0000 7ff2077fe700 1 --2- 172.21.15.37:0/1863449869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff208101620 0x7ff208199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:22.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.290+0000 7ff2077fe700 1 -- 172.21.15.37:0/1863449869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff208101fd0 msgr2=0x7ff208199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:22.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.290+0000 7ff2077fe700 1 --2- 172.21.15.37:0/1863449869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff208101fd0 0x7ff208199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:22.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.290+0000 7ff2077fe700 1 -- 172.21.15.37:0/1863449869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2081a5060 con 0x7ff2081066b0 2024-06-10T00:54:22.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.290+0000 7ff206ffd700 1 --2- 172.21.15.37:0/1863449869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff208101620 0x7ff208199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:54:22.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.290+0000 7ff2077fe700 1 --2- 172.21.15.37:0/1863449869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2081066b0 0x7ff2081a4b20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff200006070 tx=0x7ff200004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:22.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.291+0000 7ff1effff700 1 -- 172.21.15.37:0/1863449869 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff200021070 con 0x7ff2081066b0 2024-06-10T00:54:22.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.291+0000 7ff1effff700 1 -- 172.21.15.37:0/1863449869 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff200007b80 con 0x7ff2081066b0 2024-06-10T00:54:22.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.291+0000 7ff20d6f1700 1 -- 172.21.15.37:0/1863449869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2081a5230 con 0x7ff2081066b0 2024-06-10T00:54:22.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.291+0000 7ff1effff700 1 -- 172.21.15.37:0/1863449869 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff200005330 con 0x7ff2081066b0 2024-06-10T00:54:22.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.291+0000 7ff20d6f1700 1 -- 172.21.15.37:0/1863449869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2081a56e0 con 0x7ff2081066b0 2024-06-10T00:54:22.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.292+0000 7ff20d6f1700 1 -- 172.21.15.37:0/1863449869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff208069180 con 0x7ff2081066b0 2024-06-10T00:54:22.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.293+0000 7ff1effff700 1 -- 172.21.15.37:0/1863449869 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff200024070 con 0x7ff2081066b0 2024-06-10T00:54:22.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.294+0000 7ff1effff700 1 --2- 172.21.15.37:0/1863449869 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff1f00a2cb0 0x7ff1f00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:22.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.294+0000 7ff206ffd700 1 --2- 172.21.15.37:0/1863449869 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff1f00a2cb0 0x7ff1f00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:22.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.294+0000 7ff1effff700 1 -- 172.21.15.37:0/1863449869 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9237+0+0 (secure 0 0 0) 0x7ff20000f020 con 0x7ff2081066b0 2024-06-10T00:54:22.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.295+0000 7ff206ffd700 1 --2- 172.21.15.37:0/1863449869 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff1f00a2cb0 0x7ff1f00a5170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8005950 tx=0x7ff1f80058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:22.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.299+0000 7ff1effff700 1 -- 172.21.15.37:0/1863449869 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff20002b020 con 0x7ff2081066b0 2024-06-10T00:54:22.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.504+0000 7ff20d6f1700 1 -- 172.21.15.37:0/1863449869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff20802ddc0 con 0x7ff2081066b0 2024-06-10T00:54:22.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.506+0000 7ff1effff700 1 -- 172.21.15.37:0/1863449869 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+24853 (secure 0 0 0) 0x7ff20009d620 con 0x7ff2081066b0 2024-06-10T00:54:22.507 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:54:22.507 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":99,"fsid":"68793514-26c2-11ef-bc9b-c7b262605968","created":"2024-06-10T00:44:24.106713+0000","modified":"2024-06-10T00:54:21.540789+0000","last_up_change":"2024-06-10T00:51:31.562156+0000","last_in_change":"2024-06-10T00:51:11.294301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:47:44.773911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:52:31.684780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:52:34.885503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:52:44.415437+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1df6046a-6801-4950-9193-54303fed169b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6803","nonce":3594704094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6805","nonce":3594704094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6809","nonce":3594704094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6807","nonce":3594704094}]},"public_addr":"172.21.15.37:6803/3594704094","cluster_addr":"172.21.15.37:6805/3594704094","heartbeat_back_addr":"172.21.15.37:6809/3594704094","heartbeat_front_addr":"172.21.15.37:6807/3594704094","state":["exists","up"]},{"osd":1,"uuid":"4e1c8944-94e4-4e46-9efe-56229fe6df1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6801","nonce":3427727155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6803","nonce":3427727155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6807","nonce":3427727155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6805","nonce":3427727155}]},"public_addr":"172.21.15.94:6801/3427727155","cluster_addr":"172.21.15.94:6803/3427727155","heartbeat_back_addr":"172.21.15.94:6807/3427727155","heartbeat_front_addr":"172.21.15.94:6805/3427727155","state":["exists","up"]},{"osd":2,"uuid":"fd2fcedc-4c88-478f-8025-13e5944a27ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6801","nonce":4002498292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6803","nonce":4002498292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6807","nonce":4002498292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6805","nonce":4002498292}]},"public_addr":"172.21.15.167:6801/4002498292","cluster_addr":"172.21.15.167:6803/4002498292","heartbeat_back_addr":"172.21.15.167:6807/4002498292","heartbeat_front_addr":"172.21.15.167:6805/4002498292","state":["exists","up"]},{"osd":3,"uuid":"d0153676-818c-4dcc-83b1-287435a0c8a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6811","nonce":3425469324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6813","nonce":3425469324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6817","nonce":3425469324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6815","nonce":3425469324}]},"public_addr":"172.21.15.37:6811/3425469324","cluster_addr":"172.21.15.37:6813/3425469324","heartbeat_back_addr":"172.21.15.37:6817/3425469324","heartbeat_front_addr":"172.21.15.37:6815/3425469324","state":["exists","up"]},{"osd":4,"uuid":"172c333c-00c0-4e94-8870-acb3b32b2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6809","nonce":1740627431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6811","nonce":1740627431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6815","nonce":1740627431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6813","nonce":1740627431}]},"public_addr":"172.21.15.94:6809/1740627431","cluster_addr":"172.21.15.94:6811/1740627431","heartbeat_back_addr":"172.21.15.94:6815/1740627431","heartbeat_front_addr":"172.21.15.94:6813/1740627431","state":["exists","up"]},{"osd":5,"uuid":"dbce656e-5f93-4bda-baf2-fc3cc03d8282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6809","nonce":221805236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6811","nonce":221805236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6815","nonce":221805236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6813","nonce":221805236}]},"public_addr":"172.21.15.167:6809/221805236","cluster_addr":"172.21.15.167:6811/221805236","heartbeat_back_addr":"172.21.15.167:6815/221805236","heartbeat_front_addr":"172.21.15.167:6813/221805236","state":["exists","up"]},{"osd":6,"uuid":"fcbda266-6d60-43b2-a51c-4e562476d1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6819","nonce":472670875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6821","nonce":472670875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6825","nonce":472670875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6823","nonce":472670875}]},"public_addr":"172.21.15.37:6819/472670875","cluster_addr":"172.21.15.37:6821/472670875","heartbeat_back_addr":"172.21.15.37:6825/472670875","heartbeat_front_addr":"172.21.15.37:6823/472670875","state":["exists","up"]},{"osd":7,"uuid":"ed829ed3-ba85-4e82-ac37-e44e9534d37c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6817","nonce":3932281552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6819","nonce":3932281552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6823","nonce":3932281552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6821","nonce":3932281552}]},"public_addr":"172.21.15.94:6817/3932281552","cluster_addr":"172.21.15.94:6819/3932281552","heartbeat_back_addr":"172.21.15.94:6823/3932281552","heartbeat_front_addr":"172.21.15.94:6821/3932281552","state":["exists","up"]},{"osd":8,"uuid":"7d6267d8-72f4-4704-b554-bafc52d11a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6817","nonce":2290216047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6819","nonce":2290216047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6823","nonce":2290216047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6821","nonce":2290216047}]},"public_addr":"172.21.15.167:6817/2290216047","cluster_addr":"172.21.15.167:6819/2290216047","heartbeat_back_addr":"172.21.15.167:6823/2290216047","heartbeat_front_addr":"172.21.15.167:6821/2290216047","state":["exists","up"]},{"osd":9,"uuid":"210fdc20-0a36-4eeb-bdf2-c0a7f130e19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6827","nonce":2063666428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6829","nonce":2063666428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6833","nonce":2063666428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6831","nonce":2063666428}]},"public_addr":"172.21.15.37:6827/2063666428","cluster_addr":"172.21.15.37:6829/2063666428","heartbeat_back_addr":"172.21.15.37:6833/2063666428","heartbeat_front_addr":"172.21.15.37:6831/2063666428","state":["exists","up"]},{"osd":10,"uuid":"e200b371-b3cf-4b92-a659-f155ca86f0a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6825","nonce":1706618899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6827","nonce":1706618899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6831","nonce":1706618899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6829","nonce":1706618899}]},"public_addr":"172.21.15.94:6825/1706618899","cluster_addr":"172.21.15.94:6827/1706618899","heartbeat_back_addr":"172.21.15.94:6831/1706618899","heartbeat_front_addr":"172.21.15.94:6829/1706618899","state":["exists","up"]},{"osd":11,"uuid":"cfa427df-37a5-4d28-ac2e-af777d607008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6825","nonce":1349892516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6827","nonce":1349892516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6831","nonce":1349892516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6829","nonce":1349892516}]},"public_addr":"172.21.15.167:6825/1349892516","cluster_addr":"172.21.15.167:6827/1349892516","heartbeat_back_addr":"172.21.15.167:6831/1349892516","heartbeat_front_addr":"172.21.15.167:6829/1349892516","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:46:51.846147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:15.763723+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:40.106800+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:05.923204+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:30.501883+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:55.565575+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:20.411360+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:46.071776+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:11.854599+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:37.608818+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:03.176767+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:29.768705+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/4084722397":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/3571803585":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/416587443":"2024-06-11T00:45:20.254342+0000","172.21.15.37:6801/2766692263":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/2026344134":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/2781321886":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6801/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/1110143997":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2766692263":"2024-06-11T00:45:20.254342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:54:22.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.508+0000 7ff20d6f1700 1 -- 172.21.15.37:0/1863449869 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff1f00a2cb0 msgr2=0x7ff1f00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:22.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.508+0000 7ff20d6f1700 1 --2- 172.21.15.37:0/1863449869 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff1f00a2cb0 0x7ff1f00a5170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8005950 tx=0x7ff1f80058e0 comp rx=0 tx=0).stop 2024-06-10T00:54:22.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.508+0000 7ff20d6f1700 1 -- 172.21.15.37:0/1863449869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2081066b0 msgr2=0x7ff2081a4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:22.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.508+0000 7ff20d6f1700 1 --2- 172.21.15.37:0/1863449869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2081066b0 0x7ff2081a4b20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff200006070 tx=0x7ff200004d00 comp rx=0 tx=0).stop 2024-06-10T00:54:22.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.509+0000 7ff20d6f1700 1 -- 172.21.15.37:0/1863449869 shutdown_connections 2024-06-10T00:54:22.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.509+0000 7ff20d6f1700 1 --2- 172.21.15.37:0/1863449869 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff1f00a2cb0 0x7ff1f00a5170 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:22.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.509+0000 7ff20d6f1700 1 --2- 172.21.15.37:0/1863449869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2081066b0 0x7ff2081a4b20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:22.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.509+0000 7ff20d6f1700 1 --2- 172.21.15.37:0/1863449869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff208101620 0x7ff208199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:22.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.509+0000 7ff20d6f1700 1 --2- 172.21.15.37:0/1863449869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff208101fd0 0x7ff208199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:22.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.509+0000 7ff20d6f1700 1 -- 172.21.15.37:0/1863449869 >> 172.21.15.37:0/1863449869 conn(0x7ff208078340 msgr2=0x7ff2080ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:22.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.509+0000 7ff20d6f1700 1 -- 172.21.15.37:0/1863449869 shutdown_connections 2024-06-10T00:54:22.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:22.509+0000 7ff20d6f1700 1 -- 172.21.15.37:0/1863449869 wait complete. 2024-06-10T00:54:22.821 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:22 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-06-10T00:54:22.821 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:22 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-06-10T00:54:22.821 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:22 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-06-10T00:54:22.821 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:22 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-06-10T00:54:22.822 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:22 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]': finished 2024-06-10T00:54:22.822 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:22 smithi037 ceph-mon[32192]: osdmap e99: 12 total, 12 up, 12 in 2024-06-10T00:54:22.822 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:22 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1863449869' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:54:22.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:22 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-06-10T00:54:22.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:22 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-06-10T00:54:22.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:22 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-06-10T00:54:22.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:22 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-06-10T00:54:22.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:22 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]': finished 2024-06-10T00:54:22.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:22 smithi167 ceph-mon[36917]: osdmap e99: 12 total, 12 up, 12 in 2024-06-10T00:54:22.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:22 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1863449869' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:54:22.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:22 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-06-10T00:54:22.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:22 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-06-10T00:54:22.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:22 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-06-10T00:54:22.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:22 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-06-10T00:54:22.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:22 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]': finished 2024-06-10T00:54:22.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:22 smithi094 ceph-mon[36708]: osdmap e99: 12 total, 12 up, 12 in 2024-06-10T00:54:22.865 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:22 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1863449869' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:54:23.269 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-06-10T00:54:23.269 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-06-10T00:54:23.270 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-06-10T00:54:23.270 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-06-10T00:54:23.271 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-06-10T00:54:23.271 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-06-10T00:54:23.271 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-06-10T00:54:23.272 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-06-10T00:54:23.272 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-06-10T00:54:23.272 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-06-10T00:54:23.273 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-06-10T00:54:23.273 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-06-10T00:54:23.553 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:23 smithi037 ceph-mon[32192]: pgmap v360: 129 pgs: 1 peering, 128 active+clean; 648 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 409 B/s wr, 6 op/s 2024-06-10T00:54:23.554 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:23 smithi037 ceph-mon[32192]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-10T00:54:23.554 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:23 smithi037 ceph-mon[32192]: osdmap e100: 12 total, 12 up, 12 in 2024-06-10T00:54:23.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:23 smithi167 ceph-mon[36917]: pgmap v360: 129 pgs: 1 peering, 128 active+clean; 648 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 409 B/s wr, 6 op/s 2024-06-10T00:54:23.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:23 smithi167 ceph-mon[36917]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-10T00:54:23.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:23 smithi167 ceph-mon[36917]: osdmap e100: 12 total, 12 up, 12 in 2024-06-10T00:54:23.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:23 smithi094 ceph-mon[36708]: pgmap v360: 129 pgs: 1 peering, 128 active+clean; 648 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 409 B/s wr, 6 op/s 2024-06-10T00:54:23.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:23 smithi094 ceph-mon[36708]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-10T00:54:23.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:23 smithi094 ceph-mon[36708]: osdmap e100: 12 total, 12 up, 12 in 2024-06-10T00:54:23.952 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:23.955 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:23.964 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:24.001 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:24.033 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:24.040 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:24.300 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:24.300 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:25.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:25 smithi167 ceph-mon[36917]: pgmap v362: 129 pgs: 1 peering, 128 active+clean; 648 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T00:54:25.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:25 smithi094 ceph-mon[36708]: pgmap v362: 129 pgs: 1 peering, 128 active+clean; 648 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T00:54:25.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:25 smithi037 ceph-mon[32192]: pgmap v362: 129 pgs: 1 peering, 128 active+clean; 648 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T00:54:27.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:27 smithi167 ceph-mon[36917]: pgmap v363: 129 pgs: 1 peering, 128 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 383 B/s wr, 5 op/s 2024-06-10T00:54:27.855 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:27 smithi037 ceph-mon[32192]: pgmap v363: 129 pgs: 1 peering, 128 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 383 B/s wr, 5 op/s 2024-06-10T00:54:27.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:27 smithi094 ceph-mon[36708]: pgmap v363: 129 pgs: 1 peering, 128 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 383 B/s wr, 5 op/s 2024-06-10T00:54:28.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:28 smithi037 ceph-mon[32192]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-10T00:54:28.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:28 smithi037 ceph-mon[32192]: Cluster is now healthy 2024-06-10T00:54:28.866 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:28 smithi094 ceph-mon[36708]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-10T00:54:28.866 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:28 smithi094 ceph-mon[36708]: Cluster is now healthy 2024-06-10T00:54:29.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:28 smithi167 ceph-mon[36917]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-10T00:54:29.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:28 smithi167 ceph-mon[36917]: Cluster is now healthy 2024-06-10T00:54:29.743 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:29 smithi037 ceph-mon[32192]: pgmap v364: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 383 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:54:29.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.535+0000 7fc559ae2700 1 -- 172.21.15.37:0/1119926468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc55406a170 msgr2=0x7fc55406a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:29.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.535+0000 7fc559ae2700 1 --2- 172.21.15.37:0/1119926468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc55406a170 0x7fc55406a5d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc5480099c0 tx=0x7fc548009cd0 comp rx=0 tx=0).stop 2024-06-10T00:54:29.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.535+0000 7fc559ae2700 1 -- 172.21.15.37:0/1119926468 shutdown_connections 2024-06-10T00:54:29.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.535+0000 7fc559ae2700 1 --2- 172.21.15.37:0/1119926468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc55406ab10 0x7fc55410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:29.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.535+0000 7fc559ae2700 1 --2- 172.21.15.37:0/1119926468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5541084e0 0x7fc5541088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:29.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.535+0000 7fc559ae2700 1 --2- 172.21.15.37:0/1119926468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc55406a170 0x7fc55406a5d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.535+0000 7fc559ae2700 1 -- 172.21.15.37:0/1119926468 >> 172.21.15.37:0/1119926468 conn(0x7fc5540782c0 msgr2=0x7fc5540786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.535+0000 7fc559ae2700 1 -- 172.21.15.37:0/1119926468 shutdown_connections 2024-06-10T00:54:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.535+0000 7fc559ae2700 1 -- 172.21.15.37:0/1119926468 wait complete. 2024-06-10T00:54:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.535+0000 7fc559ae2700 1 Processor -- start 2024-06-10T00:54:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.536+0000 7fc559ae2700 1 -- start start 2024-06-10T00:54:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.536+0000 7fc559ae2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc55406a170 0x7fc554101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.536+0000 7fc559ae2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc55406ab10 0x7fc5541019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.536+0000 7fc559ae2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5541084e0 0x7fc554101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.536+0000 7fc559ae2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc554112790 con 0x7fc55406ab10 2024-06-10T00:54:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.536+0000 7fc559ae2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc554112610 con 0x7fc55406a170 2024-06-10T00:54:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.536+0000 7fc559ae2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc554112910 con 0x7fc5541084e0 2024-06-10T00:54:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.536+0000 7fc552ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc55406ab10 0x7fc5541019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.536+0000 7fc552ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc55406ab10 0x7fc5541019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43054/0 (socket says 172.21.15.37:43054) 2024-06-10T00:54:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.536+0000 7fc552ffd700 1 -- 172.21.15.37:0/3003574396 learned_addr learned my addr 172.21.15.37:0/3003574396 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:54:29.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.536+0000 7fc553fff700 1 --2- 172.21.15.37:0/3003574396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5541084e0 0x7fc554101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:29.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.536+0000 7fc5537fe700 1 --2- 172.21.15.37:0/3003574396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc55406a170 0x7fc554101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:29.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.536+0000 7fc552ffd700 1 -- 172.21.15.37:0/3003574396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5541084e0 msgr2=0x7fc554101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:29.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.536+0000 7fc552ffd700 1 --2- 172.21.15.37:0/3003574396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5541084e0 0x7fc554101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:29.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.536+0000 7fc552ffd700 1 -- 172.21.15.37:0/3003574396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc55406a170 msgr2=0x7fc554101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:29.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.536+0000 7fc552ffd700 1 --2- 172.21.15.37:0/3003574396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc55406a170 0x7fc554101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:29.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.536+0000 7fc552ffd700 1 -- 172.21.15.37:0/3003574396 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5541a7180 con 0x7fc55406ab10 2024-06-10T00:54:29.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.537+0000 7fc552ffd700 1 --2- 172.21.15.37:0/3003574396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc55406ab10 0x7fc5541019a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc548009990 tx=0x7fc548009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:29.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.537+0000 7fc550ff9700 1 -- 172.21.15.37:0/3003574396 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc548026070 con 0x7fc55406ab10 2024-06-10T00:54:29.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.537+0000 7fc550ff9700 1 -- 172.21.15.37:0/3003574396 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc548003780 con 0x7fc55406ab10 2024-06-10T00:54:29.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.537+0000 7fc559ae2700 1 -- 172.21.15.37:0/3003574396 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5541a7410 con 0x7fc55406ab10 2024-06-10T00:54:29.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.537+0000 7fc550ff9700 1 -- 172.21.15.37:0/3003574396 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54800fc90 con 0x7fc55406ab10 2024-06-10T00:54:29.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.537+0000 7fc559ae2700 1 -- 172.21.15.37:0/3003574396 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5541a7860 con 0x7fc55406ab10 2024-06-10T00:54:29.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.538+0000 7fc559ae2700 1 -- 172.21.15.37:0/3003574396 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc554062ba0 con 0x7fc55406ab10 2024-06-10T00:54:29.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.539+0000 7fc550ff9700 1 -- 172.21.15.37:0/3003574396 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc548029070 con 0x7fc55406ab10 2024-06-10T00:54:29.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.540+0000 7fc550ff9700 1 --2- 172.21.15.37:0/3003574396 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc53c0a2cb0 0x7fc53c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:29.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.540+0000 7fc550ff9700 1 -- 172.21.15.37:0/3003574396 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fc5480d0770 con 0x7fc55406ab10 2024-06-10T00:54:29.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.540+0000 7fc5537fe700 1 --2- 172.21.15.37:0/3003574396 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc53c0a2cb0 0x7fc53c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:29.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.540+0000 7fc550ff9700 1 --2- 172.21.15.37:0/3003574396 >> [v2:172.21.15.167:6816/2290216047,v1:172.21.15.167:6817/2290216047] conn(0x7fc53c0ae090 0x7fc53c0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:29.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.540+0000 7fc550ff9700 1 -- 172.21.15.37:0/3003574396 --> [v2:172.21.15.167:6816/2290216047,v1:172.21.15.167:6817/2290216047] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc5480043d0 con 0x7fc53c0ae090 2024-06-10T00:54:29.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.540+0000 7fc550ff9700 1 -- 172.21.15.37:0/3003574396 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fc5480d0b20 con 0x7fc55406ab10 2024-06-10T00:54:29.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.540+0000 7fc5537fe700 1 --2- 172.21.15.37:0/3003574396 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc53c0a2cb0 0x7fc53c0a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc544005fd0 tx=0x7fc54400af20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:29.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.541+0000 7fc553fff700 1 --2- 172.21.15.37:0/3003574396 >> [v2:172.21.15.167:6816/2290216047,v1:172.21.15.167:6817/2290216047] conn(0x7fc53c0ae090 0x7fc53c0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:29.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.541+0000 7fc553fff700 1 --2- 172.21.15.37:0/3003574396 >> [v2:172.21.15.167:6816/2290216047,v1:172.21.15.167:6817/2290216047] conn(0x7fc53c0ae090 0x7fc53c0b0530 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:29.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.542+0000 7fc550ff9700 1 -- 172.21.15.37:0/3003574396 <== osd.8 v2:172.21.15.167:6816/2290216047 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc5480043d0 con 0x7fc53c0ae090 2024-06-10T00:54:29.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.559+0000 7fc559ae2700 1 -- 172.21.15.37:0/3003574396 --> [v2:172.21.15.167:6816/2290216047,v1:172.21.15.167:6817/2290216047] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc554062fa0 con 0x7fc53c0ae090 2024-06-10T00:54:29.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.560+0000 7fc550ff9700 1 -- 172.21.15.37:0/3003574396 <== osd.8 v2:172.21.15.167:6816/2290216047 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc554062fa0 con 0x7fc53c0ae090 2024-06-10T00:54:29.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.560+0000 7fc559ae2700 1 -- 172.21.15.37:0/3003574396 >> [v2:172.21.15.167:6816/2290216047,v1:172.21.15.167:6817/2290216047] conn(0x7fc53c0ae090 msgr2=0x7fc53c0b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:29.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.560+0000 7fc559ae2700 1 --2- 172.21.15.37:0/3003574396 >> [v2:172.21.15.167:6816/2290216047,v1:172.21.15.167:6817/2290216047] conn(0x7fc53c0ae090 0x7fc53c0b0530 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:29.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.560+0000 7fc559ae2700 1 -- 172.21.15.37:0/3003574396 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc53c0a2cb0 msgr2=0x7fc53c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:29.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.560+0000 7fc559ae2700 1 --2- 172.21.15.37:0/3003574396 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc53c0a2cb0 0x7fc53c0a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc544005fd0 tx=0x7fc54400af20 comp rx=0 tx=0).stop 2024-06-10T00:54:29.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.560+0000 7fc559ae2700 1 -- 172.21.15.37:0/3003574396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc55406ab10 msgr2=0x7fc5541019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:29.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.560+0000 7fc559ae2700 1 --2- 172.21.15.37:0/3003574396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc55406ab10 0x7fc5541019a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc548009990 tx=0x7fc548009f80 comp rx=0 tx=0).stop 2024-06-10T00:54:29.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.560+0000 7fc553fff700 1 -- 172.21.15.37:0/3003574396 reap_dead start 2024-06-10T00:54:29.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.560+0000 7fc559ae2700 1 -- 172.21.15.37:0/3003574396 shutdown_connections 2024-06-10T00:54:29.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.560+0000 7fc559ae2700 1 -- 172.21.15.37:0/3003574396 >> 172.21.15.37:0/3003574396 conn(0x7fc5540782c0 msgr2=0x7fc5540ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:29.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.560+0000 7fc559ae2700 1 -- 172.21.15.37:0/3003574396 shutdown_connections 2024-06-10T00:54:29.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:29.560+0000 7fc559ae2700 1 -- 172.21.15.37:0/3003574396 wait complete. 2024-06-10T00:54:29.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:29 smithi094 ceph-mon[36708]: pgmap v364: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 383 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:54:30.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:29 smithi167 ceph-mon[36917]: pgmap v364: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 383 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:54:30.837 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:30.838 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:30.838 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:30.838 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:31.888 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:31 smithi037 ceph-mon[32192]: pgmap v365: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 348 B/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:54:32.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:31 smithi167 ceph-mon[36917]: pgmap v365: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 348 B/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:54:32.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:31 smithi094 ceph-mon[36708]: pgmap v365: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 348 B/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:54:33.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.542+0000 7f4638b7f700 1 -- 172.21.15.37:0/1664718719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4634101390 msgr2=0x7f46341017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:33.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.542+0000 7f4638b7f700 1 --2- 172.21.15.37:0/1664718719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4634101390 0x7f46341017f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4628009990 tx=0x7f4628009ca0 comp rx=0 tx=0).stop 2024-06-10T00:54:33.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.542+0000 7f4638b7f700 1 -- 172.21.15.37:0/1664718719 shutdown_connections 2024-06-10T00:54:33.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.542+0000 7f4638b7f700 1 --2- 172.21.15.37:0/1664718719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4634101d30 0x7f463410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:33.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.542+0000 7f4638b7f700 1 --2- 172.21.15.37:0/1664718719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46341073d0 0x7f46341077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:33.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.542+0000 7f4638b7f700 1 --2- 172.21.15.37:0/1664718719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4634101390 0x7f46341017f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:33.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.542+0000 7f4638b7f700 1 -- 172.21.15.37:0/1664718719 >> 172.21.15.37:0/1664718719 conn(0x7f46340782c0 msgr2=0x7f46340786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:33.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.542+0000 7f4638b7f700 1 -- 172.21.15.37:0/1664718719 shutdown_connections 2024-06-10T00:54:33.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.542+0000 7f4638b7f700 1 -- 172.21.15.37:0/1664718719 wait complete. 2024-06-10T00:54:33.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.543+0000 7f4638b7f700 1 Processor -- start 2024-06-10T00:54:33.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.543+0000 7f4638b7f700 1 -- start start 2024-06-10T00:54:33.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.543+0000 7f4638b7f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4634101390 0x7f463419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:33.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.543+0000 7f4638b7f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4634101d30 0x7f463419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:33.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.543+0000 7f4638b7f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46341073d0 0x7f4634198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.543+0000 7f4638b7f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f46341116d0 con 0x7f46341073d0 2024-06-10T00:54:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.543+0000 7f4638b7f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4634111550 con 0x7f4634101390 2024-06-10T00:54:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.543+0000 7f4638b7f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4634111850 con 0x7f4634101d30 2024-06-10T00:54:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.543+0000 7f4632d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46341073d0 0x7f4634198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.543+0000 7f4632d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46341073d0 0x7f4634198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49822/0 (socket says 172.21.15.37:49822) 2024-06-10T00:54:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.543+0000 7f4632d9d700 1 -- 172.21.15.37:0/1703710017 learned_addr learned my addr 172.21.15.37:0/1703710017 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:54:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.543+0000 7f463259c700 1 --2- 172.21.15.37:0/1703710017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4634101390 0x7f463419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.543+0000 7f4631d9b700 1 --2- 172.21.15.37:0/1703710017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4634101d30 0x7f463419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.544+0000 7f4632d9d700 1 -- 172.21.15.37:0/1703710017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4634101d30 msgr2=0x7f463419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.544+0000 7f4632d9d700 1 --2- 172.21.15.37:0/1703710017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4634101d30 0x7f463419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.544+0000 7f4632d9d700 1 -- 172.21.15.37:0/1703710017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4634101390 msgr2=0x7f463419e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.544+0000 7f4632d9d700 1 --2- 172.21.15.37:0/1703710017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4634101390 0x7f463419e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.544+0000 7f4632d9d700 1 -- 172.21.15.37:0/1703710017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46341990e0 con 0x7f46341073d0 2024-06-10T00:54:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.544+0000 7f463259c700 1 --2- 172.21.15.37:0/1703710017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4634101390 0x7f463419e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:54:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.544+0000 7f4632d9d700 1 --2- 172.21.15.37:0/1703710017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46341073d0 0x7f4634198980 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f462c00b970 tx=0x7f462c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:33.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.544+0000 7f46237fe700 1 -- 172.21.15.37:0/1703710017 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c019070 con 0x7f46341073d0 2024-06-10T00:54:33.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.544+0000 7f4638b7f700 1 -- 172.21.15.37:0/1703710017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46341993d0 con 0x7f46341073d0 2024-06-10T00:54:33.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.544+0000 7f46237fe700 1 -- 172.21.15.37:0/1703710017 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f462c014440 con 0x7f46341073d0 2024-06-10T00:54:33.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.544+0000 7f46237fe700 1 -- 172.21.15.37:0/1703710017 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c0075d0 con 0x7f46341073d0 2024-06-10T00:54:33.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.544+0000 7f4638b7f700 1 -- 172.21.15.37:0/1703710017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46341a5690 con 0x7f46341073d0 2024-06-10T00:54:33.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.545+0000 7f46217fa700 1 -- 172.21.15.37:0/1703710017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4634062be0 con 0x7f46341073d0 2024-06-10T00:54:33.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.547+0000 7f46237fe700 1 -- 172.21.15.37:0/1703710017 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f462c01c070 con 0x7f46341073d0 2024-06-10T00:54:33.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.547+0000 7f46237fe700 1 --2- 172.21.15.37:0/1703710017 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f461c0a2cb0 0x7f461c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:33.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.548+0000 7f46237fe700 1 -- 172.21.15.37:0/1703710017 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f462c0cf120 con 0x7f46341073d0 2024-06-10T00:54:33.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.548+0000 7f463259c700 1 --2- 172.21.15.37:0/1703710017 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f461c0a2cb0 0x7f461c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:33.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.548+0000 7f46237fe700 1 --2- 172.21.15.37:0/1703710017 >> [v2:172.21.15.37:6818/472670875,v1:172.21.15.37:6819/472670875] conn(0x7f461c0ae090 0x7f461c0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:33.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.548+0000 7f46237fe700 1 -- 172.21.15.37:0/1703710017 --> [v2:172.21.15.37:6818/472670875,v1:172.21.15.37:6819/472670875] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f462c004860 con 0x7f461c0ae090 2024-06-10T00:54:33.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.548+0000 7f46237fe700 1 -- 172.21.15.37:0/1703710017 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f462c0cf570 con 0x7f46341073d0 2024-06-10T00:54:33.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.548+0000 7f4631d9b700 1 --2- 172.21.15.37:0/1703710017 >> [v2:172.21.15.37:6818/472670875,v1:172.21.15.37:6819/472670875] conn(0x7f461c0ae090 0x7f461c0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:33.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.548+0000 7f463259c700 1 --2- 172.21.15.37:0/1703710017 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f461c0a2cb0 0x7f461c0a5170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4634199ac0 tx=0x7f462400a680 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:33.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.548+0000 7f4631d9b700 1 --2- 172.21.15.37:0/1703710017 >> [v2:172.21.15.37:6818/472670875,v1:172.21.15.37:6819/472670875] conn(0x7f461c0ae090 0x7f461c0b0530 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:33.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.549+0000 7f46237fe700 1 -- 172.21.15.37:0/1703710017 <== osd.6 v2:172.21.15.37:6818/472670875 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f462c004860 con 0x7f461c0ae090 2024-06-10T00:54:33.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.566+0000 7f46217fa700 1 -- 172.21.15.37:0/1703710017 --> [v2:172.21.15.37:6818/472670875,v1:172.21.15.37:6819/472670875] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4634063060 con 0x7f461c0ae090 2024-06-10T00:54:33.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.566+0000 7f46237fe700 1 -- 172.21.15.37:0/1703710017 <== osd.6 v2:172.21.15.37:6818/472670875 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4634063060 con 0x7f461c0ae090 2024-06-10T00:54:33.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.566+0000 7f46217fa700 1 -- 172.21.15.37:0/1703710017 >> [v2:172.21.15.37:6818/472670875,v1:172.21.15.37:6819/472670875] conn(0x7f461c0ae090 msgr2=0x7f461c0b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:33.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.566+0000 7f46217fa700 1 --2- 172.21.15.37:0/1703710017 >> [v2:172.21.15.37:6818/472670875,v1:172.21.15.37:6819/472670875] conn(0x7f461c0ae090 0x7f461c0b0530 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.566+0000 7f46217fa700 1 -- 172.21.15.37:0/1703710017 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f461c0a2cb0 msgr2=0x7f461c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.566+0000 7f46217fa700 1 --2- 172.21.15.37:0/1703710017 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f461c0a2cb0 0x7f461c0a5170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4634199ac0 tx=0x7f462400a680 comp rx=0 tx=0).stop 2024-06-10T00:54:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.566+0000 7f46217fa700 1 -- 172.21.15.37:0/1703710017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46341073d0 msgr2=0x7f4634198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.566+0000 7f46217fa700 1 --2- 172.21.15.37:0/1703710017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46341073d0 0x7f4634198980 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f462c00b970 tx=0x7f462c00bd30 comp rx=0 tx=0).stop 2024-06-10T00:54:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.567+0000 7f4632d9d700 1 -- 172.21.15.37:0/1703710017 reap_dead start 2024-06-10T00:54:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.567+0000 7f46217fa700 1 -- 172.21.15.37:0/1703710017 shutdown_connections 2024-06-10T00:54:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.567+0000 7f46217fa700 1 -- 172.21.15.37:0/1703710017 >> 172.21.15.37:0/1703710017 conn(0x7f46340782c0 msgr2=0x7f46341005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.567+0000 7f46217fa700 1 -- 172.21.15.37:0/1703710017 shutdown_connections 2024-06-10T00:54:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:33.567+0000 7f46217fa700 1 -- 172.21.15.37:0/1703710017 wait complete. 2024-06-10T00:54:33.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:33 smithi167 ceph-mon[36917]: pgmap v366: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:54:33.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:33 smithi037 ceph-mon[32192]: pgmap v366: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:54:33.916 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:33 smithi094 ceph-mon[36708]: pgmap v366: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:54:35.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:35 smithi167 ceph-mon[36917]: pgmap v367: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 259 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:54:35.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:35 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:54:35.843 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:35 smithi094 ceph-mon[36708]: pgmap v367: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 259 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:54:35.843 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:35 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:54:35.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:35 smithi037 ceph-mon[32192]: pgmap v367: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 259 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:54:35.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:35 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:54:36.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.866+0000 7f60ee18f700 1 -- 172.21.15.37:0/3768302985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60e8101fd0 msgr2=0x7f60e8105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:36.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.866+0000 7f60ee18f700 1 --2- 172.21.15.37:0/3768302985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60e8101fd0 0x7f60e8105fd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f60dc0099c0 tx=0x7f60dc009cd0 comp rx=0 tx=0).stop 2024-06-10T00:54:36.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.867+0000 7f60ee18f700 1 -- 172.21.15.37:0/3768302985 shutdown_connections 2024-06-10T00:54:36.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.867+0000 7f60ee18f700 1 --2- 172.21.15.37:0/3768302985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60e81066b0 0x7f60e810e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:36.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.867+0000 7f60ee18f700 1 --2- 172.21.15.37:0/3768302985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60e8101620 0x7f60e8101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:36.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.867+0000 7f60ee18f700 1 --2- 172.21.15.37:0/3768302985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60e8101fd0 0x7f60e8105fd0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:36.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.867+0000 7f60ee18f700 1 -- 172.21.15.37:0/3768302985 >> 172.21.15.37:0/3768302985 conn(0x7f60e8078340 msgr2=0x7f60e8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:36.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.867+0000 7f60ee18f700 1 -- 172.21.15.37:0/3768302985 shutdown_connections 2024-06-10T00:54:36.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.867+0000 7f60ee18f700 1 -- 172.21.15.37:0/3768302985 wait complete. 2024-06-10T00:54:36.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.867+0000 7f60ee18f700 1 Processor -- start 2024-06-10T00:54:36.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.867+0000 7f60ee18f700 1 -- start start 2024-06-10T00:54:36.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.868+0000 7f60ee18f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60e8101620 0x7f60e8103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:36.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.868+0000 7f60ee18f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60e8101fd0 0x7f60e81035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:36.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.868+0000 7f60ee18f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60e81066b0 0x7f60e8103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:36.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.868+0000 7f60ee18f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f60e8111690 con 0x7f60e81066b0 2024-06-10T00:54:36.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.868+0000 7f60ee18f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f60e8111510 con 0x7f60e8101fd0 2024-06-10T00:54:36.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.868+0000 7f60ee18f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f60e8111810 con 0x7f60e8101620 2024-06-10T00:54:36.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.868+0000 7f60e7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60e81066b0 0x7f60e8103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:36.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.868+0000 7f60e77fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60e8101620 0x7f60e8103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:36.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.868+0000 7f60e77fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60e8101620 0x7f60e8103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:53656/0 (socket says 172.21.15.37:53656) 2024-06-10T00:54:36.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.868+0000 7f60e7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60e81066b0 0x7f60e8103af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49834/0 (socket says 172.21.15.37:49834) 2024-06-10T00:54:36.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.868+0000 7f60e6ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60e8101fd0 0x7f60e81035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:36.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.868+0000 7f60e77fe700 1 -- 172.21.15.37:0/1761351513 learned_addr learned my addr 172.21.15.37:0/1761351513 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:54:36.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.868+0000 7f60e7fff700 1 -- 172.21.15.37:0/1761351513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60e8101620 msgr2=0x7f60e8103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:36.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.868+0000 7f60e7fff700 1 --2- 172.21.15.37:0/1761351513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60e8101620 0x7f60e8103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:36.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.868+0000 7f60e7fff700 1 -- 172.21.15.37:0/1761351513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60e8101fd0 msgr2=0x7f60e81035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:36.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.868+0000 7f60e7fff700 1 --2- 172.21.15.37:0/1761351513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60e8101fd0 0x7f60e81035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:36.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.868+0000 7f60e7fff700 1 -- 172.21.15.37:0/1761351513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e81a71c0 con 0x7f60e81066b0 2024-06-10T00:54:36.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.868+0000 7f60e7fff700 1 --2- 172.21.15.37:0/1761351513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60e81066b0 0x7f60e8103af0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f60e0007f10 tx=0x7f60e000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:36.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.869+0000 7f60e4ff9700 1 -- 172.21.15.37:0/1761351513 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e0004db0 con 0x7f60e81066b0 2024-06-10T00:54:36.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.869+0000 7f60e4ff9700 1 -- 172.21.15.37:0/1761351513 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60e0003a00 con 0x7f60e81066b0 2024-06-10T00:54:36.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.869+0000 7f60ee18f700 1 -- 172.21.15.37:0/1761351513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e81a74b0 con 0x7f60e81066b0 2024-06-10T00:54:36.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.869+0000 7f60e4ff9700 1 -- 172.21.15.37:0/1761351513 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e0004680 con 0x7f60e81066b0 2024-06-10T00:54:36.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.869+0000 7f60ee18f700 1 -- 172.21.15.37:0/1761351513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e81a79c0 con 0x7f60e81066b0 2024-06-10T00:54:36.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.869+0000 7f60ee18f700 1 -- 172.21.15.37:0/1761351513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f60e8062ba0 con 0x7f60e81066b0 2024-06-10T00:54:36.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.871+0000 7f60e4ff9700 1 -- 172.21.15.37:0/1761351513 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f60e001c020 con 0x7f60e81066b0 2024-06-10T00:54:36.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.871+0000 7f60e4ff9700 1 --2- 172.21.15.37:0/1761351513 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f60d00a2d30 0x7f60d00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:36.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.872+0000 7f60e77fe700 1 --2- 172.21.15.37:0/1761351513 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f60d00a2d30 0x7f60d00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:36.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.872+0000 7f60e4ff9700 1 -- 172.21.15.37:0/1761351513 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f60e00cf970 con 0x7f60e81066b0 2024-06-10T00:54:36.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.872+0000 7f60e4ff9700 1 --2- 172.21.15.37:0/1761351513 >> [v2:172.21.15.37:6810/3425469324,v1:172.21.15.37:6811/3425469324] conn(0x7f60d00ae110 0x7f60d00b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:36.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.872+0000 7f60e4ff9700 1 -- 172.21.15.37:0/1761351513 --> [v2:172.21.15.37:6810/3425469324,v1:172.21.15.37:6811/3425469324] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f60e00020b0 con 0x7f60d00ae110 2024-06-10T00:54:36.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.872+0000 7f60e4ff9700 1 -- 172.21.15.37:0/1761351513 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f60e00cfd20 con 0x7f60e81066b0 2024-06-10T00:54:36.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.872+0000 7f60e6ffd700 1 --2- 172.21.15.37:0/1761351513 >> [v2:172.21.15.37:6810/3425469324,v1:172.21.15.37:6811/3425469324] conn(0x7f60d00ae110 0x7f60d00b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:36.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.872+0000 7f60e77fe700 1 --2- 172.21.15.37:0/1761351513 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f60d00a2d30 0x7f60d00a51f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f60d8009d60 tx=0x7f60d8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:36.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.872+0000 7f60e6ffd700 1 --2- 172.21.15.37:0/1761351513 >> [v2:172.21.15.37:6810/3425469324,v1:172.21.15.37:6811/3425469324] conn(0x7f60d00ae110 0x7f60d00b05b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:36.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.873+0000 7f60e4ff9700 1 -- 172.21.15.37:0/1761351513 <== osd.3 v2:172.21.15.37:6810/3425469324 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f60e00020b0 con 0x7f60d00ae110 2024-06-10T00:54:36.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.889+0000 7f60ee18f700 1 -- 172.21.15.37:0/1761351513 --> [v2:172.21.15.37:6810/3425469324,v1:172.21.15.37:6811/3425469324] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f60e8063020 con 0x7f60d00ae110 2024-06-10T00:54:36.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.890+0000 7f60e4ff9700 1 -- 172.21.15.37:0/1761351513 <== osd.3 v2:172.21.15.37:6810/3425469324 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f60e8063020 con 0x7f60d00ae110 2024-06-10T00:54:36.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.890+0000 7f60ee18f700 1 -- 172.21.15.37:0/1761351513 >> [v2:172.21.15.37:6810/3425469324,v1:172.21.15.37:6811/3425469324] conn(0x7f60d00ae110 msgr2=0x7f60d00b05b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:36.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.890+0000 7f60ee18f700 1 --2- 172.21.15.37:0/1761351513 >> [v2:172.21.15.37:6810/3425469324,v1:172.21.15.37:6811/3425469324] conn(0x7f60d00ae110 0x7f60d00b05b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:36.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.890+0000 7f60ee18f700 1 -- 172.21.15.37:0/1761351513 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f60d00a2d30 msgr2=0x7f60d00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:36.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.890+0000 7f60ee18f700 1 --2- 172.21.15.37:0/1761351513 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f60d00a2d30 0x7f60d00a51f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f60d8009d60 tx=0x7f60d8009260 comp rx=0 tx=0).stop 2024-06-10T00:54:36.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.890+0000 7f60ee18f700 1 -- 172.21.15.37:0/1761351513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60e81066b0 msgr2=0x7f60e8103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:36.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.890+0000 7f60ee18f700 1 --2- 172.21.15.37:0/1761351513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60e81066b0 0x7f60e8103af0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f60e0007f10 tx=0x7f60e000d3b0 comp rx=0 tx=0).stop 2024-06-10T00:54:36.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.890+0000 7f60e7fff700 1 -- 172.21.15.37:0/1761351513 reap_dead start 2024-06-10T00:54:36.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.890+0000 7f60ee18f700 1 -- 172.21.15.37:0/1761351513 shutdown_connections 2024-06-10T00:54:36.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.890+0000 7f60ee18f700 1 -- 172.21.15.37:0/1761351513 >> 172.21.15.37:0/1761351513 conn(0x7f60e8078340 msgr2=0x7f60e806aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:36.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.890+0000 7f60ee18f700 1 -- 172.21.15.37:0/1761351513 shutdown_connections 2024-06-10T00:54:36.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:36.890+0000 7f60ee18f700 1 -- 172.21.15.37:0/1761351513 wait complete. 2024-06-10T00:54:37.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:37 smithi037 ceph-mon[32192]: pgmap v368: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:54:37.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:37 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:54:37.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:37 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:54:37.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:37 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:54:37.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:37 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:54:37.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:37 smithi167 ceph-mon[36917]: pgmap v368: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:54:37.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:37 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:54:37.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:37 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:54:37.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:37 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:54:37.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:37 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:54:37.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:37 smithi094 ceph-mon[36708]: pgmap v368: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:54:37.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:37 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:54:37.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:37 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:54:37.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:37 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:54:37.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:37 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:54:38.500 INFO:teuthology.orchestra.run.smithi037.stdout:201863462977 2024-06-10T00:54:38.501 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-06-10T00:54:38.813 INFO:teuthology.orchestra.run.smithi037.stdout:257698037814 2024-06-10T00:54:38.814 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-06-10T00:54:39.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.438+0000 7f191c800700 1 -- 172.21.15.37:0/4008277319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1918106600 msgr2=0x7f191810e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:39.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.438+0000 7f191c800700 1 --2- 172.21.15.37:0/4008277319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1918106600 0x7f191810e1d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f191000b0d0 tx=0x7f191000b3e0 comp rx=0 tx=0).stop 2024-06-10T00:54:39.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.438+0000 7f191c800700 1 -- 172.21.15.37:0/4008277319 shutdown_connections 2024-06-10T00:54:39.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.438+0000 7f191c800700 1 --2- 172.21.15.37:0/4008277319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1918106600 0x7f191810e1d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:39.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.438+0000 7f191c800700 1 --2- 172.21.15.37:0/4008277319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1918101520 0x7f1918101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:39.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.438+0000 7f191c800700 1 --2- 172.21.15.37:0/4008277319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1918101ed0 0x7f1918105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:39.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.438+0000 7f191c800700 1 -- 172.21.15.37:0/4008277319 >> 172.21.15.37:0/4008277319 conn(0x7f19180fcdb0 msgr2=0x7f19180ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:39.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.439+0000 7f191c800700 1 -- 172.21.15.37:0/4008277319 shutdown_connections 2024-06-10T00:54:39.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.439+0000 7f191c800700 1 -- 172.21.15.37:0/4008277319 wait complete. 2024-06-10T00:54:39.441 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:39 smithi037 ceph-mon[32192]: pgmap v369: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:54:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.439+0000 7f191c800700 1 Processor -- start 2024-06-10T00:54:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.440+0000 7f191c800700 1 -- start start 2024-06-10T00:54:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.440+0000 7f191c800700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1918101520 0x7f19180fd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.440+0000 7f191c800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1918101ed0 0x7f19180fdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.440+0000 7f191c800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1918106600 0x7f19180fe0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.440+0000 7f191c800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1918111610 con 0x7f1918101ed0 2024-06-10T00:54:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.440+0000 7f191c800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1918111490 con 0x7f1918106600 2024-06-10T00:54:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.440+0000 7f191c800700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1918111790 con 0x7f1918101520 2024-06-10T00:54:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.440+0000 7f191659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1918101ed0 0x7f19180fdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.440+0000 7f191659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1918101ed0 0x7f19180fdb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49848/0 (socket says 172.21.15.37:49848) 2024-06-10T00:54:39.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.440+0000 7f191659c700 1 -- 172.21.15.37:0/1724908286 learned_addr learned my addr 172.21.15.37:0/1724908286 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:54:39.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.440+0000 7f1916d9d700 1 --2- 172.21.15.37:0/1724908286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1918101520 0x7f19180fd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:39.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.440+0000 7f191759e700 1 --2- 172.21.15.37:0/1724908286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1918106600 0x7f19180fe0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:39.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.441+0000 7f191659c700 1 -- 172.21.15.37:0/1724908286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1918101520 msgr2=0x7f19180fd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:39.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.441+0000 7f191659c700 1 --2- 172.21.15.37:0/1724908286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1918101520 0x7f19180fd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:39.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.441+0000 7f191659c700 1 -- 172.21.15.37:0/1724908286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1918106600 msgr2=0x7f19180fe0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:39.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.441+0000 7f191659c700 1 --2- 172.21.15.37:0/1724908286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1918106600 0x7f19180fe0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:39.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.441+0000 7f191659c700 1 -- 172.21.15.37:0/1724908286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19181a6fc0 con 0x7f1918101ed0 2024-06-10T00:54:39.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.441+0000 7f191659c700 1 --2- 172.21.15.37:0/1724908286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1918101ed0 0x7f19180fdb90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f190c00d950 tx=0x7f190c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:39.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.441+0000 7f18fffff700 1 -- 172.21.15.37:0/1724908286 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c017070 con 0x7f1918101ed0 2024-06-10T00:54:39.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.441+0000 7f191c800700 1 -- 172.21.15.37:0/1724908286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19181a72b0 con 0x7f1918101ed0 2024-06-10T00:54:39.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.441+0000 7f18fffff700 1 -- 172.21.15.37:0/1724908286 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f190c005020 con 0x7f1918101ed0 2024-06-10T00:54:39.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.441+0000 7f18fffff700 1 -- 172.21.15.37:0/1724908286 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c00fbc0 con 0x7f1918101ed0 2024-06-10T00:54:39.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.441+0000 7f191c800700 1 -- 172.21.15.37:0/1724908286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19181a77a0 con 0x7f1918101ed0 2024-06-10T00:54:39.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.442+0000 7f191c800700 1 -- 172.21.15.37:0/1724908286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1918062ba0 con 0x7f1918101ed0 2024-06-10T00:54:39.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.444+0000 7f18fffff700 1 -- 172.21.15.37:0/1724908286 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f190c00fd60 con 0x7f1918101ed0 2024-06-10T00:54:39.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.445+0000 7f18fffff700 1 --2- 172.21.15.37:0/1724908286 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f19000a2cb0 0x7f19000a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:39.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.445+0000 7f18fffff700 1 -- 172.21.15.37:0/1724908286 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f190c0cf170 con 0x7f1918101ed0 2024-06-10T00:54:39.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.445+0000 7f1916d9d700 1 --2- 172.21.15.37:0/1724908286 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f19000a2cb0 0x7f19000a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:39.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.445+0000 7f18fffff700 1 --2- 172.21.15.37:0/1724908286 >> [v2:172.21.15.167:6808/221805236,v1:172.21.15.167:6809/221805236] conn(0x7f19000ae090 0x7f19000b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:39.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.445+0000 7f18fffff700 1 -- 172.21.15.37:0/1724908286 --> [v2:172.21.15.167:6808/221805236,v1:172.21.15.167:6809/221805236] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f19000b0c00 con 0x7f19000ae090 2024-06-10T00:54:39.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.445+0000 7f18fffff700 1 -- 172.21.15.37:0/1724908286 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f190c0cf520 con 0x7f1918101ed0 2024-06-10T00:54:39.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.446+0000 7f191759e700 1 --2- 172.21.15.37:0/1724908286 >> [v2:172.21.15.167:6808/221805236,v1:172.21.15.167:6809/221805236] conn(0x7f19000ae090 0x7f19000b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:39.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.446+0000 7f1916d9d700 1 --2- 172.21.15.37:0/1724908286 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f19000a2cb0 0x7f19000a5170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f190800ac50 tx=0x7f190800a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:39.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.446+0000 7f191759e700 1 --2- 172.21.15.37:0/1724908286 >> [v2:172.21.15.167:6808/221805236,v1:172.21.15.167:6809/221805236] conn(0x7f19000ae090 0x7f19000b0530 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:39.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.447+0000 7f18fffff700 1 -- 172.21.15.37:0/1724908286 <== osd.5 v2:172.21.15.167:6808/221805236 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f19000b0c00 con 0x7f19000ae090 2024-06-10T00:54:39.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.464+0000 7f191c800700 1 -- 172.21.15.37:0/1724908286 --> [v2:172.21.15.167:6808/221805236,v1:172.21.15.167:6809/221805236] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1918063020 con 0x7f19000ae090 2024-06-10T00:54:39.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.465+0000 7f18fffff700 1 -- 172.21.15.37:0/1724908286 <== osd.5 v2:172.21.15.167:6808/221805236 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1918063020 con 0x7f19000ae090 2024-06-10T00:54:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.465+0000 7f191c800700 1 -- 172.21.15.37:0/1724908286 >> [v2:172.21.15.167:6808/221805236,v1:172.21.15.167:6809/221805236] conn(0x7f19000ae090 msgr2=0x7f19000b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.465+0000 7f191c800700 1 --2- 172.21.15.37:0/1724908286 >> [v2:172.21.15.167:6808/221805236,v1:172.21.15.167:6809/221805236] conn(0x7f19000ae090 0x7f19000b0530 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.465+0000 7f191c800700 1 -- 172.21.15.37:0/1724908286 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f19000a2cb0 msgr2=0x7f19000a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.465+0000 7f191c800700 1 --2- 172.21.15.37:0/1724908286 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f19000a2cb0 0x7f19000a5170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f190800ac50 tx=0x7f190800a380 comp rx=0 tx=0).stop 2024-06-10T00:54:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.465+0000 7f191c800700 1 -- 172.21.15.37:0/1724908286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1918101ed0 msgr2=0x7f19180fdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.465+0000 7f191c800700 1 --2- 172.21.15.37:0/1724908286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1918101ed0 0x7f19180fdb90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f190c00d950 tx=0x7f190c00dc60 comp rx=0 tx=0).stop 2024-06-10T00:54:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.465+0000 7f191759e700 1 -- 172.21.15.37:0/1724908286 reap_dead start 2024-06-10T00:54:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.465+0000 7f191c800700 1 -- 172.21.15.37:0/1724908286 shutdown_connections 2024-06-10T00:54:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.465+0000 7f191c800700 1 -- 172.21.15.37:0/1724908286 >> 172.21.15.37:0/1724908286 conn(0x7f19180fcdb0 msgr2=0x7f191806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.465+0000 7f191c800700 1 -- 172.21.15.37:0/1724908286 shutdown_connections 2024-06-10T00:54:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.465+0000 7f191c800700 1 -- 172.21.15.37:0/1724908286 wait complete. 2024-06-10T00:54:39.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.744+0000 7f03485a8700 1 -- 172.21.15.37:0/1834098979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0340107840 msgr2=0x7f034010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:39.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.744+0000 7f03485a8700 1 --2- 172.21.15.37:0/1834098979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0340107840 0x7f034010f3c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f033800b0d0 tx=0x7f033800b3e0 comp rx=0 tx=0).stop 2024-06-10T00:54:39.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.744+0000 7f03485a8700 1 -- 172.21.15.37:0/1834098979 shutdown_connections 2024-06-10T00:54:39.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.744+0000 7f03485a8700 1 --2- 172.21.15.37:0/1834098979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0340107840 0x7f034010f3c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:39.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.744+0000 7f03485a8700 1 --2- 172.21.15.37:0/1834098979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0340102740 0x7f0340102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:39.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.744+0000 7f03485a8700 1 --2- 172.21.15.37:0/1834098979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03401030f0 0x7f0340107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:39.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.744+0000 7f03485a8700 1 -- 172.21.15.37:0/1834098979 >> 172.21.15.37:0/1834098979 conn(0x7f03400795f0 msgr2=0x7f0340079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:39.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.744+0000 7f03485a8700 1 -- 172.21.15.37:0/1834098979 shutdown_connections 2024-06-10T00:54:39.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.744+0000 7f03485a8700 1 -- 172.21.15.37:0/1834098979 wait complete. 2024-06-10T00:54:39.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.745+0000 7f03485a8700 1 Processor -- start 2024-06-10T00:54:39.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.745+0000 7f03485a8700 1 -- start start 2024-06-10T00:54:39.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.745+0000 7f03485a8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0340102740 0x7f034019e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:39.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.745+0000 7f03485a8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03401030f0 0x7f034019ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:39.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.745+0000 7f03485a8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0340107840 0x7f0340198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:39.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.745+0000 7f03485a8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0340112760 con 0x7f0340102740 2024-06-10T00:54:39.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.745+0000 7f03485a8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f03401125e0 con 0x7f0340107840 2024-06-10T00:54:39.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.745+0000 7f03485a8700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f03401128e0 con 0x7f03401030f0 2024-06-10T00:54:39.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.745+0000 7f0346344700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0340102740 0x7f034019e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:39.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.745+0000 7f0346344700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0340102740 0x7f034019e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49874/0 (socket says 172.21.15.37:49874) 2024-06-10T00:54:39.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.745+0000 7f0346344700 1 -- 172.21.15.37:0/3646557194 learned_addr learned my addr 172.21.15.37:0/3646557194 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:54:39.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.745+0000 7f0345b43700 1 --2- 172.21.15.37:0/3646557194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03401030f0 0x7f034019ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:39.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.746+0000 7f0346b45700 1 --2- 172.21.15.37:0/3646557194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0340107840 0x7f0340198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:39.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.746+0000 7f0346344700 1 -- 172.21.15.37:0/3646557194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03401030f0 msgr2=0x7f034019ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:39.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.746+0000 7f0346344700 1 --2- 172.21.15.37:0/3646557194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03401030f0 0x7f034019ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:39.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.746+0000 7f0346344700 1 -- 172.21.15.37:0/3646557194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0340107840 msgr2=0x7f0340198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:39.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.746+0000 7f0346344700 1 --2- 172.21.15.37:0/3646557194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0340107840 0x7f0340198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:39.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.746+0000 7f0346344700 1 -- 172.21.15.37:0/3646557194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03401990c0 con 0x7f0340102740 2024-06-10T00:54:39.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.746+0000 7f0346b45700 1 --2- 172.21.15.37:0/3646557194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0340107840 0x7f0340198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:54:39.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.746+0000 7f0346344700 1 --2- 172.21.15.37:0/3646557194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0340102740 0x7f034019e750 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f033c00c9b0 tx=0x7f033c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:39.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.746+0000 7f03377fe700 1 -- 172.21.15.37:0/3646557194 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c007b00 con 0x7f0340102740 2024-06-10T00:54:39.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.746+0000 7f03377fe700 1 -- 172.21.15.37:0/3646557194 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f033c007ca0 con 0x7f0340102740 2024-06-10T00:54:39.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.746+0000 7f03485a8700 1 -- 172.21.15.37:0/3646557194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03401993b0 con 0x7f0340102740 2024-06-10T00:54:39.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.746+0000 7f03377fe700 1 -- 172.21.15.37:0/3646557194 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c018730 con 0x7f0340102740 2024-06-10T00:54:39.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.746+0000 7f03485a8700 1 -- 172.21.15.37:0/3646557194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03401a5620 con 0x7f0340102740 2024-06-10T00:54:39.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.747+0000 7f03485a8700 1 -- 172.21.15.37:0/3646557194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0340062c20 con 0x7f0340102740 2024-06-10T00:54:39.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.748+0000 7f03377fe700 1 -- 172.21.15.37:0/3646557194 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f033c0188d0 con 0x7f0340102740 2024-06-10T00:54:39.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.749+0000 7f03377fe700 1 --2- 172.21.15.37:0/3646557194 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f032c0a2cb0 0x7f032c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:39.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.749+0000 7f03377fe700 1 -- 172.21.15.37:0/3646557194 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f033c0cfa00 con 0x7f0340102740 2024-06-10T00:54:39.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.749+0000 7f03377fe700 1 --2- 172.21.15.37:0/3646557194 >> [v2:172.21.15.94:6816/3932281552,v1:172.21.15.94:6817/3932281552] conn(0x7f032c0ae090 0x7f032c0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:39.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.749+0000 7f03377fe700 1 -- 172.21.15.37:0/3646557194 --> [v2:172.21.15.94:6816/3932281552,v1:172.21.15.94:6817/3932281552] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f033c00a0a0 con 0x7f032c0ae090 2024-06-10T00:54:39.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.749+0000 7f03377fe700 1 -- 172.21.15.37:0/3646557194 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f033c0cfe50 con 0x7f0340102740 2024-06-10T00:54:39.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.750+0000 7f0345b43700 1 --2- 172.21.15.37:0/3646557194 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f032c0a2cb0 0x7f032c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:39.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.750+0000 7f0346b45700 1 --2- 172.21.15.37:0/3646557194 >> [v2:172.21.15.94:6816/3932281552,v1:172.21.15.94:6817/3932281552] conn(0x7f032c0ae090 0x7f032c0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:39.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.750+0000 7f0345b43700 1 --2- 172.21.15.37:0/3646557194 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f032c0a2cb0 0x7f032c0a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0340108bd0 tx=0x7f03300073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:39.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.751+0000 7f0346b45700 1 --2- 172.21.15.37:0/3646557194 >> [v2:172.21.15.94:6816/3932281552,v1:172.21.15.94:6817/3932281552] conn(0x7f032c0ae090 0x7f032c0b0530 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:39.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.751+0000 7f03377fe700 1 -- 172.21.15.37:0/3646557194 <== osd.7 v2:172.21.15.94:6816/3932281552 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f033c00a0a0 con 0x7f032c0ae090 2024-06-10T00:54:39.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.769+0000 7f03485a8700 1 -- 172.21.15.37:0/3646557194 --> [v2:172.21.15.94:6816/3932281552,v1:172.21.15.94:6817/3932281552] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0340063020 con 0x7f032c0ae090 2024-06-10T00:54:39.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.770+0000 7f03377fe700 1 -- 172.21.15.37:0/3646557194 <== osd.7 v2:172.21.15.94:6816/3932281552 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0340063020 con 0x7f032c0ae090 2024-06-10T00:54:39.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.770+0000 7f03485a8700 1 -- 172.21.15.37:0/3646557194 >> [v2:172.21.15.94:6816/3932281552,v1:172.21.15.94:6817/3932281552] conn(0x7f032c0ae090 msgr2=0x7f032c0b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:39.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.770+0000 7f03485a8700 1 --2- 172.21.15.37:0/3646557194 >> [v2:172.21.15.94:6816/3932281552,v1:172.21.15.94:6817/3932281552] conn(0x7f032c0ae090 0x7f032c0b0530 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:39.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.771+0000 7f03485a8700 1 -- 172.21.15.37:0/3646557194 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f032c0a2cb0 msgr2=0x7f032c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:39.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.771+0000 7f03485a8700 1 --2- 172.21.15.37:0/3646557194 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f032c0a2cb0 0x7f032c0a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0340108bd0 tx=0x7f03300073d0 comp rx=0 tx=0).stop 2024-06-10T00:54:39.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.771+0000 7f03485a8700 1 -- 172.21.15.37:0/3646557194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0340102740 msgr2=0x7f034019e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:39.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.771+0000 7f03485a8700 1 --2- 172.21.15.37:0/3646557194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0340102740 0x7f034019e750 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f033c00c9b0 tx=0x7f033c00ccc0 comp rx=0 tx=0).stop 2024-06-10T00:54:39.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.771+0000 7f0346b45700 1 -- 172.21.15.37:0/3646557194 reap_dead start 2024-06-10T00:54:39.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.771+0000 7f03485a8700 1 -- 172.21.15.37:0/3646557194 shutdown_connections 2024-06-10T00:54:39.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.771+0000 7f03485a8700 1 -- 172.21.15.37:0/3646557194 >> 172.21.15.37:0/3646557194 conn(0x7f03400795f0 msgr2=0x7f0340100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:39.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.771+0000 7f03485a8700 1 -- 172.21.15.37:0/3646557194 shutdown_connections 2024-06-10T00:54:39.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:39.771+0000 7f03485a8700 1 -- 172.21.15.37:0/3646557194 wait complete. 2024-06-10T00:54:39.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:39 smithi167 ceph-mon[36917]: pgmap v369: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:54:39.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:39 smithi094 ceph-mon[36708]: pgmap v369: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-06-10T00:54:41.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:41 smithi037 ceph-mon[32192]: pgmap v370: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T00:54:41.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.693+0000 7effc54c2700 1 -- 172.21.15.37:0/3259159328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc0101390 msgr2=0x7effc0101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:41.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.693+0000 7effc54c2700 1 --2- 172.21.15.37:0/3259159328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc0101390 0x7effc0101810 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7effb4009990 tx=0x7effb4009ca0 comp rx=0 tx=0).stop 2024-06-10T00:54:41.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.693+0000 7effc54c2700 1 -- 172.21.15.37:0/3259159328 shutdown_connections 2024-06-10T00:54:41.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.693+0000 7effc54c2700 1 --2- 172.21.15.37:0/3259159328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effc0101d50 0x7effc010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:41.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.693+0000 7effc54c2700 1 --2- 172.21.15.37:0/3259159328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7effc0107400 0x7effc01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:41.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.693+0000 7effc54c2700 1 --2- 172.21.15.37:0/3259159328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc0101390 0x7effc0101810 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:41.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.693+0000 7effc54c2700 1 -- 172.21.15.37:0/3259159328 >> 172.21.15.37:0/3259159328 conn(0x7effc0078340 msgr2=0x7effc0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:41.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.693+0000 7effc54c2700 1 -- 172.21.15.37:0/3259159328 shutdown_connections 2024-06-10T00:54:41.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.693+0000 7effc54c2700 1 -- 172.21.15.37:0/3259159328 wait complete. 2024-06-10T00:54:41.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.693+0000 7effc54c2700 1 Processor -- start 2024-06-10T00:54:41.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.693+0000 7effc54c2700 1 -- start start 2024-06-10T00:54:41.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.694+0000 7effc54c2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7effc0101390 0x7effc019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:41.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.694+0000 7effc54c2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc0101d50 0x7effc019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:41.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.694+0000 7effc54c2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effc0107400 0x7effc019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:41.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.694+0000 7effc54c2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7effc01116c0 con 0x7effc0101d50 2024-06-10T00:54:41.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.694+0000 7effc54c2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7effc0111540 con 0x7effc0101390 2024-06-10T00:54:41.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.694+0000 7effc54c2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7effc0111840 con 0x7effc0107400 2024-06-10T00:54:41.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.694+0000 7effbe7fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc0101d50 0x7effc019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:41.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.694+0000 7effbe7fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc0101d50 0x7effc019b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49928/0 (socket says 172.21.15.37:49928) 2024-06-10T00:54:41.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.694+0000 7effbe7fc700 1 -- 172.21.15.37:0/3403237701 learned_addr learned my addr 172.21.15.37:0/3403237701 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:54:41.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.694+0000 7effbf7fe700 1 --2- 172.21.15.37:0/3403237701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effc0107400 0x7effc019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:41.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.694+0000 7effbeffd700 1 --2- 172.21.15.37:0/3403237701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7effc0101390 0x7effc019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:41.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.694+0000 7effbe7fc700 1 -- 172.21.15.37:0/3403237701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effc0107400 msgr2=0x7effc019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:41.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.694+0000 7effbe7fc700 1 --2- 172.21.15.37:0/3403237701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effc0107400 0x7effc019f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:41.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.694+0000 7effbe7fc700 1 -- 172.21.15.37:0/3403237701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7effc0101390 msgr2=0x7effc019ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:41.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.694+0000 7effbe7fc700 1 --2- 172.21.15.37:0/3403237701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7effc0101390 0x7effc019ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:41.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.694+0000 7effbe7fc700 1 -- 172.21.15.37:0/3403237701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc019fe80 con 0x7effc0101d50 2024-06-10T00:54:41.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.694+0000 7effbf7fe700 1 --2- 172.21.15.37:0/3403237701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effc0107400 0x7effc019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:54:41.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.695+0000 7effbe7fc700 1 --2- 172.21.15.37:0/3403237701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc0101d50 0x7effc019b3b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7effb4000c00 tx=0x7effb400ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:41.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.695+0000 7effa7fff700 1 -- 172.21.15.37:0/3403237701 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb4027070 con 0x7effc0101d50 2024-06-10T00:54:41.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.695+0000 7effc54c2700 1 -- 172.21.15.37:0/3403237701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc01a0110 con 0x7effc0101d50 2024-06-10T00:54:41.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.695+0000 7effa7fff700 1 -- 172.21.15.37:0/3403237701 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effb4003d70 con 0x7effc0101d50 2024-06-10T00:54:41.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.695+0000 7effa7fff700 1 -- 172.21.15.37:0/3403237701 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb400f460 con 0x7effc0101d50 2024-06-10T00:54:41.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.695+0000 7effc54c2700 1 -- 172.21.15.37:0/3403237701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effc01a79f0 con 0x7effc0101d50 2024-06-10T00:54:41.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.696+0000 7effc54c2700 1 -- 172.21.15.37:0/3403237701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7effc00ffde0 con 0x7effc0101d50 2024-06-10T00:54:41.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.697+0000 7effa7fff700 1 -- 172.21.15.37:0/3403237701 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7effb402a070 con 0x7effc0101d50 2024-06-10T00:54:41.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.698+0000 7effa7fff700 1 --2- 172.21.15.37:0/3403237701 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7effa80a2cb0 0x7effa80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:41.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.698+0000 7effa7fff700 1 -- 172.21.15.37:0/3403237701 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7effb40d10b0 con 0x7effc0101d50 2024-06-10T00:54:41.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.698+0000 7effbeffd700 1 --2- 172.21.15.37:0/3403237701 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7effa80a2cb0 0x7effa80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:41.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.698+0000 7effa7fff700 1 --2- 172.21.15.37:0/3403237701 >> [v2:172.21.15.94:6808/1740627431,v1:172.21.15.94:6809/1740627431] conn(0x7effa80ae090 0x7effa80b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:41.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.698+0000 7effa7fff700 1 -- 172.21.15.37:0/3403237701 --> [v2:172.21.15.94:6808/1740627431,v1:172.21.15.94:6809/1740627431] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7effb4001d30 con 0x7effa80ae090 2024-06-10T00:54:41.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.698+0000 7effa7fff700 1 -- 172.21.15.37:0/3403237701 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7effb40d1500 con 0x7effc0101d50 2024-06-10T00:54:41.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.699+0000 7effbeffd700 1 --2- 172.21.15.37:0/3403237701 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7effa80a2cb0 0x7effa80a5170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7effb0009be0 tx=0x7effb0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:41.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.699+0000 7effbf7fe700 1 --2- 172.21.15.37:0/3403237701 >> [v2:172.21.15.94:6808/1740627431,v1:172.21.15.94:6809/1740627431] conn(0x7effa80ae090 0x7effa80b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:41.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.699+0000 7effbf7fe700 1 --2- 172.21.15.37:0/3403237701 >> [v2:172.21.15.94:6808/1740627431,v1:172.21.15.94:6809/1740627431] conn(0x7effa80ae090 0x7effa80b0530 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:41.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.700+0000 7effa7fff700 1 -- 172.21.15.37:0/3403237701 <== osd.4 v2:172.21.15.94:6808/1740627431 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7effb4001d30 con 0x7effa80ae090 2024-06-10T00:54:41.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.716+0000 7effc54c2700 1 -- 172.21.15.37:0/3403237701 --> [v2:172.21.15.94:6808/1740627431,v1:172.21.15.94:6809/1740627431] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7effc0102ab0 con 0x7effa80ae090 2024-06-10T00:54:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.717+0000 7effa7fff700 1 -- 172.21.15.37:0/3403237701 <== osd.4 v2:172.21.15.94:6808/1740627431 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7effc0102ab0 con 0x7effa80ae090 2024-06-10T00:54:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.718+0000 7effc54c2700 1 -- 172.21.15.37:0/3403237701 >> [v2:172.21.15.94:6808/1740627431,v1:172.21.15.94:6809/1740627431] conn(0x7effa80ae090 msgr2=0x7effa80b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.718+0000 7effc54c2700 1 --2- 172.21.15.37:0/3403237701 >> [v2:172.21.15.94:6808/1740627431,v1:172.21.15.94:6809/1740627431] conn(0x7effa80ae090 0x7effa80b0530 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.718+0000 7effc54c2700 1 -- 172.21.15.37:0/3403237701 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7effa80a2cb0 msgr2=0x7effa80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.718+0000 7effc54c2700 1 --2- 172.21.15.37:0/3403237701 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7effa80a2cb0 0x7effa80a5170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7effb0009be0 tx=0x7effb0009260 comp rx=0 tx=0).stop 2024-06-10T00:54:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.718+0000 7effc54c2700 1 -- 172.21.15.37:0/3403237701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc0101d50 msgr2=0x7effc019b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.718+0000 7effc54c2700 1 --2- 172.21.15.37:0/3403237701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc0101d50 0x7effc019b3b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7effb4000c00 tx=0x7effb400ba90 comp rx=0 tx=0).stop 2024-06-10T00:54:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.718+0000 7effbf7fe700 1 -- 172.21.15.37:0/3403237701 reap_dead start 2024-06-10T00:54:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.718+0000 7effc54c2700 1 -- 172.21.15.37:0/3403237701 shutdown_connections 2024-06-10T00:54:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.718+0000 7effc54c2700 1 -- 172.21.15.37:0/3403237701 >> 172.21.15.37:0/3403237701 conn(0x7effc0078340 msgr2=0x7effc00fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.718+0000 7effc54c2700 1 -- 172.21.15.37:0/3403237701 shutdown_connections 2024-06-10T00:54:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.718+0000 7effc54c2700 1 -- 172.21.15.37:0/3403237701 wait complete. 2024-06-10T00:54:41.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:41 smithi167 ceph-mon[36917]: pgmap v370: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T00:54:41.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:41 smithi094 ceph-mon[36708]: pgmap v370: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T00:54:42.509 INFO:teuthology.orchestra.run.smithi037.stdout:146028888140 2024-06-10T00:54:42.510 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-06-10T00:54:42.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.794+0000 7fa7a1e55700 1 -- 172.21.15.37:0/4115406611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa79c101390 msgr2=0x7fa79c101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:42.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.794+0000 7fa7a1e55700 1 --2- 172.21.15.37:0/4115406611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa79c101390 0x7fa79c101810 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa784009990 tx=0x7fa784009ca0 comp rx=0 tx=0).stop 2024-06-10T00:54:42.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.794+0000 7fa7a1e55700 1 -- 172.21.15.37:0/4115406611 shutdown_connections 2024-06-10T00:54:42.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.794+0000 7fa7a1e55700 1 --2- 172.21.15.37:0/4115406611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa79c101d50 0x7fa79c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:42.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.794+0000 7fa7a1e55700 1 --2- 172.21.15.37:0/4115406611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa79c107400 0x7fa79c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:42.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.794+0000 7fa7a1e55700 1 --2- 172.21.15.37:0/4115406611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa79c101390 0x7fa79c101810 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:42.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.794+0000 7fa7a1e55700 1 -- 172.21.15.37:0/4115406611 >> 172.21.15.37:0/4115406611 conn(0x7fa79c078340 msgr2=0x7fa79c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:42.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.795+0000 7fa7a1e55700 1 -- 172.21.15.37:0/4115406611 shutdown_connections 2024-06-10T00:54:42.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.795+0000 7fa7a1e55700 1 -- 172.21.15.37:0/4115406611 wait complete. 2024-06-10T00:54:42.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.795+0000 7fa7a1e55700 1 Processor -- start 2024-06-10T00:54:42.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.795+0000 7fa7a1e55700 1 -- start start 2024-06-10T00:54:42.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.796+0000 7fa7a1e55700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa79c101390 0x7fa79c199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:42.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.796+0000 7fa7a1e55700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa79c101d50 0x7fa79c199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:42.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.796+0000 7fa7a1e55700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa79c107400 0x7fa79c1a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:42.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.796+0000 7fa7a1e55700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c111670 con 0x7fa79c107400 2024-06-10T00:54:42.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.796+0000 7fa7a1e55700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c1114f0 con 0x7fa79c101390 2024-06-10T00:54:42.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.796+0000 7fa7a1e55700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c1117f0 con 0x7fa79c101d50 2024-06-10T00:54:42.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.796+0000 7fa79bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa79c107400 0x7fa79c1a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:42.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.796+0000 7fa79b7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa79c101390 0x7fa79c199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:42.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.796+0000 7fa79bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa79c107400 0x7fa79c1a4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54520/0 (socket says 172.21.15.37:54520) 2024-06-10T00:54:42.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.796+0000 7fa79b7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa79c101390 0x7fa79c199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:40434/0 (socket says 172.21.15.37:40434) 2024-06-10T00:54:42.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.796+0000 7fa79bfff700 1 -- 172.21.15.37:0/4201693999 learned_addr learned my addr 172.21.15.37:0/4201693999 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:54:42.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.796+0000 7fa79affd700 1 --2- 172.21.15.37:0/4201693999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa79c101d50 0x7fa79c199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:42.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.796+0000 7fa79bfff700 1 -- 172.21.15.37:0/4201693999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa79c101d50 msgr2=0x7fa79c199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:42.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.796+0000 7fa79bfff700 1 --2- 172.21.15.37:0/4201693999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa79c101d50 0x7fa79c199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:42.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.796+0000 7fa79bfff700 1 -- 172.21.15.37:0/4201693999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa79c101390 msgr2=0x7fa79c199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:42.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.796+0000 7fa79bfff700 1 --2- 172.21.15.37:0/4201693999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa79c101390 0x7fa79c199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:42.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.796+0000 7fa79bfff700 1 -- 172.21.15.37:0/4201693999 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa79c1a5030 con 0x7fa79c107400 2024-06-10T00:54:42.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.796+0000 7fa79bfff700 1 --2- 172.21.15.37:0/4201693999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa79c107400 0x7fa79c1a4af0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa794007d70 tx=0x7fa79400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:42.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.796+0000 7fa798ff9700 1 -- 172.21.15.37:0/4201693999 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa794016020 con 0x7fa79c107400 2024-06-10T00:54:42.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.796+0000 7fa7a1e55700 1 -- 172.21.15.37:0/4201693999 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa79c1a5260 con 0x7fa79c107400 2024-06-10T00:54:42.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.797+0000 7fa7a1e55700 1 -- 172.21.15.37:0/4201693999 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa79c1a5770 con 0x7fa79c107400 2024-06-10T00:54:42.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.797+0000 7fa798ff9700 1 -- 172.21.15.37:0/4201693999 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa79400ac90 con 0x7fa79c107400 2024-06-10T00:54:42.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.797+0000 7fa798ff9700 1 -- 172.21.15.37:0/4201693999 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79401b660 con 0x7fa79c107400 2024-06-10T00:54:42.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.797+0000 7fa7a1e55700 1 -- 172.21.15.37:0/4201693999 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa79c062b60 con 0x7fa79c107400 2024-06-10T00:54:42.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.799+0000 7fa798ff9700 1 -- 172.21.15.37:0/4201693999 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa794019020 con 0x7fa79c107400 2024-06-10T00:54:42.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.800+0000 7fa798ff9700 1 --2- 172.21.15.37:0/4201693999 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa7880a2cb0 0x7fa7880a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:42.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.800+0000 7fa79b7fe700 1 --2- 172.21.15.37:0/4201693999 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa7880a2cb0 0x7fa7880a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:42.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.800+0000 7fa798ff9700 1 -- 172.21.15.37:0/4201693999 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fa7940cf4f0 con 0x7fa79c107400 2024-06-10T00:54:42.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.800+0000 7fa798ff9700 1 --2- 172.21.15.37:0/4201693999 >> [v2:172.21.15.167:6800/4002498292,v1:172.21.15.167:6801/4002498292] conn(0x7fa7880ae090 0x7fa7880b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:42.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.800+0000 7fa798ff9700 1 -- 172.21.15.37:0/4201693999 --> [v2:172.21.15.167:6800/4002498292,v1:172.21.15.167:6801/4002498292] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa7880b0c00 con 0x7fa7880ae090 2024-06-10T00:54:42.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.800+0000 7fa798ff9700 1 -- 172.21.15.37:0/4201693999 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fa7940cf940 con 0x7fa79c107400 2024-06-10T00:54:42.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.800+0000 7fa79b7fe700 1 --2- 172.21.15.37:0/4201693999 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa7880a2cb0 0x7fa7880a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa78c005950 tx=0x7fa78c00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:42.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.800+0000 7fa79affd700 1 --2- 172.21.15.37:0/4201693999 >> [v2:172.21.15.167:6800/4002498292,v1:172.21.15.167:6801/4002498292] conn(0x7fa7880ae090 0x7fa7880b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:42.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.801+0000 7fa79affd700 1 --2- 172.21.15.37:0/4201693999 >> [v2:172.21.15.167:6800/4002498292,v1:172.21.15.167:6801/4002498292] conn(0x7fa7880ae090 0x7fa7880b0530 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:42.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.802+0000 7fa798ff9700 1 -- 172.21.15.37:0/4201693999 <== osd.2 v2:172.21.15.167:6800/4002498292 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa7880b0c00 con 0x7fa7880ae090 2024-06-10T00:54:42.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.818+0000 7fa7a1e55700 1 -- 172.21.15.37:0/4201693999 --> [v2:172.21.15.167:6800/4002498292,v1:172.21.15.167:6801/4002498292] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa79c062f60 con 0x7fa7880ae090 2024-06-10T00:54:42.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.819+0000 7fa798ff9700 1 -- 172.21.15.37:0/4201693999 <== osd.2 v2:172.21.15.167:6800/4002498292 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa79c062f60 con 0x7fa7880ae090 2024-06-10T00:54:42.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.819+0000 7fa7a1e55700 1 -- 172.21.15.37:0/4201693999 >> [v2:172.21.15.167:6800/4002498292,v1:172.21.15.167:6801/4002498292] conn(0x7fa7880ae090 msgr2=0x7fa7880b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:42.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.819+0000 7fa7a1e55700 1 --2- 172.21.15.37:0/4201693999 >> [v2:172.21.15.167:6800/4002498292,v1:172.21.15.167:6801/4002498292] conn(0x7fa7880ae090 0x7fa7880b0530 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:42.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.819+0000 7fa7a1e55700 1 -- 172.21.15.37:0/4201693999 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa7880a2cb0 msgr2=0x7fa7880a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:42.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.819+0000 7fa7a1e55700 1 --2- 172.21.15.37:0/4201693999 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa7880a2cb0 0x7fa7880a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa78c005950 tx=0x7fa78c00b410 comp rx=0 tx=0).stop 2024-06-10T00:54:42.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.819+0000 7fa7a1e55700 1 -- 172.21.15.37:0/4201693999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa79c107400 msgr2=0x7fa79c1a4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:42.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.819+0000 7fa7a1e55700 1 --2- 172.21.15.37:0/4201693999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa79c107400 0x7fa79c1a4af0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa794007d70 tx=0x7fa79400a340 comp rx=0 tx=0).stop 2024-06-10T00:54:42.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.819+0000 7fa79bfff700 1 -- 172.21.15.37:0/4201693999 reap_dead start 2024-06-10T00:54:42.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.819+0000 7fa7a1e55700 1 -- 172.21.15.37:0/4201693999 shutdown_connections 2024-06-10T00:54:42.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.819+0000 7fa7a1e55700 1 -- 172.21.15.37:0/4201693999 >> 172.21.15.37:0/4201693999 conn(0x7fa79c078340 msgr2=0x7fa79c0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:42.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.820+0000 7fa7a1e55700 1 -- 172.21.15.37:0/4201693999 shutdown_connections 2024-06-10T00:54:42.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.820+0000 7fa7a1e55700 1 -- 172.21.15.37:0/4201693999 wait complete. 2024-06-10T00:54:43.495 INFO:teuthology.orchestra.run.smithi037.stdout:167503724615 2024-06-10T00:54:43.496 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-06-10T00:54:43.583 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:43 smithi037 ceph-mon[32192]: pgmap v371: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:54:43.613 INFO:teuthology.orchestra.run.smithi037.stdout:77309411415 2024-06-10T00:54:43.614 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-06-10T00:54:43.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:43 smithi167 ceph-mon[36917]: pgmap v371: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:54:43.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:43 smithi094 ceph-mon[36708]: pgmap v371: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:54:44.442 INFO:teuthology.orchestra.run.smithi037.stdout:111669149776 2024-06-10T00:54:44.443 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-06-10T00:54:45.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.374+0000 7fb1f93df700 1 -- 172.21.15.37:0/3166892773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1f40746c0 msgr2=0x7fb1f4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:45.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.374+0000 7fb1f93df700 1 --2- 172.21.15.37:0/3166892773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1f40746c0 0x7fb1f4074aa0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4009990 tx=0x7fb1e4009ca0 comp rx=0 tx=0).stop 2024-06-10T00:54:45.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.374+0000 7fb1f93df700 1 -- 172.21.15.37:0/3166892773 shutdown_connections 2024-06-10T00:54:45.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.374+0000 7fb1f93df700 1 --2- 172.21.15.37:0/3166892773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1f410ee90 0x7fb1f4111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:45.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.374+0000 7fb1f93df700 1 --2- 172.21.15.37:0/3166892773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1f40746c0 0x7fb1f4074aa0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:45.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.374+0000 7fb1f93df700 1 --2- 172.21.15.37:0/3166892773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1f4074fe0 0x7fb1f410e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:45.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.374+0000 7fb1f93df700 1 -- 172.21.15.37:0/3166892773 >> 172.21.15.37:0/3166892773 conn(0x7fb1f4079530 msgr2=0x7fb1f4079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:45.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.374+0000 7fb1f93df700 1 -- 172.21.15.37:0/3166892773 shutdown_connections 2024-06-10T00:54:45.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.374+0000 7fb1f93df700 1 -- 172.21.15.37:0/3166892773 wait complete. 2024-06-10T00:54:45.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.374+0000 7fb1f93df700 1 Processor -- start 2024-06-10T00:54:45.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.375+0000 7fb1f93df700 1 -- start start 2024-06-10T00:54:45.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.375+0000 7fb1f93df700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1f40746c0 0x7fb1f419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:45.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.375+0000 7fb1f93df700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1f4074fe0 0x7fb1f419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:45.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.375+0000 7fb1f93df700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1f410ee90 0x7fb1f419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:45.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.375+0000 7fb1f93df700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f4114990 con 0x7fb1f4074fe0 2024-06-10T00:54:45.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.375+0000 7fb1f93df700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f4114810 con 0x7fb1f410ee90 2024-06-10T00:54:45.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.375+0000 7fb1f93df700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f4114b10 con 0x7fb1f40746c0 2024-06-10T00:54:45.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.375+0000 7fb1f27fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1f4074fe0 0x7fb1f419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:45.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.375+0000 7fb1f27fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1f4074fe0 0x7fb1f419b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54532/0 (socket says 172.21.15.37:54532) 2024-06-10T00:54:45.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.375+0000 7fb1f27fc700 1 -- 172.21.15.37:0/1692493461 learned_addr learned my addr 172.21.15.37:0/1692493461 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:54:45.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.375+0000 7fb1f2ffd700 1 --2- 172.21.15.37:0/1692493461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1f40746c0 0x7fb1f419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:45.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.375+0000 7fb1f37fe700 1 --2- 172.21.15.37:0/1692493461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1f410ee90 0x7fb1f419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:45.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.376+0000 7fb1f27fc700 1 -- 172.21.15.37:0/1692493461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1f40746c0 msgr2=0x7fb1f419ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:45.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.376+0000 7fb1f27fc700 1 --2- 172.21.15.37:0/1692493461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1f40746c0 0x7fb1f419ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:45.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.376+0000 7fb1f27fc700 1 -- 172.21.15.37:0/1692493461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1f410ee90 msgr2=0x7fb1f419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:45.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.376+0000 7fb1f27fc700 1 --2- 172.21.15.37:0/1692493461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1f410ee90 0x7fb1f419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:45.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.376+0000 7fb1f27fc700 1 -- 172.21.15.37:0/1692493461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f419fe20 con 0x7fb1f4074fe0 2024-06-10T00:54:45.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.376+0000 7fb1f37fe700 1 --2- 172.21.15.37:0/1692493461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1f410ee90 0x7fb1f419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:54:45.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.376+0000 7fb1f27fc700 1 --2- 172.21.15.37:0/1692493461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1f4074fe0 0x7fb1f419b3a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb1e800c920 tx=0x7fb1e800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:45.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.376+0000 7fb1dbfff700 1 -- 172.21.15.37:0/1692493461 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8007b00 con 0x7fb1f4074fe0 2024-06-10T00:54:45.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.376+0000 7fb1f93df700 1 -- 172.21.15.37:0/1692493461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f41a0110 con 0x7fb1f4074fe0 2024-06-10T00:54:45.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.376+0000 7fb1dbfff700 1 -- 172.21.15.37:0/1692493461 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1e8004d10 con 0x7fb1f4074fe0 2024-06-10T00:54:45.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.376+0000 7fb1dbfff700 1 -- 172.21.15.37:0/1692493461 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8005680 con 0x7fb1f4074fe0 2024-06-10T00:54:45.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.376+0000 7fb1f93df700 1 -- 172.21.15.37:0/1692493461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f41a7990 con 0x7fb1f4074fe0 2024-06-10T00:54:45.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.377+0000 7fb1f93df700 1 -- 172.21.15.37:0/1692493461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb1f4062c20 con 0x7fb1f4074fe0 2024-06-10T00:54:45.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.378+0000 7fb1dbfff700 1 -- 172.21.15.37:0/1692493461 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb1e8004750 con 0x7fb1f4074fe0 2024-06-10T00:54:45.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.379+0000 7fb1dbfff700 1 --2- 172.21.15.37:0/1692493461 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb1dc0a2cb0 0x7fb1dc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:45.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.379+0000 7fb1f2ffd700 1 --2- 172.21.15.37:0/1692493461 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb1dc0a2cb0 0x7fb1dc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:45.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.379+0000 7fb1dbfff700 1 -- 172.21.15.37:0/1692493461 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fb1e80cf5a0 con 0x7fb1f4074fe0 2024-06-10T00:54:45.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.379+0000 7fb1dbfff700 1 --2- 172.21.15.37:0/1692493461 >> [v2:172.21.15.167:6824/1349892516,v1:172.21.15.167:6825/1349892516] conn(0x7fb1dc0ae090 0x7fb1dc0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:45.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.379+0000 7fb1dbfff700 1 -- 172.21.15.37:0/1692493461 --> [v2:172.21.15.167:6824/1349892516,v1:172.21.15.167:6825/1349892516] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb1dc0b0c00 con 0x7fb1dc0ae090 2024-06-10T00:54:45.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.379+0000 7fb1dbfff700 1 -- 172.21.15.37:0/1692493461 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fb1e80cf950 con 0x7fb1f4074fe0 2024-06-10T00:54:45.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.379+0000 7fb1f2ffd700 1 --2- 172.21.15.37:0/1692493461 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb1dc0a2cb0 0x7fb1dc0a5170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb1e400b5c0 tx=0x7fb1e4005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:45.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.380+0000 7fb1f37fe700 1 --2- 172.21.15.37:0/1692493461 >> [v2:172.21.15.167:6824/1349892516,v1:172.21.15.167:6825/1349892516] conn(0x7fb1dc0ae090 0x7fb1dc0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:45.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.380+0000 7fb1f37fe700 1 --2- 172.21.15.37:0/1692493461 >> [v2:172.21.15.167:6824/1349892516,v1:172.21.15.167:6825/1349892516] conn(0x7fb1dc0ae090 0x7fb1dc0b0530 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:45.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.381+0000 7fb1dbfff700 1 -- 172.21.15.37:0/1692493461 <== osd.11 v2:172.21.15.167:6824/1349892516 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb1dc0b0c00 con 0x7fb1dc0ae090 2024-06-10T00:54:45.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.399+0000 7fb1f93df700 1 -- 172.21.15.37:0/1692493461 --> [v2:172.21.15.167:6824/1349892516,v1:172.21.15.167:6825/1349892516] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb1f40630a0 con 0x7fb1dc0ae090 2024-06-10T00:54:45.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.400+0000 7fb1dbfff700 1 -- 172.21.15.37:0/1692493461 <== osd.11 v2:172.21.15.167:6824/1349892516 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb1f40630a0 con 0x7fb1dc0ae090 2024-06-10T00:54:45.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.400+0000 7fb1f93df700 1 -- 172.21.15.37:0/1692493461 >> [v2:172.21.15.167:6824/1349892516,v1:172.21.15.167:6825/1349892516] conn(0x7fb1dc0ae090 msgr2=0x7fb1dc0b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:45.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.400+0000 7fb1f93df700 1 --2- 172.21.15.37:0/1692493461 >> [v2:172.21.15.167:6824/1349892516,v1:172.21.15.167:6825/1349892516] conn(0x7fb1dc0ae090 0x7fb1dc0b0530 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:45.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.400+0000 7fb1f93df700 1 -- 172.21.15.37:0/1692493461 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb1dc0a2cb0 msgr2=0x7fb1dc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:45.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.400+0000 7fb1f93df700 1 --2- 172.21.15.37:0/1692493461 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb1dc0a2cb0 0x7fb1dc0a5170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb1e400b5c0 tx=0x7fb1e4005960 comp rx=0 tx=0).stop 2024-06-10T00:54:45.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.400+0000 7fb1f93df700 1 -- 172.21.15.37:0/1692493461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1f4074fe0 msgr2=0x7fb1f419b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:45.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.400+0000 7fb1f93df700 1 --2- 172.21.15.37:0/1692493461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1f4074fe0 0x7fb1f419b3a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb1e800c920 tx=0x7fb1e800cc30 comp rx=0 tx=0).stop 2024-06-10T00:54:45.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.400+0000 7fb1f37fe700 1 -- 172.21.15.37:0/1692493461 reap_dead start 2024-06-10T00:54:45.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.400+0000 7fb1f93df700 1 -- 172.21.15.37:0/1692493461 shutdown_connections 2024-06-10T00:54:45.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.400+0000 7fb1f93df700 1 -- 172.21.15.37:0/1692493461 >> 172.21.15.37:0/1692493461 conn(0x7fb1f4079530 msgr2=0x7fb1f40ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:45.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.401+0000 7fb1f93df700 1 -- 172.21.15.37:0/1692493461 shutdown_connections 2024-06-10T00:54:45.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.401+0000 7fb1f93df700 1 -- 172.21.15.37:0/1692493461 wait complete. 2024-06-10T00:54:45.583 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:45 smithi037 ceph-mon[32192]: pgmap v372: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T00:54:45.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.598+0000 7f8f45740700 1 -- 172.21.15.37:0/235715370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f401066b0 msgr2=0x7f8f4010e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:45.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.598+0000 7f8f45740700 1 --2- 172.21.15.37:0/235715370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f401066b0 0x7f8f4010e280 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8f3800b0d0 tx=0x7f8f3800b3e0 comp rx=0 tx=0).stop 2024-06-10T00:54:45.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.599+0000 7f8f45740700 1 -- 172.21.15.37:0/235715370 shutdown_connections 2024-06-10T00:54:45.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.599+0000 7f8f45740700 1 --2- 172.21.15.37:0/235715370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f401066b0 0x7f8f4010e280 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:45.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.599+0000 7f8f45740700 1 --2- 172.21.15.37:0/235715370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f40101620 0x7f8f40101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:45.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.599+0000 7f8f45740700 1 --2- 172.21.15.37:0/235715370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f40101fd0 0x7f8f40105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:45.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.599+0000 7f8f45740700 1 -- 172.21.15.37:0/235715370 >> 172.21.15.37:0/235715370 conn(0x7f8f40078340 msgr2=0x7f8f40078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:45.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.599+0000 7f8f45740700 1 -- 172.21.15.37:0/235715370 shutdown_connections 2024-06-10T00:54:45.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.599+0000 7f8f45740700 1 -- 172.21.15.37:0/235715370 wait complete. 2024-06-10T00:54:45.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.599+0000 7f8f45740700 1 Processor -- start 2024-06-10T00:54:45.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.599+0000 7f8f45740700 1 -- start start 2024-06-10T00:54:45.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.600+0000 7f8f45740700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f40101620 0x7f8f4019e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:45.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.600+0000 7f8f45740700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f40101fd0 0x7f8f4019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:45.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.600+0000 7f8f45740700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f401066b0 0x7f8f40198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:45.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.600+0000 7f8f45740700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8f40111710 con 0x7f8f401066b0 2024-06-10T00:54:45.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.600+0000 7f8f45740700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8f40111590 con 0x7f8f40101fd0 2024-06-10T00:54:45.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.600+0000 7f8f45740700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8f40111890 con 0x7f8f40101620 2024-06-10T00:54:45.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.600+0000 7f8f3f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f401066b0 0x7f8f40198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:45.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.600+0000 7f8f3f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f401066b0 0x7f8f40198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54560/0 (socket says 172.21.15.37:54560) 2024-06-10T00:54:45.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.600+0000 7f8f3f7fe700 1 -- 172.21.15.37:0/1157747634 learned_addr learned my addr 172.21.15.37:0/1157747634 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:54:45.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.600+0000 7f8f3effd700 1 --2- 172.21.15.37:0/1157747634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f40101620 0x7f8f4019e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:45.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.600+0000 7f8f3e7fc700 1 --2- 172.21.15.37:0/1157747634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f40101fd0 0x7f8f4019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:45.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.600+0000 7f8f3f7fe700 1 -- 172.21.15.37:0/1157747634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f40101620 msgr2=0x7f8f4019e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:45.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.600+0000 7f8f3f7fe700 1 --2- 172.21.15.37:0/1157747634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f40101620 0x7f8f4019e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:45.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.600+0000 7f8f3f7fe700 1 -- 172.21.15.37:0/1157747634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f40101fd0 msgr2=0x7f8f4019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:45.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.600+0000 7f8f3f7fe700 1 --2- 172.21.15.37:0/1157747634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f40101fd0 0x7f8f4019ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:45.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.600+0000 7f8f3f7fe700 1 -- 172.21.15.37:0/1157747634 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f40199250 con 0x7f8f401066b0 2024-06-10T00:54:45.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.600+0000 7f8f3f7fe700 1 --2- 172.21.15.37:0/1157747634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f401066b0 0x7f8f40198af0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8f38006070 tx=0x7f8f38004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:45.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.601+0000 7f8f27fff700 1 -- 172.21.15.37:0/1157747634 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38021070 con 0x7f8f401066b0 2024-06-10T00:54:45.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.601+0000 7f8f45740700 1 -- 172.21.15.37:0/1157747634 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f401994e0 con 0x7f8f401066b0 2024-06-10T00:54:45.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.601+0000 7f8f27fff700 1 -- 172.21.15.37:0/1157747634 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f38007d00 con 0x7f8f401066b0 2024-06-10T00:54:45.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.601+0000 7f8f27fff700 1 -- 172.21.15.37:0/1157747634 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38005470 con 0x7f8f401066b0 2024-06-10T00:54:45.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.601+0000 7f8f45740700 1 -- 172.21.15.37:0/1157747634 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f401a57b0 con 0x7f8f401066b0 2024-06-10T00:54:45.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.601+0000 7f8f45740700 1 -- 172.21.15.37:0/1157747634 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8f4006af10 con 0x7f8f401066b0 2024-06-10T00:54:45.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.603+0000 7f8f27fff700 1 -- 172.21.15.37:0/1157747634 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8f38024070 con 0x7f8f401066b0 2024-06-10T00:54:45.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.603+0000 7f8f27fff700 1 --2- 172.21.15.37:0/1157747634 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8f280a2cb0 0x7f8f280a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:45.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.604+0000 7f8f3effd700 1 --2- 172.21.15.37:0/1157747634 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8f280a2cb0 0x7f8f280a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:45.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.604+0000 7f8f27fff700 1 -- 172.21.15.37:0/1157747634 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f8f380d07b0 con 0x7f8f401066b0 2024-06-10T00:54:45.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.604+0000 7f8f27fff700 1 --2- 172.21.15.37:0/1157747634 >> [v2:172.21.15.94:6824/1706618899,v1:172.21.15.94:6825/1706618899] conn(0x7f8f280ae090 0x7f8f280b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:45.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.604+0000 7f8f27fff700 1 -- 172.21.15.37:0/1157747634 --> [v2:172.21.15.94:6824/1706618899,v1:172.21.15.94:6825/1706618899] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8f3800cbc0 con 0x7f8f280ae090 2024-06-10T00:54:45.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.604+0000 7f8f27fff700 1 -- 172.21.15.37:0/1157747634 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f8f380d0c00 con 0x7f8f401066b0 2024-06-10T00:54:45.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.604+0000 7f8f3effd700 1 --2- 172.21.15.37:0/1157747634 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8f280a2cb0 0x7f8f280a5170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8f40106200 tx=0x7f8f30009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:45.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.604+0000 7f8f3e7fc700 1 --2- 172.21.15.37:0/1157747634 >> [v2:172.21.15.94:6824/1706618899,v1:172.21.15.94:6825/1706618899] conn(0x7f8f280ae090 0x7f8f280b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:45.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.605+0000 7f8f3e7fc700 1 --2- 172.21.15.37:0/1157747634 >> [v2:172.21.15.94:6824/1706618899,v1:172.21.15.94:6825/1706618899] conn(0x7f8f280ae090 0x7f8f280b0530 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:45.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.606+0000 7f8f27fff700 1 -- 172.21.15.37:0/1157747634 <== osd.10 v2:172.21.15.94:6824/1706618899 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8f3800cbc0 con 0x7f8f280ae090 2024-06-10T00:54:45.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.622+0000 7f8f45740700 1 -- 172.21.15.37:0/1157747634 --> [v2:172.21.15.94:6824/1706618899,v1:172.21.15.94:6825/1706618899] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8f40102ab0 con 0x7f8f280ae090 2024-06-10T00:54:45.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.623+0000 7f8f27fff700 1 -- 172.21.15.37:0/1157747634 <== osd.10 v2:172.21.15.94:6824/1706618899 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8f40102ab0 con 0x7f8f280ae090 2024-06-10T00:54:45.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.623+0000 7f8f45740700 1 -- 172.21.15.37:0/1157747634 >> [v2:172.21.15.94:6824/1706618899,v1:172.21.15.94:6825/1706618899] conn(0x7f8f280ae090 msgr2=0x7f8f280b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:45.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.623+0000 7f8f45740700 1 --2- 172.21.15.37:0/1157747634 >> [v2:172.21.15.94:6824/1706618899,v1:172.21.15.94:6825/1706618899] conn(0x7f8f280ae090 0x7f8f280b0530 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:45.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.623+0000 7f8f45740700 1 -- 172.21.15.37:0/1157747634 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8f280a2cb0 msgr2=0x7f8f280a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:45.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.623+0000 7f8f45740700 1 --2- 172.21.15.37:0/1157747634 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8f280a2cb0 0x7f8f280a5170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8f40106200 tx=0x7f8f30009190 comp rx=0 tx=0).stop 2024-06-10T00:54:45.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.623+0000 7f8f45740700 1 -- 172.21.15.37:0/1157747634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f401066b0 msgr2=0x7f8f40198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:45.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.623+0000 7f8f45740700 1 --2- 172.21.15.37:0/1157747634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f401066b0 0x7f8f40198af0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8f38006070 tx=0x7f8f38004e40 comp rx=0 tx=0).stop 2024-06-10T00:54:45.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.623+0000 7f8f3f7fe700 1 -- 172.21.15.37:0/1157747634 reap_dead start 2024-06-10T00:54:45.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.623+0000 7f8f45740700 1 -- 172.21.15.37:0/1157747634 shutdown_connections 2024-06-10T00:54:45.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.623+0000 7f8f45740700 1 -- 172.21.15.37:0/1157747634 >> 172.21.15.37:0/1157747634 conn(0x7f8f40078340 msgr2=0x7f8f401052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:45.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.623+0000 7f8f45740700 1 -- 172.21.15.37:0/1157747634 shutdown_connections 2024-06-10T00:54:45.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:45.623+0000 7f8f45740700 1 -- 172.21.15.37:0/1157747634 wait complete. 2024-06-10T00:54:45.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:45 smithi167 ceph-mon[36917]: pgmap v372: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T00:54:45.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:45 smithi094 ceph-mon[36708]: pgmap v372: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T00:54:45.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.574+0000 7f4a78cac700 1 -- 172.21.15.37:0/3314022051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a7406b3a0 msgr2=0x7f4a7410f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:45.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.574+0000 7f4a78cac700 1 --2- 172.21.15.37:0/3314022051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a7406b3a0 0x7f4a7410f310 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c00b0a0 tx=0x7f4a6c00b3b0 comp rx=0 tx=0).stop 2024-06-10T00:54:45.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.575+0000 7f4a78cac700 1 -- 172.21.15.37:0/3314022051 shutdown_connections 2024-06-10T00:54:45.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.575+0000 7f4a78cac700 1 --2- 172.21.15.37:0/3314022051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a7406b3a0 0x7f4a7410f310 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:45.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.575+0000 7f4a78cac700 1 --2- 172.21.15.37:0/3314022051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a741084d0 0x7f4a741088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:45.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.575+0000 7f4a78cac700 1 --2- 172.21.15.37:0/3314022051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a7406aa00 0x7f4a7406ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:45.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.575+0000 7f4a78cac700 1 -- 172.21.15.37:0/3314022051 >> 172.21.15.37:0/3314022051 conn(0x7f4a74078330 msgr2=0x7f4a74078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:45.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.575+0000 7f4a78cac700 1 -- 172.21.15.37:0/3314022051 shutdown_connections 2024-06-10T00:54:45.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.575+0000 7f4a78cac700 1 -- 172.21.15.37:0/3314022051 wait complete. 2024-06-10T00:54:45.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.575+0000 7f4a78cac700 1 Processor -- start 2024-06-10T00:54:45.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.576+0000 7f4a78cac700 1 -- start start 2024-06-10T00:54:45.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.576+0000 7f4a78cac700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a7406aa00 0x7f4a74101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:45.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.576+0000 7f4a78cac700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a7406b3a0 0x7f4a74101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:45.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.576+0000 7f4a78cac700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a741084d0 0x7f4a74101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:45.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.576+0000 7f4a78cac700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4a74112750 con 0x7f4a7406aa00 2024-06-10T00:54:45.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.576+0000 7f4a78cac700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4a741125d0 con 0x7f4a7406b3a0 2024-06-10T00:54:45.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.576+0000 7f4a78cac700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4a741128d0 con 0x7f4a741084d0 2024-06-10T00:54:45.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.576+0000 7f4a7259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a7406aa00 0x7f4a74101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:45.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.576+0000 7f4a7259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a7406aa00 0x7f4a74101450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49906/0 (socket says 172.21.15.37:49906) 2024-06-10T00:54:45.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.576+0000 7f4a7259c700 1 -- 172.21.15.37:0/2734219154 learned_addr learned my addr 172.21.15.37:0/2734219154 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:54:45.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.576+0000 7f4a71d9b700 1 --2- 172.21.15.37:0/2734219154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a7406b3a0 0x7f4a74101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:45.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.576+0000 7f4a72d9d700 1 --2- 172.21.15.37:0/2734219154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a741084d0 0x7f4a74101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:45.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.577+0000 7f4a7259c700 1 -- 172.21.15.37:0/2734219154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a741084d0 msgr2=0x7f4a74101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:45.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.577+0000 7f4a7259c700 1 --2- 172.21.15.37:0/2734219154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a741084d0 0x7f4a74101ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:45.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.577+0000 7f4a7259c700 1 -- 172.21.15.37:0/2734219154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a7406b3a0 msgr2=0x7f4a74101990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:45.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.577+0000 7f4a7259c700 1 --2- 172.21.15.37:0/2734219154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a7406b3a0 0x7f4a74101990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:45.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.577+0000 7f4a7259c700 1 -- 172.21.15.37:0/2734219154 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a741a7150 con 0x7f4a7406aa00 2024-06-10T00:54:45.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.577+0000 7f4a71d9b700 1 --2- 172.21.15.37:0/2734219154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a7406b3a0 0x7f4a74101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:54:45.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.577+0000 7f4a7259c700 1 --2- 172.21.15.37:0/2734219154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a7406aa00 0x7f4a74101450 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400d980 tx=0x7f4a6400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:45.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.577+0000 7f4a637fe700 1 -- 172.21.15.37:0/2734219154 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a64016070 con 0x7f4a7406aa00 2024-06-10T00:54:45.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.577+0000 7f4a78cac700 1 -- 172.21.15.37:0/2734219154 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a741a7440 con 0x7f4a7406aa00 2024-06-10T00:54:45.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.577+0000 7f4a78cac700 1 -- 172.21.15.37:0/2734219154 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a741a79d0 con 0x7f4a7406aa00 2024-06-10T00:54:45.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.577+0000 7f4a637fe700 1 -- 172.21.15.37:0/2734219154 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a64004d10 con 0x7f4a7406aa00 2024-06-10T00:54:45.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.578+0000 7f4a78cac700 1 -- 172.21.15.37:0/2734219154 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a54001030 con 0x7f4a7406aa00 2024-06-10T00:54:45.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.578+0000 7f4a637fe700 1 -- 172.21.15.37:0/2734219154 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a64005670 con 0x7f4a7406aa00 2024-06-10T00:54:45.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.581+0000 7f4a637fe700 1 -- 172.21.15.37:0/2734219154 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4a64004220 con 0x7f4a7406aa00 2024-06-10T00:54:45.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.582+0000 7f4a637fe700 1 --2- 172.21.15.37:0/2734219154 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4a5c0a2cb0 0x7f4a5c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:45.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.582+0000 7f4a71d9b700 1 --2- 172.21.15.37:0/2734219154 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4a5c0a2cb0 0x7f4a5c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:45.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.582+0000 7f4a637fe700 1 -- 172.21.15.37:0/2734219154 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f4a640cf870 con 0x7f4a7406aa00 2024-06-10T00:54:45.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.582+0000 7f4a637fe700 1 --2- 172.21.15.37:0/2734219154 >> [v2:172.21.15.37:6802/3594704094,v1:172.21.15.37:6803/3594704094] conn(0x7f4a5c0ae090 0x7f4a5c0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:45.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.582+0000 7f4a637fe700 1 -- 172.21.15.37:0/2734219154 --> [v2:172.21.15.37:6802/3594704094,v1:172.21.15.37:6803/3594704094] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a5c0b0c00 con 0x7f4a5c0ae090 2024-06-10T00:54:45.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.582+0000 7f4a637fe700 1 -- 172.21.15.37:0/2734219154 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f4a640cfc20 con 0x7f4a7406aa00 2024-06-10T00:54:45.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.582+0000 7f4a72d9d700 1 --2- 172.21.15.37:0/2734219154 >> [v2:172.21.15.37:6802/3594704094,v1:172.21.15.37:6803/3594704094] conn(0x7f4a5c0ae090 0x7f4a5c0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:45.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.582+0000 7f4a71d9b700 1 --2- 172.21.15.37:0/2734219154 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4a5c0a2cb0 0x7f4a5c0a5170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4a740788c0 tx=0x7f4a68009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:45.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.583+0000 7f4a72d9d700 1 --2- 172.21.15.37:0/2734219154 >> [v2:172.21.15.37:6802/3594704094,v1:172.21.15.37:6803/3594704094] conn(0x7f4a5c0ae090 0x7f4a5c0b0530 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:45.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.583+0000 7f4a637fe700 1 -- 172.21.15.37:0/2734219154 <== osd.0 v2:172.21.15.37:6802/3594704094 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a5c0b0c00 con 0x7f4a5c0ae090 2024-06-10T00:54:45.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.600+0000 7f4a78cac700 1 -- 172.21.15.37:0/2734219154 --> [v2:172.21.15.37:6802/3594704094,v1:172.21.15.37:6803/3594704094] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4a54002e60 con 0x7f4a5c0ae090 2024-06-10T00:54:45.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.600+0000 7f4a637fe700 1 -- 172.21.15.37:0/2734219154 <== osd.0 v2:172.21.15.37:6802/3594704094 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4a54002e60 con 0x7f4a5c0ae090 2024-06-10T00:54:45.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.601+0000 7f4a78cac700 1 -- 172.21.15.37:0/2734219154 >> [v2:172.21.15.37:6802/3594704094,v1:172.21.15.37:6803/3594704094] conn(0x7f4a5c0ae090 msgr2=0x7f4a5c0b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:45.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.601+0000 7f4a78cac700 1 --2- 172.21.15.37:0/2734219154 >> [v2:172.21.15.37:6802/3594704094,v1:172.21.15.37:6803/3594704094] conn(0x7f4a5c0ae090 0x7f4a5c0b0530 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:45.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.601+0000 7f4a78cac700 1 -- 172.21.15.37:0/2734219154 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4a5c0a2cb0 msgr2=0x7f4a5c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:45.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.601+0000 7f4a78cac700 1 --2- 172.21.15.37:0/2734219154 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4a5c0a2cb0 0x7f4a5c0a5170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4a740788c0 tx=0x7f4a68009260 comp rx=0 tx=0).stop 2024-06-10T00:54:45.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.601+0000 7f4a78cac700 1 -- 172.21.15.37:0/2734219154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a7406aa00 msgr2=0x7f4a74101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:45.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.601+0000 7f4a78cac700 1 --2- 172.21.15.37:0/2734219154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a7406aa00 0x7f4a74101450 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400d980 tx=0x7f4a6400dc90 comp rx=0 tx=0).stop 2024-06-10T00:54:45.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.601+0000 7f4a72d9d700 1 -- 172.21.15.37:0/2734219154 reap_dead start 2024-06-10T00:54:45.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.601+0000 7f4a78cac700 1 -- 172.21.15.37:0/2734219154 shutdown_connections 2024-06-10T00:54:45.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.601+0000 7f4a78cac700 1 -- 172.21.15.37:0/2734219154 >> 172.21.15.37:0/2734219154 conn(0x7f4a74078330 msgr2=0x7f4a740ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:45.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.601+0000 7f4a78cac700 1 -- 172.21.15.37:0/2734219154 shutdown_connections 2024-06-10T00:54:45.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:41.601+0000 7f4a78cac700 1 -- 172.21.15.37:0/2734219154 wait complete. 2024-06-10T00:54:46.573 INFO:teuthology.orchestra.run.smithi037.stdout:34359738465 2024-06-10T00:54:46.574 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-06-10T00:54:47.147 INFO:teuthology.orchestra.run.smithi037.stdout:223338299454 2024-06-10T00:54:47.148 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-06-10T00:54:47.616 INFO:teuthology.orchestra.run.smithi037.stdout:347892351018 2024-06-10T00:54:47.616 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-06-10T00:54:47.724 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:47 smithi037 ceph-mon[32192]: pgmap v373: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-06-10T00:54:47.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:47 smithi167 ceph-mon[36917]: pgmap v373: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-06-10T00:54:47.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:47 smithi094 ceph-mon[36708]: pgmap v373: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-06-10T00:54:48.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.321+0000 7f5b3adbe700 1 -- 172.21.15.37:0/2161075534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b34101d30 msgr2=0x7f5b3410e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:48.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.321+0000 7f5b3adbe700 1 --2- 172.21.15.37:0/2161075534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b34101d30 0x7f5b3410e1d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c00b0a0 tx=0x7f5b2c00b3b0 comp rx=0 tx=0).stop 2024-06-10T00:54:48.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.321+0000 7f5b3adbe700 1 -- 172.21.15.37:0/2161075534 shutdown_connections 2024-06-10T00:54:48.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.321+0000 7f5b3adbe700 1 --2- 172.21.15.37:0/2161075534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b34101d30 0x7f5b3410e1d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:48.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.321+0000 7f5b3adbe700 1 --2- 172.21.15.37:0/2161075534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b34107420 0x7f5b34107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:48.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.321+0000 7f5b3adbe700 1 --2- 172.21.15.37:0/2161075534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b34101370 0x7f5b341017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:48.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.321+0000 7f5b3adbe700 1 -- 172.21.15.37:0/2161075534 >> 172.21.15.37:0/2161075534 conn(0x7f5b34078340 msgr2=0x7f5b34078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:48.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.322+0000 7f5b3adbe700 1 -- 172.21.15.37:0/2161075534 shutdown_connections 2024-06-10T00:54:48.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.322+0000 7f5b3adbe700 1 -- 172.21.15.37:0/2161075534 wait complete. 2024-06-10T00:54:48.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.322+0000 7f5b3adbe700 1 Processor -- start 2024-06-10T00:54:48.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.322+0000 7f5b3adbe700 1 -- start start 2024-06-10T00:54:48.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.323+0000 7f5b3adbe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b34101370 0x7f5b3419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:48.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.323+0000 7f5b3adbe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b34101d30 0x7f5b3419dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:48.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.323+0000 7f5b3adbe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b34107420 0x7f5b34197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:48.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.323+0000 7f5b3adbe700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5b34111630 con 0x7f5b34107420 2024-06-10T00:54:48.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.323+0000 7f5b3adbe700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5b341114b0 con 0x7f5b34101d30 2024-06-10T00:54:48.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.323+0000 7f5b3adbe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5b341117b0 con 0x7f5b34101370 2024-06-10T00:54:48.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.323+0000 7f5b3935b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b34107420 0x7f5b34197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:48.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.323+0000 7f5b3935b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b34107420 0x7f5b34197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54596/0 (socket says 172.21.15.37:54596) 2024-06-10T00:54:48.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.323+0000 7f5b3935b700 1 -- 172.21.15.37:0/1909236494 learned_addr learned my addr 172.21.15.37:0/1909236494 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:54:48.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.323+0000 7f5b38b5a700 1 --2- 172.21.15.37:0/1909236494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b34101370 0x7f5b3419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:48.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.323+0000 7f5b33fff700 1 --2- 172.21.15.37:0/1909236494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b34101d30 0x7f5b3419dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:48.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.323+0000 7f5b3935b700 1 -- 172.21.15.37:0/1909236494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b34101370 msgr2=0x7f5b3419d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:48.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.323+0000 7f5b3935b700 1 --2- 172.21.15.37:0/1909236494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b34101370 0x7f5b3419d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:48.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.323+0000 7f5b3935b700 1 -- 172.21.15.37:0/1909236494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b34101d30 msgr2=0x7f5b3419dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:48.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.323+0000 7f5b3935b700 1 --2- 172.21.15.37:0/1909236494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b34101d30 0x7f5b3419dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:48.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.323+0000 7f5b3935b700 1 -- 172.21.15.37:0/1909236494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b34198010 con 0x7f5b34107420 2024-06-10T00:54:48.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.323+0000 7f5b3935b700 1 --2- 172.21.15.37:0/1909236494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b34107420 0x7f5b34197830 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0062a0 tx=0x7f5b2c004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:48.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.324+0000 7f5b31ffb700 1 -- 172.21.15.37:0/1909236494 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c020070 con 0x7f5b34107420 2024-06-10T00:54:48.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.324+0000 7f5b3adbe700 1 -- 172.21.15.37:0/1909236494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b341982a0 con 0x7f5b34107420 2024-06-10T00:54:48.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.324+0000 7f5b31ffb700 1 -- 172.21.15.37:0/1909236494 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b2c003c00 con 0x7f5b34107420 2024-06-10T00:54:48.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.324+0000 7f5b31ffb700 1 -- 172.21.15.37:0/1909236494 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c004880 con 0x7f5b34107420 2024-06-10T00:54:48.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.324+0000 7f5b3adbe700 1 -- 172.21.15.37:0/1909236494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b3406abc0 con 0x7f5b34107420 2024-06-10T00:54:48.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.325+0000 7f5b3adbe700 1 -- 172.21.15.37:0/1909236494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5b3406b170 con 0x7f5b34107420 2024-06-10T00:54:48.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.326+0000 7f5b31ffb700 1 -- 172.21.15.37:0/1909236494 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5b2c003db0 con 0x7f5b34107420 2024-06-10T00:54:48.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.327+0000 7f5b31ffb700 1 --2- 172.21.15.37:0/1909236494 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5b240a2cb0 0x7f5b240a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:48.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.327+0000 7f5b38b5a700 1 --2- 172.21.15.37:0/1909236494 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5b240a2cb0 0x7f5b240a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:48.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.327+0000 7f5b31ffb700 1 -- 172.21.15.37:0/1909236494 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f5b2c00f030 con 0x7f5b34107420 2024-06-10T00:54:48.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.327+0000 7f5b31ffb700 1 --2- 172.21.15.37:0/1909236494 >> [v2:172.21.15.94:6800/3427727155,v1:172.21.15.94:6801/3427727155] conn(0x7f5b240ae090 0x7f5b240b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:48.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.327+0000 7f5b31ffb700 1 -- 172.21.15.37:0/1909236494 --> [v2:172.21.15.94:6800/3427727155,v1:172.21.15.94:6801/3427727155] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5b2c0020b0 con 0x7f5b240ae090 2024-06-10T00:54:48.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.327+0000 7f5b31ffb700 1 -- 172.21.15.37:0/1909236494 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f5b2c0d22f0 con 0x7f5b34107420 2024-06-10T00:54:48.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.327+0000 7f5b38b5a700 1 --2- 172.21.15.37:0/1909236494 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5b240a2cb0 0x7f5b240a5170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800abd0 tx=0x7f5b2800a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:48.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.327+0000 7f5b33fff700 1 --2- 172.21.15.37:0/1909236494 >> [v2:172.21.15.94:6800/3427727155,v1:172.21.15.94:6801/3427727155] conn(0x7f5b240ae090 0x7f5b240b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:48.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.328+0000 7f5b33fff700 1 --2- 172.21.15.37:0/1909236494 >> [v2:172.21.15.94:6800/3427727155,v1:172.21.15.94:6801/3427727155] conn(0x7f5b240ae090 0x7f5b240b0530 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:48.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.329+0000 7f5b31ffb700 1 -- 172.21.15.37:0/1909236494 <== osd.1 v2:172.21.15.94:6800/3427727155 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5b2c0020b0 con 0x7f5b240ae090 2024-06-10T00:54:48.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.345+0000 7f5b3adbe700 1 -- 172.21.15.37:0/1909236494 --> [v2:172.21.15.94:6800/3427727155,v1:172.21.15.94:6801/3427727155] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5b3406b5f0 con 0x7f5b240ae090 2024-06-10T00:54:48.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.345+0000 7f5b31ffb700 1 -- 172.21.15.37:0/1909236494 <== osd.1 v2:172.21.15.94:6800/3427727155 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5b3406b5f0 con 0x7f5b240ae090 2024-06-10T00:54:48.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.346+0000 7f5b3adbe700 1 -- 172.21.15.37:0/1909236494 >> [v2:172.21.15.94:6800/3427727155,v1:172.21.15.94:6801/3427727155] conn(0x7f5b240ae090 msgr2=0x7f5b240b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:48.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.346+0000 7f5b3adbe700 1 --2- 172.21.15.37:0/1909236494 >> [v2:172.21.15.94:6800/3427727155,v1:172.21.15.94:6801/3427727155] conn(0x7f5b240ae090 0x7f5b240b0530 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:48.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.346+0000 7f5b3adbe700 1 -- 172.21.15.37:0/1909236494 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5b240a2cb0 msgr2=0x7f5b240a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:48.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.346+0000 7f5b3adbe700 1 --2- 172.21.15.37:0/1909236494 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5b240a2cb0 0x7f5b240a5170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800abd0 tx=0x7f5b2800a380 comp rx=0 tx=0).stop 2024-06-10T00:54:48.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.346+0000 7f5b3adbe700 1 -- 172.21.15.37:0/1909236494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b34107420 msgr2=0x7f5b34197830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:48.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.346+0000 7f5b3adbe700 1 --2- 172.21.15.37:0/1909236494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b34107420 0x7f5b34197830 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0062a0 tx=0x7f5b2c004d70 comp rx=0 tx=0).stop 2024-06-10T00:54:48.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.346+0000 7f5b3935b700 1 -- 172.21.15.37:0/1909236494 reap_dead start 2024-06-10T00:54:48.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.346+0000 7f5b3adbe700 1 -- 172.21.15.37:0/1909236494 shutdown_connections 2024-06-10T00:54:48.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.346+0000 7f5b3adbe700 1 -- 172.21.15.37:0/1909236494 >> 172.21.15.37:0/1909236494 conn(0x7f5b34078340 msgr2=0x7f5b340fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:48.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.346+0000 7f5b3adbe700 1 -- 172.21.15.37:0/1909236494 shutdown_connections 2024-06-10T00:54:48.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.346+0000 7f5b3adbe700 1 -- 172.21.15.37:0/1909236494 wait complete. 2024-06-10T00:54:49.054 INFO:teuthology.orchestra.run.smithi037.stdout:313532612655 2024-06-10T00:54:49.054 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-06-10T00:54:49.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.211+0000 7f5434614700 1 -- 172.21.15.37:0/2346111089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f542c106720 msgr2=0x7f542c10e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:49.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.211+0000 7f5434614700 1 --2- 172.21.15.37:0/2346111089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f542c106720 0x7f542c10e2c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f54240099c0 tx=0x7f5424009cd0 comp rx=0 tx=0).stop 2024-06-10T00:54:49.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.211+0000 7f5434614700 1 -- 172.21.15.37:0/2346111089 shutdown_connections 2024-06-10T00:54:49.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.211+0000 7f5434614700 1 --2- 172.21.15.37:0/2346111089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f542c106720 0x7f542c10e2c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:49.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.211+0000 7f5434614700 1 --2- 172.21.15.37:0/2346111089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f542c1019d0 0x7f542c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:49.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.211+0000 7f5434614700 1 --2- 172.21.15.37:0/2346111089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f542c102380 0x7f542c105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.211+0000 7f5434614700 1 -- 172.21.15.37:0/2346111089 >> 172.21.15.37:0/2346111089 conn(0x7f542c078340 msgr2=0x7f542c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.211+0000 7f5434614700 1 -- 172.21.15.37:0/2346111089 shutdown_connections 2024-06-10T00:54:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.211+0000 7f5434614700 1 -- 172.21.15.37:0/2346111089 wait complete. 2024-06-10T00:54:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.212+0000 7f5434614700 1 Processor -- start 2024-06-10T00:54:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.212+0000 7f5434614700 1 -- start start 2024-06-10T00:54:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.212+0000 7f5434614700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f542c1019d0 0x7f542c197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.212+0000 7f5434614700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f542c102380 0x7f542c197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.212+0000 7f5434614700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f542c106720 0x7f542c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.212+0000 7f5434614700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f542c1117b0 con 0x7f542c1019d0 2024-06-10T00:54:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.212+0000 7f5434614700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f542c111630 con 0x7f542c102380 2024-06-10T00:54:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.212+0000 7f5434614700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f542c111930 con 0x7f542c106720 2024-06-10T00:54:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.212+0000 7f54323b0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f542c1019d0 0x7f542c197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:49.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.212+0000 7f54323b0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f542c1019d0 0x7f542c197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54578/0 (socket says 172.21.15.37:54578) 2024-06-10T00:54:49.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.212+0000 7f54323b0700 1 -- 172.21.15.37:0/192939704 learned_addr learned my addr 172.21.15.37:0/192939704 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:54:49.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.212+0000 7f5431baf700 1 --2- 172.21.15.37:0/192939704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f542c102380 0x7f542c197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:49.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.212+0000 7f5432bb1700 1 --2- 172.21.15.37:0/192939704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f542c106720 0x7f542c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:49.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.212+0000 7f54323b0700 1 -- 172.21.15.37:0/192939704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f542c106720 msgr2=0x7f542c1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:49.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.212+0000 7f54323b0700 1 --2- 172.21.15.37:0/192939704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f542c106720 0x7f542c1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:49.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.212+0000 7f54323b0700 1 -- 172.21.15.37:0/192939704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f542c102380 msgr2=0x7f542c197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:49.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.212+0000 7f54323b0700 1 --2- 172.21.15.37:0/192939704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f542c102380 0x7f542c197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:49.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.212+0000 7f54323b0700 1 -- 172.21.15.37:0/192939704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f542c1a7190 con 0x7f542c1019d0 2024-06-10T00:54:49.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.212+0000 7f5432bb1700 1 --2- 172.21.15.37:0/192939704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f542c106720 0x7f542c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:54:49.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.213+0000 7f54323b0700 1 --2- 172.21.15.37:0/192939704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f542c1019d0 0x7f542c197920 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f541c00ba50 tx=0x7f541c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:49.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.213+0000 7f542b7fe700 1 -- 172.21.15.37:0/192939704 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c019070 con 0x7f542c1019d0 2024-06-10T00:54:49.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.213+0000 7f542b7fe700 1 -- 172.21.15.37:0/192939704 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f541c014440 con 0x7f542c1019d0 2024-06-10T00:54:49.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.213+0000 7f5434614700 1 -- 172.21.15.37:0/192939704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f542c1a7480 con 0x7f542c1019d0 2024-06-10T00:54:49.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.213+0000 7f542b7fe700 1 -- 172.21.15.37:0/192939704 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c0075d0 con 0x7f542c1019d0 2024-06-10T00:54:49.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.213+0000 7f5434614700 1 -- 172.21.15.37:0/192939704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f542c1a7a10 con 0x7f542c1019d0 2024-06-10T00:54:49.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.214+0000 7f5434614700 1 -- 172.21.15.37:0/192939704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f542c062c20 con 0x7f542c1019d0 2024-06-10T00:54:49.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.215+0000 7f542b7fe700 1 -- 172.21.15.37:0/192939704 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f541c01c070 con 0x7f542c1019d0 2024-06-10T00:54:49.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.216+0000 7f542b7fe700 1 --2- 172.21.15.37:0/192939704 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f54140a2d30 0x7f54140a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:49.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.216+0000 7f5431baf700 1 --2- 172.21.15.37:0/192939704 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f54140a2d30 0x7f54140a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:49.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.216+0000 7f542b7fe700 1 -- 172.21.15.37:0/192939704 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f541c0cefc0 con 0x7f542c1019d0 2024-06-10T00:54:49.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.216+0000 7f542b7fe700 1 --2- 172.21.15.37:0/192939704 >> [v2:172.21.15.37:6826/2063666428,v1:172.21.15.37:6827/2063666428] conn(0x7f54140ae110 0x7f54140b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:49.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.216+0000 7f542b7fe700 1 -- 172.21.15.37:0/192939704 --> [v2:172.21.15.37:6826/2063666428,v1:172.21.15.37:6827/2063666428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f54140b0c80 con 0x7f54140ae110 2024-06-10T00:54:49.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.216+0000 7f542b7fe700 1 -- 172.21.15.37:0/192939704 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f541c0cf410 con 0x7f542c1019d0 2024-06-10T00:54:49.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.217+0000 7f5432bb1700 1 --2- 172.21.15.37:0/192939704 >> [v2:172.21.15.37:6826/2063666428,v1:172.21.15.37:6827/2063666428] conn(0x7f54140ae110 0x7f54140b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:49.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.217+0000 7f5431baf700 1 --2- 172.21.15.37:0/192939704 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f54140a2d30 0x7f54140a51f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f542c199a80 tx=0x7f5420009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:49.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.217+0000 7f5432bb1700 1 --2- 172.21.15.37:0/192939704 >> [v2:172.21.15.37:6826/2063666428,v1:172.21.15.37:6827/2063666428] conn(0x7f54140ae110 0x7f54140b05b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:49.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.218+0000 7f542b7fe700 1 -- 172.21.15.37:0/192939704 <== osd.9 v2:172.21.15.37:6826/2063666428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f54140b0c80 con 0x7f54140ae110 2024-06-10T00:54:49.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.235+0000 7f5434614700 1 -- 172.21.15.37:0/192939704 --> [v2:172.21.15.37:6826/2063666428,v1:172.21.15.37:6827/2063666428] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f542c063020 con 0x7f54140ae110 2024-06-10T00:54:49.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.235+0000 7f542b7fe700 1 -- 172.21.15.37:0/192939704 <== osd.9 v2:172.21.15.37:6826/2063666428 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f542c063020 con 0x7f54140ae110 2024-06-10T00:54:49.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.235+0000 7f5434614700 1 -- 172.21.15.37:0/192939704 >> [v2:172.21.15.37:6826/2063666428,v1:172.21.15.37:6827/2063666428] conn(0x7f54140ae110 msgr2=0x7f54140b05b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:49.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.235+0000 7f5434614700 1 --2- 172.21.15.37:0/192939704 >> [v2:172.21.15.37:6826/2063666428,v1:172.21.15.37:6827/2063666428] conn(0x7f54140ae110 0x7f54140b05b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:49.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.236+0000 7f5434614700 1 -- 172.21.15.37:0/192939704 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f54140a2d30 msgr2=0x7f54140a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:49.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.236+0000 7f5434614700 1 --2- 172.21.15.37:0/192939704 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f54140a2d30 0x7f54140a51f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f542c199a80 tx=0x7f5420009190 comp rx=0 tx=0).stop 2024-06-10T00:54:49.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.236+0000 7f5434614700 1 -- 172.21.15.37:0/192939704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f542c1019d0 msgr2=0x7f542c197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:49.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.236+0000 7f5434614700 1 --2- 172.21.15.37:0/192939704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f542c1019d0 0x7f542c197920 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f541c00ba50 tx=0x7f541c00bd60 comp rx=0 tx=0).stop 2024-06-10T00:54:49.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.236+0000 7f5432bb1700 1 -- 172.21.15.37:0/192939704 reap_dead start 2024-06-10T00:54:49.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.236+0000 7f5434614700 1 -- 172.21.15.37:0/192939704 shutdown_connections 2024-06-10T00:54:49.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.236+0000 7f5434614700 1 -- 172.21.15.37:0/192939704 >> 172.21.15.37:0/192939704 conn(0x7f542c078340 msgr2=0x7f542c0ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:49.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.236+0000 7f5434614700 1 -- 172.21.15.37:0/192939704 shutdown_connections 2024-06-10T00:54:49.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:48.236+0000 7f5434614700 1 -- 172.21.15.37:0/192939704 wait complete. 2024-06-10T00:54:49.340 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:49.340 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:49.342 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:49.348 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:49.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:49 smithi037 ceph-mon[32192]: pgmap v374: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-06-10T00:54:49.759 INFO:teuthology.orchestra.run.smithi037.stdout:292057776181 2024-06-10T00:54:49.759 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-06-10T00:54:49.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:49 smithi167 ceph-mon[36917]: pgmap v374: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-06-10T00:54:49.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:49 smithi094 ceph-mon[36708]: pgmap v374: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-06-10T00:54:50.238 INFO:teuthology.orchestra.run.smithi037.stdout:55834574941 2024-06-10T00:54:50.239 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-06-10T00:54:51.567 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:51 smithi037 ceph-mon[32192]: pgmap v375: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:54:51.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:51 smithi167 ceph-mon[36917]: pgmap v375: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:54:51.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:51 smithi094 ceph-mon[36708]: pgmap v375: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:54:53.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:53 smithi037 ceph-mon[32192]: pgmap v376: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:54:53.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:53 smithi167 ceph-mon[36917]: pgmap v376: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:54:53.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:53 smithi094 ceph-mon[36708]: pgmap v376: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:54:53.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.894+0000 7f53a724b700 1 -- 172.21.15.37:0/1191225622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53a006aa00 msgr2=0x7f53a006ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:53.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.894+0000 7f53a724b700 1 --2- 172.21.15.37:0/1191225622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53a006aa00 0x7f53a006ae80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f538c009990 tx=0x7f538c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:54:53.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.894+0000 7f53a724b700 1 -- 172.21.15.37:0/1191225622 shutdown_connections 2024-06-10T00:54:53.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.894+0000 7f53a724b700 1 --2- 172.21.15.37:0/1191225622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53a006b3c0 0x7f53a010f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:53.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.894+0000 7f53a724b700 1 --2- 172.21.15.37:0/1191225622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53a01084f0 0x7f53a01088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:53.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.894+0000 7f53a724b700 1 --2- 172.21.15.37:0/1191225622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53a006aa00 0x7f53a006ae80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:53.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.894+0000 7f53a724b700 1 -- 172.21.15.37:0/1191225622 >> 172.21.15.37:0/1191225622 conn(0x7f53a0078300 msgr2=0x7f53a0078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:53.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.894+0000 7f53a724b700 1 -- 172.21.15.37:0/1191225622 shutdown_connections 2024-06-10T00:54:53.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.894+0000 7f53a724b700 1 -- 172.21.15.37:0/1191225622 wait complete. 2024-06-10T00:54:53.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.895+0000 7f53a724b700 1 Processor -- start 2024-06-10T00:54:53.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.895+0000 7f53a724b700 1 -- start start 2024-06-10T00:54:53.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.895+0000 7f53a724b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53a006aa00 0x7f53a019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:53.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.895+0000 7f53a724b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53a006b3c0 0x7f53a019edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:53.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.895+0000 7f53a724b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53a01084f0 0x7f53a0198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:53.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.895+0000 7f53a724b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f53a0112770 con 0x7f53a006aa00 2024-06-10T00:54:53.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.895+0000 7f53a724b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f53a01125f0 con 0x7f53a01084f0 2024-06-10T00:54:53.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.895+0000 7f53a724b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f53a01128f0 con 0x7f53a006b3c0 2024-06-10T00:54:53.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.895+0000 7f53a4fe7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53a006aa00 0x7f53a019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:53.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.895+0000 7f53a4fe7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53a006aa00 0x7f53a019e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49630/0 (socket says 172.21.15.37:49630) 2024-06-10T00:54:53.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.895+0000 7f53a4fe7700 1 -- 172.21.15.37:0/2877229414 learned_addr learned my addr 172.21.15.37:0/2877229414 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:54:53.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.895+0000 7f539ffff700 1 --2- 172.21.15.37:0/2877229414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53a006b3c0 0x7f53a019edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:53.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.896+0000 7f53a57e8700 1 --2- 172.21.15.37:0/2877229414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53a01084f0 0x7f53a0198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:53.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.896+0000 7f53a4fe7700 1 -- 172.21.15.37:0/2877229414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53a006b3c0 msgr2=0x7f53a019edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:53.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.896+0000 7f53a4fe7700 1 --2- 172.21.15.37:0/2877229414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53a006b3c0 0x7f53a019edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:53.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.896+0000 7f53a4fe7700 1 -- 172.21.15.37:0/2877229414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53a01084f0 msgr2=0x7f53a0198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:53.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.896+0000 7f53a4fe7700 1 --2- 172.21.15.37:0/2877229414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53a01084f0 0x7f53a0198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:53.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.896+0000 7f53a4fe7700 1 -- 172.21.15.37:0/2877229414 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a01991b0 con 0x7f53a006aa00 2024-06-10T00:54:53.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.896+0000 7f53a4fe7700 1 --2- 172.21.15.37:0/2877229414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53a006aa00 0x7f53a019e890 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f539400d980 tx=0x7f539400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:53.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.896+0000 7f539dffb700 1 -- 172.21.15.37:0/2877229414 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394017070 con 0x7f53a006aa00 2024-06-10T00:54:53.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.896+0000 7f539dffb700 1 -- 172.21.15.37:0/2877229414 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5394005020 con 0x7f53a006aa00 2024-06-10T00:54:53.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.896+0000 7f53a724b700 1 -- 172.21.15.37:0/2877229414 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a01994a0 con 0x7f53a006aa00 2024-06-10T00:54:53.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.896+0000 7f539dffb700 1 -- 172.21.15.37:0/2877229414 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394009bc0 con 0x7f53a006aa00 2024-06-10T00:54:53.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.896+0000 7f53a724b700 1 -- 172.21.15.37:0/2877229414 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a01a5760 con 0x7f53a006aa00 2024-06-10T00:54:53.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.897+0000 7f53a724b700 1 -- 172.21.15.37:0/2877229414 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53a0069180 con 0x7f53a006aa00 2024-06-10T00:54:53.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.898+0000 7f539dffb700 1 -- 172.21.15.37:0/2877229414 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f53940051d0 con 0x7f53a006aa00 2024-06-10T00:54:53.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.899+0000 7f539dffb700 1 --2- 172.21.15.37:0/2877229414 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f53900a2d30 0x7f53900a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:53.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.899+0000 7f539ffff700 1 --2- 172.21.15.37:0/2877229414 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f53900a2d30 0x7f53900a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:53.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.899+0000 7f539dffb700 1 -- 172.21.15.37:0/2877229414 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f53940cf4b0 con 0x7f53a006aa00 2024-06-10T00:54:53.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.900+0000 7f539ffff700 1 --2- 172.21.15.37:0/2877229414 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f53900a2d30 0x7f53900a51f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f538c00b2e0 tx=0x7f538c005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:53.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:53.904+0000 7f539dffb700 1 -- 172.21.15.37:0/2877229414 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5394020020 con 0x7f53a006aa00 2024-06-10T00:54:54.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:54.116+0000 7f53a724b700 1 -- 172.21.15.37:0/2877229414 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f53a002dca0 con 0x7f53a006aa00 2024-06-10T00:54:54.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:54.117+0000 7f539dffb700 1 -- 172.21.15.37:0/2877229414 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f539409be60 con 0x7f53a006aa00 2024-06-10T00:54:54.117 INFO:teuthology.orchestra.run.smithi037.stdout:146028888142 2024-06-10T00:54:54.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:54.119+0000 7f53a724b700 1 -- 172.21.15.37:0/2877229414 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f53900a2d30 msgr2=0x7f53900a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:54.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:54.119+0000 7f53a724b700 1 --2- 172.21.15.37:0/2877229414 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f53900a2d30 0x7f53900a51f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f538c00b2e0 tx=0x7f538c005960 comp rx=0 tx=0).stop 2024-06-10T00:54:54.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:54.119+0000 7f53a724b700 1 -- 172.21.15.37:0/2877229414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53a006aa00 msgr2=0x7f53a019e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:54.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:54.119+0000 7f53a724b700 1 --2- 172.21.15.37:0/2877229414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53a006aa00 0x7f53a019e890 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f539400d980 tx=0x7f539400dc90 comp rx=0 tx=0).stop 2024-06-10T00:54:54.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:54.120+0000 7f53a724b700 1 -- 172.21.15.37:0/2877229414 shutdown_connections 2024-06-10T00:54:54.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:54.120+0000 7f53a724b700 1 --2- 172.21.15.37:0/2877229414 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f53900a2d30 0x7f53900a51f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:54.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:54.120+0000 7f53a724b700 1 --2- 172.21.15.37:0/2877229414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53a01084f0 0x7f53a0198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:54.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:54.120+0000 7f53a724b700 1 --2- 172.21.15.37:0/2877229414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53a006aa00 0x7f53a019e890 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:54.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:54.120+0000 7f53a724b700 1 --2- 172.21.15.37:0/2877229414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53a006b3c0 0x7f53a019edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:54.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:54.120+0000 7f53a724b700 1 -- 172.21.15.37:0/2877229414 >> 172.21.15.37:0/2877229414 conn(0x7f53a0078300 msgr2=0x7f53a00fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:54.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:54.120+0000 7f53a724b700 1 -- 172.21.15.37:0/2877229414 shutdown_connections 2024-06-10T00:54:54.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:54.120+0000 7f53a724b700 1 -- 172.21.15.37:0/2877229414 wait complete. 2024-06-10T00:54:54.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:54 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2877229414' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-10T00:54:54.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:54 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2877229414' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-10T00:54:54.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:54 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2877229414' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-10T00:54:55.027 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:55.030 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:55.033 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888140 got 146028888142 for osd.4 2024-06-10T00:54:55.033 DEBUG:teuthology.parallel:result is None 2024-06-10T00:54:55.587 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:55 smithi037 ceph-mon[32192]: pgmap v377: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T00:54:55.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:55 smithi167 ceph-mon[36917]: pgmap v377: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T00:54:55.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:55 smithi094 ceph-mon[36708]: pgmap v377: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T00:54:57.560 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:57.779 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:57 smithi037 ceph-mon[32192]: pgmap v378: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:54:57.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:57 smithi167 ceph-mon[36917]: pgmap v378: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:54:57.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:57 smithi094 ceph-mon[36708]: pgmap v378: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:54:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.253+0000 7f65e0593700 1 -- 172.21.15.37:0/1861236082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65d8101d40 msgr2=0x7f65d810e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.253+0000 7f65e0593700 1 --2- 172.21.15.37:0/1861236082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65d8101d40 0x7f65d810e230 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f65d000b0a0 tx=0x7f65d000b3b0 comp rx=0 tx=0).stop 2024-06-10T00:54:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.253+0000 7f65e0593700 1 -- 172.21.15.37:0/1861236082 shutdown_connections 2024-06-10T00:54:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.253+0000 7f65e0593700 1 --2- 172.21.15.37:0/1861236082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65d8101d40 0x7f65d810e230 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.253+0000 7f65e0593700 1 --2- 172.21.15.37:0/1861236082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65d81073f0 0x7f65d81077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.253+0000 7f65e0593700 1 --2- 172.21.15.37:0/1861236082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65d8101380 0x7f65d8101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.253+0000 7f65e0593700 1 -- 172.21.15.37:0/1861236082 >> 172.21.15.37:0/1861236082 conn(0x7f65d8078300 msgr2=0x7f65d8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.253+0000 7f65e0593700 1 -- 172.21.15.37:0/1861236082 shutdown_connections 2024-06-10T00:54:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.254+0000 7f65e0593700 1 -- 172.21.15.37:0/1861236082 wait complete. 2024-06-10T00:54:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.254+0000 7f65e0593700 1 Processor -- start 2024-06-10T00:54:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.254+0000 7f65e0593700 1 -- start start 2024-06-10T00:54:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.255+0000 7f65e0593700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65d8101380 0x7f65d81977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.255+0000 7f65e0593700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65d8101d40 0x7f65d8197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.255+0000 7f65ddb2e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65d8101d40 0x7f65d8197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.255+0000 7f65ddb2e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65d8101d40 0x7f65d8197d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49662/0 (socket says 172.21.15.37:49662) 2024-06-10T00:54:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.255+0000 7f65ddb2e700 1 -- 172.21.15.37:0/4100449601 learned_addr learned my addr 172.21.15.37:0/4100449601 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:54:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.255+0000 7f65e0593700 1 --2- 172.21.15.37:0/4100449601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65d81073f0 0x7f65d8198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.255+0000 7f65e0593700 1 -- 172.21.15.37:0/4100449601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f65d8111660 con 0x7f65d8101d40 2024-06-10T00:54:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.255+0000 7f65e0593700 1 -- 172.21.15.37:0/4100449601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f65d81114e0 con 0x7f65d81073f0 2024-06-10T00:54:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.255+0000 7f65e0593700 1 -- 172.21.15.37:0/4100449601 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f65d81117e0 con 0x7f65d8101380 2024-06-10T00:54:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.255+0000 7f65de32f700 1 --2- 172.21.15.37:0/4100449601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65d8101380 0x7f65d81977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.255+0000 7f65deb30700 1 --2- 172.21.15.37:0/4100449601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65d81073f0 0x7f65d8198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.255+0000 7f65ddb2e700 1 -- 172.21.15.37:0/4100449601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65d8101380 msgr2=0x7f65d81977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.255+0000 7f65ddb2e700 1 --2- 172.21.15.37:0/4100449601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65d8101380 0x7f65d81977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.255+0000 7f65ddb2e700 1 -- 172.21.15.37:0/4100449601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65d81073f0 msgr2=0x7f65d8198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:57.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.255+0000 7f65ddb2e700 1 --2- 172.21.15.37:0/4100449601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65d81073f0 0x7f65d8198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:57.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.255+0000 7f65ddb2e700 1 -- 172.21.15.37:0/4100449601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d81a70d0 con 0x7f65d8101d40 2024-06-10T00:54:57.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.256+0000 7f65ddb2e700 1 --2- 172.21.15.37:0/4100449601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65d8101d40 0x7f65d8197d10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f65c800d950 tx=0x7f65c800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:57.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.256+0000 7f65cf7fe700 1 -- 172.21.15.37:0/4100449601 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c8016070 con 0x7f65d8101d40 2024-06-10T00:54:57.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.256+0000 7f65e0593700 1 -- 172.21.15.37:0/4100449601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65d81a73c0 con 0x7f65d8101d40 2024-06-10T00:54:57.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.256+0000 7f65cf7fe700 1 -- 172.21.15.37:0/4100449601 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65c8009e20 con 0x7f65d8101d40 2024-06-10T00:54:57.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.256+0000 7f65cf7fe700 1 -- 172.21.15.37:0/4100449601 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c800f620 con 0x7f65d8101d40 2024-06-10T00:54:57.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.256+0000 7f65e0593700 1 -- 172.21.15.37:0/4100449601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65d81a7950 con 0x7f65d8101d40 2024-06-10T00:54:57.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.257+0000 7f65e0593700 1 -- 172.21.15.37:0/4100449601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65d8069180 con 0x7f65d8101d40 2024-06-10T00:54:57.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.258+0000 7f65cf7fe700 1 -- 172.21.15.37:0/4100449601 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f65c8019070 con 0x7f65d8101d40 2024-06-10T00:54:57.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.259+0000 7f65cf7fe700 1 --2- 172.21.15.37:0/4100449601 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f65c40a2d30 0x7f65c40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:57.887 INFO:teuthology.orchestra.run.smithi037.stdout:167503724618 2024-06-10T00:54:57.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.259+0000 7f65cf7fe700 1 -- 172.21.15.37:0/4100449601 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f65c80cf250 con 0x7f65d8101d40 2024-06-10T00:54:57.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.259+0000 7f65de32f700 1 --2- 172.21.15.37:0/4100449601 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f65c40a2d30 0x7f65c40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:57.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.260+0000 7f65de32f700 1 --2- 172.21.15.37:0/4100449601 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f65c40a2d30 0x7f65c40a51f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f65d400abd0 tx=0x7f65d400a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:57.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.264+0000 7f65cf7fe700 1 -- 172.21.15.37:0/4100449601 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f65c809bc00 con 0x7f65d8101d40 2024-06-10T00:54:57.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.479+0000 7f65e0593700 1 -- 172.21.15.37:0/4100449601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f65d802dca0 con 0x7f65d8101d40 2024-06-10T00:54:57.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.480+0000 7f65cf7fe700 1 -- 172.21.15.37:0/4100449601 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f65c809b500 con 0x7f65d8101d40 2024-06-10T00:54:57.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.482+0000 7f65e0593700 1 -- 172.21.15.37:0/4100449601 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f65c40a2d30 msgr2=0x7f65c40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:57.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.482+0000 7f65e0593700 1 --2- 172.21.15.37:0/4100449601 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f65c40a2d30 0x7f65c40a51f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f65d400abd0 tx=0x7f65d400a380 comp rx=0 tx=0).stop 2024-06-10T00:54:57.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.482+0000 7f65e0593700 1 -- 172.21.15.37:0/4100449601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65d8101d40 msgr2=0x7f65d8197d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:57.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.482+0000 7f65e0593700 1 --2- 172.21.15.37:0/4100449601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65d8101d40 0x7f65d8197d10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f65c800d950 tx=0x7f65c800dc60 comp rx=0 tx=0).stop 2024-06-10T00:54:57.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.482+0000 7f65e0593700 1 -- 172.21.15.37:0/4100449601 shutdown_connections 2024-06-10T00:54:57.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.482+0000 7f65e0593700 1 --2- 172.21.15.37:0/4100449601 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f65c40a2d30 0x7f65c40a51f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:57.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.482+0000 7f65e0593700 1 --2- 172.21.15.37:0/4100449601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65d81073f0 0x7f65d8198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:57.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.482+0000 7f65e0593700 1 --2- 172.21.15.37:0/4100449601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65d8101380 0x7f65d81977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:57.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.482+0000 7f65e0593700 1 --2- 172.21.15.37:0/4100449601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65d8101d40 0x7f65d8197d10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:57.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.482+0000 7f65e0593700 1 -- 172.21.15.37:0/4100449601 >> 172.21.15.37:0/4100449601 conn(0x7f65d8078300 msgr2=0x7f65d80ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:57.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.482+0000 7f65e0593700 1 -- 172.21.15.37:0/4100449601 shutdown_connections 2024-06-10T00:54:57.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:57.482+0000 7f65e0593700 1 -- 172.21.15.37:0/4100449601 wait complete. 2024-06-10T00:54:58.325 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:58.516 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:58 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/4100449601' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-10T00:54:58.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:58 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/4100449601' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-10T00:54:58.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:58 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/4100449601' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-10T00:54:59.349 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724615 got 167503724618 for osd.5 2024-06-10T00:54:59.349 DEBUG:teuthology.parallel:result is None 2024-06-10T00:54:59.624 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:54:59 smithi037 ceph-mon[32192]: pgmap v379: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:54:59.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.645+0000 7f73661ef700 1 -- 172.21.15.37:0/1340998930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f736006b810 msgr2=0x7f73601070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:59.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.645+0000 7f73661ef700 1 --2- 172.21.15.37:0/1340998930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f736006b810 0x7f73601070d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f73540099c0 tx=0x7f7354009cd0 comp rx=0 tx=0).stop 2024-06-10T00:54:59.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.645+0000 7f73661ef700 1 -- 172.21.15.37:0/1340998930 shutdown_connections 2024-06-10T00:54:59.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.645+0000 7f73661ef700 1 --2- 172.21.15.37:0/1340998930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73601077b0 0x7f736010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:59.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.645+0000 7f73661ef700 1 --2- 172.21.15.37:0/1340998930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f736006ae60 0x7f736006b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:59.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.645+0000 7f73661ef700 1 --2- 172.21.15.37:0/1340998930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f736006b810 0x7f73601070d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:59.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.645+0000 7f73661ef700 1 -- 172.21.15.37:0/1340998930 >> 172.21.15.37:0/1340998930 conn(0x7f73600773e0 msgr2=0x7f73600777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:59.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.645+0000 7f73661ef700 1 -- 172.21.15.37:0/1340998930 shutdown_connections 2024-06-10T00:54:59.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.645+0000 7f73661ef700 1 -- 172.21.15.37:0/1340998930 wait complete. 2024-06-10T00:54:59.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.645+0000 7f73661ef700 1 Processor -- start 2024-06-10T00:54:59.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.646+0000 7f73661ef700 1 -- start start 2024-06-10T00:54:59.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.646+0000 7f73661ef700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f736006ae60 0x7f736019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:59.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.646+0000 7f73661ef700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f736006b810 0x7f736019edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:59.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.646+0000 7f73661ef700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73601077b0 0x7f7360198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:59.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.646+0000 7f73661ef700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7360112830 con 0x7f736006b810 2024-06-10T00:54:59.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.646+0000 7f73661ef700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f73601126b0 con 0x7f73601077b0 2024-06-10T00:54:59.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.646+0000 7f73661ef700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f73601129b0 con 0x7f736006ae60 2024-06-10T00:54:59.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.646+0000 7f735effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f736006b810 0x7f736019edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:59.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.646+0000 7f735effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f736006b810 0x7f736019edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49680/0 (socket says 172.21.15.37:49680) 2024-06-10T00:54:59.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.646+0000 7f735effd700 1 -- 172.21.15.37:0/4267845232 learned_addr learned my addr 172.21.15.37:0/4267845232 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:54:59.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.646+0000 7f735ffff700 1 --2- 172.21.15.37:0/4267845232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73601077b0 0x7f7360198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:59.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.646+0000 7f735f7fe700 1 --2- 172.21.15.37:0/4267845232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f736006ae60 0x7f736019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:59.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.646+0000 7f735effd700 1 -- 172.21.15.37:0/4267845232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f736006ae60 msgr2=0x7f736019e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:59.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.646+0000 7f735effd700 1 --2- 172.21.15.37:0/4267845232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f736006ae60 0x7f736019e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:59.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.646+0000 7f735effd700 1 -- 172.21.15.37:0/4267845232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73601077b0 msgr2=0x7f7360198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:59.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.646+0000 7f735effd700 1 --2- 172.21.15.37:0/4267845232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73601077b0 0x7f7360198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:59.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.646+0000 7f735effd700 1 -- 172.21.15.37:0/4267845232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73601991b0 con 0x7f736006b810 2024-06-10T00:54:59.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.647+0000 7f735effd700 1 --2- 172.21.15.37:0/4267845232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f736006b810 0x7f736019edd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7354009fd0 tx=0x7f7354005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:59.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.647+0000 7f735cff9700 1 -- 172.21.15.37:0/4267845232 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7354004830 con 0x7f736006b810 2024-06-10T00:54:59.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.647+0000 7f73661ef700 1 -- 172.21.15.37:0/4267845232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7360199440 con 0x7f736006b810 2024-06-10T00:54:59.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.647+0000 7f735cff9700 1 -- 172.21.15.37:0/4267845232 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7354005020 con 0x7f736006b810 2024-06-10T00:54:59.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.647+0000 7f735cff9700 1 -- 172.21.15.37:0/4267845232 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7354003f90 con 0x7f736006b810 2024-06-10T00:54:59.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.647+0000 7f73661ef700 1 -- 172.21.15.37:0/4267845232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73601a5760 con 0x7f736006b810 2024-06-10T00:54:59.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.648+0000 7f73661ef700 1 -- 172.21.15.37:0/4267845232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7360069180 con 0x7f736006b810 2024-06-10T00:54:59.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.649+0000 7f735cff9700 1 -- 172.21.15.37:0/4267845232 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7354005540 con 0x7f736006b810 2024-06-10T00:54:59.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.650+0000 7f735cff9700 1 --2- 172.21.15.37:0/4267845232 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f73480a2cb0 0x7f73480a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:54:59.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.650+0000 7f735f7fe700 1 --2- 172.21.15.37:0/4267845232 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f73480a2cb0 0x7f73480a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:54:59.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.650+0000 7f735cff9700 1 -- 172.21.15.37:0/4267845232 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f73540d0600 con 0x7f736006b810 2024-06-10T00:54:59.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.650+0000 7f735f7fe700 1 --2- 172.21.15.37:0/4267845232 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f73480a2cb0 0x7f73480a5170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7350006fd0 tx=0x7f7350008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:54:59.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.654+0000 7f735cff9700 1 -- 172.21.15.37:0/4267845232 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7354013070 con 0x7f736006b810 2024-06-10T00:54:59.755 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:54:59.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:54:59 smithi167 ceph-mon[36917]: pgmap v379: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:54:59.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:54:59 smithi094 ceph-mon[36708]: pgmap v379: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:54:59.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.864+0000 7f73661ef700 1 -- 172.21.15.37:0/4267845232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f736002dca0 con 0x7f736006b810 2024-06-10T00:54:59.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.865+0000 7f735cff9700 1 -- 172.21.15.37:0/4267845232 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f735409d0a0 con 0x7f736006b810 2024-06-10T00:54:59.865 INFO:teuthology.orchestra.run.smithi037.stdout:201863462982 2024-06-10T00:54:59.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.867+0000 7f73661ef700 1 -- 172.21.15.37:0/4267845232 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f73480a2cb0 msgr2=0x7f73480a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:59.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.867+0000 7f73661ef700 1 --2- 172.21.15.37:0/4267845232 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f73480a2cb0 0x7f73480a5170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7350006fd0 tx=0x7f7350008040 comp rx=0 tx=0).stop 2024-06-10T00:54:59.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.867+0000 7f73661ef700 1 -- 172.21.15.37:0/4267845232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f736006b810 msgr2=0x7f736019edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:54:59.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.867+0000 7f73661ef700 1 --2- 172.21.15.37:0/4267845232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f736006b810 0x7f736019edd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7354009fd0 tx=0x7f7354005f10 comp rx=0 tx=0).stop 2024-06-10T00:54:59.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.867+0000 7f73661ef700 1 -- 172.21.15.37:0/4267845232 shutdown_connections 2024-06-10T00:54:59.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.867+0000 7f73661ef700 1 --2- 172.21.15.37:0/4267845232 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f73480a2cb0 0x7f73480a5170 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:59.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.867+0000 7f73661ef700 1 --2- 172.21.15.37:0/4267845232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73601077b0 0x7f7360198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:59.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.867+0000 7f73661ef700 1 --2- 172.21.15.37:0/4267845232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f736006ae60 0x7f736019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:59.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.867+0000 7f73661ef700 1 --2- 172.21.15.37:0/4267845232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f736006b810 0x7f736019edd0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:54:59.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.867+0000 7f73661ef700 1 -- 172.21.15.37:0/4267845232 >> 172.21.15.37:0/4267845232 conn(0x7f73600773e0 msgr2=0x7f7360100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:54:59.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.867+0000 7f73661ef700 1 -- 172.21.15.37:0/4267845232 shutdown_connections 2024-06-10T00:54:59.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:54:59.867+0000 7f73661ef700 1 -- 172.21.15.37:0/4267845232 wait complete. 2024-06-10T00:55:00.791 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:00 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/4267845232' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-10T00:55:00.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.799+0000 7f66d7ce1700 1 -- 172.21.15.37:0/1156401237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66d006ac60 msgr2=0x7f66d006b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:00.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.799+0000 7f66d7ce1700 1 --2- 172.21.15.37:0/1156401237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66d006ac60 0x7f66d006b0e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f66c00099c0 tx=0x7f66c0009cd0 comp rx=0 tx=0).stop 2024-06-10T00:55:00.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.799+0000 7f66d7ce1700 1 -- 172.21.15.37:0/1156401237 shutdown_connections 2024-06-10T00:55:00.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.799+0000 7f66d7ce1700 1 --2- 172.21.15.37:0/1156401237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66d006b620 0x7f66d010f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:00.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.799+0000 7f66d7ce1700 1 --2- 172.21.15.37:0/1156401237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66d0108580 0x7f66d0108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:00.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.799+0000 7f66d7ce1700 1 --2- 172.21.15.37:0/1156401237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66d006ac60 0x7f66d006b0e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:00.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.799+0000 7f66d7ce1700 1 -- 172.21.15.37:0/1156401237 >> 172.21.15.37:0/1156401237 conn(0x7f66d0078430 msgr2=0x7f66d0078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:00.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.800+0000 7f66d7ce1700 1 -- 172.21.15.37:0/1156401237 shutdown_connections 2024-06-10T00:55:00.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.800+0000 7f66d7ce1700 1 -- 172.21.15.37:0/1156401237 wait complete. 2024-06-10T00:55:00.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.800+0000 7f66d7ce1700 1 Processor -- start 2024-06-10T00:55:00.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.800+0000 7f66d7ce1700 1 -- start start 2024-06-10T00:55:00.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.800+0000 7f66d7ce1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66d006ac60 0x7f66d0101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:00.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.800+0000 7f66d7ce1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66d006b620 0x7f66d0101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:00.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.800+0000 7f66d7ce1700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66d0108580 0x7f66d0101e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:00.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.800+0000 7f66d7ce1700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f66d01127c0 con 0x7f66d006b620 2024-06-10T00:55:00.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.800+0000 7f66d7ce1700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f66d0112640 con 0x7f66d006ac60 2024-06-10T00:55:00.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.800+0000 7f66d7ce1700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f66d0112940 con 0x7f66d0108580 2024-06-10T00:55:00.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.801+0000 7f66d527c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66d006b620 0x7f66d0101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:00.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.801+0000 7f66d527c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66d006b620 0x7f66d0101940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49688/0 (socket says 172.21.15.37:49688) 2024-06-10T00:55:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.801+0000 7f66d527c700 1 -- 172.21.15.37:0/2058280264 learned_addr learned my addr 172.21.15.37:0/2058280264 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.801+0000 7f66d5a7d700 1 --2- 172.21.15.37:0/2058280264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66d006ac60 0x7f66d0101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.801+0000 7f66d627e700 1 --2- 172.21.15.37:0/2058280264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66d0108580 0x7f66d0101e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.801+0000 7f66d527c700 1 -- 172.21.15.37:0/2058280264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66d0108580 msgr2=0x7f66d0101e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.801+0000 7f66d527c700 1 --2- 172.21.15.37:0/2058280264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66d0108580 0x7f66d0101e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.801+0000 7f66d527c700 1 -- 172.21.15.37:0/2058280264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66d006ac60 msgr2=0x7f66d0101400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.801+0000 7f66d527c700 1 --2- 172.21.15.37:0/2058280264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66d006ac60 0x7f66d0101400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.801+0000 7f66d527c700 1 -- 172.21.15.37:0/2058280264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d01a70d0 con 0x7f66d006b620 2024-06-10T00:55:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.801+0000 7f66d527c700 1 --2- 172.21.15.37:0/2058280264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66d006b620 0x7f66d0101940 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f66c00059d0 tx=0x7f66c00038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.801+0000 7f66c6ffd700 1 -- 172.21.15.37:0/2058280264 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c0027070 con 0x7f66d006b620 2024-06-10T00:55:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.801+0000 7f66c6ffd700 1 -- 172.21.15.37:0/2058280264 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66c0004050 con 0x7f66d006b620 2024-06-10T00:55:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.801+0000 7f66d7ce1700 1 -- 172.21.15.37:0/2058280264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d01a7360 con 0x7f66d006b620 2024-06-10T00:55:00.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.801+0000 7f66c6ffd700 1 -- 172.21.15.37:0/2058280264 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c0022db0 con 0x7f66d006b620 2024-06-10T00:55:00.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.801+0000 7f66d7ce1700 1 -- 172.21.15.37:0/2058280264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d01a77b0 con 0x7f66d006b620 2024-06-10T00:55:00.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.802+0000 7f66d7ce1700 1 -- 172.21.15.37:0/2058280264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66d0069180 con 0x7f66d006b620 2024-06-10T00:55:00.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.804+0000 7f66c6ffd700 1 -- 172.21.15.37:0/2058280264 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f66c002a070 con 0x7f66d006b620 2024-06-10T00:55:00.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.804+0000 7f66c6ffd700 1 --2- 172.21.15.37:0/2058280264 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f66bc0a2cb0 0x7f66bc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:00.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.804+0000 7f66d5a7d700 1 --2- 172.21.15.37:0/2058280264 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f66bc0a2cb0 0x7f66bc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:00.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.804+0000 7f66c6ffd700 1 -- 172.21.15.37:0/2058280264 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f66c00d0b00 con 0x7f66d006b620 2024-06-10T00:55:00.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.805+0000 7f66d5a7d700 1 --2- 172.21.15.37:0/2058280264 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f66bc0a2cb0 0x7f66bc0a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f66cc005950 tx=0x7f66cc00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:00.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:00.808+0000 7f66c6ffd700 1 -- 172.21.15.37:0/2058280264 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f66c0031020 con 0x7f66d006b620 2024-06-10T00:55:00.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:00 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/4267845232' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-10T00:55:00.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:00 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/4267845232' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-10T00:55:01.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:01.018+0000 7f66d7ce1700 1 -- 172.21.15.37:0/2058280264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f66d002df00 con 0x7f66d006b620 2024-06-10T00:55:01.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:01.018+0000 7f66c6ffd700 1 -- 172.21.15.37:0/2058280264 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f66c0013070 con 0x7f66d006b620 2024-06-10T00:55:01.019 INFO:teuthology.orchestra.run.smithi037.stdout:257698037820 2024-06-10T00:55:01.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:01.021+0000 7f66d7ce1700 1 -- 172.21.15.37:0/2058280264 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f66bc0a2cb0 msgr2=0x7f66bc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:01.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:01.021+0000 7f66d7ce1700 1 --2- 172.21.15.37:0/2058280264 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f66bc0a2cb0 0x7f66bc0a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f66cc005950 tx=0x7f66cc00b410 comp rx=0 tx=0).stop 2024-06-10T00:55:01.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:01.021+0000 7f66d7ce1700 1 -- 172.21.15.37:0/2058280264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66d006b620 msgr2=0x7f66d0101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:01.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:01.021+0000 7f66d7ce1700 1 --2- 172.21.15.37:0/2058280264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66d006b620 0x7f66d0101940 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f66c00059d0 tx=0x7f66c00038b0 comp rx=0 tx=0).stop 2024-06-10T00:55:01.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:01.021+0000 7f66d7ce1700 1 -- 172.21.15.37:0/2058280264 shutdown_connections 2024-06-10T00:55:01.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:01.021+0000 7f66d7ce1700 1 --2- 172.21.15.37:0/2058280264 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f66bc0a2cb0 0x7f66bc0a5170 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:01.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:01.021+0000 7f66d7ce1700 1 --2- 172.21.15.37:0/2058280264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66d0108580 0x7f66d0101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:01.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:01.021+0000 7f66d7ce1700 1 --2- 172.21.15.37:0/2058280264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66d006ac60 0x7f66d0101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:01.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:01.021+0000 7f66d7ce1700 1 --2- 172.21.15.37:0/2058280264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66d006b620 0x7f66d0101940 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:01.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:01.021+0000 7f66d7ce1700 1 -- 172.21.15.37:0/2058280264 >> 172.21.15.37:0/2058280264 conn(0x7f66d0078430 msgr2=0x7f66d00ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:01.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:01.021+0000 7f66d7ce1700 1 -- 172.21.15.37:0/2058280264 shutdown_connections 2024-06-10T00:55:01.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:01.021+0000 7f66d7ce1700 1 -- 172.21.15.37:0/2058280264 wait complete. 2024-06-10T00:55:01.638 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:55:01.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:01 smithi037 ceph-mon[32192]: pgmap v380: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:55:01.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:01 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2058280264' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-10T00:55:01.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:01 smithi167 ceph-mon[36917]: pgmap v380: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:55:01.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:01 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2058280264' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-10T00:55:01.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:01 smithi094 ceph-mon[36708]: pgmap v380: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:55:01.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:01 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2058280264' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-10T00:55:02.073 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462977 got 201863462982 for osd.6 2024-06-10T00:55:02.074 DEBUG:teuthology.parallel:result is None 2024-06-10T00:55:02.270 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:55:03.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:03 smithi037 ceph-mon[32192]: pgmap v381: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:55:03.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:03 smithi167 ceph-mon[36917]: pgmap v381: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:55:03.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:03 smithi094 ceph-mon[36708]: pgmap v381: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:55:05.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.486+0000 7f0c1dc9c700 1 -- 172.21.15.37:0/1141251987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c18102740 msgr2=0x7f0c18102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:05.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.486+0000 7f0c1dc9c700 1 --2- 172.21.15.37:0/1141251987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c18102740 0x7f0c18102b20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0c080099c0 tx=0x7f0c08009cd0 comp rx=0 tx=0).stop 2024-06-10T00:55:05.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.486+0000 7f0c1dc9c700 1 -- 172.21.15.37:0/1141251987 shutdown_connections 2024-06-10T00:55:05.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.486+0000 7f0c1dc9c700 1 --2- 172.21.15.37:0/1141251987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c18107840 0x7f0c1810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:05.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.486+0000 7f0c1dc9c700 1 --2- 172.21.15.37:0/1141251987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c18102740 0x7f0c18102b20 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:05.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.486+0000 7f0c1dc9c700 1 --2- 172.21.15.37:0/1141251987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c181030f0 0x7f0c18107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:05.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.486+0000 7f0c1dc9c700 1 -- 172.21.15.37:0/1141251987 >> 172.21.15.37:0/1141251987 conn(0x7f0c180795f0 msgr2=0x7f0c18079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:05.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.486+0000 7f0c1dc9c700 1 -- 172.21.15.37:0/1141251987 shutdown_connections 2024-06-10T00:55:05.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.486+0000 7f0c1dc9c700 1 -- 172.21.15.37:0/1141251987 wait complete. 2024-06-10T00:55:05.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.487+0000 7f0c1dc9c700 1 Processor -- start 2024-06-10T00:55:05.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.487+0000 7f0c1dc9c700 1 -- start start 2024-06-10T00:55:05.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.487+0000 7f0c1dc9c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c18102740 0x7f0c18074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:05.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.487+0000 7f0c1dc9c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c181030f0 0x7f0c18074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:05.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.488+0000 7f0c1dc9c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c18107840 0x7f0c18075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:05.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.488+0000 7f0c1dc9c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0c18112760 con 0x7f0c18102740 2024-06-10T00:55:05.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.488+0000 7f0c1dc9c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0c181125e0 con 0x7f0c181030f0 2024-06-10T00:55:05.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.488+0000 7f0c1dc9c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0c181128e0 con 0x7f0c18107840 2024-06-10T00:55:05.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.488+0000 7f0c177fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c18102740 0x7f0c18074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:05.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.488+0000 7f0c177fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c18102740 0x7f0c18074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43798/0 (socket says 172.21.15.37:43798) 2024-06-10T00:55:05.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.488+0000 7f0c177fe700 1 -- 172.21.15.37:0/18848554 learned_addr learned my addr 172.21.15.37:0/18848554 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:05.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.488+0000 7f0c17fff700 1 --2- 172.21.15.37:0/18848554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c18107840 0x7f0c18075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:05.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.488+0000 7f0c16ffd700 1 --2- 172.21.15.37:0/18848554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c181030f0 0x7f0c18074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:05.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.488+0000 7f0c177fe700 1 -- 172.21.15.37:0/18848554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c18107840 msgr2=0x7f0c18075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:05.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.488+0000 7f0c177fe700 1 --2- 172.21.15.37:0/18848554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c18107840 0x7f0c18075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:05.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.488+0000 7f0c177fe700 1 -- 172.21.15.37:0/18848554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c181030f0 msgr2=0x7f0c18074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:05.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.488+0000 7f0c177fe700 1 --2- 172.21.15.37:0/18848554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c181030f0 0x7f0c18074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:05.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.488+0000 7f0c177fe700 1 -- 172.21.15.37:0/18848554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c181a7070 con 0x7f0c18102740 2024-06-10T00:55:05.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.488+0000 7f0c17fff700 1 --2- 172.21.15.37:0/18848554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c18107840 0x7f0c18075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:55:05.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.488+0000 7f0c177fe700 1 --2- 172.21.15.37:0/18848554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c18102740 0x7f0c18074790 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0c080059d0 tx=0x7f0c080038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:05.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.488+0000 7f0c14ff9700 1 -- 172.21.15.37:0/18848554 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08027070 con 0x7f0c18102740 2024-06-10T00:55:05.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.488+0000 7f0c1dc9c700 1 -- 172.21.15.37:0/18848554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c181a7300 con 0x7f0c18102740 2024-06-10T00:55:05.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.488+0000 7f0c14ff9700 1 -- 172.21.15.37:0/18848554 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c08004050 con 0x7f0c18102740 2024-06-10T00:55:05.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.488+0000 7f0c1dc9c700 1 -- 172.21.15.37:0/18848554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c181a77b0 con 0x7f0c18102740 2024-06-10T00:55:05.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.489+0000 7f0c14ff9700 1 -- 172.21.15.37:0/18848554 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08022d50 con 0x7f0c18102740 2024-06-10T00:55:05.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.489+0000 7f0c1dc9c700 1 -- 172.21.15.37:0/18848554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c1810aa80 con 0x7f0c18102740 2024-06-10T00:55:05.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.490+0000 7f0c14ff9700 1 -- 172.21.15.37:0/18848554 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0c0802a070 con 0x7f0c18102740 2024-06-10T00:55:05.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.491+0000 7f0c14ff9700 1 --2- 172.21.15.37:0/18848554 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0c000a2cb0 0x7f0c000a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:05.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.491+0000 7f0c14ff9700 1 -- 172.21.15.37:0/18848554 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f0c080d0610 con 0x7f0c18102740 2024-06-10T00:55:05.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.491+0000 7f0c16ffd700 1 --2- 172.21.15.37:0/18848554 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0c000a2cb0 0x7f0c000a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:05.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.492+0000 7f0c16ffd700 1 --2- 172.21.15.37:0/18848554 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0c000a2cb0 0x7f0c000a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0c18108bb0 tx=0x7f0c0c00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:05.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.496+0000 7f0c14ff9700 1 -- 172.21.15.37:0/18848554 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0c0805e850 con 0x7f0c18102740 2024-06-10T00:55:05.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.707+0000 7f0c1dc9c700 1 -- 172.21.15.37:0/18848554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f0c18075ca0 con 0x7f0c18102740 2024-06-10T00:55:05.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.707+0000 7f0c14ff9700 1 -- 172.21.15.37:0/18848554 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f0c18075ca0 con 0x7f0c18102740 2024-06-10T00:55:05.708 INFO:teuthology.orchestra.run.smithi037.stdout:313532612659 2024-06-10T00:55:05.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.710+0000 7f0c1dc9c700 1 -- 172.21.15.37:0/18848554 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0c000a2cb0 msgr2=0x7f0c000a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:05.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.710+0000 7f0c1dc9c700 1 --2- 172.21.15.37:0/18848554 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0c000a2cb0 0x7f0c000a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0c18108bb0 tx=0x7f0c0c00b410 comp rx=0 tx=0).stop 2024-06-10T00:55:05.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.710+0000 7f0c1dc9c700 1 -- 172.21.15.37:0/18848554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c18102740 msgr2=0x7f0c18074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:05.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.710+0000 7f0c1dc9c700 1 --2- 172.21.15.37:0/18848554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c18102740 0x7f0c18074790 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0c080059d0 tx=0x7f0c080038b0 comp rx=0 tx=0).stop 2024-06-10T00:55:05.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.710+0000 7f0c1dc9c700 1 -- 172.21.15.37:0/18848554 shutdown_connections 2024-06-10T00:55:05.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.710+0000 7f0c1dc9c700 1 --2- 172.21.15.37:0/18848554 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0c000a2cb0 0x7f0c000a5170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:05.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.710+0000 7f0c1dc9c700 1 --2- 172.21.15.37:0/18848554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c18107840 0x7f0c18075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:05.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.710+0000 7f0c1dc9c700 1 --2- 172.21.15.37:0/18848554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c18102740 0x7f0c18074790 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:05.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.710+0000 7f0c1dc9c700 1 --2- 172.21.15.37:0/18848554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c181030f0 0x7f0c18074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:05.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.710+0000 7f0c1dc9c700 1 -- 172.21.15.37:0/18848554 >> 172.21.15.37:0/18848554 conn(0x7f0c180795f0 msgr2=0x7f0c18100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:05.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.710+0000 7f0c1dc9c700 1 -- 172.21.15.37:0/18848554 shutdown_connections 2024-06-10T00:55:05.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.710+0000 7f0c1dc9c700 1 -- 172.21.15.37:0/18848554 wait complete. 2024-06-10T00:55:05.724 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:05 smithi037 ceph-mon[32192]: pgmap v382: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T00:55:05.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.729+0000 7fbd50040700 1 -- 172.21.15.37:0/2234007562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd4806b560 msgr2=0x7fbd4810e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:05.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.729+0000 7fbd50040700 1 --2- 172.21.15.37:0/2234007562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd4806b560 0x7fbd4810e9a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbd38009990 tx=0x7fbd38009ca0 comp rx=0 tx=0).stop 2024-06-10T00:55:05.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.729+0000 7fbd50040700 1 -- 172.21.15.37:0/2234007562 shutdown_connections 2024-06-10T00:55:05.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.729+0000 7fbd50040700 1 --2- 172.21.15.37:0/2234007562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd4810eee0 0x7fbd481112d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:05.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.729+0000 7fbd50040700 1 --2- 172.21.15.37:0/2234007562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd4806ac40 0x7fbd4806b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:05.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.729+0000 7fbd50040700 1 --2- 172.21.15.37:0/2234007562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd4806b560 0x7fbd4810e9a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:05.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.729+0000 7fbd50040700 1 -- 172.21.15.37:0/2234007562 >> 172.21.15.37:0/2234007562 conn(0x7fbd48078430 msgr2=0x7fbd48078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:05.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.729+0000 7fbd50040700 1 -- 172.21.15.37:0/2234007562 shutdown_connections 2024-06-10T00:55:05.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.729+0000 7fbd50040700 1 -- 172.21.15.37:0/2234007562 wait complete. 2024-06-10T00:55:05.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.730+0000 7fbd50040700 1 Processor -- start 2024-06-10T00:55:05.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.730+0000 7fbd50040700 1 -- start start 2024-06-10T00:55:05.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.730+0000 7fbd50040700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd4806ac40 0x7fbd4819af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:05.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.730+0000 7fbd50040700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd4806b560 0x7fbd4819b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:05.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.730+0000 7fbd50040700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd4810eee0 0x7fbd4819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:05.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.730+0000 7fbd50040700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481149f0 con 0x7fbd4806b560 2024-06-10T00:55:05.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.730+0000 7fbd50040700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbd48114870 con 0x7fbd4806ac40 2024-06-10T00:55:05.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.730+0000 7fbd50040700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbd48114b70 con 0x7fbd4810eee0 2024-06-10T00:55:05.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.731+0000 7fbd4d5db700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd4806b560 0x7fbd4819b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:05.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.731+0000 7fbd4e5dd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd4810eee0 0x7fbd4819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:05.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.731+0000 7fbd4d5db700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd4806b560 0x7fbd4819b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43814/0 (socket says 172.21.15.37:43814) 2024-06-10T00:55:05.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.731+0000 7fbd4d5db700 1 -- 172.21.15.37:0/2490952236 learned_addr learned my addr 172.21.15.37:0/2490952236 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:05.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.731+0000 7fbd4dddc700 1 --2- 172.21.15.37:0/2490952236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd4806ac40 0x7fbd4819af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:05.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.731+0000 7fbd4dddc700 1 -- 172.21.15.37:0/2490952236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd4810eee0 msgr2=0x7fbd4819f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:05.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.731+0000 7fbd4dddc700 1 --2- 172.21.15.37:0/2490952236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd4810eee0 0x7fbd4819f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:05.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.731+0000 7fbd4dddc700 1 -- 172.21.15.37:0/2490952236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd4806b560 msgr2=0x7fbd4819b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:05.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.731+0000 7fbd4dddc700 1 --2- 172.21.15.37:0/2490952236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd4806b560 0x7fbd4819b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:05.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.731+0000 7fbd4dddc700 1 -- 172.21.15.37:0/2490952236 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4819ff30 con 0x7fbd4806ac40 2024-06-10T00:55:05.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.731+0000 7fbd4e5dd700 1 --2- 172.21.15.37:0/2490952236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd4810eee0 0x7fbd4819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:55:05.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.731+0000 7fbd4d5db700 1 --2- 172.21.15.37:0/2490952236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd4806b560 0x7fbd4819b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:55:05.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.731+0000 7fbd4dddc700 1 --2- 172.21.15.37:0/2490952236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd4806ac40 0x7fbd4819af20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400d950 tx=0x7fbd4400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:05.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.732+0000 7fbd3effd700 1 -- 172.21.15.37:0/2490952236 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd44016070 con 0x7fbd4806ac40 2024-06-10T00:55:05.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.732+0000 7fbd3effd700 1 -- 172.21.15.37:0/2490952236 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd4400fd70 con 0x7fbd4806ac40 2024-06-10T00:55:05.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.732+0000 7fbd3effd700 1 -- 172.21.15.37:0/2490952236 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4400e3b0 con 0x7fbd4806ac40 2024-06-10T00:55:05.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.732+0000 7fbd50040700 1 -- 172.21.15.37:0/2490952236 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd481a0220 con 0x7fbd4806ac40 2024-06-10T00:55:05.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.732+0000 7fbd50040700 1 -- 172.21.15.37:0/2490952236 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd481a7aa0 con 0x7fbd4806ac40 2024-06-10T00:55:05.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.733+0000 7fbd50040700 1 -- 172.21.15.37:0/2490952236 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd48069180 con 0x7fbd4806ac40 2024-06-10T00:55:05.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.735+0000 7fbd3effd700 1 -- 172.21.15.37:0/2490952236 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbd44019070 con 0x7fbd4806ac40 2024-06-10T00:55:05.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.735+0000 7fbd3effd700 1 --2- 172.21.15.37:0/2490952236 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fbd340a2cb0 0x7fbd340a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:05.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.736+0000 7fbd3effd700 1 -- 172.21.15.37:0/2490952236 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fbd440cf250 con 0x7fbd4806ac40 2024-06-10T00:55:05.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.736+0000 7fbd4d5db700 1 --2- 172.21.15.37:0/2490952236 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fbd340a2cb0 0x7fbd340a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:05.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.736+0000 7fbd4d5db700 1 --2- 172.21.15.37:0/2490952236 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fbd340a2cb0 0x7fbd340a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbd481028f0 tx=0x7fbd38005c00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:05.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.739+0000 7fbd3effd700 1 -- 172.21.15.37:0/2490952236 <== mon.2 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbd44020030 con 0x7fbd4806ac40 2024-06-10T00:55:05.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:05 smithi167 ceph-mon[36917]: pgmap v382: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T00:55:05.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:05 smithi094 ceph-mon[36708]: pgmap v382: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T00:55:05.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.947+0000 7fbd50040700 1 -- 172.21.15.37:0/2490952236 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fbd4819c2b0 con 0x7fbd4806ac40 2024-06-10T00:55:05.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.948+0000 7fbd3effd700 1 -- 172.21.15.37:0/2490952236 <== mon.2 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbd4819c2b0 con 0x7fbd4806ac40 2024-06-10T00:55:05.949 INFO:teuthology.orchestra.run.smithi037.stdout:111669149782 2024-06-10T00:55:05.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.950+0000 7fbd50040700 1 -- 172.21.15.37:0/2490952236 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fbd340a2cb0 msgr2=0x7fbd340a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:05.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.950+0000 7fbd50040700 1 --2- 172.21.15.37:0/2490952236 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fbd340a2cb0 0x7fbd340a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbd481028f0 tx=0x7fbd38005c00 comp rx=0 tx=0).stop 2024-06-10T00:55:05.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.950+0000 7fbd50040700 1 -- 172.21.15.37:0/2490952236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd4806ac40 msgr2=0x7fbd4819af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:05.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.950+0000 7fbd50040700 1 --2- 172.21.15.37:0/2490952236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd4806ac40 0x7fbd4819af20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400d950 tx=0x7fbd4400dc60 comp rx=0 tx=0).stop 2024-06-10T00:55:05.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.950+0000 7fbd50040700 1 -- 172.21.15.37:0/2490952236 shutdown_connections 2024-06-10T00:55:05.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.950+0000 7fbd50040700 1 --2- 172.21.15.37:0/2490952236 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fbd340a2cb0 0x7fbd340a5170 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:05.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.950+0000 7fbd50040700 1 --2- 172.21.15.37:0/2490952236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd4810eee0 0x7fbd4819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:05.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.950+0000 7fbd50040700 1 --2- 172.21.15.37:0/2490952236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd4806ac40 0x7fbd4819af20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:05.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.950+0000 7fbd50040700 1 --2- 172.21.15.37:0/2490952236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd4806b560 0x7fbd4819b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:05.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.950+0000 7fbd50040700 1 -- 172.21.15.37:0/2490952236 >> 172.21.15.37:0/2490952236 conn(0x7fbd48078430 msgr2=0x7fbd48110370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:05.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.950+0000 7fbd50040700 1 -- 172.21.15.37:0/2490952236 shutdown_connections 2024-06-10T00:55:05.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:05.950+0000 7fbd50040700 1 -- 172.21.15.37:0/2490952236 wait complete. 2024-06-10T00:55:06.766 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:06 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/18848554' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-10T00:55:06.767 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:06 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2490952236' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-10T00:55:06.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:06 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/18848554' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-10T00:55:06.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:06 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2490952236' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-10T00:55:06.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:06 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/18848554' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-10T00:55:06.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:06 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2490952236' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-10T00:55:07.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.119+0000 7f5e910d0700 1 -- 172.21.15.37:0/2719383263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e8c108500 msgr2=0x7f5e8c1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:07.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.119+0000 7f5e910d0700 1 --2- 172.21.15.37:0/2719383263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e8c108500 0x7f5e8c1088e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00b0a0 tx=0x7f5e7c00b3b0 comp rx=0 tx=0).stop 2024-06-10T00:55:07.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.119+0000 7f5e910d0700 1 -- 172.21.15.37:0/2719383263 shutdown_connections 2024-06-10T00:55:07.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.119+0000 7f5e910d0700 1 --2- 172.21.15.37:0/2719383263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e8c06ab30 0x7f5e8c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:07.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.119+0000 7f5e910d0700 1 --2- 172.21.15.37:0/2719383263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e8c108500 0x7f5e8c1088e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:07.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.119+0000 7f5e910d0700 1 --2- 172.21.15.37:0/2719383263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e8c06a170 0x7f5e8c06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:07.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.119+0000 7f5e910d0700 1 -- 172.21.15.37:0/2719383263 >> 172.21.15.37:0/2719383263 conn(0x7f5e8c078340 msgr2=0x7f5e8c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:07.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.119+0000 7f5e910d0700 1 -- 172.21.15.37:0/2719383263 shutdown_connections 2024-06-10T00:55:07.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.119+0000 7f5e910d0700 1 -- 172.21.15.37:0/2719383263 wait complete. 2024-06-10T00:55:07.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.120+0000 7f5e910d0700 1 Processor -- start 2024-06-10T00:55:07.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.120+0000 7f5e910d0700 1 -- start start 2024-06-10T00:55:07.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.120+0000 7f5e910d0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e8c06a170 0x7f5e8c19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:07.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.120+0000 7f5e910d0700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e8c06ab30 0x7f5e8c19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:07.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.120+0000 7f5e910d0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e8c108500 0x7f5e8c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:07.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.120+0000 7f5e910d0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c112760 con 0x7f5e8c06a170 2024-06-10T00:55:07.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.120+0000 7f5e910d0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c1125e0 con 0x7f5e8c108500 2024-06-10T00:55:07.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.120+0000 7f5e910d0700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c1128e0 con 0x7f5e8c06ab30 2024-06-10T00:55:07.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.120+0000 7f5e8ad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e8c06a170 0x7f5e8c19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:07.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.120+0000 7f5e8ad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e8c06a170 0x7f5e8c19e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43838/0 (socket says 172.21.15.37:43838) 2024-06-10T00:55:07.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.120+0000 7f5e8ad9d700 1 -- 172.21.15.37:0/2313406497 learned_addr learned my addr 172.21.15.37:0/2313406497 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:07.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.120+0000 7f5e8b59e700 1 --2- 172.21.15.37:0/2313406497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e8c108500 0x7f5e8c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:07.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.121+0000 7f5e8a59c700 1 --2- 172.21.15.37:0/2313406497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e8c06ab30 0x7f5e8c19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:07.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.121+0000 7f5e8ad9d700 1 -- 172.21.15.37:0/2313406497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e8c06ab30 msgr2=0x7f5e8c19edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:07.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.121+0000 7f5e8ad9d700 1 --2- 172.21.15.37:0/2313406497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e8c06ab30 0x7f5e8c19edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:07.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.121+0000 7f5e8ad9d700 1 -- 172.21.15.37:0/2313406497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e8c108500 msgr2=0x7f5e8c198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:07.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.121+0000 7f5e8ad9d700 1 --2- 172.21.15.37:0/2313406497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e8c108500 0x7f5e8c198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:07.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.121+0000 7f5e8ad9d700 1 -- 172.21.15.37:0/2313406497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8c199200 con 0x7f5e8c06a170 2024-06-10T00:55:07.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.121+0000 7f5e8ad9d700 1 --2- 172.21.15.37:0/2313406497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e8c06a170 0x7f5e8c19e890 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c000f80 tx=0x7f5e7c00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:07.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.121+0000 7f5e73fff700 1 -- 172.21.15.37:0/2313406497 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c021070 con 0x7f5e8c06a170 2024-06-10T00:55:07.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.121+0000 7f5e73fff700 1 -- 172.21.15.37:0/2313406497 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e7c007d00 con 0x7f5e8c06a170 2024-06-10T00:55:07.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.121+0000 7f5e910d0700 1 -- 172.21.15.37:0/2313406497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e8c199490 con 0x7f5e8c06a170 2024-06-10T00:55:07.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.121+0000 7f5e73fff700 1 -- 172.21.15.37:0/2313406497 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c005500 con 0x7f5e8c06a170 2024-06-10T00:55:07.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.121+0000 7f5e910d0700 1 -- 172.21.15.37:0/2313406497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e8c1a5760 con 0x7f5e8c06a170 2024-06-10T00:55:07.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.122+0000 7f5e910d0700 1 -- 172.21.15.37:0/2313406497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e8c10a9e0 con 0x7f5e8c06a170 2024-06-10T00:55:07.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.124+0000 7f5e73fff700 1 -- 172.21.15.37:0/2313406497 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5e7c024070 con 0x7f5e8c06a170 2024-06-10T00:55:07.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.124+0000 7f5e73fff700 1 --2- 172.21.15.37:0/2313406497 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5e740a2cb0 0x7f5e740a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:07.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.124+0000 7f5e8a59c700 1 --2- 172.21.15.37:0/2313406497 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5e740a2cb0 0x7f5e740a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:07.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.124+0000 7f5e73fff700 1 -- 172.21.15.37:0/2313406497 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f5e7c00f020 con 0x7f5e8c06a170 2024-06-10T00:55:07.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.125+0000 7f5e8a59c700 1 --2- 172.21.15.37:0/2313406497 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5e740a2cb0 0x7f5e740a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5e80005ea0 tx=0x7f5e80005e30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:07.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.129+0000 7f5e73fff700 1 -- 172.21.15.37:0/2313406497 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5e7c02b020 con 0x7f5e8c06a170 2024-06-10T00:55:07.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.338+0000 7f5e910d0700 1 -- 172.21.15.37:0/2313406497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f5e8c199f80 con 0x7f5e8c06a170 2024-06-10T00:55:07.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.339+0000 7f5e73fff700 1 -- 172.21.15.37:0/2313406497 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5e7c09d790 con 0x7f5e8c06a170 2024-06-10T00:55:07.339 INFO:teuthology.orchestra.run.smithi037.stdout:77309411420 2024-06-10T00:55:07.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.341+0000 7f5e910d0700 1 -- 172.21.15.37:0/2313406497 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5e740a2cb0 msgr2=0x7f5e740a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:07.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.341+0000 7f5e910d0700 1 --2- 172.21.15.37:0/2313406497 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5e740a2cb0 0x7f5e740a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5e80005ea0 tx=0x7f5e80005e30 comp rx=0 tx=0).stop 2024-06-10T00:55:07.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.341+0000 7f5e910d0700 1 -- 172.21.15.37:0/2313406497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e8c06a170 msgr2=0x7f5e8c19e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:07.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.341+0000 7f5e910d0700 1 --2- 172.21.15.37:0/2313406497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e8c06a170 0x7f5e8c19e890 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c000f80 tx=0x7f5e7c00be60 comp rx=0 tx=0).stop 2024-06-10T00:55:07.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.341+0000 7f5e910d0700 1 -- 172.21.15.37:0/2313406497 shutdown_connections 2024-06-10T00:55:07.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.341+0000 7f5e910d0700 1 --2- 172.21.15.37:0/2313406497 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5e740a2cb0 0x7f5e740a5170 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:07.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.341+0000 7f5e910d0700 1 --2- 172.21.15.37:0/2313406497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e8c108500 0x7f5e8c198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:07.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.341+0000 7f5e910d0700 1 --2- 172.21.15.37:0/2313406497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e8c06a170 0x7f5e8c19e890 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:07.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.341+0000 7f5e910d0700 1 --2- 172.21.15.37:0/2313406497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e8c06ab30 0x7f5e8c19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:07.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.341+0000 7f5e910d0700 1 -- 172.21.15.37:0/2313406497 >> 172.21.15.37:0/2313406497 conn(0x7f5e8c078340 msgr2=0x7f5e8c0fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:07.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.341+0000 7f5e910d0700 1 -- 172.21.15.37:0/2313406497 shutdown_connections 2024-06-10T00:55:07.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:07.341+0000 7f5e910d0700 1 -- 172.21.15.37:0/2313406497 wait complete. 2024-06-10T00:55:07.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:07 smithi037 ceph-mon[32192]: pgmap v383: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:55:07.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:07 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2313406497' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-10T00:55:07.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:07 smithi167 ceph-mon[36917]: pgmap v383: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:55:07.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:07 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2313406497' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-10T00:55:07.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:07 smithi094 ceph-mon[36708]: pgmap v383: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:55:07.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:07 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2313406497' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-10T00:55:08.640 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037814 got 257698037820 for osd.8 2024-06-10T00:55:08.640 DEBUG:teuthology.parallel:result is None 2024-06-10T00:55:08.662 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:55:09.138 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612655 got 313532612659 for osd.10 2024-06-10T00:55:09.138 DEBUG:teuthology.parallel:result is None 2024-06-10T00:55:09.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:09 smithi037 ceph-mon[32192]: pgmap v384: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:55:09.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:09 smithi167 ceph-mon[36917]: pgmap v384: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:55:09.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:09 smithi094 ceph-mon[36708]: pgmap v384: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:55:09.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.971+0000 7f32e1743700 1 -- 172.21.15.37:0/2250386112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32dc101440 msgr2=0x7f32dc10f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:09.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.971+0000 7f32e1743700 1 --2- 172.21.15.37:0/2250386112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32dc101440 0x7f32dc10f330 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f32d400b0a0 tx=0x7f32d400b3b0 comp rx=0 tx=0).stop 2024-06-10T00:55:09.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.971+0000 7f32e1743700 1 -- 172.21.15.37:0/2250386112 shutdown_connections 2024-06-10T00:55:09.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.971+0000 7f32e1743700 1 --2- 172.21.15.37:0/2250386112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32dc101440 0x7f32dc10f330 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:09.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.971+0000 7f32e1743700 1 --2- 172.21.15.37:0/2250386112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32dc108510 0x7f32dc1088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:09.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.971+0000 7f32e1743700 1 --2- 172.21.15.37:0/2250386112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32dc100a80 0x7f32dc100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:09.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.971+0000 7f32e1743700 1 -- 172.21.15.37:0/2250386112 >> 172.21.15.37:0/2250386112 conn(0x7f32dc077a00 msgr2=0x7f32dc0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:09.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.971+0000 7f32e1743700 1 -- 172.21.15.37:0/2250386112 shutdown_connections 2024-06-10T00:55:09.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.971+0000 7f32e1743700 1 -- 172.21.15.37:0/2250386112 wait complete. 2024-06-10T00:55:09.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.972+0000 7f32e1743700 1 Processor -- start 2024-06-10T00:55:09.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.972+0000 7f32e1743700 1 -- start start 2024-06-10T00:55:09.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.972+0000 7f32e1743700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32dc100a80 0x7f32dc19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:09.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.972+0000 7f32e1743700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32dc101440 0x7f32dc19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:09.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.972+0000 7f32e1743700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32dc108510 0x7f32dc1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:09.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.972+0000 7f32e1743700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc112790 con 0x7f32dc101440 2024-06-10T00:55:09.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.972+0000 7f32e1743700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc112610 con 0x7f32dc100a80 2024-06-10T00:55:09.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.972+0000 7f32e1743700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc112910 con 0x7f32dc108510 2024-06-10T00:55:09.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.973+0000 7f32da7fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32dc101440 0x7f32dc19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:09.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.973+0000 7f32daffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32dc100a80 0x7f32dc19e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:09.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.973+0000 7f32da7fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32dc101440 0x7f32dc19ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43850/0 (socket says 172.21.15.37:43850) 2024-06-10T00:55:09.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.973+0000 7f32da7fc700 1 -- 172.21.15.37:0/20280571 learned_addr learned my addr 172.21.15.37:0/20280571 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:09.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.973+0000 7f32db7fe700 1 --2- 172.21.15.37:0/20280571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32dc108510 0x7f32dc1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:09.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.973+0000 7f32daffd700 1 -- 172.21.15.37:0/20280571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32dc108510 msgr2=0x7f32dc1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:09.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.973+0000 7f32daffd700 1 --2- 172.21.15.37:0/20280571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32dc108510 0x7f32dc1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:09.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.973+0000 7f32daffd700 1 -- 172.21.15.37:0/20280571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32dc101440 msgr2=0x7f32dc19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:09.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.973+0000 7f32daffd700 1 --2- 172.21.15.37:0/20280571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32dc101440 0x7f32dc19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:09.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.973+0000 7f32daffd700 1 -- 172.21.15.37:0/20280571 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32dc199110 con 0x7f32dc100a80 2024-06-10T00:55:09.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.973+0000 7f32daffd700 1 --2- 172.21.15.37:0/20280571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32dc100a80 0x7f32dc19e7f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f32cc00d750 tx=0x7f32cc00da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:09.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.973+0000 7f32c3fff700 1 -- 172.21.15.37:0/20280571 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc017070 con 0x7f32dc100a80 2024-06-10T00:55:09.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.973+0000 7f32e1743700 1 -- 172.21.15.37:0/20280571 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32dc199400 con 0x7f32dc100a80 2024-06-10T00:55:09.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.973+0000 7f32c3fff700 1 -- 172.21.15.37:0/20280571 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32cc005020 con 0x7f32dc100a80 2024-06-10T00:55:09.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.973+0000 7f32c3fff700 1 -- 172.21.15.37:0/20280571 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc00fb70 con 0x7f32dc100a80 2024-06-10T00:55:09.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.974+0000 7f32e1743700 1 -- 172.21.15.37:0/20280571 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32dc1a56c0 con 0x7f32dc100a80 2024-06-10T00:55:09.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.974+0000 7f32e1743700 1 -- 172.21.15.37:0/20280571 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32dc069180 con 0x7f32dc100a80 2024-06-10T00:55:09.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.976+0000 7f32c3fff700 1 -- 172.21.15.37:0/20280571 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f32cc0051d0 con 0x7f32dc100a80 2024-06-10T00:55:09.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.977+0000 7f32c3fff700 1 --2- 172.21.15.37:0/20280571 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f32c40a2cb0 0x7f32c40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:09.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.977+0000 7f32da7fc700 1 --2- 172.21.15.37:0/20280571 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f32c40a2cb0 0x7f32c40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:09.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.977+0000 7f32c3fff700 1 -- 172.21.15.37:0/20280571 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f32cc0cf100 con 0x7f32dc100a80 2024-06-10T00:55:09.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.977+0000 7f32da7fc700 1 --2- 172.21.15.37:0/20280571 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f32c40a2cb0 0x7f32c40a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009680 tx=0x7f32d0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:09.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:09.981+0000 7f32c3fff700 1 -- 172.21.15.37:0/20280571 <== mon.2 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f32cc09bab0 con 0x7f32dc100a80 2024-06-10T00:55:10.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:10.191+0000 7f32e1743700 1 -- 172.21.15.37:0/20280571 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f32dc199ee0 con 0x7f32dc100a80 2024-06-10T00:55:10.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:10.191+0000 7f32c3fff700 1 -- 172.21.15.37:0/20280571 <== mon.2 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f32cc09b3b0 con 0x7f32dc100a80 2024-06-10T00:55:10.192 INFO:teuthology.orchestra.run.smithi037.stdout:34359738470 2024-06-10T00:55:10.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:10.193+0000 7f32e1743700 1 -- 172.21.15.37:0/20280571 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f32c40a2cb0 msgr2=0x7f32c40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:10.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:10.193+0000 7f32e1743700 1 --2- 172.21.15.37:0/20280571 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f32c40a2cb0 0x7f32c40a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009680 tx=0x7f32d0009260 comp rx=0 tx=0).stop 2024-06-10T00:55:10.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:10.193+0000 7f32e1743700 1 -- 172.21.15.37:0/20280571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32dc100a80 msgr2=0x7f32dc19e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:10.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:10.193+0000 7f32e1743700 1 --2- 172.21.15.37:0/20280571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32dc100a80 0x7f32dc19e7f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f32cc00d750 tx=0x7f32cc00da60 comp rx=0 tx=0).stop 2024-06-10T00:55:10.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:10.193+0000 7f32e1743700 1 -- 172.21.15.37:0/20280571 shutdown_connections 2024-06-10T00:55:10.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:10.193+0000 7f32e1743700 1 --2- 172.21.15.37:0/20280571 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f32c40a2cb0 0x7f32c40a5170 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:10.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:10.193+0000 7f32e1743700 1 --2- 172.21.15.37:0/20280571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32dc108510 0x7f32dc1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:10.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:10.193+0000 7f32e1743700 1 --2- 172.21.15.37:0/20280571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32dc100a80 0x7f32dc19e7f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:10.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:10.193+0000 7f32e1743700 1 --2- 172.21.15.37:0/20280571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32dc101440 0x7f32dc19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:10.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:10.193+0000 7f32e1743700 1 -- 172.21.15.37:0/20280571 >> 172.21.15.37:0/20280571 conn(0x7f32dc077a00 msgr2=0x7f32dc0ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:10.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:10.194+0000 7f32e1743700 1 -- 172.21.15.37:0/20280571 shutdown_connections 2024-06-10T00:55:10.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:10.194+0000 7f32e1743700 1 -- 172.21.15.37:0/20280571 wait complete. 2024-06-10T00:55:10.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:10 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/20280571' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-10T00:55:10.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:10 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/20280571' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-10T00:55:10.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:10 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/20280571' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-10T00:55:10.899 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149776 got 111669149782 for osd.3 2024-06-10T00:55:10.899 DEBUG:teuthology.parallel:result is None 2024-06-10T00:55:11.662 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:11 smithi037 ceph-mon[32192]: pgmap v385: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:55:11.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:11 smithi167 ceph-mon[36917]: pgmap v385: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:55:11.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:11 smithi094 ceph-mon[36708]: pgmap v385: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:55:11.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.979+0000 7f70b59da700 1 -- 172.21.15.37:0/1807150266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70b0107570 msgr2=0x7f70b0107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:11.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.979+0000 7f70b59da700 1 --2- 172.21.15.37:0/1807150266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70b0107570 0x7f70b0107950 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f70a0009990 tx=0x7f70a0009ca0 comp rx=0 tx=0).stop 2024-06-10T00:55:11.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.979+0000 7f70b59da700 1 -- 172.21.15.37:0/1807150266 shutdown_connections 2024-06-10T00:55:11.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.979+0000 7f70b59da700 1 --2- 172.21.15.37:0/1807150266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70b0101810 0x7f70b010f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:11.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.979+0000 7f70b59da700 1 --2- 172.21.15.37:0/1807150266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70b0107570 0x7f70b0107950 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:11.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.979+0000 7f70b59da700 1 --2- 172.21.15.37:0/1807150266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70b0100e50 0x7f70b01012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:11.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.979+0000 7f70b59da700 1 -- 172.21.15.37:0/1807150266 >> 172.21.15.37:0/1807150266 conn(0x7f70b00766b0 msgr2=0x7f70b0076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:11.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.979+0000 7f70b59da700 1 -- 172.21.15.37:0/1807150266 shutdown_connections 2024-06-10T00:55:11.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.979+0000 7f70b59da700 1 -- 172.21.15.37:0/1807150266 wait complete. 2024-06-10T00:55:11.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.980+0000 7f70b59da700 1 Processor -- start 2024-06-10T00:55:11.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.980+0000 7f70b59da700 1 -- start start 2024-06-10T00:55:11.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.980+0000 7f70b59da700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70b0100e50 0x7f70b019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:11.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.980+0000 7f70b59da700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70b0101810 0x7f70b019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:11.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.980+0000 7f70b59da700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70b0107570 0x7f70b01989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:11.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.980+0000 7f70b59da700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f70b01127b0 con 0x7f70b0100e50 2024-06-10T00:55:11.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.980+0000 7f70b59da700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f70b0112630 con 0x7f70b0107570 2024-06-10T00:55:11.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.980+0000 7f70b59da700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f70b0112930 con 0x7f70b0101810 2024-06-10T00:55:11.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.981+0000 7f70aeffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70b0100e50 0x7f70b019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:11.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.981+0000 7f70aeffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70b0100e50 0x7f70b019e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41430/0 (socket says 172.21.15.37:41430) 2024-06-10T00:55:11.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.981+0000 7f70aeffd700 1 -- 172.21.15.37:0/3712900517 learned_addr learned my addr 172.21.15.37:0/3712900517 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:11.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.981+0000 7f70af7fe700 1 --2- 172.21.15.37:0/3712900517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70b0107570 0x7f70b01989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:11.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.981+0000 7f70a67fc700 1 --2- 172.21.15.37:0/3712900517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70b0101810 0x7f70b019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:11.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.981+0000 7f70aeffd700 1 -- 172.21.15.37:0/3712900517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70b0101810 msgr2=0x7f70b019ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:11.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.981+0000 7f70aeffd700 1 --2- 172.21.15.37:0/3712900517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70b0101810 0x7f70b019ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:11.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.981+0000 7f70aeffd700 1 -- 172.21.15.37:0/3712900517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70b0107570 msgr2=0x7f70b01989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:11.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.981+0000 7f70aeffd700 1 --2- 172.21.15.37:0/3712900517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70b0107570 0x7f70b01989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:11.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.981+0000 7f70aeffd700 1 -- 172.21.15.37:0/3712900517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b0199150 con 0x7f70b0100e50 2024-06-10T00:55:11.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.981+0000 7f70aeffd700 1 --2- 172.21.15.37:0/3712900517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70b0100e50 0x7f70b019e7e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f70a0000c00 tx=0x7f70a00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:11.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.981+0000 7f70b49d8700 1 -- 172.21.15.37:0/3712900517 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a0027070 con 0x7f70b0100e50 2024-06-10T00:55:11.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.981+0000 7f70b49d8700 1 -- 172.21.15.37:0/3712900517 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a0003e40 con 0x7f70b0100e50 2024-06-10T00:55:11.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.981+0000 7f70b59da700 1 -- 172.21.15.37:0/3712900517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70b01993e0 con 0x7f70b0100e50 2024-06-10T00:55:11.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.981+0000 7f70b49d8700 1 -- 172.21.15.37:0/3712900517 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a0022ce0 con 0x7f70b0100e50 2024-06-10T00:55:11.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.981+0000 7f70b59da700 1 -- 172.21.15.37:0/3712900517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70b01a56b0 con 0x7f70b0100e50 2024-06-10T00:55:11.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.982+0000 7f70b59da700 1 -- 172.21.15.37:0/3712900517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70b006a660 con 0x7f70b0100e50 2024-06-10T00:55:11.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.984+0000 7f70b49d8700 1 -- 172.21.15.37:0/3712900517 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f70a002a070 con 0x7f70b0100e50 2024-06-10T00:55:11.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.984+0000 7f70b49d8700 1 --2- 172.21.15.37:0/3712900517 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f709c0a2cb0 0x7f709c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:11.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.984+0000 7f70a67fc700 1 --2- 172.21.15.37:0/3712900517 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f709c0a2cb0 0x7f709c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:11.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.984+0000 7f70b49d8700 1 -- 172.21.15.37:0/3712900517 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f70a00d0aa0 con 0x7f70b0100e50 2024-06-10T00:55:11.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.985+0000 7f70a67fc700 1 --2- 172.21.15.37:0/3712900517 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f709c0a2cb0 0x7f709c0a5170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f70b0076b40 tx=0x7f709800a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:11.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:11.988+0000 7f70b49d8700 1 -- 172.21.15.37:0/3712900517 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f70a0031070 con 0x7f70b0100e50 2024-06-10T00:55:12.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.199+0000 7f70b59da700 1 -- 172.21.15.37:0/3712900517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f70b0069180 con 0x7f70b0100e50 2024-06-10T00:55:12.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.200+0000 7f70b49d8700 1 -- 172.21.15.37:0/3712900517 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f70b0069180 con 0x7f70b0100e50 2024-06-10T00:55:12.200 INFO:teuthology.orchestra.run.smithi037.stdout:292057776185 2024-06-10T00:55:12.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.202+0000 7f70b59da700 1 -- 172.21.15.37:0/3712900517 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f709c0a2cb0 msgr2=0x7f709c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:12.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.202+0000 7f70b59da700 1 --2- 172.21.15.37:0/3712900517 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f709c0a2cb0 0x7f709c0a5170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f70b0076b40 tx=0x7f709800a380 comp rx=0 tx=0).stop 2024-06-10T00:55:12.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.202+0000 7f70b59da700 1 -- 172.21.15.37:0/3712900517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70b0100e50 msgr2=0x7f70b019e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:12.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.202+0000 7f70b59da700 1 --2- 172.21.15.37:0/3712900517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70b0100e50 0x7f70b019e7e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f70a0000c00 tx=0x7f70a00047b0 comp rx=0 tx=0).stop 2024-06-10T00:55:12.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.202+0000 7f70b59da700 1 -- 172.21.15.37:0/3712900517 shutdown_connections 2024-06-10T00:55:12.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.202+0000 7f70b59da700 1 --2- 172.21.15.37:0/3712900517 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f709c0a2cb0 0x7f709c0a5170 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:12.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.202+0000 7f70b59da700 1 --2- 172.21.15.37:0/3712900517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70b0107570 0x7f70b01989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:12.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.202+0000 7f70b59da700 1 --2- 172.21.15.37:0/3712900517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70b0100e50 0x7f70b019e7e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:12.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.202+0000 7f70b59da700 1 --2- 172.21.15.37:0/3712900517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70b0101810 0x7f70b019ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:12.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.202+0000 7f70b59da700 1 -- 172.21.15.37:0/3712900517 >> 172.21.15.37:0/3712900517 conn(0x7f70b00766b0 msgr2=0x7f70b00ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:12.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.202+0000 7f70b59da700 1 -- 172.21.15.37:0/3712900517 shutdown_connections 2024-06-10T00:55:12.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.202+0000 7f70b59da700 1 -- 172.21.15.37:0/3712900517 wait complete. 2024-06-10T00:55:12.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.215+0000 7f71c4148700 1 -- 172.21.15.37:0/485776201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71bc102ef0 msgr2=0x7f71bc10f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:12.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.215+0000 7f71c4148700 1 --2- 172.21.15.37:0/485776201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71bc102ef0 0x7f71bc10f3a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f71b400b0d0 tx=0x7f71b400b3e0 comp rx=0 tx=0).stop 2024-06-10T00:55:12.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.215+0000 7f71c4148700 1 -- 172.21.15.37:0/485776201 shutdown_connections 2024-06-10T00:55:12.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.215+0000 7f71c4148700 1 --2- 172.21.15.37:0/485776201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71bc102ef0 0x7f71bc10f3a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:12.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.215+0000 7f71c4148700 1 --2- 172.21.15.37:0/485776201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71bc1085d0 0x7f71bc1089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:12.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.215+0000 7f71c4148700 1 --2- 172.21.15.37:0/485776201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71bc102530 0x7f71bc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:12.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.215+0000 7f71c4148700 1 -- 172.21.15.37:0/485776201 >> 172.21.15.37:0/485776201 conn(0x7f71bc079640 msgr2=0x7f71bc079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:12.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.215+0000 7f71c4148700 1 -- 172.21.15.37:0/485776201 shutdown_connections 2024-06-10T00:55:12.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.215+0000 7f71c4148700 1 -- 172.21.15.37:0/485776201 wait complete. 2024-06-10T00:55:12.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.216+0000 7f71c4148700 1 Processor -- start 2024-06-10T00:55:12.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.216+0000 7f71c4148700 1 -- start start 2024-06-10T00:55:12.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.216+0000 7f71c4148700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71bc102530 0x7f71bc19ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:12.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.216+0000 7f71c4148700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71bc102ef0 0x7f71bc19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:12.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.216+0000 7f71c4148700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71bc1085d0 0x7f71bc198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:12.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.216+0000 7f71c4148700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc112860 con 0x7f71bc102530 2024-06-10T00:55:12.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.216+0000 7f71c4148700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc1126e0 con 0x7f71bc1085d0 2024-06-10T00:55:12.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.216+0000 7f71c4148700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc1129e0 con 0x7f71bc102ef0 2024-06-10T00:55:12.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.216+0000 7f71c1ee4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71bc102530 0x7f71bc19ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:12.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.217+0000 7f71c1ee4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71bc102530 0x7f71bc19ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41458/0 (socket says 172.21.15.37:41458) 2024-06-10T00:55:12.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.217+0000 7f71c1ee4700 1 -- 172.21.15.37:0/882068626 learned_addr learned my addr 172.21.15.37:0/882068626 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:12.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.217+0000 7f71c16e3700 1 --2- 172.21.15.37:0/882068626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71bc102ef0 0x7f71bc19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:12.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.217+0000 7f71c26e5700 1 --2- 172.21.15.37:0/882068626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71bc1085d0 0x7f71bc198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:12.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.217+0000 7f71c1ee4700 1 -- 172.21.15.37:0/882068626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71bc102ef0 msgr2=0x7f71bc19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:12.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.217+0000 7f71c1ee4700 1 --2- 172.21.15.37:0/882068626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71bc102ef0 0x7f71bc19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:12.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.217+0000 7f71c1ee4700 1 -- 172.21.15.37:0/882068626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71bc1085d0 msgr2=0x7f71bc198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:12.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.217+0000 7f71c1ee4700 1 --2- 172.21.15.37:0/882068626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71bc1085d0 0x7f71bc198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:12.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.217+0000 7f71c1ee4700 1 -- 172.21.15.37:0/882068626 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71bc199440 con 0x7f71bc102530 2024-06-10T00:55:12.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.217+0000 7f71c26e5700 1 --2- 172.21.15.37:0/882068626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71bc1085d0 0x7f71bc198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:55:12.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.217+0000 7f71c1ee4700 1 --2- 172.21.15.37:0/882068626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71bc102530 0x7f71bc19ea40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f71b800d950 tx=0x7f71b800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:12.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.217+0000 7f71b2ffd700 1 -- 172.21.15.37:0/882068626 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8016070 con 0x7f71bc102530 2024-06-10T00:55:12.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.217+0000 7f71b2ffd700 1 -- 172.21.15.37:0/882068626 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71b8009e20 con 0x7f71bc102530 2024-06-10T00:55:12.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.217+0000 7f71b2ffd700 1 -- 172.21.15.37:0/882068626 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b800f620 con 0x7f71bc102530 2024-06-10T00:55:12.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.217+0000 7f71c4148700 1 -- 172.21.15.37:0/882068626 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71bc199730 con 0x7f71bc102530 2024-06-10T00:55:12.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.217+0000 7f71c4148700 1 -- 172.21.15.37:0/882068626 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71bc1a59c0 con 0x7f71bc102530 2024-06-10T00:55:12.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.219+0000 7f71c4148700 1 -- 172.21.15.37:0/882068626 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71bc103c40 con 0x7f71bc102530 2024-06-10T00:55:12.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.221+0000 7f71b2ffd700 1 -- 172.21.15.37:0/882068626 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f71b8019070 con 0x7f71bc102530 2024-06-10T00:55:12.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.222+0000 7f71b2ffd700 1 --2- 172.21.15.37:0/882068626 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f71a80a2cb0 0x7f71a80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:12.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.222+0000 7f71b2ffd700 1 -- 172.21.15.37:0/882068626 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f71b80cf120 con 0x7f71bc102530 2024-06-10T00:55:12.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.222+0000 7f71c16e3700 1 --2- 172.21.15.37:0/882068626 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f71a80a2cb0 0x7f71a80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:12.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.222+0000 7f71c16e3700 1 --2- 172.21.15.37:0/882068626 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f71a80a2cb0 0x7f71a80a5170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f71bc19a260 tx=0x7f71ac009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:12.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.226+0000 7f71b2ffd700 1 -- 172.21.15.37:0/882068626 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f71b809bb30 con 0x7f71bc102530 2024-06-10T00:55:12.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.436+0000 7f71c4148700 1 -- 172.21.15.37:0/882068626 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f71bc199fd0 con 0x7f71bc102530 2024-06-10T00:55:12.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.436+0000 7f71b2ffd700 1 -- 172.21.15.37:0/882068626 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f71b809b430 con 0x7f71bc102530 2024-06-10T00:55:12.437 INFO:teuthology.orchestra.run.smithi037.stdout:55834574946 2024-06-10T00:55:12.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.439+0000 7f71c4148700 1 -- 172.21.15.37:0/882068626 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f71a80a2cb0 msgr2=0x7f71a80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:12.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.439+0000 7f71c4148700 1 --2- 172.21.15.37:0/882068626 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f71a80a2cb0 0x7f71a80a5170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f71bc19a260 tx=0x7f71ac009260 comp rx=0 tx=0).stop 2024-06-10T00:55:12.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.439+0000 7f71c4148700 1 -- 172.21.15.37:0/882068626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71bc102530 msgr2=0x7f71bc19ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:12.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.439+0000 7f71c4148700 1 --2- 172.21.15.37:0/882068626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71bc102530 0x7f71bc19ea40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f71b800d950 tx=0x7f71b800dc60 comp rx=0 tx=0).stop 2024-06-10T00:55:12.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.439+0000 7f71c4148700 1 -- 172.21.15.37:0/882068626 shutdown_connections 2024-06-10T00:55:12.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.439+0000 7f71c4148700 1 --2- 172.21.15.37:0/882068626 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f71a80a2cb0 0x7f71a80a5170 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:12.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.439+0000 7f71c4148700 1 --2- 172.21.15.37:0/882068626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71bc1085d0 0x7f71bc198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:12.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.439+0000 7f71c4148700 1 --2- 172.21.15.37:0/882068626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71bc102530 0x7f71bc19ea40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:12.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.439+0000 7f71c4148700 1 --2- 172.21.15.37:0/882068626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71bc102ef0 0x7f71bc19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:12.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.439+0000 7f71c4148700 1 -- 172.21.15.37:0/882068626 >> 172.21.15.37:0/882068626 conn(0x7f71bc079640 msgr2=0x7f71bc101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:12.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.439+0000 7f71c4148700 1 -- 172.21.15.37:0/882068626 shutdown_connections 2024-06-10T00:55:12.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:12.439+0000 7f71c4148700 1 -- 172.21.15.37:0/882068626 wait complete. 2024-06-10T00:55:12.766 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:12 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3712900517' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-10T00:55:12.766 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:12 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/882068626' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-10T00:55:12.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:12 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3712900517' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-10T00:55:12.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:12 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/882068626' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-10T00:55:12.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:12 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3712900517' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-10T00:55:12.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:12 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/882068626' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-10T00:55:13.245 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776181 got 292057776185 for osd.9 2024-06-10T00:55:13.246 DEBUG:teuthology.parallel:result is None 2024-06-10T00:55:13.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:13 smithi167 ceph-mon[36917]: pgmap v386: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:55:13.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:13 smithi094 ceph-mon[36708]: pgmap v386: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:55:13.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:13 smithi037 ceph-mon[32192]: pgmap v386: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:55:13.971 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738465 got 34359738470 for osd.0 2024-06-10T00:55:13.971 DEBUG:teuthology.parallel:result is None 2024-06-10T00:55:15.018 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411415 got 77309411420 for osd.2 2024-06-10T00:55:15.019 DEBUG:teuthology.parallel:result is None 2024-06-10T00:55:15.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:15 smithi167 ceph-mon[36917]: pgmap v387: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T00:55:15.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:15 smithi094 ceph-mon[36708]: pgmap v387: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T00:55:15.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:15 smithi037 ceph-mon[32192]: pgmap v387: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T00:55:15.984 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574941 got 55834574946 for osd.1 2024-06-10T00:55:15.985 DEBUG:teuthology.parallel:result is None 2024-06-10T00:55:16.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.435+0000 7f10b9fcd700 1 -- 172.21.15.37:0/3888375485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10b41084f0 msgr2=0x7f10b41088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:16.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.435+0000 7f10b9fcd700 1 --2- 172.21.15.37:0/3888375485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10b41084f0 0x7f10b41088d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f10a4009990 tx=0x7f10a4009ca0 comp rx=0 tx=0).stop 2024-06-10T00:55:16.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.435+0000 7f10b9fcd700 1 -- 172.21.15.37:0/3888375485 shutdown_connections 2024-06-10T00:55:16.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.435+0000 7f10b9fcd700 1 --2- 172.21.15.37:0/3888375485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10b4102e80 0x7f10b410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:16.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.435+0000 7f10b9fcd700 1 --2- 172.21.15.37:0/3888375485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10b41084f0 0x7f10b41088d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:16.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.435+0000 7f10b9fcd700 1 --2- 172.21.15.37:0/3888375485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10b41024e0 0x7f10b4102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:16.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.435+0000 7f10b9fcd700 1 -- 172.21.15.37:0/3888375485 >> 172.21.15.37:0/3888375485 conn(0x7f10b4079530 msgr2=0x7f10b4079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:16.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.435+0000 7f10b9fcd700 1 -- 172.21.15.37:0/3888375485 shutdown_connections 2024-06-10T00:55:16.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.435+0000 7f10b9fcd700 1 -- 172.21.15.37:0/3888375485 wait complete. 2024-06-10T00:55:16.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.435+0000 7f10b9fcd700 1 Processor -- start 2024-06-10T00:55:16.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.435+0000 7f10b9fcd700 1 -- start start 2024-06-10T00:55:16.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.436+0000 7f10b9fcd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10b41024e0 0x7f10b4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:16.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.436+0000 7f10b9fcd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10b4102e80 0x7f10b4074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:16.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.436+0000 7f10b9fcd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10b41084f0 0x7f10b4075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:16.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.436+0000 7f10b9fcd700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f10b4112730 con 0x7f10b41024e0 2024-06-10T00:55:16.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.436+0000 7f10b9fcd700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f10b41125b0 con 0x7f10b4102e80 2024-06-10T00:55:16.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.436+0000 7f10b9fcd700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f10b41128b0 con 0x7f10b41084f0 2024-06-10T00:55:16.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.436+0000 7f10b37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10b41024e0 0x7f10b4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:16.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.436+0000 7f10b37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10b41024e0 0x7f10b4074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41470/0 (socket says 172.21.15.37:41470) 2024-06-10T00:55:16.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.436+0000 7f10b37fe700 1 -- 172.21.15.37:0/2440126191 learned_addr learned my addr 172.21.15.37:0/2440126191 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:16.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.436+0000 7f10b3fff700 1 --2- 172.21.15.37:0/2440126191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10b41084f0 0x7f10b4075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:16.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.436+0000 7f10aaffd700 1 --2- 172.21.15.37:0/2440126191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10b4102e80 0x7f10b4074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:16.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.436+0000 7f10b37fe700 1 -- 172.21.15.37:0/2440126191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10b41084f0 msgr2=0x7f10b4075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:16.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.436+0000 7f10b37fe700 1 --2- 172.21.15.37:0/2440126191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10b41084f0 0x7f10b4075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:16.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.436+0000 7f10b37fe700 1 -- 172.21.15.37:0/2440126191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10b4102e80 msgr2=0x7f10b4074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:16.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.436+0000 7f10b37fe700 1 --2- 172.21.15.37:0/2440126191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10b4102e80 0x7f10b4074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:16.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.436+0000 7f10b37fe700 1 -- 172.21.15.37:0/2440126191 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10b41a7110 con 0x7f10b41024e0 2024-06-10T00:55:16.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.436+0000 7f10b37fe700 1 --2- 172.21.15.37:0/2440126191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10b41024e0 0x7f10b4074790 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f10a400bff0 tx=0x7f10a4005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:16.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.437+0000 7f10b17fa700 1 -- 172.21.15.37:0/2440126191 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a4027070 con 0x7f10b41024e0 2024-06-10T00:55:16.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.437+0000 7f10b9fcd700 1 -- 172.21.15.37:0/2440126191 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10b41a73a0 con 0x7f10b41024e0 2024-06-10T00:55:16.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.437+0000 7f10b17fa700 1 -- 172.21.15.37:0/2440126191 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10a4004150 con 0x7f10b41024e0 2024-06-10T00:55:16.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.437+0000 7f10b17fa700 1 -- 172.21.15.37:0/2440126191 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a400f460 con 0x7f10b41024e0 2024-06-10T00:55:16.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.437+0000 7f10b9fcd700 1 -- 172.21.15.37:0/2440126191 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10b41a78d0 con 0x7f10b41024e0 2024-06-10T00:55:16.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.438+0000 7f10b9fcd700 1 -- 172.21.15.37:0/2440126191 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10b4069180 con 0x7f10b41024e0 2024-06-10T00:55:16.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.439+0000 7f10b17fa700 1 -- 172.21.15.37:0/2440126191 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f10a402a070 con 0x7f10b41024e0 2024-06-10T00:55:16.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.440+0000 7f10b17fa700 1 --2- 172.21.15.37:0/2440126191 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f10a00a2cb0 0x7f10a00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:16.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.440+0000 7f10aaffd700 1 --2- 172.21.15.37:0/2440126191 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f10a00a2cb0 0x7f10a00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:16.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.440+0000 7f10b17fa700 1 -- 172.21.15.37:0/2440126191 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f10a40d0c50 con 0x7f10b41024e0 2024-06-10T00:55:16.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.440+0000 7f10aaffd700 1 --2- 172.21.15.37:0/2440126191 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f10a00a2cb0 0x7f10a00a5170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f109c005fd0 tx=0x7f109c00a560 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:16.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.444+0000 7f10b17fa700 1 -- 172.21.15.37:0/2440126191 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f10a4031020 con 0x7f10b41024e0 2024-06-10T00:55:16.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.653+0000 7f10b9fcd700 1 -- 172.21.15.37:0/2440126191 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f10b402dca0 con 0x7f10b41024e0 2024-06-10T00:55:16.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.653+0000 7f10b17fa700 1 -- 172.21.15.37:0/2440126191 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f10a4013070 con 0x7f10b41024e0 2024-06-10T00:55:16.654 INFO:teuthology.orchestra.run.smithi037.stdout:223338299461 2024-06-10T00:55:16.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.655+0000 7f10b9fcd700 1 -- 172.21.15.37:0/2440126191 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f10a00a2cb0 msgr2=0x7f10a00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:16.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.655+0000 7f10b9fcd700 1 --2- 172.21.15.37:0/2440126191 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f10a00a2cb0 0x7f10a00a5170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f109c005fd0 tx=0x7f109c00a560 comp rx=0 tx=0).stop 2024-06-10T00:55:16.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.655+0000 7f10b9fcd700 1 -- 172.21.15.37:0/2440126191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10b41024e0 msgr2=0x7f10b4074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:16.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.655+0000 7f10b9fcd700 1 --2- 172.21.15.37:0/2440126191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10b41024e0 0x7f10b4074790 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f10a400bff0 tx=0x7f10a4005640 comp rx=0 tx=0).stop 2024-06-10T00:55:16.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.655+0000 7f10b9fcd700 1 -- 172.21.15.37:0/2440126191 shutdown_connections 2024-06-10T00:55:16.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.655+0000 7f10b9fcd700 1 --2- 172.21.15.37:0/2440126191 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f10a00a2cb0 0x7f10a00a5170 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:16.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.655+0000 7f10b9fcd700 1 --2- 172.21.15.37:0/2440126191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10b41084f0 0x7f10b4075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:16.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.655+0000 7f10b9fcd700 1 --2- 172.21.15.37:0/2440126191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10b41024e0 0x7f10b4074790 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:16.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.655+0000 7f10b9fcd700 1 --2- 172.21.15.37:0/2440126191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10b4102e80 0x7f10b4074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:16.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.655+0000 7f10b9fcd700 1 -- 172.21.15.37:0/2440126191 >> 172.21.15.37:0/2440126191 conn(0x7f10b4079530 msgr2=0x7f10b4100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:16.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.655+0000 7f10b9fcd700 1 -- 172.21.15.37:0/2440126191 shutdown_connections 2024-06-10T00:55:16.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:16.656+0000 7f10b9fcd700 1 -- 172.21.15.37:0/2440126191 wait complete. 2024-06-10T00:55:17.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.109+0000 7f6b8f226700 1 -- 172.21.15.37:0/511976612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b88108500 msgr2=0x7f6b881088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:17.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.109+0000 7f6b8f226700 1 --2- 172.21.15.37:0/511976612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b88108500 0x7f6b881088e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6b78009990 tx=0x7f6b78009ca0 comp rx=0 tx=0).stop 2024-06-10T00:55:17.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.109+0000 7f6b8f226700 1 -- 172.21.15.37:0/511976612 shutdown_connections 2024-06-10T00:55:17.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.109+0000 7f6b8f226700 1 --2- 172.21.15.37:0/511976612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b8806ab30 0x7f6b8810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:17.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.109+0000 7f6b8f226700 1 --2- 172.21.15.37:0/511976612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b88108500 0x7f6b881088e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:17.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.109+0000 7f6b8f226700 1 --2- 172.21.15.37:0/511976612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b8806a170 0x7f6b8806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:17.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.109+0000 7f6b8f226700 1 -- 172.21.15.37:0/511976612 >> 172.21.15.37:0/511976612 conn(0x7f6b88078340 msgr2=0x7f6b88078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:17.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.109+0000 7f6b8f226700 1 -- 172.21.15.37:0/511976612 shutdown_connections 2024-06-10T00:55:17.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.109+0000 7f6b8f226700 1 -- 172.21.15.37:0/511976612 wait complete. 2024-06-10T00:55:17.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.110+0000 7f6b8f226700 1 Processor -- start 2024-06-10T00:55:17.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.110+0000 7f6b8f226700 1 -- start start 2024-06-10T00:55:17.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.110+0000 7f6b8f226700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b8806a170 0x7f6b88101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:17.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.110+0000 7f6b8f226700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b8806ab30 0x7f6b881019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:17.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.110+0000 7f6b8f226700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b88108500 0x7f6b88101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:17.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.110+0000 7f6b8f226700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6b881127e0 con 0x7f6b88108500 2024-06-10T00:55:17.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.110+0000 7f6b8f226700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6b88112660 con 0x7f6b8806a170 2024-06-10T00:55:17.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.110+0000 7f6b8f226700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6b88112960 con 0x7f6b8806ab30 2024-06-10T00:55:17.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.110+0000 7f6b8d7c3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b88108500 0x7f6b88101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:17.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.110+0000 7f6b8d7c3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b88108500 0x7f6b88101ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41490/0 (socket says 172.21.15.37:41490) 2024-06-10T00:55:17.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.110+0000 7f6b8cfc2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b8806a170 0x7f6b88101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:17.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.110+0000 7f6b8d7c3700 1 -- 172.21.15.37:0/3428630172 learned_addr learned my addr 172.21.15.37:0/3428630172 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:17.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.111+0000 7f6b87fff700 1 --2- 172.21.15.37:0/3428630172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b8806ab30 0x7f6b881019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:17.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.111+0000 7f6b8d7c3700 1 -- 172.21.15.37:0/3428630172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b8806ab30 msgr2=0x7f6b881019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:17.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.111+0000 7f6b8d7c3700 1 --2- 172.21.15.37:0/3428630172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b8806ab30 0x7f6b881019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:17.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.111+0000 7f6b8d7c3700 1 -- 172.21.15.37:0/3428630172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b8806a170 msgr2=0x7f6b88101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:17.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.111+0000 7f6b8d7c3700 1 --2- 172.21.15.37:0/3428630172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b8806a170 0x7f6b88101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:17.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.111+0000 7f6b8d7c3700 1 -- 172.21.15.37:0/3428630172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b881a71a0 con 0x7f6b88108500 2024-06-10T00:55:17.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.111+0000 7f6b8cfc2700 1 --2- 172.21.15.37:0/3428630172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b8806a170 0x7f6b88101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:55:17.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.111+0000 7f6b8d7c3700 1 --2- 172.21.15.37:0/3428630172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b88108500 0x7f6b88101ee0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6b80007ee0 tx=0x7f6b8000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:17.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.111+0000 7f6b85ffb700 1 -- 172.21.15.37:0/3428630172 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8000dbd0 con 0x7f6b88108500 2024-06-10T00:55:17.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.111+0000 7f6b85ffb700 1 -- 172.21.15.37:0/3428630172 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b8000dd70 con 0x7f6b88108500 2024-06-10T00:55:17.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.111+0000 7f6b8f226700 1 -- 172.21.15.37:0/3428630172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b881a7490 con 0x7f6b88108500 2024-06-10T00:55:17.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.111+0000 7f6b85ffb700 1 -- 172.21.15.37:0/3428630172 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b80005a60 con 0x7f6b88108500 2024-06-10T00:55:17.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.111+0000 7f6b8f226700 1 -- 172.21.15.37:0/3428630172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b881a79d0 con 0x7f6b88108500 2024-06-10T00:55:17.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.112+0000 7f6b8f226700 1 -- 172.21.15.37:0/3428630172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b8810a9e0 con 0x7f6b88108500 2024-06-10T00:55:17.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.114+0000 7f6b85ffb700 1 -- 172.21.15.37:0/3428630172 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6b8001a020 con 0x7f6b88108500 2024-06-10T00:55:17.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.114+0000 7f6b85ffb700 1 --2- 172.21.15.37:0/3428630172 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f6b700a2d30 0x7f6b700a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:17.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.115+0000 7f6b8cfc2700 1 --2- 172.21.15.37:0/3428630172 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f6b700a2d30 0x7f6b700a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:17.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.115+0000 7f6b85ffb700 1 -- 172.21.15.37:0/3428630172 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f6b800cecb0 con 0x7f6b88108500 2024-06-10T00:55:17.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.115+0000 7f6b8cfc2700 1 --2- 172.21.15.37:0/3428630172 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f6b700a2d30 0x7f6b700a51f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6b7800b5c0 tx=0x7f6b78005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:17.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.119+0000 7f6b85ffb700 1 -- 172.21.15.37:0/3428630172 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6b80010070 con 0x7f6b88108500 2024-06-10T00:55:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.329+0000 7f6b8f226700 1 -- 172.21.15.37:0/3428630172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f6b8810a3c0 con 0x7f6b88108500 2024-06-10T00:55:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.329+0000 7f6b85ffb700 1 -- 172.21.15.37:0/3428630172 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f6b8009b6c0 con 0x7f6b88108500 2024-06-10T00:55:17.330 INFO:teuthology.orchestra.run.smithi037.stdout:347892351024 2024-06-10T00:55:17.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.332+0000 7f6b8f226700 1 -- 172.21.15.37:0/3428630172 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f6b700a2d30 msgr2=0x7f6b700a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:17.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.332+0000 7f6b8f226700 1 --2- 172.21.15.37:0/3428630172 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f6b700a2d30 0x7f6b700a51f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6b7800b5c0 tx=0x7f6b78005960 comp rx=0 tx=0).stop 2024-06-10T00:55:17.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.332+0000 7f6b8f226700 1 -- 172.21.15.37:0/3428630172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b88108500 msgr2=0x7f6b88101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:17.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.332+0000 7f6b8f226700 1 --2- 172.21.15.37:0/3428630172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b88108500 0x7f6b88101ee0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6b80007ee0 tx=0x7f6b8000d3b0 comp rx=0 tx=0).stop 2024-06-10T00:55:17.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.332+0000 7f6b8f226700 1 -- 172.21.15.37:0/3428630172 shutdown_connections 2024-06-10T00:55:17.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.332+0000 7f6b8f226700 1 --2- 172.21.15.37:0/3428630172 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f6b700a2d30 0x7f6b700a51f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:17.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.332+0000 7f6b8f226700 1 --2- 172.21.15.37:0/3428630172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b88108500 0x7f6b88101ee0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:17.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.332+0000 7f6b8f226700 1 --2- 172.21.15.37:0/3428630172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b8806a170 0x7f6b88101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:17.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.332+0000 7f6b8f226700 1 --2- 172.21.15.37:0/3428630172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b8806ab30 0x7f6b881019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:17.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.332+0000 7f6b8f226700 1 -- 172.21.15.37:0/3428630172 >> 172.21.15.37:0/3428630172 conn(0x7f6b88078340 msgr2=0x7f6b88100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:17.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.332+0000 7f6b8f226700 1 -- 172.21.15.37:0/3428630172 shutdown_connections 2024-06-10T00:55:17.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:17.332+0000 7f6b8f226700 1 -- 172.21.15.37:0/3428630172 wait complete. 2024-06-10T00:55:17.615 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:17 smithi037 ceph-mon[32192]: pgmap v388: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:55:17.616 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:17 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2440126191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-10T00:55:17.616 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:17 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3428630172' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-10T00:55:17.644 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299454 got 223338299461 for osd.7 2024-06-10T00:55:17.644 DEBUG:teuthology.parallel:result is None 2024-06-10T00:55:17.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:17 smithi167 ceph-mon[36917]: pgmap v388: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:55:17.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:17 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2440126191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-10T00:55:17.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:17 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3428630172' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-10T00:55:17.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:17 smithi094 ceph-mon[36708]: pgmap v388: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:55:17.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:17 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2440126191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-10T00:55:17.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:17 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3428630172' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-10T00:55:18.325 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351018 got 347892351024 for osd.11 2024-06-10T00:55:18.325 DEBUG:teuthology.parallel:result is None 2024-06-10T00:55:18.325 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-10T00:55:18.326 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-10T00:55:18.600 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:55:19.741 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:19 smithi037 ceph-mon[32192]: pgmap v389: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:55:19.741 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:19 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:55:19.741 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:19 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:55:19.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:19 smithi167 ceph-mon[36917]: pgmap v389: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:55:19.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:19 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:55:19.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:19 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:55:19.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:19 smithi094 ceph-mon[36708]: pgmap v389: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:55:19.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:55:19.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:19 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:55:20.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.358+0000 7f96b1ce0700 1 -- 172.21.15.37:0/454197751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96ac101390 msgr2=0x7f96ac101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:20.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.358+0000 7f96b1ce0700 1 --2- 172.21.15.37:0/454197751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96ac101390 0x7f96ac101810 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f969c009990 tx=0x7f969c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:55:20.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.359+0000 7f96b1ce0700 1 -- 172.21.15.37:0/454197751 shutdown_connections 2024-06-10T00:55:20.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.359+0000 7f96b1ce0700 1 --2- 172.21.15.37:0/454197751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96ac101d50 0x7f96ac10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:20.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.359+0000 7f96b1ce0700 1 --2- 172.21.15.37:0/454197751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96ac107400 0x7f96ac1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:20.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.359+0000 7f96b1ce0700 1 --2- 172.21.15.37:0/454197751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96ac101390 0x7f96ac101810 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:20.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.359+0000 7f96b1ce0700 1 -- 172.21.15.37:0/454197751 >> 172.21.15.37:0/454197751 conn(0x7f96ac078340 msgr2=0x7f96ac078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:20.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.365+0000 7f96b1ce0700 1 -- 172.21.15.37:0/454197751 shutdown_connections 2024-06-10T00:55:20.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.365+0000 7f96b1ce0700 1 -- 172.21.15.37:0/454197751 wait complete. 2024-06-10T00:55:20.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.366+0000 7f96b1ce0700 1 Processor -- start 2024-06-10T00:55:20.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.367+0000 7f96b1ce0700 1 -- start start 2024-06-10T00:55:20.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.367+0000 7f96b1ce0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96ac101390 0x7f96ac10a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:20.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.368+0000 7f96b1ce0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96ac101d50 0x7f96ac10a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:20.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.368+0000 7f96b1ce0700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96ac107400 0x7f96ac10aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:20.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.368+0000 7f96b1ce0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f96ac1106a0 con 0x7f96ac101d50 2024-06-10T00:55:20.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.368+0000 7f96b1ce0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f96ac110520 con 0x7f96ac101390 2024-06-10T00:55:20.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.368+0000 7f96b1ce0700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f96ac110820 con 0x7f96ac107400 2024-06-10T00:55:20.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.368+0000 7f96ab7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96ac101390 0x7f96ac10a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:20.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.368+0000 7f96ab7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96ac101390 0x7f96ac10a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:52014/0 (socket says 172.21.15.37:52014) 2024-06-10T00:55:20.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.368+0000 7f96ab7fe700 1 -- 172.21.15.37:0/3019662586 learned_addr learned my addr 172.21.15.37:0/3019662586 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:20.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.368+0000 7f96abfff700 1 --2- 172.21.15.37:0/3019662586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96ac107400 0x7f96ac10aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:20.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.368+0000 7f96aaffd700 1 --2- 172.21.15.37:0/3019662586 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96ac101d50 0x7f96ac10a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:20.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.368+0000 7f96ab7fe700 1 -- 172.21.15.37:0/3019662586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96ac107400 msgr2=0x7f96ac10aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:20.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.368+0000 7f96ab7fe700 1 --2- 172.21.15.37:0/3019662586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96ac107400 0x7f96ac10aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:20.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.368+0000 7f96ab7fe700 1 -- 172.21.15.37:0/3019662586 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96ac101d50 msgr2=0x7f96ac10a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:20.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.368+0000 7f96ab7fe700 1 --2- 172.21.15.37:0/3019662586 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96ac101d50 0x7f96ac10a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:20.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.368+0000 7f96ab7fe700 1 -- 172.21.15.37:0/3019662586 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ac116010 con 0x7f96ac101390 2024-06-10T00:55:20.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.369+0000 7f96ab7fe700 1 --2- 172.21.15.37:0/3019662586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96ac101390 0x7f96ac10a430 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f96a000b580 tx=0x7f96a000b890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:20.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.370+0000 7f96a8ff9700 1 -- 172.21.15.37:0/3019662586 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0015020 con 0x7f96ac101390 2024-06-10T00:55:20.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.370+0000 7f96b1ce0700 1 -- 172.21.15.37:0/3019662586 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96ac116300 con 0x7f96ac101390 2024-06-10T00:55:20.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.370+0000 7f96b1ce0700 1 -- 172.21.15.37:0/3019662586 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96ac116890 con 0x7f96ac101390 2024-06-10T00:55:20.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.370+0000 7f96a8ff9700 1 -- 172.21.15.37:0/3019662586 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96a0011d10 con 0x7f96ac101390 2024-06-10T00:55:20.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.371+0000 7f96a8ff9700 1 -- 172.21.15.37:0/3019662586 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a000f2e0 con 0x7f96ac101390 2024-06-10T00:55:20.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.371+0000 7f96b1ce0700 1 -- 172.21.15.37:0/3019662586 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9698005330 con 0x7f96ac101390 2024-06-10T00:55:20.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.374+0000 7f96a8ff9700 1 -- 172.21.15.37:0/3019662586 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f96a0018020 con 0x7f96ac101390 2024-06-10T00:55:20.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.374+0000 7f96a8ff9700 1 --2- 172.21.15.37:0/3019662586 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f96940a2de0 0x7f96940a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:20.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.375+0000 7f96a8ff9700 1 -- 172.21.15.37:0/3019662586 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f96a00cf020 con 0x7f96ac101390 2024-06-10T00:55:20.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.375+0000 7f96aaffd700 1 --2- 172.21.15.37:0/3019662586 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f96940a2de0 0x7f96940a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:20.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.375+0000 7f96aaffd700 1 --2- 172.21.15.37:0/3019662586 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f96940a2de0 0x7f96940a52a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f96ac10bbf0 tx=0x7f969c01a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:20.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.381+0000 7f96a8ff9700 1 -- 172.21.15.37:0/3019662586 <== mon.2 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f96a001f020 con 0x7f96ac101390 2024-06-10T00:55:20.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.588+0000 7f96b1ce0700 1 -- 172.21.15.37:0/3019662586 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9698000bc0 con 0x7f96940a2de0 2024-06-10T00:55:20.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.597+0000 7f96a8ff9700 1 -- 172.21.15.37:0/3019662586 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340776 (secure 0 0 0) 0x7f969c07b340 con 0x7f96940a2de0 2024-06-10T00:55:20.603 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:55:20.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.601+0000 7f96b1ce0700 1 -- 172.21.15.37:0/3019662586 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f96940a2de0 msgr2=0x7f96940a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:20.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.601+0000 7f96b1ce0700 1 --2- 172.21.15.37:0/3019662586 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f96940a2de0 0x7f96940a52a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f96ac10bbf0 tx=0x7f969c01a040 comp rx=0 tx=0).stop 2024-06-10T00:55:20.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.601+0000 7f96b1ce0700 1 -- 172.21.15.37:0/3019662586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96ac101390 msgr2=0x7f96ac10a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:20.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.601+0000 7f96b1ce0700 1 --2- 172.21.15.37:0/3019662586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96ac101390 0x7f96ac10a430 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f96a000b580 tx=0x7f96a000b890 comp rx=0 tx=0).stop 2024-06-10T00:55:20.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.601+0000 7f96b1ce0700 1 -- 172.21.15.37:0/3019662586 shutdown_connections 2024-06-10T00:55:20.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.601+0000 7f96b1ce0700 1 --2- 172.21.15.37:0/3019662586 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f96940a2de0 0x7f96940a52a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:20.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.601+0000 7f96b1ce0700 1 --2- 172.21.15.37:0/3019662586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96ac107400 0x7f96ac10aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:20.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.601+0000 7f96b1ce0700 1 --2- 172.21.15.37:0/3019662586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96ac101390 0x7f96ac10a430 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:20.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.601+0000 7f96b1ce0700 1 --2- 172.21.15.37:0/3019662586 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96ac101d50 0x7f96ac10a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:20.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.601+0000 7f96b1ce0700 1 -- 172.21.15.37:0/3019662586 >> 172.21.15.37:0/3019662586 conn(0x7f96ac078340 msgr2=0x7f96ac0ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:20.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.601+0000 7f96b1ce0700 1 -- 172.21.15.37:0/3019662586 shutdown_connections 2024-06-10T00:55:20.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:20.601+0000 7f96b1ce0700 1 -- 172.21.15.37:0/3019662586 wait complete. 2024-06-10T00:55:20.607 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-06-10T00:55:20.613 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:20 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:20.613 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:20 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:55:20.613 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:20 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:55:20.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:20 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:20.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:20 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:55:20.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:20 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:55:20.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:20 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:55:20.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:20 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:55:20.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:20 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:55:21.292 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":390,"stamp":"2024-06-10T00:55:20.387788+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664694,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":730,"num_read_kb":490,"num_write":354,"num_write_kb":3067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":286,"ondisk_log_size":286,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370956,"kb_used_data":13424,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570868,"statfs":{"total":1151940427776,"available":1151560568832,"internally_reserved":0,"allocated":13746176,"data_stored":8847224,"data_compressed":497625,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003409"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.559397+0000","last_change":"2024-06-10T00:52:35.971067+0000","last_active":"2024-06-10T00:54:22.559397+0000","last_peered":"2024-06-10T00:54:22.559397+0000","last_clean":"2024-06-10T00:54:22.559397+0000","last_became_active":"2024-06-10T00:52:35.970292+0000","last_became_peered":"2024-06-10T00:52:35.970292+0000","last_unstale":"2024-06-10T00:54:22.559397+0000","last_undegraded":"2024-06-10T00:54:22.559397+0000","last_fullsized":"2024-06-10T00:54:22.559397+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:57:37.287238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878547+0000","last_change":"2024-06-10T00:52:35.960421+0000","last_active":"2024-06-10T00:54:22.878547+0000","last_peered":"2024-06-10T00:54:22.878547+0000","last_clean":"2024-06-10T00:54:22.878547+0000","last_became_active":"2024-06-10T00:52:35.960070+0000","last_became_peered":"2024-06-10T00:52:35.960070+0000","last_unstale":"2024-06-10T00:54:22.878547+0000","last_undegraded":"2024-06-10T00:54:22.878547+0000","last_fullsized":"2024-06-10T00:54:22.878547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:48:02.616434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878431+0000","last_change":"2024-06-10T00:52:35.964778+0000","last_active":"2024-06-10T00:54:22.878431+0000","last_peered":"2024-06-10T00:54:22.878431+0000","last_clean":"2024-06-10T00:54:22.878431+0000","last_became_active":"2024-06-10T00:52:35.964463+0000","last_became_peered":"2024-06-10T00:52:35.964463+0000","last_unstale":"2024-06-10T00:54:22.878431+0000","last_undegraded":"2024-06-10T00:54:22.878431+0000","last_fullsized":"2024-06-10T00:54:22.878431+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:52:44.987363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668250+0000","last_change":"2024-06-10T00:52:35.963226+0000","last_active":"2024-06-10T00:54:22.668250+0000","last_peered":"2024-06-10T00:54:22.668250+0000","last_clean":"2024-06-10T00:54:22.668250+0000","last_became_active":"2024-06-10T00:52:35.962862+0000","last_became_peered":"2024-06-10T00:52:35.962862+0000","last_unstale":"2024-06-10T00:54:22.668250+0000","last_undegraded":"2024-06-10T00:54:22.668250+0000","last_fullsized":"2024-06-10T00:54:22.668250+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:08:56.418071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556100+0000","last_change":"2024-06-10T00:52:35.957427+0000","last_active":"2024-06-10T00:54:22.556100+0000","last_peered":"2024-06-10T00:54:22.556100+0000","last_clean":"2024-06-10T00:54:22.556100+0000","last_became_active":"2024-06-10T00:52:35.957049+0000","last_became_peered":"2024-06-10T00:52:35.957049+0000","last_unstale":"2024-06-10T00:54:22.556100+0000","last_undegraded":"2024-06-10T00:54:22.556100+0000","last_fullsized":"2024-06-10T00:54:22.556100+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:16:04.257210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554114+0000","last_change":"2024-06-10T00:52:35.954251+0000","last_active":"2024-06-10T00:54:22.554114+0000","last_peered":"2024-06-10T00:54:22.554114+0000","last_clean":"2024-06-10T00:54:22.554114+0000","last_became_active":"2024-06-10T00:52:35.953908+0000","last_became_peered":"2024-06-10T00:52:35.953908+0000","last_unstale":"2024-06-10T00:54:22.554114+0000","last_undegraded":"2024-06-10T00:54:22.554114+0000","last_fullsized":"2024-06-10T00:54:22.554114+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:55:25.125836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404257+0000","last_change":"2024-06-10T00:52:35.956822+0000","last_active":"2024-06-10T00:54:24.404257+0000","last_peered":"2024-06-10T00:54:24.404257+0000","last_clean":"2024-06-10T00:54:24.404257+0000","last_became_active":"2024-06-10T00:52:35.956466+0000","last_became_peered":"2024-06-10T00:52:35.956466+0000","last_unstale":"2024-06-10T00:54:24.404257+0000","last_undegraded":"2024-06-10T00:54:24.404257+0000","last_fullsized":"2024-06-10T00:54:24.404257+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:36:41.534392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601335+0000","last_change":"2024-06-10T00:52:35.962978+0000","last_active":"2024-06-10T00:54:22.601335+0000","last_peered":"2024-06-10T00:54:22.601335+0000","last_clean":"2024-06-10T00:54:22.601335+0000","last_became_active":"2024-06-10T00:52:35.962609+0000","last_became_peered":"2024-06-10T00:52:35.962609+0000","last_unstale":"2024-06-10T00:54:22.601335+0000","last_undegraded":"2024-06-10T00:54:22.601335+0000","last_fullsized":"2024-06-10T00:54:22.601335+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:20:57.469577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.562007+0000","last_change":"2024-06-10T00:52:35.963203+0000","last_active":"2024-06-10T00:54:22.562007+0000","last_peered":"2024-06-10T00:54:22.562007+0000","last_clean":"2024-06-10T00:54:22.562007+0000","last_became_active":"2024-06-10T00:52:35.962789+0000","last_became_peered":"2024-06-10T00:52:35.962789+0000","last_unstale":"2024-06-10T00:54:22.562007+0000","last_undegraded":"2024-06-10T00:54:22.562007+0000","last_fullsized":"2024-06-10T00:54:22.562007+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:00:17.819193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554495+0000","last_change":"2024-06-10T00:52:35.966940+0000","last_active":"2024-06-10T00:54:22.554495+0000","last_peered":"2024-06-10T00:54:22.554495+0000","last_clean":"2024-06-10T00:54:22.554495+0000","last_became_active":"2024-06-10T00:52:35.966554+0000","last_became_peered":"2024-06-10T00:52:35.966554+0000","last_unstale":"2024-06-10T00:54:22.554495+0000","last_undegraded":"2024-06-10T00:54:22.554495+0000","last_fullsized":"2024-06-10T00:54:22.554495+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:00.374437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404575+0000","last_change":"2024-06-10T00:52:35.960927+0000","last_active":"2024-06-10T00:54:24.404575+0000","last_peered":"2024-06-10T00:54:24.404575+0000","last_clean":"2024-06-10T00:54:24.404575+0000","last_became_active":"2024-06-10T00:52:35.960577+0000","last_became_peered":"2024-06-10T00:52:35.960577+0000","last_unstale":"2024-06-10T00:54:24.404575+0000","last_undegraded":"2024-06-10T00:54:24.404575+0000","last_fullsized":"2024-06-10T00:54:24.404575+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:50:02.366695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668213+0000","last_change":"2024-06-10T00:52:35.964435+0000","last_active":"2024-06-10T00:54:22.668213+0000","last_peered":"2024-06-10T00:54:22.668213+0000","last_clean":"2024-06-10T00:54:22.668213+0000","last_became_active":"2024-06-10T00:52:35.964069+0000","last_became_peered":"2024-06-10T00:52:35.964069+0000","last_unstale":"2024-06-10T00:54:22.668213+0000","last_undegraded":"2024-06-10T00:54:22.668213+0000","last_fullsized":"2024-06-10T00:54:22.668213+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:06.708551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553481+0000","last_change":"2024-06-10T00:52:35.966722+0000","last_active":"2024-06-10T00:54:22.553481+0000","last_peered":"2024-06-10T00:54:22.553481+0000","last_clean":"2024-06-10T00:54:22.553481+0000","last_became_active":"2024-06-10T00:52:35.966366+0000","last_became_peered":"2024-06-10T00:52:35.966366+0000","last_unstale":"2024-06-10T00:54:22.553481+0000","last_undegraded":"2024-06-10T00:54:22.553481+0000","last_fullsized":"2024-06-10T00:54:22.553481+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:57:46.021047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878151+0000","last_change":"2024-06-10T00:52:35.955307+0000","last_active":"2024-06-10T00:54:22.878151+0000","last_peered":"2024-06-10T00:54:22.878151+0000","last_clean":"2024-06-10T00:54:22.878151+0000","last_became_active":"2024-06-10T00:52:35.954140+0000","last_became_peered":"2024-06-10T00:52:35.954140+0000","last_unstale":"2024-06-10T00:54:22.878151+0000","last_undegraded":"2024-06-10T00:54:22.878151+0000","last_fullsized":"2024-06-10T00:54:22.878151+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:47:22.493598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878142+0000","last_change":"2024-06-10T00:52:35.960592+0000","last_active":"2024-06-10T00:54:22.878142+0000","last_peered":"2024-06-10T00:54:22.878142+0000","last_clean":"2024-06-10T00:54:22.878142+0000","last_became_active":"2024-06-10T00:52:35.960263+0000","last_became_peered":"2024-06-10T00:52:35.960263+0000","last_unstale":"2024-06-10T00:54:22.878142+0000","last_undegraded":"2024-06-10T00:54:22.878142+0000","last_fullsized":"2024-06-10T00:54:22.878142+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:47:11.658666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561312+0000","last_change":"2024-06-10T00:52:35.965796+0000","last_active":"2024-06-10T00:54:22.561312+0000","last_peered":"2024-06-10T00:54:22.561312+0000","last_clean":"2024-06-10T00:54:22.561312+0000","last_became_active":"2024-06-10T00:52:35.963965+0000","last_became_peered":"2024-06-10T00:52:35.963965+0000","last_unstale":"2024-06-10T00:54:22.561312+0000","last_undegraded":"2024-06-10T00:54:22.561312+0000","last_fullsized":"2024-06-10T00:54:22.561312+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:29:47.149852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.559195+0000","last_change":"2024-06-10T00:52:35.970714+0000","last_active":"2024-06-10T00:54:22.559195+0000","last_peered":"2024-06-10T00:54:22.559195+0000","last_clean":"2024-06-10T00:54:22.559195+0000","last_became_active":"2024-06-10T00:52:35.970173+0000","last_became_peered":"2024-06-10T00:52:35.970173+0000","last_unstale":"2024-06-10T00:54:22.559195+0000","last_undegraded":"2024-06-10T00:54:22.559195+0000","last_fullsized":"2024-06-10T00:54:22.559195+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:33:28.183156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556519+0000","last_change":"2024-06-10T00:52:35.963052+0000","last_active":"2024-06-10T00:54:22.556519+0000","last_peered":"2024-06-10T00:54:22.556519+0000","last_clean":"2024-06-10T00:54:22.556519+0000","last_became_active":"2024-06-10T00:52:35.962674+0000","last_became_peered":"2024-06-10T00:52:35.962674+0000","last_unstale":"2024-06-10T00:54:22.556519+0000","last_undegraded":"2024-06-10T00:54:22.556519+0000","last_fullsized":"2024-06-10T00:54:22.556519+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:56:38.836854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554975+0000","last_change":"2024-06-10T00:53:22.130336+0000","last_active":"2024-06-10T00:54:22.554975+0000","last_peered":"2024-06-10T00:54:22.554975+0000","last_clean":"2024-06-10T00:54:22.554975+0000","last_became_active":"2024-06-10T00:53:22.129910+0000","last_became_peered":"2024-06-10T00:53:22.129910+0000","last_unstale":"2024-06-10T00:54:22.554975+0000","last_undegraded":"2024-06-10T00:54:22.554975+0000","last_fullsized":"2024-06-10T00:54:22.554975+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:23:50.600821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554286+0000","last_change":"2024-06-10T00:52:45.997276+0000","last_active":"2024-06-10T00:54:22.554286+0000","last_peered":"2024-06-10T00:54:22.554286+0000","last_clean":"2024-06-10T00:54:22.554286+0000","last_became_active":"2024-06-10T00:52:45.995638+0000","last_became_peered":"2024-06-10T00:52:45.995638+0000","last_unstale":"2024-06-10T00:54:22.554286+0000","last_undegraded":"2024-06-10T00:54:22.554286+0000","last_fullsized":"2024-06-10T00:54:22.554286+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:14:20.922348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556499+0000","last_change":"2024-06-10T00:52:32.760888+0000","last_active":"2024-06-10T00:54:22.556499+0000","last_peered":"2024-06-10T00:54:22.556499+0000","last_clean":"2024-06-10T00:54:22.556499+0000","last_became_active":"2024-06-10T00:52:32.760531+0000","last_became_peered":"2024-06-10T00:52:32.760531+0000","last_unstale":"2024-06-10T00:54:22.556499+0000","last_undegraded":"2024-06-10T00:54:22.556499+0000","last_fullsized":"2024-06-10T00:54:22.556499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:05:59.052251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601475+0000","last_change":"2024-06-10T00:52:35.967626+0000","last_active":"2024-06-10T00:54:22.601475+0000","last_peered":"2024-06-10T00:54:22.601475+0000","last_clean":"2024-06-10T00:54:22.601475+0000","last_became_active":"2024-06-10T00:52:35.967295+0000","last_became_peered":"2024-06-10T00:52:35.967295+0000","last_unstale":"2024-06-10T00:54:22.601475+0000","last_undegraded":"2024-06-10T00:54:22.601475+0000","last_fullsized":"2024-06-10T00:54:22.601475+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:07:55.054327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668105+0000","last_change":"2024-06-10T00:52:45.993974+0000","last_active":"2024-06-10T00:54:22.668105+0000","last_peered":"2024-06-10T00:54:22.668105+0000","last_clean":"2024-06-10T00:54:22.668105+0000","last_became_active":"2024-06-10T00:52:45.993242+0000","last_became_peered":"2024-06-10T00:52:45.993242+0000","last_unstale":"2024-06-10T00:54:22.668105+0000","last_undegraded":"2024-06-10T00:54:22.668105+0000","last_fullsized":"2024-06-10T00:54:22.668105+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:28:12.810368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601807+0000","last_change":"2024-06-10T00:52:32.959905+0000","last_active":"2024-06-10T00:54:22.601807+0000","last_peered":"2024-06-10T00:54:22.601807+0000","last_clean":"2024-06-10T00:54:22.601807+0000","last_became_active":"2024-06-10T00:52:32.959551+0000","last_became_peered":"2024-06-10T00:52:32.959551+0000","last_unstale":"2024-06-10T00:54:22.601807+0000","last_undegraded":"2024-06-10T00:54:22.601807+0000","last_fullsized":"2024-06-10T00:54:22.601807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:33:48.029129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878378+0000","last_change":"2024-06-10T00:52:35.965790+0000","last_active":"2024-06-10T00:54:22.878378+0000","last_peered":"2024-06-10T00:54:22.878378+0000","last_clean":"2024-06-10T00:54:22.878378+0000","last_became_active":"2024-06-10T00:52:35.965475+0000","last_became_peered":"2024-06-10T00:52:35.965475+0000","last_unstale":"2024-06-10T00:54:22.878378+0000","last_undegraded":"2024-06-10T00:54:22.878378+0000","last_fullsized":"2024-06-10T00:54:22.878378+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:24:02.894411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.667953+0000","last_change":"2024-06-10T00:52:45.994464+0000","last_active":"2024-06-10T00:54:22.667953+0000","last_peered":"2024-06-10T00:54:22.667953+0000","last_clean":"2024-06-10T00:54:22.667953+0000","last_became_active":"2024-06-10T00:52:45.994022+0000","last_became_peered":"2024-06-10T00:52:45.994022+0000","last_unstale":"2024-06-10T00:54:22.667953+0000","last_undegraded":"2024-06-10T00:54:22.667953+0000","last_fullsized":"2024-06-10T00:54:22.667953+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:27:25.063074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556658+0000","last_change":"2024-06-10T00:52:32.763885+0000","last_active":"2024-06-10T00:54:22.556658+0000","last_peered":"2024-06-10T00:54:22.556658+0000","last_clean":"2024-06-10T00:54:22.556658+0000","last_became_active":"2024-06-10T00:52:32.763606+0000","last_became_peered":"2024-06-10T00:52:32.763606+0000","last_unstale":"2024-06-10T00:54:22.556658+0000","last_undegraded":"2024-06-10T00:54:22.556658+0000","last_fullsized":"2024-06-10T00:54:22.556658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:01:58.161168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561645+0000","last_change":"2024-06-10T00:52:35.966948+0000","last_active":"2024-06-10T00:54:22.561645+0000","last_peered":"2024-06-10T00:54:22.561645+0000","last_clean":"2024-06-10T00:54:22.561645+0000","last_became_active":"2024-06-10T00:52:35.966345+0000","last_became_peered":"2024-06-10T00:52:35.966345+0000","last_unstale":"2024-06-10T00:54:22.561645+0000","last_undegraded":"2024-06-10T00:54:22.561645+0000","last_fullsized":"2024-06-10T00:54:22.561645+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:58:27.169201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.667762+0000","last_change":"2024-06-10T00:52:45.993645+0000","last_active":"2024-06-10T00:54:22.667762+0000","last_peered":"2024-06-10T00:54:22.667762+0000","last_clean":"2024-06-10T00:54:22.667762+0000","last_became_active":"2024-06-10T00:52:45.993183+0000","last_became_peered":"2024-06-10T00:52:45.993183+0000","last_unstale":"2024-06-10T00:54:22.667762+0000","last_undegraded":"2024-06-10T00:54:22.667762+0000","last_fullsized":"2024-06-10T00:54:22.667762+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:47:00.541427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.565600+0000","last_change":"2024-06-10T00:54:22.565600+0000","last_active":"2024-06-10T00:54:22.565600+0000","last_peered":"2024-06-10T00:54:22.565600+0000","last_clean":"2024-06-10T00:54:22.565600+0000","last_became_active":"2024-06-10T00:54:22.562561+0000","last_became_peered":"2024-06-10T00:54:22.562561+0000","last_unstale":"2024-06-10T00:54:22.565600+0000","last_undegraded":"2024-06-10T00:54:22.565600+0000","last_fullsized":"2024-06-10T00:54:22.565600+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:47:58.748050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561500+0000","last_change":"2024-06-10T00:53:22.199668+0000","last_active":"2024-06-10T00:54:22.561500+0000","last_peered":"2024-06-10T00:54:22.561500+0000","last_clean":"2024-06-10T00:54:22.561500+0000","last_became_active":"2024-06-10T00:53:22.199329+0000","last_became_peered":"2024-06-10T00:53:22.199329+0000","last_unstale":"2024-06-10T00:54:22.561500+0000","last_undegraded":"2024-06-10T00:54:22.561500+0000","last_fullsized":"2024-06-10T00:54:22.561500+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:37.640478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554689+0000","last_change":"2024-06-10T00:52:45.995767+0000","last_active":"2024-06-10T00:54:22.554689+0000","last_peered":"2024-06-10T00:54:22.554689+0000","last_clean":"2024-06-10T00:54:22.554689+0000","last_became_active":"2024-06-10T00:52:45.994659+0000","last_became_peered":"2024-06-10T00:52:45.994659+0000","last_unstale":"2024-06-10T00:54:22.554689+0000","last_undegraded":"2024-06-10T00:54:22.554689+0000","last_fullsized":"2024-06-10T00:54:22.554689+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:03:45.652248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"98'6","reported_seq":148,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:15.438312+0000","last_change":"2024-06-10T00:52:32.761637+0000","last_active":"2024-06-10T00:55:15.438312+0000","last_peered":"2024-06-10T00:55:15.438312+0000","last_clean":"2024-06-10T00:55:15.438312+0000","last_became_active":"2024-06-10T00:52:32.761144+0000","last_became_peered":"2024-06-10T00:52:32.761144+0000","last_unstale":"2024-06-10T00:55:15.438312+0000","last_undegraded":"2024-06-10T00:55:15.438312+0000","last_fullsized":"2024-06-10T00:55:15.438312+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:55:05.616564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.557749+0000","last_change":"2024-06-10T00:52:35.962683+0000","last_active":"2024-06-10T00:54:22.557749+0000","last_peered":"2024-06-10T00:54:22.557749+0000","last_clean":"2024-06-10T00:54:22.557749+0000","last_became_active":"2024-06-10T00:52:35.962327+0000","last_became_peered":"2024-06-10T00:52:35.962327+0000","last_unstale":"2024-06-10T00:54:22.557749+0000","last_undegraded":"2024-06-10T00:54:22.557749+0000","last_fullsized":"2024-06-10T00:54:22.557749+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:38:19.063306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554563+0000","last_change":"2024-06-10T00:52:45.996882+0000","last_active":"2024-06-10T00:54:22.554563+0000","last_peered":"2024-06-10T00:54:22.554563+0000","last_clean":"2024-06-10T00:54:22.554563+0000","last_became_active":"2024-06-10T00:52:45.995767+0000","last_became_peered":"2024-06-10T00:52:45.995767+0000","last_unstale":"2024-06-10T00:54:22.554563+0000","last_undegraded":"2024-06-10T00:54:22.554563+0000","last_fullsized":"2024-06-10T00:54:22.554563+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:28:03.766944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553970+0000","last_change":"2024-06-10T00:53:22.200546+0000","last_active":"2024-06-10T00:54:22.553970+0000","last_peered":"2024-06-10T00:54:22.553970+0000","last_clean":"2024-06-10T00:54:22.553970+0000","last_became_active":"2024-06-10T00:53:22.200059+0000","last_became_peered":"2024-06-10T00:53:22.200059+0000","last_unstale":"2024-06-10T00:54:22.553970+0000","last_undegraded":"2024-06-10T00:54:22.553970+0000","last_fullsized":"2024-06-10T00:54:22.553970+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:08:47.682594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561766+0000","last_change":"2024-06-10T00:52:35.963105+0000","last_active":"2024-06-10T00:54:22.561766+0000","last_peered":"2024-06-10T00:54:22.561766+0000","last_clean":"2024-06-10T00:54:22.561766+0000","last_became_active":"2024-06-10T00:52:35.962627+0000","last_became_peered":"2024-06-10T00:52:35.962627+0000","last_unstale":"2024-06-10T00:54:22.561766+0000","last_undegraded":"2024-06-10T00:54:22.561766+0000","last_fullsized":"2024-06-10T00:54:22.561766+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:55:51.231795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.559190+0000","last_change":"2024-06-10T00:52:45.992557+0000","last_active":"2024-06-10T00:54:22.559190+0000","last_peered":"2024-06-10T00:54:22.559190+0000","last_clean":"2024-06-10T00:54:22.559190+0000","last_became_active":"2024-06-10T00:52:45.992166+0000","last_became_peered":"2024-06-10T00:52:45.992166+0000","last_unstale":"2024-06-10T00:54:22.559190+0000","last_undegraded":"2024-06-10T00:54:22.559190+0000","last_fullsized":"2024-06-10T00:54:22.559190+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:49:22.368409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553428+0000","last_change":"2024-06-10T00:52:32.751867+0000","last_active":"2024-06-10T00:54:22.553428+0000","last_peered":"2024-06-10T00:54:22.553428+0000","last_clean":"2024-06-10T00:54:22.553428+0000","last_became_active":"2024-06-10T00:52:32.751516+0000","last_became_peered":"2024-06-10T00:52:32.751516+0000","last_unstale":"2024-06-10T00:54:22.553428+0000","last_undegraded":"2024-06-10T00:54:22.553428+0000","last_fullsized":"2024-06-10T00:54:22.553428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:37:55.920613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601243+0000","last_change":"2024-06-10T00:52:35.957772+0000","last_active":"2024-06-10T00:54:22.601243+0000","last_peered":"2024-06-10T00:54:22.601243+0000","last_clean":"2024-06-10T00:54:22.601243+0000","last_became_active":"2024-06-10T00:52:35.957470+0000","last_became_peered":"2024-06-10T00:52:35.957470+0000","last_unstale":"2024-06-10T00:54:22.601243+0000","last_undegraded":"2024-06-10T00:54:22.601243+0000","last_fullsized":"2024-06-10T00:54:22.601243+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:16:43.645292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878568+0000","last_change":"2024-06-10T00:52:45.996807+0000","last_active":"2024-06-10T00:54:22.878568+0000","last_peered":"2024-06-10T00:54:22.878568+0000","last_clean":"2024-06-10T00:54:22.878568+0000","last_became_active":"2024-06-10T00:52:45.996378+0000","last_became_peered":"2024-06-10T00:52:45.996378+0000","last_unstale":"2024-06-10T00:54:22.878568+0000","last_undegraded":"2024-06-10T00:54:22.878568+0000","last_fullsized":"2024-06-10T00:54:22.878568+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:40:54.251790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'1","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561873+0000","last_change":"2024-06-10T00:52:32.759625+0000","last_active":"2024-06-10T00:54:22.561873+0000","last_peered":"2024-06-10T00:54:22.561873+0000","last_clean":"2024-06-10T00:54:22.561873+0000","last_became_active":"2024-06-10T00:52:32.759299+0000","last_became_peered":"2024-06-10T00:52:32.759299+0000","last_unstale":"2024-06-10T00:54:22.561873+0000","last_undegraded":"2024-06-10T00:54:22.561873+0000","last_fullsized":"2024-06-10T00:54:22.561873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:11:44.966453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.555979+0000","last_change":"2024-06-10T00:52:35.957672+0000","last_active":"2024-06-10T00:54:22.555979+0000","last_peered":"2024-06-10T00:54:22.555979+0000","last_clean":"2024-06-10T00:54:22.555979+0000","last_became_active":"2024-06-10T00:52:35.957328+0000","last_became_peered":"2024-06-10T00:52:35.957328+0000","last_unstale":"2024-06-10T00:54:22.555979+0000","last_undegraded":"2024-06-10T00:54:22.555979+0000","last_fullsized":"2024-06-10T00:54:22.555979+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:43:59.658178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601278+0000","last_change":"2024-06-10T00:52:45.996677+0000","last_active":"2024-06-10T00:54:22.601278+0000","last_peered":"2024-06-10T00:54:22.601278+0000","last_clean":"2024-06-10T00:54:22.601278+0000","last_became_active":"2024-06-10T00:52:45.994483+0000","last_became_peered":"2024-06-10T00:52:45.994483+0000","last_unstale":"2024-06-10T00:54:22.601278+0000","last_undegraded":"2024-06-10T00:54:22.601278+0000","last_fullsized":"2024-06-10T00:54:22.601278+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:03:28.712072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"98'3","reported_seq":143,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:14.436273+0000","last_change":"2024-06-10T00:52:32.754829+0000","last_active":"2024-06-10T00:55:14.436273+0000","last_peered":"2024-06-10T00:55:14.436273+0000","last_clean":"2024-06-10T00:55:14.436273+0000","last_became_active":"2024-06-10T00:52:32.754386+0000","last_became_peered":"2024-06-10T00:52:32.754386+0000","last_unstale":"2024-06-10T00:55:14.436273+0000","last_undegraded":"2024-06-10T00:55:14.436273+0000","last_fullsized":"2024-06-10T00:55:14.436273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:00.870324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":95,"num_read_kb":95,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.562006+0000","last_change":"2024-06-10T00:52:35.966871+0000","last_active":"2024-06-10T00:54:22.562006+0000","last_peered":"2024-06-10T00:54:22.562006+0000","last_clean":"2024-06-10T00:54:22.562006+0000","last_became_active":"2024-06-10T00:52:35.966492+0000","last_became_peered":"2024-06-10T00:52:35.966492+0000","last_unstale":"2024-06-10T00:54:22.562006+0000","last_undegraded":"2024-06-10T00:54:22.562006+0000","last_fullsized":"2024-06-10T00:54:22.562006+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:24:41.048935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.879950+0000","last_change":"2024-06-10T00:54:22.879457+0000","last_active":"2024-06-10T00:54:22.879950+0000","last_peered":"2024-06-10T00:54:22.879950+0000","last_clean":"2024-06-10T00:54:22.879950+0000","last_became_active":"2024-06-10T00:54:22.878978+0000","last_became_peered":"2024-06-10T00:54:22.878978+0000","last_unstale":"2024-06-10T00:54:22.879950+0000","last_undegraded":"2024-06-10T00:54:22.879950+0000","last_fullsized":"2024-06-10T00:54:22.879950+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:46:38.819250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404659+0000","last_change":"2024-06-10T00:52:32.765794+0000","last_active":"2024-06-10T00:54:24.404659+0000","last_peered":"2024-06-10T00:54:24.404659+0000","last_clean":"2024-06-10T00:54:24.404659+0000","last_became_active":"2024-06-10T00:52:32.765276+0000","last_became_peered":"2024-06-10T00:52:32.765276+0000","last_unstale":"2024-06-10T00:54:24.404659+0000","last_undegraded":"2024-06-10T00:54:24.404659+0000","last_fullsized":"2024-06-10T00:54:24.404659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:06:51.209012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878545+0000","last_change":"2024-06-10T00:52:35.954897+0000","last_active":"2024-06-10T00:54:22.878545+0000","last_peered":"2024-06-10T00:54:22.878545+0000","last_clean":"2024-06-10T00:54:22.878545+0000","last_became_active":"2024-06-10T00:52:35.954565+0000","last_became_peered":"2024-06-10T00:52:35.954565+0000","last_unstale":"2024-06-10T00:54:22.878545+0000","last_undegraded":"2024-06-10T00:54:22.878545+0000","last_fullsized":"2024-06-10T00:54:22.878545+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:39:26.702804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668190+0000","last_change":"2024-06-10T00:52:45.992194+0000","last_active":"2024-06-10T00:54:22.668190+0000","last_peered":"2024-06-10T00:54:22.668190+0000","last_clean":"2024-06-10T00:54:22.668190+0000","last_became_active":"2024-06-10T00:52:45.991453+0000","last_became_peered":"2024-06-10T00:52:45.991453+0000","last_unstale":"2024-06-10T00:54:22.668190+0000","last_undegraded":"2024-06-10T00:54:22.668190+0000","last_fullsized":"2024-06-10T00:54:22.668190+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:11:10.775610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561610+0000","last_change":"2024-06-10T00:52:32.962214+0000","last_active":"2024-06-10T00:54:22.561610+0000","last_peered":"2024-06-10T00:54:22.561610+0000","last_clean":"2024-06-10T00:54:22.561610+0000","last_became_active":"2024-06-10T00:52:32.961835+0000","last_became_peered":"2024-06-10T00:52:32.961835+0000","last_unstale":"2024-06-10T00:54:22.561610+0000","last_undegraded":"2024-06-10T00:54:22.561610+0000","last_fullsized":"2024-06-10T00:54:22.561610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:36:44.724122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561818+0000","last_change":"2024-06-10T00:52:35.963413+0000","last_active":"2024-06-10T00:54:22.561818+0000","last_peered":"2024-06-10T00:54:22.561818+0000","last_clean":"2024-06-10T00:54:22.561818+0000","last_became_active":"2024-06-10T00:52:35.962922+0000","last_became_peered":"2024-06-10T00:52:35.962922+0000","last_unstale":"2024-06-10T00:54:22.561818+0000","last_undegraded":"2024-06-10T00:54:22.561818+0000","last_fullsized":"2024-06-10T00:54:22.561818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:05:32.164696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":34,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.560580+0000","last_change":"2024-06-10T00:54:22.560196+0000","last_active":"2024-06-10T00:54:22.560580+0000","last_peered":"2024-06-10T00:54:22.560580+0000","last_clean":"2024-06-10T00:54:22.560580+0000","last_became_active":"2024-06-10T00:54:22.559826+0000","last_became_peered":"2024-06-10T00:54:22.559826+0000","last_unstale":"2024-06-10T00:54:22.560580+0000","last_undegraded":"2024-06-10T00:54:22.560580+0000","last_fullsized":"2024-06-10T00:54:22.560580+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:09:28.456394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'38","reported_seq":283,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:15.450221+0000","last_change":"2024-06-10T00:52:32.761592+0000","last_active":"2024-06-10T00:55:15.450221+0000","last_peered":"2024-06-10T00:55:15.450221+0000","last_clean":"2024-06-10T00:55:15.450221+0000","last_became_active":"2024-06-10T00:52:32.761169+0000","last_became_peered":"2024-06-10T00:52:32.761169+0000","last_unstale":"2024-06-10T00:55:15.450221+0000","last_undegraded":"2024-06-10T00:55:15.450221+0000","last_fullsized":"2024-06-10T00:55:15.450221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:08:33.015035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70092,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":200,"num_read_kb":170,"num_write":39,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556908+0000","last_change":"2024-06-10T00:52:35.955304+0000","last_active":"2024-06-10T00:54:22.556908+0000","last_peered":"2024-06-10T00:54:22.556908+0000","last_clean":"2024-06-10T00:54:22.556908+0000","last_became_active":"2024-06-10T00:52:35.954949+0000","last_became_peered":"2024-06-10T00:52:35.954949+0000","last_unstale":"2024-06-10T00:54:22.556908+0000","last_undegraded":"2024-06-10T00:54:22.556908+0000","last_fullsized":"2024-06-10T00:54:22.556908+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:56:42.890022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561827+0000","last_change":"2024-06-10T00:52:45.992974+0000","last_active":"2024-06-10T00:54:22.561827+0000","last_peered":"2024-06-10T00:54:22.561827+0000","last_clean":"2024-06-10T00:54:22.561827+0000","last_became_active":"2024-06-10T00:52:45.992490+0000","last_became_peered":"2024-06-10T00:52:45.992490+0000","last_unstale":"2024-06-10T00:54:22.561827+0000","last_undegraded":"2024-06-10T00:54:22.561827+0000","last_fullsized":"2024-06-10T00:54:22.561827+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:03:45.543476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554437+0000","last_change":"2024-06-10T00:52:32.762347+0000","last_active":"2024-06-10T00:54:22.554437+0000","last_peered":"2024-06-10T00:54:22.554437+0000","last_clean":"2024-06-10T00:54:22.554437+0000","last_became_active":"2024-06-10T00:52:32.761049+0000","last_became_peered":"2024-06-10T00:52:32.761049+0000","last_unstale":"2024-06-10T00:54:22.554437+0000","last_undegraded":"2024-06-10T00:54:22.554437+0000","last_fullsized":"2024-06-10T00:54:22.554437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:29:38.384451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553361+0000","last_change":"2024-06-10T00:52:35.961299+0000","last_active":"2024-06-10T00:54:22.553361+0000","last_peered":"2024-06-10T00:54:22.553361+0000","last_clean":"2024-06-10T00:54:22.553361+0000","last_became_active":"2024-06-10T00:52:35.960966+0000","last_became_peered":"2024-06-10T00:52:35.960966+0000","last_unstale":"2024-06-10T00:54:22.553361+0000","last_undegraded":"2024-06-10T00:54:22.553361+0000","last_fullsized":"2024-06-10T00:54:22.553361+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:32:21.269689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.559474+0000","last_change":"2024-06-10T00:52:46.002091+0000","last_active":"2024-06-10T00:54:22.559474+0000","last_peered":"2024-06-10T00:54:22.559474+0000","last_clean":"2024-06-10T00:54:22.559474+0000","last_became_active":"2024-06-10T00:52:46.001677+0000","last_became_peered":"2024-06-10T00:52:46.001677+0000","last_unstale":"2024-06-10T00:54:22.559474+0000","last_undegraded":"2024-06-10T00:54:22.559474+0000","last_fullsized":"2024-06-10T00:54:22.559474+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:16:56.405720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'3","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.562287+0000","last_change":"2024-06-10T00:52:32.756765+0000","last_active":"2024-06-10T00:54:22.562287+0000","last_peered":"2024-06-10T00:54:22.562287+0000","last_clean":"2024-06-10T00:54:22.562287+0000","last_became_active":"2024-06-10T00:52:32.756470+0000","last_became_peered":"2024-06-10T00:52:32.756470+0000","last_unstale":"2024-06-10T00:54:22.562287+0000","last_undegraded":"2024-06-10T00:54:22.562287+0000","last_fullsized":"2024-06-10T00:54:22.562287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:29:24.854913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404448+0000","last_change":"2024-06-10T00:52:35.960693+0000","last_active":"2024-06-10T00:54:24.404448+0000","last_peered":"2024-06-10T00:54:24.404448+0000","last_clean":"2024-06-10T00:54:24.404448+0000","last_became_active":"2024-06-10T00:52:35.958397+0000","last_became_peered":"2024-06-10T00:52:35.958397+0000","last_unstale":"2024-06-10T00:54:24.404448+0000","last_undegraded":"2024-06-10T00:54:24.404448+0000","last_fullsized":"2024-06-10T00:54:24.404448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:23:47.556462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556200+0000","last_change":"2024-06-10T00:52:46.001475+0000","last_active":"2024-06-10T00:54:22.556200+0000","last_peered":"2024-06-10T00:54:22.556200+0000","last_clean":"2024-06-10T00:54:22.556200+0000","last_became_active":"2024-06-10T00:52:46.000983+0000","last_became_peered":"2024-06-10T00:52:46.000983+0000","last_unstale":"2024-06-10T00:54:22.556200+0000","last_undegraded":"2024-06-10T00:54:22.556200+0000","last_fullsized":"2024-06-10T00:54:22.556200+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:23:56.285356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"98'3","reported_seq":146,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:16.438974+0000","last_change":"2024-06-10T00:52:32.756665+0000","last_active":"2024-06-10T00:55:16.438974+0000","last_peered":"2024-06-10T00:55:16.438974+0000","last_clean":"2024-06-10T00:55:16.438974+0000","last_became_active":"2024-06-10T00:52:32.756221+0000","last_became_peered":"2024-06-10T00:52:32.756221+0000","last_unstale":"2024-06-10T00:55:16.438974+0000","last_undegraded":"2024-06-10T00:55:16.438974+0000","last_fullsized":"2024-06-10T00:55:16.438974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:50:34.789063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404268+0000","last_change":"2024-06-10T00:53:22.125844+0000","last_active":"2024-06-10T00:54:24.404268+0000","last_peered":"2024-06-10T00:54:24.404268+0000","last_clean":"2024-06-10T00:54:24.404268+0000","last_became_active":"2024-06-10T00:53:22.125449+0000","last_became_peered":"2024-06-10T00:53:22.125449+0000","last_unstale":"2024-06-10T00:54:24.404268+0000","last_undegraded":"2024-06-10T00:54:24.404268+0000","last_fullsized":"2024-06-10T00:54:24.404268+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:08:42.936702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553904+0000","last_change":"2024-06-10T00:52:45.999300+0000","last_active":"2024-06-10T00:54:22.553904+0000","last_peered":"2024-06-10T00:54:22.553904+0000","last_clean":"2024-06-10T00:54:22.553904+0000","last_became_active":"2024-06-10T00:52:45.998860+0000","last_became_peered":"2024-06-10T00:52:45.998860+0000","last_unstale":"2024-06-10T00:54:22.553904+0000","last_undegraded":"2024-06-10T00:54:22.553904+0000","last_fullsized":"2024-06-10T00:54:22.553904+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:14:24.936870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":75,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.557182+0000","last_change":"2024-06-10T00:53:22.210710+0000","last_active":"2024-06-10T00:54:22.557182+0000","last_peered":"2024-06-10T00:54:22.557182+0000","last_clean":"2024-06-10T00:54:22.557182+0000","last_became_active":"2024-06-10T00:53:22.202461+0000","last_became_peered":"2024-06-10T00:53:22.202461+0000","last_unstale":"2024-06-10T00:54:22.557182+0000","last_undegraded":"2024-06-10T00:54:22.557182+0000","last_fullsized":"2024-06-10T00:54:22.557182+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:28:14.671776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561714+0000","last_change":"2024-06-10T00:52:35.966960+0000","last_active":"2024-06-10T00:54:22.561714+0000","last_peered":"2024-06-10T00:54:22.561714+0000","last_clean":"2024-06-10T00:54:22.561714+0000","last_became_active":"2024-06-10T00:52:35.966630+0000","last_became_peered":"2024-06-10T00:52:35.966630+0000","last_unstale":"2024-06-10T00:54:22.561714+0000","last_undegraded":"2024-06-10T00:54:22.561714+0000","last_fullsized":"2024-06-10T00:54:22.561714+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:04.117405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561586+0000","last_change":"2024-06-10T00:52:45.992955+0000","last_active":"2024-06-10T00:54:22.561586+0000","last_peered":"2024-06-10T00:54:22.561586+0000","last_clean":"2024-06-10T00:54:22.561586+0000","last_became_active":"2024-06-10T00:52:45.992457+0000","last_became_peered":"2024-06-10T00:52:45.992457+0000","last_unstale":"2024-06-10T00:54:22.561586+0000","last_undegraded":"2024-06-10T00:54:22.561586+0000","last_fullsized":"2024-06-10T00:54:22.561586+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:13:49.491183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556768+0000","last_change":"2024-06-10T00:52:32.763794+0000","last_active":"2024-06-10T00:54:22.556768+0000","last_peered":"2024-06-10T00:54:22.556768+0000","last_clean":"2024-06-10T00:54:22.556768+0000","last_became_active":"2024-06-10T00:52:32.763406+0000","last_became_peered":"2024-06-10T00:52:32.763406+0000","last_unstale":"2024-06-10T00:54:22.556768+0000","last_undegraded":"2024-06-10T00:54:22.556768+0000","last_fullsized":"2024-06-10T00:54:22.556768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:50:33.580163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.559163+0000","last_change":"2024-06-10T00:53:22.130891+0000","last_active":"2024-06-10T00:54:22.559163+0000","last_peered":"2024-06-10T00:54:22.559163+0000","last_clean":"2024-06-10T00:54:22.559163+0000","last_became_active":"2024-06-10T00:53:22.130508+0000","last_became_peered":"2024-06-10T00:53:22.130508+0000","last_unstale":"2024-06-10T00:54:22.559163+0000","last_undegraded":"2024-06-10T00:54:22.559163+0000","last_fullsized":"2024-06-10T00:54:22.559163+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:28:01.919825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":34,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.563757+0000","last_change":"2024-06-10T00:54:22.563272+0000","last_active":"2024-06-10T00:54:22.563757+0000","last_peered":"2024-06-10T00:54:22.563757+0000","last_clean":"2024-06-10T00:54:22.563757+0000","last_became_active":"2024-06-10T00:54:22.562767+0000","last_became_peered":"2024-06-10T00:54:22.562767+0000","last_unstale":"2024-06-10T00:54:22.563757+0000","last_undegraded":"2024-06-10T00:54:22.563757+0000","last_fullsized":"2024-06-10T00:54:22.563757+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:52:20.739801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556364+0000","last_change":"2024-06-10T00:52:32.960515+0000","last_active":"2024-06-10T00:54:22.556364+0000","last_peered":"2024-06-10T00:54:22.556364+0000","last_clean":"2024-06-10T00:54:22.556364+0000","last_became_active":"2024-06-10T00:52:32.960163+0000","last_became_peered":"2024-06-10T00:52:32.960163+0000","last_unstale":"2024-06-10T00:54:22.556364+0000","last_undegraded":"2024-06-10T00:54:22.556364+0000","last_fullsized":"2024-06-10T00:54:22.556364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:39:51.374998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554132+0000","last_change":"2024-06-10T00:52:35.961884+0000","last_active":"2024-06-10T00:54:22.554132+0000","last_peered":"2024-06-10T00:54:22.554132+0000","last_clean":"2024-06-10T00:54:22.554132+0000","last_became_active":"2024-06-10T00:52:35.961497+0000","last_became_peered":"2024-06-10T00:52:35.961497+0000","last_unstale":"2024-06-10T00:54:22.554132+0000","last_undegraded":"2024-06-10T00:54:22.554132+0000","last_fullsized":"2024-06-10T00:54:22.554132+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:16:09.284184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553591+0000","last_change":"2024-06-10T00:52:45.990846+0000","last_active":"2024-06-10T00:54:22.553591+0000","last_peered":"2024-06-10T00:54:22.553591+0000","last_clean":"2024-06-10T00:54:22.553591+0000","last_became_active":"2024-06-10T00:52:45.990441+0000","last_became_peered":"2024-06-10T00:52:45.990441+0000","last_unstale":"2024-06-10T00:54:22.553591+0000","last_undegraded":"2024-06-10T00:54:22.553591+0000","last_fullsized":"2024-06-10T00:54:22.553591+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:19:35.690378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404369+0000","last_change":"2024-06-10T00:52:32.753913+0000","last_active":"2024-06-10T00:54:24.404369+0000","last_peered":"2024-06-10T00:54:24.404369+0000","last_clean":"2024-06-10T00:54:24.404369+0000","last_became_active":"2024-06-10T00:52:32.753174+0000","last_became_peered":"2024-06-10T00:52:32.753174+0000","last_unstale":"2024-06-10T00:54:24.404369+0000","last_undegraded":"2024-06-10T00:54:24.404369+0000","last_fullsized":"2024-06-10T00:54:24.404369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:13:20.048635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553967+0000","last_change":"2024-06-10T00:52:35.965783+0000","last_active":"2024-06-10T00:54:22.553967+0000","last_peered":"2024-06-10T00:54:22.553967+0000","last_clean":"2024-06-10T00:54:22.553967+0000","last_became_active":"2024-06-10T00:52:35.965432+0000","last_became_peered":"2024-06-10T00:52:35.965432+0000","last_unstale":"2024-06-10T00:54:22.553967+0000","last_undegraded":"2024-06-10T00:54:22.553967+0000","last_fullsized":"2024-06-10T00:54:22.553967+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:52:01.240318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553790+0000","last_change":"2024-06-10T00:52:46.241662+0000","last_active":"2024-06-10T00:54:22.553790+0000","last_peered":"2024-06-10T00:54:22.553790+0000","last_clean":"2024-06-10T00:54:22.553790+0000","last_became_active":"2024-06-10T00:52:46.241211+0000","last_became_peered":"2024-06-10T00:52:46.241211+0000","last_unstale":"2024-06-10T00:54:22.553790+0000","last_undegraded":"2024-06-10T00:54:22.553790+0000","last_fullsized":"2024-06-10T00:54:22.553790+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:12.041290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.555853+0000","last_change":"2024-06-10T00:53:22.200590+0000","last_active":"2024-06-10T00:54:22.555853+0000","last_peered":"2024-06-10T00:54:22.555853+0000","last_clean":"2024-06-10T00:54:22.555853+0000","last_became_active":"2024-06-10T00:53:22.200046+0000","last_became_peered":"2024-06-10T00:53:22.200046+0000","last_unstale":"2024-06-10T00:54:22.555853+0000","last_undegraded":"2024-06-10T00:54:22.555853+0000","last_fullsized":"2024-06-10T00:54:22.555853+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:43:22.378316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"98'4","reported_seq":145,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:16.438519+0000","last_change":"2024-06-10T00:52:32.959904+0000","last_active":"2024-06-10T00:55:16.438519+0000","last_peered":"2024-06-10T00:55:16.438519+0000","last_clean":"2024-06-10T00:55:16.438519+0000","last_became_active":"2024-06-10T00:52:32.959506+0000","last_became_peered":"2024-06-10T00:52:32.959506+0000","last_unstale":"2024-06-10T00:55:16.438519+0000","last_undegraded":"2024-06-10T00:55:16.438519+0000","last_fullsized":"2024-06-10T00:55:16.438519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:23:15.950345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404433+0000","last_change":"2024-06-10T00:52:45.993305+0000","last_active":"2024-06-10T00:54:24.404433+0000","last_peered":"2024-06-10T00:54:24.404433+0000","last_clean":"2024-06-10T00:54:24.404433+0000","last_became_active":"2024-06-10T00:52:45.992682+0000","last_became_peered":"2024-06-10T00:52:45.992682+0000","last_unstale":"2024-06-10T00:54:24.404433+0000","last_undegraded":"2024-06-10T00:54:24.404433+0000","last_fullsized":"2024-06-10T00:54:24.404433+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:49:41.817126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601421+0000","last_change":"2024-06-10T00:52:35.965893+0000","last_active":"2024-06-10T00:54:22.601421+0000","last_peered":"2024-06-10T00:54:22.601421+0000","last_clean":"2024-06-10T00:54:22.601421+0000","last_became_active":"2024-06-10T00:52:35.965426+0000","last_became_peered":"2024-06-10T00:52:35.965426+0000","last_unstale":"2024-06-10T00:54:22.601421+0000","last_undegraded":"2024-06-10T00:54:22.601421+0000","last_fullsized":"2024-06-10T00:54:22.601421+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:29:19.640836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668002+0000","last_change":"2024-06-10T00:52:32.753631+0000","last_active":"2024-06-10T00:54:22.668002+0000","last_peered":"2024-06-10T00:54:22.668002+0000","last_clean":"2024-06-10T00:54:22.668002+0000","last_became_active":"2024-06-10T00:52:32.753260+0000","last_became_peered":"2024-06-10T00:52:32.753260+0000","last_unstale":"2024-06-10T00:54:22.668002+0000","last_undegraded":"2024-06-10T00:54:22.668002+0000","last_fullsized":"2024-06-10T00:54:22.668002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:12:55.932676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":368,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.403988+0000","last_change":"2024-06-10T00:51:09.267750+0000","last_active":"2024-06-10T00:54:24.403988+0000","last_peered":"2024-06-10T00:54:24.403988+0000","last_clean":"2024-06-10T00:54:24.403988+0000","last_became_active":"2024-06-10T00:51:09.267246+0000","last_became_peered":"2024-06-10T00:51:09.267246+0000","last_unstale":"2024-06-10T00:54:24.403988+0000","last_undegraded":"2024-06-10T00:54:24.403988+0000","last_fullsized":"2024-06-10T00:54:24.403988+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:47:44.965452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:47:44.965452+0000","last_clean_scrub_stamp":"2024-06-10T00:47:44.965452+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:21.100090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561655+0000","last_change":"2024-06-10T00:52:46.242152+0000","last_active":"2024-06-10T00:54:22.561655+0000","last_peered":"2024-06-10T00:54:22.561655+0000","last_clean":"2024-06-10T00:54:22.561655+0000","last_became_active":"2024-06-10T00:52:46.241542+0000","last_became_peered":"2024-06-10T00:52:46.241542+0000","last_unstale":"2024-06-10T00:54:22.561655+0000","last_undegraded":"2024-06-10T00:54:22.561655+0000","last_fullsized":"2024-06-10T00:54:22.561655+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:55:54.695465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553309+0000","last_change":"2024-06-10T00:52:35.963911+0000","last_active":"2024-06-10T00:54:22.553309+0000","last_peered":"2024-06-10T00:54:22.553309+0000","last_clean":"2024-06-10T00:54:22.553309+0000","last_became_active":"2024-06-10T00:52:35.963571+0000","last_became_peered":"2024-06-10T00:52:35.963571+0000","last_unstale":"2024-06-10T00:54:22.553309+0000","last_undegraded":"2024-06-10T00:54:22.553309+0000","last_fullsized":"2024-06-10T00:54:22.553309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:27:16.757428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":18,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.558463+0000","last_change":"2024-06-10T00:53:22.129035+0000","last_active":"2024-06-10T00:54:22.558463+0000","last_peered":"2024-06-10T00:54:22.558463+0000","last_clean":"2024-06-10T00:54:22.558463+0000","last_became_active":"2024-06-10T00:53:22.128561+0000","last_became_peered":"2024-06-10T00:53:22.128561+0000","last_unstale":"2024-06-10T00:54:22.558463+0000","last_undegraded":"2024-06-10T00:54:22.558463+0000","last_fullsized":"2024-06-10T00:54:22.558463+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:02:43.313118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.558735+0000","last_change":"2024-06-10T00:52:45.996907+0000","last_active":"2024-06-10T00:54:22.558735+0000","last_peered":"2024-06-10T00:54:22.558735+0000","last_clean":"2024-06-10T00:54:22.558735+0000","last_became_active":"2024-06-10T00:52:45.996554+0000","last_became_peered":"2024-06-10T00:52:45.996554+0000","last_unstale":"2024-06-10T00:54:22.558735+0000","last_undegraded":"2024-06-10T00:54:22.558735+0000","last_fullsized":"2024-06-10T00:54:22.558735+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:00:46.824071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556594+0000","last_change":"2024-06-10T00:52:32.960223+0000","last_active":"2024-06-10T00:54:22.556594+0000","last_peered":"2024-06-10T00:54:22.556594+0000","last_clean":"2024-06-10T00:54:22.556594+0000","last_became_active":"2024-06-10T00:52:32.959871+0000","last_became_peered":"2024-06-10T00:52:32.959871+0000","last_unstale":"2024-06-10T00:54:22.556594+0000","last_undegraded":"2024-06-10T00:54:22.556594+0000","last_fullsized":"2024-06-10T00:54:22.556594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:18:17.099140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553610+0000","last_change":"2024-06-10T00:52:35.962516+0000","last_active":"2024-06-10T00:54:22.553610+0000","last_peered":"2024-06-10T00:54:22.553610+0000","last_clean":"2024-06-10T00:54:22.553610+0000","last_became_active":"2024-06-10T00:52:35.962187+0000","last_became_peered":"2024-06-10T00:52:35.962187+0000","last_unstale":"2024-06-10T00:54:22.553610+0000","last_undegraded":"2024-06-10T00:54:22.553610+0000","last_fullsized":"2024-06-10T00:54:22.553610+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:10:44.295192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601233+0000","last_change":"2024-06-10T00:52:46.241554+0000","last_active":"2024-06-10T00:54:22.601233+0000","last_peered":"2024-06-10T00:54:22.601233+0000","last_clean":"2024-06-10T00:54:22.601233+0000","last_became_active":"2024-06-10T00:52:46.241142+0000","last_became_peered":"2024-06-10T00:52:46.241142+0000","last_unstale":"2024-06-10T00:54:22.601233+0000","last_undegraded":"2024-06-10T00:54:22.601233+0000","last_fullsized":"2024-06-10T00:54:22.601233+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:58:07.233080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"98'10","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556481+0000","last_change":"2024-06-10T00:52:32.761854+0000","last_active":"2024-06-10T00:54:22.556481+0000","last_peered":"2024-06-10T00:54:22.556481+0000","last_clean":"2024-06-10T00:54:22.556481+0000","last_became_active":"2024-06-10T00:52:32.761522+0000","last_became_peered":"2024-06-10T00:52:32.761522+0000","last_unstale":"2024-06-10T00:54:22.556481+0000","last_undegraded":"2024-06-10T00:54:22.556481+0000","last_fullsized":"2024-06-10T00:54:22.556481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:06:43.269001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601795+0000","last_change":"2024-06-10T00:52:35.959250+0000","last_active":"2024-06-10T00:54:22.601795+0000","last_peered":"2024-06-10T00:54:22.601795+0000","last_clean":"2024-06-10T00:54:22.601795+0000","last_became_active":"2024-06-10T00:52:35.958925+0000","last_became_peered":"2024-06-10T00:52:35.958925+0000","last_unstale":"2024-06-10T00:54:22.601795+0000","last_undegraded":"2024-06-10T00:54:22.601795+0000","last_fullsized":"2024-06-10T00:54:22.601795+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:11:12.355505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554191+0000","last_change":"2024-06-10T00:52:45.991819+0000","last_active":"2024-06-10T00:54:22.554191+0000","last_peered":"2024-06-10T00:54:22.554191+0000","last_clean":"2024-06-10T00:54:22.554191+0000","last_became_active":"2024-06-10T00:52:45.991408+0000","last_became_peered":"2024-06-10T00:52:45.991408+0000","last_unstale":"2024-06-10T00:54:22.554191+0000","last_undegraded":"2024-06-10T00:54:22.554191+0000","last_fullsized":"2024-06-10T00:54:22.554191+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:16:34.912359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668551+0000","last_change":"2024-06-10T00:52:32.869094+0000","last_active":"2024-06-10T00:54:22.668551+0000","last_peered":"2024-06-10T00:54:22.668551+0000","last_clean":"2024-06-10T00:54:22.668551+0000","last_became_active":"2024-06-10T00:52:32.868664+0000","last_became_peered":"2024-06-10T00:52:32.868664+0000","last_unstale":"2024-06-10T00:54:22.668551+0000","last_undegraded":"2024-06-10T00:54:22.668551+0000","last_fullsized":"2024-06-10T00:54:22.668551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:50:38.051090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878659+0000","last_change":"2024-06-10T00:52:35.952356+0000","last_active":"2024-06-10T00:54:22.878659+0000","last_peered":"2024-06-10T00:54:22.878659+0000","last_clean":"2024-06-10T00:54:22.878659+0000","last_became_active":"2024-06-10T00:52:35.952002+0000","last_became_peered":"2024-06-10T00:52:35.952002+0000","last_unstale":"2024-06-10T00:54:22.878659+0000","last_undegraded":"2024-06-10T00:54:22.878659+0000","last_fullsized":"2024-06-10T00:54:22.878659+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:12:28.581522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668366+0000","last_change":"2024-06-10T00:52:46.241773+0000","last_active":"2024-06-10T00:54:22.668366+0000","last_peered":"2024-06-10T00:54:22.668366+0000","last_clean":"2024-06-10T00:54:22.668366+0000","last_became_active":"2024-06-10T00:52:46.241272+0000","last_became_peered":"2024-06-10T00:52:46.241272+0000","last_unstale":"2024-06-10T00:54:22.668366+0000","last_undegraded":"2024-06-10T00:54:22.668366+0000","last_fullsized":"2024-06-10T00:54:22.668366+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:07:28.337536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561360+0000","last_change":"2024-06-10T00:52:32.960100+0000","last_active":"2024-06-10T00:54:22.561360+0000","last_peered":"2024-06-10T00:54:22.561360+0000","last_clean":"2024-06-10T00:54:22.561360+0000","last_became_active":"2024-06-10T00:52:32.959779+0000","last_became_peered":"2024-06-10T00:52:32.959779+0000","last_unstale":"2024-06-10T00:54:22.561360+0000","last_undegraded":"2024-06-10T00:54:22.561360+0000","last_fullsized":"2024-06-10T00:54:22.561360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:09.555753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556021+0000","last_change":"2024-06-10T00:52:35.955586+0000","last_active":"2024-06-10T00:54:22.556021+0000","last_peered":"2024-06-10T00:54:22.556021+0000","last_clean":"2024-06-10T00:54:22.556021+0000","last_became_active":"2024-06-10T00:52:35.955216+0000","last_became_peered":"2024-06-10T00:52:35.955216+0000","last_unstale":"2024-06-10T00:54:22.556021+0000","last_undegraded":"2024-06-10T00:54:22.556021+0000","last_fullsized":"2024-06-10T00:54:22.556021+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:23:12.197182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.603221+0000","last_change":"2024-06-10T00:54:22.602670+0000","last_active":"2024-06-10T00:54:22.603221+0000","last_peered":"2024-06-10T00:54:22.603221+0000","last_clean":"2024-06-10T00:54:22.603221+0000","last_became_active":"2024-06-10T00:54:22.602218+0000","last_became_peered":"2024-06-10T00:54:22.602218+0000","last_unstale":"2024-06-10T00:54:22.603221+0000","last_undegraded":"2024-06-10T00:54:22.603221+0000","last_fullsized":"2024-06-10T00:54:22.603221+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:07.399025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"98'3","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601358+0000","last_change":"2024-06-10T00:52:32.753444+0000","last_active":"2024-06-10T00:54:22.601358+0000","last_peered":"2024-06-10T00:54:22.601358+0000","last_clean":"2024-06-10T00:54:22.601358+0000","last_became_active":"2024-06-10T00:52:32.753149+0000","last_became_peered":"2024-06-10T00:52:32.753149+0000","last_unstale":"2024-06-10T00:54:22.601358+0000","last_undegraded":"2024-06-10T00:54:22.601358+0000","last_fullsized":"2024-06-10T00:54:22.601358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:28.241950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.559055+0000","last_change":"2024-06-10T00:52:35.966718+0000","last_active":"2024-06-10T00:54:22.559055+0000","last_peered":"2024-06-10T00:54:22.559055+0000","last_clean":"2024-06-10T00:54:22.559055+0000","last_became_active":"2024-06-10T00:52:35.965907+0000","last_became_peered":"2024-06-10T00:52:35.965907+0000","last_unstale":"2024-06-10T00:54:22.559055+0000","last_undegraded":"2024-06-10T00:54:22.559055+0000","last_fullsized":"2024-06-10T00:54:22.559055+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:21:34.261306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668092+0000","last_change":"2024-06-10T00:52:45.991034+0000","last_active":"2024-06-10T00:54:22.668092+0000","last_peered":"2024-06-10T00:54:22.668092+0000","last_clean":"2024-06-10T00:54:22.668092+0000","last_became_active":"2024-06-10T00:52:45.990546+0000","last_became_peered":"2024-06-10T00:52:45.990546+0000","last_unstale":"2024-06-10T00:54:22.668092+0000","last_undegraded":"2024-06-10T00:54:22.668092+0000","last_fullsized":"2024-06-10T00:54:22.668092+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:09:23.412922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'1","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554448+0000","last_change":"2024-06-10T00:52:32.764024+0000","last_active":"2024-06-10T00:54:22.554448+0000","last_peered":"2024-06-10T00:54:22.554448+0000","last_clean":"2024-06-10T00:54:22.554448+0000","last_became_active":"2024-06-10T00:52:32.763449+0000","last_became_peered":"2024-06-10T00:52:32.763449+0000","last_unstale":"2024-06-10T00:54:22.554448+0000","last_undegraded":"2024-06-10T00:54:22.554448+0000","last_fullsized":"2024-06-10T00:54:22.554448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:50:23.605873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601482+0000","last_change":"2024-06-10T00:52:35.955570+0000","last_active":"2024-06-10T00:54:22.601482+0000","last_peered":"2024-06-10T00:54:22.601482+0000","last_clean":"2024-06-10T00:54:22.601482+0000","last_became_active":"2024-06-10T00:52:35.955248+0000","last_became_peered":"2024-06-10T00:52:35.955248+0000","last_unstale":"2024-06-10T00:54:22.601482+0000","last_undegraded":"2024-06-10T00:54:22.601482+0000","last_fullsized":"2024-06-10T00:54:22.601482+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:26.124499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668028+0000","last_change":"2024-06-10T00:52:46.241776+0000","last_active":"2024-06-10T00:54:22.668028+0000","last_peered":"2024-06-10T00:54:22.668028+0000","last_clean":"2024-06-10T00:54:22.668028+0000","last_became_active":"2024-06-10T00:52:46.241276+0000","last_became_peered":"2024-06-10T00:52:46.241276+0000","last_unstale":"2024-06-10T00:54:22.668028+0000","last_undegraded":"2024-06-10T00:54:22.668028+0000","last_fullsized":"2024-06-10T00:54:22.668028+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:08.027938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":58,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556321+0000","last_change":"2024-06-10T00:53:22.204034+0000","last_active":"2024-06-10T00:54:22.556321+0000","last_peered":"2024-06-10T00:54:22.556321+0000","last_clean":"2024-06-10T00:54:22.556321+0000","last_became_active":"2024-06-10T00:53:22.199988+0000","last_became_peered":"2024-06-10T00:53:22.199988+0000","last_unstale":"2024-06-10T00:54:22.556321+0000","last_undegraded":"2024-06-10T00:54:22.556321+0000","last_fullsized":"2024-06-10T00:54:22.556321+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:39:54.012360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404417+0000","last_change":"2024-06-10T00:52:35.959182+0000","last_active":"2024-06-10T00:54:24.404417+0000","last_peered":"2024-06-10T00:54:24.404417+0000","last_clean":"2024-06-10T00:54:24.404417+0000","last_became_active":"2024-06-10T00:52:35.958828+0000","last_became_peered":"2024-06-10T00:52:35.958828+0000","last_unstale":"2024-06-10T00:54:24.404417+0000","last_undegraded":"2024-06-10T00:54:24.404417+0000","last_fullsized":"2024-06-10T00:54:24.404417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:29:58.857659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561603+0000","last_change":"2024-06-10T00:52:45.989739+0000","last_active":"2024-06-10T00:54:22.561603+0000","last_peered":"2024-06-10T00:54:22.561603+0000","last_clean":"2024-06-10T00:54:22.561603+0000","last_became_active":"2024-06-10T00:52:45.989279+0000","last_became_peered":"2024-06-10T00:52:45.989279+0000","last_unstale":"2024-06-10T00:54:22.561603+0000","last_undegraded":"2024-06-10T00:54:22.561603+0000","last_fullsized":"2024-06-10T00:54:22.561603+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:19:17.551947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554798+0000","last_change":"2024-06-10T00:52:32.959662+0000","last_active":"2024-06-10T00:54:22.554798+0000","last_peered":"2024-06-10T00:54:22.554798+0000","last_clean":"2024-06-10T00:54:22.554798+0000","last_became_active":"2024-06-10T00:52:32.959286+0000","last_became_peered":"2024-06-10T00:52:32.959286+0000","last_unstale":"2024-06-10T00:54:22.554798+0000","last_undegraded":"2024-06-10T00:54:22.554798+0000","last_fullsized":"2024-06-10T00:54:22.554798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:26:04.361567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404695+0000","last_change":"2024-06-10T00:52:35.960137+0000","last_active":"2024-06-10T00:54:24.404695+0000","last_peered":"2024-06-10T00:54:24.404695+0000","last_clean":"2024-06-10T00:54:24.404695+0000","last_became_active":"2024-06-10T00:52:35.959584+0000","last_became_peered":"2024-06-10T00:52:35.959584+0000","last_unstale":"2024-06-10T00:54:24.404695+0000","last_undegraded":"2024-06-10T00:54:24.404695+0000","last_fullsized":"2024-06-10T00:54:24.404695+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:23:38.702685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878137+0000","last_change":"2024-06-10T00:52:46.002268+0000","last_active":"2024-06-10T00:54:22.878137+0000","last_peered":"2024-06-10T00:54:22.878137+0000","last_clean":"2024-06-10T00:54:22.878137+0000","last_became_active":"2024-06-10T00:52:46.001769+0000","last_became_peered":"2024-06-10T00:52:46.001769+0000","last_unstale":"2024-06-10T00:54:22.878137+0000","last_undegraded":"2024-06-10T00:54:22.878137+0000","last_fullsized":"2024-06-10T00:54:22.878137+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:33:51.365951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561629+0000","last_change":"2024-06-10T00:52:32.757112+0000","last_active":"2024-06-10T00:54:22.561629+0000","last_peered":"2024-06-10T00:54:22.561629+0000","last_clean":"2024-06-10T00:54:22.561629+0000","last_became_active":"2024-06-10T00:52:32.755589+0000","last_became_peered":"2024-06-10T00:52:32.755589+0000","last_unstale":"2024-06-10T00:54:22.561629+0000","last_undegraded":"2024-06-10T00:54:22.561629+0000","last_fullsized":"2024-06-10T00:54:22.561629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:15:12.814489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404450+0000","last_change":"2024-06-10T00:52:35.955925+0000","last_active":"2024-06-10T00:54:24.404450+0000","last_peered":"2024-06-10T00:54:24.404450+0000","last_clean":"2024-06-10T00:54:24.404450+0000","last_became_active":"2024-06-10T00:52:35.955556+0000","last_became_peered":"2024-06-10T00:52:35.955556+0000","last_unstale":"2024-06-10T00:54:24.404450+0000","last_undegraded":"2024-06-10T00:54:24.404450+0000","last_fullsized":"2024-06-10T00:54:24.404450+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:42:18.324484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561041+0000","last_change":"2024-06-10T00:52:45.988810+0000","last_active":"2024-06-10T00:54:22.561041+0000","last_peered":"2024-06-10T00:54:22.561041+0000","last_clean":"2024-06-10T00:54:22.561041+0000","last_became_active":"2024-06-10T00:52:45.988041+0000","last_became_peered":"2024-06-10T00:52:45.988041+0000","last_unstale":"2024-06-10T00:54:22.561041+0000","last_undegraded":"2024-06-10T00:54:22.561041+0000","last_fullsized":"2024-06-10T00:54:22.561041+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:14:08.979802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553851+0000","last_change":"2024-06-10T00:52:32.765493+0000","last_active":"2024-06-10T00:54:22.553851+0000","last_peered":"2024-06-10T00:54:22.553851+0000","last_clean":"2024-06-10T00:54:22.553851+0000","last_became_active":"2024-06-10T00:52:32.765193+0000","last_became_peered":"2024-06-10T00:52:32.765193+0000","last_unstale":"2024-06-10T00:54:22.553851+0000","last_undegraded":"2024-06-10T00:54:22.553851+0000","last_fullsized":"2024-06-10T00:54:22.553851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:38:11.554673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404761+0000","last_change":"2024-06-10T00:52:35.960216+0000","last_active":"2024-06-10T00:54:24.404761+0000","last_peered":"2024-06-10T00:54:24.404761+0000","last_clean":"2024-06-10T00:54:24.404761+0000","last_became_active":"2024-06-10T00:52:35.959664+0000","last_became_peered":"2024-06-10T00:52:35.959664+0000","last_unstale":"2024-06-10T00:54:24.404761+0000","last_undegraded":"2024-06-10T00:54:24.404761+0000","last_fullsized":"2024-06-10T00:54:24.404761+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:41:58.086330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.558863+0000","last_change":"2024-06-10T00:52:35.967043+0000","last_active":"2024-06-10T00:54:22.558863+0000","last_peered":"2024-06-10T00:54:22.558863+0000","last_clean":"2024-06-10T00:54:22.558863+0000","last_became_active":"2024-06-10T00:52:35.966366+0000","last_became_peered":"2024-06-10T00:52:35.966366+0000","last_unstale":"2024-06-10T00:54:22.558863+0000","last_undegraded":"2024-06-10T00:54:22.558863+0000","last_fullsized":"2024-06-10T00:54:22.558863+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:27:06.047113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554483+0000","last_change":"2024-06-10T00:52:35.960517+0000","last_active":"2024-06-10T00:54:22.554483+0000","last_peered":"2024-06-10T00:54:22.554483+0000","last_clean":"2024-06-10T00:54:22.554483+0000","last_became_active":"2024-06-10T00:52:35.957895+0000","last_became_peered":"2024-06-10T00:52:35.957895+0000","last_unstale":"2024-06-10T00:54:22.554483+0000","last_undegraded":"2024-06-10T00:54:22.554483+0000","last_fullsized":"2024-06-10T00:54:22.554483+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:04:34.438070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554310+0000","last_change":"2024-06-10T00:52:35.962163+0000","last_active":"2024-06-10T00:54:22.554310+0000","last_peered":"2024-06-10T00:54:22.554310+0000","last_clean":"2024-06-10T00:54:22.554310+0000","last_became_active":"2024-06-10T00:52:35.961768+0000","last_became_peered":"2024-06-10T00:52:35.961768+0000","last_unstale":"2024-06-10T00:54:22.554310+0000","last_undegraded":"2024-06-10T00:54:22.554310+0000","last_fullsized":"2024-06-10T00:54:22.554310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:35.793995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878357+0000","last_change":"2024-06-10T00:53:22.130686+0000","last_active":"2024-06-10T00:54:22.878357+0000","last_peered":"2024-06-10T00:54:22.878357+0000","last_clean":"2024-06-10T00:54:22.878357+0000","last_became_active":"2024-06-10T00:53:22.130198+0000","last_became_peered":"2024-06-10T00:53:22.130198+0000","last_unstale":"2024-06-10T00:54:22.878357+0000","last_undegraded":"2024-06-10T00:54:22.878357+0000","last_fullsized":"2024-06-10T00:54:22.878357+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:50:26.773977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553686+0000","last_change":"2024-06-10T00:52:35.957255+0000","last_active":"2024-06-10T00:54:22.553686+0000","last_peered":"2024-06-10T00:54:22.553686+0000","last_clean":"2024-06-10T00:54:22.553686+0000","last_became_active":"2024-06-10T00:52:35.956923+0000","last_became_peered":"2024-06-10T00:52:35.956923+0000","last_unstale":"2024-06-10T00:54:22.553686+0000","last_undegraded":"2024-06-10T00:54:22.553686+0000","last_fullsized":"2024-06-10T00:54:22.553686+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:33:25.743503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.558986+0000","last_change":"2024-06-10T00:52:35.964258+0000","last_active":"2024-06-10T00:54:22.558986+0000","last_peered":"2024-06-10T00:54:22.558986+0000","last_clean":"2024-06-10T00:54:22.558986+0000","last_became_active":"2024-06-10T00:52:35.963862+0000","last_became_peered":"2024-06-10T00:52:35.963862+0000","last_unstale":"2024-06-10T00:54:22.558986+0000","last_undegraded":"2024-06-10T00:54:22.558986+0000","last_fullsized":"2024-06-10T00:54:22.558986+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:48:58.554411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554874+0000","last_change":"2024-06-10T00:52:35.965046+0000","last_active":"2024-06-10T00:54:22.554874+0000","last_peered":"2024-06-10T00:54:22.554874+0000","last_clean":"2024-06-10T00:54:22.554874+0000","last_became_active":"2024-06-10T00:52:35.964496+0000","last_became_peered":"2024-06-10T00:52:35.964496+0000","last_unstale":"2024-06-10T00:54:22.554874+0000","last_undegraded":"2024-06-10T00:54:22.554874+0000","last_fullsized":"2024-06-10T00:54:22.554874+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:09:42.979731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601886+0000","last_change":"2024-06-10T00:52:35.964287+0000","last_active":"2024-06-10T00:54:22.601886+0000","last_peered":"2024-06-10T00:54:22.601886+0000","last_clean":"2024-06-10T00:54:22.601886+0000","last_became_active":"2024-06-10T00:52:35.963920+0000","last_became_peered":"2024-06-10T00:52:35.963920+0000","last_unstale":"2024-06-10T00:54:22.601886+0000","last_undegraded":"2024-06-10T00:54:22.601886+0000","last_fullsized":"2024-06-10T00:54:22.601886+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:41:54.593274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.562011+0000","last_change":"2024-06-10T00:52:35.961440+0000","last_active":"2024-06-10T00:54:22.562011+0000","last_peered":"2024-06-10T00:54:22.562011+0000","last_clean":"2024-06-10T00:54:22.562011+0000","last_became_active":"2024-06-10T00:52:35.960066+0000","last_became_peered":"2024-06-10T00:52:35.960066+0000","last_unstale":"2024-06-10T00:54:22.562011+0000","last_undegraded":"2024-06-10T00:54:22.562011+0000","last_fullsized":"2024-06-10T00:54:22.562011+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:01.024596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601396+0000","last_change":"2024-06-10T00:52:35.964464+0000","last_active":"2024-06-10T00:54:22.601396+0000","last_peered":"2024-06-10T00:54:22.601396+0000","last_clean":"2024-06-10T00:54:22.601396+0000","last_became_active":"2024-06-10T00:52:35.964159+0000","last_became_peered":"2024-06-10T00:52:35.964159+0000","last_unstale":"2024-06-10T00:54:22.601396+0000","last_undegraded":"2024-06-10T00:54:22.601396+0000","last_fullsized":"2024-06-10T00:54:22.601396+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:18.660859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561660+0000","last_change":"2024-06-10T00:52:35.961539+0000","last_active":"2024-06-10T00:54:22.561660+0000","last_peered":"2024-06-10T00:54:22.561660+0000","last_clean":"2024-06-10T00:54:22.561660+0000","last_became_active":"2024-06-10T00:52:35.958738+0000","last_became_peered":"2024-06-10T00:52:35.958738+0000","last_unstale":"2024-06-10T00:54:22.561660+0000","last_undegraded":"2024-06-10T00:54:22.561660+0000","last_fullsized":"2024-06-10T00:54:22.561660+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:06:48.057551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553534+0000","last_change":"2024-06-10T00:52:35.962563+0000","last_active":"2024-06-10T00:54:22.553534+0000","last_peered":"2024-06-10T00:54:22.553534+0000","last_clean":"2024-06-10T00:54:22.553534+0000","last_became_active":"2024-06-10T00:52:35.962255+0000","last_became_peered":"2024-06-10T00:52:35.962255+0000","last_unstale":"2024-06-10T00:54:22.553534+0000","last_undegraded":"2024-06-10T00:54:22.553534+0000","last_fullsized":"2024-06-10T00:54:22.553534+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:33:35.689446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.559600+0000","last_change":"2024-06-10T00:52:35.959616+0000","last_active":"2024-06-10T00:54:22.559600+0000","last_peered":"2024-06-10T00:54:22.559600+0000","last_clean":"2024-06-10T00:54:22.559600+0000","last_became_active":"2024-06-10T00:52:35.959251+0000","last_became_peered":"2024-06-10T00:52:35.959251+0000","last_unstale":"2024-06-10T00:54:22.559600+0000","last_undegraded":"2024-06-10T00:54:22.559600+0000","last_fullsized":"2024-06-10T00:54:22.559600+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:39:23.933279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284806,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74092,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":604,"num_read_kb":381,"num_write":171,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":222276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351025,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1101824,"data_stored":573101,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612662,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28240,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716912,"statfs":{"total":95995035648,"available":95966117888,"internally_reserved":0,"allocated":1253376,"data_stored":1230461,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776187,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1122304,"data_stored":577433,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037824,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1118208,"data_stored":573257,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299462,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1167360,"data_stored":1169165,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462986,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1097728,"data_stored":568513,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1171456,"data_stored":638515,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":569901,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411423,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32336,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712816,"statfs":{"total":95995035648,"available":95961923584,"internally_reserved":0,"allocated":1187840,"data_stored":640143,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149785,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1110016,"data_stored":568707,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888148,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1118208,"data_stored":573275,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724623,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1175552,"data_stored":1164753,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":70092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T00:55:21.298 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-10T00:55:21.591 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:55:21.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:21 smithi167 ceph-mon[36917]: pgmap v390: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:55:21.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:21 smithi167 ceph-mon[36917]: from='client.24740 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:55:21.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:21 smithi167 ceph-mon[36917]: mgrmap e18: x(active, since 10m), standbys: y, z 2024-06-10T00:55:21.854 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:21 smithi037 ceph-mon[32192]: pgmap v390: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:55:21.854 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:21 smithi037 ceph-mon[32192]: from='client.24740 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:55:21.854 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:21 smithi037 ceph-mon[32192]: mgrmap e18: x(active, since 10m), standbys: y, z 2024-06-10T00:55:21.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:21 smithi094 ceph-mon[36708]: pgmap v390: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:55:21.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:21 smithi094 ceph-mon[36708]: from='client.24740 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:55:21.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:21 smithi094 ceph-mon[36708]: mgrmap e18: x(active, since 10m), standbys: y, z 2024-06-10T00:55:23.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.430+0000 7f3281d58700 1 -- 172.21.15.37:0/525649887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f327c101390 msgr2=0x7f327c101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.430+0000 7f3281d58700 1 --2- 172.21.15.37:0/525649887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f327c101390 0x7f327c101810 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3270009990 tx=0x7f3270009ca0 comp rx=0 tx=0).stop 2024-06-10T00:55:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.430+0000 7f3281d58700 1 -- 172.21.15.37:0/525649887 shutdown_connections 2024-06-10T00:55:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.430+0000 7f3281d58700 1 --2- 172.21.15.37:0/525649887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f327c101d50 0x7f327c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.430+0000 7f3281d58700 1 --2- 172.21.15.37:0/525649887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f327c107400 0x7f327c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.430+0000 7f3281d58700 1 --2- 172.21.15.37:0/525649887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f327c101390 0x7f327c101810 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.430+0000 7f3281d58700 1 -- 172.21.15.37:0/525649887 >> 172.21.15.37:0/525649887 conn(0x7f327c078340 msgr2=0x7f327c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.430+0000 7f3281d58700 1 -- 172.21.15.37:0/525649887 shutdown_connections 2024-06-10T00:55:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.430+0000 7f3281d58700 1 -- 172.21.15.37:0/525649887 wait complete. 2024-06-10T00:55:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.431+0000 7f3281d58700 1 Processor -- start 2024-06-10T00:55:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.431+0000 7f3281d58700 1 -- start start 2024-06-10T00:55:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.432+0000 7f3281d58700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f327c101390 0x7f327c103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.432+0000 7f3281d58700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f327c101d50 0x7f327c1035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:23.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.432+0000 7f3281d58700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f327c107400 0x7f327c103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:23.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.432+0000 7f3281d58700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f327c111670 con 0x7f327c107400 2024-06-10T00:55:23.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.432+0000 7f3281d58700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f327c1114f0 con 0x7f327c101d50 2024-06-10T00:55:23.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.432+0000 7f3281d58700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f327c1117f0 con 0x7f327c101390 2024-06-10T00:55:23.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.432+0000 7f327bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f327c107400 0x7f327c103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:23.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.432+0000 7f327bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f327c107400 0x7f327c103af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34170/0 (socket says 172.21.15.37:34170) 2024-06-10T00:55:23.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.432+0000 7f327bfff700 1 -- 172.21.15.37:0/3543226256 learned_addr learned my addr 172.21.15.37:0/3543226256 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:23.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.432+0000 7f327b7fe700 1 --2- 172.21.15.37:0/3543226256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f327c101390 0x7f327c103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:23.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.432+0000 7f327affd700 1 --2- 172.21.15.37:0/3543226256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f327c101d50 0x7f327c1035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:23.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.432+0000 7f327bfff700 1 -- 172.21.15.37:0/3543226256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f327c101390 msgr2=0x7f327c103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:23.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.432+0000 7f327bfff700 1 --2- 172.21.15.37:0/3543226256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f327c101390 0x7f327c103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:23.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.432+0000 7f327bfff700 1 -- 172.21.15.37:0/3543226256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f327c101d50 msgr2=0x7f327c1035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:23.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.432+0000 7f327bfff700 1 --2- 172.21.15.37:0/3543226256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f327c101d50 0x7f327c1035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:23.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.432+0000 7f327bfff700 1 -- 172.21.15.37:0/3543226256 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f327c1a7130 con 0x7f327c107400 2024-06-10T00:55:23.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.432+0000 7f327affd700 1 --2- 172.21.15.37:0/3543226256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f327c101d50 0x7f327c1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:55:23.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.432+0000 7f327bfff700 1 --2- 172.21.15.37:0/3543226256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f327c107400 0x7f327c103af0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f327400ada0 tx=0x7f327400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:23.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.433+0000 7f3278ff9700 1 -- 172.21.15.37:0/3543226256 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3274017020 con 0x7f327c107400 2024-06-10T00:55:23.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.433+0000 7f3278ff9700 1 -- 172.21.15.37:0/3543226256 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f327400a0d0 con 0x7f327c107400 2024-06-10T00:55:23.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.433+0000 7f3281d58700 1 -- 172.21.15.37:0/3543226256 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f327c1a7420 con 0x7f327c107400 2024-06-10T00:55:23.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.433+0000 7f3278ff9700 1 -- 172.21.15.37:0/3543226256 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3274005480 con 0x7f327c107400 2024-06-10T00:55:23.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.433+0000 7f3281d58700 1 -- 172.21.15.37:0/3543226256 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f327c1a7910 con 0x7f327c107400 2024-06-10T00:55:23.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.434+0000 7f3281d58700 1 -- 172.21.15.37:0/3543226256 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f327c069180 con 0x7f327c107400 2024-06-10T00:55:23.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.435+0000 7f3278ff9700 1 -- 172.21.15.37:0/3543226256 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3274005620 con 0x7f327c107400 2024-06-10T00:55:23.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.436+0000 7f3278ff9700 1 --2- 172.21.15.37:0/3543226256 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f32640a2c20 0x7f32640a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:23.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.436+0000 7f327b7fe700 1 --2- 172.21.15.37:0/3543226256 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f32640a2c20 0x7f32640a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:23.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.436+0000 7f3278ff9700 1 -- 172.21.15.37:0/3543226256 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f32740cf1f0 con 0x7f327c107400 2024-06-10T00:55:23.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.436+0000 7f327b7fe700 1 --2- 172.21.15.37:0/3543226256 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f32640a2c20 0x7f32640a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f326c005950 tx=0x7f326c00a300 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:23.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.439+0000 7f3278ff9700 1 -- 172.21.15.37:0/3543226256 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3274021020 con 0x7f327c107400 2024-06-10T00:55:23.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.643+0000 7f3281d58700 1 -- 172.21.15.37:0/3543226256 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f327c1051b0 con 0x7f32640a2c20 2024-06-10T00:55:23.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.653+0000 7f3278ff9700 1 -- 172.21.15.37:0/3543226256 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340883 (secure 0 0 0) 0x7f327c1051b0 con 0x7f32640a2c20 2024-06-10T00:55:23.658 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:55:23.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.656+0000 7f3281d58700 1 -- 172.21.15.37:0/3543226256 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f32640a2c20 msgr2=0x7f32640a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:23.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.656+0000 7f3281d58700 1 --2- 172.21.15.37:0/3543226256 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f32640a2c20 0x7f32640a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f326c005950 tx=0x7f326c00a300 comp rx=0 tx=0).stop 2024-06-10T00:55:23.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.656+0000 7f3281d58700 1 -- 172.21.15.37:0/3543226256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f327c107400 msgr2=0x7f327c103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:23.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.656+0000 7f3281d58700 1 --2- 172.21.15.37:0/3543226256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f327c107400 0x7f327c103af0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f327400ada0 tx=0x7f327400c330 comp rx=0 tx=0).stop 2024-06-10T00:55:23.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.656+0000 7f3281d58700 1 -- 172.21.15.37:0/3543226256 shutdown_connections 2024-06-10T00:55:23.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.656+0000 7f3281d58700 1 --2- 172.21.15.37:0/3543226256 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f32640a2c20 0x7f32640a50e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:23.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.656+0000 7f3281d58700 1 --2- 172.21.15.37:0/3543226256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f327c107400 0x7f327c103af0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:23.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.656+0000 7f3281d58700 1 --2- 172.21.15.37:0/3543226256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f327c101390 0x7f327c103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:23.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.656+0000 7f3281d58700 1 --2- 172.21.15.37:0/3543226256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f327c101d50 0x7f327c1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:23.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.656+0000 7f3281d58700 1 -- 172.21.15.37:0/3543226256 >> 172.21.15.37:0/3543226256 conn(0x7f327c078340 msgr2=0x7f327c100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:23.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.656+0000 7f3281d58700 1 -- 172.21.15.37:0/3543226256 shutdown_connections 2024-06-10T00:55:23.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:23.656+0000 7f3281d58700 1 -- 172.21.15.37:0/3543226256 wait complete. 2024-06-10T00:55:23.662 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-06-10T00:55:23.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:23 smithi167 ceph-mon[36917]: pgmap v391: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-10T00:55:23.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:23 smithi094 ceph-mon[36708]: pgmap v391: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-10T00:55:23.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:23 smithi037 ceph-mon[32192]: pgmap v391: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-10T00:55:24.467 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-06-10T00:55:22.388274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665084,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":752,"num_read_kb":502,"num_write":359,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":441,"ondisk_log_size":441,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370952,"kb_used_data":13428,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570872,"statfs":{"total":1151940427776,"available":1151560572928,"internally_reserved":0,"allocated":13750272,"data_stored":8847614,"data_compressed":497625,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":42,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003274"},"pg_stats":[{"pgid":"3.3f","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373740+0000","last_change":"2024-06-10T00:52:35.971067+0000","last_active":"2024-06-10T00:55:20.373740+0000","last_peered":"2024-06-10T00:55:20.373740+0000","last_clean":"2024-06-10T00:55:20.373740+0000","last_became_active":"2024-06-10T00:52:35.970292+0000","last_became_peered":"2024-06-10T00:52:35.970292+0000","last_unstale":"2024-06-10T00:55:20.373740+0000","last_undegraded":"2024-06-10T00:55:20.373740+0000","last_fullsized":"2024-06-10T00:55:20.373740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:57:37.287238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878547+0000","last_change":"2024-06-10T00:52:35.960421+0000","last_active":"2024-06-10T00:54:22.878547+0000","last_peered":"2024-06-10T00:54:22.878547+0000","last_clean":"2024-06-10T00:54:22.878547+0000","last_became_active":"2024-06-10T00:52:35.960070+0000","last_became_peered":"2024-06-10T00:52:35.960070+0000","last_unstale":"2024-06-10T00:54:22.878547+0000","last_undegraded":"2024-06-10T00:54:22.878547+0000","last_fullsized":"2024-06-10T00:54:22.878547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:48:02.616434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878431+0000","last_change":"2024-06-10T00:52:35.964778+0000","last_active":"2024-06-10T00:54:22.878431+0000","last_peered":"2024-06-10T00:54:22.878431+0000","last_clean":"2024-06-10T00:54:22.878431+0000","last_became_active":"2024-06-10T00:52:35.964463+0000","last_became_peered":"2024-06-10T00:52:35.964463+0000","last_unstale":"2024-06-10T00:54:22.878431+0000","last_undegraded":"2024-06-10T00:54:22.878431+0000","last_fullsized":"2024-06-10T00:54:22.878431+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:52:44.987363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668250+0000","last_change":"2024-06-10T00:52:35.963226+0000","last_active":"2024-06-10T00:54:22.668250+0000","last_peered":"2024-06-10T00:54:22.668250+0000","last_clean":"2024-06-10T00:54:22.668250+0000","last_became_active":"2024-06-10T00:52:35.962862+0000","last_became_peered":"2024-06-10T00:52:35.962862+0000","last_unstale":"2024-06-10T00:54:22.668250+0000","last_undegraded":"2024-06-10T00:54:22.668250+0000","last_fullsized":"2024-06-10T00:54:22.668250+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:08:56.418071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556100+0000","last_change":"2024-06-10T00:52:35.957427+0000","last_active":"2024-06-10T00:54:22.556100+0000","last_peered":"2024-06-10T00:54:22.556100+0000","last_clean":"2024-06-10T00:54:22.556100+0000","last_became_active":"2024-06-10T00:52:35.957049+0000","last_became_peered":"2024-06-10T00:52:35.957049+0000","last_unstale":"2024-06-10T00:54:22.556100+0000","last_undegraded":"2024-06-10T00:54:22.556100+0000","last_fullsized":"2024-06-10T00:54:22.556100+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:16:04.257210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.355809+0000","last_change":"2024-06-10T00:52:35.954251+0000","last_active":"2024-06-10T00:55:20.355809+0000","last_peered":"2024-06-10T00:55:20.355809+0000","last_clean":"2024-06-10T00:55:20.355809+0000","last_became_active":"2024-06-10T00:52:35.953908+0000","last_became_peered":"2024-06-10T00:52:35.953908+0000","last_unstale":"2024-06-10T00:55:20.355809+0000","last_undegraded":"2024-06-10T00:55:20.355809+0000","last_fullsized":"2024-06-10T00:55:20.355809+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:55:25.125836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404257+0000","last_change":"2024-06-10T00:52:35.956822+0000","last_active":"2024-06-10T00:54:24.404257+0000","last_peered":"2024-06-10T00:54:24.404257+0000","last_clean":"2024-06-10T00:54:24.404257+0000","last_became_active":"2024-06-10T00:52:35.956466+0000","last_became_peered":"2024-06-10T00:52:35.956466+0000","last_unstale":"2024-06-10T00:54:24.404257+0000","last_undegraded":"2024-06-10T00:54:24.404257+0000","last_fullsized":"2024-06-10T00:54:24.404257+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:36:41.534392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601335+0000","last_change":"2024-06-10T00:52:35.962978+0000","last_active":"2024-06-10T00:54:22.601335+0000","last_peered":"2024-06-10T00:54:22.601335+0000","last_clean":"2024-06-10T00:54:22.601335+0000","last_became_active":"2024-06-10T00:52:35.962609+0000","last_became_peered":"2024-06-10T00:52:35.962609+0000","last_unstale":"2024-06-10T00:54:22.601335+0000","last_undegraded":"2024-06-10T00:54:22.601335+0000","last_fullsized":"2024-06-10T00:54:22.601335+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:20:57.469577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.562007+0000","last_change":"2024-06-10T00:52:35.963203+0000","last_active":"2024-06-10T00:54:22.562007+0000","last_peered":"2024-06-10T00:54:22.562007+0000","last_clean":"2024-06-10T00:54:22.562007+0000","last_became_active":"2024-06-10T00:52:35.962789+0000","last_became_peered":"2024-06-10T00:52:35.962789+0000","last_unstale":"2024-06-10T00:54:22.562007+0000","last_undegraded":"2024-06-10T00:54:22.562007+0000","last_fullsized":"2024-06-10T00:54:22.562007+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:00:17.819193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"100'16","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.356376+0000","last_change":"2024-06-10T00:52:35.966940+0000","last_active":"2024-06-10T00:55:20.356376+0000","last_peered":"2024-06-10T00:55:20.356376+0000","last_clean":"2024-06-10T00:55:20.356376+0000","last_became_active":"2024-06-10T00:52:35.966554+0000","last_became_peered":"2024-06-10T00:52:35.966554+0000","last_unstale":"2024-06-10T00:55:20.356376+0000","last_undegraded":"2024-06-10T00:55:20.356376+0000","last_fullsized":"2024-06-10T00:55:20.356376+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:00.374437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404575+0000","last_change":"2024-06-10T00:52:35.960927+0000","last_active":"2024-06-10T00:54:24.404575+0000","last_peered":"2024-06-10T00:54:24.404575+0000","last_clean":"2024-06-10T00:54:24.404575+0000","last_became_active":"2024-06-10T00:52:35.960577+0000","last_became_peered":"2024-06-10T00:52:35.960577+0000","last_unstale":"2024-06-10T00:54:24.404575+0000","last_undegraded":"2024-06-10T00:54:24.404575+0000","last_fullsized":"2024-06-10T00:54:24.404575+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:50:02.366695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668213+0000","last_change":"2024-06-10T00:52:35.964435+0000","last_active":"2024-06-10T00:54:22.668213+0000","last_peered":"2024-06-10T00:54:22.668213+0000","last_clean":"2024-06-10T00:54:22.668213+0000","last_became_active":"2024-06-10T00:52:35.964069+0000","last_became_peered":"2024-06-10T00:52:35.964069+0000","last_unstale":"2024-06-10T00:54:22.668213+0000","last_undegraded":"2024-06-10T00:54:22.668213+0000","last_fullsized":"2024-06-10T00:54:22.668213+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:06.708551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553481+0000","last_change":"2024-06-10T00:52:35.966722+0000","last_active":"2024-06-10T00:54:22.553481+0000","last_peered":"2024-06-10T00:54:22.553481+0000","last_clean":"2024-06-10T00:54:22.553481+0000","last_became_active":"2024-06-10T00:52:35.966366+0000","last_became_peered":"2024-06-10T00:52:35.966366+0000","last_unstale":"2024-06-10T00:54:22.553481+0000","last_undegraded":"2024-06-10T00:54:22.553481+0000","last_fullsized":"2024-06-10T00:54:22.553481+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:57:46.021047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878151+0000","last_change":"2024-06-10T00:52:35.955307+0000","last_active":"2024-06-10T00:54:22.878151+0000","last_peered":"2024-06-10T00:54:22.878151+0000","last_clean":"2024-06-10T00:54:22.878151+0000","last_became_active":"2024-06-10T00:52:35.954140+0000","last_became_peered":"2024-06-10T00:52:35.954140+0000","last_unstale":"2024-06-10T00:54:22.878151+0000","last_undegraded":"2024-06-10T00:54:22.878151+0000","last_fullsized":"2024-06-10T00:54:22.878151+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:47:22.493598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878142+0000","last_change":"2024-06-10T00:52:35.960592+0000","last_active":"2024-06-10T00:54:22.878142+0000","last_peered":"2024-06-10T00:54:22.878142+0000","last_clean":"2024-06-10T00:54:22.878142+0000","last_became_active":"2024-06-10T00:52:35.960263+0000","last_became_peered":"2024-06-10T00:52:35.960263+0000","last_unstale":"2024-06-10T00:54:22.878142+0000","last_undegraded":"2024-06-10T00:54:22.878142+0000","last_fullsized":"2024-06-10T00:54:22.878142+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:47:11.658666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561312+0000","last_change":"2024-06-10T00:52:35.965796+0000","last_active":"2024-06-10T00:54:22.561312+0000","last_peered":"2024-06-10T00:54:22.561312+0000","last_clean":"2024-06-10T00:54:22.561312+0000","last_became_active":"2024-06-10T00:52:35.963965+0000","last_became_peered":"2024-06-10T00:52:35.963965+0000","last_unstale":"2024-06-10T00:54:22.561312+0000","last_undegraded":"2024-06-10T00:54:22.561312+0000","last_fullsized":"2024-06-10T00:54:22.561312+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:29:47.149852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374090+0000","last_change":"2024-06-10T00:52:35.970714+0000","last_active":"2024-06-10T00:55:20.374090+0000","last_peered":"2024-06-10T00:55:20.374090+0000","last_clean":"2024-06-10T00:55:20.374090+0000","last_became_active":"2024-06-10T00:52:35.970173+0000","last_became_peered":"2024-06-10T00:52:35.970173+0000","last_unstale":"2024-06-10T00:55:20.374090+0000","last_undegraded":"2024-06-10T00:55:20.374090+0000","last_fullsized":"2024-06-10T00:55:20.374090+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:33:28.183156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.370737+0000","last_change":"2024-06-10T00:52:35.963052+0000","last_active":"2024-06-10T00:55:20.370737+0000","last_peered":"2024-06-10T00:55:20.370737+0000","last_clean":"2024-06-10T00:55:20.370737+0000","last_became_active":"2024-06-10T00:52:35.962674+0000","last_became_peered":"2024-06-10T00:52:35.962674+0000","last_unstale":"2024-06-10T00:55:20.370737+0000","last_undegraded":"2024-06-10T00:55:20.370737+0000","last_fullsized":"2024-06-10T00:55:20.370737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:56:38.836854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554975+0000","last_change":"2024-06-10T00:53:22.130336+0000","last_active":"2024-06-10T00:54:22.554975+0000","last_peered":"2024-06-10T00:54:22.554975+0000","last_clean":"2024-06-10T00:54:22.554975+0000","last_became_active":"2024-06-10T00:53:22.129910+0000","last_became_peered":"2024-06-10T00:53:22.129910+0000","last_unstale":"2024-06-10T00:54:22.554975+0000","last_undegraded":"2024-06-10T00:54:22.554975+0000","last_fullsized":"2024-06-10T00:54:22.554975+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:23:50.600821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554286+0000","last_change":"2024-06-10T00:52:45.997276+0000","last_active":"2024-06-10T00:54:22.554286+0000","last_peered":"2024-06-10T00:54:22.554286+0000","last_clean":"2024-06-10T00:54:22.554286+0000","last_became_active":"2024-06-10T00:52:45.995638+0000","last_became_peered":"2024-06-10T00:52:45.995638+0000","last_unstale":"2024-06-10T00:54:22.554286+0000","last_undegraded":"2024-06-10T00:54:22.554286+0000","last_fullsized":"2024-06-10T00:54:22.554286+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:14:20.922348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556499+0000","last_change":"2024-06-10T00:52:32.760888+0000","last_active":"2024-06-10T00:54:22.556499+0000","last_peered":"2024-06-10T00:54:22.556499+0000","last_clean":"2024-06-10T00:54:22.556499+0000","last_became_active":"2024-06-10T00:52:32.760531+0000","last_became_peered":"2024-06-10T00:52:32.760531+0000","last_unstale":"2024-06-10T00:54:22.556499+0000","last_undegraded":"2024-06-10T00:54:22.556499+0000","last_fullsized":"2024-06-10T00:54:22.556499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:05:59.052251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601475+0000","last_change":"2024-06-10T00:52:35.967626+0000","last_active":"2024-06-10T00:54:22.601475+0000","last_peered":"2024-06-10T00:54:22.601475+0000","last_clean":"2024-06-10T00:54:22.601475+0000","last_became_active":"2024-06-10T00:52:35.967295+0000","last_became_peered":"2024-06-10T00:52:35.967295+0000","last_unstale":"2024-06-10T00:54:22.601475+0000","last_undegraded":"2024-06-10T00:54:22.601475+0000","last_fullsized":"2024-06-10T00:54:22.601475+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:07:55.054327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668105+0000","last_change":"2024-06-10T00:52:45.993974+0000","last_active":"2024-06-10T00:54:22.668105+0000","last_peered":"2024-06-10T00:54:22.668105+0000","last_clean":"2024-06-10T00:54:22.668105+0000","last_became_active":"2024-06-10T00:52:45.993242+0000","last_became_peered":"2024-06-10T00:52:45.993242+0000","last_unstale":"2024-06-10T00:54:22.668105+0000","last_undegraded":"2024-06-10T00:54:22.668105+0000","last_fullsized":"2024-06-10T00:54:22.668105+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:28:12.810368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601807+0000","last_change":"2024-06-10T00:52:32.959905+0000","last_active":"2024-06-10T00:54:22.601807+0000","last_peered":"2024-06-10T00:54:22.601807+0000","last_clean":"2024-06-10T00:54:22.601807+0000","last_became_active":"2024-06-10T00:52:32.959551+0000","last_became_peered":"2024-06-10T00:52:32.959551+0000","last_unstale":"2024-06-10T00:54:22.601807+0000","last_undegraded":"2024-06-10T00:54:22.601807+0000","last_fullsized":"2024-06-10T00:54:22.601807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:33:48.029129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878378+0000","last_change":"2024-06-10T00:52:35.965790+0000","last_active":"2024-06-10T00:54:22.878378+0000","last_peered":"2024-06-10T00:54:22.878378+0000","last_clean":"2024-06-10T00:54:22.878378+0000","last_became_active":"2024-06-10T00:52:35.965475+0000","last_became_peered":"2024-06-10T00:52:35.965475+0000","last_unstale":"2024-06-10T00:54:22.878378+0000","last_undegraded":"2024-06-10T00:54:22.878378+0000","last_fullsized":"2024-06-10T00:54:22.878378+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:24:02.894411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.667953+0000","last_change":"2024-06-10T00:52:45.994464+0000","last_active":"2024-06-10T00:54:22.667953+0000","last_peered":"2024-06-10T00:54:22.667953+0000","last_clean":"2024-06-10T00:54:22.667953+0000","last_became_active":"2024-06-10T00:52:45.994022+0000","last_became_peered":"2024-06-10T00:52:45.994022+0000","last_unstale":"2024-06-10T00:54:22.667953+0000","last_undegraded":"2024-06-10T00:54:22.667953+0000","last_fullsized":"2024-06-10T00:54:22.667953+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:27:25.063074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556658+0000","last_change":"2024-06-10T00:52:32.763885+0000","last_active":"2024-06-10T00:54:22.556658+0000","last_peered":"2024-06-10T00:54:22.556658+0000","last_clean":"2024-06-10T00:54:22.556658+0000","last_became_active":"2024-06-10T00:52:32.763606+0000","last_became_peered":"2024-06-10T00:52:32.763606+0000","last_unstale":"2024-06-10T00:54:22.556658+0000","last_undegraded":"2024-06-10T00:54:22.556658+0000","last_fullsized":"2024-06-10T00:54:22.556658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:01:58.161168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561645+0000","last_change":"2024-06-10T00:52:35.966948+0000","last_active":"2024-06-10T00:54:22.561645+0000","last_peered":"2024-06-10T00:54:22.561645+0000","last_clean":"2024-06-10T00:54:22.561645+0000","last_became_active":"2024-06-10T00:52:35.966345+0000","last_became_peered":"2024-06-10T00:52:35.966345+0000","last_unstale":"2024-06-10T00:54:22.561645+0000","last_undegraded":"2024-06-10T00:54:22.561645+0000","last_fullsized":"2024-06-10T00:54:22.561645+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:58:27.169201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.667762+0000","last_change":"2024-06-10T00:52:45.993645+0000","last_active":"2024-06-10T00:54:22.667762+0000","last_peered":"2024-06-10T00:54:22.667762+0000","last_clean":"2024-06-10T00:54:22.667762+0000","last_became_active":"2024-06-10T00:52:45.993183+0000","last_became_peered":"2024-06-10T00:52:45.993183+0000","last_unstale":"2024-06-10T00:54:22.667762+0000","last_undegraded":"2024-06-10T00:54:22.667762+0000","last_fullsized":"2024-06-10T00:54:22.667762+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:47:00.541427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.565600+0000","last_change":"2024-06-10T00:54:22.565600+0000","last_active":"2024-06-10T00:54:22.565600+0000","last_peered":"2024-06-10T00:54:22.565600+0000","last_clean":"2024-06-10T00:54:22.565600+0000","last_became_active":"2024-06-10T00:54:22.562561+0000","last_became_peered":"2024-06-10T00:54:22.562561+0000","last_unstale":"2024-06-10T00:54:22.565600+0000","last_undegraded":"2024-06-10T00:54:22.565600+0000","last_fullsized":"2024-06-10T00:54:22.565600+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:47:58.748050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"100'15","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365162+0000","last_change":"2024-06-10T00:53:22.199668+0000","last_active":"2024-06-10T00:55:20.365162+0000","last_peered":"2024-06-10T00:55:20.365162+0000","last_clean":"2024-06-10T00:55:20.365162+0000","last_became_active":"2024-06-10T00:53:22.199329+0000","last_became_peered":"2024-06-10T00:53:22.199329+0000","last_unstale":"2024-06-10T00:55:20.365162+0000","last_undegraded":"2024-06-10T00:55:20.365162+0000","last_fullsized":"2024-06-10T00:55:20.365162+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:37.640478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554689+0000","last_change":"2024-06-10T00:52:45.995767+0000","last_active":"2024-06-10T00:54:22.554689+0000","last_peered":"2024-06-10T00:54:22.554689+0000","last_clean":"2024-06-10T00:54:22.554689+0000","last_became_active":"2024-06-10T00:52:45.994659+0000","last_became_peered":"2024-06-10T00:52:45.994659+0000","last_unstale":"2024-06-10T00:54:22.554689+0000","last_undegraded":"2024-06-10T00:54:22.554689+0000","last_fullsized":"2024-06-10T00:54:22.554689+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:03:45.652248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'7","reported_seq":155,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.445849+0000","last_change":"2024-06-10T00:52:32.761637+0000","last_active":"2024-06-10T00:55:20.445849+0000","last_peered":"2024-06-10T00:55:20.445849+0000","last_clean":"2024-06-10T00:55:20.445849+0000","last_became_active":"2024-06-10T00:52:32.761144+0000","last_became_peered":"2024-06-10T00:52:32.761144+0000","last_unstale":"2024-06-10T00:55:20.445849+0000","last_undegraded":"2024-06-10T00:55:20.445849+0000","last_fullsized":"2024-06-10T00:55:20.445849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:55:05.616564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.557749+0000","last_change":"2024-06-10T00:52:35.962683+0000","last_active":"2024-06-10T00:54:22.557749+0000","last_peered":"2024-06-10T00:54:22.557749+0000","last_clean":"2024-06-10T00:54:22.557749+0000","last_became_active":"2024-06-10T00:52:35.962327+0000","last_became_peered":"2024-06-10T00:52:35.962327+0000","last_unstale":"2024-06-10T00:54:22.557749+0000","last_undegraded":"2024-06-10T00:54:22.557749+0000","last_fullsized":"2024-06-10T00:54:22.557749+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:38:19.063306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554563+0000","last_change":"2024-06-10T00:52:45.996882+0000","last_active":"2024-06-10T00:54:22.554563+0000","last_peered":"2024-06-10T00:54:22.554563+0000","last_clean":"2024-06-10T00:54:22.554563+0000","last_became_active":"2024-06-10T00:52:45.995767+0000","last_became_peered":"2024-06-10T00:52:45.995767+0000","last_unstale":"2024-06-10T00:54:22.554563+0000","last_undegraded":"2024-06-10T00:54:22.554563+0000","last_fullsized":"2024-06-10T00:54:22.554563+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:28:03.766944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553970+0000","last_change":"2024-06-10T00:53:22.200546+0000","last_active":"2024-06-10T00:54:22.553970+0000","last_peered":"2024-06-10T00:54:22.553970+0000","last_clean":"2024-06-10T00:54:22.553970+0000","last_became_active":"2024-06-10T00:53:22.200059+0000","last_became_peered":"2024-06-10T00:53:22.200059+0000","last_unstale":"2024-06-10T00:54:22.553970+0000","last_undegraded":"2024-06-10T00:54:22.553970+0000","last_fullsized":"2024-06-10T00:54:22.553970+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:08:47.682594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.356418+0000","last_change":"2024-06-10T00:52:35.963105+0000","last_active":"2024-06-10T00:55:20.356418+0000","last_peered":"2024-06-10T00:55:20.356418+0000","last_clean":"2024-06-10T00:55:20.356418+0000","last_became_active":"2024-06-10T00:52:35.962627+0000","last_became_peered":"2024-06-10T00:52:35.962627+0000","last_unstale":"2024-06-10T00:55:20.356418+0000","last_undegraded":"2024-06-10T00:55:20.356418+0000","last_fullsized":"2024-06-10T00:55:20.356418+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:55:51.231795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.559190+0000","last_change":"2024-06-10T00:52:45.992557+0000","last_active":"2024-06-10T00:54:22.559190+0000","last_peered":"2024-06-10T00:54:22.559190+0000","last_clean":"2024-06-10T00:54:22.559190+0000","last_became_active":"2024-06-10T00:52:45.992166+0000","last_became_peered":"2024-06-10T00:52:45.992166+0000","last_unstale":"2024-06-10T00:54:22.559190+0000","last_undegraded":"2024-06-10T00:54:22.559190+0000","last_fullsized":"2024-06-10T00:54:22.559190+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:49:22.368409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553428+0000","last_change":"2024-06-10T00:52:32.751867+0000","last_active":"2024-06-10T00:54:22.553428+0000","last_peered":"2024-06-10T00:54:22.553428+0000","last_clean":"2024-06-10T00:54:22.553428+0000","last_became_active":"2024-06-10T00:52:32.751516+0000","last_became_peered":"2024-06-10T00:52:32.751516+0000","last_unstale":"2024-06-10T00:54:22.553428+0000","last_undegraded":"2024-06-10T00:54:22.553428+0000","last_fullsized":"2024-06-10T00:54:22.553428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:37:55.920613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601243+0000","last_change":"2024-06-10T00:52:35.957772+0000","last_active":"2024-06-10T00:54:22.601243+0000","last_peered":"2024-06-10T00:54:22.601243+0000","last_clean":"2024-06-10T00:54:22.601243+0000","last_became_active":"2024-06-10T00:52:35.957470+0000","last_became_peered":"2024-06-10T00:52:35.957470+0000","last_unstale":"2024-06-10T00:54:22.601243+0000","last_undegraded":"2024-06-10T00:54:22.601243+0000","last_fullsized":"2024-06-10T00:54:22.601243+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:16:43.645292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878568+0000","last_change":"2024-06-10T00:52:45.996807+0000","last_active":"2024-06-10T00:54:22.878568+0000","last_peered":"2024-06-10T00:54:22.878568+0000","last_clean":"2024-06-10T00:54:22.878568+0000","last_became_active":"2024-06-10T00:52:45.996378+0000","last_became_peered":"2024-06-10T00:52:45.996378+0000","last_unstale":"2024-06-10T00:54:22.878568+0000","last_undegraded":"2024-06-10T00:54:22.878568+0000","last_fullsized":"2024-06-10T00:54:22.878568+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:40:54.251790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'1","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561873+0000","last_change":"2024-06-10T00:52:32.759625+0000","last_active":"2024-06-10T00:54:22.561873+0000","last_peered":"2024-06-10T00:54:22.561873+0000","last_clean":"2024-06-10T00:54:22.561873+0000","last_became_active":"2024-06-10T00:52:32.759299+0000","last_became_peered":"2024-06-10T00:52:32.759299+0000","last_unstale":"2024-06-10T00:54:22.561873+0000","last_undegraded":"2024-06-10T00:54:22.561873+0000","last_fullsized":"2024-06-10T00:54:22.561873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:11:44.966453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365673+0000","last_change":"2024-06-10T00:52:35.957672+0000","last_active":"2024-06-10T00:55:20.365673+0000","last_peered":"2024-06-10T00:55:20.365673+0000","last_clean":"2024-06-10T00:55:20.365673+0000","last_became_active":"2024-06-10T00:52:35.957328+0000","last_became_peered":"2024-06-10T00:52:35.957328+0000","last_unstale":"2024-06-10T00:55:20.365673+0000","last_undegraded":"2024-06-10T00:55:20.365673+0000","last_fullsized":"2024-06-10T00:55:20.365673+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:43:59.658178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601278+0000","last_change":"2024-06-10T00:52:45.996677+0000","last_active":"2024-06-10T00:54:22.601278+0000","last_peered":"2024-06-10T00:54:22.601278+0000","last_clean":"2024-06-10T00:54:22.601278+0000","last_became_active":"2024-06-10T00:52:45.994483+0000","last_became_peered":"2024-06-10T00:52:45.994483+0000","last_unstale":"2024-06-10T00:54:22.601278+0000","last_undegraded":"2024-06-10T00:54:22.601278+0000","last_fullsized":"2024-06-10T00:54:22.601278+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:03:28.712072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'4","reported_seq":149,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.322365+0000","last_change":"2024-06-10T00:52:32.754829+0000","last_active":"2024-06-10T00:55:20.322365+0000","last_peered":"2024-06-10T00:55:20.322365+0000","last_clean":"2024-06-10T00:55:20.322365+0000","last_became_active":"2024-06-10T00:52:32.754386+0000","last_became_peered":"2024-06-10T00:52:32.754386+0000","last_unstale":"2024-06-10T00:55:20.322365+0000","last_undegraded":"2024-06-10T00:55:20.322365+0000","last_fullsized":"2024-06-10T00:55:20.322365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:00.870324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.562006+0000","last_change":"2024-06-10T00:52:35.966871+0000","last_active":"2024-06-10T00:54:22.562006+0000","last_peered":"2024-06-10T00:54:22.562006+0000","last_clean":"2024-06-10T00:54:22.562006+0000","last_became_active":"2024-06-10T00:52:35.966492+0000","last_became_peered":"2024-06-10T00:52:35.966492+0000","last_unstale":"2024-06-10T00:54:22.562006+0000","last_undegraded":"2024-06-10T00:54:22.562006+0000","last_fullsized":"2024-06-10T00:54:22.562006+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:24:41.048935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.879950+0000","last_change":"2024-06-10T00:54:22.879457+0000","last_active":"2024-06-10T00:54:22.879950+0000","last_peered":"2024-06-10T00:54:22.879950+0000","last_clean":"2024-06-10T00:54:22.879950+0000","last_became_active":"2024-06-10T00:54:22.878978+0000","last_became_peered":"2024-06-10T00:54:22.878978+0000","last_unstale":"2024-06-10T00:54:22.879950+0000","last_undegraded":"2024-06-10T00:54:22.879950+0000","last_fullsized":"2024-06-10T00:54:22.879950+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:46:38.819250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404659+0000","last_change":"2024-06-10T00:52:32.765794+0000","last_active":"2024-06-10T00:54:24.404659+0000","last_peered":"2024-06-10T00:54:24.404659+0000","last_clean":"2024-06-10T00:54:24.404659+0000","last_became_active":"2024-06-10T00:52:32.765276+0000","last_became_peered":"2024-06-10T00:52:32.765276+0000","last_unstale":"2024-06-10T00:54:24.404659+0000","last_undegraded":"2024-06-10T00:54:24.404659+0000","last_fullsized":"2024-06-10T00:54:24.404659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:06:51.209012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878545+0000","last_change":"2024-06-10T00:52:35.954897+0000","last_active":"2024-06-10T00:54:22.878545+0000","last_peered":"2024-06-10T00:54:22.878545+0000","last_clean":"2024-06-10T00:54:22.878545+0000","last_became_active":"2024-06-10T00:52:35.954565+0000","last_became_peered":"2024-06-10T00:52:35.954565+0000","last_unstale":"2024-06-10T00:54:22.878545+0000","last_undegraded":"2024-06-10T00:54:22.878545+0000","last_fullsized":"2024-06-10T00:54:22.878545+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:39:26.702804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668190+0000","last_change":"2024-06-10T00:52:45.992194+0000","last_active":"2024-06-10T00:54:22.668190+0000","last_peered":"2024-06-10T00:54:22.668190+0000","last_clean":"2024-06-10T00:54:22.668190+0000","last_became_active":"2024-06-10T00:52:45.991453+0000","last_became_peered":"2024-06-10T00:52:45.991453+0000","last_unstale":"2024-06-10T00:54:22.668190+0000","last_undegraded":"2024-06-10T00:54:22.668190+0000","last_fullsized":"2024-06-10T00:54:22.668190+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:11:10.775610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561610+0000","last_change":"2024-06-10T00:52:32.962214+0000","last_active":"2024-06-10T00:54:22.561610+0000","last_peered":"2024-06-10T00:54:22.561610+0000","last_clean":"2024-06-10T00:54:22.561610+0000","last_became_active":"2024-06-10T00:52:32.961835+0000","last_became_peered":"2024-06-10T00:52:32.961835+0000","last_unstale":"2024-06-10T00:54:22.561610+0000","last_undegraded":"2024-06-10T00:54:22.561610+0000","last_fullsized":"2024-06-10T00:54:22.561610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:36:44.724122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561818+0000","last_change":"2024-06-10T00:52:35.963413+0000","last_active":"2024-06-10T00:54:22.561818+0000","last_peered":"2024-06-10T00:54:22.561818+0000","last_clean":"2024-06-10T00:54:22.561818+0000","last_became_active":"2024-06-10T00:52:35.962922+0000","last_became_peered":"2024-06-10T00:52:35.962922+0000","last_unstale":"2024-06-10T00:54:22.561818+0000","last_undegraded":"2024-06-10T00:54:22.561818+0000","last_fullsized":"2024-06-10T00:54:22.561818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:05:32.164696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":34,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.560580+0000","last_change":"2024-06-10T00:54:22.560196+0000","last_active":"2024-06-10T00:54:22.560580+0000","last_peered":"2024-06-10T00:54:22.560580+0000","last_clean":"2024-06-10T00:54:22.560580+0000","last_became_active":"2024-06-10T00:54:22.559826+0000","last_became_peered":"2024-06-10T00:54:22.559826+0000","last_unstale":"2024-06-10T00:54:22.560580+0000","last_undegraded":"2024-06-10T00:54:22.560580+0000","last_fullsized":"2024-06-10T00:54:22.560580+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:09:28.456394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'41","reported_seq":297,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.457551+0000","last_change":"2024-06-10T00:52:32.761592+0000","last_active":"2024-06-10T00:55:20.457551+0000","last_peered":"2024-06-10T00:55:20.457551+0000","last_clean":"2024-06-10T00:55:20.457551+0000","last_became_active":"2024-06-10T00:52:32.761169+0000","last_became_peered":"2024-06-10T00:52:32.761169+0000","last_unstale":"2024-06-10T00:55:20.457551+0000","last_undegraded":"2024-06-10T00:55:20.457551+0000","last_fullsized":"2024-06-10T00:55:20.457551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:08:33.015035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70341,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":211,"num_read_kb":176,"num_write":42,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556908+0000","last_change":"2024-06-10T00:52:35.955304+0000","last_active":"2024-06-10T00:54:22.556908+0000","last_peered":"2024-06-10T00:54:22.556908+0000","last_clean":"2024-06-10T00:54:22.556908+0000","last_became_active":"2024-06-10T00:52:35.954949+0000","last_became_peered":"2024-06-10T00:52:35.954949+0000","last_unstale":"2024-06-10T00:54:22.556908+0000","last_undegraded":"2024-06-10T00:54:22.556908+0000","last_fullsized":"2024-06-10T00:54:22.556908+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:56:42.890022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561827+0000","last_change":"2024-06-10T00:52:45.992974+0000","last_active":"2024-06-10T00:54:22.561827+0000","last_peered":"2024-06-10T00:54:22.561827+0000","last_clean":"2024-06-10T00:54:22.561827+0000","last_became_active":"2024-06-10T00:52:45.992490+0000","last_became_peered":"2024-06-10T00:52:45.992490+0000","last_unstale":"2024-06-10T00:54:22.561827+0000","last_undegraded":"2024-06-10T00:54:22.561827+0000","last_fullsized":"2024-06-10T00:54:22.561827+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:03:45.543476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554437+0000","last_change":"2024-06-10T00:52:32.762347+0000","last_active":"2024-06-10T00:54:22.554437+0000","last_peered":"2024-06-10T00:54:22.554437+0000","last_clean":"2024-06-10T00:54:22.554437+0000","last_became_active":"2024-06-10T00:52:32.761049+0000","last_became_peered":"2024-06-10T00:52:32.761049+0000","last_unstale":"2024-06-10T00:54:22.554437+0000","last_undegraded":"2024-06-10T00:54:22.554437+0000","last_fullsized":"2024-06-10T00:54:22.554437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:29:38.384451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553361+0000","last_change":"2024-06-10T00:52:35.961299+0000","last_active":"2024-06-10T00:54:22.553361+0000","last_peered":"2024-06-10T00:54:22.553361+0000","last_clean":"2024-06-10T00:54:22.553361+0000","last_became_active":"2024-06-10T00:52:35.960966+0000","last_became_peered":"2024-06-10T00:52:35.960966+0000","last_unstale":"2024-06-10T00:54:22.553361+0000","last_undegraded":"2024-06-10T00:54:22.553361+0000","last_fullsized":"2024-06-10T00:54:22.553361+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:32:21.269689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.559474+0000","last_change":"2024-06-10T00:52:46.002091+0000","last_active":"2024-06-10T00:54:22.559474+0000","last_peered":"2024-06-10T00:54:22.559474+0000","last_clean":"2024-06-10T00:54:22.559474+0000","last_became_active":"2024-06-10T00:52:46.001677+0000","last_became_peered":"2024-06-10T00:52:46.001677+0000","last_unstale":"2024-06-10T00:54:22.559474+0000","last_undegraded":"2024-06-10T00:54:22.559474+0000","last_fullsized":"2024-06-10T00:54:22.559474+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:16:56.405720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'3","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.562287+0000","last_change":"2024-06-10T00:52:32.756765+0000","last_active":"2024-06-10T00:54:22.562287+0000","last_peered":"2024-06-10T00:54:22.562287+0000","last_clean":"2024-06-10T00:54:22.562287+0000","last_became_active":"2024-06-10T00:52:32.756470+0000","last_became_peered":"2024-06-10T00:52:32.756470+0000","last_unstale":"2024-06-10T00:54:22.562287+0000","last_undegraded":"2024-06-10T00:54:22.562287+0000","last_fullsized":"2024-06-10T00:54:22.562287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:29:24.854913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404448+0000","last_change":"2024-06-10T00:52:35.960693+0000","last_active":"2024-06-10T00:54:24.404448+0000","last_peered":"2024-06-10T00:54:24.404448+0000","last_clean":"2024-06-10T00:54:24.404448+0000","last_became_active":"2024-06-10T00:52:35.958397+0000","last_became_peered":"2024-06-10T00:52:35.958397+0000","last_unstale":"2024-06-10T00:54:24.404448+0000","last_undegraded":"2024-06-10T00:54:24.404448+0000","last_fullsized":"2024-06-10T00:54:24.404448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:23:47.556462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556200+0000","last_change":"2024-06-10T00:52:46.001475+0000","last_active":"2024-06-10T00:54:22.556200+0000","last_peered":"2024-06-10T00:54:22.556200+0000","last_clean":"2024-06-10T00:54:22.556200+0000","last_became_active":"2024-06-10T00:52:46.000983+0000","last_became_peered":"2024-06-10T00:52:46.000983+0000","last_unstale":"2024-06-10T00:54:22.556200+0000","last_undegraded":"2024-06-10T00:54:22.556200+0000","last_fullsized":"2024-06-10T00:54:22.556200+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:23:56.285356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"98'3","reported_seq":146,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:16.438974+0000","last_change":"2024-06-10T00:52:32.756665+0000","last_active":"2024-06-10T00:55:16.438974+0000","last_peered":"2024-06-10T00:55:16.438974+0000","last_clean":"2024-06-10T00:55:16.438974+0000","last_became_active":"2024-06-10T00:52:32.756221+0000","last_became_peered":"2024-06-10T00:52:32.756221+0000","last_unstale":"2024-06-10T00:55:16.438974+0000","last_undegraded":"2024-06-10T00:55:16.438974+0000","last_fullsized":"2024-06-10T00:55:16.438974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:50:34.789063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404268+0000","last_change":"2024-06-10T00:53:22.125844+0000","last_active":"2024-06-10T00:54:24.404268+0000","last_peered":"2024-06-10T00:54:24.404268+0000","last_clean":"2024-06-10T00:54:24.404268+0000","last_became_active":"2024-06-10T00:53:22.125449+0000","last_became_peered":"2024-06-10T00:53:22.125449+0000","last_unstale":"2024-06-10T00:54:24.404268+0000","last_undegraded":"2024-06-10T00:54:24.404268+0000","last_fullsized":"2024-06-10T00:54:24.404268+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:08:42.936702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553904+0000","last_change":"2024-06-10T00:52:45.999300+0000","last_active":"2024-06-10T00:54:22.553904+0000","last_peered":"2024-06-10T00:54:22.553904+0000","last_clean":"2024-06-10T00:54:22.553904+0000","last_became_active":"2024-06-10T00:52:45.998860+0000","last_became_peered":"2024-06-10T00:52:45.998860+0000","last_unstale":"2024-06-10T00:54:22.553904+0000","last_undegraded":"2024-06-10T00:54:22.553904+0000","last_fullsized":"2024-06-10T00:54:22.553904+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:14:24.936870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":75,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.557182+0000","last_change":"2024-06-10T00:53:22.210710+0000","last_active":"2024-06-10T00:54:22.557182+0000","last_peered":"2024-06-10T00:54:22.557182+0000","last_clean":"2024-06-10T00:54:22.557182+0000","last_became_active":"2024-06-10T00:53:22.202461+0000","last_became_peered":"2024-06-10T00:53:22.202461+0000","last_unstale":"2024-06-10T00:54:22.557182+0000","last_undegraded":"2024-06-10T00:54:22.557182+0000","last_fullsized":"2024-06-10T00:54:22.557182+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:28:14.671776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364560+0000","last_change":"2024-06-10T00:52:35.966960+0000","last_active":"2024-06-10T00:55:20.364560+0000","last_peered":"2024-06-10T00:55:20.364560+0000","last_clean":"2024-06-10T00:55:20.364560+0000","last_became_active":"2024-06-10T00:52:35.966630+0000","last_became_peered":"2024-06-10T00:52:35.966630+0000","last_unstale":"2024-06-10T00:55:20.364560+0000","last_undegraded":"2024-06-10T00:55:20.364560+0000","last_fullsized":"2024-06-10T00:55:20.364560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:04.117405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561586+0000","last_change":"2024-06-10T00:52:45.992955+0000","last_active":"2024-06-10T00:54:22.561586+0000","last_peered":"2024-06-10T00:54:22.561586+0000","last_clean":"2024-06-10T00:54:22.561586+0000","last_became_active":"2024-06-10T00:52:45.992457+0000","last_became_peered":"2024-06-10T00:52:45.992457+0000","last_unstale":"2024-06-10T00:54:22.561586+0000","last_undegraded":"2024-06-10T00:54:22.561586+0000","last_fullsized":"2024-06-10T00:54:22.561586+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:13:49.491183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556768+0000","last_change":"2024-06-10T00:52:32.763794+0000","last_active":"2024-06-10T00:54:22.556768+0000","last_peered":"2024-06-10T00:54:22.556768+0000","last_clean":"2024-06-10T00:54:22.556768+0000","last_became_active":"2024-06-10T00:52:32.763406+0000","last_became_peered":"2024-06-10T00:52:32.763406+0000","last_unstale":"2024-06-10T00:54:22.556768+0000","last_undegraded":"2024-06-10T00:54:22.556768+0000","last_fullsized":"2024-06-10T00:54:22.556768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:50:33.580163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"100'5","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374120+0000","last_change":"2024-06-10T00:53:22.130891+0000","last_active":"2024-06-10T00:55:20.374120+0000","last_peered":"2024-06-10T00:55:20.374120+0000","last_clean":"2024-06-10T00:55:20.374120+0000","last_became_active":"2024-06-10T00:53:22.130508+0000","last_became_peered":"2024-06-10T00:53:22.130508+0000","last_unstale":"2024-06-10T00:55:20.374120+0000","last_undegraded":"2024-06-10T00:55:20.374120+0000","last_fullsized":"2024-06-10T00:55:20.374120+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:28:01.919825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":34,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.563757+0000","last_change":"2024-06-10T00:54:22.563272+0000","last_active":"2024-06-10T00:54:22.563757+0000","last_peered":"2024-06-10T00:54:22.563757+0000","last_clean":"2024-06-10T00:54:22.563757+0000","last_became_active":"2024-06-10T00:54:22.562767+0000","last_became_peered":"2024-06-10T00:54:22.562767+0000","last_unstale":"2024-06-10T00:54:22.563757+0000","last_undegraded":"2024-06-10T00:54:22.563757+0000","last_fullsized":"2024-06-10T00:54:22.563757+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:52:20.739801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556364+0000","last_change":"2024-06-10T00:52:32.960515+0000","last_active":"2024-06-10T00:54:22.556364+0000","last_peered":"2024-06-10T00:54:22.556364+0000","last_clean":"2024-06-10T00:54:22.556364+0000","last_became_active":"2024-06-10T00:52:32.960163+0000","last_became_peered":"2024-06-10T00:52:32.960163+0000","last_unstale":"2024-06-10T00:54:22.556364+0000","last_undegraded":"2024-06-10T00:54:22.556364+0000","last_fullsized":"2024-06-10T00:54:22.556364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:39:51.374998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.359152+0000","last_change":"2024-06-10T00:52:35.961884+0000","last_active":"2024-06-10T00:55:20.359152+0000","last_peered":"2024-06-10T00:55:20.359152+0000","last_clean":"2024-06-10T00:55:20.359152+0000","last_became_active":"2024-06-10T00:52:35.961497+0000","last_became_peered":"2024-06-10T00:52:35.961497+0000","last_unstale":"2024-06-10T00:55:20.359152+0000","last_undegraded":"2024-06-10T00:55:20.359152+0000","last_fullsized":"2024-06-10T00:55:20.359152+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:16:09.284184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553591+0000","last_change":"2024-06-10T00:52:45.990846+0000","last_active":"2024-06-10T00:54:22.553591+0000","last_peered":"2024-06-10T00:54:22.553591+0000","last_clean":"2024-06-10T00:54:22.553591+0000","last_became_active":"2024-06-10T00:52:45.990441+0000","last_became_peered":"2024-06-10T00:52:45.990441+0000","last_unstale":"2024-06-10T00:54:22.553591+0000","last_undegraded":"2024-06-10T00:54:22.553591+0000","last_fullsized":"2024-06-10T00:54:22.553591+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:19:35.690378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404369+0000","last_change":"2024-06-10T00:52:32.753913+0000","last_active":"2024-06-10T00:54:24.404369+0000","last_peered":"2024-06-10T00:54:24.404369+0000","last_clean":"2024-06-10T00:54:24.404369+0000","last_became_active":"2024-06-10T00:52:32.753174+0000","last_became_peered":"2024-06-10T00:52:32.753174+0000","last_unstale":"2024-06-10T00:54:24.404369+0000","last_undegraded":"2024-06-10T00:54:24.404369+0000","last_fullsized":"2024-06-10T00:54:24.404369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:13:20.048635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.359023+0000","last_change":"2024-06-10T00:52:35.965783+0000","last_active":"2024-06-10T00:55:20.359023+0000","last_peered":"2024-06-10T00:55:20.359023+0000","last_clean":"2024-06-10T00:55:20.359023+0000","last_became_active":"2024-06-10T00:52:35.965432+0000","last_became_peered":"2024-06-10T00:52:35.965432+0000","last_unstale":"2024-06-10T00:55:20.359023+0000","last_undegraded":"2024-06-10T00:55:20.359023+0000","last_fullsized":"2024-06-10T00:55:20.359023+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:52:01.240318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553790+0000","last_change":"2024-06-10T00:52:46.241662+0000","last_active":"2024-06-10T00:54:22.553790+0000","last_peered":"2024-06-10T00:54:22.553790+0000","last_clean":"2024-06-10T00:54:22.553790+0000","last_became_active":"2024-06-10T00:52:46.241211+0000","last_became_peered":"2024-06-10T00:52:46.241211+0000","last_unstale":"2024-06-10T00:54:22.553790+0000","last_undegraded":"2024-06-10T00:54:22.553790+0000","last_fullsized":"2024-06-10T00:54:22.553790+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:12.041290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"100'8","reported_seq":23,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364272+0000","last_change":"2024-06-10T00:53:22.200590+0000","last_active":"2024-06-10T00:55:20.364272+0000","last_peered":"2024-06-10T00:55:20.364272+0000","last_clean":"2024-06-10T00:55:20.364272+0000","last_became_active":"2024-06-10T00:53:22.200046+0000","last_became_peered":"2024-06-10T00:53:22.200046+0000","last_unstale":"2024-06-10T00:55:20.364272+0000","last_undegraded":"2024-06-10T00:55:20.364272+0000","last_fullsized":"2024-06-10T00:55:20.364272+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:43:22.378316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"98'4","reported_seq":145,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:16.438519+0000","last_change":"2024-06-10T00:52:32.959904+0000","last_active":"2024-06-10T00:55:16.438519+0000","last_peered":"2024-06-10T00:55:16.438519+0000","last_clean":"2024-06-10T00:55:16.438519+0000","last_became_active":"2024-06-10T00:52:32.959506+0000","last_became_peered":"2024-06-10T00:52:32.959506+0000","last_unstale":"2024-06-10T00:55:16.438519+0000","last_undegraded":"2024-06-10T00:55:16.438519+0000","last_fullsized":"2024-06-10T00:55:16.438519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:23:15.950345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404433+0000","last_change":"2024-06-10T00:52:45.993305+0000","last_active":"2024-06-10T00:54:24.404433+0000","last_peered":"2024-06-10T00:54:24.404433+0000","last_clean":"2024-06-10T00:54:24.404433+0000","last_became_active":"2024-06-10T00:52:45.992682+0000","last_became_peered":"2024-06-10T00:52:45.992682+0000","last_unstale":"2024-06-10T00:54:24.404433+0000","last_undegraded":"2024-06-10T00:54:24.404433+0000","last_fullsized":"2024-06-10T00:54:24.404433+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:49:41.817126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601421+0000","last_change":"2024-06-10T00:52:35.965893+0000","last_active":"2024-06-10T00:54:22.601421+0000","last_peered":"2024-06-10T00:54:22.601421+0000","last_clean":"2024-06-10T00:54:22.601421+0000","last_became_active":"2024-06-10T00:52:35.965426+0000","last_became_peered":"2024-06-10T00:52:35.965426+0000","last_unstale":"2024-06-10T00:54:22.601421+0000","last_undegraded":"2024-06-10T00:54:22.601421+0000","last_fullsized":"2024-06-10T00:54:22.601421+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:29:19.640836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668002+0000","last_change":"2024-06-10T00:52:32.753631+0000","last_active":"2024-06-10T00:54:22.668002+0000","last_peered":"2024-06-10T00:54:22.668002+0000","last_clean":"2024-06-10T00:54:22.668002+0000","last_became_active":"2024-06-10T00:52:32.753260+0000","last_became_peered":"2024-06-10T00:52:32.753260+0000","last_unstale":"2024-06-10T00:54:22.668002+0000","last_undegraded":"2024-06-10T00:54:22.668002+0000","last_fullsized":"2024-06-10T00:54:22.668002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:12:55.932676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":368,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.403988+0000","last_change":"2024-06-10T00:51:09.267750+0000","last_active":"2024-06-10T00:54:24.403988+0000","last_peered":"2024-06-10T00:54:24.403988+0000","last_clean":"2024-06-10T00:54:24.403988+0000","last_became_active":"2024-06-10T00:51:09.267246+0000","last_became_peered":"2024-06-10T00:51:09.267246+0000","last_unstale":"2024-06-10T00:54:24.403988+0000","last_undegraded":"2024-06-10T00:54:24.403988+0000","last_fullsized":"2024-06-10T00:54:24.403988+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:47:44.965452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:47:44.965452+0000","last_clean_scrub_stamp":"2024-06-10T00:47:44.965452+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:21.100090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561655+0000","last_change":"2024-06-10T00:52:46.242152+0000","last_active":"2024-06-10T00:54:22.561655+0000","last_peered":"2024-06-10T00:54:22.561655+0000","last_clean":"2024-06-10T00:54:22.561655+0000","last_became_active":"2024-06-10T00:52:46.241542+0000","last_became_peered":"2024-06-10T00:52:46.241542+0000","last_unstale":"2024-06-10T00:54:22.561655+0000","last_undegraded":"2024-06-10T00:54:22.561655+0000","last_fullsized":"2024-06-10T00:54:22.561655+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:55:54.695465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553309+0000","last_change":"2024-06-10T00:52:35.963911+0000","last_active":"2024-06-10T00:54:22.553309+0000","last_peered":"2024-06-10T00:54:22.553309+0000","last_clean":"2024-06-10T00:54:22.553309+0000","last_became_active":"2024-06-10T00:52:35.963571+0000","last_became_peered":"2024-06-10T00:52:35.963571+0000","last_unstale":"2024-06-10T00:54:22.553309+0000","last_undegraded":"2024-06-10T00:54:22.553309+0000","last_fullsized":"2024-06-10T00:54:22.553309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:27:16.757428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.558529+0000","last_change":"2024-06-10T00:53:22.129035+0000","last_active":"2024-06-10T00:54:22.558529+0000","last_peered":"2024-06-10T00:54:22.558529+0000","last_clean":"2024-06-10T00:54:22.558529+0000","last_became_active":"2024-06-10T00:53:22.128561+0000","last_became_peered":"2024-06-10T00:53:22.128561+0000","last_unstale":"2024-06-10T00:54:22.558529+0000","last_undegraded":"2024-06-10T00:54:22.558529+0000","last_fullsized":"2024-06-10T00:54:22.558529+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:02:43.313118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.558735+0000","last_change":"2024-06-10T00:52:45.996907+0000","last_active":"2024-06-10T00:54:22.558735+0000","last_peered":"2024-06-10T00:54:22.558735+0000","last_clean":"2024-06-10T00:54:22.558735+0000","last_became_active":"2024-06-10T00:52:45.996554+0000","last_became_peered":"2024-06-10T00:52:45.996554+0000","last_unstale":"2024-06-10T00:54:22.558735+0000","last_undegraded":"2024-06-10T00:54:22.558735+0000","last_fullsized":"2024-06-10T00:54:22.558735+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:00:46.824071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556594+0000","last_change":"2024-06-10T00:52:32.960223+0000","last_active":"2024-06-10T00:54:22.556594+0000","last_peered":"2024-06-10T00:54:22.556594+0000","last_clean":"2024-06-10T00:54:22.556594+0000","last_became_active":"2024-06-10T00:52:32.959871+0000","last_became_peered":"2024-06-10T00:52:32.959871+0000","last_unstale":"2024-06-10T00:54:22.556594+0000","last_undegraded":"2024-06-10T00:54:22.556594+0000","last_fullsized":"2024-06-10T00:54:22.556594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:18:17.099140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553610+0000","last_change":"2024-06-10T00:52:35.962516+0000","last_active":"2024-06-10T00:54:22.553610+0000","last_peered":"2024-06-10T00:54:22.553610+0000","last_clean":"2024-06-10T00:54:22.553610+0000","last_became_active":"2024-06-10T00:52:35.962187+0000","last_became_peered":"2024-06-10T00:52:35.962187+0000","last_unstale":"2024-06-10T00:54:22.553610+0000","last_undegraded":"2024-06-10T00:54:22.553610+0000","last_fullsized":"2024-06-10T00:54:22.553610+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:10:44.295192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601233+0000","last_change":"2024-06-10T00:52:46.241554+0000","last_active":"2024-06-10T00:54:22.601233+0000","last_peered":"2024-06-10T00:54:22.601233+0000","last_clean":"2024-06-10T00:54:22.601233+0000","last_became_active":"2024-06-10T00:52:46.241142+0000","last_became_peered":"2024-06-10T00:52:46.241142+0000","last_unstale":"2024-06-10T00:54:22.601233+0000","last_undegraded":"2024-06-10T00:54:22.601233+0000","last_fullsized":"2024-06-10T00:54:22.601233+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:58:07.233080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"98'10","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556481+0000","last_change":"2024-06-10T00:52:32.761854+0000","last_active":"2024-06-10T00:54:22.556481+0000","last_peered":"2024-06-10T00:54:22.556481+0000","last_clean":"2024-06-10T00:54:22.556481+0000","last_became_active":"2024-06-10T00:52:32.761522+0000","last_became_peered":"2024-06-10T00:52:32.761522+0000","last_unstale":"2024-06-10T00:54:22.556481+0000","last_undegraded":"2024-06-10T00:54:22.556481+0000","last_fullsized":"2024-06-10T00:54:22.556481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:06:43.269001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601795+0000","last_change":"2024-06-10T00:52:35.959250+0000","last_active":"2024-06-10T00:54:22.601795+0000","last_peered":"2024-06-10T00:54:22.601795+0000","last_clean":"2024-06-10T00:54:22.601795+0000","last_became_active":"2024-06-10T00:52:35.958925+0000","last_became_peered":"2024-06-10T00:52:35.958925+0000","last_unstale":"2024-06-10T00:54:22.601795+0000","last_undegraded":"2024-06-10T00:54:22.601795+0000","last_fullsized":"2024-06-10T00:54:22.601795+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:11:12.355505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554191+0000","last_change":"2024-06-10T00:52:45.991819+0000","last_active":"2024-06-10T00:54:22.554191+0000","last_peered":"2024-06-10T00:54:22.554191+0000","last_clean":"2024-06-10T00:54:22.554191+0000","last_became_active":"2024-06-10T00:52:45.991408+0000","last_became_peered":"2024-06-10T00:52:45.991408+0000","last_unstale":"2024-06-10T00:54:22.554191+0000","last_undegraded":"2024-06-10T00:54:22.554191+0000","last_fullsized":"2024-06-10T00:54:22.554191+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:16:34.912359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668551+0000","last_change":"2024-06-10T00:52:32.869094+0000","last_active":"2024-06-10T00:54:22.668551+0000","last_peered":"2024-06-10T00:54:22.668551+0000","last_clean":"2024-06-10T00:54:22.668551+0000","last_became_active":"2024-06-10T00:52:32.868664+0000","last_became_peered":"2024-06-10T00:52:32.868664+0000","last_unstale":"2024-06-10T00:54:22.668551+0000","last_undegraded":"2024-06-10T00:54:22.668551+0000","last_fullsized":"2024-06-10T00:54:22.668551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:50:38.051090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878659+0000","last_change":"2024-06-10T00:52:35.952356+0000","last_active":"2024-06-10T00:54:22.878659+0000","last_peered":"2024-06-10T00:54:22.878659+0000","last_clean":"2024-06-10T00:54:22.878659+0000","last_became_active":"2024-06-10T00:52:35.952002+0000","last_became_peered":"2024-06-10T00:52:35.952002+0000","last_unstale":"2024-06-10T00:54:22.878659+0000","last_undegraded":"2024-06-10T00:54:22.878659+0000","last_fullsized":"2024-06-10T00:54:22.878659+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:12:28.581522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668366+0000","last_change":"2024-06-10T00:52:46.241773+0000","last_active":"2024-06-10T00:54:22.668366+0000","last_peered":"2024-06-10T00:54:22.668366+0000","last_clean":"2024-06-10T00:54:22.668366+0000","last_became_active":"2024-06-10T00:52:46.241272+0000","last_became_peered":"2024-06-10T00:52:46.241272+0000","last_unstale":"2024-06-10T00:54:22.668366+0000","last_undegraded":"2024-06-10T00:54:22.668366+0000","last_fullsized":"2024-06-10T00:54:22.668366+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:07:28.337536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561360+0000","last_change":"2024-06-10T00:52:32.960100+0000","last_active":"2024-06-10T00:54:22.561360+0000","last_peered":"2024-06-10T00:54:22.561360+0000","last_clean":"2024-06-10T00:54:22.561360+0000","last_became_active":"2024-06-10T00:52:32.959779+0000","last_became_peered":"2024-06-10T00:52:32.959779+0000","last_unstale":"2024-06-10T00:54:22.561360+0000","last_undegraded":"2024-06-10T00:54:22.561360+0000","last_fullsized":"2024-06-10T00:54:22.561360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:09.555753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556021+0000","last_change":"2024-06-10T00:52:35.955586+0000","last_active":"2024-06-10T00:54:22.556021+0000","last_peered":"2024-06-10T00:54:22.556021+0000","last_clean":"2024-06-10T00:54:22.556021+0000","last_became_active":"2024-06-10T00:52:35.955216+0000","last_became_peered":"2024-06-10T00:52:35.955216+0000","last_unstale":"2024-06-10T00:54:22.556021+0000","last_undegraded":"2024-06-10T00:54:22.556021+0000","last_fullsized":"2024-06-10T00:54:22.556021+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:23:12.197182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.603221+0000","last_change":"2024-06-10T00:54:22.602670+0000","last_active":"2024-06-10T00:54:22.603221+0000","last_peered":"2024-06-10T00:54:22.603221+0000","last_clean":"2024-06-10T00:54:22.603221+0000","last_became_active":"2024-06-10T00:54:22.602218+0000","last_became_peered":"2024-06-10T00:54:22.602218+0000","last_unstale":"2024-06-10T00:54:22.603221+0000","last_undegraded":"2024-06-10T00:54:22.603221+0000","last_fullsized":"2024-06-10T00:54:22.603221+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:07.399025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"98'3","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601358+0000","last_change":"2024-06-10T00:52:32.753444+0000","last_active":"2024-06-10T00:54:22.601358+0000","last_peered":"2024-06-10T00:54:22.601358+0000","last_clean":"2024-06-10T00:54:22.601358+0000","last_became_active":"2024-06-10T00:52:32.753149+0000","last_became_peered":"2024-06-10T00:52:32.753149+0000","last_unstale":"2024-06-10T00:54:22.601358+0000","last_undegraded":"2024-06-10T00:54:22.601358+0000","last_fullsized":"2024-06-10T00:54:22.601358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:28.241950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365900+0000","last_change":"2024-06-10T00:52:35.966718+0000","last_active":"2024-06-10T00:55:20.365900+0000","last_peered":"2024-06-10T00:55:20.365900+0000","last_clean":"2024-06-10T00:55:20.365900+0000","last_became_active":"2024-06-10T00:52:35.965907+0000","last_became_peered":"2024-06-10T00:52:35.965907+0000","last_unstale":"2024-06-10T00:55:20.365900+0000","last_undegraded":"2024-06-10T00:55:20.365900+0000","last_fullsized":"2024-06-10T00:55:20.365900+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:21:34.261306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668092+0000","last_change":"2024-06-10T00:52:45.991034+0000","last_active":"2024-06-10T00:54:22.668092+0000","last_peered":"2024-06-10T00:54:22.668092+0000","last_clean":"2024-06-10T00:54:22.668092+0000","last_became_active":"2024-06-10T00:52:45.990546+0000","last_became_peered":"2024-06-10T00:52:45.990546+0000","last_unstale":"2024-06-10T00:54:22.668092+0000","last_undegraded":"2024-06-10T00:54:22.668092+0000","last_fullsized":"2024-06-10T00:54:22.668092+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:09:23.412922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'1","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554448+0000","last_change":"2024-06-10T00:52:32.764024+0000","last_active":"2024-06-10T00:54:22.554448+0000","last_peered":"2024-06-10T00:54:22.554448+0000","last_clean":"2024-06-10T00:54:22.554448+0000","last_became_active":"2024-06-10T00:52:32.763449+0000","last_became_peered":"2024-06-10T00:52:32.763449+0000","last_unstale":"2024-06-10T00:54:22.554448+0000","last_undegraded":"2024-06-10T00:54:22.554448+0000","last_fullsized":"2024-06-10T00:54:22.554448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:50:23.605873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601482+0000","last_change":"2024-06-10T00:52:35.955570+0000","last_active":"2024-06-10T00:54:22.601482+0000","last_peered":"2024-06-10T00:54:22.601482+0000","last_clean":"2024-06-10T00:54:22.601482+0000","last_became_active":"2024-06-10T00:52:35.955248+0000","last_became_peered":"2024-06-10T00:52:35.955248+0000","last_unstale":"2024-06-10T00:54:22.601482+0000","last_undegraded":"2024-06-10T00:54:22.601482+0000","last_fullsized":"2024-06-10T00:54:22.601482+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:26.124499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668028+0000","last_change":"2024-06-10T00:52:46.241776+0000","last_active":"2024-06-10T00:54:22.668028+0000","last_peered":"2024-06-10T00:54:22.668028+0000","last_clean":"2024-06-10T00:54:22.668028+0000","last_became_active":"2024-06-10T00:52:46.241276+0000","last_became_peered":"2024-06-10T00:52:46.241276+0000","last_unstale":"2024-06-10T00:54:22.668028+0000","last_undegraded":"2024-06-10T00:54:22.668028+0000","last_fullsized":"2024-06-10T00:54:22.668028+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:08.027938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":58,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556321+0000","last_change":"2024-06-10T00:53:22.204034+0000","last_active":"2024-06-10T00:54:22.556321+0000","last_peered":"2024-06-10T00:54:22.556321+0000","last_clean":"2024-06-10T00:54:22.556321+0000","last_became_active":"2024-06-10T00:53:22.199988+0000","last_became_peered":"2024-06-10T00:53:22.199988+0000","last_unstale":"2024-06-10T00:54:22.556321+0000","last_undegraded":"2024-06-10T00:54:22.556321+0000","last_fullsized":"2024-06-10T00:54:22.556321+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:39:54.012360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404417+0000","last_change":"2024-06-10T00:52:35.959182+0000","last_active":"2024-06-10T00:54:24.404417+0000","last_peered":"2024-06-10T00:54:24.404417+0000","last_clean":"2024-06-10T00:54:24.404417+0000","last_became_active":"2024-06-10T00:52:35.958828+0000","last_became_peered":"2024-06-10T00:52:35.958828+0000","last_unstale":"2024-06-10T00:54:24.404417+0000","last_undegraded":"2024-06-10T00:54:24.404417+0000","last_fullsized":"2024-06-10T00:54:24.404417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:29:58.857659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561603+0000","last_change":"2024-06-10T00:52:45.989739+0000","last_active":"2024-06-10T00:54:22.561603+0000","last_peered":"2024-06-10T00:54:22.561603+0000","last_clean":"2024-06-10T00:54:22.561603+0000","last_became_active":"2024-06-10T00:52:45.989279+0000","last_became_peered":"2024-06-10T00:52:45.989279+0000","last_unstale":"2024-06-10T00:54:22.561603+0000","last_undegraded":"2024-06-10T00:54:22.561603+0000","last_fullsized":"2024-06-10T00:54:22.561603+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:19:17.551947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554798+0000","last_change":"2024-06-10T00:52:32.959662+0000","last_active":"2024-06-10T00:54:22.554798+0000","last_peered":"2024-06-10T00:54:22.554798+0000","last_clean":"2024-06-10T00:54:22.554798+0000","last_became_active":"2024-06-10T00:52:32.959286+0000","last_became_peered":"2024-06-10T00:52:32.959286+0000","last_unstale":"2024-06-10T00:54:22.554798+0000","last_undegraded":"2024-06-10T00:54:22.554798+0000","last_fullsized":"2024-06-10T00:54:22.554798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:26:04.361567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404695+0000","last_change":"2024-06-10T00:52:35.960137+0000","last_active":"2024-06-10T00:54:24.404695+0000","last_peered":"2024-06-10T00:54:24.404695+0000","last_clean":"2024-06-10T00:54:24.404695+0000","last_became_active":"2024-06-10T00:52:35.959584+0000","last_became_peered":"2024-06-10T00:52:35.959584+0000","last_unstale":"2024-06-10T00:54:24.404695+0000","last_undegraded":"2024-06-10T00:54:24.404695+0000","last_fullsized":"2024-06-10T00:54:24.404695+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:23:38.702685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878137+0000","last_change":"2024-06-10T00:52:46.002268+0000","last_active":"2024-06-10T00:54:22.878137+0000","last_peered":"2024-06-10T00:54:22.878137+0000","last_clean":"2024-06-10T00:54:22.878137+0000","last_became_active":"2024-06-10T00:52:46.001769+0000","last_became_peered":"2024-06-10T00:52:46.001769+0000","last_unstale":"2024-06-10T00:54:22.878137+0000","last_undegraded":"2024-06-10T00:54:22.878137+0000","last_fullsized":"2024-06-10T00:54:22.878137+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:33:51.365951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561629+0000","last_change":"2024-06-10T00:52:32.757112+0000","last_active":"2024-06-10T00:54:22.561629+0000","last_peered":"2024-06-10T00:54:22.561629+0000","last_clean":"2024-06-10T00:54:22.561629+0000","last_became_active":"2024-06-10T00:52:32.755589+0000","last_became_peered":"2024-06-10T00:52:32.755589+0000","last_unstale":"2024-06-10T00:54:22.561629+0000","last_undegraded":"2024-06-10T00:54:22.561629+0000","last_fullsized":"2024-06-10T00:54:22.561629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:15:12.814489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404450+0000","last_change":"2024-06-10T00:52:35.955925+0000","last_active":"2024-06-10T00:54:24.404450+0000","last_peered":"2024-06-10T00:54:24.404450+0000","last_clean":"2024-06-10T00:54:24.404450+0000","last_became_active":"2024-06-10T00:52:35.955556+0000","last_became_peered":"2024-06-10T00:52:35.955556+0000","last_unstale":"2024-06-10T00:54:24.404450+0000","last_undegraded":"2024-06-10T00:54:24.404450+0000","last_fullsized":"2024-06-10T00:54:24.404450+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:42:18.324484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561041+0000","last_change":"2024-06-10T00:52:45.988810+0000","last_active":"2024-06-10T00:54:22.561041+0000","last_peered":"2024-06-10T00:54:22.561041+0000","last_clean":"2024-06-10T00:54:22.561041+0000","last_became_active":"2024-06-10T00:52:45.988041+0000","last_became_peered":"2024-06-10T00:52:45.988041+0000","last_unstale":"2024-06-10T00:54:22.561041+0000","last_undegraded":"2024-06-10T00:54:22.561041+0000","last_fullsized":"2024-06-10T00:54:22.561041+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:14:08.979802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553851+0000","last_change":"2024-06-10T00:52:32.765493+0000","last_active":"2024-06-10T00:54:22.553851+0000","last_peered":"2024-06-10T00:54:22.553851+0000","last_clean":"2024-06-10T00:54:22.553851+0000","last_became_active":"2024-06-10T00:52:32.765193+0000","last_became_peered":"2024-06-10T00:52:32.765193+0000","last_unstale":"2024-06-10T00:54:22.553851+0000","last_undegraded":"2024-06-10T00:54:22.553851+0000","last_fullsized":"2024-06-10T00:54:22.553851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:38:11.554673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404761+0000","last_change":"2024-06-10T00:52:35.960216+0000","last_active":"2024-06-10T00:54:24.404761+0000","last_peered":"2024-06-10T00:54:24.404761+0000","last_clean":"2024-06-10T00:54:24.404761+0000","last_became_active":"2024-06-10T00:52:35.959664+0000","last_became_peered":"2024-06-10T00:52:35.959664+0000","last_unstale":"2024-06-10T00:54:24.404761+0000","last_undegraded":"2024-06-10T00:54:24.404761+0000","last_fullsized":"2024-06-10T00:54:24.404761+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:41:58.086330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375612+0000","last_change":"2024-06-10T00:52:35.967043+0000","last_active":"2024-06-10T00:55:20.375612+0000","last_peered":"2024-06-10T00:55:20.375612+0000","last_clean":"2024-06-10T00:55:20.375612+0000","last_became_active":"2024-06-10T00:52:35.966366+0000","last_became_peered":"2024-06-10T00:52:35.966366+0000","last_unstale":"2024-06-10T00:55:20.375612+0000","last_undegraded":"2024-06-10T00:55:20.375612+0000","last_fullsized":"2024-06-10T00:55:20.375612+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:27:06.047113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554483+0000","last_change":"2024-06-10T00:52:35.960517+0000","last_active":"2024-06-10T00:54:22.554483+0000","last_peered":"2024-06-10T00:54:22.554483+0000","last_clean":"2024-06-10T00:54:22.554483+0000","last_became_active":"2024-06-10T00:52:35.957895+0000","last_became_peered":"2024-06-10T00:52:35.957895+0000","last_unstale":"2024-06-10T00:54:22.554483+0000","last_undegraded":"2024-06-10T00:54:22.554483+0000","last_fullsized":"2024-06-10T00:54:22.554483+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:04:34.438070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554310+0000","last_change":"2024-06-10T00:52:35.962163+0000","last_active":"2024-06-10T00:54:22.554310+0000","last_peered":"2024-06-10T00:54:22.554310+0000","last_clean":"2024-06-10T00:54:22.554310+0000","last_became_active":"2024-06-10T00:52:35.961768+0000","last_became_peered":"2024-06-10T00:52:35.961768+0000","last_unstale":"2024-06-10T00:54:22.554310+0000","last_undegraded":"2024-06-10T00:54:22.554310+0000","last_fullsized":"2024-06-10T00:54:22.554310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:35.793995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878357+0000","last_change":"2024-06-10T00:53:22.130686+0000","last_active":"2024-06-10T00:54:22.878357+0000","last_peered":"2024-06-10T00:54:22.878357+0000","last_clean":"2024-06-10T00:54:22.878357+0000","last_became_active":"2024-06-10T00:53:22.130198+0000","last_became_peered":"2024-06-10T00:53:22.130198+0000","last_unstale":"2024-06-10T00:54:22.878357+0000","last_undegraded":"2024-06-10T00:54:22.878357+0000","last_fullsized":"2024-06-10T00:54:22.878357+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:50:26.773977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553686+0000","last_change":"2024-06-10T00:52:35.957255+0000","last_active":"2024-06-10T00:54:22.553686+0000","last_peered":"2024-06-10T00:54:22.553686+0000","last_clean":"2024-06-10T00:54:22.553686+0000","last_became_active":"2024-06-10T00:52:35.956923+0000","last_became_peered":"2024-06-10T00:52:35.956923+0000","last_unstale":"2024-06-10T00:54:22.553686+0000","last_undegraded":"2024-06-10T00:54:22.553686+0000","last_fullsized":"2024-06-10T00:54:22.553686+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:33:25.743503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375063+0000","last_change":"2024-06-10T00:52:35.964258+0000","last_active":"2024-06-10T00:55:20.375063+0000","last_peered":"2024-06-10T00:55:20.375063+0000","last_clean":"2024-06-10T00:55:20.375063+0000","last_became_active":"2024-06-10T00:52:35.963862+0000","last_became_peered":"2024-06-10T00:52:35.963862+0000","last_unstale":"2024-06-10T00:55:20.375063+0000","last_undegraded":"2024-06-10T00:55:20.375063+0000","last_fullsized":"2024-06-10T00:55:20.375063+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:48:58.554411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554874+0000","last_change":"2024-06-10T00:52:35.965046+0000","last_active":"2024-06-10T00:54:22.554874+0000","last_peered":"2024-06-10T00:54:22.554874+0000","last_clean":"2024-06-10T00:54:22.554874+0000","last_became_active":"2024-06-10T00:52:35.964496+0000","last_became_peered":"2024-06-10T00:52:35.964496+0000","last_unstale":"2024-06-10T00:54:22.554874+0000","last_undegraded":"2024-06-10T00:54:22.554874+0000","last_fullsized":"2024-06-10T00:54:22.554874+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:09:42.979731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601886+0000","last_change":"2024-06-10T00:52:35.964287+0000","last_active":"2024-06-10T00:54:22.601886+0000","last_peered":"2024-06-10T00:54:22.601886+0000","last_clean":"2024-06-10T00:54:22.601886+0000","last_became_active":"2024-06-10T00:52:35.963920+0000","last_became_peered":"2024-06-10T00:52:35.963920+0000","last_unstale":"2024-06-10T00:54:22.601886+0000","last_undegraded":"2024-06-10T00:54:22.601886+0000","last_fullsized":"2024-06-10T00:54:22.601886+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:41:54.593274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.367606+0000","last_change":"2024-06-10T00:52:35.961440+0000","last_active":"2024-06-10T00:55:20.367606+0000","last_peered":"2024-06-10T00:55:20.367606+0000","last_clean":"2024-06-10T00:55:20.367606+0000","last_became_active":"2024-06-10T00:52:35.960066+0000","last_became_peered":"2024-06-10T00:52:35.960066+0000","last_unstale":"2024-06-10T00:55:20.367606+0000","last_undegraded":"2024-06-10T00:55:20.367606+0000","last_fullsized":"2024-06-10T00:55:20.367606+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:01.024596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601396+0000","last_change":"2024-06-10T00:52:35.964464+0000","last_active":"2024-06-10T00:54:22.601396+0000","last_peered":"2024-06-10T00:54:22.601396+0000","last_clean":"2024-06-10T00:54:22.601396+0000","last_became_active":"2024-06-10T00:52:35.964159+0000","last_became_peered":"2024-06-10T00:52:35.964159+0000","last_unstale":"2024-06-10T00:54:22.601396+0000","last_undegraded":"2024-06-10T00:54:22.601396+0000","last_fullsized":"2024-06-10T00:54:22.601396+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:18.660859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561660+0000","last_change":"2024-06-10T00:52:35.961539+0000","last_active":"2024-06-10T00:54:22.561660+0000","last_peered":"2024-06-10T00:54:22.561660+0000","last_clean":"2024-06-10T00:54:22.561660+0000","last_became_active":"2024-06-10T00:52:35.958738+0000","last_became_peered":"2024-06-10T00:52:35.958738+0000","last_unstale":"2024-06-10T00:54:22.561660+0000","last_undegraded":"2024-06-10T00:54:22.561660+0000","last_fullsized":"2024-06-10T00:54:22.561660+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:06:48.057551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553534+0000","last_change":"2024-06-10T00:52:35.962563+0000","last_active":"2024-06-10T00:54:22.553534+0000","last_peered":"2024-06-10T00:54:22.553534+0000","last_clean":"2024-06-10T00:54:22.553534+0000","last_became_active":"2024-06-10T00:52:35.962255+0000","last_became_peered":"2024-06-10T00:52:35.962255+0000","last_unstale":"2024-06-10T00:54:22.553534+0000","last_undegraded":"2024-06-10T00:54:22.553534+0000","last_fullsized":"2024-06-10T00:54:22.553534+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:33:35.689446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372437+0000","last_change":"2024-06-10T00:52:35.959616+0000","last_active":"2024-06-10T00:55:20.372437+0000","last_peered":"2024-06-10T00:55:20.372437+0000","last_clean":"2024-06-10T00:55:20.372437+0000","last_became_active":"2024-06-10T00:52:35.959251+0000","last_became_peered":"2024-06-10T00:52:35.959251+0000","last_unstale":"2024-06-10T00:55:20.372437+0000","last_undegraded":"2024-06-10T00:55:20.372437+0000","last_fullsized":"2024-06-10T00:55:20.372437+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:39:23.933279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284806,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74482,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":626,"num_read_kb":393,"num_write":176,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":446464,"data_stored":222666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351026,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":573101,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612663,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28244,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716908,"statfs":{"total":95995035648,"available":95966113792,"internally_reserved":0,"allocated":1257472,"data_stored":1230710,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776188,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1122304,"data_stored":577433,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037824,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1118208,"data_stored":573257,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299462,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1167360,"data_stored":1169165,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462986,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1097728,"data_stored":568513,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1171456,"data_stored":638515,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":569901,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411423,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32336,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712816,"statfs":{"total":95995035648,"available":95961923584,"internally_reserved":0,"allocated":1187840,"data_stored":640143,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149785,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1110016,"data_stored":568707,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888148,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1118208,"data_stored":573275,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724624,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":1164894,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":70092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":71919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T00:55:24.472 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-10T00:55:24.472 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-06-10T00:55:24.472 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-06-10T00:55:24.472 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph health --format=json 2024-06-10T00:55:24.731 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:55:24.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:24 smithi167 ceph-mon[36917]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:55:24.853 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:24 smithi037 ceph-mon[32192]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:55:24.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:24 smithi094 ceph-mon[36708]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:55:25.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:25 smithi167 ceph-mon[36917]: pgmap v392: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-10T00:55:25.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:25 smithi094 ceph-mon[36708]: pgmap v392: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-10T00:55:25.865 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:25 smithi037 ceph-mon[32192]: pgmap v392: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-10T00:55:26.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.450+0000 7f84a496e700 1 -- 172.21.15.37:0/4076148474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84a0102380 msgr2=0x7f84a0105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:26.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.450+0000 7f84a496e700 1 --2- 172.21.15.37:0/4076148474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84a0102380 0x7f84a0105ff0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f84940099c0 tx=0x7f8494009cd0 comp rx=0 tx=0).stop 2024-06-10T00:55:26.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.450+0000 7f84a496e700 1 -- 172.21.15.37:0/4076148474 shutdown_connections 2024-06-10T00:55:26.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.450+0000 7f84a496e700 1 --2- 172.21.15.37:0/4076148474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84a0106720 0x7f84a010e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:26.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.450+0000 7f84a496e700 1 --2- 172.21.15.37:0/4076148474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84a01019d0 0x7f84a0101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:26.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.450+0000 7f84a496e700 1 --2- 172.21.15.37:0/4076148474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84a0102380 0x7f84a0105ff0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:26.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.450+0000 7f84a496e700 1 -- 172.21.15.37:0/4076148474 >> 172.21.15.37:0/4076148474 conn(0x7f84a0078340 msgr2=0x7f84a0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:26.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.450+0000 7f84a496e700 1 -- 172.21.15.37:0/4076148474 shutdown_connections 2024-06-10T00:55:26.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.450+0000 7f84a496e700 1 -- 172.21.15.37:0/4076148474 wait complete. 2024-06-10T00:55:26.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.451+0000 7f84a496e700 1 Processor -- start 2024-06-10T00:55:26.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.451+0000 7f84a496e700 1 -- start start 2024-06-10T00:55:26.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.451+0000 7f84a496e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84a01019d0 0x7f84a0103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:26.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.451+0000 7f84a496e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84a0102380 0x7f84a01035d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:26.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.452+0000 7f84a496e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84a0106720 0x7f84a0103b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:26.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.452+0000 7f84a496e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f84a0111690 con 0x7f84a0102380 2024-06-10T00:55:26.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.452+0000 7f84a496e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f84a0111510 con 0x7f84a0106720 2024-06-10T00:55:26.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.452+0000 7f84a496e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f84a0111810 con 0x7f84a01019d0 2024-06-10T00:55:26.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.452+0000 7f849dd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84a0102380 0x7f84a01035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:26.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.452+0000 7f849e59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84a01019d0 0x7f84a0103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:26.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.452+0000 7f849dd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84a0102380 0x7f84a01035d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34194/0 (socket says 172.21.15.37:34194) 2024-06-10T00:55:26.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.452+0000 7f849dd9b700 1 -- 172.21.15.37:0/3650517074 learned_addr learned my addr 172.21.15.37:0/3650517074 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:26.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.452+0000 7f849ed9d700 1 --2- 172.21.15.37:0/3650517074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84a0106720 0x7f84a0103b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:26.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.452+0000 7f849dd9b700 1 -- 172.21.15.37:0/3650517074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84a01019d0 msgr2=0x7f84a0103090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:26.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.452+0000 7f849dd9b700 1 --2- 172.21.15.37:0/3650517074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84a01019d0 0x7f84a0103090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:26.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.452+0000 7f849dd9b700 1 -- 172.21.15.37:0/3650517074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84a0106720 msgr2=0x7f84a0103b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:26.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.452+0000 7f849dd9b700 1 --2- 172.21.15.37:0/3650517074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84a0106720 0x7f84a0103b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:26.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.452+0000 7f849dd9b700 1 -- 172.21.15.37:0/3650517074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a01a7130 con 0x7f84a0102380 2024-06-10T00:55:26.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.452+0000 7f849ed9d700 1 --2- 172.21.15.37:0/3650517074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84a0106720 0x7f84a0103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:55:26.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.452+0000 7f849dd9b700 1 --2- 172.21.15.37:0/3650517074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84a0102380 0x7f84a01035d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f849400b580 tx=0x7f84940038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:26.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.453+0000 7f848f7fe700 1 -- 172.21.15.37:0/3650517074 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8494027070 con 0x7f84a0102380 2024-06-10T00:55:26.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.453+0000 7f84a496e700 1 -- 172.21.15.37:0/3650517074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a01a73c0 con 0x7f84a0102380 2024-06-10T00:55:26.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.453+0000 7f848f7fe700 1 -- 172.21.15.37:0/3650517074 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8494003e40 con 0x7f84a0102380 2024-06-10T00:55:26.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.453+0000 7f848f7fe700 1 -- 172.21.15.37:0/3650517074 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8494022d10 con 0x7f84a0102380 2024-06-10T00:55:26.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.453+0000 7f84a496e700 1 -- 172.21.15.37:0/3650517074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a01a7870 con 0x7f84a0102380 2024-06-10T00:55:26.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.454+0000 7f84a496e700 1 -- 172.21.15.37:0/3650517074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84a0102ad0 con 0x7f84a0102380 2024-06-10T00:55:26.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.455+0000 7f848f7fe700 1 -- 172.21.15.37:0/3650517074 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f849402a070 con 0x7f84a0102380 2024-06-10T00:55:26.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.456+0000 7f848f7fe700 1 --2- 172.21.15.37:0/3650517074 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f84880a2c20 0x7f84880a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:26.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.456+0000 7f849e59c700 1 --2- 172.21.15.37:0/3650517074 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f84880a2c20 0x7f84880a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:26.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.456+0000 7f848f7fe700 1 -- 172.21.15.37:0/3650517074 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f84940d09f0 con 0x7f84a0102380 2024-06-10T00:55:26.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.456+0000 7f849e59c700 1 --2- 172.21.15.37:0/3650517074 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f84880a2c20 0x7f84880a50e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8490009550 tx=0x7f84900091b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:26.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.459+0000 7f848f7fe700 1 -- 172.21.15.37:0/3650517074 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8494031020 con 0x7f84a0102380 2024-06-10T00:55:26.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.711+0000 7f84a496e700 1 -- 172.21.15.37:0/3650517074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f84a002dca0 con 0x7f84a0102380 2024-06-10T00:55:26.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.712+0000 7f848f7fe700 1 -- 172.21.15.37:0/3650517074 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f8494013070 con 0x7f84a0102380 2024-06-10T00:55:26.712 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:55:26.712 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-06-10T00:55:26.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.714+0000 7f84a496e700 1 -- 172.21.15.37:0/3650517074 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f84880a2c20 msgr2=0x7f84880a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:26.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.714+0000 7f84a496e700 1 --2- 172.21.15.37:0/3650517074 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f84880a2c20 0x7f84880a50e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8490009550 tx=0x7f84900091b0 comp rx=0 tx=0).stop 2024-06-10T00:55:26.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.714+0000 7f84a496e700 1 -- 172.21.15.37:0/3650517074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84a0102380 msgr2=0x7f84a01035d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:26.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.714+0000 7f84a496e700 1 --2- 172.21.15.37:0/3650517074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84a0102380 0x7f84a01035d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f849400b580 tx=0x7f84940038b0 comp rx=0 tx=0).stop 2024-06-10T00:55:26.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.714+0000 7f84a496e700 1 -- 172.21.15.37:0/3650517074 shutdown_connections 2024-06-10T00:55:26.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.714+0000 7f84a496e700 1 --2- 172.21.15.37:0/3650517074 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f84880a2c20 0x7f84880a50e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:26.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.714+0000 7f84a496e700 1 --2- 172.21.15.37:0/3650517074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84a0106720 0x7f84a0103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:26.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.714+0000 7f84a496e700 1 --2- 172.21.15.37:0/3650517074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84a01019d0 0x7f84a0103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:26.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.714+0000 7f84a496e700 1 --2- 172.21.15.37:0/3650517074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84a0102380 0x7f84a01035d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:26.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.714+0000 7f84a496e700 1 -- 172.21.15.37:0/3650517074 >> 172.21.15.37:0/3650517074 conn(0x7f84a0078340 msgr2=0x7f84a006ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:26.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.714+0000 7f84a496e700 1 -- 172.21.15.37:0/3650517074 shutdown_connections 2024-06-10T00:55:26.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:26.714+0000 7f84a496e700 1 -- 172.21.15.37:0/3650517074 wait complete. 2024-06-10T00:55:27.482 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-06-10T00:55:27.483 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:55:27.739 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:55:27.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:27 smithi167 ceph-mon[36917]: pgmap v393: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-10T00:55:27.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:27 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3650517074' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-10T00:55:27.862 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:27 smithi037 ceph-mon[32192]: pgmap v393: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-10T00:55:27.862 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:27 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3650517074' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-10T00:55:27.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:27 smithi094 ceph-mon[36708]: pgmap v393: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-10T00:55:27.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:27 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3650517074' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-10T00:55:29.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.419+0000 7f5f74b36700 1 -- 172.21.15.37:0/3276828805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f70101390 msgr2=0x7f5f70101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:29.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.419+0000 7f5f74b36700 1 --2- 172.21.15.37:0/3276828805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f70101390 0x7f5f70101810 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009990 tx=0x7f5f64009ca0 comp rx=0 tx=0).stop 2024-06-10T00:55:29.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.419+0000 7f5f74b36700 1 -- 172.21.15.37:0/3276828805 shutdown_connections 2024-06-10T00:55:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.419+0000 7f5f74b36700 1 --2- 172.21.15.37:0/3276828805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f70101d50 0x7f5f7010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.419+0000 7f5f74b36700 1 --2- 172.21.15.37:0/3276828805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f70107400 0x7f5f701077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.419+0000 7f5f74b36700 1 --2- 172.21.15.37:0/3276828805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f70101390 0x7f5f70101810 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.419+0000 7f5f74b36700 1 -- 172.21.15.37:0/3276828805 >> 172.21.15.37:0/3276828805 conn(0x7f5f70078340 msgr2=0x7f5f70078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.419+0000 7f5f74b36700 1 -- 172.21.15.37:0/3276828805 shutdown_connections 2024-06-10T00:55:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.419+0000 7f5f74b36700 1 -- 172.21.15.37:0/3276828805 wait complete. 2024-06-10T00:55:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.420+0000 7f5f74b36700 1 Processor -- start 2024-06-10T00:55:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.420+0000 7f5f74b36700 1 -- start start 2024-06-10T00:55:29.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.420+0000 7f5f74b36700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f70101390 0x7f5f7019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:29.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.420+0000 7f5f74b36700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f70101d50 0x7f5f7019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:29.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.420+0000 7f5f74b36700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f70107400 0x7f5f70198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:29.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.420+0000 7f5f74b36700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5f70111670 con 0x7f5f70107400 2024-06-10T00:55:29.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.420+0000 7f5f74b36700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5f701114f0 con 0x7f5f70101390 2024-06-10T00:55:29.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.420+0000 7f5f74b36700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5f701117f0 con 0x7f5f70101d50 2024-06-10T00:55:29.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.420+0000 7f5f6ed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f70107400 0x7f5f70198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:29.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.420+0000 7f5f6ed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f70107400 0x7f5f70198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34214/0 (socket says 172.21.15.37:34214) 2024-06-10T00:55:29.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.420+0000 7f5f6ed9d700 1 -- 172.21.15.37:0/1387083049 learned_addr learned my addr 172.21.15.37:0/1387083049 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:29.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.421+0000 7f5f6e59c700 1 --2- 172.21.15.37:0/1387083049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f70101390 0x7f5f7019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:29.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.421+0000 7f5f6dd9b700 1 --2- 172.21.15.37:0/1387083049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f70101d50 0x7f5f7019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:29.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.421+0000 7f5f6ed9d700 1 -- 172.21.15.37:0/1387083049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f70101d50 msgr2=0x7f5f7019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:29.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.421+0000 7f5f6ed9d700 1 --2- 172.21.15.37:0/1387083049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f70101d50 0x7f5f7019eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:29.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.421+0000 7f5f6ed9d700 1 -- 172.21.15.37:0/1387083049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f70101390 msgr2=0x7f5f7019e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:29.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.421+0000 7f5f6ed9d700 1 --2- 172.21.15.37:0/1387083049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f70101390 0x7f5f7019e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:29.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.421+0000 7f5f6ed9d700 1 -- 172.21.15.37:0/1387083049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f701991d0 con 0x7f5f70107400 2024-06-10T00:55:29.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.421+0000 7f5f6dd9b700 1 --2- 172.21.15.37:0/1387083049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f70101d50 0x7f5f7019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:55:29.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.421+0000 7f5f6ed9d700 1 --2- 172.21.15.37:0/1387083049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f70107400 0x7f5f70198a70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800ada0 tx=0x7f5f6800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:29.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.421+0000 7f5f5f7fe700 1 -- 172.21.15.37:0/1387083049 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f68017020 con 0x7f5f70107400 2024-06-10T00:55:29.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.421+0000 7f5f74b36700 1 -- 172.21.15.37:0/1387083049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f701994c0 con 0x7f5f70107400 2024-06-10T00:55:29.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.421+0000 7f5f5f7fe700 1 -- 172.21.15.37:0/1387083049 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f68004bb0 con 0x7f5f70107400 2024-06-10T00:55:29.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.421+0000 7f5f5f7fe700 1 -- 172.21.15.37:0/1387083049 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f68005520 con 0x7f5f70107400 2024-06-10T00:55:29.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.421+0000 7f5f74b36700 1 -- 172.21.15.37:0/1387083049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f701a5730 con 0x7f5f70107400 2024-06-10T00:55:29.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.422+0000 7f5f74b36700 1 -- 172.21.15.37:0/1387083049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f70069180 con 0x7f5f70107400 2024-06-10T00:55:29.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.423+0000 7f5f5f7fe700 1 -- 172.21.15.37:0/1387083049 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5f680056c0 con 0x7f5f70107400 2024-06-10T00:55:29.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.424+0000 7f5f5f7fe700 1 --2- 172.21.15.37:0/1387083049 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5f580a2c20 0x7f5f580a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:29.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.424+0000 7f5f6e59c700 1 --2- 172.21.15.37:0/1387083049 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5f580a2c20 0x7f5f580a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:29.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.424+0000 7f5f5f7fe700 1 -- 172.21.15.37:0/1387083049 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f5f680cec10 con 0x7f5f70107400 2024-06-10T00:55:29.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.425+0000 7f5f6e59c700 1 --2- 172.21.15.37:0/1387083049 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5f580a2c20 0x7f5f580a50e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5f700ffbe0 tx=0x7f5f6000b5b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:29.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.429+0000 7f5f5f7fe700 1 -- 172.21.15.37:0/1387083049 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5f68021020 con 0x7f5f70107400 2024-06-10T00:55:29.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.686+0000 7f5f74b36700 1 -- 172.21.15.37:0/1387083049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f701a59c0 con 0x7f5f70107400 2024-06-10T00:55:29.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.686+0000 7f5f5f7fe700 1 -- 172.21.15.37:0/1387083049 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11867 (secure 0 0 0) 0x7f5f6809b810 con 0x7f5f70107400 2024-06-10T00:55:29.687 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:55:29.687 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T00:53:44.125049+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24458,24472,24451]},"id":1}]} 2024-06-10T00:55:29.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.689+0000 7f5f74b36700 1 -- 172.21.15.37:0/1387083049 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5f580a2c20 msgr2=0x7f5f580a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:29.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.689+0000 7f5f74b36700 1 --2- 172.21.15.37:0/1387083049 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5f580a2c20 0x7f5f580a50e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5f700ffbe0 tx=0x7f5f6000b5b0 comp rx=0 tx=0).stop 2024-06-10T00:55:29.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.689+0000 7f5f74b36700 1 -- 172.21.15.37:0/1387083049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f70107400 msgr2=0x7f5f70198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:29.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.689+0000 7f5f74b36700 1 --2- 172.21.15.37:0/1387083049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f70107400 0x7f5f70198a70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800ada0 tx=0x7f5f6800c330 comp rx=0 tx=0).stop 2024-06-10T00:55:29.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.689+0000 7f5f74b36700 1 -- 172.21.15.37:0/1387083049 shutdown_connections 2024-06-10T00:55:29.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.689+0000 7f5f74b36700 1 --2- 172.21.15.37:0/1387083049 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5f580a2c20 0x7f5f580a50e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:29.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.689+0000 7f5f74b36700 1 --2- 172.21.15.37:0/1387083049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f70107400 0x7f5f70198a70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:29.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.689+0000 7f5f74b36700 1 --2- 172.21.15.37:0/1387083049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f70101390 0x7f5f7019e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:29.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.689+0000 7f5f74b36700 1 --2- 172.21.15.37:0/1387083049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f70101d50 0x7f5f7019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:29.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.689+0000 7f5f74b36700 1 -- 172.21.15.37:0/1387083049 >> 172.21.15.37:0/1387083049 conn(0x7f5f70078340 msgr2=0x7f5f70100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:29.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.689+0000 7f5f74b36700 1 -- 172.21.15.37:0/1387083049 shutdown_connections 2024-06-10T00:55:29.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:29.689+0000 7f5f74b36700 1 -- 172.21.15.37:0/1387083049 wait complete. 2024-06-10T00:55:29.690 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-06-10T00:55:29.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:29 smithi167 ceph-mon[36917]: pgmap v394: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-10T00:55:29.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:29 smithi094 ceph-mon[36708]: pgmap v394: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-10T00:55:29.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:29 smithi037 ceph-mon[32192]: pgmap v394: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-10T00:55:30.484 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:55:30.739 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:55:30.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:30 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1387083049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:55:30.862 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:30 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1387083049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:55:30.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:30 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1387083049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:55:31.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:31 smithi167 ceph-mon[36917]: pgmap v395: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-10T00:55:31.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:31 smithi094 ceph-mon[36708]: pgmap v395: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-10T00:55:31.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:31 smithi037 ceph-mon[32192]: pgmap v395: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-06-10T00:55:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.528+0000 7f95016ba700 1 -- 172.21.15.37:0/2723873520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94fc107400 msgr2=0x7f94fc1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.528+0000 7f95016ba700 1 --2- 172.21.15.37:0/2723873520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94fc107400 0x7f94fc1077e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00b0a0 tx=0x7f94ec00b3b0 comp rx=0 tx=0).stop 2024-06-10T00:55:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.528+0000 7f95016ba700 1 -- 172.21.15.37:0/2723873520 shutdown_connections 2024-06-10T00:55:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.528+0000 7f95016ba700 1 --2- 172.21.15.37:0/2723873520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94fc101d50 0x7f94fc10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.528+0000 7f95016ba700 1 --2- 172.21.15.37:0/2723873520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94fc107400 0x7f94fc1077e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.528+0000 7f95016ba700 1 --2- 172.21.15.37:0/2723873520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94fc101390 0x7f94fc101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.528+0000 7f95016ba700 1 -- 172.21.15.37:0/2723873520 >> 172.21.15.37:0/2723873520 conn(0x7f94fc078340 msgr2=0x7f94fc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.528+0000 7f95016ba700 1 -- 172.21.15.37:0/2723873520 shutdown_connections 2024-06-10T00:55:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.528+0000 7f95016ba700 1 -- 172.21.15.37:0/2723873520 wait complete. 2024-06-10T00:55:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.529+0000 7f95016ba700 1 Processor -- start 2024-06-10T00:55:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.529+0000 7f95016ba700 1 -- start start 2024-06-10T00:55:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.529+0000 7f95016ba700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94fc101390 0x7f94fc199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.529+0000 7f95016ba700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94fc101d50 0x7f94fc19a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.529+0000 7f95016ba700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94fc107400 0x7f94fc1a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.529+0000 7f95016ba700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc1116c0 con 0x7f94fc107400 2024-06-10T00:55:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.529+0000 7f95016ba700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc111540 con 0x7f94fc101390 2024-06-10T00:55:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.529+0000 7f95016ba700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc111840 con 0x7f94fc101d50 2024-06-10T00:55:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.530+0000 7f94fb7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94fc107400 0x7f94fc1a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.530+0000 7f94fb7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94fc107400 0x7f94fc1a4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47534/0 (socket says 172.21.15.37:47534) 2024-06-10T00:55:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.530+0000 7f94fb7fe700 1 -- 172.21.15.37:0/256935198 learned_addr learned my addr 172.21.15.37:0/256935198 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.530+0000 7f94faffd700 1 --2- 172.21.15.37:0/256935198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94fc101390 0x7f94fc199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.530+0000 7f94fa7fc700 1 --2- 172.21.15.37:0/256935198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94fc101d50 0x7f94fc19a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.530+0000 7f94fb7fe700 1 -- 172.21.15.37:0/256935198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94fc101d50 msgr2=0x7f94fc19a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.530+0000 7f94fb7fe700 1 --2- 172.21.15.37:0/256935198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94fc101d50 0x7f94fc19a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.530+0000 7f94fb7fe700 1 -- 172.21.15.37:0/256935198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94fc101390 msgr2=0x7f94fc199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.530+0000 7f94fb7fe700 1 --2- 172.21.15.37:0/256935198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94fc101390 0x7f94fc199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.530+0000 7f94fb7fe700 1 -- 172.21.15.37:0/256935198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94fc1a50e0 con 0x7f94fc107400 2024-06-10T00:55:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.530+0000 7f94fb7fe700 1 --2- 172.21.15.37:0/256935198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94fc107400 0x7f94fc1a4ba0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f94f400c970 tx=0x7f94f400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.530+0000 7f94f3fff700 1 -- 172.21.15.37:0/256935198 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f4004a30 con 0x7f94fc107400 2024-06-10T00:55:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.530+0000 7f95016ba700 1 -- 172.21.15.37:0/256935198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94fc1a5370 con 0x7f94fc107400 2024-06-10T00:55:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.530+0000 7f94f3fff700 1 -- 172.21.15.37:0/256935198 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94f4005500 con 0x7f94fc107400 2024-06-10T00:55:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.530+0000 7f94f3fff700 1 -- 172.21.15.37:0/256935198 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f400ecf0 con 0x7f94fc107400 2024-06-10T00:55:32.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.531+0000 7f95016ba700 1 -- 172.21.15.37:0/256935198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94fc1a5830 con 0x7f94fc107400 2024-06-10T00:55:32.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.531+0000 7f95016ba700 1 -- 172.21.15.37:0/256935198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94fc069180 con 0x7f94fc107400 2024-06-10T00:55:32.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.533+0000 7f94f3fff700 1 -- 172.21.15.37:0/256935198 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f94f4005000 con 0x7f94fc107400 2024-06-10T00:55:32.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.533+0000 7f94f3fff700 1 --2- 172.21.15.37:0/256935198 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f94e80a2c20 0x7f94e80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:32.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.533+0000 7f94f3fff700 1 -- 172.21.15.37:0/256935198 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f94f40cf270 con 0x7f94fc107400 2024-06-10T00:55:32.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.533+0000 7f94faffd700 1 --2- 172.21.15.37:0/256935198 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f94e80a2c20 0x7f94e80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:32.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.534+0000 7f94faffd700 1 --2- 172.21.15.37:0/256935198 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f94e80a2c20 0x7f94e80a50e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f94ec009fd0 tx=0x7f94ec006210 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:32.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.537+0000 7f94f3fff700 1 -- 172.21.15.37:0/256935198 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f94f4014030 con 0x7f94fc107400 2024-06-10T00:55:32.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.790+0000 7f95016ba700 1 -- 172.21.15.37:0/256935198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94fc02dca0 con 0x7f94fc107400 2024-06-10T00:55:32.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.791+0000 7f94f3fff700 1 -- 172.21.15.37:0/256935198 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11867 (secure 0 0 0) 0x7f94f409bde0 con 0x7f94fc107400 2024-06-10T00:55:32.792 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:55:32.792 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T00:53:44.125049+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24458,24472,24451]},"id":1}]} 2024-06-10T00:55:32.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.793+0000 7f95016ba700 1 -- 172.21.15.37:0/256935198 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f94e80a2c20 msgr2=0x7f94e80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:32.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.793+0000 7f95016ba700 1 --2- 172.21.15.37:0/256935198 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f94e80a2c20 0x7f94e80a50e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f94ec009fd0 tx=0x7f94ec006210 comp rx=0 tx=0).stop 2024-06-10T00:55:32.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.793+0000 7f95016ba700 1 -- 172.21.15.37:0/256935198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94fc107400 msgr2=0x7f94fc1a4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:32.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.793+0000 7f95016ba700 1 --2- 172.21.15.37:0/256935198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94fc107400 0x7f94fc1a4ba0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f94f400c970 tx=0x7f94f400cc80 comp rx=0 tx=0).stop 2024-06-10T00:55:32.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.793+0000 7f95016ba700 1 -- 172.21.15.37:0/256935198 shutdown_connections 2024-06-10T00:55:32.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.793+0000 7f95016ba700 1 --2- 172.21.15.37:0/256935198 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f94e80a2c20 0x7f94e80a50e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:32.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.793+0000 7f95016ba700 1 --2- 172.21.15.37:0/256935198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94fc107400 0x7f94fc1a4ba0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:32.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.793+0000 7f95016ba700 1 --2- 172.21.15.37:0/256935198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94fc101390 0x7f94fc199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:32.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.793+0000 7f95016ba700 1 --2- 172.21.15.37:0/256935198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94fc101d50 0x7f94fc19a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:32.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.793+0000 7f95016ba700 1 -- 172.21.15.37:0/256935198 >> 172.21.15.37:0/256935198 conn(0x7f94fc078340 msgr2=0x7f94fc1000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:32.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.793+0000 7f95016ba700 1 -- 172.21.15.37:0/256935198 shutdown_connections 2024-06-10T00:55:32.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:32.793+0000 7f95016ba700 1 -- 172.21.15.37:0/256935198 wait complete. 2024-06-10T00:55:32.795 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-06-10T00:55:33.527 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T00:55:33.783 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:55:33.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:33 smithi167 ceph-mon[36917]: pgmap v396: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-10T00:55:33.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:33 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/256935198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:55:33.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:33 smithi094 ceph-mon[36708]: pgmap v396: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-10T00:55:33.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:33 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/256935198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:55:33.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:33 smithi037 ceph-mon[32192]: pgmap v396: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-10T00:55:33.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:33 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/256935198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:55:35.606 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:35 smithi037 ceph-mon[32192]: pgmap v397: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-10T00:55:35.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.609+0000 7fc744ec7700 1 -- 172.21.15.37:0/1627980900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc740102510 msgr2=0x7fc740102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:35.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.609+0000 7fc744ec7700 1 --2- 172.21.15.37:0/1627980900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc740102510 0x7fc740102990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc728009990 tx=0x7fc728009ca0 comp rx=0 tx=0).stop 2024-06-10T00:55:35.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.609+0000 7fc744ec7700 1 -- 172.21.15.37:0/1627980900 shutdown_connections 2024-06-10T00:55:35.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.609+0000 7fc744ec7700 1 --2- 172.21.15.37:0/1627980900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc740102ed0 0x7fc74010f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:35.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.609+0000 7fc744ec7700 1 --2- 172.21.15.37:0/1627980900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc740108520 0x7fc740108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:35.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.609+0000 7fc744ec7700 1 --2- 172.21.15.37:0/1627980900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc740102510 0x7fc740102990 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:35.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.609+0000 7fc744ec7700 1 -- 172.21.15.37:0/1627980900 >> 172.21.15.37:0/1627980900 conn(0x7fc740079560 msgr2=0x7fc740079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:35.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.609+0000 7fc744ec7700 1 -- 172.21.15.37:0/1627980900 shutdown_connections 2024-06-10T00:55:35.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.609+0000 7fc744ec7700 1 -- 172.21.15.37:0/1627980900 wait complete. 2024-06-10T00:55:35.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.610+0000 7fc744ec7700 1 Processor -- start 2024-06-10T00:55:35.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.610+0000 7fc744ec7700 1 -- start start 2024-06-10T00:55:35.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.610+0000 7fc744ec7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc740102510 0x7fc740074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:35.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.610+0000 7fc744ec7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc740102ed0 0x7fc740074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:35.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.610+0000 7fc744ec7700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc740108520 0x7fc740075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:35.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.610+0000 7fc744ec7700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc7401127c0 con 0x7fc740102510 2024-06-10T00:55:35.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.610+0000 7fc744ec7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc740112640 con 0x7fc740102ed0 2024-06-10T00:55:35.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.610+0000 7fc744ec7700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc740112940 con 0x7fc740108520 2024-06-10T00:55:35.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.611+0000 7fc73e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc740102510 0x7fc740074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:35.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.611+0000 7fc73e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc740102510 0x7fc740074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47546/0 (socket says 172.21.15.37:47546) 2024-06-10T00:55:35.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.611+0000 7fc73e59c700 1 -- 172.21.15.37:0/392179731 learned_addr learned my addr 172.21.15.37:0/392179731 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:35.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.611+0000 7fc73dd9b700 1 --2- 172.21.15.37:0/392179731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc740102ed0 0x7fc740074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:35.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.611+0000 7fc73ed9d700 1 --2- 172.21.15.37:0/392179731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc740108520 0x7fc740075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:35.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.611+0000 7fc73e59c700 1 -- 172.21.15.37:0/392179731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc740108520 msgr2=0x7fc740075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:35.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.611+0000 7fc73e59c700 1 --2- 172.21.15.37:0/392179731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc740108520 0x7fc740075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:35.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.611+0000 7fc73e59c700 1 -- 172.21.15.37:0/392179731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc740102ed0 msgr2=0x7fc740074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:35.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.611+0000 7fc73e59c700 1 --2- 172.21.15.37:0/392179731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc740102ed0 0x7fc740074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:35.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.611+0000 7fc73e59c700 1 -- 172.21.15.37:0/392179731 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7401a70c0 con 0x7fc740102510 2024-06-10T00:55:35.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.611+0000 7fc73ed9d700 1 --2- 172.21.15.37:0/392179731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc740108520 0x7fc740075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:55:35.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.611+0000 7fc73e59c700 1 --2- 172.21.15.37:0/392179731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc740102510 0x7fc740074790 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc73000d980 tx=0x7fc73000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:35.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.611+0000 7fc7377fe700 1 -- 172.21.15.37:0/392179731 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc730017070 con 0x7fc740102510 2024-06-10T00:55:35.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.611+0000 7fc7377fe700 1 -- 172.21.15.37:0/392179731 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc730005020 con 0x7fc740102510 2024-06-10T00:55:35.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.611+0000 7fc744ec7700 1 -- 172.21.15.37:0/392179731 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7401a73b0 con 0x7fc740102510 2024-06-10T00:55:35.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.611+0000 7fc7377fe700 1 -- 172.21.15.37:0/392179731 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc730009bc0 con 0x7fc740102510 2024-06-10T00:55:35.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.611+0000 7fc744ec7700 1 -- 172.21.15.37:0/392179731 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7401a7870 con 0x7fc740102510 2024-06-10T00:55:35.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.612+0000 7fc744ec7700 1 -- 172.21.15.37:0/392179731 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc740069180 con 0x7fc740102510 2024-06-10T00:55:35.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.614+0000 7fc7377fe700 1 -- 172.21.15.37:0/392179731 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc7300051d0 con 0x7fc740102510 2024-06-10T00:55:35.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.614+0000 7fc7377fe700 1 --2- 172.21.15.37:0/392179731 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc72c0a2c20 0x7fc72c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:35.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.615+0000 7fc73dd9b700 1 --2- 172.21.15.37:0/392179731 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc72c0a2c20 0x7fc72c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:35.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.615+0000 7fc7377fe700 1 -- 172.21.15.37:0/392179731 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fc7300cf350 con 0x7fc740102510 2024-06-10T00:55:35.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.615+0000 7fc73dd9b700 1 --2- 172.21.15.37:0/392179731 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc72c0a2c20 0x7fc72c0a50e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc72800b5c0 tx=0x7fc728005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:35.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.619+0000 7fc7377fe700 1 -- 172.21.15.37:0/392179731 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc730020020 con 0x7fc740102510 2024-06-10T00:55:35.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.826+0000 7fc744ec7700 1 -- 172.21.15.37:0/392179731 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc74002de60 con 0x7fc740102510 2024-06-10T00:55:35.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.828+0000 7fc7377fe700 1 -- 172.21.15.37:0/392179731 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24854 (secure 0 0 0) 0x7fc73009bec0 con 0x7fc740102510 2024-06-10T00:55:35.829 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:55:35.829 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":100,"fsid":"68793514-26c2-11ef-bc9b-c7b262605968","created":"2024-06-10T00:44:24.106713+0000","modified":"2024-06-10T00:54:22.546967+0000","last_up_change":"2024-06-10T00:51:31.562156+0000","last_in_change":"2024-06-10T00:51:11.294301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:47:44.773911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:52:31.684780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:52:34.885503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:52:44.415437+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1df6046a-6801-4950-9193-54303fed169b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6803","nonce":3594704094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6805","nonce":3594704094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6809","nonce":3594704094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6807","nonce":3594704094}]},"public_addr":"172.21.15.37:6803/3594704094","cluster_addr":"172.21.15.37:6805/3594704094","heartbeat_back_addr":"172.21.15.37:6809/3594704094","heartbeat_front_addr":"172.21.15.37:6807/3594704094","state":["exists","up"]},{"osd":1,"uuid":"4e1c8944-94e4-4e46-9efe-56229fe6df1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6801","nonce":3427727155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6803","nonce":3427727155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6807","nonce":3427727155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6805","nonce":3427727155}]},"public_addr":"172.21.15.94:6801/3427727155","cluster_addr":"172.21.15.94:6803/3427727155","heartbeat_back_addr":"172.21.15.94:6807/3427727155","heartbeat_front_addr":"172.21.15.94:6805/3427727155","state":["exists","up"]},{"osd":2,"uuid":"fd2fcedc-4c88-478f-8025-13e5944a27ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6801","nonce":4002498292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6803","nonce":4002498292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6807","nonce":4002498292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6805","nonce":4002498292}]},"public_addr":"172.21.15.167:6801/4002498292","cluster_addr":"172.21.15.167:6803/4002498292","heartbeat_back_addr":"172.21.15.167:6807/4002498292","heartbeat_front_addr":"172.21.15.167:6805/4002498292","state":["exists","up"]},{"osd":3,"uuid":"d0153676-818c-4dcc-83b1-287435a0c8a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6811","nonce":3425469324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6813","nonce":3425469324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6817","nonce":3425469324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6815","nonce":3425469324}]},"public_addr":"172.21.15.37:6811/3425469324","cluster_addr":"172.21.15.37:6813/3425469324","heartbeat_back_addr":"172.21.15.37:6817/3425469324","heartbeat_front_addr":"172.21.15.37:6815/3425469324","state":["exists","up"]},{"osd":4,"uuid":"172c333c-00c0-4e94-8870-acb3b32b2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6809","nonce":1740627431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6811","nonce":1740627431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6815","nonce":1740627431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6813","nonce":1740627431}]},"public_addr":"172.21.15.94:6809/1740627431","cluster_addr":"172.21.15.94:6811/1740627431","heartbeat_back_addr":"172.21.15.94:6815/1740627431","heartbeat_front_addr":"172.21.15.94:6813/1740627431","state":["exists","up"]},{"osd":5,"uuid":"dbce656e-5f93-4bda-baf2-fc3cc03d8282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6809","nonce":221805236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6811","nonce":221805236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6815","nonce":221805236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6813","nonce":221805236}]},"public_addr":"172.21.15.167:6809/221805236","cluster_addr":"172.21.15.167:6811/221805236","heartbeat_back_addr":"172.21.15.167:6815/221805236","heartbeat_front_addr":"172.21.15.167:6813/221805236","state":["exists","up"]},{"osd":6,"uuid":"fcbda266-6d60-43b2-a51c-4e562476d1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6819","nonce":472670875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6821","nonce":472670875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6825","nonce":472670875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6823","nonce":472670875}]},"public_addr":"172.21.15.37:6819/472670875","cluster_addr":"172.21.15.37:6821/472670875","heartbeat_back_addr":"172.21.15.37:6825/472670875","heartbeat_front_addr":"172.21.15.37:6823/472670875","state":["exists","up"]},{"osd":7,"uuid":"ed829ed3-ba85-4e82-ac37-e44e9534d37c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6817","nonce":3932281552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6819","nonce":3932281552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6823","nonce":3932281552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6821","nonce":3932281552}]},"public_addr":"172.21.15.94:6817/3932281552","cluster_addr":"172.21.15.94:6819/3932281552","heartbeat_back_addr":"172.21.15.94:6823/3932281552","heartbeat_front_addr":"172.21.15.94:6821/3932281552","state":["exists","up"]},{"osd":8,"uuid":"7d6267d8-72f4-4704-b554-bafc52d11a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6817","nonce":2290216047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6819","nonce":2290216047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6823","nonce":2290216047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6821","nonce":2290216047}]},"public_addr":"172.21.15.167:6817/2290216047","cluster_addr":"172.21.15.167:6819/2290216047","heartbeat_back_addr":"172.21.15.167:6823/2290216047","heartbeat_front_addr":"172.21.15.167:6821/2290216047","state":["exists","up"]},{"osd":9,"uuid":"210fdc20-0a36-4eeb-bdf2-c0a7f130e19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6827","nonce":2063666428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6829","nonce":2063666428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6833","nonce":2063666428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6831","nonce":2063666428}]},"public_addr":"172.21.15.37:6827/2063666428","cluster_addr":"172.21.15.37:6829/2063666428","heartbeat_back_addr":"172.21.15.37:6833/2063666428","heartbeat_front_addr":"172.21.15.37:6831/2063666428","state":["exists","up"]},{"osd":10,"uuid":"e200b371-b3cf-4b92-a659-f155ca86f0a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6825","nonce":1706618899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6827","nonce":1706618899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6831","nonce":1706618899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6829","nonce":1706618899}]},"public_addr":"172.21.15.94:6825/1706618899","cluster_addr":"172.21.15.94:6827/1706618899","heartbeat_back_addr":"172.21.15.94:6831/1706618899","heartbeat_front_addr":"172.21.15.94:6829/1706618899","state":["exists","up"]},{"osd":11,"uuid":"cfa427df-37a5-4d28-ac2e-af777d607008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6825","nonce":1349892516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6827","nonce":1349892516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6831","nonce":1349892516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6829","nonce":1349892516}]},"public_addr":"172.21.15.167:6825/1349892516","cluster_addr":"172.21.15.167:6827/1349892516","heartbeat_back_addr":"172.21.15.167:6831/1349892516","heartbeat_front_addr":"172.21.15.167:6829/1349892516","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:46:51.846147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:15.763723+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:40.106800+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:05.923204+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:30.501883+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:55.565575+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:20.411360+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:46.071776+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:11.854599+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:37.608818+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:03.176767+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:29.768705+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/4084722397":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/3571803585":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/416587443":"2024-06-11T00:45:20.254342+0000","172.21.15.37:6801/2766692263":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/2026344134":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/2781321886":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6801/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/1110143997":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2766692263":"2024-06-11T00:45:20.254342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:55:35.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.830+0000 7fc744ec7700 1 -- 172.21.15.37:0/392179731 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc72c0a2c20 msgr2=0x7fc72c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:35.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.830+0000 7fc744ec7700 1 --2- 172.21.15.37:0/392179731 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc72c0a2c20 0x7fc72c0a50e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc72800b5c0 tx=0x7fc728005960 comp rx=0 tx=0).stop 2024-06-10T00:55:35.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.830+0000 7fc744ec7700 1 -- 172.21.15.37:0/392179731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc740102510 msgr2=0x7fc740074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:35.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.830+0000 7fc744ec7700 1 --2- 172.21.15.37:0/392179731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc740102510 0x7fc740074790 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc73000d980 tx=0x7fc73000dc90 comp rx=0 tx=0).stop 2024-06-10T00:55:35.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.831+0000 7fc744ec7700 1 -- 172.21.15.37:0/392179731 shutdown_connections 2024-06-10T00:55:35.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.831+0000 7fc744ec7700 1 --2- 172.21.15.37:0/392179731 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc72c0a2c20 0x7fc72c0a50e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:35.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.831+0000 7fc744ec7700 1 --2- 172.21.15.37:0/392179731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc740108520 0x7fc740075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:35.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.831+0000 7fc744ec7700 1 --2- 172.21.15.37:0/392179731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc740102510 0x7fc740074790 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:35.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.831+0000 7fc744ec7700 1 --2- 172.21.15.37:0/392179731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc740102ed0 0x7fc740074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:35.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.831+0000 7fc744ec7700 1 -- 172.21.15.37:0/392179731 >> 172.21.15.37:0/392179731 conn(0x7fc740079560 msgr2=0x7fc740101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:35.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.831+0000 7fc744ec7700 1 -- 172.21.15.37:0/392179731 shutdown_connections 2024-06-10T00:55:35.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:35.831+0000 7fc744ec7700 1 -- 172.21.15.37:0/392179731 wait complete. 2024-06-10T00:55:35.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:35 smithi167 ceph-mon[36917]: pgmap v397: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-10T00:55:35.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:35 smithi094 ceph-mon[36708]: pgmap v397: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-10T00:55:36.600 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:36 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/392179731' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:55:36.602 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:55:36.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:36 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/392179731' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:55:36.861 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:55:36.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:36 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/392179731' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:55:37.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:37 smithi094 ceph-mon[36708]: pgmap v398: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-06-10T00:55:37.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:37 smithi037 ceph-mon[32192]: pgmap v398: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-06-10T00:55:38.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:37 smithi167 ceph-mon[36917]: pgmap v398: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-06-10T00:55:38.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.682+0000 7ffb1d636700 1 -- 172.21.15.37:0/3431719322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb18101fc0 msgr2=0x7ffb18106010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:38.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.682+0000 7ffb1d636700 1 --2- 172.21.15.37:0/3431719322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb18101fc0 0x7ffb18106010 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0099c0 tx=0x7ffb0c009cd0 comp rx=0 tx=0).stop 2024-06-10T00:55:38.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.682+0000 7ffb1d636700 1 -- 172.21.15.37:0/3431719322 shutdown_connections 2024-06-10T00:55:38.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.682+0000 7ffb1d636700 1 --2- 172.21.15.37:0/3431719322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb181066f0 0x7ffb1810e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:38.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.682+0000 7ffb1d636700 1 --2- 172.21.15.37:0/3431719322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb18101610 0x7ffb181019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:38.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.682+0000 7ffb1d636700 1 --2- 172.21.15.37:0/3431719322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb18101fc0 0x7ffb18106010 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:38.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.682+0000 7ffb1d636700 1 -- 172.21.15.37:0/3431719322 >> 172.21.15.37:0/3431719322 conn(0x7ffb18078430 msgr2=0x7ffb18078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:38.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.682+0000 7ffb1d636700 1 -- 172.21.15.37:0/3431719322 shutdown_connections 2024-06-10T00:55:38.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.682+0000 7ffb1d636700 1 -- 172.21.15.37:0/3431719322 wait complete. 2024-06-10T00:55:38.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.683+0000 7ffb1d636700 1 Processor -- start 2024-06-10T00:55:38.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.683+0000 7ffb1d636700 1 -- start start 2024-06-10T00:55:38.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.683+0000 7ffb1d636700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb18101610 0x7ffb18198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:38.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.683+0000 7ffb1d636700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb18101fc0 0x7ffb18199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:38.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.683+0000 7ffb1d636700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb181066f0 0x7ffb1819d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:38.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.683+0000 7ffb1d636700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffb18111780 con 0x7ffb18101610 2024-06-10T00:55:38.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.683+0000 7ffb1d636700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffb18111600 con 0x7ffb18101fc0 2024-06-10T00:55:38.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.683+0000 7ffb1d636700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffb18111900 con 0x7ffb181066f0 2024-06-10T00:55:38.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.684+0000 7ffb16ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb18101610 0x7ffb18198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:38.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.684+0000 7ffb16ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb18101610 0x7ffb18198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47576/0 (socket says 172.21.15.37:47576) 2024-06-10T00:55:38.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.684+0000 7ffb16ffd700 1 -- 172.21.15.37:0/3588600729 learned_addr learned my addr 172.21.15.37:0/3588600729 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:38.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.684+0000 7ffb167fc700 1 --2- 172.21.15.37:0/3588600729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb18101fc0 0x7ffb18199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:38.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.684+0000 7ffb177fe700 1 --2- 172.21.15.37:0/3588600729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb181066f0 0x7ffb1819d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:38.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.684+0000 7ffb16ffd700 1 -- 172.21.15.37:0/3588600729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb181066f0 msgr2=0x7ffb1819d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:38.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.684+0000 7ffb16ffd700 1 --2- 172.21.15.37:0/3588600729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb181066f0 0x7ffb1819d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:38.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.684+0000 7ffb16ffd700 1 -- 172.21.15.37:0/3588600729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb18101fc0 msgr2=0x7ffb18199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:38.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.684+0000 7ffb16ffd700 1 --2- 172.21.15.37:0/3588600729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb18101fc0 0x7ffb18199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:38.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.684+0000 7ffb16ffd700 1 -- 172.21.15.37:0/3588600729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb1819dd30 con 0x7ffb18101610 2024-06-10T00:55:38.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.684+0000 7ffb16ffd700 1 --2- 172.21.15.37:0/3588600729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb18101610 0x7ffb18198d20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800d950 tx=0x7ffb0800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:38.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.684+0000 7ffafffff700 1 -- 172.21.15.37:0/3588600729 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb08016070 con 0x7ffb18101610 2024-06-10T00:55:38.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.684+0000 7ffafffff700 1 -- 172.21.15.37:0/3588600729 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb08009e20 con 0x7ffb18101610 2024-06-10T00:55:38.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.684+0000 7ffb1d636700 1 -- 172.21.15.37:0/3588600729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb1819e020 con 0x7ffb18101610 2024-06-10T00:55:38.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.684+0000 7ffafffff700 1 -- 172.21.15.37:0/3588600729 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0800f620 con 0x7ffb18101610 2024-06-10T00:55:38.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.684+0000 7ffb1d636700 1 -- 172.21.15.37:0/3588600729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb181a58a0 con 0x7ffb18101610 2024-06-10T00:55:38.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.685+0000 7ffb1d636700 1 -- 172.21.15.37:0/3588600729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb18069180 con 0x7ffb18101610 2024-06-10T00:55:38.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.687+0000 7ffafffff700 1 -- 172.21.15.37:0/3588600729 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffb08019070 con 0x7ffb18101610 2024-06-10T00:55:38.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.687+0000 7ffafffff700 1 --2- 172.21.15.37:0/3588600729 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ffb000a2c20 0x7ffb000a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:38.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.687+0000 7ffb167fc700 1 --2- 172.21.15.37:0/3588600729 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ffb000a2c20 0x7ffb000a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:38.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.687+0000 7ffafffff700 1 -- 172.21.15.37:0/3588600729 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7ffb080ceff0 con 0x7ffb18101610 2024-06-10T00:55:38.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.688+0000 7ffb167fc700 1 --2- 172.21.15.37:0/3588600729 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ffb000a2c20 0x7ffb000a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c000c00 tx=0x7ffb0c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:38.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.691+0000 7ffafffff700 1 -- 172.21.15.37:0/3588600729 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ffb0809bb60 con 0x7ffb18101610 2024-06-10T00:55:38.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.943+0000 7ffb1d636700 1 -- 172.21.15.37:0/3588600729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb1802dca0 con 0x7ffb18101610 2024-06-10T00:55:38.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.945+0000 7ffafffff700 1 -- 172.21.15.37:0/3588600729 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11867 (secure 0 0 0) 0x7ffb0809b460 con 0x7ffb18101610 2024-06-10T00:55:38.945 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:55:38.945 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T00:53:44.125049+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24458,24472,24451]},"id":1}]} 2024-06-10T00:55:38.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.947+0000 7ffb1d636700 1 -- 172.21.15.37:0/3588600729 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ffb000a2c20 msgr2=0x7ffb000a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:38.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.947+0000 7ffb1d636700 1 --2- 172.21.15.37:0/3588600729 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ffb000a2c20 0x7ffb000a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c000c00 tx=0x7ffb0c005fb0 comp rx=0 tx=0).stop 2024-06-10T00:55:38.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.947+0000 7ffb1d636700 1 -- 172.21.15.37:0/3588600729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb18101610 msgr2=0x7ffb18198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:38.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.947+0000 7ffb1d636700 1 --2- 172.21.15.37:0/3588600729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb18101610 0x7ffb18198d20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800d950 tx=0x7ffb0800dc60 comp rx=0 tx=0).stop 2024-06-10T00:55:38.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.947+0000 7ffb1d636700 1 -- 172.21.15.37:0/3588600729 shutdown_connections 2024-06-10T00:55:38.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.947+0000 7ffb1d636700 1 --2- 172.21.15.37:0/3588600729 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ffb000a2c20 0x7ffb000a50e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:38.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.947+0000 7ffb1d636700 1 --2- 172.21.15.37:0/3588600729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb181066f0 0x7ffb1819d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:38.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.947+0000 7ffb1d636700 1 --2- 172.21.15.37:0/3588600729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb18101610 0x7ffb18198d20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:38.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.947+0000 7ffb1d636700 1 --2- 172.21.15.37:0/3588600729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb18101fc0 0x7ffb18199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:38.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.947+0000 7ffb1d636700 1 -- 172.21.15.37:0/3588600729 >> 172.21.15.37:0/3588600729 conn(0x7ffb18078430 msgr2=0x7ffb1806aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:38.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.947+0000 7ffb1d636700 1 -- 172.21.15.37:0/3588600729 shutdown_connections 2024-06-10T00:55:38.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:38.947+0000 7ffb1d636700 1 -- 172.21.15.37:0/3588600729 wait complete. 2024-06-10T00:55:38.949 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-06-10T00:55:39.677 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-10T00:52:37.869277+0000', 'modified': '2024-06-10T00:53:44.125049+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24472, 'mds_1': 24458, 'mds_2': 24451}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24413': {'gid': 24413, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.94:6835/570288048', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6834', 'nonce': 570288048}, {'type': 'v1', 'addr': '172.21.15.94:6835', 'nonce': 570288048}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.94:6837/3801637791', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6836', 'nonce': 3801637791}, {'type': 'v1', 'addr': '172.21.15.94:6837', 'nonce': 3801637791}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6835/3066042611', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 3066042611}, {'type': 'v1', 'addr': '172.21.15.167:6835', 'nonce': 3066042611}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.167:6837/2720148065', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 2720148065}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 2720148065}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.94:6839/946269331', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6838', 'nonce': 946269331}, {'type': 'v1', 'addr': '172.21.15.94:6839', 'nonce': 946269331}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.167:6839/3945638144', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 3945638144}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 3945638144}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24472, 'qdb_cluster': [24458, 24472, 24451]} 2024-06-10T00:55:39.678 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-06-10T00:55:39.678 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.c status 2024-06-10T00:55:39.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:39 smithi094 ceph-mon[36708]: pgmap v399: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:55:39.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:39 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3588600729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:55:39.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:39 smithi037 ceph-mon[32192]: pgmap v399: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:55:39.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:39 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3588600729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:55:39.937 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:55:40.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:39 smithi167 ceph-mon[36917]: pgmap v399: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:55:40.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:39 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3588600729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:55:41.664 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:41 smithi037 ceph-mon[32192]: pgmap v400: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:55:41.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.667+0000 7ff40c367700 1 -- 172.21.15.37:0/3237330045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff404100e30 msgr2=0x7ff4041012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:41.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.667+0000 7ff40c367700 1 --2- 172.21.15.37:0/3237330045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff404100e30 0x7ff4041012b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80099c0 tx=0x7ff3f8009cd0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.667+0000 7ff40c367700 1 -- 172.21.15.37:0/3237330045 shutdown_connections 2024-06-10T00:55:41.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.667+0000 7ff40c367700 1 --2- 172.21.15.37:0/3237330045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4041017f0 0x7ff40410f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.667+0000 7ff40c367700 1 --2- 172.21.15.37:0/3237330045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff404107500 0x7ff4041078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.667+0000 7ff40c367700 1 --2- 172.21.15.37:0/3237330045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff404100e30 0x7ff4041012b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.667+0000 7ff40c367700 1 -- 172.21.15.37:0/3237330045 >> 172.21.15.37:0/3237330045 conn(0x7ff404076730 msgr2=0x7ff404076b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:41.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.667+0000 7ff40c367700 1 -- 172.21.15.37:0/3237330045 shutdown_connections 2024-06-10T00:55:41.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.667+0000 7ff40c367700 1 -- 172.21.15.37:0/3237330045 wait complete. 2024-06-10T00:55:41.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.668+0000 7ff40c367700 1 Processor -- start 2024-06-10T00:55:41.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.668+0000 7ff40c367700 1 -- start start 2024-06-10T00:55:41.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.668+0000 7ff40c367700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff404100e30 0x7ff40419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:41.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.668+0000 7ff40c367700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4041017f0 0x7ff40419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:41.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.668+0000 7ff40c367700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff404107500 0x7ff40419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:41.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.668+0000 7ff40c367700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff4041127c0 con 0x7ff4041017f0 2024-06-10T00:55:41.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.668+0000 7ff40c367700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff404112640 con 0x7ff404107500 2024-06-10T00:55:41.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.668+0000 7ff40c367700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff404112940 con 0x7ff404100e30 2024-06-10T00:55:41.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.668+0000 7ff409902700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4041017f0 0x7ff40419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:41.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.668+0000 7ff409902700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4041017f0 0x7ff40419b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47588/0 (socket says 172.21.15.37:47588) 2024-06-10T00:55:41.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.668+0000 7ff409902700 1 -- 172.21.15.37:0/2513102367 learned_addr learned my addr 172.21.15.37:0/2513102367 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:41.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.668+0000 7ff40a904700 1 --2- 172.21.15.37:0/2513102367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff404107500 0x7ff40419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:41.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.668+0000 7ff40a103700 1 --2- 172.21.15.37:0/2513102367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff404100e30 0x7ff40419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:41.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.669+0000 7ff409902700 1 -- 172.21.15.37:0/2513102367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff404100e30 msgr2=0x7ff40419adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:41.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.669+0000 7ff409902700 1 --2- 172.21.15.37:0/2513102367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff404100e30 0x7ff40419adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.669+0000 7ff409902700 1 -- 172.21.15.37:0/2513102367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff404107500 msgr2=0x7ff40419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:41.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.669+0000 7ff409902700 1 --2- 172.21.15.37:0/2513102367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff404107500 0x7ff40419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.669+0000 7ff409902700 1 -- 172.21.15.37:0/2513102367 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40419fe00 con 0x7ff4041017f0 2024-06-10T00:55:41.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.669+0000 7ff409902700 1 --2- 172.21.15.37:0/2513102367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4041017f0 0x7ff40419b330 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8006010 tx=0x7ff3f8009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:41.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.669+0000 7ff4037fe700 1 -- 172.21.15.37:0/2513102367 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8026070 con 0x7ff4041017f0 2024-06-10T00:55:41.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.669+0000 7ff4037fe700 1 -- 172.21.15.37:0/2513102367 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3f8004cd0 con 0x7ff4041017f0 2024-06-10T00:55:41.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.669+0000 7ff40c367700 1 -- 172.21.15.37:0/2513102367 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4041a0090 con 0x7ff4041017f0 2024-06-10T00:55:41.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.669+0000 7ff4037fe700 1 -- 172.21.15.37:0/2513102367 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8003ba0 con 0x7ff4041017f0 2024-06-10T00:55:41.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.669+0000 7ff40c367700 1 -- 172.21.15.37:0/2513102367 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4041a7970 con 0x7ff4041017f0 2024-06-10T00:55:41.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.671+0000 7ff4037fe700 1 -- 172.21.15.37:0/2513102367 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff3f8029070 con 0x7ff4041017f0 2024-06-10T00:55:41.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.672+0000 7ff4037fe700 1 --2- 172.21.15.37:0/2513102367 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff3ec0a2c20 0x7ff3ec0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:41.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.672+0000 7ff40a103700 1 --2- 172.21.15.37:0/2513102367 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff3ec0a2c20 0x7ff3ec0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:41.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.672+0000 7ff4037fe700 1 -- 172.21.15.37:0/2513102367 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7ff3f80cfd20 con 0x7ff4041017f0 2024-06-10T00:55:41.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.672+0000 7ff40a103700 1 --2- 172.21.15.37:0/2513102367 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff3ec0a2c20 0x7ff3ec0a50e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4003f90 tx=0x7ff3f4009180 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:41.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.673+0000 7ff416beab80 1 Processor -- start 2024-06-10T00:55:41.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.673+0000 7ff416beab80 1 -- start start 2024-06-10T00:55:41.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.673+0000 7ff416beab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb242c1f10 0x55cb242c22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:41.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.673+0000 7ff416beab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb242c2830 0x55cb242bc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:41.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.673+0000 7ff416beab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb242bc750 0x55cb242bcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:41.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.673+0000 7ff416beab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cb241aef10 con 0x55cb242bc750 2024-06-10T00:55:41.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.673+0000 7ff416beab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cb2420d800 con 0x55cb242c2830 2024-06-10T00:55:41.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.673+0000 7ff416beab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cb24175410 con 0x55cb242c1f10 2024-06-10T00:55:41.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.673+0000 7ff40a904700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb242bc750 0x55cb242bcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:41.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.673+0000 7ff40a904700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb242bc750 0x55cb242bcb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47600/0 (socket says 172.21.15.37:47600) 2024-06-10T00:55:41.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.673+0000 7ff40a904700 1 -- 172.21.15.37:0/3020997749 learned_addr learned my addr 172.21.15.37:0/3020997749 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:41.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.673+0000 7ff40a103700 1 --2- 172.21.15.37:0/3020997749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb242c1f10 0x55cb242c22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:41.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.674+0000 7ff409902700 1 --2- 172.21.15.37:0/3020997749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb242c2830 0x55cb242bc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:41.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.674+0000 7ff40a904700 1 -- 172.21.15.37:0/3020997749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb242c1f10 msgr2=0x55cb242c22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:55:41.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.674+0000 7ff40a904700 1 --2- 172.21.15.37:0/3020997749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb242c1f10 0x55cb242c22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.674+0000 7ff40a904700 1 -- 172.21.15.37:0/3020997749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb242c2830 msgr2=0x55cb242bc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:55:41.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.674+0000 7ff40a904700 1 --2- 172.21.15.37:0/3020997749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb242c2830 0x55cb242bc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.674+0000 7ff40a904700 1 -- 172.21.15.37:0/3020997749 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb2415c880 con 0x55cb242bc750 2024-06-10T00:55:41.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.674+0000 7ff409902700 1 --2- 172.21.15.37:0/3020997749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb242c2830 0x55cb242bc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:55:41.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.674+0000 7ff40a904700 1 --2- 172.21.15.37:0/3020997749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb242bc750 0x55cb242bcb70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc00b6c0 tx=0x7ff3fc00b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e6309cc96d7f3f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:41.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.674+0000 7ff400d70700 1 -- 172.21.15.37:0/3020997749 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc004a30 con 0x55cb242bc750 2024-06-10T00:55:41.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.674+0000 7ff400d70700 1 -- 172.21.15.37:0/3020997749 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3fc015070 con 0x55cb242bc750 2024-06-10T00:55:41.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.674+0000 7ff400d70700 1 -- 172.21.15.37:0/3020997749 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc003ac0 con 0x55cb242bc750 2024-06-10T00:55:41.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.675+0000 7ff416beab80 1 -- 172.21.15.37:0/3020997749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb242bc750 msgr2=0x55cb242bcb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:41.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.675+0000 7ff416beab80 1 --2- 172.21.15.37:0/3020997749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb242bc750 0x55cb242bcb70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc00b6c0 tx=0x7ff3fc00b9d0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.675+0000 7ff416beab80 1 -- 172.21.15.37:0/3020997749 shutdown_connections 2024-06-10T00:55:41.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.675+0000 7ff416beab80 1 --2- 172.21.15.37:0/3020997749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb242bc750 0x55cb242bcb70 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.675+0000 7ff416beab80 1 --2- 172.21.15.37:0/3020997749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb242c1f10 0x55cb242c22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.675+0000 7ff416beab80 1 --2- 172.21.15.37:0/3020997749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb242c2830 0x55cb242bc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.675+0000 7ff416beab80 1 -- 172.21.15.37:0/3020997749 >> 172.21.15.37:0/3020997749 conn(0x55cb241e9030 msgr2=0x55cb242b71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:41.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.675+0000 7ff416beab80 1 -- 172.21.15.37:0/3020997749 shutdown_connections 2024-06-10T00:55:41.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.675+0000 7ff416beab80 1 -- 172.21.15.37:0/3020997749 wait complete. 2024-06-10T00:55:41.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.675+0000 7ff416beab80 1 -- start start 2024-06-10T00:55:41.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.675+0000 7ff416beab80 1 Processor -- start 2024-06-10T00:55:41.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.676+0000 7ff416beab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb242bc750 0x55cb242c8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:41.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.676+0000 7ff416beab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb242c1f10 0x55cb242c5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:41.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.676+0000 7ff416beab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb242c2830 0x55cb242c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:41.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.676+0000 7ff416beab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cb241adf70 con 0x55cb242bc750 2024-06-10T00:55:41.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.676+0000 7ff416beab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cb242097c0 con 0x55cb242c1f10 2024-06-10T00:55:41.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.676+0000 7ff416beab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cb242c9210 con 0x55cb242c2830 2024-06-10T00:55:41.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.676+0000 7ff416beab80 10 client.0 _ll_register_callbacks cb 0x55cb241a3560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:55:41.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.676+0000 7ff40a103700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb242bc750 0x55cb242c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:41.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.676+0000 7ff40a103700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb242bc750 0x55cb242c8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47610/0 (socket says 172.21.15.37:47610) 2024-06-10T00:55:41.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.676+0000 7ff40a103700 1 -- 172.21.15.37:0/2516757586 learned_addr learned my addr 172.21.15.37:0/2516757586 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:41.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.676+0000 7ff409902700 1 --2- 172.21.15.37:0/2516757586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb242c1f10 0x55cb242c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:41.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.676+0000 7ff40a904700 1 --2- 172.21.15.37:0/2516757586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb242c2830 0x55cb242c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:41.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.676+0000 7ff40a103700 1 -- 172.21.15.37:0/2516757586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb242c2830 msgr2=0x55cb242c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:55:41.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.676+0000 7ff40a103700 1 --2- 172.21.15.37:0/2516757586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb242c2830 0x55cb242c5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.676+0000 7ff40a103700 1 -- 172.21.15.37:0/2516757586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb242c1f10 msgr2=0x55cb242c5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:55:41.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.676+0000 7ff40a103700 1 --2- 172.21.15.37:0/2516757586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb242c1f10 0x55cb242c5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.676+0000 7ff40a103700 1 -- 172.21.15.37:0/2516757586 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb241d3700 con 0x55cb242bc750 2024-06-10T00:55:41.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.676+0000 7ff40a904700 1 --2- 172.21.15.37:0/2516757586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb242c2830 0x55cb242c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:55:41.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.676+0000 7ff40a103700 1 --2- 172.21.15.37:0/2516757586 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb242bc750 0x55cb242c8520 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4007ab0 tx=0x7ff3f4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95801844c374b7d7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:41.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.676+0000 7ff3e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:55:41.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.677+0000 7ff3e9ffb700 1 -- 172.21.15.37:0/2516757586 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4026020 con 0x55cb242bc750 2024-06-10T00:55:41.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.677+0000 7ff416beab80 1 -- 172.21.15.37:0/2516757586 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3fc015070 con 0x55cb242bc750 2024-06-10T00:55:41.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.677+0000 7ff3e9ffb700 1 -- 172.21.15.37:0/2516757586 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7ff3f4015ad0 con 0x55cb242bc750 2024-06-10T00:55:41.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.677+0000 7ff3e9ffb700 1 -- 172.21.15.37:0/2516757586 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff3f4031050 con 0x55cb242bc750 2024-06-10T00:55:41.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.677+0000 7ff416beab80 10 client.15054 fetch_fsmap learned FSMap version 26 2024-06-10T00:55:41.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.677+0000 7ff416beab80 1 -- 172.21.15.37:0/2516757586 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cb242d3a30 con 0x55cb242bc750 2024-06-10T00:55:41.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.677+0000 7ff3e9ffb700 1 -- 172.21.15.37:0/2516757586 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3fc015070 con 0x55cb242bc750 2024-06-10T00:55:41.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.677+0000 7ff3e9ffb700 1 -- 172.21.15.37:0/2516757586 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ff3f40163f0 con 0x55cb242bc750 2024-06-10T00:55:41.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.677+0000 7ff416beab80 10 client.15054 fetch_fsmap finished waiting for FSMap version 26 2024-06-10T00:55:41.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.677+0000 7ff416beab80 10 client.15054 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-10T00:55:41.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.677+0000 7ff416beab80 20 client.15054 populate_metadata read hostname 'smithi037' 2024-06-10T00:55:41.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.677+0000 7ff416beab80 1 --2- 172.21.15.37:0/2516757586 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55cb242d60b0 0x55cb242d7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:41.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.677+0000 7ff416beab80 4 client.15054 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:55:41.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.677+0000 7ff416beab80 1 -- 172.21.15.37:0/2516757586 --> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb2421ad00 con 0x55cb242d60b0 2024-06-10T00:55:41.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.678+0000 7ff409902700 1 --2- 172.21.15.37:0/2516757586 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55cb242d60b0 0x55cb242d7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:41.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.678+0000 7ff409902700 1 --2- 172.21.15.37:0/2516757586 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55cb242d60b0 0x55cb242d7520 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=86b2a1ced871c6e6 server_cookie=94f794e5c0c520d5 in_seq=0 out_seq=0 2024-06-10T00:55:41.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.678+0000 7ff3e9ffb700 10 client.15054 ms_handle_connect on v2:172.21.15.167:6836/2720148065 2024-06-10T00:55:41.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.679+0000 7ff3e9ffb700 1 -- 172.21.15.37:0/2516757586 <== mds.2 v2:172.21.15.167:6836/2720148065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cb2421ad00 con 0x55cb242d60b0 2024-06-10T00:55:41.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.679+0000 7ff3e9ffb700 10 client.15054 handle_command_reply: tid=0 2024-06-10T00:55:41.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.679+0000 7ff416beab80 1 client.15054 shutdown 2024-06-10T00:55:41.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.679+0000 7ff416beab80 1 -- 172.21.15.37:0/2516757586 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb242bc750 msgr2=0x55cb242c8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:41.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.679+0000 7ff416beab80 1 --2- 172.21.15.37:0/2516757586 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb242bc750 0x55cb242c8520 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4007ab0 tx=0x7ff3f4004b40 comp rx=0 tx=0).stop 2024-06-10T00:55:41.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.679+0000 7ff416beab80 1 -- 172.21.15.37:0/2516757586 shutdown_connections 2024-06-10T00:55:41.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.679+0000 7ff416beab80 1 --2- 172.21.15.37:0/2516757586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb242c2830 0x55cb242c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.679+0000 7ff416beab80 1 --2- 172.21.15.37:0/2516757586 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb242bc750 0x55cb242c8520 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.679+0000 7ff416beab80 1 --2- 172.21.15.37:0/2516757586 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55cb242d60b0 0x55cb242d7520 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.679+0000 7ff416beab80 1 --2- 172.21.15.37:0/2516757586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb242c1f10 0x55cb242c5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.679+0000 7ff416beab80 1 -- 172.21.15.37:0/2516757586 >> 172.21.15.37:0/2516757586 conn(0x55cb241e9030 msgr2=0x55cb242b66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:41.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.679+0000 7ff3e9ffb700 0 client.15054 ms_handle_reset on v2:172.21.15.167:6836/2720148065 2024-06-10T00:55:41.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.679+0000 7ff416beab80 1 -- 172.21.15.37:0/2516757586 shutdown_connections 2024-06-10T00:55:41.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.680+0000 7ff416beab80 1 -- 172.21.15.37:0/2516757586 wait complete. 2024-06-10T00:55:41.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.680+0000 7ff416beab80 20 client.15054 trim_cache size 0 max 16384 2024-06-10T00:55:41.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.701+0000 7ff416beab80 1 Processor -- start 2024-06-10T00:55:41.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.701+0000 7ff416beab80 1 -- start start 2024-06-10T00:55:41.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.701+0000 7ff416beab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb242bc750 0x55cb242bc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:41.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.701+0000 7ff416beab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb242c1f10 0x55cb242c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:41.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.701+0000 7ff416beab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb242c2830 0x55cb242c3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:41.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.701+0000 7ff416beab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cb242d5860 con 0x55cb242c2830 2024-06-10T00:55:41.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.701+0000 7ff416beab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cb242d5f30 con 0x55cb242c1f10 2024-06-10T00:55:41.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.701+0000 7ff416beab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cb242d6600 con 0x55cb242bc750 2024-06-10T00:55:41.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.702+0000 7ff40a904700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb242c2830 0x55cb242c3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:41.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.702+0000 7ff40a904700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb242c2830 0x55cb242c3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47622/0 (socket says 172.21.15.37:47622) 2024-06-10T00:55:41.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.702+0000 7ff40a904700 1 -- 172.21.15.37:0/1313146461 learned_addr learned my addr 172.21.15.37:0/1313146461 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:41.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.702+0000 7ff409902700 1 --2- 172.21.15.37:0/1313146461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb242c1f10 0x55cb242c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:41.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.702+0000 7ff40a103700 1 --2- 172.21.15.37:0/1313146461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb242bc750 0x55cb242bc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:41.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.702+0000 7ff40a904700 1 -- 172.21.15.37:0/1313146461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb242bc750 msgr2=0x55cb242bc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:55:41.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.702+0000 7ff40a904700 1 --2- 172.21.15.37:0/1313146461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb242bc750 0x55cb242bc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.702+0000 7ff40a904700 1 -- 172.21.15.37:0/1313146461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb242c1f10 msgr2=0x55cb242c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:55:41.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.702+0000 7ff40a904700 1 --2- 172.21.15.37:0/1313146461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb242c1f10 0x55cb242c5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.702+0000 7ff40a904700 1 -- 172.21.15.37:0/1313146461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb242b8a00 con 0x55cb242c2830 2024-06-10T00:55:41.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.702+0000 7ff409902700 1 --2- 172.21.15.37:0/1313146461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb242c1f10 0x55cb242c5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:55:41.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.702+0000 7ff40a904700 1 --2- 172.21.15.37:0/1313146461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb242c2830 0x55cb242c3320 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc00b470 tx=0x7ff3fc0051f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3897fdc2a2cae0b8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:41.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.702+0000 7ff3ebfff700 1 -- 172.21.15.37:0/1313146461 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc028070 con 0x55cb242c2830 2024-06-10T00:55:41.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.702+0000 7ff3ebfff700 1 -- 172.21.15.37:0/1313146461 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3fc015040 con 0x55cb242c2830 2024-06-10T00:55:41.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.702+0000 7ff3ebfff700 1 -- 172.21.15.37:0/1313146461 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc003ab0 con 0x55cb242c2830 2024-06-10T00:55:41.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.703+0000 7ff416beab80 1 -- 172.21.15.37:0/1313146461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb242c2830 msgr2=0x55cb242c3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:41.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.703+0000 7ff416beab80 1 --2- 172.21.15.37:0/1313146461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb242c2830 0x55cb242c3320 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc00b470 tx=0x7ff3fc0051f0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.703+0000 7ff416beab80 1 -- 172.21.15.37:0/1313146461 shutdown_connections 2024-06-10T00:55:41.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.703+0000 7ff416beab80 1 --2- 172.21.15.37:0/1313146461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb242c2830 0x55cb242c3320 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.703+0000 7ff416beab80 1 --2- 172.21.15.37:0/1313146461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb242bc750 0x55cb242bc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.703+0000 7ff416beab80 1 --2- 172.21.15.37:0/1313146461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb242c1f10 0x55cb242c5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.703+0000 7ff416beab80 1 -- 172.21.15.37:0/1313146461 >> 172.21.15.37:0/1313146461 conn(0x55cb241e9030 msgr2=0x55cb242c2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:41.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.703+0000 7ff416beab80 1 -- 172.21.15.37:0/1313146461 shutdown_connections 2024-06-10T00:55:41.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.703+0000 7ff416beab80 1 -- 172.21.15.37:0/1313146461 wait complete. 2024-06-10T00:55:41.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.703+0000 7ff416beab80 1 -- start start 2024-06-10T00:55:41.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.703+0000 7ff416beab80 1 Processor -- start 2024-06-10T00:55:41.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.704+0000 7ff416beab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb242bc750 0x55cb242e11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:41.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.704+0000 7ff416beab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb242c1f10 0x55cb242b6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:41.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.704+0000 7ff416beab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb242c2830 0x55cb242b6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:41.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.704+0000 7ff416beab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cb242d73a0 con 0x55cb242bc750 2024-06-10T00:55:41.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.704+0000 7ff416beab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cb242d6cd0 con 0x55cb242c2830 2024-06-10T00:55:41.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.704+0000 7ff416beab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cb242d7a70 con 0x55cb242c1f10 2024-06-10T00:55:41.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.704+0000 7ff416beab80 10 client.0 _ll_register_callbacks cb 0x55cb241a3560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:55:41.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.704+0000 7ff40a103700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb242bc750 0x55cb242e11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:41.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.704+0000 7ff40a103700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb242bc750 0x55cb242e11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47630/0 (socket says 172.21.15.37:47630) 2024-06-10T00:55:41.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.704+0000 7ff40a103700 1 -- 172.21.15.37:0/1158121143 learned_addr learned my addr 172.21.15.37:0/1158121143 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:41.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.704+0000 7ff409902700 1 --2- 172.21.15.37:0/1158121143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb242c1f10 0x55cb242b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:41.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.704+0000 7ff40a904700 1 --2- 172.21.15.37:0/1158121143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb242c2830 0x55cb242b6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:41.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.704+0000 7ff40a103700 1 -- 172.21.15.37:0/1158121143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb242c1f10 msgr2=0x55cb242b6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:55:41.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.704+0000 7ff40a103700 1 --2- 172.21.15.37:0/1158121143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb242c1f10 0x55cb242b6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.704+0000 7ff40a103700 1 -- 172.21.15.37:0/1158121143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb242c2830 msgr2=0x55cb242b6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:55:41.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.704+0000 7ff40a103700 1 --2- 172.21.15.37:0/1158121143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb242c2830 0x55cb242b6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.704+0000 7ff40a103700 1 -- 172.21.15.37:0/1158121143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb242b7340 con 0x55cb242bc750 2024-06-10T00:55:41.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.704+0000 7ff409902700 1 --2- 172.21.15.37:0/1158121143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb242c1f10 0x55cb242b6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:55:41.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.704+0000 7ff40a904700 1 --2- 172.21.15.37:0/1158121143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb242c2830 0x55cb242b6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:55:41.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.704+0000 7ff40a103700 1 --2- 172.21.15.37:0/1158121143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb242bc750 0x55cb242e11c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4015040 tx=0x7ff3f4016750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4ab793278ba804 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:41.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.704+0000 7ff3e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:55:41.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.705+0000 7ff3e9ffb700 1 -- 172.21.15.37:0/1158121143 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4026040 con 0x55cb242bc750 2024-06-10T00:55:41.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.705+0000 7ff416beab80 1 -- 172.21.15.37:0/1158121143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3fc015040 con 0x55cb242bc750 2024-06-10T00:55:41.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.705+0000 7ff3e9ffb700 1 -- 172.21.15.37:0/1158121143 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7ff3f4004bb0 con 0x55cb242bc750 2024-06-10T00:55:41.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.705+0000 7ff3e9ffb700 1 -- 172.21.15.37:0/1158121143 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff3f403e050 con 0x55cb242bc750 2024-06-10T00:55:41.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.705+0000 7ff416beab80 10 client.15060 fetch_fsmap learned FSMap version 26 2024-06-10T00:55:41.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.705+0000 7ff416beab80 1 -- 172.21.15.37:0/1158121143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cb24341ad0 con 0x55cb242bc750 2024-06-10T00:55:41.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.705+0000 7ff3e9ffb700 1 -- 172.21.15.37:0/1158121143 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3fc015040 con 0x55cb242bc750 2024-06-10T00:55:41.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.705+0000 7ff3e9ffb700 1 -- 172.21.15.37:0/1158121143 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ff3f4022470 con 0x55cb242bc750 2024-06-10T00:55:41.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.705+0000 7ff416beab80 10 client.15060 fetch_fsmap finished waiting for FSMap version 26 2024-06-10T00:55:41.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.705+0000 7ff416beab80 10 client.15060 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-10T00:55:41.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.705+0000 7ff416beab80 20 client.15060 populate_metadata read hostname 'smithi037' 2024-06-10T00:55:41.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.705+0000 7ff416beab80 1 --2- 172.21.15.37:0/1158121143 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55cb24346750 0x55cb24345460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:41.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.705+0000 7ff416beab80 4 client.15060 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T00:55:41.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.705+0000 7ff416beab80 1 -- 172.21.15.37:0/1158121143 --> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] -- command(tid 0: {"prefix": "status"}) -- 0x55cb24215280 con 0x55cb24346750 2024-06-10T00:55:41.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.706+0000 7ff409902700 1 --2- 172.21.15.37:0/1158121143 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55cb24346750 0x55cb24345460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:41.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.706+0000 7ff409902700 1 --2- 172.21.15.37:0/1158121143 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55cb24346750 0x55cb24345460 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e4bd5887031e9de server_cookie=373b6c865fb6ee55 in_seq=0 out_seq=0 2024-06-10T00:55:41.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.706+0000 7ff3e9ffb700 10 client.15060 ms_handle_connect on v2:172.21.15.167:6836/2720148065 2024-06-10T00:55:41.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.706+0000 7ff3e9ffb700 1 -- 172.21.15.37:0/1158121143 <== mds.2 v2:172.21.15.167:6836/2720148065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55cb24215280 con 0x55cb24346750 2024-06-10T00:55:41.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.706+0000 7ff3e9ffb700 10 client.15060 handle_command_reply: tid=0 2024-06-10T00:55:41.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.706+0000 7ff416beab80 1 client.15060 shutdown 2024-06-10T00:55:41.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.706+0000 7ff416beab80 1 -- 172.21.15.37:0/1158121143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb242bc750 msgr2=0x55cb242e11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:41.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.706+0000 7ff416beab80 1 --2- 172.21.15.37:0/1158121143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb242bc750 0x55cb242e11c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4015040 tx=0x7ff3f4016750 comp rx=0 tx=0).stop 2024-06-10T00:55:41.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.706+0000 7ff416beab80 1 -- 172.21.15.37:0/1158121143 shutdown_connections 2024-06-10T00:55:41.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.706+0000 7ff416beab80 1 --2- 172.21.15.37:0/1158121143 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55cb24346750 0x55cb24345460 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.706+0000 7ff416beab80 1 --2- 172.21.15.37:0/1158121143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb242c2830 0x55cb242b6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.706+0000 7ff416beab80 1 --2- 172.21.15.37:0/1158121143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb242bc750 0x55cb242e11c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.706+0000 7ff416beab80 1 --2- 172.21.15.37:0/1158121143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cb242c1f10 0x55cb242b6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.706+0000 7ff3e9ffb700 0 client.15060 ms_handle_reset on v2:172.21.15.167:6836/2720148065 2024-06-10T00:55:41.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.706+0000 7ff416beab80 1 -- 172.21.15.37:0/1158121143 >> 172.21.15.37:0/1158121143 conn(0x55cb241e9030 msgr2=0x55cb242c45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:41.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.707+0000 7ff416beab80 1 -- 172.21.15.37:0/1158121143 shutdown_connections 2024-06-10T00:55:41.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.707+0000 7ff416beab80 1 -- 172.21.15.37:0/1158121143 wait complete. 2024-06-10T00:55:41.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.707+0000 7ff416beab80 20 client.15060 trim_cache size 0 max 16384 2024-06-10T00:55:41.713 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T00:55:41.714 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "68793514-26c2-11ef-bc9b-c7b262605968", 2024-06-10T00:55:41.714 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 2, 2024-06-10T00:55:41.714 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24451, 2024-06-10T00:55:41.714 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-06-10T00:55:41.714 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-06-10T00:55:41.714 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-06-10T00:55:41.714 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 122.42730973, 2024-06-10T00:55:41.714 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-06-10T00:55:41.714 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 100, 2024-06-10T00:55:41.714 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 98, 2024-06-10T00:55:41.714 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 217.20634865 2024-06-10T00:55:41.714 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T00:55:41.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.707+0000 7ff40c367700 1 -- 172.21.15.37:0/2513102367 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff3ec0a2c20 msgr2=0x7ff3ec0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:41.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.707+0000 7ff40c367700 1 --2- 172.21.15.37:0/2513102367 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff3ec0a2c20 0x7ff3ec0a50e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4003f90 tx=0x7ff3f4009180 comp rx=0 tx=0).stop 2024-06-10T00:55:41.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.707+0000 7ff40c367700 1 -- 172.21.15.37:0/2513102367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4041017f0 msgr2=0x7ff40419b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:41.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.707+0000 7ff40c367700 1 --2- 172.21.15.37:0/2513102367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4041017f0 0x7ff40419b330 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8006010 tx=0x7ff3f8009f80 comp rx=0 tx=0).stop 2024-06-10T00:55:41.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.707+0000 7ff40c367700 1 -- 172.21.15.37:0/2513102367 shutdown_connections 2024-06-10T00:55:41.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.707+0000 7ff40c367700 1 --2- 172.21.15.37:0/2513102367 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff3ec0a2c20 0x7ff3ec0a50e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.707+0000 7ff40c367700 1 --2- 172.21.15.37:0/2513102367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff404107500 0x7ff40419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.707+0000 7ff40c367700 1 --2- 172.21.15.37:0/2513102367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff404100e30 0x7ff40419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.707+0000 7ff40c367700 1 --2- 172.21.15.37:0/2513102367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4041017f0 0x7ff40419b330 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:41.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.707+0000 7ff40c367700 1 -- 172.21.15.37:0/2513102367 >> 172.21.15.37:0/2513102367 conn(0x7ff404076730 msgr2=0x7ff40410d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:41.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.707+0000 7ff40c367700 1 -- 172.21.15.37:0/2513102367 shutdown_connections 2024-06-10T00:55:41.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:41.707+0000 7ff40c367700 1 -- 172.21.15.37:0/2513102367 wait complete. 2024-06-10T00:55:41.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:41 smithi094 ceph-mon[36708]: pgmap v400: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:55:42.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:41 smithi167 ceph-mon[36917]: pgmap v400: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:55:42.480 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-10T00:55:42.752 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:55:43.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:43 smithi094 ceph-mon[36708]: pgmap v401: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:55:43.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:43 smithi037 ceph-mon[32192]: pgmap v401: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:55:44.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:43 smithi167 ceph-mon[36917]: pgmap v401: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:55:44.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.490+0000 7f2a745fe700 1 -- 172.21.15.37:0/666241489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a6c1066f0 msgr2=0x7f2a6c10e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:44.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.490+0000 7f2a745fe700 1 --2- 172.21.15.37:0/666241489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a6c1066f0 0x7f2a6c10e2c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2a6400b0d0 tx=0x7f2a6400b3e0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.490+0000 7f2a745fe700 1 -- 172.21.15.37:0/666241489 shutdown_connections 2024-06-10T00:55:44.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.490+0000 7f2a745fe700 1 --2- 172.21.15.37:0/666241489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a6c1066f0 0x7f2a6c10e2c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.490+0000 7f2a745fe700 1 --2- 172.21.15.37:0/666241489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a6c101610 0x7f2a6c1019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.490+0000 7f2a745fe700 1 --2- 172.21.15.37:0/666241489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a6c101fc0 0x7f2a6c106010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.490+0000 7f2a745fe700 1 -- 172.21.15.37:0/666241489 >> 172.21.15.37:0/666241489 conn(0x7f2a6c078430 msgr2=0x7f2a6c078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:44.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.490+0000 7f2a745fe700 1 -- 172.21.15.37:0/666241489 shutdown_connections 2024-06-10T00:55:44.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.490+0000 7f2a745fe700 1 -- 172.21.15.37:0/666241489 wait complete. 2024-06-10T00:55:44.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.490+0000 7f2a745fe700 1 Processor -- start 2024-06-10T00:55:44.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.490+0000 7f2a745fe700 1 -- start start 2024-06-10T00:55:44.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.491+0000 7f2a745fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a6c101610 0x7f2a6c1030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:44.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.491+0000 7f2a745fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a6c101fc0 0x7f2a6c1035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:44.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.491+0000 7f2a745fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a6c1066f0 0x7f2a6c103b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:44.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.491+0000 7f2a745fe700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c111700 con 0x7f2a6c1066f0 2024-06-10T00:55:44.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.491+0000 7f2a745fe700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c111580 con 0x7f2a6c101fc0 2024-06-10T00:55:44.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.491+0000 7f2a745fe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c111880 con 0x7f2a6c101610 2024-06-10T00:55:44.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.491+0000 7f2a72b9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a6c1066f0 0x7f2a6c103b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:44.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.491+0000 7f2a72b9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a6c1066f0 0x7f2a6c103b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49988/0 (socket says 172.21.15.37:49988) 2024-06-10T00:55:44.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.491+0000 7f2a72b9b700 1 -- 172.21.15.37:0/1430882174 learned_addr learned my addr 172.21.15.37:0/1430882174 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:44.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.491+0000 7f2a7239a700 1 --2- 172.21.15.37:0/1430882174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a6c101610 0x7f2a6c1030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:44.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.491+0000 7f2a71b99700 1 --2- 172.21.15.37:0/1430882174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a6c101fc0 0x7f2a6c1035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:44.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.491+0000 7f2a72b9b700 1 -- 172.21.15.37:0/1430882174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a6c101610 msgr2=0x7f2a6c1030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:44.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.491+0000 7f2a72b9b700 1 --2- 172.21.15.37:0/1430882174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a6c101610 0x7f2a6c1030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.491+0000 7f2a72b9b700 1 -- 172.21.15.37:0/1430882174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a6c101fc0 msgr2=0x7f2a6c1035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:44.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.491+0000 7f2a72b9b700 1 --2- 172.21.15.37:0/1430882174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a6c101fc0 0x7f2a6c1035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.491+0000 7f2a72b9b700 1 -- 172.21.15.37:0/1430882174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a6c1a7150 con 0x7f2a6c1066f0 2024-06-10T00:55:44.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.491+0000 7f2a72b9b700 1 --2- 172.21.15.37:0/1430882174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a6c1066f0 0x7f2a6c103b30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2a64006070 tx=0x7f2a64004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:44.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.491+0000 7f2a6b7fe700 1 -- 172.21.15.37:0/1430882174 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64021070 con 0x7f2a6c1066f0 2024-06-10T00:55:44.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.492+0000 7f2a745fe700 1 -- 172.21.15.37:0/1430882174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a6c1a73e0 con 0x7f2a6c1066f0 2024-06-10T00:55:44.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.492+0000 7f2a6b7fe700 1 -- 172.21.15.37:0/1430882174 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a64007b80 con 0x7f2a6c1066f0 2024-06-10T00:55:44.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.492+0000 7f2a6b7fe700 1 -- 172.21.15.37:0/1430882174 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a640052a0 con 0x7f2a6c1066f0 2024-06-10T00:55:44.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.492+0000 7f2a745fe700 1 -- 172.21.15.37:0/1430882174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a6c1a7890 con 0x7f2a6c1066f0 2024-06-10T00:55:44.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.494+0000 7f2a6b7fe700 1 -- 172.21.15.37:0/1430882174 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2a64024070 con 0x7f2a6c1066f0 2024-06-10T00:55:44.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.494+0000 7f2a6b7fe700 1 --2- 172.21.15.37:0/1430882174 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2a540a2c20 0x7f2a540a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:44.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.495+0000 7f2a7239a700 1 --2- 172.21.15.37:0/1430882174 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2a540a2c20 0x7f2a540a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:44.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.495+0000 7f2a6b7fe700 1 -- 172.21.15.37:0/1430882174 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f2a6400f020 con 0x7f2a6c1066f0 2024-06-10T00:55:44.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.495+0000 7f2a7239a700 1 --2- 172.21.15.37:0/1430882174 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2a540a2c20 0x7f2a540a50e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c004630 tx=0x7f2a5c00b3c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:44.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.495+0000 7f2a7ee81b80 1 Processor -- start 2024-06-10T00:55:44.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.495+0000 7f2a7ee81b80 1 -- start start 2024-06-10T00:55:44.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.496+0000 7f2a7ee81b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3890dd5f0 0x55b3890dd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:44.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.496+0000 7f2a7ee81b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3890ddf10 0x55b3890d7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:44.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.496+0000 7f2a7ee81b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3890d7ed0 0x55b3890d82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:44.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.496+0000 7f2a7ee81b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b3890297c0 con 0x55b3890d7ed0 2024-06-10T00:55:44.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.496+0000 7f2a7ee81b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b388fd16d0 con 0x55b3890ddf10 2024-06-10T00:55:44.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.496+0000 7f2a7ee81b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b389025780 con 0x55b3890dd5f0 2024-06-10T00:55:44.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.496+0000 7f2a72b9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3890d7ed0 0x55b3890d82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:44.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.496+0000 7f2a7239a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3890dd5f0 0x55b3890dd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:44.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.496+0000 7f2a71b99700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3890ddf10 0x55b3890d7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:44.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.496+0000 7f2a72b9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3890d7ed0 0x55b3890d82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49992/0 (socket says 172.21.15.37:49992) 2024-06-10T00:55:44.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.496+0000 7f2a72b9b700 1 -- 172.21.15.37:0/4034699717 learned_addr learned my addr 172.21.15.37:0/4034699717 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:44.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.496+0000 7f2a72b9b700 1 -- 172.21.15.37:0/4034699717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3890dd5f0 msgr2=0x55b3890dd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:55:44.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.496+0000 7f2a72b9b700 1 --2- 172.21.15.37:0/4034699717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3890dd5f0 0x55b3890dd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.496+0000 7f2a72b9b700 1 -- 172.21.15.37:0/4034699717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3890ddf10 msgr2=0x55b3890d7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:55:44.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.496+0000 7f2a72b9b700 1 --2- 172.21.15.37:0/4034699717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3890ddf10 0x55b3890d7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.496+0000 7f2a72b9b700 1 -- 172.21.15.37:0/4034699717 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b388fc7f00 con 0x55b3890d7ed0 2024-06-10T00:55:44.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.496+0000 7f2a72b9b700 1 --2- 172.21.15.37:0/4034699717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3890d7ed0 0x55b3890d82f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2a640998b0 tx=0x7f2a640097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b31706ec8f4cfc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:44.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.497+0000 7f2a68ff9700 1 -- 172.21.15.37:0/4034699717 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6409d5e0 con 0x55b3890d7ed0 2024-06-10T00:55:44.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.497+0000 7f2a68ff9700 1 -- 172.21.15.37:0/4034699717 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a6409cee0 con 0x55b3890d7ed0 2024-06-10T00:55:44.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.497+0000 7f2a68ff9700 1 -- 172.21.15.37:0/4034699717 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a640a5630 con 0x55b3890d7ed0 2024-06-10T00:55:44.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.497+0000 7f2a7ee81b80 1 -- 172.21.15.37:0/4034699717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3890d7ed0 msgr2=0x55b3890d82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:44.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.497+0000 7f2a7ee81b80 1 --2- 172.21.15.37:0/4034699717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3890d7ed0 0x55b3890d82f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2a640998b0 tx=0x7f2a640097a0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.497+0000 7f2a7ee81b80 1 -- 172.21.15.37:0/4034699717 shutdown_connections 2024-06-10T00:55:44.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.497+0000 7f2a7ee81b80 1 --2- 172.21.15.37:0/4034699717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3890d7ed0 0x55b3890d82f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.497+0000 7f2a7ee81b80 1 --2- 172.21.15.37:0/4034699717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3890dd5f0 0x55b3890dd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.497+0000 7f2a7ee81b80 1 --2- 172.21.15.37:0/4034699717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3890ddf10 0x55b3890d7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.497+0000 7f2a7ee81b80 1 -- 172.21.15.37:0/4034699717 >> 172.21.15.37:0/4034699717 conn(0x55b389002c70 msgr2=0x55b3890d2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:44.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.498+0000 7f2a7ee81b80 1 -- 172.21.15.37:0/4034699717 shutdown_connections 2024-06-10T00:55:44.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.498+0000 7f2a7ee81b80 1 -- 172.21.15.37:0/4034699717 wait complete. 2024-06-10T00:55:44.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.498+0000 7f2a7ee81b80 1 -- start start 2024-06-10T00:55:44.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.498+0000 7f2a7ee81b80 1 Processor -- start 2024-06-10T00:55:44.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.498+0000 7f2a7ee81b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3890d7ed0 0x55b3890e3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:44.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.498+0000 7f2a7ee81b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3890dd5f0 0x55b3890e0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:44.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.498+0000 7f2a7ee81b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3890ddf10 0x55b3890e1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:44.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.498+0000 7f2a7ee81b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b3890e46a0 con 0x55b3890dd5f0 2024-06-10T00:55:44.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.498+0000 7f2a7ee81b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b388fc9ba0 con 0x55b3890d7ed0 2024-06-10T00:55:44.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.498+0000 7f2a7ee81b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b3890e4820 con 0x55b3890ddf10 2024-06-10T00:55:44.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.498+0000 7f2a7ee81b80 10 client.0 _ll_register_callbacks cb 0x55b388fc38d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:55:44.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.498+0000 7f2a71b99700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3890dd5f0 0x55b3890e0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:44.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.498+0000 7f2a71b99700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3890dd5f0 0x55b3890e0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49998/0 (socket says 172.21.15.37:49998) 2024-06-10T00:55:44.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.498+0000 7f2a71b99700 1 -- 172.21.15.37:0/718885455 learned_addr learned my addr 172.21.15.37:0/718885455 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:44.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.498+0000 7f2a7239a700 1 --2- 172.21.15.37:0/718885455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3890d7ed0 0x55b3890e3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:44.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.499+0000 7f2a72b9b700 1 --2- 172.21.15.37:0/718885455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3890ddf10 0x55b3890e1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:44.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.499+0000 7f2a71b99700 1 -- 172.21.15.37:0/718885455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3890ddf10 msgr2=0x55b3890e1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:55:44.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.499+0000 7f2a71b99700 1 --2- 172.21.15.37:0/718885455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3890ddf10 0x55b3890e1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.499+0000 7f2a71b99700 1 -- 172.21.15.37:0/718885455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3890d7ed0 msgr2=0x55b3890e3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:55:44.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.499+0000 7f2a71b99700 1 --2- 172.21.15.37:0/718885455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3890d7ed0 0x55b3890e3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.499+0000 7f2a71b99700 1 -- 172.21.15.37:0/718885455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b388f78880 con 0x55b3890dd5f0 2024-06-10T00:55:44.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.499+0000 7f2a71b99700 1 --2- 172.21.15.37:0/718885455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3890dd5f0 0x55b3890e0fc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000be70 tx=0x7f2a60011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10df331097a6873a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:44.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.499+0000 7f2a51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:55:44.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.499+0000 7f2a51ffb700 1 -- 172.21.15.37:0/718885455 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a60004830 con 0x55b3890dd5f0 2024-06-10T00:55:44.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.499+0000 7f2a51ffb700 1 -- 172.21.15.37:0/718885455 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f2a600049d0 con 0x55b3890dd5f0 2024-06-10T00:55:44.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.499+0000 7f2a7ee81b80 1 -- 172.21.15.37:0/718885455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a6409cee0 con 0x55b3890dd5f0 2024-06-10T00:55:44.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.499+0000 7f2a51ffb700 1 -- 172.21.15.37:0/718885455 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2a60025050 con 0x55b3890dd5f0 2024-06-10T00:55:44.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.499+0000 7f2a7ee81b80 10 client.15072 fetch_fsmap learned FSMap version 26 2024-06-10T00:55:44.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.499+0000 7f2a7ee81b80 1 -- 172.21.15.37:0/718885455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b3890ef020 con 0x55b3890dd5f0 2024-06-10T00:55:44.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.499+0000 7f2a51ffb700 1 -- 172.21.15.37:0/718885455 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a6409cee0 con 0x55b3890dd5f0 2024-06-10T00:55:44.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.500+0000 7f2a51ffb700 1 -- 172.21.15.37:0/718885455 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f2a6001a930 con 0x55b3890dd5f0 2024-06-10T00:55:44.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.500+0000 7f2a7ee81b80 10 client.15072 fetch_fsmap finished waiting for FSMap version 26 2024-06-10T00:55:44.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.500+0000 7f2a7ee81b80 10 client.15072 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-10T00:55:44.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.500+0000 7f2a7ee81b80 20 client.15072 populate_metadata read hostname 'smithi037' 2024-06-10T00:55:44.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.500+0000 7f2a7ee81b80 1 --2- 172.21.15.37:0/718885455 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55b3890f16c0 0x55b3890f2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:44.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.500+0000 7f2a7ee81b80 4 client.15072 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:55:44.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.500+0000 7f2a7ee81b80 1 -- 172.21.15.37:0/718885455 --> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b389036cc0 con 0x55b3890f16c0 2024-06-10T00:55:44.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.500+0000 7f2a7239a700 1 --2- 172.21.15.37:0/718885455 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55b3890f16c0 0x55b3890f2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:44.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.501+0000 7f2a51ffb700 10 client.15072 ms_handle_connect on v2:172.21.15.94:6838/946269331 2024-06-10T00:55:44.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.501+0000 7f2a7239a700 1 --2- 172.21.15.37:0/718885455 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55b3890f16c0 0x55b3890f2b30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ccd8c4bca358b699 server_cookie=364606bd05b1649a in_seq=0 out_seq=0 2024-06-10T00:55:44.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.501+0000 7f2a51ffb700 1 -- 172.21.15.37:0/718885455 <== mds.1 v2:172.21.15.94:6838/946269331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b389036cc0 con 0x55b3890f16c0 2024-06-10T00:55:44.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.501+0000 7f2a51ffb700 10 client.15072 handle_command_reply: tid=0 2024-06-10T00:55:44.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.502+0000 7f2a7ee81b80 1 client.15072 shutdown 2024-06-10T00:55:44.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.502+0000 7f2a7ee81b80 1 -- 172.21.15.37:0/718885455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3890dd5f0 msgr2=0x55b3890e0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:44.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.502+0000 7f2a7ee81b80 1 --2- 172.21.15.37:0/718885455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3890dd5f0 0x55b3890e0fc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000be70 tx=0x7f2a60011720 comp rx=0 tx=0).stop 2024-06-10T00:55:44.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.502+0000 7f2a7ee81b80 1 -- 172.21.15.37:0/718885455 shutdown_connections 2024-06-10T00:55:44.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.502+0000 7f2a7ee81b80 1 --2- 172.21.15.37:0/718885455 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55b3890f16c0 0x55b3890f2b30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.502+0000 7f2a7ee81b80 1 --2- 172.21.15.37:0/718885455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3890ddf10 0x55b3890e1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.502+0000 7f2a7ee81b80 1 --2- 172.21.15.37:0/718885455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3890d7ed0 0x55b3890e3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.502+0000 7f2a7ee81b80 1 --2- 172.21.15.37:0/718885455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3890dd5f0 0x55b3890e0fc0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.502+0000 7f2a51ffb700 0 client.15072 ms_handle_reset on v2:172.21.15.94:6838/946269331 2024-06-10T00:55:44.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.502+0000 7f2a7ee81b80 1 -- 172.21.15.37:0/718885455 >> 172.21.15.37:0/718885455 conn(0x55b389002c70 msgr2=0x55b3890d6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:44.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.502+0000 7f2a7ee81b80 1 -- 172.21.15.37:0/718885455 shutdown_connections 2024-06-10T00:55:44.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.502+0000 7f2a7ee81b80 1 -- 172.21.15.37:0/718885455 wait complete. 2024-06-10T00:55:44.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.502+0000 7f2a7ee81b80 20 client.15072 trim_cache size 0 max 16384 2024-06-10T00:55:44.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.524+0000 7f2a7ee81b80 1 Processor -- start 2024-06-10T00:55:44.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.524+0000 7f2a7ee81b80 1 -- start start 2024-06-10T00:55:44.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.524+0000 7f2a7ee81b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3890d7ed0 0x55b3890d7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:44.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.524+0000 7f2a7ee81b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3890dd5f0 0x55b3890de3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:44.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.524+0000 7f2a7ee81b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3890ddf10 0x55b3890de910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:44.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.524+0000 7f2a7ee81b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b3890f0e70 con 0x55b3890d7ed0 2024-06-10T00:55:44.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.524+0000 7f2a7ee81b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b3890f1540 con 0x55b3890ddf10 2024-06-10T00:55:44.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.524+0000 7f2a7ee81b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b3890f1c10 con 0x55b3890dd5f0 2024-06-10T00:55:44.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.524+0000 7f2a7239a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3890d7ed0 0x55b3890d7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:44.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.524+0000 7f2a7239a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3890d7ed0 0x55b3890d7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50006/0 (socket says 172.21.15.37:50006) 2024-06-10T00:55:44.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.524+0000 7f2a7239a700 1 -- 172.21.15.37:0/3392556220 learned_addr learned my addr 172.21.15.37:0/3392556220 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:44.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.525+0000 7f2a72b9b700 1 --2- 172.21.15.37:0/3392556220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3890ddf10 0x55b3890de910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:44.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.525+0000 7f2a7239a700 1 -- 172.21.15.37:0/3392556220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3890dd5f0 msgr2=0x55b3890de3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:55:44.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.525+0000 7f2a71b99700 1 --2- 172.21.15.37:0/3392556220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3890dd5f0 0x55b3890de3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:44.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.525+0000 7f2a7239a700 1 --2- 172.21.15.37:0/3392556220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3890dd5f0 0x55b3890de3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.525+0000 7f2a7239a700 1 -- 172.21.15.37:0/3392556220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3890ddf10 msgr2=0x55b3890de910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:55:44.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.525+0000 7f2a7239a700 1 --2- 172.21.15.37:0/3392556220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3890ddf10 0x55b3890de910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.525+0000 7f2a7239a700 1 -- 172.21.15.37:0/3392556220 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3890df530 con 0x55b3890d7ed0 2024-06-10T00:55:44.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.525+0000 7f2a72b9b700 1 --2- 172.21.15.37:0/3392556220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3890ddf10 0x55b3890de910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:55:44.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.525+0000 7f2a71b99700 1 --2- 172.21.15.37:0/3392556220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3890dd5f0 0x55b3890de3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:55:44.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.525+0000 7f2a7239a700 1 --2- 172.21.15.37:0/3392556220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3890d7ed0 0x55b3890d7990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x55b3890f36d0 tx=0x7f2a5c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d37463cdaed3ce8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:44.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.525+0000 7f2a53fff700 1 -- 172.21.15.37:0/3392556220 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c0209a0 con 0x55b3890d7ed0 2024-06-10T00:55:44.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.525+0000 7f2a53fff700 1 -- 172.21.15.37:0/3392556220 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a5c020b40 con 0x55b3890d7ed0 2024-06-10T00:55:44.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.525+0000 7f2a53fff700 1 -- 172.21.15.37:0/3392556220 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c029750 con 0x55b3890d7ed0 2024-06-10T00:55:44.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.526+0000 7f2a7ee81b80 1 -- 172.21.15.37:0/3392556220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3890d7ed0 msgr2=0x55b3890d7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:44.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.526+0000 7f2a7ee81b80 1 --2- 172.21.15.37:0/3392556220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3890d7ed0 0x55b3890d7990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x55b3890f36d0 tx=0x7f2a5c013000 comp rx=0 tx=0).stop 2024-06-10T00:55:44.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.526+0000 7f2a7ee81b80 1 -- 172.21.15.37:0/3392556220 shutdown_connections 2024-06-10T00:55:44.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.526+0000 7f2a7ee81b80 1 --2- 172.21.15.37:0/3392556220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3890ddf10 0x55b3890de910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.526+0000 7f2a7ee81b80 1 --2- 172.21.15.37:0/3392556220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3890d7ed0 0x55b3890d7990 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.526+0000 7f2a7ee81b80 1 --2- 172.21.15.37:0/3392556220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3890dd5f0 0x55b3890de3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.526+0000 7f2a7ee81b80 1 -- 172.21.15.37:0/3392556220 >> 172.21.15.37:0/3392556220 conn(0x55b389002c70 msgr2=0x55b3890df8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:44.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.526+0000 7f2a7ee81b80 1 -- 172.21.15.37:0/3392556220 shutdown_connections 2024-06-10T00:55:44.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.526+0000 7f2a7ee81b80 1 -- 172.21.15.37:0/3392556220 wait complete. 2024-06-10T00:55:44.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.526+0000 7f2a7ee81b80 1 -- start start 2024-06-10T00:55:44.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.526+0000 7f2a7ee81b80 1 Processor -- start 2024-06-10T00:55:44.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.526+0000 7f2a7ee81b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3890d7ed0 0x55b3890fc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:44.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.527+0000 7f2a7ee81b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3890dd5f0 0x55b3890d6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:44.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.527+0000 7f2a7ee81b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3890ddf10 0x55b3890d7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:44.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.527+0000 7f2a7ee81b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b3890f29b0 con 0x55b3890dd5f0 2024-06-10T00:55:44.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.527+0000 7f2a7ee81b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b3890f22e0 con 0x55b3890ddf10 2024-06-10T00:55:44.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.527+0000 7f2a7ee81b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b3890f3080 con 0x55b3890d7ed0 2024-06-10T00:55:44.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.527+0000 7f2a7ee81b80 10 client.0 _ll_register_callbacks cb 0x55b388fc38d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:55:44.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.527+0000 7f2a71b99700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3890dd5f0 0x55b3890d6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:44.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.527+0000 7f2a71b99700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3890dd5f0 0x55b3890d6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50008/0 (socket says 172.21.15.37:50008) 2024-06-10T00:55:44.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.527+0000 7f2a71b99700 1 -- 172.21.15.37:0/1359403215 learned_addr learned my addr 172.21.15.37:0/1359403215 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:44.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.527+0000 7f2a7239a700 1 --2- 172.21.15.37:0/1359403215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3890d7ed0 0x55b3890fc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:44.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.527+0000 7f2a72b9b700 1 --2- 172.21.15.37:0/1359403215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3890ddf10 0x55b3890d7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:44.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.527+0000 7f2a71b99700 1 -- 172.21.15.37:0/1359403215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3890d7ed0 msgr2=0x55b3890fc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:55:44.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.527+0000 7f2a71b99700 1 --2- 172.21.15.37:0/1359403215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3890d7ed0 0x55b3890fc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.527+0000 7f2a71b99700 1 -- 172.21.15.37:0/1359403215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3890ddf10 msgr2=0x55b3890d7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:55:44.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.527+0000 7f2a71b99700 1 --2- 172.21.15.37:0/1359403215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3890ddf10 0x55b3890d7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.527+0000 7f2a71b99700 1 -- 172.21.15.37:0/1359403215 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3890fa2f0 con 0x55b3890dd5f0 2024-06-10T00:55:44.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.527+0000 7f2a72b9b700 1 --2- 172.21.15.37:0/1359403215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3890ddf10 0x55b3890d7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:55:44.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.527+0000 7f2a71b99700 1 --2- 172.21.15.37:0/1359403215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3890dd5f0 0x55b3890d6aa0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2a6001af40 tx=0x7f2a60005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd97c01bf52eabc2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:44.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.527+0000 7f2a51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:55:44.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.527+0000 7f2a51ffb700 1 -- 172.21.15.37:0/1359403215 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a60005540 con 0x55b3890dd5f0 2024-06-10T00:55:44.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.528+0000 7f2a51ffb700 1 -- 172.21.15.37:0/1359403215 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f2a60010700 con 0x55b3890dd5f0 2024-06-10T00:55:44.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.528+0000 7f2a7ee81b80 1 -- 172.21.15.37:0/1359403215 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a5c020b40 con 0x55b3890dd5f0 2024-06-10T00:55:44.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.528+0000 7f2a51ffb700 1 -- 172.21.15.37:0/1359403215 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2a60036050 con 0x55b3890dd5f0 2024-06-10T00:55:44.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.528+0000 7f2a7ee81b80 10 client.15078 fetch_fsmap learned FSMap version 26 2024-06-10T00:55:44.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.528+0000 7f2a7ee81b80 1 -- 172.21.15.37:0/1359403215 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b38915d060 con 0x55b3890dd5f0 2024-06-10T00:55:44.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.528+0000 7f2a51ffb700 1 -- 172.21.15.37:0/1359403215 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a5c020b40 con 0x55b3890dd5f0 2024-06-10T00:55:44.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.528+0000 7f2a51ffb700 1 -- 172.21.15.37:0/1359403215 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f2a60003680 con 0x55b3890dd5f0 2024-06-10T00:55:44.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.528+0000 7f2a7ee81b80 10 client.15078 fetch_fsmap finished waiting for FSMap version 26 2024-06-10T00:55:44.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.528+0000 7f2a7ee81b80 10 client.15078 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-10T00:55:44.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.528+0000 7f2a7ee81b80 20 client.15078 populate_metadata read hostname 'smithi037' 2024-06-10T00:55:44.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.528+0000 7f2a7ee81b80 1 --2- 172.21.15.37:0/1359403215 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55b389161c60 0x55b389160970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:44.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.528+0000 7f2a7ee81b80 4 client.15078 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T00:55:44.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.528+0000 7f2a7ee81b80 1 -- 172.21.15.37:0/1359403215 --> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] -- command(tid 0: {"prefix": "status"}) -- 0x55b389031240 con 0x55b389161c60 2024-06-10T00:55:44.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.529+0000 7f2a7239a700 1 --2- 172.21.15.37:0/1359403215 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55b389161c60 0x55b389160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:44.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.529+0000 7f2a7239a700 1 --2- 172.21.15.37:0/1359403215 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55b389161c60 0x55b389160970 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=50f7e302c0d38185 server_cookie=7da986ab12ec7938 in_seq=0 out_seq=0 2024-06-10T00:55:44.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.529+0000 7f2a51ffb700 10 client.15078 ms_handle_connect on v2:172.21.15.94:6838/946269331 2024-06-10T00:55:44.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.529+0000 7f2a51ffb700 1 -- 172.21.15.37:0/1359403215 <== mds.1 v2:172.21.15.94:6838/946269331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55b389031240 con 0x55b389161c60 2024-06-10T00:55:44.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.529+0000 7f2a51ffb700 10 client.15078 handle_command_reply: tid=0 2024-06-10T00:55:44.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.529+0000 7f2a7ee81b80 1 client.15078 shutdown 2024-06-10T00:55:44.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.529+0000 7f2a7ee81b80 1 -- 172.21.15.37:0/1359403215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3890dd5f0 msgr2=0x55b3890d6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:44.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.529+0000 7f2a7ee81b80 1 --2- 172.21.15.37:0/1359403215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3890dd5f0 0x55b3890d6aa0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2a6001af40 tx=0x7f2a60005fd0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.529+0000 7f2a7ee81b80 1 -- 172.21.15.37:0/1359403215 shutdown_connections 2024-06-10T00:55:44.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.529+0000 7f2a7ee81b80 1 --2- 172.21.15.37:0/1359403215 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55b389161c60 0x55b389160970 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.529+0000 7f2a7ee81b80 1 --2- 172.21.15.37:0/1359403215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3890ddf10 0x55b3890d7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.529+0000 7f2a7ee81b80 1 --2- 172.21.15.37:0/1359403215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3890d7ed0 0x55b3890fc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.529+0000 7f2a7ee81b80 1 --2- 172.21.15.37:0/1359403215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3890dd5f0 0x55b3890d6aa0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.529+0000 7f2a7ee81b80 1 -- 172.21.15.37:0/1359403215 >> 172.21.15.37:0/1359403215 conn(0x55b389002c70 msgr2=0x55b3890de3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:44.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.529+0000 7f2a51ffb700 0 client.15078 ms_handle_reset on v2:172.21.15.94:6838/946269331 2024-06-10T00:55:44.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.529+0000 7f2a7ee81b80 1 -- 172.21.15.37:0/1359403215 shutdown_connections 2024-06-10T00:55:44.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.530+0000 7f2a7ee81b80 1 -- 172.21.15.37:0/1359403215 wait complete. 2024-06-10T00:55:44.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.530+0000 7f2a7ee81b80 20 client.15078 trim_cache size 0 max 16384 2024-06-10T00:55:44.535 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T00:55:44.535 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "68793514-26c2-11ef-bc9b-c7b262605968", 2024-06-10T00:55:44.535 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 1, 2024-06-10T00:55:44.535 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24458, 2024-06-10T00:55:44.535 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-06-10T00:55:44.535 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-06-10T00:55:44.535 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-06-10T00:55:44.536 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 126.25680481400001, 2024-06-10T00:55:44.536 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-06-10T00:55:44.536 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 100, 2024-06-10T00:55:44.536 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 98, 2024-06-10T00:55:44.536 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 203.970561567 2024-06-10T00:55:44.536 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T00:55:44.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.530+0000 7f2a745fe700 1 -- 172.21.15.37:0/1430882174 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2a540a2c20 msgr2=0x7f2a540a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:44.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.530+0000 7f2a745fe700 1 --2- 172.21.15.37:0/1430882174 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2a540a2c20 0x7f2a540a50e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c004630 tx=0x7f2a5c00b3c0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.530+0000 7f2a745fe700 1 -- 172.21.15.37:0/1430882174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a6c1066f0 msgr2=0x7f2a6c103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:44.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.530+0000 7f2a745fe700 1 --2- 172.21.15.37:0/1430882174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a6c1066f0 0x7f2a6c103b30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2a64006070 tx=0x7f2a64004d00 comp rx=0 tx=0).stop 2024-06-10T00:55:44.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.530+0000 7f2a745fe700 1 -- 172.21.15.37:0/1430882174 shutdown_connections 2024-06-10T00:55:44.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.530+0000 7f2a745fe700 1 --2- 172.21.15.37:0/1430882174 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2a540a2c20 0x7f2a540a50e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.530+0000 7f2a745fe700 1 --2- 172.21.15.37:0/1430882174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a6c1066f0 0x7f2a6c103b30 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.530+0000 7f2a745fe700 1 --2- 172.21.15.37:0/1430882174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a6c101610 0x7f2a6c1030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.530+0000 7f2a745fe700 1 --2- 172.21.15.37:0/1430882174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a6c101fc0 0x7f2a6c1035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:44.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.530+0000 7f2a745fe700 1 -- 172.21.15.37:0/1430882174 >> 172.21.15.37:0/1430882174 conn(0x7f2a6c078430 msgr2=0x7f2a6c06ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:44.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.530+0000 7f2a745fe700 1 -- 172.21.15.37:0/1430882174 shutdown_connections 2024-06-10T00:55:44.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:44.530+0000 7f2a745fe700 1 -- 172.21.15.37:0/1430882174 wait complete. 2024-06-10T00:55:45.211 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-10T00:55:45.465 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:55:45.850 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:45 smithi037 ceph-mon[32192]: pgmap v402: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T00:55:45.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:45 smithi094 ceph-mon[36708]: pgmap v402: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T00:55:46.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:45 smithi167 ceph-mon[36917]: pgmap v402: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T00:55:47.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.220+0000 7fca1d531700 1 -- 172.21.15.37:0/2570611619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca1806a170 msgr2=0x7fca1806a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:47.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.220+0000 7fca1d531700 1 --2- 172.21.15.37:0/2570611619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca1806a170 0x7fca1806a5f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fca0c009990 tx=0x7fca0c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.220+0000 7fca1d531700 1 -- 172.21.15.37:0/2570611619 shutdown_connections 2024-06-10T00:55:47.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.220+0000 7fca1d531700 1 --2- 172.21.15.37:0/2570611619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca1806ab30 0x7fca1810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.220+0000 7fca1d531700 1 --2- 172.21.15.37:0/2570611619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca18108500 0x7fca181088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.220+0000 7fca1d531700 1 --2- 172.21.15.37:0/2570611619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca1806a170 0x7fca1806a5f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.220+0000 7fca1d531700 1 -- 172.21.15.37:0/2570611619 >> 172.21.15.37:0/2570611619 conn(0x7fca18078340 msgr2=0x7fca18078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:47.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.220+0000 7fca1d531700 1 -- 172.21.15.37:0/2570611619 shutdown_connections 2024-06-10T00:55:47.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.220+0000 7fca1d531700 1 -- 172.21.15.37:0/2570611619 wait complete. 2024-06-10T00:55:47.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.221+0000 7fca1d531700 1 Processor -- start 2024-06-10T00:55:47.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.221+0000 7fca1d531700 1 -- start start 2024-06-10T00:55:47.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.222+0000 7fca1d531700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca1806a170 0x7fca1819e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:47.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.222+0000 7fca1d531700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca1806ab30 0x7fca1819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:47.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.222+0000 7fca1d531700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca18108500 0x7fca18198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:47.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.222+0000 7fca1d531700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fca18112760 con 0x7fca1806a170 2024-06-10T00:55:47.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.222+0000 7fca1d531700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fca181125e0 con 0x7fca1806ab30 2024-06-10T00:55:47.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.222+0000 7fca1d531700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fca181128e0 con 0x7fca18108500 2024-06-10T00:55:47.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.222+0000 7fca16ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca1806a170 0x7fca1819e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:47.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.222+0000 7fca16ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca1806a170 0x7fca1819e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50022/0 (socket says 172.21.15.37:50022) 2024-06-10T00:55:47.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.222+0000 7fca16ffd700 1 -- 172.21.15.37:0/2365625798 learned_addr learned my addr 172.21.15.37:0/2365625798 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:47.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.222+0000 7fca167fc700 1 --2- 172.21.15.37:0/2365625798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca1806ab30 0x7fca1819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:47.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.222+0000 7fca177fe700 1 --2- 172.21.15.37:0/2365625798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca18108500 0x7fca18198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:47.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.222+0000 7fca16ffd700 1 -- 172.21.15.37:0/2365625798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca18108500 msgr2=0x7fca18198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:47.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.222+0000 7fca16ffd700 1 --2- 172.21.15.37:0/2365625798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca18108500 0x7fca18198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.222+0000 7fca16ffd700 1 -- 172.21.15.37:0/2365625798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca1806ab30 msgr2=0x7fca1819edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:47.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.222+0000 7fca16ffd700 1 --2- 172.21.15.37:0/2365625798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca1806ab30 0x7fca1819edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.222+0000 7fca16ffd700 1 -- 172.21.15.37:0/2365625798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca181991a0 con 0x7fca1806a170 2024-06-10T00:55:47.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.223+0000 7fca16ffd700 1 --2- 172.21.15.37:0/2365625798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca1806a170 0x7fca1819e880 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fca080060b0 tx=0x7fca0800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:47.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.223+0000 7fc9fffff700 1 -- 172.21.15.37:0/2365625798 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0801a070 con 0x7fca1806a170 2024-06-10T00:55:47.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.223+0000 7fca1d531700 1 -- 172.21.15.37:0/2365625798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca18199490 con 0x7fca1806a170 2024-06-10T00:55:47.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.223+0000 7fc9fffff700 1 -- 172.21.15.37:0/2365625798 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca0800ddd0 con 0x7fca1806a170 2024-06-10T00:55:47.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.223+0000 7fc9fffff700 1 -- 172.21.15.37:0/2365625798 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca080147a0 con 0x7fca1806a170 2024-06-10T00:55:47.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.223+0000 7fca1d531700 1 -- 172.21.15.37:0/2365625798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca181a5750 con 0x7fca1806a170 2024-06-10T00:55:47.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.225+0000 7fc9fffff700 1 -- 172.21.15.37:0/2365625798 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fca0801d070 con 0x7fca1806a170 2024-06-10T00:55:47.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.226+0000 7fc9fffff700 1 --2- 172.21.15.37:0/2365625798 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fca000a2c20 0x7fca000a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:47.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.226+0000 7fca167fc700 1 --2- 172.21.15.37:0/2365625798 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fca000a2c20 0x7fca000a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:47.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.226+0000 7fc9fffff700 1 -- 172.21.15.37:0/2365625798 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fca080cec80 con 0x7fca1806a170 2024-06-10T00:55:47.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.227+0000 7fca167fc700 1 --2- 172.21.15.37:0/2365625798 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fca000a2c20 0x7fca000a50e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00b5c0 tx=0x7fca0c005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:47.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.228+0000 7fca27db4b80 1 Processor -- start 2024-06-10T00:55:47.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.228+0000 7fca27db4b80 1 -- start start 2024-06-10T00:55:47.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.228+0000 7fca27db4b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b12e21e5f0 0x55b12e21e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:47.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.228+0000 7fca27db4b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b12e21ef10 0x55b12e218990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:47.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.228+0000 7fca27db4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b12e218ed0 0x55b12e2192f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:47.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.228+0000 7fca27db4b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b12e16a7c0 con 0x55b12e21e5f0 2024-06-10T00:55:47.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.228+0000 7fca27db4b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b12e1126d0 con 0x55b12e21ef10 2024-06-10T00:55:47.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.228+0000 7fca27db4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b12e166780 con 0x55b12e218ed0 2024-06-10T00:55:47.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.228+0000 7fca16ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b12e21e5f0 0x55b12e21e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:47.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.228+0000 7fca16ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b12e21e5f0 0x55b12e21e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50036/0 (socket says 172.21.15.37:50036) 2024-06-10T00:55:47.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.228+0000 7fca16ffd700 1 -- 172.21.15.37:0/354658785 learned_addr learned my addr 172.21.15.37:0/354658785 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:47.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.229+0000 7fca167fc700 1 --2- 172.21.15.37:0/354658785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b12e21ef10 0x55b12e218990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:47.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.229+0000 7fca177fe700 1 --2- 172.21.15.37:0/354658785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b12e218ed0 0x55b12e2192f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:47.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.229+0000 7fca16ffd700 1 -- 172.21.15.37:0/354658785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b12e218ed0 msgr2=0x55b12e2192f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:55:47.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.229+0000 7fca16ffd700 1 --2- 172.21.15.37:0/354658785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b12e218ed0 0x55b12e2192f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.229+0000 7fca16ffd700 1 -- 172.21.15.37:0/354658785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b12e21ef10 msgr2=0x55b12e218990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:55:47.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.229+0000 7fca16ffd700 1 --2- 172.21.15.37:0/354658785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b12e21ef10 0x55b12e218990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.229+0000 7fca16ffd700 1 -- 172.21.15.37:0/354658785 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b12e108f00 con 0x55b12e21e5f0 2024-06-10T00:55:47.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.229+0000 7fca16ffd700 1 --2- 172.21.15.37:0/354658785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b12e21e5f0 0x55b12e21e9d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fca08009510 tx=0x7fca0809ab50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa5b3d7bef05115 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:47.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.229+0000 7fc9fd7fa700 1 -- 172.21.15.37:0/354658785 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca080a3d60 con 0x55b12e21e5f0 2024-06-10T00:55:47.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.229+0000 7fc9fd7fa700 1 -- 172.21.15.37:0/354658785 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca080993f0 con 0x55b12e21e5f0 2024-06-10T00:55:47.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.229+0000 7fc9fd7fa700 1 -- 172.21.15.37:0/354658785 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0809b350 con 0x55b12e21e5f0 2024-06-10T00:55:47.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.230+0000 7fca27db4b80 1 -- 172.21.15.37:0/354658785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b12e21e5f0 msgr2=0x55b12e21e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:47.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.230+0000 7fca27db4b80 1 --2- 172.21.15.37:0/354658785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b12e21e5f0 0x55b12e21e9d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fca08009510 tx=0x7fca0809ab50 comp rx=0 tx=0).stop 2024-06-10T00:55:47.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.230+0000 7fca27db4b80 1 -- 172.21.15.37:0/354658785 shutdown_connections 2024-06-10T00:55:47.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.230+0000 7fca27db4b80 1 --2- 172.21.15.37:0/354658785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b12e218ed0 0x55b12e2192f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.230+0000 7fca27db4b80 1 --2- 172.21.15.37:0/354658785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b12e21e5f0 0x55b12e21e9d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.230+0000 7fca27db4b80 1 --2- 172.21.15.37:0/354658785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b12e21ef10 0x55b12e218990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.230+0000 7fca27db4b80 1 -- 172.21.15.37:0/354658785 >> 172.21.15.37:0/354658785 conn(0x55b12e143c70 msgr2=0x55b12e213800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:47.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.230+0000 7fca27db4b80 1 -- 172.21.15.37:0/354658785 shutdown_connections 2024-06-10T00:55:47.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.230+0000 7fca27db4b80 1 -- 172.21.15.37:0/354658785 wait complete. 2024-06-10T00:55:47.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.231+0000 7fca27db4b80 1 -- start start 2024-06-10T00:55:47.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.231+0000 7fca27db4b80 1 Processor -- start 2024-06-10T00:55:47.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.231+0000 7fca27db4b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b12e218ed0 0x55b12e224cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:47.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.231+0000 7fca27db4b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b12e21e5f0 0x55b12e221fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:47.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.231+0000 7fca27db4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b12e21ef10 0x55b12e222510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:47.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.231+0000 7fca27db4b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b12e229d90 con 0x55b12e21e5f0 2024-06-10T00:55:47.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.231+0000 7fca27db4b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b12e10aba0 con 0x55b12e218ed0 2024-06-10T00:55:47.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.231+0000 7fca27db4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b12e229f10 con 0x55b12e21ef10 2024-06-10T00:55:47.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.231+0000 7fca27db4b80 10 client.0 _ll_register_callbacks cb 0x55b12e1048d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:55:47.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.231+0000 7fca16ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b12e218ed0 0x55b12e224cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:47.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.231+0000 7fca16ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b12e218ed0 0x55b12e224cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:60408/0 (socket says 172.21.15.37:60408) 2024-06-10T00:55:47.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.231+0000 7fca16ffd700 1 -- 172.21.15.37:0/4194402355 learned_addr learned my addr 172.21.15.37:0/4194402355 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:47.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.232+0000 7fca177fe700 1 --2- 172.21.15.37:0/4194402355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b12e21ef10 0x55b12e222510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:47.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.232+0000 7fca167fc700 1 --2- 172.21.15.37:0/4194402355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b12e21e5f0 0x55b12e221fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:47.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.232+0000 7fca16ffd700 1 -- 172.21.15.37:0/4194402355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b12e21ef10 msgr2=0x55b12e222510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:55:47.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.232+0000 7fca16ffd700 1 --2- 172.21.15.37:0/4194402355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b12e21ef10 0x55b12e222510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.232+0000 7fca16ffd700 1 -- 172.21.15.37:0/4194402355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b12e21e5f0 msgr2=0x55b12e221fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:55:47.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.232+0000 7fca16ffd700 1 --2- 172.21.15.37:0/4194402355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b12e21e5f0 0x55b12e221fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.232+0000 7fca16ffd700 1 -- 172.21.15.37:0/4194402355 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b12e0b9880 con 0x55b12e218ed0 2024-06-10T00:55:47.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.232+0000 7fca177fe700 1 --2- 172.21.15.37:0/4194402355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b12e21ef10 0x55b12e222510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:55:47.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.232+0000 7fca167fc700 1 --2- 172.21.15.37:0/4194402355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b12e21e5f0 0x55b12e221fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:55:47.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.232+0000 7fca16ffd700 1 --2- 172.21.15.37:0/4194402355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b12e218ed0 0x55b12e224cd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fca08097ac0 tx=0x7fca0809b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a60bc0f28324951c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:47.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.232+0000 7fc9faff5700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T00:55:47.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.232+0000 7fc9faff5700 1 -- 172.21.15.37:0/4194402355 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0809b550 con 0x55b12e218ed0 2024-06-10T00:55:47.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.232+0000 7fca27db4b80 1 -- 172.21.15.37:0/4194402355 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca080993f0 con 0x55b12e218ed0 2024-06-10T00:55:47.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.232+0000 7fc9faff5700 1 -- 172.21.15.37:0/4194402355 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fca080a38d0 con 0x55b12e218ed0 2024-06-10T00:55:47.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.232+0000 7fc9faff5700 1 -- 172.21.15.37:0/4194402355 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fca08015ba0 con 0x55b12e218ed0 2024-06-10T00:55:47.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.233+0000 7fca27db4b80 10 client.24806 fetch_fsmap learned FSMap version 26 2024-06-10T00:55:47.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.233+0000 7fca27db4b80 1 -- 172.21.15.37:0/4194402355 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b12e2346e0 con 0x55b12e218ed0 2024-06-10T00:55:47.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.233+0000 7fc9faff5700 1 -- 172.21.15.37:0/4194402355 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca080993f0 con 0x55b12e218ed0 2024-06-10T00:55:47.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.233+0000 7fc9faff5700 1 -- 172.21.15.37:0/4194402355 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fca080b5070 con 0x55b12e218ed0 2024-06-10T00:55:47.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.233+0000 7fca27db4b80 10 client.24806 fetch_fsmap finished waiting for FSMap version 26 2024-06-10T00:55:47.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.233+0000 7fca27db4b80 10 client.24806 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T00:55:47.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.233+0000 7fca27db4b80 20 client.24806 populate_metadata read hostname 'smithi037' 2024-06-10T00:55:47.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.233+0000 7fca27db4b80 1 --2- 172.21.15.37:0/4194402355 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b12e236880 0x55b12e237cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:47.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.233+0000 7fca27db4b80 4 client.24806 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:55:47.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.233+0000 7fca27db4b80 1 -- 172.21.15.37:0/4194402355 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b12e177cc0 con 0x55b12e236880 2024-06-10T00:55:47.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.234+0000 7fca167fc700 1 --2- 172.21.15.37:0/4194402355 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b12e236880 0x55b12e237cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:47.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.234+0000 7fc9faff5700 10 client.24806 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:55:47.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.234+0000 7fca167fc700 1 --2- 172.21.15.37:0/4194402355 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b12e236880 0x55b12e237cf0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da859cd189e2beb server_cookie=9df9300c2da0b6b3 in_seq=0 out_seq=0 2024-06-10T00:55:47.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.235+0000 7fc9faff5700 1 -- 172.21.15.37:0/4194402355 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b12e177cc0 con 0x55b12e236880 2024-06-10T00:55:47.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.235+0000 7fc9faff5700 10 client.24806 handle_command_reply: tid=0 2024-06-10T00:55:47.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.235+0000 7fca27db4b80 1 client.24806 shutdown 2024-06-10T00:55:47.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.235+0000 7fca27db4b80 1 -- 172.21.15.37:0/4194402355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b12e218ed0 msgr2=0x55b12e224cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:47.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.235+0000 7fca27db4b80 1 --2- 172.21.15.37:0/4194402355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b12e218ed0 0x55b12e224cd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fca08097ac0 tx=0x7fca0809b960 comp rx=0 tx=0).stop 2024-06-10T00:55:47.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.235+0000 7fca27db4b80 1 -- 172.21.15.37:0/4194402355 shutdown_connections 2024-06-10T00:55:47.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.235+0000 7fca27db4b80 1 --2- 172.21.15.37:0/4194402355 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b12e236880 0x55b12e237cf0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.235+0000 7fca27db4b80 1 --2- 172.21.15.37:0/4194402355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b12e21ef10 0x55b12e222510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.235+0000 7fca27db4b80 1 --2- 172.21.15.37:0/4194402355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b12e218ed0 0x55b12e224cd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.235+0000 7fca27db4b80 1 --2- 172.21.15.37:0/4194402355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b12e21e5f0 0x55b12e221fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.235+0000 7fca27db4b80 1 -- 172.21.15.37:0/4194402355 >> 172.21.15.37:0/4194402355 conn(0x55b12e143c70 msgr2=0x55b12e217980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:47.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.236+0000 7fc9faff5700 0 client.24806 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:55:47.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.236+0000 7fca27db4b80 1 -- 172.21.15.37:0/4194402355 shutdown_connections 2024-06-10T00:55:47.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.236+0000 7fca27db4b80 1 -- 172.21.15.37:0/4194402355 wait complete. 2024-06-10T00:55:47.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.236+0000 7fca27db4b80 20 client.24806 trim_cache size 0 max 16384 2024-06-10T00:55:47.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.258+0000 7fca27db4b80 1 Processor -- start 2024-06-10T00:55:47.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.258+0000 7fca27db4b80 1 -- start start 2024-06-10T00:55:47.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.258+0000 7fca27db4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b12e218ed0 0x55b12e218990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:47.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.258+0000 7fca27db4b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b12e21e5f0 0x55b12e2139a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:47.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.258+0000 7fca27db4b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b12e21ef10 0x55b12e20d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:47.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.258+0000 7fca27db4b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b12e236030 con 0x55b12e21e5f0 2024-06-10T00:55:47.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.258+0000 7fca27db4b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b12e236700 con 0x55b12e21ef10 2024-06-10T00:55:47.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.258+0000 7fca27db4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b12e236dd0 con 0x55b12e218ed0 2024-06-10T00:55:47.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.258+0000 7fca167fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b12e21e5f0 0x55b12e2139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:47.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.258+0000 7fca167fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b12e21e5f0 0x55b12e2139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50044/0 (socket says 172.21.15.37:50044) 2024-06-10T00:55:47.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.258+0000 7fca167fc700 1 -- 172.21.15.37:0/1713943801 learned_addr learned my addr 172.21.15.37:0/1713943801 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:47.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.259+0000 7fca16ffd700 1 --2- 172.21.15.37:0/1713943801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b12e218ed0 0x55b12e218990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:47.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.259+0000 7fca177fe700 1 --2- 172.21.15.37:0/1713943801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b12e21ef10 0x55b12e20d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:47.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.259+0000 7fca167fc700 1 -- 172.21.15.37:0/1713943801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b12e218ed0 msgr2=0x55b12e218990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:55:47.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.259+0000 7fca167fc700 1 --2- 172.21.15.37:0/1713943801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b12e218ed0 0x55b12e218990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.259+0000 7fca167fc700 1 -- 172.21.15.37:0/1713943801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b12e21ef10 msgr2=0x55b12e20d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:55:47.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.259+0000 7fca167fc700 1 --2- 172.21.15.37:0/1713943801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b12e21ef10 0x55b12e20d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.259+0000 7fca167fc700 1 -- 172.21.15.37:0/1713943801 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b12e224910 con 0x55b12e21e5f0 2024-06-10T00:55:47.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.259+0000 7fca16ffd700 1 --2- 172.21.15.37:0/1713943801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b12e218ed0 0x55b12e218990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:55:47.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.259+0000 7fca177fe700 1 --2- 172.21.15.37:0/1713943801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b12e21ef10 0x55b12e20d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:55:47.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.259+0000 7fca167fc700 1 --2- 172.21.15.37:0/1713943801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b12e21e5f0 0x55b12e2139a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fca0c009990 tx=0x7fca0c023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fcdb66a75d0a327 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:47.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.259+0000 7fc9fcff9700 1 -- 172.21.15.37:0/1713943801 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c00fa40 con 0x55b12e21e5f0 2024-06-10T00:55:47.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.259+0000 7fc9fcff9700 1 -- 172.21.15.37:0/1713943801 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca0c00fbe0 con 0x55b12e21e5f0 2024-06-10T00:55:47.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.259+0000 7fc9fcff9700 1 -- 172.21.15.37:0/1713943801 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c024410 con 0x55b12e21e5f0 2024-06-10T00:55:47.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.260+0000 7fca27db4b80 1 -- 172.21.15.37:0/1713943801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b12e21e5f0 msgr2=0x55b12e2139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:47.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.260+0000 7fca27db4b80 1 --2- 172.21.15.37:0/1713943801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b12e21e5f0 0x55b12e2139a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fca0c009990 tx=0x7fca0c023000 comp rx=0 tx=0).stop 2024-06-10T00:55:47.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.260+0000 7fca27db4b80 1 -- 172.21.15.37:0/1713943801 shutdown_connections 2024-06-10T00:55:47.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.260+0000 7fca27db4b80 1 --2- 172.21.15.37:0/1713943801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b12e21ef10 0x55b12e20d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.260+0000 7fca27db4b80 1 --2- 172.21.15.37:0/1713943801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b12e218ed0 0x55b12e218990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.260+0000 7fca27db4b80 1 --2- 172.21.15.37:0/1713943801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b12e21e5f0 0x55b12e2139a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.260+0000 7fca27db4b80 1 -- 172.21.15.37:0/1713943801 >> 172.21.15.37:0/1713943801 conn(0x55b12e143c70 msgr2=0x55b12e21f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:47.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.260+0000 7fca27db4b80 1 -- 172.21.15.37:0/1713943801 shutdown_connections 2024-06-10T00:55:47.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.260+0000 7fca27db4b80 1 -- 172.21.15.37:0/1713943801 wait complete. 2024-06-10T00:55:47.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.260+0000 7fca27db4b80 1 -- start start 2024-06-10T00:55:47.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.261+0000 7fca27db4b80 1 Processor -- start 2024-06-10T00:55:47.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.261+0000 7fca27db4b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b12e218ed0 0x55b12e2414c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:47.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.261+0000 7fca27db4b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b12e21e5f0 0x55b12e241a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:47.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.261+0000 7fca27db4b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b12e21ef10 0x55b12e23aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:47.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.261+0000 7fca27db4b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b12e237b70 con 0x55b12e21e5f0 2024-06-10T00:55:47.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.261+0000 7fca27db4b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b12e2374a0 con 0x55b12e218ed0 2024-06-10T00:55:47.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.261+0000 7fca27db4b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b12e238240 con 0x55b12e21ef10 2024-06-10T00:55:47.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.261+0000 7fca27db4b80 10 client.0 _ll_register_callbacks cb 0x55b12e1048d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:55:47.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.261+0000 7fca167fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b12e21e5f0 0x55b12e241a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:47.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.261+0000 7fca167fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b12e21e5f0 0x55b12e241a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50058/0 (socket says 172.21.15.37:50058) 2024-06-10T00:55:47.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.261+0000 7fca167fc700 1 -- 172.21.15.37:0/2482310129 learned_addr learned my addr 172.21.15.37:0/2482310129 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:47.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.261+0000 7fca16ffd700 1 --2- 172.21.15.37:0/2482310129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b12e218ed0 0x55b12e2414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:47.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.261+0000 7fca177fe700 1 --2- 172.21.15.37:0/2482310129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b12e21ef10 0x55b12e23aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:47.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.261+0000 7fca167fc700 1 -- 172.21.15.37:0/2482310129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b12e21ef10 msgr2=0x55b12e23aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:55:47.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.261+0000 7fca167fc700 1 --2- 172.21.15.37:0/2482310129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b12e21ef10 0x55b12e23aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.261+0000 7fca167fc700 1 -- 172.21.15.37:0/2482310129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b12e218ed0 msgr2=0x55b12e2414c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:55:47.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.261+0000 7fca167fc700 1 --2- 172.21.15.37:0/2482310129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b12e218ed0 0x55b12e2414c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.261+0000 7fca167fc700 1 -- 172.21.15.37:0/2482310129 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b12e23b260 con 0x55b12e21e5f0 2024-06-10T00:55:47.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.261+0000 7fca16ffd700 1 --2- 172.21.15.37:0/2482310129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b12e218ed0 0x55b12e2414c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:55:47.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.262+0000 7fca167fc700 1 --2- 172.21.15.37:0/2482310129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b12e21e5f0 0x55b12e241a00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fca0c0098e0 tx=0x7fca0c003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf940b0aa274a3e2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:47.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.262+0000 7fc9faff5700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:55:47.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.262+0000 7fc9faff5700 1 -- 172.21.15.37:0/2482310129 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c031610 con 0x55b12e21e5f0 2024-06-10T00:55:47.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.262+0000 7fc9faff5700 1 -- 172.21.15.37:0/2482310129 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fca0c03c420 con 0x55b12e21e5f0 2024-06-10T00:55:47.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.262+0000 7fca27db4b80 1 -- 172.21.15.37:0/2482310129 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca0c00fbe0 con 0x55b12e21e5f0 2024-06-10T00:55:47.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.262+0000 7fc9faff5700 1 -- 172.21.15.37:0/2482310129 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fca0c0317b0 con 0x55b12e21e5f0 2024-06-10T00:55:47.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.262+0000 7fca27db4b80 10 client.15093 fetch_fsmap learned FSMap version 26 2024-06-10T00:55:47.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.262+0000 7fca27db4b80 1 -- 172.21.15.37:0/2482310129 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b12e23fc20 con 0x55b12e21e5f0 2024-06-10T00:55:47.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.262+0000 7fc9faff5700 1 -- 172.21.15.37:0/2482310129 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca0c00fbe0 con 0x55b12e21e5f0 2024-06-10T00:55:47.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.262+0000 7fc9faff5700 1 -- 172.21.15.37:0/2482310129 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fca0809f120 con 0x55b12e21e5f0 2024-06-10T00:55:47.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.262+0000 7fca27db4b80 10 client.15093 fetch_fsmap finished waiting for FSMap version 26 2024-06-10T00:55:47.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.262+0000 7fca27db4b80 10 client.15093 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T00:55:47.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.262+0000 7fca27db4b80 20 client.15093 populate_metadata read hostname 'smithi037' 2024-06-10T00:55:47.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.262+0000 7fca27db4b80 1 --2- 172.21.15.37:0/2482310129 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b12e2a63f0 0x55b12e2a5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:47.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.262+0000 7fca27db4b80 4 client.15093 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T00:55:47.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.262+0000 7fca27db4b80 1 -- 172.21.15.37:0/2482310129 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "status"}) -- 0x55b12e172240 con 0x55b12e2a63f0 2024-06-10T00:55:47.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.263+0000 7fca16ffd700 1 --2- 172.21.15.37:0/2482310129 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b12e2a63f0 0x55b12e2a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:47.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.263+0000 7fca16ffd700 1 --2- 172.21.15.37:0/2482310129 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b12e2a63f0 0x55b12e2a5100 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=348c4b32fb0b9c3 server_cookie=e752112b5c85b6d7 in_seq=0 out_seq=0 2024-06-10T00:55:47.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.263+0000 7fc9faff5700 10 client.15093 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:55:47.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.263+0000 7fc9faff5700 1 -- 172.21.15.37:0/2482310129 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55b12e172240 con 0x55b12e2a63f0 2024-06-10T00:55:47.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.263+0000 7fc9faff5700 10 client.15093 handle_command_reply: tid=0 2024-06-10T00:55:47.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.264+0000 7fca27db4b80 1 client.15093 shutdown 2024-06-10T00:55:47.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.264+0000 7fca27db4b80 1 -- 172.21.15.37:0/2482310129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b12e21e5f0 msgr2=0x55b12e241a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:47.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.264+0000 7fca27db4b80 1 --2- 172.21.15.37:0/2482310129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b12e21e5f0 0x55b12e241a00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fca0c0098e0 tx=0x7fca0c003730 comp rx=0 tx=0).stop 2024-06-10T00:55:47.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.264+0000 7fca27db4b80 1 -- 172.21.15.37:0/2482310129 shutdown_connections 2024-06-10T00:55:47.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.264+0000 7fca27db4b80 1 --2- 172.21.15.37:0/2482310129 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b12e2a63f0 0x55b12e2a5100 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.264+0000 7fca27db4b80 1 --2- 172.21.15.37:0/2482310129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b12e21ef10 0x55b12e23aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.264+0000 7fca27db4b80 1 --2- 172.21.15.37:0/2482310129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b12e218ed0 0x55b12e2414c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.264+0000 7fca27db4b80 1 --2- 172.21.15.37:0/2482310129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b12e21e5f0 0x55b12e241a00 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.264+0000 7fc9faff5700 0 client.15093 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:55:47.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.264+0000 7fca27db4b80 1 -- 172.21.15.37:0/2482310129 >> 172.21.15.37:0/2482310129 conn(0x55b12e143c70 msgr2=0x55b12e21fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:47.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.264+0000 7fca27db4b80 1 -- 172.21.15.37:0/2482310129 shutdown_connections 2024-06-10T00:55:47.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.264+0000 7fca27db4b80 1 -- 172.21.15.37:0/2482310129 wait complete. 2024-06-10T00:55:47.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.264+0000 7fca27db4b80 20 client.15093 trim_cache size 0 max 16384 2024-06-10T00:55:47.272 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T00:55:47.272 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "68793514-26c2-11ef-bc9b-c7b262605968", 2024-06-10T00:55:47.272 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-06-10T00:55:47.272 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24472, 2024-06-10T00:55:47.272 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-06-10T00:55:47.272 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-06-10T00:55:47.272 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-06-10T00:55:47.273 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 189.31519584599999, 2024-06-10T00:55:47.273 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-06-10T00:55:47.273 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 100, 2024-06-10T00:55:47.273 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 98, 2024-06-10T00:55:47.273 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 216.964579921 2024-06-10T00:55:47.273 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T00:55:47.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.264+0000 7fca1d531700 1 -- 172.21.15.37:0/2365625798 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fca000a2c20 msgr2=0x7fca000a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:47.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.264+0000 7fca1d531700 1 --2- 172.21.15.37:0/2365625798 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fca000a2c20 0x7fca000a50e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00b5c0 tx=0x7fca0c005960 comp rx=0 tx=0).stop 2024-06-10T00:55:47.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.264+0000 7fca1d531700 1 -- 172.21.15.37:0/2365625798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca1806a170 msgr2=0x7fca1819e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:47.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.264+0000 7fca1d531700 1 --2- 172.21.15.37:0/2365625798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca1806a170 0x7fca1819e880 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fca080060b0 tx=0x7fca0800d3b0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.264+0000 7fca1d531700 1 -- 172.21.15.37:0/2365625798 shutdown_connections 2024-06-10T00:55:47.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.264+0000 7fca1d531700 1 --2- 172.21.15.37:0/2365625798 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fca000a2c20 0x7fca000a50e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.264+0000 7fca1d531700 1 --2- 172.21.15.37:0/2365625798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca18108500 0x7fca18198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.265+0000 7fca1d531700 1 --2- 172.21.15.37:0/2365625798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca1806a170 0x7fca1819e880 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.265+0000 7fca1d531700 1 --2- 172.21.15.37:0/2365625798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca1806ab30 0x7fca1819edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:47.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.265+0000 7fca1d531700 1 -- 172.21.15.37:0/2365625798 >> 172.21.15.37:0/2365625798 conn(0x7fca18078340 msgr2=0x7fca18100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:47.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.265+0000 7fca1d531700 1 -- 172.21.15.37:0/2365625798 shutdown_connections 2024-06-10T00:55:47.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:47.265+0000 7fca1d531700 1 -- 172.21.15.37:0/2365625798 wait complete. 2024-06-10T00:55:47.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:47 smithi094 ceph-mon[36708]: pgmap v403: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:55:47.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:47 smithi037 ceph-mon[32192]: pgmap v403: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:55:48.020 INFO:tasks.cephadm:Setup complete, yielding 2024-06-10T00:55:48.020 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-10T00:55:48.032 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-06-10T00:55:48.032 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch status' 2024-06-10T00:55:48.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:47 smithi167 ceph-mon[36917]: pgmap v403: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:55:48.287 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:55:49.731 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:49 smithi037 ceph-mon[32192]: pgmap v404: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:55:49.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:49 smithi094 ceph-mon[36708]: pgmap v404: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:55:49.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.961+0000 7ff58f86b700 1 -- 172.21.15.37:0/1320744738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff588103710 msgr2=0x7ff588103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:49.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.961+0000 7ff58f86b700 1 --2- 172.21.15.37:0/1320744738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff588103710 0x7ff588103b70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff5780099c0 tx=0x7ff578009cd0 comp rx=0 tx=0).stop 2024-06-10T00:55:49.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.961+0000 7ff58f86b700 1 -- 172.21.15.37:0/1320744738 shutdown_connections 2024-06-10T00:55:49.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.961+0000 7ff58f86b700 1 --2- 172.21.15.37:0/1320744738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5881040b0 0x7ff58810a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:49.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.961+0000 7ff58f86b700 1 --2- 172.21.15.37:0/1320744738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff588102510 0x7ff588102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:49.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.961+0000 7ff58f86b700 1 --2- 172.21.15.37:0/1320744738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff588103710 0x7ff588103b70 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:49.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.961+0000 7ff58f86b700 1 -- 172.21.15.37:0/1320744738 >> 172.21.15.37:0/1320744738 conn(0x7ff5880fdab0 msgr2=0x7ff5880ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:49.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.962+0000 7ff58f86b700 1 -- 172.21.15.37:0/1320744738 shutdown_connections 2024-06-10T00:55:49.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.962+0000 7ff58f86b700 1 -- 172.21.15.37:0/1320744738 wait complete. 2024-06-10T00:55:49.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.962+0000 7ff58f86b700 1 Processor -- start 2024-06-10T00:55:49.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.962+0000 7ff58f86b700 1 -- start start 2024-06-10T00:55:49.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.962+0000 7ff58f86b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff588102510 0x7ff588076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:49.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.962+0000 7ff58f86b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff588103710 0x7ff588074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:49.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.962+0000 7ff58f86b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5881040b0 0x7ff588074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:49.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.962+0000 7ff58f86b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff58810e170 con 0x7ff5881040b0 2024-06-10T00:55:49.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.962+0000 7ff58f86b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff58810dff0 con 0x7ff588102510 2024-06-10T00:55:49.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.962+0000 7ff58f86b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff58810e2f0 con 0x7ff588103710 2024-06-10T00:55:49.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.963+0000 7ff58d607700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff588102510 0x7ff588076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:49.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.963+0000 7ff58de08700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5881040b0 0x7ff588074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:49.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.963+0000 7ff58d607700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff588102510 0x7ff588076230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:60458/0 (socket says 172.21.15.37:60458) 2024-06-10T00:55:49.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.963+0000 7ff58d607700 1 -- 172.21.15.37:0/3303429908 learned_addr learned my addr 172.21.15.37:0/3303429908 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:49.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.963+0000 7ff58ce06700 1 --2- 172.21.15.37:0/3303429908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff588103710 0x7ff588074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:49.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.963+0000 7ff58d607700 1 -- 172.21.15.37:0/3303429908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff588103710 msgr2=0x7ff588074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:49.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.963+0000 7ff58d607700 1 --2- 172.21.15.37:0/3303429908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff588103710 0x7ff588074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:49.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.963+0000 7ff58d607700 1 -- 172.21.15.37:0/3303429908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5881040b0 msgr2=0x7ff588074dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:49.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.963+0000 7ff58d607700 1 --2- 172.21.15.37:0/3303429908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5881040b0 0x7ff588074dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:49.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.963+0000 7ff58d607700 1 -- 172.21.15.37:0/3303429908 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff588075670 con 0x7ff588102510 2024-06-10T00:55:49.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.963+0000 7ff58ce06700 1 --2- 172.21.15.37:0/3303429908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff588103710 0x7ff588074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:55:49.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.963+0000 7ff58de08700 1 --2- 172.21.15.37:0/3303429908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5881040b0 0x7ff588074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:55:49.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.963+0000 7ff58d607700 1 --2- 172.21.15.37:0/3303429908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff588102510 0x7ff588076230 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff58400d980 tx=0x7ff58400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:49.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.963+0000 7ff57e7fc700 1 -- 172.21.15.37:0/3303429908 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584017030 con 0x7ff588102510 2024-06-10T00:55:49.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.963+0000 7ff58f86b700 1 -- 172.21.15.37:0/3303429908 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5881a2910 con 0x7ff588102510 2024-06-10T00:55:49.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.963+0000 7ff57e7fc700 1 -- 172.21.15.37:0/3303429908 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff584004500 con 0x7ff588102510 2024-06-10T00:55:49.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.963+0000 7ff57e7fc700 1 -- 172.21.15.37:0/3303429908 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584003aa0 con 0x7ff588102510 2024-06-10T00:55:49.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.963+0000 7ff58f86b700 1 -- 172.21.15.37:0/3303429908 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5881a2e50 con 0x7ff588102510 2024-06-10T00:55:49.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.964+0000 7ff58f86b700 1 -- 172.21.15.37:0/3303429908 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff588069180 con 0x7ff588102510 2024-06-10T00:55:49.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.966+0000 7ff57e7fc700 1 -- 172.21.15.37:0/3303429908 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff58401a070 con 0x7ff588102510 2024-06-10T00:55:49.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.966+0000 7ff57e7fc700 1 --2- 172.21.15.37:0/3303429908 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff5740a2c20 0x7ff5740a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:49.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.966+0000 7ff58ce06700 1 --2- 172.21.15.37:0/3303429908 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff5740a2c20 0x7ff5740a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:49.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.967+0000 7ff57e7fc700 1 -- 172.21.15.37:0/3303429908 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7ff5840cf180 con 0x7ff588102510 2024-06-10T00:55:49.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.967+0000 7ff58ce06700 1 --2- 172.21.15.37:0/3303429908 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff5740a2c20 0x7ff5740a50e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff578003870 tx=0x7ff578005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:49.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:49.970+0000 7ff57e7fc700 1 -- 172.21.15.37:0/3303429908 <== mon.2 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff58409bc90 con 0x7ff588102510 2024-06-10T00:55:50.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:49 smithi167 ceph-mon[36917]: pgmap v404: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:55:50.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:50.202+0000 7ff58f86b700 1 -- 172.21.15.37:0/3303429908 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7ff5880628d0 con 0x7ff5740a2c20 2024-06-10T00:55:50.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:50.203+0000 7ff57e7fc700 1 -- 172.21.15.37:0/3303429908 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7ff5880628d0 con 0x7ff5740a2c20 2024-06-10T00:55:50.204 INFO:teuthology.orchestra.run.smithi037.stdout:Backend: cephadm 2024-06-10T00:55:50.204 INFO:teuthology.orchestra.run.smithi037.stdout:Available: Yes 2024-06-10T00:55:50.204 INFO:teuthology.orchestra.run.smithi037.stdout:Paused: No 2024-06-10T00:55:50.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:50.205+0000 7ff58f86b700 1 -- 172.21.15.37:0/3303429908 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff5740a2c20 msgr2=0x7ff5740a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:50.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:50.205+0000 7ff58f86b700 1 --2- 172.21.15.37:0/3303429908 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff5740a2c20 0x7ff5740a50e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff578003870 tx=0x7ff578005fb0 comp rx=0 tx=0).stop 2024-06-10T00:55:50.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:50.205+0000 7ff58f86b700 1 -- 172.21.15.37:0/3303429908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff588102510 msgr2=0x7ff588076230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:50.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:50.205+0000 7ff58f86b700 1 --2- 172.21.15.37:0/3303429908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff588102510 0x7ff588076230 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff58400d980 tx=0x7ff58400dc90 comp rx=0 tx=0).stop 2024-06-10T00:55:50.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:50.205+0000 7ff58f86b700 1 -- 172.21.15.37:0/3303429908 shutdown_connections 2024-06-10T00:55:50.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:50.205+0000 7ff58f86b700 1 --2- 172.21.15.37:0/3303429908 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff5740a2c20 0x7ff5740a50e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:50.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:50.205+0000 7ff58f86b700 1 --2- 172.21.15.37:0/3303429908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5881040b0 0x7ff588074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:50.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:50.205+0000 7ff58f86b700 1 --2- 172.21.15.37:0/3303429908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff588102510 0x7ff588076230 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:50.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:50.205+0000 7ff58f86b700 1 --2- 172.21.15.37:0/3303429908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff588103710 0x7ff588074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:50.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:50.205+0000 7ff58f86b700 1 -- 172.21.15.37:0/3303429908 >> 172.21.15.37:0/3303429908 conn(0x7ff5880fdab0 msgr2=0x7ff588106a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:50.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:50.205+0000 7ff58f86b700 1 -- 172.21.15.37:0/3303429908 shutdown_connections 2024-06-10T00:55:50.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:50.205+0000 7ff58f86b700 1 -- 172.21.15.37:0/3303429908 wait complete. 2024-06-10T00:55:50.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:50 smithi094 ceph-mon[36708]: from='client.24815 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:55:50.866 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:50 smithi037 ceph-mon[32192]: from='client.24815 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:55:50.900 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ps' 2024-06-10T00:55:51.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:50 smithi167 ceph-mon[36917]: from='client.24815 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:55:51.156 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:55:51.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:51 smithi094 ceph-mon[36708]: pgmap v405: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:55:51.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:51 smithi037 ceph-mon[32192]: pgmap v405: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:55:52.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:51 smithi167 ceph-mon[36917]: pgmap v405: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:55:52.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.849+0000 7f92b510a700 1 -- 172.21.15.37:0/4177134648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92b0102f90 msgr2=0x7f92b0109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:52.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.849+0000 7f92b510a700 1 --2- 172.21.15.37:0/4177134648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92b0102f90 0x7f92b0109840 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f92a800b0a0 tx=0x7f92a800b3b0 comp rx=0 tx=0).stop 2024-06-10T00:55:52.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.849+0000 7f92b510a700 1 -- 172.21.15.37:0/4177134648 shutdown_connections 2024-06-10T00:55:52.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.849+0000 7f92b510a700 1 --2- 172.21.15.37:0/4177134648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92b0102f90 0x7f92b0109840 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:52.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.849+0000 7f92b510a700 1 --2- 172.21.15.37:0/4177134648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92b01013f0 0x7f92b0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:52.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.849+0000 7f92b510a700 1 --2- 172.21.15.37:0/4177134648 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92b01025f0 0x7f92b0102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:52.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.849+0000 7f92b510a700 1 -- 172.21.15.37:0/4177134648 >> 172.21.15.37:0/4177134648 conn(0x7f92b00fc9b0 msgr2=0x7f92b00fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:52.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.850+0000 7f92b510a700 1 -- 172.21.15.37:0/4177134648 shutdown_connections 2024-06-10T00:55:52.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.850+0000 7f92b510a700 1 -- 172.21.15.37:0/4177134648 wait complete. 2024-06-10T00:55:52.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.850+0000 7f92b510a700 1 Processor -- start 2024-06-10T00:55:52.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.850+0000 7f92b510a700 1 -- start start 2024-06-10T00:55:52.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.851+0000 7f92b510a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92b01013f0 0x7f92b019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:52.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.851+0000 7f92b510a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92b01025f0 0x7f92b019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:52.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.851+0000 7f92b510a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92b0102f90 0x7f92b01a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:52.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.851+0000 7f92b510a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f92b010e160 con 0x7f92b01013f0 2024-06-10T00:55:52.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.851+0000 7f92b510a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f92b010dfe0 con 0x7f92b01025f0 2024-06-10T00:55:52.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.851+0000 7f92b510a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f92b010e2e0 con 0x7f92b0102f90 2024-06-10T00:55:52.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.851+0000 7f92aed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92b01013f0 0x7f92b019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:52.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.851+0000 7f92aed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92b01013f0 0x7f92b019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45326/0 (socket says 172.21.15.37:45326) 2024-06-10T00:55:52.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.851+0000 7f92aed9d700 1 -- 172.21.15.37:0/3144756925 learned_addr learned my addr 172.21.15.37:0/3144756925 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:52.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.851+0000 7f92ae59c700 1 --2- 172.21.15.37:0/3144756925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92b01025f0 0x7f92b019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:52.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.851+0000 7f92af59e700 1 --2- 172.21.15.37:0/3144756925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92b0102f90 0x7f92b01a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:52.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.851+0000 7f92aed9d700 1 -- 172.21.15.37:0/3144756925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92b0102f90 msgr2=0x7f92b01a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:52.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.851+0000 7f92aed9d700 1 --2- 172.21.15.37:0/3144756925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92b0102f90 0x7f92b01a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:52.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.851+0000 7f92aed9d700 1 -- 172.21.15.37:0/3144756925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92b01025f0 msgr2=0x7f92b019ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:52.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.851+0000 7f92aed9d700 1 --2- 172.21.15.37:0/3144756925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92b01025f0 0x7f92b019ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:52.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.851+0000 7f92aed9d700 1 -- 172.21.15.37:0/3144756925 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b01a23a0 con 0x7f92b01013f0 2024-06-10T00:55:52.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.851+0000 7f92af59e700 1 --2- 172.21.15.37:0/3144756925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92b0102f90 0x7f92b01a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:55:52.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.852+0000 7f92aed9d700 1 --2- 172.21.15.37:0/3144756925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92b01013f0 0x7f92b019a740 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f92a000c9c0 tx=0x7f92a000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:52.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.852+0000 7f92a7fff700 1 -- 172.21.15.37:0/3144756925 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a0007ac0 con 0x7f92b01013f0 2024-06-10T00:55:52.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.852+0000 7f92b510a700 1 -- 172.21.15.37:0/3144756925 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b01a2690 con 0x7f92b01013f0 2024-06-10T00:55:52.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.852+0000 7f92a7fff700 1 -- 172.21.15.37:0/3144756925 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92a0007c60 con 0x7f92b01013f0 2024-06-10T00:55:52.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.852+0000 7f92a7fff700 1 -- 172.21.15.37:0/3144756925 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a0018660 con 0x7f92b01013f0 2024-06-10T00:55:52.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.852+0000 7f92b510a700 1 -- 172.21.15.37:0/3144756925 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b01a2c20 con 0x7f92b01013f0 2024-06-10T00:55:52.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.853+0000 7f92b510a700 1 -- 172.21.15.37:0/3144756925 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b0069180 con 0x7f92b01013f0 2024-06-10T00:55:52.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.854+0000 7f92a7fff700 1 -- 172.21.15.37:0/3144756925 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f92a0004020 con 0x7f92b01013f0 2024-06-10T00:55:52.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.855+0000 7f92a7fff700 1 --2- 172.21.15.37:0/3144756925 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f929c0a2c20 0x7f929c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:52.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.855+0000 7f92ae59c700 1 --2- 172.21.15.37:0/3144756925 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f929c0a2c20 0x7f929c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:52.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.855+0000 7f92a7fff700 1 -- 172.21.15.37:0/3144756925 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f92a00cf540 con 0x7f92b01013f0 2024-06-10T00:55:52.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.856+0000 7f92ae59c700 1 --2- 172.21.15.37:0/3144756925 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f929c0a2c20 0x7f929c0a50e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f929800ac50 tx=0x7f929800a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:52.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:52.859+0000 7f92a7fff700 1 -- 172.21.15.37:0/3144756925 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f92a0014030 con 0x7f92b01013f0 2024-06-10T00:55:53.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:53.075+0000 7f92b510a700 1 -- 172.21.15.37:0/3144756925 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f92b010b7a0 con 0x7f929c0a2c20 2024-06-10T00:55:53.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:53.088+0000 7f92a7fff700 1 -- 172.21.15.37:0/3144756925 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f92b010b7a0 con 0x7f929c0a2c20 2024-06-10T00:55:53.090 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-10T00:55:53.090 INFO:teuthology.orchestra.run.smithi037.stdout:mds.a smithi037 running (3m) 34s ago 3m 15.8M - 19.0.0-2946-g3d89accf b045ac545f82 e6d28d7dca9d 2024-06-10T00:55:53.090 INFO:teuthology.orchestra.run.smithi037.stdout:mds.b smithi094 running (3m) 76s ago 3m 19.8M - 19.0.0-2946-g3d89accf b045ac545f82 6afed0933a54 2024-06-10T00:55:53.090 INFO:teuthology.orchestra.run.smithi037.stdout:mds.c smithi167 running (3m) 76s ago 3m 18.1M - 19.0.0-2946-g3d89accf b045ac545f82 73324939eab3 2024-06-10T00:55:53.090 INFO:teuthology.orchestra.run.smithi037.stdout:mds.d smithi037 running (3m) 34s ago 3m 16.8M - 19.0.0-2946-g3d89accf b045ac545f82 2637085aa9ef 2024-06-10T00:55:53.090 INFO:teuthology.orchestra.run.smithi037.stdout:mds.e smithi094 running (3m) 76s ago 3m 17.4M - 19.0.0-2946-g3d89accf b045ac545f82 e7986ba36127 2024-06-10T00:55:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:mds.f smithi167 running (3m) 76s ago 3m 20.9M - 19.0.0-2946-g3d89accf b045ac545f82 3f963f7d3e44 2024-06-10T00:55:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:mds.g smithi037 running (3m) 34s ago 3m 13.4M - 19.0.0-2946-g3d89accf b045ac545f82 fd0e2ed2a2e4 2024-06-10T00:55:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:mds.h smithi094 running (3m) 76s ago 3m 17.2M - 19.0.0-2946-g3d89accf b045ac545f82 d3cac041c120 2024-06-10T00:55:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:mds.i smithi167 running (3m) 76s ago 3m 28.5M - 19.0.0-2946-g3d89accf b045ac545f82 016d4410edc1 2024-06-10T00:55:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:mds.j smithi037 running (4m) 34s ago 4m 14.6M - 19.0.0-2946-g3d89accf b045ac545f82 c1217c6e90fe 2024-06-10T00:55:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:mds.k smithi094 running (4m) 76s ago 4m 14.6M - 19.0.0-2946-g3d89accf b045ac545f82 2b9d0a4608f3 2024-06-10T00:55:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:mds.l smithi167 running (4m) 76s ago 4m 16.0M - 19.0.0-2946-g3d89accf b045ac545f82 035edf753fae 2024-06-10T00:55:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.x smithi037 *:9283,8765 running (11m) 34s ago 11m 517M - 19.0.0-2946-g3d89accf b045ac545f82 8038bdd9f3fd 2024-06-10T00:55:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.y smithi094 *:8443,8765 running (9m) 76s ago 9m 454M - 19.0.0-2946-g3d89accf b045ac545f82 166781b6b467 2024-06-10T00:55:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.z smithi167 *:8443,8765 running (9m) 76s ago 9m 456M - 19.0.0-2946-g3d89accf b045ac545f82 fbbed67eb804 2024-06-10T00:55:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:mon.a smithi037 running (11m) 34s ago 11m 50.2M 2048M 19.0.0-2946-g3d89accf b045ac545f82 b72c961cf1e0 2024-06-10T00:55:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:mon.b smithi094 running (9m) 76s ago 9m 42.7M 2048M 19.0.0-2946-g3d89accf b045ac545f82 17bea145dd2e 2024-06-10T00:55:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:mon.c smithi167 running (9m) 76s ago 9m 39.8M 2048M 19.0.0-2946-g3d89accf b045ac545f82 97970bf31e27 2024-06-10T00:55:53.092 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (9m) 34s ago 9m 49.2M 4273M 19.0.0-2946-g3d89accf b045ac545f82 b53f47f0c6fa 2024-06-10T00:55:53.092 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi094 running (8m) 76s ago 8m 57.3M 4272M 19.0.0-2946-g3d89accf b045ac545f82 c9a33d71ab04 2024-06-10T00:55:53.092 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi167 running (8m) 76s ago 8m 57.0M 4272M 19.0.0-2946-g3d89accf b045ac545f82 d35e34d3bfbf 2024-06-10T00:55:53.092 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi037 running (7m) 34s ago 7m 55.9M 4273M 19.0.0-2946-g3d89accf b045ac545f82 3d40a4974a8a 2024-06-10T00:55:53.092 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi094 running (7m) 76s ago 7m 45.6M 4272M 19.0.0-2946-g3d89accf b045ac545f82 af25f95213d2 2024-06-10T00:55:53.092 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi167 running (7m) 76s ago 7m 45.6M 4272M 19.0.0-2946-g3d89accf b045ac545f82 328b54319753 2024-06-10T00:55:53.092 INFO:teuthology.orchestra.run.smithi037.stdout:osd.6 smithi037 running (6m) 34s ago 6m 42.9M 4273M 19.0.0-2946-g3d89accf b045ac545f82 1db8efd002c5 2024-06-10T00:55:53.092 INFO:teuthology.orchestra.run.smithi037.stdout:osd.7 smithi094 running (6m) 76s ago 6m 56.8M 4272M 19.0.0-2946-g3d89accf b045ac545f82 e405938ba63c 2024-06-10T00:55:53.092 INFO:teuthology.orchestra.run.smithi037.stdout:osd.8 smithi167 running (5m) 76s ago 5m 45.6M 4272M 19.0.0-2946-g3d89accf b045ac545f82 acefb6e84b07 2024-06-10T00:55:53.092 INFO:teuthology.orchestra.run.smithi037.stdout:osd.9 smithi037 running (5m) 34s ago 5m 52.9M 4273M 19.0.0-2946-g3d89accf b045ac545f82 1a37515c2356 2024-06-10T00:55:53.092 INFO:teuthology.orchestra.run.smithi037.stdout:osd.10 smithi094 running (4m) 76s ago 4m 44.3M 4272M 19.0.0-2946-g3d89accf b045ac545f82 d6e0886ceed6 2024-06-10T00:55:53.092 INFO:teuthology.orchestra.run.smithi037.stdout:osd.11 smithi167 running (4m) 76s ago 4m 53.4M 4272M 19.0.0-2946-g3d89accf b045ac545f82 dafbd393f144 2024-06-10T00:55:53.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:53.091+0000 7f92b510a700 1 -- 172.21.15.37:0/3144756925 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f929c0a2c20 msgr2=0x7f929c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:53.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:53.091+0000 7f92b510a700 1 --2- 172.21.15.37:0/3144756925 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f929c0a2c20 0x7f929c0a50e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f929800ac50 tx=0x7f929800a380 comp rx=0 tx=0).stop 2024-06-10T00:55:53.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:53.091+0000 7f92b510a700 1 -- 172.21.15.37:0/3144756925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92b01013f0 msgr2=0x7f92b019a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:53.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:53.091+0000 7f92b510a700 1 --2- 172.21.15.37:0/3144756925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92b01013f0 0x7f92b019a740 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f92a000c9c0 tx=0x7f92a000ccd0 comp rx=0 tx=0).stop 2024-06-10T00:55:53.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:53.091+0000 7f92b510a700 1 -- 172.21.15.37:0/3144756925 shutdown_connections 2024-06-10T00:55:53.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:53.091+0000 7f92b510a700 1 --2- 172.21.15.37:0/3144756925 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f929c0a2c20 0x7f929c0a50e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:53.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:53.091+0000 7f92b510a700 1 --2- 172.21.15.37:0/3144756925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92b0102f90 0x7f92b01a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:53.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:53.091+0000 7f92b510a700 1 --2- 172.21.15.37:0/3144756925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92b01013f0 0x7f92b019a740 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:53.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:53.091+0000 7f92b510a700 1 --2- 172.21.15.37:0/3144756925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92b01025f0 0x7f92b019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:53.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:53.091+0000 7f92b510a700 1 -- 172.21.15.37:0/3144756925 >> 172.21.15.37:0/3144756925 conn(0x7f92b00fc9b0 msgr2=0x7f92b010a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:53.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:53.092+0000 7f92b510a700 1 -- 172.21.15.37:0/3144756925 shutdown_connections 2024-06-10T00:55:53.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:53.092+0000 7f92b510a700 1 -- 172.21.15.37:0/3144756925 wait complete. 2024-06-10T00:55:53.772 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ls' 2024-06-10T00:55:53.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:53 smithi037 ceph-mon[32192]: pgmap v406: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:55:53.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:53 smithi037 ceph-mon[32192]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:55:54.023 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:55:54.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:53 smithi167 ceph-mon[36917]: pgmap v406: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:55:54.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:53 smithi167 ceph-mon[36917]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:55:54.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:53 smithi094 ceph-mon[36708]: pgmap v406: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:55:54.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:53 smithi094 ceph-mon[36708]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:55:55.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.754+0000 7f9056248700 1 -- 172.21.15.37:0/136358725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9050104090 msgr2=0x7f905010a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:55.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.754+0000 7f9056248700 1 --2- 172.21.15.37:0/136358725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9050104090 0x7f905010a940 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f904800b0d0 tx=0x7f904800b3e0 comp rx=0 tx=0).stop 2024-06-10T00:55:55.756 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:55 smithi037 ceph-mon[32192]: pgmap v407: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T00:55:55.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.754+0000 7f9056248700 1 -- 172.21.15.37:0/136358725 shutdown_connections 2024-06-10T00:55:55.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.754+0000 7f9056248700 1 --2- 172.21.15.37:0/136358725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9050104090 0x7f905010a940 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:55.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.754+0000 7f9056248700 1 --2- 172.21.15.37:0/136358725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90501024f0 0x7f9050102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:55.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.754+0000 7f9056248700 1 --2- 172.21.15.37:0/136358725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90501036f0 0x7f9050103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:55.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.754+0000 7f9056248700 1 -- 172.21.15.37:0/136358725 >> 172.21.15.37:0/136358725 conn(0x7f90500fda90 msgr2=0x7f90500ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:55.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.754+0000 7f9056248700 1 -- 172.21.15.37:0/136358725 shutdown_connections 2024-06-10T00:55:55.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.754+0000 7f9056248700 1 -- 172.21.15.37:0/136358725 wait complete. 2024-06-10T00:55:55.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.755+0000 7f9056248700 1 Processor -- start 2024-06-10T00:55:55.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.755+0000 7f9056248700 1 -- start start 2024-06-10T00:55:55.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.755+0000 7f9056248700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90501024f0 0x7f90501986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:55.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.755+0000 7f9056248700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90501036f0 0x7f9050198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:55.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.755+0000 7f9056248700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9050104090 0x7f905019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:55.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.755+0000 7f9056248700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f905010e130 con 0x7f90501024f0 2024-06-10T00:55:55.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.755+0000 7f9056248700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f905010dfb0 con 0x7f9050104090 2024-06-10T00:55:55.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.755+0000 7f9056248700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f905010e2b0 con 0x7f90501036f0 2024-06-10T00:55:55.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.755+0000 7f904f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90501024f0 0x7f90501986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:55.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.755+0000 7f904f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90501024f0 0x7f90501986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45344/0 (socket says 172.21.15.37:45344) 2024-06-10T00:55:55.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.755+0000 7f904f7fe700 1 -- 172.21.15.37:0/1250100947 learned_addr learned my addr 172.21.15.37:0/1250100947 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:55.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.756+0000 7f904ffff700 1 --2- 172.21.15.37:0/1250100947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9050104090 0x7f905019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:55.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.756+0000 7f904effd700 1 --2- 172.21.15.37:0/1250100947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90501036f0 0x7f9050198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:55.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.756+0000 7f904f7fe700 1 -- 172.21.15.37:0/1250100947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90501036f0 msgr2=0x7f9050198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:55.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.756+0000 7f904f7fe700 1 --2- 172.21.15.37:0/1250100947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90501036f0 0x7f9050198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:55.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.756+0000 7f904f7fe700 1 -- 172.21.15.37:0/1250100947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9050104090 msgr2=0x7f905019fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:55.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.756+0000 7f904f7fe700 1 --2- 172.21.15.37:0/1250100947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9050104090 0x7f905019fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:55.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.756+0000 7f904f7fe700 1 -- 172.21.15.37:0/1250100947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90501a0310 con 0x7f90501024f0 2024-06-10T00:55:55.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.756+0000 7f904effd700 1 --2- 172.21.15.37:0/1250100947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90501036f0 0x7f9050198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:55:55.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.756+0000 7f904f7fe700 1 --2- 172.21.15.37:0/1250100947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90501024f0 0x7f90501986b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f903800d9b0 tx=0x7f903800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:55.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.756+0000 7f904cff9700 1 -- 172.21.15.37:0/1250100947 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9038016070 con 0x7f90501024f0 2024-06-10T00:55:55.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.756+0000 7f904cff9700 1 -- 172.21.15.37:0/1250100947 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f903800be30 con 0x7f90501024f0 2024-06-10T00:55:55.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.756+0000 7f9056248700 1 -- 172.21.15.37:0/1250100947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90501a0600 con 0x7f90501024f0 2024-06-10T00:55:55.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.756+0000 7f904cff9700 1 -- 172.21.15.37:0/1250100947 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903800f3c0 con 0x7f90501024f0 2024-06-10T00:55:55.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.756+0000 7f9056248700 1 -- 172.21.15.37:0/1250100947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90501a0b90 con 0x7f90501024f0 2024-06-10T00:55:55.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.757+0000 7f9056248700 1 -- 172.21.15.37:0/1250100947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9050069180 con 0x7f90501024f0 2024-06-10T00:55:55.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.759+0000 7f904cff9700 1 -- 172.21.15.37:0/1250100947 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9038019070 con 0x7f90501024f0 2024-06-10T00:55:55.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.760+0000 7f904cff9700 1 --2- 172.21.15.37:0/1250100947 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f903c0a2ca0 0x7f903c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:55.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.760+0000 7f904effd700 1 --2- 172.21.15.37:0/1250100947 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f903c0a2ca0 0x7f903c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:55.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.760+0000 7f904cff9700 1 -- 172.21.15.37:0/1250100947 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f90380cf770 con 0x7f90501024f0 2024-06-10T00:55:55.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.760+0000 7f904effd700 1 --2- 172.21.15.37:0/1250100947 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f903c0a2ca0 0x7f903c0a5160 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f90400099d0 tx=0x7f9040008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:55.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.764+0000 7f904cff9700 1 -- 172.21.15.37:0/1250100947 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9038020020 con 0x7f90501024f0 2024-06-10T00:55:55.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.982+0000 7f9056248700 1 -- 172.21.15.37:0/1250100947 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f90500628d0 con 0x7f903c0a2ca0 2024-06-10T00:55:55.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.988+0000 7f904cff9700 1 -- 172.21.15.37:0/1250100947 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f90500628d0 con 0x7f903c0a2ca0 2024-06-10T00:55:55.990 INFO:teuthology.orchestra.run.smithi037.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-06-10T00:55:55.990 INFO:teuthology.orchestra.run.smithi037.stdout:mds.all 12/12 79s ago 4m smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi094=b;smithi094=e;smithi094=h;smithi094=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-06-10T00:55:55.990 INFO:teuthology.orchestra.run.smithi037.stdout:mgr 3/3 79s ago 9m smithi037=x;smithi094=y;smithi167=z;count:3 2024-06-10T00:55:55.990 INFO:teuthology.orchestra.run.smithi037.stdout:mon 3/3 79s ago 9m smithi037:172.21.15.37=a;smithi094:172.21.15.94=b;smithi167:172.21.15.167=c;count:3 2024-06-10T00:55:55.990 INFO:teuthology.orchestra.run.smithi037.stdout:osd 12 79s ago - 2024-06-10T00:55:55.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.990+0000 7f9056248700 1 -- 172.21.15.37:0/1250100947 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f903c0a2ca0 msgr2=0x7f903c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:55.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.990+0000 7f9056248700 1 --2- 172.21.15.37:0/1250100947 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f903c0a2ca0 0x7f903c0a5160 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f90400099d0 tx=0x7f9040008040 comp rx=0 tx=0).stop 2024-06-10T00:55:55.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.990+0000 7f9056248700 1 -- 172.21.15.37:0/1250100947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90501024f0 msgr2=0x7f90501986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:55.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.990+0000 7f9056248700 1 --2- 172.21.15.37:0/1250100947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90501024f0 0x7f90501986b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f903800d9b0 tx=0x7f903800dcc0 comp rx=0 tx=0).stop 2024-06-10T00:55:55.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.990+0000 7f9056248700 1 -- 172.21.15.37:0/1250100947 shutdown_connections 2024-06-10T00:55:55.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.990+0000 7f9056248700 1 --2- 172.21.15.37:0/1250100947 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f903c0a2ca0 0x7f903c0a5160 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:55.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.990+0000 7f9056248700 1 --2- 172.21.15.37:0/1250100947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9050104090 0x7f905019fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:55.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.990+0000 7f9056248700 1 --2- 172.21.15.37:0/1250100947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90501024f0 0x7f90501986b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:55.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.990+0000 7f9056248700 1 --2- 172.21.15.37:0/1250100947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90501036f0 0x7f9050198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:55.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.990+0000 7f9056248700 1 -- 172.21.15.37:0/1250100947 >> 172.21.15.37:0/1250100947 conn(0x7f90500fda90 msgr2=0x7f9050106990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:55.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.991+0000 7f9056248700 1 -- 172.21.15.37:0/1250100947 shutdown_connections 2024-06-10T00:55:55.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:55.991+0000 7f9056248700 1 -- 172.21.15.37:0/1250100947 wait complete. 2024-06-10T00:55:56.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:55 smithi167 ceph-mon[36917]: pgmap v407: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T00:55:56.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:55 smithi094 ceph-mon[36708]: pgmap v407: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T00:55:56.700 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch host ls' 2024-06-10T00:55:56.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:56 smithi037 ceph-mon[32192]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:55:56.954 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:55:57.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:56 smithi167 ceph-mon[36917]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:55:57.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:56 smithi094 ceph-mon[36708]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:55:57.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:57 smithi037 ceph-mon[32192]: pgmap v408: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:55:58.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:57 smithi167 ceph-mon[36917]: pgmap v408: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:55:58.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:57 smithi094 ceph-mon[36708]: pgmap v408: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:55:58.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.669+0000 7fdee6abe700 1 -- 172.21.15.37:0/226105787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee0108260 msgr2=0x7fdee010a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:58.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.669+0000 7fdee6abe700 1 --2- 172.21.15.37:0/226105787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee0108260 0x7fdee010a670 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fded800b0a0 tx=0x7fded800b3b0 comp rx=0 tx=0).stop 2024-06-10T00:55:58.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.669+0000 7fdee6abe700 1 -- 172.21.15.37:0/226105787 shutdown_connections 2024-06-10T00:55:58.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.669+0000 7fdee6abe700 1 --2- 172.21.15.37:0/226105787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee0108260 0x7fdee010a670 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:58.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.669+0000 7fdee6abe700 1 --2- 172.21.15.37:0/226105787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdee0103000 0x7fdee01053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:58.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.669+0000 7fdee6abe700 1 --2- 172.21.15.37:0/226105787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdee0105930 0x7fdee0107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:58.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.669+0000 7fdee6abe700 1 -- 172.21.15.37:0/226105787 >> 172.21.15.37:0/226105787 conn(0x7fdee00fc9b0 msgr2=0x7fdee00fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:58.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.669+0000 7fdee6abe700 1 -- 172.21.15.37:0/226105787 shutdown_connections 2024-06-10T00:55:58.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.669+0000 7fdee6abe700 1 -- 172.21.15.37:0/226105787 wait complete. 2024-06-10T00:55:58.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.670+0000 7fdee6abe700 1 Processor -- start 2024-06-10T00:55:58.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.670+0000 7fdee6abe700 1 -- start start 2024-06-10T00:55:58.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.670+0000 7fdee6abe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdee0103000 0x7fdee0198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:58.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.670+0000 7fdee6abe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee0105930 0x7fdee0198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:58.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.670+0000 7fdee6abe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdee0108260 0x7fdee019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:58.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.670+0000 7fdee6abe700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdee010e120 con 0x7fdee0105930 2024-06-10T00:55:58.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.670+0000 7fdee6abe700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdee010dfa0 con 0x7fdee0108260 2024-06-10T00:55:58.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.670+0000 7fdee6abe700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdee010e2a0 con 0x7fdee0103000 2024-06-10T00:55:58.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.670+0000 7fdedffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee0105930 0x7fdee0198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:58.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.670+0000 7fdedffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee0105930 0x7fdee0198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45356/0 (socket says 172.21.15.37:45356) 2024-06-10T00:55:58.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.670+0000 7fdedffff700 1 -- 172.21.15.37:0/506481137 learned_addr learned my addr 172.21.15.37:0/506481137 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:55:58.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.670+0000 7fdee505b700 1 --2- 172.21.15.37:0/506481137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdee0108260 0x7fdee019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:58.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.670+0000 7fdee485a700 1 --2- 172.21.15.37:0/506481137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdee0103000 0x7fdee0198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:58.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.670+0000 7fdedffff700 1 -- 172.21.15.37:0/506481137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdee0103000 msgr2=0x7fdee0198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:58.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.670+0000 7fdedffff700 1 --2- 172.21.15.37:0/506481137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdee0103000 0x7fdee0198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:58.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.670+0000 7fdedffff700 1 -- 172.21.15.37:0/506481137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdee0108260 msgr2=0x7fdee019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:58.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.670+0000 7fdedffff700 1 --2- 172.21.15.37:0/506481137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdee0108260 0x7fdee019fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:58.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.670+0000 7fdedffff700 1 -- 172.21.15.37:0/506481137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee01a02d0 con 0x7fdee0105930 2024-06-10T00:55:58.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.670+0000 7fdee485a700 1 --2- 172.21.15.37:0/506481137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdee0103000 0x7fdee0198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:55:58.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.671+0000 7fdedffff700 1 --2- 172.21.15.37:0/506481137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee0105930 0x7fdee0198bb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00d980 tx=0x7fdecc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:58.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.671+0000 7fdeddffb700 1 -- 172.21.15.37:0/506481137 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc016070 con 0x7fdee0105930 2024-06-10T00:55:58.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.671+0000 7fdee6abe700 1 -- 172.21.15.37:0/506481137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee01a05c0 con 0x7fdee0105930 2024-06-10T00:55:58.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.671+0000 7fdeddffb700 1 -- 172.21.15.37:0/506481137 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdecc00de70 con 0x7fdee0105930 2024-06-10T00:55:58.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.671+0000 7fdeddffb700 1 -- 172.21.15.37:0/506481137 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc00f3c0 con 0x7fdee0105930 2024-06-10T00:55:58.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.671+0000 7fdee6abe700 1 -- 172.21.15.37:0/506481137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee01a0b00 con 0x7fdee0105930 2024-06-10T00:55:58.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.673+0000 7fdee6abe700 1 -- 172.21.15.37:0/506481137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdee0069180 con 0x7fdee0105930 2024-06-10T00:55:58.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.673+0000 7fdeddffb700 1 -- 172.21.15.37:0/506481137 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdecc019070 con 0x7fdee0105930 2024-06-10T00:55:58.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.674+0000 7fdeddffb700 1 --2- 172.21.15.37:0/506481137 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fded00a2c20 0x7fded00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:55:58.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.674+0000 7fdeddffb700 1 -- 172.21.15.37:0/506481137 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fdecc0cefc0 con 0x7fdee0105930 2024-06-10T00:55:58.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.674+0000 7fdee485a700 1 --2- 172.21.15.37:0/506481137 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fded00a2c20 0x7fded00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:55:58.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.674+0000 7fdee485a700 1 --2- 172.21.15.37:0/506481137 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fded00a2c20 0x7fded00a50e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fded400bcf0 tx=0x7fded400b2c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:55:58.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.678+0000 7fdeddffb700 1 -- 172.21.15.37:0/506481137 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdecc09bb30 con 0x7fdee0105930 2024-06-10T00:55:58.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.889+0000 7fdee6abe700 1 -- 172.21.15.37:0/506481137 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fdee00628d0 con 0x7fded00a2c20 2024-06-10T00:55:58.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.891+0000 7fdeddffb700 1 -- 172.21.15.37:0/506481137 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fdee00628d0 con 0x7fded00a2c20 2024-06-10T00:55:58.891 INFO:teuthology.orchestra.run.smithi037.stdout:HOST ADDR LABELS STATUS 2024-06-10T00:55:58.891 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037 172.21.15.37 2024-06-10T00:55:58.892 INFO:teuthology.orchestra.run.smithi037.stdout:smithi094 172.21.15.94 2024-06-10T00:55:58.892 INFO:teuthology.orchestra.run.smithi037.stdout:smithi167 172.21.15.167 2024-06-10T00:55:58.892 INFO:teuthology.orchestra.run.smithi037.stdout:3 hosts in cluster 2024-06-10T00:55:58.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.893+0000 7fdee6abe700 1 -- 172.21.15.37:0/506481137 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fded00a2c20 msgr2=0x7fded00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:58.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.893+0000 7fdee6abe700 1 --2- 172.21.15.37:0/506481137 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fded00a2c20 0x7fded00a50e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fded400bcf0 tx=0x7fded400b2c0 comp rx=0 tx=0).stop 2024-06-10T00:55:58.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.893+0000 7fdee6abe700 1 -- 172.21.15.37:0/506481137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee0105930 msgr2=0x7fdee0198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:55:58.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.893+0000 7fdee6abe700 1 --2- 172.21.15.37:0/506481137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee0105930 0x7fdee0198bb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00d980 tx=0x7fdecc00dc90 comp rx=0 tx=0).stop 2024-06-10T00:55:58.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.893+0000 7fdee6abe700 1 -- 172.21.15.37:0/506481137 shutdown_connections 2024-06-10T00:55:58.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.893+0000 7fdee6abe700 1 --2- 172.21.15.37:0/506481137 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fded00a2c20 0x7fded00a50e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:58.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.893+0000 7fdee6abe700 1 --2- 172.21.15.37:0/506481137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdee0108260 0x7fdee019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:58.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.893+0000 7fdee6abe700 1 --2- 172.21.15.37:0/506481137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdee0103000 0x7fdee0198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:58.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.893+0000 7fdee6abe700 1 --2- 172.21.15.37:0/506481137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee0105930 0x7fdee0198bb0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:55:58.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.893+0000 7fdee6abe700 1 -- 172.21.15.37:0/506481137 >> 172.21.15.37:0/506481137 conn(0x7fdee00fc9b0 msgr2=0x7fdee0103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:55:58.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.893+0000 7fdee6abe700 1 -- 172.21.15.37:0/506481137 shutdown_connections 2024-06-10T00:55:58.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:55:58.893+0000 7fdee6abe700 1 -- 172.21.15.37:0/506481137 wait complete. 2024-06-10T00:55:59.599 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch device ls' 2024-06-10T00:55:59.851 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:55:59.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:59 smithi037 ceph-mon[32192]: pgmap v409: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:55:59.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:55:59 smithi037 ceph-mon[32192]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:56:00.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:59 smithi167 ceph-mon[36917]: pgmap v409: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:56:00.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:55:59 smithi167 ceph-mon[36917]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:56:00.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:59 smithi094 ceph-mon[36708]: pgmap v409: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:56:00.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:55:59 smithi094 ceph-mon[36708]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:56:01.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.587+0000 7fe63b527700 1 -- 172.21.15.37:0/3970288660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe634104060 msgr2=0x7fe63410a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:01.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.587+0000 7fe63b527700 1 --2- 172.21.15.37:0/3970288660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe634104060 0x7fe63410a910 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00b0d0 tx=0x7fe62c00b3e0 comp rx=0 tx=0).stop 2024-06-10T00:56:01.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.587+0000 7fe63b527700 1 -- 172.21.15.37:0/3970288660 shutdown_connections 2024-06-10T00:56:01.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.587+0000 7fe63b527700 1 --2- 172.21.15.37:0/3970288660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe634104060 0x7fe63410a910 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:01.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.587+0000 7fe63b527700 1 --2- 172.21.15.37:0/3970288660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6341024c0 0x7fe6341028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:01.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.587+0000 7fe63b527700 1 --2- 172.21.15.37:0/3970288660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6341036c0 0x7fe634103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:01.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.587+0000 7fe63b527700 1 -- 172.21.15.37:0/3970288660 >> 172.21.15.37:0/3970288660 conn(0x7fe6340fda60 msgr2=0x7fe6340ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:01.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.588+0000 7fe63b527700 1 -- 172.21.15.37:0/3970288660 shutdown_connections 2024-06-10T00:56:01.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.588+0000 7fe63b527700 1 -- 172.21.15.37:0/3970288660 wait complete. 2024-06-10T00:56:01.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.589+0000 7fe63b527700 1 Processor -- start 2024-06-10T00:56:01.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.589+0000 7fe63b527700 1 -- start start 2024-06-10T00:56:01.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.589+0000 7fe63b527700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6341024c0 0x7fe6341986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:01.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.589+0000 7fe63b527700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6341036c0 0x7fe634198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:01.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.589+0000 7fe63b527700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe634104060 0x7fe63419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:01.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.589+0000 7fe63b527700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe63410e100 con 0x7fe634104060 2024-06-10T00:56:01.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.589+0000 7fe63b527700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe63410df80 con 0x7fe6341036c0 2024-06-10T00:56:01.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.589+0000 7fe63b527700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe63410e280 con 0x7fe6341024c0 2024-06-10T00:56:01.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.589+0000 7fe639ac4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe634104060 0x7fe63419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:01.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.589+0000 7fe639ac4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe634104060 0x7fe63419fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45384/0 (socket says 172.21.15.37:45384) 2024-06-10T00:56:01.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.589+0000 7fe639ac4700 1 -- 172.21.15.37:0/2365977557 learned_addr learned my addr 172.21.15.37:0/2365977557 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:01.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.589+0000 7fe6392c3700 1 --2- 172.21.15.37:0/2365977557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6341024c0 0x7fe6341986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:01.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.589+0000 7fe638ac2700 1 --2- 172.21.15.37:0/2365977557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6341036c0 0x7fe634198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:01.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.590+0000 7fe639ac4700 1 -- 172.21.15.37:0/2365977557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6341024c0 msgr2=0x7fe6341986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:01.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.590+0000 7fe639ac4700 1 --2- 172.21.15.37:0/2365977557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6341024c0 0x7fe6341986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:01.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.590+0000 7fe639ac4700 1 -- 172.21.15.37:0/2365977557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6341036c0 msgr2=0x7fe634198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:01.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.590+0000 7fe639ac4700 1 --2- 172.21.15.37:0/2365977557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6341036c0 0x7fe634198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:01.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.590+0000 7fe639ac4700 1 -- 172.21.15.37:0/2365977557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6341a0350 con 0x7fe634104060 2024-06-10T00:56:01.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.590+0000 7fe6392c3700 1 --2- 172.21.15.37:0/2365977557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6341024c0 0x7fe6341986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:56:01.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.590+0000 7fe638ac2700 1 --2- 172.21.15.37:0/2365977557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6341036c0 0x7fe634198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:56:01.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.590+0000 7fe639ac4700 1 --2- 172.21.15.37:0/2365977557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe634104060 0x7fe63419fcb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe62c004da0 tx=0x7fe62c004e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:01.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.590+0000 7fe62a7fc700 1 -- 172.21.15.37:0/2365977557 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c021020 con 0x7fe634104060 2024-06-10T00:56:01.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.590+0000 7fe62a7fc700 1 -- 172.21.15.37:0/2365977557 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe62c007c80 con 0x7fe634104060 2024-06-10T00:56:01.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.590+0000 7fe63b527700 1 -- 172.21.15.37:0/2365977557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6341a05e0 con 0x7fe634104060 2024-06-10T00:56:01.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.590+0000 7fe62a7fc700 1 -- 172.21.15.37:0/2365977557 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c005400 con 0x7fe634104060 2024-06-10T00:56:01.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.590+0000 7fe63b527700 1 -- 172.21.15.37:0/2365977557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6341a0a90 con 0x7fe634104060 2024-06-10T00:56:01.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.591+0000 7fe63b527700 1 -- 172.21.15.37:0/2365977557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe634069180 con 0x7fe634104060 2024-06-10T00:56:01.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.593+0000 7fe62a7fc700 1 -- 172.21.15.37:0/2365977557 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe62c024020 con 0x7fe634104060 2024-06-10T00:56:01.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.594+0000 7fe62a7fc700 1 --2- 172.21.15.37:0/2365977557 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe6200a2c20 0x7fe6200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:01.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.594+0000 7fe6392c3700 1 --2- 172.21.15.37:0/2365977557 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe6200a2c20 0x7fe6200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:01.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.594+0000 7fe62a7fc700 1 -- 172.21.15.37:0/2365977557 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fe62c00f020 con 0x7fe634104060 2024-06-10T00:56:01.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.594+0000 7fe6392c3700 1 --2- 172.21.15.37:0/2365977557 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe6200a2c20 0x7fe6200a50e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe630005fd0 tx=0x7fe630005f00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:01.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.597+0000 7fe62a7fc700 1 -- 172.21.15.37:0/2365977557 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe62c09d730 con 0x7fe634104060 2024-06-10T00:56:01.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.818+0000 7fe63b527700 1 -- 172.21.15.37:0/2365977557 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fe6340628d0 con 0x7fe6200a2c20 2024-06-10T00:56:01.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.820+0000 7fe62a7fc700 1 -- 172.21.15.37:0/2365977557 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fe6340628d0 con 0x7fe6200a2c20 2024-06-10T00:56:01.820 INFO:teuthology.orchestra.run.smithi037.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-06-10T00:56:01.821 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331005T400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-06-10T00:56:01.821 INFO:teuthology.orchestra.run.smithi037.stdout:smithi094 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513001L480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-06-10T00:56:01.821 INFO:teuthology.orchestra.run.smithi037.stdout:smithi167 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62330080400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-06-10T00:56:01.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.822+0000 7fe63b527700 1 -- 172.21.15.37:0/2365977557 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe6200a2c20 msgr2=0x7fe6200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:01.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.822+0000 7fe63b527700 1 --2- 172.21.15.37:0/2365977557 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe6200a2c20 0x7fe6200a50e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe630005fd0 tx=0x7fe630005f00 comp rx=0 tx=0).stop 2024-06-10T00:56:01.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.822+0000 7fe63b527700 1 -- 172.21.15.37:0/2365977557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe634104060 msgr2=0x7fe63419fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:01.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.822+0000 7fe63b527700 1 --2- 172.21.15.37:0/2365977557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe634104060 0x7fe63419fcb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe62c004da0 tx=0x7fe62c004e80 comp rx=0 tx=0).stop 2024-06-10T00:56:01.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.822+0000 7fe63b527700 1 -- 172.21.15.37:0/2365977557 shutdown_connections 2024-06-10T00:56:01.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.822+0000 7fe63b527700 1 --2- 172.21.15.37:0/2365977557 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe6200a2c20 0x7fe6200a50e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:01.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.822+0000 7fe63b527700 1 --2- 172.21.15.37:0/2365977557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe634104060 0x7fe63419fcb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:01.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.822+0000 7fe63b527700 1 --2- 172.21.15.37:0/2365977557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6341024c0 0x7fe6341986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:01.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.822+0000 7fe63b527700 1 --2- 172.21.15.37:0/2365977557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6341036c0 0x7fe634198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:01.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.822+0000 7fe63b527700 1 -- 172.21.15.37:0/2365977557 >> 172.21.15.37:0/2365977557 conn(0x7fe6340fda60 msgr2=0x7fe634106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:01.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.823+0000 7fe63b527700 1 -- 172.21.15.37:0/2365977557 shutdown_connections 2024-06-10T00:56:01.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:01.823+0000 7fe63b527700 1 -- 172.21.15.37:0/2365977557 wait complete. 2024-06-10T00:56:01.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:01 smithi037 ceph-mon[32192]: pgmap v410: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:56:02.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:01 smithi167 ceph-mon[36917]: pgmap v410: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:56:02.118 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:01 smithi094 ceph-mon[36708]: pgmap v410: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:56:02.512 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-10T00:56:02.524 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-06-10T00:56:02.524 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs dump' 2024-06-10T00:56:02.781 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:56:02.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:02 smithi037 ceph-mon[32192]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:56:03.016 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:02 smithi094 ceph-mon[36708]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:56:03.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:02 smithi167 ceph-mon[36917]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:56:03.880 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:03 smithi037 ceph-mon[32192]: pgmap v411: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:56:03.990 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:03 smithi167 ceph-mon[36917]: pgmap v411: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:56:04.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:03 smithi094 ceph-mon[36708]: pgmap v411: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:56:04.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.470+0000 7f2efa3a8700 1 -- 172.21.15.37:0/1313040133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ef41024e0 msgr2=0x7f2ef4102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:04.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.470+0000 7f2efa3a8700 1 --2- 172.21.15.37:0/1313040133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ef41024e0 0x7f2ef4102900 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40099c0 tx=0x7f2ee4009cd0 comp rx=0 tx=0).stop 2024-06-10T00:56:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.470+0000 7f2efa3a8700 1 -- 172.21.15.37:0/1313040133 shutdown_connections 2024-06-10T00:56:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.470+0000 7f2efa3a8700 1 --2- 172.21.15.37:0/1313040133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ef4104080 0x7f2ef410a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.470+0000 7f2efa3a8700 1 --2- 172.21.15.37:0/1313040133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ef41024e0 0x7f2ef4102900 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.470+0000 7f2efa3a8700 1 --2- 172.21.15.37:0/1313040133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ef41036e0 0x7f2ef4103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.470+0000 7f2efa3a8700 1 -- 172.21.15.37:0/1313040133 >> 172.21.15.37:0/1313040133 conn(0x7f2ef40fda40 msgr2=0x7f2ef40ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.470+0000 7f2efa3a8700 1 -- 172.21.15.37:0/1313040133 shutdown_connections 2024-06-10T00:56:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.470+0000 7f2efa3a8700 1 -- 172.21.15.37:0/1313040133 wait complete. 2024-06-10T00:56:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.471+0000 7f2efa3a8700 1 Processor -- start 2024-06-10T00:56:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.471+0000 7f2efa3a8700 1 -- start start 2024-06-10T00:56:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.471+0000 7f2efa3a8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ef41024e0 0x7f2ef4198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.471+0000 7f2efa3a8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ef41036e0 0x7f2ef4198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.471+0000 7f2efa3a8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ef4104080 0x7f2ef419fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.471+0000 7f2efa3a8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef410e1a0 con 0x7f2ef41036e0 2024-06-10T00:56:04.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.471+0000 7f2efa3a8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef410e020 con 0x7f2ef41024e0 2024-06-10T00:56:04.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.471+0000 7f2efa3a8700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef410e320 con 0x7f2ef4104080 2024-06-10T00:56:04.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.471+0000 7f2eebfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ef41036e0 0x7f2ef4198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:04.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.471+0000 7f2eebfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ef41036e0 0x7f2ef4198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52032/0 (socket says 172.21.15.37:52032) 2024-06-10T00:56:04.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.471+0000 7f2eebfff700 1 -- 172.21.15.37:0/2136464833 learned_addr learned my addr 172.21.15.37:0/2136464833 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:04.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.471+0000 7f2ef8945700 1 --2- 172.21.15.37:0/2136464833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ef4104080 0x7f2ef419fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:04.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.471+0000 7f2ef3fff700 1 --2- 172.21.15.37:0/2136464833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ef41024e0 0x7f2ef4198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:04.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.472+0000 7f2eebfff700 1 -- 172.21.15.37:0/2136464833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ef4104080 msgr2=0x7f2ef419fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:04.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.472+0000 7f2eebfff700 1 --2- 172.21.15.37:0/2136464833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ef4104080 0x7f2ef419fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:04.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.472+0000 7f2eebfff700 1 -- 172.21.15.37:0/2136464833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ef41024e0 msgr2=0x7f2ef4198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:04.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.472+0000 7f2eebfff700 1 --2- 172.21.15.37:0/2136464833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ef41024e0 0x7f2ef4198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:04.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.472+0000 7f2eebfff700 1 -- 172.21.15.37:0/2136464833 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef41a04d0 con 0x7f2ef41036e0 2024-06-10T00:56:04.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.472+0000 7f2eebfff700 1 --2- 172.21.15.37:0/2136464833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ef41036e0 0x7f2ef4198db0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00b560 tx=0x7f2edc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.472+0000 7f2ef1ffb700 1 -- 172.21.15.37:0/2136464833 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc015020 con 0x7f2ef41036e0 2024-06-10T00:56:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.472+0000 7f2efa3a8700 1 -- 172.21.15.37:0/2136464833 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef41a07c0 con 0x7f2ef41036e0 2024-06-10T00:56:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.472+0000 7f2ef1ffb700 1 -- 172.21.15.37:0/2136464833 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2edc011d10 con 0x7f2ef41036e0 2024-06-10T00:56:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.472+0000 7f2ef1ffb700 1 -- 172.21.15.37:0/2136464833 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc00f2e0 con 0x7f2ef41036e0 2024-06-10T00:56:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.472+0000 7f2efa3a8700 1 -- 172.21.15.37:0/2136464833 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef41a0dd0 con 0x7f2ef41036e0 2024-06-10T00:56:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.473+0000 7f2efa3a8700 1 -- 172.21.15.37:0/2136464833 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ef4069180 con 0x7f2ef41036e0 2024-06-10T00:56:04.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.475+0000 7f2ef1ffb700 1 -- 172.21.15.37:0/2136464833 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2edc018020 con 0x7f2ef41036e0 2024-06-10T00:56:04.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.476+0000 7f2ef1ffb700 1 --2- 172.21.15.37:0/2136464833 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2ee00a2ca0 0x7f2ee00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:04.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.476+0000 7f2ef3fff700 1 --2- 172.21.15.37:0/2136464833 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2ee00a2ca0 0x7f2ee00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:04.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.476+0000 7f2ef1ffb700 1 -- 172.21.15.37:0/2136464833 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f2edc0cef90 con 0x7f2ef41036e0 2024-06-10T00:56:04.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.476+0000 7f2ef3fff700 1 --2- 172.21.15.37:0/2136464833 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2ee00a2ca0 0x7f2ee00a5160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400b580 tx=0x7f2ee4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:04.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.479+0000 7f2ef1ffb700 1 -- 172.21.15.37:0/2136464833 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2edc01f020 con 0x7f2ef41036e0 2024-06-10T00:56:04.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.733+0000 7f2efa3a8700 1 -- 172.21.15.37:0/2136464833 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f2ef402dd20 con 0x7f2ef41036e0 2024-06-10T00:56:04.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.733+0000 7f2ef1ffb700 1 -- 172.21.15.37:0/2136464833 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3048 (secure 0 0 0) 0x7f2edc09bb00 con 0x7f2ef41036e0 2024-06-10T00:56:04.734 INFO:teuthology.orchestra.run.smithi037.stdout:e26 2024-06-10T00:56:04.734 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-10T00:56:04.734 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-10T00:56:04.734 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-06-10T00:56:04.734 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:56:04.735 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-06-10T00:56:04.735 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-06-10T00:56:04.735 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 26 2024-06-10T00:56:04.735 INFO:teuthology.orchestra.run.smithi037.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-06-10T00:56:04.735 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-06-10T00:52:37.869277+0000 2024-06-10T00:56:04.735 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-06-10T00:53:44.125049+0000 2024-06-10T00:56:04.735 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-06-10T00:56:04.735 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-06-10T00:56:04.735 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 300 2024-06-10T00:56:04.735 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-06-10T00:56:04.735 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-06-10T00:56:04.735 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-06-10T00:56:04.735 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-06-10T00:56:04.735 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-06-10T00:56:04.735 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 0 2024-06-10T00:56:04.736 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-10T00:56:04.736 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 3 2024-06-10T00:56:04.736 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1,2 2024-06-10T00:56:04.736 INFO:teuthology.orchestra.run.smithi037.stdout:up {0=24472,1=24458,2=24451} 2024-06-10T00:56:04.736 INFO:teuthology.orchestra.run.smithi037.stdout:failed 2024-06-10T00:56:04.736 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-06-10T00:56:04.736 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-06-10T00:56:04.736 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3,4] 2024-06-10T00:56:04.736 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-06-10T00:56:04.736 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-06-10T00:56:04.736 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-06-10T00:56:04.736 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-06-10T00:56:04.736 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 0 2024-06-10T00:56:04.736 INFO:teuthology.orchestra.run.smithi037.stdout:qdb_cluster leader: 24472 members: 24458,24472,24451 2024-06-10T00:56:04.737 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.i{0:24472} state up:active seq 8 addr [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T00:56:04.737 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.f{0:24445} state up:standby-replay seq 1 addr [v2:172.21.15.167:6834/3066042611,v1:172.21.15.167:6835/3066042611] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T00:56:04.737 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.b{1:24458} state up:active seq 21 addr [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T00:56:04.737 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.h{1:24431} state up:standby-replay seq 1 addr [v2:172.21.15.94:6836/3801637791,v1:172.21.15.94:6837/3801637791] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T00:56:04.737 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.c{2:24451} state up:active seq 25 addr [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T00:56:04.737 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.e{2:24413} state up:standby-replay seq 1 addr [v2:172.21.15.94:6834/570288048,v1:172.21.15.94:6835/570288048] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T00:56:04.737 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:56:04.737 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:56:04.737 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-06-10T00:56:04.737 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:56:04.737 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.j{-1:14526} state up:standby seq 1 addr [v2:172.21.15.37:6834/1021220889,v1:172.21.15.37:6835/1021220889] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T00:56:04.737 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.d{-1:14538} state up:standby seq 1 addr [v2:172.21.15.37:6836/4093234668,v1:172.21.15.37:6837/4093234668] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T00:56:04.737 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.g{-1:14586} state up:standby seq 1 addr [v2:172.21.15.37:6838/595511083,v1:172.21.15.37:6839/595511083] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T00:56:04.738 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.a{-1:14598} state up:standby seq 1 addr [v2:172.21.15.37:6840/2289348383,v1:172.21.15.37:6841/2289348383] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T00:56:04.738 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.k{-1:24398} state up:standby seq 1 addr [v2:172.21.15.94:6832/2915505496,v1:172.21.15.94:6833/2915505496] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T00:56:04.738 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.l{-1:24412} state up:standby seq 1 addr [v2:172.21.15.167:6832/2828145715,v1:172.21.15.167:6833/2828145715] compat {c=[1],r=[1],i=[1fff]}] 2024-06-10T00:56:04.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.737+0000 7f2efa3a8700 1 -- 172.21.15.37:0/2136464833 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2ee00a2ca0 msgr2=0x7f2ee00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:04.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.737+0000 7f2efa3a8700 1 --2- 172.21.15.37:0/2136464833 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2ee00a2ca0 0x7f2ee00a5160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400b580 tx=0x7f2ee4005fb0 comp rx=0 tx=0).stop 2024-06-10T00:56:04.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.737+0000 7f2efa3a8700 1 -- 172.21.15.37:0/2136464833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ef41036e0 msgr2=0x7f2ef4198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:04.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.737+0000 7f2efa3a8700 1 --2- 172.21.15.37:0/2136464833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ef41036e0 0x7f2ef4198db0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00b560 tx=0x7f2edc00b870 comp rx=0 tx=0).stop 2024-06-10T00:56:04.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.737+0000 7f2efa3a8700 1 -- 172.21.15.37:0/2136464833 shutdown_connections 2024-06-10T00:56:04.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.737+0000 7f2efa3a8700 1 --2- 172.21.15.37:0/2136464833 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2ee00a2ca0 0x7f2ee00a5160 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:04.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.737+0000 7f2efa3a8700 1 --2- 172.21.15.37:0/2136464833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ef4104080 0x7f2ef419fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:04.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.737+0000 7f2efa3a8700 1 --2- 172.21.15.37:0/2136464833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ef41024e0 0x7f2ef4198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:04.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.737+0000 7f2efa3a8700 1 --2- 172.21.15.37:0/2136464833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ef41036e0 0x7f2ef4198db0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:04.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.737+0000 7f2efa3a8700 1 -- 172.21.15.37:0/2136464833 >> 172.21.15.37:0/2136464833 conn(0x7f2ef40fda40 msgr2=0x7f2ef4106ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:04.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.737+0000 7f2efa3a8700 1 -- 172.21.15.37:0/2136464833 shutdown_connections 2024-06-10T00:56:04.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:04.737+0000 7f2efa3a8700 1 -- 172.21.15.37:0/2136464833 wait complete. 2024-06-10T00:56:04.739 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-06-10T00:56:05.508 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- bash -c 'ceph osd dump' 2024-06-10T00:56:05.767 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:56:05.890 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:05 smithi037 ceph-mon[32192]: pgmap v412: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T00:56:05.890 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:05 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2136464833' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-10T00:56:06.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:05 smithi167 ceph-mon[36917]: pgmap v412: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T00:56:06.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:05 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2136464833' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-10T00:56:06.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:05 smithi094 ceph-mon[36708]: pgmap v412: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T00:56:06.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:05 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2136464833' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-10T00:56:07.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.473+0000 7f17edfe8700 1 -- 172.21.15.37:0/1692884674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17e8102fa0 msgr2=0x7f17e8105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:07.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.473+0000 7f17edfe8700 1 --2- 172.21.15.37:0/1692884674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17e8102fa0 0x7f17e8105390 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f17d8009990 tx=0x7f17d8009ca0 comp rx=0 tx=0).stop 2024-06-10T00:56:07.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.473+0000 7f17edfe8700 1 -- 172.21.15.37:0/1692884674 shutdown_connections 2024-06-10T00:56:07.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.473+0000 7f17edfe8700 1 --2- 172.21.15.37:0/1692884674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17e8108200 0x7f17e810a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:07.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.473+0000 7f17edfe8700 1 --2- 172.21.15.37:0/1692884674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17e8102fa0 0x7f17e8105390 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:07.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.473+0000 7f17edfe8700 1 --2- 172.21.15.37:0/1692884674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17e81058d0 0x7f17e8107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:07.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.473+0000 7f17edfe8700 1 -- 172.21.15.37:0/1692884674 >> 172.21.15.37:0/1692884674 conn(0x7f17e80fc9b0 msgr2=0x7f17e80fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:07.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.473+0000 7f17edfe8700 1 -- 172.21.15.37:0/1692884674 shutdown_connections 2024-06-10T00:56:07.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.473+0000 7f17edfe8700 1 -- 172.21.15.37:0/1692884674 wait complete. 2024-06-10T00:56:07.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.473+0000 7f17edfe8700 1 Processor -- start 2024-06-10T00:56:07.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.474+0000 7f17edfe8700 1 -- start start 2024-06-10T00:56:07.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.474+0000 7f17edfe8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17e8102fa0 0x7f17e8196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:07.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.474+0000 7f17edfe8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17e81058d0 0x7f17e8196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:07.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.474+0000 7f17edfe8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17e8108200 0x7f17e81a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:07.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.474+0000 7f17edfe8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f17e810e120 con 0x7f17e8108200 2024-06-10T00:56:07.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.474+0000 7f17edfe8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f17e810dfa0 con 0x7f17e81058d0 2024-06-10T00:56:07.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.474+0000 7f17edfe8700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f17e810e2a0 con 0x7f17e8102fa0 2024-06-10T00:56:07.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.474+0000 7f17e7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17e8108200 0x7f17e81a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:07.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.474+0000 7f17e7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17e8108200 0x7f17e81a6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52050/0 (socket says 172.21.15.37:52050) 2024-06-10T00:56:07.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.474+0000 7f17e7fff700 1 -- 172.21.15.37:0/2057996460 learned_addr learned my addr 172.21.15.37:0/2057996460 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:07.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.474+0000 7f17e77fe700 1 --2- 172.21.15.37:0/2057996460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17e8102fa0 0x7f17e8196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:07.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.474+0000 7f17e6ffd700 1 --2- 172.21.15.37:0/2057996460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17e81058d0 0x7f17e8196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:07.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.475+0000 7f17e7fff700 1 -- 172.21.15.37:0/2057996460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17e8102fa0 msgr2=0x7f17e8196550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:07.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.475+0000 7f17e7fff700 1 --2- 172.21.15.37:0/2057996460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17e8102fa0 0x7f17e8196550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:07.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.475+0000 7f17e7fff700 1 -- 172.21.15.37:0/2057996460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17e81058d0 msgr2=0x7f17e8196a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:07.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.475+0000 7f17e7fff700 1 --2- 172.21.15.37:0/2057996460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17e81058d0 0x7f17e8196a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:07.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.475+0000 7f17e7fff700 1 -- 172.21.15.37:0/2057996460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e81a69c0 con 0x7f17e8108200 2024-06-10T00:56:07.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.475+0000 7f17e7fff700 1 --2- 172.21.15.37:0/2057996460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17e8108200 0x7f17e81a6430 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f17e000ada0 tx=0x7f17e000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:07.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.475+0000 7f17e4ff9700 1 -- 172.21.15.37:0/2057996460 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0017020 con 0x7f17e8108200 2024-06-10T00:56:07.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.475+0000 7f17edfe8700 1 -- 172.21.15.37:0/2057996460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17e81a6cb0 con 0x7f17e8108200 2024-06-10T00:56:07.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.475+0000 7f17e4ff9700 1 -- 172.21.15.37:0/2057996460 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17e000cdb0 con 0x7f17e8108200 2024-06-10T00:56:07.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.475+0000 7f17e4ff9700 1 -- 172.21.15.37:0/2057996460 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e00053a0 con 0x7f17e8108200 2024-06-10T00:56:07.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.475+0000 7f17edfe8700 1 -- 172.21.15.37:0/2057996460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17e81a71a0 con 0x7f17e8108200 2024-06-10T00:56:07.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.476+0000 7f17edfe8700 1 -- 172.21.15.37:0/2057996460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17e8069180 con 0x7f17e8108200 2024-06-10T00:56:07.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.478+0000 7f17e4ff9700 1 -- 172.21.15.37:0/2057996460 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f17e000c8d0 con 0x7f17e8108200 2024-06-10T00:56:07.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.478+0000 7f17e4ff9700 1 --2- 172.21.15.37:0/2057996460 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f17d00a2c20 0x7f17d00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:07.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.478+0000 7f17e77fe700 1 --2- 172.21.15.37:0/2057996460 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f17d00a2c20 0x7f17d00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:07.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.478+0000 7f17e4ff9700 1 -- 172.21.15.37:0/2057996460 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f17e00cf080 con 0x7f17e8108200 2024-06-10T00:56:07.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.479+0000 7f17e77fe700 1 --2- 172.21.15.37:0/2057996460 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f17d00a2c20 0x7f17d00a50e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f17d800b5c0 tx=0x7f17d8005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:07.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.482+0000 7f17e4ff9700 1 -- 172.21.15.37:0/2057996460 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f17e0021020 con 0x7f17e8108200 2024-06-10T00:56:07.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.691+0000 7f17edfe8700 1 -- 172.21.15.37:0/2057996460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f17e802dca0 con 0x7f17e8108200 2024-06-10T00:56:07.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.692+0000 7f17e4ff9700 1 -- 172.21.15.37:0/2057996460 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v100) ==== 56+0+5720 (secure 0 0 0) 0x7f17e009bc80 con 0x7f17e8108200 2024-06-10T00:56:07.693 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 100 2024-06-10T00:56:07.693 INFO:teuthology.orchestra.run.smithi037.stdout:fsid 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T00:56:07.693 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-06-10T00:44:24.106713+0000 2024-06-10T00:56:07.693 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-06-10T00:54:22.546967+0000 2024-06-10T00:56:07.693 INFO:teuthology.orchestra.run.smithi037.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-06-10T00:56:07.693 INFO:teuthology.orchestra.run.smithi037.stdout:crush_version 27 2024-06-10T00:56:07.693 INFO:teuthology.orchestra.run.smithi037.stdout:full_ratio 0.95 2024-06-10T00:56:07.693 INFO:teuthology.orchestra.run.smithi037.stdout:backfillfull_ratio 0.9 2024-06-10T00:56:07.693 INFO:teuthology.orchestra.run.smithi037.stdout:nearfull_ratio 0.85 2024-06-10T00:56:07.693 INFO:teuthology.orchestra.run.smithi037.stdout:require_min_compat_client luminous 2024-06-10T00:56:07.693 INFO:teuthology.orchestra.run.smithi037.stdout:min_compat_client luminous 2024-06-10T00:56:07.693 INFO:teuthology.orchestra.run.smithi037.stdout:require_osd_release squid 2024-06-10T00:56:07.694 INFO:teuthology.orchestra.run.smithi037.stdout:stretch_mode_enabled false 2024-06-10T00:56:07.694 INFO:teuthology.orchestra.run.smithi037.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-06-10T00:56:07.694 INFO:teuthology.orchestra.run.smithi037.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-06-10T00:56:07.694 INFO:teuthology.orchestra.run.smithi037.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 89 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-06-10T00:56:07.694 INFO:teuthology.orchestra.run.smithi037.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 96 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-06-10T00:56:07.694 INFO:teuthology.orchestra.run.smithi037.stdout:max_osd 12 2024-06-10T00:56:07.694 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 up in weight 1 up_from 8 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6802/3594704094,v1:172.21.15.37:6803/3594704094] [v2:172.21.15.37:6804/3594704094,v1:172.21.15.37:6805/3594704094] exists,up 1df6046a-6801-4950-9193-54303fed169b 2024-06-10T00:56:07.694 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 up in weight 1 up_from 13 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6800/3427727155,v1:172.21.15.94:6801/3427727155] [v2:172.21.15.94:6802/3427727155,v1:172.21.15.94:6803/3427727155] exists,up 4e1c8944-94e4-4e46-9efe-56229fe6df1a 2024-06-10T00:56:07.694 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 up in weight 1 up_from 18 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6800/4002498292,v1:172.21.15.167:6801/4002498292] [v2:172.21.15.167:6802/4002498292,v1:172.21.15.167:6803/4002498292] exists,up fd2fcedc-4c88-478f-8025-13e5944a27ea 2024-06-10T00:56:07.694 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 up in weight 1 up_from 26 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6810/3425469324,v1:172.21.15.37:6811/3425469324] [v2:172.21.15.37:6812/3425469324,v1:172.21.15.37:6813/3425469324] exists,up d0153676-818c-4dcc-83b1-287435a0c8a8 2024-06-10T00:56:07.694 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 up in weight 1 up_from 34 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6808/1740627431,v1:172.21.15.94:6809/1740627431] [v2:172.21.15.94:6810/1740627431,v1:172.21.15.94:6811/1740627431] exists,up 172c333c-00c0-4e94-8870-acb3b32b2b1b 2024-06-10T00:56:07.694 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 up in weight 1 up_from 39 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6808/221805236,v1:172.21.15.167:6809/221805236] [v2:172.21.15.167:6810/221805236,v1:172.21.15.167:6811/221805236] exists,up dbce656e-5f93-4bda-baf2-fc3cc03d8282 2024-06-10T00:56:07.695 INFO:teuthology.orchestra.run.smithi037.stdout:osd.6 up in weight 1 up_from 47 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6818/472670875,v1:172.21.15.37:6819/472670875] [v2:172.21.15.37:6820/472670875,v1:172.21.15.37:6821/472670875] exists,up fcbda266-6d60-43b2-a51c-4e562476d1a9 2024-06-10T00:56:07.695 INFO:teuthology.orchestra.run.smithi037.stdout:osd.7 up in weight 1 up_from 52 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6816/3932281552,v1:172.21.15.94:6817/3932281552] [v2:172.21.15.94:6818/3932281552,v1:172.21.15.94:6819/3932281552] exists,up ed829ed3-ba85-4e82-ac37-e44e9534d37c 2024-06-10T00:56:07.695 INFO:teuthology.orchestra.run.smithi037.stdout:osd.8 up in weight 1 up_from 60 up_thru 99 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6816/2290216047,v1:172.21.15.167:6817/2290216047] [v2:172.21.15.167:6818/2290216047,v1:172.21.15.167:6819/2290216047] exists,up 7d6267d8-72f4-4704-b554-bafc52d11a11 2024-06-10T00:56:07.695 INFO:teuthology.orchestra.run.smithi037.stdout:osd.9 up in weight 1 up_from 68 up_thru 99 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6826/2063666428,v1:172.21.15.37:6827/2063666428] [v2:172.21.15.37:6828/2063666428,v1:172.21.15.37:6829/2063666428] exists,up 210fdc20-0a36-4eeb-bdf2-c0a7f130e19d 2024-06-10T00:56:07.695 INFO:teuthology.orchestra.run.smithi037.stdout:osd.10 up in weight 1 up_from 73 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6824/1706618899,v1:172.21.15.94:6825/1706618899] [v2:172.21.15.94:6826/1706618899,v1:172.21.15.94:6827/1706618899] exists,up e200b371-b3cf-4b92-a659-f155ca86f0a7 2024-06-10T00:56:07.695 INFO:teuthology.orchestra.run.smithi037.stdout:osd.11 up in weight 1 up_from 81 up_thru 99 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6824/1349892516,v1:172.21.15.167:6825/1349892516] [v2:172.21.15.167:6826/1349892516,v1:172.21.15.167:6827/1349892516] exists,up cfa427df-37a5-4d28-ac2e-af777d607008 2024-06-10T00:56:07.695 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.0 [7,11] 2024-06-10T00:56:07.695 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.b [2,4] 2024-06-10T00:56:07.695 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.10 [2,4] 2024-06-10T00:56:07.695 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.1a [8,1] 2024-06-10T00:56:07.695 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.1c [8,11] 2024-06-10T00:56:07.695 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.0 [5,9] 2024-06-10T00:56:07.695 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.e [4,6] 2024-06-10T00:56:07.695 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.10 [5,9] 2024-06-10T00:56:07.696 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.1d [4,8] 2024-06-10T00:56:07.696 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.23 [4,7] 2024-06-10T00:56:07.696 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.2d [3,9] 2024-06-10T00:56:07.696 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 4.2 [9,1] 2024-06-10T00:56:07.696 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 4.f [9,8] 2024-06-10T00:56:07.696 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 4.10 [9,8] 2024-06-10T00:56:07.696 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 4.12 [5,10] 2024-06-10T00:56:07.696 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/4084722397 expires 2024-06-11T00:45:20.254342+0000 2024-06-10T00:56:07.696 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/3571803585 expires 2024-06-11T00:45:20.254342+0000 2024-06-10T00:56:07.696 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/416587443 expires 2024-06-11T00:45:20.254342+0000 2024-06-10T00:56:07.696 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6801/2766692263 expires 2024-06-11T00:45:20.254342+0000 2024-06-10T00:56:07.696 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/2026344134 expires 2024-06-11T00:44:53.913969+0000 2024-06-10T00:56:07.696 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/2781321886 expires 2024-06-11T00:44:53.913969+0000 2024-06-10T00:56:07.696 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6800/2803174880 expires 2024-06-11T00:44:53.913969+0000 2024-06-10T00:56:07.697 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6801/2803174880 expires 2024-06-11T00:44:53.913969+0000 2024-06-10T00:56:07.697 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/1110143997 expires 2024-06-11T00:44:53.913969+0000 2024-06-10T00:56:07.697 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6800/2766692263 expires 2024-06-11T00:45:20.254342+0000 2024-06-10T00:56:07.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.694+0000 7f17edfe8700 1 -- 172.21.15.37:0/2057996460 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f17d00a2c20 msgr2=0x7f17d00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:07.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.694+0000 7f17edfe8700 1 --2- 172.21.15.37:0/2057996460 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f17d00a2c20 0x7f17d00a50e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f17d800b5c0 tx=0x7f17d8005fd0 comp rx=0 tx=0).stop 2024-06-10T00:56:07.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.694+0000 7f17edfe8700 1 -- 172.21.15.37:0/2057996460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17e8108200 msgr2=0x7f17e81a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:07.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.694+0000 7f17edfe8700 1 --2- 172.21.15.37:0/2057996460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17e8108200 0x7f17e81a6430 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f17e000ada0 tx=0x7f17e000c330 comp rx=0 tx=0).stop 2024-06-10T00:56:07.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.694+0000 7f17edfe8700 1 -- 172.21.15.37:0/2057996460 shutdown_connections 2024-06-10T00:56:07.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.694+0000 7f17edfe8700 1 --2- 172.21.15.37:0/2057996460 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f17d00a2c20 0x7f17d00a50e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:07.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.694+0000 7f17edfe8700 1 --2- 172.21.15.37:0/2057996460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17e8108200 0x7f17e81a6430 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:07.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.694+0000 7f17edfe8700 1 --2- 172.21.15.37:0/2057996460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17e8102fa0 0x7f17e8196550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:07.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.694+0000 7f17edfe8700 1 --2- 172.21.15.37:0/2057996460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17e81058d0 0x7f17e8196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:07.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.694+0000 7f17edfe8700 1 -- 172.21.15.37:0/2057996460 >> 172.21.15.37:0/2057996460 conn(0x7f17e80fc9b0 msgr2=0x7f17e8102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:07.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.694+0000 7f17edfe8700 1 -- 172.21.15.37:0/2057996460 shutdown_connections 2024-06-10T00:56:07.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:07.694+0000 7f17edfe8700 1 -- 172.21.15.37:0/2057996460 wait complete. 2024-06-10T00:56:07.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:07 smithi037 ceph-mon[32192]: pgmap v413: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:56:08.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:07 smithi167 ceph-mon[36917]: pgmap v413: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:56:08.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:07 smithi094 ceph-mon[36708]: pgmap v413: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:56:08.372 INFO:teuthology.run_tasks:Running task fs.ready... 2024-06-10T00:56:08.383 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:56:08.640 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:56:08.900 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:08 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2057996460' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-10T00:56:09.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:08 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2057996460' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-10T00:56:09.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:08 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2057996460' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-10T00:56:09.764 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:09 smithi037 ceph-mon[32192]: pgmap v414: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:56:10.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:09 smithi167 ceph-mon[36917]: pgmap v414: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:56:10.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:09 smithi094 ceph-mon[36708]: pgmap v414: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:56:10.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.317+0000 7f23462eb700 1 -- 172.21.15.37:0/1017583697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2340074fe0 msgr2=0x7f2340113120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:10.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.317+0000 7f23462eb700 1 --2- 172.21.15.37:0/1017583697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2340074fe0 0x7f2340113120 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2334009990 tx=0x7f2334009ca0 comp rx=0 tx=0).stop 2024-06-10T00:56:10.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.317+0000 7f23462eb700 1 -- 172.21.15.37:0/1017583697 shutdown_connections 2024-06-10T00:56:10.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.317+0000 7f23462eb700 1 --2- 172.21.15.37:0/1017583697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2340113660 0x7f2340115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:10.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.317+0000 7f23462eb700 1 --2- 172.21.15.37:0/1017583697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23400746c0 0x7f2340074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:10.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.317+0000 7f23462eb700 1 --2- 172.21.15.37:0/1017583697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2340074fe0 0x7f2340113120 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:10.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.317+0000 7f23462eb700 1 -- 172.21.15.37:0/1017583697 >> 172.21.15.37:0/1017583697 conn(0x7f23400795f0 msgr2=0x7f2340079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:10.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.317+0000 7f23462eb700 1 -- 172.21.15.37:0/1017583697 shutdown_connections 2024-06-10T00:56:10.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.317+0000 7f23462eb700 1 -- 172.21.15.37:0/1017583697 wait complete. 2024-06-10T00:56:10.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.317+0000 7f23462eb700 1 Processor -- start 2024-06-10T00:56:10.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.318+0000 7f23462eb700 1 -- start start 2024-06-10T00:56:10.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.318+0000 7f23462eb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23400746c0 0x7f234019f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:10.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.318+0000 7f23462eb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2340074fe0 0x7f234019f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:10.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.318+0000 7f23462eb700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2340113660 0x7f23401a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:10.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.318+0000 7f23462eb700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2340118e90 con 0x7f23400746c0 2024-06-10T00:56:10.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.318+0000 7f23462eb700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2340118d10 con 0x7f2340074fe0 2024-06-10T00:56:10.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.318+0000 7f23462eb700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2340119010 con 0x7f2340113660 2024-06-10T00:56:10.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.318+0000 7f233ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23400746c0 0x7f234019f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:10.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.318+0000 7f233ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23400746c0 0x7f234019f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52064/0 (socket says 172.21.15.37:52064) 2024-06-10T00:56:10.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.318+0000 7f233ffff700 1 -- 172.21.15.37:0/3695577613 learned_addr learned my addr 172.21.15.37:0/3695577613 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:10.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.318+0000 7f233f7fe700 1 --2- 172.21.15.37:0/3695577613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2340074fe0 0x7f234019f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:10.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.318+0000 7f2344888700 1 --2- 172.21.15.37:0/3695577613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2340113660 0x7f23401a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:10.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.318+0000 7f233ffff700 1 -- 172.21.15.37:0/3695577613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2340113660 msgr2=0x7f23401a3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:10.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.318+0000 7f233ffff700 1 --2- 172.21.15.37:0/3695577613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2340113660 0x7f23401a3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:10.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.318+0000 7f233ffff700 1 -- 172.21.15.37:0/3695577613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2340074fe0 msgr2=0x7f234019f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:10.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.318+0000 7f233ffff700 1 --2- 172.21.15.37:0/3695577613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2340074fe0 0x7f234019f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:10.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.319+0000 7f233ffff700 1 -- 172.21.15.37:0/3695577613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23401a43b0 con 0x7f23400746c0 2024-06-10T00:56:10.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.319+0000 7f2344888700 1 --2- 172.21.15.37:0/3695577613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2340113660 0x7f23401a3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:56:10.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.319+0000 7f233ffff700 1 --2- 172.21.15.37:0/3695577613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23400746c0 0x7f234019f3a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f233000b530 tx=0x7f233000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:10.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.319+0000 7f233d7fa700 1 -- 172.21.15.37:0/3695577613 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2330015020 con 0x7f23400746c0 2024-06-10T00:56:10.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.319+0000 7f23462eb700 1 -- 172.21.15.37:0/3695577613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23401a46a0 con 0x7f23400746c0 2024-06-10T00:56:10.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.319+0000 7f233d7fa700 1 -- 172.21.15.37:0/3695577613 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2330011d10 con 0x7f23400746c0 2024-06-10T00:56:10.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.319+0000 7f233d7fa700 1 -- 172.21.15.37:0/3695577613 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2330010e50 con 0x7f23400746c0 2024-06-10T00:56:10.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.319+0000 7f23462eb700 1 -- 172.21.15.37:0/3695577613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23401abf20 con 0x7f23400746c0 2024-06-10T00:56:10.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.320+0000 7f23462eb700 1 -- 172.21.15.37:0/3695577613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2340069180 con 0x7f23400746c0 2024-06-10T00:56:10.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.321+0000 7f233d7fa700 1 -- 172.21.15.37:0/3695577613 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2330018020 con 0x7f23400746c0 2024-06-10T00:56:10.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.322+0000 7f233d7fa700 1 --2- 172.21.15.37:0/3695577613 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f23280a2dc0 0x7f23280a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:10.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.322+0000 7f233f7fe700 1 --2- 172.21.15.37:0/3695577613 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f23280a2dc0 0x7f23280a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:10.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.322+0000 7f233d7fa700 1 -- 172.21.15.37:0/3695577613 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f23300cf1e0 con 0x7f23400746c0 2024-06-10T00:56:10.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.322+0000 7f233f7fe700 1 --2- 172.21.15.37:0/3695577613 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f23280a2dc0 0x7f23280a5280 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2334009750 tx=0x7f233400b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:10.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.325+0000 7f233d7fa700 1 -- 172.21.15.37:0/3695577613 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f233001f020 con 0x7f23400746c0 2024-06-10T00:56:10.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.577+0000 7f23462eb700 1 -- 172.21.15.37:0/3695577613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23401ac0f0 con 0x7f23400746c0 2024-06-10T00:56:10.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.578+0000 7f233d7fa700 1 -- 172.21.15.37:0/3695577613 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11867 (secure 0 0 0) 0x7f233009bd50 con 0x7f23400746c0 2024-06-10T00:56:10.578 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:56:10.579 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T00:53:44.125049+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24458,24472,24451]},"id":1}]} 2024-06-10T00:56:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.580+0000 7f23462eb700 1 -- 172.21.15.37:0/3695577613 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f23280a2dc0 msgr2=0x7f23280a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.580+0000 7f23462eb700 1 --2- 172.21.15.37:0/3695577613 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f23280a2dc0 0x7f23280a5280 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2334009750 tx=0x7f233400b540 comp rx=0 tx=0).stop 2024-06-10T00:56:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.580+0000 7f23462eb700 1 -- 172.21.15.37:0/3695577613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23400746c0 msgr2=0x7f234019f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.580+0000 7f23462eb700 1 --2- 172.21.15.37:0/3695577613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23400746c0 0x7f234019f3a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f233000b530 tx=0x7f233000b8f0 comp rx=0 tx=0).stop 2024-06-10T00:56:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.580+0000 7f23462eb700 1 -- 172.21.15.37:0/3695577613 shutdown_connections 2024-06-10T00:56:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.580+0000 7f23462eb700 1 --2- 172.21.15.37:0/3695577613 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f23280a2dc0 0x7f23280a5280 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.580+0000 7f23462eb700 1 --2- 172.21.15.37:0/3695577613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2340113660 0x7f23401a3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.580+0000 7f23462eb700 1 --2- 172.21.15.37:0/3695577613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23400746c0 0x7f234019f3a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.580+0000 7f23462eb700 1 --2- 172.21.15.37:0/3695577613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2340074fe0 0x7f234019f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.580+0000 7f23462eb700 1 -- 172.21.15.37:0/3695577613 >> 172.21.15.37:0/3695577613 conn(0x7f23400795f0 msgr2=0x7f234010f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:10.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.580+0000 7f23462eb700 1 -- 172.21.15.37:0/3695577613 shutdown_connections 2024-06-10T00:56:10.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:10.580+0000 7f23462eb700 1 -- 172.21.15.37:0/3695577613 wait complete. 2024-06-10T00:56:10.582 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-06-10T00:56:10.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:10 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3695577613' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:56:11.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:10 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3695577613' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:56:11.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:10 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3695577613' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:56:11.208 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:56:11.465 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:56:11.844 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:11 smithi037 ceph-mon[32192]: pgmap v415: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:56:12.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:11 smithi167 ceph-mon[36917]: pgmap v415: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:56:12.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:11 smithi094 ceph-mon[36708]: pgmap v415: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:56:13.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.136+0000 7ff82466b700 1 -- 172.21.15.37:0/1411452531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff81c1073f0 msgr2=0x7ff81c1077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:13.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.136+0000 7ff82466b700 1 --2- 172.21.15.37:0/1411452531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff81c1073f0 0x7ff81c1077d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff818009990 tx=0x7ff818009ca0 comp rx=0 tx=0).stop 2024-06-10T00:56:13.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.136+0000 7ff82466b700 1 -- 172.21.15.37:0/1411452531 shutdown_connections 2024-06-10T00:56:13.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.136+0000 7ff82466b700 1 --2- 172.21.15.37:0/1411452531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff81c101d40 0x7ff81c10e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:13.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.136+0000 7ff82466b700 1 --2- 172.21.15.37:0/1411452531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff81c1073f0 0x7ff81c1077d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:13.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.136+0000 7ff82466b700 1 --2- 172.21.15.37:0/1411452531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff81c101380 0x7ff81c101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:13.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.136+0000 7ff82466b700 1 -- 172.21.15.37:0/1411452531 >> 172.21.15.37:0/1411452531 conn(0x7ff81c078300 msgr2=0x7ff81c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:13.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.136+0000 7ff82466b700 1 -- 172.21.15.37:0/1411452531 shutdown_connections 2024-06-10T00:56:13.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.136+0000 7ff82466b700 1 -- 172.21.15.37:0/1411452531 wait complete. 2024-06-10T00:56:13.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.137+0000 7ff82466b700 1 Processor -- start 2024-06-10T00:56:13.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.137+0000 7ff82466b700 1 -- start start 2024-06-10T00:56:13.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.137+0000 7ff82466b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff81c101380 0x7ff81c19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:13.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.137+0000 7ff82466b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff81c101d40 0x7ff81c19ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:13.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.137+0000 7ff82466b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff81c1073f0 0x7ff81c198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:13.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.137+0000 7ff82466b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c111690 con 0x7ff81c1073f0 2024-06-10T00:56:13.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.137+0000 7ff82466b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c111510 con 0x7ff81c101380 2024-06-10T00:56:13.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.137+0000 7ff82466b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c111810 con 0x7ff81c101d40 2024-06-10T00:56:13.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.137+0000 7ff822c08700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff81c1073f0 0x7ff81c198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:13.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.137+0000 7ff822c08700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff81c1073f0 0x7ff81c198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57462/0 (socket says 172.21.15.37:57462) 2024-06-10T00:56:13.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.137+0000 7ff822c08700 1 -- 172.21.15.37:0/1918125744 learned_addr learned my addr 172.21.15.37:0/1918125744 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:13.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.137+0000 7ff822407700 1 --2- 172.21.15.37:0/1918125744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff81c101380 0x7ff81c19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:13.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.137+0000 7ff821c06700 1 --2- 172.21.15.37:0/1918125744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff81c101d40 0x7ff81c19ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:13.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.138+0000 7ff822c08700 1 -- 172.21.15.37:0/1918125744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff81c101d40 msgr2=0x7ff81c19ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:13.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.138+0000 7ff822c08700 1 --2- 172.21.15.37:0/1918125744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff81c101d40 0x7ff81c19ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:13.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.138+0000 7ff822c08700 1 -- 172.21.15.37:0/1918125744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff81c101380 msgr2=0x7ff81c19e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:13.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.138+0000 7ff822c08700 1 --2- 172.21.15.37:0/1918125744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff81c101380 0x7ff81c19e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:13.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.138+0000 7ff822c08700 1 -- 172.21.15.37:0/1918125744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81c1992a0 con 0x7ff81c1073f0 2024-06-10T00:56:13.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.138+0000 7ff821c06700 1 --2- 172.21.15.37:0/1918125744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff81c101d40 0x7ff81c19ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:56:13.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.138+0000 7ff822c08700 1 --2- 172.21.15.37:0/1918125744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff81c1073f0 0x7ff81c198b40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff814007ee0 tx=0x7ff81400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:13.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.138+0000 7ff8137fe700 1 -- 172.21.15.37:0/1918125744 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81400dbd0 con 0x7ff81c1073f0 2024-06-10T00:56:13.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.138+0000 7ff8137fe700 1 -- 172.21.15.37:0/1918125744 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff81400dd70 con 0x7ff81c1073f0 2024-06-10T00:56:13.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.138+0000 7ff82466b700 1 -- 172.21.15.37:0/1918125744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff81c199590 con 0x7ff81c1073f0 2024-06-10T00:56:13.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.138+0000 7ff8137fe700 1 -- 172.21.15.37:0/1918125744 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814005a60 con 0x7ff81c1073f0 2024-06-10T00:56:13.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.138+0000 7ff82466b700 1 -- 172.21.15.37:0/1918125744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff81c1a5800 con 0x7ff81c1073f0 2024-06-10T00:56:13.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.139+0000 7ff82466b700 1 -- 172.21.15.37:0/1918125744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff81c069180 con 0x7ff81c1073f0 2024-06-10T00:56:13.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.141+0000 7ff8137fe700 1 -- 172.21.15.37:0/1918125744 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff81401a020 con 0x7ff81c1073f0 2024-06-10T00:56:13.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.142+0000 7ff8137fe700 1 --2- 172.21.15.37:0/1918125744 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff8080a2ca0 0x7ff8080a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:13.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.143+0000 7ff822407700 1 --2- 172.21.15.37:0/1918125744 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff8080a2ca0 0x7ff8080a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:13.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.143+0000 7ff8137fe700 1 -- 172.21.15.37:0/1918125744 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7ff8140cf150 con 0x7ff81c1073f0 2024-06-10T00:56:13.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.143+0000 7ff822407700 1 --2- 172.21.15.37:0/1918125744 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff8080a2ca0 0x7ff8080a5160 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff81800b5c0 tx=0x7ff818005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:13.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.146+0000 7ff8137fe700 1 -- 172.21.15.37:0/1918125744 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff814010070 con 0x7ff81c1073f0 2024-06-10T00:56:13.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.404+0000 7ff82466b700 1 -- 172.21.15.37:0/1918125744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff81c103360 con 0x7ff81c1073f0 2024-06-10T00:56:13.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.405+0000 7ff8137fe700 1 -- 172.21.15.37:0/1918125744 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11867 (secure 0 0 0) 0x7ff81409bcc0 con 0x7ff81c1073f0 2024-06-10T00:56:13.405 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:56:13.405 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T00:53:44.125049+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24458,24472,24451]},"id":1}]} 2024-06-10T00:56:13.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.407+0000 7ff82466b700 1 -- 172.21.15.37:0/1918125744 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff8080a2ca0 msgr2=0x7ff8080a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:13.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.407+0000 7ff82466b700 1 --2- 172.21.15.37:0/1918125744 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff8080a2ca0 0x7ff8080a5160 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff81800b5c0 tx=0x7ff818005960 comp rx=0 tx=0).stop 2024-06-10T00:56:13.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.407+0000 7ff82466b700 1 -- 172.21.15.37:0/1918125744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff81c1073f0 msgr2=0x7ff81c198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:13.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.407+0000 7ff82466b700 1 --2- 172.21.15.37:0/1918125744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff81c1073f0 0x7ff81c198b40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff814007ee0 tx=0x7ff81400d3b0 comp rx=0 tx=0).stop 2024-06-10T00:56:13.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.407+0000 7ff82466b700 1 -- 172.21.15.37:0/1918125744 shutdown_connections 2024-06-10T00:56:13.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.407+0000 7ff82466b700 1 --2- 172.21.15.37:0/1918125744 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff8080a2ca0 0x7ff8080a5160 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:13.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.407+0000 7ff82466b700 1 --2- 172.21.15.37:0/1918125744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff81c1073f0 0x7ff81c198b40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:13.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.407+0000 7ff82466b700 1 --2- 172.21.15.37:0/1918125744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff81c101380 0x7ff81c19e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:13.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.407+0000 7ff82466b700 1 --2- 172.21.15.37:0/1918125744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff81c101d40 0x7ff81c19ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:13.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.407+0000 7ff82466b700 1 -- 172.21.15.37:0/1918125744 >> 172.21.15.37:0/1918125744 conn(0x7ff81c078300 msgr2=0x7ff81c1056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:13.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.407+0000 7ff82466b700 1 -- 172.21.15.37:0/1918125744 shutdown_connections 2024-06-10T00:56:13.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:13.407+0000 7ff82466b700 1 -- 172.21.15.37:0/1918125744 wait complete. 2024-06-10T00:56:13.409 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-06-10T00:56:14.060 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:13 smithi037 ceph-mon[32192]: pgmap v416: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:56:14.060 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:13 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1918125744' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:56:14.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:13 smithi167 ceph-mon[36917]: pgmap v416: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:56:14.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:13 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1918125744' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:56:14.091 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T00:56:14.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:13 smithi094 ceph-mon[36708]: pgmap v416: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:56:14.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:13 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1918125744' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:56:14.345 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:56:15.859 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:15 smithi037 ceph-mon[32192]: pgmap v417: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T00:56:16.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:15 smithi167 ceph-mon[36917]: pgmap v417: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T00:56:16.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:15 smithi094 ceph-mon[36708]: pgmap v417: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T00:56:16.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.123+0000 7efd77bd2700 1 -- 172.21.15.37:0/1624151208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd70107400 msgr2=0x7efd701077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:16.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.123+0000 7efd77bd2700 1 --2- 172.21.15.37:0/1624151208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd70107400 0x7efd701077e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7efd6c009990 tx=0x7efd6c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:56:16.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.123+0000 7efd77bd2700 1 -- 172.21.15.37:0/1624151208 shutdown_connections 2024-06-10T00:56:16.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.123+0000 7efd77bd2700 1 --2- 172.21.15.37:0/1624151208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd70101d50 0x7efd7010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:16.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.123+0000 7efd77bd2700 1 --2- 172.21.15.37:0/1624151208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd70107400 0x7efd701077e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:16.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.123+0000 7efd77bd2700 1 --2- 172.21.15.37:0/1624151208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd70101390 0x7efd70101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:16.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.123+0000 7efd77bd2700 1 -- 172.21.15.37:0/1624151208 >> 172.21.15.37:0/1624151208 conn(0x7efd70078340 msgr2=0x7efd70078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:16.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.123+0000 7efd77bd2700 1 -- 172.21.15.37:0/1624151208 shutdown_connections 2024-06-10T00:56:16.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.123+0000 7efd77bd2700 1 -- 172.21.15.37:0/1624151208 wait complete. 2024-06-10T00:56:16.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.124+0000 7efd77bd2700 1 Processor -- start 2024-06-10T00:56:16.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.124+0000 7efd77bd2700 1 -- start start 2024-06-10T00:56:16.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.124+0000 7efd77bd2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd70101390 0x7efd7019e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:16.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.124+0000 7efd77bd2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd70101d50 0x7efd7019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:16.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.124+0000 7efd77bd2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd70107400 0x7efd70198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:16.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.124+0000 7efd77bd2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efd70111670 con 0x7efd70101390 2024-06-10T00:56:16.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.124+0000 7efd77bd2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efd701114f0 con 0x7efd70107400 2024-06-10T00:56:16.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.124+0000 7efd77bd2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efd701117f0 con 0x7efd70101d50 2024-06-10T00:56:16.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.124+0000 7efd7596e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd70101390 0x7efd7019e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:16.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.124+0000 7efd7596e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd70101390 0x7efd7019e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57474/0 (socket says 172.21.15.37:57474) 2024-06-10T00:56:16.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.124+0000 7efd7596e700 1 -- 172.21.15.37:0/2129065207 learned_addr learned my addr 172.21.15.37:0/2129065207 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:16.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.124+0000 7efd7516d700 1 --2- 172.21.15.37:0/2129065207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd70101d50 0x7efd7019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:16.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.124+0000 7efd7616f700 1 --2- 172.21.15.37:0/2129065207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd70107400 0x7efd70198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:16.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.125+0000 7efd7596e700 1 -- 172.21.15.37:0/2129065207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd70101d50 msgr2=0x7efd7019ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:16.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.125+0000 7efd7596e700 1 --2- 172.21.15.37:0/2129065207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd70101d50 0x7efd7019ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:16.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.125+0000 7efd7596e700 1 -- 172.21.15.37:0/2129065207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd70107400 msgr2=0x7efd70198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:16.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.125+0000 7efd7596e700 1 --2- 172.21.15.37:0/2129065207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd70107400 0x7efd70198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:16.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.125+0000 7efd7596e700 1 -- 172.21.15.37:0/2129065207 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd701990c0 con 0x7efd70101390 2024-06-10T00:56:16.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.125+0000 7efd7596e700 1 --2- 172.21.15.37:0/2129065207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd70101390 0x7efd7019e7a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7efd6c000c00 tx=0x7efd6c003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:16.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.125+0000 7efd62ffd700 1 -- 172.21.15.37:0/2129065207 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6c027070 con 0x7efd70101390 2024-06-10T00:56:16.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.125+0000 7efd77bd2700 1 -- 172.21.15.37:0/2129065207 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd70199350 con 0x7efd70101390 2024-06-10T00:56:16.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.125+0000 7efd62ffd700 1 -- 172.21.15.37:0/2129065207 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd6c004190 con 0x7efd70101390 2024-06-10T00:56:16.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.125+0000 7efd62ffd700 1 -- 172.21.15.37:0/2129065207 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6c00f460 con 0x7efd70101390 2024-06-10T00:56:16.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.125+0000 7efd77bd2700 1 -- 172.21.15.37:0/2129065207 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd701a5670 con 0x7efd70101390 2024-06-10T00:56:16.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.126+0000 7efd77bd2700 1 -- 172.21.15.37:0/2129065207 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd70069180 con 0x7efd70101390 2024-06-10T00:56:16.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.127+0000 7efd62ffd700 1 -- 172.21.15.37:0/2129065207 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efd6c02a070 con 0x7efd70101390 2024-06-10T00:56:16.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.128+0000 7efd62ffd700 1 --2- 172.21.15.37:0/2129065207 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7efd5c0a2c20 0x7efd5c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:16.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.128+0000 7efd7516d700 1 --2- 172.21.15.37:0/2129065207 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7efd5c0a2c20 0x7efd5c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:16.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.128+0000 7efd62ffd700 1 -- 172.21.15.37:0/2129065207 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7efd6c0d0820 con 0x7efd70101390 2024-06-10T00:56:16.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.128+0000 7efd7516d700 1 --2- 172.21.15.37:0/2129065207 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7efd5c0a2c20 0x7efd5c0a50e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efd64005fd0 tx=0x7efd6400a560 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:16.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.131+0000 7efd62ffd700 1 -- 172.21.15.37:0/2129065207 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efd6c013070 con 0x7efd70101390 2024-06-10T00:56:16.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.338+0000 7efd77bd2700 1 -- 172.21.15.37:0/2129065207 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd701a5840 con 0x7efd70101390 2024-06-10T00:56:16.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.339+0000 7efd62ffd700 1 -- 172.21.15.37:0/2129065207 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24854 (secure 0 0 0) 0x7efd6c09d3c0 con 0x7efd70101390 2024-06-10T00:56:16.340 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:56:16.340 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":100,"fsid":"68793514-26c2-11ef-bc9b-c7b262605968","created":"2024-06-10T00:44:24.106713+0000","modified":"2024-06-10T00:54:22.546967+0000","last_up_change":"2024-06-10T00:51:31.562156+0000","last_in_change":"2024-06-10T00:51:11.294301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:47:44.773911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:52:31.684780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:52:34.885503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:52:44.415437+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1df6046a-6801-4950-9193-54303fed169b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6803","nonce":3594704094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6805","nonce":3594704094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6809","nonce":3594704094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6807","nonce":3594704094}]},"public_addr":"172.21.15.37:6803/3594704094","cluster_addr":"172.21.15.37:6805/3594704094","heartbeat_back_addr":"172.21.15.37:6809/3594704094","heartbeat_front_addr":"172.21.15.37:6807/3594704094","state":["exists","up"]},{"osd":1,"uuid":"4e1c8944-94e4-4e46-9efe-56229fe6df1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6801","nonce":3427727155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6803","nonce":3427727155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6807","nonce":3427727155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6805","nonce":3427727155}]},"public_addr":"172.21.15.94:6801/3427727155","cluster_addr":"172.21.15.94:6803/3427727155","heartbeat_back_addr":"172.21.15.94:6807/3427727155","heartbeat_front_addr":"172.21.15.94:6805/3427727155","state":["exists","up"]},{"osd":2,"uuid":"fd2fcedc-4c88-478f-8025-13e5944a27ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6801","nonce":4002498292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6803","nonce":4002498292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6807","nonce":4002498292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6805","nonce":4002498292}]},"public_addr":"172.21.15.167:6801/4002498292","cluster_addr":"172.21.15.167:6803/4002498292","heartbeat_back_addr":"172.21.15.167:6807/4002498292","heartbeat_front_addr":"172.21.15.167:6805/4002498292","state":["exists","up"]},{"osd":3,"uuid":"d0153676-818c-4dcc-83b1-287435a0c8a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6811","nonce":3425469324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6813","nonce":3425469324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6817","nonce":3425469324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6815","nonce":3425469324}]},"public_addr":"172.21.15.37:6811/3425469324","cluster_addr":"172.21.15.37:6813/3425469324","heartbeat_back_addr":"172.21.15.37:6817/3425469324","heartbeat_front_addr":"172.21.15.37:6815/3425469324","state":["exists","up"]},{"osd":4,"uuid":"172c333c-00c0-4e94-8870-acb3b32b2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6809","nonce":1740627431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6811","nonce":1740627431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6815","nonce":1740627431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6813","nonce":1740627431}]},"public_addr":"172.21.15.94:6809/1740627431","cluster_addr":"172.21.15.94:6811/1740627431","heartbeat_back_addr":"172.21.15.94:6815/1740627431","heartbeat_front_addr":"172.21.15.94:6813/1740627431","state":["exists","up"]},{"osd":5,"uuid":"dbce656e-5f93-4bda-baf2-fc3cc03d8282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6809","nonce":221805236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6811","nonce":221805236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6815","nonce":221805236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6813","nonce":221805236}]},"public_addr":"172.21.15.167:6809/221805236","cluster_addr":"172.21.15.167:6811/221805236","heartbeat_back_addr":"172.21.15.167:6815/221805236","heartbeat_front_addr":"172.21.15.167:6813/221805236","state":["exists","up"]},{"osd":6,"uuid":"fcbda266-6d60-43b2-a51c-4e562476d1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6819","nonce":472670875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6821","nonce":472670875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6825","nonce":472670875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6823","nonce":472670875}]},"public_addr":"172.21.15.37:6819/472670875","cluster_addr":"172.21.15.37:6821/472670875","heartbeat_back_addr":"172.21.15.37:6825/472670875","heartbeat_front_addr":"172.21.15.37:6823/472670875","state":["exists","up"]},{"osd":7,"uuid":"ed829ed3-ba85-4e82-ac37-e44e9534d37c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6817","nonce":3932281552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6819","nonce":3932281552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6823","nonce":3932281552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6821","nonce":3932281552}]},"public_addr":"172.21.15.94:6817/3932281552","cluster_addr":"172.21.15.94:6819/3932281552","heartbeat_back_addr":"172.21.15.94:6823/3932281552","heartbeat_front_addr":"172.21.15.94:6821/3932281552","state":["exists","up"]},{"osd":8,"uuid":"7d6267d8-72f4-4704-b554-bafc52d11a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6817","nonce":2290216047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6819","nonce":2290216047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6823","nonce":2290216047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6821","nonce":2290216047}]},"public_addr":"172.21.15.167:6817/2290216047","cluster_addr":"172.21.15.167:6819/2290216047","heartbeat_back_addr":"172.21.15.167:6823/2290216047","heartbeat_front_addr":"172.21.15.167:6821/2290216047","state":["exists","up"]},{"osd":9,"uuid":"210fdc20-0a36-4eeb-bdf2-c0a7f130e19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6827","nonce":2063666428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6829","nonce":2063666428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6833","nonce":2063666428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6831","nonce":2063666428}]},"public_addr":"172.21.15.37:6827/2063666428","cluster_addr":"172.21.15.37:6829/2063666428","heartbeat_back_addr":"172.21.15.37:6833/2063666428","heartbeat_front_addr":"172.21.15.37:6831/2063666428","state":["exists","up"]},{"osd":10,"uuid":"e200b371-b3cf-4b92-a659-f155ca86f0a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6825","nonce":1706618899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6827","nonce":1706618899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6831","nonce":1706618899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6829","nonce":1706618899}]},"public_addr":"172.21.15.94:6825/1706618899","cluster_addr":"172.21.15.94:6827/1706618899","heartbeat_back_addr":"172.21.15.94:6831/1706618899","heartbeat_front_addr":"172.21.15.94:6829/1706618899","state":["exists","up"]},{"osd":11,"uuid":"cfa427df-37a5-4d28-ac2e-af777d607008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6825","nonce":1349892516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6827","nonce":1349892516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6831","nonce":1349892516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6829","nonce":1349892516}]},"public_addr":"172.21.15.167:6825/1349892516","cluster_addr":"172.21.15.167:6827/1349892516","heartbeat_back_addr":"172.21.15.167:6831/1349892516","heartbeat_front_addr":"172.21.15.167:6829/1349892516","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:46:51.846147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:15.763723+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:40.106800+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:05.923204+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:30.501883+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:55.565575+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:20.411360+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:46.071776+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:11.854599+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:37.608818+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:03.176767+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:29.768705+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/4084722397":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/3571803585":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/416587443":"2024-06-11T00:45:20.254342+0000","172.21.15.37:6801/2766692263":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/2026344134":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/2781321886":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6801/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/1110143997":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2766692263":"2024-06-11T00:45:20.254342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:56:16.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.342+0000 7efd77bd2700 1 -- 172.21.15.37:0/2129065207 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7efd5c0a2c20 msgr2=0x7efd5c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:16.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.342+0000 7efd77bd2700 1 --2- 172.21.15.37:0/2129065207 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7efd5c0a2c20 0x7efd5c0a50e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efd64005fd0 tx=0x7efd6400a560 comp rx=0 tx=0).stop 2024-06-10T00:56:16.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.342+0000 7efd77bd2700 1 -- 172.21.15.37:0/2129065207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd70101390 msgr2=0x7efd7019e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:16.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.342+0000 7efd77bd2700 1 --2- 172.21.15.37:0/2129065207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd70101390 0x7efd7019e7a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7efd6c000c00 tx=0x7efd6c003730 comp rx=0 tx=0).stop 2024-06-10T00:56:16.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.342+0000 7efd77bd2700 1 -- 172.21.15.37:0/2129065207 shutdown_connections 2024-06-10T00:56:16.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.342+0000 7efd77bd2700 1 --2- 172.21.15.37:0/2129065207 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7efd5c0a2c20 0x7efd5c0a50e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:16.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.342+0000 7efd77bd2700 1 --2- 172.21.15.37:0/2129065207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd70107400 0x7efd70198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:16.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.342+0000 7efd77bd2700 1 --2- 172.21.15.37:0/2129065207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd70101390 0x7efd7019e7a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:16.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.342+0000 7efd77bd2700 1 --2- 172.21.15.37:0/2129065207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd70101d50 0x7efd7019ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:16.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.342+0000 7efd77bd2700 1 -- 172.21.15.37:0/2129065207 >> 172.21.15.37:0/2129065207 conn(0x7efd70078340 msgr2=0x7efd701005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:16.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.342+0000 7efd77bd2700 1 -- 172.21.15.37:0/2129065207 shutdown_connections 2024-06-10T00:56:16.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:16.342+0000 7efd77bd2700 1 -- 172.21.15.37:0/2129065207 wait complete. 2024-06-10T00:56:16.711 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:16 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2129065207' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:56:17.025 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-10T00:52:37.869277+0000', 'modified': '2024-06-10T00:53:44.125049+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24472, 'mds_1': 24458, 'mds_2': 24451}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24413': {'gid': 24413, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.94:6835/570288048', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6834', 'nonce': 570288048}, {'type': 'v1', 'addr': '172.21.15.94:6835', 'nonce': 570288048}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.94:6837/3801637791', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6836', 'nonce': 3801637791}, {'type': 'v1', 'addr': '172.21.15.94:6837', 'nonce': 3801637791}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6835/3066042611', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 3066042611}, {'type': 'v1', 'addr': '172.21.15.167:6835', 'nonce': 3066042611}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.167:6837/2720148065', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 2720148065}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 2720148065}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.94:6839/946269331', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6838', 'nonce': 946269331}, {'type': 'v1', 'addr': '172.21.15.94:6839', 'nonce': 946269331}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.167:6839/3945638144', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 3945638144}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 3945638144}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24472, 'qdb_cluster': [24458, 24472, 24451]} 2024-06-10T00:56:17.025 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-06-10T00:56:17.026 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.c status 2024-06-10T00:56:17.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:16 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2129065207' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:56:17.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:16 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2129065207' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:56:17.280 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:56:18.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:17 smithi167 ceph-mon[36917]: pgmap v418: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:56:18.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:17 smithi094 ceph-mon[36708]: pgmap v418: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:56:18.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:17 smithi037 ceph-mon[32192]: pgmap v418: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:56:19.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.039+0000 7f7e8ff79700 1 -- 172.21.15.37:0/2840450231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e88103150 msgr2=0x7f7e88107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:19.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.039+0000 7f7e8ff79700 1 --2- 172.21.15.37:0/2840450231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e88103150 0x7f7e88107140 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7e780099c0 tx=0x7f7e78009cd0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.039+0000 7f7e8ff79700 1 -- 172.21.15.37:0/2840450231 shutdown_connections 2024-06-10T00:56:19.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.039+0000 7f7e8ff79700 1 --2- 172.21.15.37:0/2840450231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e88107870 0x7f7e8810f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.039+0000 7f7e8ff79700 1 --2- 172.21.15.37:0/2840450231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e881027a0 0x7f7e88102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.039+0000 7f7e8ff79700 1 --2- 172.21.15.37:0/2840450231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e88103150 0x7f7e88107140 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.039+0000 7f7e8ff79700 1 -- 172.21.15.37:0/2840450231 >> 172.21.15.37:0/2840450231 conn(0x7f7e88079520 msgr2=0x7f7e88079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:19.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.039+0000 7f7e8ff79700 1 -- 172.21.15.37:0/2840450231 shutdown_connections 2024-06-10T00:56:19.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.039+0000 7f7e8ff79700 1 -- 172.21.15.37:0/2840450231 wait complete. 2024-06-10T00:56:19.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.040+0000 7f7e8ff79700 1 Processor -- start 2024-06-10T00:56:19.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.040+0000 7f7e8ff79700 1 -- start start 2024-06-10T00:56:19.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.040+0000 7f7e8ff79700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e881027a0 0x7f7e8819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:19.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.040+0000 7f7e8ff79700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e88103150 0x7f7e8819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:19.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.041+0000 7f7e8ff79700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e88107870 0x7f7e88198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:19.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.041+0000 7f7e8ff79700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7e88112880 con 0x7f7e88107870 2024-06-10T00:56:19.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.041+0000 7f7e8ff79700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7e88112700 con 0x7f7e88103150 2024-06-10T00:56:19.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.041+0000 7f7e8ff79700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7e88112a00 con 0x7f7e881027a0 2024-06-10T00:56:19.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.041+0000 7f7e8e516700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e88107870 0x7f7e88198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:19.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.041+0000 7f7e8e516700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e88107870 0x7f7e88198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57492/0 (socket says 172.21.15.37:57492) 2024-06-10T00:56:19.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.041+0000 7f7e8e516700 1 -- 172.21.15.37:0/3521082326 learned_addr learned my addr 172.21.15.37:0/3521082326 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:19.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.041+0000 7f7e8dd15700 1 --2- 172.21.15.37:0/3521082326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e881027a0 0x7f7e8819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:19.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.041+0000 7f7e8d514700 1 --2- 172.21.15.37:0/3521082326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e88103150 0x7f7e8819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:19.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.041+0000 7f7e8e516700 1 -- 172.21.15.37:0/3521082326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e881027a0 msgr2=0x7f7e8819e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:19.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.041+0000 7f7e8e516700 1 --2- 172.21.15.37:0/3521082326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e881027a0 0x7f7e8819e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.041+0000 7f7e8e516700 1 -- 172.21.15.37:0/3521082326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e88103150 msgr2=0x7f7e8819ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:19.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.041+0000 7f7e8e516700 1 --2- 172.21.15.37:0/3521082326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e88103150 0x7f7e8819ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.041+0000 7f7e8e516700 1 -- 172.21.15.37:0/3521082326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e88199260 con 0x7f7e88107870 2024-06-10T00:56:19.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.041+0000 7f7e8e516700 1 --2- 172.21.15.37:0/3521082326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e88107870 0x7f7e88198b00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000b970 tx=0x7f7e8000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:19.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.041+0000 7f7e7effd700 1 -- 172.21.15.37:0/3521082326 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8000d860 con 0x7f7e88107870 2024-06-10T00:56:19.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.042+0000 7f7e7effd700 1 -- 172.21.15.37:0/3521082326 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e8000e430 con 0x7f7e88107870 2024-06-10T00:56:19.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.042+0000 7f7e8ff79700 1 -- 172.21.15.37:0/3521082326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e88199550 con 0x7f7e88107870 2024-06-10T00:56:19.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.042+0000 7f7e7effd700 1 -- 172.21.15.37:0/3521082326 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80017660 con 0x7f7e88107870 2024-06-10T00:56:19.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.042+0000 7f7e8ff79700 1 -- 172.21.15.37:0/3521082326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e881a5810 con 0x7f7e88107870 2024-06-10T00:56:19.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.044+0000 7f7e7effd700 1 -- 172.21.15.37:0/3521082326 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7e8000e5e0 con 0x7f7e88107870 2024-06-10T00:56:19.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.045+0000 7f7e7effd700 1 --2- 172.21.15.37:0/3521082326 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7e740a2c20 0x7f7e740a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:19.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.045+0000 7f7e8dd15700 1 --2- 172.21.15.37:0/3521082326 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7e740a2c20 0x7f7e740a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:19.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.045+0000 7f7e7effd700 1 -- 172.21.15.37:0/3521082326 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f7e800cf050 con 0x7f7e88107870 2024-06-10T00:56:19.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.045+0000 7f7e8dd15700 1 --2- 172.21.15.37:0/3521082326 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7e740a2c20 0x7f7e740a50e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009a10 tx=0x7f7e84009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:19.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.047+0000 7f7e9a7fcb80 1 Processor -- start 2024-06-10T00:56:19.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.047+0000 7f7e9a7fcb80 1 -- start start 2024-06-10T00:56:19.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.047+0000 7f7e9a7fcb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae9b5a85f0 0x55ae9b5a89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:19.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.048+0000 7f7e9a7fcb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae9b5a8f10 0x55ae9b5a2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:19.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.048+0000 7f7e9a7fcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae9b5a2ed0 0x55ae9b5a32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:19.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.048+0000 7f7e9a7fcb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ae9b4f47c0 con 0x55ae9b5a85f0 2024-06-10T00:56:19.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.048+0000 7f7e9a7fcb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ae9b49c6d0 con 0x55ae9b5a8f10 2024-06-10T00:56:19.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.048+0000 7f7e9a7fcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ae9b4f0780 con 0x55ae9b5a2ed0 2024-06-10T00:56:19.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.048+0000 7f7e8dd15700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae9b5a85f0 0x55ae9b5a89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:19.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.048+0000 7f7e8dd15700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae9b5a85f0 0x55ae9b5a89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57494/0 (socket says 172.21.15.37:57494) 2024-06-10T00:56:19.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.048+0000 7f7e8dd15700 1 -- 172.21.15.37:0/3661784082 learned_addr learned my addr 172.21.15.37:0/3661784082 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:19.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.048+0000 7f7e8d514700 1 --2- 172.21.15.37:0/3661784082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae9b5a8f10 0x55ae9b5a2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:19.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.048+0000 7f7e8e516700 1 --2- 172.21.15.37:0/3661784082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae9b5a2ed0 0x55ae9b5a32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:19.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.048+0000 7f7e8dd15700 1 -- 172.21.15.37:0/3661784082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae9b5a2ed0 msgr2=0x55ae9b5a32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:19.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.048+0000 7f7e8dd15700 1 --2- 172.21.15.37:0/3661784082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae9b5a2ed0 0x55ae9b5a32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.048+0000 7f7e8dd15700 1 -- 172.21.15.37:0/3661784082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae9b5a8f10 msgr2=0x55ae9b5a2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:19.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.048+0000 7f7e8dd15700 1 --2- 172.21.15.37:0/3661784082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae9b5a8f10 0x55ae9b5a2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.048+0000 7f7e8dd15700 1 -- 172.21.15.37:0/3661784082 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae9b492f00 con 0x55ae9b5a85f0 2024-06-10T00:56:19.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.048+0000 7f7e8dd15700 1 --2- 172.21.15.37:0/3661784082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae9b5a85f0 0x55ae9b5a89d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400ea00 tx=0x7f7e84011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6975e5add22be0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:19.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.049+0000 7f7e73fff700 1 -- 172.21.15.37:0/3661784082 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8401b040 con 0x55ae9b5a85f0 2024-06-10T00:56:19.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.049+0000 7f7e73fff700 1 -- 172.21.15.37:0/3661784082 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e84024070 con 0x55ae9b5a85f0 2024-06-10T00:56:19.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.049+0000 7f7e73fff700 1 -- 172.21.15.37:0/3661784082 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84013740 con 0x55ae9b5a85f0 2024-06-10T00:56:19.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.049+0000 7f7e9a7fcb80 1 -- 172.21.15.37:0/3661784082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae9b5a85f0 msgr2=0x55ae9b5a89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:19.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.049+0000 7f7e9a7fcb80 1 --2- 172.21.15.37:0/3661784082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae9b5a85f0 0x55ae9b5a89d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400ea00 tx=0x7f7e84011000 comp rx=0 tx=0).stop 2024-06-10T00:56:19.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.049+0000 7f7e9a7fcb80 1 -- 172.21.15.37:0/3661784082 shutdown_connections 2024-06-10T00:56:19.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.049+0000 7f7e9a7fcb80 1 --2- 172.21.15.37:0/3661784082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae9b5a2ed0 0x55ae9b5a32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.049+0000 7f7e9a7fcb80 1 --2- 172.21.15.37:0/3661784082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae9b5a85f0 0x55ae9b5a89d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.049+0000 7f7e9a7fcb80 1 --2- 172.21.15.37:0/3661784082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae9b5a8f10 0x55ae9b5a2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.049+0000 7f7e9a7fcb80 1 -- 172.21.15.37:0/3661784082 >> 172.21.15.37:0/3661784082 conn(0x55ae9b4cdc70 msgr2=0x55ae9b59d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:19.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.050+0000 7f7e9a7fcb80 1 -- 172.21.15.37:0/3661784082 shutdown_connections 2024-06-10T00:56:19.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.050+0000 7f7e9a7fcb80 1 -- 172.21.15.37:0/3661784082 wait complete. 2024-06-10T00:56:19.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.050+0000 7f7e9a7fcb80 1 -- start start 2024-06-10T00:56:19.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.050+0000 7f7e9a7fcb80 1 Processor -- start 2024-06-10T00:56:19.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.050+0000 7f7e9a7fcb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae9b5a2ed0 0x55ae9b5aec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:19.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.050+0000 7f7e9a7fcb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae9b5a85f0 0x55ae9b5abf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:19.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.050+0000 7f7e9a7fcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae9b5a8f10 0x55ae9b5ac4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:19.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.050+0000 7f7e9a7fcb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ae9b5af980 con 0x55ae9b5a2ed0 2024-06-10T00:56:19.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.050+0000 7f7e9a7fcb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ae9b494ba0 con 0x55ae9b5a85f0 2024-06-10T00:56:19.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.050+0000 7f7e9a7fcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ae9b5afb00 con 0x55ae9b5a8f10 2024-06-10T00:56:19.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.051+0000 7f7e9a7fcb80 10 client.0 _ll_register_callbacks cb 0x55ae9b48e8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:56:19.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.051+0000 7f7e8dd15700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae9b5a2ed0 0x55ae9b5aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:19.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.051+0000 7f7e8dd15700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae9b5a2ed0 0x55ae9b5aec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57510/0 (socket says 172.21.15.37:57510) 2024-06-10T00:56:19.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.051+0000 7f7e8dd15700 1 -- 172.21.15.37:0/816168112 learned_addr learned my addr 172.21.15.37:0/816168112 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:19.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.051+0000 7f7e8e516700 1 --2- 172.21.15.37:0/816168112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae9b5a8f10 0x55ae9b5ac4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:19.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.051+0000 7f7e8d514700 1 --2- 172.21.15.37:0/816168112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae9b5a85f0 0x55ae9b5abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:19.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.051+0000 7f7e8dd15700 1 -- 172.21.15.37:0/816168112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae9b5a8f10 msgr2=0x55ae9b5ac4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:19.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.051+0000 7f7e8dd15700 1 --2- 172.21.15.37:0/816168112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae9b5a8f10 0x55ae9b5ac4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.051+0000 7f7e8dd15700 1 -- 172.21.15.37:0/816168112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae9b5a85f0 msgr2=0x55ae9b5abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:19.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.051+0000 7f7e8dd15700 1 --2- 172.21.15.37:0/816168112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae9b5a85f0 0x55ae9b5abf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.051+0000 7f7e8dd15700 1 -- 172.21.15.37:0/816168112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae9b443880 con 0x55ae9b5a2ed0 2024-06-10T00:56:19.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.051+0000 7f7e8dd15700 1 --2- 172.21.15.37:0/816168112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae9b5a2ed0 0x55ae9b5aec60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7e84017010 tx=0x7f7e840159a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8908c9b31beafe6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:19.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.051+0000 7f7e717fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:56:19.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.051+0000 7f7e717fa700 1 -- 172.21.15.37:0/816168112 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84013860 con 0x55ae9b5a2ed0 2024-06-10T00:56:19.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.051+0000 7f7e717fa700 1 -- 172.21.15.37:0/816168112 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f7e84013a00 con 0x55ae9b5a2ed0 2024-06-10T00:56:19.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.051+0000 7f7e9a7fcb80 1 -- 172.21.15.37:0/816168112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e84024070 con 0x55ae9b5a2ed0 2024-06-10T00:56:19.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.052+0000 7f7e717fa700 1 -- 172.21.15.37:0/816168112 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7e84009710 con 0x55ae9b5a2ed0 2024-06-10T00:56:19.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.052+0000 7f7e9a7fcb80 10 client.15165 fetch_fsmap learned FSMap version 26 2024-06-10T00:56:19.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.052+0000 7f7e9a7fcb80 1 -- 172.21.15.37:0/816168112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ae9b5ba290 con 0x55ae9b5a2ed0 2024-06-10T00:56:19.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.052+0000 7f7e717fa700 1 -- 172.21.15.37:0/816168112 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e84024070 con 0x55ae9b5a2ed0 2024-06-10T00:56:19.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.052+0000 7f7e717fa700 1 -- 172.21.15.37:0/816168112 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f7e8401e030 con 0x55ae9b5a2ed0 2024-06-10T00:56:19.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.052+0000 7f7e9a7fcb80 10 client.15165 fetch_fsmap finished waiting for FSMap version 26 2024-06-10T00:56:19.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.052+0000 7f7e9a7fcb80 10 client.15165 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-10T00:56:19.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.052+0000 7f7e9a7fcb80 20 client.15165 populate_metadata read hostname 'smithi037' 2024-06-10T00:56:19.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.052+0000 7f7e9a7fcb80 1 --2- 172.21.15.37:0/816168112 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55ae9b5bc4e0 0x55ae9b5bd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:19.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.052+0000 7f7e9a7fcb80 4 client.15165 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:56:19.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.052+0000 7f7e9a7fcb80 1 -- 172.21.15.37:0/816168112 --> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae9b501cc0 con 0x55ae9b5bc4e0 2024-06-10T00:56:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.052+0000 7f7e8d514700 1 --2- 172.21.15.37:0/816168112 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55ae9b5bc4e0 0x55ae9b5bd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.053+0000 7f7e8d514700 1 --2- 172.21.15.37:0/816168112 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55ae9b5bc4e0 0x55ae9b5bd950 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f15d793c9a2c6f9 server_cookie=cce25bc58e67fa3d in_seq=0 out_seq=0 2024-06-10T00:56:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.053+0000 7f7e717fa700 10 client.15165 ms_handle_connect on v2:172.21.15.167:6836/2720148065 2024-06-10T00:56:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.054+0000 7f7e717fa700 1 -- 172.21.15.37:0/816168112 <== mds.2 v2:172.21.15.167:6836/2720148065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ae9b501cc0 con 0x55ae9b5bc4e0 2024-06-10T00:56:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.054+0000 7f7e717fa700 10 client.15165 handle_command_reply: tid=0 2024-06-10T00:56:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.054+0000 7f7e9a7fcb80 1 client.15165 shutdown 2024-06-10T00:56:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.054+0000 7f7e9a7fcb80 1 -- 172.21.15.37:0/816168112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae9b5a2ed0 msgr2=0x55ae9b5aec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.054+0000 7f7e9a7fcb80 1 --2- 172.21.15.37:0/816168112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae9b5a2ed0 0x55ae9b5aec60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7e84017010 tx=0x7f7e840159a0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.054+0000 7f7e9a7fcb80 1 -- 172.21.15.37:0/816168112 shutdown_connections 2024-06-10T00:56:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.054+0000 7f7e9a7fcb80 1 --2- 172.21.15.37:0/816168112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae9b5a8f10 0x55ae9b5ac4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.054+0000 7f7e9a7fcb80 1 --2- 172.21.15.37:0/816168112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae9b5a2ed0 0x55ae9b5aec60 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.054+0000 7f7e9a7fcb80 1 --2- 172.21.15.37:0/816168112 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55ae9b5bc4e0 0x55ae9b5bd950 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.054+0000 7f7e9a7fcb80 1 --2- 172.21.15.37:0/816168112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae9b5a85f0 0x55ae9b5abf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.054+0000 7f7e717fa700 0 client.15165 ms_handle_reset on v2:172.21.15.167:6836/2720148065 2024-06-10T00:56:19.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.054+0000 7f7e9a7fcb80 1 -- 172.21.15.37:0/816168112 >> 172.21.15.37:0/816168112 conn(0x55ae9b4cdc70 msgr2=0x55ae9b5a1980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:19.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.054+0000 7f7e9a7fcb80 1 -- 172.21.15.37:0/816168112 shutdown_connections 2024-06-10T00:56:19.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.054+0000 7f7e9a7fcb80 1 -- 172.21.15.37:0/816168112 wait complete. 2024-06-10T00:56:19.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.054+0000 7f7e9a7fcb80 20 client.15165 trim_cache size 0 max 16384 2024-06-10T00:56:19.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.076+0000 7f7e9a7fcb80 1 Processor -- start 2024-06-10T00:56:19.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.076+0000 7f7e9a7fcb80 1 -- start start 2024-06-10T00:56:19.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.076+0000 7f7e9a7fcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae9b5a2ed0 0x55ae9b5a2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:19.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.076+0000 7f7e9a7fcb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae9b5a85f0 0x55ae9b5ac4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:19.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.076+0000 7f7e9a7fcb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae9b5a8f10 0x55ae9b5c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:19.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.076+0000 7f7e9a7fcb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ae9b5bbc90 con 0x55ae9b5a85f0 2024-06-10T00:56:19.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.076+0000 7f7e9a7fcb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ae9b5bc360 con 0x55ae9b5a8f10 2024-06-10T00:56:19.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.076+0000 7f7e9a7fcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ae9b5bca30 con 0x55ae9b5a2ed0 2024-06-10T00:56:19.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.076+0000 7f7e8d514700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae9b5a85f0 0x55ae9b5ac4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:19.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.076+0000 7f7e8d514700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae9b5a85f0 0x55ae9b5ac4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57516/0 (socket says 172.21.15.37:57516) 2024-06-10T00:56:19.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.076+0000 7f7e8d514700 1 -- 172.21.15.37:0/3124686114 learned_addr learned my addr 172.21.15.37:0/3124686114 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:19.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.076+0000 7f7e8dd15700 1 --2- 172.21.15.37:0/3124686114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae9b5a2ed0 0x55ae9b5a2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:19.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.077+0000 7f7e8e516700 1 --2- 172.21.15.37:0/3124686114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae9b5a8f10 0x55ae9b5c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:19.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.077+0000 7f7e8d514700 1 -- 172.21.15.37:0/3124686114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae9b5a2ed0 msgr2=0x55ae9b5a2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:19.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.077+0000 7f7e8d514700 1 --2- 172.21.15.37:0/3124686114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae9b5a2ed0 0x55ae9b5a2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.077+0000 7f7e8d514700 1 -- 172.21.15.37:0/3124686114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae9b5a8f10 msgr2=0x55ae9b5c71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:19.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.077+0000 7f7e8d514700 1 --2- 172.21.15.37:0/3124686114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae9b5a8f10 0x55ae9b5c71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.077+0000 7f7e8d514700 1 -- 172.21.15.37:0/3124686114 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae9b5aa4d0 con 0x55ae9b5a85f0 2024-06-10T00:56:19.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.077+0000 7f7e8e516700 1 --2- 172.21.15.37:0/3124686114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae9b5a8f10 0x55ae9b5c71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:56:19.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.077+0000 7f7e8d514700 1 --2- 172.21.15.37:0/3124686114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae9b5a85f0 0x55ae9b5ac4d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009750 tx=0x7f7e7801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f460ef9fbea7f10f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:19.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.077+0000 7f7e737fe700 1 -- 172.21.15.37:0/3124686114 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7800bc60 con 0x55ae9b5a85f0 2024-06-10T00:56:19.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.077+0000 7f7e737fe700 1 -- 172.21.15.37:0/3124686114 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e78003e40 con 0x55ae9b5a85f0 2024-06-10T00:56:19.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.077+0000 7f7e737fe700 1 -- 172.21.15.37:0/3124686114 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7800f6c0 con 0x55ae9b5a85f0 2024-06-10T00:56:19.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.078+0000 7f7e9a7fcb80 1 -- 172.21.15.37:0/3124686114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae9b5a85f0 msgr2=0x55ae9b5ac4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:19.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.078+0000 7f7e9a7fcb80 1 --2- 172.21.15.37:0/3124686114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae9b5a85f0 0x55ae9b5ac4d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009750 tx=0x7f7e7801a040 comp rx=0 tx=0).stop 2024-06-10T00:56:19.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.078+0000 7f7e9a7fcb80 1 -- 172.21.15.37:0/3124686114 shutdown_connections 2024-06-10T00:56:19.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.078+0000 7f7e9a7fcb80 1 --2- 172.21.15.37:0/3124686114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae9b5a8f10 0x55ae9b5c71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.078+0000 7f7e9a7fcb80 1 --2- 172.21.15.37:0/3124686114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae9b5a2ed0 0x55ae9b5a2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.078+0000 7f7e9a7fcb80 1 --2- 172.21.15.37:0/3124686114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae9b5a85f0 0x55ae9b5ac4d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.078+0000 7f7e9a7fcb80 1 -- 172.21.15.37:0/3124686114 >> 172.21.15.37:0/3124686114 conn(0x55ae9b4cdc70 msgr2=0x55ae9b5aa850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:19.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.078+0000 7f7e9a7fcb80 1 -- 172.21.15.37:0/3124686114 shutdown_connections 2024-06-10T00:56:19.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.078+0000 7f7e9a7fcb80 1 -- 172.21.15.37:0/3124686114 wait complete. 2024-06-10T00:56:19.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.078+0000 7f7e9a7fcb80 1 -- start start 2024-06-10T00:56:19.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.078+0000 7f7e9a7fcb80 1 Processor -- start 2024-06-10T00:56:19.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.079+0000 7f7e9a7fcb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae9b5a2ed0 0x55ae9b5ab4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:19.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.079+0000 7f7e9a7fcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae9b5a85f0 0x55ae9b5aba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:19.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.079+0000 7f7e9a7fcb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae9b5a8f10 0x55ae9b5abf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:19.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.079+0000 7f7e9a7fcb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ae9b5bd7d0 con 0x55ae9b5a2ed0 2024-06-10T00:56:19.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.079+0000 7f7e9a7fcb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ae9b5bd100 con 0x55ae9b5a8f10 2024-06-10T00:56:19.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.079+0000 7f7e9a7fcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ae9b5bdea0 con 0x55ae9b5a85f0 2024-06-10T00:56:19.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.079+0000 7f7e9a7fcb80 10 client.0 _ll_register_callbacks cb 0x55ae9b48e8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:56:19.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.079+0000 7f7e8dd15700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae9b5a2ed0 0x55ae9b5ab4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:19.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.079+0000 7f7e8dd15700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae9b5a2ed0 0x55ae9b5ab4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57530/0 (socket says 172.21.15.37:57530) 2024-06-10T00:56:19.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.079+0000 7f7e8dd15700 1 -- 172.21.15.37:0/1322115022 learned_addr learned my addr 172.21.15.37:0/1322115022 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:19.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.079+0000 7f7e8d514700 1 --2- 172.21.15.37:0/1322115022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae9b5a85f0 0x55ae9b5aba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:19.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.079+0000 7f7e8e516700 1 --2- 172.21.15.37:0/1322115022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae9b5a8f10 0x55ae9b5abf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:19.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.079+0000 7f7e8dd15700 1 -- 172.21.15.37:0/1322115022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae9b5a85f0 msgr2=0x55ae9b5aba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:19.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.079+0000 7f7e8dd15700 1 --2- 172.21.15.37:0/1322115022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae9b5a85f0 0x55ae9b5aba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.079+0000 7f7e8dd15700 1 -- 172.21.15.37:0/1322115022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae9b5a8f10 msgr2=0x55ae9b5abf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:19.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.079+0000 7f7e8dd15700 1 --2- 172.21.15.37:0/1322115022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae9b5a8f10 0x55ae9b5abf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.079+0000 7f7e8dd15700 1 -- 172.21.15.37:0/1322115022 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae9b62ae20 con 0x55ae9b5a2ed0 2024-06-10T00:56:19.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.079+0000 7f7e8e516700 1 --2- 172.21.15.37:0/1322115022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae9b5a8f10 0x55ae9b5abf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:56:19.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.079+0000 7f7e8dd15700 1 --2- 172.21.15.37:0/1322115022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae9b5a2ed0 0x55ae9b5ab4c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7e840098b0 tx=0x7f7e84015cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4fe7d85cc04195 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:19.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.079+0000 7f7e717fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:56:19.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.079+0000 7f7e717fa700 1 -- 172.21.15.37:0/1322115022 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84028800 con 0x55ae9b5a2ed0 2024-06-10T00:56:19.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.079+0000 7f7e717fa700 1 -- 172.21.15.37:0/1322115022 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f7e84013660 con 0x55ae9b5a2ed0 2024-06-10T00:56:19.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.079+0000 7f7e9a7fcb80 1 -- 172.21.15.37:0/1322115022 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ae9b4fc240 con 0x55ae9b5a2ed0 2024-06-10T00:56:19.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.080+0000 7f7e717fa700 1 -- 172.21.15.37:0/1322115022 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7e84013860 con 0x55ae9b5a2ed0 2024-06-10T00:56:19.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.080+0000 7f7e9a7fcb80 10 client.15171 fetch_fsmap learned FSMap version 26 2024-06-10T00:56:19.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.080+0000 7f7e9a7fcb80 1 -- 172.21.15.37:0/1322115022 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ae9b62c070 con 0x55ae9b5a2ed0 2024-06-10T00:56:19.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.080+0000 7f7e717fa700 1 -- 172.21.15.37:0/1322115022 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ae9b4fc240 con 0x55ae9b5a2ed0 2024-06-10T00:56:19.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.080+0000 7f7e717fa700 1 -- 172.21.15.37:0/1322115022 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f7e8401e040 con 0x55ae9b5a2ed0 2024-06-10T00:56:19.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.080+0000 7f7e9a7fcb80 10 client.15171 fetch_fsmap finished waiting for FSMap version 26 2024-06-10T00:56:19.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.080+0000 7f7e9a7fcb80 10 client.15171 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-10T00:56:19.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.080+0000 7f7e9a7fcb80 20 client.15171 populate_metadata read hostname 'smithi037' 2024-06-10T00:56:19.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.080+0000 7f7e9a7fcb80 1 --2- 172.21.15.37:0/1322115022 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55ae9b630a60 0x55ae9b62f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:19.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.080+0000 7f7e9a7fcb80 4 client.15171 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T00:56:19.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.080+0000 7f7e9a7fcb80 1 -- 172.21.15.37:0/1322115022 --> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] -- command(tid 0: {"prefix": "status"}) -- 0x55ae9b376200 con 0x55ae9b630a60 2024-06-10T00:56:19.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.080+0000 7f7e8d514700 1 --2- 172.21.15.37:0/1322115022 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55ae9b630a60 0x55ae9b62f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:19.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.081+0000 7f7e8d514700 1 --2- 172.21.15.37:0/1322115022 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55ae9b630a60 0x55ae9b62f770 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7e8c81367abe074e server_cookie=ea1d21c314b64da5 in_seq=0 out_seq=0 2024-06-10T00:56:19.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.081+0000 7f7e717fa700 10 client.15171 ms_handle_connect on v2:172.21.15.167:6836/2720148065 2024-06-10T00:56:19.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.081+0000 7f7e717fa700 1 -- 172.21.15.37:0/1322115022 <== mds.2 v2:172.21.15.167:6836/2720148065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55ae9b376200 con 0x55ae9b630a60 2024-06-10T00:56:19.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.081+0000 7f7e717fa700 10 client.15171 handle_command_reply: tid=0 2024-06-10T00:56:19.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.081+0000 7f7e9a7fcb80 1 client.15171 shutdown 2024-06-10T00:56:19.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.081+0000 7f7e9a7fcb80 1 -- 172.21.15.37:0/1322115022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae9b5a2ed0 msgr2=0x55ae9b5ab4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:19.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.081+0000 7f7e9a7fcb80 1 --2- 172.21.15.37:0/1322115022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae9b5a2ed0 0x55ae9b5ab4c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7e840098b0 tx=0x7f7e84015cf0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.081+0000 7f7e9a7fcb80 1 -- 172.21.15.37:0/1322115022 shutdown_connections 2024-06-10T00:56:19.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.081+0000 7f7e9a7fcb80 1 --2- 172.21.15.37:0/1322115022 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55ae9b630a60 0x55ae9b62f770 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.081+0000 7f7e9a7fcb80 1 --2- 172.21.15.37:0/1322115022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ae9b5a8f10 0x55ae9b5abf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.081+0000 7f7e717fa700 0 client.15171 ms_handle_reset on v2:172.21.15.167:6836/2720148065 2024-06-10T00:56:19.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.081+0000 7f7e9a7fcb80 1 --2- 172.21.15.37:0/1322115022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae9b5a2ed0 0x55ae9b5ab4c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.081+0000 7f7e9a7fcb80 1 --2- 172.21.15.37:0/1322115022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae9b5a85f0 0x55ae9b5aba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.081+0000 7f7e9a7fcb80 1 -- 172.21.15.37:0/1322115022 >> 172.21.15.37:0/1322115022 conn(0x55ae9b4cdc70 msgr2=0x55ae9b5a92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.081+0000 7f7e9a7fcb80 1 -- 172.21.15.37:0/1322115022 shutdown_connections 2024-06-10T00:56:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.081+0000 7f7e9a7fcb80 1 -- 172.21.15.37:0/1322115022 wait complete. 2024-06-10T00:56:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.081+0000 7f7e9a7fcb80 20 client.15171 trim_cache size 0 max 16384 2024-06-10T00:56:19.088 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T00:56:19.088 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "68793514-26c2-11ef-bc9b-c7b262605968", 2024-06-10T00:56:19.088 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 2, 2024-06-10T00:56:19.088 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24451, 2024-06-10T00:56:19.088 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-06-10T00:56:19.089 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-06-10T00:56:19.089 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-06-10T00:56:19.089 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 159.802030634, 2024-06-10T00:56:19.089 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-06-10T00:56:19.089 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 100, 2024-06-10T00:56:19.089 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 98, 2024-06-10T00:56:19.089 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 254.58106965100001 2024-06-10T00:56:19.089 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T00:56:19.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.082+0000 7f7e8ff79700 1 -- 172.21.15.37:0/3521082326 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7e740a2c20 msgr2=0x7f7e740a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:19.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.082+0000 7f7e8ff79700 1 --2- 172.21.15.37:0/3521082326 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7e740a2c20 0x7f7e740a50e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009a10 tx=0x7f7e84009190 comp rx=0 tx=0).stop 2024-06-10T00:56:19.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.082+0000 7f7e8ff79700 1 -- 172.21.15.37:0/3521082326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e88107870 msgr2=0x7f7e88198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:19.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.082+0000 7f7e8ff79700 1 --2- 172.21.15.37:0/3521082326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e88107870 0x7f7e88198b00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000b970 tx=0x7f7e8000bc80 comp rx=0 tx=0).stop 2024-06-10T00:56:19.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.082+0000 7f7e8ff79700 1 -- 172.21.15.37:0/3521082326 shutdown_connections 2024-06-10T00:56:19.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.082+0000 7f7e8ff79700 1 --2- 172.21.15.37:0/3521082326 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7e740a2c20 0x7f7e740a50e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.082+0000 7f7e8ff79700 1 --2- 172.21.15.37:0/3521082326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e88107870 0x7f7e88198b00 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.082+0000 7f7e8ff79700 1 --2- 172.21.15.37:0/3521082326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e881027a0 0x7f7e8819e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.082+0000 7f7e8ff79700 1 --2- 172.21.15.37:0/3521082326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e88103150 0x7f7e8819ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:19.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.082+0000 7f7e8ff79700 1 -- 172.21.15.37:0/3521082326 >> 172.21.15.37:0/3521082326 conn(0x7f7e88079520 msgr2=0x7f7e881089c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:19.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.082+0000 7f7e8ff79700 1 -- 172.21.15.37:0/3521082326 shutdown_connections 2024-06-10T00:56:19.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:19.082+0000 7f7e8ff79700 1 -- 172.21.15.37:0/3521082326 wait complete. 2024-06-10T00:56:19.839 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:19 smithi037 ceph-mon[32192]: pgmap v419: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:56:19.869 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-10T00:56:20.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:19 smithi167 ceph-mon[36917]: pgmap v419: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:56:20.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:19 smithi094 ceph-mon[36708]: pgmap v419: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:56:20.141 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:56:20.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:20 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:56:20.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:20 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:56:20.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:20 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:56:20.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:20 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:56:20.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:20 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:56:21.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:20 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:56:21.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:20 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:56:21.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:20 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:56:21.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:20 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:56:21.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:20 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:56:21.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:20 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:56:21.114 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:20 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:56:21.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:20 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:56:21.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:20 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:56:21.115 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:20 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:56:21.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.881+0000 7f04dcf64700 1 -- 172.21.15.37:0/3294104936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04d8107400 msgr2=0x7f04d81077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:21.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.881+0000 7f04dcf64700 1 --2- 172.21.15.37:0/3294104936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04d8107400 0x7f04d81077e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009990 tx=0x7f04c8009ca0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.881+0000 7f04dcf64700 1 -- 172.21.15.37:0/3294104936 shutdown_connections 2024-06-10T00:56:21.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.881+0000 7f04dcf64700 1 --2- 172.21.15.37:0/3294104936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04d8101d50 0x7f04d810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.881+0000 7f04dcf64700 1 --2- 172.21.15.37:0/3294104936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04d8107400 0x7f04d81077e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.881+0000 7f04dcf64700 1 --2- 172.21.15.37:0/3294104936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04d8101390 0x7f04d8101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.881+0000 7f04dcf64700 1 -- 172.21.15.37:0/3294104936 >> 172.21.15.37:0/3294104936 conn(0x7f04d8078340 msgr2=0x7f04d8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:21.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.881+0000 7f04dcf64700 1 -- 172.21.15.37:0/3294104936 shutdown_connections 2024-06-10T00:56:21.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.881+0000 7f04dcf64700 1 -- 172.21.15.37:0/3294104936 wait complete. 2024-06-10T00:56:21.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.882+0000 7f04dcf64700 1 Processor -- start 2024-06-10T00:56:21.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.882+0000 7f04dcf64700 1 -- start start 2024-06-10T00:56:21.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.882+0000 7f04dcf64700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04d8101390 0x7f04d819e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:21.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.882+0000 7f04dcf64700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04d8101d50 0x7f04d819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:21.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.882+0000 7f04dcf64700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04d8107400 0x7f04d8198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:21.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.883+0000 7f04dcf64700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81116f0 con 0x7f04d8101390 2024-06-10T00:56:21.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.883+0000 7f04dcf64700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f04d8111570 con 0x7f04d8101d50 2024-06-10T00:56:21.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.883+0000 7f04dcf64700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f04d8111870 con 0x7f04d8107400 2024-06-10T00:56:21.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.883+0000 7f04d659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04d8101390 0x7f04d819e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:21.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.883+0000 7f04d659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04d8101390 0x7f04d819e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50266/0 (socket says 172.21.15.37:50266) 2024-06-10T00:56:21.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.883+0000 7f04d659c700 1 -- 172.21.15.37:0/1651715218 learned_addr learned my addr 172.21.15.37:0/1651715218 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:21.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.883+0000 7f04d6d9d700 1 --2- 172.21.15.37:0/1651715218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04d8107400 0x7f04d8198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:21.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.883+0000 7f04d5d9b700 1 --2- 172.21.15.37:0/1651715218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04d8101d50 0x7f04d819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:21.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.883+0000 7f04d659c700 1 -- 172.21.15.37:0/1651715218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04d8107400 msgr2=0x7f04d8198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:21.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.883+0000 7f04d659c700 1 --2- 172.21.15.37:0/1651715218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04d8107400 0x7f04d8198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.883+0000 7f04d659c700 1 -- 172.21.15.37:0/1651715218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04d8101d50 msgr2=0x7f04d819ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:21.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.883+0000 7f04d659c700 1 --2- 172.21.15.37:0/1651715218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04d8101d50 0x7f04d819ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.883+0000 7f04d659c700 1 -- 172.21.15.37:0/1651715218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d8199230 con 0x7f04d8101390 2024-06-10T00:56:21.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.884+0000 7f04d659c700 1 --2- 172.21.15.37:0/1651715218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04d8101390 0x7f04d819e910 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f04c800bff0 tx=0x7f04c8005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:21.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.884+0000 7f04c77fe700 1 -- 172.21.15.37:0/1651715218 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8027070 con 0x7f04d8101390 2024-06-10T00:56:21.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.884+0000 7f04dcf64700 1 -- 172.21.15.37:0/1651715218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d81994c0 con 0x7f04d8101390 2024-06-10T00:56:21.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.884+0000 7f04c77fe700 1 -- 172.21.15.37:0/1651715218 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c8004050 con 0x7f04d8101390 2024-06-10T00:56:21.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.884+0000 7f04c77fe700 1 -- 172.21.15.37:0/1651715218 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8022d00 con 0x7f04d8101390 2024-06-10T00:56:21.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.884+0000 7f04dcf64700 1 -- 172.21.15.37:0/1651715218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d81a57e0 con 0x7f04d8101390 2024-06-10T00:56:21.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.887+0000 7f04c77fe700 1 -- 172.21.15.37:0/1651715218 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f04c802a070 con 0x7f04d8101390 2024-06-10T00:56:21.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.887+0000 7f04c77fe700 1 --2- 172.21.15.37:0/1651715218 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f04c00a2ca0 0x7f04c00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:21.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.887+0000 7f04d5d9b700 1 --2- 172.21.15.37:0/1651715218 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f04c00a2ca0 0x7f04c00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:21.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.887+0000 7f04c77fe700 1 -- 172.21.15.37:0/1651715218 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f04c80d02b0 con 0x7f04d8101390 2024-06-10T00:56:21.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.888+0000 7f04d5d9b700 1 --2- 172.21.15.37:0/1651715218 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f04c00a2ca0 0x7f04c00a5160 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f04d80ffb20 tx=0x7f04cc009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:21.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.888+0000 7f04e77e7b80 1 Processor -- start 2024-06-10T00:56:21.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.888+0000 7f04e77e7b80 1 -- start start 2024-06-10T00:56:21.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.888+0000 7f04e77e7b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f584f35f0 0x563f584f39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:21.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.888+0000 7f04e77e7b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f584f3f10 0x563f584ed990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:21.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.889+0000 7f04e77e7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f584eded0 0x563f584ee2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:21.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.889+0000 7f04e77e7b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563f5843f7c0 con 0x563f584f35f0 2024-06-10T00:56:21.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.889+0000 7f04e77e7b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563f583e76d0 con 0x563f584f3f10 2024-06-10T00:56:21.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.889+0000 7f04e77e7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563f5843b780 con 0x563f584eded0 2024-06-10T00:56:21.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.889+0000 7f04d659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f584f35f0 0x563f584f39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:21.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.889+0000 7f04d659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f584f35f0 0x563f584f39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50270/0 (socket says 172.21.15.37:50270) 2024-06-10T00:56:21.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.889+0000 7f04d659c700 1 -- 172.21.15.37:0/864645526 learned_addr learned my addr 172.21.15.37:0/864645526 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:21.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.889+0000 7f04d5d9b700 1 --2- 172.21.15.37:0/864645526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f584f3f10 0x563f584ed990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:21.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.889+0000 7f04d6d9d700 1 --2- 172.21.15.37:0/864645526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f584eded0 0x563f584ee2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:21.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.889+0000 7f04d659c700 1 -- 172.21.15.37:0/864645526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f584eded0 msgr2=0x563f584ee2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:21.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.889+0000 7f04d659c700 1 --2- 172.21.15.37:0/864645526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f584eded0 0x563f584ee2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.889+0000 7f04d659c700 1 -- 172.21.15.37:0/864645526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f584f3f10 msgr2=0x563f584ed990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:21.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.889+0000 7f04d659c700 1 --2- 172.21.15.37:0/864645526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f584f3f10 0x563f584ed990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.889+0000 7f04d659c700 1 -- 172.21.15.37:0/864645526 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f583ddf00 con 0x563f584f35f0 2024-06-10T00:56:21.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.889+0000 7f04d5d9b700 1 --2- 172.21.15.37:0/864645526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f584f3f10 0x563f584ed990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:56:21.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.889+0000 7f04d659c700 1 --2- 172.21.15.37:0/864645526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f584f35f0 0x563f584f39d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f04c8099120 tx=0x7f04c8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c539c4ef927cd686 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:21.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.889+0000 7f04c4ff9700 1 -- 172.21.15.37:0/864645526 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c809ce50 con 0x563f584f35f0 2024-06-10T00:56:21.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.889+0000 7f04c4ff9700 1 -- 172.21.15.37:0/864645526 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c809c750 con 0x563f584f35f0 2024-06-10T00:56:21.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.889+0000 7f04c4ff9700 1 -- 172.21.15.37:0/864645526 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c809c9b0 con 0x563f584f35f0 2024-06-10T00:56:21.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.890+0000 7f04e77e7b80 1 -- 172.21.15.37:0/864645526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f584f35f0 msgr2=0x563f584f39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:21.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.890+0000 7f04e77e7b80 1 --2- 172.21.15.37:0/864645526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f584f35f0 0x563f584f39d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f04c8099120 tx=0x7f04c8009ca0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.890+0000 7f04e77e7b80 1 -- 172.21.15.37:0/864645526 shutdown_connections 2024-06-10T00:56:21.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.890+0000 7f04e77e7b80 1 --2- 172.21.15.37:0/864645526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f584eded0 0x563f584ee2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.890+0000 7f04e77e7b80 1 --2- 172.21.15.37:0/864645526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f584f35f0 0x563f584f39d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.890+0000 7f04e77e7b80 1 --2- 172.21.15.37:0/864645526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f584f3f10 0x563f584ed990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.890+0000 7f04e77e7b80 1 -- 172.21.15.37:0/864645526 >> 172.21.15.37:0/864645526 conn(0x563f58418c70 msgr2=0x563f584e8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:21.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.890+0000 7f04e77e7b80 1 -- 172.21.15.37:0/864645526 shutdown_connections 2024-06-10T00:56:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.890+0000 7f04e77e7b80 1 -- 172.21.15.37:0/864645526 wait complete. 2024-06-10T00:56:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.891+0000 7f04e77e7b80 1 -- start start 2024-06-10T00:56:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.891+0000 7f04e77e7b80 1 Processor -- start 2024-06-10T00:56:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.891+0000 7f04e77e7b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f584eded0 0x563f584f9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.891+0000 7f04e77e7b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f584f35f0 0x563f584f6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.891+0000 7f04e77e7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f584f3f10 0x563f584f7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.891+0000 7f04e77e7b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563f584fa6a0 con 0x563f584f35f0 2024-06-10T00:56:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.891+0000 7f04e77e7b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563f583dfba0 con 0x563f584eded0 2024-06-10T00:56:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.891+0000 7f04e77e7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563f584fa820 con 0x563f584f3f10 2024-06-10T00:56:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.891+0000 7f04e77e7b80 10 client.0 _ll_register_callbacks cb 0x563f583d98d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:56:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.891+0000 7f04d659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f584eded0 0x563f584f9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.891+0000 7f04d659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f584eded0 0x563f584f9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:53126/0 (socket says 172.21.15.37:53126) 2024-06-10T00:56:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.891+0000 7f04d659c700 1 -- 172.21.15.37:0/1554975842 learned_addr learned my addr 172.21.15.37:0/1554975842 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.891+0000 7f04d6d9d700 1 --2- 172.21.15.37:0/1554975842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f584f3f10 0x563f584f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.892+0000 7f04d5d9b700 1 --2- 172.21.15.37:0/1554975842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f584f35f0 0x563f584f6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.892+0000 7f04d659c700 1 -- 172.21.15.37:0/1554975842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f584f3f10 msgr2=0x563f584f7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.892+0000 7f04d659c700 1 --2- 172.21.15.37:0/1554975842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f584f3f10 0x563f584f7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.892+0000 7f04d659c700 1 -- 172.21.15.37:0/1554975842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f584f35f0 msgr2=0x563f584f6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.892+0000 7f04d659c700 1 --2- 172.21.15.37:0/1554975842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f584f35f0 0x563f584f6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.892+0000 7f04d659c700 1 -- 172.21.15.37:0/1554975842 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f5838e880 con 0x563f584eded0 2024-06-10T00:56:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.892+0000 7f04d659c700 1 --2- 172.21.15.37:0/1554975842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f584eded0 0x563f584f9c50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f04c80a4e00 tx=0x7f04c8005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc446e8eb060a20 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.892+0000 7f04bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T00:56:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.892+0000 7f04bdffb700 1 -- 172.21.15.37:0/1554975842 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c80b5020 con 0x563f584eded0 2024-06-10T00:56:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.892+0000 7f04bdffb700 1 -- 172.21.15.37:0/1554975842 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f04c80b0be0 con 0x563f584eded0 2024-06-10T00:56:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.892+0000 7f04e77e7b80 1 -- 172.21.15.37:0/1554975842 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04c809c750 con 0x563f584eded0 2024-06-10T00:56:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.893+0000 7f04bdffb700 1 -- 172.21.15.37:0/1554975842 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f04c80a0b00 con 0x563f584eded0 2024-06-10T00:56:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.893+0000 7f04e77e7b80 10 client.24872 fetch_fsmap learned FSMap version 26 2024-06-10T00:56:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.893+0000 7f04e77e7b80 1 -- 172.21.15.37:0/1554975842 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563f584fa6a0 con 0x563f584eded0 2024-06-10T00:56:21.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.893+0000 7f04bdffb700 1 -- 172.21.15.37:0/1554975842 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c809c750 con 0x563f584eded0 2024-06-10T00:56:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.893+0000 7f04bdffb700 1 -- 172.21.15.37:0/1554975842 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f04c8085ce0 con 0x563f584eded0 2024-06-10T00:56:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.893+0000 7f04e77e7b80 10 client.24872 fetch_fsmap finished waiting for FSMap version 26 2024-06-10T00:56:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.893+0000 7f04e77e7b80 10 client.24872 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-10T00:56:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.893+0000 7f04e77e7b80 20 client.24872 populate_metadata read hostname 'smithi037' 2024-06-10T00:56:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.893+0000 7f04e77e7b80 1 --2- 172.21.15.37:0/1554975842 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x563f58507360 0x563f585087d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:21.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.893+0000 7f04e77e7b80 4 client.24872 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:56:21.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.893+0000 7f04e77e7b80 1 -- 172.21.15.37:0/1554975842 --> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f5844ccc0 con 0x563f58507360 2024-06-10T00:56:21.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.894+0000 7f04d5d9b700 1 --2- 172.21.15.37:0/1554975842 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x563f58507360 0x563f585087d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:21.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.894+0000 7f04d5d9b700 1 --2- 172.21.15.37:0/1554975842 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x563f58507360 0x563f585087d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=30fdddc511de9f7f server_cookie=c19da4f022fbd477 in_seq=0 out_seq=0 2024-06-10T00:56:21.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.894+0000 7f04bdffb700 10 client.24872 ms_handle_connect on v2:172.21.15.94:6838/946269331 2024-06-10T00:56:21.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.895+0000 7f04bdffb700 1 -- 172.21.15.37:0/1554975842 <== mds.1 v2:172.21.15.94:6838/946269331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563f5844ccc0 con 0x563f58507360 2024-06-10T00:56:21.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.895+0000 7f04bdffb700 10 client.24872 handle_command_reply: tid=0 2024-06-10T00:56:21.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.895+0000 7f04e77e7b80 1 client.24872 shutdown 2024-06-10T00:56:21.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.895+0000 7f04e77e7b80 1 -- 172.21.15.37:0/1554975842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f584eded0 msgr2=0x563f584f9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:21.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.895+0000 7f04e77e7b80 1 --2- 172.21.15.37:0/1554975842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f584eded0 0x563f584f9c50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f04c80a4e00 tx=0x7f04c8005770 comp rx=0 tx=0).stop 2024-06-10T00:56:21.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.895+0000 7f04e77e7b80 1 -- 172.21.15.37:0/1554975842 shutdown_connections 2024-06-10T00:56:21.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.895+0000 7f04e77e7b80 1 --2- 172.21.15.37:0/1554975842 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x563f58507360 0x563f585087d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.895+0000 7f04e77e7b80 1 --2- 172.21.15.37:0/1554975842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f584f3f10 0x563f584f7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.895+0000 7f04e77e7b80 1 --2- 172.21.15.37:0/1554975842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f584eded0 0x563f584f9c50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.895+0000 7f04e77e7b80 1 --2- 172.21.15.37:0/1554975842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f584f35f0 0x563f584f6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.895+0000 7f04bdffb700 0 client.24872 ms_handle_reset on v2:172.21.15.94:6838/946269331 2024-06-10T00:56:21.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.895+0000 7f04e77e7b80 1 -- 172.21.15.37:0/1554975842 >> 172.21.15.37:0/1554975842 conn(0x563f58418c70 msgr2=0x563f584e85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:21.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.895+0000 7f04e77e7b80 1 -- 172.21.15.37:0/1554975842 shutdown_connections 2024-06-10T00:56:21.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.895+0000 7f04e77e7b80 1 -- 172.21.15.37:0/1554975842 wait complete. 2024-06-10T00:56:21.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.895+0000 7f04e77e7b80 20 client.24872 trim_cache size 0 max 16384 2024-06-10T00:56:21.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.917+0000 7f04e77e7b80 1 Processor -- start 2024-06-10T00:56:21.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.917+0000 7f04e77e7b80 1 -- start start 2024-06-10T00:56:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.917+0000 7f04e77e7b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f584eded0 0x563f584ed990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.917+0000 7f04e77e7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f584f35f0 0x563f584f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.917+0000 7f04e77e7b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f584f3f10 0x563f584e1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.917+0000 7f04e77e7b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563f58506b10 con 0x563f584eded0 2024-06-10T00:56:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.917+0000 7f04e77e7b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563f585071e0 con 0x563f584f3f10 2024-06-10T00:56:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.917+0000 7f04e77e7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563f585078b0 con 0x563f584f35f0 2024-06-10T00:56:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.917+0000 7f04d659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f584eded0 0x563f584ed990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.917+0000 7f04d659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f584eded0 0x563f584ed990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50292/0 (socket says 172.21.15.37:50292) 2024-06-10T00:56:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.917+0000 7f04d659c700 1 -- 172.21.15.37:0/1119759334 learned_addr learned my addr 172.21.15.37:0/1119759334 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.918+0000 7f04d6d9d700 1 --2- 172.21.15.37:0/1119759334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f584f3f10 0x563f584e1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.918+0000 7f04d5d9b700 1 --2- 172.21.15.37:0/1119759334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f584f35f0 0x563f584f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.918+0000 7f04d659c700 1 -- 172.21.15.37:0/1119759334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f584f35f0 msgr2=0x563f584f4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.918+0000 7f04d659c700 1 --2- 172.21.15.37:0/1119759334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f584f35f0 0x563f584f4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.918+0000 7f04d659c700 1 -- 172.21.15.37:0/1119759334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f584f3f10 msgr2=0x563f584e1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.918+0000 7f04d659c700 1 --2- 172.21.15.37:0/1119759334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f584f3f10 0x563f584e1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.918+0000 7f04d659c700 1 -- 172.21.15.37:0/1119759334 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f584e8330 con 0x563f584eded0 2024-06-10T00:56:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.918+0000 7f04d659c700 1 --2- 172.21.15.37:0/1119759334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f584eded0 0x563f584ed990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f04c8099120 tx=0x7f04c809ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc36e9a34171a623 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.918+0000 7f04bffff700 1 -- 172.21.15.37:0/1119759334 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c80b5050 con 0x563f584eded0 2024-06-10T00:56:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.918+0000 7f04bffff700 1 -- 172.21.15.37:0/1119759334 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c809ce50 con 0x563f584eded0 2024-06-10T00:56:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.918+0000 7f04bffff700 1 -- 172.21.15.37:0/1119759334 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c80870b0 con 0x563f584eded0 2024-06-10T00:56:21.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.919+0000 7f04e77e7b80 1 -- 172.21.15.37:0/1119759334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f584eded0 msgr2=0x563f584ed990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:21.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.919+0000 7f04e77e7b80 1 --2- 172.21.15.37:0/1119759334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f584eded0 0x563f584ed990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f04c8099120 tx=0x7f04c809ac60 comp rx=0 tx=0).stop 2024-06-10T00:56:21.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.919+0000 7f04e77e7b80 1 -- 172.21.15.37:0/1119759334 shutdown_connections 2024-06-10T00:56:21.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.919+0000 7f04e77e7b80 1 --2- 172.21.15.37:0/1119759334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f584f3f10 0x563f584e1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.919+0000 7f04e77e7b80 1 --2- 172.21.15.37:0/1119759334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f584eded0 0x563f584ed990 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.919+0000 7f04e77e7b80 1 --2- 172.21.15.37:0/1119759334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f584f35f0 0x563f584f4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.919+0000 7f04e77e7b80 1 -- 172.21.15.37:0/1119759334 >> 172.21.15.37:0/1119759334 conn(0x563f58418c70 msgr2=0x563f584f42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:21.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.919+0000 7f04e77e7b80 1 -- 172.21.15.37:0/1119759334 shutdown_connections 2024-06-10T00:56:21.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.919+0000 7f04e77e7b80 1 -- 172.21.15.37:0/1119759334 wait complete. 2024-06-10T00:56:21.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.919+0000 7f04e77e7b80 1 -- start start 2024-06-10T00:56:21.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.919+0000 7f04e77e7b80 1 Processor -- start 2024-06-10T00:56:21.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.920+0000 7f04e77e7b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f584eded0 0x563f58512300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:21.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.920+0000 7f04e77e7b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f584f35f0 0x563f584ec980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:21.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.920+0000 7f04e77e7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f584f3f10 0x563f584ecec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:21.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.920+0000 7f04e77e7b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563f58508650 con 0x563f584eded0 2024-06-10T00:56:21.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.920+0000 7f04e77e7b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563f58507f80 con 0x563f584f35f0 2024-06-10T00:56:21.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.920+0000 7f04e77e7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563f58508d20 con 0x563f584f3f10 2024-06-10T00:56:21.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.920+0000 7f04e77e7b80 10 client.0 _ll_register_callbacks cb 0x563f583d98d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:56:21.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.920+0000 7f04d659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f584eded0 0x563f58512300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:21.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.920+0000 7f04d659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f584eded0 0x563f58512300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50296/0 (socket says 172.21.15.37:50296) 2024-06-10T00:56:21.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.920+0000 7f04d659c700 1 -- 172.21.15.37:0/2603394066 learned_addr learned my addr 172.21.15.37:0/2603394066 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:21.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.920+0000 7f04d6d9d700 1 --2- 172.21.15.37:0/2603394066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f584f3f10 0x563f584ecec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:21.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.920+0000 7f04d5d9b700 1 --2- 172.21.15.37:0/2603394066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f584f35f0 0x563f584ec980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:21.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.920+0000 7f04d659c700 1 -- 172.21.15.37:0/2603394066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f584f3f10 msgr2=0x563f584ecec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:21.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.920+0000 7f04d659c700 1 --2- 172.21.15.37:0/2603394066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f584f3f10 0x563f584ecec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.920+0000 7f04d659c700 1 -- 172.21.15.37:0/2603394066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f584f35f0 msgr2=0x563f584ec980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:21.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.920+0000 7f04d659c700 1 --2- 172.21.15.37:0/2603394066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f584f35f0 0x563f584ec980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.920+0000 7f04d659c700 1 -- 172.21.15.37:0/2603394066 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f584ed6c0 con 0x563f584eded0 2024-06-10T00:56:21.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.920+0000 7f04d5d9b700 1 --2- 172.21.15.37:0/2603394066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f584f35f0 0x563f584ec980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:56:21.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.920+0000 7f04d659c700 1 --2- 172.21.15.37:0/2603394066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f584eded0 0x563f58512300 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f04c8003900 tx=0x7f04c809c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97224a4e3c2c84a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:21.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.920+0000 7f04bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:56:21.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.920+0000 7f04bdffb700 1 -- 172.21.15.37:0/2603394066 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c80b5040 con 0x563f584eded0 2024-06-10T00:56:21.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.920+0000 7f04bdffb700 1 -- 172.21.15.37:0/2603394066 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f04c8087020 con 0x563f584eded0 2024-06-10T00:56:21.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.920+0000 7f04e77e7b80 1 -- 172.21.15.37:0/2603394066 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04c809ce50 con 0x563f584eded0 2024-06-10T00:56:21.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.921+0000 7f04bdffb700 1 -- 172.21.15.37:0/2603394066 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f04c80a5330 con 0x563f584eded0 2024-06-10T00:56:21.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.921+0000 7f04e77e7b80 10 client.15189 fetch_fsmap learned FSMap version 26 2024-06-10T00:56:21.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.921+0000 7f04e77e7b80 1 -- 172.21.15.37:0/2603394066 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563f584ed6c0 con 0x563f584eded0 2024-06-10T00:56:21.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.921+0000 7f04bdffb700 1 -- 172.21.15.37:0/2603394066 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c809ce50 con 0x563f584eded0 2024-06-10T00:56:21.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.921+0000 7f04bdffb700 1 -- 172.21.15.37:0/2603394066 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f04c8085ce0 con 0x563f584eded0 2024-06-10T00:56:21.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.921+0000 7f04e77e7b80 10 client.15189 fetch_fsmap finished waiting for FSMap version 26 2024-06-10T00:56:21.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.921+0000 7f04e77e7b80 10 client.15189 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-10T00:56:21.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.921+0000 7f04e77e7b80 20 client.15189 populate_metadata read hostname 'smithi037' 2024-06-10T00:56:21.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.921+0000 7f04e77e7b80 1 --2- 172.21.15.37:0/2603394066 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x563f58576fc0 0x563f58575cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:21.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.921+0000 7f04e77e7b80 4 client.15189 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T00:56:21.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.921+0000 7f04e77e7b80 1 -- 172.21.15.37:0/2603394066 --> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] -- command(tid 0: {"prefix": "status"}) -- 0x563f58447240 con 0x563f58576fc0 2024-06-10T00:56:21.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.922+0000 7f04d5d9b700 1 --2- 172.21.15.37:0/2603394066 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x563f58576fc0 0x563f58575cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:21.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.922+0000 7f04d5d9b700 1 --2- 172.21.15.37:0/2603394066 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x563f58576fc0 0x563f58575cd0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c5c0b83fc8011c30 server_cookie=26e4803aababab9a in_seq=0 out_seq=0 2024-06-10T00:56:21.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.922+0000 7f04bdffb700 10 client.15189 ms_handle_connect on v2:172.21.15.94:6838/946269331 2024-06-10T00:56:21.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.922+0000 7f04bdffb700 1 -- 172.21.15.37:0/2603394066 <== mds.1 v2:172.21.15.94:6838/946269331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x563f58447240 con 0x563f58576fc0 2024-06-10T00:56:21.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.922+0000 7f04bdffb700 10 client.15189 handle_command_reply: tid=0 2024-06-10T00:56:21.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.922+0000 7f04e77e7b80 1 client.15189 shutdown 2024-06-10T00:56:21.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.922+0000 7f04e77e7b80 1 -- 172.21.15.37:0/2603394066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f584eded0 msgr2=0x563f58512300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:21.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.922+0000 7f04e77e7b80 1 --2- 172.21.15.37:0/2603394066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f584eded0 0x563f58512300 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f04c8003900 tx=0x7f04c809c3c0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.922+0000 7f04e77e7b80 1 -- 172.21.15.37:0/2603394066 shutdown_connections 2024-06-10T00:56:21.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.922+0000 7f04e77e7b80 1 --2- 172.21.15.37:0/2603394066 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x563f58576fc0 0x563f58575cd0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.922+0000 7f04e77e7b80 1 --2- 172.21.15.37:0/2603394066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f584f3f10 0x563f584ecec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.922+0000 7f04bdffb700 0 client.15189 ms_handle_reset on v2:172.21.15.94:6838/946269331 2024-06-10T00:56:21.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.922+0000 7f04e77e7b80 1 --2- 172.21.15.37:0/2603394066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f584eded0 0x563f58512300 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.922+0000 7f04e77e7b80 1 --2- 172.21.15.37:0/2603394066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f584f35f0 0x563f584ec980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.922+0000 7f04e77e7b80 1 -- 172.21.15.37:0/2603394066 >> 172.21.15.37:0/2603394066 conn(0x563f58418c70 msgr2=0x563f584e1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:21.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.922+0000 7f04e77e7b80 1 -- 172.21.15.37:0/2603394066 shutdown_connections 2024-06-10T00:56:21.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.922+0000 7f04e77e7b80 1 -- 172.21.15.37:0/2603394066 wait complete. 2024-06-10T00:56:21.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.923+0000 7f04e77e7b80 20 client.15189 trim_cache size 0 max 16384 2024-06-10T00:56:21.932 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T00:56:21.932 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "68793514-26c2-11ef-bc9b-c7b262605968", 2024-06-10T00:56:21.932 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 1, 2024-06-10T00:56:21.932 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24458, 2024-06-10T00:56:21.932 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-06-10T00:56:21.932 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-06-10T00:56:21.933 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-06-10T00:56:21.933 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 163.649708584, 2024-06-10T00:56:21.933 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-06-10T00:56:21.933 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 100, 2024-06-10T00:56:21.933 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 98, 2024-06-10T00:56:21.933 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 241.363466012 2024-06-10T00:56:21.933 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T00:56:21.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.923+0000 7f04dcf64700 1 -- 172.21.15.37:0/1651715218 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f04c00a2ca0 msgr2=0x7f04c00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:21.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.923+0000 7f04dcf64700 1 --2- 172.21.15.37:0/1651715218 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f04c00a2ca0 0x7f04c00a5160 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f04d80ffb20 tx=0x7f04cc009190 comp rx=0 tx=0).stop 2024-06-10T00:56:21.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.923+0000 7f04dcf64700 1 -- 172.21.15.37:0/1651715218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04d8101390 msgr2=0x7f04d819e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:21.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.923+0000 7f04dcf64700 1 --2- 172.21.15.37:0/1651715218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04d8101390 0x7f04d819e910 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f04c800bff0 tx=0x7f04c8005590 comp rx=0 tx=0).stop 2024-06-10T00:56:21.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.923+0000 7f04dcf64700 1 -- 172.21.15.37:0/1651715218 shutdown_connections 2024-06-10T00:56:21.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.923+0000 7f04dcf64700 1 --2- 172.21.15.37:0/1651715218 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f04c00a2ca0 0x7f04c00a5160 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.923+0000 7f04dcf64700 1 --2- 172.21.15.37:0/1651715218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04d8107400 0x7f04d8198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.923+0000 7f04dcf64700 1 --2- 172.21.15.37:0/1651715218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04d8101390 0x7f04d819e910 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.923+0000 7f04dcf64700 1 --2- 172.21.15.37:0/1651715218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04d8101d50 0x7f04d819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:21.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.923+0000 7f04dcf64700 1 -- 172.21.15.37:0/1651715218 >> 172.21.15.37:0/1651715218 conn(0x7f04d8078340 msgr2=0x7f04d81006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:21.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.923+0000 7f04dcf64700 1 -- 172.21.15.37:0/1651715218 shutdown_connections 2024-06-10T00:56:21.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:21.923+0000 7f04dcf64700 1 -- 172.21.15.37:0/1651715218 wait complete. 2024-06-10T00:56:22.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:21 smithi037 ceph-mon[32192]: pgmap v420: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:56:22.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:56:22.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:56:22.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:56:22.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:56:22.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:21 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:56:22.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:21 smithi167 ceph-mon[36917]: pgmap v420: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:56:22.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:56:22.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:56:22.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:56:22.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:56:22.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:21 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:56:22.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:21 smithi094 ceph-mon[36708]: pgmap v420: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:56:22.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:56:22.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:56:22.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:56:22.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:56:22.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:21 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:56:22.670 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-10T00:56:22.921 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:56:24.033 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:23 smithi037 ceph-mon[32192]: pgmap v421: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:56:24.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:23 smithi167 ceph-mon[36917]: pgmap v421: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:56:24.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:23 smithi094 ceph-mon[36708]: pgmap v421: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:56:24.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.594+0000 7f91168fc700 1 -- 172.21.15.37:0/241325087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9110101d40 msgr2=0x7f911010e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:24.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.594+0000 7f91168fc700 1 --2- 172.21.15.37:0/241325087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9110101d40 0x7f911010e230 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f910800b0a0 tx=0x7f910800b3b0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.594+0000 7f91168fc700 1 -- 172.21.15.37:0/241325087 shutdown_connections 2024-06-10T00:56:24.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.594+0000 7f91168fc700 1 --2- 172.21.15.37:0/241325087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9110101d40 0x7f911010e230 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.594+0000 7f91168fc700 1 --2- 172.21.15.37:0/241325087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91101073f0 0x7f91101077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.594+0000 7f91168fc700 1 --2- 172.21.15.37:0/241325087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9110101380 0x7f9110101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.594+0000 7f91168fc700 1 -- 172.21.15.37:0/241325087 >> 172.21.15.37:0/241325087 conn(0x7f9110078300 msgr2=0x7f9110078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:24.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.594+0000 7f91168fc700 1 -- 172.21.15.37:0/241325087 shutdown_connections 2024-06-10T00:56:24.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.594+0000 7f91168fc700 1 -- 172.21.15.37:0/241325087 wait complete. 2024-06-10T00:56:24.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.595+0000 7f91168fc700 1 Processor -- start 2024-06-10T00:56:24.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.595+0000 7f91168fc700 1 -- start start 2024-06-10T00:56:24.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.595+0000 7f91168fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9110101380 0x7f911019aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:24.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.595+0000 7f91168fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9110101d40 0x7f911019b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:24.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.595+0000 7f91168fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91101073f0 0x7f911019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:24.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.595+0000 7f91168fc700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9110111760 con 0x7f9110101d40 2024-06-10T00:56:24.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.595+0000 7f91168fc700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f91101115e0 con 0x7f91101073f0 2024-06-10T00:56:24.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.595+0000 7f91168fc700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f91101118e0 con 0x7f9110101380 2024-06-10T00:56:24.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.596+0000 7f9107fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9110101d40 0x7f911019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:24.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.596+0000 7f9107fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9110101d40 0x7f911019b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50304/0 (socket says 172.21.15.37:50304) 2024-06-10T00:56:24.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.596+0000 7f9107fff700 1 -- 172.21.15.37:0/1299590073 learned_addr learned my addr 172.21.15.37:0/1299590073 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:24.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.596+0000 7f9114e99700 1 --2- 172.21.15.37:0/1299590073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91101073f0 0x7f911019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:24.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.596+0000 7f910ffff700 1 --2- 172.21.15.37:0/1299590073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9110101380 0x7f911019aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:24.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.596+0000 7f9107fff700 1 -- 172.21.15.37:0/1299590073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9110101380 msgr2=0x7f911019aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:24.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.596+0000 7f9107fff700 1 --2- 172.21.15.37:0/1299590073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9110101380 0x7f911019aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.596+0000 7f9107fff700 1 -- 172.21.15.37:0/1299590073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91101073f0 msgr2=0x7f911019f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:24.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.596+0000 7f9107fff700 1 --2- 172.21.15.37:0/1299590073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91101073f0 0x7f911019f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.596+0000 7f9107fff700 1 -- 172.21.15.37:0/1299590073 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f911019fee0 con 0x7f9110101d40 2024-06-10T00:56:24.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.596+0000 7f9107fff700 1 --2- 172.21.15.37:0/1299590073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9110101d40 0x7f911019b410 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f90f800b530 tx=0x7f90f800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:24.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.596+0000 7f910dffb700 1 -- 172.21.15.37:0/1299590073 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8015020 con 0x7f9110101d40 2024-06-10T00:56:24.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.596+0000 7f910dffb700 1 -- 172.21.15.37:0/1299590073 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90f8011d10 con 0x7f9110101d40 2024-06-10T00:56:24.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.596+0000 7f91168fc700 1 -- 172.21.15.37:0/1299590073 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91101a01d0 con 0x7f9110101d40 2024-06-10T00:56:24.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.596+0000 7f910dffb700 1 -- 172.21.15.37:0/1299590073 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f800f2e0 con 0x7f9110101d40 2024-06-10T00:56:24.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.596+0000 7f91168fc700 1 -- 172.21.15.37:0/1299590073 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91101a7a50 con 0x7f9110101d40 2024-06-10T00:56:24.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.599+0000 7f910dffb700 1 -- 172.21.15.37:0/1299590073 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f90f8018020 con 0x7f9110101d40 2024-06-10T00:56:24.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.599+0000 7f910dffb700 1 --2- 172.21.15.37:0/1299590073 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f90fc0a2c20 0x7f90fc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:24.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.599+0000 7f910ffff700 1 --2- 172.21.15.37:0/1299590073 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f90fc0a2c20 0x7f90fc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:24.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.599+0000 7f910dffb700 1 -- 172.21.15.37:0/1299590073 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f90f80cecf0 con 0x7f9110101d40 2024-06-10T00:56:24.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.600+0000 7f910ffff700 1 --2- 172.21.15.37:0/1299590073 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f90fc0a2c20 0x7f90fc0a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9100006fd0 tx=0x7f9100008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:24.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.600+0000 7f912117fb80 1 Processor -- start 2024-06-10T00:56:24.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.600+0000 7f912117fb80 1 -- start start 2024-06-10T00:56:24.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.600+0000 7f912117fb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5376c25f0 0x55f5376c29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:24.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.600+0000 7f912117fb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5376c2f10 0x55f5376bc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:24.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.600+0000 7f912117fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5376bced0 0x55f5376bd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:24.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.600+0000 7f912117fb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f53760e7c0 con 0x55f5376c2f10 2024-06-10T00:56:24.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.600+0000 7f912117fb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f5375b66d0 con 0x55f5376c25f0 2024-06-10T00:56:24.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.600+0000 7f912117fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f53760a780 con 0x55f5376bced0 2024-06-10T00:56:24.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.600+0000 7f9107fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5376c2f10 0x55f5376bc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:24.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.600+0000 7f9107fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5376c2f10 0x55f5376bc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50314/0 (socket says 172.21.15.37:50314) 2024-06-10T00:56:24.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.601+0000 7f9107fff700 1 -- 172.21.15.37:0/1030277076 learned_addr learned my addr 172.21.15.37:0/1030277076 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:24.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.601+0000 7f910ffff700 1 --2- 172.21.15.37:0/1030277076 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5376c25f0 0x55f5376c29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:24.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.601+0000 7f9114e99700 1 --2- 172.21.15.37:0/1030277076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5376bced0 0x55f5376bd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:24.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.601+0000 7f9107fff700 1 -- 172.21.15.37:0/1030277076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5376bced0 msgr2=0x55f5376bd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:24.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.601+0000 7f9107fff700 1 --2- 172.21.15.37:0/1030277076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5376bced0 0x55f5376bd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.601+0000 7f9107fff700 1 -- 172.21.15.37:0/1030277076 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5376c25f0 msgr2=0x55f5376c29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:24.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.601+0000 7f9107fff700 1 --2- 172.21.15.37:0/1030277076 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5376c25f0 0x55f5376c29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.601+0000 7f9107fff700 1 -- 172.21.15.37:0/1030277076 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5375acf00 con 0x55f5376c2f10 2024-06-10T00:56:24.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.601+0000 7f910ffff700 1 --2- 172.21.15.37:0/1030277076 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5376c25f0 0x55f5376c29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:56:24.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.601+0000 7f9107fff700 1 --2- 172.21.15.37:0/1030277076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5376c2f10 0x55f5376bc990 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f90f80975c0 tx=0x7f90f809b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33df5ba74a77ed6d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:24.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.601+0000 7f91067fc700 1 -- 172.21.15.37:0/1030277076 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f80a3dd0 con 0x55f5376c2f10 2024-06-10T00:56:24.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.601+0000 7f91067fc700 1 -- 172.21.15.37:0/1030277076 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90f8010380 con 0x55f5376c2f10 2024-06-10T00:56:24.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.601+0000 7f91067fc700 1 -- 172.21.15.37:0/1030277076 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f809b3c0 con 0x55f5376c2f10 2024-06-10T00:56:24.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.602+0000 7f912117fb80 1 -- 172.21.15.37:0/1030277076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5376c2f10 msgr2=0x55f5376bc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:24.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.602+0000 7f912117fb80 1 --2- 172.21.15.37:0/1030277076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5376c2f10 0x55f5376bc990 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f90f80975c0 tx=0x7f90f809b860 comp rx=0 tx=0).stop 2024-06-10T00:56:24.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.602+0000 7f912117fb80 1 -- 172.21.15.37:0/1030277076 shutdown_connections 2024-06-10T00:56:24.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.602+0000 7f912117fb80 1 --2- 172.21.15.37:0/1030277076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5376bced0 0x55f5376bd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.602+0000 7f912117fb80 1 --2- 172.21.15.37:0/1030277076 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5376c25f0 0x55f5376c29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.602+0000 7f912117fb80 1 --2- 172.21.15.37:0/1030277076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5376c2f10 0x55f5376bc990 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.602+0000 7f912117fb80 1 -- 172.21.15.37:0/1030277076 >> 172.21.15.37:0/1030277076 conn(0x55f5375e7c70 msgr2=0x55f5376b7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:24.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.602+0000 7f912117fb80 1 -- 172.21.15.37:0/1030277076 shutdown_connections 2024-06-10T00:56:24.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.602+0000 7f912117fb80 1 -- 172.21.15.37:0/1030277076 wait complete. 2024-06-10T00:56:24.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.602+0000 7f912117fb80 1 -- start start 2024-06-10T00:56:24.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.602+0000 7f912117fb80 1 Processor -- start 2024-06-10T00:56:24.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.603+0000 7f912117fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5376bced0 0x55f5376c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:24.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.603+0000 7f912117fb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5376c25f0 0x55f5376c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:24.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.603+0000 7f912117fb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5376c2f10 0x55f5376c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:24.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.603+0000 7f912117fb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f5376c9920 con 0x55f5376c25f0 2024-06-10T00:56:24.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.603+0000 7f912117fb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f5375aeba0 con 0x55f5376c2f10 2024-06-10T00:56:24.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.603+0000 7f912117fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f5376c9aa0 con 0x55f5376bced0 2024-06-10T00:56:24.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.603+0000 7f912117fb80 10 client.0 _ll_register_callbacks cb 0x55f5375a88d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:56:24.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.603+0000 7f9107fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5376c25f0 0x55f5376c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:24.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.603+0000 7f9107fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5376c25f0 0x55f5376c5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50326/0 (socket says 172.21.15.37:50326) 2024-06-10T00:56:24.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.603+0000 7f9107fff700 1 -- 172.21.15.37:0/821760882 learned_addr learned my addr 172.21.15.37:0/821760882 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:24.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.603+0000 7f910ffff700 1 --2- 172.21.15.37:0/821760882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5376bced0 0x55f5376c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:24.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.603+0000 7f9114e99700 1 --2- 172.21.15.37:0/821760882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5376c2f10 0x55f5376c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:24.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.603+0000 7f9107fff700 1 -- 172.21.15.37:0/821760882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5376bced0 msgr2=0x55f5376c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:24.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.603+0000 7f9107fff700 1 --2- 172.21.15.37:0/821760882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5376bced0 0x55f5376c8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.603+0000 7f9107fff700 1 -- 172.21.15.37:0/821760882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5376c2f10 msgr2=0x55f5376c64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:24.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.603+0000 7f9107fff700 1 --2- 172.21.15.37:0/821760882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5376c2f10 0x55f5376c64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.603+0000 7f9107fff700 1 -- 172.21.15.37:0/821760882 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f53755d880 con 0x55f5376c25f0 2024-06-10T00:56:24.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.603+0000 7f9107fff700 1 --2- 172.21.15.37:0/821760882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5376c25f0 0x55f5376c5f80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f90f80105c0 tx=0x7f90f809f680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9dbbbcbeaa11ced server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:24.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.603+0000 7f90ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:56:24.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.604+0000 7f90ef7fe700 1 -- 172.21.15.37:0/821760882 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8010690 con 0x55f5376c25f0 2024-06-10T00:56:24.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.604+0000 7f90ef7fe700 1 -- 172.21.15.37:0/821760882 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f90f80bb0b0 con 0x55f5376c25f0 2024-06-10T00:56:24.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.604+0000 7f912117fb80 1 -- 172.21.15.37:0/821760882 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90f8010380 con 0x55f5376c25f0 2024-06-10T00:56:24.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.604+0000 7f90ef7fe700 1 -- 172.21.15.37:0/821760882 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f90f809b3c0 con 0x55f5376c25f0 2024-06-10T00:56:24.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.604+0000 7f912117fb80 10 client.15201 fetch_fsmap learned FSMap version 26 2024-06-10T00:56:24.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.604+0000 7f912117fb80 1 -- 172.21.15.37:0/821760882 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f5376c9920 con 0x55f5376c25f0 2024-06-10T00:56:24.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.604+0000 7f90ef7fe700 1 -- 172.21.15.37:0/821760882 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90f8010380 con 0x55f5376c25f0 2024-06-10T00:56:24.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.604+0000 7f90ef7fe700 1 -- 172.21.15.37:0/821760882 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f90f809a5b0 con 0x55f5376c25f0 2024-06-10T00:56:24.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.605+0000 7f912117fb80 10 client.15201 fetch_fsmap finished waiting for FSMap version 26 2024-06-10T00:56:24.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.605+0000 7f912117fb80 10 client.15201 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T00:56:24.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.605+0000 7f912117fb80 20 client.15201 populate_metadata read hostname 'smithi037' 2024-06-10T00:56:24.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.605+0000 7f912117fb80 1 --2- 172.21.15.37:0/821760882 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f5376d6300 0x55f5376d7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:24.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.605+0000 7f912117fb80 4 client.15201 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:56:24.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.605+0000 7f912117fb80 1 -- 172.21.15.37:0/821760882 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f53761bcc0 con 0x55f5376d6300 2024-06-10T00:56:24.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.605+0000 7f910ffff700 1 --2- 172.21.15.37:0/821760882 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f5376d6300 0x55f5376d7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:24.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.606+0000 7f910ffff700 1 --2- 172.21.15.37:0/821760882 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f5376d6300 0x55f5376d7770 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da75b405e1288c4f server_cookie=a09224b033075cc5 in_seq=0 out_seq=0 2024-06-10T00:56:24.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.606+0000 7f90ef7fe700 10 client.15201 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:56:24.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.606+0000 7f90ef7fe700 1 -- 172.21.15.37:0/821760882 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f53761bcc0 con 0x55f5376d6300 2024-06-10T00:56:24.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.606+0000 7f90ef7fe700 10 client.15201 handle_command_reply: tid=0 2024-06-10T00:56:24.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.606+0000 7f912117fb80 1 client.15201 shutdown 2024-06-10T00:56:24.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.607+0000 7f912117fb80 1 -- 172.21.15.37:0/821760882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5376c25f0 msgr2=0x55f5376c5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:24.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.607+0000 7f912117fb80 1 --2- 172.21.15.37:0/821760882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5376c25f0 0x55f5376c5f80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f90f80105c0 tx=0x7f90f809f680 comp rx=0 tx=0).stop 2024-06-10T00:56:24.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.607+0000 7f912117fb80 1 -- 172.21.15.37:0/821760882 shutdown_connections 2024-06-10T00:56:24.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.607+0000 7f912117fb80 1 --2- 172.21.15.37:0/821760882 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f5376d6300 0x55f5376d7770 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.607+0000 7f90ef7fe700 0 client.15201 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:56:24.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.607+0000 7f912117fb80 1 --2- 172.21.15.37:0/821760882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5376c2f10 0x55f5376c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.607+0000 7f912117fb80 1 --2- 172.21.15.37:0/821760882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5376bced0 0x55f5376c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.607+0000 7f912117fb80 1 --2- 172.21.15.37:0/821760882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5376c25f0 0x55f5376c5f80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.607+0000 7f912117fb80 1 -- 172.21.15.37:0/821760882 >> 172.21.15.37:0/821760882 conn(0x55f5375e7c70 msgr2=0x55f5376b75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:24.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.607+0000 7f912117fb80 1 -- 172.21.15.37:0/821760882 shutdown_connections 2024-06-10T00:56:24.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.607+0000 7f912117fb80 1 -- 172.21.15.37:0/821760882 wait complete. 2024-06-10T00:56:24.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.607+0000 7f912117fb80 20 client.15201 trim_cache size 0 max 16384 2024-06-10T00:56:24.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.629+0000 7f912117fb80 1 Processor -- start 2024-06-10T00:56:24.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.629+0000 7f912117fb80 1 -- start start 2024-06-10T00:56:24.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.629+0000 7f912117fb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5376bced0 0x55f5376bc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:24.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.629+0000 7f912117fb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5376c25f0 0x55f5376c3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:24.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.629+0000 7f912117fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5376c2f10 0x55f5376b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:24.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.629+0000 7f912117fb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f5376d5ab0 con 0x55f5376bced0 2024-06-10T00:56:24.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.629+0000 7f912117fb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f5376d6180 con 0x55f5376c25f0 2024-06-10T00:56:24.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.629+0000 7f912117fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f5376d6850 con 0x55f5376c2f10 2024-06-10T00:56:24.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.629+0000 7f910ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5376bced0 0x55f5376bc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:24.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.629+0000 7f910ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5376bced0 0x55f5376bc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50334/0 (socket says 172.21.15.37:50334) 2024-06-10T00:56:24.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.629+0000 7f910ffff700 1 -- 172.21.15.37:0/2551243857 learned_addr learned my addr 172.21.15.37:0/2551243857 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:24.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.629+0000 7f9114e99700 1 --2- 172.21.15.37:0/2551243857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5376c2f10 0x55f5376b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:24.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.629+0000 7f9107fff700 1 --2- 172.21.15.37:0/2551243857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5376c25f0 0x55f5376c3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:24.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.630+0000 7f910ffff700 1 -- 172.21.15.37:0/2551243857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5376c2f10 msgr2=0x55f5376b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:24.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.630+0000 7f910ffff700 1 --2- 172.21.15.37:0/2551243857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5376c2f10 0x55f5376b0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.630+0000 7f910ffff700 1 -- 172.21.15.37:0/2551243857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5376c25f0 msgr2=0x55f5376c3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:24.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.630+0000 7f910ffff700 1 --2- 172.21.15.37:0/2551243857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5376c25f0 0x55f5376c3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.630+0000 7f910ffff700 1 -- 172.21.15.37:0/2551243857 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5376c4000 con 0x55f5376bced0 2024-06-10T00:56:24.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.630+0000 7f9107fff700 1 --2- 172.21.15.37:0/2551243857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5376c25f0 0x55f5376c3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:56:24.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.630+0000 7f910ffff700 1 --2- 172.21.15.37:0/2551243857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5376bced0 0x55f5376bc990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55f5376c7140 tx=0x7f9100013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35d78decde224b3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:24.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.630+0000 7f9105ffb700 1 -- 172.21.15.37:0/2551243857 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910000d720 con 0x55f5376bced0 2024-06-10T00:56:24.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.630+0000 7f9105ffb700 1 -- 172.21.15.37:0/2551243857 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9100031070 con 0x55f5376bced0 2024-06-10T00:56:24.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.630+0000 7f9105ffb700 1 -- 172.21.15.37:0/2551243857 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910002c8e0 con 0x55f5376bced0 2024-06-10T00:56:24.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.631+0000 7f912117fb80 1 -- 172.21.15.37:0/2551243857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5376bced0 msgr2=0x55f5376bc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:24.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.631+0000 7f912117fb80 1 --2- 172.21.15.37:0/2551243857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5376bced0 0x55f5376bc990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55f5376c7140 tx=0x7f9100013000 comp rx=0 tx=0).stop 2024-06-10T00:56:24.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.631+0000 7f912117fb80 1 -- 172.21.15.37:0/2551243857 shutdown_connections 2024-06-10T00:56:24.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.631+0000 7f912117fb80 1 --2- 172.21.15.37:0/2551243857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5376c2f10 0x55f5376b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.631+0000 7f912117fb80 1 --2- 172.21.15.37:0/2551243857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5376bced0 0x55f5376bc990 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.631+0000 7f912117fb80 1 --2- 172.21.15.37:0/2551243857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5376c25f0 0x55f5376c3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.631+0000 7f912117fb80 1 -- 172.21.15.37:0/2551243857 >> 172.21.15.37:0/2551243857 conn(0x55f5375e7c70 msgr2=0x55f5376c33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:24.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.631+0000 7f912117fb80 1 -- 172.21.15.37:0/2551243857 shutdown_connections 2024-06-10T00:56:24.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.631+0000 7f912117fb80 1 -- 172.21.15.37:0/2551243857 wait complete. 2024-06-10T00:56:24.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.631+0000 7f912117fb80 1 -- start start 2024-06-10T00:56:24.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.631+0000 7f912117fb80 1 Processor -- start 2024-06-10T00:56:24.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.632+0000 7f912117fb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5376bced0 0x55f5376e1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:24.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.632+0000 7f912117fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5376c25f0 0x55f5376bbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:24.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.632+0000 7f912117fb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5376c2f10 0x55f5376bc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:24.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.632+0000 7f912117fb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f5376d75f0 con 0x55f5376c2f10 2024-06-10T00:56:24.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.632+0000 7f912117fb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f5376d6f20 con 0x55f5376bced0 2024-06-10T00:56:24.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.632+0000 7f912117fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f5376d7cc0 con 0x55f5376c25f0 2024-06-10T00:56:24.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.632+0000 7f912117fb80 10 client.0 _ll_register_callbacks cb 0x55f5375a88d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:56:24.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.632+0000 7f9114e99700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5376c2f10 0x55f5376bc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:24.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.632+0000 7f9114e99700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5376c2f10 0x55f5376bc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50344/0 (socket says 172.21.15.37:50344) 2024-06-10T00:56:24.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.632+0000 7f9114e99700 1 -- 172.21.15.37:0/499299500 learned_addr learned my addr 172.21.15.37:0/499299500 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:24.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.632+0000 7f9107fff700 1 --2- 172.21.15.37:0/499299500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5376c25f0 0x55f5376bbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:24.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.632+0000 7f910ffff700 1 --2- 172.21.15.37:0/499299500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5376bced0 0x55f5376e1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:24.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.632+0000 7f9114e99700 1 -- 172.21.15.37:0/499299500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5376c25f0 msgr2=0x55f5376bbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:24.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.632+0000 7f9114e99700 1 --2- 172.21.15.37:0/499299500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5376c25f0 0x55f5376bbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.632+0000 7f9114e99700 1 -- 172.21.15.37:0/499299500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5376bced0 msgr2=0x55f5376e1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:24.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.632+0000 7f9114e99700 1 --2- 172.21.15.37:0/499299500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5376bced0 0x55f5376e1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.632+0000 7f9114e99700 1 -- 172.21.15.37:0/499299500 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5376def90 con 0x55f5376c2f10 2024-06-10T00:56:24.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.632+0000 7f9114e99700 1 --2- 172.21.15.37:0/499299500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5376c2f10 0x55f5376bc240 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9108003c30 tx=0x7f9108003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24710ab2e12717c6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:24.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.632+0000 7f90ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:56:24.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.633+0000 7f90ef7fe700 1 -- 172.21.15.37:0/499299500 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9108029060 con 0x55f5376c2f10 2024-06-10T00:56:24.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.633+0000 7f90ef7fe700 1 -- 172.21.15.37:0/499299500 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f91080074e0 con 0x55f5376c2f10 2024-06-10T00:56:24.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.633+0000 7f912117fb80 1 -- 172.21.15.37:0/499299500 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9100031070 con 0x55f5376c2f10 2024-06-10T00:56:24.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.633+0000 7f90ef7fe700 1 -- 172.21.15.37:0/499299500 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f91080076e0 con 0x55f5376c2f10 2024-06-10T00:56:24.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.633+0000 7f912117fb80 10 client.15207 fetch_fsmap learned FSMap version 26 2024-06-10T00:56:24.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.633+0000 7f912117fb80 1 -- 172.21.15.37:0/499299500 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f537741d70 con 0x55f5376c2f10 2024-06-10T00:56:24.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.633+0000 7f90ef7fe700 1 -- 172.21.15.37:0/499299500 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9100031070 con 0x55f5376c2f10 2024-06-10T00:56:24.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.633+0000 7f90ef7fe700 1 -- 172.21.15.37:0/499299500 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f9108023070 con 0x55f5376c2f10 2024-06-10T00:56:24.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.633+0000 7f912117fb80 10 client.15207 fetch_fsmap finished waiting for FSMap version 26 2024-06-10T00:56:24.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.633+0000 7f912117fb80 10 client.15207 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T00:56:24.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.633+0000 7f912117fb80 20 client.15207 populate_metadata read hostname 'smithi037' 2024-06-10T00:56:24.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.633+0000 7f912117fb80 1 --2- 172.21.15.37:0/499299500 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f537746920 0x55f537745630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:24.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.633+0000 7f912117fb80 4 client.15207 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T00:56:24.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.633+0000 7f912117fb80 1 -- 172.21.15.37:0/499299500 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "status"}) -- 0x55f537616240 con 0x55f537746920 2024-06-10T00:56:24.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.634+0000 7f910ffff700 1 --2- 172.21.15.37:0/499299500 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f537746920 0x55f537745630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:24.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.634+0000 7f90ef7fe700 10 client.15207 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:56:24.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.634+0000 7f910ffff700 1 --2- 172.21.15.37:0/499299500 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f537746920 0x55f537745630 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f51fd2e62422760e server_cookie=28a11e0e30f50a6 in_seq=0 out_seq=0 2024-06-10T00:56:24.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.634+0000 7f90ef7fe700 1 -- 172.21.15.37:0/499299500 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55f537616240 con 0x55f537746920 2024-06-10T00:56:24.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.634+0000 7f90ef7fe700 10 client.15207 handle_command_reply: tid=0 2024-06-10T00:56:24.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.634+0000 7f912117fb80 1 client.15207 shutdown 2024-06-10T00:56:24.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.634+0000 7f912117fb80 1 -- 172.21.15.37:0/499299500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5376c2f10 msgr2=0x55f5376bc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:24.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.634+0000 7f912117fb80 1 --2- 172.21.15.37:0/499299500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5376c2f10 0x55f5376bc240 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9108003c30 tx=0x7f9108003c60 comp rx=0 tx=0).stop 2024-06-10T00:56:24.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.634+0000 7f912117fb80 1 -- 172.21.15.37:0/499299500 shutdown_connections 2024-06-10T00:56:24.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.634+0000 7f912117fb80 1 --2- 172.21.15.37:0/499299500 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f537746920 0x55f537745630 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.634+0000 7f912117fb80 1 --2- 172.21.15.37:0/499299500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5376c2f10 0x55f5376bc240 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.635+0000 7f912117fb80 1 --2- 172.21.15.37:0/499299500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5376bced0 0x55f5376e1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.635+0000 7f912117fb80 1 --2- 172.21.15.37:0/499299500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5376c25f0 0x55f5376bbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.635+0000 7f90ef7fe700 0 client.15207 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:56:24.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.635+0000 7f912117fb80 1 -- 172.21.15.37:0/499299500 >> 172.21.15.37:0/499299500 conn(0x55f5375e7c70 msgr2=0x55f5376b0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:24.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.635+0000 7f912117fb80 1 -- 172.21.15.37:0/499299500 shutdown_connections 2024-06-10T00:56:24.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.635+0000 7f912117fb80 1 -- 172.21.15.37:0/499299500 wait complete. 2024-06-10T00:56:24.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.635+0000 7f912117fb80 20 client.15207 trim_cache size 0 max 16384 2024-06-10T00:56:24.640 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T00:56:24.640 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "68793514-26c2-11ef-bc9b-c7b262605968", 2024-06-10T00:56:24.640 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-06-10T00:56:24.640 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24472, 2024-06-10T00:56:24.641 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-06-10T00:56:24.641 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-06-10T00:56:24.641 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-06-10T00:56:24.641 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 226.68598864200001, 2024-06-10T00:56:24.641 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-06-10T00:56:24.641 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 100, 2024-06-10T00:56:24.641 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 98, 2024-06-10T00:56:24.641 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 254.33537259900001 2024-06-10T00:56:24.641 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T00:56:24.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.635+0000 7f91168fc700 1 -- 172.21.15.37:0/1299590073 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f90fc0a2c20 msgr2=0x7f90fc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:24.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.635+0000 7f91168fc700 1 --2- 172.21.15.37:0/1299590073 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f90fc0a2c20 0x7f90fc0a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9100006fd0 tx=0x7f9100008040 comp rx=0 tx=0).stop 2024-06-10T00:56:24.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.635+0000 7f91168fc700 1 -- 172.21.15.37:0/1299590073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9110101d40 msgr2=0x7f911019b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:24.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.635+0000 7f91168fc700 1 --2- 172.21.15.37:0/1299590073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9110101d40 0x7f911019b410 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f90f800b530 tx=0x7f90f800b840 comp rx=0 tx=0).stop 2024-06-10T00:56:24.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.635+0000 7f91168fc700 1 -- 172.21.15.37:0/1299590073 shutdown_connections 2024-06-10T00:56:24.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.635+0000 7f91168fc700 1 --2- 172.21.15.37:0/1299590073 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f90fc0a2c20 0x7f90fc0a50e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.635+0000 7f91168fc700 1 --2- 172.21.15.37:0/1299590073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91101073f0 0x7f911019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.635+0000 7f91168fc700 1 --2- 172.21.15.37:0/1299590073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9110101380 0x7f911019aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.635+0000 7f91168fc700 1 --2- 172.21.15.37:0/1299590073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9110101d40 0x7f911019b410 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:24.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.635+0000 7f91168fc700 1 -- 172.21.15.37:0/1299590073 >> 172.21.15.37:0/1299590073 conn(0x7f9110078300 msgr2=0x7f911006aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:24.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.635+0000 7f91168fc700 1 -- 172.21.15.37:0/1299590073 shutdown_connections 2024-06-10T00:56:24.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:24.636+0000 7f91168fc700 1 -- 172.21.15.37:0/1299590073 wait complete. 2024-06-10T00:56:25.326 INFO:teuthology.run_tasks:Running task sequential... 2024-06-10T00:56:25.339 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-06-10T00:56:25.340 INFO:teuthology.task.sequential:In sequential, running task print... 2024-06-10T00:56:25.340 INFO:teuthology.task.print:Enabling mgr modules 2024-06-10T00:56:25.340 INFO:teuthology.run_tasks:Running task kclient... 2024-06-10T00:56:25.373 INFO:tasks.kclient:Mounting kernel clients... 2024-06-10T00:56:25.374 INFO:tasks.kclient:config is {'client.0': {}} 2024-06-10T00:56:25.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:56:25.374 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-06-10T00:56:25.393 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-06-10T00:56:25.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:56:25.394 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-10T00:56:25.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:56:25.464 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-06-10T00:56:25.559 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-06-10T00:56:25.559 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-10T00:56:25.559 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-10T00:56:25.559 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-06-10T00:56:25.813 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:56:26.187 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:25 smithi037 ceph-mon[32192]: pgmap v422: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T00:56:26.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:25 smithi167 ceph-mon[36917]: pgmap v422: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T00:56:26.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:25 smithi094 ceph-mon[36708]: pgmap v422: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-10T00:56:27.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.517+0000 7fc74975e700 1 -- 172.21.15.37:0/427311914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc74406a170 msgr2=0x7fc74406a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:27.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.517+0000 7fc74975e700 1 --2- 172.21.15.37:0/427311914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc74406a170 0x7fc74406a5f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc738009990 tx=0x7fc738009ca0 comp rx=0 tx=0).stop 2024-06-10T00:56:27.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.517+0000 7fc74975e700 1 -- 172.21.15.37:0/427311914 shutdown_connections 2024-06-10T00:56:27.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.517+0000 7fc74975e700 1 --2- 172.21.15.37:0/427311914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc74406ab30 0x7fc74410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:27.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.517+0000 7fc74975e700 1 --2- 172.21.15.37:0/427311914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc744108500 0x7fc7441088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:27.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.517+0000 7fc74975e700 1 --2- 172.21.15.37:0/427311914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc74406a170 0x7fc74406a5f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:27.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.517+0000 7fc74975e700 1 -- 172.21.15.37:0/427311914 >> 172.21.15.37:0/427311914 conn(0x7fc744078340 msgr2=0x7fc744078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:27.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.517+0000 7fc74975e700 1 -- 172.21.15.37:0/427311914 shutdown_connections 2024-06-10T00:56:27.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.517+0000 7fc74975e700 1 -- 172.21.15.37:0/427311914 wait complete. 2024-06-10T00:56:27.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.518+0000 7fc74975e700 1 Processor -- start 2024-06-10T00:56:27.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.518+0000 7fc74975e700 1 -- start start 2024-06-10T00:56:27.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.518+0000 7fc74975e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc74406a170 0x7fc74419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:27.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.518+0000 7fc74975e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc74406ab30 0x7fc74419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:27.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.518+0000 7fc74975e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc744108500 0x7fc744198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:27.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.518+0000 7fc74975e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc7441127e0 con 0x7fc74406a170 2024-06-10T00:56:27.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.518+0000 7fc74975e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc744112660 con 0x7fc74406ab30 2024-06-10T00:56:27.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.518+0000 7fc74975e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc744112960 con 0x7fc744108500 2024-06-10T00:56:27.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.518+0000 7fc742ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc74406a170 0x7fc74419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:27.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.518+0000 7fc742ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc74406a170 0x7fc74419e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50366/0 (socket says 172.21.15.37:50366) 2024-06-10T00:56:27.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.518+0000 7fc742ffd700 1 -- 172.21.15.37:0/4079559890 learned_addr learned my addr 172.21.15.37:0/4079559890 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:27.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.519+0000 7fc7437fe700 1 --2- 172.21.15.37:0/4079559890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc744108500 0x7fc744198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:27.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.519+0000 7fc7427fc700 1 --2- 172.21.15.37:0/4079559890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc74406ab30 0x7fc74419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:27.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.519+0000 7fc742ffd700 1 -- 172.21.15.37:0/4079559890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc744108500 msgr2=0x7fc744198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:27.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.519+0000 7fc742ffd700 1 --2- 172.21.15.37:0/4079559890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc744108500 0x7fc744198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:27.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.519+0000 7fc742ffd700 1 -- 172.21.15.37:0/4079559890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc74406ab30 msgr2=0x7fc74419ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:27.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.519+0000 7fc742ffd700 1 --2- 172.21.15.37:0/4079559890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc74406ab30 0x7fc74419ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:27.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.519+0000 7fc742ffd700 1 -- 172.21.15.37:0/4079559890 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc744199200 con 0x7fc74406a170 2024-06-10T00:56:27.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.519+0000 7fc7437fe700 1 --2- 172.21.15.37:0/4079559890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc744108500 0x7fc744198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:56:27.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.519+0000 7fc7427fc700 1 --2- 172.21.15.37:0/4079559890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc74406ab30 0x7fc74419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:56:27.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.519+0000 7fc742ffd700 1 --2- 172.21.15.37:0/4079559890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc74406a170 0x7fc74419e8e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc73400c920 tx=0x7fc73400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:27.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.519+0000 7fc72bfff700 1 -- 172.21.15.37:0/4079559890 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc734007b00 con 0x7fc74406a170 2024-06-10T00:56:27.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.519+0000 7fc74975e700 1 -- 172.21.15.37:0/4079559890 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7441994f0 con 0x7fc74406a170 2024-06-10T00:56:27.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.519+0000 7fc74975e700 1 -- 172.21.15.37:0/4079559890 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7441a57b0 con 0x7fc74406a170 2024-06-10T00:56:27.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.519+0000 7fc72bfff700 1 -- 172.21.15.37:0/4079559890 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc734004d10 con 0x7fc74406a170 2024-06-10T00:56:27.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.519+0000 7fc72bfff700 1 -- 172.21.15.37:0/4079559890 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7340056e0 con 0x7fc74406a170 2024-06-10T00:56:27.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.521+0000 7fc74975e700 1 -- 172.21.15.37:0/4079559890 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc744069180 con 0x7fc74406a170 2024-06-10T00:56:27.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.522+0000 7fc72bfff700 1 -- 172.21.15.37:0/4079559890 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc734007ca0 con 0x7fc74406a170 2024-06-10T00:56:27.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.522+0000 7fc72bfff700 1 --2- 172.21.15.37:0/4079559890 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc72c0a2c20 0x7fc72c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:27.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.522+0000 7fc7427fc700 1 --2- 172.21.15.37:0/4079559890 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc72c0a2c20 0x7fc72c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:27.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.522+0000 7fc72bfff700 1 -- 172.21.15.37:0/4079559890 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fc7340cf0d0 con 0x7fc74406a170 2024-06-10T00:56:27.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.523+0000 7fc7427fc700 1 --2- 172.21.15.37:0/4079559890 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc72c0a2c20 0x7fc72c0a50e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc738006010 tx=0x7fc73800b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:27.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.527+0000 7fc72bfff700 1 -- 172.21.15.37:0/4079559890 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc734014030 con 0x7fc74406a170 2024-06-10T00:56:27.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.781+0000 7fc74975e700 1 -- 172.21.15.37:0/4079559890 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc74402dca0 con 0x7fc74406a170 2024-06-10T00:56:27.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.782+0000 7fc72bfff700 1 -- 172.21.15.37:0/4079559890 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+88 (secure 0 0 0) 0x7fc73409bc40 con 0x7fc74406a170 2024-06-10T00:56:27.782 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-06-10T00:56:27.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.784+0000 7fc74975e700 1 -- 172.21.15.37:0/4079559890 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc72c0a2c20 msgr2=0x7fc72c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:27.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.784+0000 7fc74975e700 1 --2- 172.21.15.37:0/4079559890 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc72c0a2c20 0x7fc72c0a50e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc738006010 tx=0x7fc73800b540 comp rx=0 tx=0).stop 2024-06-10T00:56:27.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.784+0000 7fc74975e700 1 -- 172.21.15.37:0/4079559890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc74406a170 msgr2=0x7fc74419e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:27.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.784+0000 7fc74975e700 1 --2- 172.21.15.37:0/4079559890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc74406a170 0x7fc74419e8e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc73400c920 tx=0x7fc73400cce0 comp rx=0 tx=0).stop 2024-06-10T00:56:27.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.784+0000 7fc74975e700 1 -- 172.21.15.37:0/4079559890 shutdown_connections 2024-06-10T00:56:27.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.784+0000 7fc74975e700 1 --2- 172.21.15.37:0/4079559890 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc72c0a2c20 0x7fc72c0a50e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:27.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.784+0000 7fc74975e700 1 --2- 172.21.15.37:0/4079559890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc744108500 0x7fc744198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:27.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.784+0000 7fc74975e700 1 --2- 172.21.15.37:0/4079559890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc74406a170 0x7fc74419e8e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:27.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.784+0000 7fc74975e700 1 --2- 172.21.15.37:0/4079559890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc74406ab30 0x7fc74419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:27.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.784+0000 7fc74975e700 1 -- 172.21.15.37:0/4079559890 >> 172.21.15.37:0/4079559890 conn(0x7fc744078340 msgr2=0x7fc7440fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:27.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.784+0000 7fc74975e700 1 -- 172.21.15.37:0/4079559890 shutdown_connections 2024-06-10T00:56:27.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:27.784+0000 7fc74975e700 1 -- 172.21.15.37:0/4079559890 wait complete. 2024-06-10T00:56:28.119 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:27 smithi037 ceph-mon[32192]: pgmap v423: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:56:28.120 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:27 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/4079559890' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-10T00:56:28.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:27 smithi167 ceph-mon[36917]: pgmap v423: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:56:28.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:27 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/4079559890' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-10T00:56:28.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:27 smithi094 ceph-mon[36708]: pgmap v423: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-10T00:56:28.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:27 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/4079559890' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-10T00:56:28.421 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-06-10T00:56:28.421 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-10T00:56:28.421 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-06-10T00:56:28.422 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-06-10T00:56:28.422 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-10T00:56:28.422 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-06-10T00:56:28.422 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-10T00:56:28.422 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-10T00:56:28.422 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-06-10T00:56:28.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:56:28.422 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-06-10T00:56:28.441 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-10T00:56:28.442 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-10T00:56:28.442 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-06-10T00:56:28.442 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-06-10T00:56:28.442 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-06-10T00:56:28.442 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-06-10T00:56:28.442 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-10T00:56:28.442 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-06-10T00:56:28.442 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-06-10T00:56:28.442 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-10T00:56:28.442 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-06-10T00:56:28.442 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-06-10T00:56:28.443 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-10T00:56:28.443 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-06-10T00:56:28.443 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-06-10T00:56:28.443 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-06-10T00:56:28.443 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 41998sec preferred_lft 41998sec 2024-06-10T00:56:28.443 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link noprefixroute 2024-06-10T00:56:28.443 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-06-10T00:56:28.443 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-10T00:56:28.443 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-06-10T00:56:28.443 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-06-10T00:56:28.443 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-10T00:56:28.444 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T00:56:28.444 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-06-10T00:56:28.444 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add name ceph-brx type bridge 2024-06-10T00:56:28.444 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr flush dev ceph-brx 2024-06-10T00:56:28.444 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set ceph-brx up 2024-06-10T00:56:28.444 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-10T00:56:28.444 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-06-10T00:56:28.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T00:56:28.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:56:28.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:56:28.596 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-10T00:56:28.621 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-06-10T00:56:28.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:56:28.623 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-06-10T00:56:28.681 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-06-10T00:56:28.681 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-10T00:56:28.681 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-06-10T00:56:28.682 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-06-10T00:56:28.682 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-10T00:56:28.682 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T00:56:28.682 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-06-10T00:56:28.682 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-06-10T00:56:28.682 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-06-10T00:56:28.682 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-06-10T00:56:28.682 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-06-10T00:56:28.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T00:56:28.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:56:28.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:56:28.811 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-06-10T00:56:28.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:56:28.828 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list-id 2024-06-10T00:56:28.884 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T00:56:28.884 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-06-10T00:56:28.884 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-10T00:56:28.884 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-06-10T00:56:28.884 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-06-10T00:56:28.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T00:56:28.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:56:29.005 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-06-10T00:56:29.005 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T00:56:29.006 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-06-10T00:56:29.006 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-06-10T00:56:29.006 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-10T00:56:29.006 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-06-10T00:56:29.006 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-06-10T00:56:29.006 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-06-10T00:56:29.006 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-06-10T00:56:29.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T00:56:29.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:56:29.132 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-10T00:56:29.133 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-06-10T00:56:29.133 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-06-10T00:56:29.133 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set dev brx.0 master ceph-brx 2024-06-10T00:56:29.133 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-06-10T00:56:29.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T00:56:29.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:56:29.212 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-10T00:56:29.212 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-06-10T00:56:29.228 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-06-10T00:56:29.228 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-10T00:56:29.228 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-06-10T00:56:29.284 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-06-10T00:56:29.284 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-06-10T00:56:29.284 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/f3e8e6e7-e4b5-44c3-b060-dff807fa8df4 2024-06-10T00:56:29.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:56:29.284 DEBUG:teuthology.orchestra.run.smithi037:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/f3e8e6e7-e4b5-44c3-b060-dff807fa8df4 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-06-10T00:56:29.368 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-06-10T00:56:29.369 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-06-10T00:56:29.369 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-06-10T00:56:29.520 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-06-10T00:56:29.520 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-06-10T00:56:29.520 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-06-10T00:56:29.521 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.94:3300,172.21.15.167:3300" 2024-06-10T00:56:29.521 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.37:3300,172.21.15.94:3300,172.21.15.167:3300:/volumes/qa/sv_1/f3e8e6e7-e4b5-44c3-b060-dff807fa8df4 2024-06-10T00:56:29.521 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,key=0,fsid=68793514-26c2-11ef-bc9b-c7b262605968" will pass to kernel 2024-06-10T00:56:29.521 INFO:teuthology.orchestra.run.smithi037.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-06-10T00:56:29.522 INFO:teuthology.orchestra.run.smithi037.stdout: You just mounted a file system that supports labels which does not 2024-06-10T00:56:29.522 INFO:teuthology.orchestra.run.smithi037.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-06-10T00:56:29.522 INFO:teuthology.orchestra.run.smithi037.stdout: applications will generate AVC messages and not be allowed access to 2024-06-10T00:56:29.522 INFO:teuthology.orchestra.run.smithi037.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-06-10T00:56:29.524 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-06-10T00:56:29.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T00:56:29.524 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-06-10T00:56:29.595 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-10T00:56:29.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-10T00:56:29.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:56:29.694 INFO:teuthology.orchestra.run.smithi037.stdout:client24904 2024-06-10T00:56:29.694 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-06-10T00:56:29.695 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-10T00:56:29.711 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-10T00:56:29.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-10T00:56:29.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:56:29.810 INFO:teuthology.orchestra.run.smithi037.stdout:client24904 2024-06-10T00:56:29.811 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-10T00:56:29.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-10T00:56:29.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:56:29.868 INFO:teuthology.orchestra.run.smithi037.stdout:client24904 2024-06-10T00:56:29.869 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/68793514-26c2-11ef-bc9b-c7b262605968.client24904/status') 2024-06-10T00:56:29.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-10T00:56:29.929 INFO:teuthology.orchestra.run.smithi037.stdout:instance: client.24904 (3)172.21.15.37:0/4225843740 2024-06-10T00:56:29.929 INFO:teuthology.orchestra.run.smithi037.stdout:blocklisted: false 2024-06-10T00:56:29.929 INFO:teuthology.orchestra.run.smithi037.stderr:0+1 records in 2024-06-10T00:56:29.929 INFO:teuthology.orchestra.run.smithi037.stderr:0+1 records out 2024-06-10T00:56:29.929 INFO:teuthology.orchestra.run.smithi037.stderr:71 bytes copied, 2.4157e-05 s, 2.9 MB/s 2024-06-10T00:56:29.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-10T00:56:29.937 INFO:teuthology.run_tasks:Running task exec... 2024-06-10T00:56:29.946 INFO:teuthology.task.exec:Executing custom commands... 2024-06-10T00:56:29.946 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-06-10T00:56:29.947 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-06-10T00:56:30.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.054+0000 7f46f9e7d640 1 -- 172.21.15.37:0/2822646371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46f4102530 msgr2=0x7f46f4104920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:30.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.055+0000 7f46f9e7d640 1 --2- 172.21.15.37:0/2822646371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46f4102530 0x7f46f4104920 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f46e4009900 tx=0x7f46e402a410 comp rx=0 tx=0).stop 2024-06-10T00:56:30.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.055+0000 7f46f9e7d640 1 -- 172.21.15.37:0/2822646371 shutdown_connections 2024-06-10T00:56:30.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.055+0000 7f46f9e7d640 1 --2- 172.21.15.37:0/2822646371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46f4107790 0x7f46f4109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:30.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.055+0000 7f46f9e7d640 1 --2- 172.21.15.37:0/2822646371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46f4104e60 0x7f46f4107250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:30.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.055+0000 7f46f9e7d640 1 --2- 172.21.15.37:0/2822646371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46f4102530 0x7f46f4104920 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:30.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.055+0000 7f46f9e7d640 1 -- 172.21.15.37:0/2822646371 >> 172.21.15.37:0/2822646371 conn(0x7f46f40fbf20 msgr2=0x7f46f40fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:30.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.055+0000 7f46f9e7d640 1 -- 172.21.15.37:0/2822646371 shutdown_connections 2024-06-10T00:56:30.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.055+0000 7f46f9e7d640 1 -- 172.21.15.37:0/2822646371 wait complete. 2024-06-10T00:56:30.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.055+0000 7f46f9e7d640 1 Processor -- start 2024-06-10T00:56:30.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.055+0000 7f46f9e7d640 1 -- start start 2024-06-10T00:56:30.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.056+0000 7f46f9e7d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46f4102530 0x7f46f4197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:30.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.056+0000 7f46f9e7d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46f4104e60 0x7f46f4198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:30.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.056+0000 7f46f9e7d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46f4107790 0x7f46f419f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:30.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.056+0000 7f46f9e7d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f46f410d740 con 0x7f46f4102530 2024-06-10T00:56:30.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.056+0000 7f46f9e7d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f46f410d5c0 con 0x7f46f4104e60 2024-06-10T00:56:30.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.056+0000 7f46f9e7d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f46f410d8c0 con 0x7f46f4107790 2024-06-10T00:56:30.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.056+0000 7f46f37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46f4102530 0x7f46f4197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:30.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.056+0000 7f46f37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46f4102530 0x7f46f4197cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50382/0 (socket says 172.21.15.37:50382) 2024-06-10T00:56:30.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.056+0000 7f46f37fe640 1 -- 172.21.15.37:0/3588129861 learned_addr learned my addr 172.21.15.37:0/3588129861 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:30.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.056+0000 7f46f3fff640 1 --2- 172.21.15.37:0/3588129861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46f4107790 0x7f46f419f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:30.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.056+0000 7f46f2ffd640 1 --2- 172.21.15.37:0/3588129861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46f4104e60 0x7f46f4198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:30.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.056+0000 7f46f37fe640 1 -- 172.21.15.37:0/3588129861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46f4107790 msgr2=0x7f46f419f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:30.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.056+0000 7f46f37fe640 1 --2- 172.21.15.37:0/3588129861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46f4107790 0x7f46f419f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:30.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.056+0000 7f46f37fe640 1 -- 172.21.15.37:0/3588129861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46f4104e60 msgr2=0x7f46f4198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:30.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.056+0000 7f46f37fe640 1 --2- 172.21.15.37:0/3588129861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46f4104e60 0x7f46f4198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:30.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.056+0000 7f46f37fe640 1 -- 172.21.15.37:0/3588129861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f419f8c0 con 0x7f46f4102530 2024-06-10T00:56:30.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.056+0000 7f46f37fe640 1 --2- 172.21.15.37:0/3588129861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46f4102530 0x7f46f4197cd0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f46e4009230 tx=0x7f46e4004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:30.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.056+0000 7f46f0ff9640 1 -- 172.21.15.37:0/3588129861 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e403f030 con 0x7f46f4102530 2024-06-10T00:56:30.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.056+0000 7f46f0ff9640 1 -- 172.21.15.37:0/3588129861 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46e4003ce0 con 0x7f46f4102530 2024-06-10T00:56:30.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.056+0000 7f46f9e7d640 1 -- 172.21.15.37:0/3588129861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f419fb50 con 0x7f46f4102530 2024-06-10T00:56:30.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.057+0000 7f46f0ff9640 1 -- 172.21.15.37:0/3588129861 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4035070 con 0x7f46f4102530 2024-06-10T00:56:30.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.057+0000 7f46f9e7d640 1 -- 172.21.15.37:0/3588129861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f41a0040 con 0x7f46f4102530 2024-06-10T00:56:30.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.058+0000 7f46f9e7d640 1 -- 172.21.15.37:0/3588129861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46f40fda10 con 0x7f46f4102530 2024-06-10T00:56:30.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.059+0000 7f46f0ff9640 1 -- 172.21.15.37:0/3588129861 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f46e4042030 con 0x7f46f4102530 2024-06-10T00:56:30.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.059+0000 7f46f0ff9640 1 --2- 172.21.15.37:0/3588129861 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f46dc0a2c30 0x7f46dc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:30.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.060+0000 7f46f2ffd640 1 --2- 172.21.15.37:0/3588129861 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f46dc0a2c30 0x7f46dc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:30.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.060+0000 7f46f0ff9640 1 -- 172.21.15.37:0/3588129861 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f46e40efb40 con 0x7f46f4102530 2024-06-10T00:56:30.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.060+0000 7f46f2ffd640 1 --2- 172.21.15.37:0/3588129861 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f46dc0a2c30 0x7f46dc0a50f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f46f4199180 tx=0x7f46e8008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:30.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.064+0000 7f46f0ff9640 1 -- 172.21.15.37:0/3588129861 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f46e40bc6b0 con 0x7f46f4102530 2024-06-10T00:56:30.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.289+0000 7f46f9e7d640 1 -- 172.21.15.37:0/3588129861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f46f4062640 con 0x7f46f4102530 2024-06-10T00:56:30.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:29 smithi167 ceph-mon[36917]: pgmap v424: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:56:30.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:29 smithi094 ceph-mon[36708]: pgmap v424: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:56:30.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:29 smithi037 ceph-mon[32192]: pgmap v424: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:56:30.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.994+0000 7f46f0ff9640 1 -- 172.21.15.37:0/3588129861 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v27) ==== 118+0+0 (secure 0 0 0) 0x7f46e40bbfb0 con 0x7f46f4102530 2024-06-10T00:56:30.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.997+0000 7f46f9e7d640 1 -- 172.21.15.37:0/3588129861 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f46dc0a2c30 msgr2=0x7f46dc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:30.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.997+0000 7f46f9e7d640 1 --2- 172.21.15.37:0/3588129861 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f46dc0a2c30 0x7f46dc0a50f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f46f4199180 tx=0x7f46e8008040 comp rx=0 tx=0).stop 2024-06-10T00:56:30.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.997+0000 7f46f9e7d640 1 -- 172.21.15.37:0/3588129861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46f4102530 msgr2=0x7f46f4197cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:30.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.997+0000 7f46f9e7d640 1 --2- 172.21.15.37:0/3588129861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46f4102530 0x7f46f4197cd0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f46e4009230 tx=0x7f46e4004430 comp rx=0 tx=0).stop 2024-06-10T00:56:30.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.997+0000 7f46f9e7d640 1 -- 172.21.15.37:0/3588129861 shutdown_connections 2024-06-10T00:56:30.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.997+0000 7f46f9e7d640 1 --2- 172.21.15.37:0/3588129861 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f46dc0a2c30 0x7f46dc0a50f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:30.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.997+0000 7f46f9e7d640 1 --2- 172.21.15.37:0/3588129861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46f4107790 0x7f46f419f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:30.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.997+0000 7f46f9e7d640 1 --2- 172.21.15.37:0/3588129861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46f4104e60 0x7f46f4198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:30.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.997+0000 7f46f9e7d640 1 --2- 172.21.15.37:0/3588129861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46f4102530 0x7f46f4197cd0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:30.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.997+0000 7f46f9e7d640 1 -- 172.21.15.37:0/3588129861 >> 172.21.15.37:0/3588129861 conn(0x7f46f40fbf20 msgr2=0x7f46f4102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:30.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.998+0000 7f46f9e7d640 1 -- 172.21.15.37:0/3588129861 shutdown_connections 2024-06-10T00:56:30.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:30.998+0000 7f46f9e7d640 1 -- 172.21.15.37:0/3588129861 wait complete. 2024-06-10T00:56:31.012 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-06-10T00:56:31.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.125+0000 7f1628cee640 1 -- 172.21.15.37:0/3509822633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1624102bf0 msgr2=0x7f1624103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:31.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.125+0000 7f1628cee640 1 --2- 172.21.15.37:0/3509822633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1624102bf0 0x7f1624103070 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f160c009900 tx=0x7f160c02a410 comp rx=0 tx=0).stop 2024-06-10T00:56:31.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.125+0000 7f1628cee640 1 -- 172.21.15.37:0/3509822633 shutdown_connections 2024-06-10T00:56:31.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.125+0000 7f1628cee640 1 --2- 172.21.15.37:0/3509822633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16241035b0 0x7f1624109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.125+0000 7f1628cee640 1 --2- 172.21.15.37:0/3509822633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1624102bf0 0x7f1624103070 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.125+0000 7f1628cee640 1 --2- 172.21.15.37:0/3509822633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1624101a00 0x7f1624101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.125+0000 7f1628cee640 1 -- 172.21.15.37:0/3509822633 >> 172.21.15.37:0/3509822633 conn(0x7f16240fcf90 msgr2=0x7f16240ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:31.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.125+0000 7f1628cee640 1 -- 172.21.15.37:0/3509822633 shutdown_connections 2024-06-10T00:56:31.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.125+0000 7f1628cee640 1 -- 172.21.15.37:0/3509822633 wait complete. 2024-06-10T00:56:31.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.126+0000 7f1628cee640 1 Processor -- start 2024-06-10T00:56:31.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.126+0000 7f1628cee640 1 -- start start 2024-06-10T00:56:31.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.126+0000 7f1628cee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1624101a00 0x7f1624199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:31.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.126+0000 7f1628cee640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1624102bf0 0x7f162419a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:31.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.126+0000 7f1628cee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16241035b0 0x7f16241a1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:31.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.126+0000 7f1628cee640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f162410d680 con 0x7f1624101a00 2024-06-10T00:56:31.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.126+0000 7f1628cee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f162410d500 con 0x7f16241035b0 2024-06-10T00:56:31.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.126+0000 7f1628cee640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f162410d800 con 0x7f1624102bf0 2024-06-10T00:56:31.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.126+0000 7f162259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1624101a00 0x7f1624199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:31.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.126+0000 7f162259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1624101a00 0x7f1624199e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50406/0 (socket says 172.21.15.37:50406) 2024-06-10T00:56:31.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.126+0000 7f162259c640 1 -- 172.21.15.37:0/841414520 learned_addr learned my addr 172.21.15.37:0/841414520 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:31.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.127+0000 7f1621d9b640 1 --2- 172.21.15.37:0/841414520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1624102bf0 0x7f162419a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:31.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.127+0000 7f1622d9d640 1 --2- 172.21.15.37:0/841414520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16241035b0 0x7f16241a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:31.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.127+0000 7f162259c640 1 -- 172.21.15.37:0/841414520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1624102bf0 msgr2=0x7f162419a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:31.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.127+0000 7f162259c640 1 --2- 172.21.15.37:0/841414520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1624102bf0 0x7f162419a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.127+0000 7f162259c640 1 -- 172.21.15.37:0/841414520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16241035b0 msgr2=0x7f16241a1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:31.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.127+0000 7f162259c640 1 --2- 172.21.15.37:0/841414520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16241035b0 0x7f16241a1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.127+0000 7f162259c640 1 -- 172.21.15.37:0/841414520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16241a1a70 con 0x7f1624101a00 2024-06-10T00:56:31.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.127+0000 7f1622d9d640 1 --2- 172.21.15.37:0/841414520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16241035b0 0x7f16241a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:56:31.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.127+0000 7f162259c640 1 --2- 172.21.15.37:0/841414520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1624101a00 0x7f1624199e50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f161400d950 tx=0x7f161400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:31.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.127+0000 7f161b7fe640 1 -- 172.21.15.37:0/841414520 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1614017070 con 0x7f1624101a00 2024-06-10T00:56:31.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.127+0000 7f1628cee640 1 -- 172.21.15.37:0/841414520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16241a1d60 con 0x7f1624101a00 2024-06-10T00:56:31.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.127+0000 7f161b7fe640 1 -- 172.21.15.37:0/841414520 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1614009c30 con 0x7f1624101a00 2024-06-10T00:56:31.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.127+0000 7f161b7fe640 1 -- 172.21.15.37:0/841414520 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161400f600 con 0x7f1624101a00 2024-06-10T00:56:31.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.127+0000 7f1628cee640 1 -- 172.21.15.37:0/841414520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16241a21d0 con 0x7f1624101a00 2024-06-10T00:56:31.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.128+0000 7f1628cee640 1 -- 172.21.15.37:0/841414520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16241a23a0 con 0x7f1624101a00 2024-06-10T00:56:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.129+0000 7f161b7fe640 1 -- 172.21.15.37:0/841414520 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f161401a070 con 0x7f1624101a00 2024-06-10T00:56:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.130+0000 7f161b7fe640 1 --2- 172.21.15.37:0/841414520 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f16100a2c30 0x7f16100a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.130+0000 7f1621d9b640 1 --2- 172.21.15.37:0/841414520 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f16100a2c30 0x7f16100a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.130+0000 7f161b7fe640 1 -- 172.21.15.37:0/841414520 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f16140ceea0 con 0x7f1624101a00 2024-06-10T00:56:31.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.131+0000 7f1621d9b640 1 --2- 172.21.15.37:0/841414520 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f16100a2c30 0x7f16100a50f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f160c0314e0 tx=0x7f160c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:31.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.134+0000 7f161b7fe640 1 -- 172.21.15.37:0/841414520 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1614009de0 con 0x7f1624101a00 2024-06-10T00:56:31.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:30 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3588129861' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-06-10T00:56:31.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.348+0000 7f1628cee640 1 -- 172.21.15.37:0/841414520 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f16240752f0 con 0x7f16100a2c30 2024-06-10T00:56:31.354 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:30 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3588129861' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-06-10T00:56:31.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.362+0000 7f161b7fe640 1 -- 172.21.15.37:0/841414520 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f16240752f0 con 0x7f16100a2c30 2024-06-10T00:56:31.363 INFO:teuthology.orchestra.run.smithi037.stdout:{} 2024-06-10T00:56:31.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:30 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3588129861' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-06-10T00:56:31.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.364+0000 7f1628cee640 1 -- 172.21.15.37:0/841414520 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f16100a2c30 msgr2=0x7f16100a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:31.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.364+0000 7f1628cee640 1 --2- 172.21.15.37:0/841414520 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f16100a2c30 0x7f16100a50f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f160c0314e0 tx=0x7f160c000bc0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.364+0000 7f1628cee640 1 -- 172.21.15.37:0/841414520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1624101a00 msgr2=0x7f1624199e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:31.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.364+0000 7f1628cee640 1 --2- 172.21.15.37:0/841414520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1624101a00 0x7f1624199e50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f161400d950 tx=0x7f161400de10 comp rx=0 tx=0).stop 2024-06-10T00:56:31.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.364+0000 7f1628cee640 1 -- 172.21.15.37:0/841414520 shutdown_connections 2024-06-10T00:56:31.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.364+0000 7f1628cee640 1 --2- 172.21.15.37:0/841414520 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f16100a2c30 0x7f16100a50f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.364+0000 7f1628cee640 1 --2- 172.21.15.37:0/841414520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16241035b0 0x7f16241a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.364+0000 7f1628cee640 1 --2- 172.21.15.37:0/841414520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1624102bf0 0x7f162419a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.364+0000 7f1628cee640 1 --2- 172.21.15.37:0/841414520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1624101a00 0x7f1624199e50 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.364+0000 7f1628cee640 1 -- 172.21.15.37:0/841414520 >> 172.21.15.37:0/841414520 conn(0x7f16240fcf90 msgr2=0x7f1624073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:31.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.364+0000 7f1628cee640 1 -- 172.21.15.37:0/841414520 shutdown_connections 2024-06-10T00:56:31.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.364+0000 7f1628cee640 1 -- 172.21.15.37:0/841414520 wait complete. 2024-06-10T00:56:31.376 INFO:teuthology.run_tasks:Running task check-counter... 2024-06-10T00:56:31.386 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-06-10T00:56:31.424 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-10T00:56:31.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.529+0000 7f40c8238640 1 -- 172.21.15.37:0/2364292455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40c0102480 msgr2=0x7f40c0104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:31.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.529+0000 7f40c8238640 1 --2- 172.21.15.37:0/2364292455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40c0102480 0x7f40c0104870 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f40bc009900 tx=0x7f40bc02a410 comp rx=0 tx=0).stop 2024-06-10T00:56:31.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.529+0000 7f40c8238640 1 -- 172.21.15.37:0/2364292455 shutdown_connections 2024-06-10T00:56:31.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.529+0000 7f40c8238640 1 --2- 172.21.15.37:0/2364292455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40c01076e0 0x7f40c0109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.529+0000 7f40c8238640 1 --2- 172.21.15.37:0/2364292455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40c0104db0 0x7f40c01071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.529+0000 7f40c8238640 1 --2- 172.21.15.37:0/2364292455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40c0102480 0x7f40c0104870 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.529+0000 7f40c8238640 1 -- 172.21.15.37:0/2364292455 >> 172.21.15.37:0/2364292455 conn(0x7f40c00fbe90 msgr2=0x7f40c00fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:31.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.529+0000 7f40c8238640 1 -- 172.21.15.37:0/2364292455 shutdown_connections 2024-06-10T00:56:31.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.529+0000 7f40c8238640 1 -- 172.21.15.37:0/2364292455 wait complete. 2024-06-10T00:56:31.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.530+0000 7f40c8238640 1 Processor -- start 2024-06-10T00:56:31.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.530+0000 7f40c8238640 1 -- start start 2024-06-10T00:56:31.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.530+0000 7f40c8238640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40c0102480 0x7f40c0197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:31.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.530+0000 7f40c8238640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40c0104db0 0x7f40c01982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:31.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.530+0000 7f40c8238640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40c01076e0 0x7f40c019f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:31.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.530+0000 7f40c8238640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f40c010d680 con 0x7f40c01076e0 2024-06-10T00:56:31.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.530+0000 7f40c8238640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f40c010d500 con 0x7f40c0102480 2024-06-10T00:56:31.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.530+0000 7f40c8238640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f40c010d800 con 0x7f40c0104db0 2024-06-10T00:56:31.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.530+0000 7f40c67d5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40c01076e0 0x7f40c019f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:31.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.530+0000 7f40c67d5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40c01076e0 0x7f40c019f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50450/0 (socket says 172.21.15.37:50450) 2024-06-10T00:56:31.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.530+0000 7f40c67d5640 1 -- 172.21.15.37:0/3917752779 learned_addr learned my addr 172.21.15.37:0/3917752779 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:31.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.530+0000 7f40c5fd4640 1 --2- 172.21.15.37:0/3917752779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40c0102480 0x7f40c0197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:31.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.530+0000 7f40c57d3640 1 --2- 172.21.15.37:0/3917752779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40c0104db0 0x7f40c01982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:31.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.530+0000 7f40c67d5640 1 -- 172.21.15.37:0/3917752779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40c0104db0 msgr2=0x7f40c01982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:31.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.530+0000 7f40c67d5640 1 --2- 172.21.15.37:0/3917752779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40c0104db0 0x7f40c01982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.530+0000 7f40c67d5640 1 -- 172.21.15.37:0/3917752779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40c0102480 msgr2=0x7f40c0197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:31.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.530+0000 7f40c67d5640 1 --2- 172.21.15.37:0/3917752779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40c0102480 0x7f40c0197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.530+0000 7f40c67d5640 1 -- 172.21.15.37:0/3917752779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c019f980 con 0x7f40c01076e0 2024-06-10T00:56:31.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.530+0000 7f40c57d3640 1 --2- 172.21.15.37:0/3917752779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40c0104db0 0x7f40c01982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:56:31.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.531+0000 7f40c67d5640 1 --2- 172.21.15.37:0/3917752779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40c01076e0 0x7f40c019f310 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f40b800eaa0 tx=0x7f40b800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:31.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.531+0000 7f40b6ffd640 1 -- 172.21.15.37:0/3917752779 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b800cdb0 con 0x7f40c01076e0 2024-06-10T00:56:31.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.531+0000 7f40b6ffd640 1 -- 172.21.15.37:0/3917752779 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40b8005020 con 0x7f40c01076e0 2024-06-10T00:56:31.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.531+0000 7f40c8238640 1 -- 172.21.15.37:0/3917752779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40c019fc70 con 0x7f40c01076e0 2024-06-10T00:56:31.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.531+0000 7f40b6ffd640 1 -- 172.21.15.37:0/3917752779 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b80116b0 con 0x7f40c01076e0 2024-06-10T00:56:31.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.531+0000 7f40c8238640 1 -- 172.21.15.37:0/3917752779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40c01a0170 con 0x7f40c01076e0 2024-06-10T00:56:31.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.532+0000 7f40c8238640 1 -- 172.21.15.37:0/3917752779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40c00fd980 con 0x7f40c01076e0 2024-06-10T00:56:31.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.533+0000 7f40b6ffd640 1 -- 172.21.15.37:0/3917752779 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f40b8005560 con 0x7f40c01076e0 2024-06-10T00:56:31.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.534+0000 7f40b6ffd640 1 --2- 172.21.15.37:0/3917752779 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f40ac0a2b90 0x7f40ac0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:31.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.534+0000 7f40c5fd4640 1 --2- 172.21.15.37:0/3917752779 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f40ac0a2b90 0x7f40ac0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:31.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.534+0000 7f40b6ffd640 1 -- 172.21.15.37:0/3917752779 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f40b80cf680 con 0x7f40c01076e0 2024-06-10T00:56:31.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.535+0000 7f40c5fd4640 1 --2- 172.21.15.37:0/3917752779 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f40ac0a2b90 0x7f40ac0a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f40bc0314e0 tx=0x7f40bc000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:31.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.539+0000 7f40b6ffd640 1 -- 172.21.15.37:0/3917752779 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f40b8016030 con 0x7f40c01076e0 2024-06-10T00:56:31.646 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:56:31 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:56:31.353+0000 7fcb31c8c700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:56:31.646 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:56:31 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:56:31.353+0000 7fcb31c8c700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:56:31.646 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:56:31 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:56:31.353+0000 7fcb31c8c700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:56:31.646 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:56:31 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:56:31.353+0000 7fcb31c8c700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:56:31.646 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 00:56:31 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T00:56:31.353+0000 7fcb31c8c700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-10T00:56:31.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.727+0000 7f40c8238640 1 -- 172.21.15.37:0/3917752779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f40c0062640 con 0x7f40c01076e0 2024-06-10T00:56:31.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.728+0000 7f40b6ffd640 1 -- 172.21.15.37:0/3917752779 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24854 (secure 0 0 0) 0x7f40b809c1f0 con 0x7f40c01076e0 2024-06-10T00:56:31.729 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:56:31.729 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":100,"fsid":"68793514-26c2-11ef-bc9b-c7b262605968","created":"2024-06-10T00:44:24.106713+0000","modified":"2024-06-10T00:54:22.546967+0000","last_up_change":"2024-06-10T00:51:31.562156+0000","last_in_change":"2024-06-10T00:51:11.294301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:47:44.773911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:52:31.684780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:52:34.885503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:52:44.415437+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1df6046a-6801-4950-9193-54303fed169b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6803","nonce":3594704094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6805","nonce":3594704094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6809","nonce":3594704094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6807","nonce":3594704094}]},"public_addr":"172.21.15.37:6803/3594704094","cluster_addr":"172.21.15.37:6805/3594704094","heartbeat_back_addr":"172.21.15.37:6809/3594704094","heartbeat_front_addr":"172.21.15.37:6807/3594704094","state":["exists","up"]},{"osd":1,"uuid":"4e1c8944-94e4-4e46-9efe-56229fe6df1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6801","nonce":3427727155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6803","nonce":3427727155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6807","nonce":3427727155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6805","nonce":3427727155}]},"public_addr":"172.21.15.94:6801/3427727155","cluster_addr":"172.21.15.94:6803/3427727155","heartbeat_back_addr":"172.21.15.94:6807/3427727155","heartbeat_front_addr":"172.21.15.94:6805/3427727155","state":["exists","up"]},{"osd":2,"uuid":"fd2fcedc-4c88-478f-8025-13e5944a27ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6801","nonce":4002498292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6803","nonce":4002498292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6807","nonce":4002498292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6805","nonce":4002498292}]},"public_addr":"172.21.15.167:6801/4002498292","cluster_addr":"172.21.15.167:6803/4002498292","heartbeat_back_addr":"172.21.15.167:6807/4002498292","heartbeat_front_addr":"172.21.15.167:6805/4002498292","state":["exists","up"]},{"osd":3,"uuid":"d0153676-818c-4dcc-83b1-287435a0c8a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6811","nonce":3425469324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6813","nonce":3425469324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6817","nonce":3425469324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6815","nonce":3425469324}]},"public_addr":"172.21.15.37:6811/3425469324","cluster_addr":"172.21.15.37:6813/3425469324","heartbeat_back_addr":"172.21.15.37:6817/3425469324","heartbeat_front_addr":"172.21.15.37:6815/3425469324","state":["exists","up"]},{"osd":4,"uuid":"172c333c-00c0-4e94-8870-acb3b32b2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6809","nonce":1740627431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6811","nonce":1740627431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6815","nonce":1740627431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6813","nonce":1740627431}]},"public_addr":"172.21.15.94:6809/1740627431","cluster_addr":"172.21.15.94:6811/1740627431","heartbeat_back_addr":"172.21.15.94:6815/1740627431","heartbeat_front_addr":"172.21.15.94:6813/1740627431","state":["exists","up"]},{"osd":5,"uuid":"dbce656e-5f93-4bda-baf2-fc3cc03d8282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6809","nonce":221805236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6811","nonce":221805236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6815","nonce":221805236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6813","nonce":221805236}]},"public_addr":"172.21.15.167:6809/221805236","cluster_addr":"172.21.15.167:6811/221805236","heartbeat_back_addr":"172.21.15.167:6815/221805236","heartbeat_front_addr":"172.21.15.167:6813/221805236","state":["exists","up"]},{"osd":6,"uuid":"fcbda266-6d60-43b2-a51c-4e562476d1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6819","nonce":472670875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6821","nonce":472670875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6825","nonce":472670875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6823","nonce":472670875}]},"public_addr":"172.21.15.37:6819/472670875","cluster_addr":"172.21.15.37:6821/472670875","heartbeat_back_addr":"172.21.15.37:6825/472670875","heartbeat_front_addr":"172.21.15.37:6823/472670875","state":["exists","up"]},{"osd":7,"uuid":"ed829ed3-ba85-4e82-ac37-e44e9534d37c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6817","nonce":3932281552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6819","nonce":3932281552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6823","nonce":3932281552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6821","nonce":3932281552}]},"public_addr":"172.21.15.94:6817/3932281552","cluster_addr":"172.21.15.94:6819/3932281552","heartbeat_back_addr":"172.21.15.94:6823/3932281552","heartbeat_front_addr":"172.21.15.94:6821/3932281552","state":["exists","up"]},{"osd":8,"uuid":"7d6267d8-72f4-4704-b554-bafc52d11a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6817","nonce":2290216047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6819","nonce":2290216047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6823","nonce":2290216047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6821","nonce":2290216047}]},"public_addr":"172.21.15.167:6817/2290216047","cluster_addr":"172.21.15.167:6819/2290216047","heartbeat_back_addr":"172.21.15.167:6823/2290216047","heartbeat_front_addr":"172.21.15.167:6821/2290216047","state":["exists","up"]},{"osd":9,"uuid":"210fdc20-0a36-4eeb-bdf2-c0a7f130e19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6827","nonce":2063666428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6829","nonce":2063666428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6833","nonce":2063666428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6831","nonce":2063666428}]},"public_addr":"172.21.15.37:6827/2063666428","cluster_addr":"172.21.15.37:6829/2063666428","heartbeat_back_addr":"172.21.15.37:6833/2063666428","heartbeat_front_addr":"172.21.15.37:6831/2063666428","state":["exists","up"]},{"osd":10,"uuid":"e200b371-b3cf-4b92-a659-f155ca86f0a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6825","nonce":1706618899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6827","nonce":1706618899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6831","nonce":1706618899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6829","nonce":1706618899}]},"public_addr":"172.21.15.94:6825/1706618899","cluster_addr":"172.21.15.94:6827/1706618899","heartbeat_back_addr":"172.21.15.94:6831/1706618899","heartbeat_front_addr":"172.21.15.94:6829/1706618899","state":["exists","up"]},{"osd":11,"uuid":"cfa427df-37a5-4d28-ac2e-af777d607008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6825","nonce":1349892516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6827","nonce":1349892516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6831","nonce":1349892516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6829","nonce":1349892516}]},"public_addr":"172.21.15.167:6825/1349892516","cluster_addr":"172.21.15.167:6827/1349892516","heartbeat_back_addr":"172.21.15.167:6831/1349892516","heartbeat_front_addr":"172.21.15.167:6829/1349892516","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:46:51.846147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:15.763723+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:40.106800+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:05.923204+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:30.501883+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:55.565575+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:20.411360+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:46.071776+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:11.854599+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:37.608818+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:03.176767+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:29.768705+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/4084722397":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/3571803585":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/416587443":"2024-06-11T00:45:20.254342+0000","172.21.15.37:6801/2766692263":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/2026344134":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/2781321886":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6801/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/1110143997":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2766692263":"2024-06-11T00:45:20.254342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:56:31.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.730+0000 7f40c8238640 1 -- 172.21.15.37:0/3917752779 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f40ac0a2b90 msgr2=0x7f40ac0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:31.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.730+0000 7f40c8238640 1 --2- 172.21.15.37:0/3917752779 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f40ac0a2b90 0x7f40ac0a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f40bc0314e0 tx=0x7f40bc000bc0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.730+0000 7f40c8238640 1 -- 172.21.15.37:0/3917752779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40c01076e0 msgr2=0x7f40c019f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:31.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.730+0000 7f40c8238640 1 --2- 172.21.15.37:0/3917752779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40c01076e0 0x7f40c019f310 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f40b800eaa0 tx=0x7f40b800ef60 comp rx=0 tx=0).stop 2024-06-10T00:56:31.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.730+0000 7f40c8238640 1 -- 172.21.15.37:0/3917752779 shutdown_connections 2024-06-10T00:56:31.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.730+0000 7f40c8238640 1 --2- 172.21.15.37:0/3917752779 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f40ac0a2b90 0x7f40ac0a5050 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.730+0000 7f40c8238640 1 --2- 172.21.15.37:0/3917752779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40c01076e0 0x7f40c019f310 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.730+0000 7f40c8238640 1 --2- 172.21.15.37:0/3917752779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40c0104db0 0x7f40c01982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.730+0000 7f40c8238640 1 --2- 172.21.15.37:0/3917752779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40c0102480 0x7f40c0197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.730+0000 7f40c8238640 1 -- 172.21.15.37:0/3917752779 >> 172.21.15.37:0/3917752779 conn(0x7f40c00fbe90 msgr2=0x7f40c0102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:31.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.730+0000 7f40c8238640 1 -- 172.21.15.37:0/3917752779 shutdown_connections 2024-06-10T00:56:31.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.730+0000 7f40c8238640 1 -- 172.21.15.37:0/3917752779 wait complete. 2024-06-10T00:56:31.743 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-10T00:47:44.773911+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-10T00:52:31.684780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-10T00:52:34.885503+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-06-10T00:52:44.415437+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-06-10T00:56:31.743 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-10T00:56:31.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.850+0000 7f3dec864640 1 -- 172.21.15.37:0/1773586423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3de8102480 msgr2=0x7f3de8104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:31.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.850+0000 7f3dec864640 1 --2- 172.21.15.37:0/1773586423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3de8102480 0x7f3de8104870 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009900 tx=0x7f3dd802a410 comp rx=0 tx=0).stop 2024-06-10T00:56:31.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.850+0000 7f3dec864640 1 -- 172.21.15.37:0/1773586423 shutdown_connections 2024-06-10T00:56:31.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.850+0000 7f3dec864640 1 --2- 172.21.15.37:0/1773586423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3de81076e0 0x7f3de8109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.850+0000 7f3dec864640 1 --2- 172.21.15.37:0/1773586423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3de8104db0 0x7f3de81071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.850+0000 7f3dec864640 1 --2- 172.21.15.37:0/1773586423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3de8102480 0x7f3de8104870 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.850+0000 7f3dec864640 1 -- 172.21.15.37:0/1773586423 >> 172.21.15.37:0/1773586423 conn(0x7f3de80fbe70 msgr2=0x7f3de80fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:31.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.851+0000 7f3dec864640 1 -- 172.21.15.37:0/1773586423 shutdown_connections 2024-06-10T00:56:31.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.851+0000 7f3dec864640 1 -- 172.21.15.37:0/1773586423 wait complete. 2024-06-10T00:56:31.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.851+0000 7f3dec864640 1 Processor -- start 2024-06-10T00:56:31.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.851+0000 7f3dec864640 1 -- start start 2024-06-10T00:56:31.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.851+0000 7f3dec864640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3de8102480 0x7f3de8197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:31.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.851+0000 7f3dec864640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3de8104db0 0x7f3de8198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:31.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.851+0000 7f3dec864640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3de81076e0 0x7f3de819f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:31.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.851+0000 7f3dec864640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3de810d6b0 con 0x7f3de8104db0 2024-06-10T00:56:31.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.851+0000 7f3dec864640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3de810d530 con 0x7f3de8102480 2024-06-10T00:56:31.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.851+0000 7f3dec864640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3de810d830 con 0x7f3de81076e0 2024-06-10T00:56:31.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.852+0000 7f3de5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3de8104db0 0x7f3de8198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:31.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.852+0000 7f3de5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3de8104db0 0x7f3de8198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47086/0 (socket says 172.21.15.37:47086) 2024-06-10T00:56:31.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.852+0000 7f3de5d9b640 1 -- 172.21.15.37:0/28297353 learned_addr learned my addr 172.21.15.37:0/28297353 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:31.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.852+0000 7f3de659c640 1 --2- 172.21.15.37:0/28297353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3de8102480 0x7f3de8197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:31.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.852+0000 7f3de6d9d640 1 --2- 172.21.15.37:0/28297353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3de81076e0 0x7f3de819f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:31.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.852+0000 7f3de5d9b640 1 -- 172.21.15.37:0/28297353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3de81076e0 msgr2=0x7f3de819f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:31.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.852+0000 7f3de5d9b640 1 --2- 172.21.15.37:0/28297353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3de81076e0 0x7f3de819f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.852+0000 7f3de5d9b640 1 -- 172.21.15.37:0/28297353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3de8102480 msgr2=0x7f3de8197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:31.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.852+0000 7f3de5d9b640 1 --2- 172.21.15.37:0/28297353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3de8102480 0x7f3de8197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:31.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.852+0000 7f3de5d9b640 1 -- 172.21.15.37:0/28297353 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de819f960 con 0x7f3de8104db0 2024-06-10T00:56:31.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.852+0000 7f3de5d9b640 1 --2- 172.21.15.37:0/28297353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3de8104db0 0x7f3de8198290 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00d740 tx=0x7f3ddc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:31.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.852+0000 7f3dd77fe640 1 -- 172.21.15.37:0/28297353 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc017070 con 0x7f3de8104db0 2024-06-10T00:56:31.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.852+0000 7f3dec864640 1 -- 172.21.15.37:0/28297353 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3de819fc50 con 0x7f3de8104db0 2024-06-10T00:56:31.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.852+0000 7f3dd77fe640 1 -- 172.21.15.37:0/28297353 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ddc00de50 con 0x7f3de8104db0 2024-06-10T00:56:31.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.852+0000 7f3dd77fe640 1 -- 172.21.15.37:0/28297353 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc004e00 con 0x7f3de8104db0 2024-06-10T00:56:31.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.852+0000 7f3dec864640 1 -- 172.21.15.37:0/28297353 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3de81a0150 con 0x7f3de8104db0 2024-06-10T00:56:31.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.853+0000 7f3dec864640 1 -- 172.21.15.37:0/28297353 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3de80fd960 con 0x7f3de8104db0 2024-06-10T00:56:31.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.854+0000 7f3dd77fe640 1 -- 172.21.15.37:0/28297353 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3ddc004fa0 con 0x7f3de8104db0 2024-06-10T00:56:31.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.855+0000 7f3dd77fe640 1 --2- 172.21.15.37:0/28297353 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f3dd00a2b90 0x7f3dd00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:31.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.855+0000 7f3dd77fe640 1 -- 172.21.15.37:0/28297353 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f3ddc0cee00 con 0x7f3de8104db0 2024-06-10T00:56:31.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.855+0000 7f3de659c640 1 --2- 172.21.15.37:0/28297353 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f3dd00a2b90 0x7f3dd00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:31.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.856+0000 7f3de659c640 1 --2- 172.21.15.37:0/28297353 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f3dd00a2b90 0x7f3dd00a5050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3de8199220 tx=0x7f3dd8000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:31.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:31.860+0000 7f3dd77fe640 1 -- 172.21.15.37:0/28297353 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3ddc09b320 con 0x7f3de8104db0 2024-06-10T00:56:32.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.045+0000 7f3dec864640 1 -- 172.21.15.37:0/28297353 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3de80625d0 con 0x7f3de8104db0 2024-06-10T00:56:32.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.046+0000 7f3dd77fe640 1 -- 172.21.15.37:0/28297353 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7f3ddc09f620 con 0x7f3de8104db0 2024-06-10T00:56:32.046 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-06-10T00:56:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.048+0000 7f3dec864640 1 -- 172.21.15.37:0/28297353 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f3dd00a2b90 msgr2=0x7f3dd00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.048+0000 7f3dec864640 1 --2- 172.21.15.37:0/28297353 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f3dd00a2b90 0x7f3dd00a5050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3de8199220 tx=0x7f3dd8000bc0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.048+0000 7f3dec864640 1 -- 172.21.15.37:0/28297353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3de8104db0 msgr2=0x7f3de8198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.048+0000 7f3dec864640 1 --2- 172.21.15.37:0/28297353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3de8104db0 0x7f3de8198290 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00d740 tx=0x7f3ddc00dc00 comp rx=0 tx=0).stop 2024-06-10T00:56:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.048+0000 7f3dec864640 1 -- 172.21.15.37:0/28297353 shutdown_connections 2024-06-10T00:56:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.048+0000 7f3dec864640 1 --2- 172.21.15.37:0/28297353 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f3dd00a2b90 0x7f3dd00a5050 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.048+0000 7f3dec864640 1 --2- 172.21.15.37:0/28297353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3de81076e0 0x7f3de819f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.048+0000 7f3dec864640 1 --2- 172.21.15.37:0/28297353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3de8104db0 0x7f3de8198290 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.048+0000 7f3dec864640 1 --2- 172.21.15.37:0/28297353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3de8102480 0x7f3de8197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.048+0000 7f3dec864640 1 -- 172.21.15.37:0/28297353 >> 172.21.15.37:0/28297353 conn(0x7f3de80fbe70 msgr2=0x7f3de8102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.048+0000 7f3dec864640 1 -- 172.21.15.37:0/28297353 shutdown_connections 2024-06-10T00:56:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.048+0000 7f3dec864640 1 -- 172.21.15.37:0/28297353 wait complete. 2024-06-10T00:56:32.060 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-10T00:56:32.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.167+0000 7f05d54fc640 1 -- 172.21.15.37:0/949316718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05d0102480 msgr2=0x7f05d0104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:32.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.167+0000 7f05d54fc640 1 --2- 172.21.15.37:0/949316718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05d0102480 0x7f05d0104870 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009900 tx=0x7f05c002a410 comp rx=0 tx=0).stop 2024-06-10T00:56:32.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.167+0000 7f05d54fc640 1 -- 172.21.15.37:0/949316718 shutdown_connections 2024-06-10T00:56:32.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.167+0000 7f05d54fc640 1 --2- 172.21.15.37:0/949316718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05d01076e0 0x7f05d0109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.167+0000 7f05d54fc640 1 --2- 172.21.15.37:0/949316718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05d0104db0 0x7f05d01071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.167+0000 7f05d54fc640 1 --2- 172.21.15.37:0/949316718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05d0102480 0x7f05d0104870 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.167+0000 7f05d54fc640 1 -- 172.21.15.37:0/949316718 >> 172.21.15.37:0/949316718 conn(0x7f05d00fbe90 msgr2=0x7f05d00fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:32.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.167+0000 7f05d54fc640 1 -- 172.21.15.37:0/949316718 shutdown_connections 2024-06-10T00:56:32.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.167+0000 7f05d54fc640 1 -- 172.21.15.37:0/949316718 wait complete. 2024-06-10T00:56:32.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.168+0000 7f05d54fc640 1 Processor -- start 2024-06-10T00:56:32.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.168+0000 7f05d54fc640 1 -- start start 2024-06-10T00:56:32.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.168+0000 7f05d54fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05d0102480 0x7f05d0197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:32.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.168+0000 7f05d54fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05d0104db0 0x7f05d01982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:32.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.168+0000 7f05d54fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05d01076e0 0x7f05d019f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:32.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.168+0000 7f05d54fc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f05d010d6b0 con 0x7f05d0102480 2024-06-10T00:56:32.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.168+0000 7f05d54fc640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f05d010d530 con 0x7f05d01076e0 2024-06-10T00:56:32.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.168+0000 7f05d54fc640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f05d010d830 con 0x7f05d0104db0 2024-06-10T00:56:32.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.168+0000 7f05ceffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05d0102480 0x7f05d0197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:32.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.168+0000 7f05ceffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05d0102480 0x7f05d0197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47102/0 (socket says 172.21.15.37:47102) 2024-06-10T00:56:32.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.168+0000 7f05ceffd640 1 -- 172.21.15.37:0/3799056635 learned_addr learned my addr 172.21.15.37:0/3799056635 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:32.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.168+0000 7f05ce7fc640 1 --2- 172.21.15.37:0/3799056635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05d0104db0 0x7f05d01982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:32.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.168+0000 7f05ceffd640 1 -- 172.21.15.37:0/3799056635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05d0104db0 msgr2=0x7f05d01982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:32.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.168+0000 7f05ceffd640 1 --2- 172.21.15.37:0/3799056635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05d0104db0 0x7f05d01982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.168+0000 7f05ceffd640 1 -- 172.21.15.37:0/3799056635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05d01076e0 msgr2=0x7f05d019f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:32.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.168+0000 7f05ceffd640 1 --2- 172.21.15.37:0/3799056635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05d01076e0 0x7f05d019f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.168+0000 7f05ceffd640 1 -- 172.21.15.37:0/3799056635 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d019f9d0 con 0x7f05d0102480 2024-06-10T00:56:32.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.168+0000 7f05ceffd640 1 --2- 172.21.15.37:0/3799056635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05d0102480 0x7f05d0197db0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009230 tx=0x7f05c0031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:32.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.169+0000 7f05b7fff640 1 -- 172.21.15.37:0/3799056635 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c003f030 con 0x7f05d0102480 2024-06-10T00:56:32.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.169+0000 7f05b7fff640 1 -- 172.21.15.37:0/3799056635 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05c0003ad0 con 0x7f05d0102480 2024-06-10T00:56:32.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.169+0000 7f05d54fc640 1 -- 172.21.15.37:0/3799056635 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05d019fc60 con 0x7f05d0102480 2024-06-10T00:56:32.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.169+0000 7f05b7fff640 1 -- 172.21.15.37:0/3799056635 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c0035070 con 0x7f05d0102480 2024-06-10T00:56:32.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.169+0000 7f05d54fc640 1 -- 172.21.15.37:0/3799056635 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05d01a01a0 con 0x7f05d0102480 2024-06-10T00:56:32.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.170+0000 7f05d54fc640 1 -- 172.21.15.37:0/3799056635 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05d00fd980 con 0x7f05d0102480 2024-06-10T00:56:32.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.171+0000 7f05b7fff640 1 -- 172.21.15.37:0/3799056635 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f05c0042030 con 0x7f05d0102480 2024-06-10T00:56:32.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.172+0000 7f05b7fff640 1 --2- 172.21.15.37:0/3799056635 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f05b80a2c10 0x7f05b80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:32.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.172+0000 7f05ce7fc640 1 --2- 172.21.15.37:0/3799056635 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f05b80a2c10 0x7f05b80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:32.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.172+0000 7f05b7fff640 1 -- 172.21.15.37:0/3799056635 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f05c00ef690 con 0x7f05d0102480 2024-06-10T00:56:32.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.172+0000 7f05ce7fc640 1 --2- 172.21.15.37:0/3799056635 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f05b80a2c10 0x7f05b80a50d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f05d01992c0 tx=0x7f05c4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:32.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.177+0000 7f05b7fff640 1 -- 172.21.15.37:0/3799056635 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f05c00bc200 con 0x7f05d0102480 2024-06-10T00:56:32.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:31 smithi167 ceph-mon[36917]: pgmap v425: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:56:32.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:31 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3588129861' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-06-10T00:56:32.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:31 smithi167 ceph-mon[36917]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T00:56:32.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:31 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3917752779' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:56:32.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.362+0000 7f05d54fc640 1 -- 172.21.15.37:0/3799056635 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f05d0062640 con 0x7f05d0102480 2024-06-10T00:56:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.363+0000 7f05b7fff640 1 -- 172.21.15.37:0/3799056635 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7f05c00bbb00 con 0x7f05d0102480 2024-06-10T00:56:32.363 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-06-10T00:56:32.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:31 smithi094 ceph-mon[36708]: pgmap v425: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:56:32.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:31 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3588129861' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-06-10T00:56:32.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:31 smithi094 ceph-mon[36708]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T00:56:32.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:31 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3917752779' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:56:32.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.365+0000 7f05d54fc640 1 -- 172.21.15.37:0/3799056635 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f05b80a2c10 msgr2=0x7f05b80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:32.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.365+0000 7f05d54fc640 1 --2- 172.21.15.37:0/3799056635 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f05b80a2c10 0x7f05b80a50d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f05d01992c0 tx=0x7f05c4005f90 comp rx=0 tx=0).stop 2024-06-10T00:56:32.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.365+0000 7f05d54fc640 1 -- 172.21.15.37:0/3799056635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05d0102480 msgr2=0x7f05d0197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:32.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.365+0000 7f05d54fc640 1 --2- 172.21.15.37:0/3799056635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05d0102480 0x7f05d0197db0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009230 tx=0x7f05c0031b60 comp rx=0 tx=0).stop 2024-06-10T00:56:32.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.365+0000 7f05d54fc640 1 -- 172.21.15.37:0/3799056635 shutdown_connections 2024-06-10T00:56:32.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.365+0000 7f05d54fc640 1 --2- 172.21.15.37:0/3799056635 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f05b80a2c10 0x7f05b80a50d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.365+0000 7f05d54fc640 1 --2- 172.21.15.37:0/3799056635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05d01076e0 0x7f05d019f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.365+0000 7f05d54fc640 1 --2- 172.21.15.37:0/3799056635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05d0104db0 0x7f05d01982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.365+0000 7f05d54fc640 1 --2- 172.21.15.37:0/3799056635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05d0102480 0x7f05d0197db0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.365+0000 7f05d54fc640 1 -- 172.21.15.37:0/3799056635 >> 172.21.15.37:0/3799056635 conn(0x7f05d00fbe90 msgr2=0x7f05d0102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:32.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.365+0000 7f05d54fc640 1 -- 172.21.15.37:0/3799056635 shutdown_connections 2024-06-10T00:56:32.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.365+0000 7f05d54fc640 1 -- 172.21.15.37:0/3799056635 wait complete. 2024-06-10T00:56:32.375 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:31 smithi037 ceph-mon[32192]: pgmap v425: 129 pgs: 129 active+clean; 649 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:56:32.376 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:31 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3588129861' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-06-10T00:56:32.376 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:31 smithi037 ceph-mon[32192]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T00:56:32.376 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:31 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3917752779' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:56:32.377 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-10T00:56:32.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.485+0000 7fe40099e640 1 -- 172.21.15.37:0/4011150997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3fc1007b0 msgr2=0x7fe3fc100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:32.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.485+0000 7fe40099e640 1 --2- 172.21.15.37:0/4011150997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3fc1007b0 0x7fe3fc100bd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009860 tx=0x7fe3ec02a340 comp rx=0 tx=0).stop 2024-06-10T00:56:32.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.485+0000 7fe40099e640 1 -- 172.21.15.37:0/4011150997 shutdown_connections 2024-06-10T00:56:32.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.485+0000 7fe40099e640 1 --2- 172.21.15.37:0/4011150997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3fc102360 0x7fe3fc108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.485+0000 7fe40099e640 1 --2- 172.21.15.37:0/4011150997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3fc1019a0 0x7fe3fc101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.485+0000 7fe40099e640 1 --2- 172.21.15.37:0/4011150997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3fc1007b0 0x7fe3fc100bd0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.485+0000 7fe40099e640 1 -- 172.21.15.37:0/4011150997 >> 172.21.15.37:0/4011150997 conn(0x7fe3fc0fbd40 msgr2=0x7fe3fc0fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:32.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.485+0000 7fe40099e640 1 -- 172.21.15.37:0/4011150997 shutdown_connections 2024-06-10T00:56:32.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.485+0000 7fe40099e640 1 -- 172.21.15.37:0/4011150997 wait complete. 2024-06-10T00:56:32.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.486+0000 7fe40099e640 1 Processor -- start 2024-06-10T00:56:32.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.486+0000 7fe40099e640 1 -- start start 2024-06-10T00:56:32.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.486+0000 7fe40099e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3fc1007b0 0x7fe3fc197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:32.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.486+0000 7fe40099e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3fc1019a0 0x7fe3fc1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:32.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.486+0000 7fe40099e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3fc102360 0x7fe3fc19f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:32.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.486+0000 7fe40099e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe3fc10c420 con 0x7fe3fc102360 2024-06-10T00:56:32.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.486+0000 7fe40099e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe3fc10c2a0 con 0x7fe3fc1019a0 2024-06-10T00:56:32.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.486+0000 7fe40099e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe3fc10c5a0 con 0x7fe3fc1007b0 2024-06-10T00:56:32.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.486+0000 7fe3fbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3fc102360 0x7fe3fc19f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:32.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.486+0000 7fe3fbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3fc102360 0x7fe3fc19f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47124/0 (socket says 172.21.15.37:47124) 2024-06-10T00:56:32.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.486+0000 7fe3fbfff640 1 -- 172.21.15.37:0/2516585632 learned_addr learned my addr 172.21.15.37:0/2516585632 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:32.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.486+0000 7fe3f3fff640 1 --2- 172.21.15.37:0/2516585632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3fc1019a0 0x7fe3fc1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:32.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.487+0000 7fe3fb7fe640 1 --2- 172.21.15.37:0/2516585632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3fc1007b0 0x7fe3fc197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:32.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.487+0000 7fe3fbfff640 1 -- 172.21.15.37:0/2516585632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3fc1007b0 msgr2=0x7fe3fc197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:32.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.487+0000 7fe3fbfff640 1 --2- 172.21.15.37:0/2516585632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3fc1007b0 0x7fe3fc197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.487+0000 7fe3fbfff640 1 -- 172.21.15.37:0/2516585632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3fc1019a0 msgr2=0x7fe3fc1981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:32.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.487+0000 7fe3fbfff640 1 --2- 172.21.15.37:0/2516585632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3fc1019a0 0x7fe3fc1981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.487+0000 7fe3fbfff640 1 -- 172.21.15.37:0/2516585632 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3fc19f830 con 0x7fe3fc102360 2024-06-10T00:56:32.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.487+0000 7fe3fbfff640 1 --2- 172.21.15.37:0/2516585632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3fc102360 0x7fe3fc19f190 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4009fd0 tx=0x7fe3f400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:32.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.487+0000 7fe3f97fa640 1 -- 172.21.15.37:0/2516585632 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4004b20 con 0x7fe3fc102360 2024-06-10T00:56:32.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.487+0000 7fe40099e640 1 -- 172.21.15.37:0/2516585632 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3fc19fb20 con 0x7fe3fc102360 2024-06-10T00:56:32.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.487+0000 7fe3f97fa640 1 -- 172.21.15.37:0/2516585632 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3f4003e50 con 0x7fe3fc102360 2024-06-10T00:56:32.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.487+0000 7fe3f97fa640 1 -- 172.21.15.37:0/2516585632 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f400b630 con 0x7fe3fc102360 2024-06-10T00:56:32.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.487+0000 7fe40099e640 1 -- 172.21.15.37:0/2516585632 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3fc1a0070 con 0x7fe3fc102360 2024-06-10T00:56:32.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.488+0000 7fe40099e640 1 -- 172.21.15.37:0/2516585632 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3fc0686e0 con 0x7fe3fc102360 2024-06-10T00:56:32.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.489+0000 7fe3f97fa640 1 -- 172.21.15.37:0/2516585632 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe3f401d020 con 0x7fe3fc102360 2024-06-10T00:56:32.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.490+0000 7fe3f97fa640 1 --2- 172.21.15.37:0/2516585632 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe3e80a2b90 0x7fe3e80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:32.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.490+0000 7fe3fb7fe640 1 --2- 172.21.15.37:0/2516585632 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe3e80a2b90 0x7fe3e80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:32.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.490+0000 7fe3f97fa640 1 -- 172.21.15.37:0/2516585632 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fe3f40cffa0 con 0x7fe3fc102360 2024-06-10T00:56:32.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.491+0000 7fe3fb7fe640 1 --2- 172.21.15.37:0/2516585632 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe3e80a2b90 0x7fe3e80a5050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec035040 tx=0x7fe3ec030550 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:32.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.493+0000 7fe3f97fa640 1 -- 172.21.15.37:0/2516585632 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe3f4024030 con 0x7fe3fc102360 2024-06-10T00:56:32.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.678+0000 7fe40099e640 1 -- 172.21.15.37:0/2516585632 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe3fc1a03a0 con 0x7fe3fc102360 2024-06-10T00:56:32.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.679+0000 7fe3f97fa640 1 -- 172.21.15.37:0/2516585632 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7fe3f409cab0 con 0x7fe3fc102360 2024-06-10T00:56:32.679 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-06-10T00:56:32.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.681+0000 7fe40099e640 1 -- 172.21.15.37:0/2516585632 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe3e80a2b90 msgr2=0x7fe3e80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:32.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.681+0000 7fe40099e640 1 --2- 172.21.15.37:0/2516585632 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe3e80a2b90 0x7fe3e80a5050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec035040 tx=0x7fe3ec030550 comp rx=0 tx=0).stop 2024-06-10T00:56:32.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.681+0000 7fe40099e640 1 -- 172.21.15.37:0/2516585632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3fc102360 msgr2=0x7fe3fc19f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:32.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.681+0000 7fe40099e640 1 --2- 172.21.15.37:0/2516585632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3fc102360 0x7fe3fc19f190 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4009fd0 tx=0x7fe3f400c800 comp rx=0 tx=0).stop 2024-06-10T00:56:32.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.681+0000 7fe40099e640 1 -- 172.21.15.37:0/2516585632 shutdown_connections 2024-06-10T00:56:32.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.681+0000 7fe40099e640 1 --2- 172.21.15.37:0/2516585632 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe3e80a2b90 0x7fe3e80a5050 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.681+0000 7fe40099e640 1 --2- 172.21.15.37:0/2516585632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3fc102360 0x7fe3fc19f190 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.681+0000 7fe40099e640 1 --2- 172.21.15.37:0/2516585632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3fc1019a0 0x7fe3fc1981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.681+0000 7fe40099e640 1 --2- 172.21.15.37:0/2516585632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3fc1007b0 0x7fe3fc197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.681+0000 7fe40099e640 1 -- 172.21.15.37:0/2516585632 >> 172.21.15.37:0/2516585632 conn(0x7fe3fc0fbd40 msgr2=0x7fe3fc0fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:32.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.681+0000 7fe40099e640 1 -- 172.21.15.37:0/2516585632 shutdown_connections 2024-06-10T00:56:32.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.681+0000 7fe40099e640 1 -- 172.21.15.37:0/2516585632 wait complete. 2024-06-10T00:56:32.692 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-06-10T00:56:32.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.800+0000 7fb32d0e1640 1 -- 172.21.15.37:0/3478066008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb328100900 msgr2=0x7fb328100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:32.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.800+0000 7fb32d0e1640 1 --2- 172.21.15.37:0/3478066008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb328100900 0x7fb328100d20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb318009900 tx=0x7fb31802a410 comp rx=0 tx=0).stop 2024-06-10T00:56:32.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.800+0000 7fb32d0e1640 1 -- 172.21.15.37:0/3478066008 shutdown_connections 2024-06-10T00:56:32.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.800+0000 7fb32d0e1640 1 --2- 172.21.15.37:0/3478066008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3281024b0 0x7fb328108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.800+0000 7fb32d0e1640 1 --2- 172.21.15.37:0/3478066008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb328101af0 0x7fb328101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.800+0000 7fb32d0e1640 1 --2- 172.21.15.37:0/3478066008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb328100900 0x7fb328100d20 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.800+0000 7fb32d0e1640 1 -- 172.21.15.37:0/3478066008 >> 172.21.15.37:0/3478066008 conn(0x7fb3280fbe90 msgr2=0x7fb3280fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:32.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.800+0000 7fb32d0e1640 1 -- 172.21.15.37:0/3478066008 shutdown_connections 2024-06-10T00:56:32.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.800+0000 7fb32d0e1640 1 -- 172.21.15.37:0/3478066008 wait complete. 2024-06-10T00:56:32.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.801+0000 7fb32d0e1640 1 Processor -- start 2024-06-10T00:56:32.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.801+0000 7fb32d0e1640 1 -- start start 2024-06-10T00:56:32.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.801+0000 7fb32d0e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb328100900 0x7fb32806ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:32.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.801+0000 7fb32d0e1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb328101af0 0x7fb32819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:32.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.801+0000 7fb32d0e1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3281024b0 0x7fb3281a1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:32.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.801+0000 7fb32d0e1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb32810c550 con 0x7fb328100900 2024-06-10T00:56:32.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.801+0000 7fb32d0e1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb32810c3d0 con 0x7fb3281024b0 2024-06-10T00:56:32.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.801+0000 7fb32d0e1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb32810c6d0 con 0x7fb328101af0 2024-06-10T00:56:32.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.801+0000 7fb326d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb328100900 0x7fb32806ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:32.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.801+0000 7fb326d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb328100900 0x7fb32806ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47130/0 (socket says 172.21.15.37:47130) 2024-06-10T00:56:32.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.801+0000 7fb326d9d640 1 -- 172.21.15.37:0/2697997915 learned_addr learned my addr 172.21.15.37:0/2697997915 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:32.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.801+0000 7fb32759e640 1 --2- 172.21.15.37:0/2697997915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3281024b0 0x7fb3281a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:32.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.801+0000 7fb32659c640 1 --2- 172.21.15.37:0/2697997915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb328101af0 0x7fb32819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:32.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.801+0000 7fb326d9d640 1 -- 172.21.15.37:0/2697997915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb328101af0 msgr2=0x7fb32819eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:32.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.801+0000 7fb326d9d640 1 --2- 172.21.15.37:0/2697997915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb328101af0 0x7fb32819eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.801+0000 7fb326d9d640 1 -- 172.21.15.37:0/2697997915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3281024b0 msgr2=0x7fb3281a1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:32.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.801+0000 7fb326d9d640 1 --2- 172.21.15.37:0/2697997915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3281024b0 0x7fb3281a1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.801+0000 7fb326d9d640 1 -- 172.21.15.37:0/2697997915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3281a1bf0 con 0x7fb328100900 2024-06-10T00:56:32.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.801+0000 7fb32759e640 1 --2- 172.21.15.37:0/2697997915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3281024b0 0x7fb3281a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:56:32.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.802+0000 7fb326d9d640 1 --2- 172.21.15.37:0/2697997915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb328100900 0x7fb32806ab10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb318031610 tx=0x7fb3180043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:32.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.802+0000 7fb30ffff640 1 -- 172.21.15.37:0/2697997915 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31803f030 con 0x7fb328100900 2024-06-10T00:56:32.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.802+0000 7fb30ffff640 1 -- 172.21.15.37:0/2697997915 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb318003ce0 con 0x7fb328100900 2024-06-10T00:56:32.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.802+0000 7fb32d0e1640 1 -- 172.21.15.37:0/2697997915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3281a1e20 con 0x7fb328100900 2024-06-10T00:56:32.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.802+0000 7fb30ffff640 1 -- 172.21.15.37:0/2697997915 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb318035070 con 0x7fb328100900 2024-06-10T00:56:32.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.802+0000 7fb32d0e1640 1 -- 172.21.15.37:0/2697997915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3281a2230 con 0x7fb328100900 2024-06-10T00:56:32.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.803+0000 7fb32d0e1640 1 -- 172.21.15.37:0/2697997915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb328107d10 con 0x7fb328100900 2024-06-10T00:56:32.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.804+0000 7fb30ffff640 1 -- 172.21.15.37:0/2697997915 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb318042030 con 0x7fb328100900 2024-06-10T00:56:32.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.805+0000 7fb30ffff640 1 --2- 172.21.15.37:0/2697997915 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb3100a2bc0 0x7fb3100a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:32.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.805+0000 7fb32659c640 1 --2- 172.21.15.37:0/2697997915 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb3100a2bc0 0x7fb3100a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:32.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.805+0000 7fb30ffff640 1 -- 172.21.15.37:0/2697997915 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fb3180efa90 con 0x7fb328100900 2024-06-10T00:56:32.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.805+0000 7fb32659c640 1 --2- 172.21.15.37:0/2697997915 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb3100a2bc0 0x7fb3100a5080 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb31c005fd0 tx=0x7fb31c005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:32.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.809+0000 7fb30ffff640 1 -- 172.21.15.37:0/2697997915 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb3180bc5a0 con 0x7fb328100900 2024-06-10T00:56:32.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.994+0000 7fb32d0e1640 1 -- 172.21.15.37:0/2697997915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb328061de0 con 0x7fb328100900 2024-06-10T00:56:32.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.994+0000 7fb30ffff640 1 -- 172.21.15.37:0/2697997915 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v100) ==== 103+0+11 (secure 0 0 0) 0x7fb3180bbea0 con 0x7fb328100900 2024-06-10T00:56:32.995 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-06-10T00:56:32.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.997+0000 7fb32d0e1640 1 -- 172.21.15.37:0/2697997915 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb3100a2bc0 msgr2=0x7fb3100a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:32.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.997+0000 7fb32d0e1640 1 --2- 172.21.15.37:0/2697997915 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb3100a2bc0 0x7fb3100a5080 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb31c005fd0 tx=0x7fb31c005850 comp rx=0 tx=0).stop 2024-06-10T00:56:32.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.997+0000 7fb32d0e1640 1 -- 172.21.15.37:0/2697997915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb328100900 msgr2=0x7fb32806ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:32.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.997+0000 7fb32d0e1640 1 --2- 172.21.15.37:0/2697997915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb328100900 0x7fb32806ab10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb318031610 tx=0x7fb3180043b0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.997+0000 7fb32d0e1640 1 -- 172.21.15.37:0/2697997915 shutdown_connections 2024-06-10T00:56:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.997+0000 7fb32d0e1640 1 --2- 172.21.15.37:0/2697997915 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb3100a2bc0 0x7fb3100a5080 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.997+0000 7fb32d0e1640 1 --2- 172.21.15.37:0/2697997915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3281024b0 0x7fb3281a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.997+0000 7fb32d0e1640 1 --2- 172.21.15.37:0/2697997915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb328101af0 0x7fb32819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.997+0000 7fb32d0e1640 1 --2- 172.21.15.37:0/2697997915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb328100900 0x7fb32806ab10 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.997+0000 7fb32d0e1640 1 -- 172.21.15.37:0/2697997915 >> 172.21.15.37:0/2697997915 conn(0x7fb3280fbe90 msgr2=0x7fb328104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.997+0000 7fb32d0e1640 1 -- 172.21.15.37:0/2697997915 shutdown_connections 2024-06-10T00:56:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:32.997+0000 7fb32d0e1640 1 -- 172.21.15.37:0/2697997915 wait complete. 2024-06-10T00:56:33.012 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-06-10T00:56:33.013 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:56:33.264 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:56:33.266 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:33 smithi094 ceph-mon[36708]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:56:33.266 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:33 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/28297353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T00:56:33.266 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:33 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3799056635' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-10T00:56:33.266 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:33 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2516585632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-10T00:56:33.266 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:33 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2697997915' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-10T00:56:33.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:33 smithi167 ceph-mon[36917]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:56:33.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:33 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/28297353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T00:56:33.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:33 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3799056635' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-10T00:56:33.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:33 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2516585632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-10T00:56:33.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:33 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2697997915' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-10T00:56:33.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:33 smithi037 ceph-mon[32192]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-06-10T00:56:33.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:33 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/28297353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T00:56:33.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:33 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3799056635' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-10T00:56:33.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:33 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2516585632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-10T00:56:33.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:33 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2697997915' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-10T00:56:34.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:34 smithi167 ceph-mon[36917]: pgmap v426: 129 pgs: 129 active+clean; 654 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 853 B/s wr, 7 op/s 2024-06-10T00:56:34.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:34 smithi167 ceph-mon[36917]: mgrmap e19: x(active, since 11m), standbys: y, z 2024-06-10T00:56:34.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:34 smithi094 ceph-mon[36708]: pgmap v426: 129 pgs: 129 active+clean; 654 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 853 B/s wr, 7 op/s 2024-06-10T00:56:34.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:34 smithi094 ceph-mon[36708]: mgrmap e19: x(active, since 11m), standbys: y, z 2024-06-10T00:56:34.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:34 smithi037 ceph-mon[32192]: pgmap v426: 129 pgs: 129 active+clean; 654 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 853 B/s wr, 7 op/s 2024-06-10T00:56:34.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:34 smithi037 ceph-mon[32192]: mgrmap e19: x(active, since 11m), standbys: y, z 2024-06-10T00:56:35.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.081+0000 7f57d08bc700 1 -- 172.21.15.37:0/2151987657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57cc101d40 msgr2=0x7f57cc10e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:35.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.081+0000 7f57d08bc700 1 --2- 172.21.15.37:0/2151987657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57cc101d40 0x7f57cc10e230 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009990 tx=0x7f57c4009ca0 comp rx=0 tx=0).stop 2024-06-10T00:56:35.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.081+0000 7f57d08bc700 1 -- 172.21.15.37:0/2151987657 shutdown_connections 2024-06-10T00:56:35.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.081+0000 7f57d08bc700 1 --2- 172.21.15.37:0/2151987657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57cc101d40 0x7f57cc10e230 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:35.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.081+0000 7f57d08bc700 1 --2- 172.21.15.37:0/2151987657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57cc1073f0 0x7f57cc1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:35.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.081+0000 7f57d08bc700 1 --2- 172.21.15.37:0/2151987657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57cc101380 0x7f57cc101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:35.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.081+0000 7f57d08bc700 1 -- 172.21.15.37:0/2151987657 >> 172.21.15.37:0/2151987657 conn(0x7f57cc078300 msgr2=0x7f57cc078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:35.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.081+0000 7f57d08bc700 1 -- 172.21.15.37:0/2151987657 shutdown_connections 2024-06-10T00:56:35.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.081+0000 7f57d08bc700 1 -- 172.21.15.37:0/2151987657 wait complete. 2024-06-10T00:56:35.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.082+0000 7f57d08bc700 1 Processor -- start 2024-06-10T00:56:35.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.082+0000 7f57d08bc700 1 -- start start 2024-06-10T00:56:35.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.082+0000 7f57d08bc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57cc101380 0x7f57cc19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:35.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.082+0000 7f57d08bc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57cc101d40 0x7f57cc19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:35.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.082+0000 7f57d08bc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57cc1073f0 0x7f57cc1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:35.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.082+0000 7f57d08bc700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc111660 con 0x7f57cc101d40 2024-06-10T00:56:35.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.082+0000 7f57d08bc700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc1114e0 con 0x7f57cc1073f0 2024-06-10T00:56:35.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.082+0000 7f57d08bc700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc1117e0 con 0x7f57cc101380 2024-06-10T00:56:35.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.082+0000 7f57c9d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57cc101d40 0x7f57cc19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:35.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.082+0000 7f57c9d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57cc101d40 0x7f57cc19ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47144/0 (socket says 172.21.15.37:47144) 2024-06-10T00:56:35.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.082+0000 7f57c9d9b700 1 -- 172.21.15.37:0/1370056769 learned_addr learned my addr 172.21.15.37:0/1370056769 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:35.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.082+0000 7f57ca59c700 1 --2- 172.21.15.37:0/1370056769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57cc101380 0x7f57cc19e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:35.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.082+0000 7f57cad9d700 1 --2- 172.21.15.37:0/1370056769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57cc1073f0 0x7f57cc1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:35.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.082+0000 7f57c9d9b700 1 -- 172.21.15.37:0/1370056769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57cc101380 msgr2=0x7f57cc19e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:35.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.082+0000 7f57c9d9b700 1 --2- 172.21.15.37:0/1370056769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57cc101380 0x7f57cc19e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:35.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.082+0000 7f57c9d9b700 1 -- 172.21.15.37:0/1370056769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57cc1073f0 msgr2=0x7f57cc1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:35.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.082+0000 7f57c9d9b700 1 --2- 172.21.15.37:0/1370056769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57cc1073f0 0x7f57cc1989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:35.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.082+0000 7f57c9d9b700 1 -- 172.21.15.37:0/1370056769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57cc199110 con 0x7f57cc101d40 2024-06-10T00:56:35.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.083+0000 7f57c9d9b700 1 --2- 172.21.15.37:0/1370056769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57cc101d40 0x7f57cc19ed30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f57b400d950 tx=0x7f57b400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:35.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.083+0000 7f57c37fe700 1 -- 172.21.15.37:0/1370056769 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b4016070 con 0x7f57cc101d40 2024-06-10T00:56:35.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.083+0000 7f57d08bc700 1 -- 172.21.15.37:0/1370056769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57cc199400 con 0x7f57cc101d40 2024-06-10T00:56:35.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.083+0000 7f57c37fe700 1 -- 172.21.15.37:0/1370056769 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57b4009e20 con 0x7f57cc101d40 2024-06-10T00:56:35.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.083+0000 7f57c37fe700 1 -- 172.21.15.37:0/1370056769 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b400f620 con 0x7f57cc101d40 2024-06-10T00:56:35.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.083+0000 7f57d08bc700 1 -- 172.21.15.37:0/1370056769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57cc1a56c0 con 0x7f57cc101d40 2024-06-10T00:56:35.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.084+0000 7f57d08bc700 1 -- 172.21.15.37:0/1370056769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57cc069180 con 0x7f57cc101d40 2024-06-10T00:56:35.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.085+0000 7f57c37fe700 1 -- 172.21.15.37:0/1370056769 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f57b4019070 con 0x7f57cc101d40 2024-06-10T00:56:35.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.086+0000 7f57c37fe700 1 --2- 172.21.15.37:0/1370056769 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f57b80a2cb0 0x7f57b80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:35.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.086+0000 7f57ca59c700 1 --2- 172.21.15.37:0/1370056769 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f57b80a2cb0 0x7f57b80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:35.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.086+0000 7f57c37fe700 1 -- 172.21.15.37:0/1370056769 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f57b40cf180 con 0x7f57cc101d40 2024-06-10T00:56:35.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.086+0000 7f57ca59c700 1 --2- 172.21.15.37:0/1370056769 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f57b80a2cb0 0x7f57b80a5170 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009550 tx=0x7f57bc0091b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:35.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.089+0000 7f57c37fe700 1 -- 172.21.15.37:0/1370056769 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f57b409bb90 con 0x7f57cc101d40 2024-06-10T00:56:35.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.341+0000 7f57d08bc700 1 -- 172.21.15.37:0/1370056769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57cc1a58f0 con 0x7f57cc101d40 2024-06-10T00:56:35.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.341+0000 7f57c37fe700 1 -- 172.21.15.37:0/1370056769 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11867 (secure 0 0 0) 0x7f57b409b490 con 0x7f57cc101d40 2024-06-10T00:56:35.342 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:56:35.342 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T00:56:30.291171+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24451,24458,24472]},"id":1}]} 2024-06-10T00:56:35.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.344+0000 7f57d08bc700 1 -- 172.21.15.37:0/1370056769 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f57b80a2cb0 msgr2=0x7f57b80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:35.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.344+0000 7f57d08bc700 1 --2- 172.21.15.37:0/1370056769 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f57b80a2cb0 0x7f57b80a5170 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009550 tx=0x7f57bc0091b0 comp rx=0 tx=0).stop 2024-06-10T00:56:35.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.344+0000 7f57d08bc700 1 -- 172.21.15.37:0/1370056769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57cc101d40 msgr2=0x7f57cc19ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:35.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.344+0000 7f57d08bc700 1 --2- 172.21.15.37:0/1370056769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57cc101d40 0x7f57cc19ed30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f57b400d950 tx=0x7f57b400dc60 comp rx=0 tx=0).stop 2024-06-10T00:56:35.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.344+0000 7f57d08bc700 1 -- 172.21.15.37:0/1370056769 shutdown_connections 2024-06-10T00:56:35.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.344+0000 7f57d08bc700 1 --2- 172.21.15.37:0/1370056769 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f57b80a2cb0 0x7f57b80a5170 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:35.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.344+0000 7f57d08bc700 1 --2- 172.21.15.37:0/1370056769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57cc1073f0 0x7f57cc1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:35.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.344+0000 7f57d08bc700 1 --2- 172.21.15.37:0/1370056769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57cc101380 0x7f57cc19e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:35.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.344+0000 7f57d08bc700 1 --2- 172.21.15.37:0/1370056769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57cc101d40 0x7f57cc19ed30 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:35.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.344+0000 7f57d08bc700 1 -- 172.21.15.37:0/1370056769 >> 172.21.15.37:0/1370056769 conn(0x7f57cc078300 msgr2=0x7f57cc1005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:35.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.344+0000 7f57d08bc700 1 -- 172.21.15.37:0/1370056769 shutdown_connections 2024-06-10T00:56:35.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:35.344+0000 7f57d08bc700 1 -- 172.21.15.37:0/1370056769 wait complete. 2024-06-10T00:56:35.346 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-06-10T00:56:36.253 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-06-10T00:56:36.253 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-06-10T00:56:36.253 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-10T00:56:36.280 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:36 smithi037 ceph-mon[32192]: pgmap v427: 129 pgs: 129 active+clean; 654 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 6 op/s 2024-06-10T00:56:36.280 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:36 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1370056769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:56:36.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:36 smithi167 ceph-mon[36917]: pgmap v427: 129 pgs: 129 active+clean; 654 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 6 op/s 2024-06-10T00:56:36.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:36 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1370056769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:56:36.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:36 smithi094 ceph-mon[36708]: pgmap v427: 129 pgs: 129 active+clean; 654 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 6 op/s 2024-06-10T00:56:36.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:36 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1370056769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:56:36.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.369+0000 7f22ea770640 1 -- 172.21.15.37:0/2661055638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22e4100900 msgr2=0x7f22e4100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:36.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.369+0000 7f22ea770640 1 --2- 172.21.15.37:0/2661055638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22e4100900 0x7f22e4100d20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009900 tx=0x7f22d402a410 comp rx=0 tx=0).stop 2024-06-10T00:56:36.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.369+0000 7f22ea770640 1 -- 172.21.15.37:0/2661055638 shutdown_connections 2024-06-10T00:56:36.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.369+0000 7f22ea770640 1 --2- 172.21.15.37:0/2661055638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22e41024b0 0x7f22e4108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:36.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.369+0000 7f22ea770640 1 --2- 172.21.15.37:0/2661055638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22e4101af0 0x7f22e4101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:36.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.369+0000 7f22ea770640 1 --2- 172.21.15.37:0/2661055638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22e4100900 0x7f22e4100d20 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:36.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.369+0000 7f22ea770640 1 -- 172.21.15.37:0/2661055638 >> 172.21.15.37:0/2661055638 conn(0x7f22e40fbe90 msgr2=0x7f22e40fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:36.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.369+0000 7f22ea770640 1 -- 172.21.15.37:0/2661055638 shutdown_connections 2024-06-10T00:56:36.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.369+0000 7f22ea770640 1 -- 172.21.15.37:0/2661055638 wait complete. 2024-06-10T00:56:36.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.370+0000 7f22ea770640 1 Processor -- start 2024-06-10T00:56:36.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.370+0000 7f22ea770640 1 -- start start 2024-06-10T00:56:36.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.370+0000 7f22ea770640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22e4100900 0x7f22e4196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:36.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.370+0000 7f22ea770640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22e4101af0 0x7f22e4197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:36.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.370+0000 7f22ea770640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22e41024b0 0x7f22e419e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:36.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.370+0000 7f22ea770640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f22e410c580 con 0x7f22e4101af0 2024-06-10T00:56:36.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.370+0000 7f22ea770640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f22e410c400 con 0x7f22e41024b0 2024-06-10T00:56:36.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.370+0000 7f22ea770640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f22e410c700 con 0x7f22e4100900 2024-06-10T00:56:36.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.370+0000 7f22dbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22e4101af0 0x7f22e4197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:36.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.370+0000 7f22dbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22e4101af0 0x7f22e4197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47158/0 (socket says 172.21.15.37:47158) 2024-06-10T00:56:36.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.370+0000 7f22dbfff640 1 -- 172.21.15.37:0/2520816136 learned_addr learned my addr 172.21.15.37:0/2520816136 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:36.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.370+0000 7f22e3fff640 1 --2- 172.21.15.37:0/2520816136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22e4100900 0x7f22e4196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:36.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.370+0000 7f22dbfff640 1 -- 172.21.15.37:0/2520816136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22e4100900 msgr2=0x7f22e4196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:36.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.370+0000 7f22dbfff640 1 --2- 172.21.15.37:0/2520816136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22e4100900 0x7f22e4196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:36.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.370+0000 7f22dbfff640 1 -- 172.21.15.37:0/2520816136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22e41024b0 msgr2=0x7f22e419e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:36.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.370+0000 7f22dbfff640 1 --2- 172.21.15.37:0/2520816136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22e41024b0 0x7f22e419e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:36.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.370+0000 7f22dbfff640 1 -- 172.21.15.37:0/2520816136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e419e800 con 0x7f22e4101af0 2024-06-10T00:56:36.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.370+0000 7f22e3fff640 1 --2- 172.21.15.37:0/2520816136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22e4100900 0x7f22e4196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:56:36.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.371+0000 7f22dbfff640 1 --2- 172.21.15.37:0/2520816136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22e4101af0 0x7f22e4197170 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00d950 tx=0x7f22cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:36.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.371+0000 7f22e1ffb640 1 -- 172.21.15.37:0/2520816136 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22cc017030 con 0x7f22e4101af0 2024-06-10T00:56:36.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.371+0000 7f22ea770640 1 -- 172.21.15.37:0/2520816136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22e419eaf0 con 0x7f22e4101af0 2024-06-10T00:56:36.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.371+0000 7f22e1ffb640 1 -- 172.21.15.37:0/2520816136 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22cc004020 con 0x7f22e4101af0 2024-06-10T00:56:36.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.371+0000 7f22e1ffb640 1 -- 172.21.15.37:0/2520816136 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22cc0047a0 con 0x7f22e4101af0 2024-06-10T00:56:36.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.371+0000 7f22ea770640 1 -- 172.21.15.37:0/2520816136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22e419f040 con 0x7f22e4101af0 2024-06-10T00:56:36.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.372+0000 7f22ea770640 1 -- 172.21.15.37:0/2520816136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22e4107ca0 con 0x7f22e4101af0 2024-06-10T00:56:36.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.373+0000 7f22e1ffb640 1 -- 172.21.15.37:0/2520816136 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f22cc0041d0 con 0x7f22e4101af0 2024-06-10T00:56:36.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.374+0000 7f22e1ffb640 1 --2- 172.21.15.37:0/2520816136 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f22d00a2c20 0x7f22d00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:36.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.374+0000 7f22e3fff640 1 --2- 172.21.15.37:0/2520816136 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f22d00a2c20 0x7f22d00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:36.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.374+0000 7f22e1ffb640 1 -- 172.21.15.37:0/2520816136 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f22cc0cf120 con 0x7f22e4101af0 2024-06-10T00:56:36.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.374+0000 7f22e3fff640 1 --2- 172.21.15.37:0/2520816136 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f22d00a2c20 0x7f22d00a50e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f22d4006010 tx=0x7f22d4032cf0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:36.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.379+0000 7f22e1ffb640 1 -- 172.21.15.37:0/2520816136 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f22cc09bad0 con 0x7f22e4101af0 2024-06-10T00:56:36.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.561+0000 7f22ea770640 1 -- 172.21.15.37:0/2520816136 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f22e4061de0 con 0x7f22d00a2c20 2024-06-10T00:56:36.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.571+0000 7f22e1ffb640 1 -- 172.21.15.37:0/2520816136 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340965 (secure 0 0 0) 0x7f22e4061de0 con 0x7f22d00a2c20 2024-06-10T00:56:36.576 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:56:36.578 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-06-10T00:56:36.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.575+0000 7f22ea770640 1 -- 172.21.15.37:0/2520816136 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f22d00a2c20 msgr2=0x7f22d00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:36.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.575+0000 7f22ea770640 1 --2- 172.21.15.37:0/2520816136 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f22d00a2c20 0x7f22d00a50e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f22d4006010 tx=0x7f22d4032cf0 comp rx=0 tx=0).stop 2024-06-10T00:56:36.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.575+0000 7f22ea770640 1 -- 172.21.15.37:0/2520816136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22e4101af0 msgr2=0x7f22e4197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:36.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.575+0000 7f22ea770640 1 --2- 172.21.15.37:0/2520816136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22e4101af0 0x7f22e4197170 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00d950 tx=0x7f22cc00de10 comp rx=0 tx=0).stop 2024-06-10T00:56:36.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.575+0000 7f22ea770640 1 -- 172.21.15.37:0/2520816136 shutdown_connections 2024-06-10T00:56:36.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.575+0000 7f22ea770640 1 --2- 172.21.15.37:0/2520816136 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f22d00a2c20 0x7f22d00a50e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:36.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.575+0000 7f22ea770640 1 --2- 172.21.15.37:0/2520816136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22e41024b0 0x7f22e419e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:36.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.575+0000 7f22ea770640 1 --2- 172.21.15.37:0/2520816136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22e4101af0 0x7f22e4197170 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:36.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.575+0000 7f22ea770640 1 --2- 172.21.15.37:0/2520816136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22e4100900 0x7f22e4196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:36.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.575+0000 7f22ea770640 1 -- 172.21.15.37:0/2520816136 >> 172.21.15.37:0/2520816136 conn(0x7f22e40fbe90 msgr2=0x7f22e4104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:36.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.575+0000 7f22ea770640 1 -- 172.21.15.37:0/2520816136 shutdown_connections 2024-06-10T00:56:36.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.575+0000 7f22ea770640 1 -- 172.21.15.37:0/2520816136 wait complete. 2024-06-10T00:56:36.587 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":428,"stamp":"2024-06-10T00:56:36.410576+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":672924,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1201,"num_read_kb":737,"num_write":374,"num_write_kb":3088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":802,"ondisk_log_size":802,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370920,"kb_used_data":13448,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570904,"statfs":{"total":1151940427776,"available":1151560605696,"internally_reserved":0,"allocated":13770752,"data_stored":8866874,"data_compressed":497625,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7840,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":50,"num_write":13,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003319"},"pg_stats":[{"pgid":"3.3f","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373740+0000","last_change":"2024-06-10T00:52:35.971067+0000","last_active":"2024-06-10T00:55:20.373740+0000","last_peered":"2024-06-10T00:55:20.373740+0000","last_clean":"2024-06-10T00:55:20.373740+0000","last_became_active":"2024-06-10T00:52:35.970292+0000","last_became_peered":"2024-06-10T00:52:35.970292+0000","last_unstale":"2024-06-10T00:55:20.373740+0000","last_undegraded":"2024-06-10T00:55:20.373740+0000","last_fullsized":"2024-06-10T00:55:20.373740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:57:37.287238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364432+0000","last_change":"2024-06-10T00:52:35.960421+0000","last_active":"2024-06-10T00:55:20.364432+0000","last_peered":"2024-06-10T00:55:20.364432+0000","last_clean":"2024-06-10T00:55:20.364432+0000","last_became_active":"2024-06-10T00:52:35.960070+0000","last_became_peered":"2024-06-10T00:52:35.960070+0000","last_unstale":"2024-06-10T00:55:20.364432+0000","last_undegraded":"2024-06-10T00:55:20.364432+0000","last_fullsized":"2024-06-10T00:55:20.364432+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:48:02.616434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375701+0000","last_change":"2024-06-10T00:52:35.964778+0000","last_active":"2024-06-10T00:55:20.375701+0000","last_peered":"2024-06-10T00:55:20.375701+0000","last_clean":"2024-06-10T00:55:20.375701+0000","last_became_active":"2024-06-10T00:52:35.964463+0000","last_became_peered":"2024-06-10T00:52:35.964463+0000","last_unstale":"2024-06-10T00:55:20.375701+0000","last_undegraded":"2024-06-10T00:55:20.375701+0000","last_fullsized":"2024-06-10T00:55:20.375701+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:52:44.987363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.360279+0000","last_change":"2024-06-10T00:52:35.963226+0000","last_active":"2024-06-10T00:55:20.360279+0000","last_peered":"2024-06-10T00:55:20.360279+0000","last_clean":"2024-06-10T00:55:20.360279+0000","last_became_active":"2024-06-10T00:52:35.962862+0000","last_became_peered":"2024-06-10T00:52:35.962862+0000","last_unstale":"2024-06-10T00:55:20.360279+0000","last_undegraded":"2024-06-10T00:55:20.360279+0000","last_fullsized":"2024-06-10T00:55:20.360279+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:08:56.418071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.360258+0000","last_change":"2024-06-10T00:52:35.957427+0000","last_active":"2024-06-10T00:55:20.360258+0000","last_peered":"2024-06-10T00:55:20.360258+0000","last_clean":"2024-06-10T00:55:20.360258+0000","last_became_active":"2024-06-10T00:52:35.957049+0000","last_became_peered":"2024-06-10T00:52:35.957049+0000","last_unstale":"2024-06-10T00:55:20.360258+0000","last_undegraded":"2024-06-10T00:55:20.360258+0000","last_fullsized":"2024-06-10T00:55:20.360258+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:16:04.257210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.355809+0000","last_change":"2024-06-10T00:52:35.954251+0000","last_active":"2024-06-10T00:55:20.355809+0000","last_peered":"2024-06-10T00:55:20.355809+0000","last_clean":"2024-06-10T00:55:20.355809+0000","last_became_active":"2024-06-10T00:52:35.953908+0000","last_became_peered":"2024-06-10T00:52:35.953908+0000","last_unstale":"2024-06-10T00:55:20.355809+0000","last_undegraded":"2024-06-10T00:55:20.355809+0000","last_fullsized":"2024-06-10T00:55:20.355809+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:55:25.125836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364343+0000","last_change":"2024-06-10T00:52:35.956822+0000","last_active":"2024-06-10T00:55:20.364343+0000","last_peered":"2024-06-10T00:55:20.364343+0000","last_clean":"2024-06-10T00:55:20.364343+0000","last_became_active":"2024-06-10T00:52:35.956466+0000","last_became_peered":"2024-06-10T00:52:35.956466+0000","last_unstale":"2024-06-10T00:55:20.364343+0000","last_undegraded":"2024-06-10T00:55:20.364343+0000","last_fullsized":"2024-06-10T00:55:20.364343+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:36:41.534392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.369347+0000","last_change":"2024-06-10T00:52:35.962978+0000","last_active":"2024-06-10T00:55:20.369347+0000","last_peered":"2024-06-10T00:55:20.369347+0000","last_clean":"2024-06-10T00:55:20.369347+0000","last_became_active":"2024-06-10T00:52:35.962609+0000","last_became_peered":"2024-06-10T00:52:35.962609+0000","last_unstale":"2024-06-10T00:55:20.369347+0000","last_undegraded":"2024-06-10T00:55:20.369347+0000","last_fullsized":"2024-06-10T00:55:20.369347+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:20:57.469577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.377074+0000","last_change":"2024-06-10T00:52:35.963203+0000","last_active":"2024-06-10T00:55:20.377074+0000","last_peered":"2024-06-10T00:55:20.377074+0000","last_clean":"2024-06-10T00:55:20.377074+0000","last_became_active":"2024-06-10T00:52:35.962789+0000","last_became_peered":"2024-06-10T00:52:35.962789+0000","last_unstale":"2024-06-10T00:55:20.377074+0000","last_undegraded":"2024-06-10T00:55:20.377074+0000","last_fullsized":"2024-06-10T00:55:20.377074+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:00:17.819193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"100'16","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.356376+0000","last_change":"2024-06-10T00:52:35.966940+0000","last_active":"2024-06-10T00:55:20.356376+0000","last_peered":"2024-06-10T00:55:20.356376+0000","last_clean":"2024-06-10T00:55:20.356376+0000","last_became_active":"2024-06-10T00:52:35.966554+0000","last_became_peered":"2024-06-10T00:52:35.966554+0000","last_unstale":"2024-06-10T00:55:20.356376+0000","last_undegraded":"2024-06-10T00:55:20.356376+0000","last_fullsized":"2024-06-10T00:55:20.356376+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:00.374437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372673+0000","last_change":"2024-06-10T00:52:35.960927+0000","last_active":"2024-06-10T00:55:20.372673+0000","last_peered":"2024-06-10T00:55:20.372673+0000","last_clean":"2024-06-10T00:55:20.372673+0000","last_became_active":"2024-06-10T00:52:35.960577+0000","last_became_peered":"2024-06-10T00:52:35.960577+0000","last_unstale":"2024-06-10T00:55:20.372673+0000","last_undegraded":"2024-06-10T00:55:20.372673+0000","last_fullsized":"2024-06-10T00:55:20.372673+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:50:02.366695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.355167+0000","last_change":"2024-06-10T00:52:35.964435+0000","last_active":"2024-06-10T00:55:20.355167+0000","last_peered":"2024-06-10T00:55:20.355167+0000","last_clean":"2024-06-10T00:55:20.355167+0000","last_became_active":"2024-06-10T00:52:35.964069+0000","last_became_peered":"2024-06-10T00:52:35.964069+0000","last_unstale":"2024-06-10T00:55:20.355167+0000","last_undegraded":"2024-06-10T00:55:20.355167+0000","last_fullsized":"2024-06-10T00:55:20.355167+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:06.708551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.361787+0000","last_change":"2024-06-10T00:52:35.966722+0000","last_active":"2024-06-10T00:55:20.361787+0000","last_peered":"2024-06-10T00:55:20.361787+0000","last_clean":"2024-06-10T00:55:20.361787+0000","last_became_active":"2024-06-10T00:52:35.966366+0000","last_became_peered":"2024-06-10T00:52:35.966366+0000","last_unstale":"2024-06-10T00:55:20.361787+0000","last_undegraded":"2024-06-10T00:55:20.361787+0000","last_fullsized":"2024-06-10T00:55:20.361787+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:57:46.021047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.371736+0000","last_change":"2024-06-10T00:52:35.955307+0000","last_active":"2024-06-10T00:55:20.371736+0000","last_peered":"2024-06-10T00:55:20.371736+0000","last_clean":"2024-06-10T00:55:20.371736+0000","last_became_active":"2024-06-10T00:52:35.954140+0000","last_became_peered":"2024-06-10T00:52:35.954140+0000","last_unstale":"2024-06-10T00:55:20.371736+0000","last_undegraded":"2024-06-10T00:55:20.371736+0000","last_fullsized":"2024-06-10T00:55:20.371736+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:47:22.493598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373915+0000","last_change":"2024-06-10T00:52:35.960592+0000","last_active":"2024-06-10T00:55:20.373915+0000","last_peered":"2024-06-10T00:55:20.373915+0000","last_clean":"2024-06-10T00:55:20.373915+0000","last_became_active":"2024-06-10T00:52:35.960263+0000","last_became_peered":"2024-06-10T00:52:35.960263+0000","last_unstale":"2024-06-10T00:55:20.373915+0000","last_undegraded":"2024-06-10T00:55:20.373915+0000","last_fullsized":"2024-06-10T00:55:20.373915+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:47:11.658666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372126+0000","last_change":"2024-06-10T00:52:35.965796+0000","last_active":"2024-06-10T00:55:20.372126+0000","last_peered":"2024-06-10T00:55:20.372126+0000","last_clean":"2024-06-10T00:55:20.372126+0000","last_became_active":"2024-06-10T00:52:35.963965+0000","last_became_peered":"2024-06-10T00:52:35.963965+0000","last_unstale":"2024-06-10T00:55:20.372126+0000","last_undegraded":"2024-06-10T00:55:20.372126+0000","last_fullsized":"2024-06-10T00:55:20.372126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:29:47.149852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374090+0000","last_change":"2024-06-10T00:52:35.970714+0000","last_active":"2024-06-10T00:55:20.374090+0000","last_peered":"2024-06-10T00:55:20.374090+0000","last_clean":"2024-06-10T00:55:20.374090+0000","last_became_active":"2024-06-10T00:52:35.970173+0000","last_became_peered":"2024-06-10T00:52:35.970173+0000","last_unstale":"2024-06-10T00:55:20.374090+0000","last_undegraded":"2024-06-10T00:55:20.374090+0000","last_fullsized":"2024-06-10T00:55:20.374090+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:33:28.183156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.370737+0000","last_change":"2024-06-10T00:52:35.963052+0000","last_active":"2024-06-10T00:55:20.370737+0000","last_peered":"2024-06-10T00:55:20.370737+0000","last_clean":"2024-06-10T00:55:20.370737+0000","last_became_active":"2024-06-10T00:52:35.962674+0000","last_became_peered":"2024-06-10T00:52:35.962674+0000","last_unstale":"2024-06-10T00:55:20.370737+0000","last_undegraded":"2024-06-10T00:55:20.370737+0000","last_fullsized":"2024-06-10T00:55:20.370737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:56:38.836854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"100'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.363943+0000","last_change":"2024-06-10T00:53:22.130336+0000","last_active":"2024-06-10T00:55:20.363943+0000","last_peered":"2024-06-10T00:55:20.363943+0000","last_clean":"2024-06-10T00:55:20.363943+0000","last_became_active":"2024-06-10T00:53:22.129910+0000","last_became_peered":"2024-06-10T00:53:22.129910+0000","last_unstale":"2024-06-10T00:55:20.363943+0000","last_undegraded":"2024-06-10T00:55:20.363943+0000","last_fullsized":"2024-06-10T00:55:20.363943+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:23:50.600821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554286+0000","last_change":"2024-06-10T00:52:45.997276+0000","last_active":"2024-06-10T00:54:22.554286+0000","last_peered":"2024-06-10T00:54:22.554286+0000","last_clean":"2024-06-10T00:54:22.554286+0000","last_became_active":"2024-06-10T00:52:45.995638+0000","last_became_peered":"2024-06-10T00:52:45.995638+0000","last_unstale":"2024-06-10T00:54:22.554286+0000","last_undegraded":"2024-06-10T00:54:22.554286+0000","last_fullsized":"2024-06-10T00:54:22.554286+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:14:20.922348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556499+0000","last_change":"2024-06-10T00:52:32.760888+0000","last_active":"2024-06-10T00:54:22.556499+0000","last_peered":"2024-06-10T00:54:22.556499+0000","last_clean":"2024-06-10T00:54:22.556499+0000","last_became_active":"2024-06-10T00:52:32.760531+0000","last_became_peered":"2024-06-10T00:52:32.760531+0000","last_unstale":"2024-06-10T00:54:22.556499+0000","last_undegraded":"2024-06-10T00:54:22.556499+0000","last_fullsized":"2024-06-10T00:54:22.556499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:05:59.052251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.371793+0000","last_change":"2024-06-10T00:52:35.967626+0000","last_active":"2024-06-10T00:55:20.371793+0000","last_peered":"2024-06-10T00:55:20.371793+0000","last_clean":"2024-06-10T00:55:20.371793+0000","last_became_active":"2024-06-10T00:52:35.967295+0000","last_became_peered":"2024-06-10T00:52:35.967295+0000","last_unstale":"2024-06-10T00:55:20.371793+0000","last_undegraded":"2024-06-10T00:55:20.371793+0000","last_fullsized":"2024-06-10T00:55:20.371793+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:07:55.054327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668105+0000","last_change":"2024-06-10T00:52:45.993974+0000","last_active":"2024-06-10T00:54:22.668105+0000","last_peered":"2024-06-10T00:54:22.668105+0000","last_clean":"2024-06-10T00:54:22.668105+0000","last_became_active":"2024-06-10T00:52:45.993242+0000","last_became_peered":"2024-06-10T00:52:45.993242+0000","last_unstale":"2024-06-10T00:54:22.668105+0000","last_undegraded":"2024-06-10T00:54:22.668105+0000","last_fullsized":"2024-06-10T00:54:22.668105+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:28:12.810368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:33.628114+0000","last_change":"2024-06-10T00:52:32.959905+0000","last_active":"2024-06-10T00:54:33.628114+0000","last_peered":"2024-06-10T00:54:33.628114+0000","last_clean":"2024-06-10T00:54:33.628114+0000","last_became_active":"2024-06-10T00:52:32.959551+0000","last_became_peered":"2024-06-10T00:52:32.959551+0000","last_unstale":"2024-06-10T00:54:33.628114+0000","last_undegraded":"2024-06-10T00:54:33.628114+0000","last_fullsized":"2024-06-10T00:54:33.628114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:33:48.029129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375782+0000","last_change":"2024-06-10T00:52:35.965790+0000","last_active":"2024-06-10T00:55:20.375782+0000","last_peered":"2024-06-10T00:55:20.375782+0000","last_clean":"2024-06-10T00:55:20.375782+0000","last_became_active":"2024-06-10T00:52:35.965475+0000","last_became_peered":"2024-06-10T00:52:35.965475+0000","last_unstale":"2024-06-10T00:55:20.375782+0000","last_undegraded":"2024-06-10T00:55:20.375782+0000","last_fullsized":"2024-06-10T00:55:20.375782+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:24:02.894411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.667953+0000","last_change":"2024-06-10T00:52:45.994464+0000","last_active":"2024-06-10T00:54:22.667953+0000","last_peered":"2024-06-10T00:54:22.667953+0000","last_clean":"2024-06-10T00:54:22.667953+0000","last_became_active":"2024-06-10T00:52:45.994022+0000","last_became_peered":"2024-06-10T00:52:45.994022+0000","last_unstale":"2024-06-10T00:54:22.667953+0000","last_undegraded":"2024-06-10T00:54:22.667953+0000","last_fullsized":"2024-06-10T00:54:22.667953+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:27:25.063074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556658+0000","last_change":"2024-06-10T00:52:32.763885+0000","last_active":"2024-06-10T00:54:22.556658+0000","last_peered":"2024-06-10T00:54:22.556658+0000","last_clean":"2024-06-10T00:54:22.556658+0000","last_became_active":"2024-06-10T00:52:32.763606+0000","last_became_peered":"2024-06-10T00:52:32.763606+0000","last_unstale":"2024-06-10T00:54:22.556658+0000","last_undegraded":"2024-06-10T00:54:22.556658+0000","last_fullsized":"2024-06-10T00:54:22.556658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:01:58.161168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372890+0000","last_change":"2024-06-10T00:52:35.966948+0000","last_active":"2024-06-10T00:55:20.372890+0000","last_peered":"2024-06-10T00:55:20.372890+0000","last_clean":"2024-06-10T00:55:20.372890+0000","last_became_active":"2024-06-10T00:52:35.966345+0000","last_became_peered":"2024-06-10T00:52:35.966345+0000","last_unstale":"2024-06-10T00:55:20.372890+0000","last_undegraded":"2024-06-10T00:55:20.372890+0000","last_fullsized":"2024-06-10T00:55:20.372890+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:58:27.169201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.667762+0000","last_change":"2024-06-10T00:52:45.993645+0000","last_active":"2024-06-10T00:54:22.667762+0000","last_peered":"2024-06-10T00:54:22.667762+0000","last_clean":"2024-06-10T00:54:22.667762+0000","last_became_active":"2024-06-10T00:52:45.993183+0000","last_became_peered":"2024-06-10T00:52:45.993183+0000","last_unstale":"2024-06-10T00:54:22.667762+0000","last_undegraded":"2024-06-10T00:54:22.667762+0000","last_fullsized":"2024-06-10T00:54:22.667762+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:47:00.541427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.565600+0000","last_change":"2024-06-10T00:54:22.565600+0000","last_active":"2024-06-10T00:54:22.565600+0000","last_peered":"2024-06-10T00:54:22.565600+0000","last_clean":"2024-06-10T00:54:22.565600+0000","last_became_active":"2024-06-10T00:54:22.562561+0000","last_became_peered":"2024-06-10T00:54:22.562561+0000","last_unstale":"2024-06-10T00:54:22.565600+0000","last_undegraded":"2024-06-10T00:54:22.565600+0000","last_fullsized":"2024-06-10T00:54:22.565600+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:47:58.748050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"100'15","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365162+0000","last_change":"2024-06-10T00:53:22.199668+0000","last_active":"2024-06-10T00:55:20.365162+0000","last_peered":"2024-06-10T00:55:20.365162+0000","last_clean":"2024-06-10T00:55:20.365162+0000","last_became_active":"2024-06-10T00:53:22.199329+0000","last_became_peered":"2024-06-10T00:53:22.199329+0000","last_unstale":"2024-06-10T00:55:20.365162+0000","last_undegraded":"2024-06-10T00:55:20.365162+0000","last_fullsized":"2024-06-10T00:55:20.365162+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:37.640478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554689+0000","last_change":"2024-06-10T00:52:45.995767+0000","last_active":"2024-06-10T00:54:22.554689+0000","last_peered":"2024-06-10T00:54:22.554689+0000","last_clean":"2024-06-10T00:54:22.554689+0000","last_became_active":"2024-06-10T00:52:45.994659+0000","last_became_peered":"2024-06-10T00:52:45.994659+0000","last_unstale":"2024-06-10T00:54:22.554689+0000","last_undegraded":"2024-06-10T00:54:22.554689+0000","last_fullsized":"2024-06-10T00:54:22.554689+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:03:45.652248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'8","reported_seq":227,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:30.554987+0000","last_change":"2024-06-10T00:52:32.761637+0000","last_active":"2024-06-10T00:56:30.554987+0000","last_peered":"2024-06-10T00:56:30.554987+0000","last_clean":"2024-06-10T00:56:30.554987+0000","last_became_active":"2024-06-10T00:52:32.761144+0000","last_became_peered":"2024-06-10T00:52:32.761144+0000","last_unstale":"2024-06-10T00:56:30.554987+0000","last_undegraded":"2024-06-10T00:56:30.554987+0000","last_fullsized":"2024-06-10T00:56:30.554987+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:55:05.616564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1461,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.362574+0000","last_change":"2024-06-10T00:52:35.962683+0000","last_active":"2024-06-10T00:55:20.362574+0000","last_peered":"2024-06-10T00:55:20.362574+0000","last_clean":"2024-06-10T00:55:20.362574+0000","last_became_active":"2024-06-10T00:52:35.962327+0000","last_became_peered":"2024-06-10T00:52:35.962327+0000","last_unstale":"2024-06-10T00:55:20.362574+0000","last_undegraded":"2024-06-10T00:55:20.362574+0000","last_fullsized":"2024-06-10T00:55:20.362574+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:38:19.063306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554563+0000","last_change":"2024-06-10T00:52:45.996882+0000","last_active":"2024-06-10T00:54:22.554563+0000","last_peered":"2024-06-10T00:54:22.554563+0000","last_clean":"2024-06-10T00:54:22.554563+0000","last_became_active":"2024-06-10T00:52:45.995767+0000","last_became_peered":"2024-06-10T00:52:45.995767+0000","last_unstale":"2024-06-10T00:54:22.554563+0000","last_undegraded":"2024-06-10T00:54:22.554563+0000","last_fullsized":"2024-06-10T00:54:22.554563+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:28:03.766944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553970+0000","last_change":"2024-06-10T00:53:22.200546+0000","last_active":"2024-06-10T00:54:22.553970+0000","last_peered":"2024-06-10T00:54:22.553970+0000","last_clean":"2024-06-10T00:54:22.553970+0000","last_became_active":"2024-06-10T00:53:22.200059+0000","last_became_peered":"2024-06-10T00:53:22.200059+0000","last_unstale":"2024-06-10T00:54:22.553970+0000","last_undegraded":"2024-06-10T00:54:22.553970+0000","last_fullsized":"2024-06-10T00:54:22.553970+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:08:47.682594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.356418+0000","last_change":"2024-06-10T00:52:35.963105+0000","last_active":"2024-06-10T00:55:20.356418+0000","last_peered":"2024-06-10T00:55:20.356418+0000","last_clean":"2024-06-10T00:55:20.356418+0000","last_became_active":"2024-06-10T00:52:35.962627+0000","last_became_peered":"2024-06-10T00:52:35.962627+0000","last_unstale":"2024-06-10T00:55:20.356418+0000","last_undegraded":"2024-06-10T00:55:20.356418+0000","last_fullsized":"2024-06-10T00:55:20.356418+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:55:51.231795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.559190+0000","last_change":"2024-06-10T00:52:45.992557+0000","last_active":"2024-06-10T00:54:22.559190+0000","last_peered":"2024-06-10T00:54:22.559190+0000","last_clean":"2024-06-10T00:54:22.559190+0000","last_became_active":"2024-06-10T00:52:45.992166+0000","last_became_peered":"2024-06-10T00:52:45.992166+0000","last_unstale":"2024-06-10T00:54:22.559190+0000","last_undegraded":"2024-06-10T00:54:22.559190+0000","last_fullsized":"2024-06-10T00:54:22.559190+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:49:22.368409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553428+0000","last_change":"2024-06-10T00:52:32.751867+0000","last_active":"2024-06-10T00:54:22.553428+0000","last_peered":"2024-06-10T00:54:22.553428+0000","last_clean":"2024-06-10T00:54:22.553428+0000","last_became_active":"2024-06-10T00:52:32.751516+0000","last_became_peered":"2024-06-10T00:52:32.751516+0000","last_unstale":"2024-06-10T00:54:22.553428+0000","last_undegraded":"2024-06-10T00:54:22.553428+0000","last_fullsized":"2024-06-10T00:54:22.553428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:37:55.920613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.370097+0000","last_change":"2024-06-10T00:52:35.957772+0000","last_active":"2024-06-10T00:55:20.370097+0000","last_peered":"2024-06-10T00:55:20.370097+0000","last_clean":"2024-06-10T00:55:20.370097+0000","last_became_active":"2024-06-10T00:52:35.957470+0000","last_became_peered":"2024-06-10T00:52:35.957470+0000","last_unstale":"2024-06-10T00:55:20.370097+0000","last_undegraded":"2024-06-10T00:55:20.370097+0000","last_fullsized":"2024-06-10T00:55:20.370097+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:16:43.645292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878568+0000","last_change":"2024-06-10T00:52:45.996807+0000","last_active":"2024-06-10T00:54:22.878568+0000","last_peered":"2024-06-10T00:54:22.878568+0000","last_clean":"2024-06-10T00:54:22.878568+0000","last_became_active":"2024-06-10T00:52:45.996378+0000","last_became_peered":"2024-06-10T00:52:45.996378+0000","last_unstale":"2024-06-10T00:54:22.878568+0000","last_undegraded":"2024-06-10T00:54:22.878568+0000","last_fullsized":"2024-06-10T00:54:22.878568+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:40:54.251790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'1","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561873+0000","last_change":"2024-06-10T00:52:32.759625+0000","last_active":"2024-06-10T00:54:22.561873+0000","last_peered":"2024-06-10T00:54:22.561873+0000","last_clean":"2024-06-10T00:54:22.561873+0000","last_became_active":"2024-06-10T00:52:32.759299+0000","last_became_peered":"2024-06-10T00:52:32.759299+0000","last_unstale":"2024-06-10T00:54:22.561873+0000","last_undegraded":"2024-06-10T00:54:22.561873+0000","last_fullsized":"2024-06-10T00:54:22.561873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:11:44.966453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365673+0000","last_change":"2024-06-10T00:52:35.957672+0000","last_active":"2024-06-10T00:55:20.365673+0000","last_peered":"2024-06-10T00:55:20.365673+0000","last_clean":"2024-06-10T00:55:20.365673+0000","last_became_active":"2024-06-10T00:52:35.957328+0000","last_became_peered":"2024-06-10T00:52:35.957328+0000","last_unstale":"2024-06-10T00:55:20.365673+0000","last_undegraded":"2024-06-10T00:55:20.365673+0000","last_fullsized":"2024-06-10T00:55:20.365673+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:43:59.658178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601278+0000","last_change":"2024-06-10T00:52:45.996677+0000","last_active":"2024-06-10T00:54:22.601278+0000","last_peered":"2024-06-10T00:54:22.601278+0000","last_clean":"2024-06-10T00:54:22.601278+0000","last_became_active":"2024-06-10T00:52:45.994483+0000","last_became_peered":"2024-06-10T00:52:45.994483+0000","last_unstale":"2024-06-10T00:54:22.601278+0000","last_undegraded":"2024-06-10T00:54:22.601278+0000","last_fullsized":"2024-06-10T00:54:22.601278+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:03:28.712072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'5","reported_seq":225,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:34.560617+0000","last_change":"2024-06-10T00:52:32.754829+0000","last_active":"2024-06-10T00:56:34.560617+0000","last_peered":"2024-06-10T00:56:34.560617+0000","last_clean":"2024-06-10T00:56:34.560617+0000","last_became_active":"2024-06-10T00:52:32.754386+0000","last_became_peered":"2024-06-10T00:52:32.754386+0000","last_unstale":"2024-06-10T00:56:34.560617+0000","last_undegraded":"2024-06-10T00:56:34.560617+0000","last_fullsized":"2024-06-10T00:56:34.560617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:00.870324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":175,"num_read_kb":175,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.371610+0000","last_change":"2024-06-10T00:52:35.966871+0000","last_active":"2024-06-10T00:55:20.371610+0000","last_peered":"2024-06-10T00:55:20.371610+0000","last_clean":"2024-06-10T00:55:20.371610+0000","last_became_active":"2024-06-10T00:52:35.966492+0000","last_became_peered":"2024-06-10T00:52:35.966492+0000","last_unstale":"2024-06-10T00:55:20.371610+0000","last_undegraded":"2024-06-10T00:55:20.371610+0000","last_fullsized":"2024-06-10T00:55:20.371610+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:24:41.048935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.879950+0000","last_change":"2024-06-10T00:54:22.879457+0000","last_active":"2024-06-10T00:54:22.879950+0000","last_peered":"2024-06-10T00:54:22.879950+0000","last_clean":"2024-06-10T00:54:22.879950+0000","last_became_active":"2024-06-10T00:54:22.878978+0000","last_became_peered":"2024-06-10T00:54:22.878978+0000","last_unstale":"2024-06-10T00:54:22.879950+0000","last_undegraded":"2024-06-10T00:54:22.879950+0000","last_fullsized":"2024-06-10T00:54:22.879950+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:46:38.819250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404659+0000","last_change":"2024-06-10T00:52:32.765794+0000","last_active":"2024-06-10T00:54:24.404659+0000","last_peered":"2024-06-10T00:54:24.404659+0000","last_clean":"2024-06-10T00:54:24.404659+0000","last_became_active":"2024-06-10T00:52:32.765276+0000","last_became_peered":"2024-06-10T00:52:32.765276+0000","last_unstale":"2024-06-10T00:54:24.404659+0000","last_undegraded":"2024-06-10T00:54:24.404659+0000","last_fullsized":"2024-06-10T00:54:24.404659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:06:51.209012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365186+0000","last_change":"2024-06-10T00:52:35.954897+0000","last_active":"2024-06-10T00:55:20.365186+0000","last_peered":"2024-06-10T00:55:20.365186+0000","last_clean":"2024-06-10T00:55:20.365186+0000","last_became_active":"2024-06-10T00:52:35.954565+0000","last_became_peered":"2024-06-10T00:52:35.954565+0000","last_unstale":"2024-06-10T00:55:20.365186+0000","last_undegraded":"2024-06-10T00:55:20.365186+0000","last_fullsized":"2024-06-10T00:55:20.365186+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:39:26.702804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668190+0000","last_change":"2024-06-10T00:52:45.992194+0000","last_active":"2024-06-10T00:54:22.668190+0000","last_peered":"2024-06-10T00:54:22.668190+0000","last_clean":"2024-06-10T00:54:22.668190+0000","last_became_active":"2024-06-10T00:52:45.991453+0000","last_became_peered":"2024-06-10T00:52:45.991453+0000","last_unstale":"2024-06-10T00:54:22.668190+0000","last_undegraded":"2024-06-10T00:54:22.668190+0000","last_fullsized":"2024-06-10T00:54:22.668190+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:11:10.775610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561610+0000","last_change":"2024-06-10T00:52:32.962214+0000","last_active":"2024-06-10T00:54:22.561610+0000","last_peered":"2024-06-10T00:54:22.561610+0000","last_clean":"2024-06-10T00:54:22.561610+0000","last_became_active":"2024-06-10T00:52:32.961835+0000","last_became_peered":"2024-06-10T00:52:32.961835+0000","last_unstale":"2024-06-10T00:54:22.561610+0000","last_undegraded":"2024-06-10T00:54:22.561610+0000","last_fullsized":"2024-06-10T00:54:22.561610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:36:44.724122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.377485+0000","last_change":"2024-06-10T00:52:35.963413+0000","last_active":"2024-06-10T00:55:20.377485+0000","last_peered":"2024-06-10T00:55:20.377485+0000","last_clean":"2024-06-10T00:55:20.377485+0000","last_became_active":"2024-06-10T00:52:35.962922+0000","last_became_peered":"2024-06-10T00:52:35.962922+0000","last_unstale":"2024-06-10T00:55:20.377485+0000","last_undegraded":"2024-06-10T00:55:20.377485+0000","last_fullsized":"2024-06-10T00:55:20.377485+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:05:32.164696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":34,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.560580+0000","last_change":"2024-06-10T00:54:22.560196+0000","last_active":"2024-06-10T00:54:22.560580+0000","last_peered":"2024-06-10T00:54:22.560580+0000","last_clean":"2024-06-10T00:54:22.560580+0000","last_became_active":"2024-06-10T00:54:22.559826+0000","last_became_peered":"2024-06-10T00:54:22.559826+0000","last_unstale":"2024-06-10T00:54:22.560580+0000","last_undegraded":"2024-06-10T00:54:22.560580+0000","last_fullsized":"2024-06-10T00:54:22.560580+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:09:28.456394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'46","reported_seq":455,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:35.566931+0000","last_change":"2024-06-10T00:52:32.761592+0000","last_active":"2024-06-10T00:56:35.566931+0000","last_peered":"2024-06-10T00:56:35.566931+0000","last_clean":"2024-06-10T00:56:35.566931+0000","last_became_active":"2024-06-10T00:52:32.761169+0000","last_became_peered":"2024-06-10T00:52:32.761169+0000","last_unstale":"2024-06-10T00:56:35.566931+0000","last_undegraded":"2024-06-10T00:56:35.566931+0000","last_fullsized":"2024-06-10T00:56:35.566931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:08:33.015035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77873,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":364,"num_read_kb":260,"num_write":47,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.351972+0000","last_change":"2024-06-10T00:52:35.955304+0000","last_active":"2024-06-10T00:55:20.351972+0000","last_peered":"2024-06-10T00:55:20.351972+0000","last_clean":"2024-06-10T00:55:20.351972+0000","last_became_active":"2024-06-10T00:52:35.954949+0000","last_became_peered":"2024-06-10T00:52:35.954949+0000","last_unstale":"2024-06-10T00:55:20.351972+0000","last_undegraded":"2024-06-10T00:55:20.351972+0000","last_fullsized":"2024-06-10T00:55:20.351972+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:56:42.890022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561827+0000","last_change":"2024-06-10T00:52:45.992974+0000","last_active":"2024-06-10T00:54:22.561827+0000","last_peered":"2024-06-10T00:54:22.561827+0000","last_clean":"2024-06-10T00:54:22.561827+0000","last_became_active":"2024-06-10T00:52:45.992490+0000","last_became_peered":"2024-06-10T00:52:45.992490+0000","last_unstale":"2024-06-10T00:54:22.561827+0000","last_undegraded":"2024-06-10T00:54:22.561827+0000","last_fullsized":"2024-06-10T00:54:22.561827+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:03:45.543476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554437+0000","last_change":"2024-06-10T00:52:32.762347+0000","last_active":"2024-06-10T00:54:22.554437+0000","last_peered":"2024-06-10T00:54:22.554437+0000","last_clean":"2024-06-10T00:54:22.554437+0000","last_became_active":"2024-06-10T00:52:32.761049+0000","last_became_peered":"2024-06-10T00:52:32.761049+0000","last_unstale":"2024-06-10T00:54:22.554437+0000","last_undegraded":"2024-06-10T00:54:22.554437+0000","last_fullsized":"2024-06-10T00:54:22.554437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:29:38.384451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364495+0000","last_change":"2024-06-10T00:52:35.961299+0000","last_active":"2024-06-10T00:55:20.364495+0000","last_peered":"2024-06-10T00:55:20.364495+0000","last_clean":"2024-06-10T00:55:20.364495+0000","last_became_active":"2024-06-10T00:52:35.960966+0000","last_became_peered":"2024-06-10T00:52:35.960966+0000","last_unstale":"2024-06-10T00:55:20.364495+0000","last_undegraded":"2024-06-10T00:55:20.364495+0000","last_fullsized":"2024-06-10T00:55:20.364495+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:32:21.269689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.559474+0000","last_change":"2024-06-10T00:52:46.002091+0000","last_active":"2024-06-10T00:54:22.559474+0000","last_peered":"2024-06-10T00:54:22.559474+0000","last_clean":"2024-06-10T00:54:22.559474+0000","last_became_active":"2024-06-10T00:52:46.001677+0000","last_became_peered":"2024-06-10T00:52:46.001677+0000","last_unstale":"2024-06-10T00:54:22.559474+0000","last_undegraded":"2024-06-10T00:54:22.559474+0000","last_fullsized":"2024-06-10T00:54:22.559474+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:16:56.405720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'3","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.562287+0000","last_change":"2024-06-10T00:52:32.756765+0000","last_active":"2024-06-10T00:54:22.562287+0000","last_peered":"2024-06-10T00:54:22.562287+0000","last_clean":"2024-06-10T00:54:22.562287+0000","last_became_active":"2024-06-10T00:52:32.756470+0000","last_became_peered":"2024-06-10T00:52:32.756470+0000","last_unstale":"2024-06-10T00:54:22.562287+0000","last_undegraded":"2024-06-10T00:54:22.562287+0000","last_fullsized":"2024-06-10T00:54:22.562287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:29:24.854913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"100'15","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364256+0000","last_change":"2024-06-10T00:52:35.960693+0000","last_active":"2024-06-10T00:55:20.364256+0000","last_peered":"2024-06-10T00:55:20.364256+0000","last_clean":"2024-06-10T00:55:20.364256+0000","last_became_active":"2024-06-10T00:52:35.958397+0000","last_became_peered":"2024-06-10T00:52:35.958397+0000","last_unstale":"2024-06-10T00:55:20.364256+0000","last_undegraded":"2024-06-10T00:55:20.364256+0000","last_fullsized":"2024-06-10T00:55:20.364256+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:23:47.556462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556200+0000","last_change":"2024-06-10T00:52:46.001475+0000","last_active":"2024-06-10T00:54:22.556200+0000","last_peered":"2024-06-10T00:54:22.556200+0000","last_clean":"2024-06-10T00:54:22.556200+0000","last_became_active":"2024-06-10T00:52:46.000983+0000","last_became_peered":"2024-06-10T00:52:46.000983+0000","last_unstale":"2024-06-10T00:54:22.556200+0000","last_undegraded":"2024-06-10T00:54:22.556200+0000","last_fullsized":"2024-06-10T00:54:22.556200+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:23:56.285356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"98'3","reported_seq":221,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:31.559742+0000","last_change":"2024-06-10T00:52:32.756665+0000","last_active":"2024-06-10T00:56:31.559742+0000","last_peered":"2024-06-10T00:56:31.559742+0000","last_clean":"2024-06-10T00:56:31.559742+0000","last_became_active":"2024-06-10T00:52:32.756221+0000","last_became_peered":"2024-06-10T00:52:32.756221+0000","last_unstale":"2024-06-10T00:56:31.559742+0000","last_undegraded":"2024-06-10T00:56:31.559742+0000","last_fullsized":"2024-06-10T00:56:31.559742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:50:34.789063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":175,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"100'6","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364652+0000","last_change":"2024-06-10T00:53:22.125844+0000","last_active":"2024-06-10T00:55:20.364652+0000","last_peered":"2024-06-10T00:55:20.364652+0000","last_clean":"2024-06-10T00:55:20.364652+0000","last_became_active":"2024-06-10T00:53:22.125449+0000","last_became_peered":"2024-06-10T00:53:22.125449+0000","last_unstale":"2024-06-10T00:55:20.364652+0000","last_undegraded":"2024-06-10T00:55:20.364652+0000","last_fullsized":"2024-06-10T00:55:20.364652+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:08:42.936702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553904+0000","last_change":"2024-06-10T00:52:45.999300+0000","last_active":"2024-06-10T00:54:22.553904+0000","last_peered":"2024-06-10T00:54:22.553904+0000","last_clean":"2024-06-10T00:54:22.553904+0000","last_became_active":"2024-06-10T00:52:45.998860+0000","last_became_peered":"2024-06-10T00:52:45.998860+0000","last_unstale":"2024-06-10T00:54:22.553904+0000","last_undegraded":"2024-06-10T00:54:22.553904+0000","last_fullsized":"2024-06-10T00:54:22.553904+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:14:24.936870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":75,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.557182+0000","last_change":"2024-06-10T00:53:22.210710+0000","last_active":"2024-06-10T00:54:22.557182+0000","last_peered":"2024-06-10T00:54:22.557182+0000","last_clean":"2024-06-10T00:54:22.557182+0000","last_became_active":"2024-06-10T00:53:22.202461+0000","last_became_peered":"2024-06-10T00:53:22.202461+0000","last_unstale":"2024-06-10T00:54:22.557182+0000","last_undegraded":"2024-06-10T00:54:22.557182+0000","last_fullsized":"2024-06-10T00:54:22.557182+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:28:14.671776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364560+0000","last_change":"2024-06-10T00:52:35.966960+0000","last_active":"2024-06-10T00:55:20.364560+0000","last_peered":"2024-06-10T00:55:20.364560+0000","last_clean":"2024-06-10T00:55:20.364560+0000","last_became_active":"2024-06-10T00:52:35.966630+0000","last_became_peered":"2024-06-10T00:52:35.966630+0000","last_unstale":"2024-06-10T00:55:20.364560+0000","last_undegraded":"2024-06-10T00:55:20.364560+0000","last_fullsized":"2024-06-10T00:55:20.364560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:04.117405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561586+0000","last_change":"2024-06-10T00:52:45.992955+0000","last_active":"2024-06-10T00:54:22.561586+0000","last_peered":"2024-06-10T00:54:22.561586+0000","last_clean":"2024-06-10T00:54:22.561586+0000","last_became_active":"2024-06-10T00:52:45.992457+0000","last_became_peered":"2024-06-10T00:52:45.992457+0000","last_unstale":"2024-06-10T00:54:22.561586+0000","last_undegraded":"2024-06-10T00:54:22.561586+0000","last_fullsized":"2024-06-10T00:54:22.561586+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:13:49.491183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556768+0000","last_change":"2024-06-10T00:52:32.763794+0000","last_active":"2024-06-10T00:54:22.556768+0000","last_peered":"2024-06-10T00:54:22.556768+0000","last_clean":"2024-06-10T00:54:22.556768+0000","last_became_active":"2024-06-10T00:52:32.763406+0000","last_became_peered":"2024-06-10T00:52:32.763406+0000","last_unstale":"2024-06-10T00:54:22.556768+0000","last_undegraded":"2024-06-10T00:54:22.556768+0000","last_fullsized":"2024-06-10T00:54:22.556768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:50:33.580163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"100'5","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374120+0000","last_change":"2024-06-10T00:53:22.130891+0000","last_active":"2024-06-10T00:55:20.374120+0000","last_peered":"2024-06-10T00:55:20.374120+0000","last_clean":"2024-06-10T00:55:20.374120+0000","last_became_active":"2024-06-10T00:53:22.130508+0000","last_became_peered":"2024-06-10T00:53:22.130508+0000","last_unstale":"2024-06-10T00:55:20.374120+0000","last_undegraded":"2024-06-10T00:55:20.374120+0000","last_fullsized":"2024-06-10T00:55:20.374120+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:28:01.919825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":34,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.563757+0000","last_change":"2024-06-10T00:54:22.563272+0000","last_active":"2024-06-10T00:54:22.563757+0000","last_peered":"2024-06-10T00:54:22.563757+0000","last_clean":"2024-06-10T00:54:22.563757+0000","last_became_active":"2024-06-10T00:54:22.562767+0000","last_became_peered":"2024-06-10T00:54:22.562767+0000","last_unstale":"2024-06-10T00:54:22.563757+0000","last_undegraded":"2024-06-10T00:54:22.563757+0000","last_fullsized":"2024-06-10T00:54:22.563757+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:52:20.739801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556364+0000","last_change":"2024-06-10T00:52:32.960515+0000","last_active":"2024-06-10T00:54:22.556364+0000","last_peered":"2024-06-10T00:54:22.556364+0000","last_clean":"2024-06-10T00:54:22.556364+0000","last_became_active":"2024-06-10T00:52:32.960163+0000","last_became_peered":"2024-06-10T00:52:32.960163+0000","last_unstale":"2024-06-10T00:54:22.556364+0000","last_undegraded":"2024-06-10T00:54:22.556364+0000","last_fullsized":"2024-06-10T00:54:22.556364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:39:51.374998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.359152+0000","last_change":"2024-06-10T00:52:35.961884+0000","last_active":"2024-06-10T00:55:20.359152+0000","last_peered":"2024-06-10T00:55:20.359152+0000","last_clean":"2024-06-10T00:55:20.359152+0000","last_became_active":"2024-06-10T00:52:35.961497+0000","last_became_peered":"2024-06-10T00:52:35.961497+0000","last_unstale":"2024-06-10T00:55:20.359152+0000","last_undegraded":"2024-06-10T00:55:20.359152+0000","last_fullsized":"2024-06-10T00:55:20.359152+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:16:09.284184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553591+0000","last_change":"2024-06-10T00:52:45.990846+0000","last_active":"2024-06-10T00:54:22.553591+0000","last_peered":"2024-06-10T00:54:22.553591+0000","last_clean":"2024-06-10T00:54:22.553591+0000","last_became_active":"2024-06-10T00:52:45.990441+0000","last_became_peered":"2024-06-10T00:52:45.990441+0000","last_unstale":"2024-06-10T00:54:22.553591+0000","last_undegraded":"2024-06-10T00:54:22.553591+0000","last_fullsized":"2024-06-10T00:54:22.553591+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:19:35.690378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404369+0000","last_change":"2024-06-10T00:52:32.753913+0000","last_active":"2024-06-10T00:54:24.404369+0000","last_peered":"2024-06-10T00:54:24.404369+0000","last_clean":"2024-06-10T00:54:24.404369+0000","last_became_active":"2024-06-10T00:52:32.753174+0000","last_became_peered":"2024-06-10T00:52:32.753174+0000","last_unstale":"2024-06-10T00:54:24.404369+0000","last_undegraded":"2024-06-10T00:54:24.404369+0000","last_fullsized":"2024-06-10T00:54:24.404369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:13:20.048635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.359023+0000","last_change":"2024-06-10T00:52:35.965783+0000","last_active":"2024-06-10T00:55:20.359023+0000","last_peered":"2024-06-10T00:55:20.359023+0000","last_clean":"2024-06-10T00:55:20.359023+0000","last_became_active":"2024-06-10T00:52:35.965432+0000","last_became_peered":"2024-06-10T00:52:35.965432+0000","last_unstale":"2024-06-10T00:55:20.359023+0000","last_undegraded":"2024-06-10T00:55:20.359023+0000","last_fullsized":"2024-06-10T00:55:20.359023+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:52:01.240318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553790+0000","last_change":"2024-06-10T00:52:46.241662+0000","last_active":"2024-06-10T00:54:22.553790+0000","last_peered":"2024-06-10T00:54:22.553790+0000","last_clean":"2024-06-10T00:54:22.553790+0000","last_became_active":"2024-06-10T00:52:46.241211+0000","last_became_peered":"2024-06-10T00:52:46.241211+0000","last_unstale":"2024-06-10T00:54:22.553790+0000","last_undegraded":"2024-06-10T00:54:22.553790+0000","last_fullsized":"2024-06-10T00:54:22.553790+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:12.041290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"100'8","reported_seq":24,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.367303+0000","last_change":"2024-06-10T00:53:22.200590+0000","last_active":"2024-06-10T00:55:20.367303+0000","last_peered":"2024-06-10T00:55:20.367303+0000","last_clean":"2024-06-10T00:55:20.367303+0000","last_became_active":"2024-06-10T00:53:22.200046+0000","last_became_peered":"2024-06-10T00:53:22.200046+0000","last_unstale":"2024-06-10T00:55:20.367303+0000","last_undegraded":"2024-06-10T00:55:20.367303+0000","last_fullsized":"2024-06-10T00:55:20.367303+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:43:22.378316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"98'4","reported_seq":220,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:31.559275+0000","last_change":"2024-06-10T00:52:32.959904+0000","last_active":"2024-06-10T00:56:31.559275+0000","last_peered":"2024-06-10T00:56:31.559275+0000","last_clean":"2024-06-10T00:56:31.559275+0000","last_became_active":"2024-06-10T00:52:32.959506+0000","last_became_peered":"2024-06-10T00:52:32.959506+0000","last_unstale":"2024-06-10T00:56:31.559275+0000","last_undegraded":"2024-06-10T00:56:31.559275+0000","last_fullsized":"2024-06-10T00:56:31.559275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:23:15.950345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":172,"num_read_kb":172,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404433+0000","last_change":"2024-06-10T00:52:45.993305+0000","last_active":"2024-06-10T00:54:24.404433+0000","last_peered":"2024-06-10T00:54:24.404433+0000","last_clean":"2024-06-10T00:54:24.404433+0000","last_became_active":"2024-06-10T00:52:45.992682+0000","last_became_peered":"2024-06-10T00:52:45.992682+0000","last_unstale":"2024-06-10T00:54:24.404433+0000","last_undegraded":"2024-06-10T00:54:24.404433+0000","last_fullsized":"2024-06-10T00:54:24.404433+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:49:41.817126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375771+0000","last_change":"2024-06-10T00:52:35.965893+0000","last_active":"2024-06-10T00:55:20.375771+0000","last_peered":"2024-06-10T00:55:20.375771+0000","last_clean":"2024-06-10T00:55:20.375771+0000","last_became_active":"2024-06-10T00:52:35.965426+0000","last_became_peered":"2024-06-10T00:52:35.965426+0000","last_unstale":"2024-06-10T00:55:20.375771+0000","last_undegraded":"2024-06-10T00:55:20.375771+0000","last_fullsized":"2024-06-10T00:55:20.375771+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:29:19.640836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668002+0000","last_change":"2024-06-10T00:52:32.753631+0000","last_active":"2024-06-10T00:54:22.668002+0000","last_peered":"2024-06-10T00:54:22.668002+0000","last_clean":"2024-06-10T00:54:22.668002+0000","last_became_active":"2024-06-10T00:52:32.753260+0000","last_became_peered":"2024-06-10T00:52:32.753260+0000","last_unstale":"2024-06-10T00:54:22.668002+0000","last_undegraded":"2024-06-10T00:54:22.668002+0000","last_fullsized":"2024-06-10T00:54:22.668002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:12:55.932676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":369,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404055+0000","last_change":"2024-06-10T00:51:09.267750+0000","last_active":"2024-06-10T00:54:24.404055+0000","last_peered":"2024-06-10T00:54:24.404055+0000","last_clean":"2024-06-10T00:54:24.404055+0000","last_became_active":"2024-06-10T00:51:09.267246+0000","last_became_peered":"2024-06-10T00:51:09.267246+0000","last_unstale":"2024-06-10T00:54:24.404055+0000","last_undegraded":"2024-06-10T00:54:24.404055+0000","last_fullsized":"2024-06-10T00:54:24.404055+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:47:44.965452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:47:44.965452+0000","last_clean_scrub_stamp":"2024-06-10T00:47:44.965452+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:21.100090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561655+0000","last_change":"2024-06-10T00:52:46.242152+0000","last_active":"2024-06-10T00:54:22.561655+0000","last_peered":"2024-06-10T00:54:22.561655+0000","last_clean":"2024-06-10T00:54:22.561655+0000","last_became_active":"2024-06-10T00:52:46.241542+0000","last_became_peered":"2024-06-10T00:52:46.241542+0000","last_unstale":"2024-06-10T00:54:22.561655+0000","last_undegraded":"2024-06-10T00:54:22.561655+0000","last_fullsized":"2024-06-10T00:54:22.561655+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:55:54.695465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364883+0000","last_change":"2024-06-10T00:52:35.963911+0000","last_active":"2024-06-10T00:55:20.364883+0000","last_peered":"2024-06-10T00:55:20.364883+0000","last_clean":"2024-06-10T00:55:20.364883+0000","last_became_active":"2024-06-10T00:52:35.963571+0000","last_became_peered":"2024-06-10T00:52:35.963571+0000","last_unstale":"2024-06-10T00:55:20.364883+0000","last_undegraded":"2024-06-10T00:55:20.364883+0000","last_fullsized":"2024-06-10T00:55:20.364883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:27:16.757428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.558529+0000","last_change":"2024-06-10T00:53:22.129035+0000","last_active":"2024-06-10T00:54:22.558529+0000","last_peered":"2024-06-10T00:54:22.558529+0000","last_clean":"2024-06-10T00:54:22.558529+0000","last_became_active":"2024-06-10T00:53:22.128561+0000","last_became_peered":"2024-06-10T00:53:22.128561+0000","last_unstale":"2024-06-10T00:54:22.558529+0000","last_undegraded":"2024-06-10T00:54:22.558529+0000","last_fullsized":"2024-06-10T00:54:22.558529+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:02:43.313118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.558735+0000","last_change":"2024-06-10T00:52:45.996907+0000","last_active":"2024-06-10T00:54:22.558735+0000","last_peered":"2024-06-10T00:54:22.558735+0000","last_clean":"2024-06-10T00:54:22.558735+0000","last_became_active":"2024-06-10T00:52:45.996554+0000","last_became_peered":"2024-06-10T00:52:45.996554+0000","last_unstale":"2024-06-10T00:54:22.558735+0000","last_undegraded":"2024-06-10T00:54:22.558735+0000","last_fullsized":"2024-06-10T00:54:22.558735+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:00:46.824071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556594+0000","last_change":"2024-06-10T00:52:32.960223+0000","last_active":"2024-06-10T00:54:22.556594+0000","last_peered":"2024-06-10T00:54:22.556594+0000","last_clean":"2024-06-10T00:54:22.556594+0000","last_became_active":"2024-06-10T00:52:32.959871+0000","last_became_peered":"2024-06-10T00:52:32.959871+0000","last_unstale":"2024-06-10T00:54:22.556594+0000","last_undegraded":"2024-06-10T00:54:22.556594+0000","last_fullsized":"2024-06-10T00:54:22.556594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:18:17.099140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.369555+0000","last_change":"2024-06-10T00:52:35.962516+0000","last_active":"2024-06-10T00:55:20.369555+0000","last_peered":"2024-06-10T00:55:20.369555+0000","last_clean":"2024-06-10T00:55:20.369555+0000","last_became_active":"2024-06-10T00:52:35.962187+0000","last_became_peered":"2024-06-10T00:52:35.962187+0000","last_unstale":"2024-06-10T00:55:20.369555+0000","last_undegraded":"2024-06-10T00:55:20.369555+0000","last_fullsized":"2024-06-10T00:55:20.369555+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:10:44.295192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601233+0000","last_change":"2024-06-10T00:52:46.241554+0000","last_active":"2024-06-10T00:54:22.601233+0000","last_peered":"2024-06-10T00:54:22.601233+0000","last_clean":"2024-06-10T00:54:22.601233+0000","last_became_active":"2024-06-10T00:52:46.241142+0000","last_became_peered":"2024-06-10T00:52:46.241142+0000","last_unstale":"2024-06-10T00:54:22.601233+0000","last_undegraded":"2024-06-10T00:54:22.601233+0000","last_fullsized":"2024-06-10T00:54:22.601233+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:58:07.233080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'13","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:29.987897+0000","last_change":"2024-06-10T00:52:32.761854+0000","last_active":"2024-06-10T00:56:29.987897+0000","last_peered":"2024-06-10T00:56:29.987897+0000","last_clean":"2024-06-10T00:56:29.987897+0000","last_became_active":"2024-06-10T00:52:32.761522+0000","last_became_peered":"2024-06-10T00:52:32.761522+0000","last_unstale":"2024-06-10T00:56:29.987897+0000","last_undegraded":"2024-06-10T00:56:29.987897+0000","last_fullsized":"2024-06-10T00:56:29.987897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:06:43.269001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.370762+0000","last_change":"2024-06-10T00:52:35.959250+0000","last_active":"2024-06-10T00:55:20.370762+0000","last_peered":"2024-06-10T00:55:20.370762+0000","last_clean":"2024-06-10T00:55:20.370762+0000","last_became_active":"2024-06-10T00:52:35.958925+0000","last_became_peered":"2024-06-10T00:52:35.958925+0000","last_unstale":"2024-06-10T00:55:20.370762+0000","last_undegraded":"2024-06-10T00:55:20.370762+0000","last_fullsized":"2024-06-10T00:55:20.370762+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:11:12.355505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554191+0000","last_change":"2024-06-10T00:52:45.991819+0000","last_active":"2024-06-10T00:54:22.554191+0000","last_peered":"2024-06-10T00:54:22.554191+0000","last_clean":"2024-06-10T00:54:22.554191+0000","last_became_active":"2024-06-10T00:52:45.991408+0000","last_became_peered":"2024-06-10T00:52:45.991408+0000","last_unstale":"2024-06-10T00:54:22.554191+0000","last_undegraded":"2024-06-10T00:54:22.554191+0000","last_fullsized":"2024-06-10T00:54:22.554191+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:16:34.912359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668551+0000","last_change":"2024-06-10T00:52:32.869094+0000","last_active":"2024-06-10T00:54:22.668551+0000","last_peered":"2024-06-10T00:54:22.668551+0000","last_clean":"2024-06-10T00:54:22.668551+0000","last_became_active":"2024-06-10T00:52:32.868664+0000","last_became_peered":"2024-06-10T00:52:32.868664+0000","last_unstale":"2024-06-10T00:54:22.668551+0000","last_undegraded":"2024-06-10T00:54:22.668551+0000","last_fullsized":"2024-06-10T00:54:22.668551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:50:38.051090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374722+0000","last_change":"2024-06-10T00:52:35.952356+0000","last_active":"2024-06-10T00:55:20.374722+0000","last_peered":"2024-06-10T00:55:20.374722+0000","last_clean":"2024-06-10T00:55:20.374722+0000","last_became_active":"2024-06-10T00:52:35.952002+0000","last_became_peered":"2024-06-10T00:52:35.952002+0000","last_unstale":"2024-06-10T00:55:20.374722+0000","last_undegraded":"2024-06-10T00:55:20.374722+0000","last_fullsized":"2024-06-10T00:55:20.374722+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:12:28.581522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668366+0000","last_change":"2024-06-10T00:52:46.241773+0000","last_active":"2024-06-10T00:54:22.668366+0000","last_peered":"2024-06-10T00:54:22.668366+0000","last_clean":"2024-06-10T00:54:22.668366+0000","last_became_active":"2024-06-10T00:52:46.241272+0000","last_became_peered":"2024-06-10T00:52:46.241272+0000","last_unstale":"2024-06-10T00:54:22.668366+0000","last_undegraded":"2024-06-10T00:54:22.668366+0000","last_fullsized":"2024-06-10T00:54:22.668366+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:07:28.337536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561360+0000","last_change":"2024-06-10T00:52:32.960100+0000","last_active":"2024-06-10T00:54:22.561360+0000","last_peered":"2024-06-10T00:54:22.561360+0000","last_clean":"2024-06-10T00:54:22.561360+0000","last_became_active":"2024-06-10T00:52:32.959779+0000","last_became_peered":"2024-06-10T00:52:32.959779+0000","last_unstale":"2024-06-10T00:54:22.561360+0000","last_undegraded":"2024-06-10T00:54:22.561360+0000","last_fullsized":"2024-06-10T00:54:22.561360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:09.555753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'2","reported_seq":40,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.326494+0000","last_change":"2024-06-10T00:52:35.955586+0000","last_active":"2024-06-10T00:55:20.326494+0000","last_peered":"2024-06-10T00:55:20.326494+0000","last_clean":"2024-06-10T00:55:20.326494+0000","last_became_active":"2024-06-10T00:52:35.955216+0000","last_became_peered":"2024-06-10T00:52:35.955216+0000","last_unstale":"2024-06-10T00:55:20.326494+0000","last_undegraded":"2024-06-10T00:55:20.326494+0000","last_fullsized":"2024-06-10T00:55:20.326494+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:23:12.197182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.603221+0000","last_change":"2024-06-10T00:54:22.602670+0000","last_active":"2024-06-10T00:54:22.603221+0000","last_peered":"2024-06-10T00:54:22.603221+0000","last_clean":"2024-06-10T00:54:22.603221+0000","last_became_active":"2024-06-10T00:54:22.602218+0000","last_became_peered":"2024-06-10T00:54:22.602218+0000","last_unstale":"2024-06-10T00:54:22.603221+0000","last_undegraded":"2024-06-10T00:54:22.603221+0000","last_fullsized":"2024-06-10T00:54:22.603221+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:07.399025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"98'3","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601358+0000","last_change":"2024-06-10T00:52:32.753444+0000","last_active":"2024-06-10T00:54:22.601358+0000","last_peered":"2024-06-10T00:54:22.601358+0000","last_clean":"2024-06-10T00:54:22.601358+0000","last_became_active":"2024-06-10T00:52:32.753149+0000","last_became_peered":"2024-06-10T00:52:32.753149+0000","last_unstale":"2024-06-10T00:54:22.601358+0000","last_undegraded":"2024-06-10T00:54:22.601358+0000","last_fullsized":"2024-06-10T00:54:22.601358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:28.241950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365900+0000","last_change":"2024-06-10T00:52:35.966718+0000","last_active":"2024-06-10T00:55:20.365900+0000","last_peered":"2024-06-10T00:55:20.365900+0000","last_clean":"2024-06-10T00:55:20.365900+0000","last_became_active":"2024-06-10T00:52:35.965907+0000","last_became_peered":"2024-06-10T00:52:35.965907+0000","last_unstale":"2024-06-10T00:55:20.365900+0000","last_undegraded":"2024-06-10T00:55:20.365900+0000","last_fullsized":"2024-06-10T00:55:20.365900+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:21:34.261306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668092+0000","last_change":"2024-06-10T00:52:45.991034+0000","last_active":"2024-06-10T00:54:22.668092+0000","last_peered":"2024-06-10T00:54:22.668092+0000","last_clean":"2024-06-10T00:54:22.668092+0000","last_became_active":"2024-06-10T00:52:45.990546+0000","last_became_peered":"2024-06-10T00:52:45.990546+0000","last_unstale":"2024-06-10T00:54:22.668092+0000","last_undegraded":"2024-06-10T00:54:22.668092+0000","last_fullsized":"2024-06-10T00:54:22.668092+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:09:23.412922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"100'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:42.999786+0000","last_change":"2024-06-10T00:52:32.764024+0000","last_active":"2024-06-10T00:54:42.999786+0000","last_peered":"2024-06-10T00:54:42.999786+0000","last_clean":"2024-06-10T00:54:42.999786+0000","last_became_active":"2024-06-10T00:52:32.763449+0000","last_became_peered":"2024-06-10T00:52:32.763449+0000","last_unstale":"2024-06-10T00:54:42.999786+0000","last_undegraded":"2024-06-10T00:54:42.999786+0000","last_fullsized":"2024-06-10T00:54:42.999786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:50:23.605873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373684+0000","last_change":"2024-06-10T00:52:35.955570+0000","last_active":"2024-06-10T00:55:20.373684+0000","last_peered":"2024-06-10T00:55:20.373684+0000","last_clean":"2024-06-10T00:55:20.373684+0000","last_became_active":"2024-06-10T00:52:35.955248+0000","last_became_peered":"2024-06-10T00:52:35.955248+0000","last_unstale":"2024-06-10T00:55:20.373684+0000","last_undegraded":"2024-06-10T00:55:20.373684+0000","last_fullsized":"2024-06-10T00:55:20.373684+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:26.124499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668028+0000","last_change":"2024-06-10T00:52:46.241776+0000","last_active":"2024-06-10T00:54:22.668028+0000","last_peered":"2024-06-10T00:54:22.668028+0000","last_clean":"2024-06-10T00:54:22.668028+0000","last_became_active":"2024-06-10T00:52:46.241276+0000","last_became_peered":"2024-06-10T00:52:46.241276+0000","last_unstale":"2024-06-10T00:54:22.668028+0000","last_undegraded":"2024-06-10T00:54:22.668028+0000","last_fullsized":"2024-06-10T00:54:22.668028+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:08.027938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":58,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556321+0000","last_change":"2024-06-10T00:53:22.204034+0000","last_active":"2024-06-10T00:54:22.556321+0000","last_peered":"2024-06-10T00:54:22.556321+0000","last_clean":"2024-06-10T00:54:22.556321+0000","last_became_active":"2024-06-10T00:53:22.199988+0000","last_became_peered":"2024-06-10T00:53:22.199988+0000","last_unstale":"2024-06-10T00:54:22.556321+0000","last_undegraded":"2024-06-10T00:54:22.556321+0000","last_fullsized":"2024-06-10T00:54:22.556321+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:39:54.012360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373708+0000","last_change":"2024-06-10T00:52:35.959182+0000","last_active":"2024-06-10T00:55:20.373708+0000","last_peered":"2024-06-10T00:55:20.373708+0000","last_clean":"2024-06-10T00:55:20.373708+0000","last_became_active":"2024-06-10T00:52:35.958828+0000","last_became_peered":"2024-06-10T00:52:35.958828+0000","last_unstale":"2024-06-10T00:55:20.373708+0000","last_undegraded":"2024-06-10T00:55:20.373708+0000","last_fullsized":"2024-06-10T00:55:20.373708+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:29:58.857659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561603+0000","last_change":"2024-06-10T00:52:45.989739+0000","last_active":"2024-06-10T00:54:22.561603+0000","last_peered":"2024-06-10T00:54:22.561603+0000","last_clean":"2024-06-10T00:54:22.561603+0000","last_became_active":"2024-06-10T00:52:45.989279+0000","last_became_peered":"2024-06-10T00:52:45.989279+0000","last_unstale":"2024-06-10T00:54:22.561603+0000","last_undegraded":"2024-06-10T00:54:22.561603+0000","last_fullsized":"2024-06-10T00:54:22.561603+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:19:17.551947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554798+0000","last_change":"2024-06-10T00:52:32.959662+0000","last_active":"2024-06-10T00:54:22.554798+0000","last_peered":"2024-06-10T00:54:22.554798+0000","last_clean":"2024-06-10T00:54:22.554798+0000","last_became_active":"2024-06-10T00:52:32.959286+0000","last_became_peered":"2024-06-10T00:52:32.959286+0000","last_unstale":"2024-06-10T00:54:22.554798+0000","last_undegraded":"2024-06-10T00:54:22.554798+0000","last_fullsized":"2024-06-10T00:54:22.554798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:26:04.361567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373843+0000","last_change":"2024-06-10T00:52:35.960137+0000","last_active":"2024-06-10T00:55:20.373843+0000","last_peered":"2024-06-10T00:55:20.373843+0000","last_clean":"2024-06-10T00:55:20.373843+0000","last_became_active":"2024-06-10T00:52:35.959584+0000","last_became_peered":"2024-06-10T00:52:35.959584+0000","last_unstale":"2024-06-10T00:55:20.373843+0000","last_undegraded":"2024-06-10T00:55:20.373843+0000","last_fullsized":"2024-06-10T00:55:20.373843+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:23:38.702685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878137+0000","last_change":"2024-06-10T00:52:46.002268+0000","last_active":"2024-06-10T00:54:22.878137+0000","last_peered":"2024-06-10T00:54:22.878137+0000","last_clean":"2024-06-10T00:54:22.878137+0000","last_became_active":"2024-06-10T00:52:46.001769+0000","last_became_peered":"2024-06-10T00:52:46.001769+0000","last_unstale":"2024-06-10T00:54:22.878137+0000","last_undegraded":"2024-06-10T00:54:22.878137+0000","last_fullsized":"2024-06-10T00:54:22.878137+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:33:51.365951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561629+0000","last_change":"2024-06-10T00:52:32.757112+0000","last_active":"2024-06-10T00:54:22.561629+0000","last_peered":"2024-06-10T00:54:22.561629+0000","last_clean":"2024-06-10T00:54:22.561629+0000","last_became_active":"2024-06-10T00:52:32.755589+0000","last_became_peered":"2024-06-10T00:52:32.755589+0000","last_unstale":"2024-06-10T00:54:22.561629+0000","last_undegraded":"2024-06-10T00:54:22.561629+0000","last_fullsized":"2024-06-10T00:54:22.561629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:15:12.814489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364097+0000","last_change":"2024-06-10T00:52:35.955925+0000","last_active":"2024-06-10T00:55:20.364097+0000","last_peered":"2024-06-10T00:55:20.364097+0000","last_clean":"2024-06-10T00:55:20.364097+0000","last_became_active":"2024-06-10T00:52:35.955556+0000","last_became_peered":"2024-06-10T00:52:35.955556+0000","last_unstale":"2024-06-10T00:55:20.364097+0000","last_undegraded":"2024-06-10T00:55:20.364097+0000","last_fullsized":"2024-06-10T00:55:20.364097+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:42:18.324484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561041+0000","last_change":"2024-06-10T00:52:45.988810+0000","last_active":"2024-06-10T00:54:22.561041+0000","last_peered":"2024-06-10T00:54:22.561041+0000","last_clean":"2024-06-10T00:54:22.561041+0000","last_became_active":"2024-06-10T00:52:45.988041+0000","last_became_peered":"2024-06-10T00:52:45.988041+0000","last_unstale":"2024-06-10T00:54:22.561041+0000","last_undegraded":"2024-06-10T00:54:22.561041+0000","last_fullsized":"2024-06-10T00:54:22.561041+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:14:08.979802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553851+0000","last_change":"2024-06-10T00:52:32.765493+0000","last_active":"2024-06-10T00:54:22.553851+0000","last_peered":"2024-06-10T00:54:22.553851+0000","last_clean":"2024-06-10T00:54:22.553851+0000","last_became_active":"2024-06-10T00:52:32.765193+0000","last_became_peered":"2024-06-10T00:52:32.765193+0000","last_unstale":"2024-06-10T00:54:22.553851+0000","last_undegraded":"2024-06-10T00:54:22.553851+0000","last_fullsized":"2024-06-10T00:54:22.553851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:38:11.554673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374319+0000","last_change":"2024-06-10T00:52:35.960216+0000","last_active":"2024-06-10T00:55:20.374319+0000","last_peered":"2024-06-10T00:55:20.374319+0000","last_clean":"2024-06-10T00:55:20.374319+0000","last_became_active":"2024-06-10T00:52:35.959664+0000","last_became_peered":"2024-06-10T00:52:35.959664+0000","last_unstale":"2024-06-10T00:55:20.374319+0000","last_undegraded":"2024-06-10T00:55:20.374319+0000","last_fullsized":"2024-06-10T00:55:20.374319+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:41:58.086330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375612+0000","last_change":"2024-06-10T00:52:35.967043+0000","last_active":"2024-06-10T00:55:20.375612+0000","last_peered":"2024-06-10T00:55:20.375612+0000","last_clean":"2024-06-10T00:55:20.375612+0000","last_became_active":"2024-06-10T00:52:35.966366+0000","last_became_peered":"2024-06-10T00:52:35.966366+0000","last_unstale":"2024-06-10T00:55:20.375612+0000","last_undegraded":"2024-06-10T00:55:20.375612+0000","last_fullsized":"2024-06-10T00:55:20.375612+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:27:06.047113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"100'12","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.369745+0000","last_change":"2024-06-10T00:52:35.960517+0000","last_active":"2024-06-10T00:55:20.369745+0000","last_peered":"2024-06-10T00:55:20.369745+0000","last_clean":"2024-06-10T00:55:20.369745+0000","last_became_active":"2024-06-10T00:52:35.957895+0000","last_became_peered":"2024-06-10T00:52:35.957895+0000","last_unstale":"2024-06-10T00:55:20.369745+0000","last_undegraded":"2024-06-10T00:55:20.369745+0000","last_fullsized":"2024-06-10T00:55:20.369745+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:04:34.438070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.369777+0000","last_change":"2024-06-10T00:52:35.962163+0000","last_active":"2024-06-10T00:55:20.369777+0000","last_peered":"2024-06-10T00:55:20.369777+0000","last_clean":"2024-06-10T00:55:20.369777+0000","last_became_active":"2024-06-10T00:52:35.961768+0000","last_became_peered":"2024-06-10T00:52:35.961768+0000","last_unstale":"2024-06-10T00:55:20.369777+0000","last_undegraded":"2024-06-10T00:55:20.369777+0000","last_fullsized":"2024-06-10T00:55:20.369777+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:35.793995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"100'7","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373779+0000","last_change":"2024-06-10T00:53:22.130686+0000","last_active":"2024-06-10T00:55:20.373779+0000","last_peered":"2024-06-10T00:55:20.373779+0000","last_clean":"2024-06-10T00:55:20.373779+0000","last_became_active":"2024-06-10T00:53:22.130198+0000","last_became_peered":"2024-06-10T00:53:22.130198+0000","last_unstale":"2024-06-10T00:55:20.373779+0000","last_undegraded":"2024-06-10T00:55:20.373779+0000","last_fullsized":"2024-06-10T00:55:20.373779+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:50:26.773977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372642+0000","last_change":"2024-06-10T00:52:35.957255+0000","last_active":"2024-06-10T00:55:20.372642+0000","last_peered":"2024-06-10T00:55:20.372642+0000","last_clean":"2024-06-10T00:55:20.372642+0000","last_became_active":"2024-06-10T00:52:35.956923+0000","last_became_peered":"2024-06-10T00:52:35.956923+0000","last_unstale":"2024-06-10T00:55:20.372642+0000","last_undegraded":"2024-06-10T00:55:20.372642+0000","last_fullsized":"2024-06-10T00:55:20.372642+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:33:25.743503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375063+0000","last_change":"2024-06-10T00:52:35.964258+0000","last_active":"2024-06-10T00:55:20.375063+0000","last_peered":"2024-06-10T00:55:20.375063+0000","last_clean":"2024-06-10T00:55:20.375063+0000","last_became_active":"2024-06-10T00:52:35.963862+0000","last_became_peered":"2024-06-10T00:52:35.963862+0000","last_unstale":"2024-06-10T00:55:20.375063+0000","last_undegraded":"2024-06-10T00:55:20.375063+0000","last_fullsized":"2024-06-10T00:55:20.375063+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:48:58.554411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.377085+0000","last_change":"2024-06-10T00:52:35.965046+0000","last_active":"2024-06-10T00:55:20.377085+0000","last_peered":"2024-06-10T00:55:20.377085+0000","last_clean":"2024-06-10T00:55:20.377085+0000","last_became_active":"2024-06-10T00:52:35.964496+0000","last_became_peered":"2024-06-10T00:52:35.964496+0000","last_unstale":"2024-06-10T00:55:20.377085+0000","last_undegraded":"2024-06-10T00:55:20.377085+0000","last_fullsized":"2024-06-10T00:55:20.377085+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:09:42.979731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"100'2","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.370981+0000","last_change":"2024-06-10T00:52:35.964287+0000","last_active":"2024-06-10T00:55:20.370981+0000","last_peered":"2024-06-10T00:55:20.370981+0000","last_clean":"2024-06-10T00:55:20.370981+0000","last_became_active":"2024-06-10T00:52:35.963920+0000","last_became_peered":"2024-06-10T00:52:35.963920+0000","last_unstale":"2024-06-10T00:55:20.370981+0000","last_undegraded":"2024-06-10T00:55:20.370981+0000","last_fullsized":"2024-06-10T00:55:20.370981+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:41:54.593274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.367606+0000","last_change":"2024-06-10T00:52:35.961440+0000","last_active":"2024-06-10T00:55:20.367606+0000","last_peered":"2024-06-10T00:55:20.367606+0000","last_clean":"2024-06-10T00:55:20.367606+0000","last_became_active":"2024-06-10T00:52:35.960066+0000","last_became_peered":"2024-06-10T00:52:35.960066+0000","last_unstale":"2024-06-10T00:55:20.367606+0000","last_undegraded":"2024-06-10T00:55:20.367606+0000","last_fullsized":"2024-06-10T00:55:20.367606+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:01.024596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372367+0000","last_change":"2024-06-10T00:52:35.964464+0000","last_active":"2024-06-10T00:55:20.372367+0000","last_peered":"2024-06-10T00:55:20.372367+0000","last_clean":"2024-06-10T00:55:20.372367+0000","last_became_active":"2024-06-10T00:52:35.964159+0000","last_became_peered":"2024-06-10T00:52:35.964159+0000","last_unstale":"2024-06-10T00:55:20.372367+0000","last_undegraded":"2024-06-10T00:55:20.372367+0000","last_fullsized":"2024-06-10T00:55:20.372367+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:18.660859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.377213+0000","last_change":"2024-06-10T00:52:35.961539+0000","last_active":"2024-06-10T00:55:20.377213+0000","last_peered":"2024-06-10T00:55:20.377213+0000","last_clean":"2024-06-10T00:55:20.377213+0000","last_became_active":"2024-06-10T00:52:35.958738+0000","last_became_peered":"2024-06-10T00:52:35.958738+0000","last_unstale":"2024-06-10T00:55:20.377213+0000","last_undegraded":"2024-06-10T00:55:20.377213+0000","last_fullsized":"2024-06-10T00:55:20.377213+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:06:48.057551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374134+0000","last_change":"2024-06-10T00:52:35.962563+0000","last_active":"2024-06-10T00:55:20.374134+0000","last_peered":"2024-06-10T00:55:20.374134+0000","last_clean":"2024-06-10T00:55:20.374134+0000","last_became_active":"2024-06-10T00:52:35.962255+0000","last_became_peered":"2024-06-10T00:52:35.962255+0000","last_unstale":"2024-06-10T00:55:20.374134+0000","last_undegraded":"2024-06-10T00:55:20.374134+0000","last_fullsized":"2024-06-10T00:55:20.374134+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:33:35.689446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372437+0000","last_change":"2024-06-10T00:52:35.959616+0000","last_active":"2024-06-10T00:55:20.372437+0000","last_peered":"2024-06-10T00:55:20.372437+0000","last_clean":"2024-06-10T00:55:20.372437+0000","last_became_active":"2024-06-10T00:52:35.959251+0000","last_became_peered":"2024-06-10T00:52:35.959251+0000","last_unstale":"2024-06-10T00:55:20.372437+0000","last_undegraded":"2024-06-10T00:55:20.372437+0000","last_fullsized":"2024-06-10T00:55:20.372437+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:39:23.933279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284806,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":82322,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1075,"num_read_kb":628,"num_write":191,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":241926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":163,"ondisk_log_size":163,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351040,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":573101,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612678,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28248,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716904,"statfs":{"total":95995035648,"available":95966109696,"internally_reserved":0,"allocated":1261568,"data_stored":1238242,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776203,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":577433,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037839,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":573257,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299477,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36476,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708676,"statfs":{"total":95995035648,"available":95957684224,"internally_reserved":0,"allocated":1167360,"data_stored":1169614,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463001,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":568513,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36488,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708664,"statfs":{"total":95995035648,"available":95957671936,"internally_reserved":0,"allocated":1179648,"data_stored":643776,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738488,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":569901,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411438,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32344,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712808,"statfs":{"total":95995035648,"available":95961915392,"internally_reserved":0,"allocated":1196032,"data_stored":645853,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149800,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":568707,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888163,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":573275,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724638,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":1165202,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":75353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":77430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":79451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T00:56:36.592 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-10T00:56:36.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.732+0000 7f6efb0d4640 1 -- 172.21.15.37:0/3966152595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef41008f0 msgr2=0x7f6ef4100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:36.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.732+0000 7f6efb0d4640 1 --2- 172.21.15.37:0/3966152595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef41008f0 0x7f6ef4100d10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4009900 tx=0x7f6ee402a410 comp rx=0 tx=0).stop 2024-06-10T00:56:36.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.732+0000 7f6efb0d4640 1 -- 172.21.15.37:0/3966152595 shutdown_connections 2024-06-10T00:56:36.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.732+0000 7f6efb0d4640 1 --2- 172.21.15.37:0/3966152595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ef41024a0 0x7f6ef4108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:36.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.732+0000 7f6efb0d4640 1 --2- 172.21.15.37:0/3966152595 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ef4101ae0 0x7f6ef4101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:36.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.732+0000 7f6efb0d4640 1 --2- 172.21.15.37:0/3966152595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef41008f0 0x7f6ef4100d10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:36.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.732+0000 7f6efb0d4640 1 -- 172.21.15.37:0/3966152595 >> 172.21.15.37:0/3966152595 conn(0x7f6ef40fbea0 msgr2=0x7f6ef40fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:36.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.732+0000 7f6efb0d4640 1 -- 172.21.15.37:0/3966152595 shutdown_connections 2024-06-10T00:56:36.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.732+0000 7f6efb0d4640 1 -- 172.21.15.37:0/3966152595 wait complete. 2024-06-10T00:56:36.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.732+0000 7f6efb0d4640 1 Processor -- start 2024-06-10T00:56:36.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.732+0000 7f6efb0d4640 1 -- start start 2024-06-10T00:56:36.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.733+0000 7f6efb0d4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ef41008f0 0x7f6ef4197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:36.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.733+0000 7f6efb0d4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ef4101ae0 0x7f6ef4198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:36.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.733+0000 7f6efb0d4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef41024a0 0x7f6ef419f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:36.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.733+0000 7f6efb0d4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef410c590 con 0x7f6ef41024a0 2024-06-10T00:56:36.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.733+0000 7f6efb0d4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef410c410 con 0x7f6ef41008f0 2024-06-10T00:56:36.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.733+0000 7f6efb0d4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef410c710 con 0x7f6ef4101ae0 2024-06-10T00:56:36.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.733+0000 7f6ef9671640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef41024a0 0x7f6ef419f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:36.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.733+0000 7f6ef9671640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef41024a0 0x7f6ef419f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47168/0 (socket says 172.21.15.37:47168) 2024-06-10T00:56:36.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.733+0000 7f6ef9671640 1 -- 172.21.15.37:0/714717845 learned_addr learned my addr 172.21.15.37:0/714717845 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:36.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.733+0000 7f6ef8e70640 1 --2- 172.21.15.37:0/714717845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ef41008f0 0x7f6ef4197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:36.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.733+0000 7f6ef3fff640 1 --2- 172.21.15.37:0/714717845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ef4101ae0 0x7f6ef4198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:36.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.733+0000 7f6ef9671640 1 -- 172.21.15.37:0/714717845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ef4101ae0 msgr2=0x7f6ef4198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:36.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.733+0000 7f6ef9671640 1 --2- 172.21.15.37:0/714717845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ef4101ae0 0x7f6ef4198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:36.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.733+0000 7f6ef9671640 1 -- 172.21.15.37:0/714717845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ef41008f0 msgr2=0x7f6ef4197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:36.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.733+0000 7f6ef9671640 1 --2- 172.21.15.37:0/714717845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ef41008f0 0x7f6ef4197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:36.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.733+0000 7f6ef9671640 1 -- 172.21.15.37:0/714717845 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef419f950 con 0x7f6ef41024a0 2024-06-10T00:56:36.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.733+0000 7f6ef8e70640 1 --2- 172.21.15.37:0/714717845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ef41008f0 0x7f6ef4197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:56:36.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.733+0000 7f6ef9671640 1 --2- 172.21.15.37:0/714717845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef41024a0 0x7f6ef419f2b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6eec00eaa0 tx=0x7f6eec00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:36.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.733+0000 7f6ef1ffb640 1 -- 172.21.15.37:0/714717845 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec00cdb0 con 0x7f6ef41024a0 2024-06-10T00:56:36.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.733+0000 7f6ef1ffb640 1 -- 172.21.15.37:0/714717845 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eec005020 con 0x7f6ef41024a0 2024-06-10T00:56:36.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.733+0000 7f6efb0d4640 1 -- 172.21.15.37:0/714717845 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef419fc40 con 0x7f6ef41024a0 2024-06-10T00:56:36.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.733+0000 7f6ef1ffb640 1 -- 172.21.15.37:0/714717845 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec0116b0 con 0x7f6ef41024a0 2024-06-10T00:56:36.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.734+0000 7f6efb0d4640 1 -- 172.21.15.37:0/714717845 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef41a0190 con 0x7f6ef41024a0 2024-06-10T00:56:36.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.735+0000 7f6efb0d4640 1 -- 172.21.15.37:0/714717845 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ef41a03c0 con 0x7f6ef41024a0 2024-06-10T00:56:36.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.735+0000 7f6ef1ffb640 1 -- 172.21.15.37:0/714717845 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f6eec005560 con 0x7f6ef41024a0 2024-06-10T00:56:36.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.736+0000 7f6ef1ffb640 1 --2- 172.21.15.37:0/714717845 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f6ee00a2ca0 0x7f6ee00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:36.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.736+0000 7f6ef8e70640 1 --2- 172.21.15.37:0/714717845 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f6ee00a2ca0 0x7f6ee00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:36.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.736+0000 7f6ef1ffb640 1 -- 172.21.15.37:0/714717845 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f6eec0cf870 con 0x7f6ef41024a0 2024-06-10T00:56:36.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.737+0000 7f6ef8e70640 1 --2- 172.21.15.37:0/714717845 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f6ee00a2ca0 0x7f6ee00a5160 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4006010 tx=0x7f6ee4000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:36.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.741+0000 7f6ef1ffb640 1 -- 172.21.15.37:0/714717845 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6eec016030 con 0x7f6ef41024a0 2024-06-10T00:56:36.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.922+0000 7f6efb0d4640 1 -- 172.21.15.37:0/714717845 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6ef41a0590 con 0x7f6ee00a2ca0 2024-06-10T00:56:36.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.932+0000 7f6ef1ffb640 1 -- 172.21.15.37:0/714717845 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340965 (secure 0 0 0) 0x7f6ef41a0590 con 0x7f6ee00a2ca0 2024-06-10T00:56:36.936 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:56:36.937 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-06-10T00:56:36.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.935+0000 7f6efb0d4640 1 -- 172.21.15.37:0/714717845 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f6ee00a2ca0 msgr2=0x7f6ee00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:36.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.935+0000 7f6efb0d4640 1 --2- 172.21.15.37:0/714717845 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f6ee00a2ca0 0x7f6ee00a5160 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4006010 tx=0x7f6ee4000bc0 comp rx=0 tx=0).stop 2024-06-10T00:56:36.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.935+0000 7f6efb0d4640 1 -- 172.21.15.37:0/714717845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef41024a0 msgr2=0x7f6ef419f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:36.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.935+0000 7f6efb0d4640 1 --2- 172.21.15.37:0/714717845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef41024a0 0x7f6ef419f2b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6eec00eaa0 tx=0x7f6eec00ef60 comp rx=0 tx=0).stop 2024-06-10T00:56:36.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.936+0000 7f6efb0d4640 1 -- 172.21.15.37:0/714717845 shutdown_connections 2024-06-10T00:56:36.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.936+0000 7f6efb0d4640 1 --2- 172.21.15.37:0/714717845 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f6ee00a2ca0 0x7f6ee00a5160 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:36.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.936+0000 7f6efb0d4640 1 --2- 172.21.15.37:0/714717845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef41024a0 0x7f6ef419f2b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:36.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.936+0000 7f6efb0d4640 1 --2- 172.21.15.37:0/714717845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ef4101ae0 0x7f6ef4198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:36.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.936+0000 7f6efb0d4640 1 --2- 172.21.15.37:0/714717845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ef41008f0 0x7f6ef4197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:36.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.936+0000 7f6efb0d4640 1 -- 172.21.15.37:0/714717845 >> 172.21.15.37:0/714717845 conn(0x7f6ef40fbea0 msgr2=0x7f6ef40fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:36.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.936+0000 7f6efb0d4640 1 -- 172.21.15.37:0/714717845 shutdown_connections 2024-06-10T00:56:36.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:36.936+0000 7f6efb0d4640 1 -- 172.21.15.37:0/714717845 wait complete. 2024-06-10T00:56:36.947 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":428,"stamp":"2024-06-10T00:56:36.410576+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":672924,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1201,"num_read_kb":737,"num_write":374,"num_write_kb":3088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":802,"ondisk_log_size":802,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370920,"kb_used_data":13448,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570904,"statfs":{"total":1151940427776,"available":1151560605696,"internally_reserved":0,"allocated":13770752,"data_stored":8866874,"data_compressed":497625,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7840,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":50,"num_write":13,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003319"},"pg_stats":[{"pgid":"3.3f","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373740+0000","last_change":"2024-06-10T00:52:35.971067+0000","last_active":"2024-06-10T00:55:20.373740+0000","last_peered":"2024-06-10T00:55:20.373740+0000","last_clean":"2024-06-10T00:55:20.373740+0000","last_became_active":"2024-06-10T00:52:35.970292+0000","last_became_peered":"2024-06-10T00:52:35.970292+0000","last_unstale":"2024-06-10T00:55:20.373740+0000","last_undegraded":"2024-06-10T00:55:20.373740+0000","last_fullsized":"2024-06-10T00:55:20.373740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:57:37.287238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364432+0000","last_change":"2024-06-10T00:52:35.960421+0000","last_active":"2024-06-10T00:55:20.364432+0000","last_peered":"2024-06-10T00:55:20.364432+0000","last_clean":"2024-06-10T00:55:20.364432+0000","last_became_active":"2024-06-10T00:52:35.960070+0000","last_became_peered":"2024-06-10T00:52:35.960070+0000","last_unstale":"2024-06-10T00:55:20.364432+0000","last_undegraded":"2024-06-10T00:55:20.364432+0000","last_fullsized":"2024-06-10T00:55:20.364432+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:48:02.616434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375701+0000","last_change":"2024-06-10T00:52:35.964778+0000","last_active":"2024-06-10T00:55:20.375701+0000","last_peered":"2024-06-10T00:55:20.375701+0000","last_clean":"2024-06-10T00:55:20.375701+0000","last_became_active":"2024-06-10T00:52:35.964463+0000","last_became_peered":"2024-06-10T00:52:35.964463+0000","last_unstale":"2024-06-10T00:55:20.375701+0000","last_undegraded":"2024-06-10T00:55:20.375701+0000","last_fullsized":"2024-06-10T00:55:20.375701+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:52:44.987363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.360279+0000","last_change":"2024-06-10T00:52:35.963226+0000","last_active":"2024-06-10T00:55:20.360279+0000","last_peered":"2024-06-10T00:55:20.360279+0000","last_clean":"2024-06-10T00:55:20.360279+0000","last_became_active":"2024-06-10T00:52:35.962862+0000","last_became_peered":"2024-06-10T00:52:35.962862+0000","last_unstale":"2024-06-10T00:55:20.360279+0000","last_undegraded":"2024-06-10T00:55:20.360279+0000","last_fullsized":"2024-06-10T00:55:20.360279+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:08:56.418071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.360258+0000","last_change":"2024-06-10T00:52:35.957427+0000","last_active":"2024-06-10T00:55:20.360258+0000","last_peered":"2024-06-10T00:55:20.360258+0000","last_clean":"2024-06-10T00:55:20.360258+0000","last_became_active":"2024-06-10T00:52:35.957049+0000","last_became_peered":"2024-06-10T00:52:35.957049+0000","last_unstale":"2024-06-10T00:55:20.360258+0000","last_undegraded":"2024-06-10T00:55:20.360258+0000","last_fullsized":"2024-06-10T00:55:20.360258+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:16:04.257210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.355809+0000","last_change":"2024-06-10T00:52:35.954251+0000","last_active":"2024-06-10T00:55:20.355809+0000","last_peered":"2024-06-10T00:55:20.355809+0000","last_clean":"2024-06-10T00:55:20.355809+0000","last_became_active":"2024-06-10T00:52:35.953908+0000","last_became_peered":"2024-06-10T00:52:35.953908+0000","last_unstale":"2024-06-10T00:55:20.355809+0000","last_undegraded":"2024-06-10T00:55:20.355809+0000","last_fullsized":"2024-06-10T00:55:20.355809+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:55:25.125836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364343+0000","last_change":"2024-06-10T00:52:35.956822+0000","last_active":"2024-06-10T00:55:20.364343+0000","last_peered":"2024-06-10T00:55:20.364343+0000","last_clean":"2024-06-10T00:55:20.364343+0000","last_became_active":"2024-06-10T00:52:35.956466+0000","last_became_peered":"2024-06-10T00:52:35.956466+0000","last_unstale":"2024-06-10T00:55:20.364343+0000","last_undegraded":"2024-06-10T00:55:20.364343+0000","last_fullsized":"2024-06-10T00:55:20.364343+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:36:41.534392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.369347+0000","last_change":"2024-06-10T00:52:35.962978+0000","last_active":"2024-06-10T00:55:20.369347+0000","last_peered":"2024-06-10T00:55:20.369347+0000","last_clean":"2024-06-10T00:55:20.369347+0000","last_became_active":"2024-06-10T00:52:35.962609+0000","last_became_peered":"2024-06-10T00:52:35.962609+0000","last_unstale":"2024-06-10T00:55:20.369347+0000","last_undegraded":"2024-06-10T00:55:20.369347+0000","last_fullsized":"2024-06-10T00:55:20.369347+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:20:57.469577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.377074+0000","last_change":"2024-06-10T00:52:35.963203+0000","last_active":"2024-06-10T00:55:20.377074+0000","last_peered":"2024-06-10T00:55:20.377074+0000","last_clean":"2024-06-10T00:55:20.377074+0000","last_became_active":"2024-06-10T00:52:35.962789+0000","last_became_peered":"2024-06-10T00:52:35.962789+0000","last_unstale":"2024-06-10T00:55:20.377074+0000","last_undegraded":"2024-06-10T00:55:20.377074+0000","last_fullsized":"2024-06-10T00:55:20.377074+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:00:17.819193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"100'16","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.356376+0000","last_change":"2024-06-10T00:52:35.966940+0000","last_active":"2024-06-10T00:55:20.356376+0000","last_peered":"2024-06-10T00:55:20.356376+0000","last_clean":"2024-06-10T00:55:20.356376+0000","last_became_active":"2024-06-10T00:52:35.966554+0000","last_became_peered":"2024-06-10T00:52:35.966554+0000","last_unstale":"2024-06-10T00:55:20.356376+0000","last_undegraded":"2024-06-10T00:55:20.356376+0000","last_fullsized":"2024-06-10T00:55:20.356376+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:00.374437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372673+0000","last_change":"2024-06-10T00:52:35.960927+0000","last_active":"2024-06-10T00:55:20.372673+0000","last_peered":"2024-06-10T00:55:20.372673+0000","last_clean":"2024-06-10T00:55:20.372673+0000","last_became_active":"2024-06-10T00:52:35.960577+0000","last_became_peered":"2024-06-10T00:52:35.960577+0000","last_unstale":"2024-06-10T00:55:20.372673+0000","last_undegraded":"2024-06-10T00:55:20.372673+0000","last_fullsized":"2024-06-10T00:55:20.372673+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:50:02.366695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.355167+0000","last_change":"2024-06-10T00:52:35.964435+0000","last_active":"2024-06-10T00:55:20.355167+0000","last_peered":"2024-06-10T00:55:20.355167+0000","last_clean":"2024-06-10T00:55:20.355167+0000","last_became_active":"2024-06-10T00:52:35.964069+0000","last_became_peered":"2024-06-10T00:52:35.964069+0000","last_unstale":"2024-06-10T00:55:20.355167+0000","last_undegraded":"2024-06-10T00:55:20.355167+0000","last_fullsized":"2024-06-10T00:55:20.355167+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:06.708551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.361787+0000","last_change":"2024-06-10T00:52:35.966722+0000","last_active":"2024-06-10T00:55:20.361787+0000","last_peered":"2024-06-10T00:55:20.361787+0000","last_clean":"2024-06-10T00:55:20.361787+0000","last_became_active":"2024-06-10T00:52:35.966366+0000","last_became_peered":"2024-06-10T00:52:35.966366+0000","last_unstale":"2024-06-10T00:55:20.361787+0000","last_undegraded":"2024-06-10T00:55:20.361787+0000","last_fullsized":"2024-06-10T00:55:20.361787+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:57:46.021047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.371736+0000","last_change":"2024-06-10T00:52:35.955307+0000","last_active":"2024-06-10T00:55:20.371736+0000","last_peered":"2024-06-10T00:55:20.371736+0000","last_clean":"2024-06-10T00:55:20.371736+0000","last_became_active":"2024-06-10T00:52:35.954140+0000","last_became_peered":"2024-06-10T00:52:35.954140+0000","last_unstale":"2024-06-10T00:55:20.371736+0000","last_undegraded":"2024-06-10T00:55:20.371736+0000","last_fullsized":"2024-06-10T00:55:20.371736+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:47:22.493598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373915+0000","last_change":"2024-06-10T00:52:35.960592+0000","last_active":"2024-06-10T00:55:20.373915+0000","last_peered":"2024-06-10T00:55:20.373915+0000","last_clean":"2024-06-10T00:55:20.373915+0000","last_became_active":"2024-06-10T00:52:35.960263+0000","last_became_peered":"2024-06-10T00:52:35.960263+0000","last_unstale":"2024-06-10T00:55:20.373915+0000","last_undegraded":"2024-06-10T00:55:20.373915+0000","last_fullsized":"2024-06-10T00:55:20.373915+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:47:11.658666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372126+0000","last_change":"2024-06-10T00:52:35.965796+0000","last_active":"2024-06-10T00:55:20.372126+0000","last_peered":"2024-06-10T00:55:20.372126+0000","last_clean":"2024-06-10T00:55:20.372126+0000","last_became_active":"2024-06-10T00:52:35.963965+0000","last_became_peered":"2024-06-10T00:52:35.963965+0000","last_unstale":"2024-06-10T00:55:20.372126+0000","last_undegraded":"2024-06-10T00:55:20.372126+0000","last_fullsized":"2024-06-10T00:55:20.372126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:29:47.149852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374090+0000","last_change":"2024-06-10T00:52:35.970714+0000","last_active":"2024-06-10T00:55:20.374090+0000","last_peered":"2024-06-10T00:55:20.374090+0000","last_clean":"2024-06-10T00:55:20.374090+0000","last_became_active":"2024-06-10T00:52:35.970173+0000","last_became_peered":"2024-06-10T00:52:35.970173+0000","last_unstale":"2024-06-10T00:55:20.374090+0000","last_undegraded":"2024-06-10T00:55:20.374090+0000","last_fullsized":"2024-06-10T00:55:20.374090+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:33:28.183156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.370737+0000","last_change":"2024-06-10T00:52:35.963052+0000","last_active":"2024-06-10T00:55:20.370737+0000","last_peered":"2024-06-10T00:55:20.370737+0000","last_clean":"2024-06-10T00:55:20.370737+0000","last_became_active":"2024-06-10T00:52:35.962674+0000","last_became_peered":"2024-06-10T00:52:35.962674+0000","last_unstale":"2024-06-10T00:55:20.370737+0000","last_undegraded":"2024-06-10T00:55:20.370737+0000","last_fullsized":"2024-06-10T00:55:20.370737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:56:38.836854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"100'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.363943+0000","last_change":"2024-06-10T00:53:22.130336+0000","last_active":"2024-06-10T00:55:20.363943+0000","last_peered":"2024-06-10T00:55:20.363943+0000","last_clean":"2024-06-10T00:55:20.363943+0000","last_became_active":"2024-06-10T00:53:22.129910+0000","last_became_peered":"2024-06-10T00:53:22.129910+0000","last_unstale":"2024-06-10T00:55:20.363943+0000","last_undegraded":"2024-06-10T00:55:20.363943+0000","last_fullsized":"2024-06-10T00:55:20.363943+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:23:50.600821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554286+0000","last_change":"2024-06-10T00:52:45.997276+0000","last_active":"2024-06-10T00:54:22.554286+0000","last_peered":"2024-06-10T00:54:22.554286+0000","last_clean":"2024-06-10T00:54:22.554286+0000","last_became_active":"2024-06-10T00:52:45.995638+0000","last_became_peered":"2024-06-10T00:52:45.995638+0000","last_unstale":"2024-06-10T00:54:22.554286+0000","last_undegraded":"2024-06-10T00:54:22.554286+0000","last_fullsized":"2024-06-10T00:54:22.554286+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:14:20.922348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556499+0000","last_change":"2024-06-10T00:52:32.760888+0000","last_active":"2024-06-10T00:54:22.556499+0000","last_peered":"2024-06-10T00:54:22.556499+0000","last_clean":"2024-06-10T00:54:22.556499+0000","last_became_active":"2024-06-10T00:52:32.760531+0000","last_became_peered":"2024-06-10T00:52:32.760531+0000","last_unstale":"2024-06-10T00:54:22.556499+0000","last_undegraded":"2024-06-10T00:54:22.556499+0000","last_fullsized":"2024-06-10T00:54:22.556499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:05:59.052251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.371793+0000","last_change":"2024-06-10T00:52:35.967626+0000","last_active":"2024-06-10T00:55:20.371793+0000","last_peered":"2024-06-10T00:55:20.371793+0000","last_clean":"2024-06-10T00:55:20.371793+0000","last_became_active":"2024-06-10T00:52:35.967295+0000","last_became_peered":"2024-06-10T00:52:35.967295+0000","last_unstale":"2024-06-10T00:55:20.371793+0000","last_undegraded":"2024-06-10T00:55:20.371793+0000","last_fullsized":"2024-06-10T00:55:20.371793+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:07:55.054327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668105+0000","last_change":"2024-06-10T00:52:45.993974+0000","last_active":"2024-06-10T00:54:22.668105+0000","last_peered":"2024-06-10T00:54:22.668105+0000","last_clean":"2024-06-10T00:54:22.668105+0000","last_became_active":"2024-06-10T00:52:45.993242+0000","last_became_peered":"2024-06-10T00:52:45.993242+0000","last_unstale":"2024-06-10T00:54:22.668105+0000","last_undegraded":"2024-06-10T00:54:22.668105+0000","last_fullsized":"2024-06-10T00:54:22.668105+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:28:12.810368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:33.628114+0000","last_change":"2024-06-10T00:52:32.959905+0000","last_active":"2024-06-10T00:54:33.628114+0000","last_peered":"2024-06-10T00:54:33.628114+0000","last_clean":"2024-06-10T00:54:33.628114+0000","last_became_active":"2024-06-10T00:52:32.959551+0000","last_became_peered":"2024-06-10T00:52:32.959551+0000","last_unstale":"2024-06-10T00:54:33.628114+0000","last_undegraded":"2024-06-10T00:54:33.628114+0000","last_fullsized":"2024-06-10T00:54:33.628114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:33:48.029129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375782+0000","last_change":"2024-06-10T00:52:35.965790+0000","last_active":"2024-06-10T00:55:20.375782+0000","last_peered":"2024-06-10T00:55:20.375782+0000","last_clean":"2024-06-10T00:55:20.375782+0000","last_became_active":"2024-06-10T00:52:35.965475+0000","last_became_peered":"2024-06-10T00:52:35.965475+0000","last_unstale":"2024-06-10T00:55:20.375782+0000","last_undegraded":"2024-06-10T00:55:20.375782+0000","last_fullsized":"2024-06-10T00:55:20.375782+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:24:02.894411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.667953+0000","last_change":"2024-06-10T00:52:45.994464+0000","last_active":"2024-06-10T00:54:22.667953+0000","last_peered":"2024-06-10T00:54:22.667953+0000","last_clean":"2024-06-10T00:54:22.667953+0000","last_became_active":"2024-06-10T00:52:45.994022+0000","last_became_peered":"2024-06-10T00:52:45.994022+0000","last_unstale":"2024-06-10T00:54:22.667953+0000","last_undegraded":"2024-06-10T00:54:22.667953+0000","last_fullsized":"2024-06-10T00:54:22.667953+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:27:25.063074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556658+0000","last_change":"2024-06-10T00:52:32.763885+0000","last_active":"2024-06-10T00:54:22.556658+0000","last_peered":"2024-06-10T00:54:22.556658+0000","last_clean":"2024-06-10T00:54:22.556658+0000","last_became_active":"2024-06-10T00:52:32.763606+0000","last_became_peered":"2024-06-10T00:52:32.763606+0000","last_unstale":"2024-06-10T00:54:22.556658+0000","last_undegraded":"2024-06-10T00:54:22.556658+0000","last_fullsized":"2024-06-10T00:54:22.556658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:01:58.161168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372890+0000","last_change":"2024-06-10T00:52:35.966948+0000","last_active":"2024-06-10T00:55:20.372890+0000","last_peered":"2024-06-10T00:55:20.372890+0000","last_clean":"2024-06-10T00:55:20.372890+0000","last_became_active":"2024-06-10T00:52:35.966345+0000","last_became_peered":"2024-06-10T00:52:35.966345+0000","last_unstale":"2024-06-10T00:55:20.372890+0000","last_undegraded":"2024-06-10T00:55:20.372890+0000","last_fullsized":"2024-06-10T00:55:20.372890+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:58:27.169201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.667762+0000","last_change":"2024-06-10T00:52:45.993645+0000","last_active":"2024-06-10T00:54:22.667762+0000","last_peered":"2024-06-10T00:54:22.667762+0000","last_clean":"2024-06-10T00:54:22.667762+0000","last_became_active":"2024-06-10T00:52:45.993183+0000","last_became_peered":"2024-06-10T00:52:45.993183+0000","last_unstale":"2024-06-10T00:54:22.667762+0000","last_undegraded":"2024-06-10T00:54:22.667762+0000","last_fullsized":"2024-06-10T00:54:22.667762+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:47:00.541427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.565600+0000","last_change":"2024-06-10T00:54:22.565600+0000","last_active":"2024-06-10T00:54:22.565600+0000","last_peered":"2024-06-10T00:54:22.565600+0000","last_clean":"2024-06-10T00:54:22.565600+0000","last_became_active":"2024-06-10T00:54:22.562561+0000","last_became_peered":"2024-06-10T00:54:22.562561+0000","last_unstale":"2024-06-10T00:54:22.565600+0000","last_undegraded":"2024-06-10T00:54:22.565600+0000","last_fullsized":"2024-06-10T00:54:22.565600+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:47:58.748050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"100'15","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365162+0000","last_change":"2024-06-10T00:53:22.199668+0000","last_active":"2024-06-10T00:55:20.365162+0000","last_peered":"2024-06-10T00:55:20.365162+0000","last_clean":"2024-06-10T00:55:20.365162+0000","last_became_active":"2024-06-10T00:53:22.199329+0000","last_became_peered":"2024-06-10T00:53:22.199329+0000","last_unstale":"2024-06-10T00:55:20.365162+0000","last_undegraded":"2024-06-10T00:55:20.365162+0000","last_fullsized":"2024-06-10T00:55:20.365162+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:37.640478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554689+0000","last_change":"2024-06-10T00:52:45.995767+0000","last_active":"2024-06-10T00:54:22.554689+0000","last_peered":"2024-06-10T00:54:22.554689+0000","last_clean":"2024-06-10T00:54:22.554689+0000","last_became_active":"2024-06-10T00:52:45.994659+0000","last_became_peered":"2024-06-10T00:52:45.994659+0000","last_unstale":"2024-06-10T00:54:22.554689+0000","last_undegraded":"2024-06-10T00:54:22.554689+0000","last_fullsized":"2024-06-10T00:54:22.554689+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:03:45.652248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'8","reported_seq":227,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:30.554987+0000","last_change":"2024-06-10T00:52:32.761637+0000","last_active":"2024-06-10T00:56:30.554987+0000","last_peered":"2024-06-10T00:56:30.554987+0000","last_clean":"2024-06-10T00:56:30.554987+0000","last_became_active":"2024-06-10T00:52:32.761144+0000","last_became_peered":"2024-06-10T00:52:32.761144+0000","last_unstale":"2024-06-10T00:56:30.554987+0000","last_undegraded":"2024-06-10T00:56:30.554987+0000","last_fullsized":"2024-06-10T00:56:30.554987+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:55:05.616564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1461,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.362574+0000","last_change":"2024-06-10T00:52:35.962683+0000","last_active":"2024-06-10T00:55:20.362574+0000","last_peered":"2024-06-10T00:55:20.362574+0000","last_clean":"2024-06-10T00:55:20.362574+0000","last_became_active":"2024-06-10T00:52:35.962327+0000","last_became_peered":"2024-06-10T00:52:35.962327+0000","last_unstale":"2024-06-10T00:55:20.362574+0000","last_undegraded":"2024-06-10T00:55:20.362574+0000","last_fullsized":"2024-06-10T00:55:20.362574+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:38:19.063306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554563+0000","last_change":"2024-06-10T00:52:45.996882+0000","last_active":"2024-06-10T00:54:22.554563+0000","last_peered":"2024-06-10T00:54:22.554563+0000","last_clean":"2024-06-10T00:54:22.554563+0000","last_became_active":"2024-06-10T00:52:45.995767+0000","last_became_peered":"2024-06-10T00:52:45.995767+0000","last_unstale":"2024-06-10T00:54:22.554563+0000","last_undegraded":"2024-06-10T00:54:22.554563+0000","last_fullsized":"2024-06-10T00:54:22.554563+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:28:03.766944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553970+0000","last_change":"2024-06-10T00:53:22.200546+0000","last_active":"2024-06-10T00:54:22.553970+0000","last_peered":"2024-06-10T00:54:22.553970+0000","last_clean":"2024-06-10T00:54:22.553970+0000","last_became_active":"2024-06-10T00:53:22.200059+0000","last_became_peered":"2024-06-10T00:53:22.200059+0000","last_unstale":"2024-06-10T00:54:22.553970+0000","last_undegraded":"2024-06-10T00:54:22.553970+0000","last_fullsized":"2024-06-10T00:54:22.553970+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:08:47.682594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.356418+0000","last_change":"2024-06-10T00:52:35.963105+0000","last_active":"2024-06-10T00:55:20.356418+0000","last_peered":"2024-06-10T00:55:20.356418+0000","last_clean":"2024-06-10T00:55:20.356418+0000","last_became_active":"2024-06-10T00:52:35.962627+0000","last_became_peered":"2024-06-10T00:52:35.962627+0000","last_unstale":"2024-06-10T00:55:20.356418+0000","last_undegraded":"2024-06-10T00:55:20.356418+0000","last_fullsized":"2024-06-10T00:55:20.356418+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:55:51.231795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.559190+0000","last_change":"2024-06-10T00:52:45.992557+0000","last_active":"2024-06-10T00:54:22.559190+0000","last_peered":"2024-06-10T00:54:22.559190+0000","last_clean":"2024-06-10T00:54:22.559190+0000","last_became_active":"2024-06-10T00:52:45.992166+0000","last_became_peered":"2024-06-10T00:52:45.992166+0000","last_unstale":"2024-06-10T00:54:22.559190+0000","last_undegraded":"2024-06-10T00:54:22.559190+0000","last_fullsized":"2024-06-10T00:54:22.559190+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:49:22.368409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553428+0000","last_change":"2024-06-10T00:52:32.751867+0000","last_active":"2024-06-10T00:54:22.553428+0000","last_peered":"2024-06-10T00:54:22.553428+0000","last_clean":"2024-06-10T00:54:22.553428+0000","last_became_active":"2024-06-10T00:52:32.751516+0000","last_became_peered":"2024-06-10T00:52:32.751516+0000","last_unstale":"2024-06-10T00:54:22.553428+0000","last_undegraded":"2024-06-10T00:54:22.553428+0000","last_fullsized":"2024-06-10T00:54:22.553428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:37:55.920613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.370097+0000","last_change":"2024-06-10T00:52:35.957772+0000","last_active":"2024-06-10T00:55:20.370097+0000","last_peered":"2024-06-10T00:55:20.370097+0000","last_clean":"2024-06-10T00:55:20.370097+0000","last_became_active":"2024-06-10T00:52:35.957470+0000","last_became_peered":"2024-06-10T00:52:35.957470+0000","last_unstale":"2024-06-10T00:55:20.370097+0000","last_undegraded":"2024-06-10T00:55:20.370097+0000","last_fullsized":"2024-06-10T00:55:20.370097+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:16:43.645292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878568+0000","last_change":"2024-06-10T00:52:45.996807+0000","last_active":"2024-06-10T00:54:22.878568+0000","last_peered":"2024-06-10T00:54:22.878568+0000","last_clean":"2024-06-10T00:54:22.878568+0000","last_became_active":"2024-06-10T00:52:45.996378+0000","last_became_peered":"2024-06-10T00:52:45.996378+0000","last_unstale":"2024-06-10T00:54:22.878568+0000","last_undegraded":"2024-06-10T00:54:22.878568+0000","last_fullsized":"2024-06-10T00:54:22.878568+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:40:54.251790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'1","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561873+0000","last_change":"2024-06-10T00:52:32.759625+0000","last_active":"2024-06-10T00:54:22.561873+0000","last_peered":"2024-06-10T00:54:22.561873+0000","last_clean":"2024-06-10T00:54:22.561873+0000","last_became_active":"2024-06-10T00:52:32.759299+0000","last_became_peered":"2024-06-10T00:52:32.759299+0000","last_unstale":"2024-06-10T00:54:22.561873+0000","last_undegraded":"2024-06-10T00:54:22.561873+0000","last_fullsized":"2024-06-10T00:54:22.561873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:11:44.966453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365673+0000","last_change":"2024-06-10T00:52:35.957672+0000","last_active":"2024-06-10T00:55:20.365673+0000","last_peered":"2024-06-10T00:55:20.365673+0000","last_clean":"2024-06-10T00:55:20.365673+0000","last_became_active":"2024-06-10T00:52:35.957328+0000","last_became_peered":"2024-06-10T00:52:35.957328+0000","last_unstale":"2024-06-10T00:55:20.365673+0000","last_undegraded":"2024-06-10T00:55:20.365673+0000","last_fullsized":"2024-06-10T00:55:20.365673+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:43:59.658178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601278+0000","last_change":"2024-06-10T00:52:45.996677+0000","last_active":"2024-06-10T00:54:22.601278+0000","last_peered":"2024-06-10T00:54:22.601278+0000","last_clean":"2024-06-10T00:54:22.601278+0000","last_became_active":"2024-06-10T00:52:45.994483+0000","last_became_peered":"2024-06-10T00:52:45.994483+0000","last_unstale":"2024-06-10T00:54:22.601278+0000","last_undegraded":"2024-06-10T00:54:22.601278+0000","last_fullsized":"2024-06-10T00:54:22.601278+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:03:28.712072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'5","reported_seq":225,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:34.560617+0000","last_change":"2024-06-10T00:52:32.754829+0000","last_active":"2024-06-10T00:56:34.560617+0000","last_peered":"2024-06-10T00:56:34.560617+0000","last_clean":"2024-06-10T00:56:34.560617+0000","last_became_active":"2024-06-10T00:52:32.754386+0000","last_became_peered":"2024-06-10T00:52:32.754386+0000","last_unstale":"2024-06-10T00:56:34.560617+0000","last_undegraded":"2024-06-10T00:56:34.560617+0000","last_fullsized":"2024-06-10T00:56:34.560617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:00.870324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":175,"num_read_kb":175,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.371610+0000","last_change":"2024-06-10T00:52:35.966871+0000","last_active":"2024-06-10T00:55:20.371610+0000","last_peered":"2024-06-10T00:55:20.371610+0000","last_clean":"2024-06-10T00:55:20.371610+0000","last_became_active":"2024-06-10T00:52:35.966492+0000","last_became_peered":"2024-06-10T00:52:35.966492+0000","last_unstale":"2024-06-10T00:55:20.371610+0000","last_undegraded":"2024-06-10T00:55:20.371610+0000","last_fullsized":"2024-06-10T00:55:20.371610+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:24:41.048935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.879950+0000","last_change":"2024-06-10T00:54:22.879457+0000","last_active":"2024-06-10T00:54:22.879950+0000","last_peered":"2024-06-10T00:54:22.879950+0000","last_clean":"2024-06-10T00:54:22.879950+0000","last_became_active":"2024-06-10T00:54:22.878978+0000","last_became_peered":"2024-06-10T00:54:22.878978+0000","last_unstale":"2024-06-10T00:54:22.879950+0000","last_undegraded":"2024-06-10T00:54:22.879950+0000","last_fullsized":"2024-06-10T00:54:22.879950+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:46:38.819250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404659+0000","last_change":"2024-06-10T00:52:32.765794+0000","last_active":"2024-06-10T00:54:24.404659+0000","last_peered":"2024-06-10T00:54:24.404659+0000","last_clean":"2024-06-10T00:54:24.404659+0000","last_became_active":"2024-06-10T00:52:32.765276+0000","last_became_peered":"2024-06-10T00:52:32.765276+0000","last_unstale":"2024-06-10T00:54:24.404659+0000","last_undegraded":"2024-06-10T00:54:24.404659+0000","last_fullsized":"2024-06-10T00:54:24.404659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:06:51.209012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365186+0000","last_change":"2024-06-10T00:52:35.954897+0000","last_active":"2024-06-10T00:55:20.365186+0000","last_peered":"2024-06-10T00:55:20.365186+0000","last_clean":"2024-06-10T00:55:20.365186+0000","last_became_active":"2024-06-10T00:52:35.954565+0000","last_became_peered":"2024-06-10T00:52:35.954565+0000","last_unstale":"2024-06-10T00:55:20.365186+0000","last_undegraded":"2024-06-10T00:55:20.365186+0000","last_fullsized":"2024-06-10T00:55:20.365186+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:39:26.702804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668190+0000","last_change":"2024-06-10T00:52:45.992194+0000","last_active":"2024-06-10T00:54:22.668190+0000","last_peered":"2024-06-10T00:54:22.668190+0000","last_clean":"2024-06-10T00:54:22.668190+0000","last_became_active":"2024-06-10T00:52:45.991453+0000","last_became_peered":"2024-06-10T00:52:45.991453+0000","last_unstale":"2024-06-10T00:54:22.668190+0000","last_undegraded":"2024-06-10T00:54:22.668190+0000","last_fullsized":"2024-06-10T00:54:22.668190+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:11:10.775610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561610+0000","last_change":"2024-06-10T00:52:32.962214+0000","last_active":"2024-06-10T00:54:22.561610+0000","last_peered":"2024-06-10T00:54:22.561610+0000","last_clean":"2024-06-10T00:54:22.561610+0000","last_became_active":"2024-06-10T00:52:32.961835+0000","last_became_peered":"2024-06-10T00:52:32.961835+0000","last_unstale":"2024-06-10T00:54:22.561610+0000","last_undegraded":"2024-06-10T00:54:22.561610+0000","last_fullsized":"2024-06-10T00:54:22.561610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:36:44.724122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.377485+0000","last_change":"2024-06-10T00:52:35.963413+0000","last_active":"2024-06-10T00:55:20.377485+0000","last_peered":"2024-06-10T00:55:20.377485+0000","last_clean":"2024-06-10T00:55:20.377485+0000","last_became_active":"2024-06-10T00:52:35.962922+0000","last_became_peered":"2024-06-10T00:52:35.962922+0000","last_unstale":"2024-06-10T00:55:20.377485+0000","last_undegraded":"2024-06-10T00:55:20.377485+0000","last_fullsized":"2024-06-10T00:55:20.377485+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:05:32.164696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":34,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.560580+0000","last_change":"2024-06-10T00:54:22.560196+0000","last_active":"2024-06-10T00:54:22.560580+0000","last_peered":"2024-06-10T00:54:22.560580+0000","last_clean":"2024-06-10T00:54:22.560580+0000","last_became_active":"2024-06-10T00:54:22.559826+0000","last_became_peered":"2024-06-10T00:54:22.559826+0000","last_unstale":"2024-06-10T00:54:22.560580+0000","last_undegraded":"2024-06-10T00:54:22.560580+0000","last_fullsized":"2024-06-10T00:54:22.560580+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:09:28.456394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'46","reported_seq":455,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:35.566931+0000","last_change":"2024-06-10T00:52:32.761592+0000","last_active":"2024-06-10T00:56:35.566931+0000","last_peered":"2024-06-10T00:56:35.566931+0000","last_clean":"2024-06-10T00:56:35.566931+0000","last_became_active":"2024-06-10T00:52:32.761169+0000","last_became_peered":"2024-06-10T00:52:32.761169+0000","last_unstale":"2024-06-10T00:56:35.566931+0000","last_undegraded":"2024-06-10T00:56:35.566931+0000","last_fullsized":"2024-06-10T00:56:35.566931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:08:33.015035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77873,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":364,"num_read_kb":260,"num_write":47,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.351972+0000","last_change":"2024-06-10T00:52:35.955304+0000","last_active":"2024-06-10T00:55:20.351972+0000","last_peered":"2024-06-10T00:55:20.351972+0000","last_clean":"2024-06-10T00:55:20.351972+0000","last_became_active":"2024-06-10T00:52:35.954949+0000","last_became_peered":"2024-06-10T00:52:35.954949+0000","last_unstale":"2024-06-10T00:55:20.351972+0000","last_undegraded":"2024-06-10T00:55:20.351972+0000","last_fullsized":"2024-06-10T00:55:20.351972+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:56:42.890022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561827+0000","last_change":"2024-06-10T00:52:45.992974+0000","last_active":"2024-06-10T00:54:22.561827+0000","last_peered":"2024-06-10T00:54:22.561827+0000","last_clean":"2024-06-10T00:54:22.561827+0000","last_became_active":"2024-06-10T00:52:45.992490+0000","last_became_peered":"2024-06-10T00:52:45.992490+0000","last_unstale":"2024-06-10T00:54:22.561827+0000","last_undegraded":"2024-06-10T00:54:22.561827+0000","last_fullsized":"2024-06-10T00:54:22.561827+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:03:45.543476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554437+0000","last_change":"2024-06-10T00:52:32.762347+0000","last_active":"2024-06-10T00:54:22.554437+0000","last_peered":"2024-06-10T00:54:22.554437+0000","last_clean":"2024-06-10T00:54:22.554437+0000","last_became_active":"2024-06-10T00:52:32.761049+0000","last_became_peered":"2024-06-10T00:52:32.761049+0000","last_unstale":"2024-06-10T00:54:22.554437+0000","last_undegraded":"2024-06-10T00:54:22.554437+0000","last_fullsized":"2024-06-10T00:54:22.554437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:29:38.384451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364495+0000","last_change":"2024-06-10T00:52:35.961299+0000","last_active":"2024-06-10T00:55:20.364495+0000","last_peered":"2024-06-10T00:55:20.364495+0000","last_clean":"2024-06-10T00:55:20.364495+0000","last_became_active":"2024-06-10T00:52:35.960966+0000","last_became_peered":"2024-06-10T00:52:35.960966+0000","last_unstale":"2024-06-10T00:55:20.364495+0000","last_undegraded":"2024-06-10T00:55:20.364495+0000","last_fullsized":"2024-06-10T00:55:20.364495+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:32:21.269689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.559474+0000","last_change":"2024-06-10T00:52:46.002091+0000","last_active":"2024-06-10T00:54:22.559474+0000","last_peered":"2024-06-10T00:54:22.559474+0000","last_clean":"2024-06-10T00:54:22.559474+0000","last_became_active":"2024-06-10T00:52:46.001677+0000","last_became_peered":"2024-06-10T00:52:46.001677+0000","last_unstale":"2024-06-10T00:54:22.559474+0000","last_undegraded":"2024-06-10T00:54:22.559474+0000","last_fullsized":"2024-06-10T00:54:22.559474+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:16:56.405720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'3","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.562287+0000","last_change":"2024-06-10T00:52:32.756765+0000","last_active":"2024-06-10T00:54:22.562287+0000","last_peered":"2024-06-10T00:54:22.562287+0000","last_clean":"2024-06-10T00:54:22.562287+0000","last_became_active":"2024-06-10T00:52:32.756470+0000","last_became_peered":"2024-06-10T00:52:32.756470+0000","last_unstale":"2024-06-10T00:54:22.562287+0000","last_undegraded":"2024-06-10T00:54:22.562287+0000","last_fullsized":"2024-06-10T00:54:22.562287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:29:24.854913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"100'15","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364256+0000","last_change":"2024-06-10T00:52:35.960693+0000","last_active":"2024-06-10T00:55:20.364256+0000","last_peered":"2024-06-10T00:55:20.364256+0000","last_clean":"2024-06-10T00:55:20.364256+0000","last_became_active":"2024-06-10T00:52:35.958397+0000","last_became_peered":"2024-06-10T00:52:35.958397+0000","last_unstale":"2024-06-10T00:55:20.364256+0000","last_undegraded":"2024-06-10T00:55:20.364256+0000","last_fullsized":"2024-06-10T00:55:20.364256+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:23:47.556462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556200+0000","last_change":"2024-06-10T00:52:46.001475+0000","last_active":"2024-06-10T00:54:22.556200+0000","last_peered":"2024-06-10T00:54:22.556200+0000","last_clean":"2024-06-10T00:54:22.556200+0000","last_became_active":"2024-06-10T00:52:46.000983+0000","last_became_peered":"2024-06-10T00:52:46.000983+0000","last_unstale":"2024-06-10T00:54:22.556200+0000","last_undegraded":"2024-06-10T00:54:22.556200+0000","last_fullsized":"2024-06-10T00:54:22.556200+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:23:56.285356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"98'3","reported_seq":221,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:31.559742+0000","last_change":"2024-06-10T00:52:32.756665+0000","last_active":"2024-06-10T00:56:31.559742+0000","last_peered":"2024-06-10T00:56:31.559742+0000","last_clean":"2024-06-10T00:56:31.559742+0000","last_became_active":"2024-06-10T00:52:32.756221+0000","last_became_peered":"2024-06-10T00:52:32.756221+0000","last_unstale":"2024-06-10T00:56:31.559742+0000","last_undegraded":"2024-06-10T00:56:31.559742+0000","last_fullsized":"2024-06-10T00:56:31.559742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:50:34.789063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":175,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"100'6","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364652+0000","last_change":"2024-06-10T00:53:22.125844+0000","last_active":"2024-06-10T00:55:20.364652+0000","last_peered":"2024-06-10T00:55:20.364652+0000","last_clean":"2024-06-10T00:55:20.364652+0000","last_became_active":"2024-06-10T00:53:22.125449+0000","last_became_peered":"2024-06-10T00:53:22.125449+0000","last_unstale":"2024-06-10T00:55:20.364652+0000","last_undegraded":"2024-06-10T00:55:20.364652+0000","last_fullsized":"2024-06-10T00:55:20.364652+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:08:42.936702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553904+0000","last_change":"2024-06-10T00:52:45.999300+0000","last_active":"2024-06-10T00:54:22.553904+0000","last_peered":"2024-06-10T00:54:22.553904+0000","last_clean":"2024-06-10T00:54:22.553904+0000","last_became_active":"2024-06-10T00:52:45.998860+0000","last_became_peered":"2024-06-10T00:52:45.998860+0000","last_unstale":"2024-06-10T00:54:22.553904+0000","last_undegraded":"2024-06-10T00:54:22.553904+0000","last_fullsized":"2024-06-10T00:54:22.553904+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:14:24.936870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":75,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.557182+0000","last_change":"2024-06-10T00:53:22.210710+0000","last_active":"2024-06-10T00:54:22.557182+0000","last_peered":"2024-06-10T00:54:22.557182+0000","last_clean":"2024-06-10T00:54:22.557182+0000","last_became_active":"2024-06-10T00:53:22.202461+0000","last_became_peered":"2024-06-10T00:53:22.202461+0000","last_unstale":"2024-06-10T00:54:22.557182+0000","last_undegraded":"2024-06-10T00:54:22.557182+0000","last_fullsized":"2024-06-10T00:54:22.557182+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:28:14.671776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364560+0000","last_change":"2024-06-10T00:52:35.966960+0000","last_active":"2024-06-10T00:55:20.364560+0000","last_peered":"2024-06-10T00:55:20.364560+0000","last_clean":"2024-06-10T00:55:20.364560+0000","last_became_active":"2024-06-10T00:52:35.966630+0000","last_became_peered":"2024-06-10T00:52:35.966630+0000","last_unstale":"2024-06-10T00:55:20.364560+0000","last_undegraded":"2024-06-10T00:55:20.364560+0000","last_fullsized":"2024-06-10T00:55:20.364560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:04.117405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561586+0000","last_change":"2024-06-10T00:52:45.992955+0000","last_active":"2024-06-10T00:54:22.561586+0000","last_peered":"2024-06-10T00:54:22.561586+0000","last_clean":"2024-06-10T00:54:22.561586+0000","last_became_active":"2024-06-10T00:52:45.992457+0000","last_became_peered":"2024-06-10T00:52:45.992457+0000","last_unstale":"2024-06-10T00:54:22.561586+0000","last_undegraded":"2024-06-10T00:54:22.561586+0000","last_fullsized":"2024-06-10T00:54:22.561586+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:13:49.491183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556768+0000","last_change":"2024-06-10T00:52:32.763794+0000","last_active":"2024-06-10T00:54:22.556768+0000","last_peered":"2024-06-10T00:54:22.556768+0000","last_clean":"2024-06-10T00:54:22.556768+0000","last_became_active":"2024-06-10T00:52:32.763406+0000","last_became_peered":"2024-06-10T00:52:32.763406+0000","last_unstale":"2024-06-10T00:54:22.556768+0000","last_undegraded":"2024-06-10T00:54:22.556768+0000","last_fullsized":"2024-06-10T00:54:22.556768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:50:33.580163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"100'5","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374120+0000","last_change":"2024-06-10T00:53:22.130891+0000","last_active":"2024-06-10T00:55:20.374120+0000","last_peered":"2024-06-10T00:55:20.374120+0000","last_clean":"2024-06-10T00:55:20.374120+0000","last_became_active":"2024-06-10T00:53:22.130508+0000","last_became_peered":"2024-06-10T00:53:22.130508+0000","last_unstale":"2024-06-10T00:55:20.374120+0000","last_undegraded":"2024-06-10T00:55:20.374120+0000","last_fullsized":"2024-06-10T00:55:20.374120+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:28:01.919825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":34,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.563757+0000","last_change":"2024-06-10T00:54:22.563272+0000","last_active":"2024-06-10T00:54:22.563757+0000","last_peered":"2024-06-10T00:54:22.563757+0000","last_clean":"2024-06-10T00:54:22.563757+0000","last_became_active":"2024-06-10T00:54:22.562767+0000","last_became_peered":"2024-06-10T00:54:22.562767+0000","last_unstale":"2024-06-10T00:54:22.563757+0000","last_undegraded":"2024-06-10T00:54:22.563757+0000","last_fullsized":"2024-06-10T00:54:22.563757+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:52:20.739801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556364+0000","last_change":"2024-06-10T00:52:32.960515+0000","last_active":"2024-06-10T00:54:22.556364+0000","last_peered":"2024-06-10T00:54:22.556364+0000","last_clean":"2024-06-10T00:54:22.556364+0000","last_became_active":"2024-06-10T00:52:32.960163+0000","last_became_peered":"2024-06-10T00:52:32.960163+0000","last_unstale":"2024-06-10T00:54:22.556364+0000","last_undegraded":"2024-06-10T00:54:22.556364+0000","last_fullsized":"2024-06-10T00:54:22.556364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:39:51.374998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.359152+0000","last_change":"2024-06-10T00:52:35.961884+0000","last_active":"2024-06-10T00:55:20.359152+0000","last_peered":"2024-06-10T00:55:20.359152+0000","last_clean":"2024-06-10T00:55:20.359152+0000","last_became_active":"2024-06-10T00:52:35.961497+0000","last_became_peered":"2024-06-10T00:52:35.961497+0000","last_unstale":"2024-06-10T00:55:20.359152+0000","last_undegraded":"2024-06-10T00:55:20.359152+0000","last_fullsized":"2024-06-10T00:55:20.359152+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:16:09.284184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553591+0000","last_change":"2024-06-10T00:52:45.990846+0000","last_active":"2024-06-10T00:54:22.553591+0000","last_peered":"2024-06-10T00:54:22.553591+0000","last_clean":"2024-06-10T00:54:22.553591+0000","last_became_active":"2024-06-10T00:52:45.990441+0000","last_became_peered":"2024-06-10T00:52:45.990441+0000","last_unstale":"2024-06-10T00:54:22.553591+0000","last_undegraded":"2024-06-10T00:54:22.553591+0000","last_fullsized":"2024-06-10T00:54:22.553591+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:19:35.690378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404369+0000","last_change":"2024-06-10T00:52:32.753913+0000","last_active":"2024-06-10T00:54:24.404369+0000","last_peered":"2024-06-10T00:54:24.404369+0000","last_clean":"2024-06-10T00:54:24.404369+0000","last_became_active":"2024-06-10T00:52:32.753174+0000","last_became_peered":"2024-06-10T00:52:32.753174+0000","last_unstale":"2024-06-10T00:54:24.404369+0000","last_undegraded":"2024-06-10T00:54:24.404369+0000","last_fullsized":"2024-06-10T00:54:24.404369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:13:20.048635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.359023+0000","last_change":"2024-06-10T00:52:35.965783+0000","last_active":"2024-06-10T00:55:20.359023+0000","last_peered":"2024-06-10T00:55:20.359023+0000","last_clean":"2024-06-10T00:55:20.359023+0000","last_became_active":"2024-06-10T00:52:35.965432+0000","last_became_peered":"2024-06-10T00:52:35.965432+0000","last_unstale":"2024-06-10T00:55:20.359023+0000","last_undegraded":"2024-06-10T00:55:20.359023+0000","last_fullsized":"2024-06-10T00:55:20.359023+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:52:01.240318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553790+0000","last_change":"2024-06-10T00:52:46.241662+0000","last_active":"2024-06-10T00:54:22.553790+0000","last_peered":"2024-06-10T00:54:22.553790+0000","last_clean":"2024-06-10T00:54:22.553790+0000","last_became_active":"2024-06-10T00:52:46.241211+0000","last_became_peered":"2024-06-10T00:52:46.241211+0000","last_unstale":"2024-06-10T00:54:22.553790+0000","last_undegraded":"2024-06-10T00:54:22.553790+0000","last_fullsized":"2024-06-10T00:54:22.553790+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:12.041290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"100'8","reported_seq":24,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.367303+0000","last_change":"2024-06-10T00:53:22.200590+0000","last_active":"2024-06-10T00:55:20.367303+0000","last_peered":"2024-06-10T00:55:20.367303+0000","last_clean":"2024-06-10T00:55:20.367303+0000","last_became_active":"2024-06-10T00:53:22.200046+0000","last_became_peered":"2024-06-10T00:53:22.200046+0000","last_unstale":"2024-06-10T00:55:20.367303+0000","last_undegraded":"2024-06-10T00:55:20.367303+0000","last_fullsized":"2024-06-10T00:55:20.367303+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:43:22.378316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"98'4","reported_seq":220,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:31.559275+0000","last_change":"2024-06-10T00:52:32.959904+0000","last_active":"2024-06-10T00:56:31.559275+0000","last_peered":"2024-06-10T00:56:31.559275+0000","last_clean":"2024-06-10T00:56:31.559275+0000","last_became_active":"2024-06-10T00:52:32.959506+0000","last_became_peered":"2024-06-10T00:52:32.959506+0000","last_unstale":"2024-06-10T00:56:31.559275+0000","last_undegraded":"2024-06-10T00:56:31.559275+0000","last_fullsized":"2024-06-10T00:56:31.559275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:23:15.950345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":172,"num_read_kb":172,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404433+0000","last_change":"2024-06-10T00:52:45.993305+0000","last_active":"2024-06-10T00:54:24.404433+0000","last_peered":"2024-06-10T00:54:24.404433+0000","last_clean":"2024-06-10T00:54:24.404433+0000","last_became_active":"2024-06-10T00:52:45.992682+0000","last_became_peered":"2024-06-10T00:52:45.992682+0000","last_unstale":"2024-06-10T00:54:24.404433+0000","last_undegraded":"2024-06-10T00:54:24.404433+0000","last_fullsized":"2024-06-10T00:54:24.404433+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:49:41.817126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375771+0000","last_change":"2024-06-10T00:52:35.965893+0000","last_active":"2024-06-10T00:55:20.375771+0000","last_peered":"2024-06-10T00:55:20.375771+0000","last_clean":"2024-06-10T00:55:20.375771+0000","last_became_active":"2024-06-10T00:52:35.965426+0000","last_became_peered":"2024-06-10T00:52:35.965426+0000","last_unstale":"2024-06-10T00:55:20.375771+0000","last_undegraded":"2024-06-10T00:55:20.375771+0000","last_fullsized":"2024-06-10T00:55:20.375771+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:29:19.640836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668002+0000","last_change":"2024-06-10T00:52:32.753631+0000","last_active":"2024-06-10T00:54:22.668002+0000","last_peered":"2024-06-10T00:54:22.668002+0000","last_clean":"2024-06-10T00:54:22.668002+0000","last_became_active":"2024-06-10T00:52:32.753260+0000","last_became_peered":"2024-06-10T00:52:32.753260+0000","last_unstale":"2024-06-10T00:54:22.668002+0000","last_undegraded":"2024-06-10T00:54:22.668002+0000","last_fullsized":"2024-06-10T00:54:22.668002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:12:55.932676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":369,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404055+0000","last_change":"2024-06-10T00:51:09.267750+0000","last_active":"2024-06-10T00:54:24.404055+0000","last_peered":"2024-06-10T00:54:24.404055+0000","last_clean":"2024-06-10T00:54:24.404055+0000","last_became_active":"2024-06-10T00:51:09.267246+0000","last_became_peered":"2024-06-10T00:51:09.267246+0000","last_unstale":"2024-06-10T00:54:24.404055+0000","last_undegraded":"2024-06-10T00:54:24.404055+0000","last_fullsized":"2024-06-10T00:54:24.404055+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:47:44.965452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:47:44.965452+0000","last_clean_scrub_stamp":"2024-06-10T00:47:44.965452+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:21.100090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561655+0000","last_change":"2024-06-10T00:52:46.242152+0000","last_active":"2024-06-10T00:54:22.561655+0000","last_peered":"2024-06-10T00:54:22.561655+0000","last_clean":"2024-06-10T00:54:22.561655+0000","last_became_active":"2024-06-10T00:52:46.241542+0000","last_became_peered":"2024-06-10T00:52:46.241542+0000","last_unstale":"2024-06-10T00:54:22.561655+0000","last_undegraded":"2024-06-10T00:54:22.561655+0000","last_fullsized":"2024-06-10T00:54:22.561655+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:55:54.695465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364883+0000","last_change":"2024-06-10T00:52:35.963911+0000","last_active":"2024-06-10T00:55:20.364883+0000","last_peered":"2024-06-10T00:55:20.364883+0000","last_clean":"2024-06-10T00:55:20.364883+0000","last_became_active":"2024-06-10T00:52:35.963571+0000","last_became_peered":"2024-06-10T00:52:35.963571+0000","last_unstale":"2024-06-10T00:55:20.364883+0000","last_undegraded":"2024-06-10T00:55:20.364883+0000","last_fullsized":"2024-06-10T00:55:20.364883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:27:16.757428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.558529+0000","last_change":"2024-06-10T00:53:22.129035+0000","last_active":"2024-06-10T00:54:22.558529+0000","last_peered":"2024-06-10T00:54:22.558529+0000","last_clean":"2024-06-10T00:54:22.558529+0000","last_became_active":"2024-06-10T00:53:22.128561+0000","last_became_peered":"2024-06-10T00:53:22.128561+0000","last_unstale":"2024-06-10T00:54:22.558529+0000","last_undegraded":"2024-06-10T00:54:22.558529+0000","last_fullsized":"2024-06-10T00:54:22.558529+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:02:43.313118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.558735+0000","last_change":"2024-06-10T00:52:45.996907+0000","last_active":"2024-06-10T00:54:22.558735+0000","last_peered":"2024-06-10T00:54:22.558735+0000","last_clean":"2024-06-10T00:54:22.558735+0000","last_became_active":"2024-06-10T00:52:45.996554+0000","last_became_peered":"2024-06-10T00:52:45.996554+0000","last_unstale":"2024-06-10T00:54:22.558735+0000","last_undegraded":"2024-06-10T00:54:22.558735+0000","last_fullsized":"2024-06-10T00:54:22.558735+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:00:46.824071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556594+0000","last_change":"2024-06-10T00:52:32.960223+0000","last_active":"2024-06-10T00:54:22.556594+0000","last_peered":"2024-06-10T00:54:22.556594+0000","last_clean":"2024-06-10T00:54:22.556594+0000","last_became_active":"2024-06-10T00:52:32.959871+0000","last_became_peered":"2024-06-10T00:52:32.959871+0000","last_unstale":"2024-06-10T00:54:22.556594+0000","last_undegraded":"2024-06-10T00:54:22.556594+0000","last_fullsized":"2024-06-10T00:54:22.556594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:18:17.099140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.369555+0000","last_change":"2024-06-10T00:52:35.962516+0000","last_active":"2024-06-10T00:55:20.369555+0000","last_peered":"2024-06-10T00:55:20.369555+0000","last_clean":"2024-06-10T00:55:20.369555+0000","last_became_active":"2024-06-10T00:52:35.962187+0000","last_became_peered":"2024-06-10T00:52:35.962187+0000","last_unstale":"2024-06-10T00:55:20.369555+0000","last_undegraded":"2024-06-10T00:55:20.369555+0000","last_fullsized":"2024-06-10T00:55:20.369555+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:10:44.295192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601233+0000","last_change":"2024-06-10T00:52:46.241554+0000","last_active":"2024-06-10T00:54:22.601233+0000","last_peered":"2024-06-10T00:54:22.601233+0000","last_clean":"2024-06-10T00:54:22.601233+0000","last_became_active":"2024-06-10T00:52:46.241142+0000","last_became_peered":"2024-06-10T00:52:46.241142+0000","last_unstale":"2024-06-10T00:54:22.601233+0000","last_undegraded":"2024-06-10T00:54:22.601233+0000","last_fullsized":"2024-06-10T00:54:22.601233+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:58:07.233080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'13","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:29.987897+0000","last_change":"2024-06-10T00:52:32.761854+0000","last_active":"2024-06-10T00:56:29.987897+0000","last_peered":"2024-06-10T00:56:29.987897+0000","last_clean":"2024-06-10T00:56:29.987897+0000","last_became_active":"2024-06-10T00:52:32.761522+0000","last_became_peered":"2024-06-10T00:52:32.761522+0000","last_unstale":"2024-06-10T00:56:29.987897+0000","last_undegraded":"2024-06-10T00:56:29.987897+0000","last_fullsized":"2024-06-10T00:56:29.987897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:06:43.269001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.370762+0000","last_change":"2024-06-10T00:52:35.959250+0000","last_active":"2024-06-10T00:55:20.370762+0000","last_peered":"2024-06-10T00:55:20.370762+0000","last_clean":"2024-06-10T00:55:20.370762+0000","last_became_active":"2024-06-10T00:52:35.958925+0000","last_became_peered":"2024-06-10T00:52:35.958925+0000","last_unstale":"2024-06-10T00:55:20.370762+0000","last_undegraded":"2024-06-10T00:55:20.370762+0000","last_fullsized":"2024-06-10T00:55:20.370762+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:11:12.355505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554191+0000","last_change":"2024-06-10T00:52:45.991819+0000","last_active":"2024-06-10T00:54:22.554191+0000","last_peered":"2024-06-10T00:54:22.554191+0000","last_clean":"2024-06-10T00:54:22.554191+0000","last_became_active":"2024-06-10T00:52:45.991408+0000","last_became_peered":"2024-06-10T00:52:45.991408+0000","last_unstale":"2024-06-10T00:54:22.554191+0000","last_undegraded":"2024-06-10T00:54:22.554191+0000","last_fullsized":"2024-06-10T00:54:22.554191+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:16:34.912359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668551+0000","last_change":"2024-06-10T00:52:32.869094+0000","last_active":"2024-06-10T00:54:22.668551+0000","last_peered":"2024-06-10T00:54:22.668551+0000","last_clean":"2024-06-10T00:54:22.668551+0000","last_became_active":"2024-06-10T00:52:32.868664+0000","last_became_peered":"2024-06-10T00:52:32.868664+0000","last_unstale":"2024-06-10T00:54:22.668551+0000","last_undegraded":"2024-06-10T00:54:22.668551+0000","last_fullsized":"2024-06-10T00:54:22.668551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:50:38.051090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374722+0000","last_change":"2024-06-10T00:52:35.952356+0000","last_active":"2024-06-10T00:55:20.374722+0000","last_peered":"2024-06-10T00:55:20.374722+0000","last_clean":"2024-06-10T00:55:20.374722+0000","last_became_active":"2024-06-10T00:52:35.952002+0000","last_became_peered":"2024-06-10T00:52:35.952002+0000","last_unstale":"2024-06-10T00:55:20.374722+0000","last_undegraded":"2024-06-10T00:55:20.374722+0000","last_fullsized":"2024-06-10T00:55:20.374722+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:12:28.581522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668366+0000","last_change":"2024-06-10T00:52:46.241773+0000","last_active":"2024-06-10T00:54:22.668366+0000","last_peered":"2024-06-10T00:54:22.668366+0000","last_clean":"2024-06-10T00:54:22.668366+0000","last_became_active":"2024-06-10T00:52:46.241272+0000","last_became_peered":"2024-06-10T00:52:46.241272+0000","last_unstale":"2024-06-10T00:54:22.668366+0000","last_undegraded":"2024-06-10T00:54:22.668366+0000","last_fullsized":"2024-06-10T00:54:22.668366+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:07:28.337536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561360+0000","last_change":"2024-06-10T00:52:32.960100+0000","last_active":"2024-06-10T00:54:22.561360+0000","last_peered":"2024-06-10T00:54:22.561360+0000","last_clean":"2024-06-10T00:54:22.561360+0000","last_became_active":"2024-06-10T00:52:32.959779+0000","last_became_peered":"2024-06-10T00:52:32.959779+0000","last_unstale":"2024-06-10T00:54:22.561360+0000","last_undegraded":"2024-06-10T00:54:22.561360+0000","last_fullsized":"2024-06-10T00:54:22.561360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:09.555753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'2","reported_seq":40,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.326494+0000","last_change":"2024-06-10T00:52:35.955586+0000","last_active":"2024-06-10T00:55:20.326494+0000","last_peered":"2024-06-10T00:55:20.326494+0000","last_clean":"2024-06-10T00:55:20.326494+0000","last_became_active":"2024-06-10T00:52:35.955216+0000","last_became_peered":"2024-06-10T00:52:35.955216+0000","last_unstale":"2024-06-10T00:55:20.326494+0000","last_undegraded":"2024-06-10T00:55:20.326494+0000","last_fullsized":"2024-06-10T00:55:20.326494+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:23:12.197182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.603221+0000","last_change":"2024-06-10T00:54:22.602670+0000","last_active":"2024-06-10T00:54:22.603221+0000","last_peered":"2024-06-10T00:54:22.603221+0000","last_clean":"2024-06-10T00:54:22.603221+0000","last_became_active":"2024-06-10T00:54:22.602218+0000","last_became_peered":"2024-06-10T00:54:22.602218+0000","last_unstale":"2024-06-10T00:54:22.603221+0000","last_undegraded":"2024-06-10T00:54:22.603221+0000","last_fullsized":"2024-06-10T00:54:22.603221+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:07.399025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"98'3","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601358+0000","last_change":"2024-06-10T00:52:32.753444+0000","last_active":"2024-06-10T00:54:22.601358+0000","last_peered":"2024-06-10T00:54:22.601358+0000","last_clean":"2024-06-10T00:54:22.601358+0000","last_became_active":"2024-06-10T00:52:32.753149+0000","last_became_peered":"2024-06-10T00:52:32.753149+0000","last_unstale":"2024-06-10T00:54:22.601358+0000","last_undegraded":"2024-06-10T00:54:22.601358+0000","last_fullsized":"2024-06-10T00:54:22.601358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:28.241950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365900+0000","last_change":"2024-06-10T00:52:35.966718+0000","last_active":"2024-06-10T00:55:20.365900+0000","last_peered":"2024-06-10T00:55:20.365900+0000","last_clean":"2024-06-10T00:55:20.365900+0000","last_became_active":"2024-06-10T00:52:35.965907+0000","last_became_peered":"2024-06-10T00:52:35.965907+0000","last_unstale":"2024-06-10T00:55:20.365900+0000","last_undegraded":"2024-06-10T00:55:20.365900+0000","last_fullsized":"2024-06-10T00:55:20.365900+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:21:34.261306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668092+0000","last_change":"2024-06-10T00:52:45.991034+0000","last_active":"2024-06-10T00:54:22.668092+0000","last_peered":"2024-06-10T00:54:22.668092+0000","last_clean":"2024-06-10T00:54:22.668092+0000","last_became_active":"2024-06-10T00:52:45.990546+0000","last_became_peered":"2024-06-10T00:52:45.990546+0000","last_unstale":"2024-06-10T00:54:22.668092+0000","last_undegraded":"2024-06-10T00:54:22.668092+0000","last_fullsized":"2024-06-10T00:54:22.668092+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:09:23.412922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"100'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:42.999786+0000","last_change":"2024-06-10T00:52:32.764024+0000","last_active":"2024-06-10T00:54:42.999786+0000","last_peered":"2024-06-10T00:54:42.999786+0000","last_clean":"2024-06-10T00:54:42.999786+0000","last_became_active":"2024-06-10T00:52:32.763449+0000","last_became_peered":"2024-06-10T00:52:32.763449+0000","last_unstale":"2024-06-10T00:54:42.999786+0000","last_undegraded":"2024-06-10T00:54:42.999786+0000","last_fullsized":"2024-06-10T00:54:42.999786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:50:23.605873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373684+0000","last_change":"2024-06-10T00:52:35.955570+0000","last_active":"2024-06-10T00:55:20.373684+0000","last_peered":"2024-06-10T00:55:20.373684+0000","last_clean":"2024-06-10T00:55:20.373684+0000","last_became_active":"2024-06-10T00:52:35.955248+0000","last_became_peered":"2024-06-10T00:52:35.955248+0000","last_unstale":"2024-06-10T00:55:20.373684+0000","last_undegraded":"2024-06-10T00:55:20.373684+0000","last_fullsized":"2024-06-10T00:55:20.373684+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:26.124499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668028+0000","last_change":"2024-06-10T00:52:46.241776+0000","last_active":"2024-06-10T00:54:22.668028+0000","last_peered":"2024-06-10T00:54:22.668028+0000","last_clean":"2024-06-10T00:54:22.668028+0000","last_became_active":"2024-06-10T00:52:46.241276+0000","last_became_peered":"2024-06-10T00:52:46.241276+0000","last_unstale":"2024-06-10T00:54:22.668028+0000","last_undegraded":"2024-06-10T00:54:22.668028+0000","last_fullsized":"2024-06-10T00:54:22.668028+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:08.027938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":58,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556321+0000","last_change":"2024-06-10T00:53:22.204034+0000","last_active":"2024-06-10T00:54:22.556321+0000","last_peered":"2024-06-10T00:54:22.556321+0000","last_clean":"2024-06-10T00:54:22.556321+0000","last_became_active":"2024-06-10T00:53:22.199988+0000","last_became_peered":"2024-06-10T00:53:22.199988+0000","last_unstale":"2024-06-10T00:54:22.556321+0000","last_undegraded":"2024-06-10T00:54:22.556321+0000","last_fullsized":"2024-06-10T00:54:22.556321+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:39:54.012360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373708+0000","last_change":"2024-06-10T00:52:35.959182+0000","last_active":"2024-06-10T00:55:20.373708+0000","last_peered":"2024-06-10T00:55:20.373708+0000","last_clean":"2024-06-10T00:55:20.373708+0000","last_became_active":"2024-06-10T00:52:35.958828+0000","last_became_peered":"2024-06-10T00:52:35.958828+0000","last_unstale":"2024-06-10T00:55:20.373708+0000","last_undegraded":"2024-06-10T00:55:20.373708+0000","last_fullsized":"2024-06-10T00:55:20.373708+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:29:58.857659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561603+0000","last_change":"2024-06-10T00:52:45.989739+0000","last_active":"2024-06-10T00:54:22.561603+0000","last_peered":"2024-06-10T00:54:22.561603+0000","last_clean":"2024-06-10T00:54:22.561603+0000","last_became_active":"2024-06-10T00:52:45.989279+0000","last_became_peered":"2024-06-10T00:52:45.989279+0000","last_unstale":"2024-06-10T00:54:22.561603+0000","last_undegraded":"2024-06-10T00:54:22.561603+0000","last_fullsized":"2024-06-10T00:54:22.561603+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:19:17.551947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554798+0000","last_change":"2024-06-10T00:52:32.959662+0000","last_active":"2024-06-10T00:54:22.554798+0000","last_peered":"2024-06-10T00:54:22.554798+0000","last_clean":"2024-06-10T00:54:22.554798+0000","last_became_active":"2024-06-10T00:52:32.959286+0000","last_became_peered":"2024-06-10T00:52:32.959286+0000","last_unstale":"2024-06-10T00:54:22.554798+0000","last_undegraded":"2024-06-10T00:54:22.554798+0000","last_fullsized":"2024-06-10T00:54:22.554798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:26:04.361567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373843+0000","last_change":"2024-06-10T00:52:35.960137+0000","last_active":"2024-06-10T00:55:20.373843+0000","last_peered":"2024-06-10T00:55:20.373843+0000","last_clean":"2024-06-10T00:55:20.373843+0000","last_became_active":"2024-06-10T00:52:35.959584+0000","last_became_peered":"2024-06-10T00:52:35.959584+0000","last_unstale":"2024-06-10T00:55:20.373843+0000","last_undegraded":"2024-06-10T00:55:20.373843+0000","last_fullsized":"2024-06-10T00:55:20.373843+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:23:38.702685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878137+0000","last_change":"2024-06-10T00:52:46.002268+0000","last_active":"2024-06-10T00:54:22.878137+0000","last_peered":"2024-06-10T00:54:22.878137+0000","last_clean":"2024-06-10T00:54:22.878137+0000","last_became_active":"2024-06-10T00:52:46.001769+0000","last_became_peered":"2024-06-10T00:52:46.001769+0000","last_unstale":"2024-06-10T00:54:22.878137+0000","last_undegraded":"2024-06-10T00:54:22.878137+0000","last_fullsized":"2024-06-10T00:54:22.878137+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:33:51.365951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561629+0000","last_change":"2024-06-10T00:52:32.757112+0000","last_active":"2024-06-10T00:54:22.561629+0000","last_peered":"2024-06-10T00:54:22.561629+0000","last_clean":"2024-06-10T00:54:22.561629+0000","last_became_active":"2024-06-10T00:52:32.755589+0000","last_became_peered":"2024-06-10T00:52:32.755589+0000","last_unstale":"2024-06-10T00:54:22.561629+0000","last_undegraded":"2024-06-10T00:54:22.561629+0000","last_fullsized":"2024-06-10T00:54:22.561629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:15:12.814489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364097+0000","last_change":"2024-06-10T00:52:35.955925+0000","last_active":"2024-06-10T00:55:20.364097+0000","last_peered":"2024-06-10T00:55:20.364097+0000","last_clean":"2024-06-10T00:55:20.364097+0000","last_became_active":"2024-06-10T00:52:35.955556+0000","last_became_peered":"2024-06-10T00:52:35.955556+0000","last_unstale":"2024-06-10T00:55:20.364097+0000","last_undegraded":"2024-06-10T00:55:20.364097+0000","last_fullsized":"2024-06-10T00:55:20.364097+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:42:18.324484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561041+0000","last_change":"2024-06-10T00:52:45.988810+0000","last_active":"2024-06-10T00:54:22.561041+0000","last_peered":"2024-06-10T00:54:22.561041+0000","last_clean":"2024-06-10T00:54:22.561041+0000","last_became_active":"2024-06-10T00:52:45.988041+0000","last_became_peered":"2024-06-10T00:52:45.988041+0000","last_unstale":"2024-06-10T00:54:22.561041+0000","last_undegraded":"2024-06-10T00:54:22.561041+0000","last_fullsized":"2024-06-10T00:54:22.561041+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:14:08.979802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553851+0000","last_change":"2024-06-10T00:52:32.765493+0000","last_active":"2024-06-10T00:54:22.553851+0000","last_peered":"2024-06-10T00:54:22.553851+0000","last_clean":"2024-06-10T00:54:22.553851+0000","last_became_active":"2024-06-10T00:52:32.765193+0000","last_became_peered":"2024-06-10T00:52:32.765193+0000","last_unstale":"2024-06-10T00:54:22.553851+0000","last_undegraded":"2024-06-10T00:54:22.553851+0000","last_fullsized":"2024-06-10T00:54:22.553851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:38:11.554673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374319+0000","last_change":"2024-06-10T00:52:35.960216+0000","last_active":"2024-06-10T00:55:20.374319+0000","last_peered":"2024-06-10T00:55:20.374319+0000","last_clean":"2024-06-10T00:55:20.374319+0000","last_became_active":"2024-06-10T00:52:35.959664+0000","last_became_peered":"2024-06-10T00:52:35.959664+0000","last_unstale":"2024-06-10T00:55:20.374319+0000","last_undegraded":"2024-06-10T00:55:20.374319+0000","last_fullsized":"2024-06-10T00:55:20.374319+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:41:58.086330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375612+0000","last_change":"2024-06-10T00:52:35.967043+0000","last_active":"2024-06-10T00:55:20.375612+0000","last_peered":"2024-06-10T00:55:20.375612+0000","last_clean":"2024-06-10T00:55:20.375612+0000","last_became_active":"2024-06-10T00:52:35.966366+0000","last_became_peered":"2024-06-10T00:52:35.966366+0000","last_unstale":"2024-06-10T00:55:20.375612+0000","last_undegraded":"2024-06-10T00:55:20.375612+0000","last_fullsized":"2024-06-10T00:55:20.375612+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:27:06.047113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"100'12","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.369745+0000","last_change":"2024-06-10T00:52:35.960517+0000","last_active":"2024-06-10T00:55:20.369745+0000","last_peered":"2024-06-10T00:55:20.369745+0000","last_clean":"2024-06-10T00:55:20.369745+0000","last_became_active":"2024-06-10T00:52:35.957895+0000","last_became_peered":"2024-06-10T00:52:35.957895+0000","last_unstale":"2024-06-10T00:55:20.369745+0000","last_undegraded":"2024-06-10T00:55:20.369745+0000","last_fullsized":"2024-06-10T00:55:20.369745+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:04:34.438070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.369777+0000","last_change":"2024-06-10T00:52:35.962163+0000","last_active":"2024-06-10T00:55:20.369777+0000","last_peered":"2024-06-10T00:55:20.369777+0000","last_clean":"2024-06-10T00:55:20.369777+0000","last_became_active":"2024-06-10T00:52:35.961768+0000","last_became_peered":"2024-06-10T00:52:35.961768+0000","last_unstale":"2024-06-10T00:55:20.369777+0000","last_undegraded":"2024-06-10T00:55:20.369777+0000","last_fullsized":"2024-06-10T00:55:20.369777+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:35.793995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"100'7","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373779+0000","last_change":"2024-06-10T00:53:22.130686+0000","last_active":"2024-06-10T00:55:20.373779+0000","last_peered":"2024-06-10T00:55:20.373779+0000","last_clean":"2024-06-10T00:55:20.373779+0000","last_became_active":"2024-06-10T00:53:22.130198+0000","last_became_peered":"2024-06-10T00:53:22.130198+0000","last_unstale":"2024-06-10T00:55:20.373779+0000","last_undegraded":"2024-06-10T00:55:20.373779+0000","last_fullsized":"2024-06-10T00:55:20.373779+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:50:26.773977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372642+0000","last_change":"2024-06-10T00:52:35.957255+0000","last_active":"2024-06-10T00:55:20.372642+0000","last_peered":"2024-06-10T00:55:20.372642+0000","last_clean":"2024-06-10T00:55:20.372642+0000","last_became_active":"2024-06-10T00:52:35.956923+0000","last_became_peered":"2024-06-10T00:52:35.956923+0000","last_unstale":"2024-06-10T00:55:20.372642+0000","last_undegraded":"2024-06-10T00:55:20.372642+0000","last_fullsized":"2024-06-10T00:55:20.372642+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:33:25.743503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375063+0000","last_change":"2024-06-10T00:52:35.964258+0000","last_active":"2024-06-10T00:55:20.375063+0000","last_peered":"2024-06-10T00:55:20.375063+0000","last_clean":"2024-06-10T00:55:20.375063+0000","last_became_active":"2024-06-10T00:52:35.963862+0000","last_became_peered":"2024-06-10T00:52:35.963862+0000","last_unstale":"2024-06-10T00:55:20.375063+0000","last_undegraded":"2024-06-10T00:55:20.375063+0000","last_fullsized":"2024-06-10T00:55:20.375063+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:48:58.554411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.377085+0000","last_change":"2024-06-10T00:52:35.965046+0000","last_active":"2024-06-10T00:55:20.377085+0000","last_peered":"2024-06-10T00:55:20.377085+0000","last_clean":"2024-06-10T00:55:20.377085+0000","last_became_active":"2024-06-10T00:52:35.964496+0000","last_became_peered":"2024-06-10T00:52:35.964496+0000","last_unstale":"2024-06-10T00:55:20.377085+0000","last_undegraded":"2024-06-10T00:55:20.377085+0000","last_fullsized":"2024-06-10T00:55:20.377085+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:09:42.979731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"100'2","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.370981+0000","last_change":"2024-06-10T00:52:35.964287+0000","last_active":"2024-06-10T00:55:20.370981+0000","last_peered":"2024-06-10T00:55:20.370981+0000","last_clean":"2024-06-10T00:55:20.370981+0000","last_became_active":"2024-06-10T00:52:35.963920+0000","last_became_peered":"2024-06-10T00:52:35.963920+0000","last_unstale":"2024-06-10T00:55:20.370981+0000","last_undegraded":"2024-06-10T00:55:20.370981+0000","last_fullsized":"2024-06-10T00:55:20.370981+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:41:54.593274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.367606+0000","last_change":"2024-06-10T00:52:35.961440+0000","last_active":"2024-06-10T00:55:20.367606+0000","last_peered":"2024-06-10T00:55:20.367606+0000","last_clean":"2024-06-10T00:55:20.367606+0000","last_became_active":"2024-06-10T00:52:35.960066+0000","last_became_peered":"2024-06-10T00:52:35.960066+0000","last_unstale":"2024-06-10T00:55:20.367606+0000","last_undegraded":"2024-06-10T00:55:20.367606+0000","last_fullsized":"2024-06-10T00:55:20.367606+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:01.024596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372367+0000","last_change":"2024-06-10T00:52:35.964464+0000","last_active":"2024-06-10T00:55:20.372367+0000","last_peered":"2024-06-10T00:55:20.372367+0000","last_clean":"2024-06-10T00:55:20.372367+0000","last_became_active":"2024-06-10T00:52:35.964159+0000","last_became_peered":"2024-06-10T00:52:35.964159+0000","last_unstale":"2024-06-10T00:55:20.372367+0000","last_undegraded":"2024-06-10T00:55:20.372367+0000","last_fullsized":"2024-06-10T00:55:20.372367+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:18.660859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.377213+0000","last_change":"2024-06-10T00:52:35.961539+0000","last_active":"2024-06-10T00:55:20.377213+0000","last_peered":"2024-06-10T00:55:20.377213+0000","last_clean":"2024-06-10T00:55:20.377213+0000","last_became_active":"2024-06-10T00:52:35.958738+0000","last_became_peered":"2024-06-10T00:52:35.958738+0000","last_unstale":"2024-06-10T00:55:20.377213+0000","last_undegraded":"2024-06-10T00:55:20.377213+0000","last_fullsized":"2024-06-10T00:55:20.377213+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:06:48.057551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374134+0000","last_change":"2024-06-10T00:52:35.962563+0000","last_active":"2024-06-10T00:55:20.374134+0000","last_peered":"2024-06-10T00:55:20.374134+0000","last_clean":"2024-06-10T00:55:20.374134+0000","last_became_active":"2024-06-10T00:52:35.962255+0000","last_became_peered":"2024-06-10T00:52:35.962255+0000","last_unstale":"2024-06-10T00:55:20.374134+0000","last_undegraded":"2024-06-10T00:55:20.374134+0000","last_fullsized":"2024-06-10T00:55:20.374134+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:33:35.689446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372437+0000","last_change":"2024-06-10T00:52:35.959616+0000","last_active":"2024-06-10T00:55:20.372437+0000","last_peered":"2024-06-10T00:55:20.372437+0000","last_clean":"2024-06-10T00:55:20.372437+0000","last_became_active":"2024-06-10T00:52:35.959251+0000","last_became_peered":"2024-06-10T00:52:35.959251+0000","last_unstale":"2024-06-10T00:55:20.372437+0000","last_undegraded":"2024-06-10T00:55:20.372437+0000","last_fullsized":"2024-06-10T00:55:20.372437+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:39:23.933279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284806,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":82322,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1075,"num_read_kb":628,"num_write":191,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":241926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":163,"ondisk_log_size":163,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351040,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":573101,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612678,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28248,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716904,"statfs":{"total":95995035648,"available":95966109696,"internally_reserved":0,"allocated":1261568,"data_stored":1238242,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776203,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":577433,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037839,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":573257,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299477,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36476,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708676,"statfs":{"total":95995035648,"available":95957684224,"internally_reserved":0,"allocated":1167360,"data_stored":1169614,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463001,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":568513,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36488,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708664,"statfs":{"total":95995035648,"available":95957671936,"internally_reserved":0,"allocated":1179648,"data_stored":643776,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738488,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":569901,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411438,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32344,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712808,"statfs":{"total":95995035648,"available":95961915392,"internally_reserved":0,"allocated":1196032,"data_stored":645853,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149800,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":568707,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888163,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":573275,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724638,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":1165202,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":75353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":77430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":79451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T00:56:36.952 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-06-10T00:56:36.952 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-10T00:56:37.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.093+0000 7fa496321640 1 -- 172.21.15.37:0/1165734206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4901077e0 msgr2=0x7fa490109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:37.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.093+0000 7fa496321640 1 --2- 172.21.15.37:0/1165734206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4901077e0 0x7fa490109bd0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa488009ca0 tx=0x7fa48802a8c0 comp rx=0 tx=0).stop 2024-06-10T00:56:37.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.093+0000 7fa496321640 1 -- 172.21.15.37:0/1165734206 shutdown_connections 2024-06-10T00:56:37.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.093+0000 7fa496321640 1 --2- 172.21.15.37:0/1165734206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4901077e0 0x7fa490109bd0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:37.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.093+0000 7fa496321640 1 --2- 172.21.15.37:0/1165734206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa490104eb0 0x7fa4901072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:37.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.093+0000 7fa496321640 1 --2- 172.21.15.37:0/1165734206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa490102580 0x7fa490104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:37.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.093+0000 7fa496321640 1 -- 172.21.15.37:0/1165734206 >> 172.21.15.37:0/1165734206 conn(0x7fa4900fbef0 msgr2=0x7fa4900fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:37.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.093+0000 7fa496321640 1 -- 172.21.15.37:0/1165734206 shutdown_connections 2024-06-10T00:56:37.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.093+0000 7fa496321640 1 -- 172.21.15.37:0/1165734206 wait complete. 2024-06-10T00:56:37.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.093+0000 7fa496321640 1 Processor -- start 2024-06-10T00:56:37.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.093+0000 7fa496321640 1 -- start start 2024-06-10T00:56:37.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.094+0000 7fa496321640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa490102580 0x7fa490197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:37.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.094+0000 7fa496321640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa490104eb0 0x7fa490198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:37.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.094+0000 7fa496321640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4901077e0 0x7fa49019f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:37.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.094+0000 7fa496321640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa49010d690 con 0x7fa490104eb0 2024-06-10T00:56:37.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.094+0000 7fa496321640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa49010d510 con 0x7fa4901077e0 2024-06-10T00:56:37.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.094+0000 7fa496321640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa49010d810 con 0x7fa490102580 2024-06-10T00:56:37.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.094+0000 7fa48f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa490104eb0 0x7fa490198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:37.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.094+0000 7fa48f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa490104eb0 0x7fa490198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47182/0 (socket says 172.21.15.37:47182) 2024-06-10T00:56:37.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.094+0000 7fa48f7fe640 1 -- 172.21.15.37:0/1685279247 learned_addr learned my addr 172.21.15.37:0/1685279247 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:37.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.094+0000 7fa48ffff640 1 --2- 172.21.15.37:0/1685279247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa490102580 0x7fa490197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:37.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.094+0000 7fa48f7fe640 1 -- 172.21.15.37:0/1685279247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa490102580 msgr2=0x7fa490197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:37.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.094+0000 7fa48f7fe640 1 --2- 172.21.15.37:0/1685279247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa490102580 0x7fa490197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:37.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.094+0000 7fa48f7fe640 1 -- 172.21.15.37:0/1685279247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4901077e0 msgr2=0x7fa49019f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:37.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.094+0000 7fa48f7fe640 1 --2- 172.21.15.37:0/1685279247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4901077e0 0x7fa49019f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:37.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.094+0000 7fa48f7fe640 1 -- 172.21.15.37:0/1685279247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa49019f910 con 0x7fa490104eb0 2024-06-10T00:56:37.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.094+0000 7fa48f7fe640 1 --2- 172.21.15.37:0/1685279247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa490104eb0 0x7fa490198240 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa48400d740 tx=0x7fa48400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:37.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.094+0000 7fa48d7fa640 1 -- 172.21.15.37:0/1685279247 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa484017070 con 0x7fa490104eb0 2024-06-10T00:56:37.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.095+0000 7fa48d7fa640 1 -- 172.21.15.37:0/1685279247 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa48400de50 con 0x7fa490104eb0 2024-06-10T00:56:37.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.095+0000 7fa496321640 1 -- 172.21.15.37:0/1685279247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa49019fc00 con 0x7fa490104eb0 2024-06-10T00:56:37.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.095+0000 7fa48d7fa640 1 -- 172.21.15.37:0/1685279247 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa484004e00 con 0x7fa490104eb0 2024-06-10T00:56:37.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.095+0000 7fa496321640 1 -- 172.21.15.37:0/1685279247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4901a0100 con 0x7fa490104eb0 2024-06-10T00:56:37.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.095+0000 7fa496321640 1 -- 172.21.15.37:0/1685279247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4900fd9e0 con 0x7fa490104eb0 2024-06-10T00:56:37.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.097+0000 7fa48d7fa640 1 -- 172.21.15.37:0/1685279247 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fa484004fa0 con 0x7fa490104eb0 2024-06-10T00:56:37.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.097+0000 7fa48d7fa640 1 --2- 172.21.15.37:0/1685279247 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa4780a2c20 0x7fa4780a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:37.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.097+0000 7fa48d7fa640 1 -- 172.21.15.37:0/1685279247 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fa4840cf050 con 0x7fa490104eb0 2024-06-10T00:56:37.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.097+0000 7fa48ffff640 1 --2- 172.21.15.37:0/1685279247 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa4780a2c20 0x7fa4780a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:37.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.098+0000 7fa48ffff640 1 --2- 172.21.15.37:0/1685279247 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa4780a2c20 0x7fa4780a50e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa4800058c0 tx=0x7fa48000a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:37.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.102+0000 7fa48d7fa640 1 -- 172.21.15.37:0/1685279247 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa48409ba60 con 0x7fa490104eb0 2024-06-10T00:56:37.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.283+0000 7fa496321640 1 -- 172.21.15.37:0/1685279247 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa490061de0 con 0x7fa4780a2c20 2024-06-10T00:56:37.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.292+0000 7fa48d7fa640 1 -- 172.21.15.37:0/1685279247 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340965 (secure 0 0 0) 0x7fa490061de0 con 0x7fa4780a2c20 2024-06-10T00:56:37.296 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:56:37.298 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-06-10T00:56:37.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.296+0000 7fa496321640 1 -- 172.21.15.37:0/1685279247 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa4780a2c20 msgr2=0x7fa4780a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:37.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.296+0000 7fa496321640 1 --2- 172.21.15.37:0/1685279247 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa4780a2c20 0x7fa4780a50e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa4800058c0 tx=0x7fa48000a2d0 comp rx=0 tx=0).stop 2024-06-10T00:56:37.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.296+0000 7fa496321640 1 -- 172.21.15.37:0/1685279247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa490104eb0 msgr2=0x7fa490198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:37.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.296+0000 7fa496321640 1 --2- 172.21.15.37:0/1685279247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa490104eb0 0x7fa490198240 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa48400d740 tx=0x7fa48400dc00 comp rx=0 tx=0).stop 2024-06-10T00:56:37.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.296+0000 7fa496321640 1 -- 172.21.15.37:0/1685279247 shutdown_connections 2024-06-10T00:56:37.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.296+0000 7fa496321640 1 --2- 172.21.15.37:0/1685279247 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa4780a2c20 0x7fa4780a50e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:37.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.296+0000 7fa496321640 1 --2- 172.21.15.37:0/1685279247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4901077e0 0x7fa49019f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:37.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.296+0000 7fa496321640 1 --2- 172.21.15.37:0/1685279247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa490104eb0 0x7fa490198240 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:37.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.296+0000 7fa496321640 1 --2- 172.21.15.37:0/1685279247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa490102580 0x7fa490197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:37.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.296+0000 7fa496321640 1 -- 172.21.15.37:0/1685279247 >> 172.21.15.37:0/1685279247 conn(0x7fa4900fbef0 msgr2=0x7fa490102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:37.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.296+0000 7fa496321640 1 -- 172.21.15.37:0/1685279247 shutdown_connections 2024-06-10T00:56:37.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:37.296+0000 7fa496321640 1 -- 172.21.15.37:0/1685279247 wait complete. 2024-06-10T00:56:37.308 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":428,"stamp":"2024-06-10T00:56:36.410576+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":672924,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1201,"num_read_kb":737,"num_write":374,"num_write_kb":3088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":802,"ondisk_log_size":802,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370920,"kb_used_data":13448,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570904,"statfs":{"total":1151940427776,"available":1151560605696,"internally_reserved":0,"allocated":13770752,"data_stored":8866874,"data_compressed":497625,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7840,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":50,"num_write":13,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003319"},"pg_stats":[{"pgid":"3.3f","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373740+0000","last_change":"2024-06-10T00:52:35.971067+0000","last_active":"2024-06-10T00:55:20.373740+0000","last_peered":"2024-06-10T00:55:20.373740+0000","last_clean":"2024-06-10T00:55:20.373740+0000","last_became_active":"2024-06-10T00:52:35.970292+0000","last_became_peered":"2024-06-10T00:52:35.970292+0000","last_unstale":"2024-06-10T00:55:20.373740+0000","last_undegraded":"2024-06-10T00:55:20.373740+0000","last_fullsized":"2024-06-10T00:55:20.373740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:57:37.287238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364432+0000","last_change":"2024-06-10T00:52:35.960421+0000","last_active":"2024-06-10T00:55:20.364432+0000","last_peered":"2024-06-10T00:55:20.364432+0000","last_clean":"2024-06-10T00:55:20.364432+0000","last_became_active":"2024-06-10T00:52:35.960070+0000","last_became_peered":"2024-06-10T00:52:35.960070+0000","last_unstale":"2024-06-10T00:55:20.364432+0000","last_undegraded":"2024-06-10T00:55:20.364432+0000","last_fullsized":"2024-06-10T00:55:20.364432+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:48:02.616434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375701+0000","last_change":"2024-06-10T00:52:35.964778+0000","last_active":"2024-06-10T00:55:20.375701+0000","last_peered":"2024-06-10T00:55:20.375701+0000","last_clean":"2024-06-10T00:55:20.375701+0000","last_became_active":"2024-06-10T00:52:35.964463+0000","last_became_peered":"2024-06-10T00:52:35.964463+0000","last_unstale":"2024-06-10T00:55:20.375701+0000","last_undegraded":"2024-06-10T00:55:20.375701+0000","last_fullsized":"2024-06-10T00:55:20.375701+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:52:44.987363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.360279+0000","last_change":"2024-06-10T00:52:35.963226+0000","last_active":"2024-06-10T00:55:20.360279+0000","last_peered":"2024-06-10T00:55:20.360279+0000","last_clean":"2024-06-10T00:55:20.360279+0000","last_became_active":"2024-06-10T00:52:35.962862+0000","last_became_peered":"2024-06-10T00:52:35.962862+0000","last_unstale":"2024-06-10T00:55:20.360279+0000","last_undegraded":"2024-06-10T00:55:20.360279+0000","last_fullsized":"2024-06-10T00:55:20.360279+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:08:56.418071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.360258+0000","last_change":"2024-06-10T00:52:35.957427+0000","last_active":"2024-06-10T00:55:20.360258+0000","last_peered":"2024-06-10T00:55:20.360258+0000","last_clean":"2024-06-10T00:55:20.360258+0000","last_became_active":"2024-06-10T00:52:35.957049+0000","last_became_peered":"2024-06-10T00:52:35.957049+0000","last_unstale":"2024-06-10T00:55:20.360258+0000","last_undegraded":"2024-06-10T00:55:20.360258+0000","last_fullsized":"2024-06-10T00:55:20.360258+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:16:04.257210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.355809+0000","last_change":"2024-06-10T00:52:35.954251+0000","last_active":"2024-06-10T00:55:20.355809+0000","last_peered":"2024-06-10T00:55:20.355809+0000","last_clean":"2024-06-10T00:55:20.355809+0000","last_became_active":"2024-06-10T00:52:35.953908+0000","last_became_peered":"2024-06-10T00:52:35.953908+0000","last_unstale":"2024-06-10T00:55:20.355809+0000","last_undegraded":"2024-06-10T00:55:20.355809+0000","last_fullsized":"2024-06-10T00:55:20.355809+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:55:25.125836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364343+0000","last_change":"2024-06-10T00:52:35.956822+0000","last_active":"2024-06-10T00:55:20.364343+0000","last_peered":"2024-06-10T00:55:20.364343+0000","last_clean":"2024-06-10T00:55:20.364343+0000","last_became_active":"2024-06-10T00:52:35.956466+0000","last_became_peered":"2024-06-10T00:52:35.956466+0000","last_unstale":"2024-06-10T00:55:20.364343+0000","last_undegraded":"2024-06-10T00:55:20.364343+0000","last_fullsized":"2024-06-10T00:55:20.364343+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:36:41.534392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.369347+0000","last_change":"2024-06-10T00:52:35.962978+0000","last_active":"2024-06-10T00:55:20.369347+0000","last_peered":"2024-06-10T00:55:20.369347+0000","last_clean":"2024-06-10T00:55:20.369347+0000","last_became_active":"2024-06-10T00:52:35.962609+0000","last_became_peered":"2024-06-10T00:52:35.962609+0000","last_unstale":"2024-06-10T00:55:20.369347+0000","last_undegraded":"2024-06-10T00:55:20.369347+0000","last_fullsized":"2024-06-10T00:55:20.369347+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:20:57.469577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.377074+0000","last_change":"2024-06-10T00:52:35.963203+0000","last_active":"2024-06-10T00:55:20.377074+0000","last_peered":"2024-06-10T00:55:20.377074+0000","last_clean":"2024-06-10T00:55:20.377074+0000","last_became_active":"2024-06-10T00:52:35.962789+0000","last_became_peered":"2024-06-10T00:52:35.962789+0000","last_unstale":"2024-06-10T00:55:20.377074+0000","last_undegraded":"2024-06-10T00:55:20.377074+0000","last_fullsized":"2024-06-10T00:55:20.377074+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:00:17.819193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"100'16","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.356376+0000","last_change":"2024-06-10T00:52:35.966940+0000","last_active":"2024-06-10T00:55:20.356376+0000","last_peered":"2024-06-10T00:55:20.356376+0000","last_clean":"2024-06-10T00:55:20.356376+0000","last_became_active":"2024-06-10T00:52:35.966554+0000","last_became_peered":"2024-06-10T00:52:35.966554+0000","last_unstale":"2024-06-10T00:55:20.356376+0000","last_undegraded":"2024-06-10T00:55:20.356376+0000","last_fullsized":"2024-06-10T00:55:20.356376+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:00.374437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372673+0000","last_change":"2024-06-10T00:52:35.960927+0000","last_active":"2024-06-10T00:55:20.372673+0000","last_peered":"2024-06-10T00:55:20.372673+0000","last_clean":"2024-06-10T00:55:20.372673+0000","last_became_active":"2024-06-10T00:52:35.960577+0000","last_became_peered":"2024-06-10T00:52:35.960577+0000","last_unstale":"2024-06-10T00:55:20.372673+0000","last_undegraded":"2024-06-10T00:55:20.372673+0000","last_fullsized":"2024-06-10T00:55:20.372673+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:50:02.366695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.355167+0000","last_change":"2024-06-10T00:52:35.964435+0000","last_active":"2024-06-10T00:55:20.355167+0000","last_peered":"2024-06-10T00:55:20.355167+0000","last_clean":"2024-06-10T00:55:20.355167+0000","last_became_active":"2024-06-10T00:52:35.964069+0000","last_became_peered":"2024-06-10T00:52:35.964069+0000","last_unstale":"2024-06-10T00:55:20.355167+0000","last_undegraded":"2024-06-10T00:55:20.355167+0000","last_fullsized":"2024-06-10T00:55:20.355167+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:06.708551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.361787+0000","last_change":"2024-06-10T00:52:35.966722+0000","last_active":"2024-06-10T00:55:20.361787+0000","last_peered":"2024-06-10T00:55:20.361787+0000","last_clean":"2024-06-10T00:55:20.361787+0000","last_became_active":"2024-06-10T00:52:35.966366+0000","last_became_peered":"2024-06-10T00:52:35.966366+0000","last_unstale":"2024-06-10T00:55:20.361787+0000","last_undegraded":"2024-06-10T00:55:20.361787+0000","last_fullsized":"2024-06-10T00:55:20.361787+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:57:46.021047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.371736+0000","last_change":"2024-06-10T00:52:35.955307+0000","last_active":"2024-06-10T00:55:20.371736+0000","last_peered":"2024-06-10T00:55:20.371736+0000","last_clean":"2024-06-10T00:55:20.371736+0000","last_became_active":"2024-06-10T00:52:35.954140+0000","last_became_peered":"2024-06-10T00:52:35.954140+0000","last_unstale":"2024-06-10T00:55:20.371736+0000","last_undegraded":"2024-06-10T00:55:20.371736+0000","last_fullsized":"2024-06-10T00:55:20.371736+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:47:22.493598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373915+0000","last_change":"2024-06-10T00:52:35.960592+0000","last_active":"2024-06-10T00:55:20.373915+0000","last_peered":"2024-06-10T00:55:20.373915+0000","last_clean":"2024-06-10T00:55:20.373915+0000","last_became_active":"2024-06-10T00:52:35.960263+0000","last_became_peered":"2024-06-10T00:52:35.960263+0000","last_unstale":"2024-06-10T00:55:20.373915+0000","last_undegraded":"2024-06-10T00:55:20.373915+0000","last_fullsized":"2024-06-10T00:55:20.373915+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:47:11.658666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372126+0000","last_change":"2024-06-10T00:52:35.965796+0000","last_active":"2024-06-10T00:55:20.372126+0000","last_peered":"2024-06-10T00:55:20.372126+0000","last_clean":"2024-06-10T00:55:20.372126+0000","last_became_active":"2024-06-10T00:52:35.963965+0000","last_became_peered":"2024-06-10T00:52:35.963965+0000","last_unstale":"2024-06-10T00:55:20.372126+0000","last_undegraded":"2024-06-10T00:55:20.372126+0000","last_fullsized":"2024-06-10T00:55:20.372126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:29:47.149852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374090+0000","last_change":"2024-06-10T00:52:35.970714+0000","last_active":"2024-06-10T00:55:20.374090+0000","last_peered":"2024-06-10T00:55:20.374090+0000","last_clean":"2024-06-10T00:55:20.374090+0000","last_became_active":"2024-06-10T00:52:35.970173+0000","last_became_peered":"2024-06-10T00:52:35.970173+0000","last_unstale":"2024-06-10T00:55:20.374090+0000","last_undegraded":"2024-06-10T00:55:20.374090+0000","last_fullsized":"2024-06-10T00:55:20.374090+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:33:28.183156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.370737+0000","last_change":"2024-06-10T00:52:35.963052+0000","last_active":"2024-06-10T00:55:20.370737+0000","last_peered":"2024-06-10T00:55:20.370737+0000","last_clean":"2024-06-10T00:55:20.370737+0000","last_became_active":"2024-06-10T00:52:35.962674+0000","last_became_peered":"2024-06-10T00:52:35.962674+0000","last_unstale":"2024-06-10T00:55:20.370737+0000","last_undegraded":"2024-06-10T00:55:20.370737+0000","last_fullsized":"2024-06-10T00:55:20.370737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:56:38.836854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"100'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.363943+0000","last_change":"2024-06-10T00:53:22.130336+0000","last_active":"2024-06-10T00:55:20.363943+0000","last_peered":"2024-06-10T00:55:20.363943+0000","last_clean":"2024-06-10T00:55:20.363943+0000","last_became_active":"2024-06-10T00:53:22.129910+0000","last_became_peered":"2024-06-10T00:53:22.129910+0000","last_unstale":"2024-06-10T00:55:20.363943+0000","last_undegraded":"2024-06-10T00:55:20.363943+0000","last_fullsized":"2024-06-10T00:55:20.363943+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:23:50.600821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554286+0000","last_change":"2024-06-10T00:52:45.997276+0000","last_active":"2024-06-10T00:54:22.554286+0000","last_peered":"2024-06-10T00:54:22.554286+0000","last_clean":"2024-06-10T00:54:22.554286+0000","last_became_active":"2024-06-10T00:52:45.995638+0000","last_became_peered":"2024-06-10T00:52:45.995638+0000","last_unstale":"2024-06-10T00:54:22.554286+0000","last_undegraded":"2024-06-10T00:54:22.554286+0000","last_fullsized":"2024-06-10T00:54:22.554286+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:14:20.922348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556499+0000","last_change":"2024-06-10T00:52:32.760888+0000","last_active":"2024-06-10T00:54:22.556499+0000","last_peered":"2024-06-10T00:54:22.556499+0000","last_clean":"2024-06-10T00:54:22.556499+0000","last_became_active":"2024-06-10T00:52:32.760531+0000","last_became_peered":"2024-06-10T00:52:32.760531+0000","last_unstale":"2024-06-10T00:54:22.556499+0000","last_undegraded":"2024-06-10T00:54:22.556499+0000","last_fullsized":"2024-06-10T00:54:22.556499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:05:59.052251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.371793+0000","last_change":"2024-06-10T00:52:35.967626+0000","last_active":"2024-06-10T00:55:20.371793+0000","last_peered":"2024-06-10T00:55:20.371793+0000","last_clean":"2024-06-10T00:55:20.371793+0000","last_became_active":"2024-06-10T00:52:35.967295+0000","last_became_peered":"2024-06-10T00:52:35.967295+0000","last_unstale":"2024-06-10T00:55:20.371793+0000","last_undegraded":"2024-06-10T00:55:20.371793+0000","last_fullsized":"2024-06-10T00:55:20.371793+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:07:55.054327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668105+0000","last_change":"2024-06-10T00:52:45.993974+0000","last_active":"2024-06-10T00:54:22.668105+0000","last_peered":"2024-06-10T00:54:22.668105+0000","last_clean":"2024-06-10T00:54:22.668105+0000","last_became_active":"2024-06-10T00:52:45.993242+0000","last_became_peered":"2024-06-10T00:52:45.993242+0000","last_unstale":"2024-06-10T00:54:22.668105+0000","last_undegraded":"2024-06-10T00:54:22.668105+0000","last_fullsized":"2024-06-10T00:54:22.668105+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:28:12.810368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:33.628114+0000","last_change":"2024-06-10T00:52:32.959905+0000","last_active":"2024-06-10T00:54:33.628114+0000","last_peered":"2024-06-10T00:54:33.628114+0000","last_clean":"2024-06-10T00:54:33.628114+0000","last_became_active":"2024-06-10T00:52:32.959551+0000","last_became_peered":"2024-06-10T00:52:32.959551+0000","last_unstale":"2024-06-10T00:54:33.628114+0000","last_undegraded":"2024-06-10T00:54:33.628114+0000","last_fullsized":"2024-06-10T00:54:33.628114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:33:48.029129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375782+0000","last_change":"2024-06-10T00:52:35.965790+0000","last_active":"2024-06-10T00:55:20.375782+0000","last_peered":"2024-06-10T00:55:20.375782+0000","last_clean":"2024-06-10T00:55:20.375782+0000","last_became_active":"2024-06-10T00:52:35.965475+0000","last_became_peered":"2024-06-10T00:52:35.965475+0000","last_unstale":"2024-06-10T00:55:20.375782+0000","last_undegraded":"2024-06-10T00:55:20.375782+0000","last_fullsized":"2024-06-10T00:55:20.375782+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:24:02.894411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.667953+0000","last_change":"2024-06-10T00:52:45.994464+0000","last_active":"2024-06-10T00:54:22.667953+0000","last_peered":"2024-06-10T00:54:22.667953+0000","last_clean":"2024-06-10T00:54:22.667953+0000","last_became_active":"2024-06-10T00:52:45.994022+0000","last_became_peered":"2024-06-10T00:52:45.994022+0000","last_unstale":"2024-06-10T00:54:22.667953+0000","last_undegraded":"2024-06-10T00:54:22.667953+0000","last_fullsized":"2024-06-10T00:54:22.667953+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:27:25.063074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556658+0000","last_change":"2024-06-10T00:52:32.763885+0000","last_active":"2024-06-10T00:54:22.556658+0000","last_peered":"2024-06-10T00:54:22.556658+0000","last_clean":"2024-06-10T00:54:22.556658+0000","last_became_active":"2024-06-10T00:52:32.763606+0000","last_became_peered":"2024-06-10T00:52:32.763606+0000","last_unstale":"2024-06-10T00:54:22.556658+0000","last_undegraded":"2024-06-10T00:54:22.556658+0000","last_fullsized":"2024-06-10T00:54:22.556658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:01:58.161168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372890+0000","last_change":"2024-06-10T00:52:35.966948+0000","last_active":"2024-06-10T00:55:20.372890+0000","last_peered":"2024-06-10T00:55:20.372890+0000","last_clean":"2024-06-10T00:55:20.372890+0000","last_became_active":"2024-06-10T00:52:35.966345+0000","last_became_peered":"2024-06-10T00:52:35.966345+0000","last_unstale":"2024-06-10T00:55:20.372890+0000","last_undegraded":"2024-06-10T00:55:20.372890+0000","last_fullsized":"2024-06-10T00:55:20.372890+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:58:27.169201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.667762+0000","last_change":"2024-06-10T00:52:45.993645+0000","last_active":"2024-06-10T00:54:22.667762+0000","last_peered":"2024-06-10T00:54:22.667762+0000","last_clean":"2024-06-10T00:54:22.667762+0000","last_became_active":"2024-06-10T00:52:45.993183+0000","last_became_peered":"2024-06-10T00:52:45.993183+0000","last_unstale":"2024-06-10T00:54:22.667762+0000","last_undegraded":"2024-06-10T00:54:22.667762+0000","last_fullsized":"2024-06-10T00:54:22.667762+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:47:00.541427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.565600+0000","last_change":"2024-06-10T00:54:22.565600+0000","last_active":"2024-06-10T00:54:22.565600+0000","last_peered":"2024-06-10T00:54:22.565600+0000","last_clean":"2024-06-10T00:54:22.565600+0000","last_became_active":"2024-06-10T00:54:22.562561+0000","last_became_peered":"2024-06-10T00:54:22.562561+0000","last_unstale":"2024-06-10T00:54:22.565600+0000","last_undegraded":"2024-06-10T00:54:22.565600+0000","last_fullsized":"2024-06-10T00:54:22.565600+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:47:58.748050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"100'15","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365162+0000","last_change":"2024-06-10T00:53:22.199668+0000","last_active":"2024-06-10T00:55:20.365162+0000","last_peered":"2024-06-10T00:55:20.365162+0000","last_clean":"2024-06-10T00:55:20.365162+0000","last_became_active":"2024-06-10T00:53:22.199329+0000","last_became_peered":"2024-06-10T00:53:22.199329+0000","last_unstale":"2024-06-10T00:55:20.365162+0000","last_undegraded":"2024-06-10T00:55:20.365162+0000","last_fullsized":"2024-06-10T00:55:20.365162+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:37.640478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554689+0000","last_change":"2024-06-10T00:52:45.995767+0000","last_active":"2024-06-10T00:54:22.554689+0000","last_peered":"2024-06-10T00:54:22.554689+0000","last_clean":"2024-06-10T00:54:22.554689+0000","last_became_active":"2024-06-10T00:52:45.994659+0000","last_became_peered":"2024-06-10T00:52:45.994659+0000","last_unstale":"2024-06-10T00:54:22.554689+0000","last_undegraded":"2024-06-10T00:54:22.554689+0000","last_fullsized":"2024-06-10T00:54:22.554689+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:03:45.652248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'8","reported_seq":227,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:30.554987+0000","last_change":"2024-06-10T00:52:32.761637+0000","last_active":"2024-06-10T00:56:30.554987+0000","last_peered":"2024-06-10T00:56:30.554987+0000","last_clean":"2024-06-10T00:56:30.554987+0000","last_became_active":"2024-06-10T00:52:32.761144+0000","last_became_peered":"2024-06-10T00:52:32.761144+0000","last_unstale":"2024-06-10T00:56:30.554987+0000","last_undegraded":"2024-06-10T00:56:30.554987+0000","last_fullsized":"2024-06-10T00:56:30.554987+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:55:05.616564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1461,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.362574+0000","last_change":"2024-06-10T00:52:35.962683+0000","last_active":"2024-06-10T00:55:20.362574+0000","last_peered":"2024-06-10T00:55:20.362574+0000","last_clean":"2024-06-10T00:55:20.362574+0000","last_became_active":"2024-06-10T00:52:35.962327+0000","last_became_peered":"2024-06-10T00:52:35.962327+0000","last_unstale":"2024-06-10T00:55:20.362574+0000","last_undegraded":"2024-06-10T00:55:20.362574+0000","last_fullsized":"2024-06-10T00:55:20.362574+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:38:19.063306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554563+0000","last_change":"2024-06-10T00:52:45.996882+0000","last_active":"2024-06-10T00:54:22.554563+0000","last_peered":"2024-06-10T00:54:22.554563+0000","last_clean":"2024-06-10T00:54:22.554563+0000","last_became_active":"2024-06-10T00:52:45.995767+0000","last_became_peered":"2024-06-10T00:52:45.995767+0000","last_unstale":"2024-06-10T00:54:22.554563+0000","last_undegraded":"2024-06-10T00:54:22.554563+0000","last_fullsized":"2024-06-10T00:54:22.554563+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:28:03.766944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553970+0000","last_change":"2024-06-10T00:53:22.200546+0000","last_active":"2024-06-10T00:54:22.553970+0000","last_peered":"2024-06-10T00:54:22.553970+0000","last_clean":"2024-06-10T00:54:22.553970+0000","last_became_active":"2024-06-10T00:53:22.200059+0000","last_became_peered":"2024-06-10T00:53:22.200059+0000","last_unstale":"2024-06-10T00:54:22.553970+0000","last_undegraded":"2024-06-10T00:54:22.553970+0000","last_fullsized":"2024-06-10T00:54:22.553970+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:08:47.682594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.356418+0000","last_change":"2024-06-10T00:52:35.963105+0000","last_active":"2024-06-10T00:55:20.356418+0000","last_peered":"2024-06-10T00:55:20.356418+0000","last_clean":"2024-06-10T00:55:20.356418+0000","last_became_active":"2024-06-10T00:52:35.962627+0000","last_became_peered":"2024-06-10T00:52:35.962627+0000","last_unstale":"2024-06-10T00:55:20.356418+0000","last_undegraded":"2024-06-10T00:55:20.356418+0000","last_fullsized":"2024-06-10T00:55:20.356418+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:55:51.231795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.559190+0000","last_change":"2024-06-10T00:52:45.992557+0000","last_active":"2024-06-10T00:54:22.559190+0000","last_peered":"2024-06-10T00:54:22.559190+0000","last_clean":"2024-06-10T00:54:22.559190+0000","last_became_active":"2024-06-10T00:52:45.992166+0000","last_became_peered":"2024-06-10T00:52:45.992166+0000","last_unstale":"2024-06-10T00:54:22.559190+0000","last_undegraded":"2024-06-10T00:54:22.559190+0000","last_fullsized":"2024-06-10T00:54:22.559190+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:49:22.368409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553428+0000","last_change":"2024-06-10T00:52:32.751867+0000","last_active":"2024-06-10T00:54:22.553428+0000","last_peered":"2024-06-10T00:54:22.553428+0000","last_clean":"2024-06-10T00:54:22.553428+0000","last_became_active":"2024-06-10T00:52:32.751516+0000","last_became_peered":"2024-06-10T00:52:32.751516+0000","last_unstale":"2024-06-10T00:54:22.553428+0000","last_undegraded":"2024-06-10T00:54:22.553428+0000","last_fullsized":"2024-06-10T00:54:22.553428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:37:55.920613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.370097+0000","last_change":"2024-06-10T00:52:35.957772+0000","last_active":"2024-06-10T00:55:20.370097+0000","last_peered":"2024-06-10T00:55:20.370097+0000","last_clean":"2024-06-10T00:55:20.370097+0000","last_became_active":"2024-06-10T00:52:35.957470+0000","last_became_peered":"2024-06-10T00:52:35.957470+0000","last_unstale":"2024-06-10T00:55:20.370097+0000","last_undegraded":"2024-06-10T00:55:20.370097+0000","last_fullsized":"2024-06-10T00:55:20.370097+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:16:43.645292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878568+0000","last_change":"2024-06-10T00:52:45.996807+0000","last_active":"2024-06-10T00:54:22.878568+0000","last_peered":"2024-06-10T00:54:22.878568+0000","last_clean":"2024-06-10T00:54:22.878568+0000","last_became_active":"2024-06-10T00:52:45.996378+0000","last_became_peered":"2024-06-10T00:52:45.996378+0000","last_unstale":"2024-06-10T00:54:22.878568+0000","last_undegraded":"2024-06-10T00:54:22.878568+0000","last_fullsized":"2024-06-10T00:54:22.878568+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:40:54.251790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'1","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561873+0000","last_change":"2024-06-10T00:52:32.759625+0000","last_active":"2024-06-10T00:54:22.561873+0000","last_peered":"2024-06-10T00:54:22.561873+0000","last_clean":"2024-06-10T00:54:22.561873+0000","last_became_active":"2024-06-10T00:52:32.759299+0000","last_became_peered":"2024-06-10T00:52:32.759299+0000","last_unstale":"2024-06-10T00:54:22.561873+0000","last_undegraded":"2024-06-10T00:54:22.561873+0000","last_fullsized":"2024-06-10T00:54:22.561873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:11:44.966453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365673+0000","last_change":"2024-06-10T00:52:35.957672+0000","last_active":"2024-06-10T00:55:20.365673+0000","last_peered":"2024-06-10T00:55:20.365673+0000","last_clean":"2024-06-10T00:55:20.365673+0000","last_became_active":"2024-06-10T00:52:35.957328+0000","last_became_peered":"2024-06-10T00:52:35.957328+0000","last_unstale":"2024-06-10T00:55:20.365673+0000","last_undegraded":"2024-06-10T00:55:20.365673+0000","last_fullsized":"2024-06-10T00:55:20.365673+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:43:59.658178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601278+0000","last_change":"2024-06-10T00:52:45.996677+0000","last_active":"2024-06-10T00:54:22.601278+0000","last_peered":"2024-06-10T00:54:22.601278+0000","last_clean":"2024-06-10T00:54:22.601278+0000","last_became_active":"2024-06-10T00:52:45.994483+0000","last_became_peered":"2024-06-10T00:52:45.994483+0000","last_unstale":"2024-06-10T00:54:22.601278+0000","last_undegraded":"2024-06-10T00:54:22.601278+0000","last_fullsized":"2024-06-10T00:54:22.601278+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:03:28.712072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'5","reported_seq":225,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:34.560617+0000","last_change":"2024-06-10T00:52:32.754829+0000","last_active":"2024-06-10T00:56:34.560617+0000","last_peered":"2024-06-10T00:56:34.560617+0000","last_clean":"2024-06-10T00:56:34.560617+0000","last_became_active":"2024-06-10T00:52:32.754386+0000","last_became_peered":"2024-06-10T00:52:32.754386+0000","last_unstale":"2024-06-10T00:56:34.560617+0000","last_undegraded":"2024-06-10T00:56:34.560617+0000","last_fullsized":"2024-06-10T00:56:34.560617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:00.870324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":175,"num_read_kb":175,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.371610+0000","last_change":"2024-06-10T00:52:35.966871+0000","last_active":"2024-06-10T00:55:20.371610+0000","last_peered":"2024-06-10T00:55:20.371610+0000","last_clean":"2024-06-10T00:55:20.371610+0000","last_became_active":"2024-06-10T00:52:35.966492+0000","last_became_peered":"2024-06-10T00:52:35.966492+0000","last_unstale":"2024-06-10T00:55:20.371610+0000","last_undegraded":"2024-06-10T00:55:20.371610+0000","last_fullsized":"2024-06-10T00:55:20.371610+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:24:41.048935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.879950+0000","last_change":"2024-06-10T00:54:22.879457+0000","last_active":"2024-06-10T00:54:22.879950+0000","last_peered":"2024-06-10T00:54:22.879950+0000","last_clean":"2024-06-10T00:54:22.879950+0000","last_became_active":"2024-06-10T00:54:22.878978+0000","last_became_peered":"2024-06-10T00:54:22.878978+0000","last_unstale":"2024-06-10T00:54:22.879950+0000","last_undegraded":"2024-06-10T00:54:22.879950+0000","last_fullsized":"2024-06-10T00:54:22.879950+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:46:38.819250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404659+0000","last_change":"2024-06-10T00:52:32.765794+0000","last_active":"2024-06-10T00:54:24.404659+0000","last_peered":"2024-06-10T00:54:24.404659+0000","last_clean":"2024-06-10T00:54:24.404659+0000","last_became_active":"2024-06-10T00:52:32.765276+0000","last_became_peered":"2024-06-10T00:52:32.765276+0000","last_unstale":"2024-06-10T00:54:24.404659+0000","last_undegraded":"2024-06-10T00:54:24.404659+0000","last_fullsized":"2024-06-10T00:54:24.404659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:06:51.209012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365186+0000","last_change":"2024-06-10T00:52:35.954897+0000","last_active":"2024-06-10T00:55:20.365186+0000","last_peered":"2024-06-10T00:55:20.365186+0000","last_clean":"2024-06-10T00:55:20.365186+0000","last_became_active":"2024-06-10T00:52:35.954565+0000","last_became_peered":"2024-06-10T00:52:35.954565+0000","last_unstale":"2024-06-10T00:55:20.365186+0000","last_undegraded":"2024-06-10T00:55:20.365186+0000","last_fullsized":"2024-06-10T00:55:20.365186+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:39:26.702804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668190+0000","last_change":"2024-06-10T00:52:45.992194+0000","last_active":"2024-06-10T00:54:22.668190+0000","last_peered":"2024-06-10T00:54:22.668190+0000","last_clean":"2024-06-10T00:54:22.668190+0000","last_became_active":"2024-06-10T00:52:45.991453+0000","last_became_peered":"2024-06-10T00:52:45.991453+0000","last_unstale":"2024-06-10T00:54:22.668190+0000","last_undegraded":"2024-06-10T00:54:22.668190+0000","last_fullsized":"2024-06-10T00:54:22.668190+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:11:10.775610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561610+0000","last_change":"2024-06-10T00:52:32.962214+0000","last_active":"2024-06-10T00:54:22.561610+0000","last_peered":"2024-06-10T00:54:22.561610+0000","last_clean":"2024-06-10T00:54:22.561610+0000","last_became_active":"2024-06-10T00:52:32.961835+0000","last_became_peered":"2024-06-10T00:52:32.961835+0000","last_unstale":"2024-06-10T00:54:22.561610+0000","last_undegraded":"2024-06-10T00:54:22.561610+0000","last_fullsized":"2024-06-10T00:54:22.561610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:36:44.724122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.377485+0000","last_change":"2024-06-10T00:52:35.963413+0000","last_active":"2024-06-10T00:55:20.377485+0000","last_peered":"2024-06-10T00:55:20.377485+0000","last_clean":"2024-06-10T00:55:20.377485+0000","last_became_active":"2024-06-10T00:52:35.962922+0000","last_became_peered":"2024-06-10T00:52:35.962922+0000","last_unstale":"2024-06-10T00:55:20.377485+0000","last_undegraded":"2024-06-10T00:55:20.377485+0000","last_fullsized":"2024-06-10T00:55:20.377485+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:05:32.164696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":34,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.560580+0000","last_change":"2024-06-10T00:54:22.560196+0000","last_active":"2024-06-10T00:54:22.560580+0000","last_peered":"2024-06-10T00:54:22.560580+0000","last_clean":"2024-06-10T00:54:22.560580+0000","last_became_active":"2024-06-10T00:54:22.559826+0000","last_became_peered":"2024-06-10T00:54:22.559826+0000","last_unstale":"2024-06-10T00:54:22.560580+0000","last_undegraded":"2024-06-10T00:54:22.560580+0000","last_fullsized":"2024-06-10T00:54:22.560580+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:09:28.456394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'46","reported_seq":455,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:35.566931+0000","last_change":"2024-06-10T00:52:32.761592+0000","last_active":"2024-06-10T00:56:35.566931+0000","last_peered":"2024-06-10T00:56:35.566931+0000","last_clean":"2024-06-10T00:56:35.566931+0000","last_became_active":"2024-06-10T00:52:32.761169+0000","last_became_peered":"2024-06-10T00:52:32.761169+0000","last_unstale":"2024-06-10T00:56:35.566931+0000","last_undegraded":"2024-06-10T00:56:35.566931+0000","last_fullsized":"2024-06-10T00:56:35.566931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:08:33.015035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77873,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":364,"num_read_kb":260,"num_write":47,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.351972+0000","last_change":"2024-06-10T00:52:35.955304+0000","last_active":"2024-06-10T00:55:20.351972+0000","last_peered":"2024-06-10T00:55:20.351972+0000","last_clean":"2024-06-10T00:55:20.351972+0000","last_became_active":"2024-06-10T00:52:35.954949+0000","last_became_peered":"2024-06-10T00:52:35.954949+0000","last_unstale":"2024-06-10T00:55:20.351972+0000","last_undegraded":"2024-06-10T00:55:20.351972+0000","last_fullsized":"2024-06-10T00:55:20.351972+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:56:42.890022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561827+0000","last_change":"2024-06-10T00:52:45.992974+0000","last_active":"2024-06-10T00:54:22.561827+0000","last_peered":"2024-06-10T00:54:22.561827+0000","last_clean":"2024-06-10T00:54:22.561827+0000","last_became_active":"2024-06-10T00:52:45.992490+0000","last_became_peered":"2024-06-10T00:52:45.992490+0000","last_unstale":"2024-06-10T00:54:22.561827+0000","last_undegraded":"2024-06-10T00:54:22.561827+0000","last_fullsized":"2024-06-10T00:54:22.561827+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:03:45.543476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554437+0000","last_change":"2024-06-10T00:52:32.762347+0000","last_active":"2024-06-10T00:54:22.554437+0000","last_peered":"2024-06-10T00:54:22.554437+0000","last_clean":"2024-06-10T00:54:22.554437+0000","last_became_active":"2024-06-10T00:52:32.761049+0000","last_became_peered":"2024-06-10T00:52:32.761049+0000","last_unstale":"2024-06-10T00:54:22.554437+0000","last_undegraded":"2024-06-10T00:54:22.554437+0000","last_fullsized":"2024-06-10T00:54:22.554437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:29:38.384451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364495+0000","last_change":"2024-06-10T00:52:35.961299+0000","last_active":"2024-06-10T00:55:20.364495+0000","last_peered":"2024-06-10T00:55:20.364495+0000","last_clean":"2024-06-10T00:55:20.364495+0000","last_became_active":"2024-06-10T00:52:35.960966+0000","last_became_peered":"2024-06-10T00:52:35.960966+0000","last_unstale":"2024-06-10T00:55:20.364495+0000","last_undegraded":"2024-06-10T00:55:20.364495+0000","last_fullsized":"2024-06-10T00:55:20.364495+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:32:21.269689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.559474+0000","last_change":"2024-06-10T00:52:46.002091+0000","last_active":"2024-06-10T00:54:22.559474+0000","last_peered":"2024-06-10T00:54:22.559474+0000","last_clean":"2024-06-10T00:54:22.559474+0000","last_became_active":"2024-06-10T00:52:46.001677+0000","last_became_peered":"2024-06-10T00:52:46.001677+0000","last_unstale":"2024-06-10T00:54:22.559474+0000","last_undegraded":"2024-06-10T00:54:22.559474+0000","last_fullsized":"2024-06-10T00:54:22.559474+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:16:56.405720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'3","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.562287+0000","last_change":"2024-06-10T00:52:32.756765+0000","last_active":"2024-06-10T00:54:22.562287+0000","last_peered":"2024-06-10T00:54:22.562287+0000","last_clean":"2024-06-10T00:54:22.562287+0000","last_became_active":"2024-06-10T00:52:32.756470+0000","last_became_peered":"2024-06-10T00:52:32.756470+0000","last_unstale":"2024-06-10T00:54:22.562287+0000","last_undegraded":"2024-06-10T00:54:22.562287+0000","last_fullsized":"2024-06-10T00:54:22.562287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:29:24.854913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"100'15","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364256+0000","last_change":"2024-06-10T00:52:35.960693+0000","last_active":"2024-06-10T00:55:20.364256+0000","last_peered":"2024-06-10T00:55:20.364256+0000","last_clean":"2024-06-10T00:55:20.364256+0000","last_became_active":"2024-06-10T00:52:35.958397+0000","last_became_peered":"2024-06-10T00:52:35.958397+0000","last_unstale":"2024-06-10T00:55:20.364256+0000","last_undegraded":"2024-06-10T00:55:20.364256+0000","last_fullsized":"2024-06-10T00:55:20.364256+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:23:47.556462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556200+0000","last_change":"2024-06-10T00:52:46.001475+0000","last_active":"2024-06-10T00:54:22.556200+0000","last_peered":"2024-06-10T00:54:22.556200+0000","last_clean":"2024-06-10T00:54:22.556200+0000","last_became_active":"2024-06-10T00:52:46.000983+0000","last_became_peered":"2024-06-10T00:52:46.000983+0000","last_unstale":"2024-06-10T00:54:22.556200+0000","last_undegraded":"2024-06-10T00:54:22.556200+0000","last_fullsized":"2024-06-10T00:54:22.556200+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:23:56.285356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"98'3","reported_seq":221,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:31.559742+0000","last_change":"2024-06-10T00:52:32.756665+0000","last_active":"2024-06-10T00:56:31.559742+0000","last_peered":"2024-06-10T00:56:31.559742+0000","last_clean":"2024-06-10T00:56:31.559742+0000","last_became_active":"2024-06-10T00:52:32.756221+0000","last_became_peered":"2024-06-10T00:52:32.756221+0000","last_unstale":"2024-06-10T00:56:31.559742+0000","last_undegraded":"2024-06-10T00:56:31.559742+0000","last_fullsized":"2024-06-10T00:56:31.559742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:50:34.789063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":175,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"100'6","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364652+0000","last_change":"2024-06-10T00:53:22.125844+0000","last_active":"2024-06-10T00:55:20.364652+0000","last_peered":"2024-06-10T00:55:20.364652+0000","last_clean":"2024-06-10T00:55:20.364652+0000","last_became_active":"2024-06-10T00:53:22.125449+0000","last_became_peered":"2024-06-10T00:53:22.125449+0000","last_unstale":"2024-06-10T00:55:20.364652+0000","last_undegraded":"2024-06-10T00:55:20.364652+0000","last_fullsized":"2024-06-10T00:55:20.364652+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:08:42.936702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553904+0000","last_change":"2024-06-10T00:52:45.999300+0000","last_active":"2024-06-10T00:54:22.553904+0000","last_peered":"2024-06-10T00:54:22.553904+0000","last_clean":"2024-06-10T00:54:22.553904+0000","last_became_active":"2024-06-10T00:52:45.998860+0000","last_became_peered":"2024-06-10T00:52:45.998860+0000","last_unstale":"2024-06-10T00:54:22.553904+0000","last_undegraded":"2024-06-10T00:54:22.553904+0000","last_fullsized":"2024-06-10T00:54:22.553904+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:14:24.936870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":75,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.557182+0000","last_change":"2024-06-10T00:53:22.210710+0000","last_active":"2024-06-10T00:54:22.557182+0000","last_peered":"2024-06-10T00:54:22.557182+0000","last_clean":"2024-06-10T00:54:22.557182+0000","last_became_active":"2024-06-10T00:53:22.202461+0000","last_became_peered":"2024-06-10T00:53:22.202461+0000","last_unstale":"2024-06-10T00:54:22.557182+0000","last_undegraded":"2024-06-10T00:54:22.557182+0000","last_fullsized":"2024-06-10T00:54:22.557182+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:28:14.671776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364560+0000","last_change":"2024-06-10T00:52:35.966960+0000","last_active":"2024-06-10T00:55:20.364560+0000","last_peered":"2024-06-10T00:55:20.364560+0000","last_clean":"2024-06-10T00:55:20.364560+0000","last_became_active":"2024-06-10T00:52:35.966630+0000","last_became_peered":"2024-06-10T00:52:35.966630+0000","last_unstale":"2024-06-10T00:55:20.364560+0000","last_undegraded":"2024-06-10T00:55:20.364560+0000","last_fullsized":"2024-06-10T00:55:20.364560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:04.117405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561586+0000","last_change":"2024-06-10T00:52:45.992955+0000","last_active":"2024-06-10T00:54:22.561586+0000","last_peered":"2024-06-10T00:54:22.561586+0000","last_clean":"2024-06-10T00:54:22.561586+0000","last_became_active":"2024-06-10T00:52:45.992457+0000","last_became_peered":"2024-06-10T00:52:45.992457+0000","last_unstale":"2024-06-10T00:54:22.561586+0000","last_undegraded":"2024-06-10T00:54:22.561586+0000","last_fullsized":"2024-06-10T00:54:22.561586+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:13:49.491183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556768+0000","last_change":"2024-06-10T00:52:32.763794+0000","last_active":"2024-06-10T00:54:22.556768+0000","last_peered":"2024-06-10T00:54:22.556768+0000","last_clean":"2024-06-10T00:54:22.556768+0000","last_became_active":"2024-06-10T00:52:32.763406+0000","last_became_peered":"2024-06-10T00:52:32.763406+0000","last_unstale":"2024-06-10T00:54:22.556768+0000","last_undegraded":"2024-06-10T00:54:22.556768+0000","last_fullsized":"2024-06-10T00:54:22.556768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:50:33.580163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"100'5","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374120+0000","last_change":"2024-06-10T00:53:22.130891+0000","last_active":"2024-06-10T00:55:20.374120+0000","last_peered":"2024-06-10T00:55:20.374120+0000","last_clean":"2024-06-10T00:55:20.374120+0000","last_became_active":"2024-06-10T00:53:22.130508+0000","last_became_peered":"2024-06-10T00:53:22.130508+0000","last_unstale":"2024-06-10T00:55:20.374120+0000","last_undegraded":"2024-06-10T00:55:20.374120+0000","last_fullsized":"2024-06-10T00:55:20.374120+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:28:01.919825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":34,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.563757+0000","last_change":"2024-06-10T00:54:22.563272+0000","last_active":"2024-06-10T00:54:22.563757+0000","last_peered":"2024-06-10T00:54:22.563757+0000","last_clean":"2024-06-10T00:54:22.563757+0000","last_became_active":"2024-06-10T00:54:22.562767+0000","last_became_peered":"2024-06-10T00:54:22.562767+0000","last_unstale":"2024-06-10T00:54:22.563757+0000","last_undegraded":"2024-06-10T00:54:22.563757+0000","last_fullsized":"2024-06-10T00:54:22.563757+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:52:20.739801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556364+0000","last_change":"2024-06-10T00:52:32.960515+0000","last_active":"2024-06-10T00:54:22.556364+0000","last_peered":"2024-06-10T00:54:22.556364+0000","last_clean":"2024-06-10T00:54:22.556364+0000","last_became_active":"2024-06-10T00:52:32.960163+0000","last_became_peered":"2024-06-10T00:52:32.960163+0000","last_unstale":"2024-06-10T00:54:22.556364+0000","last_undegraded":"2024-06-10T00:54:22.556364+0000","last_fullsized":"2024-06-10T00:54:22.556364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:39:51.374998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.359152+0000","last_change":"2024-06-10T00:52:35.961884+0000","last_active":"2024-06-10T00:55:20.359152+0000","last_peered":"2024-06-10T00:55:20.359152+0000","last_clean":"2024-06-10T00:55:20.359152+0000","last_became_active":"2024-06-10T00:52:35.961497+0000","last_became_peered":"2024-06-10T00:52:35.961497+0000","last_unstale":"2024-06-10T00:55:20.359152+0000","last_undegraded":"2024-06-10T00:55:20.359152+0000","last_fullsized":"2024-06-10T00:55:20.359152+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:16:09.284184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553591+0000","last_change":"2024-06-10T00:52:45.990846+0000","last_active":"2024-06-10T00:54:22.553591+0000","last_peered":"2024-06-10T00:54:22.553591+0000","last_clean":"2024-06-10T00:54:22.553591+0000","last_became_active":"2024-06-10T00:52:45.990441+0000","last_became_peered":"2024-06-10T00:52:45.990441+0000","last_unstale":"2024-06-10T00:54:22.553591+0000","last_undegraded":"2024-06-10T00:54:22.553591+0000","last_fullsized":"2024-06-10T00:54:22.553591+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:19:35.690378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404369+0000","last_change":"2024-06-10T00:52:32.753913+0000","last_active":"2024-06-10T00:54:24.404369+0000","last_peered":"2024-06-10T00:54:24.404369+0000","last_clean":"2024-06-10T00:54:24.404369+0000","last_became_active":"2024-06-10T00:52:32.753174+0000","last_became_peered":"2024-06-10T00:52:32.753174+0000","last_unstale":"2024-06-10T00:54:24.404369+0000","last_undegraded":"2024-06-10T00:54:24.404369+0000","last_fullsized":"2024-06-10T00:54:24.404369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:13:20.048635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.359023+0000","last_change":"2024-06-10T00:52:35.965783+0000","last_active":"2024-06-10T00:55:20.359023+0000","last_peered":"2024-06-10T00:55:20.359023+0000","last_clean":"2024-06-10T00:55:20.359023+0000","last_became_active":"2024-06-10T00:52:35.965432+0000","last_became_peered":"2024-06-10T00:52:35.965432+0000","last_unstale":"2024-06-10T00:55:20.359023+0000","last_undegraded":"2024-06-10T00:55:20.359023+0000","last_fullsized":"2024-06-10T00:55:20.359023+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:52:01.240318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553790+0000","last_change":"2024-06-10T00:52:46.241662+0000","last_active":"2024-06-10T00:54:22.553790+0000","last_peered":"2024-06-10T00:54:22.553790+0000","last_clean":"2024-06-10T00:54:22.553790+0000","last_became_active":"2024-06-10T00:52:46.241211+0000","last_became_peered":"2024-06-10T00:52:46.241211+0000","last_unstale":"2024-06-10T00:54:22.553790+0000","last_undegraded":"2024-06-10T00:54:22.553790+0000","last_fullsized":"2024-06-10T00:54:22.553790+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:12.041290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"100'8","reported_seq":24,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.367303+0000","last_change":"2024-06-10T00:53:22.200590+0000","last_active":"2024-06-10T00:55:20.367303+0000","last_peered":"2024-06-10T00:55:20.367303+0000","last_clean":"2024-06-10T00:55:20.367303+0000","last_became_active":"2024-06-10T00:53:22.200046+0000","last_became_peered":"2024-06-10T00:53:22.200046+0000","last_unstale":"2024-06-10T00:55:20.367303+0000","last_undegraded":"2024-06-10T00:55:20.367303+0000","last_fullsized":"2024-06-10T00:55:20.367303+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:43:22.378316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"98'4","reported_seq":220,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:31.559275+0000","last_change":"2024-06-10T00:52:32.959904+0000","last_active":"2024-06-10T00:56:31.559275+0000","last_peered":"2024-06-10T00:56:31.559275+0000","last_clean":"2024-06-10T00:56:31.559275+0000","last_became_active":"2024-06-10T00:52:32.959506+0000","last_became_peered":"2024-06-10T00:52:32.959506+0000","last_unstale":"2024-06-10T00:56:31.559275+0000","last_undegraded":"2024-06-10T00:56:31.559275+0000","last_fullsized":"2024-06-10T00:56:31.559275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:23:15.950345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":172,"num_read_kb":172,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404433+0000","last_change":"2024-06-10T00:52:45.993305+0000","last_active":"2024-06-10T00:54:24.404433+0000","last_peered":"2024-06-10T00:54:24.404433+0000","last_clean":"2024-06-10T00:54:24.404433+0000","last_became_active":"2024-06-10T00:52:45.992682+0000","last_became_peered":"2024-06-10T00:52:45.992682+0000","last_unstale":"2024-06-10T00:54:24.404433+0000","last_undegraded":"2024-06-10T00:54:24.404433+0000","last_fullsized":"2024-06-10T00:54:24.404433+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:49:41.817126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375771+0000","last_change":"2024-06-10T00:52:35.965893+0000","last_active":"2024-06-10T00:55:20.375771+0000","last_peered":"2024-06-10T00:55:20.375771+0000","last_clean":"2024-06-10T00:55:20.375771+0000","last_became_active":"2024-06-10T00:52:35.965426+0000","last_became_peered":"2024-06-10T00:52:35.965426+0000","last_unstale":"2024-06-10T00:55:20.375771+0000","last_undegraded":"2024-06-10T00:55:20.375771+0000","last_fullsized":"2024-06-10T00:55:20.375771+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:29:19.640836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668002+0000","last_change":"2024-06-10T00:52:32.753631+0000","last_active":"2024-06-10T00:54:22.668002+0000","last_peered":"2024-06-10T00:54:22.668002+0000","last_clean":"2024-06-10T00:54:22.668002+0000","last_became_active":"2024-06-10T00:52:32.753260+0000","last_became_peered":"2024-06-10T00:52:32.753260+0000","last_unstale":"2024-06-10T00:54:22.668002+0000","last_undegraded":"2024-06-10T00:54:22.668002+0000","last_fullsized":"2024-06-10T00:54:22.668002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:12:55.932676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":369,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404055+0000","last_change":"2024-06-10T00:51:09.267750+0000","last_active":"2024-06-10T00:54:24.404055+0000","last_peered":"2024-06-10T00:54:24.404055+0000","last_clean":"2024-06-10T00:54:24.404055+0000","last_became_active":"2024-06-10T00:51:09.267246+0000","last_became_peered":"2024-06-10T00:51:09.267246+0000","last_unstale":"2024-06-10T00:54:24.404055+0000","last_undegraded":"2024-06-10T00:54:24.404055+0000","last_fullsized":"2024-06-10T00:54:24.404055+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:47:44.965452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:47:44.965452+0000","last_clean_scrub_stamp":"2024-06-10T00:47:44.965452+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:21.100090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561655+0000","last_change":"2024-06-10T00:52:46.242152+0000","last_active":"2024-06-10T00:54:22.561655+0000","last_peered":"2024-06-10T00:54:22.561655+0000","last_clean":"2024-06-10T00:54:22.561655+0000","last_became_active":"2024-06-10T00:52:46.241542+0000","last_became_peered":"2024-06-10T00:52:46.241542+0000","last_unstale":"2024-06-10T00:54:22.561655+0000","last_undegraded":"2024-06-10T00:54:22.561655+0000","last_fullsized":"2024-06-10T00:54:22.561655+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:55:54.695465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364883+0000","last_change":"2024-06-10T00:52:35.963911+0000","last_active":"2024-06-10T00:55:20.364883+0000","last_peered":"2024-06-10T00:55:20.364883+0000","last_clean":"2024-06-10T00:55:20.364883+0000","last_became_active":"2024-06-10T00:52:35.963571+0000","last_became_peered":"2024-06-10T00:52:35.963571+0000","last_unstale":"2024-06-10T00:55:20.364883+0000","last_undegraded":"2024-06-10T00:55:20.364883+0000","last_fullsized":"2024-06-10T00:55:20.364883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:27:16.757428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.558529+0000","last_change":"2024-06-10T00:53:22.129035+0000","last_active":"2024-06-10T00:54:22.558529+0000","last_peered":"2024-06-10T00:54:22.558529+0000","last_clean":"2024-06-10T00:54:22.558529+0000","last_became_active":"2024-06-10T00:53:22.128561+0000","last_became_peered":"2024-06-10T00:53:22.128561+0000","last_unstale":"2024-06-10T00:54:22.558529+0000","last_undegraded":"2024-06-10T00:54:22.558529+0000","last_fullsized":"2024-06-10T00:54:22.558529+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:02:43.313118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.558735+0000","last_change":"2024-06-10T00:52:45.996907+0000","last_active":"2024-06-10T00:54:22.558735+0000","last_peered":"2024-06-10T00:54:22.558735+0000","last_clean":"2024-06-10T00:54:22.558735+0000","last_became_active":"2024-06-10T00:52:45.996554+0000","last_became_peered":"2024-06-10T00:52:45.996554+0000","last_unstale":"2024-06-10T00:54:22.558735+0000","last_undegraded":"2024-06-10T00:54:22.558735+0000","last_fullsized":"2024-06-10T00:54:22.558735+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:00:46.824071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556594+0000","last_change":"2024-06-10T00:52:32.960223+0000","last_active":"2024-06-10T00:54:22.556594+0000","last_peered":"2024-06-10T00:54:22.556594+0000","last_clean":"2024-06-10T00:54:22.556594+0000","last_became_active":"2024-06-10T00:52:32.959871+0000","last_became_peered":"2024-06-10T00:52:32.959871+0000","last_unstale":"2024-06-10T00:54:22.556594+0000","last_undegraded":"2024-06-10T00:54:22.556594+0000","last_fullsized":"2024-06-10T00:54:22.556594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:18:17.099140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.369555+0000","last_change":"2024-06-10T00:52:35.962516+0000","last_active":"2024-06-10T00:55:20.369555+0000","last_peered":"2024-06-10T00:55:20.369555+0000","last_clean":"2024-06-10T00:55:20.369555+0000","last_became_active":"2024-06-10T00:52:35.962187+0000","last_became_peered":"2024-06-10T00:52:35.962187+0000","last_unstale":"2024-06-10T00:55:20.369555+0000","last_undegraded":"2024-06-10T00:55:20.369555+0000","last_fullsized":"2024-06-10T00:55:20.369555+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:10:44.295192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601233+0000","last_change":"2024-06-10T00:52:46.241554+0000","last_active":"2024-06-10T00:54:22.601233+0000","last_peered":"2024-06-10T00:54:22.601233+0000","last_clean":"2024-06-10T00:54:22.601233+0000","last_became_active":"2024-06-10T00:52:46.241142+0000","last_became_peered":"2024-06-10T00:52:46.241142+0000","last_unstale":"2024-06-10T00:54:22.601233+0000","last_undegraded":"2024-06-10T00:54:22.601233+0000","last_fullsized":"2024-06-10T00:54:22.601233+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:58:07.233080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'13","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:29.987897+0000","last_change":"2024-06-10T00:52:32.761854+0000","last_active":"2024-06-10T00:56:29.987897+0000","last_peered":"2024-06-10T00:56:29.987897+0000","last_clean":"2024-06-10T00:56:29.987897+0000","last_became_active":"2024-06-10T00:52:32.761522+0000","last_became_peered":"2024-06-10T00:52:32.761522+0000","last_unstale":"2024-06-10T00:56:29.987897+0000","last_undegraded":"2024-06-10T00:56:29.987897+0000","last_fullsized":"2024-06-10T00:56:29.987897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:06:43.269001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.370762+0000","last_change":"2024-06-10T00:52:35.959250+0000","last_active":"2024-06-10T00:55:20.370762+0000","last_peered":"2024-06-10T00:55:20.370762+0000","last_clean":"2024-06-10T00:55:20.370762+0000","last_became_active":"2024-06-10T00:52:35.958925+0000","last_became_peered":"2024-06-10T00:52:35.958925+0000","last_unstale":"2024-06-10T00:55:20.370762+0000","last_undegraded":"2024-06-10T00:55:20.370762+0000","last_fullsized":"2024-06-10T00:55:20.370762+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:11:12.355505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554191+0000","last_change":"2024-06-10T00:52:45.991819+0000","last_active":"2024-06-10T00:54:22.554191+0000","last_peered":"2024-06-10T00:54:22.554191+0000","last_clean":"2024-06-10T00:54:22.554191+0000","last_became_active":"2024-06-10T00:52:45.991408+0000","last_became_peered":"2024-06-10T00:52:45.991408+0000","last_unstale":"2024-06-10T00:54:22.554191+0000","last_undegraded":"2024-06-10T00:54:22.554191+0000","last_fullsized":"2024-06-10T00:54:22.554191+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:16:34.912359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668551+0000","last_change":"2024-06-10T00:52:32.869094+0000","last_active":"2024-06-10T00:54:22.668551+0000","last_peered":"2024-06-10T00:54:22.668551+0000","last_clean":"2024-06-10T00:54:22.668551+0000","last_became_active":"2024-06-10T00:52:32.868664+0000","last_became_peered":"2024-06-10T00:52:32.868664+0000","last_unstale":"2024-06-10T00:54:22.668551+0000","last_undegraded":"2024-06-10T00:54:22.668551+0000","last_fullsized":"2024-06-10T00:54:22.668551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:50:38.051090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374722+0000","last_change":"2024-06-10T00:52:35.952356+0000","last_active":"2024-06-10T00:55:20.374722+0000","last_peered":"2024-06-10T00:55:20.374722+0000","last_clean":"2024-06-10T00:55:20.374722+0000","last_became_active":"2024-06-10T00:52:35.952002+0000","last_became_peered":"2024-06-10T00:52:35.952002+0000","last_unstale":"2024-06-10T00:55:20.374722+0000","last_undegraded":"2024-06-10T00:55:20.374722+0000","last_fullsized":"2024-06-10T00:55:20.374722+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:12:28.581522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668366+0000","last_change":"2024-06-10T00:52:46.241773+0000","last_active":"2024-06-10T00:54:22.668366+0000","last_peered":"2024-06-10T00:54:22.668366+0000","last_clean":"2024-06-10T00:54:22.668366+0000","last_became_active":"2024-06-10T00:52:46.241272+0000","last_became_peered":"2024-06-10T00:52:46.241272+0000","last_unstale":"2024-06-10T00:54:22.668366+0000","last_undegraded":"2024-06-10T00:54:22.668366+0000","last_fullsized":"2024-06-10T00:54:22.668366+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:07:28.337536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561360+0000","last_change":"2024-06-10T00:52:32.960100+0000","last_active":"2024-06-10T00:54:22.561360+0000","last_peered":"2024-06-10T00:54:22.561360+0000","last_clean":"2024-06-10T00:54:22.561360+0000","last_became_active":"2024-06-10T00:52:32.959779+0000","last_became_peered":"2024-06-10T00:52:32.959779+0000","last_unstale":"2024-06-10T00:54:22.561360+0000","last_undegraded":"2024-06-10T00:54:22.561360+0000","last_fullsized":"2024-06-10T00:54:22.561360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:09.555753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'2","reported_seq":40,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.326494+0000","last_change":"2024-06-10T00:52:35.955586+0000","last_active":"2024-06-10T00:55:20.326494+0000","last_peered":"2024-06-10T00:55:20.326494+0000","last_clean":"2024-06-10T00:55:20.326494+0000","last_became_active":"2024-06-10T00:52:35.955216+0000","last_became_peered":"2024-06-10T00:52:35.955216+0000","last_unstale":"2024-06-10T00:55:20.326494+0000","last_undegraded":"2024-06-10T00:55:20.326494+0000","last_fullsized":"2024-06-10T00:55:20.326494+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:23:12.197182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.603221+0000","last_change":"2024-06-10T00:54:22.602670+0000","last_active":"2024-06-10T00:54:22.603221+0000","last_peered":"2024-06-10T00:54:22.603221+0000","last_clean":"2024-06-10T00:54:22.603221+0000","last_became_active":"2024-06-10T00:54:22.602218+0000","last_became_peered":"2024-06-10T00:54:22.602218+0000","last_unstale":"2024-06-10T00:54:22.603221+0000","last_undegraded":"2024-06-10T00:54:22.603221+0000","last_fullsized":"2024-06-10T00:54:22.603221+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:07.399025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"98'3","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601358+0000","last_change":"2024-06-10T00:52:32.753444+0000","last_active":"2024-06-10T00:54:22.601358+0000","last_peered":"2024-06-10T00:54:22.601358+0000","last_clean":"2024-06-10T00:54:22.601358+0000","last_became_active":"2024-06-10T00:52:32.753149+0000","last_became_peered":"2024-06-10T00:52:32.753149+0000","last_unstale":"2024-06-10T00:54:22.601358+0000","last_undegraded":"2024-06-10T00:54:22.601358+0000","last_fullsized":"2024-06-10T00:54:22.601358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:28.241950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365900+0000","last_change":"2024-06-10T00:52:35.966718+0000","last_active":"2024-06-10T00:55:20.365900+0000","last_peered":"2024-06-10T00:55:20.365900+0000","last_clean":"2024-06-10T00:55:20.365900+0000","last_became_active":"2024-06-10T00:52:35.965907+0000","last_became_peered":"2024-06-10T00:52:35.965907+0000","last_unstale":"2024-06-10T00:55:20.365900+0000","last_undegraded":"2024-06-10T00:55:20.365900+0000","last_fullsized":"2024-06-10T00:55:20.365900+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:21:34.261306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668092+0000","last_change":"2024-06-10T00:52:45.991034+0000","last_active":"2024-06-10T00:54:22.668092+0000","last_peered":"2024-06-10T00:54:22.668092+0000","last_clean":"2024-06-10T00:54:22.668092+0000","last_became_active":"2024-06-10T00:52:45.990546+0000","last_became_peered":"2024-06-10T00:52:45.990546+0000","last_unstale":"2024-06-10T00:54:22.668092+0000","last_undegraded":"2024-06-10T00:54:22.668092+0000","last_fullsized":"2024-06-10T00:54:22.668092+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:09:23.412922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"100'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:42.999786+0000","last_change":"2024-06-10T00:52:32.764024+0000","last_active":"2024-06-10T00:54:42.999786+0000","last_peered":"2024-06-10T00:54:42.999786+0000","last_clean":"2024-06-10T00:54:42.999786+0000","last_became_active":"2024-06-10T00:52:32.763449+0000","last_became_peered":"2024-06-10T00:52:32.763449+0000","last_unstale":"2024-06-10T00:54:42.999786+0000","last_undegraded":"2024-06-10T00:54:42.999786+0000","last_fullsized":"2024-06-10T00:54:42.999786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:50:23.605873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373684+0000","last_change":"2024-06-10T00:52:35.955570+0000","last_active":"2024-06-10T00:55:20.373684+0000","last_peered":"2024-06-10T00:55:20.373684+0000","last_clean":"2024-06-10T00:55:20.373684+0000","last_became_active":"2024-06-10T00:52:35.955248+0000","last_became_peered":"2024-06-10T00:52:35.955248+0000","last_unstale":"2024-06-10T00:55:20.373684+0000","last_undegraded":"2024-06-10T00:55:20.373684+0000","last_fullsized":"2024-06-10T00:55:20.373684+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:26.124499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668028+0000","last_change":"2024-06-10T00:52:46.241776+0000","last_active":"2024-06-10T00:54:22.668028+0000","last_peered":"2024-06-10T00:54:22.668028+0000","last_clean":"2024-06-10T00:54:22.668028+0000","last_became_active":"2024-06-10T00:52:46.241276+0000","last_became_peered":"2024-06-10T00:52:46.241276+0000","last_unstale":"2024-06-10T00:54:22.668028+0000","last_undegraded":"2024-06-10T00:54:22.668028+0000","last_fullsized":"2024-06-10T00:54:22.668028+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:08.027938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":58,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556321+0000","last_change":"2024-06-10T00:53:22.204034+0000","last_active":"2024-06-10T00:54:22.556321+0000","last_peered":"2024-06-10T00:54:22.556321+0000","last_clean":"2024-06-10T00:54:22.556321+0000","last_became_active":"2024-06-10T00:53:22.199988+0000","last_became_peered":"2024-06-10T00:53:22.199988+0000","last_unstale":"2024-06-10T00:54:22.556321+0000","last_undegraded":"2024-06-10T00:54:22.556321+0000","last_fullsized":"2024-06-10T00:54:22.556321+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:39:54.012360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373708+0000","last_change":"2024-06-10T00:52:35.959182+0000","last_active":"2024-06-10T00:55:20.373708+0000","last_peered":"2024-06-10T00:55:20.373708+0000","last_clean":"2024-06-10T00:55:20.373708+0000","last_became_active":"2024-06-10T00:52:35.958828+0000","last_became_peered":"2024-06-10T00:52:35.958828+0000","last_unstale":"2024-06-10T00:55:20.373708+0000","last_undegraded":"2024-06-10T00:55:20.373708+0000","last_fullsized":"2024-06-10T00:55:20.373708+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:29:58.857659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561603+0000","last_change":"2024-06-10T00:52:45.989739+0000","last_active":"2024-06-10T00:54:22.561603+0000","last_peered":"2024-06-10T00:54:22.561603+0000","last_clean":"2024-06-10T00:54:22.561603+0000","last_became_active":"2024-06-10T00:52:45.989279+0000","last_became_peered":"2024-06-10T00:52:45.989279+0000","last_unstale":"2024-06-10T00:54:22.561603+0000","last_undegraded":"2024-06-10T00:54:22.561603+0000","last_fullsized":"2024-06-10T00:54:22.561603+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:19:17.551947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554798+0000","last_change":"2024-06-10T00:52:32.959662+0000","last_active":"2024-06-10T00:54:22.554798+0000","last_peered":"2024-06-10T00:54:22.554798+0000","last_clean":"2024-06-10T00:54:22.554798+0000","last_became_active":"2024-06-10T00:52:32.959286+0000","last_became_peered":"2024-06-10T00:52:32.959286+0000","last_unstale":"2024-06-10T00:54:22.554798+0000","last_undegraded":"2024-06-10T00:54:22.554798+0000","last_fullsized":"2024-06-10T00:54:22.554798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:26:04.361567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373843+0000","last_change":"2024-06-10T00:52:35.960137+0000","last_active":"2024-06-10T00:55:20.373843+0000","last_peered":"2024-06-10T00:55:20.373843+0000","last_clean":"2024-06-10T00:55:20.373843+0000","last_became_active":"2024-06-10T00:52:35.959584+0000","last_became_peered":"2024-06-10T00:52:35.959584+0000","last_unstale":"2024-06-10T00:55:20.373843+0000","last_undegraded":"2024-06-10T00:55:20.373843+0000","last_fullsized":"2024-06-10T00:55:20.373843+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:23:38.702685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878137+0000","last_change":"2024-06-10T00:52:46.002268+0000","last_active":"2024-06-10T00:54:22.878137+0000","last_peered":"2024-06-10T00:54:22.878137+0000","last_clean":"2024-06-10T00:54:22.878137+0000","last_became_active":"2024-06-10T00:52:46.001769+0000","last_became_peered":"2024-06-10T00:52:46.001769+0000","last_unstale":"2024-06-10T00:54:22.878137+0000","last_undegraded":"2024-06-10T00:54:22.878137+0000","last_fullsized":"2024-06-10T00:54:22.878137+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:33:51.365951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561629+0000","last_change":"2024-06-10T00:52:32.757112+0000","last_active":"2024-06-10T00:54:22.561629+0000","last_peered":"2024-06-10T00:54:22.561629+0000","last_clean":"2024-06-10T00:54:22.561629+0000","last_became_active":"2024-06-10T00:52:32.755589+0000","last_became_peered":"2024-06-10T00:52:32.755589+0000","last_unstale":"2024-06-10T00:54:22.561629+0000","last_undegraded":"2024-06-10T00:54:22.561629+0000","last_fullsized":"2024-06-10T00:54:22.561629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:15:12.814489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364097+0000","last_change":"2024-06-10T00:52:35.955925+0000","last_active":"2024-06-10T00:55:20.364097+0000","last_peered":"2024-06-10T00:55:20.364097+0000","last_clean":"2024-06-10T00:55:20.364097+0000","last_became_active":"2024-06-10T00:52:35.955556+0000","last_became_peered":"2024-06-10T00:52:35.955556+0000","last_unstale":"2024-06-10T00:55:20.364097+0000","last_undegraded":"2024-06-10T00:55:20.364097+0000","last_fullsized":"2024-06-10T00:55:20.364097+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:42:18.324484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561041+0000","last_change":"2024-06-10T00:52:45.988810+0000","last_active":"2024-06-10T00:54:22.561041+0000","last_peered":"2024-06-10T00:54:22.561041+0000","last_clean":"2024-06-10T00:54:22.561041+0000","last_became_active":"2024-06-10T00:52:45.988041+0000","last_became_peered":"2024-06-10T00:52:45.988041+0000","last_unstale":"2024-06-10T00:54:22.561041+0000","last_undegraded":"2024-06-10T00:54:22.561041+0000","last_fullsized":"2024-06-10T00:54:22.561041+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:14:08.979802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553851+0000","last_change":"2024-06-10T00:52:32.765493+0000","last_active":"2024-06-10T00:54:22.553851+0000","last_peered":"2024-06-10T00:54:22.553851+0000","last_clean":"2024-06-10T00:54:22.553851+0000","last_became_active":"2024-06-10T00:52:32.765193+0000","last_became_peered":"2024-06-10T00:52:32.765193+0000","last_unstale":"2024-06-10T00:54:22.553851+0000","last_undegraded":"2024-06-10T00:54:22.553851+0000","last_fullsized":"2024-06-10T00:54:22.553851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:38:11.554673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374319+0000","last_change":"2024-06-10T00:52:35.960216+0000","last_active":"2024-06-10T00:55:20.374319+0000","last_peered":"2024-06-10T00:55:20.374319+0000","last_clean":"2024-06-10T00:55:20.374319+0000","last_became_active":"2024-06-10T00:52:35.959664+0000","last_became_peered":"2024-06-10T00:52:35.959664+0000","last_unstale":"2024-06-10T00:55:20.374319+0000","last_undegraded":"2024-06-10T00:55:20.374319+0000","last_fullsized":"2024-06-10T00:55:20.374319+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:41:58.086330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375612+0000","last_change":"2024-06-10T00:52:35.967043+0000","last_active":"2024-06-10T00:55:20.375612+0000","last_peered":"2024-06-10T00:55:20.375612+0000","last_clean":"2024-06-10T00:55:20.375612+0000","last_became_active":"2024-06-10T00:52:35.966366+0000","last_became_peered":"2024-06-10T00:52:35.966366+0000","last_unstale":"2024-06-10T00:55:20.375612+0000","last_undegraded":"2024-06-10T00:55:20.375612+0000","last_fullsized":"2024-06-10T00:55:20.375612+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:27:06.047113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"100'12","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.369745+0000","last_change":"2024-06-10T00:52:35.960517+0000","last_active":"2024-06-10T00:55:20.369745+0000","last_peered":"2024-06-10T00:55:20.369745+0000","last_clean":"2024-06-10T00:55:20.369745+0000","last_became_active":"2024-06-10T00:52:35.957895+0000","last_became_peered":"2024-06-10T00:52:35.957895+0000","last_unstale":"2024-06-10T00:55:20.369745+0000","last_undegraded":"2024-06-10T00:55:20.369745+0000","last_fullsized":"2024-06-10T00:55:20.369745+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:04:34.438070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.369777+0000","last_change":"2024-06-10T00:52:35.962163+0000","last_active":"2024-06-10T00:55:20.369777+0000","last_peered":"2024-06-10T00:55:20.369777+0000","last_clean":"2024-06-10T00:55:20.369777+0000","last_became_active":"2024-06-10T00:52:35.961768+0000","last_became_peered":"2024-06-10T00:52:35.961768+0000","last_unstale":"2024-06-10T00:55:20.369777+0000","last_undegraded":"2024-06-10T00:55:20.369777+0000","last_fullsized":"2024-06-10T00:55:20.369777+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:35.793995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"100'7","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373779+0000","last_change":"2024-06-10T00:53:22.130686+0000","last_active":"2024-06-10T00:55:20.373779+0000","last_peered":"2024-06-10T00:55:20.373779+0000","last_clean":"2024-06-10T00:55:20.373779+0000","last_became_active":"2024-06-10T00:53:22.130198+0000","last_became_peered":"2024-06-10T00:53:22.130198+0000","last_unstale":"2024-06-10T00:55:20.373779+0000","last_undegraded":"2024-06-10T00:55:20.373779+0000","last_fullsized":"2024-06-10T00:55:20.373779+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:50:26.773977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372642+0000","last_change":"2024-06-10T00:52:35.957255+0000","last_active":"2024-06-10T00:55:20.372642+0000","last_peered":"2024-06-10T00:55:20.372642+0000","last_clean":"2024-06-10T00:55:20.372642+0000","last_became_active":"2024-06-10T00:52:35.956923+0000","last_became_peered":"2024-06-10T00:52:35.956923+0000","last_unstale":"2024-06-10T00:55:20.372642+0000","last_undegraded":"2024-06-10T00:55:20.372642+0000","last_fullsized":"2024-06-10T00:55:20.372642+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:33:25.743503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375063+0000","last_change":"2024-06-10T00:52:35.964258+0000","last_active":"2024-06-10T00:55:20.375063+0000","last_peered":"2024-06-10T00:55:20.375063+0000","last_clean":"2024-06-10T00:55:20.375063+0000","last_became_active":"2024-06-10T00:52:35.963862+0000","last_became_peered":"2024-06-10T00:52:35.963862+0000","last_unstale":"2024-06-10T00:55:20.375063+0000","last_undegraded":"2024-06-10T00:55:20.375063+0000","last_fullsized":"2024-06-10T00:55:20.375063+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:48:58.554411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.377085+0000","last_change":"2024-06-10T00:52:35.965046+0000","last_active":"2024-06-10T00:55:20.377085+0000","last_peered":"2024-06-10T00:55:20.377085+0000","last_clean":"2024-06-10T00:55:20.377085+0000","last_became_active":"2024-06-10T00:52:35.964496+0000","last_became_peered":"2024-06-10T00:52:35.964496+0000","last_unstale":"2024-06-10T00:55:20.377085+0000","last_undegraded":"2024-06-10T00:55:20.377085+0000","last_fullsized":"2024-06-10T00:55:20.377085+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:09:42.979731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"100'2","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.370981+0000","last_change":"2024-06-10T00:52:35.964287+0000","last_active":"2024-06-10T00:55:20.370981+0000","last_peered":"2024-06-10T00:55:20.370981+0000","last_clean":"2024-06-10T00:55:20.370981+0000","last_became_active":"2024-06-10T00:52:35.963920+0000","last_became_peered":"2024-06-10T00:52:35.963920+0000","last_unstale":"2024-06-10T00:55:20.370981+0000","last_undegraded":"2024-06-10T00:55:20.370981+0000","last_fullsized":"2024-06-10T00:55:20.370981+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:41:54.593274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.367606+0000","last_change":"2024-06-10T00:52:35.961440+0000","last_active":"2024-06-10T00:55:20.367606+0000","last_peered":"2024-06-10T00:55:20.367606+0000","last_clean":"2024-06-10T00:55:20.367606+0000","last_became_active":"2024-06-10T00:52:35.960066+0000","last_became_peered":"2024-06-10T00:52:35.960066+0000","last_unstale":"2024-06-10T00:55:20.367606+0000","last_undegraded":"2024-06-10T00:55:20.367606+0000","last_fullsized":"2024-06-10T00:55:20.367606+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:01.024596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372367+0000","last_change":"2024-06-10T00:52:35.964464+0000","last_active":"2024-06-10T00:55:20.372367+0000","last_peered":"2024-06-10T00:55:20.372367+0000","last_clean":"2024-06-10T00:55:20.372367+0000","last_became_active":"2024-06-10T00:52:35.964159+0000","last_became_peered":"2024-06-10T00:52:35.964159+0000","last_unstale":"2024-06-10T00:55:20.372367+0000","last_undegraded":"2024-06-10T00:55:20.372367+0000","last_fullsized":"2024-06-10T00:55:20.372367+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:18.660859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.377213+0000","last_change":"2024-06-10T00:52:35.961539+0000","last_active":"2024-06-10T00:55:20.377213+0000","last_peered":"2024-06-10T00:55:20.377213+0000","last_clean":"2024-06-10T00:55:20.377213+0000","last_became_active":"2024-06-10T00:52:35.958738+0000","last_became_peered":"2024-06-10T00:52:35.958738+0000","last_unstale":"2024-06-10T00:55:20.377213+0000","last_undegraded":"2024-06-10T00:55:20.377213+0000","last_fullsized":"2024-06-10T00:55:20.377213+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:06:48.057551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374134+0000","last_change":"2024-06-10T00:52:35.962563+0000","last_active":"2024-06-10T00:55:20.374134+0000","last_peered":"2024-06-10T00:55:20.374134+0000","last_clean":"2024-06-10T00:55:20.374134+0000","last_became_active":"2024-06-10T00:52:35.962255+0000","last_became_peered":"2024-06-10T00:52:35.962255+0000","last_unstale":"2024-06-10T00:55:20.374134+0000","last_undegraded":"2024-06-10T00:55:20.374134+0000","last_fullsized":"2024-06-10T00:55:20.374134+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:33:35.689446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372437+0000","last_change":"2024-06-10T00:52:35.959616+0000","last_active":"2024-06-10T00:55:20.372437+0000","last_peered":"2024-06-10T00:55:20.372437+0000","last_clean":"2024-06-10T00:55:20.372437+0000","last_became_active":"2024-06-10T00:52:35.959251+0000","last_became_peered":"2024-06-10T00:52:35.959251+0000","last_unstale":"2024-06-10T00:55:20.372437+0000","last_undegraded":"2024-06-10T00:55:20.372437+0000","last_fullsized":"2024-06-10T00:55:20.372437+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:39:23.933279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284806,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":82322,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1075,"num_read_kb":628,"num_write":191,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":241926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":163,"ondisk_log_size":163,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351040,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":573101,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612678,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28248,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716904,"statfs":{"total":95995035648,"available":95966109696,"internally_reserved":0,"allocated":1261568,"data_stored":1238242,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776203,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":577433,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037839,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":573257,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299477,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36476,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708676,"statfs":{"total":95995035648,"available":95957684224,"internally_reserved":0,"allocated":1167360,"data_stored":1169614,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463001,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":568513,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36488,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708664,"statfs":{"total":95995035648,"available":95957671936,"internally_reserved":0,"allocated":1179648,"data_stored":643776,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738488,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":569901,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411438,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32344,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712808,"statfs":{"total":95995035648,"available":95961915392,"internally_reserved":0,"allocated":1196032,"data_stored":645853,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149800,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":568707,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888163,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":573275,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724638,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":1165202,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":75353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":77430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":79451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T00:56:37.313 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:56:37.596 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:56:38.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:38 smithi167 ceph-mon[36917]: pgmap v428: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-06-10T00:56:38.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:38 smithi167 ceph-mon[36917]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:56:38.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:38 smithi167 ceph-mon[36917]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:56:38.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:38 smithi094 ceph-mon[36708]: pgmap v428: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-06-10T00:56:38.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:38 smithi094 ceph-mon[36708]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:56:38.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:38 smithi094 ceph-mon[36708]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:56:38.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:38 smithi037 ceph-mon[32192]: pgmap v428: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-06-10T00:56:38.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:38 smithi037 ceph-mon[32192]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:56:38.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:38 smithi037 ceph-mon[32192]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:56:39.223 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:39 smithi037 ceph-mon[32192]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:56:39.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:39 smithi167 ceph-mon[36917]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:56:39.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.332+0000 7f758b5e3700 1 -- 172.21.15.37:0/457835067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75841030f0 msgr2=0x7f7584107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:39.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.332+0000 7f758b5e3700 1 --2- 172.21.15.37:0/457835067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75841030f0 0x7f7584107110 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f75740099c0 tx=0x7f7574009cd0 comp rx=0 tx=0).stop 2024-06-10T00:56:39.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.332+0000 7f758b5e3700 1 -- 172.21.15.37:0/457835067 shutdown_connections 2024-06-10T00:56:39.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.332+0000 7f758b5e3700 1 --2- 172.21.15.37:0/457835067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7584107840 0x7f758410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:39.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.332+0000 7f758b5e3700 1 --2- 172.21.15.37:0/457835067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7584102740 0x7f7584102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:39.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.332+0000 7f758b5e3700 1 --2- 172.21.15.37:0/457835067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75841030f0 0x7f7584107110 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:39.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.332+0000 7f758b5e3700 1 -- 172.21.15.37:0/457835067 >> 172.21.15.37:0/457835067 conn(0x7f75840795f0 msgr2=0x7f7584079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:39.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.332+0000 7f758b5e3700 1 -- 172.21.15.37:0/457835067 shutdown_connections 2024-06-10T00:56:39.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.332+0000 7f758b5e3700 1 -- 172.21.15.37:0/457835067 wait complete. 2024-06-10T00:56:39.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.333+0000 7f758b5e3700 1 Processor -- start 2024-06-10T00:56:39.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.333+0000 7f758b5e3700 1 -- start start 2024-06-10T00:56:39.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.333+0000 7f758b5e3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7584102740 0x7f758419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:39.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.333+0000 7f758b5e3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75841030f0 0x7f758419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:39.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.333+0000 7f758b5e3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7584107840 0x7f7584198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:39.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.333+0000 7f758b5e3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f75841127d0 con 0x7f7584107840 2024-06-10T00:56:39.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.333+0000 7f758b5e3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7584112650 con 0x7f75841030f0 2024-06-10T00:56:39.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.333+0000 7f758b5e3700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7584112950 con 0x7f7584102740 2024-06-10T00:56:39.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.333+0000 7f7589b80700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7584107840 0x7f7584198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:39.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.333+0000 7f7589b80700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7584107840 0x7f7584198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47190/0 (socket says 172.21.15.37:47190) 2024-06-10T00:56:39.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.333+0000 7f7589b80700 1 -- 172.21.15.37:0/3295728156 learned_addr learned my addr 172.21.15.37:0/3295728156 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:39.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.333+0000 7f758937f700 1 --2- 172.21.15.37:0/3295728156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7584102740 0x7f758419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:39.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.333+0000 7f7588b7e700 1 --2- 172.21.15.37:0/3295728156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75841030f0 0x7f758419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:39.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.333+0000 7f7589b80700 1 -- 172.21.15.37:0/3295728156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7584102740 msgr2=0x7f758419e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:39.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.333+0000 7f7589b80700 1 --2- 172.21.15.37:0/3295728156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7584102740 0x7f758419e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:39.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.334+0000 7f7589b80700 1 -- 172.21.15.37:0/3295728156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75841030f0 msgr2=0x7f758419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:39.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.334+0000 7f7589b80700 1 --2- 172.21.15.37:0/3295728156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75841030f0 0x7f758419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:39.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.334+0000 7f7589b80700 1 -- 172.21.15.37:0/3295728156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75841991b0 con 0x7f7584107840 2024-06-10T00:56:39.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.334+0000 7f7588b7e700 1 --2- 172.21.15.37:0/3295728156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75841030f0 0x7f758419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:56:39.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.334+0000 7f7589b80700 1 --2- 172.21.15.37:0/3295728156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7584107840 0x7f7584198a50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f757c00b970 tx=0x7f757c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:39.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.334+0000 7f757a7fc700 1 -- 172.21.15.37:0/3295728156 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c019070 con 0x7f7584107840 2024-06-10T00:56:39.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.334+0000 7f758b5e3700 1 -- 172.21.15.37:0/3295728156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75841994a0 con 0x7f7584107840 2024-06-10T00:56:39.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.334+0000 7f757a7fc700 1 -- 172.21.15.37:0/3295728156 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f757c00ce10 con 0x7f7584107840 2024-06-10T00:56:39.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.334+0000 7f757a7fc700 1 -- 172.21.15.37:0/3295728156 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c007370 con 0x7f7584107840 2024-06-10T00:56:39.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.334+0000 7f758b5e3700 1 -- 172.21.15.37:0/3295728156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75841a5710 con 0x7f7584107840 2024-06-10T00:56:39.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.335+0000 7f758b5e3700 1 -- 172.21.15.37:0/3295728156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7584069180 con 0x7f7584107840 2024-06-10T00:56:39.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.336+0000 7f757a7fc700 1 -- 172.21.15.37:0/3295728156 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f757c01c070 con 0x7f7584107840 2024-06-10T00:56:39.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.337+0000 7f757a7fc700 1 --2- 172.21.15.37:0/3295728156 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f75700a2cb0 0x7f75700a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:39.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.337+0000 7f758937f700 1 --2- 172.21.15.37:0/3295728156 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f75700a2cb0 0x7f75700a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:39.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.337+0000 7f757a7fc700 1 -- 172.21.15.37:0/3295728156 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f757c0ced90 con 0x7f7584107840 2024-06-10T00:56:39.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.338+0000 7f758937f700 1 --2- 172.21.15.37:0/3295728156 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f75700a2cb0 0x7f75700a5170 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f75841073f0 tx=0x7f758000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:39.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.341+0000 7f757a7fc700 1 -- 172.21.15.37:0/3295728156 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f757c09b850 con 0x7f7584107840 2024-06-10T00:56:39.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:39 smithi094 ceph-mon[36708]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:56:39.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.601+0000 7f758b5e3700 1 -- 172.21.15.37:0/3295728156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7584199f30 con 0x7f7584107840 2024-06-10T00:56:39.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.602+0000 7f757a7fc700 1 -- 172.21.15.37:0/3295728156 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11867 (secure 0 0 0) 0x7f757c09b150 con 0x7f7584107840 2024-06-10T00:56:39.602 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:56:39.602 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T00:56:30.291171+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24451,24458,24472]},"id":1}]} 2024-06-10T00:56:39.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.604+0000 7f758b5e3700 1 -- 172.21.15.37:0/3295728156 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f75700a2cb0 msgr2=0x7f75700a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:39.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.604+0000 7f758b5e3700 1 --2- 172.21.15.37:0/3295728156 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f75700a2cb0 0x7f75700a5170 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f75841073f0 tx=0x7f758000b410 comp rx=0 tx=0).stop 2024-06-10T00:56:39.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.604+0000 7f758b5e3700 1 -- 172.21.15.37:0/3295728156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7584107840 msgr2=0x7f7584198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:39.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.604+0000 7f758b5e3700 1 --2- 172.21.15.37:0/3295728156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7584107840 0x7f7584198a50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f757c00b970 tx=0x7f757c00bc80 comp rx=0 tx=0).stop 2024-06-10T00:56:39.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.604+0000 7f758b5e3700 1 -- 172.21.15.37:0/3295728156 shutdown_connections 2024-06-10T00:56:39.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.604+0000 7f758b5e3700 1 --2- 172.21.15.37:0/3295728156 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f75700a2cb0 0x7f75700a5170 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:39.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.604+0000 7f758b5e3700 1 --2- 172.21.15.37:0/3295728156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7584107840 0x7f7584198a50 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:39.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.604+0000 7f758b5e3700 1 --2- 172.21.15.37:0/3295728156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7584102740 0x7f758419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:39.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.604+0000 7f758b5e3700 1 --2- 172.21.15.37:0/3295728156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75841030f0 0x7f758419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:39.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.604+0000 7f758b5e3700 1 -- 172.21.15.37:0/3295728156 >> 172.21.15.37:0/3295728156 conn(0x7f75840795f0 msgr2=0x7f7584101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:39.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.604+0000 7f758b5e3700 1 -- 172.21.15.37:0/3295728156 shutdown_connections 2024-06-10T00:56:39.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:39.604+0000 7f758b5e3700 1 -- 172.21.15.37:0/3295728156 wait complete. 2024-06-10T00:56:39.605 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-06-10T00:56:40.203 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:40 smithi037 ceph-mon[32192]: pgmap v429: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-06-10T00:56:40.203 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:40 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3295728156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:56:40.239 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T00:56:40.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:40 smithi167 ceph-mon[36917]: pgmap v429: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-06-10T00:56:40.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:40 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3295728156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:56:40.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:40 smithi094 ceph-mon[36708]: pgmap v429: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-06-10T00:56:40.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:40 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3295728156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:56:40.496 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:56:42.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.192+0000 7f7269360700 1 -- 172.21.15.37:0/2352337880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7264108580 msgr2=0x7f7264108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:42.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.192+0000 7f7269360700 1 --2- 172.21.15.37:0/2352337880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7264108580 0x7f7264108960 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f72540099c0 tx=0x7f7254009cd0 comp rx=0 tx=0).stop 2024-06-10T00:56:42.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.192+0000 7f7269360700 1 -- 172.21.15.37:0/2352337880 shutdown_connections 2024-06-10T00:56:42.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.192+0000 7f7269360700 1 --2- 172.21.15.37:0/2352337880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f726406b620 0x7f726410f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:42.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.192+0000 7f7269360700 1 --2- 172.21.15.37:0/2352337880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7264108580 0x7f7264108960 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:42.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.192+0000 7f7269360700 1 --2- 172.21.15.37:0/2352337880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f726406ac60 0x7f726406b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:42.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.192+0000 7f7269360700 1 -- 172.21.15.37:0/2352337880 >> 172.21.15.37:0/2352337880 conn(0x7f7264078430 msgr2=0x7f7264078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:42.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.192+0000 7f7269360700 1 -- 172.21.15.37:0/2352337880 shutdown_connections 2024-06-10T00:56:42.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.192+0000 7f7269360700 1 -- 172.21.15.37:0/2352337880 wait complete. 2024-06-10T00:56:42.194 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:42 smithi037 ceph-mon[32192]: pgmap v430: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-06-10T00:56:42.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.193+0000 7f7269360700 1 Processor -- start 2024-06-10T00:56:42.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.193+0000 7f7269360700 1 -- start start 2024-06-10T00:56:42.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.193+0000 7f7269360700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f726406ac60 0x7f726419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:42.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.193+0000 7f7269360700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f726406b620 0x7f726419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:42.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.193+0000 7f7269360700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7264108580 0x7f726419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:42.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.193+0000 7f7269360700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7264112840 con 0x7f726406b620 2024-06-10T00:56:42.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.193+0000 7f7269360700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f72641126c0 con 0x7f726406ac60 2024-06-10T00:56:42.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.193+0000 7f7269360700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f72641129c0 con 0x7f7264108580 2024-06-10T00:56:42.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.193+0000 7f72627fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f726406b620 0x7f726419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:42.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.194+0000 7f7262ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f726406ac60 0x7f726419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:42.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.194+0000 7f72627fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f726406b620 0x7f726419b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59366/0 (socket says 172.21.15.37:59366) 2024-06-10T00:56:42.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.194+0000 7f72627fc700 1 -- 172.21.15.37:0/896194040 learned_addr learned my addr 172.21.15.37:0/896194040 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:42.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.194+0000 7f72627fc700 1 -- 172.21.15.37:0/896194040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7264108580 msgr2=0x7f726419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:42.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.194+0000 7f72637fe700 1 --2- 172.21.15.37:0/896194040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7264108580 0x7f726419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:42.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.194+0000 7f72627fc700 1 --2- 172.21.15.37:0/896194040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7264108580 0x7f726419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:42.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.194+0000 7f72627fc700 1 -- 172.21.15.37:0/896194040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f726406ac60 msgr2=0x7f726419ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:42.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.194+0000 7f72627fc700 1 --2- 172.21.15.37:0/896194040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f726406ac60 0x7f726419ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:42.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.194+0000 7f72627fc700 1 -- 172.21.15.37:0/896194040 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f726419fe40 con 0x7f726406b620 2024-06-10T00:56:42.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.194+0000 7f7262ffd700 1 --2- 172.21.15.37:0/896194040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f726406ac60 0x7f726419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:56:42.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.194+0000 7f72637fe700 1 --2- 172.21.15.37:0/896194040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7264108580 0x7f726419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:56:42.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.194+0000 7f72627fc700 1 --2- 172.21.15.37:0/896194040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f726406b620 0x7f726419b370 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f725800d950 tx=0x7f725800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:42.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.194+0000 7f724bfff700 1 -- 172.21.15.37:0/896194040 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258016070 con 0x7f726406b620 2024-06-10T00:56:42.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.194+0000 7f7269360700 1 -- 172.21.15.37:0/896194040 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72641a0130 con 0x7f726406b620 2024-06-10T00:56:42.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.194+0000 7f724bfff700 1 -- 172.21.15.37:0/896194040 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7258009e20 con 0x7f726406b620 2024-06-10T00:56:42.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.194+0000 7f724bfff700 1 -- 172.21.15.37:0/896194040 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725800f620 con 0x7f726406b620 2024-06-10T00:56:42.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.194+0000 7f7269360700 1 -- 172.21.15.37:0/896194040 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72641a7a30 con 0x7f726406b620 2024-06-10T00:56:42.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.195+0000 7f7269360700 1 -- 172.21.15.37:0/896194040 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7264069180 con 0x7f726406b620 2024-06-10T00:56:42.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.197+0000 7f724bfff700 1 -- 172.21.15.37:0/896194040 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f7258019070 con 0x7f726406b620 2024-06-10T00:56:42.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.197+0000 7f724bfff700 1 --2- 172.21.15.37:0/896194040 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f724c0a2cb0 0x7f724c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:42.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.198+0000 7f7262ffd700 1 --2- 172.21.15.37:0/896194040 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f724c0a2cb0 0x7f724c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:42.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.198+0000 7f724bfff700 1 -- 172.21.15.37:0/896194040 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f72580cf1c0 con 0x7f726406b620 2024-06-10T00:56:42.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.198+0000 7f7262ffd700 1 --2- 172.21.15.37:0/896194040 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f724c0a2cb0 0x7f724c0a5170 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7254000c00 tx=0x7f7254005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:42.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.201+0000 7f724bfff700 1 -- 172.21.15.37:0/896194040 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f725809bbd0 con 0x7f726406b620 2024-06-10T00:56:42.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:42 smithi167 ceph-mon[36917]: pgmap v430: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-06-10T00:56:42.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:42 smithi094 ceph-mon[36708]: pgmap v430: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-06-10T00:56:42.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.406+0000 7f7269360700 1 -- 172.21.15.37:0/896194040 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f72641a7c60 con 0x7f726406b620 2024-06-10T00:56:42.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.407+0000 7f724bfff700 1 -- 172.21.15.37:0/896194040 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24854 (secure 0 0 0) 0x7f725809b4d0 con 0x7f726406b620 2024-06-10T00:56:42.408 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:56:42.408 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":100,"fsid":"68793514-26c2-11ef-bc9b-c7b262605968","created":"2024-06-10T00:44:24.106713+0000","modified":"2024-06-10T00:54:22.546967+0000","last_up_change":"2024-06-10T00:51:31.562156+0000","last_in_change":"2024-06-10T00:51:11.294301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:47:44.773911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:52:31.684780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:52:34.885503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:52:44.415437+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1df6046a-6801-4950-9193-54303fed169b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6803","nonce":3594704094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6805","nonce":3594704094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6809","nonce":3594704094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6807","nonce":3594704094}]},"public_addr":"172.21.15.37:6803/3594704094","cluster_addr":"172.21.15.37:6805/3594704094","heartbeat_back_addr":"172.21.15.37:6809/3594704094","heartbeat_front_addr":"172.21.15.37:6807/3594704094","state":["exists","up"]},{"osd":1,"uuid":"4e1c8944-94e4-4e46-9efe-56229fe6df1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6801","nonce":3427727155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6803","nonce":3427727155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6807","nonce":3427727155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6805","nonce":3427727155}]},"public_addr":"172.21.15.94:6801/3427727155","cluster_addr":"172.21.15.94:6803/3427727155","heartbeat_back_addr":"172.21.15.94:6807/3427727155","heartbeat_front_addr":"172.21.15.94:6805/3427727155","state":["exists","up"]},{"osd":2,"uuid":"fd2fcedc-4c88-478f-8025-13e5944a27ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6801","nonce":4002498292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6803","nonce":4002498292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6807","nonce":4002498292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6805","nonce":4002498292}]},"public_addr":"172.21.15.167:6801/4002498292","cluster_addr":"172.21.15.167:6803/4002498292","heartbeat_back_addr":"172.21.15.167:6807/4002498292","heartbeat_front_addr":"172.21.15.167:6805/4002498292","state":["exists","up"]},{"osd":3,"uuid":"d0153676-818c-4dcc-83b1-287435a0c8a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6811","nonce":3425469324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6813","nonce":3425469324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6817","nonce":3425469324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6815","nonce":3425469324}]},"public_addr":"172.21.15.37:6811/3425469324","cluster_addr":"172.21.15.37:6813/3425469324","heartbeat_back_addr":"172.21.15.37:6817/3425469324","heartbeat_front_addr":"172.21.15.37:6815/3425469324","state":["exists","up"]},{"osd":4,"uuid":"172c333c-00c0-4e94-8870-acb3b32b2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6809","nonce":1740627431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6811","nonce":1740627431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6815","nonce":1740627431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6813","nonce":1740627431}]},"public_addr":"172.21.15.94:6809/1740627431","cluster_addr":"172.21.15.94:6811/1740627431","heartbeat_back_addr":"172.21.15.94:6815/1740627431","heartbeat_front_addr":"172.21.15.94:6813/1740627431","state":["exists","up"]},{"osd":5,"uuid":"dbce656e-5f93-4bda-baf2-fc3cc03d8282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6809","nonce":221805236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6811","nonce":221805236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6815","nonce":221805236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6813","nonce":221805236}]},"public_addr":"172.21.15.167:6809/221805236","cluster_addr":"172.21.15.167:6811/221805236","heartbeat_back_addr":"172.21.15.167:6815/221805236","heartbeat_front_addr":"172.21.15.167:6813/221805236","state":["exists","up"]},{"osd":6,"uuid":"fcbda266-6d60-43b2-a51c-4e562476d1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6819","nonce":472670875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6821","nonce":472670875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6825","nonce":472670875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6823","nonce":472670875}]},"public_addr":"172.21.15.37:6819/472670875","cluster_addr":"172.21.15.37:6821/472670875","heartbeat_back_addr":"172.21.15.37:6825/472670875","heartbeat_front_addr":"172.21.15.37:6823/472670875","state":["exists","up"]},{"osd":7,"uuid":"ed829ed3-ba85-4e82-ac37-e44e9534d37c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6817","nonce":3932281552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6819","nonce":3932281552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6823","nonce":3932281552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6821","nonce":3932281552}]},"public_addr":"172.21.15.94:6817/3932281552","cluster_addr":"172.21.15.94:6819/3932281552","heartbeat_back_addr":"172.21.15.94:6823/3932281552","heartbeat_front_addr":"172.21.15.94:6821/3932281552","state":["exists","up"]},{"osd":8,"uuid":"7d6267d8-72f4-4704-b554-bafc52d11a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6817","nonce":2290216047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6819","nonce":2290216047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6823","nonce":2290216047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6821","nonce":2290216047}]},"public_addr":"172.21.15.167:6817/2290216047","cluster_addr":"172.21.15.167:6819/2290216047","heartbeat_back_addr":"172.21.15.167:6823/2290216047","heartbeat_front_addr":"172.21.15.167:6821/2290216047","state":["exists","up"]},{"osd":9,"uuid":"210fdc20-0a36-4eeb-bdf2-c0a7f130e19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6827","nonce":2063666428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6829","nonce":2063666428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6833","nonce":2063666428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6831","nonce":2063666428}]},"public_addr":"172.21.15.37:6827/2063666428","cluster_addr":"172.21.15.37:6829/2063666428","heartbeat_back_addr":"172.21.15.37:6833/2063666428","heartbeat_front_addr":"172.21.15.37:6831/2063666428","state":["exists","up"]},{"osd":10,"uuid":"e200b371-b3cf-4b92-a659-f155ca86f0a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6825","nonce":1706618899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6827","nonce":1706618899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6831","nonce":1706618899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6829","nonce":1706618899}]},"public_addr":"172.21.15.94:6825/1706618899","cluster_addr":"172.21.15.94:6827/1706618899","heartbeat_back_addr":"172.21.15.94:6831/1706618899","heartbeat_front_addr":"172.21.15.94:6829/1706618899","state":["exists","up"]},{"osd":11,"uuid":"cfa427df-37a5-4d28-ac2e-af777d607008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6825","nonce":1349892516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6827","nonce":1349892516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6831","nonce":1349892516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6829","nonce":1349892516}]},"public_addr":"172.21.15.167:6825/1349892516","cluster_addr":"172.21.15.167:6827/1349892516","heartbeat_back_addr":"172.21.15.167:6831/1349892516","heartbeat_front_addr":"172.21.15.167:6829/1349892516","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:46:51.846147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:15.763723+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:40.106800+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:05.923204+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:30.501883+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:55.565575+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:20.411360+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:46.071776+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:11.854599+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:37.608818+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:03.176767+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:29.768705+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/4084722397":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/3571803585":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/416587443":"2024-06-11T00:45:20.254342+0000","172.21.15.37:6801/2766692263":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/2026344134":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/2781321886":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6801/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/1110143997":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2766692263":"2024-06-11T00:45:20.254342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:56:42.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.410+0000 7f7269360700 1 -- 172.21.15.37:0/896194040 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f724c0a2cb0 msgr2=0x7f724c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:42.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.410+0000 7f7269360700 1 --2- 172.21.15.37:0/896194040 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f724c0a2cb0 0x7f724c0a5170 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7254000c00 tx=0x7f7254005fb0 comp rx=0 tx=0).stop 2024-06-10T00:56:42.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.410+0000 7f7269360700 1 -- 172.21.15.37:0/896194040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f726406b620 msgr2=0x7f726419b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:42.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.410+0000 7f7269360700 1 --2- 172.21.15.37:0/896194040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f726406b620 0x7f726419b370 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f725800d950 tx=0x7f725800dc60 comp rx=0 tx=0).stop 2024-06-10T00:56:42.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.410+0000 7f7269360700 1 -- 172.21.15.37:0/896194040 shutdown_connections 2024-06-10T00:56:42.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.410+0000 7f7269360700 1 --2- 172.21.15.37:0/896194040 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f724c0a2cb0 0x7f724c0a5170 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:42.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.410+0000 7f7269360700 1 --2- 172.21.15.37:0/896194040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7264108580 0x7f726419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:42.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.410+0000 7f7269360700 1 --2- 172.21.15.37:0/896194040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f726406ac60 0x7f726419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:42.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.410+0000 7f7269360700 1 --2- 172.21.15.37:0/896194040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f726406b620 0x7f726419b370 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:42.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.410+0000 7f7269360700 1 -- 172.21.15.37:0/896194040 >> 172.21.15.37:0/896194040 conn(0x7f7264078430 msgr2=0x7f726410d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:42.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.410+0000 7f7269360700 1 -- 172.21.15.37:0/896194040 shutdown_connections 2024-06-10T00:56:42.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:42.410+0000 7f7269360700 1 -- 172.21.15.37:0/896194040 wait complete. 2024-06-10T00:56:43.211 DEBUG:tasks.fwd_scrub:Yielding 2024-06-10T00:56:43.211 INFO:teuthology.run_tasks:Running task background_exec... 2024-06-10T00:56:43.222 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-06-10T00:56:43.222 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-06-10T00:56:43.222 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-06-10T00:56:43.223 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T00:56:43.223 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T00:56:43.225 INFO:teuthology.run_tasks:Running task quiescer... 2024-06-10T00:56:43.261 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-10T00:56:43.261 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-10T00:56:43.284 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:43 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/896194040' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:56:43.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:43 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/896194040' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:56:43.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:43 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/896194040' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:56:43.493 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:56:43.524 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:56:44.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:44 smithi167 ceph-mon[36917]: pgmap v431: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-06-10T00:56:44.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:44 smithi094 ceph-mon[36708]: pgmap v431: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-06-10T00:56:44.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:44 smithi037 ceph-mon[32192]: pgmap v431: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-06-10T00:56:46.112 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:46 smithi037 ceph-mon[32192]: pgmap v432: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-10T00:56:46.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:46 smithi167 ceph-mon[36917]: pgmap v432: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-10T00:56:46.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:46 smithi094 ceph-mon[36708]: pgmap v432: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-10T00:56:46.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.427+0000 7f23b4f5e700 1 -- 172.21.15.37:0/4009958382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23b0113660 msgr2=0x7f23b0115a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:46.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.427+0000 7f23b4f5e700 1 --2- 172.21.15.37:0/4009958382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23b0113660 0x7f23b0115a50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f23a800b0a0 tx=0x7f23a800b3b0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.427+0000 7f23b4f5e700 1 -- 172.21.15.37:0/4009958382 shutdown_connections 2024-06-10T00:56:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.427+0000 7f23b4f5e700 1 --2- 172.21.15.37:0/4009958382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23b0113660 0x7f23b0115a50 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.427+0000 7f23b4f5e700 1 --2- 172.21.15.37:0/4009958382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23b00746c0 0x7f23b0074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.427+0000 7f23b4f5e700 1 --2- 172.21.15.37:0/4009958382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b0074fe0 0x7f23b0113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.427+0000 7f23b4f5e700 1 -- 172.21.15.37:0/4009958382 >> 172.21.15.37:0/4009958382 conn(0x7f23b00795f0 msgr2=0x7f23b0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.427+0000 7f23b4f5e700 1 -- 172.21.15.37:0/4009958382 shutdown_connections 2024-06-10T00:56:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.427+0000 7f23b4f5e700 1 -- 172.21.15.37:0/4009958382 wait complete. 2024-06-10T00:56:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.428+0000 7f23b4f5e700 1 Processor -- start 2024-06-10T00:56:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.428+0000 7f23b4f5e700 1 -- start start 2024-06-10T00:56:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.428+0000 7f23b4f5e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b00746c0 0x7f23b019f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.428+0000 7f23b4f5e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23b0074fe0 0x7f23b019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.428+0000 7f23b4f5e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23b0113660 0x7f23b01a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.428+0000 7f23b4f5e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f23b0118d80 con 0x7f23b0074fe0 2024-06-10T00:56:46.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.428+0000 7f23b4f5e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23b0118c00 con 0x7f23b00746c0 2024-06-10T00:56:46.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.428+0000 7f23b4f5e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f23b0118f00 con 0x7f23b0113660 2024-06-10T00:56:46.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.428+0000 7f23add9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23b0074fe0 0x7f23b019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:46.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.428+0000 7f23add9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23b0074fe0 0x7f23b019f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59376/0 (socket says 172.21.15.37:59376) 2024-06-10T00:56:46.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.428+0000 7f23add9b700 1 -- 172.21.15.37:0/4145817850 learned_addr learned my addr 172.21.15.37:0/4145817850 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:46.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.428+0000 7f23ae59c700 1 --2- 172.21.15.37:0/4145817850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b00746c0 0x7f23b019f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:46.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.428+0000 7f23aed9d700 1 --2- 172.21.15.37:0/4145817850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23b0113660 0x7f23b01a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:46.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.429+0000 7f23add9b700 1 -- 172.21.15.37:0/4145817850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23b0113660 msgr2=0x7f23b01a3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:46.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.429+0000 7f23add9b700 1 --2- 172.21.15.37:0/4145817850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23b0113660 0x7f23b01a3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.429+0000 7f23add9b700 1 -- 172.21.15.37:0/4145817850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b00746c0 msgr2=0x7f23b019f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:46.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.429+0000 7f23add9b700 1 --2- 172.21.15.37:0/4145817850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b00746c0 0x7f23b019f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.429+0000 7f23add9b700 1 -- 172.21.15.37:0/4145817850 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b01a4160 con 0x7f23b0074fe0 2024-06-10T00:56:46.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.429+0000 7f23add9b700 1 --2- 172.21.15.37:0/4145817850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23b0074fe0 0x7f23b019f690 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f239800d950 tx=0x7f239800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:46.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.429+0000 7f23a77fe700 1 -- 172.21.15.37:0/4145817850 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2398016070 con 0x7f23b0074fe0 2024-06-10T00:56:46.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.429+0000 7f23b4f5e700 1 -- 172.21.15.37:0/4145817850 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23b01a4450 con 0x7f23b0074fe0 2024-06-10T00:56:46.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.429+0000 7f23a77fe700 1 -- 172.21.15.37:0/4145817850 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f239800fd70 con 0x7f23b0074fe0 2024-06-10T00:56:46.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.429+0000 7f23a77fe700 1 -- 172.21.15.37:0/4145817850 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239800e610 con 0x7f23b0074fe0 2024-06-10T00:56:46.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.429+0000 7f23b4f5e700 1 -- 172.21.15.37:0/4145817850 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23b01abcd0 con 0x7f23b0074fe0 2024-06-10T00:56:46.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.430+0000 7f23b4f5e700 1 -- 172.21.15.37:0/4145817850 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23b0069180 con 0x7f23b0074fe0 2024-06-10T00:56:46.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.431+0000 7f23a77fe700 1 -- 172.21.15.37:0/4145817850 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f2398019070 con 0x7f23b0074fe0 2024-06-10T00:56:46.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.432+0000 7f23a77fe700 1 --2- 172.21.15.37:0/4145817850 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f239c0a2cb0 0x7f239c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:46.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.432+0000 7f23ae59c700 1 --2- 172.21.15.37:0/4145817850 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f239c0a2cb0 0x7f239c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:46.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.432+0000 7f23a77fe700 1 -- 172.21.15.37:0/4145817850 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f23980cefc0 con 0x7f23b0074fe0 2024-06-10T00:56:46.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.432+0000 7f23ae59c700 1 --2- 172.21.15.37:0/4145817850 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f239c0a2cb0 0x7f239c0a5170 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f23a000ac50 tx=0x7f23a000a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:46.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.436+0000 7f23a77fe700 1 -- 172.21.15.37:0/4145817850 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f239809b9d0 con 0x7f23b0074fe0 2024-06-10T00:56:46.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.639+0000 7f23b4f5e700 1 -- 172.21.15.37:0/4145817850 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f23b0112a60 con 0x7f239c0a2cb0 2024-06-10T00:56:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.652+0000 7f23a77fe700 1 -- 172.21.15.37:0/4145817850 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340963 (secure 0 0 0) 0x7f23b0112a60 con 0x7f239c0a2cb0 2024-06-10T00:56:46.657 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:56:46.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.656+0000 7f23b4f5e700 1 -- 172.21.15.37:0/4145817850 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f239c0a2cb0 msgr2=0x7f239c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:46.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.656+0000 7f23b4f5e700 1 --2- 172.21.15.37:0/4145817850 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f239c0a2cb0 0x7f239c0a5170 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f23a000ac50 tx=0x7f23a000a380 comp rx=0 tx=0).stop 2024-06-10T00:56:46.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.656+0000 7f23b4f5e700 1 -- 172.21.15.37:0/4145817850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23b0074fe0 msgr2=0x7f23b019f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:46.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.656+0000 7f23b4f5e700 1 --2- 172.21.15.37:0/4145817850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23b0074fe0 0x7f23b019f690 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f239800d950 tx=0x7f239800dc60 comp rx=0 tx=0).stop 2024-06-10T00:56:46.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.656+0000 7f23b4f5e700 1 -- 172.21.15.37:0/4145817850 shutdown_connections 2024-06-10T00:56:46.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.656+0000 7f23b4f5e700 1 --2- 172.21.15.37:0/4145817850 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f239c0a2cb0 0x7f239c0a5170 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.656+0000 7f23b4f5e700 1 --2- 172.21.15.37:0/4145817850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23b0113660 0x7f23b01a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.656+0000 7f23b4f5e700 1 --2- 172.21.15.37:0/4145817850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b00746c0 0x7f23b019f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.656+0000 7f23b4f5e700 1 --2- 172.21.15.37:0/4145817850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23b0074fe0 0x7f23b019f690 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.656+0000 7f23b4f5e700 1 -- 172.21.15.37:0/4145817850 >> 172.21.15.37:0/4145817850 conn(0x7f23b00795f0 msgr2=0x7f23b0112350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:46.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.656+0000 7f23b4f5e700 1 -- 172.21.15.37:0/4145817850 shutdown_connections 2024-06-10T00:56:46.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.656+0000 7f23b4f5e700 1 -- 172.21.15.37:0/4145817850 wait complete. 2024-06-10T00:56:46.660 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-06-10T00:56:46.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.831+0000 7f068c577700 1 -- 172.21.15.37:0/777567283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06841066f0 msgr2=0x7f068410e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:46.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.831+0000 7f068c577700 1 --2- 172.21.15.37:0/777567283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06841066f0 0x7f068410e2c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f067c00b0d0 tx=0x7f067c00b3e0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.832+0000 7f068c577700 1 -- 172.21.15.37:0/777567283 shutdown_connections 2024-06-10T00:56:46.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.832+0000 7f068c577700 1 --2- 172.21.15.37:0/777567283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06841066f0 0x7f068410e2c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.832+0000 7f068c577700 1 --2- 172.21.15.37:0/777567283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0684101610 0x7f06841019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.832+0000 7f068c577700 1 --2- 172.21.15.37:0/777567283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0684101fc0 0x7f0684106010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.832+0000 7f068c577700 1 -- 172.21.15.37:0/777567283 >> 172.21.15.37:0/777567283 conn(0x7f0684078430 msgr2=0x7f0684078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:46.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.832+0000 7f068c577700 1 -- 172.21.15.37:0/777567283 shutdown_connections 2024-06-10T00:56:46.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.832+0000 7f068c577700 1 -- 172.21.15.37:0/777567283 wait complete. 2024-06-10T00:56:46.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.832+0000 7f068c577700 1 Processor -- start 2024-06-10T00:56:46.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.832+0000 7f068c577700 1 -- start start 2024-06-10T00:56:46.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.832+0000 7f068c577700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0684101610 0x7f068419d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:46.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.832+0000 7f068c577700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0684101fc0 0x7f068419dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:46.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.832+0000 7f068c577700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06841066f0 0x7f0684197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:46.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.832+0000 7f068c577700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f06841116e0 con 0x7f06841066f0 2024-06-10T00:56:46.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.832+0000 7f068c577700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0684111560 con 0x7f0684101610 2024-06-10T00:56:46.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.832+0000 7f068c577700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0684111860 con 0x7f0684101fc0 2024-06-10T00:56:46.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.833+0000 7f068ab14700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06841066f0 0x7f0684197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:46.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.833+0000 7f068ab14700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06841066f0 0x7f0684197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59396/0 (socket says 172.21.15.37:59396) 2024-06-10T00:56:46.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.833+0000 7f068ab14700 1 -- 172.21.15.37:0/1238373075 learned_addr learned my addr 172.21.15.37:0/1238373075 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:46.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.833+0000 7f068a313700 1 --2- 172.21.15.37:0/1238373075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0684101610 0x7f068419d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:46.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.833+0000 7f0689b12700 1 --2- 172.21.15.37:0/1238373075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0684101fc0 0x7f068419dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:46.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.833+0000 7f068ab14700 1 -- 172.21.15.37:0/1238373075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0684101fc0 msgr2=0x7f068419dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:46.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.833+0000 7f068ab14700 1 --2- 172.21.15.37:0/1238373075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0684101fc0 0x7f068419dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.833+0000 7f068ab14700 1 -- 172.21.15.37:0/1238373075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0684101610 msgr2=0x7f068419d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:46.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.833+0000 7f068ab14700 1 --2- 172.21.15.37:0/1238373075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0684101610 0x7f068419d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.833+0000 7f068ab14700 1 -- 172.21.15.37:0/1238373075 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0684198000 con 0x7f06841066f0 2024-06-10T00:56:46.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.833+0000 7f068ab14700 1 --2- 172.21.15.37:0/1238373075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06841066f0 0x7f0684197820 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f067c006070 tx=0x7f067c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:46.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.833+0000 7f067b7fe700 1 -- 172.21.15.37:0/1238373075 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c021070 con 0x7f06841066f0 2024-06-10T00:56:46.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.833+0000 7f067b7fe700 1 -- 172.21.15.37:0/1238373075 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f067c007b80 con 0x7f06841066f0 2024-06-10T00:56:46.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.833+0000 7f068c577700 1 -- 172.21.15.37:0/1238373075 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0684198290 con 0x7f06841066f0 2024-06-10T00:56:46.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.833+0000 7f067b7fe700 1 -- 172.21.15.37:0/1238373075 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c0052a0 con 0x7f06841066f0 2024-06-10T00:56:46.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.833+0000 7f068c577700 1 -- 172.21.15.37:0/1238373075 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06841a4500 con 0x7f06841066f0 2024-06-10T00:56:46.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.836+0000 7f067b7fe700 1 -- 172.21.15.37:0/1238373075 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f067c024070 con 0x7f06841066f0 2024-06-10T00:56:46.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.836+0000 7f067b7fe700 1 --2- 172.21.15.37:0/1238373075 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f06700a2cb0 0x7f06700a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:46.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.836+0000 7f068a313700 1 --2- 172.21.15.37:0/1238373075 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f06700a2cb0 0x7f06700a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:46.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.836+0000 7f067b7fe700 1 -- 172.21.15.37:0/1238373075 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f067c00f020 con 0x7f06841066f0 2024-06-10T00:56:46.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.837+0000 7f068a313700 1 --2- 172.21.15.37:0/1238373075 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f06700a2cb0 0x7f06700a5170 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0680005950 tx=0x7f068000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:46.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.837+0000 7f0696dfab80 1 Processor -- start 2024-06-10T00:56:46.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.837+0000 7f0696dfab80 1 -- start start 2024-06-10T00:56:46.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.837+0000 7f0696dfab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf1acbd080 0x55cf1acbb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:46.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.837+0000 7f0696dfab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf1acbb820 0x55cf1acbbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:46.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.837+0000 7f0696dfab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf1acc9960 0x55cf1accbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:46.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.837+0000 7f0696dfab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cf1abadfe0 con 0x55cf1acbd080 2024-06-10T00:56:46.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.837+0000 7f0696dfab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cf1ac0c8d0 con 0x55cf1acc9960 2024-06-10T00:56:46.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.837+0000 7f0696dfab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cf1ab744e0 con 0x55cf1acbb820 2024-06-10T00:56:46.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.838+0000 7f068a313700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf1acbd080 0x55cf1acbb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:46.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.838+0000 7f068a313700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf1acbd080 0x55cf1acbb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59412/0 (socket says 172.21.15.37:59412) 2024-06-10T00:56:46.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.838+0000 7f068a313700 1 -- 172.21.15.37:0/117685742 learned_addr learned my addr 172.21.15.37:0/117685742 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:46.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.838+0000 7f0689b12700 1 --2- 172.21.15.37:0/117685742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf1acbb820 0x55cf1acbbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:46.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.838+0000 7f068ab14700 1 --2- 172.21.15.37:0/117685742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf1acc9960 0x55cf1accbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:46.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.838+0000 7f068a313700 1 -- 172.21.15.37:0/117685742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf1acbb820 msgr2=0x55cf1acbbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:46.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.838+0000 7f068a313700 1 --2- 172.21.15.37:0/117685742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf1acbb820 0x55cf1acbbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.838+0000 7f068a313700 1 -- 172.21.15.37:0/117685742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf1acc9960 msgr2=0x55cf1accbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:46.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.838+0000 7f068a313700 1 --2- 172.21.15.37:0/117685742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf1acc9960 0x55cf1accbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.838+0000 7f068a313700 1 -- 172.21.15.37:0/117685742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf1ab5b950 con 0x55cf1acbd080 2024-06-10T00:56:46.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.838+0000 7f068a313700 1 --2- 172.21.15.37:0/117685742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf1acbd080 0x55cf1acbb2e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0680010fa0 tx=0x7f0680013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0ac5b3684a6d04 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:46.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.838+0000 7f0678ff9700 1 -- 172.21.15.37:0/117685742 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068000be60 con 0x55cf1acbd080 2024-06-10T00:56:46.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.838+0000 7f0678ff9700 1 -- 172.21.15.37:0/117685742 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f068001eb00 con 0x55cf1acbd080 2024-06-10T00:56:46.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.838+0000 7f0678ff9700 1 -- 172.21.15.37:0/117685742 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068001cc20 con 0x55cf1acbd080 2024-06-10T00:56:46.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.839+0000 7f0696dfab80 1 -- 172.21.15.37:0/117685742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf1acbd080 msgr2=0x55cf1acbb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:46.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.839+0000 7f0696dfab80 1 --2- 172.21.15.37:0/117685742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf1acbd080 0x55cf1acbb2e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0680010fa0 tx=0x7f0680013000 comp rx=0 tx=0).stop 2024-06-10T00:56:46.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.839+0000 7f0696dfab80 1 -- 172.21.15.37:0/117685742 shutdown_connections 2024-06-10T00:56:46.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.839+0000 7f0696dfab80 1 --2- 172.21.15.37:0/117685742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf1acc9960 0x55cf1accbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.839+0000 7f0696dfab80 1 --2- 172.21.15.37:0/117685742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf1acbd080 0x55cf1acbb2e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.839+0000 7f0696dfab80 1 --2- 172.21.15.37:0/117685742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf1acbb820 0x55cf1acbbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.839+0000 7f0696dfab80 1 -- 172.21.15.37:0/117685742 >> 172.21.15.37:0/117685742 conn(0x55cf1abe8100 msgr2=0x55cf1acb62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:46.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.839+0000 7f0696dfab80 1 -- 172.21.15.37:0/117685742 shutdown_connections 2024-06-10T00:56:46.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.839+0000 7f0696dfab80 1 -- 172.21.15.37:0/117685742 wait complete. 2024-06-10T00:56:46.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.839+0000 7f0696dfab80 1 -- start start 2024-06-10T00:56:46.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.840+0000 7f0696dfab80 1 Processor -- start 2024-06-10T00:56:46.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.840+0000 7f0696dfab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf1acbb820 0x55cf1acc9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:46.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.840+0000 7f0696dfab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf1acbd080 0x55cf1acc6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:46.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.840+0000 7f0696dfab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf1acc9960 0x55cf1acc68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:46.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.840+0000 7f0696dfab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cf1abad040 con 0x55cf1acbd080 2024-06-10T00:56:46.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.840+0000 7f0696dfab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cf1ac08890 con 0x55cf1acc9960 2024-06-10T00:56:46.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.840+0000 7f0696dfab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cf1accc700 con 0x55cf1acbb820 2024-06-10T00:56:46.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.840+0000 7f0696dfab80 10 client.0 _ll_register_callbacks cb 0x55cf1aba2630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:56:46.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.840+0000 7f0689b12700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf1acbd080 0x55cf1acc6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:46.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.840+0000 7f0689b12700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf1acbd080 0x55cf1acc6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59424/0 (socket says 172.21.15.37:59424) 2024-06-10T00:56:46.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.840+0000 7f0689b12700 1 -- 172.21.15.37:0/2579608169 learned_addr learned my addr 172.21.15.37:0/2579608169 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:46.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.840+0000 7f068ab14700 1 --2- 172.21.15.37:0/2579608169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf1acc9960 0x55cf1acc68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:46.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.840+0000 7f068a313700 1 --2- 172.21.15.37:0/2579608169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf1acbb820 0x55cf1acc9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:46.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.840+0000 7f0689b12700 1 -- 172.21.15.37:0/2579608169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf1acbb820 msgr2=0x55cf1acc9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:46.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.840+0000 7f0689b12700 1 --2- 172.21.15.37:0/2579608169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf1acbb820 0x55cf1acc9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.840+0000 7f0689b12700 1 -- 172.21.15.37:0/2579608169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf1acc9960 msgr2=0x55cf1acc68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:46.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.840+0000 7f0689b12700 1 --2- 172.21.15.37:0/2579608169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf1acc9960 0x55cf1acc68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.840+0000 7f0689b12700 1 -- 172.21.15.37:0/2579608169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf1abd27d0 con 0x55cf1acbd080 2024-06-10T00:56:46.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.840+0000 7f0689b12700 1 --2- 172.21.15.37:0/2579608169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf1acbd080 0x55cf1acc6380 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0674012790 tx=0x7f0674012aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5190ccd2acd6135 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:46.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.840+0000 7f066dffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:56:46.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.841+0000 7f066dffb700 1 -- 172.21.15.37:0/2579608169 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067401f060 con 0x55cf1acbd080 2024-06-10T00:56:46.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.841+0000 7f066dffb700 1 -- 172.21.15.37:0/2579608169 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f0674004460 con 0x55cf1acbd080 2024-06-10T00:56:46.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.841+0000 7f0696dfab80 1 -- 172.21.15.37:0/2579608169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f068001eb00 con 0x55cf1acbd080 2024-06-10T00:56:46.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.841+0000 7f066dffb700 1 -- 172.21.15.37:0/2579608169 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0674026050 con 0x55cf1acbd080 2024-06-10T00:56:46.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.841+0000 7f0696dfab80 10 client.15315 fetch_fsmap learned FSMap version 27 2024-06-10T00:56:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.841+0000 7f0696dfab80 1 -- 172.21.15.37:0/2579608169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cf1acd6e70 con 0x55cf1acbd080 2024-06-10T00:56:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.841+0000 7f066dffb700 1 -- 172.21.15.37:0/2579608169 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f068001eb00 con 0x55cf1acbd080 2024-06-10T00:56:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.841+0000 7f066dffb700 1 -- 172.21.15.37:0/2579608169 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0674010a40 con 0x55cf1acbd080 2024-06-10T00:56:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.841+0000 7f0696dfab80 10 client.15315 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:56:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.841+0000 7f0696dfab80 10 client.15315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T00:56:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.841+0000 7f0696dfab80 20 client.15315 populate_metadata read hostname 'smithi037' 2024-06-10T00:56:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.841+0000 7f0696dfab80 1 --2- 172.21.15.37:0/2579608169 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55cf1acd9500 0x55cf1acdb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.841+0000 7f0696dfab80 4 client.15315 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:56:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.841+0000 7f0696dfab80 1 -- 172.21.15.37:0/2579608169 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf1ac19dd0 con 0x55cf1acd9500 2024-06-10T00:56:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.842+0000 7f068a313700 1 --2- 172.21.15.37:0/2579608169 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55cf1acd9500 0x55cf1acdb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.842+0000 7f068a313700 1 --2- 172.21.15.37:0/2579608169 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55cf1acd9500 0x55cf1acdb8f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4772f9ddfd1a240f server_cookie=d6ebe87a582b4fdb in_seq=0 out_seq=0 2024-06-10T00:56:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.842+0000 7f066dffb700 10 client.15315 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:56:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.843+0000 7f066dffb700 1 -- 172.21.15.37:0/2579608169 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cf1ac19dd0 con 0x55cf1acd9500 2024-06-10T00:56:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.843+0000 7f066dffb700 10 client.15315 handle_command_reply: tid=0 2024-06-10T00:56:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.843+0000 7f0696dfab80 1 client.15315 shutdown 2024-06-10T00:56:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.843+0000 7f0696dfab80 1 -- 172.21.15.37:0/2579608169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf1acbd080 msgr2=0x55cf1acc6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.843+0000 7f0696dfab80 1 --2- 172.21.15.37:0/2579608169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf1acbd080 0x55cf1acc6380 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0674012790 tx=0x7f0674012aa0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.843+0000 7f0696dfab80 1 -- 172.21.15.37:0/2579608169 shutdown_connections 2024-06-10T00:56:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.843+0000 7f0696dfab80 1 --2- 172.21.15.37:0/2579608169 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55cf1acd9500 0x55cf1acdb8f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.843+0000 7f0696dfab80 1 --2- 172.21.15.37:0/2579608169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf1acc9960 0x55cf1acc68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.843+0000 7f0696dfab80 1 --2- 172.21.15.37:0/2579608169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf1acbb820 0x55cf1acc9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.843+0000 7f066dffb700 0 client.15315 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:56:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.843+0000 7f0696dfab80 1 --2- 172.21.15.37:0/2579608169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf1acbd080 0x55cf1acc6380 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.843+0000 7f0696dfab80 1 -- 172.21.15.37:0/2579608169 >> 172.21.15.37:0/2579608169 conn(0x55cf1abe8100 msgr2=0x55cf1acb5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.843+0000 7f0696dfab80 1 -- 172.21.15.37:0/2579608169 shutdown_connections 2024-06-10T00:56:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.843+0000 7f0696dfab80 1 -- 172.21.15.37:0/2579608169 wait complete. 2024-06-10T00:56:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.843+0000 7f0696dfab80 20 client.15315 trim_cache size 0 max 16384 2024-06-10T00:56:46.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.864+0000 7f0696dfab80 1 Processor -- start 2024-06-10T00:56:46.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.864+0000 7f0696dfab80 1 -- start start 2024-06-10T00:56:46.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.864+0000 7f0696dfab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf1acbb820 0x55cf1acc52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:46.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.864+0000 7f0696dfab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf1acbd080 0x55cf1acb5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:46.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.864+0000 7f0696dfab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf1acc9960 0x55cf1acb6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:46.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.864+0000 7f0696dfab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f06841aea50 con 0x55cf1acbd080 2024-06-10T00:56:46.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.864+0000 7f0696dfab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f06841ac2e0 con 0x55cf1acc9960 2024-06-10T00:56:46.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.864+0000 7f0696dfab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f06841aa130 con 0x55cf1acbb820 2024-06-10T00:56:46.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.864+0000 7f0689b12700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf1acbd080 0x55cf1acb5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:46.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.864+0000 7f0689b12700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf1acbd080 0x55cf1acb5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59440/0 (socket says 172.21.15.37:59440) 2024-06-10T00:56:46.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.864+0000 7f0689b12700 1 -- 172.21.15.37:0/730369693 learned_addr learned my addr 172.21.15.37:0/730369693 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:46.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.865+0000 7f068a313700 1 --2- 172.21.15.37:0/730369693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf1acbb820 0x55cf1acc52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:46.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.865+0000 7f0689b12700 1 -- 172.21.15.37:0/730369693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf1acbb820 msgr2=0x55cf1acc52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:46.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.865+0000 7f0689b12700 1 --2- 172.21.15.37:0/730369693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf1acbb820 0x55cf1acc52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.865+0000 7f0689b12700 1 -- 172.21.15.37:0/730369693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf1acc9960 msgr2=0x55cf1acb6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:46.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.865+0000 7f0689b12700 1 --2- 172.21.15.37:0/730369693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf1acc9960 0x55cf1acb6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.865+0000 7f0689b12700 1 -- 172.21.15.37:0/730369693 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf1acc93f0 con 0x55cf1acbd080 2024-06-10T00:56:46.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.865+0000 7f068a313700 1 --2- 172.21.15.37:0/730369693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf1acbb820 0x55cf1acc52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:56:46.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.865+0000 7f0689b12700 1 --2- 172.21.15.37:0/730369693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf1acbd080 0x55cf1acb5f00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0674010e90 tx=0x7f067400e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b833f1e9d3d175c7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:46.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.865+0000 7f066ffff700 1 -- 172.21.15.37:0/730369693 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067402f070 con 0x55cf1acbd080 2024-06-10T00:56:46.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.865+0000 7f066ffff700 1 -- 172.21.15.37:0/730369693 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06740057b0 con 0x55cf1acbd080 2024-06-10T00:56:46.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.865+0000 7f066ffff700 1 -- 172.21.15.37:0/730369693 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0674010830 con 0x55cf1acbd080 2024-06-10T00:56:46.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.866+0000 7f0696dfab80 1 -- 172.21.15.37:0/730369693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf1acbd080 msgr2=0x55cf1acb5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:46.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.866+0000 7f0696dfab80 1 --2- 172.21.15.37:0/730369693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf1acbd080 0x55cf1acb5f00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0674010e90 tx=0x7f067400e650 comp rx=0 tx=0).stop 2024-06-10T00:56:46.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.866+0000 7f0696dfab80 1 -- 172.21.15.37:0/730369693 shutdown_connections 2024-06-10T00:56:46.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.866+0000 7f0696dfab80 1 --2- 172.21.15.37:0/730369693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf1acc9960 0x55cf1acb6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.866+0000 7f0696dfab80 1 --2- 172.21.15.37:0/730369693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf1acbb820 0x55cf1acc52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.866+0000 7f0696dfab80 1 --2- 172.21.15.37:0/730369693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf1acbd080 0x55cf1acb5f00 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.866+0000 7f0696dfab80 1 -- 172.21.15.37:0/730369693 >> 172.21.15.37:0/730369693 conn(0x55cf1abe8100 msgr2=0x55cf1acc8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:46.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.866+0000 7f0696dfab80 1 -- 172.21.15.37:0/730369693 shutdown_connections 2024-06-10T00:56:46.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.866+0000 7f0696dfab80 1 -- 172.21.15.37:0/730369693 wait complete. 2024-06-10T00:56:46.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.866+0000 7f0696dfab80 1 -- start start 2024-06-10T00:56:46.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.866+0000 7f0696dfab80 1 Processor -- start 2024-06-10T00:56:46.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.866+0000 7f0696dfab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf1acbb820 0x55cf1acbf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:46.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.866+0000 7f0696dfab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf1acbd080 0x55cf1acbfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:46.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.866+0000 7f0696dfab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf1acc9960 0x55cf1acc02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:46.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.867+0000 7f0696dfab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f06841aa7a0 con 0x55cf1acc9960 2024-06-10T00:56:46.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.867+0000 7f0696dfab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f06841abc10 con 0x55cf1acbb820 2024-06-10T00:56:46.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.867+0000 7f0696dfab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f06841aae70 con 0x55cf1acbd080 2024-06-10T00:56:46.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.867+0000 7f068ab14700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf1acc9960 0x55cf1acc02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:46.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.867+0000 7f0696dfab80 10 client.0 _ll_register_callbacks cb 0x55cf1aba2630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:56:46.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.867+0000 7f068ab14700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf1acc9960 0x55cf1acc02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59456/0 (socket says 172.21.15.37:59456) 2024-06-10T00:56:46.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.867+0000 7f068ab14700 1 -- 172.21.15.37:0/3667028989 learned_addr learned my addr 172.21.15.37:0/3667028989 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:46.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.867+0000 7f068a313700 1 --2- 172.21.15.37:0/3667028989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf1acbb820 0x55cf1acbf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:46.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.867+0000 7f0689b12700 1 --2- 172.21.15.37:0/3667028989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf1acbd080 0x55cf1acbfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:46.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.867+0000 7f068ab14700 1 -- 172.21.15.37:0/3667028989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf1acbd080 msgr2=0x55cf1acbfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:46.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.867+0000 7f068ab14700 1 --2- 172.21.15.37:0/3667028989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf1acbd080 0x55cf1acbfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.867+0000 7f068ab14700 1 -- 172.21.15.37:0/3667028989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf1acbb820 msgr2=0x55cf1acbf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:46.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.867+0000 7f068ab14700 1 --2- 172.21.15.37:0/3667028989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf1acbb820 0x55cf1acbf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.867+0000 7f068ab14700 1 -- 172.21.15.37:0/3667028989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf1acd09c0 con 0x55cf1acc9960 2024-06-10T00:56:46.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.867+0000 7f068ab14700 1 --2- 172.21.15.37:0/3667028989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf1acc9960 0x55cf1acc02f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f067c0998d0 tx=0x7f067c09c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a311aea4af7d5d82 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:46.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.867+0000 7f066dffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:56:46.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.867+0000 7f066dffb700 1 -- 172.21.15.37:0/3667028989 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c0a12b0 con 0x55cf1acc9960 2024-06-10T00:56:46.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.867+0000 7f066dffb700 1 -- 172.21.15.37:0/3667028989 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f067c007d30 con 0x55cf1acc9960 2024-06-10T00:56:46.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.867+0000 7f0696dfab80 1 -- 172.21.15.37:0/3667028989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06740057b0 con 0x55cf1acc9960 2024-06-10T00:56:46.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.867+0000 7f066dffb700 1 -- 172.21.15.37:0/3667028989 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f067c0a5ae0 con 0x55cf1acc9960 2024-06-10T00:56:46.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.867+0000 7f0696dfab80 10 client.15321 fetch_fsmap learned FSMap version 27 2024-06-10T00:56:46.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.867+0000 7f0696dfab80 1 -- 172.21.15.37:0/3667028989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cf1acd1cb0 con 0x55cf1acc9960 2024-06-10T00:56:46.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.868+0000 7f066dffb700 1 -- 172.21.15.37:0/3667028989 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06740057b0 con 0x55cf1acc9960 2024-06-10T00:56:46.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.868+0000 7f066dffb700 1 -- 172.21.15.37:0/3667028989 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f067c011070 con 0x55cf1acc9960 2024-06-10T00:56:46.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.868+0000 7f0696dfab80 10 client.15321 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:56:46.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.868+0000 7f0696dfab80 10 client.15321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T00:56:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.868+0000 7f0696dfab80 20 client.15321 populate_metadata read hostname 'smithi037' 2024-06-10T00:56:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.868+0000 7f0696dfab80 1 --2- 172.21.15.37:0/3667028989 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55cf1acd2310 0x55cf1ad4c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.868+0000 7f0696dfab80 4 client.15321 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T00:56:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.868+0000 7f0696dfab80 1 -- 172.21.15.37:0/3667028989 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cf1ac14350 con 0x55cf1acd2310 2024-06-10T00:56:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.868+0000 7f068a313700 1 --2- 172.21.15.37:0/3667028989 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55cf1acd2310 0x55cf1ad4c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.868+0000 7f068a313700 1 --2- 172.21.15.37:0/3667028989 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55cf1acd2310 0x55cf1ad4c9a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffc51a5dcb7b91aa server_cookie=839cc88545f057fd in_seq=0 out_seq=0 2024-06-10T00:56:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.868+0000 7f066dffb700 10 client.15321 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:56:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.870+0000 7f066dffb700 1 -- 172.21.15.37:0/3667028989 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cf1ac14350 con 0x55cf1acd2310 2024-06-10T00:56:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.870+0000 7f066dffb700 10 client.15321 handle_command_reply: tid=0 2024-06-10T00:56:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.870+0000 7f0696dfab80 1 client.15321 shutdown 2024-06-10T00:56:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.870+0000 7f0696dfab80 1 -- 172.21.15.37:0/3667028989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf1acc9960 msgr2=0x55cf1acc02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.870+0000 7f0696dfab80 1 --2- 172.21.15.37:0/3667028989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf1acc9960 0x55cf1acc02f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f067c0998d0 tx=0x7f067c09c860 comp rx=0 tx=0).stop 2024-06-10T00:56:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.870+0000 7f0696dfab80 1 -- 172.21.15.37:0/3667028989 shutdown_connections 2024-06-10T00:56:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.870+0000 7f0696dfab80 1 --2- 172.21.15.37:0/3667028989 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55cf1acd2310 0x55cf1ad4c9a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.870+0000 7f0696dfab80 1 --2- 172.21.15.37:0/3667028989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf1acc9960 0x55cf1acc02f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.870+0000 7f0696dfab80 1 --2- 172.21.15.37:0/3667028989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf1acbb820 0x55cf1acbf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.870+0000 7f0696dfab80 1 --2- 172.21.15.37:0/3667028989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf1acbd080 0x55cf1acbfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.870+0000 7f0696dfab80 1 -- 172.21.15.37:0/3667028989 >> 172.21.15.37:0/3667028989 conn(0x55cf1abe8100 msgr2=0x55cf1acc5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:46.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.870+0000 7f066dffb700 0 client.15321 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:56:46.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.870+0000 7f0696dfab80 1 -- 172.21.15.37:0/3667028989 shutdown_connections 2024-06-10T00:56:46.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.870+0000 7f0696dfab80 1 -- 172.21.15.37:0/3667028989 wait complete. 2024-06-10T00:56:46.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.870+0000 7f0696dfab80 20 client.15321 trim_cache size 0 max 16384 2024-06-10T00:56:46.878 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T00:56:46.878 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-06-10T00:56:46.879 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "8221b7f4-fd6c-44aa-ad29-07188e3e5f28", 2024-06-10T00:56:46.879 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-06-10T00:56:46.879 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T00:56:46.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.871+0000 7f068c577700 1 -- 172.21.15.37:0/1238373075 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f06700a2cb0 msgr2=0x7f06700a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:46.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.871+0000 7f068c577700 1 --2- 172.21.15.37:0/1238373075 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f06700a2cb0 0x7f06700a5170 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0680005950 tx=0x7f068000b410 comp rx=0 tx=0).stop 2024-06-10T00:56:46.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.871+0000 7f068c577700 1 -- 172.21.15.37:0/1238373075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06841066f0 msgr2=0x7f0684197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:46.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.871+0000 7f068c577700 1 --2- 172.21.15.37:0/1238373075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06841066f0 0x7f0684197820 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f067c006070 tx=0x7f067c004d00 comp rx=0 tx=0).stop 2024-06-10T00:56:46.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.871+0000 7f068c577700 1 -- 172.21.15.37:0/1238373075 shutdown_connections 2024-06-10T00:56:46.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.871+0000 7f068c577700 1 --2- 172.21.15.37:0/1238373075 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f06700a2cb0 0x7f06700a5170 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.871+0000 7f068c577700 1 --2- 172.21.15.37:0/1238373075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06841066f0 0x7f0684197820 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.871+0000 7f068c577700 1 --2- 172.21.15.37:0/1238373075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0684101610 0x7f068419d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.871+0000 7f068c577700 1 --2- 172.21.15.37:0/1238373075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0684101fc0 0x7f068419dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:46.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.871+0000 7f068c577700 1 -- 172.21.15.37:0/1238373075 >> 172.21.15.37:0/1238373075 conn(0x7f0684078430 msgr2=0x7f06840ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:46.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.871+0000 7f068c577700 1 -- 172.21.15.37:0/1238373075 shutdown_connections 2024-06-10T00:56:46.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:46.871+0000 7f068c577700 1 -- 172.21.15.37:0/1238373075 wait complete. 2024-06-10T00:56:47.722 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T00:56:48.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:48 smithi167 ceph-mon[36917]: pgmap v433: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-10T00:56:48.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:48 smithi167 ceph-mon[36917]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:56:48.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:48 smithi167 ceph-mon[36917]: scrub queued for path: / 2024-06-10T00:56:48.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:48 smithi167 ceph-mon[36917]: scrub summary: idle+waiting paths [/] 2024-06-10T00:56:48.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:48 smithi167 ceph-mon[36917]: scrub summary: active paths [/] 2024-06-10T00:56:48.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:48 smithi167 ceph-mon[36917]: scrub summary: idle+waiting paths [/] 2024-06-10T00:56:48.362 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:48 smithi037 ceph-mon[32192]: pgmap v433: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-10T00:56:48.362 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:48 smithi037 ceph-mon[32192]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:56:48.362 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:48 smithi037 ceph-mon[32192]: scrub queued for path: / 2024-06-10T00:56:48.362 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:48 smithi037 ceph-mon[32192]: scrub summary: idle+waiting paths [/] 2024-06-10T00:56:48.362 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:48 smithi037 ceph-mon[32192]: scrub summary: active paths [/] 2024-06-10T00:56:48.362 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:48 smithi037 ceph-mon[32192]: scrub summary: idle+waiting paths [/] 2024-06-10T00:56:48.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:48 smithi094 ceph-mon[36708]: pgmap v433: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-10T00:56:48.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:48 smithi094 ceph-mon[36708]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:56:48.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:48 smithi094 ceph-mon[36708]: scrub queued for path: / 2024-06-10T00:56:48.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:48 smithi094 ceph-mon[36708]: scrub summary: idle+waiting paths [/] 2024-06-10T00:56:48.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:48 smithi094 ceph-mon[36708]: scrub summary: active paths [/] 2024-06-10T00:56:48.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:48 smithi094 ceph-mon[36708]: scrub summary: idle+waiting paths [/] 2024-06-10T00:56:48.393 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":433,"stamp":"2024-06-10T00:56:46.413110+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":672924,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1261,"num_read_kb":767,"num_write":374,"num_write_kb":3088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":802,"ondisk_log_size":802,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370920,"kb_used_data":13448,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570904,"statfs":{"total":1151940427776,"available":1151560605696,"internally_reserved":0,"allocated":13770752,"data_stored":8871914,"data_compressed":497625,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2910,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":44,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003250"},"pg_stats":[{"pgid":"3.3f","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373740+0000","last_change":"2024-06-10T00:52:35.971067+0000","last_active":"2024-06-10T00:55:20.373740+0000","last_peered":"2024-06-10T00:55:20.373740+0000","last_clean":"2024-06-10T00:55:20.373740+0000","last_became_active":"2024-06-10T00:52:35.970292+0000","last_became_peered":"2024-06-10T00:52:35.970292+0000","last_unstale":"2024-06-10T00:55:20.373740+0000","last_undegraded":"2024-06-10T00:55:20.373740+0000","last_fullsized":"2024-06-10T00:55:20.373740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:57:37.287238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.371246+0000","last_change":"2024-06-10T00:52:35.960421+0000","last_active":"2024-06-10T00:55:20.371246+0000","last_peered":"2024-06-10T00:55:20.371246+0000","last_clean":"2024-06-10T00:55:20.371246+0000","last_became_active":"2024-06-10T00:52:35.960070+0000","last_became_peered":"2024-06-10T00:52:35.960070+0000","last_unstale":"2024-06-10T00:55:20.371246+0000","last_undegraded":"2024-06-10T00:55:20.371246+0000","last_fullsized":"2024-06-10T00:55:20.371246+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:48:02.616434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375920+0000","last_change":"2024-06-10T00:52:35.964778+0000","last_active":"2024-06-10T00:55:20.375920+0000","last_peered":"2024-06-10T00:55:20.375920+0000","last_clean":"2024-06-10T00:55:20.375920+0000","last_became_active":"2024-06-10T00:52:35.964463+0000","last_became_peered":"2024-06-10T00:52:35.964463+0000","last_unstale":"2024-06-10T00:55:20.375920+0000","last_undegraded":"2024-06-10T00:55:20.375920+0000","last_fullsized":"2024-06-10T00:55:20.375920+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:52:44.987363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.360455+0000","last_change":"2024-06-10T00:52:35.963226+0000","last_active":"2024-06-10T00:55:20.360455+0000","last_peered":"2024-06-10T00:55:20.360455+0000","last_clean":"2024-06-10T00:55:20.360455+0000","last_became_active":"2024-06-10T00:52:35.962862+0000","last_became_peered":"2024-06-10T00:52:35.962862+0000","last_unstale":"2024-06-10T00:55:20.360455+0000","last_undegraded":"2024-06-10T00:55:20.360455+0000","last_fullsized":"2024-06-10T00:55:20.360455+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:08:56.418071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.360258+0000","last_change":"2024-06-10T00:52:35.957427+0000","last_active":"2024-06-10T00:55:20.360258+0000","last_peered":"2024-06-10T00:55:20.360258+0000","last_clean":"2024-06-10T00:55:20.360258+0000","last_became_active":"2024-06-10T00:52:35.957049+0000","last_became_peered":"2024-06-10T00:52:35.957049+0000","last_unstale":"2024-06-10T00:55:20.360258+0000","last_undegraded":"2024-06-10T00:55:20.360258+0000","last_fullsized":"2024-06-10T00:55:20.360258+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:16:04.257210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.355809+0000","last_change":"2024-06-10T00:52:35.954251+0000","last_active":"2024-06-10T00:55:20.355809+0000","last_peered":"2024-06-10T00:55:20.355809+0000","last_clean":"2024-06-10T00:55:20.355809+0000","last_became_active":"2024-06-10T00:52:35.953908+0000","last_became_peered":"2024-06-10T00:52:35.953908+0000","last_unstale":"2024-06-10T00:55:20.355809+0000","last_undegraded":"2024-06-10T00:55:20.355809+0000","last_fullsized":"2024-06-10T00:55:20.355809+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:55:25.125836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364343+0000","last_change":"2024-06-10T00:52:35.956822+0000","last_active":"2024-06-10T00:55:20.364343+0000","last_peered":"2024-06-10T00:55:20.364343+0000","last_clean":"2024-06-10T00:55:20.364343+0000","last_became_active":"2024-06-10T00:52:35.956466+0000","last_became_peered":"2024-06-10T00:52:35.956466+0000","last_unstale":"2024-06-10T00:55:20.364343+0000","last_undegraded":"2024-06-10T00:55:20.364343+0000","last_fullsized":"2024-06-10T00:55:20.364343+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:36:41.534392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.369347+0000","last_change":"2024-06-10T00:52:35.962978+0000","last_active":"2024-06-10T00:55:20.369347+0000","last_peered":"2024-06-10T00:55:20.369347+0000","last_clean":"2024-06-10T00:55:20.369347+0000","last_became_active":"2024-06-10T00:52:35.962609+0000","last_became_peered":"2024-06-10T00:52:35.962609+0000","last_unstale":"2024-06-10T00:55:20.369347+0000","last_undegraded":"2024-06-10T00:55:20.369347+0000","last_fullsized":"2024-06-10T00:55:20.369347+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:20:57.469577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.377074+0000","last_change":"2024-06-10T00:52:35.963203+0000","last_active":"2024-06-10T00:55:20.377074+0000","last_peered":"2024-06-10T00:55:20.377074+0000","last_clean":"2024-06-10T00:55:20.377074+0000","last_became_active":"2024-06-10T00:52:35.962789+0000","last_became_peered":"2024-06-10T00:52:35.962789+0000","last_unstale":"2024-06-10T00:55:20.377074+0000","last_undegraded":"2024-06-10T00:55:20.377074+0000","last_fullsized":"2024-06-10T00:55:20.377074+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:00:17.819193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"100'16","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.356376+0000","last_change":"2024-06-10T00:52:35.966940+0000","last_active":"2024-06-10T00:55:20.356376+0000","last_peered":"2024-06-10T00:55:20.356376+0000","last_clean":"2024-06-10T00:55:20.356376+0000","last_became_active":"2024-06-10T00:52:35.966554+0000","last_became_peered":"2024-06-10T00:52:35.966554+0000","last_unstale":"2024-06-10T00:55:20.356376+0000","last_undegraded":"2024-06-10T00:55:20.356376+0000","last_fullsized":"2024-06-10T00:55:20.356376+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:00.374437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372673+0000","last_change":"2024-06-10T00:52:35.960927+0000","last_active":"2024-06-10T00:55:20.372673+0000","last_peered":"2024-06-10T00:55:20.372673+0000","last_clean":"2024-06-10T00:55:20.372673+0000","last_became_active":"2024-06-10T00:52:35.960577+0000","last_became_peered":"2024-06-10T00:52:35.960577+0000","last_unstale":"2024-06-10T00:55:20.372673+0000","last_undegraded":"2024-06-10T00:55:20.372673+0000","last_fullsized":"2024-06-10T00:55:20.372673+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:50:02.366695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.355167+0000","last_change":"2024-06-10T00:52:35.964435+0000","last_active":"2024-06-10T00:55:20.355167+0000","last_peered":"2024-06-10T00:55:20.355167+0000","last_clean":"2024-06-10T00:55:20.355167+0000","last_became_active":"2024-06-10T00:52:35.964069+0000","last_became_peered":"2024-06-10T00:52:35.964069+0000","last_unstale":"2024-06-10T00:55:20.355167+0000","last_undegraded":"2024-06-10T00:55:20.355167+0000","last_fullsized":"2024-06-10T00:55:20.355167+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:06.708551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.361787+0000","last_change":"2024-06-10T00:52:35.966722+0000","last_active":"2024-06-10T00:55:20.361787+0000","last_peered":"2024-06-10T00:55:20.361787+0000","last_clean":"2024-06-10T00:55:20.361787+0000","last_became_active":"2024-06-10T00:52:35.966366+0000","last_became_peered":"2024-06-10T00:52:35.966366+0000","last_unstale":"2024-06-10T00:55:20.361787+0000","last_undegraded":"2024-06-10T00:55:20.361787+0000","last_fullsized":"2024-06-10T00:55:20.361787+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:57:46.021047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.371736+0000","last_change":"2024-06-10T00:52:35.955307+0000","last_active":"2024-06-10T00:55:20.371736+0000","last_peered":"2024-06-10T00:55:20.371736+0000","last_clean":"2024-06-10T00:55:20.371736+0000","last_became_active":"2024-06-10T00:52:35.954140+0000","last_became_peered":"2024-06-10T00:52:35.954140+0000","last_unstale":"2024-06-10T00:55:20.371736+0000","last_undegraded":"2024-06-10T00:55:20.371736+0000","last_fullsized":"2024-06-10T00:55:20.371736+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:47:22.493598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373915+0000","last_change":"2024-06-10T00:52:35.960592+0000","last_active":"2024-06-10T00:55:20.373915+0000","last_peered":"2024-06-10T00:55:20.373915+0000","last_clean":"2024-06-10T00:55:20.373915+0000","last_became_active":"2024-06-10T00:52:35.960263+0000","last_became_peered":"2024-06-10T00:52:35.960263+0000","last_unstale":"2024-06-10T00:55:20.373915+0000","last_undegraded":"2024-06-10T00:55:20.373915+0000","last_fullsized":"2024-06-10T00:55:20.373915+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:47:11.658666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372126+0000","last_change":"2024-06-10T00:52:35.965796+0000","last_active":"2024-06-10T00:55:20.372126+0000","last_peered":"2024-06-10T00:55:20.372126+0000","last_clean":"2024-06-10T00:55:20.372126+0000","last_became_active":"2024-06-10T00:52:35.963965+0000","last_became_peered":"2024-06-10T00:52:35.963965+0000","last_unstale":"2024-06-10T00:55:20.372126+0000","last_undegraded":"2024-06-10T00:55:20.372126+0000","last_fullsized":"2024-06-10T00:55:20.372126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:29:47.149852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374090+0000","last_change":"2024-06-10T00:52:35.970714+0000","last_active":"2024-06-10T00:55:20.374090+0000","last_peered":"2024-06-10T00:55:20.374090+0000","last_clean":"2024-06-10T00:55:20.374090+0000","last_became_active":"2024-06-10T00:52:35.970173+0000","last_became_peered":"2024-06-10T00:52:35.970173+0000","last_unstale":"2024-06-10T00:55:20.374090+0000","last_undegraded":"2024-06-10T00:55:20.374090+0000","last_fullsized":"2024-06-10T00:55:20.374090+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:33:28.183156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.370737+0000","last_change":"2024-06-10T00:52:35.963052+0000","last_active":"2024-06-10T00:55:20.370737+0000","last_peered":"2024-06-10T00:55:20.370737+0000","last_clean":"2024-06-10T00:55:20.370737+0000","last_became_active":"2024-06-10T00:52:35.962674+0000","last_became_peered":"2024-06-10T00:52:35.962674+0000","last_unstale":"2024-06-10T00:55:20.370737+0000","last_undegraded":"2024-06-10T00:55:20.370737+0000","last_fullsized":"2024-06-10T00:55:20.370737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:56:38.836854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"100'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.363943+0000","last_change":"2024-06-10T00:53:22.130336+0000","last_active":"2024-06-10T00:55:20.363943+0000","last_peered":"2024-06-10T00:55:20.363943+0000","last_clean":"2024-06-10T00:55:20.363943+0000","last_became_active":"2024-06-10T00:53:22.129910+0000","last_became_peered":"2024-06-10T00:53:22.129910+0000","last_unstale":"2024-06-10T00:55:20.363943+0000","last_undegraded":"2024-06-10T00:55:20.363943+0000","last_fullsized":"2024-06-10T00:55:20.363943+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:23:50.600821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554286+0000","last_change":"2024-06-10T00:52:45.997276+0000","last_active":"2024-06-10T00:54:22.554286+0000","last_peered":"2024-06-10T00:54:22.554286+0000","last_clean":"2024-06-10T00:54:22.554286+0000","last_became_active":"2024-06-10T00:52:45.995638+0000","last_became_peered":"2024-06-10T00:52:45.995638+0000","last_unstale":"2024-06-10T00:54:22.554286+0000","last_undegraded":"2024-06-10T00:54:22.554286+0000","last_fullsized":"2024-06-10T00:54:22.554286+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:14:20.922348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:40.422280+0000","last_change":"2024-06-10T00:52:32.760888+0000","last_active":"2024-06-10T00:54:40.422280+0000","last_peered":"2024-06-10T00:54:40.422280+0000","last_clean":"2024-06-10T00:54:40.422280+0000","last_became_active":"2024-06-10T00:52:32.760531+0000","last_became_peered":"2024-06-10T00:52:32.760531+0000","last_unstale":"2024-06-10T00:54:40.422280+0000","last_undegraded":"2024-06-10T00:54:40.422280+0000","last_fullsized":"2024-06-10T00:54:40.422280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:05:59.052251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"100'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372743+0000","last_change":"2024-06-10T00:52:35.967626+0000","last_active":"2024-06-10T00:55:20.372743+0000","last_peered":"2024-06-10T00:55:20.372743+0000","last_clean":"2024-06-10T00:55:20.372743+0000","last_became_active":"2024-06-10T00:52:35.967295+0000","last_became_peered":"2024-06-10T00:52:35.967295+0000","last_unstale":"2024-06-10T00:55:20.372743+0000","last_undegraded":"2024-06-10T00:55:20.372743+0000","last_fullsized":"2024-06-10T00:55:20.372743+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:07:55.054327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668105+0000","last_change":"2024-06-10T00:52:45.993974+0000","last_active":"2024-06-10T00:54:22.668105+0000","last_peered":"2024-06-10T00:54:22.668105+0000","last_clean":"2024-06-10T00:54:22.668105+0000","last_became_active":"2024-06-10T00:52:45.993242+0000","last_became_peered":"2024-06-10T00:52:45.993242+0000","last_unstale":"2024-06-10T00:54:22.668105+0000","last_undegraded":"2024-06-10T00:54:22.668105+0000","last_fullsized":"2024-06-10T00:54:22.668105+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:28:12.810368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:33.628114+0000","last_change":"2024-06-10T00:52:32.959905+0000","last_active":"2024-06-10T00:54:33.628114+0000","last_peered":"2024-06-10T00:54:33.628114+0000","last_clean":"2024-06-10T00:54:33.628114+0000","last_became_active":"2024-06-10T00:52:32.959551+0000","last_became_peered":"2024-06-10T00:52:32.959551+0000","last_unstale":"2024-06-10T00:54:33.628114+0000","last_undegraded":"2024-06-10T00:54:33.628114+0000","last_fullsized":"2024-06-10T00:54:33.628114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:33:48.029129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.377301+0000","last_change":"2024-06-10T00:52:35.965790+0000","last_active":"2024-06-10T00:55:20.377301+0000","last_peered":"2024-06-10T00:55:20.377301+0000","last_clean":"2024-06-10T00:55:20.377301+0000","last_became_active":"2024-06-10T00:52:35.965475+0000","last_became_peered":"2024-06-10T00:52:35.965475+0000","last_unstale":"2024-06-10T00:55:20.377301+0000","last_undegraded":"2024-06-10T00:55:20.377301+0000","last_fullsized":"2024-06-10T00:55:20.377301+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:24:02.894411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.667953+0000","last_change":"2024-06-10T00:52:45.994464+0000","last_active":"2024-06-10T00:54:22.667953+0000","last_peered":"2024-06-10T00:54:22.667953+0000","last_clean":"2024-06-10T00:54:22.667953+0000","last_became_active":"2024-06-10T00:52:45.994022+0000","last_became_peered":"2024-06-10T00:52:45.994022+0000","last_unstale":"2024-06-10T00:54:22.667953+0000","last_undegraded":"2024-06-10T00:54:22.667953+0000","last_fullsized":"2024-06-10T00:54:22.667953+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:27:25.063074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556658+0000","last_change":"2024-06-10T00:52:32.763885+0000","last_active":"2024-06-10T00:54:22.556658+0000","last_peered":"2024-06-10T00:54:22.556658+0000","last_clean":"2024-06-10T00:54:22.556658+0000","last_became_active":"2024-06-10T00:52:32.763606+0000","last_became_peered":"2024-06-10T00:52:32.763606+0000","last_unstale":"2024-06-10T00:54:22.556658+0000","last_undegraded":"2024-06-10T00:54:22.556658+0000","last_fullsized":"2024-06-10T00:54:22.556658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:01:58.161168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372890+0000","last_change":"2024-06-10T00:52:35.966948+0000","last_active":"2024-06-10T00:55:20.372890+0000","last_peered":"2024-06-10T00:55:20.372890+0000","last_clean":"2024-06-10T00:55:20.372890+0000","last_became_active":"2024-06-10T00:52:35.966345+0000","last_became_peered":"2024-06-10T00:52:35.966345+0000","last_unstale":"2024-06-10T00:55:20.372890+0000","last_undegraded":"2024-06-10T00:55:20.372890+0000","last_fullsized":"2024-06-10T00:55:20.372890+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:58:27.169201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.667762+0000","last_change":"2024-06-10T00:52:45.993645+0000","last_active":"2024-06-10T00:54:22.667762+0000","last_peered":"2024-06-10T00:54:22.667762+0000","last_clean":"2024-06-10T00:54:22.667762+0000","last_became_active":"2024-06-10T00:52:45.993183+0000","last_became_peered":"2024-06-10T00:52:45.993183+0000","last_unstale":"2024-06-10T00:54:22.667762+0000","last_undegraded":"2024-06-10T00:54:22.667762+0000","last_fullsized":"2024-06-10T00:54:22.667762+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:47:00.541427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.565600+0000","last_change":"2024-06-10T00:54:22.565600+0000","last_active":"2024-06-10T00:54:22.565600+0000","last_peered":"2024-06-10T00:54:22.565600+0000","last_clean":"2024-06-10T00:54:22.565600+0000","last_became_active":"2024-06-10T00:54:22.562561+0000","last_became_peered":"2024-06-10T00:54:22.562561+0000","last_unstale":"2024-06-10T00:54:22.565600+0000","last_undegraded":"2024-06-10T00:54:22.565600+0000","last_fullsized":"2024-06-10T00:54:22.565600+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:47:58.748050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"100'15","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365162+0000","last_change":"2024-06-10T00:53:22.199668+0000","last_active":"2024-06-10T00:55:20.365162+0000","last_peered":"2024-06-10T00:55:20.365162+0000","last_clean":"2024-06-10T00:55:20.365162+0000","last_became_active":"2024-06-10T00:53:22.199329+0000","last_became_peered":"2024-06-10T00:53:22.199329+0000","last_unstale":"2024-06-10T00:55:20.365162+0000","last_undegraded":"2024-06-10T00:55:20.365162+0000","last_fullsized":"2024-06-10T00:55:20.365162+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:37.640478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554689+0000","last_change":"2024-06-10T00:52:45.995767+0000","last_active":"2024-06-10T00:54:22.554689+0000","last_peered":"2024-06-10T00:54:22.554689+0000","last_clean":"2024-06-10T00:54:22.554689+0000","last_became_active":"2024-06-10T00:52:45.994659+0000","last_became_peered":"2024-06-10T00:52:45.994659+0000","last_unstale":"2024-06-10T00:54:22.554689+0000","last_undegraded":"2024-06-10T00:54:22.554689+0000","last_fullsized":"2024-06-10T00:54:22.554689+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:03:45.652248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'8","reported_seq":237,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:40.570510+0000","last_change":"2024-06-10T00:52:32.761637+0000","last_active":"2024-06-10T00:56:40.570510+0000","last_peered":"2024-06-10T00:56:40.570510+0000","last_clean":"2024-06-10T00:56:40.570510+0000","last_became_active":"2024-06-10T00:52:32.761144+0000","last_became_peered":"2024-06-10T00:52:32.761144+0000","last_unstale":"2024-06-10T00:56:40.570510+0000","last_undegraded":"2024-06-10T00:56:40.570510+0000","last_fullsized":"2024-06-10T00:56:40.570510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:55:05.616564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1461,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.362574+0000","last_change":"2024-06-10T00:52:35.962683+0000","last_active":"2024-06-10T00:55:20.362574+0000","last_peered":"2024-06-10T00:55:20.362574+0000","last_clean":"2024-06-10T00:55:20.362574+0000","last_became_active":"2024-06-10T00:52:35.962327+0000","last_became_peered":"2024-06-10T00:52:35.962327+0000","last_unstale":"2024-06-10T00:55:20.362574+0000","last_undegraded":"2024-06-10T00:55:20.362574+0000","last_fullsized":"2024-06-10T00:55:20.362574+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:38:19.063306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554563+0000","last_change":"2024-06-10T00:52:45.996882+0000","last_active":"2024-06-10T00:54:22.554563+0000","last_peered":"2024-06-10T00:54:22.554563+0000","last_clean":"2024-06-10T00:54:22.554563+0000","last_became_active":"2024-06-10T00:52:45.995767+0000","last_became_peered":"2024-06-10T00:52:45.995767+0000","last_unstale":"2024-06-10T00:54:22.554563+0000","last_undegraded":"2024-06-10T00:54:22.554563+0000","last_fullsized":"2024-06-10T00:54:22.554563+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:28:03.766944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553970+0000","last_change":"2024-06-10T00:53:22.200546+0000","last_active":"2024-06-10T00:54:22.553970+0000","last_peered":"2024-06-10T00:54:22.553970+0000","last_clean":"2024-06-10T00:54:22.553970+0000","last_became_active":"2024-06-10T00:53:22.200059+0000","last_became_peered":"2024-06-10T00:53:22.200059+0000","last_unstale":"2024-06-10T00:54:22.553970+0000","last_undegraded":"2024-06-10T00:54:22.553970+0000","last_fullsized":"2024-06-10T00:54:22.553970+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:08:47.682594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.356418+0000","last_change":"2024-06-10T00:52:35.963105+0000","last_active":"2024-06-10T00:55:20.356418+0000","last_peered":"2024-06-10T00:55:20.356418+0000","last_clean":"2024-06-10T00:55:20.356418+0000","last_became_active":"2024-06-10T00:52:35.962627+0000","last_became_peered":"2024-06-10T00:52:35.962627+0000","last_unstale":"2024-06-10T00:55:20.356418+0000","last_undegraded":"2024-06-10T00:55:20.356418+0000","last_fullsized":"2024-06-10T00:55:20.356418+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:55:51.231795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.559190+0000","last_change":"2024-06-10T00:52:45.992557+0000","last_active":"2024-06-10T00:54:22.559190+0000","last_peered":"2024-06-10T00:54:22.559190+0000","last_clean":"2024-06-10T00:54:22.559190+0000","last_became_active":"2024-06-10T00:52:45.992166+0000","last_became_peered":"2024-06-10T00:52:45.992166+0000","last_unstale":"2024-06-10T00:54:22.559190+0000","last_undegraded":"2024-06-10T00:54:22.559190+0000","last_fullsized":"2024-06-10T00:54:22.559190+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:49:22.368409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553428+0000","last_change":"2024-06-10T00:52:32.751867+0000","last_active":"2024-06-10T00:54:22.553428+0000","last_peered":"2024-06-10T00:54:22.553428+0000","last_clean":"2024-06-10T00:54:22.553428+0000","last_became_active":"2024-06-10T00:52:32.751516+0000","last_became_peered":"2024-06-10T00:52:32.751516+0000","last_unstale":"2024-06-10T00:54:22.553428+0000","last_undegraded":"2024-06-10T00:54:22.553428+0000","last_fullsized":"2024-06-10T00:54:22.553428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:37:55.920613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.370097+0000","last_change":"2024-06-10T00:52:35.957772+0000","last_active":"2024-06-10T00:55:20.370097+0000","last_peered":"2024-06-10T00:55:20.370097+0000","last_clean":"2024-06-10T00:55:20.370097+0000","last_became_active":"2024-06-10T00:52:35.957470+0000","last_became_peered":"2024-06-10T00:52:35.957470+0000","last_unstale":"2024-06-10T00:55:20.370097+0000","last_undegraded":"2024-06-10T00:55:20.370097+0000","last_fullsized":"2024-06-10T00:55:20.370097+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:16:43.645292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878568+0000","last_change":"2024-06-10T00:52:45.996807+0000","last_active":"2024-06-10T00:54:22.878568+0000","last_peered":"2024-06-10T00:54:22.878568+0000","last_clean":"2024-06-10T00:54:22.878568+0000","last_became_active":"2024-06-10T00:52:45.996378+0000","last_became_peered":"2024-06-10T00:52:45.996378+0000","last_unstale":"2024-06-10T00:54:22.878568+0000","last_undegraded":"2024-06-10T00:54:22.878568+0000","last_fullsized":"2024-06-10T00:54:22.878568+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:40:54.251790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'1","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561873+0000","last_change":"2024-06-10T00:52:32.759625+0000","last_active":"2024-06-10T00:54:22.561873+0000","last_peered":"2024-06-10T00:54:22.561873+0000","last_clean":"2024-06-10T00:54:22.561873+0000","last_became_active":"2024-06-10T00:52:32.759299+0000","last_became_peered":"2024-06-10T00:52:32.759299+0000","last_unstale":"2024-06-10T00:54:22.561873+0000","last_undegraded":"2024-06-10T00:54:22.561873+0000","last_fullsized":"2024-06-10T00:54:22.561873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:11:44.966453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365673+0000","last_change":"2024-06-10T00:52:35.957672+0000","last_active":"2024-06-10T00:55:20.365673+0000","last_peered":"2024-06-10T00:55:20.365673+0000","last_clean":"2024-06-10T00:55:20.365673+0000","last_became_active":"2024-06-10T00:52:35.957328+0000","last_became_peered":"2024-06-10T00:52:35.957328+0000","last_unstale":"2024-06-10T00:55:20.365673+0000","last_undegraded":"2024-06-10T00:55:20.365673+0000","last_fullsized":"2024-06-10T00:55:20.365673+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:43:59.658178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601278+0000","last_change":"2024-06-10T00:52:45.996677+0000","last_active":"2024-06-10T00:54:22.601278+0000","last_peered":"2024-06-10T00:54:22.601278+0000","last_clean":"2024-06-10T00:54:22.601278+0000","last_became_active":"2024-06-10T00:52:45.994483+0000","last_became_peered":"2024-06-10T00:52:45.994483+0000","last_unstale":"2024-06-10T00:54:22.601278+0000","last_undegraded":"2024-06-10T00:54:22.601278+0000","last_fullsized":"2024-06-10T00:54:22.601278+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:03:28.712072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'5","reported_seq":235,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:44.576351+0000","last_change":"2024-06-10T00:52:32.754829+0000","last_active":"2024-06-10T00:56:44.576351+0000","last_peered":"2024-06-10T00:56:44.576351+0000","last_clean":"2024-06-10T00:56:44.576351+0000","last_became_active":"2024-06-10T00:52:32.754386+0000","last_became_peered":"2024-06-10T00:52:32.754386+0000","last_unstale":"2024-06-10T00:56:44.576351+0000","last_undegraded":"2024-06-10T00:56:44.576351+0000","last_fullsized":"2024-06-10T00:56:44.576351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:00.870324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":185,"num_read_kb":185,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.371610+0000","last_change":"2024-06-10T00:52:35.966871+0000","last_active":"2024-06-10T00:55:20.371610+0000","last_peered":"2024-06-10T00:55:20.371610+0000","last_clean":"2024-06-10T00:55:20.371610+0000","last_became_active":"2024-06-10T00:52:35.966492+0000","last_became_peered":"2024-06-10T00:52:35.966492+0000","last_unstale":"2024-06-10T00:55:20.371610+0000","last_undegraded":"2024-06-10T00:55:20.371610+0000","last_fullsized":"2024-06-10T00:55:20.371610+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:24:41.048935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:23.582564+0000","last_change":"2024-06-10T00:54:22.879457+0000","last_active":"2024-06-10T00:54:23.582564+0000","last_peered":"2024-06-10T00:54:23.582564+0000","last_clean":"2024-06-10T00:54:23.582564+0000","last_became_active":"2024-06-10T00:54:22.878978+0000","last_became_peered":"2024-06-10T00:54:22.878978+0000","last_unstale":"2024-06-10T00:54:23.582564+0000","last_undegraded":"2024-06-10T00:54:23.582564+0000","last_fullsized":"2024-06-10T00:54:23.582564+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:46:38.819250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404659+0000","last_change":"2024-06-10T00:52:32.765794+0000","last_active":"2024-06-10T00:54:24.404659+0000","last_peered":"2024-06-10T00:54:24.404659+0000","last_clean":"2024-06-10T00:54:24.404659+0000","last_became_active":"2024-06-10T00:52:32.765276+0000","last_became_peered":"2024-06-10T00:52:32.765276+0000","last_unstale":"2024-06-10T00:54:24.404659+0000","last_undegraded":"2024-06-10T00:54:24.404659+0000","last_fullsized":"2024-06-10T00:54:24.404659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:06:51.209012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365186+0000","last_change":"2024-06-10T00:52:35.954897+0000","last_active":"2024-06-10T00:55:20.365186+0000","last_peered":"2024-06-10T00:55:20.365186+0000","last_clean":"2024-06-10T00:55:20.365186+0000","last_became_active":"2024-06-10T00:52:35.954565+0000","last_became_peered":"2024-06-10T00:52:35.954565+0000","last_unstale":"2024-06-10T00:55:20.365186+0000","last_undegraded":"2024-06-10T00:55:20.365186+0000","last_fullsized":"2024-06-10T00:55:20.365186+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:39:26.702804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668190+0000","last_change":"2024-06-10T00:52:45.992194+0000","last_active":"2024-06-10T00:54:22.668190+0000","last_peered":"2024-06-10T00:54:22.668190+0000","last_clean":"2024-06-10T00:54:22.668190+0000","last_became_active":"2024-06-10T00:52:45.991453+0000","last_became_peered":"2024-06-10T00:52:45.991453+0000","last_unstale":"2024-06-10T00:54:22.668190+0000","last_undegraded":"2024-06-10T00:54:22.668190+0000","last_fullsized":"2024-06-10T00:54:22.668190+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:11:10.775610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561610+0000","last_change":"2024-06-10T00:52:32.962214+0000","last_active":"2024-06-10T00:54:22.561610+0000","last_peered":"2024-06-10T00:54:22.561610+0000","last_clean":"2024-06-10T00:54:22.561610+0000","last_became_active":"2024-06-10T00:52:32.961835+0000","last_became_peered":"2024-06-10T00:52:32.961835+0000","last_unstale":"2024-06-10T00:54:22.561610+0000","last_undegraded":"2024-06-10T00:54:22.561610+0000","last_fullsized":"2024-06-10T00:54:22.561610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:36:44.724122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.377485+0000","last_change":"2024-06-10T00:52:35.963413+0000","last_active":"2024-06-10T00:55:20.377485+0000","last_peered":"2024-06-10T00:55:20.377485+0000","last_clean":"2024-06-10T00:55:20.377485+0000","last_became_active":"2024-06-10T00:52:35.962922+0000","last_became_peered":"2024-06-10T00:52:35.962922+0000","last_unstale":"2024-06-10T00:55:20.377485+0000","last_undegraded":"2024-06-10T00:55:20.377485+0000","last_fullsized":"2024-06-10T00:55:20.377485+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:05:32.164696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":34,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.560580+0000","last_change":"2024-06-10T00:54:22.560196+0000","last_active":"2024-06-10T00:54:22.560580+0000","last_peered":"2024-06-10T00:54:22.560580+0000","last_clean":"2024-06-10T00:54:22.560580+0000","last_became_active":"2024-06-10T00:54:22.559826+0000","last_became_peered":"2024-06-10T00:54:22.559826+0000","last_unstale":"2024-06-10T00:54:22.560580+0000","last_undegraded":"2024-06-10T00:54:22.560580+0000","last_fullsized":"2024-06-10T00:54:22.560580+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:09:28.456394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'46","reported_seq":475,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:45.580653+0000","last_change":"2024-06-10T00:52:32.761592+0000","last_active":"2024-06-10T00:56:45.580653+0000","last_peered":"2024-06-10T00:56:45.580653+0000","last_clean":"2024-06-10T00:56:45.580653+0000","last_became_active":"2024-06-10T00:52:32.761169+0000","last_became_peered":"2024-06-10T00:52:32.761169+0000","last_unstale":"2024-06-10T00:56:45.580653+0000","last_undegraded":"2024-06-10T00:56:45.580653+0000","last_fullsized":"2024-06-10T00:56:45.580653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:08:33.015035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77873,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":384,"num_read_kb":270,"num_write":47,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.351972+0000","last_change":"2024-06-10T00:52:35.955304+0000","last_active":"2024-06-10T00:55:20.351972+0000","last_peered":"2024-06-10T00:55:20.351972+0000","last_clean":"2024-06-10T00:55:20.351972+0000","last_became_active":"2024-06-10T00:52:35.954949+0000","last_became_peered":"2024-06-10T00:52:35.954949+0000","last_unstale":"2024-06-10T00:55:20.351972+0000","last_undegraded":"2024-06-10T00:55:20.351972+0000","last_fullsized":"2024-06-10T00:55:20.351972+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:56:42.890022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561827+0000","last_change":"2024-06-10T00:52:45.992974+0000","last_active":"2024-06-10T00:54:22.561827+0000","last_peered":"2024-06-10T00:54:22.561827+0000","last_clean":"2024-06-10T00:54:22.561827+0000","last_became_active":"2024-06-10T00:52:45.992490+0000","last_became_peered":"2024-06-10T00:52:45.992490+0000","last_unstale":"2024-06-10T00:54:22.561827+0000","last_undegraded":"2024-06-10T00:54:22.561827+0000","last_fullsized":"2024-06-10T00:54:22.561827+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:03:45.543476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554437+0000","last_change":"2024-06-10T00:52:32.762347+0000","last_active":"2024-06-10T00:54:22.554437+0000","last_peered":"2024-06-10T00:54:22.554437+0000","last_clean":"2024-06-10T00:54:22.554437+0000","last_became_active":"2024-06-10T00:52:32.761049+0000","last_became_peered":"2024-06-10T00:52:32.761049+0000","last_unstale":"2024-06-10T00:54:22.554437+0000","last_undegraded":"2024-06-10T00:54:22.554437+0000","last_fullsized":"2024-06-10T00:54:22.554437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:29:38.384451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364495+0000","last_change":"2024-06-10T00:52:35.961299+0000","last_active":"2024-06-10T00:55:20.364495+0000","last_peered":"2024-06-10T00:55:20.364495+0000","last_clean":"2024-06-10T00:55:20.364495+0000","last_became_active":"2024-06-10T00:52:35.960966+0000","last_became_peered":"2024-06-10T00:52:35.960966+0000","last_unstale":"2024-06-10T00:55:20.364495+0000","last_undegraded":"2024-06-10T00:55:20.364495+0000","last_fullsized":"2024-06-10T00:55:20.364495+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:32:21.269689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.559474+0000","last_change":"2024-06-10T00:52:46.002091+0000","last_active":"2024-06-10T00:54:22.559474+0000","last_peered":"2024-06-10T00:54:22.559474+0000","last_clean":"2024-06-10T00:54:22.559474+0000","last_became_active":"2024-06-10T00:52:46.001677+0000","last_became_peered":"2024-06-10T00:52:46.001677+0000","last_unstale":"2024-06-10T00:54:22.559474+0000","last_undegraded":"2024-06-10T00:54:22.559474+0000","last_fullsized":"2024-06-10T00:54:22.559474+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:16:56.405720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'3","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.562287+0000","last_change":"2024-06-10T00:52:32.756765+0000","last_active":"2024-06-10T00:54:22.562287+0000","last_peered":"2024-06-10T00:54:22.562287+0000","last_clean":"2024-06-10T00:54:22.562287+0000","last_became_active":"2024-06-10T00:52:32.756470+0000","last_became_peered":"2024-06-10T00:52:32.756470+0000","last_unstale":"2024-06-10T00:54:22.562287+0000","last_undegraded":"2024-06-10T00:54:22.562287+0000","last_fullsized":"2024-06-10T00:54:22.562287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:29:24.854913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"100'15","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364256+0000","last_change":"2024-06-10T00:52:35.960693+0000","last_active":"2024-06-10T00:55:20.364256+0000","last_peered":"2024-06-10T00:55:20.364256+0000","last_clean":"2024-06-10T00:55:20.364256+0000","last_became_active":"2024-06-10T00:52:35.958397+0000","last_became_peered":"2024-06-10T00:52:35.958397+0000","last_unstale":"2024-06-10T00:55:20.364256+0000","last_undegraded":"2024-06-10T00:55:20.364256+0000","last_fullsized":"2024-06-10T00:55:20.364256+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:23:47.556462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556200+0000","last_change":"2024-06-10T00:52:46.001475+0000","last_active":"2024-06-10T00:54:22.556200+0000","last_peered":"2024-06-10T00:54:22.556200+0000","last_clean":"2024-06-10T00:54:22.556200+0000","last_became_active":"2024-06-10T00:52:46.000983+0000","last_became_peered":"2024-06-10T00:52:46.000983+0000","last_unstale":"2024-06-10T00:54:22.556200+0000","last_undegraded":"2024-06-10T00:54:22.556200+0000","last_fullsized":"2024-06-10T00:54:22.556200+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:23:56.285356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"98'3","reported_seq":231,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:41.572246+0000","last_change":"2024-06-10T00:52:32.756665+0000","last_active":"2024-06-10T00:56:41.572246+0000","last_peered":"2024-06-10T00:56:41.572246+0000","last_clean":"2024-06-10T00:56:41.572246+0000","last_became_active":"2024-06-10T00:52:32.756221+0000","last_became_peered":"2024-06-10T00:52:32.756221+0000","last_unstale":"2024-06-10T00:56:41.572246+0000","last_undegraded":"2024-06-10T00:56:41.572246+0000","last_fullsized":"2024-06-10T00:56:41.572246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:50:34.789063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":185,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"100'6","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364652+0000","last_change":"2024-06-10T00:53:22.125844+0000","last_active":"2024-06-10T00:55:20.364652+0000","last_peered":"2024-06-10T00:55:20.364652+0000","last_clean":"2024-06-10T00:55:20.364652+0000","last_became_active":"2024-06-10T00:53:22.125449+0000","last_became_peered":"2024-06-10T00:53:22.125449+0000","last_unstale":"2024-06-10T00:55:20.364652+0000","last_undegraded":"2024-06-10T00:55:20.364652+0000","last_fullsized":"2024-06-10T00:55:20.364652+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:08:42.936702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553904+0000","last_change":"2024-06-10T00:52:45.999300+0000","last_active":"2024-06-10T00:54:22.553904+0000","last_peered":"2024-06-10T00:54:22.553904+0000","last_clean":"2024-06-10T00:54:22.553904+0000","last_became_active":"2024-06-10T00:52:45.998860+0000","last_became_peered":"2024-06-10T00:52:45.998860+0000","last_unstale":"2024-06-10T00:54:22.553904+0000","last_undegraded":"2024-06-10T00:54:22.553904+0000","last_fullsized":"2024-06-10T00:54:22.553904+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:14:24.936870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":75,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.557182+0000","last_change":"2024-06-10T00:53:22.210710+0000","last_active":"2024-06-10T00:54:22.557182+0000","last_peered":"2024-06-10T00:54:22.557182+0000","last_clean":"2024-06-10T00:54:22.557182+0000","last_became_active":"2024-06-10T00:53:22.202461+0000","last_became_peered":"2024-06-10T00:53:22.202461+0000","last_unstale":"2024-06-10T00:54:22.557182+0000","last_undegraded":"2024-06-10T00:54:22.557182+0000","last_fullsized":"2024-06-10T00:54:22.557182+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:28:14.671776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364560+0000","last_change":"2024-06-10T00:52:35.966960+0000","last_active":"2024-06-10T00:55:20.364560+0000","last_peered":"2024-06-10T00:55:20.364560+0000","last_clean":"2024-06-10T00:55:20.364560+0000","last_became_active":"2024-06-10T00:52:35.966630+0000","last_became_peered":"2024-06-10T00:52:35.966630+0000","last_unstale":"2024-06-10T00:55:20.364560+0000","last_undegraded":"2024-06-10T00:55:20.364560+0000","last_fullsized":"2024-06-10T00:55:20.364560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:04.117405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561586+0000","last_change":"2024-06-10T00:52:45.992955+0000","last_active":"2024-06-10T00:54:22.561586+0000","last_peered":"2024-06-10T00:54:22.561586+0000","last_clean":"2024-06-10T00:54:22.561586+0000","last_became_active":"2024-06-10T00:52:45.992457+0000","last_became_peered":"2024-06-10T00:52:45.992457+0000","last_unstale":"2024-06-10T00:54:22.561586+0000","last_undegraded":"2024-06-10T00:54:22.561586+0000","last_fullsized":"2024-06-10T00:54:22.561586+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:13:49.491183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556768+0000","last_change":"2024-06-10T00:52:32.763794+0000","last_active":"2024-06-10T00:54:22.556768+0000","last_peered":"2024-06-10T00:54:22.556768+0000","last_clean":"2024-06-10T00:54:22.556768+0000","last_became_active":"2024-06-10T00:52:32.763406+0000","last_became_peered":"2024-06-10T00:52:32.763406+0000","last_unstale":"2024-06-10T00:54:22.556768+0000","last_undegraded":"2024-06-10T00:54:22.556768+0000","last_fullsized":"2024-06-10T00:54:22.556768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:50:33.580163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"100'5","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374120+0000","last_change":"2024-06-10T00:53:22.130891+0000","last_active":"2024-06-10T00:55:20.374120+0000","last_peered":"2024-06-10T00:55:20.374120+0000","last_clean":"2024-06-10T00:55:20.374120+0000","last_became_active":"2024-06-10T00:53:22.130508+0000","last_became_peered":"2024-06-10T00:53:22.130508+0000","last_unstale":"2024-06-10T00:55:20.374120+0000","last_undegraded":"2024-06-10T00:55:20.374120+0000","last_fullsized":"2024-06-10T00:55:20.374120+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:28:01.919825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":34,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.563757+0000","last_change":"2024-06-10T00:54:22.563272+0000","last_active":"2024-06-10T00:54:22.563757+0000","last_peered":"2024-06-10T00:54:22.563757+0000","last_clean":"2024-06-10T00:54:22.563757+0000","last_became_active":"2024-06-10T00:54:22.562767+0000","last_became_peered":"2024-06-10T00:54:22.562767+0000","last_unstale":"2024-06-10T00:54:22.563757+0000","last_undegraded":"2024-06-10T00:54:22.563757+0000","last_fullsized":"2024-06-10T00:54:22.563757+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:52:20.739801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:40.422237+0000","last_change":"2024-06-10T00:52:32.960515+0000","last_active":"2024-06-10T00:54:40.422237+0000","last_peered":"2024-06-10T00:54:40.422237+0000","last_clean":"2024-06-10T00:54:40.422237+0000","last_became_active":"2024-06-10T00:52:32.960163+0000","last_became_peered":"2024-06-10T00:52:32.960163+0000","last_unstale":"2024-06-10T00:54:40.422237+0000","last_undegraded":"2024-06-10T00:54:40.422237+0000","last_fullsized":"2024-06-10T00:54:40.422237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:39:51.374998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"100'4","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374185+0000","last_change":"2024-06-10T00:52:35.961884+0000","last_active":"2024-06-10T00:55:20.374185+0000","last_peered":"2024-06-10T00:55:20.374185+0000","last_clean":"2024-06-10T00:55:20.374185+0000","last_became_active":"2024-06-10T00:52:35.961497+0000","last_became_peered":"2024-06-10T00:52:35.961497+0000","last_unstale":"2024-06-10T00:55:20.374185+0000","last_undegraded":"2024-06-10T00:55:20.374185+0000","last_fullsized":"2024-06-10T00:55:20.374185+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:16:09.284184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553591+0000","last_change":"2024-06-10T00:52:45.990846+0000","last_active":"2024-06-10T00:54:22.553591+0000","last_peered":"2024-06-10T00:54:22.553591+0000","last_clean":"2024-06-10T00:54:22.553591+0000","last_became_active":"2024-06-10T00:52:45.990441+0000","last_became_peered":"2024-06-10T00:52:45.990441+0000","last_unstale":"2024-06-10T00:54:22.553591+0000","last_undegraded":"2024-06-10T00:54:22.553591+0000","last_fullsized":"2024-06-10T00:54:22.553591+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:19:35.690378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:39.101650+0000","last_change":"2024-06-10T00:52:32.753913+0000","last_active":"2024-06-10T00:54:39.101650+0000","last_peered":"2024-06-10T00:54:39.101650+0000","last_clean":"2024-06-10T00:54:39.101650+0000","last_became_active":"2024-06-10T00:52:32.753174+0000","last_became_peered":"2024-06-10T00:52:32.753174+0000","last_unstale":"2024-06-10T00:54:39.101650+0000","last_undegraded":"2024-06-10T00:54:39.101650+0000","last_fullsized":"2024-06-10T00:54:39.101650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:13:20.048635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"100'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.359209+0000","last_change":"2024-06-10T00:52:35.965783+0000","last_active":"2024-06-10T00:55:20.359209+0000","last_peered":"2024-06-10T00:55:20.359209+0000","last_clean":"2024-06-10T00:55:20.359209+0000","last_became_active":"2024-06-10T00:52:35.965432+0000","last_became_peered":"2024-06-10T00:52:35.965432+0000","last_unstale":"2024-06-10T00:55:20.359209+0000","last_undegraded":"2024-06-10T00:55:20.359209+0000","last_fullsized":"2024-06-10T00:55:20.359209+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:52:01.240318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553790+0000","last_change":"2024-06-10T00:52:46.241662+0000","last_active":"2024-06-10T00:54:22.553790+0000","last_peered":"2024-06-10T00:54:22.553790+0000","last_clean":"2024-06-10T00:54:22.553790+0000","last_became_active":"2024-06-10T00:52:46.241211+0000","last_became_peered":"2024-06-10T00:52:46.241211+0000","last_unstale":"2024-06-10T00:54:22.553790+0000","last_undegraded":"2024-06-10T00:54:22.553790+0000","last_fullsized":"2024-06-10T00:54:22.553790+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:12.041290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"100'8","reported_seq":24,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.367303+0000","last_change":"2024-06-10T00:53:22.200590+0000","last_active":"2024-06-10T00:55:20.367303+0000","last_peered":"2024-06-10T00:55:20.367303+0000","last_clean":"2024-06-10T00:55:20.367303+0000","last_became_active":"2024-06-10T00:53:22.200046+0000","last_became_peered":"2024-06-10T00:53:22.200046+0000","last_unstale":"2024-06-10T00:55:20.367303+0000","last_undegraded":"2024-06-10T00:55:20.367303+0000","last_fullsized":"2024-06-10T00:55:20.367303+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:43:22.378316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"98'4","reported_seq":231,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:42.573128+0000","last_change":"2024-06-10T00:52:32.959904+0000","last_active":"2024-06-10T00:56:42.573128+0000","last_peered":"2024-06-10T00:56:42.573128+0000","last_clean":"2024-06-10T00:56:42.573128+0000","last_became_active":"2024-06-10T00:52:32.959506+0000","last_became_peered":"2024-06-10T00:52:32.959506+0000","last_unstale":"2024-06-10T00:56:42.573128+0000","last_undegraded":"2024-06-10T00:56:42.573128+0000","last_fullsized":"2024-06-10T00:56:42.573128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:23:15.950345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":182,"num_read_kb":182,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404433+0000","last_change":"2024-06-10T00:52:45.993305+0000","last_active":"2024-06-10T00:54:24.404433+0000","last_peered":"2024-06-10T00:54:24.404433+0000","last_clean":"2024-06-10T00:54:24.404433+0000","last_became_active":"2024-06-10T00:52:45.992682+0000","last_became_peered":"2024-06-10T00:52:45.992682+0000","last_unstale":"2024-06-10T00:54:24.404433+0000","last_undegraded":"2024-06-10T00:54:24.404433+0000","last_fullsized":"2024-06-10T00:54:24.404433+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:49:41.817126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375771+0000","last_change":"2024-06-10T00:52:35.965893+0000","last_active":"2024-06-10T00:55:20.375771+0000","last_peered":"2024-06-10T00:55:20.375771+0000","last_clean":"2024-06-10T00:55:20.375771+0000","last_became_active":"2024-06-10T00:52:35.965426+0000","last_became_peered":"2024-06-10T00:52:35.965426+0000","last_unstale":"2024-06-10T00:55:20.375771+0000","last_undegraded":"2024-06-10T00:55:20.375771+0000","last_fullsized":"2024-06-10T00:55:20.375771+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:29:19.640836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:38.154378+0000","last_change":"2024-06-10T00:52:32.753631+0000","last_active":"2024-06-10T00:54:38.154378+0000","last_peered":"2024-06-10T00:54:38.154378+0000","last_clean":"2024-06-10T00:54:38.154378+0000","last_became_active":"2024-06-10T00:52:32.753260+0000","last_became_peered":"2024-06-10T00:52:32.753260+0000","last_unstale":"2024-06-10T00:54:38.154378+0000","last_undegraded":"2024-06-10T00:54:38.154378+0000","last_fullsized":"2024-06-10T00:54:38.154378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:12:55.932676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":369,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404055+0000","last_change":"2024-06-10T00:51:09.267750+0000","last_active":"2024-06-10T00:54:24.404055+0000","last_peered":"2024-06-10T00:54:24.404055+0000","last_clean":"2024-06-10T00:54:24.404055+0000","last_became_active":"2024-06-10T00:51:09.267246+0000","last_became_peered":"2024-06-10T00:51:09.267246+0000","last_unstale":"2024-06-10T00:54:24.404055+0000","last_undegraded":"2024-06-10T00:54:24.404055+0000","last_fullsized":"2024-06-10T00:54:24.404055+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:47:44.965452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:47:44.965452+0000","last_clean_scrub_stamp":"2024-06-10T00:47:44.965452+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:21.100090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561655+0000","last_change":"2024-06-10T00:52:46.242152+0000","last_active":"2024-06-10T00:54:22.561655+0000","last_peered":"2024-06-10T00:54:22.561655+0000","last_clean":"2024-06-10T00:54:22.561655+0000","last_became_active":"2024-06-10T00:52:46.241542+0000","last_became_peered":"2024-06-10T00:52:46.241542+0000","last_unstale":"2024-06-10T00:54:22.561655+0000","last_undegraded":"2024-06-10T00:54:22.561655+0000","last_fullsized":"2024-06-10T00:54:22.561655+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:55:54.695465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364883+0000","last_change":"2024-06-10T00:52:35.963911+0000","last_active":"2024-06-10T00:55:20.364883+0000","last_peered":"2024-06-10T00:55:20.364883+0000","last_clean":"2024-06-10T00:55:20.364883+0000","last_became_active":"2024-06-10T00:52:35.963571+0000","last_became_peered":"2024-06-10T00:52:35.963571+0000","last_unstale":"2024-06-10T00:55:20.364883+0000","last_undegraded":"2024-06-10T00:55:20.364883+0000","last_fullsized":"2024-06-10T00:55:20.364883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:27:16.757428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.558529+0000","last_change":"2024-06-10T00:53:22.129035+0000","last_active":"2024-06-10T00:54:22.558529+0000","last_peered":"2024-06-10T00:54:22.558529+0000","last_clean":"2024-06-10T00:54:22.558529+0000","last_became_active":"2024-06-10T00:53:22.128561+0000","last_became_peered":"2024-06-10T00:53:22.128561+0000","last_unstale":"2024-06-10T00:54:22.558529+0000","last_undegraded":"2024-06-10T00:54:22.558529+0000","last_fullsized":"2024-06-10T00:54:22.558529+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:02:43.313118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.558735+0000","last_change":"2024-06-10T00:52:45.996907+0000","last_active":"2024-06-10T00:54:22.558735+0000","last_peered":"2024-06-10T00:54:22.558735+0000","last_clean":"2024-06-10T00:54:22.558735+0000","last_became_active":"2024-06-10T00:52:45.996554+0000","last_became_peered":"2024-06-10T00:52:45.996554+0000","last_unstale":"2024-06-10T00:54:22.558735+0000","last_undegraded":"2024-06-10T00:54:22.558735+0000","last_fullsized":"2024-06-10T00:54:22.558735+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:00:46.824071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'1","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:38.584029+0000","last_change":"2024-06-10T00:52:32.960223+0000","last_active":"2024-06-10T00:54:38.584029+0000","last_peered":"2024-06-10T00:54:38.584029+0000","last_clean":"2024-06-10T00:54:38.584029+0000","last_became_active":"2024-06-10T00:52:32.959871+0000","last_became_peered":"2024-06-10T00:52:32.959871+0000","last_unstale":"2024-06-10T00:54:38.584029+0000","last_undegraded":"2024-06-10T00:54:38.584029+0000","last_fullsized":"2024-06-10T00:54:38.584029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:18:17.099140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.369555+0000","last_change":"2024-06-10T00:52:35.962516+0000","last_active":"2024-06-10T00:55:20.369555+0000","last_peered":"2024-06-10T00:55:20.369555+0000","last_clean":"2024-06-10T00:55:20.369555+0000","last_became_active":"2024-06-10T00:52:35.962187+0000","last_became_peered":"2024-06-10T00:52:35.962187+0000","last_unstale":"2024-06-10T00:55:20.369555+0000","last_undegraded":"2024-06-10T00:55:20.369555+0000","last_fullsized":"2024-06-10T00:55:20.369555+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:10:44.295192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601233+0000","last_change":"2024-06-10T00:52:46.241554+0000","last_active":"2024-06-10T00:54:22.601233+0000","last_peered":"2024-06-10T00:54:22.601233+0000","last_clean":"2024-06-10T00:54:22.601233+0000","last_became_active":"2024-06-10T00:52:46.241142+0000","last_became_peered":"2024-06-10T00:52:46.241142+0000","last_unstale":"2024-06-10T00:54:22.601233+0000","last_undegraded":"2024-06-10T00:54:22.601233+0000","last_fullsized":"2024-06-10T00:54:22.601233+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:58:07.233080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'13","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:29.987897+0000","last_change":"2024-06-10T00:52:32.761854+0000","last_active":"2024-06-10T00:56:29.987897+0000","last_peered":"2024-06-10T00:56:29.987897+0000","last_clean":"2024-06-10T00:56:29.987897+0000","last_became_active":"2024-06-10T00:52:32.761522+0000","last_became_peered":"2024-06-10T00:52:32.761522+0000","last_unstale":"2024-06-10T00:56:29.987897+0000","last_undegraded":"2024-06-10T00:56:29.987897+0000","last_fullsized":"2024-06-10T00:56:29.987897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:06:43.269001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.370762+0000","last_change":"2024-06-10T00:52:35.959250+0000","last_active":"2024-06-10T00:55:20.370762+0000","last_peered":"2024-06-10T00:55:20.370762+0000","last_clean":"2024-06-10T00:55:20.370762+0000","last_became_active":"2024-06-10T00:52:35.958925+0000","last_became_peered":"2024-06-10T00:52:35.958925+0000","last_unstale":"2024-06-10T00:55:20.370762+0000","last_undegraded":"2024-06-10T00:55:20.370762+0000","last_fullsized":"2024-06-10T00:55:20.370762+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:11:12.355505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554191+0000","last_change":"2024-06-10T00:52:45.991819+0000","last_active":"2024-06-10T00:54:22.554191+0000","last_peered":"2024-06-10T00:54:22.554191+0000","last_clean":"2024-06-10T00:54:22.554191+0000","last_became_active":"2024-06-10T00:52:45.991408+0000","last_became_peered":"2024-06-10T00:52:45.991408+0000","last_unstale":"2024-06-10T00:54:22.554191+0000","last_undegraded":"2024-06-10T00:54:22.554191+0000","last_fullsized":"2024-06-10T00:54:22.554191+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:16:34.912359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668551+0000","last_change":"2024-06-10T00:52:32.869094+0000","last_active":"2024-06-10T00:54:22.668551+0000","last_peered":"2024-06-10T00:54:22.668551+0000","last_clean":"2024-06-10T00:54:22.668551+0000","last_became_active":"2024-06-10T00:52:32.868664+0000","last_became_peered":"2024-06-10T00:52:32.868664+0000","last_unstale":"2024-06-10T00:54:22.668551+0000","last_undegraded":"2024-06-10T00:54:22.668551+0000","last_fullsized":"2024-06-10T00:54:22.668551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:50:38.051090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374722+0000","last_change":"2024-06-10T00:52:35.952356+0000","last_active":"2024-06-10T00:55:20.374722+0000","last_peered":"2024-06-10T00:55:20.374722+0000","last_clean":"2024-06-10T00:55:20.374722+0000","last_became_active":"2024-06-10T00:52:35.952002+0000","last_became_peered":"2024-06-10T00:52:35.952002+0000","last_unstale":"2024-06-10T00:55:20.374722+0000","last_undegraded":"2024-06-10T00:55:20.374722+0000","last_fullsized":"2024-06-10T00:55:20.374722+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:12:28.581522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668366+0000","last_change":"2024-06-10T00:52:46.241773+0000","last_active":"2024-06-10T00:54:22.668366+0000","last_peered":"2024-06-10T00:54:22.668366+0000","last_clean":"2024-06-10T00:54:22.668366+0000","last_became_active":"2024-06-10T00:52:46.241272+0000","last_became_peered":"2024-06-10T00:52:46.241272+0000","last_unstale":"2024-06-10T00:54:22.668366+0000","last_undegraded":"2024-06-10T00:54:22.668366+0000","last_fullsized":"2024-06-10T00:54:22.668366+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:07:28.337536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561360+0000","last_change":"2024-06-10T00:52:32.960100+0000","last_active":"2024-06-10T00:54:22.561360+0000","last_peered":"2024-06-10T00:54:22.561360+0000","last_clean":"2024-06-10T00:54:22.561360+0000","last_became_active":"2024-06-10T00:52:32.959779+0000","last_became_peered":"2024-06-10T00:52:32.959779+0000","last_unstale":"2024-06-10T00:54:22.561360+0000","last_undegraded":"2024-06-10T00:54:22.561360+0000","last_fullsized":"2024-06-10T00:54:22.561360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:09.555753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'2","reported_seq":40,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.326494+0000","last_change":"2024-06-10T00:52:35.955586+0000","last_active":"2024-06-10T00:55:20.326494+0000","last_peered":"2024-06-10T00:55:20.326494+0000","last_clean":"2024-06-10T00:55:20.326494+0000","last_became_active":"2024-06-10T00:52:35.955216+0000","last_became_peered":"2024-06-10T00:52:35.955216+0000","last_unstale":"2024-06-10T00:55:20.326494+0000","last_undegraded":"2024-06-10T00:55:20.326494+0000","last_fullsized":"2024-06-10T00:55:20.326494+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:23:12.197182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:23.582601+0000","last_change":"2024-06-10T00:54:22.602670+0000","last_active":"2024-06-10T00:54:23.582601+0000","last_peered":"2024-06-10T00:54:23.582601+0000","last_clean":"2024-06-10T00:54:23.582601+0000","last_became_active":"2024-06-10T00:54:22.602218+0000","last_became_peered":"2024-06-10T00:54:22.602218+0000","last_unstale":"2024-06-10T00:54:23.582601+0000","last_undegraded":"2024-06-10T00:54:23.582601+0000","last_fullsized":"2024-06-10T00:54:23.582601+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:07.399025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"98'3","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601358+0000","last_change":"2024-06-10T00:52:32.753444+0000","last_active":"2024-06-10T00:54:22.601358+0000","last_peered":"2024-06-10T00:54:22.601358+0000","last_clean":"2024-06-10T00:54:22.601358+0000","last_became_active":"2024-06-10T00:52:32.753149+0000","last_became_peered":"2024-06-10T00:52:32.753149+0000","last_unstale":"2024-06-10T00:54:22.601358+0000","last_undegraded":"2024-06-10T00:54:22.601358+0000","last_fullsized":"2024-06-10T00:54:22.601358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:28.241950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365900+0000","last_change":"2024-06-10T00:52:35.966718+0000","last_active":"2024-06-10T00:55:20.365900+0000","last_peered":"2024-06-10T00:55:20.365900+0000","last_clean":"2024-06-10T00:55:20.365900+0000","last_became_active":"2024-06-10T00:52:35.965907+0000","last_became_peered":"2024-06-10T00:52:35.965907+0000","last_unstale":"2024-06-10T00:55:20.365900+0000","last_undegraded":"2024-06-10T00:55:20.365900+0000","last_fullsized":"2024-06-10T00:55:20.365900+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:21:34.261306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668092+0000","last_change":"2024-06-10T00:52:45.991034+0000","last_active":"2024-06-10T00:54:22.668092+0000","last_peered":"2024-06-10T00:54:22.668092+0000","last_clean":"2024-06-10T00:54:22.668092+0000","last_became_active":"2024-06-10T00:52:45.990546+0000","last_became_peered":"2024-06-10T00:52:45.990546+0000","last_unstale":"2024-06-10T00:54:22.668092+0000","last_undegraded":"2024-06-10T00:54:22.668092+0000","last_fullsized":"2024-06-10T00:54:22.668092+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:09:23.412922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"100'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:42.999786+0000","last_change":"2024-06-10T00:52:32.764024+0000","last_active":"2024-06-10T00:54:42.999786+0000","last_peered":"2024-06-10T00:54:42.999786+0000","last_clean":"2024-06-10T00:54:42.999786+0000","last_became_active":"2024-06-10T00:52:32.763449+0000","last_became_peered":"2024-06-10T00:52:32.763449+0000","last_unstale":"2024-06-10T00:54:42.999786+0000","last_undegraded":"2024-06-10T00:54:42.999786+0000","last_fullsized":"2024-06-10T00:54:42.999786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:50:23.605873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373684+0000","last_change":"2024-06-10T00:52:35.955570+0000","last_active":"2024-06-10T00:55:20.373684+0000","last_peered":"2024-06-10T00:55:20.373684+0000","last_clean":"2024-06-10T00:55:20.373684+0000","last_became_active":"2024-06-10T00:52:35.955248+0000","last_became_peered":"2024-06-10T00:52:35.955248+0000","last_unstale":"2024-06-10T00:55:20.373684+0000","last_undegraded":"2024-06-10T00:55:20.373684+0000","last_fullsized":"2024-06-10T00:55:20.373684+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:26.124499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668028+0000","last_change":"2024-06-10T00:52:46.241776+0000","last_active":"2024-06-10T00:54:22.668028+0000","last_peered":"2024-06-10T00:54:22.668028+0000","last_clean":"2024-06-10T00:54:22.668028+0000","last_became_active":"2024-06-10T00:52:46.241276+0000","last_became_peered":"2024-06-10T00:52:46.241276+0000","last_unstale":"2024-06-10T00:54:22.668028+0000","last_undegraded":"2024-06-10T00:54:22.668028+0000","last_fullsized":"2024-06-10T00:54:22.668028+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:08.027938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":58,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556321+0000","last_change":"2024-06-10T00:53:22.204034+0000","last_active":"2024-06-10T00:54:22.556321+0000","last_peered":"2024-06-10T00:54:22.556321+0000","last_clean":"2024-06-10T00:54:22.556321+0000","last_became_active":"2024-06-10T00:53:22.199988+0000","last_became_peered":"2024-06-10T00:53:22.199988+0000","last_unstale":"2024-06-10T00:54:22.556321+0000","last_undegraded":"2024-06-10T00:54:22.556321+0000","last_fullsized":"2024-06-10T00:54:22.556321+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:39:54.012360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373708+0000","last_change":"2024-06-10T00:52:35.959182+0000","last_active":"2024-06-10T00:55:20.373708+0000","last_peered":"2024-06-10T00:55:20.373708+0000","last_clean":"2024-06-10T00:55:20.373708+0000","last_became_active":"2024-06-10T00:52:35.958828+0000","last_became_peered":"2024-06-10T00:52:35.958828+0000","last_unstale":"2024-06-10T00:55:20.373708+0000","last_undegraded":"2024-06-10T00:55:20.373708+0000","last_fullsized":"2024-06-10T00:55:20.373708+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:29:58.857659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561603+0000","last_change":"2024-06-10T00:52:45.989739+0000","last_active":"2024-06-10T00:54:22.561603+0000","last_peered":"2024-06-10T00:54:22.561603+0000","last_clean":"2024-06-10T00:54:22.561603+0000","last_became_active":"2024-06-10T00:52:45.989279+0000","last_became_peered":"2024-06-10T00:52:45.989279+0000","last_unstale":"2024-06-10T00:54:22.561603+0000","last_undegraded":"2024-06-10T00:54:22.561603+0000","last_fullsized":"2024-06-10T00:54:22.561603+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:19:17.551947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554798+0000","last_change":"2024-06-10T00:52:32.959662+0000","last_active":"2024-06-10T00:54:22.554798+0000","last_peered":"2024-06-10T00:54:22.554798+0000","last_clean":"2024-06-10T00:54:22.554798+0000","last_became_active":"2024-06-10T00:52:32.959286+0000","last_became_peered":"2024-06-10T00:52:32.959286+0000","last_unstale":"2024-06-10T00:54:22.554798+0000","last_undegraded":"2024-06-10T00:54:22.554798+0000","last_fullsized":"2024-06-10T00:54:22.554798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:26:04.361567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373843+0000","last_change":"2024-06-10T00:52:35.960137+0000","last_active":"2024-06-10T00:55:20.373843+0000","last_peered":"2024-06-10T00:55:20.373843+0000","last_clean":"2024-06-10T00:55:20.373843+0000","last_became_active":"2024-06-10T00:52:35.959584+0000","last_became_peered":"2024-06-10T00:52:35.959584+0000","last_unstale":"2024-06-10T00:55:20.373843+0000","last_undegraded":"2024-06-10T00:55:20.373843+0000","last_fullsized":"2024-06-10T00:55:20.373843+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:23:38.702685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878137+0000","last_change":"2024-06-10T00:52:46.002268+0000","last_active":"2024-06-10T00:54:22.878137+0000","last_peered":"2024-06-10T00:54:22.878137+0000","last_clean":"2024-06-10T00:54:22.878137+0000","last_became_active":"2024-06-10T00:52:46.001769+0000","last_became_peered":"2024-06-10T00:52:46.001769+0000","last_unstale":"2024-06-10T00:54:22.878137+0000","last_undegraded":"2024-06-10T00:54:22.878137+0000","last_fullsized":"2024-06-10T00:54:22.878137+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:33:51.365951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561629+0000","last_change":"2024-06-10T00:52:32.757112+0000","last_active":"2024-06-10T00:54:22.561629+0000","last_peered":"2024-06-10T00:54:22.561629+0000","last_clean":"2024-06-10T00:54:22.561629+0000","last_became_active":"2024-06-10T00:52:32.755589+0000","last_became_peered":"2024-06-10T00:52:32.755589+0000","last_unstale":"2024-06-10T00:54:22.561629+0000","last_undegraded":"2024-06-10T00:54:22.561629+0000","last_fullsized":"2024-06-10T00:54:22.561629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:15:12.814489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364097+0000","last_change":"2024-06-10T00:52:35.955925+0000","last_active":"2024-06-10T00:55:20.364097+0000","last_peered":"2024-06-10T00:55:20.364097+0000","last_clean":"2024-06-10T00:55:20.364097+0000","last_became_active":"2024-06-10T00:52:35.955556+0000","last_became_peered":"2024-06-10T00:52:35.955556+0000","last_unstale":"2024-06-10T00:55:20.364097+0000","last_undegraded":"2024-06-10T00:55:20.364097+0000","last_fullsized":"2024-06-10T00:55:20.364097+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:42:18.324484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561041+0000","last_change":"2024-06-10T00:52:45.988810+0000","last_active":"2024-06-10T00:54:22.561041+0000","last_peered":"2024-06-10T00:54:22.561041+0000","last_clean":"2024-06-10T00:54:22.561041+0000","last_became_active":"2024-06-10T00:52:45.988041+0000","last_became_peered":"2024-06-10T00:52:45.988041+0000","last_unstale":"2024-06-10T00:54:22.561041+0000","last_undegraded":"2024-06-10T00:54:22.561041+0000","last_fullsized":"2024-06-10T00:54:22.561041+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:14:08.979802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553851+0000","last_change":"2024-06-10T00:52:32.765493+0000","last_active":"2024-06-10T00:54:22.553851+0000","last_peered":"2024-06-10T00:54:22.553851+0000","last_clean":"2024-06-10T00:54:22.553851+0000","last_became_active":"2024-06-10T00:52:32.765193+0000","last_became_peered":"2024-06-10T00:52:32.765193+0000","last_unstale":"2024-06-10T00:54:22.553851+0000","last_undegraded":"2024-06-10T00:54:22.553851+0000","last_fullsized":"2024-06-10T00:54:22.553851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:38:11.554673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374319+0000","last_change":"2024-06-10T00:52:35.960216+0000","last_active":"2024-06-10T00:55:20.374319+0000","last_peered":"2024-06-10T00:55:20.374319+0000","last_clean":"2024-06-10T00:55:20.374319+0000","last_became_active":"2024-06-10T00:52:35.959664+0000","last_became_peered":"2024-06-10T00:52:35.959664+0000","last_unstale":"2024-06-10T00:55:20.374319+0000","last_undegraded":"2024-06-10T00:55:20.374319+0000","last_fullsized":"2024-06-10T00:55:20.374319+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:41:58.086330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375612+0000","last_change":"2024-06-10T00:52:35.967043+0000","last_active":"2024-06-10T00:55:20.375612+0000","last_peered":"2024-06-10T00:55:20.375612+0000","last_clean":"2024-06-10T00:55:20.375612+0000","last_became_active":"2024-06-10T00:52:35.966366+0000","last_became_peered":"2024-06-10T00:52:35.966366+0000","last_unstale":"2024-06-10T00:55:20.375612+0000","last_undegraded":"2024-06-10T00:55:20.375612+0000","last_fullsized":"2024-06-10T00:55:20.375612+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:27:06.047113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"100'12","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.369745+0000","last_change":"2024-06-10T00:52:35.960517+0000","last_active":"2024-06-10T00:55:20.369745+0000","last_peered":"2024-06-10T00:55:20.369745+0000","last_clean":"2024-06-10T00:55:20.369745+0000","last_became_active":"2024-06-10T00:52:35.957895+0000","last_became_peered":"2024-06-10T00:52:35.957895+0000","last_unstale":"2024-06-10T00:55:20.369745+0000","last_undegraded":"2024-06-10T00:55:20.369745+0000","last_fullsized":"2024-06-10T00:55:20.369745+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:04:34.438070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.369777+0000","last_change":"2024-06-10T00:52:35.962163+0000","last_active":"2024-06-10T00:55:20.369777+0000","last_peered":"2024-06-10T00:55:20.369777+0000","last_clean":"2024-06-10T00:55:20.369777+0000","last_became_active":"2024-06-10T00:52:35.961768+0000","last_became_peered":"2024-06-10T00:52:35.961768+0000","last_unstale":"2024-06-10T00:55:20.369777+0000","last_undegraded":"2024-06-10T00:55:20.369777+0000","last_fullsized":"2024-06-10T00:55:20.369777+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:35.793995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"100'7","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373779+0000","last_change":"2024-06-10T00:53:22.130686+0000","last_active":"2024-06-10T00:55:20.373779+0000","last_peered":"2024-06-10T00:55:20.373779+0000","last_clean":"2024-06-10T00:55:20.373779+0000","last_became_active":"2024-06-10T00:53:22.130198+0000","last_became_peered":"2024-06-10T00:53:22.130198+0000","last_unstale":"2024-06-10T00:55:20.373779+0000","last_undegraded":"2024-06-10T00:55:20.373779+0000","last_fullsized":"2024-06-10T00:55:20.373779+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:50:26.773977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372642+0000","last_change":"2024-06-10T00:52:35.957255+0000","last_active":"2024-06-10T00:55:20.372642+0000","last_peered":"2024-06-10T00:55:20.372642+0000","last_clean":"2024-06-10T00:55:20.372642+0000","last_became_active":"2024-06-10T00:52:35.956923+0000","last_became_peered":"2024-06-10T00:52:35.956923+0000","last_unstale":"2024-06-10T00:55:20.372642+0000","last_undegraded":"2024-06-10T00:55:20.372642+0000","last_fullsized":"2024-06-10T00:55:20.372642+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:33:25.743503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375063+0000","last_change":"2024-06-10T00:52:35.964258+0000","last_active":"2024-06-10T00:55:20.375063+0000","last_peered":"2024-06-10T00:55:20.375063+0000","last_clean":"2024-06-10T00:55:20.375063+0000","last_became_active":"2024-06-10T00:52:35.963862+0000","last_became_peered":"2024-06-10T00:52:35.963862+0000","last_unstale":"2024-06-10T00:55:20.375063+0000","last_undegraded":"2024-06-10T00:55:20.375063+0000","last_fullsized":"2024-06-10T00:55:20.375063+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:48:58.554411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.377085+0000","last_change":"2024-06-10T00:52:35.965046+0000","last_active":"2024-06-10T00:55:20.377085+0000","last_peered":"2024-06-10T00:55:20.377085+0000","last_clean":"2024-06-10T00:55:20.377085+0000","last_became_active":"2024-06-10T00:52:35.964496+0000","last_became_peered":"2024-06-10T00:52:35.964496+0000","last_unstale":"2024-06-10T00:55:20.377085+0000","last_undegraded":"2024-06-10T00:55:20.377085+0000","last_fullsized":"2024-06-10T00:55:20.377085+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:09:42.979731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"100'2","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.370981+0000","last_change":"2024-06-10T00:52:35.964287+0000","last_active":"2024-06-10T00:55:20.370981+0000","last_peered":"2024-06-10T00:55:20.370981+0000","last_clean":"2024-06-10T00:55:20.370981+0000","last_became_active":"2024-06-10T00:52:35.963920+0000","last_became_peered":"2024-06-10T00:52:35.963920+0000","last_unstale":"2024-06-10T00:55:20.370981+0000","last_undegraded":"2024-06-10T00:55:20.370981+0000","last_fullsized":"2024-06-10T00:55:20.370981+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:41:54.593274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.367606+0000","last_change":"2024-06-10T00:52:35.961440+0000","last_active":"2024-06-10T00:55:20.367606+0000","last_peered":"2024-06-10T00:55:20.367606+0000","last_clean":"2024-06-10T00:55:20.367606+0000","last_became_active":"2024-06-10T00:52:35.960066+0000","last_became_peered":"2024-06-10T00:52:35.960066+0000","last_unstale":"2024-06-10T00:55:20.367606+0000","last_undegraded":"2024-06-10T00:55:20.367606+0000","last_fullsized":"2024-06-10T00:55:20.367606+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:01.024596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372367+0000","last_change":"2024-06-10T00:52:35.964464+0000","last_active":"2024-06-10T00:55:20.372367+0000","last_peered":"2024-06-10T00:55:20.372367+0000","last_clean":"2024-06-10T00:55:20.372367+0000","last_became_active":"2024-06-10T00:52:35.964159+0000","last_became_peered":"2024-06-10T00:52:35.964159+0000","last_unstale":"2024-06-10T00:55:20.372367+0000","last_undegraded":"2024-06-10T00:55:20.372367+0000","last_fullsized":"2024-06-10T00:55:20.372367+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:18.660859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.377213+0000","last_change":"2024-06-10T00:52:35.961539+0000","last_active":"2024-06-10T00:55:20.377213+0000","last_peered":"2024-06-10T00:55:20.377213+0000","last_clean":"2024-06-10T00:55:20.377213+0000","last_became_active":"2024-06-10T00:52:35.958738+0000","last_became_peered":"2024-06-10T00:52:35.958738+0000","last_unstale":"2024-06-10T00:55:20.377213+0000","last_undegraded":"2024-06-10T00:55:20.377213+0000","last_fullsized":"2024-06-10T00:55:20.377213+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:06:48.057551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374134+0000","last_change":"2024-06-10T00:52:35.962563+0000","last_active":"2024-06-10T00:55:20.374134+0000","last_peered":"2024-06-10T00:55:20.374134+0000","last_clean":"2024-06-10T00:55:20.374134+0000","last_became_active":"2024-06-10T00:52:35.962255+0000","last_became_peered":"2024-06-10T00:52:35.962255+0000","last_unstale":"2024-06-10T00:55:20.374134+0000","last_undegraded":"2024-06-10T00:55:20.374134+0000","last_fullsized":"2024-06-10T00:55:20.374134+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:33:35.689446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372437+0000","last_change":"2024-06-10T00:52:35.959616+0000","last_active":"2024-06-10T00:55:20.372437+0000","last_peered":"2024-06-10T00:55:20.372437+0000","last_clean":"2024-06-10T00:55:20.372437+0000","last_became_active":"2024-06-10T00:52:35.959251+0000","last_became_peered":"2024-06-10T00:52:35.959251+0000","last_unstale":"2024-06-10T00:55:20.372437+0000","last_undegraded":"2024-06-10T00:55:20.372437+0000","last_fullsized":"2024-06-10T00:55:20.372437+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:39:23.933279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284806,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":82322,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1135,"num_read_kb":658,"num_write":191,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":246966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":163,"ondisk_log_size":163,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351042,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":573101,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612680,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28248,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716904,"statfs":{"total":95995035648,"available":95966109696,"internally_reserved":0,"allocated":1261568,"data_stored":1238242,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776205,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":577433,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037841,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":573257,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299479,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36476,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708676,"statfs":{"total":95995035648,"available":95957684224,"internally_reserved":0,"allocated":1167360,"data_stored":1169614,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463003,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":568513,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36488,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708664,"statfs":{"total":95995035648,"available":95957671936,"internally_reserved":0,"allocated":1179648,"data_stored":646296,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738490,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":569901,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411440,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32344,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712808,"statfs":{"total":95995035648,"available":95961915392,"internally_reserved":0,"allocated":1196032,"data_stored":648373,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149802,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":568707,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888165,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":573275,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724640,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":1165202,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":77873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":79950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":79451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T00:56:48.399 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-10T00:56:48.513 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:56:50.249 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:50 smithi037 ceph-mon[32192]: pgmap v434: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:56:50.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.248+0000 7f366807d700 1 -- 172.21.15.37:0/634576426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3660108500 msgr2=0x7f36601088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:50.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.248+0000 7f366807d700 1 --2- 172.21.15.37:0/634576426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3660108500 0x7f36601088e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f365000b0a0 tx=0x7f365000b3b0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.248+0000 7f366807d700 1 -- 172.21.15.37:0/634576426 shutdown_connections 2024-06-10T00:56:50.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.248+0000 7f366807d700 1 --2- 172.21.15.37:0/634576426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f366006ab30 0x7f366010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.248+0000 7f366807d700 1 --2- 172.21.15.37:0/634576426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3660108500 0x7f36601088e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.248+0000 7f366807d700 1 --2- 172.21.15.37:0/634576426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f366006a170 0x7f366006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.248+0000 7f366807d700 1 -- 172.21.15.37:0/634576426 >> 172.21.15.37:0/634576426 conn(0x7f3660078340 msgr2=0x7f3660078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:50.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.248+0000 7f366807d700 1 -- 172.21.15.37:0/634576426 shutdown_connections 2024-06-10T00:56:50.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.249+0000 7f366807d700 1 -- 172.21.15.37:0/634576426 wait complete. 2024-06-10T00:56:50.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.249+0000 7f366807d700 1 Processor -- start 2024-06-10T00:56:50.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.249+0000 7f366807d700 1 -- start start 2024-06-10T00:56:50.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.250+0000 7f366807d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f366006a170 0x7f3660101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:50.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.250+0000 7f366807d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f366006ab30 0x7f36601019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:50.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.250+0000 7f366807d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3660108500 0x7f3660101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:50.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.250+0000 7f366807d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3660112760 con 0x7f366006a170 2024-06-10T00:56:50.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.250+0000 7f366807d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f36601125e0 con 0x7f366006ab30 2024-06-10T00:56:50.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.250+0000 7f366807d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f36601128e0 con 0x7f3660108500 2024-06-10T00:56:50.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.250+0000 7f3665e19700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f366006a170 0x7f3660101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:50.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.250+0000 7f3665e19700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f366006a170 0x7f3660101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59472/0 (socket says 172.21.15.37:59472) 2024-06-10T00:56:50.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.250+0000 7f3665e19700 1 -- 172.21.15.37:0/2821561628 learned_addr learned my addr 172.21.15.37:0/2821561628 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:50.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.250+0000 7f366661a700 1 --2- 172.21.15.37:0/2821561628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3660108500 0x7f3660101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:50.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.250+0000 7f3665618700 1 --2- 172.21.15.37:0/2821561628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f366006ab30 0x7f36601019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:50.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.250+0000 7f3665e19700 1 -- 172.21.15.37:0/2821561628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3660108500 msgr2=0x7f3660101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:50.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.250+0000 7f3665e19700 1 --2- 172.21.15.37:0/2821561628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3660108500 0x7f3660101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.250+0000 7f3665e19700 1 -- 172.21.15.37:0/2821561628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f366006ab30 msgr2=0x7f36601019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:50.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.250+0000 7f3665e19700 1 --2- 172.21.15.37:0/2821561628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f366006ab30 0x7f36601019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.250+0000 7f3665e19700 1 -- 172.21.15.37:0/2821561628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36601a7110 con 0x7f366006a170 2024-06-10T00:56:50.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.250+0000 7f3665618700 1 --2- 172.21.15.37:0/2821561628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f366006ab30 0x7f36601019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:56:50.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.250+0000 7f366661a700 1 --2- 172.21.15.37:0/2821561628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3660108500 0x7f3660101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:56:50.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.250+0000 7f3665e19700 1 --2- 172.21.15.37:0/2821561628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f366006a170 0x7f3660101460 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f365000b070 tx=0x7f3650004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.250+0000 7f365effd700 1 -- 172.21.15.37:0/2821561628 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3650021070 con 0x7f366006a170 2024-06-10T00:56:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.250+0000 7f365effd700 1 -- 172.21.15.37:0/2821561628 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f365001d930 con 0x7f366006a170 2024-06-10T00:56:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.251+0000 7f366807d700 1 -- 172.21.15.37:0/2821561628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36601a73a0 con 0x7f366006a170 2024-06-10T00:56:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.251+0000 7f365effd700 1 -- 172.21.15.37:0/2821561628 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3650005440 con 0x7f366006a170 2024-06-10T00:56:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.251+0000 7f366807d700 1 -- 172.21.15.37:0/2821561628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36601a78a0 con 0x7f366006a170 2024-06-10T00:56:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.253+0000 7f365effd700 1 -- 172.21.15.37:0/2821561628 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f3650024070 con 0x7f366006a170 2024-06-10T00:56:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.253+0000 7f365effd700 1 --2- 172.21.15.37:0/2821561628 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f36480a2d30 0x7f36480a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.254+0000 7f3665618700 1 --2- 172.21.15.37:0/2821561628 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f36480a2d30 0x7f36480a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.254+0000 7f365effd700 1 -- 172.21.15.37:0/2821561628 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f365000f020 con 0x7f366006a170 2024-06-10T00:56:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.254+0000 7f3665618700 1 --2- 172.21.15.37:0/2821561628 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f36480a2d30 0x7f36480a51f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3654005fd0 tx=0x7f3654005ee0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.254+0000 7f3672900b80 1 Processor -- start 2024-06-10T00:56:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.255+0000 7f3672900b80 1 -- start start 2024-06-10T00:56:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.255+0000 7f3672900b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be5deb8640 0x55be5deb8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.255+0000 7f3672900b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be5deb8f60 0x55be5deb29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.255+0000 7f3672900b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be5deb2f20 0x55be5deb3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.255+0000 7f3672900b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55be5de04810 con 0x55be5deb8640 2024-06-10T00:56:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.255+0000 7f3672900b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55be5ddac720 con 0x55be5deb2f20 2024-06-10T00:56:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.255+0000 7f3672900b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55be5de007d0 con 0x55be5deb8f60 2024-06-10T00:56:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.255+0000 7f3665e19700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be5deb8640 0x55be5deb8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.255+0000 7f3665e19700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be5deb8640 0x55be5deb8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59482/0 (socket says 172.21.15.37:59482) 2024-06-10T00:56:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.255+0000 7f3665e19700 1 -- 172.21.15.37:0/1415006981 learned_addr learned my addr 172.21.15.37:0/1415006981 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.255+0000 7f366661a700 1 --2- 172.21.15.37:0/1415006981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be5deb2f20 0x55be5deb3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.255+0000 7f3665618700 1 --2- 172.21.15.37:0/1415006981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be5deb8f60 0x55be5deb29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.255+0000 7f3665e19700 1 -- 172.21.15.37:0/1415006981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be5deb8f60 msgr2=0x55be5deb29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.255+0000 7f3665e19700 1 --2- 172.21.15.37:0/1415006981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be5deb8f60 0x55be5deb29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.255+0000 7f3665e19700 1 -- 172.21.15.37:0/1415006981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be5deb2f20 msgr2=0x55be5deb3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:50.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.255+0000 7f3665e19700 1 --2- 172.21.15.37:0/1415006981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be5deb2f20 0x55be5deb3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.255+0000 7f3665e19700 1 -- 172.21.15.37:0/1415006981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be5dda2f50 con 0x55be5deb8640 2024-06-10T00:56:50.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.255+0000 7f3665e19700 1 --2- 172.21.15.37:0/1415006981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be5deb8640 0x55be5deb8a20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3650000f80 tx=0x7f365009cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607056de5cee6156 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:50.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.256+0000 7f3647fff700 1 -- 172.21.15.37:0/1415006981 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36500a5d90 con 0x55be5deb8640 2024-06-10T00:56:50.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.256+0000 7f3647fff700 1 -- 172.21.15.37:0/1415006981 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f365009b4b0 con 0x55be5deb8640 2024-06-10T00:56:50.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.256+0000 7f3647fff700 1 -- 172.21.15.37:0/1415006981 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36500bd200 con 0x55be5deb8640 2024-06-10T00:56:50.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.256+0000 7f3672900b80 1 -- 172.21.15.37:0/1415006981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be5deb8640 msgr2=0x55be5deb8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:50.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.256+0000 7f3672900b80 1 --2- 172.21.15.37:0/1415006981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be5deb8640 0x55be5deb8a20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3650000f80 tx=0x7f365009cc10 comp rx=0 tx=0).stop 2024-06-10T00:56:50.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.256+0000 7f3672900b80 1 -- 172.21.15.37:0/1415006981 shutdown_connections 2024-06-10T00:56:50.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.256+0000 7f3672900b80 1 --2- 172.21.15.37:0/1415006981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be5deb2f20 0x55be5deb3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.256+0000 7f3672900b80 1 --2- 172.21.15.37:0/1415006981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be5deb8640 0x55be5deb8a20 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.256+0000 7f3672900b80 1 --2- 172.21.15.37:0/1415006981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be5deb8f60 0x55be5deb29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.256+0000 7f3672900b80 1 -- 172.21.15.37:0/1415006981 >> 172.21.15.37:0/1415006981 conn(0x55be5ddddcc0 msgr2=0x55be5dead850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:50.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.257+0000 7f3672900b80 1 -- 172.21.15.37:0/1415006981 shutdown_connections 2024-06-10T00:56:50.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.257+0000 7f3672900b80 1 -- 172.21.15.37:0/1415006981 wait complete. 2024-06-10T00:56:50.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.257+0000 7f3672900b80 1 -- start start 2024-06-10T00:56:50.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.257+0000 7f3672900b80 1 Processor -- start 2024-06-10T00:56:50.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.258+0000 7f3672900b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be5deb2f20 0x55be5debecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:50.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.258+0000 7f3672900b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be5deb8640 0x55be5debbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:50.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.258+0000 7f3672900b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be5deb8f60 0x55be5debc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:50.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.258+0000 7f3672900b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55be5debf6f0 con 0x55be5deb8640 2024-06-10T00:56:50.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.258+0000 7f3672900b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55be5dda4bf0 con 0x55be5deb8f60 2024-06-10T00:56:50.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.258+0000 7f3672900b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55be5debf870 con 0x55be5deb2f20 2024-06-10T00:56:50.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.258+0000 7f3672900b80 10 client.0 _ll_register_callbacks cb 0x55be5dd9e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:56:50.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.258+0000 7f3665618700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be5deb8640 0x55be5debbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:50.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.258+0000 7f3665618700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be5deb8640 0x55be5debbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59494/0 (socket says 172.21.15.37:59494) 2024-06-10T00:56:50.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.258+0000 7f3665618700 1 -- 172.21.15.37:0/4217350949 learned_addr learned my addr 172.21.15.37:0/4217350949 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:50.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.258+0000 7f3665e19700 1 --2- 172.21.15.37:0/4217350949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be5deb2f20 0x55be5debecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:50.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.258+0000 7f366661a700 1 --2- 172.21.15.37:0/4217350949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be5deb8f60 0x55be5debc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:50.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.258+0000 7f3665618700 1 -- 172.21.15.37:0/4217350949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be5deb2f20 msgr2=0x55be5debecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:50.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.258+0000 7f3665618700 1 --2- 172.21.15.37:0/4217350949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be5deb2f20 0x55be5debecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.258+0000 7f3665618700 1 -- 172.21.15.37:0/4217350949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be5deb8f60 msgr2=0x55be5debc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:50.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.258+0000 7f3665618700 1 --2- 172.21.15.37:0/4217350949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be5deb8f60 0x55be5debc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.258+0000 7f3665618700 1 -- 172.21.15.37:0/4217350949 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be5dd538d0 con 0x55be5deb8640 2024-06-10T00:56:50.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.258+0000 7f3665618700 1 --2- 172.21.15.37:0/4217350949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be5deb8640 0x55be5debbfd0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3654011720 tx=0x7f36540047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48384dc93015a7c7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:50.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.258+0000 7f36457fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:56:50.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.258+0000 7f36457fa700 1 -- 172.21.15.37:0/4217350949 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3654023070 con 0x55be5deb8640 2024-06-10T00:56:50.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.258+0000 7f36457fa700 1 -- 172.21.15.37:0/4217350949 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f3654004220 con 0x55be5deb8640 2024-06-10T00:56:50.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.258+0000 7f3672900b80 1 -- 172.21.15.37:0/4217350949 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f365009b4b0 con 0x55be5deb8640 2024-06-10T00:56:50.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.259+0000 7f36457fa700 1 -- 172.21.15.37:0/4217350949 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f365402e050 con 0x55be5deb8640 2024-06-10T00:56:50.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.259+0000 7f3672900b80 10 client.15333 fetch_fsmap learned FSMap version 27 2024-06-10T00:56:50.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.259+0000 7f3672900b80 1 -- 172.21.15.37:0/4217350949 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55be5dec9fd0 con 0x55be5deb8640 2024-06-10T00:56:50.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.259+0000 7f36457fa700 1 -- 172.21.15.37:0/4217350949 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f365009b4b0 con 0x55be5deb8640 2024-06-10T00:56:50.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.259+0000 7f36457fa700 1 -- 172.21.15.37:0/4217350949 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f365400f040 con 0x55be5deb8640 2024-06-10T00:56:50.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.259+0000 7f3672900b80 10 client.15333 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:56:50.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.259+0000 7f3672900b80 10 client.15333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T00:56:50.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.259+0000 7f3672900b80 20 client.15333 populate_metadata read hostname 'smithi037' 2024-06-10T00:56:50.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.259+0000 7f3672900b80 1 --2- 172.21.15.37:0/4217350949 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55be5decc5b0 0x55be5dece9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:50.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.259+0000 7f3672900b80 4 client.15333 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:56:50.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.259+0000 7f3672900b80 1 -- 172.21.15.37:0/4217350949 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be5de11d10 con 0x55be5decc5b0 2024-06-10T00:56:50.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.259+0000 7f3665e19700 1 --2- 172.21.15.37:0/4217350949 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55be5decc5b0 0x55be5dece9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:50.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.260+0000 7f3665e19700 1 --2- 172.21.15.37:0/4217350949 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55be5decc5b0 0x55be5dece9d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5cea21a6ab1c68d server_cookie=d3a7a3f5775f6f0f in_seq=0 out_seq=0 2024-06-10T00:56:50.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.260+0000 7f36457fa700 10 client.15333 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:56:50.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.261+0000 7f36457fa700 1 -- 172.21.15.37:0/4217350949 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55be5de11d10 con 0x55be5decc5b0 2024-06-10T00:56:50.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.261+0000 7f36457fa700 10 client.15333 handle_command_reply: tid=0 2024-06-10T00:56:50.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.261+0000 7f3672900b80 1 client.15333 shutdown 2024-06-10T00:56:50.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.261+0000 7f3672900b80 1 -- 172.21.15.37:0/4217350949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be5deb8640 msgr2=0x55be5debbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:50.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.261+0000 7f3672900b80 1 --2- 172.21.15.37:0/4217350949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be5deb8640 0x55be5debbfd0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3654011720 tx=0x7f36540047c0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.261+0000 7f3672900b80 1 -- 172.21.15.37:0/4217350949 shutdown_connections 2024-06-10T00:56:50.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.261+0000 7f3672900b80 1 --2- 172.21.15.37:0/4217350949 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55be5decc5b0 0x55be5dece9d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.261+0000 7f3672900b80 1 --2- 172.21.15.37:0/4217350949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be5deb8f60 0x55be5debc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.261+0000 7f3672900b80 1 --2- 172.21.15.37:0/4217350949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be5deb2f20 0x55be5debecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.261+0000 7f3672900b80 1 --2- 172.21.15.37:0/4217350949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be5deb8640 0x55be5debbfd0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.261+0000 7f36457fa700 0 client.15333 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:56:50.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.261+0000 7f3672900b80 1 -- 172.21.15.37:0/4217350949 >> 172.21.15.37:0/4217350949 conn(0x55be5ddddcc0 msgr2=0x55be5dead5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:50.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.261+0000 7f3672900b80 1 -- 172.21.15.37:0/4217350949 shutdown_connections 2024-06-10T00:56:50.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.261+0000 7f3672900b80 1 -- 172.21.15.37:0/4217350949 wait complete. 2024-06-10T00:56:50.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.261+0000 7f3672900b80 20 client.15333 trim_cache size 0 max 16384 2024-06-10T00:56:50.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.282+0000 7f3672900b80 1 Processor -- start 2024-06-10T00:56:50.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.282+0000 7f3672900b80 1 -- start start 2024-06-10T00:56:50.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.282+0000 7f3672900b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be5deb2f20 0x55be5deb29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:50.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.282+0000 7f3672900b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be5deb8640 0x55be5dea7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:50.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.282+0000 7f3672900b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be5deb8f60 0x55be5ded4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:50.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.282+0000 7f3672900b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f36601b2f20 con 0x55be5deb2f20 2024-06-10T00:56:50.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.282+0000 7f3672900b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f36601b07b0 con 0x55be5deb8640 2024-06-10T00:56:50.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.282+0000 7f3672900b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f36601ae600 con 0x55be5deb8f60 2024-06-10T00:56:50.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.282+0000 7f3665e19700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be5deb2f20 0x55be5deb29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:50.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.282+0000 7f3665e19700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be5deb2f20 0x55be5deb29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59496/0 (socket says 172.21.15.37:59496) 2024-06-10T00:56:50.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.282+0000 7f3665e19700 1 -- 172.21.15.37:0/777084532 learned_addr learned my addr 172.21.15.37:0/777084532 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:50.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.282+0000 7f3665618700 1 --2- 172.21.15.37:0/777084532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be5deb8640 0x55be5dea7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:50.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.282+0000 7f3665e19700 1 -- 172.21.15.37:0/777084532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be5deb8f60 msgr2=0x55be5ded4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:50.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.282+0000 7f3665e19700 1 --2- 172.21.15.37:0/777084532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be5deb8f60 0x55be5ded4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.282+0000 7f3665e19700 1 -- 172.21.15.37:0/777084532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be5deb8640 msgr2=0x55be5dea7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:50.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.282+0000 7f3665e19700 1 --2- 172.21.15.37:0/777084532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be5deb8640 0x55be5dea7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.282+0000 7f3665e19700 1 -- 172.21.15.37:0/777084532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be5decf030 con 0x55be5deb2f20 2024-06-10T00:56:50.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.283+0000 7f3665e19700 1 --2- 172.21.15.37:0/777084532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be5deb2f20 0x55be5deb29e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f36500ce450 tx=0x7f3650009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89bbf726fffa8bd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:50.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.283+0000 7f36477fe700 1 -- 172.21.15.37:0/777084532 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365009b4b0 con 0x55be5deb2f20 2024-06-10T00:56:50.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.283+0000 7f36477fe700 1 -- 172.21.15.37:0/777084532 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36500074e0 con 0x55be5deb2f20 2024-06-10T00:56:50.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.283+0000 7f36477fe700 1 -- 172.21.15.37:0/777084532 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36500a1120 con 0x55be5deb2f20 2024-06-10T00:56:50.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.283+0000 7f3672900b80 1 -- 172.21.15.37:0/777084532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be5deb2f20 msgr2=0x55be5deb29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:50.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.283+0000 7f3672900b80 1 --2- 172.21.15.37:0/777084532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be5deb2f20 0x55be5deb29e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f36500ce450 tx=0x7f3650009c70 comp rx=0 tx=0).stop 2024-06-10T00:56:50.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.283+0000 7f3672900b80 1 -- 172.21.15.37:0/777084532 shutdown_connections 2024-06-10T00:56:50.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.283+0000 7f3672900b80 1 --2- 172.21.15.37:0/777084532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be5deb8f60 0x55be5ded4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.283+0000 7f3672900b80 1 --2- 172.21.15.37:0/777084532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be5deb2f20 0x55be5deb29e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.283+0000 7f3672900b80 1 --2- 172.21.15.37:0/777084532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be5deb8640 0x55be5dea7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.283+0000 7f3672900b80 1 -- 172.21.15.37:0/777084532 >> 172.21.15.37:0/777084532 conn(0x55be5ddddcc0 msgr2=0x55be5deb99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:50.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.283+0000 7f3672900b80 1 -- 172.21.15.37:0/777084532 shutdown_connections 2024-06-10T00:56:50.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.284+0000 7f3672900b80 1 -- 172.21.15.37:0/777084532 wait complete. 2024-06-10T00:56:50.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.284+0000 7f3672900b80 1 -- start start 2024-06-10T00:56:50.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.284+0000 7f3672900b80 1 Processor -- start 2024-06-10T00:56:50.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.284+0000 7f3672900b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be5deb2f20 0x55be5dec4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:50.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.284+0000 7f3672900b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be5deb8640 0x55be5dec5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:50.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.284+0000 7f3672900b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be5deb8f60 0x55be5dec1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:50.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.284+0000 7f3672900b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f36601aec70 con 0x55be5deb2f20 2024-06-10T00:56:50.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.284+0000 7f3672900b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f36601b00e0 con 0x55be5deb8f60 2024-06-10T00:56:50.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.284+0000 7f3672900b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f36601af340 con 0x55be5deb8640 2024-06-10T00:56:50.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.284+0000 7f3672900b80 10 client.0 _ll_register_callbacks cb 0x55be5dd9e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:56:50.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.284+0000 7f3665e19700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be5deb2f20 0x55be5dec4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:50.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.284+0000 7f3665e19700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be5deb2f20 0x55be5dec4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59500/0 (socket says 172.21.15.37:59500) 2024-06-10T00:56:50.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.284+0000 7f3665e19700 1 -- 172.21.15.37:0/1966083420 learned_addr learned my addr 172.21.15.37:0/1966083420 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:50.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.285+0000 7f366661a700 1 --2- 172.21.15.37:0/1966083420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be5deb8f60 0x55be5dec1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:50.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.285+0000 7f3665618700 1 --2- 172.21.15.37:0/1966083420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be5deb8640 0x55be5dec5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:50.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.285+0000 7f3665e19700 1 -- 172.21.15.37:0/1966083420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be5deb8640 msgr2=0x55be5dec5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:50.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.285+0000 7f3665e19700 1 --2- 172.21.15.37:0/1966083420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be5deb8640 0x55be5dec5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.285+0000 7f3665e19700 1 -- 172.21.15.37:0/1966083420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be5deb8f60 msgr2=0x55be5dec1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:50.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.285+0000 7f3665e19700 1 --2- 172.21.15.37:0/1966083420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be5deb8f60 0x55be5dec1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.285+0000 7f3665e19700 1 -- 172.21.15.37:0/1966083420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be5dec1fd0 con 0x55be5deb2f20 2024-06-10T00:56:50.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.285+0000 7f366661a700 1 --2- 172.21.15.37:0/1966083420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be5deb8f60 0x55be5dec1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:56:50.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.285+0000 7f3665618700 1 --2- 172.21.15.37:0/1966083420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be5deb8640 0x55be5dec5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:56:50.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.285+0000 7f3665e19700 1 --2- 172.21.15.37:0/1966083420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be5deb2f20 0x55be5dec4ac0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3650000f80 tx=0x7f365009b300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26a96ec9e0e70a8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:50.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.285+0000 7f36457fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:56:50.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.285+0000 7f36457fa700 1 -- 172.21.15.37:0/1966083420 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365001dae0 con 0x55be5deb2f20 2024-06-10T00:56:50.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.285+0000 7f36457fa700 1 -- 172.21.15.37:0/1966083420 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f36500a1120 con 0x55be5deb2f20 2024-06-10T00:56:50.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.285+0000 7f3672900b80 1 -- 172.21.15.37:0/1966083420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36500074e0 con 0x55be5deb2f20 2024-06-10T00:56:50.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.285+0000 7f36457fa700 1 -- 172.21.15.37:0/1966083420 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3650085050 con 0x55be5deb2f20 2024-06-10T00:56:50.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.285+0000 7f3672900b80 10 client.15339 fetch_fsmap learned FSMap version 27 2024-06-10T00:56:50.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.285+0000 7f3672900b80 1 -- 172.21.15.37:0/1966083420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55be5df379d0 con 0x55be5deb2f20 2024-06-10T00:56:50.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.285+0000 7f36457fa700 1 -- 172.21.15.37:0/1966083420 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36500074e0 con 0x55be5deb2f20 2024-06-10T00:56:50.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.286+0000 7f36457fa700 1 -- 172.21.15.37:0/1966083420 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3650011070 con 0x55be5deb2f20 2024-06-10T00:56:50.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.286+0000 7f3672900b80 10 client.15339 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:56:50.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.286+0000 7f3672900b80 10 client.15339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T00:56:50.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.286+0000 7f3672900b80 20 client.15339 populate_metadata read hostname 'smithi037' 2024-06-10T00:56:50.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.286+0000 7f3672900b80 1 --2- 172.21.15.37:0/1966083420 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55be5df37bb0 0x55be5df39fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:50.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.286+0000 7f3672900b80 4 client.15339 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T00:56:50.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.286+0000 7f3672900b80 1 -- 172.21.15.37:0/1966083420 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55be5de0c290 con 0x55be5df37bb0 2024-06-10T00:56:50.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.286+0000 7f3665618700 1 --2- 172.21.15.37:0/1966083420 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55be5df37bb0 0x55be5df39fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:50.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.286+0000 7f3665618700 1 --2- 172.21.15.37:0/1966083420 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55be5df37bb0 0x55be5df39fa0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a288947cdf795565 server_cookie=53453fcd81abf5ea in_seq=0 out_seq=0 2024-06-10T00:56:50.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.287+0000 7f36457fa700 10 client.15339 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:56:50.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.287+0000 7f36457fa700 1 -- 172.21.15.37:0/1966083420 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55be5de0c290 con 0x55be5df37bb0 2024-06-10T00:56:50.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.287+0000 7f36457fa700 10 client.15339 handle_command_reply: tid=0 2024-06-10T00:56:50.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.287+0000 7f3672900b80 1 client.15339 shutdown 2024-06-10T00:56:50.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.287+0000 7f3672900b80 1 -- 172.21.15.37:0/1966083420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be5deb2f20 msgr2=0x55be5dec4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:50.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.287+0000 7f3672900b80 1 --2- 172.21.15.37:0/1966083420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be5deb2f20 0x55be5dec4ac0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3650000f80 tx=0x7f365009b300 comp rx=0 tx=0).stop 2024-06-10T00:56:50.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.287+0000 7f3672900b80 1 -- 172.21.15.37:0/1966083420 shutdown_connections 2024-06-10T00:56:50.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.287+0000 7f3672900b80 1 --2- 172.21.15.37:0/1966083420 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55be5df37bb0 0x55be5df39fa0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.287+0000 7f3672900b80 1 --2- 172.21.15.37:0/1966083420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be5deb8f60 0x55be5dec1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.287+0000 7f36457fa700 0 client.15339 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:56:50.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.287+0000 7f3672900b80 1 --2- 172.21.15.37:0/1966083420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be5deb2f20 0x55be5dec4ac0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.287+0000 7f3672900b80 1 --2- 172.21.15.37:0/1966083420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55be5deb8640 0x55be5dec5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.287+0000 7f3672900b80 1 -- 172.21.15.37:0/1966083420 >> 172.21.15.37:0/1966083420 conn(0x55be5ddddcc0 msgr2=0x55be5deba8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:50.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.287+0000 7f3672900b80 1 -- 172.21.15.37:0/1966083420 shutdown_connections 2024-06-10T00:56:50.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.287+0000 7f3672900b80 1 -- 172.21.15.37:0/1966083420 wait complete. 2024-06-10T00:56:50.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.287+0000 7f3672900b80 20 client.15339 trim_cache size 0 max 16384 2024-06-10T00:56:50.338 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T00:56:50.339 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T00:56:50.339 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-06-10T00:56:50.339 INFO:teuthology.orchestra.run.smithi037.stdout: "8221b7f4-fd6c-44aa-ad29-07188e3e5f28": { 2024-06-10T00:56:50.339 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-06-10T00:56:50.339 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "8221b7f4-fd6c-44aa-ad29-07188e3e5f28", 2024-06-10T00:56:50.339 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-06-10T00:56:50.339 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T00:56:50.339 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T00:56:50.339 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T00:56:50.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.288+0000 7f366807d700 1 -- 172.21.15.37:0/2821561628 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f36480a2d30 msgr2=0x7f36480a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:50.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.288+0000 7f366807d700 1 --2- 172.21.15.37:0/2821561628 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f36480a2d30 0x7f36480a51f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3654005fd0 tx=0x7f3654005ee0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.288+0000 7f366807d700 1 -- 172.21.15.37:0/2821561628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f366006a170 msgr2=0x7f3660101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:50.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.288+0000 7f366807d700 1 --2- 172.21.15.37:0/2821561628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f366006a170 0x7f3660101460 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f365000b070 tx=0x7f3650004ef0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.288+0000 7f366807d700 1 -- 172.21.15.37:0/2821561628 shutdown_connections 2024-06-10T00:56:50.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.288+0000 7f366807d700 1 --2- 172.21.15.37:0/2821561628 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f36480a2d30 0x7f36480a51f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.288+0000 7f366807d700 1 --2- 172.21.15.37:0/2821561628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3660108500 0x7f3660101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.288+0000 7f366807d700 1 --2- 172.21.15.37:0/2821561628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f366006a170 0x7f3660101460 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.288+0000 7f366807d700 1 --2- 172.21.15.37:0/2821561628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f366006ab30 0x7f36601019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:50.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.288+0000 7f366807d700 1 -- 172.21.15.37:0/2821561628 >> 172.21.15.37:0/2821561628 conn(0x7f3660078340 msgr2=0x7f36600ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:50.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.288+0000 7f366807d700 1 -- 172.21.15.37:0/2821561628 shutdown_connections 2024-06-10T00:56:50.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:50.288+0000 7f366807d700 1 -- 172.21.15.37:0/2821561628 wait complete. 2024-06-10T00:56:50.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:50 smithi167 ceph-mon[36917]: pgmap v434: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:56:50.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:50 smithi094 ceph-mon[36708]: pgmap v434: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T00:56:51.156 INFO:tasks.cephfs.filesystem:scrub status for tag:8221b7f4-fd6c-44aa-ad29-07188e3e5f28 - {'path': '/', 'tag': '8221b7f4-fd6c-44aa-ad29-07188e3e5f28', 'options': 'recursive,force'} 2024-06-10T00:56:51.200 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:56:52.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:52 smithi167 ceph-mon[36917]: pgmap v435: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:56:52.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:52 smithi037 ceph-mon[32192]: pgmap v435: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:56:52.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:52 smithi094 ceph-mon[36708]: pgmap v435: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T00:56:53.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.085+0000 7fd411947700 1 -- 172.21.15.37:0/3952289658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd40c107860 msgr2=0x7fd40c10f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:53.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.085+0000 7fd411947700 1 --2- 172.21.15.37:0/3952289658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd40c107860 0x7fd40c10f3e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd4040099c0 tx=0x7fd404009cd0 comp rx=0 tx=0).stop 2024-06-10T00:56:53.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.085+0000 7fd411947700 1 -- 172.21.15.37:0/3952289658 shutdown_connections 2024-06-10T00:56:53.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.085+0000 7fd411947700 1 --2- 172.21.15.37:0/3952289658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd40c107860 0x7fd40c10f3e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:53.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.085+0000 7fd411947700 1 --2- 172.21.15.37:0/3952289658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd40c102770 0x7fd40c102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:53.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.085+0000 7fd411947700 1 --2- 172.21.15.37:0/3952289658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd40c103120 0x7fd40c107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:53.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.085+0000 7fd411947700 1 -- 172.21.15.37:0/3952289658 >> 172.21.15.37:0/3952289658 conn(0x7fd40c079600 msgr2=0x7fd40c079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:53.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.085+0000 7fd411947700 1 -- 172.21.15.37:0/3952289658 shutdown_connections 2024-06-10T00:56:53.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.085+0000 7fd411947700 1 -- 172.21.15.37:0/3952289658 wait complete. 2024-06-10T00:56:53.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.086+0000 7fd411947700 1 Processor -- start 2024-06-10T00:56:53.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.086+0000 7fd411947700 1 -- start start 2024-06-10T00:56:53.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.086+0000 7fd411947700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd40c102770 0x7fd40c19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:53.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.086+0000 7fd411947700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd40c103120 0x7fd40c19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:53.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.086+0000 7fd411947700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd40c107860 0x7fd40c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:53.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.086+0000 7fd411947700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c1127a0 con 0x7fd40c103120 2024-06-10T00:56:53.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.086+0000 7fd411947700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c112620 con 0x7fd40c102770 2024-06-10T00:56:53.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.087+0000 7fd411947700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c112920 con 0x7fd40c107860 2024-06-10T00:56:53.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.087+0000 7fd40a7fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd40c103120 0x7fd40c19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:53.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.087+0000 7fd40a7fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd40c103120 0x7fd40c19ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39248/0 (socket says 172.21.15.37:39248) 2024-06-10T00:56:53.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.087+0000 7fd40a7fc700 1 -- 172.21.15.37:0/374460411 learned_addr learned my addr 172.21.15.37:0/374460411 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:53.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.087+0000 7fd40affd700 1 --2- 172.21.15.37:0/374460411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd40c102770 0x7fd40c19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:53.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.087+0000 7fd40b7fe700 1 --2- 172.21.15.37:0/374460411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd40c107860 0x7fd40c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:53.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.087+0000 7fd40a7fc700 1 -- 172.21.15.37:0/374460411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd40c107860 msgr2=0x7fd40c198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:53.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.087+0000 7fd40a7fc700 1 --2- 172.21.15.37:0/374460411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd40c107860 0x7fd40c198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:53.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.087+0000 7fd40a7fc700 1 -- 172.21.15.37:0/374460411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd40c102770 msgr2=0x7fd40c19e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:53.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.087+0000 7fd40a7fc700 1 --2- 172.21.15.37:0/374460411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd40c102770 0x7fd40c19e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:53.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.087+0000 7fd40a7fc700 1 -- 172.21.15.37:0/374460411 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd40c199060 con 0x7fd40c103120 2024-06-10T00:56:53.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.087+0000 7fd40a7fc700 1 --2- 172.21.15.37:0/374460411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd40c103120 0x7fd40c19ec30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd40000d980 tx=0x7fd40000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:53.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.087+0000 7fd410945700 1 -- 172.21.15.37:0/374460411 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd400017070 con 0x7fd40c103120 2024-06-10T00:56:53.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.087+0000 7fd410945700 1 -- 172.21.15.37:0/374460411 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd400005020 con 0x7fd40c103120 2024-06-10T00:56:53.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.087+0000 7fd411947700 1 -- 172.21.15.37:0/374460411 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd40c199350 con 0x7fd40c103120 2024-06-10T00:56:53.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.087+0000 7fd410945700 1 -- 172.21.15.37:0/374460411 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40000fbf0 con 0x7fd40c103120 2024-06-10T00:56:53.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.088+0000 7fd411947700 1 -- 172.21.15.37:0/374460411 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd40c1a55c0 con 0x7fd40c103120 2024-06-10T00:56:53.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.089+0000 7fd411947700 1 -- 172.21.15.37:0/374460411 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd40c069180 con 0x7fd40c103120 2024-06-10T00:56:53.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.090+0000 7fd410945700 1 -- 172.21.15.37:0/374460411 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fd4000051d0 con 0x7fd40c103120 2024-06-10T00:56:53.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.090+0000 7fd410945700 1 --2- 172.21.15.37:0/374460411 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd3f40a2d30 0x7fd3f40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:53.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.090+0000 7fd40affd700 1 --2- 172.21.15.37:0/374460411 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd3f40a2d30 0x7fd3f40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:53.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.090+0000 7fd410945700 1 -- 172.21.15.37:0/374460411 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fd4000cf380 con 0x7fd40c103120 2024-06-10T00:56:53.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.091+0000 7fd40affd700 1 --2- 172.21.15.37:0/374460411 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd3f40a2d30 0x7fd3f40a51f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd40c107280 tx=0x7fd3fc0079e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:53.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.095+0000 7fd410945700 1 -- 172.21.15.37:0/374460411 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd400020020 con 0x7fd40c103120 2024-06-10T00:56:53.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.297+0000 7fd411947700 1 -- 172.21.15.37:0/374460411 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd40c100b30 con 0x7fd3f40a2d30 2024-06-10T00:56:53.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.307+0000 7fd410945700 1 -- 172.21.15.37:0/374460411 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340960 (secure 0 0 0) 0x7fd40c100b30 con 0x7fd3f40a2d30 2024-06-10T00:56:53.316 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:56:53.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.310+0000 7fd411947700 1 -- 172.21.15.37:0/374460411 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd3f40a2d30 msgr2=0x7fd3f40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:53.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.310+0000 7fd411947700 1 --2- 172.21.15.37:0/374460411 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd3f40a2d30 0x7fd3f40a51f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd40c107280 tx=0x7fd3fc0079e0 comp rx=0 tx=0).stop 2024-06-10T00:56:53.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.310+0000 7fd411947700 1 -- 172.21.15.37:0/374460411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd40c103120 msgr2=0x7fd40c19ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:53.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.310+0000 7fd411947700 1 --2- 172.21.15.37:0/374460411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd40c103120 0x7fd40c19ec30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd40000d980 tx=0x7fd40000dc90 comp rx=0 tx=0).stop 2024-06-10T00:56:53.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.310+0000 7fd411947700 1 -- 172.21.15.37:0/374460411 shutdown_connections 2024-06-10T00:56:53.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.310+0000 7fd411947700 1 --2- 172.21.15.37:0/374460411 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd3f40a2d30 0x7fd3f40a51f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:53.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.310+0000 7fd411947700 1 --2- 172.21.15.37:0/374460411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd40c107860 0x7fd40c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:53.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.310+0000 7fd411947700 1 --2- 172.21.15.37:0/374460411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd40c102770 0x7fd40c19e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:53.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.310+0000 7fd411947700 1 --2- 172.21.15.37:0/374460411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd40c103120 0x7fd40c19ec30 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:53.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.310+0000 7fd411947700 1 -- 172.21.15.37:0/374460411 >> 172.21.15.37:0/374460411 conn(0x7fd40c079600 msgr2=0x7fd40c100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:53.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.310+0000 7fd411947700 1 -- 172.21.15.37:0/374460411 shutdown_connections 2024-06-10T00:56:53.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:53.310+0000 7fd411947700 1 -- 172.21.15.37:0/374460411 wait complete. 2024-06-10T00:56:53.318 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-06-10T00:56:54.091 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2024-06-10T00:56:52.414673+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":672924,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1298,"num_read_kb":784,"num_write":374,"num_write_kb":3088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":802,"ondisk_log_size":802,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370920,"kb_used_data":13448,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570904,"statfs":{"total":1151940427776,"available":1151560605696,"internally_reserved":0,"allocated":13770752,"data_stored":8871914,"data_compressed":497625,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":42,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003067"},"pg_stats":[{"pgid":"3.3f","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374277+0000","last_change":"2024-06-10T00:52:35.971067+0000","last_active":"2024-06-10T00:55:20.374277+0000","last_peered":"2024-06-10T00:55:20.374277+0000","last_clean":"2024-06-10T00:55:20.374277+0000","last_became_active":"2024-06-10T00:52:35.970292+0000","last_became_peered":"2024-06-10T00:52:35.970292+0000","last_unstale":"2024-06-10T00:55:20.374277+0000","last_undegraded":"2024-06-10T00:55:20.374277+0000","last_fullsized":"2024-06-10T00:55:20.374277+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:57:37.287238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.371246+0000","last_change":"2024-06-10T00:52:35.960421+0000","last_active":"2024-06-10T00:55:20.371246+0000","last_peered":"2024-06-10T00:55:20.371246+0000","last_clean":"2024-06-10T00:55:20.371246+0000","last_became_active":"2024-06-10T00:52:35.960070+0000","last_became_peered":"2024-06-10T00:52:35.960070+0000","last_unstale":"2024-06-10T00:55:20.371246+0000","last_undegraded":"2024-06-10T00:55:20.371246+0000","last_fullsized":"2024-06-10T00:55:20.371246+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:48:02.616434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375920+0000","last_change":"2024-06-10T00:52:35.964778+0000","last_active":"2024-06-10T00:55:20.375920+0000","last_peered":"2024-06-10T00:55:20.375920+0000","last_clean":"2024-06-10T00:55:20.375920+0000","last_became_active":"2024-06-10T00:52:35.964463+0000","last_became_peered":"2024-06-10T00:52:35.964463+0000","last_unstale":"2024-06-10T00:55:20.375920+0000","last_undegraded":"2024-06-10T00:55:20.375920+0000","last_fullsized":"2024-06-10T00:55:20.375920+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:52:44.987363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.360455+0000","last_change":"2024-06-10T00:52:35.963226+0000","last_active":"2024-06-10T00:55:20.360455+0000","last_peered":"2024-06-10T00:55:20.360455+0000","last_clean":"2024-06-10T00:55:20.360455+0000","last_became_active":"2024-06-10T00:52:35.962862+0000","last_became_peered":"2024-06-10T00:52:35.962862+0000","last_unstale":"2024-06-10T00:55:20.360455+0000","last_undegraded":"2024-06-10T00:55:20.360455+0000","last_fullsized":"2024-06-10T00:55:20.360455+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:08:56.418071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.360258+0000","last_change":"2024-06-10T00:52:35.957427+0000","last_active":"2024-06-10T00:55:20.360258+0000","last_peered":"2024-06-10T00:55:20.360258+0000","last_clean":"2024-06-10T00:55:20.360258+0000","last_became_active":"2024-06-10T00:52:35.957049+0000","last_became_peered":"2024-06-10T00:52:35.957049+0000","last_unstale":"2024-06-10T00:55:20.360258+0000","last_undegraded":"2024-06-10T00:55:20.360258+0000","last_fullsized":"2024-06-10T00:55:20.360258+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:16:04.257210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.355809+0000","last_change":"2024-06-10T00:52:35.954251+0000","last_active":"2024-06-10T00:55:20.355809+0000","last_peered":"2024-06-10T00:55:20.355809+0000","last_clean":"2024-06-10T00:55:20.355809+0000","last_became_active":"2024-06-10T00:52:35.953908+0000","last_became_peered":"2024-06-10T00:52:35.953908+0000","last_unstale":"2024-06-10T00:55:20.355809+0000","last_undegraded":"2024-06-10T00:55:20.355809+0000","last_fullsized":"2024-06-10T00:55:20.355809+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:55:25.125836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364343+0000","last_change":"2024-06-10T00:52:35.956822+0000","last_active":"2024-06-10T00:55:20.364343+0000","last_peered":"2024-06-10T00:55:20.364343+0000","last_clean":"2024-06-10T00:55:20.364343+0000","last_became_active":"2024-06-10T00:52:35.956466+0000","last_became_peered":"2024-06-10T00:52:35.956466+0000","last_unstale":"2024-06-10T00:55:20.364343+0000","last_undegraded":"2024-06-10T00:55:20.364343+0000","last_fullsized":"2024-06-10T00:55:20.364343+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:36:41.534392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.369347+0000","last_change":"2024-06-10T00:52:35.962978+0000","last_active":"2024-06-10T00:55:20.369347+0000","last_peered":"2024-06-10T00:55:20.369347+0000","last_clean":"2024-06-10T00:55:20.369347+0000","last_became_active":"2024-06-10T00:52:35.962609+0000","last_became_peered":"2024-06-10T00:52:35.962609+0000","last_unstale":"2024-06-10T00:55:20.369347+0000","last_undegraded":"2024-06-10T00:55:20.369347+0000","last_fullsized":"2024-06-10T00:55:20.369347+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:20:57.469577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.377074+0000","last_change":"2024-06-10T00:52:35.963203+0000","last_active":"2024-06-10T00:55:20.377074+0000","last_peered":"2024-06-10T00:55:20.377074+0000","last_clean":"2024-06-10T00:55:20.377074+0000","last_became_active":"2024-06-10T00:52:35.962789+0000","last_became_peered":"2024-06-10T00:52:35.962789+0000","last_unstale":"2024-06-10T00:55:20.377074+0000","last_undegraded":"2024-06-10T00:55:20.377074+0000","last_fullsized":"2024-06-10T00:55:20.377074+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:00:17.819193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"100'16","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.356376+0000","last_change":"2024-06-10T00:52:35.966940+0000","last_active":"2024-06-10T00:55:20.356376+0000","last_peered":"2024-06-10T00:55:20.356376+0000","last_clean":"2024-06-10T00:55:20.356376+0000","last_became_active":"2024-06-10T00:52:35.966554+0000","last_became_peered":"2024-06-10T00:52:35.966554+0000","last_unstale":"2024-06-10T00:55:20.356376+0000","last_undegraded":"2024-06-10T00:55:20.356376+0000","last_fullsized":"2024-06-10T00:55:20.356376+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:00.374437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372673+0000","last_change":"2024-06-10T00:52:35.960927+0000","last_active":"2024-06-10T00:55:20.372673+0000","last_peered":"2024-06-10T00:55:20.372673+0000","last_clean":"2024-06-10T00:55:20.372673+0000","last_became_active":"2024-06-10T00:52:35.960577+0000","last_became_peered":"2024-06-10T00:52:35.960577+0000","last_unstale":"2024-06-10T00:55:20.372673+0000","last_undegraded":"2024-06-10T00:55:20.372673+0000","last_fullsized":"2024-06-10T00:55:20.372673+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:50:02.366695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.355167+0000","last_change":"2024-06-10T00:52:35.964435+0000","last_active":"2024-06-10T00:55:20.355167+0000","last_peered":"2024-06-10T00:55:20.355167+0000","last_clean":"2024-06-10T00:55:20.355167+0000","last_became_active":"2024-06-10T00:52:35.964069+0000","last_became_peered":"2024-06-10T00:52:35.964069+0000","last_unstale":"2024-06-10T00:55:20.355167+0000","last_undegraded":"2024-06-10T00:55:20.355167+0000","last_fullsized":"2024-06-10T00:55:20.355167+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:06.708551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.361787+0000","last_change":"2024-06-10T00:52:35.966722+0000","last_active":"2024-06-10T00:55:20.361787+0000","last_peered":"2024-06-10T00:55:20.361787+0000","last_clean":"2024-06-10T00:55:20.361787+0000","last_became_active":"2024-06-10T00:52:35.966366+0000","last_became_peered":"2024-06-10T00:52:35.966366+0000","last_unstale":"2024-06-10T00:55:20.361787+0000","last_undegraded":"2024-06-10T00:55:20.361787+0000","last_fullsized":"2024-06-10T00:55:20.361787+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:57:46.021047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.371736+0000","last_change":"2024-06-10T00:52:35.955307+0000","last_active":"2024-06-10T00:55:20.371736+0000","last_peered":"2024-06-10T00:55:20.371736+0000","last_clean":"2024-06-10T00:55:20.371736+0000","last_became_active":"2024-06-10T00:52:35.954140+0000","last_became_peered":"2024-06-10T00:52:35.954140+0000","last_unstale":"2024-06-10T00:55:20.371736+0000","last_undegraded":"2024-06-10T00:55:20.371736+0000","last_fullsized":"2024-06-10T00:55:20.371736+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:47:22.493598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373915+0000","last_change":"2024-06-10T00:52:35.960592+0000","last_active":"2024-06-10T00:55:20.373915+0000","last_peered":"2024-06-10T00:55:20.373915+0000","last_clean":"2024-06-10T00:55:20.373915+0000","last_became_active":"2024-06-10T00:52:35.960263+0000","last_became_peered":"2024-06-10T00:52:35.960263+0000","last_unstale":"2024-06-10T00:55:20.373915+0000","last_undegraded":"2024-06-10T00:55:20.373915+0000","last_fullsized":"2024-06-10T00:55:20.373915+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:47:11.658666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372126+0000","last_change":"2024-06-10T00:52:35.965796+0000","last_active":"2024-06-10T00:55:20.372126+0000","last_peered":"2024-06-10T00:55:20.372126+0000","last_clean":"2024-06-10T00:55:20.372126+0000","last_became_active":"2024-06-10T00:52:35.963965+0000","last_became_peered":"2024-06-10T00:52:35.963965+0000","last_unstale":"2024-06-10T00:55:20.372126+0000","last_undegraded":"2024-06-10T00:55:20.372126+0000","last_fullsized":"2024-06-10T00:55:20.372126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:29:47.149852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374090+0000","last_change":"2024-06-10T00:52:35.970714+0000","last_active":"2024-06-10T00:55:20.374090+0000","last_peered":"2024-06-10T00:55:20.374090+0000","last_clean":"2024-06-10T00:55:20.374090+0000","last_became_active":"2024-06-10T00:52:35.970173+0000","last_became_peered":"2024-06-10T00:52:35.970173+0000","last_unstale":"2024-06-10T00:55:20.374090+0000","last_undegraded":"2024-06-10T00:55:20.374090+0000","last_fullsized":"2024-06-10T00:55:20.374090+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:33:28.183156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.370737+0000","last_change":"2024-06-10T00:52:35.963052+0000","last_active":"2024-06-10T00:55:20.370737+0000","last_peered":"2024-06-10T00:55:20.370737+0000","last_clean":"2024-06-10T00:55:20.370737+0000","last_became_active":"2024-06-10T00:52:35.962674+0000","last_became_peered":"2024-06-10T00:52:35.962674+0000","last_unstale":"2024-06-10T00:55:20.370737+0000","last_undegraded":"2024-06-10T00:55:20.370737+0000","last_fullsized":"2024-06-10T00:55:20.370737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:56:38.836854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"100'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.363943+0000","last_change":"2024-06-10T00:53:22.130336+0000","last_active":"2024-06-10T00:55:20.363943+0000","last_peered":"2024-06-10T00:55:20.363943+0000","last_clean":"2024-06-10T00:55:20.363943+0000","last_became_active":"2024-06-10T00:53:22.129910+0000","last_became_peered":"2024-06-10T00:53:22.129910+0000","last_unstale":"2024-06-10T00:55:20.363943+0000","last_undegraded":"2024-06-10T00:55:20.363943+0000","last_fullsized":"2024-06-10T00:55:20.363943+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:23:50.600821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554286+0000","last_change":"2024-06-10T00:52:45.997276+0000","last_active":"2024-06-10T00:54:22.554286+0000","last_peered":"2024-06-10T00:54:22.554286+0000","last_clean":"2024-06-10T00:54:22.554286+0000","last_became_active":"2024-06-10T00:52:45.995638+0000","last_became_peered":"2024-06-10T00:52:45.995638+0000","last_unstale":"2024-06-10T00:54:22.554286+0000","last_undegraded":"2024-06-10T00:54:22.554286+0000","last_fullsized":"2024-06-10T00:54:22.554286+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:14:20.922348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:45.433552+0000","last_change":"2024-06-10T00:52:32.760888+0000","last_active":"2024-06-10T00:56:45.433552+0000","last_peered":"2024-06-10T00:56:45.433552+0000","last_clean":"2024-06-10T00:56:45.433552+0000","last_became_active":"2024-06-10T00:52:32.760531+0000","last_became_peered":"2024-06-10T00:52:32.760531+0000","last_unstale":"2024-06-10T00:56:45.433552+0000","last_undegraded":"2024-06-10T00:56:45.433552+0000","last_fullsized":"2024-06-10T00:56:45.433552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:05:59.052251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"100'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372743+0000","last_change":"2024-06-10T00:52:35.967626+0000","last_active":"2024-06-10T00:55:20.372743+0000","last_peered":"2024-06-10T00:55:20.372743+0000","last_clean":"2024-06-10T00:55:20.372743+0000","last_became_active":"2024-06-10T00:52:35.967295+0000","last_became_peered":"2024-06-10T00:52:35.967295+0000","last_unstale":"2024-06-10T00:55:20.372743+0000","last_undegraded":"2024-06-10T00:55:20.372743+0000","last_fullsized":"2024-06-10T00:55:20.372743+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:07:55.054327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668105+0000","last_change":"2024-06-10T00:52:45.993974+0000","last_active":"2024-06-10T00:54:22.668105+0000","last_peered":"2024-06-10T00:54:22.668105+0000","last_clean":"2024-06-10T00:54:22.668105+0000","last_became_active":"2024-06-10T00:52:45.993242+0000","last_became_peered":"2024-06-10T00:52:45.993242+0000","last_unstale":"2024-06-10T00:54:22.668105+0000","last_undegraded":"2024-06-10T00:54:22.668105+0000","last_fullsized":"2024-06-10T00:54:22.668105+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:28:12.810368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:33.628114+0000","last_change":"2024-06-10T00:52:32.959905+0000","last_active":"2024-06-10T00:54:33.628114+0000","last_peered":"2024-06-10T00:54:33.628114+0000","last_clean":"2024-06-10T00:54:33.628114+0000","last_became_active":"2024-06-10T00:52:32.959551+0000","last_became_peered":"2024-06-10T00:52:32.959551+0000","last_unstale":"2024-06-10T00:54:33.628114+0000","last_undegraded":"2024-06-10T00:54:33.628114+0000","last_fullsized":"2024-06-10T00:54:33.628114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:33:48.029129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.377301+0000","last_change":"2024-06-10T00:52:35.965790+0000","last_active":"2024-06-10T00:55:20.377301+0000","last_peered":"2024-06-10T00:55:20.377301+0000","last_clean":"2024-06-10T00:55:20.377301+0000","last_became_active":"2024-06-10T00:52:35.965475+0000","last_became_peered":"2024-06-10T00:52:35.965475+0000","last_unstale":"2024-06-10T00:55:20.377301+0000","last_undegraded":"2024-06-10T00:55:20.377301+0000","last_fullsized":"2024-06-10T00:55:20.377301+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:24:02.894411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.667953+0000","last_change":"2024-06-10T00:52:45.994464+0000","last_active":"2024-06-10T00:54:22.667953+0000","last_peered":"2024-06-10T00:54:22.667953+0000","last_clean":"2024-06-10T00:54:22.667953+0000","last_became_active":"2024-06-10T00:52:45.994022+0000","last_became_peered":"2024-06-10T00:52:45.994022+0000","last_unstale":"2024-06-10T00:54:22.667953+0000","last_undegraded":"2024-06-10T00:54:22.667953+0000","last_fullsized":"2024-06-10T00:54:22.667953+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:27:25.063074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556658+0000","last_change":"2024-06-10T00:52:32.763885+0000","last_active":"2024-06-10T00:54:22.556658+0000","last_peered":"2024-06-10T00:54:22.556658+0000","last_clean":"2024-06-10T00:54:22.556658+0000","last_became_active":"2024-06-10T00:52:32.763606+0000","last_became_peered":"2024-06-10T00:52:32.763606+0000","last_unstale":"2024-06-10T00:54:22.556658+0000","last_undegraded":"2024-06-10T00:54:22.556658+0000","last_fullsized":"2024-06-10T00:54:22.556658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:01:58.161168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372890+0000","last_change":"2024-06-10T00:52:35.966948+0000","last_active":"2024-06-10T00:55:20.372890+0000","last_peered":"2024-06-10T00:55:20.372890+0000","last_clean":"2024-06-10T00:55:20.372890+0000","last_became_active":"2024-06-10T00:52:35.966345+0000","last_became_peered":"2024-06-10T00:52:35.966345+0000","last_unstale":"2024-06-10T00:55:20.372890+0000","last_undegraded":"2024-06-10T00:55:20.372890+0000","last_fullsized":"2024-06-10T00:55:20.372890+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:58:27.169201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.667762+0000","last_change":"2024-06-10T00:52:45.993645+0000","last_active":"2024-06-10T00:54:22.667762+0000","last_peered":"2024-06-10T00:54:22.667762+0000","last_clean":"2024-06-10T00:54:22.667762+0000","last_became_active":"2024-06-10T00:52:45.993183+0000","last_became_peered":"2024-06-10T00:52:45.993183+0000","last_unstale":"2024-06-10T00:54:22.667762+0000","last_undegraded":"2024-06-10T00:54:22.667762+0000","last_fullsized":"2024-06-10T00:54:22.667762+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:47:00.541427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.566044+0000","last_change":"2024-06-10T00:54:22.565600+0000","last_active":"2024-06-10T00:54:22.566044+0000","last_peered":"2024-06-10T00:54:22.566044+0000","last_clean":"2024-06-10T00:54:22.566044+0000","last_became_active":"2024-06-10T00:54:22.562561+0000","last_became_peered":"2024-06-10T00:54:22.562561+0000","last_unstale":"2024-06-10T00:54:22.566044+0000","last_undegraded":"2024-06-10T00:54:22.566044+0000","last_fullsized":"2024-06-10T00:54:22.566044+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:47:58.748050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"100'15","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365162+0000","last_change":"2024-06-10T00:53:22.199668+0000","last_active":"2024-06-10T00:55:20.365162+0000","last_peered":"2024-06-10T00:55:20.365162+0000","last_clean":"2024-06-10T00:55:20.365162+0000","last_became_active":"2024-06-10T00:53:22.199329+0000","last_became_peered":"2024-06-10T00:53:22.199329+0000","last_unstale":"2024-06-10T00:55:20.365162+0000","last_undegraded":"2024-06-10T00:55:20.365162+0000","last_fullsized":"2024-06-10T00:55:20.365162+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:37.640478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554689+0000","last_change":"2024-06-10T00:52:45.995767+0000","last_active":"2024-06-10T00:54:22.554689+0000","last_peered":"2024-06-10T00:54:22.554689+0000","last_clean":"2024-06-10T00:54:22.554689+0000","last_became_active":"2024-06-10T00:52:45.994659+0000","last_became_peered":"2024-06-10T00:52:45.994659+0000","last_unstale":"2024-06-10T00:54:22.554689+0000","last_undegraded":"2024-06-10T00:54:22.554689+0000","last_fullsized":"2024-06-10T00:54:22.554689+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:03:45.652248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'8","reported_seq":247,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:50.585916+0000","last_change":"2024-06-10T00:52:32.761637+0000","last_active":"2024-06-10T00:56:50.585916+0000","last_peered":"2024-06-10T00:56:50.585916+0000","last_clean":"2024-06-10T00:56:50.585916+0000","last_became_active":"2024-06-10T00:52:32.761144+0000","last_became_peered":"2024-06-10T00:52:32.761144+0000","last_unstale":"2024-06-10T00:56:50.585916+0000","last_undegraded":"2024-06-10T00:56:50.585916+0000","last_fullsized":"2024-06-10T00:56:50.585916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:55:05.616564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1461,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":196,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.362574+0000","last_change":"2024-06-10T00:52:35.962683+0000","last_active":"2024-06-10T00:55:20.362574+0000","last_peered":"2024-06-10T00:55:20.362574+0000","last_clean":"2024-06-10T00:55:20.362574+0000","last_became_active":"2024-06-10T00:52:35.962327+0000","last_became_peered":"2024-06-10T00:52:35.962327+0000","last_unstale":"2024-06-10T00:55:20.362574+0000","last_undegraded":"2024-06-10T00:55:20.362574+0000","last_fullsized":"2024-06-10T00:55:20.362574+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:38:19.063306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554563+0000","last_change":"2024-06-10T00:52:45.996882+0000","last_active":"2024-06-10T00:54:22.554563+0000","last_peered":"2024-06-10T00:54:22.554563+0000","last_clean":"2024-06-10T00:54:22.554563+0000","last_became_active":"2024-06-10T00:52:45.995767+0000","last_became_peered":"2024-06-10T00:52:45.995767+0000","last_unstale":"2024-06-10T00:54:22.554563+0000","last_undegraded":"2024-06-10T00:54:22.554563+0000","last_fullsized":"2024-06-10T00:54:22.554563+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:28:03.766944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553970+0000","last_change":"2024-06-10T00:53:22.200546+0000","last_active":"2024-06-10T00:54:22.553970+0000","last_peered":"2024-06-10T00:54:22.553970+0000","last_clean":"2024-06-10T00:54:22.553970+0000","last_became_active":"2024-06-10T00:53:22.200059+0000","last_became_peered":"2024-06-10T00:53:22.200059+0000","last_unstale":"2024-06-10T00:54:22.553970+0000","last_undegraded":"2024-06-10T00:54:22.553970+0000","last_fullsized":"2024-06-10T00:54:22.553970+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:08:47.682594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.356418+0000","last_change":"2024-06-10T00:52:35.963105+0000","last_active":"2024-06-10T00:55:20.356418+0000","last_peered":"2024-06-10T00:55:20.356418+0000","last_clean":"2024-06-10T00:55:20.356418+0000","last_became_active":"2024-06-10T00:52:35.962627+0000","last_became_peered":"2024-06-10T00:52:35.962627+0000","last_unstale":"2024-06-10T00:55:20.356418+0000","last_undegraded":"2024-06-10T00:55:20.356418+0000","last_fullsized":"2024-06-10T00:55:20.356418+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:55:51.231795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.559190+0000","last_change":"2024-06-10T00:52:45.992557+0000","last_active":"2024-06-10T00:54:22.559190+0000","last_peered":"2024-06-10T00:54:22.559190+0000","last_clean":"2024-06-10T00:54:22.559190+0000","last_became_active":"2024-06-10T00:52:45.992166+0000","last_became_peered":"2024-06-10T00:52:45.992166+0000","last_unstale":"2024-06-10T00:54:22.559190+0000","last_undegraded":"2024-06-10T00:54:22.559190+0000","last_fullsized":"2024-06-10T00:54:22.559190+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:49:22.368409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553428+0000","last_change":"2024-06-10T00:52:32.751867+0000","last_active":"2024-06-10T00:54:22.553428+0000","last_peered":"2024-06-10T00:54:22.553428+0000","last_clean":"2024-06-10T00:54:22.553428+0000","last_became_active":"2024-06-10T00:52:32.751516+0000","last_became_peered":"2024-06-10T00:52:32.751516+0000","last_unstale":"2024-06-10T00:54:22.553428+0000","last_undegraded":"2024-06-10T00:54:22.553428+0000","last_fullsized":"2024-06-10T00:54:22.553428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:37:55.920613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.370097+0000","last_change":"2024-06-10T00:52:35.957772+0000","last_active":"2024-06-10T00:55:20.370097+0000","last_peered":"2024-06-10T00:55:20.370097+0000","last_clean":"2024-06-10T00:55:20.370097+0000","last_became_active":"2024-06-10T00:52:35.957470+0000","last_became_peered":"2024-06-10T00:52:35.957470+0000","last_unstale":"2024-06-10T00:55:20.370097+0000","last_undegraded":"2024-06-10T00:55:20.370097+0000","last_fullsized":"2024-06-10T00:55:20.370097+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:16:43.645292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:49.523884+0000","last_change":"2024-06-10T00:52:45.996807+0000","last_active":"2024-06-10T00:54:49.523884+0000","last_peered":"2024-06-10T00:54:49.523884+0000","last_clean":"2024-06-10T00:54:49.523884+0000","last_became_active":"2024-06-10T00:52:45.996378+0000","last_became_peered":"2024-06-10T00:52:45.996378+0000","last_unstale":"2024-06-10T00:54:49.523884+0000","last_undegraded":"2024-06-10T00:54:49.523884+0000","last_fullsized":"2024-06-10T00:54:49.523884+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:40:54.251790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'1","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561873+0000","last_change":"2024-06-10T00:52:32.759625+0000","last_active":"2024-06-10T00:54:22.561873+0000","last_peered":"2024-06-10T00:54:22.561873+0000","last_clean":"2024-06-10T00:54:22.561873+0000","last_became_active":"2024-06-10T00:52:32.759299+0000","last_became_peered":"2024-06-10T00:52:32.759299+0000","last_unstale":"2024-06-10T00:54:22.561873+0000","last_undegraded":"2024-06-10T00:54:22.561873+0000","last_fullsized":"2024-06-10T00:54:22.561873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:11:44.966453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365673+0000","last_change":"2024-06-10T00:52:35.957672+0000","last_active":"2024-06-10T00:55:20.365673+0000","last_peered":"2024-06-10T00:55:20.365673+0000","last_clean":"2024-06-10T00:55:20.365673+0000","last_became_active":"2024-06-10T00:52:35.957328+0000","last_became_peered":"2024-06-10T00:52:35.957328+0000","last_unstale":"2024-06-10T00:55:20.365673+0000","last_undegraded":"2024-06-10T00:55:20.365673+0000","last_fullsized":"2024-06-10T00:55:20.365673+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:43:59.658178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601278+0000","last_change":"2024-06-10T00:52:45.996677+0000","last_active":"2024-06-10T00:54:22.601278+0000","last_peered":"2024-06-10T00:54:22.601278+0000","last_clean":"2024-06-10T00:54:22.601278+0000","last_became_active":"2024-06-10T00:52:45.994483+0000","last_became_peered":"2024-06-10T00:52:45.994483+0000","last_unstale":"2024-06-10T00:54:22.601278+0000","last_undegraded":"2024-06-10T00:54:22.601278+0000","last_fullsized":"2024-06-10T00:54:22.601278+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:03:28.712072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'5","reported_seq":240,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:49.583797+0000","last_change":"2024-06-10T00:52:32.754829+0000","last_active":"2024-06-10T00:56:49.583797+0000","last_peered":"2024-06-10T00:56:49.583797+0000","last_clean":"2024-06-10T00:56:49.583797+0000","last_became_active":"2024-06-10T00:52:32.754386+0000","last_became_peered":"2024-06-10T00:52:32.754386+0000","last_unstale":"2024-06-10T00:56:49.583797+0000","last_undegraded":"2024-06-10T00:56:49.583797+0000","last_fullsized":"2024-06-10T00:56:49.583797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:00.870324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.371610+0000","last_change":"2024-06-10T00:52:35.966871+0000","last_active":"2024-06-10T00:55:20.371610+0000","last_peered":"2024-06-10T00:55:20.371610+0000","last_clean":"2024-06-10T00:55:20.371610+0000","last_became_active":"2024-06-10T00:52:35.966492+0000","last_became_peered":"2024-06-10T00:52:35.966492+0000","last_unstale":"2024-06-10T00:55:20.371610+0000","last_undegraded":"2024-06-10T00:55:20.371610+0000","last_fullsized":"2024-06-10T00:55:20.371610+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:24:41.048935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:23.582564+0000","last_change":"2024-06-10T00:54:22.879457+0000","last_active":"2024-06-10T00:54:23.582564+0000","last_peered":"2024-06-10T00:54:23.582564+0000","last_clean":"2024-06-10T00:54:23.582564+0000","last_became_active":"2024-06-10T00:54:22.878978+0000","last_became_peered":"2024-06-10T00:54:22.878978+0000","last_unstale":"2024-06-10T00:54:23.582564+0000","last_undegraded":"2024-06-10T00:54:23.582564+0000","last_fullsized":"2024-06-10T00:54:23.582564+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:46:38.819250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404659+0000","last_change":"2024-06-10T00:52:32.765794+0000","last_active":"2024-06-10T00:54:24.404659+0000","last_peered":"2024-06-10T00:54:24.404659+0000","last_clean":"2024-06-10T00:54:24.404659+0000","last_became_active":"2024-06-10T00:52:32.765276+0000","last_became_peered":"2024-06-10T00:52:32.765276+0000","last_unstale":"2024-06-10T00:54:24.404659+0000","last_undegraded":"2024-06-10T00:54:24.404659+0000","last_fullsized":"2024-06-10T00:54:24.404659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:06:51.209012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365186+0000","last_change":"2024-06-10T00:52:35.954897+0000","last_active":"2024-06-10T00:55:20.365186+0000","last_peered":"2024-06-10T00:55:20.365186+0000","last_clean":"2024-06-10T00:55:20.365186+0000","last_became_active":"2024-06-10T00:52:35.954565+0000","last_became_peered":"2024-06-10T00:52:35.954565+0000","last_unstale":"2024-06-10T00:55:20.365186+0000","last_undegraded":"2024-06-10T00:55:20.365186+0000","last_fullsized":"2024-06-10T00:55:20.365186+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:39:26.702804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668190+0000","last_change":"2024-06-10T00:52:45.992194+0000","last_active":"2024-06-10T00:54:22.668190+0000","last_peered":"2024-06-10T00:54:22.668190+0000","last_clean":"2024-06-10T00:54:22.668190+0000","last_became_active":"2024-06-10T00:52:45.991453+0000","last_became_peered":"2024-06-10T00:52:45.991453+0000","last_unstale":"2024-06-10T00:54:22.668190+0000","last_undegraded":"2024-06-10T00:54:22.668190+0000","last_fullsized":"2024-06-10T00:54:22.668190+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:11:10.775610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561610+0000","last_change":"2024-06-10T00:52:32.962214+0000","last_active":"2024-06-10T00:54:22.561610+0000","last_peered":"2024-06-10T00:54:22.561610+0000","last_clean":"2024-06-10T00:54:22.561610+0000","last_became_active":"2024-06-10T00:52:32.961835+0000","last_became_peered":"2024-06-10T00:52:32.961835+0000","last_unstale":"2024-06-10T00:54:22.561610+0000","last_undegraded":"2024-06-10T00:54:22.561610+0000","last_fullsized":"2024-06-10T00:54:22.561610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:36:44.724122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.377485+0000","last_change":"2024-06-10T00:52:35.963413+0000","last_active":"2024-06-10T00:55:20.377485+0000","last_peered":"2024-06-10T00:55:20.377485+0000","last_clean":"2024-06-10T00:55:20.377485+0000","last_became_active":"2024-06-10T00:52:35.962922+0000","last_became_peered":"2024-06-10T00:52:35.962922+0000","last_unstale":"2024-06-10T00:55:20.377485+0000","last_undegraded":"2024-06-10T00:55:20.377485+0000","last_fullsized":"2024-06-10T00:55:20.377485+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:05:32.164696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":34,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.560580+0000","last_change":"2024-06-10T00:54:22.560196+0000","last_active":"2024-06-10T00:54:22.560580+0000","last_peered":"2024-06-10T00:54:22.560580+0000","last_clean":"2024-06-10T00:54:22.560580+0000","last_became_active":"2024-06-10T00:54:22.559826+0000","last_became_peered":"2024-06-10T00:54:22.559826+0000","last_unstale":"2024-06-10T00:54:22.560580+0000","last_undegraded":"2024-06-10T00:54:22.560580+0000","last_fullsized":"2024-06-10T00:54:22.560580+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:09:28.456394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'46","reported_seq":486,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:50.589095+0000","last_change":"2024-06-10T00:52:32.761592+0000","last_active":"2024-06-10T00:56:50.589095+0000","last_peered":"2024-06-10T00:56:50.589095+0000","last_clean":"2024-06-10T00:56:50.589095+0000","last_became_active":"2024-06-10T00:52:32.761169+0000","last_became_peered":"2024-06-10T00:52:32.761169+0000","last_unstale":"2024-06-10T00:56:50.589095+0000","last_undegraded":"2024-06-10T00:56:50.589095+0000","last_fullsized":"2024-06-10T00:56:50.589095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:08:33.015035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77873,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":395,"num_read_kb":276,"num_write":47,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.351972+0000","last_change":"2024-06-10T00:52:35.955304+0000","last_active":"2024-06-10T00:55:20.351972+0000","last_peered":"2024-06-10T00:55:20.351972+0000","last_clean":"2024-06-10T00:55:20.351972+0000","last_became_active":"2024-06-10T00:52:35.954949+0000","last_became_peered":"2024-06-10T00:52:35.954949+0000","last_unstale":"2024-06-10T00:55:20.351972+0000","last_undegraded":"2024-06-10T00:55:20.351972+0000","last_fullsized":"2024-06-10T00:55:20.351972+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:56:42.890022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561827+0000","last_change":"2024-06-10T00:52:45.992974+0000","last_active":"2024-06-10T00:54:22.561827+0000","last_peered":"2024-06-10T00:54:22.561827+0000","last_clean":"2024-06-10T00:54:22.561827+0000","last_became_active":"2024-06-10T00:52:45.992490+0000","last_became_peered":"2024-06-10T00:52:45.992490+0000","last_unstale":"2024-06-10T00:54:22.561827+0000","last_undegraded":"2024-06-10T00:54:22.561827+0000","last_fullsized":"2024-06-10T00:54:22.561827+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:03:45.543476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554437+0000","last_change":"2024-06-10T00:52:32.762347+0000","last_active":"2024-06-10T00:54:22.554437+0000","last_peered":"2024-06-10T00:54:22.554437+0000","last_clean":"2024-06-10T00:54:22.554437+0000","last_became_active":"2024-06-10T00:52:32.761049+0000","last_became_peered":"2024-06-10T00:52:32.761049+0000","last_unstale":"2024-06-10T00:54:22.554437+0000","last_undegraded":"2024-06-10T00:54:22.554437+0000","last_fullsized":"2024-06-10T00:54:22.554437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:29:38.384451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364495+0000","last_change":"2024-06-10T00:52:35.961299+0000","last_active":"2024-06-10T00:55:20.364495+0000","last_peered":"2024-06-10T00:55:20.364495+0000","last_clean":"2024-06-10T00:55:20.364495+0000","last_became_active":"2024-06-10T00:52:35.960966+0000","last_became_peered":"2024-06-10T00:52:35.960966+0000","last_unstale":"2024-06-10T00:55:20.364495+0000","last_undegraded":"2024-06-10T00:55:20.364495+0000","last_fullsized":"2024-06-10T00:55:20.364495+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:32:21.269689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.559474+0000","last_change":"2024-06-10T00:52:46.002091+0000","last_active":"2024-06-10T00:54:22.559474+0000","last_peered":"2024-06-10T00:54:22.559474+0000","last_clean":"2024-06-10T00:54:22.559474+0000","last_became_active":"2024-06-10T00:52:46.001677+0000","last_became_peered":"2024-06-10T00:52:46.001677+0000","last_unstale":"2024-06-10T00:54:22.559474+0000","last_undegraded":"2024-06-10T00:54:22.559474+0000","last_fullsized":"2024-06-10T00:54:22.559474+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:16:56.405720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'3","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.562287+0000","last_change":"2024-06-10T00:52:32.756765+0000","last_active":"2024-06-10T00:54:22.562287+0000","last_peered":"2024-06-10T00:54:22.562287+0000","last_clean":"2024-06-10T00:54:22.562287+0000","last_became_active":"2024-06-10T00:52:32.756470+0000","last_became_peered":"2024-06-10T00:52:32.756470+0000","last_unstale":"2024-06-10T00:54:22.562287+0000","last_undegraded":"2024-06-10T00:54:22.562287+0000","last_fullsized":"2024-06-10T00:54:22.562287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:29:24.854913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"100'15","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364256+0000","last_change":"2024-06-10T00:52:35.960693+0000","last_active":"2024-06-10T00:55:20.364256+0000","last_peered":"2024-06-10T00:55:20.364256+0000","last_clean":"2024-06-10T00:55:20.364256+0000","last_became_active":"2024-06-10T00:52:35.958397+0000","last_became_peered":"2024-06-10T00:52:35.958397+0000","last_unstale":"2024-06-10T00:55:20.364256+0000","last_undegraded":"2024-06-10T00:55:20.364256+0000","last_fullsized":"2024-06-10T00:55:20.364256+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:23:47.556462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556200+0000","last_change":"2024-06-10T00:52:46.001475+0000","last_active":"2024-06-10T00:54:22.556200+0000","last_peered":"2024-06-10T00:54:22.556200+0000","last_clean":"2024-06-10T00:54:22.556200+0000","last_became_active":"2024-06-10T00:52:46.000983+0000","last_became_peered":"2024-06-10T00:52:46.000983+0000","last_unstale":"2024-06-10T00:54:22.556200+0000","last_undegraded":"2024-06-10T00:54:22.556200+0000","last_fullsized":"2024-06-10T00:54:22.556200+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:23:56.285356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"98'3","reported_seq":236,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:46.579414+0000","last_change":"2024-06-10T00:52:32.756665+0000","last_active":"2024-06-10T00:56:46.579414+0000","last_peered":"2024-06-10T00:56:46.579414+0000","last_clean":"2024-06-10T00:56:46.579414+0000","last_became_active":"2024-06-10T00:52:32.756221+0000","last_became_peered":"2024-06-10T00:52:32.756221+0000","last_unstale":"2024-06-10T00:56:46.579414+0000","last_undegraded":"2024-06-10T00:56:46.579414+0000","last_fullsized":"2024-06-10T00:56:46.579414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:50:34.789063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":190,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"100'6","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364652+0000","last_change":"2024-06-10T00:53:22.125844+0000","last_active":"2024-06-10T00:55:20.364652+0000","last_peered":"2024-06-10T00:55:20.364652+0000","last_clean":"2024-06-10T00:55:20.364652+0000","last_became_active":"2024-06-10T00:53:22.125449+0000","last_became_peered":"2024-06-10T00:53:22.125449+0000","last_unstale":"2024-06-10T00:55:20.364652+0000","last_undegraded":"2024-06-10T00:55:20.364652+0000","last_fullsized":"2024-06-10T00:55:20.364652+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:08:42.936702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553904+0000","last_change":"2024-06-10T00:52:45.999300+0000","last_active":"2024-06-10T00:54:22.553904+0000","last_peered":"2024-06-10T00:54:22.553904+0000","last_clean":"2024-06-10T00:54:22.553904+0000","last_became_active":"2024-06-10T00:52:45.998860+0000","last_became_peered":"2024-06-10T00:52:45.998860+0000","last_unstale":"2024-06-10T00:54:22.553904+0000","last_undegraded":"2024-06-10T00:54:22.553904+0000","last_fullsized":"2024-06-10T00:54:22.553904+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:14:24.936870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":75,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.557182+0000","last_change":"2024-06-10T00:53:22.210710+0000","last_active":"2024-06-10T00:54:22.557182+0000","last_peered":"2024-06-10T00:54:22.557182+0000","last_clean":"2024-06-10T00:54:22.557182+0000","last_became_active":"2024-06-10T00:53:22.202461+0000","last_became_peered":"2024-06-10T00:53:22.202461+0000","last_unstale":"2024-06-10T00:54:22.557182+0000","last_undegraded":"2024-06-10T00:54:22.557182+0000","last_fullsized":"2024-06-10T00:54:22.557182+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:28:14.671776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364560+0000","last_change":"2024-06-10T00:52:35.966960+0000","last_active":"2024-06-10T00:55:20.364560+0000","last_peered":"2024-06-10T00:55:20.364560+0000","last_clean":"2024-06-10T00:55:20.364560+0000","last_became_active":"2024-06-10T00:52:35.966630+0000","last_became_peered":"2024-06-10T00:52:35.966630+0000","last_unstale":"2024-06-10T00:55:20.364560+0000","last_undegraded":"2024-06-10T00:55:20.364560+0000","last_fullsized":"2024-06-10T00:55:20.364560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:04.117405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561586+0000","last_change":"2024-06-10T00:52:45.992955+0000","last_active":"2024-06-10T00:54:22.561586+0000","last_peered":"2024-06-10T00:54:22.561586+0000","last_clean":"2024-06-10T00:54:22.561586+0000","last_became_active":"2024-06-10T00:52:45.992457+0000","last_became_peered":"2024-06-10T00:52:45.992457+0000","last_unstale":"2024-06-10T00:54:22.561586+0000","last_undegraded":"2024-06-10T00:54:22.561586+0000","last_fullsized":"2024-06-10T00:54:22.561586+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:13:49.491183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556768+0000","last_change":"2024-06-10T00:52:32.763794+0000","last_active":"2024-06-10T00:54:22.556768+0000","last_peered":"2024-06-10T00:54:22.556768+0000","last_clean":"2024-06-10T00:54:22.556768+0000","last_became_active":"2024-06-10T00:52:32.763406+0000","last_became_peered":"2024-06-10T00:52:32.763406+0000","last_unstale":"2024-06-10T00:54:22.556768+0000","last_undegraded":"2024-06-10T00:54:22.556768+0000","last_fullsized":"2024-06-10T00:54:22.556768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:50:33.580163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"100'5","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374120+0000","last_change":"2024-06-10T00:53:22.130891+0000","last_active":"2024-06-10T00:55:20.374120+0000","last_peered":"2024-06-10T00:55:20.374120+0000","last_clean":"2024-06-10T00:55:20.374120+0000","last_became_active":"2024-06-10T00:53:22.130508+0000","last_became_peered":"2024-06-10T00:53:22.130508+0000","last_unstale":"2024-06-10T00:55:20.374120+0000","last_undegraded":"2024-06-10T00:55:20.374120+0000","last_fullsized":"2024-06-10T00:55:20.374120+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:28:01.919825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":34,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.563757+0000","last_change":"2024-06-10T00:54:22.563272+0000","last_active":"2024-06-10T00:54:22.563757+0000","last_peered":"2024-06-10T00:54:22.563757+0000","last_clean":"2024-06-10T00:54:22.563757+0000","last_became_active":"2024-06-10T00:54:22.562767+0000","last_became_peered":"2024-06-10T00:54:22.562767+0000","last_unstale":"2024-06-10T00:54:22.563757+0000","last_undegraded":"2024-06-10T00:54:22.563757+0000","last_fullsized":"2024-06-10T00:54:22.563757+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:52:20.739801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:40.422237+0000","last_change":"2024-06-10T00:52:32.960515+0000","last_active":"2024-06-10T00:54:40.422237+0000","last_peered":"2024-06-10T00:54:40.422237+0000","last_clean":"2024-06-10T00:54:40.422237+0000","last_became_active":"2024-06-10T00:52:32.960163+0000","last_became_peered":"2024-06-10T00:52:32.960163+0000","last_unstale":"2024-06-10T00:54:40.422237+0000","last_undegraded":"2024-06-10T00:54:40.422237+0000","last_fullsized":"2024-06-10T00:54:40.422237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:39:51.374998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"100'4","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374185+0000","last_change":"2024-06-10T00:52:35.961884+0000","last_active":"2024-06-10T00:55:20.374185+0000","last_peered":"2024-06-10T00:55:20.374185+0000","last_clean":"2024-06-10T00:55:20.374185+0000","last_became_active":"2024-06-10T00:52:35.961497+0000","last_became_peered":"2024-06-10T00:52:35.961497+0000","last_unstale":"2024-06-10T00:55:20.374185+0000","last_undegraded":"2024-06-10T00:55:20.374185+0000","last_fullsized":"2024-06-10T00:55:20.374185+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:16:09.284184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553591+0000","last_change":"2024-06-10T00:52:45.990846+0000","last_active":"2024-06-10T00:54:22.553591+0000","last_peered":"2024-06-10T00:54:22.553591+0000","last_clean":"2024-06-10T00:54:22.553591+0000","last_became_active":"2024-06-10T00:52:45.990441+0000","last_became_peered":"2024-06-10T00:52:45.990441+0000","last_unstale":"2024-06-10T00:54:22.553591+0000","last_undegraded":"2024-06-10T00:54:22.553591+0000","last_fullsized":"2024-06-10T00:54:22.553591+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:19:35.690378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:39.101650+0000","last_change":"2024-06-10T00:52:32.753913+0000","last_active":"2024-06-10T00:54:39.101650+0000","last_peered":"2024-06-10T00:54:39.101650+0000","last_clean":"2024-06-10T00:54:39.101650+0000","last_became_active":"2024-06-10T00:52:32.753174+0000","last_became_peered":"2024-06-10T00:52:32.753174+0000","last_unstale":"2024-06-10T00:54:39.101650+0000","last_undegraded":"2024-06-10T00:54:39.101650+0000","last_fullsized":"2024-06-10T00:54:39.101650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:13:20.048635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"100'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.359209+0000","last_change":"2024-06-10T00:52:35.965783+0000","last_active":"2024-06-10T00:55:20.359209+0000","last_peered":"2024-06-10T00:55:20.359209+0000","last_clean":"2024-06-10T00:55:20.359209+0000","last_became_active":"2024-06-10T00:52:35.965432+0000","last_became_peered":"2024-06-10T00:52:35.965432+0000","last_unstale":"2024-06-10T00:55:20.359209+0000","last_undegraded":"2024-06-10T00:55:20.359209+0000","last_fullsized":"2024-06-10T00:55:20.359209+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:52:01.240318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553790+0000","last_change":"2024-06-10T00:52:46.241662+0000","last_active":"2024-06-10T00:54:22.553790+0000","last_peered":"2024-06-10T00:54:22.553790+0000","last_clean":"2024-06-10T00:54:22.553790+0000","last_became_active":"2024-06-10T00:52:46.241211+0000","last_became_peered":"2024-06-10T00:52:46.241211+0000","last_unstale":"2024-06-10T00:54:22.553790+0000","last_undegraded":"2024-06-10T00:54:22.553790+0000","last_fullsized":"2024-06-10T00:54:22.553790+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:12.041290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"100'8","reported_seq":24,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.367303+0000","last_change":"2024-06-10T00:53:22.200590+0000","last_active":"2024-06-10T00:55:20.367303+0000","last_peered":"2024-06-10T00:55:20.367303+0000","last_clean":"2024-06-10T00:55:20.367303+0000","last_became_active":"2024-06-10T00:53:22.200046+0000","last_became_peered":"2024-06-10T00:53:22.200046+0000","last_unstale":"2024-06-10T00:55:20.367303+0000","last_undegraded":"2024-06-10T00:55:20.367303+0000","last_fullsized":"2024-06-10T00:55:20.367303+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:43:22.378316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"98'4","reported_seq":236,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:46.578903+0000","last_change":"2024-06-10T00:52:32.959904+0000","last_active":"2024-06-10T00:56:46.578903+0000","last_peered":"2024-06-10T00:56:46.578903+0000","last_clean":"2024-06-10T00:56:46.578903+0000","last_became_active":"2024-06-10T00:52:32.959506+0000","last_became_peered":"2024-06-10T00:52:32.959506+0000","last_unstale":"2024-06-10T00:56:46.578903+0000","last_undegraded":"2024-06-10T00:56:46.578903+0000","last_fullsized":"2024-06-10T00:56:46.578903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:23:15.950345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":187,"num_read_kb":187,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404433+0000","last_change":"2024-06-10T00:52:45.993305+0000","last_active":"2024-06-10T00:54:24.404433+0000","last_peered":"2024-06-10T00:54:24.404433+0000","last_clean":"2024-06-10T00:54:24.404433+0000","last_became_active":"2024-06-10T00:52:45.992682+0000","last_became_peered":"2024-06-10T00:52:45.992682+0000","last_unstale":"2024-06-10T00:54:24.404433+0000","last_undegraded":"2024-06-10T00:54:24.404433+0000","last_fullsized":"2024-06-10T00:54:24.404433+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:49:41.817126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375925+0000","last_change":"2024-06-10T00:52:35.965893+0000","last_active":"2024-06-10T00:55:20.375925+0000","last_peered":"2024-06-10T00:55:20.375925+0000","last_clean":"2024-06-10T00:55:20.375925+0000","last_became_active":"2024-06-10T00:52:35.965426+0000","last_became_peered":"2024-06-10T00:52:35.965426+0000","last_unstale":"2024-06-10T00:55:20.375925+0000","last_undegraded":"2024-06-10T00:55:20.375925+0000","last_fullsized":"2024-06-10T00:55:20.375925+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:29:19.640836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:38.154378+0000","last_change":"2024-06-10T00:52:32.753631+0000","last_active":"2024-06-10T00:54:38.154378+0000","last_peered":"2024-06-10T00:54:38.154378+0000","last_clean":"2024-06-10T00:54:38.154378+0000","last_became_active":"2024-06-10T00:52:32.753260+0000","last_became_peered":"2024-06-10T00:52:32.753260+0000","last_unstale":"2024-06-10T00:54:38.154378+0000","last_undegraded":"2024-06-10T00:54:38.154378+0000","last_fullsized":"2024-06-10T00:54:38.154378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:12:55.932676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":369,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404055+0000","last_change":"2024-06-10T00:51:09.267750+0000","last_active":"2024-06-10T00:54:24.404055+0000","last_peered":"2024-06-10T00:54:24.404055+0000","last_clean":"2024-06-10T00:54:24.404055+0000","last_became_active":"2024-06-10T00:51:09.267246+0000","last_became_peered":"2024-06-10T00:51:09.267246+0000","last_unstale":"2024-06-10T00:54:24.404055+0000","last_undegraded":"2024-06-10T00:54:24.404055+0000","last_fullsized":"2024-06-10T00:54:24.404055+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:47:44.965452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:47:44.965452+0000","last_clean_scrub_stamp":"2024-06-10T00:47:44.965452+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:21.100090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561655+0000","last_change":"2024-06-10T00:52:46.242152+0000","last_active":"2024-06-10T00:54:22.561655+0000","last_peered":"2024-06-10T00:54:22.561655+0000","last_clean":"2024-06-10T00:54:22.561655+0000","last_became_active":"2024-06-10T00:52:46.241542+0000","last_became_peered":"2024-06-10T00:52:46.241542+0000","last_unstale":"2024-06-10T00:54:22.561655+0000","last_undegraded":"2024-06-10T00:54:22.561655+0000","last_fullsized":"2024-06-10T00:54:22.561655+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:55:54.695465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365104+0000","last_change":"2024-06-10T00:52:35.963911+0000","last_active":"2024-06-10T00:55:20.365104+0000","last_peered":"2024-06-10T00:55:20.365104+0000","last_clean":"2024-06-10T00:55:20.365104+0000","last_became_active":"2024-06-10T00:52:35.963571+0000","last_became_peered":"2024-06-10T00:52:35.963571+0000","last_unstale":"2024-06-10T00:55:20.365104+0000","last_undegraded":"2024-06-10T00:55:20.365104+0000","last_fullsized":"2024-06-10T00:55:20.365104+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:27:16.757428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.558529+0000","last_change":"2024-06-10T00:53:22.129035+0000","last_active":"2024-06-10T00:54:22.558529+0000","last_peered":"2024-06-10T00:54:22.558529+0000","last_clean":"2024-06-10T00:54:22.558529+0000","last_became_active":"2024-06-10T00:53:22.128561+0000","last_became_peered":"2024-06-10T00:53:22.128561+0000","last_unstale":"2024-06-10T00:54:22.558529+0000","last_undegraded":"2024-06-10T00:54:22.558529+0000","last_fullsized":"2024-06-10T00:54:22.558529+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:02:43.313118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.558735+0000","last_change":"2024-06-10T00:52:45.996907+0000","last_active":"2024-06-10T00:54:22.558735+0000","last_peered":"2024-06-10T00:54:22.558735+0000","last_clean":"2024-06-10T00:54:22.558735+0000","last_became_active":"2024-06-10T00:52:45.996554+0000","last_became_peered":"2024-06-10T00:52:45.996554+0000","last_unstale":"2024-06-10T00:54:22.558735+0000","last_undegraded":"2024-06-10T00:54:22.558735+0000","last_fullsized":"2024-06-10T00:54:22.558735+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:00:46.824071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'1","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:38.584029+0000","last_change":"2024-06-10T00:52:32.960223+0000","last_active":"2024-06-10T00:54:38.584029+0000","last_peered":"2024-06-10T00:54:38.584029+0000","last_clean":"2024-06-10T00:54:38.584029+0000","last_became_active":"2024-06-10T00:52:32.959871+0000","last_became_peered":"2024-06-10T00:52:32.959871+0000","last_unstale":"2024-06-10T00:54:38.584029+0000","last_undegraded":"2024-06-10T00:54:38.584029+0000","last_fullsized":"2024-06-10T00:54:38.584029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:18:17.099140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.369679+0000","last_change":"2024-06-10T00:52:35.962516+0000","last_active":"2024-06-10T00:55:20.369679+0000","last_peered":"2024-06-10T00:55:20.369679+0000","last_clean":"2024-06-10T00:55:20.369679+0000","last_became_active":"2024-06-10T00:52:35.962187+0000","last_became_peered":"2024-06-10T00:52:35.962187+0000","last_unstale":"2024-06-10T00:55:20.369679+0000","last_undegraded":"2024-06-10T00:55:20.369679+0000","last_fullsized":"2024-06-10T00:55:20.369679+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:10:44.295192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:48.629265+0000","last_change":"2024-06-10T00:52:46.241554+0000","last_active":"2024-06-10T00:54:48.629265+0000","last_peered":"2024-06-10T00:54:48.629265+0000","last_clean":"2024-06-10T00:54:48.629265+0000","last_became_active":"2024-06-10T00:52:46.241142+0000","last_became_peered":"2024-06-10T00:52:46.241142+0000","last_unstale":"2024-06-10T00:54:48.629265+0000","last_undegraded":"2024-06-10T00:54:48.629265+0000","last_fullsized":"2024-06-10T00:54:48.629265+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:58:07.233080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'13","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:29.987897+0000","last_change":"2024-06-10T00:52:32.761854+0000","last_active":"2024-06-10T00:56:29.987897+0000","last_peered":"2024-06-10T00:56:29.987897+0000","last_clean":"2024-06-10T00:56:29.987897+0000","last_became_active":"2024-06-10T00:52:32.761522+0000","last_became_peered":"2024-06-10T00:52:32.761522+0000","last_unstale":"2024-06-10T00:56:29.987897+0000","last_undegraded":"2024-06-10T00:56:29.987897+0000","last_fullsized":"2024-06-10T00:56:29.987897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:06:43.269001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.370762+0000","last_change":"2024-06-10T00:52:35.959250+0000","last_active":"2024-06-10T00:55:20.370762+0000","last_peered":"2024-06-10T00:55:20.370762+0000","last_clean":"2024-06-10T00:55:20.370762+0000","last_became_active":"2024-06-10T00:52:35.958925+0000","last_became_peered":"2024-06-10T00:52:35.958925+0000","last_unstale":"2024-06-10T00:55:20.370762+0000","last_undegraded":"2024-06-10T00:55:20.370762+0000","last_fullsized":"2024-06-10T00:55:20.370762+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:11:12.355505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554191+0000","last_change":"2024-06-10T00:52:45.991819+0000","last_active":"2024-06-10T00:54:22.554191+0000","last_peered":"2024-06-10T00:54:22.554191+0000","last_clean":"2024-06-10T00:54:22.554191+0000","last_became_active":"2024-06-10T00:52:45.991408+0000","last_became_peered":"2024-06-10T00:52:45.991408+0000","last_unstale":"2024-06-10T00:54:22.554191+0000","last_undegraded":"2024-06-10T00:54:22.554191+0000","last_fullsized":"2024-06-10T00:54:22.554191+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:16:34.912359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668551+0000","last_change":"2024-06-10T00:52:32.869094+0000","last_active":"2024-06-10T00:54:22.668551+0000","last_peered":"2024-06-10T00:54:22.668551+0000","last_clean":"2024-06-10T00:54:22.668551+0000","last_became_active":"2024-06-10T00:52:32.868664+0000","last_became_peered":"2024-06-10T00:52:32.868664+0000","last_unstale":"2024-06-10T00:54:22.668551+0000","last_undegraded":"2024-06-10T00:54:22.668551+0000","last_fullsized":"2024-06-10T00:54:22.668551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:50:38.051090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374722+0000","last_change":"2024-06-10T00:52:35.952356+0000","last_active":"2024-06-10T00:55:20.374722+0000","last_peered":"2024-06-10T00:55:20.374722+0000","last_clean":"2024-06-10T00:55:20.374722+0000","last_became_active":"2024-06-10T00:52:35.952002+0000","last_became_peered":"2024-06-10T00:52:35.952002+0000","last_unstale":"2024-06-10T00:55:20.374722+0000","last_undegraded":"2024-06-10T00:55:20.374722+0000","last_fullsized":"2024-06-10T00:55:20.374722+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:12:28.581522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668366+0000","last_change":"2024-06-10T00:52:46.241773+0000","last_active":"2024-06-10T00:54:22.668366+0000","last_peered":"2024-06-10T00:54:22.668366+0000","last_clean":"2024-06-10T00:54:22.668366+0000","last_became_active":"2024-06-10T00:52:46.241272+0000","last_became_peered":"2024-06-10T00:52:46.241272+0000","last_unstale":"2024-06-10T00:54:22.668366+0000","last_undegraded":"2024-06-10T00:54:22.668366+0000","last_fullsized":"2024-06-10T00:54:22.668366+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:07:28.337536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561360+0000","last_change":"2024-06-10T00:52:32.960100+0000","last_active":"2024-06-10T00:54:22.561360+0000","last_peered":"2024-06-10T00:54:22.561360+0000","last_clean":"2024-06-10T00:54:22.561360+0000","last_became_active":"2024-06-10T00:52:32.959779+0000","last_became_peered":"2024-06-10T00:52:32.959779+0000","last_unstale":"2024-06-10T00:54:22.561360+0000","last_undegraded":"2024-06-10T00:54:22.561360+0000","last_fullsized":"2024-06-10T00:54:22.561360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:09.555753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'2","reported_seq":40,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.326494+0000","last_change":"2024-06-10T00:52:35.955586+0000","last_active":"2024-06-10T00:55:20.326494+0000","last_peered":"2024-06-10T00:55:20.326494+0000","last_clean":"2024-06-10T00:55:20.326494+0000","last_became_active":"2024-06-10T00:52:35.955216+0000","last_became_peered":"2024-06-10T00:52:35.955216+0000","last_unstale":"2024-06-10T00:55:20.326494+0000","last_undegraded":"2024-06-10T00:55:20.326494+0000","last_fullsized":"2024-06-10T00:55:20.326494+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:23:12.197182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:23.582601+0000","last_change":"2024-06-10T00:54:22.602670+0000","last_active":"2024-06-10T00:54:23.582601+0000","last_peered":"2024-06-10T00:54:23.582601+0000","last_clean":"2024-06-10T00:54:23.582601+0000","last_became_active":"2024-06-10T00:54:22.602218+0000","last_became_peered":"2024-06-10T00:54:22.602218+0000","last_unstale":"2024-06-10T00:54:23.582601+0000","last_undegraded":"2024-06-10T00:54:23.582601+0000","last_fullsized":"2024-06-10T00:54:23.582601+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:07.399025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"98'3","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601358+0000","last_change":"2024-06-10T00:52:32.753444+0000","last_active":"2024-06-10T00:54:22.601358+0000","last_peered":"2024-06-10T00:54:22.601358+0000","last_clean":"2024-06-10T00:54:22.601358+0000","last_became_active":"2024-06-10T00:52:32.753149+0000","last_became_peered":"2024-06-10T00:52:32.753149+0000","last_unstale":"2024-06-10T00:54:22.601358+0000","last_undegraded":"2024-06-10T00:54:22.601358+0000","last_fullsized":"2024-06-10T00:54:22.601358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:28.241950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365900+0000","last_change":"2024-06-10T00:52:35.966718+0000","last_active":"2024-06-10T00:55:20.365900+0000","last_peered":"2024-06-10T00:55:20.365900+0000","last_clean":"2024-06-10T00:55:20.365900+0000","last_became_active":"2024-06-10T00:52:35.965907+0000","last_became_peered":"2024-06-10T00:52:35.965907+0000","last_unstale":"2024-06-10T00:55:20.365900+0000","last_undegraded":"2024-06-10T00:55:20.365900+0000","last_fullsized":"2024-06-10T00:55:20.365900+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:21:34.261306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668092+0000","last_change":"2024-06-10T00:52:45.991034+0000","last_active":"2024-06-10T00:54:22.668092+0000","last_peered":"2024-06-10T00:54:22.668092+0000","last_clean":"2024-06-10T00:54:22.668092+0000","last_became_active":"2024-06-10T00:52:45.990546+0000","last_became_peered":"2024-06-10T00:52:45.990546+0000","last_unstale":"2024-06-10T00:54:22.668092+0000","last_undegraded":"2024-06-10T00:54:22.668092+0000","last_fullsized":"2024-06-10T00:54:22.668092+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:09:23.412922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"100'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:42.999786+0000","last_change":"2024-06-10T00:52:32.764024+0000","last_active":"2024-06-10T00:54:42.999786+0000","last_peered":"2024-06-10T00:54:42.999786+0000","last_clean":"2024-06-10T00:54:42.999786+0000","last_became_active":"2024-06-10T00:52:32.763449+0000","last_became_peered":"2024-06-10T00:52:32.763449+0000","last_unstale":"2024-06-10T00:54:42.999786+0000","last_undegraded":"2024-06-10T00:54:42.999786+0000","last_fullsized":"2024-06-10T00:54:42.999786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:50:23.605873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373684+0000","last_change":"2024-06-10T00:52:35.955570+0000","last_active":"2024-06-10T00:55:20.373684+0000","last_peered":"2024-06-10T00:55:20.373684+0000","last_clean":"2024-06-10T00:55:20.373684+0000","last_became_active":"2024-06-10T00:52:35.955248+0000","last_became_peered":"2024-06-10T00:52:35.955248+0000","last_unstale":"2024-06-10T00:55:20.373684+0000","last_undegraded":"2024-06-10T00:55:20.373684+0000","last_fullsized":"2024-06-10T00:55:20.373684+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:26.124499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668028+0000","last_change":"2024-06-10T00:52:46.241776+0000","last_active":"2024-06-10T00:54:22.668028+0000","last_peered":"2024-06-10T00:54:22.668028+0000","last_clean":"2024-06-10T00:54:22.668028+0000","last_became_active":"2024-06-10T00:52:46.241276+0000","last_became_peered":"2024-06-10T00:52:46.241276+0000","last_unstale":"2024-06-10T00:54:22.668028+0000","last_undegraded":"2024-06-10T00:54:22.668028+0000","last_fullsized":"2024-06-10T00:54:22.668028+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:08.027938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":58,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556321+0000","last_change":"2024-06-10T00:53:22.204034+0000","last_active":"2024-06-10T00:54:22.556321+0000","last_peered":"2024-06-10T00:54:22.556321+0000","last_clean":"2024-06-10T00:54:22.556321+0000","last_became_active":"2024-06-10T00:53:22.199988+0000","last_became_peered":"2024-06-10T00:53:22.199988+0000","last_unstale":"2024-06-10T00:54:22.556321+0000","last_undegraded":"2024-06-10T00:54:22.556321+0000","last_fullsized":"2024-06-10T00:54:22.556321+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:39:54.012360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373708+0000","last_change":"2024-06-10T00:52:35.959182+0000","last_active":"2024-06-10T00:55:20.373708+0000","last_peered":"2024-06-10T00:55:20.373708+0000","last_clean":"2024-06-10T00:55:20.373708+0000","last_became_active":"2024-06-10T00:52:35.958828+0000","last_became_peered":"2024-06-10T00:52:35.958828+0000","last_unstale":"2024-06-10T00:55:20.373708+0000","last_undegraded":"2024-06-10T00:55:20.373708+0000","last_fullsized":"2024-06-10T00:55:20.373708+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:29:58.857659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561603+0000","last_change":"2024-06-10T00:52:45.989739+0000","last_active":"2024-06-10T00:54:22.561603+0000","last_peered":"2024-06-10T00:54:22.561603+0000","last_clean":"2024-06-10T00:54:22.561603+0000","last_became_active":"2024-06-10T00:52:45.989279+0000","last_became_peered":"2024-06-10T00:52:45.989279+0000","last_unstale":"2024-06-10T00:54:22.561603+0000","last_undegraded":"2024-06-10T00:54:22.561603+0000","last_fullsized":"2024-06-10T00:54:22.561603+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:19:17.551947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554798+0000","last_change":"2024-06-10T00:52:32.959662+0000","last_active":"2024-06-10T00:54:22.554798+0000","last_peered":"2024-06-10T00:54:22.554798+0000","last_clean":"2024-06-10T00:54:22.554798+0000","last_became_active":"2024-06-10T00:52:32.959286+0000","last_became_peered":"2024-06-10T00:52:32.959286+0000","last_unstale":"2024-06-10T00:54:22.554798+0000","last_undegraded":"2024-06-10T00:54:22.554798+0000","last_fullsized":"2024-06-10T00:54:22.554798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:26:04.361567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373843+0000","last_change":"2024-06-10T00:52:35.960137+0000","last_active":"2024-06-10T00:55:20.373843+0000","last_peered":"2024-06-10T00:55:20.373843+0000","last_clean":"2024-06-10T00:55:20.373843+0000","last_became_active":"2024-06-10T00:52:35.959584+0000","last_became_peered":"2024-06-10T00:52:35.959584+0000","last_unstale":"2024-06-10T00:55:20.373843+0000","last_undegraded":"2024-06-10T00:55:20.373843+0000","last_fullsized":"2024-06-10T00:55:20.373843+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:23:38.702685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878137+0000","last_change":"2024-06-10T00:52:46.002268+0000","last_active":"2024-06-10T00:54:22.878137+0000","last_peered":"2024-06-10T00:54:22.878137+0000","last_clean":"2024-06-10T00:54:22.878137+0000","last_became_active":"2024-06-10T00:52:46.001769+0000","last_became_peered":"2024-06-10T00:52:46.001769+0000","last_unstale":"2024-06-10T00:54:22.878137+0000","last_undegraded":"2024-06-10T00:54:22.878137+0000","last_fullsized":"2024-06-10T00:54:22.878137+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:33:51.365951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561629+0000","last_change":"2024-06-10T00:52:32.757112+0000","last_active":"2024-06-10T00:54:22.561629+0000","last_peered":"2024-06-10T00:54:22.561629+0000","last_clean":"2024-06-10T00:54:22.561629+0000","last_became_active":"2024-06-10T00:52:32.755589+0000","last_became_peered":"2024-06-10T00:52:32.755589+0000","last_unstale":"2024-06-10T00:54:22.561629+0000","last_undegraded":"2024-06-10T00:54:22.561629+0000","last_fullsized":"2024-06-10T00:54:22.561629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:15:12.814489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364097+0000","last_change":"2024-06-10T00:52:35.955925+0000","last_active":"2024-06-10T00:55:20.364097+0000","last_peered":"2024-06-10T00:55:20.364097+0000","last_clean":"2024-06-10T00:55:20.364097+0000","last_became_active":"2024-06-10T00:52:35.955556+0000","last_became_peered":"2024-06-10T00:52:35.955556+0000","last_unstale":"2024-06-10T00:55:20.364097+0000","last_undegraded":"2024-06-10T00:55:20.364097+0000","last_fullsized":"2024-06-10T00:55:20.364097+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:42:18.324484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561041+0000","last_change":"2024-06-10T00:52:45.988810+0000","last_active":"2024-06-10T00:54:22.561041+0000","last_peered":"2024-06-10T00:54:22.561041+0000","last_clean":"2024-06-10T00:54:22.561041+0000","last_became_active":"2024-06-10T00:52:45.988041+0000","last_became_peered":"2024-06-10T00:52:45.988041+0000","last_unstale":"2024-06-10T00:54:22.561041+0000","last_undegraded":"2024-06-10T00:54:22.561041+0000","last_fullsized":"2024-06-10T00:54:22.561041+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:14:08.979802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553851+0000","last_change":"2024-06-10T00:52:32.765493+0000","last_active":"2024-06-10T00:54:22.553851+0000","last_peered":"2024-06-10T00:54:22.553851+0000","last_clean":"2024-06-10T00:54:22.553851+0000","last_became_active":"2024-06-10T00:52:32.765193+0000","last_became_peered":"2024-06-10T00:52:32.765193+0000","last_unstale":"2024-06-10T00:54:22.553851+0000","last_undegraded":"2024-06-10T00:54:22.553851+0000","last_fullsized":"2024-06-10T00:54:22.553851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:38:11.554673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374319+0000","last_change":"2024-06-10T00:52:35.960216+0000","last_active":"2024-06-10T00:55:20.374319+0000","last_peered":"2024-06-10T00:55:20.374319+0000","last_clean":"2024-06-10T00:55:20.374319+0000","last_became_active":"2024-06-10T00:52:35.959664+0000","last_became_peered":"2024-06-10T00:52:35.959664+0000","last_unstale":"2024-06-10T00:55:20.374319+0000","last_undegraded":"2024-06-10T00:55:20.374319+0000","last_fullsized":"2024-06-10T00:55:20.374319+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:41:58.086330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375776+0000","last_change":"2024-06-10T00:52:35.967043+0000","last_active":"2024-06-10T00:55:20.375776+0000","last_peered":"2024-06-10T00:55:20.375776+0000","last_clean":"2024-06-10T00:55:20.375776+0000","last_became_active":"2024-06-10T00:52:35.966366+0000","last_became_peered":"2024-06-10T00:52:35.966366+0000","last_unstale":"2024-06-10T00:55:20.375776+0000","last_undegraded":"2024-06-10T00:55:20.375776+0000","last_fullsized":"2024-06-10T00:55:20.375776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:27:06.047113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"100'12","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.369850+0000","last_change":"2024-06-10T00:52:35.960517+0000","last_active":"2024-06-10T00:55:20.369850+0000","last_peered":"2024-06-10T00:55:20.369850+0000","last_clean":"2024-06-10T00:55:20.369850+0000","last_became_active":"2024-06-10T00:52:35.957895+0000","last_became_peered":"2024-06-10T00:52:35.957895+0000","last_unstale":"2024-06-10T00:55:20.369850+0000","last_undegraded":"2024-06-10T00:55:20.369850+0000","last_fullsized":"2024-06-10T00:55:20.369850+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:04:34.438070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374305+0000","last_change":"2024-06-10T00:52:35.962163+0000","last_active":"2024-06-10T00:55:20.374305+0000","last_peered":"2024-06-10T00:55:20.374305+0000","last_clean":"2024-06-10T00:55:20.374305+0000","last_became_active":"2024-06-10T00:52:35.961768+0000","last_became_peered":"2024-06-10T00:52:35.961768+0000","last_unstale":"2024-06-10T00:55:20.374305+0000","last_undegraded":"2024-06-10T00:55:20.374305+0000","last_fullsized":"2024-06-10T00:55:20.374305+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:35.793995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"100'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374044+0000","last_change":"2024-06-10T00:53:22.130686+0000","last_active":"2024-06-10T00:55:20.374044+0000","last_peered":"2024-06-10T00:55:20.374044+0000","last_clean":"2024-06-10T00:55:20.374044+0000","last_became_active":"2024-06-10T00:53:22.130198+0000","last_became_peered":"2024-06-10T00:53:22.130198+0000","last_unstale":"2024-06-10T00:55:20.374044+0000","last_undegraded":"2024-06-10T00:55:20.374044+0000","last_fullsized":"2024-06-10T00:55:20.374044+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:50:26.773977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372642+0000","last_change":"2024-06-10T00:52:35.957255+0000","last_active":"2024-06-10T00:55:20.372642+0000","last_peered":"2024-06-10T00:55:20.372642+0000","last_clean":"2024-06-10T00:55:20.372642+0000","last_became_active":"2024-06-10T00:52:35.956923+0000","last_became_peered":"2024-06-10T00:52:35.956923+0000","last_unstale":"2024-06-10T00:55:20.372642+0000","last_undegraded":"2024-06-10T00:55:20.372642+0000","last_fullsized":"2024-06-10T00:55:20.372642+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:33:25.743503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375063+0000","last_change":"2024-06-10T00:52:35.964258+0000","last_active":"2024-06-10T00:55:20.375063+0000","last_peered":"2024-06-10T00:55:20.375063+0000","last_clean":"2024-06-10T00:55:20.375063+0000","last_became_active":"2024-06-10T00:52:35.963862+0000","last_became_peered":"2024-06-10T00:52:35.963862+0000","last_unstale":"2024-06-10T00:55:20.375063+0000","last_undegraded":"2024-06-10T00:55:20.375063+0000","last_fullsized":"2024-06-10T00:55:20.375063+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:48:58.554411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.377085+0000","last_change":"2024-06-10T00:52:35.965046+0000","last_active":"2024-06-10T00:55:20.377085+0000","last_peered":"2024-06-10T00:55:20.377085+0000","last_clean":"2024-06-10T00:55:20.377085+0000","last_became_active":"2024-06-10T00:52:35.964496+0000","last_became_peered":"2024-06-10T00:52:35.964496+0000","last_unstale":"2024-06-10T00:55:20.377085+0000","last_undegraded":"2024-06-10T00:55:20.377085+0000","last_fullsized":"2024-06-10T00:55:20.377085+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:09:42.979731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"100'2","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.370981+0000","last_change":"2024-06-10T00:52:35.964287+0000","last_active":"2024-06-10T00:55:20.370981+0000","last_peered":"2024-06-10T00:55:20.370981+0000","last_clean":"2024-06-10T00:55:20.370981+0000","last_became_active":"2024-06-10T00:52:35.963920+0000","last_became_peered":"2024-06-10T00:52:35.963920+0000","last_unstale":"2024-06-10T00:55:20.370981+0000","last_undegraded":"2024-06-10T00:55:20.370981+0000","last_fullsized":"2024-06-10T00:55:20.370981+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:41:54.593274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.367606+0000","last_change":"2024-06-10T00:52:35.961440+0000","last_active":"2024-06-10T00:55:20.367606+0000","last_peered":"2024-06-10T00:55:20.367606+0000","last_clean":"2024-06-10T00:55:20.367606+0000","last_became_active":"2024-06-10T00:52:35.960066+0000","last_became_peered":"2024-06-10T00:52:35.960066+0000","last_unstale":"2024-06-10T00:55:20.367606+0000","last_undegraded":"2024-06-10T00:55:20.367606+0000","last_fullsized":"2024-06-10T00:55:20.367606+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:01.024596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372367+0000","last_change":"2024-06-10T00:52:35.964464+0000","last_active":"2024-06-10T00:55:20.372367+0000","last_peered":"2024-06-10T00:55:20.372367+0000","last_clean":"2024-06-10T00:55:20.372367+0000","last_became_active":"2024-06-10T00:52:35.964159+0000","last_became_peered":"2024-06-10T00:52:35.964159+0000","last_unstale":"2024-06-10T00:55:20.372367+0000","last_undegraded":"2024-06-10T00:55:20.372367+0000","last_fullsized":"2024-06-10T00:55:20.372367+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:18.660859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.377213+0000","last_change":"2024-06-10T00:52:35.961539+0000","last_active":"2024-06-10T00:55:20.377213+0000","last_peered":"2024-06-10T00:55:20.377213+0000","last_clean":"2024-06-10T00:55:20.377213+0000","last_became_active":"2024-06-10T00:52:35.958738+0000","last_became_peered":"2024-06-10T00:52:35.958738+0000","last_unstale":"2024-06-10T00:55:20.377213+0000","last_undegraded":"2024-06-10T00:55:20.377213+0000","last_fullsized":"2024-06-10T00:55:20.377213+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:06:48.057551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374134+0000","last_change":"2024-06-10T00:52:35.962563+0000","last_active":"2024-06-10T00:55:20.374134+0000","last_peered":"2024-06-10T00:55:20.374134+0000","last_clean":"2024-06-10T00:55:20.374134+0000","last_became_active":"2024-06-10T00:52:35.962255+0000","last_became_peered":"2024-06-10T00:52:35.962255+0000","last_unstale":"2024-06-10T00:55:20.374134+0000","last_undegraded":"2024-06-10T00:55:20.374134+0000","last_fullsized":"2024-06-10T00:55:20.374134+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:33:35.689446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372437+0000","last_change":"2024-06-10T00:52:35.959616+0000","last_active":"2024-06-10T00:55:20.372437+0000","last_peered":"2024-06-10T00:55:20.372437+0000","last_clean":"2024-06-10T00:55:20.372437+0000","last_became_active":"2024-06-10T00:52:35.959251+0000","last_became_peered":"2024-06-10T00:52:35.959251+0000","last_unstale":"2024-06-10T00:55:20.372437+0000","last_undegraded":"2024-06-10T00:55:20.372437+0000","last_fullsized":"2024-06-10T00:55:20.372437+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:39:23.933279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284806,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":82322,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1172,"num_read_kb":675,"num_write":191,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":246966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":163,"ondisk_log_size":163,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351044,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":573101,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612681,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28248,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716904,"statfs":{"total":95995035648,"available":95966109696,"internally_reserved":0,"allocated":1261568,"data_stored":1238242,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776206,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":577433,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037842,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":573257,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299480,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36476,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708676,"statfs":{"total":95995035648,"available":95957684224,"internally_reserved":0,"allocated":1167360,"data_stored":1169614,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463004,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":568513,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574966,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36488,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708664,"statfs":{"total":95995035648,"available":95957671936,"internally_reserved":0,"allocated":1179648,"data_stored":646296,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738491,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":569901,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32344,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712808,"statfs":{"total":95995035648,"available":95961915392,"internally_reserved":0,"allocated":1196032,"data_stored":648373,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149803,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":568707,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888166,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":573275,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724642,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":1165202,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":77873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":79950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":79451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T00:56:54.096 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-10T00:56:54.097 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-10T00:56:54.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:54 smithi167 ceph-mon[36917]: pgmap v436: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-06-10T00:56:54.352 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:56:54.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:54 smithi094 ceph-mon[36708]: pgmap v436: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-06-10T00:56:54.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:54 smithi037 ceph-mon[32192]: pgmap v436: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-06-10T00:56:55.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:55 smithi167 ceph-mon[36917]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:56:55.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:55 smithi094 ceph-mon[36708]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:56:55.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:55 smithi037 ceph-mon[32192]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:56:56.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.117+0000 7f6329074700 1 -- 172.21.15.37:0/2453825191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6324107400 msgr2=0x7f63241077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:56.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.117+0000 7f6329074700 1 --2- 172.21.15.37:0/2453825191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6324107400 0x7f63241077e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6314009990 tx=0x7f6314009ca0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.117+0000 7f6329074700 1 -- 172.21.15.37:0/2453825191 shutdown_connections 2024-06-10T00:56:56.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.117+0000 7f6329074700 1 --2- 172.21.15.37:0/2453825191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6324101d50 0x7f632410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.117+0000 7f6329074700 1 --2- 172.21.15.37:0/2453825191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6324107400 0x7f63241077e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.117+0000 7f6329074700 1 --2- 172.21.15.37:0/2453825191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6324101390 0x7f6324101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.117+0000 7f6329074700 1 -- 172.21.15.37:0/2453825191 >> 172.21.15.37:0/2453825191 conn(0x7f6324078340 msgr2=0x7f6324078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:56.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.117+0000 7f6329074700 1 -- 172.21.15.37:0/2453825191 shutdown_connections 2024-06-10T00:56:56.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.117+0000 7f6329074700 1 -- 172.21.15.37:0/2453825191 wait complete. 2024-06-10T00:56:56.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.118+0000 7f6329074700 1 Processor -- start 2024-06-10T00:56:56.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.118+0000 7f6329074700 1 -- start start 2024-06-10T00:56:56.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.118+0000 7f6329074700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6324101390 0x7f6324199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:56.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.118+0000 7f6329074700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6324101d50 0x7f632419a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:56.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.118+0000 7f6329074700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6324107400 0x7f632419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:56.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.118+0000 7f6329074700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f63241116a0 con 0x7f6324101d50 2024-06-10T00:56:56.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.118+0000 7f6329074700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6324111520 con 0x7f6324107400 2024-06-10T00:56:56.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.118+0000 7f6329074700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6324111820 con 0x7f6324101390 2024-06-10T00:56:56.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.118+0000 7f632259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6324101d50 0x7f632419a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:56.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.118+0000 7f632259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6324101d50 0x7f632419a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39270/0 (socket says 172.21.15.37:39270) 2024-06-10T00:56:56.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.118+0000 7f632259c700 1 -- 172.21.15.37:0/1646275343 learned_addr learned my addr 172.21.15.37:0/1646275343 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:56.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.118+0000 7f6322d9d700 1 --2- 172.21.15.37:0/1646275343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6324101390 0x7f6324199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:56.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.119+0000 7f632359e700 1 --2- 172.21.15.37:0/1646275343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6324107400 0x7f632419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:56.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.119+0000 7f6322d9d700 1 -- 172.21.15.37:0/1646275343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6324107400 msgr2=0x7f632419a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:56.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.119+0000 7f6322d9d700 1 --2- 172.21.15.37:0/1646275343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6324107400 0x7f632419a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.119+0000 7f6322d9d700 1 -- 172.21.15.37:0/1646275343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6324101d50 msgr2=0x7f632419a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:56.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.119+0000 7f6322d9d700 1 --2- 172.21.15.37:0/1646275343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6324101d50 0x7f632419a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.119+0000 7f6322d9d700 1 -- 172.21.15.37:0/1646275343 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63241a4f90 con 0x7f6324101390 2024-06-10T00:56:56.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.119+0000 7f632259c700 1 --2- 172.21.15.37:0/1646275343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6324101d50 0x7f632419a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:56:56.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.119+0000 7f6322d9d700 1 --2- 172.21.15.37:0/1646275343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6324101390 0x7f6324199ba0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f63140059d0 tx=0x7f63140047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:56.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.119+0000 7f631bfff700 1 -- 172.21.15.37:0/1646275343 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6314027070 con 0x7f6324101390 2024-06-10T00:56:56.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.119+0000 7f631bfff700 1 -- 172.21.15.37:0/1646275343 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6314003740 con 0x7f6324101390 2024-06-10T00:56:56.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.119+0000 7f6329074700 1 -- 172.21.15.37:0/1646275343 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63241a5220 con 0x7f6324101390 2024-06-10T00:56:56.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.119+0000 7f631bfff700 1 -- 172.21.15.37:0/1646275343 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6314022b50 con 0x7f6324101390 2024-06-10T00:56:56.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.119+0000 7f6329074700 1 -- 172.21.15.37:0/1646275343 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63241a5670 con 0x7f6324101390 2024-06-10T00:56:56.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.120+0000 7f6329074700 1 -- 172.21.15.37:0/1646275343 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6324069180 con 0x7f6324101390 2024-06-10T00:56:56.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.122+0000 7f631bfff700 1 -- 172.21.15.37:0/1646275343 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f6314022d10 con 0x7f6324101390 2024-06-10T00:56:56.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.122+0000 7f631bfff700 1 --2- 172.21.15.37:0/1646275343 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f63100a2cb0 0x7f63100a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:56.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.122+0000 7f632259c700 1 --2- 172.21.15.37:0/1646275343 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f63100a2cb0 0x7f63100a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:56.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.122+0000 7f631bfff700 1 -- 172.21.15.37:0/1646275343 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f63140d0ad0 con 0x7f6324101390 2024-06-10T00:56:56.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.123+0000 7f632259c700 1 --2- 172.21.15.37:0/1646275343 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f63100a2cb0 0x7f63100a5170 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6324078840 tx=0x7f630c007480 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:56.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.126+0000 7f631bfff700 1 -- 172.21.15.37:0/1646275343 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6314013070 con 0x7f6324101390 2024-06-10T00:56:56.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:56 smithi037 ceph-mon[32192]: pgmap v437: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-06-10T00:56:56.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.327+0000 7f6329074700 1 -- 172.21.15.37:0/1646275343 --> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f63240628d0 con 0x7f63100a2cb0 2024-06-10T00:56:56.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.336+0000 7f631bfff700 1 -- 172.21.15.37:0/1646275343 <== mgr.14148 v2:172.21.15.37:6800/2387001309 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340960 (secure 0 0 0) 0x7f63240628d0 con 0x7f63100a2cb0 2024-06-10T00:56:56.341 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:56:56.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.341+0000 7f6329074700 1 -- 172.21.15.37:0/1646275343 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f63100a2cb0 msgr2=0x7f63100a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:56.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.341+0000 7f6329074700 1 --2- 172.21.15.37:0/1646275343 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f63100a2cb0 0x7f63100a5170 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6324078840 tx=0x7f630c007480 comp rx=0 tx=0).stop 2024-06-10T00:56:56.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.341+0000 7f6329074700 1 -- 172.21.15.37:0/1646275343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6324101390 msgr2=0x7f6324199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:56.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.341+0000 7f6329074700 1 --2- 172.21.15.37:0/1646275343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6324101390 0x7f6324199ba0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f63140059d0 tx=0x7f63140047b0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.341+0000 7f6329074700 1 -- 172.21.15.37:0/1646275343 shutdown_connections 2024-06-10T00:56:56.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.341+0000 7f6329074700 1 --2- 172.21.15.37:0/1646275343 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f63100a2cb0 0x7f63100a5170 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.341+0000 7f6329074700 1 --2- 172.21.15.37:0/1646275343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6324107400 0x7f632419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.341+0000 7f6329074700 1 --2- 172.21.15.37:0/1646275343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6324101390 0x7f6324199ba0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.341+0000 7f6329074700 1 --2- 172.21.15.37:0/1646275343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6324101d50 0x7f632419a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.341+0000 7f6329074700 1 -- 172.21.15.37:0/1646275343 >> 172.21.15.37:0/1646275343 conn(0x7f6324078340 msgr2=0x7f63240fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:56.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.341+0000 7f6329074700 1 -- 172.21.15.37:0/1646275343 shutdown_connections 2024-06-10T00:56:56.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:56.341+0000 7f6329074700 1 -- 172.21.15.37:0/1646275343 wait complete. 2024-06-10T00:56:56.344 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-06-10T00:56:56.344 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:56 smithi167 ceph-mon[36917]: pgmap v437: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-06-10T00:56:56.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:56 smithi094 ceph-mon[36708]: pgmap v437: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-06-10T00:56:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.365+0000 7fab7ad6a640 1 -- 172.21.15.37:0/3270616045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab741035b0 msgr2=0x7fab74109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.365+0000 7fab7ad6a640 1 --2- 172.21.15.37:0/3270616045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab741035b0 0x7fab74109e40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009ca0 tx=0x7fab6c02a8c0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.365+0000 7fab7ad6a640 1 -- 172.21.15.37:0/3270616045 shutdown_connections 2024-06-10T00:56:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.365+0000 7fab7ad6a640 1 --2- 172.21.15.37:0/3270616045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab741035b0 0x7fab74109e40 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.365+0000 7fab7ad6a640 1 --2- 172.21.15.37:0/3270616045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab74102bf0 0x7fab74103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.365+0000 7fab7ad6a640 1 --2- 172.21.15.37:0/3270616045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab74101a00 0x7fab74101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.365+0000 7fab7ad6a640 1 -- 172.21.15.37:0/3270616045 >> 172.21.15.37:0/3270616045 conn(0x7fab740fcf90 msgr2=0x7fab740ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.365+0000 7fab7ad6a640 1 -- 172.21.15.37:0/3270616045 shutdown_connections 2024-06-10T00:56:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.365+0000 7fab7ad6a640 1 -- 172.21.15.37:0/3270616045 wait complete. 2024-06-10T00:56:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.365+0000 7fab7ad6a640 1 Processor -- start 2024-06-10T00:56:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.365+0000 7fab7ad6a640 1 -- start start 2024-06-10T00:56:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.365+0000 7fab7ad6a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab74101a00 0x7fab74075750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.365+0000 7fab7ad6a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab74102bf0 0x7fab74073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.365+0000 7fab7ad6a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab741035b0 0x7fab740742d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.365+0000 7fab7ad6a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fab7410d680 con 0x7fab741035b0 2024-06-10T00:56:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.365+0000 7fab7ad6a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab7410d500 con 0x7fab74102bf0 2024-06-10T00:56:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.365+0000 7fab7ad6a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fab7410d800 con 0x7fab74101a00 2024-06-10T00:56:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.366+0000 7fab79307640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab741035b0 0x7fab740742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.366+0000 7fab79307640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab741035b0 0x7fab740742d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39288/0 (socket says 172.21.15.37:39288) 2024-06-10T00:56:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.366+0000 7fab79307640 1 -- 172.21.15.37:0/1716904557 learned_addr learned my addr 172.21.15.37:0/1716904557 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.366+0000 7fab6bfff640 1 --2- 172.21.15.37:0/1716904557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab74102bf0 0x7fab74073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.366+0000 7fab78b06640 1 --2- 172.21.15.37:0/1716904557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab74101a00 0x7fab74075750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.366+0000 7fab79307640 1 -- 172.21.15.37:0/1716904557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab74101a00 msgr2=0x7fab74075750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.366+0000 7fab79307640 1 --2- 172.21.15.37:0/1716904557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab74101a00 0x7fab74075750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.366+0000 7fab79307640 1 -- 172.21.15.37:0/1716904557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab74102bf0 msgr2=0x7fab74073d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.366+0000 7fab79307640 1 --2- 172.21.15.37:0/1716904557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab74102bf0 0x7fab74073d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.366+0000 7fab79307640 1 -- 172.21.15.37:0/1716904557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab74074b60 con 0x7fab741035b0 2024-06-10T00:56:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.366+0000 7fab78b06640 1 --2- 172.21.15.37:0/1716904557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab74101a00 0x7fab74075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:56:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.366+0000 7fab79307640 1 --2- 172.21.15.37:0/1716904557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab741035b0 0x7fab740742d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fab6c005e30 tx=0x7fab6c031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.366+0000 7fab69ffb640 1 -- 172.21.15.37:0/1716904557 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c003b70 con 0x7fab741035b0 2024-06-10T00:56:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.366+0000 7fab7ad6a640 1 -- 172.21.15.37:0/1716904557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab741a1c90 con 0x7fab741035b0 2024-06-10T00:56:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.367+0000 7fab69ffb640 1 -- 172.21.15.37:0/1716904557 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab6c003d10 con 0x7fab741035b0 2024-06-10T00:56:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.367+0000 7fab69ffb640 1 -- 172.21.15.37:0/1716904557 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c02fd30 con 0x7fab741035b0 2024-06-10T00:56:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.367+0000 7fab7ad6a640 1 -- 172.21.15.37:0/1716904557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab741a2180 con 0x7fab741035b0 2024-06-10T00:56:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.369+0000 7fab69ffb640 1 -- 172.21.15.37:0/1716904557 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fab6c004050 con 0x7fab741035b0 2024-06-10T00:56:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.370+0000 7fab69ffb640 1 --2- 172.21.15.37:0/1716904557 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fab5c0a2cd0 0x7fab5c0a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:56.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.370+0000 7fab78b06640 1 --2- 172.21.15.37:0/1716904557 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fab5c0a2cd0 0x7fab5c0a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.370+0000 7fab69ffb640 1 -- 172.21.15.37:0/1716904557 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fab6c0f0750 con 0x7fab741035b0 2024-06-10T00:56:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.370+0000 7fab7da70740 1 Processor -- start 2024-06-10T00:56:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.370+0000 7fab78b06640 1 --2- 172.21.15.37:0/1716904557 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fab5c0a2cd0 0x7fab5c0a5190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fab700058c0 tx=0x7fab70005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.370+0000 7fab7da70740 1 -- start start 2024-06-10T00:56:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.371+0000 7fab7da70740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633816ec010 0x5633816ec430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.371+0000 7fab7da70740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5633816eca00 0x5633816f1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.371+0000 7fab7da70740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633816f1b70 0x5633816f4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.371+0000 7fab7da70740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56338164f4f0 con 0x5633816ec010 2024-06-10T00:56:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.371+0000 7fab7da70740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5633815ed870 con 0x5633816eca00 2024-06-10T00:56:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.371+0000 7fab7da70740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56338164b4b0 con 0x5633816f1b70 2024-06-10T00:56:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.371+0000 7fab78b06640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633816ec010 0x5633816ec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.371+0000 7fab78b06640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633816ec010 0x5633816ec430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39290/0 (socket says 172.21.15.37:39290) 2024-06-10T00:56:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.371+0000 7fab78b06640 1 -- 172.21.15.37:0/2722878878 learned_addr learned my addr 172.21.15.37:0/2722878878 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.371+0000 7fab6bfff640 1 --2- 172.21.15.37:0/2722878878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5633816eca00 0x5633816f1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.371+0000 7fab78b06640 1 -- 172.21.15.37:0/2722878878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633816f1b70 msgr2=0x5633816f4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.371+0000 7fab78b06640 1 --2- 172.21.15.37:0/2722878878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633816f1b70 0x5633816f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.371+0000 7fab78b06640 1 -- 172.21.15.37:0/2722878878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5633816eca00 msgr2=0x5633816f1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.371+0000 7fab78b06640 1 --2- 172.21.15.37:0/2722878878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5633816eca00 0x5633816f1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.371+0000 7fab78b06640 1 -- 172.21.15.37:0/2722878878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633816099b0 con 0x5633816ec010 2024-06-10T00:56:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.371+0000 7fab6bfff640 1 --2- 172.21.15.37:0/2722878878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5633816eca00 0x5633816f1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:56:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.371+0000 7fab78b06640 1 --2- 172.21.15.37:0/2722878878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633816ec010 0x5633816ec430 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fab7000b450 tx=0x7fab70012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61f441924e70dd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.371+0000 7fab5affd640 1 -- 172.21.15.37:0/2722878878 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab70022070 con 0x5633816ec010 2024-06-10T00:56:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.371+0000 7fab5affd640 1 -- 172.21.15.37:0/2722878878 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab7001eb30 con 0x5633816ec010 2024-06-10T00:56:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.371+0000 7fab5affd640 1 -- 172.21.15.37:0/2722878878 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab70010bb0 con 0x5633816ec010 2024-06-10T00:56:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.372+0000 7fab7da70740 1 -- 172.21.15.37:0/2722878878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633816ec010 msgr2=0x5633816ec430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.372+0000 7fab7da70740 1 --2- 172.21.15.37:0/2722878878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633816ec010 0x5633816ec430 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fab7000b450 tx=0x7fab70012000 comp rx=0 tx=0).stop 2024-06-10T00:56:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.372+0000 7fab7da70740 1 -- 172.21.15.37:0/2722878878 shutdown_connections 2024-06-10T00:56:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.372+0000 7fab7da70740 1 --2- 172.21.15.37:0/2722878878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633816f1b70 0x5633816f4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.372+0000 7fab7da70740 1 --2- 172.21.15.37:0/2722878878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5633816eca00 0x5633816f1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.372+0000 7fab7da70740 1 --2- 172.21.15.37:0/2722878878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633816ec010 0x5633816ec430 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.372+0000 7fab7da70740 1 -- 172.21.15.37:0/2722878878 >> 172.21.15.37:0/2722878878 conn(0x5633816e7680 msgr2=0x5633816e7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.372+0000 7fab7da70740 1 -- 172.21.15.37:0/2722878878 shutdown_connections 2024-06-10T00:56:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.372+0000 7fab7da70740 1 -- 172.21.15.37:0/2722878878 wait complete. 2024-06-10T00:56:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.373+0000 7fab7da70740 1 -- start start 2024-06-10T00:56:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.373+0000 7fab7da70740 1 Processor -- start 2024-06-10T00:56:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.373+0000 7fab7da70740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5633816ec010 0x5633816f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.373+0000 7fab7da70740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633816eca00 0x5633816f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.373+0000 7fab7da70740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633816f1b70 0x5633816fda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.373+0000 7fab7da70740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5633816f4b20 con 0x5633816f1b70 2024-06-10T00:56:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.373+0000 7fab7da70740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5633815d8ae0 con 0x5633816ec010 2024-06-10T00:56:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.373+0000 7fab7da70740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5633816f4ca0 con 0x5633816eca00 2024-06-10T00:56:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.373+0000 7fab7da70740 10 client.0 _ll_register_callbacks cb 0x5633816119b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:56:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.373+0000 7fab79307640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633816f1b70 0x5633816fda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.373+0000 7fab79307640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633816f1b70 0x5633816fda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39292/0 (socket says 172.21.15.37:39292) 2024-06-10T00:56:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.373+0000 7fab79307640 1 -- 172.21.15.37:0/1808131214 learned_addr learned my addr 172.21.15.37:0/1808131214 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.373+0000 7fab6bfff640 1 --2- 172.21.15.37:0/1808131214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633816eca00 0x5633816f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.374+0000 7fab78b06640 1 --2- 172.21.15.37:0/1808131214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5633816ec010 0x5633816f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.374+0000 7fab79307640 1 -- 172.21.15.37:0/1808131214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633816eca00 msgr2=0x5633816f6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.374+0000 7fab79307640 1 --2- 172.21.15.37:0/1808131214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633816eca00 0x5633816f6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.374+0000 7fab79307640 1 -- 172.21.15.37:0/1808131214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5633816ec010 msgr2=0x5633816f64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.374+0000 7fab79307640 1 --2- 172.21.15.37:0/1808131214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5633816ec010 0x5633816f64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.374+0000 7fab79307640 1 -- 172.21.15.37:0/1808131214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563381363210 con 0x5633816f1b70 2024-06-10T00:56:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.374+0000 7fab79307640 1 --2- 172.21.15.37:0/1808131214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633816f1b70 0x5633816fda50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fab6c0b90f0 tx=0x7fab6c0bc890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17dc56dba62409b9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.374+0000 7fab53fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:56:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.374+0000 7fab53fff640 1 -- 172.21.15.37:0/1808131214 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c0bcc90 con 0x5633816f1b70 2024-06-10T00:56:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.374+0000 7fab7da70740 1 -- 172.21.15.37:0/1808131214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab7001eb30 con 0x5633816f1b70 2024-06-10T00:56:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.374+0000 7fab53fff640 1 -- 172.21.15.37:0/1808131214 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fab6c0f0430 con 0x5633816f1b70 2024-06-10T00:56:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.375+0000 7fab53fff640 1 -- 172.21.15.37:0/1808131214 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fab6c0c1040 con 0x5633816f1b70 2024-06-10T00:56:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.375+0000 7fab7da70740 10 client.15363 fetch_fsmap learned FSMap version 27 2024-06-10T00:56:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.375+0000 7fab7da70740 1 -- 172.21.15.37:0/1808131214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56338142f620 con 0x5633816f1b70 2024-06-10T00:56:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.375+0000 7fab53fff640 1 -- 172.21.15.37:0/1808131214 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab7001eb30 con 0x5633816f1b70 2024-06-10T00:56:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.375+0000 7fab53fff640 1 -- 172.21.15.37:0/1808131214 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fab6c0bcef0 con 0x5633816f1b70 2024-06-10T00:56:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.375+0000 7fab7da70740 10 client.15363 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:56:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.375+0000 7fab7da70740 10 client.15363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T00:56:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.375+0000 7fab7da70740 20 client.15363 populate_metadata read hostname 'smithi037' 2024-06-10T00:56:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.375+0000 7fab7da70740 1 --2- 172.21.15.37:0/1808131214 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x563381702420 0x563381704860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.375+0000 7fab7da70740 4 client.15363 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:56:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.375+0000 7fab7da70740 1 -- 172.21.15.37:0/1808131214 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563381662f10 con 0x563381702420 2024-06-10T00:56:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.375+0000 7fab78b06640 1 --2- 172.21.15.37:0/1808131214 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x563381702420 0x563381704860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.376+0000 7fab78b06640 1 --2- 172.21.15.37:0/1808131214 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x563381702420 0x563381704860 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa4202057e2021ec server_cookie=c8d26e2cbb37fcd8 in_seq=0 out_seq=0 2024-06-10T00:56:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.376+0000 7fab53fff640 10 client.15363 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:56:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.377+0000 7fab53fff640 1 -- 172.21.15.37:0/1808131214 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563381662f10 con 0x563381702420 2024-06-10T00:56:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.377+0000 7fab53fff640 10 client.15363 handle_command_reply: tid=0 2024-06-10T00:56:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.377+0000 7fab7da70740 1 client.15363 shutdown 2024-06-10T00:56:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.377+0000 7fab7da70740 1 -- 172.21.15.37:0/1808131214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633816f1b70 msgr2=0x5633816fda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.377+0000 7fab7da70740 1 --2- 172.21.15.37:0/1808131214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633816f1b70 0x5633816fda50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fab6c0b90f0 tx=0x7fab6c0bc890 comp rx=0 tx=0).stop 2024-06-10T00:56:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.377+0000 7fab7da70740 1 -- 172.21.15.37:0/1808131214 shutdown_connections 2024-06-10T00:56:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.377+0000 7fab7da70740 1 --2- 172.21.15.37:0/1808131214 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x563381702420 0x563381704860 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.377+0000 7fab7da70740 1 --2- 172.21.15.37:0/1808131214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633816f1b70 0x5633816fda50 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.377+0000 7fab7da70740 1 --2- 172.21.15.37:0/1808131214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633816eca00 0x5633816f6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.377+0000 7fab53fff640 0 client.15363 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:56:56.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.377+0000 7fab7da70740 1 --2- 172.21.15.37:0/1808131214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5633816ec010 0x5633816f64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.377+0000 7fab7da70740 1 -- 172.21.15.37:0/1808131214 >> 172.21.15.37:0/1808131214 conn(0x5633816e7680 msgr2=0x5633816e67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:56.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.377+0000 7fab7da70740 1 -- 172.21.15.37:0/1808131214 shutdown_connections 2024-06-10T00:56:56.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.377+0000 7fab7da70740 1 -- 172.21.15.37:0/1808131214 wait complete. 2024-06-10T00:56:56.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.377+0000 7fab7da70740 20 client.15363 trim_cache size 0 max 16384 2024-06-10T00:56:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.392+0000 7fab7da70740 1 Processor -- start 2024-06-10T00:56:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.393+0000 7fab7da70740 1 -- start start 2024-06-10T00:56:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.393+0000 7fab7da70740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633816ec010 0x5633816f50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.393+0000 7fab7da70740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633816eca00 0x5633816fda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.393+0000 7fab7da70740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5633816f1b70 0x5633816ede60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.393+0000 7fab7da70740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fab741af030 con 0x5633816eca00 2024-06-10T00:56:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.393+0000 7fab7da70740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab741ac900 con 0x5633816f1b70 2024-06-10T00:56:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.393+0000 7fab7da70740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fab741aa750 con 0x5633816ec010 2024-06-10T00:56:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.393+0000 7fab6bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633816eca00 0x5633816fda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.393+0000 7fab6bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633816eca00 0x5633816fda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39296/0 (socket says 172.21.15.37:39296) 2024-06-10T00:56:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.393+0000 7fab6bfff640 1 -- 172.21.15.37:0/4229068222 learned_addr learned my addr 172.21.15.37:0/4229068222 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.393+0000 7fab78b06640 1 --2- 172.21.15.37:0/4229068222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633816ec010 0x5633816f50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.393+0000 7fab79307640 1 --2- 172.21.15.37:0/4229068222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5633816f1b70 0x5633816ede60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.393+0000 7fab6bfff640 1 -- 172.21.15.37:0/4229068222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633816ec010 msgr2=0x5633816f50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.393+0000 7fab6bfff640 1 --2- 172.21.15.37:0/4229068222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633816ec010 0x5633816f50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.393+0000 7fab6bfff640 1 -- 172.21.15.37:0/4229068222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5633816f1b70 msgr2=0x5633816ede60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.393+0000 7fab6bfff640 1 --2- 172.21.15.37:0/4229068222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5633816f1b70 0x5633816ede60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.393+0000 7fab6bfff640 1 -- 172.21.15.37:0/4229068222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633816e8350 con 0x5633816eca00 2024-06-10T00:56:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.393+0000 7fab6bfff640 1 --2- 172.21.15.37:0/4229068222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633816eca00 0x5633816fda10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fab740754c0 tx=0x7fab64012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e221ac0c9b1184 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.394+0000 7fab5a7fc640 1 -- 172.21.15.37:0/4229068222 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab64005020 con 0x5633816eca00 2024-06-10T00:56:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.394+0000 7fab5a7fc640 1 -- 172.21.15.37:0/4229068222 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab640047a0 con 0x5633816eca00 2024-06-10T00:56:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.394+0000 7fab5a7fc640 1 -- 172.21.15.37:0/4229068222 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6400f7a0 con 0x5633816eca00 2024-06-10T00:56:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.394+0000 7fab7da70740 1 -- 172.21.15.37:0/4229068222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633816eca00 msgr2=0x5633816fda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.394+0000 7fab7da70740 1 --2- 172.21.15.37:0/4229068222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633816eca00 0x5633816fda10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fab740754c0 tx=0x7fab64012820 comp rx=0 tx=0).stop 2024-06-10T00:56:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.394+0000 7fab7da70740 1 -- 172.21.15.37:0/4229068222 shutdown_connections 2024-06-10T00:56:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.394+0000 7fab7da70740 1 --2- 172.21.15.37:0/4229068222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5633816f1b70 0x5633816ede60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.394+0000 7fab7da70740 1 --2- 172.21.15.37:0/4229068222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633816eca00 0x5633816fda10 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.394+0000 7fab7da70740 1 --2- 172.21.15.37:0/4229068222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633816ec010 0x5633816f50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.394+0000 7fab7da70740 1 -- 172.21.15.37:0/4229068222 >> 172.21.15.37:0/4229068222 conn(0x5633816e7680 msgr2=0x5633816e7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.395+0000 7fab7da70740 1 -- 172.21.15.37:0/4229068222 shutdown_connections 2024-06-10T00:56:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.395+0000 7fab7da70740 1 -- 172.21.15.37:0/4229068222 wait complete. 2024-06-10T00:56:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.395+0000 7fab7da70740 1 -- start start 2024-06-10T00:56:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.395+0000 7fab7da70740 1 Processor -- start 2024-06-10T00:56:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.395+0000 7fab7da70740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633816ec010 0x5633816faab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.395+0000 7fab7da70740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633816eca00 0x5633816faff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.395+0000 7fab7da70740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5633816f1b70 0x5633816fc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.395+0000 7fab7da70740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fab741aadc0 con 0x5633816ec010 2024-06-10T00:56:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.395+0000 7fab7da70740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab741ac230 con 0x5633816f1b70 2024-06-10T00:56:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.395+0000 7fab7da70740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fab741ab490 con 0x5633816eca00 2024-06-10T00:56:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.395+0000 7fab7da70740 10 client.0 _ll_register_callbacks cb 0x7fab6c02f690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:56:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.395+0000 7fab78b06640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633816ec010 0x5633816faab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.395+0000 7fab78b06640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633816ec010 0x5633816faab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39312/0 (socket says 172.21.15.37:39312) 2024-06-10T00:56:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.395+0000 7fab78b06640 1 -- 172.21.15.37:0/2581855911 learned_addr learned my addr 172.21.15.37:0/2581855911 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.395+0000 7fab6bfff640 1 --2- 172.21.15.37:0/2581855911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633816eca00 0x5633816faff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.395+0000 7fab79307640 1 --2- 172.21.15.37:0/2581855911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5633816f1b70 0x5633816fc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.396+0000 7fab78b06640 1 -- 172.21.15.37:0/2581855911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633816eca00 msgr2=0x5633816faff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.396+0000 7fab78b06640 1 --2- 172.21.15.37:0/2581855911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633816eca00 0x5633816faff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.396+0000 7fab78b06640 1 -- 172.21.15.37:0/2581855911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5633816f1b70 msgr2=0x5633816fc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:56:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.396+0000 7fab78b06640 1 --2- 172.21.15.37:0/2581855911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5633816f1b70 0x5633816fc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.396+0000 7fab78b06640 1 -- 172.21.15.37:0/2581855911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633816fc5f0 con 0x5633816ec010 2024-06-10T00:56:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.396+0000 7fab78b06640 1 --2- 172.21.15.37:0/2581855911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633816ec010 0x5633816faab0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fab7000b450 tx=0x7fab7000bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48032cba70cfade0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.396+0000 7fab53fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:56:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.396+0000 7fab53fff640 1 -- 172.21.15.37:0/2581855911 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab70037030 con 0x5633816ec010 2024-06-10T00:56:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.396+0000 7fab53fff640 1 -- 172.21.15.37:0/2581855911 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fab70005490 con 0x5633816ec010 2024-06-10T00:56:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.396+0000 7fab7da70740 1 -- 172.21.15.37:0/2581855911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab640047a0 con 0x5633816ec010 2024-06-10T00:56:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.396+0000 7fab53fff640 1 -- 172.21.15.37:0/2581855911 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fab70043050 con 0x5633816ec010 2024-06-10T00:56:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.396+0000 7fab7da70740 10 client.15369 fetch_fsmap learned FSMap version 27 2024-06-10T00:56:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.396+0000 7fab7da70740 1 -- 172.21.15.37:0/2581855911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5633816fd900 con 0x5633816ec010 2024-06-10T00:56:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.396+0000 7fab53fff640 1 -- 172.21.15.37:0/2581855911 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab7001e450 con 0x5633816ec010 2024-06-10T00:56:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.397+0000 7fab53fff640 1 -- 172.21.15.37:0/2581855911 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fab7001ece0 con 0x5633816ec010 2024-06-10T00:56:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.397+0000 7fab7da70740 10 client.15369 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:56:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.397+0000 7fab7da70740 10 client.15369 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T00:56:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.397+0000 7fab7da70740 20 client.15369 populate_metadata read hostname 'smithi037' 2024-06-10T00:56:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.397+0000 7fab7da70740 1 --2- 172.21.15.37:0/2581855911 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x563381702420 0x56338176f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.397+0000 7fab7da70740 4 client.15369 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T00:56:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.397+0000 7fab7da70740 1 -- 172.21.15.37:0/2581855911 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fab640047a0 con 0x563381702420 2024-06-10T00:56:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.397+0000 7fab6bfff640 1 --2- 172.21.15.37:0/2581855911 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x563381702420 0x56338176f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.397+0000 7fab53fff640 10 client.15369 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:56:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.397+0000 7fab6bfff640 1 --2- 172.21.15.37:0/2581855911 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x563381702420 0x56338176f160 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff6439039c295f43 server_cookie=98928a3f171f6d7 in_seq=0 out_seq=0 2024-06-10T00:56:56.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.409+0000 7fab53fff640 1 -- 172.21.15.37:0/2581855911 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fab640047a0 con 0x563381702420 2024-06-10T00:56:56.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.409+0000 7fab53fff640 10 client.15369 handle_command_reply: tid=0 2024-06-10T00:56:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.409+0000 7fab7da70740 1 client.15369 shutdown 2024-06-10T00:56:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.410+0000 7fab7da70740 1 -- 172.21.15.37:0/2581855911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633816ec010 msgr2=0x5633816faab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.410+0000 7fab7da70740 1 --2- 172.21.15.37:0/2581855911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633816ec010 0x5633816faab0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fab7000b450 tx=0x7fab7000bd80 comp rx=0 tx=0).stop 2024-06-10T00:56:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.410+0000 7fab7da70740 1 -- 172.21.15.37:0/2581855911 shutdown_connections 2024-06-10T00:56:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.410+0000 7fab7da70740 1 --2- 172.21.15.37:0/2581855911 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x563381702420 0x56338176f160 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.410+0000 7fab7da70740 1 --2- 172.21.15.37:0/2581855911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5633816f1b70 0x5633816fc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.410+0000 7fab7da70740 1 --2- 172.21.15.37:0/2581855911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633816eca00 0x5633816faff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.410+0000 7fab53fff640 0 client.15369 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:56:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.410+0000 7fab7da70740 1 --2- 172.21.15.37:0/2581855911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633816ec010 0x5633816faab0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.410+0000 7fab7da70740 1 -- 172.21.15.37:0/2581855911 >> 172.21.15.37:0/2581855911 conn(0x5633816e7680 msgr2=0x5633816f63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.410+0000 7fab7da70740 1 -- 172.21.15.37:0/2581855911 shutdown_connections 2024-06-10T00:56:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.410+0000 7fab7da70740 1 -- 172.21.15.37:0/2581855911 wait complete. 2024-06-10T00:56:56.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.410+0000 7fab7da70740 20 client.15369 trim_cache size 0 max 16384 2024-06-10T00:56:56.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T00:56:56.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T00:56:56.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T00:56:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.410+0000 7fab7ad6a640 1 -- 172.21.15.37:0/1716904557 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fab5c0a2cd0 msgr2=0x7fab5c0a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.410+0000 7fab7ad6a640 1 --2- 172.21.15.37:0/1716904557 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fab5c0a2cd0 0x7fab5c0a5190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fab700058c0 tx=0x7fab70005850 comp rx=0 tx=0).stop 2024-06-10T00:56:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.410+0000 7fab7ad6a640 1 -- 172.21.15.37:0/1716904557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab741035b0 msgr2=0x7fab740742d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.410+0000 7fab7ad6a640 1 --2- 172.21.15.37:0/1716904557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab741035b0 0x7fab740742d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fab6c005e30 tx=0x7fab6c031ee0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.411+0000 7fab7ad6a640 1 -- 172.21.15.37:0/1716904557 shutdown_connections 2024-06-10T00:56:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.411+0000 7fab7ad6a640 1 --2- 172.21.15.37:0/1716904557 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fab5c0a2cd0 0x7fab5c0a5190 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.411+0000 7fab7ad6a640 1 --2- 172.21.15.37:0/1716904557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab741035b0 0x7fab740742d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.411+0000 7fab7ad6a640 1 --2- 172.21.15.37:0/1716904557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab74102bf0 0x7fab74073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.411+0000 7fab7ad6a640 1 --2- 172.21.15.37:0/1716904557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab74101a00 0x7fab74075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.411+0000 7fab7ad6a640 1 -- 172.21.15.37:0/1716904557 >> 172.21.15.37:0/1716904557 conn(0x7fab740fcf90 msgr2=0x7fab74105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.411+0000 7fab7ad6a640 1 -- 172.21.15.37:0/1716904557 shutdown_connections 2024-06-10T00:56:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:56:56.411+0000 7fab7ad6a640 1 -- 172.21.15.37:0/1716904557 wait complete. 2024-06-10T00:56:57.043 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":437,"stamp":"2024-06-10T00:56:54.415269+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":672924,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1308,"num_read_kb":789,"num_write":374,"num_write_kb":3088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":802,"ondisk_log_size":802,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370920,"kb_used_data":13448,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570904,"statfs":{"total":1151940427776,"available":1151560605696,"internally_reserved":0,"allocated":13770752,"data_stored":8871914,"data_compressed":497625,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":37,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003178"},"pg_stats":[{"pgid":"3.3f","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374277+0000","last_change":"2024-06-10T00:52:35.971067+0000","last_active":"2024-06-10T00:55:20.374277+0000","last_peered":"2024-06-10T00:55:20.374277+0000","last_clean":"2024-06-10T00:55:20.374277+0000","last_became_active":"2024-06-10T00:52:35.970292+0000","last_became_peered":"2024-06-10T00:52:35.970292+0000","last_unstale":"2024-06-10T00:55:20.374277+0000","last_undegraded":"2024-06-10T00:55:20.374277+0000","last_fullsized":"2024-06-10T00:55:20.374277+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:57:37.287238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.371246+0000","last_change":"2024-06-10T00:52:35.960421+0000","last_active":"2024-06-10T00:55:20.371246+0000","last_peered":"2024-06-10T00:55:20.371246+0000","last_clean":"2024-06-10T00:55:20.371246+0000","last_became_active":"2024-06-10T00:52:35.960070+0000","last_became_peered":"2024-06-10T00:52:35.960070+0000","last_unstale":"2024-06-10T00:55:20.371246+0000","last_undegraded":"2024-06-10T00:55:20.371246+0000","last_fullsized":"2024-06-10T00:55:20.371246+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:48:02.616434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375920+0000","last_change":"2024-06-10T00:52:35.964778+0000","last_active":"2024-06-10T00:55:20.375920+0000","last_peered":"2024-06-10T00:55:20.375920+0000","last_clean":"2024-06-10T00:55:20.375920+0000","last_became_active":"2024-06-10T00:52:35.964463+0000","last_became_peered":"2024-06-10T00:52:35.964463+0000","last_unstale":"2024-06-10T00:55:20.375920+0000","last_undegraded":"2024-06-10T00:55:20.375920+0000","last_fullsized":"2024-06-10T00:55:20.375920+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:52:44.987363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.360455+0000","last_change":"2024-06-10T00:52:35.963226+0000","last_active":"2024-06-10T00:55:20.360455+0000","last_peered":"2024-06-10T00:55:20.360455+0000","last_clean":"2024-06-10T00:55:20.360455+0000","last_became_active":"2024-06-10T00:52:35.962862+0000","last_became_peered":"2024-06-10T00:52:35.962862+0000","last_unstale":"2024-06-10T00:55:20.360455+0000","last_undegraded":"2024-06-10T00:55:20.360455+0000","last_fullsized":"2024-06-10T00:55:20.360455+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:08:56.418071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.360258+0000","last_change":"2024-06-10T00:52:35.957427+0000","last_active":"2024-06-10T00:55:20.360258+0000","last_peered":"2024-06-10T00:55:20.360258+0000","last_clean":"2024-06-10T00:55:20.360258+0000","last_became_active":"2024-06-10T00:52:35.957049+0000","last_became_peered":"2024-06-10T00:52:35.957049+0000","last_unstale":"2024-06-10T00:55:20.360258+0000","last_undegraded":"2024-06-10T00:55:20.360258+0000","last_fullsized":"2024-06-10T00:55:20.360258+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:16:04.257210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.355809+0000","last_change":"2024-06-10T00:52:35.954251+0000","last_active":"2024-06-10T00:55:20.355809+0000","last_peered":"2024-06-10T00:55:20.355809+0000","last_clean":"2024-06-10T00:55:20.355809+0000","last_became_active":"2024-06-10T00:52:35.953908+0000","last_became_peered":"2024-06-10T00:52:35.953908+0000","last_unstale":"2024-06-10T00:55:20.355809+0000","last_undegraded":"2024-06-10T00:55:20.355809+0000","last_fullsized":"2024-06-10T00:55:20.355809+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:55:25.125836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364343+0000","last_change":"2024-06-10T00:52:35.956822+0000","last_active":"2024-06-10T00:55:20.364343+0000","last_peered":"2024-06-10T00:55:20.364343+0000","last_clean":"2024-06-10T00:55:20.364343+0000","last_became_active":"2024-06-10T00:52:35.956466+0000","last_became_peered":"2024-06-10T00:52:35.956466+0000","last_unstale":"2024-06-10T00:55:20.364343+0000","last_undegraded":"2024-06-10T00:55:20.364343+0000","last_fullsized":"2024-06-10T00:55:20.364343+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:36:41.534392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.369347+0000","last_change":"2024-06-10T00:52:35.962978+0000","last_active":"2024-06-10T00:55:20.369347+0000","last_peered":"2024-06-10T00:55:20.369347+0000","last_clean":"2024-06-10T00:55:20.369347+0000","last_became_active":"2024-06-10T00:52:35.962609+0000","last_became_peered":"2024-06-10T00:52:35.962609+0000","last_unstale":"2024-06-10T00:55:20.369347+0000","last_undegraded":"2024-06-10T00:55:20.369347+0000","last_fullsized":"2024-06-10T00:55:20.369347+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:20:57.469577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.377074+0000","last_change":"2024-06-10T00:52:35.963203+0000","last_active":"2024-06-10T00:55:20.377074+0000","last_peered":"2024-06-10T00:55:20.377074+0000","last_clean":"2024-06-10T00:55:20.377074+0000","last_became_active":"2024-06-10T00:52:35.962789+0000","last_became_peered":"2024-06-10T00:52:35.962789+0000","last_unstale":"2024-06-10T00:55:20.377074+0000","last_undegraded":"2024-06-10T00:55:20.377074+0000","last_fullsized":"2024-06-10T00:55:20.377074+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:00:17.819193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"100'16","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.356376+0000","last_change":"2024-06-10T00:52:35.966940+0000","last_active":"2024-06-10T00:55:20.356376+0000","last_peered":"2024-06-10T00:55:20.356376+0000","last_clean":"2024-06-10T00:55:20.356376+0000","last_became_active":"2024-06-10T00:52:35.966554+0000","last_became_peered":"2024-06-10T00:52:35.966554+0000","last_unstale":"2024-06-10T00:55:20.356376+0000","last_undegraded":"2024-06-10T00:55:20.356376+0000","last_fullsized":"2024-06-10T00:55:20.356376+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:09:00.374437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372673+0000","last_change":"2024-06-10T00:52:35.960927+0000","last_active":"2024-06-10T00:55:20.372673+0000","last_peered":"2024-06-10T00:55:20.372673+0000","last_clean":"2024-06-10T00:55:20.372673+0000","last_became_active":"2024-06-10T00:52:35.960577+0000","last_became_peered":"2024-06-10T00:52:35.960577+0000","last_unstale":"2024-06-10T00:55:20.372673+0000","last_undegraded":"2024-06-10T00:55:20.372673+0000","last_fullsized":"2024-06-10T00:55:20.372673+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:50:02.366695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.355167+0000","last_change":"2024-06-10T00:52:35.964435+0000","last_active":"2024-06-10T00:55:20.355167+0000","last_peered":"2024-06-10T00:55:20.355167+0000","last_clean":"2024-06-10T00:55:20.355167+0000","last_became_active":"2024-06-10T00:52:35.964069+0000","last_became_peered":"2024-06-10T00:52:35.964069+0000","last_unstale":"2024-06-10T00:55:20.355167+0000","last_undegraded":"2024-06-10T00:55:20.355167+0000","last_fullsized":"2024-06-10T00:55:20.355167+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:06.708551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.361787+0000","last_change":"2024-06-10T00:52:35.966722+0000","last_active":"2024-06-10T00:55:20.361787+0000","last_peered":"2024-06-10T00:55:20.361787+0000","last_clean":"2024-06-10T00:55:20.361787+0000","last_became_active":"2024-06-10T00:52:35.966366+0000","last_became_peered":"2024-06-10T00:52:35.966366+0000","last_unstale":"2024-06-10T00:55:20.361787+0000","last_undegraded":"2024-06-10T00:55:20.361787+0000","last_fullsized":"2024-06-10T00:55:20.361787+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:57:46.021047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.371736+0000","last_change":"2024-06-10T00:52:35.955307+0000","last_active":"2024-06-10T00:55:20.371736+0000","last_peered":"2024-06-10T00:55:20.371736+0000","last_clean":"2024-06-10T00:55:20.371736+0000","last_became_active":"2024-06-10T00:52:35.954140+0000","last_became_peered":"2024-06-10T00:52:35.954140+0000","last_unstale":"2024-06-10T00:55:20.371736+0000","last_undegraded":"2024-06-10T00:55:20.371736+0000","last_fullsized":"2024-06-10T00:55:20.371736+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:47:22.493598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373915+0000","last_change":"2024-06-10T00:52:35.960592+0000","last_active":"2024-06-10T00:55:20.373915+0000","last_peered":"2024-06-10T00:55:20.373915+0000","last_clean":"2024-06-10T00:55:20.373915+0000","last_became_active":"2024-06-10T00:52:35.960263+0000","last_became_peered":"2024-06-10T00:52:35.960263+0000","last_unstale":"2024-06-10T00:55:20.373915+0000","last_undegraded":"2024-06-10T00:55:20.373915+0000","last_fullsized":"2024-06-10T00:55:20.373915+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:47:11.658666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372126+0000","last_change":"2024-06-10T00:52:35.965796+0000","last_active":"2024-06-10T00:55:20.372126+0000","last_peered":"2024-06-10T00:55:20.372126+0000","last_clean":"2024-06-10T00:55:20.372126+0000","last_became_active":"2024-06-10T00:52:35.963965+0000","last_became_peered":"2024-06-10T00:52:35.963965+0000","last_unstale":"2024-06-10T00:55:20.372126+0000","last_undegraded":"2024-06-10T00:55:20.372126+0000","last_fullsized":"2024-06-10T00:55:20.372126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:29:47.149852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374090+0000","last_change":"2024-06-10T00:52:35.970714+0000","last_active":"2024-06-10T00:55:20.374090+0000","last_peered":"2024-06-10T00:55:20.374090+0000","last_clean":"2024-06-10T00:55:20.374090+0000","last_became_active":"2024-06-10T00:52:35.970173+0000","last_became_peered":"2024-06-10T00:52:35.970173+0000","last_unstale":"2024-06-10T00:55:20.374090+0000","last_undegraded":"2024-06-10T00:55:20.374090+0000","last_fullsized":"2024-06-10T00:55:20.374090+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:33:28.183156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.370737+0000","last_change":"2024-06-10T00:52:35.963052+0000","last_active":"2024-06-10T00:55:20.370737+0000","last_peered":"2024-06-10T00:55:20.370737+0000","last_clean":"2024-06-10T00:55:20.370737+0000","last_became_active":"2024-06-10T00:52:35.962674+0000","last_became_peered":"2024-06-10T00:52:35.962674+0000","last_unstale":"2024-06-10T00:55:20.370737+0000","last_undegraded":"2024-06-10T00:55:20.370737+0000","last_fullsized":"2024-06-10T00:55:20.370737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:56:38.836854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"100'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.363943+0000","last_change":"2024-06-10T00:53:22.130336+0000","last_active":"2024-06-10T00:55:20.363943+0000","last_peered":"2024-06-10T00:55:20.363943+0000","last_clean":"2024-06-10T00:55:20.363943+0000","last_became_active":"2024-06-10T00:53:22.129910+0000","last_became_peered":"2024-06-10T00:53:22.129910+0000","last_unstale":"2024-06-10T00:55:20.363943+0000","last_undegraded":"2024-06-10T00:55:20.363943+0000","last_fullsized":"2024-06-10T00:55:20.363943+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:23:50.600821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554286+0000","last_change":"2024-06-10T00:52:45.997276+0000","last_active":"2024-06-10T00:54:22.554286+0000","last_peered":"2024-06-10T00:54:22.554286+0000","last_clean":"2024-06-10T00:54:22.554286+0000","last_became_active":"2024-06-10T00:52:45.995638+0000","last_became_peered":"2024-06-10T00:52:45.995638+0000","last_unstale":"2024-06-10T00:54:22.554286+0000","last_undegraded":"2024-06-10T00:54:22.554286+0000","last_fullsized":"2024-06-10T00:54:22.554286+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:14:20.922348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'9","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:45.433552+0000","last_change":"2024-06-10T00:52:32.760888+0000","last_active":"2024-06-10T00:56:45.433552+0000","last_peered":"2024-06-10T00:56:45.433552+0000","last_clean":"2024-06-10T00:56:45.433552+0000","last_became_active":"2024-06-10T00:52:32.760531+0000","last_became_peered":"2024-06-10T00:52:32.760531+0000","last_unstale":"2024-06-10T00:56:45.433552+0000","last_undegraded":"2024-06-10T00:56:45.433552+0000","last_fullsized":"2024-06-10T00:56:45.433552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:05:59.052251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"100'5","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372743+0000","last_change":"2024-06-10T00:52:35.967626+0000","last_active":"2024-06-10T00:55:20.372743+0000","last_peered":"2024-06-10T00:55:20.372743+0000","last_clean":"2024-06-10T00:55:20.372743+0000","last_became_active":"2024-06-10T00:52:35.967295+0000","last_became_peered":"2024-06-10T00:52:35.967295+0000","last_unstale":"2024-06-10T00:55:20.372743+0000","last_undegraded":"2024-06-10T00:55:20.372743+0000","last_fullsized":"2024-06-10T00:55:20.372743+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:07:55.054327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668105+0000","last_change":"2024-06-10T00:52:45.993974+0000","last_active":"2024-06-10T00:54:22.668105+0000","last_peered":"2024-06-10T00:54:22.668105+0000","last_clean":"2024-06-10T00:54:22.668105+0000","last_became_active":"2024-06-10T00:52:45.993242+0000","last_became_peered":"2024-06-10T00:52:45.993242+0000","last_unstale":"2024-06-10T00:54:22.668105+0000","last_undegraded":"2024-06-10T00:54:22.668105+0000","last_fullsized":"2024-06-10T00:54:22.668105+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:28:12.810368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:33.628114+0000","last_change":"2024-06-10T00:52:32.959905+0000","last_active":"2024-06-10T00:54:33.628114+0000","last_peered":"2024-06-10T00:54:33.628114+0000","last_clean":"2024-06-10T00:54:33.628114+0000","last_became_active":"2024-06-10T00:52:32.959551+0000","last_became_peered":"2024-06-10T00:52:32.959551+0000","last_unstale":"2024-06-10T00:54:33.628114+0000","last_undegraded":"2024-06-10T00:54:33.628114+0000","last_fullsized":"2024-06-10T00:54:33.628114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:33:48.029129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.377301+0000","last_change":"2024-06-10T00:52:35.965790+0000","last_active":"2024-06-10T00:55:20.377301+0000","last_peered":"2024-06-10T00:55:20.377301+0000","last_clean":"2024-06-10T00:55:20.377301+0000","last_became_active":"2024-06-10T00:52:35.965475+0000","last_became_peered":"2024-06-10T00:52:35.965475+0000","last_unstale":"2024-06-10T00:55:20.377301+0000","last_undegraded":"2024-06-10T00:55:20.377301+0000","last_fullsized":"2024-06-10T00:55:20.377301+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:24:02.894411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.667953+0000","last_change":"2024-06-10T00:52:45.994464+0000","last_active":"2024-06-10T00:54:22.667953+0000","last_peered":"2024-06-10T00:54:22.667953+0000","last_clean":"2024-06-10T00:54:22.667953+0000","last_became_active":"2024-06-10T00:52:45.994022+0000","last_became_peered":"2024-06-10T00:52:45.994022+0000","last_unstale":"2024-06-10T00:54:22.667953+0000","last_undegraded":"2024-06-10T00:54:22.667953+0000","last_fullsized":"2024-06-10T00:54:22.667953+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:27:25.063074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'4","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556658+0000","last_change":"2024-06-10T00:52:32.763885+0000","last_active":"2024-06-10T00:54:22.556658+0000","last_peered":"2024-06-10T00:54:22.556658+0000","last_clean":"2024-06-10T00:54:22.556658+0000","last_became_active":"2024-06-10T00:52:32.763606+0000","last_became_peered":"2024-06-10T00:52:32.763606+0000","last_unstale":"2024-06-10T00:54:22.556658+0000","last_undegraded":"2024-06-10T00:54:22.556658+0000","last_fullsized":"2024-06-10T00:54:22.556658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:01:58.161168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372890+0000","last_change":"2024-06-10T00:52:35.966948+0000","last_active":"2024-06-10T00:55:20.372890+0000","last_peered":"2024-06-10T00:55:20.372890+0000","last_clean":"2024-06-10T00:55:20.372890+0000","last_became_active":"2024-06-10T00:52:35.966345+0000","last_became_peered":"2024-06-10T00:52:35.966345+0000","last_unstale":"2024-06-10T00:55:20.372890+0000","last_undegraded":"2024-06-10T00:55:20.372890+0000","last_fullsized":"2024-06-10T00:55:20.372890+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:58:27.169201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.667762+0000","last_change":"2024-06-10T00:52:45.993645+0000","last_active":"2024-06-10T00:54:22.667762+0000","last_peered":"2024-06-10T00:54:22.667762+0000","last_clean":"2024-06-10T00:54:22.667762+0000","last_became_active":"2024-06-10T00:52:45.993183+0000","last_became_peered":"2024-06-10T00:52:45.993183+0000","last_unstale":"2024-06-10T00:54:22.667762+0000","last_undegraded":"2024-06-10T00:54:22.667762+0000","last_fullsized":"2024-06-10T00:54:22.667762+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:47:00.541427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.566044+0000","last_change":"2024-06-10T00:54:22.565600+0000","last_active":"2024-06-10T00:54:22.566044+0000","last_peered":"2024-06-10T00:54:22.566044+0000","last_clean":"2024-06-10T00:54:22.566044+0000","last_became_active":"2024-06-10T00:54:22.562561+0000","last_became_peered":"2024-06-10T00:54:22.562561+0000","last_unstale":"2024-06-10T00:54:22.566044+0000","last_undegraded":"2024-06-10T00:54:22.566044+0000","last_fullsized":"2024-06-10T00:54:22.566044+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:47:58.748050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"100'15","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365162+0000","last_change":"2024-06-10T00:53:22.199668+0000","last_active":"2024-06-10T00:55:20.365162+0000","last_peered":"2024-06-10T00:55:20.365162+0000","last_clean":"2024-06-10T00:55:20.365162+0000","last_became_active":"2024-06-10T00:53:22.199329+0000","last_became_peered":"2024-06-10T00:53:22.199329+0000","last_unstale":"2024-06-10T00:55:20.365162+0000","last_undegraded":"2024-06-10T00:55:20.365162+0000","last_fullsized":"2024-06-10T00:55:20.365162+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:37.640478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554689+0000","last_change":"2024-06-10T00:52:45.995767+0000","last_active":"2024-06-10T00:54:22.554689+0000","last_peered":"2024-06-10T00:54:22.554689+0000","last_clean":"2024-06-10T00:54:22.554689+0000","last_became_active":"2024-06-10T00:52:45.994659+0000","last_became_peered":"2024-06-10T00:52:45.994659+0000","last_unstale":"2024-06-10T00:54:22.554689+0000","last_undegraded":"2024-06-10T00:54:22.554689+0000","last_fullsized":"2024-06-10T00:54:22.554689+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:03:45.652248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'8","reported_seq":247,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:50.585916+0000","last_change":"2024-06-10T00:52:32.761637+0000","last_active":"2024-06-10T00:56:50.585916+0000","last_peered":"2024-06-10T00:56:50.585916+0000","last_clean":"2024-06-10T00:56:50.585916+0000","last_became_active":"2024-06-10T00:52:32.761144+0000","last_became_peered":"2024-06-10T00:52:32.761144+0000","last_unstale":"2024-06-10T00:56:50.585916+0000","last_undegraded":"2024-06-10T00:56:50.585916+0000","last_fullsized":"2024-06-10T00:56:50.585916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:55:05.616564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1461,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":196,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.362574+0000","last_change":"2024-06-10T00:52:35.962683+0000","last_active":"2024-06-10T00:55:20.362574+0000","last_peered":"2024-06-10T00:55:20.362574+0000","last_clean":"2024-06-10T00:55:20.362574+0000","last_became_active":"2024-06-10T00:52:35.962327+0000","last_became_peered":"2024-06-10T00:52:35.962327+0000","last_unstale":"2024-06-10T00:55:20.362574+0000","last_undegraded":"2024-06-10T00:55:20.362574+0000","last_fullsized":"2024-06-10T00:55:20.362574+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:38:19.063306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554563+0000","last_change":"2024-06-10T00:52:45.996882+0000","last_active":"2024-06-10T00:54:22.554563+0000","last_peered":"2024-06-10T00:54:22.554563+0000","last_clean":"2024-06-10T00:54:22.554563+0000","last_became_active":"2024-06-10T00:52:45.995767+0000","last_became_peered":"2024-06-10T00:52:45.995767+0000","last_unstale":"2024-06-10T00:54:22.554563+0000","last_undegraded":"2024-06-10T00:54:22.554563+0000","last_fullsized":"2024-06-10T00:54:22.554563+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:28:03.766944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'3","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553970+0000","last_change":"2024-06-10T00:53:22.200546+0000","last_active":"2024-06-10T00:54:22.553970+0000","last_peered":"2024-06-10T00:54:22.553970+0000","last_clean":"2024-06-10T00:54:22.553970+0000","last_became_active":"2024-06-10T00:53:22.200059+0000","last_became_peered":"2024-06-10T00:53:22.200059+0000","last_unstale":"2024-06-10T00:54:22.553970+0000","last_undegraded":"2024-06-10T00:54:22.553970+0000","last_fullsized":"2024-06-10T00:54:22.553970+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:08:47.682594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.356418+0000","last_change":"2024-06-10T00:52:35.963105+0000","last_active":"2024-06-10T00:55:20.356418+0000","last_peered":"2024-06-10T00:55:20.356418+0000","last_clean":"2024-06-10T00:55:20.356418+0000","last_became_active":"2024-06-10T00:52:35.962627+0000","last_became_peered":"2024-06-10T00:52:35.962627+0000","last_unstale":"2024-06-10T00:55:20.356418+0000","last_undegraded":"2024-06-10T00:55:20.356418+0000","last_fullsized":"2024-06-10T00:55:20.356418+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T00:55:51.231795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.559190+0000","last_change":"2024-06-10T00:52:45.992557+0000","last_active":"2024-06-10T00:54:22.559190+0000","last_peered":"2024-06-10T00:54:22.559190+0000","last_clean":"2024-06-10T00:54:22.559190+0000","last_became_active":"2024-06-10T00:52:45.992166+0000","last_became_peered":"2024-06-10T00:52:45.992166+0000","last_unstale":"2024-06-10T00:54:22.559190+0000","last_undegraded":"2024-06-10T00:54:22.559190+0000","last_fullsized":"2024-06-10T00:54:22.559190+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:49:22.368409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553428+0000","last_change":"2024-06-10T00:52:32.751867+0000","last_active":"2024-06-10T00:54:22.553428+0000","last_peered":"2024-06-10T00:54:22.553428+0000","last_clean":"2024-06-10T00:54:22.553428+0000","last_became_active":"2024-06-10T00:52:32.751516+0000","last_became_peered":"2024-06-10T00:52:32.751516+0000","last_unstale":"2024-06-10T00:54:22.553428+0000","last_undegraded":"2024-06-10T00:54:22.553428+0000","last_fullsized":"2024-06-10T00:54:22.553428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:37:55.920613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.370097+0000","last_change":"2024-06-10T00:52:35.957772+0000","last_active":"2024-06-10T00:55:20.370097+0000","last_peered":"2024-06-10T00:55:20.370097+0000","last_clean":"2024-06-10T00:55:20.370097+0000","last_became_active":"2024-06-10T00:52:35.957470+0000","last_became_peered":"2024-06-10T00:52:35.957470+0000","last_unstale":"2024-06-10T00:55:20.370097+0000","last_undegraded":"2024-06-10T00:55:20.370097+0000","last_fullsized":"2024-06-10T00:55:20.370097+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:16:43.645292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:49.523884+0000","last_change":"2024-06-10T00:52:45.996807+0000","last_active":"2024-06-10T00:54:49.523884+0000","last_peered":"2024-06-10T00:54:49.523884+0000","last_clean":"2024-06-10T00:54:49.523884+0000","last_became_active":"2024-06-10T00:52:45.996378+0000","last_became_peered":"2024-06-10T00:52:45.996378+0000","last_unstale":"2024-06-10T00:54:49.523884+0000","last_undegraded":"2024-06-10T00:54:49.523884+0000","last_fullsized":"2024-06-10T00:54:49.523884+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:40:54.251790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'1","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561873+0000","last_change":"2024-06-10T00:52:32.759625+0000","last_active":"2024-06-10T00:54:22.561873+0000","last_peered":"2024-06-10T00:54:22.561873+0000","last_clean":"2024-06-10T00:54:22.561873+0000","last_became_active":"2024-06-10T00:52:32.759299+0000","last_became_peered":"2024-06-10T00:52:32.759299+0000","last_unstale":"2024-06-10T00:54:22.561873+0000","last_undegraded":"2024-06-10T00:54:22.561873+0000","last_fullsized":"2024-06-10T00:54:22.561873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:11:44.966453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365673+0000","last_change":"2024-06-10T00:52:35.957672+0000","last_active":"2024-06-10T00:55:20.365673+0000","last_peered":"2024-06-10T00:55:20.365673+0000","last_clean":"2024-06-10T00:55:20.365673+0000","last_became_active":"2024-06-10T00:52:35.957328+0000","last_became_peered":"2024-06-10T00:52:35.957328+0000","last_unstale":"2024-06-10T00:55:20.365673+0000","last_undegraded":"2024-06-10T00:55:20.365673+0000","last_fullsized":"2024-06-10T00:55:20.365673+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:43:59.658178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.601278+0000","last_change":"2024-06-10T00:52:45.996677+0000","last_active":"2024-06-10T00:54:22.601278+0000","last_peered":"2024-06-10T00:54:22.601278+0000","last_clean":"2024-06-10T00:54:22.601278+0000","last_became_active":"2024-06-10T00:52:45.994483+0000","last_became_peered":"2024-06-10T00:52:45.994483+0000","last_unstale":"2024-06-10T00:54:22.601278+0000","last_undegraded":"2024-06-10T00:54:22.601278+0000","last_fullsized":"2024-06-10T00:54:22.601278+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:03:28.712072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'5","reported_seq":240,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:49.583797+0000","last_change":"2024-06-10T00:52:32.754829+0000","last_active":"2024-06-10T00:56:49.583797+0000","last_peered":"2024-06-10T00:56:49.583797+0000","last_clean":"2024-06-10T00:56:49.583797+0000","last_became_active":"2024-06-10T00:52:32.754386+0000","last_became_peered":"2024-06-10T00:52:32.754386+0000","last_unstale":"2024-06-10T00:56:49.583797+0000","last_undegraded":"2024-06-10T00:56:49.583797+0000","last_fullsized":"2024-06-10T00:56:49.583797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:42:00.870324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.371610+0000","last_change":"2024-06-10T00:52:35.966871+0000","last_active":"2024-06-10T00:55:20.371610+0000","last_peered":"2024-06-10T00:55:20.371610+0000","last_clean":"2024-06-10T00:55:20.371610+0000","last_became_active":"2024-06-10T00:52:35.966492+0000","last_became_peered":"2024-06-10T00:52:35.966492+0000","last_unstale":"2024-06-10T00:55:20.371610+0000","last_undegraded":"2024-06-10T00:55:20.371610+0000","last_fullsized":"2024-06-10T00:55:20.371610+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:24:41.048935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:23.582564+0000","last_change":"2024-06-10T00:54:22.879457+0000","last_active":"2024-06-10T00:54:23.582564+0000","last_peered":"2024-06-10T00:54:23.582564+0000","last_clean":"2024-06-10T00:54:23.582564+0000","last_became_active":"2024-06-10T00:54:22.878978+0000","last_became_peered":"2024-06-10T00:54:22.878978+0000","last_unstale":"2024-06-10T00:54:23.582564+0000","last_undegraded":"2024-06-10T00:54:23.582564+0000","last_fullsized":"2024-06-10T00:54:23.582564+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:46:38.819250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404659+0000","last_change":"2024-06-10T00:52:32.765794+0000","last_active":"2024-06-10T00:54:24.404659+0000","last_peered":"2024-06-10T00:54:24.404659+0000","last_clean":"2024-06-10T00:54:24.404659+0000","last_became_active":"2024-06-10T00:52:32.765276+0000","last_became_peered":"2024-06-10T00:52:32.765276+0000","last_unstale":"2024-06-10T00:54:24.404659+0000","last_undegraded":"2024-06-10T00:54:24.404659+0000","last_fullsized":"2024-06-10T00:54:24.404659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:06:51.209012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365186+0000","last_change":"2024-06-10T00:52:35.954897+0000","last_active":"2024-06-10T00:55:20.365186+0000","last_peered":"2024-06-10T00:55:20.365186+0000","last_clean":"2024-06-10T00:55:20.365186+0000","last_became_active":"2024-06-10T00:52:35.954565+0000","last_became_peered":"2024-06-10T00:52:35.954565+0000","last_unstale":"2024-06-10T00:55:20.365186+0000","last_undegraded":"2024-06-10T00:55:20.365186+0000","last_fullsized":"2024-06-10T00:55:20.365186+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:39:26.702804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668190+0000","last_change":"2024-06-10T00:52:45.992194+0000","last_active":"2024-06-10T00:54:22.668190+0000","last_peered":"2024-06-10T00:54:22.668190+0000","last_clean":"2024-06-10T00:54:22.668190+0000","last_became_active":"2024-06-10T00:52:45.991453+0000","last_became_peered":"2024-06-10T00:52:45.991453+0000","last_unstale":"2024-06-10T00:54:22.668190+0000","last_undegraded":"2024-06-10T00:54:22.668190+0000","last_fullsized":"2024-06-10T00:54:22.668190+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:11:10.775610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561610+0000","last_change":"2024-06-10T00:52:32.962214+0000","last_active":"2024-06-10T00:54:22.561610+0000","last_peered":"2024-06-10T00:54:22.561610+0000","last_clean":"2024-06-10T00:54:22.561610+0000","last_became_active":"2024-06-10T00:52:32.961835+0000","last_became_peered":"2024-06-10T00:52:32.961835+0000","last_unstale":"2024-06-10T00:54:22.561610+0000","last_undegraded":"2024-06-10T00:54:22.561610+0000","last_fullsized":"2024-06-10T00:54:22.561610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:36:44.724122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.377485+0000","last_change":"2024-06-10T00:52:35.963413+0000","last_active":"2024-06-10T00:55:20.377485+0000","last_peered":"2024-06-10T00:55:20.377485+0000","last_clean":"2024-06-10T00:55:20.377485+0000","last_became_active":"2024-06-10T00:52:35.962922+0000","last_became_peered":"2024-06-10T00:52:35.962922+0000","last_unstale":"2024-06-10T00:55:20.377485+0000","last_undegraded":"2024-06-10T00:55:20.377485+0000","last_fullsized":"2024-06-10T00:55:20.377485+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:05:32.164696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":34,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.560580+0000","last_change":"2024-06-10T00:54:22.560196+0000","last_active":"2024-06-10T00:54:22.560580+0000","last_peered":"2024-06-10T00:54:22.560580+0000","last_clean":"2024-06-10T00:54:22.560580+0000","last_became_active":"2024-06-10T00:54:22.559826+0000","last_became_peered":"2024-06-10T00:54:22.559826+0000","last_unstale":"2024-06-10T00:54:22.560580+0000","last_undegraded":"2024-06-10T00:54:22.560580+0000","last_fullsized":"2024-06-10T00:54:22.560580+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:09:28.456394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'46","reported_seq":486,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:50.589095+0000","last_change":"2024-06-10T00:52:32.761592+0000","last_active":"2024-06-10T00:56:50.589095+0000","last_peered":"2024-06-10T00:56:50.589095+0000","last_clean":"2024-06-10T00:56:50.589095+0000","last_became_active":"2024-06-10T00:52:32.761169+0000","last_became_peered":"2024-06-10T00:52:32.761169+0000","last_unstale":"2024-06-10T00:56:50.589095+0000","last_undegraded":"2024-06-10T00:56:50.589095+0000","last_fullsized":"2024-06-10T00:56:50.589095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:08:33.015035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77873,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":395,"num_read_kb":276,"num_write":47,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.351972+0000","last_change":"2024-06-10T00:52:35.955304+0000","last_active":"2024-06-10T00:55:20.351972+0000","last_peered":"2024-06-10T00:55:20.351972+0000","last_clean":"2024-06-10T00:55:20.351972+0000","last_became_active":"2024-06-10T00:52:35.954949+0000","last_became_peered":"2024-06-10T00:52:35.954949+0000","last_unstale":"2024-06-10T00:55:20.351972+0000","last_undegraded":"2024-06-10T00:55:20.351972+0000","last_fullsized":"2024-06-10T00:55:20.351972+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:56:42.890022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561827+0000","last_change":"2024-06-10T00:52:45.992974+0000","last_active":"2024-06-10T00:54:22.561827+0000","last_peered":"2024-06-10T00:54:22.561827+0000","last_clean":"2024-06-10T00:54:22.561827+0000","last_became_active":"2024-06-10T00:52:45.992490+0000","last_became_peered":"2024-06-10T00:52:45.992490+0000","last_unstale":"2024-06-10T00:54:22.561827+0000","last_undegraded":"2024-06-10T00:54:22.561827+0000","last_fullsized":"2024-06-10T00:54:22.561827+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:03:45.543476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'6","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554437+0000","last_change":"2024-06-10T00:52:32.762347+0000","last_active":"2024-06-10T00:54:22.554437+0000","last_peered":"2024-06-10T00:54:22.554437+0000","last_clean":"2024-06-10T00:54:22.554437+0000","last_became_active":"2024-06-10T00:52:32.761049+0000","last_became_peered":"2024-06-10T00:52:32.761049+0000","last_unstale":"2024-06-10T00:54:22.554437+0000","last_undegraded":"2024-06-10T00:54:22.554437+0000","last_fullsized":"2024-06-10T00:54:22.554437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:29:38.384451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364495+0000","last_change":"2024-06-10T00:52:35.961299+0000","last_active":"2024-06-10T00:55:20.364495+0000","last_peered":"2024-06-10T00:55:20.364495+0000","last_clean":"2024-06-10T00:55:20.364495+0000","last_became_active":"2024-06-10T00:52:35.960966+0000","last_became_peered":"2024-06-10T00:52:35.960966+0000","last_unstale":"2024-06-10T00:55:20.364495+0000","last_undegraded":"2024-06-10T00:55:20.364495+0000","last_fullsized":"2024-06-10T00:55:20.364495+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:32:21.269689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.559474+0000","last_change":"2024-06-10T00:52:46.002091+0000","last_active":"2024-06-10T00:54:22.559474+0000","last_peered":"2024-06-10T00:54:22.559474+0000","last_clean":"2024-06-10T00:54:22.559474+0000","last_became_active":"2024-06-10T00:52:46.001677+0000","last_became_peered":"2024-06-10T00:52:46.001677+0000","last_unstale":"2024-06-10T00:54:22.559474+0000","last_undegraded":"2024-06-10T00:54:22.559474+0000","last_fullsized":"2024-06-10T00:54:22.559474+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:16:56.405720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'3","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.562287+0000","last_change":"2024-06-10T00:52:32.756765+0000","last_active":"2024-06-10T00:54:22.562287+0000","last_peered":"2024-06-10T00:54:22.562287+0000","last_clean":"2024-06-10T00:54:22.562287+0000","last_became_active":"2024-06-10T00:52:32.756470+0000","last_became_peered":"2024-06-10T00:52:32.756470+0000","last_unstale":"2024-06-10T00:54:22.562287+0000","last_undegraded":"2024-06-10T00:54:22.562287+0000","last_fullsized":"2024-06-10T00:54:22.562287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:29:24.854913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"100'15","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364256+0000","last_change":"2024-06-10T00:52:35.960693+0000","last_active":"2024-06-10T00:55:20.364256+0000","last_peered":"2024-06-10T00:55:20.364256+0000","last_clean":"2024-06-10T00:55:20.364256+0000","last_became_active":"2024-06-10T00:52:35.958397+0000","last_became_peered":"2024-06-10T00:52:35.958397+0000","last_unstale":"2024-06-10T00:55:20.364256+0000","last_undegraded":"2024-06-10T00:55:20.364256+0000","last_fullsized":"2024-06-10T00:55:20.364256+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:23:47.556462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556200+0000","last_change":"2024-06-10T00:52:46.001475+0000","last_active":"2024-06-10T00:54:22.556200+0000","last_peered":"2024-06-10T00:54:22.556200+0000","last_clean":"2024-06-10T00:54:22.556200+0000","last_became_active":"2024-06-10T00:52:46.000983+0000","last_became_peered":"2024-06-10T00:52:46.000983+0000","last_unstale":"2024-06-10T00:54:22.556200+0000","last_undegraded":"2024-06-10T00:54:22.556200+0000","last_fullsized":"2024-06-10T00:54:22.556200+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:23:56.285356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"98'3","reported_seq":241,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:51.586997+0000","last_change":"2024-06-10T00:52:32.756665+0000","last_active":"2024-06-10T00:56:51.586997+0000","last_peered":"2024-06-10T00:56:51.586997+0000","last_clean":"2024-06-10T00:56:51.586997+0000","last_became_active":"2024-06-10T00:52:32.756221+0000","last_became_peered":"2024-06-10T00:52:32.756221+0000","last_unstale":"2024-06-10T00:56:51.586997+0000","last_undegraded":"2024-06-10T00:56:51.586997+0000","last_fullsized":"2024-06-10T00:56:51.586997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:50:34.789063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":195,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"100'6","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364652+0000","last_change":"2024-06-10T00:53:22.125844+0000","last_active":"2024-06-10T00:55:20.364652+0000","last_peered":"2024-06-10T00:55:20.364652+0000","last_clean":"2024-06-10T00:55:20.364652+0000","last_became_active":"2024-06-10T00:53:22.125449+0000","last_became_peered":"2024-06-10T00:53:22.125449+0000","last_unstale":"2024-06-10T00:55:20.364652+0000","last_undegraded":"2024-06-10T00:55:20.364652+0000","last_fullsized":"2024-06-10T00:55:20.364652+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:08:42.936702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553904+0000","last_change":"2024-06-10T00:52:45.999300+0000","last_active":"2024-06-10T00:54:22.553904+0000","last_peered":"2024-06-10T00:54:22.553904+0000","last_clean":"2024-06-10T00:54:22.553904+0000","last_became_active":"2024-06-10T00:52:45.998860+0000","last_became_peered":"2024-06-10T00:52:45.998860+0000","last_unstale":"2024-06-10T00:54:22.553904+0000","last_undegraded":"2024-06-10T00:54:22.553904+0000","last_fullsized":"2024-06-10T00:54:22.553904+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:14:24.936870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'15","reported_seq":75,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.557182+0000","last_change":"2024-06-10T00:53:22.210710+0000","last_active":"2024-06-10T00:54:22.557182+0000","last_peered":"2024-06-10T00:54:22.557182+0000","last_clean":"2024-06-10T00:54:22.557182+0000","last_became_active":"2024-06-10T00:53:22.202461+0000","last_became_peered":"2024-06-10T00:53:22.202461+0000","last_unstale":"2024-06-10T00:54:22.557182+0000","last_undegraded":"2024-06-10T00:54:22.557182+0000","last_fullsized":"2024-06-10T00:54:22.557182+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:28:14.671776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364560+0000","last_change":"2024-06-10T00:52:35.966960+0000","last_active":"2024-06-10T00:55:20.364560+0000","last_peered":"2024-06-10T00:55:20.364560+0000","last_clean":"2024-06-10T00:55:20.364560+0000","last_became_active":"2024-06-10T00:52:35.966630+0000","last_became_peered":"2024-06-10T00:52:35.966630+0000","last_unstale":"2024-06-10T00:55:20.364560+0000","last_undegraded":"2024-06-10T00:55:20.364560+0000","last_fullsized":"2024-06-10T00:55:20.364560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:04.117405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561586+0000","last_change":"2024-06-10T00:52:45.992955+0000","last_active":"2024-06-10T00:54:22.561586+0000","last_peered":"2024-06-10T00:54:22.561586+0000","last_clean":"2024-06-10T00:54:22.561586+0000","last_became_active":"2024-06-10T00:52:45.992457+0000","last_became_peered":"2024-06-10T00:52:45.992457+0000","last_unstale":"2024-06-10T00:54:22.561586+0000","last_undegraded":"2024-06-10T00:54:22.561586+0000","last_fullsized":"2024-06-10T00:54:22.561586+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:13:49.491183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556768+0000","last_change":"2024-06-10T00:52:32.763794+0000","last_active":"2024-06-10T00:54:22.556768+0000","last_peered":"2024-06-10T00:54:22.556768+0000","last_clean":"2024-06-10T00:54:22.556768+0000","last_became_active":"2024-06-10T00:52:32.763406+0000","last_became_peered":"2024-06-10T00:52:32.763406+0000","last_unstale":"2024-06-10T00:54:22.556768+0000","last_undegraded":"2024-06-10T00:54:22.556768+0000","last_fullsized":"2024-06-10T00:54:22.556768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:50:33.580163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"100'5","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374120+0000","last_change":"2024-06-10T00:53:22.130891+0000","last_active":"2024-06-10T00:55:20.374120+0000","last_peered":"2024-06-10T00:55:20.374120+0000","last_clean":"2024-06-10T00:55:20.374120+0000","last_became_active":"2024-06-10T00:53:22.130508+0000","last_became_peered":"2024-06-10T00:53:22.130508+0000","last_unstale":"2024-06-10T00:55:20.374120+0000","last_undegraded":"2024-06-10T00:55:20.374120+0000","last_fullsized":"2024-06-10T00:55:20.374120+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:28:01.919825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":34,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.563757+0000","last_change":"2024-06-10T00:54:22.563272+0000","last_active":"2024-06-10T00:54:22.563757+0000","last_peered":"2024-06-10T00:54:22.563757+0000","last_clean":"2024-06-10T00:54:22.563757+0000","last_became_active":"2024-06-10T00:54:22.562767+0000","last_became_peered":"2024-06-10T00:54:22.562767+0000","last_unstale":"2024-06-10T00:54:22.563757+0000","last_undegraded":"2024-06-10T00:54:22.563757+0000","last_fullsized":"2024-06-10T00:54:22.563757+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:52:20.739801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:40.422237+0000","last_change":"2024-06-10T00:52:32.960515+0000","last_active":"2024-06-10T00:54:40.422237+0000","last_peered":"2024-06-10T00:54:40.422237+0000","last_clean":"2024-06-10T00:54:40.422237+0000","last_became_active":"2024-06-10T00:52:32.960163+0000","last_became_peered":"2024-06-10T00:52:32.960163+0000","last_unstale":"2024-06-10T00:54:40.422237+0000","last_undegraded":"2024-06-10T00:54:40.422237+0000","last_fullsized":"2024-06-10T00:54:40.422237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:39:51.374998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"100'4","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374185+0000","last_change":"2024-06-10T00:52:35.961884+0000","last_active":"2024-06-10T00:55:20.374185+0000","last_peered":"2024-06-10T00:55:20.374185+0000","last_clean":"2024-06-10T00:55:20.374185+0000","last_became_active":"2024-06-10T00:52:35.961497+0000","last_became_peered":"2024-06-10T00:52:35.961497+0000","last_unstale":"2024-06-10T00:55:20.374185+0000","last_undegraded":"2024-06-10T00:55:20.374185+0000","last_fullsized":"2024-06-10T00:55:20.374185+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:16:09.284184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553591+0000","last_change":"2024-06-10T00:52:45.990846+0000","last_active":"2024-06-10T00:54:22.553591+0000","last_peered":"2024-06-10T00:54:22.553591+0000","last_clean":"2024-06-10T00:54:22.553591+0000","last_became_active":"2024-06-10T00:52:45.990441+0000","last_became_peered":"2024-06-10T00:52:45.990441+0000","last_unstale":"2024-06-10T00:54:22.553591+0000","last_undegraded":"2024-06-10T00:54:22.553591+0000","last_fullsized":"2024-06-10T00:54:22.553591+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:19:35.690378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'1","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:39.101650+0000","last_change":"2024-06-10T00:52:32.753913+0000","last_active":"2024-06-10T00:54:39.101650+0000","last_peered":"2024-06-10T00:54:39.101650+0000","last_clean":"2024-06-10T00:54:39.101650+0000","last_became_active":"2024-06-10T00:52:32.753174+0000","last_became_peered":"2024-06-10T00:52:32.753174+0000","last_unstale":"2024-06-10T00:54:39.101650+0000","last_undegraded":"2024-06-10T00:54:39.101650+0000","last_fullsized":"2024-06-10T00:54:39.101650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:13:20.048635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"100'7","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.359209+0000","last_change":"2024-06-10T00:52:35.965783+0000","last_active":"2024-06-10T00:55:20.359209+0000","last_peered":"2024-06-10T00:55:20.359209+0000","last_clean":"2024-06-10T00:55:20.359209+0000","last_became_active":"2024-06-10T00:52:35.965432+0000","last_became_peered":"2024-06-10T00:52:35.965432+0000","last_unstale":"2024-06-10T00:55:20.359209+0000","last_undegraded":"2024-06-10T00:55:20.359209+0000","last_fullsized":"2024-06-10T00:55:20.359209+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:52:01.240318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553790+0000","last_change":"2024-06-10T00:52:46.241662+0000","last_active":"2024-06-10T00:54:22.553790+0000","last_peered":"2024-06-10T00:54:22.553790+0000","last_clean":"2024-06-10T00:54:22.553790+0000","last_became_active":"2024-06-10T00:52:46.241211+0000","last_became_peered":"2024-06-10T00:52:46.241211+0000","last_unstale":"2024-06-10T00:54:22.553790+0000","last_undegraded":"2024-06-10T00:54:22.553790+0000","last_fullsized":"2024-06-10T00:54:22.553790+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:12.041290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"100'8","reported_seq":24,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.367303+0000","last_change":"2024-06-10T00:53:22.200590+0000","last_active":"2024-06-10T00:55:20.367303+0000","last_peered":"2024-06-10T00:55:20.367303+0000","last_clean":"2024-06-10T00:55:20.367303+0000","last_became_active":"2024-06-10T00:53:22.200046+0000","last_became_peered":"2024-06-10T00:53:22.200046+0000","last_unstale":"2024-06-10T00:55:20.367303+0000","last_undegraded":"2024-06-10T00:55:20.367303+0000","last_fullsized":"2024-06-10T00:55:20.367303+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:43:22.378316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"98'4","reported_seq":241,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:51.586439+0000","last_change":"2024-06-10T00:52:32.959904+0000","last_active":"2024-06-10T00:56:51.586439+0000","last_peered":"2024-06-10T00:56:51.586439+0000","last_clean":"2024-06-10T00:56:51.586439+0000","last_became_active":"2024-06-10T00:52:32.959506+0000","last_became_peered":"2024-06-10T00:52:32.959506+0000","last_unstale":"2024-06-10T00:56:51.586439+0000","last_undegraded":"2024-06-10T00:56:51.586439+0000","last_fullsized":"2024-06-10T00:56:51.586439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:23:15.950345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":192,"num_read_kb":192,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404433+0000","last_change":"2024-06-10T00:52:45.993305+0000","last_active":"2024-06-10T00:54:24.404433+0000","last_peered":"2024-06-10T00:54:24.404433+0000","last_clean":"2024-06-10T00:54:24.404433+0000","last_became_active":"2024-06-10T00:52:45.992682+0000","last_became_peered":"2024-06-10T00:52:45.992682+0000","last_unstale":"2024-06-10T00:54:24.404433+0000","last_undegraded":"2024-06-10T00:54:24.404433+0000","last_fullsized":"2024-06-10T00:54:24.404433+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:49:41.817126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375925+0000","last_change":"2024-06-10T00:52:35.965893+0000","last_active":"2024-06-10T00:55:20.375925+0000","last_peered":"2024-06-10T00:55:20.375925+0000","last_clean":"2024-06-10T00:55:20.375925+0000","last_became_active":"2024-06-10T00:52:35.965426+0000","last_became_peered":"2024-06-10T00:52:35.965426+0000","last_unstale":"2024-06-10T00:55:20.375925+0000","last_undegraded":"2024-06-10T00:55:20.375925+0000","last_fullsized":"2024-06-10T00:55:20.375925+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:29:19.640836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'3","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:38.154378+0000","last_change":"2024-06-10T00:52:32.753631+0000","last_active":"2024-06-10T00:54:38.154378+0000","last_peered":"2024-06-10T00:54:38.154378+0000","last_clean":"2024-06-10T00:54:38.154378+0000","last_became_active":"2024-06-10T00:52:32.753260+0000","last_became_peered":"2024-06-10T00:52:32.753260+0000","last_unstale":"2024-06-10T00:54:38.154378+0000","last_undegraded":"2024-06-10T00:54:38.154378+0000","last_fullsized":"2024-06-10T00:54:38.154378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:12:55.932676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":369,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:24.404055+0000","last_change":"2024-06-10T00:51:09.267750+0000","last_active":"2024-06-10T00:54:24.404055+0000","last_peered":"2024-06-10T00:54:24.404055+0000","last_clean":"2024-06-10T00:54:24.404055+0000","last_became_active":"2024-06-10T00:51:09.267246+0000","last_became_peered":"2024-06-10T00:51:09.267246+0000","last_unstale":"2024-06-10T00:54:24.404055+0000","last_undegraded":"2024-06-10T00:54:24.404055+0000","last_fullsized":"2024-06-10T00:54:24.404055+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:47:44.965452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:47:44.965452+0000","last_clean_scrub_stamp":"2024-06-10T00:47:44.965452+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:48:21.100090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561655+0000","last_change":"2024-06-10T00:52:46.242152+0000","last_active":"2024-06-10T00:54:22.561655+0000","last_peered":"2024-06-10T00:54:22.561655+0000","last_clean":"2024-06-10T00:54:22.561655+0000","last_became_active":"2024-06-10T00:52:46.241542+0000","last_became_peered":"2024-06-10T00:52:46.241542+0000","last_unstale":"2024-06-10T00:54:22.561655+0000","last_undegraded":"2024-06-10T00:54:22.561655+0000","last_fullsized":"2024-06-10T00:54:22.561655+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:55:54.695465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365104+0000","last_change":"2024-06-10T00:52:35.963911+0000","last_active":"2024-06-10T00:55:20.365104+0000","last_peered":"2024-06-10T00:55:20.365104+0000","last_clean":"2024-06-10T00:55:20.365104+0000","last_became_active":"2024-06-10T00:52:35.963571+0000","last_became_peered":"2024-06-10T00:52:35.963571+0000","last_unstale":"2024-06-10T00:55:20.365104+0000","last_undegraded":"2024-06-10T00:55:20.365104+0000","last_fullsized":"2024-06-10T00:55:20.365104+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:27:16.757428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'3","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.558529+0000","last_change":"2024-06-10T00:53:22.129035+0000","last_active":"2024-06-10T00:54:22.558529+0000","last_peered":"2024-06-10T00:54:22.558529+0000","last_clean":"2024-06-10T00:54:22.558529+0000","last_became_active":"2024-06-10T00:53:22.128561+0000","last_became_peered":"2024-06-10T00:53:22.128561+0000","last_unstale":"2024-06-10T00:54:22.558529+0000","last_undegraded":"2024-06-10T00:54:22.558529+0000","last_fullsized":"2024-06-10T00:54:22.558529+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:02:43.313118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.558735+0000","last_change":"2024-06-10T00:52:45.996907+0000","last_active":"2024-06-10T00:54:22.558735+0000","last_peered":"2024-06-10T00:54:22.558735+0000","last_clean":"2024-06-10T00:54:22.558735+0000","last_became_active":"2024-06-10T00:52:45.996554+0000","last_became_peered":"2024-06-10T00:52:45.996554+0000","last_unstale":"2024-06-10T00:54:22.558735+0000","last_undegraded":"2024-06-10T00:54:22.558735+0000","last_fullsized":"2024-06-10T00:54:22.558735+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:00:46.824071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'1","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:38.584029+0000","last_change":"2024-06-10T00:52:32.960223+0000","last_active":"2024-06-10T00:54:38.584029+0000","last_peered":"2024-06-10T00:54:38.584029+0000","last_clean":"2024-06-10T00:54:38.584029+0000","last_became_active":"2024-06-10T00:52:32.959871+0000","last_became_peered":"2024-06-10T00:52:32.959871+0000","last_unstale":"2024-06-10T00:54:38.584029+0000","last_undegraded":"2024-06-10T00:54:38.584029+0000","last_fullsized":"2024-06-10T00:54:38.584029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:18:17.099140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.369679+0000","last_change":"2024-06-10T00:52:35.962516+0000","last_active":"2024-06-10T00:55:20.369679+0000","last_peered":"2024-06-10T00:55:20.369679+0000","last_clean":"2024-06-10T00:55:20.369679+0000","last_became_active":"2024-06-10T00:52:35.962187+0000","last_became_peered":"2024-06-10T00:52:35.962187+0000","last_unstale":"2024-06-10T00:55:20.369679+0000","last_undegraded":"2024-06-10T00:55:20.369679+0000","last_fullsized":"2024-06-10T00:55:20.369679+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:10:44.295192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:48.629265+0000","last_change":"2024-06-10T00:52:46.241554+0000","last_active":"2024-06-10T00:54:48.629265+0000","last_peered":"2024-06-10T00:54:48.629265+0000","last_clean":"2024-06-10T00:54:48.629265+0000","last_became_active":"2024-06-10T00:52:46.241142+0000","last_became_peered":"2024-06-10T00:52:46.241142+0000","last_unstale":"2024-06-10T00:54:48.629265+0000","last_undegraded":"2024-06-10T00:54:48.629265+0000","last_fullsized":"2024-06-10T00:54:48.629265+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:58:07.233080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'13","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:29.987897+0000","last_change":"2024-06-10T00:52:32.761854+0000","last_active":"2024-06-10T00:56:29.987897+0000","last_peered":"2024-06-10T00:56:29.987897+0000","last_clean":"2024-06-10T00:56:29.987897+0000","last_became_active":"2024-06-10T00:52:32.761522+0000","last_became_peered":"2024-06-10T00:52:32.761522+0000","last_unstale":"2024-06-10T00:56:29.987897+0000","last_undegraded":"2024-06-10T00:56:29.987897+0000","last_fullsized":"2024-06-10T00:56:29.987897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:06:43.269001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.370762+0000","last_change":"2024-06-10T00:52:35.959250+0000","last_active":"2024-06-10T00:55:20.370762+0000","last_peered":"2024-06-10T00:55:20.370762+0000","last_clean":"2024-06-10T00:55:20.370762+0000","last_became_active":"2024-06-10T00:52:35.958925+0000","last_became_peered":"2024-06-10T00:52:35.958925+0000","last_unstale":"2024-06-10T00:55:20.370762+0000","last_undegraded":"2024-06-10T00:55:20.370762+0000","last_fullsized":"2024-06-10T00:55:20.370762+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:11:12.355505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554191+0000","last_change":"2024-06-10T00:52:45.991819+0000","last_active":"2024-06-10T00:54:22.554191+0000","last_peered":"2024-06-10T00:54:22.554191+0000","last_clean":"2024-06-10T00:54:22.554191+0000","last_became_active":"2024-06-10T00:52:45.991408+0000","last_became_peered":"2024-06-10T00:52:45.991408+0000","last_unstale":"2024-06-10T00:54:22.554191+0000","last_undegraded":"2024-06-10T00:54:22.554191+0000","last_fullsized":"2024-06-10T00:54:22.554191+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:16:34.912359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'4","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:41.719222+0000","last_change":"2024-06-10T00:52:32.869094+0000","last_active":"2024-06-10T00:54:41.719222+0000","last_peered":"2024-06-10T00:54:41.719222+0000","last_clean":"2024-06-10T00:54:41.719222+0000","last_became_active":"2024-06-10T00:52:32.868664+0000","last_became_peered":"2024-06-10T00:52:32.868664+0000","last_unstale":"2024-06-10T00:54:41.719222+0000","last_undegraded":"2024-06-10T00:54:41.719222+0000","last_fullsized":"2024-06-10T00:54:41.719222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T04:50:38.051090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374722+0000","last_change":"2024-06-10T00:52:35.952356+0000","last_active":"2024-06-10T00:55:20.374722+0000","last_peered":"2024-06-10T00:55:20.374722+0000","last_clean":"2024-06-10T00:55:20.374722+0000","last_became_active":"2024-06-10T00:52:35.952002+0000","last_became_peered":"2024-06-10T00:52:35.952002+0000","last_unstale":"2024-06-10T00:55:20.374722+0000","last_undegraded":"2024-06-10T00:55:20.374722+0000","last_fullsized":"2024-06-10T00:55:20.374722+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:12:28.581522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668366+0000","last_change":"2024-06-10T00:52:46.241773+0000","last_active":"2024-06-10T00:54:22.668366+0000","last_peered":"2024-06-10T00:54:22.668366+0000","last_clean":"2024-06-10T00:54:22.668366+0000","last_became_active":"2024-06-10T00:52:46.241272+0000","last_became_peered":"2024-06-10T00:52:46.241272+0000","last_unstale":"2024-06-10T00:54:22.668366+0000","last_undegraded":"2024-06-10T00:54:22.668366+0000","last_fullsized":"2024-06-10T00:54:22.668366+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:07:28.337536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:43.509696+0000","last_change":"2024-06-10T00:52:32.960100+0000","last_active":"2024-06-10T00:54:43.509696+0000","last_peered":"2024-06-10T00:54:43.509696+0000","last_clean":"2024-06-10T00:54:43.509696+0000","last_became_active":"2024-06-10T00:52:32.959779+0000","last_became_peered":"2024-06-10T00:52:32.959779+0000","last_unstale":"2024-06-10T00:54:43.509696+0000","last_undegraded":"2024-06-10T00:54:43.509696+0000","last_fullsized":"2024-06-10T00:54:43.509696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:09.555753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'2","reported_seq":40,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.326494+0000","last_change":"2024-06-10T00:52:35.955586+0000","last_active":"2024-06-10T00:55:20.326494+0000","last_peered":"2024-06-10T00:55:20.326494+0000","last_clean":"2024-06-10T00:55:20.326494+0000","last_became_active":"2024-06-10T00:52:35.955216+0000","last_became_peered":"2024-06-10T00:52:35.955216+0000","last_unstale":"2024-06-10T00:55:20.326494+0000","last_undegraded":"2024-06-10T00:55:20.326494+0000","last_fullsized":"2024-06-10T00:55:20.326494+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:23:12.197182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:23.582601+0000","last_change":"2024-06-10T00:54:22.602670+0000","last_active":"2024-06-10T00:54:23.582601+0000","last_peered":"2024-06-10T00:54:23.582601+0000","last_clean":"2024-06-10T00:54:23.582601+0000","last_became_active":"2024-06-10T00:54:22.602218+0000","last_became_peered":"2024-06-10T00:54:22.602218+0000","last_unstale":"2024-06-10T00:54:23.582601+0000","last_undegraded":"2024-06-10T00:54:23.582601+0000","last_fullsized":"2024-06-10T00:54:23.582601+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:09:07.399025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:43.628895+0000","last_change":"2024-06-10T00:52:32.753444+0000","last_active":"2024-06-10T00:54:43.628895+0000","last_peered":"2024-06-10T00:54:43.628895+0000","last_clean":"2024-06-10T00:54:43.628895+0000","last_became_active":"2024-06-10T00:52:32.753149+0000","last_became_peered":"2024-06-10T00:52:32.753149+0000","last_unstale":"2024-06-10T00:54:43.628895+0000","last_undegraded":"2024-06-10T00:54:43.628895+0000","last_fullsized":"2024-06-10T00:54:43.628895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T05:24:28.241950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.365900+0000","last_change":"2024-06-10T00:52:35.966718+0000","last_active":"2024-06-10T00:55:20.365900+0000","last_peered":"2024-06-10T00:55:20.365900+0000","last_clean":"2024-06-10T00:55:20.365900+0000","last_became_active":"2024-06-10T00:52:35.965907+0000","last_became_peered":"2024-06-10T00:52:35.965907+0000","last_unstale":"2024-06-10T00:55:20.365900+0000","last_undegraded":"2024-06-10T00:55:20.365900+0000","last_fullsized":"2024-06-10T00:55:20.365900+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:21:34.261306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668092+0000","last_change":"2024-06-10T00:52:45.991034+0000","last_active":"2024-06-10T00:54:22.668092+0000","last_peered":"2024-06-10T00:54:22.668092+0000","last_clean":"2024-06-10T00:54:22.668092+0000","last_became_active":"2024-06-10T00:52:45.990546+0000","last_became_peered":"2024-06-10T00:52:45.990546+0000","last_unstale":"2024-06-10T00:54:22.668092+0000","last_undegraded":"2024-06-10T00:54:22.668092+0000","last_fullsized":"2024-06-10T00:54:22.668092+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:09:23.412922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"100'2","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:56:30.301571+0000","last_change":"2024-06-10T00:52:32.764024+0000","last_active":"2024-06-10T00:56:30.301571+0000","last_peered":"2024-06-10T00:56:30.301571+0000","last_clean":"2024-06-10T00:56:30.301571+0000","last_became_active":"2024-06-10T00:52:32.763449+0000","last_became_peered":"2024-06-10T00:52:32.763449+0000","last_unstale":"2024-06-10T00:56:30.301571+0000","last_undegraded":"2024-06-10T00:56:30.301571+0000","last_fullsized":"2024-06-10T00:56:30.301571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:50:23.605873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373684+0000","last_change":"2024-06-10T00:52:35.955570+0000","last_active":"2024-06-10T00:55:20.373684+0000","last_peered":"2024-06-10T00:55:20.373684+0000","last_clean":"2024-06-10T00:55:20.373684+0000","last_became_active":"2024-06-10T00:52:35.955248+0000","last_became_peered":"2024-06-10T00:52:35.955248+0000","last_unstale":"2024-06-10T00:55:20.373684+0000","last_undegraded":"2024-06-10T00:55:20.373684+0000","last_fullsized":"2024-06-10T00:55:20.373684+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:51:26.124499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.668028+0000","last_change":"2024-06-10T00:52:46.241776+0000","last_active":"2024-06-10T00:54:22.668028+0000","last_peered":"2024-06-10T00:54:22.668028+0000","last_clean":"2024-06-10T00:54:22.668028+0000","last_became_active":"2024-06-10T00:52:46.241276+0000","last_became_peered":"2024-06-10T00:52:46.241276+0000","last_unstale":"2024-06-10T00:54:22.668028+0000","last_undegraded":"2024-06-10T00:54:22.668028+0000","last_fullsized":"2024-06-10T00:54:22.668028+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:12:08.027938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":58,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.556321+0000","last_change":"2024-06-10T00:53:22.204034+0000","last_active":"2024-06-10T00:54:22.556321+0000","last_peered":"2024-06-10T00:54:22.556321+0000","last_clean":"2024-06-10T00:54:22.556321+0000","last_became_active":"2024-06-10T00:53:22.199988+0000","last_became_peered":"2024-06-10T00:53:22.199988+0000","last_unstale":"2024-06-10T00:54:22.556321+0000","last_undegraded":"2024-06-10T00:54:22.556321+0000","last_fullsized":"2024-06-10T00:54:22.556321+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:39:54.012360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373708+0000","last_change":"2024-06-10T00:52:35.959182+0000","last_active":"2024-06-10T00:55:20.373708+0000","last_peered":"2024-06-10T00:55:20.373708+0000","last_clean":"2024-06-10T00:55:20.373708+0000","last_became_active":"2024-06-10T00:52:35.958828+0000","last_became_peered":"2024-06-10T00:52:35.958828+0000","last_unstale":"2024-06-10T00:55:20.373708+0000","last_undegraded":"2024-06-10T00:55:20.373708+0000","last_fullsized":"2024-06-10T00:55:20.373708+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:29:58.857659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561603+0000","last_change":"2024-06-10T00:52:45.989739+0000","last_active":"2024-06-10T00:54:22.561603+0000","last_peered":"2024-06-10T00:54:22.561603+0000","last_clean":"2024-06-10T00:54:22.561603+0000","last_became_active":"2024-06-10T00:52:45.989279+0000","last_became_peered":"2024-06-10T00:52:45.989279+0000","last_unstale":"2024-06-10T00:54:22.561603+0000","last_undegraded":"2024-06-10T00:54:22.561603+0000","last_fullsized":"2024-06-10T00:54:22.561603+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:19:17.551947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'3","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.554798+0000","last_change":"2024-06-10T00:52:32.959662+0000","last_active":"2024-06-10T00:54:22.554798+0000","last_peered":"2024-06-10T00:54:22.554798+0000","last_clean":"2024-06-10T00:54:22.554798+0000","last_became_active":"2024-06-10T00:52:32.959286+0000","last_became_peered":"2024-06-10T00:52:32.959286+0000","last_unstale":"2024-06-10T00:54:22.554798+0000","last_undegraded":"2024-06-10T00:54:22.554798+0000","last_fullsized":"2024-06-10T00:54:22.554798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:26:04.361567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.373843+0000","last_change":"2024-06-10T00:52:35.960137+0000","last_active":"2024-06-10T00:55:20.373843+0000","last_peered":"2024-06-10T00:55:20.373843+0000","last_clean":"2024-06-10T00:55:20.373843+0000","last_became_active":"2024-06-10T00:52:35.959584+0000","last_became_peered":"2024-06-10T00:52:35.959584+0000","last_unstale":"2024-06-10T00:55:20.373843+0000","last_undegraded":"2024-06-10T00:55:20.373843+0000","last_fullsized":"2024-06-10T00:55:20.373843+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:23:38.702685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.878137+0000","last_change":"2024-06-10T00:52:46.002268+0000","last_active":"2024-06-10T00:54:22.878137+0000","last_peered":"2024-06-10T00:54:22.878137+0000","last_clean":"2024-06-10T00:54:22.878137+0000","last_became_active":"2024-06-10T00:52:46.001769+0000","last_became_peered":"2024-06-10T00:52:46.001769+0000","last_unstale":"2024-06-10T00:54:22.878137+0000","last_undegraded":"2024-06-10T00:54:22.878137+0000","last_fullsized":"2024-06-10T00:54:22.878137+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:33:51.365951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561629+0000","last_change":"2024-06-10T00:52:32.757112+0000","last_active":"2024-06-10T00:54:22.561629+0000","last_peered":"2024-06-10T00:54:22.561629+0000","last_clean":"2024-06-10T00:54:22.561629+0000","last_became_active":"2024-06-10T00:52:32.755589+0000","last_became_peered":"2024-06-10T00:52:32.755589+0000","last_unstale":"2024-06-10T00:54:22.561629+0000","last_undegraded":"2024-06-10T00:54:22.561629+0000","last_fullsized":"2024-06-10T00:54:22.561629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:15:12.814489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.364097+0000","last_change":"2024-06-10T00:52:35.955925+0000","last_active":"2024-06-10T00:55:20.364097+0000","last_peered":"2024-06-10T00:55:20.364097+0000","last_clean":"2024-06-10T00:55:20.364097+0000","last_became_active":"2024-06-10T00:52:35.955556+0000","last_became_peered":"2024-06-10T00:52:35.955556+0000","last_unstale":"2024-06-10T00:55:20.364097+0000","last_undegraded":"2024-06-10T00:55:20.364097+0000","last_fullsized":"2024-06-10T00:55:20.364097+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:42:18.324484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":27,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.561041+0000","last_change":"2024-06-10T00:52:45.988810+0000","last_active":"2024-06-10T00:54:22.561041+0000","last_peered":"2024-06-10T00:54:22.561041+0000","last_clean":"2024-06-10T00:54:22.561041+0000","last_became_active":"2024-06-10T00:52:45.988041+0000","last_became_peered":"2024-06-10T00:52:45.988041+0000","last_unstale":"2024-06-10T00:54:22.561041+0000","last_undegraded":"2024-06-10T00:54:22.561041+0000","last_fullsized":"2024-06-10T00:54:22.561041+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:44.972916+0000","last_clean_scrub_stamp":"2024-06-10T00:52:44.972916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T07:14:08.979802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:54:22.553851+0000","last_change":"2024-06-10T00:52:32.765493+0000","last_active":"2024-06-10T00:54:22.553851+0000","last_peered":"2024-06-10T00:54:22.553851+0000","last_clean":"2024-06-10T00:54:22.553851+0000","last_became_active":"2024-06-10T00:52:32.765193+0000","last_became_peered":"2024-06-10T00:52:32.765193+0000","last_unstale":"2024-06-10T00:54:22.553851+0000","last_undegraded":"2024-06-10T00:54:22.553851+0000","last_fullsized":"2024-06-10T00:54:22.553851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:31.734970+0000","last_clean_scrub_stamp":"2024-06-10T00:52:31.734970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T06:38:11.554673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374319+0000","last_change":"2024-06-10T00:52:35.960216+0000","last_active":"2024-06-10T00:55:20.374319+0000","last_peered":"2024-06-10T00:55:20.374319+0000","last_clean":"2024-06-10T00:55:20.374319+0000","last_became_active":"2024-06-10T00:52:35.959664+0000","last_became_peered":"2024-06-10T00:52:35.959664+0000","last_unstale":"2024-06-10T00:55:20.374319+0000","last_undegraded":"2024-06-10T00:55:20.374319+0000","last_fullsized":"2024-06-10T00:55:20.374319+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:41:58.086330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375776+0000","last_change":"2024-06-10T00:52:35.967043+0000","last_active":"2024-06-10T00:55:20.375776+0000","last_peered":"2024-06-10T00:55:20.375776+0000","last_clean":"2024-06-10T00:55:20.375776+0000","last_became_active":"2024-06-10T00:52:35.966366+0000","last_became_peered":"2024-06-10T00:52:35.966366+0000","last_unstale":"2024-06-10T00:55:20.375776+0000","last_undegraded":"2024-06-10T00:55:20.375776+0000","last_fullsized":"2024-06-10T00:55:20.375776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T08:27:06.047113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"100'12","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.369850+0000","last_change":"2024-06-10T00:52:35.960517+0000","last_active":"2024-06-10T00:55:20.369850+0000","last_peered":"2024-06-10T00:55:20.369850+0000","last_clean":"2024-06-10T00:55:20.369850+0000","last_became_active":"2024-06-10T00:52:35.957895+0000","last_became_peered":"2024-06-10T00:52:35.957895+0000","last_unstale":"2024-06-10T00:55:20.369850+0000","last_undegraded":"2024-06-10T00:55:20.369850+0000","last_fullsized":"2024-06-10T00:55:20.369850+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:04:34.438070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"100'8","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374305+0000","last_change":"2024-06-10T00:52:35.962163+0000","last_active":"2024-06-10T00:55:20.374305+0000","last_peered":"2024-06-10T00:55:20.374305+0000","last_clean":"2024-06-10T00:55:20.374305+0000","last_became_active":"2024-06-10T00:52:35.961768+0000","last_became_peered":"2024-06-10T00:52:35.961768+0000","last_unstale":"2024-06-10T00:55:20.374305+0000","last_undegraded":"2024-06-10T00:55:20.374305+0000","last_fullsized":"2024-06-10T00:55:20.374305+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:32:35.793995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"100'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374044+0000","last_change":"2024-06-10T00:53:22.130686+0000","last_active":"2024-06-10T00:55:20.374044+0000","last_peered":"2024-06-10T00:55:20.374044+0000","last_clean":"2024-06-10T00:55:20.374044+0000","last_became_active":"2024-06-10T00:53:22.130198+0000","last_became_peered":"2024-06-10T00:53:22.130198+0000","last_unstale":"2024-06-10T00:55:20.374044+0000","last_undegraded":"2024-06-10T00:55:20.374044+0000","last_fullsized":"2024-06-10T00:55:20.374044+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T01:50:26.773977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372642+0000","last_change":"2024-06-10T00:52:35.957255+0000","last_active":"2024-06-10T00:55:20.372642+0000","last_peered":"2024-06-10T00:55:20.372642+0000","last_clean":"2024-06-10T00:55:20.372642+0000","last_became_active":"2024-06-10T00:52:35.956923+0000","last_became_peered":"2024-06-10T00:52:35.956923+0000","last_unstale":"2024-06-10T00:55:20.372642+0000","last_undegraded":"2024-06-10T00:55:20.372642+0000","last_fullsized":"2024-06-10T00:55:20.372642+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T03:33:25.743503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.375063+0000","last_change":"2024-06-10T00:52:35.964258+0000","last_active":"2024-06-10T00:55:20.375063+0000","last_peered":"2024-06-10T00:55:20.375063+0000","last_clean":"2024-06-10T00:55:20.375063+0000","last_became_active":"2024-06-10T00:52:35.963862+0000","last_became_peered":"2024-06-10T00:52:35.963862+0000","last_unstale":"2024-06-10T00:55:20.375063+0000","last_undegraded":"2024-06-10T00:55:20.375063+0000","last_fullsized":"2024-06-10T00:55:20.375063+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T12:48:58.554411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.377085+0000","last_change":"2024-06-10T00:52:35.965046+0000","last_active":"2024-06-10T00:55:20.377085+0000","last_peered":"2024-06-10T00:55:20.377085+0000","last_clean":"2024-06-10T00:55:20.377085+0000","last_became_active":"2024-06-10T00:52:35.964496+0000","last_became_peered":"2024-06-10T00:52:35.964496+0000","last_unstale":"2024-06-10T00:55:20.377085+0000","last_undegraded":"2024-06-10T00:55:20.377085+0000","last_fullsized":"2024-06-10T00:55:20.377085+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:09:42.979731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"100'2","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.370981+0000","last_change":"2024-06-10T00:52:35.964287+0000","last_active":"2024-06-10T00:55:20.370981+0000","last_peered":"2024-06-10T00:55:20.370981+0000","last_clean":"2024-06-10T00:55:20.370981+0000","last_became_active":"2024-06-10T00:52:35.963920+0000","last_became_peered":"2024-06-10T00:52:35.963920+0000","last_unstale":"2024-06-10T00:55:20.370981+0000","last_undegraded":"2024-06-10T00:55:20.370981+0000","last_fullsized":"2024-06-10T00:55:20.370981+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:41:54.593274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.367606+0000","last_change":"2024-06-10T00:52:35.961440+0000","last_active":"2024-06-10T00:55:20.367606+0000","last_peered":"2024-06-10T00:55:20.367606+0000","last_clean":"2024-06-10T00:55:20.367606+0000","last_became_active":"2024-06-10T00:52:35.960066+0000","last_became_peered":"2024-06-10T00:52:35.960066+0000","last_unstale":"2024-06-10T00:55:20.367606+0000","last_undegraded":"2024-06-10T00:55:20.367606+0000","last_fullsized":"2024-06-10T00:55:20.367606+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:58:01.024596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372367+0000","last_change":"2024-06-10T00:52:35.964464+0000","last_active":"2024-06-10T00:55:20.372367+0000","last_peered":"2024-06-10T00:55:20.372367+0000","last_clean":"2024-06-10T00:55:20.372367+0000","last_became_active":"2024-06-10T00:52:35.964159+0000","last_became_peered":"2024-06-10T00:52:35.964159+0000","last_unstale":"2024-06-10T00:55:20.372367+0000","last_undegraded":"2024-06-10T00:55:20.372367+0000","last_fullsized":"2024-06-10T00:55:20.372367+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T10:56:18.660859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.377213+0000","last_change":"2024-06-10T00:52:35.961539+0000","last_active":"2024-06-10T00:55:20.377213+0000","last_peered":"2024-06-10T00:55:20.377213+0000","last_clean":"2024-06-10T00:55:20.377213+0000","last_became_active":"2024-06-10T00:52:35.958738+0000","last_became_peered":"2024-06-10T00:52:35.958738+0000","last_unstale":"2024-06-10T00:55:20.377213+0000","last_undegraded":"2024-06-10T00:55:20.377213+0000","last_fullsized":"2024-06-10T00:55:20.377213+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T02:06:48.057551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.374134+0000","last_change":"2024-06-10T00:52:35.962563+0000","last_active":"2024-06-10T00:55:20.374134+0000","last_peered":"2024-06-10T00:55:20.374134+0000","last_clean":"2024-06-10T00:55:20.374134+0000","last_became_active":"2024-06-10T00:52:35.962255+0000","last_became_peered":"2024-06-10T00:52:35.962255+0000","last_unstale":"2024-06-10T00:55:20.374134+0000","last_undegraded":"2024-06-10T00:55:20.374134+0000","last_fullsized":"2024-06-10T00:55:20.374134+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T09:33:35.689446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-10T00:55:20.372437+0000","last_change":"2024-06-10T00:52:35.959616+0000","last_active":"2024-06-10T00:55:20.372437+0000","last_peered":"2024-06-10T00:55:20.372437+0000","last_clean":"2024-06-10T00:55:20.372437+0000","last_became_active":"2024-06-10T00:52:35.959251+0000","last_became_peered":"2024-06-10T00:52:35.959251+0000","last_unstale":"2024-06-10T00:55:20.372437+0000","last_undegraded":"2024-06-10T00:55:20.372437+0000","last_fullsized":"2024-06-10T00:55:20.372437+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-10T00:52:34.935656+0000","last_clean_scrub_stamp":"2024-06-10T00:52:34.935656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-11T11:39:23.933279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284806,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":82322,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1182,"num_read_kb":680,"num_write":191,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":246966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":163,"ondisk_log_size":163,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351044,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":573101,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612681,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28248,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716904,"statfs":{"total":95995035648,"available":95966109696,"internally_reserved":0,"allocated":1261568,"data_stored":1238242,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776206,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":577433,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037843,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":573257,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299481,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36476,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708676,"statfs":{"total":95995035648,"available":95957684224,"internally_reserved":0,"allocated":1167360,"data_stored":1169614,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463005,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":568513,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36488,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708664,"statfs":{"total":95995035648,"available":95957671936,"internally_reserved":0,"allocated":1179648,"data_stored":646296,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738491,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":569901,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411442,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32344,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712808,"statfs":{"total":95995035648,"available":95961915392,"internally_reserved":0,"allocated":1196032,"data_stored":648373,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149804,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":568707,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888167,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":573275,"data_compressed":32415,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724642,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":1165202,"data_compressed":68630,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34577,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36215,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":77873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":79950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":79451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-10T00:56:57.048 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:56:57.337 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:56:58.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:58 smithi167 ceph-mon[36917]: from='client.25009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:56:58.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:56:58 smithi167 ceph-mon[36917]: pgmap v438: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-06-10T00:56:58.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:58 smithi094 ceph-mon[36708]: from='client.25009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:56:58.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:56:58 smithi094 ceph-mon[36708]: pgmap v438: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-06-10T00:56:58.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:58 smithi037 ceph-mon[32192]: from='client.25009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-10T00:56:58.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:56:58 smithi037 ceph-mon[32192]: pgmap v438: 129 pgs: 129 active+clean; 657 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-06-10T00:56:59.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.008+0000 7f432c0b2700 1 -- 172.21.15.37:0/2147119766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43241066b0 msgr2=0x7f432410e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:59.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.008+0000 7f432c0b2700 1 --2- 172.21.15.37:0/2147119766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43241066b0 0x7f432410e280 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f431c00b0d0 tx=0x7f431c00b3e0 comp rx=0 tx=0).stop 2024-06-10T00:56:59.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.008+0000 7f432c0b2700 1 -- 172.21.15.37:0/2147119766 shutdown_connections 2024-06-10T00:56:59.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.008+0000 7f432c0b2700 1 --2- 172.21.15.37:0/2147119766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43241066b0 0x7f432410e280 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:59.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.008+0000 7f432c0b2700 1 --2- 172.21.15.37:0/2147119766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4324101620 0x7f4324101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:59.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.008+0000 7f432c0b2700 1 --2- 172.21.15.37:0/2147119766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4324101fd0 0x7f4324105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:59.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.008+0000 7f432c0b2700 1 -- 172.21.15.37:0/2147119766 >> 172.21.15.37:0/2147119766 conn(0x7f4324078340 msgr2=0x7f4324078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:59.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.008+0000 7f432c0b2700 1 -- 172.21.15.37:0/2147119766 shutdown_connections 2024-06-10T00:56:59.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.008+0000 7f432c0b2700 1 -- 172.21.15.37:0/2147119766 wait complete. 2024-06-10T00:56:59.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.009+0000 7f432c0b2700 1 Processor -- start 2024-06-10T00:56:59.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.009+0000 7f432c0b2700 1 -- start start 2024-06-10T00:56:59.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.009+0000 7f432c0b2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4324101620 0x7f432419e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:59.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.009+0000 7f432c0b2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4324101fd0 0x7f432419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:59.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.009+0000 7f432c0b2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43241066b0 0x7f4324198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:59.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.009+0000 7f432c0b2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f43241116c0 con 0x7f4324101fd0 2024-06-10T00:56:59.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.009+0000 7f432c0b2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4324111540 con 0x7f43241066b0 2024-06-10T00:56:59.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.009+0000 7f432c0b2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4324111840 con 0x7f4324101620 2024-06-10T00:56:59.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.009+0000 7f432964d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4324101fd0 0x7f432419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:59.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.009+0000 7f432964d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4324101fd0 0x7f432419ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39330/0 (socket says 172.21.15.37:39330) 2024-06-10T00:56:59.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.009+0000 7f432964d700 1 -- 172.21.15.37:0/2748433 learned_addr learned my addr 172.21.15.37:0/2748433 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:56:59.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.009+0000 7f4329e4e700 1 --2- 172.21.15.37:0/2748433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4324101620 0x7f432419e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:59.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.009+0000 7f432a64f700 1 --2- 172.21.15.37:0/2748433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43241066b0 0x7f4324198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:59.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.009+0000 7f432964d700 1 -- 172.21.15.37:0/2748433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4324101620 msgr2=0x7f432419e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:59.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.009+0000 7f432964d700 1 --2- 172.21.15.37:0/2748433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4324101620 0x7f432419e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:59.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.009+0000 7f432964d700 1 -- 172.21.15.37:0/2748433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43241066b0 msgr2=0x7f4324198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:59.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.009+0000 7f432964d700 1 --2- 172.21.15.37:0/2748433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43241066b0 0x7f4324198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:59.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.009+0000 7f432964d700 1 -- 172.21.15.37:0/2748433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4324199260 con 0x7f4324101fd0 2024-06-10T00:56:59.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.010+0000 7f432a64f700 1 --2- 172.21.15.37:0/2748433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43241066b0 0x7f4324198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:56:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.010+0000 7f432964d700 1 --2- 172.21.15.37:0/2748433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4324101fd0 0x7f432419ee30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f431400d980 tx=0x7f431400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.010+0000 7f431affd700 1 -- 172.21.15.37:0/2748433 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314017070 con 0x7f4324101fd0 2024-06-10T00:56:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.010+0000 7f431affd700 1 -- 172.21.15.37:0/2748433 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4314009e10 con 0x7f4324101fd0 2024-06-10T00:56:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.010+0000 7f432c0b2700 1 -- 172.21.15.37:0/2748433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4324199550 con 0x7f4324101fd0 2024-06-10T00:56:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.010+0000 7f431affd700 1 -- 172.21.15.37:0/2748433 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431400f620 con 0x7f4324101fd0 2024-06-10T00:56:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.010+0000 7f432c0b2700 1 -- 172.21.15.37:0/2748433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43241a57c0 con 0x7f4324101fd0 2024-06-10T00:56:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.011+0000 7f432c0b2700 1 -- 172.21.15.37:0/2748433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4324069180 con 0x7f4324101fd0 2024-06-10T00:56:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.012+0000 7f431affd700 1 -- 172.21.15.37:0/2748433 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f4314010460 con 0x7f4324101fd0 2024-06-10T00:56:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.013+0000 7f431affd700 1 --2- 172.21.15.37:0/2748433 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f43100a2cb0 0x7f43100a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:56:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.013+0000 7f4329e4e700 1 --2- 172.21.15.37:0/2748433 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f43100a2cb0 0x7f43100a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:56:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.013+0000 7f431affd700 1 -- 172.21.15.37:0/2748433 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f43140cf040 con 0x7f4324101fd0 2024-06-10T00:56:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.013+0000 7f4329e4e700 1 --2- 172.21.15.37:0/2748433 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f43100a2cb0 0x7f43100a5170 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4324078aa0 tx=0x7f432000b3c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:56:59.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.016+0000 7f431affd700 1 -- 172.21.15.37:0/2748433 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f431409ba50 con 0x7f4324101fd0 2024-06-10T00:56:59.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.273+0000 7f432c0b2700 1 -- 172.21.15.37:0/2748433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43241a59f0 con 0x7f4324101fd0 2024-06-10T00:56:59.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.274+0000 7f431affd700 1 -- 172.21.15.37:0/2748433 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11867 (secure 0 0 0) 0x7f431409b350 con 0x7f4324101fd0 2024-06-10T00:56:59.274 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:56:59.274 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T00:56:30.291171+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24451,24458,24472]},"id":1}]} 2024-06-10T00:56:59.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.276+0000 7f432c0b2700 1 -- 172.21.15.37:0/2748433 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f43100a2cb0 msgr2=0x7f43100a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:59.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.276+0000 7f432c0b2700 1 --2- 172.21.15.37:0/2748433 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f43100a2cb0 0x7f43100a5170 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4324078aa0 tx=0x7f432000b3c0 comp rx=0 tx=0).stop 2024-06-10T00:56:59.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.276+0000 7f432c0b2700 1 -- 172.21.15.37:0/2748433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4324101fd0 msgr2=0x7f432419ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:56:59.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.276+0000 7f432c0b2700 1 --2- 172.21.15.37:0/2748433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4324101fd0 0x7f432419ee30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f431400d980 tx=0x7f431400dc90 comp rx=0 tx=0).stop 2024-06-10T00:56:59.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.276+0000 7f432c0b2700 1 -- 172.21.15.37:0/2748433 shutdown_connections 2024-06-10T00:56:59.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.276+0000 7f432c0b2700 1 --2- 172.21.15.37:0/2748433 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f43100a2cb0 0x7f43100a5170 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:59.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.276+0000 7f432c0b2700 1 --2- 172.21.15.37:0/2748433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43241066b0 0x7f4324198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:59.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.276+0000 7f432c0b2700 1 --2- 172.21.15.37:0/2748433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4324101620 0x7f432419e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:59.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.276+0000 7f432c0b2700 1 --2- 172.21.15.37:0/2748433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4324101fd0 0x7f432419ee30 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:56:59.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.276+0000 7f432c0b2700 1 -- 172.21.15.37:0/2748433 >> 172.21.15.37:0/2748433 conn(0x7f4324078340 msgr2=0x7f4324105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:56:59.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.276+0000 7f432c0b2700 1 -- 172.21.15.37:0/2748433 shutdown_connections 2024-06-10T00:56:59.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:56:59.276+0000 7f432c0b2700 1 -- 172.21.15.37:0/2748433 wait complete. 2024-06-10T00:56:59.277 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-06-10T00:57:00.082 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:57:00.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:00 smithi167 ceph-mon[36917]: pgmap v439: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 8 op/s 2024-06-10T00:57:00.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:00 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2748433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:57:00.340 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:57:00.359 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:00 smithi037 ceph-mon[32192]: pgmap v439: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 8 op/s 2024-06-10T00:57:00.359 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:00 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2748433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:57:00.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:00 smithi094 ceph-mon[36708]: pgmap v439: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 8 op/s 2024-06-10T00:57:00.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:00 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2748433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:57:01.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:01 smithi167 ceph-mon[36917]: scrub summary: idle 2024-06-10T00:57:01.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:01 smithi094 ceph-mon[36708]: scrub summary: idle 2024-06-10T00:57:01.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:01 smithi037 ceph-mon[32192]: scrub summary: idle 2024-06-10T00:57:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.145+0000 7f9c44bc2700 1 -- 172.21.15.37:0/84178136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c40102740 msgr2=0x7f9c40102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.145+0000 7f9c44bc2700 1 --2- 172.21.15.37:0/84178136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c40102740 0x7f9c40102b20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9c300099c0 tx=0x7f9c30009cd0 comp rx=0 tx=0).stop 2024-06-10T00:57:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.145+0000 7f9c44bc2700 1 -- 172.21.15.37:0/84178136 shutdown_connections 2024-06-10T00:57:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.145+0000 7f9c44bc2700 1 --2- 172.21.15.37:0/84178136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c40107840 0x7f9c4010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.145+0000 7f9c44bc2700 1 --2- 172.21.15.37:0/84178136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c40102740 0x7f9c40102b20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.145+0000 7f9c44bc2700 1 --2- 172.21.15.37:0/84178136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c401030f0 0x7f9c40107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.145+0000 7f9c44bc2700 1 -- 172.21.15.37:0/84178136 >> 172.21.15.37:0/84178136 conn(0x7f9c400795f0 msgr2=0x7f9c40079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.145+0000 7f9c44bc2700 1 -- 172.21.15.37:0/84178136 shutdown_connections 2024-06-10T00:57:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.145+0000 7f9c44bc2700 1 -- 172.21.15.37:0/84178136 wait complete. 2024-06-10T00:57:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.146+0000 7f9c44bc2700 1 Processor -- start 2024-06-10T00:57:02.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.146+0000 7f9c44bc2700 1 -- start start 2024-06-10T00:57:02.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.147+0000 7f9c44bc2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c40102740 0x7f9c4019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:02.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.147+0000 7f9c44bc2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c401030f0 0x7f9c4019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:02.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.147+0000 7f9c44bc2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c40107840 0x7f9c4019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:02.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.147+0000 7f9c44bc2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9c40112780 con 0x7f9c40102740 2024-06-10T00:57:02.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.147+0000 7f9c44bc2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9c40112600 con 0x7f9c401030f0 2024-06-10T00:57:02.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.147+0000 7f9c44bc2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9c40112900 con 0x7f9c40107840 2024-06-10T00:57:02.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.147+0000 7f9c3e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c40102740 0x7f9c4019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:02.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.147+0000 7f9c3e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c40102740 0x7f9c4019ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51316/0 (socket says 172.21.15.37:51316) 2024-06-10T00:57:02.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.147+0000 7f9c3e59c700 1 -- 172.21.15.37:0/1757612307 learned_addr learned my addr 172.21.15.37:0/1757612307 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:02.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.147+0000 7f9c35d9b700 1 --2- 172.21.15.37:0/1757612307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c401030f0 0x7f9c4019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:02.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.147+0000 7f9c3ed9d700 1 --2- 172.21.15.37:0/1757612307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c40107840 0x7f9c4019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:02.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.147+0000 7f9c3e59c700 1 -- 172.21.15.37:0/1757612307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c40107840 msgr2=0x7f9c4019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:02.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.147+0000 7f9c3e59c700 1 --2- 172.21.15.37:0/1757612307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c40107840 0x7f9c4019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:02.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.147+0000 7f9c3e59c700 1 -- 172.21.15.37:0/1757612307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c401030f0 msgr2=0x7f9c4019b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:02.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.147+0000 7f9c3e59c700 1 --2- 172.21.15.37:0/1757612307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c401030f0 0x7f9c4019b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:02.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.147+0000 7f9c3e59c700 1 -- 172.21.15.37:0/1757612307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4019fe10 con 0x7f9c40102740 2024-06-10T00:57:02.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.147+0000 7f9c3e59c700 1 --2- 172.21.15.37:0/1757612307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c40102740 0x7f9c4019ae00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9c30006010 tx=0x7f9c30009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:02.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.148+0000 7f9c37fff700 1 -- 172.21.15.37:0/1757612307 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c30026070 con 0x7f9c40102740 2024-06-10T00:57:02.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.148+0000 7f9c37fff700 1 -- 172.21.15.37:0/1757612307 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c30004cd0 con 0x7f9c40102740 2024-06-10T00:57:02.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.148+0000 7f9c37fff700 1 -- 172.21.15.37:0/1757612307 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c30003ba0 con 0x7f9c40102740 2024-06-10T00:57:02.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.148+0000 7f9c44bc2700 1 -- 172.21.15.37:0/1757612307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c401a00a0 con 0x7f9c40102740 2024-06-10T00:57:02.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.148+0000 7f9c44bc2700 1 -- 172.21.15.37:0/1757612307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c401a7980 con 0x7f9c40102740 2024-06-10T00:57:02.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.149+0000 7f9c44bc2700 1 -- 172.21.15.37:0/1757612307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c40069180 con 0x7f9c40102740 2024-06-10T00:57:02.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.150+0000 7f9c37fff700 1 -- 172.21.15.37:0/1757612307 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f9c30029070 con 0x7f9c40102740 2024-06-10T00:57:02.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.151+0000 7f9c37fff700 1 --2- 172.21.15.37:0/1757612307 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9c2c0a2d30 0x7f9c2c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:02.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.151+0000 7f9c35d9b700 1 --2- 172.21.15.37:0/1757612307 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9c2c0a2d30 0x7f9c2c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:02.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.151+0000 7f9c37fff700 1 -- 172.21.15.37:0/1757612307 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f9c300d0240 con 0x7f9c40102740 2024-06-10T00:57:02.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.151+0000 7f9c35d9b700 1 --2- 172.21.15.37:0/1757612307 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9c2c0a2d30 0x7f9c2c0a51f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9c4010f770 tx=0x7f9c2800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:02.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.154+0000 7f9c37fff700 1 -- 172.21.15.37:0/1757612307 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9c30030020 con 0x7f9c40102740 2024-06-10T00:57:02.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:02 smithi167 ceph-mon[36917]: pgmap v440: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 7 op/s 2024-06-10T00:57:02.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:02 smithi094 ceph-mon[36708]: pgmap v440: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 7 op/s 2024-06-10T00:57:02.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:02 smithi037 ceph-mon[32192]: pgmap v440: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 7 op/s 2024-06-10T00:57:02.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.410+0000 7f9c44bc2700 1 -- 172.21.15.37:0/1757612307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c4019c160 con 0x7f9c40102740 2024-06-10T00:57:02.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.411+0000 7f9c37fff700 1 -- 172.21.15.37:0/1757612307 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11867 (secure 0 0 0) 0x7f9c30013070 con 0x7f9c40102740 2024-06-10T00:57:02.412 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:57:02.412 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T00:56:30.291171+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24451,24458,24472]},"id":1}]} 2024-06-10T00:57:02.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.413+0000 7f9c44bc2700 1 -- 172.21.15.37:0/1757612307 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9c2c0a2d30 msgr2=0x7f9c2c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:02.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.413+0000 7f9c44bc2700 1 --2- 172.21.15.37:0/1757612307 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9c2c0a2d30 0x7f9c2c0a51f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9c4010f770 tx=0x7f9c2800b410 comp rx=0 tx=0).stop 2024-06-10T00:57:02.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.413+0000 7f9c44bc2700 1 -- 172.21.15.37:0/1757612307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c40102740 msgr2=0x7f9c4019ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:02.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.413+0000 7f9c44bc2700 1 --2- 172.21.15.37:0/1757612307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c40102740 0x7f9c4019ae00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9c30006010 tx=0x7f9c30009f80 comp rx=0 tx=0).stop 2024-06-10T00:57:02.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.413+0000 7f9c44bc2700 1 -- 172.21.15.37:0/1757612307 shutdown_connections 2024-06-10T00:57:02.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.413+0000 7f9c44bc2700 1 --2- 172.21.15.37:0/1757612307 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9c2c0a2d30 0x7f9c2c0a51f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:02.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.413+0000 7f9c44bc2700 1 --2- 172.21.15.37:0/1757612307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c40107840 0x7f9c4019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:02.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.413+0000 7f9c44bc2700 1 --2- 172.21.15.37:0/1757612307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c40102740 0x7f9c4019ae00 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:02.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.413+0000 7f9c44bc2700 1 --2- 172.21.15.37:0/1757612307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c401030f0 0x7f9c4019b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:02.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.413+0000 7f9c44bc2700 1 -- 172.21.15.37:0/1757612307 >> 172.21.15.37:0/1757612307 conn(0x7f9c400795f0 msgr2=0x7f9c40100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:02.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.414+0000 7f9c44bc2700 1 -- 172.21.15.37:0/1757612307 shutdown_connections 2024-06-10T00:57:02.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:02.414+0000 7f9c44bc2700 1 -- 172.21.15.37:0/1757612307 wait complete. 2024-06-10T00:57:02.415 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-06-10T00:57:03.092 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T00:57:03.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:03 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1757612307' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:57:03.343 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:57:03.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:03 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1757612307' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:57:03.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:03 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1757612307' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:57:04.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:04 smithi167 ceph-mon[36917]: pgmap v441: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-06-10T00:57:04.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:04 smithi094 ceph-mon[36708]: pgmap v441: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-06-10T00:57:04.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:04 smithi037 ceph-mon[32192]: pgmap v441: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-06-10T00:57:05.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.142+0000 7f88d9639700 1 -- 172.21.15.37:0/1508279048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d4101650 msgr2=0x7f88d4101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:05.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.142+0000 7f88d9639700 1 --2- 172.21.15.37:0/1508279048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d4101650 0x7f88d4101a30 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f88c40099c0 tx=0x7f88c4009cd0 comp rx=0 tx=0).stop 2024-06-10T00:57:05.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.142+0000 7f88d9639700 1 -- 172.21.15.37:0/1508279048 shutdown_connections 2024-06-10T00:57:05.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.142+0000 7f88d9639700 1 --2- 172.21.15.37:0/1508279048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88d41066a0 0x7f88d410e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:05.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.142+0000 7f88d9639700 1 --2- 172.21.15.37:0/1508279048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d4101650 0x7f88d4101a30 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:05.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.142+0000 7f88d9639700 1 --2- 172.21.15.37:0/1508279048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d4102000 0x7f88d4105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:05.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.142+0000 7f88d9639700 1 -- 172.21.15.37:0/1508279048 >> 172.21.15.37:0/1508279048 conn(0x7f88d40773e0 msgr2=0x7f88d40777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:05.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.142+0000 7f88d9639700 1 -- 172.21.15.37:0/1508279048 shutdown_connections 2024-06-10T00:57:05.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.142+0000 7f88d9639700 1 -- 172.21.15.37:0/1508279048 wait complete. 2024-06-10T00:57:05.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.143+0000 7f88d9639700 1 Processor -- start 2024-06-10T00:57:05.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.143+0000 7f88d9639700 1 -- start start 2024-06-10T00:57:05.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.143+0000 7f88d9639700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d4101650 0x7f88d410ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:05.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.143+0000 7f88d9639700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d4102000 0x7f88d410ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:05.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.143+0000 7f88d9639700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88d41066a0 0x7f88d410f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:05.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.143+0000 7f88d9639700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f88d4112840 con 0x7f88d4102000 2024-06-10T00:57:05.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.143+0000 7f88d9639700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f88d41126c0 con 0x7f88d4101650 2024-06-10T00:57:05.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.143+0000 7f88d9639700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f88d41129c0 con 0x7f88d41066a0 2024-06-10T00:57:05.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.144+0000 7f88d27fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d4102000 0x7f88d410ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:05.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.144+0000 7f88d27fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d4102000 0x7f88d410ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51332/0 (socket says 172.21.15.37:51332) 2024-06-10T00:57:05.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.144+0000 7f88d2ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d4101650 0x7f88d410ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:05.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.144+0000 7f88d27fc700 1 -- 172.21.15.37:0/1529866162 learned_addr learned my addr 172.21.15.37:0/1529866162 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:05.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.144+0000 7f88d37fe700 1 --2- 172.21.15.37:0/1529866162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88d41066a0 0x7f88d410f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:05.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.144+0000 7f88d27fc700 1 -- 172.21.15.37:0/1529866162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88d41066a0 msgr2=0x7f88d410f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:05.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.144+0000 7f88d27fc700 1 --2- 172.21.15.37:0/1529866162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88d41066a0 0x7f88d410f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:05.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.144+0000 7f88d27fc700 1 -- 172.21.15.37:0/1529866162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d4101650 msgr2=0x7f88d410ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:05.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.144+0000 7f88d27fc700 1 --2- 172.21.15.37:0/1529866162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d4101650 0x7f88d410ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:05.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.144+0000 7f88d27fc700 1 -- 172.21.15.37:0/1529866162 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d41a7170 con 0x7f88d4102000 2024-06-10T00:57:05.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.144+0000 7f88d27fc700 1 --2- 172.21.15.37:0/1529866162 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d4102000 0x7f88d410ef90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f88c800c8f0 tx=0x7f88c800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:05.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.144+0000 7f88bbfff700 1 -- 172.21.15.37:0/1529866162 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c80078d0 con 0x7f88d4102000 2024-06-10T00:57:05.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.144+0000 7f88bbfff700 1 -- 172.21.15.37:0/1529866162 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88c800f450 con 0x7f88d4102000 2024-06-10T00:57:05.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.144+0000 7f88d9639700 1 -- 172.21.15.37:0/1529866162 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d41a7460 con 0x7f88d4102000 2024-06-10T00:57:05.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.144+0000 7f88bbfff700 1 -- 172.21.15.37:0/1529866162 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c8018640 con 0x7f88d4102000 2024-06-10T00:57:05.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.144+0000 7f88d9639700 1 -- 172.21.15.37:0/1529866162 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d41a79a0 con 0x7f88d4102000 2024-06-10T00:57:05.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.145+0000 7f88d9639700 1 -- 172.21.15.37:0/1529866162 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88d4069180 con 0x7f88d4102000 2024-06-10T00:57:05.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.146+0000 7f88bbfff700 1 -- 172.21.15.37:0/1529866162 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f88c800f600 con 0x7f88d4102000 2024-06-10T00:57:05.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.147+0000 7f88bbfff700 1 --2- 172.21.15.37:0/1529866162 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f88bc0a2cb0 0x7f88bc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:05.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.148+0000 7f88d2ffd700 1 --2- 172.21.15.37:0/1529866162 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f88bc0a2cb0 0x7f88bc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:05.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.148+0000 7f88bbfff700 1 -- 172.21.15.37:0/1529866162 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f88c80cf3e0 con 0x7f88d4102000 2024-06-10T00:57:05.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.148+0000 7f88d2ffd700 1 --2- 172.21.15.37:0/1529866162 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f88bc0a2cb0 0x7f88bc0a5170 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f88c4006010 tx=0x7f88c40058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:05.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.152+0000 7f88bbfff700 1 -- 172.21.15.37:0/1529866162 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f88c8014030 con 0x7f88d4102000 2024-06-10T00:57:05.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.359+0000 7f88d9639700 1 -- 172.21.15.37:0/1529866162 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88d402de60 con 0x7f88d4102000 2024-06-10T00:57:05.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.361+0000 7f88bbfff700 1 -- 172.21.15.37:0/1529866162 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24854 (secure 0 0 0) 0x7f88c809bdf0 con 0x7f88d4102000 2024-06-10T00:57:05.363 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:57:05.363 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":100,"fsid":"68793514-26c2-11ef-bc9b-c7b262605968","created":"2024-06-10T00:44:24.106713+0000","modified":"2024-06-10T00:54:22.546967+0000","last_up_change":"2024-06-10T00:51:31.562156+0000","last_in_change":"2024-06-10T00:51:11.294301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:47:44.773911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:52:31.684780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:52:34.885503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:52:44.415437+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1df6046a-6801-4950-9193-54303fed169b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6803","nonce":3594704094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6805","nonce":3594704094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6809","nonce":3594704094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6807","nonce":3594704094}]},"public_addr":"172.21.15.37:6803/3594704094","cluster_addr":"172.21.15.37:6805/3594704094","heartbeat_back_addr":"172.21.15.37:6809/3594704094","heartbeat_front_addr":"172.21.15.37:6807/3594704094","state":["exists","up"]},{"osd":1,"uuid":"4e1c8944-94e4-4e46-9efe-56229fe6df1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6801","nonce":3427727155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6803","nonce":3427727155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6807","nonce":3427727155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6805","nonce":3427727155}]},"public_addr":"172.21.15.94:6801/3427727155","cluster_addr":"172.21.15.94:6803/3427727155","heartbeat_back_addr":"172.21.15.94:6807/3427727155","heartbeat_front_addr":"172.21.15.94:6805/3427727155","state":["exists","up"]},{"osd":2,"uuid":"fd2fcedc-4c88-478f-8025-13e5944a27ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6801","nonce":4002498292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6803","nonce":4002498292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6807","nonce":4002498292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6805","nonce":4002498292}]},"public_addr":"172.21.15.167:6801/4002498292","cluster_addr":"172.21.15.167:6803/4002498292","heartbeat_back_addr":"172.21.15.167:6807/4002498292","heartbeat_front_addr":"172.21.15.167:6805/4002498292","state":["exists","up"]},{"osd":3,"uuid":"d0153676-818c-4dcc-83b1-287435a0c8a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6811","nonce":3425469324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6813","nonce":3425469324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6817","nonce":3425469324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6815","nonce":3425469324}]},"public_addr":"172.21.15.37:6811/3425469324","cluster_addr":"172.21.15.37:6813/3425469324","heartbeat_back_addr":"172.21.15.37:6817/3425469324","heartbeat_front_addr":"172.21.15.37:6815/3425469324","state":["exists","up"]},{"osd":4,"uuid":"172c333c-00c0-4e94-8870-acb3b32b2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6809","nonce":1740627431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6811","nonce":1740627431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6815","nonce":1740627431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6813","nonce":1740627431}]},"public_addr":"172.21.15.94:6809/1740627431","cluster_addr":"172.21.15.94:6811/1740627431","heartbeat_back_addr":"172.21.15.94:6815/1740627431","heartbeat_front_addr":"172.21.15.94:6813/1740627431","state":["exists","up"]},{"osd":5,"uuid":"dbce656e-5f93-4bda-baf2-fc3cc03d8282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6809","nonce":221805236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6811","nonce":221805236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6815","nonce":221805236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6813","nonce":221805236}]},"public_addr":"172.21.15.167:6809/221805236","cluster_addr":"172.21.15.167:6811/221805236","heartbeat_back_addr":"172.21.15.167:6815/221805236","heartbeat_front_addr":"172.21.15.167:6813/221805236","state":["exists","up"]},{"osd":6,"uuid":"fcbda266-6d60-43b2-a51c-4e562476d1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6819","nonce":472670875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6821","nonce":472670875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6825","nonce":472670875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6823","nonce":472670875}]},"public_addr":"172.21.15.37:6819/472670875","cluster_addr":"172.21.15.37:6821/472670875","heartbeat_back_addr":"172.21.15.37:6825/472670875","heartbeat_front_addr":"172.21.15.37:6823/472670875","state":["exists","up"]},{"osd":7,"uuid":"ed829ed3-ba85-4e82-ac37-e44e9534d37c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6817","nonce":3932281552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6819","nonce":3932281552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6823","nonce":3932281552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6821","nonce":3932281552}]},"public_addr":"172.21.15.94:6817/3932281552","cluster_addr":"172.21.15.94:6819/3932281552","heartbeat_back_addr":"172.21.15.94:6823/3932281552","heartbeat_front_addr":"172.21.15.94:6821/3932281552","state":["exists","up"]},{"osd":8,"uuid":"7d6267d8-72f4-4704-b554-bafc52d11a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6817","nonce":2290216047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6819","nonce":2290216047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6823","nonce":2290216047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6821","nonce":2290216047}]},"public_addr":"172.21.15.167:6817/2290216047","cluster_addr":"172.21.15.167:6819/2290216047","heartbeat_back_addr":"172.21.15.167:6823/2290216047","heartbeat_front_addr":"172.21.15.167:6821/2290216047","state":["exists","up"]},{"osd":9,"uuid":"210fdc20-0a36-4eeb-bdf2-c0a7f130e19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6827","nonce":2063666428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6829","nonce":2063666428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6833","nonce":2063666428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6831","nonce":2063666428}]},"public_addr":"172.21.15.37:6827/2063666428","cluster_addr":"172.21.15.37:6829/2063666428","heartbeat_back_addr":"172.21.15.37:6833/2063666428","heartbeat_front_addr":"172.21.15.37:6831/2063666428","state":["exists","up"]},{"osd":10,"uuid":"e200b371-b3cf-4b92-a659-f155ca86f0a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6825","nonce":1706618899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6827","nonce":1706618899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6831","nonce":1706618899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6829","nonce":1706618899}]},"public_addr":"172.21.15.94:6825/1706618899","cluster_addr":"172.21.15.94:6827/1706618899","heartbeat_back_addr":"172.21.15.94:6831/1706618899","heartbeat_front_addr":"172.21.15.94:6829/1706618899","state":["exists","up"]},{"osd":11,"uuid":"cfa427df-37a5-4d28-ac2e-af777d607008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6825","nonce":1349892516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6827","nonce":1349892516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6831","nonce":1349892516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6829","nonce":1349892516}]},"public_addr":"172.21.15.167:6825/1349892516","cluster_addr":"172.21.15.167:6827/1349892516","heartbeat_back_addr":"172.21.15.167:6831/1349892516","heartbeat_front_addr":"172.21.15.167:6829/1349892516","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:46:51.846147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:15.763723+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:40.106800+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:05.923204+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:30.501883+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:55.565575+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:20.411360+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:46.071776+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:11.854599+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:37.608818+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:03.176767+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:29.768705+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/4084722397":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/3571803585":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/416587443":"2024-06-11T00:45:20.254342+0000","172.21.15.37:6801/2766692263":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/2026344134":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/2781321886":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6801/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/1110143997":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2766692263":"2024-06-11T00:45:20.254342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T00:57:05.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.363+0000 7f88d9639700 1 -- 172.21.15.37:0/1529866162 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f88bc0a2cb0 msgr2=0x7f88bc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:05.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.363+0000 7f88d9639700 1 --2- 172.21.15.37:0/1529866162 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f88bc0a2cb0 0x7f88bc0a5170 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f88c4006010 tx=0x7f88c40058e0 comp rx=0 tx=0).stop 2024-06-10T00:57:05.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.363+0000 7f88d9639700 1 -- 172.21.15.37:0/1529866162 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d4102000 msgr2=0x7f88d410ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:05.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.363+0000 7f88d9639700 1 --2- 172.21.15.37:0/1529866162 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d4102000 0x7f88d410ef90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f88c800c8f0 tx=0x7f88c800cc00 comp rx=0 tx=0).stop 2024-06-10T00:57:05.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.363+0000 7f88d9639700 1 -- 172.21.15.37:0/1529866162 shutdown_connections 2024-06-10T00:57:05.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.363+0000 7f88d9639700 1 --2- 172.21.15.37:0/1529866162 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f88bc0a2cb0 0x7f88bc0a5170 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:05.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.363+0000 7f88d9639700 1 --2- 172.21.15.37:0/1529866162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88d41066a0 0x7f88d410f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:05.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.363+0000 7f88d9639700 1 --2- 172.21.15.37:0/1529866162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88d4101650 0x7f88d410ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:05.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.363+0000 7f88d9639700 1 --2- 172.21.15.37:0/1529866162 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d4102000 0x7f88d410ef90 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:05.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.363+0000 7f88d9639700 1 -- 172.21.15.37:0/1529866162 >> 172.21.15.37:0/1529866162 conn(0x7f88d40773e0 msgr2=0x7f88d40ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:05.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.363+0000 7f88d9639700 1 -- 172.21.15.37:0/1529866162 shutdown_connections 2024-06-10T00:57:05.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:05.363+0000 7f88d9639700 1 -- 172.21.15.37:0/1529866162 wait complete. 2024-06-10T00:57:06.182 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-06-10T00:57:06.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:06 smithi167 ceph-mon[36917]: pgmap v442: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-10T00:57:06.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:06 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1529866162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:57:06.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:06 smithi094 ceph-mon[36708]: pgmap v442: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-10T00:57:06.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:06 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1529866162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:57:06.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:06 smithi037 ceph-mon[32192]: pgmap v442: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-10T00:57:06.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:06 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1529866162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T00:57:06.441 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:57:08.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:08 smithi037 ceph-mon[32192]: pgmap v443: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-10T00:57:08.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:08 smithi167 ceph-mon[36917]: pgmap v443: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-10T00:57:08.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:08 smithi094 ceph-mon[36708]: pgmap v443: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-10T00:57:09.092 INFO:teuthology.orchestra.run.smithi037.stdout:{"run_dir":"/var/run/ceph"} 2024-06-10T00:57:09.093 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-06-10T00:57:09.093 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 178 2024-06-10T00:57:09.093 DEBUG:tasks.quiescer:Yielding 2024-06-10T00:57:09.093 INFO:teuthology.run_tasks:Running task workunit... 2024-06-10T00:57:09.106 INFO:tasks.workunit:Pulling workunits from ref 7b7a5d375cf4e4afab96b9d4d229b699ae8304bb 2024-06-10T00:57:09.107 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-06-10T00:57:09.107 INFO:tasks.workunit:timeout=3h 2024-06-10T00:57:09.107 INFO:tasks.workunit:cleanup=True 2024-06-10T00:57:09.107 DEBUG:teuthology.orchestra.run.smithi037:> stat -- /home/ubuntu/cephtest/mnt.0 2024-06-10T00:57:09.108 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:57:09.125 INFO:teuthology.orchestra.run.smithi037.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-06-10T00:57:09.125 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-06-10T00:57:09.125 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-06-10T00:57:09.125 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-10T00:57:09.125 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-06-10T00:57:09.126 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-06-10 00:53:55.733255913 +0000 2024-06-10T00:57:09.126 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-06-10 00:53:55.733255913 +0000 2024-06-10T00:57:09.126 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-06-10 00:56:29.588442519 +0000 2024-06-10T00:57:09.126 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-06-10 00:53:55.733255913 +0000 2024-06-10T00:57:09.126 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-06-10T00:57:09.126 DEBUG:teuthology.orchestra.run.smithi037:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-06-10T00:57:09.200 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 7b7a5d375cf4e4afab96b9d4d229b699ae8304bb 2024-06-10T00:57:09.256 INFO:tasks.workunit.client.0.smithi037.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-06-10T00:57:09.368 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:57:09.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.501+0000 7fc3d48ae640 1 -- 172.21.15.37:0/80554864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d0101b10 msgr2=0x7fc3d0101f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:09.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.501+0000 7fc3d48ae640 1 --2- 172.21.15.37:0/80554864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d0101b10 0x7fc3d0101f90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4009900 tx=0x7fc3c402a410 comp rx=0 tx=0).stop 2024-06-10T00:57:09.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.501+0000 7fc3d48ae640 1 -- 172.21.15.37:0/80554864 shutdown_connections 2024-06-10T00:57:09.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.501+0000 7fc3d48ae640 1 --2- 172.21.15.37:0/80554864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3d01024d0 0x7fc3d0108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.501+0000 7fc3d48ae640 1 --2- 172.21.15.37:0/80554864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d0101b10 0x7fc3d0101f90 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.501+0000 7fc3d48ae640 1 --2- 172.21.15.37:0/80554864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3d0100920 0x7fc3d0100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.501+0000 7fc3d48ae640 1 -- 172.21.15.37:0/80554864 >> 172.21.15.37:0/80554864 conn(0x7fc3d00fbeb0 msgr2=0x7fc3d00fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:09.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.502+0000 7fc3d48ae640 1 -- 172.21.15.37:0/80554864 shutdown_connections 2024-06-10T00:57:09.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.502+0000 7fc3d48ae640 1 -- 172.21.15.37:0/80554864 wait complete. 2024-06-10T00:57:09.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.503+0000 7fc3d48ae640 1 Processor -- start 2024-06-10T00:57:09.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.503+0000 7fc3d48ae640 1 -- start start 2024-06-10T00:57:09.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.503+0000 7fc3d48ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d0100920 0x7fc3d0197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.503+0000 7fc3d48ae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3d0101b10 0x7fc3d0198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.503+0000 7fc3d48ae640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3d01024d0 0x7fc3d019f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.503+0000 7fc3d48ae640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d010c5f0 con 0x7fc3d0100920 2024-06-10T00:57:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.503+0000 7fc3d48ae640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d010c470 con 0x7fc3d0101b10 2024-06-10T00:57:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.503+0000 7fc3d48ae640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d010c770 con 0x7fc3d01024d0 2024-06-10T00:57:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.503+0000 7fc3ce59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d0100920 0x7fc3d0197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.503+0000 7fc3ce59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d0100920 0x7fc3d0197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51340/0 (socket says 172.21.15.37:51340) 2024-06-10T00:57:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.503+0000 7fc3ce59c640 1 -- 172.21.15.37:0/2982044280 learned_addr learned my addr 172.21.15.37:0/2982044280 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.503+0000 7fc3cdd9b640 1 --2- 172.21.15.37:0/2982044280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3d0101b10 0x7fc3d0198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.503+0000 7fc3ced9d640 1 --2- 172.21.15.37:0/2982044280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3d01024d0 0x7fc3d019f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.503+0000 7fc3ce59c640 1 -- 172.21.15.37:0/2982044280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3d01024d0 msgr2=0x7fc3d019f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.503+0000 7fc3ce59c640 1 --2- 172.21.15.37:0/2982044280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3d01024d0 0x7fc3d019f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.503+0000 7fc3ce59c640 1 -- 172.21.15.37:0/2982044280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3d0101b10 msgr2=0x7fc3d0198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.503+0000 7fc3ce59c640 1 --2- 172.21.15.37:0/2982044280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3d0101b10 0x7fc3d0198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.503+0000 7fc3ce59c640 1 -- 172.21.15.37:0/2982044280 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d019f960 con 0x7fc3d0100920 2024-06-10T00:57:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.503+0000 7fc3ced9d640 1 --2- 172.21.15.37:0/2982044280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3d01024d0 0x7fc3d019f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:57:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.503+0000 7fc3cdd9b640 1 --2- 172.21.15.37:0/2982044280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3d0101b10 0x7fc3d0198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:57:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.504+0000 7fc3ce59c640 1 --2- 172.21.15.37:0/2982044280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d0100920 0x7fc3d0197d50 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc3c000e9c0 tx=0x7fc3c000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.504+0000 7fc3bf7fe640 1 -- 172.21.15.37:0/2982044280 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c000cd80 con 0x7fc3d0100920 2024-06-10T00:57:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.504+0000 7fc3d48ae640 1 -- 172.21.15.37:0/2982044280 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3d019fc50 con 0x7fc3d0100920 2024-06-10T00:57:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.504+0000 7fc3bf7fe640 1 -- 172.21.15.37:0/2982044280 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3c00035f0 con 0x7fc3d0100920 2024-06-10T00:57:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.504+0000 7fc3bf7fe640 1 -- 172.21.15.37:0/2982044280 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c0010690 con 0x7fc3d0100920 2024-06-10T00:57:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.504+0000 7fc3d48ae640 1 -- 172.21.15.37:0/2982044280 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3d01a01a0 con 0x7fc3d0100920 2024-06-10T00:57:09.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.506+0000 7fc3bf7fe640 1 -- 172.21.15.37:0/2982044280 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fc3c00037a0 con 0x7fc3d0100920 2024-06-10T00:57:09.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.506+0000 7fc3d75b7740 1 Processor -- start 2024-06-10T00:57:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.507+0000 7fc3d75b7740 1 -- start start 2024-06-10T00:57:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.507+0000 7fc3d75b7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6a8fd010 0x55dc6a8fd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.507+0000 7fc3d75b7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6a8fda00 0x55dc6a902500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.507+0000 7fc3d75b7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6a902b70 0x55dc6a905000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.507+0000 7fc3d75b7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dc6a8604f0 con 0x55dc6a902b70 2024-06-10T00:57:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.507+0000 7fc3d75b7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55dc6a7fe870 con 0x55dc6a8fda00 2024-06-10T00:57:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.507+0000 7fc3d75b7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dc6a85c4b0 con 0x55dc6a8fd010 2024-06-10T00:57:09.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.507+0000 7fc3bf7fe640 1 --2- 172.21.15.37:0/2982044280 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3b80a2cd0 0x7fc3b80a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:09.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.508+0000 7fc3ce59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6a8fda00 0x55dc6a902500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:09.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.508+0000 7fc3ce59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6a8fda00 0x55dc6a902500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:43450/0 (socket says 172.21.15.37:43450) 2024-06-10T00:57:09.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.508+0000 7fc3ce59c640 1 -- 172.21.15.37:0/1179315537 learned_addr learned my addr 172.21.15.37:0/1179315537 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:09.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.508+0000 7fc3ced9d640 1 --2- 172.21.15.37:0/1179315537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6a8fd010 0x55dc6a8fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.508+0000 7fc3ced9d640 1 --2- 172.21.15.37:0/2982044280 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3b80a2cd0 0x7fc3b80a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.508+0000 7fc3cdd9b640 1 --2- 172.21.15.37:0/1179315537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6a902b70 0x55dc6a905000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.508+0000 7fc3bf7fe640 1 -- 172.21.15.37:0/2982044280 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fc3c00cfba0 con 0x7fc3d0100920 2024-06-10T00:57:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.508+0000 7fc3ced9d640 1 -- 172.21.15.37:0/1179315537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6a8fda00 msgr2=0x55dc6a902500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.508+0000 7fc3ced9d640 1 --2- 172.21.15.37:0/1179315537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6a8fda00 0x55dc6a902500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.508+0000 7fc3ced9d640 1 -- 172.21.15.37:0/1179315537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6a902b70 msgr2=0x55dc6a905000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.508+0000 7fc3ced9d640 1 --2- 172.21.15.37:0/1179315537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6a902b70 0x55dc6a905000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.508+0000 7fc3ced9d640 1 -- 172.21.15.37:0/1179315537 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc6a81a9b0 con 0x55dc6a8fd010 2024-06-10T00:57:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.508+0000 7fc3ced9d640 1 --2- 172.21.15.37:0/2982044280 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3b80a2cd0 0x7fc3b80a5190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8010ca0 tx=0x7fc3c801d000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.508+0000 7fc3ced9d640 1 --2- 172.21.15.37:0/1179315537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6a8fd010 0x55dc6a8fd430 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8005fd0 tx=0x7fc3c80107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=327bd48ff591d8b5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.509+0000 7fc3bcff9640 1 -- 172.21.15.37:0/1179315537 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8012890 con 0x55dc6a8fd010 2024-06-10T00:57:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.509+0000 7fc3bcff9640 1 -- 172.21.15.37:0/1179315537 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3c8009d20 con 0x55dc6a8fd010 2024-06-10T00:57:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.509+0000 7fc3bcff9640 1 -- 172.21.15.37:0/1179315537 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c80054a0 con 0x55dc6a8fd010 2024-06-10T00:57:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.509+0000 7fc3d75b7740 1 -- 172.21.15.37:0/1179315537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6a8fd010 msgr2=0x55dc6a8fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.509+0000 7fc3d75b7740 1 --2- 172.21.15.37:0/1179315537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6a8fd010 0x55dc6a8fd430 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8005fd0 tx=0x7fc3c80107a0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.510+0000 7fc3d75b7740 1 -- 172.21.15.37:0/1179315537 shutdown_connections 2024-06-10T00:57:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.510+0000 7fc3d75b7740 1 --2- 172.21.15.37:0/1179315537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6a902b70 0x55dc6a905000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.510+0000 7fc3d75b7740 1 --2- 172.21.15.37:0/1179315537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6a8fda00 0x55dc6a902500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.510+0000 7fc3d75b7740 1 --2- 172.21.15.37:0/1179315537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6a8fd010 0x55dc6a8fd430 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.510+0000 7fc3d75b7740 1 -- 172.21.15.37:0/1179315537 >> 172.21.15.37:0/1179315537 conn(0x55dc6a8f8680 msgr2=0x55dc6a8f8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.510+0000 7fc3d75b7740 1 -- 172.21.15.37:0/1179315537 shutdown_connections 2024-06-10T00:57:09.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.510+0000 7fc3d75b7740 1 -- 172.21.15.37:0/1179315537 wait complete. 2024-06-10T00:57:09.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.510+0000 7fc3d75b7740 1 -- start start 2024-06-10T00:57:09.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.510+0000 7fc3d75b7740 1 Processor -- start 2024-06-10T00:57:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.511+0000 7fc3d75b7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6a8fd010 0x55dc6a907130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.511+0000 7fc3d75b7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6a8fda00 0x55dc6a907670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.511+0000 7fc3d75b7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6a902b70 0x55dc6a90e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.511+0000 7fc3d75b7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dc6a905a60 con 0x55dc6a902b70 2024-06-10T00:57:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.511+0000 7fc3d75b7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55dc6a7e9ae0 con 0x55dc6a8fda00 2024-06-10T00:57:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.511+0000 7fc3d75b7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dc6a905be0 con 0x55dc6a8fd010 2024-06-10T00:57:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.511+0000 7fc3d75b7740 10 client.0 _ll_register_callbacks cb 0x55dc6a8229b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.511+0000 7fc3ced9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6a902b70 0x55dc6a90e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.511+0000 7fc3ced9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6a902b70 0x55dc6a90e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51368/0 (socket says 172.21.15.37:51368) 2024-06-10T00:57:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.511+0000 7fc3ced9d640 1 -- 172.21.15.37:0/3851045892 learned_addr learned my addr 172.21.15.37:0/3851045892 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.511+0000 7fc3ce59c640 1 --2- 172.21.15.37:0/3851045892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6a8fd010 0x55dc6a907130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.512+0000 7fc3ced9d640 1 -- 172.21.15.37:0/3851045892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6a8fd010 msgr2=0x55dc6a907130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.512+0000 7fc3ced9d640 1 --2- 172.21.15.37:0/3851045892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6a8fd010 0x55dc6a907130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.512+0000 7fc3ced9d640 1 -- 172.21.15.37:0/3851045892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6a8fda00 msgr2=0x55dc6a907670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:09.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.512+0000 7fc3cdd9b640 1 --2- 172.21.15.37:0/3851045892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6a8fda00 0x55dc6a907670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.512+0000 7fc3ced9d640 1 --2- 172.21.15.37:0/3851045892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6a8fda00 0x55dc6a907670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.512+0000 7fc3ced9d640 1 -- 172.21.15.37:0/3851045892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc6a574210 con 0x55dc6a902b70 2024-06-10T00:57:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.512+0000 7fc3cdd9b640 1 --2- 172.21.15.37:0/3851045892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6a8fda00 0x55dc6a907670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:57:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.512+0000 7fc3ced9d640 1 --2- 172.21.15.37:0/3851045892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6a902b70 0x55dc6a90e6b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8027040 tx=0x7fc3c800de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f50892763789658 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.512+0000 7fc3b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:57:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.512+0000 7fc3b5ffb640 1 -- 172.21.15.37:0/3851045892 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8039070 con 0x55dc6a902b70 2024-06-10T00:57:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.512+0000 7fc3b5ffb640 1 -- 172.21.15.37:0/3851045892 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fc3c802d3d0 con 0x55dc6a902b70 2024-06-10T00:57:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.512+0000 7fc3d75b7740 1 -- 172.21.15.37:0/3851045892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3c8009d20 con 0x55dc6a902b70 2024-06-10T00:57:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.513+0000 7fc3b5ffb640 1 -- 172.21.15.37:0/3851045892 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc3c8045050 con 0x55dc6a902b70 2024-06-10T00:57:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.513+0000 7fc3d75b7740 10 client.15399 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.513+0000 7fc3d75b7740 1 -- 172.21.15.37:0/3851045892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dc6a640620 con 0x55dc6a902b70 2024-06-10T00:57:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.513+0000 7fc3b5ffb640 1 -- 172.21.15.37:0/3851045892 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3c8009d20 con 0x55dc6a902b70 2024-06-10T00:57:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.513+0000 7fc3b5ffb640 1 -- 172.21.15.37:0/3851045892 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc3c801e420 con 0x55dc6a902b70 2024-06-10T00:57:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.513+0000 7fc3d75b7740 10 client.15399 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.513+0000 7fc3d75b7740 10 client.15399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T00:57:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.513+0000 7fc3d75b7740 20 client.15399 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.513+0000 7fc3d75b7740 1 --2- 172.21.15.37:0/3851045892 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55dc6a912dd0 0x55dc6a9151c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.513+0000 7fc3d75b7740 4 client.15399 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:57:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.513+0000 7fc3d75b7740 1 -- 172.21.15.37:0/3851045892 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc6a873f10 con 0x55dc6a912dd0 2024-06-10T00:57:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.513+0000 7fc3ce59c640 1 --2- 172.21.15.37:0/3851045892 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55dc6a912dd0 0x55dc6a9151c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.514+0000 7fc3ce59c640 1 --2- 172.21.15.37:0/3851045892 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55dc6a912dd0 0x55dc6a9151c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac0f1148f23617c8 server_cookie=415e06e202b4ecfa in_seq=0 out_seq=0 2024-06-10T00:57:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.514+0000 7fc3b5ffb640 10 client.15399 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.515+0000 7fc3b5ffb640 1 -- 172.21.15.37:0/3851045892 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55dc6a873f10 con 0x55dc6a912dd0 2024-06-10T00:57:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.515+0000 7fc3b5ffb640 10 client.15399 handle_command_reply: tid=0 2024-06-10T00:57:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.515+0000 7fc3d75b7740 1 client.15399 shutdown 2024-06-10T00:57:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.515+0000 7fc3d75b7740 1 -- 172.21.15.37:0/3851045892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6a902b70 msgr2=0x55dc6a90e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.515+0000 7fc3d75b7740 1 --2- 172.21.15.37:0/3851045892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6a902b70 0x55dc6a90e6b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8027040 tx=0x7fc3c800de50 comp rx=0 tx=0).stop 2024-06-10T00:57:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.515+0000 7fc3d75b7740 1 -- 172.21.15.37:0/3851045892 shutdown_connections 2024-06-10T00:57:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.515+0000 7fc3d75b7740 1 --2- 172.21.15.37:0/3851045892 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55dc6a912dd0 0x55dc6a9151c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.515+0000 7fc3d75b7740 1 --2- 172.21.15.37:0/3851045892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6a902b70 0x55dc6a90e6b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.515+0000 7fc3d75b7740 1 --2- 172.21.15.37:0/3851045892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6a8fda00 0x55dc6a907670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.515+0000 7fc3b5ffb640 0 client.15399 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.515+0000 7fc3d75b7740 1 --2- 172.21.15.37:0/3851045892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6a8fd010 0x55dc6a907130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.515+0000 7fc3d75b7740 1 -- 172.21.15.37:0/3851045892 >> 172.21.15.37:0/3851045892 conn(0x55dc6a8f8680 msgr2=0x55dc6a8f77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.515+0000 7fc3d75b7740 1 -- 172.21.15.37:0/3851045892 shutdown_connections 2024-06-10T00:57:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.515+0000 7fc3d75b7740 1 -- 172.21.15.37:0/3851045892 wait complete. 2024-06-10T00:57:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.515+0000 7fc3d75b7740 20 client.15399 trim_cache size 0 max 16384 2024-06-10T00:57:09.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.530+0000 7fc3d75b7740 1 Processor -- start 2024-06-10T00:57:09.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.530+0000 7fc3d75b7740 1 -- start start 2024-06-10T00:57:09.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.530+0000 7fc3d75b7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6a8fd010 0x55dc6a907130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:09.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.530+0000 7fc3d75b7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6a8fda00 0x55dc6a907670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.530+0000 7fc3d75b7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6a902b70 0x55dc6a90e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.530+0000 7fc3d75b7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d01aaea0 con 0x55dc6a8fd010 2024-06-10T00:57:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.530+0000 7fc3d75b7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d01a8770 con 0x55dc6a8fda00 2024-06-10T00:57:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.530+0000 7fc3d75b7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d01a65c0 con 0x55dc6a902b70 2024-06-10T00:57:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.530+0000 7fc3ce59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6a8fd010 0x55dc6a907130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.530+0000 7fc3ce59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6a8fd010 0x55dc6a907130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51382/0 (socket says 172.21.15.37:51382) 2024-06-10T00:57:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.530+0000 7fc3ce59c640 1 -- 172.21.15.37:0/2078069580 learned_addr learned my addr 172.21.15.37:0/2078069580 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.531+0000 7fc3cdd9b640 1 --2- 172.21.15.37:0/2078069580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6a8fda00 0x55dc6a907670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.531+0000 7fc3ced9d640 1 --2- 172.21.15.37:0/2078069580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6a902b70 0x55dc6a90e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.531+0000 7fc3ce59c640 1 -- 172.21.15.37:0/2078069580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6a902b70 msgr2=0x55dc6a90e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.531+0000 7fc3ce59c640 1 --2- 172.21.15.37:0/2078069580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6a902b70 0x55dc6a90e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.531+0000 7fc3ce59c640 1 -- 172.21.15.37:0/2078069580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6a8fda00 msgr2=0x55dc6a907670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.531+0000 7fc3ce59c640 1 --2- 172.21.15.37:0/2078069580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6a8fda00 0x55dc6a907670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.531+0000 7fc3ce59c640 1 -- 172.21.15.37:0/2078069580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc6a915530 con 0x55dc6a8fd010 2024-06-10T00:57:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.531+0000 7fc3ce59c640 1 --2- 172.21.15.37:0/2078069580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6a8fd010 0x55dc6a907130 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc3c009bbb0 tx=0x7fc3c00cfe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39fb98bb01e295d4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.531+0000 7fc3b7fff640 1 -- 172.21.15.37:0/2078069580 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c009c5b0 con 0x55dc6a8fd010 2024-06-10T00:57:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.531+0000 7fc3b7fff640 1 -- 172.21.15.37:0/2078069580 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3c009beb0 con 0x55dc6a8fd010 2024-06-10T00:57:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.531+0000 7fc3b7fff640 1 -- 172.21.15.37:0/2078069580 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c00cd3b0 con 0x55dc6a8fd010 2024-06-10T00:57:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.532+0000 7fc3d75b7740 1 -- 172.21.15.37:0/2078069580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6a8fd010 msgr2=0x55dc6a907130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.532+0000 7fc3d75b7740 1 --2- 172.21.15.37:0/2078069580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6a8fd010 0x55dc6a907130 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc3c009bbb0 tx=0x7fc3c00cfe60 comp rx=0 tx=0).stop 2024-06-10T00:57:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.532+0000 7fc3d75b7740 1 -- 172.21.15.37:0/2078069580 shutdown_connections 2024-06-10T00:57:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.532+0000 7fc3d75b7740 1 --2- 172.21.15.37:0/2078069580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6a902b70 0x55dc6a90e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.532+0000 7fc3d75b7740 1 --2- 172.21.15.37:0/2078069580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6a8fda00 0x55dc6a907670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.532+0000 7fc3d75b7740 1 --2- 172.21.15.37:0/2078069580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6a8fd010 0x55dc6a907130 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.532+0000 7fc3d75b7740 1 -- 172.21.15.37:0/2078069580 >> 172.21.15.37:0/2078069580 conn(0x55dc6a8f8680 msgr2=0x55dc6a902220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.532+0000 7fc3d75b7740 1 -- 172.21.15.37:0/2078069580 shutdown_connections 2024-06-10T00:57:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.532+0000 7fc3d75b7740 1 -- 172.21.15.37:0/2078069580 wait complete. 2024-06-10T00:57:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.532+0000 7fc3d75b7740 1 -- start start 2024-06-10T00:57:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.532+0000 7fc3d75b7740 1 Processor -- start 2024-06-10T00:57:09.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.532+0000 7fc3d75b7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6a8fd010 0x55dc6a90e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:09.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.532+0000 7fc3d75b7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6a8fda00 0x55dc6a90b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:09.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.532+0000 7fc3d75b7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6a902b70 0x55dc6a90b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:09.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.532+0000 7fc3d75b7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d01a6c30 con 0x55dc6a8fda00 2024-06-10T00:57:09.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.532+0000 7fc3d75b7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d01a80a0 con 0x55dc6a902b70 2024-06-10T00:57:09.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.532+0000 7fc3d75b7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d01a7300 con 0x55dc6a8fd010 2024-06-10T00:57:09.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.532+0000 7fc3d75b7740 10 client.0 _ll_register_callbacks cb 0x7fc3c0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:09.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.533+0000 7fc3cdd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6a8fda00 0x55dc6a90b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:09.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.533+0000 7fc3cdd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6a8fda00 0x55dc6a90b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51390/0 (socket says 172.21.15.37:51390) 2024-06-10T00:57:09.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.533+0000 7fc3cdd9b640 1 -- 172.21.15.37:0/585793628 learned_addr learned my addr 172.21.15.37:0/585793628 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:09.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.533+0000 7fc3ced9d640 1 --2- 172.21.15.37:0/585793628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6a902b70 0x55dc6a90b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:09.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.533+0000 7fc3ce59c640 1 --2- 172.21.15.37:0/585793628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6a8fd010 0x55dc6a90e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:09.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.533+0000 7fc3cdd9b640 1 -- 172.21.15.37:0/585793628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6a8fd010 msgr2=0x55dc6a90e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:09.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.533+0000 7fc3cdd9b640 1 --2- 172.21.15.37:0/585793628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6a8fd010 0x55dc6a90e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.533+0000 7fc3cdd9b640 1 -- 172.21.15.37:0/585793628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6a902b70 msgr2=0x55dc6a90b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:09.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.533+0000 7fc3cdd9b640 1 --2- 172.21.15.37:0/585793628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6a902b70 0x55dc6a90b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.533+0000 7fc3cdd9b640 1 -- 172.21.15.37:0/585793628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc6a97bb20 con 0x55dc6a8fda00 2024-06-10T00:57:09.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.533+0000 7fc3ced9d640 1 --2- 172.21.15.37:0/585793628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6a902b70 0x55dc6a90b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:57:09.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.533+0000 7fc3cdd9b640 1 --2- 172.21.15.37:0/585793628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6a8fda00 0x55dc6a90b0f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4009230 tx=0x7fc3c4005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986fe74a53b662f1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:09.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.533+0000 7fc3b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:57:09.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.533+0000 7fc3b5ffb640 1 -- 172.21.15.37:0/585793628 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4035040 con 0x55dc6a8fda00 2024-06-10T00:57:09.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.533+0000 7fc3b5ffb640 1 -- 172.21.15.37:0/585793628 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fc3c40316b0 con 0x55dc6a8fda00 2024-06-10T00:57:09.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.533+0000 7fc3d75b7740 1 -- 172.21.15.37:0/585793628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3c009beb0 con 0x55dc6a8fda00 2024-06-10T00:57:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.533+0000 7fc3b5ffb640 1 -- 172.21.15.37:0/585793628 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc3c404e050 con 0x55dc6a8fda00 2024-06-10T00:57:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.533+0000 7fc3d75b7740 10 client.15405 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.533+0000 7fc3d75b7740 1 -- 172.21.15.37:0/585793628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dc6a97ce70 con 0x55dc6a8fda00 2024-06-10T00:57:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.534+0000 7fc3b5ffb640 1 -- 172.21.15.37:0/585793628 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3c009beb0 con 0x55dc6a8fda00 2024-06-10T00:57:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.534+0000 7fc3b5ffb640 1 -- 172.21.15.37:0/585793628 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc3c4043020 con 0x55dc6a8fda00 2024-06-10T00:57:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.534+0000 7fc3d75b7740 10 client.15405 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.534+0000 7fc3d75b7740 10 client.15405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T00:57:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.534+0000 7fc3d75b7740 20 client.15405 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.534+0000 7fc3d75b7740 1 --2- 172.21.15.37:0/585793628 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55dc6a97d8a0 0x55dc6a97fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.534+0000 7fc3d75b7740 4 client.15405 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T00:57:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.534+0000 7fc3d75b7740 1 -- 172.21.15.37:0/585793628 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dc6a86dc20 con 0x55dc6a97d8a0 2024-06-10T00:57:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.534+0000 7fc3ce59c640 1 --2- 172.21.15.37:0/585793628 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55dc6a97d8a0 0x55dc6a97fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.535+0000 7fc3ce59c640 1 --2- 172.21.15.37:0/585793628 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55dc6a97d8a0 0x55dc6a97fcc0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7db98f2bcbe8afed server_cookie=b3b5591cbf085ee9 in_seq=0 out_seq=0 2024-06-10T00:57:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.535+0000 7fc3b5ffb640 10 client.15405 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.542+0000 7fc3b5ffb640 1 -- 172.21.15.37:0/585793628 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55dc6a86dc20 con 0x55dc6a97d8a0 2024-06-10T00:57:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.542+0000 7fc3b5ffb640 10 client.15405 handle_command_reply: tid=0 2024-06-10T00:57:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.542+0000 7fc3d75b7740 1 client.15405 shutdown 2024-06-10T00:57:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.542+0000 7fc3d75b7740 1 -- 172.21.15.37:0/585793628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6a8fda00 msgr2=0x55dc6a90b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.542+0000 7fc3d75b7740 1 --2- 172.21.15.37:0/585793628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6a8fda00 0x55dc6a90b0f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4009230 tx=0x7fc3c4005270 comp rx=0 tx=0).stop 2024-06-10T00:57:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.543+0000 7fc3d75b7740 1 -- 172.21.15.37:0/585793628 shutdown_connections 2024-06-10T00:57:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.543+0000 7fc3d75b7740 1 --2- 172.21.15.37:0/585793628 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55dc6a97d8a0 0x55dc6a97fcc0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.543+0000 7fc3d75b7740 1 --2- 172.21.15.37:0/585793628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6a902b70 0x55dc6a90b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.543+0000 7fc3b5ffb640 0 client.15405 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.543+0000 7fc3d75b7740 1 --2- 172.21.15.37:0/585793628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6a8fda00 0x55dc6a90b0f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.543+0000 7fc3d75b7740 1 --2- 172.21.15.37:0/585793628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6a8fd010 0x55dc6a90e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.543+0000 7fc3d75b7740 1 -- 172.21.15.37:0/585793628 >> 172.21.15.37:0/585793628 conn(0x55dc6a8f8680 msgr2=0x55dc6a8febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.543+0000 7fc3d75b7740 1 -- 172.21.15.37:0/585793628 shutdown_connections 2024-06-10T00:57:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.543+0000 7fc3d75b7740 1 -- 172.21.15.37:0/585793628 wait complete. 2024-06-10T00:57:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.543+0000 7fc3d75b7740 20 client.15405 trim_cache size 0 max 16384 2024-06-10T00:57:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T00:57:09.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T00:57:09.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T00:57:09.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.543+0000 7fc3d48ae640 1 -- 172.21.15.37:0/2982044280 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3b80a2cd0 msgr2=0x7fc3b80a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:09.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.544+0000 7fc3d48ae640 1 --2- 172.21.15.37:0/2982044280 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3b80a2cd0 0x7fc3b80a5190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8010ca0 tx=0x7fc3c801d000 comp rx=0 tx=0).stop 2024-06-10T00:57:09.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.544+0000 7fc3d48ae640 1 -- 172.21.15.37:0/2982044280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d0100920 msgr2=0x7fc3d0197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:09.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.544+0000 7fc3d48ae640 1 --2- 172.21.15.37:0/2982044280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d0100920 0x7fc3d0197d50 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc3c000e9c0 tx=0x7fc3c000ee80 comp rx=0 tx=0).stop 2024-06-10T00:57:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.544+0000 7fc3d48ae640 1 -- 172.21.15.37:0/2982044280 shutdown_connections 2024-06-10T00:57:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.544+0000 7fc3d48ae640 1 --2- 172.21.15.37:0/2982044280 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3b80a2cd0 0x7fc3b80a5190 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.544+0000 7fc3d48ae640 1 --2- 172.21.15.37:0/2982044280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3d01024d0 0x7fc3d019f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.544+0000 7fc3d48ae640 1 --2- 172.21.15.37:0/2982044280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3d0101b10 0x7fc3d0198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.544+0000 7fc3d48ae640 1 --2- 172.21.15.37:0/2982044280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3d0100920 0x7fc3d0197d50 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.544+0000 7fc3d48ae640 1 -- 172.21.15.37:0/2982044280 >> 172.21.15.37:0/2982044280 conn(0x7fc3d00fbeb0 msgr2=0x7fc3d00fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.544+0000 7fc3d48ae640 1 -- 172.21.15.37:0/2982044280 shutdown_connections 2024-06-10T00:57:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:09.544+0000 7fc3d48ae640 1 -- 172.21.15.37:0/2982044280 wait complete. 2024-06-10T00:57:10.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:10 smithi094 ceph-mon[36708]: pgmap v444: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-10T00:57:10.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:10 smithi037 ceph-mon[32192]: pgmap v444: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-10T00:57:10.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:10 smithi167 ceph-mon[36917]: pgmap v444: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-10T00:57:11.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.146+0000 7f6543094700 1 -- 172.21.15.37:0/943766596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f653c107840 msgr2=0x7f653c10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:11.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.146+0000 7f6543094700 1 --2- 172.21.15.37:0/943766596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f653c107840 0x7f653c10f3c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f652c0099c0 tx=0x7f652c009cd0 comp rx=0 tx=0).stop 2024-06-10T00:57:11.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.146+0000 7f6543094700 1 -- 172.21.15.37:0/943766596 shutdown_connections 2024-06-10T00:57:11.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.146+0000 7f6543094700 1 --2- 172.21.15.37:0/943766596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f653c107840 0x7f653c10f3c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:11.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.146+0000 7f6543094700 1 --2- 172.21.15.37:0/943766596 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f653c102740 0x7f653c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:11.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.146+0000 7f6543094700 1 --2- 172.21.15.37:0/943766596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f653c1030f0 0x7f653c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:11.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.146+0000 7f6543094700 1 -- 172.21.15.37:0/943766596 >> 172.21.15.37:0/943766596 conn(0x7f653c0795f0 msgr2=0x7f653c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:11.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.146+0000 7f6543094700 1 -- 172.21.15.37:0/943766596 shutdown_connections 2024-06-10T00:57:11.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.147+0000 7f6543094700 1 -- 172.21.15.37:0/943766596 wait complete. 2024-06-10T00:57:11.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.147+0000 7f6543094700 1 Processor -- start 2024-06-10T00:57:11.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.147+0000 7f6543094700 1 -- start start 2024-06-10T00:57:11.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.147+0000 7f6543094700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f653c102740 0x7f653c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:11.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.147+0000 7f6543094700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f653c1030f0 0x7f653c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:11.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.147+0000 7f6543094700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f653c107840 0x7f653c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:11.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.147+0000 7f6543094700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f653c1127d0 con 0x7f653c1030f0 2024-06-10T00:57:11.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.147+0000 7f6543094700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f653c112650 con 0x7f653c107840 2024-06-10T00:57:11.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.147+0000 7f6543094700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f653c112950 con 0x7f653c102740 2024-06-10T00:57:11.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.147+0000 7f653bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f653c1030f0 0x7f653c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:11.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.148+0000 7f653bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f653c1030f0 0x7f653c074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51416/0 (socket says 172.21.15.37:51416) 2024-06-10T00:57:11.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.148+0000 7f6540e30700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f653c102740 0x7f653c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:11.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.148+0000 7f653bfff700 1 -- 172.21.15.37:0/722153897 learned_addr learned my addr 172.21.15.37:0/722153897 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:11.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.148+0000 7f6541631700 1 --2- 172.21.15.37:0/722153897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f653c107840 0x7f653c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:11.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.148+0000 7f653bfff700 1 -- 172.21.15.37:0/722153897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f653c102740 msgr2=0x7f653c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:11.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.148+0000 7f653bfff700 1 --2- 172.21.15.37:0/722153897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f653c102740 0x7f653c074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:11.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.148+0000 7f653bfff700 1 -- 172.21.15.37:0/722153897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f653c107840 msgr2=0x7f653c075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:11.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.148+0000 7f653bfff700 1 --2- 172.21.15.37:0/722153897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f653c107840 0x7f653c075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:11.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.148+0000 7f653bfff700 1 -- 172.21.15.37:0/722153897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f653c1a70f0 con 0x7f653c1030f0 2024-06-10T00:57:11.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.148+0000 7f653bfff700 1 --2- 172.21.15.37:0/722153897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f653c1030f0 0x7f653c074cd0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f653400ef30 tx=0x7f653400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:11.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.148+0000 7f6539ffb700 1 -- 172.21.15.37:0/722153897 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65340092e0 con 0x7f653c1030f0 2024-06-10T00:57:11.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.148+0000 7f6543094700 1 -- 172.21.15.37:0/722153897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f653c1a7380 con 0x7f653c1030f0 2024-06-10T00:57:11.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.148+0000 7f6539ffb700 1 -- 172.21.15.37:0/722153897 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6534004bb0 con 0x7f653c1030f0 2024-06-10T00:57:11.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.148+0000 7f6539ffb700 1 -- 172.21.15.37:0/722153897 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6534004280 con 0x7f653c1030f0 2024-06-10T00:57:11.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.149+0000 7f6543094700 1 -- 172.21.15.37:0/722153897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f653c1a78c0 con 0x7f653c1030f0 2024-06-10T00:57:11.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.149+0000 7f6543094700 1 -- 172.21.15.37:0/722153897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f653c069180 con 0x7f653c1030f0 2024-06-10T00:57:11.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.150+0000 7f6539ffb700 1 -- 172.21.15.37:0/722153897 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f653401b030 con 0x7f653c1030f0 2024-06-10T00:57:11.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.151+0000 7f6539ffb700 1 --2- 172.21.15.37:0/722153897 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f65280a2cb0 0x7f65280a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:11.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.151+0000 7f6540e30700 1 --2- 172.21.15.37:0/722153897 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f65280a2cb0 0x7f65280a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:11.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.151+0000 7f6539ffb700 1 -- 172.21.15.37:0/722153897 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f65340cf0c0 con 0x7f653c1030f0 2024-06-10T00:57:11.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.152+0000 7f6540e30700 1 --2- 172.21.15.37:0/722153897 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f65280a2cb0 0x7f65280a5170 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6524005950 tx=0x7f652400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:11.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.157+0000 7f6539ffb700 1 -- 172.21.15.37:0/722153897 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6534022020 con 0x7f653c1030f0 2024-06-10T00:57:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.404+0000 7f6543094700 1 -- 172.21.15.37:0/722153897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f653c02de60 con 0x7f653c1030f0 2024-06-10T00:57:11.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.405+0000 7f6539ffb700 1 -- 172.21.15.37:0/722153897 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11867 (secure 0 0 0) 0x7f6534010040 con 0x7f653c1030f0 2024-06-10T00:57:11.406 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:57:11.406 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T00:56:30.291171+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24451,24458,24472]},"id":1}]} 2024-06-10T00:57:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.407+0000 7f6543094700 1 -- 172.21.15.37:0/722153897 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f65280a2cb0 msgr2=0x7f65280a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.407+0000 7f6543094700 1 --2- 172.21.15.37:0/722153897 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f65280a2cb0 0x7f65280a5170 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6524005950 tx=0x7f652400b410 comp rx=0 tx=0).stop 2024-06-10T00:57:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.407+0000 7f6543094700 1 -- 172.21.15.37:0/722153897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f653c1030f0 msgr2=0x7f653c074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.407+0000 7f6543094700 1 --2- 172.21.15.37:0/722153897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f653c1030f0 0x7f653c074cd0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f653400ef30 tx=0x7f653400c3a0 comp rx=0 tx=0).stop 2024-06-10T00:57:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.407+0000 7f6543094700 1 -- 172.21.15.37:0/722153897 shutdown_connections 2024-06-10T00:57:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.407+0000 7f6543094700 1 --2- 172.21.15.37:0/722153897 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f65280a2cb0 0x7f65280a5170 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.407+0000 7f6543094700 1 --2- 172.21.15.37:0/722153897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f653c107840 0x7f653c075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.407+0000 7f6543094700 1 --2- 172.21.15.37:0/722153897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f653c102740 0x7f653c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.407+0000 7f6543094700 1 --2- 172.21.15.37:0/722153897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f653c1030f0 0x7f653c074cd0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.407+0000 7f6543094700 1 -- 172.21.15.37:0/722153897 >> 172.21.15.37:0/722153897 conn(0x7f653c0795f0 msgr2=0x7f653c100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.407+0000 7f6543094700 1 -- 172.21.15.37:0/722153897 shutdown_connections 2024-06-10T00:57:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:11.407+0000 7f6543094700 1 -- 172.21.15.37:0/722153897 wait complete. 2024-06-10T00:57:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-06-10T00:57:12.165 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-10T00:52:37.869277+0000', 'modified': '2024-06-10T00:56:30.291171+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24472, 'mds_1': 24458, 'mds_2': 24451}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24413': {'gid': 24413, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.94:6835/570288048', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6834', 'nonce': 570288048}, {'type': 'v1', 'addr': '172.21.15.94:6835', 'nonce': 570288048}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.94:6837/3801637791', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6836', 'nonce': 3801637791}, {'type': 'v1', 'addr': '172.21.15.94:6837', 'nonce': 3801637791}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6835/3066042611', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 3066042611}, {'type': 'v1', 'addr': '172.21.15.167:6835', 'nonce': 3066042611}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.167:6837/2720148065', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 2720148065}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 2720148065}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.94:6839/946269331', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6838', 'nonce': 946269331}, {'type': 'v1', 'addr': '172.21.15.94:6839', 'nonce': 946269331}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.167:6839/3945638144', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 3945638144}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 3945638144}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24472, 'qdb_cluster': [24451, 24458, 24472]} 2024-06-10T00:57:12.165 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-06-10T00:57:12.165 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.c status 2024-06-10T00:57:12.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:12 smithi094 ceph-mon[36708]: pgmap v445: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-10T00:57:12.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:12 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/722153897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:57:12.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:12 smithi037 ceph-mon[32192]: pgmap v445: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-10T00:57:12.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:12 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/722153897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:57:12.461 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:57:12.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:12 smithi167 ceph-mon[36917]: pgmap v445: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-10T00:57:12.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:12 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/722153897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:57:14.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.308+0000 7fa640107700 1 -- 172.21.15.37:0/1466259531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa638107840 msgr2=0x7fa63810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:14.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.308+0000 7fa640107700 1 --2- 172.21.15.37:0/1466259531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa638107840 0x7fa63810f3c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa63000b0d0 tx=0x7fa63000b3e0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.308+0000 7fa640107700 1 -- 172.21.15.37:0/1466259531 shutdown_connections 2024-06-10T00:57:14.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.308+0000 7fa640107700 1 --2- 172.21.15.37:0/1466259531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa638107840 0x7fa63810f3c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.308+0000 7fa640107700 1 --2- 172.21.15.37:0/1466259531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa638102740 0x7fa638102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.308+0000 7fa640107700 1 --2- 172.21.15.37:0/1466259531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6381030f0 0x7fa638107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.308+0000 7fa640107700 1 -- 172.21.15.37:0/1466259531 >> 172.21.15.37:0/1466259531 conn(0x7fa6380795f0 msgr2=0x7fa638079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:14.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.308+0000 7fa640107700 1 -- 172.21.15.37:0/1466259531 shutdown_connections 2024-06-10T00:57:14.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.308+0000 7fa640107700 1 -- 172.21.15.37:0/1466259531 wait complete. 2024-06-10T00:57:14.310 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:14 smithi037 ceph-mon[32192]: pgmap v446: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-06-10T00:57:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.309+0000 7fa640107700 1 Processor -- start 2024-06-10T00:57:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.309+0000 7fa640107700 1 -- start start 2024-06-10T00:57:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.310+0000 7fa640107700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa638102740 0x7fa63819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.310+0000 7fa640107700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6381030f0 0x7fa63819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.310+0000 7fa640107700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa638107840 0x7fa6381989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.310+0000 7fa640107700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa6381127d0 con 0x7fa638107840 2024-06-10T00:57:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.310+0000 7fa640107700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa638112650 con 0x7fa638102740 2024-06-10T00:57:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.310+0000 7fa640107700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa638112950 con 0x7fa6381030f0 2024-06-10T00:57:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.310+0000 7fa63e6a4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa638107840 0x7fa6381989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.310+0000 7fa63e6a4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa638107840 0x7fa6381989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37224/0 (socket says 172.21.15.37:37224) 2024-06-10T00:57:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.310+0000 7fa63e6a4700 1 -- 172.21.15.37:0/115010757 learned_addr learned my addr 172.21.15.37:0/115010757 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.310+0000 7fa63dea3700 1 --2- 172.21.15.37:0/115010757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa638102740 0x7fa63819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:14.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.310+0000 7fa63d6a2700 1 --2- 172.21.15.37:0/115010757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6381030f0 0x7fa63819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:14.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.310+0000 7fa63e6a4700 1 -- 172.21.15.37:0/115010757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6381030f0 msgr2=0x7fa63819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:14.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.310+0000 7fa63e6a4700 1 --2- 172.21.15.37:0/115010757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6381030f0 0x7fa63819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.310+0000 7fa63e6a4700 1 -- 172.21.15.37:0/115010757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa638102740 msgr2=0x7fa63819e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:14.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.310+0000 7fa63e6a4700 1 --2- 172.21.15.37:0/115010757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa638102740 0x7fa63819e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.310+0000 7fa63e6a4700 1 -- 172.21.15.37:0/115010757 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa638199110 con 0x7fa638107840 2024-06-10T00:57:14.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.310+0000 7fa63e6a4700 1 --2- 172.21.15.37:0/115010757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa638107840 0x7fa6381989b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa63000d010 tx=0x7fa6300079a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:14.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.310+0000 7fa62affd700 1 -- 172.21.15.37:0/115010757 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa630003c00 con 0x7fa638107840 2024-06-10T00:57:14.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.311+0000 7fa62affd700 1 -- 172.21.15.37:0/115010757 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa630007e10 con 0x7fa638107840 2024-06-10T00:57:14.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.311+0000 7fa640107700 1 -- 172.21.15.37:0/115010757 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6381993a0 con 0x7fa638107840 2024-06-10T00:57:14.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.311+0000 7fa62affd700 1 -- 172.21.15.37:0/115010757 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63001b8d0 con 0x7fa638107840 2024-06-10T00:57:14.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.311+0000 7fa640107700 1 -- 172.21.15.37:0/115010757 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6381a56c0 con 0x7fa638107840 2024-06-10T00:57:14.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.314+0000 7fa62affd700 1 -- 172.21.15.37:0/115010757 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fa63001ba70 con 0x7fa638107840 2024-06-10T00:57:14.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.314+0000 7fa62affd700 1 --2- 172.21.15.37:0/115010757 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa6240a2cb0 0x7fa6240a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:14.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.314+0000 7fa63dea3700 1 --2- 172.21.15.37:0/115010757 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa6240a2cb0 0x7fa6240a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:14.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.315+0000 7fa62affd700 1 -- 172.21.15.37:0/115010757 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fa63000f020 con 0x7fa638107840 2024-06-10T00:57:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.315+0000 7fa64a98ab80 1 Processor -- start 2024-06-10T00:57:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.315+0000 7fa63dea3700 1 --2- 172.21.15.37:0/115010757 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa6240a2cb0 0x7fa6240a5170 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa638199af0 tx=0x7fa62c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.315+0000 7fa64a98ab80 1 -- start start 2024-06-10T00:57:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.315+0000 7fa64a98ab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69fa06f10 0x55a69fa072f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.315+0000 7fa64a98ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a69fa07830 0x55a69fa01210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.315+0000 7fa64a98ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a69fa01750 0x55a69fa01b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.315+0000 7fa64a98ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a69f8f3f10 con 0x55a69fa07830 2024-06-10T00:57:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.315+0000 7fa64a98ab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a69f952800 con 0x55a69fa06f10 2024-06-10T00:57:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.315+0000 7fa64a98ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a69f8ba410 con 0x55a69fa01750 2024-06-10T00:57:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.315+0000 7fa63d6a2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a69fa07830 0x55a69fa01210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:14.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.316+0000 7fa63d6a2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a69fa07830 0x55a69fa01210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37238/0 (socket says 172.21.15.37:37238) 2024-06-10T00:57:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.316+0000 7fa63d6a2700 1 -- 172.21.15.37:0/1431845355 learned_addr learned my addr 172.21.15.37:0/1431845355 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.316+0000 7fa63dea3700 1 --2- 172.21.15.37:0/1431845355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69fa06f10 0x55a69fa072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.316+0000 7fa63e6a4700 1 --2- 172.21.15.37:0/1431845355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a69fa01750 0x55a69fa01b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.316+0000 7fa63d6a2700 1 -- 172.21.15.37:0/1431845355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a69fa01750 msgr2=0x55a69fa01b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.316+0000 7fa63d6a2700 1 --2- 172.21.15.37:0/1431845355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a69fa01750 0x55a69fa01b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.316+0000 7fa63d6a2700 1 -- 172.21.15.37:0/1431845355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69fa06f10 msgr2=0x55a69fa072f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.316+0000 7fa63d6a2700 1 --2- 172.21.15.37:0/1431845355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69fa06f10 0x55a69fa072f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.316+0000 7fa63d6a2700 1 -- 172.21.15.37:0/1431845355 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a69f8a1880 con 0x55a69fa07830 2024-06-10T00:57:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.316+0000 7fa63d6a2700 1 --2- 172.21.15.37:0/1431845355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a69fa07830 0x55a69fa01210 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa634007c90 tx=0x7fa6340103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621c58425d5cf5f5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.316+0000 7fa623fff700 1 -- 172.21.15.37:0/1431845355 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa634010ca0 con 0x55a69fa07830 2024-06-10T00:57:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.316+0000 7fa623fff700 1 -- 172.21.15.37:0/1431845355 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa634015070 con 0x55a69fa07830 2024-06-10T00:57:14.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.316+0000 7fa623fff700 1 -- 172.21.15.37:0/1431845355 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa634003e10 con 0x55a69fa07830 2024-06-10T00:57:14.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.317+0000 7fa64a98ab80 1 -- 172.21.15.37:0/1431845355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a69fa07830 msgr2=0x55a69fa01210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:14.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.317+0000 7fa64a98ab80 1 --2- 172.21.15.37:0/1431845355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a69fa07830 0x55a69fa01210 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa634007c90 tx=0x7fa6340103d0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.317+0000 7fa64a98ab80 1 -- 172.21.15.37:0/1431845355 shutdown_connections 2024-06-10T00:57:14.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.317+0000 7fa64a98ab80 1 --2- 172.21.15.37:0/1431845355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a69fa01750 0x55a69fa01b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.317+0000 7fa64a98ab80 1 --2- 172.21.15.37:0/1431845355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69fa06f10 0x55a69fa072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.317+0000 7fa64a98ab80 1 --2- 172.21.15.37:0/1431845355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a69fa07830 0x55a69fa01210 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.317+0000 7fa64a98ab80 1 -- 172.21.15.37:0/1431845355 >> 172.21.15.37:0/1431845355 conn(0x55a69f92e030 msgr2=0x55a69f9fc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:14.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.317+0000 7fa64a98ab80 1 -- 172.21.15.37:0/1431845355 shutdown_connections 2024-06-10T00:57:14.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.317+0000 7fa64a98ab80 1 -- 172.21.15.37:0/1431845355 wait complete. 2024-06-10T00:57:14.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.318+0000 7fa64a98ab80 1 -- start start 2024-06-10T00:57:14.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.318+0000 7fa64a98ab80 1 Processor -- start 2024-06-10T00:57:14.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.318+0000 7fa64a98ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a69fa01750 0x55a69fa0d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:14.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.318+0000 7fa64a98ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a69fa06f10 0x55a69fa0a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:14.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.318+0000 7fa64a98ab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69fa07830 0x55a69fa0ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:14.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.318+0000 7fa64a98ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a69f8f2f70 con 0x55a69fa01750 2024-06-10T00:57:14.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.318+0000 7fa64a98ab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a69f94e7c0 con 0x55a69fa07830 2024-06-10T00:57:14.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.318+0000 7fa64a98ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a69fa0e210 con 0x55a69fa06f10 2024-06-10T00:57:14.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.318+0000 7fa64a98ab80 10 client.0 _ll_register_callbacks cb 0x55a69f8e8560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:14.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.318+0000 7fa63dea3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a69fa01750 0x55a69fa0d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:14.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.318+0000 7fa63dea3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a69fa01750 0x55a69fa0d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37244/0 (socket says 172.21.15.37:37244) 2024-06-10T00:57:14.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.318+0000 7fa63dea3700 1 -- 172.21.15.37:0/696551587 learned_addr learned my addr 172.21.15.37:0/696551587 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:14.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.318+0000 7fa63e6a4700 1 --2- 172.21.15.37:0/696551587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69fa07830 0x55a69fa0ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:14.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.318+0000 7fa63d6a2700 1 --2- 172.21.15.37:0/696551587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a69fa06f10 0x55a69fa0a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:14.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.319+0000 7fa63dea3700 1 -- 172.21.15.37:0/696551587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a69fa06f10 msgr2=0x55a69fa0a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:14.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.319+0000 7fa63dea3700 1 --2- 172.21.15.37:0/696551587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a69fa06f10 0x55a69fa0a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.319+0000 7fa63dea3700 1 -- 172.21.15.37:0/696551587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69fa07830 msgr2=0x55a69fa0ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:14.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.319+0000 7fa63dea3700 1 --2- 172.21.15.37:0/696551587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69fa07830 0x55a69fa0ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.319+0000 7fa63dea3700 1 -- 172.21.15.37:0/696551587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a69f918700 con 0x55a69fa01750 2024-06-10T00:57:14.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.319+0000 7fa63dea3700 1 --2- 172.21.15.37:0/696551587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a69fa01750 0x55a69fa0d5a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa62c004600 tx=0x7fa62c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=355785e1b4d01358 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:14.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.319+0000 7fa6217fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:57:14.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.319+0000 7fa6217fa700 1 -- 172.21.15.37:0/696551587 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c01e830 con 0x55a69fa01750 2024-06-10T00:57:14.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.319+0000 7fa6217fa700 1 -- 172.21.15.37:0/696551587 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fa62c01e9d0 con 0x55a69fa01750 2024-06-10T00:57:14.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.319+0000 7fa64a98ab80 1 -- 172.21.15.37:0/696551587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa634015070 con 0x55a69fa01750 2024-06-10T00:57:14.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.319+0000 7fa6217fa700 1 -- 172.21.15.37:0/696551587 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa62c031050 con 0x55a69fa01750 2024-06-10T00:57:14.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.319+0000 7fa64a98ab80 10 client.15423 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:14.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.319+0000 7fa64a98ab80 1 -- 172.21.15.37:0/696551587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a69fa189f0 con 0x55a69fa01750 2024-06-10T00:57:14.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.319+0000 7fa6217fa700 1 -- 172.21.15.37:0/696551587 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa634015070 con 0x55a69fa01750 2024-06-10T00:57:14.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.320+0000 7fa6217fa700 1 -- 172.21.15.37:0/696551587 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa62c026020 con 0x55a69fa01750 2024-06-10T00:57:14.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.320+0000 7fa64a98ab80 10 client.15423 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:14.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.320+0000 7fa64a98ab80 10 client.15423 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-10T00:57:14.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.320+0000 7fa64a98ab80 20 client.15423 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:14.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.320+0000 7fa64a98ab80 1 --2- 172.21.15.37:0/696551587 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55a69fa1b040 0x55a69fa1c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:14.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.320+0000 7fa64a98ab80 4 client.15423 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:57:14.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.320+0000 7fa64a98ab80 1 -- 172.21.15.37:0/696551587 --> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a69f95fd00 con 0x55a69fa1b040 2024-06-10T00:57:14.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.320+0000 7fa63d6a2700 1 --2- 172.21.15.37:0/696551587 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55a69fa1b040 0x55a69fa1c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:14.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.321+0000 7fa63d6a2700 1 --2- 172.21.15.37:0/696551587 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55a69fa1b040 0x55a69fa1c4b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c86e63406cb6103a server_cookie=a21d363e4f361b66 in_seq=0 out_seq=0 2024-06-10T00:57:14.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.321+0000 7fa6217fa700 10 client.15423 ms_handle_connect on v2:172.21.15.167:6836/2720148065 2024-06-10T00:57:14.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.321+0000 7fa6217fa700 1 -- 172.21.15.37:0/696551587 <== mds.2 v2:172.21.15.167:6836/2720148065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a69f95fd00 con 0x55a69fa1b040 2024-06-10T00:57:14.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.321+0000 7fa6217fa700 10 client.15423 handle_command_reply: tid=0 2024-06-10T00:57:14.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.321+0000 7fa64a98ab80 1 client.15423 shutdown 2024-06-10T00:57:14.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.322+0000 7fa64a98ab80 1 -- 172.21.15.37:0/696551587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a69fa01750 msgr2=0x55a69fa0d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:14.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.322+0000 7fa64a98ab80 1 --2- 172.21.15.37:0/696551587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a69fa01750 0x55a69fa0d5a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa62c004600 tx=0x7fa62c0047c0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.322+0000 7fa64a98ab80 1 -- 172.21.15.37:0/696551587 shutdown_connections 2024-06-10T00:57:14.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.322+0000 7fa64a98ab80 1 --2- 172.21.15.37:0/696551587 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55a69fa1b040 0x55a69fa1c4b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.322+0000 7fa64a98ab80 1 --2- 172.21.15.37:0/696551587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69fa07830 0x55a69fa0ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.322+0000 7fa64a98ab80 1 --2- 172.21.15.37:0/696551587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a69fa01750 0x55a69fa0d5a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.322+0000 7fa6217fa700 0 client.15423 ms_handle_reset on v2:172.21.15.167:6836/2720148065 2024-06-10T00:57:14.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.322+0000 7fa64a98ab80 1 --2- 172.21.15.37:0/696551587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a69fa06f10 0x55a69fa0a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.322+0000 7fa64a98ab80 1 -- 172.21.15.37:0/696551587 >> 172.21.15.37:0/696551587 conn(0x55a69f92e030 msgr2=0x55a69f9fb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:14.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.322+0000 7fa64a98ab80 1 -- 172.21.15.37:0/696551587 shutdown_connections 2024-06-10T00:57:14.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.322+0000 7fa64a98ab80 1 -- 172.21.15.37:0/696551587 wait complete. 2024-06-10T00:57:14.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.322+0000 7fa64a98ab80 20 client.15423 trim_cache size 0 max 16384 2024-06-10T00:57:14.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.344+0000 7fa64a98ab80 1 Processor -- start 2024-06-10T00:57:14.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.344+0000 7fa64a98ab80 1 -- start start 2024-06-10T00:57:14.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.344+0000 7fa64a98ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a69fa01750 0x55a69f9fbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:14.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.344+0000 7fa64a98ab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69fa06f10 0x55a69f9fc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:14.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.344+0000 7fa64a98ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a69fa07830 0x55a69fa07c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:14.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.344+0000 7fa64a98ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a69fa1a7f0 con 0x55a69fa01750 2024-06-10T00:57:14.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.344+0000 7fa64a98ab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a69fa1aec0 con 0x55a69fa06f10 2024-06-10T00:57:14.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.344+0000 7fa64a98ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a69fa1b590 con 0x55a69fa07830 2024-06-10T00:57:14.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.344+0000 7fa63dea3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a69fa01750 0x55a69f9fbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:14.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.344+0000 7fa63dea3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a69fa01750 0x55a69f9fbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37252/0 (socket says 172.21.15.37:37252) 2024-06-10T00:57:14.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.344+0000 7fa63dea3700 1 -- 172.21.15.37:0/4172286707 learned_addr learned my addr 172.21.15.37:0/4172286707 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:14.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.345+0000 7fa63d6a2700 1 --2- 172.21.15.37:0/4172286707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69fa06f10 0x55a69f9fc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:14.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.345+0000 7fa63e6a4700 1 --2- 172.21.15.37:0/4172286707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a69fa07830 0x55a69fa07c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:14.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.345+0000 7fa63dea3700 1 -- 172.21.15.37:0/4172286707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a69fa07830 msgr2=0x55a69fa07c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:14.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.345+0000 7fa63dea3700 1 --2- 172.21.15.37:0/4172286707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a69fa07830 0x55a69fa07c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.345+0000 7fa63dea3700 1 -- 172.21.15.37:0/4172286707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69fa06f10 msgr2=0x55a69f9fc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:14.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.345+0000 7fa63dea3700 1 --2- 172.21.15.37:0/4172286707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69fa06f10 0x55a69f9fc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.345+0000 7fa63dea3700 1 -- 172.21.15.37:0/4172286707 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a69fa0e210 con 0x55a69fa01750 2024-06-10T00:57:14.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.345+0000 7fa63d6a2700 1 --2- 172.21.15.37:0/4172286707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69fa06f10 0x55a69f9fc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:57:14.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.345+0000 7fa63dea3700 1 --2- 172.21.15.37:0/4172286707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a69fa01750 0x55a69f9fbe30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa62c0046b0 tx=0x7fa62c0046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f763e7e0a28cf9f0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:14.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.345+0000 7fa6237fe700 1 -- 172.21.15.37:0/4172286707 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c011a40 con 0x55a69fa01750 2024-06-10T00:57:14.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.345+0000 7fa6237fe700 1 -- 172.21.15.37:0/4172286707 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa62c003cf0 con 0x55a69fa01750 2024-06-10T00:57:14.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.345+0000 7fa6237fe700 1 -- 172.21.15.37:0/4172286707 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c00edb0 con 0x55a69fa01750 2024-06-10T00:57:14.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.346+0000 7fa64a98ab80 1 -- 172.21.15.37:0/4172286707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a69fa01750 msgr2=0x55a69f9fbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:14.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.346+0000 7fa64a98ab80 1 --2- 172.21.15.37:0/4172286707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a69fa01750 0x55a69f9fbe30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa62c0046b0 tx=0x7fa62c0046e0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.346+0000 7fa64a98ab80 1 -- 172.21.15.37:0/4172286707 shutdown_connections 2024-06-10T00:57:14.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.346+0000 7fa64a98ab80 1 --2- 172.21.15.37:0/4172286707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a69fa07830 0x55a69fa07c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.346+0000 7fa64a98ab80 1 --2- 172.21.15.37:0/4172286707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a69fa01750 0x55a69f9fbe30 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.346+0000 7fa64a98ab80 1 --2- 172.21.15.37:0/4172286707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69fa06f10 0x55a69f9fc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.346+0000 7fa64a98ab80 1 -- 172.21.15.37:0/4172286707 >> 172.21.15.37:0/4172286707 conn(0x55a69f92e030 msgr2=0x55a69fa09190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:14.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.346+0000 7fa64a98ab80 1 -- 172.21.15.37:0/4172286707 shutdown_connections 2024-06-10T00:57:14.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.346+0000 7fa64a98ab80 1 -- 172.21.15.37:0/4172286707 wait complete. 2024-06-10T00:57:14.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.346+0000 7fa64a98ab80 1 -- start start 2024-06-10T00:57:14.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.346+0000 7fa64a98ab80 1 Processor -- start 2024-06-10T00:57:14.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.347+0000 7fa64a98ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a69fa01750 0x55a69fa25930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:14.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.347+0000 7fa64a98ab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69fa06f10 0x55a69fa25e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:14.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.347+0000 7fa64a98ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a69fa07830 0x55a69fa208e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:14.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.347+0000 7fa64a98ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a69fa1c330 con 0x55a69fa01750 2024-06-10T00:57:14.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.347+0000 7fa64a98ab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a69fa1bc60 con 0x55a69fa06f10 2024-06-10T00:57:14.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.347+0000 7fa64a98ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a69fa1ca00 con 0x55a69fa07830 2024-06-10T00:57:14.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.347+0000 7fa64a98ab80 10 client.0 _ll_register_callbacks cb 0x55a69f8e8560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:14.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.347+0000 7fa63dea3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a69fa01750 0x55a69fa25930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:14.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.347+0000 7fa63dea3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a69fa01750 0x55a69fa25930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37266/0 (socket says 172.21.15.37:37266) 2024-06-10T00:57:14.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.347+0000 7fa63dea3700 1 -- 172.21.15.37:0/2622358114 learned_addr learned my addr 172.21.15.37:0/2622358114 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:14.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.347+0000 7fa63d6a2700 1 --2- 172.21.15.37:0/2622358114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69fa06f10 0x55a69fa25e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:14.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.347+0000 7fa63e6a4700 1 --2- 172.21.15.37:0/2622358114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a69fa07830 0x55a69fa208e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:14.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.347+0000 7fa63dea3700 1 -- 172.21.15.37:0/2622358114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a69fa07830 msgr2=0x55a69fa208e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:14.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.347+0000 7fa63dea3700 1 --2- 172.21.15.37:0/2622358114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a69fa07830 0x55a69fa208e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.347+0000 7fa63dea3700 1 -- 172.21.15.37:0/2622358114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69fa06f10 msgr2=0x55a69fa25e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:14.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.347+0000 7fa63dea3700 1 --2- 172.21.15.37:0/2622358114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69fa06f10 0x55a69fa25e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.347+0000 7fa63dea3700 1 -- 172.21.15.37:0/2622358114 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a69fa26590 con 0x55a69fa01750 2024-06-10T00:57:14.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.347+0000 7fa63dea3700 1 --2- 172.21.15.37:0/2622358114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a69fa01750 0x55a69fa25930 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa62c016040 tx=0x7fa62c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5198f4eedbca762 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:14.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.347+0000 7fa6217fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:57:14.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.347+0000 7fa6217fa700 1 -- 172.21.15.37:0/2622358114 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c03b070 con 0x55a69fa01750 2024-06-10T00:57:14.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.348+0000 7fa6217fa700 1 -- 172.21.15.37:0/2622358114 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fa62c01a7d0 con 0x55a69fa01750 2024-06-10T00:57:14.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.348+0000 7fa64a98ab80 1 -- 172.21.15.37:0/2622358114 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa62c003cf0 con 0x55a69fa01750 2024-06-10T00:57:14.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.348+0000 7fa6217fa700 1 -- 172.21.15.37:0/2622358114 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa62c047050 con 0x55a69fa01750 2024-06-10T00:57:14.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.348+0000 7fa64a98ab80 10 client.15429 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:14.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.348+0000 7fa64a98ab80 1 -- 172.21.15.37:0/2622358114 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a69fa86f30 con 0x55a69fa01750 2024-06-10T00:57:14.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.348+0000 7fa6217fa700 1 -- 172.21.15.37:0/2622358114 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa62c003cf0 con 0x55a69fa01750 2024-06-10T00:57:14.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.348+0000 7fa6217fa700 1 -- 172.21.15.37:0/2622358114 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa62c034030 con 0x55a69fa01750 2024-06-10T00:57:14.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.348+0000 7fa64a98ab80 10 client.15429 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:14.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.348+0000 7fa64a98ab80 10 client.15429 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-10T00:57:14.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.348+0000 7fa64a98ab80 20 client.15429 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:14.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.348+0000 7fa64a98ab80 1 --2- 172.21.15.37:0/2622358114 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55a69fa8b6a0 0x55a69fa8a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:14.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.348+0000 7fa64a98ab80 4 client.15429 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T00:57:14.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.348+0000 7fa64a98ab80 1 -- 172.21.15.37:0/2622358114 --> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] -- command(tid 0: {"prefix": "status"}) -- 0x55a69f95a280 con 0x55a69fa8b6a0 2024-06-10T00:57:14.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.349+0000 7fa63d6a2700 1 --2- 172.21.15.37:0/2622358114 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55a69fa8b6a0 0x55a69fa8a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:14.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.349+0000 7fa6217fa700 10 client.15429 ms_handle_connect on v2:172.21.15.167:6836/2720148065 2024-06-10T00:57:14.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.349+0000 7fa63d6a2700 1 --2- 172.21.15.37:0/2622358114 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55a69fa8b6a0 0x55a69fa8a3b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=704a9ec224d0e581 server_cookie=ee3df51692592fc in_seq=0 out_seq=0 2024-06-10T00:57:14.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.349+0000 7fa6217fa700 1 -- 172.21.15.37:0/2622358114 <== mds.2 v2:172.21.15.167:6836/2720148065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+329 (crc 0 0 0) 0x55a69f95a280 con 0x55a69fa8b6a0 2024-06-10T00:57:14.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.349+0000 7fa6217fa700 10 client.15429 handle_command_reply: tid=0 2024-06-10T00:57:14.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.349+0000 7fa64a98ab80 1 client.15429 shutdown 2024-06-10T00:57:14.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.349+0000 7fa64a98ab80 1 -- 172.21.15.37:0/2622358114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a69fa01750 msgr2=0x55a69fa25930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:14.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.349+0000 7fa64a98ab80 1 --2- 172.21.15.37:0/2622358114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a69fa01750 0x55a69fa25930 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa62c016040 tx=0x7fa62c00ece0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.349+0000 7fa64a98ab80 1 -- 172.21.15.37:0/2622358114 shutdown_connections 2024-06-10T00:57:14.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.349+0000 7fa64a98ab80 1 --2- 172.21.15.37:0/2622358114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a69fa07830 0x55a69fa208e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.349+0000 7fa64a98ab80 1 --2- 172.21.15.37:0/2622358114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a69fa01750 0x55a69fa25930 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.349+0000 7fa64a98ab80 1 --2- 172.21.15.37:0/2622358114 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55a69fa8b6a0 0x55a69fa8a3b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.349+0000 7fa64a98ab80 1 --2- 172.21.15.37:0/2622358114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a69fa06f10 0x55a69fa25e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.349+0000 7fa6217fa700 0 client.15429 ms_handle_reset on v2:172.21.15.167:6836/2720148065 2024-06-10T00:57:14.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.349+0000 7fa64a98ab80 1 -- 172.21.15.37:0/2622358114 >> 172.21.15.37:0/2622358114 conn(0x55a69f92e030 msgr2=0x55a69fa07c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:14.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.350+0000 7fa64a98ab80 1 -- 172.21.15.37:0/2622358114 shutdown_connections 2024-06-10T00:57:14.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.350+0000 7fa64a98ab80 1 -- 172.21.15.37:0/2622358114 wait complete. 2024-06-10T00:57:14.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.350+0000 7fa64a98ab80 20 client.15429 trim_cache size 0 max 16384 2024-06-10T00:57:14.355 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T00:57:14.355 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "68793514-26c2-11ef-bc9b-c7b262605968", 2024-06-10T00:57:14.355 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 2, 2024-06-10T00:57:14.355 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24451, 2024-06-10T00:57:14.355 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-06-10T00:57:14.355 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-06-10T00:57:14.356 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-06-10T00:57:14.356 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 215.070258362, 2024-06-10T00:57:14.356 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 27, 2024-06-10T00:57:14.356 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 100, 2024-06-10T00:57:14.356 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 100, 2024-06-10T00:57:14.356 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 309.84929934299998 2024-06-10T00:57:14.356 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T00:57:14.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.350+0000 7fa640107700 1 -- 172.21.15.37:0/115010757 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa6240a2cb0 msgr2=0x7fa6240a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:14.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.350+0000 7fa640107700 1 --2- 172.21.15.37:0/115010757 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa6240a2cb0 0x7fa6240a5170 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa638199af0 tx=0x7fa62c009260 comp rx=0 tx=0).stop 2024-06-10T00:57:14.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.350+0000 7fa640107700 1 -- 172.21.15.37:0/115010757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa638107840 msgr2=0x7fa6381989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:14.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.350+0000 7fa640107700 1 --2- 172.21.15.37:0/115010757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa638107840 0x7fa6381989b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa63000d010 tx=0x7fa6300079a0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.350+0000 7fa640107700 1 -- 172.21.15.37:0/115010757 shutdown_connections 2024-06-10T00:57:14.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.350+0000 7fa640107700 1 --2- 172.21.15.37:0/115010757 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa6240a2cb0 0x7fa6240a5170 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.350+0000 7fa640107700 1 --2- 172.21.15.37:0/115010757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa638107840 0x7fa6381989b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.350+0000 7fa640107700 1 --2- 172.21.15.37:0/115010757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa638102740 0x7fa63819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.350+0000 7fa640107700 1 --2- 172.21.15.37:0/115010757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6381030f0 0x7fa63819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:14.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.350+0000 7fa640107700 1 -- 172.21.15.37:0/115010757 >> 172.21.15.37:0/115010757 conn(0x7fa6380795f0 msgr2=0x7fa6381005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:14.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.350+0000 7fa640107700 1 -- 172.21.15.37:0/115010757 shutdown_connections 2024-06-10T00:57:14.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:14.350+0000 7fa640107700 1 -- 172.21.15.37:0/115010757 wait complete. 2024-06-10T00:57:14.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:14 smithi094 ceph-mon[36708]: pgmap v446: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-06-10T00:57:14.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:14 smithi167 ceph-mon[36917]: pgmap v446: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-06-10T00:57:15.069 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-10T00:57:15.326 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:57:16.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:16 smithi094 ceph-mon[36708]: pgmap v447: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-06-10T00:57:16.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:16 smithi037 ceph-mon[32192]: pgmap v447: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-06-10T00:57:16.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:16 smithi167 ceph-mon[36917]: pgmap v447: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-06-10T00:57:17.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.138+0000 7f5efe7b7700 1 -- 172.21.15.37:0/986006815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ef8103120 msgr2=0x7f5ef8107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:17.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.138+0000 7f5efe7b7700 1 --2- 172.21.15.37:0/986006815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ef8103120 0x7f5ef8107130 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00099c0 tx=0x7f5ee0009cd0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.138+0000 7f5efe7b7700 1 -- 172.21.15.37:0/986006815 shutdown_connections 2024-06-10T00:57:17.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.138+0000 7f5efe7b7700 1 --2- 172.21.15.37:0/986006815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ef8107860 0x7f5ef810f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.138+0000 7f5efe7b7700 1 --2- 172.21.15.37:0/986006815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ef8102770 0x7f5ef8102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.138+0000 7f5efe7b7700 1 --2- 172.21.15.37:0/986006815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ef8103120 0x7f5ef8107130 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.138+0000 7f5efe7b7700 1 -- 172.21.15.37:0/986006815 >> 172.21.15.37:0/986006815 conn(0x7f5ef8079600 msgr2=0x7f5ef8079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:17.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.138+0000 7f5efe7b7700 1 -- 172.21.15.37:0/986006815 shutdown_connections 2024-06-10T00:57:17.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.138+0000 7f5efe7b7700 1 -- 172.21.15.37:0/986006815 wait complete. 2024-06-10T00:57:17.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.139+0000 7f5efe7b7700 1 Processor -- start 2024-06-10T00:57:17.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.139+0000 7f5efe7b7700 1 -- start start 2024-06-10T00:57:17.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.139+0000 7f5efe7b7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ef8102770 0x7f5ef8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:17.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.139+0000 7f5efe7b7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ef8103120 0x7f5ef8074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:17.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.139+0000 7f5efe7b7700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ef8107860 0x7f5ef8075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:17.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.139+0000 7f5efe7b7700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef8112770 con 0x7f5ef8102770 2024-06-10T00:57:17.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.139+0000 7f5efe7b7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef81125f0 con 0x7f5ef8103120 2024-06-10T00:57:17.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.139+0000 7f5efe7b7700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef81128f0 con 0x7f5ef8107860 2024-06-10T00:57:17.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.139+0000 7f5ef7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ef8102770 0x7f5ef8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:17.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.139+0000 7f5ef7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ef8102770 0x7f5ef8074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37292/0 (socket says 172.21.15.37:37292) 2024-06-10T00:57:17.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.139+0000 7f5ef7fff700 1 -- 172.21.15.37:0/3486145242 learned_addr learned my addr 172.21.15.37:0/3486145242 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:17.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.139+0000 7f5eeffff700 1 --2- 172.21.15.37:0/3486145242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ef8103120 0x7f5ef8074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:17.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.139+0000 7f5efcd54700 1 --2- 172.21.15.37:0/3486145242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ef8107860 0x7f5ef8075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:17.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.139+0000 7f5ef7fff700 1 -- 172.21.15.37:0/3486145242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ef8107860 msgr2=0x7f5ef8075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:17.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.139+0000 7f5ef7fff700 1 --2- 172.21.15.37:0/3486145242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ef8107860 0x7f5ef8075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.139+0000 7f5ef7fff700 1 -- 172.21.15.37:0/3486145242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ef8103120 msgr2=0x7f5ef8074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:17.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.139+0000 7f5ef7fff700 1 --2- 172.21.15.37:0/3486145242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ef8103120 0x7f5ef8074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.139+0000 7f5ef7fff700 1 -- 172.21.15.37:0/3486145242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef81a70a0 con 0x7f5ef8102770 2024-06-10T00:57:17.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.139+0000 7f5eeffff700 1 --2- 172.21.15.37:0/3486145242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ef8103120 0x7f5ef8074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:57:17.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.140+0000 7f5ef7fff700 1 --2- 172.21.15.37:0/3486145242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ef8102770 0x7f5ef8074790 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800c9b0 tx=0x7f5ee800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:17.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.140+0000 7f5ef5ffb700 1 -- 172.21.15.37:0/3486145242 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8007b00 con 0x7f5ef8102770 2024-06-10T00:57:17.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.140+0000 7f5ef5ffb700 1 -- 172.21.15.37:0/3486145242 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ee8007ca0 con 0x7f5ef8102770 2024-06-10T00:57:17.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.140+0000 7f5efe7b7700 1 -- 172.21.15.37:0/3486145242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef81a7390 con 0x7f5ef8102770 2024-06-10T00:57:17.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.140+0000 7f5ef5ffb700 1 -- 172.21.15.37:0/3486145242 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8018780 con 0x7f5ef8102770 2024-06-10T00:57:17.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.140+0000 7f5efe7b7700 1 -- 172.21.15.37:0/3486145242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef81a78d0 con 0x7f5ef8102770 2024-06-10T00:57:17.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.142+0000 7f5ef5ffb700 1 -- 172.21.15.37:0/3486145242 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f5ee800f450 con 0x7f5ef8102770 2024-06-10T00:57:17.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.143+0000 7f5ef5ffb700 1 --2- 172.21.15.37:0/3486145242 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5ee40a2cb0 0x7f5ee40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:17.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.143+0000 7f5ef5ffb700 1 -- 172.21.15.37:0/3486145242 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f5ee80cf630 con 0x7f5ef8102770 2024-06-10T00:57:17.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.143+0000 7f5eeffff700 1 --2- 172.21.15.37:0/3486145242 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5ee40a2cb0 0x7f5ee40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:17.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.144+0000 7f5eeffff700 1 --2- 172.21.15.37:0/3486145242 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5ee40a2cb0 0x7f5ee40a5170 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8108c40 tx=0x7f5ee0005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:17.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.144+0000 7f5f0903ab80 1 Processor -- start 2024-06-10T00:57:17.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.144+0000 7f5f0903ab80 1 -- start start 2024-06-10T00:57:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.144+0000 7f5f0903ab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556be9c9e5f0 0x556be9c9e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.144+0000 7f5f0903ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556be9c9ef10 0x556be9c98990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.144+0000 7f5f0903ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556be9c98ed0 0x556be9c992f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.144+0000 7f5f0903ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556be9bea7c0 con 0x556be9c98ed0 2024-06-10T00:57:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.144+0000 7f5f0903ab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556be9b926d0 con 0x556be9c9e5f0 2024-06-10T00:57:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.144+0000 7f5f0903ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556be9be6780 con 0x556be9c9ef10 2024-06-10T00:57:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.144+0000 7f5efcd54700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556be9c98ed0 0x556be9c992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.144+0000 7f5efcd54700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556be9c98ed0 0x556be9c992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37306/0 (socket says 172.21.15.37:37306) 2024-06-10T00:57:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.144+0000 7f5efcd54700 1 -- 172.21.15.37:0/2694157370 learned_addr learned my addr 172.21.15.37:0/2694157370 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.145+0000 7f5ef7fff700 1 --2- 172.21.15.37:0/2694157370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556be9c9e5f0 0x556be9c9e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.145+0000 7f5eeffff700 1 --2- 172.21.15.37:0/2694157370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556be9c9ef10 0x556be9c98990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.145+0000 7f5efcd54700 1 -- 172.21.15.37:0/2694157370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556be9c9ef10 msgr2=0x556be9c98990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.145+0000 7f5efcd54700 1 --2- 172.21.15.37:0/2694157370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556be9c9ef10 0x556be9c98990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.145+0000 7f5efcd54700 1 -- 172.21.15.37:0/2694157370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556be9c9e5f0 msgr2=0x556be9c9e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.145+0000 7f5efcd54700 1 --2- 172.21.15.37:0/2694157370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556be9c9e5f0 0x556be9c9e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.145+0000 7f5efcd54700 1 -- 172.21.15.37:0/2694157370 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556be9b88f00 con 0x556be9c98ed0 2024-06-10T00:57:17.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.145+0000 7f5ef7fff700 1 --2- 172.21.15.37:0/2694157370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556be9c9e5f0 0x556be9c9e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:57:17.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.145+0000 7f5efcd54700 1 --2- 172.21.15.37:0/2694157370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556be9c98ed0 0x556be9c992f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0009d20 tx=0x7f5ef00077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404d06aa00b040f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:17.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.145+0000 7f5eee7fc700 1 -- 172.21.15.37:0/2694157370 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0004bb0 con 0x556be9c98ed0 2024-06-10T00:57:17.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.145+0000 7f5eee7fc700 1 -- 172.21.15.37:0/2694157370 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ef0004d50 con 0x556be9c98ed0 2024-06-10T00:57:17.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.145+0000 7f5eee7fc700 1 -- 172.21.15.37:0/2694157370 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0014670 con 0x556be9c98ed0 2024-06-10T00:57:17.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.146+0000 7f5f0903ab80 1 -- 172.21.15.37:0/2694157370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556be9c98ed0 msgr2=0x556be9c992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:17.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.146+0000 7f5f0903ab80 1 --2- 172.21.15.37:0/2694157370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556be9c98ed0 0x556be9c992f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0009d20 tx=0x7f5ef00077e0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.146+0000 7f5f0903ab80 1 -- 172.21.15.37:0/2694157370 shutdown_connections 2024-06-10T00:57:17.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.146+0000 7f5f0903ab80 1 --2- 172.21.15.37:0/2694157370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556be9c98ed0 0x556be9c992f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.146+0000 7f5f0903ab80 1 --2- 172.21.15.37:0/2694157370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556be9c9e5f0 0x556be9c9e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.146+0000 7f5f0903ab80 1 --2- 172.21.15.37:0/2694157370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556be9c9ef10 0x556be9c98990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.146+0000 7f5f0903ab80 1 -- 172.21.15.37:0/2694157370 >> 172.21.15.37:0/2694157370 conn(0x556be9bc3c70 msgr2=0x556be9c93800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:17.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.146+0000 7f5f0903ab80 1 -- 172.21.15.37:0/2694157370 shutdown_connections 2024-06-10T00:57:17.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.146+0000 7f5f0903ab80 1 -- 172.21.15.37:0/2694157370 wait complete. 2024-06-10T00:57:17.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.146+0000 7f5f0903ab80 1 -- start start 2024-06-10T00:57:17.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.146+0000 7f5f0903ab80 1 Processor -- start 2024-06-10T00:57:17.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.147+0000 7f5f0903ab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556be9c98ed0 0x556be9ca4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:17.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.147+0000 7f5f0903ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556be9c9e5f0 0x556be9ca1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:17.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.147+0000 7f5f0903ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556be9c9ef10 0x556be9ca2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:17.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.147+0000 7f5f0903ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556be9ca5980 con 0x556be9c9e5f0 2024-06-10T00:57:17.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.147+0000 7f5f0903ab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556be9b8aba0 con 0x556be9c98ed0 2024-06-10T00:57:17.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.147+0000 7f5f0903ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556be9ca5b00 con 0x556be9c9ef10 2024-06-10T00:57:17.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.147+0000 7f5f0903ab80 10 client.0 _ll_register_callbacks cb 0x556be9b848d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:17.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.147+0000 7f5eeffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556be9c9e5f0 0x556be9ca1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:17.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.147+0000 7f5eeffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556be9c9e5f0 0x556be9ca1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37318/0 (socket says 172.21.15.37:37318) 2024-06-10T00:57:17.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.147+0000 7f5eeffff700 1 -- 172.21.15.37:0/148909556 learned_addr learned my addr 172.21.15.37:0/148909556 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:17.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.147+0000 7f5ef7fff700 1 --2- 172.21.15.37:0/148909556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556be9c98ed0 0x556be9ca4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:17.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.147+0000 7f5efcd54700 1 --2- 172.21.15.37:0/148909556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556be9c9ef10 0x556be9ca2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:17.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.147+0000 7f5eeffff700 1 -- 172.21.15.37:0/148909556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556be9c9ef10 msgr2=0x556be9ca2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:17.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.147+0000 7f5eeffff700 1 --2- 172.21.15.37:0/148909556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556be9c9ef10 0x556be9ca2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.147+0000 7f5eeffff700 1 -- 172.21.15.37:0/148909556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556be9c98ed0 msgr2=0x556be9ca4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:17.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.147+0000 7f5eeffff700 1 --2- 172.21.15.37:0/148909556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556be9c98ed0 0x556be9ca4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.147+0000 7f5eeffff700 1 -- 172.21.15.37:0/148909556 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556be9b39880 con 0x556be9c9e5f0 2024-06-10T00:57:17.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.148+0000 7f5eeffff700 1 --2- 172.21.15.37:0/148909556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556be9c9e5f0 0x556be9ca1f50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0015000 tx=0x7f5ee0009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7743288dda8360 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:17.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.148+0000 7f5ed77fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:57:17.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.148+0000 7f5ed77fe700 1 -- 172.21.15.37:0/148909556 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0003d70 con 0x556be9c9e5f0 2024-06-10T00:57:17.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.148+0000 7f5ed77fe700 1 -- 172.21.15.37:0/148909556 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f5ee000f870 con 0x556be9c9e5f0 2024-06-10T00:57:17.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.148+0000 7f5f0903ab80 1 -- 172.21.15.37:0/148909556 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ef0004d50 con 0x556be9c9e5f0 2024-06-10T00:57:17.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.148+0000 7f5ed77fe700 1 -- 172.21.15.37:0/148909556 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5ee000fa70 con 0x556be9c9e5f0 2024-06-10T00:57:17.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.148+0000 7f5f0903ab80 10 client.15441 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:17.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.148+0000 7f5f0903ab80 1 -- 172.21.15.37:0/148909556 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556be9cb0260 con 0x556be9c9e5f0 2024-06-10T00:57:17.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.148+0000 7f5ed77fe700 1 -- 172.21.15.37:0/148909556 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ef0004d50 con 0x556be9c9e5f0 2024-06-10T00:57:17.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.148+0000 7f5ed77fe700 1 -- 172.21.15.37:0/148909556 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5ee0024410 con 0x556be9c9e5f0 2024-06-10T00:57:17.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.148+0000 7f5f0903ab80 10 client.15441 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:17.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.148+0000 7f5f0903ab80 10 client.15441 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-10T00:57:17.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.148+0000 7f5f0903ab80 20 client.15441 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:17.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.148+0000 7f5f0903ab80 1 --2- 172.21.15.37:0/148909556 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x556be9cb2900 0x556be9cb3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:17.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.148+0000 7f5f0903ab80 4 client.15441 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:57:17.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.148+0000 7f5f0903ab80 1 -- 172.21.15.37:0/148909556 --> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556be9bf7cc0 con 0x556be9cb2900 2024-06-10T00:57:17.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.149+0000 7f5ef7fff700 1 --2- 172.21.15.37:0/148909556 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x556be9cb2900 0x556be9cb3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:17.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.149+0000 7f5ef7fff700 1 --2- 172.21.15.37:0/148909556 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x556be9cb2900 0x556be9cb3d70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2fad247961b76bd3 server_cookie=6f3df6e5dd42cad6 in_seq=0 out_seq=0 2024-06-10T00:57:17.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.149+0000 7f5ed77fe700 10 client.15441 ms_handle_connect on v2:172.21.15.94:6838/946269331 2024-06-10T00:57:17.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.150+0000 7f5ed77fe700 1 -- 172.21.15.37:0/148909556 <== mds.1 v2:172.21.15.94:6838/946269331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556be9bf7cc0 con 0x556be9cb2900 2024-06-10T00:57:17.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.150+0000 7f5ed77fe700 10 client.15441 handle_command_reply: tid=0 2024-06-10T00:57:17.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.150+0000 7f5f0903ab80 1 client.15441 shutdown 2024-06-10T00:57:17.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.150+0000 7f5f0903ab80 1 -- 172.21.15.37:0/148909556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556be9c9e5f0 msgr2=0x556be9ca1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:17.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.150+0000 7f5f0903ab80 1 --2- 172.21.15.37:0/148909556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556be9c9e5f0 0x556be9ca1f50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0015000 tx=0x7f5ee0009cd0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.150+0000 7f5f0903ab80 1 -- 172.21.15.37:0/148909556 shutdown_connections 2024-06-10T00:57:17.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.150+0000 7f5f0903ab80 1 --2- 172.21.15.37:0/148909556 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x556be9cb2900 0x556be9cb3d70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.150+0000 7f5f0903ab80 1 --2- 172.21.15.37:0/148909556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556be9c9ef10 0x556be9ca2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.150+0000 7f5f0903ab80 1 --2- 172.21.15.37:0/148909556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556be9c98ed0 0x556be9ca4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.150+0000 7f5f0903ab80 1 --2- 172.21.15.37:0/148909556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556be9c9e5f0 0x556be9ca1f50 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.150+0000 7f5f0903ab80 1 -- 172.21.15.37:0/148909556 >> 172.21.15.37:0/148909556 conn(0x556be9bc3c70 msgr2=0x556be9c97980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:17.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.150+0000 7f5ed77fe700 0 client.15441 ms_handle_reset on v2:172.21.15.94:6838/946269331 2024-06-10T00:57:17.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.150+0000 7f5f0903ab80 1 -- 172.21.15.37:0/148909556 shutdown_connections 2024-06-10T00:57:17.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.150+0000 7f5f0903ab80 1 -- 172.21.15.37:0/148909556 wait complete. 2024-06-10T00:57:17.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.150+0000 7f5f0903ab80 20 client.15441 trim_cache size 0 max 16384 2024-06-10T00:57:17.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.172+0000 7f5f0903ab80 1 Processor -- start 2024-06-10T00:57:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.172+0000 7f5f0903ab80 1 -- start start 2024-06-10T00:57:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.172+0000 7f5f0903ab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556be9c98ed0 0x556be9c98990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.172+0000 7f5f0903ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556be9c9e5f0 0x556be9c9fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.172+0000 7f5f0903ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556be9c9ef10 0x556be9c8cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.172+0000 7f5f0903ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556be9cb20b0 con 0x556be9c9ef10 2024-06-10T00:57:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.172+0000 7f5f0903ab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556be9cb2780 con 0x556be9c98ed0 2024-06-10T00:57:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.172+0000 7f5f0903ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556be9cb2e50 con 0x556be9c9e5f0 2024-06-10T00:57:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.172+0000 7f5efcd54700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556be9c9ef10 0x556be9c8cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.172+0000 7f5efcd54700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556be9c9ef10 0x556be9c8cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37324/0 (socket says 172.21.15.37:37324) 2024-06-10T00:57:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.172+0000 7f5efcd54700 1 -- 172.21.15.37:0/561078926 learned_addr learned my addr 172.21.15.37:0/561078926 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.173+0000 7f5eeffff700 1 --2- 172.21.15.37:0/561078926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556be9c9e5f0 0x556be9c9fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.173+0000 7f5ef7fff700 1 --2- 172.21.15.37:0/561078926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556be9c98ed0 0x556be9c98990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.173+0000 7f5efcd54700 1 -- 172.21.15.37:0/561078926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556be9c9e5f0 msgr2=0x556be9c9fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:17.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.173+0000 7f5efcd54700 1 --2- 172.21.15.37:0/561078926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556be9c9e5f0 0x556be9c9fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.173+0000 7f5efcd54700 1 -- 172.21.15.37:0/561078926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556be9c98ed0 msgr2=0x556be9c98990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:17.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.173+0000 7f5efcd54700 1 --2- 172.21.15.37:0/561078926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556be9c98ed0 0x556be9c98990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.173+0000 7f5efcd54700 1 -- 172.21.15.37:0/561078926 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556be9ca5390 con 0x556be9c9ef10 2024-06-10T00:57:17.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.173+0000 7f5efcd54700 1 --2- 172.21.15.37:0/561078926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556be9c9ef10 0x556be9c8cdb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0017000 tx=0x7f5ef00092b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d20fa6b4bbb9e2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:17.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.173+0000 7f5eedffb700 1 -- 172.21.15.37:0/561078926 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef001f4b0 con 0x556be9c9ef10 2024-06-10T00:57:17.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.173+0000 7f5eedffb700 1 -- 172.21.15.37:0/561078926 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ef001fb30 con 0x556be9c9ef10 2024-06-10T00:57:17.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.173+0000 7f5eedffb700 1 -- 172.21.15.37:0/561078926 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0028b00 con 0x556be9c9ef10 2024-06-10T00:57:17.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.174+0000 7f5f0903ab80 1 -- 172.21.15.37:0/561078926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556be9c9ef10 msgr2=0x556be9c8cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:17.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.174+0000 7f5f0903ab80 1 --2- 172.21.15.37:0/561078926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556be9c9ef10 0x556be9c8cdb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0017000 tx=0x7f5ef00092b0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.174+0000 7f5f0903ab80 1 -- 172.21.15.37:0/561078926 shutdown_connections 2024-06-10T00:57:17.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.174+0000 7f5f0903ab80 1 --2- 172.21.15.37:0/561078926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556be9c9ef10 0x556be9c8cdb0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.174+0000 7f5f0903ab80 1 --2- 172.21.15.37:0/561078926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556be9c98ed0 0x556be9c98990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.174+0000 7f5f0903ab80 1 --2- 172.21.15.37:0/561078926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556be9c9e5f0 0x556be9c9fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.174+0000 7f5f0903ab80 1 -- 172.21.15.37:0/561078926 >> 172.21.15.37:0/561078926 conn(0x556be9bc3c70 msgr2=0x556be9c9f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.174+0000 7f5f0903ab80 1 -- 172.21.15.37:0/561078926 shutdown_connections 2024-06-10T00:57:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.174+0000 7f5f0903ab80 1 -- 172.21.15.37:0/561078926 wait complete. 2024-06-10T00:57:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.174+0000 7f5f0903ab80 1 -- start start 2024-06-10T00:57:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.174+0000 7f5f0903ab80 1 Processor -- start 2024-06-10T00:57:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.175+0000 7f5f0903ab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556be9c98ed0 0x556be9cbdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.175+0000 7f5f0903ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556be9c9e5f0 0x556be9cb6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.175+0000 7f5f0903ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556be9c9ef10 0x556be9cb7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.175+0000 7f5f0903ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556be9cb3bf0 con 0x556be9c9ef10 2024-06-10T00:57:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.175+0000 7f5f0903ab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556be9cb3520 con 0x556be9c98ed0 2024-06-10T00:57:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.175+0000 7f5f0903ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556be9cb42c0 con 0x556be9c9e5f0 2024-06-10T00:57:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.175+0000 7f5f0903ab80 10 client.0 _ll_register_callbacks cb 0x556be9b848d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.175+0000 7f5efcd54700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556be9c9ef10 0x556be9cb7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:17.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.175+0000 7f5eeffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556be9c9e5f0 0x556be9cb6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:17.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.175+0000 7f5efcd54700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556be9c9ef10 0x556be9cb7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37340/0 (socket says 172.21.15.37:37340) 2024-06-10T00:57:17.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.175+0000 7f5efcd54700 1 -- 172.21.15.37:0/265088116 learned_addr learned my addr 172.21.15.37:0/265088116 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:17.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.175+0000 7f5ef7fff700 1 --2- 172.21.15.37:0/265088116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556be9c98ed0 0x556be9cbdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:17.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.175+0000 7f5efcd54700 1 -- 172.21.15.37:0/265088116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556be9c9e5f0 msgr2=0x556be9cb6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:17.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.175+0000 7f5efcd54700 1 --2- 172.21.15.37:0/265088116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556be9c9e5f0 0x556be9cb6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.175+0000 7f5efcd54700 1 -- 172.21.15.37:0/265088116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556be9c98ed0 msgr2=0x556be9cbdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:17.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.175+0000 7f5efcd54700 1 --2- 172.21.15.37:0/265088116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556be9c98ed0 0x556be9cbdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.175+0000 7f5efcd54700 1 -- 172.21.15.37:0/265088116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556be9cbb5d0 con 0x556be9c9ef10 2024-06-10T00:57:17.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.175+0000 7f5ef7fff700 1 --2- 172.21.15.37:0/265088116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556be9c98ed0 0x556be9cbdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:57:17.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.175+0000 7f5efcd54700 1 --2- 172.21.15.37:0/265088116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556be9c9ef10 0x556be9cb7550 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0007820 tx=0x7f5ef0007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180a5758bdb0250a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:17.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.175+0000 7f5ed77fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:57:17.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.176+0000 7f5ed77fe700 1 -- 172.21.15.37:0/265088116 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef00288a0 con 0x556be9c9ef10 2024-06-10T00:57:17.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.176+0000 7f5ed77fe700 1 -- 172.21.15.37:0/265088116 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f5ef0028a40 con 0x556be9c9ef10 2024-06-10T00:57:17.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.176+0000 7f5f0903ab80 1 -- 172.21.15.37:0/265088116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ef001fb30 con 0x556be9c9ef10 2024-06-10T00:57:17.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.176+0000 7f5ed77fe700 1 -- 172.21.15.37:0/265088116 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5ef0039050 con 0x556be9c9ef10 2024-06-10T00:57:17.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.176+0000 7f5f0903ab80 10 client.15447 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:17.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.176+0000 7f5f0903ab80 1 -- 172.21.15.37:0/265088116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556be9d1e380 con 0x556be9c9ef10 2024-06-10T00:57:17.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.176+0000 7f5ed77fe700 1 -- 172.21.15.37:0/265088116 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ef001fb30 con 0x556be9c9ef10 2024-06-10T00:57:17.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.176+0000 7f5ed77fe700 1 -- 172.21.15.37:0/265088116 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5ef0026020 con 0x556be9c9ef10 2024-06-10T00:57:17.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.176+0000 7f5f0903ab80 10 client.15447 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:17.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.176+0000 7f5f0903ab80 10 client.15447 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-10T00:57:17.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.176+0000 7f5f0903ab80 20 client.15447 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:17.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.176+0000 7f5f0903ab80 1 --2- 172.21.15.37:0/265088116 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x556be9d22840 0x556be9d21550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:17.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.176+0000 7f5f0903ab80 4 client.15447 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T00:57:17.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.176+0000 7f5f0903ab80 1 -- 172.21.15.37:0/265088116 --> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] -- command(tid 0: {"prefix": "status"}) -- 0x556be9bf2240 con 0x556be9d22840 2024-06-10T00:57:17.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.177+0000 7f5ef7fff700 1 --2- 172.21.15.37:0/265088116 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x556be9d22840 0x556be9d21550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:17.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.177+0000 7f5ef7fff700 1 --2- 172.21.15.37:0/265088116 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x556be9d22840 0x556be9d21550 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=25ddff1d0bb83f86 server_cookie=31ac1b10ed9a3c2a in_seq=0 out_seq=0 2024-06-10T00:57:17.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.177+0000 7f5ed77fe700 10 client.15447 ms_handle_connect on v2:172.21.15.94:6838/946269331 2024-06-10T00:57:17.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.177+0000 7f5ed77fe700 1 -- 172.21.15.37:0/265088116 <== mds.1 v2:172.21.15.94:6838/946269331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x556be9bf2240 con 0x556be9d22840 2024-06-10T00:57:17.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.177+0000 7f5ed77fe700 10 client.15447 handle_command_reply: tid=0 2024-06-10T00:57:17.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.177+0000 7f5f0903ab80 1 client.15447 shutdown 2024-06-10T00:57:17.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.177+0000 7f5f0903ab80 1 -- 172.21.15.37:0/265088116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556be9c9ef10 msgr2=0x556be9cb7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:17.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.177+0000 7f5f0903ab80 1 --2- 172.21.15.37:0/265088116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556be9c9ef10 0x556be9cb7550 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0007820 tx=0x7f5ef0007f10 comp rx=0 tx=0).stop 2024-06-10T00:57:17.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.177+0000 7f5f0903ab80 1 -- 172.21.15.37:0/265088116 shutdown_connections 2024-06-10T00:57:17.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.177+0000 7f5f0903ab80 1 --2- 172.21.15.37:0/265088116 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x556be9d22840 0x556be9d21550 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.177+0000 7f5f0903ab80 1 --2- 172.21.15.37:0/265088116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556be9c9ef10 0x556be9cb7550 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.177+0000 7f5f0903ab80 1 --2- 172.21.15.37:0/265088116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556be9c98ed0 0x556be9cbdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.177+0000 7f5ed77fe700 0 client.15447 ms_handle_reset on v2:172.21.15.94:6838/946269331 2024-06-10T00:57:17.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.177+0000 7f5f0903ab80 1 --2- 172.21.15.37:0/265088116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556be9c9e5f0 0x556be9cb6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.177+0000 7f5f0903ab80 1 -- 172.21.15.37:0/265088116 >> 172.21.15.37:0/265088116 conn(0x556be9bc3c70 msgr2=0x556be9c8cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:17.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.177+0000 7f5f0903ab80 1 -- 172.21.15.37:0/265088116 shutdown_connections 2024-06-10T00:57:17.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.178+0000 7f5f0903ab80 1 -- 172.21.15.37:0/265088116 wait complete. 2024-06-10T00:57:17.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.178+0000 7f5f0903ab80 20 client.15447 trim_cache size 0 max 16384 2024-06-10T00:57:17.183 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T00:57:17.183 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "68793514-26c2-11ef-bc9b-c7b262605968", 2024-06-10T00:57:17.183 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 1, 2024-06-10T00:57:17.183 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24458, 2024-06-10T00:57:17.183 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-06-10T00:57:17.183 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-06-10T00:57:17.183 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-06-10T00:57:17.183 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 218.90470444900001, 2024-06-10T00:57:17.184 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 27, 2024-06-10T00:57:17.184 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 100, 2024-06-10T00:57:17.184 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 100, 2024-06-10T00:57:17.184 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 296.61846167599998 2024-06-10T00:57:17.184 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T00:57:17.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.178+0000 7f5efe7b7700 1 -- 172.21.15.37:0/3486145242 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5ee40a2cb0 msgr2=0x7f5ee40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:17.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.178+0000 7f5efe7b7700 1 --2- 172.21.15.37:0/3486145242 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5ee40a2cb0 0x7f5ee40a5170 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8108c40 tx=0x7f5ee0005fd0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.178+0000 7f5efe7b7700 1 -- 172.21.15.37:0/3486145242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ef8102770 msgr2=0x7f5ef8074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.178+0000 7f5efe7b7700 1 --2- 172.21.15.37:0/3486145242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ef8102770 0x7f5ef8074790 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800c9b0 tx=0x7f5ee800cd70 comp rx=0 tx=0).stop 2024-06-10T00:57:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.178+0000 7f5efe7b7700 1 -- 172.21.15.37:0/3486145242 shutdown_connections 2024-06-10T00:57:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.178+0000 7f5efe7b7700 1 --2- 172.21.15.37:0/3486145242 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5ee40a2cb0 0x7f5ee40a5170 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.178+0000 7f5efe7b7700 1 --2- 172.21.15.37:0/3486145242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ef8107860 0x7f5ef8075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.178+0000 7f5efe7b7700 1 --2- 172.21.15.37:0/3486145242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ef8102770 0x7f5ef8074790 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.178+0000 7f5efe7b7700 1 --2- 172.21.15.37:0/3486145242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ef8103120 0x7f5ef8074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.178+0000 7f5efe7b7700 1 -- 172.21.15.37:0/3486145242 >> 172.21.15.37:0/3486145242 conn(0x7f5ef8079600 msgr2=0x7f5ef8100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.178+0000 7f5efe7b7700 1 -- 172.21.15.37:0/3486145242 shutdown_connections 2024-06-10T00:57:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:17.178+0000 7f5efe7b7700 1 -- 172.21.15.37:0/3486145242 wait complete. 2024-06-10T00:57:17.897 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-10T00:57:18.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:18 smithi037 ceph-mon[32192]: pgmap v448: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-06-10T00:57:18.153 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:57:18.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:18 smithi094 ceph-mon[36708]: pgmap v448: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-06-10T00:57:18.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:18 smithi167 ceph-mon[36917]: pgmap v448: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-06-10T00:57:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.006+0000 7f5d89af9700 1 -- 172.21.15.37:0/2029536695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d8410ee90 msgr2=0x7f5d84111280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.006+0000 7f5d89af9700 1 --2- 172.21.15.37:0/2029536695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d8410ee90 0x7f5d84111280 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00b0a0 tx=0x7f5d7c00b3b0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.006+0000 7f5d89af9700 1 -- 172.21.15.37:0/2029536695 shutdown_connections 2024-06-10T00:57:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.006+0000 7f5d89af9700 1 --2- 172.21.15.37:0/2029536695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d8410ee90 0x7f5d84111280 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.006+0000 7f5d89af9700 1 --2- 172.21.15.37:0/2029536695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d840746c0 0x7f5d84074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.006+0000 7f5d89af9700 1 --2- 172.21.15.37:0/2029536695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d84074fe0 0x7f5d8410e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.006+0000 7f5d89af9700 1 -- 172.21.15.37:0/2029536695 >> 172.21.15.37:0/2029536695 conn(0x7f5d840795f0 msgr2=0x7f5d84079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.006+0000 7f5d89af9700 1 -- 172.21.15.37:0/2029536695 shutdown_connections 2024-06-10T00:57:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.006+0000 7f5d89af9700 1 -- 172.21.15.37:0/2029536695 wait complete. 2024-06-10T00:57:20.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.007+0000 7f5d89af9700 1 Processor -- start 2024-06-10T00:57:20.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.007+0000 7f5d89af9700 1 -- start start 2024-06-10T00:57:20.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.007+0000 7f5d89af9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d840746c0 0x7f5d8419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:20.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.007+0000 7f5d89af9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d84074fe0 0x7f5d8419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:20.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.007+0000 7f5d89af9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d8410ee90 0x7f5d8419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:20.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.007+0000 7f5d89af9700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5d84114980 con 0x7f5d8410ee90 2024-06-10T00:57:20.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.007+0000 7f5d89af9700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5d84114800 con 0x7f5d84074fe0 2024-06-10T00:57:20.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.007+0000 7f5d89af9700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5d84114b00 con 0x7f5d840746c0 2024-06-10T00:57:20.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.007+0000 7f5d83fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d8410ee90 0x7f5d8419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:20.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.007+0000 7f5d83fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d8410ee90 0x7f5d8419f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37354/0 (socket says 172.21.15.37:37354) 2024-06-10T00:57:20.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.007+0000 7f5d83fff700 1 -- 172.21.15.37:0/4290754050 learned_addr learned my addr 172.21.15.37:0/4290754050 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:20.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.007+0000 7f5d82ffd700 1 --2- 172.21.15.37:0/4290754050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d84074fe0 0x7f5d8419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:20.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.007+0000 7f5d837fe700 1 --2- 172.21.15.37:0/4290754050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d840746c0 0x7f5d8419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:20.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.007+0000 7f5d83fff700 1 -- 172.21.15.37:0/4290754050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d840746c0 msgr2=0x7f5d8419adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:20.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.007+0000 7f5d83fff700 1 --2- 172.21.15.37:0/4290754050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d840746c0 0x7f5d8419adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.007+0000 7f5d83fff700 1 -- 172.21.15.37:0/4290754050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d84074fe0 msgr2=0x7f5d8419b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:20.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.008+0000 7f5d83fff700 1 --2- 172.21.15.37:0/4290754050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d84074fe0 0x7f5d8419b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.008+0000 7f5d83fff700 1 -- 172.21.15.37:0/4290754050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d8419fe00 con 0x7f5d8410ee90 2024-06-10T00:57:20.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.008+0000 7f5d83fff700 1 --2- 172.21.15.37:0/4290754050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d8410ee90 0x7f5d8419f6c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00b780 tx=0x7f5d7c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:20.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.008+0000 7f5d80ff9700 1 -- 172.21.15.37:0/4290754050 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c021070 con 0x7f5d8410ee90 2024-06-10T00:57:20.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.008+0000 7f5d80ff9700 1 -- 172.21.15.37:0/4290754050 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d7c007b80 con 0x7f5d8410ee90 2024-06-10T00:57:20.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.008+0000 7f5d89af9700 1 -- 172.21.15.37:0/4290754050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d841a0090 con 0x7f5d8410ee90 2024-06-10T00:57:20.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.008+0000 7f5d80ff9700 1 -- 172.21.15.37:0/4290754050 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0052a0 con 0x7f5d8410ee90 2024-06-10T00:57:20.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.008+0000 7f5d89af9700 1 -- 172.21.15.37:0/4290754050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d841a7970 con 0x7f5d8410ee90 2024-06-10T00:57:20.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.010+0000 7f5d80ff9700 1 -- 172.21.15.37:0/4290754050 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f5d7c024070 con 0x7f5d8410ee90 2024-06-10T00:57:20.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.011+0000 7f5d80ff9700 1 --2- 172.21.15.37:0/4290754050 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5d6c0a2cb0 0x7f5d6c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:20.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.011+0000 7f5d837fe700 1 --2- 172.21.15.37:0/4290754050 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5d6c0a2cb0 0x7f5d6c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:20.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.011+0000 7f5d80ff9700 1 -- 172.21.15.37:0/4290754050 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f5d7c00f020 con 0x7f5d8410ee90 2024-06-10T00:57:20.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.011+0000 7f5d837fe700 1 --2- 172.21.15.37:0/4290754050 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5d6c0a2cb0 0x7f5d6c0a5170 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5d74009c60 tx=0x7f5d74009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:20.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.011+0000 7f5d9437cb80 1 Processor -- start 2024-06-10T00:57:20.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.012+0000 7f5d9437cb80 1 -- start start 2024-06-10T00:57:20.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.012+0000 7f5d9437cb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5d1e09f10 0x55a5d1e0a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:20.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.012+0000 7f5d9437cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5d1e0a830 0x55a5d1e04210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:20.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.012+0000 7f5d9437cb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5d1e04750 0x55a5d1e04b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:20.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.012+0000 7f5d9437cb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a5d1cf6f10 con 0x55a5d1e04750 2024-06-10T00:57:20.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.012+0000 7f5d9437cb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a5d1d55800 con 0x55a5d1e09f10 2024-06-10T00:57:20.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.012+0000 7f5d9437cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a5d1cbd410 con 0x55a5d1e0a830 2024-06-10T00:57:20.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.012+0000 7f5d83fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5d1e04750 0x55a5d1e04b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:20.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.012+0000 7f5d83fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5d1e04750 0x55a5d1e04b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37356/0 (socket says 172.21.15.37:37356) 2024-06-10T00:57:20.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.012+0000 7f5d83fff700 1 -- 172.21.15.37:0/823492251 learned_addr learned my addr 172.21.15.37:0/823492251 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:20.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.012+0000 7f5d82ffd700 1 --2- 172.21.15.37:0/823492251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5d1e0a830 0x55a5d1e04210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:20.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.012+0000 7f5d837fe700 1 --2- 172.21.15.37:0/823492251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5d1e09f10 0x55a5d1e0a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:20.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.012+0000 7f5d83fff700 1 -- 172.21.15.37:0/823492251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5d1e0a830 msgr2=0x55a5d1e04210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:20.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.012+0000 7f5d83fff700 1 --2- 172.21.15.37:0/823492251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5d1e0a830 0x55a5d1e04210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.012+0000 7f5d83fff700 1 -- 172.21.15.37:0/823492251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5d1e09f10 msgr2=0x55a5d1e0a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:20.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.012+0000 7f5d83fff700 1 --2- 172.21.15.37:0/823492251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5d1e09f10 0x55a5d1e0a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.012+0000 7f5d83fff700 1 -- 172.21.15.37:0/823492251 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5d1ca4880 con 0x55a5d1e04750 2024-06-10T00:57:20.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.012+0000 7f5d82ffd700 1 --2- 172.21.15.37:0/823492251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5d1e0a830 0x55a5d1e04210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:57:20.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.012+0000 7f5d83fff700 1 --2- 172.21.15.37:0/823492251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5d1e04750 0x55a5d1e04b70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00b070 tx=0x7f5d7c099150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d0154197075ecd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:20.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.013+0000 7f5d69ffb700 1 -- 172.21.15.37:0/823492251 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0074e0 con 0x55a5d1e04750 2024-06-10T00:57:20.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.013+0000 7f5d69ffb700 1 -- 172.21.15.37:0/823492251 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d7c09cd80 con 0x55a5d1e04750 2024-06-10T00:57:20.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.013+0000 7f5d69ffb700 1 -- 172.21.15.37:0/823492251 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c09d220 con 0x55a5d1e04750 2024-06-10T00:57:20.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.013+0000 7f5d9437cb80 1 -- 172.21.15.37:0/823492251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5d1e04750 msgr2=0x55a5d1e04b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:20.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.013+0000 7f5d9437cb80 1 --2- 172.21.15.37:0/823492251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5d1e04750 0x55a5d1e04b70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00b070 tx=0x7f5d7c099150 comp rx=0 tx=0).stop 2024-06-10T00:57:20.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.013+0000 7f5d9437cb80 1 -- 172.21.15.37:0/823492251 shutdown_connections 2024-06-10T00:57:20.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.013+0000 7f5d9437cb80 1 --2- 172.21.15.37:0/823492251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5d1e04750 0x55a5d1e04b70 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.013+0000 7f5d9437cb80 1 --2- 172.21.15.37:0/823492251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5d1e09f10 0x55a5d1e0a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.013+0000 7f5d9437cb80 1 --2- 172.21.15.37:0/823492251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5d1e0a830 0x55a5d1e04210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.013+0000 7f5d9437cb80 1 -- 172.21.15.37:0/823492251 >> 172.21.15.37:0/823492251 conn(0x55a5d1d31030 msgr2=0x55a5d1dff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:20.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.014+0000 7f5d9437cb80 1 -- 172.21.15.37:0/823492251 shutdown_connections 2024-06-10T00:57:20.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.014+0000 7f5d9437cb80 1 -- 172.21.15.37:0/823492251 wait complete. 2024-06-10T00:57:20.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.014+0000 7f5d9437cb80 1 -- start start 2024-06-10T00:57:20.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.014+0000 7f5d9437cb80 1 Processor -- start 2024-06-10T00:57:20.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.014+0000 7f5d9437cb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5d1e04750 0x55a5d1e105d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:20.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.014+0000 7f5d9437cb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5d1e09f10 0x55a5d1e0d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:20.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.014+0000 7f5d9437cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5d1e0a830 0x55a5d1e0de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:20.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.014+0000 7f5d9437cb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a5d1cf5f70 con 0x55a5d1e09f10 2024-06-10T00:57:20.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.014+0000 7f5d9437cb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a5d1d517c0 con 0x55a5d1e04750 2024-06-10T00:57:20.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.014+0000 7f5d9437cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a5d1e10fc0 con 0x55a5d1e0a830 2024-06-10T00:57:20.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.014+0000 7f5d9437cb80 10 client.0 _ll_register_callbacks cb 0x55a5d1ceb560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:20.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.015+0000 7f5d82ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5d1e09f10 0x55a5d1e0d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:20.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.015+0000 7f5d837fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5d1e04750 0x55a5d1e105d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:20.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.015+0000 7f5d837fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5d1e04750 0x55a5d1e105d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:47022/0 (socket says 172.21.15.37:47022) 2024-06-10T00:57:20.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.015+0000 7f5d837fe700 1 -- 172.21.15.37:0/3583452714 learned_addr learned my addr 172.21.15.37:0/3583452714 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:20.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.015+0000 7f5d83fff700 1 --2- 172.21.15.37:0/3583452714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5d1e0a830 0x55a5d1e0de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:20.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.015+0000 7f5d837fe700 1 -- 172.21.15.37:0/3583452714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5d1e0a830 msgr2=0x55a5d1e0de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:20.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.015+0000 7f5d837fe700 1 --2- 172.21.15.37:0/3583452714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5d1e0a830 0x55a5d1e0de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.015+0000 7f5d837fe700 1 -- 172.21.15.37:0/3583452714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5d1e09f10 msgr2=0x55a5d1e0d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:20.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.015+0000 7f5d837fe700 1 --2- 172.21.15.37:0/3583452714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5d1e09f10 0x55a5d1e0d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.015+0000 7f5d837fe700 1 -- 172.21.15.37:0/3583452714 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5d1d1b700 con 0x55a5d1e04750 2024-06-10T00:57:20.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.015+0000 7f5d837fe700 1 --2- 172.21.15.37:0/3583452714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5d1e04750 0x55a5d1e105d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5d7401ec10 tx=0x7f5d740047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c98743407886d15a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:20.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.015+0000 7f5d62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T00:57:20.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.015+0000 7f5d62ffd700 1 -- 172.21.15.37:0/3583452714 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7402b060 con 0x55a5d1e04750 2024-06-10T00:57:20.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.015+0000 7f5d9437cb80 1 -- 172.21.15.37:0/3583452714 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d7c09cd80 con 0x55a5d1e04750 2024-06-10T00:57:20.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.015+0000 7f5d62ffd700 1 -- 172.21.15.37:0/3583452714 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f5d7400ec50 con 0x55a5d1e04750 2024-06-10T00:57:20.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.016+0000 7f5d62ffd700 1 -- 172.21.15.37:0/3583452714 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5d74032050 con 0x55a5d1e04750 2024-06-10T00:57:20.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.016+0000 7f5d9437cb80 10 client.25079 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:20.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.016+0000 7f5d9437cb80 1 -- 172.21.15.37:0/3583452714 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a5d1e1b720 con 0x55a5d1e04750 2024-06-10T00:57:20.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.016+0000 7f5d62ffd700 1 -- 172.21.15.37:0/3583452714 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d7c09cd80 con 0x55a5d1e04750 2024-06-10T00:57:20.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.016+0000 7f5d62ffd700 1 -- 172.21.15.37:0/3583452714 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5d74025030 con 0x55a5d1e04750 2024-06-10T00:57:20.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.016+0000 7f5d9437cb80 10 client.25079 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:20.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.016+0000 7f5d9437cb80 10 client.25079 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T00:57:20.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.016+0000 7f5d9437cb80 20 client.25079 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:20.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.016+0000 7f5d9437cb80 1 --2- 172.21.15.37:0/3583452714 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a5d1e1dd70 0x55a5d1e1f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:20.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.016+0000 7f5d9437cb80 4 client.25079 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:57:20.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.016+0000 7f5d9437cb80 1 -- 172.21.15.37:0/3583452714 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5d1d62d00 con 0x55a5d1e1dd70 2024-06-10T00:57:20.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.016+0000 7f5d82ffd700 1 --2- 172.21.15.37:0/3583452714 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a5d1e1dd70 0x55a5d1e1f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:20.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.017+0000 7f5d82ffd700 1 --2- 172.21.15.37:0/3583452714 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a5d1e1dd70 0x55a5d1e1f1e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea64ce1c569fec4e server_cookie=cafb9bbded709f9f in_seq=0 out_seq=0 2024-06-10T00:57:20.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.017+0000 7f5d62ffd700 10 client.25079 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:20.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.018+0000 7f5d62ffd700 1 -- 172.21.15.37:0/3583452714 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a5d1d62d00 con 0x55a5d1e1dd70 2024-06-10T00:57:20.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.018+0000 7f5d62ffd700 10 client.25079 handle_command_reply: tid=0 2024-06-10T00:57:20.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.018+0000 7f5d9437cb80 1 client.25079 shutdown 2024-06-10T00:57:20.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.018+0000 7f5d9437cb80 1 -- 172.21.15.37:0/3583452714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5d1e04750 msgr2=0x55a5d1e105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:20.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.018+0000 7f5d9437cb80 1 --2- 172.21.15.37:0/3583452714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5d1e04750 0x55a5d1e105d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5d7401ec10 tx=0x7f5d740047c0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.018+0000 7f5d9437cb80 1 -- 172.21.15.37:0/3583452714 shutdown_connections 2024-06-10T00:57:20.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.018+0000 7f5d9437cb80 1 --2- 172.21.15.37:0/3583452714 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a5d1e1dd70 0x55a5d1e1f1e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.018+0000 7f5d9437cb80 1 --2- 172.21.15.37:0/3583452714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5d1e0a830 0x55a5d1e0de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.018+0000 7f5d9437cb80 1 --2- 172.21.15.37:0/3583452714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5d1e04750 0x55a5d1e105d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.018+0000 7f5d62ffd700 0 client.25079 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:20.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.018+0000 7f5d9437cb80 1 --2- 172.21.15.37:0/3583452714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5d1e09f10 0x55a5d1e0d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.018+0000 7f5d9437cb80 1 -- 172.21.15.37:0/3583452714 >> 172.21.15.37:0/3583452714 conn(0x55a5d1d31030 msgr2=0x55a5d1dfe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:20.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.018+0000 7f5d9437cb80 1 -- 172.21.15.37:0/3583452714 shutdown_connections 2024-06-10T00:57:20.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.018+0000 7f5d9437cb80 1 -- 172.21.15.37:0/3583452714 wait complete. 2024-06-10T00:57:20.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.018+0000 7f5d9437cb80 20 client.25079 trim_cache size 0 max 16384 2024-06-10T00:57:20.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.040+0000 7f5d9437cb80 1 Processor -- start 2024-06-10T00:57:20.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.040+0000 7f5d9437cb80 1 -- start start 2024-06-10T00:57:20.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.040+0000 7f5d9437cb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5d1e04750 0x55a5d1e04210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:20.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.040+0000 7f5d9437cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5d1e09f10 0x55a5d1e0b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:20.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.040+0000 7f5d9437cb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5d1e0a830 0x55a5d1dfb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:20.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.040+0000 7f5d9437cb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a5d1e1d520 con 0x55a5d1e0a830 2024-06-10T00:57:20.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.040+0000 7f5d9437cb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a5d1e1dbf0 con 0x55a5d1e04750 2024-06-10T00:57:20.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.040+0000 7f5d9437cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a5d1e1e2c0 con 0x55a5d1e09f10 2024-06-10T00:57:20.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.040+0000 7f5d83fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5d1e0a830 0x55a5d1dfb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:20.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.040+0000 7f5d83fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5d1e0a830 0x55a5d1dfb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37374/0 (socket says 172.21.15.37:37374) 2024-06-10T00:57:20.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.040+0000 7f5d83fff700 1 -- 172.21.15.37:0/426396671 learned_addr learned my addr 172.21.15.37:0/426396671 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:20.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.040+0000 7f5d82ffd700 1 --2- 172.21.15.37:0/426396671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5d1e09f10 0x55a5d1e0b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:20.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.040+0000 7f5d837fe700 1 --2- 172.21.15.37:0/426396671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5d1e04750 0x55a5d1e04210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:20.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.040+0000 7f5d83fff700 1 -- 172.21.15.37:0/426396671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5d1e09f10 msgr2=0x55a5d1e0b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:20.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.040+0000 7f5d83fff700 1 --2- 172.21.15.37:0/426396671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5d1e09f10 0x55a5d1e0b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.040+0000 7f5d83fff700 1 -- 172.21.15.37:0/426396671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5d1e04750 msgr2=0x55a5d1e04210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:20.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.040+0000 7f5d83fff700 1 --2- 172.21.15.37:0/426396671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5d1e04750 0x55a5d1e04210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.040+0000 7f5d83fff700 1 -- 172.21.15.37:0/426396671 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5d1e0be90 con 0x55a5d1e0a830 2024-06-10T00:57:20.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.040+0000 7f5d82ffd700 1 --2- 172.21.15.37:0/426396671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5d1e09f10 0x55a5d1e0b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:57:20.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.040+0000 7f5d83fff700 1 --2- 172.21.15.37:0/426396671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5d1e0a830 0x55a5d1dfb320 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c09c850 tx=0x7f5d7c028040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91b618ee4d91afd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:20.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.041+0000 7f5d697fa700 1 -- 172.21.15.37:0/426396671 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0a5960 con 0x55a5d1e0a830 2024-06-10T00:57:20.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.041+0000 7f5d697fa700 1 -- 172.21.15.37:0/426396671 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d7c0074e0 con 0x55a5d1e0a830 2024-06-10T00:57:20.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.041+0000 7f5d697fa700 1 -- 172.21.15.37:0/426396671 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c09d240 con 0x55a5d1e0a830 2024-06-10T00:57:20.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.041+0000 7f5d9437cb80 1 -- 172.21.15.37:0/426396671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5d1e0a830 msgr2=0x55a5d1dfb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:20.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.041+0000 7f5d9437cb80 1 --2- 172.21.15.37:0/426396671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5d1e0a830 0x55a5d1dfb320 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c09c850 tx=0x7f5d7c028040 comp rx=0 tx=0).stop 2024-06-10T00:57:20.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.041+0000 7f5d9437cb80 1 -- 172.21.15.37:0/426396671 shutdown_connections 2024-06-10T00:57:20.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.041+0000 7f5d9437cb80 1 --2- 172.21.15.37:0/426396671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5d1e0a830 0x55a5d1dfb320 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.041+0000 7f5d9437cb80 1 --2- 172.21.15.37:0/426396671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5d1e04750 0x55a5d1e04210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.041+0000 7f5d9437cb80 1 --2- 172.21.15.37:0/426396671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5d1e09f10 0x55a5d1e0b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.041+0000 7f5d9437cb80 1 -- 172.21.15.37:0/426396671 >> 172.21.15.37:0/426396671 conn(0x55a5d1d31030 msgr2=0x55a5d1e0c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:20.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.041+0000 7f5d9437cb80 1 -- 172.21.15.37:0/426396671 shutdown_connections 2024-06-10T00:57:20.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.041+0000 7f5d9437cb80 1 -- 172.21.15.37:0/426396671 wait complete. 2024-06-10T00:57:20.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.042+0000 7f5d9437cb80 1 -- start start 2024-06-10T00:57:20.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.042+0000 7f5d9437cb80 1 Processor -- start 2024-06-10T00:57:20.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.042+0000 7f5d9437cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5d1e04750 0x55a5d1e226b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:20.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.042+0000 7f5d9437cb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5d1e09f10 0x55a5d1e22bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:20.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.042+0000 7f5d9437cb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5d1e0a830 0x55a5d1e26e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:20.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.042+0000 7f5d9437cb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a5d1e1f060 con 0x55a5d1e0a830 2024-06-10T00:57:20.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.042+0000 7f5d9437cb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a5d1e1e990 con 0x55a5d1e09f10 2024-06-10T00:57:20.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.042+0000 7f5d9437cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a5d1e1f730 con 0x55a5d1e04750 2024-06-10T00:57:20.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.042+0000 7f5d9437cb80 10 client.0 _ll_register_callbacks cb 0x55a5d1ceb560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:20.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.042+0000 7f5d83fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5d1e0a830 0x55a5d1e26e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:20.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.042+0000 7f5d83fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5d1e0a830 0x55a5d1e26e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37376/0 (socket says 172.21.15.37:37376) 2024-06-10T00:57:20.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.042+0000 7f5d83fff700 1 -- 172.21.15.37:0/1970634882 learned_addr learned my addr 172.21.15.37:0/1970634882 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:20.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.042+0000 7f5d82ffd700 1 --2- 172.21.15.37:0/1970634882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5d1e09f10 0x55a5d1e22bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:20.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.042+0000 7f5d837fe700 1 --2- 172.21.15.37:0/1970634882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5d1e04750 0x55a5d1e226b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:20.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.043+0000 7f5d83fff700 1 -- 172.21.15.37:0/1970634882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5d1e04750 msgr2=0x55a5d1e226b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:20.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.043+0000 7f5d83fff700 1 --2- 172.21.15.37:0/1970634882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5d1e04750 0x55a5d1e226b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.043+0000 7f5d83fff700 1 -- 172.21.15.37:0/1970634882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5d1e09f10 msgr2=0x55a5d1e22bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:20.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.043+0000 7f5d83fff700 1 --2- 172.21.15.37:0/1970634882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5d1e09f10 0x55a5d1e22bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.043+0000 7f5d83fff700 1 -- 172.21.15.37:0/1970634882 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5d1e234f0 con 0x55a5d1e0a830 2024-06-10T00:57:20.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.043+0000 7f5d83fff700 1 --2- 172.21.15.37:0/1970634882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5d1e0a830 0x55a5d1e26e70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c028010 tx=0x7f5d7c099130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce4f89905668684 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:20.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.043+0000 7f5d62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:57:20.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.043+0000 7f5d62ffd700 1 -- 172.21.15.37:0/1970634882 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c083020 con 0x55a5d1e0a830 2024-06-10T00:57:20.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.043+0000 7f5d62ffd700 1 -- 172.21.15.37:0/1970634882 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f5d7c0a54d0 con 0x55a5d1e0a830 2024-06-10T00:57:20.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.043+0000 7f5d9437cb80 1 -- 172.21.15.37:0/1970634882 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d7c0074e0 con 0x55a5d1e0a830 2024-06-10T00:57:20.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.043+0000 7f5d62ffd700 1 -- 172.21.15.37:0/1970634882 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5d7c0a5960 con 0x55a5d1e0a830 2024-06-10T00:57:20.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.043+0000 7f5d9437cb80 10 client.15465 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:20.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.043+0000 7f5d9437cb80 1 -- 172.21.15.37:0/1970634882 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a5d1e89c60 con 0x55a5d1e0a830 2024-06-10T00:57:20.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.043+0000 7f5d62ffd700 1 -- 172.21.15.37:0/1970634882 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d7c0074e0 con 0x55a5d1e0a830 2024-06-10T00:57:20.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.043+0000 7f5d62ffd700 1 -- 172.21.15.37:0/1970634882 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5d7c011070 con 0x55a5d1e0a830 2024-06-10T00:57:20.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.044+0000 7f5d9437cb80 10 client.15465 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:20.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.044+0000 7f5d9437cb80 10 client.15465 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-10T00:57:20.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.044+0000 7f5d9437cb80 20 client.15465 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:20.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.044+0000 7f5d9437cb80 1 --2- 172.21.15.37:0/1970634882 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a5d1e8e200 0x55a5d1e8cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:20.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.044+0000 7f5d9437cb80 4 client.15465 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-10T00:57:20.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.044+0000 7f5d9437cb80 1 -- 172.21.15.37:0/1970634882 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "status"}) -- 0x55a5d1d5d280 con 0x55a5d1e8e200 2024-06-10T00:57:20.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.044+0000 7f5d837fe700 1 --2- 172.21.15.37:0/1970634882 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a5d1e8e200 0x55a5d1e8cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:20.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.044+0000 7f5d837fe700 1 --2- 172.21.15.37:0/1970634882 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a5d1e8e200 0x55a5d1e8cf10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33cfd9e2d694dd10 server_cookie=6db306edd24d8391 in_seq=0 out_seq=0 2024-06-10T00:57:20.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.044+0000 7f5d62ffd700 10 client.15465 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:20.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.045+0000 7f5d62ffd700 1 -- 172.21.15.37:0/1970634882 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55a5d1d5d280 con 0x55a5d1e8e200 2024-06-10T00:57:20.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.045+0000 7f5d62ffd700 10 client.15465 handle_command_reply: tid=0 2024-06-10T00:57:20.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.045+0000 7f5d9437cb80 1 client.15465 shutdown 2024-06-10T00:57:20.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.045+0000 7f5d9437cb80 1 -- 172.21.15.37:0/1970634882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5d1e0a830 msgr2=0x55a5d1e26e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:20.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.045+0000 7f5d9437cb80 1 --2- 172.21.15.37:0/1970634882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5d1e0a830 0x55a5d1e26e70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c028010 tx=0x7f5d7c099130 comp rx=0 tx=0).stop 2024-06-10T00:57:20.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.045+0000 7f5d9437cb80 1 -- 172.21.15.37:0/1970634882 shutdown_connections 2024-06-10T00:57:20.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.045+0000 7f5d9437cb80 1 --2- 172.21.15.37:0/1970634882 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a5d1e8e200 0x55a5d1e8cf10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.045+0000 7f5d9437cb80 1 --2- 172.21.15.37:0/1970634882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5d1e0a830 0x55a5d1e26e70 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.045+0000 7f5d9437cb80 1 --2- 172.21.15.37:0/1970634882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a5d1e04750 0x55a5d1e226b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.045+0000 7f5d9437cb80 1 --2- 172.21.15.37:0/1970634882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a5d1e09f10 0x55a5d1e22bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.045+0000 7f5d9437cb80 1 -- 172.21.15.37:0/1970634882 >> 172.21.15.37:0/1970634882 conn(0x55a5d1d31030 msgr2=0x55a5d1e0b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:20.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.045+0000 7f5d62ffd700 0 client.15465 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:20.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.045+0000 7f5d9437cb80 1 -- 172.21.15.37:0/1970634882 shutdown_connections 2024-06-10T00:57:20.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.045+0000 7f5d9437cb80 1 -- 172.21.15.37:0/1970634882 wait complete. 2024-06-10T00:57:20.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.045+0000 7f5d9437cb80 20 client.15465 trim_cache size 0 max 16384 2024-06-10T00:57:20.051 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T00:57:20.051 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "68793514-26c2-11ef-bc9b-c7b262605968", 2024-06-10T00:57:20.051 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-06-10T00:57:20.051 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24472, 2024-06-10T00:57:20.051 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-06-10T00:57:20.051 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-06-10T00:57:20.051 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-06-10T00:57:20.051 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 282.096228827, 2024-06-10T00:57:20.051 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 27, 2024-06-10T00:57:20.051 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 100, 2024-06-10T00:57:20.051 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 100, 2024-06-10T00:57:20.052 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 309.745612134 2024-06-10T00:57:20.052 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T00:57:20.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.045+0000 7f5d89af9700 1 -- 172.21.15.37:0/4290754050 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5d6c0a2cb0 msgr2=0x7f5d6c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:20.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.045+0000 7f5d89af9700 1 --2- 172.21.15.37:0/4290754050 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5d6c0a2cb0 0x7f5d6c0a5170 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5d74009c60 tx=0x7f5d74009260 comp rx=0 tx=0).stop 2024-06-10T00:57:20.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.045+0000 7f5d89af9700 1 -- 172.21.15.37:0/4290754050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d8410ee90 msgr2=0x7f5d8419f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:20.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.045+0000 7f5d89af9700 1 --2- 172.21.15.37:0/4290754050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d8410ee90 0x7f5d8419f6c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00b780 tx=0x7f5d7c004d00 comp rx=0 tx=0).stop 2024-06-10T00:57:20.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.045+0000 7f5d89af9700 1 -- 172.21.15.37:0/4290754050 shutdown_connections 2024-06-10T00:57:20.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.046+0000 7f5d89af9700 1 --2- 172.21.15.37:0/4290754050 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5d6c0a2cb0 0x7f5d6c0a5170 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.046+0000 7f5d89af9700 1 --2- 172.21.15.37:0/4290754050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d8410ee90 0x7f5d8419f6c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.046+0000 7f5d89af9700 1 --2- 172.21.15.37:0/4290754050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d840746c0 0x7f5d8419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.046+0000 7f5d89af9700 1 --2- 172.21.15.37:0/4290754050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d84074fe0 0x7f5d8419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:20.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.046+0000 7f5d89af9700 1 -- 172.21.15.37:0/4290754050 >> 172.21.15.37:0/4290754050 conn(0x7f5d840795f0 msgr2=0x7f5d840ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:20.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.046+0000 7f5d89af9700 1 -- 172.21.15.37:0/4290754050 shutdown_connections 2024-06-10T00:57:20.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:20.046+0000 7f5d89af9700 1 -- 172.21.15.37:0/4290754050 wait complete. 2024-06-10T00:57:20.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:20 smithi094 ceph-mon[36708]: pgmap v449: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-06-10T00:57:20.376 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:20 smithi037 ceph-mon[32192]: pgmap v449: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-06-10T00:57:20.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:20 smithi167 ceph-mon[36917]: pgmap v449: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-06-10T00:57:20.842 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-06-10T00:57:21.157 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T00:57:21.424 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:57:22.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:22 smithi037 ceph-mon[32192]: pgmap v450: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-06-10T00:57:22.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:22 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:57:22.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:22 smithi094 ceph-mon[36708]: pgmap v450: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-06-10T00:57:22.365 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:22 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:57:22.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:22 smithi167 ceph-mon[36917]: pgmap v450: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-06-10T00:57:22.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:22 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:57:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.673+0000 7f6403142640 1 -- 172.21.15.37:0/4009175594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63fc0ffcb0 msgr2=0x7f63fc1000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.673+0000 7f6403142640 1 --2- 172.21.15.37:0/4009175594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63fc0ffcb0 0x7f63fc1000d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009930 tx=0x7f63f002a420 comp rx=0 tx=0).stop 2024-06-10T00:57:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.673+0000 7f6403142640 1 -- 172.21.15.37:0/4009175594 shutdown_connections 2024-06-10T00:57:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.673+0000 7f6403142640 1 --2- 172.21.15.37:0/4009175594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63fc103640 0x7f63fc109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.673+0000 7f6403142640 1 --2- 172.21.15.37:0/4009175594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63fc102c80 0x7f63fc103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.673+0000 7f6403142640 1 --2- 172.21.15.37:0/4009175594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63fc0ffcb0 0x7f63fc1000d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.673+0000 7f6403142640 1 -- 172.21.15.37:0/4009175594 >> 172.21.15.37:0/4009175594 conn(0x7f63fc0fb600 msgr2=0x7f63fc0fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.673+0000 7f6403142640 1 -- 172.21.15.37:0/4009175594 shutdown_connections 2024-06-10T00:57:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.673+0000 7f6403142640 1 -- 172.21.15.37:0/4009175594 wait complete. 2024-06-10T00:57:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.674+0000 7f6403142640 1 Processor -- start 2024-06-10T00:57:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.674+0000 7f6403142640 1 -- start start 2024-06-10T00:57:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.674+0000 7f6403142640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63fc0ffcb0 0x7f63fc1025a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.674+0000 7f6403142640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63fc102c80 0x7f63fc100c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.674+0000 7f6403142640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63fc103640 0x7f63fc101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.674+0000 7f6403142640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc10d680 con 0x7f63fc103640 2024-06-10T00:57:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.674+0000 7f6403142640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc10d500 con 0x7f63fc102c80 2024-06-10T00:57:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.674+0000 7f6403142640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc10d800 con 0x7f63fc0ffcb0 2024-06-10T00:57:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.674+0000 7f64016df640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63fc103640 0x7f63fc101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.674+0000 7f64016df640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63fc103640 0x7f63fc101170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55000/0 (socket says 172.21.15.37:55000) 2024-06-10T00:57:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.674+0000 7f64016df640 1 -- 172.21.15.37:0/4117522552 learned_addr learned my addr 172.21.15.37:0/4117522552 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.674+0000 7f6400ede640 1 --2- 172.21.15.37:0/4117522552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63fc0ffcb0 0x7f63fc1025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.674+0000 7f63fbfff640 1 --2- 172.21.15.37:0/4117522552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63fc102c80 0x7f63fc100c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.674+0000 7f64016df640 1 -- 172.21.15.37:0/4117522552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63fc0ffcb0 msgr2=0x7f63fc1025a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.674+0000 7f64016df640 1 --2- 172.21.15.37:0/4117522552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63fc0ffcb0 0x7f63fc1025a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.674+0000 7f64016df640 1 -- 172.21.15.37:0/4117522552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63fc102c80 msgr2=0x7f63fc100c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.674+0000 7f64016df640 1 --2- 172.21.15.37:0/4117522552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63fc102c80 0x7f63fc100c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.674+0000 7f64016df640 1 -- 172.21.15.37:0/4117522552 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63fc101a30 con 0x7f63fc103640 2024-06-10T00:57:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.674+0000 7f63fbfff640 1 --2- 172.21.15.37:0/4117522552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63fc102c80 0x7f63fc100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:57:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.675+0000 7f64016df640 1 --2- 172.21.15.37:0/4117522552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63fc103640 0x7f63fc101170 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f63f400eaa0 tx=0x7f63f400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.675+0000 7f63f9ffb640 1 -- 172.21.15.37:0/4117522552 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f400cdb0 con 0x7f63fc103640 2024-06-10T00:57:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.675+0000 7f63f9ffb640 1 -- 172.21.15.37:0/4117522552 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63f4005020 con 0x7f63fc103640 2024-06-10T00:57:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.675+0000 7f6403142640 1 -- 172.21.15.37:0/4117522552 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63fc1a1e40 con 0x7f63fc103640 2024-06-10T00:57:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.675+0000 7f63f9ffb640 1 -- 172.21.15.37:0/4117522552 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f40116b0 con 0x7f63fc103640 2024-06-10T00:57:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.675+0000 7f6403142640 1 -- 172.21.15.37:0/4117522552 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63fc1a2300 con 0x7f63fc103640 2024-06-10T00:57:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.678+0000 7f63f9ffb640 1 -- 172.21.15.37:0/4117522552 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f63f4005560 con 0x7f63fc103640 2024-06-10T00:57:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.678+0000 7f6405e53740 1 Processor -- start 2024-06-10T00:57:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.678+0000 7f6405e53740 1 -- start start 2024-06-10T00:57:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.678+0000 7f6405e53740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649528ed010 0x5649528ed430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.678+0000 7f6405e53740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5649528eda00 0x5649528f2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.678+0000 7f6405e53740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649528f2b70 0x5649528f5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.678+0000 7f6405e53740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5649528504f0 con 0x5649528eda00 2024-06-10T00:57:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.678+0000 7f6405e53740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5649527ee870 con 0x5649528f2b70 2024-06-10T00:57:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.678+0000 7f6405e53740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56495284c4b0 con 0x5649528ed010 2024-06-10T00:57:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.678+0000 7f63f9ffb640 1 --2- 172.21.15.37:0/4117522552 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f63ec0a2d50 0x7f63ec0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.678+0000 7f6400ede640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5649528eda00 0x5649528f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.679+0000 7f63fbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649528f2b70 0x5649528f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.679+0000 7f6400ede640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5649528eda00 0x5649528f2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55002/0 (socket says 172.21.15.37:55002) 2024-06-10T00:57:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.679+0000 7f6400ede640 1 -- 172.21.15.37:0/2733003117 learned_addr learned my addr 172.21.15.37:0/2733003117 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.679+0000 7f63fbfff640 1 --2- 172.21.15.37:0/2733003117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649528ed010 0x5649528ed430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.679+0000 7f64016df640 1 --2- 172.21.15.37:0/4117522552 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f63ec0a2d50 0x7f63ec0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.679+0000 7f63f9ffb640 1 -- 172.21.15.37:0/4117522552 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f63f40cf7e0 con 0x7f63fc103640 2024-06-10T00:57:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.679+0000 7f6400ede640 1 -- 172.21.15.37:0/2733003117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649528ed010 msgr2=0x5649528ed430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.679+0000 7f6400ede640 1 --2- 172.21.15.37:0/2733003117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649528ed010 0x5649528ed430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.679+0000 7f6400ede640 1 -- 172.21.15.37:0/2733003117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649528f2b70 msgr2=0x5649528f5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.679+0000 7f6400ede640 1 --2- 172.21.15.37:0/2733003117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649528f2b70 0x5649528f5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.679+0000 7f6400ede640 1 -- 172.21.15.37:0/2733003117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56495280a9b0 con 0x5649528eda00 2024-06-10T00:57:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.679+0000 7f6400ede640 1 --2- 172.21.15.37:0/2733003117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5649528eda00 0x5649528f2500 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009900 tx=0x7f63f0005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6300bfe5806b96a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.679+0000 7f64016df640 1 --2- 172.21.15.37:0/4117522552 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f63ec0a2d50 0x7f63ec0a5210 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f63f40984c0 tx=0x7f63f4066770 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.679+0000 7f63e6ffd640 1 -- 172.21.15.37:0/2733003117 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f0040070 con 0x5649528eda00 2024-06-10T00:57:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.680+0000 7f63e6ffd640 1 -- 172.21.15.37:0/2733003117 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63f0032e20 con 0x5649528eda00 2024-06-10T00:57:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.680+0000 7f63e6ffd640 1 -- 172.21.15.37:0/2733003117 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f0035040 con 0x5649528eda00 2024-06-10T00:57:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.680+0000 7f6405e53740 1 -- 172.21.15.37:0/2733003117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5649528eda00 msgr2=0x5649528f2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.680+0000 7f6405e53740 1 --2- 172.21.15.37:0/2733003117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5649528eda00 0x5649528f2500 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009900 tx=0x7f63f0005c00 comp rx=0 tx=0).stop 2024-06-10T00:57:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.680+0000 7f6405e53740 1 -- 172.21.15.37:0/2733003117 shutdown_connections 2024-06-10T00:57:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.680+0000 7f6405e53740 1 --2- 172.21.15.37:0/2733003117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649528f2b70 0x5649528f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.680+0000 7f6405e53740 1 --2- 172.21.15.37:0/2733003117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5649528eda00 0x5649528f2500 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.680+0000 7f6405e53740 1 --2- 172.21.15.37:0/2733003117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649528ed010 0x5649528ed430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.680+0000 7f6405e53740 1 -- 172.21.15.37:0/2733003117 >> 172.21.15.37:0/2733003117 conn(0x5649528e8680 msgr2=0x5649528e8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.680+0000 7f6405e53740 1 -- 172.21.15.37:0/2733003117 shutdown_connections 2024-06-10T00:57:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.681+0000 7f6405e53740 1 -- 172.21.15.37:0/2733003117 wait complete. 2024-06-10T00:57:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.681+0000 7f6405e53740 1 -- start start 2024-06-10T00:57:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.681+0000 7f6405e53740 1 Processor -- start 2024-06-10T00:57:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.681+0000 7f6405e53740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5649528ed010 0x5649528f7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.682+0000 7f6405e53740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649528eda00 0x5649528f7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.682+0000 7f6405e53740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649528f2b70 0x5649528fe7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.682+0000 7f6405e53740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5649528f5ac0 con 0x5649528ed010 2024-06-10T00:57:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.682+0000 7f6405e53740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5649527d9ae0 con 0x5649528f2b70 2024-06-10T00:57:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.682+0000 7f6405e53740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5649528f5c40 con 0x5649528eda00 2024-06-10T00:57:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.682+0000 7f6405e53740 10 client.0 _ll_register_callbacks cb 0x5649528129b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.682+0000 7f63fbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5649528ed010 0x5649528f7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.682+0000 7f63fbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5649528ed010 0x5649528f7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55008/0 (socket says 172.21.15.37:55008) 2024-06-10T00:57:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.682+0000 7f63fbfff640 1 -- 172.21.15.37:0/3251288370 learned_addr learned my addr 172.21.15.37:0/3251288370 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.682+0000 7f6400ede640 1 --2- 172.21.15.37:0/3251288370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649528eda00 0x5649528f7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.682+0000 7f63fbfff640 1 --2- 172.21.15.37:0/3251288370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649528f2b70 0x5649528fe7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.682+0000 7f63fbfff640 1 -- 172.21.15.37:0/3251288370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649528eda00 msgr2=0x5649528f7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.682+0000 7f63fbfff640 1 --2- 172.21.15.37:0/3251288370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649528eda00 0x5649528f7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.682+0000 7f63fbfff640 1 -- 172.21.15.37:0/3251288370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649528f2b70 msgr2=0x5649528fe7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.682+0000 7f63fbfff640 1 --2- 172.21.15.37:0/3251288370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649528f2b70 0x5649528fe7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.682+0000 7f63fbfff640 1 -- 172.21.15.37:0/3251288370 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564952564210 con 0x5649528ed010 2024-06-10T00:57:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.682+0000 7f63fbfff640 1 --2- 172.21.15.37:0/3251288370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5649528ed010 0x5649528f7230 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f63e800c3d0 tx=0x7f63e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd564b86039ac9d9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.682+0000 7f63dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:57:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.682+0000 7f63dbfff640 1 -- 172.21.15.37:0/3251288370 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e8005020 con 0x5649528ed010 2024-06-10T00:57:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.682+0000 7f63dbfff640 1 -- 172.21.15.37:0/3251288370 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f63e80051c0 con 0x5649528ed010 2024-06-10T00:57:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.683+0000 7f6405e53740 1 -- 172.21.15.37:0/3251288370 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63f0032e20 con 0x5649528ed010 2024-06-10T00:57:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.683+0000 7f63dbfff640 1 -- 172.21.15.37:0/3251288370 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f63e8027050 con 0x5649528ed010 2024-06-10T00:57:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.683+0000 7f6405e53740 10 client.15477 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.683+0000 7f6405e53740 1 -- 172.21.15.37:0/3251288370 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564952630620 con 0x5649528ed010 2024-06-10T00:57:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.683+0000 7f63dbfff640 1 -- 172.21.15.37:0/3251288370 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63f0032e20 con 0x5649528ed010 2024-06-10T00:57:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.683+0000 7f63dbfff640 1 -- 172.21.15.37:0/3251288370 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f63e80214b0 con 0x5649528ed010 2024-06-10T00:57:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.683+0000 7f6405e53740 10 client.15477 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.683+0000 7f6405e53740 10 client.15477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T00:57:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.683+0000 7f6405e53740 20 client.15477 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.683+0000 7f6405e53740 1 --2- 172.21.15.37:0/3251288370 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564952903060 0x5649529054d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.683+0000 7f6405e53740 4 client.15477 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:57:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.683+0000 7f6405e53740 1 -- 172.21.15.37:0/3251288370 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564952863f10 con 0x564952903060 2024-06-10T00:57:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.684+0000 7f6400ede640 1 --2- 172.21.15.37:0/3251288370 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564952903060 0x5649529054d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.684+0000 7f6400ede640 1 --2- 172.21.15.37:0/3251288370 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564952903060 0x5649529054d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b11ef4768a0e10a0 server_cookie=ffef2b745e6305d6 in_seq=0 out_seq=0 2024-06-10T00:57:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.684+0000 7f63dbfff640 10 client.15477 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.685+0000 7f63dbfff640 1 -- 172.21.15.37:0/3251288370 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564952863f10 con 0x564952903060 2024-06-10T00:57:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.685+0000 7f63dbfff640 10 client.15477 handle_command_reply: tid=0 2024-06-10T00:57:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.685+0000 7f6405e53740 1 client.15477 shutdown 2024-06-10T00:57:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.685+0000 7f6405e53740 1 -- 172.21.15.37:0/3251288370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5649528ed010 msgr2=0x5649528f7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.685+0000 7f6405e53740 1 --2- 172.21.15.37:0/3251288370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5649528ed010 0x5649528f7230 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f63e800c3d0 tx=0x7f63e8004730 comp rx=0 tx=0).stop 2024-06-10T00:57:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.685+0000 7f6405e53740 1 -- 172.21.15.37:0/3251288370 shutdown_connections 2024-06-10T00:57:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.685+0000 7f6405e53740 1 --2- 172.21.15.37:0/3251288370 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564952903060 0x5649529054d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.685+0000 7f6405e53740 1 --2- 172.21.15.37:0/3251288370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649528f2b70 0x5649528fe7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.685+0000 7f63dbfff640 0 client.15477 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.685+0000 7f6405e53740 1 --2- 172.21.15.37:0/3251288370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649528eda00 0x5649528f7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.685+0000 7f6405e53740 1 --2- 172.21.15.37:0/3251288370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5649528ed010 0x5649528f7230 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.685+0000 7f6405e53740 1 -- 172.21.15.37:0/3251288370 >> 172.21.15.37:0/3251288370 conn(0x5649528e8680 msgr2=0x5649528e77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.685+0000 7f6405e53740 1 -- 172.21.15.37:0/3251288370 shutdown_connections 2024-06-10T00:57:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.685+0000 7f6405e53740 1 -- 172.21.15.37:0/3251288370 wait complete. 2024-06-10T00:57:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.685+0000 7f6405e53740 20 client.15477 trim_cache size 0 max 16384 2024-06-10T00:57:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.701+0000 7f6405e53740 1 Processor -- start 2024-06-10T00:57:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.701+0000 7f6405e53740 1 -- start start 2024-06-10T00:57:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.701+0000 7f6405e53740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649528ed010 0x5649528f6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.701+0000 7f6405e53740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5649528eda00 0x5649528fe770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.701+0000 7f6405e53740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649528f2b70 0x5649528f7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.701+0000 7f6405e53740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc1af1c0 con 0x5649528eda00 2024-06-10T00:57:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.701+0000 7f6405e53740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc1aca90 con 0x5649528ed010 2024-06-10T00:57:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.701+0000 7f6405e53740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc1aa8e0 con 0x5649528f2b70 2024-06-10T00:57:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.701+0000 7f6400ede640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5649528eda00 0x5649528fe770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.701+0000 7f6400ede640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5649528eda00 0x5649528fe770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55018/0 (socket says 172.21.15.37:55018) 2024-06-10T00:57:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.701+0000 7f6400ede640 1 -- 172.21.15.37:0/2305489376 learned_addr learned my addr 172.21.15.37:0/2305489376 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.701+0000 7f63fbfff640 1 --2- 172.21.15.37:0/2305489376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649528f2b70 0x5649528f7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.701+0000 7f63fbfff640 1 --2- 172.21.15.37:0/2305489376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649528ed010 0x5649528f6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.702+0000 7f6400ede640 1 -- 172.21.15.37:0/2305489376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649528f2b70 msgr2=0x5649528f7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.702+0000 7f6400ede640 1 --2- 172.21.15.37:0/2305489376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649528f2b70 0x5649528f7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.702+0000 7f6400ede640 1 -- 172.21.15.37:0/2305489376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649528ed010 msgr2=0x5649528f6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.702+0000 7f6400ede640 1 --2- 172.21.15.37:0/2305489376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649528ed010 0x5649528f6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.702+0000 7f6400ede640 1 -- 172.21.15.37:0/2305489376 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649528ed7a0 con 0x5649528eda00 2024-06-10T00:57:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.702+0000 7f6400ede640 1 --2- 172.21.15.37:0/2305489376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5649528eda00 0x5649528fe770 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009900 tx=0x7f63f002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406925f86cd73e3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.702+0000 7f63e67fc640 1 -- 172.21.15.37:0/2305489376 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f0053030 con 0x5649528eda00 2024-06-10T00:57:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.702+0000 7f63e67fc640 1 -- 172.21.15.37:0/2305489376 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63f0032da0 con 0x5649528eda00 2024-06-10T00:57:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.702+0000 7f63e67fc640 1 -- 172.21.15.37:0/2305489376 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f0035040 con 0x5649528eda00 2024-06-10T00:57:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.703+0000 7f6405e53740 1 -- 172.21.15.37:0/2305489376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5649528eda00 msgr2=0x5649528fe770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.703+0000 7f6405e53740 1 --2- 172.21.15.37:0/2305489376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5649528eda00 0x5649528fe770 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009900 tx=0x7f63f002a420 comp rx=0 tx=0).stop 2024-06-10T00:57:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.703+0000 7f6405e53740 1 -- 172.21.15.37:0/2305489376 shutdown_connections 2024-06-10T00:57:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.703+0000 7f6405e53740 1 --2- 172.21.15.37:0/2305489376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649528f2b70 0x5649528f7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.703+0000 7f6405e53740 1 --2- 172.21.15.37:0/2305489376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5649528eda00 0x5649528fe770 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.703+0000 7f6405e53740 1 --2- 172.21.15.37:0/2305489376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649528ed010 0x5649528f6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.703+0000 7f6405e53740 1 -- 172.21.15.37:0/2305489376 >> 172.21.15.37:0/2305489376 conn(0x5649528e8680 msgr2=0x5649528eec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.703+0000 7f6405e53740 1 -- 172.21.15.37:0/2305489376 shutdown_connections 2024-06-10T00:57:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.703+0000 7f6405e53740 1 -- 172.21.15.37:0/2305489376 wait complete. 2024-06-10T00:57:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.703+0000 7f6405e53740 1 -- start start 2024-06-10T00:57:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.703+0000 7f6405e53740 1 Processor -- start 2024-06-10T00:57:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.704+0000 7f6405e53740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649528ed010 0x5649528fe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.704+0000 7f6405e53740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649528eda00 0x5649528fb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.704+0000 7f6405e53740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5649528f2b70 0x5649528fb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.704+0000 7f6405e53740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc1aaf50 con 0x5649528f2b70 2024-06-10T00:57:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.704+0000 7f6405e53740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc1ac3c0 con 0x5649528eda00 2024-06-10T00:57:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.704+0000 7f6405e53740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc1ab620 con 0x5649528ed010 2024-06-10T00:57:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.704+0000 7f6405e53740 10 client.0 _ll_register_callbacks cb 0x7f63e800ce10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.704+0000 7f63fbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5649528f2b70 0x5649528fb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.704+0000 7f6400ede640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649528eda00 0x5649528fb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.704+0000 7f6400ede640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649528eda00 0x5649528fb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:60558/0 (socket says 172.21.15.37:60558) 2024-06-10T00:57:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.704+0000 7f63fbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5649528f2b70 0x5649528fb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55022/0 (socket says 172.21.15.37:55022) 2024-06-10T00:57:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.704+0000 7f6400ede640 1 -- 172.21.15.37:0/1031799005 learned_addr learned my addr 172.21.15.37:0/1031799005 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.704+0000 7f63fbfff640 1 --2- 172.21.15.37:0/1031799005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649528ed010 0x5649528fe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.704+0000 7f6400ede640 1 -- 172.21.15.37:0/1031799005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649528ed010 msgr2=0x5649528fe5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.704+0000 7f6400ede640 1 --2- 172.21.15.37:0/1031799005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649528ed010 0x5649528fe5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.704+0000 7f6400ede640 1 -- 172.21.15.37:0/1031799005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5649528f2b70 msgr2=0x5649528fb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.704+0000 7f6400ede640 1 --2- 172.21.15.37:0/1031799005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5649528f2b70 0x5649528fb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.704+0000 7f6400ede640 1 -- 172.21.15.37:0/1031799005 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56495296bda0 con 0x5649528eda00 2024-06-10T00:57:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.704+0000 7f63fbfff640 1 --2- 172.21.15.37:0/1031799005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649528ed010 0x5649528fe5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:57:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.705+0000 7f6400ede640 1 --2- 172.21.15.37:0/1031799005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649528eda00 0x5649528fb080 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009850 tx=0x7f63f0005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ce9c23419f37066 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.705+0000 7f63dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T00:57:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.705+0000 7f63dbfff640 1 -- 172.21.15.37:0/1031799005 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f0053040 con 0x5649528eda00 2024-06-10T00:57:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.705+0000 7f63dbfff640 1 -- 172.21.15.37:0/1031799005 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f63f004e440 con 0x5649528eda00 2024-06-10T00:57:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.705+0000 7f6405e53740 1 -- 172.21.15.37:0/1031799005 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63f0032da0 con 0x5649528eda00 2024-06-10T00:57:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.705+0000 7f63dbfff640 1 -- 172.21.15.37:0/1031799005 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f63f0066050 con 0x5649528eda00 2024-06-10T00:57:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.705+0000 7f6405e53740 10 client.25100 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.705+0000 7f6405e53740 1 -- 172.21.15.37:0/1031799005 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56495296cf30 con 0x5649528eda00 2024-06-10T00:57:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.705+0000 7f63dbfff640 1 -- 172.21.15.37:0/1031799005 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63f0032da0 con 0x5649528eda00 2024-06-10T00:57:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.705+0000 7f63dbfff640 1 -- 172.21.15.37:0/1031799005 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f63f003c7c0 con 0x5649528eda00 2024-06-10T00:57:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.706+0000 7f6405e53740 10 client.25100 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.706+0000 7f6405e53740 10 client.25100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T00:57:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.706+0000 7f6405e53740 20 client.25100 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.706+0000 7f6405e53740 1 --2- 172.21.15.37:0/1031799005 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564952903060 0x56495296f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.706+0000 7f6405e53740 4 client.25100 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T00:57:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.706+0000 7f6405e53740 1 -- 172.21.15.37:0/1031799005 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56495285dc20 con 0x564952903060 2024-06-10T00:57:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.706+0000 7f63fbfff640 1 --2- 172.21.15.37:0/1031799005 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564952903060 0x56495296f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.706+0000 7f63dbfff640 10 client.25100 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.706+0000 7f63fbfff640 1 --2- 172.21.15.37:0/1031799005 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564952903060 0x56495296f4f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df7422190509229 server_cookie=323cc1871c98bcda in_seq=0 out_seq=0 2024-06-10T00:57:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.709+0000 7f63dbfff640 1 -- 172.21.15.37:0/1031799005 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56495285dc20 con 0x564952903060 2024-06-10T00:57:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.709+0000 7f63dbfff640 10 client.25100 handle_command_reply: tid=0 2024-06-10T00:57:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.709+0000 7f6405e53740 1 client.25100 shutdown 2024-06-10T00:57:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.710+0000 7f6405e53740 1 -- 172.21.15.37:0/1031799005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649528eda00 msgr2=0x5649528fb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.710+0000 7f6405e53740 1 --2- 172.21.15.37:0/1031799005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649528eda00 0x5649528fb080 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009850 tx=0x7f63f0005fd0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.710+0000 7f6405e53740 1 -- 172.21.15.37:0/1031799005 shutdown_connections 2024-06-10T00:57:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.710+0000 7f6405e53740 1 --2- 172.21.15.37:0/1031799005 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564952903060 0x56495296f4f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.710+0000 7f6405e53740 1 --2- 172.21.15.37:0/1031799005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5649528f2b70 0x5649528fb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.710+0000 7f63dbfff640 0 client.25100 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.710+0000 7f6405e53740 1 --2- 172.21.15.37:0/1031799005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5649528eda00 0x5649528fb080 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.710+0000 7f6405e53740 1 --2- 172.21.15.37:0/1031799005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5649528ed010 0x5649528fe5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.710+0000 7f6405e53740 1 -- 172.21.15.37:0/1031799005 >> 172.21.15.37:0/1031799005 conn(0x5649528e8680 msgr2=0x5649528f6dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.710+0000 7f6405e53740 1 -- 172.21.15.37:0/1031799005 shutdown_connections 2024-06-10T00:57:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.710+0000 7f6405e53740 1 -- 172.21.15.37:0/1031799005 wait complete. 2024-06-10T00:57:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.710+0000 7f6405e53740 20 client.25100 trim_cache size 0 max 16384 2024-06-10T00:57:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T00:57:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T00:57:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T00:57:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.711+0000 7f6403142640 1 -- 172.21.15.37:0/4117522552 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f63ec0a2d50 msgr2=0x7f63ec0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.711+0000 7f6403142640 1 --2- 172.21.15.37:0/4117522552 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f63ec0a2d50 0x7f63ec0a5210 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f63f40984c0 tx=0x7f63f4066770 comp rx=0 tx=0).stop 2024-06-10T00:57:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.711+0000 7f6403142640 1 -- 172.21.15.37:0/4117522552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63fc103640 msgr2=0x7f63fc101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.711+0000 7f6403142640 1 --2- 172.21.15.37:0/4117522552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63fc103640 0x7f63fc101170 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f63f400eaa0 tx=0x7f63f400ef60 comp rx=0 tx=0).stop 2024-06-10T00:57:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.711+0000 7f6403142640 1 -- 172.21.15.37:0/4117522552 shutdown_connections 2024-06-10T00:57:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.711+0000 7f6403142640 1 --2- 172.21.15.37:0/4117522552 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f63ec0a2d50 0x7f63ec0a5210 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.711+0000 7f6403142640 1 --2- 172.21.15.37:0/4117522552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63fc103640 0x7f63fc101170 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.711+0000 7f6403142640 1 --2- 172.21.15.37:0/4117522552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63fc102c80 0x7f63fc100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.711+0000 7f6403142640 1 --2- 172.21.15.37:0/4117522552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63fc0ffcb0 0x7f63fc1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.711+0000 7f6403142640 1 -- 172.21.15.37:0/4117522552 >> 172.21.15.37:0/4117522552 conn(0x7f63fc0fb600 msgr2=0x7f63fc106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.711+0000 7f6403142640 1 -- 172.21.15.37:0/4117522552 shutdown_connections 2024-06-10T00:57:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:22.711+0000 7f6403142640 1 -- 172.21.15.37:0/4117522552 wait complete. 2024-06-10T00:57:23.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.139+0000 7f281192b700 1 -- 172.21.15.37:0/1916491650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f280c06b3c0 msgr2=0x7f280c10f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:23.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.139+0000 7f281192b700 1 --2- 172.21.15.37:0/1916491650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f280c06b3c0 0x7f280c10f310 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f280400b0a0 tx=0x7f280400b3b0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.139+0000 7f281192b700 1 -- 172.21.15.37:0/1916491650 shutdown_connections 2024-06-10T00:57:23.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.139+0000 7f281192b700 1 --2- 172.21.15.37:0/1916491650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f280c06b3c0 0x7f280c10f310 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.139+0000 7f281192b700 1 --2- 172.21.15.37:0/1916491650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f280c1084f0 0x7f280c1088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.139+0000 7f281192b700 1 --2- 172.21.15.37:0/1916491650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f280c06aa00 0x7f280c06ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.139+0000 7f281192b700 1 -- 172.21.15.37:0/1916491650 >> 172.21.15.37:0/1916491650 conn(0x7f280c078300 msgr2=0x7f280c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:23.141 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:23 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:23.141 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:23 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:57:23.141 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:23 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:57:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.139+0000 7f281192b700 1 -- 172.21.15.37:0/1916491650 shutdown_connections 2024-06-10T00:57:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.139+0000 7f281192b700 1 -- 172.21.15.37:0/1916491650 wait complete. 2024-06-10T00:57:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.140+0000 7f281192b700 1 Processor -- start 2024-06-10T00:57:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.140+0000 7f281192b700 1 -- start start 2024-06-10T00:57:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.141+0000 7f281192b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f280c06aa00 0x7f280c19add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.141+0000 7f281192b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f280c06b3c0 0x7f280c19b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.141+0000 7f281192b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f280c1084f0 0x7f280c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.141+0000 7f281192b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f280c102010 con 0x7f280c06b3c0 2024-06-10T00:57:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.141+0000 7f281192b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f280c101e90 con 0x7f280c1084f0 2024-06-10T00:57:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.141+0000 7f281192b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f280c102190 con 0x7f280c06aa00 2024-06-10T00:57:23.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.141+0000 7f280a7fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f280c06b3c0 0x7f280c19b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.141+0000 7f280affd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f280c06aa00 0x7f280c19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.141+0000 7f280affd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f280c06aa00 0x7f280c19add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:54950/0 (socket says 172.21.15.37:54950) 2024-06-10T00:57:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.141+0000 7f280affd700 1 -- 172.21.15.37:0/353517175 learned_addr learned my addr 172.21.15.37:0/353517175 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.141+0000 7f280b7fe700 1 --2- 172.21.15.37:0/353517175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f280c1084f0 0x7f280c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.141+0000 7f280a7fc700 1 -- 172.21.15.37:0/353517175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f280c06aa00 msgr2=0x7f280c19add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.141+0000 7f280a7fc700 1 --2- 172.21.15.37:0/353517175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f280c06aa00 0x7f280c19add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.141+0000 7f280a7fc700 1 -- 172.21.15.37:0/353517175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f280c1084f0 msgr2=0x7f280c19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.141+0000 7f280a7fc700 1 --2- 172.21.15.37:0/353517175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f280c1084f0 0x7f280c19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.141+0000 7f280a7fc700 1 -- 172.21.15.37:0/353517175 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280c19fd90 con 0x7f280c06b3c0 2024-06-10T00:57:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.141+0000 7f280affd700 1 --2- 172.21.15.37:0/353517175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f280c06aa00 0x7f280c19add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:57:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.142+0000 7f280a7fc700 1 --2- 172.21.15.37:0/353517175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f280c06b3c0 0x7f280c19b310 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f280000d950 tx=0x7f280000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:23.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.142+0000 7f2810929700 1 -- 172.21.15.37:0/353517175 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2800016070 con 0x7f280c06b3c0 2024-06-10T00:57:23.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.142+0000 7f281192b700 1 -- 172.21.15.37:0/353517175 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f280c1a0080 con 0x7f280c06b3c0 2024-06-10T00:57:23.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.142+0000 7f281192b700 1 -- 172.21.15.37:0/353517175 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f280c1a7930 con 0x7f280c06b3c0 2024-06-10T00:57:23.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.142+0000 7f2810929700 1 -- 172.21.15.37:0/353517175 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f280000fd70 con 0x7f280c06b3c0 2024-06-10T00:57:23.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.142+0000 7f2810929700 1 -- 172.21.15.37:0/353517175 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280000e3b0 con 0x7f280c06b3c0 2024-06-10T00:57:23.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.145+0000 7f2810929700 1 -- 172.21.15.37:0/353517175 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f2800019070 con 0x7f280c06b3c0 2024-06-10T00:57:23.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.145+0000 7f2810929700 1 --2- 172.21.15.37:0/353517175 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f27f40a2d30 0x7f27f40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:23.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.145+0000 7f2810929700 1 -- 172.21.15.37:0/353517175 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f28000cee10 con 0x7f280c06b3c0 2024-06-10T00:57:23.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.145+0000 7f280affd700 1 --2- 172.21.15.37:0/353517175 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f27f40a2d30 0x7f27f40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:23.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.146+0000 7f280affd700 1 --2- 172.21.15.37:0/353517175 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f27f40a2d30 0x7f27f40a51f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f27fc0104e0 tx=0x7f27fc00b480 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:23.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.146+0000 7f281c1aeb80 1 Processor -- start 2024-06-10T00:57:23.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.146+0000 7f281c1aeb80 1 -- start start 2024-06-10T00:57:23.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.147+0000 7f281c1aeb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558112904640 0x558112904a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:23.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.147+0000 7f281c1aeb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558112904f60 0x5581128fe9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:23.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.147+0000 7f281c1aeb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581128fef20 0x5581128ff340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:23.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.147+0000 7f281c1aeb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558112850810 con 0x5581128fef20 2024-06-10T00:57:23.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.147+0000 7f281c1aeb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5581127f8720 con 0x558112904640 2024-06-10T00:57:23.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.147+0000 7f281c1aeb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55811284c7d0 con 0x558112904f60 2024-06-10T00:57:23.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.147+0000 7f280b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581128fef20 0x5581128ff340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:23.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.147+0000 7f280affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558112904640 0x558112904a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:23.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.147+0000 7f280affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558112904640 0x558112904a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:60594/0 (socket says 172.21.15.37:60594) 2024-06-10T00:57:23.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.147+0000 7f280affd700 1 -- 172.21.15.37:0/1534651731 learned_addr learned my addr 172.21.15.37:0/1534651731 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:23.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.147+0000 7f280a7fc700 1 --2- 172.21.15.37:0/1534651731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558112904f60 0x5581128fe9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:23.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.147+0000 7f280affd700 1 -- 172.21.15.37:0/1534651731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558112904f60 msgr2=0x5581128fe9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:23.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.147+0000 7f280affd700 1 --2- 172.21.15.37:0/1534651731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558112904f60 0x5581128fe9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.147+0000 7f280affd700 1 -- 172.21.15.37:0/1534651731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581128fef20 msgr2=0x5581128ff340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:23.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.147+0000 7f280affd700 1 --2- 172.21.15.37:0/1534651731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581128fef20 0x5581128ff340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.147+0000 7f280affd700 1 -- 172.21.15.37:0/1534651731 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581127eef50 con 0x558112904640 2024-06-10T00:57:23.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.147+0000 7f280b7fe700 1 --2- 172.21.15.37:0/1534651731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581128fef20 0x5581128ff340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:57:23.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.147+0000 7f280affd700 1 --2- 172.21.15.37:0/1534651731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558112904640 0x558112904a20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f27fc0116e0 tx=0x7f27fc013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e03614367c215a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:23.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.148+0000 7f27f1ffb700 1 -- 172.21.15.37:0/1534651731 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc01e620 con 0x558112904640 2024-06-10T00:57:23.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.148+0000 7f27f1ffb700 1 -- 172.21.15.37:0/1534651731 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27fc01eca0 con 0x558112904640 2024-06-10T00:57:23.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.148+0000 7f27f1ffb700 1 -- 172.21.15.37:0/1534651731 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc01cd80 con 0x558112904640 2024-06-10T00:57:23.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.148+0000 7f281c1aeb80 1 -- 172.21.15.37:0/1534651731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558112904640 msgr2=0x558112904a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:23.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.148+0000 7f281c1aeb80 1 --2- 172.21.15.37:0/1534651731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558112904640 0x558112904a20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f27fc0116e0 tx=0x7f27fc013000 comp rx=0 tx=0).stop 2024-06-10T00:57:23.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.148+0000 7f281c1aeb80 1 -- 172.21.15.37:0/1534651731 shutdown_connections 2024-06-10T00:57:23.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.148+0000 7f281c1aeb80 1 --2- 172.21.15.37:0/1534651731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581128fef20 0x5581128ff340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.148+0000 7f281c1aeb80 1 --2- 172.21.15.37:0/1534651731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558112904640 0x558112904a20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.148+0000 7f281c1aeb80 1 --2- 172.21.15.37:0/1534651731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558112904f60 0x5581128fe9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.148+0000 7f281c1aeb80 1 -- 172.21.15.37:0/1534651731 >> 172.21.15.37:0/1534651731 conn(0x558112829cc0 msgr2=0x5581128f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:23.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.149+0000 7f281c1aeb80 1 -- 172.21.15.37:0/1534651731 shutdown_connections 2024-06-10T00:57:23.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.149+0000 7f281c1aeb80 1 -- 172.21.15.37:0/1534651731 wait complete. 2024-06-10T00:57:23.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.149+0000 7f281c1aeb80 1 -- start start 2024-06-10T00:57:23.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.149+0000 7f281c1aeb80 1 Processor -- start 2024-06-10T00:57:23.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.150+0000 7f281c1aeb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581128fef20 0x55811290acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:23.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.150+0000 7f281c1aeb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558112904640 0x558112907fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:23.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.150+0000 7f281c1aeb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558112904f60 0x558112908500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:23.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.150+0000 7f281c1aeb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55811290b9d0 con 0x5581128fef20 2024-06-10T00:57:23.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.150+0000 7f281c1aeb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5581127f0bf0 con 0x558112904f60 2024-06-10T00:57:23.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.150+0000 7f281c1aeb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55811290bb50 con 0x558112904640 2024-06-10T00:57:23.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.150+0000 7f281c1aeb80 10 client.0 _ll_register_callbacks cb 0x5581127ea920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:23.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.150+0000 7f280affd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581128fef20 0x55811290acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:23.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.150+0000 7f280affd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581128fef20 0x55811290acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55064/0 (socket says 172.21.15.37:55064) 2024-06-10T00:57:23.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.150+0000 7f280affd700 1 -- 172.21.15.37:0/509695890 learned_addr learned my addr 172.21.15.37:0/509695890 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:23.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.150+0000 7f280a7fc700 1 --2- 172.21.15.37:0/509695890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558112904640 0x558112907fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:23.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.150+0000 7f280b7fe700 1 --2- 172.21.15.37:0/509695890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558112904f60 0x558112908500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:23.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.150+0000 7f280affd700 1 -- 172.21.15.37:0/509695890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558112904640 msgr2=0x558112907fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:23.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.150+0000 7f280affd700 1 --2- 172.21.15.37:0/509695890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558112904640 0x558112907fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.150+0000 7f280affd700 1 -- 172.21.15.37:0/509695890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558112904f60 msgr2=0x558112908500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:23.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.150+0000 7f280affd700 1 --2- 172.21.15.37:0/509695890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558112904f60 0x558112908500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.150+0000 7f280affd700 1 -- 172.21.15.37:0/509695890 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55811279f8d0 con 0x5581128fef20 2024-06-10T00:57:23.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.150+0000 7f280a7fc700 1 --2- 172.21.15.37:0/509695890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558112904640 0x558112907fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:57:23.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.150+0000 7f280affd700 1 --2- 172.21.15.37:0/509695890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581128fef20 0x55811290acc0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f27fc0116b0 tx=0x7f27fc00bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8e072eecbbd4aa server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:23.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.150+0000 7f27eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:57:23.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.151+0000 7f27eaffd700 1 -- 172.21.15.37:0/509695890 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc036060 con 0x5581128fef20 2024-06-10T00:57:23.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.151+0000 7f27eaffd700 1 -- 172.21.15.37:0/509695890 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f27fc01c9c0 con 0x5581128fef20 2024-06-10T00:57:23.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.151+0000 7f281c1aeb80 1 -- 172.21.15.37:0/509695890 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27fc01eca0 con 0x5581128fef20 2024-06-10T00:57:23.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.151+0000 7f27eaffd700 1 -- 172.21.15.37:0/509695890 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f27fc03d050 con 0x5581128fef20 2024-06-10T00:57:23.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.151+0000 7f281c1aeb80 10 client.15495 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:23.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.151+0000 7f281c1aeb80 1 -- 172.21.15.37:0/509695890 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55811291eb20 con 0x5581128fef20 2024-06-10T00:57:23.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.151+0000 7f27eaffd700 1 -- 172.21.15.37:0/509695890 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27fc01eca0 con 0x5581128fef20 2024-06-10T00:57:23.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.151+0000 7f27eaffd700 1 -- 172.21.15.37:0/509695890 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f27fc01e5d0 con 0x5581128fef20 2024-06-10T00:57:23.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.151+0000 7f281c1aeb80 10 client.15495 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:23.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.151+0000 7f281c1aeb80 10 client.15495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T00:57:23.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.151+0000 7f281c1aeb80 20 client.15495 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:23.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.151+0000 7f281c1aeb80 1 --2- 172.21.15.37:0/509695890 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558112921100 0x5581129234f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:23.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.151+0000 7f281c1aeb80 4 client.15495 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:57:23.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.151+0000 7f281c1aeb80 1 -- 172.21.15.37:0/509695890 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55811285dd10 con 0x558112921100 2024-06-10T00:57:23.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.152+0000 7f280a7fc700 1 --2- 172.21.15.37:0/509695890 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558112921100 0x5581129234f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:23.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.152+0000 7f27eaffd700 10 client.15495 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:23.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.152+0000 7f280a7fc700 1 --2- 172.21.15.37:0/509695890 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558112921100 0x5581129234f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3728547d950d203 server_cookie=a5b5141c8fa51ce7 in_seq=0 out_seq=0 2024-06-10T00:57:23.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.153+0000 7f27eaffd700 1 -- 172.21.15.37:0/509695890 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55811285dd10 con 0x558112921100 2024-06-10T00:57:23.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.153+0000 7f27eaffd700 10 client.15495 handle_command_reply: tid=0 2024-06-10T00:57:23.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.153+0000 7f281c1aeb80 1 client.15495 shutdown 2024-06-10T00:57:23.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.153+0000 7f281c1aeb80 1 -- 172.21.15.37:0/509695890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581128fef20 msgr2=0x55811290acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:23.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.153+0000 7f281c1aeb80 1 --2- 172.21.15.37:0/509695890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581128fef20 0x55811290acc0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f27fc0116b0 tx=0x7f27fc00bee0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.153+0000 7f281c1aeb80 1 -- 172.21.15.37:0/509695890 shutdown_connections 2024-06-10T00:57:23.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.153+0000 7f281c1aeb80 1 --2- 172.21.15.37:0/509695890 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558112921100 0x5581129234f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.153+0000 7f281c1aeb80 1 --2- 172.21.15.37:0/509695890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558112904f60 0x558112908500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.153+0000 7f27eaffd700 0 client.15495 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:23.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.153+0000 7f281c1aeb80 1 --2- 172.21.15.37:0/509695890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581128fef20 0x55811290acc0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.153+0000 7f281c1aeb80 1 --2- 172.21.15.37:0/509695890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558112904640 0x558112907fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.153+0000 7f281c1aeb80 1 -- 172.21.15.37:0/509695890 >> 172.21.15.37:0/509695890 conn(0x558112829cc0 msgr2=0x5581128fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:23.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.153+0000 7f281c1aeb80 1 -- 172.21.15.37:0/509695890 shutdown_connections 2024-06-10T00:57:23.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.153+0000 7f281c1aeb80 1 -- 172.21.15.37:0/509695890 wait complete. 2024-06-10T00:57:23.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.153+0000 7f281c1aeb80 20 client.15495 trim_cache size 0 max 16384 2024-06-10T00:57:23.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.192+0000 7f281c1aeb80 1 Processor -- start 2024-06-10T00:57:23.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.192+0000 7f281c1aeb80 1 -- start start 2024-06-10T00:57:23.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.192+0000 7f281c1aeb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581128fef20 0x5581128fe9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:23.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.192+0000 7f281c1aeb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558112904640 0x5581128f3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:23.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.192+0000 7f281c1aeb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558112904f60 0x5581129097f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:23.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.192+0000 7f281c1aeb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f27f8011500 con 0x5581128fef20 2024-06-10T00:57:23.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.192+0000 7f281c1aeb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f27f800ed90 con 0x558112904640 2024-06-10T00:57:23.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.192+0000 7f281c1aeb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f27f800cbe0 con 0x558112904f60 2024-06-10T00:57:23.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.192+0000 7f280affd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581128fef20 0x5581128fe9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:23.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.192+0000 7f280affd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581128fef20 0x5581128fe9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55074/0 (socket says 172.21.15.37:55074) 2024-06-10T00:57:23.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.192+0000 7f280affd700 1 -- 172.21.15.37:0/2812479459 learned_addr learned my addr 172.21.15.37:0/2812479459 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:23.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.192+0000 7f280b7fe700 1 --2- 172.21.15.37:0/2812479459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558112904f60 0x5581129097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:23.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.192+0000 7f280a7fc700 1 --2- 172.21.15.37:0/2812479459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558112904640 0x5581128f3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:23.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.192+0000 7f280affd700 1 -- 172.21.15.37:0/2812479459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558112904f60 msgr2=0x5581129097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:23.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.192+0000 7f280affd700 1 --2- 172.21.15.37:0/2812479459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558112904f60 0x5581129097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.192+0000 7f280affd700 1 -- 172.21.15.37:0/2812479459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558112904640 msgr2=0x5581128f3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:23.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.192+0000 7f280affd700 1 --2- 172.21.15.37:0/2812479459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558112904640 0x5581128f3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.192+0000 7f280affd700 1 -- 172.21.15.37:0/2812479459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558112908f60 con 0x5581128fef20 2024-06-10T00:57:23.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.192+0000 7f280a7fc700 1 --2- 172.21.15.37:0/2812479459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558112904640 0x5581128f3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:57:23.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.192+0000 7f280affd700 1 --2- 172.21.15.37:0/2812479459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581128fef20 0x5581128fe9e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f27fc011680 tx=0x7f27fc00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db64563cc325962a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:23.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.193+0000 7f27f17fa700 1 -- 172.21.15.37:0/2812479459 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc036070 con 0x5581128fef20 2024-06-10T00:57:23.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.193+0000 7f27f17fa700 1 -- 172.21.15.37:0/2812479459 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27fc01c3b0 con 0x5581128fef20 2024-06-10T00:57:23.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.193+0000 7f27f17fa700 1 -- 172.21.15.37:0/2812479459 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc01e660 con 0x5581128fef20 2024-06-10T00:57:23.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.193+0000 7f281c1aeb80 1 -- 172.21.15.37:0/2812479459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581128fef20 msgr2=0x5581128fe9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:23.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.193+0000 7f281c1aeb80 1 --2- 172.21.15.37:0/2812479459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581128fef20 0x5581128fe9e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f27fc011680 tx=0x7f27fc00bfd0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.193+0000 7f281c1aeb80 1 -- 172.21.15.37:0/2812479459 shutdown_connections 2024-06-10T00:57:23.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.194+0000 7f281c1aeb80 1 --2- 172.21.15.37:0/2812479459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558112904f60 0x5581129097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.194+0000 7f281c1aeb80 1 --2- 172.21.15.37:0/2812479459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581128fef20 0x5581128fe9e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.194+0000 7f281c1aeb80 1 --2- 172.21.15.37:0/2812479459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558112904640 0x5581128f3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.194+0000 7f281c1aeb80 1 -- 172.21.15.37:0/2812479459 >> 172.21.15.37:0/2812479459 conn(0x558112829cc0 msgr2=0x5581129059b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:23.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.194+0000 7f281c1aeb80 1 -- 172.21.15.37:0/2812479459 shutdown_connections 2024-06-10T00:57:23.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.194+0000 7f281c1aeb80 1 -- 172.21.15.37:0/2812479459 wait complete. 2024-06-10T00:57:23.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.194+0000 7f281c1aeb80 1 -- start start 2024-06-10T00:57:23.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.194+0000 7f281c1aeb80 1 Processor -- start 2024-06-10T00:57:23.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.195+0000 7f281c1aeb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581128fef20 0x558112984380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:23.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.195+0000 7f281c1aeb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558112904640 0x5581129848c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:23.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.195+0000 7f281c1aeb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558112904f60 0x558112926080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:23.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.195+0000 7f281c1aeb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f27f800d250 con 0x5581128fef20 2024-06-10T00:57:23.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.195+0000 7f281c1aeb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f27f800e6c0 con 0x558112904640 2024-06-10T00:57:23.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.195+0000 7f281c1aeb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f27f800d920 con 0x558112904f60 2024-06-10T00:57:23.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.195+0000 7f281c1aeb80 10 client.0 _ll_register_callbacks cb 0x5581127ea920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:23.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.195+0000 7f280affd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581128fef20 0x558112984380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:23.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.195+0000 7f280affd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581128fef20 0x558112984380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55086/0 (socket says 172.21.15.37:55086) 2024-06-10T00:57:23.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.195+0000 7f280affd700 1 -- 172.21.15.37:0/2638300751 learned_addr learned my addr 172.21.15.37:0/2638300751 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:23.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.195+0000 7f280a7fc700 1 --2- 172.21.15.37:0/2638300751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558112904640 0x5581129848c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:23.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.195+0000 7f280b7fe700 1 --2- 172.21.15.37:0/2638300751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558112904f60 0x558112926080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:23.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.195+0000 7f280affd700 1 -- 172.21.15.37:0/2638300751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558112904f60 msgr2=0x558112926080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:23.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.195+0000 7f280affd700 1 --2- 172.21.15.37:0/2638300751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558112904f60 0x558112926080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.195+0000 7f280affd700 1 -- 172.21.15.37:0/2638300751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558112904640 msgr2=0x5581129848c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:23.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.195+0000 7f280affd700 1 --2- 172.21.15.37:0/2638300751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558112904640 0x5581129848c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.195+0000 7f280affd700 1 -- 172.21.15.37:0/2638300751 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558112926890 con 0x5581128fef20 2024-06-10T00:57:23.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.195+0000 7f280a7fc700 1 --2- 172.21.15.37:0/2638300751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558112904640 0x5581129848c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:57:23.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.195+0000 7f280affd700 1 --2- 172.21.15.37:0/2638300751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581128fef20 0x558112984380 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f27fc029040 tx=0x7f27fc00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856d4a1bfe861480 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:23.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.195+0000 7f27eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:57:23.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.195+0000 7f27eaffd700 1 -- 172.21.15.37:0/2638300751 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc036040 con 0x5581128fef20 2024-06-10T00:57:23.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.196+0000 7f27eaffd700 1 -- 172.21.15.37:0/2638300751 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f27fc01e420 con 0x5581128fef20 2024-06-10T00:57:23.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.196+0000 7f281c1aeb80 1 -- 172.21.15.37:0/2638300751 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27fc01c3b0 con 0x5581128fef20 2024-06-10T00:57:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.196+0000 7f27eaffd700 1 -- 172.21.15.37:0/2638300751 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f27fc01e660 con 0x5581128fef20 2024-06-10T00:57:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.196+0000 7f281c1aeb80 10 client.15501 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.196+0000 7f281c1aeb80 1 -- 172.21.15.37:0/2638300751 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55811298ba60 con 0x5581128fef20 2024-06-10T00:57:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.196+0000 7f27eaffd700 1 -- 172.21.15.37:0/2638300751 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27fc01c3b0 con 0x5581128fef20 2024-06-10T00:57:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.196+0000 7f27eaffd700 1 -- 172.21.15.37:0/2638300751 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f27fc04a9a0 con 0x5581128fef20 2024-06-10T00:57:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.196+0000 7f281c1aeb80 10 client.15501 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.196+0000 7f281c1aeb80 10 client.15501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T00:57:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.196+0000 7f281c1aeb80 20 client.15501 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.196+0000 7f281c1aeb80 1 --2- 172.21.15.37:0/2638300751 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558112921100 0x55811298e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.196+0000 7f281c1aeb80 4 client.15501 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T00:57:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.196+0000 7f281c1aeb80 1 -- 172.21.15.37:0/2638300751 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558112858290 con 0x558112921100 2024-06-10T00:57:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.196+0000 7f280a7fc700 1 --2- 172.21.15.37:0/2638300751 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558112921100 0x55811298e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.197+0000 7f280a7fc700 1 --2- 172.21.15.37:0/2638300751 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558112921100 0x55811298e150 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=121aa569de5962dd server_cookie=6bd8839277ebb6b7 in_seq=0 out_seq=0 2024-06-10T00:57:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.197+0000 7f27eaffd700 10 client.15501 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:23.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.197+0000 7f27eaffd700 1 -- 172.21.15.37:0/2638300751 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558112858290 con 0x558112921100 2024-06-10T00:57:23.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.197+0000 7f27eaffd700 10 client.15501 handle_command_reply: tid=0 2024-06-10T00:57:23.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.197+0000 7f281c1aeb80 1 client.15501 shutdown 2024-06-10T00:57:23.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.197+0000 7f281c1aeb80 1 -- 172.21.15.37:0/2638300751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581128fef20 msgr2=0x558112984380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:23.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.197+0000 7f281c1aeb80 1 --2- 172.21.15.37:0/2638300751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581128fef20 0x558112984380 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f27fc029040 tx=0x7f27fc00bf40 comp rx=0 tx=0).stop 2024-06-10T00:57:23.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.197+0000 7f281c1aeb80 1 -- 172.21.15.37:0/2638300751 shutdown_connections 2024-06-10T00:57:23.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.197+0000 7f281c1aeb80 1 --2- 172.21.15.37:0/2638300751 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558112921100 0x55811298e150 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.197+0000 7f281c1aeb80 1 --2- 172.21.15.37:0/2638300751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558112904f60 0x558112926080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.197+0000 7f281c1aeb80 1 --2- 172.21.15.37:0/2638300751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581128fef20 0x558112984380 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.197+0000 7f27eaffd700 0 client.15501 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:23.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.197+0000 7f281c1aeb80 1 --2- 172.21.15.37:0/2638300751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558112904640 0x5581129848c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.197+0000 7f281c1aeb80 1 -- 172.21.15.37:0/2638300751 >> 172.21.15.37:0/2638300751 conn(0x558112829cc0 msgr2=0x5581129097f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:23.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.197+0000 7f281c1aeb80 1 -- 172.21.15.37:0/2638300751 shutdown_connections 2024-06-10T00:57:23.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.197+0000 7f281c1aeb80 1 -- 172.21.15.37:0/2638300751 wait complete. 2024-06-10T00:57:23.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.198+0000 7f281c1aeb80 20 client.15501 trim_cache size 0 max 16384 2024-06-10T00:57:23.204 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T00:57:23.204 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-06-10T00:57:23.204 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-06-10T00:57:23.204 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T00:57:23.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.198+0000 7f281192b700 1 -- 172.21.15.37:0/353517175 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f27f40a2d30 msgr2=0x7f27f40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:23.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.198+0000 7f281192b700 1 --2- 172.21.15.37:0/353517175 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f27f40a2d30 0x7f27f40a51f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f27fc0104e0 tx=0x7f27fc00b480 comp rx=0 tx=0).stop 2024-06-10T00:57:23.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.198+0000 7f281192b700 1 -- 172.21.15.37:0/353517175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f280c06b3c0 msgr2=0x7f280c19b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:23.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.198+0000 7f281192b700 1 --2- 172.21.15.37:0/353517175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f280c06b3c0 0x7f280c19b310 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f280000d950 tx=0x7f280000dc60 comp rx=0 tx=0).stop 2024-06-10T00:57:23.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.198+0000 7f281192b700 1 -- 172.21.15.37:0/353517175 shutdown_connections 2024-06-10T00:57:23.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.198+0000 7f281192b700 1 --2- 172.21.15.37:0/353517175 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f27f40a2d30 0x7f27f40a51f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.198+0000 7f281192b700 1 --2- 172.21.15.37:0/353517175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f280c1084f0 0x7f280c19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.198+0000 7f281192b700 1 --2- 172.21.15.37:0/353517175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f280c06aa00 0x7f280c19add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.198+0000 7f281192b700 1 --2- 172.21.15.37:0/353517175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f280c06b3c0 0x7f280c19b310 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:23.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.198+0000 7f281192b700 1 -- 172.21.15.37:0/353517175 >> 172.21.15.37:0/353517175 conn(0x7f280c078300 msgr2=0x7f280c0ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:23.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.198+0000 7f281192b700 1 -- 172.21.15.37:0/353517175 shutdown_connections 2024-06-10T00:57:23.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:23.198+0000 7f281192b700 1 -- 172.21.15.37:0/353517175 wait complete. 2024-06-10T00:57:23.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:23 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:23.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:23 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:57:23.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:23 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:57:23.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:23 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:57:23.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:23 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:57:23.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:23 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:57:24.029 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T00:57:24.030 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:57:24.309 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:57:24.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:24 smithi037 ceph-mon[32192]: pgmap v451: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-06-10T00:57:24.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:24 smithi167 ceph-mon[36917]: pgmap v451: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-06-10T00:57:24.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:24 smithi094 ceph-mon[36708]: pgmap v451: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-06-10T00:57:26.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:26 smithi037 ceph-mon[32192]: pgmap v452: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-06-10T00:57:26.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.142+0000 7f77aa662700 1 -- 172.21.15.37:0/2750173999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a4101390 msgr2=0x7f77a4101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:26.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.142+0000 7f77aa662700 1 --2- 172.21.15.37:0/2750173999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a4101390 0x7f77a4101810 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7798009990 tx=0x7f7798009ca0 comp rx=0 tx=0).stop 2024-06-10T00:57:26.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.142+0000 7f77aa662700 1 -- 172.21.15.37:0/2750173999 shutdown_connections 2024-06-10T00:57:26.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.142+0000 7f77aa662700 1 --2- 172.21.15.37:0/2750173999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77a4101d50 0x7f77a410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:26.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.142+0000 7f77aa662700 1 --2- 172.21.15.37:0/2750173999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a4107400 0x7f77a41077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:26.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.142+0000 7f77aa662700 1 --2- 172.21.15.37:0/2750173999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a4101390 0x7f77a4101810 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:26.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.142+0000 7f77aa662700 1 -- 172.21.15.37:0/2750173999 >> 172.21.15.37:0/2750173999 conn(0x7f77a4078340 msgr2=0x7f77a4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:26.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.142+0000 7f77aa662700 1 -- 172.21.15.37:0/2750173999 shutdown_connections 2024-06-10T00:57:26.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.142+0000 7f77aa662700 1 -- 172.21.15.37:0/2750173999 wait complete. 2024-06-10T00:57:26.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.143+0000 7f77aa662700 1 Processor -- start 2024-06-10T00:57:26.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.143+0000 7f77aa662700 1 -- start start 2024-06-10T00:57:26.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.143+0000 7f77aa662700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a4101390 0x7f77a419e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:26.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.143+0000 7f77aa662700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a4101d50 0x7f77a419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:26.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.143+0000 7f77aa662700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77a4107400 0x7f77a41989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:26.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.143+0000 7f77aa662700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f77a4111670 con 0x7f77a4101d50 2024-06-10T00:57:26.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.143+0000 7f77aa662700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41114f0 con 0x7f77a4101390 2024-06-10T00:57:26.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.143+0000 7f77aa662700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41117f0 con 0x7f77a4107400 2024-06-10T00:57:26.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.144+0000 7f77a37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a4101d50 0x7f77a419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:26.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.144+0000 7f77a3fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a4101390 0x7f77a419e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:26.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.144+0000 7f77a37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a4101d50 0x7f77a419ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55102/0 (socket says 172.21.15.37:55102) 2024-06-10T00:57:26.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.144+0000 7f77a37fe700 1 -- 172.21.15.37:0/3257975002 learned_addr learned my addr 172.21.15.37:0/3257975002 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:26.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.144+0000 7f77a8bff700 1 --2- 172.21.15.37:0/3257975002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77a4107400 0x7f77a41989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:26.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.144+0000 7f77a37fe700 1 -- 172.21.15.37:0/3257975002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77a4107400 msgr2=0x7f77a41989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:26.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.144+0000 7f77a37fe700 1 --2- 172.21.15.37:0/3257975002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77a4107400 0x7f77a41989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:26.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.144+0000 7f77a37fe700 1 -- 172.21.15.37:0/3257975002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a4101390 msgr2=0x7f77a419e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:26.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.144+0000 7f77a37fe700 1 --2- 172.21.15.37:0/3257975002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a4101390 0x7f77a419e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:26.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.144+0000 7f77a37fe700 1 -- 172.21.15.37:0/3257975002 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a4199100 con 0x7f77a4101d50 2024-06-10T00:57:26.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.144+0000 7f77a37fe700 1 --2- 172.21.15.37:0/3257975002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a4101d50 0x7f77a419ed20 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f779800bff0 tx=0x7f7798005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:26.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.144+0000 7f77a17fa700 1 -- 172.21.15.37:0/3257975002 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798027070 con 0x7f77a4101d50 2024-06-10T00:57:26.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.144+0000 7f77aa662700 1 -- 172.21.15.37:0/3257975002 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a4199390 con 0x7f77a4101d50 2024-06-10T00:57:26.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.144+0000 7f77a17fa700 1 -- 172.21.15.37:0/3257975002 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77980040d0 con 0x7f77a4101d50 2024-06-10T00:57:26.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.144+0000 7f77a17fa700 1 -- 172.21.15.37:0/3257975002 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798022e00 con 0x7f77a4101d50 2024-06-10T00:57:26.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.144+0000 7f77aa662700 1 -- 172.21.15.37:0/3257975002 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a41a56b0 con 0x7f77a4101d50 2024-06-10T00:57:26.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.146+0000 7f77aa662700 1 -- 172.21.15.37:0/3257975002 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77a4069180 con 0x7f77a4101d50 2024-06-10T00:57:26.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.147+0000 7f77a17fa700 1 -- 172.21.15.37:0/3257975002 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f779802a070 con 0x7f77a4101d50 2024-06-10T00:57:26.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.148+0000 7f77a17fa700 1 --2- 172.21.15.37:0/3257975002 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f778c0a2d30 0x7f778c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:26.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.148+0000 7f77a3fff700 1 --2- 172.21.15.37:0/3257975002 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f778c0a2d30 0x7f778c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:26.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.148+0000 7f77a17fa700 1 -- 172.21.15.37:0/3257975002 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f77980d0330 con 0x7f77a4101d50 2024-06-10T00:57:26.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.149+0000 7f77a3fff700 1 --2- 172.21.15.37:0/3257975002 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f778c0a2d30 0x7f778c0a51f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f77a4199ae0 tx=0x7f779400a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:26.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.153+0000 7f77a17fa700 1 -- 172.21.15.37:0/3257975002 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f779805e850 con 0x7f77a4101d50 2024-06-10T00:57:26.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.423+0000 7f77aa662700 1 -- 172.21.15.37:0/3257975002 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77a41a5880 con 0x7f77a4101d50 2024-06-10T00:57:26.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.424+0000 7f77a17fa700 1 -- 172.21.15.37:0/3257975002 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11867 (secure 0 0 0) 0x7f7798013070 con 0x7f77a4101d50 2024-06-10T00:57:26.425 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:57:26.425 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T00:56:30.291171+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24451,24458,24472]},"id":1}]} 2024-06-10T00:57:26.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.426+0000 7f77aa662700 1 -- 172.21.15.37:0/3257975002 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f778c0a2d30 msgr2=0x7f778c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:26.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.426+0000 7f77aa662700 1 --2- 172.21.15.37:0/3257975002 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f778c0a2d30 0x7f778c0a51f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f77a4199ae0 tx=0x7f779400a380 comp rx=0 tx=0).stop 2024-06-10T00:57:26.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.426+0000 7f77aa662700 1 -- 172.21.15.37:0/3257975002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a4101d50 msgr2=0x7f77a419ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:26.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.426+0000 7f77aa662700 1 --2- 172.21.15.37:0/3257975002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a4101d50 0x7f77a419ed20 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f779800bff0 tx=0x7f7798005590 comp rx=0 tx=0).stop 2024-06-10T00:57:26.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.426+0000 7f77aa662700 1 -- 172.21.15.37:0/3257975002 shutdown_connections 2024-06-10T00:57:26.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.427+0000 7f77aa662700 1 --2- 172.21.15.37:0/3257975002 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f778c0a2d30 0x7f778c0a51f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:26.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.427+0000 7f77aa662700 1 --2- 172.21.15.37:0/3257975002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77a4107400 0x7f77a41989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:26.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.427+0000 7f77aa662700 1 --2- 172.21.15.37:0/3257975002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a4101390 0x7f77a419e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:26.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.427+0000 7f77aa662700 1 --2- 172.21.15.37:0/3257975002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77a4101d50 0x7f77a419ed20 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:26.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.427+0000 7f77aa662700 1 -- 172.21.15.37:0/3257975002 >> 172.21.15.37:0/3257975002 conn(0x7f77a4078340 msgr2=0x7f77a4100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:26.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.427+0000 7f77aa662700 1 -- 172.21.15.37:0/3257975002 shutdown_connections 2024-06-10T00:57:26.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:26.427+0000 7f77aa662700 1 -- 172.21.15.37:0/3257975002 wait complete. 2024-06-10T00:57:26.428 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-06-10T00:57:26.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:26 smithi167 ceph-mon[36917]: pgmap v452: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-06-10T00:57:26.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:26 smithi094 ceph-mon[36708]: pgmap v452: 129 pgs: 129 active+clean; 668 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-06-10T00:57:27.124 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T00:57:27.391 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:57:27.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:27 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3257975002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:57:27.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:27 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3257975002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:57:27.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:27 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3257975002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:57:28.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:28 smithi037 ceph-mon[32192]: pgmap v453: 129 pgs: 129 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T00:57:28.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:28 smithi167 ceph-mon[36917]: pgmap v453: 129 pgs: 129 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T00:57:28.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:28 smithi094 ceph-mon[36708]: pgmap v453: 129 pgs: 129 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T00:57:29.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.268+0000 7f111c82f700 1 -- 172.21.15.37:0/3550296001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1118101d50 msgr2=0x7f111810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:29.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.268+0000 7f111c82f700 1 --2- 172.21.15.37:0/3550296001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1118101d50 0x7f111810e240 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f111000b0a0 tx=0x7f111000b3b0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.268+0000 7f111c82f700 1 -- 172.21.15.37:0/3550296001 shutdown_connections 2024-06-10T00:57:29.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.268+0000 7f111c82f700 1 --2- 172.21.15.37:0/3550296001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1118101d50 0x7f111810e240 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.268+0000 7f111c82f700 1 --2- 172.21.15.37:0/3550296001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1118107400 0x7f11181077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.268+0000 7f111c82f700 1 --2- 172.21.15.37:0/3550296001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1118101390 0x7f1118101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.268+0000 7f111c82f700 1 -- 172.21.15.37:0/3550296001 >> 172.21.15.37:0/3550296001 conn(0x7f1118078340 msgr2=0x7f1118078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:29.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.268+0000 7f111c82f700 1 -- 172.21.15.37:0/3550296001 shutdown_connections 2024-06-10T00:57:29.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.268+0000 7f111c82f700 1 -- 172.21.15.37:0/3550296001 wait complete. 2024-06-10T00:57:29.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.269+0000 7f111c82f700 1 Processor -- start 2024-06-10T00:57:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.269+0000 7f111c82f700 1 -- start start 2024-06-10T00:57:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.269+0000 7f111c82f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1118101390 0x7f111819e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.269+0000 7f111c82f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1118101d50 0x7f111819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.269+0000 7f111c82f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1118107400 0x7f1118198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.269+0000 7f111c82f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11181116f0 con 0x7f1118101d50 2024-06-10T00:57:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.269+0000 7f111c82f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1118111570 con 0x7f1118101390 2024-06-10T00:57:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.269+0000 7f111c82f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1118111870 con 0x7f1118107400 2024-06-10T00:57:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.270+0000 7f1115d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1118101d50 0x7f111819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.270+0000 7f111659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1118101390 0x7f111819e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.270+0000 7f1115d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1118101d50 0x7f111819ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55122/0 (socket says 172.21.15.37:55122) 2024-06-10T00:57:29.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.270+0000 7f1115d9b700 1 -- 172.21.15.37:0/3556402880 learned_addr learned my addr 172.21.15.37:0/3556402880 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:29.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.270+0000 7f1116d9d700 1 --2- 172.21.15.37:0/3556402880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1118107400 0x7f1118198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:29.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.270+0000 7f1115d9b700 1 -- 172.21.15.37:0/3556402880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1118107400 msgr2=0x7f1118198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:29.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.270+0000 7f1115d9b700 1 --2- 172.21.15.37:0/3556402880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1118107400 0x7f1118198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.270+0000 7f1115d9b700 1 -- 172.21.15.37:0/3556402880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1118101390 msgr2=0x7f111819e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:29.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.270+0000 7f1115d9b700 1 --2- 172.21.15.37:0/3556402880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1118101390 0x7f111819e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.270+0000 7f1115d9b700 1 -- 172.21.15.37:0/3556402880 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11181991c0 con 0x7f1118101d50 2024-06-10T00:57:29.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.270+0000 7f1115d9b700 1 --2- 172.21.15.37:0/3556402880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1118101d50 0x7f111819ed90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f110c00d980 tx=0x7f110c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:29.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.270+0000 7f11077fe700 1 -- 172.21.15.37:0/3556402880 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c017070 con 0x7f1118101d50 2024-06-10T00:57:29.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.270+0000 7f11077fe700 1 -- 172.21.15.37:0/3556402880 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f110c005020 con 0x7f1118101d50 2024-06-10T00:57:29.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.270+0000 7f111c82f700 1 -- 172.21.15.37:0/3556402880 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11181994b0 con 0x7f1118101d50 2024-06-10T00:57:29.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.270+0000 7f11077fe700 1 -- 172.21.15.37:0/3556402880 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c009bc0 con 0x7f1118101d50 2024-06-10T00:57:29.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.271+0000 7f111c82f700 1 -- 172.21.15.37:0/3556402880 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11181a5720 con 0x7f1118101d50 2024-06-10T00:57:29.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.273+0000 7f11077fe700 1 -- 172.21.15.37:0/3556402880 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f110c0051d0 con 0x7f1118101d50 2024-06-10T00:57:29.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.273+0000 7f11077fe700 1 --2- 172.21.15.37:0/3556402880 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f11000a2d30 0x7f11000a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:29.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.273+0000 7f111659c700 1 --2- 172.21.15.37:0/3556402880 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f11000a2d30 0x7f11000a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:29.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.273+0000 7f11077fe700 1 -- 172.21.15.37:0/3556402880 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f110c0cf3e0 con 0x7f1118101d50 2024-06-10T00:57:29.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.274+0000 7f111659c700 1 --2- 172.21.15.37:0/3556402880 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f11000a2d30 0x7f11000a51f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1108005950 tx=0x7f110800a300 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:29.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.275+0000 7f11270b2b80 1 Processor -- start 2024-06-10T00:57:29.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.275+0000 7f11270b2b80 1 -- start start 2024-06-10T00:57:29.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.275+0000 7f11270b2b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558434d0af60 0x558434d0b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:29.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.275+0000 7f11270b2b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558434d0b880 0x558434d05260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:29.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.275+0000 7f11270b2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558434d057a0 0x558434d05bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:29.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.275+0000 7f11270b2b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558434bf7f60 con 0x558434d0af60 2024-06-10T00:57:29.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.275+0000 7f11270b2b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558434c56850 con 0x558434d0b880 2024-06-10T00:57:29.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.275+0000 7f11270b2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558434bbe460 con 0x558434d057a0 2024-06-10T00:57:29.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.275+0000 7f111659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558434d0af60 0x558434d0b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:29.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.275+0000 7f111659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558434d0af60 0x558434d0b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55130/0 (socket says 172.21.15.37:55130) 2024-06-10T00:57:29.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.275+0000 7f111659c700 1 -- 172.21.15.37:0/2384463714 learned_addr learned my addr 172.21.15.37:0/2384463714 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:29.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.276+0000 7f1115d9b700 1 --2- 172.21.15.37:0/2384463714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558434d0b880 0x558434d05260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:29.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.276+0000 7f111659c700 1 -- 172.21.15.37:0/2384463714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558434d057a0 msgr2=0x558434d05bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:29.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.276+0000 7f1116d9d700 1 --2- 172.21.15.37:0/2384463714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558434d057a0 0x558434d05bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:29.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.276+0000 7f111659c700 1 --2- 172.21.15.37:0/2384463714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558434d057a0 0x558434d05bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.276+0000 7f111659c700 1 -- 172.21.15.37:0/2384463714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558434d0b880 msgr2=0x558434d05260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:29.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.276+0000 7f111659c700 1 --2- 172.21.15.37:0/2384463714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558434d0b880 0x558434d05260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.276+0000 7f111659c700 1 -- 172.21.15.37:0/2384463714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558434ba58d0 con 0x558434d0af60 2024-06-10T00:57:29.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.276+0000 7f1115d9b700 1 --2- 172.21.15.37:0/2384463714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558434d0b880 0x558434d05260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:57:29.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.276+0000 7f1116d9d700 1 --2- 172.21.15.37:0/2384463714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558434d057a0 0x558434d05bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:57:29.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.276+0000 7f111659c700 1 --2- 172.21.15.37:0/2384463714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558434d0af60 0x558434d0b340 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f110800fd30 tx=0x7f1108012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=126ba04a66255a2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:29.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.276+0000 7f1104ff9700 1 -- 172.21.15.37:0/2384463714 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1108016070 con 0x558434d0af60 2024-06-10T00:57:29.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.276+0000 7f1104ff9700 1 -- 172.21.15.37:0/2384463714 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1108010da0 con 0x558434d0af60 2024-06-10T00:57:29.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.276+0000 7f1104ff9700 1 -- 172.21.15.37:0/2384463714 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110801b420 con 0x558434d0af60 2024-06-10T00:57:29.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.277+0000 7f11270b2b80 1 -- 172.21.15.37:0/2384463714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558434d0af60 msgr2=0x558434d0b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:29.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.277+0000 7f11270b2b80 1 --2- 172.21.15.37:0/2384463714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558434d0af60 0x558434d0b340 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f110800fd30 tx=0x7f1108012000 comp rx=0 tx=0).stop 2024-06-10T00:57:29.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.277+0000 7f11270b2b80 1 -- 172.21.15.37:0/2384463714 shutdown_connections 2024-06-10T00:57:29.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.277+0000 7f11270b2b80 1 --2- 172.21.15.37:0/2384463714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558434d057a0 0x558434d05bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.277+0000 7f11270b2b80 1 --2- 172.21.15.37:0/2384463714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558434d0af60 0x558434d0b340 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.277+0000 7f11270b2b80 1 --2- 172.21.15.37:0/2384463714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558434d0b880 0x558434d05260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.277+0000 7f11270b2b80 1 -- 172.21.15.37:0/2384463714 >> 172.21.15.37:0/2384463714 conn(0x558434c32080 msgr2=0x558434d00220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:29.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.277+0000 7f11270b2b80 1 -- 172.21.15.37:0/2384463714 shutdown_connections 2024-06-10T00:57:29.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.277+0000 7f11270b2b80 1 -- 172.21.15.37:0/2384463714 wait complete. 2024-06-10T00:57:29.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.277+0000 7f11270b2b80 1 -- start start 2024-06-10T00:57:29.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.278+0000 7f11270b2b80 1 Processor -- start 2024-06-10T00:57:29.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.278+0000 7f11270b2b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558434d057a0 0x558434d115e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:29.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.278+0000 7f11270b2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558434d0af60 0x558434d0e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:29.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.278+0000 7f11270b2b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558434d0b880 0x558434d0ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:29.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.278+0000 7f11270b2b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558434bf6fc0 con 0x558434d0b880 2024-06-10T00:57:29.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.278+0000 7f11270b2b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558434c52810 con 0x558434d057a0 2024-06-10T00:57:29.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.278+0000 7f11270b2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558434d12260 con 0x558434d0af60 2024-06-10T00:57:29.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.278+0000 7f11270b2b80 10 client.0 _ll_register_callbacks cb 0x558434bec5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:29.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.278+0000 7f1116d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558434d0b880 0x558434d0ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:29.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.278+0000 7f1116d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558434d0b880 0x558434d0ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55138/0 (socket says 172.21.15.37:55138) 2024-06-10T00:57:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.278+0000 7f1116d9d700 1 -- 172.21.15.37:0/694243547 learned_addr learned my addr 172.21.15.37:0/694243547 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.278+0000 7f111659c700 1 --2- 172.21.15.37:0/694243547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558434d057a0 0x558434d115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.278+0000 7f1115d9b700 1 --2- 172.21.15.37:0/694243547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558434d0af60 0x558434d0e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.278+0000 7f1116d9d700 1 -- 172.21.15.37:0/694243547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558434d0af60 msgr2=0x558434d0e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.278+0000 7f1116d9d700 1 --2- 172.21.15.37:0/694243547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558434d0af60 0x558434d0e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.278+0000 7f1116d9d700 1 -- 172.21.15.37:0/694243547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558434d057a0 msgr2=0x558434d115e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.278+0000 7f1116d9d700 1 --2- 172.21.15.37:0/694243547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558434d057a0 0x558434d115e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.278+0000 7f1116d9d700 1 -- 172.21.15.37:0/694243547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558434c1c750 con 0x558434d0b880 2024-06-10T00:57:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.279+0000 7f1116d9d700 1 --2- 172.21.15.37:0/694243547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558434d0b880 0x558434d0ee20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1110015000 tx=0x7f1110004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7724c45141183684 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.279+0000 7f10fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:57:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.279+0000 7f10fdffb700 1 -- 172.21.15.37:0/694243547 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11100076e0 con 0x558434d0b880 2024-06-10T00:57:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.279+0000 7f10fdffb700 1 -- 172.21.15.37:0/694243547 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f1110007880 con 0x558434d0b880 2024-06-10T00:57:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.279+0000 7f11270b2b80 1 -- 172.21.15.37:0/694243547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1108010da0 con 0x558434d0b880 2024-06-10T00:57:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.279+0000 7f10fdffb700 1 -- 172.21.15.37:0/694243547 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f111002f050 con 0x558434d0b880 2024-06-10T00:57:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.279+0000 7f11270b2b80 10 client.15519 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.279+0000 7f11270b2b80 1 -- 172.21.15.37:0/694243547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558434d1c9d0 con 0x558434d0b880 2024-06-10T00:57:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.279+0000 7f10fdffb700 1 -- 172.21.15.37:0/694243547 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1108010da0 con 0x558434d0b880 2024-06-10T00:57:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.280+0000 7f10fdffb700 1 -- 172.21.15.37:0/694243547 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1110011070 con 0x558434d0b880 2024-06-10T00:57:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.280+0000 7f11270b2b80 10 client.15519 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.280+0000 7f11270b2b80 10 client.15519 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-10T00:57:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.280+0000 7f11270b2b80 20 client.15519 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.280+0000 7f11270b2b80 1 --2- 172.21.15.37:0/694243547 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x558434d1f010 0x558434d21400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.280+0000 7f11270b2b80 4 client.15519 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:57:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.280+0000 7f11270b2b80 1 -- 172.21.15.37:0/694243547 --> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558434c63d50 con 0x558434d1f010 2024-06-10T00:57:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.280+0000 7f111659c700 1 --2- 172.21.15.37:0/694243547 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x558434d1f010 0x558434d21400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.281+0000 7f111659c700 1 --2- 172.21.15.37:0/694243547 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x558434d1f010 0x558434d21400 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f2d58212734adc12 server_cookie=82fbdcd7bed26eb1 in_seq=0 out_seq=0 2024-06-10T00:57:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.281+0000 7f10fdffb700 10 client.15519 ms_handle_connect on v2:172.21.15.167:6836/2720148065 2024-06-10T00:57:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.281+0000 7f10fdffb700 1 -- 172.21.15.37:0/694243547 <== mds.2 v2:172.21.15.167:6836/2720148065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558434c63d50 con 0x558434d1f010 2024-06-10T00:57:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.281+0000 7f10fdffb700 10 client.15519 handle_command_reply: tid=0 2024-06-10T00:57:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.281+0000 7f11270b2b80 1 client.15519 shutdown 2024-06-10T00:57:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.281+0000 7f11270b2b80 1 -- 172.21.15.37:0/694243547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558434d0b880 msgr2=0x558434d0ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.281+0000 7f11270b2b80 1 --2- 172.21.15.37:0/694243547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558434d0b880 0x558434d0ee20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1110015000 tx=0x7f1110004c80 comp rx=0 tx=0).stop 2024-06-10T00:57:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.282+0000 7f11270b2b80 1 -- 172.21.15.37:0/694243547 shutdown_connections 2024-06-10T00:57:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.282+0000 7f11270b2b80 1 --2- 172.21.15.37:0/694243547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558434d0b880 0x558434d0ee20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.282+0000 7f11270b2b80 1 --2- 172.21.15.37:0/694243547 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x558434d1f010 0x558434d21400 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.282+0000 7f11270b2b80 1 --2- 172.21.15.37:0/694243547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558434d057a0 0x558434d115e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.282+0000 7f10fdffb700 0 client.15519 ms_handle_reset on v2:172.21.15.167:6836/2720148065 2024-06-10T00:57:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.282+0000 7f11270b2b80 1 --2- 172.21.15.37:0/694243547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558434d0af60 0x558434d0e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.282+0000 7f11270b2b80 1 -- 172.21.15.37:0/694243547 >> 172.21.15.37:0/694243547 conn(0x558434c32080 msgr2=0x558434cff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.282+0000 7f11270b2b80 1 -- 172.21.15.37:0/694243547 shutdown_connections 2024-06-10T00:57:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.282+0000 7f11270b2b80 1 -- 172.21.15.37:0/694243547 wait complete. 2024-06-10T00:57:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.282+0000 7f11270b2b80 20 client.15519 trim_cache size 0 max 16384 2024-06-10T00:57:29.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.299+0000 7f11270b2b80 1 Processor -- start 2024-06-10T00:57:29.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.299+0000 7f11270b2b80 1 -- start start 2024-06-10T00:57:29.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.299+0000 7f11270b2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558434d057a0 0x558434d0e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:29.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.299+0000 7f11270b2b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558434d0af60 0x558434d11120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:29.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.299+0000 7f11270b2b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558434d0b880 0x558434cffe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:29.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.299+0000 7f11270b2b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11181b0d30 con 0x558434d0b880 2024-06-10T00:57:29.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.299+0000 7f11270b2b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f11181ae5c0 con 0x558434d0af60 2024-06-10T00:57:29.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.299+0000 7f11270b2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f11181ac410 con 0x558434d057a0 2024-06-10T00:57:29.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.299+0000 7f1116d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558434d0b880 0x558434cffe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:29.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.300+0000 7f1116d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558434d0b880 0x558434cffe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55144/0 (socket says 172.21.15.37:55144) 2024-06-10T00:57:29.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.300+0000 7f1116d9d700 1 -- 172.21.15.37:0/3128249046 learned_addr learned my addr 172.21.15.37:0/3128249046 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:29.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.300+0000 7f111659c700 1 --2- 172.21.15.37:0/3128249046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558434d057a0 0x558434d0e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:29.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.300+0000 7f1115d9b700 1 --2- 172.21.15.37:0/3128249046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558434d0af60 0x558434d11120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:29.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.300+0000 7f1116d9d700 1 -- 172.21.15.37:0/3128249046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558434d057a0 msgr2=0x558434d0e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:29.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.300+0000 7f1116d9d700 1 --2- 172.21.15.37:0/3128249046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558434d057a0 0x558434d0e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.300+0000 7f1116d9d700 1 -- 172.21.15.37:0/3128249046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558434d0af60 msgr2=0x558434d11120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:29.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.300+0000 7f1116d9d700 1 --2- 172.21.15.37:0/3128249046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558434d0af60 0x558434d11120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.300+0000 7f1116d9d700 1 -- 172.21.15.37:0/3128249046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558434d0ce50 con 0x558434d0b880 2024-06-10T00:57:29.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.300+0000 7f111659c700 1 --2- 172.21.15.37:0/3128249046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558434d057a0 0x558434d0e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:57:29.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.300+0000 7f1116d9d700 1 --2- 172.21.15.37:0/3128249046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558434d0b880 0x558434cffe80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1110009a20 tx=0x7f1110005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f755ea5e78f13e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:29.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.300+0000 7f10fffff700 1 -- 172.21.15.37:0/3128249046 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110038070 con 0x558434d0b880 2024-06-10T00:57:29.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.300+0000 7f10fffff700 1 -- 172.21.15.37:0/3128249046 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1110005520 con 0x558434d0b880 2024-06-10T00:57:29.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.300+0000 7f10fffff700 1 -- 172.21.15.37:0/3128249046 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110026d10 con 0x558434d0b880 2024-06-10T00:57:29.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.301+0000 7f11270b2b80 1 -- 172.21.15.37:0/3128249046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558434d0b880 msgr2=0x558434cffe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:29.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.301+0000 7f11270b2b80 1 --2- 172.21.15.37:0/3128249046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558434d0b880 0x558434cffe80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1110009a20 tx=0x7f1110005b90 comp rx=0 tx=0).stop 2024-06-10T00:57:29.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.301+0000 7f11270b2b80 1 -- 172.21.15.37:0/3128249046 shutdown_connections 2024-06-10T00:57:29.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.301+0000 7f11270b2b80 1 --2- 172.21.15.37:0/3128249046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558434d0b880 0x558434cffe80 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.301+0000 7f11270b2b80 1 --2- 172.21.15.37:0/3128249046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558434d057a0 0x558434d0e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.301+0000 7f11270b2b80 1 --2- 172.21.15.37:0/3128249046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558434d0af60 0x558434d11120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.301+0000 7f11270b2b80 1 -- 172.21.15.37:0/3128249046 >> 172.21.15.37:0/3128249046 conn(0x558434c32080 msgr2=0x558434d04250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:29.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.302+0000 7f11270b2b80 1 -- 172.21.15.37:0/3128249046 shutdown_connections 2024-06-10T00:57:29.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.302+0000 7f11270b2b80 1 -- 172.21.15.37:0/3128249046 wait complete. 2024-06-10T00:57:29.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.302+0000 7f11270b2b80 1 -- start start 2024-06-10T00:57:29.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.302+0000 7f11270b2b80 1 Processor -- start 2024-06-10T00:57:29.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.303+0000 7f11270b2b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558434d057a0 0x558434d17550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:29.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.303+0000 7f11270b2b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558434d0af60 0x558434d17a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:29.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.303+0000 7f11270b2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558434d0b880 0x558434d25e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:29.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.303+0000 7f11270b2b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11181aca80 con 0x558434d0af60 2024-06-10T00:57:29.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.303+0000 7f11270b2b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f11181adef0 con 0x558434d057a0 2024-06-10T00:57:29.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.303+0000 7f11270b2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f11181ad150 con 0x558434d0b880 2024-06-10T00:57:29.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.303+0000 7f11270b2b80 10 client.0 _ll_register_callbacks cb 0x558434bec5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:29.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.303+0000 7f1116d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558434d0b880 0x558434d25e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:29.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.303+0000 7f1116d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558434d0b880 0x558434d25e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:55052/0 (socket says 172.21.15.37:55052) 2024-06-10T00:57:29.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.303+0000 7f1116d9d700 1 -- 172.21.15.37:0/2093915721 learned_addr learned my addr 172.21.15.37:0/2093915721 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:29.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.303+0000 7f111659c700 1 --2- 172.21.15.37:0/2093915721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558434d057a0 0x558434d17550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:29.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.303+0000 7f1115d9b700 1 --2- 172.21.15.37:0/2093915721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558434d0af60 0x558434d17a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:29.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.303+0000 7f1116d9d700 1 -- 172.21.15.37:0/2093915721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558434d057a0 msgr2=0x558434d17550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:29.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.303+0000 7f1116d9d700 1 --2- 172.21.15.37:0/2093915721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558434d057a0 0x558434d17550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.303+0000 7f1116d9d700 1 -- 172.21.15.37:0/2093915721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558434d0af60 msgr2=0x558434d17a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:29.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.303+0000 7f1116d9d700 1 --2- 172.21.15.37:0/2093915721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558434d0af60 0x558434d17a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.303+0000 7f1116d9d700 1 -- 172.21.15.37:0/2093915721 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558434d263f0 con 0x558434d0b880 2024-06-10T00:57:29.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.303+0000 7f1115d9b700 1 --2- 172.21.15.37:0/2093915721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558434d0af60 0x558434d17a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:57:29.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.304+0000 7f1116d9d700 1 --2- 172.21.15.37:0/2093915721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558434d0b880 0x558434d25e80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1110009bc0 tx=0x7f11100056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3c0c2617932c9d8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:29.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.304+0000 7f10fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T00:57:29.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.304+0000 7f10fdffb700 1 -- 172.21.15.37:0/2093915721 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110038070 con 0x558434d0b880 2024-06-10T00:57:29.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.304+0000 7f10fdffb700 1 -- 172.21.15.37:0/2093915721 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f1110005000 con 0x558434d0b880 2024-06-10T00:57:29.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.304+0000 7f11270b2b80 1 -- 172.21.15.37:0/2093915721 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1110005520 con 0x558434d0b880 2024-06-10T00:57:29.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.304+0000 7f10fdffb700 1 -- 172.21.15.37:0/2093915721 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1110045050 con 0x558434d0b880 2024-06-10T00:57:29.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.304+0000 7f11270b2b80 10 client.25168 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:29.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.304+0000 7f11270b2b80 1 -- 172.21.15.37:0/2093915721 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558434d8d8d0 con 0x558434d0b880 2024-06-10T00:57:29.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.304+0000 7f10fdffb700 1 -- 172.21.15.37:0/2093915721 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1110005520 con 0x558434d0b880 2024-06-10T00:57:29.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.305+0000 7f10fdffb700 1 -- 172.21.15.37:0/2093915721 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1110011040 con 0x558434d0b880 2024-06-10T00:57:29.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.305+0000 7f11270b2b80 10 client.25168 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:29.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.305+0000 7f11270b2b80 10 client.25168 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-10T00:57:29.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.305+0000 7f11270b2b80 20 client.25168 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:29.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.305+0000 7f11270b2b80 1 --2- 172.21.15.37:0/2093915721 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x558434d1f010 0x558434d8ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:29.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.305+0000 7f11270b2b80 4 client.25168 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:57:29.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.305+0000 7f11270b2b80 1 -- 172.21.15.37:0/2093915721 --> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558434c5e2d0 con 0x558434d1f010 2024-06-10T00:57:29.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.305+0000 7f111659c700 1 --2- 172.21.15.37:0/2093915721 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x558434d1f010 0x558434d8ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:29.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.305+0000 7f10fdffb700 10 client.25168 ms_handle_connect on v2:172.21.15.167:6836/2720148065 2024-06-10T00:57:29.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.305+0000 7f111659c700 1 --2- 172.21.15.37:0/2093915721 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x558434d1f010 0x558434d8ff60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c29a29dd23ff88fc server_cookie=67a61bdff64c338 in_seq=0 out_seq=0 2024-06-10T00:57:29.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.305+0000 7f10fdffb700 1 -- 172.21.15.37:0/2093915721 <== mds.2 v2:172.21.15.167:6836/2720148065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558434c5e2d0 con 0x558434d1f010 2024-06-10T00:57:29.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.306+0000 7f10fdffb700 10 client.25168 handle_command_reply: tid=0 2024-06-10T00:57:29.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.306+0000 7f11270b2b80 1 client.25168 shutdown 2024-06-10T00:57:29.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.306+0000 7f11270b2b80 1 -- 172.21.15.37:0/2093915721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558434d0b880 msgr2=0x558434d25e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:29.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.306+0000 7f11270b2b80 1 --2- 172.21.15.37:0/2093915721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558434d0b880 0x558434d25e80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1110009bc0 tx=0x7f11100056d0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.306+0000 7f11270b2b80 1 -- 172.21.15.37:0/2093915721 shutdown_connections 2024-06-10T00:57:29.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.306+0000 7f11270b2b80 1 --2- 172.21.15.37:0/2093915721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558434d0b880 0x558434d25e80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.306+0000 7f11270b2b80 1 --2- 172.21.15.37:0/2093915721 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x558434d1f010 0x558434d8ff60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.306+0000 7f11270b2b80 1 --2- 172.21.15.37:0/2093915721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558434d057a0 0x558434d17550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.306+0000 7f11270b2b80 1 --2- 172.21.15.37:0/2093915721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558434d0af60 0x558434d17a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.306+0000 7f11270b2b80 1 -- 172.21.15.37:0/2093915721 >> 172.21.15.37:0/2093915721 conn(0x558434c32080 msgr2=0x558434d0c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:29.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.306+0000 7f10fdffb700 0 client.25168 ms_handle_reset on v2:172.21.15.167:6836/2720148065 2024-06-10T00:57:29.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.306+0000 7f11270b2b80 1 -- 172.21.15.37:0/2093915721 shutdown_connections 2024-06-10T00:57:29.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.306+0000 7f11270b2b80 1 -- 172.21.15.37:0/2093915721 wait complete. 2024-06-10T00:57:29.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.306+0000 7f11270b2b80 20 client.25168 trim_cache size 0 max 16384 2024-06-10T00:57:29.313 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T00:57:29.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.307+0000 7f111c82f700 1 -- 172.21.15.37:0/3556402880 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f11000a2d30 msgr2=0x7f11000a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:29.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.307+0000 7f111c82f700 1 --2- 172.21.15.37:0/3556402880 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f11000a2d30 0x7f11000a51f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1108005950 tx=0x7f110800a300 comp rx=0 tx=0).stop 2024-06-10T00:57:29.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.307+0000 7f111c82f700 1 -- 172.21.15.37:0/3556402880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1118101d50 msgr2=0x7f111819ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:29.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.307+0000 7f111c82f700 1 --2- 172.21.15.37:0/3556402880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1118101d50 0x7f111819ed90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f110c00d980 tx=0x7f110c00dc90 comp rx=0 tx=0).stop 2024-06-10T00:57:29.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.307+0000 7f111c82f700 1 -- 172.21.15.37:0/3556402880 shutdown_connections 2024-06-10T00:57:29.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.307+0000 7f111c82f700 1 --2- 172.21.15.37:0/3556402880 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f11000a2d30 0x7f11000a51f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.307+0000 7f111c82f700 1 --2- 172.21.15.37:0/3556402880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1118107400 0x7f1118198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.307+0000 7f111c82f700 1 --2- 172.21.15.37:0/3556402880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1118101390 0x7f111819e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.307+0000 7f111c82f700 1 --2- 172.21.15.37:0/3556402880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1118101d50 0x7f111819ed90 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:29.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.307+0000 7f111c82f700 1 -- 172.21.15.37:0/3556402880 >> 172.21.15.37:0/3556402880 conn(0x7f1118078340 msgr2=0x7f11181005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:29.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.307+0000 7f111c82f700 1 -- 172.21.15.37:0/3556402880 shutdown_connections 2024-06-10T00:57:29.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:29.307+0000 7f111c82f700 1 -- 172.21.15.37:0/3556402880 wait complete. 2024-06-10T00:57:30.160 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T00:57:30.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:30 smithi037 ceph-mon[32192]: pgmap v454: 129 pgs: 129 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T00:57:30.425 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:57:30.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:30 smithi167 ceph-mon[36917]: pgmap v454: 129 pgs: 129 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T00:57:30.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:30 smithi094 ceph-mon[36708]: pgmap v454: 129 pgs: 129 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T00:57:32.257 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:32 smithi037 ceph-mon[32192]: pgmap v455: 129 pgs: 129 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-10T00:57:32.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.268+0000 7f2d6a8f2700 1 -- 172.21.15.37:0/1648196924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d64107870 msgr2=0x7f2d6410f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.268+0000 7f2d6a8f2700 1 --2- 172.21.15.37:0/1648196924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d64107870 0x7f2d6410f3f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c00b0d0 tx=0x7f2d5c00b3e0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.268+0000 7f2d6a8f2700 1 -- 172.21.15.37:0/1648196924 shutdown_connections 2024-06-10T00:57:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.268+0000 7f2d6a8f2700 1 --2- 172.21.15.37:0/1648196924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d64107870 0x7f2d6410f3f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.268+0000 7f2d6a8f2700 1 --2- 172.21.15.37:0/1648196924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d641027a0 0x7f2d64102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.268+0000 7f2d6a8f2700 1 --2- 172.21.15.37:0/1648196924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d64103150 0x7f2d64107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.268+0000 7f2d6a8f2700 1 -- 172.21.15.37:0/1648196924 >> 172.21.15.37:0/1648196924 conn(0x7f2d64079520 msgr2=0x7f2d64079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.269+0000 7f2d6a8f2700 1 -- 172.21.15.37:0/1648196924 shutdown_connections 2024-06-10T00:57:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.269+0000 7f2d6a8f2700 1 -- 172.21.15.37:0/1648196924 wait complete. 2024-06-10T00:57:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.269+0000 7f2d6a8f2700 1 Processor -- start 2024-06-10T00:57:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.269+0000 7f2d6a8f2700 1 -- start start 2024-06-10T00:57:32.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.270+0000 7f2d6a8f2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d641027a0 0x7f2d64074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:32.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.270+0000 7f2d6a8f2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d64103150 0x7f2d64074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:32.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.270+0000 7f2d6a8f2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d64107870 0x7f2d64075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:32.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.270+0000 7f2d6a8f2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2d641127b0 con 0x7f2d64103150 2024-06-10T00:57:32.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.270+0000 7f2d6a8f2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2d64112630 con 0x7f2d64107870 2024-06-10T00:57:32.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.270+0000 7f2d6a8f2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2d64112930 con 0x7f2d641027a0 2024-06-10T00:57:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.270+0000 7f2d637fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d64103150 0x7f2d64074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.270+0000 7f2d637fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d64103150 0x7f2d64074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44352/0 (socket says 172.21.15.37:44352) 2024-06-10T00:57:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.270+0000 7f2d637fe700 1 -- 172.21.15.37:0/2767402167 learned_addr learned my addr 172.21.15.37:0/2767402167 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.270+0000 7f2d63fff700 1 --2- 172.21.15.37:0/2767402167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d641027a0 0x7f2d64074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.270+0000 7f2d68e8f700 1 --2- 172.21.15.37:0/2767402167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d64107870 0x7f2d64075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.270+0000 7f2d637fe700 1 -- 172.21.15.37:0/2767402167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d641027a0 msgr2=0x7f2d64074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.270+0000 7f2d637fe700 1 --2- 172.21.15.37:0/2767402167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d641027a0 0x7f2d64074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.270+0000 7f2d637fe700 1 -- 172.21.15.37:0/2767402167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d64107870 msgr2=0x7f2d64075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.270+0000 7f2d637fe700 1 --2- 172.21.15.37:0/2767402167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d64107870 0x7f2d64075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.270+0000 7f2d637fe700 1 -- 172.21.15.37:0/2767402167 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d641a7040 con 0x7f2d64103150 2024-06-10T00:57:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.270+0000 7f2d68e8f700 1 --2- 172.21.15.37:0/2767402167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d64107870 0x7f2d64075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:57:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.271+0000 7f2d637fe700 1 --2- 172.21.15.37:0/2767402167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d64103150 0x7f2d64074cd0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800d950 tx=0x7f2d5800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.271+0000 7f2d617fa700 1 -- 172.21.15.37:0/2767402167 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d58016070 con 0x7f2d64103150 2024-06-10T00:57:32.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.271+0000 7f2d6a8f2700 1 -- 172.21.15.37:0/2767402167 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d641a7330 con 0x7f2d64103150 2024-06-10T00:57:32.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.271+0000 7f2d617fa700 1 -- 172.21.15.37:0/2767402167 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d58009e20 con 0x7f2d64103150 2024-06-10T00:57:32.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.271+0000 7f2d617fa700 1 -- 172.21.15.37:0/2767402167 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5800f620 con 0x7f2d64103150 2024-06-10T00:57:32.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.271+0000 7f2d6a8f2700 1 -- 172.21.15.37:0/2767402167 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d641a78c0 con 0x7f2d64103150 2024-06-10T00:57:32.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.273+0000 7f2d617fa700 1 -- 172.21.15.37:0/2767402167 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f2d58019070 con 0x7f2d64103150 2024-06-10T00:57:32.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.274+0000 7f2d617fa700 1 --2- 172.21.15.37:0/2767402167 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2d4c0a2d30 0x7f2d4c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:32.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.274+0000 7f2d63fff700 1 --2- 172.21.15.37:0/2767402167 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2d4c0a2d30 0x7f2d4c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:32.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.274+0000 7f2d617fa700 1 -- 172.21.15.37:0/2767402167 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f2d580cf490 con 0x7f2d64103150 2024-06-10T00:57:32.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.275+0000 7f2d63fff700 1 --2- 172.21.15.37:0/2767402167 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2d4c0a2d30 0x7f2d4c0a51f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2d54004630 tx=0x7f2d5400b3c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:32.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.275+0000 7f2d75175b80 1 Processor -- start 2024-06-10T00:57:32.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.275+0000 7f2d75175b80 1 -- start start 2024-06-10T00:57:32.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.276+0000 7f2d75175b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600ce649f60 0x5600ce64a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:32.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.276+0000 7f2d75175b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ce64a880 0x5600ce644260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:32.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.276+0000 7f2d75175b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600ce6447a0 0x5600ce644bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:32.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.276+0000 7f2d75175b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5600ce536f60 con 0x5600ce649f60 2024-06-10T00:57:32.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.276+0000 7f2d75175b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5600ce595850 con 0x5600ce6447a0 2024-06-10T00:57:32.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.276+0000 7f2d75175b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5600ce4fd460 con 0x5600ce64a880 2024-06-10T00:57:32.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.276+0000 7f2d63fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600ce649f60 0x5600ce64a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:32.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.276+0000 7f2d63fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600ce649f60 0x5600ce64a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44356/0 (socket says 172.21.15.37:44356) 2024-06-10T00:57:32.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.276+0000 7f2d63fff700 1 -- 172.21.15.37:0/123542284 learned_addr learned my addr 172.21.15.37:0/123542284 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:32.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.276+0000 7f2d68e8f700 1 --2- 172.21.15.37:0/123542284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600ce6447a0 0x5600ce644bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:32.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.276+0000 7f2d637fe700 1 --2- 172.21.15.37:0/123542284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ce64a880 0x5600ce644260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:32.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.276+0000 7f2d63fff700 1 -- 172.21.15.37:0/123542284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ce64a880 msgr2=0x5600ce644260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:32.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.276+0000 7f2d63fff700 1 --2- 172.21.15.37:0/123542284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ce64a880 0x5600ce644260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.276+0000 7f2d63fff700 1 -- 172.21.15.37:0/123542284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600ce6447a0 msgr2=0x5600ce644bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:32.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.276+0000 7f2d63fff700 1 --2- 172.21.15.37:0/123542284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600ce6447a0 0x5600ce644bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.276+0000 7f2d63fff700 1 -- 172.21.15.37:0/123542284 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600ce4e48d0 con 0x5600ce649f60 2024-06-10T00:57:32.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.276+0000 7f2d68e8f700 1 --2- 172.21.15.37:0/123542284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600ce6447a0 0x5600ce644bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:57:32.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.276+0000 7f2d63fff700 1 --2- 172.21.15.37:0/123542284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600ce649f60 0x5600ce64a340 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2d54003b10 tx=0x7f2d54013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d560628c01f29f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:32.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.277+0000 7f2d4a7fc700 1 -- 172.21.15.37:0/123542284 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d54011ad0 con 0x5600ce649f60 2024-06-10T00:57:32.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.277+0000 7f2d4a7fc700 1 -- 172.21.15.37:0/123542284 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d54011c70 con 0x5600ce649f60 2024-06-10T00:57:32.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.277+0000 7f2d4a7fc700 1 -- 172.21.15.37:0/123542284 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5401c610 con 0x5600ce649f60 2024-06-10T00:57:32.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.277+0000 7f2d75175b80 1 -- 172.21.15.37:0/123542284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600ce649f60 msgr2=0x5600ce64a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:32.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.277+0000 7f2d75175b80 1 --2- 172.21.15.37:0/123542284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600ce649f60 0x5600ce64a340 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2d54003b10 tx=0x7f2d54013000 comp rx=0 tx=0).stop 2024-06-10T00:57:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.278+0000 7f2d75175b80 1 -- 172.21.15.37:0/123542284 shutdown_connections 2024-06-10T00:57:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.278+0000 7f2d75175b80 1 --2- 172.21.15.37:0/123542284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600ce6447a0 0x5600ce644bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.278+0000 7f2d75175b80 1 --2- 172.21.15.37:0/123542284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600ce649f60 0x5600ce64a340 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.278+0000 7f2d75175b80 1 --2- 172.21.15.37:0/123542284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ce64a880 0x5600ce644260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.278+0000 7f2d75175b80 1 -- 172.21.15.37:0/123542284 >> 172.21.15.37:0/123542284 conn(0x5600ce571080 msgr2=0x5600ce63f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.278+0000 7f2d75175b80 1 -- 172.21.15.37:0/123542284 shutdown_connections 2024-06-10T00:57:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.278+0000 7f2d75175b80 1 -- 172.21.15.37:0/123542284 wait complete. 2024-06-10T00:57:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.278+0000 7f2d75175b80 1 -- start start 2024-06-10T00:57:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.278+0000 7f2d75175b80 1 Processor -- start 2024-06-10T00:57:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.279+0000 7f2d75175b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600ce6447a0 0x5600ce6505c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.279+0000 7f2d75175b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600ce649f60 0x5600ce64d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.279+0000 7f2d75175b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ce64a880 0x5600ce64de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.279+0000 7f2d75175b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5600ce535fc0 con 0x5600ce6447a0 2024-06-10T00:57:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.279+0000 7f2d75175b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5600ce591810 con 0x5600ce649f60 2024-06-10T00:57:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.279+0000 7f2d75175b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5600ce6512b0 con 0x5600ce64a880 2024-06-10T00:57:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.279+0000 7f2d75175b80 10 client.0 _ll_register_callbacks cb 0x5600ce52b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.279+0000 7f2d63fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600ce6447a0 0x5600ce6505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.279+0000 7f2d63fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600ce6447a0 0x5600ce6505c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44366/0 (socket says 172.21.15.37:44366) 2024-06-10T00:57:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.279+0000 7f2d63fff700 1 -- 172.21.15.37:0/1614838823 learned_addr learned my addr 172.21.15.37:0/1614838823 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.279+0000 7f2d68e8f700 1 --2- 172.21.15.37:0/1614838823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ce64a880 0x5600ce64de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.279+0000 7f2d637fe700 1 --2- 172.21.15.37:0/1614838823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600ce649f60 0x5600ce64d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.279+0000 7f2d63fff700 1 -- 172.21.15.37:0/1614838823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ce64a880 msgr2=0x5600ce64de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.279+0000 7f2d63fff700 1 --2- 172.21.15.37:0/1614838823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ce64a880 0x5600ce64de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.279+0000 7f2d63fff700 1 -- 172.21.15.37:0/1614838823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600ce649f60 msgr2=0x5600ce64d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.279+0000 7f2d63fff700 1 --2- 172.21.15.37:0/1614838823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600ce649f60 0x5600ce64d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.279+0000 7f2d63fff700 1 -- 172.21.15.37:0/1614838823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600ce55b750 con 0x5600ce6447a0 2024-06-10T00:57:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.279+0000 7f2d63fff700 1 --2- 172.21.15.37:0/1614838823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600ce6447a0 0x5600ce6505c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2d54011720 tx=0x7f2d5401ffb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba38317e9594d645 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.279+0000 7f2d437fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:57:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.280+0000 7f2d437fe700 1 -- 172.21.15.37:0/1614838823 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d54035060 con 0x5600ce6447a0 2024-06-10T00:57:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.280+0000 7f2d437fe700 1 -- 172.21.15.37:0/1614838823 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f2d5401e420 con 0x5600ce6447a0 2024-06-10T00:57:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.280+0000 7f2d75175b80 1 -- 172.21.15.37:0/1614838823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d54011c70 con 0x5600ce6447a0 2024-06-10T00:57:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.280+0000 7f2d437fe700 1 -- 172.21.15.37:0/1614838823 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2d5403c050 con 0x5600ce6447a0 2024-06-10T00:57:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.280+0000 7f2d75175b80 10 client.15537 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.280+0000 7f2d75175b80 1 -- 172.21.15.37:0/1614838823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5600ce65ba40 con 0x5600ce6447a0 2024-06-10T00:57:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.280+0000 7f2d437fe700 1 -- 172.21.15.37:0/1614838823 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d54011c70 con 0x5600ce6447a0 2024-06-10T00:57:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.280+0000 7f2d437fe700 1 -- 172.21.15.37:0/1614838823 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2d54004830 con 0x5600ce6447a0 2024-06-10T00:57:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.281+0000 7f2d75175b80 10 client.15537 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.281+0000 7f2d75175b80 10 client.15537 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-06-10T00:57:32.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.281+0000 7f2d75175b80 20 client.15537 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:32.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.281+0000 7f2d75175b80 1 --2- 172.21.15.37:0/1614838823 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x5600ce65dfc0 0x5600ce6603b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:32.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.281+0000 7f2d75175b80 4 client.15537 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:57:32.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.281+0000 7f2d75175b80 1 -- 172.21.15.37:0/1614838823 --> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600ce5a2d50 con 0x5600ce65dfc0 2024-06-10T00:57:32.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.281+0000 7f2d637fe700 1 --2- 172.21.15.37:0/1614838823 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x5600ce65dfc0 0x5600ce6603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:32.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.281+0000 7f2d637fe700 1 --2- 172.21.15.37:0/1614838823 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x5600ce65dfc0 0x5600ce6603b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec3396be2d109157 server_cookie=3c7965b01a2177da in_seq=0 out_seq=0 2024-06-10T00:57:32.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.281+0000 7f2d437fe700 10 client.15537 ms_handle_connect on v2:172.21.15.94:6838/946269331 2024-06-10T00:57:32.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.282+0000 7f2d437fe700 1 -- 172.21.15.37:0/1614838823 <== mds.1 v2:172.21.15.94:6838/946269331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5600ce5a2d50 con 0x5600ce65dfc0 2024-06-10T00:57:32.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.282+0000 7f2d437fe700 10 client.15537 handle_command_reply: tid=0 2024-06-10T00:57:32.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.282+0000 7f2d75175b80 1 client.15537 shutdown 2024-06-10T00:57:32.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.282+0000 7f2d75175b80 1 -- 172.21.15.37:0/1614838823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600ce6447a0 msgr2=0x5600ce6505c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:32.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.282+0000 7f2d75175b80 1 --2- 172.21.15.37:0/1614838823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600ce6447a0 0x5600ce6505c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2d54011720 tx=0x7f2d5401ffb0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.282+0000 7f2d75175b80 1 -- 172.21.15.37:0/1614838823 shutdown_connections 2024-06-10T00:57:32.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.282+0000 7f2d75175b80 1 --2- 172.21.15.37:0/1614838823 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x5600ce65dfc0 0x5600ce6603b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.282+0000 7f2d75175b80 1 --2- 172.21.15.37:0/1614838823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ce64a880 0x5600ce64de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.282+0000 7f2d75175b80 1 --2- 172.21.15.37:0/1614838823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600ce6447a0 0x5600ce6505c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.282+0000 7f2d437fe700 0 client.15537 ms_handle_reset on v2:172.21.15.94:6838/946269331 2024-06-10T00:57:32.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.282+0000 7f2d75175b80 1 --2- 172.21.15.37:0/1614838823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600ce649f60 0x5600ce64d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.282+0000 7f2d75175b80 1 -- 172.21.15.37:0/1614838823 >> 172.21.15.37:0/1614838823 conn(0x5600ce571080 msgr2=0x5600ce63e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:32.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.282+0000 7f2d75175b80 1 -- 172.21.15.37:0/1614838823 shutdown_connections 2024-06-10T00:57:32.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.283+0000 7f2d75175b80 1 -- 172.21.15.37:0/1614838823 wait complete. 2024-06-10T00:57:32.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.283+0000 7f2d75175b80 20 client.15537 trim_cache size 0 max 16384 2024-06-10T00:57:32.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.298+0000 7f2d75175b80 1 Processor -- start 2024-06-10T00:57:32.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.298+0000 7f2d75175b80 1 -- start start 2024-06-10T00:57:32.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.299+0000 7f2d75175b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600ce6447a0 0x5600ce63ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:32.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.299+0000 7f2d75175b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ce649f60 0x5600ce63f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:32.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.299+0000 7f2d75175b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600ce64a880 0x5600ce64ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:32.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.299+0000 7f2d75175b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2d641b3010 con 0x5600ce6447a0 2024-06-10T00:57:32.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.299+0000 7f2d75175b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2d641b08a0 con 0x5600ce64a880 2024-06-10T00:57:32.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.299+0000 7f2d75175b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2d641ae6f0 con 0x5600ce649f60 2024-06-10T00:57:32.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.299+0000 7f2d63fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600ce6447a0 0x5600ce63ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:32.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.299+0000 7f2d68e8f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600ce64a880 0x5600ce64ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:32.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.299+0000 7f2d63fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600ce6447a0 0x5600ce63ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44368/0 (socket says 172.21.15.37:44368) 2024-06-10T00:57:32.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.299+0000 7f2d63fff700 1 -- 172.21.15.37:0/1314971467 learned_addr learned my addr 172.21.15.37:0/1314971467 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:32.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.299+0000 7f2d637fe700 1 --2- 172.21.15.37:0/1314971467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ce649f60 0x5600ce63f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:32.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.299+0000 7f2d68e8f700 1 -- 172.21.15.37:0/1314971467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ce649f60 msgr2=0x5600ce63f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:32.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.299+0000 7f2d68e8f700 1 --2- 172.21.15.37:0/1314971467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ce649f60 0x5600ce63f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.299+0000 7f2d68e8f700 1 -- 172.21.15.37:0/1314971467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600ce6447a0 msgr2=0x5600ce63ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:32.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.299+0000 7f2d68e8f700 1 --2- 172.21.15.37:0/1314971467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600ce6447a0 0x5600ce63ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.299+0000 7f2d68e8f700 1 -- 172.21.15.37:0/1314971467 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600ce64be50 con 0x5600ce64a880 2024-06-10T00:57:32.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.299+0000 7f2d63fff700 1 --2- 172.21.15.37:0/1314971467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600ce6447a0 0x5600ce63ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:57:32.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.299+0000 7f2d68e8f700 1 --2- 172.21.15.37:0/1314971467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600ce64a880 0x5600ce64ad20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c009fd0 tx=0x7f2d5c009e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c45a6f2b091ae6e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:32.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.300+0000 7f2d49ffb700 1 -- 172.21.15.37:0/1314971467 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c003a00 con 0x5600ce64a880 2024-06-10T00:57:32.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.300+0000 7f2d49ffb700 1 -- 172.21.15.37:0/1314971467 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d5c003ba0 con 0x5600ce64a880 2024-06-10T00:57:32.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.300+0000 7f2d49ffb700 1 -- 172.21.15.37:0/1314971467 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c005260 con 0x5600ce64a880 2024-06-10T00:57:32.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.300+0000 7f2d75175b80 1 -- 172.21.15.37:0/1314971467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600ce64a880 msgr2=0x5600ce64ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:32.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.300+0000 7f2d75175b80 1 --2- 172.21.15.37:0/1314971467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600ce64a880 0x5600ce64ad20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c009fd0 tx=0x7f2d5c009e20 comp rx=0 tx=0).stop 2024-06-10T00:57:32.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.300+0000 7f2d75175b80 1 -- 172.21.15.37:0/1314971467 shutdown_connections 2024-06-10T00:57:32.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.300+0000 7f2d75175b80 1 --2- 172.21.15.37:0/1314971467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600ce64a880 0x5600ce64ad20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.300+0000 7f2d75175b80 1 --2- 172.21.15.37:0/1314971467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600ce6447a0 0x5600ce63ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.300+0000 7f2d75175b80 1 --2- 172.21.15.37:0/1314971467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ce649f60 0x5600ce63f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.300+0000 7f2d75175b80 1 -- 172.21.15.37:0/1314971467 >> 172.21.15.37:0/1314971467 conn(0x5600ce571080 msgr2=0x5600ce647e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:32.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.301+0000 7f2d75175b80 1 -- 172.21.15.37:0/1314971467 shutdown_connections 2024-06-10T00:57:32.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.301+0000 7f2d75175b80 1 -- 172.21.15.37:0/1314971467 wait complete. 2024-06-10T00:57:32.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.301+0000 7f2d75175b80 1 -- start start 2024-06-10T00:57:32.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.301+0000 7f2d75175b80 1 Processor -- start 2024-06-10T00:57:32.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.301+0000 7f2d75175b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600ce6447a0 0x5600ce6603b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:32.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.301+0000 7f2d75175b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600ce649f60 0x5600ce6608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:32.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.301+0000 7f2d75175b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ce64a880 0x5600ce660e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:32.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.301+0000 7f2d75175b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2d641aed60 con 0x5600ce649f60 2024-06-10T00:57:32.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.302+0000 7f2d75175b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2d641b01d0 con 0x5600ce6447a0 2024-06-10T00:57:32.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.302+0000 7f2d75175b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2d641af430 con 0x5600ce64a880 2024-06-10T00:57:32.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.302+0000 7f2d75175b80 10 client.0 _ll_register_callbacks cb 0x5600ce52b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:32.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.302+0000 7f2d637fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600ce649f60 0x5600ce6608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:32.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.302+0000 7f2d637fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600ce649f60 0x5600ce6608f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44370/0 (socket says 172.21.15.37:44370) 2024-06-10T00:57:32.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.302+0000 7f2d637fe700 1 -- 172.21.15.37:0/3052295803 learned_addr learned my addr 172.21.15.37:0/3052295803 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:32.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.302+0000 7f2d63fff700 1 --2- 172.21.15.37:0/3052295803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600ce6447a0 0x5600ce6603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:32.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.302+0000 7f2d68e8f700 1 --2- 172.21.15.37:0/3052295803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ce64a880 0x5600ce660e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:32.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.302+0000 7f2d637fe700 1 -- 172.21.15.37:0/3052295803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ce64a880 msgr2=0x5600ce660e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:32.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.302+0000 7f2d637fe700 1 --2- 172.21.15.37:0/3052295803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ce64a880 0x5600ce660e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.302+0000 7f2d637fe700 1 -- 172.21.15.37:0/3052295803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600ce6447a0 msgr2=0x5600ce6603b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:32.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.302+0000 7f2d637fe700 1 --2- 172.21.15.37:0/3052295803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600ce6447a0 0x5600ce6603b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.302+0000 7f2d637fe700 1 -- 172.21.15.37:0/3052295803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600ce6c6b50 con 0x5600ce649f60 2024-06-10T00:57:32.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.302+0000 7f2d637fe700 1 --2- 172.21.15.37:0/3052295803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600ce649f60 0x5600ce6608f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2d5809b4a0 tx=0x7f2d58005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7334441a8bf6fd93 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:32.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.302+0000 7f2d437fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:57:32.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.302+0000 7f2d437fe700 1 -- 172.21.15.37:0/3052295803 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d580b8070 con 0x5600ce649f60 2024-06-10T00:57:32.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.302+0000 7f2d75175b80 1 -- 172.21.15.37:0/3052295803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d5c003ba0 con 0x5600ce649f60 2024-06-10T00:57:32.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.302+0000 7f2d437fe700 1 -- 172.21.15.37:0/3052295803 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f2d58010780 con 0x5600ce649f60 2024-06-10T00:57:32.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.303+0000 7f2d437fe700 1 -- 172.21.15.37:0/3052295803 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2d58020040 con 0x5600ce649f60 2024-06-10T00:57:32.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.303+0000 7f2d75175b80 10 client.15543 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:32.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.303+0000 7f2d75175b80 1 -- 172.21.15.37:0/3052295803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5600ce6c7e70 con 0x5600ce649f60 2024-06-10T00:57:32.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.303+0000 7f2d437fe700 1 -- 172.21.15.37:0/3052295803 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d5c003ba0 con 0x5600ce649f60 2024-06-10T00:57:32.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.303+0000 7f2d437fe700 1 -- 172.21.15.37:0/3052295803 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2d580cf140 con 0x5600ce649f60 2024-06-10T00:57:32.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.303+0000 7f2d75175b80 10 client.15543 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:32.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.303+0000 7f2d75175b80 10 client.15543 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-06-10T00:57:32.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.303+0000 7f2d75175b80 20 client.15543 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:32.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.303+0000 7f2d75175b80 1 --2- 172.21.15.37:0/3052295803 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x5600ce65dfc0 0x5600ce6ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:32.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.303+0000 7f2d75175b80 4 client.15543 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:57:32.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.303+0000 7f2d75175b80 1 -- 172.21.15.37:0/3052295803 --> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5600ce59d2d0 con 0x5600ce65dfc0 2024-06-10T00:57:32.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.303+0000 7f2d63fff700 1 --2- 172.21.15.37:0/3052295803 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x5600ce65dfc0 0x5600ce6ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:32.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.304+0000 7f2d63fff700 1 --2- 172.21.15.37:0/3052295803 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x5600ce65dfc0 0x5600ce6ca590 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a22af7575cffb7ae server_cookie=42cec8f1a09af56f in_seq=0 out_seq=0 2024-06-10T00:57:32.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.304+0000 7f2d437fe700 10 client.15543 ms_handle_connect on v2:172.21.15.94:6838/946269331 2024-06-10T00:57:32.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.304+0000 7f2d437fe700 1 -- 172.21.15.37:0/3052295803 <== mds.1 v2:172.21.15.94:6838/946269331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5600ce59d2d0 con 0x5600ce65dfc0 2024-06-10T00:57:32.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.304+0000 7f2d437fe700 10 client.15543 handle_command_reply: tid=0 2024-06-10T00:57:32.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.304+0000 7f2d75175b80 1 client.15543 shutdown 2024-06-10T00:57:32.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.304+0000 7f2d75175b80 1 -- 172.21.15.37:0/3052295803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600ce649f60 msgr2=0x5600ce6608f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:32.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.304+0000 7f2d75175b80 1 --2- 172.21.15.37:0/3052295803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600ce649f60 0x5600ce6608f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2d5809b4a0 tx=0x7f2d58005dc0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.304+0000 7f2d75175b80 1 -- 172.21.15.37:0/3052295803 shutdown_connections 2024-06-10T00:57:32.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.304+0000 7f2d75175b80 1 --2- 172.21.15.37:0/3052295803 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x5600ce65dfc0 0x5600ce6ca590 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.304+0000 7f2d75175b80 1 --2- 172.21.15.37:0/3052295803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5600ce64a880 0x5600ce660e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.304+0000 7f2d75175b80 1 --2- 172.21.15.37:0/3052295803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5600ce6447a0 0x5600ce6603b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.304+0000 7f2d75175b80 1 --2- 172.21.15.37:0/3052295803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600ce649f60 0x5600ce6608f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.304+0000 7f2d75175b80 1 -- 172.21.15.37:0/3052295803 >> 172.21.15.37:0/3052295803 conn(0x5600ce571080 msgr2=0x5600ce64ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:32.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.304+0000 7f2d437fe700 0 client.15543 ms_handle_reset on v2:172.21.15.94:6838/946269331 2024-06-10T00:57:32.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.304+0000 7f2d75175b80 1 -- 172.21.15.37:0/3052295803 shutdown_connections 2024-06-10T00:57:32.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.304+0000 7f2d75175b80 1 -- 172.21.15.37:0/3052295803 wait complete. 2024-06-10T00:57:32.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.304+0000 7f2d75175b80 20 client.15543 trim_cache size 0 max 16384 2024-06-10T00:57:32.309 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T00:57:32.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.305+0000 7f2d6a8f2700 1 -- 172.21.15.37:0/2767402167 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2d4c0a2d30 msgr2=0x7f2d4c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:32.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.305+0000 7f2d6a8f2700 1 --2- 172.21.15.37:0/2767402167 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2d4c0a2d30 0x7f2d4c0a51f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2d54004630 tx=0x7f2d5400b3c0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.305+0000 7f2d6a8f2700 1 -- 172.21.15.37:0/2767402167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d64103150 msgr2=0x7f2d64074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:32.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.305+0000 7f2d6a8f2700 1 --2- 172.21.15.37:0/2767402167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d64103150 0x7f2d64074cd0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800d950 tx=0x7f2d5800dc60 comp rx=0 tx=0).stop 2024-06-10T00:57:32.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.305+0000 7f2d6a8f2700 1 -- 172.21.15.37:0/2767402167 shutdown_connections 2024-06-10T00:57:32.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.305+0000 7f2d6a8f2700 1 --2- 172.21.15.37:0/2767402167 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2d4c0a2d30 0x7f2d4c0a51f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.305+0000 7f2d6a8f2700 1 --2- 172.21.15.37:0/2767402167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d64107870 0x7f2d64075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.305+0000 7f2d6a8f2700 1 --2- 172.21.15.37:0/2767402167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d641027a0 0x7f2d64074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.305+0000 7f2d6a8f2700 1 --2- 172.21.15.37:0/2767402167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d64103150 0x7f2d64074cd0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:32.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.305+0000 7f2d6a8f2700 1 -- 172.21.15.37:0/2767402167 >> 172.21.15.37:0/2767402167 conn(0x7f2d64079520 msgr2=0x7f2d64100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:32.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.305+0000 7f2d6a8f2700 1 -- 172.21.15.37:0/2767402167 shutdown_connections 2024-06-10T00:57:32.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:32.305+0000 7f2d6a8f2700 1 -- 172.21.15.37:0/2767402167 wait complete. 2024-06-10T00:57:32.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:32 smithi167 ceph-mon[36917]: pgmap v455: 129 pgs: 129 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-10T00:57:32.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:32 smithi094 ceph-mon[36708]: pgmap v455: 129 pgs: 129 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-10T00:57:33.100 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T00:57:33.384 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:57:34.391 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:34 smithi167 ceph-mon[36917]: pgmap v456: 129 pgs: 129 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-10T00:57:34.423 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:34 smithi094 ceph-mon[36708]: pgmap v456: 129 pgs: 129 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-10T00:57:34.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:34 smithi037 ceph-mon[32192]: pgmap v456: 129 pgs: 129 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-10T00:57:35.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.192+0000 7fefe1bc5700 1 -- 172.21.15.37:0/3466962464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefdc107400 msgr2=0x7fefdc1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:35.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.192+0000 7fefe1bc5700 1 --2- 172.21.15.37:0/3466962464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefdc107400 0x7fefdc1077e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fefcc009990 tx=0x7fefcc009ca0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.192+0000 7fefe1bc5700 1 -- 172.21.15.37:0/3466962464 shutdown_connections 2024-06-10T00:57:35.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.192+0000 7fefe1bc5700 1 --2- 172.21.15.37:0/3466962464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefdc101d50 0x7fefdc10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.192+0000 7fefe1bc5700 1 --2- 172.21.15.37:0/3466962464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefdc107400 0x7fefdc1077e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.192+0000 7fefe1bc5700 1 --2- 172.21.15.37:0/3466962464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefdc101390 0x7fefdc101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.192+0000 7fefe1bc5700 1 -- 172.21.15.37:0/3466962464 >> 172.21.15.37:0/3466962464 conn(0x7fefdc078340 msgr2=0x7fefdc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:35.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.192+0000 7fefe1bc5700 1 -- 172.21.15.37:0/3466962464 shutdown_connections 2024-06-10T00:57:35.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.192+0000 7fefe1bc5700 1 -- 172.21.15.37:0/3466962464 wait complete. 2024-06-10T00:57:35.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.193+0000 7fefe1bc5700 1 Processor -- start 2024-06-10T00:57:35.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.193+0000 7fefe1bc5700 1 -- start start 2024-06-10T00:57:35.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.193+0000 7fefe1bc5700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefdc101390 0x7fefdc19e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.193+0000 7fefe1bc5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefdc101d50 0x7fefdc19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.193+0000 7fefe1bc5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefdc107400 0x7fefdc198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.193+0000 7fefe1bc5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc111670 con 0x7fefdc107400 2024-06-10T00:57:35.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.193+0000 7fefe1bc5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc1114f0 con 0x7fefdc101d50 2024-06-10T00:57:35.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.193+0000 7fefe1bc5700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc1117f0 con 0x7fefdc101390 2024-06-10T00:57:35.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.193+0000 7fefdbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefdc107400 0x7fefdc198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.194+0000 7fefdbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefdc107400 0x7fefdc198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44396/0 (socket says 172.21.15.37:44396) 2024-06-10T00:57:35.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.194+0000 7fefdbfff700 1 -- 172.21.15.37:0/724019268 learned_addr learned my addr 172.21.15.37:0/724019268 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:35.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.194+0000 7fefdaffd700 1 --2- 172.21.15.37:0/724019268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefdc101d50 0x7fefdc19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.194+0000 7fefdb7fe700 1 --2- 172.21.15.37:0/724019268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefdc101390 0x7fefdc19e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.194+0000 7fefdbfff700 1 -- 172.21.15.37:0/724019268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefdc101390 msgr2=0x7fefdc19e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:35.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.194+0000 7fefdbfff700 1 --2- 172.21.15.37:0/724019268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefdc101390 0x7fefdc19e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.194+0000 7fefdbfff700 1 -- 172.21.15.37:0/724019268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefdc101d50 msgr2=0x7fefdc19ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:35.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.194+0000 7fefdbfff700 1 --2- 172.21.15.37:0/724019268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefdc101d50 0x7fefdc19ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.194+0000 7fefdbfff700 1 -- 172.21.15.37:0/724019268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefdc199220 con 0x7fefdc107400 2024-06-10T00:57:35.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.194+0000 7fefdaffd700 1 --2- 172.21.15.37:0/724019268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefdc101d50 0x7fefdc19ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:57:35.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.194+0000 7fefdbfff700 1 --2- 172.21.15.37:0/724019268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefdc107400 0x7fefdc198ac0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fefd4007ee0 tx=0x7fefd400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:35.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.194+0000 7fefd8ff9700 1 -- 172.21.15.37:0/724019268 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4004bb0 con 0x7fefdc107400 2024-06-10T00:57:35.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.194+0000 7fefd8ff9700 1 -- 172.21.15.37:0/724019268 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefd4004d50 con 0x7fefdc107400 2024-06-10T00:57:35.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.195+0000 7fefd8ff9700 1 -- 172.21.15.37:0/724019268 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4005960 con 0x7fefdc107400 2024-06-10T00:57:35.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.195+0000 7fefe1bc5700 1 -- 172.21.15.37:0/724019268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefdc199510 con 0x7fefdc107400 2024-06-10T00:57:35.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.195+0000 7fefe1bc5700 1 -- 172.21.15.37:0/724019268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefdc1a57d0 con 0x7fefdc107400 2024-06-10T00:57:35.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.198+0000 7fefd8ff9700 1 -- 172.21.15.37:0/724019268 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fefd401a020 con 0x7fefdc107400 2024-06-10T00:57:35.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.198+0000 7fefd8ff9700 1 --2- 172.21.15.37:0/724019268 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fefc40a2cb0 0x7fefc40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.199+0000 7fefdb7fe700 1 --2- 172.21.15.37:0/724019268 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fefc40a2cb0 0x7fefc40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.199+0000 7fefd8ff9700 1 -- 172.21.15.37:0/724019268 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fefd40cec90 con 0x7fefdc107400 2024-06-10T00:57:35.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.199+0000 7fefdb7fe700 1 --2- 172.21.15.37:0/724019268 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fefc40a2cb0 0x7fefc40a5170 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fefcc00b5c0 tx=0x7fefcc005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:35.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.199+0000 7fefec448b80 1 Processor -- start 2024-06-10T00:57:35.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.199+0000 7fefec448b80 1 -- start start 2024-06-10T00:57:35.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.200+0000 7fefec448b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598e6b9a470 0x5598e6b9a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.200+0000 7fefec448b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598e6b94670 0x5598e6b94af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.200+0000 7fefec448b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598e6b95030 0x5598e6ba1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.200+0000 7fefec448b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5598e6a82f60 con 0x5598e6b9a470 2024-06-10T00:57:35.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.200+0000 7fefec448b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5598e6ae1850 con 0x5598e6b95030 2024-06-10T00:57:35.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.200+0000 7fefec448b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5598e6a49460 con 0x5598e6b94670 2024-06-10T00:57:35.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.200+0000 7fefdb7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598e6b9a470 0x5598e6b9a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.200+0000 7fefdb7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598e6b9a470 0x5598e6b9a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44410/0 (socket says 172.21.15.37:44410) 2024-06-10T00:57:35.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.200+0000 7fefdb7fe700 1 -- 172.21.15.37:0/1610457861 learned_addr learned my addr 172.21.15.37:0/1610457861 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:35.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.200+0000 7fefdaffd700 1 --2- 172.21.15.37:0/1610457861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598e6b94670 0x5598e6b94af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.200+0000 7fefdbfff700 1 --2- 172.21.15.37:0/1610457861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598e6b95030 0x5598e6ba1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.200+0000 7fefdb7fe700 1 -- 172.21.15.37:0/1610457861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598e6b94670 msgr2=0x5598e6b94af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:35.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.200+0000 7fefdb7fe700 1 --2- 172.21.15.37:0/1610457861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598e6b94670 0x5598e6b94af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.200+0000 7fefdb7fe700 1 -- 172.21.15.37:0/1610457861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598e6b95030 msgr2=0x5598e6ba1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:35.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.200+0000 7fefdb7fe700 1 --2- 172.21.15.37:0/1610457861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598e6b95030 0x5598e6ba1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.200+0000 7fefdb7fe700 1 -- 172.21.15.37:0/1610457861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598e6a308d0 con 0x5598e6b9a470 2024-06-10T00:57:35.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.200+0000 7fefdbfff700 1 --2- 172.21.15.37:0/1610457861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598e6b95030 0x5598e6ba1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:57:35.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.200+0000 7fefdaffd700 1 --2- 172.21.15.37:0/1610457861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598e6b94670 0x5598e6b94af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:57:35.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.201+0000 7fefdb7fe700 1 --2- 172.21.15.37:0/1610457861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598e6b9a470 0x5598e6b9a850 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fefcc009990 tx=0x7fefcc020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31db5b65a92e1247 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:35.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.201+0000 7fefc1ffb700 1 -- 172.21.15.37:0/1610457861 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc00f4e0 con 0x5598e6b9a470 2024-06-10T00:57:35.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.201+0000 7fefc1ffb700 1 -- 172.21.15.37:0/1610457861 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefcc00fb60 con 0x5598e6b9a470 2024-06-10T00:57:35.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.201+0000 7fefc1ffb700 1 -- 172.21.15.37:0/1610457861 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc024cd0 con 0x5598e6b9a470 2024-06-10T00:57:35.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.202+0000 7fefec448b80 1 -- 172.21.15.37:0/1610457861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598e6b9a470 msgr2=0x5598e6b9a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:35.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.202+0000 7fefec448b80 1 --2- 172.21.15.37:0/1610457861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598e6b9a470 0x5598e6b9a850 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fefcc009990 tx=0x7fefcc020000 comp rx=0 tx=0).stop 2024-06-10T00:57:35.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.202+0000 7fefec448b80 1 -- 172.21.15.37:0/1610457861 shutdown_connections 2024-06-10T00:57:35.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.202+0000 7fefec448b80 1 --2- 172.21.15.37:0/1610457861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598e6b95030 0x5598e6ba1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.202+0000 7fefec448b80 1 --2- 172.21.15.37:0/1610457861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598e6b9a470 0x5598e6b9a850 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.202+0000 7fefec448b80 1 --2- 172.21.15.37:0/1610457861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598e6b94670 0x5598e6b94af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.202+0000 7fefec448b80 1 -- 172.21.15.37:0/1610457861 >> 172.21.15.37:0/1610457861 conn(0x5598e6abd080 msgr2=0x5598e6b8b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:35.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.202+0000 7fefec448b80 1 -- 172.21.15.37:0/1610457861 shutdown_connections 2024-06-10T00:57:35.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.202+0000 7fefec448b80 1 -- 172.21.15.37:0/1610457861 wait complete. 2024-06-10T00:57:35.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.202+0000 7fefec448b80 1 -- start start 2024-06-10T00:57:35.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.202+0000 7fefec448b80 1 Processor -- start 2024-06-10T00:57:35.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.203+0000 7fefec448b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598e6b94670 0x5598e6ba0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.203+0000 7fefec448b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598e6b95030 0x5598e6b9e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.203+0000 7fefec448b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598e6b9a470 0x5598e6b9e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.203+0000 7fefec448b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5598e6a81fc0 con 0x5598e6b9a470 2024-06-10T00:57:35.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.203+0000 7fefec448b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5598e6add810 con 0x5598e6b94670 2024-06-10T00:57:35.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.203+0000 7fefec448b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5598e6ba1750 con 0x5598e6b95030 2024-06-10T00:57:35.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.203+0000 7fefec448b80 10 client.0 _ll_register_callbacks cb 0x5598e6a775b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:35.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.203+0000 7fefdbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598e6b9a470 0x5598e6b9e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.203+0000 7fefdbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598e6b9a470 0x5598e6b9e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44416/0 (socket says 172.21.15.37:44416) 2024-06-10T00:57:35.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.203+0000 7fefdbfff700 1 -- 172.21.15.37:0/1450868023 learned_addr learned my addr 172.21.15.37:0/1450868023 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:35.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.203+0000 7fefdaffd700 1 --2- 172.21.15.37:0/1450868023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598e6b95030 0x5598e6b9e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.203+0000 7fefdb7fe700 1 --2- 172.21.15.37:0/1450868023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598e6b94670 0x5598e6ba0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.203+0000 7fefdbfff700 1 -- 172.21.15.37:0/1450868023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598e6b95030 msgr2=0x5598e6b9e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:35.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.203+0000 7fefdbfff700 1 --2- 172.21.15.37:0/1450868023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598e6b95030 0x5598e6b9e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.203+0000 7fefdbfff700 1 -- 172.21.15.37:0/1450868023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598e6b94670 msgr2=0x5598e6ba0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:35.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.203+0000 7fefdbfff700 1 --2- 172.21.15.37:0/1450868023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598e6b94670 0x5598e6ba0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.203+0000 7fefdbfff700 1 -- 172.21.15.37:0/1450868023 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598e6aa7750 con 0x5598e6b9a470 2024-06-10T00:57:35.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.203+0000 7fefdbfff700 1 --2- 172.21.15.37:0/1450868023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598e6b9a470 0x5598e6b9e690 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fefd40a36e0 tx=0x7fefd409ae80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b8ca1422d4f870 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:35.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.203+0000 7fefbaffd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:57:35.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.204+0000 7fefbaffd700 1 -- 172.21.15.37:0/1450868023 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4022450 con 0x5598e6b9a470 2024-06-10T00:57:35.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.204+0000 7fefbaffd700 1 -- 172.21.15.37:0/1450868023 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fefd4004f00 con 0x5598e6b9a470 2024-06-10T00:57:35.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.204+0000 7fefec448b80 1 -- 172.21.15.37:0/1450868023 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefcc00fb60 con 0x5598e6b9a470 2024-06-10T00:57:35.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.204+0000 7fefbaffd700 1 -- 172.21.15.37:0/1450868023 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fefd409f350 con 0x5598e6b9a470 2024-06-10T00:57:35.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.204+0000 7fefec448b80 10 client.15555 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:35.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.204+0000 7fefec448b80 1 -- 172.21.15.37:0/1450868023 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5598e6babd10 con 0x5598e6b9a470 2024-06-10T00:57:35.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.204+0000 7fefbaffd700 1 -- 172.21.15.37:0/1450868023 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefcc00fb60 con 0x5598e6b9a470 2024-06-10T00:57:35.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.204+0000 7fefbaffd700 1 -- 172.21.15.37:0/1450868023 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fefd409b200 con 0x5598e6b9a470 2024-06-10T00:57:35.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.204+0000 7fefec448b80 10 client.15555 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:35.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.204+0000 7fefec448b80 10 client.15555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T00:57:35.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.204+0000 7fefec448b80 20 client.15555 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:35.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.204+0000 7fefec448b80 1 --2- 172.21.15.37:0/1450868023 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5598e6bae2f0 0x5598e6bb0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.204+0000 7fefec448b80 4 client.15555 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:57:35.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.204+0000 7fefec448b80 1 -- 172.21.15.37:0/1450868023 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598e6aeed50 con 0x5598e6bae2f0 2024-06-10T00:57:35.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.204+0000 7fefdb7fe700 1 --2- 172.21.15.37:0/1450868023 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5598e6bae2f0 0x5598e6bb0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.205+0000 7fefdb7fe700 1 --2- 172.21.15.37:0/1450868023 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5598e6bae2f0 0x5598e6bb0750 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c16461ebc523276f server_cookie=e3efc92417414c59 in_seq=0 out_seq=0 2024-06-10T00:57:35.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.205+0000 7fefbaffd700 10 client.15555 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:35.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.206+0000 7fefbaffd700 1 -- 172.21.15.37:0/1450868023 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5598e6aeed50 con 0x5598e6bae2f0 2024-06-10T00:57:35.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.206+0000 7fefbaffd700 10 client.15555 handle_command_reply: tid=0 2024-06-10T00:57:35.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.206+0000 7fefec448b80 1 client.15555 shutdown 2024-06-10T00:57:35.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.206+0000 7fefec448b80 1 -- 172.21.15.37:0/1450868023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598e6b9a470 msgr2=0x5598e6b9e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:35.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.206+0000 7fefec448b80 1 --2- 172.21.15.37:0/1450868023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598e6b9a470 0x5598e6b9e690 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fefd40a36e0 tx=0x7fefd409ae80 comp rx=0 tx=0).stop 2024-06-10T00:57:35.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.206+0000 7fefec448b80 1 -- 172.21.15.37:0/1450868023 shutdown_connections 2024-06-10T00:57:35.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.206+0000 7fefec448b80 1 --2- 172.21.15.37:0/1450868023 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5598e6bae2f0 0x5598e6bb0750 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.206+0000 7fefec448b80 1 --2- 172.21.15.37:0/1450868023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598e6b9a470 0x5598e6b9e690 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.206+0000 7fefec448b80 1 --2- 172.21.15.37:0/1450868023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598e6b94670 0x5598e6ba0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.206+0000 7fefbaffd700 0 client.15555 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:35.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.206+0000 7fefec448b80 1 --2- 172.21.15.37:0/1450868023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598e6b95030 0x5598e6b9e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.206+0000 7fefec448b80 1 -- 172.21.15.37:0/1450868023 >> 172.21.15.37:0/1450868023 conn(0x5598e6abd080 msgr2=0x5598e6b8a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:35.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.206+0000 7fefec448b80 1 -- 172.21.15.37:0/1450868023 shutdown_connections 2024-06-10T00:57:35.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.206+0000 7fefec448b80 1 -- 172.21.15.37:0/1450868023 wait complete. 2024-06-10T00:57:35.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.206+0000 7fefec448b80 20 client.15555 trim_cache size 0 max 16384 2024-06-10T00:57:35.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.221+0000 7fefec448b80 1 Processor -- start 2024-06-10T00:57:35.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.222+0000 7fefec448b80 1 -- start start 2024-06-10T00:57:35.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.222+0000 7fefec448b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598e6b94670 0x5598e6b99970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.222+0000 7fefec448b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598e6b95030 0x5598e6b8ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.222+0000 7fefec448b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598e6b9a470 0x5598e6b8b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.222+0000 7fefec448b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc1aea80 con 0x5598e6b9a470 2024-06-10T00:57:35.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.222+0000 7fefec448b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc1ac310 con 0x5598e6b95030 2024-06-10T00:57:35.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.222+0000 7fefec448b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc1aa160 con 0x5598e6b94670 2024-06-10T00:57:35.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.222+0000 7fefdbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598e6b9a470 0x5598e6b8b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.222+0000 7fefdbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598e6b9a470 0x5598e6b8b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44428/0 (socket says 172.21.15.37:44428) 2024-06-10T00:57:35.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.222+0000 7fefdbfff700 1 -- 172.21.15.37:0/3864459987 learned_addr learned my addr 172.21.15.37:0/3864459987 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:35.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.222+0000 7fefdaffd700 1 --2- 172.21.15.37:0/3864459987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598e6b95030 0x5598e6b8ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.222+0000 7fefdb7fe700 1 --2- 172.21.15.37:0/3864459987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598e6b94670 0x5598e6b99970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.222+0000 7fefdbfff700 1 -- 172.21.15.37:0/3864459987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598e6b94670 msgr2=0x5598e6b99970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:35.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.222+0000 7fefdbfff700 1 --2- 172.21.15.37:0/3864459987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598e6b94670 0x5598e6b99970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.222+0000 7fefdbfff700 1 -- 172.21.15.37:0/3864459987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598e6b95030 msgr2=0x5598e6b8ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:35.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.222+0000 7fefdbfff700 1 --2- 172.21.15.37:0/3864459987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598e6b95030 0x5598e6b8ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.222+0000 7fefdbfff700 1 -- 172.21.15.37:0/3864459987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598e6b8be40 con 0x5598e6b9a470 2024-06-10T00:57:35.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.222+0000 7fefdaffd700 1 --2- 172.21.15.37:0/3864459987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598e6b95030 0x5598e6b8ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:57:35.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.223+0000 7fefdbfff700 1 --2- 172.21.15.37:0/3864459987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598e6b9a470 0x5598e6b8b3c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fefd409af30 tx=0x7fefd409b120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efda5f5813ed5f9a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:35.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.223+0000 7fefc17fa700 1 -- 172.21.15.37:0/3864459987 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4097240 con 0x5598e6b9a470 2024-06-10T00:57:35.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.223+0000 7fefc17fa700 1 -- 172.21.15.37:0/3864459987 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefd409b6a0 con 0x5598e6b9a470 2024-06-10T00:57:35.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.223+0000 7fefc17fa700 1 -- 172.21.15.37:0/3864459987 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4004f00 con 0x5598e6b9a470 2024-06-10T00:57:35.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.224+0000 7fefec448b80 1 -- 172.21.15.37:0/3864459987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598e6b9a470 msgr2=0x5598e6b8b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:35.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.224+0000 7fefec448b80 1 --2- 172.21.15.37:0/3864459987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598e6b9a470 0x5598e6b8b3c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fefd409af30 tx=0x7fefd409b120 comp rx=0 tx=0).stop 2024-06-10T00:57:35.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.224+0000 7fefec448b80 1 -- 172.21.15.37:0/3864459987 shutdown_connections 2024-06-10T00:57:35.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.224+0000 7fefec448b80 1 --2- 172.21.15.37:0/3864459987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598e6b9a470 0x5598e6b8b3c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.224+0000 7fefec448b80 1 --2- 172.21.15.37:0/3864459987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598e6b94670 0x5598e6b99970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.224+0000 7fefec448b80 1 --2- 172.21.15.37:0/3864459987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598e6b95030 0x5598e6b8ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.224+0000 7fefec448b80 1 -- 172.21.15.37:0/3864459987 >> 172.21.15.37:0/3864459987 conn(0x5598e6abd080 msgr2=0x5598e6b99260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:35.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.224+0000 7fefec448b80 1 -- 172.21.15.37:0/3864459987 shutdown_connections 2024-06-10T00:57:35.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.224+0000 7fefec448b80 1 -- 172.21.15.37:0/3864459987 wait complete. 2024-06-10T00:57:35.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.224+0000 7fefec448b80 1 -- start start 2024-06-10T00:57:35.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.224+0000 7fefec448b80 1 Processor -- start 2024-06-10T00:57:35.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.225+0000 7fefec448b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598e6b94670 0x5598e6ba6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.225+0000 7fefec448b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598e6b95030 0x5598e6ba6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.225+0000 7fefec448b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598e6b9a470 0x5598e6bb53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.225+0000 7fefec448b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc1aa7d0 con 0x5598e6b94670 2024-06-10T00:57:35.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.225+0000 7fefec448b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc1abc40 con 0x5598e6b95030 2024-06-10T00:57:35.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.225+0000 7fefec448b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc1aaea0 con 0x5598e6b9a470 2024-06-10T00:57:35.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.225+0000 7fefec448b80 10 client.0 _ll_register_callbacks cb 0x5598e6a775b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:35.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.225+0000 7fefdb7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598e6b94670 0x5598e6ba6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.225+0000 7fefdb7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598e6b94670 0x5598e6ba6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44444/0 (socket says 172.21.15.37:44444) 2024-06-10T00:57:35.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.225+0000 7fefdb7fe700 1 -- 172.21.15.37:0/2947096661 learned_addr learned my addr 172.21.15.37:0/2947096661 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:35.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.225+0000 7fefdaffd700 1 --2- 172.21.15.37:0/2947096661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598e6b95030 0x5598e6ba6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.225+0000 7fefdbfff700 1 --2- 172.21.15.37:0/2947096661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598e6b9a470 0x5598e6bb53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.225+0000 7fefdb7fe700 1 -- 172.21.15.37:0/2947096661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598e6b9a470 msgr2=0x5598e6bb53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:35.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.225+0000 7fefdb7fe700 1 --2- 172.21.15.37:0/2947096661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598e6b9a470 0x5598e6bb53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.225+0000 7fefdb7fe700 1 -- 172.21.15.37:0/2947096661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598e6b95030 msgr2=0x5598e6ba6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:35.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.225+0000 7fefdb7fe700 1 --2- 172.21.15.37:0/2947096661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598e6b95030 0x5598e6ba6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.225+0000 7fefdb7fe700 1 -- 172.21.15.37:0/2947096661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598e6b9ac80 con 0x5598e6b94670 2024-06-10T00:57:35.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.225+0000 7fefdb7fe700 1 --2- 172.21.15.37:0/2947096661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598e6b94670 0x5598e6ba6a40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fefcc009350 tx=0x7fefcc005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bdab89479aebd3e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:35.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.225+0000 7fefbaffd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:57:35.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.226+0000 7fefbaffd700 1 -- 172.21.15.37:0/2947096661 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc048060 con 0x5598e6b94670 2024-06-10T00:57:35.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.226+0000 7fefec448b80 1 -- 172.21.15.37:0/2947096661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefd409b6a0 con 0x5598e6b94670 2024-06-10T00:57:35.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.226+0000 7fefbaffd700 1 -- 172.21.15.37:0/2947096661 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fefcc035040 con 0x5598e6b94670 2024-06-10T00:57:35.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.226+0000 7fefbaffd700 1 -- 172.21.15.37:0/2947096661 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fefcc04f050 con 0x5598e6b94670 2024-06-10T00:57:35.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.226+0000 7fefec448b80 10 client.15561 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:35.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.226+0000 7fefec448b80 1 -- 172.21.15.37:0/2947096661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5598e6c18730 con 0x5598e6b94670 2024-06-10T00:57:35.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.226+0000 7fefbaffd700 1 -- 172.21.15.37:0/2947096661 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefd409b6a0 con 0x5598e6b94670 2024-06-10T00:57:35.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.226+0000 7fefbaffd700 1 -- 172.21.15.37:0/2947096661 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fefcc00fcc0 con 0x5598e6b94670 2024-06-10T00:57:35.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.226+0000 7fefec448b80 10 client.15561 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:35.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.226+0000 7fefec448b80 10 client.15561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T00:57:35.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.226+0000 7fefec448b80 20 client.15561 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:35.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.226+0000 7fefec448b80 1 --2- 172.21.15.37:0/2947096661 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5598e6c189c0 0x5598e6c1adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.226+0000 7fefec448b80 4 client.15561 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:57:35.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.226+0000 7fefec448b80 1 -- 172.21.15.37:0/2947096661 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5598e6ae92d0 con 0x5598e6c189c0 2024-06-10T00:57:35.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.227+0000 7fefdaffd700 1 --2- 172.21.15.37:0/2947096661 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5598e6c189c0 0x5598e6c1adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.227+0000 7fefdaffd700 1 --2- 172.21.15.37:0/2947096661 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5598e6c189c0 0x5598e6c1adb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb17a92d306bfcc1 server_cookie=1a4ed1ab7013bab6 in_seq=0 out_seq=0 2024-06-10T00:57:35.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.227+0000 7fefbaffd700 10 client.15561 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:35.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.227+0000 7fefbaffd700 1 -- 172.21.15.37:0/2947096661 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5598e6ae92d0 con 0x5598e6c189c0 2024-06-10T00:57:35.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.227+0000 7fefbaffd700 10 client.15561 handle_command_reply: tid=0 2024-06-10T00:57:35.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.227+0000 7fefec448b80 1 client.15561 shutdown 2024-06-10T00:57:35.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.228+0000 7fefec448b80 1 -- 172.21.15.37:0/2947096661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598e6b94670 msgr2=0x5598e6ba6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:35.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.228+0000 7fefec448b80 1 --2- 172.21.15.37:0/2947096661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598e6b94670 0x5598e6ba6a40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fefcc009350 tx=0x7fefcc005fb0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.228+0000 7fefec448b80 1 -- 172.21.15.37:0/2947096661 shutdown_connections 2024-06-10T00:57:35.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.228+0000 7fefec448b80 1 --2- 172.21.15.37:0/2947096661 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5598e6c189c0 0x5598e6c1adb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.228+0000 7fefec448b80 1 --2- 172.21.15.37:0/2947096661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5598e6b9a470 0x5598e6bb53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.228+0000 7fefbaffd700 0 client.15561 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:35.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.228+0000 7fefec448b80 1 --2- 172.21.15.37:0/2947096661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598e6b94670 0x5598e6ba6a40 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.228+0000 7fefec448b80 1 --2- 172.21.15.37:0/2947096661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5598e6b95030 0x5598e6ba6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.228+0000 7fefec448b80 1 -- 172.21.15.37:0/2947096661 >> 172.21.15.37:0/2947096661 conn(0x5598e6abd080 msgr2=0x5598e6b97e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:35.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.228+0000 7fefec448b80 1 -- 172.21.15.37:0/2947096661 shutdown_connections 2024-06-10T00:57:35.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.228+0000 7fefec448b80 1 -- 172.21.15.37:0/2947096661 wait complete. 2024-06-10T00:57:35.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.228+0000 7fefec448b80 20 client.15561 trim_cache size 0 max 16384 2024-06-10T00:57:35.235 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T00:57:35.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.228+0000 7fefe1bc5700 1 -- 172.21.15.37:0/724019268 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fefc40a2cb0 msgr2=0x7fefc40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:35.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.229+0000 7fefe1bc5700 1 --2- 172.21.15.37:0/724019268 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fefc40a2cb0 0x7fefc40a5170 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fefcc00b5c0 tx=0x7fefcc005960 comp rx=0 tx=0).stop 2024-06-10T00:57:35.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.229+0000 7fefe1bc5700 1 -- 172.21.15.37:0/724019268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefdc107400 msgr2=0x7fefdc198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:35.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.229+0000 7fefe1bc5700 1 --2- 172.21.15.37:0/724019268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefdc107400 0x7fefdc198ac0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fefd4007ee0 tx=0x7fefd400d3b0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.229+0000 7fefe1bc5700 1 -- 172.21.15.37:0/724019268 shutdown_connections 2024-06-10T00:57:35.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.229+0000 7fefe1bc5700 1 --2- 172.21.15.37:0/724019268 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fefc40a2cb0 0x7fefc40a5170 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.229+0000 7fefe1bc5700 1 --2- 172.21.15.37:0/724019268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefdc107400 0x7fefdc198ac0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.229+0000 7fefe1bc5700 1 --2- 172.21.15.37:0/724019268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefdc101390 0x7fefdc19e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.229+0000 7fefe1bc5700 1 --2- 172.21.15.37:0/724019268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fefdc101d50 0x7fefdc19ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.229+0000 7fefe1bc5700 1 -- 172.21.15.37:0/724019268 >> 172.21.15.37:0/724019268 conn(0x7fefdc078340 msgr2=0x7fefdc105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.229+0000 7fefe1bc5700 1 -- 172.21.15.37:0/724019268 shutdown_connections 2024-06-10T00:57:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:35.229+0000 7fefe1bc5700 1 -- 172.21.15.37:0/724019268 wait complete. 2024-06-10T00:57:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.807+0000 7faa0c961640 1 -- 172.21.15.37:0/1018448612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa08104db0 msgr2=0x7faa081071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.807+0000 7faa0c961640 1 --2- 172.21.15.37:0/1018448612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa08104db0 0x7faa081071a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc009900 tx=0x7fa9fc02a410 comp rx=0 tx=0).stop 2024-06-10T00:57:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.807+0000 7faa0c961640 1 -- 172.21.15.37:0/1018448612 shutdown_connections 2024-06-10T00:57:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.807+0000 7faa0c961640 1 --2- 172.21.15.37:0/1018448612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa081076e0 0x7faa08109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.807+0000 7faa0c961640 1 --2- 172.21.15.37:0/1018448612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa08104db0 0x7faa081071a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.807+0000 7faa0c961640 1 --2- 172.21.15.37:0/1018448612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa08102480 0x7faa08104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.807+0000 7faa0c961640 1 -- 172.21.15.37:0/1018448612 >> 172.21.15.37:0/1018448612 conn(0x7faa080fbe70 msgr2=0x7faa080fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.807+0000 7faa0c961640 1 -- 172.21.15.37:0/1018448612 shutdown_connections 2024-06-10T00:57:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.807+0000 7faa0c961640 1 -- 172.21.15.37:0/1018448612 wait complete. 2024-06-10T00:57:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.808+0000 7faa0c961640 1 Processor -- start 2024-06-10T00:57:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.808+0000 7faa0c961640 1 -- start start 2024-06-10T00:57:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.808+0000 7faa0c961640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa08102480 0x7faa08197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.809+0000 7faa0c961640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa08104db0 0x7faa081981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.809+0000 7faa0c961640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa081076e0 0x7faa0819f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.809+0000 7faa0c961640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faa0810d6e0 con 0x7faa08102480 2024-06-10T00:57:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.809+0000 7faa0c961640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faa0810d560 con 0x7faa08104db0 2024-06-10T00:57:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.809+0000 7faa0c961640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faa0810d860 con 0x7faa081076e0 2024-06-10T00:57:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.809+0000 7faa0659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa08102480 0x7faa08197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.809+0000 7faa0659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa08102480 0x7faa08197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44458/0 (socket says 172.21.15.37:44458) 2024-06-10T00:57:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.809+0000 7faa0659c640 1 -- 172.21.15.37:0/3932724977 learned_addr learned my addr 172.21.15.37:0/3932724977 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.809+0000 7faa05d9b640 1 --2- 172.21.15.37:0/3932724977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa08104db0 0x7faa081981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.809+0000 7faa06d9d640 1 --2- 172.21.15.37:0/3932724977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa081076e0 0x7faa0819f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.809+0000 7faa0659c640 1 -- 172.21.15.37:0/3932724977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa081076e0 msgr2=0x7faa0819f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.809+0000 7faa0659c640 1 --2- 172.21.15.37:0/3932724977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa081076e0 0x7faa0819f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.809+0000 7faa0659c640 1 -- 172.21.15.37:0/3932724977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa08104db0 msgr2=0x7faa081981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.809+0000 7faa0659c640 1 --2- 172.21.15.37:0/3932724977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa08104db0 0x7faa081981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.809+0000 7faa0659c640 1 -- 172.21.15.37:0/3932724977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0819f8e0 con 0x7faa08102480 2024-06-10T00:57:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.809+0000 7faa0659c640 1 --2- 172.21.15.37:0/3932724977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa08102480 0x7faa08197ca0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009ae0 tx=0x7fa9f800ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.810+0000 7fa9f77fe640 1 -- 172.21.15.37:0/3932724977 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8007920 con 0x7faa08102480 2024-06-10T00:57:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.810+0000 7fa9f77fe640 1 -- 172.21.15.37:0/3932724977 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9f8005020 con 0x7faa08102480 2024-06-10T00:57:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.810+0000 7faa0c961640 1 -- 172.21.15.37:0/3932724977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa081002d0 con 0x7faa08102480 2024-06-10T00:57:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.810+0000 7fa9f77fe640 1 -- 172.21.15.37:0/3932724977 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8004e00 con 0x7faa08102480 2024-06-10T00:57:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.810+0000 7faa0c961640 1 -- 172.21.15.37:0/3932724977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa081007f0 con 0x7faa08102480 2024-06-10T00:57:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.812+0000 7fa9f77fe640 1 -- 172.21.15.37:0/3932724977 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fa9f80051d0 con 0x7faa08102480 2024-06-10T00:57:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.813+0000 7faa0f66e740 1 Processor -- start 2024-06-10T00:57:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.813+0000 7faa0f66e740 1 -- start start 2024-06-10T00:57:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.814+0000 7fa9f77fe640 1 --2- 172.21.15.37:0/3932724977 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa9f00a2d50 0x7fa9f00a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.814+0000 7faa0f66e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fbc149010 0x556fbc149430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.814+0000 7faa0f66e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fbc149a00 0x556fbc14e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.814+0000 7faa0f66e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fbc14eb70 0x556fbc151000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.814+0000 7faa0f66e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556fbc0ac4f0 con 0x556fbc149a00 2024-06-10T00:57:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.814+0000 7faa0f66e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556fbc04a870 con 0x556fbc149010 2024-06-10T00:57:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.814+0000 7faa0f66e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556fbc0a84b0 con 0x556fbc14eb70 2024-06-10T00:57:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.814+0000 7fa9f77fe640 1 -- 172.21.15.37:0/3932724977 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fa9f80d2b20 con 0x7faa08102480 2024-06-10T00:57:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.814+0000 7faa05d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fbc149a00 0x556fbc14e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.814+0000 7faa05d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fbc149a00 0x556fbc14e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44474/0 (socket says 172.21.15.37:44474) 2024-06-10T00:57:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.814+0000 7faa05d9b640 1 -- 172.21.15.37:0/3986973253 learned_addr learned my addr 172.21.15.37:0/3986973253 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.814+0000 7faa06d9d640 1 --2- 172.21.15.37:0/3932724977 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa9f00a2d50 0x7fa9f00a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.814+0000 7faa0659c640 1 --2- 172.21.15.37:0/3986973253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fbc149010 0x556fbc149430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.814+0000 7faa05d9b640 1 -- 172.21.15.37:0/3986973253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fbc14eb70 msgr2=0x556fbc151000 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T00:57:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.814+0000 7faa05d9b640 1 --2- 172.21.15.37:0/3986973253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fbc14eb70 0x556fbc151000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.814+0000 7faa05d9b640 1 -- 172.21.15.37:0/3986973253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fbc149010 msgr2=0x556fbc149430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.814+0000 7faa05d9b640 1 --2- 172.21.15.37:0/3986973253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fbc149010 0x556fbc149430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.814+0000 7faa05d9b640 1 -- 172.21.15.37:0/3986973253 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fbc0669b0 con 0x556fbc149a00 2024-06-10T00:57:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.814+0000 7faa0659c640 1 --2- 172.21.15.37:0/3986973253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fbc149010 0x556fbc149430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:57:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.814+0000 7faa05d9b640 1 --2- 172.21.15.37:0/3986973253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fbc149a00 0x556fbc14e500 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc006010 tx=0x7fa9fc004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e97fbb3bdbc4f7a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.814+0000 7faa06d9d640 1 --2- 172.21.15.37:0/3932724977 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa9f00a2d50 0x7fa9f00a5210 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7faa00005fd0 tx=0x7faa00009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.815+0000 7fa9f4ff9640 1 -- 172.21.15.37:0/3986973253 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc035040 con 0x556fbc149a00 2024-06-10T00:57:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.815+0000 7fa9f4ff9640 1 -- 172.21.15.37:0/3986973253 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9fc005520 con 0x556fbc149a00 2024-06-10T00:57:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.815+0000 7fa9f4ff9640 1 -- 172.21.15.37:0/3986973253 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc031910 con 0x556fbc149a00 2024-06-10T00:57:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.815+0000 7faa0f66e740 1 -- 172.21.15.37:0/3986973253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fbc149a00 msgr2=0x556fbc14e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.815+0000 7faa0f66e740 1 --2- 172.21.15.37:0/3986973253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fbc149a00 0x556fbc14e500 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc006010 tx=0x7fa9fc004170 comp rx=0 tx=0).stop 2024-06-10T00:57:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.816+0000 7faa0f66e740 1 -- 172.21.15.37:0/3986973253 shutdown_connections 2024-06-10T00:57:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.816+0000 7faa0f66e740 1 --2- 172.21.15.37:0/3986973253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fbc14eb70 0x556fbc151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.816+0000 7faa0f66e740 1 --2- 172.21.15.37:0/3986973253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fbc149a00 0x556fbc14e500 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.816+0000 7faa0f66e740 1 --2- 172.21.15.37:0/3986973253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fbc149010 0x556fbc149430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.816+0000 7faa0f66e740 1 -- 172.21.15.37:0/3986973253 >> 172.21.15.37:0/3986973253 conn(0x556fbc144680 msgr2=0x556fbc144a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.816+0000 7faa0f66e740 1 -- 172.21.15.37:0/3986973253 shutdown_connections 2024-06-10T00:57:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.816+0000 7faa0f66e740 1 -- 172.21.15.37:0/3986973253 wait complete. 2024-06-10T00:57:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.816+0000 7faa0f66e740 1 -- start start 2024-06-10T00:57:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.816+0000 7faa0f66e740 1 Processor -- start 2024-06-10T00:57:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.816+0000 7faa0f66e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fbc149010 0x556fbc153200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.817+0000 7faa0f66e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fbc149a00 0x556fbc153740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.817+0000 7faa0659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fbc149010 0x556fbc153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.817+0000 7faa0f66e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fbc14eb70 0x556fbc15a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.817+0000 7faa0659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fbc149010 0x556fbc153200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44480/0 (socket says 172.21.15.37:44480) 2024-06-10T00:57:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.817+0000 7faa0659c640 1 -- 172.21.15.37:0/320466323 learned_addr learned my addr 172.21.15.37:0/320466323 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.817+0000 7faa0f66e740 1 -- 172.21.15.37:0/320466323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556fbc151ac0 con 0x556fbc149010 2024-06-10T00:57:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.817+0000 7faa0f66e740 1 -- 172.21.15.37:0/320466323 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556fbc035ae0 con 0x556fbc14eb70 2024-06-10T00:57:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.817+0000 7faa0f66e740 1 -- 172.21.15.37:0/320466323 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556fbc151c40 con 0x556fbc149a00 2024-06-10T00:57:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.817+0000 7faa05d9b640 1 --2- 172.21.15.37:0/320466323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fbc149a00 0x556fbc153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.817+0000 7faa0f66e740 10 client.0 _ll_register_callbacks cb 0x556fbc06e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.817+0000 7faa0659c640 1 -- 172.21.15.37:0/320466323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fbc149a00 msgr2=0x556fbc153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.817+0000 7faa0659c640 1 --2- 172.21.15.37:0/320466323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fbc149a00 0x556fbc153740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.817+0000 7faa0659c640 1 -- 172.21.15.37:0/320466323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fbc14eb70 msgr2=0x556fbc15a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.817+0000 7faa0659c640 1 --2- 172.21.15.37:0/320466323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fbc14eb70 0x556fbc15a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.817+0000 7faa0659c640 1 -- 172.21.15.37:0/320466323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fbbdc0210 con 0x556fbc149010 2024-06-10T00:57:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.817+0000 7faa0659c640 1 --2- 172.21.15.37:0/320466323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fbc149010 0x556fbc153200 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa9f809b8b0 tx=0x7fa9f8004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb08f43425abf3c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.817+0000 7fa9edffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:57:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.817+0000 7fa9edffb640 1 -- 172.21.15.37:0/320466323 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f809eee0 con 0x556fbc149010 2024-06-10T00:57:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.818+0000 7fa9edffb640 1 -- 172.21.15.37:0/320466323 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fa9f80d0330 con 0x556fbc149010 2024-06-10T00:57:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.818+0000 7faa0f66e740 1 -- 172.21.15.37:0/320466323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9fc005520 con 0x556fbc149010 2024-06-10T00:57:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.818+0000 7fa9edffb640 1 -- 172.21.15.37:0/320466323 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa9f80a31e0 con 0x556fbc149010 2024-06-10T00:57:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.818+0000 7faa0f66e740 10 client.15573 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.818+0000 7faa0f66e740 1 -- 172.21.15.37:0/320466323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556fbbe8c620 con 0x556fbc149010 2024-06-10T00:57:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.818+0000 7fa9edffb640 1 -- 172.21.15.37:0/320466323 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9fc005520 con 0x556fbc149010 2024-06-10T00:57:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.818+0000 7fa9edffb640 1 -- 172.21.15.37:0/320466323 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa9f809e330 con 0x556fbc149010 2024-06-10T00:57:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.818+0000 7faa0f66e740 10 client.15573 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.818+0000 7faa0f66e740 10 client.15573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T00:57:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.818+0000 7faa0f66e740 20 client.15573 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.818+0000 7faa0f66e740 1 --2- 172.21.15.37:0/320466323 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x556fbc15f4b0 0x556fbc161940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.818+0000 7faa0f66e740 4 client.15573 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:57:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.818+0000 7faa0f66e740 1 -- 172.21.15.37:0/320466323 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fbc0bff10 con 0x556fbc15f4b0 2024-06-10T00:57:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.819+0000 7faa05d9b640 1 --2- 172.21.15.37:0/320466323 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x556fbc15f4b0 0x556fbc161940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.819+0000 7faa05d9b640 1 --2- 172.21.15.37:0/320466323 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x556fbc15f4b0 0x556fbc161940 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18317d68517eab5 server_cookie=f784283bacce255f in_seq=0 out_seq=0 2024-06-10T00:57:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.819+0000 7fa9edffb640 10 client.15573 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.820+0000 7fa9edffb640 1 -- 172.21.15.37:0/320466323 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556fbc0bff10 con 0x556fbc15f4b0 2024-06-10T00:57:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.820+0000 7fa9edffb640 10 client.15573 handle_command_reply: tid=0 2024-06-10T00:57:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.820+0000 7faa0f66e740 1 client.15573 shutdown 2024-06-10T00:57:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.820+0000 7faa0f66e740 1 -- 172.21.15.37:0/320466323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fbc149010 msgr2=0x556fbc153200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.820+0000 7faa0f66e740 1 --2- 172.21.15.37:0/320466323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fbc149010 0x556fbc153200 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa9f809b8b0 tx=0x7fa9f8004800 comp rx=0 tx=0).stop 2024-06-10T00:57:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.820+0000 7faa0f66e740 1 -- 172.21.15.37:0/320466323 shutdown_connections 2024-06-10T00:57:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.820+0000 7faa0f66e740 1 --2- 172.21.15.37:0/320466323 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x556fbc15f4b0 0x556fbc161940 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.820+0000 7faa0f66e740 1 --2- 172.21.15.37:0/320466323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fbc14eb70 0x556fbc15a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.820+0000 7faa0f66e740 1 --2- 172.21.15.37:0/320466323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fbc149a00 0x556fbc153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.820+0000 7faa0f66e740 1 --2- 172.21.15.37:0/320466323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fbc149010 0x556fbc153200 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.820+0000 7faa0f66e740 1 -- 172.21.15.37:0/320466323 >> 172.21.15.37:0/320466323 conn(0x556fbc144680 msgr2=0x556fbc1437e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.820+0000 7fa9edffb640 0 client.15573 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.820+0000 7faa0f66e740 1 -- 172.21.15.37:0/320466323 shutdown_connections 2024-06-10T00:57:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.820+0000 7faa0f66e740 1 -- 172.21.15.37:0/320466323 wait complete. 2024-06-10T00:57:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.820+0000 7faa0f66e740 20 client.15573 trim_cache size 0 max 16384 2024-06-10T00:57:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.836+0000 7faa0f66e740 1 Processor -- start 2024-06-10T00:57:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.836+0000 7faa0f66e740 1 -- start start 2024-06-10T00:57:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.836+0000 7faa0f66e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fbc149a00 0x556fbc14ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.836+0000 7faa0f66e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fbc14eb70 0x556fbc14b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.836+0000 7faa0f66e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fbc15a740 0x556fbc147590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.836+0000 7faa0f66e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faa081aad80 con 0x556fbc14eb70 2024-06-10T00:57:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.836+0000 7faa0f66e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faa081a8650 con 0x556fbc15a740 2024-06-10T00:57:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.836+0000 7faa0f66e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faa081a64a0 con 0x556fbc149a00 2024-06-10T00:57:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.836+0000 7faa05d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fbc14eb70 0x556fbc14b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.836+0000 7faa05d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fbc14eb70 0x556fbc14b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44492/0 (socket says 172.21.15.37:44492) 2024-06-10T00:57:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.836+0000 7faa05d9b640 1 -- 172.21.15.37:0/3719145203 learned_addr learned my addr 172.21.15.37:0/3719145203 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.836+0000 7faa0659c640 1 --2- 172.21.15.37:0/3719145203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fbc149a00 0x556fbc14ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.836+0000 7faa06d9d640 1 --2- 172.21.15.37:0/3719145203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fbc15a740 0x556fbc147590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.836+0000 7faa05d9b640 1 -- 172.21.15.37:0/3719145203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fbc149a00 msgr2=0x556fbc14ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.836+0000 7faa05d9b640 1 --2- 172.21.15.37:0/3719145203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fbc149a00 0x556fbc14ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.836+0000 7faa05d9b640 1 -- 172.21.15.37:0/3719145203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fbc15a740 msgr2=0x556fbc147590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.837+0000 7faa05d9b640 1 --2- 172.21.15.37:0/3719145203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fbc15a740 0x556fbc147590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.837+0000 7faa05d9b640 1 -- 172.21.15.37:0/3719145203 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fbc1444b0 con 0x556fbc14eb70 2024-06-10T00:57:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.837+0000 7faa06d9d640 1 --2- 172.21.15.37:0/3719145203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fbc15a740 0x556fbc147590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:57:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.837+0000 7faa05d9b640 1 --2- 172.21.15.37:0/3719145203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fbc14eb70 0x556fbc14b180 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc009900 tx=0x7fa9fc031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f357d4fc4b26e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.837+0000 7fa9effff640 1 -- 172.21.15.37:0/3719145203 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc035040 con 0x556fbc14eb70 2024-06-10T00:57:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.837+0000 7fa9effff640 1 -- 172.21.15.37:0/3719145203 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9fc004460 con 0x556fbc14eb70 2024-06-10T00:57:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.837+0000 7fa9effff640 1 -- 172.21.15.37:0/3719145203 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc003ac0 con 0x556fbc14eb70 2024-06-10T00:57:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.838+0000 7faa0f66e740 1 -- 172.21.15.37:0/3719145203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fbc14eb70 msgr2=0x556fbc14b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.838+0000 7faa0f66e740 1 --2- 172.21.15.37:0/3719145203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fbc14eb70 0x556fbc14b180 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc009900 tx=0x7fa9fc031400 comp rx=0 tx=0).stop 2024-06-10T00:57:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.838+0000 7faa0f66e740 1 -- 172.21.15.37:0/3719145203 shutdown_connections 2024-06-10T00:57:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.838+0000 7faa0f66e740 1 --2- 172.21.15.37:0/3719145203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fbc15a740 0x556fbc147590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.838+0000 7faa0f66e740 1 --2- 172.21.15.37:0/3719145203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fbc14eb70 0x556fbc14b180 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.838+0000 7faa0f66e740 1 --2- 172.21.15.37:0/3719145203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fbc149a00 0x556fbc14ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.838+0000 7faa0f66e740 1 -- 172.21.15.37:0/3719145203 >> 172.21.15.37:0/3719145203 conn(0x556fbc144680 msgr2=0x556fbc14e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.838+0000 7faa0f66e740 1 -- 172.21.15.37:0/3719145203 shutdown_connections 2024-06-10T00:57:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.838+0000 7faa0f66e740 1 -- 172.21.15.37:0/3719145203 wait complete. 2024-06-10T00:57:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.838+0000 7faa0f66e740 1 -- start start 2024-06-10T00:57:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.839+0000 7faa0f66e740 1 Processor -- start 2024-06-10T00:57:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.839+0000 7faa0f66e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fbc149a00 0x556fbc159d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.839+0000 7faa0f66e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fbc14eb70 0x556fbc156810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.839+0000 7faa0f66e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fbc15a740 0x556fbc156d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.839+0000 7faa0f66e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faa081a6b10 con 0x556fbc14eb70 2024-06-10T00:57:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.839+0000 7faa0f66e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faa081a7f80 con 0x556fbc15a740 2024-06-10T00:57:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.839+0000 7faa0f66e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faa081a71e0 con 0x556fbc149a00 2024-06-10T00:57:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.839+0000 7faa05d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fbc14eb70 0x556fbc156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.839+0000 7faa0f66e740 10 client.0 _ll_register_callbacks cb 0x7fa9f809ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.839+0000 7faa0659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fbc149a00 0x556fbc159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.839+0000 7faa05d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fbc14eb70 0x556fbc156810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44508/0 (socket says 172.21.15.37:44508) 2024-06-10T00:57:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.839+0000 7faa05d9b640 1 -- 172.21.15.37:0/4062430498 learned_addr learned my addr 172.21.15.37:0/4062430498 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.839+0000 7faa06d9d640 1 --2- 172.21.15.37:0/4062430498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fbc15a740 0x556fbc156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.839+0000 7faa05d9b640 1 -- 172.21.15.37:0/4062430498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fbc149a00 msgr2=0x556fbc159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.839+0000 7faa05d9b640 1 --2- 172.21.15.37:0/4062430498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fbc149a00 0x556fbc159d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.839+0000 7faa05d9b640 1 -- 172.21.15.37:0/4062430498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fbc15a740 msgr2=0x556fbc156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.839+0000 7faa05d9b640 1 --2- 172.21.15.37:0/4062430498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fbc15a740 0x556fbc156d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.839+0000 7faa05d9b640 1 -- 172.21.15.37:0/4062430498 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fbc15a3a0 con 0x556fbc14eb70 2024-06-10T00:57:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.840+0000 7faa05d9b640 1 --2- 172.21.15.37:0/4062430498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fbc14eb70 0x556fbc156810 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc02aa00 tx=0x7fa9fc004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a69271ff16c1296 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.840+0000 7fa9edffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:57:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.840+0000 7fa9edffb640 1 -- 172.21.15.37:0/4062430498 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc035030 con 0x556fbc14eb70 2024-06-10T00:57:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.840+0000 7fa9edffb640 1 -- 172.21.15.37:0/4062430498 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fa9fc0316b0 con 0x556fbc14eb70 2024-06-10T00:57:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.840+0000 7faa0f66e740 1 -- 172.21.15.37:0/4062430498 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9fc004460 con 0x556fbc14eb70 2024-06-10T00:57:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.840+0000 7fa9edffb640 1 -- 172.21.15.37:0/4062430498 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa9fc06c050 con 0x556fbc14eb70 2024-06-10T00:57:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.840+0000 7faa0f66e740 10 client.15579 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.840+0000 7faa0f66e740 1 -- 172.21.15.37:0/4062430498 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556fbc1c90e0 con 0x556fbc14eb70 2024-06-10T00:57:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.840+0000 7fa9edffb640 1 -- 172.21.15.37:0/4062430498 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9fc004460 con 0x556fbc14eb70 2024-06-10T00:57:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.840+0000 7fa9edffb640 1 -- 172.21.15.37:0/4062430498 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa9fc03cab0 con 0x556fbc14eb70 2024-06-10T00:57:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.840+0000 7faa0f66e740 10 client.15579 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.840+0000 7faa0f66e740 10 client.15579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T00:57:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.841+0000 7faa0f66e740 20 client.15579 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.841+0000 7faa0f66e740 1 --2- 172.21.15.37:0/4062430498 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x556fbc1c9650 0x556fbc1cba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.841+0000 7faa0f66e740 4 client.15579 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T00:57:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.841+0000 7faa0f66e740 1 -- 172.21.15.37:0/4062430498 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556fbc0b9c20 con 0x556fbc1c9650 2024-06-10T00:57:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.841+0000 7faa0659c640 1 --2- 172.21.15.37:0/4062430498 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x556fbc1c9650 0x556fbc1cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.841+0000 7faa0659c640 1 --2- 172.21.15.37:0/4062430498 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x556fbc1c9650 0x556fbc1cba40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efb291e42a499fe9 server_cookie=6beaece369229b34 in_seq=0 out_seq=0 2024-06-10T00:57:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.841+0000 7fa9edffb640 10 client.15579 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:35.849 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-10T00:57:35.850 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-06-10T00:57:35.850 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:57:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.845+0000 7fa9edffb640 1 -- 172.21.15.37:0/4062430498 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556fbc0b9c20 con 0x556fbc1c9650 2024-06-10T00:57:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.845+0000 7fa9edffb640 10 client.15579 handle_command_reply: tid=0 2024-06-10T00:57:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.845+0000 7faa0f66e740 1 client.15579 shutdown 2024-06-10T00:57:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.845+0000 7faa0f66e740 1 -- 172.21.15.37:0/4062430498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fbc14eb70 msgr2=0x556fbc156810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.845+0000 7faa0f66e740 1 --2- 172.21.15.37:0/4062430498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fbc14eb70 0x556fbc156810 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc02aa00 tx=0x7fa9fc004170 comp rx=0 tx=0).stop 2024-06-10T00:57:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.845+0000 7faa0f66e740 1 -- 172.21.15.37:0/4062430498 shutdown_connections 2024-06-10T00:57:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.845+0000 7faa0f66e740 1 --2- 172.21.15.37:0/4062430498 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x556fbc1c9650 0x556fbc1cba40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.845+0000 7faa0f66e740 1 --2- 172.21.15.37:0/4062430498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fbc15a740 0x556fbc156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.845+0000 7faa0f66e740 1 --2- 172.21.15.37:0/4062430498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fbc14eb70 0x556fbc156810 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.845+0000 7faa0f66e740 1 --2- 172.21.15.37:0/4062430498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556fbc149a00 0x556fbc159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.845+0000 7faa0f66e740 1 -- 172.21.15.37:0/4062430498 >> 172.21.15.37:0/4062430498 conn(0x556fbc144680 msgr2=0x556fbc14de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.845+0000 7fa9edffb640 0 client.15579 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.845+0000 7faa0f66e740 1 -- 172.21.15.37:0/4062430498 shutdown_connections 2024-06-10T00:57:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.845+0000 7faa0f66e740 1 -- 172.21.15.37:0/4062430498 wait complete. 2024-06-10T00:57:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.845+0000 7faa0f66e740 20 client.15579 trim_cache size 0 max 16384 2024-06-10T00:57:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T00:57:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T00:57:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T00:57:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.846+0000 7faa0c961640 1 -- 172.21.15.37:0/3932724977 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa9f00a2d50 msgr2=0x7fa9f00a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.846+0000 7faa0c961640 1 --2- 172.21.15.37:0/3932724977 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa9f00a2d50 0x7fa9f00a5210 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7faa00005fd0 tx=0x7faa00009500 comp rx=0 tx=0).stop 2024-06-10T00:57:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.846+0000 7faa0c961640 1 -- 172.21.15.37:0/3932724977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa08102480 msgr2=0x7faa08197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.846+0000 7faa0c961640 1 --2- 172.21.15.37:0/3932724977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa08102480 0x7faa08197ca0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009ae0 tx=0x7fa9f800ff80 comp rx=0 tx=0).stop 2024-06-10T00:57:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.846+0000 7faa0c961640 1 -- 172.21.15.37:0/3932724977 shutdown_connections 2024-06-10T00:57:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.846+0000 7faa0c961640 1 --2- 172.21.15.37:0/3932724977 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa9f00a2d50 0x7fa9f00a5210 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.846+0000 7faa0c961640 1 --2- 172.21.15.37:0/3932724977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa081076e0 0x7faa0819f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.846+0000 7faa0c961640 1 --2- 172.21.15.37:0/3932724977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa08104db0 0x7faa081981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.846+0000 7faa0c961640 1 --2- 172.21.15.37:0/3932724977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa08102480 0x7faa08197ca0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.846+0000 7faa0c961640 1 -- 172.21.15.37:0/3932724977 >> 172.21.15.37:0/3932724977 conn(0x7faa080fbe70 msgr2=0x7faa08102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.846+0000 7faa0c961640 1 -- 172.21.15.37:0/3932724977 shutdown_connections 2024-06-10T00:57:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:35.846+0000 7faa0c961640 1 -- 172.21.15.37:0/3932724977 wait complete. 2024-06-10T00:57:36.175 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:57:36.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:36 smithi037 ceph-mon[32192]: pgmap v457: 129 pgs: 129 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T00:57:36.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:36 smithi167 ceph-mon[36917]: pgmap v457: 129 pgs: 129 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T00:57:36.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:36 smithi094 ceph-mon[36708]: pgmap v457: 129 pgs: 129 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T00:57:36.916 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T00:57:36.917 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T00:57:37.580 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:57:38.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.127+0000 7fe893391700 1 -- 172.21.15.37:0/3802058667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe88c107400 msgr2=0x7fe88c1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.127+0000 7fe893391700 1 --2- 172.21.15.37:0/3802058667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe88c107400 0x7fe88c1077e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe888009990 tx=0x7fe888009ca0 comp rx=0 tx=0).stop 2024-06-10T00:57:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.128+0000 7fe893391700 1 -- 172.21.15.37:0/3802058667 shutdown_connections 2024-06-10T00:57:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.128+0000 7fe893391700 1 --2- 172.21.15.37:0/3802058667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe88c101d50 0x7fe88c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.128+0000 7fe893391700 1 --2- 172.21.15.37:0/3802058667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe88c107400 0x7fe88c1077e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.128+0000 7fe893391700 1 --2- 172.21.15.37:0/3802058667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe88c101390 0x7fe88c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.128+0000 7fe893391700 1 -- 172.21.15.37:0/3802058667 >> 172.21.15.37:0/3802058667 conn(0x7fe88c078340 msgr2=0x7fe88c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.128+0000 7fe893391700 1 -- 172.21.15.37:0/3802058667 shutdown_connections 2024-06-10T00:57:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.128+0000 7fe893391700 1 -- 172.21.15.37:0/3802058667 wait complete. 2024-06-10T00:57:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.128+0000 7fe893391700 1 Processor -- start 2024-06-10T00:57:38.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.128+0000 7fe893391700 1 -- start start 2024-06-10T00:57:38.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.129+0000 7fe893391700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe88c101390 0x7fe88c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:38.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.129+0000 7fe893391700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe88c101d50 0x7fe88c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:38.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.129+0000 7fe893391700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe88c107400 0x7fe88c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:38.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.129+0000 7fe893391700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c112790 con 0x7fe88c107400 2024-06-10T00:57:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.129+0000 7fe893391700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c112610 con 0x7fe88c101390 2024-06-10T00:57:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.129+0000 7fe893391700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c112910 con 0x7fe88c101d50 2024-06-10T00:57:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.129+0000 7fe89192e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe88c107400 0x7fe88c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.129+0000 7fe89192e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe88c107400 0x7fe88c198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44528/0 (socket says 172.21.15.37:44528) 2024-06-10T00:57:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.129+0000 7fe89192e700 1 -- 172.21.15.37:0/33693331 learned_addr learned my addr 172.21.15.37:0/33693331 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.129+0000 7fe89112d700 1 --2- 172.21.15.37:0/33693331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe88c101390 0x7fe88c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.129+0000 7fe89092c700 1 --2- 172.21.15.37:0/33693331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe88c101d50 0x7fe88c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.129+0000 7fe89192e700 1 -- 172.21.15.37:0/33693331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe88c101d50 msgr2=0x7fe88c19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.129+0000 7fe89192e700 1 --2- 172.21.15.37:0/33693331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe88c101d50 0x7fe88c19eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.129+0000 7fe89192e700 1 -- 172.21.15.37:0/33693331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe88c101390 msgr2=0x7fe88c19e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.129+0000 7fe89192e700 1 --2- 172.21.15.37:0/33693331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe88c101390 0x7fe88c19e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.129+0000 7fe89192e700 1 -- 172.21.15.37:0/33693331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88c1991d0 con 0x7fe88c107400 2024-06-10T00:57:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.130+0000 7fe89192e700 1 --2- 172.21.15.37:0/33693331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe88c107400 0x7fe88c198a70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe884007ee0 tx=0x7fe88400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.130+0000 7fe8827fc700 1 -- 172.21.15.37:0/33693331 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe884004bb0 con 0x7fe88c107400 2024-06-10T00:57:38.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.130+0000 7fe8827fc700 1 -- 172.21.15.37:0/33693331 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe884004d50 con 0x7fe88c107400 2024-06-10T00:57:38.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.130+0000 7fe893391700 1 -- 172.21.15.37:0/33693331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88c1994c0 con 0x7fe88c107400 2024-06-10T00:57:38.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.130+0000 7fe8827fc700 1 -- 172.21.15.37:0/33693331 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe884005960 con 0x7fe88c107400 2024-06-10T00:57:38.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.130+0000 7fe893391700 1 -- 172.21.15.37:0/33693331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe88c10ec00 con 0x7fe88c107400 2024-06-10T00:57:38.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.131+0000 7fe893391700 1 -- 172.21.15.37:0/33693331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe88c069180 con 0x7fe88c107400 2024-06-10T00:57:38.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.133+0000 7fe8827fc700 1 -- 172.21.15.37:0/33693331 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fe88401a020 con 0x7fe88c107400 2024-06-10T00:57:38.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.134+0000 7fe8827fc700 1 --2- 172.21.15.37:0/33693331 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe8780a2d30 0x7fe8780a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:38.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.135+0000 7fe8827fc700 1 -- 172.21.15.37:0/33693331 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fe8840cec20 con 0x7fe88c107400 2024-06-10T00:57:38.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.135+0000 7fe89112d700 1 --2- 172.21.15.37:0/33693331 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe8780a2d30 0x7fe8780a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:38.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.135+0000 7fe89112d700 1 --2- 172.21.15.37:0/33693331 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe8780a2d30 0x7fe8780a51f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0ffa10 tx=0x7fe888005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:38.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.142+0000 7fe8827fc700 1 -- 172.21.15.37:0/33693331 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe884010070 con 0x7fe88c107400 2024-06-10T00:57:38.267 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:38 smithi037 ceph-mon[32192]: pgmap v458: 129 pgs: 129 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-10T00:57:38.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.486+0000 7fe893391700 1 -- 172.21.15.37:0/33693331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe88c02dca0 con 0x7fe88c107400 2024-06-10T00:57:38.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.487+0000 7fe8827fc700 1 -- 172.21.15.37:0/33693331 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11867 (secure 0 0 0) 0x7fe88409b630 con 0x7fe88c107400 2024-06-10T00:57:38.487 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:57:38.487 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T00:56:30.291171+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24451,24458,24472]},"id":1}]} 2024-06-10T00:57:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.489+0000 7fe893391700 1 -- 172.21.15.37:0/33693331 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe8780a2d30 msgr2=0x7fe8780a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.489+0000 7fe893391700 1 --2- 172.21.15.37:0/33693331 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe8780a2d30 0x7fe8780a51f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0ffa10 tx=0x7fe888005960 comp rx=0 tx=0).stop 2024-06-10T00:57:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.489+0000 7fe893391700 1 -- 172.21.15.37:0/33693331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe88c107400 msgr2=0x7fe88c198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.489+0000 7fe893391700 1 --2- 172.21.15.37:0/33693331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe88c107400 0x7fe88c198a70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe884007ee0 tx=0x7fe88400d3b0 comp rx=0 tx=0).stop 2024-06-10T00:57:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.489+0000 7fe893391700 1 -- 172.21.15.37:0/33693331 shutdown_connections 2024-06-10T00:57:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.489+0000 7fe893391700 1 --2- 172.21.15.37:0/33693331 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe8780a2d30 0x7fe8780a51f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.489+0000 7fe893391700 1 --2- 172.21.15.37:0/33693331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe88c107400 0x7fe88c198a70 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.489+0000 7fe893391700 1 --2- 172.21.15.37:0/33693331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe88c101390 0x7fe88c19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.489+0000 7fe893391700 1 --2- 172.21.15.37:0/33693331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe88c101d50 0x7fe88c19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.489+0000 7fe893391700 1 -- 172.21.15.37:0/33693331 >> 172.21.15.37:0/33693331 conn(0x7fe88c078340 msgr2=0x7fe88c1000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.489+0000 7fe893391700 1 -- 172.21.15.37:0/33693331 shutdown_connections 2024-06-10T00:57:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:38.489+0000 7fe893391700 1 -- 172.21.15.37:0/33693331 wait complete. 2024-06-10T00:57:38.491 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-06-10T00:57:38.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:38 smithi167 ceph-mon[36917]: pgmap v458: 129 pgs: 129 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-10T00:57:38.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:38 smithi094 ceph-mon[36708]: pgmap v458: 129 pgs: 129 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-10T00:57:39.174 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24472 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-06-10T00:57:39.174 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24472 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-06-10T00:57:39.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:39 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/33693331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:57:39.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:39 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/33693331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:57:39.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:39 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/33693331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:57:40.215 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:57:40.501 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:40 smithi037 ceph-mon[32192]: pgmap v459: 129 pgs: 129 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T00:57:40.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:40 smithi167 ceph-mon[36917]: pgmap v459: 129 pgs: 129 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T00:57:40.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:40 smithi094 ceph-mon[36708]: pgmap v459: 129 pgs: 129 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T00:57:41.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.462+0000 7f4323809700 1 -- 172.21.15.37:0/3597633284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f431c101380 msgr2=0x7f431c101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:41.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.462+0000 7f4323809700 1 --2- 172.21.15.37:0/3597633284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f431c101380 0x7f431c101800 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f430c009990 tx=0x7f430c009ca0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.462+0000 7f4323809700 1 -- 172.21.15.37:0/3597633284 shutdown_connections 2024-06-10T00:57:41.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.462+0000 7f4323809700 1 --2- 172.21.15.37:0/3597633284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f431c101d40 0x7f431c10e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.462+0000 7f4323809700 1 --2- 172.21.15.37:0/3597633284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f431c1073f0 0x7f431c1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.462+0000 7f4323809700 1 --2- 172.21.15.37:0/3597633284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f431c101380 0x7f431c101800 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.462+0000 7f4323809700 1 -- 172.21.15.37:0/3597633284 >> 172.21.15.37:0/3597633284 conn(0x7f431c078300 msgr2=0x7f431c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:41.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.463+0000 7f4323809700 1 -- 172.21.15.37:0/3597633284 shutdown_connections 2024-06-10T00:57:41.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.463+0000 7f4323809700 1 -- 172.21.15.37:0/3597633284 wait complete. 2024-06-10T00:57:41.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.463+0000 7f4323809700 1 Processor -- start 2024-06-10T00:57:41.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.463+0000 7f4323809700 1 -- start start 2024-06-10T00:57:41.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.464+0000 7f4323809700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f431c101380 0x7f431c19e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:41.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.464+0000 7f4323809700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f431c101d40 0x7f431c19ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:41.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.464+0000 7f4323809700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f431c1073f0 0x7f431c198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:41.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.464+0000 7f4323809700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1127e0 con 0x7f431c101380 2024-06-10T00:57:41.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.464+0000 7f4323809700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f431c112660 con 0x7f431c1073f0 2024-06-10T00:57:41.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.464+0000 7f43215a5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f431c101380 0x7f431c19e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:41.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.464+0000 7f4323809700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f431c112960 con 0x7f431c101d40 2024-06-10T00:57:41.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.464+0000 7f43215a5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f431c101380 0x7f431c19e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44560/0 (socket says 172.21.15.37:44560) 2024-06-10T00:57:41.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.464+0000 7f43215a5700 1 -- 172.21.15.37:0/2545165056 learned_addr learned my addr 172.21.15.37:0/2545165056 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:41.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.464+0000 7f4320da4700 1 --2- 172.21.15.37:0/2545165056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f431c101d40 0x7f431c19ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:41.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.464+0000 7f43215a5700 1 -- 172.21.15.37:0/2545165056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f431c101d40 msgr2=0x7f431c19ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:41.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.464+0000 7f43215a5700 1 --2- 172.21.15.37:0/2545165056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f431c101d40 0x7f431c19ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.464+0000 7f43215a5700 1 -- 172.21.15.37:0/2545165056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f431c1073f0 msgr2=0x7f431c198880 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-10T00:57:41.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.464+0000 7f43215a5700 1 --2- 172.21.15.37:0/2545165056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f431c1073f0 0x7f431c198880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.464+0000 7f43215a5700 1 -- 172.21.15.37:0/2545165056 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431c198fe0 con 0x7f431c101380 2024-06-10T00:57:41.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.465+0000 7f43215a5700 1 --2- 172.21.15.37:0/2545165056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f431c101380 0x7f431c19e6c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f431800d950 tx=0x7f431800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:41.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.465+0000 7f43127fc700 1 -- 172.21.15.37:0/2545165056 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4318016070 con 0x7f431c101380 2024-06-10T00:57:41.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.465+0000 7f43127fc700 1 -- 172.21.15.37:0/2545165056 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4318009e20 con 0x7f431c101380 2024-06-10T00:57:41.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.465+0000 7f4323809700 1 -- 172.21.15.37:0/2545165056 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f431c1992d0 con 0x7f431c101380 2024-06-10T00:57:41.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.465+0000 7f43127fc700 1 -- 172.21.15.37:0/2545165056 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431800f620 con 0x7f431c101380 2024-06-10T00:57:41.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.465+0000 7f4323809700 1 -- 172.21.15.37:0/2545165056 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f431c1a5590 con 0x7f431c101380 2024-06-10T00:57:41.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.469+0000 7f43127fc700 1 -- 172.21.15.37:0/2545165056 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f4318019070 con 0x7f431c101380 2024-06-10T00:57:41.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.470+0000 7f43127fc700 1 --2- 172.21.15.37:0/2545165056 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f43080a2e80 0x7f43080a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:41.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.470+0000 7f43127fc700 1 -- 172.21.15.37:0/2545165056 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f43180cef80 con 0x7f431c101380 2024-06-10T00:57:41.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.470+0000 7f4320da4700 1 --2- 172.21.15.37:0/2545165056 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f43080a2e80 0x7f43080a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:41.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.471+0000 7f432e08cb80 1 Processor -- start 2024-06-10T00:57:41.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.471+0000 7f4320da4700 1 --2- 172.21.15.37:0/2545165056 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f43080a2e80 0x7f43080a5340 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f430c009350 tx=0x7f430c00b560 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:41.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.471+0000 7f432e08cb80 1 -- start start 2024-06-10T00:57:41.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.471+0000 7f432e08cb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5627c0c386c0 0x5627c0c38aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:41.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.472+0000 7f432e08cb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627c0c38fe0 0x5627c0c32a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:41.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.472+0000 7f43215a5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5627c0c386c0 0x5627c0c38aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:41.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.472+0000 7f43215a5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5627c0c386c0 0x5627c0c38aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44566/0 (socket says 172.21.15.37:44566) 2024-06-10T00:57:41.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.472+0000 7f432e08cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627c0c32fa0 0x5627c0c333c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:41.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.472+0000 7f4320da4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627c0c38fe0 0x5627c0c32a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:41.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.472+0000 7f4320da4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627c0c38fe0 0x5627c0c32a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:43686/0 (socket says 172.21.15.37:43686) 2024-06-10T00:57:41.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.472+0000 7f43215a5700 1 -- 172.21.15.37:0/4114563677 learned_addr learned my addr 172.21.15.37:0/4114563677 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:41.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.473+0000 7f432e08cb80 1 -- 172.21.15.37:0/4114563677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5627c0b84890 con 0x5627c0c386c0 2024-06-10T00:57:41.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.473+0000 7f432e08cb80 1 -- 172.21.15.37:0/4114563677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5627c0b2c7a0 con 0x5627c0c38fe0 2024-06-10T00:57:41.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.473+0000 7f432e08cb80 1 -- 172.21.15.37:0/4114563677 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5627c0b80850 con 0x5627c0c32fa0 2024-06-10T00:57:41.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.473+0000 7f4321da6700 1 --2- 172.21.15.37:0/4114563677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627c0c32fa0 0x5627c0c333c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:41.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.473+0000 7f43215a5700 1 -- 172.21.15.37:0/4114563677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627c0c32fa0 msgr2=0x5627c0c333c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:41.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.473+0000 7f43215a5700 1 --2- 172.21.15.37:0/4114563677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627c0c32fa0 0x5627c0c333c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.473+0000 7f43215a5700 1 -- 172.21.15.37:0/4114563677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627c0c38fe0 msgr2=0x5627c0c32a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:41.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.473+0000 7f43215a5700 1 --2- 172.21.15.37:0/4114563677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627c0c38fe0 0x5627c0c32a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.473+0000 7f43215a5700 1 -- 172.21.15.37:0/4114563677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627c0b22fd0 con 0x5627c0c386c0 2024-06-10T00:57:41.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.473+0000 7f43215a5700 1 --2- 172.21.15.37:0/4114563677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5627c0c386c0 0x5627c0c38aa0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4318097c00 tx=0x7f431809b2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb22d18ec9838bc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:41.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.474+0000 7f43077fe700 1 -- 172.21.15.37:0/4114563677 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43180b4070 con 0x5627c0c386c0 2024-06-10T00:57:41.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.474+0000 7f43077fe700 1 -- 172.21.15.37:0/4114563677 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43180b4210 con 0x5627c0c386c0 2024-06-10T00:57:41.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.474+0000 7f43077fe700 1 -- 172.21.15.37:0/4114563677 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431809f3d0 con 0x5627c0c386c0 2024-06-10T00:57:41.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.475+0000 7f432e08cb80 1 -- 172.21.15.37:0/4114563677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5627c0c386c0 msgr2=0x5627c0c38aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:41.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.475+0000 7f432e08cb80 1 --2- 172.21.15.37:0/4114563677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5627c0c386c0 0x5627c0c38aa0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4318097c00 tx=0x7f431809b2b0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.475+0000 7f432e08cb80 1 -- 172.21.15.37:0/4114563677 shutdown_connections 2024-06-10T00:57:41.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.475+0000 7f432e08cb80 1 --2- 172.21.15.37:0/4114563677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627c0c32fa0 0x5627c0c333c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.475+0000 7f432e08cb80 1 --2- 172.21.15.37:0/4114563677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5627c0c386c0 0x5627c0c38aa0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.475+0000 7f432e08cb80 1 --2- 172.21.15.37:0/4114563677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627c0c38fe0 0x5627c0c32a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.475+0000 7f432e08cb80 1 -- 172.21.15.37:0/4114563677 >> 172.21.15.37:0/4114563677 conn(0x5627c0b5dd40 msgr2=0x5627c0c2d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:41.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.475+0000 7f432e08cb80 1 -- 172.21.15.37:0/4114563677 shutdown_connections 2024-06-10T00:57:41.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.475+0000 7f432e08cb80 1 -- 172.21.15.37:0/4114563677 wait complete. 2024-06-10T00:57:41.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.475+0000 7f432e08cb80 1 -- start start 2024-06-10T00:57:41.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.476+0000 7f432e08cb80 1 Processor -- start 2024-06-10T00:57:41.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.476+0000 7f432e08cb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5627c0c32fa0 0x5627c0c3ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:41.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.476+0000 7f432e08cb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627c0c386c0 0x5627c0c3c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:41.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.476+0000 7f432e08cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627c0c38fe0 0x5627c0c3c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:41.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.476+0000 7f432e08cb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5627c0c3fa50 con 0x5627c0c32fa0 2024-06-10T00:57:41.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.476+0000 7f43215a5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5627c0c32fa0 0x5627c0c3ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:41.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.476+0000 7f43215a5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5627c0c32fa0 0x5627c0c3ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44576/0 (socket says 172.21.15.37:44576) 2024-06-10T00:57:41.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.476+0000 7f43215a5700 1 -- 172.21.15.37:0/1636125862 learned_addr learned my addr 172.21.15.37:0/1636125862 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:41.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.476+0000 7f432e08cb80 1 -- 172.21.15.37:0/1636125862 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5627c0b24c70 con 0x5627c0c386c0 2024-06-10T00:57:41.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.476+0000 7f4320da4700 1 --2- 172.21.15.37:0/1636125862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627c0c386c0 0x5627c0c3c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:41.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.477+0000 7f432e08cb80 1 -- 172.21.15.37:0/1636125862 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5627c0c3fbd0 con 0x5627c0c38fe0 2024-06-10T00:57:41.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.477+0000 7f4321da6700 1 --2- 172.21.15.37:0/1636125862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627c0c38fe0 0x5627c0c3c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:41.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.477+0000 7f43215a5700 1 -- 172.21.15.37:0/1636125862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627c0c38fe0 msgr2=0x5627c0c3c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:41.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.477+0000 7f43215a5700 1 --2- 172.21.15.37:0/1636125862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627c0c38fe0 0x5627c0c3c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.477+0000 7f43215a5700 1 -- 172.21.15.37:0/1636125862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627c0c386c0 msgr2=0x5627c0c3c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:41.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.477+0000 7f43215a5700 1 --2- 172.21.15.37:0/1636125862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627c0c386c0 0x5627c0c3c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.477+0000 7f43215a5700 1 -- 172.21.15.37:0/1636125862 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627c0ad3950 con 0x5627c0c32fa0 2024-06-10T00:57:41.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.477+0000 7f4321da6700 1 --2- 172.21.15.37:0/1636125862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627c0c38fe0 0x5627c0c3c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:57:41.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.477+0000 7f432e08cb80 10 client.0 _ll_register_callbacks cb 0x5627c0b1e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:41.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.477+0000 7f4304ff9700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:57:41.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.477+0000 7f43215a5700 1 --2- 172.21.15.37:0/1636125862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5627c0c32fa0 0x5627c0c3ed00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4318009520 tx=0x7f431809f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b994d6edcf04b659 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:41.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.477+0000 7f432e08cb80 1 -- 172.21.15.37:0/1636125862 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5627c0b91d90 con 0x5627c0c32fa0 2024-06-10T00:57:41.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.478+0000 7f4304ff9700 1 -- 172.21.15.37:0/1636125862 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43180b8070 con 0x5627c0c32fa0 2024-06-10T00:57:41.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.478+0000 7f4304ff9700 1 -- 172.21.15.37:0/1636125862 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f431809f170 con 0x5627c0c32fa0 2024-06-10T00:57:41.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.478+0000 7f4304ff9700 1 -- 172.21.15.37:0/1636125862 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f43180b4040 con 0x5627c0c32fa0 2024-06-10T00:57:41.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.478+0000 7f432e08cb80 10 client.15597 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:41.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.478+0000 7f432e08cb80 1 -- 172.21.15.37:0/1636125862 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=27}) -- 0x5627c0c49e10 con 0x5627c0c32fa0 2024-06-10T00:57:41.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.478+0000 7f4304ff9700 1 -- 172.21.15.37:0/1636125862 <== mon.0 v2:172.21.15.37:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f43180af280 con 0x5627c0c32fa0 2024-06-10T00:57:41.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.478+0000 7f432e08cb80 10 client.15597 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:41.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.479+0000 7f432e08cb80 10 client.15597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T00:57:41.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.479+0000 7f432e08cb80 20 client.15597 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:41.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.479+0000 7f432e08cb80 1 --2- 172.21.15.37:0/1636125862 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5627c0c4c2e0 0x5627c0c4e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:41.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.479+0000 7f432e08cb80 4 client.15597 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:57:41.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.479+0000 7f432e08cb80 1 -- 172.21.15.37:0/1636125862 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627c0b8c310 con 0x5627c0c4c2e0 2024-06-10T00:57:41.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.479+0000 7f4320da4700 1 --2- 172.21.15.37:0/1636125862 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5627c0c4c2e0 0x5627c0c4e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:41.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.480+0000 7f4320da4700 1 --2- 172.21.15.37:0/1636125862 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5627c0c4c2e0 0x5627c0c4e6d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62b7f1e629f64bfd server_cookie=3b42643a9a7bea24 in_seq=0 out_seq=0 2024-06-10T00:57:41.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.480+0000 7f4304ff9700 10 client.15597 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:41.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.480+0000 7f4304ff9700 1 -- 172.21.15.37:0/1636125862 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5627c0b8c310 con 0x5627c0c4c2e0 2024-06-10T00:57:41.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.480+0000 7f4304ff9700 10 client.15597 handle_command_reply: tid=0 2024-06-10T00:57:41.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.480+0000 7f432e08cb80 1 client.15597 shutdown 2024-06-10T00:57:41.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.481+0000 7f432e08cb80 1 -- 172.21.15.37:0/1636125862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5627c0c32fa0 msgr2=0x5627c0c3ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:41.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.481+0000 7f432e08cb80 1 --2- 172.21.15.37:0/1636125862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5627c0c32fa0 0x5627c0c3ed00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4318009520 tx=0x7f431809f720 comp rx=0 tx=0).stop 2024-06-10T00:57:41.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.481+0000 7f432e08cb80 1 -- 172.21.15.37:0/1636125862 shutdown_connections 2024-06-10T00:57:41.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.481+0000 7f432e08cb80 1 --2- 172.21.15.37:0/1636125862 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5627c0c4c2e0 0x5627c0c4e6d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.481+0000 7f432e08cb80 1 --2- 172.21.15.37:0/1636125862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627c0c38fe0 0x5627c0c3c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.481+0000 7f432e08cb80 1 --2- 172.21.15.37:0/1636125862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5627c0c32fa0 0x5627c0c3ed00 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.481+0000 7f432e08cb80 1 --2- 172.21.15.37:0/1636125862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627c0c386c0 0x5627c0c3c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.481+0000 7f432e08cb80 1 -- 172.21.15.37:0/1636125862 >> 172.21.15.37:0/1636125862 conn(0x5627c0b5dd40 msgr2=0x5627c0c2d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:41.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.481+0000 7f4304ff9700 0 client.15597 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:41.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.481+0000 7f432e08cb80 1 -- 172.21.15.37:0/1636125862 shutdown_connections 2024-06-10T00:57:41.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.481+0000 7f432e08cb80 1 -- 172.21.15.37:0/1636125862 wait complete. 2024-06-10T00:57:41.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.481+0000 7f432e08cb80 20 client.15597 trim_cache size 0 max 16384 2024-06-10T00:57:41.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.522+0000 7f432e08cb80 1 Processor -- start 2024-06-10T00:57:41.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.522+0000 7f432e08cb80 1 -- start start 2024-06-10T00:57:41.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.523+0000 7f432e08cb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5627c0c32fa0 0x5627c0c32a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:41.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.523+0000 7f432e08cb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627c0c386c0 0x5627c0c39c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:41.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.523+0000 7f432e08cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627c0c38fe0 0x5627c0c27590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:41.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.523+0000 7f43215a5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5627c0c32fa0 0x5627c0c32a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:41.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.523+0000 7f43215a5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5627c0c32fa0 0x5627c0c32a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44588/0 (socket says 172.21.15.37:44588) 2024-06-10T00:57:41.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.523+0000 7f43215a5700 1 -- 172.21.15.37:0/2808239836 learned_addr learned my addr 172.21.15.37:0/2808239836 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:41.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.523+0000 7f432e08cb80 1 -- 172.21.15.37:0/2808239836 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1b0cf0 con 0x5627c0c32fa0 2024-06-10T00:57:41.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.523+0000 7f432e08cb80 1 -- 172.21.15.37:0/2808239836 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1ae580 con 0x5627c0c386c0 2024-06-10T00:57:41.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.523+0000 7f432e08cb80 1 -- 172.21.15.37:0/2808239836 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1ac3d0 con 0x5627c0c38fe0 2024-06-10T00:57:41.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.523+0000 7f4320da4700 1 --2- 172.21.15.37:0/2808239836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627c0c386c0 0x5627c0c39c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:41.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.523+0000 7f43215a5700 1 -- 172.21.15.37:0/2808239836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627c0c38fe0 msgr2=0x5627c0c27590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:41.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.523+0000 7f43215a5700 1 --2- 172.21.15.37:0/2808239836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627c0c38fe0 0x5627c0c27590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.523+0000 7f43215a5700 1 -- 172.21.15.37:0/2808239836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627c0c386c0 msgr2=0x5627c0c39c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:41.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.523+0000 7f43215a5700 1 --2- 172.21.15.37:0/2808239836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627c0c386c0 0x5627c0c39c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.523+0000 7f43215a5700 1 -- 172.21.15.37:0/2808239836 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627c0c2e5a0 con 0x5627c0c32fa0 2024-06-10T00:57:41.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.524+0000 7f43215a5700 1 --2- 172.21.15.37:0/2808239836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5627c0c32fa0 0x5627c0c32a60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4318097690 tx=0x7f43180b0000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a0803b82074160 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:41.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.524+0000 7f4306ffd700 1 -- 172.21.15.37:0/2808239836 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43180b8040 con 0x5627c0c32fa0 2024-06-10T00:57:41.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.524+0000 7f4306ffd700 1 -- 172.21.15.37:0/2808239836 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5627c0b91d90 con 0x5627c0c32fa0 2024-06-10T00:57:41.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.524+0000 7f4306ffd700 1 -- 172.21.15.37:0/2808239836 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43180bb0b0 con 0x5627c0c32fa0 2024-06-10T00:57:41.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.525+0000 7f432e08cb80 1 -- 172.21.15.37:0/2808239836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5627c0c32fa0 msgr2=0x5627c0c32a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:41.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.525+0000 7f432e08cb80 1 --2- 172.21.15.37:0/2808239836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5627c0c32fa0 0x5627c0c32a60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4318097690 tx=0x7f43180b0000 comp rx=0 tx=0).stop 2024-06-10T00:57:41.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.525+0000 7f432e08cb80 1 -- 172.21.15.37:0/2808239836 shutdown_connections 2024-06-10T00:57:41.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.525+0000 7f432e08cb80 1 --2- 172.21.15.37:0/2808239836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627c0c38fe0 0x5627c0c27590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.525+0000 7f432e08cb80 1 --2- 172.21.15.37:0/2808239836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5627c0c32fa0 0x5627c0c32a60 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.525+0000 7f432e08cb80 1 --2- 172.21.15.37:0/2808239836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627c0c386c0 0x5627c0c39c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.525+0000 7f432e08cb80 1 -- 172.21.15.37:0/2808239836 >> 172.21.15.37:0/2808239836 conn(0x5627c0b5dd40 msgr2=0x5627c0c26e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:41.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.526+0000 7f432e08cb80 1 -- 172.21.15.37:0/2808239836 shutdown_connections 2024-06-10T00:57:41.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.526+0000 7f432e08cb80 1 -- 172.21.15.37:0/2808239836 wait complete. 2024-06-10T00:57:41.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.526+0000 7f432e08cb80 1 -- start start 2024-06-10T00:57:41.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.526+0000 7f432e08cb80 1 Processor -- start 2024-06-10T00:57:41.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.527+0000 7f432e08cb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627c0c32fa0 0x5627c0cb2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:41.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.527+0000 7f432e08cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627c0c386c0 0x5627c0c44ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:41.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.527+0000 7f432e08cb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5627c0c38fe0 0x5627c0c45430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:41.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.527+0000 7f432e08cb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1aca40 con 0x5627c0c38fe0 2024-06-10T00:57:41.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.527+0000 7f432e08cb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1adeb0 con 0x5627c0c32fa0 2024-06-10T00:57:41.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.527+0000 7f4321da6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5627c0c38fe0 0x5627c0c45430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:41.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.527+0000 7f4321da6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5627c0c38fe0 0x5627c0c45430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44594/0 (socket says 172.21.15.37:44594) 2024-06-10T00:57:41.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.527+0000 7f4321da6700 1 -- 172.21.15.37:0/1080684136 learned_addr learned my addr 172.21.15.37:0/1080684136 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:41.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.527+0000 7f4320da4700 1 --2- 172.21.15.37:0/1080684136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627c0c386c0 0x5627c0c44ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:41.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.527+0000 7f43215a5700 1 --2- 172.21.15.37:0/1080684136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627c0c32fa0 0x5627c0cb2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:41.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.528+0000 7f432e08cb80 1 -- 172.21.15.37:0/1080684136 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1ad110 con 0x5627c0c386c0 2024-06-10T00:57:41.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.528+0000 7f432e08cb80 10 client.0 _ll_register_callbacks cb 0x5627c0b1e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:41.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.528+0000 7f4321da6700 1 -- 172.21.15.37:0/1080684136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627c0c386c0 msgr2=0x5627c0c44ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:41.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.528+0000 7f4321da6700 1 --2- 172.21.15.37:0/1080684136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627c0c386c0 0x5627c0c44ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.528+0000 7f4321da6700 1 -- 172.21.15.37:0/1080684136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627c0c32fa0 msgr2=0x5627c0cb2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:41.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.528+0000 7f4321da6700 1 --2- 172.21.15.37:0/1080684136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627c0c32fa0 0x5627c0cb2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.528+0000 7f4321da6700 1 -- 172.21.15.37:0/1080684136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627c0c459f0 con 0x5627c0c38fe0 2024-06-10T00:57:41.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.528+0000 7f432e08cb80 1 -- 172.21.15.37:0/1080684136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5627c0b91d90 con 0x5627c0c38fe0 2024-06-10T00:57:41.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.528+0000 7f4321da6700 1 --2- 172.21.15.37:0/1080684136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5627c0c38fe0 0x5627c0c45430 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4314005bd0 tx=0x7f4314004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50016ffcdcba2ecc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:41.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.528+0000 7f4304ff9700 10 client.15603 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:57:41.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.529+0000 7f4304ff9700 1 -- 172.21.15.37:0/1080684136 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431401a020 con 0x5627c0c38fe0 2024-06-10T00:57:41.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.529+0000 7f4304ff9700 1 -- 172.21.15.37:0/1080684136 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f4314016670 con 0x5627c0c38fe0 2024-06-10T00:57:41.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.529+0000 7f4304ff9700 1 -- 172.21.15.37:0/1080684136 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f43140168f0 con 0x5627c0c38fe0 2024-06-10T00:57:41.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.529+0000 7f432e08cb80 10 client.15603 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:41.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.529+0000 7f432e08cb80 1 -- 172.21.15.37:0/1080684136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=27}) -- 0x5627c0cbfad0 con 0x5627c0c38fe0 2024-06-10T00:57:41.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.529+0000 7f4304ff9700 1 -- 172.21.15.37:0/1080684136 <== mon.0 v2:172.21.15.37:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4314027060 con 0x5627c0c38fe0 2024-06-10T00:57:41.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.529+0000 7f432e08cb80 10 client.15603 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:41.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.529+0000 7f432e08cb80 10 client.15603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T00:57:41.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.529+0000 7f432e08cb80 20 client.15603 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:41.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.529+0000 7f432e08cb80 1 --2- 172.21.15.37:0/1080684136 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5627c0cbffa0 0x5627c0cc23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:41.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.529+0000 7f432e08cb80 4 client.15603 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T00:57:41.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.529+0000 7f432e08cb80 1 -- 172.21.15.37:0/1080684136 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5627c0a062d0 con 0x5627c0cbffa0 2024-06-10T00:57:41.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.530+0000 7f43215a5700 1 --2- 172.21.15.37:0/1080684136 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5627c0cbffa0 0x5627c0cc23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:41.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.530+0000 7f4304ff9700 10 client.15603 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:41.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.530+0000 7f43215a5700 1 --2- 172.21.15.37:0/1080684136 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5627c0cbffa0 0x5627c0cc23c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a268f4d3a7a1b36b server_cookie=aa4a1af52539e5af in_seq=0 out_seq=0 2024-06-10T00:57:41.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.532+0000 7f4304ff9700 1 -- 172.21.15.37:0/1080684136 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5627c0a062d0 con 0x5627c0cbffa0 2024-06-10T00:57:41.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.532+0000 7f4304ff9700 10 client.15603 handle_command_reply: tid=0 2024-06-10T00:57:41.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.532+0000 7f432e08cb80 1 client.15603 shutdown 2024-06-10T00:57:41.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.532+0000 7f432e08cb80 1 -- 172.21.15.37:0/1080684136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5627c0c38fe0 msgr2=0x5627c0c45430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:41.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.532+0000 7f432e08cb80 1 --2- 172.21.15.37:0/1080684136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5627c0c38fe0 0x5627c0c45430 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4314005bd0 tx=0x7f4314004320 comp rx=0 tx=0).stop 2024-06-10T00:57:41.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.532+0000 7f432e08cb80 1 -- 172.21.15.37:0/1080684136 shutdown_connections 2024-06-10T00:57:41.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.532+0000 7f432e08cb80 1 --2- 172.21.15.37:0/1080684136 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5627c0cbffa0 0x5627c0cc23c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.532+0000 7f432e08cb80 1 --2- 172.21.15.37:0/1080684136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5627c0c38fe0 0x5627c0c45430 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.532+0000 7f432e08cb80 1 --2- 172.21.15.37:0/1080684136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627c0c32fa0 0x5627c0cb2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.532+0000 7f432e08cb80 1 --2- 172.21.15.37:0/1080684136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5627c0c386c0 0x5627c0c44ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.532+0000 7f432e08cb80 1 -- 172.21.15.37:0/1080684136 >> 172.21.15.37:0/1080684136 conn(0x5627c0b5dd40 msgr2=0x5627c0c3d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:41.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.532+0000 7f4304ff9700 0 client.15603 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:41.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.532+0000 7f432e08cb80 1 -- 172.21.15.37:0/1080684136 shutdown_connections 2024-06-10T00:57:41.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.532+0000 7f432e08cb80 1 -- 172.21.15.37:0/1080684136 wait complete. 2024-06-10T00:57:41.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.533+0000 7f432e08cb80 20 client.15603 trim_cache size 0 max 16384 2024-06-10T00:57:41.535 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T00:57:41.535 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-06-10T00:57:41.535 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "1a88450c-ee00-4379-bc75-c7682bbc5717", 2024-06-10T00:57:41.535 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-06-10T00:57:41.536 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T00:57:41.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.534+0000 7f4323809700 1 -- 172.21.15.37:0/2545165056 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f43080a2e80 msgr2=0x7f43080a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:41.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.534+0000 7f4323809700 1 --2- 172.21.15.37:0/2545165056 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f43080a2e80 0x7f43080a5340 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f430c009350 tx=0x7f430c00b560 comp rx=0 tx=0).stop 2024-06-10T00:57:41.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.534+0000 7f4323809700 1 -- 172.21.15.37:0/2545165056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f431c101380 msgr2=0x7f431c19e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:41.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.534+0000 7f4323809700 1 --2- 172.21.15.37:0/2545165056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f431c101380 0x7f431c19e6c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f431800d950 tx=0x7f431800dc60 comp rx=0 tx=0).stop 2024-06-10T00:57:41.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.535+0000 7f4323809700 1 -- 172.21.15.37:0/2545165056 shutdown_connections 2024-06-10T00:57:41.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.535+0000 7f4323809700 1 --2- 172.21.15.37:0/2545165056 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f43080a2e80 0x7f43080a5340 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.535+0000 7f4323809700 1 --2- 172.21.15.37:0/2545165056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f431c1073f0 0x7f431c198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.535+0000 7f4323809700 1 --2- 172.21.15.37:0/2545165056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f431c101380 0x7f431c19e6c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.535+0000 7f4323809700 1 --2- 172.21.15.37:0/2545165056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f431c101d40 0x7f431c19ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:41.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.535+0000 7f4323809700 1 -- 172.21.15.37:0/2545165056 >> 172.21.15.37:0/2545165056 conn(0x7f431c078300 msgr2=0x7f431c0ffab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:41.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.535+0000 7f4323809700 1 -- 172.21.15.37:0/2545165056 shutdown_connections 2024-06-10T00:57:41.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:41.535+0000 7f4323809700 1 -- 172.21.15.37:0/2545165056 wait complete. 2024-06-10T00:57:42.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:42 smithi037 ceph-mon[32192]: pgmap v460: 129 pgs: 129 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-10T00:57:42.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:42 smithi167 ceph-mon[36917]: pgmap v460: 129 pgs: 129 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-10T00:57:42.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:42 smithi094 ceph-mon[36708]: pgmap v460: 129 pgs: 129 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-10T00:57:42.882 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T00:57:43.375 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:43 smithi037 ceph-mon[32192]: scrub queued for path: / 2024-06-10T00:57:43.375 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:43 smithi037 ceph-mon[32192]: scrub summary: idle+waiting paths [/] 2024-06-10T00:57:43.375 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:43 smithi037 ceph-mon[32192]: scrub summary: active paths [/] 2024-06-10T00:57:43.375 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:43 smithi037 ceph-mon[32192]: scrub summary: idle+waiting paths [/] 2024-06-10T00:57:43.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.458+0000 7ff4c60ee700 1 -- 172.21.15.37:0/3534782805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4c0106750 msgr2=0x7ff4c010e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:43.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.458+0000 7ff4c60ee700 1 --2- 172.21.15.37:0/3534782805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4c0106750 0x7ff4c010e2d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800b0d0 tx=0x7ff4b800b3e0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.458+0000 7ff4c60ee700 1 -- 172.21.15.37:0/3534782805 shutdown_connections 2024-06-10T00:57:43.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.458+0000 7ff4c60ee700 1 --2- 172.21.15.37:0/3534782805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4c0106750 0x7ff4c010e2d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.458+0000 7ff4c60ee700 1 --2- 172.21.15.37:0/3534782805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4c0101df0 0x7ff4c01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.458+0000 7ff4c60ee700 1 --2- 172.21.15.37:0/3534782805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4c01027a0 0x7ff4c0106020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.458+0000 7ff4c60ee700 1 -- 172.21.15.37:0/3534782805 >> 172.21.15.37:0/3534782805 conn(0x7ff4c0077a30 msgr2=0x7ff4c00fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:43.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.459+0000 7ff4c60ee700 1 -- 172.21.15.37:0/3534782805 shutdown_connections 2024-06-10T00:57:43.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.459+0000 7ff4c60ee700 1 -- 172.21.15.37:0/3534782805 wait complete. 2024-06-10T00:57:43.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.460+0000 7ff4c60ee700 1 Processor -- start 2024-06-10T00:57:43.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.460+0000 7ff4c60ee700 1 -- start start 2024-06-10T00:57:43.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.461+0000 7ff4c60ee700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4c0101df0 0x7ff4c0113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:43.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.461+0000 7ff4c60ee700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4c01027a0 0x7ff4c0073f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:43.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.461+0000 7ff4c60ee700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4c0106750 0x7ff4c0111e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:43.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.461+0000 7ff4c60ee700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c01116c0 con 0x7ff4c0106750 2024-06-10T00:57:43.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.461+0000 7ff4c60ee700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c0111540 con 0x7ff4c01027a0 2024-06-10T00:57:43.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.461+0000 7ff4c60ee700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c0111840 con 0x7ff4c0101df0 2024-06-10T00:57:43.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.461+0000 7ff4beffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4c01027a0 0x7ff4c0073f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:43.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.461+0000 7ff4beffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4c01027a0 0x7ff4c0073f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:37970/0 (socket says 172.21.15.37:37970) 2024-06-10T00:57:43.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.461+0000 7ff4beffd700 1 -- 172.21.15.37:0/3758692664 learned_addr learned my addr 172.21.15.37:0/3758692664 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:43.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.462+0000 7ff4bf7fe700 1 --2- 172.21.15.37:0/3758692664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4c0101df0 0x7ff4c0113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:43.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.462+0000 7ff4bffff700 1 --2- 172.21.15.37:0/3758692664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4c0106750 0x7ff4c0111e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:43.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.462+0000 7ff4beffd700 1 -- 172.21.15.37:0/3758692664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4c0101df0 msgr2=0x7ff4c0113950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:43.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.462+0000 7ff4beffd700 1 --2- 172.21.15.37:0/3758692664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4c0101df0 0x7ff4c0113950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.462+0000 7ff4beffd700 1 -- 172.21.15.37:0/3758692664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4c0106750 msgr2=0x7ff4c0111e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:43.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.462+0000 7ff4beffd700 1 --2- 172.21.15.37:0/3758692664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4c0106750 0x7ff4c0111e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.462+0000 7ff4beffd700 1 -- 172.21.15.37:0/3758692664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c01124c0 con 0x7ff4c01027a0 2024-06-10T00:57:43.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.462+0000 7ff4bffff700 1 --2- 172.21.15.37:0/3758692664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4c0106750 0x7ff4c0111e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:57:43.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.462+0000 7ff4beffd700 1 --2- 172.21.15.37:0/3758692664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4c01027a0 0x7ff4c0073f10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400d9b0 tx=0x7ff4b400dd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:43.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.463+0000 7ff4bcff9700 1 -- 172.21.15.37:0/3758692664 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4016070 con 0x7ff4c01027a0 2024-06-10T00:57:43.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.463+0000 7ff4c60ee700 1 -- 172.21.15.37:0/3758692664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c01127b0 con 0x7ff4c01027a0 2024-06-10T00:57:43.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.463+0000 7ff4c60ee700 1 -- 172.21.15.37:0/3758692664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c01a9db0 con 0x7ff4c01027a0 2024-06-10T00:57:43.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.465+0000 7ff4bcff9700 1 -- 172.21.15.37:0/3758692664 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4b4004500 con 0x7ff4c01027a0 2024-06-10T00:57:43.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.466+0000 7ff4bcff9700 1 -- 172.21.15.37:0/3758692664 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4004d60 con 0x7ff4c01027a0 2024-06-10T00:57:43.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.467+0000 7ff4bcff9700 1 -- 172.21.15.37:0/3758692664 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7ff4b4005020 con 0x7ff4c01027a0 2024-06-10T00:57:43.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.471+0000 7ff4bcff9700 1 --2- 172.21.15.37:0/3758692664 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff4a80a2e60 0x7ff4a80a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:43.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.471+0000 7ff4bf7fe700 1 --2- 172.21.15.37:0/3758692664 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff4a80a2e60 0x7ff4a80a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:43.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.471+0000 7ff4d0971b80 1 Processor -- start 2024-06-10T00:57:43.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.472+0000 7ff4bf7fe700 1 --2- 172.21.15.37:0/3758692664 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff4a80a2e60 0x7ff4a80a5320 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0005950 tx=0x7ff4b00058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:43.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.472+0000 7ff4bcff9700 1 -- 172.21.15.37:0/3758692664 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7ff4b40d03f0 con 0x7ff4c01027a0 2024-06-10T00:57:43.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.472+0000 7ff4d0971b80 1 -- start start 2024-06-10T00:57:43.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.473+0000 7ff4d0971b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558849957590 0x558849957970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:43.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.473+0000 7ff4d0971b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558849957eb0 0x5588499517a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:43.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.473+0000 7ff4d0971b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558849951ce0 0x558849952100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:43.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.473+0000 7ff4bf7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558849957590 0x558849957970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:43.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.473+0000 7ff4bf7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558849957590 0x558849957970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:37978/0 (socket says 172.21.15.37:37978) 2024-06-10T00:57:43.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.473+0000 7ff4beffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558849957eb0 0x5588499517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:43.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.473+0000 7ff4beffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558849957eb0 0x5588499517a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35478/0 (socket says 172.21.15.37:35478) 2024-06-10T00:57:43.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.474+0000 7ff4bf7fe700 1 -- 172.21.15.37:0/2533302655 learned_addr learned my addr 172.21.15.37:0/2533302655 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:43.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.474+0000 7ff4d0971b80 1 -- 172.21.15.37:0/2533302655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5588498440f0 con 0x558849957eb0 2024-06-10T00:57:43.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.474+0000 7ff4d0971b80 1 -- 172.21.15.37:0/2533302655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5588498a29e0 con 0x558849957590 2024-06-10T00:57:43.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.474+0000 7ff4d0971b80 1 -- 172.21.15.37:0/2533302655 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55884980a5f0 con 0x558849951ce0 2024-06-10T00:57:43.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.474+0000 7ff4bffff700 1 --2- 172.21.15.37:0/2533302655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558849951ce0 0x558849952100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:43.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.474+0000 7ff4beffd700 1 -- 172.21.15.37:0/2533302655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558849951ce0 msgr2=0x558849952100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:43.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.474+0000 7ff4beffd700 1 --2- 172.21.15.37:0/2533302655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558849951ce0 0x558849952100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.474+0000 7ff4beffd700 1 -- 172.21.15.37:0/2533302655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558849957590 msgr2=0x558849957970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:43.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.474+0000 7ff4beffd700 1 --2- 172.21.15.37:0/2533302655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558849957590 0x558849957970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.474+0000 7ff4beffd700 1 -- 172.21.15.37:0/2533302655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588497f1a60 con 0x558849957eb0 2024-06-10T00:57:43.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.475+0000 7ff4beffd700 1 --2- 172.21.15.37:0/2533302655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558849957eb0 0x5588499517a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40cd950 tx=0x7ff4b409c080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3db21bd1fbe8e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:43.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.475+0000 7ff4a5ffb700 1 -- 172.21.15.37:0/2533302655 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b40870c0 con 0x558849957eb0 2024-06-10T00:57:43.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.475+0000 7ff4a5ffb700 1 -- 172.21.15.37:0/2533302655 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4b409c6a0 con 0x558849957eb0 2024-06-10T00:57:43.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.475+0000 7ff4a5ffb700 1 -- 172.21.15.37:0/2533302655 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b40a50e0 con 0x558849957eb0 2024-06-10T00:57:43.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.476+0000 7ff4d0971b80 1 -- 172.21.15.37:0/2533302655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558849957eb0 msgr2=0x5588499517a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:43.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.476+0000 7ff4d0971b80 1 --2- 172.21.15.37:0/2533302655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558849957eb0 0x5588499517a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40cd950 tx=0x7ff4b409c080 comp rx=0 tx=0).stop 2024-06-10T00:57:43.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.476+0000 7ff4d0971b80 1 -- 172.21.15.37:0/2533302655 shutdown_connections 2024-06-10T00:57:43.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.476+0000 7ff4d0971b80 1 --2- 172.21.15.37:0/2533302655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558849951ce0 0x558849952100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.476+0000 7ff4d0971b80 1 --2- 172.21.15.37:0/2533302655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558849957590 0x558849957970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.476+0000 7ff4d0971b80 1 --2- 172.21.15.37:0/2533302655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558849957eb0 0x5588499517a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.476+0000 7ff4d0971b80 1 -- 172.21.15.37:0/2533302655 >> 172.21.15.37:0/2533302655 conn(0x55884987e210 msgr2=0x55884994ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:43.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.476+0000 7ff4d0971b80 1 -- 172.21.15.37:0/2533302655 shutdown_connections 2024-06-10T00:57:43.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.476+0000 7ff4d0971b80 1 -- 172.21.15.37:0/2533302655 wait complete. 2024-06-10T00:57:43.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.477+0000 7ff4d0971b80 1 -- start start 2024-06-10T00:57:43.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.477+0000 7ff4d0971b80 1 Processor -- start 2024-06-10T00:57:43.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.477+0000 7ff4d0971b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558849951ce0 0x55884995dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:43.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.478+0000 7ff4bf7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558849951ce0 0x55884995dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:43.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.478+0000 7ff4bf7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558849951ce0 0x55884995dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35486/0 (socket says 172.21.15.37:35486) 2024-06-10T00:57:43.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.478+0000 7ff4bf7fe700 1 -- 172.21.15.37:0/1201669250 learned_addr learned my addr 172.21.15.37:0/1201669250 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:43.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.478+0000 7ff4d0971b80 1 --2- 172.21.15.37:0/1201669250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558849957590 0x55884995af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:43.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.478+0000 7ff4d0971b80 1 --2- 172.21.15.37:0/1201669250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558849957eb0 0x55884995b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:43.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.478+0000 7ff4d0971b80 1 -- 172.21.15.37:0/1201669250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558849843150 con 0x558849951ce0 2024-06-10T00:57:43.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.478+0000 7ff4d0971b80 1 -- 172.21.15.37:0/1201669250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55884989e9a0 con 0x558849957590 2024-06-10T00:57:43.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.478+0000 7ff4d0971b80 1 -- 172.21.15.37:0/1201669250 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55884995e7d0 con 0x558849957eb0 2024-06-10T00:57:43.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.478+0000 7ff4d0971b80 10 client.0 _ll_register_callbacks cb 0x558849838740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:43.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.478+0000 7ff4beffd700 1 --2- 172.21.15.37:0/1201669250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558849957590 0x55884995af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:43.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.478+0000 7ff4bffff700 1 --2- 172.21.15.37:0/1201669250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558849957eb0 0x55884995b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:43.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.479+0000 7ff4bffff700 1 -- 172.21.15.37:0/1201669250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558849957590 msgr2=0x55884995af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:43.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.479+0000 7ff4bffff700 1 --2- 172.21.15.37:0/1201669250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558849957590 0x55884995af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.479+0000 7ff4bffff700 1 -- 172.21.15.37:0/1201669250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558849951ce0 msgr2=0x55884995dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:43.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.479+0000 7ff4bffff700 1 --2- 172.21.15.37:0/1201669250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558849951ce0 0x55884995dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.479+0000 7ff4bffff700 1 -- 172.21.15.37:0/1201669250 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588498688e0 con 0x558849957eb0 2024-06-10T00:57:43.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.479+0000 7ff4bf7fe700 1 --2- 172.21.15.37:0/1201669250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558849951ce0 0x55884995dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:57:43.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.479+0000 7ff4bffff700 1 --2- 172.21.15.37:0/1201669250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558849957eb0 0x55884995b480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009fd0 tx=0x7ff4b8004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4806acfe3cb1d0f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:43.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.479+0000 7ff49effd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T00:57:43.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.479+0000 7ff49effd700 1 -- 172.21.15.37:0/1201669250 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b80076f0 con 0x558849957eb0 2024-06-10T00:57:43.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.479+0000 7ff4d0971b80 1 -- 172.21.15.37:0/1201669250 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4b409c6a0 con 0x558849957eb0 2024-06-10T00:57:43.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.480+0000 7ff49effd700 1 -- 172.21.15.37:0/1201669250 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7ff4b8007890 con 0x558849957eb0 2024-06-10T00:57:43.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.480+0000 7ff49effd700 1 -- 172.21.15.37:0/1201669250 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff4b802f050 con 0x558849957eb0 2024-06-10T00:57:43.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.480+0000 7ff4d0971b80 10 client.25249 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:43.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.480+0000 7ff4d0971b80 1 -- 172.21.15.37:0/1201669250 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558849968dd0 con 0x558849957eb0 2024-06-10T00:57:43.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.480+0000 7ff49effd700 1 -- 172.21.15.37:0/1201669250 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4b8005040 con 0x558849957eb0 2024-06-10T00:57:43.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.481+0000 7ff49effd700 1 -- 172.21.15.37:0/1201669250 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff4b8011070 con 0x558849957eb0 2024-06-10T00:57:43.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.481+0000 7ff4d0971b80 10 client.25249 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:43.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.481+0000 7ff4d0971b80 10 client.25249 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T00:57:43.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.481+0000 7ff4d0971b80 20 client.25249 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:43.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.481+0000 7ff4d0971b80 1 --2- 172.21.15.37:0/1201669250 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55884996b3b0 0x55884996d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:43.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.481+0000 7ff4d0971b80 4 client.25249 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:57:43.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.481+0000 7ff4d0971b80 1 -- 172.21.15.37:0/1201669250 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff4b409c6a0 con 0x55884996b3b0 2024-06-10T00:57:43.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.481+0000 7ff4bf7fe700 1 --2- 172.21.15.37:0/1201669250 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55884996b3b0 0x55884996d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:43.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.482+0000 7ff4bf7fe700 1 --2- 172.21.15.37:0/1201669250 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55884996b3b0 0x55884996d7a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17b235290ab7b2cf server_cookie=b9b193338baaf4c in_seq=0 out_seq=0 2024-06-10T00:57:43.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.482+0000 7ff49effd700 10 client.25249 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:43.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.483+0000 7ff49effd700 1 -- 172.21.15.37:0/1201669250 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7ff4b409c6a0 con 0x55884996b3b0 2024-06-10T00:57:43.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.483+0000 7ff49effd700 10 client.25249 handle_command_reply: tid=0 2024-06-10T00:57:43.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.483+0000 7ff4d0971b80 1 client.25249 shutdown 2024-06-10T00:57:43.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.483+0000 7ff4d0971b80 1 -- 172.21.15.37:0/1201669250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558849957eb0 msgr2=0x55884995b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:43.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.483+0000 7ff4d0971b80 1 --2- 172.21.15.37:0/1201669250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558849957eb0 0x55884995b480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009fd0 tx=0x7ff4b8004c80 comp rx=0 tx=0).stop 2024-06-10T00:57:43.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.484+0000 7ff4d0971b80 1 -- 172.21.15.37:0/1201669250 shutdown_connections 2024-06-10T00:57:43.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.484+0000 7ff4d0971b80 1 --2- 172.21.15.37:0/1201669250 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55884996b3b0 0x55884996d7a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.484+0000 7ff4d0971b80 1 --2- 172.21.15.37:0/1201669250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558849957eb0 0x55884995b480 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.484+0000 7ff49effd700 0 client.25249 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:43.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.484+0000 7ff4d0971b80 1 --2- 172.21.15.37:0/1201669250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558849951ce0 0x55884995dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.485+0000 7ff4d0971b80 1 --2- 172.21.15.37:0/1201669250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558849957590 0x55884995af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.485+0000 7ff4d0971b80 1 -- 172.21.15.37:0/1201669250 >> 172.21.15.37:0/1201669250 conn(0x55884987e210 msgr2=0x55884994b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:43.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.485+0000 7ff4d0971b80 1 -- 172.21.15.37:0/1201669250 shutdown_connections 2024-06-10T00:57:43.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.485+0000 7ff4d0971b80 1 -- 172.21.15.37:0/1201669250 wait complete. 2024-06-10T00:57:43.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.485+0000 7ff4d0971b80 20 client.25249 trim_cache size 0 max 16384 2024-06-10T00:57:43.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.527+0000 7ff4d0971b80 1 Processor -- start 2024-06-10T00:57:43.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.527+0000 7ff4d0971b80 1 -- start start 2024-06-10T00:57:43.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.527+0000 7ff4d0971b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558849951ce0 0x558849959f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:43.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.528+0000 7ff4d0971b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558849957590 0x558849956890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:43.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.528+0000 7ff4bf7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558849951ce0 0x558849959f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:43.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.528+0000 7ff4d0971b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558849957eb0 0x55884995d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:43.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.528+0000 7ff4d0971b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac011500 con 0x558849957590 2024-06-10T00:57:43.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.529+0000 7ff4bffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558849957eb0 0x55884995d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:43.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.529+0000 7ff4bffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558849957eb0 0x55884995d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:37994/0 (socket says 172.21.15.37:37994) 2024-06-10T00:57:43.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.529+0000 7ff4bffff700 1 -- 172.21.15.37:0/3283237664 learned_addr learned my addr 172.21.15.37:0/3283237664 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:43.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.529+0000 7ff4beffd700 1 --2- 172.21.15.37:0/3283237664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558849957590 0x558849956890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:43.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.529+0000 7ff4d0971b80 1 -- 172.21.15.37:0/3283237664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac00ed90 con 0x558849957eb0 2024-06-10T00:57:43.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.529+0000 7ff4d0971b80 1 -- 172.21.15.37:0/3283237664 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac00cbe0 con 0x558849951ce0 2024-06-10T00:57:43.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.530+0000 7ff4bffff700 1 -- 172.21.15.37:0/3283237664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558849951ce0 msgr2=0x558849959f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:43.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.530+0000 7ff4bffff700 1 --2- 172.21.15.37:0/3283237664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558849951ce0 0x558849959f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.530+0000 7ff4bffff700 1 -- 172.21.15.37:0/3283237664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558849957590 msgr2=0x558849956890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:43.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.530+0000 7ff4bffff700 1 --2- 172.21.15.37:0/3283237664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558849957590 0x558849956890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.530+0000 7ff4bffff700 1 -- 172.21.15.37:0/3283237664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588499594b0 con 0x558849957eb0 2024-06-10T00:57:43.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.530+0000 7ff4bffff700 1 --2- 172.21.15.37:0/3283237664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558849957eb0 0x55884995d780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009bb0 tx=0x7ff4b80099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afd77513953393b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:43.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.530+0000 7ff4a57fa700 1 -- 172.21.15.37:0/3283237664 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8009c50 con 0x558849957eb0 2024-06-10T00:57:43.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.530+0000 7ff4a57fa700 1 -- 172.21.15.37:0/3283237664 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4b8005e70 con 0x558849957eb0 2024-06-10T00:57:43.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.530+0000 7ff4a57fa700 1 -- 172.21.15.37:0/3283237664 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8005040 con 0x558849957eb0 2024-06-10T00:57:43.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.531+0000 7ff4d0971b80 1 -- 172.21.15.37:0/3283237664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558849957eb0 msgr2=0x55884995d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:43.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.531+0000 7ff4d0971b80 1 --2- 172.21.15.37:0/3283237664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558849957eb0 0x55884995d780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009bb0 tx=0x7ff4b80099e0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.531+0000 7ff4d0971b80 1 -- 172.21.15.37:0/3283237664 shutdown_connections 2024-06-10T00:57:43.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.531+0000 7ff4d0971b80 1 --2- 172.21.15.37:0/3283237664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558849957eb0 0x55884995d780 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.531+0000 7ff4d0971b80 1 --2- 172.21.15.37:0/3283237664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558849951ce0 0x558849959f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.531+0000 7ff4d0971b80 1 --2- 172.21.15.37:0/3283237664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558849957590 0x558849956890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.531+0000 7ff4d0971b80 1 -- 172.21.15.37:0/3283237664 >> 172.21.15.37:0/3283237664 conn(0x55884987e210 msgr2=0x5588499589c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:43.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.532+0000 7ff4d0971b80 1 -- 172.21.15.37:0/3283237664 shutdown_connections 2024-06-10T00:57:43.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.532+0000 7ff4d0971b80 1 -- 172.21.15.37:0/3283237664 wait complete. 2024-06-10T00:57:43.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.532+0000 7ff4d0971b80 1 -- start start 2024-06-10T00:57:43.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.532+0000 7ff4d0971b80 1 Processor -- start 2024-06-10T00:57:43.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.533+0000 7ff4d0971b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558849951ce0 0x558849963a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:43.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.533+0000 7ff4d0971b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558849957590 0x558849963fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:43.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.533+0000 7ff4d0971b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558849957eb0 0x558849960a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:43.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.533+0000 7ff4d0971b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac00d250 con 0x558849957eb0 2024-06-10T00:57:43.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.533+0000 7ff4d0971b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac00e6c0 con 0x558849957590 2024-06-10T00:57:43.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.533+0000 7ff4d0971b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac00d920 con 0x558849951ce0 2024-06-10T00:57:43.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.533+0000 7ff4d0971b80 10 client.0 _ll_register_callbacks cb 0x558849838740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:43.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.533+0000 7ff4bffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558849957eb0 0x558849960a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:43.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.533+0000 7ff4bffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558849957eb0 0x558849960a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35514/0 (socket says 172.21.15.37:35514) 2024-06-10T00:57:43.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.533+0000 7ff4bffff700 1 -- 172.21.15.37:0/3613725957 learned_addr learned my addr 172.21.15.37:0/3613725957 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:43.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.534+0000 7ff4bffff700 1 -- 172.21.15.37:0/3613725957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558849951ce0 msgr2=0x558849963a90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T00:57:43.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.534+0000 7ff4bffff700 1 --2- 172.21.15.37:0/3613725957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558849951ce0 0x558849963a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.534+0000 7ff4bffff700 1 -- 172.21.15.37:0/3613725957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558849957590 msgr2=0x558849963fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T00:57:43.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.534+0000 7ff4bffff700 1 --2- 172.21.15.37:0/3613725957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558849957590 0x558849963fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.534+0000 7ff4bffff700 1 -- 172.21.15.37:0/3613725957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588499646d0 con 0x558849957eb0 2024-06-10T00:57:43.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.534+0000 7ff4bffff700 1 --2- 172.21.15.37:0/3613725957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558849957eb0 0x558849960a60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8004cc0 tx=0x7ff4b8007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5117099e3a95fb72 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:43.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.534+0000 7ff49effd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:57:43.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.534+0000 7ff49effd700 1 -- 172.21.15.37:0/3613725957 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b80076f0 con 0x558849957eb0 2024-06-10T00:57:43.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.534+0000 7ff4d0971b80 1 -- 172.21.15.37:0/3613725957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4b8005e70 con 0x558849957eb0 2024-06-10T00:57:43.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.535+0000 7ff49effd700 1 -- 172.21.15.37:0/3613725957 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7ff4b8009490 con 0x558849957eb0 2024-06-10T00:57:43.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.535+0000 7ff49effd700 1 -- 172.21.15.37:0/3613725957 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff4b8006020 con 0x558849957eb0 2024-06-10T00:57:43.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.535+0000 7ff4d0971b80 10 client.15621 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:43.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.535+0000 7ff4d0971b80 1 -- 172.21.15.37:0/3613725957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5588499d65f0 con 0x558849957eb0 2024-06-10T00:57:43.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.535+0000 7ff49effd700 1 -- 172.21.15.37:0/3613725957 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4b8005e70 con 0x558849957eb0 2024-06-10T00:57:43.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.536+0000 7ff49effd700 1 -- 172.21.15.37:0/3613725957 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff4b8011040 con 0x558849957eb0 2024-06-10T00:57:43.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.536+0000 7ff4d0971b80 10 client.15621 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:43.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.536+0000 7ff4d0971b80 10 client.15621 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T00:57:43.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.536+0000 7ff4d0971b80 20 client.15621 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:43.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.536+0000 7ff4d0971b80 1 --2- 172.21.15.37:0/3613725957 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55884996b3b0 0x5588499d8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:43.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.537+0000 7ff4d0971b80 4 client.15621 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-06-10T00:57:43.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.537+0000 7ff4d0971b80 1 -- 172.21.15.37:0/3613725957 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x5588498afee0 con 0x55884996b3b0 2024-06-10T00:57:43.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.537+0000 7ff4bf7fe700 1 --2- 172.21.15.37:0/3613725957 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55884996b3b0 0x5588499d8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:43.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.537+0000 7ff4bf7fe700 1 --2- 172.21.15.37:0/3613725957 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55884996b3b0 0x5588499d8d40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95e4d70cee34360e server_cookie=a7fc70f33cc7b8be in_seq=0 out_seq=0 2024-06-10T00:57:43.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.537+0000 7ff49effd700 10 client.15621 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:43.546 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:57:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.545+0000 7ff49effd700 1 -- 172.21.15.37:0/3613725957 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5588498afee0 con 0x55884996b3b0 2024-06-10T00:57:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.545+0000 7ff49effd700 10 client.15621 handle_command_reply: tid=0 2024-06-10T00:57:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.545+0000 7ff4d0971b80 1 client.15621 shutdown 2024-06-10T00:57:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.546+0000 7ff4d0971b80 1 -- 172.21.15.37:0/3613725957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558849957eb0 msgr2=0x558849960a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.546+0000 7ff4d0971b80 1 --2- 172.21.15.37:0/3613725957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558849957eb0 0x558849960a60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8004cc0 tx=0x7ff4b8007e00 comp rx=0 tx=0).stop 2024-06-10T00:57:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.546+0000 7ff4d0971b80 1 -- 172.21.15.37:0/3613725957 shutdown_connections 2024-06-10T00:57:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.546+0000 7ff4d0971b80 1 --2- 172.21.15.37:0/3613725957 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55884996b3b0 0x5588499d8d40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.546+0000 7ff4d0971b80 1 --2- 172.21.15.37:0/3613725957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558849957eb0 0x558849960a60 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.546+0000 7ff4d0971b80 1 --2- 172.21.15.37:0/3613725957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558849951ce0 0x558849963a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.546+0000 7ff4d0971b80 1 --2- 172.21.15.37:0/3613725957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558849957590 0x558849963fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.546+0000 7ff4d0971b80 1 -- 172.21.15.37:0/3613725957 >> 172.21.15.37:0/3613725957 conn(0x55884987e210 msgr2=0x558849956890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:43.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.546+0000 7ff49effd700 0 client.15621 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:43.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.546+0000 7ff4d0971b80 1 -- 172.21.15.37:0/3613725957 shutdown_connections 2024-06-10T00:57:43.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.546+0000 7ff4d0971b80 1 -- 172.21.15.37:0/3613725957 wait complete. 2024-06-10T00:57:43.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.547+0000 7ff4d0971b80 20 client.15621 trim_cache size 0 max 16384 2024-06-10T00:57:43.549 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T00:57:43.549 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-06-10T00:57:43.549 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24472, 2024-06-10T00:57:43.549 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 2, 2024-06-10T00:57:43.549 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-06-10T00:57:43.549 INFO:teuthology.orchestra.run.smithi037.stdout: "f993ef51": { 2024-06-10T00:57:43.550 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 2, 2024-06-10T00:57:43.550 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-06-10T00:57:43.550 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T00:57:43.550 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-06-10T00:57:43.550 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T00:57:43.550 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T00:57:43.550 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-06-10T00:57:43.550 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 150.0, 2024-06-10T00:57:43.550 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-06-10T00:57:43.550 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-06-10T00:57:43.551 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-06-10T00:57:43.551 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T00:57:43.551 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-06-10T00:57:43.551 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T00:57:43.551 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T00:57:43.551 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T00:57:43.551 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T00:57:43.551 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T00:57:43.551 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T00:57:43.551 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T00:57:43.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.548+0000 7ff4c60ee700 1 -- 172.21.15.37:0/3758692664 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff4a80a2e60 msgr2=0x7ff4a80a5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:43.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.548+0000 7ff4c60ee700 1 --2- 172.21.15.37:0/3758692664 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff4a80a2e60 0x7ff4a80a5320 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0005950 tx=0x7ff4b00058e0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.548+0000 7ff4c60ee700 1 -- 172.21.15.37:0/3758692664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4c01027a0 msgr2=0x7ff4c0073f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:43.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.548+0000 7ff4c60ee700 1 --2- 172.21.15.37:0/3758692664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4c01027a0 0x7ff4c0073f10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400d9b0 tx=0x7ff4b400dd70 comp rx=0 tx=0).stop 2024-06-10T00:57:43.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.549+0000 7ff4c60ee700 1 -- 172.21.15.37:0/3758692664 shutdown_connections 2024-06-10T00:57:43.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.549+0000 7ff4c60ee700 1 --2- 172.21.15.37:0/3758692664 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff4a80a2e60 0x7ff4a80a5320 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.549+0000 7ff4c60ee700 1 --2- 172.21.15.37:0/3758692664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4c0106750 0x7ff4c0111e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.549+0000 7ff4c60ee700 1 --2- 172.21.15.37:0/3758692664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4c0101df0 0x7ff4c0113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.549+0000 7ff4c60ee700 1 --2- 172.21.15.37:0/3758692664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4c01027a0 0x7ff4c0073f10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:43.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.549+0000 7ff4c60ee700 1 -- 172.21.15.37:0/3758692664 >> 172.21.15.37:0/3758692664 conn(0x7ff4c0077a30 msgr2=0x7ff4c01079d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:43.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.549+0000 7ff4c60ee700 1 -- 172.21.15.37:0/3758692664 shutdown_connections 2024-06-10T00:57:43.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:43.549+0000 7ff4c60ee700 1 -- 172.21.15.37:0/3758692664 wait complete. 2024-06-10T00:57:43.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:43 smithi167 ceph-mon[36917]: scrub queued for path: / 2024-06-10T00:57:43.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:43 smithi167 ceph-mon[36917]: scrub summary: idle+waiting paths [/] 2024-06-10T00:57:43.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:43 smithi167 ceph-mon[36917]: scrub summary: active paths [/] 2024-06-10T00:57:43.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:43 smithi167 ceph-mon[36917]: scrub summary: idle+waiting paths [/] 2024-06-10T00:57:43.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:43 smithi094 ceph-mon[36708]: scrub queued for path: / 2024-06-10T00:57:43.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:43 smithi094 ceph-mon[36708]: scrub summary: idle+waiting paths [/] 2024-06-10T00:57:43.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:43 smithi094 ceph-mon[36708]: scrub summary: active paths [/] 2024-06-10T00:57:43.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:43 smithi094 ceph-mon[36708]: scrub summary: idle+waiting paths [/] 2024-06-10T00:57:44.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:44 smithi037 ceph-mon[32192]: pgmap v461: 129 pgs: 129 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-06-10T00:57:44.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:44 smithi167 ceph-mon[36917]: pgmap v461: 129 pgs: 129 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-06-10T00:57:44.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:44 smithi094 ceph-mon[36708]: pgmap v461: 129 pgs: 129 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-06-10T00:57:44.719 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f993ef51', quiesce took 8.9 seconds. Will release after: 21.1 2024-06-10T00:57:46.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.205+0000 7f247df6a700 1 -- 172.21.15.37:0/284426812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2478102530 msgr2=0x7f24781029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:46.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.205+0000 7f247df6a700 1 --2- 172.21.15.37:0/284426812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2478102530 0x7f24781029b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f246c0099c0 tx=0x7f246c009cd0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.205+0000 7f247df6a700 1 -- 172.21.15.37:0/284426812 shutdown_connections 2024-06-10T00:57:46.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.205+0000 7f247df6a700 1 --2- 172.21.15.37:0/284426812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2478102ef0 0x7f247810f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.205+0000 7f247df6a700 1 --2- 172.21.15.37:0/284426812 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24781085d0 0x7f24781089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.205+0000 7f247df6a700 1 --2- 172.21.15.37:0/284426812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2478102530 0x7f24781029b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.205+0000 7f247df6a700 1 -- 172.21.15.37:0/284426812 >> 172.21.15.37:0/284426812 conn(0x7f2478079640 msgr2=0x7f2478079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:46.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.206+0000 7f247df6a700 1 -- 172.21.15.37:0/284426812 shutdown_connections 2024-06-10T00:57:46.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.206+0000 7f247df6a700 1 -- 172.21.15.37:0/284426812 wait complete. 2024-06-10T00:57:46.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.207+0000 7f247df6a700 1 Processor -- start 2024-06-10T00:57:46.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.207+0000 7f247df6a700 1 -- start start 2024-06-10T00:57:46.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.208+0000 7f247df6a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2478102530 0x7f2478074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:46.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.208+0000 7f247df6a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2478102ef0 0x7f2478074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:46.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.208+0000 7f247df6a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24781085d0 0x7f2478075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:46.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.208+0000 7f247df6a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2478112860 con 0x7f2478102530 2024-06-10T00:57:46.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.208+0000 7f247df6a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f24781126e0 con 0x7f2478102ef0 2024-06-10T00:57:46.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.208+0000 7f247df6a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f24781129e0 con 0x7f24781085d0 2024-06-10T00:57:46.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.208+0000 7f2476ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2478102ef0 0x7f2478074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:46.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.208+0000 7f2477fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24781085d0 0x7f2478075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:46.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.208+0000 7f2476ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2478102ef0 0x7f2478074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:38024/0 (socket says 172.21.15.37:38024) 2024-06-10T00:57:46.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.208+0000 7f2476ffd700 1 -- 172.21.15.37:0/3030003863 learned_addr learned my addr 172.21.15.37:0/3030003863 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:46.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.208+0000 7f24777fe700 1 --2- 172.21.15.37:0/3030003863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2478102530 0x7f2478074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:46.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.209+0000 7f2476ffd700 1 -- 172.21.15.37:0/3030003863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24781085d0 msgr2=0x7f2478075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:46.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.209+0000 7f2476ffd700 1 --2- 172.21.15.37:0/3030003863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24781085d0 0x7f2478075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.209+0000 7f2476ffd700 1 -- 172.21.15.37:0/3030003863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2478102530 msgr2=0x7f2478074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:46.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.209+0000 7f2476ffd700 1 --2- 172.21.15.37:0/3030003863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2478102530 0x7f2478074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.209+0000 7f2476ffd700 1 -- 172.21.15.37:0/3030003863 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24781a71a0 con 0x7f2478102ef0 2024-06-10T00:57:46.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.209+0000 7f2477fff700 1 --2- 172.21.15.37:0/3030003863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24781085d0 0x7f2478075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:57:46.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.209+0000 7f24777fe700 1 --2- 172.21.15.37:0/3030003863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2478102530 0x7f2478074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:57:46.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.209+0000 7f2476ffd700 1 --2- 172.21.15.37:0/3030003863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2478102ef0 0x7f2478074cd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f246c005910 tx=0x7f246c00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:46.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.209+0000 7f2474ff9700 1 -- 172.21.15.37:0/3030003863 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c027070 con 0x7f2478102ef0 2024-06-10T00:57:46.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.209+0000 7f2474ff9700 1 -- 172.21.15.37:0/3030003863 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f246c003680 con 0x7f2478102ef0 2024-06-10T00:57:46.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.209+0000 7f247df6a700 1 -- 172.21.15.37:0/3030003863 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24781989b0 con 0x7f2478102ef0 2024-06-10T00:57:46.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.209+0000 7f247df6a700 1 -- 172.21.15.37:0/3030003863 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2478198e20 con 0x7f2478102ef0 2024-06-10T00:57:46.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.210+0000 7f2474ff9700 1 -- 172.21.15.37:0/3030003863 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c0173f0 con 0x7f2478102ef0 2024-06-10T00:57:46.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.213+0000 7f2474ff9700 1 -- 172.21.15.37:0/3030003863 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f246c02a070 con 0x7f2478102ef0 2024-06-10T00:57:46.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.214+0000 7f2474ff9700 1 --2- 172.21.15.37:0/3030003863 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f24600a2db0 0x7f24600a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:46.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.214+0000 7f2474ff9700 1 -- 172.21.15.37:0/3030003863 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f246c0d00e0 con 0x7f2478102ef0 2024-06-10T00:57:46.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.214+0000 7f24777fe700 1 --2- 172.21.15.37:0/3030003863 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f24600a2db0 0x7f24600a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:46.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.214+0000 7f24887edb80 1 Processor -- start 2024-06-10T00:57:46.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.215+0000 7f24777fe700 1 --2- 172.21.15.37:0/3030003863 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f24600a2db0 0x7f24600a5270 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2468005950 tx=0x7f246800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:46.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.215+0000 7f24887edb80 1 -- start start 2024-06-10T00:57:46.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.215+0000 7f24887edb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0e5d64640 0x55c0e5d64a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:46.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.215+0000 7f24887edb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0e5d64f60 0x55c0e5d5e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:46.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.215+0000 7f24887edb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0e5d5ef20 0x55c0e5d5f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:46.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.215+0000 7f24887edb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c0e5cb0810 con 0x55c0e5d64640 2024-06-10T00:57:46.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.215+0000 7f24887edb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c0e5c58720 con 0x55c0e5d64f60 2024-06-10T00:57:46.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.215+0000 7f24887edb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c0e5cac7d0 con 0x55c0e5d5ef20 2024-06-10T00:57:46.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.215+0000 7f24777fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0e5d64640 0x55c0e5d64a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:46.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.215+0000 7f24777fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0e5d64640 0x55c0e5d64a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35546/0 (socket says 172.21.15.37:35546) 2024-06-10T00:57:46.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.215+0000 7f24777fe700 1 -- 172.21.15.37:0/2063258480 learned_addr learned my addr 172.21.15.37:0/2063258480 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:46.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.215+0000 7f2476ffd700 1 --2- 172.21.15.37:0/2063258480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0e5d64f60 0x55c0e5d5e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:46.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.216+0000 7f2477fff700 1 --2- 172.21.15.37:0/2063258480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0e5d5ef20 0x55c0e5d5f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:46.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.216+0000 7f2476ffd700 1 -- 172.21.15.37:0/2063258480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0e5d5ef20 msgr2=0x55c0e5d5f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:46.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.216+0000 7f2476ffd700 1 --2- 172.21.15.37:0/2063258480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0e5d5ef20 0x55c0e5d5f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.216+0000 7f2476ffd700 1 -- 172.21.15.37:0/2063258480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0e5d64640 msgr2=0x55c0e5d64a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:46.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.216+0000 7f2476ffd700 1 --2- 172.21.15.37:0/2063258480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0e5d64640 0x55c0e5d64a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.216+0000 7f2476ffd700 1 -- 172.21.15.37:0/2063258480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0e5c4ef50 con 0x55c0e5d64f60 2024-06-10T00:57:46.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.216+0000 7f24777fe700 1 --2- 172.21.15.37:0/2063258480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0e5d64640 0x55c0e5d64a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:57:46.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.216+0000 7f2476ffd700 1 --2- 172.21.15.37:0/2063258480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0e5d64f60 0x55c0e5d5e9e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f246c0cd640 tx=0x7f246c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48322ffd0d11a6c9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:46.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.216+0000 7f245dffb700 1 -- 172.21.15.37:0/2063258480 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c09cb20 con 0x55c0e5d64f60 2024-06-10T00:57:46.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.216+0000 7f245dffb700 1 -- 172.21.15.37:0/2063258480 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f246c09c420 con 0x55c0e5d64f60 2024-06-10T00:57:46.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.216+0000 7f245dffb700 1 -- 172.21.15.37:0/2063258480 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c0a4dd0 con 0x55c0e5d64f60 2024-06-10T00:57:46.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.217+0000 7f24887edb80 1 -- 172.21.15.37:0/2063258480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0e5d64f60 msgr2=0x55c0e5d5e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:46.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.217+0000 7f24887edb80 1 --2- 172.21.15.37:0/2063258480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0e5d64f60 0x55c0e5d5e9e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f246c0cd640 tx=0x7f246c009310 comp rx=0 tx=0).stop 2024-06-10T00:57:46.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.217+0000 7f24887edb80 1 -- 172.21.15.37:0/2063258480 shutdown_connections 2024-06-10T00:57:46.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.217+0000 7f24887edb80 1 --2- 172.21.15.37:0/2063258480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0e5d5ef20 0x55c0e5d5f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.217+0000 7f24887edb80 1 --2- 172.21.15.37:0/2063258480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0e5d64640 0x55c0e5d64a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.217+0000 7f24887edb80 1 --2- 172.21.15.37:0/2063258480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0e5d64f60 0x55c0e5d5e9e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.217+0000 7f24887edb80 1 -- 172.21.15.37:0/2063258480 >> 172.21.15.37:0/2063258480 conn(0x55c0e5c89cc0 msgr2=0x55c0e5d59850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:46.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.217+0000 7f24887edb80 1 -- 172.21.15.37:0/2063258480 shutdown_connections 2024-06-10T00:57:46.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.217+0000 7f24887edb80 1 -- 172.21.15.37:0/2063258480 wait complete. 2024-06-10T00:57:46.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.218+0000 7f24887edb80 1 -- start start 2024-06-10T00:57:46.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.218+0000 7f24887edb80 1 Processor -- start 2024-06-10T00:57:46.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.218+0000 7f24887edb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0e5d5ef20 0x55c0e5d6ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:46.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.218+0000 7f24887edb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0e5d64640 0x55c0e5d68030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:46.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.219+0000 7f24887edb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0e5d64f60 0x55c0e5d68570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:46.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.219+0000 7f24887edb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c0e5d6b6f0 con 0x55c0e5d64f60 2024-06-10T00:57:46.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.219+0000 7f24887edb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c0e5c50bf0 con 0x55c0e5d64640 2024-06-10T00:57:46.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.219+0000 7f24887edb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c0e5d6b870 con 0x55c0e5d5ef20 2024-06-10T00:57:46.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.219+0000 7f24887edb80 10 client.0 _ll_register_callbacks cb 0x55c0e5c4a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:46.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.219+0000 7f2476ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0e5d64640 0x55c0e5d68030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:46.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.219+0000 7f2477fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0e5d64f60 0x55c0e5d68570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:46.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.219+0000 7f2476ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0e5d64640 0x55c0e5d68030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:38050/0 (socket says 172.21.15.37:38050) 2024-06-10T00:57:46.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.219+0000 7f2476ffd700 1 -- 172.21.15.37:0/39519260 learned_addr learned my addr 172.21.15.37:0/39519260 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:46.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.219+0000 7f24777fe700 1 --2- 172.21.15.37:0/39519260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0e5d5ef20 0x55c0e5d6ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:46.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.219+0000 7f2476ffd700 1 -- 172.21.15.37:0/39519260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0e5d5ef20 msgr2=0x55c0e5d6ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:46.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.219+0000 7f2476ffd700 1 --2- 172.21.15.37:0/39519260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0e5d5ef20 0x55c0e5d6ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.219+0000 7f2476ffd700 1 -- 172.21.15.37:0/39519260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0e5d64f60 msgr2=0x55c0e5d68570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:46.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.219+0000 7f2476ffd700 1 --2- 172.21.15.37:0/39519260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0e5d64f60 0x55c0e5d68570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.219+0000 7f2476ffd700 1 -- 172.21.15.37:0/39519260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0e5bff8d0 con 0x55c0e5d64640 2024-06-10T00:57:46.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.219+0000 7f2476ffd700 1 --2- 172.21.15.37:0/39519260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0e5d64640 0x55c0e5d68030 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f246c009990 tx=0x7f246c0a4da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7dd7d72fd3ae608 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:46.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.219+0000 7f2456ffd700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T00:57:46.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.220+0000 7f2456ffd700 1 -- 172.21.15.37:0/39519260 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c09cb20 con 0x55c0e5d64640 2024-06-10T00:57:46.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.220+0000 7f2456ffd700 1 -- 172.21.15.37:0/39519260 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f246c09ba80 con 0x55c0e5d64640 2024-06-10T00:57:46.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.220+0000 7f24887edb80 1 -- 172.21.15.37:0/39519260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f246c09c420 con 0x55c0e5d64640 2024-06-10T00:57:46.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.220+0000 7f2456ffd700 1 -- 172.21.15.37:0/39519260 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f246c0a5000 con 0x55c0e5d64640 2024-06-10T00:57:46.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.220+0000 7f24887edb80 10 client.25226 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:46.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.220+0000 7f24887edb80 1 -- 172.21.15.37:0/39519260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c0e5d6b6f0 con 0x55c0e5d64640 2024-06-10T00:57:46.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.220+0000 7f2456ffd700 1 -- 172.21.15.37:0/39519260 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f246c09c420 con 0x55c0e5d64640 2024-06-10T00:57:46.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.220+0000 7f2456ffd700 1 -- 172.21.15.37:0/39519260 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f246c09b2e0 con 0x55c0e5d64640 2024-06-10T00:57:46.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.221+0000 7f24887edb80 10 client.25226 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:46.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.221+0000 7f24887edb80 10 client.25226 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T00:57:46.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.221+0000 7f24887edb80 20 client.25226 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:46.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.221+0000 7f24887edb80 1 --2- 172.21.15.37:0/39519260 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c0e5d78320 0x55c0e5d7a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:46.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.221+0000 7f24887edb80 4 client.25226 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:57:46.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.221+0000 7f24887edb80 1 -- 172.21.15.37:0/39519260 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0e5cbdd10 con 0x55c0e5d78320 2024-06-10T00:57:46.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.221+0000 7f24777fe700 1 --2- 172.21.15.37:0/39519260 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c0e5d78320 0x55c0e5d7a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:46.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.222+0000 7f2456ffd700 10 client.25226 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:46.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.222+0000 7f24777fe700 1 --2- 172.21.15.37:0/39519260 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c0e5d78320 0x55c0e5d7a710 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c61a6fa8caecf73 server_cookie=206117d98b45d5e2 in_seq=0 out_seq=0 2024-06-10T00:57:46.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.222+0000 7f2456ffd700 1 -- 172.21.15.37:0/39519260 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c0e5cbdd10 con 0x55c0e5d78320 2024-06-10T00:57:46.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.222+0000 7f2456ffd700 10 client.25226 handle_command_reply: tid=0 2024-06-10T00:57:46.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.222+0000 7f24887edb80 1 client.25226 shutdown 2024-06-10T00:57:46.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.223+0000 7f24887edb80 1 -- 172.21.15.37:0/39519260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0e5d64640 msgr2=0x55c0e5d68030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:46.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.223+0000 7f24887edb80 1 --2- 172.21.15.37:0/39519260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0e5d64640 0x55c0e5d68030 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f246c009990 tx=0x7f246c0a4da0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.223+0000 7f24887edb80 1 -- 172.21.15.37:0/39519260 shutdown_connections 2024-06-10T00:57:46.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.223+0000 7f24887edb80 1 --2- 172.21.15.37:0/39519260 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c0e5d78320 0x55c0e5d7a710 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.223+0000 7f24887edb80 1 --2- 172.21.15.37:0/39519260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0e5d64f60 0x55c0e5d68570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.223+0000 7f24887edb80 1 --2- 172.21.15.37:0/39519260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0e5d5ef20 0x55c0e5d6ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.223+0000 7f24887edb80 1 --2- 172.21.15.37:0/39519260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0e5d64640 0x55c0e5d68030 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.223+0000 7f24887edb80 1 -- 172.21.15.37:0/39519260 >> 172.21.15.37:0/39519260 conn(0x55c0e5c89cc0 msgr2=0x55c0e5d595f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:46.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.223+0000 7f2456ffd700 0 client.25226 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:46.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.223+0000 7f24887edb80 1 -- 172.21.15.37:0/39519260 shutdown_connections 2024-06-10T00:57:46.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.223+0000 7f24887edb80 1 -- 172.21.15.37:0/39519260 wait complete. 2024-06-10T00:57:46.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.223+0000 7f24887edb80 20 client.25226 trim_cache size 0 max 16384 2024-06-10T00:57:46.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.260+0000 7f24887edb80 1 Processor -- start 2024-06-10T00:57:46.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.260+0000 7f24887edb80 1 -- start start 2024-06-10T00:57:46.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.260+0000 7f24887edb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0e5d5ef20 0x55c0e5d74990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:46.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.260+0000 7f24887edb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0e5d64640 0x55c0e5d5e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:46.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.260+0000 7f24887edb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0e5d64f60 0x55c0e5d65400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:46.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.260+0000 7f24887edb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f24781aeb40 con 0x55c0e5d64f60 2024-06-10T00:57:46.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.260+0000 7f24887edb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f24781ac3d0 con 0x55c0e5d64640 2024-06-10T00:57:46.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.260+0000 7f24887edb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f24781aa220 con 0x55c0e5d5ef20 2024-06-10T00:57:46.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.260+0000 7f2477fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0e5d64f60 0x55c0e5d65400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:46.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.260+0000 7f2477fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0e5d64f60 0x55c0e5d65400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35554/0 (socket says 172.21.15.37:35554) 2024-06-10T00:57:46.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.260+0000 7f2477fff700 1 -- 172.21.15.37:0/1399251723 learned_addr learned my addr 172.21.15.37:0/1399251723 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:46.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.260+0000 7f24777fe700 1 --2- 172.21.15.37:0/1399251723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0e5d5ef20 0x55c0e5d74990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:46.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.260+0000 7f2476ffd700 1 --2- 172.21.15.37:0/1399251723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0e5d64640 0x55c0e5d5e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:46.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.261+0000 7f2477fff700 1 -- 172.21.15.37:0/1399251723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0e5d5ef20 msgr2=0x55c0e5d74990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:46.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.261+0000 7f2477fff700 1 --2- 172.21.15.37:0/1399251723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0e5d5ef20 0x55c0e5d74990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.261+0000 7f2477fff700 1 -- 172.21.15.37:0/1399251723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0e5d64640 msgr2=0x55c0e5d5e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:46.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.261+0000 7f2477fff700 1 --2- 172.21.15.37:0/1399251723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0e5d64640 0x55c0e5d5e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.261+0000 7f2477fff700 1 -- 172.21.15.37:0/1399251723 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0e5d6b510 con 0x55c0e5d64f60 2024-06-10T00:57:46.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.261+0000 7f2477fff700 1 --2- 172.21.15.37:0/1399251723 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0e5d64f60 0x55c0e5d65400 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2470018820 tx=0x7f247000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1a9ac1a8e5724a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:46.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.261+0000 7f245d7fa700 1 -- 172.21.15.37:0/1399251723 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2470024070 con 0x55c0e5d64f60 2024-06-10T00:57:46.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.261+0000 7f245d7fa700 1 -- 172.21.15.37:0/1399251723 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2470004bb0 con 0x55c0e5d64f60 2024-06-10T00:57:46.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.261+0000 7f245d7fa700 1 -- 172.21.15.37:0/1399251723 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247001f6e0 con 0x55c0e5d64f60 2024-06-10T00:57:46.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.262+0000 7f24887edb80 1 -- 172.21.15.37:0/1399251723 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0e5d64f60 msgr2=0x55c0e5d65400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:46.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.262+0000 7f24887edb80 1 --2- 172.21.15.37:0/1399251723 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0e5d64f60 0x55c0e5d65400 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2470018820 tx=0x7f247000b040 comp rx=0 tx=0).stop 2024-06-10T00:57:46.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.262+0000 7f24887edb80 1 -- 172.21.15.37:0/1399251723 shutdown_connections 2024-06-10T00:57:46.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.262+0000 7f24887edb80 1 --2- 172.21.15.37:0/1399251723 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0e5d64f60 0x55c0e5d65400 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.262+0000 7f24887edb80 1 --2- 172.21.15.37:0/1399251723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0e5d5ef20 0x55c0e5d74990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.262+0000 7f24887edb80 1 --2- 172.21.15.37:0/1399251723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0e5d64640 0x55c0e5d5e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.262+0000 7f24887edb80 1 -- 172.21.15.37:0/1399251723 >> 172.21.15.37:0/1399251723 conn(0x55c0e5c89cc0 msgr2=0x55c0e5d66920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:46.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.262+0000 7f24887edb80 1 -- 172.21.15.37:0/1399251723 shutdown_connections 2024-06-10T00:57:46.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.262+0000 7f24887edb80 1 -- 172.21.15.37:0/1399251723 wait complete. 2024-06-10T00:57:46.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.263+0000 7f24887edb80 1 -- start start 2024-06-10T00:57:46.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.263+0000 7f24887edb80 1 Processor -- start 2024-06-10T00:57:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.263+0000 7f24887edb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0e5d5ef20 0x55c0e5d7fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.263+0000 7f24887edb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0e5d64640 0x55c0e5d7ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.263+0000 7f24887edb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0e5d64f60 0x55c0e5d804c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.263+0000 7f24887edb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f24781aa890 con 0x55c0e5d5ef20 2024-06-10T00:57:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.263+0000 7f24887edb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f24781abd00 con 0x55c0e5d64640 2024-06-10T00:57:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.263+0000 7f24887edb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f24781aaf60 con 0x55c0e5d64f60 2024-06-10T00:57:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.264+0000 7f24887edb80 10 client.0 _ll_register_callbacks cb 0x55c0e5c4a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.264+0000 7f2476ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0e5d64640 0x55c0e5d7ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.264+0000 7f2476ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0e5d64640 0x55c0e5d7ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:38070/0 (socket says 172.21.15.37:38070) 2024-06-10T00:57:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.264+0000 7f2476ffd700 1 -- 172.21.15.37:0/3651008096 learned_addr learned my addr 172.21.15.37:0/3651008096 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.264+0000 7f2477fff700 1 --2- 172.21.15.37:0/3651008096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0e5d64f60 0x55c0e5d804c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.264+0000 7f2476ffd700 1 -- 172.21.15.37:0/3651008096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0e5d64f60 msgr2=0x55c0e5d804c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.264+0000 7f2476ffd700 1 --2- 172.21.15.37:0/3651008096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0e5d64f60 0x55c0e5d804c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.264+0000 7f2476ffd700 1 -- 172.21.15.37:0/3651008096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0e5d5ef20 msgr2=0x55c0e5d7fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:46.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.264+0000 7f2476ffd700 1 --2- 172.21.15.37:0/3651008096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0e5d5ef20 0x55c0e5d7fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.264+0000 7f2476ffd700 1 -- 172.21.15.37:0/3651008096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0e5de1ef0 con 0x55c0e5d64640 2024-06-10T00:57:46.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.264+0000 7f2476ffd700 1 --2- 172.21.15.37:0/3651008096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0e5d64640 0x55c0e5d7ff80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f246c09c0d0 tx=0x7f246c005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83be65ee8334d51f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:46.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.264+0000 7f2456ffd700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T00:57:46.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.264+0000 7f2456ffd700 1 -- 172.21.15.37:0/3651008096 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c09cb20 con 0x55c0e5d64640 2024-06-10T00:57:46.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.264+0000 7f2456ffd700 1 -- 172.21.15.37:0/3651008096 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f246c0cd580 con 0x55c0e5d64640 2024-06-10T00:57:46.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.264+0000 7f24887edb80 1 -- 172.21.15.37:0/3651008096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2470004bb0 con 0x55c0e5d64640 2024-06-10T00:57:46.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.265+0000 7f2456ffd700 1 -- 172.21.15.37:0/3651008096 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f246c09c420 con 0x55c0e5d64640 2024-06-10T00:57:46.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.265+0000 7f24887edb80 10 client.25232 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:46.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.265+0000 7f24887edb80 1 -- 172.21.15.37:0/3651008096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c0e5de1ef0 con 0x55c0e5d64640 2024-06-10T00:57:46.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.265+0000 7f2456ffd700 1 -- 172.21.15.37:0/3651008096 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2470004bb0 con 0x55c0e5d64640 2024-06-10T00:57:46.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.265+0000 7f2456ffd700 1 -- 172.21.15.37:0/3651008096 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f246c098170 con 0x55c0e5d64640 2024-06-10T00:57:46.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.265+0000 7f24887edb80 10 client.25232 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:46.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.265+0000 7f24887edb80 10 client.25232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T00:57:46.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.265+0000 7f24887edb80 20 client.25232 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:46.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.265+0000 7f24887edb80 1 --2- 172.21.15.37:0/3651008096 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c0e5de3400 0x55c0e5de5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:46.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.265+0000 7f24887edb80 4 client.25232 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T00:57:46.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.265+0000 7f24887edb80 1 -- 172.21.15.37:0/3651008096 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c0e5cb8290 con 0x55c0e5de3400 2024-06-10T00:57:46.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.266+0000 7f24777fe700 1 --2- 172.21.15.37:0/3651008096 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c0e5de3400 0x55c0e5de5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:46.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.266+0000 7f24777fe700 1 --2- 172.21.15.37:0/3651008096 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c0e5de3400 0x55c0e5de5820 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=652fa37a206e5f4d server_cookie=43fccea184d1ab85 in_seq=0 out_seq=0 2024-06-10T00:57:46.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.266+0000 7f2456ffd700 10 client.25232 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:46.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.266+0000 7f2456ffd700 1 -- 172.21.15.37:0/3651008096 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c0e5cb8290 con 0x55c0e5de3400 2024-06-10T00:57:46.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.266+0000 7f2456ffd700 10 client.25232 handle_command_reply: tid=0 2024-06-10T00:57:46.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.266+0000 7f24887edb80 1 client.25232 shutdown 2024-06-10T00:57:46.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.267+0000 7f24887edb80 1 -- 172.21.15.37:0/3651008096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0e5d64640 msgr2=0x55c0e5d7ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:46.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.267+0000 7f24887edb80 1 --2- 172.21.15.37:0/3651008096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0e5d64640 0x55c0e5d7ff80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f246c09c0d0 tx=0x7f246c005e80 comp rx=0 tx=0).stop 2024-06-10T00:57:46.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.267+0000 7f24887edb80 1 -- 172.21.15.37:0/3651008096 shutdown_connections 2024-06-10T00:57:46.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.267+0000 7f24887edb80 1 --2- 172.21.15.37:0/3651008096 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c0e5de3400 0x55c0e5de5820 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.267+0000 7f2456ffd700 0 client.25232 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:46.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.267+0000 7f24887edb80 1 --2- 172.21.15.37:0/3651008096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0e5d64f60 0x55c0e5d804c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.267+0000 7f24887edb80 1 --2- 172.21.15.37:0/3651008096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0e5d5ef20 0x55c0e5d7fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.267+0000 7f24887edb80 1 --2- 172.21.15.37:0/3651008096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0e5d64640 0x55c0e5d7ff80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.267+0000 7f24887edb80 1 -- 172.21.15.37:0/3651008096 >> 172.21.15.37:0/3651008096 conn(0x55c0e5c89cc0 msgr2=0x55c0e5d66670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:46.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.267+0000 7f24887edb80 1 -- 172.21.15.37:0/3651008096 shutdown_connections 2024-06-10T00:57:46.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.267+0000 7f24887edb80 1 -- 172.21.15.37:0/3651008096 wait complete. 2024-06-10T00:57:46.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.267+0000 7f24887edb80 20 client.25232 trim_cache size 0 max 16384 2024-06-10T00:57:46.271 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T00:57:46.271 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-06-10T00:57:46.271 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-06-10T00:57:46.271 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T00:57:46.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.268+0000 7f247df6a700 1 -- 172.21.15.37:0/3030003863 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f24600a2db0 msgr2=0x7f24600a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:46.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.268+0000 7f247df6a700 1 --2- 172.21.15.37:0/3030003863 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f24600a2db0 0x7f24600a5270 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2468005950 tx=0x7f246800b410 comp rx=0 tx=0).stop 2024-06-10T00:57:46.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.268+0000 7f247df6a700 1 -- 172.21.15.37:0/3030003863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2478102ef0 msgr2=0x7f2478074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:46.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.268+0000 7f247df6a700 1 --2- 172.21.15.37:0/3030003863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2478102ef0 0x7f2478074cd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f246c005910 tx=0x7f246c00fa00 comp rx=0 tx=0).stop 2024-06-10T00:57:46.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.268+0000 7f247df6a700 1 -- 172.21.15.37:0/3030003863 shutdown_connections 2024-06-10T00:57:46.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.268+0000 7f247df6a700 1 --2- 172.21.15.37:0/3030003863 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f24600a2db0 0x7f24600a5270 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.268+0000 7f247df6a700 1 --2- 172.21.15.37:0/3030003863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24781085d0 0x7f2478075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.268+0000 7f247df6a700 1 --2- 172.21.15.37:0/3030003863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2478102530 0x7f2478074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.268+0000 7f247df6a700 1 --2- 172.21.15.37:0/3030003863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2478102ef0 0x7f2478074cd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:46.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.268+0000 7f247df6a700 1 -- 172.21.15.37:0/3030003863 >> 172.21.15.37:0/3030003863 conn(0x7f2478079640 msgr2=0x7f2478101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:46.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.268+0000 7f247df6a700 1 -- 172.21.15.37:0/3030003863 shutdown_connections 2024-06-10T00:57:46.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:46.268+0000 7f247df6a700 1 -- 172.21.15.37:0/3030003863 wait complete. 2024-06-10T00:57:46.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:46 smithi037 ceph-mon[32192]: pgmap v462: 129 pgs: 129 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-06-10T00:57:46.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:46 smithi037 ceph-mon[32192]: scrub summary: idle 2024-06-10T00:57:46.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:46 smithi167 ceph-mon[36917]: pgmap v462: 129 pgs: 129 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-06-10T00:57:46.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:46 smithi167 ceph-mon[36917]: scrub summary: idle 2024-06-10T00:57:46.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:46 smithi094 ceph-mon[36708]: pgmap v462: 129 pgs: 129 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-06-10T00:57:46.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:46 smithi094 ceph-mon[36708]: scrub summary: idle 2024-06-10T00:57:47.081 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T00:57:47.081 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:57:47.470 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:57:48.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:48 smithi167 ceph-mon[36917]: pgmap v463: 129 pgs: 129 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-06-10T00:57:48.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:48 smithi094 ceph-mon[36708]: pgmap v463: 129 pgs: 129 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-06-10T00:57:48.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:48 smithi037 ceph-mon[32192]: pgmap v463: 129 pgs: 129 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-06-10T00:57:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.982+0000 7fb8b9a38640 1 -- 172.21.15.37:0/1260940801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8b4100900 msgr2=0x7fb8b4100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.982+0000 7fb8b9a38640 1 --2- 172.21.15.37:0/1260940801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8b4100900 0x7fb8b4100d20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4009950 tx=0x7fb8a402a480 comp rx=0 tx=0).stop 2024-06-10T00:57:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.982+0000 7fb8b9a38640 1 -- 172.21.15.37:0/1260940801 shutdown_connections 2024-06-10T00:57:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.982+0000 7fb8b9a38640 1 --2- 172.21.15.37:0/1260940801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8b41024b0 0x7fb8b4108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.982+0000 7fb8b9a38640 1 --2- 172.21.15.37:0/1260940801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8b4101af0 0x7fb8b4101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.982+0000 7fb8b9a38640 1 --2- 172.21.15.37:0/1260940801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8b4100900 0x7fb8b4100d20 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.982+0000 7fb8b9a38640 1 -- 172.21.15.37:0/1260940801 >> 172.21.15.37:0/1260940801 conn(0x7fb8b40fbe70 msgr2=0x7fb8b40fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.982+0000 7fb8b9a38640 1 -- 172.21.15.37:0/1260940801 shutdown_connections 2024-06-10T00:57:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.982+0000 7fb8b9a38640 1 -- 172.21.15.37:0/1260940801 wait complete. 2024-06-10T00:57:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.983+0000 7fb8b9a38640 1 Processor -- start 2024-06-10T00:57:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.983+0000 7fb8b9a38640 1 -- start start 2024-06-10T00:57:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.983+0000 7fb8b9a38640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8b4100900 0x7fb8b4069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.983+0000 7fb8b9a38640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8b4101af0 0x7fb8b419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.983+0000 7fb8b27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8b4101af0 0x7fb8b419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.983+0000 7fb8b27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8b4101af0 0x7fb8b419efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35592/0 (socket says 172.21.15.37:35592) 2024-06-10T00:57:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.984+0000 7fb8b27fc640 1 -- 172.21.15.37:0/3596683665 learned_addr learned my addr 172.21.15.37:0/3596683665 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.984+0000 7fb8b2ffd640 1 --2- 172.21.15.37:0/3596683665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8b4100900 0x7fb8b4069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.984+0000 7fb8b9a38640 1 --2- 172.21.15.37:0/3596683665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8b41024b0 0x7fb8b41a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.984+0000 7fb8b9a38640 1 -- 172.21.15.37:0/3596683665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b410c6c0 con 0x7fb8b4101af0 2024-06-10T00:57:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.984+0000 7fb8b9a38640 1 -- 172.21.15.37:0/3596683665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b410c540 con 0x7fb8b41024b0 2024-06-10T00:57:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.984+0000 7fb8b9a38640 1 -- 172.21.15.37:0/3596683665 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b410c840 con 0x7fb8b4100900 2024-06-10T00:57:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.984+0000 7fb8b37fe640 1 --2- 172.21.15.37:0/3596683665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8b41024b0 0x7fb8b41a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.984+0000 7fb8b27fc640 1 -- 172.21.15.37:0/3596683665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8b4100900 msgr2=0x7fb8b4069b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.984+0000 7fb8b27fc640 1 --2- 172.21.15.37:0/3596683665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8b4100900 0x7fb8b4069b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.984+0000 7fb8b27fc640 1 -- 172.21.15.37:0/3596683665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8b41024b0 msgr2=0x7fb8b41a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.984+0000 7fb8b27fc640 1 --2- 172.21.15.37:0/3596683665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8b41024b0 0x7fb8b41a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.984+0000 7fb8b27fc640 1 -- 172.21.15.37:0/3596683665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b41a1c80 con 0x7fb8b4101af0 2024-06-10T00:57:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.984+0000 7fb8b2ffd640 1 --2- 172.21.15.37:0/3596683665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8b4100900 0x7fb8b4069b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:57:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.985+0000 7fb8b27fc640 1 --2- 172.21.15.37:0/3596683665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8b4101af0 0x7fb8b419efe0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800b700 tx=0x7fb8a800bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.985+0000 7fb8b8a36640 1 -- 172.21.15.37:0/3596683665 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8019070 con 0x7fb8b4101af0 2024-06-10T00:57:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.985+0000 7fb8b8a36640 1 -- 172.21.15.37:0/3596683665 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8a800ccc0 con 0x7fb8b4101af0 2024-06-10T00:57:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.985+0000 7fb8b9a38640 1 -- 172.21.15.37:0/3596683665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8b41a1f70 con 0x7fb8b4101af0 2024-06-10T00:57:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.987+0000 7fb8b8a36640 1 -- 172.21.15.37:0/3596683665 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8012340 con 0x7fb8b4101af0 2024-06-10T00:57:48.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.987+0000 7fb8b9a38640 1 -- 172.21.15.37:0/3596683665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8b41a2500 con 0x7fb8b4101af0 2024-06-10T00:57:48.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.989+0000 7fb8b8a36640 1 -- 172.21.15.37:0/3596683665 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7fb8a801c070 con 0x7fb8b4101af0 2024-06-10T00:57:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.990+0000 7fb8b8a36640 1 --2- 172.21.15.37:0/3596683665 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb89c0a2c20 0x7fb89c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.990+0000 7fb8b2ffd640 1 --2- 172.21.15.37:0/3596683665 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb89c0a2c20 0x7fb89c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.990+0000 7fb8b8a36640 1 -- 172.21.15.37:0/3596683665 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fb8a80d01c0 con 0x7fb8b4101af0 2024-06-10T00:57:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.990+0000 7fb8bc823740 1 Processor -- start 2024-06-10T00:57:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.990+0000 7fb8bc823740 1 -- start start 2024-06-10T00:57:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.990+0000 7fb8bc823740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5573f6241010 0x5573f6241430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.991+0000 7fb8bc823740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f6241a00 0x5573f6246500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.991+0000 7fb8bc823740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f6246b70 0x5573f6249000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:48.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.991+0000 7fb8bc823740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5573f61a44f0 con 0x5573f6241a00 2024-06-10T00:57:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.991+0000 7fb8bc823740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5573f6142870 con 0x5573f6246b70 2024-06-10T00:57:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.991+0000 7fb8bc823740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5573f61a04b0 con 0x5573f6241010 2024-06-10T00:57:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.991+0000 7fb8b2ffd640 1 --2- 172.21.15.37:0/3596683665 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb89c0a2c20 0x7fb89c0a50e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4006010 tx=0x7fb8a4000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.991+0000 7fb8b27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f6241a00 0x5573f6246500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.991+0000 7fb8b27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f6241a00 0x5573f6246500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35604/0 (socket says 172.21.15.37:35604) 2024-06-10T00:57:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.991+0000 7fb8b27fc640 1 -- 172.21.15.37:0/2859960648 learned_addr learned my addr 172.21.15.37:0/2859960648 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.991+0000 7fb8b2ffd640 1 --2- 172.21.15.37:0/2859960648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5573f6241010 0x5573f6241430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.991+0000 7fb8b37fe640 1 --2- 172.21.15.37:0/2859960648 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f6246b70 0x5573f6249000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.991+0000 7fb8b27fc640 1 -- 172.21.15.37:0/2859960648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5573f6241010 msgr2=0x5573f6241430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.991+0000 7fb8b27fc640 1 --2- 172.21.15.37:0/2859960648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5573f6241010 0x5573f6241430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.991+0000 7fb8b27fc640 1 -- 172.21.15.37:0/2859960648 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f6246b70 msgr2=0x5573f6249000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.991+0000 7fb8b27fc640 1 --2- 172.21.15.37:0/2859960648 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f6246b70 0x5573f6249000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.991+0000 7fb8b27fc640 1 -- 172.21.15.37:0/2859960648 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573f615e9b0 con 0x5573f6241a00 2024-06-10T00:57:48.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.991+0000 7fb8b37fe640 1 --2- 172.21.15.37:0/2859960648 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f6246b70 0x5573f6249000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:57:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.991+0000 7fb8b27fc640 1 --2- 172.21.15.37:0/2859960648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f6241a00 0x5573f6246500 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800bcf0 tx=0x7fb8a80047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82578673771c4921 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.992+0000 7fb899ffb640 1 -- 172.21.15.37:0/2859960648 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a80b3070 con 0x5573f6241a00 2024-06-10T00:57:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.992+0000 7fb899ffb640 1 -- 172.21.15.37:0/2859960648 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8a80ba040 con 0x5573f6241a00 2024-06-10T00:57:48.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.992+0000 7fb899ffb640 1 -- 172.21.15.37:0/2859960648 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a80d2060 con 0x5573f6241a00 2024-06-10T00:57:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.992+0000 7fb8bc823740 1 -- 172.21.15.37:0/2859960648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f6241a00 msgr2=0x5573f6246500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.992+0000 7fb8bc823740 1 --2- 172.21.15.37:0/2859960648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f6241a00 0x5573f6246500 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800bcf0 tx=0x7fb8a80047d0 comp rx=0 tx=0).stop 2024-06-10T00:57:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.992+0000 7fb8bc823740 1 -- 172.21.15.37:0/2859960648 shutdown_connections 2024-06-10T00:57:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.992+0000 7fb8bc823740 1 --2- 172.21.15.37:0/2859960648 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f6246b70 0x5573f6249000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.992+0000 7fb8bc823740 1 --2- 172.21.15.37:0/2859960648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f6241a00 0x5573f6246500 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.992+0000 7fb8bc823740 1 --2- 172.21.15.37:0/2859960648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5573f6241010 0x5573f6241430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.992+0000 7fb8bc823740 1 -- 172.21.15.37:0/2859960648 >> 172.21.15.37:0/2859960648 conn(0x5573f623c680 msgr2=0x5573f623ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.993+0000 7fb8bc823740 1 -- 172.21.15.37:0/2859960648 shutdown_connections 2024-06-10T00:57:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.993+0000 7fb8bc823740 1 -- 172.21.15.37:0/2859960648 wait complete. 2024-06-10T00:57:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.993+0000 7fb8bc823740 1 -- start start 2024-06-10T00:57:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.993+0000 7fb8bc823740 1 Processor -- start 2024-06-10T00:57:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.993+0000 7fb8bc823740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f6241010 0x5573f624b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.993+0000 7fb8bc823740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5573f6241a00 0x5573f624b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.994+0000 7fb8bc823740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f6246b70 0x5573f6252840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.994+0000 7fb8bc823740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5573f6249ac0 con 0x5573f6246b70 2024-06-10T00:57:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.994+0000 7fb8bc823740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5573f612dae0 con 0x5573f6241010 2024-06-10T00:57:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.994+0000 7fb8bc823740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5573f6249c40 con 0x5573f6241a00 2024-06-10T00:57:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.994+0000 7fb8bc823740 10 client.0 _ll_register_callbacks cb 0x5573f61669b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.994+0000 7fb8b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f6246b70 0x5573f6252840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.994+0000 7fb8b2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f6241010 0x5573f624b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.994+0000 7fb8b2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f6241010 0x5573f624b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:38112/0 (socket says 172.21.15.37:38112) 2024-06-10T00:57:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.994+0000 7fb8b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f6246b70 0x5573f6252840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35614/0 (socket says 172.21.15.37:35614) 2024-06-10T00:57:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.994+0000 7fb8b2ffd640 1 -- 172.21.15.37:0/1305495592 learned_addr learned my addr 172.21.15.37:0/1305495592 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.994+0000 7fb8b27fc640 1 --2- 172.21.15.37:0/1305495592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5573f6241a00 0x5573f624b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.994+0000 7fb8b2ffd640 1 -- 172.21.15.37:0/1305495592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5573f6241a00 msgr2=0x5573f624b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.994+0000 7fb8b2ffd640 1 --2- 172.21.15.37:0/1305495592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5573f6241a00 0x5573f624b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.994+0000 7fb8b2ffd640 1 -- 172.21.15.37:0/1305495592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f6246b70 msgr2=0x5573f6252840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.994+0000 7fb8b2ffd640 1 --2- 172.21.15.37:0/1305495592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f6246b70 0x5573f6252840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.994+0000 7fb8b2ffd640 1 -- 172.21.15.37:0/1305495592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573f5eb8210 con 0x5573f6241010 2024-06-10T00:57:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.994+0000 7fb8b27fc640 1 --2- 172.21.15.37:0/1305495592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5573f6241a00 0x5573f624b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:57:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.994+0000 7fb8b37fe640 1 --2- 172.21.15.37:0/1305495592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f6246b70 0x5573f6252840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:57:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.994+0000 7fb8b2ffd640 1 --2- 172.21.15.37:0/1305495592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f6241010 0x5573f624b2c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4009950 tx=0x7fb8a4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ed2b64adc31c602 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.994+0000 7fb892ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T00:57:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.995+0000 7fb892ffd640 1 -- 172.21.15.37:0/1305495592 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a404d020 con 0x5573f6241010 2024-06-10T00:57:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.995+0000 7fb8bc823740 1 -- 172.21.15.37:0/1305495592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8a80ba040 con 0x5573f6241010 2024-06-10T00:57:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.995+0000 7fb892ffd640 1 -- 172.21.15.37:0/1305495592 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fb8a40317b0 con 0x5573f6241010 2024-06-10T00:57:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.995+0000 7fb892ffd640 1 -- 172.21.15.37:0/1305495592 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb8a4058050 con 0x5573f6241010 2024-06-10T00:57:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.995+0000 7fb8bc823740 10 client.25241 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.995+0000 7fb8bc823740 1 -- 172.21.15.37:0/1305495592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5573f5f84620 con 0x5573f6241010 2024-06-10T00:57:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.995+0000 7fb892ffd640 1 -- 172.21.15.37:0/1305495592 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8a80ba040 con 0x5573f6241010 2024-06-10T00:57:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.996+0000 7fb892ffd640 1 -- 172.21.15.37:0/1305495592 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb8a403a040 con 0x5573f6241010 2024-06-10T00:57:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.996+0000 7fb8bc823740 10 client.25241 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:49.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.996+0000 7fb8bc823740 10 client.25241 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T00:57:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.996+0000 7fb8bc823740 20 client.25241 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.996+0000 7fb8bc823740 1 --2- 172.21.15.37:0/1305495592 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5573f6257130 0x5573f6259540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.996+0000 7fb8bc823740 4 client.25241 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:57:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.996+0000 7fb8bc823740 1 -- 172.21.15.37:0/1305495592 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573f61b7f10 con 0x5573f6257130 2024-06-10T00:57:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.996+0000 7fb8b27fc640 1 --2- 172.21.15.37:0/1305495592 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5573f6257130 0x5573f6259540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.996+0000 7fb892ffd640 10 client.25241 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.996+0000 7fb8b27fc640 1 --2- 172.21.15.37:0/1305495592 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5573f6257130 0x5573f6259540 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a43683405cee6621 server_cookie=3660accf3060dd8f in_seq=0 out_seq=0 2024-06-10T00:57:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.997+0000 7fb892ffd640 1 -- 172.21.15.37:0/1305495592 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5573f61b7f10 con 0x5573f6257130 2024-06-10T00:57:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.997+0000 7fb892ffd640 10 client.25241 handle_command_reply: tid=0 2024-06-10T00:57:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.997+0000 7fb8bc823740 1 client.25241 shutdown 2024-06-10T00:57:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.998+0000 7fb8bc823740 1 -- 172.21.15.37:0/1305495592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f6241010 msgr2=0x5573f624b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.998+0000 7fb8bc823740 1 --2- 172.21.15.37:0/1305495592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f6241010 0x5573f624b2c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4009950 tx=0x7fb8a4009260 comp rx=0 tx=0).stop 2024-06-10T00:57:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.998+0000 7fb8bc823740 1 -- 172.21.15.37:0/1305495592 shutdown_connections 2024-06-10T00:57:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.998+0000 7fb8bc823740 1 --2- 172.21.15.37:0/1305495592 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5573f6257130 0x5573f6259540 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.998+0000 7fb8bc823740 1 --2- 172.21.15.37:0/1305495592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f6246b70 0x5573f6252840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.998+0000 7fb8bc823740 1 --2- 172.21.15.37:0/1305495592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5573f6241a00 0x5573f624b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.998+0000 7fb8bc823740 1 --2- 172.21.15.37:0/1305495592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f6241010 0x5573f624b2c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.998+0000 7fb8bc823740 1 -- 172.21.15.37:0/1305495592 >> 172.21.15.37:0/1305495592 conn(0x5573f623c680 msgr2=0x5573f623b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.998+0000 7fb892ffd640 0 client.25241 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.998+0000 7fb8bc823740 1 -- 172.21.15.37:0/1305495592 shutdown_connections 2024-06-10T00:57:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.998+0000 7fb8bc823740 1 -- 172.21.15.37:0/1305495592 wait complete. 2024-06-10T00:57:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:48.998+0000 7fb8bc823740 20 client.25241 trim_cache size 0 max 16384 2024-06-10T00:57:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.013+0000 7fb8bc823740 1 Processor -- start 2024-06-10T00:57:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.013+0000 7fb8bc823740 1 -- start start 2024-06-10T00:57:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.014+0000 7fb8bc823740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f6241010 0x5573f6237820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.014+0000 7fb8bc823740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f6241a00 0x5573f6252800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.014+0000 7fb8bc823740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5573f6246b70 0x5573f624aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.014+0000 7fb8bc823740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b41af1f0 con 0x5573f6241a00 2024-06-10T00:57:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.014+0000 7fb8bc823740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b41acac0 con 0x5573f6241010 2024-06-10T00:57:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.014+0000 7fb8bc823740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b41aa910 con 0x5573f6246b70 2024-06-10T00:57:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.014+0000 7fb8b27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f6241a00 0x5573f6252800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.014+0000 7fb8b27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f6241a00 0x5573f6252800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35620/0 (socket says 172.21.15.37:35620) 2024-06-10T00:57:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.014+0000 7fb8b27fc640 1 -- 172.21.15.37:0/1733021409 learned_addr learned my addr 172.21.15.37:0/1733021409 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.014+0000 7fb8b2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f6241010 0x5573f6237820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.014+0000 7fb8b37fe640 1 --2- 172.21.15.37:0/1733021409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5573f6246b70 0x5573f624aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.014+0000 7fb8b27fc640 1 -- 172.21.15.37:0/1733021409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5573f6246b70 msgr2=0x5573f624aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.014+0000 7fb8b27fc640 1 --2- 172.21.15.37:0/1733021409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5573f6246b70 0x5573f624aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.014+0000 7fb8b27fc640 1 -- 172.21.15.37:0/1733021409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f6241010 msgr2=0x5573f6237820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.014+0000 7fb8b27fc640 1 --2- 172.21.15.37:0/1733021409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f6241010 0x5573f6237820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.014+0000 7fb8b27fc640 1 -- 172.21.15.37:0/1733021409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573f6241de0 con 0x5573f6241a00 2024-06-10T00:57:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.014+0000 7fb8b2ffd640 1 --2- 172.21.15.37:0/1733021409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f6241010 0x5573f6237820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:57:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.015+0000 7fb8b27fc640 1 --2- 172.21.15.37:0/1733021409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f6241a00 0x5573f6252800 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb8a809a240 tx=0x7fb8a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9deb2d0dfe0b9d93 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.015+0000 7fb8997fa640 1 -- 172.21.15.37:0/1733021409 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a80b8070 con 0x5573f6241a00 2024-06-10T00:57:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.015+0000 7fb8997fa640 1 -- 172.21.15.37:0/1733021409 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8a80ba040 con 0x5573f6241a00 2024-06-10T00:57:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.015+0000 7fb8997fa640 1 -- 172.21.15.37:0/1733021409 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a80b3040 con 0x5573f6241a00 2024-06-10T00:57:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.016+0000 7fb8bc823740 1 -- 172.21.15.37:0/1733021409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f6241a00 msgr2=0x5573f6252800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.016+0000 7fb8bc823740 1 --2- 172.21.15.37:0/1733021409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f6241a00 0x5573f6252800 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb8a809a240 tx=0x7fb8a80046c0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.016+0000 7fb8bc823740 1 -- 172.21.15.37:0/1733021409 shutdown_connections 2024-06-10T00:57:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.016+0000 7fb8bc823740 1 --2- 172.21.15.37:0/1733021409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5573f6246b70 0x5573f624aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.016+0000 7fb8bc823740 1 --2- 172.21.15.37:0/1733021409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f6241a00 0x5573f6252800 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.016+0000 7fb8bc823740 1 --2- 172.21.15.37:0/1733021409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f6241010 0x5573f6237820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.016+0000 7fb8bc823740 1 -- 172.21.15.37:0/1733021409 >> 172.21.15.37:0/1733021409 conn(0x5573f623c680 msgr2=0x5573f623ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.016+0000 7fb8bc823740 1 -- 172.21.15.37:0/1733021409 shutdown_connections 2024-06-10T00:57:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.016+0000 7fb8bc823740 1 -- 172.21.15.37:0/1733021409 wait complete. 2024-06-10T00:57:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.016+0000 7fb8bc823740 1 -- start start 2024-06-10T00:57:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.017+0000 7fb8bc823740 1 Processor -- start 2024-06-10T00:57:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.017+0000 7fb8bc823740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5573f6241010 0x5573f6252250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.017+0000 7fb8bc823740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f6241a00 0x5573f6252790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.017+0000 7fb8bc823740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f6246b70 0x5573f624ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.017+0000 7fb8bc823740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b41aaf80 con 0x5573f6241a00 2024-06-10T00:57:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.017+0000 7fb8bc823740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b41ac3f0 con 0x5573f6246b70 2024-06-10T00:57:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.017+0000 7fb8bc823740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b41ab650 con 0x5573f6241010 2024-06-10T00:57:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.017+0000 7fb8bc823740 10 client.0 _ll_register_callbacks cb 0x7fb8a4004480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.017+0000 7fb8b2ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5573f6241010 0x5573f6252250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:49.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.017+0000 7fb8b2ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5573f6241010 0x5573f6252250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:36156/0 (socket says 172.21.15.37:36156) 2024-06-10T00:57:49.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.017+0000 7fb8b2ffd640 1 -- 172.21.15.37:0/778957032 learned_addr learned my addr 172.21.15.37:0/778957032 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:49.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.017+0000 7fb8b37fe640 1 --2- 172.21.15.37:0/778957032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f6246b70 0x5573f624ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:49.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.018+0000 7fb8b27fc640 1 --2- 172.21.15.37:0/778957032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f6241a00 0x5573f6252790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:49.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.018+0000 7fb8b2ffd640 1 -- 172.21.15.37:0/778957032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f6246b70 msgr2=0x5573f624ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:49.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.018+0000 7fb8b2ffd640 1 --2- 172.21.15.37:0/778957032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f6246b70 0x5573f624ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.018+0000 7fb8b2ffd640 1 -- 172.21.15.37:0/778957032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f6241a00 msgr2=0x5573f6252790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:49.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.018+0000 7fb8b2ffd640 1 --2- 172.21.15.37:0/778957032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f6241a00 0x5573f6252790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.018+0000 7fb8b2ffd640 1 -- 172.21.15.37:0/778957032 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573f624f5a0 con 0x5573f6241010 2024-06-10T00:57:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.018+0000 7fb8b27fc640 1 --2- 172.21.15.37:0/778957032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f6241a00 0x5573f6252790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:57:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.018+0000 7fb8b2ffd640 1 --2- 172.21.15.37:0/778957032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5573f6241010 0x5573f6252250 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4009950 tx=0x7fb8a403c770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c491946251066472 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.018+0000 7fb892ffd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T00:57:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.018+0000 7fb892ffd640 1 -- 172.21.15.37:0/778957032 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4060020 con 0x5573f6241010 2024-06-10T00:57:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.018+0000 7fb8bc823740 1 -- 172.21.15.37:0/778957032 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8a80ba040 con 0x5573f6241010 2024-06-10T00:57:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.018+0000 7fb892ffd640 1 -- 172.21.15.37:0/778957032 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fb8a403cd20 con 0x5573f6241010 2024-06-10T00:57:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.018+0000 7fb892ffd640 1 -- 172.21.15.37:0/778957032 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb8a406b050 con 0x5573f6241010 2024-06-10T00:57:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.018+0000 7fb8bc823740 10 client.25282 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.018+0000 7fb8bc823740 1 -- 172.21.15.37:0/778957032 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5573f62c0c10 con 0x5573f6241010 2024-06-10T00:57:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.019+0000 7fb892ffd640 1 -- 172.21.15.37:0/778957032 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8a80ba040 con 0x5573f6241010 2024-06-10T00:57:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.019+0000 7fb892ffd640 1 -- 172.21.15.37:0/778957032 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb8a403a070 con 0x5573f6241010 2024-06-10T00:57:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.019+0000 7fb8bc823740 10 client.25282 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.019+0000 7fb8bc823740 10 client.25282 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T00:57:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.019+0000 7fb8bc823740 20 client.25282 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.019+0000 7fb8bc823740 1 --2- 172.21.15.37:0/778957032 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5573f62c15e0 0x5573f62c3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.019+0000 7fb8bc823740 4 client.25282 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T00:57:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.019+0000 7fb8bc823740 1 -- 172.21.15.37:0/778957032 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5573f61b1c20 con 0x5573f62c15e0 2024-06-10T00:57:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.019+0000 7fb8b27fc640 1 --2- 172.21.15.37:0/778957032 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5573f62c15e0 0x5573f62c3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.020+0000 7fb892ffd640 10 client.25282 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.020+0000 7fb8b27fc640 1 --2- 172.21.15.37:0/778957032 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5573f62c15e0 0x5573f62c3a00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d76731ca1257d42 server_cookie=ae309bf939e040d7 in_seq=0 out_seq=0 2024-06-10T00:57:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.027+0000 7fb892ffd640 1 -- 172.21.15.37:0/778957032 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5573f61b1c20 con 0x5573f62c15e0 2024-06-10T00:57:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.027+0000 7fb892ffd640 10 client.25282 handle_command_reply: tid=0 2024-06-10T00:57:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.027+0000 7fb8bc823740 1 client.25282 shutdown 2024-06-10T00:57:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.028+0000 7fb8bc823740 1 -- 172.21.15.37:0/778957032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5573f6241010 msgr2=0x5573f6252250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.028+0000 7fb8bc823740 1 --2- 172.21.15.37:0/778957032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5573f6241010 0x5573f6252250 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4009950 tx=0x7fb8a403c770 comp rx=0 tx=0).stop 2024-06-10T00:57:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.028+0000 7fb8bc823740 1 -- 172.21.15.37:0/778957032 shutdown_connections 2024-06-10T00:57:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.028+0000 7fb8bc823740 1 --2- 172.21.15.37:0/778957032 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5573f62c15e0 0x5573f62c3a00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.028+0000 7fb892ffd640 0 client.25282 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:57:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.028+0000 7fb8bc823740 1 --2- 172.21.15.37:0/778957032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f6246b70 0x5573f624ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.028+0000 7fb8bc823740 1 --2- 172.21.15.37:0/778957032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5573f6241a00 0x5573f6252790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.028+0000 7fb8bc823740 1 --2- 172.21.15.37:0/778957032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5573f6241010 0x5573f6252250 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.028+0000 7fb8bc823740 1 -- 172.21.15.37:0/778957032 >> 172.21.15.37:0/778957032 conn(0x5573f623c680 msgr2=0x5573f6245fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.028+0000 7fb8bc823740 1 -- 172.21.15.37:0/778957032 shutdown_connections 2024-06-10T00:57:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.028+0000 7fb8bc823740 1 -- 172.21.15.37:0/778957032 wait complete. 2024-06-10T00:57:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.028+0000 7fb8bc823740 20 client.25282 trim_cache size 0 max 16384 2024-06-10T00:57:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T00:57:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T00:57:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T00:57:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.028+0000 7fb8b9a38640 1 -- 172.21.15.37:0/3596683665 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb89c0a2c20 msgr2=0x7fb89c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.028+0000 7fb8b9a38640 1 --2- 172.21.15.37:0/3596683665 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb89c0a2c20 0x7fb89c0a50e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4006010 tx=0x7fb8a4000bc0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.028+0000 7fb8b9a38640 1 -- 172.21.15.37:0/3596683665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8b4101af0 msgr2=0x7fb8b419efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.028+0000 7fb8b9a38640 1 --2- 172.21.15.37:0/3596683665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8b4101af0 0x7fb8b419efe0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800b700 tx=0x7fb8a800bbc0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.029+0000 7fb8b9a38640 1 -- 172.21.15.37:0/3596683665 shutdown_connections 2024-06-10T00:57:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.029+0000 7fb8b9a38640 1 --2- 172.21.15.37:0/3596683665 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb89c0a2c20 0x7fb89c0a50e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.029+0000 7fb8b9a38640 1 --2- 172.21.15.37:0/3596683665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8b41024b0 0x7fb8b41a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.029+0000 7fb8b9a38640 1 --2- 172.21.15.37:0/3596683665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8b4101af0 0x7fb8b419efe0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.029+0000 7fb8b9a38640 1 --2- 172.21.15.37:0/3596683665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8b4100900 0x7fb8b4069b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.029+0000 7fb8b9a38640 1 -- 172.21.15.37:0/3596683665 >> 172.21.15.37:0/3596683665 conn(0x7fb8b40fbe70 msgr2=0x7fb8b4104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.029+0000 7fb8b9a38640 1 -- 172.21.15.37:0/3596683665 shutdown_connections 2024-06-10T00:57:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:57:49.029+0000 7fb8b9a38640 1 -- 172.21.15.37:0/3596683665 wait complete. 2024-06-10T00:57:49.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.372+0000 7f581ad5e700 1 -- 172.21.15.37:0/456515646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5814101440 msgr2=0x7f581410f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:49.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.372+0000 7f581ad5e700 1 --2- 172.21.15.37:0/456515646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5814101440 0x7f581410f530 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f580c00b0a0 tx=0x7f580c00b3b0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.372+0000 7f581ad5e700 1 -- 172.21.15.37:0/456515646 shutdown_connections 2024-06-10T00:57:49.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.372+0000 7f581ad5e700 1 --2- 172.21.15.37:0/456515646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5814101440 0x7f581410f530 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.372+0000 7f581ad5e700 1 --2- 172.21.15.37:0/456515646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58141086f0 0x7f5814108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.372+0000 7f581ad5e700 1 --2- 172.21.15.37:0/456515646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5814100a80 0x7f5814100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.372+0000 7f581ad5e700 1 -- 172.21.15.37:0/456515646 >> 172.21.15.37:0/456515646 conn(0x7f5814077a00 msgr2=0x7f58140fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:49.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.372+0000 7f581ad5e700 1 -- 172.21.15.37:0/456515646 shutdown_connections 2024-06-10T00:57:49.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.372+0000 7f581ad5e700 1 -- 172.21.15.37:0/456515646 wait complete. 2024-06-10T00:57:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.373+0000 7f581ad5e700 1 Processor -- start 2024-06-10T00:57:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.373+0000 7f581ad5e700 1 -- start start 2024-06-10T00:57:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.373+0000 7f581ad5e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5814100a80 0x7f5814101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.373+0000 7f581ad5e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5814101440 0x7f58141021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.373+0000 7f581ad5e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58141086f0 0x7f58141026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.373+0000 7f581ad5e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5814112a70 con 0x7f58141086f0 2024-06-10T00:57:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.373+0000 7f581ad5e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f58141128f0 con 0x7f5814100a80 2024-06-10T00:57:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.373+0000 7f581ad5e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5814112bf0 con 0x7f5814101440 2024-06-10T00:57:49.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.374+0000 7f58192fb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58141086f0 0x7f58141026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:49.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.374+0000 7f5818afa700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5814100a80 0x7f5814101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:49.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.374+0000 7f58192fb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58141086f0 0x7f58141026e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35634/0 (socket says 172.21.15.37:35634) 2024-06-10T00:57:49.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.374+0000 7f58192fb700 1 -- 172.21.15.37:0/487157761 learned_addr learned my addr 172.21.15.37:0/487157761 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:49.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.374+0000 7f5813fff700 1 --2- 172.21.15.37:0/487157761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5814101440 0x7f58141021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:49.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.374+0000 7f5818afa700 1 -- 172.21.15.37:0/487157761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5814101440 msgr2=0x7f58141021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:49.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.374+0000 7f5818afa700 1 --2- 172.21.15.37:0/487157761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5814101440 0x7f58141021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.374+0000 7f5818afa700 1 -- 172.21.15.37:0/487157761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58141086f0 msgr2=0x7f58141026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:49.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.374+0000 7f5818afa700 1 --2- 172.21.15.37:0/487157761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58141086f0 0x7f58141026e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.374+0000 7f5818afa700 1 -- 172.21.15.37:0/487157761 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58141a7400 con 0x7f5814100a80 2024-06-10T00:57:49.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.374+0000 7f58192fb700 1 --2- 172.21.15.37:0/487157761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58141086f0 0x7f58141026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:57:49.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.374+0000 7f5818afa700 1 --2- 172.21.15.37:0/487157761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5814100a80 0x7f5814101c60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5814069990 tx=0x7f580800b750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:49.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.375+0000 7f5811ffb700 1 -- 172.21.15.37:0/487157761 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808015020 con 0x7f5814100a80 2024-06-10T00:57:49.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.375+0000 7f581ad5e700 1 -- 172.21.15.37:0/487157761 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58141a76f0 con 0x7f5814100a80 2024-06-10T00:57:49.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.375+0000 7f5811ffb700 1 -- 172.21.15.37:0/487157761 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f580800be50 con 0x7f5814100a80 2024-06-10T00:57:49.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.375+0000 7f5811ffb700 1 -- 172.21.15.37:0/487157761 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808010cb0 con 0x7f5814100a80 2024-06-10T00:57:49.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.375+0000 7f581ad5e700 1 -- 172.21.15.37:0/487157761 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58141a7c80 con 0x7f5814100a80 2024-06-10T00:57:49.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.376+0000 7f581ad5e700 1 -- 172.21.15.37:0/487157761 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f581410ac60 con 0x7f5814100a80 2024-06-10T00:57:49.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.377+0000 7f5811ffb700 1 -- 172.21.15.37:0/487157761 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 137884+0+0 (secure 0 0 0) 0x7f5808018020 con 0x7f5814100a80 2024-06-10T00:57:49.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.378+0000 7f5811ffb700 1 --2- 172.21.15.37:0/487157761 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f58040a2eb0 0x7f58040a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:49.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.378+0000 7f5811ffb700 1 -- 172.21.15.37:0/487157761 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f5808010a50 con 0x7f5814100a80 2024-06-10T00:57:49.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.378+0000 7f5813fff700 1 --2- 172.21.15.37:0/487157761 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f58040a2eb0 0x7f58040a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:49.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.379+0000 7f5813fff700 1 --2- 172.21.15.37:0/487157761 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f58040a2eb0 0x7f58040a5370 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5814107d30 tx=0x7f5800009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:49.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.383+0000 7f5811ffb700 1 -- 172.21.15.37:0/487157761 <== mon.2 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f580801f020 con 0x7f5814100a80 2024-06-10T00:57:49.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.698+0000 7f581ad5e700 1 -- 172.21.15.37:0/487157761 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5814103140 con 0x7f5814100a80 2024-06-10T00:57:49.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.699+0000 7f5811ffb700 1 -- 172.21.15.37:0/487157761 <== mon.2 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11867 (secure 0 0 0) 0x7f580809b450 con 0x7f5814100a80 2024-06-10T00:57:49.700 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:57:49.700 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T00:56:30.291171+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24451,24458,24472]},"id":1}]} 2024-06-10T00:57:49.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.702+0000 7f581ad5e700 1 -- 172.21.15.37:0/487157761 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f58040a2eb0 msgr2=0x7f58040a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:49.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.702+0000 7f581ad5e700 1 --2- 172.21.15.37:0/487157761 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f58040a2eb0 0x7f58040a5370 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5814107d30 tx=0x7f5800009190 comp rx=0 tx=0).stop 2024-06-10T00:57:49.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.702+0000 7f581ad5e700 1 -- 172.21.15.37:0/487157761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5814100a80 msgr2=0x7f5814101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:49.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.702+0000 7f581ad5e700 1 --2- 172.21.15.37:0/487157761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5814100a80 0x7f5814101c60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5814069990 tx=0x7f580800b750 comp rx=0 tx=0).stop 2024-06-10T00:57:49.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.702+0000 7f581ad5e700 1 -- 172.21.15.37:0/487157761 shutdown_connections 2024-06-10T00:57:49.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.702+0000 7f581ad5e700 1 --2- 172.21.15.37:0/487157761 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f58040a2eb0 0x7f58040a5370 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.702+0000 7f581ad5e700 1 --2- 172.21.15.37:0/487157761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58141086f0 0x7f58141026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.702+0000 7f581ad5e700 1 --2- 172.21.15.37:0/487157761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5814100a80 0x7f5814101c60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.702+0000 7f581ad5e700 1 --2- 172.21.15.37:0/487157761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5814101440 0x7f58141021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:49.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.702+0000 7f581ad5e700 1 -- 172.21.15.37:0/487157761 >> 172.21.15.37:0/487157761 conn(0x7f5814077a00 msgr2=0x7f58140ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:49.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.703+0000 7f581ad5e700 1 -- 172.21.15.37:0/487157761 shutdown_connections 2024-06-10T00:57:49.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:49.703+0000 7f581ad5e700 1 -- 172.21.15.37:0/487157761 wait complete. 2024-06-10T00:57:49.704 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-06-10T00:57:50.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:50 smithi037 ceph-mon[32192]: pgmap v464: 129 pgs: 129 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 7 op/s 2024-06-10T00:57:50.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:50 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/487157761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:57:50.513 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T00:57:50.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:50 smithi167 ceph-mon[36917]: pgmap v464: 129 pgs: 129 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 7 op/s 2024-06-10T00:57:50.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:50 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/487157761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:57:50.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:50 smithi094 ceph-mon[36708]: pgmap v464: 129 pgs: 129 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 7 op/s 2024-06-10T00:57:50.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:50 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/487157761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:57:50.857 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:57:52.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:52 smithi037 ceph-mon[32192]: pgmap v465: 129 pgs: 129 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 6 op/s 2024-06-10T00:57:52.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:52 smithi037 ceph-mon[32192]: mgrmap e20: x(active, since 12m), standbys: y, z 2024-06-10T00:57:52.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:52 smithi167 ceph-mon[36917]: pgmap v465: 129 pgs: 129 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 6 op/s 2024-06-10T00:57:52.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:52 smithi167 ceph-mon[36917]: mgrmap e20: x(active, since 12m), standbys: y, z 2024-06-10T00:57:52.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:52 smithi094 ceph-mon[36708]: pgmap v465: 129 pgs: 129 active+clean; 680 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 938 B/s wr, 6 op/s 2024-06-10T00:57:52.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:52 smithi094 ceph-mon[36708]: mgrmap e20: x(active, since 12m), standbys: y, z 2024-06-10T00:57:53.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.066+0000 7eff63147700 1 -- 172.21.15.37:0/3331910743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff5c101d40 msgr2=0x7eff5c10e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:53.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.066+0000 7eff63147700 1 --2- 172.21.15.37:0/3331910743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff5c101d40 0x7eff5c10e230 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7eff5400b0a0 tx=0x7eff5400b3b0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.067+0000 7eff63147700 1 -- 172.21.15.37:0/3331910743 shutdown_connections 2024-06-10T00:57:53.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.067+0000 7eff63147700 1 --2- 172.21.15.37:0/3331910743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff5c101d40 0x7eff5c10e230 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.067+0000 7eff63147700 1 --2- 172.21.15.37:0/3331910743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff5c1073f0 0x7eff5c1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.067+0000 7eff63147700 1 --2- 172.21.15.37:0/3331910743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff5c101380 0x7eff5c101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.067+0000 7eff63147700 1 -- 172.21.15.37:0/3331910743 >> 172.21.15.37:0/3331910743 conn(0x7eff5c078300 msgr2=0x7eff5c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:53.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.067+0000 7eff63147700 1 -- 172.21.15.37:0/3331910743 shutdown_connections 2024-06-10T00:57:53.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.067+0000 7eff63147700 1 -- 172.21.15.37:0/3331910743 wait complete. 2024-06-10T00:57:53.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.068+0000 7eff63147700 1 Processor -- start 2024-06-10T00:57:53.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.068+0000 7eff63147700 1 -- start start 2024-06-10T00:57:53.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.068+0000 7eff63147700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff5c101380 0x7eff5c196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:53.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.068+0000 7eff63147700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff5c101d40 0x7eff5c197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:53.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.068+0000 7eff63147700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff5c1073f0 0x7eff5c19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:53.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.068+0000 7eff63147700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c111660 con 0x7eff5c101d40 2024-06-10T00:57:53.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.068+0000 7eff63147700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c1114e0 con 0x7eff5c101380 2024-06-10T00:57:53.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.068+0000 7eff63147700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c1117e0 con 0x7eff5c1073f0 2024-06-10T00:57:53.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.069+0000 7eff5bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff5c101d40 0x7eff5c197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:53.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.069+0000 7eff5bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff5c101d40 0x7eff5c197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51172/0 (socket says 172.21.15.37:51172) 2024-06-10T00:57:53.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.069+0000 7eff5bfff700 1 -- 172.21.15.37:0/1956643318 learned_addr learned my addr 172.21.15.37:0/1956643318 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:53.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.069+0000 7eff60ee3700 1 --2- 172.21.15.37:0/1956643318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff5c101380 0x7eff5c196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:53.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.069+0000 7eff5bfff700 1 -- 172.21.15.37:0/1956643318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff5c1073f0 msgr2=0x7eff5c19b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:53.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.069+0000 7eff616e4700 1 --2- 172.21.15.37:0/1956643318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff5c1073f0 0x7eff5c19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:53.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.069+0000 7eff5bfff700 1 --2- 172.21.15.37:0/1956643318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff5c1073f0 0x7eff5c19b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.069+0000 7eff5bfff700 1 -- 172.21.15.37:0/1956643318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff5c101380 msgr2=0x7eff5c196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:53.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.069+0000 7eff5bfff700 1 --2- 172.21.15.37:0/1956643318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff5c101380 0x7eff5c196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.069+0000 7eff5bfff700 1 -- 172.21.15.37:0/1956643318 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5c19ba60 con 0x7eff5c101d40 2024-06-10T00:57:53.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.069+0000 7eff60ee3700 1 --2- 172.21.15.37:0/1956643318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff5c101380 0x7eff5c196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:57:53.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.069+0000 7eff5bfff700 1 --2- 172.21.15.37:0/1956643318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff5c101d40 0x7eff5c197240 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7eff4800dac0 tx=0x7eff4800ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:53.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.070+0000 7eff59ffb700 1 -- 172.21.15.37:0/1956643318 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff48016070 con 0x7eff5c101d40 2024-06-10T00:57:53.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.070+0000 7eff59ffb700 1 -- 172.21.15.37:0/1956643318 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff48004d10 con 0x7eff5c101d40 2024-06-10T00:57:53.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.070+0000 7eff63147700 1 -- 172.21.15.37:0/1956643318 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff5c19bd50 con 0x7eff5c101d40 2024-06-10T00:57:53.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.070+0000 7eff59ffb700 1 -- 172.21.15.37:0/1956643318 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4800bd10 con 0x7eff5c101d40 2024-06-10T00:57:53.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.070+0000 7eff63147700 1 -- 172.21.15.37:0/1956643318 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff5c1b4760 con 0x7eff5c101d40 2024-06-10T00:57:53.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.074+0000 7eff59ffb700 1 -- 172.21.15.37:0/1956643318 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7eff48004220 con 0x7eff5c101d40 2024-06-10T00:57:53.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.075+0000 7eff59ffb700 1 --2- 172.21.15.37:0/1956643318 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7eff4c0a2ca0 0x7eff4c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:53.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.075+0000 7eff60ee3700 1 --2- 172.21.15.37:0/1956643318 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7eff4c0a2ca0 0x7eff4c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:53.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.075+0000 7eff6d9cab80 1 Processor -- start 2024-06-10T00:57:53.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.075+0000 7eff6d9cab80 1 -- start start 2024-06-10T00:57:53.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.076+0000 7eff60ee3700 1 --2- 172.21.15.37:0/1956643318 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7eff4c0a2ca0 0x7eff4c0a5160 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7eff50009fd0 tx=0x7eff50009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:53.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.076+0000 7eff59ffb700 1 -- 172.21.15.37:0/1956643318 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7eff480cf2b0 con 0x7eff5c101d40 2024-06-10T00:57:53.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.076+0000 7eff6d9cab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a2cb69640 0x562a2cb69a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:53.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.076+0000 7eff6d9cab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a2cb69f60 0x562a2cb639e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:53.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.076+0000 7eff6d9cab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2cb63f20 0x562a2cb64340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:53.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.076+0000 7eff60ee3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a2cb69640 0x562a2cb69a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:53.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.076+0000 7eff60ee3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a2cb69640 0x562a2cb69a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51174/0 (socket says 172.21.15.37:51174) 2024-06-10T00:57:53.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.077+0000 7eff60ee3700 1 -- 172.21.15.37:0/3630225350 learned_addr learned my addr 172.21.15.37:0/3630225350 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:53.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.077+0000 7eff5bfff700 1 --2- 172.21.15.37:0/3630225350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a2cb69f60 0x562a2cb639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:53.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.077+0000 7eff6d9cab80 1 -- 172.21.15.37:0/3630225350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562a2cab5810 con 0x562a2cb69640 2024-06-10T00:57:53.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.077+0000 7eff6d9cab80 1 -- 172.21.15.37:0/3630225350 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562a2ca5d720 con 0x562a2cb69f60 2024-06-10T00:57:53.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.077+0000 7eff6d9cab80 1 -- 172.21.15.37:0/3630225350 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562a2cab17d0 con 0x562a2cb63f20 2024-06-10T00:57:53.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.077+0000 7eff616e4700 1 --2- 172.21.15.37:0/3630225350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2cb63f20 0x562a2cb64340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:53.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.077+0000 7eff5bfff700 1 -- 172.21.15.37:0/3630225350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2cb63f20 msgr2=0x562a2cb64340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:53.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.077+0000 7eff5bfff700 1 --2- 172.21.15.37:0/3630225350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2cb63f20 0x562a2cb64340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.077+0000 7eff5bfff700 1 -- 172.21.15.37:0/3630225350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a2cb69640 msgr2=0x562a2cb69a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:53.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.077+0000 7eff5bfff700 1 --2- 172.21.15.37:0/3630225350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a2cb69640 0x562a2cb69a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.077+0000 7eff5bfff700 1 -- 172.21.15.37:0/3630225350 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a2ca53f50 con 0x562a2cb69f60 2024-06-10T00:57:53.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.077+0000 7eff60ee3700 1 --2- 172.21.15.37:0/3630225350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a2cb69640 0x562a2cb69a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:57:53.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.078+0000 7eff5bfff700 1 --2- 172.21.15.37:0/3630225350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a2cb69f60 0x562a2cb639e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7eff4800b540 tx=0x7eff4809b0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af91b96d5205636f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:53.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.078+0000 7eff46ffd700 1 -- 172.21.15.37:0/3630225350 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4809b6c0 con 0x562a2cb69f60 2024-06-10T00:57:53.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.078+0000 7eff46ffd700 1 -- 172.21.15.37:0/3630225350 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff4809fa70 con 0x562a2cb69f60 2024-06-10T00:57:53.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.078+0000 7eff46ffd700 1 -- 172.21.15.37:0/3630225350 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff480a3ea0 con 0x562a2cb69f60 2024-06-10T00:57:53.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.079+0000 7eff6d9cab80 1 -- 172.21.15.37:0/3630225350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a2cb69f60 msgr2=0x562a2cb639e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:53.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.079+0000 7eff6d9cab80 1 --2- 172.21.15.37:0/3630225350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a2cb69f60 0x562a2cb639e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7eff4800b540 tx=0x7eff4809b0a0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.079+0000 7eff6d9cab80 1 -- 172.21.15.37:0/3630225350 shutdown_connections 2024-06-10T00:57:53.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.079+0000 7eff6d9cab80 1 --2- 172.21.15.37:0/3630225350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2cb63f20 0x562a2cb64340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.079+0000 7eff6d9cab80 1 --2- 172.21.15.37:0/3630225350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a2cb69640 0x562a2cb69a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.079+0000 7eff6d9cab80 1 --2- 172.21.15.37:0/3630225350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a2cb69f60 0x562a2cb639e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.079+0000 7eff6d9cab80 1 -- 172.21.15.37:0/3630225350 >> 172.21.15.37:0/3630225350 conn(0x562a2ca8ecc0 msgr2=0x562a2cb5e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:53.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.079+0000 7eff6d9cab80 1 -- 172.21.15.37:0/3630225350 shutdown_connections 2024-06-10T00:57:53.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.079+0000 7eff6d9cab80 1 -- 172.21.15.37:0/3630225350 wait complete. 2024-06-10T00:57:53.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.079+0000 7eff6d9cab80 1 -- start start 2024-06-10T00:57:53.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.080+0000 7eff6d9cab80 1 Processor -- start 2024-06-10T00:57:53.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.080+0000 7eff6d9cab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a2cb63f20 0x562a2cb6fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:53.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.080+0000 7eff6d9cab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a2cb69640 0x562a2cb6d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:53.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.080+0000 7eff6d9cab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2cb69f60 0x562a2cb6d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:53.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.080+0000 7eff6d9cab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562a2cb70910 con 0x562a2cb63f20 2024-06-10T00:57:53.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.080+0000 7eff6d9cab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562a2ca55bf0 con 0x562a2cb69640 2024-06-10T00:57:53.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.080+0000 7eff6d9cab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562a2cb70a90 con 0x562a2cb69f60 2024-06-10T00:57:53.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.080+0000 7eff6d9cab80 10 client.0 _ll_register_callbacks cb 0x562a2ca4f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:53.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.080+0000 7eff5bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a2cb69640 0x562a2cb6d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:53.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.080+0000 7eff60ee3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a2cb63f20 0x562a2cb6fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:53.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.081+0000 7eff616e4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2cb69f60 0x562a2cb6d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:53.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.081+0000 7eff60ee3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a2cb63f20 0x562a2cb6fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51176/0 (socket says 172.21.15.37:51176) 2024-06-10T00:57:53.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.081+0000 7eff60ee3700 1 -- 172.21.15.37:0/654603252 learned_addr learned my addr 172.21.15.37:0/654603252 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:53.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.081+0000 7eff5bfff700 1 -- 172.21.15.37:0/654603252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2cb69f60 msgr2=0x562a2cb6d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:53.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.081+0000 7eff5bfff700 1 --2- 172.21.15.37:0/654603252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2cb69f60 0x562a2cb6d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.081+0000 7eff5bfff700 1 -- 172.21.15.37:0/654603252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a2cb63f20 msgr2=0x562a2cb6fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:53.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.081+0000 7eff5bfff700 1 --2- 172.21.15.37:0/654603252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a2cb63f20 0x562a2cb6fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.081+0000 7eff5bfff700 1 -- 172.21.15.37:0/654603252 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a2ca048d0 con 0x562a2cb69640 2024-06-10T00:57:53.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.081+0000 7eff60ee3700 1 --2- 172.21.15.37:0/654603252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a2cb63f20 0x562a2cb6fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:57:53.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.082+0000 7eff5bfff700 1 --2- 172.21.15.37:0/654603252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a2cb69640 0x562a2cb6d100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x562a2cb70580 tx=0x7eff480cf1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6addcf24e51f858f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:53.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.082+0000 7eff3bfff700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T00:57:53.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.082+0000 7eff3bfff700 1 -- 172.21.15.37:0/654603252 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff480b8060 con 0x562a2cb69640 2024-06-10T00:57:53.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.082+0000 7eff3bfff700 1 -- 172.21.15.37:0/654603252 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7eff480afbe0 con 0x562a2cb69640 2024-06-10T00:57:53.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.082+0000 7eff6d9cab80 1 -- 172.21.15.37:0/654603252 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff4809fa70 con 0x562a2cb69640 2024-06-10T00:57:53.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.082+0000 7eff3bfff700 1 -- 172.21.15.37:0/654603252 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7eff480afde0 con 0x562a2cb69640 2024-06-10T00:57:53.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.082+0000 7eff6d9cab80 10 client.25262 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:53.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.082+0000 7eff6d9cab80 1 -- 172.21.15.37:0/654603252 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562a2cb7b290 con 0x562a2cb69640 2024-06-10T00:57:53.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.083+0000 7eff3bfff700 1 -- 172.21.15.37:0/654603252 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff4809fa70 con 0x562a2cb69640 2024-06-10T00:57:53.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.083+0000 7eff3bfff700 1 -- 172.21.15.37:0/654603252 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7eff4809ab10 con 0x562a2cb69640 2024-06-10T00:57:53.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.083+0000 7eff6d9cab80 10 client.25262 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:53.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.083+0000 7eff6d9cab80 10 client.25262 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-10T00:57:53.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.083+0000 7eff6d9cab80 20 client.25262 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:53.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.083+0000 7eff6d9cab80 1 --2- 172.21.15.37:0/654603252 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x562a2cb7d8f0 0x562a2cb7fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:53.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.083+0000 7eff6d9cab80 4 client.25262 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:57:53.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.083+0000 7eff6d9cab80 1 -- 172.21.15.37:0/654603252 --> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a2cac2d10 con 0x562a2cb7d8f0 2024-06-10T00:57:53.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.083+0000 7eff60ee3700 1 --2- 172.21.15.37:0/654603252 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x562a2cb7d8f0 0x562a2cb7fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:53.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.084+0000 7eff3bfff700 10 client.25262 ms_handle_connect on v2:172.21.15.167:6836/2720148065 2024-06-10T00:57:53.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.084+0000 7eff60ee3700 1 --2- 172.21.15.37:0/654603252 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x562a2cb7d8f0 0x562a2cb7fd10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bd55a3a30d80c72a server_cookie=cb7b6d79fab6747a in_seq=0 out_seq=0 2024-06-10T00:57:53.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.085+0000 7eff3bfff700 1 -- 172.21.15.37:0/654603252 <== mds.2 v2:172.21.15.167:6836/2720148065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562a2cac2d10 con 0x562a2cb7d8f0 2024-06-10T00:57:53.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.085+0000 7eff3bfff700 10 client.25262 handle_command_reply: tid=0 2024-06-10T00:57:53.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.085+0000 7eff6d9cab80 1 client.25262 shutdown 2024-06-10T00:57:53.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.085+0000 7eff6d9cab80 1 -- 172.21.15.37:0/654603252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a2cb69640 msgr2=0x562a2cb6d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:53.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.085+0000 7eff6d9cab80 1 --2- 172.21.15.37:0/654603252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a2cb69640 0x562a2cb6d100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x562a2cb70580 tx=0x7eff480cf1d0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.085+0000 7eff6d9cab80 1 -- 172.21.15.37:0/654603252 shutdown_connections 2024-06-10T00:57:53.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.085+0000 7eff6d9cab80 1 --2- 172.21.15.37:0/654603252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2cb69f60 0x562a2cb6d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.085+0000 7eff6d9cab80 1 --2- 172.21.15.37:0/654603252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a2cb63f20 0x562a2cb6fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.085+0000 7eff6d9cab80 1 --2- 172.21.15.37:0/654603252 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x562a2cb7d8f0 0x562a2cb7fd10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.085+0000 7eff6d9cab80 1 --2- 172.21.15.37:0/654603252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a2cb69640 0x562a2cb6d100 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.085+0000 7eff6d9cab80 1 -- 172.21.15.37:0/654603252 >> 172.21.15.37:0/654603252 conn(0x562a2ca8ecc0 msgr2=0x562a2cb5e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:53.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.085+0000 7eff3bfff700 0 client.25262 ms_handle_reset on v2:172.21.15.167:6836/2720148065 2024-06-10T00:57:53.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.085+0000 7eff6d9cab80 1 -- 172.21.15.37:0/654603252 shutdown_connections 2024-06-10T00:57:53.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.085+0000 7eff6d9cab80 1 -- 172.21.15.37:0/654603252 wait complete. 2024-06-10T00:57:53.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.085+0000 7eff6d9cab80 20 client.25262 trim_cache size 0 max 16384 2024-06-10T00:57:53.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.117+0000 7eff6d9cab80 1 Processor -- start 2024-06-10T00:57:53.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.117+0000 7eff6d9cab80 1 -- start start 2024-06-10T00:57:53.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.117+0000 7eff6d9cab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a2cb63f20 0x562a2cb639e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:53.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.117+0000 7eff6d9cab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2cb69640 0x562a2cb6ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:53.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.117+0000 7eff6d9cab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a2cb69f60 0x562a2cb85620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:53.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.117+0000 7eff6d9cab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c1bfb20 con 0x562a2cb63f20 2024-06-10T00:57:53.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.117+0000 7eff6d9cab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c1bd3b0 con 0x562a2cb69f60 2024-06-10T00:57:53.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.117+0000 7eff6d9cab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c1bb200 con 0x562a2cb69640 2024-06-10T00:57:53.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.117+0000 7eff60ee3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a2cb63f20 0x562a2cb639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:53.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.117+0000 7eff60ee3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a2cb63f20 0x562a2cb639e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51192/0 (socket says 172.21.15.37:51192) 2024-06-10T00:57:53.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.117+0000 7eff60ee3700 1 -- 172.21.15.37:0/771698074 learned_addr learned my addr 172.21.15.37:0/771698074 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:53.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.117+0000 7eff616e4700 1 --2- 172.21.15.37:0/771698074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a2cb69f60 0x562a2cb85620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:53.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.117+0000 7eff5bfff700 1 --2- 172.21.15.37:0/771698074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2cb69640 0x562a2cb6ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:53.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.118+0000 7eff60ee3700 1 -- 172.21.15.37:0/771698074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2cb69640 msgr2=0x562a2cb6ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:53.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.118+0000 7eff60ee3700 1 --2- 172.21.15.37:0/771698074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2cb69640 0x562a2cb6ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.118+0000 7eff60ee3700 1 -- 172.21.15.37:0/771698074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a2cb69f60 msgr2=0x562a2cb85620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:53.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.118+0000 7eff60ee3700 1 --2- 172.21.15.37:0/771698074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a2cb69f60 0x562a2cb85620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.118+0000 7eff60ee3700 1 -- 172.21.15.37:0/771698074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a2cb5f500 con 0x562a2cb63f20 2024-06-10T00:57:53.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.118+0000 7eff5bfff700 1 --2- 172.21.15.37:0/771698074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2cb69640 0x562a2cb6ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:57:53.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.118+0000 7eff60ee3700 1 --2- 172.21.15.37:0/771698074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a2cb63f20 0x562a2cb639e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7eff50013010 tx=0x7eff50014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a65d472bd40023 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:53.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.118+0000 7eff467fc700 1 -- 172.21.15.37:0/771698074 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5001fbc0 con 0x562a2cb63f20 2024-06-10T00:57:53.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.118+0000 7eff467fc700 1 -- 172.21.15.37:0/771698074 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff5000ee10 con 0x562a2cb63f20 2024-06-10T00:57:53.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.118+0000 7eff467fc700 1 -- 172.21.15.37:0/771698074 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50028770 con 0x562a2cb63f20 2024-06-10T00:57:53.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.119+0000 7eff6d9cab80 1 -- 172.21.15.37:0/771698074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a2cb63f20 msgr2=0x562a2cb639e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:53.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.119+0000 7eff6d9cab80 1 --2- 172.21.15.37:0/771698074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a2cb63f20 0x562a2cb639e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7eff50013010 tx=0x7eff50014000 comp rx=0 tx=0).stop 2024-06-10T00:57:53.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.119+0000 7eff6d9cab80 1 -- 172.21.15.37:0/771698074 shutdown_connections 2024-06-10T00:57:53.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.119+0000 7eff6d9cab80 1 --2- 172.21.15.37:0/771698074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a2cb69f60 0x562a2cb85620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.119+0000 7eff6d9cab80 1 --2- 172.21.15.37:0/771698074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a2cb63f20 0x562a2cb639e0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.119+0000 7eff6d9cab80 1 --2- 172.21.15.37:0/771698074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2cb69640 0x562a2cb6ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.119+0000 7eff6d9cab80 1 -- 172.21.15.37:0/771698074 >> 172.21.15.37:0/771698074 conn(0x562a2ca8ecc0 msgr2=0x562a2cb6a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:53.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.119+0000 7eff6d9cab80 1 -- 172.21.15.37:0/771698074 shutdown_connections 2024-06-10T00:57:53.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.119+0000 7eff6d9cab80 1 -- 172.21.15.37:0/771698074 wait complete. 2024-06-10T00:57:53.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.120+0000 7eff6d9cab80 1 -- start start 2024-06-10T00:57:53.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.120+0000 7eff6d9cab80 1 Processor -- start 2024-06-10T00:57:53.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.120+0000 7eff6d9cab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a2cb63f20 0x562a2cb85c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:53.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.120+0000 7eff6d9cab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2cb69640 0x562a2cb75d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:53.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.120+0000 7eff6d9cab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a2cb69f60 0x562a2cb76260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:53.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.120+0000 7eff60ee3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a2cb63f20 0x562a2cb85c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:53.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.120+0000 7eff60ee3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a2cb63f20 0x562a2cb85c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:34426/0 (socket says 172.21.15.37:34426) 2024-06-10T00:57:53.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.121+0000 7eff5bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2cb69640 0x562a2cb75d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:53.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.121+0000 7eff5bfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2cb69640 0x562a2cb75d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:48812/0 (socket says 172.21.15.37:48812) 2024-06-10T00:57:53.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.121+0000 7eff616e4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a2cb69f60 0x562a2cb76260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:53.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.121+0000 7eff616e4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a2cb69f60 0x562a2cb76260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51202/0 (socket says 172.21.15.37:51202) 2024-06-10T00:57:53.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.121+0000 7eff60ee3700 1 -- 172.21.15.37:0/2857969929 learned_addr learned my addr 172.21.15.37:0/2857969929 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:57:53.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.121+0000 7eff6d9cab80 1 -- 172.21.15.37:0/2857969929 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c1bb870 con 0x562a2cb69f60 2024-06-10T00:57:53.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.121+0000 7eff6d9cab80 1 -- 172.21.15.37:0/2857969929 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c1bcce0 con 0x562a2cb63f20 2024-06-10T00:57:53.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.121+0000 7eff6d9cab80 1 -- 172.21.15.37:0/2857969929 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c1bbf40 con 0x562a2cb69640 2024-06-10T00:57:53.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.121+0000 7eff6d9cab80 10 client.0 _ll_register_callbacks cb 0x562a2ca4f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:57:53.772 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T00:57:53.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.121+0000 7eff616e4700 1 -- 172.21.15.37:0/2857969929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2cb69640 msgr2=0x562a2cb75d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:53.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.121+0000 7eff616e4700 1 --2- 172.21.15.37:0/2857969929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2cb69640 0x562a2cb75d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.121+0000 7eff616e4700 1 -- 172.21.15.37:0/2857969929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a2cb63f20 msgr2=0x562a2cb85c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:57:53.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.121+0000 7eff616e4700 1 --2- 172.21.15.37:0/2857969929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a2cb63f20 0x562a2cb85c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.121+0000 7eff616e4700 1 -- 172.21.15.37:0/2857969929 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a2cb768b0 con 0x562a2cb69f60 2024-06-10T00:57:53.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.121+0000 7eff60ee3700 1 --2- 172.21.15.37:0/2857969929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a2cb63f20 0x562a2cb85c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:57:53.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.121+0000 7eff616e4700 1 --2- 172.21.15.37:0/2857969929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a2cb69f60 0x562a2cb76260 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7eff5400b780 tx=0x7eff5400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5ee61f8131fb9e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:57:53.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.121+0000 7eff3bfff700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:57:53.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.122+0000 7eff3bfff700 1 -- 172.21.15.37:0/2857969929 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff540074e0 con 0x562a2cb69f60 2024-06-10T00:57:53.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.122+0000 7eff3bfff700 1 -- 172.21.15.37:0/2857969929 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7eff54007680 con 0x562a2cb69f60 2024-06-10T00:57:53.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.122+0000 7eff6d9cab80 1 -- 172.21.15.37:0/2857969929 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff5000ee10 con 0x562a2cb69f60 2024-06-10T00:57:53.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.122+0000 7eff3bfff700 1 -- 172.21.15.37:0/2857969929 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7eff54007880 con 0x562a2cb69f60 2024-06-10T00:57:53.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.122+0000 7eff6d9cab80 10 client.15675 fetch_fsmap learned FSMap version 27 2024-06-10T00:57:53.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.122+0000 7eff6d9cab80 1 -- 172.21.15.37:0/2857969929 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562a2cbe75f0 con 0x562a2cb69f60 2024-06-10T00:57:53.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.122+0000 7eff3bfff700 1 -- 172.21.15.37:0/2857969929 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff5000ee10 con 0x562a2cb69f60 2024-06-10T00:57:53.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.122+0000 7eff3bfff700 1 -- 172.21.15.37:0/2857969929 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7eff54034070 con 0x562a2cb69f60 2024-06-10T00:57:53.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.122+0000 7eff6d9cab80 10 client.15675 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:57:53.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.122+0000 7eff6d9cab80 10 client.15675 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-10T00:57:53.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.122+0000 7eff6d9cab80 20 client.15675 populate_metadata read hostname 'smithi037' 2024-06-10T00:57:53.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.122+0000 7eff6d9cab80 1 --2- 172.21.15.37:0/2857969929 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x562a2cbe7be0 0x562a2cbe9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:57:53.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.122+0000 7eff6d9cab80 4 client.15675 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:57:53.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.122+0000 7eff6d9cab80 1 -- 172.21.15.37:0/2857969929 --> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a2cabd290 con 0x562a2cbe7be0 2024-06-10T00:57:53.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.123+0000 7eff60ee3700 1 --2- 172.21.15.37:0/2857969929 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x562a2cbe7be0 0x562a2cbe9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:57:53.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.123+0000 7eff60ee3700 1 --2- 172.21.15.37:0/2857969929 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x562a2cbe7be0 0x562a2cbe9ff0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c644852edff8a6cd server_cookie=68bab401d016809a in_seq=0 out_seq=0 2024-06-10T00:57:53.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.123+0000 7eff3bfff700 10 client.15675 ms_handle_connect on v2:172.21.15.167:6836/2720148065 2024-06-10T00:57:53.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.123+0000 7eff3bfff700 1 -- 172.21.15.37:0/2857969929 <== mds.2 v2:172.21.15.167:6836/2720148065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a2cabd290 con 0x562a2cbe7be0 2024-06-10T00:57:53.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.123+0000 7eff3bfff700 10 client.15675 handle_command_reply: tid=0 2024-06-10T00:57:53.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.123+0000 7eff6d9cab80 1 client.15675 shutdown 2024-06-10T00:57:53.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.124+0000 7eff6d9cab80 1 -- 172.21.15.37:0/2857969929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a2cb69f60 msgr2=0x562a2cb76260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:53.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.124+0000 7eff6d9cab80 1 --2- 172.21.15.37:0/2857969929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a2cb69f60 0x562a2cb76260 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7eff5400b780 tx=0x7eff5400be90 comp rx=0 tx=0).stop 2024-06-10T00:57:53.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.124+0000 7eff6d9cab80 1 -- 172.21.15.37:0/2857969929 shutdown_connections 2024-06-10T00:57:53.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.124+0000 7eff6d9cab80 1 --2- 172.21.15.37:0/2857969929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a2cb69f60 0x562a2cb76260 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.124+0000 7eff6d9cab80 1 --2- 172.21.15.37:0/2857969929 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x562a2cbe7be0 0x562a2cbe9ff0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.124+0000 7eff3bfff700 0 client.15675 ms_handle_reset on v2:172.21.15.167:6836/2720148065 2024-06-10T00:57:53.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.124+0000 7eff6d9cab80 1 --2- 172.21.15.37:0/2857969929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a2cb63f20 0x562a2cb85c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.124+0000 7eff6d9cab80 1 --2- 172.21.15.37:0/2857969929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562a2cb69640 0x562a2cb75d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.124+0000 7eff6d9cab80 1 -- 172.21.15.37:0/2857969929 >> 172.21.15.37:0/2857969929 conn(0x562a2ca8ecc0 msgr2=0x562a2cb6b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:53.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.124+0000 7eff6d9cab80 1 -- 172.21.15.37:0/2857969929 shutdown_connections 2024-06-10T00:57:53.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.124+0000 7eff6d9cab80 1 -- 172.21.15.37:0/2857969929 wait complete. 2024-06-10T00:57:53.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.124+0000 7eff6d9cab80 20 client.15675 trim_cache size 0 max 16384 2024-06-10T00:57:53.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.125+0000 7eff63147700 1 -- 172.21.15.37:0/1956643318 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7eff4c0a2ca0 msgr2=0x7eff4c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:53.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.125+0000 7eff63147700 1 --2- 172.21.15.37:0/1956643318 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7eff4c0a2ca0 0x7eff4c0a5160 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7eff50009fd0 tx=0x7eff50009260 comp rx=0 tx=0).stop 2024-06-10T00:57:53.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.125+0000 7eff63147700 1 -- 172.21.15.37:0/1956643318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff5c101d40 msgr2=0x7eff5c197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:57:53.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.125+0000 7eff63147700 1 --2- 172.21.15.37:0/1956643318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff5c101d40 0x7eff5c197240 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7eff4800dac0 tx=0x7eff4800ddd0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.125+0000 7eff63147700 1 -- 172.21.15.37:0/1956643318 shutdown_connections 2024-06-10T00:57:53.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.125+0000 7eff63147700 1 --2- 172.21.15.37:0/1956643318 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7eff4c0a2ca0 0x7eff4c0a5160 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.125+0000 7eff63147700 1 --2- 172.21.15.37:0/1956643318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff5c1073f0 0x7eff5c19b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.125+0000 7eff63147700 1 --2- 172.21.15.37:0/1956643318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff5c101380 0x7eff5c196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.125+0000 7eff63147700 1 --2- 172.21.15.37:0/1956643318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff5c101d40 0x7eff5c197240 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:57:53.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.125+0000 7eff63147700 1 -- 172.21.15.37:0/1956643318 >> 172.21.15.37:0/1956643318 conn(0x7eff5c078300 msgr2=0x7eff5c0ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:57:53.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.125+0000 7eff63147700 1 -- 172.21.15.37:0/1956643318 shutdown_connections 2024-06-10T00:57:53.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:57:53.125+0000 7eff63147700 1 -- 172.21.15.37:0/1956643318 wait complete. 2024-06-10T00:57:54.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:54 smithi167 ceph-mon[36917]: pgmap v466: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-06-10T00:57:54.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:54 smithi094 ceph-mon[36708]: pgmap v466: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-06-10T00:57:54.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:54 smithi037 ceph-mon[32192]: pgmap v466: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-06-10T00:57:56.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:56 smithi167 ceph-mon[36917]: pgmap v467: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 8 op/s 2024-06-10T00:57:56.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:56 smithi094 ceph-mon[36708]: pgmap v467: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 8 op/s 2024-06-10T00:57:56.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:56 smithi037 ceph-mon[32192]: pgmap v467: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 8 op/s 2024-06-10T00:57:57.838 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T00:57:58.193 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:57:58.490 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:57:58 smithi037 ceph-mon[32192]: pgmap v468: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-06-10T00:57:58.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:57:58 smithi167 ceph-mon[36917]: pgmap v468: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-06-10T00:57:58.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:57:58 smithi094 ceph-mon[36708]: pgmap v468: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-06-10T00:58:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.104+0000 7f2002b28700 1 -- 172.21.15.37:0/2967528029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ffc06a170 msgr2=0x7f1ffc06a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.104+0000 7f2002b28700 1 --2- 172.21.15.37:0/2967528029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ffc06a170 0x7f1ffc06a5f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009990 tx=0x7f1fe8009ca0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.104+0000 7f2002b28700 1 -- 172.21.15.37:0/2967528029 shutdown_connections 2024-06-10T00:58:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.104+0000 7f2002b28700 1 --2- 172.21.15.37:0/2967528029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ffc06ab30 0x7f1ffc10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.104+0000 7f2002b28700 1 --2- 172.21.15.37:0/2967528029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ffc108500 0x7f1ffc1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.104+0000 7f2002b28700 1 --2- 172.21.15.37:0/2967528029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ffc06a170 0x7f1ffc06a5f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.104+0000 7f2002b28700 1 -- 172.21.15.37:0/2967528029 >> 172.21.15.37:0/2967528029 conn(0x7f1ffc078340 msgr2=0x7f1ffc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:00.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.104+0000 7f2002b28700 1 -- 172.21.15.37:0/2967528029 shutdown_connections 2024-06-10T00:58:00.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.104+0000 7f2002b28700 1 -- 172.21.15.37:0/2967528029 wait complete. 2024-06-10T00:58:00.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.104+0000 7f2002b28700 1 Processor -- start 2024-06-10T00:58:00.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.105+0000 7f2002b28700 1 -- start start 2024-06-10T00:58:00.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.105+0000 7f2002b28700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ffc06a170 0x7f1ffc19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:00.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.105+0000 7f2002b28700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ffc06ab30 0x7f1ffc19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:00.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.105+0000 7f2002b28700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ffc108500 0x7f1ffc1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:00.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.105+0000 7f2002b28700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc112780 con 0x7f1ffc06a170 2024-06-10T00:58:00.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.105+0000 7f2002b28700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc112600 con 0x7f1ffc06ab30 2024-06-10T00:58:00.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.105+0000 7f2002b28700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc112900 con 0x7f1ffc108500 2024-06-10T00:58:00.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.105+0000 7f20008c4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ffc06a170 0x7f1ffc19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:00.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.105+0000 7f20008c4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ffc06a170 0x7f1ffc19e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51226/0 (socket says 172.21.15.37:51226) 2024-06-10T00:58:00.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.105+0000 7f20008c4700 1 -- 172.21.15.37:0/4121630333 learned_addr learned my addr 172.21.15.37:0/4121630333 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:00.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.105+0000 7f1ffbfff700 1 --2- 172.21.15.37:0/4121630333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ffc06ab30 0x7f1ffc19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:00.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.105+0000 7f20010c5700 1 --2- 172.21.15.37:0/4121630333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ffc108500 0x7f1ffc1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:00.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.106+0000 7f1ffbfff700 1 -- 172.21.15.37:0/4121630333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ffc108500 msgr2=0x7f1ffc1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:00.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.106+0000 7f1ffbfff700 1 --2- 172.21.15.37:0/4121630333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ffc108500 0x7f1ffc1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.106+0000 7f1ffbfff700 1 -- 172.21.15.37:0/4121630333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ffc06a170 msgr2=0x7f1ffc19e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:00.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.106+0000 7f1ffbfff700 1 --2- 172.21.15.37:0/4121630333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ffc06a170 0x7f1ffc19e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.106+0000 7f1ffbfff700 1 -- 172.21.15.37:0/4121630333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ffc199120 con 0x7f1ffc06ab30 2024-06-10T00:58:00.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.106+0000 7f1ffbfff700 1 --2- 172.21.15.37:0/4121630333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ffc06ab30 0x7f1ffc19ed40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80059d0 tx=0x7f1fe800bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:00.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.106+0000 7f1ff9ffb700 1 -- 172.21.15.37:0/4121630333 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe802b070 con 0x7f1ffc06ab30 2024-06-10T00:58:00.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.106+0000 7f2002b28700 1 -- 172.21.15.37:0/4121630333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ffc1993b0 con 0x7f1ffc06ab30 2024-06-10T00:58:00.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.106+0000 7f1ff9ffb700 1 -- 172.21.15.37:0/4121630333 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fe8003680 con 0x7f1ffc06ab30 2024-06-10T00:58:00.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.106+0000 7f1ff9ffb700 1 -- 172.21.15.37:0/4121630333 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe80266d0 con 0x7f1ffc06ab30 2024-06-10T00:58:00.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.107+0000 7f2002b28700 1 -- 172.21.15.37:0/4121630333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ffc1997a0 con 0x7f1ffc06ab30 2024-06-10T00:58:00.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.110+0000 7f1ff9ffb700 1 -- 172.21.15.37:0/4121630333 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1fe802e070 con 0x7f1ffc06ab30 2024-06-10T00:58:00.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.111+0000 7f1ff9ffb700 1 --2- 172.21.15.37:0/4121630333 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1fec0a2d20 0x7f1fec0a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:00.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.111+0000 7f1ff9ffb700 1 -- 172.21.15.37:0/4121630333 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f1fe80d47b0 con 0x7f1ffc06ab30 2024-06-10T00:58:00.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.111+0000 7f20008c4700 1 --2- 172.21.15.37:0/4121630333 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1fec0a2d20 0x7f1fec0a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:00.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.112+0000 7f20008c4700 1 --2- 172.21.15.37:0/4121630333 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1fec0a2d20 0x7f1fec0a51e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc199b00 tx=0x7f1ff0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:00.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.112+0000 7f200d3abb80 1 Processor -- start 2024-06-10T00:58:00.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.112+0000 7f200d3abb80 1 -- start start 2024-06-10T00:58:00.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.113+0000 7f200d3abb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c637afcf60 0x55c637afd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:00.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.113+0000 7f200d3abb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c637afd880 0x55c637af7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.113+0000 7f200d3abb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c637af77a0 0x55c637af7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.113+0000 7f1ffbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c637afd880 0x55c637af7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.113+0000 7f1ffbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c637afd880 0x55c637af7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51232/0 (socket says 172.21.15.37:51232) 2024-06-10T00:58:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.113+0000 7f20008c4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c637afcf60 0x55c637afd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.113+0000 7f200d3abb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c6379e9f60 con 0x55c637afd880 2024-06-10T00:58:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.113+0000 7f200d3abb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c637a48850 con 0x55c637afcf60 2024-06-10T00:58:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.113+0000 7f20008c4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c637afcf60 0x55c637afd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:34456/0 (socket says 172.21.15.37:34456) 2024-06-10T00:58:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.113+0000 7f200d3abb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c6379b0460 con 0x55c637af77a0 2024-06-10T00:58:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.113+0000 7f20008c4700 1 -- 172.21.15.37:0/4236088417 learned_addr learned my addr 172.21.15.37:0/4236088417 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.113+0000 7f20008c4700 1 -- 172.21.15.37:0/4236088417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c637af77a0 msgr2=0x55c637af7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.113+0000 7f20008c4700 1 --2- 172.21.15.37:0/4236088417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c637af77a0 0x55c637af7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.113+0000 7f20008c4700 1 -- 172.21.15.37:0/4236088417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c637afd880 msgr2=0x55c637af7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:00.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.113+0000 7f20008c4700 1 --2- 172.21.15.37:0/4236088417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c637afd880 0x55c637af7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.113+0000 7f20008c4700 1 -- 172.21.15.37:0/4236088417 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6379978d0 con 0x55c637afcf60 2024-06-10T00:58:00.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.113+0000 7f1ffbfff700 1 --2- 172.21.15.37:0/4236088417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c637afd880 0x55c637af7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:58:00.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.114+0000 7f20008c4700 1 --2- 172.21.15.37:0/4236088417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c637afcf60 0x55c637afd340 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000f550 tx=0x7f1ff0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68e1232c0615628b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:00.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.114+0000 7f1fe6ffd700 1 -- 172.21.15.37:0/4236088417 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0026400 con 0x55c637afcf60 2024-06-10T00:58:00.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.114+0000 7f1fe6ffd700 1 -- 172.21.15.37:0/4236088417 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ff001e8d0 con 0x55c637afcf60 2024-06-10T00:58:00.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.114+0000 7f1fe6ffd700 1 -- 172.21.15.37:0/4236088417 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff001c860 con 0x55c637afcf60 2024-06-10T00:58:00.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.115+0000 7f200d3abb80 1 -- 172.21.15.37:0/4236088417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c637afcf60 msgr2=0x55c637afd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:00.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.115+0000 7f200d3abb80 1 --2- 172.21.15.37:0/4236088417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c637afcf60 0x55c637afd340 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000f550 tx=0x7f1ff0014000 comp rx=0 tx=0).stop 2024-06-10T00:58:00.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.115+0000 7f200d3abb80 1 -- 172.21.15.37:0/4236088417 shutdown_connections 2024-06-10T00:58:00.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.115+0000 7f200d3abb80 1 --2- 172.21.15.37:0/4236088417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c637af77a0 0x55c637af7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.115+0000 7f200d3abb80 1 --2- 172.21.15.37:0/4236088417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c637afcf60 0x55c637afd340 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.115+0000 7f200d3abb80 1 --2- 172.21.15.37:0/4236088417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c637afd880 0x55c637af7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.115+0000 7f200d3abb80 1 -- 172.21.15.37:0/4236088417 >> 172.21.15.37:0/4236088417 conn(0x55c637a24080 msgr2=0x55c637af2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:00.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.115+0000 7f200d3abb80 1 -- 172.21.15.37:0/4236088417 shutdown_connections 2024-06-10T00:58:00.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.115+0000 7f200d3abb80 1 -- 172.21.15.37:0/4236088417 wait complete. 2024-06-10T00:58:00.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.116+0000 7f200d3abb80 1 -- start start 2024-06-10T00:58:00.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.116+0000 7f200d3abb80 1 Processor -- start 2024-06-10T00:58:00.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.116+0000 7f200d3abb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c637af77a0 0x55c637b036c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:00.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.116+0000 7f200d3abb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c637afcf60 0x55c637b009c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.116+0000 7f200d3abb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c637afd880 0x55c637b00f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.117+0000 7f20010c5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c637afd880 0x55c637b00f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.117+0000 7f20008c4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c637af77a0 0x55c637b036c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.117+0000 7f20010c5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c637afd880 0x55c637b00f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51244/0 (socket says 172.21.15.37:51244) 2024-06-10T00:58:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.117+0000 7f1ffbfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c637afcf60 0x55c637b009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.117+0000 7f20008c4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c637af77a0 0x55c637b036c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:48844/0 (socket says 172.21.15.37:48844) 2024-06-10T00:58:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.117+0000 7f20010c5700 1 -- 172.21.15.37:0/2328405096 learned_addr learned my addr 172.21.15.37:0/2328405096 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.117+0000 7f200d3abb80 1 -- 172.21.15.37:0/2328405096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c6379e8fc0 con 0x55c637afd880 2024-06-10T00:58:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.117+0000 7f200d3abb80 1 -- 172.21.15.37:0/2328405096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c637a44810 con 0x55c637afcf60 2024-06-10T00:58:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.117+0000 7f200d3abb80 1 -- 172.21.15.37:0/2328405096 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c637b04260 con 0x55c637af77a0 2024-06-10T00:58:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.117+0000 7f20010c5700 1 -- 172.21.15.37:0/2328405096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c637af77a0 msgr2=0x55c637b036c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.117+0000 7f20010c5700 1 --2- 172.21.15.37:0/2328405096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c637af77a0 0x55c637b036c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.117+0000 7f20010c5700 1 -- 172.21.15.37:0/2328405096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c637afcf60 msgr2=0x55c637b009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.117+0000 7f20010c5700 1 --2- 172.21.15.37:0/2328405096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c637afcf60 0x55c637b009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.117+0000 7f20010c5700 1 -- 172.21.15.37:0/2328405096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c637a0e750 con 0x55c637afd880 2024-06-10T00:58:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.118+0000 7f20010c5700 1 --2- 172.21.15.37:0/2328405096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c637afd880 0x55c637b00f00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400ff10 tx=0x7f1ff400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3168ba15df7fc257 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.118+0000 7f1fdbfff700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:58:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.118+0000 7f1fdbfff700 1 -- 172.21.15.37:0/2328405096 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4029030 con 0x55c637afd880 2024-06-10T00:58:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.118+0000 7f1fdbfff700 1 -- 172.21.15.37:0/2328405096 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f1ff4010040 con 0x55c637afd880 2024-06-10T00:58:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.118+0000 7f200d3abb80 10 client.0 _ll_register_callbacks cb 0x55c6379de5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:58:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.118+0000 7f200d3abb80 1 -- 172.21.15.37:0/2328405096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ff001e8d0 con 0x55c637afd880 2024-06-10T00:58:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.118+0000 7f1fdbfff700 1 -- 172.21.15.37:0/2328405096 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1ff4033050 con 0x55c637afd880 2024-06-10T00:58:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.118+0000 7f200d3abb80 10 client.15687 fetch_fsmap learned FSMap version 27 2024-06-10T00:58:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.118+0000 7f200d3abb80 1 -- 172.21.15.37:0/2328405096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55c637b05830 con 0x55c637afd880 2024-06-10T00:58:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.119+0000 7f1fdbfff700 1 -- 172.21.15.37:0/2328405096 <== mon.0 v2:172.21.15.37:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1ff400d9c0 con 0x55c637afd880 2024-06-10T00:58:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.119+0000 7f200d3abb80 10 client.15687 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:58:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.119+0000 7f200d3abb80 10 client.15687 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-06-10T00:58:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.119+0000 7f200d3abb80 20 client.15687 populate_metadata read hostname 'smithi037' 2024-06-10T00:58:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.119+0000 7f200d3abb80 1 --2- 172.21.15.37:0/2328405096 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55c637b07e00 0x55c637b0a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.119+0000 7f200d3abb80 4 client.15687 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:58:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.119+0000 7f200d3abb80 1 -- 172.21.15.37:0/2328405096 --> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c637a55d50 con 0x55c637b07e00 2024-06-10T00:58:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.119+0000 7f20008c4700 1 --2- 172.21.15.37:0/2328405096 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55c637b07e00 0x55c637b0a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.119+0000 7f1fdbfff700 10 client.15687 ms_handle_connect on v2:172.21.15.94:6838/946269331 2024-06-10T00:58:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.119+0000 7f20008c4700 1 --2- 172.21.15.37:0/2328405096 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55c637b07e00 0x55c637b0a290 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=27e6f2d637066016 server_cookie=6db69d8c296aac4 in_seq=0 out_seq=0 2024-06-10T00:58:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.120+0000 7f1fdbfff700 1 -- 172.21.15.37:0/2328405096 <== mds.1 v2:172.21.15.94:6838/946269331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c637a55d50 con 0x55c637b07e00 2024-06-10T00:58:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.120+0000 7f1fdbfff700 10 client.15687 handle_command_reply: tid=0 2024-06-10T00:58:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.120+0000 7f200d3abb80 1 client.15687 shutdown 2024-06-10T00:58:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.120+0000 7f200d3abb80 1 -- 172.21.15.37:0/2328405096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c637afd880 msgr2=0x55c637b00f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.120+0000 7f200d3abb80 1 --2- 172.21.15.37:0/2328405096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c637afd880 0x55c637b00f00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400ff10 tx=0x7f1ff400d430 comp rx=0 tx=0).stop 2024-06-10T00:58:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.121+0000 7f200d3abb80 1 -- 172.21.15.37:0/2328405096 shutdown_connections 2024-06-10T00:58:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.121+0000 7f200d3abb80 1 --2- 172.21.15.37:0/2328405096 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55c637b07e00 0x55c637b0a290 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.121+0000 7f1fdbfff700 0 client.15687 ms_handle_reset on v2:172.21.15.94:6838/946269331 2024-06-10T00:58:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.121+0000 7f200d3abb80 1 --2- 172.21.15.37:0/2328405096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c637afd880 0x55c637b00f00 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.121+0000 7f200d3abb80 1 --2- 172.21.15.37:0/2328405096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c637af77a0 0x55c637b036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.121+0000 7f200d3abb80 1 --2- 172.21.15.37:0/2328405096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c637afcf60 0x55c637b009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.121+0000 7f200d3abb80 1 -- 172.21.15.37:0/2328405096 >> 172.21.15.37:0/2328405096 conn(0x55c637a24080 msgr2=0x55c637af1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.121+0000 7f200d3abb80 1 -- 172.21.15.37:0/2328405096 shutdown_connections 2024-06-10T00:58:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.121+0000 7f200d3abb80 1 -- 172.21.15.37:0/2328405096 wait complete. 2024-06-10T00:58:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.121+0000 7f200d3abb80 20 client.15687 trim_cache size 0 max 16384 2024-06-10T00:58:00.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.151+0000 7f200d3abb80 1 Processor -- start 2024-06-10T00:58:00.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.151+0000 7f200d3abb80 1 -- start start 2024-06-10T00:58:00.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.151+0000 7f200d3abb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c637af77a0 0x55c637afc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:00.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.151+0000 7f200d3abb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c637afcf60 0x55c637b03200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:00.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.151+0000 7f200d3abb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c637afd880 0x55c637af1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:00.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.151+0000 7f20008c4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c637af77a0 0x55c637afc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:00.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.151+0000 7f20008c4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c637af77a0 0x55c637afc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:34468/0 (socket says 172.21.15.37:34468) 2024-06-10T00:58:00.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.151+0000 7f1ffbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c637afcf60 0x55c637b03200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:00.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.151+0000 7f1ffbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c637afcf60 0x55c637b03200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:48850/0 (socket says 172.21.15.37:48850) 2024-06-10T00:58:00.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.152+0000 7f20010c5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c637afd880 0x55c637af1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:00.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.152+0000 7f20010c5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c637afd880 0x55c637af1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51260/0 (socket says 172.21.15.37:51260) 2024-06-10T00:58:00.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.152+0000 7f200d3abb80 1 -- 172.21.15.37:0/3214399287 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc1ae920 con 0x55c637afd880 2024-06-10T00:58:00.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.152+0000 7f20008c4700 1 -- 172.21.15.37:0/3214399287 learned_addr learned my addr 172.21.15.37:0/3214399287 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:00.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.152+0000 7f200d3abb80 1 -- 172.21.15.37:0/3214399287 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc1ac1b0 con 0x55c637af77a0 2024-06-10T00:58:00.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.152+0000 7f200d3abb80 1 -- 172.21.15.37:0/3214399287 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc1aa000 con 0x55c637afcf60 2024-06-10T00:58:00.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.152+0000 7f20008c4700 1 -- 172.21.15.37:0/3214399287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c637afcf60 msgr2=0x55c637b03200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:00.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.152+0000 7f20008c4700 1 --2- 172.21.15.37:0/3214399287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c637afcf60 0x55c637b03200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.152+0000 7f20008c4700 1 -- 172.21.15.37:0/3214399287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c637afd880 msgr2=0x55c637af1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.152+0000 7f20008c4700 1 --2- 172.21.15.37:0/3214399287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c637afd880 0x55c637af1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.152+0000 7f20008c4700 1 -- 172.21.15.37:0/3214399287 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c637afef50 con 0x55c637af77a0 2024-06-10T00:58:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.152+0000 7f1ffbfff700 1 --2- 172.21.15.37:0/3214399287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c637afcf60 0x55c637b03200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:58:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.152+0000 7f20008c4700 1 --2- 172.21.15.37:0/3214399287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c637af77a0 0x55c637afc1e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000f550 tx=0x7f1ff001c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14ff761d4b499438 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.152+0000 7f1fe67fc700 1 -- 172.21.15.37:0/3214399287 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff001e5f0 con 0x55c637af77a0 2024-06-10T00:58:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.152+0000 7f1fe67fc700 1 -- 172.21.15.37:0/3214399287 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ff0038070 con 0x55c637af77a0 2024-06-10T00:58:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.153+0000 7f1fe67fc700 1 -- 172.21.15.37:0/3214399287 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0004970 con 0x55c637af77a0 2024-06-10T00:58:00.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.153+0000 7f200d3abb80 1 -- 172.21.15.37:0/3214399287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c637af77a0 msgr2=0x55c637afc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:00.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.153+0000 7f200d3abb80 1 --2- 172.21.15.37:0/3214399287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c637af77a0 0x55c637afc1e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000f550 tx=0x7f1ff001c380 comp rx=0 tx=0).stop 2024-06-10T00:58:00.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.153+0000 7f200d3abb80 1 -- 172.21.15.37:0/3214399287 shutdown_connections 2024-06-10T00:58:00.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.153+0000 7f200d3abb80 1 --2- 172.21.15.37:0/3214399287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c637afd880 0x55c637af1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.153+0000 7f200d3abb80 1 --2- 172.21.15.37:0/3214399287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c637af77a0 0x55c637afc1e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.154+0000 7f200d3abb80 1 --2- 172.21.15.37:0/3214399287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c637afcf60 0x55c637b03200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.154+0000 7f200d3abb80 1 -- 172.21.15.37:0/3214399287 >> 172.21.15.37:0/3214399287 conn(0x55c637a24080 msgr2=0x55c637afbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.154+0000 7f200d3abb80 1 -- 172.21.15.37:0/3214399287 shutdown_connections 2024-06-10T00:58:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.154+0000 7f200d3abb80 1 -- 172.21.15.37:0/3214399287 wait complete. 2024-06-10T00:58:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.154+0000 7f200d3abb80 1 -- start start 2024-06-10T00:58:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.154+0000 7f200d3abb80 1 Processor -- start 2024-06-10T00:58:00.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.155+0000 7f200d3abb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c637af77a0 0x55c637b0dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:00.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.155+0000 7f200d3abb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c637afcf60 0x55c637b0e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:00.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.155+0000 7f200d3abb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c637afd880 0x55c637b0e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:00.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.155+0000 7f20008c4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c637af77a0 0x55c637b0dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:00.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.155+0000 7f1ffbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c637afcf60 0x55c637b0e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:00.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.155+0000 7f20008c4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c637af77a0 0x55c637b0dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51266/0 (socket says 172.21.15.37:51266) 2024-06-10T00:58:00.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.155+0000 7f1ffbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c637afcf60 0x55c637b0e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:48858/0 (socket says 172.21.15.37:48858) 2024-06-10T00:58:00.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.155+0000 7f20008c4700 1 -- 172.21.15.37:0/1300218762 learned_addr learned my addr 172.21.15.37:0/1300218762 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:00.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.155+0000 7f200d3abb80 1 -- 172.21.15.37:0/1300218762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc1aa670 con 0x55c637af77a0 2024-06-10T00:58:00.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.155+0000 7f200d3abb80 1 -- 172.21.15.37:0/1300218762 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc1abae0 con 0x55c637afd880 2024-06-10T00:58:00.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.155+0000 7f200d3abb80 1 -- 172.21.15.37:0/1300218762 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc1aad40 con 0x55c637afcf60 2024-06-10T00:58:00.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.155+0000 7f20010c5700 1 --2- 172.21.15.37:0/1300218762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c637afd880 0x55c637b0e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:00.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.156+0000 7f200d3abb80 10 client.0 _ll_register_callbacks cb 0x55c6379de5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:58:00.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.156+0000 7f20008c4700 1 -- 172.21.15.37:0/1300218762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c637afcf60 msgr2=0x55c637b0e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:00.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.156+0000 7f20008c4700 1 --2- 172.21.15.37:0/1300218762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c637afcf60 0x55c637b0e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.156+0000 7f20008c4700 1 -- 172.21.15.37:0/1300218762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c637afd880 msgr2=0x55c637b0e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:00.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.156+0000 7f20008c4700 1 --2- 172.21.15.37:0/1300218762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c637afd880 0x55c637b0e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.156+0000 7f20008c4700 1 -- 172.21.15.37:0/1300218762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c637b7b030 con 0x55c637af77a0 2024-06-10T00:58:00.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.156+0000 7f200d3abb80 1 -- 172.21.15.37:0/1300218762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ff0038070 con 0x55c637af77a0 2024-06-10T00:58:00.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.156+0000 7f20008c4700 1 --2- 172.21.15.37:0/1300218762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c637af77a0 0x55c637b0dac0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000a010 tx=0x7f1ff0009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8376d51fcf6b0bff server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:00.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.156+0000 7f1fdbfff700 10 client.15693 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:58:00.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.156+0000 7f1fdbfff700 1 -- 172.21.15.37:0/1300218762 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0005530 con 0x55c637af77a0 2024-06-10T00:58:00.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.156+0000 7f1fdbfff700 1 -- 172.21.15.37:0/1300218762 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f1ff001e3f0 con 0x55c637af77a0 2024-06-10T00:58:00.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.156+0000 7f1fdbfff700 1 -- 172.21.15.37:0/1300218762 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1ff001e650 con 0x55c637af77a0 2024-06-10T00:58:00.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.156+0000 7f200d3abb80 10 client.15693 fetch_fsmap learned FSMap version 27 2024-06-10T00:58:00.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.157+0000 7f200d3abb80 1 -- 172.21.15.37:0/1300218762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55c637b7c140 con 0x55c637af77a0 2024-06-10T00:58:00.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.157+0000 7f1fdbfff700 1 -- 172.21.15.37:0/1300218762 <== mon.0 v2:172.21.15.37:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1ff001ea80 con 0x55c637af77a0 2024-06-10T00:58:00.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.157+0000 7f200d3abb80 10 client.15693 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:58:00.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.157+0000 7f200d3abb80 10 client.15693 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-06-10T00:58:00.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.157+0000 7f200d3abb80 20 client.15693 populate_metadata read hostname 'smithi037' 2024-06-10T00:58:00.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.157+0000 7f200d3abb80 1 --2- 172.21.15.37:0/1300218762 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55c637b07e00 0x55c637b7e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:00.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.157+0000 7f200d3abb80 4 client.15693 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:58:00.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.157+0000 7f200d3abb80 1 -- 172.21.15.37:0/1300218762 --> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f1ff0038070 con 0x55c637b07e00 2024-06-10T00:58:00.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.157+0000 7f1ffbfff700 1 --2- 172.21.15.37:0/1300218762 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55c637b07e00 0x55c637b7e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:00.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.158+0000 7f1ffbfff700 1 --2- 172.21.15.37:0/1300218762 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55c637b07e00 0x55c637b7e3c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d0d52aa9f276b9d7 server_cookie=fe461938df2f4c6f in_seq=0 out_seq=0 2024-06-10T00:58:00.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.158+0000 7f1fdbfff700 10 client.15693 ms_handle_connect on v2:172.21.15.94:6838/946269331 2024-06-10T00:58:00.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.158+0000 7f1fdbfff700 1 -- 172.21.15.37:0/1300218762 <== mds.1 v2:172.21.15.94:6838/946269331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f1ff0038070 con 0x55c637b07e00 2024-06-10T00:58:00.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.158+0000 7f1fdbfff700 10 client.15693 handle_command_reply: tid=0 2024-06-10T00:58:00.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.158+0000 7f200d3abb80 1 client.15693 shutdown 2024-06-10T00:58:00.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.158+0000 7f200d3abb80 1 -- 172.21.15.37:0/1300218762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c637af77a0 msgr2=0x55c637b0dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:00.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.158+0000 7f200d3abb80 1 --2- 172.21.15.37:0/1300218762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c637af77a0 0x55c637b0dac0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000a010 tx=0x7f1ff0009e40 comp rx=0 tx=0).stop 2024-06-10T00:58:00.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.158+0000 7f200d3abb80 1 -- 172.21.15.37:0/1300218762 shutdown_connections 2024-06-10T00:58:00.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.158+0000 7f200d3abb80 1 --2- 172.21.15.37:0/1300218762 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55c637b07e00 0x55c637b7e3c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.158+0000 7f1fdbfff700 0 client.15693 ms_handle_reset on v2:172.21.15.94:6838/946269331 2024-06-10T00:58:00.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.158+0000 7f200d3abb80 1 --2- 172.21.15.37:0/1300218762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c637afd880 0x55c637b0e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.158+0000 7f200d3abb80 1 --2- 172.21.15.37:0/1300218762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c637af77a0 0x55c637b0dac0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.158+0000 7f200d3abb80 1 --2- 172.21.15.37:0/1300218762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c637afcf60 0x55c637b0e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.158+0000 7f200d3abb80 1 -- 172.21.15.37:0/1300218762 >> 172.21.15.37:0/1300218762 conn(0x55c637a24080 msgr2=0x55c637afc360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:00.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.159+0000 7f200d3abb80 1 -- 172.21.15.37:0/1300218762 shutdown_connections 2024-06-10T00:58:00.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.159+0000 7f200d3abb80 1 -- 172.21.15.37:0/1300218762 wait complete. 2024-06-10T00:58:00.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.159+0000 7f200d3abb80 20 client.15693 trim_cache size 0 max 16384 2024-06-10T00:58:00.175 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T00:58:00.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.159+0000 7f2002b28700 1 -- 172.21.15.37:0/4121630333 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1fec0a2d20 msgr2=0x7f1fec0a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:00.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.159+0000 7f2002b28700 1 --2- 172.21.15.37:0/4121630333 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1fec0a2d20 0x7f1fec0a51e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc199b00 tx=0x7f1ff0009260 comp rx=0 tx=0).stop 2024-06-10T00:58:00.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.159+0000 7f2002b28700 1 -- 172.21.15.37:0/4121630333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ffc06ab30 msgr2=0x7f1ffc19ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:00.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.159+0000 7f2002b28700 1 --2- 172.21.15.37:0/4121630333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ffc06ab30 0x7f1ffc19ed40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80059d0 tx=0x7f1fe800bab0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.160+0000 7f2002b28700 1 -- 172.21.15.37:0/4121630333 shutdown_connections 2024-06-10T00:58:00.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.160+0000 7f2002b28700 1 --2- 172.21.15.37:0/4121630333 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1fec0a2d20 0x7f1fec0a51e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.160+0000 7f2002b28700 1 --2- 172.21.15.37:0/4121630333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ffc108500 0x7f1ffc1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.160+0000 7f2002b28700 1 --2- 172.21.15.37:0/4121630333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ffc06a170 0x7f1ffc19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.160+0000 7f2002b28700 1 --2- 172.21.15.37:0/4121630333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ffc06ab30 0x7f1ffc19ed40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:00.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.160+0000 7f2002b28700 1 -- 172.21.15.37:0/4121630333 >> 172.21.15.37:0/4121630333 conn(0x7f1ffc078340 msgr2=0x7f1ffc0ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:00.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.160+0000 7f2002b28700 1 -- 172.21.15.37:0/4121630333 shutdown_connections 2024-06-10T00:58:00.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:00.160+0000 7f2002b28700 1 -- 172.21.15.37:0/4121630333 wait complete. 2024-06-10T00:58:00.519 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:00 smithi037 ceph-mon[32192]: pgmap v469: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1023 B/s wr, 8 op/s 2024-06-10T00:58:00.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:00 smithi167 ceph-mon[36917]: pgmap v469: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1023 B/s wr, 8 op/s 2024-06-10T00:58:00.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:00 smithi094 ceph-mon[36708]: pgmap v469: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1023 B/s wr, 8 op/s 2024-06-10T00:58:00.958 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T00:58:01.254 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:58:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.169+0000 7f77c6e12640 1 -- 172.21.15.37:0/2918540636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77c00ffce0 msgr2=0x7f77c0100100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.169+0000 7f77c6e12640 1 --2- 172.21.15.37:0/2918540636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77c00ffce0 0x7f77c0100100 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009930 tx=0x7f77b402e830 comp rx=0 tx=0).stop 2024-06-10T00:58:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.169+0000 7f77c6e12640 1 -- 172.21.15.37:0/2918540636 shutdown_connections 2024-06-10T00:58:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.169+0000 7f77c6e12640 1 --2- 172.21.15.37:0/2918540636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77c0103670 0x7f77c0109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.169+0000 7f77c6e12640 1 --2- 172.21.15.37:0/2918540636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77c0102cb0 0x7f77c0103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.169+0000 7f77c6e12640 1 --2- 172.21.15.37:0/2918540636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77c00ffce0 0x7f77c0100100 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.169+0000 7f77c6e12640 1 -- 172.21.15.37:0/2918540636 >> 172.21.15.37:0/2918540636 conn(0x7f77c00fb610 msgr2=0x7f77c00fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.169+0000 7f77c6e12640 1 -- 172.21.15.37:0/2918540636 shutdown_connections 2024-06-10T00:58:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.170+0000 7f77c6e12640 1 -- 172.21.15.37:0/2918540636 wait complete. 2024-06-10T00:58:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.170+0000 7f77c6e12640 1 Processor -- start 2024-06-10T00:58:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.170+0000 7f77c6e12640 1 -- start start 2024-06-10T00:58:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.171+0000 7f77c6e12640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77c00ffce0 0x7f77c0195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.171+0000 7f77c6e12640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77c0102cb0 0x7f77c0195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.171+0000 7f77c6e12640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77c0103670 0x7f77c019d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.171+0000 7f77c4bae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77c00ffce0 0x7f77c0195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.171+0000 7f77c6e12640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f77c010b500 con 0x7f77c00ffce0 2024-06-10T00:58:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.171+0000 7f77c4bae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77c00ffce0 0x7f77c0195aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58062/0 (socket says 172.21.15.37:58062) 2024-06-10T00:58:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.171+0000 7f77c4bae640 1 -- 172.21.15.37:0/2775447912 learned_addr learned my addr 172.21.15.37:0/2775447912 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.171+0000 7f77c6e12640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f77c010b380 con 0x7f77c0102cb0 2024-06-10T00:58:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.171+0000 7f77c6e12640 1 -- 172.21.15.37:0/2775447912 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f77c010b680 con 0x7f77c0103670 2024-06-10T00:58:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.171+0000 7f77bffff640 1 --2- 172.21.15.37:0/2775447912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77c0102cb0 0x7f77c0195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.171+0000 7f77c53af640 1 --2- 172.21.15.37:0/2775447912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77c0103670 0x7f77c019d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.171+0000 7f77c4bae640 1 -- 172.21.15.37:0/2775447912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77c0103670 msgr2=0x7f77c019d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.171+0000 7f77c4bae640 1 --2- 172.21.15.37:0/2775447912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77c0103670 0x7f77c019d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.171+0000 7f77c4bae640 1 -- 172.21.15.37:0/2775447912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77c0102cb0 msgr2=0x7f77c0195fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.171+0000 7f77c4bae640 1 --2- 172.21.15.37:0/2775447912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77c0102cb0 0x7f77c0195fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.171+0000 7f77c4bae640 1 -- 172.21.15.37:0/2775447912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c019d6c0 con 0x7f77c00ffce0 2024-06-10T00:58:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.171+0000 7f77c4bae640 1 --2- 172.21.15.37:0/2775447912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77c00ffce0 0x7f77c0195aa0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f77b4006010 tx=0x7f77b4004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.172+0000 7f77bdffb640 1 -- 172.21.15.37:0/2775447912 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b4047020 con 0x7f77c00ffce0 2024-06-10T00:58:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.172+0000 7f77bdffb640 1 -- 172.21.15.37:0/2775447912 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77b4005000 con 0x7f77c00ffce0 2024-06-10T00:58:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.172+0000 7f77c6e12640 1 -- 172.21.15.37:0/2775447912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c019d950 con 0x7f77c00ffce0 2024-06-10T00:58:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.172+0000 7f77bdffb640 1 -- 172.21.15.37:0/2775447912 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b4003850 con 0x7f77c00ffce0 2024-06-10T00:58:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.172+0000 7f77c6e12640 1 -- 172.21.15.37:0/2775447912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c019def0 con 0x7f77c00ffce0 2024-06-10T00:58:02.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.175+0000 7f77bdffb640 1 -- 172.21.15.37:0/2775447912 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f77b404a020 con 0x7f77c00ffce0 2024-06-10T00:58:02.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.176+0000 7f77c9bfc740 1 Processor -- start 2024-06-10T00:58:02.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.176+0000 7f77c9bfc740 1 -- start start 2024-06-10T00:58:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.176+0000 7f77c9bfc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594955ae010 0x5594955ae430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.176+0000 7f77c9bfc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594955aea00 0x5594955b3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.176+0000 7f77c53af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594955ae010 0x5594955ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.176+0000 7f77c53af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594955ae010 0x5594955ae430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58074/0 (socket says 172.21.15.37:58074) 2024-06-10T00:58:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.176+0000 7f77c9bfc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594955b3b70 0x5594955b6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.176+0000 7f77c53af640 1 -- 172.21.15.37:0/3655914215 learned_addr learned my addr 172.21.15.37:0/3655914215 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.176+0000 7f77c9bfc740 1 -- 172.21.15.37:0/3655914215 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5594955114f0 con 0x5594955ae010 2024-06-10T00:58:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.176+0000 7f77c9bfc740 1 -- 172.21.15.37:0/3655914215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5594954af870 con 0x5594955aea00 2024-06-10T00:58:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.176+0000 7f77c9bfc740 1 -- 172.21.15.37:0/3655914215 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55949550d4b0 con 0x5594955b3b70 2024-06-10T00:58:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.177+0000 7f77c4bae640 1 --2- 172.21.15.37:0/3655914215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594955aea00 0x5594955b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.177+0000 7f77c53af640 1 -- 172.21.15.37:0/3655914215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594955b3b70 msgr2=0x5594955b6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.177+0000 7f77bffff640 1 --2- 172.21.15.37:0/3655914215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594955b3b70 0x5594955b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.177+0000 7f77c53af640 1 --2- 172.21.15.37:0/3655914215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594955b3b70 0x5594955b6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.177+0000 7f77c53af640 1 -- 172.21.15.37:0/3655914215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594955aea00 msgr2=0x5594955b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.177+0000 7f77c53af640 1 --2- 172.21.15.37:0/3655914215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594955aea00 0x5594955b3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.177+0000 7f77c53af640 1 -- 172.21.15.37:0/3655914215 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594954cb9b0 con 0x5594955ae010 2024-06-10T00:58:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.177+0000 7f77bffff640 1 --2- 172.21.15.37:0/3655914215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594955b3b70 0x5594955b6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:58:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.177+0000 7f77c53af640 1 --2- 172.21.15.37:0/3655914215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594955ae010 0x5594955ae430 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009fc0 tx=0x7f77b8010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270c0bc67382a63b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.178+0000 7f77bdffb640 1 --2- 172.21.15.37:0/2775447912 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f77b00a2cb0 0x7f77b00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.178+0000 7f77c4bae640 1 --2- 172.21.15.37:0/2775447912 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f77b00a2cb0 0x7f77b00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.178+0000 7f77aaffd640 1 -- 172.21.15.37:0/3655914215 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8003b70 con 0x5594955ae010 2024-06-10T00:58:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.178+0000 7f77aaffd640 1 -- 172.21.15.37:0/3655914215 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77b8003d10 con 0x5594955ae010 2024-06-10T00:58:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.178+0000 7f77aaffd640 1 -- 172.21.15.37:0/3655914215 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8011aa0 con 0x5594955ae010 2024-06-10T00:58:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.178+0000 7f77bdffb640 1 -- 172.21.15.37:0/2775447912 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f77b40f3f10 con 0x7f77c00ffce0 2024-06-10T00:58:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.179+0000 7f77c9bfc740 1 -- 172.21.15.37:0/3655914215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594955ae010 msgr2=0x5594955ae430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.179+0000 7f77c4bae640 1 --2- 172.21.15.37:0/2775447912 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f77b00a2cb0 0x7f77b00a5170 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f77b40c8a60 tx=0x7f77b4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.179+0000 7f77c9bfc740 1 --2- 172.21.15.37:0/3655914215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594955ae010 0x5594955ae430 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009fc0 tx=0x7f77b8010860 comp rx=0 tx=0).stop 2024-06-10T00:58:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.179+0000 7f77c9bfc740 1 -- 172.21.15.37:0/3655914215 shutdown_connections 2024-06-10T00:58:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.179+0000 7f77c9bfc740 1 --2- 172.21.15.37:0/3655914215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594955b3b70 0x5594955b6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.179+0000 7f77c9bfc740 1 --2- 172.21.15.37:0/3655914215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594955aea00 0x5594955b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.179+0000 7f77c9bfc740 1 --2- 172.21.15.37:0/3655914215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594955ae010 0x5594955ae430 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.179+0000 7f77c9bfc740 1 -- 172.21.15.37:0/3655914215 >> 172.21.15.37:0/3655914215 conn(0x5594955a9680 msgr2=0x5594955a9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.179+0000 7f77c9bfc740 1 -- 172.21.15.37:0/3655914215 shutdown_connections 2024-06-10T00:58:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.179+0000 7f77c9bfc740 1 -- 172.21.15.37:0/3655914215 wait complete. 2024-06-10T00:58:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.180+0000 7f77c9bfc740 1 -- start start 2024-06-10T00:58:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.180+0000 7f77c9bfc740 1 Processor -- start 2024-06-10T00:58:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.180+0000 7f77c9bfc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594955ae010 0x5594955b8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.180+0000 7f77c9bfc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594955aea00 0x5594955b8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.180+0000 7f77c9bfc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594955b3b70 0x5594955bf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.180+0000 7f77c9bfc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5594955b6b20 con 0x5594955b3b70 2024-06-10T00:58:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.180+0000 7f77c53af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594955b3b70 0x5594955bf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.180+0000 7f77c53af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594955b3b70 0x5594955bf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58082/0 (socket says 172.21.15.37:58082) 2024-06-10T00:58:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.180+0000 7f77c53af640 1 -- 172.21.15.37:0/2786648660 learned_addr learned my addr 172.21.15.37:0/2786648660 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.180+0000 7f77c9bfc740 1 -- 172.21.15.37:0/2786648660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55949549aae0 con 0x5594955aea00 2024-06-10T00:58:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.180+0000 7f77c9bfc740 1 -- 172.21.15.37:0/2786648660 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5594955b6ca0 con 0x5594955ae010 2024-06-10T00:58:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.181+0000 7f77bffff640 1 --2- 172.21.15.37:0/2786648660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594955aea00 0x5594955b8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.181+0000 7f77c53af640 1 -- 172.21.15.37:0/2786648660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594955ae010 msgr2=0x5594955b8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.181+0000 7f77c53af640 1 --2- 172.21.15.37:0/2786648660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594955ae010 0x5594955b8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.181+0000 7f77c53af640 1 -- 172.21.15.37:0/2786648660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594955aea00 msgr2=0x5594955b8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.181+0000 7f77c53af640 1 --2- 172.21.15.37:0/2786648660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594955aea00 0x5594955b8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.181+0000 7f77c53af640 1 -- 172.21.15.37:0/2786648660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559495225210 con 0x5594955b3b70 2024-06-10T00:58:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.181+0000 7f77c53af640 1 --2- 172.21.15.37:0/2786648660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594955b3b70 0x5594955bf8c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f77b8003cb0 tx=0x7f77b8010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24aad0bc823174e6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.181+0000 7f77c9bfc740 10 client.0 _ll_register_callbacks cb 0x5594954d39b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:58:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.181+0000 7f77c9bfc740 1 -- 172.21.15.37:0/2786648660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77b8003d10 con 0x5594955b3b70 2024-06-10T00:58:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.181+0000 7f779ffff640 10 client.15705 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:58:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.181+0000 7f779ffff640 1 -- 172.21.15.37:0/2786648660 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8011e30 con 0x5594955b3b70 2024-06-10T00:58:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.182+0000 7f779ffff640 1 -- 172.21.15.37:0/2786648660 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f77b8004c50 con 0x5594955b3b70 2024-06-10T00:58:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.182+0000 7f779ffff640 1 -- 172.21.15.37:0/2786648660 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f77b8003ec0 con 0x5594955b3b70 2024-06-10T00:58:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.182+0000 7f77c9bfc740 10 client.15705 fetch_fsmap learned FSMap version 27 2024-06-10T00:58:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.182+0000 7f77c9bfc740 1 -- 172.21.15.37:0/2786648660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=27}) -- 0x5594952f1620 con 0x5594955b3b70 2024-06-10T00:58:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.183+0000 7f779ffff640 1 -- 172.21.15.37:0/2786648660 <== mon.0 v2:172.21.15.37:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f77b8004060 con 0x5594955b3b70 2024-06-10T00:58:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.183+0000 7f77c9bfc740 10 client.15705 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:58:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.183+0000 7f77c9bfc740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T00:58:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.183+0000 7f77c9bfc740 20 client.15705 populate_metadata read hostname 'smithi037' 2024-06-10T00:58:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.183+0000 7f77c9bfc740 1 --2- 172.21.15.37:0/2786648660 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5594955c3d20 0x5594955c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.183+0000 7f77c9bfc740 4 client.15705 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:58:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.183+0000 7f77c9bfc740 1 -- 172.21.15.37:0/2786648660 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559495524f10 con 0x5594955c3d20 2024-06-10T00:58:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.183+0000 7f77bffff640 1 --2- 172.21.15.37:0/2786648660 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5594955c3d20 0x5594955c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.184+0000 7f77bffff640 1 --2- 172.21.15.37:0/2786648660 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5594955c3d20 0x5594955c61b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d48f5078c7bb512 server_cookie=a12ef8a0a0d0b26b in_seq=0 out_seq=0 2024-06-10T00:58:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.184+0000 7f779ffff640 10 client.15705 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:02.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.185+0000 7f779ffff640 1 -- 172.21.15.37:0/2786648660 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559495524f10 con 0x5594955c3d20 2024-06-10T00:58:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.185+0000 7f779ffff640 10 client.15705 handle_command_reply: tid=0 2024-06-10T00:58:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.185+0000 7f77c9bfc740 1 client.15705 shutdown 2024-06-10T00:58:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.185+0000 7f77c9bfc740 1 -- 172.21.15.37:0/2786648660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594955b3b70 msgr2=0x5594955bf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.185+0000 7f77c9bfc740 1 --2- 172.21.15.37:0/2786648660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594955b3b70 0x5594955bf8c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f77b8003cb0 tx=0x7f77b8010fd0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.185+0000 7f77c9bfc740 1 -- 172.21.15.37:0/2786648660 shutdown_connections 2024-06-10T00:58:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.185+0000 7f77c9bfc740 1 --2- 172.21.15.37:0/2786648660 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5594955c3d20 0x5594955c61b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.185+0000 7f77c9bfc740 1 --2- 172.21.15.37:0/2786648660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594955b3b70 0x5594955bf8c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.185+0000 7f77c9bfc740 1 --2- 172.21.15.37:0/2786648660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594955aea00 0x5594955b8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.185+0000 7f77c9bfc740 1 --2- 172.21.15.37:0/2786648660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594955ae010 0x5594955b8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.185+0000 7f77c9bfc740 1 -- 172.21.15.37:0/2786648660 >> 172.21.15.37:0/2786648660 conn(0x5594955a9680 msgr2=0x5594955a87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.185+0000 7f779ffff640 0 client.15705 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.185+0000 7f77c9bfc740 1 -- 172.21.15.37:0/2786648660 shutdown_connections 2024-06-10T00:58:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.185+0000 7f77c9bfc740 1 -- 172.21.15.37:0/2786648660 wait complete. 2024-06-10T00:58:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.185+0000 7f77c9bfc740 20 client.15705 trim_cache size 0 max 16384 2024-06-10T00:58:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.216+0000 7f77c9bfc740 1 Processor -- start 2024-06-10T00:58:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.216+0000 7f77c9bfc740 1 -- start start 2024-06-10T00:58:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.216+0000 7f77c9bfc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594955ae010 0x5594955a8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.217+0000 7f77c9bfc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594955aea00 0x5594955a4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.217+0000 7f77c9bfc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594955b3b70 0x5594955b8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.217+0000 7f77c9bfc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f77c01aae90 con 0x5594955aea00 2024-06-10T00:58:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.217+0000 7f77c9bfc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f77c01a8760 con 0x5594955ae010 2024-06-10T00:58:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.217+0000 7f77c53af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594955ae010 0x5594955a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.217+0000 7f77c53af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594955ae010 0x5594955a8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:34960/0 (socket says 172.21.15.37:34960) 2024-06-10T00:58:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.217+0000 7f77c53af640 1 -- 172.21.15.37:0/3118241483 learned_addr learned my addr 172.21.15.37:0/3118241483 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.217+0000 7f77bffff640 1 --2- 172.21.15.37:0/3118241483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594955aea00 0x5594955a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.217+0000 7f77c9bfc740 1 -- 172.21.15.37:0/3118241483 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f77c01a65b0 con 0x5594955b3b70 2024-06-10T00:58:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.217+0000 7f77c53af640 1 --2- 172.21.15.37:0/3118241483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594955b3b70 0x5594955b8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.217+0000 7f77bffff640 1 -- 172.21.15.37:0/3118241483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594955b3b70 msgr2=0x5594955b8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.217+0000 7f77bffff640 1 --2- 172.21.15.37:0/3118241483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594955b3b70 0x5594955b8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.217+0000 7f77bffff640 1 -- 172.21.15.37:0/3118241483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594955ae010 msgr2=0x5594955a8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.217+0000 7f77bffff640 1 --2- 172.21.15.37:0/3118241483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594955ae010 0x5594955a8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.217+0000 7f77bffff640 1 -- 172.21.15.37:0/3118241483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594955a94a0 con 0x5594955aea00 2024-06-10T00:58:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.217+0000 7f77c53af640 1 --2- 172.21.15.37:0/3118241483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594955ae010 0x5594955a8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:58:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.218+0000 7f77bffff640 1 --2- 172.21.15.37:0/3118241483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594955aea00 0x5594955a4560 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x5594955c6a00 tx=0x7f77ac00b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41a3ba3dc1c9d47 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.220+0000 7f77aa7fc640 1 -- 172.21.15.37:0/3118241483 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac01f070 con 0x5594955aea00 2024-06-10T00:58:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.220+0000 7f77aa7fc640 1 -- 172.21.15.37:0/3118241483 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77ac004730 con 0x5594955aea00 2024-06-10T00:58:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.220+0000 7f77aa7fc640 1 -- 172.21.15.37:0/3118241483 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac01b5b0 con 0x5594955aea00 2024-06-10T00:58:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.221+0000 7f77c9bfc740 1 -- 172.21.15.37:0/3118241483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594955aea00 msgr2=0x5594955a4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.221+0000 7f77c9bfc740 1 --2- 172.21.15.37:0/3118241483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594955aea00 0x5594955a4560 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x5594955c6a00 tx=0x7f77ac00b4d0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.221+0000 7f77c9bfc740 1 -- 172.21.15.37:0/3118241483 shutdown_connections 2024-06-10T00:58:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.221+0000 7f77c9bfc740 1 --2- 172.21.15.37:0/3118241483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594955b3b70 0x5594955b8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.221+0000 7f77c9bfc740 1 --2- 172.21.15.37:0/3118241483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594955aea00 0x5594955a4560 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.221+0000 7f77c9bfc740 1 --2- 172.21.15.37:0/3118241483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594955ae010 0x5594955a8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.221+0000 7f77c9bfc740 1 -- 172.21.15.37:0/3118241483 >> 172.21.15.37:0/3118241483 conn(0x5594955a9680 msgr2=0x5594955a9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.221+0000 7f77c9bfc740 1 -- 172.21.15.37:0/3118241483 shutdown_connections 2024-06-10T00:58:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.221+0000 7f77c9bfc740 1 -- 172.21.15.37:0/3118241483 wait complete. 2024-06-10T00:58:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.221+0000 7f77c9bfc740 1 -- start start 2024-06-10T00:58:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.221+0000 7f77c9bfc740 1 Processor -- start 2024-06-10T00:58:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.222+0000 7f77c9bfc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594955ae010 0x5594955bef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.222+0000 7f77c9bfc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594955b3b70 0x5594955bb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.222+0000 7f77c9bfc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594955bbf30 0x55949562ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.222+0000 7f77c9bfc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f77c01a6c20 con 0x5594955bbf30 2024-06-10T00:58:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.222+0000 7f77c9bfc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f77c01a8090 con 0x5594955ae010 2024-06-10T00:58:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.222+0000 7f77c9bfc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f77c01a72f0 con 0x5594955b3b70 2024-06-10T00:58:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.222+0000 7f77c9bfc740 10 client.0 _ll_register_callbacks cb 0x7f77b8009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:58:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.222+0000 7f77c53af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594955bbf30 0x55949562ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.222+0000 7f77c53af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594955bbf30 0x55949562ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58110/0 (socket says 172.21.15.37:58110) 2024-06-10T00:58:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.222+0000 7f77c53af640 1 -- 172.21.15.37:0/809320800 learned_addr learned my addr 172.21.15.37:0/809320800 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.222+0000 7f77bffff640 1 --2- 172.21.15.37:0/809320800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594955b3b70 0x5594955bb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.222+0000 7f77c53af640 1 -- 172.21.15.37:0/809320800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594955b3b70 msgr2=0x5594955bb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.223+0000 7f77c53af640 1 --2- 172.21.15.37:0/809320800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594955b3b70 0x5594955bb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.223+0000 7f77c53af640 1 -- 172.21.15.37:0/809320800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594955ae010 msgr2=0x5594955bef00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T00:58:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.223+0000 7f77c53af640 1 --2- 172.21.15.37:0/809320800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594955ae010 0x5594955bef00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.223+0000 7f77c53af640 1 -- 172.21.15.37:0/809320800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594955bc5b0 con 0x5594955bbf30 2024-06-10T00:58:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.223+0000 7f77c53af640 1 --2- 172.21.15.37:0/809320800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594955bbf30 0x55949562ca80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f77b8010d60 tx=0x7f77b8010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e44db43fc49757 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.223+0000 7f779ffff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:58:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.223+0000 7f779ffff640 1 -- 172.21.15.37:0/809320800 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b803c020 con 0x5594955bbf30 2024-06-10T00:58:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.223+0000 7f779ffff640 1 -- 172.21.15.37:0/809320800 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f77b8002030 con 0x5594955bbf30 2024-06-10T00:58:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.223+0000 7f77c9bfc740 1 -- 172.21.15.37:0/809320800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77ac004730 con 0x5594955bbf30 2024-06-10T00:58:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.223+0000 7f779ffff640 1 -- 172.21.15.37:0/809320800 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f77b8034450 con 0x5594955bbf30 2024-06-10T00:58:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.223+0000 7f77c9bfc740 10 client.15711 fetch_fsmap learned FSMap version 27 2024-06-10T00:58:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.223+0000 7f77c9bfc740 1 -- 172.21.15.37:0/809320800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55949562e0a0 con 0x5594955bbf30 2024-06-10T00:58:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.224+0000 7f779ffff640 1 -- 172.21.15.37:0/809320800 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77ac004730 con 0x5594955bbf30 2024-06-10T00:58:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.224+0000 7f779ffff640 1 -- 172.21.15.37:0/809320800 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f77b80384f0 con 0x5594955bbf30 2024-06-10T00:58:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.224+0000 7f77c9bfc740 10 client.15711 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:58:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.224+0000 7f77c9bfc740 10 client.15711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T00:58:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.224+0000 7f77c9bfc740 20 client.15711 populate_metadata read hostname 'smithi037' 2024-06-10T00:58:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.224+0000 7f77c9bfc740 1 --2- 172.21.15.37:0/809320800 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55949562e8f0 0x559495630d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.224+0000 7f77c9bfc740 4 client.15711 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T00:58:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.224+0000 7f77c9bfc740 1 -- 172.21.15.37:0/809320800 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55949551ec20 con 0x55949562e8f0 2024-06-10T00:58:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.224+0000 7f77bffff640 1 --2- 172.21.15.37:0/809320800 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55949562e8f0 0x559495630d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.225+0000 7f77bffff640 1 --2- 172.21.15.37:0/809320800 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55949562e8f0 0x559495630d10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6726c49da7d32014 server_cookie=57414eb9a42b4fde in_seq=0 out_seq=0 2024-06-10T00:58:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.225+0000 7f779ffff640 10 client.15711 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.230+0000 7f779ffff640 1 -- 172.21.15.37:0/809320800 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55949551ec20 con 0x55949562e8f0 2024-06-10T00:58:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.230+0000 7f779ffff640 10 client.15711 handle_command_reply: tid=0 2024-06-10T00:58:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.230+0000 7f77c9bfc740 1 client.15711 shutdown 2024-06-10T00:58:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.230+0000 7f77c9bfc740 1 -- 172.21.15.37:0/809320800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594955bbf30 msgr2=0x55949562ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.230+0000 7f77c9bfc740 1 --2- 172.21.15.37:0/809320800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594955bbf30 0x55949562ca80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f77b8010d60 tx=0x7f77b8010860 comp rx=0 tx=0).stop 2024-06-10T00:58:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.231+0000 7f77c9bfc740 1 -- 172.21.15.37:0/809320800 shutdown_connections 2024-06-10T00:58:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.231+0000 7f77c9bfc740 1 --2- 172.21.15.37:0/809320800 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55949562e8f0 0x559495630d10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.231+0000 7f779ffff640 0 client.15711 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.231+0000 7f77c9bfc740 1 --2- 172.21.15.37:0/809320800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594955bbf30 0x55949562ca80 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.231+0000 7f77c9bfc740 1 --2- 172.21.15.37:0/809320800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594955b3b70 0x5594955bb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.231+0000 7f77c9bfc740 1 --2- 172.21.15.37:0/809320800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594955ae010 0x5594955bef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.231+0000 7f77c9bfc740 1 -- 172.21.15.37:0/809320800 >> 172.21.15.37:0/809320800 conn(0x5594955a9680 msgr2=0x5594955a4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.231+0000 7f77c9bfc740 1 -- 172.21.15.37:0/809320800 shutdown_connections 2024-06-10T00:58:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.231+0000 7f77c9bfc740 1 -- 172.21.15.37:0/809320800 wait complete. 2024-06-10T00:58:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.231+0000 7f77c9bfc740 20 client.15711 trim_cache size 0 max 16384 2024-06-10T00:58:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T00:58:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T00:58:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T00:58:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.231+0000 7f77c6e12640 1 -- 172.21.15.37:0/2775447912 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f77b00a2cb0 msgr2=0x7f77b00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.231+0000 7f77c6e12640 1 --2- 172.21.15.37:0/2775447912 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f77b00a2cb0 0x7f77b00a5170 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f77b40c8a60 tx=0x7f77b4009260 comp rx=0 tx=0).stop 2024-06-10T00:58:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.232+0000 7f77c6e12640 1 -- 172.21.15.37:0/2775447912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77c00ffce0 msgr2=0x7f77c0195aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.232+0000 7f77c6e12640 1 --2- 172.21.15.37:0/2775447912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77c00ffce0 0x7f77c0195aa0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f77b4006010 tx=0x7f77b4004ee0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.232+0000 7f77c6e12640 1 -- 172.21.15.37:0/2775447912 shutdown_connections 2024-06-10T00:58:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.232+0000 7f77c6e12640 1 --2- 172.21.15.37:0/2775447912 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f77b00a2cb0 0x7f77b00a5170 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.232+0000 7f77c6e12640 1 --2- 172.21.15.37:0/2775447912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77c0103670 0x7f77c019d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.232+0000 7f77c6e12640 1 --2- 172.21.15.37:0/2775447912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77c0102cb0 0x7f77c0195fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.232+0000 7f77c6e12640 1 --2- 172.21.15.37:0/2775447912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77c00ffce0 0x7f77c0195aa0 secure :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f77b4006010 tx=0x7f77b4004ee0 comp rx=0 tx=0).stop 2024-06-10T00:58:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.232+0000 7f77c6e12640 1 -- 172.21.15.37:0/2775447912 >> 172.21.15.37:0/2775447912 conn(0x7f77c00fb610 msgr2=0x7f77c0105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.232+0000 7f77c6e12640 1 -- 172.21.15.37:0/2775447912 shutdown_connections 2024-06-10T00:58:02.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:02.232+0000 7f77c6e12640 1 -- 172.21.15.37:0/2775447912 wait complete. 2024-06-10T00:58:02.536 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:02 smithi037 ceph-mon[32192]: pgmap v470: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-10T00:58:02.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:02 smithi167 ceph-mon[36917]: pgmap v470: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-10T00:58:02.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:02 smithi094 ceph-mon[36708]: pgmap v470: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-10T00:58:03.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.168+0000 7f53ccb10700 1 -- 172.21.15.37:0/1272639645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53c8108500 msgr2=0x7f53c81088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:03.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.168+0000 7f53ccb10700 1 --2- 172.21.15.37:0/1272639645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53c8108500 0x7f53c81088e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f53b8009990 tx=0x7f53b8009ca0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.168+0000 7f53ccb10700 1 -- 172.21.15.37:0/1272639645 shutdown_connections 2024-06-10T00:58:03.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.168+0000 7f53ccb10700 1 --2- 172.21.15.37:0/1272639645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53c806ab30 0x7f53c810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.168+0000 7f53ccb10700 1 --2- 172.21.15.37:0/1272639645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53c8108500 0x7f53c81088e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.168+0000 7f53ccb10700 1 --2- 172.21.15.37:0/1272639645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53c806a170 0x7f53c806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.168+0000 7f53ccb10700 1 -- 172.21.15.37:0/1272639645 >> 172.21.15.37:0/1272639645 conn(0x7f53c8078340 msgr2=0x7f53c8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:03.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.168+0000 7f53ccb10700 1 -- 172.21.15.37:0/1272639645 shutdown_connections 2024-06-10T00:58:03.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.168+0000 7f53ccb10700 1 -- 172.21.15.37:0/1272639645 wait complete. 2024-06-10T00:58:03.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.169+0000 7f53ccb10700 1 Processor -- start 2024-06-10T00:58:03.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.169+0000 7f53ccb10700 1 -- start start 2024-06-10T00:58:03.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.170+0000 7f53ccb10700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53c806a170 0x7f53c8101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:03.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.170+0000 7f53ccb10700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53c806ab30 0x7f53c81019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:03.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.170+0000 7f53ccb10700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53c8108500 0x7f53c8101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:03.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.170+0000 7f53ccb10700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f53c8112760 con 0x7f53c8108500 2024-06-10T00:58:03.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.170+0000 7f53ccb10700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f53c81125e0 con 0x7f53c806a170 2024-06-10T00:58:03.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.170+0000 7f53ccb10700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f53c81128e0 con 0x7f53c806ab30 2024-06-10T00:58:03.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.170+0000 7f53c6d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53c8108500 0x7f53c8101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:03.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.170+0000 7f53c5d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53c806ab30 0x7f53c81019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:03.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.170+0000 7f53c6d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53c8108500 0x7f53c8101ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58124/0 (socket says 172.21.15.37:58124) 2024-06-10T00:58:03.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.170+0000 7f53c5d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53c806ab30 0x7f53c81019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:44642/0 (socket says 172.21.15.37:44642) 2024-06-10T00:58:03.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.170+0000 7f53c6d9d700 1 -- 172.21.15.37:0/376944628 learned_addr learned my addr 172.21.15.37:0/376944628 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:03.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.170+0000 7f53c659c700 1 --2- 172.21.15.37:0/376944628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53c806a170 0x7f53c8101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:03.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.171+0000 7f53c5d9b700 1 -- 172.21.15.37:0/376944628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53c806a170 msgr2=0x7f53c8101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:03.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.171+0000 7f53c5d9b700 1 --2- 172.21.15.37:0/376944628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53c806a170 0x7f53c8101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.171+0000 7f53c5d9b700 1 -- 172.21.15.37:0/376944628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53c8108500 msgr2=0x7f53c8101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:03.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.171+0000 7f53c5d9b700 1 --2- 172.21.15.37:0/376944628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53c8108500 0x7f53c8101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.171+0000 7f53c5d9b700 1 -- 172.21.15.37:0/376944628 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c81a7180 con 0x7f53c806ab30 2024-06-10T00:58:03.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.171+0000 7f53c6d9d700 1 --2- 172.21.15.37:0/376944628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53c8108500 0x7f53c8101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:58:03.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.171+0000 7f53c5d9b700 1 --2- 172.21.15.37:0/376944628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53c806ab30 0x7f53c81019a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00d750 tx=0x7f53bc00da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:03.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.171+0000 7f53b77fe700 1 -- 172.21.15.37:0/376944628 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc014030 con 0x7f53c806ab30 2024-06-10T00:58:03.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.171+0000 7f53ccb10700 1 -- 172.21.15.37:0/376944628 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c81989f0 con 0x7f53c806ab30 2024-06-10T00:58:03.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.171+0000 7f53ccb10700 1 -- 172.21.15.37:0/376944628 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c8198f80 con 0x7f53c806ab30 2024-06-10T00:58:03.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.171+0000 7f53b77fe700 1 -- 172.21.15.37:0/376944628 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53bc004500 con 0x7f53c806ab30 2024-06-10T00:58:03.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.171+0000 7f53b77fe700 1 -- 172.21.15.37:0/376944628 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc005400 con 0x7f53c806ab30 2024-06-10T00:58:03.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.174+0000 7f53b77fe700 1 -- 172.21.15.37:0/376944628 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f53bc004750 con 0x7f53c806ab30 2024-06-10T00:58:03.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.174+0000 7f53b77fe700 1 --2- 172.21.15.37:0/376944628 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f53b00a2ca0 0x7f53b00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:03.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.175+0000 7f53b77fe700 1 -- 172.21.15.37:0/376944628 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f53bc0cec80 con 0x7f53c806ab30 2024-06-10T00:58:03.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.175+0000 7f53c659c700 1 --2- 172.21.15.37:0/376944628 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f53b00a2ca0 0x7f53b00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:03.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.175+0000 7f53c659c700 1 --2- 172.21.15.37:0/376944628 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f53b00a2ca0 0x7f53b00a5160 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f53b8004000 tx=0x7f53b8005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:03.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.177+0000 7f53d7393b80 1 Processor -- start 2024-06-10T00:58:03.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.177+0000 7f53d7393b80 1 -- start start 2024-06-10T00:58:03.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.178+0000 7f53d7393b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0f3617640 0x55b0f3617a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:03.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.178+0000 7f53d7393b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f3617f60 0x55b0f36119e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:03.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.178+0000 7f53d7393b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0f3611f20 0x55b0f3612340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:03.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.178+0000 7f53c659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0f3617640 0x55b0f3617a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:03.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.178+0000 7f53c659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0f3617640 0x55b0f3617a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58130/0 (socket says 172.21.15.37:58130) 2024-06-10T00:58:03.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.178+0000 7f53d7393b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b0f3563810 con 0x55b0f3617640 2024-06-10T00:58:03.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.178+0000 7f53c659c700 1 -- 172.21.15.37:0/2894765127 learned_addr learned my addr 172.21.15.37:0/2894765127 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:03.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.178+0000 7f53c5d9b700 1 --2- 172.21.15.37:0/2894765127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f3617f60 0x55b0f36119e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:03.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.178+0000 7f53d7393b80 1 -- 172.21.15.37:0/2894765127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b0f350b720 con 0x55b0f3611f20 2024-06-10T00:58:03.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.178+0000 7f53c6d9d700 1 --2- 172.21.15.37:0/2894765127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0f3611f20 0x55b0f3612340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:03.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.179+0000 7f53d7393b80 1 -- 172.21.15.37:0/2894765127 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b0f355f7d0 con 0x55b0f3617f60 2024-06-10T00:58:03.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.179+0000 7f53c6d9d700 1 -- 172.21.15.37:0/2894765127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f3617f60 msgr2=0x55b0f36119e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:03.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.179+0000 7f53c6d9d700 1 --2- 172.21.15.37:0/2894765127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f3617f60 0x55b0f36119e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.179+0000 7f53c6d9d700 1 -- 172.21.15.37:0/2894765127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0f3617640 msgr2=0x55b0f3617a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:03.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.179+0000 7f53c6d9d700 1 --2- 172.21.15.37:0/2894765127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0f3617640 0x55b0f3617a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.179+0000 7f53c6d9d700 1 -- 172.21.15.37:0/2894765127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0f3501f50 con 0x55b0f3611f20 2024-06-10T00:58:03.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.179+0000 7f53c659c700 1 --2- 172.21.15.37:0/2894765127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0f3617640 0x55b0f3617a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:58:03.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.179+0000 7f53c6d9d700 1 --2- 172.21.15.37:0/2894765127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0f3611f20 0x55b0f3612340 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009430 tx=0x7f53c0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5382f18225bb94 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:03.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.180+0000 7f53b4ff9700 1 -- 172.21.15.37:0/2894765127 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c0004bb0 con 0x55b0f3611f20 2024-06-10T00:58:03.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.180+0000 7f53b4ff9700 1 -- 172.21.15.37:0/2894765127 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53c0004d50 con 0x55b0f3611f20 2024-06-10T00:58:03.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.180+0000 7f53b4ff9700 1 -- 172.21.15.37:0/2894765127 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c001f6f0 con 0x55b0f3611f20 2024-06-10T00:58:03.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.180+0000 7f53d7393b80 1 -- 172.21.15.37:0/2894765127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0f3611f20 msgr2=0x55b0f3612340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:03.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.180+0000 7f53d7393b80 1 --2- 172.21.15.37:0/2894765127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0f3611f20 0x55b0f3612340 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009430 tx=0x7f53c0009920 comp rx=0 tx=0).stop 2024-06-10T00:58:03.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.181+0000 7f53d7393b80 1 -- 172.21.15.37:0/2894765127 shutdown_connections 2024-06-10T00:58:03.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.181+0000 7f53d7393b80 1 --2- 172.21.15.37:0/2894765127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0f3611f20 0x55b0f3612340 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.181+0000 7f53d7393b80 1 --2- 172.21.15.37:0/2894765127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0f3617640 0x55b0f3617a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.181+0000 7f53d7393b80 1 --2- 172.21.15.37:0/2894765127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f3617f60 0x55b0f36119e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.181+0000 7f53d7393b80 1 -- 172.21.15.37:0/2894765127 >> 172.21.15.37:0/2894765127 conn(0x55b0f353ccc0 msgr2=0x55b0f360c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:03.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.181+0000 7f53d7393b80 1 -- 172.21.15.37:0/2894765127 shutdown_connections 2024-06-10T00:58:03.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.181+0000 7f53d7393b80 1 -- 172.21.15.37:0/2894765127 wait complete. 2024-06-10T00:58:03.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.181+0000 7f53d7393b80 1 -- start start 2024-06-10T00:58:03.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.181+0000 7f53d7393b80 1 Processor -- start 2024-06-10T00:58:03.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.181+0000 7f53d7393b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0f3611f20 0x55b0f361dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:03.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.181+0000 7f53d7393b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0f3617640 0x55b0f361b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:03.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.182+0000 7f53d7393b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f3617f60 0x55b0f361b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:03.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.182+0000 7f53d7393b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b0f361e970 con 0x55b0f3617640 2024-06-10T00:58:03.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.182+0000 7f53d7393b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b0f3503bf0 con 0x55b0f3611f20 2024-06-10T00:58:03.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.182+0000 7f53d7393b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b0f361eaf0 con 0x55b0f3617f60 2024-06-10T00:58:03.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.182+0000 7f53d7393b80 10 client.0 _ll_register_callbacks cb 0x55b0f34fd920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:58:03.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.182+0000 7f53c5d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0f3617640 0x55b0f361b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:03.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.182+0000 7f53c5d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0f3617640 0x55b0f361b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58142/0 (socket says 172.21.15.37:58142) 2024-06-10T00:58:03.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.182+0000 7f53c5d9b700 1 -- 172.21.15.37:0/2526609060 learned_addr learned my addr 172.21.15.37:0/2526609060 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:03.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.182+0000 7f53c5d9b700 1 -- 172.21.15.37:0/2526609060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f3617f60 msgr2=0x55b0f361b550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T00:58:03.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.182+0000 7f53c5d9b700 1 --2- 172.21.15.37:0/2526609060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f3617f60 0x55b0f361b550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.182+0000 7f53c5d9b700 1 -- 172.21.15.37:0/2526609060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0f3611f20 msgr2=0x55b0f361dd10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T00:58:03.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.182+0000 7f53c5d9b700 1 --2- 172.21.15.37:0/2526609060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0f3611f20 0x55b0f361dd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.182+0000 7f53c5d9b700 1 -- 172.21.15.37:0/2526609060 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0f34b28d0 con 0x55b0f3617640 2024-06-10T00:58:03.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.182+0000 7f53c5d9b700 1 --2- 172.21.15.37:0/2526609060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0f3617640 0x55b0f361b010 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f53bc09adf0 tx=0x7f53bc09b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44167525ac7159c7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:03.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.182+0000 7f53adffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:58:03.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.182+0000 7f53adffb700 1 -- 172.21.15.37:0/2526609060 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc01d040 con 0x55b0f3617640 2024-06-10T00:58:03.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.183+0000 7f53d7393b80 1 -- 172.21.15.37:0/2526609060 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53c0004d50 con 0x55b0f3617640 2024-06-10T00:58:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.183+0000 7f53adffb700 1 -- 172.21.15.37:0/2526609060 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f53bc09f160 con 0x55b0f3617640 2024-06-10T00:58:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.183+0000 7f53adffb700 1 -- 172.21.15.37:0/2526609060 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f53bc0055a0 con 0x55b0f3617640 2024-06-10T00:58:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.183+0000 7f53d7393b80 10 client.15723 fetch_fsmap learned FSMap version 27 2024-06-10T00:58:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.183+0000 7f53d7393b80 1 -- 172.21.15.37:0/2526609060 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b0f36292a0 con 0x55b0f3617640 2024-06-10T00:58:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.183+0000 7f53adffb700 1 -- 172.21.15.37:0/2526609060 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53c0004d50 con 0x55b0f3617640 2024-06-10T00:58:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.184+0000 7f53adffb700 1 -- 172.21.15.37:0/2526609060 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f53bc0b5050 con 0x55b0f3617640 2024-06-10T00:58:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.184+0000 7f53d7393b80 10 client.15723 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:58:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.184+0000 7f53d7393b80 10 client.15723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T00:58:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.184+0000 7f53d7393b80 20 client.15723 populate_metadata read hostname 'smithi037' 2024-06-10T00:58:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.184+0000 7f53d7393b80 1 --2- 172.21.15.37:0/2526609060 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b0f362b900 0x55b0f362dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.184+0000 7f53d7393b80 4 client.15723 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:58:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.184+0000 7f53d7393b80 1 -- 172.21.15.37:0/2526609060 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0f3570d10 con 0x55b0f362b900 2024-06-10T00:58:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.184+0000 7f53c659c700 1 --2- 172.21.15.37:0/2526609060 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b0f362b900 0x55b0f362dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.185+0000 7f53c659c700 1 --2- 172.21.15.37:0/2526609060 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b0f362b900 0x55b0f362dcf0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f78c85cdf71ae79b server_cookie=61943d98906083eb in_seq=0 out_seq=0 2024-06-10T00:58:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.185+0000 7f53adffb700 10 client.15723 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:03.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.185+0000 7f53adffb700 1 -- 172.21.15.37:0/2526609060 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b0f3570d10 con 0x55b0f362b900 2024-06-10T00:58:03.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.185+0000 7f53adffb700 10 client.15723 handle_command_reply: tid=0 2024-06-10T00:58:03.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.185+0000 7f53d7393b80 1 client.15723 shutdown 2024-06-10T00:58:03.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.186+0000 7f53d7393b80 1 -- 172.21.15.37:0/2526609060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0f3617640 msgr2=0x55b0f361b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:03.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.186+0000 7f53d7393b80 1 --2- 172.21.15.37:0/2526609060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0f3617640 0x55b0f361b010 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f53bc09adf0 tx=0x7f53bc09b950 comp rx=0 tx=0).stop 2024-06-10T00:58:03.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.186+0000 7f53d7393b80 1 -- 172.21.15.37:0/2526609060 shutdown_connections 2024-06-10T00:58:03.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.186+0000 7f53d7393b80 1 --2- 172.21.15.37:0/2526609060 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b0f362b900 0x55b0f362dcf0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.186+0000 7f53adffb700 0 client.15723 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:03.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.186+0000 7f53d7393b80 1 --2- 172.21.15.37:0/2526609060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f3617f60 0x55b0f361b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.186+0000 7f53d7393b80 1 --2- 172.21.15.37:0/2526609060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0f3611f20 0x55b0f361dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.186+0000 7f53d7393b80 1 --2- 172.21.15.37:0/2526609060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0f3617640 0x55b0f361b010 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.186+0000 7f53d7393b80 1 -- 172.21.15.37:0/2526609060 >> 172.21.15.37:0/2526609060 conn(0x55b0f353ccc0 msgr2=0x55b0f36109d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:03.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.186+0000 7f53d7393b80 1 -- 172.21.15.37:0/2526609060 shutdown_connections 2024-06-10T00:58:03.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.186+0000 7f53d7393b80 1 -- 172.21.15.37:0/2526609060 wait complete. 2024-06-10T00:58:03.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.186+0000 7f53d7393b80 20 client.15723 trim_cache size 0 max 16384 2024-06-10T00:58:03.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.201+0000 7f53d7393b80 1 Processor -- start 2024-06-10T00:58:03.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.202+0000 7f53d7393b80 1 -- start start 2024-06-10T00:58:03.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.202+0000 7f53d7393b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f3611f20 0x55b0f36119e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:03.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.202+0000 7f53d7393b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0f3617640 0x55b0f36155c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:03.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.202+0000 7f53d7393b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0f3617f60 0x55b0f3633550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:03.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.202+0000 7f53d7393b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f53c81aeb40 con 0x55b0f3617640 2024-06-10T00:58:03.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.202+0000 7f53d7393b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f53c81ac3d0 con 0x55b0f3617f60 2024-06-10T00:58:03.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.202+0000 7f53d7393b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f53c81aa220 con 0x55b0f3611f20 2024-06-10T00:58:03.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.202+0000 7f53c5d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0f3617640 0x55b0f36155c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:03.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.202+0000 7f53c5d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0f3617640 0x55b0f36155c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58146/0 (socket says 172.21.15.37:58146) 2024-06-10T00:58:03.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.202+0000 7f53c5d9b700 1 -- 172.21.15.37:0/1265051121 learned_addr learned my addr 172.21.15.37:0/1265051121 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:03.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.202+0000 7f53c659c700 1 --2- 172.21.15.37:0/1265051121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f3611f20 0x55b0f36119e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:03.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.202+0000 7f53c5d9b700 1 -- 172.21.15.37:0/1265051121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f3611f20 msgr2=0x55b0f36119e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:03.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.202+0000 7f53c5d9b700 1 --2- 172.21.15.37:0/1265051121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f3611f20 0x55b0f36119e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.202+0000 7f53c5d9b700 1 -- 172.21.15.37:0/1265051121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0f3617f60 msgr2=0x55b0f3633550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:03.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.202+0000 7f53c6d9d700 1 --2- 172.21.15.37:0/1265051121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0f3617f60 0x55b0f3633550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:03.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.203+0000 7f53c5d9b700 1 --2- 172.21.15.37:0/1265051121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0f3617f60 0x55b0f3633550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.203+0000 7f53c5d9b700 1 -- 172.21.15.37:0/1265051121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0f360d500 con 0x55b0f3617640 2024-06-10T00:58:03.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.203+0000 7f53c5d9b700 1 --2- 172.21.15.37:0/1265051121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0f3617640 0x55b0f36155c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f53bc09b080 tx=0x7f53bc0a3c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f124b8e37f3da5c0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:03.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.203+0000 7f53c6d9d700 1 --2- 172.21.15.37:0/1265051121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0f3617f60 0x55b0f3633550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:58:03.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.203+0000 7f53affff700 1 -- 172.21.15.37:0/1265051121 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc07e070 con 0x55b0f3617640 2024-06-10T00:58:03.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.203+0000 7f53affff700 1 -- 172.21.15.37:0/1265051121 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53bc0bb0d0 con 0x55b0f3617640 2024-06-10T00:58:03.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.203+0000 7f53affff700 1 -- 172.21.15.37:0/1265051121 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc01d040 con 0x55b0f3617640 2024-06-10T00:58:03.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.204+0000 7f53d7393b80 1 -- 172.21.15.37:0/1265051121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0f3617640 msgr2=0x55b0f36155c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:03.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.204+0000 7f53d7393b80 1 --2- 172.21.15.37:0/1265051121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0f3617640 0x55b0f36155c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f53bc09b080 tx=0x7f53bc0a3c90 comp rx=0 tx=0).stop 2024-06-10T00:58:03.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.204+0000 7f53d7393b80 1 -- 172.21.15.37:0/1265051121 shutdown_connections 2024-06-10T00:58:03.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.204+0000 7f53d7393b80 1 --2- 172.21.15.37:0/1265051121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0f3617f60 0x55b0f3633550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.204+0000 7f53d7393b80 1 --2- 172.21.15.37:0/1265051121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f3611f20 0x55b0f36119e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.204+0000 7f53d7393b80 1 --2- 172.21.15.37:0/1265051121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0f3617640 0x55b0f36155c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.204+0000 7f53d7393b80 1 -- 172.21.15.37:0/1265051121 >> 172.21.15.37:0/1265051121 conn(0x55b0f353ccc0 msgr2=0x55b0f3614eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:03.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.204+0000 7f53d7393b80 1 -- 172.21.15.37:0/1265051121 shutdown_connections 2024-06-10T00:58:03.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.204+0000 7f53d7393b80 1 -- 172.21.15.37:0/1265051121 wait complete. 2024-06-10T00:58:03.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.205+0000 7f53d7393b80 1 -- start start 2024-06-10T00:58:03.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.205+0000 7f53d7393b80 1 Processor -- start 2024-06-10T00:58:03.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.205+0000 7f53d7393b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f3611f20 0x55b0f3624390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:03.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.205+0000 7f53d7393b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0f3617640 0x55b0f3632e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:03.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.205+0000 7f53d7393b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0f3617f60 0x55b0f36333d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:03.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.205+0000 7f53d7393b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f53c81aa890 con 0x55b0f3617640 2024-06-10T00:58:03.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.205+0000 7f53d7393b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f53c81abd00 con 0x55b0f3617f60 2024-06-10T00:58:03.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.205+0000 7f53d7393b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f53c81aaf60 con 0x55b0f3611f20 2024-06-10T00:58:03.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.205+0000 7f53d7393b80 10 client.0 _ll_register_callbacks cb 0x55b0f34fd920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:58:03.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.205+0000 7f53c5d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0f3617640 0x55b0f3632e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:03.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.205+0000 7f53c5d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0f3617640 0x55b0f3632e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58160/0 (socket says 172.21.15.37:58160) 2024-06-10T00:58:03.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.205+0000 7f53c5d9b700 1 -- 172.21.15.37:0/2931706718 learned_addr learned my addr 172.21.15.37:0/2931706718 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:03.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.205+0000 7f53c659c700 1 --2- 172.21.15.37:0/2931706718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f3611f20 0x55b0f3624390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:03.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.206+0000 7f53c5d9b700 1 -- 172.21.15.37:0/2931706718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f3611f20 msgr2=0x55b0f3624390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:03.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.206+0000 7f53c5d9b700 1 --2- 172.21.15.37:0/2931706718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f3611f20 0x55b0f3624390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.206+0000 7f53c5d9b700 1 -- 172.21.15.37:0/2931706718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0f3617f60 msgr2=0x55b0f36333d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T00:58:03.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.206+0000 7f53c5d9b700 1 --2- 172.21.15.37:0/2931706718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0f3617f60 0x55b0f36333d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.206+0000 7f53c5d9b700 1 -- 172.21.15.37:0/2931706718 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0f3633a90 con 0x55b0f3617640 2024-06-10T00:58:03.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.206+0000 7f53c659c700 1 --2- 172.21.15.37:0/2931706718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f3611f20 0x55b0f3624390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:58:03.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.206+0000 7f53c5d9b700 1 --2- 172.21.15.37:0/2931706718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0f3617640 0x55b0f3632e90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f53bc0b4260 tx=0x7f53bc09ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98652627cff13f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:03.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.206+0000 7f53adffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:58:03.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.206+0000 7f53adffb700 1 -- 172.21.15.37:0/2931706718 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc07e030 con 0x55b0f3617640 2024-06-10T00:58:03.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.206+0000 7f53adffb700 1 -- 172.21.15.37:0/2931706718 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f53bc09a810 con 0x55b0f3617640 2024-06-10T00:58:03.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.206+0000 7f53d7393b80 1 -- 172.21.15.37:0/2931706718 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53bc0bb0d0 con 0x55b0f3617640 2024-06-10T00:58:03.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.206+0000 7f53adffb700 1 -- 172.21.15.37:0/2931706718 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f53bc01d070 con 0x55b0f3617640 2024-06-10T00:58:03.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.207+0000 7f53d7393b80 10 client.15729 fetch_fsmap learned FSMap version 27 2024-06-10T00:58:03.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.207+0000 7f53d7393b80 1 -- 172.21.15.37:0/2931706718 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b0f3696380 con 0x55b0f3617640 2024-06-10T00:58:03.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.207+0000 7f53adffb700 1 -- 172.21.15.37:0/2931706718 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53bc0bb0d0 con 0x55b0f3617640 2024-06-10T00:58:03.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.207+0000 7f53adffb700 1 -- 172.21.15.37:0/2931706718 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f53bc0b5070 con 0x55b0f3617640 2024-06-10T00:58:03.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.207+0000 7f53d7393b80 10 client.15729 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:58:03.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.207+0000 7f53d7393b80 10 client.15729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T00:58:03.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.207+0000 7f53d7393b80 20 client.15729 populate_metadata read hostname 'smithi037' 2024-06-10T00:58:03.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.207+0000 7f53d7393b80 1 --2- 172.21.15.37:0/2931706718 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b0f362b900 0x55b0f36987a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:03.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.207+0000 7f53d7393b80 4 client.15729 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:58:03.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.207+0000 7f53d7393b80 1 -- 172.21.15.37:0/2931706718 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b0f356b290 con 0x55b0f362b900 2024-06-10T00:58:03.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.207+0000 7f53c659c700 1 --2- 172.21.15.37:0/2931706718 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b0f362b900 0x55b0f36987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:03.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.208+0000 7f53adffb700 10 client.15729 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:03.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.208+0000 7f53c659c700 1 --2- 172.21.15.37:0/2931706718 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b0f362b900 0x55b0f36987a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7544a7d407bbbfd0 server_cookie=c39490460ec8df57 in_seq=0 out_seq=0 2024-06-10T00:58:03.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.208+0000 7f53adffb700 1 -- 172.21.15.37:0/2931706718 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b0f356b290 con 0x55b0f362b900 2024-06-10T00:58:03.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.208+0000 7f53adffb700 10 client.15729 handle_command_reply: tid=0 2024-06-10T00:58:03.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.208+0000 7f53d7393b80 1 client.15729 shutdown 2024-06-10T00:58:03.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.209+0000 7f53d7393b80 1 -- 172.21.15.37:0/2931706718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0f3617640 msgr2=0x55b0f3632e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:03.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.209+0000 7f53d7393b80 1 --2- 172.21.15.37:0/2931706718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0f3617640 0x55b0f3632e90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f53bc0b4260 tx=0x7f53bc09ac30 comp rx=0 tx=0).stop 2024-06-10T00:58:03.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.209+0000 7f53d7393b80 1 -- 172.21.15.37:0/2931706718 shutdown_connections 2024-06-10T00:58:03.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.209+0000 7f53d7393b80 1 --2- 172.21.15.37:0/2931706718 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b0f362b900 0x55b0f36987a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.209+0000 7f53d7393b80 1 --2- 172.21.15.37:0/2931706718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b0f3617f60 0x55b0f36333d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.209+0000 7f53d7393b80 1 --2- 172.21.15.37:0/2931706718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b0f3611f20 0x55b0f3624390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.209+0000 7f53d7393b80 1 --2- 172.21.15.37:0/2931706718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0f3617640 0x55b0f3632e90 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.209+0000 7f53d7393b80 1 -- 172.21.15.37:0/2931706718 >> 172.21.15.37:0/2931706718 conn(0x55b0f353ccc0 msgr2=0x55b0f3605e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:03.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.209+0000 7f53adffb700 0 client.15729 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:03.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.213+0000 7f53d7393b80 1 -- 172.21.15.37:0/2931706718 shutdown_connections 2024-06-10T00:58:03.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.213+0000 7f53d7393b80 1 -- 172.21.15.37:0/2931706718 wait complete. 2024-06-10T00:58:03.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.213+0000 7f53d7393b80 20 client.15729 trim_cache size 0 max 16384 2024-06-10T00:58:03.213 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T00:58:03.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.213+0000 7f53ccb10700 1 -- 172.21.15.37:0/376944628 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f53b00a2ca0 msgr2=0x7f53b00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:03.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.213+0000 7f53ccb10700 1 --2- 172.21.15.37:0/376944628 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f53b00a2ca0 0x7f53b00a5160 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f53b8004000 tx=0x7f53b8005960 comp rx=0 tx=0).stop 2024-06-10T00:58:03.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.213+0000 7f53ccb10700 1 -- 172.21.15.37:0/376944628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53c806ab30 msgr2=0x7f53c81019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:03.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.213+0000 7f53ccb10700 1 --2- 172.21.15.37:0/376944628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53c806ab30 0x7f53c81019a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00d750 tx=0x7f53bc00da60 comp rx=0 tx=0).stop 2024-06-10T00:58:03.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.214+0000 7f53ccb10700 1 -- 172.21.15.37:0/376944628 shutdown_connections 2024-06-10T00:58:03.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.214+0000 7f53ccb10700 1 --2- 172.21.15.37:0/376944628 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f53b00a2ca0 0x7f53b00a5160 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.214+0000 7f53ccb10700 1 --2- 172.21.15.37:0/376944628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53c8108500 0x7f53c8101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.214+0000 7f53ccb10700 1 --2- 172.21.15.37:0/376944628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53c806a170 0x7f53c8101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.214+0000 7f53ccb10700 1 --2- 172.21.15.37:0/376944628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53c806ab30 0x7f53c81019a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:03.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.214+0000 7f53ccb10700 1 -- 172.21.15.37:0/376944628 >> 172.21.15.37:0/376944628 conn(0x7f53c8078340 msgr2=0x7f53c81000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:03.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.214+0000 7f53ccb10700 1 -- 172.21.15.37:0/376944628 shutdown_connections 2024-06-10T00:58:03.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:03.214+0000 7f53ccb10700 1 -- 172.21.15.37:0/376944628 wait complete. 2024-06-10T00:58:04.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:04 smithi167 ceph-mon[36917]: pgmap v471: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 8 op/s 2024-06-10T00:58:04.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:04 smithi094 ceph-mon[36708]: pgmap v471: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 8 op/s 2024-06-10T00:58:04.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:04 smithi037 ceph-mon[32192]: pgmap v471: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 8 op/s 2024-06-10T00:58:04.910 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T00:58:04.911 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T00:58:05.172 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:58:05.820 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:58:06.237 INFO:tasks.workunit.client.0.smithi037.stderr:Updating files: 92% (11577/12576) Updating files: 93% (11696/12576) Updating files: 94% (11822/12576) Updating files: 95% (11948/12576) Updating files: 96% (12073/12576) Updating files: 97% (12199/12576) Updating files: 98% (12325/12576) Updating files: 99% (12451/12576) Updating files: 100% (12576/12576) Updating files: 100% (12576/12576), done. 2024-06-10T00:58:06.501 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:06 smithi037 ceph-mon[32192]: pgmap v472: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-10T00:58:06.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:06 smithi167 ceph-mon[36917]: pgmap v472: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-10T00:58:06.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:06 smithi094 ceph-mon[36708]: pgmap v472: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-10T00:58:06.757 INFO:tasks.workunit.client.0.smithi037.stderr:Note: switching to '7b7a5d375cf4e4afab96b9d4d229b699ae8304bb'. 2024-06-10T00:58:06.757 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-06-10T00:58:06.757 INFO:tasks.workunit.client.0.smithi037.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-06-10T00:58:06.757 INFO:tasks.workunit.client.0.smithi037.stderr:changes and commit them, and you can discard any commits you make in this 2024-06-10T00:58:06.757 INFO:tasks.workunit.client.0.smithi037.stderr:state without impacting any branches by switching back to a branch. 2024-06-10T00:58:06.758 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-06-10T00:58:06.758 INFO:tasks.workunit.client.0.smithi037.stderr:If you want to create a new branch to retain commits you create, you may 2024-06-10T00:58:06.758 INFO:tasks.workunit.client.0.smithi037.stderr:do so (now or later) by using -c with the switch command. Example: 2024-06-10T00:58:06.758 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-06-10T00:58:06.758 INFO:tasks.workunit.client.0.smithi037.stderr: git switch -c 2024-06-10T00:58:06.758 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-06-10T00:58:06.758 INFO:tasks.workunit.client.0.smithi037.stderr:Or undo this operation with: 2024-06-10T00:58:06.758 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-06-10T00:58:06.758 INFO:tasks.workunit.client.0.smithi037.stderr: git switch - 2024-06-10T00:58:06.758 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-06-10T00:58:06.758 INFO:tasks.workunit.client.0.smithi037.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-06-10T00:58:06.758 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-06-10T00:58:06.758 INFO:tasks.workunit.client.0.smithi037.stderr:HEAD is now at 7b7a5d375cf Merge pull request #57941 from zdover23/wip-doc-2024-06-09-backport-57939-to-squid 2024-06-10T00:58:06.762 DEBUG:teuthology.orchestra.run.smithi037:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-06-10T00:58:06.836 INFO:tasks.workunit.client.0.smithi037.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-06-10T00:58:06.838 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-10T00:58:06.838 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-06-10T00:58:07.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.171+0000 7f2af3c58700 1 -- 172.21.15.37:0/1734717389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2aec102510 msgr2=0x7f2aec102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:07.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.171+0000 7f2af3c58700 1 --2- 172.21.15.37:0/1734717389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2aec102510 0x7f2aec102990 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009990 tx=0x7f2adc009ca0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.171+0000 7f2af3c58700 1 -- 172.21.15.37:0/1734717389 shutdown_connections 2024-06-10T00:58:07.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.171+0000 7f2af3c58700 1 --2- 172.21.15.37:0/1734717389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2aec102ed0 0x7f2aec10f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.171+0000 7f2af3c58700 1 --2- 172.21.15.37:0/1734717389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2aec108520 0x7f2aec108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.171+0000 7f2af3c58700 1 --2- 172.21.15.37:0/1734717389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2aec102510 0x7f2aec102990 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.172+0000 7f2af3c58700 1 -- 172.21.15.37:0/1734717389 >> 172.21.15.37:0/1734717389 conn(0x7f2aec079640 msgr2=0x7f2aec079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:07.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.172+0000 7f2af3c58700 1 -- 172.21.15.37:0/1734717389 shutdown_connections 2024-06-10T00:58:07.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.172+0000 7f2af3c58700 1 -- 172.21.15.37:0/1734717389 wait complete. 2024-06-10T00:58:07.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.172+0000 7f2af3c58700 1 Processor -- start 2024-06-10T00:58:07.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.172+0000 7f2af3c58700 1 -- start start 2024-06-10T00:58:07.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.172+0000 7f2af3c58700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2aec102510 0x7f2aec19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:07.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.172+0000 7f2af3c58700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2aec102ed0 0x7f2aec19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:07.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.173+0000 7f2af3c58700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2aec108520 0x7f2aec1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:07.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.173+0000 7f2af3c58700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec1127b0 con 0x7f2aec108520 2024-06-10T00:58:07.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.173+0000 7f2af3c58700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec112630 con 0x7f2aec102510 2024-06-10T00:58:07.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.173+0000 7f2af3c58700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec112930 con 0x7f2aec102ed0 2024-06-10T00:58:07.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.173+0000 7f2af21f5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2aec108520 0x7f2aec1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:07.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.173+0000 7f2af21f5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2aec108520 0x7f2aec1989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58184/0 (socket says 172.21.15.37:58184) 2024-06-10T00:58:07.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.173+0000 7f2af21f5700 1 -- 172.21.15.37:0/2933153057 learned_addr learned my addr 172.21.15.37:0/2933153057 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:07.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.173+0000 7f2af19f4700 1 --2- 172.21.15.37:0/2933153057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2aec102510 0x7f2aec19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:07.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.173+0000 7f2af11f3700 1 --2- 172.21.15.37:0/2933153057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2aec102ed0 0x7f2aec19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:07.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.173+0000 7f2af21f5700 1 -- 172.21.15.37:0/2933153057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2aec102ed0 msgr2=0x7f2aec19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:07.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.173+0000 7f2af21f5700 1 --2- 172.21.15.37:0/2933153057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2aec102ed0 0x7f2aec19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.173+0000 7f2af21f5700 1 -- 172.21.15.37:0/2933153057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2aec102510 msgr2=0x7f2aec19e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:07.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.173+0000 7f2af21f5700 1 --2- 172.21.15.37:0/2933153057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2aec102510 0x7f2aec19e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.173+0000 7f2af21f5700 1 -- 172.21.15.37:0/2933153057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aec199130 con 0x7f2aec108520 2024-06-10T00:58:07.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.173+0000 7f2af11f3700 1 --2- 172.21.15.37:0/2933153057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2aec102ed0 0x7f2aec19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:58:07.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.173+0000 7f2af21f5700 1 --2- 172.21.15.37:0/2933153057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2aec108520 0x7f2aec1989d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400ab50 tx=0x7f2ae400ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:07.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.173+0000 7f2ae2ffd700 1 -- 172.21.15.37:0/2933153057 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4017020 con 0x7f2aec108520 2024-06-10T00:58:07.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.173+0000 7f2ae2ffd700 1 -- 172.21.15.37:0/2933153057 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ae4004bb0 con 0x7f2aec108520 2024-06-10T00:58:07.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.173+0000 7f2af3c58700 1 -- 172.21.15.37:0/2933153057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aec199420 con 0x7f2aec108520 2024-06-10T00:58:07.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.173+0000 7f2ae2ffd700 1 -- 172.21.15.37:0/2933153057 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4005310 con 0x7f2aec108520 2024-06-10T00:58:07.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.173+0000 7f2af3c58700 1 -- 172.21.15.37:0/2933153057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aec1a56e0 con 0x7f2aec108520 2024-06-10T00:58:07.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.176+0000 7f2ae2ffd700 1 -- 172.21.15.37:0/2933153057 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2ae400c620 con 0x7f2aec108520 2024-06-10T00:58:07.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.176+0000 7f2ae2ffd700 1 --2- 172.21.15.37:0/2933153057 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2ad80a2ca0 0x7f2ad80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:07.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.177+0000 7f2af19f4700 1 --2- 172.21.15.37:0/2933153057 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2ad80a2ca0 0x7f2ad80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:07.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.177+0000 7f2ae2ffd700 1 -- 172.21.15.37:0/2933153057 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f2ae40ce9d0 con 0x7f2aec108520 2024-06-10T00:58:07.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.177+0000 7f2af19f4700 1 --2- 172.21.15.37:0/2933153057 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2ad80a2ca0 0x7f2ad80a5160 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8005950 tx=0x7f2ae800a300 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:07.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.177+0000 7f2afe4dbb80 1 Processor -- start 2024-06-10T00:58:07.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.177+0000 7f2afe4dbb80 1 -- start start 2024-06-10T00:58:07.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.178+0000 7f2afe4dbb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c7a89fe0 0x5635c7a8a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:07.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.178+0000 7f2afe4dbb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5635c7a8a900 0x5635c7a842e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:07.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.178+0000 7f2afe4dbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635c7a84820 0x5635c7a84c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:07.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.178+0000 7f2afe4dbb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5635c7976fe0 con 0x5635c7a8a900 2024-06-10T00:58:07.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.178+0000 7f2afe4dbb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5635c79d58d0 con 0x5635c7a89fe0 2024-06-10T00:58:07.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.178+0000 7f2afe4dbb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5635c793d4e0 con 0x5635c7a84820 2024-06-10T00:58:07.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.178+0000 7f2af11f3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5635c7a8a900 0x5635c7a842e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:07.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.178+0000 7f2af11f3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5635c7a8a900 0x5635c7a842e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58200/0 (socket says 172.21.15.37:58200) 2024-06-10T00:58:07.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.178+0000 7f2af11f3700 1 -- 172.21.15.37:0/4009189200 learned_addr learned my addr 172.21.15.37:0/4009189200 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:07.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.178+0000 7f2af19f4700 1 --2- 172.21.15.37:0/4009189200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c7a89fe0 0x5635c7a8a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:07.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.178+0000 7f2af21f5700 1 --2- 172.21.15.37:0/4009189200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635c7a84820 0x5635c7a84c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:07.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.178+0000 7f2af11f3700 1 -- 172.21.15.37:0/4009189200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635c7a84820 msgr2=0x5635c7a84c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:07.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.178+0000 7f2af11f3700 1 --2- 172.21.15.37:0/4009189200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635c7a84820 0x5635c7a84c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.178+0000 7f2af11f3700 1 -- 172.21.15.37:0/4009189200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c7a89fe0 msgr2=0x5635c7a8a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:07.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.178+0000 7f2af11f3700 1 --2- 172.21.15.37:0/4009189200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c7a89fe0 0x5635c7a8a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.178+0000 7f2af11f3700 1 -- 172.21.15.37:0/4009189200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635c7924950 con 0x5635c7a8a900 2024-06-10T00:58:07.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.178+0000 7f2af11f3700 1 --2- 172.21.15.37:0/4009189200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5635c7a8a900 0x5635c7a842e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2adc006010 tx=0x7f2adc009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a91c9e648b2b56 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:07.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.178+0000 7f2ad7fff700 1 -- 172.21.15.37:0/4009189200 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc027070 con 0x5635c7a8a900 2024-06-10T00:58:07.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.179+0000 7f2ad7fff700 1 -- 172.21.15.37:0/4009189200 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2adc0055b0 con 0x5635c7a8a900 2024-06-10T00:58:07.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.179+0000 7f2ad7fff700 1 -- 172.21.15.37:0/4009189200 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc00bcd0 con 0x5635c7a8a900 2024-06-10T00:58:07.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.179+0000 7f2afe4dbb80 1 -- 172.21.15.37:0/4009189200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5635c7a8a900 msgr2=0x5635c7a842e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:07.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.179+0000 7f2afe4dbb80 1 --2- 172.21.15.37:0/4009189200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5635c7a8a900 0x5635c7a842e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2adc006010 tx=0x7f2adc009f60 comp rx=0 tx=0).stop 2024-06-10T00:58:07.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.179+0000 7f2afe4dbb80 1 -- 172.21.15.37:0/4009189200 shutdown_connections 2024-06-10T00:58:07.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.179+0000 7f2afe4dbb80 1 --2- 172.21.15.37:0/4009189200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635c7a84820 0x5635c7a84c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.179+0000 7f2afe4dbb80 1 --2- 172.21.15.37:0/4009189200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c7a89fe0 0x5635c7a8a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.179+0000 7f2afe4dbb80 1 --2- 172.21.15.37:0/4009189200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5635c7a8a900 0x5635c7a842e0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.179+0000 7f2afe4dbb80 1 -- 172.21.15.37:0/4009189200 >> 172.21.15.37:0/4009189200 conn(0x5635c79b1100 msgr2=0x5635c7a7f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:07.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.179+0000 7f2afe4dbb80 1 -- 172.21.15.37:0/4009189200 shutdown_connections 2024-06-10T00:58:07.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.179+0000 7f2afe4dbb80 1 -- 172.21.15.37:0/4009189200 wait complete. 2024-06-10T00:58:07.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.180+0000 7f2afe4dbb80 1 -- start start 2024-06-10T00:58:07.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.180+0000 7f2afe4dbb80 1 Processor -- start 2024-06-10T00:58:07.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.180+0000 7f2afe4dbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635c7a84820 0x5635c7a90670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:07.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.180+0000 7f2afe4dbb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c7a89fe0 0x5635c7a8d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:07.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.180+0000 7f2afe4dbb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5635c7a8a900 0x5635c7a8deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:07.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.180+0000 7f2afe4dbb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5635c7976040 con 0x5635c7a8a900 2024-06-10T00:58:07.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.180+0000 7f2afe4dbb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5635c79d1890 con 0x5635c7a89fe0 2024-06-10T00:58:07.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.180+0000 7f2afe4dbb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5635c7a91280 con 0x5635c7a84820 2024-06-10T00:58:07.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.180+0000 7f2afe4dbb80 10 client.0 _ll_register_callbacks cb 0x5635c796b630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:58:07.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.181+0000 7f2af21f5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5635c7a8a900 0x5635c7a8deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:07.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.181+0000 7f2af21f5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5635c7a8a900 0x5635c7a8deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58204/0 (socket says 172.21.15.37:58204) 2024-06-10T00:58:07.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.181+0000 7f2af21f5700 1 -- 172.21.15.37:0/251604251 learned_addr learned my addr 172.21.15.37:0/251604251 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:07.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.181+0000 7f2af11f3700 1 --2- 172.21.15.37:0/251604251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c7a89fe0 0x5635c7a8d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:07.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.181+0000 7f2af19f4700 1 --2- 172.21.15.37:0/251604251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635c7a84820 0x5635c7a90670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:07.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.181+0000 7f2af21f5700 1 -- 172.21.15.37:0/251604251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635c7a84820 msgr2=0x5635c7a90670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:07.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.181+0000 7f2af21f5700 1 --2- 172.21.15.37:0/251604251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635c7a84820 0x5635c7a90670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.181+0000 7f2af21f5700 1 -- 172.21.15.37:0/251604251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c7a89fe0 msgr2=0x5635c7a8d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:07.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.181+0000 7f2af21f5700 1 --2- 172.21.15.37:0/251604251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c7a89fe0 0x5635c7a8d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.181+0000 7f2af21f5700 1 -- 172.21.15.37:0/251604251 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635c799b7d0 con 0x5635c7a8a900 2024-06-10T00:58:07.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.181+0000 7f2af19f4700 1 --2- 172.21.15.37:0/251604251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635c7a84820 0x5635c7a90670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:58:07.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.181+0000 7f2af21f5700 1 --2- 172.21.15.37:0/251604251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5635c7a8a900 0x5635c7a8deb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2ae409ae60 tx=0x7f2ae409b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=811da7fb7f8a912f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:07.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.181+0000 7f2ad57fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:58:07.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.181+0000 7f2ad57fa700 1 -- 172.21.15.37:0/251604251 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae40a3ab0 con 0x5635c7a8a900 2024-06-10T00:58:07.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.181+0000 7f2ad57fa700 1 -- 172.21.15.37:0/251604251 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f2ae409b130 con 0x5635c7a8a900 2024-06-10T00:58:07.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.181+0000 7f2afe4dbb80 1 -- 172.21.15.37:0/251604251 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2adc0055b0 con 0x5635c7a8a900 2024-06-10T00:58:07.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.181+0000 7f2ad57fa700 1 -- 172.21.15.37:0/251604251 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2ae409b330 con 0x5635c7a8a900 2024-06-10T00:58:07.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.181+0000 7f2afe4dbb80 10 client.15741 fetch_fsmap learned FSMap version 27 2024-06-10T00:58:07.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.181+0000 7f2afe4dbb80 1 -- 172.21.15.37:0/251604251 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5635c7a9ba60 con 0x5635c7a8a900 2024-06-10T00:58:07.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.181+0000 7f2ad57fa700 1 -- 172.21.15.37:0/251604251 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2adc0055b0 con 0x5635c7a8a900 2024-06-10T00:58:07.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.182+0000 7f2ad57fa700 1 -- 172.21.15.37:0/251604251 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2ae4012070 con 0x5635c7a8a900 2024-06-10T00:58:07.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.182+0000 7f2afe4dbb80 10 client.15741 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:58:07.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.182+0000 7f2afe4dbb80 10 client.15741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T00:58:07.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.182+0000 7f2afe4dbb80 20 client.15741 populate_metadata read hostname 'smithi037' 2024-06-10T00:58:07.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.182+0000 7f2afe4dbb80 1 --2- 172.21.15.37:0/251604251 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5635c7a9dff0 0x5635c7aa03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:07.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.182+0000 7f2afe4dbb80 4 client.15741 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:58:07.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.182+0000 7f2afe4dbb80 1 -- 172.21.15.37:0/251604251 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635c79e2dd0 con 0x5635c7a9dff0 2024-06-10T00:58:07.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.182+0000 7f2af19f4700 1 --2- 172.21.15.37:0/251604251 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5635c7a9dff0 0x5635c7aa03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:07.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.183+0000 7f2af19f4700 1 --2- 172.21.15.37:0/251604251 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5635c7a9dff0 0x5635c7aa03e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aca3ca86b7fe1ca7 server_cookie=f345c64686a52f8d in_seq=0 out_seq=0 2024-06-10T00:58:07.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.183+0000 7f2ad57fa700 10 client.15741 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:07.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.183+0000 7f2ad57fa700 1 -- 172.21.15.37:0/251604251 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5635c79e2dd0 con 0x5635c7a9dff0 2024-06-10T00:58:07.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.183+0000 7f2ad57fa700 10 client.15741 handle_command_reply: tid=0 2024-06-10T00:58:07.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.183+0000 7f2afe4dbb80 1 client.15741 shutdown 2024-06-10T00:58:07.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.184+0000 7f2afe4dbb80 1 -- 172.21.15.37:0/251604251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5635c7a8a900 msgr2=0x5635c7a8deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:07.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.184+0000 7f2afe4dbb80 1 --2- 172.21.15.37:0/251604251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5635c7a8a900 0x5635c7a8deb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2ae409ae60 tx=0x7f2ae409b700 comp rx=0 tx=0).stop 2024-06-10T00:58:07.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.184+0000 7f2afe4dbb80 1 -- 172.21.15.37:0/251604251 shutdown_connections 2024-06-10T00:58:07.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.184+0000 7f2afe4dbb80 1 --2- 172.21.15.37:0/251604251 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5635c7a9dff0 0x5635c7aa03e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.184+0000 7f2afe4dbb80 1 --2- 172.21.15.37:0/251604251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5635c7a8a900 0x5635c7a8deb0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.184+0000 7f2afe4dbb80 1 --2- 172.21.15.37:0/251604251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635c7a84820 0x5635c7a90670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.184+0000 7f2ad57fa700 0 client.15741 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:07.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.184+0000 7f2afe4dbb80 1 --2- 172.21.15.37:0/251604251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c7a89fe0 0x5635c7a8d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.184+0000 7f2afe4dbb80 1 -- 172.21.15.37:0/251604251 >> 172.21.15.37:0/251604251 conn(0x5635c79b1100 msgr2=0x5635c7a7e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:07.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.184+0000 7f2afe4dbb80 1 -- 172.21.15.37:0/251604251 shutdown_connections 2024-06-10T00:58:07.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.184+0000 7f2afe4dbb80 1 -- 172.21.15.37:0/251604251 wait complete. 2024-06-10T00:58:07.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.184+0000 7f2afe4dbb80 20 client.15741 trim_cache size 0 max 16384 2024-06-10T00:58:07.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.205+0000 7f2afe4dbb80 1 Processor -- start 2024-06-10T00:58:07.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.205+0000 7f2afe4dbb80 1 -- start start 2024-06-10T00:58:07.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.205+0000 7f2afe4dbb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c7a84820 0x5635c7a8caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:07.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.205+0000 7f2afe4dbb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5635c7a89fe0 0x5635c7a7ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:07.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.205+0000 7f2afe4dbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635c7a8a900 0x5635c7a7f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:07.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.205+0000 7f2afe4dbb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec1b0c00 con 0x5635c7a89fe0 2024-06-10T00:58:07.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.205+0000 7f2afe4dbb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec1ae490 con 0x5635c7a84820 2024-06-10T00:58:07.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.205+0000 7f2afe4dbb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec1ac2e0 con 0x5635c7a8a900 2024-06-10T00:58:07.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.205+0000 7f2af11f3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5635c7a89fe0 0x5635c7a7ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:07.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.205+0000 7f2af11f3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5635c7a89fe0 0x5635c7a7ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58220/0 (socket says 172.21.15.37:58220) 2024-06-10T00:58:07.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.205+0000 7f2af11f3700 1 -- 172.21.15.37:0/1211224804 learned_addr learned my addr 172.21.15.37:0/1211224804 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:07.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.205+0000 7f2af21f5700 1 --2- 172.21.15.37:0/1211224804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635c7a8a900 0x5635c7a7f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:07.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.205+0000 7f2af19f4700 1 --2- 172.21.15.37:0/1211224804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c7a84820 0x5635c7a8caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:07.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.206+0000 7f2af11f3700 1 -- 172.21.15.37:0/1211224804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635c7a8a900 msgr2=0x5635c7a7f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:07.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.206+0000 7f2af11f3700 1 --2- 172.21.15.37:0/1211224804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635c7a8a900 0x5635c7a7f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.206+0000 7f2af11f3700 1 -- 172.21.15.37:0/1211224804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c7a84820 msgr2=0x5635c7a8caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:07.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.206+0000 7f2af11f3700 1 --2- 172.21.15.37:0/1211224804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c7a84820 0x5635c7a8caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.206+0000 7f2af11f3700 1 -- 172.21.15.37:0/1211224804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635c7a8e870 con 0x5635c7a89fe0 2024-06-10T00:58:07.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.206+0000 7f2af11f3700 1 --2- 172.21.15.37:0/1211224804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5635c7a89fe0 0x5635c7a7ef00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2adc000c00 tx=0x7f2adc003af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df43f758513b3e96 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:07.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.206+0000 7f2ad77fe700 1 -- 172.21.15.37:0/1211224804 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc032070 con 0x5635c7a89fe0 2024-06-10T00:58:07.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.206+0000 7f2ad77fe700 1 -- 172.21.15.37:0/1211224804 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2adc0055b0 con 0x5635c7a89fe0 2024-06-10T00:58:07.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.206+0000 7f2ad77fe700 1 -- 172.21.15.37:0/1211224804 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc027040 con 0x5635c7a89fe0 2024-06-10T00:58:07.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.207+0000 7f2afe4dbb80 1 -- 172.21.15.37:0/1211224804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5635c7a89fe0 msgr2=0x5635c7a7ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:07.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.207+0000 7f2afe4dbb80 1 --2- 172.21.15.37:0/1211224804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5635c7a89fe0 0x5635c7a7ef00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2adc000c00 tx=0x7f2adc003af0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.207+0000 7f2afe4dbb80 1 -- 172.21.15.37:0/1211224804 shutdown_connections 2024-06-10T00:58:07.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.207+0000 7f2afe4dbb80 1 --2- 172.21.15.37:0/1211224804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635c7a8a900 0x5635c7a7f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.207+0000 7f2afe4dbb80 1 --2- 172.21.15.37:0/1211224804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c7a84820 0x5635c7a8caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.207+0000 7f2afe4dbb80 1 --2- 172.21.15.37:0/1211224804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5635c7a89fe0 0x5635c7a7ef00 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.207+0000 7f2afe4dbb80 1 -- 172.21.15.37:0/1211224804 >> 172.21.15.37:0/1211224804 conn(0x5635c79b1100 msgr2=0x5635c7a8b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:07.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.207+0000 7f2afe4dbb80 1 -- 172.21.15.37:0/1211224804 shutdown_connections 2024-06-10T00:58:07.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.207+0000 7f2afe4dbb80 1 -- 172.21.15.37:0/1211224804 wait complete. 2024-06-10T00:58:07.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.207+0000 7f2afe4dbb80 1 -- start start 2024-06-10T00:58:07.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.208+0000 7f2afe4dbb80 1 Processor -- start 2024-06-10T00:58:07.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.208+0000 7f2afe4dbb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5635c7a84820 0x5635c7a96d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:07.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.208+0000 7f2afe4dbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635c7a89fe0 0x5635c7a93510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:07.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.208+0000 7f2afe4dbb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c7a8a900 0x5635c7a93a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:07.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.208+0000 7f2afe4dbb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec1ac950 con 0x5635c7a84820 2024-06-10T00:58:07.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.208+0000 7f2afe4dbb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec1addc0 con 0x5635c7a8a900 2024-06-10T00:58:07.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.208+0000 7f2afe4dbb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec1ad020 con 0x5635c7a89fe0 2024-06-10T00:58:07.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.208+0000 7f2afe4dbb80 10 client.0 _ll_register_callbacks cb 0x5635c796b630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:58:07.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.208+0000 7f2af19f4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5635c7a84820 0x5635c7a96d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:07.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.208+0000 7f2af19f4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5635c7a84820 0x5635c7a96d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58232/0 (socket says 172.21.15.37:58232) 2024-06-10T00:58:07.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.208+0000 7f2af19f4700 1 -- 172.21.15.37:0/244543458 learned_addr learned my addr 172.21.15.37:0/244543458 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:07.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.208+0000 7f2af21f5700 1 --2- 172.21.15.37:0/244543458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c7a8a900 0x5635c7a93a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:07.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.208+0000 7f2af11f3700 1 --2- 172.21.15.37:0/244543458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635c7a89fe0 0x5635c7a93510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:07.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.208+0000 7f2af19f4700 1 -- 172.21.15.37:0/244543458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635c7a89fe0 msgr2=0x5635c7a93510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:07.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.208+0000 7f2af19f4700 1 --2- 172.21.15.37:0/244543458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635c7a89fe0 0x5635c7a93510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.209+0000 7f2af19f4700 1 -- 172.21.15.37:0/244543458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c7a8a900 msgr2=0x5635c7a93a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:07.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.209+0000 7f2af19f4700 1 --2- 172.21.15.37:0/244543458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c7a8a900 0x5635c7a93a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.209+0000 7f2af19f4700 1 -- 172.21.15.37:0/244543458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635c7a94320 con 0x5635c7a84820 2024-06-10T00:58:07.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.209+0000 7f2af21f5700 1 --2- 172.21.15.37:0/244543458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c7a8a900 0x5635c7a93a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:58:07.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.209+0000 7f2af19f4700 1 --2- 172.21.15.37:0/244543458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5635c7a84820 0x5635c7a96d00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x5635c7aa0c30 tx=0x7f2ae80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1eabba86bec604 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:07.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.209+0000 7f2ad57fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:58:07.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.209+0000 7f2ad57fa700 1 -- 172.21.15.37:0/244543458 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8016070 con 0x5635c7a84820 2024-06-10T00:58:07.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.209+0000 7f2ad57fa700 1 -- 172.21.15.37:0/244543458 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f2ae8005450 con 0x5635c7a84820 2024-06-10T00:58:07.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.209+0000 7f2afe4dbb80 1 -- 172.21.15.37:0/244543458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2adc0055b0 con 0x5635c7a84820 2024-06-10T00:58:07.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.209+0000 7f2ad57fa700 1 -- 172.21.15.37:0/244543458 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2ae8005650 con 0x5635c7a84820 2024-06-10T00:58:07.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.209+0000 7f2afe4dbb80 10 client.15747 fetch_fsmap learned FSMap version 27 2024-06-10T00:58:07.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.209+0000 7f2afe4dbb80 1 -- 172.21.15.37:0/244543458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5635c7b09db0 con 0x5635c7a84820 2024-06-10T00:58:07.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.209+0000 7f2ad57fa700 1 -- 172.21.15.37:0/244543458 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2adc0055b0 con 0x5635c7a84820 2024-06-10T00:58:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.209+0000 7f2ad57fa700 1 -- 172.21.15.37:0/244543458 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2ae8031610 con 0x5635c7a84820 2024-06-10T00:58:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.210+0000 7f2afe4dbb80 10 client.15747 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:58:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.210+0000 7f2afe4dbb80 10 client.15747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T00:58:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.210+0000 7f2afe4dbb80 20 client.15747 populate_metadata read hostname 'smithi037' 2024-06-10T00:58:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.210+0000 7f2afe4dbb80 1 --2- 172.21.15.37:0/244543458 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5635c7b0a410 0x5635c7b0c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.210+0000 7f2afe4dbb80 4 client.15747 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T00:58:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.210+0000 7f2afe4dbb80 1 -- 172.21.15.37:0/244543458 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5635c79dd350 con 0x5635c7b0a410 2024-06-10T00:58:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.210+0000 7f2af11f3700 1 --2- 172.21.15.37:0/244543458 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5635c7b0a410 0x5635c7b0c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.210+0000 7f2af11f3700 1 --2- 172.21.15.37:0/244543458 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5635c7b0a410 0x5635c7b0c820 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84d2b2f0b67b1ece server_cookie=1056956b877d85fc in_seq=0 out_seq=0 2024-06-10T00:58:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.210+0000 7f2ad57fa700 10 client.15747 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.212+0000 7f2ad57fa700 1 -- 172.21.15.37:0/244543458 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5635c79dd350 con 0x5635c7b0a410 2024-06-10T00:58:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.212+0000 7f2ad57fa700 10 client.15747 handle_command_reply: tid=0 2024-06-10T00:58:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.212+0000 7f2afe4dbb80 1 client.15747 shutdown 2024-06-10T00:58:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.212+0000 7f2afe4dbb80 1 -- 172.21.15.37:0/244543458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5635c7a84820 msgr2=0x5635c7a96d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.212+0000 7f2afe4dbb80 1 --2- 172.21.15.37:0/244543458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5635c7a84820 0x5635c7a96d00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x5635c7aa0c30 tx=0x7f2ae80047c0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.212+0000 7f2afe4dbb80 1 -- 172.21.15.37:0/244543458 shutdown_connections 2024-06-10T00:58:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.212+0000 7f2afe4dbb80 1 --2- 172.21.15.37:0/244543458 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5635c7b0a410 0x5635c7b0c820 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.212+0000 7f2afe4dbb80 1 --2- 172.21.15.37:0/244543458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c7a8a900 0x5635c7a93a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.212+0000 7f2ad57fa700 0 client.15747 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:07.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.212+0000 7f2afe4dbb80 1 --2- 172.21.15.37:0/244543458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5635c7a84820 0x5635c7a96d00 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.212+0000 7f2afe4dbb80 1 --2- 172.21.15.37:0/244543458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635c7a89fe0 0x5635c7a93510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.212+0000 7f2afe4dbb80 1 -- 172.21.15.37:0/244543458 >> 172.21.15.37:0/244543458 conn(0x5635c79b1100 msgr2=0x5635c7a8caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:07.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.212+0000 7f2afe4dbb80 1 -- 172.21.15.37:0/244543458 shutdown_connections 2024-06-10T00:58:07.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.212+0000 7f2afe4dbb80 1 -- 172.21.15.37:0/244543458 wait complete. 2024-06-10T00:58:07.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.212+0000 7f2afe4dbb80 20 client.15747 trim_cache size 0 max 16384 2024-06-10T00:58:07.216 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T00:58:07.216 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-06-10T00:58:07.216 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "3fd4a4ad-7046-4b20-931d-89553933d670", 2024-06-10T00:58:07.216 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-06-10T00:58:07.216 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T00:58:07.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.213+0000 7f2af3c58700 1 -- 172.21.15.37:0/2933153057 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2ad80a2ca0 msgr2=0x7f2ad80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:07.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.213+0000 7f2af3c58700 1 --2- 172.21.15.37:0/2933153057 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2ad80a2ca0 0x7f2ad80a5160 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8005950 tx=0x7f2ae800a300 comp rx=0 tx=0).stop 2024-06-10T00:58:07.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.213+0000 7f2af3c58700 1 -- 172.21.15.37:0/2933153057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2aec108520 msgr2=0x7f2aec1989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:07.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.213+0000 7f2af3c58700 1 --2- 172.21.15.37:0/2933153057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2aec108520 0x7f2aec1989d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400ab50 tx=0x7f2ae400ae60 comp rx=0 tx=0).stop 2024-06-10T00:58:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.213+0000 7f2af3c58700 1 -- 172.21.15.37:0/2933153057 shutdown_connections 2024-06-10T00:58:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.213+0000 7f2af3c58700 1 --2- 172.21.15.37:0/2933153057 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2ad80a2ca0 0x7f2ad80a5160 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.213+0000 7f2af3c58700 1 --2- 172.21.15.37:0/2933153057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2aec108520 0x7f2aec1989d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.213+0000 7f2af3c58700 1 --2- 172.21.15.37:0/2933153057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2aec102510 0x7f2aec19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.213+0000 7f2af3c58700 1 --2- 172.21.15.37:0/2933153057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2aec102ed0 0x7f2aec19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.213+0000 7f2af3c58700 1 -- 172.21.15.37:0/2933153057 >> 172.21.15.37:0/2933153057 conn(0x7f2aec079640 msgr2=0x7f2aec101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.213+0000 7f2af3c58700 1 -- 172.21.15.37:0/2933153057 shutdown_connections 2024-06-10T00:58:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:07.213+0000 7f2af3c58700 1 -- 172.21.15.37:0/2933153057 wait complete. 2024-06-10T00:58:07.539 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:58:08.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:08 smithi037 ceph-mon[32192]: pgmap v473: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-10T00:58:08.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:08 smithi037 ceph-mon[32192]: scrub queued for path: / 2024-06-10T00:58:08.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:08 smithi037 ceph-mon[32192]: scrub summary: idle+waiting paths [/] 2024-06-10T00:58:08.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:08 smithi037 ceph-mon[32192]: scrub summary: active paths [/] 2024-06-10T00:58:08.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:08 smithi037 ceph-mon[32192]: scrub summary: idle+waiting paths [/] 2024-06-10T00:58:08.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:08 smithi167 ceph-mon[36917]: pgmap v473: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-10T00:58:08.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:08 smithi167 ceph-mon[36917]: scrub queued for path: / 2024-06-10T00:58:08.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:08 smithi167 ceph-mon[36917]: scrub summary: idle+waiting paths [/] 2024-06-10T00:58:08.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:08 smithi167 ceph-mon[36917]: scrub summary: active paths [/] 2024-06-10T00:58:08.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:08 smithi167 ceph-mon[36917]: scrub summary: idle+waiting paths [/] 2024-06-10T00:58:08.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:08 smithi094 ceph-mon[36708]: pgmap v473: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-10T00:58:08.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:08 smithi094 ceph-mon[36708]: scrub queued for path: / 2024-06-10T00:58:08.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:08 smithi094 ceph-mon[36708]: scrub summary: idle+waiting paths [/] 2024-06-10T00:58:08.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:08 smithi094 ceph-mon[36708]: scrub summary: active paths [/] 2024-06-10T00:58:08.615 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:08 smithi094 ceph-mon[36708]: scrub summary: idle+waiting paths [/] 2024-06-10T00:58:09.106 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-06-10T00:58:09.215 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-06-10T00:58:09.257 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-10T00:58:09.259 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-10T00:58:09.259 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-06-10T00:58:09.299 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-10T00:58:09.303 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-06-10T00:58:09.303 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-06-10T00:58:09.319 INFO:tasks.workunit:Running workunits matching fs/misc on client.0... 2024-06-10T00:58:09.320 INFO:tasks.workunit:Running workunit fs/misc/acl.sh... 2024-06-10T00:58:09.321 DEBUG:teuthology.orchestra.run.smithi037:workunit test fs/misc/acl.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b7a5d375cf4e4afab96b9d4d229b699ae8304bb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/acl.sh 2024-06-10T00:58:09.520 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T00:58:10.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.300+0000 7f4372240700 1 -- 172.21.15.37:0/3781793708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f436c1030f0 msgr2=0x7f436c107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:10.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.300+0000 7f4372240700 1 --2- 172.21.15.37:0/3781793708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f436c1030f0 0x7f436c107110 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f43600099c0 tx=0x7f4360009cd0 comp rx=0 tx=0).stop 2024-06-10T00:58:10.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.300+0000 7f4372240700 1 -- 172.21.15.37:0/3781793708 shutdown_connections 2024-06-10T00:58:10.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.300+0000 7f4372240700 1 --2- 172.21.15.37:0/3781793708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f436c107840 0x7f436c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:10.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.300+0000 7f4372240700 1 --2- 172.21.15.37:0/3781793708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f436c102740 0x7f436c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:10.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.300+0000 7f4372240700 1 --2- 172.21.15.37:0/3781793708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f436c1030f0 0x7f436c107110 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:10.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.300+0000 7f4372240700 1 -- 172.21.15.37:0/3781793708 >> 172.21.15.37:0/3781793708 conn(0x7f436c0795f0 msgr2=0x7f436c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:10.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.300+0000 7f4372240700 1 -- 172.21.15.37:0/3781793708 shutdown_connections 2024-06-10T00:58:10.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.300+0000 7f4372240700 1 -- 172.21.15.37:0/3781793708 wait complete. 2024-06-10T00:58:10.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.301+0000 7f4372240700 1 Processor -- start 2024-06-10T00:58:10.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.301+0000 7f4372240700 1 -- start start 2024-06-10T00:58:10.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.301+0000 7f4372240700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f436c102740 0x7f436c19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:10.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.301+0000 7f4372240700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f436c1030f0 0x7f436c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:10.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.301+0000 7f4372240700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f436c107840 0x7f436c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:10.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.301+0000 7f4372240700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f436c1127c0 con 0x7f436c107840 2024-06-10T00:58:10.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.301+0000 7f4372240700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f436c112640 con 0x7f436c102740 2024-06-10T00:58:10.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.301+0000 7f4372240700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f436c112940 con 0x7f436c1030f0 2024-06-10T00:58:10.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.302+0000 7f436bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f436c107840 0x7f436c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:10.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.302+0000 7f436bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f436c107840 0x7f436c198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58256/0 (socket says 172.21.15.37:58256) 2024-06-10T00:58:10.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.302+0000 7f436bfff700 1 -- 172.21.15.37:0/3316816625 learned_addr learned my addr 172.21.15.37:0/3316816625 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:10.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.302+0000 7f436b7fe700 1 --2- 172.21.15.37:0/3316816625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f436c102740 0x7f436c19e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:10.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.302+0000 7f436affd700 1 --2- 172.21.15.37:0/3316816625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f436c1030f0 0x7f436c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:10.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.302+0000 7f436bfff700 1 -- 172.21.15.37:0/3316816625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f436c1030f0 msgr2=0x7f436c19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:10.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.302+0000 7f436bfff700 1 --2- 172.21.15.37:0/3316816625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f436c1030f0 0x7f436c19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:10.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.302+0000 7f436bfff700 1 -- 172.21.15.37:0/3316816625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f436c102740 msgr2=0x7f436c19e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:10.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.302+0000 7f436bfff700 1 --2- 172.21.15.37:0/3316816625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f436c102740 0x7f436c19e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:10.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.302+0000 7f436bfff700 1 -- 172.21.15.37:0/3316816625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f436c199160 con 0x7f436c107840 2024-06-10T00:58:10.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.302+0000 7f436affd700 1 --2- 172.21.15.37:0/3316816625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f436c1030f0 0x7f436c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:58:10.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.302+0000 7f436bfff700 1 --2- 172.21.15.37:0/3316816625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f436c107840 0x7f436c198a00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4364007d70 tx=0x7f436400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:10.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.302+0000 7f4368ff9700 1 -- 172.21.15.37:0/3316816625 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364016020 con 0x7f436c107840 2024-06-10T00:58:10.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.302+0000 7f4372240700 1 -- 172.21.15.37:0/3316816625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f436c199450 con 0x7f436c107840 2024-06-10T00:58:10.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.302+0000 7f4368ff9700 1 -- 172.21.15.37:0/3316816625 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f436400ad40 con 0x7f436c107840 2024-06-10T00:58:10.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.302+0000 7f4368ff9700 1 -- 172.21.15.37:0/3316816625 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364011600 con 0x7f436c107840 2024-06-10T00:58:10.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.302+0000 7f4372240700 1 -- 172.21.15.37:0/3316816625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f436c1a56c0 con 0x7f436c107840 2024-06-10T00:58:10.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.303+0000 7f4372240700 1 -- 172.21.15.37:0/3316816625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f436c10aa10 con 0x7f436c107840 2024-06-10T00:58:10.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.304+0000 7f4368ff9700 1 -- 172.21.15.37:0/3316816625 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4364019020 con 0x7f436c107840 2024-06-10T00:58:10.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.305+0000 7f4368ff9700 1 --2- 172.21.15.37:0/3316816625 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f43540a2c20 0x7f43540a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:10.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.305+0000 7f436b7fe700 1 --2- 172.21.15.37:0/3316816625 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f43540a2c20 0x7f43540a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:10.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.305+0000 7f4368ff9700 1 -- 172.21.15.37:0/3316816625 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f43640cf790 con 0x7f436c107840 2024-06-10T00:58:10.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.306+0000 7f436b7fe700 1 --2- 172.21.15.37:0/3316816625 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f43540a2c20 0x7f43540a50e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f436c1073a0 tx=0x7f435c00b3c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:10.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.309+0000 7f4368ff9700 1 -- 172.21.15.37:0/3316816625 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4364020020 con 0x7f436c107840 2024-06-10T00:58:10.400 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:58:10.523 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:10 smithi037 ceph-mon[32192]: pgmap v474: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-10T00:58:10.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.571+0000 7f4372240700 1 -- 172.21.15.37:0/3316816625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f436c069180 con 0x7f436c107840 2024-06-10T00:58:10.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.572+0000 7f4368ff9700 1 -- 172.21.15.37:0/3316816625 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11867 (secure 0 0 0) 0x7f436409c300 con 0x7f436c107840 2024-06-10T00:58:10.573 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:58:10.573 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T00:56:30.291171+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24451,24458,24472]},"id":1}]} 2024-06-10T00:58:10.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.574+0000 7f4372240700 1 -- 172.21.15.37:0/3316816625 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f43540a2c20 msgr2=0x7f43540a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:10.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.574+0000 7f4372240700 1 --2- 172.21.15.37:0/3316816625 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f43540a2c20 0x7f43540a50e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f436c1073a0 tx=0x7f435c00b3c0 comp rx=0 tx=0).stop 2024-06-10T00:58:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.574+0000 7f4372240700 1 -- 172.21.15.37:0/3316816625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f436c107840 msgr2=0x7f436c198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.575+0000 7f4372240700 1 --2- 172.21.15.37:0/3316816625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f436c107840 0x7f436c198a00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4364007d70 tx=0x7f436400a340 comp rx=0 tx=0).stop 2024-06-10T00:58:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.575+0000 7f4372240700 1 -- 172.21.15.37:0/3316816625 shutdown_connections 2024-06-10T00:58:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.575+0000 7f4372240700 1 --2- 172.21.15.37:0/3316816625 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f43540a2c20 0x7f43540a50e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.575+0000 7f4372240700 1 --2- 172.21.15.37:0/3316816625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f436c107840 0x7f436c198a00 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.575+0000 7f4372240700 1 --2- 172.21.15.37:0/3316816625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f436c102740 0x7f436c19e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.575+0000 7f4372240700 1 --2- 172.21.15.37:0/3316816625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f436c1030f0 0x7f436c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.575+0000 7f4372240700 1 -- 172.21.15.37:0/3316816625 >> 172.21.15.37:0/3316816625 conn(0x7f436c0795f0 msgr2=0x7f436c101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.575+0000 7f4372240700 1 -- 172.21.15.37:0/3316816625 shutdown_connections 2024-06-10T00:58:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:10.575+0000 7f4372240700 1 -- 172.21.15.37:0/3316816625 wait complete. 2024-06-10T00:58:10.577 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-06-10T00:58:10.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:10 smithi167 ceph-mon[36917]: pgmap v474: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-10T00:58:10.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:10 smithi094 ceph-mon[36708]: pgmap v474: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 7 op/s 2024-06-10T00:58:11.393 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24472 quiesce db --set-id f993ef51 --release --await' 2024-06-10T00:58:11.393 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24472 quiesce db --set-id f993ef51 --release --await 2024-06-10T00:58:11.530 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:11 smithi037 ceph-mon[32192]: scrub summary: idle 2024-06-10T00:58:11.530 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:11 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3316816625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:58:11.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:11 smithi167 ceph-mon[36917]: scrub summary: idle 2024-06-10T00:58:11.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:11 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3316816625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:58:11.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:11 smithi094 ceph-mon[36708]: scrub summary: idle 2024-06-10T00:58:11.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:11 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3316816625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:58:11.887 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:58:12.586 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:12 smithi037 ceph-mon[32192]: pgmap v475: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T00:58:12.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:12 smithi167 ceph-mon[36917]: pgmap v475: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T00:58:12.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:12 smithi094 ceph-mon[36708]: pgmap v475: 129 pgs: 129 active+clean; 681 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-10T00:58:13.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.495+0000 7f92538b1700 1 -- 172.21.15.37:0/3327160908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f924c108500 msgr2=0x7f924c1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:13.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.495+0000 7f92538b1700 1 --2- 172.21.15.37:0/3327160908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f924c108500 0x7f924c1088e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9248009990 tx=0x7f9248009ca0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.495+0000 7f92538b1700 1 -- 172.21.15.37:0/3327160908 shutdown_connections 2024-06-10T00:58:13.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.495+0000 7f92538b1700 1 --2- 172.21.15.37:0/3327160908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f924c06ab30 0x7f924c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.495+0000 7f92538b1700 1 --2- 172.21.15.37:0/3327160908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f924c108500 0x7f924c1088e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.495+0000 7f92538b1700 1 --2- 172.21.15.37:0/3327160908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f924c06a170 0x7f924c06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.495+0000 7f92538b1700 1 -- 172.21.15.37:0/3327160908 >> 172.21.15.37:0/3327160908 conn(0x7f924c078340 msgr2=0x7f924c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:13.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.495+0000 7f92538b1700 1 -- 172.21.15.37:0/3327160908 shutdown_connections 2024-06-10T00:58:13.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.495+0000 7f92538b1700 1 -- 172.21.15.37:0/3327160908 wait complete. 2024-06-10T00:58:13.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.496+0000 7f92538b1700 1 Processor -- start 2024-06-10T00:58:13.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.496+0000 7f92538b1700 1 -- start start 2024-06-10T00:58:13.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.496+0000 7f92538b1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f924c06a170 0x7f924c19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:13.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.496+0000 7f92538b1700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f924c06ab30 0x7f924c19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:13.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.496+0000 7f92538b1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f924c108500 0x7f924c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:13.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.496+0000 7f92538b1700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f924c112760 con 0x7f924c06a170 2024-06-10T00:58:13.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.496+0000 7f92538b1700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1125e0 con 0x7f924c108500 2024-06-10T00:58:13.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.496+0000 7f92538b1700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1128e0 con 0x7f924c06ab30 2024-06-10T00:58:13.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.496+0000 7f925164d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f924c06a170 0x7f924c19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:13.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.496+0000 7f925164d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f924c06a170 0x7f924c19e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42052/0 (socket says 172.21.15.37:42052) 2024-06-10T00:58:13.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.496+0000 7f925164d700 1 -- 172.21.15.37:0/1754054855 learned_addr learned my addr 172.21.15.37:0/1754054855 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:13.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.497+0000 7f9251e4e700 1 --2- 172.21.15.37:0/1754054855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f924c108500 0x7f924c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:13.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.497+0000 7f9250e4c700 1 --2- 172.21.15.37:0/1754054855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f924c06ab30 0x7f924c19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:13.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.497+0000 7f925164d700 1 -- 172.21.15.37:0/1754054855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f924c06ab30 msgr2=0x7f924c19ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:13.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.497+0000 7f925164d700 1 --2- 172.21.15.37:0/1754054855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f924c06ab30 0x7f924c19ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.497+0000 7f925164d700 1 -- 172.21.15.37:0/1754054855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f924c108500 msgr2=0x7f924c198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:13.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.497+0000 7f925164d700 1 --2- 172.21.15.37:0/1754054855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f924c108500 0x7f924c198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.497+0000 7f925164d700 1 -- 172.21.15.37:0/1754054855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924c1990e0 con 0x7f924c06a170 2024-06-10T00:58:13.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.497+0000 7f9250e4c700 1 --2- 172.21.15.37:0/1754054855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f924c06ab30 0x7f924c19ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:58:13.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.497+0000 7f925164d700 1 --2- 172.21.15.37:0/1754054855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f924c06a170 0x7f924c19e7c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f92480038b0 tx=0x7f9248009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:13.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.497+0000 7f92427fc700 1 -- 172.21.15.37:0/1754054855 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9248027070 con 0x7f924c06a170 2024-06-10T00:58:13.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.497+0000 7f92427fc700 1 -- 172.21.15.37:0/1754054855 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9248004050 con 0x7f924c06a170 2024-06-10T00:58:13.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.497+0000 7f92427fc700 1 -- 172.21.15.37:0/1754054855 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9248022cd0 con 0x7f924c06a170 2024-06-10T00:58:13.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.497+0000 7f92538b1700 1 -- 172.21.15.37:0/1754054855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924c199370 con 0x7f924c06a170 2024-06-10T00:58:13.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.497+0000 7f92538b1700 1 -- 172.21.15.37:0/1754054855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f924c101390 con 0x7f924c06a170 2024-06-10T00:58:13.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.500+0000 7f92427fc700 1 -- 172.21.15.37:0/1754054855 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f924802a070 con 0x7f924c06a170 2024-06-10T00:58:13.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.500+0000 7f92427fc700 1 --2- 172.21.15.37:0/1754054855 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f92380a2ca0 0x7f92380a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:13.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.500+0000 7f92427fc700 1 -- 172.21.15.37:0/1754054855 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f92480d0230 con 0x7f924c06a170 2024-06-10T00:58:13.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.501+0000 7f9250e4c700 1 --2- 172.21.15.37:0/1754054855 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f92380a2ca0 0x7f92380a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:13.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.501+0000 7f9250e4c700 1 --2- 172.21.15.37:0/1754054855 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f92380a2ca0 0x7f92380a5160 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f923c005950 tx=0x7f923c0058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:13.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.501+0000 7f925e134b80 1 Processor -- start 2024-06-10T00:58:13.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.501+0000 7f925e134b80 1 -- start start 2024-06-10T00:58:13.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.502+0000 7f925e134b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9f848d640 0x55c9f848da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:13.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.502+0000 7f925e134b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f848df60 0x55c9f84879e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:13.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.502+0000 7f925e134b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9f8487f20 0x55c9f8488340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:13.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.502+0000 7f925e134b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c9f83d9810 con 0x55c9f848df60 2024-06-10T00:58:13.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.502+0000 7f925e134b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c9f8381720 con 0x55c9f848d640 2024-06-10T00:58:13.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.502+0000 7f925e134b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c9f83d57d0 con 0x55c9f8487f20 2024-06-10T00:58:13.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.502+0000 7f9250e4c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f848df60 0x55c9f84879e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:13.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.502+0000 7f9250e4c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f848df60 0x55c9f84879e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42068/0 (socket says 172.21.15.37:42068) 2024-06-10T00:58:13.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.502+0000 7f9250e4c700 1 -- 172.21.15.37:0/70638822 learned_addr learned my addr 172.21.15.37:0/70638822 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:13.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.502+0000 7f9251e4e700 1 --2- 172.21.15.37:0/70638822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9f8487f20 0x55c9f8488340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:13.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.502+0000 7f925164d700 1 --2- 172.21.15.37:0/70638822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9f848d640 0x55c9f848da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:13.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.502+0000 7f9250e4c700 1 -- 172.21.15.37:0/70638822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9f8487f20 msgr2=0x55c9f8488340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:13.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.502+0000 7f9250e4c700 1 --2- 172.21.15.37:0/70638822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9f8487f20 0x55c9f8488340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.502+0000 7f9250e4c700 1 -- 172.21.15.37:0/70638822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9f848d640 msgr2=0x55c9f848da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:13.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.502+0000 7f9250e4c700 1 --2- 172.21.15.37:0/70638822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9f848d640 0x55c9f848da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.502+0000 7f9250e4c700 1 -- 172.21.15.37:0/70638822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9f8377f50 con 0x55c9f848df60 2024-06-10T00:58:13.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.502+0000 7f9251e4e700 1 --2- 172.21.15.37:0/70638822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9f8487f20 0x55c9f8488340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:58:13.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.503+0000 7f9250e4c700 1 --2- 172.21.15.37:0/70638822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f848df60 0x55c9f84879e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f923c0041c0 tx=0x7f923c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b22751a19e2d70f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:13.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.503+0000 7f92377fe700 1 -- 172.21.15.37:0/70638822 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c003d10 con 0x55c9f848df60 2024-06-10T00:58:13.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.503+0000 7f92377fe700 1 -- 172.21.15.37:0/70638822 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f923c01d960 con 0x55c9f848df60 2024-06-10T00:58:13.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.503+0000 7f92377fe700 1 -- 172.21.15.37:0/70638822 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c0288c0 con 0x55c9f848df60 2024-06-10T00:58:13.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.504+0000 7f925e134b80 1 -- 172.21.15.37:0/70638822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f848df60 msgr2=0x55c9f84879e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:13.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.504+0000 7f925e134b80 1 --2- 172.21.15.37:0/70638822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f848df60 0x55c9f84879e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f923c0041c0 tx=0x7f923c016000 comp rx=0 tx=0).stop 2024-06-10T00:58:13.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.504+0000 7f925e134b80 1 -- 172.21.15.37:0/70638822 shutdown_connections 2024-06-10T00:58:13.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.504+0000 7f925e134b80 1 --2- 172.21.15.37:0/70638822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9f8487f20 0x55c9f8488340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.504+0000 7f925e134b80 1 --2- 172.21.15.37:0/70638822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9f848d640 0x55c9f848da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.504+0000 7f925e134b80 1 --2- 172.21.15.37:0/70638822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f848df60 0x55c9f84879e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.504+0000 7f925e134b80 1 -- 172.21.15.37:0/70638822 >> 172.21.15.37:0/70638822 conn(0x55c9f83b2cc0 msgr2=0x55c9f8482850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:13.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.504+0000 7f925e134b80 1 -- 172.21.15.37:0/70638822 shutdown_connections 2024-06-10T00:58:13.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.504+0000 7f925e134b80 1 -- 172.21.15.37:0/70638822 wait complete. 2024-06-10T00:58:13.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.504+0000 7f925e134b80 1 -- start start 2024-06-10T00:58:13.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.504+0000 7f925e134b80 1 Processor -- start 2024-06-10T00:58:13.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.505+0000 7f925e134b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f8487f20 0x55c9f8493d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:13.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.505+0000 7f925e134b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9f848d640 0x55c9f8491010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:13.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.505+0000 7f925e134b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9f848df60 0x55c9f8491550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:13.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.505+0000 7f925e134b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c9f84949d0 con 0x55c9f8487f20 2024-06-10T00:58:13.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.505+0000 7f925e134b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c9f8379bf0 con 0x55c9f848d640 2024-06-10T00:58:13.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.505+0000 7f925e134b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c9f8494b50 con 0x55c9f848df60 2024-06-10T00:58:13.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.505+0000 7f925e134b80 10 client.0 _ll_register_callbacks cb 0x55c9f8373920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:58:13.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.505+0000 7f925164d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f8487f20 0x55c9f8493d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:13.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.505+0000 7f925164d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f8487f20 0x55c9f8493d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42082/0 (socket says 172.21.15.37:42082) 2024-06-10T00:58:13.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.505+0000 7f925164d700 1 -- 172.21.15.37:0/2889604526 learned_addr learned my addr 172.21.15.37:0/2889604526 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:13.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.505+0000 7f9251e4e700 1 --2- 172.21.15.37:0/2889604526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9f848df60 0x55c9f8491550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:13.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.505+0000 7f9250e4c700 1 --2- 172.21.15.37:0/2889604526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9f848d640 0x55c9f8491010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:13.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.505+0000 7f925164d700 1 -- 172.21.15.37:0/2889604526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9f848df60 msgr2=0x55c9f8491550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:13.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.505+0000 7f925164d700 1 --2- 172.21.15.37:0/2889604526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9f848df60 0x55c9f8491550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.505+0000 7f925164d700 1 -- 172.21.15.37:0/2889604526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9f848d640 msgr2=0x55c9f8491010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:13.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.505+0000 7f925164d700 1 --2- 172.21.15.37:0/2889604526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9f848d640 0x55c9f8491010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.505+0000 7f925164d700 1 -- 172.21.15.37:0/2889604526 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9f83288d0 con 0x55c9f8487f20 2024-06-10T00:58:13.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.505+0000 7f925164d700 1 --2- 172.21.15.37:0/2889604526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f8487f20 0x55c9f8493d10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9248005f50 tx=0x7f92480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb97fbc7ffc9f405 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:13.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.505+0000 7f9234ff9700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:58:13.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.506+0000 7f9234ff9700 1 -- 172.21.15.37:0/2889604526 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924809ce30 con 0x55c9f8487f20 2024-06-10T00:58:13.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.506+0000 7f9234ff9700 1 -- 172.21.15.37:0/2889604526 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f924809aa30 con 0x55c9f8487f20 2024-06-10T00:58:13.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.506+0000 7f925e134b80 1 -- 172.21.15.37:0/2889604526 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f923c01d960 con 0x55c9f8487f20 2024-06-10T00:58:13.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.506+0000 7f9234ff9700 1 -- 172.21.15.37:0/2889604526 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f924809c730 con 0x55c9f8487f20 2024-06-10T00:58:13.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.506+0000 7f925e134b80 10 client.15765 fetch_fsmap learned FSMap version 27 2024-06-10T00:58:13.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.506+0000 7f925e134b80 1 -- 172.21.15.37:0/2889604526 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c9f849f280 con 0x55c9f8487f20 2024-06-10T00:58:13.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.506+0000 7f9234ff9700 1 -- 172.21.15.37:0/2889604526 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f923c01d960 con 0x55c9f8487f20 2024-06-10T00:58:13.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.506+0000 7f9234ff9700 1 -- 172.21.15.37:0/2889604526 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f92480a4e80 con 0x55c9f8487f20 2024-06-10T00:58:13.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.506+0000 7f925e134b80 10 client.15765 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:58:13.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.506+0000 7f925e134b80 10 client.15765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T00:58:13.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.506+0000 7f925e134b80 20 client.15765 populate_metadata read hostname 'smithi037' 2024-06-10T00:58:13.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.506+0000 7f925e134b80 1 --2- 172.21.15.37:0/2889604526 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c9f84a1750 0x55c9f84a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:13.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.506+0000 7f925e134b80 4 client.15765 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:58:13.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.506+0000 7f925e134b80 1 -- 172.21.15.37:0/2889604526 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9f83e6d10 con 0x55c9f84a1750 2024-06-10T00:58:13.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.507+0000 7f9250e4c700 1 --2- 172.21.15.37:0/2889604526 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c9f84a1750 0x55c9f84a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:13.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.507+0000 7f9234ff9700 10 client.15765 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:13.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.507+0000 7f9250e4c700 1 --2- 172.21.15.37:0/2889604526 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c9f84a1750 0x55c9f84a3b60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2bd79749fef844d server_cookie=e2559a6fbac96d07 in_seq=0 out_seq=0 2024-06-10T00:58:13.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.508+0000 7f9234ff9700 1 -- 172.21.15.37:0/2889604526 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c9f83e6d10 con 0x55c9f84a1750 2024-06-10T00:58:13.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.508+0000 7f9234ff9700 10 client.15765 handle_command_reply: tid=0 2024-06-10T00:58:13.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.508+0000 7f925e134b80 1 client.15765 shutdown 2024-06-10T00:58:13.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.508+0000 7f925e134b80 1 -- 172.21.15.37:0/2889604526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f8487f20 msgr2=0x55c9f8493d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:13.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.508+0000 7f925e134b80 1 --2- 172.21.15.37:0/2889604526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f8487f20 0x55c9f8493d10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9248005f50 tx=0x7f92480057f0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.508+0000 7f925e134b80 1 -- 172.21.15.37:0/2889604526 shutdown_connections 2024-06-10T00:58:13.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.508+0000 7f925e134b80 1 --2- 172.21.15.37:0/2889604526 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c9f84a1750 0x55c9f84a3b60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.508+0000 7f9234ff9700 0 client.15765 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:13.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.508+0000 7f925e134b80 1 --2- 172.21.15.37:0/2889604526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9f848df60 0x55c9f8491550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.508+0000 7f925e134b80 1 --2- 172.21.15.37:0/2889604526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f8487f20 0x55c9f8493d10 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.508+0000 7f925e134b80 1 --2- 172.21.15.37:0/2889604526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9f848d640 0x55c9f8491010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.508+0000 7f925e134b80 1 -- 172.21.15.37:0/2889604526 >> 172.21.15.37:0/2889604526 conn(0x55c9f83b2cc0 msgr2=0x55c9f84869d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:13.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.509+0000 7f925e134b80 1 -- 172.21.15.37:0/2889604526 shutdown_connections 2024-06-10T00:58:13.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.509+0000 7f925e134b80 1 -- 172.21.15.37:0/2889604526 wait complete. 2024-06-10T00:58:13.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.509+0000 7f925e134b80 20 client.15765 trim_cache size 0 max 16384 2024-06-10T00:58:13.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.530+0000 7f925e134b80 1 Processor -- start 2024-06-10T00:58:13.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.530+0000 7f925e134b80 1 -- start start 2024-06-10T00:58:13.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.530+0000 7f925e134b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9f8487f20 0x55c9f84879e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:13.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.530+0000 7f925e134b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9f848d640 0x55c9f848ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:13.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.530+0000 7f925e134b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f848df60 0x55c9f847c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:13.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.530+0000 7f925e134b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1ae970 con 0x55c9f848df60 2024-06-10T00:58:13.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.530+0000 7f925e134b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1ac200 con 0x55c9f848d640 2024-06-10T00:58:13.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.530+0000 7f925e134b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1aa050 con 0x55c9f8487f20 2024-06-10T00:58:13.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.530+0000 7f9251e4e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f848df60 0x55c9f847c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:13.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.530+0000 7f9251e4e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f848df60 0x55c9f847c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42086/0 (socket says 172.21.15.37:42086) 2024-06-10T00:58:13.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.530+0000 7f9251e4e700 1 -- 172.21.15.37:0/1181924408 learned_addr learned my addr 172.21.15.37:0/1181924408 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:13.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.530+0000 7f925164d700 1 --2- 172.21.15.37:0/1181924408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9f8487f20 0x55c9f84879e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:13.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.530+0000 7f9250e4c700 1 --2- 172.21.15.37:0/1181924408 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9f848d640 0x55c9f848ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:13.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.530+0000 7f9251e4e700 1 -- 172.21.15.37:0/1181924408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9f8487f20 msgr2=0x55c9f84879e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:13.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.530+0000 7f9251e4e700 1 --2- 172.21.15.37:0/1181924408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9f8487f20 0x55c9f84879e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.530+0000 7f9251e4e700 1 -- 172.21.15.37:0/1181924408 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9f848d640 msgr2=0x55c9f848ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:13.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.530+0000 7f9251e4e700 1 --2- 172.21.15.37:0/1181924408 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9f848d640 0x55c9f848ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.531+0000 7f9251e4e700 1 -- 172.21.15.37:0/1181924408 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9f8491f50 con 0x55c9f848df60 2024-06-10T00:58:13.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.531+0000 7f9250e4c700 1 --2- 172.21.15.37:0/1181924408 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9f848d640 0x55c9f848ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:58:13.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.531+0000 7f9251e4e700 1 --2- 172.21.15.37:0/1181924408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f848df60 0x55c9f847c510 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9244014720 tx=0x7f9244007950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a18bc18b392185 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:13.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.531+0000 7f9236ffd700 1 -- 172.21.15.37:0/1181924408 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9244016630 con 0x55c9f848df60 2024-06-10T00:58:13.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.531+0000 7f9236ffd700 1 -- 172.21.15.37:0/1181924408 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9244016cb0 con 0x55c9f848df60 2024-06-10T00:58:13.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.531+0000 7f9236ffd700 1 -- 172.21.15.37:0/1181924408 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92440153b0 con 0x55c9f848df60 2024-06-10T00:58:13.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.532+0000 7f925e134b80 1 -- 172.21.15.37:0/1181924408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f848df60 msgr2=0x55c9f847c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:13.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.532+0000 7f925e134b80 1 --2- 172.21.15.37:0/1181924408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f848df60 0x55c9f847c510 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9244014720 tx=0x7f9244007950 comp rx=0 tx=0).stop 2024-06-10T00:58:13.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.532+0000 7f925e134b80 1 -- 172.21.15.37:0/1181924408 shutdown_connections 2024-06-10T00:58:13.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.532+0000 7f925e134b80 1 --2- 172.21.15.37:0/1181924408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f848df60 0x55c9f847c510 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.532+0000 7f925e134b80 1 --2- 172.21.15.37:0/1181924408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9f8487f20 0x55c9f84879e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.532+0000 7f925e134b80 1 --2- 172.21.15.37:0/1181924408 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9f848d640 0x55c9f848ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.532+0000 7f925e134b80 1 -- 172.21.15.37:0/1181924408 >> 172.21.15.37:0/1181924408 conn(0x55c9f83b2cc0 msgr2=0x55c9f847be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:13.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.532+0000 7f925e134b80 1 -- 172.21.15.37:0/1181924408 shutdown_connections 2024-06-10T00:58:13.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.532+0000 7f925e134b80 1 -- 172.21.15.37:0/1181924408 wait complete. 2024-06-10T00:58:13.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.532+0000 7f925e134b80 1 -- start start 2024-06-10T00:58:13.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.532+0000 7f925e134b80 1 Processor -- start 2024-06-10T00:58:13.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.533+0000 7f925e134b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9f8487f20 0x55c9f8499e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:13.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.533+0000 7f925e134b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f848d640 0x55c9f849a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:13.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.533+0000 7f925e134b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9f848df60 0x55c9f8496de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:13.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.533+0000 7f925e134b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1aa6c0 con 0x55c9f848d640 2024-06-10T00:58:13.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.533+0000 7f925e134b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1abb30 con 0x55c9f848df60 2024-06-10T00:58:13.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.533+0000 7f925e134b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1aad90 con 0x55c9f8487f20 2024-06-10T00:58:13.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.533+0000 7f925e134b80 10 client.0 _ll_register_callbacks cb 0x55c9f8373920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:58:13.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.533+0000 7f9250e4c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f848d640 0x55c9f849a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:13.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.533+0000 7f925164d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9f8487f20 0x55c9f8499e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:13.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.533+0000 7f925164d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9f8487f20 0x55c9f8499e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:57166/0 (socket says 172.21.15.37:57166) 2024-06-10T00:58:13.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.533+0000 7f9251e4e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9f848df60 0x55c9f8496de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:13.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.533+0000 7f9250e4c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f848d640 0x55c9f849a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42102/0 (socket says 172.21.15.37:42102) 2024-06-10T00:58:13.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.533+0000 7f9250e4c700 1 -- 172.21.15.37:0/1156504563 learned_addr learned my addr 172.21.15.37:0/1156504563 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:13.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.533+0000 7f925164d700 1 -- 172.21.15.37:0/1156504563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9f848df60 msgr2=0x55c9f8496de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:13.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.533+0000 7f925164d700 1 --2- 172.21.15.37:0/1156504563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9f848df60 0x55c9f8496de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.533+0000 7f925164d700 1 -- 172.21.15.37:0/1156504563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f848d640 msgr2=0x55c9f849a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:13.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.533+0000 7f925164d700 1 --2- 172.21.15.37:0/1156504563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f848d640 0x55c9f849a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.533+0000 7f925164d700 1 -- 172.21.15.37:0/1156504563 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9f8497320 con 0x55c9f8487f20 2024-06-10T00:58:13.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.533+0000 7f9250e4c700 1 --2- 172.21.15.37:0/1156504563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f848d640 0x55c9f849a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:58:13.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.534+0000 7f925164d700 1 --2- 172.21.15.37:0/1156504563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9f8487f20 0x55c9f8499e10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f92480a4de0 tx=0x7f924800b540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79651e7455d22ff1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:13.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.534+0000 7f9234ff9700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T00:58:13.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.534+0000 7f9234ff9700 1 -- 172.21.15.37:0/1156504563 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924809ce30 con 0x55c9f8487f20 2024-06-10T00:58:13.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.534+0000 7f925e134b80 1 -- 172.21.15.37:0/1156504563 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9244016cb0 con 0x55c9f8487f20 2024-06-10T00:58:13.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.534+0000 7f9234ff9700 1 -- 172.21.15.37:0/1156504563 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f9248022470 con 0x55c9f8487f20 2024-06-10T00:58:13.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.534+0000 7f9234ff9700 1 -- 172.21.15.37:0/1156504563 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f92480a0ae0 con 0x55c9f8487f20 2024-06-10T00:58:13.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.534+0000 7f925e134b80 10 client.25375 fetch_fsmap learned FSMap version 27 2024-06-10T00:58:13.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.534+0000 7f925e134b80 1 -- 172.21.15.37:0/1156504563 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c9f850cab0 con 0x55c9f8487f20 2024-06-10T00:58:13.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.534+0000 7f9234ff9700 1 -- 172.21.15.37:0/1156504563 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9244016cb0 con 0x55c9f8487f20 2024-06-10T00:58:13.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.534+0000 7f9234ff9700 1 -- 172.21.15.37:0/1156504563 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f92480b0bc0 con 0x55c9f8487f20 2024-06-10T00:58:13.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.534+0000 7f925e134b80 10 client.25375 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:58:13.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.534+0000 7f925e134b80 10 client.25375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T00:58:13.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.534+0000 7f925e134b80 20 client.25375 populate_metadata read hostname 'smithi037' 2024-06-10T00:58:13.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.535+0000 7f925e134b80 1 --2- 172.21.15.37:0/1156504563 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c9f84a1750 0x55c9f850ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:13.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.535+0000 7f925e134b80 4 client.25375 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T00:58:13.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.535+0000 7f925e134b80 1 -- 172.21.15.37:0/1156504563 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c9f83e1290 con 0x55c9f84a1750 2024-06-10T00:58:13.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.535+0000 7f9250e4c700 1 --2- 172.21.15.37:0/1156504563 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c9f84a1750 0x55c9f850ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.535+0000 7f9250e4c700 1 --2- 172.21.15.37:0/1156504563 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c9f84a1750 0x55c9f850ef90 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc2f7892d06f756d server_cookie=12fe850abba76b51 in_seq=0 out_seq=0 2024-06-10T00:58:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.535+0000 7f9234ff9700 10 client.25375 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.535+0000 7f9234ff9700 1 -- 172.21.15.37:0/1156504563 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c9f83e1290 con 0x55c9f84a1750 2024-06-10T00:58:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.535+0000 7f9234ff9700 10 client.25375 handle_command_reply: tid=0 2024-06-10T00:58:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.535+0000 7f925e134b80 1 client.25375 shutdown 2024-06-10T00:58:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.536+0000 7f925e134b80 1 -- 172.21.15.37:0/1156504563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9f8487f20 msgr2=0x55c9f8499e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.536+0000 7f925e134b80 1 --2- 172.21.15.37:0/1156504563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9f8487f20 0x55c9f8499e10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f92480a4de0 tx=0x7f924800b540 comp rx=0 tx=0).stop 2024-06-10T00:58:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.536+0000 7f925e134b80 1 -- 172.21.15.37:0/1156504563 shutdown_connections 2024-06-10T00:58:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.536+0000 7f925e134b80 1 --2- 172.21.15.37:0/1156504563 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c9f84a1750 0x55c9f850ef90 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.536+0000 7f925e134b80 1 --2- 172.21.15.37:0/1156504563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9f848df60 0x55c9f8496de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.536+0000 7f925e134b80 1 --2- 172.21.15.37:0/1156504563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9f8487f20 0x55c9f8499e10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.536+0000 7f9234ff9700 0 client.25375 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.536+0000 7f925e134b80 1 --2- 172.21.15.37:0/1156504563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f848d640 0x55c9f849a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.536+0000 7f925e134b80 1 -- 172.21.15.37:0/1156504563 >> 172.21.15.37:0/1156504563 conn(0x55c9f83b2cc0 msgr2=0x55c9f848c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:13.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.536+0000 7f925e134b80 1 -- 172.21.15.37:0/1156504563 shutdown_connections 2024-06-10T00:58:13.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.536+0000 7f925e134b80 1 -- 172.21.15.37:0/1156504563 wait complete. 2024-06-10T00:58:13.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.536+0000 7f925e134b80 20 client.25375 trim_cache size 0 max 16384 2024-06-10T00:58:13.541 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T00:58:13.541 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-06-10T00:58:13.541 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-06-10T00:58:13.541 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T00:58:13.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.536+0000 7f92538b1700 1 -- 172.21.15.37:0/1754054855 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f92380a2ca0 msgr2=0x7f92380a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:13.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.536+0000 7f92538b1700 1 --2- 172.21.15.37:0/1754054855 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f92380a2ca0 0x7f92380a5160 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f923c005950 tx=0x7f923c0058e0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.536+0000 7f92538b1700 1 -- 172.21.15.37:0/1754054855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f924c06a170 msgr2=0x7f924c19e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:13.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.536+0000 7f92538b1700 1 --2- 172.21.15.37:0/1754054855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f924c06a170 0x7f924c19e7c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f92480038b0 tx=0x7f9248009f50 comp rx=0 tx=0).stop 2024-06-10T00:58:13.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.537+0000 7f92538b1700 1 -- 172.21.15.37:0/1754054855 shutdown_connections 2024-06-10T00:58:13.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.537+0000 7f92538b1700 1 --2- 172.21.15.37:0/1754054855 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f92380a2ca0 0x7f92380a5160 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.537+0000 7f92538b1700 1 --2- 172.21.15.37:0/1754054855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f924c108500 0x7f924c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.537+0000 7f92538b1700 1 --2- 172.21.15.37:0/1754054855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f924c06a170 0x7f924c19e7c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.537+0000 7f92538b1700 1 --2- 172.21.15.37:0/1754054855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f924c06ab30 0x7f924c19ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:13.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.537+0000 7f92538b1700 1 -- 172.21.15.37:0/1754054855 >> 172.21.15.37:0/1754054855 conn(0x7f924c078340 msgr2=0x7f924c0ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:13.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.537+0000 7f92538b1700 1 -- 172.21.15.37:0/1754054855 shutdown_connections 2024-06-10T00:58:13.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:13.537+0000 7f92538b1700 1 -- 172.21.15.37:0/1754054855 wait complete. 2024-06-10T00:58:14.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:14 smithi037 ceph-mon[32192]: pgmap v476: 129 pgs: 129 active+clean; 686 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-06-10T00:58:14.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:14 smithi167 ceph-mon[36917]: pgmap v476: 129 pgs: 129 active+clean; 686 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-06-10T00:58:14.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:14 smithi094 ceph-mon[36708]: pgmap v476: 129 pgs: 129 active+clean; 686 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-06-10T00:58:14.735 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T00:58:14.735 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:58:15.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.285+0000 7f9acc7f8700 1 -- 172.21.15.37:0/1341129483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac41085d0 msgr2=0x7f9ac41089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:15.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.285+0000 7f9acc7f8700 1 --2- 172.21.15.37:0/1341129483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac41085d0 0x7f9ac41089b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40099c0 tx=0x7f9ab4009cd0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.286+0000 7f9acc7f8700 1 -- 172.21.15.37:0/1341129483 shutdown_connections 2024-06-10T00:58:15.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.286+0000 7f9acc7f8700 1 --2- 172.21.15.37:0/1341129483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ac4102ef0 0x7f9ac410f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.286+0000 7f9acc7f8700 1 --2- 172.21.15.37:0/1341129483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac41085d0 0x7f9ac41089b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.286+0000 7f9acc7f8700 1 --2- 172.21.15.37:0/1341129483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ac4102530 0x7f9ac41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.286+0000 7f9acc7f8700 1 -- 172.21.15.37:0/1341129483 >> 172.21.15.37:0/1341129483 conn(0x7f9ac4079640 msgr2=0x7f9ac4079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:15.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.286+0000 7f9acc7f8700 1 -- 172.21.15.37:0/1341129483 shutdown_connections 2024-06-10T00:58:15.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.286+0000 7f9acc7f8700 1 -- 172.21.15.37:0/1341129483 wait complete. 2024-06-10T00:58:15.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.286+0000 7f9acc7f8700 1 Processor -- start 2024-06-10T00:58:15.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.287+0000 7f9acc7f8700 1 -- start start 2024-06-10T00:58:15.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.287+0000 7f9acc7f8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ac4102530 0x7f9ac419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.287+0000 7f9acc7f8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac4102ef0 0x7f9ac419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.287+0000 7f9acc7f8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ac41085d0 0x7f9ac4198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.287+0000 7f9acc7f8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac4112820 con 0x7f9ac4102ef0 2024-06-10T00:58:15.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.287+0000 7f9acc7f8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac41126a0 con 0x7f9ac4102530 2024-06-10T00:58:15.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.287+0000 7f9acc7f8700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac41129a0 con 0x7f9ac41085d0 2024-06-10T00:58:15.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.287+0000 7f9ac9d93700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac4102ef0 0x7f9ac419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.287+0000 7f9ac9d93700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac4102ef0 0x7f9ac419eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42132/0 (socket says 172.21.15.37:42132) 2024-06-10T00:58:15.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.287+0000 7f9ac9d93700 1 -- 172.21.15.37:0/3379798693 learned_addr learned my addr 172.21.15.37:0/3379798693 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:15.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.287+0000 7f9acad95700 1 --2- 172.21.15.37:0/3379798693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ac41085d0 0x7f9ac4198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.287+0000 7f9aca594700 1 --2- 172.21.15.37:0/3379798693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ac4102530 0x7f9ac419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.287+0000 7f9ac9d93700 1 -- 172.21.15.37:0/3379798693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ac41085d0 msgr2=0x7f9ac4198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:15.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.287+0000 7f9ac9d93700 1 --2- 172.21.15.37:0/3379798693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ac41085d0 0x7f9ac4198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.287+0000 7f9ac9d93700 1 -- 172.21.15.37:0/3379798693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ac4102530 msgr2=0x7f9ac419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:15.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.287+0000 7f9ac9d93700 1 --2- 172.21.15.37:0/3379798693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ac4102530 0x7f9ac419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.287+0000 7f9ac9d93700 1 -- 172.21.15.37:0/3379798693 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac4199180 con 0x7f9ac4102ef0 2024-06-10T00:58:15.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.287+0000 7f9aca594700 1 --2- 172.21.15.37:0/3379798693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ac4102530 0x7f9ac419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:58:15.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.288+0000 7f9ac9d93700 1 --2- 172.21.15.37:0/3379798693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac4102ef0 0x7f9ac419eda0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000d950 tx=0x7f9ac000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:15.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.288+0000 7f9abb7fe700 1 -- 172.21.15.37:0/3379798693 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0016070 con 0x7f9ac4102ef0 2024-06-10T00:58:15.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.288+0000 7f9acc7f8700 1 -- 172.21.15.37:0/3379798693 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ac4199470 con 0x7f9ac4102ef0 2024-06-10T00:58:15.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.288+0000 7f9acc7f8700 1 -- 172.21.15.37:0/3379798693 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ac40746c0 con 0x7f9ac4102ef0 2024-06-10T00:58:15.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.288+0000 7f9abb7fe700 1 -- 172.21.15.37:0/3379798693 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ac0009e20 con 0x7f9ac4102ef0 2024-06-10T00:58:15.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.288+0000 7f9abb7fe700 1 -- 172.21.15.37:0/3379798693 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac000f620 con 0x7f9ac4102ef0 2024-06-10T00:58:15.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.290+0000 7f9abb7fe700 1 -- 172.21.15.37:0/3379798693 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9ac0019070 con 0x7f9ac4102ef0 2024-06-10T00:58:15.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.291+0000 7f9abb7fe700 1 --2- 172.21.15.37:0/3379798693 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9ab00a2ca0 0x7f9ab00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.292+0000 7f9aca594700 1 --2- 172.21.15.37:0/3379798693 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9ab00a2ca0 0x7f9ab00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.292+0000 7f9abb7fe700 1 -- 172.21.15.37:0/3379798693 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f9ac00cf270 con 0x7f9ac4102ef0 2024-06-10T00:58:15.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.292+0000 7f9ad707bb80 1 Processor -- start 2024-06-10T00:58:15.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.292+0000 7f9ad707bb80 1 -- start start 2024-06-10T00:58:15.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.292+0000 7f9ad707bb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ee06bc500 0x561ee06bc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.292+0000 7f9ad707bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee06bce20 0x561ee06b6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.292+0000 7f9ad707bb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ee06b6c50 0x561ee06b7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.292+0000 7f9ad707bb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561ee05a9060 con 0x561ee06bc500 2024-06-10T00:58:15.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.292+0000 7f9ad707bb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561ee0607950 con 0x561ee06b6c50 2024-06-10T00:58:15.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.292+0000 7f9ad707bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561ee056f560 con 0x561ee06bce20 2024-06-10T00:58:15.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.293+0000 7f9aca594700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ee06bc500 0x561ee06bc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.293+0000 7f9aca594700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ee06bc500 0x561ee06bc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42144/0 (socket says 172.21.15.37:42144) 2024-06-10T00:58:15.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.293+0000 7f9aca594700 1 -- 172.21.15.37:0/3487525806 learned_addr learned my addr 172.21.15.37:0/3487525806 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:15.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.293+0000 7f9acad95700 1 --2- 172.21.15.37:0/3487525806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ee06b6c50 0x561ee06b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.293+0000 7f9acad95700 1 -- 172.21.15.37:0/3487525806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee06bce20 msgr2=0x561ee06b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:15.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.293+0000 7f9ac9d93700 1 --2- 172.21.15.37:0/3487525806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee06bce20 0x561ee06b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.293+0000 7f9acad95700 1 --2- 172.21.15.37:0/3487525806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee06bce20 0x561ee06b6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.293+0000 7f9acad95700 1 -- 172.21.15.37:0/3487525806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ee06bc500 msgr2=0x561ee06bc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:15.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.293+0000 7f9acad95700 1 --2- 172.21.15.37:0/3487525806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ee06bc500 0x561ee06bc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.293+0000 7f9acad95700 1 -- 172.21.15.37:0/3487525806 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ee05569d0 con 0x561ee06b6c50 2024-06-10T00:58:15.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.293+0000 7f9ac9d93700 1 --2- 172.21.15.37:0/3487525806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee06bce20 0x561ee06b6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:58:15.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.293+0000 7f9acad95700 1 --2- 172.21.15.37:0/3487525806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ee06b6c50 0x561ee06b7070 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9abc003f20 tx=0x7f9abc009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f267bd4b27dde376 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:15.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.293+0000 7f9ab8ff9700 1 -- 172.21.15.37:0/3487525806 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc004bb0 con 0x561ee06b6c50 2024-06-10T00:58:15.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.293+0000 7f9ab8ff9700 1 -- 172.21.15.37:0/3487525806 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9abc004d50 con 0x561ee06b6c50 2024-06-10T00:58:15.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.293+0000 7f9ab8ff9700 1 -- 172.21.15.37:0/3487525806 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc007db0 con 0x561ee06b6c50 2024-06-10T00:58:15.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.294+0000 7f9aca594700 1 --2- 172.21.15.37:0/3487525806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ee06bc500 0x561ee06bc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:58:15.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.294+0000 7f9ad707bb80 1 -- 172.21.15.37:0/3487525806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ee06b6c50 msgr2=0x561ee06b7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:15.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.294+0000 7f9ad707bb80 1 --2- 172.21.15.37:0/3487525806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ee06b6c50 0x561ee06b7070 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9abc003f20 tx=0x7f9abc009ba0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.294+0000 7f9aca594700 1 --2- 172.21.15.37:0/3379798693 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9ab00a2ca0 0x7f9ab00a5160 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4000c00 tx=0x7f9ab4005940 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:15.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.294+0000 7f9ad707bb80 1 -- 172.21.15.37:0/3487525806 shutdown_connections 2024-06-10T00:58:15.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.294+0000 7f9ad707bb80 1 --2- 172.21.15.37:0/3487525806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ee06b6c50 0x561ee06b7070 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.295+0000 7f9ad707bb80 1 --2- 172.21.15.37:0/3487525806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ee06bc500 0x561ee06bc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.295+0000 7f9ad707bb80 1 --2- 172.21.15.37:0/3487525806 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee06bce20 0x561ee06b6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.295+0000 7f9ad707bb80 1 -- 172.21.15.37:0/3487525806 >> 172.21.15.37:0/3487525806 conn(0x561ee05e3180 msgr2=0x561ee06b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:15.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.295+0000 7f9ad707bb80 1 -- 172.21.15.37:0/3487525806 shutdown_connections 2024-06-10T00:58:15.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.295+0000 7f9ad707bb80 1 -- 172.21.15.37:0/3487525806 wait complete. 2024-06-10T00:58:15.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.295+0000 7f9ad707bb80 1 -- start start 2024-06-10T00:58:15.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.295+0000 7f9ad707bb80 1 Processor -- start 2024-06-10T00:58:15.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.296+0000 7f9ad707bb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ee06b6c50 0x561ee06c2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.296+0000 7f9ad707bb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ee06bc500 0x561ee06bfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.296+0000 7f9ad707bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee06bce20 0x561ee06c03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.296+0000 7f9ad707bb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561ee05a80c0 con 0x561ee06bc500 2024-06-10T00:58:15.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.296+0000 7f9ad707bb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561ee0603910 con 0x561ee06b6c50 2024-06-10T00:58:15.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.296+0000 7f9ad707bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561ee06c3800 con 0x561ee06bce20 2024-06-10T00:58:15.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.296+0000 7f9ad707bb80 10 client.0 _ll_register_callbacks cb 0x561ee059d6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:58:15.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.296+0000 7f9ac9d93700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ee06bc500 0x561ee06bfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.296+0000 7f9ac9d93700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ee06bc500 0x561ee06bfeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42148/0 (socket says 172.21.15.37:42148) 2024-06-10T00:58:15.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.296+0000 7f9ac9d93700 1 -- 172.21.15.37:0/4234930350 learned_addr learned my addr 172.21.15.37:0/4234930350 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:15.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.296+0000 7f9acad95700 1 --2- 172.21.15.37:0/4234930350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee06bce20 0x561ee06c03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.296+0000 7f9ac9d93700 1 -- 172.21.15.37:0/4234930350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee06bce20 msgr2=0x561ee06c03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:15.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.296+0000 7f9ac9d93700 1 --2- 172.21.15.37:0/4234930350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee06bce20 0x561ee06c03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.296+0000 7f9ac9d93700 1 -- 172.21.15.37:0/4234930350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ee06b6c50 msgr2=0x561ee06c2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:15.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.296+0000 7f9aca594700 1 --2- 172.21.15.37:0/4234930350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ee06b6c50 0x561ee06c2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.296+0000 7f9ac9d93700 1 --2- 172.21.15.37:0/4234930350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ee06b6c50 0x561ee06c2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.296+0000 7f9ac9d93700 1 -- 172.21.15.37:0/4234930350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ee05cd850 con 0x561ee06bc500 2024-06-10T00:58:15.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.297+0000 7f9ac9d93700 1 --2- 172.21.15.37:0/4234930350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ee06bc500 0x561ee06bfeb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0097ae0 tx=0x7f9ac009b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49041b023505335 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:15.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.297+0000 7f9aadffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:58:15.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.297+0000 7f9aadffb700 1 -- 172.21.15.37:0/4234930350 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0020040 con 0x561ee06bc500 2024-06-10T00:58:15.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.297+0000 7f9aadffb700 1 -- 172.21.15.37:0/4234930350 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f9ac009b8e0 con 0x561ee06bc500 2024-06-10T00:58:15.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.297+0000 7f9ad707bb80 1 -- 172.21.15.37:0/4234930350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9abc004d50 con 0x561ee06bc500 2024-06-10T00:58:15.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.297+0000 7f9aadffb700 1 -- 172.21.15.37:0/4234930350 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9ac009bae0 con 0x561ee06bc500 2024-06-10T00:58:15.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.297+0000 7f9ad707bb80 10 client.15783 fetch_fsmap learned FSMap version 27 2024-06-10T00:58:15.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.297+0000 7f9ad707bb80 1 -- 172.21.15.37:0/4234930350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561ee06cdfb0 con 0x561ee06bc500 2024-06-10T00:58:15.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.297+0000 7f9aadffb700 1 -- 172.21.15.37:0/4234930350 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9abc004d50 con 0x561ee06bc500 2024-06-10T00:58:15.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.297+0000 7f9aadffb700 1 -- 172.21.15.37:0/4234930350 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9ac00973d0 con 0x561ee06bc500 2024-06-10T00:58:15.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.298+0000 7f9ad707bb80 10 client.15783 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:58:15.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.298+0000 7f9ad707bb80 10 client.15783 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T00:58:15.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.298+0000 7f9ad707bb80 20 client.15783 populate_metadata read hostname 'smithi037' 2024-06-10T00:58:15.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.298+0000 7f9ad707bb80 1 --2- 172.21.15.37:0/4234930350 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561ee06d05f0 0x561ee06d29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.298+0000 7f9ad707bb80 4 client.15783 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:58:15.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.298+0000 7f9ad707bb80 1 -- 172.21.15.37:0/4234930350 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ee0614e50 con 0x561ee06d05f0 2024-06-10T00:58:15.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.298+0000 7f9aca594700 1 --2- 172.21.15.37:0/4234930350 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561ee06d05f0 0x561ee06d29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.298+0000 7f9aca594700 1 --2- 172.21.15.37:0/4234930350 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561ee06d05f0 0x561ee06d29e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=970e18dce0cead5f server_cookie=25acdfae5a6f5c3d in_seq=0 out_seq=0 2024-06-10T00:58:15.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.298+0000 7f9aadffb700 10 client.15783 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:15.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.299+0000 7f9aadffb700 1 -- 172.21.15.37:0/4234930350 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561ee0614e50 con 0x561ee06d05f0 2024-06-10T00:58:15.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.299+0000 7f9aadffb700 10 client.15783 handle_command_reply: tid=0 2024-06-10T00:58:15.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.299+0000 7f9ad707bb80 1 client.15783 shutdown 2024-06-10T00:58:15.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.300+0000 7f9ad707bb80 1 -- 172.21.15.37:0/4234930350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ee06bc500 msgr2=0x561ee06bfeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:15.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.300+0000 7f9ad707bb80 1 --2- 172.21.15.37:0/4234930350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ee06bc500 0x561ee06bfeb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0097ae0 tx=0x7f9ac009b700 comp rx=0 tx=0).stop 2024-06-10T00:58:15.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.300+0000 7f9ad707bb80 1 -- 172.21.15.37:0/4234930350 shutdown_connections 2024-06-10T00:58:15.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.300+0000 7f9ad707bb80 1 --2- 172.21.15.37:0/4234930350 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561ee06d05f0 0x561ee06d29e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.300+0000 7f9ad707bb80 1 --2- 172.21.15.37:0/4234930350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee06bce20 0x561ee06c03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.300+0000 7f9ad707bb80 1 --2- 172.21.15.37:0/4234930350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ee06b6c50 0x561ee06c2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.300+0000 7f9ad707bb80 1 --2- 172.21.15.37:0/4234930350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ee06bc500 0x561ee06bfeb0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.300+0000 7f9ad707bb80 1 -- 172.21.15.37:0/4234930350 >> 172.21.15.37:0/4234930350 conn(0x561ee05e3180 msgr2=0x561ee06b0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:15.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.300+0000 7f9aadffb700 0 client.15783 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:15.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.300+0000 7f9ad707bb80 1 -- 172.21.15.37:0/4234930350 shutdown_connections 2024-06-10T00:58:15.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.300+0000 7f9ad707bb80 1 -- 172.21.15.37:0/4234930350 wait complete. 2024-06-10T00:58:15.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.300+0000 7f9ad707bb80 20 client.15783 trim_cache size 0 max 16384 2024-06-10T00:58:15.322 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:58:15.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.325+0000 7f9ad707bb80 1 Processor -- start 2024-06-10T00:58:15.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.325+0000 7f9ad707bb80 1 -- start start 2024-06-10T00:58:15.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.325+0000 7f9ad707bb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ee06b6c50 0x561ee06bb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.325+0000 7f9ad707bb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ee06bc500 0x561ee06c26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.325+0000 7f9ad707bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee06bce20 0x561ee06b1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.325+0000 7f9ad707bb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa8011500 con 0x561ee06b6c50 2024-06-10T00:58:15.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.325+0000 7f9ad707bb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa800ed90 con 0x561ee06bc500 2024-06-10T00:58:15.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.325+0000 7f9ad707bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa800cbe0 con 0x561ee06bce20 2024-06-10T00:58:15.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.325+0000 7f9aca594700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ee06b6c50 0x561ee06bb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.325+0000 7f9aca594700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ee06b6c50 0x561ee06bb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42160/0 (socket says 172.21.15.37:42160) 2024-06-10T00:58:15.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.325+0000 7f9aca594700 1 -- 172.21.15.37:0/1064270740 learned_addr learned my addr 172.21.15.37:0/1064270740 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:15.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.325+0000 7f9acad95700 1 --2- 172.21.15.37:0/1064270740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee06bce20 0x561ee06b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.325+0000 7f9ac9d93700 1 --2- 172.21.15.37:0/1064270740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ee06bc500 0x561ee06c26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.326+0000 7f9aca594700 1 -- 172.21.15.37:0/1064270740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee06bce20 msgr2=0x561ee06b1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:15.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.326+0000 7f9aca594700 1 --2- 172.21.15.37:0/1064270740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee06bce20 0x561ee06b1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.326+0000 7f9aca594700 1 -- 172.21.15.37:0/1064270740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ee06bc500 msgr2=0x561ee06c26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:15.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.326+0000 7f9aca594700 1 --2- 172.21.15.37:0/1064270740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ee06bc500 0x561ee06c26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.326+0000 7f9aca594700 1 -- 172.21.15.37:0/1064270740 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ee06a94a0 con 0x561ee06b6c50 2024-06-10T00:58:15.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.326+0000 7f9ac9d93700 1 --2- 172.21.15.37:0/1064270740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ee06bc500 0x561ee06c26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:58:15.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.326+0000 7f9aca594700 1 --2- 172.21.15.37:0/1064270740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ee06b6c50 0x561ee06bb790 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400b580 tx=0x7f9ab4009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=313ad6f857040321 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:15.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.326+0000 7f9aaffff700 1 -- 172.21.15.37:0/1064270740 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4038070 con 0x561ee06b6c50 2024-06-10T00:58:15.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.326+0000 7f9aaffff700 1 -- 172.21.15.37:0/1064270740 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ab4033400 con 0x561ee06b6c50 2024-06-10T00:58:15.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.326+0000 7f9aaffff700 1 -- 172.21.15.37:0/1064270740 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab400bea0 con 0x561ee06b6c50 2024-06-10T00:58:15.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.327+0000 7f9ad707bb80 1 -- 172.21.15.37:0/1064270740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ee06b6c50 msgr2=0x561ee06bb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:15.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.327+0000 7f9ad707bb80 1 --2- 172.21.15.37:0/1064270740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ee06b6c50 0x561ee06bb790 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400b580 tx=0x7f9ab4009cd0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.327+0000 7f9ad707bb80 1 -- 172.21.15.37:0/1064270740 shutdown_connections 2024-06-10T00:58:15.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.327+0000 7f9ad707bb80 1 --2- 172.21.15.37:0/1064270740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee06bce20 0x561ee06b1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.327+0000 7f9ad707bb80 1 --2- 172.21.15.37:0/1064270740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ee06b6c50 0x561ee06bb790 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.327+0000 7f9ad707bb80 1 --2- 172.21.15.37:0/1064270740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ee06bc500 0x561ee06c26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.327+0000 7f9ad707bb80 1 -- 172.21.15.37:0/1064270740 >> 172.21.15.37:0/1064270740 conn(0x561ee05e3180 msgr2=0x561ee06bd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:15.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.327+0000 7f9ad707bb80 1 -- 172.21.15.37:0/1064270740 shutdown_connections 2024-06-10T00:58:15.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.327+0000 7f9ad707bb80 1 -- 172.21.15.37:0/1064270740 wait complete. 2024-06-10T00:58:15.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.327+0000 7f9ad707bb80 1 -- start start 2024-06-10T00:58:15.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.328+0000 7f9ad707bb80 1 Processor -- start 2024-06-10T00:58:15.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.328+0000 7f9ad707bb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ee06b6c50 0x561ee06c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.328+0000 7f9ad707bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee06bc500 0x561ee06c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.328+0000 7f9ad707bb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ee06bce20 0x561ee06c6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.328+0000 7f9ad707bb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa800d250 con 0x561ee06bce20 2024-06-10T00:58:15.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.328+0000 7f9ad707bb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa800e6c0 con 0x561ee06b6c50 2024-06-10T00:58:15.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.328+0000 7f9ad707bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa800d920 con 0x561ee06bc500 2024-06-10T00:58:15.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.328+0000 7f9acad95700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ee06bce20 0x561ee06c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.328+0000 7f9acad95700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ee06bce20 0x561ee06c6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42170/0 (socket says 172.21.15.37:42170) 2024-06-10T00:58:15.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.328+0000 7f9acad95700 1 -- 172.21.15.37:0/190670549 learned_addr learned my addr 172.21.15.37:0/190670549 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:15.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.328+0000 7f9ad707bb80 10 client.0 _ll_register_callbacks cb 0x561ee059d6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:58:15.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.328+0000 7f9aca594700 1 --2- 172.21.15.37:0/190670549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ee06b6c50 0x561ee06c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.328+0000 7f9ac9d93700 1 --2- 172.21.15.37:0/190670549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee06bc500 0x561ee06c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.329+0000 7f9acad95700 1 -- 172.21.15.37:0/190670549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee06bc500 msgr2=0x561ee06c5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:15.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.329+0000 7f9acad95700 1 --2- 172.21.15.37:0/190670549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee06bc500 0x561ee06c5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.329+0000 7f9acad95700 1 -- 172.21.15.37:0/190670549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ee06b6c50 msgr2=0x561ee06c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:15.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.329+0000 7f9acad95700 1 --2- 172.21.15.37:0/190670549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ee06b6c50 0x561ee06c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.329+0000 7f9acad95700 1 -- 172.21.15.37:0/190670549 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ee06c6570 con 0x561ee06bce20 2024-06-10T00:58:15.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.329+0000 7f9acad95700 1 --2- 172.21.15.37:0/190670549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ee06bce20 0x561ee06c6030 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9abc003b20 tx=0x7f9abc007730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb77e160691c03b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:15.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.329+0000 7f9aadffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:58:15.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.329+0000 7f9aadffb700 1 -- 172.21.15.37:0/190670549 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc005320 con 0x561ee06bce20 2024-06-10T00:58:15.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.329+0000 7f9ad707bb80 1 -- 172.21.15.37:0/190670549 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ab4033400 con 0x561ee06bce20 2024-06-10T00:58:15.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.329+0000 7f9aadffb700 1 -- 172.21.15.37:0/190670549 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f9abc0078a0 con 0x561ee06bce20 2024-06-10T00:58:15.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.329+0000 7f9aadffb700 1 -- 172.21.15.37:0/190670549 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9abc02f050 con 0x561ee06bce20 2024-06-10T00:58:15.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.329+0000 7f9ad707bb80 10 client.15789 fetch_fsmap learned FSMap version 27 2024-06-10T00:58:15.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.329+0000 7f9ad707bb80 1 -- 172.21.15.37:0/190670549 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561ee073c000 con 0x561ee06bce20 2024-06-10T00:58:15.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.330+0000 7f9aadffb700 1 -- 172.21.15.37:0/190670549 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ab4033400 con 0x561ee06bce20 2024-06-10T00:58:15.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.330+0000 7f9aadffb700 1 -- 172.21.15.37:0/190670549 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9abc015480 con 0x561ee06bce20 2024-06-10T00:58:15.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.330+0000 7f9ad707bb80 10 client.15789 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:58:15.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.330+0000 7f9ad707bb80 10 client.15789 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T00:58:15.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.330+0000 7f9ad707bb80 20 client.15789 populate_metadata read hostname 'smithi037' 2024-06-10T00:58:15.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.330+0000 7f9ad707bb80 1 --2- 172.21.15.37:0/190670549 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561ee073c4c0 0x561ee073e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.330+0000 7f9ad707bb80 4 client.15789 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f993ef51", "release": true, "await": true}] 2024-06-10T00:58:15.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.330+0000 7f9ad707bb80 1 -- 172.21.15.37:0/190670549 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f993ef51", "release": true, "await": true}) -- 0x561ee060f3d0 con 0x561ee073c4c0 2024-06-10T00:58:15.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.330+0000 7f9aca594700 1 --2- 172.21.15.37:0/190670549 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561ee073c4c0 0x561ee073e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.331+0000 7f9aca594700 1 --2- 172.21.15.37:0/190670549 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561ee073c4c0 0x561ee073e8e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fa8ebd6b784aa48 server_cookie=7863e71eec276fb2 in_seq=0 out_seq=0 2024-06-10T00:58:15.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.331+0000 7f9aadffb700 10 client.15789 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.333+0000 7f585811d640 1 -- 172.21.15.37:0/74481604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5850101af0 msgr2=0x7f5850101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.333+0000 7f585811d640 1 --2- 172.21.15.37:0/74481604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5850101af0 0x7f5850101f70 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5840009900 tx=0x7f584002a410 comp rx=0 tx=0).stop 2024-06-10T00:58:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.333+0000 7f585811d640 1 -- 172.21.15.37:0/74481604 shutdown_connections 2024-06-10T00:58:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.333+0000 7f585811d640 1 --2- 172.21.15.37:0/74481604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58501024b0 0x7f5850108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.333+0000 7f585811d640 1 --2- 172.21.15.37:0/74481604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5850101af0 0x7f5850101f70 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.333+0000 7f585811d640 1 --2- 172.21.15.37:0/74481604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5850100900 0x7f5850100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.333+0000 7f585811d640 1 -- 172.21.15.37:0/74481604 >> 172.21.15.37:0/74481604 conn(0x7f58500fbe70 msgr2=0x7f58500fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:15.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.333+0000 7f585811d640 1 -- 172.21.15.37:0/74481604 shutdown_connections 2024-06-10T00:58:15.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.334+0000 7f585811d640 1 -- 172.21.15.37:0/74481604 wait complete. 2024-06-10T00:58:15.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.334+0000 7f9aadffb700 1 -- 172.21.15.37:0/190670549 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x561ee060f3d0 con 0x561ee073c4c0 2024-06-10T00:58:15.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.334+0000 7f9aadffb700 10 client.15789 handle_command_reply: tid=0 2024-06-10T00:58:15.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.334+0000 7f9ad707bb80 1 client.15789 shutdown 2024-06-10T00:58:15.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.334+0000 7f9ad707bb80 1 -- 172.21.15.37:0/190670549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ee06bce20 msgr2=0x561ee06c6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:15.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.334+0000 7f9ad707bb80 1 --2- 172.21.15.37:0/190670549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ee06bce20 0x561ee06c6030 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9abc003b20 tx=0x7f9abc007730 comp rx=0 tx=0).stop 2024-06-10T00:58:15.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.334+0000 7f9ad707bb80 1 -- 172.21.15.37:0/190670549 shutdown_connections 2024-06-10T00:58:15.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.334+0000 7f9ad707bb80 1 --2- 172.21.15.37:0/190670549 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561ee073c4c0 0x561ee073e8e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.334+0000 7f9ad707bb80 1 --2- 172.21.15.37:0/190670549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ee06bce20 0x561ee06c6030 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.334+0000 7f9ad707bb80 1 --2- 172.21.15.37:0/190670549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ee06b6c50 0x561ee06c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.334+0000 7f9ad707bb80 1 --2- 172.21.15.37:0/190670549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ee06bc500 0x561ee06c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.334+0000 7f9ad707bb80 1 -- 172.21.15.37:0/190670549 >> 172.21.15.37:0/190670549 conn(0x561ee05e3180 msgr2=0x561ee06bb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:15.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.334+0000 7f9aadffb700 0 client.15789 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:15.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.334+0000 7f9ad707bb80 1 -- 172.21.15.37:0/190670549 shutdown_connections 2024-06-10T00:58:15.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.334+0000 7f9ad707bb80 1 -- 172.21.15.37:0/190670549 wait complete. 2024-06-10T00:58:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.335+0000 7f585811d640 1 Processor -- start 2024-06-10T00:58:15.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.335+0000 7f9ad707bb80 20 client.15789 trim_cache size 0 max 16384 2024-06-10T00:58:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.335+0000 7f585811d640 1 -- start start 2024-06-10T00:58:15.340 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T00:58:15.340 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-06-10T00:58:15.340 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24472, 2024-06-10T00:58:15.340 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 4, 2024-06-10T00:58:15.341 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-06-10T00:58:15.341 INFO:teuthology.orchestra.run.smithi037.stdout: "f993ef51": { 2024-06-10T00:58:15.341 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 4, 2024-06-10T00:58:15.341 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-06-10T00:58:15.341 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T00:58:15.341 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-06-10T00:58:15.341 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T00:58:15.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T00:58:15.341 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-06-10T00:58:15.341 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 150.0, 2024-06-10T00:58:15.341 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-06-10T00:58:15.341 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-06-10T00:58:15.342 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-06-10T00:58:15.342 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T00:58:15.342 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-06-10T00:58:15.342 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T00:58:15.342 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T00:58:15.342 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T00:58:15.342 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T00:58:15.342 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T00:58:15.342 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T00:58:15.342 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T00:58:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.335+0000 7f585811d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5850100900 0x7f5850196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.335+0000 7f585811d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5850101af0 0x7f5850197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.335+0000 7f9acc7f8700 1 -- 172.21.15.37:0/3379798693 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9ab00a2ca0 msgr2=0x7f9ab00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:15.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.335+0000 7f9acc7f8700 1 --2- 172.21.15.37:0/3379798693 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9ab00a2ca0 0x7f9ab00a5160 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4000c00 tx=0x7f9ab4005940 comp rx=0 tx=0).stop 2024-06-10T00:58:15.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.335+0000 7f9acc7f8700 1 -- 172.21.15.37:0/3379798693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac4102ef0 msgr2=0x7f9ac419eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:15.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.335+0000 7f9acc7f8700 1 --2- 172.21.15.37:0/3379798693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac4102ef0 0x7f9ac419eda0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000d950 tx=0x7f9ac000dc60 comp rx=0 tx=0).stop 2024-06-10T00:58:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.336+0000 7f5855eb9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5850100900 0x7f5850196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.336+0000 7f9acc7f8700 1 -- 172.21.15.37:0/3379798693 shutdown_connections 2024-06-10T00:58:15.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.336+0000 7f9acc7f8700 1 --2- 172.21.15.37:0/3379798693 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9ab00a2ca0 0x7f9ab00a5160 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.336+0000 7f9acc7f8700 1 --2- 172.21.15.37:0/3379798693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ac41085d0 0x7f9ac4198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.336+0000 7f9acc7f8700 1 --2- 172.21.15.37:0/3379798693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ac4102530 0x7f9ac419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.336+0000 7f9acc7f8700 1 --2- 172.21.15.37:0/3379798693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ac4102ef0 0x7f9ac419eda0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.336+0000 7f9acc7f8700 1 -- 172.21.15.37:0/3379798693 >> 172.21.15.37:0/3379798693 conn(0x7f9ac4079640 msgr2=0x7f9ac4100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.336+0000 7f5855eb9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5850100900 0x7f5850196bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:57228/0 (socket says 172.21.15.37:57228) 2024-06-10T00:58:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.336+0000 7f5855eb9640 1 -- 172.21.15.37:0/1786641232 learned_addr learned my addr 172.21.15.37:0/1786641232 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.336+0000 7f585811d640 1 --2- 172.21.15.37:0/1786641232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58501024b0 0x7f585019e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.336+0000 7f585811d640 1 -- 172.21.15.37:0/1786641232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f585010c5d0 con 0x7f5850101af0 2024-06-10T00:58:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.336+0000 7f58556b8640 1 --2- 172.21.15.37:0/1786641232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5850101af0 0x7f5850197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.336+0000 7f9acc7f8700 1 -- 172.21.15.37:0/3379798693 shutdown_connections 2024-06-10T00:58:15.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:15.336+0000 7f9acc7f8700 1 -- 172.21.15.37:0/3379798693 wait complete. 2024-06-10T00:58:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.336+0000 7f585811d640 1 -- 172.21.15.37:0/1786641232 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f585010c450 con 0x7f58501024b0 2024-06-10T00:58:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.336+0000 7f585811d640 1 -- 172.21.15.37:0/1786641232 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f585010c750 con 0x7f5850100900 2024-06-10T00:58:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.336+0000 7f58566ba640 1 --2- 172.21.15.37:0/1786641232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58501024b0 0x7f585019e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.336+0000 7f5855eb9640 1 -- 172.21.15.37:0/1786641232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58501024b0 msgr2=0x7f585019e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.336+0000 7f5855eb9640 1 --2- 172.21.15.37:0/1786641232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58501024b0 0x7f585019e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.336+0000 7f5855eb9640 1 -- 172.21.15.37:0/1786641232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5850101af0 msgr2=0x7f5850197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.337+0000 7f5855eb9640 1 --2- 172.21.15.37:0/1786641232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5850101af0 0x7f5850197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.337+0000 7f5855eb9640 1 -- 172.21.15.37:0/1786641232 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585019e7e0 con 0x7f5850100900 2024-06-10T00:58:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.337+0000 7f58556b8640 1 --2- 172.21.15.37:0/1786641232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5850101af0 0x7f5850197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:58:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.337+0000 7f5855eb9640 1 --2- 172.21.15.37:0/1786641232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5850100900 0x7f5850196bc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f584c009f80 tx=0x7f584c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.337+0000 7f5846ffd640 1 -- 172.21.15.37:0/1786641232 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c00ce50 con 0x7f5850100900 2024-06-10T00:58:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.337+0000 7f585811d640 1 -- 172.21.15.37:0/1786641232 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58500690b0 con 0x7f5850100900 2024-06-10T00:58:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.337+0000 7f585811d640 1 -- 172.21.15.37:0/1786641232 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58500695d0 con 0x7f5850100900 2024-06-10T00:58:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.337+0000 7f5846ffd640 1 -- 172.21.15.37:0/1786641232 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f584c004030 con 0x7f5850100900 2024-06-10T00:58:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.337+0000 7f5846ffd640 1 -- 172.21.15.37:0/1786641232 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c027060 con 0x7f5850100900 2024-06-10T00:58:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.340+0000 7f5846ffd640 1 -- 172.21.15.37:0/1786641232 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f584c0041e0 con 0x7f5850100900 2024-06-10T00:58:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.340+0000 7f585ae1f740 1 Processor -- start 2024-06-10T00:58:15.349 INFO:tasks.workunit.client.0.smithi037.stderr:+ set -e 2024-06-10T00:58:15.350 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir -p testdir 2024-06-10T00:58:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.341+0000 7f585ae1f740 1 -- start start 2024-06-10T00:58:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.341+0000 7f5846ffd640 1 --2- 172.21.15.37:0/1786641232 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f583c0a2dd0 0x7f583c0a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.341+0000 7f585ae1f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629228de010 0x5629228de430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.341+0000 7f585ae1f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629228dea00 0x5629228e3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.341+0000 7f585ae1f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629228e3b70 0x5629228e6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.341+0000 7f585ae1f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5629228414f0 con 0x5629228dea00 2024-06-10T00:58:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.341+0000 7f585ae1f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5629227df870 con 0x5629228e3b70 2024-06-10T00:58:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.341+0000 7f585ae1f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56292283d4b0 con 0x5629228de010 2024-06-10T00:58:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.341+0000 7f58566ba640 1 --2- 172.21.15.37:0/1786641232 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f583c0a2dd0 0x7f583c0a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.341+0000 7f58556b8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629228dea00 0x5629228e3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.341+0000 7f58556b8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629228dea00 0x5629228e3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42192/0 (socket says 172.21.15.37:42192) 2024-06-10T00:58:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.341+0000 7f58556b8640 1 -- 172.21.15.37:0/1302559187 learned_addr learned my addr 172.21.15.37:0/1302559187 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.341+0000 7f58566ba640 1 --2- 172.21.15.37:0/1302559187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629228e3b70 0x5629228e6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.342+0000 7f5846ffd640 1 -- 172.21.15.37:0/1786641232 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f584c0d7c90 con 0x7f5850100900 2024-06-10T00:58:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.342+0000 7f58556b8640 1 -- 172.21.15.37:0/1302559187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629228de010 msgr2=0x5629228de430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.342+0000 7f58556b8640 1 --2- 172.21.15.37:0/1302559187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629228de010 0x5629228de430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.342+0000 7f58556b8640 1 -- 172.21.15.37:0/1302559187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629228e3b70 msgr2=0x5629228e6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.342+0000 7f58556b8640 1 --2- 172.21.15.37:0/1302559187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629228e3b70 0x5629228e6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.342+0000 7f58556b8640 1 -- 172.21.15.37:0/1302559187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629227fb9b0 con 0x5629228dea00 2024-06-10T00:58:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.342+0000 7f58566ba640 1 --2- 172.21.15.37:0/1302559187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629228e3b70 0x5629228e6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:58:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.342+0000 7f58566ba640 1 --2- 172.21.15.37:0/1786641232 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f583c0a2dd0 0x7f583c0a5290 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5848005fd0 tx=0x7f58480149d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.342+0000 7f58556b8640 1 --2- 172.21.15.37:0/1302559187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629228dea00 0x5629228e3500 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5840009900 tx=0x7f5840004520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1aeec7c7fdc65 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.342+0000 7f583bfff640 1 -- 172.21.15.37:0/1302559187 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5840035030 con 0x5629228dea00 2024-06-10T00:58:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.342+0000 7f583bfff640 1 -- 172.21.15.37:0/1302559187 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5840004770 con 0x5629228dea00 2024-06-10T00:58:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.342+0000 7f583bfff640 1 -- 172.21.15.37:0/1302559187 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5840031a10 con 0x5629228dea00 2024-06-10T00:58:15.353 INFO:tasks.workunit.client.0.smithi037.stderr:+ cd testdir 2024-06-10T00:58:15.353 INFO:tasks.workunit.client.0.smithi037.stderr:+ set +e 2024-06-10T00:58:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.343+0000 7f585ae1f740 1 -- 172.21.15.37:0/1302559187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629228dea00 msgr2=0x5629228e3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.343+0000 7f585ae1f740 1 --2- 172.21.15.37:0/1302559187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629228dea00 0x5629228e3500 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5840009900 tx=0x7f5840004520 comp rx=0 tx=0).stop 2024-06-10T00:58:15.354 INFO:tasks.workunit.client.0.smithi037.stderr:+ setfacl -d -m u:nobody:rw . 2024-06-10T00:58:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.343+0000 7f585ae1f740 1 -- 172.21.15.37:0/1302559187 shutdown_connections 2024-06-10T00:58:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.343+0000 7f585ae1f740 1 --2- 172.21.15.37:0/1302559187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629228e3b70 0x5629228e6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.343+0000 7f585ae1f740 1 --2- 172.21.15.37:0/1302559187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629228dea00 0x5629228e3500 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.343+0000 7f585ae1f740 1 --2- 172.21.15.37:0/1302559187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629228de010 0x5629228de430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.343+0000 7f585ae1f740 1 -- 172.21.15.37:0/1302559187 >> 172.21.15.37:0/1302559187 conn(0x5629228d9680 msgr2=0x5629228d9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.343+0000 7f585ae1f740 1 -- 172.21.15.37:0/1302559187 shutdown_connections 2024-06-10T00:58:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.343+0000 7f585ae1f740 1 -- 172.21.15.37:0/1302559187 wait complete. 2024-06-10T00:58:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.344+0000 7f585ae1f740 1 -- start start 2024-06-10T00:58:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.344+0000 7f585ae1f740 1 Processor -- start 2024-06-10T00:58:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.344+0000 7f585ae1f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629228de010 0x5629228e82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.344+0000 7f585ae1f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629228dea00 0x5629228e8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.344+0000 7f585ae1f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629228e3b70 0x5629228ef840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.344+0000 7f585ae1f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5629228e6ac0 con 0x5629228dea00 2024-06-10T00:58:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.344+0000 7f585ae1f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5629227caae0 con 0x5629228e3b70 2024-06-10T00:58:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.344+0000 7f58556b8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629228dea00 0x5629228e8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.344+0000 7f58556b8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629228dea00 0x5629228e8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42202/0 (socket says 172.21.15.37:42202) 2024-06-10T00:58:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.344+0000 7f58556b8640 1 -- 172.21.15.37:0/3420428258 learned_addr learned my addr 172.21.15.37:0/3420428258 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.344+0000 7f585ae1f740 1 -- 172.21.15.37:0/3420428258 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5629228e6c40 con 0x5629228de010 2024-06-10T00:58:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.344+0000 7f585ae1f740 10 client.0 _ll_register_callbacks cb 0x5629228039b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:58:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.344+0000 7f5855eb9640 1 --2- 172.21.15.37:0/3420428258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629228de010 0x5629228e82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.344+0000 7f58566ba640 1 --2- 172.21.15.37:0/3420428258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629228e3b70 0x5629228ef840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.345+0000 7f58556b8640 1 -- 172.21.15.37:0/3420428258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629228de010 msgr2=0x5629228e82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.345+0000 7f58556b8640 1 --2- 172.21.15.37:0/3420428258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629228de010 0x5629228e82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.345+0000 7f58556b8640 1 -- 172.21.15.37:0/3420428258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629228e3b70 msgr2=0x5629228ef840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.345+0000 7f58556b8640 1 --2- 172.21.15.37:0/3420428258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629228e3b70 0x5629228ef840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.345+0000 7f58556b8640 1 -- 172.21.15.37:0/3420428258 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562922555210 con 0x5629228dea00 2024-06-10T00:58:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.345+0000 7f58556b8640 1 --2- 172.21.15.37:0/3420428258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629228dea00 0x5629228e8800 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5840006010 tx=0x7f5840005370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cdf1a9e231e4397 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.345+0000 7f58397fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:58:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.345+0000 7f58397fa640 1 -- 172.21.15.37:0/3420428258 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5840035040 con 0x5629228dea00 2024-06-10T00:58:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.345+0000 7f58397fa640 1 -- 172.21.15.37:0/3420428258 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f5840031990 con 0x5629228dea00 2024-06-10T00:58:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.345+0000 7f585ae1f740 1 -- 172.21.15.37:0/3420428258 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5840004770 con 0x5629228dea00 2024-06-10T00:58:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.346+0000 7f58397fa640 1 -- 172.21.15.37:0/3420428258 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5840059050 con 0x5629228dea00 2024-06-10T00:58:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.346+0000 7f585ae1f740 10 client.15801 fetch_fsmap learned FSMap version 27 2024-06-10T00:58:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.346+0000 7f585ae1f740 1 -- 172.21.15.37:0/3420428258 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562922621620 con 0x5629228dea00 2024-06-10T00:58:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.346+0000 7f58397fa640 1 -- 172.21.15.37:0/3420428258 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5840004770 con 0x5629228dea00 2024-06-10T00:58:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.346+0000 7f58397fa640 1 -- 172.21.15.37:0/3420428258 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5840054030 con 0x5629228dea00 2024-06-10T00:58:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.346+0000 7f585ae1f740 10 client.15801 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:58:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.346+0000 7f585ae1f740 10 client.15801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T00:58:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.346+0000 7f585ae1f740 20 client.15801 populate_metadata read hostname 'smithi037' 2024-06-10T00:58:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.346+0000 7f585ae1f740 1 --2- 172.21.15.37:0/3420428258 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5629228f3ca0 0x5629228f6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.346+0000 7f585ae1f740 4 client.15801 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:58:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.346+0000 7f585ae1f740 1 -- 172.21.15.37:0/3420428258 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562922854f10 con 0x5629228f3ca0 2024-06-10T00:58:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.346+0000 7f5855eb9640 1 --2- 172.21.15.37:0/3420428258 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5629228f3ca0 0x5629228f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.347+0000 7f5855eb9640 1 --2- 172.21.15.37:0/3420428258 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5629228f3ca0 0x5629228f6090 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d356c99b00440113 server_cookie=4fe39c2390b2a54e in_seq=0 out_seq=0 2024-06-10T00:58:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.347+0000 7f58397fa640 10 client.15801 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.348+0000 7f58397fa640 1 -- 172.21.15.37:0/3420428258 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562922854f10 con 0x5629228f3ca0 2024-06-10T00:58:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.348+0000 7f58397fa640 10 client.15801 handle_command_reply: tid=0 2024-06-10T00:58:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.348+0000 7f585ae1f740 1 client.15801 shutdown 2024-06-10T00:58:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.348+0000 7f585ae1f740 1 -- 172.21.15.37:0/3420428258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629228dea00 msgr2=0x5629228e8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.348+0000 7f585ae1f740 1 --2- 172.21.15.37:0/3420428258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629228dea00 0x5629228e8800 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5840006010 tx=0x7f5840005370 comp rx=0 tx=0).stop 2024-06-10T00:58:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.348+0000 7f585ae1f740 1 -- 172.21.15.37:0/3420428258 shutdown_connections 2024-06-10T00:58:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.348+0000 7f585ae1f740 1 --2- 172.21.15.37:0/3420428258 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5629228f3ca0 0x5629228f6090 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.348+0000 7f585ae1f740 1 --2- 172.21.15.37:0/3420428258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629228e3b70 0x5629228ef840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.348+0000 7f585ae1f740 1 --2- 172.21.15.37:0/3420428258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629228dea00 0x5629228e8800 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.348+0000 7f58397fa640 0 client.15801 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.348+0000 7f585ae1f740 1 --2- 172.21.15.37:0/3420428258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629228de010 0x5629228e82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.348+0000 7f585ae1f740 1 -- 172.21.15.37:0/3420428258 >> 172.21.15.37:0/3420428258 conn(0x5629228d9680 msgr2=0x5629228d87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.348+0000 7f585ae1f740 1 -- 172.21.15.37:0/3420428258 shutdown_connections 2024-06-10T00:58:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.348+0000 7f585ae1f740 1 -- 172.21.15.37:0/3420428258 wait complete. 2024-06-10T00:58:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.348+0000 7f585ae1f740 20 client.15801 trim_cache size 0 max 16384 2024-06-10T00:58:15.362 INFO:tasks.workunit.client.0.smithi037.stderr:+ test 0 '!=' 0 2024-06-10T00:58:15.363 INFO:tasks.workunit.client.0.smithi037.stderr:+ set -e 2024-06-10T00:58:15.363 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=0 2024-06-10T00:58:15.363 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 0 -lt 100 ']' 2024-06-10T00:58:15.363 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 0 + 1 2024-06-10T00:58:15.363 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=1 2024-06-10T00:58:15.363 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:58:15.363 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:15.363 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.366+0000 7f585ae1f740 1 Processor -- start 2024-06-10T00:58:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.366+0000 7f585ae1f740 1 -- start start 2024-06-10T00:58:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.366+0000 7f585ae1f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629228de010 0x5629228dfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.366+0000 7f585ae1f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629228dea00 0x5629228e0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.366+0000 7f585ae1f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629228e3b70 0x5629228ef800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.366+0000 7f585ae1f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f58501aad80 con 0x5629228de010 2024-06-10T00:58:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.366+0000 7f585ae1f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f58501a8650 con 0x5629228dea00 2024-06-10T00:58:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.366+0000 7f585ae1f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f58501a64a0 con 0x5629228e3b70 2024-06-10T00:58:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.366+0000 7f5855eb9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629228de010 0x5629228dfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.367+0000 7f5855eb9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629228de010 0x5629228dfce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42206/0 (socket says 172.21.15.37:42206) 2024-06-10T00:58:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.367+0000 7f5855eb9640 1 -- 172.21.15.37:0/1171794835 learned_addr learned my addr 172.21.15.37:0/1171794835 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.367+0000 7f58556b8640 1 --2- 172.21.15.37:0/1171794835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629228dea00 0x5629228e0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.367+0000 7f58566ba640 1 --2- 172.21.15.37:0/1171794835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629228e3b70 0x5629228ef800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.367+0000 7f5855eb9640 1 -- 172.21.15.37:0/1171794835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629228e3b70 msgr2=0x5629228ef800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.367+0000 7f5855eb9640 1 --2- 172.21.15.37:0/1171794835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629228e3b70 0x5629228ef800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.367+0000 7f5855eb9640 1 -- 172.21.15.37:0/1171794835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629228dea00 msgr2=0x5629228e0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.367+0000 7f5855eb9640 1 --2- 172.21.15.37:0/1171794835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629228dea00 0x5629228e0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.367+0000 7f5855eb9640 1 -- 172.21.15.37:0/1171794835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629228da350 con 0x5629228de010 2024-06-10T00:58:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.367+0000 7f5855eb9640 1 --2- 172.21.15.37:0/1171794835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629228de010 0x5629228dfce0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f584c0a3eb0 tx=0x7f584c0199c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab861261b02bf1d0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.367+0000 7f583b7fe640 1 -- 172.21.15.37:0/1171794835 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c0acd70 con 0x5629228de010 2024-06-10T00:58:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.367+0000 7f583b7fe640 1 -- 172.21.15.37:0/1171794835 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562922854f10 con 0x5629228de010 2024-06-10T00:58:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.367+0000 7f583b7fe640 1 -- 172.21.15.37:0/1171794835 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c0ac8e0 con 0x5629228de010 2024-06-10T00:58:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.368+0000 7f585ae1f740 1 -- 172.21.15.37:0/1171794835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629228de010 msgr2=0x5629228dfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.368+0000 7f585ae1f740 1 --2- 172.21.15.37:0/1171794835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629228de010 0x5629228dfce0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f584c0a3eb0 tx=0x7f584c0199c0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.368+0000 7f585ae1f740 1 -- 172.21.15.37:0/1171794835 shutdown_connections 2024-06-10T00:58:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.368+0000 7f585ae1f740 1 --2- 172.21.15.37:0/1171794835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629228e3b70 0x5629228ef800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.368+0000 7f585ae1f740 1 --2- 172.21.15.37:0/1171794835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629228dea00 0x5629228e0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.368+0000 7f585ae1f740 1 --2- 172.21.15.37:0/1171794835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629228de010 0x5629228dfce0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.368+0000 7f585ae1f740 1 -- 172.21.15.37:0/1171794835 >> 172.21.15.37:0/1171794835 conn(0x5629228d9680 msgr2=0x5629228d9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.368+0000 7f585ae1f740 1 -- 172.21.15.37:0/1171794835 shutdown_connections 2024-06-10T00:58:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.368+0000 7f585ae1f740 1 -- 172.21.15.37:0/1171794835 wait complete. 2024-06-10T00:58:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.369+0000 7f585ae1f740 1 -- start start 2024-06-10T00:58:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.369+0000 7f585ae1f740 1 Processor -- start 2024-06-10T00:58:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.369+0000 7f585ae1f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629228de010 0x5629228ecd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.369+0000 7f585ae1f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629228dea00 0x5629228ed280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.369+0000 7f585ae1f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629228e3b70 0x5629228ee290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.369+0000 7f585ae1f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f58501a6b10 con 0x5629228dea00 2024-06-10T00:58:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.369+0000 7f585ae1f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f58501a7f80 con 0x5629228de010 2024-06-10T00:58:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.369+0000 7f585ae1f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f58501a71e0 con 0x5629228e3b70 2024-06-10T00:58:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.369+0000 7f585ae1f740 10 client.0 _ll_register_callbacks cb 0x7f58400043f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:58:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.369+0000 7f58556b8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629228dea00 0x5629228ed280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.369+0000 7f58556b8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629228dea00 0x5629228ed280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42222/0 (socket says 172.21.15.37:42222) 2024-06-10T00:58:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.369+0000 7f58556b8640 1 -- 172.21.15.37:0/2198554275 learned_addr learned my addr 172.21.15.37:0/2198554275 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.370+0000 7f5855eb9640 1 --2- 172.21.15.37:0/2198554275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629228de010 0x5629228ecd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.370+0000 7f58566ba640 1 --2- 172.21.15.37:0/2198554275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629228e3b70 0x5629228ee290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.370+0000 7f58556b8640 1 -- 172.21.15.37:0/2198554275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629228e3b70 msgr2=0x5629228ee290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.370+0000 7f58556b8640 1 --2- 172.21.15.37:0/2198554275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629228e3b70 0x5629228ee290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.370+0000 7f58556b8640 1 -- 172.21.15.37:0/2198554275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629228de010 msgr2=0x5629228ecd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.370+0000 7f58556b8640 1 --2- 172.21.15.37:0/2198554275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629228de010 0x5629228ecd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.370+0000 7f58556b8640 1 -- 172.21.15.37:0/2198554275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629228ee880 con 0x5629228dea00 2024-06-10T00:58:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.370+0000 7f58566ba640 1 --2- 172.21.15.37:0/2198554275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629228e3b70 0x5629228ee290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:58:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.370+0000 7f58556b8640 1 --2- 172.21.15.37:0/2198554275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629228dea00 0x5629228ed280 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f58400092a0 tx=0x7f584003b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852bba03ded6670 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.370+0000 7f58397fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:58:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.370+0000 7f58397fa640 1 -- 172.21.15.37:0/2198554275 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5840062060 con 0x5629228dea00 2024-06-10T00:58:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.370+0000 7f58397fa640 1 -- 172.21.15.37:0/2198554275 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f584003b8a0 con 0x5629228dea00 2024-06-10T00:58:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.370+0000 7f585ae1f740 1 -- 172.21.15.37:0/2198554275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562922854f10 con 0x5629228dea00 2024-06-10T00:58:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.370+0000 7f58397fa640 1 -- 172.21.15.37:0/2198554275 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5840069050 con 0x5629228dea00 2024-06-10T00:58:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.370+0000 7f585ae1f740 10 client.15807 fetch_fsmap learned FSMap version 27 2024-06-10T00:58:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.370+0000 7f585ae1f740 1 -- 172.21.15.37:0/2198554275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5629228efb90 con 0x5629228dea00 2024-06-10T00:58:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.371+0000 7f58397fa640 1 -- 172.21.15.37:0/2198554275 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562922854f10 con 0x5629228dea00 2024-06-10T00:58:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.371+0000 7f58397fa640 1 -- 172.21.15.37:0/2198554275 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5840054040 con 0x5629228dea00 2024-06-10T00:58:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.371+0000 7f585ae1f740 10 client.15807 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:58:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.371+0000 7f585ae1f740 10 client.15807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T00:58:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.371+0000 7f585ae1f740 20 client.15807 populate_metadata read hostname 'smithi037' 2024-06-10T00:58:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.371+0000 7f585ae1f740 1 --2- 172.21.15.37:0/2198554275 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56292295f0b0 0x5629229614d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.371+0000 7f585ae1f740 4 client.15807 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T00:58:15.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.371+0000 7f585ae1f740 1 -- 172.21.15.37:0/2198554275 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56292284ec20 con 0x56292295f0b0 2024-06-10T00:58:15.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.371+0000 7f5855eb9640 1 --2- 172.21.15.37:0/2198554275 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56292295f0b0 0x5629229614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:15.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.372+0000 7f58397fa640 10 client.15807 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:15.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.372+0000 7f5855eb9640 1 --2- 172.21.15.37:0/2198554275 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56292295f0b0 0x5629229614d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f12af91cb3e96140 server_cookie=d0046629e1906c01 in_seq=0 out_seq=0 2024-06-10T00:58:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.384+0000 7f58397fa640 1 -- 172.21.15.37:0/2198554275 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56292284ec20 con 0x56292295f0b0 2024-06-10T00:58:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.384+0000 7f58397fa640 10 client.15807 handle_command_reply: tid=0 2024-06-10T00:58:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.384+0000 7f585ae1f740 1 client.15807 shutdown 2024-06-10T00:58:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.384+0000 7f585ae1f740 1 -- 172.21.15.37:0/2198554275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629228dea00 msgr2=0x5629228ed280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.384+0000 7f585ae1f740 1 --2- 172.21.15.37:0/2198554275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629228dea00 0x5629228ed280 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f58400092a0 tx=0x7f584003b660 comp rx=0 tx=0).stop 2024-06-10T00:58:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.384+0000 7f585ae1f740 1 -- 172.21.15.37:0/2198554275 shutdown_connections 2024-06-10T00:58:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.384+0000 7f585ae1f740 1 --2- 172.21.15.37:0/2198554275 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56292295f0b0 0x5629229614d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.384+0000 7f58397fa640 0 client.15807 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.384+0000 7f585ae1f740 1 --2- 172.21.15.37:0/2198554275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629228e3b70 0x5629228ee290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.384+0000 7f585ae1f740 1 --2- 172.21.15.37:0/2198554275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629228dea00 0x5629228ed280 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.384+0000 7f585ae1f740 1 --2- 172.21.15.37:0/2198554275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5629228de010 0x5629228ecd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.384+0000 7f585ae1f740 1 -- 172.21.15.37:0/2198554275 >> 172.21.15.37:0/2198554275 conn(0x5629228d9680 msgr2=0x5629228e82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.384+0000 7f585ae1f740 1 -- 172.21.15.37:0/2198554275 shutdown_connections 2024-06-10T00:58:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.384+0000 7f585ae1f740 1 -- 172.21.15.37:0/2198554275 wait complete. 2024-06-10T00:58:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.384+0000 7f585ae1f740 20 client.15807 trim_cache size 0 max 16384 2024-06-10T00:58:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T00:58:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T00:58:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T00:58:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.385+0000 7f585811d640 1 -- 172.21.15.37:0/1786641232 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f583c0a2dd0 msgr2=0x7f583c0a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.385+0000 7f585811d640 1 --2- 172.21.15.37:0/1786641232 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f583c0a2dd0 0x7f583c0a5290 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5848005fd0 tx=0x7f58480149d0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.385+0000 7f585811d640 1 -- 172.21.15.37:0/1786641232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5850100900 msgr2=0x7f5850196bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.385+0000 7f585811d640 1 --2- 172.21.15.37:0/1786641232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5850100900 0x7f5850196bc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f584c009f80 tx=0x7f584c00c2d0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.385+0000 7f585811d640 1 -- 172.21.15.37:0/1786641232 shutdown_connections 2024-06-10T00:58:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.385+0000 7f585811d640 1 --2- 172.21.15.37:0/1786641232 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f583c0a2dd0 0x7f583c0a5290 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.385+0000 7f585811d640 1 --2- 172.21.15.37:0/1786641232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58501024b0 0x7f585019e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.385+0000 7f585811d640 1 --2- 172.21.15.37:0/1786641232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5850101af0 0x7f5850197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.385+0000 7f585811d640 1 --2- 172.21.15.37:0/1786641232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5850100900 0x7f5850196bc0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.385+0000 7f585811d640 1 -- 172.21.15.37:0/1786641232 >> 172.21.15.37:0/1786641232 conn(0x7f58500fbe70 msgr2=0x7f5850104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.385+0000 7f585811d640 1 -- 172.21.15.37:0/1786641232 shutdown_connections 2024-06-10T00:58:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:15.385+0000 7f585811d640 1 -- 172.21.15.37:0/1786641232 wait complete. 2024-06-10T00:58:15.394 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:58:15.394 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:58:15.394 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:58:16.309 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:16.309 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:16.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:16 smithi037 ceph-mon[32192]: pgmap v477: 129 pgs: 129 active+clean; 686 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 7 op/s 2024-06-10T00:58:16.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:16 smithi167 ceph-mon[36917]: pgmap v477: 129 pgs: 129 active+clean; 686 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 7 op/s 2024-06-10T00:58:16.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:16 smithi094 ceph-mon[36708]: pgmap v477: 129 pgs: 129 active+clean; 686 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 7 op/s 2024-06-10T00:58:16.623 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:58:16.623 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:58:16.669 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:16.669 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:16.669 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 1 -lt 100 ']' 2024-06-10T00:58:16.669 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 1 + 1 2024-06-10T00:58:16.808 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=2 2024-06-10T00:58:16.809 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:58:16.905 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:16.905 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:16.906 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:58:16.906 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:58:16.906 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:58:17.832 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f993ef51', total seconds elapsed: 42.0 2024-06-10T00:58:17.832 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-06-10T00:58:17.832 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:58:18.396 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:18 smithi037 ceph-mon[32192]: pgmap v478: 129 pgs: 129 active+clean; 708 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-06-10T00:58:18.557 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:18.557 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:18.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:18 smithi167 ceph-mon[36917]: pgmap v478: 129 pgs: 129 active+clean; 708 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-06-10T00:58:18.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:18 smithi094 ceph-mon[36708]: pgmap v478: 129 pgs: 129 active+clean; 708 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-06-10T00:58:18.804 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:58:18.804 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:58:18.854 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:18.854 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:18.854 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -lt 100 ']' 2024-06-10T00:58:18.854 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 2 + 1 2024-06-10T00:58:18.974 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=3 2024-06-10T00:58:18.974 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:58:18.980 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:18.980 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:18.981 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:58:18.982 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:58:18.982 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:58:19.570 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:19.570 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:19.831 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:58:19.831 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:58:19.871 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:19.872 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:19.872 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 3 -lt 100 ']' 2024-06-10T00:58:19.872 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 3 + 1 2024-06-10T00:58:19.971 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=4 2024-06-10T00:58:19.971 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:58:20.046 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:20.046 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:20.048 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:58:20.048 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:58:20.048 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:58:20.503 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:20 smithi037 ceph-mon[32192]: pgmap v479: 129 pgs: 129 active+clean; 708 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-06-10T00:58:20.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:20 smithi167 ceph-mon[36917]: pgmap v479: 129 pgs: 129 active+clean; 708 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-06-10T00:58:20.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:20 smithi094 ceph-mon[36708]: pgmap v479: 129 pgs: 129 active+clean; 708 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-06-10T00:58:21.041 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:21.042 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:21.364 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:58:21.364 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:58:21.378 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:21.379 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:21.379 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 4 -lt 100 ']' 2024-06-10T00:58:21.379 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 4 + 1 2024-06-10T00:58:21.520 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=5 2024-06-10T00:58:21.520 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:58:21.607 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:21.607 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:21.609 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:58:21.609 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:58:21.609 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:58:22.425 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:22.425 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:22.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:22 smithi167 ceph-mon[36917]: pgmap v480: 129 pgs: 129 active+clean; 708 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-06-10T00:58:22.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:22 smithi094 ceph-mon[36708]: pgmap v480: 129 pgs: 129 active+clean; 708 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-06-10T00:58:22.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:22 smithi037 ceph-mon[32192]: pgmap v480: 129 pgs: 129 active+clean; 708 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-06-10T00:58:22.772 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:58:22.772 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:58:22.795 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:22.795 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:22.795 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 5 -lt 100 ']' 2024-06-10T00:58:22.795 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 5 + 1 2024-06-10T00:58:22.880 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=6 2024-06-10T00:58:22.880 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:58:22.943 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:22.943 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:22.944 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:58:22.944 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:58:22.944 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:58:23.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:23 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:58:23.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:23 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:58:23.646 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:23 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:58:24.316 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:24.316 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:24.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:24 smithi167 ceph-mon[36917]: pgmap v481: 129 pgs: 129 active+clean; 743 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.6 KiB/s wr, 10 op/s 2024-06-10T00:58:24.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:24 smithi094 ceph-mon[36708]: pgmap v481: 129 pgs: 129 active+clean; 743 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.6 KiB/s wr, 10 op/s 2024-06-10T00:58:24.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:24 smithi037 ceph-mon[32192]: pgmap v481: 129 pgs: 129 active+clean; 743 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.6 KiB/s wr, 10 op/s 2024-06-10T00:58:24.731 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:58:24.731 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:58:24.761 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:24.761 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:24.762 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 6 -lt 100 ']' 2024-06-10T00:58:24.762 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 6 + 1 2024-06-10T00:58:24.823 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=7 2024-06-10T00:58:24.823 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:58:24.892 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:24.893 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:24.894 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:58:24.894 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:58:24.894 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:58:25.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:25 smithi167 ceph-mon[36917]: pgmap v482: 129 pgs: 129 active+clean; 743 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.1 KiB/s wr, 9 op/s 2024-06-10T00:58:25.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:25 smithi094 ceph-mon[36708]: pgmap v482: 129 pgs: 129 active+clean; 743 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.1 KiB/s wr, 9 op/s 2024-06-10T00:58:25.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:25 smithi037 ceph-mon[32192]: pgmap v482: 129 pgs: 129 active+clean; 743 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.1 KiB/s wr, 9 op/s 2024-06-10T00:58:26.116 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:26.116 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:26.563 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:58:26.563 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:58:26.578 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:26.578 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:26.578 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 7 -lt 100 ']' 2024-06-10T00:58:26.578 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 7 + 1 2024-06-10T00:58:26.647 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=8 2024-06-10T00:58:26.647 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:58:26.708 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:26.708 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:26.709 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:58:26.709 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:58:26.709 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:58:27.754 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:27 smithi037 ceph-mon[32192]: pgmap v483: 129 pgs: 129 active+clean; 789 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-06-10T00:58:27.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:27 smithi167 ceph-mon[36917]: pgmap v483: 129 pgs: 129 active+clean; 789 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-06-10T00:58:27.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:27 smithi094 ceph-mon[36708]: pgmap v483: 129 pgs: 129 active+clean; 789 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-06-10T00:58:28.350 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:28.350 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:28.446 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:58:28.446 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:58:28.461 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:28.461 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:28.461 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 8 -lt 100 ']' 2024-06-10T00:58:28.461 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 8 + 1 2024-06-10T00:58:28.662 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=9 2024-06-10T00:58:28.662 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:58:28.710 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:28.711 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:28.712 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:58:28.712 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:58:28.712 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:58:29.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:29 smithi167 ceph-mon[36917]: pgmap v484: 129 pgs: 129 active+clean; 789 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.9 KiB/s wr, 9 op/s 2024-06-10T00:58:29.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:29 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:29.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:29 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:58:29.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:29 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:58:29.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:29 smithi094 ceph-mon[36708]: pgmap v484: 129 pgs: 129 active+clean; 789 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.9 KiB/s wr, 9 op/s 2024-06-10T00:58:29.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:29 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:29.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:29 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:58:29.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:29 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:58:29.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:29 smithi037 ceph-mon[32192]: pgmap v484: 129 pgs: 129 active+clean; 789 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.9 KiB/s wr, 9 op/s 2024-06-10T00:58:29.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:29 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:58:29.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:29 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:58:29.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:29 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:58:30.590 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:30.662 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:30.971 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:58:30.971 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:58:30.995 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:30.996 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:30.996 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 9 -lt 100 ']' 2024-06-10T00:58:30.996 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 9 + 1 2024-06-10T00:58:31.110 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:58:31.188 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=10 2024-06-10T00:58:31.188 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:58:31.283 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:31.283 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:31.284 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:58:31.284 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:58:31.284 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:58:31.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:31 smithi167 ceph-mon[36917]: pgmap v485: 129 pgs: 129 active+clean; 789 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.7 KiB/s wr, 7 op/s 2024-06-10T00:58:31.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:31 smithi094 ceph-mon[36708]: pgmap v485: 129 pgs: 129 active+clean; 789 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.7 KiB/s wr, 7 op/s 2024-06-10T00:58:31.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:31 smithi037 ceph-mon[32192]: pgmap v485: 129 pgs: 129 active+clean; 789 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.7 KiB/s wr, 7 op/s 2024-06-10T00:58:32.607 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:32.607 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:33.062 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:58:33.062 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:58:33.094 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:33.094 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:33.094 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 10 -lt 100 ']' 2024-06-10T00:58:33.094 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 10 + 1 2024-06-10T00:58:33.211 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=11 2024-06-10T00:58:33.211 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:58:33.417 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:33.417 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:33.418 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:58:33.419 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:58:33.419 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:58:33.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:33 smithi037 ceph-mon[32192]: pgmap v486: 129 pgs: 129 active+clean; 816 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 8 op/s 2024-06-10T00:58:33.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:33 smithi167 ceph-mon[36917]: pgmap v486: 129 pgs: 129 active+clean; 816 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 8 op/s 2024-06-10T00:58:33.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:33 smithi094 ceph-mon[36708]: pgmap v486: 129 pgs: 129 active+clean; 816 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 8 op/s 2024-06-10T00:58:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.115+0000 7ff5b9eca640 1 -- 172.21.15.37:0/3292158876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5b4101a10 msgr2=0x7ff5b4101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.115+0000 7ff5b9eca640 1 --2- 172.21.15.37:0/3292158876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5b4101a10 0x7ff5b4101e30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4009900 tx=0x7ff5a402a3f0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.115+0000 7ff5b9eca640 1 -- 172.21.15.37:0/3292158876 shutdown_connections 2024-06-10T00:58:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.115+0000 7ff5b9eca640 1 --2- 172.21.15.37:0/3292158876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5b41035c0 0x7ff5b4109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.115+0000 7ff5b9eca640 1 --2- 172.21.15.37:0/3292158876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5b4102c00 0x7ff5b4103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.115+0000 7ff5b9eca640 1 --2- 172.21.15.37:0/3292158876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5b4101a10 0x7ff5b4101e30 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.116+0000 7ff5b9eca640 1 -- 172.21.15.37:0/3292158876 >> 172.21.15.37:0/3292158876 conn(0x7ff5b40fcf60 msgr2=0x7ff5b40ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.116+0000 7ff5b9eca640 1 -- 172.21.15.37:0/3292158876 shutdown_connections 2024-06-10T00:58:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.116+0000 7ff5b9eca640 1 -- 172.21.15.37:0/3292158876 wait complete. 2024-06-10T00:58:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.224+0000 7ff5b9eca640 1 Processor -- start 2024-06-10T00:58:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.224+0000 7ff5b9eca640 1 -- start start 2024-06-10T00:58:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.225+0000 7ff5b9eca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5b4101a10 0x7ff5b4199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.225+0000 7ff5b9eca640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5b4102c00 0x7ff5b419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.225+0000 7ff5b9eca640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5b41035c0 0x7ff5b41a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.225+0000 7ff5b9eca640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b410d6d0 con 0x7ff5b4102c00 2024-06-10T00:58:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.225+0000 7ff5b9eca640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b410d550 con 0x7ff5b4101a10 2024-06-10T00:58:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.225+0000 7ff5b9eca640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b410d850 con 0x7ff5b41035c0 2024-06-10T00:58:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.225+0000 7ff5b2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5b4102c00 0x7ff5b419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.225+0000 7ff5b2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5b4102c00 0x7ff5b419a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42772/0 (socket says 172.21.15.37:42772) 2024-06-10T00:58:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.225+0000 7ff5b2ffd640 1 -- 172.21.15.37:0/4092158727 learned_addr learned my addr 172.21.15.37:0/4092158727 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.225+0000 7ff5b3fff640 1 --2- 172.21.15.37:0/4092158727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5b41035c0 0x7ff5b41a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.225+0000 7ff5b37fe640 1 --2- 172.21.15.37:0/4092158727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5b4101a10 0x7ff5b4199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.225+0000 7ff5b2ffd640 1 -- 172.21.15.37:0/4092158727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5b41035c0 msgr2=0x7ff5b41a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.225+0000 7ff5b2ffd640 1 --2- 172.21.15.37:0/4092158727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5b41035c0 0x7ff5b41a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.225+0000 7ff5b2ffd640 1 -- 172.21.15.37:0/4092158727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5b4101a10 msgr2=0x7ff5b4199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.225+0000 7ff5b2ffd640 1 --2- 172.21.15.37:0/4092158727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5b4101a10 0x7ff5b4199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.226+0000 7ff5b2ffd640 1 -- 172.21.15.37:0/4092158727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b41a1b90 con 0x7ff5b4102c00 2024-06-10T00:58:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.226+0000 7ff5b3fff640 1 --2- 172.21.15.37:0/4092158727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5b41035c0 0x7ff5b41a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:58:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.226+0000 7ff5b2ffd640 1 --2- 172.21.15.37:0/4092158727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5b4102c00 0x7ff5b419a500 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800e9f0 tx=0x7ff5a800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.263+0000 7ff5b0ff9640 1 -- 172.21.15.37:0/4092158727 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a800ccc0 con 0x7ff5b4102c00 2024-06-10T00:58:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.263+0000 7ff5b0ff9640 1 -- 172.21.15.37:0/4092158727 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5a800ce60 con 0x7ff5b4102c00 2024-06-10T00:58:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.263+0000 7ff5b0ff9640 1 -- 172.21.15.37:0/4092158727 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8010690 con 0x7ff5b4102c00 2024-06-10T00:58:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.263+0000 7ff5b9eca640 1 -- 172.21.15.37:0/4092158727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b41a1e80 con 0x7ff5b4102c00 2024-06-10T00:58:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.263+0000 7ff5b9eca640 1 -- 172.21.15.37:0/4092158727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b41a22f0 con 0x7ff5b4102c00 2024-06-10T00:58:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.266+0000 7ff5b0ff9640 1 -- 172.21.15.37:0/4092158727 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff5a80046c0 con 0x7ff5b4102c00 2024-06-10T00:58:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.267+0000 7ff5b0ff9640 1 --2- 172.21.15.37:0/4092158727 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff59c0a2da0 0x7ff59c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.267+0000 7ff5b0ff9640 1 -- 172.21.15.37:0/4092158727 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7ff5a80cf710 con 0x7ff5b4102c00 2024-06-10T00:58:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.267+0000 7ff5b37fe640 1 --2- 172.21.15.37:0/4092158727 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff59c0a2da0 0x7ff59c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.268+0000 7ff5b37fe640 1 --2- 172.21.15.37:0/4092158727 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff59c0a2da0 0x7ff59c0a5260 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff5a402a9e0 tx=0x7ff5a40315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.753+0000 7ff5bcbd6740 1 Processor -- start 2024-06-10T00:58:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.753+0000 7ff5bcbd6740 1 -- start start 2024-06-10T00:58:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.753+0000 7ff5bcbd6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567c0145010 0x5567c0145430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.753+0000 7ff5bcbd6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567c0145a00 0x5567c014a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.753+0000 7ff5bcbd6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567c014ab70 0x5567c014d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.753+0000 7ff5bcbd6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5567c00a84f0 con 0x5567c0145a00 2024-06-10T00:58:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.753+0000 7ff5bcbd6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5567c0046870 con 0x5567c0145010 2024-06-10T00:58:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.753+0000 7ff5bcbd6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5567c00a44b0 con 0x5567c014ab70 2024-06-10T00:58:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.754+0000 7ff5b2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567c0145a00 0x5567c014a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.754+0000 7ff5b2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567c0145a00 0x5567c014a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42784/0 (socket says 172.21.15.37:42784) 2024-06-10T00:58:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.754+0000 7ff5b37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567c0145010 0x5567c0145430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.754+0000 7ff5b2ffd640 1 -- 172.21.15.37:0/2386741749 learned_addr learned my addr 172.21.15.37:0/2386741749 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.754+0000 7ff5b3fff640 1 --2- 172.21.15.37:0/2386741749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567c014ab70 0x5567c014d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.754+0000 7ff5b2ffd640 1 -- 172.21.15.37:0/2386741749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567c014ab70 msgr2=0x5567c014d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.754+0000 7ff5b2ffd640 1 --2- 172.21.15.37:0/2386741749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567c014ab70 0x5567c014d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.754+0000 7ff5b2ffd640 1 -- 172.21.15.37:0/2386741749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567c0145010 msgr2=0x5567c0145430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.754+0000 7ff5b2ffd640 1 --2- 172.21.15.37:0/2386741749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567c0145010 0x5567c0145430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.754+0000 7ff5b2ffd640 1 -- 172.21.15.37:0/2386741749 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567c00629b0 con 0x5567c0145a00 2024-06-10T00:58:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.754+0000 7ff5b37fe640 1 --2- 172.21.15.37:0/2386741749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567c0145010 0x5567c0145430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:58:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.754+0000 7ff5b2ffd640 1 --2- 172.21.15.37:0/2386741749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567c0145a00 0x5567c014a500 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff5a809b880 tx=0x7ff5a809c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe2af51b73f0ca8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.754+0000 7ff599ffb640 1 -- 172.21.15.37:0/2386741749 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a80a47f0 con 0x5567c0145a00 2024-06-10T00:58:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.755+0000 7ff599ffb640 1 -- 172.21.15.37:0/2386741749 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5a8099e80 con 0x5567c0145a00 2024-06-10T00:58:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.755+0000 7ff599ffb640 1 -- 172.21.15.37:0/2386741749 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a809afd0 con 0x5567c0145a00 2024-06-10T00:58:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.755+0000 7ff5bcbd6740 1 -- 172.21.15.37:0/2386741749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567c0145a00 msgr2=0x5567c014a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.755+0000 7ff5bcbd6740 1 --2- 172.21.15.37:0/2386741749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567c0145a00 0x5567c014a500 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff5a809b880 tx=0x7ff5a809c340 comp rx=0 tx=0).stop 2024-06-10T00:58:34.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.755+0000 7ff5bcbd6740 1 -- 172.21.15.37:0/2386741749 shutdown_connections 2024-06-10T00:58:34.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.756+0000 7ff5bcbd6740 1 --2- 172.21.15.37:0/2386741749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567c014ab70 0x5567c014d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.756+0000 7ff5bcbd6740 1 --2- 172.21.15.37:0/2386741749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567c0145a00 0x5567c014a500 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.756+0000 7ff5bcbd6740 1 --2- 172.21.15.37:0/2386741749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567c0145010 0x5567c0145430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.756+0000 7ff5bcbd6740 1 -- 172.21.15.37:0/2386741749 >> 172.21.15.37:0/2386741749 conn(0x5567c0140680 msgr2=0x5567c0140a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:34.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.756+0000 7ff5bcbd6740 1 -- 172.21.15.37:0/2386741749 shutdown_connections 2024-06-10T00:58:34.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.756+0000 7ff5bcbd6740 1 -- 172.21.15.37:0/2386741749 wait complete. 2024-06-10T00:58:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.791+0000 7ff5bcbd6740 1 -- start start 2024-06-10T00:58:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.813+0000 7ff5bcbd6740 1 Processor -- start 2024-06-10T00:58:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.813+0000 7ff5bcbd6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567c0145010 0x5567c014f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.813+0000 7ff5bcbd6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567c0145a00 0x5567c014f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.813+0000 7ff5bcbd6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567c014ab70 0x5567c0156800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.813+0000 7ff5bcbd6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5567c014db20 con 0x5567c014ab70 2024-06-10T00:58:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.813+0000 7ff5bcbd6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5567c0031ae0 con 0x5567c0145010 2024-06-10T00:58:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.813+0000 7ff5bcbd6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5567c014dca0 con 0x5567c0145a00 2024-06-10T00:58:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.814+0000 7ff5b3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567c014ab70 0x5567c0156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.814+0000 7ff5b37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567c0145010 0x5567c014f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.814+0000 7ff5b3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567c014ab70 0x5567c0156800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42798/0 (socket says 172.21.15.37:42798) 2024-06-10T00:58:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.814+0000 7ff5b3fff640 1 -- 172.21.15.37:0/4209593815 learned_addr learned my addr 172.21.15.37:0/4209593815 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.814+0000 7ff5b2ffd640 1 --2- 172.21.15.37:0/4209593815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567c0145a00 0x5567c014f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.814+0000 7ff5b3fff640 1 -- 172.21.15.37:0/4209593815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567c0145a00 msgr2=0x5567c014f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.814+0000 7ff5b3fff640 1 --2- 172.21.15.37:0/4209593815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567c0145a00 0x5567c014f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.814+0000 7ff5b3fff640 1 -- 172.21.15.37:0/4209593815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567c0145010 msgr2=0x5567c014f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.814+0000 7ff5b3fff640 1 --2- 172.21.15.37:0/4209593815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567c0145010 0x5567c014f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.814+0000 7ff5b3fff640 1 -- 172.21.15.37:0/4209593815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567bfdbc210 con 0x5567c014ab70 2024-06-10T00:58:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.814+0000 7ff5b2ffd640 1 --2- 172.21.15.37:0/4209593815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567c0145a00 0x5567c014f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:58:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.814+0000 7ff5b37fe640 1 --2- 172.21.15.37:0/4209593815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567c0145010 0x5567c014f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:58:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.814+0000 7ff5b3fff640 1 --2- 172.21.15.37:0/4209593815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567c014ab70 0x5567c0156800 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00fe20 tx=0x7ff5ac012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d5c12c1da5bde7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.814+0000 7ff592ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:58:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.814+0000 7ff592ffd640 1 -- 172.21.15.37:0/4209593815 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac003970 con 0x5567c014ab70 2024-06-10T00:58:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.815+0000 7ff592ffd640 1 -- 172.21.15.37:0/4209593815 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7ff5ac003b10 con 0x5567c014ab70 2024-06-10T00:58:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.837+0000 7ff5bcbd6740 10 client.0 _ll_register_callbacks cb 0x5567c006a9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:58:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.839+0000 7ff5bcbd6740 1 -- 172.21.15.37:0/4209593815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5a8099e80 con 0x5567c014ab70 2024-06-10T00:58:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.839+0000 7ff592ffd640 1 -- 172.21.15.37:0/4209593815 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff5ac003d10 con 0x5567c014ab70 2024-06-10T00:58:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.839+0000 7ff5bcbd6740 10 client.15819 fetch_fsmap learned FSMap version 27 2024-06-10T00:58:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.839+0000 7ff5bcbd6740 1 -- 172.21.15.37:0/4209593815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=27}) -- 0x5567bfe88620 con 0x5567c014ab70 2024-06-10T00:58:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.897+0000 7ff592ffd640 1 -- 172.21.15.37:0/4209593815 <== mon.0 v2:172.21.15.37:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff5ac004050 con 0x5567c014ab70 2024-06-10T00:58:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.897+0000 7ff5bcbd6740 10 client.15819 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:58:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.897+0000 7ff5bcbd6740 10 client.15819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T00:58:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.897+0000 7ff5bcbd6740 20 client.15819 populate_metadata read hostname 'smithi037' 2024-06-10T00:58:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.897+0000 7ff5bcbd6740 1 --2- 172.21.15.37:0/4209593815 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5567c015aee0 0x5567c015d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.897+0000 7ff5bcbd6740 4 client.15819 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:58:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.897+0000 7ff5bcbd6740 1 -- 172.21.15.37:0/4209593815 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567c00bbf10 con 0x5567c015aee0 2024-06-10T00:58:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.897+0000 7ff5b37fe640 1 --2- 172.21.15.37:0/4209593815 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5567c015aee0 0x5567c015d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.898+0000 7ff592ffd640 10 client.15819 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.898+0000 7ff5b37fe640 1 --2- 172.21.15.37:0/4209593815 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5567c015aee0 0x5567c015d3a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3608b07c1111418 server_cookie=1128fdab6fa86954 in_seq=0 out_seq=0 2024-06-10T00:58:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.899+0000 7ff592ffd640 1 -- 172.21.15.37:0/4209593815 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5567c00bbf10 con 0x5567c015aee0 2024-06-10T00:58:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.899+0000 7ff592ffd640 10 client.15819 handle_command_reply: tid=0 2024-06-10T00:58:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.899+0000 7ff5bcbd6740 1 client.15819 shutdown 2024-06-10T00:58:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.899+0000 7ff5bcbd6740 1 -- 172.21.15.37:0/4209593815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567c014ab70 msgr2=0x5567c0156800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.899+0000 7ff5bcbd6740 1 --2- 172.21.15.37:0/4209593815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567c014ab70 0x5567c0156800 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00fe20 tx=0x7ff5ac012820 comp rx=0 tx=0).stop 2024-06-10T00:58:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.899+0000 7ff5bcbd6740 1 -- 172.21.15.37:0/4209593815 shutdown_connections 2024-06-10T00:58:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.899+0000 7ff5bcbd6740 1 --2- 172.21.15.37:0/4209593815 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5567c015aee0 0x5567c015d3a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.899+0000 7ff5bcbd6740 1 --2- 172.21.15.37:0/4209593815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567c014ab70 0x5567c0156800 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.899+0000 7ff5bcbd6740 1 --2- 172.21.15.37:0/4209593815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567c0145a00 0x5567c014f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.899+0000 7ff5bcbd6740 1 --2- 172.21.15.37:0/4209593815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567c0145010 0x5567c014f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.899+0000 7ff592ffd640 0 client.15819 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.899+0000 7ff5bcbd6740 1 -- 172.21.15.37:0/4209593815 >> 172.21.15.37:0/4209593815 conn(0x5567c0140680 msgr2=0x5567c013f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.899+0000 7ff5bcbd6740 1 -- 172.21.15.37:0/4209593815 shutdown_connections 2024-06-10T00:58:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.899+0000 7ff5bcbd6740 1 -- 172.21.15.37:0/4209593815 wait complete. 2024-06-10T00:58:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.914+0000 7ff5bcbd6740 20 client.15819 trim_cache size 0 max 16384 2024-06-10T00:58:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.931+0000 7ff5bcbd6740 1 Processor -- start 2024-06-10T00:58:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.931+0000 7ff5bcbd6740 1 -- start start 2024-06-10T00:58:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.932+0000 7ff5bcbd6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567c0145010 0x5567c015d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.932+0000 7ff5bcbd6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567c0145a00 0x5567c01567c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.932+0000 7ff5bcbd6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567c014ab70 0x5567c014ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.932+0000 7ff5bcbd6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b41af260 con 0x5567c0145a00 2024-06-10T00:58:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.932+0000 7ff5bcbd6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b41acac0 con 0x5567c014ab70 2024-06-10T00:58:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.932+0000 7ff5bcbd6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b41aa910 con 0x5567c0145010 2024-06-10T00:58:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.932+0000 7ff5b2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567c0145a00 0x5567c01567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.932+0000 7ff5b2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567c0145a00 0x5567c01567c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42814/0 (socket says 172.21.15.37:42814) 2024-06-10T00:58:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.932+0000 7ff5b37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567c0145010 0x5567c015d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.932+0000 7ff5b3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567c014ab70 0x5567c014ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.932+0000 7ff5b2ffd640 1 -- 172.21.15.37:0/3873220712 learned_addr learned my addr 172.21.15.37:0/3873220712 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.932+0000 7ff5b2ffd640 1 -- 172.21.15.37:0/3873220712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567c0145010 msgr2=0x5567c015d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.932+0000 7ff5b2ffd640 1 --2- 172.21.15.37:0/3873220712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567c0145010 0x5567c015d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.932+0000 7ff5b2ffd640 1 -- 172.21.15.37:0/3873220712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567c014ab70 msgr2=0x5567c014ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.932+0000 7ff5b2ffd640 1 --2- 172.21.15.37:0/3873220712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567c014ab70 0x5567c014ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.932+0000 7ff5b2ffd640 1 -- 172.21.15.37:0/3873220712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567c01404b0 con 0x5567c0145a00 2024-06-10T00:58:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.932+0000 7ff5b3fff640 1 --2- 172.21.15.37:0/3873220712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567c014ab70 0x5567c014ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:58:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.932+0000 7ff5b2ffd640 1 --2- 172.21.15.37:0/3873220712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567c0145a00 0x5567c01567c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff5a809bb10 tx=0x7ff5a80a00b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50afae4138f4fb26 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.933+0000 7ff5997fa640 1 -- 172.21.15.37:0/3873220712 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a80a47f0 con 0x5567c0145a00 2024-06-10T00:58:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.933+0000 7ff5997fa640 1 -- 172.21.15.37:0/3873220712 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5a80a4440 con 0x5567c0145a00 2024-06-10T00:58:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.933+0000 7ff5997fa640 1 -- 172.21.15.37:0/3873220712 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a80ba540 con 0x5567c0145a00 2024-06-10T00:58:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.934+0000 7ff5bcbd6740 1 -- 172.21.15.37:0/3873220712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567c0145a00 msgr2=0x5567c01567c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.934+0000 7ff5bcbd6740 1 --2- 172.21.15.37:0/3873220712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567c0145a00 0x5567c01567c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff5a809bb10 tx=0x7ff5a80a00b0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.934+0000 7ff5bcbd6740 1 -- 172.21.15.37:0/3873220712 shutdown_connections 2024-06-10T00:58:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.934+0000 7ff5bcbd6740 1 --2- 172.21.15.37:0/3873220712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567c014ab70 0x5567c014ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.934+0000 7ff5bcbd6740 1 --2- 172.21.15.37:0/3873220712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567c0145a00 0x5567c01567c0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.934+0000 7ff5bcbd6740 1 --2- 172.21.15.37:0/3873220712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567c0145010 0x5567c015d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.934+0000 7ff5bcbd6740 1 -- 172.21.15.37:0/3873220712 >> 172.21.15.37:0/3873220712 conn(0x5567c0140680 msgr2=0x5567c014a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.934+0000 7ff5bcbd6740 1 -- 172.21.15.37:0/3873220712 shutdown_connections 2024-06-10T00:58:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.934+0000 7ff5bcbd6740 1 -- 172.21.15.37:0/3873220712 wait complete. 2024-06-10T00:58:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.934+0000 7ff5bcbd6740 1 -- start start 2024-06-10T00:58:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.934+0000 7ff5bcbd6740 1 Processor -- start 2024-06-10T00:58:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.935+0000 7ff5bcbd6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567c0145010 0x5567c01534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.935+0000 7ff5bcbd6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567c0145a00 0x5567c0153a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.935+0000 7ff5bcbd6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567c014ab70 0x5567c01c3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.935+0000 7ff5bcbd6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b41aaf80 con 0x5567c0145a00 2024-06-10T00:58:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.935+0000 7ff5bcbd6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b41ac3f0 con 0x5567c014ab70 2024-06-10T00:58:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.935+0000 7ff5bcbd6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b41ab650 con 0x5567c0145010 2024-06-10T00:58:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.935+0000 7ff5bcbd6740 10 client.0 _ll_register_callbacks cb 0x7ff5ac00f4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:58:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.935+0000 7ff5b2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567c0145a00 0x5567c0153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.935+0000 7ff5b2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567c0145a00 0x5567c0153a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42818/0 (socket says 172.21.15.37:42818) 2024-06-10T00:58:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.935+0000 7ff5b2ffd640 1 -- 172.21.15.37:0/2149233231 learned_addr learned my addr 172.21.15.37:0/2149233231 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.935+0000 7ff5b37fe640 1 --2- 172.21.15.37:0/2149233231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567c0145010 0x5567c01534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.935+0000 7ff5b3fff640 1 --2- 172.21.15.37:0/2149233231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567c014ab70 0x5567c01c3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.935+0000 7ff5b37fe640 1 -- 172.21.15.37:0/2149233231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567c014ab70 msgr2=0x5567c01c3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.935+0000 7ff5b37fe640 1 --2- 172.21.15.37:0/2149233231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567c014ab70 0x5567c01c3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.935+0000 7ff5b37fe640 1 -- 172.21.15.37:0/2149233231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567c0145a00 msgr2=0x5567c0153a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.935+0000 7ff5b37fe640 1 --2- 172.21.15.37:0/2149233231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567c0145a00 0x5567c0153a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.935+0000 7ff5b37fe640 1 -- 172.21.15.37:0/2149233231 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567c0156b00 con 0x5567c0145010 2024-06-10T00:58:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.935+0000 7ff5b2ffd640 1 --2- 172.21.15.37:0/2149233231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567c0145a00 0x5567c0153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:58:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.936+0000 7ff5b37fe640 1 --2- 172.21.15.37:0/2149233231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567c0145010 0x5567c01534f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4009230 tx=0x7ff5a402a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7ef9f9509f0ce0e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.936+0000 7ff592ffd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T00:58:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.936+0000 7ff592ffd640 1 -- 172.21.15.37:0/2149233231 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4035040 con 0x5567c0145010 2024-06-10T00:58:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.936+0000 7ff5bcbd6740 1 -- 172.21.15.37:0/2149233231 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5a80a4440 con 0x5567c0145010 2024-06-10T00:58:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.936+0000 7ff592ffd640 1 -- 172.21.15.37:0/2149233231 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7ff5a4004770 con 0x5567c0145010 2024-06-10T00:58:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.936+0000 7ff592ffd640 1 -- 172.21.15.37:0/2149233231 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff5a4004970 con 0x5567c0145010 2024-06-10T00:58:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.936+0000 7ff5bcbd6740 10 client.25420 fetch_fsmap learned FSMap version 27 2024-06-10T00:58:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.936+0000 7ff5bcbd6740 1 -- 172.21.15.37:0/2149233231 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5567c01c5200 con 0x5567c0145010 2024-06-10T00:58:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.936+0000 7ff592ffd640 1 -- 172.21.15.37:0/2149233231 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5a80a4440 con 0x5567c0145010 2024-06-10T00:58:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.937+0000 7ff592ffd640 1 -- 172.21.15.37:0/2149233231 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff5a405a070 con 0x5567c0145010 2024-06-10T00:58:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.937+0000 7ff5bcbd6740 10 client.25420 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:58:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.937+0000 7ff5bcbd6740 10 client.25420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T00:58:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.937+0000 7ff5bcbd6740 20 client.25420 populate_metadata read hostname 'smithi037' 2024-06-10T00:58:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.937+0000 7ff5bcbd6740 1 --2- 172.21.15.37:0/2149233231 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5567c01c5bb0 0x5567c01c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.937+0000 7ff5bcbd6740 4 client.25420 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T00:58:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.937+0000 7ff5bcbd6740 1 -- 172.21.15.37:0/2149233231 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5567c00b5c20 con 0x5567c01c5bb0 2024-06-10T00:58:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.937+0000 7ff5b2ffd640 1 --2- 172.21.15.37:0/2149233231 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5567c01c5bb0 0x5567c01c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.938+0000 7ff5b2ffd640 1 --2- 172.21.15.37:0/2149233231 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5567c01c5bb0 0x5567c01c7fd0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ee1f2970c7614d0 server_cookie=3919f4d249c93a38 in_seq=0 out_seq=0 2024-06-10T00:58:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.938+0000 7ff592ffd640 10 client.25420 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:34.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.955+0000 7ff592ffd640 1 -- 172.21.15.37:0/2149233231 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5567c00b5c20 con 0x5567c01c5bb0 2024-06-10T00:58:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.955+0000 7ff592ffd640 10 client.25420 handle_command_reply: tid=0 2024-06-10T00:58:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.955+0000 7ff5bcbd6740 1 client.25420 shutdown 2024-06-10T00:58:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.955+0000 7ff5bcbd6740 1 -- 172.21.15.37:0/2149233231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567c0145010 msgr2=0x5567c01534f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.955+0000 7ff5bcbd6740 1 --2- 172.21.15.37:0/2149233231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567c0145010 0x5567c01534f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4009230 tx=0x7ff5a402a3f0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.955+0000 7ff5bcbd6740 1 -- 172.21.15.37:0/2149233231 shutdown_connections 2024-06-10T00:58:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.955+0000 7ff5bcbd6740 1 --2- 172.21.15.37:0/2149233231 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5567c01c5bb0 0x5567c01c7fd0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.955+0000 7ff5bcbd6740 1 --2- 172.21.15.37:0/2149233231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567c014ab70 0x5567c01c3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.955+0000 7ff5bcbd6740 1 --2- 172.21.15.37:0/2149233231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567c0145a00 0x5567c0153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.955+0000 7ff5bcbd6740 1 --2- 172.21.15.37:0/2149233231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567c0145010 0x5567c01534f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.955+0000 7ff592ffd640 0 client.25420 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.955+0000 7ff5bcbd6740 1 -- 172.21.15.37:0/2149233231 >> 172.21.15.37:0/2149233231 conn(0x5567c0140680 msgr2=0x5567c014ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.955+0000 7ff5bcbd6740 1 -- 172.21.15.37:0/2149233231 shutdown_connections 2024-06-10T00:58:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.955+0000 7ff5bcbd6740 1 -- 172.21.15.37:0/2149233231 wait complete. 2024-06-10T00:58:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.955+0000 7ff5bcbd6740 20 client.25420 trim_cache size 0 max 16384 2024-06-10T00:58:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T00:58:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T00:58:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T00:58:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.987+0000 7ff5b9eca640 1 -- 172.21.15.37:0/4092158727 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff59c0a2da0 msgr2=0x7ff59c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.987+0000 7ff5b9eca640 1 --2- 172.21.15.37:0/4092158727 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff59c0a2da0 0x7ff59c0a5260 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff5a402a9e0 tx=0x7ff5a40315b0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.987+0000 7ff5b9eca640 1 -- 172.21.15.37:0/4092158727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5b4102c00 msgr2=0x7ff5b419a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.987+0000 7ff5b9eca640 1 --2- 172.21.15.37:0/4092158727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5b4102c00 0x7ff5b419a500 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800e9f0 tx=0x7ff5a800edf0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.987+0000 7ff5b9eca640 1 -- 172.21.15.37:0/4092158727 shutdown_connections 2024-06-10T00:58:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.987+0000 7ff5b9eca640 1 --2- 172.21.15.37:0/4092158727 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff59c0a2da0 0x7ff59c0a5260 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.987+0000 7ff5b9eca640 1 --2- 172.21.15.37:0/4092158727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5b41035c0 0x7ff5b41a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.987+0000 7ff5b9eca640 1 --2- 172.21.15.37:0/4092158727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5b4102c00 0x7ff5b419a500 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.987+0000 7ff5b9eca640 1 --2- 172.21.15.37:0/4092158727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5b4101a10 0x7ff5b4199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.987+0000 7ff5b9eca640 1 -- 172.21.15.37:0/4092158727 >> 172.21.15.37:0/4092158727 conn(0x7ff5b40fcf60 msgr2=0x7ff5b4073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.987+0000 7ff5b9eca640 1 -- 172.21.15.37:0/4092158727 shutdown_connections 2024-06-10T00:58:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:34.987+0000 7ff5b9eca640 1 -- 172.21.15.37:0/4092158727 wait complete. 2024-06-10T00:58:35.503 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:35.503 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:35.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:35 smithi167 ceph-mon[36917]: pgmap v487: 129 pgs: 129 active+clean; 816 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.9 KiB/s wr, 7 op/s 2024-06-10T00:58:35.864 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:35 smithi094 ceph-mon[36708]: pgmap v487: 129 pgs: 129 active+clean; 816 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.9 KiB/s wr, 7 op/s 2024-06-10T00:58:35.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:35 smithi037 ceph-mon[32192]: pgmap v487: 129 pgs: 129 active+clean; 816 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.9 KiB/s wr, 7 op/s 2024-06-10T00:58:35.995 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:58:35.996 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:58:36.029 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:36.029 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:36.029 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 11 -lt 100 ']' 2024-06-10T00:58:36.029 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 11 + 1 2024-06-10T00:58:36.509 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=12 2024-06-10T00:58:36.509 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:58:36.799 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:36.799 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:36.800 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:58:36.800 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:58:36.801 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:58:36.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.979+0000 7f76dfbca700 1 -- 172.21.15.37:0/1383863940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76d81073f0 msgr2=0x7f76d81077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:36.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.979+0000 7f76dfbca700 1 --2- 172.21.15.37:0/1383863940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76d81073f0 0x7f76d81077d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f76d4009990 tx=0x7f76d4009ca0 comp rx=0 tx=0).stop 2024-06-10T00:58:36.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.979+0000 7f76dfbca700 1 -- 172.21.15.37:0/1383863940 shutdown_connections 2024-06-10T00:58:36.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.979+0000 7f76dfbca700 1 --2- 172.21.15.37:0/1383863940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76d8101d40 0x7f76d810e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:36.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.979+0000 7f76dfbca700 1 --2- 172.21.15.37:0/1383863940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76d81073f0 0x7f76d81077d0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:36.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.979+0000 7f76dfbca700 1 --2- 172.21.15.37:0/1383863940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76d8101380 0x7f76d8101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:36.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.979+0000 7f76dfbca700 1 -- 172.21.15.37:0/1383863940 >> 172.21.15.37:0/1383863940 conn(0x7f76d8078300 msgr2=0x7f76d8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:36.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.979+0000 7f76dfbca700 1 -- 172.21.15.37:0/1383863940 shutdown_connections 2024-06-10T00:58:36.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.979+0000 7f76dfbca700 1 -- 172.21.15.37:0/1383863940 wait complete. 2024-06-10T00:58:36.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.980+0000 7f76dfbca700 1 Processor -- start 2024-06-10T00:58:36.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.980+0000 7f76dfbca700 1 -- start start 2024-06-10T00:58:36.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.980+0000 7f76dfbca700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76d8101380 0x7f76d8103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:36.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.980+0000 7f76dfbca700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76d8101d40 0x7f76d81035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:36.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.980+0000 7f76dfbca700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76d81073f0 0x7f76d8103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:36.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.980+0000 7f76dfbca700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f76d81116b0 con 0x7f76d8101380 2024-06-10T00:58:36.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.980+0000 7f76dfbca700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f76d8111530 con 0x7f76d8101d40 2024-06-10T00:58:36.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.980+0000 7f76dfbca700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f76d8111830 con 0x7f76d81073f0 2024-06-10T00:58:36.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.981+0000 7f76dd966700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76d8101380 0x7f76d8103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:36.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.981+0000 7f76dd966700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76d8101380 0x7f76d8103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42822/0 (socket says 172.21.15.37:42822) 2024-06-10T00:58:36.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.981+0000 7f76dd966700 1 -- 172.21.15.37:0/4275580174 learned_addr learned my addr 172.21.15.37:0/4275580174 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:36.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.981+0000 7f76de167700 1 --2- 172.21.15.37:0/4275580174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76d81073f0 0x7f76d8103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:36.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.981+0000 7f76dd165700 1 --2- 172.21.15.37:0/4275580174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76d8101d40 0x7f76d81035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:36.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.981+0000 7f76dd966700 1 -- 172.21.15.37:0/4275580174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76d81073f0 msgr2=0x7f76d8103ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:36.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.981+0000 7f76dd966700 1 --2- 172.21.15.37:0/4275580174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76d81073f0 0x7f76d8103ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:36.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.981+0000 7f76dd966700 1 -- 172.21.15.37:0/4275580174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76d8101d40 msgr2=0x7f76d81035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:36.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.981+0000 7f76dd966700 1 --2- 172.21.15.37:0/4275580174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76d8101d40 0x7f76d81035a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:36.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.981+0000 7f76dd966700 1 -- 172.21.15.37:0/4275580174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d81a71d0 con 0x7f76d8101380 2024-06-10T00:58:36.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.981+0000 7f76dd966700 1 --2- 172.21.15.37:0/4275580174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76d8101380 0x7f76d8103060 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f76d4000c00 tx=0x7f76d400ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:36.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.982+0000 7f76ceffd700 1 -- 172.21.15.37:0/4275580174 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d4027070 con 0x7f76d8101380 2024-06-10T00:58:36.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.982+0000 7f76ceffd700 1 -- 172.21.15.37:0/4275580174 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76d4003d70 con 0x7f76d8101380 2024-06-10T00:58:36.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.982+0000 7f76dfbca700 1 -- 172.21.15.37:0/4275580174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d81a7460 con 0x7f76d8101380 2024-06-10T00:58:36.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.982+0000 7f76ceffd700 1 -- 172.21.15.37:0/4275580174 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d400f460 con 0x7f76d8101380 2024-06-10T00:58:36.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.982+0000 7f76dfbca700 1 -- 172.21.15.37:0/4275580174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d81a78b0 con 0x7f76d8101380 2024-06-10T00:58:36.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.984+0000 7f76ceffd700 1 -- 172.21.15.37:0/4275580174 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f76d402a070 con 0x7f76d8101380 2024-06-10T00:58:36.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.985+0000 7f76ceffd700 1 --2- 172.21.15.37:0/4275580174 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f76c40a2c20 0x7f76c40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:36.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.985+0000 7f76dd165700 1 --2- 172.21.15.37:0/4275580174 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f76c40a2c20 0x7f76c40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:36.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.985+0000 7f76ceffd700 1 -- 172.21.15.37:0/4275580174 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f76d40d03c0 con 0x7f76d8101380 2024-06-10T00:58:36.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:36.986+0000 7f76dd165700 1 --2- 172.21.15.37:0/4275580174 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f76c40a2c20 0x7f76c40a50e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f76d8078900 tx=0x7f76c8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:37.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:37.038+0000 7f76dfbca700 1 -- 172.21.15.37:0/4275580174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76d8069180 con 0x7f76d8101380 2024-06-10T00:58:37.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:37.043+0000 7f76ceffd700 1 -- 172.21.15.37:0/4275580174 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f76d405e850 con 0x7f76d8101380 2024-06-10T00:58:37.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:37.311+0000 7f76dfbca700 1 -- 172.21.15.37:0/4275580174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76d81a7b10 con 0x7f76d8101380 2024-06-10T00:58:37.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:37.312+0000 7f76ceffd700 1 -- 172.21.15.37:0/4275580174 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11867 (secure 0 0 0) 0x7f76d4013070 con 0x7f76d8101380 2024-06-10T00:58:37.313 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:58:37.313 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T00:56:30.291171+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24451,24458,24472]},"id":1}]} 2024-06-10T00:58:37.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:37.314+0000 7f76dfbca700 1 -- 172.21.15.37:0/4275580174 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f76c40a2c20 msgr2=0x7f76c40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:37.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:37.314+0000 7f76dfbca700 1 --2- 172.21.15.37:0/4275580174 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f76c40a2c20 0x7f76c40a50e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f76d8078900 tx=0x7f76c8009260 comp rx=0 tx=0).stop 2024-06-10T00:58:37.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:37.314+0000 7f76dfbca700 1 -- 172.21.15.37:0/4275580174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76d8101380 msgr2=0x7f76d8103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:37.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:37.314+0000 7f76dfbca700 1 --2- 172.21.15.37:0/4275580174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76d8101380 0x7f76d8103060 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f76d4000c00 tx=0x7f76d400ba90 comp rx=0 tx=0).stop 2024-06-10T00:58:37.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:37.314+0000 7f76dfbca700 1 -- 172.21.15.37:0/4275580174 shutdown_connections 2024-06-10T00:58:37.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:37.314+0000 7f76dfbca700 1 --2- 172.21.15.37:0/4275580174 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f76c40a2c20 0x7f76c40a50e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:37.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:37.314+0000 7f76dfbca700 1 --2- 172.21.15.37:0/4275580174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76d81073f0 0x7f76d8103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:37.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:37.314+0000 7f76dfbca700 1 --2- 172.21.15.37:0/4275580174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76d8101380 0x7f76d8103060 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:37.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:37.314+0000 7f76dfbca700 1 --2- 172.21.15.37:0/4275580174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76d8101d40 0x7f76d81035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:37.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:37.314+0000 7f76dfbca700 1 -- 172.21.15.37:0/4275580174 >> 172.21.15.37:0/4275580174 conn(0x7f76d8078300 msgr2=0x7f76d8100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:37.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:37.315+0000 7f76dfbca700 1 -- 172.21.15.37:0/4275580174 shutdown_connections 2024-06-10T00:58:37.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:58:37.315+0000 7f76dfbca700 1 -- 172.21.15.37:0/4275580174 wait complete. 2024-06-10T00:58:37.316 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-06-10T00:58:38.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:37 smithi037 ceph-mon[32192]: pgmap v488: 129 pgs: 129 active+clean; 858 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-06-10T00:58:38.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:37 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/4275580174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:58:38.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:37 smithi167 ceph-mon[36917]: pgmap v488: 129 pgs: 129 active+clean; 858 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-06-10T00:58:38.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:37 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/4275580174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:58:38.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:37 smithi094 ceph-mon[36708]: pgmap v488: 129 pgs: 129 active+clean; 858 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-06-10T00:58:38.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:37 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/4275580174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:58:39.139 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:39.139 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:40.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:39 smithi037 ceph-mon[32192]: pgmap v489: 129 pgs: 129 active+clean; 859 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.4 KiB/s wr, 13 op/s 2024-06-10T00:58:40.161 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:58:40.161 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:58:40.192 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:40.193 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:40.193 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 12 -lt 100 ']' 2024-06-10T00:58:40.193 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 12 + 1 2024-06-10T00:58:40.229 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=13 2024-06-10T00:58:40.229 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:58:40.257 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:40.257 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:40.259 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:58:40.259 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:58:40.259 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:58:40.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:39 smithi167 ceph-mon[36917]: pgmap v489: 129 pgs: 129 active+clean; 859 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.4 KiB/s wr, 13 op/s 2024-06-10T00:58:40.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:39 smithi094 ceph-mon[36708]: pgmap v489: 129 pgs: 129 active+clean; 859 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.4 KiB/s wr, 13 op/s 2024-06-10T00:58:42.079 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:41 smithi037 ceph-mon[32192]: pgmap v490: 129 pgs: 129 active+clean; 859 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.2 KiB/s wr, 11 op/s 2024-06-10T00:58:42.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:41 smithi167 ceph-mon[36917]: pgmap v490: 129 pgs: 129 active+clean; 859 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.2 KiB/s wr, 11 op/s 2024-06-10T00:58:42.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:41 smithi094 ceph-mon[36708]: pgmap v490: 129 pgs: 129 active+clean; 859 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.2 KiB/s wr, 11 op/s 2024-06-10T00:58:42.630 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:42.630 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:43.502 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:58:43.503 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:58:43.560 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:43.560 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:43.560 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 13 -lt 100 ']' 2024-06-10T00:58:43.561 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 13 + 1 2024-06-10T00:58:43.759 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=14 2024-06-10T00:58:43.759 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:58:43.833 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:43.833 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:43.834 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:58:43.834 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:58:43.835 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:58:43.862 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T00:58:44.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:43 smithi037 ceph-mon[32192]: pgmap v491: 129 pgs: 129 active+clean; 886 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-06-10T00:58:44.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:43 smithi167 ceph-mon[36917]: pgmap v491: 129 pgs: 129 active+clean; 886 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-06-10T00:58:44.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:43 smithi094 ceph-mon[36708]: pgmap v491: 129 pgs: 129 active+clean; 886 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-06-10T00:58:45.672 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:45.672 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:46.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:45 smithi037 ceph-mon[32192]: pgmap v492: 129 pgs: 129 active+clean; 886 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.3 KiB/s wr, 12 op/s 2024-06-10T00:58:46.235 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:58:46.235 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:58:46.283 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:46.283 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:46.283 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 14 -lt 100 ']' 2024-06-10T00:58:46.284 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 14 + 1 2024-06-10T00:58:46.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:45 smithi167 ceph-mon[36917]: pgmap v492: 129 pgs: 129 active+clean; 886 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.3 KiB/s wr, 12 op/s 2024-06-10T00:58:46.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:45 smithi094 ceph-mon[36708]: pgmap v492: 129 pgs: 129 active+clean; 886 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.3 KiB/s wr, 12 op/s 2024-06-10T00:58:46.401 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=15 2024-06-10T00:58:46.401 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:58:46.427 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:46.428 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:46.429 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:58:46.429 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:58:46.429 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:58:47.846 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:47.846 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:47.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:47 smithi037 ceph-mon[32192]: pgmap v493: 129 pgs: 129 active+clean; 906 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 KiB/s wr, 13 op/s 2024-06-10T00:58:48.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:47 smithi167 ceph-mon[36917]: pgmap v493: 129 pgs: 129 active+clean; 906 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 KiB/s wr, 13 op/s 2024-06-10T00:58:48.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:47 smithi094 ceph-mon[36708]: pgmap v493: 129 pgs: 129 active+clean; 906 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 KiB/s wr, 13 op/s 2024-06-10T00:58:48.635 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:58:48.636 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:58:48.678 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:48.678 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:48.678 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 15 -lt 100 ']' 2024-06-10T00:58:48.679 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 15 + 1 2024-06-10T00:58:48.945 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=16 2024-06-10T00:58:48.945 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:58:49.051 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:49.052 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:49.053 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:58:49.053 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:58:49.053 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:58:50.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:49 smithi037 ceph-mon[32192]: pgmap v494: 129 pgs: 129 active+clean; 906 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.8 KiB/s wr, 11 op/s 2024-06-10T00:58:50.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:49 smithi167 ceph-mon[36917]: pgmap v494: 129 pgs: 129 active+clean; 906 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.8 KiB/s wr, 11 op/s 2024-06-10T00:58:50.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:49 smithi094 ceph-mon[36708]: pgmap v494: 129 pgs: 129 active+clean; 906 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.8 KiB/s wr, 11 op/s 2024-06-10T00:58:50.554 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:50.554 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:50.926 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:58:50.927 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:58:50.983 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:50.983 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:50.983 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 16 -lt 100 ']' 2024-06-10T00:58:50.983 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 16 + 1 2024-06-10T00:58:51.202 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=17 2024-06-10T00:58:51.202 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:58:51.255 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:51.256 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:51.257 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:58:51.257 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:58:51.257 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:58:52.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:51 smithi037 ceph-mon[32192]: pgmap v495: 129 pgs: 129 active+clean; 906 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-06-10T00:58:52.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:51 smithi167 ceph-mon[36917]: pgmap v495: 129 pgs: 129 active+clean; 906 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-06-10T00:58:52.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:51 smithi094 ceph-mon[36708]: pgmap v495: 129 pgs: 129 active+clean; 906 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-06-10T00:58:52.387 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:52.387 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:52.634 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:58:52.635 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:58:52.658 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:52.658 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:52.659 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 17 -lt 100 ']' 2024-06-10T00:58:52.659 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 17 + 1 2024-06-10T00:58:52.726 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=18 2024-06-10T00:58:52.726 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:58:52.769 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:52.770 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:52.771 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:58:52.771 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:58:52.771 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:58:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:53.936+0000 7f14135fb640 1 -- 172.21.15.37:0/1821815913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f140c104e20 msgr2=0x7f140c1052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:53.936+0000 7f14135fb640 1 --2- 172.21.15.37:0/1821815913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f140c104e20 0x7f140c1052a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009900 tx=0x7f13fc02a410 comp rx=0 tx=0).stop 2024-06-10T00:58:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:53.936+0000 7f14135fb640 1 -- 172.21.15.37:0/1821815913 shutdown_connections 2024-06-10T00:58:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:53.936+0000 7f14135fb640 1 --2- 172.21.15.37:0/1821815913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f140c1057e0 0x7f140c10c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:53.936+0000 7f14135fb640 1 --2- 172.21.15.37:0/1821815913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f140c104e20 0x7f140c1052a0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:53.936+0000 7f14135fb640 1 --2- 172.21.15.37:0/1821815913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f140c103c30 0x7f140c104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:53.936+0000 7f14135fb640 1 -- 172.21.15.37:0/1821815913 >> 172.21.15.37:0/1821815913 conn(0x7f140c0ff1c0 msgr2=0x7f140c1015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:53.936+0000 7f14135fb640 1 -- 172.21.15.37:0/1821815913 shutdown_connections 2024-06-10T00:58:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:53.936+0000 7f14135fb640 1 -- 172.21.15.37:0/1821815913 wait complete. 2024-06-10T00:58:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.049+0000 7f14135fb640 1 Processor -- start 2024-06-10T00:58:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.049+0000 7f14135fb640 1 -- start start 2024-06-10T00:58:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.049+0000 7f14135fb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f140c103c30 0x7f140c199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.049+0000 7f14135fb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f140c104e20 0x7f140c19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.049+0000 7f14135fb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f140c1057e0 0x7f140c1a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.049+0000 7f14135fb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f140c10f8b0 con 0x7f140c1057e0 2024-06-10T00:58:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.049+0000 7f14135fb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f140c10f730 con 0x7f140c103c30 2024-06-10T00:58:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.049+0000 7f14135fb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f140c10fa30 con 0x7f140c104e20 2024-06-10T00:58:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.049+0000 7f1411b98640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f140c1057e0 0x7f140c1a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.049+0000 7f1411b98640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f140c1057e0 0x7f140c1a1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57446/0 (socket says 172.21.15.37:57446) 2024-06-10T00:58:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.049+0000 7f1411b98640 1 -- 172.21.15.37:0/834880849 learned_addr learned my addr 172.21.15.37:0/834880849 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.049+0000 7f1410b96640 1 --2- 172.21.15.37:0/834880849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f140c104e20 0x7f140c19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.049+0000 7f1411397640 1 --2- 172.21.15.37:0/834880849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f140c103c30 0x7f140c199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.050+0000 7f1411b98640 1 -- 172.21.15.37:0/834880849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f140c104e20 msgr2=0x7f140c19a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.050+0000 7f1411b98640 1 --2- 172.21.15.37:0/834880849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f140c104e20 0x7f140c19a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.050+0000 7f1411b98640 1 -- 172.21.15.37:0/834880849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f140c103c30 msgr2=0x7f140c199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.050+0000 7f1411b98640 1 --2- 172.21.15.37:0/834880849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f140c103c30 0x7f140c199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.050+0000 7f1411b98640 1 -- 172.21.15.37:0/834880849 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f140c1a1a80 con 0x7f140c1057e0 2024-06-10T00:58:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.050+0000 7f1411b98640 1 --2- 172.21.15.37:0/834880849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f140c1057e0 0x7f140c1a1410 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f140400eb10 tx=0x7f140400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.069+0000 7f14027fc640 1 -- 172.21.15.37:0/834880849 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1404004b20 con 0x7f140c1057e0 2024-06-10T00:58:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.069+0000 7f14027fc640 1 -- 172.21.15.37:0/834880849 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1404004cc0 con 0x7f140c1057e0 2024-06-10T00:58:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.069+0000 7f14027fc640 1 -- 172.21.15.37:0/834880849 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1404010760 con 0x7f140c1057e0 2024-06-10T00:58:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.069+0000 7f14135fb640 1 -- 172.21.15.37:0/834880849 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f140c1a1d70 con 0x7f140c1057e0 2024-06-10T00:58:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.069+0000 7f14135fb640 1 -- 172.21.15.37:0/834880849 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f140c1a2230 con 0x7f140c1057e0 2024-06-10T00:58:54.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.074+0000 7f14027fc640 1 -- 172.21.15.37:0/834880849 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1404005020 con 0x7f140c1057e0 2024-06-10T00:58:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.075+0000 7f14027fc640 1 --2- 172.21.15.37:0/834880849 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f13f80a2da0 0x7f13f80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.075+0000 7f14027fc640 1 -- 172.21.15.37:0/834880849 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f14040cf9d0 con 0x7f140c1057e0 2024-06-10T00:58:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.075+0000 7f1411397640 1 --2- 172.21.15.37:0/834880849 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f13f80a2da0 0x7f13f80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:54.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.075+0000 7f1411397640 1 --2- 172.21.15.37:0/834880849 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f13f80a2da0 0x7f13f80a5260 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f14080095f0 tx=0x7f1408005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:54.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:53 smithi037 ceph-mon[32192]: pgmap v496: 129 pgs: 129 active+clean; 928 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-06-10T00:58:54.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:53 smithi167 ceph-mon[36917]: pgmap v496: 129 pgs: 129 active+clean; 928 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-06-10T00:58:54.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:53 smithi094 ceph-mon[36708]: pgmap v496: 129 pgs: 129 active+clean; 928 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-06-10T00:58:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.508+0000 7f14163e0740 1 Processor -- start 2024-06-10T00:58:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.508+0000 7f14163e0740 1 -- start start 2024-06-10T00:58:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.508+0000 7f14163e0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd810d2010 0x55cd810d2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.508+0000 7f14163e0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd810d2a00 0x55cd810d7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.508+0000 7f14163e0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd810d7b70 0x55cd810da000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.508+0000 7f14163e0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cd810354f0 con 0x55cd810d7b70 2024-06-10T00:58:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.508+0000 7f14163e0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cd80fd3870 con 0x55cd810d2a00 2024-06-10T00:58:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.508+0000 7f14163e0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cd810314b0 con 0x55cd810d2010 2024-06-10T00:58:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.508+0000 7f1411b98640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd810d7b70 0x55cd810da000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.508+0000 7f1411b98640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd810d7b70 0x55cd810da000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57450/0 (socket says 172.21.15.37:57450) 2024-06-10T00:58:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.508+0000 7f1411b98640 1 -- 172.21.15.37:0/1045551784 learned_addr learned my addr 172.21.15.37:0/1045551784 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.508+0000 7f1411397640 1 --2- 172.21.15.37:0/1045551784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd810d2010 0x55cd810d2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.508+0000 7f1410b96640 1 --2- 172.21.15.37:0/1045551784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd810d2a00 0x55cd810d7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.508+0000 7f1411b98640 1 -- 172.21.15.37:0/1045551784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd810d2010 msgr2=0x55cd810d2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.508+0000 7f1411b98640 1 --2- 172.21.15.37:0/1045551784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd810d2010 0x55cd810d2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.508+0000 7f1411b98640 1 -- 172.21.15.37:0/1045551784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd810d2a00 msgr2=0x55cd810d7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.508+0000 7f1411b98640 1 --2- 172.21.15.37:0/1045551784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd810d2a00 0x55cd810d7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.508+0000 7f1411b98640 1 -- 172.21.15.37:0/1045551784 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd80fef9b0 con 0x55cd810d7b70 2024-06-10T00:58:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.508+0000 7f1410b96640 1 --2- 172.21.15.37:0/1045551784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd810d2a00 0x55cd810d7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:58:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.509+0000 7f1411b98640 1 --2- 172.21.15.37:0/1045551784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd810d7b70 0x55cd810da000 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f14040982a0 tx=0x7f140400efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e6b3af44a583ee server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.509+0000 7f13f77fe640 1 -- 172.21.15.37:0/1045551784 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140409e070 con 0x55cd810d7b70 2024-06-10T00:58:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.509+0000 7f13f77fe640 1 -- 172.21.15.37:0/1045551784 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f140409be40 con 0x55cd810d7b70 2024-06-10T00:58:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.509+0000 7f13f77fe640 1 -- 172.21.15.37:0/1045551784 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14040a4620 con 0x55cd810d7b70 2024-06-10T00:58:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.509+0000 7f14163e0740 1 -- 172.21.15.37:0/1045551784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd810d7b70 msgr2=0x55cd810da000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.509+0000 7f14163e0740 1 --2- 172.21.15.37:0/1045551784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd810d7b70 0x55cd810da000 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f14040982a0 tx=0x7f140400efd0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.510+0000 7f14163e0740 1 -- 172.21.15.37:0/1045551784 shutdown_connections 2024-06-10T00:58:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.510+0000 7f14163e0740 1 --2- 172.21.15.37:0/1045551784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd810d7b70 0x55cd810da000 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.510+0000 7f14163e0740 1 --2- 172.21.15.37:0/1045551784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd810d2a00 0x55cd810d7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.510+0000 7f14163e0740 1 --2- 172.21.15.37:0/1045551784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd810d2010 0x55cd810d2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.510+0000 7f14163e0740 1 -- 172.21.15.37:0/1045551784 >> 172.21.15.37:0/1045551784 conn(0x55cd810cd680 msgr2=0x55cd810cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.510+0000 7f14163e0740 1 -- 172.21.15.37:0/1045551784 shutdown_connections 2024-06-10T00:58:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.510+0000 7f14163e0740 1 -- 172.21.15.37:0/1045551784 wait complete. 2024-06-10T00:58:54.569 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:54.570 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.664+0000 7f14163e0740 1 -- start start 2024-06-10T00:58:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.743+0000 7f14163e0740 1 Processor -- start 2024-06-10T00:58:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.744+0000 7f14163e0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd810d2010 0x55cd810dc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.744+0000 7f14163e0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd810d2a00 0x55cd810dc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.744+0000 7f14163e0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd810d7b70 0x55cd810e37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:54.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.744+0000 7f14163e0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cd810dab20 con 0x55cd810d2010 2024-06-10T00:58:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.744+0000 7f14163e0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cd80fbeae0 con 0x55cd810d7b70 2024-06-10T00:58:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.744+0000 7f14163e0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cd810daca0 con 0x55cd810d2a00 2024-06-10T00:58:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.744+0000 7f1411397640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd810d2010 0x55cd810dc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.744+0000 7f1411397640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd810d2010 0x55cd810dc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57456/0 (socket says 172.21.15.37:57456) 2024-06-10T00:58:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.744+0000 7f1411397640 1 -- 172.21.15.37:0/712265633 learned_addr learned my addr 172.21.15.37:0/712265633 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.744+0000 7f1410b96640 1 --2- 172.21.15.37:0/712265633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd810d2a00 0x55cd810dc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.744+0000 7f1411b98640 1 --2- 172.21.15.37:0/712265633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd810d7b70 0x55cd810e37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.745+0000 7f1411397640 1 -- 172.21.15.37:0/712265633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd810d2a00 msgr2=0x55cd810dc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.745+0000 7f1411397640 1 --2- 172.21.15.37:0/712265633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd810d2a00 0x55cd810dc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.745+0000 7f1411397640 1 -- 172.21.15.37:0/712265633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd810d7b70 msgr2=0x55cd810e37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.745+0000 7f1411397640 1 --2- 172.21.15.37:0/712265633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd810d7b70 0x55cd810e37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.745+0000 7f1411397640 1 -- 172.21.15.37:0/712265633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd80d49210 con 0x55cd810d2010 2024-06-10T00:58:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.745+0000 7f1411b98640 1 --2- 172.21.15.37:0/712265633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd810d7b70 0x55cd810e37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:58:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.745+0000 7f1411397640 1 --2- 172.21.15.37:0/712265633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd810d2010 0x55cd810dc220 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1408012c40 tx=0x7f1408004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9174eb30e1b420 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.745+0000 7f13f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:58:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.745+0000 7f13f4ff9640 1 -- 172.21.15.37:0/712265633 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1408011c20 con 0x55cd810d2010 2024-06-10T00:58:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.745+0000 7f13f4ff9640 1 -- 172.21.15.37:0/712265633 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f1408011dc0 con 0x55cd810d2010 2024-06-10T00:58:54.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.748+0000 7f14163e0740 10 client.0 _ll_register_callbacks cb 0x55cd80ff79b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:58:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.753+0000 7f14163e0740 1 -- 172.21.15.37:0/712265633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f140409be40 con 0x55cd810d2010 2024-06-10T00:58:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.753+0000 7f13f4ff9640 1 -- 172.21.15.37:0/712265633 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1408029430 con 0x55cd810d2010 2024-06-10T00:58:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.753+0000 7f14163e0740 10 client.15843 fetch_fsmap learned FSMap version 27 2024-06-10T00:58:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.753+0000 7f14163e0740 1 -- 172.21.15.37:0/712265633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55cd80e15620 con 0x55cd810d2010 2024-06-10T00:58:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.754+0000 7f13f4ff9640 1 -- 172.21.15.37:0/712265633 <== mon.0 v2:172.21.15.37:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f14080295d0 con 0x55cd810d2010 2024-06-10T00:58:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.754+0000 7f14163e0740 10 client.15843 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:58:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.754+0000 7f14163e0740 10 client.15843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T00:58:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.754+0000 7f14163e0740 20 client.15843 populate_metadata read hostname 'smithi037' 2024-06-10T00:58:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.754+0000 7f14163e0740 1 --2- 172.21.15.37:0/712265633 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55cd810e7ef0 0x55cd810ea380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.754+0000 7f14163e0740 4 client.15843 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:58:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.754+0000 7f14163e0740 1 -- 172.21.15.37:0/712265633 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd81048f10 con 0x55cd810e7ef0 2024-06-10T00:58:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.754+0000 7f1410b96640 1 --2- 172.21.15.37:0/712265633 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55cd810e7ef0 0x55cd810ea380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.755+0000 7f13f4ff9640 10 client.15843 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.755+0000 7f1410b96640 1 --2- 172.21.15.37:0/712265633 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55cd810e7ef0 0x55cd810ea380 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b33e7fbccb91b87 server_cookie=15d4891d919a1312 in_seq=0 out_seq=0 2024-06-10T00:58:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.756+0000 7f13f4ff9640 1 -- 172.21.15.37:0/712265633 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cd81048f10 con 0x55cd810e7ef0 2024-06-10T00:58:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.756+0000 7f13f4ff9640 10 client.15843 handle_command_reply: tid=0 2024-06-10T00:58:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.756+0000 7f14163e0740 1 client.15843 shutdown 2024-06-10T00:58:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.756+0000 7f14163e0740 1 -- 172.21.15.37:0/712265633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd810d2010 msgr2=0x55cd810dc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.756+0000 7f14163e0740 1 --2- 172.21.15.37:0/712265633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd810d2010 0x55cd810dc220 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1408012c40 tx=0x7f1408004730 comp rx=0 tx=0).stop 2024-06-10T00:58:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.756+0000 7f14163e0740 1 -- 172.21.15.37:0/712265633 shutdown_connections 2024-06-10T00:58:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.756+0000 7f14163e0740 1 --2- 172.21.15.37:0/712265633 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55cd810e7ef0 0x55cd810ea380 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.756+0000 7f14163e0740 1 --2- 172.21.15.37:0/712265633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd810d7b70 0x55cd810e37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.756+0000 7f14163e0740 1 --2- 172.21.15.37:0/712265633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd810d2a00 0x55cd810dc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.756+0000 7f13f4ff9640 0 client.15843 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.756+0000 7f14163e0740 1 --2- 172.21.15.37:0/712265633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd810d2010 0x55cd810dc220 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.756+0000 7f14163e0740 1 -- 172.21.15.37:0/712265633 >> 172.21.15.37:0/712265633 conn(0x55cd810cd680 msgr2=0x55cd810cc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.756+0000 7f14163e0740 1 -- 172.21.15.37:0/712265633 shutdown_connections 2024-06-10T00:58:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.756+0000 7f14163e0740 1 -- 172.21.15.37:0/712265633 wait complete. 2024-06-10T00:58:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.756+0000 7f14163e0740 20 client.15843 trim_cache size 0 max 16384 2024-06-10T00:58:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.772+0000 7f14163e0740 1 Processor -- start 2024-06-10T00:58:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.772+0000 7f14163e0740 1 -- start start 2024-06-10T00:58:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.772+0000 7f14163e0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd810d2010 0x55cd810db840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.772+0000 7f14163e0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd810d2a00 0x55cd810e3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.772+0000 7f14163e0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd810d7b70 0x55cd810dbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.772+0000 7f14163e0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f140c1af130 con 0x55cd810d2010 2024-06-10T00:58:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.772+0000 7f14163e0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f140c1aca00 con 0x55cd810d2a00 2024-06-10T00:58:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.772+0000 7f14163e0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f140c1aa850 con 0x55cd810d7b70 2024-06-10T00:58:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.772+0000 7f1411397640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd810d2010 0x55cd810db840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.772+0000 7f1411397640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd810d2010 0x55cd810db840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57462/0 (socket says 172.21.15.37:57462) 2024-06-10T00:58:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.772+0000 7f1411397640 1 -- 172.21.15.37:0/764093955 learned_addr learned my addr 172.21.15.37:0/764093955 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.772+0000 7f1410b96640 1 --2- 172.21.15.37:0/764093955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd810d2a00 0x55cd810e3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.773+0000 7f1411b98640 1 --2- 172.21.15.37:0/764093955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd810d7b70 0x55cd810dbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.773+0000 7f1411397640 1 -- 172.21.15.37:0/764093955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd810d7b70 msgr2=0x55cd810dbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.773+0000 7f1411397640 1 --2- 172.21.15.37:0/764093955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd810d7b70 0x55cd810dbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.773+0000 7f1411397640 1 -- 172.21.15.37:0/764093955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd810d2a00 msgr2=0x55cd810e3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.773+0000 7f1411397640 1 --2- 172.21.15.37:0/764093955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd810d2a00 0x55cd810e3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.773+0000 7f1411397640 1 -- 172.21.15.37:0/764093955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd810d2de0 con 0x55cd810d2010 2024-06-10T00:58:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.773+0000 7f1411b98640 1 --2- 172.21.15.37:0/764093955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd810d7b70 0x55cd810dbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:58:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.773+0000 7f1411397640 1 --2- 172.21.15.37:0/764093955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd810d2010 0x55cd810db840 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1408011820 tx=0x7f1408022070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b93b2390d1ae7a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.773+0000 7f13f6ffd640 1 -- 172.21.15.37:0/764093955 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1408012c70 con 0x55cd810d2010 2024-06-10T00:58:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.773+0000 7f13f6ffd640 1 -- 172.21.15.37:0/764093955 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f140409be40 con 0x55cd810d2010 2024-06-10T00:58:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.773+0000 7f13f6ffd640 1 -- 172.21.15.37:0/764093955 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1408029dd0 con 0x55cd810d2010 2024-06-10T00:58:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.774+0000 7f14163e0740 1 -- 172.21.15.37:0/764093955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd810d2010 msgr2=0x55cd810db840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.774+0000 7f14163e0740 1 --2- 172.21.15.37:0/764093955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd810d2010 0x55cd810db840 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1408011820 tx=0x7f1408022070 comp rx=0 tx=0).stop 2024-06-10T00:58:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.774+0000 7f14163e0740 1 -- 172.21.15.37:0/764093955 shutdown_connections 2024-06-10T00:58:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.774+0000 7f14163e0740 1 --2- 172.21.15.37:0/764093955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd810d7b70 0x55cd810dbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.774+0000 7f14163e0740 1 --2- 172.21.15.37:0/764093955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd810d2a00 0x55cd810e3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.774+0000 7f14163e0740 1 --2- 172.21.15.37:0/764093955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd810d2010 0x55cd810db840 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.774+0000 7f14163e0740 1 -- 172.21.15.37:0/764093955 >> 172.21.15.37:0/764093955 conn(0x55cd810cd680 msgr2=0x55cd810d7250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.774+0000 7f14163e0740 1 -- 172.21.15.37:0/764093955 shutdown_connections 2024-06-10T00:58:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.774+0000 7f14163e0740 1 -- 172.21.15.37:0/764093955 wait complete. 2024-06-10T00:58:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.774+0000 7f14163e0740 1 -- start start 2024-06-10T00:58:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.775+0000 7f14163e0740 1 Processor -- start 2024-06-10T00:58:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.775+0000 7f14163e0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd810d2010 0x55cd810e0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.775+0000 7f14163e0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd810d2a00 0x55cd810e08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.775+0000 7f14163e0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd810d7b70 0x55cd81150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.775+0000 7f14163e0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f140c1aaec0 con 0x55cd810d7b70 2024-06-10T00:58:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.775+0000 7f14163e0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f140c1ac330 con 0x55cd810d2010 2024-06-10T00:58:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.775+0000 7f14163e0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f140c1ab590 con 0x55cd810d2a00 2024-06-10T00:58:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.775+0000 7f14163e0740 10 client.0 _ll_register_callbacks cb 0x7f1408011b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:58:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.775+0000 7f1411b98640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd810d7b70 0x55cd81150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.775+0000 7f1411b98640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd810d7b70 0x55cd81150c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57464/0 (socket says 172.21.15.37:57464) 2024-06-10T00:58:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.775+0000 7f1411b98640 1 -- 172.21.15.37:0/2646092920 learned_addr learned my addr 172.21.15.37:0/2646092920 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:58:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.775+0000 7f1411397640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd810d2010 0x55cd810e0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.775+0000 7f1410b96640 1 --2- 172.21.15.37:0/2646092920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd810d2a00 0x55cd810e08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.775+0000 7f1411b98640 1 -- 172.21.15.37:0/2646092920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd810d2a00 msgr2=0x55cd810e08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.775+0000 7f1411b98640 1 --2- 172.21.15.37:0/2646092920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd810d2a00 0x55cd810e08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.775+0000 7f1411b98640 1 -- 172.21.15.37:0/2646092920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd810d2010 msgr2=0x55cd810e0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:58:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.775+0000 7f1411b98640 1 --2- 172.21.15.37:0/2646092920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd810d2010 0x55cd810e0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.775+0000 7f1411b98640 1 -- 172.21.15.37:0/2646092920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd810e3980 con 0x55cd810d7b70 2024-06-10T00:58:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.776+0000 7f1411b98640 1 --2- 172.21.15.37:0/2646092920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd810d7b70 0x55cd81150c50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f140409bb40 tx=0x7f140409b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27965512defdbac0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:58:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.776+0000 7f13f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:58:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.776+0000 7f13f4ff9640 1 -- 172.21.15.37:0/2646092920 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140409e040 con 0x55cd810d7b70 2024-06-10T00:58:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.776+0000 7f13f4ff9640 1 -- 172.21.15.37:0/2646092920 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f14040a4620 con 0x55cd810d7b70 2024-06-10T00:58:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.776+0000 7f14163e0740 1 -- 172.21.15.37:0/2646092920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f140409be40 con 0x55cd810d7b70 2024-06-10T00:58:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.776+0000 7f13f4ff9640 1 -- 172.21.15.37:0/2646092920 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f14040a4ab0 con 0x55cd810d7b70 2024-06-10T00:58:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.776+0000 7f14163e0740 10 client.15849 fetch_fsmap learned FSMap version 27 2024-06-10T00:58:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.776+0000 7f14163e0740 1 -- 172.21.15.37:0/2646092920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cd811520e0 con 0x55cd810d7b70 2024-06-10T00:58:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.776+0000 7f13f4ff9640 1 -- 172.21.15.37:0/2646092920 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f140409be40 con 0x55cd810d7b70 2024-06-10T00:58:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.776+0000 7f13f4ff9640 1 -- 172.21.15.37:0/2646092920 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f140409b320 con 0x55cd810d7b70 2024-06-10T00:58:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.776+0000 7f14163e0740 10 client.15849 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:58:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.776+0000 7f14163e0740 10 client.15849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T00:58:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.776+0000 7f14163e0740 20 client.15849 populate_metadata read hostname 'smithi037' 2024-06-10T00:58:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.776+0000 7f14163e0740 1 --2- 172.21.15.37:0/2646092920 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55cd811525e0 0x55cd811549d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:58:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.776+0000 7f14163e0740 4 client.15849 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T00:58:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.776+0000 7f14163e0740 1 -- 172.21.15.37:0/2646092920 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cd81042c20 con 0x55cd811525e0 2024-06-10T00:58:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.777+0000 7f1411397640 1 --2- 172.21.15.37:0/2646092920 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55cd811525e0 0x55cd811549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:58:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.777+0000 7f1411397640 1 --2- 172.21.15.37:0/2646092920 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55cd811525e0 0x55cd811549d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ed97eb056b3a3e0 server_cookie=465176ba034ece7e in_seq=0 out_seq=0 2024-06-10T00:58:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.777+0000 7f13f4ff9640 10 client.15849 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.794+0000 7f13f4ff9640 1 -- 172.21.15.37:0/2646092920 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55cd81042c20 con 0x55cd811525e0 2024-06-10T00:58:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.794+0000 7f13f4ff9640 10 client.15849 handle_command_reply: tid=0 2024-06-10T00:58:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.794+0000 7f14163e0740 1 client.15849 shutdown 2024-06-10T00:58:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.794+0000 7f14163e0740 1 -- 172.21.15.37:0/2646092920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd810d7b70 msgr2=0x55cd81150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.794+0000 7f14163e0740 1 --2- 172.21.15.37:0/2646092920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd810d7b70 0x55cd81150c50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f140409bb40 tx=0x7f140409b820 comp rx=0 tx=0).stop 2024-06-10T00:58:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.794+0000 7f14163e0740 1 -- 172.21.15.37:0/2646092920 shutdown_connections 2024-06-10T00:58:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.794+0000 7f14163e0740 1 --2- 172.21.15.37:0/2646092920 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55cd811525e0 0x55cd811549d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.794+0000 7f14163e0740 1 --2- 172.21.15.37:0/2646092920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cd810d7b70 0x55cd81150c50 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.794+0000 7f14163e0740 1 --2- 172.21.15.37:0/2646092920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd810d2a00 0x55cd810e08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.794+0000 7f14163e0740 1 --2- 172.21.15.37:0/2646092920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cd810d2010 0x55cd810e0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.794+0000 7f13f4ff9640 0 client.15849 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:58:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.794+0000 7f14163e0740 1 -- 172.21.15.37:0/2646092920 >> 172.21.15.37:0/2646092920 conn(0x55cd810cd680 msgr2=0x55cd810dbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.795+0000 7f14163e0740 1 -- 172.21.15.37:0/2646092920 shutdown_connections 2024-06-10T00:58:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.795+0000 7f14163e0740 1 -- 172.21.15.37:0/2646092920 wait complete. 2024-06-10T00:58:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.795+0000 7f14163e0740 20 client.15849 trim_cache size 0 max 16384 2024-06-10T00:58:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T00:58:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T00:58:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T00:58:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.797+0000 7f14135fb640 1 -- 172.21.15.37:0/834880849 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f13f80a2da0 msgr2=0x7f13f80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.797+0000 7f14135fb640 1 --2- 172.21.15.37:0/834880849 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f13f80a2da0 0x7f13f80a5260 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f14080095f0 tx=0x7f1408005f90 comp rx=0 tx=0).stop 2024-06-10T00:58:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.797+0000 7f14135fb640 1 -- 172.21.15.37:0/834880849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f140c1057e0 msgr2=0x7f140c1a1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:58:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.797+0000 7f14135fb640 1 --2- 172.21.15.37:0/834880849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f140c1057e0 0x7f140c1a1410 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f140400eb10 tx=0x7f140400c2d0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.797+0000 7f14135fb640 1 -- 172.21.15.37:0/834880849 shutdown_connections 2024-06-10T00:58:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.797+0000 7f14135fb640 1 --2- 172.21.15.37:0/834880849 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f13f80a2da0 0x7f13f80a5260 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.797+0000 7f14135fb640 1 --2- 172.21.15.37:0/834880849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f140c1057e0 0x7f140c1a1410 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.797+0000 7f14135fb640 1 --2- 172.21.15.37:0/834880849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f140c104e20 0x7f140c19a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.797+0000 7f14135fb640 1 --2- 172.21.15.37:0/834880849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f140c103c30 0x7f140c199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:58:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.797+0000 7f14135fb640 1 -- 172.21.15.37:0/834880849 >> 172.21.15.37:0/834880849 conn(0x7f140c0ff1c0 msgr2=0x7f140c108120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:58:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.797+0000 7f14135fb640 1 -- 172.21.15.37:0/834880849 shutdown_connections 2024-06-10T00:58:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:58:54.797+0000 7f14135fb640 1 -- 172.21.15.37:0/834880849 wait complete. 2024-06-10T00:58:55.032 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:58:55.032 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:58:55.093 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:55.093 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:55.094 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 18 -lt 100 ']' 2024-06-10T00:58:55.094 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 18 + 1 2024-06-10T00:58:55.184 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=19 2024-06-10T00:58:55.184 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:58:55.228 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:55.228 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:55.229 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:58:55.229 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:58:55.229 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:58:56.302 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:55 smithi037 ceph-mon[32192]: pgmap v497: 129 pgs: 129 active+clean; 928 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-06-10T00:58:56.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:55 smithi167 ceph-mon[36917]: pgmap v497: 129 pgs: 129 active+clean; 928 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-06-10T00:58:56.353 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:56.353 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:56.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:55 smithi094 ceph-mon[36708]: pgmap v497: 129 pgs: 129 active+clean; 928 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-06-10T00:58:56.541 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:58:56.541 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:58:56.549 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:56.549 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:56.550 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 19 -lt 100 ']' 2024-06-10T00:58:56.550 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 19 + 1 2024-06-10T00:58:56.592 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=20 2024-06-10T00:58:56.592 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:58:56.659 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:56.659 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:56.660 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:58:56.661 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:58:56.661 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:58:58.094 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:58:58.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:57 smithi037 ceph-mon[32192]: pgmap v498: 129 pgs: 129 active+clean; 978 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 10 op/s 2024-06-10T00:58:58.153 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:58.153 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:58.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:57 smithi167 ceph-mon[36917]: pgmap v498: 129 pgs: 129 active+clean; 978 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 10 op/s 2024-06-10T00:58:58.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:57 smithi094 ceph-mon[36708]: pgmap v498: 129 pgs: 129 active+clean; 978 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 10 op/s 2024-06-10T00:58:58.417 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:58:58.417 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:58:58.482 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:58.482 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:58:58.482 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 20 -lt 100 ']' 2024-06-10T00:58:58.482 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 20 + 1 2024-06-10T00:58:58.649 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=21 2024-06-10T00:58:58.649 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:58:58.764 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:58:58.764 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:58:58.765 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:58:58.766 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:58:58.766 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:59:00.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:58:59 smithi037 ceph-mon[32192]: pgmap v499: 129 pgs: 129 active+clean; 979 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.6 KiB/s wr, 13 op/s 2024-06-10T00:59:00.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:58:59 smithi167 ceph-mon[36917]: pgmap v499: 129 pgs: 129 active+clean; 979 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.6 KiB/s wr, 13 op/s 2024-06-10T00:59:00.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:58:59 smithi094 ceph-mon[36708]: pgmap v499: 129 pgs: 129 active+clean; 979 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.6 KiB/s wr, 13 op/s 2024-06-10T00:59:00.711 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:00.711 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:01.100 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:59:01.100 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:59:01.172 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:01.173 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:01.173 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 21 -lt 100 ']' 2024-06-10T00:59:01.173 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 21 + 1 2024-06-10T00:59:01.268 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=22 2024-06-10T00:59:01.268 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:59:01.336 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:01.336 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:01.337 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:59:01.337 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:59:01.337 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:59:01.988 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:01 smithi037 ceph-mon[32192]: pgmap v500: 129 pgs: 129 active+clean; 979 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.5 KiB/s wr, 12 op/s 2024-06-10T00:59:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.309+0000 7fd08bda9700 1 -- 172.21.15.37:0/798494507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08410ef30 msgr2=0x7fd084111320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.309+0000 7fd08bda9700 1 --2- 172.21.15.37:0/798494507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08410ef30 0x7fd084111320 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00b0a0 tx=0x7fd07c00b3b0 comp rx=0 tx=0).stop 2024-06-10T00:59:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.309+0000 7fd08bda9700 1 -- 172.21.15.37:0/798494507 shutdown_connections 2024-06-10T00:59:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.309+0000 7fd08bda9700 1 --2- 172.21.15.37:0/798494507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08410ef30 0x7fd084111320 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.309+0000 7fd08bda9700 1 --2- 172.21.15.37:0/798494507 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd08406aa30 0x7fd08406ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.309+0000 7fd08bda9700 1 --2- 172.21.15.37:0/798494507 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd08406b350 0x7fd08410e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.309+0000 7fd08bda9700 1 -- 172.21.15.37:0/798494507 >> 172.21.15.37:0/798494507 conn(0x7fd084078300 msgr2=0x7fd084078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.309+0000 7fd08bda9700 1 -- 172.21.15.37:0/798494507 shutdown_connections 2024-06-10T00:59:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.310+0000 7fd08bda9700 1 -- 172.21.15.37:0/798494507 wait complete. 2024-06-10T00:59:02.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.310+0000 7fd08bda9700 1 Processor -- start 2024-06-10T00:59:02.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.310+0000 7fd08bda9700 1 -- start start 2024-06-10T00:59:02.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.311+0000 7fd08bda9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08406aa30 0x7fd08419ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:02.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.311+0000 7fd08bda9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd08406b350 0x7fd08419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:02.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.311+0000 7fd08bda9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd08410ef30 0x7fd08419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:02.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.311+0000 7fd08bda9700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd084114990 con 0x7fd08406aa30 2024-06-10T00:59:02.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.311+0000 7fd08bda9700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd084114810 con 0x7fd08410ef30 2024-06-10T00:59:02.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.311+0000 7fd08bda9700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd084114b10 con 0x7fd08406b350 2024-06-10T00:59:02.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.311+0000 7fd089b45700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08406aa30 0x7fd08419ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:02.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.311+0000 7fd089b45700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08406aa30 0x7fd08419ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36592/0 (socket says 172.21.15.37:36592) 2024-06-10T00:59:02.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.311+0000 7fd089b45700 1 -- 172.21.15.37:0/3172034989 learned_addr learned my addr 172.21.15.37:0/3172034989 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:02.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.311+0000 7fd089344700 1 --2- 172.21.15.37:0/3172034989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd08406b350 0x7fd08419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:02.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.311+0000 7fd08a346700 1 --2- 172.21.15.37:0/3172034989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd08410ef30 0x7fd08419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:02.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.311+0000 7fd089b45700 1 -- 172.21.15.37:0/3172034989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd08406b350 msgr2=0x7fd08419b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:02.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.311+0000 7fd089b45700 1 --2- 172.21.15.37:0/3172034989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd08406b350 0x7fd08419b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:02.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.311+0000 7fd089b45700 1 -- 172.21.15.37:0/3172034989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd08410ef30 msgr2=0x7fd08419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:02.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.311+0000 7fd089b45700 1 --2- 172.21.15.37:0/3172034989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd08410ef30 0x7fd08419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:02.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.311+0000 7fd089b45700 1 -- 172.21.15.37:0/3172034989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08419fdf0 con 0x7fd08406aa30 2024-06-10T00:59:02.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.311+0000 7fd08a346700 1 --2- 172.21.15.37:0/3172034989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd08410ef30 0x7fd08419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:59:02.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.312+0000 7fd089b45700 1 --2- 172.21.15.37:0/3172034989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08406aa30 0x7fd08419ade0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd08000d980 tx=0x7fd08000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:02.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.312+0000 7fd07affd700 1 -- 172.21.15.37:0/3172034989 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd080016070 con 0x7fd08406aa30 2024-06-10T00:59:02.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.312+0000 7fd08bda9700 1 -- 172.21.15.37:0/3172034989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0841a00e0 con 0x7fd08406aa30 2024-06-10T00:59:02.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.312+0000 7fd07affd700 1 -- 172.21.15.37:0/3172034989 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd080004d10 con 0x7fd08406aa30 2024-06-10T00:59:02.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.312+0000 7fd07affd700 1 -- 172.21.15.37:0/3172034989 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0800055f0 con 0x7fd08406aa30 2024-06-10T00:59:02.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.312+0000 7fd08bda9700 1 -- 172.21.15.37:0/3172034989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0841a7960 con 0x7fd08406aa30 2024-06-10T00:59:02.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.315+0000 7fd07affd700 1 -- 172.21.15.37:0/3172034989 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd080004750 con 0x7fd08406aa30 2024-06-10T00:59:02.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.316+0000 7fd07affd700 1 --2- 172.21.15.37:0/3172034989 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd0700a2c20 0x7fd0700a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:02.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.316+0000 7fd089344700 1 --2- 172.21.15.37:0/3172034989 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd0700a2c20 0x7fd0700a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:02.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.316+0000 7fd07affd700 1 -- 172.21.15.37:0/3172034989 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fd0800cf260 con 0x7fd08406aa30 2024-06-10T00:59:02.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.316+0000 7fd089344700 1 --2- 172.21.15.37:0/3172034989 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd0700a2c20 0x7fd0700a50e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd084102800 tx=0x7fd07400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:02.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.326+0000 7fd08bda9700 1 -- 172.21.15.37:0/3172034989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0841019a0 con 0x7fd08406aa30 2024-06-10T00:59:02.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.331+0000 7fd07affd700 1 -- 172.21.15.37:0/3172034989 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd08001f020 con 0x7fd08406aa30 2024-06-10T00:59:02.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:01 smithi167 ceph-mon[36917]: pgmap v500: 129 pgs: 129 active+clean; 979 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.5 KiB/s wr, 12 op/s 2024-06-10T00:59:02.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:01 smithi094 ceph-mon[36708]: pgmap v500: 129 pgs: 129 active+clean; 979 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.5 KiB/s wr, 12 op/s 2024-06-10T00:59:02.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.592+0000 7fd08bda9700 1 -- 172.21.15.37:0/3172034989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd08419c140 con 0x7fd08406aa30 2024-06-10T00:59:02.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.593+0000 7fd07affd700 1 -- 172.21.15.37:0/3172034989 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11867 (secure 0 0 0) 0x7fd08009bdd0 con 0x7fd08406aa30 2024-06-10T00:59:02.593 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:59:02.593 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T00:56:30.291171+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24451,24458,24472]},"id":1}]} 2024-06-10T00:59:02.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.595+0000 7fd08bda9700 1 -- 172.21.15.37:0/3172034989 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd0700a2c20 msgr2=0x7fd0700a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:02.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.595+0000 7fd08bda9700 1 --2- 172.21.15.37:0/3172034989 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd0700a2c20 0x7fd0700a50e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd084102800 tx=0x7fd07400b410 comp rx=0 tx=0).stop 2024-06-10T00:59:02.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.595+0000 7fd08bda9700 1 -- 172.21.15.37:0/3172034989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08406aa30 msgr2=0x7fd08419ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:02.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.595+0000 7fd08bda9700 1 --2- 172.21.15.37:0/3172034989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08406aa30 0x7fd08419ade0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd08000d980 tx=0x7fd08000dc90 comp rx=0 tx=0).stop 2024-06-10T00:59:02.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.595+0000 7fd08bda9700 1 -- 172.21.15.37:0/3172034989 shutdown_connections 2024-06-10T00:59:02.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.595+0000 7fd08bda9700 1 --2- 172.21.15.37:0/3172034989 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd0700a2c20 0x7fd0700a50e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:02.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.595+0000 7fd08bda9700 1 --2- 172.21.15.37:0/3172034989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd08410ef30 0x7fd08419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:02.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.595+0000 7fd08bda9700 1 --2- 172.21.15.37:0/3172034989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08406aa30 0x7fd08419ade0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:02.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.595+0000 7fd08bda9700 1 --2- 172.21.15.37:0/3172034989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd08406b350 0x7fd08419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:02.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.595+0000 7fd08bda9700 1 -- 172.21.15.37:0/3172034989 >> 172.21.15.37:0/3172034989 conn(0x7fd084078300 msgr2=0x7fd084110340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:02.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.595+0000 7fd08bda9700 1 -- 172.21.15.37:0/3172034989 shutdown_connections 2024-06-10T00:59:02.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:02.595+0000 7fd08bda9700 1 -- 172.21.15.37:0/3172034989 wait complete. 2024-06-10T00:59:02.596 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-06-10T00:59:03.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:02 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3172034989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:59:03.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:02 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3172034989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:59:03.368 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:03.368 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:03.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:02 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3172034989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:59:03.808 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:59:03.808 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:59:03.815 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:03.815 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:03.815 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 22 -lt 100 ']' 2024-06-10T00:59:03.815 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 22 + 1 2024-06-10T00:59:03.868 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=23 2024-06-10T00:59:03.868 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:59:03.920 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:03.920 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:03.921 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:59:03.921 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:59:03.921 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:59:04.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:03 smithi167 ceph-mon[36917]: pgmap v501: 129 pgs: 129 active+clean; 1014 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-06-10T00:59:04.343 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:03 smithi037 ceph-mon[32192]: pgmap v501: 129 pgs: 129 active+clean; 1014 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-06-10T00:59:04.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:03 smithi094 ceph-mon[36708]: pgmap v501: 129 pgs: 129 active+clean; 1014 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-06-10T00:59:05.258 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:05.259 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:05.392 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24472 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-06-10T00:59:05.392 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24472 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-06-10T00:59:06.122 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:59:06.122 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:59:06.197 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:06.198 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:06.198 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 23 -lt 100 ']' 2024-06-10T00:59:06.198 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 23 + 1 2024-06-10T00:59:06.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:05 smithi167 ceph-mon[36917]: pgmap v502: 129 pgs: 129 active+clean; 1014 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2024-06-10T00:59:06.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:05 smithi094 ceph-mon[36708]: pgmap v502: 129 pgs: 129 active+clean; 1014 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2024-06-10T00:59:06.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:05 smithi037 ceph-mon[32192]: pgmap v502: 129 pgs: 129 active+clean; 1014 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2024-06-10T00:59:06.407 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=24 2024-06-10T00:59:06.407 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:59:06.465 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:06.466 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:06.467 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:59:06.467 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:59:06.467 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:59:07.627 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:07.627 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:07.999 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:59:07.999 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:59:08.014 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:08.014 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:08.014 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 24 -lt 100 ']' 2024-06-10T00:59:08.014 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 24 + 1 2024-06-10T00:59:08.083 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=25 2024-06-10T00:59:08.083 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:59:08.142 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:08.142 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:08.144 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:59:08.144 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:59:08.144 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:59:08.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:07 smithi167 ceph-mon[36917]: pgmap v503: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-10T00:59:08.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:07 smithi094 ceph-mon[36708]: pgmap v503: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-10T00:59:08.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:07 smithi037 ceph-mon[32192]: pgmap v503: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-10T00:59:10.135 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:10.135 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:10.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:09 smithi167 ceph-mon[36917]: pgmap v504: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-06-10T00:59:10.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:09 smithi094 ceph-mon[36708]: pgmap v504: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-06-10T00:59:10.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:09 smithi037 ceph-mon[32192]: pgmap v504: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-06-10T00:59:10.490 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:59:10.490 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:59:10.571 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:10.571 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:10.571 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 25 -lt 100 ']' 2024-06-10T00:59:10.571 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 25 + 1 2024-06-10T00:59:10.615 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=26 2024-06-10T00:59:10.615 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:59:10.742 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:10.742 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:10.743 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:59:10.743 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:59:10.743 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:59:12.079 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:11 smithi037 ceph-mon[32192]: pgmap v505: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.5 KiB/s wr, 7 op/s 2024-06-10T00:59:12.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:11 smithi167 ceph-mon[36917]: pgmap v505: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.5 KiB/s wr, 7 op/s 2024-06-10T00:59:12.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:11 smithi094 ceph-mon[36708]: pgmap v505: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.5 KiB/s wr, 7 op/s 2024-06-10T00:59:12.651 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:12.651 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:13.014 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:59:13.014 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:59:13.071 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:13.071 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:13.072 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 26 -lt 100 ']' 2024-06-10T00:59:13.072 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 26 + 1 2024-06-10T00:59:13.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.221+0000 7fbe1dee7640 1 -- 172.21.15.37:0/756049353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe18101a20 msgr2=0x7fbe18101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.221+0000 7fbe1dee7640 1 --2- 172.21.15.37:0/756049353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe18101a20 0x7fbe18101e40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009900 tx=0x7fbe0802a3f0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.221+0000 7fbe1dee7640 1 -- 172.21.15.37:0/756049353 shutdown_connections 2024-06-10T00:59:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.221+0000 7fbe1dee7640 1 --2- 172.21.15.37:0/756049353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe181035d0 0x7fbe18109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.221+0000 7fbe1dee7640 1 --2- 172.21.15.37:0/756049353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe18102c10 0x7fbe18103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.221+0000 7fbe1dee7640 1 --2- 172.21.15.37:0/756049353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe18101a20 0x7fbe18101e40 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.221+0000 7fbe1dee7640 1 -- 172.21.15.37:0/756049353 >> 172.21.15.37:0/756049353 conn(0x7fbe180fcf90 msgr2=0x7fbe180ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.221+0000 7fbe1dee7640 1 -- 172.21.15.37:0/756049353 shutdown_connections 2024-06-10T00:59:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.221+0000 7fbe1dee7640 1 -- 172.21.15.37:0/756049353 wait complete. 2024-06-10T00:59:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.247+0000 7fbe1dee7640 1 Processor -- start 2024-06-10T00:59:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.247+0000 7fbe1dee7640 1 -- start start 2024-06-10T00:59:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.247+0000 7fbe1dee7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe18101a20 0x7fbe18197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.247+0000 7fbe1dee7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe18102c10 0x7fbe181981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.247+0000 7fbe1dee7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe181035d0 0x7fbe1819f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.247+0000 7fbe1dee7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1810d6f0 con 0x7fbe18102c10 2024-06-10T00:59:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.247+0000 7fbe1dee7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1810d570 con 0x7fbe18101a20 2024-06-10T00:59:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.247+0000 7fbe1dee7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1810d870 con 0x7fbe181035d0 2024-06-10T00:59:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.247+0000 7fbe16ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe18102c10 0x7fbe181981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.247+0000 7fbe16ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe18102c10 0x7fbe181981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55892/0 (socket says 172.21.15.37:55892) 2024-06-10T00:59:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.247+0000 7fbe16ffd640 1 -- 172.21.15.37:0/3510774092 learned_addr learned my addr 172.21.15.37:0/3510774092 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.247+0000 7fbe17fff640 1 --2- 172.21.15.37:0/3510774092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe181035d0 0x7fbe1819f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.248+0000 7fbe177fe640 1 --2- 172.21.15.37:0/3510774092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe18101a20 0x7fbe18197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.248+0000 7fbe16ffd640 1 -- 172.21.15.37:0/3510774092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe181035d0 msgr2=0x7fbe1819f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.248+0000 7fbe16ffd640 1 --2- 172.21.15.37:0/3510774092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe181035d0 0x7fbe1819f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.248+0000 7fbe16ffd640 1 -- 172.21.15.37:0/3510774092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe18101a20 msgr2=0x7fbe18197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.248+0000 7fbe16ffd640 1 --2- 172.21.15.37:0/3510774092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe18101a20 0x7fbe18197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.248+0000 7fbe16ffd640 1 -- 172.21.15.37:0/3510774092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1819f8c0 con 0x7fbe18102c10 2024-06-10T00:59:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.248+0000 7fbe16ffd640 1 --2- 172.21.15.37:0/3510774092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe18102c10 0x7fbe181981c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00ea60 tx=0x7fbe0c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.251+0000 7fbe14ff9640 1 -- 172.21.15.37:0/3510774092 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c00cde0 con 0x7fbe18102c10 2024-06-10T00:59:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.251+0000 7fbe14ff9640 1 -- 172.21.15.37:0/3510774092 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe0c0035f0 con 0x7fbe18102c10 2024-06-10T00:59:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.251+0000 7fbe14ff9640 1 -- 172.21.15.37:0/3510774092 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c010670 con 0x7fbe18102c10 2024-06-10T00:59:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.252+0000 7fbe1dee7640 1 -- 172.21.15.37:0/3510774092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe1819fbb0 con 0x7fbe18102c10 2024-06-10T00:59:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.254+0000 7fbe14ff9640 1 -- 172.21.15.37:0/3510774092 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fbe0c0037a0 con 0x7fbe18102c10 2024-06-10T00:59:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.255+0000 7fbe14ff9640 1 --2- 172.21.15.37:0/3510774092 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fbe000a2da0 0x7fbe000a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.255+0000 7fbe177fe640 1 --2- 172.21.15.37:0/3510774092 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fbe000a2da0 0x7fbe000a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.255+0000 7fbe177fe640 1 --2- 172.21.15.37:0/3510774092 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fbe000a2da0 0x7fbe000a5260 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbe0802a9e0 tx=0x7fbe08031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:13.272 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=27 2024-06-10T00:59:13.272 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:59:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.290+0000 7fbe1dee7640 1 -- 172.21.15.37:0/3510774092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe181a00d0 con 0x7fbe18102c10 2024-06-10T00:59:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.290+0000 7fbe14ff9640 1 -- 172.21.15.37:0/3510774092 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fbe0c0992f0 con 0x7fbe18102c10 2024-06-10T00:59:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.535+0000 7fbe20bf1740 1 Processor -- start 2024-06-10T00:59:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.535+0000 7fbe20bf1740 1 -- start start 2024-06-10T00:59:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.536+0000 7fbe20bf1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ca9f0010 0x5640ca9f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.536+0000 7fbe20bf1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640ca9f0a00 0x5640ca9f5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.536+0000 7fbe20bf1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640ca9f5b70 0x5640ca9f8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.536+0000 7fbe20bf1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5640ca9534f0 con 0x5640ca9f5b70 2024-06-10T00:59:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.536+0000 7fbe20bf1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5640ca8f1870 con 0x5640ca9f0010 2024-06-10T00:59:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.536+0000 7fbe20bf1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5640ca94f4b0 con 0x5640ca9f0a00 2024-06-10T00:59:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.536+0000 7fbe17fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640ca9f5b70 0x5640ca9f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.536+0000 7fbe17fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640ca9f5b70 0x5640ca9f8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55896/0 (socket says 172.21.15.37:55896) 2024-06-10T00:59:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.536+0000 7fbe17fff640 1 -- 172.21.15.37:0/1396585264 learned_addr learned my addr 172.21.15.37:0/1396585264 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.536+0000 7fbe177fe640 1 --2- 172.21.15.37:0/1396585264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ca9f0010 0x5640ca9f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.536+0000 7fbe16ffd640 1 --2- 172.21.15.37:0/1396585264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640ca9f0a00 0x5640ca9f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.536+0000 7fbe17fff640 1 -- 172.21.15.37:0/1396585264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640ca9f0a00 msgr2=0x5640ca9f5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.536+0000 7fbe17fff640 1 --2- 172.21.15.37:0/1396585264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640ca9f0a00 0x5640ca9f5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.536+0000 7fbe17fff640 1 -- 172.21.15.37:0/1396585264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ca9f0010 msgr2=0x5640ca9f0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.536+0000 7fbe17fff640 1 --2- 172.21.15.37:0/1396585264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ca9f0010 0x5640ca9f0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.536+0000 7fbe17fff640 1 -- 172.21.15.37:0/1396585264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640ca90d9b0 con 0x5640ca9f5b70 2024-06-10T00:59:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.536+0000 7fbe16ffd640 1 --2- 172.21.15.37:0/1396585264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640ca9f0a00 0x5640ca9f5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:59:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.536+0000 7fbe17fff640 1 --2- 172.21.15.37:0/1396585264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640ca9f5b70 0x5640ca9f8000 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fbe10010370 tx=0x7fbe10010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bbfeb953b0a0505 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.536+0000 7fbdfdffb640 1 -- 172.21.15.37:0/1396585264 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe10016070 con 0x5640ca9f5b70 2024-06-10T00:59:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.537+0000 7fbdfdffb640 1 -- 172.21.15.37:0/1396585264 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe10004d20 con 0x5640ca9f5b70 2024-06-10T00:59:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.537+0000 7fbdfdffb640 1 -- 172.21.15.37:0/1396585264 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe10005420 con 0x5640ca9f5b70 2024-06-10T00:59:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.537+0000 7fbe20bf1740 1 -- 172.21.15.37:0/1396585264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640ca9f5b70 msgr2=0x5640ca9f8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.537+0000 7fbe20bf1740 1 --2- 172.21.15.37:0/1396585264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640ca9f5b70 0x5640ca9f8000 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fbe10010370 tx=0x7fbe10010830 comp rx=0 tx=0).stop 2024-06-10T00:59:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.537+0000 7fbe20bf1740 1 -- 172.21.15.37:0/1396585264 shutdown_connections 2024-06-10T00:59:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.537+0000 7fbe20bf1740 1 --2- 172.21.15.37:0/1396585264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640ca9f5b70 0x5640ca9f8000 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.537+0000 7fbe20bf1740 1 --2- 172.21.15.37:0/1396585264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640ca9f0a00 0x5640ca9f5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.537+0000 7fbe20bf1740 1 --2- 172.21.15.37:0/1396585264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ca9f0010 0x5640ca9f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.537+0000 7fbe20bf1740 1 -- 172.21.15.37:0/1396585264 >> 172.21.15.37:0/1396585264 conn(0x5640ca9eb680 msgr2=0x5640ca9eba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.537+0000 7fbe20bf1740 1 -- 172.21.15.37:0/1396585264 shutdown_connections 2024-06-10T00:59:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.537+0000 7fbe20bf1740 1 -- 172.21.15.37:0/1396585264 wait complete. 2024-06-10T00:59:13.553 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:13.553 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:13.554 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:59:13.554 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:59:13.554 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:59:13.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.757+0000 7fbe20bf1740 1 -- start start 2024-06-10T00:59:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.774+0000 7fbe20bf1740 1 Processor -- start 2024-06-10T00:59:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.774+0000 7fbe20bf1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640ca9f0010 0x5640ca9fa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.774+0000 7fbe20bf1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640ca9f0a00 0x5640ca9fa800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.775+0000 7fbe20bf1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ca9f5b70 0x5640caa017d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.775+0000 7fbe20bf1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5640ca9f8b20 con 0x5640ca9f0010 2024-06-10T00:59:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.775+0000 7fbe20bf1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5640ca8dcae0 con 0x5640ca9f5b70 2024-06-10T00:59:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.775+0000 7fbe20bf1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5640ca9f8ca0 con 0x5640ca9f0a00 2024-06-10T00:59:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.775+0000 7fbe177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640ca9f0010 0x5640ca9fa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.775+0000 7fbe177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640ca9f0010 0x5640ca9fa2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55898/0 (socket says 172.21.15.37:55898) 2024-06-10T00:59:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.775+0000 7fbe177fe640 1 -- 172.21.15.37:0/4264582434 learned_addr learned my addr 172.21.15.37:0/4264582434 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.775+0000 7fbe16ffd640 1 --2- 172.21.15.37:0/4264582434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640ca9f0a00 0x5640ca9fa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.775+0000 7fbe17fff640 1 --2- 172.21.15.37:0/4264582434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ca9f5b70 0x5640caa017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.775+0000 7fbe177fe640 1 -- 172.21.15.37:0/4264582434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640ca9f0a00 msgr2=0x5640ca9fa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.775+0000 7fbe177fe640 1 --2- 172.21.15.37:0/4264582434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640ca9f0a00 0x5640ca9fa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.775+0000 7fbe177fe640 1 -- 172.21.15.37:0/4264582434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ca9f5b70 msgr2=0x5640caa017d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.775+0000 7fbe177fe640 1 --2- 172.21.15.37:0/4264582434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ca9f5b70 0x5640caa017d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.775+0000 7fbe177fe640 1 -- 172.21.15.37:0/4264582434 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640ca667210 con 0x5640ca9f0010 2024-06-10T00:59:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.775+0000 7fbe17fff640 1 --2- 172.21.15.37:0/4264582434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ca9f5b70 0x5640caa017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:59:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.775+0000 7fbe177fe640 1 --2- 172.21.15.37:0/4264582434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640ca9f0010 0x5640ca9fa2c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009900 tx=0x7fbe08009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158d0704e55bee34 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.775+0000 7fbdf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:59:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.775+0000 7fbdf6ffd640 1 -- 172.21.15.37:0/4264582434 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0804d070 con 0x5640ca9f0010 2024-06-10T00:59:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.775+0000 7fbdf6ffd640 1 -- 172.21.15.37:0/4264582434 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fbe08005020 con 0x5640ca9f0010 2024-06-10T00:59:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.824+0000 7fbe20bf1740 10 client.0 _ll_register_callbacks cb 0x5640ca9159b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:59:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.850+0000 7fbe20bf1740 1 -- 172.21.15.37:0/4264582434 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe10004d20 con 0x5640ca9f0010 2024-06-10T00:59:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.850+0000 7fbdf6ffd640 1 -- 172.21.15.37:0/4264582434 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbe08031e40 con 0x5640ca9f0010 2024-06-10T00:59:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.850+0000 7fbe20bf1740 10 client.15867 fetch_fsmap learned FSMap version 27 2024-06-10T00:59:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.850+0000 7fbe20bf1740 1 -- 172.21.15.37:0/4264582434 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=27}) -- 0x5640ca733620 con 0x5640ca9f0010 2024-06-10T00:59:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.917+0000 7fbdf6ffd640 1 -- 172.21.15.37:0/4264582434 <== mon.0 v2:172.21.15.37:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbe08005220 con 0x5640ca9f0010 2024-06-10T00:59:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.917+0000 7fbe20bf1740 10 client.15867 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:59:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.917+0000 7fbe20bf1740 10 client.15867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T00:59:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.917+0000 7fbe20bf1740 20 client.15867 populate_metadata read hostname 'smithi037' 2024-06-10T00:59:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.917+0000 7fbe20bf1740 1 --2- 172.21.15.37:0/4264582434 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5640caa05f40 0x5640caa08400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.917+0000 7fbe20bf1740 4 client.15867 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:59:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.917+0000 7fbe20bf1740 1 -- 172.21.15.37:0/4264582434 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640ca966f10 con 0x5640caa05f40 2024-06-10T00:59:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.917+0000 7fbe16ffd640 1 --2- 172.21.15.37:0/4264582434 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5640caa05f40 0x5640caa08400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.918+0000 7fbe16ffd640 1 --2- 172.21.15.37:0/4264582434 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5640caa05f40 0x5640caa08400 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34cc3cc7462461a0 server_cookie=ec59f5ce5acd7dc4 in_seq=0 out_seq=0 2024-06-10T00:59:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.918+0000 7fbdf6ffd640 10 client.15867 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:59:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.919+0000 7fbdf6ffd640 1 -- 172.21.15.37:0/4264582434 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5640ca966f10 con 0x5640caa05f40 2024-06-10T00:59:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.919+0000 7fbdf6ffd640 10 client.15867 handle_command_reply: tid=0 2024-06-10T00:59:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.919+0000 7fbe20bf1740 1 client.15867 shutdown 2024-06-10T00:59:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.919+0000 7fbe20bf1740 1 -- 172.21.15.37:0/4264582434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640ca9f0010 msgr2=0x5640ca9fa2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.919+0000 7fbe20bf1740 1 --2- 172.21.15.37:0/4264582434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640ca9f0010 0x5640ca9fa2c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009900 tx=0x7fbe08009260 comp rx=0 tx=0).stop 2024-06-10T00:59:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.919+0000 7fbe20bf1740 1 -- 172.21.15.37:0/4264582434 shutdown_connections 2024-06-10T00:59:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.919+0000 7fbe20bf1740 1 --2- 172.21.15.37:0/4264582434 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5640caa05f40 0x5640caa08400 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.919+0000 7fbe20bf1740 1 --2- 172.21.15.37:0/4264582434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ca9f5b70 0x5640caa017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.919+0000 7fbe20bf1740 1 --2- 172.21.15.37:0/4264582434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640ca9f0a00 0x5640ca9fa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.919+0000 7fbe20bf1740 1 --2- 172.21.15.37:0/4264582434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640ca9f0010 0x5640ca9fa2c0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.919+0000 7fbdf6ffd640 0 client.15867 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:59:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.919+0000 7fbe20bf1740 1 -- 172.21.15.37:0/4264582434 >> 172.21.15.37:0/4264582434 conn(0x5640ca9eb680 msgr2=0x5640ca9ea7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.919+0000 7fbe20bf1740 1 -- 172.21.15.37:0/4264582434 shutdown_connections 2024-06-10T00:59:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.919+0000 7fbe20bf1740 1 -- 172.21.15.37:0/4264582434 wait complete. 2024-06-10T00:59:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.940+0000 7fbe20bf1740 20 client.15867 trim_cache size 0 max 16384 2024-06-10T00:59:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.955+0000 7fbe20bf1740 1 Processor -- start 2024-06-10T00:59:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.955+0000 7fbe20bf1740 1 -- start start 2024-06-10T00:59:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.955+0000 7fbe20bf1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640ca9f0010 0x5640ca9f9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.955+0000 7fbe20bf1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640ca9f0a00 0x5640caa01790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.955+0000 7fbe20bf1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ca9f5b70 0x5640ca9fa410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.955+0000 7fbe20bf1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181ad030 con 0x5640ca9f0a00 2024-06-10T00:59:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.955+0000 7fbe20bf1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181aa890 con 0x5640ca9f5b70 2024-06-10T00:59:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.955+0000 7fbe20bf1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181a86e0 con 0x5640ca9f0010 2024-06-10T00:59:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.956+0000 7fbe16ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640ca9f0a00 0x5640caa01790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.956+0000 7fbe16ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640ca9f0a00 0x5640caa01790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55902/0 (socket says 172.21.15.37:55902) 2024-06-10T00:59:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.956+0000 7fbe16ffd640 1 -- 172.21.15.37:0/3619449746 learned_addr learned my addr 172.21.15.37:0/3619449746 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.956+0000 7fbe177fe640 1 --2- 172.21.15.37:0/3619449746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640ca9f0010 0x5640ca9f9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.956+0000 7fbe17fff640 1 --2- 172.21.15.37:0/3619449746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ca9f5b70 0x5640ca9fa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.956+0000 7fbe16ffd640 1 -- 172.21.15.37:0/3619449746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640ca9f0010 msgr2=0x5640ca9f9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.956+0000 7fbe16ffd640 1 --2- 172.21.15.37:0/3619449746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640ca9f0010 0x5640ca9f9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.956+0000 7fbe16ffd640 1 -- 172.21.15.37:0/3619449746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ca9f5b70 msgr2=0x5640ca9fa410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.956+0000 7fbe16ffd640 1 --2- 172.21.15.37:0/3619449746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ca9f5b70 0x5640ca9fa410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.956+0000 7fbe16ffd640 1 -- 172.21.15.37:0/3619449746 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640ca9eb4b0 con 0x5640ca9f0a00 2024-06-10T00:59:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.956+0000 7fbe16ffd640 1 --2- 172.21.15.37:0/3619449746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640ca9f0a00 0x5640caa01790 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00c760 tx=0x7fbe0c0b1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5194babeb08ed0d8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.956+0000 7fbdfd7fa640 1 -- 172.21.15.37:0/3619449746 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c0a6070 con 0x5640ca9f0a00 2024-06-10T00:59:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.956+0000 7fbdfd7fa640 1 -- 172.21.15.37:0/3619449746 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe0c09f600 con 0x5640ca9f0a00 2024-06-10T00:59:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.956+0000 7fbdfd7fa640 1 -- 172.21.15.37:0/3619449746 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c003c50 con 0x5640ca9f0a00 2024-06-10T00:59:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.957+0000 7fbe20bf1740 1 -- 172.21.15.37:0/3619449746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640ca9f0a00 msgr2=0x5640caa01790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.957+0000 7fbe20bf1740 1 --2- 172.21.15.37:0/3619449746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640ca9f0a00 0x5640caa01790 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00c760 tx=0x7fbe0c0b1040 comp rx=0 tx=0).stop 2024-06-10T00:59:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.957+0000 7fbe20bf1740 1 -- 172.21.15.37:0/3619449746 shutdown_connections 2024-06-10T00:59:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.957+0000 7fbe20bf1740 1 --2- 172.21.15.37:0/3619449746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ca9f5b70 0x5640ca9fa410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.957+0000 7fbe20bf1740 1 --2- 172.21.15.37:0/3619449746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640ca9f0a00 0x5640caa01790 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.957+0000 7fbe20bf1740 1 --2- 172.21.15.37:0/3619449746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640ca9f0010 0x5640ca9f9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.957+0000 7fbe20bf1740 1 -- 172.21.15.37:0/3619449746 >> 172.21.15.37:0/3619449746 conn(0x5640ca9eb680 msgr2=0x5640ca9eba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.957+0000 7fbe20bf1740 1 -- 172.21.15.37:0/3619449746 shutdown_connections 2024-06-10T00:59:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.957+0000 7fbe20bf1740 1 -- 172.21.15.37:0/3619449746 wait complete. 2024-06-10T00:59:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.957+0000 7fbe20bf1740 1 -- start start 2024-06-10T00:59:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.958+0000 7fbe20bf1740 1 Processor -- start 2024-06-10T00:59:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.958+0000 7fbe20bf1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640ca9f0010 0x5640ca9fefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.958+0000 7fbe20bf1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640ca9f0a00 0x5640ca9ff500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.958+0000 7fbe20bf1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ca9f5b70 0x5640caa00510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.958+0000 7fbe20bf1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181a8d50 con 0x5640ca9f0a00 2024-06-10T00:59:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.958+0000 7fbe20bf1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181aa1c0 con 0x5640ca9f5b70 2024-06-10T00:59:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.958+0000 7fbe20bf1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181a9420 con 0x5640ca9f0010 2024-06-10T00:59:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.958+0000 7fbe20bf1740 10 client.0 _ll_register_callbacks cb 0x7fbe0c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:59:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.958+0000 7fbe16ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640ca9f0a00 0x5640ca9ff500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.958+0000 7fbe16ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640ca9f0a00 0x5640ca9ff500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55916/0 (socket says 172.21.15.37:55916) 2024-06-10T00:59:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.958+0000 7fbe16ffd640 1 -- 172.21.15.37:0/1014986440 learned_addr learned my addr 172.21.15.37:0/1014986440 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.958+0000 7fbe177fe640 1 --2- 172.21.15.37:0/1014986440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640ca9f0010 0x5640ca9fefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.958+0000 7fbe17fff640 1 --2- 172.21.15.37:0/1014986440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ca9f5b70 0x5640caa00510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.958+0000 7fbe16ffd640 1 -- 172.21.15.37:0/1014986440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640ca9f0010 msgr2=0x5640ca9fefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.958+0000 7fbe16ffd640 1 --2- 172.21.15.37:0/1014986440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640ca9f0010 0x5640ca9fefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.958+0000 7fbe16ffd640 1 -- 172.21.15.37:0/1014986440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ca9f5b70 msgr2=0x5640caa00510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.958+0000 7fbe16ffd640 1 --2- 172.21.15.37:0/1014986440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ca9f5b70 0x5640caa00510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.958+0000 7fbe16ffd640 1 -- 172.21.15.37:0/1014986440 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640caa00b00 con 0x5640ca9f0a00 2024-06-10T00:59:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.958+0000 7fbe16ffd640 1 --2- 172.21.15.37:0/1014986440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640ca9f0a00 0x5640ca9ff500 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c009230 tx=0x7fbe0c0b42a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2399baa674f3cf72 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.958+0000 7fbdf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:59:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.959+0000 7fbdf6ffd640 1 -- 172.21.15.37:0/1014986440 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c0a6070 con 0x5640ca9f0a00 2024-06-10T00:59:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.959+0000 7fbdf6ffd640 1 -- 172.21.15.37:0/1014986440 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fbe0c0a3a30 con 0x5640ca9f0a00 2024-06-10T00:59:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.959+0000 7fbe20bf1740 1 -- 172.21.15.37:0/1014986440 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe0c09f600 con 0x5640ca9f0a00 2024-06-10T00:59:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.959+0000 7fbdf6ffd640 1 -- 172.21.15.37:0/1014986440 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbe0c0a3c30 con 0x5640ca9f0a00 2024-06-10T00:59:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.959+0000 7fbe20bf1740 10 client.15873 fetch_fsmap learned FSMap version 27 2024-06-10T00:59:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.959+0000 7fbe20bf1740 1 -- 172.21.15.37:0/1014986440 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5640caa70dc0 con 0x5640ca9f0a00 2024-06-10T00:59:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.959+0000 7fbdf6ffd640 1 -- 172.21.15.37:0/1014986440 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe0c09f600 con 0x5640ca9f0a00 2024-06-10T00:59:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.959+0000 7fbdf6ffd640 1 -- 172.21.15.37:0/1014986440 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbe0c096fa0 con 0x5640ca9f0a00 2024-06-10T00:59:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.959+0000 7fbe20bf1740 10 client.15873 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:59:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.959+0000 7fbe20bf1740 10 client.15873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T00:59:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.959+0000 7fbe20bf1740 20 client.15873 populate_metadata read hostname 'smithi037' 2024-06-10T00:59:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.959+0000 7fbe20bf1740 1 --2- 172.21.15.37:0/1014986440 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5640caa71740 0x5640caa73b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.959+0000 7fbe20bf1740 4 client.15873 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T00:59:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.959+0000 7fbe20bf1740 1 -- 172.21.15.37:0/1014986440 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5640ca960c20 con 0x5640caa71740 2024-06-10T00:59:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.959+0000 7fbe177fe640 1 --2- 172.21.15.37:0/1014986440 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5640caa71740 0x5640caa73b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.960+0000 7fbe177fe640 1 --2- 172.21.15.37:0/1014986440 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5640caa71740 0x5640caa73b60 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=782439dec4179af1 server_cookie=368e70d9a8ece847 in_seq=0 out_seq=0 2024-06-10T00:59:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.960+0000 7fbdf6ffd640 10 client.15873 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:59:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.978+0000 7fbdf6ffd640 1 -- 172.21.15.37:0/1014986440 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5640ca960c20 con 0x5640caa71740 2024-06-10T00:59:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.978+0000 7fbdf6ffd640 10 client.15873 handle_command_reply: tid=0 2024-06-10T00:59:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.978+0000 7fbe20bf1740 1 client.15873 shutdown 2024-06-10T00:59:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.978+0000 7fbe20bf1740 1 -- 172.21.15.37:0/1014986440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640ca9f0a00 msgr2=0x5640ca9ff500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.978+0000 7fbe20bf1740 1 --2- 172.21.15.37:0/1014986440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640ca9f0a00 0x5640ca9ff500 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c009230 tx=0x7fbe0c0b42a0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.978+0000 7fbe20bf1740 1 -- 172.21.15.37:0/1014986440 shutdown_connections 2024-06-10T00:59:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.978+0000 7fbe20bf1740 1 --2- 172.21.15.37:0/1014986440 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5640caa71740 0x5640caa73b60 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.978+0000 7fbe20bf1740 1 --2- 172.21.15.37:0/1014986440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ca9f5b70 0x5640caa00510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.978+0000 7fbdf6ffd640 0 client.15873 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:59:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.978+0000 7fbe20bf1740 1 --2- 172.21.15.37:0/1014986440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640ca9f0a00 0x5640ca9ff500 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.978+0000 7fbe20bf1740 1 --2- 172.21.15.37:0/1014986440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640ca9f0010 0x5640ca9fefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.978+0000 7fbe20bf1740 1 -- 172.21.15.37:0/1014986440 >> 172.21.15.37:0/1014986440 conn(0x5640ca9eb680 msgr2=0x5640ca9f9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.978+0000 7fbe20bf1740 1 -- 172.21.15.37:0/1014986440 shutdown_connections 2024-06-10T00:59:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.978+0000 7fbe20bf1740 1 -- 172.21.15.37:0/1014986440 wait complete. 2024-06-10T00:59:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.978+0000 7fbe20bf1740 20 client.15873 trim_cache size 0 max 16384 2024-06-10T00:59:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T00:59:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T00:59:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T00:59:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.979+0000 7fbe1dee7640 1 -- 172.21.15.37:0/3510774092 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fbe000a2da0 msgr2=0x7fbe000a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.979+0000 7fbe1dee7640 1 --2- 172.21.15.37:0/3510774092 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fbe000a2da0 0x7fbe000a5260 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbe0802a9e0 tx=0x7fbe08031400 comp rx=0 tx=0).stop 2024-06-10T00:59:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.979+0000 7fbe1dee7640 1 -- 172.21.15.37:0/3510774092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe18102c10 msgr2=0x7fbe181981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.979+0000 7fbe1dee7640 1 --2- 172.21.15.37:0/3510774092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe18102c10 0x7fbe181981c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00ea60 tx=0x7fbe0c00ef20 comp rx=0 tx=0).stop 2024-06-10T00:59:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.979+0000 7fbe1dee7640 1 -- 172.21.15.37:0/3510774092 shutdown_connections 2024-06-10T00:59:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.979+0000 7fbe1dee7640 1 --2- 172.21.15.37:0/3510774092 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fbe000a2da0 0x7fbe000a5260 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.979+0000 7fbe1dee7640 1 --2- 172.21.15.37:0/3510774092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe181035d0 0x7fbe1819f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.979+0000 7fbe1dee7640 1 --2- 172.21.15.37:0/3510774092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe18102c10 0x7fbe181981c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.979+0000 7fbe1dee7640 1 --2- 172.21.15.37:0/3510774092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe18101a20 0x7fbe18197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.979+0000 7fbe1dee7640 1 -- 172.21.15.37:0/3510774092 >> 172.21.15.37:0/3510774092 conn(0x7fbe180fcf90 msgr2=0x7fbe18105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.979+0000 7fbe1dee7640 1 -- 172.21.15.37:0/3510774092 shutdown_connections 2024-06-10T00:59:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:13.979+0000 7fbe1dee7640 1 -- 172.21.15.37:0/3510774092 wait complete. 2024-06-10T00:59:14.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:14 smithi094 ceph-mon[36708]: pgmap v506: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 KiB/s wr, 9 op/s 2024-06-10T00:59:14.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:14 smithi037 ceph-mon[32192]: pgmap v506: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 KiB/s wr, 9 op/s 2024-06-10T00:59:14.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:14 smithi167 ceph-mon[36917]: pgmap v506: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 KiB/s wr, 9 op/s 2024-06-10T00:59:14.603 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:59:15.314 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:15 smithi037 ceph-mon[32192]: pgmap v507: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.7 KiB/s wr, 7 op/s 2024-06-10T00:59:15.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:15 smithi094 ceph-mon[36708]: pgmap v507: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.7 KiB/s wr, 7 op/s 2024-06-10T00:59:15.517 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:15.518 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:15.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:15 smithi167 ceph-mon[36917]: pgmap v507: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.7 KiB/s wr, 7 op/s 2024-06-10T00:59:15.889 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:59:15.890 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:59:15.921 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:15.922 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:15.922 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 27 -lt 100 ']' 2024-06-10T00:59:15.922 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 27 + 1 2024-06-10T00:59:16.090 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=28 2024-06-10T00:59:16.090 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:59:16.131 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:16.131 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:16.132 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:59:16.132 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:59:16.132 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:59:17.985 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:17.985 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:18.319 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:17 smithi037 ceph-mon[32192]: pgmap v508: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 10 op/s 2024-06-10T00:59:18.331 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:59:18.331 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:59:18.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:17 smithi167 ceph-mon[36917]: pgmap v508: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 10 op/s 2024-06-10T00:59:18.347 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:18.347 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:18.347 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 28 -lt 100 ']' 2024-06-10T00:59:18.347 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 28 + 1 2024-06-10T00:59:18.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:17 smithi094 ceph-mon[36708]: pgmap v508: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 10 op/s 2024-06-10T00:59:18.420 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=29 2024-06-10T00:59:18.420 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:59:18.557 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:18.557 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:18.558 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:59:18.558 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:59:18.559 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:59:20.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:19 smithi167 ceph-mon[36917]: pgmap v509: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.1 KiB/s wr, 14 op/s 2024-06-10T00:59:20.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:19 smithi094 ceph-mon[36708]: pgmap v509: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.1 KiB/s wr, 14 op/s 2024-06-10T00:59:20.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:19 smithi037 ceph-mon[32192]: pgmap v509: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.1 KiB/s wr, 14 op/s 2024-06-10T00:59:20.586 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:20.586 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:20.973 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:59:20.973 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:59:20.986 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:20.987 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:20.987 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 29 -lt 100 ']' 2024-06-10T00:59:20.987 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 29 + 1 2024-06-10T00:59:21.106 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=30 2024-06-10T00:59:21.107 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:59:21.316 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:21.316 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:21.318 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:59:21.318 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:59:21.318 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:59:22.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.073+0000 7f0559b2a700 1 -- 172.21.15.37:0/2233402950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0554101380 msgr2=0x7f0554101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:22.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.073+0000 7f0559b2a700 1 --2- 172.21.15.37:0/2233402950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0554101380 0x7f0554101800 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0548009990 tx=0x7f0548009ca0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.073+0000 7f0559b2a700 1 -- 172.21.15.37:0/2233402950 shutdown_connections 2024-06-10T00:59:22.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.073+0000 7f0559b2a700 1 --2- 172.21.15.37:0/2233402950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0554101d40 0x7f055410e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.073+0000 7f0559b2a700 1 --2- 172.21.15.37:0/2233402950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05541073c0 0x7f05541077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.073+0000 7f0559b2a700 1 --2- 172.21.15.37:0/2233402950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0554101380 0x7f0554101800 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.073+0000 7f0559b2a700 1 -- 172.21.15.37:0/2233402950 >> 172.21.15.37:0/2233402950 conn(0x7f0554078330 msgr2=0x7f0554078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:22.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.073+0000 7f0559b2a700 1 -- 172.21.15.37:0/2233402950 shutdown_connections 2024-06-10T00:59:22.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.073+0000 7f0559b2a700 1 -- 172.21.15.37:0/2233402950 wait complete. 2024-06-10T00:59:22.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.074+0000 7f0559b2a700 1 Processor -- start 2024-06-10T00:59:22.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.074+0000 7f0559b2a700 1 -- start start 2024-06-10T00:59:22.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.074+0000 7f0559b2a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0554101380 0x7f055410ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:22.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.074+0000 7f0559b2a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0554101d40 0x7f055410f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:22.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.074+0000 7f0559b2a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05541073c0 0x7f055410f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:22.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.074+0000 7f0559b2a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f05541127b0 con 0x7f05541073c0 2024-06-10T00:59:22.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.074+0000 7f0559b2a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0554112630 con 0x7f0554101380 2024-06-10T00:59:22.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.074+0000 7f0559b2a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0554112930 con 0x7f0554101d40 2024-06-10T00:59:22.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.075+0000 7f0553fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05541073c0 0x7f055410f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:22.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.075+0000 7f05537fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0554101380 0x7f055410ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:22.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.075+0000 7f05537fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0554101380 0x7f055410ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:35420/0 (socket says 172.21.15.37:35420) 2024-06-10T00:59:22.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.075+0000 7f05537fe700 1 -- 172.21.15.37:0/4231226643 learned_addr learned my addr 172.21.15.37:0/4231226643 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:22.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.075+0000 7f0552ffd700 1 --2- 172.21.15.37:0/4231226643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0554101d40 0x7f055410f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:22.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.075+0000 7f0553fff700 1 -- 172.21.15.37:0/4231226643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0554101d40 msgr2=0x7f055410f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:22.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.075+0000 7f0553fff700 1 --2- 172.21.15.37:0/4231226643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0554101d40 0x7f055410f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.075+0000 7f0553fff700 1 -- 172.21.15.37:0/4231226643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0554101380 msgr2=0x7f055410ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:22.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.075+0000 7f0553fff700 1 --2- 172.21.15.37:0/4231226643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0554101380 0x7f055410ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.075+0000 7f0553fff700 1 -- 172.21.15.37:0/4231226643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05541a7120 con 0x7f05541073c0 2024-06-10T00:59:22.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.075+0000 7f05537fe700 1 --2- 172.21.15.37:0/4231226643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0554101380 0x7f055410ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:59:22.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.075+0000 7f0553fff700 1 --2- 172.21.15.37:0/4231226643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05541073c0 0x7f055410f710 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f054c007f10 tx=0x7f054c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:22.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.076+0000 7f0550ff9700 1 -- 172.21.15.37:0/4231226643 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c004db0 con 0x7f05541073c0 2024-06-10T00:59:22.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.076+0000 7f0550ff9700 1 -- 172.21.15.37:0/4231226643 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f054c003a00 con 0x7f05541073c0 2024-06-10T00:59:22.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.076+0000 7f0550ff9700 1 -- 172.21.15.37:0/4231226643 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c004680 con 0x7f05541073c0 2024-06-10T00:59:22.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.076+0000 7f0559b2a700 1 -- 172.21.15.37:0/4231226643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05541a7410 con 0x7f05541073c0 2024-06-10T00:59:22.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.076+0000 7f0559b2a700 1 -- 172.21.15.37:0/4231226643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05541a78d0 con 0x7f05541073c0 2024-06-10T00:59:22.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.078+0000 7f0550ff9700 1 -- 172.21.15.37:0/4231226643 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f054c01c020 con 0x7f05541073c0 2024-06-10T00:59:22.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.079+0000 7f0550ff9700 1 --2- 172.21.15.37:0/4231226643 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f053c0a2ca0 0x7f053c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:22.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.079+0000 7f05537fe700 1 --2- 172.21.15.37:0/4231226643 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f053c0a2ca0 0x7f053c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:22.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.080+0000 7f0550ff9700 1 -- 172.21.15.37:0/4231226643 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f054c0ceae0 con 0x7f05541073c0 2024-06-10T00:59:22.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.080+0000 7f05537fe700 1 --2- 172.21.15.37:0/4231226643 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f053c0a2ca0 0x7f053c0a5160 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0544005950 tx=0x7f054400b500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:22.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.126+0000 7f05643adb80 1 Processor -- start 2024-06-10T00:59:22.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.126+0000 7f05643adb80 1 -- start start 2024-06-10T00:59:22.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.126+0000 7f05643adb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557966cb1640 0x557966cb1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:22.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.126+0000 7f05643adb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557966cb1f60 0x557966cab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:22.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.126+0000 7f05643adb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557966cabf20 0x557966cac340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:22.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.127+0000 7f05643adb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557966bfd810 con 0x557966cb1640 2024-06-10T00:59:22.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.127+0000 7f05643adb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557966ba5720 con 0x557966cabf20 2024-06-10T00:59:22.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.127+0000 7f05643adb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557966bf97d0 con 0x557966cb1f60 2024-06-10T00:59:22.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.127+0000 7f05537fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557966cb1640 0x557966cb1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:22.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.127+0000 7f05537fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557966cb1640 0x557966cb1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50034/0 (socket says 172.21.15.37:50034) 2024-06-10T00:59:22.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.127+0000 7f05537fe700 1 -- 172.21.15.37:0/907715619 learned_addr learned my addr 172.21.15.37:0/907715619 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:22.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.127+0000 7f0552ffd700 1 --2- 172.21.15.37:0/907715619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557966cb1f60 0x557966cab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:22.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.127+0000 7f0553fff700 1 --2- 172.21.15.37:0/907715619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557966cabf20 0x557966cac340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:22.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.127+0000 7f05537fe700 1 -- 172.21.15.37:0/907715619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557966cb1f60 msgr2=0x557966cab9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:22.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.127+0000 7f05537fe700 1 --2- 172.21.15.37:0/907715619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557966cb1f60 0x557966cab9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.127+0000 7f05537fe700 1 -- 172.21.15.37:0/907715619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557966cabf20 msgr2=0x557966cac340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:22.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.127+0000 7f05537fe700 1 --2- 172.21.15.37:0/907715619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557966cabf20 0x557966cac340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.127+0000 7f05537fe700 1 -- 172.21.15.37:0/907715619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557966b9bf50 con 0x557966cb1640 2024-06-10T00:59:22.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.127+0000 7f0552ffd700 1 --2- 172.21.15.37:0/907715619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557966cb1f60 0x557966cab9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:59:22.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.127+0000 7f0553fff700 1 --2- 172.21.15.37:0/907715619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557966cabf20 0x557966cac340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:59:22.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.127+0000 7f05537fe700 1 --2- 172.21.15.37:0/907715619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557966cb1640 0x557966cb1a20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0544011790 tx=0x7f0544013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696100b9196a94d0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:22.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.128+0000 7f0539ffb700 1 -- 172.21.15.37:0/907715619 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054401e620 con 0x557966cb1640 2024-06-10T00:59:22.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.128+0000 7f0539ffb700 1 -- 172.21.15.37:0/907715619 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f054401eca0 con 0x557966cb1640 2024-06-10T00:59:22.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.128+0000 7f0539ffb700 1 -- 172.21.15.37:0/907715619 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054401cd20 con 0x557966cb1640 2024-06-10T00:59:22.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.129+0000 7f05643adb80 1 -- 172.21.15.37:0/907715619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557966cb1640 msgr2=0x557966cb1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:22.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.129+0000 7f05643adb80 1 --2- 172.21.15.37:0/907715619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557966cb1640 0x557966cb1a20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0544011790 tx=0x7f0544013000 comp rx=0 tx=0).stop 2024-06-10T00:59:22.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.129+0000 7f05643adb80 1 -- 172.21.15.37:0/907715619 shutdown_connections 2024-06-10T00:59:22.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.129+0000 7f05643adb80 1 --2- 172.21.15.37:0/907715619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557966cabf20 0x557966cac340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.129+0000 7f05643adb80 1 --2- 172.21.15.37:0/907715619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557966cb1640 0x557966cb1a20 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.129+0000 7f05643adb80 1 --2- 172.21.15.37:0/907715619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557966cb1f60 0x557966cab9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.129+0000 7f05643adb80 1 -- 172.21.15.37:0/907715619 >> 172.21.15.37:0/907715619 conn(0x557966bd6cc0 msgr2=0x557966ca6850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:22.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.129+0000 7f05643adb80 1 -- 172.21.15.37:0/907715619 shutdown_connections 2024-06-10T00:59:22.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.129+0000 7f05643adb80 1 -- 172.21.15.37:0/907715619 wait complete. 2024-06-10T00:59:22.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.129+0000 7f05643adb80 1 -- start start 2024-06-10T00:59:22.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.130+0000 7f05643adb80 1 Processor -- start 2024-06-10T00:59:22.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.130+0000 7f05643adb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557966cabf20 0x557966cb7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:22.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.130+0000 7f05643adb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557966cb1640 0x557966cb4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:22.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.130+0000 7f05643adb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557966cb1f60 0x557966cb54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:22.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.130+0000 7f05643adb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557966cb89d0 con 0x557966cb1640 2024-06-10T00:59:22.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.130+0000 7f05643adb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557966b9dbf0 con 0x557966cabf20 2024-06-10T00:59:22.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.130+0000 7f05643adb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557966cb8b50 con 0x557966cb1f60 2024-06-10T00:59:22.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.130+0000 7f05643adb80 10 client.0 _ll_register_callbacks cb 0x557966b97920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:59:22.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.130+0000 7f0552ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557966cb1640 0x557966cb4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:22.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.130+0000 7f0552ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557966cb1640 0x557966cb4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50048/0 (socket says 172.21.15.37:50048) 2024-06-10T00:59:22.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.130+0000 7f0552ffd700 1 -- 172.21.15.37:0/2673076794 learned_addr learned my addr 172.21.15.37:0/2673076794 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:22.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.130+0000 7f05537fe700 1 --2- 172.21.15.37:0/2673076794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557966cabf20 0x557966cb7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:22.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.130+0000 7f0553fff700 1 --2- 172.21.15.37:0/2673076794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557966cb1f60 0x557966cb54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:22.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.131+0000 7f0552ffd700 1 -- 172.21.15.37:0/2673076794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557966cb1f60 msgr2=0x557966cb54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:22.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.131+0000 7f0552ffd700 1 --2- 172.21.15.37:0/2673076794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557966cb1f60 0x557966cb54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.131+0000 7f0552ffd700 1 -- 172.21.15.37:0/2673076794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557966cabf20 msgr2=0x557966cb7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:22.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.131+0000 7f0552ffd700 1 --2- 172.21.15.37:0/2673076794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557966cabf20 0x557966cb7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.131+0000 7f0552ffd700 1 -- 172.21.15.37:0/2673076794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557966b4c8d0 con 0x557966cb1640 2024-06-10T00:59:22.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.131+0000 7f0552ffd700 1 --2- 172.21.15.37:0/2673076794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557966cb1640 0x557966cb4fb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0548009990 tx=0x7f054800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c94b682bc6a0c9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:22.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.131+0000 7f0532ffd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:59:22.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.131+0000 7f0532ffd700 1 -- 172.21.15.37:0/2673076794 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0548029060 con 0x557966cb1640 2024-06-10T00:59:22.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.131+0000 7f0532ffd700 1 -- 172.21.15.37:0/2673076794 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f0548003b60 con 0x557966cb1640 2024-06-10T00:59:22.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.131+0000 7f05643adb80 1 -- 172.21.15.37:0/2673076794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f054401eca0 con 0x557966cb1640 2024-06-10T00:59:22.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.131+0000 7f0532ffd700 1 -- 172.21.15.37:0/2673076794 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0548030050 con 0x557966cb1640 2024-06-10T00:59:22.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.131+0000 7f05643adb80 10 client.15885 fetch_fsmap learned FSMap version 27 2024-06-10T00:59:22.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.131+0000 7f05643adb80 1 -- 172.21.15.37:0/2673076794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557966cc32e0 con 0x557966cb1640 2024-06-10T00:59:22.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.132+0000 7f0532ffd700 1 -- 172.21.15.37:0/2673076794 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f054401eca0 con 0x557966cb1640 2024-06-10T00:59:22.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.132+0000 7f0532ffd700 1 -- 172.21.15.37:0/2673076794 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f054800f750 con 0x557966cb1640 2024-06-10T00:59:22.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.132+0000 7f05643adb80 10 client.15885 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:59:22.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.132+0000 7f05643adb80 10 client.15885 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-10T00:59:22.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.132+0000 7f05643adb80 20 client.15885 populate_metadata read hostname 'smithi037' 2024-06-10T00:59:22.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.132+0000 7f05643adb80 1 --2- 172.21.15.37:0/2673076794 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x557966cc5920 0x557966cc7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:22.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.132+0000 7f05643adb80 4 client.15885 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:59:22.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.132+0000 7f05643adb80 1 -- 172.21.15.37:0/2673076794 --> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557966c0ad10 con 0x557966cc5920 2024-06-10T00:59:22.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.132+0000 7f05537fe700 1 --2- 172.21.15.37:0/2673076794 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x557966cc5920 0x557966cc7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:22.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.133+0000 7f05537fe700 1 --2- 172.21.15.37:0/2673076794 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x557966cc5920 0x557966cc7d10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be01a4e9c3e1406 server_cookie=f53396b9c70e8d6b in_seq=0 out_seq=0 2024-06-10T00:59:22.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.133+0000 7f0532ffd700 10 client.15885 ms_handle_connect on v2:172.21.15.167:6836/2720148065 2024-06-10T00:59:22.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.134+0000 7f0532ffd700 1 -- 172.21.15.37:0/2673076794 <== mds.2 v2:172.21.15.167:6836/2720148065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557966c0ad10 con 0x557966cc5920 2024-06-10T00:59:22.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.134+0000 7f0532ffd700 10 client.15885 handle_command_reply: tid=0 2024-06-10T00:59:22.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.134+0000 7f05643adb80 1 client.15885 shutdown 2024-06-10T00:59:22.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.134+0000 7f05643adb80 1 -- 172.21.15.37:0/2673076794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557966cb1640 msgr2=0x557966cb4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:22.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.134+0000 7f05643adb80 1 --2- 172.21.15.37:0/2673076794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557966cb1640 0x557966cb4fb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0548009990 tx=0x7f054800bf90 comp rx=0 tx=0).stop 2024-06-10T00:59:22.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.134+0000 7f05643adb80 1 -- 172.21.15.37:0/2673076794 shutdown_connections 2024-06-10T00:59:22.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.134+0000 7f05643adb80 1 --2- 172.21.15.37:0/2673076794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557966cb1f60 0x557966cb54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.134+0000 7f05643adb80 1 --2- 172.21.15.37:0/2673076794 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x557966cc5920 0x557966cc7d10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.134+0000 7f05643adb80 1 --2- 172.21.15.37:0/2673076794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557966cabf20 0x557966cb7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.134+0000 7f05643adb80 1 --2- 172.21.15.37:0/2673076794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557966cb1640 0x557966cb4fb0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.134+0000 7f0532ffd700 0 client.15885 ms_handle_reset on v2:172.21.15.167:6836/2720148065 2024-06-10T00:59:22.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.134+0000 7f05643adb80 1 -- 172.21.15.37:0/2673076794 >> 172.21.15.37:0/2673076794 conn(0x557966bd6cc0 msgr2=0x557966caa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:22.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.134+0000 7f05643adb80 1 -- 172.21.15.37:0/2673076794 shutdown_connections 2024-06-10T00:59:22.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.134+0000 7f05643adb80 1 -- 172.21.15.37:0/2673076794 wait complete. 2024-06-10T00:59:22.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.134+0000 7f05643adb80 20 client.15885 trim_cache size 0 max 16384 2024-06-10T00:59:22.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.151+0000 7f05643adb80 1 Processor -- start 2024-06-10T00:59:22.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.151+0000 7f05643adb80 1 -- start start 2024-06-10T00:59:22.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.151+0000 7f05643adb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557966cabf20 0x557966cab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:22.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.151+0000 7f05643adb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557966cb1640 0x557966ca69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:22.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.151+0000 7f05643adb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557966cb1f60 0x557966c9fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:22.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.151+0000 7f05643adb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f05541b2e40 con 0x557966cabf20 2024-06-10T00:59:22.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.151+0000 7f05643adb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f05541b06d0 con 0x557966cb1640 2024-06-10T00:59:22.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.151+0000 7f05643adb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f05541ae520 con 0x557966cb1f60 2024-06-10T00:59:22.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.151+0000 7f05537fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557966cabf20 0x557966cab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:22.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.151+0000 7f05537fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557966cabf20 0x557966cab9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50060/0 (socket says 172.21.15.37:50060) 2024-06-10T00:59:22.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.151+0000 7f05537fe700 1 -- 172.21.15.37:0/4113763689 learned_addr learned my addr 172.21.15.37:0/4113763689 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:22.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.152+0000 7f0552ffd700 1 --2- 172.21.15.37:0/4113763689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557966cb1640 0x557966ca69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:22.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.152+0000 7f0553fff700 1 --2- 172.21.15.37:0/4113763689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557966cb1f60 0x557966c9fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:22.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.152+0000 7f05537fe700 1 -- 172.21.15.37:0/4113763689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557966cb1f60 msgr2=0x557966c9fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:22.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.152+0000 7f05537fe700 1 --2- 172.21.15.37:0/4113763689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557966cb1f60 0x557966c9fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.152+0000 7f05537fe700 1 -- 172.21.15.37:0/4113763689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557966cb1640 msgr2=0x557966ca69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:22.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.152+0000 7f05537fe700 1 --2- 172.21.15.37:0/4113763689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557966cb1640 0x557966ca69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.152+0000 7f05537fe700 1 -- 172.21.15.37:0/4113763689 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557966cc8330 con 0x557966cabf20 2024-06-10T00:59:22.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.152+0000 7f0553fff700 1 --2- 172.21.15.37:0/4113763689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557966cb1f60 0x557966c9fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:59:22.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.152+0000 7f05537fe700 1 --2- 172.21.15.37:0/4113763689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557966cabf20 0x557966cab9e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f054401c890 tx=0x7f054401c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ef7c45dbd222b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:22.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.152+0000 7f05397fa700 1 -- 172.21.15.37:0/4113763689 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054401c930 con 0x557966cabf20 2024-06-10T00:59:22.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.152+0000 7f05397fa700 1 -- 172.21.15.37:0/4113763689 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0544038070 con 0x557966cabf20 2024-06-10T00:59:22.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.152+0000 7f05397fa700 1 -- 172.21.15.37:0/4113763689 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0544004950 con 0x557966cabf20 2024-06-10T00:59:22.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.153+0000 7f05643adb80 1 -- 172.21.15.37:0/4113763689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557966cabf20 msgr2=0x557966cab9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:22.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.153+0000 7f05643adb80 1 --2- 172.21.15.37:0/4113763689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557966cabf20 0x557966cab9e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f054401c890 tx=0x7f054401c8c0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.153+0000 7f05643adb80 1 -- 172.21.15.37:0/4113763689 shutdown_connections 2024-06-10T00:59:22.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.153+0000 7f05643adb80 1 --2- 172.21.15.37:0/4113763689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557966cb1f60 0x557966c9fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.153+0000 7f05643adb80 1 --2- 172.21.15.37:0/4113763689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557966cabf20 0x557966cab9e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.153+0000 7f05643adb80 1 --2- 172.21.15.37:0/4113763689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557966cb1640 0x557966ca69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.153+0000 7f05643adb80 1 -- 172.21.15.37:0/4113763689 >> 172.21.15.37:0/4113763689 conn(0x557966bd6cc0 msgr2=0x557966cb4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:22.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.153+0000 7f05643adb80 1 -- 172.21.15.37:0/4113763689 shutdown_connections 2024-06-10T00:59:22.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.153+0000 7f05643adb80 1 -- 172.21.15.37:0/4113763689 wait complete. 2024-06-10T00:59:22.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.154+0000 7f05643adb80 1 -- start start 2024-06-10T00:59:22.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.154+0000 7f05643adb80 1 Processor -- start 2024-06-10T00:59:22.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.154+0000 7f05643adb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557966cabf20 0x557966cbe660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:22.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.154+0000 7f05643adb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557966cb1640 0x557966cccf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:22.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.154+0000 7f05643adb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557966cb1f60 0x557966ccd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:22.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.154+0000 7f05643adb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f05541aeb90 con 0x557966cb1f60 2024-06-10T00:59:22.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.154+0000 7f05643adb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f05541b0000 con 0x557966cb1640 2024-06-10T00:59:22.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.154+0000 7f05643adb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f05541af260 con 0x557966cabf20 2024-06-10T00:59:22.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.154+0000 7f05643adb80 10 client.0 _ll_register_callbacks cb 0x557966b97920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:59:22.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.154+0000 7f05537fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557966cabf20 0x557966cbe660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:22.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.154+0000 7f0553fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557966cb1f60 0x557966ccd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:22.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.154+0000 7f05537fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557966cabf20 0x557966cbe660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:43164/0 (socket says 172.21.15.37:43164) 2024-06-10T00:59:22.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.154+0000 7f05537fe700 1 -- 172.21.15.37:0/321827973 learned_addr learned my addr 172.21.15.37:0/321827973 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:22.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.155+0000 7f0552ffd700 1 --2- 172.21.15.37:0/321827973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557966cb1640 0x557966cccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:22.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.155+0000 7f0553fff700 1 -- 172.21.15.37:0/321827973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557966cabf20 msgr2=0x557966cbe660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:22.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.155+0000 7f0553fff700 1 --2- 172.21.15.37:0/321827973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557966cabf20 0x557966cbe660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.155+0000 7f0553fff700 1 -- 172.21.15.37:0/321827973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557966cb1640 msgr2=0x557966cccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:22.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.155+0000 7f0553fff700 1 --2- 172.21.15.37:0/321827973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557966cb1640 0x557966cccf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.155+0000 7f0553fff700 1 -- 172.21.15.37:0/321827973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557966ccdd80 con 0x557966cb1f60 2024-06-10T00:59:22.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.155+0000 7f05537fe700 1 --2- 172.21.15.37:0/321827973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557966cabf20 0x557966cbe660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:59:22.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.155+0000 7f0552ffd700 1 --2- 172.21.15.37:0/321827973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557966cb1640 0x557966cccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:59:22.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.155+0000 7f0553fff700 1 --2- 172.21.15.37:0/321827973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557966cb1f60 0x557966ccd540 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f054c009fd0 tx=0x7f054c004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4716ed564814ec60 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:22.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.155+0000 7f0532ffd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:59:22.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.155+0000 7f0532ffd700 1 -- 172.21.15.37:0/321827973 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c019050 con 0x557966cb1f60 2024-06-10T00:59:22.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.155+0000 7f0532ffd700 1 -- 172.21.15.37:0/321827973 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f054c01c310 con 0x557966cb1f60 2024-06-10T00:59:22.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.155+0000 7f05643adb80 1 -- 172.21.15.37:0/321827973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0544038070 con 0x557966cb1f60 2024-06-10T00:59:22.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.155+0000 7f0532ffd700 1 -- 172.21.15.37:0/321827973 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f054c01c510 con 0x557966cb1f60 2024-06-10T00:59:22.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.155+0000 7f05643adb80 10 client.15891 fetch_fsmap learned FSMap version 27 2024-06-10T00:59:22.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.155+0000 7f05643adb80 1 -- 172.21.15.37:0/321827973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557966d30690 con 0x557966cb1f60 2024-06-10T00:59:22.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.156+0000 7f0532ffd700 1 -- 172.21.15.37:0/321827973 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0544038070 con 0x557966cb1f60 2024-06-10T00:59:22.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.156+0000 7f0532ffd700 1 -- 172.21.15.37:0/321827973 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f054c0b9b30 con 0x557966cb1f60 2024-06-10T00:59:22.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.156+0000 7f05643adb80 10 client.15891 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:59:22.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.156+0000 7f05643adb80 10 client.15891 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-10T00:59:22.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.156+0000 7f05643adb80 20 client.15891 populate_metadata read hostname 'smithi037' 2024-06-10T00:59:22.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.156+0000 7f05643adb80 1 --2- 172.21.15.37:0/321827973 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x557966d30d40 0x557966d33160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:22.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.156+0000 7f05643adb80 4 client.15891 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:59:22.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.156+0000 7f05643adb80 1 -- 172.21.15.37:0/321827973 --> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557966c05290 con 0x557966d30d40 2024-06-10T00:59:22.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.156+0000 7f05537fe700 1 --2- 172.21.15.37:0/321827973 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x557966d30d40 0x557966d33160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:22.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.157+0000 7f05537fe700 1 --2- 172.21.15.37:0/321827973 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x557966d30d40 0x557966d33160 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1a02547d35e9011c server_cookie=8e2e7e3e640cd6f4 in_seq=0 out_seq=0 2024-06-10T00:59:22.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.157+0000 7f0532ffd700 10 client.15891 ms_handle_connect on v2:172.21.15.167:6836/2720148065 2024-06-10T00:59:22.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.157+0000 7f0532ffd700 1 -- 172.21.15.37:0/321827973 <== mds.2 v2:172.21.15.167:6836/2720148065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557966c05290 con 0x557966d30d40 2024-06-10T00:59:22.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.157+0000 7f0532ffd700 10 client.15891 handle_command_reply: tid=0 2024-06-10T00:59:22.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.158+0000 7f05643adb80 1 client.15891 shutdown 2024-06-10T00:59:22.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.158+0000 7f05643adb80 1 -- 172.21.15.37:0/321827973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557966cb1f60 msgr2=0x557966ccd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:22.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.158+0000 7f05643adb80 1 --2- 172.21.15.37:0/321827973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557966cb1f60 0x557966ccd540 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f054c009fd0 tx=0x7f054c004f50 comp rx=0 tx=0).stop 2024-06-10T00:59:22.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.158+0000 7f05643adb80 1 -- 172.21.15.37:0/321827973 shutdown_connections 2024-06-10T00:59:22.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.158+0000 7f05643adb80 1 --2- 172.21.15.37:0/321827973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557966cb1f60 0x557966ccd540 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.158+0000 7f05643adb80 1 --2- 172.21.15.37:0/321827973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557966cabf20 0x557966cbe660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.158+0000 7f05643adb80 1 --2- 172.21.15.37:0/321827973 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x557966d30d40 0x557966d33160 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.158+0000 7f0532ffd700 0 client.15891 ms_handle_reset on v2:172.21.15.167:6836/2720148065 2024-06-10T00:59:22.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.158+0000 7f05643adb80 1 --2- 172.21.15.37:0/321827973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557966cb1640 0x557966cccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.158+0000 7f05643adb80 1 -- 172.21.15.37:0/321827973 >> 172.21.15.37:0/321827973 conn(0x557966bd6cc0 msgr2=0x557966c9fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:22.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.158+0000 7f05643adb80 1 -- 172.21.15.37:0/321827973 shutdown_connections 2024-06-10T00:59:22.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.158+0000 7f05643adb80 1 -- 172.21.15.37:0/321827973 wait complete. 2024-06-10T00:59:22.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.158+0000 7f05643adb80 20 client.15891 trim_cache size 0 max 16384 2024-06-10T00:59:22.169 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T00:59:22.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.159+0000 7f0559b2a700 1 -- 172.21.15.37:0/4231226643 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f053c0a2ca0 msgr2=0x7f053c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:22.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.159+0000 7f0559b2a700 1 --2- 172.21.15.37:0/4231226643 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f053c0a2ca0 0x7f053c0a5160 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0544005950 tx=0x7f054400b500 comp rx=0 tx=0).stop 2024-06-10T00:59:22.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.159+0000 7f0559b2a700 1 -- 172.21.15.37:0/4231226643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05541073c0 msgr2=0x7f055410f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:22.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.159+0000 7f0559b2a700 1 --2- 172.21.15.37:0/4231226643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05541073c0 0x7f055410f710 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f054c007f10 tx=0x7f054c00d3b0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.159+0000 7f0559b2a700 1 -- 172.21.15.37:0/4231226643 shutdown_connections 2024-06-10T00:59:22.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.159+0000 7f0559b2a700 1 --2- 172.21.15.37:0/4231226643 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f053c0a2ca0 0x7f053c0a5160 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.159+0000 7f0559b2a700 1 --2- 172.21.15.37:0/4231226643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05541073c0 0x7f055410f710 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.159+0000 7f0559b2a700 1 --2- 172.21.15.37:0/4231226643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0554101380 0x7f055410ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.159+0000 7f0559b2a700 1 --2- 172.21.15.37:0/4231226643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0554101d40 0x7f055410f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:22.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.159+0000 7f0559b2a700 1 -- 172.21.15.37:0/4231226643 >> 172.21.15.37:0/4231226643 conn(0x7f0554078330 msgr2=0x7f05541000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:22.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.159+0000 7f0559b2a700 1 -- 172.21.15.37:0/4231226643 shutdown_connections 2024-06-10T00:59:22.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:22.160+0000 7f0559b2a700 1 -- 172.21.15.37:0/4231226643 wait complete. 2024-06-10T00:59:22.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:21 smithi037 ceph-mon[32192]: pgmap v510: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.8 KiB/s wr, 12 op/s 2024-06-10T00:59:22.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:21 smithi167 ceph-mon[36917]: pgmap v510: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.8 KiB/s wr, 12 op/s 2024-06-10T00:59:22.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:21 smithi094 ceph-mon[36708]: pgmap v510: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.8 KiB/s wr, 12 op/s 2024-06-10T00:59:22.580 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:22.581 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:23.287 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:59:23.287 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:59:23.381 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:23.382 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:23.382 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 30 -lt 100 ']' 2024-06-10T00:59:23.382 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 30 + 1 2024-06-10T00:59:23.539 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=31 2024-06-10T00:59:23.539 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:59:23.694 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:23.694 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:23.695 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:59:23.695 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:59:23.696 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:59:24.230 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:23 smithi037 ceph-mon[32192]: pgmap v511: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-06-10T00:59:24.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:23 smithi167 ceph-mon[36917]: pgmap v511: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-06-10T00:59:24.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:23 smithi094 ceph-mon[36708]: pgmap v511: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-06-10T00:59:25.308 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:25.308 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:25.855 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:59:25.855 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:59:25.885 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:25.885 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:25.885 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 31 -lt 100 ']' 2024-06-10T00:59:25.886 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 31 + 1 2024-06-10T00:59:26.006 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=32 2024-06-10T00:59:26.006 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:59:26.151 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:26.152 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:26.153 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:59:26.153 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:59:26.153 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:59:26.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:25 smithi167 ceph-mon[36917]: pgmap v512: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.0 KiB/s wr, 12 op/s 2024-06-10T00:59:26.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:25 smithi094 ceph-mon[36708]: pgmap v512: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.0 KiB/s wr, 12 op/s 2024-06-10T00:59:26.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:25 smithi037 ceph-mon[32192]: pgmap v512: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.0 KiB/s wr, 12 op/s 2024-06-10T00:59:26.680 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T00:59:27.701 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:27.701 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:27.863 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:59:27.863 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:59:27.894 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:27.894 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:27.894 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 32 -lt 100 ']' 2024-06-10T00:59:27.895 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 32 + 1 2024-06-10T00:59:28.056 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=33 2024-06-10T00:59:28.056 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:59:28.071 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:28.072 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:28.073 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:59:28.073 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:59:28.073 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:59:28.242 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:27 smithi037 ceph-mon[32192]: pgmap v513: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-10T00:59:28.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:27 smithi167 ceph-mon[36917]: pgmap v513: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-10T00:59:28.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:27 smithi094 ceph-mon[36708]: pgmap v513: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-10T00:59:29.357 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:29.357 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:29.671 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:59:29.671 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:59:29.702 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:29.702 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:29.702 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 33 -lt 100 ']' 2024-06-10T00:59:29.702 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 33 + 1 2024-06-10T00:59:29.746 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=34 2024-06-10T00:59:29.746 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:59:29.791 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:29.791 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:29.792 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:59:29.792 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:59:29.792 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:59:30.297 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:29 smithi037 ceph-mon[32192]: pgmap v514: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.9 KiB/s wr, 11 op/s 2024-06-10T00:59:30.297 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:29 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:59:30.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:29 smithi167 ceph-mon[36917]: pgmap v514: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.9 KiB/s wr, 11 op/s 2024-06-10T00:59:30.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:29 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:59:30.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:29 smithi094 ceph-mon[36708]: pgmap v514: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.9 KiB/s wr, 11 op/s 2024-06-10T00:59:30.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:29 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T00:59:31.223 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:31.223 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:31.654 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:59:31.654 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:59:31.668 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:31.669 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:31.669 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 34 -lt 100 ']' 2024-06-10T00:59:31.669 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 34 + 1 2024-06-10T00:59:31.780 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=35 2024-06-10T00:59:31.780 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:59:31.803 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:31.804 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:31.804 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:59:31.805 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:59:31.805 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:59:32.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:31 smithi167 ceph-mon[36917]: pgmap v515: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.7 KiB/s wr, 6 op/s 2024-06-10T00:59:32.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:31 smithi094 ceph-mon[36708]: pgmap v515: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.7 KiB/s wr, 6 op/s 2024-06-10T00:59:32.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:31 smithi037 ceph-mon[32192]: pgmap v515: 129 pgs: 129 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.7 KiB/s wr, 6 op/s 2024-06-10T00:59:33.381 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:33.381 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:34.013 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:59:34.013 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:59:34.035 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:34.035 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:34.036 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 35 -lt 100 ']' 2024-06-10T00:59:34.036 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 35 + 1 2024-06-10T00:59:34.163 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=36 2024-06-10T00:59:34.163 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:59:34.178 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:34.178 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:34.179 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:59:34.180 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:59:34.180 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:59:34.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:33 smithi167 ceph-mon[36917]: pgmap v516: 129 pgs: 129 active+clean; 1.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.4 KiB/s wr, 9 op/s 2024-06-10T00:59:34.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:33 smithi094 ceph-mon[36708]: pgmap v516: 129 pgs: 129 active+clean; 1.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.4 KiB/s wr, 9 op/s 2024-06-10T00:59:34.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:33 smithi037 ceph-mon[32192]: pgmap v516: 129 pgs: 129 active+clean; 1.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.4 KiB/s wr, 9 op/s 2024-06-10T00:59:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.697+0000 7fc40f9ee640 1 -- 172.21.15.37:0/648937626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc408101a30 msgr2=0x7fc408101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.697+0000 7fc40f9ee640 1 --2- 172.21.15.37:0/648937626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc408101a30 0x7fc408101e50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc404009ca0 tx=0x7fc40402a8c0 comp rx=0 tx=0).stop 2024-06-10T00:59:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.697+0000 7fc40f9ee640 1 -- 172.21.15.37:0/648937626 shutdown_connections 2024-06-10T00:59:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.697+0000 7fc40f9ee640 1 --2- 172.21.15.37:0/648937626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4081035e0 0x7fc408109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.697+0000 7fc40f9ee640 1 --2- 172.21.15.37:0/648937626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc408102c20 0x7fc4081030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.697+0000 7fc40f9ee640 1 --2- 172.21.15.37:0/648937626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc408101a30 0x7fc408101e50 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.697+0000 7fc40f9ee640 1 -- 172.21.15.37:0/648937626 >> 172.21.15.37:0/648937626 conn(0x7fc4080fcfc0 msgr2=0x7fc4080ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.697+0000 7fc40f9ee640 1 -- 172.21.15.37:0/648937626 shutdown_connections 2024-06-10T00:59:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.697+0000 7fc40f9ee640 1 -- 172.21.15.37:0/648937626 wait complete. 2024-06-10T00:59:34.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.740+0000 7fc40f9ee640 1 Processor -- start 2024-06-10T00:59:34.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.740+0000 7fc40f9ee640 1 -- start start 2024-06-10T00:59:34.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.741+0000 7fc40f9ee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc408101a30 0x7fc408075750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:34.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.741+0000 7fc40f9ee640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc408102c20 0x7fc408073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:34.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.741+0000 7fc40f9ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4081035e0 0x7fc4080742d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.741+0000 7fc40f9ee640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc40810d6b0 con 0x7fc408101a30 2024-06-10T00:59:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.741+0000 7fc40f9ee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc40810d530 con 0x7fc4081035e0 2024-06-10T00:59:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.741+0000 7fc40f9ee640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc40810d830 con 0x7fc408102c20 2024-06-10T00:59:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.741+0000 7fc40d78a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc408101a30 0x7fc408075750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.741+0000 7fc40d78a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc408101a30 0x7fc408075750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52394/0 (socket says 172.21.15.37:52394) 2024-06-10T00:59:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.741+0000 7fc40d78a640 1 -- 172.21.15.37:0/3675190646 learned_addr learned my addr 172.21.15.37:0/3675190646 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.741+0000 7fc40df8b640 1 --2- 172.21.15.37:0/3675190646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4081035e0 0x7fc4080742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.741+0000 7fc40cf89640 1 --2- 172.21.15.37:0/3675190646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc408102c20 0x7fc408073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.741+0000 7fc40d78a640 1 -- 172.21.15.37:0/3675190646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc408102c20 msgr2=0x7fc408073d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.741+0000 7fc40d78a640 1 --2- 172.21.15.37:0/3675190646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc408102c20 0x7fc408073d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.741+0000 7fc40d78a640 1 -- 172.21.15.37:0/3675190646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4081035e0 msgr2=0x7fc4080742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.741+0000 7fc40d78a640 1 --2- 172.21.15.37:0/3675190646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4081035e0 0x7fc4080742d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.741+0000 7fc40d78a640 1 -- 172.21.15.37:0/3675190646 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc408074b60 con 0x7fc408101a30 2024-06-10T00:59:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.741+0000 7fc40df8b640 1 --2- 172.21.15.37:0/3675190646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4081035e0 0x7fc4080742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:59:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.741+0000 7fc40d78a640 1 --2- 172.21.15.37:0/3675190646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc408101a30 0x7fc408075750 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc404009bf0 tx=0x7fc404031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.755+0000 7fc3fe7fc640 1 -- 172.21.15.37:0/3675190646 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40402fa20 con 0x7fc408101a30 2024-06-10T00:59:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.755+0000 7fc3fe7fc640 1 -- 172.21.15.37:0/3675190646 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc40402fbc0 con 0x7fc408101a30 2024-06-10T00:59:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.755+0000 7fc3fe7fc640 1 -- 172.21.15.37:0/3675190646 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404046750 con 0x7fc408101a30 2024-06-10T00:59:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.755+0000 7fc40f9ee640 1 -- 172.21.15.37:0/3675190646 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4081a1d80 con 0x7fc408101a30 2024-06-10T00:59:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.756+0000 7fc40f9ee640 1 -- 172.21.15.37:0/3675190646 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4081a21c0 con 0x7fc408101a30 2024-06-10T00:59:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.757+0000 7fc3fe7fc640 1 -- 172.21.15.37:0/3675190646 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc40402fd70 con 0x7fc408101a30 2024-06-10T00:59:34.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.758+0000 7fc3fe7fc640 1 --2- 172.21.15.37:0/3675190646 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3f40a2da0 0x7fc3f40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:34.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.758+0000 7fc40cf89640 1 --2- 172.21.15.37:0/3675190646 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3f40a2da0 0x7fc3f40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:34.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.758+0000 7fc3fe7fc640 1 -- 172.21.15.37:0/3675190646 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fc4040f3170 con 0x7fc408101a30 2024-06-10T00:59:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:34.758+0000 7fc40cf89640 1 --2- 172.21.15.37:0/3675190646 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3f40a2da0 0x7fc3f40a5260 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc3f80095f0 tx=0x7fc3f8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:35.183 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:59:35.253 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:35 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:35.253 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:35 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:59:35.253 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:35 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:59:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.300+0000 7fc4127de740 1 Processor -- start 2024-06-10T00:59:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.300+0000 7fc4127de740 1 -- start start 2024-06-10T00:59:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.300+0000 7fc4127de740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561e6145010 0x5561e6145430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.300+0000 7fc4127de740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561e6145a00 0x5561e614a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.300+0000 7fc4127de740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561e614ab70 0x5561e614d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.300+0000 7fc4127de740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5561e60a84f0 con 0x5561e6145a00 2024-06-10T00:59:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.300+0000 7fc4127de740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5561e6046870 con 0x5561e614ab70 2024-06-10T00:59:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.300+0000 7fc4127de740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5561e60a44b0 con 0x5561e6145010 2024-06-10T00:59:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.301+0000 7fc40cf89640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561e6145a00 0x5561e614a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.301+0000 7fc40cf89640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561e6145a00 0x5561e614a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52398/0 (socket says 172.21.15.37:52398) 2024-06-10T00:59:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.301+0000 7fc40cf89640 1 -- 172.21.15.37:0/2734085109 learned_addr learned my addr 172.21.15.37:0/2734085109 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.301+0000 7fc40df8b640 1 --2- 172.21.15.37:0/2734085109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561e614ab70 0x5561e614d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.301+0000 7fc40d78a640 1 --2- 172.21.15.37:0/2734085109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561e6145010 0x5561e6145430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.301+0000 7fc40cf89640 1 -- 172.21.15.37:0/2734085109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561e6145010 msgr2=0x5561e6145430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.301+0000 7fc40cf89640 1 --2- 172.21.15.37:0/2734085109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561e6145010 0x5561e6145430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.301+0000 7fc40cf89640 1 -- 172.21.15.37:0/2734085109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561e614ab70 msgr2=0x5561e614d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.301+0000 7fc40cf89640 1 --2- 172.21.15.37:0/2734085109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561e614ab70 0x5561e614d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.301+0000 7fc40cf89640 1 -- 172.21.15.37:0/2734085109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561e60629b0 con 0x5561e6145a00 2024-06-10T00:59:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.301+0000 7fc40cf89640 1 --2- 172.21.15.37:0/2734085109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561e6145a00 0x5561e614a500 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8010dd0 tx=0x7fc3f8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27eb485ef00c34b5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.301+0000 7fc3f37fe640 1 -- 172.21.15.37:0/2734085109 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8023070 con 0x5561e6145a00 2024-06-10T00:59:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.301+0000 7fc3f37fe640 1 -- 172.21.15.37:0/2734085109 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3f8009390 con 0x5561e6145a00 2024-06-10T00:59:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.301+0000 7fc3f37fe640 1 -- 172.21.15.37:0/2734085109 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f80118d0 con 0x5561e6145a00 2024-06-10T00:59:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.302+0000 7fc4127de740 1 -- 172.21.15.37:0/2734085109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561e6145a00 msgr2=0x5561e614a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.302+0000 7fc4127de740 1 --2- 172.21.15.37:0/2734085109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561e6145a00 0x5561e614a500 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8010dd0 tx=0x7fc3f8013000 comp rx=0 tx=0).stop 2024-06-10T00:59:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.302+0000 7fc4127de740 1 -- 172.21.15.37:0/2734085109 shutdown_connections 2024-06-10T00:59:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.302+0000 7fc4127de740 1 --2- 172.21.15.37:0/2734085109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561e614ab70 0x5561e614d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.302+0000 7fc4127de740 1 --2- 172.21.15.37:0/2734085109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561e6145a00 0x5561e614a500 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.302+0000 7fc4127de740 1 --2- 172.21.15.37:0/2734085109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561e6145010 0x5561e6145430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.302+0000 7fc4127de740 1 -- 172.21.15.37:0/2734085109 >> 172.21.15.37:0/2734085109 conn(0x5561e6140680 msgr2=0x5561e6140a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.302+0000 7fc4127de740 1 -- 172.21.15.37:0/2734085109 shutdown_connections 2024-06-10T00:59:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.302+0000 7fc4127de740 1 -- 172.21.15.37:0/2734085109 wait complete. 2024-06-10T00:59:35.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:35 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:35.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:35 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:59:35.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:35 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:59:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.338+0000 7fc4127de740 1 -- start start 2024-06-10T00:59:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.346+0000 7fc4127de740 1 Processor -- start 2024-06-10T00:59:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.346+0000 7fc4127de740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561e6145010 0x5561e614f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.346+0000 7fc4127de740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561e6145a00 0x5561e614f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.346+0000 7fc4127de740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561e614ab70 0x5561e61567e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.346+0000 7fc4127de740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5561e614db20 con 0x5561e614ab70 2024-06-10T00:59:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.346+0000 7fc40df8b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561e614ab70 0x5561e61567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.346+0000 7fc40df8b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561e614ab70 0x5561e61567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52400/0 (socket says 172.21.15.37:52400) 2024-06-10T00:59:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.346+0000 7fc40df8b640 1 -- 172.21.15.37:0/822333759 learned_addr learned my addr 172.21.15.37:0/822333759 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.346+0000 7fc40cf89640 1 --2- 172.21.15.37:0/822333759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561e6145a00 0x5561e614f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.346+0000 7fc40d78a640 1 --2- 172.21.15.37:0/822333759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561e6145010 0x5561e614f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:35.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:35 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T00:59:35.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:35 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T00:59:35.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:35 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T00:59:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.452+0000 7fc4127de740 1 -- 172.21.15.37:0/822333759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5561e6031ae0 con 0x5561e6145010 2024-06-10T00:59:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.452+0000 7fc4127de740 1 -- 172.21.15.37:0/822333759 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5561e614dca0 con 0x5561e6145a00 2024-06-10T00:59:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.452+0000 7fc40df8b640 1 -- 172.21.15.37:0/822333759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561e6145a00 msgr2=0x5561e614f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.452+0000 7fc40df8b640 1 --2- 172.21.15.37:0/822333759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561e6145a00 0x5561e614f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.452+0000 7fc40df8b640 1 -- 172.21.15.37:0/822333759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561e6145010 msgr2=0x5561e614f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.452+0000 7fc40df8b640 1 --2- 172.21.15.37:0/822333759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561e6145010 0x5561e614f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.452+0000 7fc40df8b640 1 -- 172.21.15.37:0/822333759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561e5dbc210 con 0x5561e614ab70 2024-06-10T00:59:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.452+0000 7fc40d78a640 1 --2- 172.21.15.37:0/822333759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561e6145010 0x5561e614f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:59:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.452+0000 7fc40df8b640 1 --2- 172.21.15.37:0/822333759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561e614ab70 0x5561e61567e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc400012700 tx=0x7fc400012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5408f770e16b838 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.452+0000 7fc3f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:59:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.452+0000 7fc3f0ff9640 1 -- 172.21.15.37:0/822333759 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc400011b50 con 0x5561e614ab70 2024-06-10T00:59:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.452+0000 7fc3f0ff9640 1 -- 172.21.15.37:0/822333759 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fc400011cf0 con 0x5561e614ab70 2024-06-10T00:59:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.453+0000 7fc4127de740 10 client.0 _ll_register_callbacks cb 0x5561e606a9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:59:35.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.455+0000 7fc4127de740 1 -- 172.21.15.37:0/822333759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3f8009390 con 0x5561e614ab70 2024-06-10T00:59:35.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.455+0000 7fc3f0ff9640 1 -- 172.21.15.37:0/822333759 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc400005020 con 0x5561e614ab70 2024-06-10T00:59:35.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.455+0000 7fc4127de740 10 client.15903 fetch_fsmap learned FSMap version 27 2024-06-10T00:59:35.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.455+0000 7fc4127de740 1 -- 172.21.15.37:0/822333759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=27}) -- 0x5561e5e88620 con 0x5561e614ab70 2024-06-10T00:59:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.457+0000 7fc3f0ff9640 1 -- 172.21.15.37:0/822333759 <== mon.0 v2:172.21.15.37:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc4000051c0 con 0x5561e614ab70 2024-06-10T00:59:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.457+0000 7fc4127de740 10 client.15903 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:59:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.457+0000 7fc4127de740 10 client.15903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T00:59:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.457+0000 7fc4127de740 20 client.15903 populate_metadata read hostname 'smithi037' 2024-06-10T00:59:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.457+0000 7fc4127de740 1 --2- 172.21.15.37:0/822333759 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5561e615af60 0x5561e615d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.457+0000 7fc4127de740 4 client.15903 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:59:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.457+0000 7fc4127de740 1 -- 172.21.15.37:0/822333759 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561e60bbf10 con 0x5561e615af60 2024-06-10T00:59:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.457+0000 7fc40d78a640 1 --2- 172.21.15.37:0/822333759 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5561e615af60 0x5561e615d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.458+0000 7fc40d78a640 1 --2- 172.21.15.37:0/822333759 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5561e615af60 0x5561e615d420 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4593c5253a9a39a9 server_cookie=e856eb91a3af4d04 in_seq=0 out_seq=0 2024-06-10T00:59:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.458+0000 7fc3f0ff9640 10 client.15903 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:59:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.459+0000 7fc3f0ff9640 1 -- 172.21.15.37:0/822333759 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5561e60bbf10 con 0x5561e615af60 2024-06-10T00:59:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.459+0000 7fc3f0ff9640 10 client.15903 handle_command_reply: tid=0 2024-06-10T00:59:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.459+0000 7fc4127de740 1 client.15903 shutdown 2024-06-10T00:59:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.459+0000 7fc4127de740 1 -- 172.21.15.37:0/822333759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561e614ab70 msgr2=0x5561e61567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.459+0000 7fc4127de740 1 --2- 172.21.15.37:0/822333759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561e614ab70 0x5561e61567e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc400012700 tx=0x7fc400012bc0 comp rx=0 tx=0).stop 2024-06-10T00:59:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.459+0000 7fc4127de740 1 -- 172.21.15.37:0/822333759 shutdown_connections 2024-06-10T00:59:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.459+0000 7fc4127de740 1 --2- 172.21.15.37:0/822333759 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5561e615af60 0x5561e615d420 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.459+0000 7fc4127de740 1 --2- 172.21.15.37:0/822333759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561e614ab70 0x5561e61567e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.459+0000 7fc4127de740 1 --2- 172.21.15.37:0/822333759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561e6145a00 0x5561e614f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.459+0000 7fc4127de740 1 --2- 172.21.15.37:0/822333759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561e6145010 0x5561e614f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.459+0000 7fc3f0ff9640 0 client.15903 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:59:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.459+0000 7fc4127de740 1 -- 172.21.15.37:0/822333759 >> 172.21.15.37:0/822333759 conn(0x5561e6140680 msgr2=0x5561e613f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.459+0000 7fc4127de740 1 -- 172.21.15.37:0/822333759 shutdown_connections 2024-06-10T00:59:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.459+0000 7fc4127de740 1 -- 172.21.15.37:0/822333759 wait complete. 2024-06-10T00:59:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.460+0000 7fc4127de740 20 client.15903 trim_cache size 0 max 16384 2024-06-10T00:59:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.475+0000 7fc4127de740 1 Processor -- start 2024-06-10T00:59:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.476+0000 7fc4127de740 1 -- start start 2024-06-10T00:59:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.476+0000 7fc4127de740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561e6145010 0x5561e614f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:35.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.476+0000 7fc4127de740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561e6145a00 0x5561e61567a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.476+0000 7fc4127de740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561e614ab70 0x5561e614f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.476+0000 7fc4127de740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc4081af220 con 0x5561e614ab70 2024-06-10T00:59:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.476+0000 7fc4127de740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc4081aca80 con 0x5561e6145010 2024-06-10T00:59:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.476+0000 7fc4127de740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc4081aa8d0 con 0x5561e6145a00 2024-06-10T00:59:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.476+0000 7fc40df8b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561e614ab70 0x5561e614f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.476+0000 7fc40df8b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561e614ab70 0x5561e614f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52406/0 (socket says 172.21.15.37:52406) 2024-06-10T00:59:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.476+0000 7fc40df8b640 1 -- 172.21.15.37:0/880118605 learned_addr learned my addr 172.21.15.37:0/880118605 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.476+0000 7fc40cf89640 1 --2- 172.21.15.37:0/880118605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561e6145a00 0x5561e61567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.476+0000 7fc40d78a640 1 --2- 172.21.15.37:0/880118605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561e6145010 0x5561e614f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.476+0000 7fc40df8b640 1 -- 172.21.15.37:0/880118605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561e6145a00 msgr2=0x5561e61567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.476+0000 7fc40df8b640 1 --2- 172.21.15.37:0/880118605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561e6145a00 0x5561e61567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.476+0000 7fc40df8b640 1 -- 172.21.15.37:0/880118605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561e6145010 msgr2=0x5561e614f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.476+0000 7fc40df8b640 1 --2- 172.21.15.37:0/880118605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561e6145010 0x5561e614f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.476+0000 7fc40df8b640 1 -- 172.21.15.37:0/880118605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561e61404b0 con 0x5561e614ab70 2024-06-10T00:59:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.476+0000 7fc40df8b640 1 --2- 172.21.15.37:0/880118605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561e614ab70 0x5561e614f7a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc400012c00 tx=0x7fc40000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb8606b46a55e49 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.477+0000 7fc3f2ffd640 1 -- 172.21.15.37:0/880118605 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc400011b50 con 0x5561e614ab70 2024-06-10T00:59:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.477+0000 7fc3f2ffd640 1 -- 172.21.15.37:0/880118605 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3f8009390 con 0x5561e614ab70 2024-06-10T00:59:35.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.477+0000 7fc3f2ffd640 1 -- 172.21.15.37:0/880118605 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40001c6d0 con 0x5561e614ab70 2024-06-10T00:59:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.477+0000 7fc4127de740 1 -- 172.21.15.37:0/880118605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561e614ab70 msgr2=0x5561e614f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.477+0000 7fc4127de740 1 --2- 172.21.15.37:0/880118605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561e614ab70 0x5561e614f7a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc400012c00 tx=0x7fc40000f000 comp rx=0 tx=0).stop 2024-06-10T00:59:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.477+0000 7fc4127de740 1 -- 172.21.15.37:0/880118605 shutdown_connections 2024-06-10T00:59:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.477+0000 7fc4127de740 1 --2- 172.21.15.37:0/880118605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561e614ab70 0x5561e614f7a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.477+0000 7fc4127de740 1 --2- 172.21.15.37:0/880118605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561e6145a00 0x5561e61567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.477+0000 7fc4127de740 1 --2- 172.21.15.37:0/880118605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561e6145010 0x5561e614f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.477+0000 7fc4127de740 1 -- 172.21.15.37:0/880118605 >> 172.21.15.37:0/880118605 conn(0x5561e6140680 msgr2=0x5561e6146c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.477+0000 7fc4127de740 1 -- 172.21.15.37:0/880118605 shutdown_connections 2024-06-10T00:59:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.478+0000 7fc4127de740 1 -- 172.21.15.37:0/880118605 wait complete. 2024-06-10T00:59:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.478+0000 7fc4127de740 1 -- start start 2024-06-10T00:59:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.478+0000 7fc4127de740 1 Processor -- start 2024-06-10T00:59:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.478+0000 7fc4127de740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561e6145010 0x5561e6153b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.478+0000 7fc4127de740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561e6145a00 0x5561e61540a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.478+0000 7fc4127de740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561e614ab70 0x5561e61550b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.478+0000 7fc4127de740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc4081aaf40 con 0x5561e614ab70 2024-06-10T00:59:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.478+0000 7fc4127de740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc4081ac3b0 con 0x5561e6145010 2024-06-10T00:59:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.478+0000 7fc4127de740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc4081ab610 con 0x5561e6145a00 2024-06-10T00:59:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.478+0000 7fc4127de740 10 client.0 _ll_register_callbacks cb 0x7fc4000097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:59:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.478+0000 7fc40df8b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561e614ab70 0x5561e61550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.478+0000 7fc40df8b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561e614ab70 0x5561e61550b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52416/0 (socket says 172.21.15.37:52416) 2024-06-10T00:59:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.478+0000 7fc40df8b640 1 -- 172.21.15.37:0/2840780643 learned_addr learned my addr 172.21.15.37:0/2840780643 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.478+0000 7fc40cf89640 1 --2- 172.21.15.37:0/2840780643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561e6145a00 0x5561e61540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.478+0000 7fc40d78a640 1 --2- 172.21.15.37:0/2840780643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561e6145010 0x5561e6153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.479+0000 7fc40df8b640 1 -- 172.21.15.37:0/2840780643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561e6145a00 msgr2=0x5561e61540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.479+0000 7fc40df8b640 1 --2- 172.21.15.37:0/2840780643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561e6145a00 0x5561e61540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.479+0000 7fc40df8b640 1 -- 172.21.15.37:0/2840780643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561e6145010 msgr2=0x5561e6153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.479+0000 7fc40df8b640 1 --2- 172.21.15.37:0/2840780643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561e6145010 0x5561e6153b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.479+0000 7fc40df8b640 1 -- 172.21.15.37:0/2840780643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561e61556a0 con 0x5561e614ab70 2024-06-10T00:59:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.479+0000 7fc40cf89640 1 --2- 172.21.15.37:0/2840780643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561e6145a00 0x5561e61540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:59:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.479+0000 7fc40df8b640 1 --2- 172.21.15.37:0/2840780643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561e614ab70 0x5561e61550b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc400012700 tx=0x7fc400005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9571a9105e6196 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.479+0000 7fc3f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:59:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.479+0000 7fc3f0ff9640 1 -- 172.21.15.37:0/2840780643 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40001c470 con 0x5561e614ab70 2024-06-10T00:59:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.479+0000 7fc3f0ff9640 1 -- 172.21.15.37:0/2840780643 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fc40002d040 con 0x5561e614ab70 2024-06-10T00:59:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.479+0000 7fc4127de740 1 -- 172.21.15.37:0/2840780643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3f8009390 con 0x5561e614ab70 2024-06-10T00:59:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.479+0000 7fc3f0ff9640 1 -- 172.21.15.37:0/2840780643 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc400042050 con 0x5561e614ab70 2024-06-10T00:59:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.479+0000 7fc4127de740 10 client.15909 fetch_fsmap learned FSMap version 27 2024-06-10T00:59:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.479+0000 7fc4127de740 1 -- 172.21.15.37:0/2840780643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5561e61569b0 con 0x5561e614ab70 2024-06-10T00:59:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.479+0000 7fc3f0ff9640 1 -- 172.21.15.37:0/2840780643 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3f8009390 con 0x5561e614ab70 2024-06-10T00:59:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.479+0000 7fc3f0ff9640 1 -- 172.21.15.37:0/2840780643 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc400026420 con 0x5561e614ab70 2024-06-10T00:59:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.480+0000 7fc4127de740 10 client.15909 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:59:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.480+0000 7fc4127de740 10 client.15909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T00:59:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.480+0000 7fc4127de740 20 client.15909 populate_metadata read hostname 'smithi037' 2024-06-10T00:59:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.480+0000 7fc4127de740 1 --2- 172.21.15.37:0/2840780643 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5561e61c63b0 0x5561e61c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.480+0000 7fc4127de740 4 client.15909 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T00:59:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.480+0000 7fc4127de740 1 -- 172.21.15.37:0/2840780643 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5561e60b5c20 con 0x5561e61c63b0 2024-06-10T00:59:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.480+0000 7fc40d78a640 1 --2- 172.21.15.37:0/2840780643 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5561e61c63b0 0x5561e61c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:35.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.480+0000 7fc40d78a640 1 --2- 172.21.15.37:0/2840780643 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5561e61c63b0 0x5561e61c87d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62bfd24cb412208 server_cookie=d076bdae85668643 in_seq=0 out_seq=0 2024-06-10T00:59:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.480+0000 7fc3f0ff9640 10 client.15909 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:59:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.493+0000 7fc3f0ff9640 1 -- 172.21.15.37:0/2840780643 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5561e60b5c20 con 0x5561e61c63b0 2024-06-10T00:59:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.493+0000 7fc3f0ff9640 10 client.15909 handle_command_reply: tid=0 2024-06-10T00:59:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.493+0000 7fc4127de740 1 client.15909 shutdown 2024-06-10T00:59:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.493+0000 7fc4127de740 1 -- 172.21.15.37:0/2840780643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561e614ab70 msgr2=0x5561e61550b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.493+0000 7fc4127de740 1 --2- 172.21.15.37:0/2840780643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561e614ab70 0x5561e61550b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc400012700 tx=0x7fc400005780 comp rx=0 tx=0).stop 2024-06-10T00:59:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.493+0000 7fc4127de740 1 -- 172.21.15.37:0/2840780643 shutdown_connections 2024-06-10T00:59:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.493+0000 7fc4127de740 1 --2- 172.21.15.37:0/2840780643 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5561e61c63b0 0x5561e61c87d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.493+0000 7fc4127de740 1 --2- 172.21.15.37:0/2840780643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561e614ab70 0x5561e61550b0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:35.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.493+0000 7fc4127de740 1 --2- 172.21.15.37:0/2840780643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5561e6145a00 0x5561e61540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.493+0000 7fc4127de740 1 --2- 172.21.15.37:0/2840780643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561e6145010 0x5561e6153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.493+0000 7fc3f0ff9640 0 client.15909 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:59:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.493+0000 7fc4127de740 1 -- 172.21.15.37:0/2840780643 >> 172.21.15.37:0/2840780643 conn(0x5561e6140680 msgr2=0x5561e614f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.493+0000 7fc4127de740 1 -- 172.21.15.37:0/2840780643 shutdown_connections 2024-06-10T00:59:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.493+0000 7fc4127de740 1 -- 172.21.15.37:0/2840780643 wait complete. 2024-06-10T00:59:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.493+0000 7fc4127de740 20 client.15909 trim_cache size 0 max 16384 2024-06-10T00:59:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T00:59:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T00:59:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T00:59:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.495+0000 7fc40f9ee640 1 -- 172.21.15.37:0/3675190646 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3f40a2da0 msgr2=0x7fc3f40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.495+0000 7fc40f9ee640 1 --2- 172.21.15.37:0/3675190646 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3f40a2da0 0x7fc3f40a5260 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc3f80095f0 tx=0x7fc3f8005f90 comp rx=0 tx=0).stop 2024-06-10T00:59:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.495+0000 7fc40f9ee640 1 -- 172.21.15.37:0/3675190646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc408101a30 msgr2=0x7fc408075750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.495+0000 7fc40f9ee640 1 --2- 172.21.15.37:0/3675190646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc408101a30 0x7fc408075750 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc404009bf0 tx=0x7fc404031ee0 comp rx=0 tx=0).stop 2024-06-10T00:59:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.495+0000 7fc40f9ee640 1 -- 172.21.15.37:0/3675190646 shutdown_connections 2024-06-10T00:59:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.495+0000 7fc40f9ee640 1 --2- 172.21.15.37:0/3675190646 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3f40a2da0 0x7fc3f40a5260 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.495+0000 7fc40f9ee640 1 --2- 172.21.15.37:0/3675190646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4081035e0 0x7fc4080742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.495+0000 7fc40f9ee640 1 --2- 172.21.15.37:0/3675190646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc408102c20 0x7fc408073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.495+0000 7fc40f9ee640 1 --2- 172.21.15.37:0/3675190646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc408101a30 0x7fc408075750 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.495+0000 7fc40f9ee640 1 -- 172.21.15.37:0/3675190646 >> 172.21.15.37:0/3675190646 conn(0x7fc4080fcfc0 msgr2=0x7fc408105ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.495+0000 7fc40f9ee640 1 -- 172.21.15.37:0/3675190646 shutdown_connections 2024-06-10T00:59:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:35.495+0000 7fc40f9ee640 1 -- 172.21.15.37:0/3675190646 wait complete. 2024-06-10T00:59:35.647 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:35.648 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:36.265 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:36 smithi094 ceph-mon[36708]: pgmap v517: 129 pgs: 129 active+clean; 1.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.7 KiB/s wr, 7 op/s 2024-06-10T00:59:36.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:36 smithi167 ceph-mon[36917]: pgmap v517: 129 pgs: 129 active+clean; 1.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.7 KiB/s wr, 7 op/s 2024-06-10T00:59:36.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:36 smithi037 ceph-mon[32192]: pgmap v517: 129 pgs: 129 active+clean; 1.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.7 KiB/s wr, 7 op/s 2024-06-10T00:59:36.462 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:59:36.462 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:59:36.531 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:36.531 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:36.531 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 36 -lt 100 ']' 2024-06-10T00:59:36.531 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 36 + 1 2024-06-10T00:59:36.961 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=37 2024-06-10T00:59:36.962 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:59:37.148 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:37.148 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:37.149 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:59:37.150 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:59:37.150 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:59:38.108 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:38 smithi037 ceph-mon[32192]: pgmap v518: 129 pgs: 129 active+clean; 1.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-06-10T00:59:38.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:38 smithi167 ceph-mon[36917]: pgmap v518: 129 pgs: 129 active+clean; 1.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-06-10T00:59:38.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:38 smithi094 ceph-mon[36708]: pgmap v518: 129 pgs: 129 active+clean; 1.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-06-10T00:59:38.722 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:38.723 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:39.260 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:59:39.261 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:59:39.268 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:39.268 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:39.268 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 37 -lt 100 ']' 2024-06-10T00:59:39.268 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 37 + 1 2024-06-10T00:59:39.319 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=38 2024-06-10T00:59:39.319 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:59:39.345 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:39.346 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:39.347 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:59:39.347 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:59:39.347 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:59:39.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.594+0000 7fdcac83f700 1 -- 172.21.15.37:0/3278935606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdca81027a0 msgr2=0x7fdca8102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:39.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.594+0000 7fdcac83f700 1 --2- 172.21.15.37:0/3278935606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdca81027a0 0x7fdca8102b80 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fdc980099c0 tx=0x7fdc98009cd0 comp rx=0 tx=0).stop 2024-06-10T00:59:39.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.594+0000 7fdcac83f700 1 -- 172.21.15.37:0/3278935606 shutdown_connections 2024-06-10T00:59:39.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.594+0000 7fdcac83f700 1 --2- 172.21.15.37:0/3278935606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdca8107790 0x7fdca810f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:39.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.594+0000 7fdcac83f700 1 --2- 172.21.15.37:0/3278935606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdca81027a0 0x7fdca8102b80 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:39.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.594+0000 7fdcac83f700 1 --2- 172.21.15.37:0/3278935606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdca8103150 0x7fdca81070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:39.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.594+0000 7fdcac83f700 1 -- 172.21.15.37:0/3278935606 >> 172.21.15.37:0/3278935606 conn(0x7fdca8079560 msgr2=0x7fdca8079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:39.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.594+0000 7fdcac83f700 1 -- 172.21.15.37:0/3278935606 shutdown_connections 2024-06-10T00:59:39.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.594+0000 7fdcac83f700 1 -- 172.21.15.37:0/3278935606 wait complete. 2024-06-10T00:59:39.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.595+0000 7fdcac83f700 1 Processor -- start 2024-06-10T00:59:39.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.595+0000 7fdcac83f700 1 -- start start 2024-06-10T00:59:39.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.596+0000 7fdcac83f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdca81027a0 0x7fdca819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:39.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.596+0000 7fdcac83f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdca8103150 0x7fdca819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:39.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.596+0000 7fdcac83f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdca8107790 0x7fdca81989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:39.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.596+0000 7fdcac83f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdca81127b0 con 0x7fdca8103150 2024-06-10T00:59:39.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.596+0000 7fdcac83f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdca8112630 con 0x7fdca8107790 2024-06-10T00:59:39.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.596+0000 7fdcac83f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdca8112930 con 0x7fdca81027a0 2024-06-10T00:59:39.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.596+0000 7fdc9dd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdca8103150 0x7fdca819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:39.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.596+0000 7fdca659c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdca81027a0 0x7fdca819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:39.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.596+0000 7fdc9dd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdca8103150 0x7fdca819ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52442/0 (socket says 172.21.15.37:52442) 2024-06-10T00:59:39.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.596+0000 7fdc9dd9b700 1 -- 172.21.15.37:0/3734816094 learned_addr learned my addr 172.21.15.37:0/3734816094 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:39.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.596+0000 7fdca659c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdca81027a0 0x7fdca819e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:40282/0 (socket says 172.21.15.37:40282) 2024-06-10T00:59:39.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.596+0000 7fdca6d9d700 1 --2- 172.21.15.37:0/3734816094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdca8107790 0x7fdca81989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:39.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.596+0000 7fdca659c700 1 -- 172.21.15.37:0/3734816094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdca8107790 msgr2=0x7fdca81989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:39.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.596+0000 7fdca659c700 1 --2- 172.21.15.37:0/3734816094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdca8107790 0x7fdca81989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:39.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.596+0000 7fdca659c700 1 -- 172.21.15.37:0/3734816094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdca8103150 msgr2=0x7fdca819ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:39.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.596+0000 7fdca659c700 1 --2- 172.21.15.37:0/3734816094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdca8103150 0x7fdca819ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:39.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.596+0000 7fdca659c700 1 -- 172.21.15.37:0/3734816094 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca8199130 con 0x7fdca81027a0 2024-06-10T00:59:39.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.597+0000 7fdca659c700 1 --2- 172.21.15.37:0/3734816094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdca81027a0 0x7fdca819e7c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdc980098e0 tx=0x7fdc980047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:39.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.597+0000 7fdc9ffff700 1 -- 172.21.15.37:0/3734816094 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc98027070 con 0x7fdca81027a0 2024-06-10T00:59:39.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.597+0000 7fdc9ffff700 1 -- 172.21.15.37:0/3734816094 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc98003e40 con 0x7fdca81027a0 2024-06-10T00:59:39.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.597+0000 7fdcac83f700 1 -- 172.21.15.37:0/3734816094 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca81993c0 con 0x7fdca81027a0 2024-06-10T00:59:39.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.597+0000 7fdc9ffff700 1 -- 172.21.15.37:0/3734816094 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc98022bd0 con 0x7fdca81027a0 2024-06-10T00:59:39.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.597+0000 7fdcac83f700 1 -- 172.21.15.37:0/3734816094 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca80746c0 con 0x7fdca81027a0 2024-06-10T00:59:39.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.600+0000 7fdc9ffff700 1 -- 172.21.15.37:0/3734816094 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fdc9802a070 con 0x7fdca81027a0 2024-06-10T00:59:39.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.601+0000 7fdc9ffff700 1 --2- 172.21.15.37:0/3734816094 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdc940a2cc0 0x7fdc940a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:39.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.601+0000 7fdc9dd9b700 1 --2- 172.21.15.37:0/3734816094 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdc940a2cc0 0x7fdc940a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:39.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.601+0000 7fdc9ffff700 1 -- 172.21.15.37:0/3734816094 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fdc980d0380 con 0x7fdca81027a0 2024-06-10T00:59:39.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.602+0000 7fdc9dd9b700 1 --2- 172.21.15.37:0/3734816094 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdc940a2cc0 0x7fdc940a5180 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdca8108be0 tx=0x7fdc9000b480 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:39.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.782+0000 7fdcb70c2b80 1 Processor -- start 2024-06-10T00:59:39.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.782+0000 7fdcb70c2b80 1 -- start start 2024-06-10T00:59:39.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.782+0000 7fdcb70c2b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bdbf9aecd0 0x55bdbf9af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:39.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.782+0000 7fdcb70c2b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdbf9af5f0 0x55bdbf9a8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:39.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.782+0000 7fdcb70c2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bdbf9a9460 0x55bdbf9a98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:39.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.782+0000 7fdcb70c2b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bdbf8fa9a0 con 0x55bdbf9af5f0 2024-06-10T00:59:39.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.782+0000 7fdcb70c2b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bdbf8a28b0 con 0x55bdbf9aecd0 2024-06-10T00:59:39.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.782+0000 7fdcb70c2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bdbf8f6960 con 0x55bdbf9a9460 2024-06-10T00:59:39.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.782+0000 7fdc9dd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdbf9af5f0 0x55bdbf9a8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:39.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.782+0000 7fdc9dd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdbf9af5f0 0x55bdbf9a8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52458/0 (socket says 172.21.15.37:52458) 2024-06-10T00:59:39.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.782+0000 7fdc9dd9b700 1 -- 172.21.15.37:0/3146674570 learned_addr learned my addr 172.21.15.37:0/3146674570 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:39.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.782+0000 7fdca6d9d700 1 --2- 172.21.15.37:0/3146674570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bdbf9a9460 0x55bdbf9a98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:39.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.782+0000 7fdca659c700 1 --2- 172.21.15.37:0/3146674570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bdbf9aecd0 0x55bdbf9af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:39.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.783+0000 7fdca659c700 1 -- 172.21.15.37:0/3146674570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bdbf9a9460 msgr2=0x55bdbf9a98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:39.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.783+0000 7fdca659c700 1 --2- 172.21.15.37:0/3146674570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bdbf9a9460 0x55bdbf9a98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:39.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.783+0000 7fdca659c700 1 -- 172.21.15.37:0/3146674570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdbf9af5f0 msgr2=0x55bdbf9a8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:39.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.783+0000 7fdca659c700 1 --2- 172.21.15.37:0/3146674570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdbf9af5f0 0x55bdbf9a8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:39.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.783+0000 7fdca659c700 1 -- 172.21.15.37:0/3146674570 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdbf8990e0 con 0x55bdbf9aecd0 2024-06-10T00:59:39.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.783+0000 7fdc9dd9b700 1 --2- 172.21.15.37:0/3146674570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdbf9af5f0 0x55bdbf9a8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:59:39.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.783+0000 7fdca659c700 1 --2- 172.21.15.37:0/3146674570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bdbf9aecd0 0x55bdbf9af0b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdc9809c520 tx=0x7fdc98003890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c128340f68460ff4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:39.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.783+0000 7fdc9cd99700 1 -- 172.21.15.37:0/3146674570 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc980d0160 con 0x55bdbf9aecd0 2024-06-10T00:59:39.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.783+0000 7fdc9cd99700 1 -- 172.21.15.37:0/3146674570 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc9809c820 con 0x55bdbf9aecd0 2024-06-10T00:59:39.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.783+0000 7fdc9cd99700 1 -- 172.21.15.37:0/3146674570 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9809ca80 con 0x55bdbf9aecd0 2024-06-10T00:59:39.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.784+0000 7fdcb70c2b80 1 -- 172.21.15.37:0/3146674570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bdbf9aecd0 msgr2=0x55bdbf9af0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:39.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.784+0000 7fdcb70c2b80 1 --2- 172.21.15.37:0/3146674570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bdbf9aecd0 0x55bdbf9af0b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdc9809c520 tx=0x7fdc98003890 comp rx=0 tx=0).stop 2024-06-10T00:59:39.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.784+0000 7fdcb70c2b80 1 -- 172.21.15.37:0/3146674570 shutdown_connections 2024-06-10T00:59:39.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.784+0000 7fdcb70c2b80 1 --2- 172.21.15.37:0/3146674570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bdbf9a9460 0x55bdbf9a98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:39.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.784+0000 7fdcb70c2b80 1 --2- 172.21.15.37:0/3146674570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bdbf9aecd0 0x55bdbf9af0b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:39.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.784+0000 7fdcb70c2b80 1 --2- 172.21.15.37:0/3146674570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdbf9af5f0 0x55bdbf9a8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:39.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.784+0000 7fdcb70c2b80 1 -- 172.21.15.37:0/3146674570 >> 172.21.15.37:0/3146674570 conn(0x55bdbf8d3e50 msgr2=0x55bdbf9a41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:39.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.784+0000 7fdcb70c2b80 1 -- 172.21.15.37:0/3146674570 shutdown_connections 2024-06-10T00:59:39.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.784+0000 7fdcb70c2b80 1 -- 172.21.15.37:0/3146674570 wait complete. 2024-06-10T00:59:39.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.784+0000 7fdcb70c2b80 1 -- start start 2024-06-10T00:59:39.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.785+0000 7fdcb70c2b80 1 Processor -- start 2024-06-10T00:59:39.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.785+0000 7fdcb70c2b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bdbf9a9460 0x55bdbf9b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:39.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.785+0000 7fdcb70c2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bdbf9aecd0 0x55bdbf9b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:39.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.785+0000 7fdcb70c2b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdbf9af5f0 0x55bdbf9b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:39.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.785+0000 7fdcb70c2b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bdbf9b5d80 con 0x55bdbf9af5f0 2024-06-10T00:59:39.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.785+0000 7fdcb70c2b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bdbf89ad80 con 0x55bdbf9a9460 2024-06-10T00:59:39.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.785+0000 7fdcb70c2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bdbf9b5f00 con 0x55bdbf9aecd0 2024-06-10T00:59:39.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.785+0000 7fdcb70c2b80 10 client.0 _ll_register_callbacks cb 0x55bdbf894ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:59:39.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.785+0000 7fdca6d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdbf9af5f0 0x55bdbf9b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:39.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.785+0000 7fdca6d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdbf9af5f0 0x55bdbf9b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52462/0 (socket says 172.21.15.37:52462) 2024-06-10T00:59:39.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.785+0000 7fdca6d9d700 1 -- 172.21.15.37:0/1614294422 learned_addr learned my addr 172.21.15.37:0/1614294422 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:39.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.785+0000 7fdc9dd9b700 1 --2- 172.21.15.37:0/1614294422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bdbf9aecd0 0x55bdbf9b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:39.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.785+0000 7fdca659c700 1 --2- 172.21.15.37:0/1614294422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bdbf9a9460 0x55bdbf9b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:39.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.786+0000 7fdca6d9d700 1 -- 172.21.15.37:0/1614294422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bdbf9aecd0 msgr2=0x55bdbf9b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:39.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.786+0000 7fdca6d9d700 1 --2- 172.21.15.37:0/1614294422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bdbf9aecd0 0x55bdbf9b2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:39.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.786+0000 7fdca6d9d700 1 -- 172.21.15.37:0/1614294422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bdbf9a9460 msgr2=0x55bdbf9b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:39.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.786+0000 7fdca6d9d700 1 --2- 172.21.15.37:0/1614294422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bdbf9a9460 0x55bdbf9b5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:39.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.786+0000 7fdca6d9d700 1 -- 172.21.15.37:0/1614294422 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdbf849a60 con 0x55bdbf9af5f0 2024-06-10T00:59:39.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.786+0000 7fdc9dd9b700 1 --2- 172.21.15.37:0/1614294422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bdbf9aecd0 0x55bdbf9b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:59:39.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.786+0000 7fdca6d9d700 1 --2- 172.21.15.37:0/1614294422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdbf9af5f0 0x55bdbf9b2b90 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fdca00042a0 tx=0x7fdca00042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39dae6dabd698af3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:39.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.786+0000 7fdc8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:59:39.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.786+0000 7fdc8dffb700 1 -- 172.21.15.37:0/1614294422 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca001a070 con 0x55bdbf9af5f0 2024-06-10T00:59:39.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.786+0000 7fdcb70c2b80 1 -- 172.21.15.37:0/1614294422 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc9809c820 con 0x55bdbf9af5f0 2024-06-10T00:59:39.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.786+0000 7fdc8dffb700 1 -- 172.21.15.37:0/1614294422 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fdca0004db0 con 0x55bdbf9af5f0 2024-06-10T00:59:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.786+0000 7fdc8dffb700 1 -- 172.21.15.37:0/1614294422 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdca0025050 con 0x55bdbf9af5f0 2024-06-10T00:59:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.786+0000 7fdcb70c2b80 10 client.15921 fetch_fsmap learned FSMap version 27 2024-06-10T00:59:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.786+0000 7fdcb70c2b80 1 -- 172.21.15.37:0/1614294422 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bdbf9c06b0 con 0x55bdbf9af5f0 2024-06-10T00:59:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.787+0000 7fdc8dffb700 1 -- 172.21.15.37:0/1614294422 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc9809c820 con 0x55bdbf9af5f0 2024-06-10T00:59:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.787+0000 7fdc8dffb700 1 -- 172.21.15.37:0/1614294422 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fdca0016790 con 0x55bdbf9af5f0 2024-06-10T00:59:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.787+0000 7fdcb70c2b80 10 client.15921 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:59:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.787+0000 7fdcb70c2b80 10 client.15921 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T00:59:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.787+0000 7fdcb70c2b80 20 client.15921 populate_metadata read hostname 'smithi037' 2024-06-10T00:59:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.787+0000 7fdcb70c2b80 1 --2- 172.21.15.37:0/1614294422 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bdbf9c2c90 0x55bdbf9c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.787+0000 7fdcb70c2b80 4 client.15921 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:59:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.787+0000 7fdcb70c2b80 1 -- 172.21.15.37:0/1614294422 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdbf907ea0 con 0x55bdbf9c2c90 2024-06-10T00:59:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.787+0000 7fdca659c700 1 --2- 172.21.15.37:0/1614294422 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bdbf9c2c90 0x55bdbf9c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.788+0000 7fdca659c700 1 --2- 172.21.15.37:0/1614294422 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bdbf9c2c90 0x55bdbf9c50b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3684db0bbe7f30b1 server_cookie=44f018b14a0b0379 in_seq=0 out_seq=0 2024-06-10T00:59:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.788+0000 7fdc8dffb700 10 client.15921 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:59:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.788+0000 7fdc8dffb700 1 -- 172.21.15.37:0/1614294422 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bdbf907ea0 con 0x55bdbf9c2c90 2024-06-10T00:59:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.788+0000 7fdc8dffb700 10 client.15921 handle_command_reply: tid=0 2024-06-10T00:59:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.788+0000 7fdcb70c2b80 1 client.15921 shutdown 2024-06-10T00:59:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.789+0000 7fdcb70c2b80 1 -- 172.21.15.37:0/1614294422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdbf9af5f0 msgr2=0x55bdbf9b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.789+0000 7fdcb70c2b80 1 --2- 172.21.15.37:0/1614294422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdbf9af5f0 0x55bdbf9b2b90 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fdca00042a0 tx=0x7fdca00042d0 comp rx=0 tx=0).stop 2024-06-10T00:59:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.789+0000 7fdcb70c2b80 1 -- 172.21.15.37:0/1614294422 shutdown_connections 2024-06-10T00:59:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.789+0000 7fdcb70c2b80 1 --2- 172.21.15.37:0/1614294422 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bdbf9c2c90 0x55bdbf9c50b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:39.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.789+0000 7fdcb70c2b80 1 --2- 172.21.15.37:0/1614294422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdbf9af5f0 0x55bdbf9b2b90 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:39.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.789+0000 7fdcb70c2b80 1 --2- 172.21.15.37:0/1614294422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bdbf9a9460 0x55bdbf9b5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:39.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.789+0000 7fdcb70c2b80 1 --2- 172.21.15.37:0/1614294422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bdbf9aecd0 0x55bdbf9b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:39.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.789+0000 7fdc8dffb700 0 client.15921 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:59:39.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.789+0000 7fdcb70c2b80 1 -- 172.21.15.37:0/1614294422 >> 172.21.15.37:0/1614294422 conn(0x55bdbf8d3e50 msgr2=0x55bdbf9a7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:39.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.789+0000 7fdcb70c2b80 1 -- 172.21.15.37:0/1614294422 shutdown_connections 2024-06-10T00:59:39.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.789+0000 7fdcb70c2b80 1 -- 172.21.15.37:0/1614294422 wait complete. 2024-06-10T00:59:39.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.789+0000 7fdcb70c2b80 20 client.15921 trim_cache size 0 max 16384 2024-06-10T00:59:39.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.813+0000 7fdcb70c2b80 1 Processor -- start 2024-06-10T00:59:39.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.813+0000 7fdcb70c2b80 1 -- start start 2024-06-10T00:59:39.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.813+0000 7fdcb70c2b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdbf9a9460 0x55bdbf9b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:39.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.813+0000 7fdcb70c2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bdbf9aecd0 0x55bdbf9ca960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:39.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.813+0000 7fdcb70c2b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bdbf9af5f0 0x55bdbf9caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:39.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.813+0000 7fdcb70c2b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdca81b2d50 con 0x55bdbf9a9460 2024-06-10T00:59:39.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.813+0000 7fdcb70c2b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdca81b05e0 con 0x55bdbf9af5f0 2024-06-10T00:59:39.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.813+0000 7fdcb70c2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdca81ae430 con 0x55bdbf9aecd0 2024-06-10T00:59:39.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.813+0000 7fdca659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdbf9a9460 0x55bdbf9b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:39.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.813+0000 7fdca659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdbf9a9460 0x55bdbf9b2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52476/0 (socket says 172.21.15.37:52476) 2024-06-10T00:59:39.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.813+0000 7fdca659c700 1 -- 172.21.15.37:0/637591575 learned_addr learned my addr 172.21.15.37:0/637591575 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:39.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.813+0000 7fdca6d9d700 1 --2- 172.21.15.37:0/637591575 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bdbf9af5f0 0x55bdbf9caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:39.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.813+0000 7fdc9dd9b700 1 --2- 172.21.15.37:0/637591575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bdbf9aecd0 0x55bdbf9ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:39.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.813+0000 7fdca659c700 1 -- 172.21.15.37:0/637591575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bdbf9aecd0 msgr2=0x55bdbf9ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:39.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.813+0000 7fdca659c700 1 --2- 172.21.15.37:0/637591575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bdbf9aecd0 0x55bdbf9ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:39.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.813+0000 7fdca659c700 1 -- 172.21.15.37:0/637591575 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bdbf9af5f0 msgr2=0x55bdbf9caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:39.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.813+0000 7fdca659c700 1 --2- 172.21.15.37:0/637591575 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bdbf9af5f0 0x55bdbf9caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:39.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.813+0000 7fdca659c700 1 -- 172.21.15.37:0/637591575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdbf9b35c0 con 0x55bdbf9a9460 2024-06-10T00:59:39.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.813+0000 7fdca6d9d700 1 --2- 172.21.15.37:0/637591575 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bdbf9af5f0 0x55bdbf9caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:59:39.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.813+0000 7fdc9dd9b700 1 --2- 172.21.15.37:0/637591575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bdbf9aecd0 0x55bdbf9ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:59:39.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.813+0000 7fdca659c700 1 --2- 172.21.15.37:0/637591575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdbf9a9460 0x55bdbf9b2be0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fdc980043a0 tx=0x7fdc9801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4922f75c08e84a5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:39.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.813+0000 7fdc8ffff700 1 -- 172.21.15.37:0/637591575 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc980b5070 con 0x55bdbf9a9460 2024-06-10T00:59:39.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.814+0000 7fdc8ffff700 1 -- 172.21.15.37:0/637591575 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc9809c820 con 0x55bdbf9a9460 2024-06-10T00:59:39.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.814+0000 7fdc8ffff700 1 -- 172.21.15.37:0/637591575 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9809ca80 con 0x55bdbf9a9460 2024-06-10T00:59:39.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.814+0000 7fdcb70c2b80 1 -- 172.21.15.37:0/637591575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdbf9a9460 msgr2=0x55bdbf9b2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:39.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.814+0000 7fdcb70c2b80 1 --2- 172.21.15.37:0/637591575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdbf9a9460 0x55bdbf9b2be0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fdc980043a0 tx=0x7fdc9801a040 comp rx=0 tx=0).stop 2024-06-10T00:59:39.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.814+0000 7fdcb70c2b80 1 -- 172.21.15.37:0/637591575 shutdown_connections 2024-06-10T00:59:39.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.814+0000 7fdcb70c2b80 1 --2- 172.21.15.37:0/637591575 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bdbf9af5f0 0x55bdbf9caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:39.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.814+0000 7fdcb70c2b80 1 --2- 172.21.15.37:0/637591575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdbf9a9460 0x55bdbf9b2be0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:39.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.814+0000 7fdcb70c2b80 1 --2- 172.21.15.37:0/637591575 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bdbf9aecd0 0x55bdbf9ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:39.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.814+0000 7fdcb70c2b80 1 -- 172.21.15.37:0/637591575 >> 172.21.15.37:0/637591575 conn(0x55bdbf8d3e50 msgr2=0x55bdbf9a8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:39.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.814+0000 7fdcb70c2b80 1 -- 172.21.15.37:0/637591575 shutdown_connections 2024-06-10T00:59:39.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.815+0000 7fdcb70c2b80 1 -- 172.21.15.37:0/637591575 wait complete. 2024-06-10T00:59:39.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.815+0000 7fdcb70c2b80 1 -- start start 2024-06-10T00:59:39.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.815+0000 7fdcb70c2b80 1 Processor -- start 2024-06-10T00:59:39.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.815+0000 7fdcb70c2b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bdbf9a9460 0x55bdbf9bb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:39.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.815+0000 7fdcb70c2b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdbf9aecd0 0x55bdbf9bb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:39.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.815+0000 7fdcb70c2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bdbf9af5f0 0x55bdbf9b81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:39.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.815+0000 7fdcb70c2b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdca81aeaa0 con 0x55bdbf9aecd0 2024-06-10T00:59:39.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.815+0000 7fdcb70c2b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdca81aff10 con 0x55bdbf9a9460 2024-06-10T00:59:39.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.815+0000 7fdcb70c2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdca81af170 con 0x55bdbf9af5f0 2024-06-10T00:59:39.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.815+0000 7fdcb70c2b80 10 client.0 _ll_register_callbacks cb 0x55bdbf894ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:59:39.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.816+0000 7fdc9dd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdbf9aecd0 0x55bdbf9bb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:39.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.816+0000 7fdc9dd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdbf9aecd0 0x55bdbf9bb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52490/0 (socket says 172.21.15.37:52490) 2024-06-10T00:59:39.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.816+0000 7fdc9dd9b700 1 -- 172.21.15.37:0/4177935989 learned_addr learned my addr 172.21.15.37:0/4177935989 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:39.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.816+0000 7fdca659c700 1 --2- 172.21.15.37:0/4177935989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bdbf9a9460 0x55bdbf9bb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:39.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.816+0000 7fdca6d9d700 1 --2- 172.21.15.37:0/4177935989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bdbf9af5f0 0x55bdbf9b81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:39.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.816+0000 7fdc9dd9b700 1 -- 172.21.15.37:0/4177935989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bdbf9af5f0 msgr2=0x55bdbf9b81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:39.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.816+0000 7fdc9dd9b700 1 --2- 172.21.15.37:0/4177935989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bdbf9af5f0 0x55bdbf9b81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:39.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.816+0000 7fdc9dd9b700 1 -- 172.21.15.37:0/4177935989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bdbf9a9460 msgr2=0x55bdbf9bb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:39.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.816+0000 7fdc9dd9b700 1 --2- 172.21.15.37:0/4177935989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bdbf9a9460 0x55bdbf9bb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:39.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.816+0000 7fdc9dd9b700 1 -- 172.21.15.37:0/4177935989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdbf9b89e0 con 0x55bdbf9aecd0 2024-06-10T00:59:39.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.816+0000 7fdca659c700 1 --2- 172.21.15.37:0/4177935989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bdbf9a9460 0x55bdbf9bb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T00:59:39.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.816+0000 7fdca6d9d700 1 --2- 172.21.15.37:0/4177935989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bdbf9af5f0 0x55bdbf9b81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:59:39.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.816+0000 7fdc9dd9b700 1 --2- 172.21.15.37:0/4177935989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdbf9aecd0 0x55bdbf9bb990 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fdc9000bac0 tx=0x7fdc90013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989b525b980d0907 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:39.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.816+0000 7fdc8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:59:39.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.816+0000 7fdc8dffb700 1 -- 172.21.15.37:0/4177935989 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9002e060 con 0x55bdbf9aecd0 2024-06-10T00:59:39.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.816+0000 7fdcb70c2b80 1 -- 172.21.15.37:0/4177935989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc9809c820 con 0x55bdbf9aecd0 2024-06-10T00:59:39.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.816+0000 7fdc8dffb700 1 -- 172.21.15.37:0/4177935989 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fdc9001fca0 con 0x55bdbf9aecd0 2024-06-10T00:59:39.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.817+0000 7fdc8dffb700 1 -- 172.21.15.37:0/4177935989 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdc90035050 con 0x55bdbf9aecd0 2024-06-10T00:59:39.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.817+0000 7fdcb70c2b80 10 client.15927 fetch_fsmap learned FSMap version 27 2024-06-10T00:59:39.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.817+0000 7fdcb70c2b80 1 -- 172.21.15.37:0/4177935989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bdbfa2e500 con 0x55bdbf9aecd0 2024-06-10T00:59:39.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.817+0000 7fdc8dffb700 1 -- 172.21.15.37:0/4177935989 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc9809c820 con 0x55bdbf9aecd0 2024-06-10T00:59:39.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.817+0000 7fdc8dffb700 1 -- 172.21.15.37:0/4177935989 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fdc9001e420 con 0x55bdbf9aecd0 2024-06-10T00:59:39.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.817+0000 7fdcb70c2b80 10 client.15927 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:59:39.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.817+0000 7fdcb70c2b80 10 client.15927 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T00:59:39.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.817+0000 7fdcb70c2b80 20 client.15927 populate_metadata read hostname 'smithi037' 2024-06-10T00:59:39.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.817+0000 7fdcb70c2b80 1 --2- 172.21.15.37:0/4177935989 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bdbfa2ecb0 0x55bdbfa310d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:39.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.817+0000 7fdcb70c2b80 4 client.15927 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-06-10T00:59:39.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.817+0000 7fdcb70c2b80 1 -- 172.21.15.37:0/4177935989 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55bdbf9a9ed0 con 0x55bdbfa2ecb0 2024-06-10T00:59:39.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.817+0000 7fdca659c700 1 --2- 172.21.15.37:0/4177935989 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bdbfa2ecb0 0x55bdbfa310d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:39.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.818+0000 7fdc8dffb700 10 client.15927 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:59:39.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:39.818+0000 7fdca659c700 1 --2- 172.21.15.37:0/4177935989 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bdbfa2ecb0 0x55bdbfa310d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=280c0511ba1217ef server_cookie=fc293294f8ceb2dc in_seq=0 out_seq=0 2024-06-10T00:59:40.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:40.320+0000 7fdc8dffb700 1 -- 172.21.15.37:0/4177935989 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55bdbf9a9ed0 con 0x55bdbfa2ecb0 2024-06-10T00:59:40.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:40.320+0000 7fdc8dffb700 10 client.15927 handle_command_reply: tid=0 2024-06-10T00:59:40.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:40.320+0000 7fdcb70c2b80 1 client.15927 shutdown 2024-06-10T00:59:40.320 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:40 smithi037 ceph-mon[32192]: pgmap v519: 129 pgs: 129 active+clean; 1.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 12 op/s 2024-06-10T00:59:40.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:40.320+0000 7fdcb70c2b80 1 -- 172.21.15.37:0/4177935989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdbf9aecd0 msgr2=0x55bdbf9bb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:40.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:40.320+0000 7fdcb70c2b80 1 --2- 172.21.15.37:0/4177935989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdbf9aecd0 0x55bdbf9bb990 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fdc9000bac0 tx=0x7fdc90013070 comp rx=0 tx=0).stop 2024-06-10T00:59:40.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:40.320+0000 7fdcb70c2b80 1 -- 172.21.15.37:0/4177935989 shutdown_connections 2024-06-10T00:59:40.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:40.320+0000 7fdcb70c2b80 1 --2- 172.21.15.37:0/4177935989 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bdbfa2ecb0 0x55bdbfa310d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:40.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:40.320+0000 7fdcb70c2b80 1 --2- 172.21.15.37:0/4177935989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bdbf9af5f0 0x55bdbf9b81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:40.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:40.320+0000 7fdcb70c2b80 1 --2- 172.21.15.37:0/4177935989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bdbf9a9460 0x55bdbf9bb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:40.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:40.320+0000 7fdcb70c2b80 1 --2- 172.21.15.37:0/4177935989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdbf9aecd0 0x55bdbf9bb990 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:40.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:40.320+0000 7fdc8dffb700 0 client.15927 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:59:40.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:40.320+0000 7fdcb70c2b80 1 -- 172.21.15.37:0/4177935989 >> 172.21.15.37:0/4177935989 conn(0x55bdbf8d3e50 msgr2=0x55bdbf9af9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:40.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:40.320+0000 7fdcb70c2b80 1 -- 172.21.15.37:0/4177935989 shutdown_connections 2024-06-10T00:59:40.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:40.320+0000 7fdcb70c2b80 1 -- 172.21.15.37:0/4177935989 wait complete. 2024-06-10T00:59:40.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:40.321+0000 7fdcb70c2b80 20 client.15927 trim_cache size 0 max 16384 2024-06-10T00:59:40.322 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T00:59:40.322 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-06-10T00:59:40.322 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24472, 2024-06-10T00:59:40.322 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 6, 2024-06-10T00:59:40.322 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-06-10T00:59:40.322 INFO:teuthology.orchestra.run.smithi037.stdout: "5049796b": { 2024-06-10T00:59:40.322 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 6, 2024-06-10T00:59:40.322 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-06-10T00:59:40.322 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T00:59:40.323 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-06-10T00:59:40.323 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T00:59:40.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T00:59:40.323 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-06-10T00:59:40.323 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 150.0, 2024-06-10T00:59:40.323 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-06-10T00:59:40.323 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-06-10T00:59:40.323 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-06-10T00:59:40.323 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T00:59:40.323 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-06-10T00:59:40.323 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T00:59:40.323 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T00:59:40.323 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T00:59:40.323 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T00:59:40.324 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T00:59:40.324 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T00:59:40.324 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T00:59:40.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:40.321+0000 7fdcac83f700 1 -- 172.21.15.37:0/3734816094 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdc940a2cc0 msgr2=0x7fdc940a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:40.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:40.321+0000 7fdcac83f700 1 --2- 172.21.15.37:0/3734816094 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdc940a2cc0 0x7fdc940a5180 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdca8108be0 tx=0x7fdc9000b480 comp rx=0 tx=0).stop 2024-06-10T00:59:40.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:40.321+0000 7fdcac83f700 1 -- 172.21.15.37:0/3734816094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdca81027a0 msgr2=0x7fdca819e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:40.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:40.321+0000 7fdcac83f700 1 --2- 172.21.15.37:0/3734816094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdca81027a0 0x7fdca819e7c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdc980098e0 tx=0x7fdc980047b0 comp rx=0 tx=0).stop 2024-06-10T00:59:40.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:40.321+0000 7fdcac83f700 1 -- 172.21.15.37:0/3734816094 shutdown_connections 2024-06-10T00:59:40.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:40.321+0000 7fdcac83f700 1 --2- 172.21.15.37:0/3734816094 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdc940a2cc0 0x7fdc940a5180 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:40.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:40.321+0000 7fdcac83f700 1 --2- 172.21.15.37:0/3734816094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdca8107790 0x7fdca81989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:40.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:40.321+0000 7fdcac83f700 1 --2- 172.21.15.37:0/3734816094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdca81027a0 0x7fdca819e7c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:40.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:40.321+0000 7fdcac83f700 1 --2- 172.21.15.37:0/3734816094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdca8103150 0x7fdca819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:40.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:40.321+0000 7fdcac83f700 1 -- 172.21.15.37:0/3734816094 >> 172.21.15.37:0/3734816094 conn(0x7fdca8079560 msgr2=0x7fdca8100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:40.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:40.322+0000 7fdcac83f700 1 -- 172.21.15.37:0/3734816094 shutdown_connections 2024-06-10T00:59:40.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:40.322+0000 7fdcac83f700 1 -- 172.21.15.37:0/3734816094 wait complete. 2024-06-10T00:59:40.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:40 smithi167 ceph-mon[36917]: pgmap v519: 129 pgs: 129 active+clean; 1.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 12 op/s 2024-06-10T00:59:40.348 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:40.348 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:40.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:40 smithi094 ceph-mon[36708]: pgmap v519: 129 pgs: 129 active+clean; 1.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 12 op/s 2024-06-10T00:59:40.528 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:59:40.528 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:59:42.291 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:42 smithi037 ceph-mon[32192]: pgmap v520: 129 pgs: 129 active+clean; 1.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.7 KiB/s wr, 12 op/s 2024-06-10T00:59:42.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:42 smithi167 ceph-mon[36917]: pgmap v520: 129 pgs: 129 active+clean; 1.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.7 KiB/s wr, 12 op/s 2024-06-10T00:59:42.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:42 smithi094 ceph-mon[36708]: pgmap v520: 129 pgs: 129 active+clean; 1.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.7 KiB/s wr, 12 op/s 2024-06-10T00:59:43.169 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '5049796b', quiesce took 85.3 seconds. Will release after: -55.3 2024-06-10T00:59:43.170 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T00:59:44.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:44 smithi167 ceph-mon[36917]: pgmap v521: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-06-10T00:59:44.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:44 smithi094 ceph-mon[36708]: pgmap v521: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-06-10T00:59:44.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:44 smithi037 ceph-mon[32192]: pgmap v521: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-06-10T00:59:45.925 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:59:46.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:46 smithi167 ceph-mon[36917]: pgmap v522: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.5 KiB/s wr, 12 op/s 2024-06-10T00:59:46.348 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:46 smithi037 ceph-mon[32192]: pgmap v522: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.5 KiB/s wr, 12 op/s 2024-06-10T00:59:46.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:46 smithi094 ceph-mon[36708]: pgmap v522: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.5 KiB/s wr, 12 op/s 2024-06-10T00:59:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.919+0000 7f6232a9d700 1 -- 172.21.15.37:0/1564382071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f622c101810 msgr2=0x7f622c10f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.919+0000 7f6232a9d700 1 --2- 172.21.15.37:0/1564382071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f622c101810 0x7f622c10f280 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6224009990 tx=0x7f6224009ca0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.919+0000 7f6232a9d700 1 -- 172.21.15.37:0/1564382071 shutdown_connections 2024-06-10T00:59:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.919+0000 7f6232a9d700 1 --2- 172.21.15.37:0/1564382071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f622c101810 0x7f622c10f280 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.919+0000 7f6232a9d700 1 --2- 172.21.15.37:0/1564382071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f622c107570 0x7f622c107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.919+0000 7f6232a9d700 1 --2- 172.21.15.37:0/1564382071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f622c100e50 0x7f622c1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.919+0000 7f6232a9d700 1 -- 172.21.15.37:0/1564382071 >> 172.21.15.37:0/1564382071 conn(0x7f622c0766b0 msgr2=0x7f622c076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.919+0000 7f6232a9d700 1 -- 172.21.15.37:0/1564382071 shutdown_connections 2024-06-10T00:59:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.919+0000 7f6232a9d700 1 -- 172.21.15.37:0/1564382071 wait complete. 2024-06-10T00:59:46.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.920+0000 7f6232a9d700 1 Processor -- start 2024-06-10T00:59:46.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.920+0000 7f6232a9d700 1 -- start start 2024-06-10T00:59:46.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.920+0000 7f6232a9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f622c100e50 0x7f622c102030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:46.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.920+0000 7f6232a9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f622c101810 0x7f622c102570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:46.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.920+0000 7f6232a9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f622c107570 0x7f622c102ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:46.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.920+0000 7f6232a9d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f622c1127b0 con 0x7f622c107570 2024-06-10T00:59:46.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.920+0000 7f6232a9d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f622c112630 con 0x7f622c101810 2024-06-10T00:59:46.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.920+0000 7f6232a9d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f622c112930 con 0x7f622c100e50 2024-06-10T00:59:46.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.920+0000 7f623103a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f622c107570 0x7f622c102ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:46.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.920+0000 7f623103a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f622c107570 0x7f622c102ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40586/0 (socket says 172.21.15.37:40586) 2024-06-10T00:59:46.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.920+0000 7f623103a700 1 -- 172.21.15.37:0/463027190 learned_addr learned my addr 172.21.15.37:0/463027190 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:46.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.920+0000 7f622bfff700 1 --2- 172.21.15.37:0/463027190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f622c101810 0x7f622c102570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:46.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.920+0000 7f6230839700 1 --2- 172.21.15.37:0/463027190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f622c100e50 0x7f622c102030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:46.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.921+0000 7f623103a700 1 -- 172.21.15.37:0/463027190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f622c100e50 msgr2=0x7f622c102030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:46.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.921+0000 7f623103a700 1 --2- 172.21.15.37:0/463027190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f622c100e50 0x7f622c102030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.921+0000 7f623103a700 1 -- 172.21.15.37:0/463027190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f622c101810 msgr2=0x7f622c102570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:46.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.921+0000 7f623103a700 1 --2- 172.21.15.37:0/463027190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f622c101810 0x7f622c102570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.921+0000 7f623103a700 1 -- 172.21.15.37:0/463027190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622c1a7050 con 0x7f622c107570 2024-06-10T00:59:46.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.921+0000 7f623103a700 1 --2- 172.21.15.37:0/463027190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f622c107570 0x7f622c102ab0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f62240038b0 tx=0x7f6224005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:46.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.921+0000 7f6229ffb700 1 -- 172.21.15.37:0/463027190 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224027070 con 0x7f622c107570 2024-06-10T00:59:46.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.921+0000 7f6229ffb700 1 -- 172.21.15.37:0/463027190 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6224005690 con 0x7f622c107570 2024-06-10T00:59:46.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.921+0000 7f6232a9d700 1 -- 172.21.15.37:0/463027190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f622c1a72e0 con 0x7f622c107570 2024-06-10T00:59:46.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.921+0000 7f6229ffb700 1 -- 172.21.15.37:0/463027190 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224022cc0 con 0x7f622c107570 2024-06-10T00:59:46.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.921+0000 7f6232a9d700 1 -- 172.21.15.37:0/463027190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f622c1a7730 con 0x7f622c107570 2024-06-10T00:59:46.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.923+0000 7f6229ffb700 1 -- 172.21.15.37:0/463027190 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f622402a070 con 0x7f622c107570 2024-06-10T00:59:46.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.924+0000 7f6229ffb700 1 --2- 172.21.15.37:0/463027190 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f62180a2c20 0x7f62180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:46.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.924+0000 7f6230839700 1 --2- 172.21.15.37:0/463027190 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f62180a2c20 0x7f62180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:46.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.924+0000 7f6229ffb700 1 -- 172.21.15.37:0/463027190 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f62240d01d0 con 0x7f622c107570 2024-06-10T00:59:46.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.925+0000 7f6230839700 1 --2- 172.21.15.37:0/463027190 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f62180a2c20 0x7f62180a50e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f622c077c30 tx=0x7f621c009040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:46.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.939+0000 7f623d320b80 1 Processor -- start 2024-06-10T00:59:46.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.939+0000 7f623d320b80 1 -- start start 2024-06-10T00:59:46.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.939+0000 7f623d320b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56486708f640 0x56486708fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:46.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.939+0000 7f623d320b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56486708ff60 0x5648670899e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:46.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.940+0000 7f623d320b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564867089f20 0x56486708a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:46.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.940+0000 7f623d320b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564866fdb810 con 0x564867089f20 2024-06-10T00:59:46.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.940+0000 7f623d320b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564866f83720 con 0x56486708ff60 2024-06-10T00:59:46.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.940+0000 7f623d320b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564866fd77d0 con 0x56486708f640 2024-06-10T00:59:46.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.940+0000 7f623103a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564867089f20 0x56486708a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:46.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.940+0000 7f623103a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564867089f20 0x56486708a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40596/0 (socket says 172.21.15.37:40596) 2024-06-10T00:59:46.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.940+0000 7f623103a700 1 -- 172.21.15.37:0/2415890772 learned_addr learned my addr 172.21.15.37:0/2415890772 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:46.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.940+0000 7f6230839700 1 --2- 172.21.15.37:0/2415890772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56486708f640 0x56486708fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:46.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.940+0000 7f622bfff700 1 --2- 172.21.15.37:0/2415890772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56486708ff60 0x5648670899e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:46.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.940+0000 7f623103a700 1 -- 172.21.15.37:0/2415890772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56486708f640 msgr2=0x56486708fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:46.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.940+0000 7f623103a700 1 --2- 172.21.15.37:0/2415890772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56486708f640 0x56486708fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.940+0000 7f623103a700 1 -- 172.21.15.37:0/2415890772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56486708ff60 msgr2=0x5648670899e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:46.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.940+0000 7f623103a700 1 --2- 172.21.15.37:0/2415890772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56486708ff60 0x5648670899e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.940+0000 7f623103a700 1 -- 172.21.15.37:0/2415890772 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564866f79f50 con 0x564867089f20 2024-06-10T00:59:46.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.940+0000 7f6230839700 1 --2- 172.21.15.37:0/2415890772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56486708f640 0x56486708fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:59:46.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.940+0000 7f622bfff700 1 --2- 172.21.15.37:0/2415890772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56486708ff60 0x5648670899e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:59:46.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.940+0000 7f623103a700 1 --2- 172.21.15.37:0/2415890772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564867089f20 0x56486708a340 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f622409c660 tx=0x7f62240042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33325dcb92c7203 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:46.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.940+0000 7f6222ffd700 1 -- 172.21.15.37:0/2415890772 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224022e60 con 0x564867089f20 2024-06-10T00:59:46.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.940+0000 7f6222ffd700 1 -- 172.21.15.37:0/2415890772 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6224022470 con 0x564867089f20 2024-06-10T00:59:46.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.940+0000 7f6222ffd700 1 -- 172.21.15.37:0/2415890772 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62240a4e20 con 0x564867089f20 2024-06-10T00:59:46.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.941+0000 7f623d320b80 1 -- 172.21.15.37:0/2415890772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564867089f20 msgr2=0x56486708a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:46.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.941+0000 7f623d320b80 1 --2- 172.21.15.37:0/2415890772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564867089f20 0x56486708a340 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f622409c660 tx=0x7f62240042e0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.941+0000 7f623d320b80 1 -- 172.21.15.37:0/2415890772 shutdown_connections 2024-06-10T00:59:46.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.941+0000 7f623d320b80 1 --2- 172.21.15.37:0/2415890772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564867089f20 0x56486708a340 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.941+0000 7f623d320b80 1 --2- 172.21.15.37:0/2415890772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56486708f640 0x56486708fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.941+0000 7f623d320b80 1 --2- 172.21.15.37:0/2415890772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56486708ff60 0x5648670899e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.941+0000 7f623d320b80 1 -- 172.21.15.37:0/2415890772 >> 172.21.15.37:0/2415890772 conn(0x564866fb4cc0 msgr2=0x564867084850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:46.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.941+0000 7f623d320b80 1 -- 172.21.15.37:0/2415890772 shutdown_connections 2024-06-10T00:59:46.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.941+0000 7f623d320b80 1 -- 172.21.15.37:0/2415890772 wait complete. 2024-06-10T00:59:46.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.942+0000 7f623d320b80 1 -- start start 2024-06-10T00:59:46.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.942+0000 7f623d320b80 1 Processor -- start 2024-06-10T00:59:46.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.942+0000 7f623d320b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564867089f20 0x564867095d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:46.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.942+0000 7f623d320b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56486708f640 0x564867093010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:46.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.942+0000 7f623d320b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56486708ff60 0x564867093550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:46.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.942+0000 7f623d320b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5648670966f0 con 0x56486708ff60 2024-06-10T00:59:46.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.942+0000 7f623d320b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564866f7bbf0 con 0x56486708f640 2024-06-10T00:59:46.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.942+0000 7f623d320b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564867096870 con 0x564867089f20 2024-06-10T00:59:46.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.942+0000 7f623d320b80 10 client.0 _ll_register_callbacks cb 0x564866f75920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:59:46.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.942+0000 7f623103a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56486708ff60 0x564867093550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:46.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.942+0000 7f623103a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56486708ff60 0x564867093550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40610/0 (socket says 172.21.15.37:40610) 2024-06-10T00:59:46.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.942+0000 7f623103a700 1 -- 172.21.15.37:0/1066191747 learned_addr learned my addr 172.21.15.37:0/1066191747 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:46.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.942+0000 7f622bfff700 1 --2- 172.21.15.37:0/1066191747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56486708f640 0x564867093010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:46.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.943+0000 7f6230839700 1 --2- 172.21.15.37:0/1066191747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564867089f20 0x564867095d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:46.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.943+0000 7f623103a700 1 -- 172.21.15.37:0/1066191747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564867089f20 msgr2=0x564867095d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:46.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.943+0000 7f623103a700 1 --2- 172.21.15.37:0/1066191747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564867089f20 0x564867095d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.943+0000 7f623103a700 1 -- 172.21.15.37:0/1066191747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56486708f640 msgr2=0x564867093010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:46.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.943+0000 7f623103a700 1 --2- 172.21.15.37:0/1066191747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56486708f640 0x564867093010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.943+0000 7f623103a700 1 -- 172.21.15.37:0/1066191747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564866f2a8d0 con 0x56486708ff60 2024-06-10T00:59:46.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.943+0000 7f622bfff700 1 --2- 172.21.15.37:0/1066191747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56486708f640 0x564867093010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:59:46.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.943+0000 7f623103a700 1 --2- 172.21.15.37:0/1066191747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56486708ff60 0x564867093550 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f62240cd930 tx=0x7f62240d0160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c28c3bebff975e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:46.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.943+0000 7f620bfff700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:59:46.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.943+0000 7f620bfff700 1 -- 172.21.15.37:0/1066191747 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224022e60 con 0x56486708ff60 2024-06-10T00:59:46.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.943+0000 7f620bfff700 1 -- 172.21.15.37:0/1066191747 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f62240b0b60 con 0x56486708ff60 2024-06-10T00:59:46.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.943+0000 7f623d320b80 1 -- 172.21.15.37:0/1066191747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564866fe8d10 con 0x56486708ff60 2024-06-10T00:59:46.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.943+0000 7f620bfff700 1 -- 172.21.15.37:0/1066191747 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f622409c6d0 con 0x56486708ff60 2024-06-10T00:59:46.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.943+0000 7f623d320b80 10 client.15939 fetch_fsmap learned FSMap version 27 2024-06-10T00:59:46.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.943+0000 7f623d320b80 1 -- 172.21.15.37:0/1066191747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5648670966f0 con 0x56486708ff60 2024-06-10T00:59:46.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.943+0000 7f620bfff700 1 -- 172.21.15.37:0/1066191747 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564866fe8d10 con 0x56486708ff60 2024-06-10T00:59:46.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.944+0000 7f620bfff700 1 -- 172.21.15.37:0/1066191747 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f622409bbb0 con 0x56486708ff60 2024-06-10T00:59:46.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.944+0000 7f623d320b80 10 client.15939 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:59:46.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.944+0000 7f623d320b80 10 client.15939 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-06-10T00:59:46.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.944+0000 7f623d320b80 20 client.15939 populate_metadata read hostname 'smithi037' 2024-06-10T00:59:46.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.944+0000 7f623d320b80 1 --2- 172.21.15.37:0/1066191747 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x5648670a33e0 0x5648670a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:46.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.944+0000 7f623d320b80 4 client.15939 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:59:46.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.944+0000 7f623d320b80 1 -- 172.21.15.37:0/1066191747 --> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564866fe3290 con 0x5648670a33e0 2024-06-10T00:59:46.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.944+0000 7f6230839700 1 --2- 172.21.15.37:0/1066191747 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x5648670a33e0 0x5648670a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:46.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.944+0000 7f6230839700 1 --2- 172.21.15.37:0/1066191747 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x5648670a33e0 0x5648670a57d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b74c097ae4d777c3 server_cookie=9e30567b763ca001 in_seq=0 out_seq=0 2024-06-10T00:59:46.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.944+0000 7f620bfff700 10 client.15939 ms_handle_connect on v2:172.21.15.94:6838/946269331 2024-06-10T00:59:46.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.945+0000 7f620bfff700 1 -- 172.21.15.37:0/1066191747 <== mds.1 v2:172.21.15.94:6838/946269331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564866fe3290 con 0x5648670a33e0 2024-06-10T00:59:46.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.945+0000 7f620bfff700 10 client.15939 handle_command_reply: tid=0 2024-06-10T00:59:46.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.945+0000 7f623d320b80 1 client.15939 shutdown 2024-06-10T00:59:46.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.945+0000 7f623d320b80 1 -- 172.21.15.37:0/1066191747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56486708ff60 msgr2=0x564867093550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:46.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.945+0000 7f623d320b80 1 --2- 172.21.15.37:0/1066191747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56486708ff60 0x564867093550 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f62240cd930 tx=0x7f62240d0160 comp rx=0 tx=0).stop 2024-06-10T00:59:46.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.945+0000 7f623d320b80 1 -- 172.21.15.37:0/1066191747 shutdown_connections 2024-06-10T00:59:46.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.945+0000 7f623d320b80 1 --2- 172.21.15.37:0/1066191747 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x5648670a33e0 0x5648670a57d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.945+0000 7f623d320b80 1 --2- 172.21.15.37:0/1066191747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56486708ff60 0x564867093550 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.945+0000 7f623d320b80 1 --2- 172.21.15.37:0/1066191747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564867089f20 0x564867095d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.945+0000 7f620bfff700 0 client.15939 ms_handle_reset on v2:172.21.15.94:6838/946269331 2024-06-10T00:59:46.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.945+0000 7f623d320b80 1 --2- 172.21.15.37:0/1066191747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56486708f640 0x564867093010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.945+0000 7f623d320b80 1 -- 172.21.15.37:0/1066191747 >> 172.21.15.37:0/1066191747 conn(0x564866fb4cc0 msgr2=0x5648670889d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:46.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.946+0000 7f623d320b80 1 -- 172.21.15.37:0/1066191747 shutdown_connections 2024-06-10T00:59:46.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.946+0000 7f623d320b80 1 -- 172.21.15.37:0/1066191747 wait complete. 2024-06-10T00:59:46.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.946+0000 7f623d320b80 20 client.15939 trim_cache size 0 max 16384 2024-06-10T00:59:46.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.961+0000 7f623d320b80 1 Processor -- start 2024-06-10T00:59:46.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.961+0000 7f623d320b80 1 -- start start 2024-06-10T00:59:46.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.961+0000 7f623d320b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564867089f20 0x5648670899e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:46.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.961+0000 7f623d320b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56486708f640 0x56486707e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:46.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.961+0000 7f623d320b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56486708ff60 0x564867090bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:46.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.961+0000 7f623d320b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f622c1b2ec0 con 0x56486708f640 2024-06-10T00:59:46.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.961+0000 7f623d320b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f622c1b0750 con 0x564867089f20 2024-06-10T00:59:46.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.961+0000 7f623d320b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f622c1ae5a0 con 0x56486708ff60 2024-06-10T00:59:46.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.962+0000 7f622bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56486708f640 0x56486707e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:46.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.962+0000 7f622bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56486708f640 0x56486707e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40624/0 (socket says 172.21.15.37:40624) 2024-06-10T00:59:46.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.962+0000 7f622bfff700 1 -- 172.21.15.37:0/1579675541 learned_addr learned my addr 172.21.15.37:0/1579675541 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:46.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.962+0000 7f6230839700 1 --2- 172.21.15.37:0/1579675541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564867089f20 0x5648670899e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:46.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.962+0000 7f623103a700 1 --2- 172.21.15.37:0/1579675541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56486708ff60 0x564867090bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:46.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.962+0000 7f622bfff700 1 -- 172.21.15.37:0/1579675541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56486708ff60 msgr2=0x564867090bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:46.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.962+0000 7f622bfff700 1 --2- 172.21.15.37:0/1579675541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56486708ff60 0x564867090bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.962+0000 7f622bfff700 1 -- 172.21.15.37:0/1579675541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564867089f20 msgr2=0x5648670899e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:46.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.962+0000 7f622bfff700 1 --2- 172.21.15.37:0/1579675541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564867089f20 0x5648670899e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.962+0000 7f622bfff700 1 -- 172.21.15.37:0/1579675541 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564867091580 con 0x56486708f640 2024-06-10T00:59:46.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.962+0000 7f623103a700 1 --2- 172.21.15.37:0/1579675541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56486708ff60 0x564867090bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:59:46.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.962+0000 7f622bfff700 1 --2- 172.21.15.37:0/1579675541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56486708f640 0x56486707e4d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f62140044c0 tx=0x7f621400b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ba62e1c67bff69 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:46.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.962+0000 7f62227fc700 1 -- 172.21.15.37:0/1579675541 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6214004830 con 0x56486708f640 2024-06-10T00:59:46.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.962+0000 7f62227fc700 1 -- 172.21.15.37:0/1579675541 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6214005020 con 0x56486708f640 2024-06-10T00:59:46.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.962+0000 7f62227fc700 1 -- 172.21.15.37:0/1579675541 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6214003aa0 con 0x56486708f640 2024-06-10T00:59:46.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.963+0000 7f623d320b80 1 -- 172.21.15.37:0/1579675541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56486708f640 msgr2=0x56486707e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:46.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.963+0000 7f623d320b80 1 --2- 172.21.15.37:0/1579675541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56486708f640 0x56486707e4d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f62140044c0 tx=0x7f621400b9f0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.963+0000 7f623d320b80 1 -- 172.21.15.37:0/1579675541 shutdown_connections 2024-06-10T00:59:46.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.963+0000 7f623d320b80 1 --2- 172.21.15.37:0/1579675541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56486708ff60 0x564867090bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.963+0000 7f623d320b80 1 --2- 172.21.15.37:0/1579675541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564867089f20 0x5648670899e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.963+0000 7f623d320b80 1 --2- 172.21.15.37:0/1579675541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56486708f640 0x56486707e4d0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.963+0000 7f623d320b80 1 -- 172.21.15.37:0/1579675541 >> 172.21.15.37:0/1579675541 conn(0x564866fb4cc0 msgr2=0x5648670917d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:46.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.963+0000 7f623d320b80 1 -- 172.21.15.37:0/1579675541 shutdown_connections 2024-06-10T00:59:46.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.963+0000 7f623d320b80 1 -- 172.21.15.37:0/1579675541 wait complete. 2024-06-10T00:59:46.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.964+0000 7f623d320b80 1 -- start start 2024-06-10T00:59:46.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.964+0000 7f623d320b80 1 Processor -- start 2024-06-10T00:59:46.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.964+0000 7f623d320b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564867089f20 0x56486709c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:46.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.964+0000 7f623d320b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56486708f640 0x5648670aab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:46.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.964+0000 7f623d320b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56486708ff60 0x5648670ab270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:46.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.964+0000 7f623d320b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f622c1aec10 con 0x56486708f640 2024-06-10T00:59:46.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.964+0000 7f623d320b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f622c1b0080 con 0x564867089f20 2024-06-10T00:59:46.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.964+0000 7f623d320b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f622c1af2e0 con 0x56486708ff60 2024-06-10T00:59:46.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.964+0000 7f623d320b80 10 client.0 _ll_register_callbacks cb 0x564866f75920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:59:46.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.964+0000 7f622bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56486708f640 0x5648670aab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:46.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.964+0000 7f622bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56486708f640 0x5648670aab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40630/0 (socket says 172.21.15.37:40630) 2024-06-10T00:59:46.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.964+0000 7f622bfff700 1 -- 172.21.15.37:0/2787539513 learned_addr learned my addr 172.21.15.37:0/2787539513 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:46.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.964+0000 7f6230839700 1 --2- 172.21.15.37:0/2787539513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564867089f20 0x56486709c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:46.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.964+0000 7f623103a700 1 --2- 172.21.15.37:0/2787539513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56486708ff60 0x5648670ab270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:46.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.965+0000 7f622bfff700 1 -- 172.21.15.37:0/2787539513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56486708ff60 msgr2=0x5648670ab270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:46.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.965+0000 7f622bfff700 1 --2- 172.21.15.37:0/2787539513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56486708ff60 0x5648670ab270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.965+0000 7f622bfff700 1 -- 172.21.15.37:0/2787539513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564867089f20 msgr2=0x56486709c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:46.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.965+0000 7f622bfff700 1 --2- 172.21.15.37:0/2787539513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564867089f20 0x56486709c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.965+0000 7f622bfff700 1 -- 172.21.15.37:0/2787539513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56486710d050 con 0x56486708f640 2024-06-10T00:59:46.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.965+0000 7f623103a700 1 --2- 172.21.15.37:0/2787539513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56486708ff60 0x5648670ab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:59:46.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.965+0000 7f622bfff700 1 --2- 172.21.15.37:0/2787539513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56486708f640 0x5648670aab30 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6214005f90 tx=0x7f6214005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39e1435500d03fe server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:46.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.965+0000 7f620bfff700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:59:46.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.965+0000 7f620bfff700 1 -- 172.21.15.37:0/2787539513 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6214020030 con 0x56486708f640 2024-06-10T00:59:46.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.965+0000 7f620bfff700 1 -- 172.21.15.37:0/2787539513 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f6214004b10 con 0x56486708f640 2024-06-10T00:59:46.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.965+0000 7f623d320b80 1 -- 172.21.15.37:0/2787539513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6214005020 con 0x56486708f640 2024-06-10T00:59:46.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.965+0000 7f620bfff700 1 -- 172.21.15.37:0/2787539513 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6214030050 con 0x56486708f640 2024-06-10T00:59:46.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.965+0000 7f623d320b80 10 client.15945 fetch_fsmap learned FSMap version 27 2024-06-10T00:59:46.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.965+0000 7f623d320b80 1 -- 172.21.15.37:0/2787539513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56486710e1e0 con 0x56486708f640 2024-06-10T00:59:46.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.965+0000 7f620bfff700 1 -- 172.21.15.37:0/2787539513 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6214005020 con 0x56486708f640 2024-06-10T00:59:46.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.966+0000 7f620bfff700 1 -- 172.21.15.37:0/2787539513 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f621401c650 con 0x56486708f640 2024-06-10T00:59:46.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.966+0000 7f623d320b80 10 client.15945 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:59:46.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.966+0000 7f623d320b80 10 client.15945 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-06-10T00:59:46.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.966+0000 7f623d320b80 20 client.15945 populate_metadata read hostname 'smithi037' 2024-06-10T00:59:46.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.966+0000 7f623d320b80 1 --2- 172.21.15.37:0/2787539513 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x56486710e7c0 0x564867110be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:46.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.966+0000 7f623d320b80 4 client.15945 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:59:46.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.966+0000 7f623d320b80 1 -- 172.21.15.37:0/2787539513 --> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564866e5d250 con 0x56486710e7c0 2024-06-10T00:59:46.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.966+0000 7f6230839700 1 --2- 172.21.15.37:0/2787539513 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x56486710e7c0 0x564867110be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:46.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.966+0000 7f620bfff700 10 client.15945 ms_handle_connect on v2:172.21.15.94:6838/946269331 2024-06-10T00:59:46.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.966+0000 7f6230839700 1 --2- 172.21.15.37:0/2787539513 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x56486710e7c0 0x564867110be0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d8b521a0549448ac server_cookie=fd66e538670e01e5 in_seq=0 out_seq=0 2024-06-10T00:59:46.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.967+0000 7f620bfff700 1 -- 172.21.15.37:0/2787539513 <== mds.1 v2:172.21.15.94:6838/946269331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564866e5d250 con 0x56486710e7c0 2024-06-10T00:59:46.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.967+0000 7f620bfff700 10 client.15945 handle_command_reply: tid=0 2024-06-10T00:59:46.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.967+0000 7f623d320b80 1 client.15945 shutdown 2024-06-10T00:59:46.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.967+0000 7f623d320b80 1 -- 172.21.15.37:0/2787539513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56486708f640 msgr2=0x5648670aab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:46.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.967+0000 7f623d320b80 1 --2- 172.21.15.37:0/2787539513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56486708f640 0x5648670aab30 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6214005f90 tx=0x7f6214005670 comp rx=0 tx=0).stop 2024-06-10T00:59:46.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.967+0000 7f623d320b80 1 -- 172.21.15.37:0/2787539513 shutdown_connections 2024-06-10T00:59:46.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.967+0000 7f623d320b80 1 --2- 172.21.15.37:0/2787539513 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x56486710e7c0 0x564867110be0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.967+0000 7f623d320b80 1 --2- 172.21.15.37:0/2787539513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56486708ff60 0x5648670ab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.967+0000 7f623d320b80 1 --2- 172.21.15.37:0/2787539513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564867089f20 0x56486709c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.967+0000 7f620bfff700 0 client.15945 ms_handle_reset on v2:172.21.15.94:6838/946269331 2024-06-10T00:59:46.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.967+0000 7f623d320b80 1 --2- 172.21.15.37:0/2787539513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56486708f640 0x5648670aab30 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.967+0000 7f623d320b80 1 -- 172.21.15.37:0/2787539513 >> 172.21.15.37:0/2787539513 conn(0x564866fb4cc0 msgr2=0x564867092060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:46.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.967+0000 7f623d320b80 1 -- 172.21.15.37:0/2787539513 shutdown_connections 2024-06-10T00:59:46.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.967+0000 7f623d320b80 1 -- 172.21.15.37:0/2787539513 wait complete. 2024-06-10T00:59:46.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.967+0000 7f623d320b80 20 client.15945 trim_cache size 0 max 16384 2024-06-10T00:59:46.974 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T00:59:46.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.968+0000 7f6232a9d700 1 -- 172.21.15.37:0/463027190 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f62180a2c20 msgr2=0x7f62180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:46.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.968+0000 7f6232a9d700 1 --2- 172.21.15.37:0/463027190 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f62180a2c20 0x7f62180a50e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f622c077c30 tx=0x7f621c009040 comp rx=0 tx=0).stop 2024-06-10T00:59:46.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.968+0000 7f6232a9d700 1 -- 172.21.15.37:0/463027190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f622c107570 msgr2=0x7f622c102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:46.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.968+0000 7f6232a9d700 1 --2- 172.21.15.37:0/463027190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f622c107570 0x7f622c102ab0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f62240038b0 tx=0x7f6224005e80 comp rx=0 tx=0).stop 2024-06-10T00:59:46.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.968+0000 7f6232a9d700 1 -- 172.21.15.37:0/463027190 shutdown_connections 2024-06-10T00:59:46.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.968+0000 7f6232a9d700 1 --2- 172.21.15.37:0/463027190 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f62180a2c20 0x7f62180a50e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.968+0000 7f6232a9d700 1 --2- 172.21.15.37:0/463027190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f622c107570 0x7f622c102ab0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.968+0000 7f6232a9d700 1 --2- 172.21.15.37:0/463027190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f622c100e50 0x7f622c102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.968+0000 7f6232a9d700 1 --2- 172.21.15.37:0/463027190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f622c101810 0x7f622c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:46.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.968+0000 7f6232a9d700 1 -- 172.21.15.37:0/463027190 >> 172.21.15.37:0/463027190 conn(0x7f622c0766b0 msgr2=0x7f622c0ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:46.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.968+0000 7f6232a9d700 1 -- 172.21.15.37:0/463027190 shutdown_connections 2024-06-10T00:59:46.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:46.968+0000 7f6232a9d700 1 -- 172.21.15.37:0/463027190 wait complete. 2024-06-10T00:59:48.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:48 smithi167 ceph-mon[36917]: pgmap v523: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 13 op/s 2024-06-10T00:59:48.341 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:48 smithi037 ceph-mon[32192]: pgmap v523: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 13 op/s 2024-06-10T00:59:48.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:48 smithi094 ceph-mon[36708]: pgmap v523: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 13 op/s 2024-06-10T00:59:48.368 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T00:59:48.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.869+0000 7faeda2a8700 1 -- 172.21.15.37:0/4037229911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faed4101380 msgr2=0x7faed4101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:48.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.869+0000 7faeda2a8700 1 --2- 172.21.15.37:0/4037229911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faed4101380 0x7faed4101800 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7faebc009990 tx=0x7faebc009ca0 comp rx=0 tx=0).stop 2024-06-10T00:59:48.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.869+0000 7faeda2a8700 1 -- 172.21.15.37:0/4037229911 shutdown_connections 2024-06-10T00:59:48.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.869+0000 7faeda2a8700 1 --2- 172.21.15.37:0/4037229911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faed4101d40 0x7faed410e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:48.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.869+0000 7faeda2a8700 1 --2- 172.21.15.37:0/4037229911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed41073c0 0x7faed41077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:48.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.869+0000 7faeda2a8700 1 --2- 172.21.15.37:0/4037229911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faed4101380 0x7faed4101800 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:48.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.869+0000 7faeda2a8700 1 -- 172.21.15.37:0/4037229911 >> 172.21.15.37:0/4037229911 conn(0x7faed4078330 msgr2=0x7faed4078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:48.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.869+0000 7faeda2a8700 1 -- 172.21.15.37:0/4037229911 shutdown_connections 2024-06-10T00:59:48.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.869+0000 7faeda2a8700 1 -- 172.21.15.37:0/4037229911 wait complete. 2024-06-10T00:59:48.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.870+0000 7faeda2a8700 1 Processor -- start 2024-06-10T00:59:48.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.870+0000 7faeda2a8700 1 -- start start 2024-06-10T00:59:48.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.871+0000 7faeda2a8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faed4101380 0x7faed4199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:48.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.871+0000 7faeda2a8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faed4101d40 0x7faed4199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:48.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.871+0000 7faeda2a8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed41073c0 0x7faed41a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:48.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.871+0000 7faeda2a8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faed4111630 con 0x7faed4101d40 2024-06-10T00:59:48.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.871+0000 7faeda2a8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faed41114b0 con 0x7faed4101380 2024-06-10T00:59:48.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.871+0000 7faeda2a8700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faed41117b0 con 0x7faed41073c0 2024-06-10T00:59:48.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.871+0000 7faed37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faed4101d40 0x7faed4199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:48.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.871+0000 7faed37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faed4101d40 0x7faed4199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40642/0 (socket says 172.21.15.37:40642) 2024-06-10T00:59:48.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.871+0000 7faed37fe700 1 -- 172.21.15.37:0/3762127545 learned_addr learned my addr 172.21.15.37:0/3762127545 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:48.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.871+0000 7faed8845700 1 --2- 172.21.15.37:0/3762127545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed41073c0 0x7faed41a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:48.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.871+0000 7faed3fff700 1 --2- 172.21.15.37:0/3762127545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faed4101380 0x7faed4199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:48.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.871+0000 7faed37fe700 1 -- 172.21.15.37:0/3762127545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed41073c0 msgr2=0x7faed41a4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:48.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.871+0000 7faed37fe700 1 --2- 172.21.15.37:0/3762127545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed41073c0 0x7faed41a4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:48.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.871+0000 7faed37fe700 1 -- 172.21.15.37:0/3762127545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faed4101380 msgr2=0x7faed4199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:48.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.871+0000 7faed37fe700 1 --2- 172.21.15.37:0/3762127545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faed4101380 0x7faed4199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:48.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.871+0000 7faed37fe700 1 -- 172.21.15.37:0/3762127545 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed4197860 con 0x7faed4101d40 2024-06-10T00:59:48.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.872+0000 7faed37fe700 1 --2- 172.21.15.37:0/3762127545 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faed4101d40 0x7faed4199fb0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7faebc000c00 tx=0x7faebc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:48.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.872+0000 7faed17fa700 1 -- 172.21.15.37:0/3762127545 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc027070 con 0x7faed4101d40 2024-06-10T00:59:48.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.872+0000 7faed17fa700 1 -- 172.21.15.37:0/3762127545 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faebc003e40 con 0x7faed4101d40 2024-06-10T00:59:48.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.872+0000 7faeda2a8700 1 -- 172.21.15.37:0/3762127545 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed4197af0 con 0x7faed4101d40 2024-06-10T00:59:48.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.872+0000 7faed17fa700 1 -- 172.21.15.37:0/3762127545 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc022d10 con 0x7faed4101d40 2024-06-10T00:59:48.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.872+0000 7faeda2a8700 1 -- 172.21.15.37:0/3762127545 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed4197fd0 con 0x7faed4101d40 2024-06-10T00:59:48.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.875+0000 7faed17fa700 1 -- 172.21.15.37:0/3762127545 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7faebc02a070 con 0x7faed4101d40 2024-06-10T00:59:48.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.875+0000 7faed17fa700 1 --2- 172.21.15.37:0/3762127545 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7faec00a2c20 0x7faec00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:48.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.876+0000 7faed3fff700 1 --2- 172.21.15.37:0/3762127545 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7faec00a2c20 0x7faec00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:48.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.876+0000 7faed17fa700 1 -- 172.21.15.37:0/3762127545 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7faebc0d0260 con 0x7faed4101d40 2024-06-10T00:59:48.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.876+0000 7faed3fff700 1 --2- 172.21.15.37:0/3762127545 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7faec00a2c20 0x7faec00a50e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faec4005950 tx=0x7faec400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:48.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.878+0000 7faeda2a8700 1 -- 172.21.15.37:0/3762127545 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faed4102a70 con 0x7faed4101d40 2024-06-10T00:59:48.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:48.884+0000 7faed17fa700 1 -- 172.21.15.37:0/3762127545 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7faebc05e850 con 0x7faed4101d40 2024-06-10T00:59:48.904 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:59:49.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:49.143+0000 7faeda2a8700 1 -- 172.21.15.37:0/3762127545 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faed406aff0 con 0x7faed4101d40 2024-06-10T00:59:49.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:49.144+0000 7faed17fa700 1 -- 172.21.15.37:0/3762127545 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11867 (secure 0 0 0) 0x7faebc013070 con 0x7faed4101d40 2024-06-10T00:59:49.143 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T00:59:49.144 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T00:56:30.291171+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24451,24458,24472]},"id":1}]} 2024-06-10T00:59:49.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:49.146+0000 7faeda2a8700 1 -- 172.21.15.37:0/3762127545 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7faec00a2c20 msgr2=0x7faec00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:49.146+0000 7faeda2a8700 1 --2- 172.21.15.37:0/3762127545 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7faec00a2c20 0x7faec00a50e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faec4005950 tx=0x7faec400b410 comp rx=0 tx=0).stop 2024-06-10T00:59:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:49.146+0000 7faeda2a8700 1 -- 172.21.15.37:0/3762127545 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faed4101d40 msgr2=0x7faed4199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:49.146+0000 7faeda2a8700 1 --2- 172.21.15.37:0/3762127545 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faed4101d40 0x7faed4199fb0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7faebc000c00 tx=0x7faebc0047b0 comp rx=0 tx=0).stop 2024-06-10T00:59:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:49.146+0000 7faeda2a8700 1 -- 172.21.15.37:0/3762127545 shutdown_connections 2024-06-10T00:59:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:49.146+0000 7faeda2a8700 1 --2- 172.21.15.37:0/3762127545 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7faec00a2c20 0x7faec00a50e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:49.146+0000 7faeda2a8700 1 --2- 172.21.15.37:0/3762127545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed41073c0 0x7faed41a4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:49.146+0000 7faeda2a8700 1 --2- 172.21.15.37:0/3762127545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faed4101380 0x7faed4199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:49.146+0000 7faeda2a8700 1 --2- 172.21.15.37:0/3762127545 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faed4101d40 0x7faed4199fb0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:49.146+0000 7faeda2a8700 1 -- 172.21.15.37:0/3762127545 >> 172.21.15.37:0/3762127545 conn(0x7faed4078330 msgr2=0x7faed41000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:49.146+0000 7faeda2a8700 1 -- 172.21.15.37:0/3762127545 shutdown_connections 2024-06-10T00:59:49.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:49.146+0000 7faeda2a8700 1 -- 172.21.15.37:0/3762127545 wait complete. 2024-06-10T00:59:49.147 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-06-10T00:59:50.095 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24472 quiesce db --set-id 5049796b --release --await' 2024-06-10T00:59:50.095 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24472 quiesce db --set-id 5049796b --release --await 2024-06-10T00:59:50.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:50 smithi167 ceph-mon[36917]: pgmap v524: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.9 KiB/s wr, 12 op/s 2024-06-10T00:59:50.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:50 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3762127545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:59:50.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:50 smithi094 ceph-mon[36708]: pgmap v524: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.9 KiB/s wr, 12 op/s 2024-06-10T00:59:50.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:50 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3762127545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:59:50.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:50 smithi037 ceph-mon[32192]: pgmap v524: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.9 KiB/s wr, 12 op/s 2024-06-10T00:59:50.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:50 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3762127545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T00:59:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.844+0000 7fc022722640 1 -- 172.21.15.37:0/2481914919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc01c100070 msgr2=0x7fc01c100490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.844+0000 7fc022722640 1 --2- 172.21.15.37:0/2481914919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc01c100070 0x7fc01c100490 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009900 tx=0x7fc00c02a410 comp rx=0 tx=0).stop 2024-06-10T00:59:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.844+0000 7fc022722640 1 -- 172.21.15.37:0/2481914919 shutdown_connections 2024-06-10T00:59:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.844+0000 7fc022722640 1 --2- 172.21.15.37:0/2481914919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc01c101c20 0x7fc01c1084b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.844+0000 7fc022722640 1 --2- 172.21.15.37:0/2481914919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc01c101260 0x7fc01c1016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.844+0000 7fc022722640 1 --2- 172.21.15.37:0/2481914919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc01c100070 0x7fc01c100490 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.844+0000 7fc022722640 1 -- 172.21.15.37:0/2481914919 >> 172.21.15.37:0/2481914919 conn(0x7fc01c0fb600 msgr2=0x7fc01c0fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.844+0000 7fc022722640 1 -- 172.21.15.37:0/2481914919 shutdown_connections 2024-06-10T00:59:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.844+0000 7fc022722640 1 -- 172.21.15.37:0/2481914919 wait complete. 2024-06-10T00:59:50.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.859+0000 7fc022722640 1 Processor -- start 2024-06-10T00:59:50.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.859+0000 7fc022722640 1 -- start start 2024-06-10T00:59:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.859+0000 7fc022722640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc01c100070 0x7fc01c197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.859+0000 7fc022722640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc01c101260 0x7fc01c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.859+0000 7fc022722640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc01c101c20 0x7fc01c19f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.859+0000 7fc022722640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c10c5b0 con 0x7fc01c100070 2024-06-10T00:59:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.859+0000 7fc022722640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c10c430 con 0x7fc01c101260 2024-06-10T00:59:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.859+0000 7fc022722640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c10c730 con 0x7fc01c101c20 2024-06-10T00:59:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.859+0000 7fc01bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc01c100070 0x7fc01c197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.859+0000 7fc01bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc01c100070 0x7fc01c197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40648/0 (socket says 172.21.15.37:40648) 2024-06-10T00:59:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.859+0000 7fc01bfff640 1 -- 172.21.15.37:0/4218497405 learned_addr learned my addr 172.21.15.37:0/4218497405 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.859+0000 7fc020cbf640 1 --2- 172.21.15.37:0/4218497405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc01c101c20 0x7fc01c19f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.859+0000 7fc01b7fe640 1 --2- 172.21.15.37:0/4218497405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc01c101260 0x7fc01c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.860+0000 7fc01bfff640 1 -- 172.21.15.37:0/4218497405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc01c101c20 msgr2=0x7fc01c19f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.860+0000 7fc01bfff640 1 --2- 172.21.15.37:0/4218497405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc01c101c20 0x7fc01c19f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.860+0000 7fc01bfff640 1 -- 172.21.15.37:0/4218497405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc01c101260 msgr2=0x7fc01c198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.860+0000 7fc01bfff640 1 --2- 172.21.15.37:0/4218497405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc01c101260 0x7fc01c198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.860+0000 7fc01bfff640 1 -- 172.21.15.37:0/4218497405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01c19f900 con 0x7fc01c100070 2024-06-10T00:59:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.860+0000 7fc020cbf640 1 --2- 172.21.15.37:0/4218497405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc01c101c20 0x7fc01c19f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:59:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.860+0000 7fc01bfff640 1 --2- 172.21.15.37:0/4218497405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc01c100070 0x7fc01c197d40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc00c031440 tx=0x7fc00c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.860+0000 7fc0197fa640 1 -- 172.21.15.37:0/4218497405 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c03f030 con 0x7fc01c100070 2024-06-10T00:59:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.860+0000 7fc0197fa640 1 -- 172.21.15.37:0/4218497405 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc00c003ce0 con 0x7fc01c100070 2024-06-10T00:59:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.860+0000 7fc022722640 1 -- 172.21.15.37:0/4218497405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc01c19fb90 con 0x7fc01c100070 2024-06-10T00:59:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.860+0000 7fc0197fa640 1 -- 172.21.15.37:0/4218497405 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c035070 con 0x7fc01c100070 2024-06-10T00:59:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.860+0000 7fc022722640 1 -- 172.21.15.37:0/4218497405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc01c1a0080 con 0x7fc01c100070 2024-06-10T00:59:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.863+0000 7fc0197fa640 1 -- 172.21.15.37:0/4218497405 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc00c042030 con 0x7fc01c100070 2024-06-10T00:59:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.863+0000 7fc0197fa640 1 --2- 172.21.15.37:0/4218497405 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc0040a2da0 0x7fc0040a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.863+0000 7fc01b7fe640 1 --2- 172.21.15.37:0/4218497405 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc0040a2da0 0x7fc0040a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.863+0000 7fc0197fa640 1 -- 172.21.15.37:0/4218497405 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fc00c0f0a50 con 0x7fc01c100070 2024-06-10T00:59:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:50.985+0000 7fc01b7fe640 1 --2- 172.21.15.37:0/4218497405 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc0040a2da0 0x7fc0040a5260 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc010005fd0 tx=0x7fc010005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.001+0000 7fc025433740 1 Processor -- start 2024-06-10T00:59:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.001+0000 7fc025433740 1 -- start start 2024-06-10T00:59:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.001+0000 7fc025433740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5ee00420 0x561a5ee00840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.001+0000 7fc025433740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a5ee00e10 0x561a5ee05910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.001+0000 7fc025433740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a5ee05f80 0x561a5ee08410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.001+0000 7fc025433740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561a5ed5f4f0 con 0x561a5ee05f80 2024-06-10T00:59:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.001+0000 7fc025433740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561a5ecfd870 con 0x561a5ee00e10 2024-06-10T00:59:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.001+0000 7fc025433740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561a5ed5b4b0 con 0x561a5ee00420 2024-06-10T00:59:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.001+0000 7fc020cbf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a5ee05f80 0x561a5ee08410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.001+0000 7fc020cbf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a5ee05f80 0x561a5ee08410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40662/0 (socket says 172.21.15.37:40662) 2024-06-10T00:59:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.001+0000 7fc020cbf640 1 -- 172.21.15.37:0/1055842330 learned_addr learned my addr 172.21.15.37:0/1055842330 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.002+0000 7fc01b7fe640 1 --2- 172.21.15.37:0/1055842330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a5ee00e10 0x561a5ee05910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.002+0000 7fc020cbf640 1 -- 172.21.15.37:0/1055842330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5ee00420 msgr2=0x561a5ee00840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.002+0000 7fc01bfff640 1 --2- 172.21.15.37:0/1055842330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5ee00420 0x561a5ee00840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.002+0000 7fc020cbf640 1 --2- 172.21.15.37:0/1055842330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5ee00420 0x561a5ee00840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.002+0000 7fc020cbf640 1 -- 172.21.15.37:0/1055842330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a5ee00e10 msgr2=0x561a5ee05910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.002+0000 7fc020cbf640 1 --2- 172.21.15.37:0/1055842330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a5ee00e10 0x561a5ee05910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.002+0000 7fc020cbf640 1 -- 172.21.15.37:0/1055842330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a5ed199b0 con 0x561a5ee05f80 2024-06-10T00:59:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.002+0000 7fc01b7fe640 1 --2- 172.21.15.37:0/1055842330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a5ee00e10 0x561a5ee05910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:59:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.002+0000 7fc020cbf640 1 --2- 172.21.15.37:0/1055842330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a5ee05f80 0x561a5ee08410 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc014010450 tx=0x7fc014010910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4519e9143bebea7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.002+0000 7fc0027fc640 1 -- 172.21.15.37:0/1055842330 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc014017070 con 0x561a5ee05f80 2024-06-10T00:59:51.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.002+0000 7fc0027fc640 1 -- 172.21.15.37:0/1055842330 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc014011c40 con 0x561a5ee05f80 2024-06-10T00:59:51.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.002+0000 7fc0027fc640 1 -- 172.21.15.37:0/1055842330 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01401ad30 con 0x561a5ee05f80 2024-06-10T00:59:51.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.003+0000 7fc025433740 1 -- 172.21.15.37:0/1055842330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a5ee05f80 msgr2=0x561a5ee08410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.003+0000 7fc025433740 1 --2- 172.21.15.37:0/1055842330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a5ee05f80 0x561a5ee08410 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc014010450 tx=0x7fc014010910 comp rx=0 tx=0).stop 2024-06-10T00:59:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.003+0000 7fc025433740 1 -- 172.21.15.37:0/1055842330 shutdown_connections 2024-06-10T00:59:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.003+0000 7fc025433740 1 --2- 172.21.15.37:0/1055842330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a5ee05f80 0x561a5ee08410 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.003+0000 7fc025433740 1 --2- 172.21.15.37:0/1055842330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a5ee00e10 0x561a5ee05910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.003+0000 7fc025433740 1 --2- 172.21.15.37:0/1055842330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5ee00420 0x561a5ee00840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.003+0000 7fc025433740 1 -- 172.21.15.37:0/1055842330 >> 172.21.15.37:0/1055842330 conn(0x561a5edf7680 msgr2=0x561a5edf7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.003+0000 7fc025433740 1 -- 172.21.15.37:0/1055842330 shutdown_connections 2024-06-10T00:59:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.003+0000 7fc025433740 1 -- 172.21.15.37:0/1055842330 wait complete. 2024-06-10T00:59:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.059+0000 7fc025433740 1 -- start start 2024-06-10T00:59:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.078+0000 7fc025433740 1 Processor -- start 2024-06-10T00:59:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.078+0000 7fc025433740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a5ee00420 0x561a5ee0a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.078+0000 7fc025433740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5ee00e10 0x561a5ee0aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.078+0000 7fc025433740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a5ee05f80 0x561a5ee11be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.078+0000 7fc025433740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561a5ee08f30 con 0x561a5ee00420 2024-06-10T00:59:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.078+0000 7fc025433740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561a5ece8ae0 con 0x561a5ee05f80 2024-06-10T00:59:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.078+0000 7fc025433740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561a5ee090b0 con 0x561a5ee00e10 2024-06-10T00:59:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.078+0000 7fc01bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a5ee00420 0x561a5ee0a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.078+0000 7fc01bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a5ee00420 0x561a5ee0a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40676/0 (socket says 172.21.15.37:40676) 2024-06-10T00:59:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.078+0000 7fc01bfff640 1 -- 172.21.15.37:0/3647772018 learned_addr learned my addr 172.21.15.37:0/3647772018 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.079+0000 7fc01b7fe640 1 --2- 172.21.15.37:0/3647772018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5ee00e10 0x561a5ee0aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.079+0000 7fc020cbf640 1 --2- 172.21.15.37:0/3647772018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a5ee05f80 0x561a5ee11be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.079+0000 7fc01bfff640 1 -- 172.21.15.37:0/3647772018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5ee00e10 msgr2=0x561a5ee0aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.079+0000 7fc01bfff640 1 --2- 172.21.15.37:0/3647772018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5ee00e10 0x561a5ee0aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.079+0000 7fc01bfff640 1 -- 172.21.15.37:0/3647772018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a5ee05f80 msgr2=0x561a5ee11be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.079+0000 7fc01bfff640 1 --2- 172.21.15.37:0/3647772018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a5ee05f80 0x561a5ee11be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.079+0000 7fc01bfff640 1 -- 172.21.15.37:0/3647772018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a5ea73210 con 0x561a5ee00420 2024-06-10T00:59:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.079+0000 7fc020cbf640 1 --2- 172.21.15.37:0/3647772018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a5ee05f80 0x561a5ee11be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:59:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.079+0000 7fc01bfff640 1 --2- 172.21.15.37:0/3647772018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a5ee00420 0x561a5ee0a660 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc00c0b9290 tx=0x7fc00c0bcc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba27d3aeb0d889ba server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.079+0000 7fbffb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:59:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.079+0000 7fbffb7fe640 1 -- 172.21.15.37:0/3647772018 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c049040 con 0x561a5ee00420 2024-06-10T00:59:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.079+0000 7fbffb7fe640 1 -- 172.21.15.37:0/3647772018 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fc00c03b460 con 0x561a5ee00420 2024-06-10T00:59:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.094+0000 7fc025433740 10 client.0 _ll_register_callbacks cb 0x561a5ed219b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:59:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.101+0000 7fc025433740 1 -- 172.21.15.37:0/3647772018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc014011c40 con 0x561a5ee00420 2024-06-10T00:59:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.102+0000 7fbffb7fe640 1 -- 172.21.15.37:0/3647772018 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc00c0bb000 con 0x561a5ee00420 2024-06-10T00:59:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.102+0000 7fc025433740 10 client.15963 fetch_fsmap learned FSMap version 27 2024-06-10T00:59:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.102+0000 7fc025433740 1 -- 172.21.15.37:0/3647772018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=27}) -- 0x561a5eb3f620 con 0x561a5ee00420 2024-06-10T00:59:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.118+0000 7fbffb7fe640 1 -- 172.21.15.37:0/3647772018 <== mon.0 v2:172.21.15.37:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc00c0bd120 con 0x561a5ee00420 2024-06-10T00:59:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.118+0000 7fc025433740 10 client.15963 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:59:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.118+0000 7fc025433740 10 client.15963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T00:59:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.118+0000 7fc025433740 20 client.15963 populate_metadata read hostname 'smithi037' 2024-06-10T00:59:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.118+0000 7fc025433740 1 --2- 172.21.15.37:0/3647772018 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561a5ee16370 0x561a5ee18800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.118+0000 7fc025433740 4 client.15963 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:59:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.118+0000 7fc025433740 1 -- 172.21.15.37:0/3647772018 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a5ed72f10 con 0x561a5ee16370 2024-06-10T00:59:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.118+0000 7fc01b7fe640 1 --2- 172.21.15.37:0/3647772018 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561a5ee16370 0x561a5ee18800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.119+0000 7fc01b7fe640 1 --2- 172.21.15.37:0/3647772018 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561a5ee16370 0x561a5ee18800 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5143315e5fb0fac5 server_cookie=47e5f0527272ffb2 in_seq=0 out_seq=0 2024-06-10T00:59:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.119+0000 7fbffb7fe640 10 client.15963 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:59:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.119+0000 7fbffb7fe640 1 -- 172.21.15.37:0/3647772018 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561a5ed72f10 con 0x561a5ee16370 2024-06-10T00:59:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.119+0000 7fbffb7fe640 10 client.15963 handle_command_reply: tid=0 2024-06-10T00:59:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.120+0000 7fc025433740 1 client.15963 shutdown 2024-06-10T00:59:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.120+0000 7fc025433740 1 -- 172.21.15.37:0/3647772018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a5ee00420 msgr2=0x561a5ee0a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.120+0000 7fc025433740 1 --2- 172.21.15.37:0/3647772018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a5ee00420 0x561a5ee0a660 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc00c0b9290 tx=0x7fc00c0bcc90 comp rx=0 tx=0).stop 2024-06-10T00:59:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.120+0000 7fc025433740 1 -- 172.21.15.37:0/3647772018 shutdown_connections 2024-06-10T00:59:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.120+0000 7fc025433740 1 --2- 172.21.15.37:0/3647772018 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561a5ee16370 0x561a5ee18800 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.120+0000 7fc025433740 1 --2- 172.21.15.37:0/3647772018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a5ee05f80 0x561a5ee11be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.120+0000 7fc025433740 1 --2- 172.21.15.37:0/3647772018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5ee00e10 0x561a5ee0aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.120+0000 7fbffb7fe640 0 client.15963 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:59:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.120+0000 7fc025433740 1 --2- 172.21.15.37:0/3647772018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a5ee00420 0x561a5ee0a660 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.120+0000 7fc025433740 1 -- 172.21.15.37:0/3647772018 >> 172.21.15.37:0/3647772018 conn(0x561a5edf7680 msgr2=0x561a5edf67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.120+0000 7fc025433740 1 -- 172.21.15.37:0/3647772018 shutdown_connections 2024-06-10T00:59:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.120+0000 7fc025433740 1 -- 172.21.15.37:0/3647772018 wait complete. 2024-06-10T00:59:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.135+0000 7fc025433740 20 client.15963 trim_cache size 0 max 16384 2024-06-10T00:59:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.151+0000 7fc025433740 1 Processor -- start 2024-06-10T00:59:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.151+0000 7fc025433740 1 -- start start 2024-06-10T00:59:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.151+0000 7fc025433740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5ee00420 0x561a5edf6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.151+0000 7fc025433740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a5ee00e10 0x561a5ee11b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.151+0000 7fc025433740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a5ee05f80 0x561a5edf6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.151+0000 7fc025433740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1aaca0 con 0x561a5ee00e10 2024-06-10T00:59:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.151+0000 7fc025433740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1a8570 con 0x561a5ee05f80 2024-06-10T00:59:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.151+0000 7fc025433740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1a63c0 con 0x561a5ee00420 2024-06-10T00:59:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.151+0000 7fc01b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a5ee00e10 0x561a5ee11b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.151+0000 7fc01b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a5ee00e10 0x561a5ee11b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40684/0 (socket says 172.21.15.37:40684) 2024-06-10T00:59:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.151+0000 7fc01b7fe640 1 -- 172.21.15.37:0/808403126 learned_addr learned my addr 172.21.15.37:0/808403126 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.151+0000 7fc01bfff640 1 --2- 172.21.15.37:0/808403126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5ee00420 0x561a5edf6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.151+0000 7fc01b7fe640 1 -- 172.21.15.37:0/808403126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5ee00420 msgr2=0x561a5edf6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.151+0000 7fc01b7fe640 1 --2- 172.21.15.37:0/808403126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5ee00420 0x561a5edf6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.151+0000 7fc01b7fe640 1 -- 172.21.15.37:0/808403126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a5ee05f80 msgr2=0x561a5edf6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.151+0000 7fc01b7fe640 1 --2- 172.21.15.37:0/808403126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a5ee05f80 0x561a5edf6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.151+0000 7fc01b7fe640 1 -- 172.21.15.37:0/808403126 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a5ee18e80 con 0x561a5ee00e10 2024-06-10T00:59:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.151+0000 7fc01bfff640 1 --2- 172.21.15.37:0/808403126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5ee00420 0x561a5edf6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:59:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.151+0000 7fc01b7fe640 1 --2- 172.21.15.37:0/808403126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a5ee00e10 0x561a5ee11b70 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc010029fd0 tx=0x7fc010015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c98cd393531872c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.151+0000 7fc001ffb640 1 -- 172.21.15.37:0/808403126 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0100295b0 con 0x561a5ee00e10 2024-06-10T00:59:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.151+0000 7fc001ffb640 1 -- 172.21.15.37:0/808403126 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc01002f070 con 0x561a5ee00e10 2024-06-10T00:59:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.151+0000 7fc001ffb640 1 -- 172.21.15.37:0/808403126 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc010011bc0 con 0x561a5ee00e10 2024-06-10T00:59:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.152+0000 7fc025433740 1 -- 172.21.15.37:0/808403126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a5ee00e10 msgr2=0x561a5ee11b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.152+0000 7fc025433740 1 --2- 172.21.15.37:0/808403126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a5ee00e10 0x561a5ee11b70 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc010029fd0 tx=0x7fc010015000 comp rx=0 tx=0).stop 2024-06-10T00:59:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.152+0000 7fc025433740 1 -- 172.21.15.37:0/808403126 shutdown_connections 2024-06-10T00:59:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.152+0000 7fc025433740 1 --2- 172.21.15.37:0/808403126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a5ee05f80 0x561a5edf6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.152+0000 7fc025433740 1 --2- 172.21.15.37:0/808403126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a5ee00e10 0x561a5ee11b70 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.152+0000 7fc025433740 1 --2- 172.21.15.37:0/808403126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5ee00420 0x561a5edf6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.152+0000 7fc025433740 1 -- 172.21.15.37:0/808403126 >> 172.21.15.37:0/808403126 conn(0x561a5edf7680 msgr2=0x561a5edf5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.152+0000 7fc025433740 1 -- 172.21.15.37:0/808403126 shutdown_connections 2024-06-10T00:59:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.152+0000 7fc025433740 1 -- 172.21.15.37:0/808403126 wait complete. 2024-06-10T00:59:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.153+0000 7fc025433740 1 -- start start 2024-06-10T00:59:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.153+0000 7fc025433740 1 Processor -- start 2024-06-10T00:59:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.153+0000 7fc025433740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a5ee00420 0x561a5ee0f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.153+0000 7fc025433740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5ee00e10 0x561a5ee0f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.153+0000 7fc025433740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a5ee05f80 0x561a5ee10670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.153+0000 7fc025433740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1a6a30 con 0x561a5ee05f80 2024-06-10T00:59:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.153+0000 7fc025433740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1a7ea0 con 0x561a5ee00420 2024-06-10T00:59:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.153+0000 7fc025433740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1a7100 con 0x561a5ee00e10 2024-06-10T00:59:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.153+0000 7fc025433740 10 client.0 _ll_register_callbacks cb 0x7fc00c005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:59:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.153+0000 7fc020cbf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a5ee05f80 0x561a5ee10670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.153+0000 7fc020cbf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a5ee05f80 0x561a5ee10670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40696/0 (socket says 172.21.15.37:40696) 2024-06-10T00:59:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.153+0000 7fc020cbf640 1 -- 172.21.15.37:0/4093592455 learned_addr learned my addr 172.21.15.37:0/4093592455 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.153+0000 7fc01b7fe640 1 --2- 172.21.15.37:0/4093592455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5ee00e10 0x561a5ee0f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.153+0000 7fc01bfff640 1 --2- 172.21.15.37:0/4093592455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a5ee00420 0x561a5ee0f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.153+0000 7fc020cbf640 1 -- 172.21.15.37:0/4093592455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5ee00e10 msgr2=0x561a5ee0f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.153+0000 7fc020cbf640 1 --2- 172.21.15.37:0/4093592455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5ee00e10 0x561a5ee0f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.153+0000 7fc020cbf640 1 -- 172.21.15.37:0/4093592455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a5ee00420 msgr2=0x561a5ee0f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.153+0000 7fc020cbf640 1 --2- 172.21.15.37:0/4093592455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a5ee00420 0x561a5ee0f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.153+0000 7fc020cbf640 1 -- 172.21.15.37:0/4093592455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a5ee01ae0 con 0x561a5ee05f80 2024-06-10T00:59:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.153+0000 7fc01bfff640 1 --2- 172.21.15.37:0/4093592455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a5ee00420 0x561a5ee0f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:59:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.154+0000 7fc020cbf640 1 --2- 172.21.15.37:0/4093592455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a5ee05f80 0x561a5ee10670 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc014009540 tx=0x7fc014009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48476dc571d31c54 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.154+0000 7fbffb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:59:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.154+0000 7fbffb7fe640 1 -- 172.21.15.37:0/4093592455 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc014017050 con 0x561a5ee05f80 2024-06-10T00:59:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.154+0000 7fc025433740 1 -- 172.21.15.37:0/4093592455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc01002f070 con 0x561a5ee05f80 2024-06-10T00:59:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.154+0000 7fbffb7fe640 1 -- 172.21.15.37:0/4093592455 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fc014003b80 con 0x561a5ee05f80 2024-06-10T00:59:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.154+0000 7fbffb7fe640 1 -- 172.21.15.37:0/4093592455 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc014003d80 con 0x561a5ee05f80 2024-06-10T00:59:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.154+0000 7fc025433740 10 client.15969 fetch_fsmap learned FSMap version 27 2024-06-10T00:59:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.154+0000 7fc025433740 1 -- 172.21.15.37:0/4093592455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561a5edf74b0 con 0x561a5ee05f80 2024-06-10T00:59:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.154+0000 7fbffb7fe640 1 -- 172.21.15.37:0/4093592455 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc01002f070 con 0x561a5ee05f80 2024-06-10T00:59:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.154+0000 7fbffb7fe640 1 -- 172.21.15.37:0/4093592455 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc014004610 con 0x561a5ee05f80 2024-06-10T00:59:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.155+0000 7fc025433740 10 client.15969 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:59:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.155+0000 7fc025433740 10 client.15969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T00:59:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.155+0000 7fc025433740 20 client.15969 populate_metadata read hostname 'smithi037' 2024-06-10T00:59:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.155+0000 7fc025433740 1 --2- 172.21.15.37:0/4093592455 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561a5ee812d0 0x561a5ee836c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.155+0000 7fc025433740 4 client.15969 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T00:59:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.155+0000 7fc025433740 1 -- 172.21.15.37:0/4093592455 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561a5ed6cc20 con 0x561a5ee812d0 2024-06-10T00:59:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.155+0000 7fc01bfff640 1 --2- 172.21.15.37:0/4093592455 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561a5ee812d0 0x561a5ee836c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.155+0000 7fc01bfff640 1 --2- 172.21.15.37:0/4093592455 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561a5ee812d0 0x561a5ee836c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9cb5d3b4581b4d server_cookie=221c9db380f916ad in_seq=0 out_seq=0 2024-06-10T00:59:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.155+0000 7fbffb7fe640 10 client.15969 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:59:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.167+0000 7fbffb7fe640 1 -- 172.21.15.37:0/4093592455 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561a5ed6cc20 con 0x561a5ee812d0 2024-06-10T00:59:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.167+0000 7fbffb7fe640 10 client.15969 handle_command_reply: tid=0 2024-06-10T00:59:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.167+0000 7fc025433740 1 client.15969 shutdown 2024-06-10T00:59:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.167+0000 7fc025433740 1 -- 172.21.15.37:0/4093592455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a5ee05f80 msgr2=0x561a5ee10670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.167+0000 7fc025433740 1 --2- 172.21.15.37:0/4093592455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a5ee05f80 0x561a5ee10670 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc014009540 tx=0x7fc014009fb0 comp rx=0 tx=0).stop 2024-06-10T00:59:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.167+0000 7fc025433740 1 -- 172.21.15.37:0/4093592455 shutdown_connections 2024-06-10T00:59:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.167+0000 7fc025433740 1 --2- 172.21.15.37:0/4093592455 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561a5ee812d0 0x561a5ee836c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.167+0000 7fc025433740 1 --2- 172.21.15.37:0/4093592455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a5ee05f80 0x561a5ee10670 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.167+0000 7fc025433740 1 --2- 172.21.15.37:0/4093592455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5ee00e10 0x561a5ee0f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.167+0000 7fbffb7fe640 0 client.15969 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:59:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.167+0000 7fc025433740 1 --2- 172.21.15.37:0/4093592455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a5ee00420 0x561a5ee0f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.167+0000 7fc025433740 1 -- 172.21.15.37:0/4093592455 >> 172.21.15.37:0/4093592455 conn(0x561a5edf7680 msgr2=0x561a5edf67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.167+0000 7fc025433740 1 -- 172.21.15.37:0/4093592455 shutdown_connections 2024-06-10T00:59:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.167+0000 7fc025433740 1 -- 172.21.15.37:0/4093592455 wait complete. 2024-06-10T00:59:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.167+0000 7fc025433740 20 client.15969 trim_cache size 0 max 16384 2024-06-10T00:59:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T00:59:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T00:59:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T00:59:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.168+0000 7fc022722640 1 -- 172.21.15.37:0/4218497405 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc0040a2da0 msgr2=0x7fc0040a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.168+0000 7fc022722640 1 --2- 172.21.15.37:0/4218497405 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc0040a2da0 0x7fc0040a5260 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc010005fd0 tx=0x7fc010005850 comp rx=0 tx=0).stop 2024-06-10T00:59:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.168+0000 7fc022722640 1 -- 172.21.15.37:0/4218497405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc01c100070 msgr2=0x7fc01c197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.168+0000 7fc022722640 1 --2- 172.21.15.37:0/4218497405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc01c100070 0x7fc01c197d40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc00c031440 tx=0x7fc00c032ce0 comp rx=0 tx=0).stop 2024-06-10T00:59:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.168+0000 7fc022722640 1 -- 172.21.15.37:0/4218497405 shutdown_connections 2024-06-10T00:59:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.168+0000 7fc022722640 1 --2- 172.21.15.37:0/4218497405 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc0040a2da0 0x7fc0040a5260 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.168+0000 7fc022722640 1 --2- 172.21.15.37:0/4218497405 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc01c101c20 0x7fc01c19f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.168+0000 7fc022722640 1 --2- 172.21.15.37:0/4218497405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc01c101260 0x7fc01c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.168+0000 7fc022722640 1 --2- 172.21.15.37:0/4218497405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc01c100070 0x7fc01c197d40 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.168+0000 7fc022722640 1 -- 172.21.15.37:0/4218497405 >> 172.21.15.37:0/4218497405 conn(0x7fc01c0fb600 msgr2=0x7fc01c0fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.168+0000 7fc022722640 1 -- 172.21.15.37:0/4218497405 shutdown_connections 2024-06-10T00:59:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T00:59:51.168+0000 7fc022722640 1 -- 172.21.15.37:0/4218497405 wait complete. 2024-06-10T00:59:51.452 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T00:59:52.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:52 smithi167 ceph-mon[36917]: pgmap v525: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.8 KiB/s wr, 7 op/s 2024-06-10T00:59:52.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:52 smithi094 ceph-mon[36708]: pgmap v525: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.8 KiB/s wr, 7 op/s 2024-06-10T00:59:52.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:52 smithi037 ceph-mon[32192]: pgmap v525: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.8 KiB/s wr, 7 op/s 2024-06-10T00:59:52.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.585+0000 7f1134882700 1 -- 172.21.15.37:0/2937268033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11301066b0 msgr2=0x7f113010e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:52.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.585+0000 7f1134882700 1 --2- 172.21.15.37:0/2937268033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11301066b0 0x7f113010e280 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f112800b0d0 tx=0x7f112800b3e0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.585+0000 7f1134882700 1 -- 172.21.15.37:0/2937268033 shutdown_connections 2024-06-10T00:59:52.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.585+0000 7f1134882700 1 --2- 172.21.15.37:0/2937268033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11301066b0 0x7f113010e280 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.585+0000 7f1134882700 1 --2- 172.21.15.37:0/2937268033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1130101620 0x7f1130101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.585+0000 7f1134882700 1 --2- 172.21.15.37:0/2937268033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1130101fd0 0x7f1130105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.585+0000 7f1134882700 1 -- 172.21.15.37:0/2937268033 >> 172.21.15.37:0/2937268033 conn(0x7f1130078340 msgr2=0x7f1130078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:52.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.585+0000 7f1134882700 1 -- 172.21.15.37:0/2937268033 shutdown_connections 2024-06-10T00:59:52.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.586+0000 7f1134882700 1 -- 172.21.15.37:0/2937268033 wait complete. 2024-06-10T00:59:52.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.586+0000 7f1134882700 1 Processor -- start 2024-06-10T00:59:52.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.586+0000 7f1134882700 1 -- start start 2024-06-10T00:59:52.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.587+0000 7f1134882700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1130101620 0x7f113019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:52.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.587+0000 7f1134882700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1130101fd0 0x7f113019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:52.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.587+0000 7f1134882700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11301066b0 0x7f1130198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:52.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.587+0000 7f1134882700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11301116a0 con 0x7f1130101fd0 2024-06-10T00:59:52.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.587+0000 7f1134882700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1130111520 con 0x7f1130101620 2024-06-10T00:59:52.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.587+0000 7f1134882700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1130111820 con 0x7f11301066b0 2024-06-10T00:59:52.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.587+0000 7f112dd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1130101fd0 0x7f113019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:52.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.587+0000 7f112dd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1130101fd0 0x7f113019ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40592/0 (socket says 172.21.15.37:40592) 2024-06-10T00:59:52.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.587+0000 7f112dd9b700 1 -- 172.21.15.37:0/341233521 learned_addr learned my addr 172.21.15.37:0/341233521 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:52.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.587+0000 7f112ed9d700 1 --2- 172.21.15.37:0/341233521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11301066b0 0x7f1130198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:52.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.587+0000 7f112e59c700 1 --2- 172.21.15.37:0/341233521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1130101620 0x7f113019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:52.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.587+0000 7f112dd9b700 1 -- 172.21.15.37:0/341233521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11301066b0 msgr2=0x7f1130198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:52.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.587+0000 7f112dd9b700 1 --2- 172.21.15.37:0/341233521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11301066b0 0x7f1130198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.587+0000 7f112dd9b700 1 -- 172.21.15.37:0/341233521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1130101620 msgr2=0x7f113019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:52.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.587+0000 7f112dd9b700 1 --2- 172.21.15.37:0/341233521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1130101620 0x7f113019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.587+0000 7f112dd9b700 1 -- 172.21.15.37:0/341233521 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1130199240 con 0x7f1130101fd0 2024-06-10T00:59:52.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.587+0000 7f112e59c700 1 --2- 172.21.15.37:0/341233521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1130101620 0x7f113019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:59:52.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.588+0000 7f112dd9b700 1 --2- 172.21.15.37:0/341233521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1130101fd0 0x7f113019ee60 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f112400d980 tx=0x7f112400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:52.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.588+0000 7f111f7fe700 1 -- 172.21.15.37:0/341233521 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1124017070 con 0x7f1130101fd0 2024-06-10T00:59:52.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.588+0000 7f111f7fe700 1 -- 172.21.15.37:0/341233521 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1124009e10 con 0x7f1130101fd0 2024-06-10T00:59:52.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.588+0000 7f1134882700 1 -- 172.21.15.37:0/341233521 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1130199530 con 0x7f1130101fd0 2024-06-10T00:59:52.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.588+0000 7f111f7fe700 1 -- 172.21.15.37:0/341233521 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112400f620 con 0x7f1130101fd0 2024-06-10T00:59:52.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.588+0000 7f1134882700 1 -- 172.21.15.37:0/341233521 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11301a57f0 con 0x7f1130101fd0 2024-06-10T00:59:52.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.590+0000 7f111f7fe700 1 -- 172.21.15.37:0/341233521 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1124010460 con 0x7f1130101fd0 2024-06-10T00:59:52.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.591+0000 7f111f7fe700 1 --2- 172.21.15.37:0/341233521 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f11180a2ca0 0x7f11180a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:52.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.591+0000 7f112e59c700 1 --2- 172.21.15.37:0/341233521 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f11180a2ca0 0x7f11180a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:52.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.591+0000 7f111f7fe700 1 -- 172.21.15.37:0/341233521 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f11240ceec0 con 0x7f1130101fd0 2024-06-10T00:59:52.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.591+0000 7f112e59c700 1 --2- 172.21.15.37:0/341233521 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f11180a2ca0 0x7f11180a5160 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1120005950 tx=0x7f11200058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:52.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.593+0000 7f113f105b80 1 Processor -- start 2024-06-10T00:59:52.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.593+0000 7f113f105b80 1 -- start start 2024-06-10T00:59:52.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.593+0000 7f113f105b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559f4b64470 0x5559f4b64850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:52.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.593+0000 7f113f105b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559f4b5e670 0x5559f4b5eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:52.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.593+0000 7f113f105b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559f4b5f030 0x5559f4b6b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:52.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.593+0000 7f113f105b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5559f4a4cf60 con 0x5559f4b5e670 2024-06-10T00:59:52.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.593+0000 7f113f105b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5559f4aab850 con 0x5559f4b64470 2024-06-10T00:59:52.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.593+0000 7f113f105b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5559f4a13460 con 0x5559f4b5f030 2024-06-10T00:59:52.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.594+0000 7f112dd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559f4b5e670 0x5559f4b5eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:52.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.594+0000 7f112e59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559f4b64470 0x5559f4b64850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:52.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.594+0000 7f112dd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559f4b5e670 0x5559f4b5eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40608/0 (socket says 172.21.15.37:40608) 2024-06-10T00:59:52.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.594+0000 7f112e59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559f4b64470 0x5559f4b64850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:33546/0 (socket says 172.21.15.37:33546) 2024-06-10T00:59:52.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.594+0000 7f112dd9b700 1 -- 172.21.15.37:0/1237024923 learned_addr learned my addr 172.21.15.37:0/1237024923 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:52.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.594+0000 7f112ed9d700 1 --2- 172.21.15.37:0/1237024923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559f4b5f030 0x5559f4b6b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:52.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.594+0000 7f112dd9b700 1 -- 172.21.15.37:0/1237024923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559f4b5f030 msgr2=0x5559f4b6b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:52.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.594+0000 7f112dd9b700 1 --2- 172.21.15.37:0/1237024923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559f4b5f030 0x5559f4b6b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.594+0000 7f112dd9b700 1 -- 172.21.15.37:0/1237024923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559f4b64470 msgr2=0x5559f4b64850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:52.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.594+0000 7f112dd9b700 1 --2- 172.21.15.37:0/1237024923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559f4b64470 0x5559f4b64850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.594+0000 7f112dd9b700 1 -- 172.21.15.37:0/1237024923 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559f49fa8d0 con 0x5559f4b5e670 2024-06-10T00:59:52.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.594+0000 7f112dd9b700 1 --2- 172.21.15.37:0/1237024923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559f4b5e670 0x5559f4b5eaf0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1124097790 tx=0x7f1124097700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae2744746f6a0cf server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:52.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.594+0000 7f111cff9700 1 -- 172.21.15.37:0/1237024923 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1124020040 con 0x5559f4b5e670 2024-06-10T00:59:52.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.594+0000 7f111cff9700 1 -- 172.21.15.37:0/1237024923 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f112409b330 con 0x5559f4b5e670 2024-06-10T00:59:52.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.594+0000 7f111cff9700 1 -- 172.21.15.37:0/1237024923 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11240a3d70 con 0x5559f4b5e670 2024-06-10T00:59:52.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.595+0000 7f113f105b80 1 -- 172.21.15.37:0/1237024923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559f4b5e670 msgr2=0x5559f4b5eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:52.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.595+0000 7f113f105b80 1 --2- 172.21.15.37:0/1237024923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559f4b5e670 0x5559f4b5eaf0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1124097790 tx=0x7f1124097700 comp rx=0 tx=0).stop 2024-06-10T00:59:52.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.595+0000 7f113f105b80 1 -- 172.21.15.37:0/1237024923 shutdown_connections 2024-06-10T00:59:52.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.595+0000 7f113f105b80 1 --2- 172.21.15.37:0/1237024923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559f4b5f030 0x5559f4b6b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.595+0000 7f113f105b80 1 --2- 172.21.15.37:0/1237024923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559f4b64470 0x5559f4b64850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.595+0000 7f113f105b80 1 --2- 172.21.15.37:0/1237024923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559f4b5e670 0x5559f4b5eaf0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.595+0000 7f113f105b80 1 -- 172.21.15.37:0/1237024923 >> 172.21.15.37:0/1237024923 conn(0x5559f4a87080 msgr2=0x5559f4b55220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:52.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.595+0000 7f113f105b80 1 -- 172.21.15.37:0/1237024923 shutdown_connections 2024-06-10T00:59:52.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.595+0000 7f113f105b80 1 -- 172.21.15.37:0/1237024923 wait complete. 2024-06-10T00:59:52.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.596+0000 7f113f105b80 1 -- start start 2024-06-10T00:59:52.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.596+0000 7f113f105b80 1 Processor -- start 2024-06-10T00:59:52.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.596+0000 7f113f105b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559f4b5e670 0x5559f4b6af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:52.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.596+0000 7f113f105b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559f4b5f030 0x5559f4b68270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:52.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.596+0000 7f113f105b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559f4b64470 0x5559f4b687b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:52.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.596+0000 7f113f105b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5559f4a4bfc0 con 0x5559f4b5f030 2024-06-10T00:59:52.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.596+0000 7f113f105b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5559f4aa7810 con 0x5559f4b64470 2024-06-10T00:59:52.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.596+0000 7f113f105b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5559f4b6b750 con 0x5559f4b5e670 2024-06-10T00:59:52.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.596+0000 7f113f105b80 10 client.0 _ll_register_callbacks cb 0x5559f4a415b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:59:52.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.596+0000 7f112dd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559f4b5f030 0x5559f4b68270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:52.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.596+0000 7f112dd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559f4b5f030 0x5559f4b68270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40626/0 (socket says 172.21.15.37:40626) 2024-06-10T00:59:52.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.596+0000 7f112dd9b700 1 -- 172.21.15.37:0/1656089734 learned_addr learned my addr 172.21.15.37:0/1656089734 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:52.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.596+0000 7f112e59c700 1 --2- 172.21.15.37:0/1656089734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559f4b5e670 0x5559f4b6af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:52.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.596+0000 7f112ed9d700 1 --2- 172.21.15.37:0/1656089734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559f4b64470 0x5559f4b687b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:52.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.597+0000 7f112dd9b700 1 -- 172.21.15.37:0/1656089734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559f4b5e670 msgr2=0x5559f4b6af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:52.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.597+0000 7f112dd9b700 1 --2- 172.21.15.37:0/1656089734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559f4b5e670 0x5559f4b6af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.597+0000 7f112dd9b700 1 -- 172.21.15.37:0/1656089734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559f4b64470 msgr2=0x5559f4b687b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:52.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.597+0000 7f112dd9b700 1 --2- 172.21.15.37:0/1656089734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559f4b64470 0x5559f4b687b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.597+0000 7f112dd9b700 1 -- 172.21.15.37:0/1656089734 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559f4a71750 con 0x5559f4b5f030 2024-06-10T00:59:52.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.597+0000 7f112ed9d700 1 --2- 172.21.15.37:0/1656089734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559f4b64470 0x5559f4b687b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:59:52.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.597+0000 7f112dd9b700 1 --2- 172.21.15.37:0/1656089734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559f4b5f030 0x5559f4b68270 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f11240cc940 tx=0x7f112409baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb03fd9419ed1773 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:52.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.597+0000 7f1115ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:59:52.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.597+0000 7f1115ffb700 1 -- 172.21.15.37:0/1656089734 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11240b3070 con 0x5559f4b5f030 2024-06-10T00:59:52.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.597+0000 7f1115ffb700 1 -- 172.21.15.37:0/1656089734 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f1124010780 con 0x5559f4b5f030 2024-06-10T00:59:52.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.597+0000 7f113f105b80 1 -- 172.21.15.37:0/1656089734 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f112409b330 con 0x5559f4b5f030 2024-06-10T00:59:52.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.597+0000 7f1115ffb700 1 -- 172.21.15.37:0/1656089734 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1124020070 con 0x5559f4b5f030 2024-06-10T00:59:52.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.597+0000 7f113f105b80 10 client.15981 fetch_fsmap learned FSMap version 27 2024-06-10T00:59:52.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.597+0000 7f113f105b80 1 -- 172.21.15.37:0/1656089734 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5559f4b75d80 con 0x5559f4b5f030 2024-06-10T00:59:52.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.597+0000 7f1115ffb700 1 -- 172.21.15.37:0/1656089734 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f112409b330 con 0x5559f4b5f030 2024-06-10T00:59:52.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.598+0000 7f1115ffb700 1 -- 172.21.15.37:0/1656089734 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f112409f160 con 0x5559f4b5f030 2024-06-10T00:59:52.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.598+0000 7f113f105b80 10 client.15981 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:59:52.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.598+0000 7f113f105b80 10 client.15981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T00:59:52.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.598+0000 7f113f105b80 20 client.15981 populate_metadata read hostname 'smithi037' 2024-06-10T00:59:52.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.598+0000 7f113f105b80 1 --2- 172.21.15.37:0/1656089734 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5559f4b78250 0x5559f4b7a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:52.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.598+0000 7f113f105b80 4 client.15981 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:59:52.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.598+0000 7f113f105b80 1 -- 172.21.15.37:0/1656089734 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559f4ab8d50 con 0x5559f4b78250 2024-06-10T00:59:52.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.598+0000 7f112e59c700 1 --2- 172.21.15.37:0/1656089734 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5559f4b78250 0x5559f4b7a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:52.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.598+0000 7f112e59c700 1 --2- 172.21.15.37:0/1656089734 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5559f4b78250 0x5559f4b7a640 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f800f8362f58af2 server_cookie=bafd10c78cbfbd29 in_seq=0 out_seq=0 2024-06-10T00:59:52.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.598+0000 7f1115ffb700 10 client.15981 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:59:52.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.599+0000 7f1115ffb700 1 -- 172.21.15.37:0/1656089734 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5559f4ab8d50 con 0x5559f4b78250 2024-06-10T00:59:52.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.599+0000 7f1115ffb700 10 client.15981 handle_command_reply: tid=0 2024-06-10T00:59:52.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.599+0000 7f113f105b80 1 client.15981 shutdown 2024-06-10T00:59:52.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.599+0000 7f113f105b80 1 -- 172.21.15.37:0/1656089734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559f4b5f030 msgr2=0x5559f4b68270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:52.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.599+0000 7f113f105b80 1 --2- 172.21.15.37:0/1656089734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559f4b5f030 0x5559f4b68270 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f11240cc940 tx=0x7f112409baf0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.599+0000 7f113f105b80 1 -- 172.21.15.37:0/1656089734 shutdown_connections 2024-06-10T00:59:52.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.599+0000 7f113f105b80 1 --2- 172.21.15.37:0/1656089734 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5559f4b78250 0x5559f4b7a640 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.599+0000 7f113f105b80 1 --2- 172.21.15.37:0/1656089734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559f4b64470 0x5559f4b687b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.599+0000 7f113f105b80 1 --2- 172.21.15.37:0/1656089734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559f4b5e670 0x5559f4b6af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.599+0000 7f113f105b80 1 --2- 172.21.15.37:0/1656089734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559f4b5f030 0x5559f4b68270 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.599+0000 7f1115ffb700 0 client.15981 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:59:52.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.599+0000 7f113f105b80 1 -- 172.21.15.37:0/1656089734 >> 172.21.15.37:0/1656089734 conn(0x5559f4a87080 msgr2=0x5559f4b54700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:52.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.600+0000 7f113f105b80 1 -- 172.21.15.37:0/1656089734 shutdown_connections 2024-06-10T00:59:52.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.600+0000 7f113f105b80 1 -- 172.21.15.37:0/1656089734 wait complete. 2024-06-10T00:59:52.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.600+0000 7f113f105b80 20 client.15981 trim_cache size 0 max 16384 2024-06-10T00:59:52.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.615+0000 7f113f105b80 1 Processor -- start 2024-06-10T00:59:52.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.615+0000 7f113f105b80 1 -- start start 2024-06-10T00:59:52.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.615+0000 7f113f105b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559f4b5e670 0x5559f4b61e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:52.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.615+0000 7f113f105b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559f4b5f030 0x5559f4b623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:52.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.615+0000 7f113f105b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559f4b64470 0x5559f4b628e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:52.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.615+0000 7f113f105b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11301aeb80 con 0x5559f4b64470 2024-06-10T00:59:52.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.615+0000 7f113f105b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f11301ac410 con 0x5559f4b5f030 2024-06-10T00:59:52.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.615+0000 7f113f105b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f11301aa260 con 0x5559f4b5e670 2024-06-10T00:59:52.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.615+0000 7f112ed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559f4b64470 0x5559f4b628e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:52.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.615+0000 7f112ed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559f4b64470 0x5559f4b628e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40636/0 (socket says 172.21.15.37:40636) 2024-06-10T00:59:52.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.615+0000 7f112ed9d700 1 -- 172.21.15.37:0/3766084208 learned_addr learned my addr 172.21.15.37:0/3766084208 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:52.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.615+0000 7f112e59c700 1 --2- 172.21.15.37:0/3766084208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559f4b5e670 0x5559f4b61e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:52.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.615+0000 7f112dd9b700 1 --2- 172.21.15.37:0/3766084208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559f4b5f030 0x5559f4b623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:52.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.616+0000 7f112ed9d700 1 -- 172.21.15.37:0/3766084208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559f4b5e670 msgr2=0x5559f4b61e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:52.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.616+0000 7f112ed9d700 1 --2- 172.21.15.37:0/3766084208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559f4b5e670 0x5559f4b61e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.616+0000 7f112ed9d700 1 -- 172.21.15.37:0/3766084208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559f4b5f030 msgr2=0x5559f4b623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:52.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.616+0000 7f112ed9d700 1 --2- 172.21.15.37:0/3766084208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559f4b5f030 0x5559f4b623a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.616+0000 7f112ed9d700 1 -- 172.21.15.37:0/3766084208 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559f4b68bb0 con 0x5559f4b64470 2024-06-10T00:59:52.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.616+0000 7f112ed9d700 1 --2- 172.21.15.37:0/3766084208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559f4b64470 0x5559f4b628e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1128015000 tx=0x7f1128006210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82777116d5a303d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:52.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.616+0000 7f1117fff700 1 -- 172.21.15.37:0/3766084208 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1128003a00 con 0x5559f4b64470 2024-06-10T00:59:52.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.616+0000 7f1117fff700 1 -- 172.21.15.37:0/3766084208 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1128021070 con 0x5559f4b64470 2024-06-10T00:59:52.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.616+0000 7f1117fff700 1 -- 172.21.15.37:0/3766084208 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11280266d0 con 0x5559f4b64470 2024-06-10T00:59:52.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.617+0000 7f113f105b80 1 -- 172.21.15.37:0/3766084208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559f4b64470 msgr2=0x5559f4b628e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:52.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.617+0000 7f113f105b80 1 --2- 172.21.15.37:0/3766084208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559f4b64470 0x5559f4b628e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1128015000 tx=0x7f1128006210 comp rx=0 tx=0).stop 2024-06-10T00:59:52.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.617+0000 7f113f105b80 1 -- 172.21.15.37:0/3766084208 shutdown_connections 2024-06-10T00:59:52.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.617+0000 7f113f105b80 1 --2- 172.21.15.37:0/3766084208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559f4b64470 0x5559f4b628e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.617+0000 7f113f105b80 1 --2- 172.21.15.37:0/3766084208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559f4b5e670 0x5559f4b61e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.617+0000 7f113f105b80 1 --2- 172.21.15.37:0/3766084208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559f4b5f030 0x5559f4b623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.617+0000 7f113f105b80 1 -- 172.21.15.37:0/3766084208 >> 172.21.15.37:0/3766084208 conn(0x5559f4a87080 msgr2=0x5559f4b59ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:52.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.617+0000 7f113f105b80 1 -- 172.21.15.37:0/3766084208 shutdown_connections 2024-06-10T00:59:52.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.617+0000 7f113f105b80 1 -- 172.21.15.37:0/3766084208 wait complete. 2024-06-10T00:59:52.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.617+0000 7f113f105b80 1 -- start start 2024-06-10T00:59:52.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.617+0000 7f113f105b80 1 Processor -- start 2024-06-10T00:59:52.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.618+0000 7f113f105b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559f4b5e670 0x5559f4b7a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:52.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.618+0000 7f113f105b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559f4b5f030 0x5559f4b7ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:52.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.618+0000 7f113f105b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559f4b64470 0x5559f4b7b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:52.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.618+0000 7f113f105b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11301aa8d0 con 0x5559f4b5e670 2024-06-10T00:59:52.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.618+0000 7f113f105b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f11301abd40 con 0x5559f4b64470 2024-06-10T00:59:52.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.618+0000 7f113f105b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f11301aafa0 con 0x5559f4b5f030 2024-06-10T00:59:52.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.618+0000 7f113f105b80 10 client.0 _ll_register_callbacks cb 0x5559f4a415b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:59:52.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.618+0000 7f112e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559f4b5e670 0x5559f4b7a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:52.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.618+0000 7f112e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559f4b5e670 0x5559f4b7a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40638/0 (socket says 172.21.15.37:40638) 2024-06-10T00:59:52.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.618+0000 7f112e59c700 1 -- 172.21.15.37:0/236715576 learned_addr learned my addr 172.21.15.37:0/236715576 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:52.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.618+0000 7f112dd9b700 1 --2- 172.21.15.37:0/236715576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559f4b5f030 0x5559f4b7ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:52.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.618+0000 7f112ed9d700 1 --2- 172.21.15.37:0/236715576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559f4b64470 0x5559f4b7b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:52.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.618+0000 7f112e59c700 1 -- 172.21.15.37:0/236715576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559f4b5f030 msgr2=0x5559f4b7ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:52.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.618+0000 7f112e59c700 1 --2- 172.21.15.37:0/236715576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559f4b5f030 0x5559f4b7ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.618+0000 7f112e59c700 1 -- 172.21.15.37:0/236715576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559f4b64470 msgr2=0x5559f4b7b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:52.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.618+0000 7f112e59c700 1 --2- 172.21.15.37:0/236715576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559f4b64470 0x5559f4b7b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.618+0000 7f112e59c700 1 -- 172.21.15.37:0/236715576 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559f4be0c80 con 0x5559f4b5e670 2024-06-10T00:59:52.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.618+0000 7f112e59c700 1 --2- 172.21.15.37:0/236715576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559f4b5e670 0x5559f4b7a640 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x5559f4aa78c0 tx=0x7f11200047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a45922811e80a58 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.618+0000 7f1115ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:59:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.618+0000 7f1115ffb700 1 -- 172.21.15.37:0/236715576 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1120003880 con 0x5559f4b5e670 2024-06-10T00:59:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.618+0000 7f113f105b80 1 -- 172.21.15.37:0/236715576 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1128021070 con 0x5559f4b5e670 2024-06-10T00:59:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.618+0000 7f1115ffb700 1 -- 172.21.15.37:0/236715576 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f1120003a20 con 0x5559f4b5e670 2024-06-10T00:59:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.619+0000 7f1115ffb700 1 -- 172.21.15.37:0/236715576 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f112003a050 con 0x5559f4b5e670 2024-06-10T00:59:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.619+0000 7f113f105b80 10 client.15987 fetch_fsmap learned FSMap version 27 2024-06-10T00:59:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.619+0000 7f113f105b80 1 -- 172.21.15.37:0/236715576 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5559f4be1fc0 con 0x5559f4b5e670 2024-06-10T00:59:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.619+0000 7f1115ffb700 1 -- 172.21.15.37:0/236715576 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1128021070 con 0x5559f4b5e670 2024-06-10T00:59:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.619+0000 7f1115ffb700 1 -- 172.21.15.37:0/236715576 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1120032420 con 0x5559f4b5e670 2024-06-10T00:59:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.619+0000 7f113f105b80 10 client.15987 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:59:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.619+0000 7f113f105b80 10 client.15987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T00:59:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.619+0000 7f113f105b80 20 client.15987 populate_metadata read hostname 'smithi037' 2024-06-10T00:59:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.619+0000 7f113f105b80 1 --2- 172.21.15.37:0/236715576 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5559f4be2700 0x5559f4be4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:52.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.619+0000 7f113f105b80 4 client.15987 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T00:59:52.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.619+0000 7f113f105b80 1 -- 172.21.15.37:0/236715576 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5559f4ab32d0 con 0x5559f4be2700 2024-06-10T00:59:52.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.619+0000 7f112dd9b700 1 --2- 172.21.15.37:0/236715576 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5559f4be2700 0x5559f4be4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:52.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.620+0000 7f112dd9b700 1 --2- 172.21.15.37:0/236715576 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5559f4be2700 0x5559f4be4b20 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f65db6cb2ba965e server_cookie=841691b3bb174b6b in_seq=0 out_seq=0 2024-06-10T00:59:52.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.620+0000 7f1115ffb700 10 client.15987 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:59:52.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.620+0000 7f1115ffb700 1 -- 172.21.15.37:0/236715576 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5559f4ab32d0 con 0x5559f4be2700 2024-06-10T00:59:52.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.620+0000 7f1115ffb700 10 client.15987 handle_command_reply: tid=0 2024-06-10T00:59:52.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.620+0000 7f113f105b80 1 client.15987 shutdown 2024-06-10T00:59:52.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.620+0000 7f113f105b80 1 -- 172.21.15.37:0/236715576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559f4b5e670 msgr2=0x5559f4b7a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:52.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.620+0000 7f113f105b80 1 --2- 172.21.15.37:0/236715576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559f4b5e670 0x5559f4b7a640 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x5559f4aa78c0 tx=0x7f11200047c0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.620+0000 7f113f105b80 1 -- 172.21.15.37:0/236715576 shutdown_connections 2024-06-10T00:59:52.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.620+0000 7f113f105b80 1 --2- 172.21.15.37:0/236715576 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5559f4be2700 0x5559f4be4b20 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.620+0000 7f113f105b80 1 --2- 172.21.15.37:0/236715576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559f4b64470 0x5559f4b7b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.620+0000 7f113f105b80 1 --2- 172.21.15.37:0/236715576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5559f4b5e670 0x5559f4b7a640 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.620+0000 7f113f105b80 1 --2- 172.21.15.37:0/236715576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5559f4b5f030 0x5559f4b7ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.620+0000 7f113f105b80 1 -- 172.21.15.37:0/236715576 >> 172.21.15.37:0/236715576 conn(0x5559f4a87080 msgr2=0x5559f4b69a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:52.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.620+0000 7f1115ffb700 0 client.15987 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:59:52.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.620+0000 7f113f105b80 1 -- 172.21.15.37:0/236715576 shutdown_connections 2024-06-10T00:59:52.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.620+0000 7f113f105b80 1 -- 172.21.15.37:0/236715576 wait complete. 2024-06-10T00:59:52.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.621+0000 7f113f105b80 20 client.15987 trim_cache size 0 max 16384 2024-06-10T00:59:52.625 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T00:59:52.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.621+0000 7f1134882700 1 -- 172.21.15.37:0/341233521 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f11180a2ca0 msgr2=0x7f11180a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:52.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.621+0000 7f1134882700 1 --2- 172.21.15.37:0/341233521 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f11180a2ca0 0x7f11180a5160 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1120005950 tx=0x7f11200058e0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.621+0000 7f1134882700 1 -- 172.21.15.37:0/341233521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1130101fd0 msgr2=0x7f113019ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:52.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.621+0000 7f1134882700 1 --2- 172.21.15.37:0/341233521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1130101fd0 0x7f113019ee60 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f112400d980 tx=0x7f112400dc90 comp rx=0 tx=0).stop 2024-06-10T00:59:52.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.621+0000 7f1134882700 1 -- 172.21.15.37:0/341233521 shutdown_connections 2024-06-10T00:59:52.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.621+0000 7f1134882700 1 --2- 172.21.15.37:0/341233521 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f11180a2ca0 0x7f11180a5160 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.621+0000 7f1134882700 1 --2- 172.21.15.37:0/341233521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11301066b0 0x7f1130198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.621+0000 7f1134882700 1 --2- 172.21.15.37:0/341233521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1130101620 0x7f113019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.621+0000 7f1134882700 1 --2- 172.21.15.37:0/341233521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1130101fd0 0x7f113019ee60 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:52.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.621+0000 7f1134882700 1 -- 172.21.15.37:0/341233521 >> 172.21.15.37:0/341233521 conn(0x7f1130078340 msgr2=0x7f113006aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:52.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.621+0000 7f1134882700 1 -- 172.21.15.37:0/341233521 shutdown_connections 2024-06-10T00:59:52.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:52.621+0000 7f1134882700 1 -- 172.21.15.37:0/341233521 wait complete. 2024-06-10T00:59:54.144 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:54 smithi037 ceph-mon[32192]: pgmap v526: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-06-10T00:59:54.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:54 smithi167 ceph-mon[36917]: pgmap v526: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-06-10T00:59:54.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:54 smithi094 ceph-mon[36708]: pgmap v526: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-06-10T00:59:54.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.442+0000 7f4a7303e700 1 -- 172.21.15.37:0/2332041187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a6c102770 msgr2=0x7f4a6c102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:54.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.442+0000 7f4a7303e700 1 --2- 172.21.15.37:0/2332041187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a6c102770 0x7f4a6c102b50 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4a600099c0 tx=0x7f4a60009cd0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.442+0000 7f4a7303e700 1 -- 172.21.15.37:0/2332041187 shutdown_connections 2024-06-10T00:59:54.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.442+0000 7f4a7303e700 1 --2- 172.21.15.37:0/2332041187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a6c107860 0x7f4a6c10f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.442+0000 7f4a7303e700 1 --2- 172.21.15.37:0/2332041187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a6c102770 0x7f4a6c102b50 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.442+0000 7f4a7303e700 1 --2- 172.21.15.37:0/2332041187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a6c103120 0x7f4a6c107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.442+0000 7f4a7303e700 1 -- 172.21.15.37:0/2332041187 >> 172.21.15.37:0/2332041187 conn(0x7f4a6c079600 msgr2=0x7f4a6c079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:54.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.442+0000 7f4a7303e700 1 -- 172.21.15.37:0/2332041187 shutdown_connections 2024-06-10T00:59:54.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.442+0000 7f4a7303e700 1 -- 172.21.15.37:0/2332041187 wait complete. 2024-06-10T00:59:54.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.442+0000 7f4a7303e700 1 Processor -- start 2024-06-10T00:59:54.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.443+0000 7f4a7303e700 1 -- start start 2024-06-10T00:59:54.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.443+0000 7f4a7303e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a6c102770 0x7f4a6c19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:54.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.443+0000 7f4a7303e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a6c103120 0x7f4a6c19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:54.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.443+0000 7f4a7303e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a6c107860 0x7f4a6c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:54.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.443+0000 7f4a7303e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6c112800 con 0x7f4a6c103120 2024-06-10T00:59:54.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.443+0000 7f4a7303e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6c112680 con 0x7f4a6c102770 2024-06-10T00:59:54.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.443+0000 7f4a7303e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6c112980 con 0x7f4a6c107860 2024-06-10T00:59:54.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.443+0000 7f4a6bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a6c103120 0x7f4a6c19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:54.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.443+0000 7f4a6bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a6c103120 0x7f4a6c19ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40660/0 (socket says 172.21.15.37:40660) 2024-06-10T00:59:54.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.443+0000 7f4a6bfff700 1 -- 172.21.15.37:0/4121385156 learned_addr learned my addr 172.21.15.37:0/4121385156 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:54.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.443+0000 7f4a715db700 1 --2- 172.21.15.37:0/4121385156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a6c107860 0x7f4a6c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:54.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.443+0000 7f4a70dda700 1 --2- 172.21.15.37:0/4121385156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a6c102770 0x7f4a6c19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:54.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.443+0000 7f4a6bfff700 1 -- 172.21.15.37:0/4121385156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a6c107860 msgr2=0x7f4a6c198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:54.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.443+0000 7f4a6bfff700 1 --2- 172.21.15.37:0/4121385156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a6c107860 0x7f4a6c198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.443+0000 7f4a6bfff700 1 -- 172.21.15.37:0/4121385156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a6c102770 msgr2=0x7f4a6c19e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:54.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.443+0000 7f4a6bfff700 1 --2- 172.21.15.37:0/4121385156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a6c102770 0x7f4a6c19e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.443+0000 7f4a6bfff700 1 -- 172.21.15.37:0/4121385156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a6c199160 con 0x7f4a6c103120 2024-06-10T00:59:54.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.443+0000 7f4a70dda700 1 --2- 172.21.15.37:0/4121385156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a6c102770 0x7f4a6c19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:59:54.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.443+0000 7f4a715db700 1 --2- 172.21.15.37:0/4121385156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a6c107860 0x7f4a6c198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:59:54.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.444+0000 7f4a6bfff700 1 --2- 172.21.15.37:0/4121385156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a6c103120 0x7f4a6c19ed80 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800c9b0 tx=0x7f4a5800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:54.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.444+0000 7f4a69ffb700 1 -- 172.21.15.37:0/4121385156 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58007b00 con 0x7f4a6c103120 2024-06-10T00:59:54.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.444+0000 7f4a7303e700 1 -- 172.21.15.37:0/4121385156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a6c199450 con 0x7f4a6c103120 2024-06-10T00:59:54.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.444+0000 7f4a69ffb700 1 -- 172.21.15.37:0/4121385156 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a58007ca0 con 0x7f4a6c103120 2024-06-10T00:59:54.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.445+0000 7f4a69ffb700 1 -- 172.21.15.37:0/4121385156 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58018780 con 0x7f4a6c103120 2024-06-10T00:59:54.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.445+0000 7f4a7303e700 1 -- 172.21.15.37:0/4121385156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a6c0746c0 con 0x7f4a6c103120 2024-06-10T00:59:54.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.446+0000 7f4a69ffb700 1 -- 172.21.15.37:0/4121385156 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4a5800f450 con 0x7f4a6c103120 2024-06-10T00:59:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.447+0000 7f4a69ffb700 1 --2- 172.21.15.37:0/4121385156 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4a5c0a2c20 0x7f4a5c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.447+0000 7f4a70dda700 1 --2- 172.21.15.37:0/4121385156 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4a5c0a2c20 0x7f4a5c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.447+0000 7f4a69ffb700 1 -- 172.21.15.37:0/4121385156 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f4a580cf500 con 0x7f4a6c103120 2024-06-10T00:59:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.448+0000 7f4a70dda700 1 --2- 172.21.15.37:0/4121385156 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4a5c0a2c20 0x7f4a5c0a50e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4a60009750 tx=0x7f4a60005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.448+0000 7f4a7d8c1b80 1 Processor -- start 2024-06-10T00:59:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.448+0000 7f4a7d8c1b80 1 -- start start 2024-06-10T00:59:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.448+0000 7f4a7d8c1b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b8825ec40 0x558b8825f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.448+0000 7f4a7d8c1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b8825f560 0x558b88258e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.448+0000 7f4a7d8c1b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b882593d0 0x558b88259810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.448+0000 7f4a7d8c1b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558b881aa910 con 0x558b882593d0 2024-06-10T00:59:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.448+0000 7f4a7d8c1b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558b88152820 con 0x558b8825ec40 2024-06-10T00:59:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.448+0000 7f4a7d8c1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558b881a68d0 con 0x558b8825f560 2024-06-10T00:59:54.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.449+0000 7f4a715db700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b882593d0 0x558b88259810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:54.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.449+0000 7f4a715db700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b882593d0 0x558b88259810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40664/0 (socket says 172.21.15.37:40664) 2024-06-10T00:59:54.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.449+0000 7f4a715db700 1 -- 172.21.15.37:0/2842604254 learned_addr learned my addr 172.21.15.37:0/2842604254 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:54.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.449+0000 7f4a70dda700 1 --2- 172.21.15.37:0/2842604254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b8825ec40 0x558b8825f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:54.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.449+0000 7f4a6bfff700 1 --2- 172.21.15.37:0/2842604254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b8825f560 0x558b88258e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:54.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.449+0000 7f4a715db700 1 -- 172.21.15.37:0/2842604254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b8825f560 msgr2=0x558b88258e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:54.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.449+0000 7f4a715db700 1 --2- 172.21.15.37:0/2842604254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b8825f560 0x558b88258e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.449+0000 7f4a715db700 1 -- 172.21.15.37:0/2842604254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b8825ec40 msgr2=0x558b8825f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:54.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.449+0000 7f4a715db700 1 --2- 172.21.15.37:0/2842604254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b8825ec40 0x558b8825f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.449+0000 7f4a715db700 1 -- 172.21.15.37:0/2842604254 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b88149050 con 0x558b882593d0 2024-06-10T00:59:54.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.449+0000 7f4a715db700 1 --2- 172.21.15.37:0/2842604254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b882593d0 0x558b88259810 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400c450 tx=0x7f4a6400c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69912c14cd86eff4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:54.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.449+0000 7f4a56ffd700 1 -- 172.21.15.37:0/2842604254 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6400cb00 con 0x558b882593d0 2024-06-10T00:59:54.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.449+0000 7f4a56ffd700 1 -- 172.21.15.37:0/2842604254 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a6400cca0 con 0x558b882593d0 2024-06-10T00:59:54.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.449+0000 7f4a56ffd700 1 -- 172.21.15.37:0/2842604254 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a64015600 con 0x558b882593d0 2024-06-10T00:59:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.450+0000 7f4a7d8c1b80 1 -- 172.21.15.37:0/2842604254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b882593d0 msgr2=0x558b88259810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.450+0000 7f4a7d8c1b80 1 --2- 172.21.15.37:0/2842604254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b882593d0 0x558b88259810 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400c450 tx=0x7f4a6400c480 comp rx=0 tx=0).stop 2024-06-10T00:59:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.450+0000 7f4a7d8c1b80 1 -- 172.21.15.37:0/2842604254 shutdown_connections 2024-06-10T00:59:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.450+0000 7f4a7d8c1b80 1 --2- 172.21.15.37:0/2842604254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b882593d0 0x558b88259810 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.450+0000 7f4a7d8c1b80 1 --2- 172.21.15.37:0/2842604254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b8825ec40 0x558b8825f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.450+0000 7f4a7d8c1b80 1 --2- 172.21.15.37:0/2842604254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b8825f560 0x558b88258e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.450+0000 7f4a7d8c1b80 1 -- 172.21.15.37:0/2842604254 >> 172.21.15.37:0/2842604254 conn(0x558b88183dc0 msgr2=0x558b88254120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:54.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.450+0000 7f4a7d8c1b80 1 -- 172.21.15.37:0/2842604254 shutdown_connections 2024-06-10T00:59:54.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.450+0000 7f4a7d8c1b80 1 -- 172.21.15.37:0/2842604254 wait complete. 2024-06-10T00:59:54.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.450+0000 7f4a7d8c1b80 1 -- start start 2024-06-10T00:59:54.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.451+0000 7f4a7d8c1b80 1 Processor -- start 2024-06-10T00:59:54.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.451+0000 7f4a7d8c1b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b882593d0 0x558b882652b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:54.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.451+0000 7f4a7d8c1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b8825ec40 0x558b882625b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:54.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.451+0000 7f4a7d8c1b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b8825f560 0x558b88262af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:54.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.451+0000 7f4a7d8c1b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558b88265fd0 con 0x558b882593d0 2024-06-10T00:59:54.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.451+0000 7f4a7d8c1b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558b8814acf0 con 0x558b8825f560 2024-06-10T00:59:54.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.451+0000 7f4a7d8c1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558b88266150 con 0x558b8825ec40 2024-06-10T00:59:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.451+0000 7f4a7d8c1b80 10 client.0 _ll_register_callbacks cb 0x558b88144a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:59:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.451+0000 7f4a70dda700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b882593d0 0x558b882652b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.451+0000 7f4a70dda700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b882593d0 0x558b882652b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40680/0 (socket says 172.21.15.37:40680) 2024-06-10T00:59:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.451+0000 7f4a70dda700 1 -- 172.21.15.37:0/2039736038 learned_addr learned my addr 172.21.15.37:0/2039736038 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.451+0000 7f4a6bfff700 1 --2- 172.21.15.37:0/2039736038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b8825ec40 0x558b882625b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.451+0000 7f4a715db700 1 --2- 172.21.15.37:0/2039736038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b8825f560 0x558b88262af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.451+0000 7f4a70dda700 1 -- 172.21.15.37:0/2039736038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b8825ec40 msgr2=0x558b882625b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.451+0000 7f4a70dda700 1 --2- 172.21.15.37:0/2039736038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b8825ec40 0x558b882625b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.451+0000 7f4a70dda700 1 -- 172.21.15.37:0/2039736038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b8825f560 msgr2=0x558b88262af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.451+0000 7f4a70dda700 1 --2- 172.21.15.37:0/2039736038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b8825f560 0x558b88262af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.451+0000 7f4a70dda700 1 -- 172.21.15.37:0/2039736038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b880f99d0 con 0x558b882593d0 2024-06-10T00:59:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.451+0000 7f4a715db700 1 --2- 172.21.15.37:0/2039736038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b8825f560 0x558b88262af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:59:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.451+0000 7f4a6bfff700 1 --2- 172.21.15.37:0/2039736038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b8825ec40 0x558b882625b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T00:59:54.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.452+0000 7f4a70dda700 1 --2- 172.21.15.37:0/2039736038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b882593d0 0x558b882652b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4a600099c0 tx=0x7f4a60009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81424dc6caf95f02 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:54.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.452+0000 7f4a4bfff700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:59:54.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.452+0000 7f4a4bfff700 1 -- 172.21.15.37:0/2039736038 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a60034060 con 0x558b882593d0 2024-06-10T00:59:54.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.452+0000 7f4a7d8c1b80 1 -- 172.21.15.37:0/2039736038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a6400cca0 con 0x558b882593d0 2024-06-10T00:59:54.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.452+0000 7f4a4bfff700 1 -- 172.21.15.37:0/2039736038 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f4a6000bae0 con 0x558b882593d0 2024-06-10T00:59:54.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.452+0000 7f4a4bfff700 1 -- 172.21.15.37:0/2039736038 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4a6003b050 con 0x558b882593d0 2024-06-10T00:59:54.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.452+0000 7f4a7d8c1b80 10 client.15999 fetch_fsmap learned FSMap version 27 2024-06-10T00:59:54.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.452+0000 7f4a7d8c1b80 1 -- 172.21.15.37:0/2039736038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558b88270960 con 0x558b882593d0 2024-06-10T00:59:54.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.452+0000 7f4a4bfff700 1 -- 172.21.15.37:0/2039736038 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a6400cca0 con 0x558b882593d0 2024-06-10T00:59:54.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.453+0000 7f4a4bfff700 1 -- 172.21.15.37:0/2039736038 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4a6002f420 con 0x558b882593d0 2024-06-10T00:59:54.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.453+0000 7f4a7d8c1b80 10 client.15999 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:59:54.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.453+0000 7f4a7d8c1b80 10 client.15999 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T00:59:54.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.453+0000 7f4a7d8c1b80 20 client.15999 populate_metadata read hostname 'smithi037' 2024-06-10T00:59:54.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.453+0000 7f4a7d8c1b80 1 --2- 172.21.15.37:0/2039736038 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558b88272ef0 0x558b88275300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:54.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.453+0000 7f4a7d8c1b80 4 client.15999 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T00:59:54.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.453+0000 7f4a7d8c1b80 1 -- 172.21.15.37:0/2039736038 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b881b7e10 con 0x558b88272ef0 2024-06-10T00:59:54.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.453+0000 7f4a6bfff700 1 --2- 172.21.15.37:0/2039736038 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558b88272ef0 0x558b88275300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:54.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.453+0000 7f4a4bfff700 10 client.15999 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:59:54.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.453+0000 7f4a6bfff700 1 --2- 172.21.15.37:0/2039736038 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558b88272ef0 0x558b88275300 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffe3e3341527634b server_cookie=ea7f00b2c6154dd in_seq=0 out_seq=0 2024-06-10T00:59:54.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.454+0000 7f4a4bfff700 1 -- 172.21.15.37:0/2039736038 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558b881b7e10 con 0x558b88272ef0 2024-06-10T00:59:54.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.454+0000 7f4a4bfff700 10 client.15999 handle_command_reply: tid=0 2024-06-10T00:59:54.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.454+0000 7f4a7d8c1b80 1 client.15999 shutdown 2024-06-10T00:59:54.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.454+0000 7f4a7d8c1b80 1 -- 172.21.15.37:0/2039736038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b882593d0 msgr2=0x558b882652b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:54.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.454+0000 7f4a7d8c1b80 1 --2- 172.21.15.37:0/2039736038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b882593d0 0x558b882652b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4a600099c0 tx=0x7f4a60009cd0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.454+0000 7f4a7d8c1b80 1 -- 172.21.15.37:0/2039736038 shutdown_connections 2024-06-10T00:59:54.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.454+0000 7f4a7d8c1b80 1 --2- 172.21.15.37:0/2039736038 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558b88272ef0 0x558b88275300 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.454+0000 7f4a7d8c1b80 1 --2- 172.21.15.37:0/2039736038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b8825f560 0x558b88262af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.454+0000 7f4a7d8c1b80 1 --2- 172.21.15.37:0/2039736038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b882593d0 0x558b882652b0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.454+0000 7f4a7d8c1b80 1 --2- 172.21.15.37:0/2039736038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b8825ec40 0x558b882625b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.454+0000 7f4a7d8c1b80 1 -- 172.21.15.37:0/2039736038 >> 172.21.15.37:0/2039736038 conn(0x558b88183dc0 msgr2=0x558b88253d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:54.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.454+0000 7f4a4bfff700 0 client.15999 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:59:54.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.455+0000 7f4a7d8c1b80 1 -- 172.21.15.37:0/2039736038 shutdown_connections 2024-06-10T00:59:54.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.455+0000 7f4a7d8c1b80 1 -- 172.21.15.37:0/2039736038 wait complete. 2024-06-10T00:59:54.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.455+0000 7f4a7d8c1b80 20 client.15999 trim_cache size 0 max 16384 2024-06-10T00:59:54.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.477+0000 7f4a7d8c1b80 1 Processor -- start 2024-06-10T00:59:54.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.477+0000 7f4a7d8c1b80 1 -- start start 2024-06-10T00:59:54.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.477+0000 7f4a7d8c1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b882593d0 0x558b88258e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:54.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.477+0000 7f4a7d8c1b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b8825ec40 0x558b882615d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:54.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.477+0000 7f4a7d8c1b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8825f560 0x558b8827abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:54.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.477+0000 7f4a7d8c1b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6c1b2f60 con 0x558b8825f560 2024-06-10T00:59:54.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.477+0000 7f4a7d8c1b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6c1b07f0 con 0x558b8825ec40 2024-06-10T00:59:54.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.477+0000 7f4a7d8c1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6c1ae640 con 0x558b882593d0 2024-06-10T00:59:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.477+0000 7f4a715db700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8825f560 0x558b8827abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.477+0000 7f4a715db700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8825f560 0x558b8827abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40692/0 (socket says 172.21.15.37:40692) 2024-06-10T00:59:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.477+0000 7f4a715db700 1 -- 172.21.15.37:0/3976561420 learned_addr learned my addr 172.21.15.37:0/3976561420 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.477+0000 7f4a70dda700 1 --2- 172.21.15.37:0/3976561420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b882593d0 0x558b88258e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.477+0000 7f4a715db700 1 -- 172.21.15.37:0/3976561420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b882593d0 msgr2=0x558b88258e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.477+0000 7f4a715db700 1 --2- 172.21.15.37:0/3976561420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b882593d0 0x558b88258e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.477+0000 7f4a715db700 1 -- 172.21.15.37:0/3976561420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b8825ec40 msgr2=0x558b882615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.477+0000 7f4a6bfff700 1 --2- 172.21.15.37:0/3976561420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b8825ec40 0x558b882615d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.477+0000 7f4a715db700 1 --2- 172.21.15.37:0/3976561420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b8825ec40 0x558b882615d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.477+0000 7f4a715db700 1 -- 172.21.15.37:0/3976561420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b8825e740 con 0x558b8825f560 2024-06-10T00:59:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.477+0000 7f4a70dda700 1 --2- 172.21.15.37:0/3976561420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b882593d0 0x558b88258e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T00:59:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.478+0000 7f4a715db700 1 --2- 172.21.15.37:0/3976561420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8825f560 0x558b8827abd0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4a64004c20 tx=0x7f4a64004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9d70d39642ee8a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:54.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.478+0000 7f4a567fc700 1 -- 172.21.15.37:0/3976561420 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a64025070 con 0x558b8825f560 2024-06-10T00:59:54.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.478+0000 7f4a567fc700 1 -- 172.21.15.37:0/3976561420 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a6400a400 con 0x558b8825f560 2024-06-10T00:59:54.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.478+0000 7f4a567fc700 1 -- 172.21.15.37:0/3976561420 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a640206b0 con 0x558b8825f560 2024-06-10T00:59:54.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.478+0000 7f4a7d8c1b80 1 -- 172.21.15.37:0/3976561420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8825f560 msgr2=0x558b8827abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:54.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.478+0000 7f4a7d8c1b80 1 --2- 172.21.15.37:0/3976561420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8825f560 0x558b8827abd0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4a64004c20 tx=0x7f4a64004a50 comp rx=0 tx=0).stop 2024-06-10T00:59:54.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.478+0000 7f4a7d8c1b80 1 -- 172.21.15.37:0/3976561420 shutdown_connections 2024-06-10T00:59:54.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.478+0000 7f4a7d8c1b80 1 --2- 172.21.15.37:0/3976561420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8825f560 0x558b8827abd0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.478+0000 7f4a7d8c1b80 1 --2- 172.21.15.37:0/3976561420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b882593d0 0x558b88258e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.478+0000 7f4a7d8c1b80 1 --2- 172.21.15.37:0/3976561420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b8825ec40 0x558b882615d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.479+0000 7f4a7d8c1b80 1 -- 172.21.15.37:0/3976561420 >> 172.21.15.37:0/3976561420 conn(0x558b88183dc0 msgr2=0x558b88260ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:54.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.479+0000 7f4a7d8c1b80 1 -- 172.21.15.37:0/3976561420 shutdown_connections 2024-06-10T00:59:54.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.479+0000 7f4a7d8c1b80 1 -- 172.21.15.37:0/3976561420 wait complete. 2024-06-10T00:59:54.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.479+0000 7f4a7d8c1b80 1 -- start start 2024-06-10T00:59:54.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.479+0000 7f4a7d8c1b80 1 Processor -- start 2024-06-10T00:59:54.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.479+0000 7f4a7d8c1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b882593d0 0x558b8826b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:54.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.479+0000 7f4a7d8c1b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8825ec40 0x558b88267c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:54.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.479+0000 7f4a7d8c1b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b8825f560 0x558b88268170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:54.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.479+0000 7f4a7d8c1b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6c1aecb0 con 0x558b8825ec40 2024-06-10T00:59:54.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.479+0000 7f4a7d8c1b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6c1b0120 con 0x558b8825f560 2024-06-10T00:59:54.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.479+0000 7f4a7d8c1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6c1af380 con 0x558b882593d0 2024-06-10T00:59:54.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.479+0000 7f4a7d8c1b80 10 client.0 _ll_register_callbacks cb 0x558b88144a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T00:59:54.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.480+0000 7f4a6bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8825ec40 0x558b88267c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:54.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.480+0000 7f4a6bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8825ec40 0x558b88267c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40694/0 (socket says 172.21.15.37:40694) 2024-06-10T00:59:54.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.480+0000 7f4a6bfff700 1 -- 172.21.15.37:0/4123544381 learned_addr learned my addr 172.21.15.37:0/4123544381 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T00:59:54.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.480+0000 7f4a715db700 1 --2- 172.21.15.37:0/4123544381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b8825f560 0x558b88268170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:54.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.480+0000 7f4a70dda700 1 --2- 172.21.15.37:0/4123544381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b882593d0 0x558b8826b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:54.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.480+0000 7f4a6bfff700 1 -- 172.21.15.37:0/4123544381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b882593d0 msgr2=0x558b8826b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:54.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.480+0000 7f4a6bfff700 1 --2- 172.21.15.37:0/4123544381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b882593d0 0x558b8826b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.480+0000 7f4a6bfff700 1 -- 172.21.15.37:0/4123544381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b8825f560 msgr2=0x558b88268170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T00:59:54.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.480+0000 7f4a6bfff700 1 --2- 172.21.15.37:0/4123544381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b8825f560 0x558b88268170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.480+0000 7f4a6bfff700 1 -- 172.21.15.37:0/4123544381 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b882686b0 con 0x558b8825ec40 2024-06-10T00:59:54.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.480+0000 7f4a6bfff700 1 --2- 172.21.15.37:0/4123544381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8825ec40 0x558b88267c30 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4a580074a0 tx=0x7f4a580cca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b17be003b4d862 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T00:59:54.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.480+0000 7f4a4bfff700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T00:59:54.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.480+0000 7f4a4bfff700 1 -- 172.21.15.37:0/4123544381 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58060e60 con 0x558b8825ec40 2024-06-10T00:59:54.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.480+0000 7f4a4bfff700 1 -- 172.21.15.37:0/4123544381 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f4a58099c10 con 0x558b8825ec40 2024-06-10T00:59:54.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.480+0000 7f4a7d8c1b80 1 -- 172.21.15.37:0/4123544381 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a6400a400 con 0x558b8825ec40 2024-06-10T00:59:54.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.480+0000 7f4a4bfff700 1 -- 172.21.15.37:0/4123544381 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4a580a4580 con 0x558b8825ec40 2024-06-10T00:59:54.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.480+0000 7f4a7d8c1b80 10 client.16005 fetch_fsmap learned FSMap version 27 2024-06-10T00:59:54.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.480+0000 7f4a7d8c1b80 1 -- 172.21.15.37:0/4123544381 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558b882de920 con 0x558b8825ec40 2024-06-10T00:59:54.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.481+0000 7f4a4bfff700 1 -- 172.21.15.37:0/4123544381 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a6400a400 con 0x558b8825ec40 2024-06-10T00:59:54.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.481+0000 7f4a4bfff700 1 -- 172.21.15.37:0/4123544381 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4a580cf150 con 0x558b8825ec40 2024-06-10T00:59:54.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.481+0000 7f4a7d8c1b80 10 client.16005 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T00:59:54.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.481+0000 7f4a7d8c1b80 10 client.16005 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T00:59:54.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.481+0000 7f4a7d8c1b80 20 client.16005 populate_metadata read hostname 'smithi037' 2024-06-10T00:59:54.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.481+0000 7f4a7d8c1b80 1 --2- 172.21.15.37:0/4123544381 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558b882defe0 0x558b882e1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T00:59:54.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.481+0000 7f4a7d8c1b80 4 client.16005 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "5049796b", "release": true, "await": true}] 2024-06-10T00:59:54.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.481+0000 7f4a7d8c1b80 1 -- 172.21.15.37:0/4123544381 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "quiesce db", "set_id": "5049796b", "release": true, "await": true}) -- 0x558b881b2390 con 0x558b882defe0 2024-06-10T00:59:54.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.481+0000 7f4a70dda700 1 --2- 172.21.15.37:0/4123544381 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558b882defe0 0x558b882e1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T00:59:54.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.482+0000 7f4a70dda700 1 --2- 172.21.15.37:0/4123544381 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558b882defe0 0x558b882e1400 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b91436d957ebf268 server_cookie=85e287259ea59637 in_seq=0 out_seq=0 2024-06-10T00:59:54.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.482+0000 7f4a4bfff700 10 client.16005 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T00:59:54.485 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:54.485 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:54.485 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 38 -lt 100 ']' 2024-06-10T00:59:54.486 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 38 + 1 2024-06-10T00:59:54.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.486+0000 7f4a4bfff700 1 -- 172.21.15.37:0/4123544381 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x558b881b2390 con 0x558b882defe0 2024-06-10T00:59:54.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.486+0000 7f4a4bfff700 10 client.16005 handle_command_reply: tid=0 2024-06-10T00:59:54.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.486+0000 7f4a7d8c1b80 1 client.16005 shutdown 2024-06-10T00:59:54.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.486+0000 7f4a7d8c1b80 1 -- 172.21.15.37:0/4123544381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8825ec40 msgr2=0x558b88267c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:54.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.486+0000 7f4a7d8c1b80 1 --2- 172.21.15.37:0/4123544381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8825ec40 0x558b88267c30 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4a580074a0 tx=0x7f4a580cca80 comp rx=0 tx=0).stop 2024-06-10T00:59:54.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.486+0000 7f4a7d8c1b80 1 -- 172.21.15.37:0/4123544381 shutdown_connections 2024-06-10T00:59:54.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.486+0000 7f4a7d8c1b80 1 --2- 172.21.15.37:0/4123544381 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558b882defe0 0x558b882e1400 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.486+0000 7f4a4bfff700 0 client.16005 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T00:59:54.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.486+0000 7f4a7d8c1b80 1 --2- 172.21.15.37:0/4123544381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b8825f560 0x558b88268170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.486+0000 7f4a7d8c1b80 1 --2- 172.21.15.37:0/4123544381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558b882593d0 0x558b8826b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.486+0000 7f4a7d8c1b80 1 --2- 172.21.15.37:0/4123544381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b8825ec40 0x558b88267c30 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.486+0000 7f4a7d8c1b80 1 -- 172.21.15.37:0/4123544381 >> 172.21.15.37:0/4123544381 conn(0x558b88183dc0 msgr2=0x558b8825dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:54.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.487+0000 7f4a7d8c1b80 1 -- 172.21.15.37:0/4123544381 shutdown_connections 2024-06-10T00:59:54.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.487+0000 7f4a7d8c1b80 1 -- 172.21.15.37:0/4123544381 wait complete. 2024-06-10T00:59:54.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.487+0000 7f4a7d8c1b80 20 client.16005 trim_cache size 0 max 16384 2024-06-10T00:59:54.488 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T00:59:54.488 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-06-10T00:59:54.488 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24472, 2024-06-10T00:59:54.488 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 8, 2024-06-10T00:59:54.488 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-06-10T00:59:54.488 INFO:teuthology.orchestra.run.smithi037.stdout: "5049796b": { 2024-06-10T00:59:54.488 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 8, 2024-06-10T00:59:54.488 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-06-10T00:59:54.488 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T00:59:54.489 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-06-10T00:59:54.489 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T00:59:54.489 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T00:59:54.489 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-06-10T00:59:54.489 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 150.0, 2024-06-10T00:59:54.489 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-06-10T00:59:54.489 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-06-10T00:59:54.489 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-06-10T00:59:54.489 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T00:59:54.489 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-06-10T00:59:54.489 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T00:59:54.489 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T00:59:54.489 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T00:59:54.489 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T00:59:54.490 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T00:59:54.490 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T00:59:54.490 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T00:59:54.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.487+0000 7f4a7303e700 1 -- 172.21.15.37:0/4121385156 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4a5c0a2c20 msgr2=0x7f4a5c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:54.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.487+0000 7f4a7303e700 1 --2- 172.21.15.37:0/4121385156 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4a5c0a2c20 0x7f4a5c0a50e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4a60009750 tx=0x7f4a60005fb0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.487+0000 7f4a7303e700 1 -- 172.21.15.37:0/4121385156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a6c103120 msgr2=0x7f4a6c19ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T00:59:54.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.487+0000 7f4a7303e700 1 --2- 172.21.15.37:0/4121385156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a6c103120 0x7f4a6c19ed80 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800c9b0 tx=0x7f4a5800cd70 comp rx=0 tx=0).stop 2024-06-10T00:59:54.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.487+0000 7f4a7303e700 1 -- 172.21.15.37:0/4121385156 shutdown_connections 2024-06-10T00:59:54.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.487+0000 7f4a7303e700 1 --2- 172.21.15.37:0/4121385156 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4a5c0a2c20 0x7f4a5c0a50e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.487+0000 7f4a7303e700 1 --2- 172.21.15.37:0/4121385156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a6c107860 0x7f4a6c198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.487+0000 7f4a7303e700 1 --2- 172.21.15.37:0/4121385156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a6c102770 0x7f4a6c19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.487+0000 7f4a7303e700 1 --2- 172.21.15.37:0/4121385156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a6c103120 0x7f4a6c19ed80 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T00:59:54.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.487+0000 7f4a7303e700 1 -- 172.21.15.37:0/4121385156 >> 172.21.15.37:0/4121385156 conn(0x7f4a6c079600 msgr2=0x7f4a6c100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T00:59:54.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.487+0000 7f4a7303e700 1 -- 172.21.15.37:0/4121385156 shutdown_connections 2024-06-10T00:59:54.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T00:59:54.487+0000 7f4a7303e700 1 -- 172.21.15.37:0/4121385156 wait complete. 2024-06-10T00:59:54.537 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=39 2024-06-10T00:59:54.537 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:59:54.560 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:54.560 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:54.562 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:59:54.562 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:59:54.562 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:59:54.904 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:54.904 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:55.232 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T00:59:55.232 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T00:59:55.384 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:59:55.384 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:59:55.389 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:55.389 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:55.390 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 39 -lt 100 ']' 2024-06-10T00:59:55.390 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 39 + 1 2024-06-10T00:59:55.395 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=40 2024-06-10T00:59:55.395 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:59:55.421 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:55.421 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:55.422 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:59:55.423 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:59:55.423 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:59:56.144 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:56 smithi037 ceph-mon[32192]: pgmap v527: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-06-10T00:59:56.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:56 smithi167 ceph-mon[36917]: pgmap v527: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-06-10T00:59:56.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:56 smithi094 ceph-mon[36708]: pgmap v527: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-06-10T00:59:56.686 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:56.687 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:56.688 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '5049796b', total seconds elapsed: 98.9 2024-06-10T00:59:56.884 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:59:56.884 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:59:56.915 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:56.915 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:56.915 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 40 -lt 100 ']' 2024-06-10T00:59:56.915 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 40 + 1 2024-06-10T00:59:56.958 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=41 2024-06-10T00:59:56.959 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:59:57.012 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:57.012 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:57.013 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:59:57.014 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:59:57.014 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:59:57.672 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:57.672 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:57.950 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:59:57.951 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:59:57.974 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:57.974 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:57.974 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 41 -lt 100 ']' 2024-06-10T00:59:57.975 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 41 + 1 2024-06-10T00:59:58.000 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=42 2024-06-10T00:59:58.000 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:59:58.018 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:58.019 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:58.020 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:59:58.020 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:59:58.020 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:59:58.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 00:59:58 smithi167 ceph-mon[36917]: pgmap v528: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2024-06-10T00:59:58.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 00:59:58 smithi094 ceph-mon[36708]: pgmap v528: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2024-06-10T00:59:58.377 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 00:59:58 smithi037 ceph-mon[32192]: pgmap v528: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.8 KiB/s wr, 11 op/s 2024-06-10T00:59:58.520 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:58.520 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:58.717 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:59:58.717 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:59:58.722 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:58.723 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:58.723 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 42 -lt 100 ']' 2024-06-10T00:59:58.723 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 42 + 1 2024-06-10T00:59:58.790 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=43 2024-06-10T00:59:58.790 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T00:59:58.810 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:58.810 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:58.812 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T00:59:58.812 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T00:59:58.812 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T00:59:59.555 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T00:59:59.555 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T00:59:59.967 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T00:59:59.967 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T00:59:59.990 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:59.990 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T00:59:59.990 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 43 -lt 100 ']' 2024-06-10T00:59:59.991 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 43 + 1 2024-06-10T01:00:00.135 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=44 2024-06-10T01:00:00.135 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:00.213 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:00.213 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:00.214 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:00.215 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:00.215 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:00.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:00 smithi167 ceph-mon[36917]: pgmap v529: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-06-10T01:00:00.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:00 smithi167 ceph-mon[36917]: overall HEALTH_OK 2024-06-10T01:00:00.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:00 smithi094 ceph-mon[36708]: pgmap v529: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-06-10T01:00:00.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:00 smithi094 ceph-mon[36708]: overall HEALTH_OK 2024-06-10T01:00:00.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:00 smithi037 ceph-mon[32192]: pgmap v529: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-06-10T01:00:00.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:00 smithi037 ceph-mon[32192]: overall HEALTH_OK 2024-06-10T01:00:01.221 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:01.221 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:01.583 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:01.584 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:01.599 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:01.599 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:01.599 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 44 -lt 100 ']' 2024-06-10T01:00:01.599 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 44 + 1 2024-06-10T01:00:01.666 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=45 2024-06-10T01:00:01.666 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:01.713 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:01.713 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:01.715 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:01.715 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:01.715 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:02.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:02 smithi167 ceph-mon[36917]: pgmap v530: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-06-10T01:00:02.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:02 smithi094 ceph-mon[36708]: pgmap v530: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-06-10T01:00:02.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:02 smithi037 ceph-mon[32192]: pgmap v530: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-06-10T01:00:02.761 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:02.761 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:03.124 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:03.125 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:03.206 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:03.207 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:03.207 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 45 -lt 100 ']' 2024-06-10T01:00:03.207 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 45 + 1 2024-06-10T01:00:03.283 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=46 2024-06-10T01:00:03.283 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:03.291 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:03.291 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:03.292 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:03.293 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:03.293 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:03.993 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:00:04.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:04 smithi167 ceph-mon[36917]: pgmap v531: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 KiB/s wr, 12 op/s 2024-06-10T01:00:04.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:04 smithi094 ceph-mon[36708]: pgmap v531: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 KiB/s wr, 12 op/s 2024-06-10T01:00:04.369 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:04.369 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:04.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:04 smithi037 ceph-mon[32192]: pgmap v531: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 KiB/s wr, 12 op/s 2024-06-10T01:00:04.733 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:04.733 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:04.756 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:04.756 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:04.756 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 46 -lt 100 ']' 2024-06-10T01:00:04.756 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 46 + 1 2024-06-10T01:00:04.802 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=47 2024-06-10T01:00:04.802 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:04.861 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:04.861 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:04.862 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:04.863 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:04.863 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:06.127 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:06.127 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:06.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:06 smithi167 ceph-mon[36917]: pgmap v532: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-06-10T01:00:06.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:06 smithi094 ceph-mon[36708]: pgmap v532: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-06-10T01:00:06.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:06 smithi037 ceph-mon[32192]: pgmap v532: 129 pgs: 129 active+clean; 1.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-06-10T01:00:06.506 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:06.506 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:06.523 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:06.523 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:06.523 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 47 -lt 100 ']' 2024-06-10T01:00:06.523 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 47 + 1 2024-06-10T01:00:06.740 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=48 2024-06-10T01:00:06.740 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:07.005 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:07.006 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:07.007 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:07.007 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:07.007 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:08.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:08 smithi037 ceph-mon[32192]: pgmap v533: 129 pgs: 129 active+clean; 1.4 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 KiB/s wr, 8 op/s 2024-06-10T01:00:08.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:08 smithi167 ceph-mon[36917]: pgmap v533: 129 pgs: 129 active+clean; 1.4 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 KiB/s wr, 8 op/s 2024-06-10T01:00:08.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:08 smithi094 ceph-mon[36708]: pgmap v533: 129 pgs: 129 active+clean; 1.4 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 KiB/s wr, 8 op/s 2024-06-10T01:00:08.768 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:08.768 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:09.491 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:09.491 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:09.542 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:09.542 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:09.543 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 48 -lt 100 ']' 2024-06-10T01:00:09.543 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 48 + 1 2024-06-10T01:00:09.762 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=49 2024-06-10T01:00:09.762 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:09.835 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:09.835 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:09.836 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:09.837 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:09.837 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.900+0000 7f9950d8b640 1 -- 172.21.15.37:0/3823301974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f994c102c20 msgr2=0x7f994c1030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.900+0000 7f9950d8b640 1 --2- 172.21.15.37:0/3823301974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f994c102c20 0x7f994c1030a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9940009860 tx=0x7f994002a340 comp rx=0 tx=0).stop 2024-06-10T01:00:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.900+0000 7f9950d8b640 1 -- 172.21.15.37:0/3823301974 shutdown_connections 2024-06-10T01:00:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.900+0000 7f9950d8b640 1 --2- 172.21.15.37:0/3823301974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f994c1035e0 0x7f994c109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.900+0000 7f9950d8b640 1 --2- 172.21.15.37:0/3823301974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f994c102c20 0x7f994c1030a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.900+0000 7f9950d8b640 1 --2- 172.21.15.37:0/3823301974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f994c101a30 0x7f994c101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.900+0000 7f9950d8b640 1 -- 172.21.15.37:0/3823301974 >> 172.21.15.37:0/3823301974 conn(0x7f994c0fcfc0 msgr2=0x7f994c0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.900+0000 7f9950d8b640 1 -- 172.21.15.37:0/3823301974 shutdown_connections 2024-06-10T01:00:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.900+0000 7f9950d8b640 1 -- 172.21.15.37:0/3823301974 wait complete. 2024-06-10T01:00:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.937+0000 7f9950d8b640 1 Processor -- start 2024-06-10T01:00:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.937+0000 7f9950d8b640 1 -- start start 2024-06-10T01:00:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.937+0000 7f9950d8b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f994c101a30 0x7f994c075750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.937+0000 7f9950d8b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f994c102c20 0x7f994c073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.937+0000 7f9950d8b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f994c1035e0 0x7f994c0742d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.937+0000 7f9950d8b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f994c10d6b0 con 0x7f994c101a30 2024-06-10T01:00:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.937+0000 7f9950d8b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f994c10d530 con 0x7f994c1035e0 2024-06-10T01:00:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.937+0000 7f9950d8b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f994c10d830 con 0x7f994c102c20 2024-06-10T01:00:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.938+0000 7f994a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f994c101a30 0x7f994c075750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.938+0000 7f994a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f994c101a30 0x7f994c075750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36706/0 (socket says 172.21.15.37:36706) 2024-06-10T01:00:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.938+0000 7f994a59c640 1 -- 172.21.15.37:0/3391084390 learned_addr learned my addr 172.21.15.37:0/3391084390 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:00:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.938+0000 7f994ad9d640 1 --2- 172.21.15.37:0/3391084390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f994c1035e0 0x7f994c0742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.938+0000 7f9949d9b640 1 --2- 172.21.15.37:0/3391084390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f994c102c20 0x7f994c073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.938+0000 7f994a59c640 1 -- 172.21.15.37:0/3391084390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f994c102c20 msgr2=0x7f994c073d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.938+0000 7f994a59c640 1 --2- 172.21.15.37:0/3391084390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f994c102c20 0x7f994c073d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.938+0000 7f994a59c640 1 -- 172.21.15.37:0/3391084390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f994c1035e0 msgr2=0x7f994c0742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.938+0000 7f994a59c640 1 --2- 172.21.15.37:0/3391084390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f994c1035e0 0x7f994c0742d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.938+0000 7f994a59c640 1 -- 172.21.15.37:0/3391084390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994c074b60 con 0x7f994c101a30 2024-06-10T01:00:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.938+0000 7f994ad9d640 1 --2- 172.21.15.37:0/3391084390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f994c1035e0 0x7f994c0742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:00:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.938+0000 7f9949d9b640 1 --2- 172.21.15.37:0/3391084390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f994c102c20 0x7f994c073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:00:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.938+0000 7f994a59c640 1 --2- 172.21.15.37:0/3391084390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f994c101a30 0x7f994c075750 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f993c00e9a0 tx=0x7f993c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.947+0000 7f993b7fe640 1 -- 172.21.15.37:0/3391084390 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c00cd80 con 0x7f994c101a30 2024-06-10T01:00:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.947+0000 7f993b7fe640 1 -- 172.21.15.37:0/3391084390 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f993c010910 con 0x7f994c101a30 2024-06-10T01:00:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.947+0000 7f993b7fe640 1 -- 172.21.15.37:0/3391084390 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c018a30 con 0x7f994c101a30 2024-06-10T01:00:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.947+0000 7f9950d8b640 1 -- 172.21.15.37:0/3391084390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994c1a1db0 con 0x7f994c101a30 2024-06-10T01:00:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.950+0000 7f993b7fe640 1 -- 172.21.15.37:0/3391084390 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f993c010ac0 con 0x7f994c101a30 2024-06-10T01:00:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.951+0000 7f993b7fe640 1 --2- 172.21.15.37:0/3391084390 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f99340a2da0 0x7f99340a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.951+0000 7f9949d9b640 1 --2- 172.21.15.37:0/3391084390 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f99340a2da0 0x7f99340a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.952+0000 7f9949d9b640 1 --2- 172.21.15.37:0/3391084390 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f99340a2da0 0x7f99340a5260 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f994002a930 tx=0x7f9940030570 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.957+0000 7f9950d8b640 1 -- 172.21.15.37:0/3391084390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f994c1a21c0 con 0x7f994c101a30 2024-06-10T01:00:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:09.957+0000 7f993b7fe640 1 -- 172.21.15.37:0/3391084390 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f993c099a60 con 0x7f994c101a30 2024-06-10T01:00:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.127+0000 7f9953b70740 1 Processor -- start 2024-06-10T01:00:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.127+0000 7f9953b70740 1 -- start start 2024-06-10T01:00:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.127+0000 7f9953b70740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f06838010 0x557f06838430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.127+0000 7f9953b70740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06838a00 0x557f0683d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.127+0000 7f9953b70740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f0683db70 0x557f06840000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.127+0000 7f9953b70740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557f0679b4f0 con 0x557f06838a00 2024-06-10T01:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.127+0000 7f9953b70740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557f06739870 con 0x557f0683db70 2024-06-10T01:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.127+0000 7f9953b70740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557f067974b0 con 0x557f06838010 2024-06-10T01:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.128+0000 7f9949d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06838a00 0x557f0683d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.128+0000 7f9949d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06838a00 0x557f0683d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36722/0 (socket says 172.21.15.37:36722) 2024-06-10T01:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.128+0000 7f9949d9b640 1 -- 172.21.15.37:0/82813290 learned_addr learned my addr 172.21.15.37:0/82813290 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.128+0000 7f994a59c640 1 --2- 172.21.15.37:0/82813290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f06838010 0x557f06838430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.128+0000 7f994ad9d640 1 --2- 172.21.15.37:0/82813290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f0683db70 0x557f06840000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.128+0000 7f9949d9b640 1 -- 172.21.15.37:0/82813290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f06838010 msgr2=0x557f06838430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.128+0000 7f9949d9b640 1 --2- 172.21.15.37:0/82813290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f06838010 0x557f06838430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.128+0000 7f9949d9b640 1 -- 172.21.15.37:0/82813290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f0683db70 msgr2=0x557f06840000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.128+0000 7f9949d9b640 1 --2- 172.21.15.37:0/82813290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f0683db70 0x557f06840000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.128+0000 7f9949d9b640 1 -- 172.21.15.37:0/82813290 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f067559b0 con 0x557f06838a00 2024-06-10T01:00:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.128+0000 7f994ad9d640 1 --2- 172.21.15.37:0/82813290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f0683db70 0x557f06840000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:00:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.128+0000 7f9949d9b640 1 --2- 172.21.15.37:0/82813290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06838a00 0x557f0683d500 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9940009860 tx=0x7f994003c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12748b1878de132a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.129+0000 7f9938ff9640 1 -- 172.21.15.37:0/82813290 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9940037040 con 0x557f06838a00 2024-06-10T01:00:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.129+0000 7f9938ff9640 1 -- 172.21.15.37:0/82813290 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f994004d070 con 0x557f06838a00 2024-06-10T01:00:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.129+0000 7f9938ff9640 1 -- 172.21.15.37:0/82813290 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9940005570 con 0x557f06838a00 2024-06-10T01:00:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.130+0000 7f9953b70740 1 -- 172.21.15.37:0/82813290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06838a00 msgr2=0x557f0683d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.130+0000 7f9953b70740 1 --2- 172.21.15.37:0/82813290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06838a00 0x557f0683d500 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9940009860 tx=0x7f994003c000 comp rx=0 tx=0).stop 2024-06-10T01:00:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.130+0000 7f9953b70740 1 -- 172.21.15.37:0/82813290 shutdown_connections 2024-06-10T01:00:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.130+0000 7f9953b70740 1 --2- 172.21.15.37:0/82813290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f0683db70 0x557f06840000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.130+0000 7f9953b70740 1 --2- 172.21.15.37:0/82813290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06838a00 0x557f0683d500 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.130+0000 7f9953b70740 1 --2- 172.21.15.37:0/82813290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f06838010 0x557f06838430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.130+0000 7f9953b70740 1 -- 172.21.15.37:0/82813290 >> 172.21.15.37:0/82813290 conn(0x557f06833680 msgr2=0x557f06833a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.130+0000 7f9953b70740 1 -- 172.21.15.37:0/82813290 shutdown_connections 2024-06-10T01:00:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.130+0000 7f9953b70740 1 -- 172.21.15.37:0/82813290 wait complete. 2024-06-10T01:00:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.153+0000 7f9953b70740 1 -- start start 2024-06-10T01:00:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.167+0000 7f9953b70740 1 Processor -- start 2024-06-10T01:00:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.167+0000 7f9953b70740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f06838010 0x557f068421f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.167+0000 7f9953b70740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f06838a00 0x557f06842730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.167+0000 7f9953b70740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f0683db70 0x557f06849770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.167+0000 7f9953b70740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557f06840ac0 con 0x557f0683db70 2024-06-10T01:00:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.167+0000 7f9953b70740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557f06724ae0 con 0x557f06838010 2024-06-10T01:00:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.167+0000 7f9953b70740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557f06840c40 con 0x557f06838a00 2024-06-10T01:00:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.168+0000 7f994ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f0683db70 0x557f06849770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.168+0000 7f9949d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f06838a00 0x557f06842730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.168+0000 7f9949d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f06838a00 0x557f06842730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:37522/0 (socket says 172.21.15.37:37522) 2024-06-10T01:00:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.168+0000 7f994a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f06838010 0x557f068421f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.168+0000 7f9949d9b640 1 -- 172.21.15.37:0/4020159882 learned_addr learned my addr 172.21.15.37:0/4020159882 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:00:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.168+0000 7f994a59c640 1 -- 172.21.15.37:0/4020159882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f06838a00 msgr2=0x557f06842730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.168+0000 7f994a59c640 1 --2- 172.21.15.37:0/4020159882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f06838a00 0x557f06842730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.168+0000 7f994a59c640 1 -- 172.21.15.37:0/4020159882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f0683db70 msgr2=0x557f06849770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.168+0000 7f994a59c640 1 --2- 172.21.15.37:0/4020159882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f0683db70 0x557f06849770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.168+0000 7f994a59c640 1 -- 172.21.15.37:0/4020159882 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f064af210 con 0x557f06838010 2024-06-10T01:00:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.169+0000 7f994ad9d640 1 --2- 172.21.15.37:0/4020159882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f0683db70 0x557f06849770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:00:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.169+0000 7f994a59c640 1 --2- 172.21.15.37:0/4020159882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f06838010 0x557f068421f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f994c0754c0 tx=0x7f993c09b5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ccce9f0aef45f0c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.169+0000 7f9931ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:00:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.169+0000 7f9931ffb640 1 -- 172.21.15.37:0/4020159882 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c09fd70 con 0x557f06838010 2024-06-10T01:00:10.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.169+0000 7f9931ffb640 1 -- 172.21.15.37:0/4020159882 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f993c0a41a0 con 0x557f06838010 2024-06-10T01:00:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.175+0000 7f9953b70740 10 client.0 _ll_register_callbacks cb 0x557f0675d9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:00:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.177+0000 7f9953b70740 1 -- 172.21.15.37:0/4020159882 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f994004d070 con 0x557f06838010 2024-06-10T01:00:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.177+0000 7f9931ffb640 1 -- 172.21.15.37:0/4020159882 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f993c0a4630 con 0x557f06838010 2024-06-10T01:00:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.177+0000 7f9953b70740 10 client.25547 fetch_fsmap learned FSMap version 27 2024-06-10T01:00:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.177+0000 7f9953b70740 1 -- 172.21.15.37:0/4020159882 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({fsmap=27}) -- 0x557f0657b620 con 0x557f06838010 2024-06-10T01:00:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.179+0000 7f9931ffb640 1 -- 172.21.15.37:0/4020159882 <== mon.2 v2:172.21.15.94:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f993c0b4bc0 con 0x557f06838010 2024-06-10T01:00:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.179+0000 7f9953b70740 10 client.25547 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:00:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.179+0000 7f9953b70740 10 client.25547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:00:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.179+0000 7f9953b70740 20 client.25547 populate_metadata read hostname 'smithi037' 2024-06-10T01:00:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.179+0000 7f9953b70740 1 --2- 172.21.15.37:0/4020159882 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557f0684ddf0 0x557f06850280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.179+0000 7f9953b70740 4 client.25547 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:00:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.179+0000 7f9953b70740 1 -- 172.21.15.37:0/4020159882 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f067aef10 con 0x557f0684ddf0 2024-06-10T01:00:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.179+0000 7f9949d9b640 1 --2- 172.21.15.37:0/4020159882 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557f0684ddf0 0x557f06850280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.180+0000 7f9949d9b640 1 --2- 172.21.15.37:0/4020159882 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557f0684ddf0 0x557f06850280 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f0f9809d5516f2e server_cookie=f5d44a4231614d05 in_seq=0 out_seq=0 2024-06-10T01:00:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.180+0000 7f9931ffb640 10 client.25547 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:00:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.180+0000 7f9931ffb640 1 -- 172.21.15.37:0/4020159882 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557f067aef10 con 0x557f0684ddf0 2024-06-10T01:00:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.180+0000 7f9931ffb640 10 client.25547 handle_command_reply: tid=0 2024-06-10T01:00:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.181+0000 7f9953b70740 1 client.25547 shutdown 2024-06-10T01:00:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.181+0000 7f9953b70740 1 -- 172.21.15.37:0/4020159882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f06838010 msgr2=0x557f068421f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.181+0000 7f9953b70740 1 --2- 172.21.15.37:0/4020159882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f06838010 0x557f068421f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f994c0754c0 tx=0x7f993c09b5c0 comp rx=0 tx=0).stop 2024-06-10T01:00:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.181+0000 7f9953b70740 1 -- 172.21.15.37:0/4020159882 shutdown_connections 2024-06-10T01:00:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.181+0000 7f9953b70740 1 --2- 172.21.15.37:0/4020159882 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557f0684ddf0 0x557f06850280 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.181+0000 7f9953b70740 1 --2- 172.21.15.37:0/4020159882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f0683db70 0x557f06849770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.181+0000 7f9931ffb640 0 client.25547 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:00:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.181+0000 7f9953b70740 1 --2- 172.21.15.37:0/4020159882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f06838a00 0x557f06842730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.181+0000 7f9953b70740 1 --2- 172.21.15.37:0/4020159882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f06838010 0x557f068421f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.181+0000 7f9953b70740 1 -- 172.21.15.37:0/4020159882 >> 172.21.15.37:0/4020159882 conn(0x557f06833680 msgr2=0x557f068327e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.181+0000 7f9953b70740 1 -- 172.21.15.37:0/4020159882 shutdown_connections 2024-06-10T01:00:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.181+0000 7f9953b70740 1 -- 172.21.15.37:0/4020159882 wait complete. 2024-06-10T01:00:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.183+0000 7f9953b70740 20 client.25547 trim_cache size 0 max 16384 2024-06-10T01:00:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.203+0000 7f9953b70740 1 Processor -- start 2024-06-10T01:00:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.203+0000 7f9953b70740 1 -- start start 2024-06-10T01:00:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.203+0000 7f9953b70740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f06838010 0x557f06841e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.203+0000 7f9953b70740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06838a00 0x557f06849700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.203+0000 7f9953b70740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f0683db70 0x557f06842340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.203+0000 7f9953b70740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1af1c0 con 0x557f06838a00 2024-06-10T01:00:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.203+0000 7f9953b70740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1aca90 con 0x557f06838010 2024-06-10T01:00:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.203+0000 7f9953b70740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1aa8e0 con 0x557f0683db70 2024-06-10T01:00:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.203+0000 7f9949d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06838a00 0x557f06849700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.203+0000 7f9949d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06838a00 0x557f06849700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36738/0 (socket says 172.21.15.37:36738) 2024-06-10T01:00:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.203+0000 7f9949d9b640 1 -- 172.21.15.37:0/3489488070 learned_addr learned my addr 172.21.15.37:0/3489488070 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:00:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.204+0000 7f994a59c640 1 --2- 172.21.15.37:0/3489488070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f06838010 0x557f06841e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.204+0000 7f994ad9d640 1 --2- 172.21.15.37:0/3489488070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f0683db70 0x557f06842340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.204+0000 7f9949d9b640 1 -- 172.21.15.37:0/3489488070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f0683db70 msgr2=0x557f06842340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.204+0000 7f9949d9b640 1 --2- 172.21.15.37:0/3489488070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f0683db70 0x557f06842340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.204+0000 7f9949d9b640 1 -- 172.21.15.37:0/3489488070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f06838010 msgr2=0x557f06841e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.204+0000 7f9949d9b640 1 --2- 172.21.15.37:0/3489488070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f06838010 0x557f06841e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.204+0000 7f9949d9b640 1 -- 172.21.15.37:0/3489488070 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f06834350 con 0x557f06838a00 2024-06-10T01:00:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.204+0000 7f994a59c640 1 --2- 172.21.15.37:0/3489488070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f06838010 0x557f06841e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:00:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.204+0000 7f9949d9b640 1 --2- 172.21.15.37:0/3489488070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06838a00 0x557f06849700 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f9940009860 tx=0x7f994002a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec034e1869eee0cf server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.204+0000 7f9933fff640 1 -- 172.21.15.37:0/3489488070 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9940037040 con 0x557f06838a00 2024-06-10T01:00:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.204+0000 7f9933fff640 1 -- 172.21.15.37:0/3489488070 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99400046c0 con 0x557f06838a00 2024-06-10T01:00:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.204+0000 7f9933fff640 1 -- 172.21.15.37:0/3489488070 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99400486c0 con 0x557f06838a00 2024-06-10T01:00:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.205+0000 7f9953b70740 1 -- 172.21.15.37:0/3489488070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06838a00 msgr2=0x557f06849700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.205+0000 7f9953b70740 1 --2- 172.21.15.37:0/3489488070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06838a00 0x557f06849700 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f9940009860 tx=0x7f994002a340 comp rx=0 tx=0).stop 2024-06-10T01:00:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.205+0000 7f9953b70740 1 -- 172.21.15.37:0/3489488070 shutdown_connections 2024-06-10T01:00:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.205+0000 7f9953b70740 1 --2- 172.21.15.37:0/3489488070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f0683db70 0x557f06842340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.205+0000 7f9953b70740 1 --2- 172.21.15.37:0/3489488070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f06838a00 0x557f06849700 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.205+0000 7f9953b70740 1 --2- 172.21.15.37:0/3489488070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f06838010 0x557f06841e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.205+0000 7f9953b70740 1 -- 172.21.15.37:0/3489488070 >> 172.21.15.37:0/3489488070 conn(0x557f06833680 msgr2=0x557f06840fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.205+0000 7f9953b70740 1 -- 172.21.15.37:0/3489488070 shutdown_connections 2024-06-10T01:00:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.205+0000 7f9953b70740 1 -- 172.21.15.37:0/3489488070 wait complete. 2024-06-10T01:00:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.205+0000 7f9953b70740 1 -- start start 2024-06-10T01:00:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.206+0000 7f9953b70740 1 Processor -- start 2024-06-10T01:00:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.206+0000 7f9953b70740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f06838010 0x557f06846860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.206+0000 7f9953b70740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f06838a00 0x557f06846da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.206+0000 7f9953b70740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f0683db70 0x557f06847db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.206+0000 7f9953b70740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1aaf50 con 0x557f0683db70 2024-06-10T01:00:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.206+0000 7f9953b70740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1ac3c0 con 0x557f06838a00 2024-06-10T01:00:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.206+0000 7f9953b70740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1ab620 con 0x557f06838010 2024-06-10T01:00:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.206+0000 7f9953b70740 10 client.0 _ll_register_callbacks cb 0x7f993c097c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:00:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.206+0000 7f994ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f0683db70 0x557f06847db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.206+0000 7f994a59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f06838010 0x557f06846860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.206+0000 7f994ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f0683db70 0x557f06847db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36754/0 (socket says 172.21.15.37:36754) 2024-06-10T01:00:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.206+0000 7f994ad9d640 1 -- 172.21.15.37:0/2445634716 learned_addr learned my addr 172.21.15.37:0/2445634716 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:00:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.206+0000 7f9949d9b640 1 --2- 172.21.15.37:0/2445634716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f06838a00 0x557f06846da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.206+0000 7f994ad9d640 1 -- 172.21.15.37:0/2445634716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f06838010 msgr2=0x557f06846860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.206+0000 7f994ad9d640 1 --2- 172.21.15.37:0/2445634716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f06838010 0x557f06846860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.206+0000 7f994ad9d640 1 -- 172.21.15.37:0/2445634716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f06838a00 msgr2=0x557f06846da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.206+0000 7f994ad9d640 1 --2- 172.21.15.37:0/2445634716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f06838a00 0x557f06846da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.206+0000 7f994ad9d640 1 -- 172.21.15.37:0/2445634716 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f068483a0 con 0x557f0683db70 2024-06-10T01:00:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.206+0000 7f9949d9b640 1 --2- 172.21.15.37:0/2445634716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f06838a00 0x557f06846da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:00:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.207+0000 7f994ad9d640 1 --2- 172.21.15.37:0/2445634716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f0683db70 0x557f06847db0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f994400f670 tx=0x7f994400fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1248c3a985289fe7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.207+0000 7f9931ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:00:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.207+0000 7f9931ffb640 1 -- 172.21.15.37:0/2445634716 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9944003b70 con 0x557f0683db70 2024-06-10T01:00:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.207+0000 7f9931ffb640 1 -- 172.21.15.37:0/2445634716 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f9944003d10 con 0x557f0683db70 2024-06-10T01:00:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.207+0000 7f9953b70740 1 -- 172.21.15.37:0/2445634716 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99400046c0 con 0x557f0683db70 2024-06-10T01:00:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.207+0000 7f9931ffb640 1 -- 172.21.15.37:0/2445634716 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f994402b050 con 0x557f0683db70 2024-06-10T01:00:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.207+0000 7f9953b70740 10 client.16023 fetch_fsmap learned FSMap version 27 2024-06-10T01:00:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.207+0000 7f9953b70740 1 -- 172.21.15.37:0/2445634716 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557f06849720 con 0x557f0683db70 2024-06-10T01:00:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.207+0000 7f9931ffb640 1 -- 172.21.15.37:0/2445634716 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99400046c0 con 0x557f0683db70 2024-06-10T01:00:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.207+0000 7f9931ffb640 1 -- 172.21.15.37:0/2445634716 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f994400e9a0 con 0x557f0683db70 2024-06-10T01:00:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.208+0000 7f9953b70740 10 client.16023 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:00:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.208+0000 7f9953b70740 10 client.16023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:00:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.208+0000 7f9953b70740 20 client.16023 populate_metadata read hostname 'smithi037' 2024-06-10T01:00:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.208+0000 7f9953b70740 1 --2- 172.21.15.37:0/2445634716 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557f068b9060 0x557f068bb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.208+0000 7f9953b70740 4 client.16023 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:00:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.208+0000 7f9953b70740 1 -- 172.21.15.37:0/2445634716 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557f068396f0 con 0x557f068b9060 2024-06-10T01:00:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.208+0000 7f994a59c640 1 --2- 172.21.15.37:0/2445634716 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557f068b9060 0x557f068bb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.208+0000 7f9931ffb640 10 client.16023 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:00:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.208+0000 7f994a59c640 1 --2- 172.21.15.37:0/2445634716 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557f068b9060 0x557f068bb480 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cb2791f1f9280b2 server_cookie=71a773b78aa0165e in_seq=0 out_seq=0 2024-06-10T01:00:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.223+0000 7f9931ffb640 1 -- 172.21.15.37:0/2445634716 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557f068396f0 con 0x557f068b9060 2024-06-10T01:00:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.223+0000 7f9931ffb640 10 client.16023 handle_command_reply: tid=0 2024-06-10T01:00:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.224+0000 7f9953b70740 1 client.16023 shutdown 2024-06-10T01:00:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.224+0000 7f9953b70740 1 -- 172.21.15.37:0/2445634716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f0683db70 msgr2=0x557f06847db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.224+0000 7f9953b70740 1 --2- 172.21.15.37:0/2445634716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f0683db70 0x557f06847db0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f994400f670 tx=0x7f994400fb30 comp rx=0 tx=0).stop 2024-06-10T01:00:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.224+0000 7f9953b70740 1 -- 172.21.15.37:0/2445634716 shutdown_connections 2024-06-10T01:00:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.224+0000 7f9953b70740 1 --2- 172.21.15.37:0/2445634716 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557f068b9060 0x557f068bb480 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.224+0000 7f9953b70740 1 --2- 172.21.15.37:0/2445634716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f0683db70 0x557f06847db0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.224+0000 7f9953b70740 1 --2- 172.21.15.37:0/2445634716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557f06838a00 0x557f06846da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.224+0000 7f9931ffb640 0 client.16023 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:00:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.224+0000 7f9953b70740 1 --2- 172.21.15.37:0/2445634716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557f06838010 0x557f06846860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.224+0000 7f9953b70740 1 -- 172.21.15.37:0/2445634716 >> 172.21.15.37:0/2445634716 conn(0x557f06833680 msgr2=0x557f0683ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.224+0000 7f9953b70740 1 -- 172.21.15.37:0/2445634716 shutdown_connections 2024-06-10T01:00:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.224+0000 7f9953b70740 1 -- 172.21.15.37:0/2445634716 wait complete. 2024-06-10T01:00:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.224+0000 7f9953b70740 20 client.16023 trim_cache size 0 max 16384 2024-06-10T01:00:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:00:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:00:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:00:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.225+0000 7f9950d8b640 1 -- 172.21.15.37:0/3391084390 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f99340a2da0 msgr2=0x7f99340a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.225+0000 7f9950d8b640 1 --2- 172.21.15.37:0/3391084390 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f99340a2da0 0x7f99340a5260 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f994002a930 tx=0x7f9940030570 comp rx=0 tx=0).stop 2024-06-10T01:00:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.225+0000 7f9950d8b640 1 -- 172.21.15.37:0/3391084390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f994c101a30 msgr2=0x7f994c075750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.225+0000 7f9950d8b640 1 --2- 172.21.15.37:0/3391084390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f994c101a30 0x7f994c075750 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f993c00e9a0 tx=0x7f993c00ee60 comp rx=0 tx=0).stop 2024-06-10T01:00:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.225+0000 7f9950d8b640 1 -- 172.21.15.37:0/3391084390 shutdown_connections 2024-06-10T01:00:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.225+0000 7f9950d8b640 1 --2- 172.21.15.37:0/3391084390 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f99340a2da0 0x7f99340a5260 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.225+0000 7f9950d8b640 1 --2- 172.21.15.37:0/3391084390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f994c1035e0 0x7f994c0742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.225+0000 7f9950d8b640 1 --2- 172.21.15.37:0/3391084390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f994c102c20 0x7f994c073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.225+0000 7f9950d8b640 1 --2- 172.21.15.37:0/3391084390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f994c101a30 0x7f994c075750 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.225+0000 7f9950d8b640 1 -- 172.21.15.37:0/3391084390 >> 172.21.15.37:0/3391084390 conn(0x7f994c0fcfc0 msgr2=0x7f994c105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.225+0000 7f9950d8b640 1 -- 172.21.15.37:0/3391084390 shutdown_connections 2024-06-10T01:00:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:10.225+0000 7f9950d8b640 1 -- 172.21.15.37:0/3391084390 wait complete. 2024-06-10T01:00:10.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:10 smithi167 ceph-mon[36917]: pgmap v534: 129 pgs: 129 active+clean; 1.4 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 8 op/s 2024-06-10T01:00:10.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:10 smithi094 ceph-mon[36708]: pgmap v534: 129 pgs: 129 active+clean; 1.4 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 8 op/s 2024-06-10T01:00:10.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:10 smithi037 ceph-mon[32192]: pgmap v534: 129 pgs: 129 active+clean; 1.4 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 8 op/s 2024-06-10T01:00:10.993 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:10.993 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:11.273 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:11.273 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:11.288 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:11.288 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:11.288 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 49 -lt 100 ']' 2024-06-10T01:00:11.289 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 49 + 1 2024-06-10T01:00:11.506 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=50 2024-06-10T01:00:11.506 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:11.585 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:11.585 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:11.587 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:11.587 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:11.587 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:12.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:12 smithi167 ceph-mon[36917]: pgmap v535: 129 pgs: 129 active+clean; 1.4 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 8 op/s 2024-06-10T01:00:12.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:12 smithi094 ceph-mon[36708]: pgmap v535: 129 pgs: 129 active+clean; 1.4 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 8 op/s 2024-06-10T01:00:12.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:12 smithi037 ceph-mon[32192]: pgmap v535: 129 pgs: 129 active+clean; 1.4 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 8 op/s 2024-06-10T01:00:12.934 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:12.934 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:13.282 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:13.282 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:13.322 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:13.322 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:13.322 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 50 -lt 100 ']' 2024-06-10T01:00:13.322 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 50 + 1 2024-06-10T01:00:13.392 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=51 2024-06-10T01:00:13.392 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:13.431 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:13.432 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:13.433 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:13.433 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:13.433 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:14.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:14 smithi167 ceph-mon[36917]: pgmap v536: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-06-10T01:00:14.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:14 smithi094 ceph-mon[36708]: pgmap v536: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-06-10T01:00:14.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:14 smithi037 ceph-mon[32192]: pgmap v536: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-06-10T01:00:14.409 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:14.409 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:14.740 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:14.740 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:14.796 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:14.797 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:14.797 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 51 -lt 100 ']' 2024-06-10T01:00:14.797 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 51 + 1 2024-06-10T01:00:14.841 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=52 2024-06-10T01:00:14.841 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:14.900 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:14.901 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:14.902 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:14.902 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:14.902 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:16.092 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:16.092 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:16.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:16 smithi167 ceph-mon[36917]: pgmap v537: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.8 KiB/s wr, 11 op/s 2024-06-10T01:00:16.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:16 smithi094 ceph-mon[36708]: pgmap v537: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.8 KiB/s wr, 11 op/s 2024-06-10T01:00:16.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:16 smithi037 ceph-mon[32192]: pgmap v537: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.8 KiB/s wr, 11 op/s 2024-06-10T01:00:16.606 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:16.606 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:16.621 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:16.621 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:16.621 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 52 -lt 100 ']' 2024-06-10T01:00:16.621 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 52 + 1 2024-06-10T01:00:16.706 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=53 2024-06-10T01:00:16.706 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:16.759 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:16.759 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:16.761 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:16.761 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:16.761 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:18.009 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:18.009 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:18.281 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:18.281 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:18.296 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:18.296 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:18.296 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 53 -lt 100 ']' 2024-06-10T01:00:18.297 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 53 + 1 2024-06-10T01:00:18.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:18 smithi167 ceph-mon[36917]: pgmap v538: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-06-10T01:00:18.357 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=54 2024-06-10T01:00:18.357 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:18.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:18 smithi094 ceph-mon[36708]: pgmap v538: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-06-10T01:00:18.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:18 smithi037 ceph-mon[32192]: pgmap v538: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-06-10T01:00:18.407 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:18.408 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:18.409 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:18.409 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:18.409 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:20.092 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:20.092 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:20.223 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:20 smithi037 ceph-mon[32192]: pgmap v539: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.0 KiB/s wr, 12 op/s 2024-06-10T01:00:20.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:20 smithi167 ceph-mon[36917]: pgmap v539: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.0 KiB/s wr, 12 op/s 2024-06-10T01:00:20.338 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:20.338 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:20.353 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:20.353 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:20.353 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 54 -lt 100 ']' 2024-06-10T01:00:20.353 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 54 + 1 2024-06-10T01:00:20.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:20 smithi094 ceph-mon[36708]: pgmap v539: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.0 KiB/s wr, 12 op/s 2024-06-10T01:00:20.415 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=55 2024-06-10T01:00:20.415 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:20.440 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:20.440 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:20.442 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:20.442 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:20.442 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:21.733 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:21.733 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:22.121 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:22.121 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:22.137 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:22.138 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:22.138 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 55 -lt 100 ']' 2024-06-10T01:00:22.138 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 55 + 1 2024-06-10T01:00:22.172 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=56 2024-06-10T01:00:22.172 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:22.224 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:22.224 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:22.225 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:22.226 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:22.226 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:22.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:22 smithi167 ceph-mon[36917]: pgmap v540: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.8 KiB/s wr, 11 op/s 2024-06-10T01:00:22.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:22 smithi094 ceph-mon[36708]: pgmap v540: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.8 KiB/s wr, 11 op/s 2024-06-10T01:00:22.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:22 smithi037 ceph-mon[32192]: pgmap v540: 129 pgs: 129 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.8 KiB/s wr, 11 op/s 2024-06-10T01:00:23.291 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:23.291 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:23.821 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:23.821 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:23.837 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:23.837 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:23.837 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 56 -lt 100 ']' 2024-06-10T01:00:23.837 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 56 + 1 2024-06-10T01:00:23.906 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=57 2024-06-10T01:00:23.906 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:23.957 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:23.957 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:23.959 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:23.959 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:23.959 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:24.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:24 smithi167 ceph-mon[36917]: pgmap v541: 129 pgs: 129 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-06-10T01:00:24.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:24 smithi094 ceph-mon[36708]: pgmap v541: 129 pgs: 129 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-06-10T01:00:24.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:24 smithi037 ceph-mon[32192]: pgmap v541: 129 pgs: 129 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-06-10T01:00:25.524 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:25.524 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.623+0000 7f070152f700 1 -- 172.21.15.37:0/550149794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06fc108500 msgr2=0x7f06fc1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.623+0000 7f070152f700 1 --2- 172.21.15.37:0/550149794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06fc108500 0x7f06fc1088e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00b0a0 tx=0x7f06ec00b3b0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.623+0000 7f070152f700 1 -- 172.21.15.37:0/550149794 shutdown_connections 2024-06-10T01:00:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.623+0000 7f070152f700 1 --2- 172.21.15.37:0/550149794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06fc06ab30 0x7f06fc10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.623+0000 7f070152f700 1 --2- 172.21.15.37:0/550149794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06fc108500 0x7f06fc1088e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.623+0000 7f070152f700 1 --2- 172.21.15.37:0/550149794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06fc06a170 0x7f06fc06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.623+0000 7f070152f700 1 -- 172.21.15.37:0/550149794 >> 172.21.15.37:0/550149794 conn(0x7f06fc078340 msgr2=0x7f06fc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.623+0000 7f070152f700 1 -- 172.21.15.37:0/550149794 shutdown_connections 2024-06-10T01:00:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.623+0000 7f070152f700 1 -- 172.21.15.37:0/550149794 wait complete. 2024-06-10T01:00:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.624+0000 7f070152f700 1 Processor -- start 2024-06-10T01:00:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.624+0000 7f070152f700 1 -- start start 2024-06-10T01:00:25.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.624+0000 7f070152f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06fc06a170 0x7f06fc101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:25.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.624+0000 7f070152f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06fc06ab30 0x7f06fc1019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:25.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.624+0000 7f070152f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06fc108500 0x7f06fc101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:25.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.624+0000 7f070152f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc112780 con 0x7f06fc108500 2024-06-10T01:00:25.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.624+0000 7f070152f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc112600 con 0x7f06fc06a170 2024-06-10T01:00:25.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.624+0000 7f070152f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc112900 con 0x7f06fc06ab30 2024-06-10T01:00:25.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.624+0000 7f06fb7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06fc108500 0x7f06fc101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:25.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.624+0000 7f06faffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06fc06a170 0x7f06fc101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:25.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.624+0000 7f06fb7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06fc108500 0x7f06fc101ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38492/0 (socket says 172.21.15.37:38492) 2024-06-10T01:00:25.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.625+0000 7f06fb7fe700 1 -- 172.21.15.37:0/1658773600 learned_addr learned my addr 172.21.15.37:0/1658773600 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:00:25.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.625+0000 7f06fa7fc700 1 --2- 172.21.15.37:0/1658773600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06fc06ab30 0x7f06fc1019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:25.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.625+0000 7f06faffd700 1 -- 172.21.15.37:0/1658773600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06fc06ab30 msgr2=0x7f06fc1019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:25.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.625+0000 7f06faffd700 1 --2- 172.21.15.37:0/1658773600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06fc06ab30 0x7f06fc1019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.625+0000 7f06faffd700 1 -- 172.21.15.37:0/1658773600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06fc108500 msgr2=0x7f06fc101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:25.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.625+0000 7f06faffd700 1 --2- 172.21.15.37:0/1658773600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06fc108500 0x7f06fc101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.625+0000 7f06faffd700 1 -- 172.21.15.37:0/1658773600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06fc1a7170 con 0x7f06fc06a170 2024-06-10T01:00:25.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.625+0000 7f06fb7fe700 1 --2- 172.21.15.37:0/1658773600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06fc108500 0x7f06fc101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:00:25.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.625+0000 7f06faffd700 1 --2- 172.21.15.37:0/1658773600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06fc06a170 0x7f06fc101460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f06ec000f80 tx=0x7f06ec004d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:25.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.625+0000 7f06e3fff700 1 -- 172.21.15.37:0/1658773600 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec021070 con 0x7f06fc06a170 2024-06-10T01:00:25.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.625+0000 7f070152f700 1 -- 172.21.15.37:0/1658773600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06fc1a7400 con 0x7f06fc06a170 2024-06-10T01:00:25.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.625+0000 7f06e3fff700 1 -- 172.21.15.37:0/1658773600 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06ec007b80 con 0x7f06fc06a170 2024-06-10T01:00:25.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.626+0000 7f06e3fff700 1 -- 172.21.15.37:0/1658773600 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec0052a0 con 0x7f06fc06a170 2024-06-10T01:00:25.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.626+0000 7f070152f700 1 -- 172.21.15.37:0/1658773600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06fc1a78b0 con 0x7f06fc06a170 2024-06-10T01:00:25.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.628+0000 7f06e3fff700 1 -- 172.21.15.37:0/1658773600 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f06ec024070 con 0x7f06fc06a170 2024-06-10T01:00:25.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.629+0000 7f06e3fff700 1 --2- 172.21.15.37:0/1658773600 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f06e40a2c20 0x7f06e40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:25.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.629+0000 7f06fa7fc700 1 --2- 172.21.15.37:0/1658773600 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f06e40a2c20 0x7f06e40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:25.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.629+0000 7f06e3fff700 1 -- 172.21.15.37:0/1658773600 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f06ec00f020 con 0x7f06fc06a170 2024-06-10T01:00:25.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.629+0000 7f06fa7fc700 1 --2- 172.21.15.37:0/1658773600 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f06e40a2c20 0x7f06e40a50e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f06fc0789b0 tx=0x7f06f000b3c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:25.878 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:25.878 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:25.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.901+0000 7f070bdb2b80 1 Processor -- start 2024-06-10T01:00:25.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.901+0000 7f070bdb2b80 1 -- start start 2024-06-10T01:00:25.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.902+0000 7f070bdb2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee1bfe6c0 0x563ee1bfeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:25.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.902+0000 7f070bdb2b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ee1bfefe0 0x563ee1bf8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.902+0000 7f070bdb2b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ee1bf8fa0 0x563ee1bf93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.902+0000 7f070bdb2b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563ee1b4a890 con 0x563ee1bf8fa0 2024-06-10T01:00:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.902+0000 7f070bdb2b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563ee1af27a0 con 0x563ee1bfefe0 2024-06-10T01:00:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.902+0000 7f070bdb2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563ee1b46850 con 0x563ee1bfe6c0 2024-06-10T01:00:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.902+0000 7f06fb7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ee1bf8fa0 0x563ee1bf93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.902+0000 7f06fb7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ee1bf8fa0 0x563ee1bf93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38498/0 (socket says 172.21.15.37:38498) 2024-06-10T01:00:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.902+0000 7f06fb7fe700 1 -- 172.21.15.37:0/641904291 learned_addr learned my addr 172.21.15.37:0/641904291 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:00:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.902+0000 7f06faffd700 1 --2- 172.21.15.37:0/641904291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee1bfe6c0 0x563ee1bfeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.902+0000 7f06fa7fc700 1 --2- 172.21.15.37:0/641904291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ee1bfefe0 0x563ee1bf8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.902+0000 7f06faffd700 1 -- 172.21.15.37:0/641904291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ee1bfefe0 msgr2=0x563ee1bf8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:25.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.902+0000 7f06faffd700 1 --2- 172.21.15.37:0/641904291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ee1bfefe0 0x563ee1bf8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.902+0000 7f06faffd700 1 -- 172.21.15.37:0/641904291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ee1bf8fa0 msgr2=0x563ee1bf93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:25.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.902+0000 7f06faffd700 1 --2- 172.21.15.37:0/641904291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ee1bf8fa0 0x563ee1bf93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.902+0000 7f06faffd700 1 -- 172.21.15.37:0/641904291 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ee1ae8fd0 con 0x563ee1bfe6c0 2024-06-10T01:00:25.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.902+0000 7f06fb7fe700 1 --2- 172.21.15.37:0/641904291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ee1bf8fa0 0x563ee1bf93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:00:25.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.903+0000 7f06faffd700 1 --2- 172.21.15.37:0/641904291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee1bfe6c0 0x563ee1bfeaa0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f06ec01afd0 tx=0x7f06ec01dcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ea76bd1889201cd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:25.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.903+0000 7f06e17fa700 1 -- 172.21.15.37:0/641904291 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec0a11d0 con 0x563ee1bfe6c0 2024-06-10T01:00:25.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.903+0000 7f06e17fa700 1 -- 172.21.15.37:0/641904291 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06ec0a5a00 con 0x563ee1bfe6c0 2024-06-10T01:00:25.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.903+0000 7f06e17fa700 1 -- 172.21.15.37:0/641904291 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec09d2e0 con 0x563ee1bfe6c0 2024-06-10T01:00:25.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.904+0000 7f070bdb2b80 1 -- 172.21.15.37:0/641904291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee1bfe6c0 msgr2=0x563ee1bfeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:25.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.904+0000 7f070bdb2b80 1 --2- 172.21.15.37:0/641904291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee1bfe6c0 0x563ee1bfeaa0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f06ec01afd0 tx=0x7f06ec01dcd0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.904+0000 7f070bdb2b80 1 -- 172.21.15.37:0/641904291 shutdown_connections 2024-06-10T01:00:25.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.904+0000 7f070bdb2b80 1 --2- 172.21.15.37:0/641904291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ee1bf8fa0 0x563ee1bf93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.904+0000 7f070bdb2b80 1 --2- 172.21.15.37:0/641904291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee1bfe6c0 0x563ee1bfeaa0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.904+0000 7f070bdb2b80 1 --2- 172.21.15.37:0/641904291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ee1bfefe0 0x563ee1bf8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.904+0000 7f070bdb2b80 1 -- 172.21.15.37:0/641904291 >> 172.21.15.37:0/641904291 conn(0x563ee1b23d40 msgr2=0x563ee1bf38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:25.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.904+0000 7f070bdb2b80 1 -- 172.21.15.37:0/641904291 shutdown_connections 2024-06-10T01:00:25.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.904+0000 7f070bdb2b80 1 -- 172.21.15.37:0/641904291 wait complete. 2024-06-10T01:00:25.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.904+0000 7f070bdb2b80 1 -- start start 2024-06-10T01:00:25.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.905+0000 7f070bdb2b80 1 Processor -- start 2024-06-10T01:00:25.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.905+0000 7f070bdb2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee1bf8fa0 0x563ee1c04d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:25.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.905+0000 7f070bdb2b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ee1bfe6c0 0x563ee1c02050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:25.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.905+0000 7f070bdb2b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ee1bfefe0 0x563ee1c02590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:25.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.905+0000 7f070bdb2b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563ee1c05770 con 0x563ee1bfe6c0 2024-06-10T01:00:25.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.905+0000 7f070bdb2b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563ee1aeac70 con 0x563ee1bfefe0 2024-06-10T01:00:25.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.905+0000 7f070bdb2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563ee1c058f0 con 0x563ee1bf8fa0 2024-06-10T01:00:25.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.905+0000 7f070bdb2b80 10 client.0 _ll_register_callbacks cb 0x563ee1ae49a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:00:25.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.905+0000 7f06fa7fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ee1bfe6c0 0x563ee1c02050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:25.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.905+0000 7f06fa7fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ee1bfe6c0 0x563ee1c02050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38506/0 (socket says 172.21.15.37:38506) 2024-06-10T01:00:25.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.905+0000 7f06fa7fc700 1 -- 172.21.15.37:0/2331732248 learned_addr learned my addr 172.21.15.37:0/2331732248 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:00:25.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.905+0000 7f06faffd700 1 --2- 172.21.15.37:0/2331732248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee1bf8fa0 0x563ee1c04d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:25.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.905+0000 7f06fb7fe700 1 --2- 172.21.15.37:0/2331732248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ee1bfefe0 0x563ee1c02590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:25.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.905+0000 7f06fa7fc700 1 -- 172.21.15.37:0/2331732248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee1bf8fa0 msgr2=0x563ee1c04d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:25.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.905+0000 7f06fa7fc700 1 --2- 172.21.15.37:0/2331732248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee1bf8fa0 0x563ee1c04d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.905+0000 7f06fa7fc700 1 -- 172.21.15.37:0/2331732248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ee1bfefe0 msgr2=0x563ee1c02590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:25.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.905+0000 7f06fa7fc700 1 --2- 172.21.15.37:0/2331732248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ee1bfefe0 0x563ee1c02590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.905+0000 7f06fa7fc700 1 -- 172.21.15.37:0/2331732248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ee1a99950 con 0x563ee1bfe6c0 2024-06-10T01:00:25.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.906+0000 7f06fa7fc700 1 --2- 172.21.15.37:0/2331732248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ee1bfe6c0 0x563ee1c02050 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f06f0003ec0 tx=0x7f06f00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ccb0ba559df23b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:25.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.906+0000 7f06deff5700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:00:25.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.906+0000 7f06deff5700 1 -- 172.21.15.37:0/2331732248 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f0011ce0 con 0x563ee1bfe6c0 2024-06-10T01:00:25.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.906+0000 7f06deff5700 1 -- 172.21.15.37:0/2331732248 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f06f0004830 con 0x563ee1bfe6c0 2024-06-10T01:00:25.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.906+0000 7f070bdb2b80 1 -- 172.21.15.37:0/2331732248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06ec0a5a00 con 0x563ee1bfe6c0 2024-06-10T01:00:25.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.906+0000 7f06deff5700 1 -- 172.21.15.37:0/2331732248 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f06f0030050 con 0x563ee1bfe6c0 2024-06-10T01:00:25.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.906+0000 7f070bdb2b80 10 client.16035 fetch_fsmap learned FSMap version 27 2024-06-10T01:00:25.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.906+0000 7f070bdb2b80 1 -- 172.21.15.37:0/2331732248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563ee1c100a0 con 0x563ee1bfe6c0 2024-06-10T01:00:25.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.906+0000 7f06deff5700 1 -- 172.21.15.37:0/2331732248 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06ec0a5a00 con 0x563ee1bfe6c0 2024-06-10T01:00:25.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.907+0000 7f06deff5700 1 -- 172.21.15.37:0/2331732248 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f06f001f430 con 0x563ee1bfe6c0 2024-06-10T01:00:25.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.907+0000 7f070bdb2b80 10 client.16035 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:00:25.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.907+0000 7f070bdb2b80 10 client.16035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:00:25.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.907+0000 7f070bdb2b80 20 client.16035 populate_metadata read hostname 'smithi037' 2024-06-10T01:00:25.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.907+0000 7f070bdb2b80 1 --2- 172.21.15.37:0/2331732248 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x563ee1c126e0 0x563ee1c14ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:25.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.907+0000 7f070bdb2b80 4 client.16035 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:00:25.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.907+0000 7f070bdb2b80 1 -- 172.21.15.37:0/2331732248 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ee1b57d90 con 0x563ee1c126e0 2024-06-10T01:00:25.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.907+0000 7f06faffd700 1 --2- 172.21.15.37:0/2331732248 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x563ee1c126e0 0x563ee1c14ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:25.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.907+0000 7f06faffd700 1 --2- 172.21.15.37:0/2331732248 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x563ee1c126e0 0x563ee1c14ad0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9842c1432c33cfbd server_cookie=a888be5230edb3c4 in_seq=0 out_seq=0 2024-06-10T01:00:25.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.907+0000 7f06deff5700 10 client.16035 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:00:25.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.908+0000 7f06deff5700 1 -- 172.21.15.37:0/2331732248 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563ee1b57d90 con 0x563ee1c126e0 2024-06-10T01:00:25.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.908+0000 7f06deff5700 10 client.16035 handle_command_reply: tid=0 2024-06-10T01:00:25.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.908+0000 7f070bdb2b80 1 client.16035 shutdown 2024-06-10T01:00:25.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.908+0000 7f070bdb2b80 1 -- 172.21.15.37:0/2331732248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ee1bfe6c0 msgr2=0x563ee1c02050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:25.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.908+0000 7f070bdb2b80 1 --2- 172.21.15.37:0/2331732248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ee1bfe6c0 0x563ee1c02050 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f06f0003ec0 tx=0x7f06f00047c0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.908+0000 7f070bdb2b80 1 -- 172.21.15.37:0/2331732248 shutdown_connections 2024-06-10T01:00:25.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.908+0000 7f070bdb2b80 1 --2- 172.21.15.37:0/2331732248 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x563ee1c126e0 0x563ee1c14ad0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.909+0000 7f070bdb2b80 1 --2- 172.21.15.37:0/2331732248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ee1bfefe0 0x563ee1c02590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.909+0000 7f070bdb2b80 1 --2- 172.21.15.37:0/2331732248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee1bf8fa0 0x563ee1c04d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.909+0000 7f070bdb2b80 1 --2- 172.21.15.37:0/2331732248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ee1bfe6c0 0x563ee1c02050 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.909+0000 7f070bdb2b80 1 -- 172.21.15.37:0/2331732248 >> 172.21.15.37:0/2331732248 conn(0x563ee1b23d40 msgr2=0x563ee1bf7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:25.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.909+0000 7f06deff5700 0 client.16035 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:00:25.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.909+0000 7f070bdb2b80 1 -- 172.21.15.37:0/2331732248 shutdown_connections 2024-06-10T01:00:25.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.909+0000 7f070bdb2b80 1 -- 172.21.15.37:0/2331732248 wait complete. 2024-06-10T01:00:25.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.909+0000 7f070bdb2b80 20 client.16035 trim_cache size 0 max 16384 2024-06-10T01:00:25.919 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:25.919 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:25.919 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 57 -lt 100 ']' 2024-06-10T01:00:25.919 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 57 + 1 2024-06-10T01:00:25.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.931+0000 7f070bdb2b80 1 Processor -- start 2024-06-10T01:00:25.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.931+0000 7f070bdb2b80 1 -- start start 2024-06-10T01:00:25.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.931+0000 7f070bdb2b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ee1bf8fa0 0x563ee1bf8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:25.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.931+0000 7f070bdb2b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ee1bfe6c0 0x563ee1bed710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:25.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.931+0000 7f070bdb2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee1bfefe0 0x563ee1c1a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:25.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.931+0000 7f070bdb2b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc1b2ef0 con 0x563ee1bfe6c0 2024-06-10T01:00:25.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.931+0000 7f070bdb2b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc1b0780 con 0x563ee1bf8fa0 2024-06-10T01:00:25.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.931+0000 7f070bdb2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc1ae5d0 con 0x563ee1bfefe0 2024-06-10T01:00:25.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.931+0000 7f06fa7fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ee1bfe6c0 0x563ee1bed710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:25.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.931+0000 7f06fa7fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ee1bfe6c0 0x563ee1bed710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38510/0 (socket says 172.21.15.37:38510) 2024-06-10T01:00:25.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.931+0000 7f06fa7fc700 1 -- 172.21.15.37:0/2027470701 learned_addr learned my addr 172.21.15.37:0/2027470701 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:00:25.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.931+0000 7f06faffd700 1 --2- 172.21.15.37:0/2027470701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ee1bf8fa0 0x563ee1bf8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:25.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.932+0000 7f06fa7fc700 1 -- 172.21.15.37:0/2027470701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee1bfefe0 msgr2=0x563ee1c1a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:25.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.932+0000 7f06fa7fc700 1 --2- 172.21.15.37:0/2027470701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee1bfefe0 0x563ee1c1a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.932+0000 7f06fa7fc700 1 -- 172.21.15.37:0/2027470701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ee1bf8fa0 msgr2=0x563ee1bf8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:25.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.932+0000 7f06fa7fc700 1 --2- 172.21.15.37:0/2027470701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ee1bf8fa0 0x563ee1bf8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.932+0000 7f06fa7fc700 1 -- 172.21.15.37:0/2027470701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ee1bf9900 con 0x563ee1bfe6c0 2024-06-10T01:00:25.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.932+0000 7f06fa7fc700 1 --2- 172.21.15.37:0/2027470701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ee1bfe6c0 0x563ee1bed710 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f06f001f880 tx=0x7f06f000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f270017365cd1b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:25.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.932+0000 7f06e0ff9700 1 -- 172.21.15.37:0/2027470701 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f0038070 con 0x563ee1bfe6c0 2024-06-10T01:00:25.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.932+0000 7f06e0ff9700 1 -- 172.21.15.37:0/2027470701 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f0020e50 con 0x563ee1bfe6c0 2024-06-10T01:00:25.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.932+0000 7f06e0ff9700 1 -- 172.21.15.37:0/2027470701 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f001fe60 con 0x563ee1bfe6c0 2024-06-10T01:00:25.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.933+0000 7f070bdb2b80 1 -- 172.21.15.37:0/2027470701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ee1bfe6c0 msgr2=0x563ee1bed710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:25.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.933+0000 7f070bdb2b80 1 --2- 172.21.15.37:0/2027470701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ee1bfe6c0 0x563ee1bed710 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f06f001f880 tx=0x7f06f000be50 comp rx=0 tx=0).stop 2024-06-10T01:00:25.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.933+0000 7f070bdb2b80 1 -- 172.21.15.37:0/2027470701 shutdown_connections 2024-06-10T01:00:25.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.933+0000 7f070bdb2b80 1 --2- 172.21.15.37:0/2027470701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee1bfefe0 0x563ee1c1a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.933+0000 7f070bdb2b80 1 --2- 172.21.15.37:0/2027470701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ee1bf8fa0 0x563ee1bf8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.933+0000 7f070bdb2b80 1 --2- 172.21.15.37:0/2027470701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ee1bfe6c0 0x563ee1bed710 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.933+0000 7f070bdb2b80 1 -- 172.21.15.37:0/2027470701 >> 172.21.15.37:0/2027470701 conn(0x563ee1b23d40 msgr2=0x563ee1bffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:25.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.933+0000 7f070bdb2b80 1 -- 172.21.15.37:0/2027470701 shutdown_connections 2024-06-10T01:00:25.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.933+0000 7f070bdb2b80 1 -- 172.21.15.37:0/2027470701 wait complete. 2024-06-10T01:00:25.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.933+0000 7f070bdb2b80 1 -- start start 2024-06-10T01:00:25.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.934+0000 7f070bdb2b80 1 Processor -- start 2024-06-10T01:00:25.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.934+0000 7f070bdb2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee1bf8fa0 0x563ee1c0b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:25.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.934+0000 7f070bdb2b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ee1bfe6c0 0x563ee1c07b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:25.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.934+0000 7f070bdb2b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ee1bfefe0 0x563ee1c08290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:25.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.934+0000 7f070bdb2b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc1aec40 con 0x563ee1bfefe0 2024-06-10T01:00:25.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.934+0000 7f070bdb2b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc1b00b0 con 0x563ee1bfe6c0 2024-06-10T01:00:25.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.934+0000 7f070bdb2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc1af310 con 0x563ee1bf8fa0 2024-06-10T01:00:25.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.934+0000 7f070bdb2b80 10 client.0 _ll_register_callbacks cb 0x563ee1ae49a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:00:25.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.934+0000 7f06fb7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ee1bfefe0 0x563ee1c08290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:25.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.934+0000 7f06fb7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ee1bfefe0 0x563ee1c08290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38518/0 (socket says 172.21.15.37:38518) 2024-06-10T01:00:25.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.934+0000 7f06fb7fe700 1 -- 172.21.15.37:0/3966709033 learned_addr learned my addr 172.21.15.37:0/3966709033 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:00:25.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.934+0000 7f06fa7fc700 1 --2- 172.21.15.37:0/3966709033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ee1bfe6c0 0x563ee1c07b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:25.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.934+0000 7f06faffd700 1 --2- 172.21.15.37:0/3966709033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee1bf8fa0 0x563ee1c0b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:25.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.934+0000 7f06fb7fe700 1 -- 172.21.15.37:0/3966709033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee1bf8fa0 msgr2=0x563ee1c0b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:25.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.934+0000 7f06fb7fe700 1 --2- 172.21.15.37:0/3966709033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee1bf8fa0 0x563ee1c0b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.934+0000 7f06fb7fe700 1 -- 172.21.15.37:0/3966709033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ee1bfe6c0 msgr2=0x563ee1c07b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:25.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.934+0000 7f06fb7fe700 1 --2- 172.21.15.37:0/3966709033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ee1bfe6c0 0x563ee1c07b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.934+0000 7f06fb7fe700 1 -- 172.21.15.37:0/3966709033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ee1c7d3a0 con 0x563ee1bfefe0 2024-06-10T01:00:25.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.935+0000 7f06fb7fe700 1 --2- 172.21.15.37:0/3966709033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ee1bfefe0 0x563ee1c08290 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f06f4011a50 tx=0x7f06f4011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd3850fdc3e140f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:25.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.935+0000 7f06deff5700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:00:25.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.935+0000 7f06deff5700 1 -- 172.21.15.37:0/3966709033 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4010d70 con 0x563ee1bfefe0 2024-06-10T01:00:25.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.935+0000 7f070bdb2b80 1 -- 172.21.15.37:0/3966709033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06f0020e50 con 0x563ee1bfefe0 2024-06-10T01:00:25.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.935+0000 7f06deff5700 1 -- 172.21.15.37:0/3966709033 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f06f4004020 con 0x563ee1bfefe0 2024-06-10T01:00:25.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.935+0000 7f06deff5700 1 -- 172.21.15.37:0/3966709033 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f06f4026050 con 0x563ee1bfefe0 2024-06-10T01:00:25.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.935+0000 7f070bdb2b80 10 client.16041 fetch_fsmap learned FSMap version 27 2024-06-10T01:00:25.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.935+0000 7f070bdb2b80 1 -- 172.21.15.37:0/3966709033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563ee1c7e5c0 con 0x563ee1bfefe0 2024-06-10T01:00:25.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.935+0000 7f06deff5700 1 -- 172.21.15.37:0/3966709033 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f0020e50 con 0x563ee1bfefe0 2024-06-10T01:00:25.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.936+0000 7f06deff5700 1 -- 172.21.15.37:0/3966709033 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f06f401b400 con 0x563ee1bfefe0 2024-06-10T01:00:25.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.936+0000 7f070bdb2b80 10 client.16041 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:00:25.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.936+0000 7f070bdb2b80 10 client.16041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:00:25.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.936+0000 7f070bdb2b80 20 client.16041 populate_metadata read hostname 'smithi037' 2024-06-10T01:00:25.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.936+0000 7f070bdb2b80 1 --2- 172.21.15.37:0/3966709033 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x563ee1c7ed50 0x563ee1c81170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:25.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.936+0000 7f070bdb2b80 4 client.16041 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:00:25.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.936+0000 7f070bdb2b80 1 -- 172.21.15.37:0/3966709033 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563ee1b52310 con 0x563ee1c7ed50 2024-06-10T01:00:25.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.936+0000 7f06faffd700 1 --2- 172.21.15.37:0/3966709033 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x563ee1c7ed50 0x563ee1c81170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:25.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.936+0000 7f06deff5700 10 client.16041 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:00:25.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.936+0000 7f06faffd700 1 --2- 172.21.15.37:0/3966709033 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x563ee1c7ed50 0x563ee1c81170 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77e668ae102d0f87 server_cookie=f5c03cba68a8cba0 in_seq=0 out_seq=0 2024-06-10T01:00:25.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.938+0000 7f06deff5700 1 -- 172.21.15.37:0/3966709033 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563ee1b52310 con 0x563ee1c7ed50 2024-06-10T01:00:25.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.938+0000 7f06deff5700 10 client.16041 handle_command_reply: tid=0 2024-06-10T01:00:25.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.938+0000 7f070bdb2b80 1 client.16041 shutdown 2024-06-10T01:00:25.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.938+0000 7f070bdb2b80 1 -- 172.21.15.37:0/3966709033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ee1bfefe0 msgr2=0x563ee1c08290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:25.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.938+0000 7f070bdb2b80 1 --2- 172.21.15.37:0/3966709033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ee1bfefe0 0x563ee1c08290 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f06f4011a50 tx=0x7f06f4011d60 comp rx=0 tx=0).stop 2024-06-10T01:00:25.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.938+0000 7f070bdb2b80 1 -- 172.21.15.37:0/3966709033 shutdown_connections 2024-06-10T01:00:25.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.938+0000 7f070bdb2b80 1 --2- 172.21.15.37:0/3966709033 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x563ee1c7ed50 0x563ee1c81170 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.938+0000 7f070bdb2b80 1 --2- 172.21.15.37:0/3966709033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ee1bfefe0 0x563ee1c08290 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.938+0000 7f070bdb2b80 1 --2- 172.21.15.37:0/3966709033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee1bf8fa0 0x563ee1c0b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.938+0000 7f070bdb2b80 1 --2- 172.21.15.37:0/3966709033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ee1bfe6c0 0x563ee1c07b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.938+0000 7f070bdb2b80 1 -- 172.21.15.37:0/3966709033 >> 172.21.15.37:0/3966709033 conn(0x563ee1b23d40 msgr2=0x563ee1c03880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:25.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.938+0000 7f06deff5700 0 client.16041 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:00:25.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.938+0000 7f070bdb2b80 1 -- 172.21.15.37:0/3966709033 shutdown_connections 2024-06-10T01:00:25.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.938+0000 7f070bdb2b80 1 -- 172.21.15.37:0/3966709033 wait complete. 2024-06-10T01:00:25.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.938+0000 7f070bdb2b80 20 client.16041 trim_cache size 0 max 16384 2024-06-10T01:00:25.942 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:00:25.942 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-06-10T01:00:25.942 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "afc9e6b4-930a-4be7-a432-5fdda1c93f32", 2024-06-10T01:00:25.942 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-06-10T01:00:25.942 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:00:25.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.939+0000 7f070152f700 1 -- 172.21.15.37:0/1658773600 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f06e40a2c20 msgr2=0x7f06e40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:25.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.939+0000 7f070152f700 1 --2- 172.21.15.37:0/1658773600 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f06e40a2c20 0x7f06e40a50e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f06fc0789b0 tx=0x7f06f000b3c0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.939+0000 7f070152f700 1 -- 172.21.15.37:0/1658773600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06fc06a170 msgr2=0x7f06fc101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:25.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.939+0000 7f070152f700 1 --2- 172.21.15.37:0/1658773600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06fc06a170 0x7f06fc101460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f06ec000f80 tx=0x7f06ec004d50 comp rx=0 tx=0).stop 2024-06-10T01:00:25.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.939+0000 7f070152f700 1 -- 172.21.15.37:0/1658773600 shutdown_connections 2024-06-10T01:00:25.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.939+0000 7f070152f700 1 --2- 172.21.15.37:0/1658773600 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f06e40a2c20 0x7f06e40a50e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.939+0000 7f070152f700 1 --2- 172.21.15.37:0/1658773600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06fc108500 0x7f06fc101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.939+0000 7f070152f700 1 --2- 172.21.15.37:0/1658773600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06fc06a170 0x7f06fc101460 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.939+0000 7f070152f700 1 --2- 172.21.15.37:0/1658773600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06fc06ab30 0x7f06fc1019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:25.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.939+0000 7f070152f700 1 -- 172.21.15.37:0/1658773600 >> 172.21.15.37:0/1658773600 conn(0x7f06fc078340 msgr2=0x7f06fc100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:25.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.939+0000 7f070152f700 1 -- 172.21.15.37:0/1658773600 shutdown_connections 2024-06-10T01:00:25.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:00:25.939+0000 7f070152f700 1 -- 172.21.15.37:0/1658773600 wait complete. 2024-06-10T01:00:25.989 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=58 2024-06-10T01:00:25.989 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:26.096 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:26.096 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:26.097 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:26.097 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:26.098 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:26.310 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:26 smithi037 ceph-mon[32192]: pgmap v542: 129 pgs: 129 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.7 KiB/s wr, 9 op/s 2024-06-10T01:00:26.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:26 smithi094 ceph-mon[36708]: pgmap v542: 129 pgs: 129 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.7 KiB/s wr, 9 op/s 2024-06-10T01:00:26.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:26 smithi167 ceph-mon[36917]: pgmap v542: 129 pgs: 129 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.7 KiB/s wr, 9 op/s 2024-06-10T01:00:27.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:27 smithi094 ceph-mon[36708]: scrub queued for path: / 2024-06-10T01:00:27.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:27 smithi094 ceph-mon[36708]: scrub summary: idle+waiting paths [/] 2024-06-10T01:00:27.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:27 smithi094 ceph-mon[36708]: scrub summary: active paths [/] 2024-06-10T01:00:27.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:27 smithi094 ceph-mon[36708]: scrub summary: idle+waiting paths [/] 2024-06-10T01:00:27.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:27 smithi037 ceph-mon[32192]: scrub queued for path: / 2024-06-10T01:00:27.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:27 smithi037 ceph-mon[32192]: scrub summary: idle+waiting paths [/] 2024-06-10T01:00:27.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:27 smithi037 ceph-mon[32192]: scrub summary: active paths [/] 2024-06-10T01:00:27.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:27 smithi037 ceph-mon[32192]: scrub summary: idle+waiting paths [/] 2024-06-10T01:00:27.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:27 smithi167 ceph-mon[36917]: scrub queued for path: / 2024-06-10T01:00:27.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:27 smithi167 ceph-mon[36917]: scrub summary: idle+waiting paths [/] 2024-06-10T01:00:27.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:27 smithi167 ceph-mon[36917]: scrub summary: active paths [/] 2024-06-10T01:00:27.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:27 smithi167 ceph-mon[36917]: scrub summary: idle+waiting paths [/] 2024-06-10T01:00:27.982 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:27.982 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:28.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:28 smithi094 ceph-mon[36708]: pgmap v543: 129 pgs: 129 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-06-10T01:00:28.388 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:28.388 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:28.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:28 smithi037 ceph-mon[32192]: pgmap v543: 129 pgs: 129 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-06-10T01:00:28.419 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:28.419 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:28.419 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 58 -lt 100 ']' 2024-06-10T01:00:28.420 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 58 + 1 2024-06-10T01:00:28.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:28 smithi167 ceph-mon[36917]: pgmap v543: 129 pgs: 129 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-06-10T01:00:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.612+0000 7ff51c153640 1 -- 172.21.15.37:0/1370490170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff514075980 msgr2=0x7ff514073dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.612+0000 7ff51c153640 1 --2- 172.21.15.37:0/1370490170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff514075980 0x7ff514073dd0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff510009900 tx=0x7ff51002a480 comp rx=0 tx=0).stop 2024-06-10T01:00:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.612+0000 7ff51c153640 1 -- 172.21.15.37:0/1370490170 shutdown_connections 2024-06-10T01:00:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.612+0000 7ff51c153640 1 --2- 172.21.15.37:0/1370490170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5141098c0 0x7ff51410bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.612+0000 7ff51c153640 1 --2- 172.21.15.37:0/1370490170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5140743a0 0x7ff514074800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.612+0000 7ff51c153640 1 --2- 172.21.15.37:0/1370490170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff514075980 0x7ff514073dd0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.612+0000 7ff51c153640 1 -- 172.21.15.37:0/1370490170 >> 172.21.15.37:0/1370490170 conn(0x7ff5140fcfc0 msgr2=0x7ff5140ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.612+0000 7ff51c153640 1 -- 172.21.15.37:0/1370490170 shutdown_connections 2024-06-10T01:00:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.612+0000 7ff51c153640 1 -- 172.21.15.37:0/1370490170 wait complete. 2024-06-10T01:00:28.620 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=59 2024-06-10T01:00:28.621 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.649+0000 7ff51c153640 1 Processor -- start 2024-06-10T01:00:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.649+0000 7ff51c153640 1 -- start start 2024-06-10T01:00:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.649+0000 7ff51c153640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5140743a0 0x7ff514199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.649+0000 7ff51c153640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff514075980 0x7ff51419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.649+0000 7ff51c153640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5141098c0 0x7ff5141a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.649+0000 7ff51c153640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff51410f8f0 con 0x7ff514075980 2024-06-10T01:00:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.649+0000 7ff51c153640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff51410f770 con 0x7ff5140743a0 2024-06-10T01:00:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.649+0000 7ff51c153640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff51410fa70 con 0x7ff5141098c0 2024-06-10T01:00:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.649+0000 7ff5196ee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff514075980 0x7ff51419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.650+0000 7ff5196ee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff514075980 0x7ff51419a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38538/0 (socket says 172.21.15.37:38538) 2024-06-10T01:00:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.650+0000 7ff5196ee640 1 -- 172.21.15.37:0/1910189605 learned_addr learned my addr 172.21.15.37:0/1910189605 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:00:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.650+0000 7ff519eef640 1 --2- 172.21.15.37:0/1910189605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5140743a0 0x7ff514199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.650+0000 7ff51a6f0640 1 --2- 172.21.15.37:0/1910189605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5141098c0 0x7ff5141a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.650+0000 7ff5196ee640 1 -- 172.21.15.37:0/1910189605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5141098c0 msgr2=0x7ff5141a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.650+0000 7ff5196ee640 1 --2- 172.21.15.37:0/1910189605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5141098c0 0x7ff5141a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.650+0000 7ff5196ee640 1 -- 172.21.15.37:0/1910189605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5140743a0 msgr2=0x7ff514199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.650+0000 7ff5196ee640 1 --2- 172.21.15.37:0/1910189605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5140743a0 0x7ff514199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.650+0000 7ff5196ee640 1 -- 172.21.15.37:0/1910189605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5141a1a80 con 0x7ff514075980 2024-06-10T01:00:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.650+0000 7ff519eef640 1 --2- 172.21.15.37:0/1910189605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5140743a0 0x7ff514199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:00:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.650+0000 7ff5196ee640 1 --2- 172.21.15.37:0/1910189605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff514075980 0x7ff51419a3f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff50400ea60 tx=0x7ff50400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:28.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.665+0000 7ff50affd640 1 -- 172.21.15.37:0/1910189605 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50400cde0 con 0x7ff514075980 2024-06-10T01:00:28.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.665+0000 7ff50affd640 1 -- 172.21.15.37:0/1910189605 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5040035f0 con 0x7ff514075980 2024-06-10T01:00:28.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.665+0000 7ff50affd640 1 -- 172.21.15.37:0/1910189605 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff504010670 con 0x7ff514075980 2024-06-10T01:00:28.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.665+0000 7ff51c153640 1 -- 172.21.15.37:0/1910189605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5141a1d70 con 0x7ff514075980 2024-06-10T01:00:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.668+0000 7ff50affd640 1 -- 172.21.15.37:0/1910189605 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff5040037a0 con 0x7ff514075980 2024-06-10T01:00:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.669+0000 7ff50affd640 1 --2- 172.21.15.37:0/1910189605 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff5000a2da0 0x7ff5000a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.669+0000 7ff519eef640 1 --2- 172.21.15.37:0/1910189605 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff5000a2da0 0x7ff5000a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.670+0000 7ff519eef640 1 --2- 172.21.15.37:0/1910189605 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff5000a2da0 0x7ff5000a5260 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff510031440 tx=0x7ff5100315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:28.675 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:28.675 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:28.677 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:28.677 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:28.677 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:28.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.680+0000 7ff51c153640 1 -- 172.21.15.37:0/1910189605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5141a2210 con 0x7ff514075980 2024-06-10T01:00:28.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.681+0000 7ff50affd640 1 -- 172.21.15.37:0/1910189605 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7ff504099240 con 0x7ff514075980 2024-06-10T01:00:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.990+0000 7ff51ee5d740 1 Processor -- start 2024-06-10T01:00:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.990+0000 7ff51ee5d740 1 -- start start 2024-06-10T01:00:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.990+0000 7ff51ee5d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e00ffd1010 0x55e00ffd1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.990+0000 7ff51ee5d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e00ffd1a00 0x55e00ffd6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.990+0000 7ff51ee5d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e00ffd6b70 0x55e00ffd9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.990+0000 7ff51ee5d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e00ff344f0 con 0x55e00ffd1010 2024-06-10T01:00:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.990+0000 7ff51ee5d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e00fed2870 con 0x55e00ffd1a00 2024-06-10T01:00:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.990+0000 7ff51ee5d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e00ff304b0 con 0x55e00ffd6b70 2024-06-10T01:00:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.991+0000 7ff519eef640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e00ffd1010 0x55e00ffd1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.991+0000 7ff519eef640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e00ffd1010 0x55e00ffd1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38544/0 (socket says 172.21.15.37:38544) 2024-06-10T01:00:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.991+0000 7ff519eef640 1 -- 172.21.15.37:0/1078396262 learned_addr learned my addr 172.21.15.37:0/1078396262 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:00:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.991+0000 7ff51a6f0640 1 --2- 172.21.15.37:0/1078396262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e00ffd6b70 0x55e00ffd9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.991+0000 7ff5196ee640 1 --2- 172.21.15.37:0/1078396262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e00ffd1a00 0x55e00ffd6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.991+0000 7ff519eef640 1 -- 172.21.15.37:0/1078396262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e00ffd6b70 msgr2=0x55e00ffd9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.991+0000 7ff519eef640 1 --2- 172.21.15.37:0/1078396262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e00ffd6b70 0x55e00ffd9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.991+0000 7ff519eef640 1 -- 172.21.15.37:0/1078396262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e00ffd1a00 msgr2=0x55e00ffd6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.991+0000 7ff519eef640 1 --2- 172.21.15.37:0/1078396262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e00ffd1a00 0x55e00ffd6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.991+0000 7ff519eef640 1 -- 172.21.15.37:0/1078396262 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e00feee9b0 con 0x55e00ffd1010 2024-06-10T01:00:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.991+0000 7ff51a6f0640 1 --2- 172.21.15.37:0/1078396262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e00ffd6b70 0x55e00ffd9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:00:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.991+0000 7ff519eef640 1 --2- 172.21.15.37:0/1078396262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e00ffd1010 0x55e00ffd1430 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff510009900 tx=0x7ff51003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece6d03d974b6f6a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.992+0000 7ff4fffff640 1 -- 172.21.15.37:0/1078396262 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51004c070 con 0x55e00ffd1010 2024-06-10T01:00:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.992+0000 7ff4fffff640 1 -- 172.21.15.37:0/1078396262 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff510004770 con 0x55e00ffd1010 2024-06-10T01:00:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.992+0000 7ff4fffff640 1 -- 172.21.15.37:0/1078396262 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510035050 con 0x55e00ffd1010 2024-06-10T01:00:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.993+0000 7ff51ee5d740 1 -- 172.21.15.37:0/1078396262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e00ffd1010 msgr2=0x55e00ffd1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.993+0000 7ff51ee5d740 1 --2- 172.21.15.37:0/1078396262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e00ffd1010 0x55e00ffd1430 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff510009900 tx=0x7ff51003e000 comp rx=0 tx=0).stop 2024-06-10T01:00:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.993+0000 7ff51ee5d740 1 -- 172.21.15.37:0/1078396262 shutdown_connections 2024-06-10T01:00:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.993+0000 7ff51ee5d740 1 --2- 172.21.15.37:0/1078396262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e00ffd6b70 0x55e00ffd9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.993+0000 7ff51ee5d740 1 --2- 172.21.15.37:0/1078396262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e00ffd1a00 0x55e00ffd6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.993+0000 7ff51ee5d740 1 --2- 172.21.15.37:0/1078396262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e00ffd1010 0x55e00ffd1430 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.993+0000 7ff51ee5d740 1 -- 172.21.15.37:0/1078396262 >> 172.21.15.37:0/1078396262 conn(0x55e00ffcc680 msgr2=0x55e00ffcca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.993+0000 7ff51ee5d740 1 -- 172.21.15.37:0/1078396262 shutdown_connections 2024-06-10T01:00:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:28.993+0000 7ff51ee5d740 1 -- 172.21.15.37:0/1078396262 wait complete. 2024-06-10T01:00:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.024+0000 7ff51ee5d740 1 -- start start 2024-06-10T01:00:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.064+0000 7ff51ee5d740 1 Processor -- start 2024-06-10T01:00:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.065+0000 7ff51ee5d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e00ffd1010 0x55e00ffdb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.065+0000 7ff51ee5d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e00ffd1a00 0x55e00ffdb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.065+0000 7ff51ee5d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e00ffd6b70 0x55e00ffe2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.065+0000 7ff51ee5d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e00ffd9ac0 con 0x55e00ffd6b70 2024-06-10T01:00:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.065+0000 7ff51ee5d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e00febdae0 con 0x55e00ffd1a00 2024-06-10T01:00:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.065+0000 7ff51ee5d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e00ffd9c40 con 0x55e00ffd1010 2024-06-10T01:00:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.065+0000 7ff51a6f0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e00ffd6b70 0x55e00ffe2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.065+0000 7ff51a6f0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e00ffd6b70 0x55e00ffe2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38552/0 (socket says 172.21.15.37:38552) 2024-06-10T01:00:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.065+0000 7ff51a6f0640 1 -- 172.21.15.37:0/1054884390 learned_addr learned my addr 172.21.15.37:0/1054884390 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:00:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.065+0000 7ff519eef640 1 --2- 172.21.15.37:0/1054884390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e00ffd1010 0x55e00ffdb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.065+0000 7ff5196ee640 1 --2- 172.21.15.37:0/1054884390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e00ffd1a00 0x55e00ffdb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.065+0000 7ff51a6f0640 1 -- 172.21.15.37:0/1054884390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e00ffd1010 msgr2=0x55e00ffdb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.065+0000 7ff51a6f0640 1 --2- 172.21.15.37:0/1054884390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e00ffd1010 0x55e00ffdb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.065+0000 7ff51a6f0640 1 -- 172.21.15.37:0/1054884390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e00ffd1a00 msgr2=0x55e00ffdb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.065+0000 7ff51a6f0640 1 --2- 172.21.15.37:0/1054884390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e00ffd1a00 0x55e00ffdb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.065+0000 7ff51a6f0640 1 -- 172.21.15.37:0/1054884390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e00fc48210 con 0x55e00ffd6b70 2024-06-10T01:00:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.065+0000 7ff5196ee640 1 --2- 172.21.15.37:0/1054884390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e00ffd1a00 0x55e00ffdb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:00:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.066+0000 7ff51a6f0640 1 --2- 172.21.15.37:0/1054884390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e00ffd6b70 0x55e00ffe2750 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff50c011d10 tx=0x7ff50c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769a0ac29c0a659f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.066+0000 7ff4fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:00:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.066+0000 7ff4fd7fa640 1 -- 172.21.15.37:0/1054884390 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c01e060 con 0x55e00ffd6b70 2024-06-10T01:00:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.066+0000 7ff4fd7fa640 1 -- 172.21.15.37:0/1054884390 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7ff50c003970 con 0x55e00ffd6b70 2024-06-10T01:00:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.089+0000 7ff51ee5d740 10 client.0 _ll_register_callbacks cb 0x55e00fef69b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:00:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.091+0000 7ff51ee5d740 1 -- 172.21.15.37:0/1054884390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff510004770 con 0x55e00ffd6b70 2024-06-10T01:00:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.091+0000 7ff4fd7fa640 1 -- 172.21.15.37:0/1054884390 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff50c009e30 con 0x55e00ffd6b70 2024-06-10T01:00:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.091+0000 7ff51ee5d740 10 client.16053 fetch_fsmap learned FSMap version 27 2024-06-10T01:00:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.091+0000 7ff51ee5d740 1 -- 172.21.15.37:0/1054884390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55e00fd14620 con 0x55e00ffd6b70 2024-06-10T01:00:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.099+0000 7ff4fd7fa640 1 -- 172.21.15.37:0/1054884390 <== mon.0 v2:172.21.15.37:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff50c019070 con 0x55e00ffd6b70 2024-06-10T01:00:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.099+0000 7ff51ee5d740 10 client.16053 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:00:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.099+0000 7ff51ee5d740 10 client.16053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:00:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.099+0000 7ff51ee5d740 20 client.16053 populate_metadata read hostname 'smithi037' 2024-06-10T01:00:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.099+0000 7ff51ee5d740 1 --2- 172.21.15.37:0/1054884390 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55e00ffe6eb0 0x55e00ffe9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:29.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.099+0000 7ff51ee5d740 4 client.16053 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:00:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.099+0000 7ff51ee5d740 1 -- 172.21.15.37:0/1054884390 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e00ff47f10 con 0x55e00ffe6eb0 2024-06-10T01:00:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.099+0000 7ff519eef640 1 --2- 172.21.15.37:0/1054884390 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55e00ffe6eb0 0x55e00ffe9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.100+0000 7ff519eef640 1 --2- 172.21.15.37:0/1054884390 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55e00ffe6eb0 0x55e00ffe9370 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5916f8d481fa0ce8 server_cookie=5fb5510072de146a in_seq=0 out_seq=0 2024-06-10T01:00:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.100+0000 7ff4fd7fa640 10 client.16053 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:00:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.101+0000 7ff4fd7fa640 1 -- 172.21.15.37:0/1054884390 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e00ff47f10 con 0x55e00ffe6eb0 2024-06-10T01:00:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.101+0000 7ff4fd7fa640 10 client.16053 handle_command_reply: tid=0 2024-06-10T01:00:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.101+0000 7ff51ee5d740 1 client.16053 shutdown 2024-06-10T01:00:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.101+0000 7ff51ee5d740 1 -- 172.21.15.37:0/1054884390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e00ffd6b70 msgr2=0x55e00ffe2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.101+0000 7ff51ee5d740 1 --2- 172.21.15.37:0/1054884390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e00ffd6b70 0x55e00ffe2750 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff50c011d10 tx=0x7ff50c00f790 comp rx=0 tx=0).stop 2024-06-10T01:00:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.101+0000 7ff51ee5d740 1 -- 172.21.15.37:0/1054884390 shutdown_connections 2024-06-10T01:00:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.101+0000 7ff51ee5d740 1 --2- 172.21.15.37:0/1054884390 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55e00ffe6eb0 0x55e00ffe9370 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.101+0000 7ff51ee5d740 1 --2- 172.21.15.37:0/1054884390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e00ffd6b70 0x55e00ffe2750 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.101+0000 7ff51ee5d740 1 --2- 172.21.15.37:0/1054884390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e00ffd1a00 0x55e00ffdb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.101+0000 7ff51ee5d740 1 --2- 172.21.15.37:0/1054884390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e00ffd1010 0x55e00ffdb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.101+0000 7ff4fd7fa640 0 client.16053 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:00:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.101+0000 7ff51ee5d740 1 -- 172.21.15.37:0/1054884390 >> 172.21.15.37:0/1054884390 conn(0x55e00ffcc680 msgr2=0x55e00ffcb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.101+0000 7ff51ee5d740 1 -- 172.21.15.37:0/1054884390 shutdown_connections 2024-06-10T01:00:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.101+0000 7ff51ee5d740 1 -- 172.21.15.37:0/1054884390 wait complete. 2024-06-10T01:00:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.102+0000 7ff51ee5d740 20 client.16053 trim_cache size 0 max 16384 2024-06-10T01:00:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.118+0000 7ff51ee5d740 1 Processor -- start 2024-06-10T01:00:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.118+0000 7ff51ee5d740 1 -- start start 2024-06-10T01:00:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.118+0000 7ff51ee5d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e00ffd1010 0x55e00ffd2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.118+0000 7ff51ee5d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e00ffd1a00 0x55e00ffd3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.118+0000 7ff51ee5d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e00ffd6b70 0x55e00ffe2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.118+0000 7ff51ee5d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141af230 con 0x55e00ffd1010 2024-06-10T01:00:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.118+0000 7ff51ee5d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141aca90 con 0x55e00ffd1a00 2024-06-10T01:00:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.118+0000 7ff51ee5d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141aa8e0 con 0x55e00ffd6b70 2024-06-10T01:00:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.119+0000 7ff519eef640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e00ffd1010 0x55e00ffd2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.119+0000 7ff519eef640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e00ffd1010 0x55e00ffd2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38564/0 (socket says 172.21.15.37:38564) 2024-06-10T01:00:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.119+0000 7ff519eef640 1 -- 172.21.15.37:0/1350457480 learned_addr learned my addr 172.21.15.37:0/1350457480 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:00:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.119+0000 7ff5196ee640 1 --2- 172.21.15.37:0/1350457480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e00ffd1a00 0x55e00ffd3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.119+0000 7ff51a6f0640 1 --2- 172.21.15.37:0/1350457480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e00ffd6b70 0x55e00ffe2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.119+0000 7ff519eef640 1 -- 172.21.15.37:0/1350457480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e00ffd6b70 msgr2=0x55e00ffe2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.119+0000 7ff519eef640 1 --2- 172.21.15.37:0/1350457480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e00ffd6b70 0x55e00ffe2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.119+0000 7ff519eef640 1 -- 172.21.15.37:0/1350457480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e00ffd1a00 msgr2=0x55e00ffd3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.119+0000 7ff519eef640 1 --2- 172.21.15.37:0/1350457480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e00ffd1a00 0x55e00ffd3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.119+0000 7ff519eef640 1 -- 172.21.15.37:0/1350457480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e00ffc6ae0 con 0x55e00ffd1010 2024-06-10T01:00:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.119+0000 7ff519eef640 1 --2- 172.21.15.37:0/1350457480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e00ffd1010 0x55e00ffd2bf0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff510009900 tx=0x7ff510005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e5c88e340e446d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.119+0000 7ff4ff7fe640 1 -- 172.21.15.37:0/1350457480 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510035050 con 0x55e00ffd1010 2024-06-10T01:00:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.119+0000 7ff4ff7fe640 1 -- 172.21.15.37:0/1350457480 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff510003cd0 con 0x55e00ffd1010 2024-06-10T01:00:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.120+0000 7ff4ff7fe640 1 -- 172.21.15.37:0/1350457480 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51004c040 con 0x55e00ffd1010 2024-06-10T01:00:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.120+0000 7ff51ee5d740 1 -- 172.21.15.37:0/1350457480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e00ffd1010 msgr2=0x55e00ffd2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.120+0000 7ff51ee5d740 1 --2- 172.21.15.37:0/1350457480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e00ffd1010 0x55e00ffd2bf0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff510009900 tx=0x7ff510005fd0 comp rx=0 tx=0).stop 2024-06-10T01:00:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.120+0000 7ff51ee5d740 1 -- 172.21.15.37:0/1350457480 shutdown_connections 2024-06-10T01:00:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.120+0000 7ff51ee5d740 1 --2- 172.21.15.37:0/1350457480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e00ffd6b70 0x55e00ffe2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.120+0000 7ff51ee5d740 1 --2- 172.21.15.37:0/1350457480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e00ffd1a00 0x55e00ffd3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.120+0000 7ff51ee5d740 1 --2- 172.21.15.37:0/1350457480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e00ffd1010 0x55e00ffd2bf0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.120+0000 7ff51ee5d740 1 -- 172.21.15.37:0/1350457480 >> 172.21.15.37:0/1350457480 conn(0x55e00ffcc680 msgr2=0x55e00ffd6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.120+0000 7ff51ee5d740 1 -- 172.21.15.37:0/1350457480 shutdown_connections 2024-06-10T01:00:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.120+0000 7ff51ee5d740 1 -- 172.21.15.37:0/1350457480 wait complete. 2024-06-10T01:00:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.121+0000 7ff51ee5d740 1 -- start start 2024-06-10T01:00:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.121+0000 7ff51ee5d740 1 Processor -- start 2024-06-10T01:00:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.121+0000 7ff51ee5d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e00ffd1010 0x55e00ffe22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.121+0000 7ff51ee5d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e00ffd1a00 0x55e00ffded50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.122+0000 7ff51ee5d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e00ffd6b70 0x55e00ffdf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.122+0000 7ff51ee5d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141aaf50 con 0x55e00ffd1a00 2024-06-10T01:00:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.122+0000 7ff51ee5d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141ac3c0 con 0x55e00ffd6b70 2024-06-10T01:00:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.122+0000 7ff51ee5d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff5141ab620 con 0x55e00ffd1010 2024-06-10T01:00:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.122+0000 7ff51ee5d740 10 client.0 _ll_register_callbacks cb 0x7ff50c009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:00:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.122+0000 7ff5196ee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e00ffd1a00 0x55e00ffded50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.122+0000 7ff5196ee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e00ffd1a00 0x55e00ffded50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38574/0 (socket says 172.21.15.37:38574) 2024-06-10T01:00:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.122+0000 7ff5196ee640 1 -- 172.21.15.37:0/1042914142 learned_addr learned my addr 172.21.15.37:0/1042914142 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:00:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.122+0000 7ff519eef640 1 --2- 172.21.15.37:0/1042914142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e00ffd1010 0x55e00ffe22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.122+0000 7ff51a6f0640 1 --2- 172.21.15.37:0/1042914142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e00ffd6b70 0x55e00ffdf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.122+0000 7ff5196ee640 1 -- 172.21.15.37:0/1042914142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e00ffd1010 msgr2=0x55e00ffe22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.122+0000 7ff5196ee640 1 --2- 172.21.15.37:0/1042914142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e00ffd1010 0x55e00ffe22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.122+0000 7ff5196ee640 1 -- 172.21.15.37:0/1042914142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e00ffd6b70 msgr2=0x55e00ffdf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.122+0000 7ff5196ee640 1 --2- 172.21.15.37:0/1042914142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e00ffd6b70 0x55e00ffdf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.122+0000 7ff5196ee640 1 -- 172.21.15.37:0/1042914142 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e00ffe28e0 con 0x55e00ffd1a00 2024-06-10T01:00:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.122+0000 7ff5196ee640 1 --2- 172.21.15.37:0/1042914142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e00ffd1a00 0x55e00ffded50 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff50409b130 tx=0x7ff5040cc310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97fb274773dfea4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.122+0000 7ff4fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:00:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.122+0000 7ff4fd7fa640 1 -- 172.21.15.37:0/1042914142 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5040a3e10 con 0x55e00ffd1a00 2024-06-10T01:00:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.122+0000 7ff4fd7fa640 1 -- 172.21.15.37:0/1042914142 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7ff50409f040 con 0x55e00ffd1a00 2024-06-10T01:00:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.123+0000 7ff51ee5d740 1 -- 172.21.15.37:0/1042914142 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff510003cd0 con 0x55e00ffd1a00 2024-06-10T01:00:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.123+0000 7ff4fd7fa640 1 -- 172.21.15.37:0/1042914142 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff50409f260 con 0x55e00ffd1a00 2024-06-10T01:00:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.123+0000 7ff51ee5d740 10 client.16059 fetch_fsmap learned FSMap version 27 2024-06-10T01:00:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.123+0000 7ff51ee5d740 1 -- 172.21.15.37:0/1042914142 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e010050c00 con 0x55e00ffd1a00 2024-06-10T01:00:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.123+0000 7ff4fd7fa640 1 -- 172.21.15.37:0/1042914142 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff510003cd0 con 0x55e00ffd1a00 2024-06-10T01:00:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.123+0000 7ff4fd7fa640 1 -- 172.21.15.37:0/1042914142 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff504004090 con 0x55e00ffd1a00 2024-06-10T01:00:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.123+0000 7ff51ee5d740 10 client.16059 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:00:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.123+0000 7ff51ee5d740 10 client.16059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:00:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.123+0000 7ff51ee5d740 20 client.16059 populate_metadata read hostname 'smithi037' 2024-06-10T01:00:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.123+0000 7ff51ee5d740 1 --2- 172.21.15.37:0/1042914142 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55e010051600 0x55e010053a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.123+0000 7ff51ee5d740 4 client.16059 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:00:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.123+0000 7ff51ee5d740 1 -- 172.21.15.37:0/1042914142 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e00ff41c20 con 0x55e010051600 2024-06-10T01:00:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.124+0000 7ff519eef640 1 --2- 172.21.15.37:0/1042914142 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55e010051600 0x55e010053a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.124+0000 7ff519eef640 1 --2- 172.21.15.37:0/1042914142 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55e010051600 0x55e010053a20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64b9a90d1f2e3452 server_cookie=943a2c541d822fde in_seq=0 out_seq=0 2024-06-10T01:00:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.124+0000 7ff4fd7fa640 10 client.16059 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:00:29.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.140+0000 7ff4fd7fa640 1 -- 172.21.15.37:0/1042914142 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e00ff41c20 con 0x55e010051600 2024-06-10T01:00:29.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.140+0000 7ff4fd7fa640 10 client.16059 handle_command_reply: tid=0 2024-06-10T01:00:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.140+0000 7ff51ee5d740 1 client.16059 shutdown 2024-06-10T01:00:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.141+0000 7ff51ee5d740 1 -- 172.21.15.37:0/1042914142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e00ffd1a00 msgr2=0x55e00ffded50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.141+0000 7ff51ee5d740 1 --2- 172.21.15.37:0/1042914142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e00ffd1a00 0x55e00ffded50 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff50409b130 tx=0x7ff5040cc310 comp rx=0 tx=0).stop 2024-06-10T01:00:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.141+0000 7ff51ee5d740 1 -- 172.21.15.37:0/1042914142 shutdown_connections 2024-06-10T01:00:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.141+0000 7ff51ee5d740 1 --2- 172.21.15.37:0/1042914142 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55e010051600 0x55e010053a20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.141+0000 7ff51ee5d740 1 --2- 172.21.15.37:0/1042914142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e00ffd6b70 0x55e00ffdf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.141+0000 7ff51ee5d740 1 --2- 172.21.15.37:0/1042914142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e00ffd1a00 0x55e00ffded50 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.141+0000 7ff51ee5d740 1 --2- 172.21.15.37:0/1042914142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e00ffd1010 0x55e00ffe22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.141+0000 7ff51ee5d740 1 -- 172.21.15.37:0/1042914142 >> 172.21.15.37:0/1042914142 conn(0x55e00ffcc680 msgr2=0x55e00ffdb120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.141+0000 7ff4fd7fa640 0 client.16059 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:00:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.141+0000 7ff51ee5d740 1 -- 172.21.15.37:0/1042914142 shutdown_connections 2024-06-10T01:00:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.141+0000 7ff51ee5d740 1 -- 172.21.15.37:0/1042914142 wait complete. 2024-06-10T01:00:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.141+0000 7ff51ee5d740 20 client.16059 trim_cache size 0 max 16384 2024-06-10T01:00:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:00:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:00:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:00:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.142+0000 7ff51c153640 1 -- 172.21.15.37:0/1910189605 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff5000a2da0 msgr2=0x7ff5000a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.142+0000 7ff51c153640 1 --2- 172.21.15.37:0/1910189605 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff5000a2da0 0x7ff5000a5260 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff510031440 tx=0x7ff5100315b0 comp rx=0 tx=0).stop 2024-06-10T01:00:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.142+0000 7ff51c153640 1 -- 172.21.15.37:0/1910189605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff514075980 msgr2=0x7ff51419a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.142+0000 7ff51c153640 1 --2- 172.21.15.37:0/1910189605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff514075980 0x7ff51419a3f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff50400ea60 tx=0x7ff50400ef20 comp rx=0 tx=0).stop 2024-06-10T01:00:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.142+0000 7ff51c153640 1 -- 172.21.15.37:0/1910189605 shutdown_connections 2024-06-10T01:00:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.142+0000 7ff51c153640 1 --2- 172.21.15.37:0/1910189605 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff5000a2da0 0x7ff5000a5260 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.142+0000 7ff51c153640 1 --2- 172.21.15.37:0/1910189605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5141098c0 0x7ff5141a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.142+0000 7ff51c153640 1 --2- 172.21.15.37:0/1910189605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff514075980 0x7ff51419a3f0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.142+0000 7ff51c153640 1 --2- 172.21.15.37:0/1910189605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5140743a0 0x7ff514199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.142+0000 7ff51c153640 1 -- 172.21.15.37:0/1910189605 >> 172.21.15.37:0/1910189605 conn(0x7ff5140fcfc0 msgr2=0x7ff514105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.142+0000 7ff51c153640 1 -- 172.21.15.37:0/1910189605 shutdown_connections 2024-06-10T01:00:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:29.142+0000 7ff51c153640 1 -- 172.21.15.37:0/1910189605 wait complete. 2024-06-10T01:00:30.074 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:30.074 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:30.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:30 smithi094 ceph-mon[36708]: pgmap v544: 129 pgs: 129 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.5 KiB/s wr, 9 op/s 2024-06-10T01:00:30.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:30 smithi037 ceph-mon[32192]: pgmap v544: 129 pgs: 129 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.5 KiB/s wr, 9 op/s 2024-06-10T01:00:30.529 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:30.529 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:30.545 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:30.545 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:30.545 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 59 -lt 100 ']' 2024-06-10T01:00:30.545 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 59 + 1 2024-06-10T01:00:30.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:30 smithi167 ceph-mon[36917]: pgmap v544: 129 pgs: 129 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.5 KiB/s wr, 9 op/s 2024-06-10T01:00:30.595 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=60 2024-06-10T01:00:30.595 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:30.699 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:30.699 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:30.700 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:30.700 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:30.700 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:31.047 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:00:31.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:31 smithi094 ceph-mon[36708]: scrub summary: idle 2024-06-10T01:00:31.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:31 smithi037 ceph-mon[32192]: scrub summary: idle 2024-06-10T01:00:31.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:31 smithi167 ceph-mon[36917]: scrub summary: idle 2024-06-10T01:00:31.732 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:31.732 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:31.835 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:31.835 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:31.841 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:31.842 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:31.842 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 60 -lt 100 ']' 2024-06-10T01:00:31.842 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 60 + 1 2024-06-10T01:00:31.895 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=61 2024-06-10T01:00:31.895 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:31.951 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:31.951 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:31.952 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:31.952 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:31.953 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:32.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:32 smithi094 ceph-mon[36708]: pgmap v545: 129 pgs: 129 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 KiB/s wr, 9 op/s 2024-06-10T01:00:32.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:32 smithi037 ceph-mon[32192]: pgmap v545: 129 pgs: 129 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 KiB/s wr, 9 op/s 2024-06-10T01:00:32.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:32 smithi167 ceph-mon[36917]: pgmap v545: 129 pgs: 129 active+clean; 1.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 KiB/s wr, 9 op/s 2024-06-10T01:00:32.663 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:32.663 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:33.022 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:33.023 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:33.060 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:33.060 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:33.060 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 61 -lt 100 ']' 2024-06-10T01:00:33.060 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 61 + 1 2024-06-10T01:00:33.095 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=62 2024-06-10T01:00:33.095 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:33.147 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:33.147 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:33.149 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:33.149 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:33.149 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:33.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:33 smithi094 ceph-mon[36708]: pgmap v546: 129 pgs: 129 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-06-10T01:00:33.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:33 smithi037 ceph-mon[32192]: pgmap v546: 129 pgs: 129 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-06-10T01:00:33.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:33 smithi167 ceph-mon[36917]: pgmap v546: 129 pgs: 129 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-06-10T01:00:34.015 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:34.015 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:34.387 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:34.387 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:34.402 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:34.402 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:34.402 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 62 -lt 100 ']' 2024-06-10T01:00:34.403 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 62 + 1 2024-06-10T01:00:34.562 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=63 2024-06-10T01:00:34.562 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:34.588 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:34.588 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:34.590 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:34.590 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:34.590 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:36.290 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:35 smithi167 ceph-mon[36917]: pgmap v547: 129 pgs: 129 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.3 KiB/s wr, 12 op/s 2024-06-10T01:00:36.290 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:35 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:00:36.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:35 smithi094 ceph-mon[36708]: pgmap v547: 129 pgs: 129 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.3 KiB/s wr, 12 op/s 2024-06-10T01:00:36.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:35 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:00:36.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:35 smithi037 ceph-mon[32192]: pgmap v547: 129 pgs: 129 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.3 KiB/s wr, 12 op/s 2024-06-10T01:00:36.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:35 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:00:36.473 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:36.473 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:36.795 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:36.795 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:36.835 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:36.836 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:36.836 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 63 -lt 100 ']' 2024-06-10T01:00:36.836 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 63 + 1 2024-06-10T01:00:36.919 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=64 2024-06-10T01:00:36.920 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:36.929 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:36.929 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:36.930 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:36.930 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:36.931 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:38.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:37 smithi167 ceph-mon[36917]: pgmap v548: 129 pgs: 129 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-06-10T01:00:38.355 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:38.356 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:38.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:37 smithi094 ceph-mon[36708]: pgmap v548: 129 pgs: 129 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-06-10T01:00:38.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:37 smithi037 ceph-mon[32192]: pgmap v548: 129 pgs: 129 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-06-10T01:00:38.820 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:38.820 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:38.844 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:38.844 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:38.844 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 64 -lt 100 ']' 2024-06-10T01:00:38.845 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 64 + 1 2024-06-10T01:00:38.945 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=65 2024-06-10T01:00:38.945 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:39.065 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:39.065 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:39.066 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:39.066 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:39.067 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:40.316 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:39 smithi037 ceph-mon[32192]: pgmap v549: 129 pgs: 129 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.4 KiB/s wr, 13 op/s 2024-06-10T01:00:40.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:39 smithi167 ceph-mon[36917]: pgmap v549: 129 pgs: 129 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.4 KiB/s wr, 13 op/s 2024-06-10T01:00:40.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:39 smithi094 ceph-mon[36708]: pgmap v549: 129 pgs: 129 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.4 KiB/s wr, 13 op/s 2024-06-10T01:00:40.680 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:40.680 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:40.976 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:40.976 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:41.016 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:41.017 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:41.017 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 65 -lt 100 ']' 2024-06-10T01:00:41.017 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 65 + 1 2024-06-10T01:00:41.146 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=66 2024-06-10T01:00:41.146 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:41.186 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:41.186 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:41.187 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:41.187 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:41.187 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:41.996 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:41.996 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:41.996 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:42.286 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:42.286 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:42.286 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:00:42.286 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:41 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:00:42.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:41 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:42.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:41 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:42.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:41 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:42.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:41 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:42.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:41 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:42.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:41 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:00:42.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:41 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:00:42.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:41 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:42.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:41 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:42.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:41 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:42.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:41 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:00:42.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:41 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:00:42.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:41 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:00:42.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:41 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:00:42.497 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:42.497 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:42.953 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:42.953 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:42.975 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:42.975 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:42.976 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 66 -lt 100 ']' 2024-06-10T01:00:42.976 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 66 + 1 2024-06-10T01:00:43.154 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=67 2024-06-10T01:00:43.154 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:43.223 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:43.223 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:43.224 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:43.224 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:43.224 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:43.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:43 smithi167 ceph-mon[36917]: pgmap v550: 129 pgs: 129 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 11 op/s 2024-06-10T01:00:43.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:43 smithi167 ceph-mon[36917]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-06-10T01:00:43.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:43 smithi167 ceph-mon[36917]: Unable to set osd_memory_target on smithi037 to 185844019: error parsing value: Value '185844019' is below minimum 939524096 2024-06-10T01:00:43.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:43 smithi094 ceph-mon[36708]: pgmap v550: 129 pgs: 129 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 11 op/s 2024-06-10T01:00:43.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:43 smithi094 ceph-mon[36708]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-06-10T01:00:43.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:43 smithi094 ceph-mon[36708]: Unable to set osd_memory_target on smithi037 to 185844019: error parsing value: Value '185844019' is below minimum 939524096 2024-06-10T01:00:43.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:43 smithi037 ceph-mon[32192]: pgmap v550: 129 pgs: 129 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 11 op/s 2024-06-10T01:00:43.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:43 smithi037 ceph-mon[32192]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-06-10T01:00:43.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:43 smithi037 ceph-mon[32192]: Unable to set osd_memory_target on smithi037 to 185844019: error parsing value: Value '185844019' is below minimum 939524096 2024-06-10T01:00:44.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:44 smithi167 ceph-mon[36917]: pgmap v551: 129 pgs: 129 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-06-10T01:00:44.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:44 smithi094 ceph-mon[36708]: pgmap v551: 129 pgs: 129 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-06-10T01:00:44.372 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:44.372 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:44.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:44 smithi037 ceph-mon[32192]: pgmap v551: 129 pgs: 129 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-06-10T01:00:44.868 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:00:44.869 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:44.869 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:44.894 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:44.894 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:44.894 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 67 -lt 100 ']' 2024-06-10T01:00:44.894 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 67 + 1 2024-06-10T01:00:44.970 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=68 2024-06-10T01:00:44.970 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:45.004 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:45.004 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:45.005 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:45.005 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:45.005 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.075+0000 7effbfb5a640 1 -- 172.21.15.37:0/1524121345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effb8101a30 msgr2=0x7effb8101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.075+0000 7effbfb5a640 1 --2- 172.21.15.37:0/1524121345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effb8101a30 0x7effb8101e50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7effb4009900 tx=0x7effb402a410 comp rx=0 tx=0).stop 2024-06-10T01:00:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.075+0000 7effbfb5a640 1 -- 172.21.15.37:0/1524121345 shutdown_connections 2024-06-10T01:00:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.075+0000 7effbfb5a640 1 --2- 172.21.15.37:0/1524121345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7effb81035e0 0x7effb8109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.075+0000 7effbfb5a640 1 --2- 172.21.15.37:0/1524121345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effb8102c20 0x7effb81030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.075+0000 7effbfb5a640 1 --2- 172.21.15.37:0/1524121345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effb8101a30 0x7effb8101e50 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.075+0000 7effbfb5a640 1 -- 172.21.15.37:0/1524121345 >> 172.21.15.37:0/1524121345 conn(0x7effb80fcfc0 msgr2=0x7effb80ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.075+0000 7effbfb5a640 1 -- 172.21.15.37:0/1524121345 shutdown_connections 2024-06-10T01:00:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.075+0000 7effbfb5a640 1 -- 172.21.15.37:0/1524121345 wait complete. 2024-06-10T01:00:46.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.090+0000 7effbfb5a640 1 Processor -- start 2024-06-10T01:00:46.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.090+0000 7effbfb5a640 1 -- start start 2024-06-10T01:00:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.090+0000 7effbfb5a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7effb8101a30 0x7effb8197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.090+0000 7effbfb5a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effb8102c20 0x7effb8198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.090+0000 7effbfb5a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effb81035e0 0x7effb819f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.090+0000 7effbfb5a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7effb810d6f0 con 0x7effb81035e0 2024-06-10T01:00:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.090+0000 7effbfb5a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7effb810d570 con 0x7effb8101a30 2024-06-10T01:00:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.090+0000 7effbfb5a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7effb810d870 con 0x7effb8102c20 2024-06-10T01:00:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.091+0000 7effbe0f7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effb81035e0 0x7effb819f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.091+0000 7effbe0f7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effb81035e0 0x7effb819f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43354/0 (socket says 172.21.15.37:43354) 2024-06-10T01:00:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.091+0000 7effbe0f7640 1 -- 172.21.15.37:0/1329233465 learned_addr learned my addr 172.21.15.37:0/1329233465 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:00:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.091+0000 7effbd8f6640 1 --2- 172.21.15.37:0/1329233465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7effb8101a30 0x7effb8197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.091+0000 7effbd0f5640 1 --2- 172.21.15.37:0/1329233465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effb8102c20 0x7effb8198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.091+0000 7effbe0f7640 1 -- 172.21.15.37:0/1329233465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effb8102c20 msgr2=0x7effb8198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.091+0000 7effbe0f7640 1 --2- 172.21.15.37:0/1329233465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effb8102c20 0x7effb8198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.091+0000 7effbe0f7640 1 -- 172.21.15.37:0/1329233465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7effb8101a30 msgr2=0x7effb8197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.091+0000 7effbe0f7640 1 --2- 172.21.15.37:0/1329233465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7effb8101a30 0x7effb8197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.091+0000 7effbe0f7640 1 -- 172.21.15.37:0/1329233465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb819f930 con 0x7effb81035e0 2024-06-10T01:00:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.091+0000 7effbe0f7640 1 --2- 172.21.15.37:0/1329233465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effb81035e0 0x7effb819f290 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7effb000efd0 tx=0x7effb000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.091+0000 7effaaffd640 1 -- 172.21.15.37:0/1329233465 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb0003b70 con 0x7effb81035e0 2024-06-10T01:00:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.091+0000 7effaaffd640 1 -- 172.21.15.37:0/1329233465 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effb0003d10 con 0x7effb81035e0 2024-06-10T01:00:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.091+0000 7effbfb5a640 1 -- 172.21.15.37:0/1329233465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effb819fc20 con 0x7effb81035e0 2024-06-10T01:00:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.091+0000 7effaaffd640 1 -- 172.21.15.37:0/1329233465 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb00109f0 con 0x7effb81035e0 2024-06-10T01:00:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.094+0000 7effaaffd640 1 -- 172.21.15.37:0/1329233465 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7effb0004000 con 0x7effb81035e0 2024-06-10T01:00:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.094+0000 7effaaffd640 1 --2- 172.21.15.37:0/1329233465 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7effa40a2cb0 0x7effa40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.094+0000 7effbd8f6640 1 --2- 172.21.15.37:0/1329233465 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7effa40a2cb0 0x7effa40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.095+0000 7effbd8f6640 1 --2- 172.21.15.37:0/1329233465 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7effa40a2cb0 0x7effa40a5170 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7effb402aa00 tx=0x7effb40315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.096+0000 7effbfb5a640 1 -- 172.21.15.37:0/1329233465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effb81a0140 con 0x7effb81035e0 2024-06-10T01:00:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.096+0000 7effaaffd640 1 -- 172.21.15.37:0/1329233465 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7effb0099850 con 0x7effb81035e0 2024-06-10T01:00:46.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:46 smithi167 ceph-mon[36917]: pgmap v552: 129 pgs: 129 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 KiB/s wr, 9 op/s 2024-06-10T01:00:46.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:46 smithi094 ceph-mon[36708]: pgmap v552: 129 pgs: 129 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 KiB/s wr, 9 op/s 2024-06-10T01:00:46.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:46 smithi037 ceph-mon[32192]: pgmap v552: 129 pgs: 129 active+clean; 1.6 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 KiB/s wr, 9 op/s 2024-06-10T01:00:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.509+0000 7effc2867740 1 Processor -- start 2024-06-10T01:00:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.509+0000 7effc2867740 1 -- start start 2024-06-10T01:00:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.510+0000 7effc2867740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b41923010 0x557b41923430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.510+0000 7effc2867740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b41923a00 0x557b41928500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.510+0000 7effc2867740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b41928b70 0x557b4192b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.510+0000 7effc2867740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557b418864f0 con 0x557b41923010 2024-06-10T01:00:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.510+0000 7effc2867740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557b41824870 con 0x557b41928b70 2024-06-10T01:00:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.510+0000 7effc2867740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557b418824b0 con 0x557b41923a00 2024-06-10T01:00:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.510+0000 7effbd8f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b41923010 0x557b41923430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.510+0000 7effbd8f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b41923010 0x557b41923430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43368/0 (socket says 172.21.15.37:43368) 2024-06-10T01:00:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.510+0000 7effbd8f6640 1 -- 172.21.15.37:0/2204987949 learned_addr learned my addr 172.21.15.37:0/2204987949 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:00:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.510+0000 7effbe0f7640 1 --2- 172.21.15.37:0/2204987949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b41928b70 0x557b4192b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.510+0000 7effbd0f5640 1 --2- 172.21.15.37:0/2204987949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b41923a00 0x557b41928500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.510+0000 7effbd8f6640 1 -- 172.21.15.37:0/2204987949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b41923a00 msgr2=0x557b41928500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.510+0000 7effbd8f6640 1 --2- 172.21.15.37:0/2204987949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b41923a00 0x557b41928500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.510+0000 7effbd8f6640 1 -- 172.21.15.37:0/2204987949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b41928b70 msgr2=0x557b4192b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.510+0000 7effbd8f6640 1 --2- 172.21.15.37:0/2204987949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b41928b70 0x557b4192b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.510+0000 7effbd8f6640 1 -- 172.21.15.37:0/2204987949 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b418409b0 con 0x557b41923010 2024-06-10T01:00:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.510+0000 7effbd0f5640 1 --2- 172.21.15.37:0/2204987949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b41923a00 0x557b41928500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:00:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.510+0000 7effbd8f6640 1 --2- 172.21.15.37:0/2204987949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b41923010 0x557b41923430 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7effb8102a80 tx=0x7effb403e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3eacff90a5044e6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.511+0000 7effa3fff640 1 -- 172.21.15.37:0/2204987949 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb4035050 con 0x557b41923010 2024-06-10T01:00:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.511+0000 7effa3fff640 1 -- 172.21.15.37:0/2204987949 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effb4004770 con 0x557b41923010 2024-06-10T01:00:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.511+0000 7effa3fff640 1 -- 172.21.15.37:0/2204987949 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb4031b50 con 0x557b41923010 2024-06-10T01:00:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.512+0000 7effc2867740 1 -- 172.21.15.37:0/2204987949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b41923010 msgr2=0x557b41923430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.512+0000 7effc2867740 1 --2- 172.21.15.37:0/2204987949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b41923010 0x557b41923430 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7effb8102a80 tx=0x7effb403e000 comp rx=0 tx=0).stop 2024-06-10T01:00:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.512+0000 7effc2867740 1 -- 172.21.15.37:0/2204987949 shutdown_connections 2024-06-10T01:00:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.512+0000 7effc2867740 1 --2- 172.21.15.37:0/2204987949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b41928b70 0x557b4192b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.512+0000 7effc2867740 1 --2- 172.21.15.37:0/2204987949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b41923a00 0x557b41928500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.512+0000 7effc2867740 1 --2- 172.21.15.37:0/2204987949 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b41923010 0x557b41923430 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.512+0000 7effc2867740 1 -- 172.21.15.37:0/2204987949 >> 172.21.15.37:0/2204987949 conn(0x557b4191e680 msgr2=0x557b4191ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.512+0000 7effc2867740 1 -- 172.21.15.37:0/2204987949 shutdown_connections 2024-06-10T01:00:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.512+0000 7effc2867740 1 -- 172.21.15.37:0/2204987949 wait complete. 2024-06-10T01:00:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.526+0000 7effc2867740 1 -- start start 2024-06-10T01:00:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.527+0000 7effc2867740 1 Processor -- start 2024-06-10T01:00:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.530+0000 7effc2867740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b41923010 0x557b41931630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.530+0000 7effc2867740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b41923a00 0x557b41931b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.530+0000 7effc2867740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b41928b70 0x557b41938bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.530+0000 7effc2867740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557b4192bac0 con 0x557b41923a00 2024-06-10T01:00:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.530+0000 7effc2867740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557b4180fae0 con 0x557b41923010 2024-06-10T01:00:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.530+0000 7effc2867740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557b4192bc40 con 0x557b41928b70 2024-06-10T01:00:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.530+0000 7effbd0f5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b41923a00 0x557b41931b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.531+0000 7effbd0f5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b41923a00 0x557b41931b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43374/0 (socket says 172.21.15.37:43374) 2024-06-10T01:00:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.531+0000 7effbd0f5640 1 -- 172.21.15.37:0/2949557039 learned_addr learned my addr 172.21.15.37:0/2949557039 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:00:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.531+0000 7effbe0f7640 1 --2- 172.21.15.37:0/2949557039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b41928b70 0x557b41938bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.531+0000 7effbd8f6640 1 --2- 172.21.15.37:0/2949557039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b41923010 0x557b41931630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.531+0000 7effbd0f5640 1 -- 172.21.15.37:0/2949557039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b41928b70 msgr2=0x557b41938bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.531+0000 7effbd0f5640 1 --2- 172.21.15.37:0/2949557039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b41928b70 0x557b41938bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.531+0000 7effbd0f5640 1 -- 172.21.15.37:0/2949557039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b41923010 msgr2=0x557b41931630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.531+0000 7effbd0f5640 1 --2- 172.21.15.37:0/2949557039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b41923010 0x557b41931630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.531+0000 7effbd0f5640 1 -- 172.21.15.37:0/2949557039 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b4159a210 con 0x557b41923a00 2024-06-10T01:00:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.531+0000 7effbd8f6640 1 --2- 172.21.15.37:0/2949557039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b41923010 0x557b41931630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:00:46.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.531+0000 7effbd0f5640 1 --2- 172.21.15.37:0/2949557039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b41923a00 0x557b41931b70 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7effac012700 tx=0x7effac012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cad4d38989df38d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:46.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.531+0000 7effa17fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:00:46.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.531+0000 7effa17fa640 1 -- 172.21.15.37:0/2949557039 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac004050 con 0x557b41923a00 2024-06-10T01:00:46.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.531+0000 7effa17fa640 1 -- 172.21.15.37:0/2949557039 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7effac0041f0 con 0x557b41923a00 2024-06-10T01:00:46.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.532+0000 7effc2867740 10 client.0 _ll_register_callbacks cb 0x557b418489b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:00:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.533+0000 7effc2867740 1 -- 172.21.15.37:0/2949557039 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effb4004770 con 0x557b41923a00 2024-06-10T01:00:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.534+0000 7effa17fa640 1 -- 172.21.15.37:0/2949557039 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7effac004430 con 0x557b41923a00 2024-06-10T01:00:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.534+0000 7effc2867740 10 client.16071 fetch_fsmap learned FSMap version 27 2024-06-10T01:00:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.534+0000 7effc2867740 1 -- 172.21.15.37:0/2949557039 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=27}) -- 0x557b41666620 con 0x557b41923a00 2024-06-10T01:00:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.568+0000 7effa17fa640 1 -- 172.21.15.37:0/2949557039 <== mon.0 v2:172.21.15.37:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7effac00f8c0 con 0x557b41923a00 2024-06-10T01:00:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.568+0000 7effc2867740 10 client.16071 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:00:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.568+0000 7effc2867740 10 client.16071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:00:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.568+0000 7effc2867740 20 client.16071 populate_metadata read hostname 'smithi037' 2024-06-10T01:00:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.568+0000 7effc2867740 1 --2- 172.21.15.37:0/2949557039 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557b4193d2f0 0x557b4193f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.568+0000 7effc2867740 4 client.16071 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:00:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.568+0000 7effc2867740 1 -- 172.21.15.37:0/2949557039 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b41899f10 con 0x557b4193d2f0 2024-06-10T01:00:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.568+0000 7effbd8f6640 1 --2- 172.21.15.37:0/2949557039 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557b4193d2f0 0x557b4193f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.568+0000 7effbd8f6640 1 --2- 172.21.15.37:0/2949557039 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557b4193d2f0 0x557b4193f7b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0edf62f89c6ce9e server_cookie=6da734bfc254f693 in_seq=0 out_seq=0 2024-06-10T01:00:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.568+0000 7effa17fa640 10 client.16071 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:00:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.569+0000 7effa17fa640 1 -- 172.21.15.37:0/2949557039 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557b41899f10 con 0x557b4193d2f0 2024-06-10T01:00:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.569+0000 7effa17fa640 10 client.16071 handle_command_reply: tid=0 2024-06-10T01:00:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.569+0000 7effc2867740 1 client.16071 shutdown 2024-06-10T01:00:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.570+0000 7effc2867740 1 -- 172.21.15.37:0/2949557039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b41923a00 msgr2=0x557b41931b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.570+0000 7effc2867740 1 --2- 172.21.15.37:0/2949557039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b41923a00 0x557b41931b70 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7effac012700 tx=0x7effac012bc0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.570+0000 7effc2867740 1 -- 172.21.15.37:0/2949557039 shutdown_connections 2024-06-10T01:00:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.570+0000 7effc2867740 1 --2- 172.21.15.37:0/2949557039 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557b4193d2f0 0x557b4193f7b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.570+0000 7effc2867740 1 --2- 172.21.15.37:0/2949557039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b41928b70 0x557b41938bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.570+0000 7effc2867740 1 --2- 172.21.15.37:0/2949557039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b41923a00 0x557b41931b70 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.570+0000 7effc2867740 1 --2- 172.21.15.37:0/2949557039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b41923010 0x557b41931630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.570+0000 7effa17fa640 0 client.16071 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:00:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.570+0000 7effc2867740 1 -- 172.21.15.37:0/2949557039 >> 172.21.15.37:0/2949557039 conn(0x557b4191e680 msgr2=0x557b4191d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.570+0000 7effc2867740 1 -- 172.21.15.37:0/2949557039 shutdown_connections 2024-06-10T01:00:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.570+0000 7effc2867740 1 -- 172.21.15.37:0/2949557039 wait complete. 2024-06-10T01:00:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.597+0000 7effc2867740 20 client.16071 trim_cache size 0 max 16384 2024-06-10T01:00:46.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.612+0000 7effc2867740 1 Processor -- start 2024-06-10T01:00:46.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.612+0000 7effc2867740 1 -- start start 2024-06-10T01:00:46.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.613+0000 7effc2867740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b41923010 0x557b4191e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:46.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.613+0000 7effc2867740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b41923a00 0x557b41938b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:46.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.613+0000 7effc2867740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b41928b70 0x557b41924c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:46.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.613+0000 7effc2867740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7effb81aadc0 con 0x557b41923a00 2024-06-10T01:00:46.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.613+0000 7effc2867740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7effb81a8690 con 0x557b41923010 2024-06-10T01:00:46.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.613+0000 7effc2867740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7effb81a64e0 con 0x557b41928b70 2024-06-10T01:00:46.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.613+0000 7effbd0f5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b41923a00 0x557b41938b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.613+0000 7effbd0f5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b41923a00 0x557b41938b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43386/0 (socket says 172.21.15.37:43386) 2024-06-10T01:00:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.613+0000 7effbd0f5640 1 -- 172.21.15.37:0/3262902168 learned_addr learned my addr 172.21.15.37:0/3262902168 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:00:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.613+0000 7effbe0f7640 1 --2- 172.21.15.37:0/3262902168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b41928b70 0x557b41924c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.613+0000 7effbd8f6640 1 --2- 172.21.15.37:0/3262902168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b41923010 0x557b4191e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.613+0000 7effbd0f5640 1 -- 172.21.15.37:0/3262902168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b41928b70 msgr2=0x557b41924c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.613+0000 7effbd0f5640 1 --2- 172.21.15.37:0/3262902168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b41928b70 0x557b41924c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.613+0000 7effbd0f5640 1 -- 172.21.15.37:0/3262902168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b41923010 msgr2=0x557b4191e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.613+0000 7effbd0f5640 1 --2- 172.21.15.37:0/3262902168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b41923010 0x557b4191e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.613+0000 7effbd0f5640 1 -- 172.21.15.37:0/3262902168 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b4191f350 con 0x557b41923a00 2024-06-10T01:00:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.613+0000 7effbd8f6640 1 --2- 172.21.15.37:0/3262902168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b41923010 0x557b4191e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:00:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.613+0000 7effbe0f7640 1 --2- 172.21.15.37:0/3262902168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b41928b70 0x557b41924c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:00:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.613+0000 7effbd0f5640 1 --2- 172.21.15.37:0/3262902168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b41923a00 0x557b41938b70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7effac009e50 tx=0x7effac013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16564edd1f4e3b72 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.614+0000 7effa37fe640 1 -- 172.21.15.37:0/3262902168 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac00f790 con 0x557b41923a00 2024-06-10T01:00:46.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.614+0000 7effa37fe640 1 -- 172.21.15.37:0/3262902168 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effb4004770 con 0x557b41923a00 2024-06-10T01:00:46.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.614+0000 7effa37fe640 1 -- 172.21.15.37:0/3262902168 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac01c680 con 0x557b41923a00 2024-06-10T01:00:46.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.615+0000 7effc2867740 1 -- 172.21.15.37:0/3262902168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b41923a00 msgr2=0x557b41938b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:46.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.615+0000 7effc2867740 1 --2- 172.21.15.37:0/3262902168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b41923a00 0x557b41938b70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7effac009e50 tx=0x7effac013000 comp rx=0 tx=0).stop 2024-06-10T01:00:46.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.615+0000 7effc2867740 1 -- 172.21.15.37:0/3262902168 shutdown_connections 2024-06-10T01:00:46.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.615+0000 7effc2867740 1 --2- 172.21.15.37:0/3262902168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b41928b70 0x557b41924c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.615+0000 7effc2867740 1 --2- 172.21.15.37:0/3262902168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b41923a00 0x557b41938b70 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.615+0000 7effc2867740 1 --2- 172.21.15.37:0/3262902168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b41923010 0x557b4191e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.615+0000 7effc2867740 1 -- 172.21.15.37:0/3262902168 >> 172.21.15.37:0/3262902168 conn(0x557b4191e680 msgr2=0x557b419282b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:46.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.615+0000 7effc2867740 1 -- 172.21.15.37:0/3262902168 shutdown_connections 2024-06-10T01:00:46.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.615+0000 7effc2867740 1 -- 172.21.15.37:0/3262902168 wait complete. 2024-06-10T01:00:46.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.615+0000 7effc2867740 1 -- start start 2024-06-10T01:00:46.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.615+0000 7effc2867740 1 Processor -- start 2024-06-10T01:00:46.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.616+0000 7effc2867740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b41923010 0x557b41935d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:46.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.616+0000 7effc2867740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b41923a00 0x557b41936270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:46.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.616+0000 7effc2867740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b41928b70 0x557b41937280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:46.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.616+0000 7effc2867740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7effb81a6b50 con 0x557b41928b70 2024-06-10T01:00:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.616+0000 7effc2867740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7effb81a7fc0 con 0x557b41923a00 2024-06-10T01:00:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.616+0000 7effc2867740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7effb81a7220 con 0x557b41923010 2024-06-10T01:00:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.616+0000 7effc2867740 10 client.0 _ll_register_callbacks cb 0x7effac009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:00:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.616+0000 7effbe0f7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b41928b70 0x557b41937280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.616+0000 7effbd8f6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b41923010 0x557b41935d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.616+0000 7effbe0f7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b41928b70 0x557b41937280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43398/0 (socket says 172.21.15.37:43398) 2024-06-10T01:00:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.616+0000 7effbe0f7640 1 -- 172.21.15.37:0/3502232075 learned_addr learned my addr 172.21.15.37:0/3502232075 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:00:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.616+0000 7effbd0f5640 1 --2- 172.21.15.37:0/3502232075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b41923a00 0x557b41936270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.616+0000 7effbe0f7640 1 -- 172.21.15.37:0/3502232075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b41923010 msgr2=0x557b41935d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.616+0000 7effbe0f7640 1 --2- 172.21.15.37:0/3502232075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b41923010 0x557b41935d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.616+0000 7effbe0f7640 1 -- 172.21.15.37:0/3502232075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b41923a00 msgr2=0x557b41936270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:00:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.616+0000 7effbe0f7640 1 --2- 172.21.15.37:0/3502232075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b41923a00 0x557b41936270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.616+0000 7effbe0f7640 1 -- 172.21.15.37:0/3502232075 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b41937870 con 0x557b41928b70 2024-06-10T01:00:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.616+0000 7effbd8f6640 1 --2- 172.21.15.37:0/3502232075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b41923010 0x557b41935d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:00:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.616+0000 7effbd0f5640 1 --2- 172.21.15.37:0/3502232075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b41923a00 0x557b41936270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:00:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.616+0000 7effbe0f7640 1 --2- 172.21.15.37:0/3502232075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b41928b70 0x557b41937280 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7effb000cba0 tx=0x7effb0004880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866fb7cd075b5db3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:00:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.616+0000 7effa17fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:00:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.617+0000 7effa17fa640 1 -- 172.21.15.37:0/3502232075 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb009bc50 con 0x557b41928b70 2024-06-10T01:00:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.617+0000 7effa17fa640 1 -- 172.21.15.37:0/3502232075 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7effb0010b90 con 0x557b41928b70 2024-06-10T01:00:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.617+0000 7effc2867740 1 -- 172.21.15.37:0/3502232075 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effb4004770 con 0x557b41928b70 2024-06-10T01:00:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.617+0000 7effa17fa640 1 -- 172.21.15.37:0/3502232075 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7effb009b550 con 0x557b41928b70 2024-06-10T01:00:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.617+0000 7effc2867740 10 client.16077 fetch_fsmap learned FSMap version 27 2024-06-10T01:00:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.617+0000 7effc2867740 1 -- 172.21.15.37:0/3502232075 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557b41938b60 con 0x557b41928b70 2024-06-10T01:00:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.617+0000 7effa17fa640 1 -- 172.21.15.37:0/3502232075 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effb4004770 con 0x557b41928b70 2024-06-10T01:00:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.617+0000 7effa17fa640 1 -- 172.21.15.37:0/3502232075 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7effb0004c60 con 0x557b41928b70 2024-06-10T01:00:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.617+0000 7effc2867740 10 client.16077 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:00:46.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.617+0000 7effc2867740 10 client.16077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:00:46.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.617+0000 7effc2867740 20 client.16077 populate_metadata read hostname 'smithi037' 2024-06-10T01:00:46.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.617+0000 7effc2867740 1 --2- 172.21.15.37:0/3502232075 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557b419a84a0 0x557b419aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:00:46.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.617+0000 7effc2867740 4 client.16077 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:00:46.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.617+0000 7effc2867740 1 -- 172.21.15.37:0/3502232075 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b41893c20 con 0x557b419a84a0 2024-06-10T01:00:46.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.617+0000 7effbd8f6640 1 --2- 172.21.15.37:0/3502232075 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557b419a84a0 0x557b419aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:00:46.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.618+0000 7effbd8f6640 1 --2- 172.21.15.37:0/3502232075 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557b419a84a0 0x557b419aa8c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acb1bac7ddf71425 server_cookie=93b6d11f3fddf4ab in_seq=0 out_seq=0 2024-06-10T01:00:46.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.618+0000 7effa17fa640 10 client.16077 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:00:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.633+0000 7effa17fa640 1 -- 172.21.15.37:0/3502232075 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557b41893c20 con 0x557b419a84a0 2024-06-10T01:00:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.633+0000 7effa17fa640 10 client.16077 handle_command_reply: tid=0 2024-06-10T01:00:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.633+0000 7effc2867740 1 client.16077 shutdown 2024-06-10T01:00:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.634+0000 7effc2867740 1 -- 172.21.15.37:0/3502232075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b41928b70 msgr2=0x557b41937280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.634+0000 7effc2867740 1 --2- 172.21.15.37:0/3502232075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b41928b70 0x557b41937280 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7effb000cba0 tx=0x7effb0004880 comp rx=0 tx=0).stop 2024-06-10T01:00:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.634+0000 7effc2867740 1 -- 172.21.15.37:0/3502232075 shutdown_connections 2024-06-10T01:00:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.634+0000 7effc2867740 1 --2- 172.21.15.37:0/3502232075 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557b419a84a0 0x557b419aa8c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.634+0000 7effc2867740 1 --2- 172.21.15.37:0/3502232075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b41928b70 0x557b41937280 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.634+0000 7effc2867740 1 --2- 172.21.15.37:0/3502232075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557b41923a00 0x557b41936270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.634+0000 7effc2867740 1 --2- 172.21.15.37:0/3502232075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b41923010 0x557b41935d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.634+0000 7effa17fa640 0 client.16077 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:00:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.634+0000 7effc2867740 1 -- 172.21.15.37:0/3502232075 >> 172.21.15.37:0/3502232075 conn(0x557b4191e680 msgr2=0x557b41927e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.634+0000 7effc2867740 1 -- 172.21.15.37:0/3502232075 shutdown_connections 2024-06-10T01:00:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.634+0000 7effc2867740 1 -- 172.21.15.37:0/3502232075 wait complete. 2024-06-10T01:00:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.634+0000 7effc2867740 20 client.16077 trim_cache size 0 max 16384 2024-06-10T01:00:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:00:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:00:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:00:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.635+0000 7effbfb5a640 1 -- 172.21.15.37:0/1329233465 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7effa40a2cb0 msgr2=0x7effa40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.635+0000 7effbfb5a640 1 --2- 172.21.15.37:0/1329233465 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7effa40a2cb0 0x7effa40a5170 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7effb402aa00 tx=0x7effb40315b0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.635+0000 7effbfb5a640 1 -- 172.21.15.37:0/1329233465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effb81035e0 msgr2=0x7effb819f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:00:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.635+0000 7effbfb5a640 1 --2- 172.21.15.37:0/1329233465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effb81035e0 0x7effb819f290 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7effb000efd0 tx=0x7effb000c6a0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.635+0000 7effbfb5a640 1 -- 172.21.15.37:0/1329233465 shutdown_connections 2024-06-10T01:00:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.635+0000 7effbfb5a640 1 --2- 172.21.15.37:0/1329233465 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7effa40a2cb0 0x7effa40a5170 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.635+0000 7effbfb5a640 1 --2- 172.21.15.37:0/1329233465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effb81035e0 0x7effb819f290 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.635+0000 7effbfb5a640 1 --2- 172.21.15.37:0/1329233465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effb8102c20 0x7effb8198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.635+0000 7effbfb5a640 1 --2- 172.21.15.37:0/1329233465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7effb8101a30 0x7effb8197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:00:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.635+0000 7effbfb5a640 1 -- 172.21.15.37:0/1329233465 >> 172.21.15.37:0/1329233465 conn(0x7effb80fcfc0 msgr2=0x7effb8105ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:00:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.635+0000 7effbfb5a640 1 -- 172.21.15.37:0/1329233465 shutdown_connections 2024-06-10T01:00:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:00:46.635+0000 7effbfb5a640 1 -- 172.21.15.37:0/1329233465 wait complete. 2024-06-10T01:00:46.738 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:46.738 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:47.126 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:47.126 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:47.149 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:47.150 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:47.150 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 68 -lt 100 ']' 2024-06-10T01:00:47.150 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 68 + 1 2024-06-10T01:00:47.261 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=69 2024-06-10T01:00:47.261 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:47.288 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:47.288 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:47.289 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:47.290 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:47.290 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:48.137 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:48 smithi037 ceph-mon[32192]: pgmap v553: 129 pgs: 129 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 8 op/s 2024-06-10T01:00:48.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:48 smithi167 ceph-mon[36917]: pgmap v553: 129 pgs: 129 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 8 op/s 2024-06-10T01:00:48.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:48 smithi094 ceph-mon[36708]: pgmap v553: 129 pgs: 129 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 8 op/s 2024-06-10T01:00:48.465 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:48.465 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:48.701 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:48.701 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:48.735 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:48.735 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:48.735 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 69 -lt 100 ']' 2024-06-10T01:00:48.735 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 69 + 1 2024-06-10T01:00:48.811 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=70 2024-06-10T01:00:48.812 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:48.837 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:48.837 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:48.838 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:48.838 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:48.839 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:50.007 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:50.007 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:50.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:50 smithi167 ceph-mon[36917]: pgmap v554: 129 pgs: 129 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.7 KiB/s wr, 11 op/s 2024-06-10T01:00:50.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:50 smithi094 ceph-mon[36708]: pgmap v554: 129 pgs: 129 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.7 KiB/s wr, 11 op/s 2024-06-10T01:00:50.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:50 smithi037 ceph-mon[32192]: pgmap v554: 129 pgs: 129 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.7 KiB/s wr, 11 op/s 2024-06-10T01:00:50.668 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:50.669 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:50.719 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:50.719 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:50.719 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 70 -lt 100 ']' 2024-06-10T01:00:50.719 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 70 + 1 2024-06-10T01:00:50.910 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=71 2024-06-10T01:00:50.910 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:50.986 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:50.987 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:50.988 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:50.988 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:50.988 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:51.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:51 smithi167 ceph-mon[36917]: pgmap v555: 129 pgs: 129 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.7 KiB/s wr, 10 op/s 2024-06-10T01:00:51.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:51 smithi094 ceph-mon[36708]: pgmap v555: 129 pgs: 129 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.7 KiB/s wr, 10 op/s 2024-06-10T01:00:51.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:51 smithi037 ceph-mon[32192]: pgmap v555: 129 pgs: 129 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.7 KiB/s wr, 10 op/s 2024-06-10T01:00:51.987 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:51.987 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:52.368 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:52.369 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:52.446 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:52.447 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:52.447 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 71 -lt 100 ']' 2024-06-10T01:00:52.447 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 71 + 1 2024-06-10T01:00:52.617 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=72 2024-06-10T01:00:52.617 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:52.704 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:52.704 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:52.705 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:52.705 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:52.706 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:54.012 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:54.012 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:54.199 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:54.199 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:54.258 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:54.258 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:54.258 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 72 -lt 100 ']' 2024-06-10T01:00:54.258 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 72 + 1 2024-06-10T01:00:54.326 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=73 2024-06-10T01:00:54.326 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:54.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:53 smithi167 ceph-mon[36917]: pgmap v556: 129 pgs: 129 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-06-10T01:00:54.343 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:54.343 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:54.345 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:54.345 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:54.345 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:54.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:53 smithi094 ceph-mon[36708]: pgmap v556: 129 pgs: 129 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-06-10T01:00:54.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:53 smithi037 ceph-mon[32192]: pgmap v556: 129 pgs: 129 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-06-10T01:00:55.253 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:55.254 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:55.726 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:55.726 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:55.749 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:55.749 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:55.749 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 73 -lt 100 ']' 2024-06-10T01:00:55.749 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 73 + 1 2024-06-10T01:00:55.784 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=74 2024-06-10T01:00:55.784 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:55.829 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:55.829 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:55.830 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:55.831 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:55.831 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:56.327 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:55 smithi037 ceph-mon[32192]: pgmap v557: 129 pgs: 129 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.4 KiB/s wr, 12 op/s 2024-06-10T01:00:56.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:55 smithi167 ceph-mon[36917]: pgmap v557: 129 pgs: 129 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.4 KiB/s wr, 12 op/s 2024-06-10T01:00:56.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:55 smithi094 ceph-mon[36708]: pgmap v557: 129 pgs: 129 active+clean; 1.7 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.4 KiB/s wr, 12 op/s 2024-06-10T01:00:57.130 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:57.130 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:57.409 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:57.409 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:57.494 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:57.494 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:57.494 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 74 -lt 100 ']' 2024-06-10T01:00:57.495 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 74 + 1 2024-06-10T01:00:57.592 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=75 2024-06-10T01:00:57.592 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:57.713 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:57.713 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:57.714 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:57.714 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:57.714 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:00:58.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:00:58 smithi167 ceph-mon[36917]: pgmap v558: 129 pgs: 129 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-06-10T01:00:58.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:00:58 smithi094 ceph-mon[36708]: pgmap v558: 129 pgs: 129 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-06-10T01:00:58.385 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:00:58 smithi037 ceph-mon[32192]: pgmap v558: 129 pgs: 129 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-06-10T01:00:59.111 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:59.111 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:59.484 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:00:59.484 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:00:59.500 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:59.500 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:00:59.500 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 75 -lt 100 ']' 2024-06-10T01:00:59.500 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 75 + 1 2024-06-10T01:00:59.549 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=76 2024-06-10T01:00:59.549 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:00:59.603 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:00:59.603 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:00:59.604 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:00:59.604 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:00:59.604 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:01:00.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:00 smithi167 ceph-mon[36917]: pgmap v559: 129 pgs: 129 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 13 op/s 2024-06-10T01:01:00.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:00 smithi094 ceph-mon[36708]: pgmap v559: 129 pgs: 129 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 13 op/s 2024-06-10T01:01:00.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:00 smithi037 ceph-mon[32192]: pgmap v559: 129 pgs: 129 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 13 op/s 2024-06-10T01:01:01.011 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:01.011 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:01.492 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:01:01.492 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:01:01.550 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:01.550 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:01.550 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 76 -lt 100 ']' 2024-06-10T01:01:01.550 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 76 + 1 2024-06-10T01:01:01.600 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=77 2024-06-10T01:01:01.600 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:01:01.689 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:01.689 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:01.690 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:01:01.690 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:01:01.690 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:01:02.113 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:02 smithi094 ceph-mon[36708]: pgmap v560: 129 pgs: 129 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.7 KiB/s wr, 9 op/s 2024-06-10T01:01:02.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:02 smithi167 ceph-mon[36917]: pgmap v560: 129 pgs: 129 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.7 KiB/s wr, 9 op/s 2024-06-10T01:01:02.385 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:02 smithi037 ceph-mon[32192]: pgmap v560: 129 pgs: 129 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.7 KiB/s wr, 9 op/s 2024-06-10T01:01:02.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.543+0000 7fea30a89700 1 -- 172.21.15.37:0/1201206529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea2c1084f0 msgr2=0x7fea2c1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:02.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.543+0000 7fea30a89700 1 --2- 172.21.15.37:0/1201206529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea2c1084f0 0x7fea2c1088d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00b0a0 tx=0x7fea1c00b3b0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.543+0000 7fea30a89700 1 -- 172.21.15.37:0/1201206529 shutdown_connections 2024-06-10T01:01:02.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.543+0000 7fea30a89700 1 --2- 172.21.15.37:0/1201206529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea2c06b3c0 0x7fea2c10f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.543+0000 7fea30a89700 1 --2- 172.21.15.37:0/1201206529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea2c1084f0 0x7fea2c1088d0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.543+0000 7fea30a89700 1 --2- 172.21.15.37:0/1201206529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea2c06aa00 0x7fea2c06ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.543+0000 7fea30a89700 1 -- 172.21.15.37:0/1201206529 >> 172.21.15.37:0/1201206529 conn(0x7fea2c078300 msgr2=0x7fea2c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:02.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.543+0000 7fea30a89700 1 -- 172.21.15.37:0/1201206529 shutdown_connections 2024-06-10T01:01:02.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.543+0000 7fea30a89700 1 -- 172.21.15.37:0/1201206529 wait complete. 2024-06-10T01:01:02.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.544+0000 7fea30a89700 1 Processor -- start 2024-06-10T01:01:02.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.544+0000 7fea30a89700 1 -- start start 2024-06-10T01:01:02.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.544+0000 7fea30a89700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea2c06aa00 0x7fea2c101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:02.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.544+0000 7fea30a89700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea2c06b3c0 0x7fea2c101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:02.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.544+0000 7fea30a89700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea2c1084f0 0x7fea2c101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:02.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.544+0000 7fea30a89700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c112750 con 0x7fea2c06b3c0 2024-06-10T01:01:02.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.544+0000 7fea30a89700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1125d0 con 0x7fea2c06aa00 2024-06-10T01:01:02.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.544+0000 7fea30a89700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1128d0 con 0x7fea2c1084f0 2024-06-10T01:01:02.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.544+0000 7fea29d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea2c06b3c0 0x7fea2c101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:02.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.544+0000 7fea29d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea2c06b3c0 0x7fea2c101990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35156/0 (socket says 172.21.15.37:35156) 2024-06-10T01:01:02.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.544+0000 7fea29d9b700 1 -- 172.21.15.37:0/1266577092 learned_addr learned my addr 172.21.15.37:0/1266577092 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:02.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.544+0000 7fea2a59c700 1 --2- 172.21.15.37:0/1266577092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea2c06aa00 0x7fea2c101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:02.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.545+0000 7fea2ad9d700 1 --2- 172.21.15.37:0/1266577092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea2c1084f0 0x7fea2c101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:02.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.545+0000 7fea29d9b700 1 -- 172.21.15.37:0/1266577092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea2c1084f0 msgr2=0x7fea2c101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:02.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.545+0000 7fea29d9b700 1 --2- 172.21.15.37:0/1266577092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea2c1084f0 0x7fea2c101ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.545+0000 7fea29d9b700 1 -- 172.21.15.37:0/1266577092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea2c06aa00 msgr2=0x7fea2c101450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:02.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.545+0000 7fea29d9b700 1 --2- 172.21.15.37:0/1266577092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea2c06aa00 0x7fea2c101450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.545+0000 7fea29d9b700 1 -- 172.21.15.37:0/1266577092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea2c1a71d0 con 0x7fea2c06b3c0 2024-06-10T01:01:02.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.545+0000 7fea2a59c700 1 --2- 172.21.15.37:0/1266577092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea2c06aa00 0x7fea2c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:01:02.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.545+0000 7fea29d9b700 1 --2- 172.21.15.37:0/1266577092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea2c06b3c0 0x7fea2c101990 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fea2000e9a0 tx=0x7fea2000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:02.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.545+0000 7fea1b7fe700 1 -- 172.21.15.37:0/1266577092 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2000cbd0 con 0x7fea2c06b3c0 2024-06-10T01:01:02.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.545+0000 7fea1b7fe700 1 -- 172.21.15.37:0/1266577092 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea2000cd70 con 0x7fea2c06b3c0 2024-06-10T01:01:02.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.545+0000 7fea30a89700 1 -- 172.21.15.37:0/1266577092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea2c1a74c0 con 0x7fea2c06b3c0 2024-06-10T01:01:02.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.545+0000 7fea1b7fe700 1 -- 172.21.15.37:0/1266577092 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea200187c0 con 0x7fea2c06b3c0 2024-06-10T01:01:02.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.545+0000 7fea30a89700 1 -- 172.21.15.37:0/1266577092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea2c1a7a00 con 0x7fea2c06b3c0 2024-06-10T01:01:02.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.547+0000 7fea1b7fe700 1 -- 172.21.15.37:0/1266577092 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fea20010430 con 0x7fea2c06b3c0 2024-06-10T01:01:02.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.548+0000 7fea1b7fe700 1 --2- 172.21.15.37:0/1266577092 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fea140a2c20 0x7fea140a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:02.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.548+0000 7fea2a59c700 1 --2- 172.21.15.37:0/1266577092 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fea140a2c20 0x7fea140a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:02.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.548+0000 7fea1b7fe700 1 -- 172.21.15.37:0/1266577092 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fea200cf760 con 0x7fea2c06b3c0 2024-06-10T01:01:02.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.549+0000 7fea2a59c700 1 --2- 172.21.15.37:0/1266577092 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fea140a2c20 0x7fea140a50e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00b780 tx=0x7fea1c009780 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:02.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.663+0000 7fea3b30cb80 1 Processor -- start 2024-06-10T01:01:02.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.663+0000 7fea3b30cb80 1 -- start start 2024-06-10T01:01:02.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.664+0000 7fea3b30cb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56354e573640 0x56354e573a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:02.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.664+0000 7fea3b30cb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56354e573f60 0x56354e56d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:02.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.664+0000 7fea3b30cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56354e56df20 0x56354e56e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:02.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.664+0000 7fea3b30cb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56354e4bf810 con 0x56354e573640 2024-06-10T01:01:02.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.664+0000 7fea3b30cb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56354e467720 con 0x56354e573f60 2024-06-10T01:01:02.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.664+0000 7fea3b30cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56354e4bb7d0 con 0x56354e56df20 2024-06-10T01:01:02.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.664+0000 7fea2a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56354e573640 0x56354e573a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:02.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.664+0000 7fea2a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56354e573640 0x56354e573a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35160/0 (socket says 172.21.15.37:35160) 2024-06-10T01:01:02.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.664+0000 7fea2a59c700 1 -- 172.21.15.37:0/1641239138 learned_addr learned my addr 172.21.15.37:0/1641239138 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:02.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.664+0000 7fea2ad9d700 1 --2- 172.21.15.37:0/1641239138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56354e56df20 0x56354e56e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:02.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.664+0000 7fea29d9b700 1 --2- 172.21.15.37:0/1641239138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56354e573f60 0x56354e56d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:02.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.664+0000 7fea2a59c700 1 -- 172.21.15.37:0/1641239138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56354e56df20 msgr2=0x56354e56e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:02.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.664+0000 7fea2a59c700 1 --2- 172.21.15.37:0/1641239138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56354e56df20 0x56354e56e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.664+0000 7fea2a59c700 1 -- 172.21.15.37:0/1641239138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56354e573f60 msgr2=0x56354e56d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:02.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.664+0000 7fea2a59c700 1 --2- 172.21.15.37:0/1641239138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56354e573f60 0x56354e56d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.664+0000 7fea2a59c700 1 -- 172.21.15.37:0/1641239138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56354e45df50 con 0x56354e573640 2024-06-10T01:01:02.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.665+0000 7fea2a59c700 1 --2- 172.21.15.37:0/1641239138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56354e573640 0x56354e573a20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00b750 tx=0x7fea1c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9200a376899473 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:02.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.665+0000 7fea18ff9700 1 -- 172.21.15.37:0/1641239138 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c007940 con 0x56354e573640 2024-06-10T01:01:02.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.665+0000 7fea18ff9700 1 -- 172.21.15.37:0/1641239138 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea1c02e070 con 0x56354e573640 2024-06-10T01:01:02.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.665+0000 7fea18ff9700 1 -- 172.21.15.37:0/1641239138 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c01cdc0 con 0x56354e573640 2024-06-10T01:01:02.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.666+0000 7fea3b30cb80 1 -- 172.21.15.37:0/1641239138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56354e573640 msgr2=0x56354e573a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:02.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.666+0000 7fea3b30cb80 1 --2- 172.21.15.37:0/1641239138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56354e573640 0x56354e573a20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00b750 tx=0x7fea1c01c000 comp rx=0 tx=0).stop 2024-06-10T01:01:02.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.666+0000 7fea3b30cb80 1 -- 172.21.15.37:0/1641239138 shutdown_connections 2024-06-10T01:01:02.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.666+0000 7fea3b30cb80 1 --2- 172.21.15.37:0/1641239138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56354e56df20 0x56354e56e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.666+0000 7fea3b30cb80 1 --2- 172.21.15.37:0/1641239138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56354e573640 0x56354e573a20 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.666+0000 7fea3b30cb80 1 --2- 172.21.15.37:0/1641239138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56354e573f60 0x56354e56d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.666+0000 7fea3b30cb80 1 -- 172.21.15.37:0/1641239138 >> 172.21.15.37:0/1641239138 conn(0x56354e498cc0 msgr2=0x56354e568850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:02.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.666+0000 7fea3b30cb80 1 -- 172.21.15.37:0/1641239138 shutdown_connections 2024-06-10T01:01:02.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.666+0000 7fea3b30cb80 1 -- 172.21.15.37:0/1641239138 wait complete. 2024-06-10T01:01:02.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.666+0000 7fea3b30cb80 1 -- start start 2024-06-10T01:01:02.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.666+0000 7fea3b30cb80 1 Processor -- start 2024-06-10T01:01:02.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.667+0000 7fea3b30cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56354e56df20 0x56354e579d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:02.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.667+0000 7fea3b30cb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56354e573640 0x56354e577030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:02.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.667+0000 7fea3b30cb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56354e573f60 0x56354e577570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:02.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.667+0000 7fea3b30cb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56354e57a970 con 0x56354e573640 2024-06-10T01:01:02.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.667+0000 7fea3b30cb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56354e45fbf0 con 0x56354e573f60 2024-06-10T01:01:02.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.667+0000 7fea3b30cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56354e57aaf0 con 0x56354e56df20 2024-06-10T01:01:02.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.667+0000 7fea3b30cb80 10 client.0 _ll_register_callbacks cb 0x56354e459920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:01:02.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.667+0000 7fea29d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56354e573640 0x56354e577030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:02.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.667+0000 7fea29d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56354e573640 0x56354e577030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35174/0 (socket says 172.21.15.37:35174) 2024-06-10T01:01:02.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.667+0000 7fea29d9b700 1 -- 172.21.15.37:0/812503331 learned_addr learned my addr 172.21.15.37:0/812503331 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:02.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.667+0000 7fea2a59c700 1 --2- 172.21.15.37:0/812503331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56354e56df20 0x56354e579d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:02.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.667+0000 7fea2ad9d700 1 --2- 172.21.15.37:0/812503331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56354e573f60 0x56354e577570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:02.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.667+0000 7fea29d9b700 1 -- 172.21.15.37:0/812503331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56354e56df20 msgr2=0x56354e579d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:02.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.667+0000 7fea29d9b700 1 --2- 172.21.15.37:0/812503331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56354e56df20 0x56354e579d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.667+0000 7fea29d9b700 1 -- 172.21.15.37:0/812503331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56354e573f60 msgr2=0x56354e577570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:02.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.667+0000 7fea29d9b700 1 --2- 172.21.15.37:0/812503331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56354e573f60 0x56354e577570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.667+0000 7fea29d9b700 1 -- 172.21.15.37:0/812503331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56354e40e8d0 con 0x56354e573640 2024-06-10T01:01:02.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.667+0000 7fea2ad9d700 1 --2- 172.21.15.37:0/812503331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56354e573f60 0x56354e577570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:01:02.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.667+0000 7fea29d9b700 1 --2- 172.21.15.37:0/812503331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56354e573640 0x56354e577030 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fea200cce10 tx=0x7fea2009b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d530abc807fc6afb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:02.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.667+0000 7fea11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:01:02.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.668+0000 7fea11ffb700 1 -- 172.21.15.37:0/812503331 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea200a47e0 con 0x56354e573640 2024-06-10T01:01:02.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.668+0000 7fea11ffb700 1 -- 172.21.15.37:0/812503331 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fea2009bdd0 con 0x56354e573640 2024-06-10T01:01:02.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.668+0000 7fea3b30cb80 1 -- 172.21.15.37:0/812503331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea1c02e070 con 0x56354e573640 2024-06-10T01:01:02.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.668+0000 7fea11ffb700 1 -- 172.21.15.37:0/812503331 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fea2009bfd0 con 0x56354e573640 2024-06-10T01:01:02.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.668+0000 7fea3b30cb80 10 client.16089 fetch_fsmap learned FSMap version 27 2024-06-10T01:01:02.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.668+0000 7fea3b30cb80 1 -- 172.21.15.37:0/812503331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56354e585380 con 0x56354e573640 2024-06-10T01:01:02.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.668+0000 7fea11ffb700 1 -- 172.21.15.37:0/812503331 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea1c02e070 con 0x56354e573640 2024-06-10T01:01:02.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.668+0000 7fea11ffb700 1 -- 172.21.15.37:0/812503331 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fea2009b130 con 0x56354e573640 2024-06-10T01:01:02.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.668+0000 7fea3b30cb80 10 client.16089 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:01:02.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.668+0000 7fea3b30cb80 10 client.16089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:01:02.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.668+0000 7fea3b30cb80 20 client.16089 populate_metadata read hostname 'smithi037' 2024-06-10T01:01:02.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.668+0000 7fea3b30cb80 1 --2- 172.21.15.37:0/812503331 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56354e587910 0x56354e589d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:02.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.668+0000 7fea3b30cb80 4 client.16089 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:01:02.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.668+0000 7fea3b30cb80 1 -- 172.21.15.37:0/812503331 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56354e4ccd10 con 0x56354e587910 2024-06-10T01:01:02.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.669+0000 7fea2a59c700 1 --2- 172.21.15.37:0/812503331 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56354e587910 0x56354e589d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:02.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.669+0000 7fea2a59c700 1 --2- 172.21.15.37:0/812503331 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56354e587910 0x56354e589d30 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7629ae95fc64facf server_cookie=5a1cde5a5221f8b1 in_seq=0 out_seq=0 2024-06-10T01:01:02.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.669+0000 7fea11ffb700 10 client.16089 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:02.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.670+0000 7fea11ffb700 1 -- 172.21.15.37:0/812503331 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56354e4ccd10 con 0x56354e587910 2024-06-10T01:01:02.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.670+0000 7fea11ffb700 10 client.16089 handle_command_reply: tid=0 2024-06-10T01:01:02.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.670+0000 7fea3b30cb80 1 client.16089 shutdown 2024-06-10T01:01:02.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.670+0000 7fea3b30cb80 1 -- 172.21.15.37:0/812503331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56354e573640 msgr2=0x56354e577030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:02.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.670+0000 7fea3b30cb80 1 --2- 172.21.15.37:0/812503331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56354e573640 0x56354e577030 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fea200cce10 tx=0x7fea2009b600 comp rx=0 tx=0).stop 2024-06-10T01:01:02.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.670+0000 7fea3b30cb80 1 -- 172.21.15.37:0/812503331 shutdown_connections 2024-06-10T01:01:02.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.670+0000 7fea3b30cb80 1 --2- 172.21.15.37:0/812503331 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56354e587910 0x56354e589d30 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.670+0000 7fea3b30cb80 1 --2- 172.21.15.37:0/812503331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56354e573f60 0x56354e577570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.670+0000 7fea3b30cb80 1 --2- 172.21.15.37:0/812503331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56354e56df20 0x56354e579d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.670+0000 7fea3b30cb80 1 --2- 172.21.15.37:0/812503331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56354e573640 0x56354e577030 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.670+0000 7fea3b30cb80 1 -- 172.21.15.37:0/812503331 >> 172.21.15.37:0/812503331 conn(0x56354e498cc0 msgr2=0x56354e5685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:02.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.670+0000 7fea11ffb700 0 client.16089 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:02.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.670+0000 7fea3b30cb80 1 -- 172.21.15.37:0/812503331 shutdown_connections 2024-06-10T01:01:02.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.670+0000 7fea3b30cb80 1 -- 172.21.15.37:0/812503331 wait complete. 2024-06-10T01:01:02.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.671+0000 7fea3b30cb80 20 client.16089 trim_cache size 0 max 16384 2024-06-10T01:01:02.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.691+0000 7fea3b30cb80 1 Processor -- start 2024-06-10T01:01:02.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.691+0000 7fea3b30cb80 1 -- start start 2024-06-10T01:01:02.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.691+0000 7fea3b30cb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56354e56df20 0x56354e56d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:02.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.691+0000 7fea3b30cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56354e573640 0x56354e562690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:02.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.691+0000 7fea3b30cb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56354e573f60 0x56354e575f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:02.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.691+0000 7fea3b30cb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1b2f50 con 0x56354e56df20 2024-06-10T01:01:02.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.691+0000 7fea3b30cb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1b07e0 con 0x56354e573f60 2024-06-10T01:01:02.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.692+0000 7fea3b30cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1ae630 con 0x56354e573640 2024-06-10T01:01:02.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.692+0000 7fea2a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56354e56df20 0x56354e56d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:02.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.692+0000 7fea2a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56354e56df20 0x56354e56d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35190/0 (socket says 172.21.15.37:35190) 2024-06-10T01:01:02.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.692+0000 7fea2a59c700 1 -- 172.21.15.37:0/1489456197 learned_addr learned my addr 172.21.15.37:0/1489456197 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:02.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.692+0000 7fea2ad9d700 1 --2- 172.21.15.37:0/1489456197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56354e573f60 0x56354e575f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:02.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.692+0000 7fea29d9b700 1 --2- 172.21.15.37:0/1489456197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56354e573640 0x56354e562690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:02.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.692+0000 7fea2a59c700 1 -- 172.21.15.37:0/1489456197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56354e573640 msgr2=0x56354e562690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:02.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.692+0000 7fea2a59c700 1 --2- 172.21.15.37:0/1489456197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56354e573640 0x56354e562690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.692+0000 7fea2a59c700 1 -- 172.21.15.37:0/1489456197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56354e573f60 msgr2=0x56354e575f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:02.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.692+0000 7fea2a59c700 1 --2- 172.21.15.37:0/1489456197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56354e573f60 0x56354e575f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.692+0000 7fea2a59c700 1 -- 172.21.15.37:0/1489456197 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56354e579970 con 0x56354e56df20 2024-06-10T01:01:02.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.692+0000 7fea2a59c700 1 --2- 172.21.15.37:0/1489456197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56354e56df20 0x56354e56d9e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fea1c01c8d0 tx=0x7fea1c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2088e67b773f56 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:02.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.692+0000 7fea13fff700 1 -- 172.21.15.37:0/1489456197 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c040070 con 0x56354e56df20 2024-06-10T01:01:02.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.692+0000 7fea13fff700 1 -- 172.21.15.37:0/1489456197 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea1c021070 con 0x56354e56df20 2024-06-10T01:01:02.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.692+0000 7fea13fff700 1 -- 172.21.15.37:0/1489456197 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c03a070 con 0x56354e56df20 2024-06-10T01:01:02.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.693+0000 7fea3b30cb80 1 -- 172.21.15.37:0/1489456197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56354e56df20 msgr2=0x56354e56d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:02.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.693+0000 7fea3b30cb80 1 --2- 172.21.15.37:0/1489456197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56354e56df20 0x56354e56d9e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fea1c01c8d0 tx=0x7fea1c0099e0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.693+0000 7fea3b30cb80 1 -- 172.21.15.37:0/1489456197 shutdown_connections 2024-06-10T01:01:02.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.693+0000 7fea3b30cb80 1 --2- 172.21.15.37:0/1489456197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56354e573f60 0x56354e575f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.693+0000 7fea3b30cb80 1 --2- 172.21.15.37:0/1489456197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56354e56df20 0x56354e56d9e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.693+0000 7fea3b30cb80 1 --2- 172.21.15.37:0/1489456197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56354e573640 0x56354e562690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.693+0000 7fea3b30cb80 1 -- 172.21.15.37:0/1489456197 >> 172.21.15.37:0/1489456197 conn(0x56354e498cc0 msgr2=0x56354e5757f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:02.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.693+0000 7fea3b30cb80 1 -- 172.21.15.37:0/1489456197 shutdown_connections 2024-06-10T01:01:02.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.693+0000 7fea3b30cb80 1 -- 172.21.15.37:0/1489456197 wait complete. 2024-06-10T01:01:02.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.693+0000 7fea3b30cb80 1 -- start start 2024-06-10T01:01:02.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.694+0000 7fea3b30cb80 1 Processor -- start 2024-06-10T01:01:02.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.694+0000 7fea3b30cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56354e56df20 0x56354e580250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:02.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.694+0000 7fea3b30cb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56354e573640 0x56354e57b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:02.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.694+0000 7fea3b30cb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56354e573f60 0x56354e57bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:02.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.694+0000 7fea3b30cb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1aeca0 con 0x56354e573640 2024-06-10T01:01:02.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.694+0000 7fea3b30cb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1b0110 con 0x56354e573f60 2024-06-10T01:01:02.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.694+0000 7fea3b30cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1af370 con 0x56354e56df20 2024-06-10T01:01:02.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.694+0000 7fea3b30cb80 10 client.0 _ll_register_callbacks cb 0x56354e459920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:01:02.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.694+0000 7fea29d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56354e573640 0x56354e57b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:02.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.694+0000 7fea2a59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56354e56df20 0x56354e580250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:02.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.694+0000 7fea29d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56354e573640 0x56354e57b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35206/0 (socket says 172.21.15.37:35206) 2024-06-10T01:01:02.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.694+0000 7fea29d9b700 1 -- 172.21.15.37:0/1732385148 learned_addr learned my addr 172.21.15.37:0/1732385148 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:02.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.694+0000 7fea2ad9d700 1 --2- 172.21.15.37:0/1732385148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56354e573f60 0x56354e57bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:02.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.694+0000 7fea29d9b700 1 -- 172.21.15.37:0/1732385148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56354e56df20 msgr2=0x56354e580250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:02.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.694+0000 7fea29d9b700 1 --2- 172.21.15.37:0/1732385148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56354e56df20 0x56354e580250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.694+0000 7fea29d9b700 1 -- 172.21.15.37:0/1732385148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56354e573f60 msgr2=0x56354e57bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:02.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.694+0000 7fea29d9b700 1 --2- 172.21.15.37:0/1732385148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56354e573f60 0x56354e57bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.694+0000 7fea29d9b700 1 -- 172.21.15.37:0/1732385148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56354e57c080 con 0x56354e573640 2024-06-10T01:01:02.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.695+0000 7fea29d9b700 1 --2- 172.21.15.37:0/1732385148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56354e573640 0x56354e57b5d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fea2009b870 tx=0x7fea2000c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ceda661f8f000e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:02.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.695+0000 7fea11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:01:02.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.695+0000 7fea11ffb700 1 -- 172.21.15.37:0/1732385148 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20097e10 con 0x56354e573640 2024-06-10T01:01:02.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.695+0000 7fea11ffb700 1 -- 172.21.15.37:0/1732385148 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fea2009acc0 con 0x56354e573640 2024-06-10T01:01:02.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.695+0000 7fea3b30cb80 1 -- 172.21.15.37:0/1732385148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea1c021070 con 0x56354e573640 2024-06-10T01:01:02.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.695+0000 7fea11ffb700 1 -- 172.21.15.37:0/1732385148 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fea200a10b0 con 0x56354e573640 2024-06-10T01:01:02.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.695+0000 7fea3b30cb80 10 client.16095 fetch_fsmap learned FSMap version 27 2024-06-10T01:01:02.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.695+0000 7fea3b30cb80 1 -- 172.21.15.37:0/1732385148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56354e5f23a0 con 0x56354e573640 2024-06-10T01:01:02.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.695+0000 7fea11ffb700 1 -- 172.21.15.37:0/1732385148 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea1c021070 con 0x56354e573640 2024-06-10T01:01:02.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.695+0000 7fea11ffb700 1 -- 172.21.15.37:0/1732385148 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fea200ba350 con 0x56354e573640 2024-06-10T01:01:02.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.695+0000 7fea3b30cb80 10 client.16095 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:01:02.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.695+0000 7fea3b30cb80 10 client.16095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:01:02.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.695+0000 7fea3b30cb80 20 client.16095 populate_metadata read hostname 'smithi037' 2024-06-10T01:01:02.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.695+0000 7fea3b30cb80 1 --2- 172.21.15.37:0/1732385148 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56354e5f2b00 0x56354e5f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:02.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.695+0000 7fea3b30cb80 4 client.16095 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:01:02.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.695+0000 7fea3b30cb80 1 -- 172.21.15.37:0/1732385148 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56354e4c7290 con 0x56354e5f2b00 2024-06-10T01:01:02.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.696+0000 7fea2a59c700 1 --2- 172.21.15.37:0/1732385148 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56354e5f2b00 0x56354e5f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:02.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.696+0000 7fea2a59c700 1 --2- 172.21.15.37:0/1732385148 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56354e5f2b00 0x56354e5f4f20 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79463bc9fee936a server_cookie=70151820b173561b in_seq=0 out_seq=0 2024-06-10T01:01:02.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.696+0000 7fea11ffb700 10 client.16095 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:02.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.696+0000 7fea11ffb700 1 -- 172.21.15.37:0/1732385148 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56354e4c7290 con 0x56354e5f2b00 2024-06-10T01:01:02.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.696+0000 7fea11ffb700 10 client.16095 handle_command_reply: tid=0 2024-06-10T01:01:02.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.696+0000 7fea3b30cb80 1 client.16095 shutdown 2024-06-10T01:01:02.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.697+0000 7fea3b30cb80 1 -- 172.21.15.37:0/1732385148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56354e573640 msgr2=0x56354e57b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:02.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.697+0000 7fea3b30cb80 1 --2- 172.21.15.37:0/1732385148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56354e573640 0x56354e57b5d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fea2009b870 tx=0x7fea2000c6e0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.697+0000 7fea3b30cb80 1 -- 172.21.15.37:0/1732385148 shutdown_connections 2024-06-10T01:01:02.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.697+0000 7fea3b30cb80 1 --2- 172.21.15.37:0/1732385148 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56354e5f2b00 0x56354e5f4f20 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.697+0000 7fea3b30cb80 1 --2- 172.21.15.37:0/1732385148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56354e573f60 0x56354e57bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.697+0000 7fea3b30cb80 1 --2- 172.21.15.37:0/1732385148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56354e56df20 0x56354e580250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.697+0000 7fea3b30cb80 1 --2- 172.21.15.37:0/1732385148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56354e573640 0x56354e57b5d0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.697+0000 7fea3b30cb80 1 -- 172.21.15.37:0/1732385148 >> 172.21.15.37:0/1732385148 conn(0x56354e498cc0 msgr2=0x56354e5790f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:02.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.697+0000 7fea11ffb700 0 client.16095 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:02.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.697+0000 7fea3b30cb80 1 -- 172.21.15.37:0/1732385148 shutdown_connections 2024-06-10T01:01:02.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.697+0000 7fea3b30cb80 1 -- 172.21.15.37:0/1732385148 wait complete. 2024-06-10T01:01:02.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.697+0000 7fea3b30cb80 20 client.16095 trim_cache size 0 max 16384 2024-06-10T01:01:02.708 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:01:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-06-10T01:01:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-06-10T01:01:02.709 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:01:02.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.697+0000 7fea30a89700 1 -- 172.21.15.37:0/1266577092 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fea140a2c20 msgr2=0x7fea140a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:02.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.698+0000 7fea30a89700 1 --2- 172.21.15.37:0/1266577092 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fea140a2c20 0x7fea140a50e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00b780 tx=0x7fea1c009780 comp rx=0 tx=0).stop 2024-06-10T01:01:02.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.698+0000 7fea30a89700 1 -- 172.21.15.37:0/1266577092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea2c06b3c0 msgr2=0x7fea2c101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:02.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.698+0000 7fea30a89700 1 --2- 172.21.15.37:0/1266577092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea2c06b3c0 0x7fea2c101990 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fea2000e9a0 tx=0x7fea2000ed60 comp rx=0 tx=0).stop 2024-06-10T01:01:02.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.698+0000 7fea30a89700 1 -- 172.21.15.37:0/1266577092 shutdown_connections 2024-06-10T01:01:02.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.698+0000 7fea30a89700 1 --2- 172.21.15.37:0/1266577092 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fea140a2c20 0x7fea140a50e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.698+0000 7fea30a89700 1 --2- 172.21.15.37:0/1266577092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea2c1084f0 0x7fea2c101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.698+0000 7fea30a89700 1 --2- 172.21.15.37:0/1266577092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea2c06aa00 0x7fea2c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.698+0000 7fea30a89700 1 --2- 172.21.15.37:0/1266577092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea2c06b3c0 0x7fea2c101990 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:02.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.698+0000 7fea30a89700 1 -- 172.21.15.37:0/1266577092 >> 172.21.15.37:0/1266577092 conn(0x7fea2c078300 msgr2=0x7fea2c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:02.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.698+0000 7fea30a89700 1 -- 172.21.15.37:0/1266577092 shutdown_connections 2024-06-10T01:01:02.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:02.698+0000 7fea30a89700 1 -- 172.21.15.37:0/1266577092 wait complete. 2024-06-10T01:01:02.919 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:02.919 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:03.822 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:01:03.822 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:01:03.858 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:03.858 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:03.858 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 77 -lt 100 ']' 2024-06-10T01:01:03.858 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 77 + 1 2024-06-10T01:01:04.017 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=78 2024-06-10T01:01:04.018 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:01:04.041 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:04.041 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:04.042 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:01:04.043 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:01:04.043 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:01:04.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:04 smithi167 ceph-mon[36917]: pgmap v561: 129 pgs: 129 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-06-10T01:01:04.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:04 smithi094 ceph-mon[36708]: pgmap v561: 129 pgs: 129 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-06-10T01:01:04.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:04 smithi037 ceph-mon[32192]: pgmap v561: 129 pgs: 129 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-06-10T01:01:05.314 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:05.314 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:05.516 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:01:05.516 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:01:05.564 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:05.564 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:05.564 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 78 -lt 100 ']' 2024-06-10T01:01:05.564 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 78 + 1 2024-06-10T01:01:05.756 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=79 2024-06-10T01:01:05.756 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:01:05.853 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:05.853 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:05.854 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:01:05.855 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:01:05.855 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:01:06.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.272+0000 7f111c363640 1 -- 172.21.15.37:0/4032989312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1114101a30 msgr2=0x7f1114101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:06.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.272+0000 7f111c363640 1 --2- 172.21.15.37:0/4032989312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1114101a30 0x7f1114101e50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f1110009ca0 tx=0x7f111002a8b0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.273+0000 7f111c363640 1 -- 172.21.15.37:0/4032989312 shutdown_connections 2024-06-10T01:01:06.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.273+0000 7f111c363640 1 --2- 172.21.15.37:0/4032989312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11141035e0 0x7f1114109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.273+0000 7f111c363640 1 --2- 172.21.15.37:0/4032989312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1114102c20 0x7f11141030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.273+0000 7f111c363640 1 --2- 172.21.15.37:0/4032989312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1114101a30 0x7f1114101e50 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.273+0000 7f111c363640 1 -- 172.21.15.37:0/4032989312 >> 172.21.15.37:0/4032989312 conn(0x7f11140fcfc0 msgr2=0x7f11140ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:06.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.273+0000 7f111c363640 1 -- 172.21.15.37:0/4032989312 shutdown_connections 2024-06-10T01:01:06.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.273+0000 7f111c363640 1 -- 172.21.15.37:0/4032989312 wait complete. 2024-06-10T01:01:06.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:06 smithi167 ceph-mon[36917]: pgmap v562: 129 pgs: 129 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.6 KiB/s wr, 7 op/s 2024-06-10T01:01:06.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:06 smithi094 ceph-mon[36708]: pgmap v562: 129 pgs: 129 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.6 KiB/s wr, 7 op/s 2024-06-10T01:01:06.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:06 smithi037 ceph-mon[32192]: pgmap v562: 129 pgs: 129 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.6 KiB/s wr, 7 op/s 2024-06-10T01:01:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.425+0000 7f111c363640 1 Processor -- start 2024-06-10T01:01:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.425+0000 7f111c363640 1 -- start start 2024-06-10T01:01:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.425+0000 7f111c363640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1114101a30 0x7f1114199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.425+0000 7f111c363640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1114102c20 0x7f111419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.425+0000 7f111c363640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11141035e0 0x7f11141a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.425+0000 7f111c363640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f111410d680 con 0x7f1114102c20 2024-06-10T01:01:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.425+0000 7f111c363640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f111410d500 con 0x7f1114101a30 2024-06-10T01:01:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.425+0000 7f111c363640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f111410d800 con 0x7f11141035e0 2024-06-10T01:01:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.425+0000 7f11198fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1114102c20 0x7f111419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.426+0000 7f11198fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1114102c20 0x7f111419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35230/0 (socket says 172.21.15.37:35230) 2024-06-10T01:01:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.426+0000 7f11198fe640 1 -- 172.21.15.37:0/1595084736 learned_addr learned my addr 172.21.15.37:0/1595084736 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.426+0000 7f111a0ff640 1 --2- 172.21.15.37:0/1595084736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1114101a30 0x7f1114199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.426+0000 7f111a900640 1 --2- 172.21.15.37:0/1595084736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11141035e0 0x7f11141a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.426+0000 7f11198fe640 1 -- 172.21.15.37:0/1595084736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11141035e0 msgr2=0x7f11141a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.426+0000 7f11198fe640 1 --2- 172.21.15.37:0/1595084736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11141035e0 0x7f11141a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.426+0000 7f11198fe640 1 -- 172.21.15.37:0/1595084736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1114101a30 msgr2=0x7f1114199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.426+0000 7f11198fe640 1 --2- 172.21.15.37:0/1595084736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1114101a30 0x7f1114199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.426+0000 7f11198fe640 1 -- 172.21.15.37:0/1595084736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11141a1ad0 con 0x7f1114102c20 2024-06-10T01:01:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.426+0000 7f111a0ff640 1 --2- 172.21.15.37:0/1595084736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1114101a30 0x7f1114199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:01:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.426+0000 7f11198fe640 1 --2- 172.21.15.37:0/1595084736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1114102c20 0x7f111419a440 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f110400ea60 tx=0x7f110400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.453+0000 7f110b7fe640 1 -- 172.21.15.37:0/1595084736 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110400cde0 con 0x7f1114102c20 2024-06-10T01:01:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.453+0000 7f110b7fe640 1 -- 172.21.15.37:0/1595084736 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11040035f0 con 0x7f1114102c20 2024-06-10T01:01:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.453+0000 7f110b7fe640 1 -- 172.21.15.37:0/1595084736 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1104010670 con 0x7f1114102c20 2024-06-10T01:01:06.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.453+0000 7f111c363640 1 -- 172.21.15.37:0/1595084736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11141a1dc0 con 0x7f1114102c20 2024-06-10T01:01:06.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.456+0000 7f110b7fe640 1 -- 172.21.15.37:0/1595084736 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f11040037a0 con 0x7f1114102c20 2024-06-10T01:01:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.456+0000 7f110b7fe640 1 --2- 172.21.15.37:0/1595084736 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f11000a2da0 0x7f11000a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.457+0000 7f111a0ff640 1 --2- 172.21.15.37:0/1595084736 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f11000a2da0 0x7f11000a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.457+0000 7f111a0ff640 1 --2- 172.21.15.37:0/1595084736 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f11000a2da0 0x7f11000a5260 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1110000f80 tx=0x7f1110030750 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.466+0000 7f111c363640 1 -- 172.21.15.37:0/1595084736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11141a2280 con 0x7f1114102c20 2024-06-10T01:01:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.466+0000 7f110b7fe640 1 -- 172.21.15.37:0/1595084736 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f1104099220 con 0x7f1114102c20 2024-06-10T01:01:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.834+0000 7f111f06a740 1 Processor -- start 2024-06-10T01:01:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.835+0000 7f111f06a740 1 -- start start 2024-06-10T01:01:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.835+0000 7f111f06a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa2f2e8010 0x55aa2f2e8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.835+0000 7f111f06a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa2f2e8a00 0x55aa2f2ed500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.835+0000 7f111f06a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa2f2edb70 0x55aa2f2f0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.835+0000 7f111f06a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55aa2f24b4f0 con 0x55aa2f2e8010 2024-06-10T01:01:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.835+0000 7f111f06a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55aa2f1e9870 con 0x55aa2f2edb70 2024-06-10T01:01:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.835+0000 7f111f06a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55aa2f2474b0 con 0x55aa2f2e8a00 2024-06-10T01:01:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.835+0000 7f111a0ff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa2f2e8010 0x55aa2f2e8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.835+0000 7f111a0ff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa2f2e8010 0x55aa2f2e8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35240/0 (socket says 172.21.15.37:35240) 2024-06-10T01:01:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.835+0000 7f111a0ff640 1 -- 172.21.15.37:0/3491375867 learned_addr learned my addr 172.21.15.37:0/3491375867 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.835+0000 7f111a900640 1 --2- 172.21.15.37:0/3491375867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa2f2edb70 0x55aa2f2f0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.835+0000 7f11198fe640 1 --2- 172.21.15.37:0/3491375867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa2f2e8a00 0x55aa2f2ed500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.836+0000 7f111a0ff640 1 -- 172.21.15.37:0/3491375867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa2f2e8a00 msgr2=0x55aa2f2ed500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.836+0000 7f111a0ff640 1 --2- 172.21.15.37:0/3491375867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa2f2e8a00 0x55aa2f2ed500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.836+0000 7f111a0ff640 1 -- 172.21.15.37:0/3491375867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa2f2edb70 msgr2=0x55aa2f2f0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.836+0000 7f111a0ff640 1 --2- 172.21.15.37:0/3491375867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa2f2edb70 0x55aa2f2f0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.836+0000 7f111a0ff640 1 -- 172.21.15.37:0/3491375867 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa2f2059b0 con 0x55aa2f2e8010 2024-06-10T01:01:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.836+0000 7f111a0ff640 1 --2- 172.21.15.37:0/3491375867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa2f2e8010 0x55aa2f2e8430 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f11100307c0 tx=0x7f1110040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32fc97927e947f72 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.836+0000 7f1108bfe640 1 -- 172.21.15.37:0/3491375867 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110051670 con 0x55aa2f2e8010 2024-06-10T01:01:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.836+0000 7f1108bfe640 1 -- 172.21.15.37:0/3491375867 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f111004c070 con 0x55aa2f2e8010 2024-06-10T01:01:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.836+0000 7f1108bfe640 1 -- 172.21.15.37:0/3491375867 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111003dc90 con 0x55aa2f2e8010 2024-06-10T01:01:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.837+0000 7f111f06a740 1 -- 172.21.15.37:0/3491375867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa2f2e8010 msgr2=0x55aa2f2e8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.837+0000 7f111f06a740 1 --2- 172.21.15.37:0/3491375867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa2f2e8010 0x55aa2f2e8430 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f11100307c0 tx=0x7f1110040000 comp rx=0 tx=0).stop 2024-06-10T01:01:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.837+0000 7f111f06a740 1 -- 172.21.15.37:0/3491375867 shutdown_connections 2024-06-10T01:01:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.837+0000 7f111f06a740 1 --2- 172.21.15.37:0/3491375867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa2f2edb70 0x55aa2f2f0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.837+0000 7f111f06a740 1 --2- 172.21.15.37:0/3491375867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa2f2e8a00 0x55aa2f2ed500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.837+0000 7f111f06a740 1 --2- 172.21.15.37:0/3491375867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa2f2e8010 0x55aa2f2e8430 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.837+0000 7f111f06a740 1 -- 172.21.15.37:0/3491375867 >> 172.21.15.37:0/3491375867 conn(0x55aa2f2e3680 msgr2=0x55aa2f2e3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.838+0000 7f111f06a740 1 -- 172.21.15.37:0/3491375867 shutdown_connections 2024-06-10T01:01:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.838+0000 7f111f06a740 1 -- 172.21.15.37:0/3491375867 wait complete. 2024-06-10T01:01:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.871+0000 7f111f06a740 1 -- start start 2024-06-10T01:01:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.877+0000 7f111f06a740 1 Processor -- start 2024-06-10T01:01:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.877+0000 7f111f06a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa2f2e8010 0x55aa2f2f21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.877+0000 7f111f06a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa2f2e8a00 0x55aa2f2f2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.877+0000 7f111f06a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa2f2edb70 0x55aa2f2f9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.877+0000 7f111f06a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55aa2f2f0ac0 con 0x55aa2f2e8010 2024-06-10T01:01:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.877+0000 7f111f06a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55aa2f1d4ae0 con 0x55aa2f2e8a00 2024-06-10T01:01:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.877+0000 7f111f06a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55aa2f2f0c40 con 0x55aa2f2edb70 2024-06-10T01:01:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.877+0000 7f111a0ff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa2f2e8010 0x55aa2f2f21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.877+0000 7f111a0ff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa2f2e8010 0x55aa2f2f21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35244/0 (socket says 172.21.15.37:35244) 2024-06-10T01:01:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.877+0000 7f111a0ff640 1 -- 172.21.15.37:0/1943110543 learned_addr learned my addr 172.21.15.37:0/1943110543 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.877+0000 7f111a900640 1 --2- 172.21.15.37:0/1943110543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa2f2edb70 0x55aa2f2f9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.877+0000 7f11198fe640 1 --2- 172.21.15.37:0/1943110543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa2f2e8a00 0x55aa2f2f2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.878+0000 7f111a0ff640 1 -- 172.21.15.37:0/1943110543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa2f2edb70 msgr2=0x55aa2f2f9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.878+0000 7f111a0ff640 1 --2- 172.21.15.37:0/1943110543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa2f2edb70 0x55aa2f2f9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.878+0000 7f111a0ff640 1 -- 172.21.15.37:0/1943110543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa2f2e8a00 msgr2=0x55aa2f2f2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.878+0000 7f111a0ff640 1 --2- 172.21.15.37:0/1943110543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa2f2e8a00 0x55aa2f2f2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.878+0000 7f111a0ff640 1 -- 172.21.15.37:0/1943110543 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa2ef5f210 con 0x55aa2f2e8010 2024-06-10T01:01:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.878+0000 7f111a900640 1 --2- 172.21.15.37:0/1943110543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa2f2edb70 0x55aa2f2f9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:01:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.878+0000 7f11198fe640 1 --2- 172.21.15.37:0/1943110543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa2f2e8a00 0x55aa2f2f2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:01:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.878+0000 7f111f06a740 10 client.0 _ll_register_callbacks cb 0x55aa2f20d9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:01:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.878+0000 7f111a0ff640 1 --2- 172.21.15.37:0/1943110543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa2f2e8010 0x55aa2f2f21c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1110009bf0 tx=0x7f111003dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2909025ff8d27d24 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.878+0000 7f10fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:01:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.878+0000 7f10fdffb640 1 -- 172.21.15.37:0/1943110543 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110051880 con 0x55aa2f2e8010 2024-06-10T01:01:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.878+0000 7f10fdffb640 1 -- 172.21.15.37:0/1943110543 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f1110051a20 con 0x55aa2f2e8010 2024-06-10T01:01:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.892+0000 7f111f06a740 1 -- 172.21.15.37:0/1943110543 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f111004c070 con 0x55aa2f2e8010 2024-06-10T01:01:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.893+0000 7f10fdffb640 1 -- 172.21.15.37:0/1943110543 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1110051c20 con 0x55aa2f2e8010 2024-06-10T01:01:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.893+0000 7f111f06a740 10 client.16107 fetch_fsmap learned FSMap version 27 2024-06-10T01:01:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.893+0000 7f111f06a740 1 -- 172.21.15.37:0/1943110543 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55aa2f02b620 con 0x55aa2f2e8010 2024-06-10T01:01:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.936+0000 7f10fdffb640 1 -- 172.21.15.37:0/1943110543 <== mon.0 v2:172.21.15.37:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1110062030 con 0x55aa2f2e8010 2024-06-10T01:01:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.936+0000 7f111f06a740 10 client.16107 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:01:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.936+0000 7f111f06a740 10 client.16107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:01:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.936+0000 7f111f06a740 20 client.16107 populate_metadata read hostname 'smithi037' 2024-06-10T01:01:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.936+0000 7f111f06a740 1 --2- 172.21.15.37:0/1943110543 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55aa2f2fdaa0 0x55aa2f2ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.936+0000 7f111f06a740 4 client.16107 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:01:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.936+0000 7f111f06a740 1 -- 172.21.15.37:0/1943110543 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa2f25ef10 con 0x55aa2f2fdaa0 2024-06-10T01:01:06.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.936+0000 7f11198fe640 1 --2- 172.21.15.37:0/1943110543 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55aa2f2fdaa0 0x55aa2f2ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:06.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.937+0000 7f11198fe640 1 --2- 172.21.15.37:0/1943110543 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55aa2f2fdaa0 0x55aa2f2ffe90 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e04b20e3478f62d2 server_cookie=65610458f3467cf6 in_seq=0 out_seq=0 2024-06-10T01:01:06.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.937+0000 7f10fdffb640 10 client.16107 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.938+0000 7f10fdffb640 1 -- 172.21.15.37:0/1943110543 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55aa2f25ef10 con 0x55aa2f2fdaa0 2024-06-10T01:01:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.938+0000 7f10fdffb640 10 client.16107 handle_command_reply: tid=0 2024-06-10T01:01:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.938+0000 7f111f06a740 1 client.16107 shutdown 2024-06-10T01:01:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.938+0000 7f111f06a740 1 -- 172.21.15.37:0/1943110543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa2f2e8010 msgr2=0x55aa2f2f21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.938+0000 7f111f06a740 1 --2- 172.21.15.37:0/1943110543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa2f2e8010 0x55aa2f2f21c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1110009bf0 tx=0x7f111003dc80 comp rx=0 tx=0).stop 2024-06-10T01:01:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.938+0000 7f111f06a740 1 -- 172.21.15.37:0/1943110543 shutdown_connections 2024-06-10T01:01:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.938+0000 7f111f06a740 1 --2- 172.21.15.37:0/1943110543 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55aa2f2fdaa0 0x55aa2f2ffe90 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.938+0000 7f111f06a740 1 --2- 172.21.15.37:0/1943110543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa2f2edb70 0x55aa2f2f9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.938+0000 7f111f06a740 1 --2- 172.21.15.37:0/1943110543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa2f2e8a00 0x55aa2f2f2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.938+0000 7f111f06a740 1 --2- 172.21.15.37:0/1943110543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa2f2e8010 0x55aa2f2f21c0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.938+0000 7f10fdffb640 0 client.16107 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.938+0000 7f111f06a740 1 -- 172.21.15.37:0/1943110543 >> 172.21.15.37:0/1943110543 conn(0x55aa2f2e3680 msgr2=0x55aa2f2e27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.938+0000 7f111f06a740 1 -- 172.21.15.37:0/1943110543 shutdown_connections 2024-06-10T01:01:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.938+0000 7f111f06a740 1 -- 172.21.15.37:0/1943110543 wait complete. 2024-06-10T01:01:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.939+0000 7f111f06a740 20 client.16107 trim_cache size 0 max 16384 2024-06-10T01:01:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.955+0000 7f111f06a740 1 Processor -- start 2024-06-10T01:01:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.956+0000 7f111f06a740 1 -- start start 2024-06-10T01:01:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.956+0000 7f111f06a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa2f2e8010 0x55aa2f2e6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.956+0000 7f111f06a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa2f2e8a00 0x55aa2f2f9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.956+0000 7f111f06a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa2f2edb70 0x55aa2f2f1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.956+0000 7f111f06a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11141af150 con 0x55aa2f2e8a00 2024-06-10T01:01:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.956+0000 7f111f06a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f11141aca20 con 0x55aa2f2e8010 2024-06-10T01:01:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.956+0000 7f111f06a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f11141aa870 con 0x55aa2f2edb70 2024-06-10T01:01:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.956+0000 7f11198fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa2f2e8a00 0x55aa2f2f9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.956+0000 7f11198fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa2f2e8a00 0x55aa2f2f9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35252/0 (socket says 172.21.15.37:35252) 2024-06-10T01:01:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.956+0000 7f11198fe640 1 -- 172.21.15.37:0/4271477030 learned_addr learned my addr 172.21.15.37:0/4271477030 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.956+0000 7f111a900640 1 --2- 172.21.15.37:0/4271477030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa2f2edb70 0x55aa2f2f1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.956+0000 7f111a0ff640 1 --2- 172.21.15.37:0/4271477030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa2f2e8010 0x55aa2f2e6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.956+0000 7f11198fe640 1 -- 172.21.15.37:0/4271477030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa2f2edb70 msgr2=0x55aa2f2f1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.956+0000 7f11198fe640 1 --2- 172.21.15.37:0/4271477030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa2f2edb70 0x55aa2f2f1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.956+0000 7f11198fe640 1 -- 172.21.15.37:0/4271477030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa2f2e8010 msgr2=0x55aa2f2e6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.956+0000 7f11198fe640 1 --2- 172.21.15.37:0/4271477030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa2f2e8010 0x55aa2f2e6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.956+0000 7f11198fe640 1 -- 172.21.15.37:0/4271477030 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa2f300200 con 0x55aa2f2e8a00 2024-06-10T01:01:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.957+0000 7f11198fe640 1 --2- 172.21.15.37:0/4271477030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa2f2e8a00 0x55aa2f2f9700 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f11040975e0 tx=0x7f110400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994724b6365c6906 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.957+0000 7f10fffff640 1 -- 172.21.15.37:0/4271477030 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11040a3df0 con 0x55aa2f2e8a00 2024-06-10T01:01:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.957+0000 7f10fffff640 1 -- 172.21.15.37:0/4271477030 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11040b4130 con 0x55aa2f2e8a00 2024-06-10T01:01:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.957+0000 7f10fffff640 1 -- 172.21.15.37:0/4271477030 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11040a3960 con 0x55aa2f2e8a00 2024-06-10T01:01:06.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.958+0000 7f111f06a740 1 -- 172.21.15.37:0/4271477030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa2f2e8a00 msgr2=0x55aa2f2f9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:06.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.958+0000 7f111f06a740 1 --2- 172.21.15.37:0/4271477030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa2f2e8a00 0x55aa2f2f9700 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f11040975e0 tx=0x7f110400ef90 comp rx=0 tx=0).stop 2024-06-10T01:01:06.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.958+0000 7f111f06a740 1 -- 172.21.15.37:0/4271477030 shutdown_connections 2024-06-10T01:01:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.958+0000 7f111f06a740 1 --2- 172.21.15.37:0/4271477030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa2f2edb70 0x55aa2f2f1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.958+0000 7f111f06a740 1 --2- 172.21.15.37:0/4271477030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa2f2e8a00 0x55aa2f2f9700 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.958+0000 7f111f06a740 1 --2- 172.21.15.37:0/4271477030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa2f2e8010 0x55aa2f2e6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.958+0000 7f111f06a740 1 -- 172.21.15.37:0/4271477030 >> 172.21.15.37:0/4271477030 conn(0x55aa2f2e3680 msgr2=0x55aa2f2e6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.958+0000 7f111f06a740 1 -- 172.21.15.37:0/4271477030 shutdown_connections 2024-06-10T01:01:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.958+0000 7f111f06a740 1 -- 172.21.15.37:0/4271477030 wait complete. 2024-06-10T01:01:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.958+0000 7f111f06a740 1 -- start start 2024-06-10T01:01:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.958+0000 7f111f06a740 1 Processor -- start 2024-06-10T01:01:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.959+0000 7f111f06a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa2f2e8010 0x55aa2f2f69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.959+0000 7f111f06a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa2f2e8a00 0x55aa2f2f6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.959+0000 7f111f06a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa2f2edb70 0x55aa2f2f7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.959+0000 7f111f06a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11141aaee0 con 0x55aa2f2e8010 2024-06-10T01:01:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.959+0000 7f111f06a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f11141ac350 con 0x55aa2f2e8a00 2024-06-10T01:01:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.959+0000 7f111f06a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f11141ab5b0 con 0x55aa2f2edb70 2024-06-10T01:01:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.959+0000 7f111f06a740 10 client.0 _ll_register_callbacks cb 0x7f110c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:01:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.959+0000 7f111a0ff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa2f2e8010 0x55aa2f2f69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.959+0000 7f111a0ff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa2f2e8010 0x55aa2f2f69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35256/0 (socket says 172.21.15.37:35256) 2024-06-10T01:01:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.959+0000 7f111a0ff640 1 -- 172.21.15.37:0/3080715075 learned_addr learned my addr 172.21.15.37:0/3080715075 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.959+0000 7f111a900640 1 --2- 172.21.15.37:0/3080715075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa2f2edb70 0x55aa2f2f7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.959+0000 7f11198fe640 1 --2- 172.21.15.37:0/3080715075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa2f2e8a00 0x55aa2f2f6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.959+0000 7f111a0ff640 1 -- 172.21.15.37:0/3080715075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa2f2edb70 msgr2=0x55aa2f2f7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.959+0000 7f111a0ff640 1 --2- 172.21.15.37:0/3080715075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa2f2edb70 0x55aa2f2f7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.959+0000 7f111a0ff640 1 -- 172.21.15.37:0/3080715075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa2f2e8a00 msgr2=0x55aa2f2f6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.959+0000 7f111a0ff640 1 --2- 172.21.15.37:0/3080715075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa2f2e8a00 0x55aa2f2f6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.959+0000 7f111a0ff640 1 -- 172.21.15.37:0/3080715075 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa2f2f8510 con 0x55aa2f2e8010 2024-06-10T01:01:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.960+0000 7f111a0ff640 1 --2- 172.21.15.37:0/3080715075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa2f2e8010 0x55aa2f2f69d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f111002add0 tx=0x7f111003b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa64022c6e3e3e3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.960+0000 7f10fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:01:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.960+0000 7f10fdffb640 1 -- 172.21.15.37:0/3080715075 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110073060 con 0x55aa2f2e8010 2024-06-10T01:01:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.960+0000 7f111f06a740 1 -- 172.21.15.37:0/3080715075 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11040b4130 con 0x55aa2f2e8010 2024-06-10T01:01:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.960+0000 7f10fdffb640 1 -- 172.21.15.37:0/3080715075 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f1110051880 con 0x55aa2f2e8010 2024-06-10T01:01:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.960+0000 7f10fdffb640 1 -- 172.21.15.37:0/3080715075 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f111007a050 con 0x55aa2f2e8010 2024-06-10T01:01:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.960+0000 7f111f06a740 10 client.16113 fetch_fsmap learned FSMap version 27 2024-06-10T01:01:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.960+0000 7f111f06a740 1 -- 172.21.15.37:0/3080715075 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aa2f2f9890 con 0x55aa2f2e8010 2024-06-10T01:01:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.960+0000 7f10fdffb640 1 -- 172.21.15.37:0/3080715075 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11040b4130 con 0x55aa2f2e8010 2024-06-10T01:01:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.960+0000 7f10fdffb640 1 -- 172.21.15.37:0/3080715075 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1110062050 con 0x55aa2f2e8010 2024-06-10T01:01:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.961+0000 7f111f06a740 10 client.16113 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:01:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.961+0000 7f111f06a740 10 client.16113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:01:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.961+0000 7f111f06a740 20 client.16113 populate_metadata read hostname 'smithi037' 2024-06-10T01:01:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.961+0000 7f111f06a740 1 --2- 172.21.15.37:0/3080715075 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55aa2f368a70 0x55aa2f36ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.961+0000 7f111f06a740 4 client.16113 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:01:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.961+0000 7f111f06a740 1 -- 172.21.15.37:0/3080715075 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aa2f258c20 con 0x55aa2f368a70 2024-06-10T01:01:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.961+0000 7f11198fe640 1 --2- 172.21.15.37:0/3080715075 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55aa2f368a70 0x55aa2f36ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.961+0000 7f11198fe640 1 --2- 172.21.15.37:0/3080715075 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55aa2f368a70 0x55aa2f36ae60 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eac6daa41f60e25d server_cookie=582ce86c77ec15e8 in_seq=0 out_seq=0 2024-06-10T01:01:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.961+0000 7f10fdffb640 10 client.16113 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.978+0000 7f10fdffb640 1 -- 172.21.15.37:0/3080715075 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55aa2f258c20 con 0x55aa2f368a70 2024-06-10T01:01:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.978+0000 7f10fdffb640 10 client.16113 handle_command_reply: tid=0 2024-06-10T01:01:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.978+0000 7f111f06a740 1 client.16113 shutdown 2024-06-10T01:01:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.978+0000 7f111f06a740 1 -- 172.21.15.37:0/3080715075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa2f2e8010 msgr2=0x55aa2f2f69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.978+0000 7f111f06a740 1 --2- 172.21.15.37:0/3080715075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa2f2e8010 0x55aa2f2f69d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f111002add0 tx=0x7f111003b000 comp rx=0 tx=0).stop 2024-06-10T01:01:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.978+0000 7f111f06a740 1 -- 172.21.15.37:0/3080715075 shutdown_connections 2024-06-10T01:01:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.978+0000 7f111f06a740 1 --2- 172.21.15.37:0/3080715075 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55aa2f368a70 0x55aa2f36ae60 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.978+0000 7f111f06a740 1 --2- 172.21.15.37:0/3080715075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa2f2edb70 0x55aa2f2f7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.978+0000 7f111f06a740 1 --2- 172.21.15.37:0/3080715075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aa2f2e8a00 0x55aa2f2f6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.978+0000 7f10fdffb640 0 client.16113 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.978+0000 7f111f06a740 1 --2- 172.21.15.37:0/3080715075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa2f2e8010 0x55aa2f2f69d0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.978+0000 7f111f06a740 1 -- 172.21.15.37:0/3080715075 >> 172.21.15.37:0/3080715075 conn(0x55aa2f2e3680 msgr2=0x55aa2f2e6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.978+0000 7f111f06a740 1 -- 172.21.15.37:0/3080715075 shutdown_connections 2024-06-10T01:01:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.978+0000 7f111f06a740 1 -- 172.21.15.37:0/3080715075 wait complete. 2024-06-10T01:01:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.978+0000 7f111f06a740 20 client.16113 trim_cache size 0 max 16384 2024-06-10T01:01:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:01:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:01:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:01:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.979+0000 7f111c363640 1 -- 172.21.15.37:0/1595084736 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f11000a2da0 msgr2=0x7f11000a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.979+0000 7f111c363640 1 --2- 172.21.15.37:0/1595084736 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f11000a2da0 0x7f11000a5260 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1110000f80 tx=0x7f1110030750 comp rx=0 tx=0).stop 2024-06-10T01:01:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.979+0000 7f111c363640 1 -- 172.21.15.37:0/1595084736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1114102c20 msgr2=0x7f111419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.979+0000 7f111c363640 1 --2- 172.21.15.37:0/1595084736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1114102c20 0x7f111419a440 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f110400ea60 tx=0x7f110400ef20 comp rx=0 tx=0).stop 2024-06-10T01:01:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.979+0000 7f111c363640 1 -- 172.21.15.37:0/1595084736 shutdown_connections 2024-06-10T01:01:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.979+0000 7f111c363640 1 --2- 172.21.15.37:0/1595084736 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f11000a2da0 0x7f11000a5260 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.979+0000 7f111c363640 1 --2- 172.21.15.37:0/1595084736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11141035e0 0x7f11141a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.979+0000 7f111c363640 1 --2- 172.21.15.37:0/1595084736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1114102c20 0x7f111419a440 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.979+0000 7f111c363640 1 --2- 172.21.15.37:0/1595084736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1114101a30 0x7f1114199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.979+0000 7f111c363640 1 -- 172.21.15.37:0/1595084736 >> 172.21.15.37:0/1595084736 conn(0x7f11140fcfc0 msgr2=0x7f1114073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.979+0000 7f111c363640 1 -- 172.21.15.37:0/1595084736 shutdown_connections 2024-06-10T01:01:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:06.979+0000 7f111c363640 1 -- 172.21.15.37:0/1595084736 wait complete. 2024-06-10T01:01:07.343 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:07.344 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:07.541 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:01:07.541 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:01:07.575 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:07.575 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:07.575 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 79 -lt 100 ']' 2024-06-10T01:01:07.575 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 79 + 1 2024-06-10T01:01:07.642 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=80 2024-06-10T01:01:07.642 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:01:07.700 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:01:07.700 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:01:07.702 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:07.702 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:07.704 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:01:07.704 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:01:07.704 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:01:08.243 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:08.243 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:08.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:08 smithi167 ceph-mon[36917]: pgmap v563: 129 pgs: 129 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-06-10T01:01:08.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:08 smithi094 ceph-mon[36708]: pgmap v563: 129 pgs: 129 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-06-10T01:01:08.382 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:01:08.382 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:01:08.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:08 smithi037 ceph-mon[32192]: pgmap v563: 129 pgs: 129 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-06-10T01:01:08.396 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:08.396 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:08.396 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 80 -lt 100 ']' 2024-06-10T01:01:08.396 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 80 + 1 2024-06-10T01:01:08.457 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=81 2024-06-10T01:01:08.457 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:01:08.476 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:08.476 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:08.477 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:01:08.477 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:01:08.477 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:01:09.093 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:09.093 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:09.273 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:01:09.273 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:01:09.288 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:09.289 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:09.289 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 81 -lt 100 ']' 2024-06-10T01:01:09.289 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 81 + 1 2024-06-10T01:01:09.324 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=82 2024-06-10T01:01:09.324 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:01:09.351 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:09.351 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:09.352 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:01:09.352 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:01:09.353 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:01:09.785 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:09.785 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:09.964 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:01:09.965 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:01:09.979 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:09.979 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:09.979 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 82 -lt 100 ']' 2024-06-10T01:01:09.980 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 82 + 1 2024-06-10T01:01:10.026 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=83 2024-06-10T01:01:10.026 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:01:10.032 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:10.032 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:10.033 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:01:10.034 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:01:10.034 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:01:10.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:10 smithi167 ceph-mon[36917]: pgmap v564: 129 pgs: 129 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.2 KiB/s wr, 11 op/s 2024-06-10T01:01:10.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:10 smithi094 ceph-mon[36708]: pgmap v564: 129 pgs: 129 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.2 KiB/s wr, 11 op/s 2024-06-10T01:01:10.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:10 smithi037 ceph-mon[32192]: pgmap v564: 129 pgs: 129 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.2 KiB/s wr, 11 op/s 2024-06-10T01:01:10.676 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:10.677 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:10.974 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:01:10.974 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:01:11.005 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:11.006 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:11.006 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 83 -lt 100 ']' 2024-06-10T01:01:11.006 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 83 + 1 2024-06-10T01:01:11.081 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=84 2024-06-10T01:01:11.082 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:01:11.134 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:11.134 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:11.135 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:01:11.136 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:01:11.136 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:01:11.937 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:11.937 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:12.182 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:01:12.182 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:01:12.222 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:12.222 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:12.222 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 84 -lt 100 ']' 2024-06-10T01:01:12.222 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 84 + 1 2024-06-10T01:01:12.264 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=85 2024-06-10T01:01:12.264 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:01:12.334 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:12.334 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:12.336 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:01:12.336 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:01:12.336 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:01:12.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:12 smithi167 ceph-mon[36917]: pgmap v565: 129 pgs: 129 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-06-10T01:01:12.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:12 smithi094 ceph-mon[36708]: pgmap v565: 129 pgs: 129 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-06-10T01:01:12.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:12 smithi037 ceph-mon[32192]: pgmap v565: 129 pgs: 129 active+clean; 1.8 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-06-10T01:01:13.451 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:13.451 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:13.774 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:01:13.774 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:01:13.813 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:13.813 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:13.813 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 85 -lt 100 ']' 2024-06-10T01:01:13.814 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 85 + 1 2024-06-10T01:01:13.983 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=86 2024-06-10T01:01:13.983 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:01:14.009 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:14.009 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:14.010 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:01:14.010 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:01:14.010 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:01:14.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:14 smithi167 ceph-mon[36917]: pgmap v566: 129 pgs: 129 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-06-10T01:01:14.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:14 smithi094 ceph-mon[36708]: pgmap v566: 129 pgs: 129 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-06-10T01:01:14.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:14 smithi037 ceph-mon[32192]: pgmap v566: 129 pgs: 129 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-06-10T01:01:15.126 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:15.126 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:15.473 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:01:15.473 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:01:15.490 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:15.490 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:15.490 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 86 -lt 100 ']' 2024-06-10T01:01:15.491 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 86 + 1 2024-06-10T01:01:15.541 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=87 2024-06-10T01:01:15.541 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:01:15.600 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:15.603 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:15.603 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:01:15.603 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:01:15.603 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:01:16.288 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:16 smithi037 ceph-mon[32192]: pgmap v567: 129 pgs: 129 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-06-10T01:01:16.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:16 smithi167 ceph-mon[36917]: pgmap v567: 129 pgs: 129 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-06-10T01:01:16.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:16 smithi094 ceph-mon[36708]: pgmap v567: 129 pgs: 129 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-06-10T01:01:16.486 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:16.487 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:16.804 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:01:16.804 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:01:16.829 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:16.830 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:16.830 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 87 -lt 100 ']' 2024-06-10T01:01:16.830 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 87 + 1 2024-06-10T01:01:16.889 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=88 2024-06-10T01:01:16.889 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:01:16.907 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:16.908 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:16.909 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:01:16.909 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:01:16.909 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:01:18.109 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:18.109 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:18.112 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:18 smithi094 ceph-mon[36708]: pgmap v568: 129 pgs: 129 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-06-10T01:01:18.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:18 smithi167 ceph-mon[36917]: pgmap v568: 129 pgs: 129 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-06-10T01:01:18.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:18 smithi037 ceph-mon[32192]: pgmap v568: 129 pgs: 129 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-06-10T01:01:18.533 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:01:18.533 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:01:18.563 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:18.563 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:18.563 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 88 -lt 100 ']' 2024-06-10T01:01:18.563 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 88 + 1 2024-06-10T01:01:18.605 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=89 2024-06-10T01:01:18.606 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:01:18.658 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:18.658 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:18.660 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:01:18.660 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:01:18.660 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:01:19.517 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:19.517 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:19.898 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:01:19.898 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:01:19.913 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:19.913 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:19.913 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 89 -lt 100 ']' 2024-06-10T01:01:19.913 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 89 + 1 2024-06-10T01:01:19.982 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=90 2024-06-10T01:01:19.982 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:01:20.006 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:01:20.025 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:20.025 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:20.026 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:01:20.027 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:01:20.027 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:01:20.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:20 smithi167 ceph-mon[36917]: pgmap v569: 129 pgs: 129 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-06-10T01:01:20.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:20 smithi094 ceph-mon[36708]: pgmap v569: 129 pgs: 129 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-06-10T01:01:20.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:20 smithi037 ceph-mon[32192]: pgmap v569: 129 pgs: 129 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-06-10T01:01:21.083 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:21.083 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:21.655 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:01:21.655 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:01:21.678 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:21.679 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:21.679 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 90 -lt 100 ']' 2024-06-10T01:01:21.679 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 90 + 1 2024-06-10T01:01:21.734 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=91 2024-06-10T01:01:21.734 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:01:21.854 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:21.855 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:21.856 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:01:21.856 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:01:21.856 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:01:22.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:22 smithi167 ceph-mon[36917]: pgmap v570: 129 pgs: 129 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-06-10T01:01:22.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:22 smithi094 ceph-mon[36708]: pgmap v570: 129 pgs: 129 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-06-10T01:01:22.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:22 smithi037 ceph-mon[32192]: pgmap v570: 129 pgs: 129 active+clean; 1.9 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-06-10T01:01:22.975 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:22.975 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:23.445 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:01:23.445 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:01:23.462 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:23.462 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:23.462 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 91 -lt 100 ']' 2024-06-10T01:01:23.462 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 91 + 1 2024-06-10T01:01:23.706 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=92 2024-06-10T01:01:23.706 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:01:23.917 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:23.918 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:23.918 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:01:23.919 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:01:23.919 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:01:24.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:24 smithi167 ceph-mon[36917]: pgmap v571: 129 pgs: 129 active+clean; 2.0 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-06-10T01:01:24.339 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:24 smithi037 ceph-mon[32192]: pgmap v571: 129 pgs: 129 active+clean; 2.0 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-06-10T01:01:24.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:24 smithi094 ceph-mon[36708]: pgmap v571: 129 pgs: 129 active+clean; 2.0 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-06-10T01:01:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.646+0000 7f161021c640 1 -- 172.21.15.37:0/1430100151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16081035e0 msgr2=0x7f1608109e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.646+0000 7f161021c640 1 --2- 172.21.15.37:0/1430100151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16081035e0 0x7f1608109e70 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1600009ca0 tx=0x7f160002a8c0 comp rx=0 tx=0).stop 2024-06-10T01:01:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.646+0000 7f161021c640 1 -- 172.21.15.37:0/1430100151 shutdown_connections 2024-06-10T01:01:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.646+0000 7f161021c640 1 --2- 172.21.15.37:0/1430100151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16081035e0 0x7f1608109e70 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.646+0000 7f161021c640 1 --2- 172.21.15.37:0/1430100151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1608102c20 0x7f16081030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.646+0000 7f161021c640 1 --2- 172.21.15.37:0/1430100151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1608101a30 0x7f1608101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.647+0000 7f161021c640 1 -- 172.21.15.37:0/1430100151 >> 172.21.15.37:0/1430100151 conn(0x7f16080fcfc0 msgr2=0x7f16080ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.647+0000 7f161021c640 1 -- 172.21.15.37:0/1430100151 shutdown_connections 2024-06-10T01:01:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.647+0000 7f161021c640 1 -- 172.21.15.37:0/1430100151 wait complete. 2024-06-10T01:01:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.695+0000 7f161021c640 1 Processor -- start 2024-06-10T01:01:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.695+0000 7f161021c640 1 -- start start 2024-06-10T01:01:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.695+0000 7f161021c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1608101a30 0x7f1608199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.695+0000 7f161021c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1608102c20 0x7f160819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.695+0000 7f161021c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16081035e0 0x7f16081a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.695+0000 7f161021c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f160810d6b0 con 0x7f1608102c20 2024-06-10T01:01:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.695+0000 7f161021c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f160810d530 con 0x7f16081035e0 2024-06-10T01:01:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.695+0000 7f161021c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f160810d830 con 0x7f1608101a30 2024-06-10T01:01:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.696+0000 7f160d7b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1608102c20 0x7f160819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.696+0000 7f160d7b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1608102c20 0x7f160819a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49662/0 (socket says 172.21.15.37:49662) 2024-06-10T01:01:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.696+0000 7f160d7b7640 1 -- 172.21.15.37:0/2363745365 learned_addr learned my addr 172.21.15.37:0/2363745365 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.696+0000 7f160dfb8640 1 --2- 172.21.15.37:0/2363745365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1608101a30 0x7f1608199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.696+0000 7f160e7b9640 1 --2- 172.21.15.37:0/2363745365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16081035e0 0x7f16081a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.696+0000 7f160d7b7640 1 -- 172.21.15.37:0/2363745365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1608101a30 msgr2=0x7f1608199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.696+0000 7f160d7b7640 1 --2- 172.21.15.37:0/2363745365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1608101a30 0x7f1608199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.696+0000 7f160d7b7640 1 -- 172.21.15.37:0/2363745365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16081035e0 msgr2=0x7f16081a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.696+0000 7f160d7b7640 1 --2- 172.21.15.37:0/2363745365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16081035e0 0x7f16081a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.696+0000 7f160d7b7640 1 -- 172.21.15.37:0/2363745365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16081a1a80 con 0x7f1608102c20 2024-06-10T01:01:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.696+0000 7f160d7b7640 1 --2- 172.21.15.37:0/2363745365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1608102c20 0x7f160819a3a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f15f800ea60 tx=0x7f15f800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.700+0000 7f15feffd640 1 -- 172.21.15.37:0/2363745365 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f800cde0 con 0x7f1608102c20 2024-06-10T01:01:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.700+0000 7f15feffd640 1 -- 172.21.15.37:0/2363745365 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15f80035f0 con 0x7f1608102c20 2024-06-10T01:01:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.700+0000 7f15feffd640 1 -- 172.21.15.37:0/2363745365 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f8010670 con 0x7f1608102c20 2024-06-10T01:01:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.700+0000 7f161021c640 1 -- 172.21.15.37:0/2363745365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16081a1d70 con 0x7f1608102c20 2024-06-10T01:01:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.701+0000 7f161021c640 1 -- 172.21.15.37:0/2363745365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16081a2260 con 0x7f1608102c20 2024-06-10T01:01:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.703+0000 7f15feffd640 1 -- 172.21.15.37:0/2363745365 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f15f80037a0 con 0x7f1608102c20 2024-06-10T01:01:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.704+0000 7f15feffd640 1 --2- 172.21.15.37:0/2363745365 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f15f40a2da0 0x7f15f40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.704+0000 7f160dfb8640 1 --2- 172.21.15.37:0/2363745365 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f15f40a2da0 0x7f15f40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.704+0000 7f15feffd640 1 -- 172.21.15.37:0/2363745365 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f15f80cf630 con 0x7f1608102c20 2024-06-10T01:01:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:24.705+0000 7f160dfb8640 1 --2- 172.21.15.37:0/2363745365 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f15f40a2da0 0x7f15f40a5260 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1608102a80 tx=0x7f1604005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.041+0000 7f161300a740 1 Processor -- start 2024-06-10T01:01:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.041+0000 7f161300a740 1 -- start start 2024-06-10T01:01:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.041+0000 7f161300a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8c86e3010 0x55f8c86e3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.042+0000 7f161300a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8c86e3a00 0x55f8c86e8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.042+0000 7f161300a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8c86e8b70 0x55f8c86eb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.042+0000 7f161300a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f8c86464f0 con 0x55f8c86e3a00 2024-06-10T01:01:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.042+0000 7f161300a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f8c85e4870 con 0x55f8c86e8b70 2024-06-10T01:01:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.042+0000 7f161300a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f8c86424b0 con 0x55f8c86e3010 2024-06-10T01:01:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.042+0000 7f160d7b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8c86e3a00 0x55f8c86e8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.042+0000 7f160d7b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8c86e3a00 0x55f8c86e8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49676/0 (socket says 172.21.15.37:49676) 2024-06-10T01:01:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.042+0000 7f160d7b7640 1 -- 172.21.15.37:0/3902979846 learned_addr learned my addr 172.21.15.37:0/3902979846 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.042+0000 7f160e7b9640 1 --2- 172.21.15.37:0/3902979846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8c86e8b70 0x55f8c86eb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.042+0000 7f160dfb8640 1 --2- 172.21.15.37:0/3902979846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8c86e3010 0x55f8c86e3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.042+0000 7f160e7b9640 1 -- 172.21.15.37:0/3902979846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8c86e3010 msgr2=0x55f8c86e3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.042+0000 7f160e7b9640 1 --2- 172.21.15.37:0/3902979846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8c86e3010 0x55f8c86e3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.042+0000 7f160e7b9640 1 -- 172.21.15.37:0/3902979846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8c86e3a00 msgr2=0x55f8c86e8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.042+0000 7f160e7b9640 1 --2- 172.21.15.37:0/3902979846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8c86e3a00 0x55f8c86e8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.042+0000 7f160e7b9640 1 -- 172.21.15.37:0/3902979846 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8c86009b0 con 0x55f8c86e8b70 2024-06-10T01:01:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.043+0000 7f160e7b9640 1 --2- 172.21.15.37:0/3902979846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8c86e8b70 0x55f8c86eb000 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1600031ae0 tx=0x7f1600004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d97a878290dc2ef server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.043+0000 7f15f3fff640 1 -- 172.21.15.37:0/3902979846 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16000098c0 con 0x55f8c86e8b70 2024-06-10T01:01:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.043+0000 7f15f3fff640 1 -- 172.21.15.37:0/3902979846 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1600036040 con 0x55f8c86e8b70 2024-06-10T01:01:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.043+0000 7f15f3fff640 1 -- 172.21.15.37:0/3902979846 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1600004290 con 0x55f8c86e8b70 2024-06-10T01:01:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.044+0000 7f161300a740 1 -- 172.21.15.37:0/3902979846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8c86e8b70 msgr2=0x55f8c86eb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.044+0000 7f161300a740 1 --2- 172.21.15.37:0/3902979846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8c86e8b70 0x55f8c86eb000 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1600031ae0 tx=0x7f1600004a40 comp rx=0 tx=0).stop 2024-06-10T01:01:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.044+0000 7f161300a740 1 -- 172.21.15.37:0/3902979846 shutdown_connections 2024-06-10T01:01:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.044+0000 7f161300a740 1 --2- 172.21.15.37:0/3902979846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8c86e8b70 0x55f8c86eb000 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.044+0000 7f161300a740 1 --2- 172.21.15.37:0/3902979846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8c86e3a00 0x55f8c86e8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.044+0000 7f161300a740 1 --2- 172.21.15.37:0/3902979846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8c86e3010 0x55f8c86e3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.044+0000 7f161300a740 1 -- 172.21.15.37:0/3902979846 >> 172.21.15.37:0/3902979846 conn(0x55f8c86de680 msgr2=0x55f8c86dea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.044+0000 7f161300a740 1 -- 172.21.15.37:0/3902979846 shutdown_connections 2024-06-10T01:01:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.044+0000 7f161300a740 1 -- 172.21.15.37:0/3902979846 wait complete. 2024-06-10T01:01:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.071+0000 7f161300a740 1 -- start start 2024-06-10T01:01:25.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.076+0000 7f161300a740 1 Processor -- start 2024-06-10T01:01:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.093+0000 7f161300a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8c86e3010 0x55f8c86ed010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.093+0000 7f161300a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8c86e3a00 0x55f8c86ed550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.093+0000 7f161300a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8c86e8b70 0x55f8c86f4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.093+0000 7f161300a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f8c86eb8e0 con 0x55f8c86e3010 2024-06-10T01:01:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.093+0000 7f160dfb8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8c86e3010 0x55f8c86ed010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.093+0000 7f160d7b7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8c86e3a00 0x55f8c86ed550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.093+0000 7f160e7b9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8c86e8b70 0x55f8c86f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.093+0000 7f160d7b7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8c86e3a00 0x55f8c86ed550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:59558/0 (socket says 172.21.15.37:59558) 2024-06-10T01:01:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.093+0000 7f160e7b9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8c86e8b70 0x55f8c86f4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:53376/0 (socket says 172.21.15.37:53376) 2024-06-10T01:01:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.094+0000 7f160d7b7640 1 -- 172.21.15.37:0/2657049534 learned_addr learned my addr 172.21.15.37:0/2657049534 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.094+0000 7f161300a740 1 -- 172.21.15.37:0/2657049534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f8c85cfae0 con 0x55f8c86e8b70 2024-06-10T01:01:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.094+0000 7f161300a740 1 -- 172.21.15.37:0/2657049534 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f8c86eba60 con 0x55f8c86e3a00 2024-06-10T01:01:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.094+0000 7f160e7b9640 1 -- 172.21.15.37:0/2657049534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8c86e3a00 msgr2=0x55f8c86ed550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.094+0000 7f160e7b9640 1 --2- 172.21.15.37:0/2657049534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8c86e3a00 0x55f8c86ed550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.094+0000 7f160e7b9640 1 -- 172.21.15.37:0/2657049534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8c86e3010 msgr2=0x55f8c86ed010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.094+0000 7f160e7b9640 1 --2- 172.21.15.37:0/2657049534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8c86e3010 0x55f8c86ed010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.094+0000 7f160e7b9640 1 -- 172.21.15.37:0/2657049534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8c835a210 con 0x55f8c86e8b70 2024-06-10T01:01:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.094+0000 7f160dfb8640 1 --2- 172.21.15.37:0/2657049534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8c86e3010 0x55f8c86ed010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:01:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.094+0000 7f160d7b7640 1 --2- 172.21.15.37:0/2657049534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8c86e3a00 0x55f8c86ed550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:01:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.094+0000 7f160e7b9640 1 --2- 172.21.15.37:0/2657049534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8c86e8b70 0x55f8c86f4590 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1600009bf0 tx=0x7f160002a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f151209bc5270ac3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.094+0000 7f15f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:01:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.095+0000 7f15f17fa640 1 -- 172.21.15.37:0/2657049534 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1600053060 con 0x55f8c86e8b70 2024-06-10T01:01:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.095+0000 7f15f17fa640 1 -- 172.21.15.37:0/2657049534 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f1600041040 con 0x55f8c86e8b70 2024-06-10T01:01:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.095+0000 7f161300a740 10 client.0 _ll_register_callbacks cb 0x55f8c86089b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:01:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.102+0000 7f161300a740 1 -- 172.21.15.37:0/2657049534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1600036040 con 0x55f8c86e8b70 2024-06-10T01:01:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.103+0000 7f15f17fa640 1 -- 172.21.15.37:0/2657049534 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f16000098c0 con 0x55f8c86e8b70 2024-06-10T01:01:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.103+0000 7f161300a740 10 client.25643 fetch_fsmap learned FSMap version 27 2024-06-10T01:01:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.103+0000 7f161300a740 1 -- 172.21.15.37:0/2657049534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55f8c8426620 con 0x55f8c86e8b70 2024-06-10T01:01:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.114+0000 7f15f17fa640 1 -- 172.21.15.37:0/2657049534 <== mon.2 v2:172.21.15.94:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f160004d070 con 0x55f8c86e8b70 2024-06-10T01:01:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.114+0000 7f161300a740 10 client.25643 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:01:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.138+0000 7f161300a740 10 client.25643 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:01:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.138+0000 7f161300a740 20 client.25643 populate_metadata read hostname 'smithi037' 2024-06-10T01:01:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.138+0000 7f161300a740 1 --2- 172.21.15.37:0/2657049534 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f8c86f88f0 0x55f8c86fad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.139+0000 7f161300a740 4 client.25643 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:01:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.139+0000 7f161300a740 1 -- 172.21.15.37:0/2657049534 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8c8659f10 con 0x55f8c86f88f0 2024-06-10T01:01:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.139+0000 7f160dfb8640 1 --2- 172.21.15.37:0/2657049534 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f8c86f88f0 0x55f8c86fad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.139+0000 7f160dfb8640 1 --2- 172.21.15.37:0/2657049534 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f8c86f88f0 0x55f8c86fad20 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92906fc446f43e2f server_cookie=c3d8f93da445dda8 in_seq=0 out_seq=0 2024-06-10T01:01:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.139+0000 7f15f17fa640 10 client.25643 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.140+0000 7f15f17fa640 1 -- 172.21.15.37:0/2657049534 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f8c8659f10 con 0x55f8c86f88f0 2024-06-10T01:01:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.140+0000 7f15f17fa640 10 client.25643 handle_command_reply: tid=0 2024-06-10T01:01:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.142+0000 7f161300a740 1 client.25643 shutdown 2024-06-10T01:01:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.142+0000 7f161300a740 1 -- 172.21.15.37:0/2657049534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8c86e8b70 msgr2=0x55f8c86f4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.142+0000 7f161300a740 1 --2- 172.21.15.37:0/2657049534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8c86e8b70 0x55f8c86f4590 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1600009bf0 tx=0x7f160002a8c0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.142+0000 7f161300a740 1 -- 172.21.15.37:0/2657049534 shutdown_connections 2024-06-10T01:01:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.142+0000 7f161300a740 1 --2- 172.21.15.37:0/2657049534 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f8c86f88f0 0x55f8c86fad20 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.142+0000 7f161300a740 1 --2- 172.21.15.37:0/2657049534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8c86e8b70 0x55f8c86f4590 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.142+0000 7f161300a740 1 --2- 172.21.15.37:0/2657049534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8c86e3a00 0x55f8c86ed550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.142+0000 7f161300a740 1 --2- 172.21.15.37:0/2657049534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8c86e3010 0x55f8c86ed010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.142+0000 7f15f17fa640 0 client.25643 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.142+0000 7f161300a740 1 -- 172.21.15.37:0/2657049534 >> 172.21.15.37:0/2657049534 conn(0x55f8c86de680 msgr2=0x55f8c86dd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.142+0000 7f161300a740 1 -- 172.21.15.37:0/2657049534 shutdown_connections 2024-06-10T01:01:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.142+0000 7f161300a740 1 -- 172.21.15.37:0/2657049534 wait complete. 2024-06-10T01:01:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.143+0000 7f161300a740 20 client.25643 trim_cache size 0 max 16384 2024-06-10T01:01:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.159+0000 7f161300a740 1 Processor -- start 2024-06-10T01:01:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.159+0000 7f161300a740 1 -- start start 2024-06-10T01:01:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.159+0000 7f161300a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8c86e3010 0x55f8c86eaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.159+0000 7f161300a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8c86e3a00 0x55f8c86f4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.159+0000 7f161300a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8c86e8b70 0x55f8c86e4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.159+0000 7f161300a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f16081af080 con 0x55f8c86e3a00 2024-06-10T01:01:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.159+0000 7f161300a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f16081ac8e0 con 0x55f8c86e8b70 2024-06-10T01:01:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.159+0000 7f161300a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f16081aa730 con 0x55f8c86e3010 2024-06-10T01:01:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.159+0000 7f160d7b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8c86e3a00 0x55f8c86f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.159+0000 7f160d7b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8c86e3a00 0x55f8c86f4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49708/0 (socket says 172.21.15.37:49708) 2024-06-10T01:01:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.159+0000 7f160d7b7640 1 -- 172.21.15.37:0/4111517114 learned_addr learned my addr 172.21.15.37:0/4111517114 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.159+0000 7f160dfb8640 1 --2- 172.21.15.37:0/4111517114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8c86e3010 0x55f8c86eaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.159+0000 7f160e7b9640 1 --2- 172.21.15.37:0/4111517114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8c86e8b70 0x55f8c86e4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.159+0000 7f160d7b7640 1 -- 172.21.15.37:0/4111517114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8c86e3010 msgr2=0x55f8c86eaf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.159+0000 7f160d7b7640 1 --2- 172.21.15.37:0/4111517114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8c86e3010 0x55f8c86eaf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.159+0000 7f160d7b7640 1 -- 172.21.15.37:0/4111517114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8c86e8b70 msgr2=0x55f8c86e4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.159+0000 7f160d7b7640 1 --2- 172.21.15.37:0/4111517114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8c86e8b70 0x55f8c86e4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.159+0000 7f160d7b7640 1 -- 172.21.15.37:0/4111517114 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8c86fb3a0 con 0x55f8c86e3a00 2024-06-10T01:01:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.159+0000 7f160e7b9640 1 --2- 172.21.15.37:0/4111517114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8c86e8b70 0x55f8c86e4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:01:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.160+0000 7f160d7b7640 1 --2- 172.21.15.37:0/4111517114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8c86e3a00 0x55f8c86f4550 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f15f801f000 tx=0x7f15f80975c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d7f8d88ac8a64 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.160+0000 7f15f37fe640 1 -- 172.21.15.37:0/4111517114 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f809fdf0 con 0x55f8c86e3a00 2024-06-10T01:01:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.160+0000 7f15f37fe640 1 -- 172.21.15.37:0/4111517114 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15f80a46b0 con 0x55f8c86e3a00 2024-06-10T01:01:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.160+0000 7f15f37fe640 1 -- 172.21.15.37:0/4111517114 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f80cf280 con 0x55f8c86e3a00 2024-06-10T01:01:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.161+0000 7f161300a740 1 -- 172.21.15.37:0/4111517114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8c86e3a00 msgr2=0x55f8c86f4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.161+0000 7f161300a740 1 --2- 172.21.15.37:0/4111517114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8c86e3a00 0x55f8c86f4550 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f15f801f000 tx=0x7f15f80975c0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.161+0000 7f161300a740 1 -- 172.21.15.37:0/4111517114 shutdown_connections 2024-06-10T01:01:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.161+0000 7f161300a740 1 --2- 172.21.15.37:0/4111517114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8c86e8b70 0x55f8c86e4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.161+0000 7f161300a740 1 --2- 172.21.15.37:0/4111517114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8c86e3a00 0x55f8c86f4550 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.161+0000 7f161300a740 1 --2- 172.21.15.37:0/4111517114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8c86e3010 0x55f8c86eaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.161+0000 7f161300a740 1 -- 172.21.15.37:0/4111517114 >> 172.21.15.37:0/4111517114 conn(0x55f8c86de680 msgr2=0x55f8c86e8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.161+0000 7f161300a740 1 -- 172.21.15.37:0/4111517114 shutdown_connections 2024-06-10T01:01:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.161+0000 7f161300a740 1 -- 172.21.15.37:0/4111517114 wait complete. 2024-06-10T01:01:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.161+0000 7f161300a740 1 -- start start 2024-06-10T01:01:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.161+0000 7f161300a740 1 Processor -- start 2024-06-10T01:01:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.162+0000 7f161300a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8c86e3010 0x55f8c86f1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.162+0000 7f161300a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8c86e3a00 0x55f8c86f1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.162+0000 7f161300a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8c86e8b70 0x55f8c87615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.162+0000 7f161300a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f16081aada0 con 0x55f8c86e3a00 2024-06-10T01:01:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.162+0000 7f161300a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f16081ac210 con 0x55f8c86e3010 2024-06-10T01:01:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.162+0000 7f161300a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f16081ab470 con 0x55f8c86e8b70 2024-06-10T01:01:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.162+0000 7f161300a740 10 client.0 _ll_register_callbacks cb 0x7f15f8009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:01:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.162+0000 7f160d7b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8c86e3a00 0x55f8c86f1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.162+0000 7f160d7b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8c86e3a00 0x55f8c86f1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49722/0 (socket says 172.21.15.37:49722) 2024-06-10T01:01:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.162+0000 7f160d7b7640 1 -- 172.21.15.37:0/2440608564 learned_addr learned my addr 172.21.15.37:0/2440608564 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.162+0000 7f160dfb8640 1 --2- 172.21.15.37:0/2440608564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8c86e3010 0x55f8c86f1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.162+0000 7f160e7b9640 1 --2- 172.21.15.37:0/2440608564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8c86e8b70 0x55f8c87615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.162+0000 7f160d7b7640 1 -- 172.21.15.37:0/2440608564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8c86e8b70 msgr2=0x55f8c87615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.162+0000 7f160d7b7640 1 --2- 172.21.15.37:0/2440608564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8c86e8b70 0x55f8c87615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.162+0000 7f160d7b7640 1 -- 172.21.15.37:0/2440608564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8c86e3010 msgr2=0x55f8c86f1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.162+0000 7f160d7b7640 1 --2- 172.21.15.37:0/2440608564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8c86e3010 0x55f8c86f1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.162+0000 7f160d7b7640 1 -- 172.21.15.37:0/2440608564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8c86de4b0 con 0x55f8c86e3a00 2024-06-10T01:01:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.162+0000 7f160e7b9640 1 --2- 172.21.15.37:0/2440608564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8c86e8b70 0x55f8c87615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:01:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.162+0000 7f160d7b7640 1 --2- 172.21.15.37:0/2440608564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8c86e3a00 0x55f8c86f1890 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f15f800ea30 tx=0x7f15f80ba580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c798acacd5c5cd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.162+0000 7f15f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:01:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.163+0000 7f15f17fa640 1 -- 172.21.15.37:0/2440608564 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f80a9070 con 0x55f8c86e3a00 2024-06-10T01:01:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.163+0000 7f15f17fa640 1 -- 172.21.15.37:0/2440608564 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f15f80cf280 con 0x55f8c86e3a00 2024-06-10T01:01:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.163+0000 7f161300a740 1 -- 172.21.15.37:0/2440608564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15f80a46b0 con 0x55f8c86e3a00 2024-06-10T01:01:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.163+0000 7f15f17fa640 1 -- 172.21.15.37:0/2440608564 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f15f809ae90 con 0x55f8c86e3a00 2024-06-10T01:01:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.163+0000 7f161300a740 10 client.16131 fetch_fsmap learned FSMap version 27 2024-06-10T01:01:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.163+0000 7f161300a740 1 -- 172.21.15.37:0/2440608564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f8c8762aa0 con 0x55f8c86e3a00 2024-06-10T01:01:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.163+0000 7f15f17fa640 1 -- 172.21.15.37:0/2440608564 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15f80a46b0 con 0x55f8c86e3a00 2024-06-10T01:01:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.163+0000 7f15f17fa640 1 -- 172.21.15.37:0/2440608564 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f15f809a900 con 0x55f8c86e3a00 2024-06-10T01:01:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.163+0000 7f161300a740 10 client.16131 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:01:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.163+0000 7f161300a740 10 client.16131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:01:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.163+0000 7f161300a740 20 client.16131 populate_metadata read hostname 'smithi037' 2024-06-10T01:01:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.163+0000 7f161300a740 1 --2- 172.21.15.37:0/2440608564 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f8c8763220 0x55f8c8765640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.163+0000 7f161300a740 4 client.16131 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:01:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.163+0000 7f161300a740 1 -- 172.21.15.37:0/2440608564 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f8c86e46f0 con 0x55f8c8763220 2024-06-10T01:01:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.164+0000 7f160dfb8640 1 --2- 172.21.15.37:0/2440608564 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f8c8763220 0x55f8c8765640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:25.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.164+0000 7f15f17fa640 10 client.16131 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:25.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.164+0000 7f160dfb8640 1 --2- 172.21.15.37:0/2440608564 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f8c8763220 0x55f8c8765640 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=344b601eaffa5ee7 server_cookie=fce93af5f49ac2ab in_seq=0 out_seq=0 2024-06-10T01:01:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.182+0000 7f15f17fa640 1 -- 172.21.15.37:0/2440608564 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f8c86e46f0 con 0x55f8c8763220 2024-06-10T01:01:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.183+0000 7f15f17fa640 10 client.16131 handle_command_reply: tid=0 2024-06-10T01:01:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.183+0000 7f161300a740 1 client.16131 shutdown 2024-06-10T01:01:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.183+0000 7f161300a740 1 -- 172.21.15.37:0/2440608564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8c86e3a00 msgr2=0x55f8c86f1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.183+0000 7f161300a740 1 --2- 172.21.15.37:0/2440608564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8c86e3a00 0x55f8c86f1890 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f15f800ea30 tx=0x7f15f80ba580 comp rx=0 tx=0).stop 2024-06-10T01:01:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.183+0000 7f161300a740 1 -- 172.21.15.37:0/2440608564 shutdown_connections 2024-06-10T01:01:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.183+0000 7f161300a740 1 --2- 172.21.15.37:0/2440608564 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f8c8763220 0x55f8c8765640 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.183+0000 7f161300a740 1 --2- 172.21.15.37:0/2440608564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f8c86e8b70 0x55f8c87615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.183+0000 7f161300a740 1 --2- 172.21.15.37:0/2440608564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f8c86e3a00 0x55f8c86f1890 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.183+0000 7f15f17fa640 0 client.16131 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.183+0000 7f161300a740 1 --2- 172.21.15.37:0/2440608564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8c86e3010 0x55f8c86f1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.183+0000 7f161300a740 1 -- 172.21.15.37:0/2440608564 >> 172.21.15.37:0/2440608564 conn(0x55f8c86de680 msgr2=0x55f8c86e7e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.183+0000 7f161300a740 1 -- 172.21.15.37:0/2440608564 shutdown_connections 2024-06-10T01:01:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.183+0000 7f161300a740 1 -- 172.21.15.37:0/2440608564 wait complete. 2024-06-10T01:01:25.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.183+0000 7f161300a740 20 client.16131 trim_cache size 0 max 16384 2024-06-10T01:01:25.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:01:25.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:01:25.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:01:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.184+0000 7f161021c640 1 -- 172.21.15.37:0/2363745365 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f15f40a2da0 msgr2=0x7f15f40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.184+0000 7f161021c640 1 --2- 172.21.15.37:0/2363745365 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f15f40a2da0 0x7f15f40a5260 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1608102a80 tx=0x7f1604005f90 comp rx=0 tx=0).stop 2024-06-10T01:01:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.184+0000 7f161021c640 1 -- 172.21.15.37:0/2363745365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1608102c20 msgr2=0x7f160819a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.184+0000 7f161021c640 1 --2- 172.21.15.37:0/2363745365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1608102c20 0x7f160819a3a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f15f800ea60 tx=0x7f15f800ef20 comp rx=0 tx=0).stop 2024-06-10T01:01:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.184+0000 7f161021c640 1 -- 172.21.15.37:0/2363745365 shutdown_connections 2024-06-10T01:01:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.184+0000 7f161021c640 1 --2- 172.21.15.37:0/2363745365 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f15f40a2da0 0x7f15f40a5260 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.184+0000 7f161021c640 1 --2- 172.21.15.37:0/2363745365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16081035e0 0x7f16081a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.184+0000 7f161021c640 1 --2- 172.21.15.37:0/2363745365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1608102c20 0x7f160819a3a0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.184+0000 7f161021c640 1 --2- 172.21.15.37:0/2363745365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1608101a30 0x7f1608199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.184+0000 7f161021c640 1 -- 172.21.15.37:0/2363745365 >> 172.21.15.37:0/2363745365 conn(0x7f16080fcfc0 msgr2=0x7f1608073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.184+0000 7f161021c640 1 -- 172.21.15.37:0/2363745365 shutdown_connections 2024-06-10T01:01:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:25.184+0000 7f161021c640 1 -- 172.21.15.37:0/2363745365 wait complete. 2024-06-10T01:01:25.266 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:25.266 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:25.656 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:01:25.656 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:01:25.669 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:25.670 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:25.670 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 92 -lt 100 ']' 2024-06-10T01:01:25.670 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 92 + 1 2024-06-10T01:01:25.724 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=93 2024-06-10T01:01:25.724 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:01:25.791 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:25.791 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:25.792 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:01:25.792 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:01:25.792 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:01:26.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:26 smithi167 ceph-mon[36917]: pgmap v572: 129 pgs: 129 active+clean; 2.0 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.0 KiB/s wr, 7 op/s 2024-06-10T01:01:26.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:26 smithi094 ceph-mon[36708]: pgmap v572: 129 pgs: 129 active+clean; 2.0 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.0 KiB/s wr, 7 op/s 2024-06-10T01:01:26.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:26 smithi037 ceph-mon[32192]: pgmap v572: 129 pgs: 129 active+clean; 2.0 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.0 KiB/s wr, 7 op/s 2024-06-10T01:01:26.674 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:26.674 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:27.053 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:01:27.053 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:01:27.078 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:27.078 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:27.078 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 93 -lt 100 ']' 2024-06-10T01:01:27.078 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 93 + 1 2024-06-10T01:01:27.123 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=94 2024-06-10T01:01:27.123 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:01:27.138 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:27.138 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:27.139 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:01:27.140 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:01:27.140 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:01:27.672 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:27.672 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:27.930 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:01:27.930 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:01:27.952 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:27.952 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:27.953 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 94 -lt 100 ']' 2024-06-10T01:01:27.953 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 94 + 1 2024-06-10T01:01:28.022 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=95 2024-06-10T01:01:28.022 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:01:28.093 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:28.093 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:28.094 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:01:28.094 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:01:28.094 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:01:28.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:28 smithi167 ceph-mon[36917]: pgmap v573: 129 pgs: 129 active+clean; 2.0 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-06-10T01:01:28.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:28 smithi094 ceph-mon[36708]: pgmap v573: 129 pgs: 129 active+clean; 2.0 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-06-10T01:01:28.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:28 smithi037 ceph-mon[32192]: pgmap v573: 129 pgs: 129 active+clean; 2.0 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-06-10T01:01:28.791 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:28.791 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:29.238 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:01:29.238 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:01:29.287 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:29.287 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:29.287 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 95 -lt 100 ']' 2024-06-10T01:01:29.287 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 95 + 1 2024-06-10T01:01:29.437 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=96 2024-06-10T01:01:29.438 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:01:29.576 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:29.576 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:29.577 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:01:29.577 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:01:29.577 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:01:30.136 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:30 smithi037 ceph-mon[32192]: pgmap v574: 129 pgs: 129 active+clean; 2.0 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-06-10T01:01:30.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:30 smithi167 ceph-mon[36917]: pgmap v574: 129 pgs: 129 active+clean; 2.0 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-06-10T01:01:30.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:30 smithi094 ceph-mon[36708]: pgmap v574: 129 pgs: 129 active+clean; 2.0 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-06-10T01:01:30.663 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:30.663 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:30.963 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:01:30.963 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:01:31.002 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:31.002 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:31.002 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 96 -lt 100 ']' 2024-06-10T01:01:31.002 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 96 + 1 2024-06-10T01:01:31.037 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=97 2024-06-10T01:01:31.037 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:01:31.074 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:31.074 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:31.075 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:01:31.075 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:01:31.075 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:01:31.782 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:31.783 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:32.103 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:01:32.104 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:01:32.136 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:32.136 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:32.136 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 97 -lt 100 ']' 2024-06-10T01:01:32.136 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 97 + 1 2024-06-10T01:01:32.218 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=98 2024-06-10T01:01:32.219 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:01:32.285 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:32.285 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:32.286 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:01:32.286 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:01:32.287 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:01:32.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:32 smithi167 ceph-mon[36917]: pgmap v575: 129 pgs: 129 active+clean; 2.0 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-06-10T01:01:32.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:32 smithi094 ceph-mon[36708]: pgmap v575: 129 pgs: 129 active+clean; 2.0 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-06-10T01:01:32.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:32 smithi037 ceph-mon[32192]: pgmap v575: 129 pgs: 129 active+clean; 2.0 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-06-10T01:01:32.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.931+0000 7f47a1977700 1 -- 172.21.15.37:0/3805317000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c107840 msgr2=0x7f479c10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:32.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.931+0000 7f47a1977700 1 --2- 172.21.15.37:0/3805317000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c107840 0x7f479c10f3c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f479400b0d0 tx=0x7f479400b3e0 comp rx=0 tx=0).stop 2024-06-10T01:01:32.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.932+0000 7f47a1977700 1 -- 172.21.15.37:0/3805317000 shutdown_connections 2024-06-10T01:01:32.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.932+0000 7f47a1977700 1 --2- 172.21.15.37:0/3805317000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c107840 0x7f479c10f3c0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:32.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.932+0000 7f47a1977700 1 --2- 172.21.15.37:0/3805317000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f479c102740 0x7f479c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:32.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.932+0000 7f47a1977700 1 --2- 172.21.15.37:0/3805317000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f479c1030f0 0x7f479c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:32.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.932+0000 7f47a1977700 1 -- 172.21.15.37:0/3805317000 >> 172.21.15.37:0/3805317000 conn(0x7f479c0795f0 msgr2=0x7f479c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:32.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.932+0000 7f47a1977700 1 -- 172.21.15.37:0/3805317000 shutdown_connections 2024-06-10T01:01:32.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.932+0000 7f47a1977700 1 -- 172.21.15.37:0/3805317000 wait complete. 2024-06-10T01:01:32.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.933+0000 7f47a1977700 1 Processor -- start 2024-06-10T01:01:32.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.933+0000 7f47a1977700 1 -- start start 2024-06-10T01:01:32.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.933+0000 7f47a1977700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f479c102740 0x7f479c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:32.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.933+0000 7f47a1977700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f479c1030f0 0x7f479c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:32.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.933+0000 7f47a1977700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c107840 0x7f479c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:32.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.933+0000 7f47a1977700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f479c112780 con 0x7f479c107840 2024-06-10T01:01:32.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.933+0000 7f47a1977700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f479c112600 con 0x7f479c102740 2024-06-10T01:01:32.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.933+0000 7f47a1977700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f479c112900 con 0x7f479c1030f0 2024-06-10T01:01:32.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.934+0000 7f479affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f479c102740 0x7f479c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:32.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.934+0000 7f479affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f479c102740 0x7f479c074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:50402/0 (socket says 172.21.15.37:50402) 2024-06-10T01:01:32.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.934+0000 7f479affd700 1 -- 172.21.15.37:0/3901582591 learned_addr learned my addr 172.21.15.37:0/3901582591 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:32.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.934+0000 7f479b7fe700 1 --2- 172.21.15.37:0/3901582591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c107840 0x7f479c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:32.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.934+0000 7f479a7fc700 1 --2- 172.21.15.37:0/3901582591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f479c1030f0 0x7f479c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:32.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.934+0000 7f479affd700 1 -- 172.21.15.37:0/3901582591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f479c1030f0 msgr2=0x7f479c074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:32.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.934+0000 7f479affd700 1 --2- 172.21.15.37:0/3901582591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f479c1030f0 0x7f479c074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:32.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.934+0000 7f479affd700 1 -- 172.21.15.37:0/3901582591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c107840 msgr2=0x7f479c075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:32.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.934+0000 7f479affd700 1 --2- 172.21.15.37:0/3901582591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c107840 0x7f479c075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:32.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.934+0000 7f479affd700 1 -- 172.21.15.37:0/3901582591 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479c1a6fe0 con 0x7f479c102740 2024-06-10T01:01:32.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.934+0000 7f479affd700 1 --2- 172.21.15.37:0/3901582591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f479c102740 0x7f479c074790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f478c00c9b0 tx=0x7f478c00ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:32.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.935+0000 7f47a0975700 1 -- 172.21.15.37:0/3901582591 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c007b00 con 0x7f479c102740 2024-06-10T01:01:32.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.935+0000 7f47a0975700 1 -- 172.21.15.37:0/3901582591 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f478c00ce50 con 0x7f479c102740 2024-06-10T01:01:32.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.935+0000 7f47a1977700 1 -- 172.21.15.37:0/3901582591 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f479c1a72d0 con 0x7f479c102740 2024-06-10T01:01:32.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.935+0000 7f47a0975700 1 -- 172.21.15.37:0/3901582591 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c018660 con 0x7f479c102740 2024-06-10T01:01:32.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.935+0000 7f47a1977700 1 -- 172.21.15.37:0/3901582591 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f479c1a7860 con 0x7f479c102740 2024-06-10T01:01:32.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.937+0000 7f47a0975700 1 -- 172.21.15.37:0/3901582591 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f478c00f450 con 0x7f479c102740 2024-06-10T01:01:32.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.938+0000 7f47a0975700 1 --2- 172.21.15.37:0/3901582591 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f47840a2ca0 0x7f47840a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:32.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.938+0000 7f479a7fc700 1 --2- 172.21.15.37:0/3901582591 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f47840a2ca0 0x7f47840a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:32.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.938+0000 7f47a0975700 1 -- 172.21.15.37:0/3901582591 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f478c0cf350 con 0x7f479c102740 2024-06-10T01:01:32.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.939+0000 7f479a7fc700 1 --2- 172.21.15.37:0/3901582591 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f47840a2ca0 0x7f47840a5160 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f479c108bb0 tx=0x7f479000b3c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:32.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.940+0000 7f47a1977700 1 -- 172.21.15.37:0/3901582591 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f479c103bd0 con 0x7f479c102740 2024-06-10T01:01:32.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:32.949+0000 7f47a0975700 1 -- 172.21.15.37:0/3901582591 <== mon.2 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f478c014030 con 0x7f479c102740 2024-06-10T01:01:33.023 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:33.023 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:33.196 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:01:33.196 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:01:33.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:33.208+0000 7f47a1977700 1 -- 172.21.15.37:0/3901582591 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f479c1a7b20 con 0x7f479c102740 2024-06-10T01:01:33.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:33.208+0000 7f47a0975700 1 -- 172.21.15.37:0/3901582591 <== mon.2 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11867 (secure 0 0 0) 0x7f478c09be60 con 0x7f479c102740 2024-06-10T01:01:33.207 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:01:33.207 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T00:56:30.291171+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24451,24458,24472]},"id":1}]} 2024-06-10T01:01:33.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:33.211+0000 7f47a1977700 1 -- 172.21.15.37:0/3901582591 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f47840a2ca0 msgr2=0x7f47840a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:33.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:33.211+0000 7f47a1977700 1 --2- 172.21.15.37:0/3901582591 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f47840a2ca0 0x7f47840a5160 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f479c108bb0 tx=0x7f479000b3c0 comp rx=0 tx=0).stop 2024-06-10T01:01:33.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:33.211+0000 7f47a1977700 1 -- 172.21.15.37:0/3901582591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f479c102740 msgr2=0x7f479c074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:33.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:33.211+0000 7f47a1977700 1 --2- 172.21.15.37:0/3901582591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f479c102740 0x7f479c074790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f478c00c9b0 tx=0x7f478c00ccc0 comp rx=0 tx=0).stop 2024-06-10T01:01:33.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:33.211+0000 7f47a1977700 1 -- 172.21.15.37:0/3901582591 shutdown_connections 2024-06-10T01:01:33.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:33.211+0000 7f47a1977700 1 --2- 172.21.15.37:0/3901582591 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f47840a2ca0 0x7f47840a5160 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:33.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:33.211+0000 7f47a1977700 1 --2- 172.21.15.37:0/3901582591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c107840 0x7f479c075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:33.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:33.211+0000 7f47a1977700 1 --2- 172.21.15.37:0/3901582591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f479c102740 0x7f479c074790 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:33.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:33.211+0000 7f47a1977700 1 --2- 172.21.15.37:0/3901582591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f479c1030f0 0x7f479c074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:33.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:33.211+0000 7f47a1977700 1 -- 172.21.15.37:0/3901582591 >> 172.21.15.37:0/3901582591 conn(0x7f479c0795f0 msgr2=0x7f479c100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:33.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:33.211+0000 7f47a1977700 1 -- 172.21.15.37:0/3901582591 shutdown_connections 2024-06-10T01:01:33.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:33.211+0000 7f47a1977700 1 -- 172.21.15.37:0/3901582591 wait complete. 2024-06-10T01:01:33.211 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-06-10T01:01:33.220 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:33.220 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:33.221 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 98 -lt 100 ']' 2024-06-10T01:01:33.221 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 98 + 1 2024-06-10T01:01:33.272 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=99 2024-06-10T01:01:33.272 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:01:33.326 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:33.327 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:33.327 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:01:33.327 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:01:33.328 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:01:34.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:34 smithi167 ceph-mon[36917]: pgmap v576: 129 pgs: 129 active+clean; 2.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-06-10T01:01:34.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:34 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3901582591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:01:34.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:34 smithi094 ceph-mon[36708]: pgmap v576: 129 pgs: 129 active+clean; 2.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-06-10T01:01:34.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:34 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3901582591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:01:34.386 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:34 smithi037 ceph-mon[32192]: pgmap v576: 129 pgs: 129 active+clean; 2.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-06-10T01:01:34.386 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:34 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3901582591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:01:34.525 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:34.525 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:34.921 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:01:34.921 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:01:34.928 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:34.929 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:34.929 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 99 -lt 100 ']' 2024-06-10T01:01:34.929 INFO:tasks.workunit.client.0.smithi037.stderr:++ expr 99 + 1 2024-06-10T01:01:35.178 INFO:tasks.workunit.client.0.smithi037.stderr:+ c=100 2024-06-10T01:01:35.178 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:01:35.268 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:35.268 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:35.269 INFO:tasks.workunit.client.0.smithi037.stderr:+ c1=2 2024-06-10T01:01:35.269 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo 3 2024-06-10T01:01:35.269 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-10T01:01:36.299 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:36 smithi037 ceph-mon[32192]: pgmap v577: 129 pgs: 129 active+clean; 2.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-06-10T01:01:36.301 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfacl d1 2024-06-10T01:01:36.301 INFO:tasks.workunit.client.0.smithi037.stderr:++ grep -c nobody:rw 2024-06-10T01:01:36.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:36 smithi167 ceph-mon[36917]: pgmap v577: 129 pgs: 129 active+clean; 2.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-06-10T01:01:36.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:36 smithi094 ceph-mon[36708]: pgmap v577: 129 pgs: 129 active+clean; 2.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-06-10T01:01:36.480 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:01:36.554 INFO:tasks.workunit.client.0.smithi037.stderr:+ c2=2 2024-06-10T01:01:36.554 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:01:36.576 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:36.576 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 2 -ne 2 ']' 2024-06-10T01:01:36.577 INFO:tasks.workunit.client.0.smithi037.stderr:+ '[' 100 -lt 100 ']' 2024-06-10T01:01:36.597 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir d1 2024-06-10T01:01:36.673 INFO:tasks.workunit.client.0.smithi037.stderr:+ setfattr -n system.posix_acl_access -v 0x02000000 d1 2024-06-10T01:01:36.685 INFO:tasks.workunit.client.0.smithi037.stderr:+ setfattr -n system.posix_acl_default -v 0x02000000 . 2024-06-10T01:01:36.687 INFO:tasks.workunit.client.0.smithi037.stderr:+ expect_failure getfattr -n system.posix_acl_access d1 2024-06-10T01:01:36.687 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n system.posix_acl_access d1 2024-06-10T01:01:36.691 INFO:tasks.workunit.client.0.smithi037.stderr:d1: system.posix_acl_access: No such attribute 2024-06-10T01:01:36.691 INFO:tasks.workunit.client.0.smithi037.stderr:+ return 0 2024-06-10T01:01:36.691 INFO:tasks.workunit.client.0.smithi037.stderr:+ expect_failure getfattr -n system.posix_acl_default . 2024-06-10T01:01:36.691 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n system.posix_acl_default . 2024-06-10T01:01:36.691 INFO:tasks.workunit.client.0.smithi037.stderr:.: system.posix_acl_default: No such attribute 2024-06-10T01:01:36.691 INFO:tasks.workunit.client.0.smithi037.stderr:+ return 0 2024-06-10T01:01:36.692 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir d1 2024-06-10T01:01:36.695 INFO:tasks.workunit.client.0.smithi037.stderr:+ cd .. 2024-06-10T01:01:36.701 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir testdir 2024-06-10T01:01:36.708 INFO:tasks.workunit.client.0.smithi037.stdout:OK 2024-06-10T01:01:36.709 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo OK 2024-06-10T01:01:36.709 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-10T01:01:36.710 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-10T01:01:37.325 INFO:tasks.workunit:Running workunit fs/misc/chmod.sh... 2024-06-10T01:01:37.325 DEBUG:teuthology.orchestra.run.smithi037:workunit test fs/misc/chmod.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b7a5d375cf4e4afab96b9d4d229b699ae8304bb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh 2024-06-10T01:01:37.456 INFO:tasks.workunit.client.0.smithi037.stderr:+ set -e 2024-06-10T01:01:37.456 INFO:tasks.workunit.client.0.smithi037.stderr:+ file=test_chmod.99950 2024-06-10T01:01:37.456 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo foo 2024-06-10T01:01:37.459 INFO:tasks.workunit.client.0.smithi037.stderr:+ test 0 '!=' 0 2024-06-10T01:01:37.460 INFO:tasks.workunit.client.0.smithi037.stderr:+ chmod 400 test_chmod.99950 2024-06-10T01:01:37.462 INFO:tasks.workunit.client.0.smithi037.stderr:+ test 0 '!=' 0 2024-06-10T01:01:37.462 INFO:tasks.workunit.client.0.smithi037.stderr:+ check_perms test_chmod.99950 -r-------- 2024-06-10T01:01:37.462 INFO:tasks.workunit.client.0.smithi037.stderr:+ file=test_chmod.99950 2024-06-10T01:01:37.463 INFO:tasks.workunit.client.0.smithi037.stderr:++ ls -la test_chmod.99950 2024-06-10T01:01:37.484 INFO:tasks.workunit.client.0.smithi037.stderr:+ r='-r--------. 1 ubuntu ubuntu 4 Jun 10 01:01 test_chmod.99950' 2024-06-10T01:01:37.484 INFO:tasks.workunit.client.0.smithi037.stderr:+ test 0 '!=' 0 2024-06-10T01:01:37.484 INFO:tasks.workunit.client.0.smithi037.stderr:+ perms=-r-------- 2024-06-10T01:01:37.485 INFO:tasks.workunit.client.0.smithi037.stderr:++ echo -r--------. 1 ubuntu ubuntu 4 Jun 10 01:01 test_chmod.99950 2024-06-10T01:01:37.485 INFO:tasks.workunit.client.0.smithi037.stderr:++ awk '{print $1}' 2024-06-10T01:01:37.601 INFO:tasks.workunit.client.0.smithi037.stderr:+ test -r-------- '!=' -r--------. 2024-06-10T01:01:37.601 INFO:tasks.workunit.client.0.smithi037.stderr:++ echo -r--------. 1 ubuntu ubuntu 4 Jun 10 01:01 test_chmod.99950 2024-06-10T01:01:37.601 INFO:tasks.workunit.client.0.smithi037.stderr:++ awk '{print $1}' 2024-06-10T01:01:37.604 INFO:tasks.workunit.client.0.smithi037.stderr:+ test -r--------. '!=' -r--------. 2024-06-10T01:01:37.604 INFO:tasks.workunit.client.0.smithi037.stderr:+ set +e 2024-06-10T01:01:37.604 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo bar 2024-06-10T01:01:37.604 INFO:tasks.workunit.client.0.smithi037.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh: line 40: test_chmod.99950: Permission denied 2024-06-10T01:01:37.604 INFO:tasks.workunit.client.0.smithi037.stderr:+ test 1 = 0 2024-06-10T01:01:37.604 INFO:tasks.workunit.client.0.smithi037.stderr:+ set -e 2024-06-10T01:01:37.604 INFO:tasks.workunit.client.0.smithi037.stderr:+ chmod 600 test_chmod.99950 2024-06-10T01:01:37.605 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo bar 2024-06-10T01:01:37.605 INFO:tasks.workunit.client.0.smithi037.stderr:+ test 0 '!=' 0 2024-06-10T01:01:37.605 INFO:tasks.workunit.client.0.smithi037.stderr:+ check_perms test_chmod.99950 -rw------- 2024-06-10T01:01:37.605 INFO:tasks.workunit.client.0.smithi037.stderr:+ file=test_chmod.99950 2024-06-10T01:01:37.606 INFO:tasks.workunit.client.0.smithi037.stderr:++ ls -la test_chmod.99950 2024-06-10T01:01:37.606 INFO:tasks.workunit.client.0.smithi037.stderr:+ r='-rw-------. 1 ubuntu ubuntu 8 Jun 10 01:01 test_chmod.99950' 2024-06-10T01:01:37.606 INFO:tasks.workunit.client.0.smithi037.stderr:+ test 0 '!=' 0 2024-06-10T01:01:37.606 INFO:tasks.workunit.client.0.smithi037.stderr:+ perms=-rw------- 2024-06-10T01:01:37.607 INFO:tasks.workunit.client.0.smithi037.stderr:++ echo -rw-------. 1 ubuntu ubuntu 8 Jun 10 01:01 test_chmod.99950 2024-06-10T01:01:37.607 INFO:tasks.workunit.client.0.smithi037.stderr:++ awk '{print $1}' 2024-06-10T01:01:37.608 INFO:tasks.workunit.client.0.smithi037.stderr:+ test -rw------- '!=' -rw-------. 2024-06-10T01:01:37.609 INFO:tasks.workunit.client.0.smithi037.stderr:++ echo -rw-------. 1 ubuntu ubuntu 8 Jun 10 01:01 test_chmod.99950 2024-06-10T01:01:37.609 INFO:tasks.workunit.client.0.smithi037.stderr:++ awk '{print $1}' 2024-06-10T01:01:37.611 INFO:tasks.workunit.client.0.smithi037.stderr:+ test -rw-------. '!=' -rw-------. 2024-06-10T01:01:37.611 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo foo 2024-06-10T01:01:37.611 INFO:tasks.workunit.client.0.smithi037.stderr:+ test 0 '!=' 0 2024-06-10T01:01:37.611 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-10T01:01:37.612 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-10T01:01:37.654 INFO:tasks.workunit:Running workunit fs/misc/dac_override.sh... 2024-06-10T01:01:37.654 DEBUG:teuthology.orchestra.run.smithi037:workunit test fs/misc/dac_override.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b7a5d375cf4e4afab96b9d4d229b699ae8304bb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dac_override.sh 2024-06-10T01:01:37.719 INFO:tasks.workunit.client.0.smithi037.stderr:+ set -e 2024-06-10T01:01:37.719 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir -p testdir 2024-06-10T01:01:37.721 INFO:tasks.workunit.client.0.smithi037.stderr:+ file=test_chmod.100003 2024-06-10T01:01:37.721 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo foo 2024-06-10T01:01:37.722 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo chmod 600 testdir 2024-06-10T01:01:37.734 INFO:tasks.workunit.client.0.smithi037.stderr:+ expect_failure cat testdir/test_chmod.100003 2024-06-10T01:01:37.734 INFO:tasks.workunit.client.0.smithi037.stderr:+ cat testdir/test_chmod.100003 2024-06-10T01:01:37.735 INFO:tasks.workunit.client.0.smithi037.stderr:cat: testdir/test_chmod.100003: Permission denied 2024-06-10T01:01:37.735 INFO:tasks.workunit.client.0.smithi037.stderr:+ return 0 2024-06-10T01:01:37.735 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo cat testdir/test_chmod.100003 2024-06-10T01:01:37.745 INFO:tasks.workunit.client.0.smithi037.stdout:foo 2024-06-10T01:01:37.747 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-10T01:01:37.747 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-10T01:01:37.834 INFO:tasks.workunit:Running workunit fs/misc/direct_io.py... 2024-06-10T01:01:37.835 DEBUG:teuthology.orchestra.run.smithi037:workunit test fs/misc/direct_io.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b7a5d375cf4e4afab96b9d4d229b699ae8304bb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/direct_io.py 2024-06-10T01:01:38.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:38 smithi167 ceph-mon[36917]: pgmap v578: 129 pgs: 129 active+clean; 2.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-06-10T01:01:38.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:38 smithi094 ceph-mon[36708]: pgmap v578: 129 pgs: 129 active+clean; 2.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-06-10T01:01:38.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:38 smithi037 ceph-mon[32192]: pgmap v578: 129 pgs: 129 active+clean; 2.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-06-10T01:01:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.252+0000 7faf48568640 1 -- 172.21.15.37:0/2203143692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf40107760 msgr2=0x7faf40109b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.252+0000 7faf48568640 1 --2- 172.21.15.37:0/2203143692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf40107760 0x7faf40109b50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7faf38009ca0 tx=0x7faf3802a8c0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.253+0000 7faf48568640 1 -- 172.21.15.37:0/2203143692 shutdown_connections 2024-06-10T01:01:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.253+0000 7faf48568640 1 --2- 172.21.15.37:0/2203143692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf40107760 0x7faf40109b50 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.253+0000 7faf48568640 1 --2- 172.21.15.37:0/2203143692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf40104e30 0x7faf40107220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.253+0000 7faf48568640 1 --2- 172.21.15.37:0/2203143692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf40102500 0x7faf401048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.253+0000 7faf48568640 1 -- 172.21.15.37:0/2203143692 >> 172.21.15.37:0/2203143692 conn(0x7faf400fbe70 msgr2=0x7faf400fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.253+0000 7faf48568640 1 -- 172.21.15.37:0/2203143692 shutdown_connections 2024-06-10T01:01:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.253+0000 7faf48568640 1 -- 172.21.15.37:0/2203143692 wait complete. 2024-06-10T01:01:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.318+0000 7faf48568640 1 Processor -- start 2024-06-10T01:01:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.319+0000 7faf48568640 1 -- start start 2024-06-10T01:01:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.319+0000 7faf48568640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf40102500 0x7faf40101e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.319+0000 7faf48568640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf40104e30 0x7faf401004b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.319+0000 7faf48568640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf40107760 0x7faf401009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.319+0000 7faf48568640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faf4010d6a0 con 0x7faf40102500 2024-06-10T01:01:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.319+0000 7faf48568640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faf4010d520 con 0x7faf40107760 2024-06-10T01:01:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.319+0000 7faf48568640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faf4010d820 con 0x7faf40104e30 2024-06-10T01:01:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.319+0000 7faf46304640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf40102500 0x7faf40101e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.319+0000 7faf45b03640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf40104e30 0x7faf401004b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.319+0000 7faf46304640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf40102500 0x7faf40101e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55756/0 (socket says 172.21.15.37:55756) 2024-06-10T01:01:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.319+0000 7faf46304640 1 -- 172.21.15.37:0/2032330849 learned_addr learned my addr 172.21.15.37:0/2032330849 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.319+0000 7faf46b05640 1 --2- 172.21.15.37:0/2032330849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf40107760 0x7faf401009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.320+0000 7faf46b05640 1 -- 172.21.15.37:0/2032330849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf40104e30 msgr2=0x7faf401004b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.320+0000 7faf46b05640 1 --2- 172.21.15.37:0/2032330849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf40104e30 0x7faf401004b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.320+0000 7faf46b05640 1 -- 172.21.15.37:0/2032330849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf40102500 msgr2=0x7faf40101e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.320+0000 7faf46b05640 1 --2- 172.21.15.37:0/2032330849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf40102500 0x7faf40101e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.320+0000 7faf46b05640 1 -- 172.21.15.37:0/2032330849 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf40101280 con 0x7faf40107760 2024-06-10T01:01:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.320+0000 7faf46b05640 1 --2- 172.21.15.37:0/2032330849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf40107760 0x7faf401009f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faf38031ae0 tx=0x7faf38004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.323+0000 7faf337fe640 1 -- 172.21.15.37:0/2032330849 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38003b70 con 0x7faf40107760 2024-06-10T01:01:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.323+0000 7faf337fe640 1 -- 172.21.15.37:0/2032330849 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf38003d10 con 0x7faf40107760 2024-06-10T01:01:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.323+0000 7faf337fe640 1 -- 172.21.15.37:0/2032330849 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3802fc40 con 0x7faf40107760 2024-06-10T01:01:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.324+0000 7faf48568640 1 -- 172.21.15.37:0/2032330849 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf401a1da0 con 0x7faf40107760 2024-06-10T01:01:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.324+0000 7faf48568640 1 -- 172.21.15.37:0/2032330849 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf401a21b0 con 0x7faf40107760 2024-06-10T01:01:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.327+0000 7faf337fe640 1 -- 172.21.15.37:0/2032330849 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7faf38004050 con 0x7faf40107760 2024-06-10T01:01:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.328+0000 7faf337fe640 1 --2- 172.21.15.37:0/2032330849 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7faf2c0a2da0 0x7faf2c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.328+0000 7faf46304640 1 --2- 172.21.15.37:0/2032330849 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7faf2c0a2da0 0x7faf2c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.328+0000 7faf337fe640 1 -- 172.21.15.37:0/2032330849 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7faf380f0560 con 0x7faf40107760 2024-06-10T01:01:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.328+0000 7faf46304640 1 --2- 172.21.15.37:0/2032330849 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7faf2c0a2da0 0x7faf2c0a5260 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7faf340095f0 tx=0x7faf34005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:39.434 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:01:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.474+0000 7faf4b272740 1 Processor -- start 2024-06-10T01:01:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.475+0000 7faf4b272740 1 -- start start 2024-06-10T01:01:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.475+0000 7faf4b272740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579e9c76010 0x5579e9c76430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.475+0000 7faf4b272740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5579e9c76a00 0x5579e9c7b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.475+0000 7faf4b272740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579e9c7bb70 0x5579e9c7e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.475+0000 7faf4b272740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5579e9bd94f0 con 0x5579e9c7bb70 2024-06-10T01:01:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.475+0000 7faf4b272740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5579e9b77870 con 0x5579e9c76a00 2024-06-10T01:01:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.475+0000 7faf4b272740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5579e9bd54b0 con 0x5579e9c76010 2024-06-10T01:01:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.475+0000 7faf46b05640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579e9c7bb70 0x5579e9c7e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.475+0000 7faf46b05640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579e9c7bb70 0x5579e9c7e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55778/0 (socket says 172.21.15.37:55778) 2024-06-10T01:01:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.475+0000 7faf46b05640 1 -- 172.21.15.37:0/1060924397 learned_addr learned my addr 172.21.15.37:0/1060924397 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.475+0000 7faf46304640 1 --2- 172.21.15.37:0/1060924397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579e9c76010 0x5579e9c76430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.475+0000 7faf45b03640 1 --2- 172.21.15.37:0/1060924397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5579e9c76a00 0x5579e9c7b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.475+0000 7faf46b05640 1 -- 172.21.15.37:0/1060924397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579e9c76010 msgr2=0x5579e9c76430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.475+0000 7faf46b05640 1 --2- 172.21.15.37:0/1060924397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579e9c76010 0x5579e9c76430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.475+0000 7faf46b05640 1 -- 172.21.15.37:0/1060924397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5579e9c76a00 msgr2=0x5579e9c7b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.475+0000 7faf46b05640 1 --2- 172.21.15.37:0/1060924397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5579e9c76a00 0x5579e9c7b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.475+0000 7faf46b05640 1 -- 172.21.15.37:0/1060924397 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579e9b939b0 con 0x5579e9c7bb70 2024-06-10T01:01:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.475+0000 7faf46b05640 1 --2- 172.21.15.37:0/1060924397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579e9c7bb70 0x5579e9c7e000 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7faf380b8e30 tx=0x7faf3802f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9862b1b20d77db4e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.476+0000 7faf30bfe640 1 -- 172.21.15.37:0/1060924397 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf380bc9d0 con 0x5579e9c7bb70 2024-06-10T01:01:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.476+0000 7faf30bfe640 1 -- 172.21.15.37:0/1060924397 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf38036030 con 0x5579e9c7bb70 2024-06-10T01:01:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.476+0000 7faf30bfe640 1 -- 172.21.15.37:0/1060924397 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf380bad60 con 0x5579e9c7bb70 2024-06-10T01:01:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.476+0000 7faf4b272740 1 -- 172.21.15.37:0/1060924397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579e9c7bb70 msgr2=0x5579e9c7e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.476+0000 7faf4b272740 1 --2- 172.21.15.37:0/1060924397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579e9c7bb70 0x5579e9c7e000 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7faf380b8e30 tx=0x7faf3802f710 comp rx=0 tx=0).stop 2024-06-10T01:01:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.476+0000 7faf4b272740 1 -- 172.21.15.37:0/1060924397 shutdown_connections 2024-06-10T01:01:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.476+0000 7faf4b272740 1 --2- 172.21.15.37:0/1060924397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579e9c7bb70 0x5579e9c7e000 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.476+0000 7faf4b272740 1 --2- 172.21.15.37:0/1060924397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5579e9c76a00 0x5579e9c7b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.476+0000 7faf4b272740 1 --2- 172.21.15.37:0/1060924397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579e9c76010 0x5579e9c76430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.476+0000 7faf4b272740 1 -- 172.21.15.37:0/1060924397 >> 172.21.15.37:0/1060924397 conn(0x5579e9c71680 msgr2=0x5579e9c71a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.477+0000 7faf4b272740 1 -- 172.21.15.37:0/1060924397 shutdown_connections 2024-06-10T01:01:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.477+0000 7faf4b272740 1 -- 172.21.15.37:0/1060924397 wait complete. 2024-06-10T01:01:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.482+0000 7faf4b272740 1 -- start start 2024-06-10T01:01:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.484+0000 7faf4b272740 1 Processor -- start 2024-06-10T01:01:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.484+0000 7faf4b272740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579e9c76010 0x5579e9c7fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.484+0000 7faf4b272740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5579e9c76a00 0x5579e9c80350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.484+0000 7faf4b272740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579e9c7bb70 0x5579e9c87390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.484+0000 7faf4b272740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5579e9bd94f0 con 0x5579e9c76010 2024-06-10T01:01:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.484+0000 7faf4b272740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5579e9b62ae0 con 0x5579e9c76a00 2024-06-10T01:01:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.484+0000 7faf4b272740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5579e9c7e8e0 con 0x5579e9c7bb70 2024-06-10T01:01:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.484+0000 7faf46304640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579e9c76010 0x5579e9c7fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.484+0000 7faf46304640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579e9c76010 0x5579e9c7fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55784/0 (socket says 172.21.15.37:55784) 2024-06-10T01:01:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.484+0000 7faf46304640 1 -- 172.21.15.37:0/518148799 learned_addr learned my addr 172.21.15.37:0/518148799 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.484+0000 7faf46b05640 1 --2- 172.21.15.37:0/518148799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579e9c7bb70 0x5579e9c87390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.484+0000 7faf45b03640 1 --2- 172.21.15.37:0/518148799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5579e9c76a00 0x5579e9c80350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.484+0000 7faf46304640 1 -- 172.21.15.37:0/518148799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579e9c7bb70 msgr2=0x5579e9c87390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.484+0000 7faf46304640 1 --2- 172.21.15.37:0/518148799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579e9c7bb70 0x5579e9c87390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.484+0000 7faf46304640 1 -- 172.21.15.37:0/518148799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5579e9c76a00 msgr2=0x5579e9c80350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.484+0000 7faf46304640 1 --2- 172.21.15.37:0/518148799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5579e9c76a00 0x5579e9c80350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.484+0000 7faf46304640 1 -- 172.21.15.37:0/518148799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579e98ed210 con 0x5579e9c76010 2024-06-10T01:01:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.484+0000 7faf46304640 1 --2- 172.21.15.37:0/518148799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579e9c76010 0x5579e9c7fe10 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7faf3401b9e0 tx=0x7faf34013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37300b8889caeadd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.484+0000 7faf29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:01:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.485+0000 7faf29ffb640 1 -- 172.21.15.37:0/518148799 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3402c060 con 0x5579e9c76010 2024-06-10T01:01:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.485+0000 7faf29ffb640 1 -- 172.21.15.37:0/518148799 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7faf3401ad20 con 0x5579e9c76010 2024-06-10T01:01:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.485+0000 7faf4b272740 10 client.0 _ll_register_callbacks cb 0x5579e9b9b9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:01:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.488+0000 7faf4b272740 1 -- 172.21.15.37:0/518148799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf38036030 con 0x5579e9c76010 2024-06-10T01:01:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.488+0000 7faf29ffb640 1 -- 172.21.15.37:0/518148799 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faf34027430 con 0x5579e9c76010 2024-06-10T01:01:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.488+0000 7faf4b272740 10 client.16158 fetch_fsmap learned FSMap version 27 2024-06-10T01:01:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.488+0000 7faf4b272740 1 -- 172.21.15.37:0/518148799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=27}) -- 0x5579e99b9620 con 0x5579e9c76010 2024-06-10T01:01:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.495+0000 7faf29ffb640 1 -- 172.21.15.37:0/518148799 <== mon.0 v2:172.21.15.37:3300/0 4 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7faf34026070 con 0x5579e9c76010 2024-06-10T01:01:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.495+0000 7faf4b272740 10 client.16158 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:01:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.495+0000 7faf4b272740 10 client.16158 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:01:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.495+0000 7faf4b272740 20 client.16158 populate_metadata read hostname 'smithi037' 2024-06-10T01:01:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.495+0000 7faf4b272740 1 --2- 172.21.15.37:0/518148799 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5579e9c8ba80 0x5579e9c8df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.495+0000 7faf4b272740 4 client.16158 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:01:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.495+0000 7faf4b272740 1 -- 172.21.15.37:0/518148799 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579e9becf10 con 0x5579e9c8ba80 2024-06-10T01:01:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.495+0000 7faf45b03640 1 --2- 172.21.15.37:0/518148799 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5579e9c8ba80 0x5579e9c8df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.496+0000 7faf45b03640 1 --2- 172.21.15.37:0/518148799 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5579e9c8ba80 0x5579e9c8df20 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ada6ad7a7cfd80c server_cookie=4e36506366d77e16 in_seq=0 out_seq=0 2024-06-10T01:01:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.496+0000 7faf29ffb640 10 client.16158 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.496+0000 7faf29ffb640 1 -- 172.21.15.37:0/518148799 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5579e9becf10 con 0x5579e9c8ba80 2024-06-10T01:01:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.496+0000 7faf29ffb640 10 client.16158 handle_command_reply: tid=0 2024-06-10T01:01:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.497+0000 7faf4b272740 1 client.16158 shutdown 2024-06-10T01:01:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.497+0000 7faf4b272740 1 -- 172.21.15.37:0/518148799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579e9c76010 msgr2=0x5579e9c7fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.497+0000 7faf4b272740 1 --2- 172.21.15.37:0/518148799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579e9c76010 0x5579e9c7fe10 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7faf3401b9e0 tx=0x7faf34013040 comp rx=0 tx=0).stop 2024-06-10T01:01:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.497+0000 7faf4b272740 1 -- 172.21.15.37:0/518148799 shutdown_connections 2024-06-10T01:01:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.497+0000 7faf4b272740 1 --2- 172.21.15.37:0/518148799 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5579e9c8ba80 0x5579e9c8df20 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.497+0000 7faf4b272740 1 --2- 172.21.15.37:0/518148799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579e9c7bb70 0x5579e9c87390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.497+0000 7faf4b272740 1 --2- 172.21.15.37:0/518148799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5579e9c76a00 0x5579e9c80350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.497+0000 7faf4b272740 1 --2- 172.21.15.37:0/518148799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579e9c76010 0x5579e9c7fe10 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.497+0000 7faf4b272740 1 -- 172.21.15.37:0/518148799 >> 172.21.15.37:0/518148799 conn(0x5579e9c71680 msgr2=0x5579e9c707e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.497+0000 7faf29ffb640 0 client.16158 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.497+0000 7faf4b272740 1 -- 172.21.15.37:0/518148799 shutdown_connections 2024-06-10T01:01:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.497+0000 7faf4b272740 1 -- 172.21.15.37:0/518148799 wait complete. 2024-06-10T01:01:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.499+0000 7faf4b272740 20 client.16158 trim_cache size 0 max 16384 2024-06-10T01:01:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.515+0000 7faf4b272740 1 Processor -- start 2024-06-10T01:01:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.515+0000 7faf4b272740 1 -- start start 2024-06-10T01:01:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.515+0000 7faf4b272740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579e9c76010 0x5579e9c7f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.515+0000 7faf4b272740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5579e9c76a00 0x5579e9c87320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.515+0000 7faf4b272740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579e9c7bb70 0x5579e9c74590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.515+0000 7faf4b272740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faf401af1b0 con 0x5579e9c7bb70 2024-06-10T01:01:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.515+0000 7faf4b272740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faf401aca10 con 0x5579e9c76a00 2024-06-10T01:01:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.515+0000 7faf4b272740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faf401aa860 con 0x5579e9c76010 2024-06-10T01:01:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.515+0000 7faf46b05640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579e9c7bb70 0x5579e9c74590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.515+0000 7faf46b05640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579e9c7bb70 0x5579e9c74590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55790/0 (socket says 172.21.15.37:55790) 2024-06-10T01:01:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.515+0000 7faf45b03640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5579e9c76a00 0x5579e9c87320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.515+0000 7faf46b05640 1 -- 172.21.15.37:0/3559048897 learned_addr learned my addr 172.21.15.37:0/3559048897 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.515+0000 7faf46304640 1 --2- 172.21.15.37:0/3559048897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579e9c76010 0x5579e9c7f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.515+0000 7faf45b03640 1 -- 172.21.15.37:0/3559048897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579e9c76010 msgr2=0x5579e9c7f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.516+0000 7faf45b03640 1 --2- 172.21.15.37:0/3559048897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579e9c76010 0x5579e9c7f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.516+0000 7faf45b03640 1 -- 172.21.15.37:0/3559048897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579e9c7bb70 msgr2=0x5579e9c74590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.516+0000 7faf45b03640 1 --2- 172.21.15.37:0/3559048897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579e9c7bb70 0x5579e9c74590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.516+0000 7faf45b03640 1 -- 172.21.15.37:0/3559048897 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579e9c714b0 con 0x5579e9c76a00 2024-06-10T01:01:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.516+0000 7faf46b05640 1 --2- 172.21.15.37:0/3559048897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579e9c7bb70 0x5579e9c74590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:01:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.516+0000 7faf45b03640 1 --2- 172.21.15.37:0/3559048897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5579e9c76a00 0x5579e9c87320 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x5579e9c72550 tx=0x7faf3c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f214e8f82d460a23 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.516+0000 7faf2bfff640 1 -- 172.21.15.37:0/3559048897 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3c022030 con 0x5579e9c76a00 2024-06-10T01:01:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.516+0000 7faf2bfff640 1 -- 172.21.15.37:0/3559048897 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf3c005020 con 0x5579e9c76a00 2024-06-10T01:01:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.516+0000 7faf2bfff640 1 -- 172.21.15.37:0/3559048897 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3c004bc0 con 0x5579e9c76a00 2024-06-10T01:01:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.517+0000 7faf4b272740 1 -- 172.21.15.37:0/3559048897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5579e9c76a00 msgr2=0x5579e9c87320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.517+0000 7faf4b272740 1 --2- 172.21.15.37:0/3559048897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5579e9c76a00 0x5579e9c87320 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x5579e9c72550 tx=0x7faf3c005f90 comp rx=0 tx=0).stop 2024-06-10T01:01:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.517+0000 7faf4b272740 1 -- 172.21.15.37:0/3559048897 shutdown_connections 2024-06-10T01:01:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.517+0000 7faf4b272740 1 --2- 172.21.15.37:0/3559048897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579e9c7bb70 0x5579e9c74590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.517+0000 7faf4b272740 1 --2- 172.21.15.37:0/3559048897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5579e9c76a00 0x5579e9c87320 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.517+0000 7faf4b272740 1 --2- 172.21.15.37:0/3559048897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579e9c76010 0x5579e9c7f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.517+0000 7faf4b272740 1 -- 172.21.15.37:0/3559048897 >> 172.21.15.37:0/3559048897 conn(0x5579e9c71680 msgr2=0x5579e9c77cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.517+0000 7faf4b272740 1 -- 172.21.15.37:0/3559048897 shutdown_connections 2024-06-10T01:01:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.517+0000 7faf4b272740 1 -- 172.21.15.37:0/3559048897 wait complete. 2024-06-10T01:01:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.517+0000 7faf4b272740 1 -- start start 2024-06-10T01:01:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.517+0000 7faf4b272740 1 Processor -- start 2024-06-10T01:01:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.517+0000 7faf4b272740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5579e9c76010 0x5579e9c83f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.517+0000 7faf4b272740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579e9c76a00 0x5579e9c844d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.517+0000 7faf4b272740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579e9c7bb70 0x5579e9cf47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.517+0000 7faf4b272740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faf401aaed0 con 0x5579e9c76a00 2024-06-10T01:01:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.517+0000 7faf4b272740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faf401ac340 con 0x5579e9c76010 2024-06-10T01:01:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.517+0000 7faf4b272740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faf401ab5a0 con 0x5579e9c7bb70 2024-06-10T01:01:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.518+0000 7faf4b272740 10 client.0 _ll_register_callbacks cb 0x7faf3401b870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:01:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.518+0000 7faf45b03640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579e9c76a00 0x5579e9c844d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.518+0000 7faf45b03640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579e9c76a00 0x5579e9c844d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55802/0 (socket says 172.21.15.37:55802) 2024-06-10T01:01:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.518+0000 7faf45b03640 1 -- 172.21.15.37:0/2018951046 learned_addr learned my addr 172.21.15.37:0/2018951046 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.518+0000 7faf46304640 1 --2- 172.21.15.37:0/2018951046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5579e9c76010 0x5579e9c83f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.518+0000 7faf46b05640 1 --2- 172.21.15.37:0/2018951046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579e9c7bb70 0x5579e9cf47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.518+0000 7faf45b03640 1 -- 172.21.15.37:0/2018951046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579e9c7bb70 msgr2=0x5579e9cf47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.518+0000 7faf45b03640 1 --2- 172.21.15.37:0/2018951046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579e9c7bb70 0x5579e9cf47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.518+0000 7faf45b03640 1 -- 172.21.15.37:0/2018951046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5579e9c76010 msgr2=0x5579e9c83f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.518+0000 7faf45b03640 1 --2- 172.21.15.37:0/2018951046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5579e9c76010 0x5579e9c83f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.518+0000 7faf45b03640 1 -- 172.21.15.37:0/2018951046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579e9c755c0 con 0x5579e9c76a00 2024-06-10T01:01:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.518+0000 7faf45b03640 1 --2- 172.21.15.37:0/2018951046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579e9c76a00 0x5579e9c844d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7faf3c009da0 tx=0x7faf3c009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b06ec1f9aea685 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.518+0000 7faf29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:01:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.518+0000 7faf29ffb640 1 -- 172.21.15.37:0/2018951046 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3c022040 con 0x5579e9c76a00 2024-06-10T01:01:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.518+0000 7faf4b272740 1 -- 172.21.15.37:0/2018951046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf3c005020 con 0x5579e9c76a00 2024-06-10T01:01:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.518+0000 7faf29ffb640 1 -- 172.21.15.37:0/2018951046 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7faf3c004d80 con 0x5579e9c76a00 2024-06-10T01:01:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.519+0000 7faf29ffb640 1 -- 172.21.15.37:0/2018951046 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faf3c039050 con 0x5579e9c76a00 2024-06-10T01:01:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.519+0000 7faf4b272740 10 client.16164 fetch_fsmap learned FSMap version 27 2024-06-10T01:01:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.519+0000 7faf4b272740 1 -- 172.21.15.37:0/2018951046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5579e9cf5bb0 con 0x5579e9c76a00 2024-06-10T01:01:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.519+0000 7faf29ffb640 1 -- 172.21.15.37:0/2018951046 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf3c005020 con 0x5579e9c76a00 2024-06-10T01:01:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.519+0000 7faf29ffb640 1 -- 172.21.15.37:0/2018951046 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7faf3c028450 con 0x5579e9c76a00 2024-06-10T01:01:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.519+0000 7faf4b272740 10 client.16164 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:01:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.519+0000 7faf4b272740 10 client.16164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:01:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.519+0000 7faf4b272740 20 client.16164 populate_metadata read hostname 'smithi037' 2024-06-10T01:01:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.519+0000 7faf4b272740 1 --2- 172.21.15.37:0/2018951046 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5579e9cf6470 0x5579e9cf8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.519+0000 7faf4b272740 4 client.16164 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:01:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.519+0000 7faf4b272740 1 -- 172.21.15.37:0/2018951046 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5579e9be6c20 con 0x5579e9cf6470 2024-06-10T01:01:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.519+0000 7faf46304640 1 --2- 172.21.15.37:0/2018951046 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5579e9cf6470 0x5579e9cf8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.520+0000 7faf29ffb640 10 client.16164 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.520+0000 7faf46304640 1 --2- 172.21.15.37:0/2018951046 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5579e9cf6470 0x5579e9cf8890 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da89cfea313e89f1 server_cookie=db5cf99ffca203b in_seq=0 out_seq=0 2024-06-10T01:01:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.538+0000 7faf29ffb640 1 -- 172.21.15.37:0/2018951046 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5579e9be6c20 con 0x5579e9cf6470 2024-06-10T01:01:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.538+0000 7faf29ffb640 10 client.16164 handle_command_reply: tid=0 2024-06-10T01:01:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.538+0000 7faf4b272740 1 client.16164 shutdown 2024-06-10T01:01:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.539+0000 7faf4b272740 1 -- 172.21.15.37:0/2018951046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579e9c76a00 msgr2=0x5579e9c844d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.539+0000 7faf4b272740 1 --2- 172.21.15.37:0/2018951046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579e9c76a00 0x5579e9c844d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7faf3c009da0 tx=0x7faf3c009be0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.539+0000 7faf4b272740 1 -- 172.21.15.37:0/2018951046 shutdown_connections 2024-06-10T01:01:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.539+0000 7faf4b272740 1 --2- 172.21.15.37:0/2018951046 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5579e9cf6470 0x5579e9cf8890 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.539+0000 7faf4b272740 1 --2- 172.21.15.37:0/2018951046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579e9c7bb70 0x5579e9cf47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.539+0000 7faf4b272740 1 --2- 172.21.15.37:0/2018951046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579e9c76a00 0x5579e9c844d0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.539+0000 7faf29ffb640 0 client.16164 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.539+0000 7faf4b272740 1 --2- 172.21.15.37:0/2018951046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5579e9c76010 0x5579e9c83f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.539+0000 7faf4b272740 1 -- 172.21.15.37:0/2018951046 >> 172.21.15.37:0/2018951046 conn(0x5579e9c71680 msgr2=0x5579e9c74590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.539+0000 7faf4b272740 1 -- 172.21.15.37:0/2018951046 shutdown_connections 2024-06-10T01:01:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.539+0000 7faf4b272740 1 -- 172.21.15.37:0/2018951046 wait complete. 2024-06-10T01:01:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.539+0000 7faf4b272740 20 client.16164 trim_cache size 0 max 16384 2024-06-10T01:01:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:01:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:01:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:01:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.539+0000 7faf48568640 1 -- 172.21.15.37:0/2032330849 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7faf2c0a2da0 msgr2=0x7faf2c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.539+0000 7faf48568640 1 --2- 172.21.15.37:0/2032330849 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7faf2c0a2da0 0x7faf2c0a5260 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7faf340095f0 tx=0x7faf34005f90 comp rx=0 tx=0).stop 2024-06-10T01:01:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.539+0000 7faf48568640 1 -- 172.21.15.37:0/2032330849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf40107760 msgr2=0x7faf401009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.539+0000 7faf48568640 1 --2- 172.21.15.37:0/2032330849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf40107760 0x7faf401009f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faf38031ae0 tx=0x7faf38004ab0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.539+0000 7faf48568640 1 -- 172.21.15.37:0/2032330849 shutdown_connections 2024-06-10T01:01:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.539+0000 7faf48568640 1 --2- 172.21.15.37:0/2032330849 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7faf2c0a2da0 0x7faf2c0a5260 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.539+0000 7faf48568640 1 --2- 172.21.15.37:0/2032330849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf40107760 0x7faf401009f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.539+0000 7faf48568640 1 --2- 172.21.15.37:0/2032330849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf40104e30 0x7faf401004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.539+0000 7faf48568640 1 --2- 172.21.15.37:0/2032330849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf40102500 0x7faf40101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.539+0000 7faf48568640 1 -- 172.21.15.37:0/2032330849 >> 172.21.15.37:0/2032330849 conn(0x7faf400fbe70 msgr2=0x7faf40102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.539+0000 7faf48568640 1 -- 172.21.15.37:0/2032330849 shutdown_connections 2024-06-10T01:01:39.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:39.540+0000 7faf48568640 1 -- 172.21.15.37:0/2032330849 wait complete. 2024-06-10T01:01:39.607 INFO:tasks.workunit.client.0.smithi037.stdout:ok 2024-06-10T01:01:39.612 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-10T01:01:39.612 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-10T01:01:39.653 INFO:tasks.workunit:Running workunit fs/misc/dirfrag.sh... 2024-06-10T01:01:39.653 DEBUG:teuthology.orchestra.run.smithi037:workunit test fs/misc/dirfrag.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b7a5d375cf4e4afab96b9d4d229b699ae8304bb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dirfrag.sh 2024-06-10T01:01:39.779 INFO:tasks.workunit.client.0.smithi037.stdout:creating folder hierarchy 2024-06-10T01:01:40.180 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:40 smithi037 ceph-mon[32192]: pgmap v579: 129 pgs: 129 active+clean; 2.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-06-10T01:01:40.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:40 smithi167 ceph-mon[36917]: pgmap v579: 129 pgs: 129 active+clean; 2.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-06-10T01:01:40.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:40 smithi094 ceph-mon[36708]: pgmap v579: 129 pgs: 129 active+clean; 2.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-06-10T01:01:42.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:42 smithi037 ceph-mon[32192]: pgmap v580: 129 pgs: 129 active+clean; 2.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-06-10T01:01:42.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:42 smithi167 ceph-mon[36917]: pgmap v580: 129 pgs: 129 active+clean; 2.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-06-10T01:01:42.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:42 smithi094 ceph-mon[36708]: pgmap v580: 129 pgs: 129 active+clean; 2.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-06-10T01:01:43.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:43 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:01:43.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:43 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:01:43.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:43 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:01:43.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:43 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:01:43.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:43 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:01:43.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:43 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:01:43.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:43 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:01:43.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:43 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:01:43.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:43 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:01:43.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:43 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:01:43.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:43 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:01:43.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:43 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:01:43.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:43 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:01:43.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:43 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:01:43.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:43 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:01:43.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:43 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:01:43.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:43 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:01:43.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:43 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:01:43.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.546+0000 7fc832086700 1 -- 172.21.15.37:0/946509745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc82c06b3c0 msgr2=0x7fc82c10f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:43.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.546+0000 7fc832086700 1 --2- 172.21.15.37:0/946509745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc82c06b3c0 0x7fc82c10f310 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc82400b0a0 tx=0x7fc82400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.546+0000 7fc832086700 1 -- 172.21.15.37:0/946509745 shutdown_connections 2024-06-10T01:01:43.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.546+0000 7fc832086700 1 --2- 172.21.15.37:0/946509745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc82c06b3c0 0x7fc82c10f310 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.546+0000 7fc832086700 1 --2- 172.21.15.37:0/946509745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc82c1084f0 0x7fc82c1088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.546+0000 7fc832086700 1 --2- 172.21.15.37:0/946509745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc82c06aa00 0x7fc82c06ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.546+0000 7fc832086700 1 -- 172.21.15.37:0/946509745 >> 172.21.15.37:0/946509745 conn(0x7fc82c078300 msgr2=0x7fc82c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.546+0000 7fc832086700 1 -- 172.21.15.37:0/946509745 shutdown_connections 2024-06-10T01:01:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.546+0000 7fc832086700 1 -- 172.21.15.37:0/946509745 wait complete. 2024-06-10T01:01:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.546+0000 7fc832086700 1 Processor -- start 2024-06-10T01:01:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.546+0000 7fc832086700 1 -- start start 2024-06-10T01:01:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.547+0000 7fc832086700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc82c06aa00 0x7fc82c19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.547+0000 7fc832086700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc82c06b3c0 0x7fc82c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.547+0000 7fc832086700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc82c1084f0 0x7fc82c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.547+0000 7fc832086700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c112750 con 0x7fc82c06aa00 2024-06-10T01:01:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.547+0000 7fc832086700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c1125d0 con 0x7fc82c1084f0 2024-06-10T01:01:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.547+0000 7fc832086700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c1128d0 con 0x7fc82c06b3c0 2024-06-10T01:01:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.547+0000 7fc82b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc82c06aa00 0x7fc82c19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.547+0000 7fc823fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc82c06b3c0 0x7fc82c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:43.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.547+0000 7fc82b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc82c06aa00 0x7fc82c19e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52794/0 (socket says 172.21.15.37:52794) 2024-06-10T01:01:43.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.547+0000 7fc82b7fe700 1 -- 172.21.15.37:0/3360055522 learned_addr learned my addr 172.21.15.37:0/3360055522 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:43.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.547+0000 7fc82bfff700 1 --2- 172.21.15.37:0/3360055522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc82c1084f0 0x7fc82c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:43.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.547+0000 7fc823fff700 1 -- 172.21.15.37:0/3360055522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc82c1084f0 msgr2=0x7fc82c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:43.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.547+0000 7fc823fff700 1 --2- 172.21.15.37:0/3360055522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc82c1084f0 0x7fc82c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.547+0000 7fc823fff700 1 -- 172.21.15.37:0/3360055522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc82c06aa00 msgr2=0x7fc82c19e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:43.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.547+0000 7fc823fff700 1 --2- 172.21.15.37:0/3360055522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc82c06aa00 0x7fc82c19e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.547+0000 7fc823fff700 1 -- 172.21.15.37:0/3360055522 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82c199130 con 0x7fc82c06b3c0 2024-06-10T01:01:43.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.547+0000 7fc82b7fe700 1 --2- 172.21.15.37:0/3360055522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc82c06aa00 0x7fc82c19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:01:43.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.547+0000 7fc823fff700 1 --2- 172.21.15.37:0/3360055522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc82c06b3c0 0x7fc82c19ed50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc81400c970 tx=0x7fc81400cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:43.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.548+0000 7fc8297fa700 1 -- 172.21.15.37:0/3360055522 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc814004a30 con 0x7fc82c06b3c0 2024-06-10T01:01:43.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.548+0000 7fc832086700 1 -- 172.21.15.37:0/3360055522 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc82c199420 con 0x7fc82c06b3c0 2024-06-10T01:01:43.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.548+0000 7fc8297fa700 1 -- 172.21.15.37:0/3360055522 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc814005500 con 0x7fc82c06b3c0 2024-06-10T01:01:43.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.548+0000 7fc8297fa700 1 -- 172.21.15.37:0/3360055522 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81400ec30 con 0x7fc82c06b3c0 2024-06-10T01:01:43.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.549+0000 7fc832086700 1 -- 172.21.15.37:0/3360055522 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc82c1a56e0 con 0x7fc82c06b3c0 2024-06-10T01:01:43.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.550+0000 7fc8297fa700 1 -- 172.21.15.37:0/3360055522 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc814004bd0 con 0x7fc82c06b3c0 2024-06-10T01:01:43.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.551+0000 7fc8297fa700 1 --2- 172.21.15.37:0/3360055522 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc8180a2c20 0x7fc8180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:43.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.551+0000 7fc82b7fe700 1 --2- 172.21.15.37:0/3360055522 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc8180a2c20 0x7fc8180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:43.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.551+0000 7fc8297fa700 1 -- 172.21.15.37:0/3360055522 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fc8140cebd0 con 0x7fc82c06b3c0 2024-06-10T01:01:43.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.551+0000 7fc82b7fe700 1 --2- 172.21.15.37:0/3360055522 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc8180a2c20 0x7fc8180a50e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc82c199b10 tx=0x7fc81c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:43.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.566+0000 7fc83c909b80 1 Processor -- start 2024-06-10T01:01:43.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.566+0000 7fc83c909b80 1 -- start start 2024-06-10T01:01:43.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.566+0000 7fc83c909b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556f0e23f60 0x5556f0e24340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:43.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.566+0000 7fc83c909b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556f0e24880 0x5556f0e1e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:43.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.566+0000 7fc83c909b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556f0e1e7a0 0x5556f0e1ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:43.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.566+0000 7fc83c909b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5556f0d10f60 con 0x5556f0e23f60 2024-06-10T01:01:43.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.566+0000 7fc83c909b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5556f0d6f850 con 0x5556f0e24880 2024-06-10T01:01:43.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.566+0000 7fc83c909b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5556f0cd7460 con 0x5556f0e1e7a0 2024-06-10T01:01:43.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.567+0000 7fc82b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556f0e23f60 0x5556f0e24340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:43.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.567+0000 7fc82b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556f0e23f60 0x5556f0e24340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52802/0 (socket says 172.21.15.37:52802) 2024-06-10T01:01:43.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.567+0000 7fc82b7fe700 1 -- 172.21.15.37:0/3255379863 learned_addr learned my addr 172.21.15.37:0/3255379863 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:43.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.567+0000 7fc823fff700 1 --2- 172.21.15.37:0/3255379863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556f0e24880 0x5556f0e1e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:43.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.567+0000 7fc82bfff700 1 --2- 172.21.15.37:0/3255379863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556f0e1e7a0 0x5556f0e1ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:43.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.567+0000 7fc82b7fe700 1 -- 172.21.15.37:0/3255379863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556f0e1e7a0 msgr2=0x5556f0e1ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:43.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.567+0000 7fc82b7fe700 1 --2- 172.21.15.37:0/3255379863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556f0e1e7a0 0x5556f0e1ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.567+0000 7fc82b7fe700 1 -- 172.21.15.37:0/3255379863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556f0e24880 msgr2=0x5556f0e1e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:43.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.567+0000 7fc82b7fe700 1 --2- 172.21.15.37:0/3255379863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556f0e24880 0x5556f0e1e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.567+0000 7fc82b7fe700 1 -- 172.21.15.37:0/3255379863 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556f0cbe8d0 con 0x5556f0e23f60 2024-06-10T01:01:43.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.567+0000 7fc82b7fe700 1 --2- 172.21.15.37:0/3255379863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556f0e23f60 0x5556f0e24340 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fc81c0043b0 tx=0x7fc81c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92efd059311bf3b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:43.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.567+0000 7fc821ffb700 1 -- 172.21.15.37:0/3255379863 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c011e60 con 0x5556f0e23f60 2024-06-10T01:01:43.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.567+0000 7fc821ffb700 1 -- 172.21.15.37:0/3255379863 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc81c003ce0 con 0x5556f0e23f60 2024-06-10T01:01:43.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.567+0000 7fc821ffb700 1 -- 172.21.15.37:0/3255379863 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c026780 con 0x5556f0e23f60 2024-06-10T01:01:43.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.568+0000 7fc83c909b80 1 -- 172.21.15.37:0/3255379863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556f0e23f60 msgr2=0x5556f0e24340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:43.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.568+0000 7fc83c909b80 1 --2- 172.21.15.37:0/3255379863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556f0e23f60 0x5556f0e24340 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fc81c0043b0 tx=0x7fc81c014000 comp rx=0 tx=0).stop 2024-06-10T01:01:43.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.568+0000 7fc83c909b80 1 -- 172.21.15.37:0/3255379863 shutdown_connections 2024-06-10T01:01:43.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.568+0000 7fc83c909b80 1 --2- 172.21.15.37:0/3255379863 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556f0e1e7a0 0x5556f0e1ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.568+0000 7fc83c909b80 1 --2- 172.21.15.37:0/3255379863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556f0e23f60 0x5556f0e24340 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.568+0000 7fc83c909b80 1 --2- 172.21.15.37:0/3255379863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556f0e24880 0x5556f0e1e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.568+0000 7fc83c909b80 1 -- 172.21.15.37:0/3255379863 >> 172.21.15.37:0/3255379863 conn(0x5556f0d4b080 msgr2=0x5556f0e19220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:43.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.568+0000 7fc83c909b80 1 -- 172.21.15.37:0/3255379863 shutdown_connections 2024-06-10T01:01:43.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.568+0000 7fc83c909b80 1 -- 172.21.15.37:0/3255379863 wait complete. 2024-06-10T01:01:43.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.569+0000 7fc83c909b80 1 -- start start 2024-06-10T01:01:43.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.569+0000 7fc83c909b80 1 Processor -- start 2024-06-10T01:01:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.569+0000 7fc83c909b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556f0e1e7a0 0x5556f0e2a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.569+0000 7fc83c909b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556f0e23f60 0x5556f0e27980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.569+0000 7fc83c909b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556f0e24880 0x5556f0e27ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.569+0000 7fc83c909b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5556f0d0ffc0 con 0x5556f0e23f60 2024-06-10T01:01:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.569+0000 7fc83c909b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5556f0d6b810 con 0x5556f0e24880 2024-06-10T01:01:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.569+0000 7fc83c909b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5556f0e2b260 con 0x5556f0e1e7a0 2024-06-10T01:01:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.569+0000 7fc83c909b80 10 client.0 _ll_register_callbacks cb 0x5556f0d055b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:01:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.569+0000 7fc82b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556f0e1e7a0 0x5556f0e2a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:43.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.569+0000 7fc82b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556f0e1e7a0 0x5556f0e2a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:34866/0 (socket says 172.21.15.37:34866) 2024-06-10T01:01:43.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.569+0000 7fc823fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556f0e23f60 0x5556f0e27980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:43.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.569+0000 7fc82b7fe700 1 -- 172.21.15.37:0/1536967360 learned_addr learned my addr 172.21.15.37:0/1536967360 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:43.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.570+0000 7fc82bfff700 1 --2- 172.21.15.37:0/1536967360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556f0e24880 0x5556f0e27ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:43.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.570+0000 7fc823fff700 1 -- 172.21.15.37:0/1536967360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556f0e1e7a0 msgr2=0x5556f0e2a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:43.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.570+0000 7fc823fff700 1 --2- 172.21.15.37:0/1536967360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556f0e1e7a0 0x5556f0e2a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.570+0000 7fc823fff700 1 -- 172.21.15.37:0/1536967360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556f0e24880 msgr2=0x5556f0e27ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:43.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.570+0000 7fc823fff700 1 --2- 172.21.15.37:0/1536967360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556f0e24880 0x5556f0e27ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.570+0000 7fc823fff700 1 -- 172.21.15.37:0/1536967360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556f0d35750 con 0x5556f0e23f60 2024-06-10T01:01:43.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.570+0000 7fc82b7fe700 1 --2- 172.21.15.37:0/1536967360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556f0e1e7a0 0x5556f0e2a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:01:43.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.570+0000 7fc823fff700 1 --2- 172.21.15.37:0/1536967360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556f0e23f60 0x5556f0e27980 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fc81400cfd0 tx=0x7fc814000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7cac52863ac88b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:43.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.570+0000 7fc80affd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:01:43.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.570+0000 7fc80affd700 1 -- 172.21.15.37:0/1536967360 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8140b3070 con 0x5556f0e23f60 2024-06-10T01:01:43.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.570+0000 7fc80affd700 1 -- 172.21.15.37:0/1536967360 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fc81400edd0 con 0x5556f0e23f60 2024-06-10T01:01:43.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.570+0000 7fc83c909b80 1 -- 172.21.15.37:0/1536967360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc81c003ce0 con 0x5556f0e23f60 2024-06-10T01:01:43.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.570+0000 7fc80affd700 1 -- 172.21.15.37:0/1536967360 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc81401d070 con 0x5556f0e23f60 2024-06-10T01:01:43.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.570+0000 7fc83c909b80 10 client.16185 fetch_fsmap learned FSMap version 27 2024-06-10T01:01:43.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.570+0000 7fc83c909b80 1 -- 172.21.15.37:0/1536967360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5556f0e39f00 con 0x5556f0e23f60 2024-06-10T01:01:43.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.571+0000 7fc80affd700 1 -- 172.21.15.37:0/1536967360 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc814079e50 con 0x5556f0e23f60 2024-06-10T01:01:43.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.571+0000 7fc80affd700 1 -- 172.21.15.37:0/1536967360 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc814020060 con 0x5556f0e23f60 2024-06-10T01:01:43.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.571+0000 7fc83c909b80 10 client.16185 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:01:43.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.571+0000 7fc83c909b80 10 client.16185 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-10T01:01:43.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.571+0000 7fc83c909b80 20 client.16185 populate_metadata read hostname 'smithi037' 2024-06-10T01:01:43.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.571+0000 7fc83c909b80 1 --2- 172.21.15.37:0/1536967360 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5556f0e3c490 0x5556f0e3e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:43.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.571+0000 7fc83c909b80 4 client.16185 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:01:43.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.571+0000 7fc83c909b80 1 -- 172.21.15.37:0/1536967360 --> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556f0d7cd50 con 0x5556f0e3c490 2024-06-10T01:01:43.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.571+0000 7fc82b7fe700 1 --2- 172.21.15.37:0/1536967360 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5556f0e3c490 0x5556f0e3e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:43.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.572+0000 7fc82b7fe700 1 --2- 172.21.15.37:0/1536967360 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5556f0e3c490 0x5556f0e3e880 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2bbad374be366d54 server_cookie=aeda43173b6eea31 in_seq=0 out_seq=0 2024-06-10T01:01:43.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.572+0000 7fc80affd700 10 client.16185 ms_handle_connect on v2:172.21.15.167:6836/2720148065 2024-06-10T01:01:43.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.572+0000 7fc80affd700 1 -- 172.21.15.37:0/1536967360 <== mds.2 v2:172.21.15.167:6836/2720148065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5556f0d7cd50 con 0x5556f0e3c490 2024-06-10T01:01:43.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.572+0000 7fc80affd700 10 client.16185 handle_command_reply: tid=0 2024-06-10T01:01:43.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.572+0000 7fc83c909b80 1 client.16185 shutdown 2024-06-10T01:01:43.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.573+0000 7fc83c909b80 1 -- 172.21.15.37:0/1536967360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556f0e23f60 msgr2=0x5556f0e27980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:43.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.573+0000 7fc83c909b80 1 --2- 172.21.15.37:0/1536967360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556f0e23f60 0x5556f0e27980 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fc81400cfd0 tx=0x7fc814000bc0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.573+0000 7fc83c909b80 1 -- 172.21.15.37:0/1536967360 shutdown_connections 2024-06-10T01:01:43.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.573+0000 7fc83c909b80 1 --2- 172.21.15.37:0/1536967360 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5556f0e3c490 0x5556f0e3e880 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.573+0000 7fc83c909b80 1 --2- 172.21.15.37:0/1536967360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556f0e24880 0x5556f0e27ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.573+0000 7fc83c909b80 1 --2- 172.21.15.37:0/1536967360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556f0e1e7a0 0x5556f0e2a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.573+0000 7fc83c909b80 1 --2- 172.21.15.37:0/1536967360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556f0e23f60 0x5556f0e27980 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.573+0000 7fc80affd700 0 client.16185 ms_handle_reset on v2:172.21.15.167:6836/2720148065 2024-06-10T01:01:43.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.573+0000 7fc83c909b80 1 -- 172.21.15.37:0/1536967360 >> 172.21.15.37:0/1536967360 conn(0x5556f0d4b080 msgr2=0x5556f0e18700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:43.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.573+0000 7fc83c909b80 1 -- 172.21.15.37:0/1536967360 shutdown_connections 2024-06-10T01:01:43.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.573+0000 7fc83c909b80 1 -- 172.21.15.37:0/1536967360 wait complete. 2024-06-10T01:01:43.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.573+0000 7fc83c909b80 20 client.16185 trim_cache size 0 max 16384 2024-06-10T01:01:43.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.588+0000 7fc83c909b80 1 Processor -- start 2024-06-10T01:01:43.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.588+0000 7fc83c909b80 1 -- start start 2024-06-10T01:01:43.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.588+0000 7fc83c909b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556f0e1e7a0 0x5556f0e1e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:43.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.588+0000 7fc83c909b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556f0e23f60 0x5556f0e25370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:43.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.588+0000 7fc83c909b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556f0e24880 0x5556f0e2a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:43.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.588+0000 7fc83c909b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c1aec10 con 0x5556f0e1e7a0 2024-06-10T01:01:43.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.588+0000 7fc83c909b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c1ac4a0 con 0x5556f0e24880 2024-06-10T01:01:43.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.588+0000 7fc83c909b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c1aa2f0 con 0x5556f0e23f60 2024-06-10T01:01:43.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.589+0000 7fc823fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556f0e23f60 0x5556f0e25370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:43.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.589+0000 7fc82b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556f0e1e7a0 0x5556f0e1e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:43.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.589+0000 7fc823fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556f0e23f60 0x5556f0e25370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:34876/0 (socket says 172.21.15.37:34876) 2024-06-10T01:01:43.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.589+0000 7fc823fff700 1 -- 172.21.15.37:0/3602565126 learned_addr learned my addr 172.21.15.37:0/3602565126 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:43.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.589+0000 7fc82bfff700 1 --2- 172.21.15.37:0/3602565126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556f0e24880 0x5556f0e2a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:43.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.589+0000 7fc823fff700 1 -- 172.21.15.37:0/3602565126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556f0e24880 msgr2=0x5556f0e2a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:43.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.589+0000 7fc823fff700 1 --2- 172.21.15.37:0/3602565126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556f0e24880 0x5556f0e2a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.589+0000 7fc823fff700 1 -- 172.21.15.37:0/3602565126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556f0e1e7a0 msgr2=0x5556f0e1e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:43.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.589+0000 7fc823fff700 1 --2- 172.21.15.37:0/3602565126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556f0e1e7a0 0x5556f0e1e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.589+0000 7fc823fff700 1 -- 172.21.15.37:0/3602565126 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556f0e281f0 con 0x5556f0e23f60 2024-06-10T01:01:43.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.589+0000 7fc82b7fe700 1 --2- 172.21.15.37:0/3602565126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556f0e1e7a0 0x5556f0e1e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:01:43.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.589+0000 7fc823fff700 1 --2- 172.21.15.37:0/3602565126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556f0e23f60 0x5556f0e25370 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc8140a3bc0 tx=0x7fc81409b160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a9bf80ea106d54a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:43.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.589+0000 7fc8217fa700 1 -- 172.21.15.37:0/3602565126 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8140b3040 con 0x5556f0e23f60 2024-06-10T01:01:43.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.589+0000 7fc8217fa700 1 -- 172.21.15.37:0/3602565126 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc814079e50 con 0x5556f0e23f60 2024-06-10T01:01:43.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.589+0000 7fc8217fa700 1 -- 172.21.15.37:0/3602565126 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc814095ba0 con 0x5556f0e23f60 2024-06-10T01:01:43.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:43 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:01:43.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:43 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:01:43.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:43 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:01:43.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:43 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:01:43.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:43 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:01:43.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:43 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:01:43.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:43 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:01:43.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:43 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:01:43.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:43 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:01:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.590+0000 7fc83c909b80 1 -- 172.21.15.37:0/3602565126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556f0e23f60 msgr2=0x5556f0e25370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.590+0000 7fc83c909b80 1 --2- 172.21.15.37:0/3602565126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556f0e23f60 0x5556f0e25370 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc8140a3bc0 tx=0x7fc81409b160 comp rx=0 tx=0).stop 2024-06-10T01:01:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.590+0000 7fc83c909b80 1 -- 172.21.15.37:0/3602565126 shutdown_connections 2024-06-10T01:01:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.590+0000 7fc83c909b80 1 --2- 172.21.15.37:0/3602565126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556f0e24880 0x5556f0e2a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.590+0000 7fc83c909b80 1 --2- 172.21.15.37:0/3602565126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556f0e1e7a0 0x5556f0e1e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.590+0000 7fc83c909b80 1 --2- 172.21.15.37:0/3602565126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556f0e23f60 0x5556f0e25370 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.590+0000 7fc83c909b80 1 -- 172.21.15.37:0/3602565126 >> 172.21.15.37:0/3602565126 conn(0x5556f0d4b080 msgr2=0x5556f0e24c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.590+0000 7fc83c909b80 1 -- 172.21.15.37:0/3602565126 shutdown_connections 2024-06-10T01:01:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.590+0000 7fc83c909b80 1 -- 172.21.15.37:0/3602565126 wait complete. 2024-06-10T01:01:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.590+0000 7fc83c909b80 1 -- start start 2024-06-10T01:01:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.591+0000 7fc83c909b80 1 Processor -- start 2024-06-10T01:01:43.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.591+0000 7fc83c909b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556f0e1e7a0 0x5556f0e349b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:43.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.591+0000 7fc83c909b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556f0e23f60 0x5556f0e34ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:43.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.591+0000 7fc83c909b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556f0e24880 0x5556f0e43130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:43.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.591+0000 7fc83c909b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c1aa960 con 0x5556f0e1e7a0 2024-06-10T01:01:43.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.591+0000 7fc83c909b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c1abdd0 con 0x5556f0e24880 2024-06-10T01:01:43.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.591+0000 7fc83c909b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c1ab030 con 0x5556f0e23f60 2024-06-10T01:01:43.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.591+0000 7fc83c909b80 10 client.0 _ll_register_callbacks cb 0x5556f0e22570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:01:43.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.591+0000 7fc82b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556f0e1e7a0 0x5556f0e349b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:43.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.591+0000 7fc82b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556f0e1e7a0 0x5556f0e349b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52818/0 (socket says 172.21.15.37:52818) 2024-06-10T01:01:43.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.591+0000 7fc82b7fe700 1 -- 172.21.15.37:0/2530039855 learned_addr learned my addr 172.21.15.37:0/2530039855 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:43.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.591+0000 7fc823fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556f0e23f60 0x5556f0e34ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:43.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.591+0000 7fc82bfff700 1 --2- 172.21.15.37:0/2530039855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556f0e24880 0x5556f0e43130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:43.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.591+0000 7fc82b7fe700 1 -- 172.21.15.37:0/2530039855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556f0e23f60 msgr2=0x5556f0e34ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:43.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.591+0000 7fc82b7fe700 1 --2- 172.21.15.37:0/2530039855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556f0e23f60 0x5556f0e34ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.591+0000 7fc82b7fe700 1 -- 172.21.15.37:0/2530039855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556f0e24880 msgr2=0x5556f0e43130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.591+0000 7fc82b7fe700 1 --2- 172.21.15.37:0/2530039855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556f0e24880 0x5556f0e43130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.591+0000 7fc82b7fe700 1 -- 172.21.15.37:0/2530039855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556f0e43670 con 0x5556f0e1e7a0 2024-06-10T01:01:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.591+0000 7fc82bfff700 1 --2- 172.21.15.37:0/2530039855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556f0e24880 0x5556f0e43130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:01:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.592+0000 7fc82b7fe700 1 --2- 172.21.15.37:0/2530039855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556f0e1e7a0 0x5556f0e349b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc81c009810 tx=0x7fc81c005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed98ef23f0ad17d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.592+0000 7fc80affd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:01:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.592+0000 7fc80affd700 1 -- 172.21.15.37:0/2530039855 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c011e60 con 0x5556f0e1e7a0 2024-06-10T01:01:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.592+0000 7fc80affd700 1 -- 172.21.15.37:0/2530039855 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fc81c01e730 con 0x5556f0e1e7a0 2024-06-10T01:01:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.592+0000 7fc83c909b80 1 -- 172.21.15.37:0/2530039855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc814079e50 con 0x5556f0e1e7a0 2024-06-10T01:01:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.592+0000 7fc80affd700 1 -- 172.21.15.37:0/2530039855 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc81c042050 con 0x5556f0e1e7a0 2024-06-10T01:01:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.592+0000 7fc83c909b80 10 client.16191 fetch_fsmap learned FSMap version 27 2024-06-10T01:01:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.592+0000 7fc83c909b80 1 -- 172.21.15.37:0/2530039855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5556f0ea67c0 con 0x5556f0e1e7a0 2024-06-10T01:01:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.592+0000 7fc80affd700 1 -- 172.21.15.37:0/2530039855 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc814079e50 con 0x5556f0e1e7a0 2024-06-10T01:01:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.592+0000 7fc80affd700 1 -- 172.21.15.37:0/2530039855 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc81c004830 con 0x5556f0e1e7a0 2024-06-10T01:01:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.592+0000 7fc83c909b80 10 client.16191 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:01:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.592+0000 7fc83c909b80 10 client.16191 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-10T01:01:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.592+0000 7fc83c909b80 20 client.16191 populate_metadata read hostname 'smithi037' 2024-06-10T01:01:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.592+0000 7fc83c909b80 1 --2- 172.21.15.37:0/2530039855 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5556f0ea6bf0 0x5556f0ea9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.593+0000 7fc83c909b80 4 client.16191 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:01:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.593+0000 7fc83c909b80 1 -- 172.21.15.37:0/2530039855 --> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5556f0d772d0 con 0x5556f0ea6bf0 2024-06-10T01:01:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.593+0000 7fc823fff700 1 --2- 172.21.15.37:0/2530039855 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5556f0ea6bf0 0x5556f0ea9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.593+0000 7fc80affd700 10 client.16191 ms_handle_connect on v2:172.21.15.167:6836/2720148065 2024-06-10T01:01:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.593+0000 7fc823fff700 1 --2- 172.21.15.37:0/2530039855 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5556f0ea6bf0 0x5556f0ea9010 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ed61b10f9beedca server_cookie=f884a05a34687ab7 in_seq=0 out_seq=0 2024-06-10T01:01:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.593+0000 7fc80affd700 1 -- 172.21.15.37:0/2530039855 <== mds.2 v2:172.21.15.167:6836/2720148065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5556f0d772d0 con 0x5556f0ea6bf0 2024-06-10T01:01:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.593+0000 7fc80affd700 10 client.16191 handle_command_reply: tid=0 2024-06-10T01:01:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.593+0000 7fc83c909b80 1 client.16191 shutdown 2024-06-10T01:01:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.593+0000 7fc83c909b80 1 -- 172.21.15.37:0/2530039855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556f0e1e7a0 msgr2=0x5556f0e349b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.593+0000 7fc83c909b80 1 --2- 172.21.15.37:0/2530039855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556f0e1e7a0 0x5556f0e349b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc81c009810 tx=0x7fc81c005b90 comp rx=0 tx=0).stop 2024-06-10T01:01:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.593+0000 7fc83c909b80 1 -- 172.21.15.37:0/2530039855 shutdown_connections 2024-06-10T01:01:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.593+0000 7fc83c909b80 1 --2- 172.21.15.37:0/2530039855 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5556f0ea6bf0 0x5556f0ea9010 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.594+0000 7fc83c909b80 1 --2- 172.21.15.37:0/2530039855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556f0e24880 0x5556f0e43130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.594+0000 7fc83c909b80 1 --2- 172.21.15.37:0/2530039855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556f0e1e7a0 0x5556f0e349b0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.594+0000 7fc83c909b80 1 --2- 172.21.15.37:0/2530039855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5556f0e23f60 0x5556f0e34ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.594+0000 7fc83c909b80 1 -- 172.21.15.37:0/2530039855 >> 172.21.15.37:0/2530039855 conn(0x5556f0d4b080 msgr2=0x5556f0e254f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.594+0000 7fc80affd700 0 client.16191 ms_handle_reset on v2:172.21.15.167:6836/2720148065 2024-06-10T01:01:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.594+0000 7fc83c909b80 1 -- 172.21.15.37:0/2530039855 shutdown_connections 2024-06-10T01:01:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.594+0000 7fc83c909b80 1 -- 172.21.15.37:0/2530039855 wait complete. 2024-06-10T01:01:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.594+0000 7fc83c909b80 20 client.16191 trim_cache size 0 max 16384 2024-06-10T01:01:43.599 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T01:01:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.594+0000 7fc832086700 1 -- 172.21.15.37:0/3360055522 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc8180a2c20 msgr2=0x7fc8180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.594+0000 7fc832086700 1 --2- 172.21.15.37:0/3360055522 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc8180a2c20 0x7fc8180a50e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc82c199b10 tx=0x7fc81c009260 comp rx=0 tx=0).stop 2024-06-10T01:01:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.594+0000 7fc832086700 1 -- 172.21.15.37:0/3360055522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc82c06b3c0 msgr2=0x7fc82c19ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.594+0000 7fc832086700 1 --2- 172.21.15.37:0/3360055522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc82c06b3c0 0x7fc82c19ed50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc81400c970 tx=0x7fc81400cc80 comp rx=0 tx=0).stop 2024-06-10T01:01:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.595+0000 7fc832086700 1 -- 172.21.15.37:0/3360055522 shutdown_connections 2024-06-10T01:01:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.595+0000 7fc832086700 1 --2- 172.21.15.37:0/3360055522 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc8180a2c20 0x7fc8180a50e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.595+0000 7fc832086700 1 --2- 172.21.15.37:0/3360055522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc82c1084f0 0x7fc82c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.595+0000 7fc832086700 1 --2- 172.21.15.37:0/3360055522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc82c06aa00 0x7fc82c19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.595+0000 7fc832086700 1 --2- 172.21.15.37:0/3360055522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc82c06b3c0 0x7fc82c19ed50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.595+0000 7fc832086700 1 -- 172.21.15.37:0/3360055522 >> 172.21.15.37:0/3360055522 conn(0x7fc82c078300 msgr2=0x7fc82c0fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.595+0000 7fc832086700 1 -- 172.21.15.37:0/3360055522 shutdown_connections 2024-06-10T01:01:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:43.595+0000 7fc832086700 1 -- 172.21.15.37:0/3360055522 wait complete. 2024-06-10T01:01:44.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:44 smithi094 ceph-mon[36708]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-06-10T01:01:44.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:44 smithi094 ceph-mon[36708]: Unable to set osd_memory_target on smithi094 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-06-10T01:01:44.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:44 smithi094 ceph-mon[36708]: pgmap v581: 129 pgs: 129 active+clean; 4.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 241 KiB/s wr, 18 op/s 2024-06-10T01:01:44.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:44 smithi094 ceph-mon[36708]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-06-10T01:01:44.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:44 smithi094 ceph-mon[36708]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-06-10T01:01:44.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:44 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:44.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:44 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:01:44.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:44 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:01:44.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:44 smithi037 ceph-mon[32192]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-06-10T01:01:44.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:44 smithi037 ceph-mon[32192]: Unable to set osd_memory_target on smithi094 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-06-10T01:01:44.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:44 smithi037 ceph-mon[32192]: pgmap v581: 129 pgs: 129 active+clean; 4.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 241 KiB/s wr, 18 op/s 2024-06-10T01:01:44.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:44 smithi037 ceph-mon[32192]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-06-10T01:01:44.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:44 smithi037 ceph-mon[32192]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-06-10T01:01:44.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:44 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:44.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:44 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:01:44.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:44 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:01:44.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:44 smithi167 ceph-mon[36917]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-06-10T01:01:44.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:44 smithi167 ceph-mon[36917]: Unable to set osd_memory_target on smithi094 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-06-10T01:01:44.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:44 smithi167 ceph-mon[36917]: pgmap v581: 129 pgs: 129 active+clean; 4.6 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 241 KiB/s wr, 18 op/s 2024-06-10T01:01:44.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:44 smithi167 ceph-mon[36917]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-06-10T01:01:44.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:44 smithi167 ceph-mon[36917]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-06-10T01:01:44.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:44 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:01:44.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:44 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:01:44.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:44 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:01:44.599 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:01:44.881 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:01:46.362 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:46 smithi037 ceph-mon[32192]: pgmap v582: 129 pgs: 129 active+clean; 4.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 273 KiB/s wr, 23 op/s 2024-06-10T01:01:46.363 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:46 smithi094 ceph-mon[36708]: pgmap v582: 129 pgs: 129 active+clean; 4.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 273 KiB/s wr, 23 op/s 2024-06-10T01:01:46.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:46 smithi167 ceph-mon[36917]: pgmap v582: 129 pgs: 129 active+clean; 4.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 273 KiB/s wr, 23 op/s 2024-06-10T01:01:46.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.612+0000 7f3d2d0bf700 1 -- 172.21.15.37:0/1248789034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d28108500 msgr2=0x7f3d281088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:46.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.612+0000 7f3d2d0bf700 1 --2- 172.21.15.37:0/1248789034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d28108500 0x7f3d281088e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009990 tx=0x7f3d18009ca0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.613+0000 7f3d2d0bf700 1 -- 172.21.15.37:0/1248789034 shutdown_connections 2024-06-10T01:01:46.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.613+0000 7f3d2d0bf700 1 --2- 172.21.15.37:0/1248789034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d2806ab30 0x7f3d2810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.613+0000 7f3d2d0bf700 1 --2- 172.21.15.37:0/1248789034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d28108500 0x7f3d281088e0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.613+0000 7f3d2d0bf700 1 --2- 172.21.15.37:0/1248789034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d2806a170 0x7f3d2806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.613+0000 7f3d2d0bf700 1 -- 172.21.15.37:0/1248789034 >> 172.21.15.37:0/1248789034 conn(0x7f3d28078340 msgr2=0x7f3d28078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:46.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.613+0000 7f3d2d0bf700 1 -- 172.21.15.37:0/1248789034 shutdown_connections 2024-06-10T01:01:46.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.613+0000 7f3d2d0bf700 1 -- 172.21.15.37:0/1248789034 wait complete. 2024-06-10T01:01:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.613+0000 7f3d2d0bf700 1 Processor -- start 2024-06-10T01:01:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.614+0000 7f3d2d0bf700 1 -- start start 2024-06-10T01:01:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.614+0000 7f3d2d0bf700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d2806a170 0x7f3d28101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.614+0000 7f3d2d0bf700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d2806ab30 0x7f3d281019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.614+0000 7f3d2d0bf700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d28108500 0x7f3d28101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.614+0000 7f3d2d0bf700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3d28112760 con 0x7f3d2806ab30 2024-06-10T01:01:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.614+0000 7f3d2d0bf700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3d281125e0 con 0x7f3d28108500 2024-06-10T01:01:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.614+0000 7f3d2d0bf700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3d281128e0 con 0x7f3d2806a170 2024-06-10T01:01:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.614+0000 7f3d2659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d2806ab30 0x7f3d281019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.614+0000 7f3d2659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d2806ab30 0x7f3d281019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52840/0 (socket says 172.21.15.37:52840) 2024-06-10T01:01:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.614+0000 7f3d2659c700 1 -- 172.21.15.37:0/3064714915 learned_addr learned my addr 172.21.15.37:0/3064714915 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.614+0000 7f3d26d9d700 1 --2- 172.21.15.37:0/3064714915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d2806a170 0x7f3d28101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.614+0000 7f3d2759e700 1 --2- 172.21.15.37:0/3064714915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d28108500 0x7f3d28101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.614+0000 7f3d2659c700 1 -- 172.21.15.37:0/3064714915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d2806a170 msgr2=0x7f3d28101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.614+0000 7f3d2659c700 1 --2- 172.21.15.37:0/3064714915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d2806a170 0x7f3d28101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.615+0000 7f3d2659c700 1 -- 172.21.15.37:0/3064714915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d28108500 msgr2=0x7f3d28101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.615+0000 7f3d2659c700 1 --2- 172.21.15.37:0/3064714915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d28108500 0x7f3d28101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.615+0000 7f3d2659c700 1 -- 172.21.15.37:0/3064714915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d281a7140 con 0x7f3d2806ab30 2024-06-10T01:01:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.615+0000 7f3d2759e700 1 --2- 172.21.15.37:0/3064714915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d28108500 0x7f3d28101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:01:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.615+0000 7f3d2659c700 1 --2- 172.21.15.37:0/3064714915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d2806ab30 0x7f3d281019a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00e9a0 tx=0x7f3d1c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:46.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.615+0000 7f3d0ffff700 1 -- 172.21.15.37:0/3064714915 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c00cbd0 con 0x7f3d2806ab30 2024-06-10T01:01:46.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.615+0000 7f3d2d0bf700 1 -- 172.21.15.37:0/3064714915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d281a7430 con 0x7f3d2806ab30 2024-06-10T01:01:46.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.615+0000 7f3d0ffff700 1 -- 172.21.15.37:0/3064714915 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d1c00cd70 con 0x7f3d2806ab30 2024-06-10T01:01:46.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.615+0000 7f3d0ffff700 1 -- 172.21.15.37:0/3064714915 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c018750 con 0x7f3d2806ab30 2024-06-10T01:01:46.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.615+0000 7f3d2d0bf700 1 -- 172.21.15.37:0/3064714915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d281a7970 con 0x7f3d2806ab30 2024-06-10T01:01:46.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.619+0000 7f3d0ffff700 1 -- 172.21.15.37:0/3064714915 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3d1c0188f0 con 0x7f3d2806ab30 2024-06-10T01:01:46.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.619+0000 7f3d0ffff700 1 --2- 172.21.15.37:0/3064714915 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f3d100a2c20 0x7f3d100a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:46.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.620+0000 7f3d0ffff700 1 -- 172.21.15.37:0/3064714915 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f3d1c0cf760 con 0x7f3d2806ab30 2024-06-10T01:01:46.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.620+0000 7f3d26d9d700 1 --2- 172.21.15.37:0/3064714915 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f3d100a2c20 0x7f3d100a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:46.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.620+0000 7f3d37942b80 1 Processor -- start 2024-06-10T01:01:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.620+0000 7f3d37942b80 1 -- start start 2024-06-10T01:01:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.620+0000 7f3d26d9d700 1 --2- 172.21.15.37:0/3064714915 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f3d100a2c20 0x7f3d100a50e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800b2e0 tx=0x7f3d18005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.620+0000 7f3d37942b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56180fee1f60 0x56180fee2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.620+0000 7f3d37942b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56180fee2880 0x56180fedc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:46.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.620+0000 7f3d37942b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56180fedc7a0 0x56180fedcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:46.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.621+0000 7f3d37942b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56180fdcef60 con 0x56180fee2880 2024-06-10T01:01:46.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.621+0000 7f3d37942b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56180fe2d850 con 0x56180fee1f60 2024-06-10T01:01:46.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.621+0000 7f3d37942b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56180fd95460 con 0x56180fedc7a0 2024-06-10T01:01:46.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.621+0000 7f3d2659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56180fee2880 0x56180fedc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:46.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.621+0000 7f3d2759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56180fedc7a0 0x56180fedcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:46.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.621+0000 7f3d2659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56180fee2880 0x56180fedc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52842/0 (socket says 172.21.15.37:52842) 2024-06-10T01:01:46.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.621+0000 7f3d2659c700 1 -- 172.21.15.37:0/1190813381 learned_addr learned my addr 172.21.15.37:0/1190813381 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:46.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.621+0000 7f3d26d9d700 1 --2- 172.21.15.37:0/1190813381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56180fee1f60 0x56180fee2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:46.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.621+0000 7f3d2759e700 1 -- 172.21.15.37:0/1190813381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56180fee1f60 msgr2=0x56180fee2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:46.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.621+0000 7f3d2759e700 1 --2- 172.21.15.37:0/1190813381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56180fee1f60 0x56180fee2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.621+0000 7f3d2759e700 1 -- 172.21.15.37:0/1190813381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56180fee2880 msgr2=0x56180fedc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:46.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.621+0000 7f3d2759e700 1 --2- 172.21.15.37:0/1190813381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56180fee2880 0x56180fedc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.621+0000 7f3d2759e700 1 -- 172.21.15.37:0/1190813381 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56180fd7c8d0 con 0x56180fedc7a0 2024-06-10T01:01:46.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.621+0000 7f3d2659c700 1 --2- 172.21.15.37:0/1190813381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56180fee2880 0x56180fedc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:01:46.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.621+0000 7f3d2759e700 1 --2- 172.21.15.37:0/1190813381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56180fedc7a0 0x56180fedcbc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3d200084f0 tx=0x7f3d20008800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7341ff8b16d9bda6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:46.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.621+0000 7f3d0d7fa700 1 -- 172.21.15.37:0/1190813381 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d20003c00 con 0x56180fedc7a0 2024-06-10T01:01:46.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.622+0000 7f3d0d7fa700 1 -- 172.21.15.37:0/1190813381 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d20003da0 con 0x56180fedc7a0 2024-06-10T01:01:46.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.622+0000 7f3d0d7fa700 1 -- 172.21.15.37:0/1190813381 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d200128f0 con 0x56180fedc7a0 2024-06-10T01:01:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.622+0000 7f3d37942b80 1 -- 172.21.15.37:0/1190813381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56180fedc7a0 msgr2=0x56180fedcbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.622+0000 7f3d37942b80 1 --2- 172.21.15.37:0/1190813381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56180fedc7a0 0x56180fedcbc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3d200084f0 tx=0x7f3d20008800 comp rx=0 tx=0).stop 2024-06-10T01:01:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.622+0000 7f3d37942b80 1 -- 172.21.15.37:0/1190813381 shutdown_connections 2024-06-10T01:01:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.622+0000 7f3d37942b80 1 --2- 172.21.15.37:0/1190813381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56180fedc7a0 0x56180fedcbc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.622+0000 7f3d37942b80 1 --2- 172.21.15.37:0/1190813381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56180fee1f60 0x56180fee2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.622+0000 7f3d37942b80 1 --2- 172.21.15.37:0/1190813381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56180fee2880 0x56180fedc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.623+0000 7f3d37942b80 1 -- 172.21.15.37:0/1190813381 >> 172.21.15.37:0/1190813381 conn(0x56180fe09080 msgr2=0x56180fed7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.623+0000 7f3d37942b80 1 -- 172.21.15.37:0/1190813381 shutdown_connections 2024-06-10T01:01:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.623+0000 7f3d37942b80 1 -- 172.21.15.37:0/1190813381 wait complete. 2024-06-10T01:01:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.623+0000 7f3d37942b80 1 -- start start 2024-06-10T01:01:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.623+0000 7f3d37942b80 1 Processor -- start 2024-06-10T01:01:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.624+0000 7f3d37942b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56180fedc7a0 0x56180fee85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.624+0000 7f3d37942b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56180fee1f60 0x56180fee58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.624+0000 7f3d26d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56180fedc7a0 0x56180fee85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.624+0000 7f3d37942b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56180fee2880 0x56180fee5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.624+0000 7f3d37942b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56180fdcdfc0 con 0x56180fedc7a0 2024-06-10T01:01:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.624+0000 7f3d2659c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56180fee1f60 0x56180fee58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.624+0000 7f3d26d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56180fedc7a0 0x56180fee85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52848/0 (socket says 172.21.15.37:52848) 2024-06-10T01:01:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.624+0000 7f3d26d9d700 1 -- 172.21.15.37:0/1913472205 learned_addr learned my addr 172.21.15.37:0/1913472205 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.624+0000 7f3d37942b80 1 -- 172.21.15.37:0/1913472205 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56180fe29810 con 0x56180fee2880 2024-06-10T01:01:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.624+0000 7f3d37942b80 1 -- 172.21.15.37:0/1913472205 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56180fee9260 con 0x56180fee1f60 2024-06-10T01:01:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.624+0000 7f3d2759e700 1 --2- 172.21.15.37:0/1913472205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56180fee2880 0x56180fee5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.624+0000 7f3d37942b80 10 client.0 _ll_register_callbacks cb 0x56180fdc35b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:01:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.624+0000 7f3d2659c700 1 -- 172.21.15.37:0/1913472205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56180fee2880 msgr2=0x56180fee5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.624+0000 7f3d2659c700 1 --2- 172.21.15.37:0/1913472205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56180fee2880 0x56180fee5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.624+0000 7f3d2659c700 1 -- 172.21.15.37:0/1913472205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56180fedc7a0 msgr2=0x56180fee85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.624+0000 7f3d2659c700 1 --2- 172.21.15.37:0/1913472205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56180fedc7a0 0x56180fee85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.624+0000 7f3d2659c700 1 -- 172.21.15.37:0/1913472205 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56180fdf3750 con 0x56180fee1f60 2024-06-10T01:01:46.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.624+0000 7f3d26d9d700 1 --2- 172.21.15.37:0/1913472205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56180fedc7a0 0x56180fee85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:01:46.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.625+0000 7f3d2659c700 1 --2- 172.21.15.37:0/1913472205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56180fee1f60 0x56180fee58c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c098030 tx=0x7f3d1c018ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a9c1bc41357bcca server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:46.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.625+0000 7f3d0aff5700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:01:46.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.625+0000 7f3d0aff5700 1 -- 172.21.15.37:0/1913472205 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c09bbd0 con 0x56180fee1f60 2024-06-10T01:01:46.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.625+0000 7f3d37942b80 1 -- 172.21.15.37:0/1913472205 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d20003da0 con 0x56180fee1f60 2024-06-10T01:01:46.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.625+0000 7f3d0aff5700 1 -- 172.21.15.37:0/1913472205 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f3d1c0cf410 con 0x56180fee1f60 2024-06-10T01:01:46.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.625+0000 7f3d0aff5700 1 -- 172.21.15.37:0/1913472205 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d1c09ff80 con 0x56180fee1f60 2024-06-10T01:01:46.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.625+0000 7f3d37942b80 10 client.25771 fetch_fsmap learned FSMap version 27 2024-06-10T01:01:46.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.625+0000 7f3d37942b80 1 -- 172.21.15.37:0/1913472205 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56180fef3980 con 0x56180fee1f60 2024-06-10T01:01:46.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.625+0000 7f3d0aff5700 1 -- 172.21.15.37:0/1913472205 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d20003da0 con 0x56180fee1f60 2024-06-10T01:01:46.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.625+0000 7f3d0aff5700 1 -- 172.21.15.37:0/1913472205 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3d1c09b190 con 0x56180fee1f60 2024-06-10T01:01:46.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.626+0000 7f3d37942b80 10 client.25771 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:01:46.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.626+0000 7f3d37942b80 10 client.25771 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-06-10T01:01:46.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.626+0000 7f3d37942b80 20 client.25771 populate_metadata read hostname 'smithi037' 2024-06-10T01:01:46.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.626+0000 7f3d37942b80 1 --2- 172.21.15.37:0/1913472205 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x56180fef5fc0 0x56180fef83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:46.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.626+0000 7f3d37942b80 4 client.25771 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:01:46.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.626+0000 7f3d37942b80 1 -- 172.21.15.37:0/1913472205 --> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56180fe3ad50 con 0x56180fef5fc0 2024-06-10T01:01:46.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.626+0000 7f3d26d9d700 1 --2- 172.21.15.37:0/1913472205 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x56180fef5fc0 0x56180fef83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:46.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.626+0000 7f3d0aff5700 10 client.25771 ms_handle_connect on v2:172.21.15.94:6838/946269331 2024-06-10T01:01:46.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.626+0000 7f3d26d9d700 1 --2- 172.21.15.37:0/1913472205 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x56180fef5fc0 0x56180fef83b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5865468960602184 server_cookie=d08c4c9413c0c1f in_seq=0 out_seq=0 2024-06-10T01:01:46.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.627+0000 7f3d0aff5700 1 -- 172.21.15.37:0/1913472205 <== mds.1 v2:172.21.15.94:6838/946269331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56180fe3ad50 con 0x56180fef5fc0 2024-06-10T01:01:46.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.627+0000 7f3d0aff5700 10 client.25771 handle_command_reply: tid=0 2024-06-10T01:01:46.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.627+0000 7f3d37942b80 1 client.25771 shutdown 2024-06-10T01:01:46.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.628+0000 7f3d37942b80 1 -- 172.21.15.37:0/1913472205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56180fee1f60 msgr2=0x56180fee58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:46.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.628+0000 7f3d37942b80 1 --2- 172.21.15.37:0/1913472205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56180fee1f60 0x56180fee58c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c098030 tx=0x7f3d1c018ec0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.628+0000 7f3d37942b80 1 -- 172.21.15.37:0/1913472205 shutdown_connections 2024-06-10T01:01:46.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.628+0000 7f3d37942b80 1 --2- 172.21.15.37:0/1913472205 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x56180fef5fc0 0x56180fef83b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.628+0000 7f3d37942b80 1 --2- 172.21.15.37:0/1913472205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56180fee2880 0x56180fee5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.628+0000 7f3d0aff5700 0 client.25771 ms_handle_reset on v2:172.21.15.94:6838/946269331 2024-06-10T01:01:46.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.628+0000 7f3d37942b80 1 --2- 172.21.15.37:0/1913472205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56180fedc7a0 0x56180fee85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.628+0000 7f3d37942b80 1 --2- 172.21.15.37:0/1913472205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56180fee1f60 0x56180fee58c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.628+0000 7f3d37942b80 1 -- 172.21.15.37:0/1913472205 >> 172.21.15.37:0/1913472205 conn(0x56180fe09080 msgr2=0x56180fed6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:46.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.628+0000 7f3d37942b80 1 -- 172.21.15.37:0/1913472205 shutdown_connections 2024-06-10T01:01:46.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.628+0000 7f3d37942b80 1 -- 172.21.15.37:0/1913472205 wait complete. 2024-06-10T01:01:46.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.628+0000 7f3d37942b80 20 client.25771 trim_cache size 0 max 16384 2024-06-10T01:01:46.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.644+0000 7f3d37942b80 1 Processor -- start 2024-06-10T01:01:46.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.644+0000 7f3d37942b80 1 -- start start 2024-06-10T01:01:46.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.644+0000 7f3d37942b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56180fe09080 0x56180fedc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:46.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.644+0000 7f3d37942b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56180fedc7a0 0x56180fee5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:46.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.644+0000 7f3d37942b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56180fee1f60 0x56180fee34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:46.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.644+0000 7f3d37942b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3d281b3060 con 0x56180fe09080 2024-06-10T01:01:46.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.644+0000 7f3d37942b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3d281b08f0 con 0x56180fedc7a0 2024-06-10T01:01:46.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.644+0000 7f3d37942b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3d281ae740 con 0x56180fee1f60 2024-06-10T01:01:46.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.644+0000 7f3d26d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56180fe09080 0x56180fedc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:46.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.644+0000 7f3d26d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56180fe09080 0x56180fedc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52850/0 (socket says 172.21.15.37:52850) 2024-06-10T01:01:46.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.644+0000 7f3d2759e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56180fee1f60 0x56180fee34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:46.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.644+0000 7f3d26d9d700 1 -- 172.21.15.37:0/3561831689 learned_addr learned my addr 172.21.15.37:0/3561831689 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:46.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.644+0000 7f3d2659c700 1 --2- 172.21.15.37:0/3561831689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56180fedc7a0 0x56180fee5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:46.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.644+0000 7f3d26d9d700 1 -- 172.21.15.37:0/3561831689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56180fee1f60 msgr2=0x56180fee34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:46.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.644+0000 7f3d26d9d700 1 --2- 172.21.15.37:0/3561831689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56180fee1f60 0x56180fee34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.644+0000 7f3d26d9d700 1 -- 172.21.15.37:0/3561831689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56180fedc7a0 msgr2=0x56180fee5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:46.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.644+0000 7f3d26d9d700 1 --2- 172.21.15.37:0/3561831689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56180fedc7a0 0x56180fee5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.644+0000 7f3d26d9d700 1 -- 172.21.15.37:0/3561831689 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56180fee3e30 con 0x56180fe09080 2024-06-10T01:01:46.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.644+0000 7f3d2659c700 1 --2- 172.21.15.37:0/3561831689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56180fedc7a0 0x56180fee5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:01:46.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.645+0000 7f3d26d9d700 1 --2- 172.21.15.37:0/3561831689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56180fe09080 0x56180fedc260 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3d180098e0 tx=0x7f3d18021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1cb4031a6cc39 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:46.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.645+0000 7f3d0cff9700 1 -- 172.21.15.37:0/3561831689 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1800f460 con 0x56180fe09080 2024-06-10T01:01:46.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.645+0000 7f3d0cff9700 1 -- 172.21.15.37:0/3561831689 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d1800fae0 con 0x56180fe09080 2024-06-10T01:01:46.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.645+0000 7f3d0cff9700 1 -- 172.21.15.37:0/3561831689 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d18022a00 con 0x56180fe09080 2024-06-10T01:01:46.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.646+0000 7f3d37942b80 1 -- 172.21.15.37:0/3561831689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56180fe09080 msgr2=0x56180fedc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:46.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.646+0000 7f3d37942b80 1 --2- 172.21.15.37:0/3561831689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56180fe09080 0x56180fedc260 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3d180098e0 tx=0x7f3d18021000 comp rx=0 tx=0).stop 2024-06-10T01:01:46.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.646+0000 7f3d37942b80 1 -- 172.21.15.37:0/3561831689 shutdown_connections 2024-06-10T01:01:46.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.646+0000 7f3d37942b80 1 --2- 172.21.15.37:0/3561831689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56180fee1f60 0x56180fee34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.646+0000 7f3d37942b80 1 --2- 172.21.15.37:0/3561831689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56180fe09080 0x56180fedc260 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.646+0000 7f3d37942b80 1 --2- 172.21.15.37:0/3561831689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56180fedc7a0 0x56180fee5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.646+0000 7f3d37942b80 1 -- 172.21.15.37:0/3561831689 >> 172.21.15.37:0/3561831689 conn(0x56180fee01d0 msgr2=0x56180fee41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:46.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.646+0000 7f3d37942b80 1 -- 172.21.15.37:0/3561831689 shutdown_connections 2024-06-10T01:01:46.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.646+0000 7f3d37942b80 1 -- 172.21.15.37:0/3561831689 wait complete. 2024-06-10T01:01:46.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.646+0000 7f3d37942b80 1 -- start start 2024-06-10T01:01:46.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.646+0000 7f3d37942b80 1 Processor -- start 2024-06-10T01:01:46.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.647+0000 7f3d37942b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56180fe09080 0x56180feeed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:46.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.647+0000 7f3d37942b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56180fedc7a0 0x56180fefd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:46.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.647+0000 7f3d37942b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56180fee1f60 0x56180fefdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:46.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.647+0000 7f3d37942b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3d281aedb0 con 0x56180fedc7a0 2024-06-10T01:01:46.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.647+0000 7f3d37942b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3d281b0220 con 0x56180fee1f60 2024-06-10T01:01:46.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.647+0000 7f3d37942b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3d281af480 con 0x56180fe09080 2024-06-10T01:01:46.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.647+0000 7f3d37942b80 10 client.0 _ll_register_callbacks cb 0x56180fdc35b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:01:46.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.647+0000 7f3d2659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56180fedc7a0 0x56180fefd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:46.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.647+0000 7f3d26d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56180fe09080 0x56180feeed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:46.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.647+0000 7f3d2659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56180fedc7a0 0x56180fefd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52864/0 (socket says 172.21.15.37:52864) 2024-06-10T01:01:46.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.647+0000 7f3d2659c700 1 -- 172.21.15.37:0/1416621615 learned_addr learned my addr 172.21.15.37:0/1416621615 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:46.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.647+0000 7f3d2759e700 1 --2- 172.21.15.37:0/1416621615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56180fee1f60 0x56180fefdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:46.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.647+0000 7f3d2659c700 1 -- 172.21.15.37:0/1416621615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56180fe09080 msgr2=0x56180feeed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:46.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.647+0000 7f3d2659c700 1 --2- 172.21.15.37:0/1416621615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56180fe09080 0x56180feeed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.647+0000 7f3d2659c700 1 -- 172.21.15.37:0/1416621615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56180fee1f60 msgr2=0x56180fefdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:46.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.647+0000 7f3d2659c700 1 --2- 172.21.15.37:0/1416621615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56180fee1f60 0x56180fefdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.647+0000 7f3d2659c700 1 -- 172.21.15.37:0/1416621615 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56180fefe4e0 con 0x56180fedc7a0 2024-06-10T01:01:46.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.647+0000 7f3d2659c700 1 --2- 172.21.15.37:0/1416621615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56180fedc7a0 0x56180fefd6e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c09b8d0 tx=0x7f3d1c005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b489d762b9195ce server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:46.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.647+0000 7f3d0aff5700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:01:46.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.648+0000 7f3d0aff5700 1 -- 172.21.15.37:0/1416621615 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c07e060 con 0x56180fedc7a0 2024-06-10T01:01:46.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.648+0000 7f3d0aff5700 1 -- 172.21.15.37:0/1416621615 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f3d1c09b020 con 0x56180fedc7a0 2024-06-10T01:01:46.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.648+0000 7f3d37942b80 1 -- 172.21.15.37:0/1416621615 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d1800fae0 con 0x56180fedc7a0 2024-06-10T01:01:46.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.648+0000 7f3d0aff5700 1 -- 172.21.15.37:0/1416621615 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d1c09bbd0 con 0x56180fedc7a0 2024-06-10T01:01:46.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.648+0000 7f3d37942b80 10 client.16209 fetch_fsmap learned FSMap version 27 2024-06-10T01:01:46.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.648+0000 7f3d37942b80 1 -- 172.21.15.37:0/1416621615 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56180ff60d80 con 0x56180fedc7a0 2024-06-10T01:01:46.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.648+0000 7f3d0aff5700 1 -- 172.21.15.37:0/1416621615 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d1800fae0 con 0x56180fedc7a0 2024-06-10T01:01:46.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.648+0000 7f3d0aff5700 1 -- 172.21.15.37:0/1416621615 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3d1c09aa90 con 0x56180fedc7a0 2024-06-10T01:01:46.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.648+0000 7f3d37942b80 10 client.16209 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:01:46.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.648+0000 7f3d37942b80 10 client.16209 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-06-10T01:01:46.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.648+0000 7f3d37942b80 20 client.16209 populate_metadata read hostname 'smithi037' 2024-06-10T01:01:46.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.648+0000 7f3d37942b80 1 --2- 172.21.15.37:0/1416621615 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x56180fee2880 0x56180ff63480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:46.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.649+0000 7f3d37942b80 4 client.16209 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:01:46.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.649+0000 7f3d37942b80 1 -- 172.21.15.37:0/1416621615 --> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56180fe352d0 con 0x56180fee2880 2024-06-10T01:01:46.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.649+0000 7f3d26d9d700 1 --2- 172.21.15.37:0/1416621615 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x56180fee2880 0x56180ff63480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:46.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.649+0000 7f3d0aff5700 10 client.16209 ms_handle_connect on v2:172.21.15.94:6838/946269331 2024-06-10T01:01:46.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.649+0000 7f3d26d9d700 1 --2- 172.21.15.37:0/1416621615 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x56180fee2880 0x56180ff63480 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a0a9f5236994771b server_cookie=b689f8825e0f65a7 in_seq=0 out_seq=0 2024-06-10T01:01:46.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.650+0000 7f3d0aff5700 1 -- 172.21.15.37:0/1416621615 <== mds.1 v2:172.21.15.94:6838/946269331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56180fe352d0 con 0x56180fee2880 2024-06-10T01:01:46.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.650+0000 7f3d0aff5700 10 client.16209 handle_command_reply: tid=0 2024-06-10T01:01:46.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.650+0000 7f3d37942b80 1 client.16209 shutdown 2024-06-10T01:01:46.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.650+0000 7f3d37942b80 1 -- 172.21.15.37:0/1416621615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56180fedc7a0 msgr2=0x56180fefd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:46.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.650+0000 7f3d37942b80 1 --2- 172.21.15.37:0/1416621615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56180fedc7a0 0x56180fefd6e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c09b8d0 tx=0x7f3d1c005b40 comp rx=0 tx=0).stop 2024-06-10T01:01:46.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.650+0000 7f3d37942b80 1 -- 172.21.15.37:0/1416621615 shutdown_connections 2024-06-10T01:01:46.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.650+0000 7f3d37942b80 1 --2- 172.21.15.37:0/1416621615 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x56180fee2880 0x56180ff63480 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.650+0000 7f3d37942b80 1 --2- 172.21.15.37:0/1416621615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56180fee1f60 0x56180fefdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.650+0000 7f3d37942b80 1 --2- 172.21.15.37:0/1416621615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56180fe09080 0x56180feeed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.650+0000 7f3d0aff5700 0 client.16209 ms_handle_reset on v2:172.21.15.94:6838/946269331 2024-06-10T01:01:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.650+0000 7f3d37942b80 1 --2- 172.21.15.37:0/1416621615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56180fedc7a0 0x56180fefd6e0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.650+0000 7f3d37942b80 1 -- 172.21.15.37:0/1416621615 >> 172.21.15.37:0/1416621615 conn(0x56180fee01d0 msgr2=0x56180fee4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.650+0000 7f3d37942b80 1 -- 172.21.15.37:0/1416621615 shutdown_connections 2024-06-10T01:01:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.650+0000 7f3d37942b80 1 -- 172.21.15.37:0/1416621615 wait complete. 2024-06-10T01:01:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.650+0000 7f3d37942b80 20 client.16209 trim_cache size 0 max 16384 2024-06-10T01:01:46.652 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T01:01:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.651+0000 7f3d2d0bf700 1 -- 172.21.15.37:0/3064714915 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f3d100a2c20 msgr2=0x7f3d100a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.651+0000 7f3d2d0bf700 1 --2- 172.21.15.37:0/3064714915 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f3d100a2c20 0x7f3d100a50e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800b2e0 tx=0x7f3d18005960 comp rx=0 tx=0).stop 2024-06-10T01:01:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.651+0000 7f3d2d0bf700 1 -- 172.21.15.37:0/3064714915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d2806ab30 msgr2=0x7f3d281019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.651+0000 7f3d2d0bf700 1 --2- 172.21.15.37:0/3064714915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d2806ab30 0x7f3d281019a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00e9a0 tx=0x7f3d1c00ecb0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.651+0000 7f3d2d0bf700 1 -- 172.21.15.37:0/3064714915 shutdown_connections 2024-06-10T01:01:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.651+0000 7f3d2d0bf700 1 --2- 172.21.15.37:0/3064714915 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f3d100a2c20 0x7f3d100a50e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.651+0000 7f3d2d0bf700 1 --2- 172.21.15.37:0/3064714915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d28108500 0x7f3d28101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.651+0000 7f3d2d0bf700 1 --2- 172.21.15.37:0/3064714915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d2806a170 0x7f3d28101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.651+0000 7f3d2d0bf700 1 --2- 172.21.15.37:0/3064714915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d2806ab30 0x7f3d281019a0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.651+0000 7f3d2d0bf700 1 -- 172.21.15.37:0/3064714915 >> 172.21.15.37:0/3064714915 conn(0x7f3d28078340 msgr2=0x7f3d280fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.651+0000 7f3d2d0bf700 1 -- 172.21.15.37:0/3064714915 shutdown_connections 2024-06-10T01:01:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:46.651+0000 7f3d2d0bf700 1 -- 172.21.15.37:0/3064714915 wait complete. 2024-06-10T01:01:47.455 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:01:47.730 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:01:48.111 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:48 smithi037 ceph-mon[32192]: pgmap v583: 129 pgs: 129 active+clean; 4.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 306 KiB/s wr, 27 op/s 2024-06-10T01:01:48.118 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.2 2024-06-10T01:01:48.118 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:01:48.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:48 smithi094 ceph-mon[36708]: pgmap v583: 129 pgs: 129 active+clean; 4.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 306 KiB/s wr, 27 op/s 2024-06-10T01:01:48.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:48 smithi167 ceph-mon[36917]: pgmap v583: 129 pgs: 129 active+clean; 4.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 306 KiB/s wr, 27 op/s 2024-06-10T01:01:48.965 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:01:49.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.848+0000 7fbaf27e2700 1 -- 172.21.15.37:0/943725810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbaec106b90 msgr2=0x7fbaec106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:49.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.848+0000 7fbaf27e2700 1 --2- 172.21.15.37:0/943725810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbaec106b90 0x7fbaec106f70 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fbadc009990 tx=0x7fbadc009ca0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.848+0000 7fbaf27e2700 1 -- 172.21.15.37:0/943725810 shutdown_connections 2024-06-10T01:01:49.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.848+0000 7fbaf27e2700 1 --2- 172.21.15.37:0/943725810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaec101510 0x7fbaec10d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.848+0000 7fbaf27e2700 1 --2- 172.21.15.37:0/943725810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbaec106b90 0x7fbaec106f70 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.848+0000 7fbaf27e2700 1 --2- 172.21.15.37:0/943725810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbaec100b50 0x7fbaec100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.848+0000 7fbaf27e2700 1 -- 172.21.15.37:0/943725810 >> 172.21.15.37:0/943725810 conn(0x7fbaec077a00 msgr2=0x7fbaec0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:49.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.848+0000 7fbaf27e2700 1 -- 172.21.15.37:0/943725810 shutdown_connections 2024-06-10T01:01:49.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.848+0000 7fbaf27e2700 1 -- 172.21.15.37:0/943725810 wait complete. 2024-06-10T01:01:49.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.849+0000 7fbaf27e2700 1 Processor -- start 2024-06-10T01:01:49.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.849+0000 7fbaf27e2700 1 -- start start 2024-06-10T01:01:49.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.849+0000 7fbaf27e2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaec100b50 0x7fbaec19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:49.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.849+0000 7fbaf27e2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbaec101510 0x7fbaec19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:49.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.849+0000 7fbaf27e2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbaec106b90 0x7fbaec198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:49.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.849+0000 7fbaf27e2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec110600 con 0x7fbaec106b90 2024-06-10T01:01:49.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.849+0000 7fbaf27e2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec110480 con 0x7fbaec101510 2024-06-10T01:01:49.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.849+0000 7fbaf27e2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec110780 con 0x7fbaec100b50 2024-06-10T01:01:49.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.850+0000 7fbaf0d7f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbaec106b90 0x7fbaec198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:49.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.850+0000 7fbaf0d7f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbaec106b90 0x7fbaec198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52878/0 (socket says 172.21.15.37:52878) 2024-06-10T01:01:49.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.850+0000 7fbaf0d7f700 1 -- 172.21.15.37:0/2470345360 learned_addr learned my addr 172.21.15.37:0/2470345360 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:49.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.850+0000 7fbaebfff700 1 --2- 172.21.15.37:0/2470345360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaec100b50 0x7fbaec19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:49.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.850+0000 7fbaeb7fe700 1 --2- 172.21.15.37:0/2470345360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbaec101510 0x7fbaec19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:49.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.850+0000 7fbaf0d7f700 1 -- 172.21.15.37:0/2470345360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaec100b50 msgr2=0x7fbaec19e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:49.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.850+0000 7fbaf0d7f700 1 --2- 172.21.15.37:0/2470345360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaec100b50 0x7fbaec19e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.850+0000 7fbaf0d7f700 1 -- 172.21.15.37:0/2470345360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbaec101510 msgr2=0x7fbaec19edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:49.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.850+0000 7fbaf0d7f700 1 --2- 172.21.15.37:0/2470345360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbaec101510 0x7fbaec19edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.850+0000 7fbaf0d7f700 1 -- 172.21.15.37:0/2470345360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaec1991a0 con 0x7fbaec106b90 2024-06-10T01:01:49.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.850+0000 7fbaf0d7f700 1 --2- 172.21.15.37:0/2470345360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbaec106b90 0x7fbaec198a40 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fbae4009fa0 tx=0x7fbae40076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:49.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.850+0000 7fbae97fa700 1 -- 172.21.15.37:0/2470345360 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae4019070 con 0x7fbaec106b90 2024-06-10T01:01:49.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.850+0000 7fbaf27e2700 1 -- 172.21.15.37:0/2470345360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaec199490 con 0x7fbaec106b90 2024-06-10T01:01:49.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.850+0000 7fbaf27e2700 1 -- 172.21.15.37:0/2470345360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaec1a5750 con 0x7fbaec106b90 2024-06-10T01:01:49.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.850+0000 7fbae97fa700 1 -- 172.21.15.37:0/2470345360 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae4003c20 con 0x7fbaec106b90 2024-06-10T01:01:49.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.850+0000 7fbae97fa700 1 -- 172.21.15.37:0/2470345360 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae40147b0 con 0x7fbaec106b90 2024-06-10T01:01:49.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.853+0000 7fbae97fa700 1 -- 172.21.15.37:0/2470345360 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fbae401c070 con 0x7fbaec106b90 2024-06-10T01:01:49.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.854+0000 7fbae97fa700 1 --2- 172.21.15.37:0/2470345360 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fbad40a2ca0 0x7fbad40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:49.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.854+0000 7fbaebfff700 1 --2- 172.21.15.37:0/2470345360 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fbad40a2ca0 0x7fbad40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:49.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.854+0000 7fbae97fa700 1 -- 172.21.15.37:0/2470345360 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fbae40ced40 con 0x7fbaec106b90 2024-06-10T01:01:49.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.854+0000 7fbaebfff700 1 --2- 172.21.15.37:0/2470345360 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fbad40a2ca0 0x7fbad40a5160 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbaec199b80 tx=0x7fbadc01a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:49.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.855+0000 7fbafd065b80 1 Processor -- start 2024-06-10T01:01:49.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.855+0000 7fbafd065b80 1 -- start start 2024-06-10T01:01:49.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.855+0000 7fbafd065b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb1c4eb370 0x55bb1c4eb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:49.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.855+0000 7fbafd065b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb1c4ebc90 0x55bb1c4e5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:49.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.855+0000 7fbafd065b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb1c4e5bb0 0x55bb1c4e5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:49.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.855+0000 7fbafd065b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bb1c3d3f60 con 0x55bb1c4ebc90 2024-06-10T01:01:49.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.855+0000 7fbafd065b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bb1c432850 con 0x55bb1c4e5bb0 2024-06-10T01:01:49.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.855+0000 7fbafd065b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bb1c39a460 con 0x55bb1c4eb370 2024-06-10T01:01:49.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.855+0000 7fbaeb7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb1c4ebc90 0x55bb1c4e5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:49.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.855+0000 7fbaeb7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb1c4ebc90 0x55bb1c4e5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52888/0 (socket says 172.21.15.37:52888) 2024-06-10T01:01:49.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.855+0000 7fbaeb7fe700 1 -- 172.21.15.37:0/1555577915 learned_addr learned my addr 172.21.15.37:0/1555577915 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:49.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.855+0000 7fbaebfff700 1 --2- 172.21.15.37:0/1555577915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb1c4eb370 0x55bb1c4eb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:49.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.855+0000 7fbaf0d7f700 1 --2- 172.21.15.37:0/1555577915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb1c4e5bb0 0x55bb1c4e5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:49.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.856+0000 7fbaeb7fe700 1 -- 172.21.15.37:0/1555577915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb1c4eb370 msgr2=0x55bb1c4eb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:49.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.856+0000 7fbaeb7fe700 1 --2- 172.21.15.37:0/1555577915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb1c4eb370 0x55bb1c4eb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.856+0000 7fbaeb7fe700 1 -- 172.21.15.37:0/1555577915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb1c4e5bb0 msgr2=0x55bb1c4e5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:49.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.856+0000 7fbaeb7fe700 1 --2- 172.21.15.37:0/1555577915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb1c4e5bb0 0x55bb1c4e5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.856+0000 7fbaeb7fe700 1 -- 172.21.15.37:0/1555577915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb1c3818d0 con 0x55bb1c4ebc90 2024-06-10T01:01:49.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.856+0000 7fbaebfff700 1 --2- 172.21.15.37:0/1555577915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb1c4eb370 0x55bb1c4eb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:01:49.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.856+0000 7fbaf0d7f700 1 --2- 172.21.15.37:0/1555577915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb1c4e5bb0 0x55bb1c4e5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:01:49.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.856+0000 7fbaeb7fe700 1 --2- 172.21.15.37:0/1555577915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb1c4ebc90 0x55bb1c4e5670 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009d20 tx=0x7fbae0011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ccdcbcaa2fa379 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:49.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.856+0000 7fbad27fc700 1 -- 172.21.15.37:0/1555577915 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae001e070 con 0x55bb1c4ebc90 2024-06-10T01:01:49.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.856+0000 7fbad27fc700 1 -- 172.21.15.37:0/1555577915 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae0019b10 con 0x55bb1c4ebc90 2024-06-10T01:01:49.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.856+0000 7fbad27fc700 1 -- 172.21.15.37:0/1555577915 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae000eb70 con 0x55bb1c4ebc90 2024-06-10T01:01:49.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.857+0000 7fbafd065b80 1 -- 172.21.15.37:0/1555577915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb1c4ebc90 msgr2=0x55bb1c4e5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:49.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.857+0000 7fbafd065b80 1 --2- 172.21.15.37:0/1555577915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb1c4ebc90 0x55bb1c4e5670 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009d20 tx=0x7fbae0011720 comp rx=0 tx=0).stop 2024-06-10T01:01:49.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.857+0000 7fbafd065b80 1 -- 172.21.15.37:0/1555577915 shutdown_connections 2024-06-10T01:01:49.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.857+0000 7fbafd065b80 1 --2- 172.21.15.37:0/1555577915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb1c4e5bb0 0x55bb1c4e5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.857+0000 7fbafd065b80 1 --2- 172.21.15.37:0/1555577915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb1c4eb370 0x55bb1c4eb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.857+0000 7fbafd065b80 1 --2- 172.21.15.37:0/1555577915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb1c4ebc90 0x55bb1c4e5670 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.857+0000 7fbafd065b80 1 -- 172.21.15.37:0/1555577915 >> 172.21.15.37:0/1555577915 conn(0x55bb1c40e080 msgr2=0x55bb1c4dc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:49.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.857+0000 7fbafd065b80 1 -- 172.21.15.37:0/1555577915 shutdown_connections 2024-06-10T01:01:49.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.857+0000 7fbafd065b80 1 -- 172.21.15.37:0/1555577915 wait complete. 2024-06-10T01:01:49.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.858+0000 7fbafd065b80 1 -- start start 2024-06-10T01:01:49.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.858+0000 7fbafd065b80 1 Processor -- start 2024-06-10T01:01:49.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.858+0000 7fbafd065b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb1c4e5bb0 0x55bb1c4f1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:49.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.858+0000 7fbafd065b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb1c4eb370 0x55bb1c4eed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:49.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.858+0000 7fbafd065b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb1c4ebc90 0x55bb1c4ef240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:49.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.858+0000 7fbafd065b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bb1c3d2fc0 con 0x55bb1c4e5bb0 2024-06-10T01:01:49.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.858+0000 7fbafd065b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bb1c42e810 con 0x55bb1c4ebc90 2024-06-10T01:01:49.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.858+0000 7fbafd065b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bb1c4f2680 con 0x55bb1c4eb370 2024-06-10T01:01:49.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.858+0000 7fbafd065b80 10 client.0 _ll_register_callbacks cb 0x55bb1c3c85b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:01:49.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.858+0000 7fbaebfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb1c4e5bb0 0x55bb1c4f1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:49.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.858+0000 7fbaeb7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb1c4eb370 0x55bb1c4eed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:49.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.858+0000 7fbaebfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb1c4e5bb0 0x55bb1c4f1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52892/0 (socket says 172.21.15.37:52892) 2024-06-10T01:01:49.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.858+0000 7fbaebfff700 1 -- 172.21.15.37:0/1341235683 learned_addr learned my addr 172.21.15.37:0/1341235683 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:49.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.858+0000 7fbaf0d7f700 1 --2- 172.21.15.37:0/1341235683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb1c4ebc90 0x55bb1c4ef240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:49.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.859+0000 7fbaeb7fe700 1 -- 172.21.15.37:0/1341235683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb1c4ebc90 msgr2=0x55bb1c4ef240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:49.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.859+0000 7fbaeb7fe700 1 --2- 172.21.15.37:0/1341235683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb1c4ebc90 0x55bb1c4ef240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.859+0000 7fbaeb7fe700 1 -- 172.21.15.37:0/1341235683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb1c4e5bb0 msgr2=0x55bb1c4f1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:49.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.859+0000 7fbaeb7fe700 1 --2- 172.21.15.37:0/1341235683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb1c4e5bb0 0x55bb1c4f1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.859+0000 7fbaeb7fe700 1 -- 172.21.15.37:0/1341235683 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb1c3f8750 con 0x55bb1c4eb370 2024-06-10T01:01:49.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.859+0000 7fbaeb7fe700 1 --2- 172.21.15.37:0/1341235683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb1c4eb370 0x55bb1c4eed00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009d20 tx=0x7fbae00058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=301c1212455d657f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:49.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.859+0000 7fbacb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:01:49.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.859+0000 7fbacb7fe700 1 -- 172.21.15.37:0/1341235683 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae001e050 con 0x55bb1c4eb370 2024-06-10T01:01:49.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.859+0000 7fbacb7fe700 1 -- 172.21.15.37:0/1341235683 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fbae00198c0 con 0x55bb1c4eb370 2024-06-10T01:01:49.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.859+0000 7fbafd065b80 1 -- 172.21.15.37:0/1341235683 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbae0019b10 con 0x55bb1c4eb370 2024-06-10T01:01:49.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.859+0000 7fbacb7fe700 1 -- 172.21.15.37:0/1341235683 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbae0032050 con 0x55bb1c4eb370 2024-06-10T01:01:49.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.859+0000 7fbafd065b80 10 client.25789 fetch_fsmap learned FSMap version 27 2024-06-10T01:01:49.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.859+0000 7fbafd065b80 1 -- 172.21.15.37:0/1341235683 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bb1c4fce00 con 0x55bb1c4eb370 2024-06-10T01:01:49.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.860+0000 7fbacb7fe700 1 -- 172.21.15.37:0/1341235683 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae0019b10 con 0x55bb1c4eb370 2024-06-10T01:01:49.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.860+0000 7fbacb7fe700 1 -- 172.21.15.37:0/1341235683 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbae002d030 con 0x55bb1c4eb370 2024-06-10T01:01:49.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.860+0000 7fbafd065b80 10 client.25789 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:01:49.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.860+0000 7fbafd065b80 10 client.25789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:01:49.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.860+0000 7fbafd065b80 20 client.25789 populate_metadata read hostname 'smithi037' 2024-06-10T01:01:49.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.860+0000 7fbafd065b80 1 --2- 172.21.15.37:0/1341235683 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bb1c4ff380 0x55bb1c501770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:49.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.860+0000 7fbafd065b80 4 client.25789 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:01:49.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.860+0000 7fbafd065b80 1 -- 172.21.15.37:0/1341235683 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb1c43fd50 con 0x55bb1c4ff380 2024-06-10T01:01:49.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.860+0000 7fbaebfff700 1 --2- 172.21.15.37:0/1341235683 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bb1c4ff380 0x55bb1c501770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:49.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.860+0000 7fbacb7fe700 10 client.25789 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:49.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.860+0000 7fbaebfff700 1 --2- 172.21.15.37:0/1341235683 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bb1c4ff380 0x55bb1c501770 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cbe1c7e778a4d62 server_cookie=686650198891e734 in_seq=0 out_seq=0 2024-06-10T01:01:49.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.863+0000 7fbacb7fe700 1 -- 172.21.15.37:0/1341235683 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bb1c43fd50 con 0x55bb1c4ff380 2024-06-10T01:01:49.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.863+0000 7fbacb7fe700 10 client.25789 handle_command_reply: tid=0 2024-06-10T01:01:49.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.863+0000 7fbafd065b80 1 client.25789 shutdown 2024-06-10T01:01:49.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.863+0000 7fbafd065b80 1 -- 172.21.15.37:0/1341235683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb1c4eb370 msgr2=0x55bb1c4eed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:49.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.863+0000 7fbafd065b80 1 --2- 172.21.15.37:0/1341235683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb1c4eb370 0x55bb1c4eed00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009d20 tx=0x7fbae00058e0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.863+0000 7fbafd065b80 1 -- 172.21.15.37:0/1341235683 shutdown_connections 2024-06-10T01:01:49.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.863+0000 7fbafd065b80 1 --2- 172.21.15.37:0/1341235683 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bb1c4ff380 0x55bb1c501770 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.863+0000 7fbafd065b80 1 --2- 172.21.15.37:0/1341235683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb1c4ebc90 0x55bb1c4ef240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.863+0000 7fbafd065b80 1 --2- 172.21.15.37:0/1341235683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb1c4e5bb0 0x55bb1c4f1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.863+0000 7fbafd065b80 1 --2- 172.21.15.37:0/1341235683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb1c4eb370 0x55bb1c4eed00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.863+0000 7fbacb7fe700 0 client.25789 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:49.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.863+0000 7fbafd065b80 1 -- 172.21.15.37:0/1341235683 >> 172.21.15.37:0/1341235683 conn(0x55bb1c40e080 msgr2=0x55bb1c4db700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:49.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.863+0000 7fbafd065b80 1 -- 172.21.15.37:0/1341235683 shutdown_connections 2024-06-10T01:01:49.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.863+0000 7fbafd065b80 1 -- 172.21.15.37:0/1341235683 wait complete. 2024-06-10T01:01:49.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.863+0000 7fbafd065b80 20 client.25789 trim_cache size 0 max 16384 2024-06-10T01:01:49.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.880+0000 7fbafd065b80 1 Processor -- start 2024-06-10T01:01:49.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.880+0000 7fbafd065b80 1 -- start start 2024-06-10T01:01:49.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.881+0000 7fbafd065b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb1c4e5bb0 0x55bb1c4ec840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:49.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.881+0000 7fbafd065b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb1c4eb370 0x55bb1c4f1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:49.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.881+0000 7fbafd065b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb1c4ebc90 0x55bb1c4dbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:49.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.881+0000 7fbafd065b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec1aec20 con 0x55bb1c4e5bb0 2024-06-10T01:01:49.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.881+0000 7fbafd065b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec1ac4b0 con 0x55bb1c4ebc90 2024-06-10T01:01:49.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.881+0000 7fbafd065b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec1aa300 con 0x55bb1c4eb370 2024-06-10T01:01:49.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.881+0000 7fbaebfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb1c4e5bb0 0x55bb1c4ec840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:49.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.881+0000 7fbaf0d7f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb1c4ebc90 0x55bb1c4dbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:49.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.881+0000 7fbaeb7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb1c4eb370 0x55bb1c4f1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:49.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.881+0000 7fbaf0d7f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb1c4ebc90 0x55bb1c4dbe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:59514/0 (socket says 172.21.15.37:59514) 2024-06-10T01:01:49.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.881+0000 7fbaf0d7f700 1 -- 172.21.15.37:0/1481966282 learned_addr learned my addr 172.21.15.37:0/1481966282 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:49.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.881+0000 7fbaf0d7f700 1 -- 172.21.15.37:0/1481966282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb1c4eb370 msgr2=0x55bb1c4f1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:49.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.881+0000 7fbaf0d7f700 1 --2- 172.21.15.37:0/1481966282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb1c4eb370 0x55bb1c4f1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.881+0000 7fbaf0d7f700 1 -- 172.21.15.37:0/1481966282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb1c4e5bb0 msgr2=0x55bb1c4ec840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:49.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.881+0000 7fbaf0d7f700 1 --2- 172.21.15.37:0/1481966282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb1c4e5bb0 0x55bb1c4ec840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.881+0000 7fbaf0d7f700 1 -- 172.21.15.37:0/1481966282 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb1c4e9750 con 0x55bb1c4ebc90 2024-06-10T01:01:49.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.881+0000 7fbaeb7fe700 1 --2- 172.21.15.37:0/1481966282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb1c4eb370 0x55bb1c4f1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:01:49.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.882+0000 7fbaf0d7f700 1 --2- 172.21.15.37:0/1481966282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb1c4ebc90 0x55bb1c4dbe80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbae40cc3f0 tx=0x7fbae40ce040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff28fe1fbbe7ad1e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:49.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.882+0000 7fbad1ffb700 1 -- 172.21.15.37:0/1481966282 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae409b3b0 con 0x55bb1c4ebc90 2024-06-10T01:01:49.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.882+0000 7fbad1ffb700 1 -- 172.21.15.37:0/1481966282 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae409b550 con 0x55bb1c4ebc90 2024-06-10T01:01:49.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.882+0000 7fbad1ffb700 1 -- 172.21.15.37:0/1481966282 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae409a840 con 0x55bb1c4ebc90 2024-06-10T01:01:49.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.882+0000 7fbafd065b80 1 -- 172.21.15.37:0/1481966282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb1c4ebc90 msgr2=0x55bb1c4dbe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:49.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.882+0000 7fbafd065b80 1 --2- 172.21.15.37:0/1481966282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb1c4ebc90 0x55bb1c4dbe80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbae40cc3f0 tx=0x7fbae40ce040 comp rx=0 tx=0).stop 2024-06-10T01:01:49.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.883+0000 7fbafd065b80 1 -- 172.21.15.37:0/1481966282 shutdown_connections 2024-06-10T01:01:49.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.883+0000 7fbafd065b80 1 --2- 172.21.15.37:0/1481966282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb1c4ebc90 0x55bb1c4dbe80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.883+0000 7fbafd065b80 1 --2- 172.21.15.37:0/1481966282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb1c4e5bb0 0x55bb1c4ec840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.883+0000 7fbafd065b80 1 --2- 172.21.15.37:0/1481966282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb1c4eb370 0x55bb1c4f1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.883+0000 7fbafd065b80 1 -- 172.21.15.37:0/1481966282 >> 172.21.15.37:0/1481966282 conn(0x55bb1c40e080 msgr2=0x55bb1c4ec130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:49.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.883+0000 7fbafd065b80 1 -- 172.21.15.37:0/1481966282 shutdown_connections 2024-06-10T01:01:49.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.883+0000 7fbafd065b80 1 -- 172.21.15.37:0/1481966282 wait complete. 2024-06-10T01:01:49.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.883+0000 7fbafd065b80 1 -- start start 2024-06-10T01:01:49.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.883+0000 7fbafd065b80 1 Processor -- start 2024-06-10T01:01:49.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.883+0000 7fbafd065b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb1c4e5bb0 0x55bb1c4f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:49.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.883+0000 7fbafd065b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb1c4eb370 0x55bb1c506920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:49.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.883+0000 7fbafd065b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb1c4ebc90 0x55bb1c507060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:49.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.883+0000 7fbafd065b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec1aa970 con 0x55bb1c4eb370 2024-06-10T01:01:49.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.883+0000 7fbafd065b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec1abde0 con 0x55bb1c4e5bb0 2024-06-10T01:01:49.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.884+0000 7fbafd065b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec1ab040 con 0x55bb1c4ebc90 2024-06-10T01:01:49.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.884+0000 7fbafd065b80 10 client.0 _ll_register_callbacks cb 0x55bb1c3c85b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:01:49.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.884+0000 7fbaebfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb1c4e5bb0 0x55bb1c4f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:49.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.884+0000 7fbaeb7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb1c4eb370 0x55bb1c506920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:49.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.884+0000 7fbaebfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb1c4e5bb0 0x55bb1c4f81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:59524/0 (socket says 172.21.15.37:59524) 2024-06-10T01:01:49.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.884+0000 7fbaebfff700 1 -- 172.21.15.37:0/3706491301 learned_addr learned my addr 172.21.15.37:0/3706491301 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:49.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.884+0000 7fbaf0d7f700 1 --2- 172.21.15.37:0/3706491301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb1c4ebc90 0x55bb1c507060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:49.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.884+0000 7fbaebfff700 1 -- 172.21.15.37:0/3706491301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb1c4ebc90 msgr2=0x55bb1c507060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:49.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.884+0000 7fbaebfff700 1 --2- 172.21.15.37:0/3706491301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb1c4ebc90 0x55bb1c507060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.884+0000 7fbaebfff700 1 -- 172.21.15.37:0/3706491301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb1c4eb370 msgr2=0x55bb1c506920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:49.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.884+0000 7fbaebfff700 1 --2- 172.21.15.37:0/3706491301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb1c4eb370 0x55bb1c506920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.884+0000 7fbaebfff700 1 -- 172.21.15.37:0/3706491301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb1c568df0 con 0x55bb1c4e5bb0 2024-06-10T01:01:49.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.884+0000 7fbaebfff700 1 --2- 172.21.15.37:0/3706491301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb1c4e5bb0 0x55bb1c4f81f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbadc00b2e0 tx=0x7fbadc009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ac68b5c56b601f3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:49.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.884+0000 7fbacb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:01:49.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.884+0000 7fbacb7fe700 1 -- 172.21.15.37:0/3706491301 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc003880 con 0x55bb1c4e5bb0 2024-06-10T01:01:49.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.885+0000 7fbafd065b80 1 -- 172.21.15.37:0/3706491301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbae409b550 con 0x55bb1c4e5bb0 2024-06-10T01:01:49.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.885+0000 7fbacb7fe700 1 -- 172.21.15.37:0/3706491301 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fbadc003a20 con 0x55bb1c4e5bb0 2024-06-10T01:01:49.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.885+0000 7fbacb7fe700 1 -- 172.21.15.37:0/3706491301 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbadc044050 con 0x55bb1c4e5bb0 2024-06-10T01:01:49.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.885+0000 7fbafd065b80 10 client.25721 fetch_fsmap learned FSMap version 27 2024-06-10T01:01:49.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.885+0000 7fbafd065b80 1 -- 172.21.15.37:0/3706491301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bb1c569f80 con 0x55bb1c4e5bb0 2024-06-10T01:01:49.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.885+0000 7fbacb7fe700 1 -- 172.21.15.37:0/3706491301 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae409b550 con 0x55bb1c4e5bb0 2024-06-10T01:01:49.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.885+0000 7fbacb7fe700 1 -- 172.21.15.37:0/3706491301 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbadc03e4d0 con 0x55bb1c4e5bb0 2024-06-10T01:01:49.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.885+0000 7fbafd065b80 10 client.25721 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:01:49.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.885+0000 7fbafd065b80 10 client.25721 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:01:49.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.885+0000 7fbafd065b80 20 client.25721 populate_metadata read hostname 'smithi037' 2024-06-10T01:01:49.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.885+0000 7fbafd065b80 1 --2- 172.21.15.37:0/3706491301 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bb1c56a640 0x55bb1c56ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:49.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.885+0000 7fbafd065b80 4 client.25721 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:01:49.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.885+0000 7fbafd065b80 1 -- 172.21.15.37:0/3706491301 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb1c43a2d0 con 0x55bb1c56a640 2024-06-10T01:01:49.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.886+0000 7fbaeb7fe700 1 --2- 172.21.15.37:0/3706491301 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bb1c56a640 0x55bb1c56ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:49.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.886+0000 7fbacb7fe700 10 client.25721 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:49.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.886+0000 7fbaeb7fe700 1 --2- 172.21.15.37:0/3706491301 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bb1c56a640 0x55bb1c56ca60 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4aefbf98588ed45 server_cookie=6433999b5e8666d4 in_seq=0 out_seq=0 2024-06-10T01:01:49.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.889+0000 7fbacb7fe700 1 -- 172.21.15.37:0/3706491301 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb1c43a2d0 con 0x55bb1c56a640 2024-06-10T01:01:49.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.889+0000 7fbacb7fe700 10 client.25721 handle_command_reply: tid=0 2024-06-10T01:01:49.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.889+0000 7fbafd065b80 1 client.25721 shutdown 2024-06-10T01:01:49.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.890+0000 7fbafd065b80 1 -- 172.21.15.37:0/3706491301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb1c4e5bb0 msgr2=0x55bb1c4f81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:49.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.890+0000 7fbafd065b80 1 --2- 172.21.15.37:0/3706491301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb1c4e5bb0 0x55bb1c4f81f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbadc00b2e0 tx=0x7fbadc009ca0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.890+0000 7fbafd065b80 1 -- 172.21.15.37:0/3706491301 shutdown_connections 2024-06-10T01:01:49.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.890+0000 7fbafd065b80 1 --2- 172.21.15.37:0/3706491301 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bb1c56a640 0x55bb1c56ca60 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.890+0000 7fbafd065b80 1 --2- 172.21.15.37:0/3706491301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb1c4ebc90 0x55bb1c507060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.890+0000 7fbafd065b80 1 --2- 172.21.15.37:0/3706491301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb1c4e5bb0 0x55bb1c4f81f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.890+0000 7fbafd065b80 1 --2- 172.21.15.37:0/3706491301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb1c4eb370 0x55bb1c506920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.890+0000 7fbafd065b80 1 -- 172.21.15.37:0/3706491301 >> 172.21.15.37:0/3706491301 conn(0x55bb1c40e080 msgr2=0x55bb1c4e0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:49.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.890+0000 7fbacb7fe700 0 client.25721 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:49.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.890+0000 7fbafd065b80 1 -- 172.21.15.37:0/3706491301 shutdown_connections 2024-06-10T01:01:49.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.890+0000 7fbafd065b80 1 -- 172.21.15.37:0/3706491301 wait complete. 2024-06-10T01:01:49.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.890+0000 7fbafd065b80 20 client.25721 trim_cache size 0 max 16384 2024-06-10T01:01:49.889 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T01:01:49.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.890+0000 7fbaf27e2700 1 -- 172.21.15.37:0/2470345360 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fbad40a2ca0 msgr2=0x7fbad40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:49.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.890+0000 7fbaf27e2700 1 --2- 172.21.15.37:0/2470345360 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fbad40a2ca0 0x7fbad40a5160 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbaec199b80 tx=0x7fbadc01a040 comp rx=0 tx=0).stop 2024-06-10T01:01:49.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.890+0000 7fbaf27e2700 1 -- 172.21.15.37:0/2470345360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbaec106b90 msgr2=0x7fbaec198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:49.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.890+0000 7fbaf27e2700 1 --2- 172.21.15.37:0/2470345360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbaec106b90 0x7fbaec198a40 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fbae4009fa0 tx=0x7fbae40076c0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.891+0000 7fbaf27e2700 1 -- 172.21.15.37:0/2470345360 shutdown_connections 2024-06-10T01:01:49.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.891+0000 7fbaf27e2700 1 --2- 172.21.15.37:0/2470345360 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fbad40a2ca0 0x7fbad40a5160 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.891+0000 7fbaf27e2700 1 --2- 172.21.15.37:0/2470345360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbaec106b90 0x7fbaec198a40 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.891+0000 7fbaf27e2700 1 --2- 172.21.15.37:0/2470345360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaec100b50 0x7fbaec19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.891+0000 7fbaf27e2700 1 --2- 172.21.15.37:0/2470345360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbaec101510 0x7fbaec19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:49.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.891+0000 7fbaf27e2700 1 -- 172.21.15.37:0/2470345360 >> 172.21.15.37:0/2470345360 conn(0x7fbaec077a00 msgr2=0x7fbaec103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:49.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.891+0000 7fbaf27e2700 1 -- 172.21.15.37:0/2470345360 shutdown_connections 2024-06-10T01:01:49.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:49.891+0000 7fbaf27e2700 1 -- 172.21.15.37:0/2470345360 wait complete. 2024-06-10T01:01:50.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:50 smithi094 ceph-mon[36708]: pgmap v584: 129 pgs: 129 active+clean; 13 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 1001 KiB/s wr, 32 op/s 2024-06-10T01:01:50.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:50 smithi037 ceph-mon[32192]: pgmap v584: 129 pgs: 129 active+clean; 13 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 1001 KiB/s wr, 32 op/s 2024-06-10T01:01:50.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:50 smithi167 ceph-mon[36917]: pgmap v584: 129 pgs: 129 active+clean; 13 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 1001 KiB/s wr, 32 op/s 2024-06-10T01:01:51.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.573+0000 7f01a3de1700 1 -- 172.21.15.37:0/3284053600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f019c100a80 msgr2=0x7f019c100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:51.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.573+0000 7f01a3de1700 1 --2- 172.21.15.37:0/3284053600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f019c100a80 0x7f019c100f00 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f018c009990 tx=0x7f018c009ca0 comp rx=0 tx=0).stop 2024-06-10T01:01:51.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.573+0000 7f01a3de1700 1 -- 172.21.15.37:0/3284053600 shutdown_connections 2024-06-10T01:01:51.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.573+0000 7f01a3de1700 1 --2- 172.21.15.37:0/3284053600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f019c101520 0x7f019c111210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:51.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.573+0000 7f01a3de1700 1 --2- 172.21.15.37:0/3284053600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f019c1033a0 0x7f019c103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:51.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.573+0000 7f01a3de1700 1 --2- 172.21.15.37:0/3284053600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f019c100a80 0x7f019c100f00 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:51.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.573+0000 7f01a3de1700 1 -- 172.21.15.37:0/3284053600 >> 172.21.15.37:0/3284053600 conn(0x7f019c077a00 msgr2=0x7f019c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:51.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.573+0000 7f01a3de1700 1 -- 172.21.15.37:0/3284053600 shutdown_connections 2024-06-10T01:01:51.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.573+0000 7f01a3de1700 1 -- 172.21.15.37:0/3284053600 wait complete. 2024-06-10T01:01:51.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.574+0000 7f01a3de1700 1 Processor -- start 2024-06-10T01:01:51.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.574+0000 7f01a3de1700 1 -- start start 2024-06-10T01:01:51.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.574+0000 7f01a3de1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f019c100a80 0x7f019c19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:51.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.574+0000 7f01a3de1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f019c101520 0x7f019c19b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:51.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.574+0000 7f01a3de1700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f019c1033a0 0x7f019c19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:51.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.574+0000 7f01a3de1700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f019c114980 con 0x7f019c101520 2024-06-10T01:01:51.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.574+0000 7f01a3de1700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f019c114800 con 0x7f019c100a80 2024-06-10T01:01:51.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.574+0000 7f01a3de1700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f019c114b00 con 0x7f019c1033a0 2024-06-10T01:01:51.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.574+0000 7f01a137c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f019c101520 0x7f019c19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:51.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.574+0000 7f01a1b7d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f019c100a80 0x7f019c19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:51.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.574+0000 7f01a237e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f019c1033a0 0x7f019c19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:51.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.574+0000 7f01a1b7d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f019c100a80 0x7f019c19ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:59538/0 (socket says 172.21.15.37:59538) 2024-06-10T01:01:51.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.574+0000 7f01a1b7d700 1 -- 172.21.15.37:0/2624440269 learned_addr learned my addr 172.21.15.37:0/2624440269 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:51.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.574+0000 7f01a137c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f019c101520 0x7f019c19b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52918/0 (socket says 172.21.15.37:52918) 2024-06-10T01:01:51.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.575+0000 7f01a237e700 1 -- 172.21.15.37:0/2624440269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f019c100a80 msgr2=0x7f019c19ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:51.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.575+0000 7f01a237e700 1 --2- 172.21.15.37:0/2624440269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f019c100a80 0x7f019c19ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:51.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.575+0000 7f01a237e700 1 -- 172.21.15.37:0/2624440269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f019c101520 msgr2=0x7f019c19b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:51.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.575+0000 7f01a237e700 1 --2- 172.21.15.37:0/2624440269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f019c101520 0x7f019c19b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:51.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.575+0000 7f01a237e700 1 -- 172.21.15.37:0/2624440269 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019c19fdf0 con 0x7f019c1033a0 2024-06-10T01:01:51.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.575+0000 7f01a1b7d700 1 --2- 172.21.15.37:0/2624440269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f019c100a80 0x7f019c19ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:01:51.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.575+0000 7f01a237e700 1 --2- 172.21.15.37:0/2624440269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f019c1033a0 0x7f019c19f6b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f019400efc0 tx=0x7f019400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:51.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.575+0000 7f0192ffd700 1 -- 172.21.15.37:0/2624440269 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01940092e0 con 0x7f019c1033a0 2024-06-10T01:01:51.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.576+0000 7f0192ffd700 1 -- 172.21.15.37:0/2624440269 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0194004bb0 con 0x7f019c1033a0 2024-06-10T01:01:51.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.576+0000 7f0192ffd700 1 -- 172.21.15.37:0/2624440269 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0194004280 con 0x7f019c1033a0 2024-06-10T01:01:51.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.576+0000 7f01a3de1700 1 -- 172.21.15.37:0/2624440269 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f019c1a0080 con 0x7f019c1033a0 2024-06-10T01:01:51.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.576+0000 7f01a3de1700 1 -- 172.21.15.37:0/2624440269 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f019c1a7960 con 0x7f019c1033a0 2024-06-10T01:01:51.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.579+0000 7f0192ffd700 1 -- 172.21.15.37:0/2624440269 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f019401b030 con 0x7f019c1033a0 2024-06-10T01:01:51.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.580+0000 7f0192ffd700 1 --2- 172.21.15.37:0/2624440269 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f01880a2d70 0x7f01880a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:51.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.580+0000 7f0192ffd700 1 -- 172.21.15.37:0/2624440269 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f01940cee50 con 0x7f019c1033a0 2024-06-10T01:01:51.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.580+0000 7f01a1b7d700 1 --2- 172.21.15.37:0/2624440269 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f01880a2d70 0x7f01880a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:51.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.580+0000 7f01a1b7d700 1 --2- 172.21.15.37:0/2624440269 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f01880a2d70 0x7f01880a5230 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0198005e90 tx=0x7f0198005d60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:51.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.588+0000 7f01a3de1700 1 -- 172.21.15.37:0/2624440269 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f019c101a90 con 0x7f019c1033a0 2024-06-10T01:01:51.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.593+0000 7f0192ffd700 1 -- 172.21.15.37:0/2624440269 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0194022020 con 0x7f019c1033a0 2024-06-10T01:01:51.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.855+0000 7f01a3de1700 1 -- 172.21.15.37:0/2624440269 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f019c19c140 con 0x7f019c1033a0 2024-06-10T01:01:51.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.856+0000 7f0192ffd700 1 -- 172.21.15.37:0/2624440269 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11867 (secure 0 0 0) 0x7f0194010040 con 0x7f019c1033a0 2024-06-10T01:01:51.855 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:01:51.855 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T00:56:30.291171+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24451,24458,24472]},"id":1}]} 2024-06-10T01:01:51.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.859+0000 7f01a3de1700 1 -- 172.21.15.37:0/2624440269 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f01880a2d70 msgr2=0x7f01880a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:51.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.859+0000 7f01a3de1700 1 --2- 172.21.15.37:0/2624440269 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f01880a2d70 0x7f01880a5230 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0198005e90 tx=0x7f0198005d60 comp rx=0 tx=0).stop 2024-06-10T01:01:51.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.859+0000 7f01a3de1700 1 -- 172.21.15.37:0/2624440269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f019c1033a0 msgr2=0x7f019c19f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:51.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.859+0000 7f01a3de1700 1 --2- 172.21.15.37:0/2624440269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f019c1033a0 0x7f019c19f6b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f019400efc0 tx=0x7f019400c3a0 comp rx=0 tx=0).stop 2024-06-10T01:01:51.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.859+0000 7f01a3de1700 1 -- 172.21.15.37:0/2624440269 shutdown_connections 2024-06-10T01:01:51.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.859+0000 7f01a3de1700 1 --2- 172.21.15.37:0/2624440269 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f01880a2d70 0x7f01880a5230 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:51.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.859+0000 7f01a3de1700 1 --2- 172.21.15.37:0/2624440269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f019c1033a0 0x7f019c19f6b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:51.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.859+0000 7f01a3de1700 1 --2- 172.21.15.37:0/2624440269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f019c100a80 0x7f019c19ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:51.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.859+0000 7f01a3de1700 1 --2- 172.21.15.37:0/2624440269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f019c101520 0x7f019c19b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:51.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.859+0000 7f01a3de1700 1 -- 172.21.15.37:0/2624440269 >> 172.21.15.37:0/2624440269 conn(0x7f019c077a00 msgr2=0x7f019c0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:51.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.859+0000 7f01a3de1700 1 -- 172.21.15.37:0/2624440269 shutdown_connections 2024-06-10T01:01:51.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:51.859+0000 7f01a3de1700 1 -- 172.21.15.37:0/2624440269 wait complete. 2024-06-10T01:01:51.858 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-06-10T01:01:52.111 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:52 smithi037 ceph-mon[32192]: pgmap v585: 129 pgs: 129 active+clean; 17 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 39 op/s 2024-06-10T01:01:52.111 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:52 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2624440269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:01:52.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:52 smithi094 ceph-mon[36708]: pgmap v585: 129 pgs: 129 active+clean; 17 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 39 op/s 2024-06-10T01:01:52.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:52 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2624440269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:01:52.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:52 smithi167 ceph-mon[36917]: pgmap v585: 129 pgs: 129 active+clean; 17 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 39 op/s 2024-06-10T01:01:52.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:52 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2624440269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:01:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.763+0000 7f1761288640 1 -- 172.21.15.37:0/870765499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f175c1024c0 msgr2=0x7f175c1048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.763+0000 7f1761288640 1 --2- 172.21.15.37:0/870765499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f175c1024c0 0x7f175c1048b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f174c009900 tx=0x7f174c02a410 comp rx=0 tx=0).stop 2024-06-10T01:01:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.763+0000 7f1761288640 1 -- 172.21.15.37:0/870765499 shutdown_connections 2024-06-10T01:01:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.763+0000 7f1761288640 1 --2- 172.21.15.37:0/870765499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f175c107720 0x7f175c109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.763+0000 7f1761288640 1 --2- 172.21.15.37:0/870765499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f175c104df0 0x7f175c1071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.763+0000 7f1761288640 1 --2- 172.21.15.37:0/870765499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f175c1024c0 0x7f175c1048b0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.763+0000 7f1761288640 1 -- 172.21.15.37:0/870765499 >> 172.21.15.37:0/870765499 conn(0x7f175c0fbeb0 msgr2=0x7f175c0fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.763+0000 7f1761288640 1 -- 172.21.15.37:0/870765499 shutdown_connections 2024-06-10T01:01:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.763+0000 7f1761288640 1 -- 172.21.15.37:0/870765499 wait complete. 2024-06-10T01:01:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.764+0000 7f1761288640 1 Processor -- start 2024-06-10T01:01:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.764+0000 7f1761288640 1 -- start start 2024-06-10T01:01:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.765+0000 7f1761288640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f175c1024c0 0x7f175c197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.765+0000 7f1761288640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f175c104df0 0x7f175c1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.765+0000 7f1761288640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f175c107720 0x7f175c19f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.765+0000 7f1761288640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f175c10d6c0 con 0x7f175c1024c0 2024-06-10T01:01:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.765+0000 7f1761288640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f175c10d540 con 0x7f175c104df0 2024-06-10T01:01:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.765+0000 7f1761288640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f175c10d840 con 0x7f175c107720 2024-06-10T01:01:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.765+0000 7f175affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f175c1024c0 0x7f175c197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.765+0000 7f175affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f175c1024c0 0x7f175c197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47630/0 (socket says 172.21.15.37:47630) 2024-06-10T01:01:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.765+0000 7f175affd640 1 -- 172.21.15.37:0/1824801421 learned_addr learned my addr 172.21.15.37:0/1824801421 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.765+0000 7f175b7fe640 1 --2- 172.21.15.37:0/1824801421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f175c107720 0x7f175c19f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.765+0000 7f175a7fc640 1 --2- 172.21.15.37:0/1824801421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f175c104df0 0x7f175c1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.765+0000 7f175affd640 1 -- 172.21.15.37:0/1824801421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f175c107720 msgr2=0x7f175c19f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.765+0000 7f175affd640 1 --2- 172.21.15.37:0/1824801421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f175c107720 0x7f175c19f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.765+0000 7f175affd640 1 -- 172.21.15.37:0/1824801421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f175c104df0 msgr2=0x7f175c1981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.765+0000 7f175affd640 1 --2- 172.21.15.37:0/1824801421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f175c104df0 0x7f175c1981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.765+0000 7f175affd640 1 -- 172.21.15.37:0/1824801421 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f175c19f8c0 con 0x7f175c1024c0 2024-06-10T01:01:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.765+0000 7f175b7fe640 1 --2- 172.21.15.37:0/1824801421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f175c107720 0x7f175c19f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:01:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.765+0000 7f175affd640 1 --2- 172.21.15.37:0/1824801421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f175c1024c0 0x7f175c197c80 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f174c009230 tx=0x7f174c031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.765+0000 7f1743fff640 1 -- 172.21.15.37:0/1824801421 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c03f030 con 0x7f175c1024c0 2024-06-10T01:01:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.766+0000 7f1743fff640 1 -- 172.21.15.37:0/1824801421 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f174c003ad0 con 0x7f175c1024c0 2024-06-10T01:01:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.766+0000 7f1761288640 1 -- 172.21.15.37:0/1824801421 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f175c19fb50 con 0x7f175c1024c0 2024-06-10T01:01:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.766+0000 7f1743fff640 1 -- 172.21.15.37:0/1824801421 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c035070 con 0x7f175c1024c0 2024-06-10T01:01:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.766+0000 7f1761288640 1 -- 172.21.15.37:0/1824801421 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f175c1a0090 con 0x7f175c1024c0 2024-06-10T01:01:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.769+0000 7f1743fff640 1 -- 172.21.15.37:0/1824801421 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f174c042030 con 0x7f175c1024c0 2024-06-10T01:01:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.769+0000 7f1763f92740 1 Processor -- start 2024-06-10T01:01:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.769+0000 7f1763f92740 1 -- start start 2024-06-10T01:01:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.769+0000 7f1763f92740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652cf1ba420 0x5652cf1ba840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.769+0000 7f1763f92740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652cf1bae10 0x5652cf1bf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.769+0000 7f1763f92740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652cf1bff80 0x5652cf1c2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.769+0000 7f1763f92740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5652cf1194f0 con 0x5652cf1bff80 2024-06-10T01:01:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.769+0000 7f1763f92740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5652cf0b7870 con 0x5652cf1ba420 2024-06-10T01:01:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.769+0000 7f1763f92740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5652cf1154b0 con 0x5652cf1bae10 2024-06-10T01:01:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.769+0000 7f1743fff640 1 --2- 172.21.15.37:0/1824801421 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f17440a2cb0 0x7f17440a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.769+0000 7f175affd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652cf1bae10 0x5652cf1bf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.769+0000 7f175a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652cf1bff80 0x5652cf1c2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.769+0000 7f175affd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652cf1bae10 0x5652cf1bf910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:44258/0 (socket says 172.21.15.37:44258) 2024-06-10T01:01:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.770+0000 7f175affd640 1 -- 172.21.15.37:0/1268891874 learned_addr learned my addr 172.21.15.37:0/1268891874 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.770+0000 7f175b7fe640 1 --2- 172.21.15.37:0/1268891874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652cf1ba420 0x5652cf1ba840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.770+0000 7f175b7fe640 1 --2- 172.21.15.37:0/1824801421 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f17440a2cb0 0x7f17440a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.770+0000 7f1743fff640 1 -- 172.21.15.37:0/1824801421 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f174c0ef890 con 0x7f175c1024c0 2024-06-10T01:01:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.770+0000 7f175affd640 1 -- 172.21.15.37:0/1268891874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652cf1ba420 msgr2=0x5652cf1ba840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.770+0000 7f175affd640 1 --2- 172.21.15.37:0/1268891874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652cf1ba420 0x5652cf1ba840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.770+0000 7f175affd640 1 -- 172.21.15.37:0/1268891874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652cf1bff80 msgr2=0x5652cf1c2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.770+0000 7f175affd640 1 --2- 172.21.15.37:0/1268891874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652cf1bff80 0x5652cf1c2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.770+0000 7f175affd640 1 -- 172.21.15.37:0/1268891874 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652cf0d39b0 con 0x5652cf1bae10 2024-06-10T01:01:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.770+0000 7f175b7fe640 1 --2- 172.21.15.37:0/1268891874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652cf1ba420 0x5652cf1ba840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:01:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.770+0000 7f175affd640 1 --2- 172.21.15.37:0/1268891874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652cf1bae10 0x5652cf1bf910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f174c0ecf40 tx=0x7f174c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af37a5791fc580e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.770+0000 7f175b7fe640 1 --2- 172.21.15.37:0/1824801421 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f17440a2cb0 0x7f17440a5170 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x5652cf1bb670 tx=0x7f1754009600 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.770+0000 7f17417fa640 1 -- 172.21.15.37:0/1268891874 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c049070 con 0x5652cf1bae10 2024-06-10T01:01:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.771+0000 7f17417fa640 1 -- 172.21.15.37:0/1268891874 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f174c0bbdb0 con 0x5652cf1bae10 2024-06-10T01:01:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.771+0000 7f17417fa640 1 -- 172.21.15.37:0/1268891874 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c0b85c0 con 0x5652cf1bae10 2024-06-10T01:01:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.772+0000 7f1763f92740 1 -- 172.21.15.37:0/1268891874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652cf1bae10 msgr2=0x5652cf1bf910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.772+0000 7f1763f92740 1 --2- 172.21.15.37:0/1268891874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652cf1bae10 0x5652cf1bf910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f174c0ecf40 tx=0x7f174c009260 comp rx=0 tx=0).stop 2024-06-10T01:01:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.772+0000 7f1763f92740 1 -- 172.21.15.37:0/1268891874 shutdown_connections 2024-06-10T01:01:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.772+0000 7f1763f92740 1 --2- 172.21.15.37:0/1268891874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652cf1bff80 0x5652cf1c2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.772+0000 7f1763f92740 1 --2- 172.21.15.37:0/1268891874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652cf1bae10 0x5652cf1bf910 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.772+0000 7f1763f92740 1 --2- 172.21.15.37:0/1268891874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652cf1ba420 0x5652cf1ba840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.772+0000 7f1763f92740 1 -- 172.21.15.37:0/1268891874 >> 172.21.15.37:0/1268891874 conn(0x5652cf1b1680 msgr2=0x5652cf1b1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.772+0000 7f1763f92740 1 -- 172.21.15.37:0/1268891874 shutdown_connections 2024-06-10T01:01:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.772+0000 7f1763f92740 1 -- 172.21.15.37:0/1268891874 wait complete. 2024-06-10T01:01:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.772+0000 7f1763f92740 1 -- start start 2024-06-10T01:01:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.772+0000 7f1763f92740 1 Processor -- start 2024-06-10T01:01:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.772+0000 7f1763f92740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652cf1ba420 0x5652cf1c4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.772+0000 7f1763f92740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652cf1bae10 0x5652cf1c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.772+0000 7f1763f92740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652cf1bff80 0x5652cf1cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.772+0000 7f1763f92740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5652cf1c2cf0 con 0x5652cf1bae10 2024-06-10T01:01:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.772+0000 7f1763f92740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5652cf0a2ae0 con 0x5652cf1bff80 2024-06-10T01:01:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.772+0000 7f1763f92740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5652cf1c2e70 con 0x5652cf1ba420 2024-06-10T01:01:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.772+0000 7f1763f92740 10 client.0 _ll_register_callbacks cb 0x5652cf0db9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:01:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.772+0000 7f175a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652cf1bae10 0x5652cf1c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.773+0000 7f175b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652cf1bff80 0x5652cf1cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.772+0000 7f175affd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652cf1ba420 0x5652cf1c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.773+0000 7f175a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652cf1bae10 0x5652cf1c48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47648/0 (socket says 172.21.15.37:47648) 2024-06-10T01:01:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.773+0000 7f175b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652cf1bff80 0x5652cf1cb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:51320/0 (socket says 172.21.15.37:51320) 2024-06-10T01:01:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.773+0000 7f175affd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652cf1ba420 0x5652cf1c4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:44272/0 (socket says 172.21.15.37:44272) 2024-06-10T01:01:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.773+0000 7f175a7fc640 1 -- 172.21.15.37:0/3807958514 learned_addr learned my addr 172.21.15.37:0/3807958514 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.773+0000 7f175a7fc640 1 -- 172.21.15.37:0/3807958514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652cf1ba420 msgr2=0x5652cf1c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.773+0000 7f175a7fc640 1 --2- 172.21.15.37:0/3807958514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652cf1ba420 0x5652cf1c4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.773+0000 7f175a7fc640 1 -- 172.21.15.37:0/3807958514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652cf1bff80 msgr2=0x5652cf1cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.773+0000 7f175a7fc640 1 --2- 172.21.15.37:0/3807958514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652cf1bff80 0x5652cf1cb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.773+0000 7f175a7fc640 1 -- 172.21.15.37:0/3807958514 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652cee2d210 con 0x5652cf1bae10 2024-06-10T01:01:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.773+0000 7f175b7fe640 1 --2- 172.21.15.37:0/3807958514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652cf1bff80 0x5652cf1cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:01:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.773+0000 7f175a7fc640 1 --2- 172.21.15.37:0/3807958514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652cf1bae10 0x5652cf1c48a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f17500126e0 tx=0x7f1750012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676e7e7570a860ce server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.773+0000 7f173eff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:01:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.773+0000 7f173eff5640 1 -- 172.21.15.37:0/3807958514 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1750012190 con 0x5652cf1bae10 2024-06-10T01:01:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.773+0000 7f173eff5640 1 -- 172.21.15.37:0/3807958514 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f175000f8c0 con 0x5652cf1bae10 2024-06-10T01:01:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.773+0000 7f1763f92740 1 -- 172.21.15.37:0/3807958514 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f174c0bbdb0 con 0x5652cf1bae10 2024-06-10T01:01:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.774+0000 7f173eff5640 1 -- 172.21.15.37:0/3807958514 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f175000fac0 con 0x5652cf1bae10 2024-06-10T01:01:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.774+0000 7f1763f92740 10 client.16245 fetch_fsmap learned FSMap version 27 2024-06-10T01:01:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.774+0000 7f1763f92740 1 -- 172.21.15.37:0/3807958514 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5652ceef9620 con 0x5652cf1bae10 2024-06-10T01:01:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.774+0000 7f173eff5640 1 -- 172.21.15.37:0/3807958514 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f174c0bbdb0 con 0x5652cf1bae10 2024-06-10T01:01:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.774+0000 7f173eff5640 1 -- 172.21.15.37:0/3807958514 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f175001c420 con 0x5652cf1bae10 2024-06-10T01:01:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.774+0000 7f1763f92740 10 client.16245 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:01:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.774+0000 7f1763f92740 10 client.16245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:01:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.774+0000 7f1763f92740 20 client.16245 populate_metadata read hostname 'smithi037' 2024-06-10T01:01:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.774+0000 7f1763f92740 1 --2- 172.21.15.37:0/3807958514 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5652cf1d02e0 0x5652cf1d2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.774+0000 7f1763f92740 4 client.16245 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:01:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.774+0000 7f1763f92740 1 -- 172.21.15.37:0/3807958514 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652cf12cf10 con 0x5652cf1d02e0 2024-06-10T01:01:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.774+0000 7f175affd640 1 --2- 172.21.15.37:0/3807958514 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5652cf1d02e0 0x5652cf1d2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.775+0000 7f175affd640 1 --2- 172.21.15.37:0/3807958514 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5652cf1d02e0 0x5652cf1d2760 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbcb816c0c110ec0 server_cookie=464004ca90f650b1 in_seq=0 out_seq=0 2024-06-10T01:01:52.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.775+0000 7f173eff5640 10 client.16245 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:52.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.777+0000 7f173eff5640 1 -- 172.21.15.37:0/3807958514 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5652cf12cf10 con 0x5652cf1d02e0 2024-06-10T01:01:52.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.777+0000 7f173eff5640 10 client.16245 handle_command_reply: tid=0 2024-06-10T01:01:52.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.777+0000 7f1763f92740 1 client.16245 shutdown 2024-06-10T01:01:52.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.777+0000 7f1763f92740 1 -- 172.21.15.37:0/3807958514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652cf1bae10 msgr2=0x5652cf1c48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:52.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.777+0000 7f1763f92740 1 --2- 172.21.15.37:0/3807958514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652cf1bae10 0x5652cf1c48a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f17500126e0 tx=0x7f1750012ba0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.777+0000 7f1763f92740 1 -- 172.21.15.37:0/3807958514 shutdown_connections 2024-06-10T01:01:52.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.777+0000 7f1763f92740 1 --2- 172.21.15.37:0/3807958514 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5652cf1d02e0 0x5652cf1d2760 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.777+0000 7f1763f92740 1 --2- 172.21.15.37:0/3807958514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652cf1bff80 0x5652cf1cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.777+0000 7f173eff5640 0 client.16245 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:52.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.777+0000 7f1763f92740 1 --2- 172.21.15.37:0/3807958514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652cf1bae10 0x5652cf1c48a0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.777+0000 7f1763f92740 1 --2- 172.21.15.37:0/3807958514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652cf1ba420 0x5652cf1c4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.777+0000 7f1763f92740 1 -- 172.21.15.37:0/3807958514 >> 172.21.15.37:0/3807958514 conn(0x5652cf1b1680 msgr2=0x5652cf1b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:52.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.777+0000 7f1763f92740 1 -- 172.21.15.37:0/3807958514 shutdown_connections 2024-06-10T01:01:52.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.777+0000 7f1763f92740 1 -- 172.21.15.37:0/3807958514 wait complete. 2024-06-10T01:01:52.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.777+0000 7f1763f92740 20 client.16245 trim_cache size 0 max 16384 2024-06-10T01:01:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.794+0000 7f1763f92740 1 Processor -- start 2024-06-10T01:01:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.794+0000 7f1763f92740 1 -- start start 2024-06-10T01:01:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.795+0000 7f1763f92740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652cf1ba420 0x5652cf1c4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.795+0000 7f1763f92740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652cf1bae10 0x5652cf1c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.795+0000 7f1763f92740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652cf1bff80 0x5652cf1cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.795+0000 7f1763f92740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f175c1aae40 con 0x5652cf1bae10 2024-06-10T01:01:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.795+0000 7f1763f92740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f175c1a8710 con 0x5652cf1bff80 2024-06-10T01:01:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.795+0000 7f1763f92740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f175c1a6560 con 0x5652cf1ba420 2024-06-10T01:01:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.795+0000 7f175a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652cf1bae10 0x5652cf1c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.795+0000 7f175a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652cf1bae10 0x5652cf1c48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47658/0 (socket says 172.21.15.37:47658) 2024-06-10T01:01:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.795+0000 7f175a7fc640 1 -- 172.21.15.37:0/1785519634 learned_addr learned my addr 172.21.15.37:0/1785519634 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.795+0000 7f175affd640 1 --2- 172.21.15.37:0/1785519634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652cf1ba420 0x5652cf1c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.795+0000 7f175b7fe640 1 --2- 172.21.15.37:0/1785519634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652cf1bff80 0x5652cf1cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.795+0000 7f175a7fc640 1 -- 172.21.15.37:0/1785519634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652cf1ba420 msgr2=0x5652cf1c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.795+0000 7f175a7fc640 1 --2- 172.21.15.37:0/1785519634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652cf1ba420 0x5652cf1c4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.795+0000 7f175a7fc640 1 -- 172.21.15.37:0/1785519634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652cf1bff80 msgr2=0x5652cf1cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.795+0000 7f175a7fc640 1 --2- 172.21.15.37:0/1785519634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652cf1bff80 0x5652cf1cb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.795+0000 7f175a7fc640 1 -- 172.21.15.37:0/1785519634 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652cf1c5200 con 0x5652cf1bae10 2024-06-10T01:01:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.795+0000 7f175a7fc640 1 --2- 172.21.15.37:0/1785519634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652cf1bae10 0x5652cf1c48a0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1750009e00 tx=0x7f17500123a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537ab35f6b431455 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.795+0000 7f1740ff9640 1 -- 172.21.15.37:0/1785519634 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175000f850 con 0x5652cf1bae10 2024-06-10T01:01:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.796+0000 7f1740ff9640 1 -- 172.21.15.37:0/1785519634 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17500055c0 con 0x5652cf1bae10 2024-06-10T01:01:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.796+0000 7f1740ff9640 1 -- 172.21.15.37:0/1785519634 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175001c680 con 0x5652cf1bae10 2024-06-10T01:01:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.796+0000 7f1763f92740 1 -- 172.21.15.37:0/1785519634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652cf1bae10 msgr2=0x5652cf1c48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.796+0000 7f1763f92740 1 --2- 172.21.15.37:0/1785519634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652cf1bae10 0x5652cf1c48a0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1750009e00 tx=0x7f17500123a0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.796+0000 7f1763f92740 1 -- 172.21.15.37:0/1785519634 shutdown_connections 2024-06-10T01:01:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.796+0000 7f1763f92740 1 --2- 172.21.15.37:0/1785519634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652cf1bff80 0x5652cf1cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.796+0000 7f1763f92740 1 --2- 172.21.15.37:0/1785519634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652cf1bae10 0x5652cf1c48a0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.796+0000 7f1763f92740 1 --2- 172.21.15.37:0/1785519634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652cf1ba420 0x5652cf1c4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.796+0000 7f1763f92740 1 -- 172.21.15.37:0/1785519634 >> 172.21.15.37:0/1785519634 conn(0x5652cf1b1680 msgr2=0x5652cf1b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.796+0000 7f1763f92740 1 -- 172.21.15.37:0/1785519634 shutdown_connections 2024-06-10T01:01:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.796+0000 7f1763f92740 1 -- 172.21.15.37:0/1785519634 wait complete. 2024-06-10T01:01:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.797+0000 7f1763f92740 1 -- start start 2024-06-10T01:01:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.797+0000 7f1763f92740 1 Processor -- start 2024-06-10T01:01:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.797+0000 7f1763f92740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652cf1ba420 0x5652cf1c8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.797+0000 7f1763f92740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652cf1bae10 0x5652cf1c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.797+0000 7f1763f92740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652cf1bff80 0x5652cf239030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.797+0000 7f1763f92740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f175c1a6bd0 con 0x5652cf1ba420 2024-06-10T01:01:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.797+0000 7f1763f92740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f175c1a8040 con 0x5652cf1bae10 2024-06-10T01:01:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.797+0000 7f1763f92740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f175c1a72a0 con 0x5652cf1bff80 2024-06-10T01:01:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.797+0000 7f1763f92740 10 client.0 _ll_register_callbacks cb 0x7f1750009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:01:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.797+0000 7f175affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652cf1ba420 0x5652cf1c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.797+0000 7f175affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652cf1ba420 0x5652cf1c8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47674/0 (socket says 172.21.15.37:47674) 2024-06-10T01:01:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.797+0000 7f175affd640 1 -- 172.21.15.37:0/2243885377 learned_addr learned my addr 172.21.15.37:0/2243885377 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.797+0000 7f175b7fe640 1 --2- 172.21.15.37:0/2243885377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652cf1bff80 0x5652cf239030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.797+0000 7f175a7fc640 1 --2- 172.21.15.37:0/2243885377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652cf1bae10 0x5652cf1c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.797+0000 7f175affd640 1 -- 172.21.15.37:0/2243885377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652cf1bff80 msgr2=0x5652cf239030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.797+0000 7f175affd640 1 --2- 172.21.15.37:0/2243885377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652cf1bff80 0x5652cf239030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.797+0000 7f175affd640 1 -- 172.21.15.37:0/2243885377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652cf1bae10 msgr2=0x5652cf1c8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.797+0000 7f175affd640 1 --2- 172.21.15.37:0/2243885377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652cf1bae10 0x5652cf1c8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.797+0000 7f175affd640 1 -- 172.21.15.37:0/2243885377 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652cf1b14b0 con 0x5652cf1ba420 2024-06-10T01:01:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.798+0000 7f173eff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:01:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.798+0000 7f175affd640 1 --2- 172.21.15.37:0/2243885377 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652cf1ba420 0x5652cf1c8520 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f174c0b8210 tx=0x7f174c0c4af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b98a1e9151e956 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.798+0000 7f173eff5640 1 -- 172.21.15.37:0/2243885377 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c049040 con 0x5652cf1ba420 2024-06-10T01:01:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.798+0000 7f173eff5640 1 -- 172.21.15.37:0/2243885377 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f174c003c80 con 0x5652cf1ba420 2024-06-10T01:01:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.798+0000 7f1763f92740 1 -- 172.21.15.37:0/2243885377 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17500055c0 con 0x5652cf1ba420 2024-06-10T01:01:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.798+0000 7f173eff5640 1 -- 172.21.15.37:0/2243885377 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f174c0bbb80 con 0x5652cf1ba420 2024-06-10T01:01:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.798+0000 7f1763f92740 10 client.16251 fetch_fsmap learned FSMap version 27 2024-06-10T01:01:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.798+0000 7f1763f92740 1 -- 172.21.15.37:0/2243885377 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5652cf23a3f0 con 0x5652cf1ba420 2024-06-10T01:01:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.798+0000 7f173eff5640 1 -- 172.21.15.37:0/2243885377 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17500055c0 con 0x5652cf1ba420 2024-06-10T01:01:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.798+0000 7f173eff5640 1 -- 172.21.15.37:0/2243885377 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f174c0bc010 con 0x5652cf1ba420 2024-06-10T01:01:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.798+0000 7f1763f92740 10 client.16251 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:01:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.798+0000 7f1763f92740 10 client.16251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:01:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.798+0000 7f1763f92740 20 client.16251 populate_metadata read hostname 'smithi037' 2024-06-10T01:01:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.798+0000 7f1763f92740 1 --2- 172.21.15.37:0/2243885377 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5652cf1d02e0 0x5652cf23ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.798+0000 7f1763f92740 4 client.16251 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:01:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.798+0000 7f1763f92740 1 -- 172.21.15.37:0/2243885377 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5652cf126c20 con 0x5652cf1d02e0 2024-06-10T01:01:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.799+0000 7f175a7fc640 1 --2- 172.21.15.37:0/2243885377 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5652cf1d02e0 0x5652cf23ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.799+0000 7f173eff5640 10 client.16251 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:52.799+0000 7f175a7fc640 1 --2- 172.21.15.37:0/2243885377 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5652cf1d02e0 0x5652cf23ca30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53efb8c96edbe371 server_cookie=92826b1a1683a557 in_seq=0 out_seq=0 2024-06-10T01:01:52.910 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24472 quiesce db / --timeout 90 --expiration 138.2 --await-for 45.0' 2024-06-10T01:01:52.910 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24472 quiesce db / --timeout 90 --expiration 138.2 --await-for 45.0 2024-06-10T01:01:53.139 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:01:53.140 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:01:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:01:54.356 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:54 smithi037 ceph-mon[32192]: pgmap v586: 129 pgs: 129 active+clean; 21 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 69 op/s 2024-06-10T01:01:54.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:54 smithi094 ceph-mon[36708]: pgmap v586: 129 pgs: 129 active+clean; 21 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 69 op/s 2024-06-10T01:01:54.541 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:01:54.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:54 smithi167 ceph-mon[36917]: pgmap v586: 129 pgs: 129 active+clean; 21 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 69 op/s 2024-06-10T01:01:55.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.418+0000 7efd9c558700 1 -- 172.21.15.37:0/3482343012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd94108aa0 msgr2=0x7efd94108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:55.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.418+0000 7efd9c558700 1 --2- 172.21.15.37:0/3482343012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd94108aa0 0x7efd94108f20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efd840098f0 tx=0x7efd84009c00 comp rx=0 tx=0).stop 2024-06-10T01:01:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.419+0000 7efd9c558700 1 -- 172.21.15.37:0/3482343012 shutdown_connections 2024-06-10T01:01:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.419+0000 7efd9c558700 1 --2- 172.21.15.37:0/3482343012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd94109460 0x7efd94115970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.420+0000 7efd9c558700 1 --2- 172.21.15.37:0/3482343012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd9410eaa0 0x7efd9410ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.420+0000 7efd9c558700 1 --2- 172.21.15.37:0/3482343012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd94108aa0 0x7efd94108f20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.420+0000 7efd9c558700 1 -- 172.21.15.37:0/3482343012 >> 172.21.15.37:0/3482343012 conn(0x7efd94075c60 msgr2=0x7efd94078080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:55.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.421+0000 7efd9c558700 1 -- 172.21.15.37:0/3482343012 shutdown_connections 2024-06-10T01:01:55.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.421+0000 7efd9c558700 1 -- 172.21.15.37:0/3482343012 wait complete. 2024-06-10T01:01:55.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.423+0000 7efd9c558700 1 Processor -- start 2024-06-10T01:01:55.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.423+0000 7efd9c558700 1 -- start start 2024-06-10T01:01:55.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.424+0000 7efd9c558700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd94108aa0 0x7efd9419f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:55.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.424+0000 7efd9c558700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd94109460 0x7efd9419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:55.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.424+0000 7efd9a2f4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd94108aa0 0x7efd9419f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:55.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.424+0000 7efd9a2f4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd94108aa0 0x7efd9419f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:51352/0 (socket says 172.21.15.37:51352) 2024-06-10T01:01:55.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.424+0000 7efd9a2f4700 1 -- 172.21.15.37:0/615565899 learned_addr learned my addr 172.21.15.37:0/615565899 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:55.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.424+0000 7efd9c558700 1 --2- 172.21.15.37:0/615565899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd9410eaa0 0x7efd941a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:55.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.424+0000 7efd9c558700 1 -- 172.21.15.37:0/615565899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efd94118ed0 con 0x7efd94109460 2024-06-10T01:01:55.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.424+0000 7efd9c558700 1 -- 172.21.15.37:0/615565899 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efd94118d50 con 0x7efd94108aa0 2024-06-10T01:01:55.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.424+0000 7efd9aaf5700 1 --2- 172.21.15.37:0/615565899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd9410eaa0 0x7efd941a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:55.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.425+0000 7efd9c558700 1 -- 172.21.15.37:0/615565899 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efd94119050 con 0x7efd9410eaa0 2024-06-10T01:01:55.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.426+0000 7efd99af3700 1 --2- 172.21.15.37:0/615565899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd94109460 0x7efd9419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:55.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.428+0000 7efd99af3700 1 -- 172.21.15.37:0/615565899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd9410eaa0 msgr2=0x7efd941a3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:55.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.428+0000 7efd99af3700 1 --2- 172.21.15.37:0/615565899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd9410eaa0 0x7efd941a3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:55.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.428+0000 7efd99af3700 1 -- 172.21.15.37:0/615565899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd94108aa0 msgr2=0x7efd9419f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:55.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.428+0000 7efd99af3700 1 --2- 172.21.15.37:0/615565899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd94108aa0 0x7efd9419f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:55.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.428+0000 7efd99af3700 1 -- 172.21.15.37:0/615565899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd941a41e0 con 0x7efd94109460 2024-06-10T01:01:55.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.429+0000 7efd99af3700 1 --2- 172.21.15.37:0/615565899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd94109460 0x7efd9419f760 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7efd84005d90 tx=0x7efd840052c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:55.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.430+0000 7efd8b7fe700 1 -- 172.21.15.37:0/615565899 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8400b760 con 0x7efd94109460 2024-06-10T01:01:55.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.430+0000 7efd9c558700 1 -- 172.21.15.37:0/615565899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd941a4470 con 0x7efd94109460 2024-06-10T01:01:55.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.430+0000 7efd9c558700 1 -- 172.21.15.37:0/615565899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd941a4890 con 0x7efd94109460 2024-06-10T01:01:55.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.431+0000 7efd8b7fe700 1 -- 172.21.15.37:0/615565899 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd8400bde0 con 0x7efd94109460 2024-06-10T01:01:55.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.431+0000 7efd8b7fe700 1 -- 172.21.15.37:0/615565899 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd84026060 con 0x7efd94109460 2024-06-10T01:01:55.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.438+0000 7efd8b7fe700 1 -- 172.21.15.37:0/615565899 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7efd84017400 con 0x7efd94109460 2024-06-10T01:01:55.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.444+0000 7efda6ddbb80 1 Processor -- start 2024-06-10T01:01:55.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.444+0000 7efda6ddbb80 1 -- start start 2024-06-10T01:01:55.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.444+0000 7efda6ddbb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1f1257ce0 0x55b1f12580c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:55.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.445+0000 7efda6ddbb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1f1258600 0x55b1f1251f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:55.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.445+0000 7efd9aaf5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1f1257ce0 0x55b1f12580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:55.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.445+0000 7efd9aaf5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1f1257ce0 0x55b1f12580c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47700/0 (socket says 172.21.15.37:47700) 2024-06-10T01:01:55.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.445+0000 7efd9aaf5700 1 -- 172.21.15.37:0/1889587617 learned_addr learned my addr 172.21.15.37:0/1889587617 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:55.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.445+0000 7efda6ddbb80 1 --2- 172.21.15.37:0/1889587617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1f1252470 0x55b1f12528b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:55.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.445+0000 7efda6ddbb80 1 -- 172.21.15.37:0/1889587617 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b1f11929a0 con 0x55b1f1257ce0 2024-06-10T01:01:55.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.445+0000 7efda6ddbb80 1 -- 172.21.15.37:0/1889587617 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b1f113a8b0 con 0x55b1f1258600 2024-06-10T01:01:55.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.445+0000 7efda6ddbb80 1 -- 172.21.15.37:0/1889587617 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b1f118e960 con 0x55b1f1252470 2024-06-10T01:01:55.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.446+0000 7efd8b7fe700 1 --2- 172.21.15.37:0/615565899 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7efd800a2e10 0x7efd800a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:55.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.446+0000 7efd99af3700 1 --2- 172.21.15.37:0/1889587617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1f1252470 0x55b1f12528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:55.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.446+0000 7efd9a2f4700 1 --2- 172.21.15.37:0/1889587617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1f1258600 0x55b1f1251f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:55.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.447+0000 7efd9aaf5700 1 --2- 172.21.15.37:0/615565899 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7efd800a2e10 0x7efd800a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:55.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.447+0000 7efd9aaf5700 1 -- 172.21.15.37:0/1889587617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1f1252470 msgr2=0x55b1f12528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:55.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.447+0000 7efd9aaf5700 1 --2- 172.21.15.37:0/1889587617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1f1252470 0x55b1f12528b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:55.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.447+0000 7efd9aaf5700 1 -- 172.21.15.37:0/1889587617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1f1258600 msgr2=0x55b1f1251f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:55.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.447+0000 7efd9aaf5700 1 --2- 172.21.15.37:0/1889587617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1f1258600 0x55b1f1251f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:55.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.447+0000 7efd9aaf5700 1 -- 172.21.15.37:0/1889587617 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1f11310e0 con 0x55b1f1257ce0 2024-06-10T01:01:55.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.448+0000 7efd8b7fe700 1 -- 172.21.15.37:0/615565899 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7efd840d11f0 con 0x7efd94109460 2024-06-10T01:01:55.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.453+0000 7efd9aaf5700 1 --2- 172.21.15.37:0/1889587617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1f1257ce0 0x55b1f12580c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7efd8c007a60 tx=0x7efd8c007d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef09940eff9e75c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:55.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.453+0000 7efd9aaf5700 1 --2- 172.21.15.37:0/615565899 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7efd800a2e10 0x7efd800a52d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efd8c03d000 tx=0x7efd8c03f000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:55.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.455+0000 7efd88ff9700 1 -- 172.21.15.37:0/1889587617 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c040040 con 0x55b1f1257ce0 2024-06-10T01:01:55.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.455+0000 7efd88ff9700 1 -- 172.21.15.37:0/1889587617 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd8c039d40 con 0x55b1f1257ce0 2024-06-10T01:01:55.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.456+0000 7efda6ddbb80 1 -- 172.21.15.37:0/1889587617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1f1257ce0 msgr2=0x55b1f12580c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:55.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.456+0000 7efda6ddbb80 1 --2- 172.21.15.37:0/1889587617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1f1257ce0 0x55b1f12580c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7efd8c007a60 tx=0x7efd8c007d70 comp rx=0 tx=0).stop 2024-06-10T01:01:55.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.457+0000 7efda6ddbb80 1 -- 172.21.15.37:0/1889587617 shutdown_connections 2024-06-10T01:01:55.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.457+0000 7efda6ddbb80 1 --2- 172.21.15.37:0/1889587617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1f1252470 0x55b1f12528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:55.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.457+0000 7efda6ddbb80 1 --2- 172.21.15.37:0/1889587617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1f1257ce0 0x55b1f12580c0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:55.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.457+0000 7efda6ddbb80 1 --2- 172.21.15.37:0/1889587617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1f1258600 0x55b1f1251f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:55.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.457+0000 7efda6ddbb80 1 -- 172.21.15.37:0/1889587617 >> 172.21.15.37:0/1889587617 conn(0x55b1f116be50 msgr2=0x55b1f123c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:55.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.458+0000 7efda6ddbb80 1 -- 172.21.15.37:0/1889587617 shutdown_connections 2024-06-10T01:01:55.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.461+0000 7efda6ddbb80 1 -- 172.21.15.37:0/1889587617 wait complete. 2024-06-10T01:01:55.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.462+0000 7efda6ddbb80 1 -- start start 2024-06-10T01:01:55.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.462+0000 7efda6ddbb80 1 Processor -- start 2024-06-10T01:01:55.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.462+0000 7efda6ddbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1f1257ce0 0x55b1f125bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:55.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.462+0000 7efda6ddbb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1f125c310 0x55b1f125c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:55.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.462+0000 7efda6ddbb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1f1248770 0x55b1f124ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:55.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.462+0000 7efda6ddbb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b1f12411f0 con 0x55b1f1248770 2024-06-10T01:01:55.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.462+0000 7efda6ddbb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b1f1132d80 con 0x55b1f125c310 2024-06-10T01:01:55.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.462+0000 7efda6ddbb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b1f1241370 con 0x55b1f1257ce0 2024-06-10T01:01:55.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.462+0000 7efda6ddbb80 10 client.0 _ll_register_callbacks cb 0x55b1f112cab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:01:55.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.464+0000 7efd99af3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1f125c310 0x55b1f125c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:55.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.464+0000 7efd9aaf5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1f1248770 0x55b1f124ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:55.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.464+0000 7efd9aaf5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1f1248770 0x55b1f124ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47704/0 (socket says 172.21.15.37:47704) 2024-06-10T01:01:55.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.464+0000 7efd99af3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1f125c310 0x55b1f125c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:51370/0 (socket says 172.21.15.37:51370) 2024-06-10T01:01:55.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.464+0000 7efd9aaf5700 1 -- 172.21.15.37:0/493194781 learned_addr learned my addr 172.21.15.37:0/493194781 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:55.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.464+0000 7efd9aaf5700 1 -- 172.21.15.37:0/493194781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1f1257ce0 msgr2=0x55b1f125bdd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:01:55.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.464+0000 7efd9aaf5700 1 --2- 172.21.15.37:0/493194781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1f1257ce0 0x55b1f125bdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:55.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.464+0000 7efd9aaf5700 1 -- 172.21.15.37:0/493194781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1f125c310 msgr2=0x55b1f125c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:55.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.464+0000 7efd9aaf5700 1 --2- 172.21.15.37:0/493194781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1f125c310 0x55b1f125c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:55.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.464+0000 7efd9aaf5700 1 -- 172.21.15.37:0/493194781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1f10e1a60 con 0x55b1f1248770 2024-06-10T01:01:55.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.465+0000 7efd9aaf5700 1 --2- 172.21.15.37:0/493194781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1f1248770 0x55b1f124ac00 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009be0 tx=0x7efd8c00af60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b55a60e9abdc628 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:55.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.465+0000 7efd7dffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:01:55.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.467+0000 7efd7dffb700 1 -- 172.21.15.37:0/493194781 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c053020 con 0x55b1f1248770 2024-06-10T01:01:55.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.467+0000 7efda6ddbb80 1 -- 172.21.15.37:0/493194781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b1f119fea0 con 0x55b1f1248770 2024-06-10T01:01:55.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.467+0000 7efd7dffb700 1 -- 172.21.15.37:0/493194781 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7efd8c03c960 con 0x55b1f1248770 2024-06-10T01:01:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.469+0000 7efd7dffb700 1 -- 172.21.15.37:0/493194781 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efd8c05e050 con 0x55b1f1248770 2024-06-10T01:01:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.469+0000 7efda6ddbb80 10 client.16263 fetch_fsmap learned FSMap version 27 2024-06-10T01:01:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.469+0000 7efda6ddbb80 1 -- 172.21.15.37:0/493194781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b1f124cb20 con 0x55b1f1248770 2024-06-10T01:01:55.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.470+0000 7efd7dffb700 1 -- 172.21.15.37:0/493194781 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b1f119fea0 con 0x55b1f1248770 2024-06-10T01:01:55.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.470+0000 7efd7dffb700 1 -- 172.21.15.37:0/493194781 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7efd8c04fbf0 con 0x55b1f1248770 2024-06-10T01:01:55.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.471+0000 7efda6ddbb80 10 client.16263 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:01:55.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.471+0000 7efda6ddbb80 10 client.16263 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:01:55.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.471+0000 7efda6ddbb80 20 client.16263 populate_metadata read hostname 'smithi037' 2024-06-10T01:01:55.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.471+0000 7efda6ddbb80 1 --2- 172.21.15.37:0/493194781 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b1f124f050 0x55b1f1251470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:55.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.471+0000 7efda6ddbb80 4 client.16263 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:01:55.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.471+0000 7efda6ddbb80 1 -- 172.21.15.37:0/493194781 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1f119a420 con 0x55b1f124f050 2024-06-10T01:01:55.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.473+0000 7efd9a2f4700 1 --2- 172.21.15.37:0/493194781 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b1f124f050 0x55b1f1251470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:55.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.474+0000 7efd9a2f4700 1 --2- 172.21.15.37:0/493194781 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b1f124f050 0x55b1f1251470 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fa85f2a0f5f0662 server_cookie=357b9f29b82f3a4d in_seq=0 out_seq=0 2024-06-10T01:01:55.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.476+0000 7efd7dffb700 10 client.16263 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:55.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.482+0000 7efd7dffb700 1 -- 172.21.15.37:0/493194781 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b1f119a420 con 0x55b1f124f050 2024-06-10T01:01:55.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.482+0000 7efd7dffb700 10 client.16263 handle_command_reply: tid=0 2024-06-10T01:01:55.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.482+0000 7efda6ddbb80 1 client.16263 shutdown 2024-06-10T01:01:55.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.483+0000 7efda6ddbb80 1 -- 172.21.15.37:0/493194781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1f1248770 msgr2=0x55b1f124ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:55.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.483+0000 7efda6ddbb80 1 --2- 172.21.15.37:0/493194781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1f1248770 0x55b1f124ac00 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009be0 tx=0x7efd8c00af60 comp rx=0 tx=0).stop 2024-06-10T01:01:55.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.487+0000 7efda6ddbb80 1 -- 172.21.15.37:0/493194781 shutdown_connections 2024-06-10T01:01:55.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.487+0000 7efda6ddbb80 1 --2- 172.21.15.37:0/493194781 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b1f124f050 0x55b1f1251470 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:55.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.487+0000 7efda6ddbb80 1 --2- 172.21.15.37:0/493194781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1f1248770 0x55b1f124ac00 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:55.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.487+0000 7efda6ddbb80 1 --2- 172.21.15.37:0/493194781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1f1257ce0 0x55b1f125bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:55.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.487+0000 7efda6ddbb80 1 --2- 172.21.15.37:0/493194781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1f125c310 0x55b1f125c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:55.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.487+0000 7efda6ddbb80 1 -- 172.21.15.37:0/493194781 >> 172.21.15.37:0/493194781 conn(0x55b1f116be50 msgr2=0x55b1f123ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:55.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.488+0000 7efd7dffb700 0 client.16263 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:55.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.488+0000 7efda6ddbb80 1 -- 172.21.15.37:0/493194781 shutdown_connections 2024-06-10T01:01:55.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.488+0000 7efda6ddbb80 1 -- 172.21.15.37:0/493194781 wait complete. 2024-06-10T01:01:55.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.488+0000 7efda6ddbb80 20 client.16263 trim_cache size 0 max 16384 2024-06-10T01:01:55.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.532+0000 7efda6ddbb80 1 Processor -- start 2024-06-10T01:01:55.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.533+0000 7efda6ddbb80 1 -- start start 2024-06-10T01:01:55.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.534+0000 7efda6ddbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1f1248770 0x55b1f125bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:55.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.534+0000 7efda6ddbb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1f1257ce0 0x55b1f12599f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:55.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.534+0000 7efda6ddbb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1f125c310 0x55b1f1251ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:55.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.535+0000 7efd99af3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1f1257ce0 0x55b1f12599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:55.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.535+0000 7efd99af3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1f1257ce0 0x55b1f12599f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:51386/0 (socket says 172.21.15.37:51386) 2024-06-10T01:01:55.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.535+0000 7efd9a2f4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1f1248770 0x55b1f125bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:55.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.535+0000 7efd9a2f4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1f1248770 0x55b1f125bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:44334/0 (socket says 172.21.15.37:44334) 2024-06-10T01:01:55.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.536+0000 7efd99af3700 1 -- 172.21.15.37:0/3916879910 learned_addr learned my addr 172.21.15.37:0/3916879910 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:55.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.536+0000 7efd9aaf5700 1 --2- 172.21.15.37:0/3916879910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1f125c310 0x55b1f1251ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:55.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.536+0000 7efda6ddbb80 1 -- 172.21.15.37:0/3916879910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efd78011670 con 0x55b1f125c310 2024-06-10T01:01:55.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.537+0000 7efda6ddbb80 1 -- 172.21.15.37:0/3916879910 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efd7800ef00 con 0x55b1f1257ce0 2024-06-10T01:01:55.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.537+0000 7efda6ddbb80 1 -- 172.21.15.37:0/3916879910 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efd7800cd50 con 0x55b1f1248770 2024-06-10T01:01:55.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.537+0000 7efd99af3700 1 -- 172.21.15.37:0/3916879910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1f1248770 msgr2=0x55b1f125bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:55.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.537+0000 7efd99af3700 1 --2- 172.21.15.37:0/3916879910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1f1248770 0x55b1f125bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:55.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.537+0000 7efd99af3700 1 -- 172.21.15.37:0/3916879910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1f125c310 msgr2=0x55b1f1251ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:55.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.537+0000 7efd99af3700 1 --2- 172.21.15.37:0/3916879910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1f125c310 0x55b1f1251ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:55.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.538+0000 7efd99af3700 1 -- 172.21.15.37:0/3916879910 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1f125cd90 con 0x55b1f1257ce0 2024-06-10T01:01:55.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.539+0000 7efd99af3700 1 --2- 172.21.15.37:0/3916879910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1f1257ce0 0x55b1f12599f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efd8409d600 tx=0x7efd8401a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2321a90d613d2c17 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:55.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.541+0000 7efd7ffff700 1 -- 172.21.15.37:0/3916879910 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8409cfd0 con 0x55b1f1257ce0 2024-06-10T01:01:55.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.541+0000 7efd7ffff700 1 -- 172.21.15.37:0/3916879910 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd8409dd70 con 0x55b1f1257ce0 2024-06-10T01:01:55.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.542+0000 7efd7ffff700 1 -- 172.21.15.37:0/3916879910 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8409b970 con 0x55b1f1257ce0 2024-06-10T01:01:55.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.542+0000 7efda6ddbb80 1 -- 172.21.15.37:0/3916879910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1f1257ce0 msgr2=0x55b1f12599f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:55.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.542+0000 7efda6ddbb80 1 --2- 172.21.15.37:0/3916879910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1f1257ce0 0x55b1f12599f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efd8409d600 tx=0x7efd8401a040 comp rx=0 tx=0).stop 2024-06-10T01:01:55.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.543+0000 7efda6ddbb80 1 -- 172.21.15.37:0/3916879910 shutdown_connections 2024-06-10T01:01:55.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.543+0000 7efda6ddbb80 1 --2- 172.21.15.37:0/3916879910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1f125c310 0x55b1f1251ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:55.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.543+0000 7efda6ddbb80 1 --2- 172.21.15.37:0/3916879910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1f1248770 0x55b1f125bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:55.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.543+0000 7efda6ddbb80 1 --2- 172.21.15.37:0/3916879910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1f1257ce0 0x55b1f12599f0 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efd8409d600 tx=0x7efd8401a040 comp rx=0 tx=0).stop 2024-06-10T01:01:55.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.543+0000 7efda6ddbb80 1 -- 172.21.15.37:0/3916879910 >> 172.21.15.37:0/3916879910 conn(0x55b1f116be50 msgr2=0x55b1f125b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:55.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.543+0000 7efda6ddbb80 1 -- 172.21.15.37:0/3916879910 shutdown_connections 2024-06-10T01:01:55.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.543+0000 7efda6ddbb80 1 -- 172.21.15.37:0/3916879910 wait complete. 2024-06-10T01:01:55.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.544+0000 7efda6ddbb80 1 -- start start 2024-06-10T01:01:55.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.544+0000 7efda6ddbb80 1 Processor -- start 2024-06-10T01:01:55.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.545+0000 7efda6ddbb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1f1248770 0x55b1f1264470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:55.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.545+0000 7efda6ddbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1f125c310 0x55b1f1260b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:55.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.545+0000 7efda6ddbb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1f12610c0 0x55b1f1261540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:55.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.545+0000 7efd99af3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1f125c310 0x55b1f1260b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:55.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.545+0000 7efd99af3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1f125c310 0x55b1f1260b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:44340/0 (socket says 172.21.15.37:44340) 2024-06-10T01:01:55.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.546+0000 7efd99af3700 1 -- 172.21.15.37:0/4044983258 learned_addr learned my addr 172.21.15.37:0/4044983258 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:55.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.546+0000 7efda6ddbb80 1 -- 172.21.15.37:0/4044983258 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efd7800d3c0 con 0x55b1f12610c0 2024-06-10T01:01:55.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.546+0000 7efda6ddbb80 1 -- 172.21.15.37:0/4044983258 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efd7800e830 con 0x55b1f1248770 2024-06-10T01:01:55.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.546+0000 7efda6ddbb80 1 -- 172.21.15.37:0/4044983258 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efd7800da90 con 0x55b1f125c310 2024-06-10T01:01:55.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.546+0000 7efd9a2f4700 1 --2- 172.21.15.37:0/4044983258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1f1248770 0x55b1f1264470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:55.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.546+0000 7efda6ddbb80 10 client.0 _ll_register_callbacks cb 0x55b1f112cab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:01:55.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.551+0000 7efd9aaf5700 1 --2- 172.21.15.37:0/4044983258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1f12610c0 0x55b1f1261540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:55.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.552+0000 7efd9a2f4700 1 -- 172.21.15.37:0/4044983258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1f125c310 msgr2=0x55b1f1260b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:55.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.552+0000 7efd9a2f4700 1 --2- 172.21.15.37:0/4044983258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1f125c310 0x55b1f1260b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:55.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.552+0000 7efd9a2f4700 1 -- 172.21.15.37:0/4044983258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1f12610c0 msgr2=0x55b1f1261540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:55.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.552+0000 7efd9a2f4700 1 --2- 172.21.15.37:0/4044983258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1f12610c0 0x55b1f1261540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:55.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.552+0000 7efd9a2f4700 1 -- 172.21.15.37:0/4044983258 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1f12e8170 con 0x55b1f1248770 2024-06-10T01:01:55.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.557+0000 7efd9a2f4700 1 --2- 172.21.15.37:0/4044983258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1f1248770 0x55b1f1264470 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efd90004790 tx=0x7efd90004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de21070f9c2caef7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:55.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.558+0000 7efd7dffb700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:01:55.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.558+0000 7efd7dffb700 1 -- 172.21.15.37:0/4044983258 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90003cc0 con 0x55b1f1248770 2024-06-10T01:01:55.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.558+0000 7efd7dffb700 1 -- 172.21.15.37:0/4044983258 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7efd90004830 con 0x55b1f1248770 2024-06-10T01:01:55.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.558+0000 7efda6ddbb80 1 -- 172.21.15.37:0/4044983258 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd8409dd70 con 0x55b1f1248770 2024-06-10T01:01:55.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.562+0000 7efd7dffb700 1 -- 172.21.15.37:0/4044983258 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efd90003e60 con 0x55b1f1248770 2024-06-10T01:01:55.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.566+0000 7efda6ddbb80 10 client.25760 fetch_fsmap learned FSMap version 27 2024-06-10T01:01:55.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.566+0000 7efda6ddbb80 1 -- 172.21.15.37:0/4044983258 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b1f12e9470 con 0x55b1f1248770 2024-06-10T01:01:55.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.567+0000 7efd7dffb700 1 -- 172.21.15.37:0/4044983258 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd90005020 con 0x55b1f1248770 2024-06-10T01:01:55.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.567+0000 7efd7dffb700 1 -- 172.21.15.37:0/4044983258 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7efd90021070 con 0x55b1f1248770 2024-06-10T01:01:55.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.568+0000 7efda6ddbb80 10 client.25760 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:01:55.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.568+0000 7efda6ddbb80 10 client.25760 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:01:55.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.568+0000 7efda6ddbb80 20 client.25760 populate_metadata read hostname 'smithi037' 2024-06-10T01:01:55.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.568+0000 7efda6ddbb80 1 --2- 172.21.15.37:0/4044983258 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b1f12e9be0 0x55b1f12ebfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:55.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.568+0000 7efda6ddbb80 4 client.25760 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.2, "await_for": 45.0}] 2024-06-10T01:01:55.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.568+0000 7efda6ddbb80 1 -- 172.21.15.37:0/4044983258 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.2, "await_for": 45.0}) -- 0x7efd8409dd70 con 0x55b1f12e9be0 2024-06-10T01:01:55.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.569+0000 7efd99af3700 1 --2- 172.21.15.37:0/4044983258 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b1f12e9be0 0x55b1f12ebfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:55.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.573+0000 7efd99af3700 1 --2- 172.21.15.37:0/4044983258 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b1f12e9be0 0x55b1f12ebfd0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62473d885d160e5d server_cookie=b4e145c48fd47f07 in_seq=0 out_seq=0 2024-06-10T01:01:55.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:55.573+0000 7efd7dffb700 10 client.25760 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:56.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:56 smithi094 ceph-mon[36708]: pgmap v587: 129 pgs: 129 active+clean; 24 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 175 op/s 2024-06-10T01:01:56.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:56 smithi037 ceph-mon[32192]: pgmap v587: 129 pgs: 129 active+clean; 24 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 175 op/s 2024-06-10T01:01:56.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:56 smithi167 ceph-mon[36917]: pgmap v587: 129 pgs: 129 active+clean; 24 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 175 op/s 2024-06-10T01:01:57.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.032+0000 7f5d42a23700 1 -- 172.21.15.37:0/3184161036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d3c06b580 msgr2=0x7f5d3c10f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:57.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.032+0000 7f5d42a23700 1 --2- 172.21.15.37:0/3184161036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d3c06b580 0x7f5d3c10f320 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400b0a0 tx=0x7f5d3400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.032+0000 7f5d42a23700 1 -- 172.21.15.37:0/3184161036 shutdown_connections 2024-06-10T01:01:57.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.032+0000 7f5d42a23700 1 --2- 172.21.15.37:0/3184161036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d3c06b580 0x7f5d3c10f320 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.032+0000 7f5d42a23700 1 --2- 172.21.15.37:0/3184161036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c108500 0x7f5d3c1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.032+0000 7f5d42a23700 1 --2- 172.21.15.37:0/3184161036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d3c06abc0 0x7f5d3c06b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.033+0000 7f5d42a23700 1 -- 172.21.15.37:0/3184161036 >> 172.21.15.37:0/3184161036 conn(0x7f5d3c078340 msgr2=0x7f5d3c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:57.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.033+0000 7f5d42a23700 1 -- 172.21.15.37:0/3184161036 shutdown_connections 2024-06-10T01:01:57.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.033+0000 7f5d42a23700 1 -- 172.21.15.37:0/3184161036 wait complete. 2024-06-10T01:01:57.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.033+0000 7f5d42a23700 1 Processor -- start 2024-06-10T01:01:57.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.033+0000 7f5d42a23700 1 -- start start 2024-06-10T01:01:57.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.034+0000 7f5d42a23700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c06abc0 0x7f5d3c19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:57.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.034+0000 7f5d42a23700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d3c06b580 0x7f5d3c19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:57.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.034+0000 7f5d42a23700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d3c108500 0x7f5d3c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:57.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.034+0000 7f5d42a23700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c112760 con 0x7f5d3c108500 2024-06-10T01:01:57.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.034+0000 7f5d42a23700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c1125e0 con 0x7f5d3c06abc0 2024-06-10T01:01:57.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.034+0000 7f5d42a23700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c1128e0 con 0x7f5d3c06b580 2024-06-10T01:01:57.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.034+0000 7f5d40fc0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d3c108500 0x7f5d3c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:57.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.034+0000 7f5d40fc0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d3c108500 0x7f5d3c198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47748/0 (socket says 172.21.15.37:47748) 2024-06-10T01:01:57.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.034+0000 7f5d40fc0700 1 -- 172.21.15.37:0/2659472826 learned_addr learned my addr 172.21.15.37:0/2659472826 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:57.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.034+0000 7f5d3b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d3c06b580 0x7f5d3c19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:57.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.034+0000 7f5d3bfff700 1 --2- 172.21.15.37:0/2659472826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c06abc0 0x7f5d3c19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:57.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.034+0000 7f5d40fc0700 1 -- 172.21.15.37:0/2659472826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d3c06b580 msgr2=0x7f5d3c19ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:57.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.034+0000 7f5d40fc0700 1 --2- 172.21.15.37:0/2659472826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d3c06b580 0x7f5d3c19ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.034+0000 7f5d40fc0700 1 -- 172.21.15.37:0/2659472826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c06abc0 msgr2=0x7f5d3c19e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:57.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.034+0000 7f5d40fc0700 1 --2- 172.21.15.37:0/2659472826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c06abc0 0x7f5d3c19e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.034+0000 7f5d40fc0700 1 -- 172.21.15.37:0/2659472826 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3c1990e0 con 0x7f5d3c108500 2024-06-10T01:01:57.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.034+0000 7f5d3bfff700 1 --2- 172.21.15.37:0/2659472826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c06abc0 0x7f5d3c19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:01:57.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.034+0000 7f5d3b7fe700 1 --2- 172.21.15.37:0/2659472826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d3c06b580 0x7f5d3c19ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:01:57.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.035+0000 7f5d40fc0700 1 --2- 172.21.15.37:0/2659472826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d3c108500 0x7f5d3c198980 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400b780 tx=0x7f5d34004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:57.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.035+0000 7f5d397fa700 1 -- 172.21.15.37:0/2659472826 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d34021070 con 0x7f5d3c108500 2024-06-10T01:01:57.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.035+0000 7f5d397fa700 1 -- 172.21.15.37:0/2659472826 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d34007b80 con 0x7f5d3c108500 2024-06-10T01:01:57.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.035+0000 7f5d42a23700 1 -- 172.21.15.37:0/2659472826 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3c199370 con 0x7f5d3c108500 2024-06-10T01:01:57.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.035+0000 7f5d397fa700 1 -- 172.21.15.37:0/2659472826 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d34005370 con 0x7f5d3c108500 2024-06-10T01:01:57.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.035+0000 7f5d42a23700 1 -- 172.21.15.37:0/2659472826 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d3c1a5690 con 0x7f5d3c108500 2024-06-10T01:01:57.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.038+0000 7f5d397fa700 1 -- 172.21.15.37:0/2659472826 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5d34024070 con 0x7f5d3c108500 2024-06-10T01:01:57.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.038+0000 7f5d397fa700 1 --2- 172.21.15.37:0/2659472826 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5d240a2c20 0x7f5d240a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:57.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.039+0000 7f5d3bfff700 1 --2- 172.21.15.37:0/2659472826 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5d240a2c20 0x7f5d240a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:57.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.039+0000 7f5d397fa700 1 -- 172.21.15.37:0/2659472826 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f5d3400f020 con 0x7f5d3c108500 2024-06-10T01:01:57.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.039+0000 7f5d3bfff700 1 --2- 172.21.15.37:0/2659472826 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5d240a2c20 0x7f5d240a50e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c005ea0 tx=0x7f5d2c005e30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:57.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.039+0000 7f5d4d2a6b80 1 Processor -- start 2024-06-10T01:01:57.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.039+0000 7f5d4d2a6b80 1 -- start start 2024-06-10T01:01:57.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.039+0000 7f5d4d2a6b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a75112efe0 0x55a75112f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:57.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.039+0000 7f5d4d2a6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a75112f900 0x55a7511292e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:57.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.039+0000 7f5d4d2a6b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a751129820 0x55a751129c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:57.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.039+0000 7f5d4d2a6b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a75101bfe0 con 0x55a751129820 2024-06-10T01:01:57.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.039+0000 7f5d4d2a6b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a75107a8d0 con 0x55a75112efe0 2024-06-10T01:01:57.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.039+0000 7f5d4d2a6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a750fe24e0 con 0x55a75112f900 2024-06-10T01:01:57.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.039+0000 7f5d40fc0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a751129820 0x55a751129c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:57.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.039+0000 7f5d3bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a75112efe0 0x55a75112f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:57.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.039+0000 7f5d40fc0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a751129820 0x55a751129c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47762/0 (socket says 172.21.15.37:47762) 2024-06-10T01:01:57.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.039+0000 7f5d3b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a75112f900 0x55a7511292e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:57.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.039+0000 7f5d40fc0700 1 -- 172.21.15.37:0/518848736 learned_addr learned my addr 172.21.15.37:0/518848736 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:57.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.040+0000 7f5d3bfff700 1 -- 172.21.15.37:0/518848736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a75112f900 msgr2=0x55a7511292e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:57.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.040+0000 7f5d3bfff700 1 --2- 172.21.15.37:0/518848736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a75112f900 0x55a7511292e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.040+0000 7f5d3bfff700 1 -- 172.21.15.37:0/518848736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a751129820 msgr2=0x55a751129c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:57.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.040+0000 7f5d3bfff700 1 --2- 172.21.15.37:0/518848736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a751129820 0x55a751129c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.040+0000 7f5d3bfff700 1 -- 172.21.15.37:0/518848736 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a750fc9950 con 0x55a75112efe0 2024-06-10T01:01:57.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.040+0000 7f5d40fc0700 1 --2- 172.21.15.37:0/518848736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a751129820 0x55a751129c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:01:57.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.040+0000 7f5d3bfff700 1 --2- 172.21.15.37:0/518848736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a75112efe0 0x55a75112f3c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c005fd0 tx=0x7f5d2c011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39cc135422e3dad server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:57.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.040+0000 7f5d227fc700 1 -- 172.21.15.37:0/518848736 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c021070 con 0x55a75112efe0 2024-06-10T01:01:57.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.040+0000 7f5d227fc700 1 -- 172.21.15.37:0/518848736 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d2c004500 con 0x55a75112efe0 2024-06-10T01:01:57.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.040+0000 7f5d227fc700 1 -- 172.21.15.37:0/518848736 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c003db0 con 0x55a75112efe0 2024-06-10T01:01:57.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.041+0000 7f5d4d2a6b80 1 -- 172.21.15.37:0/518848736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a75112efe0 msgr2=0x55a75112f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:57.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.041+0000 7f5d4d2a6b80 1 --2- 172.21.15.37:0/518848736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a75112efe0 0x55a75112f3c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c005fd0 tx=0x7f5d2c011000 comp rx=0 tx=0).stop 2024-06-10T01:01:57.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.041+0000 7f5d4d2a6b80 1 -- 172.21.15.37:0/518848736 shutdown_connections 2024-06-10T01:01:57.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.041+0000 7f5d4d2a6b80 1 --2- 172.21.15.37:0/518848736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a751129820 0x55a751129c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.041+0000 7f5d4d2a6b80 1 --2- 172.21.15.37:0/518848736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a75112efe0 0x55a75112f3c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.041+0000 7f5d4d2a6b80 1 --2- 172.21.15.37:0/518848736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a75112f900 0x55a7511292e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.041+0000 7f5d4d2a6b80 1 -- 172.21.15.37:0/518848736 >> 172.21.15.37:0/518848736 conn(0x55a751056100 msgr2=0x55a7511242a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:57.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.041+0000 7f5d4d2a6b80 1 -- 172.21.15.37:0/518848736 shutdown_connections 2024-06-10T01:01:57.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.041+0000 7f5d4d2a6b80 1 -- 172.21.15.37:0/518848736 wait complete. 2024-06-10T01:01:57.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.042+0000 7f5d4d2a6b80 1 -- start start 2024-06-10T01:01:57.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.042+0000 7f5d4d2a6b80 1 Processor -- start 2024-06-10T01:01:57.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.042+0000 7f5d4d2a6b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a751129820 0x55a751135690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:57.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.042+0000 7f5d4d2a6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a75112efe0 0x55a751132990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.042+0000 7f5d4d2a6b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a75112f900 0x55a751132ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.042+0000 7f5d4d2a6b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a75101b040 con 0x55a75112f900 2024-06-10T01:01:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.042+0000 7f5d4d2a6b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a751076890 con 0x55a751129820 2024-06-10T01:01:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.042+0000 7f5d4d2a6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a7511362e0 con 0x55a75112efe0 2024-06-10T01:01:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.042+0000 7f5d4d2a6b80 10 client.0 _ll_register_callbacks cb 0x55a751010630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:01:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.042+0000 7f5d40fc0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a75112f900 0x55a751132ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.042+0000 7f5d3bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a751129820 0x55a751135690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.042+0000 7f5d3bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a751129820 0x55a751135690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:51416/0 (socket says 172.21.15.37:51416) 2024-06-10T01:01:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.042+0000 7f5d40fc0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a75112f900 0x55a751132ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47772/0 (socket says 172.21.15.37:47772) 2024-06-10T01:01:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.042+0000 7f5d3bfff700 1 -- 172.21.15.37:0/3460555261 learned_addr learned my addr 172.21.15.37:0/3460555261 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.042+0000 7f5d3b7fe700 1 --2- 172.21.15.37:0/3460555261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a75112efe0 0x55a751132990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.043+0000 7f5d40fc0700 1 -- 172.21.15.37:0/3460555261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a75112efe0 msgr2=0x55a751132990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.043+0000 7f5d40fc0700 1 --2- 172.21.15.37:0/3460555261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a75112efe0 0x55a751132990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.043+0000 7f5d40fc0700 1 -- 172.21.15.37:0/3460555261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a751129820 msgr2=0x55a751135690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.043+0000 7f5d40fc0700 1 --2- 172.21.15.37:0/3460555261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a751129820 0x55a751135690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.043+0000 7f5d40fc0700 1 -- 172.21.15.37:0/3460555261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7510407d0 con 0x55a75112f900 2024-06-10T01:01:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.043+0000 7f5d3bfff700 1 --2- 172.21.15.37:0/3460555261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a751129820 0x55a751135690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:01:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.043+0000 7f5d40fc0700 1 --2- 172.21.15.37:0/3460555261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a75112f900 0x55a751132ed0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5d3409ce40 tx=0x7f5d3409c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84fdb3384437139 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.043+0000 7f5d1b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:01:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.043+0000 7f5d1b7fe700 1 -- 172.21.15.37:0/3460555261 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d340ba020 con 0x55a75112f900 2024-06-10T01:01:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.043+0000 7f5d1b7fe700 1 -- 172.21.15.37:0/3460555261 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f5d340074e0 con 0x55a75112f900 2024-06-10T01:01:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.043+0000 7f5d4d2a6b80 1 -- 172.21.15.37:0/3460555261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d2c004500 con 0x55a75112f900 2024-06-10T01:01:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.043+0000 7f5d1b7fe700 1 -- 172.21.15.37:0/3460555261 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5d340a1260 con 0x55a75112f900 2024-06-10T01:01:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.043+0000 7f5d4d2a6b80 10 client.16281 fetch_fsmap learned FSMap version 27 2024-06-10T01:01:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.043+0000 7f5d4d2a6b80 1 -- 172.21.15.37:0/3460555261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a751140a30 con 0x55a75112f900 2024-06-10T01:01:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.044+0000 7f5d1b7fe700 1 -- 172.21.15.37:0/3460555261 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d2c004500 con 0x55a75112f900 2024-06-10T01:01:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.044+0000 7f5d1b7fe700 1 -- 172.21.15.37:0/3460555261 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5d34011070 con 0x55a75112f900 2024-06-10T01:01:57.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.044+0000 7f5d4d2a6b80 10 client.16281 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:01:57.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.044+0000 7f5d4d2a6b80 10 client.16281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:01:57.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.044+0000 7f5d4d2a6b80 20 client.16281 populate_metadata read hostname 'smithi037' 2024-06-10T01:01:57.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.044+0000 7f5d4d2a6b80 1 --2- 172.21.15.37:0/3460555261 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a751142ff0 0x55a751145430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:57.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.044+0000 7f5d4d2a6b80 4 client.16281 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:01:57.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.044+0000 7f5d4d2a6b80 1 -- 172.21.15.37:0/3460555261 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a751087dd0 con 0x55a751142ff0 2024-06-10T01:01:57.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.044+0000 7f5d3bfff700 1 --2- 172.21.15.37:0/3460555261 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a751142ff0 0x55a751145430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:57.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.045+0000 7f5d3bfff700 1 --2- 172.21.15.37:0/3460555261 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a751142ff0 0x55a751145430 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7109fc91c9237466 server_cookie=44e29005d931bf58 in_seq=0 out_seq=0 2024-06-10T01:01:57.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.045+0000 7f5d1b7fe700 10 client.16281 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:57.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.047+0000 7f5d1b7fe700 1 -- 172.21.15.37:0/3460555261 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a751087dd0 con 0x55a751142ff0 2024-06-10T01:01:57.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.047+0000 7f5d1b7fe700 10 client.16281 handle_command_reply: tid=0 2024-06-10T01:01:57.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.047+0000 7f5d4d2a6b80 1 client.16281 shutdown 2024-06-10T01:01:57.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.047+0000 7f5d4d2a6b80 1 -- 172.21.15.37:0/3460555261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a75112f900 msgr2=0x55a751132ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:57.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.047+0000 7f5d4d2a6b80 1 --2- 172.21.15.37:0/3460555261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a75112f900 0x55a751132ed0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5d3409ce40 tx=0x7f5d3409c940 comp rx=0 tx=0).stop 2024-06-10T01:01:57.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.047+0000 7f5d4d2a6b80 1 -- 172.21.15.37:0/3460555261 shutdown_connections 2024-06-10T01:01:57.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.047+0000 7f5d4d2a6b80 1 --2- 172.21.15.37:0/3460555261 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a751142ff0 0x55a751145430 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.047+0000 7f5d4d2a6b80 1 --2- 172.21.15.37:0/3460555261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a75112f900 0x55a751132ed0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.047+0000 7f5d4d2a6b80 1 --2- 172.21.15.37:0/3460555261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a751129820 0x55a751135690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.047+0000 7f5d4d2a6b80 1 --2- 172.21.15.37:0/3460555261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a75112efe0 0x55a751132990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.047+0000 7f5d1b7fe700 0 client.16281 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:57.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.047+0000 7f5d4d2a6b80 1 -- 172.21.15.37:0/3460555261 >> 172.21.15.37:0/3460555261 conn(0x55a751056100 msgr2=0x55a751123780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:57.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.047+0000 7f5d4d2a6b80 1 -- 172.21.15.37:0/3460555261 shutdown_connections 2024-06-10T01:01:57.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.047+0000 7f5d4d2a6b80 1 -- 172.21.15.37:0/3460555261 wait complete. 2024-06-10T01:01:57.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.047+0000 7f5d4d2a6b80 20 client.16281 trim_cache size 0 max 16384 2024-06-10T01:01:57.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.069+0000 7f5d4d2a6b80 1 Processor -- start 2024-06-10T01:01:57.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.069+0000 7f5d4d2a6b80 1 -- start start 2024-06-10T01:01:57.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.069+0000 7f5d4d2a6b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a751129820 0x55a751131b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:57.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.069+0000 7f5d4d2a6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a75112efe0 0x55a7511351d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:57.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.069+0000 7f5d4d2a6b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a75112f900 0x55a751123f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:57.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.069+0000 7f5d4d2a6b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c1aea90 con 0x55a75112f900 2024-06-10T01:01:57.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.069+0000 7f5d4d2a6b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c1ac320 con 0x55a751129820 2024-06-10T01:01:57.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.069+0000 7f5d4d2a6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c1aa190 con 0x55a75112efe0 2024-06-10T01:01:57.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.069+0000 7f5d3b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a75112efe0 0x55a7511351d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:57.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.069+0000 7f5d40fc0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a75112f900 0x55a751123f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:57.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.069+0000 7f5d3b7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a75112efe0 0x55a7511351d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:44390/0 (socket says 172.21.15.37:44390) 2024-06-10T01:01:57.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.069+0000 7f5d40fc0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a75112f900 0x55a751123f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47786/0 (socket says 172.21.15.37:47786) 2024-06-10T01:01:57.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.069+0000 7f5d3b7fe700 1 -- 172.21.15.37:0/1443935899 learned_addr learned my addr 172.21.15.37:0/1443935899 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:57.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.069+0000 7f5d3bfff700 1 --2- 172.21.15.37:0/1443935899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a751129820 0x55a751131b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:57.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.069+0000 7f5d3b7fe700 1 -- 172.21.15.37:0/1443935899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a751129820 msgr2=0x55a751131b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:57.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.069+0000 7f5d3b7fe700 1 --2- 172.21.15.37:0/1443935899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a751129820 0x55a751131b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.069+0000 7f5d3b7fe700 1 -- 172.21.15.37:0/1443935899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a75112f900 msgr2=0x55a751123f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:57.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.069+0000 7f5d3b7fe700 1 --2- 172.21.15.37:0/1443935899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a75112f900 0x55a751123f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.069+0000 7f5d3b7fe700 1 -- 172.21.15.37:0/1443935899 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7511250c0 con 0x55a75112efe0 2024-06-10T01:01:57.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.069+0000 7f5d40fc0700 1 --2- 172.21.15.37:0/1443935899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a75112f900 0x55a751123f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:01:57.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.070+0000 7f5d3b7fe700 1 --2- 172.21.15.37:0/1443935899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a75112efe0 0x55a7511351d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5d300127a0 tx=0x7f5d30009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86a1b50aaa650a2d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:57.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.070+0000 7f5d21ffb700 1 -- 172.21.15.37:0/1443935899 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30010430 con 0x55a75112efe0 2024-06-10T01:01:57.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.070+0000 7f5d21ffb700 1 -- 172.21.15.37:0/1443935899 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d30021070 con 0x55a75112efe0 2024-06-10T01:01:57.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.070+0000 7f5d21ffb700 1 -- 172.21.15.37:0/1443935899 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3001c970 con 0x55a75112efe0 2024-06-10T01:01:57.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.070+0000 7f5d4d2a6b80 1 -- 172.21.15.37:0/1443935899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a75112efe0 msgr2=0x55a7511351d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:57.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.070+0000 7f5d4d2a6b80 1 --2- 172.21.15.37:0/1443935899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a75112efe0 0x55a7511351d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5d300127a0 tx=0x7f5d30009260 comp rx=0 tx=0).stop 2024-06-10T01:01:57.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.071+0000 7f5d4d2a6b80 1 -- 172.21.15.37:0/1443935899 shutdown_connections 2024-06-10T01:01:57.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.071+0000 7f5d4d2a6b80 1 --2- 172.21.15.37:0/1443935899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a75112f900 0x55a751123f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.071+0000 7f5d4d2a6b80 1 --2- 172.21.15.37:0/1443935899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a751129820 0x55a751131b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.071+0000 7f5d4d2a6b80 1 --2- 172.21.15.37:0/1443935899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a75112efe0 0x55a7511351d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.071+0000 7f5d4d2a6b80 1 -- 172.21.15.37:0/1443935899 >> 172.21.15.37:0/1443935899 conn(0x55a751056100 msgr2=0x55a75112daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:57.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.071+0000 7f5d4d2a6b80 1 -- 172.21.15.37:0/1443935899 shutdown_connections 2024-06-10T01:01:57.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.071+0000 7f5d4d2a6b80 1 -- 172.21.15.37:0/1443935899 wait complete. 2024-06-10T01:01:57.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.071+0000 7f5d4d2a6b80 1 -- start start 2024-06-10T01:01:57.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.071+0000 7f5d4d2a6b80 1 Processor -- start 2024-06-10T01:01:57.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.072+0000 7f5d4d2a6b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a751129820 0x55a75113bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:57.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.072+0000 7f5d4d2a6b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a75112efe0 0x55a7511385a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:57.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.072+0000 7f5d4d2a6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a75112f900 0x55a751138ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:57.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.072+0000 7f5d4d2a6b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c1aa7e0 con 0x55a75112efe0 2024-06-10T01:01:57.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.072+0000 7f5d4d2a6b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c1abc50 con 0x55a751129820 2024-06-10T01:01:57.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.072+0000 7f5d4d2a6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c1aaeb0 con 0x55a75112f900 2024-06-10T01:01:57.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.072+0000 7f5d4d2a6b80 10 client.0 _ll_register_callbacks cb 0x55a751010630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:01:57.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.072+0000 7f5d3b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a75112efe0 0x55a7511385a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:57.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.072+0000 7f5d3bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a751129820 0x55a75113bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:57.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.072+0000 7f5d3b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a75112efe0 0x55a7511385a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47794/0 (socket says 172.21.15.37:47794) 2024-06-10T01:01:57.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.072+0000 7f5d3b7fe700 1 -- 172.21.15.37:0/3563186528 learned_addr learned my addr 172.21.15.37:0/3563186528 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:01:57.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.072+0000 7f5d40fc0700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a75112f900 0x55a751138ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:57.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.072+0000 7f5d3bfff700 1 -- 172.21.15.37:0/3563186528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a75112f900 msgr2=0x55a751138ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:57.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.072+0000 7f5d3bfff700 1 --2- 172.21.15.37:0/3563186528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a75112f900 0x55a751138ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.072+0000 7f5d3bfff700 1 -- 172.21.15.37:0/3563186528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a75112efe0 msgr2=0x55a7511385a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:01:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.072+0000 7f5d3bfff700 1 --2- 172.21.15.37:0/3563186528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a75112efe0 0x55a7511385a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.072+0000 7f5d3bfff700 1 -- 172.21.15.37:0/3563186528 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a751139120 con 0x55a751129820 2024-06-10T01:01:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.072+0000 7f5d3b7fe700 1 --2- 172.21.15.37:0/3563186528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a75112efe0 0x55a7511385a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:01:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.072+0000 7f5d3bfff700 1 --2- 172.21.15.37:0/3563186528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a751129820 0x55a75113bbe0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55a751145c70 tx=0x7f5d2c00e6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bbb56bf618d7d24 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:01:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.072+0000 7f5d1b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:01:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.073+0000 7f5d1b7fe700 1 -- 172.21.15.37:0/3563186528 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c037030 con 0x55a751129820 2024-06-10T01:01:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.073+0000 7f5d4d2a6b80 1 -- 172.21.15.37:0/3563186528 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d30021070 con 0x55a751129820 2024-06-10T01:01:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.073+0000 7f5d1b7fe700 1 -- 172.21.15.37:0/3563186528 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f5d2c033450 con 0x55a751129820 2024-06-10T01:01:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.073+0000 7f5d1b7fe700 1 -- 172.21.15.37:0/3563186528 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5d2c041050 con 0x55a751129820 2024-06-10T01:01:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.073+0000 7f5d4d2a6b80 10 client.25775 fetch_fsmap learned FSMap version 27 2024-06-10T01:01:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.073+0000 7f5d4d2a6b80 1 -- 172.21.15.37:0/3563186528 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a7511aeb70 con 0x55a751129820 2024-06-10T01:01:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.073+0000 7f5d1b7fe700 1 -- 172.21.15.37:0/3563186528 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d30021070 con 0x55a751129820 2024-06-10T01:01:57.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.073+0000 7f5d1b7fe700 1 -- 172.21.15.37:0/3563186528 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5d2c01d610 con 0x55a751129820 2024-06-10T01:01:57.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.073+0000 7f5d4d2a6b80 10 client.25775 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:01:57.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.073+0000 7f5d4d2a6b80 10 client.25775 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:01:57.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.073+0000 7f5d4d2a6b80 20 client.25775 populate_metadata read hostname 'smithi037' 2024-06-10T01:01:57.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.073+0000 7f5d4d2a6b80 1 --2- 172.21.15.37:0/3563186528 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a7511aef40 0x55a7511b1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:01:57.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.073+0000 7f5d4d2a6b80 4 client.25775 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:01:57.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.073+0000 7f5d4d2a6b80 1 -- 172.21.15.37:0/3563186528 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a751082350 con 0x55a7511aef40 2024-06-10T01:01:57.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.073+0000 7f5d3b7fe700 1 --2- 172.21.15.37:0/3563186528 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a7511aef40 0x55a7511b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:01:57.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.074+0000 7f5d1b7fe700 10 client.25775 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:57.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.074+0000 7f5d3b7fe700 1 --2- 172.21.15.37:0/3563186528 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a7511aef40 0x55a7511b1330 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8651f43943ac18cc server_cookie=563a654dceb99198 in_seq=0 out_seq=0 2024-06-10T01:01:57.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.144+0000 7f5d1b7fe700 1 -- 172.21.15.37:0/3563186528 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a751082350 con 0x55a7511aef40 2024-06-10T01:01:57.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.144+0000 7f5d1b7fe700 10 client.25775 handle_command_reply: tid=0 2024-06-10T01:01:57.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.144+0000 7f5d4d2a6b80 1 client.25775 shutdown 2024-06-10T01:01:57.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.145+0000 7f5d4d2a6b80 1 -- 172.21.15.37:0/3563186528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a751129820 msgr2=0x55a75113bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:57.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.145+0000 7f5d4d2a6b80 1 --2- 172.21.15.37:0/3563186528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a751129820 0x55a75113bbe0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55a751145c70 tx=0x7f5d2c00e6e0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.145+0000 7f5d4d2a6b80 1 -- 172.21.15.37:0/3563186528 shutdown_connections 2024-06-10T01:01:57.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.145+0000 7f5d4d2a6b80 1 --2- 172.21.15.37:0/3563186528 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a7511aef40 0x55a7511b1330 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.145+0000 7f5d4d2a6b80 1 --2- 172.21.15.37:0/3563186528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a75112f900 0x55a751138ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.145+0000 7f5d4d2a6b80 1 --2- 172.21.15.37:0/3563186528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a751129820 0x55a75113bbe0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.145+0000 7f5d1b7fe700 0 client.25775 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:57.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.145+0000 7f5d4d2a6b80 1 --2- 172.21.15.37:0/3563186528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a75112efe0 0x55a7511385a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.145+0000 7f5d4d2a6b80 1 -- 172.21.15.37:0/3563186528 >> 172.21.15.37:0/3563186528 conn(0x55a751056100 msgr2=0x55a75112daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:57.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.145+0000 7f5d4d2a6b80 1 -- 172.21.15.37:0/3563186528 shutdown_connections 2024-06-10T01:01:57.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.145+0000 7f5d4d2a6b80 1 -- 172.21.15.37:0/3563186528 wait complete. 2024-06-10T01:01:57.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.145+0000 7f5d4d2a6b80 20 client.25775 trim_cache size 0 max 16384 2024-06-10T01:01:57.144 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:01:57.145 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-06-10T01:01:57.145 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "2f56bf32-ff1d-4949-b371-c27f8b6b8de2", 2024-06-10T01:01:57.145 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-06-10T01:01:57.145 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:01:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:57.146+0000 7f173eff5640 1 -- 172.21.15.37:0/2243885377 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5652cf126c20 con 0x5652cf1d02e0 2024-06-10T01:01:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:57.146+0000 7f173eff5640 10 client.16251 handle_command_reply: tid=0 2024-06-10T01:01:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:57.146+0000 7f1763f92740 1 client.16251 shutdown 2024-06-10T01:01:57.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.146+0000 7f5d42a23700 1 -- 172.21.15.37:0/2659472826 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5d240a2c20 msgr2=0x7f5d240a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:57.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.146+0000 7f5d42a23700 1 --2- 172.21.15.37:0/2659472826 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5d240a2c20 0x7f5d240a50e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c005ea0 tx=0x7f5d2c005e30 comp rx=0 tx=0).stop 2024-06-10T01:01:57.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.146+0000 7f5d42a23700 1 -- 172.21.15.37:0/2659472826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d3c108500 msgr2=0x7f5d3c198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:57.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.146+0000 7f5d42a23700 1 --2- 172.21.15.37:0/2659472826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d3c108500 0x7f5d3c198980 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400b780 tx=0x7f5d34004d00 comp rx=0 tx=0).stop 2024-06-10T01:01:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:57.146+0000 7f1763f92740 1 -- 172.21.15.37:0/2243885377 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652cf1ba420 msgr2=0x5652cf1c8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:57.146+0000 7f1763f92740 1 --2- 172.21.15.37:0/2243885377 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652cf1ba420 0x5652cf1c8520 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f174c0b8210 tx=0x7f174c0c4af0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.146+0000 7f5d42a23700 1 -- 172.21.15.37:0/2659472826 shutdown_connections 2024-06-10T01:01:57.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.146+0000 7f5d42a23700 1 --2- 172.21.15.37:0/2659472826 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5d240a2c20 0x7f5d240a50e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.146+0000 7f5d42a23700 1 --2- 172.21.15.37:0/2659472826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d3c108500 0x7f5d3c198980 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.146+0000 7f5d42a23700 1 --2- 172.21.15.37:0/2659472826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c06abc0 0x7f5d3c19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.146+0000 7f5d42a23700 1 --2- 172.21.15.37:0/2659472826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d3c06b580 0x7f5d3c19ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.146+0000 7f5d42a23700 1 -- 172.21.15.37:0/2659472826 >> 172.21.15.37:0/2659472826 conn(0x7f5d3c078340 msgr2=0x7f5d3c0ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:57.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.146+0000 7f5d42a23700 1 -- 172.21.15.37:0/2659472826 shutdown_connections 2024-06-10T01:01:57.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:01:57.146+0000 7f5d42a23700 1 -- 172.21.15.37:0/2659472826 wait complete. 2024-06-10T01:01:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:57.146+0000 7f1763f92740 1 -- 172.21.15.37:0/2243885377 shutdown_connections 2024-06-10T01:01:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:57.146+0000 7f1763f92740 1 --2- 172.21.15.37:0/2243885377 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5652cf1d02e0 0x5652cf23ca30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:57.146+0000 7f1763f92740 1 --2- 172.21.15.37:0/2243885377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5652cf1bff80 0x5652cf239030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:57.146+0000 7f173eff5640 0 client.16251 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:01:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:57.146+0000 7f1763f92740 1 --2- 172.21.15.37:0/2243885377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652cf1bae10 0x5652cf1c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:57.146+0000 7f1763f92740 1 --2- 172.21.15.37:0/2243885377 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652cf1ba420 0x5652cf1c8520 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:57.146+0000 7f1763f92740 1 -- 172.21.15.37:0/2243885377 >> 172.21.15.37:0/2243885377 conn(0x5652cf1b1680 msgr2=0x5652cf1c42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:57.146+0000 7f1763f92740 1 -- 172.21.15.37:0/2243885377 shutdown_connections 2024-06-10T01:01:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:57.146+0000 7f1763f92740 1 -- 172.21.15.37:0/2243885377 wait complete. 2024-06-10T01:01:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:57.146+0000 7f1763f92740 20 client.16251 trim_cache size 0 max 16384 2024-06-10T01:01:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:01:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:01:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:01:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:57.147+0000 7f1761288640 1 -- 172.21.15.37:0/1824801421 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f17440a2cb0 msgr2=0x7f17440a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:57.147+0000 7f1761288640 1 --2- 172.21.15.37:0/1824801421 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f17440a2cb0 0x7f17440a5170 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x5652cf1bb670 tx=0x7f1754009600 comp rx=0 tx=0).stop 2024-06-10T01:01:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:57.147+0000 7f1761288640 1 -- 172.21.15.37:0/1824801421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f175c1024c0 msgr2=0x7f175c197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:01:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:57.147+0000 7f1761288640 1 --2- 172.21.15.37:0/1824801421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f175c1024c0 0x7f175c197c80 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f174c009230 tx=0x7f174c031b60 comp rx=0 tx=0).stop 2024-06-10T01:01:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:57.147+0000 7f1761288640 1 -- 172.21.15.37:0/1824801421 shutdown_connections 2024-06-10T01:01:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:57.147+0000 7f1761288640 1 --2- 172.21.15.37:0/1824801421 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f17440a2cb0 0x7f17440a5170 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:57.147+0000 7f1761288640 1 --2- 172.21.15.37:0/1824801421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f175c107720 0x7f175c19f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:57.147+0000 7f1761288640 1 --2- 172.21.15.37:0/1824801421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f175c104df0 0x7f175c1981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:57.147+0000 7f1761288640 1 --2- 172.21.15.37:0/1824801421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f175c1024c0 0x7f175c197c80 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:01:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:57.147+0000 7f1761288640 1 -- 172.21.15.37:0/1824801421 >> 172.21.15.37:0/1824801421 conn(0x7f175c0fbeb0 msgr2=0x7f175c102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:01:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:57.147+0000 7f1761288640 1 -- 172.21.15.37:0/1824801421 shutdown_connections 2024-06-10T01:01:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:01:57.147+0000 7f1761288640 1 -- 172.21.15.37:0/1824801421 wait complete. 2024-06-10T01:01:58.090 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:01:58.345 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:01:58.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:58 smithi094 ceph-mon[36708]: pgmap v588: 129 pgs: 129 active+clean; 24 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 308 op/s 2024-06-10T01:01:58.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:58 smithi094 ceph-mon[36708]: scrub queued for path: / 2024-06-10T01:01:58.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:58 smithi094 ceph-mon[36708]: scrub summary: idle+waiting paths [/] 2024-06-10T01:01:58.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:01:58 smithi094 ceph-mon[36708]: scrub summary: active paths [/] 2024-06-10T01:01:58.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:58 smithi037 ceph-mon[32192]: pgmap v588: 129 pgs: 129 active+clean; 24 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 308 op/s 2024-06-10T01:01:58.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:58 smithi037 ceph-mon[32192]: scrub queued for path: / 2024-06-10T01:01:58.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:58 smithi037 ceph-mon[32192]: scrub summary: idle+waiting paths [/] 2024-06-10T01:01:58.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:01:58 smithi037 ceph-mon[32192]: scrub summary: active paths [/] 2024-06-10T01:01:58.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:58 smithi167 ceph-mon[36917]: pgmap v588: 129 pgs: 129 active+clean; 24 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 308 op/s 2024-06-10T01:01:58.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:58 smithi167 ceph-mon[36917]: scrub queued for path: / 2024-06-10T01:01:58.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:58 smithi167 ceph-mon[36917]: scrub summary: idle+waiting paths [/] 2024-06-10T01:01:58.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:01:58 smithi167 ceph-mon[36917]: scrub summary: active paths [/] 2024-06-10T01:02:00.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.285+0000 7fab3a1ea700 1 -- 172.21.15.37:0/3150094048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab341024e0 msgr2=0x7fab34102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:00.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.285+0000 7fab3a1ea700 1 --2- 172.21.15.37:0/3150094048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab341024e0 0x7fab34102940 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fab28009990 tx=0x7fab28009ca0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.285+0000 7fab3a1ea700 1 -- 172.21.15.37:0/3150094048 shutdown_connections 2024-06-10T01:02:00.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.285+0000 7fab3a1ea700 1 --2- 172.21.15.37:0/3150094048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab34102e80 0x7fab3410f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.285+0000 7fab3a1ea700 1 --2- 172.21.15.37:0/3150094048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab34108650 0x7fab34108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.285+0000 7fab3a1ea700 1 --2- 172.21.15.37:0/3150094048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab341024e0 0x7fab34102940 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.285+0000 7fab3a1ea700 1 -- 172.21.15.37:0/3150094048 >> 172.21.15.37:0/3150094048 conn(0x7fab340795f0 msgr2=0x7fab34079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:00.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.285+0000 7fab3a1ea700 1 -- 172.21.15.37:0/3150094048 shutdown_connections 2024-06-10T01:02:00.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.285+0000 7fab3a1ea700 1 -- 172.21.15.37:0/3150094048 wait complete. 2024-06-10T01:02:00.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.286+0000 7fab3a1ea700 1 Processor -- start 2024-06-10T01:02:00.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.286+0000 7fab3a1ea700 1 -- start start 2024-06-10T01:02:00.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.286+0000 7fab3a1ea700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab341024e0 0x7fab34074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:00.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.286+0000 7fab3a1ea700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab34102e80 0x7fab34074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:00.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.286+0000 7fab3a1ea700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab34108650 0x7fab34075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:00.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.286+0000 7fab3a1ea700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fab341127a0 con 0x7fab34108650 2024-06-10T01:02:00.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.286+0000 7fab3a1ea700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab34112620 con 0x7fab34102e80 2024-06-10T01:02:00.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.286+0000 7fab3a1ea700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fab34112920 con 0x7fab341024e0 2024-06-10T01:02:00.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.286+0000 7fab33fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab34108650 0x7fab34075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:00.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.286+0000 7fab337fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab341024e0 0x7fab34074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:00.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.286+0000 7fab337fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab341024e0 0x7fab34074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:44418/0 (socket says 172.21.15.37:44418) 2024-06-10T01:02:00.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.286+0000 7fab32ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab34102e80 0x7fab34074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:00.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.286+0000 7fab33fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab34108650 0x7fab34075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47806/0 (socket says 172.21.15.37:47806) 2024-06-10T01:02:00.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.286+0000 7fab337fe700 1 -- 172.21.15.37:0/4282106681 learned_addr learned my addr 172.21.15.37:0/4282106681 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:00.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.287+0000 7fab337fe700 1 -- 172.21.15.37:0/4282106681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab34102e80 msgr2=0x7fab34074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:00.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.287+0000 7fab337fe700 1 --2- 172.21.15.37:0/4282106681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab34102e80 0x7fab34074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.287+0000 7fab337fe700 1 -- 172.21.15.37:0/4282106681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab34108650 msgr2=0x7fab34075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:00.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.287+0000 7fab337fe700 1 --2- 172.21.15.37:0/4282106681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab34108650 0x7fab34075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.287+0000 7fab337fe700 1 -- 172.21.15.37:0/4282106681 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab341a7140 con 0x7fab341024e0 2024-06-10T01:02:00.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.287+0000 7fab32ffd700 1 --2- 172.21.15.37:0/4282106681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab34102e80 0x7fab34074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:00.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.287+0000 7fab337fe700 1 --2- 172.21.15.37:0/4282106681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab341024e0 0x7fab34074790 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fab2400e9a0 tx=0x7fab2400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:00.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.287+0000 7fab30ff9700 1 -- 172.21.15.37:0/4282106681 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2400ca10 con 0x7fab341024e0 2024-06-10T01:02:00.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.287+0000 7fab30ff9700 1 -- 172.21.15.37:0/4282106681 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab2400cbb0 con 0x7fab341024e0 2024-06-10T01:02:00.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.287+0000 7fab3a1ea700 1 -- 172.21.15.37:0/4282106681 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab341a7430 con 0x7fab341024e0 2024-06-10T01:02:00.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.287+0000 7fab30ff9700 1 -- 172.21.15.37:0/4282106681 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab24018680 con 0x7fab341024e0 2024-06-10T01:02:00.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.287+0000 7fab3a1ea700 1 -- 172.21.15.37:0/4282106681 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab341a7990 con 0x7fab341024e0 2024-06-10T01:02:00.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.289+0000 7fab30ff9700 1 -- 172.21.15.37:0/4282106681 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fab24010430 con 0x7fab341024e0 2024-06-10T01:02:00.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.290+0000 7fab30ff9700 1 --2- 172.21.15.37:0/4282106681 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fab1c0a2ca0 0x7fab1c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:00.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.290+0000 7fab32ffd700 1 --2- 172.21.15.37:0/4282106681 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fab1c0a2ca0 0x7fab1c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:00.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.290+0000 7fab30ff9700 1 -- 172.21.15.37:0/4282106681 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fab240cf5c0 con 0x7fab341024e0 2024-06-10T01:02:00.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.291+0000 7fab32ffd700 1 --2- 172.21.15.37:0/4282106681 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fab1c0a2ca0 0x7fab1c0a5160 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fab2800b2e0 tx=0x7fab28005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:00.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.291+0000 7fab44a6db80 1 Processor -- start 2024-06-10T01:02:00.290 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:00 smithi037 ceph-mon[32192]: pgmap v589: 129 pgs: 129 active+clean; 20 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 2.10k op/s 2024-06-10T01:02:00.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.291+0000 7fab44a6db80 1 -- start start 2024-06-10T01:02:00.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.291+0000 7fab44a6db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b03238f60 0x564b03239340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:00.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.291+0000 7fab44a6db80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b03239880 0x564b03233260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:00.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.291+0000 7fab44a6db80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b032337a0 0x564b03233bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:00.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.291+0000 7fab44a6db80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564b03125f60 con 0x564b032337a0 2024-06-10T01:02:00.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.291+0000 7fab44a6db80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564b03184850 con 0x564b03239880 2024-06-10T01:02:00.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.291+0000 7fab44a6db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564b030ec460 con 0x564b03238f60 2024-06-10T01:02:00.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.291+0000 7fab337fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b03238f60 0x564b03239340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:00.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.291+0000 7fab337fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b03238f60 0x564b03239340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:44430/0 (socket says 172.21.15.37:44430) 2024-06-10T01:02:00.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.291+0000 7fab337fe700 1 -- 172.21.15.37:0/3299371592 learned_addr learned my addr 172.21.15.37:0/3299371592 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:00.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.291+0000 7fab33fff700 1 --2- 172.21.15.37:0/3299371592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b032337a0 0x564b03233bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:00.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.291+0000 7fab32ffd700 1 --2- 172.21.15.37:0/3299371592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b03239880 0x564b03233260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:00.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.292+0000 7fab337fe700 1 -- 172.21.15.37:0/3299371592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b03239880 msgr2=0x564b03233260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:00.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.292+0000 7fab337fe700 1 --2- 172.21.15.37:0/3299371592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b03239880 0x564b03233260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.292+0000 7fab337fe700 1 -- 172.21.15.37:0/3299371592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b032337a0 msgr2=0x564b03233bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:00.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.292+0000 7fab337fe700 1 --2- 172.21.15.37:0/3299371592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b032337a0 0x564b03233bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.292+0000 7fab337fe700 1 -- 172.21.15.37:0/3299371592 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b030d38d0 con 0x564b03238f60 2024-06-10T01:02:00.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.292+0000 7fab33fff700 1 --2- 172.21.15.37:0/3299371592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b032337a0 0x564b03233bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:00.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.292+0000 7fab337fe700 1 --2- 172.21.15.37:0/3299371592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b03238f60 0x564b03239340 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fab240983a0 tx=0x7fab2409b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=943ce4e802bf46a9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:00.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.292+0000 7fab19ffb700 1 -- 172.21.15.37:0/3299371592 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab240a4640 con 0x564b03238f60 2024-06-10T01:02:00.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.292+0000 7fab19ffb700 1 -- 172.21.15.37:0/3299371592 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab24099a70 con 0x564b03238f60 2024-06-10T01:02:00.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.292+0000 7fab19ffb700 1 -- 172.21.15.37:0/3299371592 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab240a4410 con 0x564b03238f60 2024-06-10T01:02:00.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.293+0000 7fab44a6db80 1 -- 172.21.15.37:0/3299371592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b03238f60 msgr2=0x564b03239340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:00.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.293+0000 7fab44a6db80 1 --2- 172.21.15.37:0/3299371592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b03238f60 0x564b03239340 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fab240983a0 tx=0x7fab2409b9d0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.293+0000 7fab44a6db80 1 -- 172.21.15.37:0/3299371592 shutdown_connections 2024-06-10T01:02:00.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.293+0000 7fab44a6db80 1 --2- 172.21.15.37:0/3299371592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b032337a0 0x564b03233bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.293+0000 7fab44a6db80 1 --2- 172.21.15.37:0/3299371592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b03238f60 0x564b03239340 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.293+0000 7fab44a6db80 1 --2- 172.21.15.37:0/3299371592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b03239880 0x564b03233260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.293+0000 7fab44a6db80 1 -- 172.21.15.37:0/3299371592 >> 172.21.15.37:0/3299371592 conn(0x564b03160080 msgr2=0x564b0322e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:00.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.293+0000 7fab44a6db80 1 -- 172.21.15.37:0/3299371592 shutdown_connections 2024-06-10T01:02:00.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.293+0000 7fab44a6db80 1 -- 172.21.15.37:0/3299371592 wait complete. 2024-06-10T01:02:00.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.293+0000 7fab44a6db80 1 -- start start 2024-06-10T01:02:00.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.294+0000 7fab44a6db80 1 Processor -- start 2024-06-10T01:02:00.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.294+0000 7fab44a6db80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b032337a0 0x564b0323f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:00.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.294+0000 7fab44a6db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b03238f60 0x564b0323ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:00.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.294+0000 7fab44a6db80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b03239880 0x564b0323cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:00.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.294+0000 7fab44a6db80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564b03124fc0 con 0x564b03239880 2024-06-10T01:02:00.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.294+0000 7fab44a6db80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564b03180810 con 0x564b032337a0 2024-06-10T01:02:00.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.294+0000 7fab44a6db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564b03240260 con 0x564b03238f60 2024-06-10T01:02:00.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.294+0000 7fab44a6db80 10 client.0 _ll_register_callbacks cb 0x564b0311a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:02:00.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.294+0000 7fab32ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b03238f60 0x564b0323ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:00.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.294+0000 7fab33fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b03239880 0x564b0323cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:00.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.294+0000 7fab32ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b03238f60 0x564b0323ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:44434/0 (socket says 172.21.15.37:44434) 2024-06-10T01:02:00.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.294+0000 7fab32ffd700 1 -- 172.21.15.37:0/1556097924 learned_addr learned my addr 172.21.15.37:0/1556097924 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:00.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.294+0000 7fab337fe700 1 --2- 172.21.15.37:0/1556097924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b032337a0 0x564b0323f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:00.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.294+0000 7fab32ffd700 1 -- 172.21.15.37:0/1556097924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b032337a0 msgr2=0x564b0323f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:00.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.294+0000 7fab32ffd700 1 --2- 172.21.15.37:0/1556097924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b032337a0 0x564b0323f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.295+0000 7fab32ffd700 1 -- 172.21.15.37:0/1556097924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b03239880 msgr2=0x564b0323cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:00.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.295+0000 7fab32ffd700 1 --2- 172.21.15.37:0/1556097924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b03239880 0x564b0323cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.295+0000 7fab32ffd700 1 -- 172.21.15.37:0/1556097924 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b0314a750 con 0x564b03238f60 2024-06-10T01:02:00.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.295+0000 7fab337fe700 1 --2- 172.21.15.37:0/1556097924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b032337a0 0x564b0323f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:00.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.295+0000 7fab33fff700 1 --2- 172.21.15.37:0/1556097924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b03239880 0x564b0323cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:00.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.295+0000 7fab32ffd700 1 --2- 172.21.15.37:0/1556097924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b03238f60 0x564b0323ca10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fab28000c00 tx=0x7fab2801a050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86341ff5776e32ab server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:00.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.295+0000 7fab12ffd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:02:00.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.295+0000 7fab12ffd700 1 -- 172.21.15.37:0/1556097924 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28036060 con 0x564b03238f60 2024-06-10T01:02:00.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.295+0000 7fab12ffd700 1 -- 172.21.15.37:0/1556097924 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fab28005440 con 0x564b03238f60 2024-06-10T01:02:00.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.295+0000 7fab44a6db80 1 -- 172.21.15.37:0/1556097924 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab24099a70 con 0x564b03238f60 2024-06-10T01:02:00.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.295+0000 7fab12ffd700 1 -- 172.21.15.37:0/1556097924 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fab2803d050 con 0x564b03238f60 2024-06-10T01:02:00.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.295+0000 7fab44a6db80 10 client.25864 fetch_fsmap learned FSMap version 27 2024-06-10T01:02:00.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.295+0000 7fab44a6db80 1 -- 172.21.15.37:0/1556097924 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564b0324ab40 con 0x564b03238f60 2024-06-10T01:02:00.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.295+0000 7fab12ffd700 1 -- 172.21.15.37:0/1556097924 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab24099a70 con 0x564b03238f60 2024-06-10T01:02:00.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.296+0000 7fab12ffd700 1 -- 172.21.15.37:0/1556097924 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fab280316c0 con 0x564b03238f60 2024-06-10T01:02:00.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.296+0000 7fab44a6db80 10 client.25864 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:02:00.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.296+0000 7fab44a6db80 10 client.25864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:02:00.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.296+0000 7fab44a6db80 20 client.25864 populate_metadata read hostname 'smithi037' 2024-06-10T01:02:00.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.296+0000 7fab44a6db80 1 --2- 172.21.15.37:0/1556097924 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564b0324d1d0 0x564b0324f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:00.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.296+0000 7fab44a6db80 4 client.25864 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:02:00.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.296+0000 7fab44a6db80 1 -- 172.21.15.37:0/1556097924 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b03191d50 con 0x564b0324d1d0 2024-06-10T01:02:00.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.296+0000 7fab337fe700 1 --2- 172.21.15.37:0/1556097924 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564b0324d1d0 0x564b0324f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:00.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.296+0000 7fab12ffd700 10 client.25864 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:00.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.296+0000 7fab337fe700 1 --2- 172.21.15.37:0/1556097924 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564b0324d1d0 0x564b0324f5c0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80c313778b5c16aa server_cookie=e14f2a9e23b4e4eb in_seq=0 out_seq=0 2024-06-10T01:02:00.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:00 smithi094 ceph-mon[36708]: pgmap v589: 129 pgs: 129 active+clean; 20 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 2.10k op/s 2024-06-10T01:02:00.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.429+0000 7fab12ffd700 1 -- 172.21.15.37:0/1556097924 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564b03191d50 con 0x564b0324d1d0 2024-06-10T01:02:00.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.429+0000 7fab12ffd700 10 client.25864 handle_command_reply: tid=0 2024-06-10T01:02:00.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.429+0000 7fab44a6db80 1 client.25864 shutdown 2024-06-10T01:02:00.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.430+0000 7fab44a6db80 1 -- 172.21.15.37:0/1556097924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b03238f60 msgr2=0x564b0323ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:00.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.430+0000 7fab44a6db80 1 --2- 172.21.15.37:0/1556097924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b03238f60 0x564b0323ca10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fab28000c00 tx=0x7fab2801a050 comp rx=0 tx=0).stop 2024-06-10T01:02:00.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.430+0000 7fab44a6db80 1 -- 172.21.15.37:0/1556097924 shutdown_connections 2024-06-10T01:02:00.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.430+0000 7fab44a6db80 1 --2- 172.21.15.37:0/1556097924 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564b0324d1d0 0x564b0324f5c0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.430+0000 7fab44a6db80 1 --2- 172.21.15.37:0/1556097924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b03239880 0x564b0323cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.430+0000 7fab44a6db80 1 --2- 172.21.15.37:0/1556097924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b032337a0 0x564b0323f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.430+0000 7fab44a6db80 1 --2- 172.21.15.37:0/1556097924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b03238f60 0x564b0323ca10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.430+0000 7fab12ffd700 0 client.25864 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:00.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.430+0000 7fab44a6db80 1 -- 172.21.15.37:0/1556097924 >> 172.21.15.37:0/1556097924 conn(0x564b03160080 msgr2=0x564b0322d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:00.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.430+0000 7fab44a6db80 1 -- 172.21.15.37:0/1556097924 shutdown_connections 2024-06-10T01:02:00.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.430+0000 7fab44a6db80 1 -- 172.21.15.37:0/1556097924 wait complete. 2024-06-10T01:02:00.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.430+0000 7fab44a6db80 20 client.25864 trim_cache size 0 max 16384 2024-06-10T01:02:00.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.451+0000 7fab44a6db80 1 Processor -- start 2024-06-10T01:02:00.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.451+0000 7fab44a6db80 1 -- start start 2024-06-10T01:02:00.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.451+0000 7fab44a6db80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b032337a0 0x564b03249380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:00.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.451+0000 7fab44a6db80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b03238f60 0x564b0323f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:00.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.451+0000 7fab44a6db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b03239880 0x564b0322de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:00.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.451+0000 7fab44a6db80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fab341b2ef0 con 0x564b03238f60 2024-06-10T01:02:00.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.451+0000 7fab44a6db80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab341b0780 con 0x564b032337a0 2024-06-10T01:02:00.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.451+0000 7fab44a6db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fab341ae5d0 con 0x564b03239880 2024-06-10T01:02:00.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.451+0000 7fab32ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b03238f60 0x564b0323f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:00.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.451+0000 7fab32ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b03238f60 0x564b0323f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47834/0 (socket says 172.21.15.37:47834) 2024-06-10T01:02:00.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.451+0000 7fab32ffd700 1 -- 172.21.15.37:0/3597011726 learned_addr learned my addr 172.21.15.37:0/3597011726 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:00.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.451+0000 7fab337fe700 1 --2- 172.21.15.37:0/3597011726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b032337a0 0x564b03249380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:00.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.451+0000 7fab33fff700 1 --2- 172.21.15.37:0/3597011726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b03239880 0x564b0322de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:00.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.451+0000 7fab32ffd700 1 -- 172.21.15.37:0/3597011726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b03239880 msgr2=0x564b0322de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:00.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.451+0000 7fab32ffd700 1 --2- 172.21.15.37:0/3597011726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b03239880 0x564b0322de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.451+0000 7fab32ffd700 1 -- 172.21.15.37:0/3597011726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b032337a0 msgr2=0x564b03249380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:00.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.451+0000 7fab32ffd700 1 --2- 172.21.15.37:0/3597011726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b032337a0 0x564b03249380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.452+0000 7fab32ffd700 1 -- 172.21.15.37:0/3597011726 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b0322a810 con 0x564b03238f60 2024-06-10T01:02:00.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.452+0000 7fab337fe700 1 --2- 172.21.15.37:0/3597011726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b032337a0 0x564b03249380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:00.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.452+0000 7fab32ffd700 1 --2- 172.21.15.37:0/3597011726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b03238f60 0x564b0323f250 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fab28031c50 tx=0x7fab2800e590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97afa34039519948 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:00.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.452+0000 7fab197fa700 1 -- 172.21.15.37:0/3597011726 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28036040 con 0x564b03238f60 2024-06-10T01:02:00.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.452+0000 7fab197fa700 1 -- 172.21.15.37:0/3597011726 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab28046070 con 0x564b03238f60 2024-06-10T01:02:00.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.452+0000 7fab197fa700 1 -- 172.21.15.37:0/3597011726 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28004b70 con 0x564b03238f60 2024-06-10T01:02:00.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.453+0000 7fab44a6db80 1 -- 172.21.15.37:0/3597011726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b03238f60 msgr2=0x564b0323f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:00.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.453+0000 7fab44a6db80 1 --2- 172.21.15.37:0/3597011726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b03238f60 0x564b0323f250 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fab28031c50 tx=0x7fab2800e590 comp rx=0 tx=0).stop 2024-06-10T01:02:00.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.453+0000 7fab44a6db80 1 -- 172.21.15.37:0/3597011726 shutdown_connections 2024-06-10T01:02:00.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.453+0000 7fab44a6db80 1 --2- 172.21.15.37:0/3597011726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b03239880 0x564b0322de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.453+0000 7fab44a6db80 1 --2- 172.21.15.37:0/3597011726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b032337a0 0x564b03249380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.453+0000 7fab44a6db80 1 --2- 172.21.15.37:0/3597011726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b03238f60 0x564b0323f250 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.453+0000 7fab44a6db80 1 -- 172.21.15.37:0/3597011726 >> 172.21.15.37:0/3597011726 conn(0x564b03160080 msgr2=0x564b0323cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:00.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.453+0000 7fab44a6db80 1 -- 172.21.15.37:0/3597011726 shutdown_connections 2024-06-10T01:02:00.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.453+0000 7fab44a6db80 1 -- 172.21.15.37:0/3597011726 wait complete. 2024-06-10T01:02:00.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.453+0000 7fab44a6db80 1 -- start start 2024-06-10T01:02:00.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.453+0000 7fab44a6db80 1 Processor -- start 2024-06-10T01:02:00.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.454+0000 7fab44a6db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b032337a0 0x564b03245db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.454+0000 7fab44a6db80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b03238f60 0x564b03254920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.454+0000 7fab44a6db80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b03239880 0x564b03255010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.454+0000 7fab44a6db80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fab341aec40 con 0x564b03238f60 2024-06-10T01:02:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.454+0000 7fab44a6db80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab341b00b0 con 0x564b03239880 2024-06-10T01:02:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.454+0000 7fab44a6db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fab341af310 con 0x564b032337a0 2024-06-10T01:02:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.454+0000 7fab44a6db80 10 client.0 _ll_register_callbacks cb 0x564b0311a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:02:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.454+0000 7fab32ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b03238f60 0x564b03254920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.454+0000 7fab32ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b03238f60 0x564b03254920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47838/0 (socket says 172.21.15.37:47838) 2024-06-10T01:02:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.454+0000 7fab32ffd700 1 -- 172.21.15.37:0/206756962 learned_addr learned my addr 172.21.15.37:0/206756962 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.454+0000 7fab337fe700 1 --2- 172.21.15.37:0/206756962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b032337a0 0x564b03245db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.454+0000 7fab33fff700 1 --2- 172.21.15.37:0/206756962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b03239880 0x564b03255010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.454+0000 7fab32ffd700 1 -- 172.21.15.37:0/206756962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b032337a0 msgr2=0x564b03245db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.454+0000 7fab32ffd700 1 --2- 172.21.15.37:0/206756962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b032337a0 0x564b03245db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.454+0000 7fab32ffd700 1 -- 172.21.15.37:0/206756962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b03239880 msgr2=0x564b03255010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:00.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.454+0000 7fab32ffd700 1 --2- 172.21.15.37:0/206756962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b03239880 0x564b03255010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.454+0000 7fab32ffd700 1 -- 172.21.15.37:0/206756962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b032b6df0 con 0x564b03238f60 2024-06-10T01:02:00.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.454+0000 7fab33fff700 1 --2- 172.21.15.37:0/206756962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b03239880 0x564b03255010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:00.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.454+0000 7fab32ffd700 1 --2- 172.21.15.37:0/206756962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b03238f60 0x564b03254920 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fab28043040 tx=0x7fab28031690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb8d3593f975a58 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:00.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.454+0000 7fab12ffd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:02:00.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.455+0000 7fab12ffd700 1 -- 172.21.15.37:0/206756962 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28005650 con 0x564b03238f60 2024-06-10T01:02:00.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.455+0000 7fab12ffd700 1 -- 172.21.15.37:0/206756962 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fab28004710 con 0x564b03238f60 2024-06-10T01:02:00.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.455+0000 7fab44a6db80 1 -- 172.21.15.37:0/206756962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab28046070 con 0x564b03238f60 2024-06-10T01:02:00.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.455+0000 7fab12ffd700 1 -- 172.21.15.37:0/206756962 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fab2805b050 con 0x564b03238f60 2024-06-10T01:02:00.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.455+0000 7fab44a6db80 10 client.16305 fetch_fsmap learned FSMap version 27 2024-06-10T01:02:00.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.455+0000 7fab44a6db80 1 -- 172.21.15.37:0/206756962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564b032b7fe0 con 0x564b03238f60 2024-06-10T01:02:00.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.455+0000 7fab12ffd700 1 -- 172.21.15.37:0/206756962 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab28046070 con 0x564b03238f60 2024-06-10T01:02:00.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.455+0000 7fab12ffd700 1 -- 172.21.15.37:0/206756962 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fab2800e9b0 con 0x564b03238f60 2024-06-10T01:02:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.455+0000 7fab44a6db80 10 client.16305 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:02:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.455+0000 7fab44a6db80 10 client.16305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:02:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.455+0000 7fab44a6db80 20 client.16305 populate_metadata read hostname 'smithi037' 2024-06-10T01:02:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.455+0000 7fab44a6db80 1 --2- 172.21.15.37:0/206756962 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564b032b8580 0x564b032ba9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.455+0000 7fab44a6db80 4 client.16305 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:02:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.455+0000 7fab44a6db80 1 -- 172.21.15.37:0/206756962 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564b0318c2d0 con 0x564b032b8580 2024-06-10T01:02:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.455+0000 7fab337fe700 1 --2- 172.21.15.37:0/206756962 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564b032b8580 0x564b032ba9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.456+0000 7fab12ffd700 10 client.16305 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.456+0000 7fab337fe700 1 --2- 172.21.15.37:0/206756962 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564b032b8580 0x564b032ba9a0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f058017e0237b117 server_cookie=21e02e018258b6fb in_seq=0 out_seq=0 2024-06-10T01:02:00.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.460+0000 7fab12ffd700 1 -- 172.21.15.37:0/206756962 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x564b0318c2d0 con 0x564b032b8580 2024-06-10T01:02:00.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.460+0000 7fab12ffd700 10 client.16305 handle_command_reply: tid=0 2024-06-10T01:02:00.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.460+0000 7fab44a6db80 1 client.16305 shutdown 2024-06-10T01:02:00.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.460+0000 7fab44a6db80 1 -- 172.21.15.37:0/206756962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b03238f60 msgr2=0x564b03254920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:00.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.460+0000 7fab44a6db80 1 --2- 172.21.15.37:0/206756962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b03238f60 0x564b03254920 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fab28043040 tx=0x7fab28031690 comp rx=0 tx=0).stop 2024-06-10T01:02:00.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.460+0000 7fab44a6db80 1 -- 172.21.15.37:0/206756962 shutdown_connections 2024-06-10T01:02:00.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.460+0000 7fab44a6db80 1 --2- 172.21.15.37:0/206756962 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564b032b8580 0x564b032ba9a0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.460+0000 7fab44a6db80 1 --2- 172.21.15.37:0/206756962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b03239880 0x564b03255010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.460+0000 7fab44a6db80 1 --2- 172.21.15.37:0/206756962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b032337a0 0x564b03245db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.460+0000 7fab12ffd700 0 client.16305 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:00.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.460+0000 7fab44a6db80 1 --2- 172.21.15.37:0/206756962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b03238f60 0x564b03254920 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.460+0000 7fab44a6db80 1 -- 172.21.15.37:0/206756962 >> 172.21.15.37:0/206756962 conn(0x564b03160080 msgr2=0x564b03237ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:00.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.460+0000 7fab44a6db80 1 -- 172.21.15.37:0/206756962 shutdown_connections 2024-06-10T01:02:00.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.461+0000 7fab44a6db80 1 -- 172.21.15.37:0/206756962 wait complete. 2024-06-10T01:02:00.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.461+0000 7fab44a6db80 20 client.16305 trim_cache size 0 max 16384 2024-06-10T01:02:00.462 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:02:00.462 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "scrub active (1760 inodes in the stack)", 2024-06-10T01:02:00.462 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-06-10T01:02:00.462 INFO:teuthology.orchestra.run.smithi037.stdout: "2f56bf32-ff1d-4949-b371-c27f8b6b8de2": { 2024-06-10T01:02:00.462 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-06-10T01:02:00.462 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "2f56bf32-ff1d-4949-b371-c27f8b6b8de2", 2024-06-10T01:02:00.462 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-06-10T01:02:00.462 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:02:00.462 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:02:00.462 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:02:00.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.461+0000 7fab3a1ea700 1 -- 172.21.15.37:0/4282106681 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fab1c0a2ca0 msgr2=0x7fab1c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:00.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.461+0000 7fab3a1ea700 1 --2- 172.21.15.37:0/4282106681 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fab1c0a2ca0 0x7fab1c0a5160 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fab2800b2e0 tx=0x7fab28005960 comp rx=0 tx=0).stop 2024-06-10T01:02:00.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.461+0000 7fab3a1ea700 1 -- 172.21.15.37:0/4282106681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab341024e0 msgr2=0x7fab34074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:00.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.461+0000 7fab3a1ea700 1 --2- 172.21.15.37:0/4282106681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab341024e0 0x7fab34074790 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fab2400e9a0 tx=0x7fab2400ecb0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.461+0000 7fab3a1ea700 1 -- 172.21.15.37:0/4282106681 shutdown_connections 2024-06-10T01:02:00.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.461+0000 7fab3a1ea700 1 --2- 172.21.15.37:0/4282106681 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fab1c0a2ca0 0x7fab1c0a5160 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.461+0000 7fab3a1ea700 1 --2- 172.21.15.37:0/4282106681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab34108650 0x7fab34075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.461+0000 7fab3a1ea700 1 --2- 172.21.15.37:0/4282106681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab341024e0 0x7fab34074790 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.461+0000 7fab3a1ea700 1 --2- 172.21.15.37:0/4282106681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab34102e80 0x7fab34074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:00.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.461+0000 7fab3a1ea700 1 -- 172.21.15.37:0/4282106681 >> 172.21.15.37:0/4282106681 conn(0x7fab340795f0 msgr2=0x7fab341010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:00.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.462+0000 7fab3a1ea700 1 -- 172.21.15.37:0/4282106681 shutdown_connections 2024-06-10T01:02:00.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:00.462+0000 7fab3a1ea700 1 -- 172.21.15.37:0/4282106681 wait complete. 2024-06-10T01:02:00.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:00 smithi167 ceph-mon[36917]: pgmap v589: 129 pgs: 129 active+clean; 20 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 2.10k op/s 2024-06-10T01:02:01.999 INFO:tasks.cephfs.filesystem:scrub status for tag:2f56bf32-ff1d-4949-b371-c27f8b6b8de2 - {'path': '/', 'tag': '2f56bf32-ff1d-4949-b371-c27f8b6b8de2', 'options': 'recursive,force'} 2024-06-10T01:02:02.362 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:02 smithi094 ceph-mon[36708]: pgmap v590: 129 pgs: 129 active+clean; 16 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 2.36k op/s 2024-06-10T01:02:02.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:02 smithi037 ceph-mon[32192]: pgmap v590: 129 pgs: 129 active+clean; 16 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 2.36k op/s 2024-06-10T01:02:02.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:02 smithi167 ceph-mon[36917]: pgmap v590: 129 pgs: 129 active+clean; 16 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 2.36k op/s 2024-06-10T01:02:04.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:04 smithi037 ceph-mon[32192]: pgmap v591: 129 pgs: 129 active+clean; 7.9 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 1.4 MiB/s wr, 2.36k op/s 2024-06-10T01:02:04.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:04 smithi167 ceph-mon[36917]: pgmap v591: 129 pgs: 129 active+clean; 7.9 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 1.4 MiB/s wr, 2.36k op/s 2024-06-10T01:02:04.612 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:04 smithi094 ceph-mon[36708]: pgmap v591: 129 pgs: 129 active+clean; 7.9 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 1.4 MiB/s wr, 2.36k op/s 2024-06-10T01:02:06.388 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:06 smithi167 ceph-mon[36917]: pgmap v592: 129 pgs: 129 active+clean; 13 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 2.54k op/s 2024-06-10T01:02:06.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:06 smithi037 ceph-mon[32192]: pgmap v592: 129 pgs: 129 active+clean; 13 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 2.54k op/s 2024-06-10T01:02:06.413 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:06 smithi094 ceph-mon[36708]: pgmap v592: 129 pgs: 129 active+clean; 13 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 2.54k op/s 2024-06-10T01:02:08.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:08 smithi037 ceph-mon[32192]: pgmap v593: 129 pgs: 129 active+clean; 13 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 1.1 MiB/s wr, 2.42k op/s 2024-06-10T01:02:08.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:08 smithi167 ceph-mon[36917]: pgmap v593: 129 pgs: 129 active+clean; 13 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 1.1 MiB/s wr, 2.42k op/s 2024-06-10T01:02:08.612 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:08 smithi094 ceph-mon[36708]: pgmap v593: 129 pgs: 129 active+clean; 13 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 1.1 MiB/s wr, 2.42k op/s 2024-06-10T01:02:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.245+0000 7f104beba640 1 -- 172.21.15.37:0/829496925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1044100900 msgr2=0x7f1044100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.245+0000 7f104beba640 1 --2- 172.21.15.37:0/829496925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1044100900 0x7f1044100d20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1040009900 tx=0x7f104002a410 comp rx=0 tx=0).stop 2024-06-10T01:02:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.245+0000 7f104beba640 1 -- 172.21.15.37:0/829496925 shutdown_connections 2024-06-10T01:02:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.245+0000 7f104beba640 1 --2- 172.21.15.37:0/829496925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10441024b0 0x7f1044108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.245+0000 7f104beba640 1 --2- 172.21.15.37:0/829496925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1044101af0 0x7f1044101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.245+0000 7f104beba640 1 --2- 172.21.15.37:0/829496925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1044100900 0x7f1044100d20 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.245+0000 7f104beba640 1 -- 172.21.15.37:0/829496925 >> 172.21.15.37:0/829496925 conn(0x7f10440fbe50 msgr2=0x7f10440fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.245+0000 7f104beba640 1 -- 172.21.15.37:0/829496925 shutdown_connections 2024-06-10T01:02:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.245+0000 7f104beba640 1 -- 172.21.15.37:0/829496925 wait complete. 2024-06-10T01:02:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.246+0000 7f104beba640 1 Processor -- start 2024-06-10T01:02:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.246+0000 7f104beba640 1 -- start start 2024-06-10T01:02:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.246+0000 7f104beba640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1044100900 0x7f1044197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.246+0000 7f104beba640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1044101af0 0x7f10441981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.246+0000 7f104beba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10441024b0 0x7f104419f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.246+0000 7f104beba640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f104410d6a0 con 0x7f10441024b0 2024-06-10T01:02:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.246+0000 7f104beba640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f104410d520 con 0x7f1044100900 2024-06-10T01:02:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.246+0000 7f104beba640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f104410d820 con 0x7f1044101af0 2024-06-10T01:02:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.246+0000 7f104a457640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10441024b0 0x7f104419f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.246+0000 7f1049c56640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1044100900 0x7f1044197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.246+0000 7f104a457640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10441024b0 0x7f104419f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44686/0 (socket says 172.21.15.37:44686) 2024-06-10T01:02:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.246+0000 7f104a457640 1 -- 172.21.15.37:0/3213468133 learned_addr learned my addr 172.21.15.37:0/3213468133 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.246+0000 7f1049455640 1 --2- 172.21.15.37:0/3213468133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1044101af0 0x7f10441981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.246+0000 7f104a457640 1 -- 172.21.15.37:0/3213468133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1044101af0 msgr2=0x7f10441981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.246+0000 7f104a457640 1 --2- 172.21.15.37:0/3213468133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1044101af0 0x7f10441981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.247+0000 7f104a457640 1 -- 172.21.15.37:0/3213468133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1044100900 msgr2=0x7f1044197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.247+0000 7f104a457640 1 --2- 172.21.15.37:0/3213468133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1044100900 0x7f1044197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.247+0000 7f104a457640 1 -- 172.21.15.37:0/3213468133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f104419f8c0 con 0x7f10441024b0 2024-06-10T01:02:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.247+0000 7f1049c56640 1 --2- 172.21.15.37:0/3213468133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1044100900 0x7f1044197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.247+0000 7f104a457640 1 --2- 172.21.15.37:0/3213468133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10441024b0 0x7f104419f220 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f103c00eb10 tx=0x7f103c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.247+0000 7f103affd640 1 -- 172.21.15.37:0/3213468133 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c004b20 con 0x7f10441024b0 2024-06-10T01:02:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.247+0000 7f104beba640 1 -- 172.21.15.37:0/3213468133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f104419fbb0 con 0x7f10441024b0 2024-06-10T01:02:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.247+0000 7f104beba640 1 -- 172.21.15.37:0/3213468133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10441a00d0 con 0x7f10441024b0 2024-06-10T01:02:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.247+0000 7f103affd640 1 -- 172.21.15.37:0/3213468133 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f103c004cc0 con 0x7f10441024b0 2024-06-10T01:02:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.247+0000 7f103affd640 1 -- 172.21.15.37:0/3213468133 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c010690 con 0x7f10441024b0 2024-06-10T01:02:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.249+0000 7f103affd640 1 -- 172.21.15.37:0/3213468133 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f103c005000 con 0x7f10441024b0 2024-06-10T01:02:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.250+0000 7f104ebc6740 1 Processor -- start 2024-06-10T01:02:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.250+0000 7f104ebc6740 1 -- start start 2024-06-10T01:02:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.250+0000 7f104ebc6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f809911010 0x55f809911430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.250+0000 7f104ebc6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f809911a00 0x55f809916500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.250+0000 7f104ebc6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f809916b70 0x55f809919000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.250+0000 7f104ebc6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f8098744f0 con 0x55f809911a00 2024-06-10T01:02:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.250+0000 7f104ebc6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f809812870 con 0x55f809916b70 2024-06-10T01:02:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.250+0000 7f104ebc6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f8098704b0 con 0x55f809911010 2024-06-10T01:02:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.250+0000 7f1049c56640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f809911a00 0x55f809916500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.250+0000 7f1049455640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f809911010 0x55f809911430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.250+0000 7f1049c56640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f809911a00 0x55f809916500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44700/0 (socket says 172.21.15.37:44700) 2024-06-10T01:02:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.250+0000 7f1049c56640 1 -- 172.21.15.37:0/3690213407 learned_addr learned my addr 172.21.15.37:0/3690213407 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.250+0000 7f1049455640 1 --2- 172.21.15.37:0/3690213407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f809916b70 0x55f809919000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.250+0000 7f103affd640 1 --2- 172.21.15.37:0/3213468133 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f10300a2c30 0x7f10300a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.251+0000 7f1049455640 1 -- 172.21.15.37:0/3690213407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f809916b70 msgr2=0x55f809919000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.251+0000 7f1049455640 1 --2- 172.21.15.37:0/3690213407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f809916b70 0x55f809919000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.251+0000 7f1049455640 1 -- 172.21.15.37:0/3690213407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f809911a00 msgr2=0x55f809916500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.251+0000 7f1049455640 1 --2- 172.21.15.37:0/3690213407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f809911a00 0x55f809916500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.251+0000 7f1049455640 1 -- 172.21.15.37:0/3690213407 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f80982e9b0 con 0x55f809911010 2024-06-10T01:02:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.251+0000 7f104a457640 1 --2- 172.21.15.37:0/3213468133 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f10300a2c30 0x7f10300a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.251+0000 7f1049c56640 1 --2- 172.21.15.37:0/3690213407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f809911a00 0x55f809916500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.251+0000 7f103affd640 1 -- 172.21.15.37:0/3213468133 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f103c0cf960 con 0x7f10441024b0 2024-06-10T01:02:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.251+0000 7f1049455640 1 --2- 172.21.15.37:0/3690213407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f809911010 0x55f809911430 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1034009730 tx=0x7f10340096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=129c428e573a7d6f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.251+0000 7f104a457640 1 --2- 172.21.15.37:0/3213468133 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f10300a2c30 0x7f10300a50f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f103c005ba0 tx=0x7f103c009a70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.251+0000 7f102ffff640 1 -- 172.21.15.37:0/3690213407 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034004050 con 0x55f809911010 2024-06-10T01:02:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.251+0000 7f102ffff640 1 -- 172.21.15.37:0/3690213407 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1034005500 con 0x55f809911010 2024-06-10T01:02:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.251+0000 7f102ffff640 1 -- 172.21.15.37:0/3690213407 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103400fdb0 con 0x55f809911010 2024-06-10T01:02:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.252+0000 7f104ebc6740 1 -- 172.21.15.37:0/3690213407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f809911010 msgr2=0x55f809911430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.252+0000 7f104ebc6740 1 --2- 172.21.15.37:0/3690213407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f809911010 0x55f809911430 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1034009730 tx=0x7f10340096c0 comp rx=0 tx=0).stop 2024-06-10T01:02:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.252+0000 7f104ebc6740 1 -- 172.21.15.37:0/3690213407 shutdown_connections 2024-06-10T01:02:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.252+0000 7f104ebc6740 1 --2- 172.21.15.37:0/3690213407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f809916b70 0x55f809919000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.252+0000 7f104ebc6740 1 --2- 172.21.15.37:0/3690213407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f809911a00 0x55f809916500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.252+0000 7f104ebc6740 1 --2- 172.21.15.37:0/3690213407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f809911010 0x55f809911430 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.252+0000 7f104ebc6740 1 -- 172.21.15.37:0/3690213407 >> 172.21.15.37:0/3690213407 conn(0x55f80990c680 msgr2=0x55f80990ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.252+0000 7f104ebc6740 1 -- 172.21.15.37:0/3690213407 shutdown_connections 2024-06-10T01:02:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.253+0000 7f104ebc6740 1 -- 172.21.15.37:0/3690213407 wait complete. 2024-06-10T01:02:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.253+0000 7f104ebc6740 1 -- start start 2024-06-10T01:02:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.253+0000 7f104ebc6740 1 Processor -- start 2024-06-10T01:02:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.253+0000 7f104ebc6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f809911010 0x55f80991b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.253+0000 7f104ebc6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f809911a00 0x55f80991b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.253+0000 7f104ebc6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f809916b70 0x55f8099227a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.253+0000 7f104ebc6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f809919ac0 con 0x55f809911010 2024-06-10T01:02:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.253+0000 7f104ebc6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f8097fdae0 con 0x55f809911a00 2024-06-10T01:02:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.253+0000 7f1049455640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f809911010 0x55f80991b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.253+0000 7f104ebc6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f809919c40 con 0x55f809916b70 2024-06-10T01:02:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.254+0000 7f1049c56640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f809911a00 0x55f80991b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.254+0000 7f1049455640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f809911010 0x55f80991b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44714/0 (socket says 172.21.15.37:44714) 2024-06-10T01:02:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.254+0000 7f1049455640 1 -- 172.21.15.37:0/2357615617 learned_addr learned my addr 172.21.15.37:0/2357615617 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.254+0000 7f104ebc6740 10 client.0 _ll_register_callbacks cb 0x55f8098369b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:02:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.254+0000 7f1049455640 1 --2- 172.21.15.37:0/2357615617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f809916b70 0x55f8099227a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.254+0000 7f1049c56640 1 -- 172.21.15.37:0/2357615617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f809916b70 msgr2=0x55f8099227a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.254+0000 7f1049c56640 1 --2- 172.21.15.37:0/2357615617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f809916b70 0x55f8099227a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.254+0000 7f1049c56640 1 -- 172.21.15.37:0/2357615617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f809911010 msgr2=0x55f80991b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.254+0000 7f1049c56640 1 --2- 172.21.15.37:0/2357615617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f809911010 0x55f80991b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.254+0000 7f1049c56640 1 -- 172.21.15.37:0/2357615617 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f809588210 con 0x55f809911a00 2024-06-10T01:02:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.254+0000 7f1049c56640 1 --2- 172.21.15.37:0/2357615617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f809911a00 0x55f80991b760 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55f809912470 tx=0x7f1040031af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b637d4cfbe0cbc3a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.254+0000 7f102d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:02:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.254+0000 7f102d7fa640 1 -- 172.21.15.37:0/2357615617 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040035040 con 0x55f809911a00 2024-06-10T01:02:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.254+0000 7f102d7fa640 1 -- 172.21.15.37:0/2357615617 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f10400042c0 con 0x55f809911a00 2024-06-10T01:02:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.254+0000 7f104ebc6740 1 -- 172.21.15.37:0/2357615617 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1034005500 con 0x55f809911a00 2024-06-10T01:02:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.255+0000 7f102d7fa640 1 -- 172.21.15.37:0/2357615617 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f104004e050 con 0x55f809911a00 2024-06-10T01:02:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.255+0000 7f104ebc6740 10 client.25805 fetch_fsmap learned FSMap version 27 2024-06-10T01:02:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.255+0000 7f104ebc6740 1 -- 172.21.15.37:0/2357615617 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f809654620 con 0x55f809911a00 2024-06-10T01:02:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.255+0000 7f102d7fa640 1 -- 172.21.15.37:0/2357615617 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1034005500 con 0x55f809911a00 2024-06-10T01:02:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.255+0000 7f102d7fa640 1 -- 172.21.15.37:0/2357615617 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f104003d720 con 0x55f809911a00 2024-06-10T01:02:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.255+0000 7f104ebc6740 10 client.25805 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:02:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.255+0000 7f104ebc6740 10 client.25805 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:02:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.255+0000 7f104ebc6740 20 client.25805 populate_metadata read hostname 'smithi037' 2024-06-10T01:02:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.255+0000 7f104ebc6740 1 --2- 172.21.15.37:0/2357615617 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f809927080 0x55f809929520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.255+0000 7f104ebc6740 4 client.25805 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:02:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.255+0000 7f104ebc6740 1 -- 172.21.15.37:0/2357615617 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f809887f10 con 0x55f809927080 2024-06-10T01:02:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.255+0000 7f1049455640 1 --2- 172.21.15.37:0/2357615617 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f809927080 0x55f809929520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.256+0000 7f102d7fa640 10 client.25805 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.256+0000 7f1049455640 1 --2- 172.21.15.37:0/2357615617 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f809927080 0x55f809929520 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=537f04a8221aacce server_cookie=4352ecdee52f9f78 in_seq=0 out_seq=0 2024-06-10T01:02:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.260+0000 7f102d7fa640 1 -- 172.21.15.37:0/2357615617 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f809887f10 con 0x55f809927080 2024-06-10T01:02:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.260+0000 7f102d7fa640 10 client.25805 handle_command_reply: tid=0 2024-06-10T01:02:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.260+0000 7f104ebc6740 1 client.25805 shutdown 2024-06-10T01:02:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.261+0000 7f104ebc6740 1 -- 172.21.15.37:0/2357615617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f809911a00 msgr2=0x55f80991b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.261+0000 7f104ebc6740 1 --2- 172.21.15.37:0/2357615617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f809911a00 0x55f80991b760 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55f809912470 tx=0x7f1040031af0 comp rx=0 tx=0).stop 2024-06-10T01:02:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.261+0000 7f104ebc6740 1 -- 172.21.15.37:0/2357615617 shutdown_connections 2024-06-10T01:02:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.261+0000 7f104ebc6740 1 --2- 172.21.15.37:0/2357615617 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f809927080 0x55f809929520 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.261+0000 7f104ebc6740 1 --2- 172.21.15.37:0/2357615617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f809916b70 0x55f8099227a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.261+0000 7f104ebc6740 1 --2- 172.21.15.37:0/2357615617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f809911a00 0x55f80991b760 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.261+0000 7f104ebc6740 1 --2- 172.21.15.37:0/2357615617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f809911010 0x55f80991b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.261+0000 7f102d7fa640 0 client.25805 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.261+0000 7f104ebc6740 1 -- 172.21.15.37:0/2357615617 >> 172.21.15.37:0/2357615617 conn(0x55f80990c680 msgr2=0x55f80990b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.261+0000 7f104ebc6740 1 -- 172.21.15.37:0/2357615617 shutdown_connections 2024-06-10T01:02:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.261+0000 7f104ebc6740 1 -- 172.21.15.37:0/2357615617 wait complete. 2024-06-10T01:02:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.261+0000 7f104ebc6740 20 client.25805 trim_cache size 0 max 16384 2024-06-10T01:02:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.278+0000 7f104ebc6740 1 Processor -- start 2024-06-10T01:02:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.278+0000 7f104ebc6740 1 -- start start 2024-06-10T01:02:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.278+0000 7f104ebc6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f809911010 0x55f80991a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.278+0000 7f104ebc6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f809911a00 0x55f809922760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.278+0000 7f104ebc6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f809916b70 0x55f809912be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.278+0000 7f104ebc6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1028012610 con 0x55f809911a00 2024-06-10T01:02:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.278+0000 7f104ebc6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f102800fee0 con 0x55f809911010 2024-06-10T01:02:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.278+0000 7f104ebc6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f102800dd30 con 0x55f809916b70 2024-06-10T01:02:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.278+0000 7f1049c56640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f809911a00 0x55f809922760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.278+0000 7f1049c56640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f809911a00 0x55f809922760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44724/0 (socket says 172.21.15.37:44724) 2024-06-10T01:02:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.278+0000 7f1049c56640 1 -- 172.21.15.37:0/680435662 learned_addr learned my addr 172.21.15.37:0/680435662 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.278+0000 7f1049455640 1 --2- 172.21.15.37:0/680435662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f809911010 0x55f80991a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.278+0000 7f1049455640 1 --2- 172.21.15.37:0/680435662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f809916b70 0x55f809912be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.278+0000 7f1049c56640 1 -- 172.21.15.37:0/680435662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f809916b70 msgr2=0x55f809912be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.278+0000 7f1049c56640 1 --2- 172.21.15.37:0/680435662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f809916b70 0x55f809912be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.278+0000 7f1049c56640 1 -- 172.21.15.37:0/680435662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f809911010 msgr2=0x55f80991a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.278+0000 7f1049c56640 1 --2- 172.21.15.37:0/680435662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f809911010 0x55f80991a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.278+0000 7f1049c56640 1 -- 172.21.15.37:0/680435662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f80990c4b0 con 0x55f809911a00 2024-06-10T01:02:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.279+0000 7f1049c56640 1 --2- 172.21.15.37:0/680435662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f809911a00 0x55f809922760 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f104003db40 tx=0x7f104002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb8c63ad0dc4bcd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.279+0000 7f102f7fe640 1 -- 172.21.15.37:0/680435662 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040035030 con 0x55f809911a00 2024-06-10T01:02:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.279+0000 7f102f7fe640 1 -- 172.21.15.37:0/680435662 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1040031b60 con 0x55f809911a00 2024-06-10T01:02:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.279+0000 7f102f7fe640 1 -- 172.21.15.37:0/680435662 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040005450 con 0x55f809911a00 2024-06-10T01:02:10.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.280+0000 7f104ebc6740 1 -- 172.21.15.37:0/680435662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f809911a00 msgr2=0x55f809922760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.280+0000 7f104ebc6740 1 --2- 172.21.15.37:0/680435662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f809911a00 0x55f809922760 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f104003db40 tx=0x7f104002a410 comp rx=0 tx=0).stop 2024-06-10T01:02:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.280+0000 7f104ebc6740 1 -- 172.21.15.37:0/680435662 shutdown_connections 2024-06-10T01:02:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.280+0000 7f104ebc6740 1 --2- 172.21.15.37:0/680435662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f809916b70 0x55f809912be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.280+0000 7f104ebc6740 1 --2- 172.21.15.37:0/680435662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f809911a00 0x55f809922760 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.280+0000 7f104ebc6740 1 --2- 172.21.15.37:0/680435662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f809911010 0x55f80991a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.280+0000 7f104ebc6740 1 -- 172.21.15.37:0/680435662 >> 172.21.15.37:0/680435662 conn(0x55f80990c680 msgr2=0x55f809916250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.280+0000 7f104ebc6740 1 -- 172.21.15.37:0/680435662 shutdown_connections 2024-06-10T01:02:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.280+0000 7f104ebc6740 1 -- 172.21.15.37:0/680435662 wait complete. 2024-06-10T01:02:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.280+0000 7f104ebc6740 1 -- start start 2024-06-10T01:02:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.280+0000 7f104ebc6740 1 Processor -- start 2024-06-10T01:02:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.280+0000 7f104ebc6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f809911010 0x55f80991f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.281+0000 7f104ebc6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f809911a00 0x55f80991f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.281+0000 7f104ebc6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f809916b70 0x55f80998fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.281+0000 7f104ebc6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f102800e3a0 con 0x55f809911a00 2024-06-10T01:02:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.281+0000 7f104ebc6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f102800f810 con 0x55f809911010 2024-06-10T01:02:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.281+0000 7f104ebc6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f102800ea70 con 0x55f809916b70 2024-06-10T01:02:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.281+0000 7f104ebc6740 10 client.0 _ll_register_callbacks cb 0x7f1040009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:02:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.281+0000 7f1049455640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f809911010 0x55f80991f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.281+0000 7f1049c56640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f809911a00 0x55f80991f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.281+0000 7f1049455640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f809911010 0x55f80991f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:55608/0 (socket says 172.21.15.37:55608) 2024-06-10T01:02:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.281+0000 7f1049455640 1 -- 172.21.15.37:0/2377887348 learned_addr learned my addr 172.21.15.37:0/2377887348 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.281+0000 7f1049c56640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f809911a00 0x55f80991f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44732/0 (socket says 172.21.15.37:44732) 2024-06-10T01:02:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.281+0000 7f1049455640 1 --2- 172.21.15.37:0/2377887348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f809916b70 0x55f80998fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.281+0000 7f1049455640 1 -- 172.21.15.37:0/2377887348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f809916b70 msgr2=0x55f80998fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.281+0000 7f1049455640 1 --2- 172.21.15.37:0/2377887348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f809916b70 0x55f80998fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.281+0000 7f1049455640 1 -- 172.21.15.37:0/2377887348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f809911a00 msgr2=0x55f80991f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.281+0000 7f1049455640 1 --2- 172.21.15.37:0/2377887348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f809911a00 0x55f80991f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.281+0000 7f1049455640 1 -- 172.21.15.37:0/2377887348 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f809922930 con 0x55f809911010 2024-06-10T01:02:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.281+0000 7f1049c56640 1 --2- 172.21.15.37:0/2377887348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f809911a00 0x55f80991f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.281+0000 7f1049455640 1 --2- 172.21.15.37:0/2377887348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f809911010 0x55f80991f320 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1034009730 tx=0x7f10340096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ce9c92d3d87316f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.281+0000 7f102d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:02:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.281+0000 7f102d7fa640 1 -- 172.21.15.37:0/2377887348 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034039040 con 0x55f809911010 2024-06-10T01:02:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.282+0000 7f104ebc6740 1 -- 172.21.15.37:0/2377887348 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1040031b60 con 0x55f809911010 2024-06-10T01:02:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.282+0000 7f102d7fa640 1 -- 172.21.15.37:0/2377887348 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f103400fab0 con 0x55f809911010 2024-06-10T01:02:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.282+0000 7f102d7fa640 1 -- 172.21.15.37:0/2377887348 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f103400fcb0 con 0x55f809911010 2024-06-10T01:02:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.282+0000 7f104ebc6740 10 client.25811 fetch_fsmap learned FSMap version 27 2024-06-10T01:02:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.282+0000 7f104ebc6740 1 -- 172.21.15.37:0/2377887348 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f809991290 con 0x55f809911010 2024-06-10T01:02:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.282+0000 7f102d7fa640 1 -- 172.21.15.37:0/2377887348 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1040031b60 con 0x55f809911010 2024-06-10T01:02:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.282+0000 7f102d7fa640 1 -- 172.21.15.37:0/2377887348 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f10340354e0 con 0x55f809911010 2024-06-10T01:02:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.282+0000 7f104ebc6740 10 client.25811 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:02:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.282+0000 7f104ebc6740 10 client.25811 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:02:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.282+0000 7f104ebc6740 20 client.25811 populate_metadata read hostname 'smithi037' 2024-06-10T01:02:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.282+0000 7f104ebc6740 1 --2- 172.21.15.37:0/2377887348 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f809991a80 0x55f809993ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.282+0000 7f104ebc6740 4 client.25811 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:02:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.282+0000 7f104ebc6740 1 -- 172.21.15.37:0/2377887348 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f809881c20 con 0x55f809991a80 2024-06-10T01:02:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.282+0000 7f1049c56640 1 --2- 172.21.15.37:0/2377887348 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f809991a80 0x55f809993ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.283+0000 7f102d7fa640 10 client.25811 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:10.283+0000 7f1049c56640 1 --2- 172.21.15.37:0/2377887348 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f809991a80 0x55f809993ea0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17f5ef49c6a5d5 server_cookie=944779976fe40fef in_seq=0 out_seq=0 2024-06-10T01:02:10.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:10 smithi037 ceph-mon[32192]: pgmap v594: 129 pgs: 129 active+clean; 19 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 2.58k op/s 2024-06-10T01:02:10.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:10 smithi167 ceph-mon[36917]: pgmap v594: 129 pgs: 129 active+clean; 19 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 2.58k op/s 2024-06-10T01:02:10.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:10 smithi094 ceph-mon[36708]: pgmap v594: 129 pgs: 129 active+clean; 19 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 2.58k op/s 2024-06-10T01:02:12.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:12 smithi037 ceph-mon[32192]: pgmap v595: 129 pgs: 129 active+clean; 20 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.5 MiB/s wr, 821 op/s 2024-06-10T01:02:12.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:12 smithi167 ceph-mon[36917]: pgmap v595: 129 pgs: 129 active+clean; 20 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.5 MiB/s wr, 821 op/s 2024-06-10T01:02:12.612 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:12 smithi094 ceph-mon[36708]: pgmap v595: 129 pgs: 129 active+clean; 20 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.5 MiB/s wr, 821 op/s 2024-06-10T01:02:13.532 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:02:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-06-10T01:02:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24472, 2024-06-10T01:02:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 10, 2024-06-10T01:02:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-06-10T01:02:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: "31515789": { 2024-06-10T01:02:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 10, 2024-06-10T01:02:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-06-10T01:02:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:02:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-06-10T01:02:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:02:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:02:13.534 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-06-10T01:02:13.534 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 138.2, 2024-06-10T01:02:13.534 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-06-10T01:02:13.534 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-06-10T01:02:13.534 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-06-10T01:02:13.534 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:02:13.535 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-06-10T01:02:13.535 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:02:13.535 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:02:13.535 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:02:13.535 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:02:13.535 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:02:13.536 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:02:13.536 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:02:13.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:13.533+0000 7efd7dffb700 1 -- 172.21.15.37:0/4044983258 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7efd8409dd70 con 0x55b1f12e9be0 2024-06-10T01:02:13.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:13.533+0000 7efd7dffb700 10 client.25760 handle_command_reply: tid=0 2024-06-10T01:02:13.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:13.533+0000 7efda6ddbb80 1 client.25760 shutdown 2024-06-10T01:02:13.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:13.533+0000 7efda6ddbb80 1 -- 172.21.15.37:0/4044983258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1f1248770 msgr2=0x55b1f1264470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:13.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:13.533+0000 7efda6ddbb80 1 --2- 172.21.15.37:0/4044983258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1f1248770 0x55b1f1264470 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efd90004790 tx=0x7efd90004320 comp rx=0 tx=0).stop 2024-06-10T01:02:13.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:13.533+0000 7efda6ddbb80 1 -- 172.21.15.37:0/4044983258 shutdown_connections 2024-06-10T01:02:13.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:13.533+0000 7efda6ddbb80 1 --2- 172.21.15.37:0/4044983258 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b1f12e9be0 0x55b1f12ebfd0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:13.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:13.533+0000 7efda6ddbb80 1 --2- 172.21.15.37:0/4044983258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b1f12610c0 0x55b1f1261540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:13.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:13.533+0000 7efda6ddbb80 1 --2- 172.21.15.37:0/4044983258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1f1248770 0x55b1f1264470 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:13.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:13.533+0000 7efd7dffb700 0 client.25760 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:13.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:13.533+0000 7efda6ddbb80 1 --2- 172.21.15.37:0/4044983258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b1f125c310 0x55b1f1260b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:13.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:13.533+0000 7efda6ddbb80 1 -- 172.21.15.37:0/4044983258 >> 172.21.15.37:0/4044983258 conn(0x55b1f116be50 msgr2=0x55b1f125de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:13.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:13.533+0000 7efda6ddbb80 1 -- 172.21.15.37:0/4044983258 shutdown_connections 2024-06-10T01:02:13.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:13.533+0000 7efda6ddbb80 1 -- 172.21.15.37:0/4044983258 wait complete. 2024-06-10T01:02:13.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:13.534+0000 7efda6ddbb80 20 client.25760 trim_cache size 0 max 16384 2024-06-10T01:02:13.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:13.534+0000 7efd9c558700 1 -- 172.21.15.37:0/615565899 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7efd800a2e10 msgr2=0x7efd800a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:13.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:13.534+0000 7efd9c558700 1 --2- 172.21.15.37:0/615565899 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7efd800a2e10 0x7efd800a52d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efd8c03d000 tx=0x7efd8c03f000 comp rx=0 tx=0).stop 2024-06-10T01:02:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:13.534+0000 7efd9c558700 1 -- 172.21.15.37:0/615565899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd94109460 msgr2=0x7efd9419f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:13.534+0000 7efd9c558700 1 --2- 172.21.15.37:0/615565899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd94109460 0x7efd9419f760 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7efd84005d90 tx=0x7efd840052c0 comp rx=0 tx=0).stop 2024-06-10T01:02:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:13.534+0000 7efd9c558700 1 -- 172.21.15.37:0/615565899 shutdown_connections 2024-06-10T01:02:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:13.534+0000 7efd9c558700 1 --2- 172.21.15.37:0/615565899 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7efd800a2e10 0x7efd800a52d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:13.534+0000 7efd9c558700 1 --2- 172.21.15.37:0/615565899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd9410eaa0 0x7efd941a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:13.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:13.534+0000 7efd9c558700 1 --2- 172.21.15.37:0/615565899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd94108aa0 0x7efd9419f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:13.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:13.534+0000 7efd9c558700 1 --2- 172.21.15.37:0/615565899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd94109460 0x7efd9419f760 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:13.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:13.534+0000 7efd9c558700 1 -- 172.21.15.37:0/615565899 >> 172.21.15.37:0/615565899 conn(0x7efd94075c60 msgr2=0x7efd940760e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:13.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:13.534+0000 7efd9c558700 1 -- 172.21.15.37:0/615565899 shutdown_connections 2024-06-10T01:02:13.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:13.534+0000 7efd9c558700 1 -- 172.21.15.37:0/615565899 wait complete. 2024-06-10T01:02:13.570 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:13.571+0000 7f102d7fa640 1 -- 172.21.15.37:0/2377887348 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f809881c20 con 0x55f809991a80 2024-06-10T01:02:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:13.571+0000 7f102d7fa640 10 client.25811 handle_command_reply: tid=0 2024-06-10T01:02:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:13.571+0000 7f104ebc6740 1 client.25811 shutdown 2024-06-10T01:02:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:13.571+0000 7f104ebc6740 1 -- 172.21.15.37:0/2377887348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f809911010 msgr2=0x55f80991f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:13.571+0000 7f104ebc6740 1 --2- 172.21.15.37:0/2377887348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f809911010 0x55f80991f320 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1034009730 tx=0x7f10340096c0 comp rx=0 tx=0).stop 2024-06-10T01:02:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:13.571+0000 7f104ebc6740 1 -- 172.21.15.37:0/2377887348 shutdown_connections 2024-06-10T01:02:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:13.571+0000 7f104ebc6740 1 --2- 172.21.15.37:0/2377887348 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f809991a80 0x55f809993ea0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:13.571+0000 7f102d7fa640 0 client.25811 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:13.571+0000 7f104ebc6740 1 --2- 172.21.15.37:0/2377887348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f809916b70 0x55f80998fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:13.571+0000 7f104ebc6740 1 --2- 172.21.15.37:0/2377887348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f809911a00 0x55f80991f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:13.571+0000 7f104ebc6740 1 --2- 172.21.15.37:0/2377887348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f809911010 0x55f80991f320 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:13.571+0000 7f104ebc6740 1 -- 172.21.15.37:0/2377887348 >> 172.21.15.37:0/2377887348 conn(0x55f80990c680 msgr2=0x55f809915e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:13.571+0000 7f104ebc6740 1 -- 172.21.15.37:0/2377887348 shutdown_connections 2024-06-10T01:02:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:13.571+0000 7f104ebc6740 1 -- 172.21.15.37:0/2377887348 wait complete. 2024-06-10T01:02:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:13.572+0000 7f104ebc6740 20 client.25811 trim_cache size 0 max 16384 2024-06-10T01:02:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:02:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:02:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:02:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:13.572+0000 7f104beba640 1 -- 172.21.15.37:0/3213468133 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f10300a2c30 msgr2=0x7f10300a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:13.572+0000 7f104beba640 1 --2- 172.21.15.37:0/3213468133 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f10300a2c30 0x7f10300a50f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f103c005ba0 tx=0x7f103c009a70 comp rx=0 tx=0).stop 2024-06-10T01:02:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:13.572+0000 7f104beba640 1 -- 172.21.15.37:0/3213468133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10441024b0 msgr2=0x7f104419f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:13.572+0000 7f104beba640 1 --2- 172.21.15.37:0/3213468133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10441024b0 0x7f104419f220 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f103c00eb10 tx=0x7f103c00c2d0 comp rx=0 tx=0).stop 2024-06-10T01:02:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:13.572+0000 7f104beba640 1 -- 172.21.15.37:0/3213468133 shutdown_connections 2024-06-10T01:02:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:13.572+0000 7f104beba640 1 --2- 172.21.15.37:0/3213468133 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f10300a2c30 0x7f10300a50f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:13.572+0000 7f104beba640 1 --2- 172.21.15.37:0/3213468133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10441024b0 0x7f104419f220 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:13.572+0000 7f104beba640 1 --2- 172.21.15.37:0/3213468133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1044101af0 0x7f10441981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:13.572+0000 7f104beba640 1 --2- 172.21.15.37:0/3213468133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1044100900 0x7f1044197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:13.572+0000 7f104beba640 1 -- 172.21.15.37:0/3213468133 >> 172.21.15.37:0/3213468133 conn(0x7f10440fbe50 msgr2=0x7f1044104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:13.572+0000 7f104beba640 1 -- 172.21.15.37:0/3213468133 shutdown_connections 2024-06-10T01:02:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:13.572+0000 7f104beba640 1 -- 172.21.15.37:0/3213468133 wait complete. 2024-06-10T01:02:14.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:14 smithi037 ceph-mon[32192]: pgmap v596: 129 pgs: 129 active+clean; 21 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 1.5 MiB/s wr, 629 op/s 2024-06-10T01:02:14.302 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '31515789', quiesce took 26.2 seconds. Will release after: -8.0 2024-06-10T01:02:14.303 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:02:14.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:14 smithi167 ceph-mon[36917]: pgmap v596: 129 pgs: 129 active+clean; 21 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 1.5 MiB/s wr, 629 op/s 2024-06-10T01:02:14.612 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:14 smithi094 ceph-mon[36708]: pgmap v596: 129 pgs: 129 active+clean; 21 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 1.5 MiB/s wr, 629 op/s 2024-06-10T01:02:14.633 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:02:16.337 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:16 smithi037 ceph-mon[32192]: scrub summary: idle+waiting paths [/] 2024-06-10T01:02:16.337 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:16 smithi037 ceph-mon[32192]: pgmap v597: 129 pgs: 129 active+clean; 13 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 1.15k op/s 2024-06-10T01:02:16.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.444+0000 7f3566132700 1 -- 172.21.15.37:0/96031672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3560107840 msgr2=0x7f356010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:16.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.444+0000 7f3566132700 1 --2- 172.21.15.37:0/96031672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3560107840 0x7f356010f3c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f355800b0d0 tx=0x7f355800b3e0 comp rx=0 tx=0).stop 2024-06-10T01:02:16.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.444+0000 7f3566132700 1 -- 172.21.15.37:0/96031672 shutdown_connections 2024-06-10T01:02:16.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.444+0000 7f3566132700 1 --2- 172.21.15.37:0/96031672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3560107840 0x7f356010f3c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:16.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.444+0000 7f3566132700 1 --2- 172.21.15.37:0/96031672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3560102740 0x7f3560102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:16.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.444+0000 7f3566132700 1 --2- 172.21.15.37:0/96031672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35601030f0 0x7f3560107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:16.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.444+0000 7f3566132700 1 -- 172.21.15.37:0/96031672 >> 172.21.15.37:0/96031672 conn(0x7f35600795f0 msgr2=0x7f3560079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:16.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.445+0000 7f3566132700 1 -- 172.21.15.37:0/96031672 shutdown_connections 2024-06-10T01:02:16.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.445+0000 7f3566132700 1 -- 172.21.15.37:0/96031672 wait complete. 2024-06-10T01:02:16.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.445+0000 7f3566132700 1 Processor -- start 2024-06-10T01:02:16.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.445+0000 7f3566132700 1 -- start start 2024-06-10T01:02:16.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.446+0000 7f3566132700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3560102740 0x7f356019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:16.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.446+0000 7f3566132700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35601030f0 0x7f356019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:16.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.446+0000 7f3566132700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3560107840 0x7f3560198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:16.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.446+0000 7f3566132700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3560112760 con 0x7f3560102740 2024-06-10T01:02:16.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.446+0000 7f3566132700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f35601125e0 con 0x7f35601030f0 2024-06-10T01:02:16.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.446+0000 7f3566132700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f35601128e0 con 0x7f3560107840 2024-06-10T01:02:16.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.446+0000 7f355f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3560102740 0x7f356019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:16.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.446+0000 7f355f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3560102740 0x7f356019e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35612/0 (socket says 172.21.15.37:35612) 2024-06-10T01:02:16.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.446+0000 7f355f7fe700 1 -- 172.21.15.37:0/1898403983 learned_addr learned my addr 172.21.15.37:0/1898403983 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:16.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.446+0000 7f355ffff700 1 --2- 172.21.15.37:0/1898403983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3560107840 0x7f3560198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:16.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.446+0000 7f3556ffd700 1 --2- 172.21.15.37:0/1898403983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35601030f0 0x7f356019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:16.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.446+0000 7f355f7fe700 1 -- 172.21.15.37:0/1898403983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3560107840 msgr2=0x7f3560198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:16.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.446+0000 7f355f7fe700 1 --2- 172.21.15.37:0/1898403983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3560107840 0x7f3560198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:16.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.446+0000 7f355f7fe700 1 -- 172.21.15.37:0/1898403983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35601030f0 msgr2=0x7f356019ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:16.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.446+0000 7f355f7fe700 1 --2- 172.21.15.37:0/1898403983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35601030f0 0x7f356019ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:16.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.446+0000 7f355f7fe700 1 -- 172.21.15.37:0/1898403983 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3560199080 con 0x7f3560102740 2024-06-10T01:02:16.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.446+0000 7f355f7fe700 1 --2- 172.21.15.37:0/1898403983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3560102740 0x7f356019e710 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f355000c9b0 tx=0x7f355000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:16.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.447+0000 7f355d7fa700 1 -- 172.21.15.37:0/1898403983 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3550007b00 con 0x7f3560102740 2024-06-10T01:02:16.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.447+0000 7f355d7fa700 1 -- 172.21.15.37:0/1898403983 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3550007ca0 con 0x7f3560102740 2024-06-10T01:02:16.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.447+0000 7f3566132700 1 -- 172.21.15.37:0/1898403983 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3560199370 con 0x7f3560102740 2024-06-10T01:02:16.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.447+0000 7f355d7fa700 1 -- 172.21.15.37:0/1898403983 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3550018780 con 0x7f3560102740 2024-06-10T01:02:16.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.447+0000 7f3566132700 1 -- 172.21.15.37:0/1898403983 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35601a55e0 con 0x7f3560102740 2024-06-10T01:02:16.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.448+0000 7f3566132700 1 -- 172.21.15.37:0/1898403983 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3560069180 con 0x7f3560102740 2024-06-10T01:02:16.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.449+0000 7f355d7fa700 1 -- 172.21.15.37:0/1898403983 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f355000f450 con 0x7f3560102740 2024-06-10T01:02:16.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.450+0000 7f355d7fa700 1 --2- 172.21.15.37:0/1898403983 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f354c0a2ca0 0x7f354c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:16.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.450+0000 7f3556ffd700 1 --2- 172.21.15.37:0/1898403983 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f354c0a2ca0 0x7f354c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:16.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.450+0000 7f355d7fa700 1 -- 172.21.15.37:0/1898403983 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f35500cf740 con 0x7f3560102740 2024-06-10T01:02:16.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.450+0000 7f3556ffd700 1 --2- 172.21.15.37:0/1898403983 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f354c0a2ca0 0x7f354c0a5160 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3560108c20 tx=0x7f3548009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:16.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.453+0000 7f355d7fa700 1 -- 172.21.15.37:0/1898403983 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3550014030 con 0x7f3560102740 2024-06-10T01:02:16.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:16 smithi167 ceph-mon[36917]: scrub summary: idle+waiting paths [/] 2024-06-10T01:02:16.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:16 smithi167 ceph-mon[36917]: pgmap v597: 129 pgs: 129 active+clean; 13 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 1.15k op/s 2024-06-10T01:02:16.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:16 smithi094 ceph-mon[36708]: scrub summary: idle+waiting paths [/] 2024-06-10T01:02:16.612 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:16 smithi094 ceph-mon[36708]: pgmap v597: 129 pgs: 129 active+clean; 13 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 1.15k op/s 2024-06-10T01:02:16.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.709+0000 7f3566132700 1 -- 172.21.15.37:0/1898403983 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f356002dca0 con 0x7f3560102740 2024-06-10T01:02:16.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.710+0000 7f355d7fa700 1 -- 172.21.15.37:0/1898403983 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11867 (secure 0 0 0) 0x7f355009c360 con 0x7f3560102740 2024-06-10T01:02:16.708 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:02:16.708 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T00:56:30.291171+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24451,24458,24472]},"id":1}]} 2024-06-10T01:02:16.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.712+0000 7f3566132700 1 -- 172.21.15.37:0/1898403983 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f354c0a2ca0 msgr2=0x7f354c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:16.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.712+0000 7f3566132700 1 --2- 172.21.15.37:0/1898403983 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f354c0a2ca0 0x7f354c0a5160 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3560108c20 tx=0x7f3548009260 comp rx=0 tx=0).stop 2024-06-10T01:02:16.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.712+0000 7f3566132700 1 -- 172.21.15.37:0/1898403983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3560102740 msgr2=0x7f356019e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:16.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.712+0000 7f3566132700 1 --2- 172.21.15.37:0/1898403983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3560102740 0x7f356019e710 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f355000c9b0 tx=0x7f355000cd70 comp rx=0 tx=0).stop 2024-06-10T01:02:16.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.712+0000 7f3566132700 1 -- 172.21.15.37:0/1898403983 shutdown_connections 2024-06-10T01:02:16.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.712+0000 7f3566132700 1 --2- 172.21.15.37:0/1898403983 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f354c0a2ca0 0x7f354c0a5160 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:16.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.712+0000 7f3566132700 1 --2- 172.21.15.37:0/1898403983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3560107840 0x7f3560198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:16.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.712+0000 7f3566132700 1 --2- 172.21.15.37:0/1898403983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3560102740 0x7f356019e710 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:16.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.712+0000 7f3566132700 1 --2- 172.21.15.37:0/1898403983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35601030f0 0x7f356019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:16.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.712+0000 7f3566132700 1 -- 172.21.15.37:0/1898403983 >> 172.21.15.37:0/1898403983 conn(0x7f35600795f0 msgr2=0x7f3560100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:16.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.712+0000 7f3566132700 1 -- 172.21.15.37:0/1898403983 shutdown_connections 2024-06-10T01:02:16.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:16.712+0000 7f3566132700 1 -- 172.21.15.37:0/1898403983 wait complete. 2024-06-10T01:02:16.711 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-06-10T01:02:17.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:17 smithi037 ceph-mon[32192]: scrub summary: idle 2024-06-10T01:02:17.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:17 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1898403983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:02:17.522 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24472 quiesce db --set-id 31515789 --release --await' 2024-06-10T01:02:17.522 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24472 quiesce db --set-id 31515789 --release --await 2024-06-10T01:02:17.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:17 smithi167 ceph-mon[36917]: scrub summary: idle 2024-06-10T01:02:17.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:17 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1898403983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:02:17.612 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:17 smithi094 ceph-mon[36708]: scrub summary: idle 2024-06-10T01:02:17.612 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:17 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1898403983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:02:17.774 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:02:18.172 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:18 smithi037 ceph-mon[32192]: pgmap v598: 129 pgs: 129 active+clean; 13 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 1.04k op/s 2024-06-10T01:02:18.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:18 smithi167 ceph-mon[36917]: pgmap v598: 129 pgs: 129 active+clean; 13 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 1.04k op/s 2024-06-10T01:02:18.612 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:18 smithi094 ceph-mon[36708]: pgmap v598: 129 pgs: 129 active+clean; 13 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 1.04k op/s 2024-06-10T01:02:19.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.787+0000 7f0754986700 1 -- 172.21.15.37:0/1206900638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07501066b0 msgr2=0x7f075010e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:19.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.787+0000 7f0754986700 1 --2- 172.21.15.37:0/1206900638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07501066b0 0x7f075010e280 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f074800b0d0 tx=0x7f074800b3e0 comp rx=0 tx=0).stop 2024-06-10T01:02:19.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.788+0000 7f0754986700 1 -- 172.21.15.37:0/1206900638 shutdown_connections 2024-06-10T01:02:19.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.788+0000 7f0754986700 1 --2- 172.21.15.37:0/1206900638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07501066b0 0x7f075010e280 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:19.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.788+0000 7f0754986700 1 --2- 172.21.15.37:0/1206900638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0750101620 0x7f0750101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:19.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.788+0000 7f0754986700 1 --2- 172.21.15.37:0/1206900638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0750101fd0 0x7f0750105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:19.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.788+0000 7f0754986700 1 -- 172.21.15.37:0/1206900638 >> 172.21.15.37:0/1206900638 conn(0x7f0750078340 msgr2=0x7f0750078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:19.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.788+0000 7f0754986700 1 -- 172.21.15.37:0/1206900638 shutdown_connections 2024-06-10T01:02:19.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.788+0000 7f0754986700 1 -- 172.21.15.37:0/1206900638 wait complete. 2024-06-10T01:02:19.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.788+0000 7f0754986700 1 Processor -- start 2024-06-10T01:02:19.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.788+0000 7f0754986700 1 -- start start 2024-06-10T01:02:19.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.789+0000 7f0754986700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0750101620 0x7f075019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:19.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.789+0000 7f0754986700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0750101fd0 0x7f075019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:19.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.789+0000 7f0754986700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07501066b0 0x7f075019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:19.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.789+0000 7f0754986700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f07501116f0 con 0x7f0750101fd0 2024-06-10T01:02:19.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.789+0000 7f0754986700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0750111570 con 0x7f0750101620 2024-06-10T01:02:19.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.789+0000 7f0754986700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0750111870 con 0x7f07501066b0 2024-06-10T01:02:19.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.789+0000 7f074dd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0750101fd0 0x7f075019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:19.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.789+0000 7f074e59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0750101620 0x7f075019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:19.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.789+0000 7f074dd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0750101fd0 0x7f075019b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35622/0 (socket says 172.21.15.37:35622) 2024-06-10T01:02:19.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.789+0000 7f074dd9b700 1 -- 172.21.15.37:0/4173289513 learned_addr learned my addr 172.21.15.37:0/4173289513 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:19.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.789+0000 7f074ed9d700 1 --2- 172.21.15.37:0/4173289513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07501066b0 0x7f075019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:19.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.789+0000 7f074dd9b700 1 -- 172.21.15.37:0/4173289513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07501066b0 msgr2=0x7f075019f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:19.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.789+0000 7f074dd9b700 1 --2- 172.21.15.37:0/4173289513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07501066b0 0x7f075019f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:19.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.789+0000 7f074dd9b700 1 -- 172.21.15.37:0/4173289513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0750101620 msgr2=0x7f075019ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:19.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.789+0000 7f074dd9b700 1 --2- 172.21.15.37:0/4173289513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0750101620 0x7f075019ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:19.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.789+0000 7f074dd9b700 1 -- 172.21.15.37:0/4173289513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075019fdd0 con 0x7f0750101fd0 2024-06-10T01:02:19.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.790+0000 7f074dd9b700 1 --2- 172.21.15.37:0/4173289513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0750101fd0 0x7f075019b350 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f074400c8f0 tx=0x7f074400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:19.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.790+0000 7f073f7fe700 1 -- 172.21.15.37:0/4173289513 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07440078d0 con 0x7f0750101fd0 2024-06-10T01:02:19.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.790+0000 7f0754986700 1 -- 172.21.15.37:0/4173289513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07501a00c0 con 0x7f0750101fd0 2024-06-10T01:02:19.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.790+0000 7f073f7fe700 1 -- 172.21.15.37:0/4173289513 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f074400f450 con 0x7f0750101fd0 2024-06-10T01:02:19.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.790+0000 7f073f7fe700 1 -- 172.21.15.37:0/4173289513 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0744018640 con 0x7f0750101fd0 2024-06-10T01:02:19.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.790+0000 7f0754986700 1 -- 172.21.15.37:0/4173289513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07501a7940 con 0x7f0750101fd0 2024-06-10T01:02:19.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.793+0000 7f073f7fe700 1 -- 172.21.15.37:0/4173289513 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f074400f600 con 0x7f0750101fd0 2024-06-10T01:02:19.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.793+0000 7f073f7fe700 1 --2- 172.21.15.37:0/4173289513 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f07380a2c20 0x7f07380a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:19.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.794+0000 7f073f7fe700 1 -- 172.21.15.37:0/4173289513 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f07440cf200 con 0x7f0750101fd0 2024-06-10T01:02:19.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.794+0000 7f074e59c700 1 --2- 172.21.15.37:0/4173289513 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f07380a2c20 0x7f07380a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:19.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.794+0000 7f074e59c700 1 --2- 172.21.15.37:0/4173289513 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f07380a2c20 0x7f07380a50e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f074000ac50 tx=0x7f074000a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:19.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.794+0000 7f075f209b80 1 Processor -- start 2024-06-10T01:02:19.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.794+0000 7f075f209b80 1 -- start start 2024-06-10T01:02:19.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.795+0000 7f075f209b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b58f426500 0x55b58f4268e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:19.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.795+0000 7f075f209b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b58f426e20 0x55b58f420710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:19.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.795+0000 7f075f209b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b58f420c50 0x55b58f421070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:19.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.795+0000 7f075f209b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b58f313060 con 0x55b58f426e20 2024-06-10T01:02:19.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.795+0000 7f075f209b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b58f371950 con 0x55b58f420c50 2024-06-10T01:02:19.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.795+0000 7f075f209b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b58f2d9560 con 0x55b58f426500 2024-06-10T01:02:19.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.795+0000 7f074dd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b58f426e20 0x55b58f420710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:19.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.795+0000 7f074dd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b58f426e20 0x55b58f420710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35630/0 (socket says 172.21.15.37:35630) 2024-06-10T01:02:19.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.795+0000 7f074dd9b700 1 -- 172.21.15.37:0/3393233693 learned_addr learned my addr 172.21.15.37:0/3393233693 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:19.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.795+0000 7f074e59c700 1 --2- 172.21.15.37:0/3393233693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b58f426500 0x55b58f4268e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:19.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.795+0000 7f074ed9d700 1 --2- 172.21.15.37:0/3393233693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b58f420c50 0x55b58f421070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:19.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.795+0000 7f074dd9b700 1 -- 172.21.15.37:0/3393233693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b58f426500 msgr2=0x55b58f4268e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:19.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.795+0000 7f074dd9b700 1 --2- 172.21.15.37:0/3393233693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b58f426500 0x55b58f4268e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:19.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.795+0000 7f074dd9b700 1 -- 172.21.15.37:0/3393233693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b58f420c50 msgr2=0x55b58f421070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:19.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.795+0000 7f074dd9b700 1 --2- 172.21.15.37:0/3393233693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b58f420c50 0x55b58f421070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:19.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.795+0000 7f074dd9b700 1 -- 172.21.15.37:0/3393233693 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b58f2c09d0 con 0x55b58f426e20 2024-06-10T01:02:19.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.795+0000 7f074dd9b700 1 --2- 172.21.15.37:0/3393233693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b58f426e20 0x55b58f420710 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f07440a3db0 tx=0x7f074409b0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c2959b04d0be1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:19.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.796+0000 7f073cff9700 1 -- 172.21.15.37:0/3393233693 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07440b4070 con 0x55b58f426e20 2024-06-10T01:02:19.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.796+0000 7f073cff9700 1 -- 172.21.15.37:0/3393233693 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f074409fa20 con 0x55b58f426e20 2024-06-10T01:02:19.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.796+0000 7f073cff9700 1 -- 172.21.15.37:0/3393233693 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074409b8d0 con 0x55b58f426e20 2024-06-10T01:02:19.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.796+0000 7f075f209b80 1 -- 172.21.15.37:0/3393233693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b58f426e20 msgr2=0x55b58f420710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:19.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.796+0000 7f075f209b80 1 --2- 172.21.15.37:0/3393233693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b58f426e20 0x55b58f420710 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f07440a3db0 tx=0x7f074409b0c0 comp rx=0 tx=0).stop 2024-06-10T01:02:19.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.796+0000 7f075f209b80 1 -- 172.21.15.37:0/3393233693 shutdown_connections 2024-06-10T01:02:19.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.796+0000 7f075f209b80 1 --2- 172.21.15.37:0/3393233693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b58f420c50 0x55b58f421070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:19.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.796+0000 7f075f209b80 1 --2- 172.21.15.37:0/3393233693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b58f426500 0x55b58f4268e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:19.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.796+0000 7f075f209b80 1 --2- 172.21.15.37:0/3393233693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b58f426e20 0x55b58f420710 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:19.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.796+0000 7f075f209b80 1 -- 172.21.15.37:0/3393233693 >> 172.21.15.37:0/3393233693 conn(0x55b58f34d180 msgr2=0x55b58f41ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:19.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.797+0000 7f075f209b80 1 -- 172.21.15.37:0/3393233693 shutdown_connections 2024-06-10T01:02:19.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.797+0000 7f075f209b80 1 -- 172.21.15.37:0/3393233693 wait complete. 2024-06-10T01:02:19.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.797+0000 7f075f209b80 1 -- start start 2024-06-10T01:02:19.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.797+0000 7f075f209b80 1 Processor -- start 2024-06-10T01:02:19.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.797+0000 7f075f209b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b58f420c50 0x55b58f42cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:19.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.797+0000 7f075f209b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b58f426500 0x55b58f429e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:19.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.797+0000 7f075f209b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b58f426e20 0x55b58f42a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:19.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.797+0000 7f075f209b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b58f3120c0 con 0x55b58f426e20 2024-06-10T01:02:19.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.797+0000 7f075f209b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b58f36d910 con 0x55b58f426500 2024-06-10T01:02:19.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.797+0000 7f075f209b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b58f42d800 con 0x55b58f420c50 2024-06-10T01:02:19.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.797+0000 7f075f209b80 10 client.0 _ll_register_callbacks cb 0x55b58f3076b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:02:19.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.797+0000 7f074ed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b58f426e20 0x55b58f42a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:19.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.797+0000 7f074ed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b58f426e20 0x55b58f42a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35644/0 (socket says 172.21.15.37:35644) 2024-06-10T01:02:19.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.797+0000 7f074ed9d700 1 -- 172.21.15.37:0/3945925799 learned_addr learned my addr 172.21.15.37:0/3945925799 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:19.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.797+0000 7f074e59c700 1 --2- 172.21.15.37:0/3945925799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b58f420c50 0x55b58f42cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:19.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.797+0000 7f074dd9b700 1 --2- 172.21.15.37:0/3945925799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b58f426500 0x55b58f429e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:19.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.798+0000 7f074ed9d700 1 -- 172.21.15.37:0/3945925799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b58f420c50 msgr2=0x55b58f42cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:19.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.798+0000 7f074ed9d700 1 --2- 172.21.15.37:0/3945925799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b58f420c50 0x55b58f42cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:19.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.798+0000 7f074ed9d700 1 -- 172.21.15.37:0/3945925799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b58f426500 msgr2=0x55b58f429e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:19.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.798+0000 7f074ed9d700 1 --2- 172.21.15.37:0/3945925799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b58f426500 0x55b58f429e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:19.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.798+0000 7f074ed9d700 1 -- 172.21.15.37:0/3945925799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b58f337850 con 0x55b58f426e20 2024-06-10T01:02:19.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.798+0000 7f074ed9d700 1 --2- 172.21.15.37:0/3945925799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b58f426e20 0x55b58f42a380 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0748015000 tx=0x7f0748004da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f90099634a7d34 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:19.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.798+0000 7f0735ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:02:19.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.798+0000 7f0735ffb700 1 -- 172.21.15.37:0/3945925799 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07480078f0 con 0x55b58f426e20 2024-06-10T01:02:19.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.798+0000 7f0735ffb700 1 -- 172.21.15.37:0/3945925799 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f0748007a90 con 0x55b58f426e20 2024-06-10T01:02:19.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.798+0000 7f075f209b80 1 -- 172.21.15.37:0/3945925799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f074409fa20 con 0x55b58f426e20 2024-06-10T01:02:19.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.798+0000 7f0735ffb700 1 -- 172.21.15.37:0/3945925799 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0748007cd0 con 0x55b58f426e20 2024-06-10T01:02:19.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.798+0000 7f075f209b80 10 client.16341 fetch_fsmap learned FSMap version 27 2024-06-10T01:02:19.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.798+0000 7f075f209b80 1 -- 172.21.15.37:0/3945925799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b58f437f60 con 0x55b58f426e20 2024-06-10T01:02:19.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.799+0000 7f0735ffb700 1 -- 172.21.15.37:0/3945925799 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f074409fa20 con 0x55b58f426e20 2024-06-10T01:02:19.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.799+0000 7f0735ffb700 1 -- 172.21.15.37:0/3945925799 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0748024070 con 0x55b58f426e20 2024-06-10T01:02:19.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.799+0000 7f075f209b80 10 client.16341 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:02:19.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.799+0000 7f075f209b80 10 client.16341 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:02:19.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.799+0000 7f075f209b80 20 client.16341 populate_metadata read hostname 'smithi037' 2024-06-10T01:02:19.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.799+0000 7f075f209b80 1 --2- 172.21.15.37:0/3945925799 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b58f43a540 0x55b58f43c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:19.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.799+0000 7f075f209b80 4 client.16341 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:02:19.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.799+0000 7f075f209b80 1 -- 172.21.15.37:0/3945925799 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b58f37ee50 con 0x55b58f43a540 2024-06-10T01:02:19.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.799+0000 7f074e59c700 1 --2- 172.21.15.37:0/3945925799 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b58f43a540 0x55b58f43c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:19.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.800+0000 7f0735ffb700 10 client.16341 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:19.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.800+0000 7f074e59c700 1 --2- 172.21.15.37:0/3945925799 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b58f43a540 0x55b58f43c930 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65f186380f812500 server_cookie=25e41e412f6cabf4 in_seq=0 out_seq=0 2024-06-10T01:02:19.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.801+0000 7f0735ffb700 1 -- 172.21.15.37:0/3945925799 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b58f37ee50 con 0x55b58f43a540 2024-06-10T01:02:19.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.801+0000 7f0735ffb700 10 client.16341 handle_command_reply: tid=0 2024-06-10T01:02:19.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.801+0000 7f075f209b80 1 client.16341 shutdown 2024-06-10T01:02:19.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.801+0000 7f075f209b80 1 -- 172.21.15.37:0/3945925799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b58f426e20 msgr2=0x55b58f42a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:19.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.801+0000 7f075f209b80 1 --2- 172.21.15.37:0/3945925799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b58f426e20 0x55b58f42a380 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0748015000 tx=0x7f0748004da0 comp rx=0 tx=0).stop 2024-06-10T01:02:19.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.801+0000 7f075f209b80 1 -- 172.21.15.37:0/3945925799 shutdown_connections 2024-06-10T01:02:19.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.801+0000 7f075f209b80 1 --2- 172.21.15.37:0/3945925799 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b58f43a540 0x55b58f43c930 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:19.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.801+0000 7f075f209b80 1 --2- 172.21.15.37:0/3945925799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b58f426e20 0x55b58f42a380 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:19.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.801+0000 7f075f209b80 1 --2- 172.21.15.37:0/3945925799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b58f420c50 0x55b58f42cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:19.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.801+0000 7f0735ffb700 0 client.16341 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:19.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.801+0000 7f075f209b80 1 --2- 172.21.15.37:0/3945925799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b58f426500 0x55b58f429e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:19.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.801+0000 7f075f209b80 1 -- 172.21.15.37:0/3945925799 >> 172.21.15.37:0/3945925799 conn(0x55b58f34d180 msgr2=0x55b58f41a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:19.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.801+0000 7f075f209b80 1 -- 172.21.15.37:0/3945925799 shutdown_connections 2024-06-10T01:02:19.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.801+0000 7f075f209b80 1 -- 172.21.15.37:0/3945925799 wait complete. 2024-06-10T01:02:19.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.801+0000 7f075f209b80 20 client.16341 trim_cache size 0 max 16384 2024-06-10T01:02:19.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.821+0000 7f075f209b80 1 Processor -- start 2024-06-10T01:02:19.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.821+0000 7f075f209b80 1 -- start start 2024-06-10T01:02:19.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.821+0000 7f075f209b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b58f420c50 0x55b58f420710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:19.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.821+0000 7f075f209b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b58f426500 0x55b58f429740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.821+0000 7f075f209b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b58f426e20 0x55b58f42c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.821+0000 7f075f209b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f07501b2e50 con 0x55b58f420c50 2024-06-10T01:02:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.821+0000 7f075f209b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f07501b06e0 con 0x55b58f426500 2024-06-10T01:02:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.821+0000 7f075f209b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f07501ae530 con 0x55b58f426e20 2024-06-10T01:02:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.822+0000 7f074e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b58f420c50 0x55b58f420710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.822+0000 7f074e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b58f420c50 0x55b58f420710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35654/0 (socket says 172.21.15.37:35654) 2024-06-10T01:02:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.822+0000 7f074e59c700 1 -- 172.21.15.37:0/3941851360 learned_addr learned my addr 172.21.15.37:0/3941851360 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.822+0000 7f074ed9d700 1 --2- 172.21.15.37:0/3941851360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b58f426e20 0x55b58f42c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.822+0000 7f074dd9b700 1 --2- 172.21.15.37:0/3941851360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b58f426500 0x55b58f429740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.822+0000 7f074e59c700 1 -- 172.21.15.37:0/3941851360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b58f426e20 msgr2=0x55b58f42c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.822+0000 7f074e59c700 1 --2- 172.21.15.37:0/3941851360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b58f426e20 0x55b58f42c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.822+0000 7f074e59c700 1 -- 172.21.15.37:0/3941851360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b58f426500 msgr2=0x55b58f429740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.822+0000 7f074e59c700 1 --2- 172.21.15.37:0/3941851360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b58f426500 0x55b58f429740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.822+0000 7f074e59c700 1 -- 172.21.15.37:0/3941851360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b58f4283b0 con 0x55b58f420c50 2024-06-10T01:02:19.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.822+0000 7f074e59c700 1 --2- 172.21.15.37:0/3941851360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b58f420c50 0x55b58f420710 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f0740004130 tx=0x7f0740012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2aff99cb39c0137 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:19.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.822+0000 7f0737fff700 1 -- 172.21.15.37:0/3941851360 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0740016070 con 0x55b58f420c50 2024-06-10T01:02:19.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.822+0000 7f0737fff700 1 -- 172.21.15.37:0/3941851360 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0740014070 con 0x55b58f420c50 2024-06-10T01:02:19.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.822+0000 7f0737fff700 1 -- 172.21.15.37:0/3941851360 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0740010630 con 0x55b58f420c50 2024-06-10T01:02:19.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.823+0000 7f075f209b80 1 -- 172.21.15.37:0/3941851360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b58f420c50 msgr2=0x55b58f420710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:19.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.823+0000 7f075f209b80 1 --2- 172.21.15.37:0/3941851360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b58f420c50 0x55b58f420710 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f0740004130 tx=0x7f0740012000 comp rx=0 tx=0).stop 2024-06-10T01:02:19.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.823+0000 7f075f209b80 1 -- 172.21.15.37:0/3941851360 shutdown_connections 2024-06-10T01:02:19.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.823+0000 7f075f209b80 1 --2- 172.21.15.37:0/3941851360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b58f426e20 0x55b58f42c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:19.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.823+0000 7f075f209b80 1 --2- 172.21.15.37:0/3941851360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b58f420c50 0x55b58f420710 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:19.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.823+0000 7f075f209b80 1 --2- 172.21.15.37:0/3941851360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b58f426500 0x55b58f429740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:19.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.823+0000 7f075f209b80 1 -- 172.21.15.37:0/3941851360 >> 172.21.15.37:0/3941851360 conn(0x55b58f34d180 msgr2=0x55b58f41abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:19.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.823+0000 7f075f209b80 1 -- 172.21.15.37:0/3941851360 shutdown_connections 2024-06-10T01:02:19.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.823+0000 7f075f209b80 1 -- 172.21.15.37:0/3941851360 wait complete. 2024-06-10T01:02:19.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.823+0000 7f075f209b80 1 -- start start 2024-06-10T01:02:19.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.823+0000 7f075f209b80 1 Processor -- start 2024-06-10T01:02:19.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.824+0000 7f075f209b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b58f420c50 0x55b58f433020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:19.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.824+0000 7f075f209b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b58f426500 0x55b58f42fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:19.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.824+0000 7f075f209b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b58f426e20 0x55b58f42ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:19.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.824+0000 7f075f209b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f07501aeba0 con 0x55b58f420c50 2024-06-10T01:02:19.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.824+0000 7f075f209b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f07501b0010 con 0x55b58f426500 2024-06-10T01:02:19.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.824+0000 7f075f209b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f07501af270 con 0x55b58f426e20 2024-06-10T01:02:19.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.824+0000 7f075f209b80 10 client.0 _ll_register_callbacks cb 0x55b58f3076b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:02:19.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.824+0000 7f074e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b58f420c50 0x55b58f433020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:19.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.824+0000 7f074e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b58f420c50 0x55b58f433020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35662/0 (socket says 172.21.15.37:35662) 2024-06-10T01:02:19.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.824+0000 7f074e59c700 1 -- 172.21.15.37:0/1065500886 learned_addr learned my addr 172.21.15.37:0/1065500886 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:19.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.824+0000 7f074ed9d700 1 --2- 172.21.15.37:0/1065500886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b58f426e20 0x55b58f42ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:19.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.824+0000 7f074dd9b700 1 --2- 172.21.15.37:0/1065500886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b58f426500 0x55b58f42fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:19.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.824+0000 7f074e59c700 1 -- 172.21.15.37:0/1065500886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b58f426e20 msgr2=0x55b58f42ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:19.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.824+0000 7f074e59c700 1 --2- 172.21.15.37:0/1065500886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b58f426e20 0x55b58f42ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:19.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.824+0000 7f074e59c700 1 -- 172.21.15.37:0/1065500886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b58f426500 msgr2=0x55b58f42fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.824+0000 7f074e59c700 1 --2- 172.21.15.37:0/1065500886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b58f426500 0x55b58f42fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.824+0000 7f074e59c700 1 -- 172.21.15.37:0/1065500886 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b58f430510 con 0x55b58f420c50 2024-06-10T01:02:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.824+0000 7f074ed9d700 1 --2- 172.21.15.37:0/1065500886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b58f426e20 0x55b58f42ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.824+0000 7f074e59c700 1 --2- 172.21.15.37:0/1065500886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b58f420c50 0x55b58f433020 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f0740004660 tx=0x7f0740003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f0c2a78d35462e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.824+0000 7f0735ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:02:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.825+0000 7f0735ffb700 1 -- 172.21.15.37:0/1065500886 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0740029670 con 0x55b58f420c50 2024-06-10T01:02:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.825+0000 7f0735ffb700 1 -- 172.21.15.37:0/1065500886 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f0740029810 con 0x55b58f420c50 2024-06-10T01:02:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.825+0000 7f075f209b80 1 -- 172.21.15.37:0/1065500886 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0740014070 con 0x55b58f420c50 2024-06-10T01:02:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.825+0000 7f0735ffb700 1 -- 172.21.15.37:0/1065500886 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0740029a10 con 0x55b58f420c50 2024-06-10T01:02:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.825+0000 7f075f209b80 10 client.16347 fetch_fsmap learned FSMap version 27 2024-06-10T01:02:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.825+0000 7f075f209b80 1 -- 172.21.15.37:0/1065500886 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b58f4a5f50 con 0x55b58f420c50 2024-06-10T01:02:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.825+0000 7f0735ffb700 1 -- 172.21.15.37:0/1065500886 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0740014070 con 0x55b58f420c50 2024-06-10T01:02:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.825+0000 7f0735ffb700 1 -- 172.21.15.37:0/1065500886 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f074003a020 con 0x55b58f420c50 2024-06-10T01:02:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.825+0000 7f075f209b80 10 client.16347 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:02:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.825+0000 7f075f209b80 10 client.16347 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:02:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.825+0000 7f075f209b80 20 client.16347 populate_metadata read hostname 'smithi037' 2024-06-10T01:02:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.825+0000 7f075f209b80 1 --2- 172.21.15.37:0/1065500886 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b58f4a64e0 0x55b58f4a88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.825+0000 7f075f209b80 4 client.16347 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "31515789", "release": true, "await": true}] 2024-06-10T01:02:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.825+0000 7f075f209b80 1 -- 172.21.15.37:0/1065500886 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "quiesce db", "set_id": "31515789", "release": true, "await": true}) -- 0x55b58f3793d0 con 0x55b58f4a64e0 2024-06-10T01:02:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.825+0000 7f074dd9b700 1 --2- 172.21.15.37:0/1065500886 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b58f4a64e0 0x55b58f4a88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.826+0000 7f074dd9b700 1 --2- 172.21.15.37:0/1065500886 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b58f4a64e0 0x55b58f4a88d0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=326b865818d49474 server_cookie=208173f0a3c039a9 in_seq=0 out_seq=0 2024-06-10T01:02:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:19.826+0000 7f0735ffb700 10 client.16347 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:20.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:20.281+0000 7f0735ffb700 1 -- 172.21.15.37:0/1065500886 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b58f3793d0 con 0x55b58f4a64e0 2024-06-10T01:02:20.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:20.281+0000 7f0735ffb700 10 client.16347 handle_command_reply: tid=0 2024-06-10T01:02:20.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:20.281+0000 7f075f209b80 1 client.16347 shutdown 2024-06-10T01:02:20.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:20.281+0000 7f075f209b80 1 -- 172.21.15.37:0/1065500886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b58f420c50 msgr2=0x55b58f433020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:20.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:20.281+0000 7f075f209b80 1 --2- 172.21.15.37:0/1065500886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b58f420c50 0x55b58f433020 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f0740004660 tx=0x7f0740003a10 comp rx=0 tx=0).stop 2024-06-10T01:02:20.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:20.281+0000 7f075f209b80 1 -- 172.21.15.37:0/1065500886 shutdown_connections 2024-06-10T01:02:20.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:20.281+0000 7f075f209b80 1 --2- 172.21.15.37:0/1065500886 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b58f4a64e0 0x55b58f4a88d0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:20.281+0000 7f075f209b80 1 --2- 172.21.15.37:0/1065500886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b58f426e20 0x55b58f42ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:20.281+0000 7f075f209b80 1 --2- 172.21.15.37:0/1065500886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b58f420c50 0x55b58f433020 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:20.281+0000 7f075f209b80 1 --2- 172.21.15.37:0/1065500886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b58f426500 0x55b58f42fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:20.281+0000 7f075f209b80 1 -- 172.21.15.37:0/1065500886 >> 172.21.15.37:0/1065500886 conn(0x55b58f34d180 msgr2=0x55b58f424f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:20.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:20.281+0000 7f0735ffb700 0 client.16347 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:20.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:20.281+0000 7f075f209b80 1 -- 172.21.15.37:0/1065500886 shutdown_connections 2024-06-10T01:02:20.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:20.282+0000 7f075f209b80 1 -- 172.21.15.37:0/1065500886 wait complete. 2024-06-10T01:02:20.281 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:02:20.281 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-06-10T01:02:20.281 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24472, 2024-06-10T01:02:20.281 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 12, 2024-06-10T01:02:20.281 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-06-10T01:02:20.281 INFO:teuthology.orchestra.run.smithi037.stdout: "31515789": { 2024-06-10T01:02:20.281 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 12, 2024-06-10T01:02:20.281 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-06-10T01:02:20.282 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:02:20.282 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-06-10T01:02:20.282 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:02:20.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:02:20.282 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-06-10T01:02:20.282 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 138.2, 2024-06-10T01:02:20.282 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-06-10T01:02:20.282 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-06-10T01:02:20.282 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-06-10T01:02:20.282 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:02:20.282 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-06-10T01:02:20.282 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:02:20.282 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:02:20.282 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:02:20.283 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:02:20.283 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:02:20.283 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:02:20.283 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:02:20.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:20.282+0000 7f075f209b80 20 client.16347 trim_cache size 0 max 16384 2024-06-10T01:02:20.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:20.282+0000 7f0754986700 1 -- 172.21.15.37:0/4173289513 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f07380a2c20 msgr2=0x7f07380a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:20.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:20.282+0000 7f0754986700 1 --2- 172.21.15.37:0/4173289513 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f07380a2c20 0x7f07380a50e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f074000ac50 tx=0x7f074000a380 comp rx=0 tx=0).stop 2024-06-10T01:02:20.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:20.282+0000 7f0754986700 1 -- 172.21.15.37:0/4173289513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0750101fd0 msgr2=0x7f075019b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:20.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:20.282+0000 7f0754986700 1 --2- 172.21.15.37:0/4173289513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0750101fd0 0x7f075019b350 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f074400c8f0 tx=0x7f074400cc00 comp rx=0 tx=0).stop 2024-06-10T01:02:20.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:20.283+0000 7f0754986700 1 -- 172.21.15.37:0/4173289513 shutdown_connections 2024-06-10T01:02:20.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:20.283+0000 7f0754986700 1 --2- 172.21.15.37:0/4173289513 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f07380a2c20 0x7f07380a50e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:20.283+0000 7f0754986700 1 --2- 172.21.15.37:0/4173289513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07501066b0 0x7f075019f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:20.283+0000 7f0754986700 1 --2- 172.21.15.37:0/4173289513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0750101620 0x7f075019ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:20.283+0000 7f0754986700 1 --2- 172.21.15.37:0/4173289513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0750101fd0 0x7f075019b350 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:20.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:20.283+0000 7f0754986700 1 -- 172.21.15.37:0/4173289513 >> 172.21.15.37:0/4173289513 conn(0x7f0750078340 msgr2=0x7f075010d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:20.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:20.283+0000 7f0754986700 1 -- 172.21.15.37:0/4173289513 shutdown_connections 2024-06-10T01:02:20.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:20.283+0000 7f0754986700 1 -- 172.21.15.37:0/4173289513 wait complete. 2024-06-10T01:02:20.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:20 smithi037 ceph-mon[32192]: pgmap v599: 129 pgs: 129 active+clean; 6.7 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 1.80k op/s 2024-06-10T01:02:20.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:20 smithi167 ceph-mon[36917]: pgmap v599: 129 pgs: 129 active+clean; 6.7 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 1.80k op/s 2024-06-10T01:02:20.612 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:20 smithi094 ceph-mon[36708]: pgmap v599: 129 pgs: 129 active+clean; 6.7 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 1.80k op/s 2024-06-10T01:02:21.078 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '31515789', total seconds elapsed: 33.0 2024-06-10T01:02:22.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:22 smithi037 ceph-mon[32192]: pgmap v600: 129 pgs: 129 active+clean; 6.7 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 1.51k op/s 2024-06-10T01:02:22.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:22 smithi167 ceph-mon[36917]: pgmap v600: 129 pgs: 129 active+clean; 6.7 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 1.51k op/s 2024-06-10T01:02:22.612 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:22 smithi094 ceph-mon[36708]: pgmap v600: 129 pgs: 129 active+clean; 6.7 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 1.51k op/s 2024-06-10T01:02:24.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:24 smithi037 ceph-mon[32192]: pgmap v601: 129 pgs: 129 active+clean; 8.6 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 1.48k op/s 2024-06-10T01:02:24.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:24 smithi167 ceph-mon[36917]: pgmap v601: 129 pgs: 129 active+clean; 8.6 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 1.48k op/s 2024-06-10T01:02:24.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:24 smithi094 ceph-mon[36708]: pgmap v601: 129 pgs: 129 active+clean; 8.6 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 1.48k op/s 2024-06-10T01:02:26.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:26 smithi167 ceph-mon[36917]: pgmap v602: 129 pgs: 129 active+clean; 10 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 1.40k op/s 2024-06-10T01:02:26.612 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:26 smithi094 ceph-mon[36708]: pgmap v602: 129 pgs: 129 active+clean; 10 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 1.40k op/s 2024-06-10T01:02:26.643 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:26 smithi037 ceph-mon[32192]: pgmap v602: 129 pgs: 129 active+clean; 10 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 1.40k op/s 2024-06-10T01:02:26.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.670+0000 7fb11dfa5640 1 -- 172.21.15.37:0/3410982611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb118102bf0 msgr2=0x7fb118103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.670+0000 7fb11dfa5640 1 --2- 172.21.15.37:0/3410982611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb118102bf0 0x7fb118103070 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb100009900 tx=0x7fb10002a410 comp rx=0 tx=0).stop 2024-06-10T01:02:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.670+0000 7fb11dfa5640 1 -- 172.21.15.37:0/3410982611 shutdown_connections 2024-06-10T01:02:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.670+0000 7fb11dfa5640 1 --2- 172.21.15.37:0/3410982611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1181035b0 0x7fb118109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.670+0000 7fb11dfa5640 1 --2- 172.21.15.37:0/3410982611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb118102bf0 0x7fb118103070 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.670+0000 7fb11dfa5640 1 --2- 172.21.15.37:0/3410982611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb118101a00 0x7fb118101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.670+0000 7fb11dfa5640 1 -- 172.21.15.37:0/3410982611 >> 172.21.15.37:0/3410982611 conn(0x7fb1180fcf90 msgr2=0x7fb1180ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.670+0000 7fb11dfa5640 1 -- 172.21.15.37:0/3410982611 shutdown_connections 2024-06-10T01:02:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.670+0000 7fb11dfa5640 1 -- 172.21.15.37:0/3410982611 wait complete. 2024-06-10T01:02:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.671+0000 7fb11dfa5640 1 Processor -- start 2024-06-10T01:02:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.671+0000 7fb11dfa5640 1 -- start start 2024-06-10T01:02:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.671+0000 7fb11dfa5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb118101a00 0x7fb118197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.671+0000 7fb11dfa5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb118102bf0 0x7fb118198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.671+0000 7fb11dfa5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1181035b0 0x7fb11819f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.671+0000 7fb11dfa5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb11810d660 con 0x7fb118101a00 2024-06-10T01:02:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.671+0000 7fb11dfa5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb11810d4e0 con 0x7fb118102bf0 2024-06-10T01:02:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.671+0000 7fb11dfa5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb11810d7e0 con 0x7fb1181035b0 2024-06-10T01:02:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.671+0000 7fb1177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb118101a00 0x7fb118197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.671+0000 7fb1177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb118101a00 0x7fb118197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46798/0 (socket says 172.21.15.37:46798) 2024-06-10T01:02:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.671+0000 7fb1177fe640 1 -- 172.21.15.37:0/1067457080 learned_addr learned my addr 172.21.15.37:0/1067457080 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.671+0000 7fb117fff640 1 --2- 172.21.15.37:0/1067457080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1181035b0 0x7fb11819f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.672+0000 7fb10ffff640 1 --2- 172.21.15.37:0/1067457080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb118102bf0 0x7fb118198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.672+0000 7fb1177fe640 1 -- 172.21.15.37:0/1067457080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1181035b0 msgr2=0x7fb11819f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.672+0000 7fb1177fe640 1 --2- 172.21.15.37:0/1067457080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1181035b0 0x7fb11819f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.672+0000 7fb1177fe640 1 -- 172.21.15.37:0/1067457080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb118102bf0 msgr2=0x7fb118198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.672+0000 7fb1177fe640 1 --2- 172.21.15.37:0/1067457080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb118102bf0 0x7fb118198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.672+0000 7fb1177fe640 1 -- 172.21.15.37:0/1067457080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11819f840 con 0x7fb118101a00 2024-06-10T01:02:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.672+0000 7fb1177fe640 1 --2- 172.21.15.37:0/1067457080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb118101a00 0x7fb118197c30 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb10800e9c0 tx=0x7fb10800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.672+0000 7fb1157fa640 1 -- 172.21.15.37:0/1067457080 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10800ccc0 con 0x7fb118101a00 2024-06-10T01:02:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.672+0000 7fb11dfa5640 1 -- 172.21.15.37:0/1067457080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb11819fb30 con 0x7fb118101a00 2024-06-10T01:02:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.672+0000 7fb1157fa640 1 -- 172.21.15.37:0/1067457080 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1080035f0 con 0x7fb118101a00 2024-06-10T01:02:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.672+0000 7fb1157fa640 1 -- 172.21.15.37:0/1067457080 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10800ce60 con 0x7fb118101a00 2024-06-10T01:02:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.672+0000 7fb11dfa5640 1 -- 172.21.15.37:0/1067457080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1181a00d0 con 0x7fb118101a00 2024-06-10T01:02:26.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.674+0000 7fb1157fa640 1 -- 172.21.15.37:0/1067457080 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb1080037a0 con 0x7fb118101a00 2024-06-10T01:02:26.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.675+0000 7fb120d8e740 1 Processor -- start 2024-06-10T01:02:26.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.675+0000 7fb120d8e740 1 -- start start 2024-06-10T01:02:26.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.675+0000 7fb120d8e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc6e379420 0x55bc6e379840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:26.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.675+0000 7fb120d8e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc6e379e10 0x55bc6e37e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:26.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.675+0000 7fb120d8e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc6e37ef80 0x55bc6e381410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.675+0000 7fb120d8e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bc6e2d84f0 con 0x55bc6e379e10 2024-06-10T01:02:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.675+0000 7fb120d8e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bc6e276870 con 0x55bc6e379420 2024-06-10T01:02:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.675+0000 7fb120d8e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bc6e2d44b0 con 0x55bc6e37ef80 2024-06-10T01:02:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.675+0000 7fb1177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc6e379e10 0x55bc6e37e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.675+0000 7fb10ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc6e37ef80 0x55bc6e381410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.675+0000 7fb1177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc6e379e10 0x55bc6e37e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46814/0 (socket says 172.21.15.37:46814) 2024-06-10T01:02:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.675+0000 7fb1177fe640 1 -- 172.21.15.37:0/435340120 learned_addr learned my addr 172.21.15.37:0/435340120 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.675+0000 7fb117fff640 1 --2- 172.21.15.37:0/435340120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc6e379420 0x55bc6e379840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.675+0000 7fb10ffff640 1 -- 172.21.15.37:0/435340120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc6e379420 msgr2=0x55bc6e379840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.675+0000 7fb10ffff640 1 --2- 172.21.15.37:0/435340120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc6e379420 0x55bc6e379840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.675+0000 7fb10ffff640 1 -- 172.21.15.37:0/435340120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc6e379e10 msgr2=0x55bc6e37e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.676+0000 7fb10ffff640 1 --2- 172.21.15.37:0/435340120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc6e379e10 0x55bc6e37e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.676+0000 7fb10ffff640 1 -- 172.21.15.37:0/435340120 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc6e2929b0 con 0x55bc6e37ef80 2024-06-10T01:02:26.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.676+0000 7fb1157fa640 1 --2- 172.21.15.37:0/1067457080 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb1040a2c30 0x7fb1040a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.676+0000 7fb117fff640 1 --2- 172.21.15.37:0/1067457080 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb1040a2c30 0x7fb1040a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.676+0000 7fb1157fa640 1 -- 172.21.15.37:0/1067457080 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fb1080cf8f0 con 0x7fb118101a00 2024-06-10T01:02:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.676+0000 7fb10ffff640 1 --2- 172.21.15.37:0/435340120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc6e37ef80 0x55bc6e381410 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb100031440 tx=0x7fb100005e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf23b96aa8387d3f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.676+0000 7fb10dffb640 1 -- 172.21.15.37:0/435340120 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100040070 con 0x55bc6e37ef80 2024-06-10T01:02:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.676+0000 7fb10dffb640 1 -- 172.21.15.37:0/435340120 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb100004030 con 0x55bc6e37ef80 2024-06-10T01:02:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.676+0000 7fb10dffb640 1 -- 172.21.15.37:0/435340120 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100035040 con 0x55bc6e37ef80 2024-06-10T01:02:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.677+0000 7fb120d8e740 1 -- 172.21.15.37:0/435340120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc6e37ef80 msgr2=0x55bc6e381410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.677+0000 7fb120d8e740 1 --2- 172.21.15.37:0/435340120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc6e37ef80 0x55bc6e381410 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb100031440 tx=0x7fb100005e00 comp rx=0 tx=0).stop 2024-06-10T01:02:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.677+0000 7fb117fff640 1 --2- 172.21.15.37:0/1067457080 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb1040a2c30 0x7fb1040a50f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55bc6e37a670 tx=0x7fb110009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.677+0000 7fb120d8e740 1 -- 172.21.15.37:0/435340120 shutdown_connections 2024-06-10T01:02:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.677+0000 7fb120d8e740 1 --2- 172.21.15.37:0/435340120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc6e37ef80 0x55bc6e381410 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.677+0000 7fb120d8e740 1 --2- 172.21.15.37:0/435340120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc6e379e10 0x55bc6e37e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.677+0000 7fb120d8e740 1 --2- 172.21.15.37:0/435340120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc6e379420 0x55bc6e379840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.677+0000 7fb120d8e740 1 -- 172.21.15.37:0/435340120 >> 172.21.15.37:0/435340120 conn(0x55bc6e370680 msgr2=0x55bc6e370a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.677+0000 7fb120d8e740 1 -- 172.21.15.37:0/435340120 shutdown_connections 2024-06-10T01:02:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.677+0000 7fb120d8e740 1 -- 172.21.15.37:0/435340120 wait complete. 2024-06-10T01:02:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.677+0000 7fb120d8e740 1 -- start start 2024-06-10T01:02:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.677+0000 7fb120d8e740 1 Processor -- start 2024-06-10T01:02:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.678+0000 7fb120d8e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc6e379420 0x55bc6e383630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.678+0000 7fb120d8e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc6e379e10 0x55bc6e383b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.678+0000 7fb120d8e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc6e37ef80 0x55bc6e38abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.678+0000 7fb120d8e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bc6e381ed0 con 0x55bc6e37ef80 2024-06-10T01:02:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.678+0000 7fb120d8e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bc6e261ae0 con 0x55bc6e379e10 2024-06-10T01:02:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.678+0000 7fb120d8e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bc6e382050 con 0x55bc6e379420 2024-06-10T01:02:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.678+0000 7fb120d8e740 10 client.0 _ll_register_callbacks cb 0x55bc6e29a9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:02:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.678+0000 7fb117fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc6e37ef80 0x55bc6e38abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.678+0000 7fb1177fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc6e379420 0x55bc6e383630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.678+0000 7fb1177fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc6e379420 0x55bc6e383630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:45390/0 (socket says 172.21.15.37:45390) 2024-06-10T01:02:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.678+0000 7fb117fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc6e37ef80 0x55bc6e38abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46818/0 (socket says 172.21.15.37:46818) 2024-06-10T01:02:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.678+0000 7fb1177fe640 1 -- 172.21.15.37:0/125734996 learned_addr learned my addr 172.21.15.37:0/125734996 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.678+0000 7fb10ffff640 1 --2- 172.21.15.37:0/125734996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc6e379e10 0x55bc6e383b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.678+0000 7fb1177fe640 1 -- 172.21.15.37:0/125734996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc6e379e10 msgr2=0x55bc6e383b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.678+0000 7fb1177fe640 1 --2- 172.21.15.37:0/125734996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc6e379e10 0x55bc6e383b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.678+0000 7fb1177fe640 1 -- 172.21.15.37:0/125734996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc6e37ef80 msgr2=0x55bc6e38abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.678+0000 7fb1177fe640 1 --2- 172.21.15.37:0/125734996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc6e37ef80 0x55bc6e38abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.678+0000 7fb1177fe640 1 -- 172.21.15.37:0/125734996 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc6dfec210 con 0x55bc6e379420 2024-06-10T01:02:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.678+0000 7fb117fff640 1 --2- 172.21.15.37:0/125734996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc6e37ef80 0x55bc6e38abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.678+0000 7fb10ffff640 1 --2- 172.21.15.37:0/125734996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc6e379e10 0x55bc6e383b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:26.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.678+0000 7fb1177fe640 1 --2- 172.21.15.37:0/125734996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc6e379420 0x55bc6e383630 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb1080cfb90 tx=0x7fb108097fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23a51121c57bc00b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.678+0000 7fb0f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:02:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.679+0000 7fb0f6ffd640 1 -- 172.21.15.37:0/125734996 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1080d2e60 con 0x55bc6e379420 2024-06-10T01:02:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.679+0000 7fb0f6ffd640 1 -- 172.21.15.37:0/125734996 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fb10809b150 con 0x55bc6e379420 2024-06-10T01:02:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.679+0000 7fb120d8e740 1 -- 172.21.15.37:0/125734996 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb100004030 con 0x55bc6e379420 2024-06-10T01:02:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.679+0000 7fb0f6ffd640 1 -- 172.21.15.37:0/125734996 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb1080b9050 con 0x55bc6e379420 2024-06-10T01:02:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.679+0000 7fb120d8e740 10 client.25918 fetch_fsmap learned FSMap version 27 2024-06-10T01:02:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.679+0000 7fb120d8e740 1 -- 172.21.15.37:0/125734996 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bc6e0b8620 con 0x55bc6e379420 2024-06-10T01:02:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.679+0000 7fb0f6ffd640 1 -- 172.21.15.37:0/125734996 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb100004030 con 0x55bc6e379420 2024-06-10T01:02:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.679+0000 7fb0f6ffd640 1 -- 172.21.15.37:0/125734996 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb1080cfce0 con 0x55bc6e379420 2024-06-10T01:02:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.679+0000 7fb120d8e740 10 client.25918 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:02:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.679+0000 7fb120d8e740 10 client.25918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:02:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.679+0000 7fb120d8e740 20 client.25918 populate_metadata read hostname 'smithi037' 2024-06-10T01:02:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.679+0000 7fb120d8e740 1 --2- 172.21.15.37:0/125734996 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bc6e38f550 0x55bc6e3919e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.679+0000 7fb120d8e740 4 client.25918 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:02:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.679+0000 7fb120d8e740 1 -- 172.21.15.37:0/125734996 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc6e2ebf10 con 0x55bc6e38f550 2024-06-10T01:02:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.679+0000 7fb10ffff640 1 --2- 172.21.15.37:0/125734996 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bc6e38f550 0x55bc6e3919e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.680+0000 7fb10ffff640 1 --2- 172.21.15.37:0/125734996 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bc6e38f550 0x55bc6e3919e0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37c9ba645ab20b13 server_cookie=b42968de44a4f1cf in_seq=0 out_seq=0 2024-06-10T01:02:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.680+0000 7fb0f6ffd640 10 client.25918 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:26.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.684+0000 7fb0f6ffd640 1 -- 172.21.15.37:0/125734996 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bc6e2ebf10 con 0x55bc6e38f550 2024-06-10T01:02:26.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.684+0000 7fb0f6ffd640 10 client.25918 handle_command_reply: tid=0 2024-06-10T01:02:26.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.684+0000 7fb120d8e740 1 client.25918 shutdown 2024-06-10T01:02:26.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.684+0000 7fb120d8e740 1 -- 172.21.15.37:0/125734996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc6e379420 msgr2=0x55bc6e383630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:26.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.684+0000 7fb120d8e740 1 --2- 172.21.15.37:0/125734996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc6e379420 0x55bc6e383630 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb1080cfb90 tx=0x7fb108097fd0 comp rx=0 tx=0).stop 2024-06-10T01:02:26.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.684+0000 7fb120d8e740 1 -- 172.21.15.37:0/125734996 shutdown_connections 2024-06-10T01:02:26.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.684+0000 7fb120d8e740 1 --2- 172.21.15.37:0/125734996 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bc6e38f550 0x55bc6e3919e0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:26.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.684+0000 7fb120d8e740 1 --2- 172.21.15.37:0/125734996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc6e37ef80 0x55bc6e38abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:26.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.684+0000 7fb120d8e740 1 --2- 172.21.15.37:0/125734996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc6e379e10 0x55bc6e383b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.684+0000 7fb0f6ffd640 0 client.25918 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.684+0000 7fb120d8e740 1 --2- 172.21.15.37:0/125734996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc6e379420 0x55bc6e383630 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.684+0000 7fb120d8e740 1 -- 172.21.15.37:0/125734996 >> 172.21.15.37:0/125734996 conn(0x55bc6e370680 msgr2=0x55bc6e36f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.684+0000 7fb120d8e740 1 -- 172.21.15.37:0/125734996 shutdown_connections 2024-06-10T01:02:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.684+0000 7fb120d8e740 1 -- 172.21.15.37:0/125734996 wait complete. 2024-06-10T01:02:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.684+0000 7fb120d8e740 20 client.25918 trim_cache size 0 max 16384 2024-06-10T01:02:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.700+0000 7fb120d8e740 1 Processor -- start 2024-06-10T01:02:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.700+0000 7fb120d8e740 1 -- start start 2024-06-10T01:02:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.700+0000 7fb120d8e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc6e379420 0x55bc6e381370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.700+0000 7fb120d8e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc6e379e10 0x55bc6e38ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.700+0000 7fb120d8e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc6e37ef80 0x55bc6e382420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.700+0000 7fb120d8e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb1181aad40 con 0x55bc6e379e10 2024-06-10T01:02:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.700+0000 7fb120d8e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb1181a8610 con 0x55bc6e379420 2024-06-10T01:02:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.700+0000 7fb120d8e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb1181a6460 con 0x55bc6e37ef80 2024-06-10T01:02:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.700+0000 7fb10ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc6e379e10 0x55bc6e38ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.700+0000 7fb117fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc6e37ef80 0x55bc6e382420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.700+0000 7fb10ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc6e379e10 0x55bc6e38ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46820/0 (socket says 172.21.15.37:46820) 2024-06-10T01:02:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.700+0000 7fb10ffff640 1 -- 172.21.15.37:0/828380237 learned_addr learned my addr 172.21.15.37:0/828380237 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.700+0000 7fb1177fe640 1 --2- 172.21.15.37:0/828380237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc6e379420 0x55bc6e381370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.700+0000 7fb10ffff640 1 -- 172.21.15.37:0/828380237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc6e37ef80 msgr2=0x55bc6e382420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.700+0000 7fb10ffff640 1 --2- 172.21.15.37:0/828380237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc6e37ef80 0x55bc6e382420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.700+0000 7fb10ffff640 1 -- 172.21.15.37:0/828380237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc6e379420 msgr2=0x55bc6e381370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.700+0000 7fb10ffff640 1 --2- 172.21.15.37:0/828380237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc6e379420 0x55bc6e381370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.700+0000 7fb10ffff640 1 -- 172.21.15.37:0/828380237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc6e3704b0 con 0x55bc6e379e10 2024-06-10T01:02:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.700+0000 7fb1177fe640 1 --2- 172.21.15.37:0/828380237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc6e379420 0x55bc6e381370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:02:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.701+0000 7fb10ffff640 1 --2- 172.21.15.37:0/828380237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc6e379e10 0x55bc6e38ab70 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb100009900 tx=0x7fb1000315d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ec2be0f2b93783 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.701+0000 7fb10d7fa640 1 -- 172.21.15.37:0/828380237 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100035040 con 0x55bc6e379e10 2024-06-10T01:02:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.701+0000 7fb10d7fa640 1 -- 172.21.15.37:0/828380237 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb100004040 con 0x55bc6e379e10 2024-06-10T01:02:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.701+0000 7fb10d7fa640 1 -- 172.21.15.37:0/828380237 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100040040 con 0x55bc6e379e10 2024-06-10T01:02:26.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.701+0000 7fb120d8e740 1 -- 172.21.15.37:0/828380237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc6e379e10 msgr2=0x55bc6e38ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:26.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.701+0000 7fb120d8e740 1 --2- 172.21.15.37:0/828380237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc6e379e10 0x55bc6e38ab70 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb100009900 tx=0x7fb1000315d0 comp rx=0 tx=0).stop 2024-06-10T01:02:26.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.702+0000 7fb120d8e740 1 -- 172.21.15.37:0/828380237 shutdown_connections 2024-06-10T01:02:26.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.702+0000 7fb120d8e740 1 --2- 172.21.15.37:0/828380237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc6e37ef80 0x55bc6e382420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:26.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.702+0000 7fb120d8e740 1 --2- 172.21.15.37:0/828380237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc6e379e10 0x55bc6e38ab70 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:26.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.702+0000 7fb120d8e740 1 --2- 172.21.15.37:0/828380237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc6e379420 0x55bc6e381370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:26.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.702+0000 7fb120d8e740 1 -- 172.21.15.37:0/828380237 >> 172.21.15.37:0/828380237 conn(0x55bc6e370680 msgr2=0x55bc6e37e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:26.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.702+0000 7fb120d8e740 1 -- 172.21.15.37:0/828380237 shutdown_connections 2024-06-10T01:02:26.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.702+0000 7fb120d8e740 1 -- 172.21.15.37:0/828380237 wait complete. 2024-06-10T01:02:26.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.702+0000 7fb120d8e740 1 -- start start 2024-06-10T01:02:26.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.702+0000 7fb120d8e740 1 Processor -- start 2024-06-10T01:02:26.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.702+0000 7fb120d8e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc6e379420 0x55bc6e38a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:26.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.702+0000 7fb120d8e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc6e379e10 0x55bc6e38a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:26.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.702+0000 7fb120d8e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc6e37ef80 0x55bc6e386f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.702+0000 7fb120d8e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb1181a6ad0 con 0x55bc6e379e10 2024-06-10T01:02:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.702+0000 7fb120d8e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb1181a7f40 con 0x55bc6e379420 2024-06-10T01:02:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.702+0000 7fb120d8e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb1181a71a0 con 0x55bc6e37ef80 2024-06-10T01:02:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.702+0000 7fb120d8e740 10 client.0 _ll_register_callbacks cb 0x7fb1080d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:02:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.703+0000 7fb10ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc6e379e10 0x55bc6e38a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.703+0000 7fb10ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc6e379e10 0x55bc6e38a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46834/0 (socket says 172.21.15.37:46834) 2024-06-10T01:02:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.703+0000 7fb10ffff640 1 -- 172.21.15.37:0/692197908 learned_addr learned my addr 172.21.15.37:0/692197908 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.703+0000 7fb1177fe640 1 --2- 172.21.15.37:0/692197908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc6e379420 0x55bc6e38a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.703+0000 7fb117fff640 1 --2- 172.21.15.37:0/692197908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc6e37ef80 0x55bc6e386f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.703+0000 7fb10ffff640 1 -- 172.21.15.37:0/692197908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc6e37ef80 msgr2=0x55bc6e386f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.703+0000 7fb10ffff640 1 --2- 172.21.15.37:0/692197908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc6e37ef80 0x55bc6e386f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.703+0000 7fb10ffff640 1 -- 172.21.15.37:0/692197908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc6e379420 msgr2=0x55bc6e38a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.703+0000 7fb10ffff640 1 --2- 172.21.15.37:0/692197908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc6e379420 0x55bc6e38a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.703+0000 7fb10ffff640 1 -- 172.21.15.37:0/692197908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc6e3876a0 con 0x55bc6e379e10 2024-06-10T01:02:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.703+0000 7fb10ffff640 1 --2- 172.21.15.37:0/692197908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc6e379e10 0x55bc6e38a990 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fb100005600 tx=0x7fb10002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deced781401504f8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.703+0000 7fb0f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:02:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.703+0000 7fb0f6ffd640 1 -- 172.21.15.37:0/692197908 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100040070 con 0x55bc6e379e10 2024-06-10T01:02:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.703+0000 7fb0f6ffd640 1 -- 172.21.15.37:0/692197908 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fb1000316b0 con 0x55bc6e379e10 2024-06-10T01:02:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.703+0000 7fb120d8e740 1 -- 172.21.15.37:0/692197908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb100004040 con 0x55bc6e379e10 2024-06-10T01:02:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.703+0000 7fb0f6ffd640 1 -- 172.21.15.37:0/692197908 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb100067050 con 0x55bc6e379e10 2024-06-10T01:02:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.703+0000 7fb120d8e740 10 client.16365 fetch_fsmap learned FSMap version 27 2024-06-10T01:02:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.703+0000 7fb120d8e740 1 -- 172.21.15.37:0/692197908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bc6e3f8eb0 con 0x55bc6e379e10 2024-06-10T01:02:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.704+0000 7fb0f6ffd640 1 -- 172.21.15.37:0/692197908 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb100004040 con 0x55bc6e379e10 2024-06-10T01:02:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.704+0000 7fb0f6ffd640 1 -- 172.21.15.37:0/692197908 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb10003c5d0 con 0x55bc6e379e10 2024-06-10T01:02:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.704+0000 7fb120d8e740 10 client.16365 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:02:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.704+0000 7fb120d8e740 10 client.16365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:02:26.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.704+0000 7fb120d8e740 20 client.16365 populate_metadata read hostname 'smithi037' 2024-06-10T01:02:26.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.704+0000 7fb120d8e740 1 --2- 172.21.15.37:0/692197908 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bc6e3f9410 0x55bc6e3fb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:26.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.704+0000 7fb120d8e740 4 client.16365 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:02:26.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.704+0000 7fb120d8e740 1 -- 172.21.15.37:0/692197908 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bc6e2e5c20 con 0x55bc6e3f9410 2024-06-10T01:02:26.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.704+0000 7fb1177fe640 1 --2- 172.21.15.37:0/692197908 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bc6e3f9410 0x55bc6e3fb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:26.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.704+0000 7fb1177fe640 1 --2- 172.21.15.37:0/692197908 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bc6e3f9410 0x55bc6e3fb830 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e706655f0dc49cb server_cookie=fb9d2b1ea8832831 in_seq=0 out_seq=0 2024-06-10T01:02:26.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:26.704+0000 7fb0f6ffd640 10 client.16365 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:28.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:28 smithi167 ceph-mon[36917]: pgmap v603: 129 pgs: 129 active+clean; 10 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 615 KiB/s wr, 877 op/s 2024-06-10T01:02:28.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:28 smithi094 ceph-mon[36708]: pgmap v603: 129 pgs: 129 active+clean; 10 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 615 KiB/s wr, 877 op/s 2024-06-10T01:02:28.644 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:28 smithi037 ceph-mon[32192]: pgmap v603: 129 pgs: 129 active+clean; 10 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 615 KiB/s wr, 877 op/s 2024-06-10T01:02:30.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:30 smithi167 ceph-mon[36917]: pgmap v604: 129 pgs: 129 active+clean; 18 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 937 op/s 2024-06-10T01:02:30.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:30 smithi094 ceph-mon[36708]: pgmap v604: 129 pgs: 129 active+clean; 18 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 937 op/s 2024-06-10T01:02:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:30.621+0000 7fb0f6ffd640 1 -- 172.21.15.37:0/692197908 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bc6e2e5c20 con 0x55bc6e3f9410 2024-06-10T01:02:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:30.621+0000 7fb0f6ffd640 10 client.16365 handle_command_reply: tid=0 2024-06-10T01:02:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:30.621+0000 7fb120d8e740 1 client.16365 shutdown 2024-06-10T01:02:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:30.621+0000 7fb120d8e740 1 -- 172.21.15.37:0/692197908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc6e379e10 msgr2=0x55bc6e38a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:30.621+0000 7fb120d8e740 1 --2- 172.21.15.37:0/692197908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc6e379e10 0x55bc6e38a990 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fb100005600 tx=0x7fb10002a410 comp rx=0 tx=0).stop 2024-06-10T01:02:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:30.621+0000 7fb120d8e740 1 -- 172.21.15.37:0/692197908 shutdown_connections 2024-06-10T01:02:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:30.621+0000 7fb120d8e740 1 --2- 172.21.15.37:0/692197908 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bc6e3f9410 0x55bc6e3fb830 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:30.621+0000 7fb120d8e740 1 --2- 172.21.15.37:0/692197908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc6e37ef80 0x55bc6e386f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:30.621+0000 7fb120d8e740 1 --2- 172.21.15.37:0/692197908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc6e379e10 0x55bc6e38a990 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:30.621+0000 7fb0f6ffd640 0 client.16365 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:30.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:30.621+0000 7fb120d8e740 1 --2- 172.21.15.37:0/692197908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bc6e379420 0x55bc6e38a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:30.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:30.621+0000 7fb120d8e740 1 -- 172.21.15.37:0/692197908 >> 172.21.15.37:0/692197908 conn(0x55bc6e370680 msgr2=0x55bc6e37e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:30.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:30.621+0000 7fb120d8e740 1 -- 172.21.15.37:0/692197908 shutdown_connections 2024-06-10T01:02:30.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:30.621+0000 7fb120d8e740 1 -- 172.21.15.37:0/692197908 wait complete. 2024-06-10T01:02:30.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:30.621+0000 7fb120d8e740 20 client.16365 trim_cache size 0 max 16384 2024-06-10T01:02:30.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:02:30.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:02:30.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:02:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:30.622+0000 7fb11dfa5640 1 -- 172.21.15.37:0/1067457080 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb1040a2c30 msgr2=0x7fb1040a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:30.622+0000 7fb11dfa5640 1 --2- 172.21.15.37:0/1067457080 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb1040a2c30 0x7fb1040a50f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55bc6e37a670 tx=0x7fb110009500 comp rx=0 tx=0).stop 2024-06-10T01:02:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:30.622+0000 7fb11dfa5640 1 -- 172.21.15.37:0/1067457080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb118101a00 msgr2=0x7fb118197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:30.622+0000 7fb11dfa5640 1 --2- 172.21.15.37:0/1067457080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb118101a00 0x7fb118197c30 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb10800e9c0 tx=0x7fb10800edc0 comp rx=0 tx=0).stop 2024-06-10T01:02:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:30.622+0000 7fb11dfa5640 1 -- 172.21.15.37:0/1067457080 shutdown_connections 2024-06-10T01:02:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:30.622+0000 7fb11dfa5640 1 --2- 172.21.15.37:0/1067457080 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb1040a2c30 0x7fb1040a50f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:30.622+0000 7fb11dfa5640 1 --2- 172.21.15.37:0/1067457080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1181035b0 0x7fb11819f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:30.622+0000 7fb11dfa5640 1 --2- 172.21.15.37:0/1067457080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb118102bf0 0x7fb118198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:30.622+0000 7fb11dfa5640 1 --2- 172.21.15.37:0/1067457080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb118101a00 0x7fb118197c30 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:30.622+0000 7fb11dfa5640 1 -- 172.21.15.37:0/1067457080 >> 172.21.15.37:0/1067457080 conn(0x7fb1180fcf90 msgr2=0x7fb118105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:30.622+0000 7fb11dfa5640 1 -- 172.21.15.37:0/1067457080 shutdown_connections 2024-06-10T01:02:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:30.622+0000 7fb11dfa5640 1 -- 172.21.15.37:0/1067457080 wait complete. 2024-06-10T01:02:30.643 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:30 smithi037 ceph-mon[32192]: pgmap v604: 129 pgs: 129 active+clean; 18 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 937 op/s 2024-06-10T01:02:32.000 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:02:32.349 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:02:32.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:32 smithi037 ceph-mon[32192]: pgmap v605: 129 pgs: 129 active+clean; 18 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.2 MiB/s wr, 442 op/s 2024-06-10T01:02:32.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:32 smithi167 ceph-mon[36917]: pgmap v605: 129 pgs: 129 active+clean; 18 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.2 MiB/s wr, 442 op/s 2024-06-10T01:02:32.612 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:32 smithi094 ceph-mon[36708]: pgmap v605: 129 pgs: 129 active+clean; 18 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.2 MiB/s wr, 442 op/s 2024-06-10T01:02:34.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:34 smithi167 ceph-mon[36917]: pgmap v606: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 1.5 MiB/s wr, 846 op/s 2024-06-10T01:02:34.612 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:34 smithi094 ceph-mon[36708]: pgmap v606: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 1.5 MiB/s wr, 846 op/s 2024-06-10T01:02:34.634 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:34 smithi037 ceph-mon[32192]: pgmap v606: 129 pgs: 129 active+clean; 12 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 1.5 MiB/s wr, 846 op/s 2024-06-10T01:02:35.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.092+0000 7fb31d4d4700 1 -- 172.21.15.37:0/2569240912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb318102ee0 msgr2=0x7fb31810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:35.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.092+0000 7fb31d4d4700 1 --2- 172.21.15.37:0/2569240912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb318102ee0 0x7fb31810f3c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb31000b0a0 tx=0x7fb31000b3b0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.092+0000 7fb31d4d4700 1 -- 172.21.15.37:0/2569240912 shutdown_connections 2024-06-10T01:02:35.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.092+0000 7fb31d4d4700 1 --2- 172.21.15.37:0/2569240912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb318102ee0 0x7fb31810f3c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.092+0000 7fb31d4d4700 1 --2- 172.21.15.37:0/2569240912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb318107450 0x7fb318107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.092+0000 7fb31d4d4700 1 --2- 172.21.15.37:0/2569240912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb318102540 0x7fb3181029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.092+0000 7fb31d4d4700 1 -- 172.21.15.37:0/2569240912 >> 172.21.15.37:0/2569240912 conn(0x7fb3180795f0 msgr2=0x7fb318079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:35.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.092+0000 7fb31d4d4700 1 -- 172.21.15.37:0/2569240912 shutdown_connections 2024-06-10T01:02:35.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.092+0000 7fb31d4d4700 1 -- 172.21.15.37:0/2569240912 wait complete. 2024-06-10T01:02:35.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.093+0000 7fb31d4d4700 1 Processor -- start 2024-06-10T01:02:35.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.093+0000 7fb31d4d4700 1 -- start start 2024-06-10T01:02:35.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.093+0000 7fb31d4d4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb318102540 0x7fb318074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:35.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.093+0000 7fb31d4d4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb318102ee0 0x7fb318074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:35.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.093+0000 7fb31d4d4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb318107450 0x7fb318075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:35.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.093+0000 7fb31d4d4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb3181127e0 con 0x7fb318102540 2024-06-10T01:02:35.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.093+0000 7fb31d4d4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb318112660 con 0x7fb318107450 2024-06-10T01:02:35.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.093+0000 7fb31d4d4700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb318112960 con 0x7fb318102ee0 2024-06-10T01:02:35.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.093+0000 7fb316ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb318102540 0x7fb318074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:35.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.093+0000 7fb316ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb318102540 0x7fb318074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50796/0 (socket says 172.21.15.37:50796) 2024-06-10T01:02:35.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.093+0000 7fb316ffd700 1 -- 172.21.15.37:0/3858518018 learned_addr learned my addr 172.21.15.37:0/3858518018 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:35.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.094+0000 7fb30e7fc700 1 --2- 172.21.15.37:0/3858518018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb318102ee0 0x7fb318074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:35.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.094+0000 7fb316ffd700 1 -- 172.21.15.37:0/3858518018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb318102ee0 msgr2=0x7fb318074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:35.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.094+0000 7fb316ffd700 1 --2- 172.21.15.37:0/3858518018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb318102ee0 0x7fb318074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.094+0000 7fb316ffd700 1 -- 172.21.15.37:0/3858518018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb318107450 msgr2=0x7fb318075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:35.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.094+0000 7fb3177fe700 1 --2- 172.21.15.37:0/3858518018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb318107450 0x7fb318075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:35.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.094+0000 7fb316ffd700 1 --2- 172.21.15.37:0/3858518018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb318107450 0x7fb318075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.094+0000 7fb316ffd700 1 -- 172.21.15.37:0/3858518018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3181a70d0 con 0x7fb318102540 2024-06-10T01:02:35.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.094+0000 7fb3177fe700 1 --2- 172.21.15.37:0/3858518018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb318107450 0x7fb318075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:02:35.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.094+0000 7fb316ffd700 1 --2- 172.21.15.37:0/3858518018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb318102540 0x7fb318074790 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb30800c990 tx=0x7fb30800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:35.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.094+0000 7fb314ff9700 1 -- 172.21.15.37:0/3858518018 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb308007b00 con 0x7fb318102540 2024-06-10T01:02:35.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.095+0000 7fb31d4d4700 1 -- 172.21.15.37:0/3858518018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3181a73c0 con 0x7fb318102540 2024-06-10T01:02:35.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.095+0000 7fb314ff9700 1 -- 172.21.15.37:0/3858518018 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb308007ca0 con 0x7fb318102540 2024-06-10T01:02:35.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.095+0000 7fb314ff9700 1 -- 172.21.15.37:0/3858518018 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb308018660 con 0x7fb318102540 2024-06-10T01:02:35.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.095+0000 7fb31d4d4700 1 -- 172.21.15.37:0/3858518018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3181a7900 con 0x7fb318102540 2024-06-10T01:02:35.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.097+0000 7fb314ff9700 1 -- 172.21.15.37:0/3858518018 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb30800f450 con 0x7fb318102540 2024-06-10T01:02:35.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.097+0000 7fb314ff9700 1 --2- 172.21.15.37:0/3858518018 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb3040a2c20 0x7fb3040a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:35.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.098+0000 7fb30e7fc700 1 --2- 172.21.15.37:0/3858518018 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb3040a2c20 0x7fb3040a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:35.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.098+0000 7fb314ff9700 1 -- 172.21.15.37:0/3858518018 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fb3080cf300 con 0x7fb318102540 2024-06-10T01:02:35.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.098+0000 7fb30e7fc700 1 --2- 172.21.15.37:0/3858518018 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb3040a2c20 0x7fb3040a50e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb318079b10 tx=0x7fb300009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:35.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.098+0000 7fb327d57b80 1 Processor -- start 2024-06-10T01:02:35.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.098+0000 7fb327d57b80 1 -- start start 2024-06-10T01:02:35.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.099+0000 7fb327d57b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b7c4e7f60 0x560b7c4e8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:35.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.099+0000 7fb327d57b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b7c4e8880 0x560b7c4e2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:35.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.099+0000 7fb327d57b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560b7c4e27a0 0x560b7c4e2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:35.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.099+0000 7fb327d57b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560b7c3d4f60 con 0x560b7c4e8880 2024-06-10T01:02:35.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.099+0000 7fb327d57b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560b7c433850 con 0x560b7c4e27a0 2024-06-10T01:02:35.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.099+0000 7fb327d57b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560b7c39b460 con 0x560b7c4e7f60 2024-06-10T01:02:35.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.099+0000 7fb30e7fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b7c4e8880 0x560b7c4e2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:35.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.099+0000 7fb30e7fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b7c4e8880 0x560b7c4e2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50800/0 (socket says 172.21.15.37:50800) 2024-06-10T01:02:35.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.099+0000 7fb316ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b7c4e7f60 0x560b7c4e8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:35.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.099+0000 7fb30e7fc700 1 -- 172.21.15.37:0/736644695 learned_addr learned my addr 172.21.15.37:0/736644695 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:35.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.099+0000 7fb3177fe700 1 --2- 172.21.15.37:0/736644695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560b7c4e27a0 0x560b7c4e2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:35.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.099+0000 7fb316ffd700 1 -- 172.21.15.37:0/736644695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560b7c4e27a0 msgr2=0x560b7c4e2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:35.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.099+0000 7fb316ffd700 1 --2- 172.21.15.37:0/736644695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560b7c4e27a0 0x560b7c4e2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.099+0000 7fb316ffd700 1 -- 172.21.15.37:0/736644695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b7c4e8880 msgr2=0x560b7c4e2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:35.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.099+0000 7fb316ffd700 1 --2- 172.21.15.37:0/736644695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b7c4e8880 0x560b7c4e2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.099+0000 7fb316ffd700 1 -- 172.21.15.37:0/736644695 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b7c3828d0 con 0x560b7c4e7f60 2024-06-10T01:02:35.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.099+0000 7fb30e7fc700 1 --2- 172.21.15.37:0/736644695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b7c4e8880 0x560b7c4e2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:35.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.100+0000 7fb316ffd700 1 --2- 172.21.15.37:0/736644695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b7c4e7f60 0x560b7c4e8340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb3080cca60 tx=0x7fb30809b5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3338cc73df0b2d5f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:35.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.100+0000 7fb30d7fa700 1 -- 172.21.15.37:0/736644695 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30809be70 con 0x560b7c4e7f60 2024-06-10T01:02:35.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.100+0000 7fb30d7fa700 1 -- 172.21.15.37:0/736644695 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb30809b770 con 0x560b7c4e7f60 2024-06-10T01:02:35.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.100+0000 7fb30d7fa700 1 -- 172.21.15.37:0/736644695 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30809bbf0 con 0x560b7c4e7f60 2024-06-10T01:02:35.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.101+0000 7fb327d57b80 1 -- 172.21.15.37:0/736644695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b7c4e7f60 msgr2=0x560b7c4e8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:35.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.101+0000 7fb327d57b80 1 --2- 172.21.15.37:0/736644695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b7c4e7f60 0x560b7c4e8340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb3080cca60 tx=0x7fb30809b5c0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.101+0000 7fb327d57b80 1 -- 172.21.15.37:0/736644695 shutdown_connections 2024-06-10T01:02:35.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.101+0000 7fb327d57b80 1 --2- 172.21.15.37:0/736644695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560b7c4e27a0 0x560b7c4e2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.101+0000 7fb327d57b80 1 --2- 172.21.15.37:0/736644695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b7c4e7f60 0x560b7c4e8340 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.101+0000 7fb327d57b80 1 --2- 172.21.15.37:0/736644695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b7c4e8880 0x560b7c4e2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.101+0000 7fb327d57b80 1 -- 172.21.15.37:0/736644695 >> 172.21.15.37:0/736644695 conn(0x560b7c40f080 msgr2=0x560b7c4dd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:35.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.101+0000 7fb327d57b80 1 -- 172.21.15.37:0/736644695 shutdown_connections 2024-06-10T01:02:35.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.101+0000 7fb327d57b80 1 -- 172.21.15.37:0/736644695 wait complete. 2024-06-10T01:02:35.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.101+0000 7fb327d57b80 1 -- start start 2024-06-10T01:02:35.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.101+0000 7fb327d57b80 1 Processor -- start 2024-06-10T01:02:35.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.102+0000 7fb327d57b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560b7c4e27a0 0x560b7c4ee690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:35.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.102+0000 7fb327d57b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b7c4e7f60 0x560b7c4eb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:35.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.102+0000 7fb327d57b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b7c4e8880 0x560b7c4ebed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:35.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.102+0000 7fb327d57b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560b7c3d3fc0 con 0x560b7c4e7f60 2024-06-10T01:02:35.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.102+0000 7fb327d57b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560b7c42f810 con 0x560b7c4e27a0 2024-06-10T01:02:35.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.102+0000 7fb327d57b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560b7c4ef200 con 0x560b7c4e8880 2024-06-10T01:02:35.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.102+0000 7fb327d57b80 10 client.0 _ll_register_callbacks cb 0x560b7c3c95b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:02:35.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.102+0000 7fb30e7fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b7c4e7f60 0x560b7c4eb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:35.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.102+0000 7fb3177fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b7c4e8880 0x560b7c4ebed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:35.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.102+0000 7fb3177fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b7c4e8880 0x560b7c4ebed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:37284/0 (socket says 172.21.15.37:37284) 2024-06-10T01:02:35.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.102+0000 7fb3177fe700 1 -- 172.21.15.37:0/1283178054 learned_addr learned my addr 172.21.15.37:0/1283178054 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:35.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.102+0000 7fb30e7fc700 1 --2- 172.21.15.37:0/1283178054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b7c4e7f60 0x560b7c4eb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50804/0 (socket says 172.21.15.37:50804) 2024-06-10T01:02:35.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.102+0000 7fb316ffd700 1 --2- 172.21.15.37:0/1283178054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560b7c4e27a0 0x560b7c4ee690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:35.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.102+0000 7fb3177fe700 1 -- 172.21.15.37:0/1283178054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560b7c4e27a0 msgr2=0x560b7c4ee690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:35.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.102+0000 7fb3177fe700 1 --2- 172.21.15.37:0/1283178054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560b7c4e27a0 0x560b7c4ee690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.102+0000 7fb3177fe700 1 -- 172.21.15.37:0/1283178054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b7c4e7f60 msgr2=0x560b7c4eb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:35.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.102+0000 7fb3177fe700 1 --2- 172.21.15.37:0/1283178054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b7c4e7f60 0x560b7c4eb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.102+0000 7fb3177fe700 1 -- 172.21.15.37:0/1283178054 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b7c3f9750 con 0x560b7c4e8880 2024-06-10T01:02:35.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.102+0000 7fb3177fe700 1 --2- 172.21.15.37:0/1283178054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b7c4e8880 0x560b7c4ebed0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb31000b070 tx=0x7fb31000bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2640f7164d6a18c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:35.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.102+0000 7fb2fe7fc700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:02:35.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.103+0000 7fb2fe7fc700 1 -- 172.21.15.37:0/1283178054 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb310022070 con 0x560b7c4e8880 2024-06-10T01:02:35.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.103+0000 7fb327d57b80 1 -- 172.21.15.37:0/1283178054 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb30809b770 con 0x560b7c4e8880 2024-06-10T01:02:35.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.103+0000 7fb2fe7fc700 1 -- 172.21.15.37:0/1283178054 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fb3100074e0 con 0x560b7c4e8880 2024-06-10T01:02:35.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.103+0000 7fb2fe7fc700 1 -- 172.21.15.37:0/1283178054 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb310004e60 con 0x560b7c4e8880 2024-06-10T01:02:35.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.103+0000 7fb327d57b80 10 client.25936 fetch_fsmap learned FSMap version 27 2024-06-10T01:02:35.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.103+0000 7fb327d57b80 1 -- 172.21.15.37:0/1283178054 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560b7c4f9a10 con 0x560b7c4e8880 2024-06-10T01:02:35.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.103+0000 7fb2fe7fc700 1 -- 172.21.15.37:0/1283178054 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb30809b770 con 0x560b7c4e8880 2024-06-10T01:02:35.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.103+0000 7fb2fe7fc700 1 -- 172.21.15.37:0/1283178054 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb310011070 con 0x560b7c4e8880 2024-06-10T01:02:35.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.103+0000 7fb327d57b80 10 client.25936 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:02:35.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.103+0000 7fb327d57b80 10 client.25936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:02:35.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.103+0000 7fb327d57b80 20 client.25936 populate_metadata read hostname 'smithi037' 2024-06-10T01:02:35.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.103+0000 7fb327d57b80 1 --2- 172.21.15.37:0/1283178054 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560b7c4fbff0 0x560b7c4fe410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:35.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.103+0000 7fb327d57b80 4 client.25936 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:02:35.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.103+0000 7fb327d57b80 1 -- 172.21.15.37:0/1283178054 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b7c440d50 con 0x560b7c4fbff0 2024-06-10T01:02:35.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.104+0000 7fb316ffd700 1 --2- 172.21.15.37:0/1283178054 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560b7c4fbff0 0x560b7c4fe410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:35.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.104+0000 7fb2fe7fc700 10 client.25936 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:35.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.104+0000 7fb316ffd700 1 --2- 172.21.15.37:0/1283178054 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560b7c4fbff0 0x560b7c4fe410 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4708851de4c760b server_cookie=99b0b9d55c306168 in_seq=0 out_seq=0 2024-06-10T01:02:35.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.106+0000 7fb2fe7fc700 1 -- 172.21.15.37:0/1283178054 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560b7c440d50 con 0x560b7c4fbff0 2024-06-10T01:02:35.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.106+0000 7fb2fe7fc700 10 client.25936 handle_command_reply: tid=0 2024-06-10T01:02:35.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.106+0000 7fb327d57b80 1 client.25936 shutdown 2024-06-10T01:02:35.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.106+0000 7fb327d57b80 1 -- 172.21.15.37:0/1283178054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b7c4e8880 msgr2=0x560b7c4ebed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:35.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.106+0000 7fb327d57b80 1 --2- 172.21.15.37:0/1283178054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b7c4e8880 0x560b7c4ebed0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb31000b070 tx=0x7fb31000bce0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.106+0000 7fb327d57b80 1 -- 172.21.15.37:0/1283178054 shutdown_connections 2024-06-10T01:02:35.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.106+0000 7fb327d57b80 1 --2- 172.21.15.37:0/1283178054 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560b7c4fbff0 0x560b7c4fe410 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.107+0000 7fb327d57b80 1 --2- 172.21.15.37:0/1283178054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b7c4e8880 0x560b7c4ebed0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.107+0000 7fb327d57b80 1 --2- 172.21.15.37:0/1283178054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560b7c4e27a0 0x560b7c4ee690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.107+0000 7fb327d57b80 1 --2- 172.21.15.37:0/1283178054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b7c4e7f60 0x560b7c4eb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.107+0000 7fb2fe7fc700 0 client.25936 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:35.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.107+0000 7fb327d57b80 1 -- 172.21.15.37:0/1283178054 >> 172.21.15.37:0/1283178054 conn(0x560b7c40f080 msgr2=0x560b7c4dc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:35.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.107+0000 7fb327d57b80 1 -- 172.21.15.37:0/1283178054 shutdown_connections 2024-06-10T01:02:35.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.107+0000 7fb327d57b80 1 -- 172.21.15.37:0/1283178054 wait complete. 2024-06-10T01:02:35.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.107+0000 7fb327d57b80 20 client.25936 trim_cache size 0 max 16384 2024-06-10T01:02:35.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.127+0000 7fb327d57b80 1 Processor -- start 2024-06-10T01:02:35.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.127+0000 7fb327d57b80 1 -- start start 2024-06-10T01:02:35.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.128+0000 7fb327d57b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b7c4e27a0 0x560b7c4ee1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:35.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.128+0000 7fb327d57b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b7c4e7f60 0x560b7c4dce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:35.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.128+0000 7fb327d57b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560b7c4e8880 0x560b7c4dd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:35.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.128+0000 7fb327d57b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb3181b2f50 con 0x560b7c4e7f60 2024-06-10T01:02:35.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.128+0000 7fb327d57b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb3181b07e0 con 0x560b7c4e8880 2024-06-10T01:02:35.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.128+0000 7fb327d57b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb3181ae630 con 0x560b7c4e27a0 2024-06-10T01:02:35.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.128+0000 7fb30e7fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b7c4e7f60 0x560b7c4dce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:35.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.128+0000 7fb30e7fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b7c4e7f60 0x560b7c4dce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50812/0 (socket says 172.21.15.37:50812) 2024-06-10T01:02:35.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.128+0000 7fb30e7fc700 1 -- 172.21.15.37:0/3419193220 learned_addr learned my addr 172.21.15.37:0/3419193220 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:35.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.128+0000 7fb316ffd700 1 --2- 172.21.15.37:0/3419193220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b7c4e27a0 0x560b7c4ee1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:35.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.128+0000 7fb3177fe700 1 --2- 172.21.15.37:0/3419193220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560b7c4e8880 0x560b7c4dd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:35.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.128+0000 7fb30e7fc700 1 -- 172.21.15.37:0/3419193220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b7c4e27a0 msgr2=0x560b7c4ee1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:35.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.128+0000 7fb30e7fc700 1 --2- 172.21.15.37:0/3419193220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b7c4e27a0 0x560b7c4ee1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.128+0000 7fb30e7fc700 1 -- 172.21.15.37:0/3419193220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560b7c4e8880 msgr2=0x560b7c4dd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:35.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.128+0000 7fb30e7fc700 1 --2- 172.21.15.37:0/3419193220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560b7c4e8880 0x560b7c4dd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.128+0000 7fb30e7fc700 1 -- 172.21.15.37:0/3419193220 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b7c4fea90 con 0x560b7c4e7f60 2024-06-10T01:02:35.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.128+0000 7fb3177fe700 1 --2- 172.21.15.37:0/3419193220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560b7c4e8880 0x560b7c4dd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:02:35.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.128+0000 7fb316ffd700 1 --2- 172.21.15.37:0/3419193220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b7c4e27a0 0x560b7c4ee1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:35.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.129+0000 7fb30e7fc700 1 --2- 172.21.15.37:0/3419193220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b7c4e7f60 0x560b7c4dce80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb300011f40 tx=0x7fb300014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2477603da0f9157 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:35.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.129+0000 7fb30cff9700 1 -- 172.21.15.37:0/3419193220 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300003880 con 0x560b7c4e7f60 2024-06-10T01:02:35.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.129+0000 7fb30cff9700 1 -- 172.21.15.37:0/3419193220 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb30002c070 con 0x560b7c4e7f60 2024-06-10T01:02:35.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.129+0000 7fb30cff9700 1 -- 172.21.15.37:0/3419193220 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300027400 con 0x560b7c4e7f60 2024-06-10T01:02:35.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.130+0000 7fb327d57b80 1 -- 172.21.15.37:0/3419193220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b7c4e7f60 msgr2=0x560b7c4dce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:35.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.130+0000 7fb327d57b80 1 --2- 172.21.15.37:0/3419193220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b7c4e7f60 0x560b7c4dce80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb300011f40 tx=0x7fb300014000 comp rx=0 tx=0).stop 2024-06-10T01:02:35.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.130+0000 7fb327d57b80 1 -- 172.21.15.37:0/3419193220 shutdown_connections 2024-06-10T01:02:35.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.130+0000 7fb327d57b80 1 --2- 172.21.15.37:0/3419193220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560b7c4e8880 0x560b7c4dd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.130+0000 7fb327d57b80 1 --2- 172.21.15.37:0/3419193220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b7c4e27a0 0x560b7c4ee1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.130+0000 7fb327d57b80 1 --2- 172.21.15.37:0/3419193220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b7c4e7f60 0x560b7c4dce80 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.130+0000 7fb327d57b80 1 -- 172.21.15.37:0/3419193220 >> 172.21.15.37:0/3419193220 conn(0x560b7c40f080 msgr2=0x560b7c4ea9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:35.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.130+0000 7fb327d57b80 1 -- 172.21.15.37:0/3419193220 shutdown_connections 2024-06-10T01:02:35.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.130+0000 7fb327d57b80 1 -- 172.21.15.37:0/3419193220 wait complete. 2024-06-10T01:02:35.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.131+0000 7fb327d57b80 1 -- start start 2024-06-10T01:02:35.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.131+0000 7fb327d57b80 1 Processor -- start 2024-06-10T01:02:35.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.131+0000 7fb327d57b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b7c4e27a0 0x560b7c5039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:35.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.131+0000 7fb327d57b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560b7c4e7f60 0x560b7c503f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:35.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.131+0000 7fb327d57b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b7c4e8880 0x560b7c504470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:35.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.131+0000 7fb327d57b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb3181aeca0 con 0x560b7c4e27a0 2024-06-10T01:02:35.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.131+0000 7fb327d57b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb3181b0110 con 0x560b7c4e7f60 2024-06-10T01:02:35.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.131+0000 7fb327d57b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb3181af370 con 0x560b7c4e8880 2024-06-10T01:02:35.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.131+0000 7fb327d57b80 10 client.0 _ll_register_callbacks cb 0x560b7c4e6540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:02:35.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.131+0000 7fb316ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b7c4e27a0 0x560b7c5039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:35.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.131+0000 7fb316ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b7c4e27a0 0x560b7c5039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50822/0 (socket says 172.21.15.37:50822) 2024-06-10T01:02:35.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.131+0000 7fb316ffd700 1 -- 172.21.15.37:0/295020732 learned_addr learned my addr 172.21.15.37:0/295020732 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:35.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.131+0000 7fb3177fe700 1 --2- 172.21.15.37:0/295020732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b7c4e8880 0x560b7c504470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:35.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.132+0000 7fb30e7fc700 1 --2- 172.21.15.37:0/295020732 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560b7c4e7f60 0x560b7c503f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:35.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.132+0000 7fb316ffd700 1 -- 172.21.15.37:0/295020732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b7c4e8880 msgr2=0x560b7c504470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:35.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.132+0000 7fb316ffd700 1 --2- 172.21.15.37:0/295020732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b7c4e8880 0x560b7c504470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.132+0000 7fb316ffd700 1 -- 172.21.15.37:0/295020732 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560b7c4e7f60 msgr2=0x560b7c503f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:35.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.132+0000 7fb316ffd700 1 --2- 172.21.15.37:0/295020732 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560b7c4e7f60 0x560b7c503f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.132+0000 7fb316ffd700 1 -- 172.21.15.37:0/295020732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b7c565db0 con 0x560b7c4e27a0 2024-06-10T01:02:35.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.132+0000 7fb3177fe700 1 --2- 172.21.15.37:0/295020732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b7c4e8880 0x560b7c504470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:35.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.132+0000 7fb30e7fc700 1 --2- 172.21.15.37:0/295020732 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560b7c4e7f60 0x560b7c503f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:02:35.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.132+0000 7fb316ffd700 1 --2- 172.21.15.37:0/295020732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b7c4e27a0 0x560b7c5039f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb3080ccd80 tx=0x7fb308005980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9107e1e4f4e1db81 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:35.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.132+0000 7fb2fe7fc700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:02:35.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.132+0000 7fb2fe7fc700 1 -- 172.21.15.37:0/295020732 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb308064030 con 0x560b7c4e27a0 2024-06-10T01:02:35.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.132+0000 7fb327d57b80 1 -- 172.21.15.37:0/295020732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb30002c070 con 0x560b7c4e27a0 2024-06-10T01:02:35.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.132+0000 7fb2fe7fc700 1 -- 172.21.15.37:0/295020732 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fb308099a70 con 0x560b7c4e27a0 2024-06-10T01:02:35.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.133+0000 7fb2fe7fc700 1 -- 172.21.15.37:0/295020732 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb308082050 con 0x560b7c4e27a0 2024-06-10T01:02:35.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.133+0000 7fb327d57b80 10 client.16383 fetch_fsmap learned FSMap version 27 2024-06-10T01:02:35.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.133+0000 7fb327d57b80 1 -- 172.21.15.37:0/295020732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560b7c567070 con 0x560b7c4e27a0 2024-06-10T01:02:35.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.133+0000 7fb2fe7fc700 1 -- 172.21.15.37:0/295020732 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb30809b770 con 0x560b7c4e27a0 2024-06-10T01:02:35.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.133+0000 7fb2fe7fc700 1 -- 172.21.15.37:0/295020732 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fb3080a3f50 con 0x560b7c4e27a0 2024-06-10T01:02:35.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.133+0000 7fb327d57b80 10 client.16383 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:02:35.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.133+0000 7fb327d57b80 10 client.16383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:02:35.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.133+0000 7fb327d57b80 20 client.16383 populate_metadata read hostname 'smithi037' 2024-06-10T01:02:35.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.133+0000 7fb327d57b80 1 --2- 172.21.15.37:0/295020732 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560b7c567360 0x560b7c569750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:35.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.133+0000 7fb327d57b80 4 client.16383 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:02:35.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.133+0000 7fb327d57b80 1 -- 172.21.15.37:0/295020732 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fb30002c070 con 0x560b7c567360 2024-06-10T01:02:35.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.134+0000 7fb30e7fc700 1 --2- 172.21.15.37:0/295020732 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560b7c567360 0x560b7c569750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:35.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.134+0000 7fb30e7fc700 1 --2- 172.21.15.37:0/295020732 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560b7c567360 0x560b7c569750 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dd610922a5d28b3 server_cookie=690313e86e156eab in_seq=0 out_seq=0 2024-06-10T01:02:35.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.134+0000 7fb2fe7fc700 10 client.16383 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:35.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.136+0000 7fb2fe7fc700 1 -- 172.21.15.37:0/295020732 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fb30002c070 con 0x560b7c567360 2024-06-10T01:02:35.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.136+0000 7fb2fe7fc700 10 client.16383 handle_command_reply: tid=0 2024-06-10T01:02:35.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.136+0000 7fb327d57b80 1 client.16383 shutdown 2024-06-10T01:02:35.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.136+0000 7fb327d57b80 1 -- 172.21.15.37:0/295020732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b7c4e27a0 msgr2=0x560b7c5039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:35.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.136+0000 7fb327d57b80 1 --2- 172.21.15.37:0/295020732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b7c4e27a0 0x560b7c5039f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb3080ccd80 tx=0x7fb308005980 comp rx=0 tx=0).stop 2024-06-10T01:02:35.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.136+0000 7fb327d57b80 1 -- 172.21.15.37:0/295020732 shutdown_connections 2024-06-10T01:02:35.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.136+0000 7fb327d57b80 1 --2- 172.21.15.37:0/295020732 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560b7c567360 0x560b7c569750 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.136+0000 7fb327d57b80 1 --2- 172.21.15.37:0/295020732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560b7c4e8880 0x560b7c504470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.136+0000 7fb327d57b80 1 --2- 172.21.15.37:0/295020732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b7c4e27a0 0x560b7c5039f0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.136+0000 7fb327d57b80 1 --2- 172.21.15.37:0/295020732 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560b7c4e7f60 0x560b7c503f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.136+0000 7fb327d57b80 1 -- 172.21.15.37:0/295020732 >> 172.21.15.37:0/295020732 conn(0x560b7c40f080 msgr2=0x560b7c4e6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:35.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.137+0000 7fb2fe7fc700 0 client.16383 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:35.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.137+0000 7fb327d57b80 1 -- 172.21.15.37:0/295020732 shutdown_connections 2024-06-10T01:02:35.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.137+0000 7fb327d57b80 1 -- 172.21.15.37:0/295020732 wait complete. 2024-06-10T01:02:35.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.137+0000 7fb327d57b80 20 client.16383 trim_cache size 0 max 16384 2024-06-10T01:02:35.137 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:02:35.137 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-06-10T01:02:35.137 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-06-10T01:02:35.137 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:02:35.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.137+0000 7fb31d4d4700 1 -- 172.21.15.37:0/3858518018 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb3040a2c20 msgr2=0x7fb3040a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:35.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.137+0000 7fb31d4d4700 1 --2- 172.21.15.37:0/3858518018 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb3040a2c20 0x7fb3040a50e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb318079b10 tx=0x7fb300009260 comp rx=0 tx=0).stop 2024-06-10T01:02:35.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.137+0000 7fb31d4d4700 1 -- 172.21.15.37:0/3858518018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb318102540 msgr2=0x7fb318074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:35.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.137+0000 7fb31d4d4700 1 --2- 172.21.15.37:0/3858518018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb318102540 0x7fb318074790 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb30800c990 tx=0x7fb30800cca0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.137+0000 7fb31d4d4700 1 -- 172.21.15.37:0/3858518018 shutdown_connections 2024-06-10T01:02:35.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.137+0000 7fb31d4d4700 1 --2- 172.21.15.37:0/3858518018 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb3040a2c20 0x7fb3040a50e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.137+0000 7fb31d4d4700 1 --2- 172.21.15.37:0/3858518018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb318107450 0x7fb318075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.137+0000 7fb31d4d4700 1 --2- 172.21.15.37:0/3858518018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb318102540 0x7fb318074790 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.137+0000 7fb31d4d4700 1 --2- 172.21.15.37:0/3858518018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb318102ee0 0x7fb318074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:35.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.137+0000 7fb31d4d4700 1 -- 172.21.15.37:0/3858518018 >> 172.21.15.37:0/3858518018 conn(0x7fb3180795f0 msgr2=0x7fb3181004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:35.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.138+0000 7fb31d4d4700 1 -- 172.21.15.37:0/3858518018 shutdown_connections 2024-06-10T01:02:35.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:35.138+0000 7fb31d4d4700 1 -- 172.21.15.37:0/3858518018 wait complete. 2024-06-10T01:02:36.158 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:02:36.158 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:02:36.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:36 smithi037 ceph-mon[32192]: pgmap v607: 129 pgs: 129 active+clean; 6.4 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 1.32k op/s 2024-06-10T01:02:36.414 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:36 smithi094 ceph-mon[36708]: pgmap v607: 129 pgs: 129 active+clean; 6.4 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 1.32k op/s 2024-06-10T01:02:36.421 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:02:36.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:36 smithi167 ceph-mon[36917]: pgmap v607: 129 pgs: 129 active+clean; 6.4 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 1.32k op/s 2024-06-10T01:02:38.415 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:38 smithi037 ceph-mon[32192]: pgmap v608: 129 pgs: 129 active+clean; 6.4 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.5 MiB/s wr, 1.32k op/s 2024-06-10T01:02:38.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:38 smithi167 ceph-mon[36917]: pgmap v608: 129 pgs: 129 active+clean; 6.4 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.5 MiB/s wr, 1.32k op/s 2024-06-10T01:02:38.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:38 smithi094 ceph-mon[36708]: pgmap v608: 129 pgs: 129 active+clean; 6.4 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.5 MiB/s wr, 1.32k op/s 2024-06-10T01:02:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.813+0000 7f0544dab700 1 -- 172.21.15.37:0/2505752673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f054006ab30 msgr2=0x7f054010f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:38.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.813+0000 7f0544dab700 1 --2- 172.21.15.37:0/2505752673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f054006ab30 0x7f054010f320 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f053800b0a0 tx=0x7f053800b3b0 comp rx=0 tx=0).stop 2024-06-10T01:02:38.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.813+0000 7f0544dab700 1 -- 172.21.15.37:0/2505752673 shutdown_connections 2024-06-10T01:02:38.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.813+0000 7f0544dab700 1 --2- 172.21.15.37:0/2505752673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f054006ab30 0x7f054010f320 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:38.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.813+0000 7f0544dab700 1 --2- 172.21.15.37:0/2505752673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0540108500 0x7f05401088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:38.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.813+0000 7f0544dab700 1 --2- 172.21.15.37:0/2505752673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f054006a170 0x7f054006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:38.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.813+0000 7f0544dab700 1 -- 172.21.15.37:0/2505752673 >> 172.21.15.37:0/2505752673 conn(0x7f0540078340 msgr2=0x7f0540078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:38.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.813+0000 7f0544dab700 1 -- 172.21.15.37:0/2505752673 shutdown_connections 2024-06-10T01:02:38.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.813+0000 7f0544dab700 1 -- 172.21.15.37:0/2505752673 wait complete. 2024-06-10T01:02:38.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.814+0000 7f0544dab700 1 Processor -- start 2024-06-10T01:02:38.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.814+0000 7f0544dab700 1 -- start start 2024-06-10T01:02:38.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.815+0000 7f0544dab700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f054006a170 0x7f054019af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:38.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.815+0000 7f0544dab700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f054006ab30 0x7f054019b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:38.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.815+0000 7f0544dab700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0540108500 0x7f054019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:38.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.815+0000 7f0544dab700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0540112880 con 0x7f054006a170 2024-06-10T01:02:38.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.815+0000 7f0544dab700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0540112700 con 0x7f054006ab30 2024-06-10T01:02:38.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.815+0000 7f0544dab700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0540112a00 con 0x7f0540108500 2024-06-10T01:02:38.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.815+0000 7f053e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f054006a170 0x7f054019af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:38.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.815+0000 7f053e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f054006a170 0x7f054019af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50848/0 (socket says 172.21.15.37:50848) 2024-06-10T01:02:38.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.815+0000 7f053e59c700 1 -- 172.21.15.37:0/3819059497 learned_addr learned my addr 172.21.15.37:0/3819059497 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.815+0000 7f053ed9d700 1 --2- 172.21.15.37:0/3819059497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0540108500 0x7f054019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.815+0000 7f0537fff700 1 --2- 172.21.15.37:0/3819059497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f054006ab30 0x7f054019b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.815+0000 7f053e59c700 1 -- 172.21.15.37:0/3819059497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0540108500 msgr2=0x7f054019f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.815+0000 7f053e59c700 1 --2- 172.21.15.37:0/3819059497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0540108500 0x7f054019f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.815+0000 7f053e59c700 1 -- 172.21.15.37:0/3819059497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f054006ab30 msgr2=0x7f054019b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.815+0000 7f053e59c700 1 --2- 172.21.15.37:0/3819059497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f054006ab30 0x7f054019b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.815+0000 7f053e59c700 1 -- 172.21.15.37:0/3819059497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054019ff20 con 0x7f054006a170 2024-06-10T01:02:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.815+0000 7f053ed9d700 1 --2- 172.21.15.37:0/3819059497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0540108500 0x7f054019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.815+0000 7f053e59c700 1 --2- 172.21.15.37:0/3819059497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f054006a170 0x7f054019af10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f053000b530 tx=0x7f053000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.816+0000 7f05377fe700 1 -- 172.21.15.37:0/3819059497 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0530015020 con 0x7f054006a170 2024-06-10T01:02:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.816+0000 7f0544dab700 1 -- 172.21.15.37:0/3819059497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05401a0210 con 0x7f054006a170 2024-06-10T01:02:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.816+0000 7f05377fe700 1 -- 172.21.15.37:0/3819059497 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0530011d10 con 0x7f054006a170 2024-06-10T01:02:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.816+0000 7f05377fe700 1 -- 172.21.15.37:0/3819059497 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053000f2e0 con 0x7f054006a170 2024-06-10T01:02:38.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.816+0000 7f0544dab700 1 -- 172.21.15.37:0/3819059497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05401a7a90 con 0x7f054006a170 2024-06-10T01:02:38.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.817+0000 7f0544dab700 1 -- 172.21.15.37:0/3819059497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0540069180 con 0x7f054006a170 2024-06-10T01:02:38.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.818+0000 7f05377fe700 1 -- 172.21.15.37:0/3819059497 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0530018020 con 0x7f054006a170 2024-06-10T01:02:38.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.819+0000 7f05377fe700 1 --2- 172.21.15.37:0/3819059497 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f052c0a2ca0 0x7f052c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:38.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.819+0000 7f0537fff700 1 --2- 172.21.15.37:0/3819059497 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f052c0a2ca0 0x7f052c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:38.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.819+0000 7f05377fe700 1 -- 172.21.15.37:0/3819059497 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f05300cf040 con 0x7f054006a170 2024-06-10T01:02:38.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.820+0000 7f0537fff700 1 --2- 172.21.15.37:0/3819059497 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f052c0a2ca0 0x7f052c0a5160 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f054010f760 tx=0x7f0528009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:38.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:38.823+0000 7f05377fe700 1 -- 172.21.15.37:0/3819059497 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f053001f020 con 0x7f054006a170 2024-06-10T01:02:39.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:39.079+0000 7f0544dab700 1 -- 172.21.15.37:0/3819059497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05401a7d50 con 0x7f054006a170 2024-06-10T01:02:39.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:39.080+0000 7f05377fe700 1 -- 172.21.15.37:0/3819059497 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11867 (secure 0 0 0) 0x7f053009bbb0 con 0x7f054006a170 2024-06-10T01:02:39.078 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:02:39.078 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T00:56:30.291171+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24451,24458,24472]},"id":1}]} 2024-06-10T01:02:39.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:39.082+0000 7f0544dab700 1 -- 172.21.15.37:0/3819059497 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f052c0a2ca0 msgr2=0x7f052c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:39.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:39.082+0000 7f0544dab700 1 --2- 172.21.15.37:0/3819059497 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f052c0a2ca0 0x7f052c0a5160 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f054010f760 tx=0x7f0528009190 comp rx=0 tx=0).stop 2024-06-10T01:02:39.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:39.082+0000 7f0544dab700 1 -- 172.21.15.37:0/3819059497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f054006a170 msgr2=0x7f054019af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:39.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:39.082+0000 7f0544dab700 1 --2- 172.21.15.37:0/3819059497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f054006a170 0x7f054019af10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f053000b530 tx=0x7f053000b840 comp rx=0 tx=0).stop 2024-06-10T01:02:39.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:39.082+0000 7f0544dab700 1 -- 172.21.15.37:0/3819059497 shutdown_connections 2024-06-10T01:02:39.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:39.082+0000 7f0544dab700 1 --2- 172.21.15.37:0/3819059497 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f052c0a2ca0 0x7f052c0a5160 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:39.082+0000 7f0544dab700 1 --2- 172.21.15.37:0/3819059497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0540108500 0x7f054019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:39.082+0000 7f0544dab700 1 --2- 172.21.15.37:0/3819059497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f054006a170 0x7f054019af10 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:39.082+0000 7f0544dab700 1 --2- 172.21.15.37:0/3819059497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f054006ab30 0x7f054019b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:39.082+0000 7f0544dab700 1 -- 172.21.15.37:0/3819059497 >> 172.21.15.37:0/3819059497 conn(0x7f0540078340 msgr2=0x7f05400ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:39.082+0000 7f0544dab700 1 -- 172.21.15.37:0/3819059497 shutdown_connections 2024-06-10T01:02:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:39.082+0000 7f0544dab700 1 -- 172.21.15.37:0/3819059497 wait complete. 2024-06-10T01:02:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-06-10T01:02:39.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:39 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3819059497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:02:39.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:39 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3819059497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:02:39.612 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:39 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3819059497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:02:39.844 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:02:40.102 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:02:40.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:40 smithi167 ceph-mon[36917]: pgmap v609: 129 pgs: 129 active+clean; 9.1 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 1.32k op/s 2024-06-10T01:02:40.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:40 smithi094 ceph-mon[36708]: pgmap v609: 129 pgs: 129 active+clean; 9.1 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 1.32k op/s 2024-06-10T01:02:40.643 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:40 smithi037 ceph-mon[32192]: pgmap v609: 129 pgs: 129 active+clean; 9.1 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 1.32k op/s 2024-06-10T01:02:42.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:42 smithi037 ceph-mon[32192]: pgmap v610: 129 pgs: 129 active+clean; 13 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 1.2 MiB/s wr, 1.17k op/s 2024-06-10T01:02:42.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.458+0000 7f95c6737700 1 -- 172.21.15.37:0/1408126161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c0106720 msgr2=0x7f95c010e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:42.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.458+0000 7f95c6737700 1 --2- 172.21.15.37:0/1408126161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c0106720 0x7f95c010e2c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f95b800b0d0 tx=0x7f95b800b3e0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.458+0000 7f95c6737700 1 -- 172.21.15.37:0/1408126161 shutdown_connections 2024-06-10T01:02:42.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.458+0000 7f95c6737700 1 --2- 172.21.15.37:0/1408126161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c0106720 0x7f95c010e2c0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.458+0000 7f95c6737700 1 --2- 172.21.15.37:0/1408126161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95c01019d0 0x7f95c0101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.458+0000 7f95c6737700 1 --2- 172.21.15.37:0/1408126161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c0102380 0x7f95c0105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.458+0000 7f95c6737700 1 -- 172.21.15.37:0/1408126161 >> 172.21.15.37:0/1408126161 conn(0x7f95c0078340 msgr2=0x7f95c0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:42.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.458+0000 7f95c6737700 1 -- 172.21.15.37:0/1408126161 shutdown_connections 2024-06-10T01:02:42.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.458+0000 7f95c6737700 1 -- 172.21.15.37:0/1408126161 wait complete. 2024-06-10T01:02:42.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.459+0000 7f95c6737700 1 Processor -- start 2024-06-10T01:02:42.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.459+0000 7f95c6737700 1 -- start start 2024-06-10T01:02:42.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.459+0000 7f95c6737700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c01019d0 0x7f95c019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:42.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.459+0000 7f95c6737700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c0102380 0x7f95c019edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:42.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.459+0000 7f95c6737700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95c0106720 0x7f95c0198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:42.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.459+0000 7f95c6737700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f95c01116e0 con 0x7f95c0102380 2024-06-10T01:02:42.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.459+0000 7f95c6737700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f95c0111560 con 0x7f95c0106720 2024-06-10T01:02:42.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.459+0000 7f95c6737700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f95c0111860 con 0x7f95c01019d0 2024-06-10T01:02:42.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.459+0000 7f95b7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c0102380 0x7f95c019edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:42.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.459+0000 7f95b7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c0102380 0x7f95c019edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55396/0 (socket says 172.21.15.37:55396) 2024-06-10T01:02:42.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.459+0000 7f95b7fff700 1 -- 172.21.15.37:0/2710127996 learned_addr learned my addr 172.21.15.37:0/2710127996 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:42.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.459+0000 7f95bffff700 1 --2- 172.21.15.37:0/2710127996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c01019d0 0x7f95c019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:42.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.459+0000 7f95c4cd4700 1 --2- 172.21.15.37:0/2710127996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95c0106720 0x7f95c0198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:42.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.460+0000 7f95b7fff700 1 -- 172.21.15.37:0/2710127996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c01019d0 msgr2=0x7f95c019e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:42.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.460+0000 7f95b7fff700 1 --2- 172.21.15.37:0/2710127996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c01019d0 0x7f95c019e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.460+0000 7f95b7fff700 1 -- 172.21.15.37:0/2710127996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95c0106720 msgr2=0x7f95c0198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:42.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.460+0000 7f95b7fff700 1 --2- 172.21.15.37:0/2710127996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95c0106720 0x7f95c0198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.460+0000 7f95b7fff700 1 -- 172.21.15.37:0/2710127996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c01991a0 con 0x7f95c0102380 2024-06-10T01:02:42.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.460+0000 7f95b7fff700 1 --2- 172.21.15.37:0/2710127996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c0102380 0x7f95c019edc0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f95a800e940 tx=0x7f95a800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:42.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.460+0000 7f95bdffb700 1 -- 172.21.15.37:0/2710127996 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a800c9e0 con 0x7f95c0102380 2024-06-10T01:02:42.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.460+0000 7f95bdffb700 1 -- 172.21.15.37:0/2710127996 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95a800cb80 con 0x7f95c0102380 2024-06-10T01:02:42.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.460+0000 7f95bdffb700 1 -- 172.21.15.37:0/2710127996 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8018660 con 0x7f95c0102380 2024-06-10T01:02:42.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.460+0000 7f95c6737700 1 -- 172.21.15.37:0/2710127996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95c0199490 con 0x7f95c0102380 2024-06-10T01:02:42.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.460+0000 7f95c6737700 1 -- 172.21.15.37:0/2710127996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95c0102fc0 con 0x7f95c0102380 2024-06-10T01:02:42.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.463+0000 7f95bdffb700 1 -- 172.21.15.37:0/2710127996 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f95a8010430 con 0x7f95c0102380 2024-06-10T01:02:42.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.463+0000 7f95bdffb700 1 --2- 172.21.15.37:0/2710127996 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f95ac0a2ca0 0x7f95ac0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:42.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.463+0000 7f95bdffb700 1 -- 172.21.15.37:0/2710127996 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f95a80cf3c0 con 0x7f95c0102380 2024-06-10T01:02:42.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.464+0000 7f95bffff700 1 --2- 172.21.15.37:0/2710127996 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f95ac0a2ca0 0x7f95ac0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:42.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.464+0000 7f95d0fbab80 1 Processor -- start 2024-06-10T01:02:42.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.464+0000 7f95bffff700 1 --2- 172.21.15.37:0/2710127996 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f95ac0a2ca0 0x7f95ac0a5160 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f95c0199b80 tx=0x7f95b00073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:42.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.464+0000 7f95d0fbab80 1 -- start start 2024-06-10T01:02:42.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.464+0000 7f95d0fbab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567792f3f60 0x5567792f4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:42.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.464+0000 7f95d0fbab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567792f4880 0x5567792ee260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:42.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.464+0000 7f95d0fbab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567792ee7a0 0x5567792eebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:42.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.464+0000 7f95d0fbab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5567791e0f60 con 0x5567792ee7a0 2024-06-10T01:02:42.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.464+0000 7f95d0fbab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55677923f850 con 0x5567792f3f60 2024-06-10T01:02:42.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.464+0000 7f95d0fbab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5567791a7460 con 0x5567792f4880 2024-06-10T01:02:42.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.464+0000 7f95b7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567792f4880 0x5567792ee260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:42.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.464+0000 7f95b7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567792f4880 0x5567792ee260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:59232/0 (socket says 172.21.15.37:59232) 2024-06-10T01:02:42.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.464+0000 7f95b7fff700 1 -- 172.21.15.37:0/3155542832 learned_addr learned my addr 172.21.15.37:0/3155542832 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:42.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.465+0000 7f95bffff700 1 --2- 172.21.15.37:0/3155542832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567792f3f60 0x5567792f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:42.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.465+0000 7f95c4cd4700 1 --2- 172.21.15.37:0/3155542832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567792ee7a0 0x5567792eebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:42.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.465+0000 7f95bffff700 1 -- 172.21.15.37:0/3155542832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567792f4880 msgr2=0x5567792ee260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:42.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.465+0000 7f95bffff700 1 --2- 172.21.15.37:0/3155542832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567792f4880 0x5567792ee260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.465+0000 7f95bffff700 1 -- 172.21.15.37:0/3155542832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567792ee7a0 msgr2=0x5567792eebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:42.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.465+0000 7f95bffff700 1 --2- 172.21.15.37:0/3155542832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567792ee7a0 0x5567792eebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.465+0000 7f95bffff700 1 -- 172.21.15.37:0/3155542832 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55677918e8d0 con 0x5567792f3f60 2024-06-10T01:02:42.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.465+0000 7f95b7fff700 1 --2- 172.21.15.37:0/3155542832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567792f4880 0x5567792ee260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:02:42.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.465+0000 7f95c4cd4700 1 --2- 172.21.15.37:0/3155542832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567792ee7a0 0x5567792eebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:02:42.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.465+0000 7f95bffff700 1 --2- 172.21.15.37:0/3155542832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567792f3f60 0x5567792f4340 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f95b0010760 tx=0x7f95b0013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ee89b99d6df26a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:42.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.466+0000 7f95b67fc700 1 -- 172.21.15.37:0/3155542832 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b0011b80 con 0x5567792f3f60 2024-06-10T01:02:42.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.466+0000 7f95b67fc700 1 -- 172.21.15.37:0/3155542832 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95b0011d20 con 0x5567792f3f60 2024-06-10T01:02:42.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.466+0000 7f95b67fc700 1 -- 172.21.15.37:0/3155542832 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b001c7e0 con 0x5567792f3f60 2024-06-10T01:02:42.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.466+0000 7f95d0fbab80 1 -- 172.21.15.37:0/3155542832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567792f3f60 msgr2=0x5567792f4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:42.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.466+0000 7f95d0fbab80 1 --2- 172.21.15.37:0/3155542832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567792f3f60 0x5567792f4340 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f95b0010760 tx=0x7f95b0013000 comp rx=0 tx=0).stop 2024-06-10T01:02:42.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.466+0000 7f95d0fbab80 1 -- 172.21.15.37:0/3155542832 shutdown_connections 2024-06-10T01:02:42.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.466+0000 7f95d0fbab80 1 --2- 172.21.15.37:0/3155542832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567792ee7a0 0x5567792eebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.466+0000 7f95d0fbab80 1 --2- 172.21.15.37:0/3155542832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567792f3f60 0x5567792f4340 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.466+0000 7f95d0fbab80 1 --2- 172.21.15.37:0/3155542832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567792f4880 0x5567792ee260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.466+0000 7f95d0fbab80 1 -- 172.21.15.37:0/3155542832 >> 172.21.15.37:0/3155542832 conn(0x55677921b080 msgr2=0x5567792e9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:42.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.467+0000 7f95d0fbab80 1 -- 172.21.15.37:0/3155542832 shutdown_connections 2024-06-10T01:02:42.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.467+0000 7f95d0fbab80 1 -- 172.21.15.37:0/3155542832 wait complete. 2024-06-10T01:02:42.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.467+0000 7f95d0fbab80 1 -- start start 2024-06-10T01:02:42.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.467+0000 7f95d0fbab80 1 Processor -- start 2024-06-10T01:02:42.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.468+0000 7f95d0fbab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567792ee7a0 0x5567792fa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:42.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.468+0000 7f95d0fbab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567792f3f60 0x5567792f7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:42.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.468+0000 7f95d0fbab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567792f4880 0x5567792f7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:42.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.468+0000 7f95d0fbab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5567791dffc0 con 0x5567792f4880 2024-06-10T01:02:42.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.468+0000 7f95d0fbab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55677923b810 con 0x5567792ee7a0 2024-06-10T01:02:42.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.468+0000 7f95d0fbab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5567792fb260 con 0x5567792f3f60 2024-06-10T01:02:42.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.468+0000 7f95d0fbab80 10 client.0 _ll_register_callbacks cb 0x5567791d55b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:02:42.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.468+0000 7f95b7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567792f3f60 0x5567792f7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:42.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.468+0000 7f95b7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567792f3f60 0x5567792f7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:59234/0 (socket says 172.21.15.37:59234) 2024-06-10T01:02:42.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.468+0000 7f95c4cd4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567792f4880 0x5567792f7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:42.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.468+0000 7f95b7fff700 1 -- 172.21.15.37:0/4213210588 learned_addr learned my addr 172.21.15.37:0/4213210588 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:42.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.468+0000 7f95bffff700 1 --2- 172.21.15.37:0/4213210588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567792ee7a0 0x5567792fa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:42.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.468+0000 7f95b7fff700 1 -- 172.21.15.37:0/4213210588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567792ee7a0 msgr2=0x5567792fa670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:42.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.468+0000 7f95b7fff700 1 --2- 172.21.15.37:0/4213210588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567792ee7a0 0x5567792fa670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.468+0000 7f95b7fff700 1 -- 172.21.15.37:0/4213210588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567792f4880 msgr2=0x5567792f7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:42.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.468+0000 7f95b7fff700 1 --2- 172.21.15.37:0/4213210588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567792f4880 0x5567792f7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.468+0000 7f95b7fff700 1 -- 172.21.15.37:0/4213210588 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556779205750 con 0x5567792f3f60 2024-06-10T01:02:42.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.468+0000 7f95b7fff700 1 --2- 172.21.15.37:0/4213210588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567792f3f60 0x5567792f7970 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009860 tx=0x7f95a8086390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5edb84d4406e8ad3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:42.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.468+0000 7f959f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:02:42.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.469+0000 7f959f7fe700 1 -- 172.21.15.37:0/4213210588 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8084e60 con 0x5567792f3f60 2024-06-10T01:02:42.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.469+0000 7f959f7fe700 1 -- 172.21.15.37:0/4213210588 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f95a8099bd0 con 0x5567792f3f60 2024-06-10T01:02:42.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.469+0000 7f95d0fbab80 1 -- 172.21.15.37:0/4213210588 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95b0011d20 con 0x5567792f3f60 2024-06-10T01:02:42.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.469+0000 7f959f7fe700 1 -- 172.21.15.37:0/4213210588 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f95a809b8d0 con 0x5567792f3f60 2024-06-10T01:02:42.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.469+0000 7f95d0fbab80 10 client.25957 fetch_fsmap learned FSMap version 27 2024-06-10T01:02:42.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.469+0000 7f95d0fbab80 1 -- 172.21.15.37:0/4213210588 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55677930e2c0 con 0x5567792f3f60 2024-06-10T01:02:42.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.469+0000 7f959f7fe700 1 -- 172.21.15.37:0/4213210588 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95b0011d20 con 0x5567792f3f60 2024-06-10T01:02:42.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.469+0000 7f959f7fe700 1 -- 172.21.15.37:0/4213210588 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f95a8010720 con 0x5567792f3f60 2024-06-10T01:02:42.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.469+0000 7f95d0fbab80 10 client.25957 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:02:42.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.469+0000 7f95d0fbab80 10 client.25957 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-10T01:02:42.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.469+0000 7f95d0fbab80 20 client.25957 populate_metadata read hostname 'smithi037' 2024-06-10T01:02:42.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.469+0000 7f95d0fbab80 1 --2- 172.21.15.37:0/4213210588 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5567793109a0 0x556779312d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:42.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.469+0000 7f95d0fbab80 4 client.25957 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:02:42.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.469+0000 7f95d0fbab80 1 -- 172.21.15.37:0/4213210588 --> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55677924cd50 con 0x5567793109a0 2024-06-10T01:02:42.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.470+0000 7f95bffff700 1 --2- 172.21.15.37:0/4213210588 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5567793109a0 0x556779312d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:42.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.470+0000 7f95bffff700 1 --2- 172.21.15.37:0/4213210588 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5567793109a0 0x556779312d90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=44a531125fddb236 server_cookie=e49486d713f2bbc0 in_seq=0 out_seq=0 2024-06-10T01:02:42.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.470+0000 7f959f7fe700 10 client.25957 ms_handle_connect on v2:172.21.15.167:6836/2720148065 2024-06-10T01:02:42.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.471+0000 7f959f7fe700 1 -- 172.21.15.37:0/4213210588 <== mds.2 v2:172.21.15.167:6836/2720148065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55677924cd50 con 0x5567793109a0 2024-06-10T01:02:42.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.471+0000 7f959f7fe700 10 client.25957 handle_command_reply: tid=0 2024-06-10T01:02:42.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.471+0000 7f95d0fbab80 1 client.25957 shutdown 2024-06-10T01:02:42.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.471+0000 7f95d0fbab80 1 -- 172.21.15.37:0/4213210588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567792f3f60 msgr2=0x5567792f7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:42.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.471+0000 7f95d0fbab80 1 --2- 172.21.15.37:0/4213210588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567792f3f60 0x5567792f7970 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009860 tx=0x7f95a8086390 comp rx=0 tx=0).stop 2024-06-10T01:02:42.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.471+0000 7f95d0fbab80 1 -- 172.21.15.37:0/4213210588 shutdown_connections 2024-06-10T01:02:42.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.471+0000 7f95d0fbab80 1 --2- 172.21.15.37:0/4213210588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567792f4880 0x5567792f7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.471+0000 7f95d0fbab80 1 --2- 172.21.15.37:0/4213210588 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5567793109a0 0x556779312d90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.471+0000 7f95d0fbab80 1 --2- 172.21.15.37:0/4213210588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567792ee7a0 0x5567792fa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.471+0000 7f95d0fbab80 1 --2- 172.21.15.37:0/4213210588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567792f3f60 0x5567792f7970 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.471+0000 7f959f7fe700 0 client.25957 ms_handle_reset on v2:172.21.15.167:6836/2720148065 2024-06-10T01:02:42.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.471+0000 7f95d0fbab80 1 -- 172.21.15.37:0/4213210588 >> 172.21.15.37:0/4213210588 conn(0x55677921b080 msgr2=0x5567792e8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:42.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.471+0000 7f95d0fbab80 1 -- 172.21.15.37:0/4213210588 shutdown_connections 2024-06-10T01:02:42.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.471+0000 7f95d0fbab80 1 -- 172.21.15.37:0/4213210588 wait complete. 2024-06-10T01:02:42.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.472+0000 7f95d0fbab80 20 client.25957 trim_cache size 0 max 16384 2024-06-10T01:02:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.487+0000 7f95d0fbab80 1 Processor -- start 2024-06-10T01:02:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.487+0000 7f95d0fbab80 1 -- start start 2024-06-10T01:02:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.487+0000 7f95d0fbab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567792ee7a0 0x5567792f6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.487+0000 7f95d0fbab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567792f3f60 0x5567792e8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.487+0000 7f95d0fbab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567792f4880 0x5567792e93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.487+0000 7f95d0fbab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f95c01aeb30 con 0x5567792f4880 2024-06-10T01:02:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.487+0000 7f95d0fbab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f95c01ac3c0 con 0x5567792f3f60 2024-06-10T01:02:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.487+0000 7f95d0fbab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f95c01aa210 con 0x5567792ee7a0 2024-06-10T01:02:42.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.487+0000 7f95c4cd4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567792f4880 0x5567792e93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:42.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.487+0000 7f95bffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567792ee7a0 0x5567792f6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:42.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.487+0000 7f95c4cd4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567792f4880 0x5567792e93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55422/0 (socket says 172.21.15.37:55422) 2024-06-10T01:02:42.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.487+0000 7f95c4cd4700 1 -- 172.21.15.37:0/3241856033 learned_addr learned my addr 172.21.15.37:0/3241856033 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:42.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.487+0000 7f95b7fff700 1 --2- 172.21.15.37:0/3241856033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567792f3f60 0x5567792e8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:42.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.487+0000 7f95c4cd4700 1 -- 172.21.15.37:0/3241856033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567792ee7a0 msgr2=0x5567792f6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:42.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.487+0000 7f95c4cd4700 1 --2- 172.21.15.37:0/3241856033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567792ee7a0 0x5567792f6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.487+0000 7f95c4cd4700 1 -- 172.21.15.37:0/3241856033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567792f3f60 msgr2=0x5567792e8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:42.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.487+0000 7f95c4cd4700 1 --2- 172.21.15.37:0/3241856033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567792f3f60 0x5567792e8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.487+0000 7f95c4cd4700 1 -- 172.21.15.37:0/3241856033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567792ee5a0 con 0x5567792f4880 2024-06-10T01:02:42.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.488+0000 7f95c4cd4700 1 --2- 172.21.15.37:0/3241856033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567792f4880 0x5567792e93c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f95b8009520 tx=0x7f95b8009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87fd57e37337294c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:42.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.488+0000 7f95b5ffb700 1 -- 172.21.15.37:0/3241856033 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b800bd20 con 0x5567792f4880 2024-06-10T01:02:42.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.488+0000 7f95b5ffb700 1 -- 172.21.15.37:0/3241856033 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95b80074e0 con 0x5567792f4880 2024-06-10T01:02:42.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.488+0000 7f95b5ffb700 1 -- 172.21.15.37:0/3241856033 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b8005300 con 0x5567792f4880 2024-06-10T01:02:42.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.489+0000 7f95d0fbab80 1 -- 172.21.15.37:0/3241856033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567792f4880 msgr2=0x5567792e93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:42.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.489+0000 7f95d0fbab80 1 --2- 172.21.15.37:0/3241856033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567792f4880 0x5567792e93c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f95b8009520 tx=0x7f95b8009550 comp rx=0 tx=0).stop 2024-06-10T01:02:42.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.489+0000 7f95d0fbab80 1 -- 172.21.15.37:0/3241856033 shutdown_connections 2024-06-10T01:02:42.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.489+0000 7f95d0fbab80 1 --2- 172.21.15.37:0/3241856033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567792f4880 0x5567792e93c0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.489+0000 7f95d0fbab80 1 --2- 172.21.15.37:0/3241856033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567792ee7a0 0x5567792f6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.489+0000 7f95d0fbab80 1 --2- 172.21.15.37:0/3241856033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567792f3f60 0x5567792e8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.489+0000 7f95d0fbab80 1 -- 172.21.15.37:0/3241856033 >> 172.21.15.37:0/3241856033 conn(0x55677921b080 msgr2=0x5567792f6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:42.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.489+0000 7f95d0fbab80 1 -- 172.21.15.37:0/3241856033 shutdown_connections 2024-06-10T01:02:42.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.489+0000 7f95d0fbab80 1 -- 172.21.15.37:0/3241856033 wait complete. 2024-06-10T01:02:42.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.489+0000 7f95d0fbab80 1 -- start start 2024-06-10T01:02:42.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.489+0000 7f95d0fbab80 1 Processor -- start 2024-06-10T01:02:42.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.490+0000 7f95d0fbab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567792ee7a0 0x5567793042f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:42.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.490+0000 7f95d0fbab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567792f3f60 0x556779304830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:42.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.490+0000 7f95d0fbab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567792f4880 0x556779379600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:42.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.490+0000 7f95d0fbab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f95c01aa880 con 0x5567792f4880 2024-06-10T01:02:42.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.490+0000 7f95d0fbab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f95c01abcf0 con 0x5567792ee7a0 2024-06-10T01:02:42.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.490+0000 7f95d0fbab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f95c01aaf50 con 0x5567792f3f60 2024-06-10T01:02:42.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.490+0000 7f95d0fbab80 10 client.0 _ll_register_callbacks cb 0x5567791d55b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:02:42.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.490+0000 7f95b7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567792f3f60 0x556779304830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:42.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.490+0000 7f95bffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567792ee7a0 0x5567793042f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:42.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.490+0000 7f95c4cd4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567792f4880 0x556779379600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:42.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.490+0000 7f95b7fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567792f3f60 0x556779304830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:59252/0 (socket says 172.21.15.37:59252) 2024-06-10T01:02:42.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.490+0000 7f95bffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567792ee7a0 0x5567793042f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:36286/0 (socket says 172.21.15.37:36286) 2024-06-10T01:02:42.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.490+0000 7f95b7fff700 1 -- 172.21.15.37:0/911940329 learned_addr learned my addr 172.21.15.37:0/911940329 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:42.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.490+0000 7f95b7fff700 1 -- 172.21.15.37:0/911940329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567792ee7a0 msgr2=0x5567793042f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:42.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.490+0000 7f95b7fff700 1 --2- 172.21.15.37:0/911940329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567792ee7a0 0x5567793042f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.490+0000 7f95b7fff700 1 -- 172.21.15.37:0/911940329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567792f4880 msgr2=0x556779379600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:42.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.491+0000 7f95b7fff700 1 --2- 172.21.15.37:0/911940329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567792f4880 0x556779379600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.491+0000 7f95b7fff700 1 -- 172.21.15.37:0/911940329 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556779379b90 con 0x5567792f3f60 2024-06-10T01:02:42.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.491+0000 7f95c4cd4700 1 --2- 172.21.15.37:0/911940329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567792f4880 0x556779379600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:42.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.491+0000 7f95b7fff700 1 --2- 172.21.15.37:0/911940329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567792f3f60 0x556779304830 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f95a80cf750 tx=0x7f95a809be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f26a8a081771c91b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:42.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.491+0000 7f959f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:02:42.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.491+0000 7f959f7fe700 1 -- 172.21.15.37:0/911940329 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8084e60 con 0x5567792f3f60 2024-06-10T01:02:42.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.491+0000 7f959f7fe700 1 -- 172.21.15.37:0/911940329 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f95a8099bd0 con 0x5567792f3f60 2024-06-10T01:02:42.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.491+0000 7f95d0fbab80 1 -- 172.21.15.37:0/911940329 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95b80074e0 con 0x5567792f3f60 2024-06-10T01:02:42.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.491+0000 7f959f7fe700 1 -- 172.21.15.37:0/911940329 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f95a80a40b0 con 0x5567792f3f60 2024-06-10T01:02:42.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.491+0000 7f95d0fbab80 10 client.25963 fetch_fsmap learned FSMap version 27 2024-06-10T01:02:42.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.491+0000 7f95d0fbab80 1 -- 172.21.15.37:0/911940329 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55677937ae30 con 0x5567792f3f60 2024-06-10T01:02:42.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.491+0000 7f959f7fe700 1 -- 172.21.15.37:0/911940329 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95b80074e0 con 0x5567792f3f60 2024-06-10T01:02:42.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.491+0000 7f959f7fe700 1 -- 172.21.15.37:0/911940329 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f95a80b4ad0 con 0x5567792f3f60 2024-06-10T01:02:42.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.492+0000 7f95d0fbab80 10 client.25963 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:02:42.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.492+0000 7f95d0fbab80 10 client.25963 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-10T01:02:42.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.492+0000 7f95d0fbab80 20 client.25963 populate_metadata read hostname 'smithi037' 2024-06-10T01:02:42.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.492+0000 7f95d0fbab80 1 --2- 172.21.15.37:0/911940329 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55677937b530 0x55677937d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:42.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.492+0000 7f95d0fbab80 4 client.25963 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:02:42.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.492+0000 7f95d0fbab80 1 -- 172.21.15.37:0/911940329 --> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5567792472d0 con 0x55677937b530 2024-06-10T01:02:42.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.492+0000 7f95bffff700 1 --2- 172.21.15.37:0/911940329 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55677937b530 0x55677937d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:42.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.492+0000 7f95bffff700 1 --2- 172.21.15.37:0/911940329 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55677937b530 0x55677937d950 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7d18f5896ede1ecf server_cookie=dbd04ba820f08a2c in_seq=0 out_seq=0 2024-06-10T01:02:42.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.492+0000 7f959f7fe700 10 client.25963 ms_handle_connect on v2:172.21.15.167:6836/2720148065 2024-06-10T01:02:42.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.492+0000 7f959f7fe700 1 -- 172.21.15.37:0/911940329 <== mds.2 v2:172.21.15.167:6836/2720148065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5567792472d0 con 0x55677937b530 2024-06-10T01:02:42.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.492+0000 7f959f7fe700 10 client.25963 handle_command_reply: tid=0 2024-06-10T01:02:42.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.492+0000 7f95d0fbab80 1 client.25963 shutdown 2024-06-10T01:02:42.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.493+0000 7f95d0fbab80 1 -- 172.21.15.37:0/911940329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567792f3f60 msgr2=0x556779304830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:42.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.493+0000 7f95d0fbab80 1 --2- 172.21.15.37:0/911940329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567792f3f60 0x556779304830 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f95a80cf750 tx=0x7f95a809be10 comp rx=0 tx=0).stop 2024-06-10T01:02:42.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.493+0000 7f95d0fbab80 1 -- 172.21.15.37:0/911940329 shutdown_connections 2024-06-10T01:02:42.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.493+0000 7f95d0fbab80 1 --2- 172.21.15.37:0/911940329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567792f4880 0x556779379600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.493+0000 7f95d0fbab80 1 --2- 172.21.15.37:0/911940329 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55677937b530 0x55677937d950 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.493+0000 7f95d0fbab80 1 --2- 172.21.15.37:0/911940329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5567792ee7a0 0x5567793042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.493+0000 7f95d0fbab80 1 --2- 172.21.15.37:0/911940329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567792f3f60 0x556779304830 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.493+0000 7f959f7fe700 0 client.25963 ms_handle_reset on v2:172.21.15.167:6836/2720148065 2024-06-10T01:02:42.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.493+0000 7f95d0fbab80 1 -- 172.21.15.37:0/911940329 >> 172.21.15.37:0/911940329 conn(0x55677921b080 msgr2=0x5567792f3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:42.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.493+0000 7f95d0fbab80 1 -- 172.21.15.37:0/911940329 shutdown_connections 2024-06-10T01:02:42.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.493+0000 7f95d0fbab80 1 -- 172.21.15.37:0/911940329 wait complete. 2024-06-10T01:02:42.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.493+0000 7f95d0fbab80 20 client.25963 trim_cache size 0 max 16384 2024-06-10T01:02:42.496 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T01:02:42.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.494+0000 7f95c6737700 1 -- 172.21.15.37:0/2710127996 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f95ac0a2ca0 msgr2=0x7f95ac0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:42.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.494+0000 7f95c6737700 1 --2- 172.21.15.37:0/2710127996 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f95ac0a2ca0 0x7f95ac0a5160 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f95c0199b80 tx=0x7f95b00073d0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.494+0000 7f95c6737700 1 -- 172.21.15.37:0/2710127996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c0102380 msgr2=0x7f95c019edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:42.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.494+0000 7f95c6737700 1 --2- 172.21.15.37:0/2710127996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c0102380 0x7f95c019edc0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f95a800e940 tx=0x7f95a800ec50 comp rx=0 tx=0).stop 2024-06-10T01:02:42.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.494+0000 7f95c6737700 1 -- 172.21.15.37:0/2710127996 shutdown_connections 2024-06-10T01:02:42.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.494+0000 7f95c6737700 1 --2- 172.21.15.37:0/2710127996 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f95ac0a2ca0 0x7f95ac0a5160 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.494+0000 7f95c6737700 1 --2- 172.21.15.37:0/2710127996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95c0106720 0x7f95c0198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.494+0000 7f95c6737700 1 --2- 172.21.15.37:0/2710127996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c01019d0 0x7f95c019e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.494+0000 7f95c6737700 1 --2- 172.21.15.37:0/2710127996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c0102380 0x7f95c019edc0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:42.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.494+0000 7f95c6737700 1 -- 172.21.15.37:0/2710127996 >> 172.21.15.37:0/2710127996 conn(0x7f95c0078340 msgr2=0x7f95c0105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:42.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.494+0000 7f95c6737700 1 -- 172.21.15.37:0/2710127996 shutdown_connections 2024-06-10T01:02:42.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:42.494+0000 7f95c6737700 1 -- 172.21.15.37:0/2710127996 wait complete. 2024-06-10T01:02:42.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:42 smithi167 ceph-mon[36917]: pgmap v610: 129 pgs: 129 active+clean; 13 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 1.2 MiB/s wr, 1.17k op/s 2024-06-10T01:02:42.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:42 smithi094 ceph-mon[36708]: pgmap v610: 129 pgs: 129 active+clean; 13 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 1.2 MiB/s wr, 1.17k op/s 2024-06-10T01:02:43.339 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:02:43.634 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:02:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.737+0000 7f4d6e13b640 1 -- 172.21.15.37:0/1729361963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d68101af0 msgr2=0x7f4d68101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.737+0000 7f4d6e13b640 1 --2- 172.21.15.37:0/1729361963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d68101af0 0x7f4d68101f70 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009900 tx=0x7f4d5c02a410 comp rx=0 tx=0).stop 2024-06-10T01:02:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.737+0000 7f4d6e13b640 1 -- 172.21.15.37:0/1729361963 shutdown_connections 2024-06-10T01:02:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.737+0000 7f4d6e13b640 1 --2- 172.21.15.37:0/1729361963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d681024b0 0x7f4d68108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.737+0000 7f4d6e13b640 1 --2- 172.21.15.37:0/1729361963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d68101af0 0x7f4d68101f70 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.737+0000 7f4d6e13b640 1 --2- 172.21.15.37:0/1729361963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d68100900 0x7f4d68100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.737+0000 7f4d6e13b640 1 -- 172.21.15.37:0/1729361963 >> 172.21.15.37:0/1729361963 conn(0x7f4d680fbe70 msgr2=0x7f4d680fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.737+0000 7f4d6e13b640 1 -- 172.21.15.37:0/1729361963 shutdown_connections 2024-06-10T01:02:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.737+0000 7f4d6e13b640 1 -- 172.21.15.37:0/1729361963 wait complete. 2024-06-10T01:02:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.738+0000 7f4d6e13b640 1 Processor -- start 2024-06-10T01:02:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.738+0000 7f4d6e13b640 1 -- start start 2024-06-10T01:02:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.738+0000 7f4d6e13b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d68100900 0x7f4d68196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.738+0000 7f4d6e13b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d68101af0 0x7f4d68197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.739+0000 7f4d6e13b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d681024b0 0x7f4d6819e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.739+0000 7f4d6e13b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6810c5b0 con 0x7f4d68101af0 2024-06-10T01:02:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.739+0000 7f4d677fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d68100900 0x7f4d68196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.739+0000 7f4d67fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d681024b0 0x7f4d6819e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.739+0000 7f4d66ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d68101af0 0x7f4d68197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.739+0000 7f4d67fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d681024b0 0x7f4d6819e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:59268/0 (socket says 172.21.15.37:59268) 2024-06-10T01:02:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.739+0000 7f4d67fff640 1 -- 172.21.15.37:0/1955005330 learned_addr learned my addr 172.21.15.37:0/1955005330 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.739+0000 7f4d6e13b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6810c430 con 0x7f4d68100900 2024-06-10T01:02:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.739+0000 7f4d6e13b640 1 -- 172.21.15.37:0/1955005330 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6810c730 con 0x7f4d681024b0 2024-06-10T01:02:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.739+0000 7f4d66ffd640 1 -- 172.21.15.37:0/1955005330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d681024b0 msgr2=0x7f4d6819e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.739+0000 7f4d66ffd640 1 --2- 172.21.15.37:0/1955005330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d681024b0 0x7f4d6819e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.739+0000 7f4d66ffd640 1 -- 172.21.15.37:0/1955005330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d68100900 msgr2=0x7f4d68196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.739+0000 7f4d66ffd640 1 --2- 172.21.15.37:0/1955005330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d68100900 0x7f4d68196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.739+0000 7f4d66ffd640 1 -- 172.21.15.37:0/1955005330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6819e7e0 con 0x7f4d68101af0 2024-06-10T01:02:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.739+0000 7f4d67fff640 1 --2- 172.21.15.37:0/1955005330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d681024b0 0x7f4d6819e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.740+0000 7f4d66ffd640 1 --2- 172.21.15.37:0/1955005330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d68101af0 0x7f4d68197100 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c02a930 tx=0x7f4d5c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.740+0000 7f4d64ff9640 1 -- 172.21.15.37:0/1955005330 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c035030 con 0x7f4d68101af0 2024-06-10T01:02:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.740+0000 7f4d64ff9640 1 -- 172.21.15.37:0/1955005330 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d5c0403e0 con 0x7f4d68101af0 2024-06-10T01:02:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.740+0000 7f4d64ff9640 1 -- 172.21.15.37:0/1955005330 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c005d50 con 0x7f4d68101af0 2024-06-10T01:02:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.740+0000 7f4d6e13b640 1 -- 172.21.15.37:0/1955005330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d68069050 con 0x7f4d68101af0 2024-06-10T01:02:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.740+0000 7f4d6e13b640 1 -- 172.21.15.37:0/1955005330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d68069400 con 0x7f4d68101af0 2024-06-10T01:02:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.743+0000 7f4d64ff9640 1 -- 172.21.15.37:0/1955005330 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4d5c040590 con 0x7f4d68101af0 2024-06-10T01:02:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.744+0000 7f4d64ff9640 1 --2- 172.21.15.37:0/1955005330 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4d500a2c30 0x7f4d500a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.744+0000 7f4d70e49740 1 Processor -- start 2024-06-10T01:02:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.744+0000 7f4d64ff9640 1 -- 172.21.15.37:0/1955005330 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f4d5c0efc90 con 0x7f4d68101af0 2024-06-10T01:02:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.744+0000 7f4d70e49740 1 -- start start 2024-06-10T01:02:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.745+0000 7f4d677fe640 1 --2- 172.21.15.37:0/1955005330 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4d500a2c30 0x7f4d500a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.745+0000 7f4d70e49740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55999bd0d820 0x55999bd0dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.745+0000 7f4d70e49740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bd0e210 0x55999bd12d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.745+0000 7f4d70e49740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55999bd13380 0x55999bd15810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.745+0000 7f4d70e49740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55999bc684f0 con 0x55999bd0e210 2024-06-10T01:02:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.745+0000 7f4d70e49740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55999bc06870 con 0x55999bd13380 2024-06-10T01:02:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.745+0000 7f4d70e49740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55999bc644b0 con 0x55999bd0d820 2024-06-10T01:02:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.745+0000 7f4d66ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bd0e210 0x55999bd12d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.745+0000 7f4d66ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bd0e210 0x55999bd12d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55462/0 (socket says 172.21.15.37:55462) 2024-06-10T01:02:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.745+0000 7f4d67fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55999bd13380 0x55999bd15810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.745+0000 7f4d66ffd640 1 -- 172.21.15.37:0/2046263322 learned_addr learned my addr 172.21.15.37:0/2046263322 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.745+0000 7f4d677fe640 1 --2- 172.21.15.37:0/1955005330 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4d500a2c30 0x7f4d500a50f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4d68101950 tx=0x7f4d58012040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.745+0000 7f4d67fff640 1 -- 172.21.15.37:0/2046263322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55999bd0d820 msgr2=0x55999bd0dc40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:02:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.745+0000 7f4d67fff640 1 --2- 172.21.15.37:0/2046263322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55999bd0d820 0x55999bd0dc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.745+0000 7f4d67fff640 1 -- 172.21.15.37:0/2046263322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bd0e210 msgr2=0x55999bd12d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.746+0000 7f4d67fff640 1 --2- 172.21.15.37:0/2046263322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bd0e210 0x55999bd12d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.746+0000 7f4d67fff640 1 -- 172.21.15.37:0/2046263322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55999bc229b0 con 0x55999bd13380 2024-06-10T01:02:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.746+0000 7f4d66ffd640 1 --2- 172.21.15.37:0/2046263322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bd0e210 0x55999bd12d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.746+0000 7f4d67fff640 1 --2- 172.21.15.37:0/2046263322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55999bd13380 0x55999bd15810 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000e800 tx=0x7f4d6000ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74d86bf005ccc379 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.746+0000 7f4d4dffb640 1 -- 172.21.15.37:0/2046263322 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d60011c20 con 0x55999bd13380 2024-06-10T01:02:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.746+0000 7f4d4dffb640 1 -- 172.21.15.37:0/2046263322 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d60011dc0 con 0x55999bd13380 2024-06-10T01:02:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.746+0000 7f4d4dffb640 1 -- 172.21.15.37:0/2046263322 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6000f720 con 0x55999bd13380 2024-06-10T01:02:43.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.747+0000 7f4d70e49740 1 -- 172.21.15.37:0/2046263322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55999bd13380 msgr2=0x55999bd15810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.747+0000 7f4d70e49740 1 --2- 172.21.15.37:0/2046263322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55999bd13380 0x55999bd15810 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000e800 tx=0x7f4d6000ecc0 comp rx=0 tx=0).stop 2024-06-10T01:02:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.747+0000 7f4d70e49740 1 -- 172.21.15.37:0/2046263322 shutdown_connections 2024-06-10T01:02:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.747+0000 7f4d70e49740 1 --2- 172.21.15.37:0/2046263322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55999bd13380 0x55999bd15810 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.747+0000 7f4d70e49740 1 --2- 172.21.15.37:0/2046263322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bd0e210 0x55999bd12d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.747+0000 7f4d70e49740 1 --2- 172.21.15.37:0/2046263322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55999bd0d820 0x55999bd0dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.747+0000 7f4d70e49740 1 -- 172.21.15.37:0/2046263322 >> 172.21.15.37:0/2046263322 conn(0x55999bd00680 msgr2=0x55999bd00a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.747+0000 7f4d70e49740 1 -- 172.21.15.37:0/2046263322 shutdown_connections 2024-06-10T01:02:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.747+0000 7f4d70e49740 1 -- 172.21.15.37:0/2046263322 wait complete. 2024-06-10T01:02:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.748+0000 7f4d70e49740 1 -- start start 2024-06-10T01:02:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.748+0000 7f4d70e49740 1 Processor -- start 2024-06-10T01:02:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.748+0000 7f4d70e49740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55999bd0d820 0x55999bd17a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.748+0000 7f4d70e49740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bd0e210 0x55999bd17fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.748+0000 7f4d70e49740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55999bd13380 0x55999bd1f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.748+0000 7f4d70e49740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55999bd16330 con 0x55999bd0e210 2024-06-10T01:02:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.748+0000 7f4d70e49740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55999bbf1ae0 con 0x55999bd0d820 2024-06-10T01:02:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.748+0000 7f4d70e49740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55999bd164b0 con 0x55999bd13380 2024-06-10T01:02:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.748+0000 7f4d70e49740 10 client.0 _ll_register_callbacks cb 0x55999bc2a9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:02:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.748+0000 7f4d66ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bd0e210 0x55999bd17fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.748+0000 7f4d66ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bd0e210 0x55999bd17fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55478/0 (socket says 172.21.15.37:55478) 2024-06-10T01:02:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.748+0000 7f4d66ffd640 1 -- 172.21.15.37:0/2447656050 learned_addr learned my addr 172.21.15.37:0/2447656050 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.748+0000 7f4d677fe640 1 --2- 172.21.15.37:0/2447656050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55999bd0d820 0x55999bd17a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.749+0000 7f4d67fff640 1 --2- 172.21.15.37:0/2447656050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55999bd13380 0x55999bd1f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.749+0000 7f4d66ffd640 1 -- 172.21.15.37:0/2447656050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55999bd13380 msgr2=0x55999bd1f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.749+0000 7f4d66ffd640 1 --2- 172.21.15.37:0/2447656050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55999bd13380 0x55999bd1f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.749+0000 7f4d66ffd640 1 -- 172.21.15.37:0/2447656050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55999bd0d820 msgr2=0x55999bd17a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.749+0000 7f4d66ffd640 1 --2- 172.21.15.37:0/2447656050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55999bd0d820 0x55999bd17a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.749+0000 7f4d66ffd640 1 -- 172.21.15.37:0/2447656050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55999b97c210 con 0x55999bd0e210 2024-06-10T01:02:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.749+0000 7f4d677fe640 1 --2- 172.21.15.37:0/2447656050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55999bd0d820 0x55999bd17a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.749+0000 7f4d66ffd640 1 --2- 172.21.15.37:0/2447656050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bd0e210 0x55999bd17fd0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0314e0 tx=0x7f4d5c0bbca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f07cfb1778b45e2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.749+0000 7f4d46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:02:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.749+0000 7f4d46ffd640 1 -- 172.21.15.37:0/2447656050 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c0bc0a0 con 0x55999bd0e210 2024-06-10T01:02:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.749+0000 7f4d46ffd640 1 -- 172.21.15.37:0/2447656050 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f4d5c0c00d0 con 0x55999bd0e210 2024-06-10T01:02:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.749+0000 7f4d70e49740 1 -- 172.21.15.37:0/2447656050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d60011dc0 con 0x55999bd0e210 2024-06-10T01:02:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.749+0000 7f4d46ffd640 1 -- 172.21.15.37:0/2447656050 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4d5c0c0450 con 0x55999bd0e210 2024-06-10T01:02:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.749+0000 7f4d70e49740 10 client.16419 fetch_fsmap learned FSMap version 27 2024-06-10T01:02:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.749+0000 7f4d70e49740 1 -- 172.21.15.37:0/2447656050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55999ba48620 con 0x55999bd0e210 2024-06-10T01:02:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.750+0000 7f4d46ffd640 1 -- 172.21.15.37:0/2447656050 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d60011dc0 con 0x55999bd0e210 2024-06-10T01:02:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.750+0000 7f4d46ffd640 1 -- 172.21.15.37:0/2447656050 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4d5c0dace0 con 0x55999bd0e210 2024-06-10T01:02:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.750+0000 7f4d70e49740 10 client.16419 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:02:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.750+0000 7f4d70e49740 10 client.16419 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:02:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.750+0000 7f4d70e49740 20 client.16419 populate_metadata read hostname 'smithi037' 2024-06-10T01:02:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.750+0000 7f4d70e49740 1 --2- 172.21.15.37:0/2447656050 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55999bd23990 0x55999bd25e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.750+0000 7f4d70e49740 4 client.16419 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:02:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.750+0000 7f4d70e49740 1 -- 172.21.15.37:0/2447656050 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55999bc7bf10 con 0x55999bd23990 2024-06-10T01:02:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.750+0000 7f4d677fe640 1 --2- 172.21.15.37:0/2447656050 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55999bd23990 0x55999bd25e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.751+0000 7f4d46ffd640 10 client.16419 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.751+0000 7f4d677fe640 1 --2- 172.21.15.37:0/2447656050 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55999bd23990 0x55999bd25e30 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcff2a30b45d1ad0 server_cookie=c153c5524033567f in_seq=0 out_seq=0 2024-06-10T01:02:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.754+0000 7f4d46ffd640 1 -- 172.21.15.37:0/2447656050 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55999bc7bf10 con 0x55999bd23990 2024-06-10T01:02:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.754+0000 7f4d46ffd640 10 client.16419 handle_command_reply: tid=0 2024-06-10T01:02:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.755+0000 7f4d70e49740 1 client.16419 shutdown 2024-06-10T01:02:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.755+0000 7f4d70e49740 1 -- 172.21.15.37:0/2447656050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bd0e210 msgr2=0x55999bd17fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.755+0000 7f4d70e49740 1 --2- 172.21.15.37:0/2447656050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bd0e210 0x55999bd17fd0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0314e0 tx=0x7f4d5c0bbca0 comp rx=0 tx=0).stop 2024-06-10T01:02:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.755+0000 7f4d70e49740 1 -- 172.21.15.37:0/2447656050 shutdown_connections 2024-06-10T01:02:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.755+0000 7f4d70e49740 1 --2- 172.21.15.37:0/2447656050 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55999bd23990 0x55999bd25e30 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.755+0000 7f4d70e49740 1 --2- 172.21.15.37:0/2447656050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55999bd13380 0x55999bd1f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.755+0000 7f4d70e49740 1 --2- 172.21.15.37:0/2447656050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bd0e210 0x55999bd17fd0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.755+0000 7f4d46ffd640 0 client.16419 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.755+0000 7f4d70e49740 1 --2- 172.21.15.37:0/2447656050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55999bd0d820 0x55999bd17a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.755+0000 7f4d70e49740 1 -- 172.21.15.37:0/2447656050 >> 172.21.15.37:0/2447656050 conn(0x55999bd00680 msgr2=0x55999bcff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.755+0000 7f4d70e49740 1 -- 172.21.15.37:0/2447656050 shutdown_connections 2024-06-10T01:02:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.755+0000 7f4d70e49740 1 -- 172.21.15.37:0/2447656050 wait complete. 2024-06-10T01:02:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.755+0000 7f4d70e49740 20 client.16419 trim_cache size 0 max 16384 2024-06-10T01:02:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.785+0000 7f4d70e49740 1 Processor -- start 2024-06-10T01:02:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.785+0000 7f4d70e49740 1 -- start start 2024-06-10T01:02:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.786+0000 7f4d70e49740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55999bd0d820 0x55999bd0f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.786+0000 7f4d70e49740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55999bd0e210 0x55999bd1efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.786+0000 7f4d70e49740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bd13380 0x55999bd0f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.786+0000 7f4d70e49740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4d681aadc0 con 0x55999bd13380 2024-06-10T01:02:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.786+0000 7f4d70e49740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4d681a8690 con 0x55999bd0e210 2024-06-10T01:02:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.786+0000 7f4d70e49740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4d681a64e0 con 0x55999bd0d820 2024-06-10T01:02:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.786+0000 7f4d67fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bd13380 0x55999bd0f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.786+0000 7f4d677fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55999bd0d820 0x55999bd0f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.786+0000 7f4d67fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bd13380 0x55999bd0f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55484/0 (socket says 172.21.15.37:55484) 2024-06-10T01:02:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.786+0000 7f4d67fff640 1 -- 172.21.15.37:0/2781808721 learned_addr learned my addr 172.21.15.37:0/2781808721 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.786+0000 7f4d66ffd640 1 --2- 172.21.15.37:0/2781808721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55999bd0e210 0x55999bd1efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.786+0000 7f4d66ffd640 1 -- 172.21.15.37:0/2781808721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55999bd0d820 msgr2=0x55999bd0f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.786+0000 7f4d66ffd640 1 --2- 172.21.15.37:0/2781808721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55999bd0d820 0x55999bd0f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.786+0000 7f4d66ffd640 1 -- 172.21.15.37:0/2781808721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bd13380 msgr2=0x55999bd0f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.786+0000 7f4d66ffd640 1 --2- 172.21.15.37:0/2781808721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bd13380 0x55999bd0f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.786+0000 7f4d66ffd640 1 -- 172.21.15.37:0/2781808721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55999bd264b0 con 0x55999bd0e210 2024-06-10T01:02:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.786+0000 7f4d677fe640 1 --2- 172.21.15.37:0/2781808721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55999bd0d820 0x55999bd0f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.787+0000 7f4d66ffd640 1 --2- 172.21.15.37:0/2781808721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55999bd0e210 0x55999bd1efd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0b8a70 tx=0x7f4d5c0ba5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29222e8961b27dbd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.787+0000 7f4d4d7fa640 1 -- 172.21.15.37:0/2781808721 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c0408c0 con 0x55999bd0e210 2024-06-10T01:02:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.787+0000 7f4d4d7fa640 1 -- 172.21.15.37:0/2781808721 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d5c0bc0a0 con 0x55999bd0e210 2024-06-10T01:02:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.787+0000 7f4d4d7fa640 1 -- 172.21.15.37:0/2781808721 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c0c4880 con 0x55999bd0e210 2024-06-10T01:02:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.788+0000 7f4d70e49740 1 -- 172.21.15.37:0/2781808721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55999bd0e210 msgr2=0x55999bd1efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.788+0000 7f4d70e49740 1 --2- 172.21.15.37:0/2781808721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55999bd0e210 0x55999bd1efd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0b8a70 tx=0x7f4d5c0ba5b0 comp rx=0 tx=0).stop 2024-06-10T01:02:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.788+0000 7f4d70e49740 1 -- 172.21.15.37:0/2781808721 shutdown_connections 2024-06-10T01:02:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.788+0000 7f4d70e49740 1 --2- 172.21.15.37:0/2781808721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bd13380 0x55999bd0f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.788+0000 7f4d70e49740 1 --2- 172.21.15.37:0/2781808721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55999bd0e210 0x55999bd1efd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.788+0000 7f4d70e49740 1 --2- 172.21.15.37:0/2781808721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55999bd0d820 0x55999bd0f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.788+0000 7f4d70e49740 1 -- 172.21.15.37:0/2781808721 >> 172.21.15.37:0/2781808721 conn(0x55999bd00680 msgr2=0x55999bd00a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.788+0000 7f4d70e49740 1 -- 172.21.15.37:0/2781808721 shutdown_connections 2024-06-10T01:02:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.788+0000 7f4d70e49740 1 -- 172.21.15.37:0/2781808721 wait complete. 2024-06-10T01:02:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.788+0000 7f4d70e49740 1 -- start start 2024-06-10T01:02:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.789+0000 7f4d70e49740 1 Processor -- start 2024-06-10T01:02:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.789+0000 7f4d70e49740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bd0d820 0x55999bd1c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.789+0000 7f4d70e49740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55999bd0e210 0x55999bd1c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.789+0000 7f4d70e49740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55999bd13380 0x55999bd1d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.789+0000 7f4d70e49740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4d681a6b50 con 0x55999bd0d820 2024-06-10T01:02:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.789+0000 7f4d70e49740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4d681a7fc0 con 0x55999bd0e210 2024-06-10T01:02:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.789+0000 7f4d70e49740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4d681a7220 con 0x55999bd13380 2024-06-10T01:02:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.789+0000 7f4d70e49740 10 client.0 _ll_register_callbacks cb 0x7f4d5c0bbe70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:02:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.789+0000 7f4d66ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55999bd0e210 0x55999bd1c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.789+0000 7f4d66ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55999bd0e210 0x55999bd1c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:36320/0 (socket says 172.21.15.37:36320) 2024-06-10T01:02:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.789+0000 7f4d66ffd640 1 -- 172.21.15.37:0/3946019435 learned_addr learned my addr 172.21.15.37:0/3946019435 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.789+0000 7f4d677fe640 1 --2- 172.21.15.37:0/3946019435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bd0d820 0x55999bd1c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.790+0000 7f4d67fff640 1 --2- 172.21.15.37:0/3946019435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55999bd13380 0x55999bd1d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.790+0000 7f4d66ffd640 1 -- 172.21.15.37:0/3946019435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55999bd13380 msgr2=0x55999bd1d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.790+0000 7f4d66ffd640 1 --2- 172.21.15.37:0/3946019435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55999bd13380 0x55999bd1d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.790+0000 7f4d66ffd640 1 -- 172.21.15.37:0/3946019435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bd0d820 msgr2=0x55999bd1c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.790+0000 7f4d66ffd640 1 --2- 172.21.15.37:0/3946019435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bd0d820 0x55999bd1c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.790+0000 7f4d66ffd640 1 -- 172.21.15.37:0/3946019435 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55999bd1db50 con 0x55999bd0e210 2024-06-10T01:02:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.790+0000 7f4d66ffd640 1 --2- 172.21.15.37:0/3946019435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55999bd0e210 0x55999bd1c550 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0314e0 tx=0x7f4d5c0b83d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8496fb46963c096d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.790+0000 7f4d46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:02:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.790+0000 7f4d46ffd640 1 -- 172.21.15.37:0/3946019435 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c0408c0 con 0x55999bd0e210 2024-06-10T01:02:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.790+0000 7f4d46ffd640 1 -- 172.21.15.37:0/3946019435 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f4d5c0b8710 con 0x55999bd0e210 2024-06-10T01:02:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.791+0000 7f4d70e49740 1 -- 172.21.15.37:0/3946019435 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d5c0bc0a0 con 0x55999bd0e210 2024-06-10T01:02:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.791+0000 7f4d46ffd640 1 -- 172.21.15.37:0/3946019435 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4d5c0c4d10 con 0x55999bd0e210 2024-06-10T01:02:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.791+0000 7f4d70e49740 10 client.25889 fetch_fsmap learned FSMap version 27 2024-06-10T01:02:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.791+0000 7f4d70e49740 1 -- 172.21.15.37:0/3946019435 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55999bd1db50 con 0x55999bd0e210 2024-06-10T01:02:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.791+0000 7f4d46ffd640 1 -- 172.21.15.37:0/3946019435 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d5c0c9070 con 0x55999bd0e210 2024-06-10T01:02:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.791+0000 7f4d46ffd640 1 -- 172.21.15.37:0/3946019435 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f4d5c0dace0 con 0x55999bd0e210 2024-06-10T01:02:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.791+0000 7f4d70e49740 10 client.25889 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:02:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.791+0000 7f4d70e49740 10 client.25889 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:02:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.791+0000 7f4d70e49740 20 client.25889 populate_metadata read hostname 'smithi037' 2024-06-10T01:02:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.791+0000 7f4d70e49740 1 --2- 172.21.15.37:0/3946019435 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55999bd8e710 0x55999bd90b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.791+0000 7f4d70e49740 4 client.25889 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:02:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.791+0000 7f4d70e49740 1 -- 172.21.15.37:0/3946019435 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55999bc75c20 con 0x55999bd8e710 2024-06-10T01:02:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.791+0000 7f4d677fe640 1 --2- 172.21.15.37:0/3946019435 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55999bd8e710 0x55999bd90b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.792+0000 7f4d46ffd640 10 client.25889 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:43.792+0000 7f4d677fe640 1 --2- 172.21.15.37:0/3946019435 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55999bd8e710 0x55999bd90b00 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8c6c3f6ffbec4b6 server_cookie=c3f015a67e7dcf1a in_seq=0 out_seq=0 2024-06-10T01:02:44.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:44 smithi167 ceph-mon[36917]: pgmap v611: 129 pgs: 129 active+clean; 21 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 904 op/s 2024-06-10T01:02:44.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:44 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:02:44.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:44 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:02:44.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:44 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:02:44.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:44 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:02:44.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:44 smithi094 ceph-mon[36708]: pgmap v611: 129 pgs: 129 active+clean; 21 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 904 op/s 2024-06-10T01:02:44.612 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:44 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:02:44.612 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:44 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:02:44.612 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:44 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:02:44.612 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:44 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:02:44.643 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:44 smithi037 ceph-mon[32192]: pgmap v611: 129 pgs: 129 active+clean; 21 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 904 op/s 2024-06-10T01:02:44.643 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:44 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:02:44.643 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:44 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:02:44.643 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:44 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:02:44.643 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:44 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:02:45.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.498+0000 7fcab516a700 1 -- 172.21.15.37:0/2475396202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab006a170 msgr2=0x7fcab006a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.498+0000 7fcab516a700 1 --2- 172.21.15.37:0/2475396202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab006a170 0x7fcab006a5f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fca98009990 tx=0x7fca98009ca0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.498+0000 7fcab516a700 1 -- 172.21.15.37:0/2475396202 shutdown_connections 2024-06-10T01:02:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.498+0000 7fcab516a700 1 --2- 172.21.15.37:0/2475396202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcab006ab30 0x7fcab010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.498+0000 7fcab516a700 1 --2- 172.21.15.37:0/2475396202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcab0108500 0x7fcab01088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.498+0000 7fcab516a700 1 --2- 172.21.15.37:0/2475396202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab006a170 0x7fcab006a5f0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.498+0000 7fcab516a700 1 -- 172.21.15.37:0/2475396202 >> 172.21.15.37:0/2475396202 conn(0x7fcab0078340 msgr2=0x7fcab0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.498+0000 7fcab516a700 1 -- 172.21.15.37:0/2475396202 shutdown_connections 2024-06-10T01:02:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.498+0000 7fcab516a700 1 -- 172.21.15.37:0/2475396202 wait complete. 2024-06-10T01:02:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.499+0000 7fcab516a700 1 Processor -- start 2024-06-10T01:02:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.499+0000 7fcab516a700 1 -- start start 2024-06-10T01:02:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.499+0000 7fcab516a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcab006a170 0x7fcab019aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.499+0000 7fcab516a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab006ab30 0x7fcab019b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.499+0000 7fcab516a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcab0108500 0x7fcab019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.499+0000 7fcab516a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcab0112780 con 0x7fcab006ab30 2024-06-10T01:02:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.499+0000 7fcab516a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcab0112600 con 0x7fcab006a170 2024-06-10T01:02:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.499+0000 7fcab516a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcab0112900 con 0x7fcab0108500 2024-06-10T01:02:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.500+0000 7fcaa659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab006ab30 0x7fcab019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.500+0000 7fcaaed9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcab006a170 0x7fcab019aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.500+0000 7fcaaed9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcab006a170 0x7fcab019aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:36344/0 (socket says 172.21.15.37:36344) 2024-06-10T01:02:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.500+0000 7fcaaed9d700 1 -- 172.21.15.37:0/416820240 learned_addr learned my addr 172.21.15.37:0/416820240 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.500+0000 7fcaaf59e700 1 --2- 172.21.15.37:0/416820240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcab0108500 0x7fcab019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.500+0000 7fcaaed9d700 1 -- 172.21.15.37:0/416820240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcab0108500 msgr2=0x7fcab019f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.500+0000 7fcaaed9d700 1 --2- 172.21.15.37:0/416820240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcab0108500 0x7fcab019f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.500+0000 7fcaaed9d700 1 -- 172.21.15.37:0/416820240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab006ab30 msgr2=0x7fcab019b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.500+0000 7fcaaed9d700 1 --2- 172.21.15.37:0/416820240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab006ab30 0x7fcab019b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.500+0000 7fcaaed9d700 1 -- 172.21.15.37:0/416820240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab019fe90 con 0x7fcab006a170 2024-06-10T01:02:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.500+0000 7fcaa659c700 1 --2- 172.21.15.37:0/416820240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab006ab30 0x7fcab019b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.500+0000 7fcaaf59e700 1 --2- 172.21.15.37:0/416820240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcab0108500 0x7fcab019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.500+0000 7fcaaed9d700 1 --2- 172.21.15.37:0/416820240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcab006a170 0x7fcab019aed0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000e9c0 tx=0x7fcaa000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.500+0000 7fcaacd99700 1 -- 172.21.15.37:0/416820240 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa000cbd0 con 0x7fcab006a170 2024-06-10T01:02:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.500+0000 7fcab516a700 1 -- 172.21.15.37:0/416820240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcab01a0180 con 0x7fcab006a170 2024-06-10T01:02:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.500+0000 7fcab516a700 1 -- 172.21.15.37:0/416820240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcab01a7a00 con 0x7fcab006a170 2024-06-10T01:02:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.500+0000 7fcaacd99700 1 -- 172.21.15.37:0/416820240 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaa000ee60 con 0x7fcab006a170 2024-06-10T01:02:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.500+0000 7fcaacd99700 1 -- 172.21.15.37:0/416820240 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0005610 con 0x7fcab006a170 2024-06-10T01:02:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.503+0000 7fcaacd99700 1 -- 172.21.15.37:0/416820240 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fcaa0021440 con 0x7fcab006a170 2024-06-10T01:02:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.504+0000 7fcaacd99700 1 --2- 172.21.15.37:0/416820240 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fca9c0a2c20 0x7fca9c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:45.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.504+0000 7fcabf9edb80 1 Processor -- start 2024-06-10T01:02:45.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.504+0000 7fcaa659c700 1 --2- 172.21.15.37:0/416820240 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fca9c0a2c20 0x7fca9c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:45.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.504+0000 7fcaacd99700 1 -- 172.21.15.37:0/416820240 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fcaa00cf670 con 0x7fcab006a170 2024-06-10T01:02:45.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.504+0000 7fcabf9edb80 1 -- start start 2024-06-10T01:02:45.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.504+0000 7fcabf9edb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ddbdbf60 0x5644ddbdc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:45.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.504+0000 7fcabf9edb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644ddbdc880 0x5644ddbd6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:45.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.504+0000 7fcabf9edb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ddbd67a0 0x5644ddbd6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:45.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.504+0000 7fcabf9edb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5644ddac8f60 con 0x5644ddbdc880 2024-06-10T01:02:45.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.504+0000 7fcabf9edb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5644ddb27850 con 0x5644ddbd67a0 2024-06-10T01:02:45.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.504+0000 7fcabf9edb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5644dda8f460 con 0x5644ddbdbf60 2024-06-10T01:02:45.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.504+0000 7fcaa659c700 1 --2- 172.21.15.37:0/416820240 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fca9c0a2c20 0x7fca9c0a50e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fca9800b2e0 tx=0x7fca98005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.504+0000 7fcaa659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644ddbdc880 0x5644ddbd6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.505+0000 7fcaa659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644ddbdc880 0x5644ddbd6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55520/0 (socket says 172.21.15.37:55520) 2024-06-10T01:02:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.505+0000 7fcaa659c700 1 -- 172.21.15.37:0/2217880275 learned_addr learned my addr 172.21.15.37:0/2217880275 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.505+0000 7fcaaed9d700 1 --2- 172.21.15.37:0/2217880275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ddbdbf60 0x5644ddbdc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.505+0000 7fcaaf59e700 1 --2- 172.21.15.37:0/2217880275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ddbd67a0 0x5644ddbd6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.505+0000 7fcaaed9d700 1 -- 172.21.15.37:0/2217880275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ddbd67a0 msgr2=0x5644ddbd6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.505+0000 7fcaaed9d700 1 --2- 172.21.15.37:0/2217880275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ddbd67a0 0x5644ddbd6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.505+0000 7fcaaed9d700 1 -- 172.21.15.37:0/2217880275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644ddbdc880 msgr2=0x5644ddbd6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.505+0000 7fcaaed9d700 1 --2- 172.21.15.37:0/2217880275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644ddbdc880 0x5644ddbd6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.505+0000 7fcaaed9d700 1 -- 172.21.15.37:0/2217880275 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644dda768d0 con 0x5644ddbdbf60 2024-06-10T01:02:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.505+0000 7fcaa659c700 1 --2- 172.21.15.37:0/2217880275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644ddbdc880 0x5644ddbd6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.505+0000 7fcaaed9d700 1 --2- 172.21.15.37:0/2217880275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ddbdbf60 0x5644ddbdc340 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00ccdd0 tx=0x7fcaa009b4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbbba619ca41679 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.505+0000 7fcaa559a700 1 -- 172.21.15.37:0/2217880275 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa009fe90 con 0x5644ddbdbf60 2024-06-10T01:02:45.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.505+0000 7fcaa559a700 1 -- 172.21.15.37:0/2217880275 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaa00b4070 con 0x5644ddbdbf60 2024-06-10T01:02:45.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.505+0000 7fcaa559a700 1 -- 172.21.15.37:0/2217880275 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa009bfa0 con 0x5644ddbdbf60 2024-06-10T01:02:45.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.506+0000 7fcabf9edb80 1 -- 172.21.15.37:0/2217880275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ddbdbf60 msgr2=0x5644ddbdc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:45.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.506+0000 7fcabf9edb80 1 --2- 172.21.15.37:0/2217880275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ddbdbf60 0x5644ddbdc340 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00ccdd0 tx=0x7fcaa009b4c0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.506+0000 7fcabf9edb80 1 -- 172.21.15.37:0/2217880275 shutdown_connections 2024-06-10T01:02:45.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.506+0000 7fcabf9edb80 1 --2- 172.21.15.37:0/2217880275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ddbd67a0 0x5644ddbd6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.506+0000 7fcabf9edb80 1 --2- 172.21.15.37:0/2217880275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ddbdbf60 0x5644ddbdc340 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.506+0000 7fcabf9edb80 1 --2- 172.21.15.37:0/2217880275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644ddbdc880 0x5644ddbd6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.506+0000 7fcabf9edb80 1 -- 172.21.15.37:0/2217880275 >> 172.21.15.37:0/2217880275 conn(0x5644ddb03080 msgr2=0x5644ddbd1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:45.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.506+0000 7fcabf9edb80 1 -- 172.21.15.37:0/2217880275 shutdown_connections 2024-06-10T01:02:45.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.506+0000 7fcabf9edb80 1 -- 172.21.15.37:0/2217880275 wait complete. 2024-06-10T01:02:45.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.506+0000 7fcabf9edb80 1 -- start start 2024-06-10T01:02:45.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.507+0000 7fcabf9edb80 1 Processor -- start 2024-06-10T01:02:45.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.507+0000 7fcabf9edb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ddbd67a0 0x5644ddbe2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:45.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.507+0000 7fcabf9edb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644ddbdbf60 0x5644ddbdf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:45.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.507+0000 7fcabf9edb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ddbdc880 0x5644ddbdfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:45.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.507+0000 7fcabf9edb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5644ddac7fc0 con 0x5644ddbdbf60 2024-06-10T01:02:45.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.507+0000 7fcabf9edb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5644ddb23810 con 0x5644ddbdc880 2024-06-10T01:02:45.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.507+0000 7fcabf9edb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5644ddbe3200 con 0x5644ddbd67a0 2024-06-10T01:02:45.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.507+0000 7fcabf9edb80 10 client.0 _ll_register_callbacks cb 0x5644ddabd5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:02:45.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.507+0000 7fcaa659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644ddbdbf60 0x5644ddbdf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:45.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.507+0000 7fcaa659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644ddbdbf60 0x5644ddbdf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55536/0 (socket says 172.21.15.37:55536) 2024-06-10T01:02:45.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.507+0000 7fcaa659c700 1 -- 172.21.15.37:0/3108061813 learned_addr learned my addr 172.21.15.37:0/3108061813 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:45.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.507+0000 7fcaaed9d700 1 --2- 172.21.15.37:0/3108061813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ddbd67a0 0x5644ddbe2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:45.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.507+0000 7fcaaf59e700 1 --2- 172.21.15.37:0/3108061813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ddbdc880 0x5644ddbdfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:45.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.507+0000 7fcaa659c700 1 -- 172.21.15.37:0/3108061813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ddbd67a0 msgr2=0x5644ddbe2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:45.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.507+0000 7fcaa659c700 1 --2- 172.21.15.37:0/3108061813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ddbd67a0 0x5644ddbe2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.507+0000 7fcaa659c700 1 -- 172.21.15.37:0/3108061813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ddbdc880 msgr2=0x5644ddbdfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:45.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.507+0000 7fcaa659c700 1 --2- 172.21.15.37:0/3108061813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ddbdc880 0x5644ddbdfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.507+0000 7fcaa659c700 1 -- 172.21.15.37:0/3108061813 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644ddaed750 con 0x5644ddbdbf60 2024-06-10T01:02:45.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.507+0000 7fcaaed9d700 1 --2- 172.21.15.37:0/3108061813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ddbd67a0 0x5644ddbe2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:45.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.508+0000 7fcaa659c700 1 --2- 172.21.15.37:0/3108061813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644ddbdbf60 0x5644ddbdf930 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fca980098e0 tx=0x7fca9801a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5eb323e70b85d3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:45.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.508+0000 7fca967fc700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:02:45.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.508+0000 7fca967fc700 1 -- 172.21.15.37:0/3108061813 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98034060 con 0x5644ddbdbf60 2024-06-10T01:02:45.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.508+0000 7fca967fc700 1 -- 172.21.15.37:0/3108061813 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fca9800f870 con 0x5644ddbdbf60 2024-06-10T01:02:45.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.508+0000 7fcabf9edb80 1 -- 172.21.15.37:0/3108061813 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaa00b4070 con 0x5644ddbdbf60 2024-06-10T01:02:45.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.508+0000 7fca967fc700 1 -- 172.21.15.37:0/3108061813 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fca9800fa70 con 0x5644ddbdbf60 2024-06-10T01:02:45.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.508+0000 7fcabf9edb80 10 client.16437 fetch_fsmap learned FSMap version 27 2024-06-10T01:02:45.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.508+0000 7fcabf9edb80 1 -- 172.21.15.37:0/3108061813 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5644ddbed9c0 con 0x5644ddbdbf60 2024-06-10T01:02:45.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.508+0000 7fca967fc700 1 -- 172.21.15.37:0/3108061813 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaa00b4070 con 0x5644ddbdbf60 2024-06-10T01:02:45.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.508+0000 7fca967fc700 1 -- 172.21.15.37:0/3108061813 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fca98004860 con 0x5644ddbdbf60 2024-06-10T01:02:45.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.508+0000 7fcabf9edb80 10 client.16437 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:02:45.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.508+0000 7fcabf9edb80 10 client.16437 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-06-10T01:02:45.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.508+0000 7fcabf9edb80 20 client.16437 populate_metadata read hostname 'smithi037' 2024-06-10T01:02:45.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.508+0000 7fcabf9edb80 1 --2- 172.21.15.37:0/3108061813 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x5644ddbf0000 0x5644ddbf23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:45.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.508+0000 7fcabf9edb80 4 client.16437 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:02:45.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.508+0000 7fcabf9edb80 1 -- 172.21.15.37:0/3108061813 --> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644ddb34d50 con 0x5644ddbf0000 2024-06-10T01:02:45.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.509+0000 7fcaaed9d700 1 --2- 172.21.15.37:0/3108061813 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x5644ddbf0000 0x5644ddbf23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:45.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.509+0000 7fca967fc700 10 client.16437 ms_handle_connect on v2:172.21.15.94:6838/946269331 2024-06-10T01:02:45.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.509+0000 7fcaaed9d700 1 --2- 172.21.15.37:0/3108061813 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x5644ddbf0000 0x5644ddbf23f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c11ef9002c625095 server_cookie=9000f2241c30c9a9 in_seq=0 out_seq=0 2024-06-10T01:02:45.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.510+0000 7fca967fc700 1 -- 172.21.15.37:0/3108061813 <== mds.1 v2:172.21.15.94:6838/946269331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5644ddb34d50 con 0x5644ddbf0000 2024-06-10T01:02:45.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.510+0000 7fca967fc700 10 client.16437 handle_command_reply: tid=0 2024-06-10T01:02:45.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.510+0000 7fcabf9edb80 1 client.16437 shutdown 2024-06-10T01:02:45.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.510+0000 7fcabf9edb80 1 -- 172.21.15.37:0/3108061813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644ddbdbf60 msgr2=0x5644ddbdf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:45.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.510+0000 7fcabf9edb80 1 --2- 172.21.15.37:0/3108061813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644ddbdbf60 0x5644ddbdf930 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fca980098e0 tx=0x7fca9801a050 comp rx=0 tx=0).stop 2024-06-10T01:02:45.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.510+0000 7fcabf9edb80 1 -- 172.21.15.37:0/3108061813 shutdown_connections 2024-06-10T01:02:45.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.510+0000 7fcabf9edb80 1 --2- 172.21.15.37:0/3108061813 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x5644ddbf0000 0x5644ddbf23f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.510+0000 7fcabf9edb80 1 --2- 172.21.15.37:0/3108061813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ddbdc880 0x5644ddbdfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.510+0000 7fcabf9edb80 1 --2- 172.21.15.37:0/3108061813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ddbd67a0 0x5644ddbe2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.510+0000 7fcabf9edb80 1 --2- 172.21.15.37:0/3108061813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644ddbdbf60 0x5644ddbdf930 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.510+0000 7fcabf9edb80 1 -- 172.21.15.37:0/3108061813 >> 172.21.15.37:0/3108061813 conn(0x5644ddb03080 msgr2=0x5644ddbd0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:45.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.510+0000 7fca967fc700 0 client.16437 ms_handle_reset on v2:172.21.15.94:6838/946269331 2024-06-10T01:02:45.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.510+0000 7fcabf9edb80 1 -- 172.21.15.37:0/3108061813 shutdown_connections 2024-06-10T01:02:45.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.510+0000 7fcabf9edb80 1 -- 172.21.15.37:0/3108061813 wait complete. 2024-06-10T01:02:45.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.510+0000 7fcabf9edb80 20 client.16437 trim_cache size 0 max 16384 2024-06-10T01:02:45.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.526+0000 7fcabf9edb80 1 Processor -- start 2024-06-10T01:02:45.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.526+0000 7fcabf9edb80 1 -- start start 2024-06-10T01:02:45.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.526+0000 7fcabf9edb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644ddb03080 0x5644ddbd6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:45.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.526+0000 7fcabf9edb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ddbd67a0 0x5644ddbdd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:45.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.526+0000 7fcabf9edb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ddbdbf60 0x5644ddbe2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:45.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.526+0000 7fcabf9edb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcab01b2fe0 con 0x5644ddb03080 2024-06-10T01:02:45.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.526+0000 7fcabf9edb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcab01b0870 con 0x5644ddbdbf60 2024-06-10T01:02:45.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.526+0000 7fcabf9edb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcab01ae6c0 con 0x5644ddbd67a0 2024-06-10T01:02:45.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.526+0000 7fcaaed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644ddb03080 0x5644ddbd6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:45.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.526+0000 7fcaaed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644ddb03080 0x5644ddbd6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55544/0 (socket says 172.21.15.37:55544) 2024-06-10T01:02:45.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.526+0000 7fcaaed9d700 1 -- 172.21.15.37:0/2296509816 learned_addr learned my addr 172.21.15.37:0/2296509816 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:45.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.526+0000 7fcaa659c700 1 --2- 172.21.15.37:0/2296509816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ddbd67a0 0x5644ddbdd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:45.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.526+0000 7fcaaf59e700 1 --2- 172.21.15.37:0/2296509816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ddbdbf60 0x5644ddbe2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:45.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.526+0000 7fcaaed9d700 1 -- 172.21.15.37:0/2296509816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ddbd67a0 msgr2=0x5644ddbdd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:45.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.526+0000 7fcaaed9d700 1 --2- 172.21.15.37:0/2296509816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ddbd67a0 0x5644ddbdd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.526+0000 7fcaaed9d700 1 -- 172.21.15.37:0/2296509816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ddbdbf60 msgr2=0x5644ddbe2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:45.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.526+0000 7fcaaed9d700 1 --2- 172.21.15.37:0/2296509816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ddbdbf60 0x5644ddbe2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.526+0000 7fcaaed9d700 1 -- 172.21.15.37:0/2296509816 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644ddbcd810 con 0x5644ddb03080 2024-06-10T01:02:45.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.526+0000 7fcaaf59e700 1 --2- 172.21.15.37:0/2296509816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ddbdbf60 0x5644ddbe2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:02:45.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.527+0000 7fcaaed9d700 1 --2- 172.21.15.37:0/2296509816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644ddb03080 0x5644ddbd6260 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00b6010 tx=0x7fcaa000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca7ebe71643c198 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:45.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.527+0000 7fcaa4d99700 1 -- 172.21.15.37:0/2296509816 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0099de0 con 0x5644ddb03080 2024-06-10T01:02:45.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.527+0000 7fcaa4d99700 1 -- 172.21.15.37:0/2296509816 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaa00b4040 con 0x5644ddb03080 2024-06-10T01:02:45.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.527+0000 7fcaa4d99700 1 -- 172.21.15.37:0/2296509816 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa00a4520 con 0x5644ddb03080 2024-06-10T01:02:45.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.527+0000 7fcabf9edb80 1 -- 172.21.15.37:0/2296509816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644ddb03080 msgr2=0x5644ddbd6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:45.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.527+0000 7fcabf9edb80 1 --2- 172.21.15.37:0/2296509816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644ddb03080 0x5644ddbd6260 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00b6010 tx=0x7fcaa000cf90 comp rx=0 tx=0).stop 2024-06-10T01:02:45.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.528+0000 7fcabf9edb80 1 -- 172.21.15.37:0/2296509816 shutdown_connections 2024-06-10T01:02:45.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.528+0000 7fcabf9edb80 1 --2- 172.21.15.37:0/2296509816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ddbdbf60 0x5644ddbe2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.528+0000 7fcabf9edb80 1 --2- 172.21.15.37:0/2296509816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644ddb03080 0x5644ddbd6260 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.528+0000 7fcabf9edb80 1 --2- 172.21.15.37:0/2296509816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ddbd67a0 0x5644ddbdd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.528+0000 7fcabf9edb80 1 -- 172.21.15.37:0/2296509816 >> 172.21.15.37:0/2296509816 conn(0x5644ddbda1e0 msgr2=0x5644ddbdcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:45.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.528+0000 7fcabf9edb80 1 -- 172.21.15.37:0/2296509816 shutdown_connections 2024-06-10T01:02:45.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.528+0000 7fcabf9edb80 1 -- 172.21.15.37:0/2296509816 wait complete. 2024-06-10T01:02:45.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.528+0000 7fcabf9edb80 1 -- start start 2024-06-10T01:02:45.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.528+0000 7fcabf9edb80 1 Processor -- start 2024-06-10T01:02:45.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.528+0000 7fcabf9edb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ddb03080 0x5644ddbe84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:45.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.528+0000 7fcabf9edb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644ddbd67a0 0x5644ddbe8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:45.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.528+0000 7fcabf9edb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ddbdbf60 0x5644ddbf6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:45.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.528+0000 7fcabf9edb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcab01aed30 con 0x5644ddbd67a0 2024-06-10T01:02:45.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.528+0000 7fcabf9edb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcab01b01a0 con 0x5644ddbdbf60 2024-06-10T01:02:45.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.528+0000 7fcabf9edb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcab01af400 con 0x5644ddb03080 2024-06-10T01:02:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.528+0000 7fcabf9edb80 10 client.0 _ll_register_callbacks cb 0x5644ddabd5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:02:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.528+0000 7fcaa659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644ddbd67a0 0x5644ddbe8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.528+0000 7fcaa659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644ddbd67a0 0x5644ddbe8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55546/0 (socket says 172.21.15.37:55546) 2024-06-10T01:02:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.528+0000 7fcaaed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ddb03080 0x5644ddbe84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.528+0000 7fcaa659c700 1 -- 172.21.15.37:0/1904137796 learned_addr learned my addr 172.21.15.37:0/1904137796 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.529+0000 7fcaaf59e700 1 --2- 172.21.15.37:0/1904137796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ddbdbf60 0x5644ddbf6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.529+0000 7fcaa659c700 1 -- 172.21.15.37:0/1904137796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ddb03080 msgr2=0x5644ddbe84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.529+0000 7fcaa659c700 1 --2- 172.21.15.37:0/1904137796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ddb03080 0x5644ddbe84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.529+0000 7fcaa659c700 1 -- 172.21.15.37:0/1904137796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ddbdbf60 msgr2=0x5644ddbf6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.529+0000 7fcaa659c700 1 --2- 172.21.15.37:0/1904137796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ddbdbf60 0x5644ddbf6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.529+0000 7fcaa659c700 1 -- 172.21.15.37:0/1904137796 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644ddbe90d0 con 0x5644ddbd67a0 2024-06-10T01:02:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.529+0000 7fcaaed9d700 1 --2- 172.21.15.37:0/1904137796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ddb03080 0x5644ddbe84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.529+0000 7fcaaf59e700 1 --2- 172.21.15.37:0/1904137796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ddbdbf60 0x5644ddbf6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:02:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.529+0000 7fcaa659c700 1 --2- 172.21.15.37:0/1904137796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644ddbd67a0 0x5644ddbe8a00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fca98009750 tx=0x7fca9800fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=822000d4fdaf8d97 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.529+0000 7fca967fc700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:02:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.529+0000 7fca967fc700 1 -- 172.21.15.37:0/1904137796 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98034040 con 0x5644ddbd67a0 2024-06-10T01:02:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.529+0000 7fca967fc700 1 -- 172.21.15.37:0/1904137796 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fca98005020 con 0x5644ddbd67a0 2024-06-10T01:02:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.529+0000 7fcabf9edb80 1 -- 172.21.15.37:0/1904137796 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaa00b4040 con 0x5644ddbd67a0 2024-06-10T01:02:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.529+0000 7fca967fc700 1 -- 172.21.15.37:0/1904137796 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fca9804e050 con 0x5644ddbd67a0 2024-06-10T01:02:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.529+0000 7fcabf9edb80 10 client.16443 fetch_fsmap learned FSMap version 27 2024-06-10T01:02:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.529+0000 7fcabf9edb80 1 -- 172.21.15.37:0/1904137796 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5644ddc5a1e0 con 0x5644ddbd67a0 2024-06-10T01:02:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.530+0000 7fca967fc700 1 -- 172.21.15.37:0/1904137796 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaa00b4040 con 0x5644ddbd67a0 2024-06-10T01:02:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.530+0000 7fca967fc700 1 -- 172.21.15.37:0/1904137796 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fca980407a0 con 0x5644ddbd67a0 2024-06-10T01:02:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.530+0000 7fcabf9edb80 10 client.16443 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:02:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.530+0000 7fcabf9edb80 10 client.16443 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-06-10T01:02:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.530+0000 7fcabf9edb80 20 client.16443 populate_metadata read hostname 'smithi037' 2024-06-10T01:02:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.530+0000 7fcabf9edb80 1 --2- 172.21.15.37:0/1904137796 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x5644ddbdc880 0x5644ddc5c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.530+0000 7fcabf9edb80 4 client.16443 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:02:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.530+0000 7fcabf9edb80 1 -- 172.21.15.37:0/1904137796 --> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5644ddb2f2d0 con 0x5644ddbdc880 2024-06-10T01:02:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.530+0000 7fcaaed9d700 1 --2- 172.21.15.37:0/1904137796 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x5644ddbdc880 0x5644ddc5c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.530+0000 7fcaaed9d700 1 --2- 172.21.15.37:0/1904137796 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x5644ddbdc880 0x5644ddc5c920 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=52b165782e35181b server_cookie=71dd2401935ae9f9 in_seq=0 out_seq=0 2024-06-10T01:02:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.530+0000 7fca967fc700 10 client.16443 ms_handle_connect on v2:172.21.15.94:6838/946269331 2024-06-10T01:02:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.531+0000 7fca967fc700 1 -- 172.21.15.37:0/1904137796 <== mds.1 v2:172.21.15.94:6838/946269331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5644ddb2f2d0 con 0x5644ddbdc880 2024-06-10T01:02:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.531+0000 7fca967fc700 10 client.16443 handle_command_reply: tid=0 2024-06-10T01:02:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.531+0000 7fcabf9edb80 1 client.16443 shutdown 2024-06-10T01:02:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.531+0000 7fcabf9edb80 1 -- 172.21.15.37:0/1904137796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644ddbd67a0 msgr2=0x5644ddbe8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.531+0000 7fcabf9edb80 1 --2- 172.21.15.37:0/1904137796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644ddbd67a0 0x5644ddbe8a00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fca98009750 tx=0x7fca9800fec0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.531+0000 7fcabf9edb80 1 -- 172.21.15.37:0/1904137796 shutdown_connections 2024-06-10T01:02:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.531+0000 7fcabf9edb80 1 --2- 172.21.15.37:0/1904137796 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x5644ddbdc880 0x5644ddc5c920 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.531+0000 7fcabf9edb80 1 --2- 172.21.15.37:0/1904137796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5644ddbdbf60 0x5644ddbf6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.531+0000 7fcabf9edb80 1 --2- 172.21.15.37:0/1904137796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5644ddb03080 0x5644ddbe84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.531+0000 7fcabf9edb80 1 --2- 172.21.15.37:0/1904137796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644ddbd67a0 0x5644ddbe8a00 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.531+0000 7fcabf9edb80 1 -- 172.21.15.37:0/1904137796 >> 172.21.15.37:0/1904137796 conn(0x5644ddbda1e0 msgr2=0x5644ddbdaa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:45.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.531+0000 7fca967fc700 0 client.16443 ms_handle_reset on v2:172.21.15.94:6838/946269331 2024-06-10T01:02:45.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.531+0000 7fcabf9edb80 1 -- 172.21.15.37:0/1904137796 shutdown_connections 2024-06-10T01:02:45.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.531+0000 7fcabf9edb80 1 -- 172.21.15.37:0/1904137796 wait complete. 2024-06-10T01:02:45.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.531+0000 7fcabf9edb80 20 client.16443 trim_cache size 0 max 16384 2024-06-10T01:02:45.534 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T01:02:45.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.531+0000 7fcab516a700 1 -- 172.21.15.37:0/416820240 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fca9c0a2c20 msgr2=0x7fca9c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:45.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.531+0000 7fcab516a700 1 --2- 172.21.15.37:0/416820240 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fca9c0a2c20 0x7fca9c0a50e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fca9800b2e0 tx=0x7fca98005960 comp rx=0 tx=0).stop 2024-06-10T01:02:45.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.531+0000 7fcab516a700 1 -- 172.21.15.37:0/416820240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcab006a170 msgr2=0x7fcab019aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:45.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.531+0000 7fcab516a700 1 --2- 172.21.15.37:0/416820240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcab006a170 0x7fcab019aed0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000e9c0 tx=0x7fcaa000ecd0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.531+0000 7fcab516a700 1 -- 172.21.15.37:0/416820240 shutdown_connections 2024-06-10T01:02:45.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.531+0000 7fcab516a700 1 --2- 172.21.15.37:0/416820240 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fca9c0a2c20 0x7fca9c0a50e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.532+0000 7fcab516a700 1 --2- 172.21.15.37:0/416820240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcab0108500 0x7fcab019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.532+0000 7fcab516a700 1 --2- 172.21.15.37:0/416820240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcab006a170 0x7fcab019aed0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.532+0000 7fcab516a700 1 --2- 172.21.15.37:0/416820240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab006ab30 0x7fcab019b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:45.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.532+0000 7fcab516a700 1 -- 172.21.15.37:0/416820240 >> 172.21.15.37:0/416820240 conn(0x7fcab0078340 msgr2=0x7fcab00ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:45.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.532+0000 7fcab516a700 1 -- 172.21.15.37:0/416820240 shutdown_connections 2024-06-10T01:02:45.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:45.532+0000 7fcab516a700 1 -- 172.21.15.37:0/416820240 wait complete. 2024-06-10T01:02:46.393 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:46 smithi037 ceph-mon[32192]: pgmap v612: 129 pgs: 129 active+clean; 21 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 489 op/s 2024-06-10T01:02:46.463 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:02:46.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:46 smithi167 ceph-mon[36917]: pgmap v612: 129 pgs: 129 active+clean; 21 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 489 op/s 2024-06-10T01:02:46.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:46 smithi094 ceph-mon[36708]: pgmap v612: 129 pgs: 129 active+clean; 21 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 489 op/s 2024-06-10T01:02:46.963 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:02:48.583 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:48 smithi037 ceph-mon[32192]: pgmap v613: 129 pgs: 129 active+clean; 21 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 54 op/s 2024-06-10T01:02:48.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:48 smithi167 ceph-mon[36917]: pgmap v613: 129 pgs: 129 active+clean; 21 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 54 op/s 2024-06-10T01:02:48.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:48 smithi094 ceph-mon[36708]: pgmap v613: 129 pgs: 129 active+clean; 21 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 54 op/s 2024-06-10T01:02:49.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.124+0000 7fd57c558700 1 -- 172.21.15.37:0/2574699728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd574108aa0 msgr2=0x7fd574108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:49.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.124+0000 7fd57c558700 1 --2- 172.21.15.37:0/2574699728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd574108aa0 0x7fd574108f20 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd56400b0a0 tx=0x7fd56400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.128+0000 7fd57c558700 1 -- 172.21.15.37:0/2574699728 shutdown_connections 2024-06-10T01:02:49.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.128+0000 7fd57c558700 1 --2- 172.21.15.37:0/2574699728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd574109460 0x7fd574115970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.128+0000 7fd57c558700 1 --2- 172.21.15.37:0/2574699728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd57410eaa0 0x7fd57410ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.128+0000 7fd57c558700 1 --2- 172.21.15.37:0/2574699728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd574108aa0 0x7fd574108f20 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.128+0000 7fd57c558700 1 -- 172.21.15.37:0/2574699728 >> 172.21.15.37:0/2574699728 conn(0x7fd574075c60 msgr2=0x7fd574078080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:49.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.130+0000 7fd57c558700 1 -- 172.21.15.37:0/2574699728 shutdown_connections 2024-06-10T01:02:49.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.130+0000 7fd57c558700 1 -- 172.21.15.37:0/2574699728 wait complete. 2024-06-10T01:02:49.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.131+0000 7fd57c558700 1 Processor -- start 2024-06-10T01:02:49.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.133+0000 7fd57c558700 1 -- start start 2024-06-10T01:02:49.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.133+0000 7fd57c558700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd574108aa0 0x7fd57407c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:49.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.133+0000 7fd57c558700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd574109460 0x7fd57407cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:49.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.133+0000 7fd57c558700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd57410eaa0 0x7fd574081120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:49.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.133+0000 7fd57c558700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd574118ed0 con 0x7fd574109460 2024-06-10T01:02:49.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.133+0000 7fd57c558700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd574118d50 con 0x7fd574108aa0 2024-06-10T01:02:49.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.133+0000 7fd57c558700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd574119050 con 0x7fd57410eaa0 2024-06-10T01:02:49.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.134+0000 7fd57a2f4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd574108aa0 0x7fd57407c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:49.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.134+0000 7fd57a2f4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd574108aa0 0x7fd57407c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:36408/0 (socket says 172.21.15.37:36408) 2024-06-10T01:02:49.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.134+0000 7fd57a2f4700 1 -- 172.21.15.37:0/3239442851 learned_addr learned my addr 172.21.15.37:0/3239442851 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:49.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.134+0000 7fd57aaf5700 1 --2- 172.21.15.37:0/3239442851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd57410eaa0 0x7fd574081120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:49.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.135+0000 7fd579af3700 1 --2- 172.21.15.37:0/3239442851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd574109460 0x7fd57407cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:49.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.136+0000 7fd57a2f4700 1 -- 172.21.15.37:0/3239442851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd57410eaa0 msgr2=0x7fd574081120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:49.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.136+0000 7fd57a2f4700 1 --2- 172.21.15.37:0/3239442851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd57410eaa0 0x7fd574081120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.136+0000 7fd57a2f4700 1 -- 172.21.15.37:0/3239442851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd574109460 msgr2=0x7fd57407cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:49.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.136+0000 7fd57a2f4700 1 --2- 172.21.15.37:0/3239442851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd574109460 0x7fd57407cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.136+0000 7fd57a2f4700 1 -- 172.21.15.37:0/3239442851 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5740817c0 con 0x7fd574108aa0 2024-06-10T01:02:49.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.137+0000 7fd57a2f4700 1 --2- 172.21.15.37:0/3239442851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd574108aa0 0x7fd57407c850 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd56800e9a0 tx=0x7fd56800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:49.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.138+0000 7fd5737fe700 1 -- 172.21.15.37:0/3239442851 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56800cbd0 con 0x7fd574108aa0 2024-06-10T01:02:49.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.138+0000 7fd5737fe700 1 -- 172.21.15.37:0/3239442851 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd56800cd70 con 0x7fd574108aa0 2024-06-10T01:02:49.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.138+0000 7fd5737fe700 1 -- 172.21.15.37:0/3239442851 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568018780 con 0x7fd574108aa0 2024-06-10T01:02:49.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.139+0000 7fd57c558700 1 -- 172.21.15.37:0/3239442851 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd574081ab0 con 0x7fd574108aa0 2024-06-10T01:02:49.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.139+0000 7fd57c558700 1 -- 172.21.15.37:0/3239442851 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5740712b0 con 0x7fd574108aa0 2024-06-10T01:02:49.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.151+0000 7fd5737fe700 1 -- 172.21.15.37:0/3239442851 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd568018920 con 0x7fd574108aa0 2024-06-10T01:02:49.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.152+0000 7fd5737fe700 1 --2- 172.21.15.37:0/3239442851 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd55c0a2cf0 0x7fd55c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:49.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.153+0000 7fd5737fe700 1 -- 172.21.15.37:0/3239442851 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fd5680cf750 con 0x7fd574108aa0 2024-06-10T01:02:49.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.153+0000 7fd579af3700 1 --2- 172.21.15.37:0/3239442851 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd55c0a2cf0 0x7fd55c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:49.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.158+0000 7fd586ddbb80 1 Processor -- start 2024-06-10T01:02:49.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.158+0000 7fd579af3700 1 --2- 172.21.15.37:0/3239442851 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd55c0a2cf0 0x7fd55c0a51b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd57407ded0 tx=0x7fd564006230 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:49.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.159+0000 7fd586ddbb80 1 -- start start 2024-06-10T01:02:49.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.159+0000 7fd586ddbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56105993df60 0x56105993e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:49.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.159+0000 7fd586ddbb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56105993e880 0x561059938260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:49.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.159+0000 7fd586ddbb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5610599387a0 0x561059938bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:49.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.159+0000 7fd586ddbb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56105982af60 con 0x5610599387a0 2024-06-10T01:02:49.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.159+0000 7fd586ddbb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561059889850 con 0x56105993e880 2024-06-10T01:02:49.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.159+0000 7fd586ddbb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5610597f1460 con 0x56105993df60 2024-06-10T01:02:49.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.159+0000 7fd579af3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56105993e880 0x561059938260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:49.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.159+0000 7fd579af3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56105993e880 0x561059938260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:36410/0 (socket says 172.21.15.37:36410) 2024-06-10T01:02:49.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.159+0000 7fd579af3700 1 -- 172.21.15.37:0/2870634113 learned_addr learned my addr 172.21.15.37:0/2870634113 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:49.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.160+0000 7fd579af3700 1 -- 172.21.15.37:0/2870634113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56105993df60 msgr2=0x56105993e340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:02:49.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.160+0000 7fd579af3700 1 --2- 172.21.15.37:0/2870634113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56105993df60 0x56105993e340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.160+0000 7fd579af3700 1 -- 172.21.15.37:0/2870634113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5610599387a0 msgr2=0x561059938bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:49.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.160+0000 7fd579af3700 1 --2- 172.21.15.37:0/2870634113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5610599387a0 0x561059938bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.160+0000 7fd579af3700 1 -- 172.21.15.37:0/2870634113 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610597d88d0 con 0x56105993e880 2024-06-10T01:02:49.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.160+0000 7fd579af3700 1 --2- 172.21.15.37:0/2870634113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56105993e880 0x561059938260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd564015000 tx=0x7fd56401c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=594513b2a70dace9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:49.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.160+0000 7fd570ff9700 1 -- 172.21.15.37:0/2870634113 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564009b80 con 0x56105993e880 2024-06-10T01:02:49.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.160+0000 7fd570ff9700 1 -- 172.21.15.37:0/2870634113 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd56402d070 con 0x56105993e880 2024-06-10T01:02:49.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.160+0000 7fd570ff9700 1 -- 172.21.15.37:0/2870634113 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564009b80 con 0x56105993e880 2024-06-10T01:02:49.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.164+0000 7fd586ddbb80 1 -- 172.21.15.37:0/2870634113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56105993e880 msgr2=0x561059938260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:49.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.164+0000 7fd586ddbb80 1 --2- 172.21.15.37:0/2870634113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56105993e880 0x561059938260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd564015000 tx=0x7fd56401c000 comp rx=0 tx=0).stop 2024-06-10T01:02:49.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.165+0000 7fd586ddbb80 1 -- 172.21.15.37:0/2870634113 shutdown_connections 2024-06-10T01:02:49.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.165+0000 7fd586ddbb80 1 --2- 172.21.15.37:0/2870634113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5610599387a0 0x561059938bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.165+0000 7fd586ddbb80 1 --2- 172.21.15.37:0/2870634113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56105993df60 0x56105993e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.165+0000 7fd586ddbb80 1 --2- 172.21.15.37:0/2870634113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56105993e880 0x561059938260 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.165+0000 7fd586ddbb80 1 -- 172.21.15.37:0/2870634113 >> 172.21.15.37:0/2870634113 conn(0x561059865080 msgr2=0x561059933220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:49.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.169+0000 7fd586ddbb80 1 -- 172.21.15.37:0/2870634113 shutdown_connections 2024-06-10T01:02:49.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.172+0000 7fd586ddbb80 1 -- 172.21.15.37:0/2870634113 wait complete. 2024-06-10T01:02:49.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.172+0000 7fd586ddbb80 1 -- start start 2024-06-10T01:02:49.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.172+0000 7fd586ddbb80 1 Processor -- start 2024-06-10T01:02:49.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.173+0000 7fd586ddbb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610599387a0 0x561059944580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:49.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.173+0000 7fd586ddbb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56105993df60 0x561059941880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:49.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.173+0000 7fd586ddbb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56105993e880 0x561059941dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:49.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.173+0000 7fd57a2f4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610599387a0 0x561059944580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:49.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.173+0000 7fd57a2f4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610599387a0 0x561059944580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:59404/0 (socket says 172.21.15.37:59404) 2024-06-10T01:02:49.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.174+0000 7fd57a2f4700 1 -- 172.21.15.37:0/2890959197 learned_addr learned my addr 172.21.15.37:0/2890959197 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:49.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.174+0000 7fd586ddbb80 1 -- 172.21.15.37:0/2890959197 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561059829fc0 con 0x56105993e880 2024-06-10T01:02:49.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.174+0000 7fd586ddbb80 1 -- 172.21.15.37:0/2890959197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561059885810 con 0x56105993df60 2024-06-10T01:02:49.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.174+0000 7fd586ddbb80 1 -- 172.21.15.37:0/2890959197 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561059945200 con 0x5610599387a0 2024-06-10T01:02:49.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.174+0000 7fd586ddbb80 10 client.0 _ll_register_callbacks cb 0x56105981f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:02:49.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.175+0000 7fd57a2f4700 1 -- 172.21.15.37:0/2890959197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56105993df60 msgr2=0x561059941880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:49.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.175+0000 7fd57a2f4700 1 --2- 172.21.15.37:0/2890959197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56105993df60 0x561059941880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.175+0000 7fd57a2f4700 1 -- 172.21.15.37:0/2890959197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56105993e880 msgr2=0x561059941dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:49.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.175+0000 7fd57a2f4700 1 --2- 172.21.15.37:0/2890959197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56105993e880 0x561059941dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.175+0000 7fd57a2f4700 1 -- 172.21.15.37:0/2890959197 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56105984f750 con 0x5610599387a0 2024-06-10T01:02:49.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.175+0000 7fd57a2f4700 1 --2- 172.21.15.37:0/2890959197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610599387a0 0x561059944580 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd5680cd1d0 tx=0x7fd56809b650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a749e2d12a885dc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:49.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.175+0000 7fd559ffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:02:49.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.175+0000 7fd559ffb700 1 -- 172.21.15.37:0/2890959197 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56809bbc0 con 0x5610599387a0 2024-06-10T01:02:49.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.175+0000 7fd586ddbb80 1 -- 172.21.15.37:0/2890959197 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd56402d070 con 0x5610599387a0 2024-06-10T01:02:49.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.176+0000 7fd559ffb700 1 -- 172.21.15.37:0/2890959197 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fd56809b010 con 0x5610599387a0 2024-06-10T01:02:49.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.176+0000 7fd559ffb700 1 -- 172.21.15.37:0/2890959197 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd56809ff70 con 0x5610599387a0 2024-06-10T01:02:49.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.176+0000 7fd586ddbb80 10 client.26005 fetch_fsmap learned FSMap version 27 2024-06-10T01:02:49.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.176+0000 7fd586ddbb80 1 -- 172.21.15.37:0/2890959197 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56105994f770 con 0x5610599387a0 2024-06-10T01:02:49.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.178+0000 7fd559ffb700 1 -- 172.21.15.37:0/2890959197 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5680a4830 con 0x5610599387a0 2024-06-10T01:02:49.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.178+0000 7fd559ffb700 1 -- 172.21.15.37:0/2890959197 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd5680a1100 con 0x5610599387a0 2024-06-10T01:02:49.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.178+0000 7fd586ddbb80 10 client.26005 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:02:49.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.178+0000 7fd586ddbb80 10 client.26005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:02:49.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.178+0000 7fd586ddbb80 20 client.26005 populate_metadata read hostname 'smithi037' 2024-06-10T01:02:49.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.178+0000 7fd586ddbb80 1 --2- 172.21.15.37:0/2890959197 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561059952260 0x5610599546c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:49.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.178+0000 7fd586ddbb80 4 client.26005 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:02:49.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.178+0000 7fd586ddbb80 1 -- 172.21.15.37:0/2890959197 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd56402d070 con 0x561059952260 2024-06-10T01:02:49.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.183+0000 7fd579af3700 1 --2- 172.21.15.37:0/2890959197 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561059952260 0x5610599546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:49.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.191+0000 7fd579af3700 1 --2- 172.21.15.37:0/2890959197 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561059952260 0x5610599546c0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea238833dcfaf87 server_cookie=8d6df9e828b0a09 in_seq=0 out_seq=0 2024-06-10T01:02:49.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.192+0000 7fd559ffb700 10 client.26005 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:49.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.198+0000 7fd559ffb700 1 -- 172.21.15.37:0/2890959197 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fd56402d070 con 0x561059952260 2024-06-10T01:02:49.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.199+0000 7fd559ffb700 10 client.26005 handle_command_reply: tid=0 2024-06-10T01:02:49.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.199+0000 7fd586ddbb80 1 client.26005 shutdown 2024-06-10T01:02:49.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.199+0000 7fd586ddbb80 1 -- 172.21.15.37:0/2890959197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610599387a0 msgr2=0x561059944580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:49.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.199+0000 7fd586ddbb80 1 --2- 172.21.15.37:0/2890959197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610599387a0 0x561059944580 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd5680cd1d0 tx=0x7fd56809b650 comp rx=0 tx=0).stop 2024-06-10T01:02:49.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.200+0000 7fd586ddbb80 1 -- 172.21.15.37:0/2890959197 shutdown_connections 2024-06-10T01:02:49.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.200+0000 7fd586ddbb80 1 --2- 172.21.15.37:0/2890959197 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561059952260 0x5610599546c0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.200+0000 7fd586ddbb80 1 --2- 172.21.15.37:0/2890959197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56105993e880 0x561059941dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.200+0000 7fd586ddbb80 1 --2- 172.21.15.37:0/2890959197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610599387a0 0x561059944580 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.200+0000 7fd586ddbb80 1 --2- 172.21.15.37:0/2890959197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56105993df60 0x561059941880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.200+0000 7fd586ddbb80 1 -- 172.21.15.37:0/2890959197 >> 172.21.15.37:0/2890959197 conn(0x561059865080 msgr2=0x561059932700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:49.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.200+0000 7fd559ffb700 0 client.26005 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:49.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.200+0000 7fd586ddbb80 1 -- 172.21.15.37:0/2890959197 shutdown_connections 2024-06-10T01:02:49.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.201+0000 7fd586ddbb80 1 -- 172.21.15.37:0/2890959197 wait complete. 2024-06-10T01:02:49.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.201+0000 7fd586ddbb80 20 client.26005 trim_cache size 0 max 16384 2024-06-10T01:02:49.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.232+0000 7fd586ddbb80 1 Processor -- start 2024-06-10T01:02:49.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.233+0000 7fd586ddbb80 1 -- start start 2024-06-10T01:02:49.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.233+0000 7fd586ddbb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610599387a0 0x561059938260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:49.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.233+0000 7fd586ddbb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56105993df60 0x56105993f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:49.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.234+0000 7fd57a2f4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610599387a0 0x561059938260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:49.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.234+0000 7fd57a2f4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610599387a0 0x561059938260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:36428/0 (socket says 172.21.15.37:36428) 2024-06-10T01:02:49.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.234+0000 7fd579af3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56105993df60 0x56105993f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:49.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.234+0000 7fd579af3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56105993df60 0x56105993f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55572/0 (socket says 172.21.15.37:55572) 2024-06-10T01:02:49.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.234+0000 7fd57a2f4700 1 -- 172.21.15.37:0/4287565956 learned_addr learned my addr 172.21.15.37:0/4287565956 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:49.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.234+0000 7fd586ddbb80 1 --2- 172.21.15.37:0/4287565956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56105993e880 0x561059940800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:49.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.234+0000 7fd586ddbb80 1 -- 172.21.15.37:0/4287565956 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd560011590 con 0x56105993df60 2024-06-10T01:02:49.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.234+0000 7fd586ddbb80 1 -- 172.21.15.37:0/4287565956 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd56000ee20 con 0x5610599387a0 2024-06-10T01:02:49.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.234+0000 7fd57aaf5700 1 --2- 172.21.15.37:0/4287565956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56105993e880 0x561059940800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:49.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.234+0000 7fd586ddbb80 1 -- 172.21.15.37:0/4287565956 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd56000cc70 con 0x56105993e880 2024-06-10T01:02:49.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.235+0000 7fd579af3700 1 -- 172.21.15.37:0/4287565956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56105993e880 msgr2=0x561059940800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:49.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.235+0000 7fd579af3700 1 --2- 172.21.15.37:0/4287565956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56105993e880 0x561059940800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.235+0000 7fd579af3700 1 -- 172.21.15.37:0/4287565956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610599387a0 msgr2=0x561059938260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:49.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.235+0000 7fd579af3700 1 --2- 172.21.15.37:0/4287565956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610599387a0 0x561059938260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.235+0000 7fd579af3700 1 -- 172.21.15.37:0/4287565956 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561059944d80 con 0x56105993df60 2024-06-10T01:02:49.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.236+0000 7fd579af3700 1 --2- 172.21.15.37:0/4287565956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56105993df60 0x56105993f4f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd564009fd0 tx=0x7fd5640099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d815f4d7b2af74a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.236+0000 7fd55bfff700 1 -- 172.21.15.37:0/4287565956 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564009b80 con 0x56105993df60 2024-06-10T01:02:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.237+0000 7fd55bfff700 1 -- 172.21.15.37:0/4287565956 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd56402d070 con 0x56105993df60 2024-06-10T01:02:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.237+0000 7fd55bfff700 1 -- 172.21.15.37:0/4287565956 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56401cbd0 con 0x56105993df60 2024-06-10T01:02:49.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.238+0000 7fd586ddbb80 1 -- 172.21.15.37:0/4287565956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56105993df60 msgr2=0x56105993f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:49.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.238+0000 7fd586ddbb80 1 --2- 172.21.15.37:0/4287565956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56105993df60 0x56105993f4f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd564009fd0 tx=0x7fd5640099e0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.239+0000 7fd586ddbb80 1 -- 172.21.15.37:0/4287565956 shutdown_connections 2024-06-10T01:02:49.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.239+0000 7fd586ddbb80 1 --2- 172.21.15.37:0/4287565956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56105993e880 0x561059940800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.239+0000 7fd586ddbb80 1 --2- 172.21.15.37:0/4287565956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610599387a0 0x561059938260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.239+0000 7fd586ddbb80 1 --2- 172.21.15.37:0/4287565956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56105993df60 0x56105993f4f0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.239+0000 7fd586ddbb80 1 -- 172.21.15.37:0/4287565956 >> 172.21.15.37:0/4287565956 conn(0x561059865080 msgr2=0x56105993c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:49.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.240+0000 7fd586ddbb80 1 -- 172.21.15.37:0/4287565956 shutdown_connections 2024-06-10T01:02:49.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.240+0000 7fd586ddbb80 1 -- 172.21.15.37:0/4287565956 wait complete. 2024-06-10T01:02:49.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.241+0000 7fd586ddbb80 1 -- start start 2024-06-10T01:02:49.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.241+0000 7fd586ddbb80 1 Processor -- start 2024-06-10T01:02:49.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.242+0000 7fd586ddbb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610599387a0 0x56105994a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:49.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.242+0000 7fd586ddbb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56105993df60 0x56105994aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:49.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.242+0000 7fd57a2f4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610599387a0 0x56105994a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:49.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.242+0000 7fd57a2f4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610599387a0 0x56105994a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:36436/0 (socket says 172.21.15.37:36436) 2024-06-10T01:02:49.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.242+0000 7fd57a2f4700 1 -- 172.21.15.37:0/2752549688 learned_addr learned my addr 172.21.15.37:0/2752549688 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:49.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.242+0000 7fd586ddbb80 1 --2- 172.21.15.37:0/2752549688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56105993e880 0x561059958fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:49.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.242+0000 7fd586ddbb80 1 -- 172.21.15.37:0/2752549688 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd56000d2e0 con 0x56105993df60 2024-06-10T01:02:49.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.242+0000 7fd586ddbb80 1 -- 172.21.15.37:0/2752549688 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd56000e750 con 0x5610599387a0 2024-06-10T01:02:49.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.242+0000 7fd586ddbb80 1 -- 172.21.15.37:0/2752549688 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd56000d9b0 con 0x56105993e880 2024-06-10T01:02:49.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.243+0000 7fd579af3700 1 --2- 172.21.15.37:0/2752549688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56105993df60 0x56105994aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:49.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.243+0000 7fd586ddbb80 10 client.0 _ll_register_callbacks cb 0x56105981f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:02:49.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.243+0000 7fd57aaf5700 1 --2- 172.21.15.37:0/2752549688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56105993e880 0x561059958fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:49.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.245+0000 7fd579af3700 1 -- 172.21.15.37:0/2752549688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56105993e880 msgr2=0x561059958fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:49.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.245+0000 7fd579af3700 1 --2- 172.21.15.37:0/2752549688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56105993e880 0x561059958fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.245+0000 7fd579af3700 1 -- 172.21.15.37:0/2752549688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610599387a0 msgr2=0x56105994a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:49.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.245+0000 7fd579af3700 1 --2- 172.21.15.37:0/2752549688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610599387a0 0x56105994a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.245+0000 7fd579af3700 1 -- 172.21.15.37:0/2752549688 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56105995aa10 con 0x56105993df60 2024-06-10T01:02:49.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.247+0000 7fd579af3700 1 --2- 172.21.15.37:0/2752549688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56105993df60 0x56105994aa60 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x56105993fff0 tx=0x7fd564007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d789e6575ebec95f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:49.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.248+0000 7fd559ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:02:49.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.250+0000 7fd559ffb700 1 -- 172.21.15.37:0/2752549688 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56403e020 con 0x56105993df60 2024-06-10T01:02:49.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.250+0000 7fd559ffb700 1 -- 172.21.15.37:0/2752549688 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fd56401c650 con 0x56105993df60 2024-06-10T01:02:49.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.250+0000 7fd586ddbb80 1 -- 172.21.15.37:0/2752549688 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd56402d070 con 0x56105993df60 2024-06-10T01:02:49.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.251+0000 7fd559ffb700 1 -- 172.21.15.37:0/2752549688 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd564009b80 con 0x56105993df60 2024-06-10T01:02:49.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.251+0000 7fd586ddbb80 10 client.16455 fetch_fsmap learned FSMap version 27 2024-06-10T01:02:49.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.251+0000 7fd586ddbb80 1 -- 172.21.15.37:0/2752549688 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5610599bc320 con 0x56105993df60 2024-06-10T01:02:49.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.252+0000 7fd559ffb700 1 -- 172.21.15.37:0/2752549688 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd56402d070 con 0x56105993df60 2024-06-10T01:02:49.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.252+0000 7fd559ffb700 1 -- 172.21.15.37:0/2752549688 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fd564011070 con 0x56105993df60 2024-06-10T01:02:49.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.252+0000 7fd586ddbb80 10 client.16455 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:02:49.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.252+0000 7fd586ddbb80 10 client.16455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:02:49.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.252+0000 7fd586ddbb80 20 client.16455 populate_metadata read hostname 'smithi037' 2024-06-10T01:02:49.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.252+0000 7fd586ddbb80 1 --2- 172.21.15.37:0/2752549688 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5610599bc4f0 0x5610599be8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:49.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.253+0000 7fd586ddbb80 4 client.16455 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:02:49.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.253+0000 7fd586ddbb80 1 -- 172.21.15.37:0/2752549688 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561059896d50 con 0x5610599bc4f0 2024-06-10T01:02:49.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.256+0000 7fd57a2f4700 1 --2- 172.21.15.37:0/2752549688 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5610599bc4f0 0x5610599be8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:49.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.259+0000 7fd57a2f4700 1 --2- 172.21.15.37:0/2752549688 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5610599bc4f0 0x5610599be8e0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2e8f66ded744a81 server_cookie=9db9389cdf8521b6 in_seq=0 out_seq=0 2024-06-10T01:02:49.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.259+0000 7fd559ffb700 10 client.16455 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:49.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.267+0000 7fd559ffb700 1 -- 172.21.15.37:0/2752549688 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561059896d50 con 0x5610599bc4f0 2024-06-10T01:02:49.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.267+0000 7fd559ffb700 10 client.16455 handle_command_reply: tid=0 2024-06-10T01:02:49.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.267+0000 7fd586ddbb80 1 client.16455 shutdown 2024-06-10T01:02:49.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.268+0000 7fd586ddbb80 1 -- 172.21.15.37:0/2752549688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56105993df60 msgr2=0x56105994aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:49.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.268+0000 7fd586ddbb80 1 --2- 172.21.15.37:0/2752549688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56105993df60 0x56105994aa60 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x56105993fff0 tx=0x7fd564007dc0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.269+0000 7fd586ddbb80 1 -- 172.21.15.37:0/2752549688 shutdown_connections 2024-06-10T01:02:49.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.269+0000 7fd586ddbb80 1 --2- 172.21.15.37:0/2752549688 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5610599bc4f0 0x5610599be8e0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.269+0000 7fd586ddbb80 1 --2- 172.21.15.37:0/2752549688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56105993e880 0x561059958fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.269+0000 7fd586ddbb80 1 --2- 172.21.15.37:0/2752549688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610599387a0 0x56105994a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.269+0000 7fd586ddbb80 1 --2- 172.21.15.37:0/2752549688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56105993df60 0x56105994aa60 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.269+0000 7fd586ddbb80 1 -- 172.21.15.37:0/2752549688 >> 172.21.15.37:0/2752549688 conn(0x561059865080 msgr2=0x5610599419c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:49.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.269+0000 7fd559ffb700 0 client.16455 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:49.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.270+0000 7fd586ddbb80 1 -- 172.21.15.37:0/2752549688 shutdown_connections 2024-06-10T01:02:49.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.270+0000 7fd586ddbb80 1 -- 172.21.15.37:0/2752549688 wait complete. 2024-06-10T01:02:49.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.271+0000 7fd586ddbb80 20 client.16455 trim_cache size 0 max 16384 2024-06-10T01:02:49.270 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T01:02:49.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.274+0000 7fd57c558700 1 -- 172.21.15.37:0/3239442851 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd55c0a2cf0 msgr2=0x7fd55c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:49.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.274+0000 7fd57c558700 1 --2- 172.21.15.37:0/3239442851 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd55c0a2cf0 0x7fd55c0a51b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd57407ded0 tx=0x7fd564006230 comp rx=0 tx=0).stop 2024-06-10T01:02:49.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.274+0000 7fd57c558700 1 -- 172.21.15.37:0/3239442851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd574108aa0 msgr2=0x7fd57407c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:49.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.274+0000 7fd57c558700 1 --2- 172.21.15.37:0/3239442851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd574108aa0 0x7fd57407c850 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd56800e9a0 tx=0x7fd56800ed60 comp rx=0 tx=0).stop 2024-06-10T01:02:49.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.275+0000 7fd57c558700 1 -- 172.21.15.37:0/3239442851 shutdown_connections 2024-06-10T01:02:49.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.275+0000 7fd57c558700 1 --2- 172.21.15.37:0/3239442851 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd55c0a2cf0 0x7fd55c0a51b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.275+0000 7fd57c558700 1 --2- 172.21.15.37:0/3239442851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd57410eaa0 0x7fd574081120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.275+0000 7fd57c558700 1 --2- 172.21.15.37:0/3239442851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd574108aa0 0x7fd57407c850 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.275+0000 7fd57c558700 1 --2- 172.21.15.37:0/3239442851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd574109460 0x7fd57407cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:49.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.275+0000 7fd57c558700 1 -- 172.21.15.37:0/3239442851 >> 172.21.15.37:0/3239442851 conn(0x7fd574075c60 msgr2=0x7fd574076d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:49.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.275+0000 7fd57c558700 1 -- 172.21.15.37:0/3239442851 shutdown_connections 2024-06-10T01:02:49.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:49.276+0000 7fd57c558700 1 -- 172.21.15.37:0/3239442851 wait complete. 2024-06-10T01:02:50.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:50 smithi167 ceph-mon[36917]: pgmap v614: 129 pgs: 129 active+clean; 26 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 1.41k op/s 2024-06-10T01:02:50.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:50 smithi094 ceph-mon[36708]: pgmap v614: 129 pgs: 129 active+clean; 26 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 1.41k op/s 2024-06-10T01:02:50.643 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:50 smithi037 ceph-mon[32192]: pgmap v614: 129 pgs: 129 active+clean; 26 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 1.41k op/s 2024-06-10T01:02:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:50.743+0000 7f4d46ffd640 1 -- 172.21.15.37:0/3946019435 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55999bc75c20 con 0x55999bd8e710 2024-06-10T01:02:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:50.743+0000 7f4d46ffd640 10 client.25889 handle_command_reply: tid=0 2024-06-10T01:02:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:50.743+0000 7f4d70e49740 1 client.25889 shutdown 2024-06-10T01:02:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:50.743+0000 7f4d70e49740 1 -- 172.21.15.37:0/3946019435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55999bd0e210 msgr2=0x55999bd1c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:50.743+0000 7f4d70e49740 1 --2- 172.21.15.37:0/3946019435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55999bd0e210 0x55999bd1c550 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0314e0 tx=0x7f4d5c0b83d0 comp rx=0 tx=0).stop 2024-06-10T01:02:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:50.743+0000 7f4d70e49740 1 -- 172.21.15.37:0/3946019435 shutdown_connections 2024-06-10T01:02:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:50.743+0000 7f4d70e49740 1 --2- 172.21.15.37:0/3946019435 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55999bd8e710 0x55999bd90b00 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:50.743+0000 7f4d70e49740 1 --2- 172.21.15.37:0/3946019435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55999bd13380 0x55999bd1d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:50.743+0000 7f4d70e49740 1 --2- 172.21.15.37:0/3946019435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55999bd0e210 0x55999bd1c550 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:50.743+0000 7f4d70e49740 1 --2- 172.21.15.37:0/3946019435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55999bd0d820 0x55999bd1c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:50.743+0000 7f4d46ffd640 0 client.25889 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:50.743+0000 7f4d70e49740 1 -- 172.21.15.37:0/3946019435 >> 172.21.15.37:0/3946019435 conn(0x55999bd00680 msgr2=0x55999bd17a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:50.743+0000 7f4d70e49740 1 -- 172.21.15.37:0/3946019435 shutdown_connections 2024-06-10T01:02:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:50.743+0000 7f4d70e49740 1 -- 172.21.15.37:0/3946019435 wait complete. 2024-06-10T01:02:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:50.744+0000 7f4d70e49740 20 client.25889 trim_cache size 0 max 16384 2024-06-10T01:02:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:02:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:02:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:02:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:50.744+0000 7f4d6e13b640 1 -- 172.21.15.37:0/1955005330 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4d500a2c30 msgr2=0x7f4d500a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:50.744+0000 7f4d6e13b640 1 --2- 172.21.15.37:0/1955005330 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4d500a2c30 0x7f4d500a50f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4d68101950 tx=0x7f4d58012040 comp rx=0 tx=0).stop 2024-06-10T01:02:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:50.744+0000 7f4d6e13b640 1 -- 172.21.15.37:0/1955005330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d68101af0 msgr2=0x7f4d68197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:50.744+0000 7f4d6e13b640 1 --2- 172.21.15.37:0/1955005330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d68101af0 0x7f4d68197100 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c02a930 tx=0x7f4d5c032ce0 comp rx=0 tx=0).stop 2024-06-10T01:02:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:50.744+0000 7f4d6e13b640 1 -- 172.21.15.37:0/1955005330 shutdown_connections 2024-06-10T01:02:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:50.744+0000 7f4d6e13b640 1 --2- 172.21.15.37:0/1955005330 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4d500a2c30 0x7f4d500a50f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:50.744+0000 7f4d6e13b640 1 --2- 172.21.15.37:0/1955005330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d681024b0 0x7f4d6819e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:50.744+0000 7f4d6e13b640 1 --2- 172.21.15.37:0/1955005330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d68101af0 0x7f4d68197100 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:50.744+0000 7f4d6e13b640 1 --2- 172.21.15.37:0/1955005330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d68100900 0x7f4d68196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:50.744+0000 7f4d6e13b640 1 -- 172.21.15.37:0/1955005330 >> 172.21.15.37:0/1955005330 conn(0x7f4d680fbe70 msgr2=0x7f4d68104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:50.744+0000 7f4d6e13b640 1 -- 172.21.15.37:0/1955005330 shutdown_connections 2024-06-10T01:02:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:02:50.744+0000 7f4d6e13b640 1 -- 172.21.15.37:0/1955005330 wait complete. 2024-06-10T01:02:51.209 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:02:51.210 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:02:51.467 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:02:52.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:52 smithi167 ceph-mon[36917]: pgmap v615: 129 pgs: 129 active+clean; 26 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 2.00k op/s 2024-06-10T01:02:52.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:52 smithi094 ceph-mon[36708]: pgmap v615: 129 pgs: 129 active+clean; 26 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 2.00k op/s 2024-06-10T01:02:52.615 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:52 smithi037 ceph-mon[32192]: pgmap v615: 129 pgs: 129 active+clean; 26 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 2.00k op/s 2024-06-10T01:02:53.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.482+0000 7fdad0e8a700 1 -- 172.21.15.37:0/3533009844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdacc100a80 msgr2=0x7fdacc100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:53.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.482+0000 7fdad0e8a700 1 --2- 172.21.15.37:0/3533009844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdacc100a80 0x7fdacc100f00 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009990 tx=0x7fdab4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.482+0000 7fdad0e8a700 1 -- 172.21.15.37:0/3533009844 shutdown_connections 2024-06-10T01:02:53.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.482+0000 7fdad0e8a700 1 --2- 172.21.15.37:0/3533009844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdacc101440 0x7fdacc10f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.482+0000 7fdad0e8a700 1 --2- 172.21.15.37:0/3533009844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdacc108510 0x7fdacc1088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.482+0000 7fdad0e8a700 1 --2- 172.21.15.37:0/3533009844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdacc100a80 0x7fdacc100f00 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.482+0000 7fdad0e8a700 1 -- 172.21.15.37:0/3533009844 >> 172.21.15.37:0/3533009844 conn(0x7fdacc077a00 msgr2=0x7fdacc0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:53.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.482+0000 7fdad0e8a700 1 -- 172.21.15.37:0/3533009844 shutdown_connections 2024-06-10T01:02:53.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.482+0000 7fdad0e8a700 1 -- 172.21.15.37:0/3533009844 wait complete. 2024-06-10T01:02:53.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.483+0000 7fdad0e8a700 1 Processor -- start 2024-06-10T01:02:53.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.483+0000 7fdad0e8a700 1 -- start start 2024-06-10T01:02:53.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.484+0000 7fdad0e8a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdacc100a80 0x7fdacc19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:53.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.484+0000 7fdad0e8a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdacc101440 0x7fdacc19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:53.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.484+0000 7fdad0e8a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdacc108510 0x7fdacc198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:53.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.484+0000 7fdad0e8a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc112790 con 0x7fdacc100a80 2024-06-10T01:02:53.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.484+0000 7fdad0e8a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc112610 con 0x7fdacc101440 2024-06-10T01:02:53.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.484+0000 7fdad0e8a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc112910 con 0x7fdacc108510 2024-06-10T01:02:53.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.484+0000 7fdaca59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdacc100a80 0x7fdacc19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:53.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.484+0000 7fdaca59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdacc100a80 0x7fdacc19e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44486/0 (socket says 172.21.15.37:44486) 2024-06-10T01:02:53.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.484+0000 7fdaca59c700 1 -- 172.21.15.37:0/3015408343 learned_addr learned my addr 172.21.15.37:0/3015408343 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:53.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.484+0000 7fdacad9d700 1 --2- 172.21.15.37:0/3015408343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdacc108510 0x7fdacc198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:53.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.484+0000 7fdac9d9b700 1 --2- 172.21.15.37:0/3015408343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdacc101440 0x7fdacc19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:53.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.484+0000 7fdaca59c700 1 -- 172.21.15.37:0/3015408343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdacc108510 msgr2=0x7fdacc198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:53.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.484+0000 7fdaca59c700 1 --2- 172.21.15.37:0/3015408343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdacc108510 0x7fdacc198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.484+0000 7fdaca59c700 1 -- 172.21.15.37:0/3015408343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdacc101440 msgr2=0x7fdacc19edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:53.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.484+0000 7fdaca59c700 1 --2- 172.21.15.37:0/3015408343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdacc101440 0x7fdacc19edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.484+0000 7fdaca59c700 1 -- 172.21.15.37:0/3015408343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdacc1991f0 con 0x7fdacc100a80 2024-06-10T01:02:53.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.484+0000 7fdacad9d700 1 --2- 172.21.15.37:0/3015408343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdacc108510 0x7fdacc198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:53.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.484+0000 7fdaca59c700 1 --2- 172.21.15.37:0/3015408343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdacc100a80 0x7fdacc19e880 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00d950 tx=0x7fdabc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:53.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.485+0000 7fdac37fe700 1 -- 172.21.15.37:0/3015408343 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc016070 con 0x7fdacc100a80 2024-06-10T01:02:53.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.485+0000 7fdac37fe700 1 -- 172.21.15.37:0/3015408343 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdabc00fd70 con 0x7fdacc100a80 2024-06-10T01:02:53.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.485+0000 7fdad0e8a700 1 -- 172.21.15.37:0/3015408343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdacc1994e0 con 0x7fdacc100a80 2024-06-10T01:02:53.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.485+0000 7fdac37fe700 1 -- 172.21.15.37:0/3015408343 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc00e610 con 0x7fdacc100a80 2024-06-10T01:02:53.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.485+0000 7fdad0e8a700 1 -- 172.21.15.37:0/3015408343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdacc1a5750 con 0x7fdacc100a80 2024-06-10T01:02:53.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.487+0000 7fdac37fe700 1 -- 172.21.15.37:0/3015408343 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fdabc019070 con 0x7fdacc100a80 2024-06-10T01:02:53.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.488+0000 7fdac37fe700 1 --2- 172.21.15.37:0/3015408343 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdab80a2c20 0x7fdab80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:53.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.488+0000 7fdac37fe700 1 -- 172.21.15.37:0/3015408343 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fdabc0cee40 con 0x7fdacc100a80 2024-06-10T01:02:53.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.488+0000 7fdac9d9b700 1 --2- 172.21.15.37:0/3015408343 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdab80a2c20 0x7fdab80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:53.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.488+0000 7fdadb70db80 1 Processor -- start 2024-06-10T01:02:53.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.489+0000 7fdadb70db80 1 -- start start 2024-06-10T01:02:53.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.489+0000 7fdac9d9b700 1 --2- 172.21.15.37:0/3015408343 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdab80a2c20 0x7fdab80a50e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdab400b2e0 tx=0x7fdab4005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:53.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.489+0000 7fdadb70db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562745fb66c0 0x562745fb6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:53.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.489+0000 7fdadb70db80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562745fb6fe0 0x562745fb0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:53.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.489+0000 7fdadb70db80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562745fb0fa0 0x562745fb13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:53.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.489+0000 7fdadb70db80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562745f02890 con 0x562745fb0fa0 2024-06-10T01:02:53.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.489+0000 7fdadb70db80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562745eaa7a0 con 0x562745fb6fe0 2024-06-10T01:02:53.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.489+0000 7fdadb70db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562745efe850 con 0x562745fb66c0 2024-06-10T01:02:53.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.489+0000 7fdacad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562745fb0fa0 0x562745fb13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:53.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.489+0000 7fdaca59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562745fb66c0 0x562745fb6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:53.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.489+0000 7fdacad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562745fb0fa0 0x562745fb13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44498/0 (socket says 172.21.15.37:44498) 2024-06-10T01:02:53.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.489+0000 7fdaca59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562745fb66c0 0x562745fb6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:57920/0 (socket says 172.21.15.37:57920) 2024-06-10T01:02:53.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.489+0000 7fdacad9d700 1 -- 172.21.15.37:0/921398815 learned_addr learned my addr 172.21.15.37:0/921398815 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:53.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.489+0000 7fdac9d9b700 1 --2- 172.21.15.37:0/921398815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562745fb6fe0 0x562745fb0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:53.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.489+0000 7fdacad9d700 1 -- 172.21.15.37:0/921398815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562745fb66c0 msgr2=0x562745fb6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:53.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.489+0000 7fdacad9d700 1 --2- 172.21.15.37:0/921398815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562745fb66c0 0x562745fb6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.489+0000 7fdacad9d700 1 -- 172.21.15.37:0/921398815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562745fb6fe0 msgr2=0x562745fb0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:53.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.489+0000 7fdacad9d700 1 --2- 172.21.15.37:0/921398815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562745fb6fe0 0x562745fb0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.489+0000 7fdacad9d700 1 -- 172.21.15.37:0/921398815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562745ea0fd0 con 0x562745fb0fa0 2024-06-10T01:02:53.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.489+0000 7fdaca59c700 1 --2- 172.21.15.37:0/921398815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562745fb66c0 0x562745fb6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:53.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.490+0000 7fdacad9d700 1 --2- 172.21.15.37:0/921398815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562745fb0fa0 0x562745fb13c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fdac4003eb0 tx=0x7fdac4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c2a08ec46b90f69 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:53.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.490+0000 7fdac0ff9700 1 -- 172.21.15.37:0/921398815 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4019020 con 0x562745fb0fa0 2024-06-10T01:02:53.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.490+0000 7fdac0ff9700 1 -- 172.21.15.37:0/921398815 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac40092d0 con 0x562745fb0fa0 2024-06-10T01:02:53.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.490+0000 7fdac0ff9700 1 -- 172.21.15.37:0/921398815 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac400e8b0 con 0x562745fb0fa0 2024-06-10T01:02:53.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.491+0000 7fdadb70db80 1 -- 172.21.15.37:0/921398815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562745fb0fa0 msgr2=0x562745fb13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:53.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.491+0000 7fdadb70db80 1 --2- 172.21.15.37:0/921398815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562745fb0fa0 0x562745fb13c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fdac4003eb0 tx=0x7fdac4009920 comp rx=0 tx=0).stop 2024-06-10T01:02:53.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.491+0000 7fdadb70db80 1 -- 172.21.15.37:0/921398815 shutdown_connections 2024-06-10T01:02:53.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.491+0000 7fdadb70db80 1 --2- 172.21.15.37:0/921398815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562745fb0fa0 0x562745fb13c0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.491+0000 7fdadb70db80 1 --2- 172.21.15.37:0/921398815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562745fb66c0 0x562745fb6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.491+0000 7fdadb70db80 1 --2- 172.21.15.37:0/921398815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562745fb6fe0 0x562745fb0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.491+0000 7fdadb70db80 1 -- 172.21.15.37:0/921398815 >> 172.21.15.37:0/921398815 conn(0x562745edbd40 msgr2=0x562745fab8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:53.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.491+0000 7fdadb70db80 1 -- 172.21.15.37:0/921398815 shutdown_connections 2024-06-10T01:02:53.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.491+0000 7fdadb70db80 1 -- 172.21.15.37:0/921398815 wait complete. 2024-06-10T01:02:53.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.491+0000 7fdadb70db80 1 -- start start 2024-06-10T01:02:53.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.491+0000 7fdadb70db80 1 Processor -- start 2024-06-10T01:02:53.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.492+0000 7fdadb70db80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562745fb0fa0 0x562745fbcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:53.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.492+0000 7fdadb70db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562745fb66c0 0x562745fba020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:53.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.492+0000 7fdadb70db80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562745fb6fe0 0x562745fba560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:53.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.492+0000 7fdadb70db80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562745fbda50 con 0x562745fb6fe0 2024-06-10T01:02:53.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.492+0000 7fdadb70db80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562745ea2c70 con 0x562745fb0fa0 2024-06-10T01:02:53.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.492+0000 7fdadb70db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562745fbdbd0 con 0x562745fb66c0 2024-06-10T01:02:53.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.492+0000 7fdadb70db80 10 client.0 _ll_register_callbacks cb 0x562745e9c9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:02:53.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.492+0000 7fdaca59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562745fb0fa0 0x562745fbcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:53.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.492+0000 7fdaca59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562745fb0fa0 0x562745fbcd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:52972/0 (socket says 172.21.15.37:52972) 2024-06-10T01:02:53.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.492+0000 7fdaca59c700 1 -- 172.21.15.37:0/2553271220 learned_addr learned my addr 172.21.15.37:0/2553271220 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:53.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.492+0000 7fdac9d9b700 1 --2- 172.21.15.37:0/2553271220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562745fb66c0 0x562745fba020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:53.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.492+0000 7fdacad9d700 1 --2- 172.21.15.37:0/2553271220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562745fb6fe0 0x562745fba560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:53.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.492+0000 7fdaca59c700 1 -- 172.21.15.37:0/2553271220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562745fb66c0 msgr2=0x562745fba020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:53.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.492+0000 7fdaca59c700 1 --2- 172.21.15.37:0/2553271220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562745fb66c0 0x562745fba020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.492+0000 7fdaca59c700 1 -- 172.21.15.37:0/2553271220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562745fb6fe0 msgr2=0x562745fba560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:53.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.492+0000 7fdaca59c700 1 --2- 172.21.15.37:0/2553271220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562745fb6fe0 0x562745fba560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.492+0000 7fdaca59c700 1 -- 172.21.15.37:0/2553271220 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562745e51950 con 0x562745fb0fa0 2024-06-10T01:02:53.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.492+0000 7fdaca59c700 1 --2- 172.21.15.37:0/2553271220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562745fb0fa0 0x562745fbcd20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdabc097c80 tx=0x7fdabc09b2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abdcce49ab65fb39 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:53.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.492+0000 7fdab1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:02:53.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.493+0000 7fdab1ffb700 1 -- 172.21.15.37:0/2553271220 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc020070 con 0x562745fb0fa0 2024-06-10T01:02:53.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.493+0000 7fdab1ffb700 1 -- 172.21.15.37:0/2553271220 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fdabc010710 con 0x562745fb0fa0 2024-06-10T01:02:53.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.493+0000 7fdadb70db80 1 -- 172.21.15.37:0/2553271220 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdac40092d0 con 0x562745fb0fa0 2024-06-10T01:02:53.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.493+0000 7fdab1ffb700 1 -- 172.21.15.37:0/2553271220 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdabc0a3f20 con 0x562745fb0fa0 2024-06-10T01:02:53.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.493+0000 7fdadb70db80 10 client.25931 fetch_fsmap learned FSMap version 27 2024-06-10T01:02:53.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.493+0000 7fdadb70db80 1 -- 172.21.15.37:0/2553271220 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562745fcc740 con 0x562745fb0fa0 2024-06-10T01:02:53.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.493+0000 7fdab1ffb700 1 -- 172.21.15.37:0/2553271220 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac40092d0 con 0x562745fb0fa0 2024-06-10T01:02:53.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.493+0000 7fdab1ffb700 1 -- 172.21.15.37:0/2553271220 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fdabc09f170 con 0x562745fb0fa0 2024-06-10T01:02:53.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.493+0000 7fdadb70db80 10 client.25931 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:02:53.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.493+0000 7fdadb70db80 10 client.25931 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:02:53.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.493+0000 7fdadb70db80 20 client.25931 populate_metadata read hostname 'smithi037' 2024-06-10T01:02:53.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.493+0000 7fdadb70db80 1 --2- 172.21.15.37:0/2553271220 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x562745fced80 0x562745fd1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:53.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.493+0000 7fdadb70db80 4 client.25931 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:02:53.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.493+0000 7fdadb70db80 1 -- 172.21.15.37:0/2553271220 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562745f0fd90 con 0x562745fced80 2024-06-10T01:02:53.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.494+0000 7fdac9d9b700 1 --2- 172.21.15.37:0/2553271220 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x562745fced80 0x562745fd1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:53.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.494+0000 7fdab1ffb700 10 client.25931 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:53.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.494+0000 7fdac9d9b700 1 --2- 172.21.15.37:0/2553271220 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x562745fced80 0x562745fd1170 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fd4afcd097262cd server_cookie=6b75276c030c24b in_seq=0 out_seq=0 2024-06-10T01:02:53.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.495+0000 7fdab1ffb700 1 -- 172.21.15.37:0/2553271220 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562745f0fd90 con 0x562745fced80 2024-06-10T01:02:53.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.495+0000 7fdab1ffb700 10 client.25931 handle_command_reply: tid=0 2024-06-10T01:02:53.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.495+0000 7fdadb70db80 1 client.25931 shutdown 2024-06-10T01:02:53.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.496+0000 7fdadb70db80 1 -- 172.21.15.37:0/2553271220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562745fb0fa0 msgr2=0x562745fbcd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:53.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.496+0000 7fdadb70db80 1 --2- 172.21.15.37:0/2553271220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562745fb0fa0 0x562745fbcd20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdabc097c80 tx=0x7fdabc09b2b0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.496+0000 7fdadb70db80 1 -- 172.21.15.37:0/2553271220 shutdown_connections 2024-06-10T01:02:53.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.496+0000 7fdadb70db80 1 --2- 172.21.15.37:0/2553271220 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x562745fced80 0x562745fd1170 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.496+0000 7fdadb70db80 1 --2- 172.21.15.37:0/2553271220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562745fb6fe0 0x562745fba560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.496+0000 7fdadb70db80 1 --2- 172.21.15.37:0/2553271220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562745fb0fa0 0x562745fbcd20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.496+0000 7fdab1ffb700 0 client.25931 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:53.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.496+0000 7fdadb70db80 1 --2- 172.21.15.37:0/2553271220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562745fb66c0 0x562745fba020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.496+0000 7fdadb70db80 1 -- 172.21.15.37:0/2553271220 >> 172.21.15.37:0/2553271220 conn(0x562745edbd40 msgr2=0x562745fafa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:53.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.496+0000 7fdadb70db80 1 -- 172.21.15.37:0/2553271220 shutdown_connections 2024-06-10T01:02:53.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.496+0000 7fdadb70db80 1 -- 172.21.15.37:0/2553271220 wait complete. 2024-06-10T01:02:53.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.496+0000 7fdadb70db80 20 client.25931 trim_cache size 0 max 16384 2024-06-10T01:02:53.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.517+0000 7fdadb70db80 1 Processor -- start 2024-06-10T01:02:53.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.517+0000 7fdadb70db80 1 -- start start 2024-06-10T01:02:53.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.517+0000 7fdadb70db80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562745fb0fa0 0x562745fb0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:53.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.517+0000 7fdadb70db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562745fb66c0 0x562745fa5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:53.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.517+0000 7fdadb70db80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562745fb6fe0 0x562745fb9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:53.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.517+0000 7fdadb70db80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc1aeb70 con 0x562745fb6fe0 2024-06-10T01:02:53.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.517+0000 7fdadb70db80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc1ac400 con 0x562745fb0fa0 2024-06-10T01:02:53.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.517+0000 7fdadb70db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc1aa250 con 0x562745fb66c0 2024-06-10T01:02:53.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.517+0000 7fdacad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562745fb6fe0 0x562745fb9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:53.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.517+0000 7fdac9d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562745fb66c0 0x562745fa5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:53.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.517+0000 7fdacad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562745fb6fe0 0x562745fb9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44518/0 (socket says 172.21.15.37:44518) 2024-06-10T01:02:53.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.517+0000 7fdacad9d700 1 -- 172.21.15.37:0/608494532 learned_addr learned my addr 172.21.15.37:0/608494532 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:53.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.517+0000 7fdaca59c700 1 --2- 172.21.15.37:0/608494532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562745fb0fa0 0x562745fb0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:53.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.518+0000 7fdacad9d700 1 -- 172.21.15.37:0/608494532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562745fb66c0 msgr2=0x562745fa5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:53.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.518+0000 7fdacad9d700 1 --2- 172.21.15.37:0/608494532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562745fb66c0 0x562745fa5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.518+0000 7fdacad9d700 1 -- 172.21.15.37:0/608494532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562745fb0fa0 msgr2=0x562745fb0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:53.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.518+0000 7fdacad9d700 1 --2- 172.21.15.37:0/608494532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562745fb0fa0 0x562745fb0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.518+0000 7fdacad9d700 1 -- 172.21.15.37:0/608494532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562745fd17a0 con 0x562745fb6fe0 2024-06-10T01:02:53.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.518+0000 7fdac9d9b700 1 --2- 172.21.15.37:0/608494532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562745fb66c0 0x562745fa5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:53.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.518+0000 7fdacad9d700 1 --2- 172.21.15.37:0/608494532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562745fb6fe0 0x562745fb9020 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009510 tx=0x7fdac4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9815da23252b87b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:53.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.518+0000 7fdab3fff700 1 -- 172.21.15.37:0/608494532 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4019040 con 0x562745fb6fe0 2024-06-10T01:02:53.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.518+0000 7fdab3fff700 1 -- 172.21.15.37:0/608494532 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac4023070 con 0x562745fb6fe0 2024-06-10T01:02:53.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.518+0000 7fdab3fff700 1 -- 172.21.15.37:0/608494532 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac400e750 con 0x562745fb6fe0 2024-06-10T01:02:53.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.519+0000 7fdadb70db80 1 -- 172.21.15.37:0/608494532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562745fb6fe0 msgr2=0x562745fb9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:53.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.519+0000 7fdadb70db80 1 --2- 172.21.15.37:0/608494532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562745fb6fe0 0x562745fb9020 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009510 tx=0x7fdac4009bd0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.519+0000 7fdadb70db80 1 -- 172.21.15.37:0/608494532 shutdown_connections 2024-06-10T01:02:53.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.519+0000 7fdadb70db80 1 --2- 172.21.15.37:0/608494532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562745fb6fe0 0x562745fb9020 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.519+0000 7fdadb70db80 1 --2- 172.21.15.37:0/608494532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562745fb0fa0 0x562745fb0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.519+0000 7fdadb70db80 1 --2- 172.21.15.37:0/608494532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562745fb66c0 0x562745fa5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.519+0000 7fdadb70db80 1 -- 172.21.15.37:0/608494532 >> 172.21.15.37:0/608494532 conn(0x562745edbd40 msgr2=0x562745fb8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:53.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.519+0000 7fdadb70db80 1 -- 172.21.15.37:0/608494532 shutdown_connections 2024-06-10T01:02:53.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.519+0000 7fdadb70db80 1 -- 172.21.15.37:0/608494532 wait complete. 2024-06-10T01:02:53.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.519+0000 7fdadb70db80 1 -- start start 2024-06-10T01:02:53.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.519+0000 7fdadb70db80 1 Processor -- start 2024-06-10T01:02:53.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.519+0000 7fdadb70db80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562745fb0fa0 0x562745fc3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:53.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.519+0000 7fdadb70db80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562745fb66c0 0x562745fc3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:53.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.519+0000 7fdadb70db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562745fb6fe0 0x562745fbf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:53.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.519+0000 7fdadb70db80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc1aa8c0 con 0x562745fb66c0 2024-06-10T01:02:53.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.519+0000 7fdadb70db80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc1abd30 con 0x562745fb0fa0 2024-06-10T01:02:53.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.519+0000 7fdadb70db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc1aaf90 con 0x562745fb6fe0 2024-06-10T01:02:53.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.520+0000 7fdadb70db80 10 client.0 _ll_register_callbacks cb 0x562745e9c9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:02:53.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.520+0000 7fdac9d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562745fb66c0 0x562745fc3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:53.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.520+0000 7fdac9d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562745fb66c0 0x562745fc3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44522/0 (socket says 172.21.15.37:44522) 2024-06-10T01:02:53.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.520+0000 7fdac9d9b700 1 -- 172.21.15.37:0/1726054064 learned_addr learned my addr 172.21.15.37:0/1726054064 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:53.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.520+0000 7fdacad9d700 1 --2- 172.21.15.37:0/1726054064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562745fb6fe0 0x562745fbf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:53.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.520+0000 7fdaca59c700 1 --2- 172.21.15.37:0/1726054064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562745fb0fa0 0x562745fc3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:53.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.520+0000 7fdac9d9b700 1 -- 172.21.15.37:0/1726054064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562745fb6fe0 msgr2=0x562745fbf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:53.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.520+0000 7fdac9d9b700 1 --2- 172.21.15.37:0/1726054064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562745fb6fe0 0x562745fbf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.520+0000 7fdac9d9b700 1 -- 172.21.15.37:0/1726054064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562745fb0fa0 msgr2=0x562745fc3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:53.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.520+0000 7fdac9d9b700 1 --2- 172.21.15.37:0/1726054064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562745fb0fa0 0x562745fc3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.520+0000 7fdac9d9b700 1 -- 172.21.15.37:0/1726054064 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562745fbffb0 con 0x562745fb66c0 2024-06-10T01:02:53.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.520+0000 7fdac9d9b700 1 --2- 172.21.15.37:0/1726054064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562745fb66c0 0x562745fc3770 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fdab4000c00 tx=0x7fdab401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39009314c56f00d7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:53.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.520+0000 7fdab1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:02:53.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.520+0000 7fdab1ffb700 1 -- 172.21.15.37:0/1726054064 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab40056e0 con 0x562745fb66c0 2024-06-10T01:02:53.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.520+0000 7fdab1ffb700 1 -- 172.21.15.37:0/1726054064 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fdab4003a90 con 0x562745fb66c0 2024-06-10T01:02:53.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.520+0000 7fdadb70db80 1 -- 172.21.15.37:0/1726054064 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdac4023070 con 0x562745fb66c0 2024-06-10T01:02:53.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.521+0000 7fdab1ffb700 1 -- 172.21.15.37:0/1726054064 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdab4003c90 con 0x562745fb66c0 2024-06-10T01:02:53.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.521+0000 7fdadb70db80 10 client.16473 fetch_fsmap learned FSMap version 27 2024-06-10T01:02:53.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.521+0000 7fdadb70db80 1 -- 172.21.15.37:0/1726054064 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56274603a720 con 0x562745fb66c0 2024-06-10T01:02:53.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.521+0000 7fdab1ffb700 1 -- 172.21.15.37:0/1726054064 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac4023070 con 0x562745fb66c0 2024-06-10T01:02:53.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.521+0000 7fdab1ffb700 1 -- 172.21.15.37:0/1726054064 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fdab4038070 con 0x562745fb66c0 2024-06-10T01:02:53.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.521+0000 7fdadb70db80 10 client.16473 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:02:53.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.521+0000 7fdadb70db80 10 client.16473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:02:53.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.521+0000 7fdadb70db80 20 client.16473 populate_metadata read hostname 'smithi037' 2024-06-10T01:02:53.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.521+0000 7fdadb70db80 1 --2- 172.21.15.37:0/1726054064 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56274603ae10 0x56274603d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:53.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.521+0000 7fdadb70db80 4 client.16473 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:02:53.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.521+0000 7fdadb70db80 1 -- 172.21.15.37:0/1726054064 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562745f0a310 con 0x56274603ae10 2024-06-10T01:02:53.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.521+0000 7fdaca59c700 1 --2- 172.21.15.37:0/1726054064 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56274603ae10 0x56274603d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:53.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.522+0000 7fdaca59c700 1 --2- 172.21.15.37:0/1726054064 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56274603ae10 0x56274603d230 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b39978d57349b93 server_cookie=410f78eb395a7e43 in_seq=0 out_seq=0 2024-06-10T01:02:53.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.522+0000 7fdab1ffb700 10 client.16473 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:53.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.662+0000 7fdab1ffb700 1 -- 172.21.15.37:0/1726054064 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562745f0a310 con 0x56274603ae10 2024-06-10T01:02:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.662+0000 7fdab1ffb700 10 client.16473 handle_command_reply: tid=0 2024-06-10T01:02:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.662+0000 7fdadb70db80 1 client.16473 shutdown 2024-06-10T01:02:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.662+0000 7fdadb70db80 1 -- 172.21.15.37:0/1726054064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562745fb66c0 msgr2=0x562745fc3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.662+0000 7fdadb70db80 1 --2- 172.21.15.37:0/1726054064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562745fb66c0 0x562745fc3770 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fdab4000c00 tx=0x7fdab401f000 comp rx=0 tx=0).stop 2024-06-10T01:02:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.663+0000 7fdadb70db80 1 -- 172.21.15.37:0/1726054064 shutdown_connections 2024-06-10T01:02:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.663+0000 7fdadb70db80 1 --2- 172.21.15.37:0/1726054064 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56274603ae10 0x56274603d230 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.663+0000 7fdadb70db80 1 --2- 172.21.15.37:0/1726054064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562745fb6fe0 0x562745fbf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.663+0000 7fdadb70db80 1 --2- 172.21.15.37:0/1726054064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562745fb0fa0 0x562745fc3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.663+0000 7fdadb70db80 1 --2- 172.21.15.37:0/1726054064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562745fb66c0 0x562745fc3770 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.663+0000 7fdadb70db80 1 -- 172.21.15.37:0/1726054064 >> 172.21.15.37:0/1726054064 conn(0x562745edbd40 msgr2=0x562745fbc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.663+0000 7fdab1ffb700 0 client.16473 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.663+0000 7fdadb70db80 1 -- 172.21.15.37:0/1726054064 shutdown_connections 2024-06-10T01:02:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.663+0000 7fdadb70db80 1 -- 172.21.15.37:0/1726054064 wait complete. 2024-06-10T01:02:53.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.663+0000 7fdadb70db80 20 client.16473 trim_cache size 0 max 16384 2024-06-10T01:02:53.662 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:02:53.662 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-06-10T01:02:53.662 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "59187c88-c6e1-417e-9ae1-b55b232cd94e", 2024-06-10T01:02:53.662 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-06-10T01:02:53.662 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:02:53.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.663+0000 7fdad0e8a700 1 -- 172.21.15.37:0/3015408343 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdab80a2c20 msgr2=0x7fdab80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:53.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.663+0000 7fdad0e8a700 1 --2- 172.21.15.37:0/3015408343 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdab80a2c20 0x7fdab80a50e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdab400b2e0 tx=0x7fdab4005960 comp rx=0 tx=0).stop 2024-06-10T01:02:53.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.663+0000 7fdad0e8a700 1 -- 172.21.15.37:0/3015408343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdacc100a80 msgr2=0x7fdacc19e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:53.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.663+0000 7fdad0e8a700 1 --2- 172.21.15.37:0/3015408343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdacc100a80 0x7fdacc19e880 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00d950 tx=0x7fdabc00dc60 comp rx=0 tx=0).stop 2024-06-10T01:02:53.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.663+0000 7fdad0e8a700 1 -- 172.21.15.37:0/3015408343 shutdown_connections 2024-06-10T01:02:53.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.663+0000 7fdad0e8a700 1 --2- 172.21.15.37:0/3015408343 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdab80a2c20 0x7fdab80a50e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.663+0000 7fdad0e8a700 1 --2- 172.21.15.37:0/3015408343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdacc108510 0x7fdacc198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.663+0000 7fdad0e8a700 1 --2- 172.21.15.37:0/3015408343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdacc100a80 0x7fdacc19e880 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.663+0000 7fdad0e8a700 1 --2- 172.21.15.37:0/3015408343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdacc101440 0x7fdacc19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:53.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.664+0000 7fdad0e8a700 1 -- 172.21.15.37:0/3015408343 >> 172.21.15.37:0/3015408343 conn(0x7fdacc077a00 msgr2=0x7fdacc0ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:53.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.664+0000 7fdad0e8a700 1 -- 172.21.15.37:0/3015408343 shutdown_connections 2024-06-10T01:02:53.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:53.664+0000 7fdad0e8a700 1 -- 172.21.15.37:0/3015408343 wait complete. 2024-06-10T01:02:54.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:54 smithi037 ceph-mon[32192]: pgmap v616: 129 pgs: 129 active+clean; 14 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 2.62k op/s 2024-06-10T01:02:54.420 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:02:54.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:54 smithi167 ceph-mon[36917]: pgmap v616: 129 pgs: 129 active+clean; 14 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 2.62k op/s 2024-06-10T01:02:54.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:54 smithi094 ceph-mon[36708]: pgmap v616: 129 pgs: 129 active+clean; 14 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 2.62k op/s 2024-06-10T01:02:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:02:54.879 INFO:tasks.quiescer.fs.[cephfs]:Total duration (58.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-10T01:02:54.879 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.45 2024-06-10T01:02:54.880 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:02:55.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:55 smithi037 ceph-mon[32192]: scrub queued for path: / 2024-06-10T01:02:55.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:55 smithi037 ceph-mon[32192]: scrub summary: idle+waiting paths [/] 2024-06-10T01:02:55.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:55 smithi037 ceph-mon[32192]: scrub summary: active paths [/] 2024-06-10T01:02:55.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:55 smithi167 ceph-mon[36917]: scrub queued for path: / 2024-06-10T01:02:55.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:55 smithi167 ceph-mon[36917]: scrub summary: idle+waiting paths [/] 2024-06-10T01:02:55.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:55 smithi167 ceph-mon[36917]: scrub summary: active paths [/] 2024-06-10T01:02:55.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:55 smithi094 ceph-mon[36708]: scrub queued for path: / 2024-06-10T01:02:55.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:55 smithi094 ceph-mon[36708]: scrub summary: idle+waiting paths [/] 2024-06-10T01:02:55.612 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:55 smithi094 ceph-mon[36708]: scrub summary: active paths [/] 2024-06-10T01:02:56.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:56 smithi167 ceph-mon[36917]: pgmap v617: 129 pgs: 129 active+clean; 11 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 1.6 MiB/s wr, 3.06k op/s 2024-06-10T01:02:56.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:56 smithi094 ceph-mon[36708]: pgmap v617: 129 pgs: 129 active+clean; 11 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 1.6 MiB/s wr, 3.06k op/s 2024-06-10T01:02:56.643 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:56 smithi037 ceph-mon[32192]: pgmap v617: 129 pgs: 129 active+clean; 11 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 1.6 MiB/s wr, 3.06k op/s 2024-06-10T01:02:57.991 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:02:58.452 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:02:58 smithi037 ceph-mon[32192]: pgmap v618: 129 pgs: 129 active+clean; 11 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 1.6 MiB/s wr, 3.07k op/s 2024-06-10T01:02:58.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:02:58 smithi167 ceph-mon[36917]: pgmap v618: 129 pgs: 129 active+clean; 11 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 1.6 MiB/s wr, 3.07k op/s 2024-06-10T01:02:58.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:02:58 smithi094 ceph-mon[36708]: pgmap v618: 129 pgs: 129 active+clean; 11 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 1.6 MiB/s wr, 3.07k op/s 2024-06-10T01:02:58.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.581+0000 7f7a31024700 1 -- 172.21.15.37:0/3027673995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a2c108500 msgr2=0x7f7a2c1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:58.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.581+0000 7f7a31024700 1 --2- 172.21.15.37:0/3027673995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a2c108500 0x7f7a2c1088e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c009990 tx=0x7f7a1c009ca0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.581+0000 7f7a31024700 1 -- 172.21.15.37:0/3027673995 shutdown_connections 2024-06-10T01:02:58.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.581+0000 7f7a31024700 1 --2- 172.21.15.37:0/3027673995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a2c06ab30 0x7f7a2c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.581+0000 7f7a31024700 1 --2- 172.21.15.37:0/3027673995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a2c108500 0x7f7a2c1088e0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.581+0000 7f7a31024700 1 --2- 172.21.15.37:0/3027673995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a2c06a170 0x7f7a2c06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.581+0000 7f7a31024700 1 -- 172.21.15.37:0/3027673995 >> 172.21.15.37:0/3027673995 conn(0x7f7a2c078340 msgr2=0x7f7a2c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:58.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.581+0000 7f7a31024700 1 -- 172.21.15.37:0/3027673995 shutdown_connections 2024-06-10T01:02:58.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.581+0000 7f7a31024700 1 -- 172.21.15.37:0/3027673995 wait complete. 2024-06-10T01:02:58.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.582+0000 7f7a31024700 1 Processor -- start 2024-06-10T01:02:58.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.582+0000 7f7a31024700 1 -- start start 2024-06-10T01:02:58.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.582+0000 7f7a31024700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a2c06a170 0x7f7a2c19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:58.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.582+0000 7f7a31024700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a2c06ab30 0x7f7a2c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:58.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.582+0000 7f7a31024700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a2c108500 0x7f7a2c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:58.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.582+0000 7f7a31024700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c112760 con 0x7f7a2c108500 2024-06-10T01:02:58.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.582+0000 7f7a31024700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c1125e0 con 0x7f7a2c06a170 2024-06-10T01:02:58.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.582+0000 7f7a31024700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c1128e0 con 0x7f7a2c06ab30 2024-06-10T01:02:58.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.583+0000 7f7a2b59e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a2c108500 0x7f7a2c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:58.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.583+0000 7f7a2b59e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a2c108500 0x7f7a2c198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44552/0 (socket says 172.21.15.37:44552) 2024-06-10T01:02:58.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.583+0000 7f7a2b59e700 1 -- 172.21.15.37:0/1591281219 learned_addr learned my addr 172.21.15.37:0/1591281219 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:58.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.583+0000 7f7a2a59c700 1 --2- 172.21.15.37:0/1591281219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a2c06ab30 0x7f7a2c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:58.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.583+0000 7f7a2ad9d700 1 --2- 172.21.15.37:0/1591281219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a2c06a170 0x7f7a2c19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:58.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.583+0000 7f7a2a59c700 1 -- 172.21.15.37:0/1591281219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a2c06a170 msgr2=0x7f7a2c19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:58.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.583+0000 7f7a2a59c700 1 --2- 172.21.15.37:0/1591281219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a2c06a170 0x7f7a2c19e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.583+0000 7f7a2a59c700 1 -- 172.21.15.37:0/1591281219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a2c108500 msgr2=0x7f7a2c198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:58.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.583+0000 7f7a2a59c700 1 --2- 172.21.15.37:0/1591281219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a2c108500 0x7f7a2c198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.583+0000 7f7a2a59c700 1 -- 172.21.15.37:0/1591281219 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2c1991c0 con 0x7f7a2c06ab30 2024-06-10T01:02:58.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.583+0000 7f7a2b59e700 1 --2- 172.21.15.37:0/1591281219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a2c108500 0x7f7a2c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:58.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.584+0000 7f7a2a59c700 1 --2- 172.21.15.37:0/1591281219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a2c06ab30 0x7f7a2c19ed90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000e9f0 tx=0x7f7a2000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:58.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.584+0000 7f7a13fff700 1 -- 172.21.15.37:0/1591281219 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2000c9c0 con 0x7f7a2c06ab30 2024-06-10T01:02:58.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.584+0000 7f7a13fff700 1 -- 172.21.15.37:0/1591281219 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a2000cb60 con 0x7f7a2c06ab30 2024-06-10T01:02:58.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.584+0000 7f7a31024700 1 -- 172.21.15.37:0/1591281219 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a2c1994b0 con 0x7f7a2c06ab30 2024-06-10T01:02:58.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.584+0000 7f7a13fff700 1 -- 172.21.15.37:0/1591281219 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a20018680 con 0x7f7a2c06ab30 2024-06-10T01:02:58.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.585+0000 7f7a31024700 1 -- 172.21.15.37:0/1591281219 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a2c101390 con 0x7f7a2c06ab30 2024-06-10T01:02:58.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.587+0000 7f7a13fff700 1 -- 172.21.15.37:0/1591281219 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7a20010430 con 0x7f7a2c06ab30 2024-06-10T01:02:58.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.588+0000 7f7a13fff700 1 --2- 172.21.15.37:0/1591281219 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7a140a2d20 0x7f7a140a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:58.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.588+0000 7f7a13fff700 1 -- 172.21.15.37:0/1591281219 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f7a200cf510 con 0x7f7a2c06ab30 2024-06-10T01:02:58.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.588+0000 7f7a2ad9d700 1 --2- 172.21.15.37:0/1591281219 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7a140a2d20 0x7f7a140a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:58.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.589+0000 7f7a2ad9d700 1 --2- 172.21.15.37:0/1591281219 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7a140a2d20 0x7f7a140a51e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00b2e0 tx=0x7f7a1c01a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:58.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.589+0000 7f7a3b8a7b80 1 Processor -- start 2024-06-10T01:02:58.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.589+0000 7f7a3b8a7b80 1 -- start start 2024-06-10T01:02:58.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.590+0000 7f7a3b8a7b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560081bbe640 0x560081bbea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:58.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.590+0000 7f7a3b8a7b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560081bbef60 0x560081bb89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:58.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.590+0000 7f7a3b8a7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560081bb8f20 0x560081bb9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:58.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.590+0000 7f7a3b8a7b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560081b0a810 con 0x560081bbe640 2024-06-10T01:02:58.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.590+0000 7f7a3b8a7b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560081ab2720 con 0x560081bbef60 2024-06-10T01:02:58.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.590+0000 7f7a3b8a7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560081b067d0 con 0x560081bb8f20 2024-06-10T01:02:58.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.590+0000 7f7a2ad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560081bbe640 0x560081bbea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:58.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.590+0000 7f7a2ad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560081bbe640 0x560081bbea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44558/0 (socket says 172.21.15.37:44558) 2024-06-10T01:02:58.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.590+0000 7f7a2ad9d700 1 -- 172.21.15.37:0/3844392347 learned_addr learned my addr 172.21.15.37:0/3844392347 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:58.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.590+0000 7f7a2a59c700 1 --2- 172.21.15.37:0/3844392347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560081bbef60 0x560081bb89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:58.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.590+0000 7f7a2b59e700 1 --2- 172.21.15.37:0/3844392347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560081bb8f20 0x560081bb9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:58.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.590+0000 7f7a2a59c700 1 -- 172.21.15.37:0/3844392347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560081bb8f20 msgr2=0x560081bb9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:58.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.590+0000 7f7a2a59c700 1 --2- 172.21.15.37:0/3844392347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560081bb8f20 0x560081bb9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.590+0000 7f7a2a59c700 1 -- 172.21.15.37:0/3844392347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560081bbe640 msgr2=0x560081bbea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:58.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.590+0000 7f7a2a59c700 1 --2- 172.21.15.37:0/3844392347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560081bbe640 0x560081bbea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.590+0000 7f7a2a59c700 1 -- 172.21.15.37:0/3844392347 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560081aa8f50 con 0x560081bbef60 2024-06-10T01:02:58.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.590+0000 7f7a2a59c700 1 --2- 172.21.15.37:0/3844392347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560081bbef60 0x560081bb89e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7a2009b620 tx=0x7f7a2009b370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97f0316154b59214 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:58.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.590+0000 7f7a117fa700 1 -- 172.21.15.37:0/3844392347 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2009b920 con 0x560081bbef60 2024-06-10T01:02:58.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.591+0000 7f7a117fa700 1 -- 172.21.15.37:0/3844392347 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a2009fcd0 con 0x560081bbef60 2024-06-10T01:02:58.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.591+0000 7f7a117fa700 1 -- 172.21.15.37:0/3844392347 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a200a4360 con 0x560081bbef60 2024-06-10T01:02:58.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.591+0000 7f7a3b8a7b80 1 -- 172.21.15.37:0/3844392347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560081bbef60 msgr2=0x560081bb89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:58.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.591+0000 7f7a3b8a7b80 1 --2- 172.21.15.37:0/3844392347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560081bbef60 0x560081bb89e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7a2009b620 tx=0x7f7a2009b370 comp rx=0 tx=0).stop 2024-06-10T01:02:58.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.591+0000 7f7a3b8a7b80 1 -- 172.21.15.37:0/3844392347 shutdown_connections 2024-06-10T01:02:58.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.591+0000 7f7a3b8a7b80 1 --2- 172.21.15.37:0/3844392347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560081bb8f20 0x560081bb9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.591+0000 7f7a3b8a7b80 1 --2- 172.21.15.37:0/3844392347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560081bbe640 0x560081bbea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.591+0000 7f7a3b8a7b80 1 --2- 172.21.15.37:0/3844392347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560081bbef60 0x560081bb89e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.591+0000 7f7a3b8a7b80 1 -- 172.21.15.37:0/3844392347 >> 172.21.15.37:0/3844392347 conn(0x560081ae3cc0 msgr2=0x560081bb3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:58.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.591+0000 7f7a3b8a7b80 1 -- 172.21.15.37:0/3844392347 shutdown_connections 2024-06-10T01:02:58.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.592+0000 7f7a3b8a7b80 1 -- 172.21.15.37:0/3844392347 wait complete. 2024-06-10T01:02:58.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.592+0000 7f7a3b8a7b80 1 -- start start 2024-06-10T01:02:58.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.592+0000 7f7a3b8a7b80 1 Processor -- start 2024-06-10T01:02:58.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.593+0000 7f7a3b8a7b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560081bb8f20 0x560081bc4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:58.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.593+0000 7f7a3b8a7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560081bbe640 0x560081bc2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:58.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.593+0000 7f7a3b8a7b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560081bbef60 0x560081bc25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:58.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.593+0000 7f7a3b8a7b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560081bc5970 con 0x560081bb8f20 2024-06-10T01:02:58.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.593+0000 7f7a3b8a7b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560081aaabf0 con 0x560081bbef60 2024-06-10T01:02:58.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.593+0000 7f7a3b8a7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560081bc5af0 con 0x560081bbe640 2024-06-10T01:02:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.593+0000 7f7a3b8a7b80 10 client.0 _ll_register_callbacks cb 0x560081aa4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:02:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.593+0000 7f7a2ad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560081bb8f20 0x560081bc4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.593+0000 7f7a2ad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560081bb8f20 0x560081bc4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44568/0 (socket says 172.21.15.37:44568) 2024-06-10T01:02:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.593+0000 7f7a2ad9d700 1 -- 172.21.15.37:0/3575506913 learned_addr learned my addr 172.21.15.37:0/3575506913 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.593+0000 7f7a2a59c700 1 --2- 172.21.15.37:0/3575506913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560081bbe640 0x560081bc2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.593+0000 7f7a2b59e700 1 --2- 172.21.15.37:0/3575506913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560081bbef60 0x560081bc25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.593+0000 7f7a2ad9d700 1 -- 172.21.15.37:0/3575506913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560081bbe640 msgr2=0x560081bc2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.593+0000 7f7a2ad9d700 1 --2- 172.21.15.37:0/3575506913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560081bbe640 0x560081bc2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.593+0000 7f7a2ad9d700 1 -- 172.21.15.37:0/3575506913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560081bbef60 msgr2=0x560081bc25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.593+0000 7f7a2ad9d700 1 --2- 172.21.15.37:0/3575506913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560081bbef60 0x560081bc25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.593+0000 7f7a2ad9d700 1 -- 172.21.15.37:0/3575506913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560081a598d0 con 0x560081bb8f20 2024-06-10T01:02:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.593+0000 7f7a2b59e700 1 --2- 172.21.15.37:0/3575506913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560081bbef60 0x560081bc25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:02:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.593+0000 7f7a2ad9d700 1 --2- 172.21.15.37:0/3575506913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560081bb8f20 0x560081bc4d80 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c0098e0 tx=0x7f7a1c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1682ff7c8a9ae83b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:58.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.593+0000 7f7a0eff5700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:02:58.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.594+0000 7f7a0eff5700 1 -- 172.21.15.37:0/3575506913 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c03c060 con 0x560081bb8f20 2024-06-10T01:02:58.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.594+0000 7f7a0eff5700 1 -- 172.21.15.37:0/3575506913 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f7a1c003880 con 0x560081bb8f20 2024-06-10T01:02:58.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.594+0000 7f7a3b8a7b80 1 -- 172.21.15.37:0/3575506913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a2009fcd0 con 0x560081bb8f20 2024-06-10T01:02:58.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.594+0000 7f7a0eff5700 1 -- 172.21.15.37:0/3575506913 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7a1c003a80 con 0x560081bb8f20 2024-06-10T01:02:58.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.594+0000 7f7a3b8a7b80 10 client.16485 fetch_fsmap learned FSMap version 27 2024-06-10T01:02:58.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.594+0000 7f7a3b8a7b80 1 -- 172.21.15.37:0/3575506913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560081bd02a0 con 0x560081bb8f20 2024-06-10T01:02:58.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.594+0000 7f7a0eff5700 1 -- 172.21.15.37:0/3575506913 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a2009fcd0 con 0x560081bb8f20 2024-06-10T01:02:58.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.594+0000 7f7a0eff5700 1 -- 172.21.15.37:0/3575506913 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7a1c036070 con 0x560081bb8f20 2024-06-10T01:02:58.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.594+0000 7f7a3b8a7b80 10 client.16485 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:02:58.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.594+0000 7f7a3b8a7b80 10 client.16485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:02:58.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.594+0000 7f7a3b8a7b80 20 client.16485 populate_metadata read hostname 'smithi037' 2024-06-10T01:02:58.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.594+0000 7f7a3b8a7b80 1 --2- 172.21.15.37:0/3575506913 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560081bd28e0 0x560081bd4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:58.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.594+0000 7f7a3b8a7b80 4 client.16485 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:02:58.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.594+0000 7f7a3b8a7b80 1 -- 172.21.15.37:0/3575506913 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560081b17d10 con 0x560081bd28e0 2024-06-10T01:02:58.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.595+0000 7f7a2a59c700 1 --2- 172.21.15.37:0/3575506913 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560081bd28e0 0x560081bd4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:58.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.595+0000 7f7a0eff5700 10 client.16485 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:58.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.595+0000 7f7a2a59c700 1 --2- 172.21.15.37:0/3575506913 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560081bd28e0 0x560081bd4cd0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=368bcff9cfd7c784 server_cookie=cdf0b38ab3772944 in_seq=0 out_seq=0 2024-06-10T01:02:58.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.598+0000 7f7a0eff5700 1 -- 172.21.15.37:0/3575506913 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560081b17d10 con 0x560081bd28e0 2024-06-10T01:02:58.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.598+0000 7f7a0eff5700 10 client.16485 handle_command_reply: tid=0 2024-06-10T01:02:58.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.598+0000 7f7a3b8a7b80 1 client.16485 shutdown 2024-06-10T01:02:58.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.598+0000 7f7a3b8a7b80 1 -- 172.21.15.37:0/3575506913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560081bb8f20 msgr2=0x560081bc4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:58.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.598+0000 7f7a3b8a7b80 1 --2- 172.21.15.37:0/3575506913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560081bb8f20 0x560081bc4d80 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c0098e0 tx=0x7f7a1c009310 comp rx=0 tx=0).stop 2024-06-10T01:02:58.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.598+0000 7f7a3b8a7b80 1 -- 172.21.15.37:0/3575506913 shutdown_connections 2024-06-10T01:02:58.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.598+0000 7f7a3b8a7b80 1 --2- 172.21.15.37:0/3575506913 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560081bd28e0 0x560081bd4cd0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.598+0000 7f7a3b8a7b80 1 --2- 172.21.15.37:0/3575506913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560081bbef60 0x560081bc25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.598+0000 7f7a3b8a7b80 1 --2- 172.21.15.37:0/3575506913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560081bb8f20 0x560081bc4d80 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.598+0000 7f7a3b8a7b80 1 --2- 172.21.15.37:0/3575506913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560081bbe640 0x560081bc2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.598+0000 7f7a3b8a7b80 1 -- 172.21.15.37:0/3575506913 >> 172.21.15.37:0/3575506913 conn(0x560081ae3cc0 msgr2=0x560081bb35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:58.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.598+0000 7f7a0eff5700 0 client.16485 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:58.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.598+0000 7f7a3b8a7b80 1 -- 172.21.15.37:0/3575506913 shutdown_connections 2024-06-10T01:02:58.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.598+0000 7f7a3b8a7b80 1 -- 172.21.15.37:0/3575506913 wait complete. 2024-06-10T01:02:58.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.599+0000 7f7a3b8a7b80 20 client.16485 trim_cache size 0 max 16384 2024-06-10T01:02:58.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.625+0000 7f7a3b8a7b80 1 Processor -- start 2024-06-10T01:02:58.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.625+0000 7f7a3b8a7b80 1 -- start start 2024-06-10T01:02:58.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.625+0000 7f7a3b8a7b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560081bb8f20 0x560081bb89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:58.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.625+0000 7f7a3b8a7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560081bbe640 0x560081bad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:58.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.625+0000 7f7a3b8a7b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560081bbef60 0x560081bda550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:58.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.625+0000 7f7a3b8a7b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c1aeb40 con 0x560081bbef60 2024-06-10T01:02:58.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.625+0000 7f7a3b8a7b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c1ac3d0 con 0x560081bb8f20 2024-06-10T01:02:58.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.625+0000 7f7a3b8a7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c1aa220 con 0x560081bbe640 2024-06-10T01:02:58.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.625+0000 7f7a2b59e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560081bbef60 0x560081bda550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:58.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.625+0000 7f7a2b59e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560081bbef60 0x560081bda550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44574/0 (socket says 172.21.15.37:44574) 2024-06-10T01:02:58.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.625+0000 7f7a2b59e700 1 -- 172.21.15.37:0/4167556606 learned_addr learned my addr 172.21.15.37:0/4167556606 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:58.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.626+0000 7f7a2a59c700 1 --2- 172.21.15.37:0/4167556606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560081bbe640 0x560081bad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:58.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.626+0000 7f7a2ad9d700 1 --2- 172.21.15.37:0/4167556606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560081bb8f20 0x560081bb89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:58.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.626+0000 7f7a2b59e700 1 -- 172.21.15.37:0/4167556606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560081bbe640 msgr2=0x560081bad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:58.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.626+0000 7f7a2b59e700 1 --2- 172.21.15.37:0/4167556606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560081bbe640 0x560081bad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.626+0000 7f7a2b59e700 1 -- 172.21.15.37:0/4167556606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560081bb8f20 msgr2=0x560081bb89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:58.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.626+0000 7f7a2b59e700 1 --2- 172.21.15.37:0/4167556606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560081bb8f20 0x560081bb89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.626+0000 7f7a2b59e700 1 -- 172.21.15.37:0/4167556606 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560081bc0610 con 0x560081bbef60 2024-06-10T01:02:58.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.626+0000 7f7a2ad9d700 1 --2- 172.21.15.37:0/4167556606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560081bb8f20 0x560081bb89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:02:58.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.626+0000 7f7a2b59e700 1 --2- 172.21.15.37:0/4167556606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560081bbef60 0x560081bda550 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f7a24007750 tx=0x7f7a2400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea9132481aefcfe server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:58.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.626+0000 7f7a10ff9700 1 -- 172.21.15.37:0/4167556606 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a24003c00 con 0x560081bbef60 2024-06-10T01:02:58.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.626+0000 7f7a10ff9700 1 -- 172.21.15.37:0/4167556606 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a24003da0 con 0x560081bbef60 2024-06-10T01:02:58.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.626+0000 7f7a10ff9700 1 -- 172.21.15.37:0/4167556606 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2400ec20 con 0x560081bbef60 2024-06-10T01:02:58.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.627+0000 7f7a3b8a7b80 1 -- 172.21.15.37:0/4167556606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560081bbef60 msgr2=0x560081bda550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:58.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.627+0000 7f7a3b8a7b80 1 --2- 172.21.15.37:0/4167556606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560081bbef60 0x560081bda550 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f7a24007750 tx=0x7f7a2400b040 comp rx=0 tx=0).stop 2024-06-10T01:02:58.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.627+0000 7f7a3b8a7b80 1 -- 172.21.15.37:0/4167556606 shutdown_connections 2024-06-10T01:02:58.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.627+0000 7f7a3b8a7b80 1 --2- 172.21.15.37:0/4167556606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560081bbef60 0x560081bda550 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.627+0000 7f7a3b8a7b80 1 --2- 172.21.15.37:0/4167556606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560081bb8f20 0x560081bb89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.627+0000 7f7a3b8a7b80 1 --2- 172.21.15.37:0/4167556606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560081bbe640 0x560081bad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.627+0000 7f7a3b8a7b80 1 -- 172.21.15.37:0/4167556606 >> 172.21.15.37:0/4167556606 conn(0x560081ae3cc0 msgr2=0x560081bace00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:58.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.627+0000 7f7a3b8a7b80 1 -- 172.21.15.37:0/4167556606 shutdown_connections 2024-06-10T01:02:58.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.627+0000 7f7a3b8a7b80 1 -- 172.21.15.37:0/4167556606 wait complete. 2024-06-10T01:02:58.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.627+0000 7f7a3b8a7b80 1 -- start start 2024-06-10T01:02:58.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.627+0000 7f7a3b8a7b80 1 Processor -- start 2024-06-10T01:02:58.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.628+0000 7f7a3b8a7b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560081bb8f20 0x560081bc6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:58.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.628+0000 7f7a3b8a7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560081bbe640 0x560081bc71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:58.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.628+0000 7f7a3b8a7b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560081bbef60 0x560081bc7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:58.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.628+0000 7f7a3b8a7b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c1aa890 con 0x560081bbef60 2024-06-10T01:02:58.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.628+0000 7f7a3b8a7b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c1abd00 con 0x560081bb8f20 2024-06-10T01:02:58.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.628+0000 7f7a3b8a7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c1aaf60 con 0x560081bbe640 2024-06-10T01:02:58.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.628+0000 7f7a3b8a7b80 10 client.0 _ll_register_callbacks cb 0x560081aa4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:02:58.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.628+0000 7f7a2b59e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560081bbef60 0x560081bc7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:58.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.628+0000 7f7a2b59e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560081bbef60 0x560081bc7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44590/0 (socket says 172.21.15.37:44590) 2024-06-10T01:02:58.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.628+0000 7f7a2a59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560081bbe640 0x560081bc71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:58.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.628+0000 7f7a2b59e700 1 -- 172.21.15.37:0/2164847171 learned_addr learned my addr 172.21.15.37:0/2164847171 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:02:58.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.628+0000 7f7a2ad9d700 1 --2- 172.21.15.37:0/2164847171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560081bb8f20 0x560081bc6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:58.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.628+0000 7f7a2b59e700 1 -- 172.21.15.37:0/2164847171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560081bbe640 msgr2=0x560081bc71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:58.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.628+0000 7f7a2b59e700 1 --2- 172.21.15.37:0/2164847171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560081bbe640 0x560081bc71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.628+0000 7f7a2b59e700 1 -- 172.21.15.37:0/2164847171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560081bb8f20 msgr2=0x560081bc6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:02:58.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.628+0000 7f7a2b59e700 1 --2- 172.21.15.37:0/2164847171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560081bb8f20 0x560081bc6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.628+0000 7f7a2b59e700 1 -- 172.21.15.37:0/2164847171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560081c3c690 con 0x560081bbef60 2024-06-10T01:02:58.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.628+0000 7f7a2a59c700 1 --2- 172.21.15.37:0/2164847171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560081bbe640 0x560081bc71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:02:58.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.628+0000 7f7a2b59e700 1 --2- 172.21.15.37:0/2164847171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560081bbef60 0x560081bc7700 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7a24007d40 tx=0x7f7a24003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada680d5e6117f2c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:02:58.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.628+0000 7f7a0eff5700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:02:58.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.629+0000 7f7a0eff5700 1 -- 172.21.15.37:0/2164847171 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2400e860 con 0x560081bbef60 2024-06-10T01:02:58.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.629+0000 7f7a0eff5700 1 -- 172.21.15.37:0/2164847171 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f7a2400ea00 con 0x560081bbef60 2024-06-10T01:02:58.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.629+0000 7f7a3b8a7b80 1 -- 172.21.15.37:0/2164847171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a24003da0 con 0x560081bbef60 2024-06-10T01:02:58.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.629+0000 7f7a0eff5700 1 -- 172.21.15.37:0/2164847171 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7a24032050 con 0x560081bbef60 2024-06-10T01:02:58.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.629+0000 7f7a3b8a7b80 10 client.16491 fetch_fsmap learned FSMap version 27 2024-06-10T01:02:58.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.629+0000 7f7a3b8a7b80 1 -- 172.21.15.37:0/2164847171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560081c3d980 con 0x560081bbef60 2024-06-10T01:02:58.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.629+0000 7f7a0eff5700 1 -- 172.21.15.37:0/2164847171 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a24003da0 con 0x560081bbef60 2024-06-10T01:02:58.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.629+0000 7f7a0eff5700 1 -- 172.21.15.37:0/2164847171 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7a2401f5f0 con 0x560081bbef60 2024-06-10T01:02:58.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.629+0000 7f7a3b8a7b80 10 client.16491 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:02:58.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.629+0000 7f7a3b8a7b80 10 client.16491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:02:58.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.629+0000 7f7a3b8a7b80 20 client.16491 populate_metadata read hostname 'smithi037' 2024-06-10T01:02:58.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.629+0000 7f7a3b8a7b80 1 --2- 172.21.15.37:0/2164847171 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560081c3def0 0x560081c40310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:02:58.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.629+0000 7f7a3b8a7b80 4 client.16491 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:02:58.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.629+0000 7f7a3b8a7b80 1 -- 172.21.15.37:0/2164847171 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560081b12290 con 0x560081c3def0 2024-06-10T01:02:58.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.630+0000 7f7a2ad9d700 1 --2- 172.21.15.37:0/2164847171 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560081c3def0 0x560081c40310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:02:58.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.630+0000 7f7a2ad9d700 1 --2- 172.21.15.37:0/2164847171 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560081c3def0 0x560081c40310 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=490e2eca600c7792 server_cookie=50dd29870c2f6f91 in_seq=0 out_seq=0 2024-06-10T01:02:58.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.630+0000 7f7a0eff5700 10 client.16491 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:58.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.633+0000 7f7a0eff5700 1 -- 172.21.15.37:0/2164847171 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x560081b12290 con 0x560081c3def0 2024-06-10T01:02:58.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.633+0000 7f7a0eff5700 10 client.16491 handle_command_reply: tid=0 2024-06-10T01:02:58.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.633+0000 7f7a3b8a7b80 1 client.16491 shutdown 2024-06-10T01:02:58.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.633+0000 7f7a3b8a7b80 1 -- 172.21.15.37:0/2164847171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560081bbef60 msgr2=0x560081bc7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:58.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.633+0000 7f7a3b8a7b80 1 --2- 172.21.15.37:0/2164847171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560081bbef60 0x560081bc7700 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7a24007d40 tx=0x7f7a24003c30 comp rx=0 tx=0).stop 2024-06-10T01:02:58.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.633+0000 7f7a3b8a7b80 1 -- 172.21.15.37:0/2164847171 shutdown_connections 2024-06-10T01:02:58.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.633+0000 7f7a3b8a7b80 1 --2- 172.21.15.37:0/2164847171 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560081c3def0 0x560081c40310 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.633+0000 7f7a0eff5700 0 client.16491 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:02:58.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.633+0000 7f7a3b8a7b80 1 --2- 172.21.15.37:0/2164847171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560081bbef60 0x560081bc7700 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.633+0000 7f7a3b8a7b80 1 --2- 172.21.15.37:0/2164847171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560081bb8f20 0x560081bc6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.633+0000 7f7a3b8a7b80 1 --2- 172.21.15.37:0/2164847171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560081bbe640 0x560081bc71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.633+0000 7f7a3b8a7b80 1 -- 172.21.15.37:0/2164847171 >> 172.21.15.37:0/2164847171 conn(0x560081ae3cc0 msgr2=0x560081bbd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:58.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.634+0000 7f7a3b8a7b80 1 -- 172.21.15.37:0/2164847171 shutdown_connections 2024-06-10T01:02:58.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.634+0000 7f7a3b8a7b80 1 -- 172.21.15.37:0/2164847171 wait complete. 2024-06-10T01:02:58.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.634+0000 7f7a3b8a7b80 20 client.16491 trim_cache size 0 max 16384 2024-06-10T01:02:58.671 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:02:58.671 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "scrub active (4577 inodes in the stack)", 2024-06-10T01:02:58.671 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-06-10T01:02:58.671 INFO:teuthology.orchestra.run.smithi037.stdout: "59187c88-c6e1-417e-9ae1-b55b232cd94e": { 2024-06-10T01:02:58.671 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-06-10T01:02:58.671 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "59187c88-c6e1-417e-9ae1-b55b232cd94e", 2024-06-10T01:02:58.671 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-06-10T01:02:58.672 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:02:58.672 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:02:58.672 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:02:58.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.634+0000 7f7a31024700 1 -- 172.21.15.37:0/1591281219 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7a140a2d20 msgr2=0x7f7a140a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:58.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.634+0000 7f7a31024700 1 --2- 172.21.15.37:0/1591281219 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7a140a2d20 0x7f7a140a51e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00b2e0 tx=0x7f7a1c01a040 comp rx=0 tx=0).stop 2024-06-10T01:02:58.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.634+0000 7f7a31024700 1 -- 172.21.15.37:0/1591281219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a2c06ab30 msgr2=0x7f7a2c19ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:02:58.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.634+0000 7f7a31024700 1 --2- 172.21.15.37:0/1591281219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a2c06ab30 0x7f7a2c19ed90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000e9f0 tx=0x7f7a2000ed00 comp rx=0 tx=0).stop 2024-06-10T01:02:58.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.634+0000 7f7a31024700 1 -- 172.21.15.37:0/1591281219 shutdown_connections 2024-06-10T01:02:58.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.634+0000 7f7a31024700 1 --2- 172.21.15.37:0/1591281219 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7a140a2d20 0x7f7a140a51e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.634+0000 7f7a31024700 1 --2- 172.21.15.37:0/1591281219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a2c108500 0x7f7a2c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.634+0000 7f7a31024700 1 --2- 172.21.15.37:0/1591281219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a2c06a170 0x7f7a2c19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.634+0000 7f7a31024700 1 --2- 172.21.15.37:0/1591281219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a2c06ab30 0x7f7a2c19ed90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:02:58.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.634+0000 7f7a31024700 1 -- 172.21.15.37:0/1591281219 >> 172.21.15.37:0/1591281219 conn(0x7f7a2c078340 msgr2=0x7f7a2c0ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:02:58.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.635+0000 7f7a31024700 1 -- 172.21.15.37:0/1591281219 shutdown_connections 2024-06-10T01:02:58.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:02:58.635+0000 7f7a31024700 1 -- 172.21.15.37:0/1591281219 wait complete. 2024-06-10T01:03:00.342 INFO:tasks.cephfs.filesystem:scrub status for tag:59187c88-c6e1-417e-9ae1-b55b232cd94e - {'path': '/', 'tag': '59187c88-c6e1-417e-9ae1-b55b232cd94e', 'options': 'recursive,force'} 2024-06-10T01:03:00.538 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:00 smithi037 ceph-mon[32192]: pgmap v619: 129 pgs: 129 active+clean; 17 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 3.17k op/s 2024-06-10T01:03:00.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:00 smithi167 ceph-mon[36917]: pgmap v619: 129 pgs: 129 active+clean; 17 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 3.17k op/s 2024-06-10T01:03:00.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:00 smithi094 ceph-mon[36708]: pgmap v619: 129 pgs: 129 active+clean; 17 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 3.17k op/s 2024-06-10T01:03:02.516 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:02 smithi037 ceph-mon[32192]: pgmap v620: 129 pgs: 129 active+clean; 17 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 1.84k op/s 2024-06-10T01:03:02.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:02 smithi167 ceph-mon[36917]: pgmap v620: 129 pgs: 129 active+clean; 17 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 1.84k op/s 2024-06-10T01:03:02.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:02 smithi094 ceph-mon[36708]: pgmap v620: 129 pgs: 129 active+clean; 17 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 1.84k op/s 2024-06-10T01:03:02.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.636+0000 7fc0e1066700 1 -- 172.21.15.37:0/1940695436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0dc102740 msgr2=0x7fc0dc102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:02.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.636+0000 7fc0e1066700 1 --2- 172.21.15.37:0/1940695436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0dc102740 0x7fc0dc102b20 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc0099c0 tx=0x7fc0cc009cd0 comp rx=0 tx=0).stop 2024-06-10T01:03:02.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.636+0000 7fc0e1066700 1 -- 172.21.15.37:0/1940695436 shutdown_connections 2024-06-10T01:03:02.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.636+0000 7fc0e1066700 1 --2- 172.21.15.37:0/1940695436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0dc107840 0x7fc0dc10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:02.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.636+0000 7fc0e1066700 1 --2- 172.21.15.37:0/1940695436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0dc102740 0x7fc0dc102b20 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:02.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.636+0000 7fc0e1066700 1 --2- 172.21.15.37:0/1940695436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0dc1030f0 0x7fc0dc107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:02.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.636+0000 7fc0e1066700 1 -- 172.21.15.37:0/1940695436 >> 172.21.15.37:0/1940695436 conn(0x7fc0dc0795f0 msgr2=0x7fc0dc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:02.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.636+0000 7fc0e1066700 1 -- 172.21.15.37:0/1940695436 shutdown_connections 2024-06-10T01:03:02.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.636+0000 7fc0e1066700 1 -- 172.21.15.37:0/1940695436 wait complete. 2024-06-10T01:03:02.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.637+0000 7fc0e1066700 1 Processor -- start 2024-06-10T01:03:02.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.637+0000 7fc0e1066700 1 -- start start 2024-06-10T01:03:02.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.637+0000 7fc0e1066700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0dc102740 0x7fc0dc19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:02.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.637+0000 7fc0e1066700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0dc1030f0 0x7fc0dc19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:02.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.637+0000 7fc0e1066700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0dc107840 0x7fc0dc198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:02.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.637+0000 7fc0e1066700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc0dc112780 con 0x7fc0dc1030f0 2024-06-10T01:03:02.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.637+0000 7fc0e1066700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc0dc112600 con 0x7fc0dc107840 2024-06-10T01:03:02.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.637+0000 7fc0e1066700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc0dc112900 con 0x7fc0dc102740 2024-06-10T01:03:02.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.637+0000 7fc0da59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0dc1030f0 0x7fc0dc19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:02.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.637+0000 7fc0dad9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0dc102740 0x7fc0dc19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:02.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.637+0000 7fc0da59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0dc1030f0 0x7fc0dc19ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57912/0 (socket says 172.21.15.37:57912) 2024-06-10T01:03:02.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.637+0000 7fc0dad9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0dc102740 0x7fc0dc19e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:37998/0 (socket says 172.21.15.37:37998) 2024-06-10T01:03:02.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.637+0000 7fc0da59c700 1 -- 172.21.15.37:0/3516255224 learned_addr learned my addr 172.21.15.37:0/3516255224 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:02.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.638+0000 7fc0db59e700 1 --2- 172.21.15.37:0/3516255224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0dc107840 0x7fc0dc198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:02.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.638+0000 7fc0dad9d700 1 -- 172.21.15.37:0/3516255224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0dc107840 msgr2=0x7fc0dc198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:02.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.638+0000 7fc0dad9d700 1 --2- 172.21.15.37:0/3516255224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0dc107840 0x7fc0dc198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:02.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.638+0000 7fc0dad9d700 1 -- 172.21.15.37:0/3516255224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0dc1030f0 msgr2=0x7fc0dc19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:02.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.638+0000 7fc0dad9d700 1 --2- 172.21.15.37:0/3516255224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0dc1030f0 0x7fc0dc19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:02.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.638+0000 7fc0dad9d700 1 -- 172.21.15.37:0/3516255224 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0dc199160 con 0x7fc0dc102740 2024-06-10T01:03:02.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.638+0000 7fc0da59c700 1 --2- 172.21.15.37:0/3516255224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0dc1030f0 0x7fc0dc19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:03:02.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.638+0000 7fc0dad9d700 1 --2- 172.21.15.37:0/3516255224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0dc102740 0x7fc0dc19e840 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc0098e0 tx=0x7fc0cc00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:02.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.638+0000 7fc0c3fff700 1 -- 172.21.15.37:0/3516255224 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc027070 con 0x7fc0dc102740 2024-06-10T01:03:02.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.638+0000 7fc0e1066700 1 -- 172.21.15.37:0/3516255224 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0dc1993f0 con 0x7fc0dc102740 2024-06-10T01:03:02.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.638+0000 7fc0c3fff700 1 -- 172.21.15.37:0/3516255224 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0cc003680 con 0x7fc0dc102740 2024-06-10T01:03:02.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.638+0000 7fc0c3fff700 1 -- 172.21.15.37:0/3516255224 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc0173f0 con 0x7fc0dc102740 2024-06-10T01:03:02.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.638+0000 7fc0e1066700 1 -- 172.21.15.37:0/3516255224 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0dc1a5710 con 0x7fc0dc102740 2024-06-10T01:03:02.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.639+0000 7fc0e1066700 1 -- 172.21.15.37:0/3516255224 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0dc069180 con 0x7fc0dc102740 2024-06-10T01:03:02.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.641+0000 7fc0c3fff700 1 -- 172.21.15.37:0/3516255224 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc0cc02a070 con 0x7fc0dc102740 2024-06-10T01:03:02.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.641+0000 7fc0c3fff700 1 --2- 172.21.15.37:0/3516255224 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc0c40a2c20 0x7fc0c40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:02.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.641+0000 7fc0c3fff700 1 -- 172.21.15.37:0/3516255224 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7fc0cc0d0530 con 0x7fc0dc102740 2024-06-10T01:03:02.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.641+0000 7fc0da59c700 1 --2- 172.21.15.37:0/3516255224 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc0c40a2c20 0x7fc0c40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:02.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.642+0000 7fc0da59c700 1 --2- 172.21.15.37:0/3516255224 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc0c40a2c20 0x7fc0c40a50e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc102060 tx=0x7fc0d0007480 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:02.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.645+0000 7fc0c3fff700 1 -- 172.21.15.37:0/3516255224 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc0cc013070 con 0x7fc0dc102740 2024-06-10T01:03:02.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.904+0000 7fc0e1066700 1 -- 172.21.15.37:0/3516255224 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0dc199f30 con 0x7fc0dc102740 2024-06-10T01:03:02.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.905+0000 7fc0c3fff700 1 -- 172.21.15.37:0/3516255224 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11867 (secure 0 0 0) 0x7fc0cc09d110 con 0x7fc0dc102740 2024-06-10T01:03:02.903 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:03:02.903 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T00:56:30.291171+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/3066042611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3066042611},{"type":"v1","addr":"172.21.15.167:6835","nonce":3066042611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24451,24458,24472]},"id":1}]} 2024-06-10T01:03:02.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.907+0000 7fc0e1066700 1 -- 172.21.15.37:0/3516255224 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc0c40a2c20 msgr2=0x7fc0c40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:02.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.907+0000 7fc0e1066700 1 --2- 172.21.15.37:0/3516255224 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc0c40a2c20 0x7fc0c40a50e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc102060 tx=0x7fc0d0007480 comp rx=0 tx=0).stop 2024-06-10T01:03:02.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.908+0000 7fc0e1066700 1 -- 172.21.15.37:0/3516255224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0dc102740 msgr2=0x7fc0dc19e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:02.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.908+0000 7fc0e1066700 1 --2- 172.21.15.37:0/3516255224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0dc102740 0x7fc0dc19e840 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc0098e0 tx=0x7fc0cc00fa00 comp rx=0 tx=0).stop 2024-06-10T01:03:02.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.908+0000 7fc0e1066700 1 -- 172.21.15.37:0/3516255224 shutdown_connections 2024-06-10T01:03:02.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.908+0000 7fc0e1066700 1 --2- 172.21.15.37:0/3516255224 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc0c40a2c20 0x7fc0c40a50e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:02.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.908+0000 7fc0e1066700 1 --2- 172.21.15.37:0/3516255224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0dc107840 0x7fc0dc198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:02.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.908+0000 7fc0e1066700 1 --2- 172.21.15.37:0/3516255224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0dc102740 0x7fc0dc19e840 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:02.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.908+0000 7fc0e1066700 1 --2- 172.21.15.37:0/3516255224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0dc1030f0 0x7fc0dc19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:02.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.908+0000 7fc0e1066700 1 -- 172.21.15.37:0/3516255224 >> 172.21.15.37:0/3516255224 conn(0x7fc0dc0795f0 msgr2=0x7fc0dc100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:02.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.908+0000 7fc0e1066700 1 -- 172.21.15.37:0/3516255224 shutdown_connections 2024-06-10T01:03:02.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:02.908+0000 7fc0e1066700 1 -- 172.21.15.37:0/3516255224 wait complete. 2024-06-10T01:03:02.906 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-06-10T01:03:03.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:03 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3516255224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:03:03.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:03 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3516255224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:03:03.639 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:03 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3516255224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:03:03.668 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24472 quiesce db / --timeout 90 --expiration 149.45 --await-for 45.0' 2024-06-10T01:03:03.669 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24472 quiesce db / --timeout 90 --expiration 149.45 --await-for 45.0 2024-06-10T01:03:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.839+0000 7f5ed0dce640 1 -- 172.21.15.37:0/2152761978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ecc100900 msgr2=0x7f5ecc100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.839+0000 7f5ed0dce640 1 --2- 172.21.15.37:0/2152761978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ecc100900 0x7f5ecc100d20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc009900 tx=0x7f5ebc02a410 comp rx=0 tx=0).stop 2024-06-10T01:03:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.839+0000 7f5ed0dce640 1 -- 172.21.15.37:0/2152761978 shutdown_connections 2024-06-10T01:03:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.839+0000 7f5ed0dce640 1 --2- 172.21.15.37:0/2152761978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ecc1024b0 0x7f5ecc108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.839+0000 7f5ed0dce640 1 --2- 172.21.15.37:0/2152761978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ecc101af0 0x7f5ecc101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.839+0000 7f5ed0dce640 1 --2- 172.21.15.37:0/2152761978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ecc100900 0x7f5ecc100d20 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.839+0000 7f5ed0dce640 1 -- 172.21.15.37:0/2152761978 >> 172.21.15.37:0/2152761978 conn(0x7f5ecc0fbe70 msgr2=0x7f5ecc0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.839+0000 7f5ed0dce640 1 -- 172.21.15.37:0/2152761978 shutdown_connections 2024-06-10T01:03:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.839+0000 7f5ed0dce640 1 -- 172.21.15.37:0/2152761978 wait complete. 2024-06-10T01:03:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.840+0000 7f5ed0dce640 1 Processor -- start 2024-06-10T01:03:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.840+0000 7f5ed0dce640 1 -- start start 2024-06-10T01:03:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.840+0000 7f5ed0dce640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ecc100900 0x7f5ecc197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.840+0000 7f5ed0dce640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ecc101af0 0x7f5ecc198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.840+0000 7f5ed0dce640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ecc1024b0 0x7f5ecc19f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.840+0000 7f5ed0dce640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc10c570 con 0x7f5ecc101af0 2024-06-10T01:03:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.840+0000 7f5ed0dce640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc10c3f0 con 0x7f5ecc1024b0 2024-06-10T01:03:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.840+0000 7f5ed0dce640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc10c6f0 con 0x7f5ecc100900 2024-06-10T01:03:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.840+0000 7f5ec9d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ecc101af0 0x7f5ecc198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.840+0000 7f5eca59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ecc100900 0x7f5ecc197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.840+0000 7f5eca59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ecc100900 0x7f5ecc197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:38002/0 (socket says 172.21.15.37:38002) 2024-06-10T01:03:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.840+0000 7f5eca59c640 1 -- 172.21.15.37:0/992728224 learned_addr learned my addr 172.21.15.37:0/992728224 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.840+0000 7f5ecad9d640 1 --2- 172.21.15.37:0/992728224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ecc1024b0 0x7f5ecc19f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.840+0000 7f5eca59c640 1 -- 172.21.15.37:0/992728224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ecc1024b0 msgr2=0x7f5ecc19f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.840+0000 7f5eca59c640 1 --2- 172.21.15.37:0/992728224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ecc1024b0 0x7f5ecc19f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.840+0000 7f5eca59c640 1 -- 172.21.15.37:0/992728224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ecc101af0 msgr2=0x7f5ecc198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.840+0000 7f5eca59c640 1 --2- 172.21.15.37:0/992728224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ecc101af0 0x7f5ecc198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.840+0000 7f5eca59c640 1 -- 172.21.15.37:0/992728224 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ecc19f900 con 0x7f5ecc100900 2024-06-10T01:03:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.840+0000 7f5ecad9d640 1 --2- 172.21.15.37:0/992728224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ecc1024b0 0x7f5ecc19f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:03:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.840+0000 7f5eca59c640 1 --2- 172.21.15.37:0/992728224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ecc100900 0x7f5ecc197cf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc031610 tx=0x7f5ebc004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.841+0000 7f5ebb7fe640 1 -- 172.21.15.37:0/992728224 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc03f030 con 0x7f5ecc100900 2024-06-10T01:03:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.841+0000 7f5ed0dce640 1 -- 172.21.15.37:0/992728224 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ecc19fb90 con 0x7f5ecc100900 2024-06-10T01:03:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.841+0000 7f5ebb7fe640 1 -- 172.21.15.37:0/992728224 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ebc003ad0 con 0x7f5ecc100900 2024-06-10T01:03:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.841+0000 7f5ebb7fe640 1 -- 172.21.15.37:0/992728224 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc035070 con 0x7f5ecc100900 2024-06-10T01:03:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.841+0000 7f5ed0dce640 1 -- 172.21.15.37:0/992728224 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ecc1a00d0 con 0x7f5ecc100900 2024-06-10T01:03:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.843+0000 7f5ebb7fe640 1 -- 172.21.15.37:0/992728224 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5ebc042030 con 0x7f5ecc100900 2024-06-10T01:03:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.843+0000 7f5ed3bb8740 1 Processor -- start 2024-06-10T01:03:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.843+0000 7f5ed3bb8740 1 -- start start 2024-06-10T01:03:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.844+0000 7f5ed3bb8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c1c83e420 0x562c1c83e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.844+0000 7f5ed3bb8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c1c83ee10 0x562c1c843910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.844+0000 7f5ed3bb8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c1c843f80 0x562c1c846410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.844+0000 7f5ed3bb8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562c1c79d4f0 con 0x562c1c843f80 2024-06-10T01:03:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.844+0000 7f5ed3bb8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562c1c73b870 con 0x562c1c83ee10 2024-06-10T01:03:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.844+0000 7f5ed3bb8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562c1c7994b0 con 0x562c1c83e420 2024-06-10T01:03:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.844+0000 7f5ecad9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c1c83e420 0x562c1c83e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.844+0000 7f5ecad9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c1c83e420 0x562c1c83e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:38010/0 (socket says 172.21.15.37:38010) 2024-06-10T01:03:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.844+0000 7f5ec9d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c1c843f80 0x562c1c846410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.844+0000 7f5ecad9d640 1 -- 172.21.15.37:0/1428398481 learned_addr learned my addr 172.21.15.37:0/1428398481 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.844+0000 7f5eca59c640 1 --2- 172.21.15.37:0/1428398481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c1c83ee10 0x562c1c843910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.844+0000 7f5ec9d9b640 1 -- 172.21.15.37:0/1428398481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c1c83e420 msgr2=0x562c1c83e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.844+0000 7f5ec9d9b640 1 --2- 172.21.15.37:0/1428398481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c1c83e420 0x562c1c83e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.844+0000 7f5ec9d9b640 1 -- 172.21.15.37:0/1428398481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c1c83ee10 msgr2=0x562c1c843910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:03.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.844+0000 7f5ec9d9b640 1 --2- 172.21.15.37:0/1428398481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c1c83ee10 0x562c1c843910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.844+0000 7f5ebb7fe640 1 --2- 172.21.15.37:0/992728224 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5eb40a2c30 0x7f5eb40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.844+0000 7f5ec9d9b640 1 -- 172.21.15.37:0/1428398481 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c1c7579b0 con 0x562c1c843f80 2024-06-10T01:03:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.844+0000 7f5ecad9d640 1 --2- 172.21.15.37:0/1428398481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c1c83e420 0x562c1c83e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:03:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.844+0000 7f5ecad9d640 1 --2- 172.21.15.37:0/992728224 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5eb40a2c30 0x7f5eb40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.845+0000 7f5ec9d9b640 1 --2- 172.21.15.37:0/1428398481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c1c843f80 0x562c1c846410 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009c00 tx=0x7f5ec0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73694e1e3f6beef2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.845+0000 7f5ebb7fe640 1 -- 172.21.15.37:0/992728224 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f5ebc0efc10 con 0x7f5ecc100900 2024-06-10T01:03:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.845+0000 7f5eb8ff9640 1 -- 172.21.15.37:0/1428398481 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0004230 con 0x562c1c843f80 2024-06-10T01:03:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.845+0000 7f5eb8ff9640 1 -- 172.21.15.37:0/1428398481 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ec00043d0 con 0x562c1c843f80 2024-06-10T01:03:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.845+0000 7f5eb8ff9640 1 -- 172.21.15.37:0/1428398481 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0004dd0 con 0x562c1c843f80 2024-06-10T01:03:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.846+0000 7f5ed3bb8740 1 -- 172.21.15.37:0/1428398481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c1c843f80 msgr2=0x562c1c846410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.846+0000 7f5ed3bb8740 1 --2- 172.21.15.37:0/1428398481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c1c843f80 0x562c1c846410 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009c00 tx=0x7f5ec0011810 comp rx=0 tx=0).stop 2024-06-10T01:03:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.846+0000 7f5ecad9d640 1 --2- 172.21.15.37:0/992728224 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5eb40a2c30 0x7f5eb40a50f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x562c1c83f670 tx=0x7f5ec4009790 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.846+0000 7f5ed3bb8740 1 -- 172.21.15.37:0/1428398481 shutdown_connections 2024-06-10T01:03:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.846+0000 7f5ed3bb8740 1 --2- 172.21.15.37:0/1428398481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c1c843f80 0x562c1c846410 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.846+0000 7f5ed3bb8740 1 --2- 172.21.15.37:0/1428398481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c1c83ee10 0x562c1c843910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.846+0000 7f5ed3bb8740 1 --2- 172.21.15.37:0/1428398481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c1c83e420 0x562c1c83e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.846+0000 7f5ed3bb8740 1 -- 172.21.15.37:0/1428398481 >> 172.21.15.37:0/1428398481 conn(0x562c1c835680 msgr2=0x562c1c835a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.846+0000 7f5ed3bb8740 1 -- 172.21.15.37:0/1428398481 shutdown_connections 2024-06-10T01:03:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.846+0000 7f5ed3bb8740 1 -- 172.21.15.37:0/1428398481 wait complete. 2024-06-10T01:03:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.846+0000 7f5ed3bb8740 1 -- start start 2024-06-10T01:03:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.846+0000 7f5ed3bb8740 1 Processor -- start 2024-06-10T01:03:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.846+0000 7f5ed3bb8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c1c83e420 0x562c1c848660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.847+0000 7f5ed3bb8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c1c83ee10 0x562c1c848ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.847+0000 7f5ed3bb8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c1c843f80 0x562c1c84fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.847+0000 7f5ed3bb8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562c1c846f30 con 0x562c1c843f80 2024-06-10T01:03:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.847+0000 7f5ed3bb8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562c1c726ae0 con 0x562c1c83e420 2024-06-10T01:03:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.847+0000 7f5ed3bb8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562c1c8470b0 con 0x562c1c83ee10 2024-06-10T01:03:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.847+0000 7f5ed3bb8740 10 client.0 _ll_register_callbacks cb 0x562c1c75f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:03:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.847+0000 7f5ecad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c1c843f80 0x562c1c84fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.847+0000 7f5ec9d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c1c83ee10 0x562c1c848ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.847+0000 7f5ecad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c1c843f80 0x562c1c84fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57950/0 (socket says 172.21.15.37:57950) 2024-06-10T01:03:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.847+0000 7f5ecad9d640 1 -- 172.21.15.37:0/4134483946 learned_addr learned my addr 172.21.15.37:0/4134483946 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.847+0000 7f5eca59c640 1 --2- 172.21.15.37:0/4134483946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c1c83e420 0x562c1c848660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.847+0000 7f5ecad9d640 1 -- 172.21.15.37:0/4134483946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c1c83ee10 msgr2=0x562c1c848ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.847+0000 7f5ecad9d640 1 --2- 172.21.15.37:0/4134483946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c1c83ee10 0x562c1c848ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.847+0000 7f5ecad9d640 1 -- 172.21.15.37:0/4134483946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c1c83e420 msgr2=0x562c1c848660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.847+0000 7f5ecad9d640 1 --2- 172.21.15.37:0/4134483946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c1c83e420 0x562c1c848660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.847+0000 7f5ecad9d640 1 -- 172.21.15.37:0/4134483946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c1c4b1210 con 0x562c1c843f80 2024-06-10T01:03:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.847+0000 7f5ecad9d640 1 --2- 172.21.15.37:0/4134483946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c1c843f80 0x562c1c84fbe0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4010940 tx=0x7f5ec4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ea22e3738c4984 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.847+0000 7f5eb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:03:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.847+0000 7f5eb1ffb640 1 -- 172.21.15.37:0/4134483946 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec400fb40 con 0x562c1c843f80 2024-06-10T01:03:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.848+0000 7f5eb1ffb640 1 -- 172.21.15.37:0/4134483946 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f5ec400fce0 con 0x562c1c843f80 2024-06-10T01:03:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.848+0000 7f5ed3bb8740 1 -- 172.21.15.37:0/4134483946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ec00043d0 con 0x562c1c843f80 2024-06-10T01:03:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.848+0000 7f5eb1ffb640 1 -- 172.21.15.37:0/4134483946 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5ec4028470 con 0x562c1c843f80 2024-06-10T01:03:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.848+0000 7f5ed3bb8740 10 client.16509 fetch_fsmap learned FSMap version 27 2024-06-10T01:03:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.848+0000 7f5ed3bb8740 1 -- 172.21.15.37:0/4134483946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562c1c57d620 con 0x562c1c843f80 2024-06-10T01:03:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.848+0000 7f5eb1ffb640 1 -- 172.21.15.37:0/4134483946 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ec00043d0 con 0x562c1c843f80 2024-06-10T01:03:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.848+0000 7f5eb1ffb640 1 -- 172.21.15.37:0/4134483946 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5ec401e720 con 0x562c1c843f80 2024-06-10T01:03:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.848+0000 7f5ed3bb8740 10 client.16509 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:03:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.848+0000 7f5ed3bb8740 10 client.16509 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:03:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.848+0000 7f5ed3bb8740 20 client.16509 populate_metadata read hostname 'smithi037' 2024-06-10T01:03:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.848+0000 7f5ed3bb8740 1 --2- 172.21.15.37:0/4134483946 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x562c1c854590 0x562c1c856a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.848+0000 7f5ed3bb8740 4 client.16509 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:03:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.848+0000 7f5ed3bb8740 1 -- 172.21.15.37:0/4134483946 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c1c7b0f10 con 0x562c1c854590 2024-06-10T01:03:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.848+0000 7f5eca59c640 1 --2- 172.21.15.37:0/4134483946 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x562c1c854590 0x562c1c856a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.849+0000 7f5eb1ffb640 10 client.16509 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:03:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.849+0000 7f5eca59c640 1 --2- 172.21.15.37:0/4134483946 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x562c1c854590 0x562c1c856a20 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e72e7344186cdced server_cookie=716929a898b900c8 in_seq=0 out_seq=0 2024-06-10T01:03:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.852+0000 7f5eb1ffb640 1 -- 172.21.15.37:0/4134483946 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562c1c7b0f10 con 0x562c1c854590 2024-06-10T01:03:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.852+0000 7f5eb1ffb640 10 client.16509 handle_command_reply: tid=0 2024-06-10T01:03:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.852+0000 7f5ed3bb8740 1 client.16509 shutdown 2024-06-10T01:03:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.852+0000 7f5ed3bb8740 1 -- 172.21.15.37:0/4134483946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c1c843f80 msgr2=0x562c1c84fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.852+0000 7f5ed3bb8740 1 --2- 172.21.15.37:0/4134483946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c1c843f80 0x562c1c84fbe0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4010940 tx=0x7f5ec4004a40 comp rx=0 tx=0).stop 2024-06-10T01:03:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.852+0000 7f5ed3bb8740 1 -- 172.21.15.37:0/4134483946 shutdown_connections 2024-06-10T01:03:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.852+0000 7f5ed3bb8740 1 --2- 172.21.15.37:0/4134483946 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x562c1c854590 0x562c1c856a20 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.852+0000 7f5ed3bb8740 1 --2- 172.21.15.37:0/4134483946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c1c843f80 0x562c1c84fbe0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.852+0000 7f5ed3bb8740 1 --2- 172.21.15.37:0/4134483946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c1c83ee10 0x562c1c848ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.852+0000 7f5ed3bb8740 1 --2- 172.21.15.37:0/4134483946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c1c83e420 0x562c1c848660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.852+0000 7f5eb1ffb640 0 client.16509 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:03:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.852+0000 7f5ed3bb8740 1 -- 172.21.15.37:0/4134483946 >> 172.21.15.37:0/4134483946 conn(0x562c1c835680 msgr2=0x562c1c8347e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.852+0000 7f5ed3bb8740 1 -- 172.21.15.37:0/4134483946 shutdown_connections 2024-06-10T01:03:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.852+0000 7f5ed3bb8740 1 -- 172.21.15.37:0/4134483946 wait complete. 2024-06-10T01:03:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.852+0000 7f5ed3bb8740 20 client.16509 trim_cache size 0 max 16384 2024-06-10T01:03:03.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.869+0000 7f5ed3bb8740 1 Processor -- start 2024-06-10T01:03:03.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.869+0000 7f5ed3bb8740 1 -- start start 2024-06-10T01:03:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.869+0000 7f5ed3bb8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c1c83e420 0x562c1c840030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.869+0000 7f5ed3bb8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c1c83ee10 0x562c1c84fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.869+0000 7f5ed3bb8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c1c843f80 0x562c1c840570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.869+0000 7f5ed3bb8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc1aade0 con 0x562c1c83e420 2024-06-10T01:03:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.869+0000 7f5ed3bb8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc1a86b0 con 0x562c1c843f80 2024-06-10T01:03:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.869+0000 7f5ed3bb8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc1a6500 con 0x562c1c83ee10 2024-06-10T01:03:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.869+0000 7f5eca59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c1c83e420 0x562c1c840030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.869+0000 7f5eca59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c1c83e420 0x562c1c840030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57952/0 (socket says 172.21.15.37:57952) 2024-06-10T01:03:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.869+0000 7f5eca59c640 1 -- 172.21.15.37:0/3947687210 learned_addr learned my addr 172.21.15.37:0/3947687210 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.869+0000 7f5ec9d9b640 1 --2- 172.21.15.37:0/3947687210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c1c83ee10 0x562c1c84fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.869+0000 7f5ecad9d640 1 --2- 172.21.15.37:0/3947687210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c1c843f80 0x562c1c840570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.869+0000 7f5eca59c640 1 -- 172.21.15.37:0/3947687210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c1c83ee10 msgr2=0x562c1c84fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.869+0000 7f5eca59c640 1 --2- 172.21.15.37:0/3947687210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c1c83ee10 0x562c1c84fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.869+0000 7f5eca59c640 1 -- 172.21.15.37:0/3947687210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c1c843f80 msgr2=0x562c1c840570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:03.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.869+0000 7f5eca59c640 1 --2- 172.21.15.37:0/3947687210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c1c843f80 0x562c1c840570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.869+0000 7f5eca59c640 1 -- 172.21.15.37:0/3947687210 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c1c8354b0 con 0x562c1c83e420 2024-06-10T01:03:03.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.869+0000 7f5ecad9d640 1 --2- 172.21.15.37:0/3947687210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c1c843f80 0x562c1c840570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:03:03.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.870+0000 7f5eca59c640 1 --2- 172.21.15.37:0/3947687210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c1c83e420 0x562c1c840030 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0314e0 tx=0x7f5ebc0dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d4590646f1cb5c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:03.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.870+0000 7f5eb3fff640 1 -- 172.21.15.37:0/3947687210 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc049040 con 0x562c1c83e420 2024-06-10T01:03:03.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.870+0000 7f5eb3fff640 1 -- 172.21.15.37:0/3947687210 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ebc0bc760 con 0x562c1c83e420 2024-06-10T01:03:03.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.870+0000 7f5eb3fff640 1 -- 172.21.15.37:0/3947687210 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc0ed410 con 0x562c1c83e420 2024-06-10T01:03:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.871+0000 7f5ed3bb8740 1 -- 172.21.15.37:0/3947687210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c1c83e420 msgr2=0x562c1c840030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.871+0000 7f5ed3bb8740 1 --2- 172.21.15.37:0/3947687210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c1c83e420 0x562c1c840030 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0314e0 tx=0x7f5ebc0dc040 comp rx=0 tx=0).stop 2024-06-10T01:03:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.871+0000 7f5ed3bb8740 1 -- 172.21.15.37:0/3947687210 shutdown_connections 2024-06-10T01:03:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.871+0000 7f5ed3bb8740 1 --2- 172.21.15.37:0/3947687210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c1c843f80 0x562c1c840570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.871+0000 7f5ed3bb8740 1 --2- 172.21.15.37:0/3947687210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c1c83ee10 0x562c1c84fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.871+0000 7f5ed3bb8740 1 --2- 172.21.15.37:0/3947687210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c1c83e420 0x562c1c840030 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.871+0000 7f5ed3bb8740 1 -- 172.21.15.37:0/3947687210 >> 172.21.15.37:0/3947687210 conn(0x562c1c835680 msgr2=0x562c1c843690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.871+0000 7f5ed3bb8740 1 -- 172.21.15.37:0/3947687210 shutdown_connections 2024-06-10T01:03:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.871+0000 7f5ed3bb8740 1 -- 172.21.15.37:0/3947687210 wait complete. 2024-06-10T01:03:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.871+0000 7f5ed3bb8740 1 -- start start 2024-06-10T01:03:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.871+0000 7f5ed3bb8740 1 Processor -- start 2024-06-10T01:03:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.871+0000 7f5ed3bb8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c1c83e420 0x562c1c84cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.871+0000 7f5ed3bb8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c1c83ee10 0x562c1c84d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.871+0000 7f5ed3bb8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c1c843f80 0x562c1c8bd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.871+0000 7f5ed3bb8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc1a6b70 con 0x562c1c83e420 2024-06-10T01:03:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.871+0000 7f5ed3bb8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc1a7fe0 con 0x562c1c843f80 2024-06-10T01:03:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.871+0000 7f5ed3bb8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc1a7240 con 0x562c1c83ee10 2024-06-10T01:03:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.871+0000 7f5ed3bb8740 10 client.0 _ll_register_callbacks cb 0x7f5ec4009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:03:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.871+0000 7f5eca59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c1c83e420 0x562c1c84cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.871+0000 7f5ec9d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c1c83ee10 0x562c1c84d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.871+0000 7f5eca59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c1c83e420 0x562c1c84cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57954/0 (socket says 172.21.15.37:57954) 2024-06-10T01:03:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.871+0000 7f5ec9d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c1c83ee10 0x562c1c84d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:38052/0 (socket says 172.21.15.37:38052) 2024-06-10T01:03:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.871+0000 7f5eca59c640 1 -- 172.21.15.37:0/365067781 learned_addr learned my addr 172.21.15.37:0/365067781 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.872+0000 7f5ecad9d640 1 --2- 172.21.15.37:0/365067781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c1c843f80 0x562c1c8bd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.872+0000 7f5eca59c640 1 -- 172.21.15.37:0/365067781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c1c83ee10 msgr2=0x562c1c84d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.872+0000 7f5eca59c640 1 --2- 172.21.15.37:0/365067781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c1c83ee10 0x562c1c84d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.872+0000 7f5eca59c640 1 -- 172.21.15.37:0/365067781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c1c843f80 msgr2=0x562c1c8bd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.872+0000 7f5eca59c640 1 --2- 172.21.15.37:0/365067781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c1c843f80 0x562c1c8bd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.872+0000 7f5eca59c640 1 -- 172.21.15.37:0/365067781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c1c8bd830 con 0x562c1c83e420 2024-06-10T01:03:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.872+0000 7f5ec9d9b640 1 --2- 172.21.15.37:0/365067781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c1c83ee10 0x562c1c84d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:03:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.872+0000 7f5ecad9d640 1 --2- 172.21.15.37:0/365067781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c1c843f80 0x562c1c8bd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:03:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.872+0000 7f5eca59c640 1 --2- 172.21.15.37:0/365067781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c1c83e420 0x562c1c84cb00 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc03d040 tx=0x7f5ebc0c4e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a1b74f578d12b5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.872+0000 7f5eb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:03:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.872+0000 7f5eb1ffb640 1 -- 172.21.15.37:0/365067781 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc049040 con 0x562c1c83e420 2024-06-10T01:03:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.872+0000 7f5eb1ffb640 1 -- 172.21.15.37:0/365067781 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f5ebc0c00f0 con 0x562c1c83e420 2024-06-10T01:03:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.872+0000 7f5ed3bb8740 1 -- 172.21.15.37:0/365067781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ebc0bc760 con 0x562c1c83e420 2024-06-10T01:03:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.872+0000 7f5eb1ffb640 1 -- 172.21.15.37:0/365067781 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5ebc0ba1a0 con 0x562c1c83e420 2024-06-10T01:03:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.872+0000 7f5ed3bb8740 10 client.16515 fetch_fsmap learned FSMap version 27 2024-06-10T01:03:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.872+0000 7f5ed3bb8740 1 -- 172.21.15.37:0/365067781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562c1c8bea70 con 0x562c1c83e420 2024-06-10T01:03:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.873+0000 7f5eb1ffb640 1 -- 172.21.15.37:0/365067781 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ebc0bc760 con 0x562c1c83e420 2024-06-10T01:03:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.873+0000 7f5eb1ffb640 1 -- 172.21.15.37:0/365067781 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5ebc03b460 con 0x562c1c83e420 2024-06-10T01:03:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.873+0000 7f5ed3bb8740 10 client.16515 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:03:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.873+0000 7f5ed3bb8740 10 client.16515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:03:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.873+0000 7f5ed3bb8740 20 client.16515 populate_metadata read hostname 'smithi037' 2024-06-10T01:03:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.873+0000 7f5ed3bb8740 1 --2- 172.21.15.37:0/365067781 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x562c1c854590 0x562c1c8c1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.873+0000 7f5ed3bb8740 4 client.16515 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:03:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.873+0000 7f5ed3bb8740 1 -- 172.21.15.37:0/365067781 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562c1c7aac20 con 0x562c1c854590 2024-06-10T01:03:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.873+0000 7f5ec9d9b640 1 --2- 172.21.15.37:0/365067781 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x562c1c854590 0x562c1c8c1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.873+0000 7f5eb1ffb640 10 client.16515 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:03:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:03.873+0000 7f5ec9d9b640 1 --2- 172.21.15.37:0/365067781 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x562c1c854590 0x562c1c8c1410 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6b6a5a74794681f server_cookie=c911d2acbdf88a19 in_seq=0 out_seq=0 2024-06-10T01:03:03.934 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:04.327 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:04 smithi037 ceph-mon[32192]: pgmap v621: 129 pgs: 129 active+clean; 21 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 1.32k op/s 2024-06-10T01:03:04.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:04 smithi167 ceph-mon[36917]: pgmap v621: 129 pgs: 129 active+clean; 21 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 1.32k op/s 2024-06-10T01:03:04.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:04 smithi094 ceph-mon[36708]: pgmap v621: 129 pgs: 129 active+clean; 21 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 1.32k op/s 2024-06-10T01:03:05.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.922+0000 7f962ada5700 1 -- 172.21.15.37:0/1472494950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f962410a930 msgr2=0x7f962410de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:05.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.922+0000 7f962ada5700 1 --2- 172.21.15.37:0/1472494950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f962410a930 0x7f962410de00 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f961c00b0a0 tx=0x7f961c00b3b0 comp rx=0 tx=0).stop 2024-06-10T01:03:05.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.923+0000 7f962ada5700 1 -- 172.21.15.37:0/1472494950 shutdown_connections 2024-06-10T01:03:05.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.923+0000 7f962ada5700 1 --2- 172.21.15.37:0/1472494950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f962410e4e0 0x7f96241160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:05.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.923+0000 7f962ada5700 1 --2- 172.21.15.37:0/1472494950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9624109f80 0x7f962410a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:05.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.923+0000 7f962ada5700 1 --2- 172.21.15.37:0/1472494950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f962410a930 0x7f962410de00 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:05.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.923+0000 7f962ada5700 1 -- 172.21.15.37:0/1472494950 >> 172.21.15.37:0/1472494950 conn(0x7f962406d3e0 msgr2=0x7f962406d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:05.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.924+0000 7f962ada5700 1 -- 172.21.15.37:0/1472494950 shutdown_connections 2024-06-10T01:03:05.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.925+0000 7f962ada5700 1 -- 172.21.15.37:0/1472494950 wait complete. 2024-06-10T01:03:05.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.926+0000 7f962ada5700 1 Processor -- start 2024-06-10T01:03:05.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.926+0000 7f962ada5700 1 -- start start 2024-06-10T01:03:05.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.926+0000 7f962ada5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9624109f80 0x7f96241a1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:05.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.926+0000 7f962ada5700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f962410a930 0x7f96241a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:05.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.926+0000 7f962ada5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f962410e4e0 0x7f96241a5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:05.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.926+0000 7f962ada5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f962411af50 con 0x7f962410e4e0 2024-06-10T01:03:05.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.926+0000 7f962ada5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f962411add0 con 0x7f9624109f80 2024-06-10T01:03:05.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.926+0000 7f962ada5700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f962411b0d0 con 0x7f962410a930 2024-06-10T01:03:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.928+0000 7f9629da3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9624109f80 0x7f96241a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.928+0000 7f96295a2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f962410a930 0x7f96241a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.928+0000 7f9629da3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9624109f80 0x7f96241a1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:43066/0 (socket says 172.21.15.37:43066) 2024-06-10T01:03:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.928+0000 7f9629da3700 1 -- 172.21.15.37:0/1843600495 learned_addr learned my addr 172.21.15.37:0/1843600495 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.928+0000 7f962a5a4700 1 --2- 172.21.15.37:0/1843600495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f962410e4e0 0x7f96241a5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.928+0000 7f9629da3700 1 -- 172.21.15.37:0/1843600495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f962410a930 msgr2=0x7f96241a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.928+0000 7f9629da3700 1 --2- 172.21.15.37:0/1843600495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f962410a930 0x7f96241a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.928+0000 7f9629da3700 1 -- 172.21.15.37:0/1843600495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f962410e4e0 msgr2=0x7f96241a5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.928+0000 7f9629da3700 1 --2- 172.21.15.37:0/1843600495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f962410e4e0 0x7f96241a5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.928+0000 7f9629da3700 1 -- 172.21.15.37:0/1843600495 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96241a6370 con 0x7f9624109f80 2024-06-10T01:03:05.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.931+0000 7f9629da3700 1 --2- 172.21.15.37:0/1843600495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9624109f80 0x7f96241a1360 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f961400e9a0 tx=0x7f961400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:05.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.933+0000 7f961affd700 1 -- 172.21.15.37:0/1843600495 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961400cbd0 con 0x7f9624109f80 2024-06-10T01:03:05.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.933+0000 7f961affd700 1 -- 172.21.15.37:0/1843600495 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f961400cd70 con 0x7f9624109f80 2024-06-10T01:03:05.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.934+0000 7f961affd700 1 -- 172.21.15.37:0/1843600495 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96140187a0 con 0x7f9624109f80 2024-06-10T01:03:05.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.937+0000 7f962ada5700 1 -- 172.21.15.37:0/1843600495 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96241a6660 con 0x7f9624109f80 2024-06-10T01:03:05.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.937+0000 7f962ada5700 1 -- 172.21.15.37:0/1843600495 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9624117740 con 0x7f9624109f80 2024-06-10T01:03:05.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.946+0000 7f963688ab80 1 Processor -- start 2024-06-10T01:03:05.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.946+0000 7f963688ab80 1 -- start start 2024-06-10T01:03:05.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.952+0000 7f963688ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d72a11090 0x558d72a11470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:05.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.952+0000 7f963688ab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d72a119b0 0x558d72a1c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:05.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.952+0000 7f963688ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d72a1c860 0x558d72a1ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:05.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.952+0000 7f963688ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558d728fc100 con 0x558d72a11090 2024-06-10T01:03:05.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.952+0000 7f963688ab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558d7295a9f0 con 0x558d72a119b0 2024-06-10T01:03:05.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.952+0000 7f963688ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558d728c2600 con 0x558d72a1c860 2024-06-10T01:03:05.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.954+0000 7f961affd700 1 -- 172.21.15.37:0/1843600495 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f961401f020 con 0x7f9624109f80 2024-06-10T01:03:05.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.955+0000 7f961affd700 1 --2- 172.21.15.37:0/1843600495 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f96100a2d00 0x7f96100a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:05.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.955+0000 7f962a5a4700 1 --2- 172.21.15.37:0/1843600495 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f96100a2d00 0x7f96100a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:05.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.955+0000 7f9629da3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d72a119b0 0x558d72a1c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:05.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.957+0000 7f96295a2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d72a1c860 0x558d72a1ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:05.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.957+0000 7f9629da3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d72a119b0 0x558d72a1c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:43070/0 (socket says 172.21.15.37:43070) 2024-06-10T01:03:05.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.957+0000 7f9629da3700 1 -- 172.21.15.37:0/3628334720 learned_addr learned my addr 172.21.15.37:0/3628334720 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:05.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.957+0000 7f961affd700 1 -- 172.21.15.37:0/1843600495 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f96140cf480 con 0x7f9624109f80 2024-06-10T01:03:05.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.958+0000 7f9629da3700 1 -- 172.21.15.37:0/3628334720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d72a1c860 msgr2=0x558d72a1ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:05.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.958+0000 7f9629da3700 1 --2- 172.21.15.37:0/3628334720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d72a1c860 0x558d72a1ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:05.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.958+0000 7f9629da3700 1 -- 172.21.15.37:0/3628334720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d72a11090 msgr2=0x558d72a11470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:05.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.958+0000 7f9629da3700 1 --2- 172.21.15.37:0/3628334720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d72a11090 0x558d72a11470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:05.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.958+0000 7f9629da3700 1 -- 172.21.15.37:0/3628334720 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d728a9a70 con 0x558d72a119b0 2024-06-10T01:03:05.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.959+0000 7f962a5a4700 1 --2- 172.21.15.37:0/1843600495 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f96100a2d00 0x7f96100a51c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9620006fd0 tx=0x7f9620009060 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:05.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.959+0000 7f9629da3700 1 --2- 172.21.15.37:0/3628334720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d72a119b0 0x558d72a1c320 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9614000c00 tx=0x7f961409b450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc53adfce23c2436 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:05.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.962+0000 7f960ffff700 1 -- 172.21.15.37:0/3628334720 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961409bf90 con 0x558d72a119b0 2024-06-10T01:03:05.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.962+0000 7f960ffff700 1 -- 172.21.15.37:0/3628334720 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f961409b890 con 0x558d72a119b0 2024-06-10T01:03:05.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.962+0000 7f960ffff700 1 -- 172.21.15.37:0/3628334720 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96140a4070 con 0x558d72a119b0 2024-06-10T01:03:05.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.964+0000 7f963688ab80 1 -- 172.21.15.37:0/3628334720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d72a119b0 msgr2=0x558d72a1c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:05.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.964+0000 7f963688ab80 1 --2- 172.21.15.37:0/3628334720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d72a119b0 0x558d72a1c320 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9614000c00 tx=0x7f961409b450 comp rx=0 tx=0).stop 2024-06-10T01:03:05.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.966+0000 7f963688ab80 1 -- 172.21.15.37:0/3628334720 shutdown_connections 2024-06-10T01:03:05.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.966+0000 7f963688ab80 1 --2- 172.21.15.37:0/3628334720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d72a1c860 0x558d72a1ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:05.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.966+0000 7f963688ab80 1 --2- 172.21.15.37:0/3628334720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d72a11090 0x558d72a11470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:05.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.966+0000 7f963688ab80 1 --2- 172.21.15.37:0/3628334720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d72a119b0 0x558d72a1c320 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:05.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.966+0000 7f963688ab80 1 -- 172.21.15.37:0/3628334720 >> 172.21.15.37:0/3628334720 conn(0x558d72936220 msgr2=0x558d72a04ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:05.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.970+0000 7f963688ab80 1 -- 172.21.15.37:0/3628334720 shutdown_connections 2024-06-10T01:03:05.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.971+0000 7f963688ab80 1 -- 172.21.15.37:0/3628334720 wait complete. 2024-06-10T01:03:05.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.971+0000 7f963688ab80 1 -- start start 2024-06-10T01:03:05.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.971+0000 7f963688ab80 1 Processor -- start 2024-06-10T01:03:05.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.972+0000 7f963688ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d72a11090 0x558d72a142c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:05.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.972+0000 7f963688ab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d72a119b0 0x558d72a14800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:05.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.972+0000 7f963688ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d72a1c860 0x558d72a1b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:05.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.972+0000 7f963688ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558d728fb160 con 0x558d72a1c860 2024-06-10T01:03:05.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.972+0000 7f963688ab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558d729569b0 con 0x558d72a119b0 2024-06-10T01:03:05.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.972+0000 7f963688ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558d72a06980 con 0x558d72a11090 2024-06-10T01:03:05.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.972+0000 7f9629da3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d72a11090 0x558d72a142c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:05.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.972+0000 7f9629da3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d72a11090 0x558d72a142c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:38086/0 (socket says 172.21.15.37:38086) 2024-06-10T01:03:05.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.972+0000 7f9629da3700 1 -- 172.21.15.37:0/3968095415 learned_addr learned my addr 172.21.15.37:0/3968095415 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:05.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.972+0000 7f963688ab80 10 client.0 _ll_register_callbacks cb 0x558d728f0750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:03:05.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.974+0000 7f962a5a4700 1 --2- 172.21.15.37:0/3968095415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d72a1c860 0x558d72a1b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:05.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.975+0000 7f96295a2700 1 --2- 172.21.15.37:0/3968095415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d72a119b0 0x558d72a14800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.977+0000 7f962a5a4700 1 -- 172.21.15.37:0/3968095415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d72a11090 msgr2=0x558d72a142c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.977+0000 7f962a5a4700 1 --2- 172.21.15.37:0/3968095415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d72a11090 0x558d72a142c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.977+0000 7f962a5a4700 1 -- 172.21.15.37:0/3968095415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d72a119b0 msgr2=0x558d72a14800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.977+0000 7f962a5a4700 1 --2- 172.21.15.37:0/3968095415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d72a119b0 0x558d72a14800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.977+0000 7f962a5a4700 1 -- 172.21.15.37:0/3968095415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d729208f0 con 0x558d72a1c860 2024-06-10T01:03:05.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.978+0000 7f962a5a4700 1 --2- 172.21.15.37:0/3968095415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d72a1c860 0x558d72a1b320 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f9620018e10 tx=0x7f9620006de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c310dd8805950b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:05.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.980+0000 7f960d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:03:05.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.980+0000 7f960d7fa700 1 -- 172.21.15.37:0/3968095415 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9620026060 con 0x558d72a1c860 2024-06-10T01:03:05.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.980+0000 7f960d7fa700 1 -- 172.21.15.37:0/3968095415 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f9620013070 con 0x558d72a1c860 2024-06-10T01:03:05.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.981+0000 7f963688ab80 1 -- 172.21.15.37:0/3968095415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f961409b890 con 0x558d72a1c860 2024-06-10T01:03:05.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.981+0000 7f960d7fa700 1 -- 172.21.15.37:0/3968095415 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9620018e40 con 0x558d72a1c860 2024-06-10T01:03:05.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.981+0000 7f963688ab80 10 client.16524 fetch_fsmap learned FSMap version 27 2024-06-10T01:03:05.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.981+0000 7f963688ab80 1 -- 172.21.15.37:0/3968095415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558d72a21380 con 0x558d72a1c860 2024-06-10T01:03:05.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.981+0000 7f960d7fa700 1 -- 172.21.15.37:0/3968095415 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f961409b890 con 0x558d72a1c860 2024-06-10T01:03:05.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.981+0000 7f960d7fa700 1 -- 172.21.15.37:0/3968095415 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9620006220 con 0x558d72a1c860 2024-06-10T01:03:05.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.982+0000 7f963688ab80 10 client.16524 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:03:05.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.982+0000 7f963688ab80 10 client.16524 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:03:05.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.982+0000 7f963688ab80 20 client.16524 populate_metadata read hostname 'smithi037' 2024-06-10T01:03:05.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.982+0000 7f963688ab80 1 --2- 172.21.15.37:0/3968095415 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558d72a23d40 0x558d72a26160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:05.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.982+0000 7f963688ab80 4 client.16524 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:03:05.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.982+0000 7f963688ab80 1 -- 172.21.15.37:0/3968095415 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d72967ef0 con 0x558d72a23d40 2024-06-10T01:03:05.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.985+0000 7f9629da3700 1 --2- 172.21.15.37:0/3968095415 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558d72a23d40 0x558d72a26160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:05.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.991+0000 7f960d7fa700 10 client.16524 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:03:05.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:05.991+0000 7f9629da3700 1 --2- 172.21.15.37:0/3968095415 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558d72a23d40 0x558d72a26160 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c56e4c7a831df21 server_cookie=85f5944ce3e25890 in_seq=0 out_seq=0 2024-06-10T01:03:06.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.008+0000 7f960d7fa700 1 -- 172.21.15.37:0/3968095415 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558d72967ef0 con 0x558d72a23d40 2024-06-10T01:03:06.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.008+0000 7f960d7fa700 10 client.16524 handle_command_reply: tid=0 2024-06-10T01:03:06.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.008+0000 7f963688ab80 1 client.16524 shutdown 2024-06-10T01:03:06.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.008+0000 7f963688ab80 1 -- 172.21.15.37:0/3968095415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d72a1c860 msgr2=0x558d72a1b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:06.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.008+0000 7f963688ab80 1 --2- 172.21.15.37:0/3968095415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d72a1c860 0x558d72a1b320 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f9620018e10 tx=0x7f9620006de0 comp rx=0 tx=0).stop 2024-06-10T01:03:06.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.008+0000 7f963688ab80 1 -- 172.21.15.37:0/3968095415 shutdown_connections 2024-06-10T01:03:06.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.008+0000 7f963688ab80 1 --2- 172.21.15.37:0/3968095415 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558d72a23d40 0x558d72a26160 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:06.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.008+0000 7f963688ab80 1 --2- 172.21.15.37:0/3968095415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d72a1c860 0x558d72a1b320 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:06.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.008+0000 7f963688ab80 1 --2- 172.21.15.37:0/3968095415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d72a11090 0x558d72a142c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:06.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.008+0000 7f963688ab80 1 --2- 172.21.15.37:0/3968095415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d72a119b0 0x558d72a14800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:06.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.008+0000 7f963688ab80 1 -- 172.21.15.37:0/3968095415 >> 172.21.15.37:0/3968095415 conn(0x558d72936220 msgr2=0x558d72a04830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:06.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.008+0000 7f960d7fa700 0 client.16524 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:03:06.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.009+0000 7f963688ab80 1 -- 172.21.15.37:0/3968095415 shutdown_connections 2024-06-10T01:03:06.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.009+0000 7f963688ab80 1 -- 172.21.15.37:0/3968095415 wait complete. 2024-06-10T01:03:06.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.009+0000 7f963688ab80 20 client.16524 trim_cache size 0 max 16384 2024-06-10T01:03:06.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.053+0000 7f963688ab80 1 Processor -- start 2024-06-10T01:03:06.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.054+0000 7f963688ab80 1 -- start start 2024-06-10T01:03:06.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.054+0000 7f963688ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d72a11090 0x558d72a04940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:06.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.055+0000 7f963688ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d72a119b0 0x558d72a136a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:06.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.055+0000 7f963688ab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d72a1c860 0x558d72a13be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:06.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.055+0000 7f963688ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9624083880 con 0x558d72a11090 2024-06-10T01:03:06.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.055+0000 7f963688ab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9624081110 con 0x558d72a1c860 2024-06-10T01:03:06.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.055+0000 7f963688ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f962407ef60 con 0x558d72a119b0 2024-06-10T01:03:06.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.055+0000 7f96295a2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d72a119b0 0x558d72a136a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:06.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.055+0000 7f96295a2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d72a119b0 0x558d72a136a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:38094/0 (socket says 172.21.15.37:38094) 2024-06-10T01:03:06.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.055+0000 7f96295a2700 1 -- 172.21.15.37:0/1040338187 learned_addr learned my addr 172.21.15.37:0/1040338187 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:06.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.055+0000 7f962a5a4700 1 --2- 172.21.15.37:0/1040338187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d72a1c860 0x558d72a13be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:06.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.056+0000 7f9629da3700 1 --2- 172.21.15.37:0/1040338187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d72a11090 0x558d72a04940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:06.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.062+0000 7f9629da3700 1 -- 172.21.15.37:0/1040338187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d72a119b0 msgr2=0x558d72a136a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:06.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.062+0000 7f9629da3700 1 --2- 172.21.15.37:0/1040338187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d72a119b0 0x558d72a136a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:06.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.062+0000 7f9629da3700 1 -- 172.21.15.37:0/1040338187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d72a1c860 msgr2=0x558d72a13be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:06.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.062+0000 7f9629da3700 1 --2- 172.21.15.37:0/1040338187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d72a1c860 0x558d72a13be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:06.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.062+0000 7f9629da3700 1 -- 172.21.15.37:0/1040338187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d72a0eb80 con 0x558d72a11090 2024-06-10T01:03:06.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.062+0000 7f9629da3700 1 --2- 172.21.15.37:0/1040338187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d72a11090 0x558d72a04940 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f96140a3fd0 tx=0x7f9614023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abaab5bdfa5b8033 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:06.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.063+0000 7f960f7fe700 1 -- 172.21.15.37:0/1040338187 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96140a8020 con 0x558d72a11090 2024-06-10T01:03:06.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.063+0000 7f960f7fe700 1 -- 172.21.15.37:0/1040338187 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f961409b890 con 0x558d72a11090 2024-06-10T01:03:06.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.063+0000 7f960f7fe700 1 -- 172.21.15.37:0/1040338187 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96140a4070 con 0x558d72a11090 2024-06-10T01:03:06.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.064+0000 7f963688ab80 1 -- 172.21.15.37:0/1040338187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d72a11090 msgr2=0x558d72a04940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:06.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.064+0000 7f963688ab80 1 --2- 172.21.15.37:0/1040338187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d72a11090 0x558d72a04940 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f96140a3fd0 tx=0x7f9614023040 comp rx=0 tx=0).stop 2024-06-10T01:03:06.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.065+0000 7f963688ab80 1 -- 172.21.15.37:0/1040338187 shutdown_connections 2024-06-10T01:03:06.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.065+0000 7f963688ab80 1 --2- 172.21.15.37:0/1040338187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d72a1c860 0x558d72a13be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:06.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.065+0000 7f963688ab80 1 --2- 172.21.15.37:0/1040338187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d72a11090 0x558d72a04940 secure :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f96140a3fd0 tx=0x7f9614023040 comp rx=0 tx=0).stop 2024-06-10T01:03:06.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.065+0000 7f963688ab80 1 --2- 172.21.15.37:0/1040338187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d72a119b0 0x558d72a136a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:06.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.065+0000 7f963688ab80 1 -- 172.21.15.37:0/1040338187 >> 172.21.15.37:0/1040338187 conn(0x558d72936220 msgr2=0x558d72a12f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:06.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.067+0000 7f963688ab80 1 -- 172.21.15.37:0/1040338187 shutdown_connections 2024-06-10T01:03:06.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.069+0000 7f963688ab80 1 -- 172.21.15.37:0/1040338187 wait complete. 2024-06-10T01:03:06.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.069+0000 7f963688ab80 1 -- start start 2024-06-10T01:03:06.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.070+0000 7f963688ab80 1 Processor -- start 2024-06-10T01:03:06.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.070+0000 7f963688ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d72a119b0 0x558d72a194f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:06.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.070+0000 7f963688ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d72a1c860 0x558d72a19a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:06.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.070+0000 7f963688ab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d72a8ba20 0x558d72a8de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:06.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.070+0000 7f963688ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f962407f5d0 con 0x558d72a1c860 2024-06-10T01:03:06.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.070+0000 7f963688ab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9624080a40 con 0x558d72a8ba20 2024-06-10T01:03:06.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.070+0000 7f963688ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f962407fca0 con 0x558d72a119b0 2024-06-10T01:03:06.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.070+0000 7f963688ab80 10 client.0 _ll_register_callbacks cb 0x558d728f0750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:03:06.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.073+0000 7f9629da3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d72a119b0 0x558d72a194f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:06.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.073+0000 7f962a5a4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d72a8ba20 0x558d72a8de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:06.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.074+0000 7f9629da3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d72a119b0 0x558d72a194f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:38110/0 (socket says 172.21.15.37:38110) 2024-06-10T01:03:06.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.074+0000 7f9629da3700 1 -- 172.21.15.37:0/1531404442 learned_addr learned my addr 172.21.15.37:0/1531404442 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:06.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.074+0000 7f9629da3700 1 -- 172.21.15.37:0/1531404442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d72a8ba20 msgr2=0x558d72a8de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:06.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.074+0000 7f9629da3700 1 --2- 172.21.15.37:0/1531404442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d72a8ba20 0x558d72a8de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:06.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.074+0000 7f9629da3700 1 -- 172.21.15.37:0/1531404442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d72a1c860 msgr2=0x558d72a19a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:03:06.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.074+0000 7f9629da3700 1 --2- 172.21.15.37:0/1531404442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d72a1c860 0x558d72a19a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:06.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.074+0000 7f9629da3700 1 -- 172.21.15.37:0/1531404442 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d72a8e3d0 con 0x558d72a119b0 2024-06-10T01:03:06.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.079+0000 7f9629da3700 1 --2- 172.21.15.37:0/1531404442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d72a119b0 0x558d72a194f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9614098260 tx=0x7f961400cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c47bcb96251e5f8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:06.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.079+0000 7f960d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:03:06.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.083+0000 7f960d7fa700 1 -- 172.21.15.37:0/1531404442 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96140a8050 con 0x558d72a119b0 2024-06-10T01:03:06.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.083+0000 7f960d7fa700 1 -- 172.21.15.37:0/1531404442 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f9614099b90 con 0x558d72a119b0 2024-06-10T01:03:06.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.084+0000 7f963688ab80 1 -- 172.21.15.37:0/1531404442 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f961409b890 con 0x558d72a119b0 2024-06-10T01:03:06.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.090+0000 7f960d7fa700 1 -- 172.21.15.37:0/1531404442 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f961401fdf0 con 0x558d72a119b0 2024-06-10T01:03:06.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.090+0000 7f963688ab80 10 client.26086 fetch_fsmap learned FSMap version 27 2024-06-10T01:03:06.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.090+0000 7f963688ab80 1 -- 172.21.15.37:0/1531404442 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558d72a8f6b0 con 0x558d72a119b0 2024-06-10T01:03:06.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.092+0000 7f960d7fa700 1 -- 172.21.15.37:0/1531404442 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f961409b890 con 0x558d72a119b0 2024-06-10T01:03:06.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.092+0000 7f960d7fa700 1 -- 172.21.15.37:0/1531404442 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f96140ab020 con 0x558d72a119b0 2024-06-10T01:03:06.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.092+0000 7f963688ab80 10 client.26086 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:03:06.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.092+0000 7f963688ab80 10 client.26086 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:03:06.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.092+0000 7f963688ab80 20 client.26086 populate_metadata read hostname 'smithi037' 2024-06-10T01:03:06.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.092+0000 7f963688ab80 1 --2- 172.21.15.37:0/1531404442 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558d72a8fc70 0x558d72a92060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:06.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.092+0000 7f963688ab80 4 client.26086 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.45, "await_for": 45.0}] 2024-06-10T01:03:06.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.092+0000 7f963688ab80 1 -- 172.21.15.37:0/1531404442 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.45, "await_for": 45.0}) -- 0x558d72962470 con 0x558d72a8fc70 2024-06-10T01:03:06.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.094+0000 7f96295a2700 1 --2- 172.21.15.37:0/1531404442 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558d72a8fc70 0x558d72a92060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:06.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.099+0000 7f960d7fa700 10 client.26086 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:03:06.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:06.099+0000 7f96295a2700 1 --2- 172.21.15.37:0/1531404442 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558d72a8fc70 0x558d72a92060 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90c9c81f9bbe57aa server_cookie=3738e10a919a287a in_seq=0 out_seq=0 2024-06-10T01:03:06.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:06 smithi167 ceph-mon[36917]: pgmap v622: 129 pgs: 129 active+clean; 25 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 758 op/s 2024-06-10T01:03:06.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:06 smithi094 ceph-mon[36708]: pgmap v622: 129 pgs: 129 active+clean; 25 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 758 op/s 2024-06-10T01:03:06.643 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:06 smithi037 ceph-mon[32192]: pgmap v622: 129 pgs: 129 active+clean; 25 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 758 op/s 2024-06-10T01:03:08.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:08 smithi167 ceph-mon[36917]: pgmap v623: 129 pgs: 129 active+clean; 29 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 443 op/s 2024-06-10T01:03:08.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:08 smithi094 ceph-mon[36708]: pgmap v623: 129 pgs: 129 active+clean; 29 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 443 op/s 2024-06-10T01:03:08.643 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:08 smithi037 ceph-mon[32192]: pgmap v623: 129 pgs: 129 active+clean; 29 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 443 op/s 2024-06-10T01:03:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:09.229+0000 7f5eb1ffb640 1 -- 172.21.15.37:0/365067781 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562c1c7aac20 con 0x562c1c854590 2024-06-10T01:03:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:09.229+0000 7f5eb1ffb640 10 client.16515 handle_command_reply: tid=0 2024-06-10T01:03:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:09.229+0000 7f5ed3bb8740 1 client.16515 shutdown 2024-06-10T01:03:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:09.229+0000 7f5ed3bb8740 1 -- 172.21.15.37:0/365067781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c1c83e420 msgr2=0x562c1c84cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:09.230+0000 7f5ed3bb8740 1 --2- 172.21.15.37:0/365067781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c1c83e420 0x562c1c84cb00 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc03d040 tx=0x7f5ebc0c4e40 comp rx=0 tx=0).stop 2024-06-10T01:03:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:09.230+0000 7f5ed3bb8740 1 -- 172.21.15.37:0/365067781 shutdown_connections 2024-06-10T01:03:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:09.230+0000 7f5ed3bb8740 1 --2- 172.21.15.37:0/365067781 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x562c1c854590 0x562c1c8c1410 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:09.230+0000 7f5ed3bb8740 1 --2- 172.21.15.37:0/365067781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562c1c843f80 0x562c1c8bd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:09.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:09.230+0000 7f5eb1ffb640 0 client.16515 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:03:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:09.230+0000 7f5ed3bb8740 1 --2- 172.21.15.37:0/365067781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562c1c83ee10 0x562c1c84d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:09.230+0000 7f5ed3bb8740 1 --2- 172.21.15.37:0/365067781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c1c83e420 0x562c1c84cb00 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:09.230+0000 7f5ed3bb8740 1 -- 172.21.15.37:0/365067781 >> 172.21.15.37:0/365067781 conn(0x562c1c835680 msgr2=0x562c1c8484a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:09.230+0000 7f5ed3bb8740 1 -- 172.21.15.37:0/365067781 shutdown_connections 2024-06-10T01:03:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:09.230+0000 7f5ed3bb8740 1 -- 172.21.15.37:0/365067781 wait complete. 2024-06-10T01:03:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:09.230+0000 7f5ed3bb8740 20 client.16515 trim_cache size 0 max 16384 2024-06-10T01:03:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:03:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:03:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:03:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:09.230+0000 7f5ed0dce640 1 -- 172.21.15.37:0/992728224 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5eb40a2c30 msgr2=0x7f5eb40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:09.230+0000 7f5ed0dce640 1 --2- 172.21.15.37:0/992728224 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5eb40a2c30 0x7f5eb40a50f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x562c1c83f670 tx=0x7f5ec4009790 comp rx=0 tx=0).stop 2024-06-10T01:03:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:09.230+0000 7f5ed0dce640 1 -- 172.21.15.37:0/992728224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ecc100900 msgr2=0x7f5ecc197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:09.230+0000 7f5ed0dce640 1 --2- 172.21.15.37:0/992728224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ecc100900 0x7f5ecc197cf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc031610 tx=0x7f5ebc004270 comp rx=0 tx=0).stop 2024-06-10T01:03:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:09.231+0000 7f5ed0dce640 1 -- 172.21.15.37:0/992728224 shutdown_connections 2024-06-10T01:03:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:09.231+0000 7f5ed0dce640 1 --2- 172.21.15.37:0/992728224 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5eb40a2c30 0x7f5eb40a50f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:09.231+0000 7f5ed0dce640 1 --2- 172.21.15.37:0/992728224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ecc1024b0 0x7f5ecc19f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:09.231+0000 7f5ed0dce640 1 --2- 172.21.15.37:0/992728224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ecc101af0 0x7f5ecc198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:09.231+0000 7f5ed0dce640 1 --2- 172.21.15.37:0/992728224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ecc100900 0x7f5ecc197cf0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:09.231+0000 7f5ed0dce640 1 -- 172.21.15.37:0/992728224 >> 172.21.15.37:0/992728224 conn(0x7f5ecc0fbe70 msgr2=0x7f5ecc0fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:09.231+0000 7f5ed0dce640 1 -- 172.21.15.37:0/992728224 shutdown_connections 2024-06-10T01:03:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:09.231+0000 7f5ed0dce640 1 -- 172.21.15.37:0/992728224 wait complete. 2024-06-10T01:03:10.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:10 smithi167 ceph-mon[36917]: pgmap v624: 129 pgs: 129 active+clean; 32 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 1.90k op/s 2024-06-10T01:03:10.586 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:03:10 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-f[67368]: *** Caught signal (Segmentation fault) ** 2024-06-10T01:03:10.586 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:03:10 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-f[67368]: in thread 7f647d51c700 thread_name:MR_Finisher 2024-06-10T01:03:10.586 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:03:10 smithi167 podman[71042]: 2024-06-10 01:03:10.334104321 +0000 UTC m=+0.026656649 container died 3f963f7d3e44deceddeb70d919ef039e2a9515c333c36a02c7c319623b2c6b34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, ceph=True) 2024-06-10T01:03:10.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:10 smithi094 ceph-mon[36708]: pgmap v624: 129 pgs: 129 active+clean; 32 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 1.90k op/s 2024-06-10T01:03:10.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:10 smithi037 ceph-mon[32192]: pgmap v624: 129 pgs: 129 active+clean; 32 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 1.90k op/s 2024-06-10T01:03:12.075 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:03:11 smithi167 podman[71042]: 2024-06-10 01:03:11.757967097 +0000 UTC m=+1.450519404 container remove 3f963f7d3e44deceddeb70d919ef039e2a9515c333c36a02c7c319623b2c6b34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-f, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T01:03:12.075 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:03:11 smithi167 systemd[1]: ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.f.service: Main process exited, code=exited, status=139/n/a 2024-06-10T01:03:12.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:12 smithi167 ceph-mon[36917]: pgmap v625: 129 pgs: 129 active+clean; 34 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 2.12k op/s 2024-06-10T01:03:12.586 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:03:12 smithi167 systemd[1]: ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.f.service: Failed with result 'exit-code'. 2024-06-10T01:03:12.586 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:03:12 smithi167 systemd[1]: ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.f.service: Consumed 12.813s CPU time. 2024-06-10T01:03:12.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:12 smithi094 ceph-mon[36708]: pgmap v625: 129 pgs: 129 active+clean; 34 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 2.12k op/s 2024-06-10T01:03:12.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:12 smithi037 ceph-mon[32192]: pgmap v625: 129 pgs: 129 active+clean; 34 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 2.12k op/s 2024-06-10T01:03:14.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:14 smithi167 ceph-mon[36917]: pgmap v626: 129 pgs: 129 active+clean; 23 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 2.23k op/s 2024-06-10T01:03:14.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:14 smithi094 ceph-mon[36708]: pgmap v626: 129 pgs: 129 active+clean; 23 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 2.23k op/s 2024-06-10T01:03:14.643 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:14 smithi037 ceph-mon[32192]: pgmap v626: 129 pgs: 129 active+clean; 23 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 2.23k op/s 2024-06-10T01:03:16.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:16 smithi167 ceph-mon[36917]: pgmap v627: 129 pgs: 129 active+clean; 13 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 2.36k op/s 2024-06-10T01:03:16.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:16 smithi094 ceph-mon[36708]: pgmap v627: 129 pgs: 129 active+clean; 13 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 2.36k op/s 2024-06-10T01:03:16.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:16 smithi037 ceph-mon[32192]: pgmap v627: 129 pgs: 129 active+clean; 13 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 2.36k op/s 2024-06-10T01:03:18.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:18 smithi167 ceph-mon[36917]: pgmap v628: 129 pgs: 129 active+clean; 16 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 2.6 MiB/s wr, 2.44k op/s 2024-06-10T01:03:18.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:18 smithi094 ceph-mon[36708]: pgmap v628: 129 pgs: 129 active+clean; 16 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 2.6 MiB/s wr, 2.44k op/s 2024-06-10T01:03:18.643 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:18 smithi037 ceph-mon[32192]: pgmap v628: 129 pgs: 129 active+clean; 16 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 2.6 MiB/s wr, 2.44k op/s 2024-06-10T01:03:20.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:20 smithi167 ceph-mon[36917]: pgmap v629: 129 pgs: 129 active+clean; 16 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 2.2 MiB/s wr, 2.36k op/s 2024-06-10T01:03:20.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:20 smithi094 ceph-mon[36708]: pgmap v629: 129 pgs: 129 active+clean; 16 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 2.2 MiB/s wr, 2.36k op/s 2024-06-10T01:03:20.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:20 smithi037 ceph-mon[32192]: pgmap v629: 129 pgs: 129 active+clean; 16 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 2.2 MiB/s wr, 2.36k op/s 2024-06-10T01:03:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.328+0000 7f5564ff9640 1 -- 172.21.15.37:0/569896732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5560102480 msgr2=0x7f5560104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.328+0000 7f5564ff9640 1 --2- 172.21.15.37:0/569896732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5560102480 0x7f5560104870 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5550009900 tx=0x7f555002a410 comp rx=0 tx=0).stop 2024-06-10T01:03:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.328+0000 7f5564ff9640 1 -- 172.21.15.37:0/569896732 shutdown_connections 2024-06-10T01:03:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.328+0000 7f5564ff9640 1 --2- 172.21.15.37:0/569896732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55601076e0 0x7f5560109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.328+0000 7f5564ff9640 1 --2- 172.21.15.37:0/569896732 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5560104db0 0x7f55601071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.328+0000 7f5564ff9640 1 --2- 172.21.15.37:0/569896732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5560102480 0x7f5560104870 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.328+0000 7f5564ff9640 1 -- 172.21.15.37:0/569896732 >> 172.21.15.37:0/569896732 conn(0x7f55600fbe70 msgr2=0x7f55600fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.328+0000 7f5564ff9640 1 -- 172.21.15.37:0/569896732 shutdown_connections 2024-06-10T01:03:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.328+0000 7f5564ff9640 1 -- 172.21.15.37:0/569896732 wait complete. 2024-06-10T01:03:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.329+0000 7f5564ff9640 1 Processor -- start 2024-06-10T01:03:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.329+0000 7f5564ff9640 1 -- start start 2024-06-10T01:03:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.329+0000 7f5564ff9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5560102480 0x7f5560101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.329+0000 7f5564ff9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5560104db0 0x7f5560100430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.329+0000 7f5564ff9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55601076e0 0x7f5560100970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.329+0000 7f5564ff9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f556010d6f0 con 0x7f55601076e0 2024-06-10T01:03:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.329+0000 7f5564ff9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f556010d570 con 0x7f5560102480 2024-06-10T01:03:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.329+0000 7f5564ff9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f556010d870 con 0x7f5560104db0 2024-06-10T01:03:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.330+0000 7f555ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55601076e0 0x7f5560100970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.330+0000 7f555dd9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5560104db0 0x7f5560100430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.330+0000 7f555ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55601076e0 0x7f5560100970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58920/0 (socket says 172.21.15.37:58920) 2024-06-10T01:03:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.330+0000 7f555ed9d640 1 -- 172.21.15.37:0/96961427 learned_addr learned my addr 172.21.15.37:0/96961427 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.330+0000 7f555dd9b640 1 --2- 172.21.15.37:0/96961427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5560104db0 0x7f5560100430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:48390/0 (socket says 172.21.15.37:48390) 2024-06-10T01:03:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.330+0000 7f555e59c640 1 --2- 172.21.15.37:0/96961427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5560102480 0x7f5560101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.330+0000 7f555ed9d640 1 -- 172.21.15.37:0/96961427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5560104db0 msgr2=0x7f5560100430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.330+0000 7f555ed9d640 1 --2- 172.21.15.37:0/96961427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5560104db0 0x7f5560100430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.330+0000 7f555ed9d640 1 -- 172.21.15.37:0/96961427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5560102480 msgr2=0x7f5560101df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.330+0000 7f555ed9d640 1 --2- 172.21.15.37:0/96961427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5560102480 0x7f5560101df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.330+0000 7f555ed9d640 1 -- 172.21.15.37:0/96961427 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5560101200 con 0x7f55601076e0 2024-06-10T01:03:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.330+0000 7f555dd9b640 1 --2- 172.21.15.37:0/96961427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5560104db0 0x7f5560100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:03:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.330+0000 7f555ed9d640 1 --2- 172.21.15.37:0/96961427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55601076e0 0x7f5560100970 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f555800eb10 tx=0x7f555800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.330+0000 7f554f7fe640 1 -- 172.21.15.37:0/96961427 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558004b20 con 0x7f55601076e0 2024-06-10T01:03:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.330+0000 7f5564ff9640 1 -- 172.21.15.37:0/96961427 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55601a1db0 con 0x7f55601076e0 2024-06-10T01:03:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.330+0000 7f554f7fe640 1 -- 172.21.15.37:0/96961427 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5558004cc0 con 0x7f55601076e0 2024-06-10T01:03:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.330+0000 7f554f7fe640 1 -- 172.21.15.37:0/96961427 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558010760 con 0x7f55601076e0 2024-06-10T01:03:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.330+0000 7f5564ff9640 1 -- 172.21.15.37:0/96961427 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55601a2270 con 0x7f55601076e0 2024-06-10T01:03:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.333+0000 7f554f7fe640 1 -- 172.21.15.37:0/96961427 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5558005020 con 0x7f55601076e0 2024-06-10T01:03:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.333+0000 7f5567de3740 1 Processor -- start 2024-06-10T01:03:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.333+0000 7f5567de3740 1 -- start start 2024-06-10T01:03:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.334+0000 7f5567de3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557941e69010 0x557941e69430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.334+0000 7f5567de3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557941e69a00 0x557941e6e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.334+0000 7f5567de3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557941e6eb70 0x557941e71000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.334+0000 7f5567de3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557941dcc4f0 con 0x557941e6eb70 2024-06-10T01:03:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.334+0000 7f5567de3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557941d6a870 con 0x557941e69a00 2024-06-10T01:03:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.334+0000 7f5567de3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557941dc84b0 con 0x557941e69010 2024-06-10T01:03:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.334+0000 7f555e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557941e69a00 0x557941e6e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.334+0000 7f555dd9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557941e69010 0x557941e69430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.334+0000 7f555e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557941e69a00 0x557941e6e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:57118/0 (socket says 172.21.15.37:57118) 2024-06-10T01:03:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.334+0000 7f555e59c640 1 -- 172.21.15.37:0/1722907517 learned_addr learned my addr 172.21.15.37:0/1722907517 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.334+0000 7f555dd9b640 1 --2- 172.21.15.37:0/1722907517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557941e6eb70 0x557941e71000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.334+0000 7f555e59c640 1 -- 172.21.15.37:0/1722907517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557941e69010 msgr2=0x557941e69430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.334+0000 7f555e59c640 1 --2- 172.21.15.37:0/1722907517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557941e69010 0x557941e69430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.334+0000 7f555e59c640 1 -- 172.21.15.37:0/1722907517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557941e6eb70 msgr2=0x557941e71000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.334+0000 7f555e59c640 1 --2- 172.21.15.37:0/1722907517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557941e6eb70 0x557941e71000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.334+0000 7f555e59c640 1 -- 172.21.15.37:0/1722907517 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557941d869b0 con 0x557941e69a00 2024-06-10T01:03:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.334+0000 7f555e59c640 1 --2- 172.21.15.37:0/1722907517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557941e69a00 0x557941e6e500 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5550000c00 tx=0x7f5550004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c6b82b45af6d68a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.334+0000 7f554f7fe640 1 --2- 172.21.15.37:0/96961427 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f55480a2cb0 0x7f55480a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.335+0000 7f555dd9b640 1 --2- 172.21.15.37:0/96961427 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f55480a2cb0 0x7f55480a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.335+0000 7f554cff9640 1 -- 172.21.15.37:0/1722907517 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550035040 con 0x557941e69a00 2024-06-10T01:03:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.335+0000 7f554f7fe640 1 -- 172.21.15.37:0/96961427 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f55580cfb00 con 0x7f55601076e0 2024-06-10T01:03:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.335+0000 7f554cff9640 1 -- 172.21.15.37:0/1722907517 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5550005520 con 0x557941e69a00 2024-06-10T01:03:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.335+0000 7f554cff9640 1 -- 172.21.15.37:0/1722907517 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550031910 con 0x557941e69a00 2024-06-10T01:03:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.335+0000 7f5567de3740 1 -- 172.21.15.37:0/1722907517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557941e69a00 msgr2=0x557941e6e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.335+0000 7f555dd9b640 1 --2- 172.21.15.37:0/96961427 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f55480a2cb0 0x7f55480a5170 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x557941e6a260 tx=0x7f5554005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.335+0000 7f5567de3740 1 --2- 172.21.15.37:0/1722907517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557941e69a00 0x557941e6e500 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5550000c00 tx=0x7f5550004040 comp rx=0 tx=0).stop 2024-06-10T01:03:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.336+0000 7f5567de3740 1 -- 172.21.15.37:0/1722907517 shutdown_connections 2024-06-10T01:03:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.336+0000 7f5567de3740 1 --2- 172.21.15.37:0/1722907517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557941e6eb70 0x557941e71000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.336+0000 7f5567de3740 1 --2- 172.21.15.37:0/1722907517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557941e69a00 0x557941e6e500 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.336+0000 7f5567de3740 1 --2- 172.21.15.37:0/1722907517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557941e69010 0x557941e69430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.336+0000 7f5567de3740 1 -- 172.21.15.37:0/1722907517 >> 172.21.15.37:0/1722907517 conn(0x557941e64680 msgr2=0x557941e64a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.336+0000 7f5567de3740 1 -- 172.21.15.37:0/1722907517 shutdown_connections 2024-06-10T01:03:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.336+0000 7f5567de3740 1 -- 172.21.15.37:0/1722907517 wait complete. 2024-06-10T01:03:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.336+0000 7f5567de3740 1 -- start start 2024-06-10T01:03:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.336+0000 7f5567de3740 1 Processor -- start 2024-06-10T01:03:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.336+0000 7f5567de3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557941e69010 0x557941e73250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.336+0000 7f5567de3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557941e69a00 0x557941e73790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.336+0000 7f5567de3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557941e6eb70 0x557941e7a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.336+0000 7f5567de3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557941e71ac0 con 0x557941e6eb70 2024-06-10T01:03:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.336+0000 7f5567de3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557941d55ae0 con 0x557941e69a00 2024-06-10T01:03:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.336+0000 7f5567de3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557941e71c40 con 0x557941e69010 2024-06-10T01:03:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.336+0000 7f555dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557941e69a00 0x557941e73790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.336+0000 7f555e59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557941e69010 0x557941e73250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.337+0000 7f555dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557941e69a00 0x557941e73790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:57122/0 (socket says 172.21.15.37:57122) 2024-06-10T01:03:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.337+0000 7f555e59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557941e69010 0x557941e73250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:48396/0 (socket says 172.21.15.37:48396) 2024-06-10T01:03:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.337+0000 7f555dd9b640 1 -- 172.21.15.37:0/345637186 learned_addr learned my addr 172.21.15.37:0/345637186 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.337+0000 7f555ed9d640 1 --2- 172.21.15.37:0/345637186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557941e6eb70 0x557941e7a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.337+0000 7f5567de3740 10 client.0 _ll_register_callbacks cb 0x557941d8e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:03:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.337+0000 7f555dd9b640 1 -- 172.21.15.37:0/345637186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557941e69010 msgr2=0x557941e73250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.337+0000 7f555dd9b640 1 --2- 172.21.15.37:0/345637186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557941e69010 0x557941e73250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.337+0000 7f555dd9b640 1 -- 172.21.15.37:0/345637186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557941e6eb70 msgr2=0x557941e7a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.337+0000 7f555dd9b640 1 --2- 172.21.15.37:0/345637186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557941e6eb70 0x557941e7a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.337+0000 7f555dd9b640 1 -- 172.21.15.37:0/345637186 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557941ae0210 con 0x557941e69a00 2024-06-10T01:03:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.337+0000 7f555dd9b640 1 --2- 172.21.15.37:0/345637186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557941e69a00 0x557941e73790 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5554011fd0 tx=0x7f5554004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65a321235b1e2d9d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.337+0000 7f5545ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:03:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.337+0000 7f5545ffb640 1 -- 172.21.15.37:0/345637186 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5554030060 con 0x557941e69a00 2024-06-10T01:03:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.337+0000 7f5567de3740 1 -- 172.21.15.37:0/345637186 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5550005520 con 0x557941e69a00 2024-06-10T01:03:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.337+0000 7f5545ffb640 1 -- 172.21.15.37:0/345637186 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f555401a050 con 0x557941e69a00 2024-06-10T01:03:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.337+0000 7f5545ffb640 1 -- 172.21.15.37:0/345637186 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5554037050 con 0x557941e69a00 2024-06-10T01:03:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.337+0000 7f5567de3740 10 client.26003 fetch_fsmap learned FSMap version 27 2024-06-10T01:03:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.338+0000 7f5567de3740 1 -- 172.21.15.37:0/345637186 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557941bac620 con 0x557941e69a00 2024-06-10T01:03:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.338+0000 7f5545ffb640 1 -- 172.21.15.37:0/345637186 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5550005520 con 0x557941e69a00 2024-06-10T01:03:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.338+0000 7f5545ffb640 1 -- 172.21.15.37:0/345637186 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f555402b670 con 0x557941e69a00 2024-06-10T01:03:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.338+0000 7f5567de3740 10 client.26003 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:03:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.338+0000 7f5567de3740 10 client.26003 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:03:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.338+0000 7f5567de3740 20 client.26003 populate_metadata read hostname 'smithi037' 2024-06-10T01:03:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.338+0000 7f5567de3740 1 --2- 172.21.15.37:0/345637186 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557941e7f0a0 0x557941e81510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.338+0000 7f5567de3740 4 client.26003 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:03:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.338+0000 7f5567de3740 1 -- 172.21.15.37:0/345637186 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557941ddff10 con 0x557941e7f0a0 2024-06-10T01:03:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.338+0000 7f555e59c640 1 --2- 172.21.15.37:0/345637186 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557941e7f0a0 0x557941e81510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.339+0000 7f555e59c640 1 --2- 172.21.15.37:0/345637186 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557941e7f0a0 0x557941e81510 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddc7492f96122bdb server_cookie=ae7b3f76811cbbeb in_seq=0 out_seq=0 2024-06-10T01:03:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.339+0000 7f5545ffb640 10 client.26003 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:03:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.342+0000 7f5545ffb640 1 -- 172.21.15.37:0/345637186 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557941ddff10 con 0x557941e7f0a0 2024-06-10T01:03:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.342+0000 7f5545ffb640 10 client.26003 handle_command_reply: tid=0 2024-06-10T01:03:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.342+0000 7f5567de3740 1 client.26003 shutdown 2024-06-10T01:03:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.342+0000 7f5567de3740 1 -- 172.21.15.37:0/345637186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557941e69a00 msgr2=0x557941e73790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.342+0000 7f5567de3740 1 --2- 172.21.15.37:0/345637186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557941e69a00 0x557941e73790 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5554011fd0 tx=0x7f5554004730 comp rx=0 tx=0).stop 2024-06-10T01:03:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.342+0000 7f5567de3740 1 -- 172.21.15.37:0/345637186 shutdown_connections 2024-06-10T01:03:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.342+0000 7f5567de3740 1 --2- 172.21.15.37:0/345637186 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557941e7f0a0 0x557941e81510 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.342+0000 7f5567de3740 1 --2- 172.21.15.37:0/345637186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557941e6eb70 0x557941e7a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.342+0000 7f5545ffb640 0 client.26003 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:03:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.342+0000 7f5567de3740 1 --2- 172.21.15.37:0/345637186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557941e69a00 0x557941e73790 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.342+0000 7f5567de3740 1 --2- 172.21.15.37:0/345637186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557941e69010 0x557941e73250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.342+0000 7f5567de3740 1 -- 172.21.15.37:0/345637186 >> 172.21.15.37:0/345637186 conn(0x557941e64680 msgr2=0x557941e637e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.342+0000 7f5567de3740 1 -- 172.21.15.37:0/345637186 shutdown_connections 2024-06-10T01:03:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.342+0000 7f5567de3740 1 -- 172.21.15.37:0/345637186 wait complete. 2024-06-10T01:03:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.342+0000 7f5567de3740 20 client.26003 trim_cache size 0 max 16384 2024-06-10T01:03:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.359+0000 7f5567de3740 1 Processor -- start 2024-06-10T01:03:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.359+0000 7f5567de3740 1 -- start start 2024-06-10T01:03:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.359+0000 7f5567de3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557941e64680 0x557941e67590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.359+0000 7f5567de3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557941e69010 0x557941e7a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.359+0000 7f5567de3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557941e69a00 0x557941e67ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.359+0000 7f5567de3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f55601af290 con 0x557941e64680 2024-06-10T01:03:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.359+0000 7f5567de3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f55601acb60 con 0x557941e69a00 2024-06-10T01:03:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.359+0000 7f5567de3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f55601aa9b0 con 0x557941e69010 2024-06-10T01:03:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.360+0000 7f555ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557941e69a00 0x557941e67ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.360+0000 7f555e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557941e64680 0x557941e67590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.360+0000 7f555ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557941e69a00 0x557941e67ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:57138/0 (socket says 172.21.15.37:57138) 2024-06-10T01:03:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.360+0000 7f555e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557941e64680 0x557941e67590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58960/0 (socket says 172.21.15.37:58960) 2024-06-10T01:03:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.360+0000 7f555ed9d640 1 -- 172.21.15.37:0/1150047660 learned_addr learned my addr 172.21.15.37:0/1150047660 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.360+0000 7f555dd9b640 1 --2- 172.21.15.37:0/1150047660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557941e69010 0x557941e7a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.360+0000 7f555ed9d640 1 -- 172.21.15.37:0/1150047660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557941e69010 msgr2=0x557941e7a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.360+0000 7f555ed9d640 1 --2- 172.21.15.37:0/1150047660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557941e69010 0x557941e7a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.360+0000 7f555ed9d640 1 -- 172.21.15.37:0/1150047660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557941e64680 msgr2=0x557941e67590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.360+0000 7f555ed9d640 1 --2- 172.21.15.37:0/1150047660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557941e64680 0x557941e67590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.360+0000 7f555ed9d640 1 -- 172.21.15.37:0/1150047660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557941e644b0 con 0x557941e69a00 2024-06-10T01:03:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.360+0000 7f555e59c640 1 --2- 172.21.15.37:0/1150047660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557941e64680 0x557941e67590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:03:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.360+0000 7f555ed9d640 1 --2- 172.21.15.37:0/1150047660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557941e69a00 0x557941e67ad0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f555809bc10 tx=0x7f555809bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cfabff670b86932 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.360+0000 7f5547fff640 1 -- 172.21.15.37:0/1150047660 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55580b4070 con 0x557941e69a00 2024-06-10T01:03:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.360+0000 7f5547fff640 1 -- 172.21.15.37:0/1150047660 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f555809c610 con 0x557941e69a00 2024-06-10T01:03:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.360+0000 7f5547fff640 1 -- 172.21.15.37:0/1150047660 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558005310 con 0x557941e69a00 2024-06-10T01:03:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.361+0000 7f5567de3740 1 -- 172.21.15.37:0/1150047660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557941e69a00 msgr2=0x557941e67ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.361+0000 7f5567de3740 1 --2- 172.21.15.37:0/1150047660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557941e69a00 0x557941e67ad0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f555809bc10 tx=0x7f555809bce0 comp rx=0 tx=0).stop 2024-06-10T01:03:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.361+0000 7f5567de3740 1 -- 172.21.15.37:0/1150047660 shutdown_connections 2024-06-10T01:03:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.361+0000 7f5567de3740 1 --2- 172.21.15.37:0/1150047660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557941e69a00 0x557941e67ad0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.361+0000 7f5567de3740 1 --2- 172.21.15.37:0/1150047660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557941e69010 0x557941e7a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.361+0000 7f5567de3740 1 --2- 172.21.15.37:0/1150047660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557941e64680 0x557941e67590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.361+0000 7f5567de3740 1 -- 172.21.15.37:0/1150047660 >> 172.21.15.37:0/1150047660 conn(0x557941e6e640 msgr2=0x557941e6ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.361+0000 7f5567de3740 1 -- 172.21.15.37:0/1150047660 shutdown_connections 2024-06-10T01:03:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.361+0000 7f5567de3740 1 -- 172.21.15.37:0/1150047660 wait complete. 2024-06-10T01:03:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.361+0000 7f5567de3740 1 -- start start 2024-06-10T01:03:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.361+0000 7f5567de3740 1 Processor -- start 2024-06-10T01:03:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.362+0000 7f5567de3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557941e64680 0x557941e77f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.362+0000 7f5567de3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557941e69010 0x557941e784a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.362+0000 7f5567de3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557941e69a00 0x557941e794b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.362+0000 7f5567de3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f55601ab020 con 0x557941e64680 2024-06-10T01:03:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.362+0000 7f5567de3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f55601ac490 con 0x557941e69010 2024-06-10T01:03:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.362+0000 7f5567de3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f55601ab6f0 con 0x557941e69a00 2024-06-10T01:03:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.362+0000 7f555e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557941e64680 0x557941e77f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.362+0000 7f555e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557941e64680 0x557941e77f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58976/0 (socket says 172.21.15.37:58976) 2024-06-10T01:03:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.362+0000 7f555e59c640 1 -- 172.21.15.37:0/3691569726 learned_addr learned my addr 172.21.15.37:0/3691569726 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.362+0000 7f5567de3740 10 client.0 _ll_register_callbacks cb 0x7f5554003ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:03:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.362+0000 7f555dd9b640 1 --2- 172.21.15.37:0/3691569726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557941e69010 0x557941e784a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.362+0000 7f555ed9d640 1 --2- 172.21.15.37:0/3691569726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557941e69a00 0x557941e794b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.362+0000 7f555e59c640 1 -- 172.21.15.37:0/3691569726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557941e69a00 msgr2=0x557941e794b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.362+0000 7f555e59c640 1 --2- 172.21.15.37:0/3691569726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557941e69a00 0x557941e794b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.362+0000 7f555e59c640 1 -- 172.21.15.37:0/3691569726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557941e69010 msgr2=0x557941e784a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.362+0000 7f555e59c640 1 --2- 172.21.15.37:0/3691569726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557941e69010 0x557941e784a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.362+0000 7f555e59c640 1 -- 172.21.15.37:0/3691569726 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557941e79aa0 con 0x557941e64680 2024-06-10T01:03:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.362+0000 7f555e59c640 1 --2- 172.21.15.37:0/3691569726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557941e64680 0x557941e77f60 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5550009900 tx=0x7f55500044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38f2341c66e0e84 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.362+0000 7f5545ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:03:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.363+0000 7f5545ffb640 1 -- 172.21.15.37:0/3691569726 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550035040 con 0x557941e64680 2024-06-10T01:03:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.363+0000 7f5545ffb640 1 -- 172.21.15.37:0/3691569726 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9237+0+0 (secure 0 0 0) 0x7f5550031ac0 con 0x557941e64680 2024-06-10T01:03:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.363+0000 7f5567de3740 1 -- 172.21.15.37:0/3691569726 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f555809c610 con 0x557941e64680 2024-06-10T01:03:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.363+0000 7f5545ffb640 1 -- 172.21.15.37:0/3691569726 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5550060050 con 0x557941e64680 2024-06-10T01:03:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.363+0000 7f5567de3740 10 client.16545 fetch_fsmap learned FSMap version 27 2024-06-10T01:03:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.363+0000 7f5567de3740 1 -- 172.21.15.37:0/3691569726 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557941ee9f20 con 0x557941e64680 2024-06-10T01:03:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.363+0000 7f5545ffb640 1 -- 172.21.15.37:0/3691569726 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f555809c610 con 0x557941e64680 2024-06-10T01:03:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.363+0000 7f5545ffb640 1 -- 172.21.15.37:0/3691569726 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5550050450 con 0x557941e64680 2024-06-10T01:03:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.363+0000 7f5567de3740 10 client.16545 fetch_fsmap finished waiting for FSMap version 27 2024-06-10T01:03:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.363+0000 7f5567de3740 10 client.16545 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:03:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.363+0000 7f5567de3740 20 client.16545 populate_metadata read hostname 'smithi037' 2024-06-10T01:03:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.363+0000 7f5567de3740 1 --2- 172.21.15.37:0/3691569726 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557941e6eb70 0x557941eec4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.363+0000 7f5567de3740 4 client.16545 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:03:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.363+0000 7f5567de3740 1 -- 172.21.15.37:0/3691569726 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557941dd9c20 con 0x557941e6eb70 2024-06-10T01:03:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.363+0000 7f555dd9b640 1 --2- 172.21.15.37:0/3691569726 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557941e6eb70 0x557941eec4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.364+0000 7f5545ffb640 10 client.16545 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:03:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:22.364+0000 7f555dd9b640 1 --2- 172.21.15.37:0/3691569726 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557941e6eb70 0x557941eec4f0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e23d5621c8d1c462 server_cookie=825e8cbe115c3a31 in_seq=0 out_seq=0 2024-06-10T01:03:22.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:22 smithi167 ceph-mon[36917]: pgmap v630: 129 pgs: 129 active+clean; 16 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 1.5 MiB/s wr, 914 op/s 2024-06-10T01:03:22.586 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:03:22 smithi167 systemd[1]: ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.f.service: Scheduled restart job, restart counter is at 1. 2024-06-10T01:03:22.586 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:03:22 smithi167 systemd[1]: Stopped Ceph mds.f for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T01:03:22.586 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:03:22 smithi167 systemd[1]: ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.f.service: Consumed 12.813s CPU time. 2024-06-10T01:03:22.586 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:03:22 smithi167 systemd[1]: Starting Ceph mds.f for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:03:22.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:22 smithi094 ceph-mon[36708]: pgmap v630: 129 pgs: 129 active+clean; 16 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 1.5 MiB/s wr, 914 op/s 2024-06-10T01:03:22.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:22 smithi037 ceph-mon[32192]: pgmap v630: 129 pgs: 129 active+clean; 16 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 1.5 MiB/s wr, 914 op/s 2024-06-10T01:03:23.056 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:03:22 smithi167 podman[71133]: 2024-06-10 01:03:22.50145998 +0000 UTC m=+0.025379801 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:03:23.336 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:03:23 smithi167 podman[71133]: 2024-06-10 01:03:23.059368722 +0000 UTC m=+0.583288526 container create c384814da1b9fad8ed928ad1a4746a1a016335f498e4bea58ae53d68e0e0e012 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-f, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-10T01:03:23.835 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:03:23 smithi167 podman[71133]: 2024-06-10 01:03:23.354877367 +0000 UTC m=+0.878797178 container init c384814da1b9fad8ed928ad1a4746a1a016335f498e4bea58ae53d68e0e0e012 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-f, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-10T01:03:23.836 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:03:23 smithi167 podman[71133]: 2024-06-10 01:03:23.368775553 +0000 UTC m=+0.892695364 container start c384814da1b9fad8ed928ad1a4746a1a016335f498e4bea58ae53d68e0e0e012 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-f, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-06-10T01:03:23.836 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:03:23 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-f[71145]: starting mds.f at 2024-06-10T01:03:23.836 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:03:23 smithi167 bash[71133]: c384814da1b9fad8ed928ad1a4746a1a016335f498e4bea58ae53d68e0e0e012 2024-06-10T01:03:23.836 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:03:23 smithi167 systemd[1]: Started Ceph mds.f for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T01:03:24.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:24 smithi167 ceph-mon[36917]: pgmap v631: 129 pgs: 129 active+clean; 21 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.7 MiB/s wr, 741 op/s 2024-06-10T01:03:24.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:24 smithi167 ceph-mon[36917]: daemon mds.f restarted 2024-06-10T01:03:24.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:24 smithi094 ceph-mon[36708]: pgmap v631: 129 pgs: 129 active+clean; 21 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.7 MiB/s wr, 741 op/s 2024-06-10T01:03:24.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:24 smithi094 ceph-mon[36708]: daemon mds.f restarted 2024-06-10T01:03:24.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:24 smithi037 ceph-mon[32192]: pgmap v631: 129 pgs: 129 active+clean; 21 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.7 MiB/s wr, 741 op/s 2024-06-10T01:03:24.643 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:24 smithi037 ceph-mon[32192]: daemon mds.f restarted 2024-06-10T01:03:25.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:25 smithi167 ceph-mon[36917]: osdmap e101: 12 total, 12 up, 12 in 2024-06-10T01:03:25.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:25 smithi167 ceph-mon[36917]: mds.? [v2:172.21.15.167:6834/317977487,v1:172.21.15.167:6835/317977487] up:boot 2024-06-10T01:03:25.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:25 smithi167 ceph-mon[36917]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-06-10T01:03:25.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:25 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T01:03:25.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:25 smithi167 ceph-mon[36917]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:03:25.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:25 smithi094 ceph-mon[36708]: osdmap e101: 12 total, 12 up, 12 in 2024-06-10T01:03:25.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:25 smithi094 ceph-mon[36708]: mds.? [v2:172.21.15.167:6834/317977487,v1:172.21.15.167:6835/317977487] up:boot 2024-06-10T01:03:25.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:25 smithi094 ceph-mon[36708]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-06-10T01:03:25.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:25 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T01:03:25.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:25 smithi094 ceph-mon[36708]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:03:25.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:25 smithi037 ceph-mon[32192]: osdmap e101: 12 total, 12 up, 12 in 2024-06-10T01:03:25.643 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:25 smithi037 ceph-mon[32192]: mds.? [v2:172.21.15.167:6834/317977487,v1:172.21.15.167:6835/317977487] up:boot 2024-06-10T01:03:25.643 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:25 smithi037 ceph-mon[32192]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-06-10T01:03:25.643 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:25 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T01:03:25.643 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:25 smithi037 ceph-mon[32192]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:03:26.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:26 smithi167 ceph-mon[36917]: pgmap v633: 129 pgs: 129 active+clean; 24 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1.9 MiB/s wr, 987 op/s 2024-06-10T01:03:26.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:26 smithi094 ceph-mon[36708]: pgmap v633: 129 pgs: 129 active+clean; 24 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1.9 MiB/s wr, 987 op/s 2024-06-10T01:03:26.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:26 smithi037 ceph-mon[32192]: pgmap v633: 129 pgs: 129 active+clean; 24 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1.9 MiB/s wr, 987 op/s 2024-06-10T01:03:28.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:28 smithi167 ceph-mon[36917]: pgmap v634: 129 pgs: 129 active+clean; 24 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.5 MiB/s wr, 941 op/s 2024-06-10T01:03:28.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:28 smithi094 ceph-mon[36708]: pgmap v634: 129 pgs: 129 active+clean; 24 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.5 MiB/s wr, 941 op/s 2024-06-10T01:03:28.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:28 smithi037 ceph-mon[32192]: pgmap v634: 129 pgs: 129 active+clean; 24 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.5 MiB/s wr, 941 op/s 2024-06-10T01:03:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:29.049+0000 7f5545ffb640 1 -- 172.21.15.37:0/3691569726 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557941dd9c20 con 0x557941e6eb70 2024-06-10T01:03:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:29.049+0000 7f5545ffb640 10 client.16545 handle_command_reply: tid=0 2024-06-10T01:03:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:29.049+0000 7f5567de3740 1 client.16545 shutdown 2024-06-10T01:03:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:29.049+0000 7f5567de3740 1 -- 172.21.15.37:0/3691569726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557941e64680 msgr2=0x557941e77f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:29.049+0000 7f5567de3740 1 --2- 172.21.15.37:0/3691569726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557941e64680 0x557941e77f60 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5550009900 tx=0x7f55500044c0 comp rx=0 tx=0).stop 2024-06-10T01:03:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:29.050+0000 7f5567de3740 1 -- 172.21.15.37:0/3691569726 shutdown_connections 2024-06-10T01:03:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:29.050+0000 7f5567de3740 1 --2- 172.21.15.37:0/3691569726 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557941e6eb70 0x557941eec4f0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:29.050+0000 7f5567de3740 1 --2- 172.21.15.37:0/3691569726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557941e69a00 0x557941e794b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:29.050+0000 7f5567de3740 1 --2- 172.21.15.37:0/3691569726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557941e69010 0x557941e784a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:29.050+0000 7f5545ffb640 0 client.16545 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:03:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:29.050+0000 7f5567de3740 1 --2- 172.21.15.37:0/3691569726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557941e64680 0x557941e77f60 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:29.050+0000 7f5567de3740 1 -- 172.21.15.37:0/3691569726 >> 172.21.15.37:0/3691569726 conn(0x557941e6e640 msgr2=0x557941e6ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:29.050+0000 7f5567de3740 1 -- 172.21.15.37:0/3691569726 shutdown_connections 2024-06-10T01:03:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:29.050+0000 7f5567de3740 1 -- 172.21.15.37:0/3691569726 wait complete. 2024-06-10T01:03:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:29.050+0000 7f5567de3740 20 client.16545 trim_cache size 0 max 16384 2024-06-10T01:03:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:03:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:03:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:03:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:29.050+0000 7f5564ff9640 1 -- 172.21.15.37:0/96961427 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f55480a2cb0 msgr2=0x7f55480a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:29.050+0000 7f5564ff9640 1 --2- 172.21.15.37:0/96961427 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f55480a2cb0 0x7f55480a5170 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x557941e6a260 tx=0x7f5554005f90 comp rx=0 tx=0).stop 2024-06-10T01:03:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:29.050+0000 7f5564ff9640 1 -- 172.21.15.37:0/96961427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55601076e0 msgr2=0x7f5560100970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:29.050+0000 7f5564ff9640 1 --2- 172.21.15.37:0/96961427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55601076e0 0x7f5560100970 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f555800eb10 tx=0x7f555800c2d0 comp rx=0 tx=0).stop 2024-06-10T01:03:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:29.050+0000 7f5564ff9640 1 -- 172.21.15.37:0/96961427 shutdown_connections 2024-06-10T01:03:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:29.050+0000 7f5564ff9640 1 --2- 172.21.15.37:0/96961427 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f55480a2cb0 0x7f55480a5170 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:29.050+0000 7f5564ff9640 1 --2- 172.21.15.37:0/96961427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55601076e0 0x7f5560100970 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:29.050+0000 7f5564ff9640 1 --2- 172.21.15.37:0/96961427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5560104db0 0x7f5560100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:29.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:29.050+0000 7f5564ff9640 1 --2- 172.21.15.37:0/96961427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5560102480 0x7f5560101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:29.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:29.050+0000 7f5564ff9640 1 -- 172.21.15.37:0/96961427 >> 172.21.15.37:0/96961427 conn(0x7f55600fbe70 msgr2=0x7f5560102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:29.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:29.051+0000 7f5564ff9640 1 -- 172.21.15.37:0/96961427 shutdown_connections 2024-06-10T01:03:29.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:29.051+0000 7f5564ff9640 1 -- 172.21.15.37:0/96961427 wait complete. 2024-06-10T01:03:30.344 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:03:30.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:30 smithi167 ceph-mon[36917]: pgmap v635: 129 pgs: 129 active+clean; 27 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 1.34k op/s 2024-06-10T01:03:30.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:30 smithi094 ceph-mon[36708]: pgmap v635: 129 pgs: 129 active+clean; 27 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 1.34k op/s 2024-06-10T01:03:30.637 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:30.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:30 smithi037 ceph-mon[32192]: pgmap v635: 129 pgs: 129 active+clean; 27 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 1.34k op/s 2024-06-10T01:03:32.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:32 smithi167 ceph-mon[36917]: pgmap v636: 129 pgs: 129 active+clean; 23 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 1.32k op/s 2024-06-10T01:03:32.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:32 smithi094 ceph-mon[36708]: pgmap v636: 129 pgs: 129 active+clean; 23 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 1.32k op/s 2024-06-10T01:03:32.611 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:32 smithi037 ceph-mon[32192]: pgmap v636: 129 pgs: 129 active+clean; 23 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 1.32k op/s 2024-06-10T01:03:32.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.871+0000 7f4973b5a700 1 -- 172.21.15.37:0/1069402777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f496c101380 msgr2=0x7f496c101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:32.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.871+0000 7f4973b5a700 1 --2- 172.21.15.37:0/1069402777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f496c101380 0x7f496c101800 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f4960009990 tx=0x7f4960009ca0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.871+0000 7f4973b5a700 1 -- 172.21.15.37:0/1069402777 shutdown_connections 2024-06-10T01:03:32.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.871+0000 7f4973b5a700 1 --2- 172.21.15.37:0/1069402777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f496c101d40 0x7f496c10e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.871+0000 7f4973b5a700 1 --2- 172.21.15.37:0/1069402777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f496c1073f0 0x7f496c1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.871+0000 7f4973b5a700 1 --2- 172.21.15.37:0/1069402777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f496c101380 0x7f496c101800 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.871+0000 7f4973b5a700 1 -- 172.21.15.37:0/1069402777 >> 172.21.15.37:0/1069402777 conn(0x7f496c078300 msgr2=0x7f496c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:32.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.871+0000 7f4973b5a700 1 -- 172.21.15.37:0/1069402777 shutdown_connections 2024-06-10T01:03:32.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.871+0000 7f4973b5a700 1 -- 172.21.15.37:0/1069402777 wait complete. 2024-06-10T01:03:32.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.872+0000 7f4973b5a700 1 Processor -- start 2024-06-10T01:03:32.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.872+0000 7f4973b5a700 1 -- start start 2024-06-10T01:03:32.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.872+0000 7f4973b5a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f496c101380 0x7f496c19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:32.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.873+0000 7f4973b5a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f496c101d40 0x7f496c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:32.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.873+0000 7f4973b5a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f496c1073f0 0x7f496c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:32.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.873+0000 7f4973b5a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f496c06b450 con 0x7f496c1073f0 2024-06-10T01:03:32.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.873+0000 7f4973b5a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f496c06b2d0 con 0x7f496c101d40 2024-06-10T01:03:32.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.873+0000 7f4973b5a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f496c06b5d0 con 0x7f496c101380 2024-06-10T01:03:32.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.873+0000 7f49718f6700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f496c101380 0x7f496c19e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:32.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.873+0000 7f49720f7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f496c1073f0 0x7f496c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:32.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.873+0000 7f49718f6700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f496c101380 0x7f496c19e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:44048/0 (socket says 172.21.15.37:44048) 2024-06-10T01:03:32.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.873+0000 7f49718f6700 1 -- 172.21.15.37:0/2776425267 learned_addr learned my addr 172.21.15.37:0/2776425267 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:32.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.873+0000 7f49710f5700 1 --2- 172.21.15.37:0/2776425267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f496c101d40 0x7f496c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:32.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.873+0000 7f49718f6700 1 -- 172.21.15.37:0/2776425267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f496c101d40 msgr2=0x7f496c19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:32.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.873+0000 7f49718f6700 1 --2- 172.21.15.37:0/2776425267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f496c101d40 0x7f496c19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.873+0000 7f49718f6700 1 -- 172.21.15.37:0/2776425267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f496c1073f0 msgr2=0x7f496c198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:32.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.873+0000 7f49718f6700 1 --2- 172.21.15.37:0/2776425267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f496c1073f0 0x7f496c198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.873+0000 7f49718f6700 1 -- 172.21.15.37:0/2776425267 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f496c199160 con 0x7f496c101380 2024-06-10T01:03:32.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.873+0000 7f49720f7700 1 --2- 172.21.15.37:0/2776425267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f496c1073f0 0x7f496c198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:03:32.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.873+0000 7f49710f5700 1 --2- 172.21.15.37:0/2776425267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f496c101d40 0x7f496c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:03:32.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.873+0000 7f49718f6700 1 --2- 172.21.15.37:0/2776425267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f496c101380 0x7f496c19e7f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f495c00ef30 tx=0x7f495c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:32.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.873+0000 7f496affd700 1 -- 172.21.15.37:0/2776425267 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c0092e0 con 0x7f496c101380 2024-06-10T01:03:32.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.874+0000 7f496affd700 1 -- 172.21.15.37:0/2776425267 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f495c004bb0 con 0x7f496c101380 2024-06-10T01:03:32.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.874+0000 7f4973b5a700 1 -- 172.21.15.37:0/2776425267 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f496c1993f0 con 0x7f496c101380 2024-06-10T01:03:32.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.874+0000 7f496affd700 1 -- 172.21.15.37:0/2776425267 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c004280 con 0x7f496c101380 2024-06-10T01:03:32.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.874+0000 7f4973b5a700 1 -- 172.21.15.37:0/2776425267 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f496c1a56c0 con 0x7f496c101380 2024-06-10T01:03:32.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.877+0000 7f496affd700 1 -- 172.21.15.37:0/2776425267 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f495c01b030 con 0x7f496c101380 2024-06-10T01:03:32.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.878+0000 7f496affd700 1 --2- 172.21.15.37:0/2776425267 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f49540a2c20 0x7f49540a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:32.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.878+0000 7f49710f5700 1 --2- 172.21.15.37:0/2776425267 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f49540a2c20 0x7f49540a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:32.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.878+0000 7f496affd700 1 -- 172.21.15.37:0/2776425267 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f495c0cef80 con 0x7f496c101380 2024-06-10T01:03:32.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.878+0000 7f497e3ddb80 1 Processor -- start 2024-06-10T01:03:32.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.879+0000 7f497e3ddb80 1 -- start start 2024-06-10T01:03:32.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.879+0000 7f49710f5700 1 --2- 172.21.15.37:0/2776425267 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f49540a2c20 0x7f49540a50e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f496000b2e0 tx=0x7f4960005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:32.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.879+0000 7f497e3ddb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555f3dcf2470 0x555f3dcf2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:32.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.879+0000 7f497e3ddb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f3dcec670 0x555f3dcecaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:32.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.879+0000 7f497e3ddb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f3dced030 0x555f3dcf9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:32.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.879+0000 7f497e3ddb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555f3dbdaf60 con 0x555f3dced030 2024-06-10T01:03:32.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.879+0000 7f497e3ddb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555f3dc39850 con 0x555f3dcec670 2024-06-10T01:03:32.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.879+0000 7f497e3ddb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555f3dba1460 con 0x555f3dcf2470 2024-06-10T01:03:32.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.879+0000 7f49718f6700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555f3dcf2470 0x555f3dcf2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:32.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.879+0000 7f49710f5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f3dcec670 0x555f3dcecaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:32.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.879+0000 7f49718f6700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555f3dcf2470 0x555f3dcf2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:44060/0 (socket says 172.21.15.37:44060) 2024-06-10T01:03:32.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.879+0000 7f49718f6700 1 -- 172.21.15.37:0/1588943332 learned_addr learned my addr 172.21.15.37:0/1588943332 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:32.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.879+0000 7f49720f7700 1 --2- 172.21.15.37:0/1588943332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f3dced030 0x555f3dcf9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:32.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.879+0000 7f49710f5700 1 -- 172.21.15.37:0/1588943332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555f3dcf2470 msgr2=0x555f3dcf2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:32.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.879+0000 7f49710f5700 1 --2- 172.21.15.37:0/1588943332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555f3dcf2470 0x555f3dcf2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.879+0000 7f49710f5700 1 -- 172.21.15.37:0/1588943332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f3dced030 msgr2=0x555f3dcf9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:32.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.879+0000 7f49710f5700 1 --2- 172.21.15.37:0/1588943332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f3dced030 0x555f3dcf9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.879+0000 7f49710f5700 1 -- 172.21.15.37:0/1588943332 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f3db888d0 con 0x555f3dcec670 2024-06-10T01:03:32.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.880+0000 7f49710f5700 1 --2- 172.21.15.37:0/1588943332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f3dcec670 0x555f3dcecaf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f49600059d0 tx=0x7f4960020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3316a9dd5d7a2a3b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:32.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.880+0000 7f4953fff700 1 -- 172.21.15.37:0/1588943332 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4960003680 con 0x555f3dcec670 2024-06-10T01:03:32.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.880+0000 7f4953fff700 1 -- 172.21.15.37:0/1588943332 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4960003d00 con 0x555f3dcec670 2024-06-10T01:03:32.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.880+0000 7f4953fff700 1 -- 172.21.15.37:0/1588943332 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4960024410 con 0x555f3dcec670 2024-06-10T01:03:32.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.881+0000 7f497e3ddb80 1 -- 172.21.15.37:0/1588943332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f3dcec670 msgr2=0x555f3dcecaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:32.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.881+0000 7f497e3ddb80 1 --2- 172.21.15.37:0/1588943332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f3dcec670 0x555f3dcecaf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f49600059d0 tx=0x7f4960020000 comp rx=0 tx=0).stop 2024-06-10T01:03:32.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.881+0000 7f497e3ddb80 1 -- 172.21.15.37:0/1588943332 shutdown_connections 2024-06-10T01:03:32.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.881+0000 7f497e3ddb80 1 --2- 172.21.15.37:0/1588943332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f3dced030 0x555f3dcf9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.881+0000 7f497e3ddb80 1 --2- 172.21.15.37:0/1588943332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555f3dcf2470 0x555f3dcf2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.881+0000 7f497e3ddb80 1 --2- 172.21.15.37:0/1588943332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f3dcec670 0x555f3dcecaf0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.881+0000 7f497e3ddb80 1 -- 172.21.15.37:0/1588943332 >> 172.21.15.37:0/1588943332 conn(0x555f3dc15080 msgr2=0x555f3dce3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:32.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.881+0000 7f497e3ddb80 1 -- 172.21.15.37:0/1588943332 shutdown_connections 2024-06-10T01:03:32.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.881+0000 7f497e3ddb80 1 -- 172.21.15.37:0/1588943332 wait complete. 2024-06-10T01:03:32.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.881+0000 7f497e3ddb80 1 -- start start 2024-06-10T01:03:32.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.881+0000 7f497e3ddb80 1 Processor -- start 2024-06-10T01:03:32.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.882+0000 7f497e3ddb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555f3dcec670 0x555f3dcf8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:32.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.882+0000 7f497e3ddb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f3dced030 0x555f3dcf61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:32.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.882+0000 7f497e3ddb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f3dcf2470 0x555f3dcf6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:32.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.882+0000 7f497e3ddb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555f3dbd9fc0 con 0x555f3dced030 2024-06-10T01:03:32.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.882+0000 7f497e3ddb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555f3dc35810 con 0x555f3dcf2470 2024-06-10T01:03:32.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.882+0000 7f497e3ddb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555f3dcf9750 con 0x555f3dcec670 2024-06-10T01:03:32.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.882+0000 7f497e3ddb80 10 client.0 _ll_register_callbacks cb 0x555f3dbcf5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:03:32.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.882+0000 7f49710f5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f3dced030 0x555f3dcf61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:32.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.882+0000 7f49710f5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f3dced030 0x555f3dcf61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47308/0 (socket says 172.21.15.37:47308) 2024-06-10T01:03:32.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.882+0000 7f49710f5700 1 -- 172.21.15.37:0/1748693805 learned_addr learned my addr 172.21.15.37:0/1748693805 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:32.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.882+0000 7f49718f6700 1 --2- 172.21.15.37:0/1748693805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555f3dcec670 0x555f3dcf8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:32.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.882+0000 7f49720f7700 1 --2- 172.21.15.37:0/1748693805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f3dcf2470 0x555f3dcf6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:32.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.882+0000 7f49710f5700 1 -- 172.21.15.37:0/1748693805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555f3dcec670 msgr2=0x555f3dcf8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:32.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.882+0000 7f49710f5700 1 --2- 172.21.15.37:0/1748693805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555f3dcec670 0x555f3dcf8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.882+0000 7f49710f5700 1 -- 172.21.15.37:0/1748693805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f3dcf2470 msgr2=0x555f3dcf6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:32.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.882+0000 7f49710f5700 1 --2- 172.21.15.37:0/1748693805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f3dcf2470 0x555f3dcf6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.882+0000 7f49710f5700 1 -- 172.21.15.37:0/1748693805 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f3dbff750 con 0x555f3dced030 2024-06-10T01:03:32.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.882+0000 7f49718f6700 1 --2- 172.21.15.37:0/1748693805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555f3dcec670 0x555f3dcf8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:03:32.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.882+0000 7f49720f7700 1 --2- 172.21.15.37:0/1748693805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f3dcf2470 0x555f3dcf6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:03:32.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.883+0000 7f49710f5700 1 --2- 172.21.15.37:0/1748693805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f3dced030 0x555f3dcf61f0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f496000c010 tx=0x7f496000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c2e05aa5258bbe server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:32.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.883+0000 7f49517fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:03:32.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.883+0000 7f49517fa700 1 -- 172.21.15.37:0/1748693805 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496003c060 con 0x555f3dced030 2024-06-10T01:03:32.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.883+0000 7f49517fa700 1 -- 172.21.15.37:0/1748693805 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f496002c040 con 0x555f3dced030 2024-06-10T01:03:32.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.883+0000 7f497e3ddb80 1 -- 172.21.15.37:0/1748693805 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4960003d00 con 0x555f3dced030 2024-06-10T01:03:32.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.883+0000 7f49517fa700 1 -- 172.21.15.37:0/1748693805 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4960003680 con 0x555f3dced030 2024-06-10T01:03:32.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.883+0000 7f497e3ddb80 10 client.16560 fetch_fsmap learned FSMap version 29 2024-06-10T01:03:32.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.883+0000 7f497e3ddb80 1 -- 172.21.15.37:0/1748693805 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555f3dd03d50 con 0x555f3dced030 2024-06-10T01:03:32.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.883+0000 7f49517fa700 1 -- 172.21.15.37:0/1748693805 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4960003d00 con 0x555f3dced030 2024-06-10T01:03:32.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.883+0000 7f49517fa700 1 -- 172.21.15.37:0/1748693805 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f4960024b40 con 0x555f3dced030 2024-06-10T01:03:32.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.883+0000 7f497e3ddb80 10 client.16560 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:03:32.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.883+0000 7f497e3ddb80 10 client.16560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:03:32.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.883+0000 7f497e3ddb80 20 client.16560 populate_metadata read hostname 'smithi037' 2024-06-10T01:03:32.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.884+0000 7f497e3ddb80 1 --2- 172.21.15.37:0/1748693805 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x555f3dd06220 0x555f3dd08610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:32.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.884+0000 7f497e3ddb80 4 client.16560 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:03:32.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.884+0000 7f497e3ddb80 1 -- 172.21.15.37:0/1748693805 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f3dc46d50 con 0x555f3dd06220 2024-06-10T01:03:32.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.884+0000 7f49718f6700 1 --2- 172.21.15.37:0/1748693805 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x555f3dd06220 0x555f3dd08610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:32.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.884+0000 7f49718f6700 1 --2- 172.21.15.37:0/1748693805 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x555f3dd06220 0x555f3dd08610 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9e042f63d1aeebb server_cookie=e7202a9cc51e25f3 in_seq=0 out_seq=0 2024-06-10T01:03:32.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.884+0000 7f49517fa700 10 client.16560 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:03:32.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.887+0000 7f49517fa700 1 -- 172.21.15.37:0/1748693805 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555f3dc46d50 con 0x555f3dd06220 2024-06-10T01:03:32.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.887+0000 7f49517fa700 10 client.16560 handle_command_reply: tid=0 2024-06-10T01:03:32.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.887+0000 7f497e3ddb80 1 client.16560 shutdown 2024-06-10T01:03:32.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.887+0000 7f497e3ddb80 1 -- 172.21.15.37:0/1748693805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f3dced030 msgr2=0x555f3dcf61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:32.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.887+0000 7f497e3ddb80 1 --2- 172.21.15.37:0/1748693805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f3dced030 0x555f3dcf61f0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f496000c010 tx=0x7f496000f690 comp rx=0 tx=0).stop 2024-06-10T01:03:32.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.887+0000 7f497e3ddb80 1 -- 172.21.15.37:0/1748693805 shutdown_connections 2024-06-10T01:03:32.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.887+0000 7f497e3ddb80 1 --2- 172.21.15.37:0/1748693805 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x555f3dd06220 0x555f3dd08610 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.887+0000 7f497e3ddb80 1 --2- 172.21.15.37:0/1748693805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f3dcf2470 0x555f3dcf6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.887+0000 7f497e3ddb80 1 --2- 172.21.15.37:0/1748693805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555f3dcec670 0x555f3dcf8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.887+0000 7f497e3ddb80 1 --2- 172.21.15.37:0/1748693805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f3dced030 0x555f3dcf61f0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.887+0000 7f49517fa700 0 client.16560 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:03:32.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.887+0000 7f497e3ddb80 1 -- 172.21.15.37:0/1748693805 >> 172.21.15.37:0/1748693805 conn(0x555f3dc15080 msgr2=0x555f3dce2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:32.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.887+0000 7f497e3ddb80 1 -- 172.21.15.37:0/1748693805 shutdown_connections 2024-06-10T01:03:32.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.887+0000 7f497e3ddb80 1 -- 172.21.15.37:0/1748693805 wait complete. 2024-06-10T01:03:32.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.887+0000 7f497e3ddb80 20 client.16560 trim_cache size 0 max 16384 2024-06-10T01:03:32.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.910+0000 7f497e3ddb80 1 Processor -- start 2024-06-10T01:03:32.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.910+0000 7f497e3ddb80 1 -- start start 2024-06-10T01:03:32.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.910+0000 7f497e3ddb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555f3dcec670 0x555f3dcf1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:32.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.910+0000 7f497e3ddb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f3dced030 0x555f3dce2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:32.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.910+0000 7f497e3ddb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f3dcf2470 0x555f3dce33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:32.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.910+0000 7f497e3ddb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f496c1aea00 con 0x555f3dced030 2024-06-10T01:03:32.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.910+0000 7f497e3ddb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f496c1ac290 con 0x555f3dcf2470 2024-06-10T01:03:32.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.910+0000 7f497e3ddb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f496c1aa100 con 0x555f3dcec670 2024-06-10T01:03:32.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.910+0000 7f49710f5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f3dced030 0x555f3dce2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:32.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.910+0000 7f49710f5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f3dced030 0x555f3dce2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47324/0 (socket says 172.21.15.37:47324) 2024-06-10T01:03:32.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.910+0000 7f49710f5700 1 -- 172.21.15.37:0/1413573276 learned_addr learned my addr 172.21.15.37:0/1413573276 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:32.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.910+0000 7f49720f7700 1 --2- 172.21.15.37:0/1413573276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f3dcf2470 0x555f3dce33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:32.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.910+0000 7f49718f6700 1 --2- 172.21.15.37:0/1413573276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555f3dcec670 0x555f3dcf1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:32.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.910+0000 7f49720f7700 1 -- 172.21.15.37:0/1413573276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555f3dcec670 msgr2=0x555f3dcf1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:32.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.910+0000 7f49720f7700 1 --2- 172.21.15.37:0/1413573276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555f3dcec670 0x555f3dcf1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.910+0000 7f49720f7700 1 -- 172.21.15.37:0/1413573276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f3dced030 msgr2=0x555f3dce2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:32.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.910+0000 7f49720f7700 1 --2- 172.21.15.37:0/1413573276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f3dced030 0x555f3dce2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.910+0000 7f49720f7700 1 -- 172.21.15.37:0/1413573276 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f3dcf4780 con 0x555f3dcf2470 2024-06-10T01:03:32.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.910+0000 7f49718f6700 1 --2- 172.21.15.37:0/1413573276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555f3dcec670 0x555f3dcf1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:03:32.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.910+0000 7f49710f5700 1 --2- 172.21.15.37:0/1413573276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f3dced030 0x555f3dce2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:03:32.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.911+0000 7f49720f7700 1 --2- 172.21.15.37:0/1413573276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f3dcf2470 0x555f3dce33c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4964012a80 tx=0x7f4964009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a352ea9624054614 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:32.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.911+0000 7f49537fe700 1 -- 172.21.15.37:0/1413573276 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4964022070 con 0x555f3dcf2470 2024-06-10T01:03:32.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.911+0000 7f49537fe700 1 -- 172.21.15.37:0/1413573276 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4964012e50 con 0x555f3dcf2470 2024-06-10T01:03:32.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.911+0000 7f49537fe700 1 -- 172.21.15.37:0/1413573276 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49640038e0 con 0x555f3dcf2470 2024-06-10T01:03:32.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.911+0000 7f497e3ddb80 1 -- 172.21.15.37:0/1413573276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f3dcf2470 msgr2=0x555f3dce33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:32.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.911+0000 7f497e3ddb80 1 --2- 172.21.15.37:0/1413573276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f3dcf2470 0x555f3dce33c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4964012a80 tx=0x7f4964009260 comp rx=0 tx=0).stop 2024-06-10T01:03:32.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.912+0000 7f497e3ddb80 1 -- 172.21.15.37:0/1413573276 shutdown_connections 2024-06-10T01:03:32.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.912+0000 7f497e3ddb80 1 --2- 172.21.15.37:0/1413573276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f3dcf2470 0x555f3dce33c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.912+0000 7f497e3ddb80 1 --2- 172.21.15.37:0/1413573276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555f3dcec670 0x555f3dcf1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.912+0000 7f497e3ddb80 1 --2- 172.21.15.37:0/1413573276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f3dced030 0x555f3dce2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.912+0000 7f497e3ddb80 1 -- 172.21.15.37:0/1413573276 >> 172.21.15.37:0/1413573276 conn(0x555f3dc15080 msgr2=0x555f3dce7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:32.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.912+0000 7f497e3ddb80 1 -- 172.21.15.37:0/1413573276 shutdown_connections 2024-06-10T01:03:32.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.912+0000 7f497e3ddb80 1 -- 172.21.15.37:0/1413573276 wait complete. 2024-06-10T01:03:32.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.912+0000 7f497e3ddb80 1 -- start start 2024-06-10T01:03:32.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.912+0000 7f497e3ddb80 1 Processor -- start 2024-06-10T01:03:32.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.913+0000 7f497e3ddb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555f3dcec670 0x555f3dd0baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:32.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.913+0000 7f497e3ddb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f3dced030 0x555f3dd08e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:32.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.913+0000 7f497e3ddb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f3dcf2470 0x555f3dd093d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:32.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.913+0000 7f497e3ddb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f496c1aa750 con 0x555f3dced030 2024-06-10T01:03:32.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.913+0000 7f497e3ddb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f496c1abbc0 con 0x555f3dcf2470 2024-06-10T01:03:32.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.913+0000 7f497e3ddb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f496c1aae20 con 0x555f3dcec670 2024-06-10T01:03:32.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.913+0000 7f497e3ddb80 10 client.0 _ll_register_callbacks cb 0x555f3dbcf5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:03:32.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.913+0000 7f49710f5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f3dced030 0x555f3dd08e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:32.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.913+0000 7f49718f6700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555f3dcec670 0x555f3dd0baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:32.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.913+0000 7f49718f6700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555f3dcec670 0x555f3dd0baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:44094/0 (socket says 172.21.15.37:44094) 2024-06-10T01:03:32.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.913+0000 7f49710f5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f3dced030 0x555f3dd08e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47334/0 (socket says 172.21.15.37:47334) 2024-06-10T01:03:32.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.913+0000 7f49718f6700 1 -- 172.21.15.37:0/2463288269 learned_addr learned my addr 172.21.15.37:0/2463288269 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:32.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.913+0000 7f49720f7700 1 --2- 172.21.15.37:0/2463288269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f3dcf2470 0x555f3dd093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:32.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.913+0000 7f49720f7700 1 -- 172.21.15.37:0/2463288269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555f3dcec670 msgr2=0x555f3dd0baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.913+0000 7f49720f7700 1 --2- 172.21.15.37:0/2463288269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555f3dcec670 0x555f3dd0baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.913+0000 7f49720f7700 1 -- 172.21.15.37:0/2463288269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f3dced030 msgr2=0x555f3dd08e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.913+0000 7f49720f7700 1 --2- 172.21.15.37:0/2463288269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f3dced030 0x555f3dd08e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.913+0000 7f49720f7700 1 -- 172.21.15.37:0/2463288269 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f3dd09940 con 0x555f3dcf2470 2024-06-10T01:03:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.913+0000 7f49710f5700 1 --2- 172.21.15.37:0/2463288269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f3dced030 0x555f3dd08e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:03:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.913+0000 7f49720f7700 1 --2- 172.21.15.37:0/2463288269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f3dcf2470 0x555f3dd093d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f496400f6a0 tx=0x7f4964012760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afed4ea7c9db7636 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.913+0000 7f49517fa700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:03:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.914+0000 7f49517fa700 1 -- 172.21.15.37:0/2463288269 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4964022070 con 0x555f3dcf2470 2024-06-10T01:03:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.914+0000 7f49517fa700 1 -- 172.21.15.37:0/2463288269 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f4964004250 con 0x555f3dcf2470 2024-06-10T01:03:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.914+0000 7f497e3ddb80 1 -- 172.21.15.37:0/2463288269 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4964012e50 con 0x555f3dcf2470 2024-06-10T01:03:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.914+0000 7f49517fa700 1 -- 172.21.15.37:0/2463288269 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4964035050 con 0x555f3dcf2470 2024-06-10T01:03:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.914+0000 7f497e3ddb80 10 client.26027 fetch_fsmap learned FSMap version 29 2024-06-10T01:03:32.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.914+0000 7f497e3ddb80 1 -- 172.21.15.37:0/2463288269 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555f3dd708f0 con 0x555f3dcf2470 2024-06-10T01:03:32.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.914+0000 7f49517fa700 1 -- 172.21.15.37:0/2463288269 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4964012e50 con 0x555f3dcf2470 2024-06-10T01:03:32.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.914+0000 7f49517fa700 1 -- 172.21.15.37:0/2463288269 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f4964030030 con 0x555f3dcf2470 2024-06-10T01:03:32.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.914+0000 7f497e3ddb80 10 client.26027 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:03:32.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.914+0000 7f497e3ddb80 10 client.26027 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:03:32.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.914+0000 7f497e3ddb80 20 client.26027 populate_metadata read hostname 'smithi037' 2024-06-10T01:03:32.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.914+0000 7f497e3ddb80 1 --2- 172.21.15.37:0/2463288269 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x555f3dd70f40 0x555f3dd73360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:32.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.914+0000 7f497e3ddb80 4 client.26027 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:03:32.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.914+0000 7f497e3ddb80 1 -- 172.21.15.37:0/2463288269 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555f3dc412d0 con 0x555f3dd70f40 2024-06-10T01:03:32.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.915+0000 7f49718f6700 1 --2- 172.21.15.37:0/2463288269 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x555f3dd70f40 0x555f3dd73360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:32.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.915+0000 7f49718f6700 1 --2- 172.21.15.37:0/2463288269 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x555f3dd70f40 0x555f3dd73360 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68ea961a5ea0a706 server_cookie=2f5ac363a2391922 in_seq=0 out_seq=0 2024-06-10T01:03:32.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.915+0000 7f49517fa700 10 client.26027 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:03:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.917+0000 7f49517fa700 1 -- 172.21.15.37:0/2463288269 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x555f3dc412d0 con 0x555f3dd70f40 2024-06-10T01:03:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.917+0000 7f49517fa700 10 client.26027 handle_command_reply: tid=0 2024-06-10T01:03:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.917+0000 7f497e3ddb80 1 client.26027 shutdown 2024-06-10T01:03:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.917+0000 7f497e3ddb80 1 -- 172.21.15.37:0/2463288269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f3dcf2470 msgr2=0x555f3dd093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.917+0000 7f497e3ddb80 1 --2- 172.21.15.37:0/2463288269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f3dcf2470 0x555f3dd093d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f496400f6a0 tx=0x7f4964012760 comp rx=0 tx=0).stop 2024-06-10T01:03:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.917+0000 7f497e3ddb80 1 -- 172.21.15.37:0/2463288269 shutdown_connections 2024-06-10T01:03:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.917+0000 7f497e3ddb80 1 --2- 172.21.15.37:0/2463288269 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x555f3dd70f40 0x555f3dd73360 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.917+0000 7f497e3ddb80 1 --2- 172.21.15.37:0/2463288269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f3dcf2470 0x555f3dd093d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.917+0000 7f497e3ddb80 1 --2- 172.21.15.37:0/2463288269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555f3dcec670 0x555f3dd0baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.917+0000 7f49517fa700 0 client.26027 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:03:32.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.917+0000 7f497e3ddb80 1 --2- 172.21.15.37:0/2463288269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f3dced030 0x555f3dd08e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.917+0000 7f497e3ddb80 1 -- 172.21.15.37:0/2463288269 >> 172.21.15.37:0/2463288269 conn(0x555f3dc15080 msgr2=0x555f3dcf5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:32.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.917+0000 7f497e3ddb80 1 -- 172.21.15.37:0/2463288269 shutdown_connections 2024-06-10T01:03:32.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.918+0000 7f497e3ddb80 1 -- 172.21.15.37:0/2463288269 wait complete. 2024-06-10T01:03:32.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.918+0000 7f497e3ddb80 20 client.26027 trim_cache size 0 max 16384 2024-06-10T01:03:32.920 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:03:32.920 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "scrub active (8826 inodes in the stack)", 2024-06-10T01:03:32.920 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-06-10T01:03:32.920 INFO:teuthology.orchestra.run.smithi037.stdout: "59187c88-c6e1-417e-9ae1-b55b232cd94e": { 2024-06-10T01:03:32.921 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-06-10T01:03:32.921 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "59187c88-c6e1-417e-9ae1-b55b232cd94e", 2024-06-10T01:03:32.921 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-06-10T01:03:32.921 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:03:32.921 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:03:32.921 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:03:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.918+0000 7f4973b5a700 1 -- 172.21.15.37:0/2776425267 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f49540a2c20 msgr2=0x7f49540a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.918+0000 7f4973b5a700 1 --2- 172.21.15.37:0/2776425267 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f49540a2c20 0x7f49540a50e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f496000b2e0 tx=0x7f4960005960 comp rx=0 tx=0).stop 2024-06-10T01:03:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.918+0000 7f4973b5a700 1 -- 172.21.15.37:0/2776425267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f496c101380 msgr2=0x7f496c19e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.918+0000 7f4973b5a700 1 --2- 172.21.15.37:0/2776425267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f496c101380 0x7f496c19e7f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f495c00ef30 tx=0x7f495c00c3a0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.918+0000 7f4973b5a700 1 -- 172.21.15.37:0/2776425267 shutdown_connections 2024-06-10T01:03:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.918+0000 7f4973b5a700 1 --2- 172.21.15.37:0/2776425267 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f49540a2c20 0x7f49540a50e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.918+0000 7f4973b5a700 1 --2- 172.21.15.37:0/2776425267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f496c1073f0 0x7f496c198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.918+0000 7f4973b5a700 1 --2- 172.21.15.37:0/2776425267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f496c101380 0x7f496c19e7f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.918+0000 7f4973b5a700 1 --2- 172.21.15.37:0/2776425267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f496c101d40 0x7f496c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.918+0000 7f4973b5a700 1 -- 172.21.15.37:0/2776425267 >> 172.21.15.37:0/2776425267 conn(0x7f496c078300 msgr2=0x7f496c0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:32.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.918+0000 7f4973b5a700 1 -- 172.21.15.37:0/2776425267 shutdown_connections 2024-06-10T01:03:32.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:32.918+0000 7f4973b5a700 1 -- 172.21.15.37:0/2776425267 wait complete. 2024-06-10T01:03:33.656 INFO:tasks.cephfs.filesystem:scrub status for tag:59187c88-c6e1-417e-9ae1-b55b232cd94e - {'path': '/', 'tag': '59187c88-c6e1-417e-9ae1-b55b232cd94e', 'options': 'recursive,force'} 2024-06-10T01:03:34.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:34 smithi167 ceph-mon[36917]: pgmap v637: 129 pgs: 129 active+clean; 11 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 1.10k op/s 2024-06-10T01:03:34.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:34 smithi094 ceph-mon[36708]: pgmap v637: 129 pgs: 129 active+clean; 11 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 1.10k op/s 2024-06-10T01:03:34.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:34 smithi037 ceph-mon[32192]: pgmap v637: 129 pgs: 129 active+clean; 11 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 1.10k op/s 2024-06-10T01:03:36.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:36 smithi167 ceph-mon[36917]: pgmap v638: 129 pgs: 129 active+clean; 17 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.8 MiB/s wr, 707 op/s 2024-06-10T01:03:36.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:36 smithi094 ceph-mon[36708]: pgmap v638: 129 pgs: 129 active+clean; 17 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.8 MiB/s wr, 707 op/s 2024-06-10T01:03:36.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:36 smithi037 ceph-mon[32192]: pgmap v638: 129 pgs: 129 active+clean; 17 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.8 MiB/s wr, 707 op/s 2024-06-10T01:03:38.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:38 smithi167 ceph-mon[36917]: pgmap v639: 129 pgs: 129 active+clean; 17 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.6 MiB/s wr, 624 op/s 2024-06-10T01:03:38.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:38 smithi094 ceph-mon[36708]: pgmap v639: 129 pgs: 129 active+clean; 17 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.6 MiB/s wr, 624 op/s 2024-06-10T01:03:38.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:38 smithi037 ceph-mon[32192]: pgmap v639: 129 pgs: 129 active+clean; 17 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.6 MiB/s wr, 624 op/s 2024-06-10T01:03:40.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:40 smithi167 ceph-mon[36917]: pgmap v640: 129 pgs: 129 active+clean; 21 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 746 op/s 2024-06-10T01:03:40.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:40 smithi094 ceph-mon[36708]: pgmap v640: 129 pgs: 129 active+clean; 21 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 746 op/s 2024-06-10T01:03:40.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:40 smithi037 ceph-mon[32192]: pgmap v640: 129 pgs: 129 active+clean; 21 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 746 op/s 2024-06-10T01:03:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.156+0000 7f40bba12640 1 -- 172.21.15.37:0/3185488348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40b41024b0 msgr2=0x7f40b4108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.156+0000 7f40bba12640 1 --2- 172.21.15.37:0/3185488348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40b41024b0 0x7f40b4108d40 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009ca0 tx=0x7f40ac02a8c0 comp rx=0 tx=0).stop 2024-06-10T01:03:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.156+0000 7f40bba12640 1 -- 172.21.15.37:0/3185488348 shutdown_connections 2024-06-10T01:03:42.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.156+0000 7f40bba12640 1 --2- 172.21.15.37:0/3185488348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40b41024b0 0x7f40b4108d40 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:42.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.156+0000 7f40bba12640 1 --2- 172.21.15.37:0/3185488348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40b4101af0 0x7f40b4101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:42.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.156+0000 7f40bba12640 1 --2- 172.21.15.37:0/3185488348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40b4100900 0x7f40b4100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:42.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.156+0000 7f40bba12640 1 -- 172.21.15.37:0/3185488348 >> 172.21.15.37:0/3185488348 conn(0x7f40b40fbe70 msgr2=0x7f40b40fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:42.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.156+0000 7f40bba12640 1 -- 172.21.15.37:0/3185488348 shutdown_connections 2024-06-10T01:03:42.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.156+0000 7f40bba12640 1 -- 172.21.15.37:0/3185488348 wait complete. 2024-06-10T01:03:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.157+0000 7f40bba12640 1 Processor -- start 2024-06-10T01:03:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.157+0000 7f40bba12640 1 -- start start 2024-06-10T01:03:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.157+0000 7f40bba12640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40b4100900 0x7f40b4197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.157+0000 7f40bba12640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40b4101af0 0x7f40b4198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.157+0000 7f40bba12640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40b41024b0 0x7f40b419f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.157+0000 7f40bba12640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f40b410c580 con 0x7f40b4101af0 2024-06-10T01:03:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.157+0000 7f40bba12640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f40b410c400 con 0x7f40b4100900 2024-06-10T01:03:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.157+0000 7f40bba12640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f40b410c700 con 0x7f40b41024b0 2024-06-10T01:03:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.157+0000 7f40b97ae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40b4100900 0x7f40b4197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.157+0000 7f40b8fad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40b4101af0 0x7f40b4198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:42.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.157+0000 7f40b97ae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40b4100900 0x7f40b4197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:56366/0 (socket says 172.21.15.37:56366) 2024-06-10T01:03:42.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.157+0000 7f40b97ae640 1 -- 172.21.15.37:0/2587684081 learned_addr learned my addr 172.21.15.37:0/2587684081 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:42.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.157+0000 7f40b9faf640 1 --2- 172.21.15.37:0/2587684081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40b41024b0 0x7f40b419f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:42.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.157+0000 7f40b97ae640 1 -- 172.21.15.37:0/2587684081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40b41024b0 msgr2=0x7f40b419f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:42.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.157+0000 7f40b97ae640 1 --2- 172.21.15.37:0/2587684081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40b41024b0 0x7f40b419f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:42.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.157+0000 7f40b97ae640 1 -- 172.21.15.37:0/2587684081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40b4101af0 msgr2=0x7f40b4198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:42.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.157+0000 7f40b97ae640 1 --2- 172.21.15.37:0/2587684081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40b4101af0 0x7f40b4198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:42.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.157+0000 7f40b97ae640 1 -- 172.21.15.37:0/2587684081 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b419f850 con 0x7f40b4100900 2024-06-10T01:03:42.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.157+0000 7f40b8fad640 1 --2- 172.21.15.37:0/2587684081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40b4101af0 0x7f40b4198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:03:42.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.158+0000 7f40b97ae640 1 --2- 172.21.15.37:0/2587684081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40b4100900 0x7f40b4197c40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f40b000e9c0 tx=0x7f40b000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:42.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.158+0000 7f40aa7fc640 1 -- 172.21.15.37:0/2587684081 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b000cd80 con 0x7f40b4100900 2024-06-10T01:03:42.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.158+0000 7f40aa7fc640 1 -- 172.21.15.37:0/2587684081 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40b00035f0 con 0x7f40b4100900 2024-06-10T01:03:42.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.158+0000 7f40bba12640 1 -- 172.21.15.37:0/2587684081 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b419fb40 con 0x7f40b4100900 2024-06-10T01:03:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.158+0000 7f40bba12640 1 -- 172.21.15.37:0/2587684081 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b41a0040 con 0x7f40b4100900 2024-06-10T01:03:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.158+0000 7f40aa7fc640 1 -- 172.21.15.37:0/2587684081 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0010690 con 0x7f40b4100900 2024-06-10T01:03:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.160+0000 7f40aa7fc640 1 -- 172.21.15.37:0/2587684081 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f40b00037a0 con 0x7f40b4100900 2024-06-10T01:03:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.161+0000 7f40aa7fc640 1 --2- 172.21.15.37:0/2587684081 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f40a00a2c30 0x7f40a00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.161+0000 7f40b8fad640 1 --2- 172.21.15.37:0/2587684081 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f40a00a2c30 0x7f40a00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.161+0000 7f40aa7fc640 1 -- 172.21.15.37:0/2587684081 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f40b00cf810 con 0x7f40b4100900 2024-06-10T01:03:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.162+0000 7f40be7fb740 1 Processor -- start 2024-06-10T01:03:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.162+0000 7f40be7fb740 1 -- start start 2024-06-10T01:03:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.162+0000 7f40b8fad640 1 --2- 172.21.15.37:0/2587684081 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f40a00a2c30 0x7f40a00a50f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f40a40095f0 tx=0x7f40a4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.162+0000 7f40be7fb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b0b868010 0x556b0b868430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.162+0000 7f40be7fb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b0b868a00 0x556b0b86d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.162+0000 7f40be7fb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b0b86db70 0x556b0b870000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.162+0000 7f40be7fb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556b0b7cb4f0 con 0x556b0b868a00 2024-06-10T01:03:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.162+0000 7f40be7fb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556b0b769870 con 0x556b0b86db70 2024-06-10T01:03:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.162+0000 7f40be7fb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556b0b7c74b0 con 0x556b0b868010 2024-06-10T01:03:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.162+0000 7f40b8fad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b0b868a00 0x556b0b86d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.162+0000 7f40b8fad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b0b868a00 0x556b0b86d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53996/0 (socket says 172.21.15.37:53996) 2024-06-10T01:03:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.162+0000 7f40b8fad640 1 -- 172.21.15.37:0/659919454 learned_addr learned my addr 172.21.15.37:0/659919454 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.162+0000 7f40b97ae640 1 --2- 172.21.15.37:0/659919454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b0b868010 0x556b0b868430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.162+0000 7f40b9faf640 1 --2- 172.21.15.37:0/659919454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b0b86db70 0x556b0b870000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.162+0000 7f40b97ae640 1 -- 172.21.15.37:0/659919454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b0b86db70 msgr2=0x556b0b870000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.162+0000 7f40b97ae640 1 --2- 172.21.15.37:0/659919454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b0b86db70 0x556b0b870000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.162+0000 7f40b97ae640 1 -- 172.21.15.37:0/659919454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b0b868a00 msgr2=0x556b0b86d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.162+0000 7f40b97ae640 1 --2- 172.21.15.37:0/659919454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b0b868a00 0x556b0b86d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.162+0000 7f40b97ae640 1 -- 172.21.15.37:0/659919454 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b0b7859b0 con 0x556b0b868010 2024-06-10T01:03:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.163+0000 7f40b97ae640 1 --2- 172.21.15.37:0/659919454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b0b868010 0x556b0b868430 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f40b00ccf70 tx=0x7f40b009b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b6e715691edbcf2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.163+0000 7f409f7fe640 1 -- 172.21.15.37:0/659919454 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b009bc80 con 0x556b0b868010 2024-06-10T01:03:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.163+0000 7f409f7fe640 1 -- 172.21.15.37:0/659919454 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40b0086040 con 0x556b0b868010 2024-06-10T01:03:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.163+0000 7f409f7fe640 1 -- 172.21.15.37:0/659919454 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0098400 con 0x556b0b868010 2024-06-10T01:03:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.164+0000 7f40be7fb740 1 -- 172.21.15.37:0/659919454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b0b868010 msgr2=0x556b0b868430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.164+0000 7f40be7fb740 1 --2- 172.21.15.37:0/659919454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b0b868010 0x556b0b868430 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f40b00ccf70 tx=0x7f40b009b880 comp rx=0 tx=0).stop 2024-06-10T01:03:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.164+0000 7f40be7fb740 1 -- 172.21.15.37:0/659919454 shutdown_connections 2024-06-10T01:03:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.164+0000 7f40be7fb740 1 --2- 172.21.15.37:0/659919454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b0b86db70 0x556b0b870000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.164+0000 7f40be7fb740 1 --2- 172.21.15.37:0/659919454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b0b868a00 0x556b0b86d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.164+0000 7f40be7fb740 1 --2- 172.21.15.37:0/659919454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b0b868010 0x556b0b868430 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.164+0000 7f40be7fb740 1 -- 172.21.15.37:0/659919454 >> 172.21.15.37:0/659919454 conn(0x556b0b863680 msgr2=0x556b0b863a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.164+0000 7f40be7fb740 1 -- 172.21.15.37:0/659919454 shutdown_connections 2024-06-10T01:03:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.164+0000 7f40be7fb740 1 -- 172.21.15.37:0/659919454 wait complete. 2024-06-10T01:03:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.164+0000 7f40be7fb740 1 -- start start 2024-06-10T01:03:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.164+0000 7f40be7fb740 1 Processor -- start 2024-06-10T01:03:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.165+0000 7f40be7fb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b0b868010 0x556b0b872220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.165+0000 7f40be7fb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b0b868a00 0x556b0b872760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.165+0000 7f40be7fb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b0b86db70 0x556b0b8797a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.165+0000 7f40be7fb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556b0b870ac0 con 0x556b0b86db70 2024-06-10T01:03:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.165+0000 7f40be7fb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556b0b754ae0 con 0x556b0b868a00 2024-06-10T01:03:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.165+0000 7f40be7fb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556b0b870c40 con 0x556b0b868010 2024-06-10T01:03:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.165+0000 7f40b9faf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b0b86db70 0x556b0b8797a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.165+0000 7f40be7fb740 10 client.0 _ll_register_callbacks cb 0x556b0b78d9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:03:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.165+0000 7f40b97ae640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b0b868010 0x556b0b872220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.165+0000 7f40b8fad640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b0b868a00 0x556b0b872760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.165+0000 7f40b9faf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b0b86db70 0x556b0b8797a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54012/0 (socket says 172.21.15.37:54012) 2024-06-10T01:03:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.165+0000 7f40b9faf640 1 -- 172.21.15.37:0/1899264178 learned_addr learned my addr 172.21.15.37:0/1899264178 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.165+0000 7f40b9faf640 1 -- 172.21.15.37:0/1899264178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b0b868010 msgr2=0x556b0b872220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.165+0000 7f40b9faf640 1 --2- 172.21.15.37:0/1899264178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b0b868010 0x556b0b872220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.165+0000 7f40b9faf640 1 -- 172.21.15.37:0/1899264178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b0b868a00 msgr2=0x556b0b872760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.165+0000 7f40b9faf640 1 --2- 172.21.15.37:0/1899264178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b0b868a00 0x556b0b872760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.165+0000 7f40b9faf640 1 -- 172.21.15.37:0/1899264178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b0b4df210 con 0x556b0b86db70 2024-06-10T01:03:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.165+0000 7f40b8fad640 1 --2- 172.21.15.37:0/1899264178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b0b868a00 0x556b0b872760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:03:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.165+0000 7f40b9faf640 1 --2- 172.21.15.37:0/1899264178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b0b86db70 0x556b0b8797a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009ca0 tx=0x7f40ac0093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d668da61a2195da server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.165+0000 7f409cff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:03:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.166+0000 7f409cff9640 1 -- 172.21.15.37:0/1899264178 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac04a060 con 0x556b0b86db70 2024-06-10T01:03:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.166+0000 7f409cff9640 1 -- 172.21.15.37:0/1899264178 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f40ac031cb0 con 0x556b0b86db70 2024-06-10T01:03:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.166+0000 7f40be7fb740 1 -- 172.21.15.37:0/1899264178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40b0086040 con 0x556b0b86db70 2024-06-10T01:03:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.166+0000 7f409cff9640 1 -- 172.21.15.37:0/1899264178 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f40ac003970 con 0x556b0b86db70 2024-06-10T01:03:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.166+0000 7f40be7fb740 10 client.16578 fetch_fsmap learned FSMap version 29 2024-06-10T01:03:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.166+0000 7f40be7fb740 1 -- 172.21.15.37:0/1899264178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556b0b5ab620 con 0x556b0b86db70 2024-06-10T01:03:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.166+0000 7f409cff9640 1 -- 172.21.15.37:0/1899264178 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40b0086040 con 0x556b0b86db70 2024-06-10T01:03:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.166+0000 7f409cff9640 1 -- 172.21.15.37:0/1899264178 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f40ac02f610 con 0x556b0b86db70 2024-06-10T01:03:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.167+0000 7f40be7fb740 10 client.16578 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:03:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.167+0000 7f40be7fb740 10 client.16578 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:03:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.167+0000 7f40be7fb740 20 client.16578 populate_metadata read hostname 'smithi037' 2024-06-10T01:03:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.167+0000 7f40be7fb740 1 --2- 172.21.15.37:0/1899264178 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x556b0b87e0a0 0x556b0b8804b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.167+0000 7f40be7fb740 4 client.16578 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:03:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.167+0000 7f40be7fb740 1 -- 172.21.15.37:0/1899264178 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b0b7def10 con 0x556b0b87e0a0 2024-06-10T01:03:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.167+0000 7f40b97ae640 1 --2- 172.21.15.37:0/1899264178 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x556b0b87e0a0 0x556b0b8804b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.167+0000 7f409cff9640 10 client.16578 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:03:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.167+0000 7f40b97ae640 1 --2- 172.21.15.37:0/1899264178 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x556b0b87e0a0 0x556b0b8804b0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=766a411d9a0aacbc server_cookie=1fd6ac708e4c0dfa in_seq=0 out_seq=0 2024-06-10T01:03:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.169+0000 7f409cff9640 1 -- 172.21.15.37:0/1899264178 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556b0b7def10 con 0x556b0b87e0a0 2024-06-10T01:03:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.169+0000 7f409cff9640 10 client.16578 handle_command_reply: tid=0 2024-06-10T01:03:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.169+0000 7f40be7fb740 1 client.16578 shutdown 2024-06-10T01:03:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.169+0000 7f40be7fb740 1 -- 172.21.15.37:0/1899264178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b0b86db70 msgr2=0x556b0b8797a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.169+0000 7f40be7fb740 1 --2- 172.21.15.37:0/1899264178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b0b86db70 0x556b0b8797a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009ca0 tx=0x7f40ac0093c0 comp rx=0 tx=0).stop 2024-06-10T01:03:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.169+0000 7f40be7fb740 1 -- 172.21.15.37:0/1899264178 shutdown_connections 2024-06-10T01:03:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.169+0000 7f40be7fb740 1 --2- 172.21.15.37:0/1899264178 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x556b0b87e0a0 0x556b0b8804b0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.169+0000 7f40be7fb740 1 --2- 172.21.15.37:0/1899264178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b0b86db70 0x556b0b8797a0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.169+0000 7f40be7fb740 1 --2- 172.21.15.37:0/1899264178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b0b868a00 0x556b0b872760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.169+0000 7f40be7fb740 1 --2- 172.21.15.37:0/1899264178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b0b868010 0x556b0b872220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.169+0000 7f409cff9640 0 client.16578 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:03:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.169+0000 7f40be7fb740 1 -- 172.21.15.37:0/1899264178 >> 172.21.15.37:0/1899264178 conn(0x556b0b863680 msgr2=0x556b0b8627e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.169+0000 7f40be7fb740 1 -- 172.21.15.37:0/1899264178 shutdown_connections 2024-06-10T01:03:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.169+0000 7f40be7fb740 1 -- 172.21.15.37:0/1899264178 wait complete. 2024-06-10T01:03:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.169+0000 7f40be7fb740 20 client.16578 trim_cache size 0 max 16384 2024-06-10T01:03:42.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.188+0000 7f40be7fb740 1 Processor -- start 2024-06-10T01:03:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.188+0000 7f40be7fb740 1 -- start start 2024-06-10T01:03:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.188+0000 7f40be7fb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b0b868010 0x556b0b872220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.188+0000 7f40be7fb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b0b868a00 0x556b0b872760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.188+0000 7f40be7fb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b0b86db70 0x556b0b866360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.188+0000 7f40be7fb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41ad0c0 con 0x556b0b868a00 2024-06-10T01:03:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.188+0000 7f40be7fb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41aa990 con 0x556b0b868010 2024-06-10T01:03:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.188+0000 7f40be7fb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41a87e0 con 0x556b0b86db70 2024-06-10T01:03:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.188+0000 7f40b8fad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b0b868a00 0x556b0b872760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.188+0000 7f40b97ae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b0b868010 0x556b0b872220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.188+0000 7f40b8fad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b0b868a00 0x556b0b872760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54022/0 (socket says 172.21.15.37:54022) 2024-06-10T01:03:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.188+0000 7f40b8fad640 1 -- 172.21.15.37:0/1278014156 learned_addr learned my addr 172.21.15.37:0/1278014156 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.188+0000 7f40b97ae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b0b868010 0x556b0b872220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:56394/0 (socket says 172.21.15.37:56394) 2024-06-10T01:03:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.188+0000 7f40b9faf640 1 --2- 172.21.15.37:0/1278014156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b0b86db70 0x556b0b866360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.189+0000 7f40b97ae640 1 -- 172.21.15.37:0/1278014156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b0b86db70 msgr2=0x556b0b866360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.189+0000 7f40b97ae640 1 --2- 172.21.15.37:0/1278014156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b0b86db70 0x556b0b866360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.189+0000 7f40b97ae640 1 -- 172.21.15.37:0/1278014156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b0b868a00 msgr2=0x556b0b872760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.189+0000 7f40b97ae640 1 --2- 172.21.15.37:0/1278014156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b0b868a00 0x556b0b872760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.189+0000 7f40b97ae640 1 -- 172.21.15.37:0/1278014156 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b0b8634b0 con 0x556b0b868010 2024-06-10T01:03:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.189+0000 7f40b8fad640 1 --2- 172.21.15.37:0/1278014156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b0b868a00 0x556b0b872760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:03:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.189+0000 7f40b9faf640 1 --2- 172.21.15.37:0/1278014156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b0b86db70 0x556b0b866360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:03:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.189+0000 7f40b97ae640 1 --2- 172.21.15.37:0/1278014156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b0b868010 0x556b0b872220 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f40b00ccf70 tx=0x7f40b009b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f25b69f4d5aca35 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.189+0000 7f409effd640 1 -- 172.21.15.37:0/1278014156 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b00ba040 con 0x556b0b868010 2024-06-10T01:03:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.189+0000 7f409effd640 1 -- 172.21.15.37:0/1278014156 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40b0086040 con 0x556b0b868010 2024-06-10T01:03:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.189+0000 7f409effd640 1 -- 172.21.15.37:0/1278014156 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b00a4460 con 0x556b0b868010 2024-06-10T01:03:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.190+0000 7f40be7fb740 1 -- 172.21.15.37:0/1278014156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b0b868010 msgr2=0x556b0b872220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.190+0000 7f40be7fb740 1 --2- 172.21.15.37:0/1278014156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b0b868010 0x556b0b872220 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f40b00ccf70 tx=0x7f40b009b8f0 comp rx=0 tx=0).stop 2024-06-10T01:03:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.190+0000 7f40be7fb740 1 -- 172.21.15.37:0/1278014156 shutdown_connections 2024-06-10T01:03:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.190+0000 7f40be7fb740 1 --2- 172.21.15.37:0/1278014156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b0b86db70 0x556b0b866360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.190+0000 7f40be7fb740 1 --2- 172.21.15.37:0/1278014156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b0b868a00 0x556b0b872760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.190+0000 7f40be7fb740 1 --2- 172.21.15.37:0/1278014156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b0b868010 0x556b0b872220 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.190+0000 7f40be7fb740 1 -- 172.21.15.37:0/1278014156 >> 172.21.15.37:0/1278014156 conn(0x556b0b863680 msgr2=0x556b0b869c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.190+0000 7f40be7fb740 1 -- 172.21.15.37:0/1278014156 shutdown_connections 2024-06-10T01:03:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.190+0000 7f40be7fb740 1 -- 172.21.15.37:0/1278014156 wait complete. 2024-06-10T01:03:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.190+0000 7f40be7fb740 1 -- start start 2024-06-10T01:03:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.190+0000 7f40be7fb740 1 Processor -- start 2024-06-10T01:03:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.191+0000 7f40be7fb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b0b868010 0x556b0b8e3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.191+0000 7f40be7fb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b0b868a00 0x556b0b8e3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.191+0000 7f40be7fb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b0b86db70 0x556b0b8eaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.191+0000 7f40be7fb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41a8e50 con 0x556b0b868a00 2024-06-10T01:03:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.191+0000 7f40be7fb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41aa2c0 con 0x556b0b868010 2024-06-10T01:03:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.191+0000 7f40be7fb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41a9520 con 0x556b0b86db70 2024-06-10T01:03:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.191+0000 7f40be7fb740 10 client.0 _ll_register_callbacks cb 0x7f40ac009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:03:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.191+0000 7f40b8fad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b0b868a00 0x556b0b8e3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.191+0000 7f40b97ae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b0b868010 0x556b0b8e3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.191+0000 7f40b8fad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b0b868a00 0x556b0b8e3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54028/0 (socket says 172.21.15.37:54028) 2024-06-10T01:03:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.191+0000 7f40b8fad640 1 -- 172.21.15.37:0/3967212966 learned_addr learned my addr 172.21.15.37:0/3967212966 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.191+0000 7f40b9faf640 1 --2- 172.21.15.37:0/3967212966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b0b86db70 0x556b0b8eaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.191+0000 7f40b97ae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b0b868010 0x556b0b8e3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:56396/0 (socket says 172.21.15.37:56396) 2024-06-10T01:03:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.191+0000 7f40b8fad640 1 -- 172.21.15.37:0/3967212966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b0b86db70 msgr2=0x556b0b8eaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.191+0000 7f40b8fad640 1 --2- 172.21.15.37:0/3967212966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b0b86db70 0x556b0b8eaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.191+0000 7f40b8fad640 1 -- 172.21.15.37:0/3967212966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b0b868010 msgr2=0x556b0b8e3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.191+0000 7f40b8fad640 1 --2- 172.21.15.37:0/3967212966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b0b868010 0x556b0b8e3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.191+0000 7f40b8fad640 1 -- 172.21.15.37:0/3967212966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b0b8eb0d0 con 0x556b0b868a00 2024-06-10T01:03:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.191+0000 7f40b97ae640 1 --2- 172.21.15.37:0/3967212966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b0b868010 0x556b0b8e3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:03:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.192+0000 7f40b8fad640 1 --2- 172.21.15.37:0/3967212966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b0b868a00 0x556b0b8e3ad0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f40a401e770 tx=0x7f40a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f47334d39d747b4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.192+0000 7f409cff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:03:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.192+0000 7f409cff9640 1 -- 172.21.15.37:0/3967212966 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a402d020 con 0x556b0b868a00 2024-06-10T01:03:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.192+0000 7f409cff9640 1 -- 172.21.15.37:0/3967212966 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f40a4029910 con 0x556b0b868a00 2024-06-10T01:03:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.192+0000 7f40be7fb740 1 -- 172.21.15.37:0/3967212966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40b0086040 con 0x556b0b868a00 2024-06-10T01:03:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.192+0000 7f409cff9640 1 -- 172.21.15.37:0/3967212966 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f40a4029b50 con 0x556b0b868a00 2024-06-10T01:03:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.192+0000 7f40be7fb740 10 client.16584 fetch_fsmap learned FSMap version 29 2024-06-10T01:03:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.192+0000 7f40be7fb740 1 -- 172.21.15.37:0/3967212966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556b0b8ec4b0 con 0x556b0b868a00 2024-06-10T01:03:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.192+0000 7f409cff9640 1 -- 172.21.15.37:0/3967212966 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40b0086040 con 0x556b0b868a00 2024-06-10T01:03:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.192+0000 7f409cff9640 1 -- 172.21.15.37:0/3967212966 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f40a400e3b0 con 0x556b0b868a00 2024-06-10T01:03:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.192+0000 7f40be7fb740 10 client.16584 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:03:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.192+0000 7f40be7fb740 10 client.16584 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:03:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.192+0000 7f40be7fb740 20 client.16584 populate_metadata read hostname 'smithi037' 2024-06-10T01:03:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.192+0000 7f40be7fb740 1 --2- 172.21.15.37:0/3967212966 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x556b0b8eced0 0x556b0b8ef2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.192+0000 7f40be7fb740 4 client.16584 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:03:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.192+0000 7f40be7fb740 1 -- 172.21.15.37:0/3967212966 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556b0b7d8c20 con 0x556b0b8eced0 2024-06-10T01:03:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.193+0000 7f40b97ae640 1 --2- 172.21.15.37:0/3967212966 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x556b0b8eced0 0x556b0b8ef2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.193+0000 7f40b97ae640 1 --2- 172.21.15.37:0/3967212966 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x556b0b8eced0 0x556b0b8ef2f0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18cd7f405388bc6 server_cookie=a6059f6ede3858ed in_seq=0 out_seq=0 2024-06-10T01:03:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:42.193+0000 7f409cff9640 10 client.16584 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:03:42.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:42 smithi167 ceph-mon[36917]: pgmap v641: 129 pgs: 129 active+clean; 21 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 971 KiB/s wr, 371 op/s 2024-06-10T01:03:42.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:42 smithi094 ceph-mon[36708]: pgmap v641: 129 pgs: 129 active+clean; 21 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 971 KiB/s wr, 371 op/s 2024-06-10T01:03:42.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:42 smithi037 ceph-mon[32192]: pgmap v641: 129 pgs: 129 active+clean; 21 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 971 KiB/s wr, 371 op/s 2024-06-10T01:03:44.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:44 smithi167 ceph-mon[36917]: pgmap v642: 129 pgs: 129 active+clean; 25 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 563 op/s 2024-06-10T01:03:44.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:44 smithi094 ceph-mon[36708]: pgmap v642: 129 pgs: 129 active+clean; 25 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 563 op/s 2024-06-10T01:03:44.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:44 smithi037 ceph-mon[32192]: pgmap v642: 129 pgs: 129 active+clean; 25 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 563 op/s 2024-06-10T01:03:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:45.409+0000 7f409cff9640 1 -- 172.21.15.37:0/3967212966 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556b0b7d8c20 con 0x556b0b8eced0 2024-06-10T01:03:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:45.409+0000 7f409cff9640 10 client.16584 handle_command_reply: tid=0 2024-06-10T01:03:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:45.409+0000 7f40be7fb740 1 client.16584 shutdown 2024-06-10T01:03:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:45.409+0000 7f40be7fb740 1 -- 172.21.15.37:0/3967212966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b0b868a00 msgr2=0x556b0b8e3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:45.409+0000 7f40be7fb740 1 --2- 172.21.15.37:0/3967212966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b0b868a00 0x556b0b8e3ad0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f40a401e770 tx=0x7f40a4004730 comp rx=0 tx=0).stop 2024-06-10T01:03:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:45.409+0000 7f40be7fb740 1 -- 172.21.15.37:0/3967212966 shutdown_connections 2024-06-10T01:03:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:45.409+0000 7f40be7fb740 1 --2- 172.21.15.37:0/3967212966 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x556b0b8eced0 0x556b0b8ef2f0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:45.409+0000 7f40be7fb740 1 --2- 172.21.15.37:0/3967212966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556b0b86db70 0x556b0b8eaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:45.409+0000 7f409cff9640 0 client.16584 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:03:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:45.409+0000 7f40be7fb740 1 --2- 172.21.15.37:0/3967212966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b0b868a00 0x556b0b8e3ad0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:45.409+0000 7f40be7fb740 1 --2- 172.21.15.37:0/3967212966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556b0b868010 0x556b0b8e3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:45.409+0000 7f40be7fb740 1 -- 172.21.15.37:0/3967212966 >> 172.21.15.37:0/3967212966 conn(0x556b0b863680 msgr2=0x556b0b869c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:45.409+0000 7f40be7fb740 1 -- 172.21.15.37:0/3967212966 shutdown_connections 2024-06-10T01:03:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:45.409+0000 7f40be7fb740 1 -- 172.21.15.37:0/3967212966 wait complete. 2024-06-10T01:03:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:45.409+0000 7f40be7fb740 20 client.16584 trim_cache size 0 max 16384 2024-06-10T01:03:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:03:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:03:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:03:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:45.410+0000 7f40bba12640 1 -- 172.21.15.37:0/2587684081 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f40a00a2c30 msgr2=0x7f40a00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:45.410+0000 7f40bba12640 1 --2- 172.21.15.37:0/2587684081 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f40a00a2c30 0x7f40a00a50f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f40a40095f0 tx=0x7f40a4005f90 comp rx=0 tx=0).stop 2024-06-10T01:03:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:45.410+0000 7f40bba12640 1 -- 172.21.15.37:0/2587684081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40b4100900 msgr2=0x7f40b4197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:45.410+0000 7f40bba12640 1 --2- 172.21.15.37:0/2587684081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40b4100900 0x7f40b4197c40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f40b000e9c0 tx=0x7f40b000ee80 comp rx=0 tx=0).stop 2024-06-10T01:03:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:45.410+0000 7f40bba12640 1 -- 172.21.15.37:0/2587684081 shutdown_connections 2024-06-10T01:03:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:45.410+0000 7f40bba12640 1 --2- 172.21.15.37:0/2587684081 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f40a00a2c30 0x7f40a00a50f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:45.410+0000 7f40bba12640 1 --2- 172.21.15.37:0/2587684081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40b41024b0 0x7f40b419f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:45.410+0000 7f40bba12640 1 --2- 172.21.15.37:0/2587684081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40b4101af0 0x7f40b4198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:45.410+0000 7f40bba12640 1 --2- 172.21.15.37:0/2587684081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40b4100900 0x7f40b4197c40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:45.410+0000 7f40bba12640 1 -- 172.21.15.37:0/2587684081 >> 172.21.15.37:0/2587684081 conn(0x7f40b40fbe70 msgr2=0x7f40b40fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:45.410+0000 7f40bba12640 1 -- 172.21.15.37:0/2587684081 shutdown_connections 2024-06-10T01:03:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:45.410+0000 7f40bba12640 1 -- 172.21.15.37:0/2587684081 wait complete. 2024-06-10T01:03:45.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:45.562+0000 7f960d7fa700 1 -- 172.21.15.37:0/1531404442 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558d72962470 con 0x558d72a8fc70 2024-06-10T01:03:45.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:45.562+0000 7f960d7fa700 10 client.26086 handle_command_reply: tid=0 2024-06-10T01:03:45.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:45.562+0000 7f963688ab80 1 client.26086 shutdown 2024-06-10T01:03:45.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:45.562+0000 7f963688ab80 1 -- 172.21.15.37:0/1531404442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d72a119b0 msgr2=0x558d72a194f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:45.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:45.562+0000 7f963688ab80 1 --2- 172.21.15.37:0/1531404442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d72a119b0 0x558d72a194f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9614098260 tx=0x7f961400cf20 comp rx=0 tx=0).stop 2024-06-10T01:03:45.561 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:45 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:03:45.561 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:45 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:03:45.561 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:45 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:03:45.561 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:45 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:03:45.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:45.562+0000 7f963688ab80 1 -- 172.21.15.37:0/1531404442 shutdown_connections 2024-06-10T01:03:45.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:45.562+0000 7f963688ab80 1 --2- 172.21.15.37:0/1531404442 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558d72a8fc70 0x558d72a92060 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:45.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:45.562+0000 7f963688ab80 1 --2- 172.21.15.37:0/1531404442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d72a8ba20 0x558d72a8de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:45.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:45.563+0000 7f963688ab80 1 --2- 172.21.15.37:0/1531404442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d72a119b0 0x558d72a194f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:45.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:45.563+0000 7f960d7fa700 0 client.26086 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:03:45.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:45.563+0000 7f963688ab80 1 --2- 172.21.15.37:0/1531404442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558d72a1c860 0x558d72a19a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:45.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:45.563+0000 7f963688ab80 1 -- 172.21.15.37:0/1531404442 >> 172.21.15.37:0/1531404442 conn(0x558d72936220 msgr2=0x558d72a13820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:45.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:45.563+0000 7f963688ab80 1 -- 172.21.15.37:0/1531404442 shutdown_connections 2024-06-10T01:03:45.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:45.563+0000 7f963688ab80 1 -- 172.21.15.37:0/1531404442 wait complete. 2024-06-10T01:03:45.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:45.563+0000 7f963688ab80 20 client.26086 trim_cache size 0 max 16384 2024-06-10T01:03:45.562 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:03:45.562 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 29, 2024-06-10T01:03:45.563 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24472, 2024-06-10T01:03:45.563 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 14, 2024-06-10T01:03:45.563 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-06-10T01:03:45.563 INFO:teuthology.orchestra.run.smithi037.stdout: "7788fb56": { 2024-06-10T01:03:45.563 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 14, 2024-06-10T01:03:45.563 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-06-10T01:03:45.563 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:03:45.563 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-06-10T01:03:45.563 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:03:45.563 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:03:45.563 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-06-10T01:03:45.563 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 149.4, 2024-06-10T01:03:45.563 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-06-10T01:03:45.564 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-06-10T01:03:45.564 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-06-10T01:03:45.564 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:03:45.564 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-06-10T01:03:45.564 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:03:45.564 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:03:45.564 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:03:45.564 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:03:45.564 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:03:45.564 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:03:45.564 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:03:45.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:45.564+0000 7f962ada5700 1 -- 172.21.15.37:0/1843600495 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f96100a2d00 msgr2=0x7f96100a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:45.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:45.564+0000 7f962ada5700 1 --2- 172.21.15.37:0/1843600495 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f96100a2d00 0x7f96100a51c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9620006fd0 tx=0x7f9620009060 comp rx=0 tx=0).stop 2024-06-10T01:03:45.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:45.564+0000 7f962ada5700 1 -- 172.21.15.37:0/1843600495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9624109f80 msgr2=0x7f96241a1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:45.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:45.564+0000 7f962ada5700 1 --2- 172.21.15.37:0/1843600495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9624109f80 0x7f96241a1360 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f961400e9a0 tx=0x7f961400ed60 comp rx=0 tx=0).stop 2024-06-10T01:03:45.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:45.564+0000 7f962ada5700 1 -- 172.21.15.37:0/1843600495 shutdown_connections 2024-06-10T01:03:45.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:45.564+0000 7f962ada5700 1 --2- 172.21.15.37:0/1843600495 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f96100a2d00 0x7f96100a51c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:45.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:45.564+0000 7f962ada5700 1 --2- 172.21.15.37:0/1843600495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f962410e4e0 0x7f96241a5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:45.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:45.564+0000 7f962ada5700 1 --2- 172.21.15.37:0/1843600495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9624109f80 0x7f96241a1360 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:45.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:45.564+0000 7f962ada5700 1 --2- 172.21.15.37:0/1843600495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f962410a930 0x7f96241a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:45.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:45.564+0000 7f962ada5700 1 -- 172.21.15.37:0/1843600495 >> 172.21.15.37:0/1843600495 conn(0x7f962406d3e0 msgr2=0x7f9624071dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:45.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:45.564+0000 7f962ada5700 1 -- 172.21.15.37:0/1843600495 shutdown_connections 2024-06-10T01:03:45.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:45.564+0000 7f962ada5700 1 -- 172.21.15.37:0/1843600495 wait complete. 2024-06-10T01:03:45.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:45 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:03:45.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:45 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:03:45.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:45 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:03:45.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:45 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:03:45.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:45 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:03:45.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:45 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:03:45.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:45 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:03:45.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:45 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:03:46.356 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7788fb56', quiesce took 51.5 seconds. Will release after: -22.05 2024-06-10T01:03:46.356 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:03:46.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:46 smithi167 ceph-mon[36917]: pgmap v643: 129 pgs: 129 active+clean; 25 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 817 op/s 2024-06-10T01:03:46.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:46 smithi094 ceph-mon[36708]: pgmap v643: 129 pgs: 129 active+clean; 25 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 817 op/s 2024-06-10T01:03:46.634 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:46.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:46 smithi037 ceph-mon[32192]: pgmap v643: 129 pgs: 129 active+clean; 25 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 817 op/s 2024-06-10T01:03:48.479 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:48 smithi037 ceph-mon[32192]: pgmap v644: 129 pgs: 129 active+clean; 29 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 912 op/s 2024-06-10T01:03:48.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:48 smithi167 ceph-mon[36917]: pgmap v644: 129 pgs: 129 active+clean; 29 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 912 op/s 2024-06-10T01:03:48.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:48 smithi094 ceph-mon[36708]: pgmap v644: 129 pgs: 129 active+clean; 29 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 912 op/s 2024-06-10T01:03:49.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.075+0000 7fd5d530f700 1 -- 172.21.15.37:0/1534547411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5d006ab30 msgr2=0x7fd5d010f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:49.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.075+0000 7fd5d530f700 1 --2- 172.21.15.37:0/1534547411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5d006ab30 0x7fd5d010f320 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800b0a0 tx=0x7fd5c800b3b0 comp rx=0 tx=0).stop 2024-06-10T01:03:49.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.075+0000 7fd5d530f700 1 -- 172.21.15.37:0/1534547411 shutdown_connections 2024-06-10T01:03:49.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.075+0000 7fd5d530f700 1 --2- 172.21.15.37:0/1534547411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5d006ab30 0x7fd5d010f320 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:49.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.075+0000 7fd5d530f700 1 --2- 172.21.15.37:0/1534547411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d0108500 0x7fd5d01088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:49.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.075+0000 7fd5d530f700 1 --2- 172.21.15.37:0/1534547411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5d006a170 0x7fd5d006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:49.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.075+0000 7fd5d530f700 1 -- 172.21.15.37:0/1534547411 >> 172.21.15.37:0/1534547411 conn(0x7fd5d0078340 msgr2=0x7fd5d0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:49.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.075+0000 7fd5d530f700 1 -- 172.21.15.37:0/1534547411 shutdown_connections 2024-06-10T01:03:49.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.075+0000 7fd5d530f700 1 -- 172.21.15.37:0/1534547411 wait complete. 2024-06-10T01:03:49.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.076+0000 7fd5d530f700 1 Processor -- start 2024-06-10T01:03:49.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.076+0000 7fd5d530f700 1 -- start start 2024-06-10T01:03:49.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.077+0000 7fd5d530f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d006a170 0x7fd5d019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:49.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.077+0000 7fd5d530f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5d006ab30 0x7fd5d019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:49.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.077+0000 7fd5d530f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5d0108500 0x7fd5d0198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:49.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.077+0000 7fd5d530f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d01127e0 con 0x7fd5d006ab30 2024-06-10T01:03:49.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.077+0000 7fd5d530f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d0112660 con 0x7fd5d006a170 2024-06-10T01:03:49.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.077+0000 7fd5d530f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d0112960 con 0x7fd5d0108500 2024-06-10T01:03:49.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.077+0000 7fd5ce7fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5d006ab30 0x7fd5d019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:49.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.077+0000 7fd5ce7fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5d006ab30 0x7fd5d019edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54042/0 (socket says 172.21.15.37:54042) 2024-06-10T01:03:49.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.077+0000 7fd5ce7fc700 1 -- 172.21.15.37:0/2912322521 learned_addr learned my addr 172.21.15.37:0/2912322521 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:49.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.077+0000 7fd5cf7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5d0108500 0x7fd5d0198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:49.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.077+0000 7fd5ceffd700 1 --2- 172.21.15.37:0/2912322521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d006a170 0x7fd5d019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:49.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.077+0000 7fd5ce7fc700 1 -- 172.21.15.37:0/2912322521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5d0108500 msgr2=0x7fd5d0198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:49.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.077+0000 7fd5ce7fc700 1 --2- 172.21.15.37:0/2912322521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5d0108500 0x7fd5d0198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:49.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.077+0000 7fd5ce7fc700 1 -- 172.21.15.37:0/2912322521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d006a170 msgr2=0x7fd5d019e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:49.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.077+0000 7fd5ce7fc700 1 --2- 172.21.15.37:0/2912322521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d006a170 0x7fd5d019e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:49.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.077+0000 7fd5ce7fc700 1 -- 172.21.15.37:0/2912322521 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d01991e0 con 0x7fd5d006ab30 2024-06-10T01:03:49.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.077+0000 7fd5ce7fc700 1 --2- 172.21.15.37:0/2912322521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5d006ab30 0x7fd5d019edb0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400d950 tx=0x7fd5c400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:49.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.077+0000 7fd5b7fff700 1 -- 172.21.15.37:0/2912322521 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4016070 con 0x7fd5d006ab30 2024-06-10T01:03:49.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.078+0000 7fd5d530f700 1 -- 172.21.15.37:0/2912322521 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d01994d0 con 0x7fd5d006ab30 2024-06-10T01:03:49.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.078+0000 7fd5b7fff700 1 -- 172.21.15.37:0/2912322521 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5c400fd70 con 0x7fd5d006ab30 2024-06-10T01:03:49.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.078+0000 7fd5b7fff700 1 -- 172.21.15.37:0/2912322521 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c400e610 con 0x7fd5d006ab30 2024-06-10T01:03:49.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.078+0000 7fd5d530f700 1 -- 172.21.15.37:0/2912322521 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d01a5740 con 0x7fd5d006ab30 2024-06-10T01:03:49.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.079+0000 7fd5d530f700 1 -- 172.21.15.37:0/2912322521 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5d010a9e0 con 0x7fd5d006ab30 2024-06-10T01:03:49.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.080+0000 7fd5b7fff700 1 -- 172.21.15.37:0/2912322521 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd5c4019070 con 0x7fd5d006ab30 2024-06-10T01:03:49.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.081+0000 7fd5b7fff700 1 --2- 172.21.15.37:0/2912322521 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd5b80a2c20 0x7fd5b80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:49.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.081+0000 7fd5b7fff700 1 -- 172.21.15.37:0/2912322521 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fd5c40ceec0 con 0x7fd5d006ab30 2024-06-10T01:03:49.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.081+0000 7fd5ceffd700 1 --2- 172.21.15.37:0/2912322521 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd5b80a2c20 0x7fd5b80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:49.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.081+0000 7fd5ceffd700 1 --2- 172.21.15.37:0/2912322521 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd5b80a2c20 0x7fd5b80a50e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0108000 tx=0x7fd5c0009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:49.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.084+0000 7fd5b7fff700 1 -- 172.21.15.37:0/2912322521 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd5c409ba30 con 0x7fd5d006ab30 2024-06-10T01:03:49.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.337+0000 7fd5d530f700 1 -- 172.21.15.37:0/2912322521 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5d0069180 con 0x7fd5d006ab30 2024-06-10T01:03:49.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.338+0000 7fd5b7fff700 1 -- 172.21.15.37:0/2912322521 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11864 (secure 0 0 0) 0x7fd5c409b330 con 0x7fd5d006ab30 2024-06-10T01:03:49.337 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:03:49.337 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T01:03:24.246183+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26110":{"gid":26110,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/317977487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":317977487},{"type":"v1","addr":"172.21.15.167:6835","nonce":317977487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24458,24472,24451]},"id":1}]} 2024-06-10T01:03:49.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.341+0000 7fd5d530f700 1 -- 172.21.15.37:0/2912322521 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd5b80a2c20 msgr2=0x7fd5b80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:49.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.341+0000 7fd5d530f700 1 --2- 172.21.15.37:0/2912322521 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd5b80a2c20 0x7fd5b80a50e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0108000 tx=0x7fd5c0009190 comp rx=0 tx=0).stop 2024-06-10T01:03:49.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.341+0000 7fd5d530f700 1 -- 172.21.15.37:0/2912322521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5d006ab30 msgr2=0x7fd5d019edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:49.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.341+0000 7fd5d530f700 1 --2- 172.21.15.37:0/2912322521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5d006ab30 0x7fd5d019edb0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400d950 tx=0x7fd5c400dc60 comp rx=0 tx=0).stop 2024-06-10T01:03:49.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.341+0000 7fd5d530f700 1 -- 172.21.15.37:0/2912322521 shutdown_connections 2024-06-10T01:03:49.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.341+0000 7fd5d530f700 1 --2- 172.21.15.37:0/2912322521 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd5b80a2c20 0x7fd5b80a50e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:49.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.341+0000 7fd5d530f700 1 --2- 172.21.15.37:0/2912322521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5d0108500 0x7fd5d0198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:49.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.341+0000 7fd5d530f700 1 --2- 172.21.15.37:0/2912322521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d006a170 0x7fd5d019e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:49.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.341+0000 7fd5d530f700 1 --2- 172.21.15.37:0/2912322521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5d006ab30 0x7fd5d019edb0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:49.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.341+0000 7fd5d530f700 1 -- 172.21.15.37:0/2912322521 >> 172.21.15.37:0/2912322521 conn(0x7fd5d0078340 msgr2=0x7fd5d00fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:49.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.341+0000 7fd5d530f700 1 -- 172.21.15.37:0/2912322521 shutdown_connections 2024-06-10T01:03:49.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:49.341+0000 7fd5d530f700 1 -- 172.21.15.37:0/2912322521 wait complete. 2024-06-10T01:03:49.339 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 29 2024-06-10T01:03:50.014 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24472 quiesce db --set-id 7788fb56 --release --await' 2024-06-10T01:03:50.014 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24472 quiesce db --set-id 7788fb56 --release --await 2024-06-10T01:03:50.271 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:50.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:50 smithi167 ceph-mon[36917]: scrub summary: idle+waiting paths [/] 2024-06-10T01:03:50.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:50 smithi167 ceph-mon[36917]: pgmap v645: 129 pgs: 129 active+clean; 6.5 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 2.53k op/s 2024-06-10T01:03:50.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:50 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2912322521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:03:50.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:50 smithi094 ceph-mon[36708]: scrub summary: idle+waiting paths [/] 2024-06-10T01:03:50.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:50 smithi094 ceph-mon[36708]: pgmap v645: 129 pgs: 129 active+clean; 6.5 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 2.53k op/s 2024-06-10T01:03:50.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:50 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2912322521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:03:50.635 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:50 smithi037 ceph-mon[32192]: scrub summary: idle+waiting paths [/] 2024-06-10T01:03:50.635 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:50 smithi037 ceph-mon[32192]: pgmap v645: 129 pgs: 129 active+clean; 6.5 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 2.53k op/s 2024-06-10T01:03:50.635 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:50 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2912322521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:03:52.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.029+0000 7f3a9e90a700 1 -- 172.21.15.37:0/1197854689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a9806b770 msgr2=0x7f3a98107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:52.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.029+0000 7f3a9e90a700 1 --2- 172.21.15.37:0/1197854689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a9806b770 0x7f3a98107130 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f3a800099c0 tx=0x7f3a80009cd0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.029+0000 7f3a9e90a700 1 -- 172.21.15.37:0/1197854689 shutdown_connections 2024-06-10T01:03:52.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.029+0000 7f3a9e90a700 1 --2- 172.21.15.37:0/1197854689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a98107860 0x7f3a9810f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.029+0000 7f3a9e90a700 1 --2- 172.21.15.37:0/1197854689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a9806adc0 0x7f3a9806b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.029+0000 7f3a9e90a700 1 --2- 172.21.15.37:0/1197854689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a9806b770 0x7f3a98107130 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.029+0000 7f3a9e90a700 1 -- 172.21.15.37:0/1197854689 >> 172.21.15.37:0/1197854689 conn(0x7f3a98078340 msgr2=0x7f3a98078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:52.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.029+0000 7f3a9e90a700 1 -- 172.21.15.37:0/1197854689 shutdown_connections 2024-06-10T01:03:52.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.029+0000 7f3a9e90a700 1 -- 172.21.15.37:0/1197854689 wait complete. 2024-06-10T01:03:52.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.030+0000 7f3a9e90a700 1 Processor -- start 2024-06-10T01:03:52.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.030+0000 7f3a9e90a700 1 -- start start 2024-06-10T01:03:52.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.030+0000 7f3a9e90a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a9806adc0 0x7f3a98101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:52.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.030+0000 7f3a9e90a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a9806b770 0x7f3a98101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:52.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.030+0000 7f3a9e90a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a98107860 0x7f3a981022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:52.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.030+0000 7f3a9e90a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3a98112770 con 0x7f3a9806b770 2024-06-10T01:03:52.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.030+0000 7f3a9e90a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3a981125f0 con 0x7f3a9806adc0 2024-06-10T01:03:52.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.030+0000 7f3a9e90a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3a981128f0 con 0x7f3a98107860 2024-06-10T01:03:52.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.030+0000 7f3a8ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a9806b770 0x7f3a98101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:52.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.030+0000 7f3a8ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a9806b770 0x7f3a98101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40988/0 (socket says 172.21.15.37:40988) 2024-06-10T01:03:52.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.030+0000 7f3a8ffff700 1 -- 172.21.15.37:0/3010357903 learned_addr learned my addr 172.21.15.37:0/3010357903 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:52.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.030+0000 7f3a9cea7700 1 --2- 172.21.15.37:0/3010357903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a98107860 0x7f3a981022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:52.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.030+0000 7f3a97fff700 1 --2- 172.21.15.37:0/3010357903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a9806adc0 0x7f3a98101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:52.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.030+0000 7f3a8ffff700 1 -- 172.21.15.37:0/3010357903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a98107860 msgr2=0x7f3a981022c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:52.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.030+0000 7f3a8ffff700 1 --2- 172.21.15.37:0/3010357903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a98107860 0x7f3a981022c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.030+0000 7f3a8ffff700 1 -- 172.21.15.37:0/3010357903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a9806adc0 msgr2=0x7f3a98101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:52.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.030+0000 7f3a8ffff700 1 --2- 172.21.15.37:0/3010357903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a9806adc0 0x7f3a98101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.030+0000 7f3a8ffff700 1 -- 172.21.15.37:0/3010357903 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a981a7070 con 0x7f3a9806b770 2024-06-10T01:03:52.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.031+0000 7f3a8ffff700 1 --2- 172.21.15.37:0/3010357903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a9806b770 0x7f3a98101d80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f3a800059d0 tx=0x7f3a800038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:52.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.031+0000 7f3a95ffb700 1 -- 172.21.15.37:0/3010357903 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a80027070 con 0x7f3a9806b770 2024-06-10T01:03:52.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.031+0000 7f3a95ffb700 1 -- 172.21.15.37:0/3010357903 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a80004050 con 0x7f3a9806b770 2024-06-10T01:03:52.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.031+0000 7f3a95ffb700 1 -- 172.21.15.37:0/3010357903 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a80022db0 con 0x7f3a9806b770 2024-06-10T01:03:52.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.031+0000 7f3a9e90a700 1 -- 172.21.15.37:0/3010357903 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a981a7300 con 0x7f3a9806b770 2024-06-10T01:03:52.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.031+0000 7f3a9e90a700 1 -- 172.21.15.37:0/3010357903 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a981a7750 con 0x7f3a9806b770 2024-06-10T01:03:52.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.033+0000 7f3a95ffb700 1 -- 172.21.15.37:0/3010357903 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3a8002a070 con 0x7f3a9806b770 2024-06-10T01:03:52.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.034+0000 7f3a95ffb700 1 --2- 172.21.15.37:0/3010357903 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f3a840a2ca0 0x7f3a840a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:52.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.034+0000 7f3a97fff700 1 --2- 172.21.15.37:0/3010357903 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f3a840a2ca0 0x7f3a840a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:52.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.034+0000 7f3a95ffb700 1 -- 172.21.15.37:0/3010357903 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f3a800d0120 con 0x7f3a9806b770 2024-06-10T01:03:52.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.034+0000 7f3a97fff700 1 --2- 172.21.15.37:0/3010357903 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f3a840a2ca0 0x7f3a840a5160 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3a88005950 tx=0x7f3a880058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:52.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.035+0000 7f3aa918db80 1 Processor -- start 2024-06-10T01:03:52.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.035+0000 7f3aa918db80 1 -- start start 2024-06-10T01:03:52.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.035+0000 7f3aa918db80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81d34d500 0x55d81d34d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:52.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.035+0000 7f3aa918db80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d81d34de20 0x55d81d347710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:52.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.035+0000 7f3aa918db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d81d347c50 0x55d81d348070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:52.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.035+0000 7f3aa918db80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d81d23a060 con 0x55d81d34de20 2024-06-10T01:03:52.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.035+0000 7f3aa918db80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d81d298950 con 0x55d81d34d500 2024-06-10T01:03:52.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.035+0000 7f3aa918db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d81d200560 con 0x55d81d347c50 2024-06-10T01:03:52.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.035+0000 7f3a8ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d81d34de20 0x55d81d347710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:52.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.035+0000 7f3a8ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d81d34de20 0x55d81d347710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41004/0 (socket says 172.21.15.37:41004) 2024-06-10T01:03:52.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.035+0000 7f3a8ffff700 1 -- 172.21.15.37:0/2516256805 learned_addr learned my addr 172.21.15.37:0/2516256805 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:52.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.035+0000 7f3a97fff700 1 --2- 172.21.15.37:0/2516256805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81d34d500 0x55d81d34d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:52.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.035+0000 7f3a9cea7700 1 --2- 172.21.15.37:0/2516256805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d81d347c50 0x55d81d348070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:52.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.035+0000 7f3a8ffff700 1 -- 172.21.15.37:0/2516256805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d81d347c50 msgr2=0x55d81d348070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:52.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.035+0000 7f3a8ffff700 1 --2- 172.21.15.37:0/2516256805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d81d347c50 0x55d81d348070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.035+0000 7f3a8ffff700 1 -- 172.21.15.37:0/2516256805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81d34d500 msgr2=0x55d81d34d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:52.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.035+0000 7f3a8ffff700 1 --2- 172.21.15.37:0/2516256805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81d34d500 0x55d81d34d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.035+0000 7f3a8ffff700 1 -- 172.21.15.37:0/2516256805 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d81d1e79d0 con 0x55d81d34de20 2024-06-10T01:03:52.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.035+0000 7f3a9cea7700 1 --2- 172.21.15.37:0/2516256805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d81d347c50 0x55d81d348070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:03:52.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.035+0000 7f3a97fff700 1 --2- 172.21.15.37:0/2516256805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81d34d500 0x55d81d34d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:03:52.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.035+0000 7f3a8ffff700 1 --2- 172.21.15.37:0/2516256805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d81d34de20 0x55d81d347710 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f3a8009c320 tx=0x7f3a80022810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c4436e713f1872 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:52.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.036+0000 7f3a8e7fc700 1 -- 172.21.15.37:0/2516256805 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8009c620 con 0x55d81d34de20 2024-06-10T01:03:52.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.036+0000 7f3a8e7fc700 1 -- 172.21.15.37:0/2516256805 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a800a09d0 con 0x55d81d34de20 2024-06-10T01:03:52.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.036+0000 7f3a8e7fc700 1 -- 172.21.15.37:0/2516256805 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8009ca80 con 0x55d81d34de20 2024-06-10T01:03:52.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.036+0000 7f3aa918db80 1 -- 172.21.15.37:0/2516256805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d81d34de20 msgr2=0x55d81d347710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:52.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.036+0000 7f3aa918db80 1 --2- 172.21.15.37:0/2516256805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d81d34de20 0x55d81d347710 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f3a8009c320 tx=0x7f3a80022810 comp rx=0 tx=0).stop 2024-06-10T01:03:52.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.036+0000 7f3aa918db80 1 -- 172.21.15.37:0/2516256805 shutdown_connections 2024-06-10T01:03:52.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.036+0000 7f3aa918db80 1 --2- 172.21.15.37:0/2516256805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d81d347c50 0x55d81d348070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.036+0000 7f3aa918db80 1 --2- 172.21.15.37:0/2516256805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81d34d500 0x55d81d34d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.036+0000 7f3aa918db80 1 --2- 172.21.15.37:0/2516256805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d81d34de20 0x55d81d347710 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.036+0000 7f3aa918db80 1 -- 172.21.15.37:0/2516256805 >> 172.21.15.37:0/2516256805 conn(0x55d81d274180 msgr2=0x55d81d342a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:52.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.036+0000 7f3aa918db80 1 -- 172.21.15.37:0/2516256805 shutdown_connections 2024-06-10T01:03:52.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.036+0000 7f3aa918db80 1 -- 172.21.15.37:0/2516256805 wait complete. 2024-06-10T01:03:52.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.037+0000 7f3aa918db80 1 -- start start 2024-06-10T01:03:52.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.037+0000 7f3aa918db80 1 Processor -- start 2024-06-10T01:03:52.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.037+0000 7f3aa918db80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d81d347c50 0x55d81d353bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:52.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.037+0000 7f3aa918db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d81d34d500 0x55d81d350eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:52.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.037+0000 7f3aa918db80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81d34de20 0x55d81d3513f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:52.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.037+0000 7f3aa918db80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d81d2390c0 con 0x55d81d347c50 2024-06-10T01:03:52.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.037+0000 7f3aa918db80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d81d294910 con 0x55d81d34de20 2024-06-10T01:03:52.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.037+0000 7f3aa918db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d81d3545b0 con 0x55d81d34d500 2024-06-10T01:03:52.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.037+0000 7f3aa918db80 10 client.0 _ll_register_callbacks cb 0x55d81d22e6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:03:52.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.037+0000 7f3a97fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d81d347c50 0x55d81d353bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:52.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.038+0000 7f3a97fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d81d347c50 0x55d81d353bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41020/0 (socket says 172.21.15.37:41020) 2024-06-10T01:03:52.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.038+0000 7f3a97fff700 1 -- 172.21.15.37:0/3389228791 learned_addr learned my addr 172.21.15.37:0/3389228791 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:52.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.038+0000 7f3a9cea7700 1 --2- 172.21.15.37:0/3389228791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81d34de20 0x55d81d3513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:52.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.038+0000 7f3a8ffff700 1 --2- 172.21.15.37:0/3389228791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d81d34d500 0x55d81d350eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:52.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.038+0000 7f3a97fff700 1 -- 172.21.15.37:0/3389228791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d81d34d500 msgr2=0x55d81d350eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:52.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.038+0000 7f3a97fff700 1 --2- 172.21.15.37:0/3389228791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d81d34d500 0x55d81d350eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.038+0000 7f3a97fff700 1 -- 172.21.15.37:0/3389228791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81d34de20 msgr2=0x55d81d3513f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:52.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.038+0000 7f3a97fff700 1 --2- 172.21.15.37:0/3389228791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81d34de20 0x55d81d3513f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.038+0000 7f3a97fff700 1 -- 172.21.15.37:0/3389228791 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d81d25e850 con 0x55d81d347c50 2024-06-10T01:03:52.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.038+0000 7f3a8ffff700 1 --2- 172.21.15.37:0/3389228791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d81d34d500 0x55d81d350eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:03:52.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.038+0000 7f3a97fff700 1 --2- 172.21.15.37:0/3389228791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d81d347c50 0x55d81d353bb0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3a8800ff70 tx=0x7f3a88004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572dd42426cf6013 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:52.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.038+0000 7f3a777fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:03:52.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.038+0000 7f3a777fe700 1 -- 172.21.15.37:0/3389228791 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8801f630 con 0x55d81d347c50 2024-06-10T01:03:52.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.038+0000 7f3a777fe700 1 -- 172.21.15.37:0/3389228791 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f3a8801f7d0 con 0x55d81d347c50 2024-06-10T01:03:52.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.038+0000 7f3aa918db80 1 -- 172.21.15.37:0/3389228791 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a800a09d0 con 0x55d81d347c50 2024-06-10T01:03:52.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.038+0000 7f3a777fe700 1 -- 172.21.15.37:0/3389228791 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3a88030050 con 0x55d81d347c50 2024-06-10T01:03:52.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.038+0000 7f3aa918db80 10 client.16602 fetch_fsmap learned FSMap version 29 2024-06-10T01:03:52.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.038+0000 7f3aa918db80 1 -- 172.21.15.37:0/3389228791 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d81d35ecb0 con 0x55d81d347c50 2024-06-10T01:03:52.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.039+0000 7f3a777fe700 1 -- 172.21.15.37:0/3389228791 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a800a09d0 con 0x55d81d347c50 2024-06-10T01:03:52.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.039+0000 7f3a777fe700 1 -- 172.21.15.37:0/3389228791 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f3a880279e0 con 0x55d81d347c50 2024-06-10T01:03:52.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.039+0000 7f3aa918db80 10 client.16602 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:03:52.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.039+0000 7f3aa918db80 10 client.16602 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:03:52.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.039+0000 7f3aa918db80 20 client.16602 populate_metadata read hostname 'smithi037' 2024-06-10T01:03:52.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.039+0000 7f3aa918db80 1 --2- 172.21.15.37:0/3389228791 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55d81d3612f0 0x55d81d363700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:52.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.039+0000 7f3aa918db80 4 client.16602 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:03:52.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.039+0000 7f3aa918db80 1 -- 172.21.15.37:0/3389228791 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d81d2a5e50 con 0x55d81d3612f0 2024-06-10T01:03:52.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.039+0000 7f3a8ffff700 1 --2- 172.21.15.37:0/3389228791 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55d81d3612f0 0x55d81d363700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:52.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.040+0000 7f3a8ffff700 1 --2- 172.21.15.37:0/3389228791 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55d81d3612f0 0x55d81d363700 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4dd55992acb63a6 server_cookie=6a3109dd4a3aa60d in_seq=0 out_seq=0 2024-06-10T01:03:52.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.040+0000 7f3a777fe700 10 client.16602 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:03:52.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.040+0000 7f3a777fe700 1 -- 172.21.15.37:0/3389228791 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d81d2a5e50 con 0x55d81d3612f0 2024-06-10T01:03:52.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.040+0000 7f3a777fe700 10 client.16602 handle_command_reply: tid=0 2024-06-10T01:03:52.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.041+0000 7f3aa918db80 1 client.16602 shutdown 2024-06-10T01:03:52.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.041+0000 7f3aa918db80 1 -- 172.21.15.37:0/3389228791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d81d347c50 msgr2=0x55d81d353bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:52.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.041+0000 7f3aa918db80 1 --2- 172.21.15.37:0/3389228791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d81d347c50 0x55d81d353bb0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3a8800ff70 tx=0x7f3a88004750 comp rx=0 tx=0).stop 2024-06-10T01:03:52.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.041+0000 7f3aa918db80 1 -- 172.21.15.37:0/3389228791 shutdown_connections 2024-06-10T01:03:52.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.041+0000 7f3aa918db80 1 --2- 172.21.15.37:0/3389228791 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55d81d3612f0 0x55d81d363700 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.041+0000 7f3aa918db80 1 --2- 172.21.15.37:0/3389228791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81d34de20 0x55d81d3513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.041+0000 7f3aa918db80 1 --2- 172.21.15.37:0/3389228791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d81d347c50 0x55d81d353bb0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.041+0000 7f3a777fe700 0 client.16602 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:03:52.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.041+0000 7f3aa918db80 1 --2- 172.21.15.37:0/3389228791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d81d34d500 0x55d81d350eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.041+0000 7f3aa918db80 1 -- 172.21.15.37:0/3389228791 >> 172.21.15.37:0/3389228791 conn(0x55d81d274180 msgr2=0x55d81d341bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:52.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.041+0000 7f3aa918db80 1 -- 172.21.15.37:0/3389228791 shutdown_connections 2024-06-10T01:03:52.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.041+0000 7f3aa918db80 1 -- 172.21.15.37:0/3389228791 wait complete. 2024-06-10T01:03:52.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.041+0000 7f3aa918db80 20 client.16602 trim_cache size 0 max 16384 2024-06-10T01:03:52.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.061+0000 7f3aa918db80 1 Processor -- start 2024-06-10T01:03:52.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.061+0000 7f3aa918db80 1 -- start start 2024-06-10T01:03:52.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.062+0000 7f3aa918db80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81d347c50 0x55d81d3536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:52.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.062+0000 7f3aa918db80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d81d34d500 0x55d81d346f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:52.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.062+0000 7f3aa918db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d81d34de20 0x55d81d342330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:52.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.062+0000 7f3aa918db80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3a981b2e20 con 0x55d81d34d500 2024-06-10T01:03:52.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.062+0000 7f3aa918db80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3a981b06b0 con 0x55d81d347c50 2024-06-10T01:03:52.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.062+0000 7f3aa918db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3a981ae500 con 0x55d81d34de20 2024-06-10T01:03:52.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.062+0000 7f3a8ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d81d34d500 0x55d81d346f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:52.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.062+0000 7f3a8ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d81d34d500 0x55d81d346f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41028/0 (socket says 172.21.15.37:41028) 2024-06-10T01:03:52.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.062+0000 7f3a8ffff700 1 -- 172.21.15.37:0/3741459481 learned_addr learned my addr 172.21.15.37:0/3741459481 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:52.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.062+0000 7f3a9cea7700 1 --2- 172.21.15.37:0/3741459481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d81d34de20 0x55d81d342330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:52.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.062+0000 7f3a97fff700 1 --2- 172.21.15.37:0/3741459481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81d347c50 0x55d81d3536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:52.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.062+0000 7f3a8ffff700 1 -- 172.21.15.37:0/3741459481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d81d34de20 msgr2=0x55d81d342330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:52.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.062+0000 7f3a8ffff700 1 --2- 172.21.15.37:0/3741459481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d81d34de20 0x55d81d342330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.062+0000 7f3a8ffff700 1 -- 172.21.15.37:0/3741459481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81d347c50 msgr2=0x55d81d3536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:52.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.062+0000 7f3a8ffff700 1 --2- 172.21.15.37:0/3741459481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81d347c50 0x55d81d3536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.062+0000 7f3a8ffff700 1 -- 172.21.15.37:0/3741459481 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d81d34f420 con 0x55d81d34d500 2024-06-10T01:03:52.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.062+0000 7f3a8ffff700 1 --2- 172.21.15.37:0/3741459481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d81d34d500 0x55d81d346f90 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f3a800cdba0 tx=0x7f3a800a3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9879a3e05262f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:52.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.062+0000 7f3a8dffb700 1 -- 172.21.15.37:0/3741459481 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8009c620 con 0x55d81d34d500 2024-06-10T01:03:52.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.063+0000 7f3a8dffb700 1 -- 172.21.15.37:0/3741459481 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a8009a6c0 con 0x55d81d34d500 2024-06-10T01:03:52.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.063+0000 7f3a8dffb700 1 -- 172.21.15.37:0/3741459481 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8009cae0 con 0x55d81d34d500 2024-06-10T01:03:52.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.063+0000 7f3aa918db80 1 -- 172.21.15.37:0/3741459481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d81d34d500 msgr2=0x55d81d346f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:52.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.063+0000 7f3aa918db80 1 --2- 172.21.15.37:0/3741459481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d81d34d500 0x55d81d346f90 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f3a800cdba0 tx=0x7f3a800a3040 comp rx=0 tx=0).stop 2024-06-10T01:03:52.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.063+0000 7f3aa918db80 1 -- 172.21.15.37:0/3741459481 shutdown_connections 2024-06-10T01:03:52.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.063+0000 7f3aa918db80 1 --2- 172.21.15.37:0/3741459481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d81d34de20 0x55d81d342330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.063+0000 7f3aa918db80 1 --2- 172.21.15.37:0/3741459481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81d347c50 0x55d81d3536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.063+0000 7f3aa918db80 1 --2- 172.21.15.37:0/3741459481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d81d34d500 0x55d81d346f90 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.063+0000 7f3aa918db80 1 -- 172.21.15.37:0/3741459481 >> 172.21.15.37:0/3741459481 conn(0x55d81d274180 msgr2=0x55d81d34c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:52.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.063+0000 7f3aa918db80 1 -- 172.21.15.37:0/3741459481 shutdown_connections 2024-06-10T01:03:52.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.063+0000 7f3aa918db80 1 -- 172.21.15.37:0/3741459481 wait complete. 2024-06-10T01:03:52.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.064+0000 7f3aa918db80 1 -- start start 2024-06-10T01:03:52.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.064+0000 7f3aa918db80 1 Processor -- start 2024-06-10T01:03:52.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.064+0000 7f3aa918db80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81d347c50 0x55d81d359f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:52.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.064+0000 7f3aa918db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d81d34d500 0x55d81d3567d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:52.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.064+0000 7f3aa918db80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d81d34de20 0x55d81d356d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:52.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.064+0000 7f3aa918db80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3a981aeb70 con 0x55d81d34de20 2024-06-10T01:03:52.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.064+0000 7f3aa918db80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3a981affe0 con 0x55d81d347c50 2024-06-10T01:03:52.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.064+0000 7f3aa918db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3a981af240 con 0x55d81d34d500 2024-06-10T01:03:52.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.064+0000 7f3aa918db80 10 client.0 _ll_register_callbacks cb 0x55d81d22e6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:03:52.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.064+0000 7f3a9cea7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d81d34de20 0x55d81d356d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:52.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.064+0000 7f3a9cea7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d81d34de20 0x55d81d356d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41030/0 (socket says 172.21.15.37:41030) 2024-06-10T01:03:52.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.064+0000 7f3a9cea7700 1 -- 172.21.15.37:0/3287758047 learned_addr learned my addr 172.21.15.37:0/3287758047 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:52.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.064+0000 7f3a97fff700 1 --2- 172.21.15.37:0/3287758047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81d347c50 0x55d81d359f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:52.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.064+0000 7f3a8ffff700 1 --2- 172.21.15.37:0/3287758047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d81d34d500 0x55d81d3567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:52.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.065+0000 7f3a9cea7700 1 -- 172.21.15.37:0/3287758047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d81d34d500 msgr2=0x55d81d3567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:52.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.065+0000 7f3a9cea7700 1 --2- 172.21.15.37:0/3287758047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d81d34d500 0x55d81d3567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.065+0000 7f3a9cea7700 1 -- 172.21.15.37:0/3287758047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81d347c50 msgr2=0x55d81d359f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:52.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.065+0000 7f3a9cea7700 1 --2- 172.21.15.37:0/3287758047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81d347c50 0x55d81d359f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:52.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.065+0000 7f3a9cea7700 1 -- 172.21.15.37:0/3287758047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d81d357500 con 0x55d81d34de20 2024-06-10T01:03:52.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.065+0000 7f3a9cea7700 1 --2- 172.21.15.37:0/3287758047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d81d34de20 0x55d81d356d10 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3a90014cf0 tx=0x7f3a900163c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8a71d244445db2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:52.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.065+0000 7f3a777fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:03:52.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.065+0000 7f3a777fe700 1 -- 172.21.15.37:0/3287758047 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a90003cd0 con 0x55d81d34de20 2024-06-10T01:03:52.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.065+0000 7f3a777fe700 1 -- 172.21.15.37:0/3287758047 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f3a900169c0 con 0x55d81d34de20 2024-06-10T01:03:52.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.065+0000 7f3aa918db80 1 -- 172.21.15.37:0/3287758047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a8009a6c0 con 0x55d81d34de20 2024-06-10T01:03:52.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.065+0000 7f3a777fe700 1 -- 172.21.15.37:0/3287758047 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3a90029050 con 0x55d81d34de20 2024-06-10T01:03:52.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.065+0000 7f3aa918db80 10 client.16608 fetch_fsmap learned FSMap version 29 2024-06-10T01:03:52.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.065+0000 7f3aa918db80 1 -- 172.21.15.37:0/3287758047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d81d3cd0c0 con 0x55d81d34de20 2024-06-10T01:03:52.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.065+0000 7f3a777fe700 1 -- 172.21.15.37:0/3287758047 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a8009a6c0 con 0x55d81d34de20 2024-06-10T01:03:52.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.066+0000 7f3a777fe700 1 -- 172.21.15.37:0/3287758047 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f3a90020470 con 0x55d81d34de20 2024-06-10T01:03:52.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.066+0000 7f3aa918db80 10 client.16608 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:03:52.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.066+0000 7f3aa918db80 10 client.16608 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:03:52.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.066+0000 7f3aa918db80 20 client.16608 populate_metadata read hostname 'smithi037' 2024-06-10T01:03:52.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.066+0000 7f3aa918db80 1 --2- 172.21.15.37:0/3287758047 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55d81d3cd800 0x55d81d3cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:52.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.066+0000 7f3aa918db80 4 client.16608 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7788fb56", "release": true, "await": true}] 2024-06-10T01:03:52.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.066+0000 7f3aa918db80 1 -- 172.21.15.37:0/3287758047 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7788fb56", "release": true, "await": true}) -- 0x55d81d2a03d0 con 0x55d81d3cd800 2024-06-10T01:03:52.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.066+0000 7f3a97fff700 1 --2- 172.21.15.37:0/3287758047 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55d81d3cd800 0x55d81d3cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:52.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.066+0000 7f3a97fff700 1 --2- 172.21.15.37:0/3287758047 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55d81d3cd800 0x55d81d3cfc20 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=746ac1ddc81263d5 server_cookie=a535315879a343bc in_seq=0 out_seq=0 2024-06-10T01:03:52.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:52.066+0000 7f3a777fe700 10 client.16608 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:03:52.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:52 smithi167 ceph-mon[36917]: scrub summary: idle 2024-06-10T01:03:52.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:52 smithi167 ceph-mon[36917]: pgmap v646: 129 pgs: 129 active+clean; 6.5 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.4 MiB/s wr, 2.64k op/s 2024-06-10T01:03:52.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:52 smithi094 ceph-mon[36708]: scrub summary: idle 2024-06-10T01:03:52.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:52 smithi094 ceph-mon[36708]: pgmap v646: 129 pgs: 129 active+clean; 6.5 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.4 MiB/s wr, 2.64k op/s 2024-06-10T01:03:52.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:52 smithi037 ceph-mon[32192]: scrub summary: idle 2024-06-10T01:03:52.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:52 smithi037 ceph-mon[32192]: pgmap v646: 129 pgs: 129 active+clean; 6.5 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.4 MiB/s wr, 2.64k op/s 2024-06-10T01:03:53.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:53.607+0000 7f3a777fe700 1 -- 172.21.15.37:0/3287758047 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d81d2a03d0 con 0x55d81d3cd800 2024-06-10T01:03:53.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:53.607+0000 7f3a777fe700 10 client.16608 handle_command_reply: tid=0 2024-06-10T01:03:53.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:53.607+0000 7f3aa918db80 1 client.16608 shutdown 2024-06-10T01:03:53.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:53.607+0000 7f3aa918db80 1 -- 172.21.15.37:0/3287758047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d81d34de20 msgr2=0x55d81d356d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:53.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:53.607+0000 7f3aa918db80 1 --2- 172.21.15.37:0/3287758047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d81d34de20 0x55d81d356d10 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3a90014cf0 tx=0x7f3a900163c0 comp rx=0 tx=0).stop 2024-06-10T01:03:53.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:53.607+0000 7f3aa918db80 1 -- 172.21.15.37:0/3287758047 shutdown_connections 2024-06-10T01:03:53.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:53.607+0000 7f3aa918db80 1 --2- 172.21.15.37:0/3287758047 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55d81d3cd800 0x55d81d3cfc20 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:53.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:53.607+0000 7f3aa918db80 1 --2- 172.21.15.37:0/3287758047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d81d34de20 0x55d81d356d10 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:53.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:53.607+0000 7f3aa918db80 1 --2- 172.21.15.37:0/3287758047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81d347c50 0x55d81d359f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:53.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:53.607+0000 7f3aa918db80 1 --2- 172.21.15.37:0/3287758047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d81d34d500 0x55d81d3567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:53.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:53.607+0000 7f3a777fe700 0 client.16608 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:03:53.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:53.607+0000 7f3aa918db80 1 -- 172.21.15.37:0/3287758047 >> 172.21.15.37:0/3287758047 conn(0x55d81d274180 msgr2=0x55d81d346f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:53.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:53.607+0000 7f3aa918db80 1 -- 172.21.15.37:0/3287758047 shutdown_connections 2024-06-10T01:03:53.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:53.607+0000 7f3aa918db80 1 -- 172.21.15.37:0/3287758047 wait complete. 2024-06-10T01:03:53.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:53.608+0000 7f3aa918db80 20 client.16608 trim_cache size 0 max 16384 2024-06-10T01:03:53.606 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:03:53.606 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 29, 2024-06-10T01:03:53.606 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24472, 2024-06-10T01:03:53.607 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 16, 2024-06-10T01:03:53.607 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-06-10T01:03:53.607 INFO:teuthology.orchestra.run.smithi037.stdout: "7788fb56": { 2024-06-10T01:03:53.607 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 16, 2024-06-10T01:03:53.607 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-06-10T01:03:53.607 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:03:53.607 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-06-10T01:03:53.607 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:03:53.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:03:53.607 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-06-10T01:03:53.607 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 149.4, 2024-06-10T01:03:53.607 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-06-10T01:03:53.607 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-06-10T01:03:53.607 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-06-10T01:03:53.608 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:03:53.608 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-06-10T01:03:53.608 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:03:53.608 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:03:53.608 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:03:53.608 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:03:53.608 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:03:53.608 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:03:53.608 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:03:53.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:53.608+0000 7f3a9e90a700 1 -- 172.21.15.37:0/3010357903 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f3a840a2ca0 msgr2=0x7f3a840a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:53.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:53.608+0000 7f3a9e90a700 1 --2- 172.21.15.37:0/3010357903 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f3a840a2ca0 0x7f3a840a5160 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3a88005950 tx=0x7f3a880058e0 comp rx=0 tx=0).stop 2024-06-10T01:03:53.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:53.608+0000 7f3a9e90a700 1 -- 172.21.15.37:0/3010357903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a9806b770 msgr2=0x7f3a98101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:53.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:53.608+0000 7f3a9e90a700 1 --2- 172.21.15.37:0/3010357903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a9806b770 0x7f3a98101d80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f3a800059d0 tx=0x7f3a800038b0 comp rx=0 tx=0).stop 2024-06-10T01:03:53.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:53.609+0000 7f3a9e90a700 1 -- 172.21.15.37:0/3010357903 shutdown_connections 2024-06-10T01:03:53.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:53.609+0000 7f3a9e90a700 1 --2- 172.21.15.37:0/3010357903 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f3a840a2ca0 0x7f3a840a5160 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:53.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:53.609+0000 7f3a9e90a700 1 --2- 172.21.15.37:0/3010357903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a98107860 0x7f3a981022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:53.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:53.609+0000 7f3a9e90a700 1 --2- 172.21.15.37:0/3010357903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a9806adc0 0x7f3a98101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:53.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:53.609+0000 7f3a9e90a700 1 --2- 172.21.15.37:0/3010357903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a9806b770 0x7f3a98101d80 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:53.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:53.609+0000 7f3a9e90a700 1 -- 172.21.15.37:0/3010357903 >> 172.21.15.37:0/3010357903 conn(0x7f3a98078340 msgr2=0x7f3a980ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:53.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:53.609+0000 7f3a9e90a700 1 -- 172.21.15.37:0/3010357903 shutdown_connections 2024-06-10T01:03:53.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:53.609+0000 7f3a9e90a700 1 -- 172.21.15.37:0/3010357903 wait complete. 2024-06-10T01:03:54.308 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7788fb56', total seconds elapsed: 59.4 2024-06-10T01:03:54.308 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.45 2024-06-10T01:03:54.308 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:03:54.568 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:54.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:54 smithi167 ceph-mon[36917]: pgmap v647: 129 pgs: 129 active+clean; 6.5 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.4 MiB/s wr, 2.90k op/s 2024-06-10T01:03:54.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:54 smithi094 ceph-mon[36708]: pgmap v647: 129 pgs: 129 active+clean; 6.5 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.4 MiB/s wr, 2.90k op/s 2024-06-10T01:03:54.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:54 smithi037 ceph-mon[32192]: pgmap v647: 129 pgs: 129 active+clean; 6.5 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.4 MiB/s wr, 2.90k op/s 2024-06-10T01:03:56.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.270+0000 7fd703d66700 1 -- 172.21.15.37:0/3658671918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6fc100e50 msgr2=0x7fd6fc1012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:56.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.270+0000 7fd703d66700 1 --2- 172.21.15.37:0/3658671918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6fc100e50 0x7fd6fc1012d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec009990 tx=0x7fd6ec009ca0 comp rx=0 tx=0).stop 2024-06-10T01:03:56.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.270+0000 7fd703d66700 1 -- 172.21.15.37:0/3658671918 shutdown_connections 2024-06-10T01:03:56.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.270+0000 7fd703d66700 1 --2- 172.21.15.37:0/3658671918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6fc101810 0x7fd6fc10f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:56.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.270+0000 7fd703d66700 1 --2- 172.21.15.37:0/3658671918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6fc107570 0x7fd6fc107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:56.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.270+0000 7fd703d66700 1 --2- 172.21.15.37:0/3658671918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6fc100e50 0x7fd6fc1012d0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:56.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.270+0000 7fd703d66700 1 -- 172.21.15.37:0/3658671918 >> 172.21.15.37:0/3658671918 conn(0x7fd6fc0766b0 msgr2=0x7fd6fc076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:56.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.270+0000 7fd703d66700 1 -- 172.21.15.37:0/3658671918 shutdown_connections 2024-06-10T01:03:56.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.270+0000 7fd703d66700 1 -- 172.21.15.37:0/3658671918 wait complete. 2024-06-10T01:03:56.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.271+0000 7fd703d66700 1 Processor -- start 2024-06-10T01:03:56.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.271+0000 7fd703d66700 1 -- start start 2024-06-10T01:03:56.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.271+0000 7fd703d66700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6fc100e50 0x7fd6fc19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:56.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.271+0000 7fd703d66700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6fc101810 0x7fd6fc19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:56.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.271+0000 7fd703d66700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6fc107570 0x7fd6fc198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:56.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.271+0000 7fd703d66700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd6fc1127b0 con 0x7fd6fc107570 2024-06-10T01:03:56.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.271+0000 7fd703d66700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd6fc112630 con 0x7fd6fc100e50 2024-06-10T01:03:56.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.271+0000 7fd703d66700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd6fc112930 con 0x7fd6fc101810 2024-06-10T01:03:56.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.271+0000 7fd702303700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6fc107570 0x7fd6fc198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:56.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.271+0000 7fd701301700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6fc101810 0x7fd6fc19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:56.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.271+0000 7fd701301700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6fc101810 0x7fd6fc19ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:51354/0 (socket says 172.21.15.37:51354) 2024-06-10T01:03:56.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.271+0000 7fd702303700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6fc107570 0x7fd6fc198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41050/0 (socket says 172.21.15.37:41050) 2024-06-10T01:03:56.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.271+0000 7fd701301700 1 -- 172.21.15.37:0/2923729435 learned_addr learned my addr 172.21.15.37:0/2923729435 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:56.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.271+0000 7fd701b02700 1 --2- 172.21.15.37:0/2923729435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6fc100e50 0x7fd6fc19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:56.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.272+0000 7fd701301700 1 -- 172.21.15.37:0/2923729435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6fc100e50 msgr2=0x7fd6fc19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:56.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.272+0000 7fd701301700 1 --2- 172.21.15.37:0/2923729435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6fc100e50 0x7fd6fc19e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:56.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.272+0000 7fd701301700 1 -- 172.21.15.37:0/2923729435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6fc107570 msgr2=0x7fd6fc198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:56.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.272+0000 7fd701301700 1 --2- 172.21.15.37:0/2923729435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6fc107570 0x7fd6fc198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:56.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.272+0000 7fd701301700 1 -- 172.21.15.37:0/2923729435 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6fc199170 con 0x7fd6fc101810 2024-06-10T01:03:56.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.272+0000 7fd701b02700 1 --2- 172.21.15.37:0/2923729435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6fc100e50 0x7fd6fc19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:03:56.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.272+0000 7fd701301700 1 --2- 172.21.15.37:0/2923729435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6fc101810 0x7fd6fc19ed90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec000c00 tx=0x7fd6ec0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:56.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.272+0000 7fd6f2ffd700 1 -- 172.21.15.37:0/2923729435 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec003880 con 0x7fd6fc101810 2024-06-10T01:03:56.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.272+0000 7fd703d66700 1 -- 172.21.15.37:0/2923729435 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6fc199400 con 0x7fd6fc101810 2024-06-10T01:03:56.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.272+0000 7fd6f2ffd700 1 -- 172.21.15.37:0/2923729435 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6ec01f4d0 con 0x7fd6fc101810 2024-06-10T01:03:56.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.272+0000 7fd6f2ffd700 1 -- 172.21.15.37:0/2923729435 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec00f6a0 con 0x7fd6fc101810 2024-06-10T01:03:56.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.272+0000 7fd703d66700 1 -- 172.21.15.37:0/2923729435 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6fc101f60 con 0x7fd6fc101810 2024-06-10T01:03:56.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.274+0000 7fd703d66700 1 -- 172.21.15.37:0/2923729435 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6fc069180 con 0x7fd6fc101810 2024-06-10T01:03:56.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.275+0000 7fd6f2ffd700 1 -- 172.21.15.37:0/2923729435 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd6ec009d10 con 0x7fd6fc101810 2024-06-10T01:03:56.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.275+0000 7fd6f2ffd700 1 --2- 172.21.15.37:0/2923729435 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd6e80a2ca0 0x7fd6e80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:56.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.276+0000 7fd701b02700 1 --2- 172.21.15.37:0/2923729435 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd6e80a2ca0 0x7fd6e80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:56.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.276+0000 7fd6f2ffd700 1 -- 172.21.15.37:0/2923729435 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fd6ec0d4610 con 0x7fd6fc101810 2024-06-10T01:03:56.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.276+0000 7fd701b02700 1 --2- 172.21.15.37:0/2923729435 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd6e80a2ca0 0x7fd6e80a5160 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc199b50 tx=0x7fd6f8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:56.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.280+0000 7fd6f2ffd700 1 -- 172.21.15.37:0/2923729435 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd6ec013070 con 0x7fd6fc101810 2024-06-10T01:03:56.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.539+0000 7fd703d66700 1 -- 172.21.15.37:0/2923729435 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6fc199f40 con 0x7fd6fc101810 2024-06-10T01:03:56.537 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:56 smithi037 ceph-mon[32192]: pgmap v648: 129 pgs: 129 active+clean; 7.1 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.0 MiB/s wr, 2.82k op/s 2024-06-10T01:03:56.537 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:03:56.537 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T01:03:24.246183+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26110":{"gid":26110,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/317977487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":317977487},{"type":"v1","addr":"172.21.15.167:6835","nonce":317977487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24458,24472,24451]},"id":1}]} 2024-06-10T01:03:56.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.539+0000 7fd6f2ffd700 1 -- 172.21.15.37:0/2923729435 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11864 (secure 0 0 0) 0x7fd6ec0a1210 con 0x7fd6fc101810 2024-06-10T01:03:56.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.542+0000 7fd703d66700 1 -- 172.21.15.37:0/2923729435 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd6e80a2ca0 msgr2=0x7fd6e80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:56.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.542+0000 7fd703d66700 1 --2- 172.21.15.37:0/2923729435 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd6e80a2ca0 0x7fd6e80a5160 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc199b50 tx=0x7fd6f8009260 comp rx=0 tx=0).stop 2024-06-10T01:03:56.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.542+0000 7fd703d66700 1 -- 172.21.15.37:0/2923729435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6fc101810 msgr2=0x7fd6fc19ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:56.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.542+0000 7fd703d66700 1 --2- 172.21.15.37:0/2923729435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6fc101810 0x7fd6fc19ed90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec000c00 tx=0x7fd6ec0047b0 comp rx=0 tx=0).stop 2024-06-10T01:03:56.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.542+0000 7fd703d66700 1 -- 172.21.15.37:0/2923729435 shutdown_connections 2024-06-10T01:03:56.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.542+0000 7fd703d66700 1 --2- 172.21.15.37:0/2923729435 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd6e80a2ca0 0x7fd6e80a5160 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:56.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.542+0000 7fd703d66700 1 --2- 172.21.15.37:0/2923729435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6fc107570 0x7fd6fc198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:56.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.542+0000 7fd703d66700 1 --2- 172.21.15.37:0/2923729435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6fc100e50 0x7fd6fc19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:56.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.542+0000 7fd703d66700 1 --2- 172.21.15.37:0/2923729435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6fc101810 0x7fd6fc19ed90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:56.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.542+0000 7fd703d66700 1 -- 172.21.15.37:0/2923729435 >> 172.21.15.37:0/2923729435 conn(0x7fd6fc0766b0 msgr2=0x7fd6fc0ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:56.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.542+0000 7fd703d66700 1 -- 172.21.15.37:0/2923729435 shutdown_connections 2024-06-10T01:03:56.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:03:56.542+0000 7fd703d66700 1 -- 172.21.15.37:0/2923729435 wait complete. 2024-06-10T01:03:56.541 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 29 2024-06-10T01:03:56.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:56 smithi167 ceph-mon[36917]: pgmap v648: 129 pgs: 129 active+clean; 7.1 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.0 MiB/s wr, 2.82k op/s 2024-06-10T01:03:56.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:56 smithi094 ceph-mon[36708]: pgmap v648: 129 pgs: 129 active+clean; 7.1 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.0 MiB/s wr, 2.82k op/s 2024-06-10T01:03:57.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:57 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2923729435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:03:57.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:03:57 smithi167 ceph-mon[36917]: pgmap v649: 129 pgs: 129 active+clean; 8.5 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 978 KiB/s wr, 2.56k op/s 2024-06-10T01:03:57.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:57 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2923729435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:03:57.611 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:03:57 smithi094 ceph-mon[36708]: pgmap v649: 129 pgs: 129 active+clean; 8.5 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 978 KiB/s wr, 2.56k op/s 2024-06-10T01:03:57.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:57 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2923729435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:03:57.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:03:57 smithi037 ceph-mon[32192]: pgmap v649: 129 pgs: 129 active+clean; 8.5 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 978 KiB/s wr, 2.56k op/s 2024-06-10T01:03:57.961 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24472 quiesce db / --timeout 90 --expiration 149.45 --await-for 45.0' 2024-06-10T01:03:57.961 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24472 quiesce db / --timeout 90 --expiration 149.45 --await-for 45.0 2024-06-10T01:03:58.225 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:03:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.505+0000 7f4ff3864640 1 -- 172.21.15.37:0/4202091857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec1024a0 msgr2=0x7f4fec104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.505+0000 7f4ff3864640 1 --2- 172.21.15.37:0/4202091857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec1024a0 0x7f4fec104890 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009900 tx=0x7f4fe802a410 comp rx=0 tx=0).stop 2024-06-10T01:03:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.505+0000 7f4ff3864640 1 -- 172.21.15.37:0/4202091857 shutdown_connections 2024-06-10T01:03:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.505+0000 7f4ff3864640 1 --2- 172.21.15.37:0/4202091857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fec107700 0x7f4fec109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.505+0000 7f4ff3864640 1 --2- 172.21.15.37:0/4202091857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fec104dd0 0x7f4fec1071c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.505+0000 7f4ff3864640 1 --2- 172.21.15.37:0/4202091857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec1024a0 0x7f4fec104890 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.505+0000 7f4ff3864640 1 -- 172.21.15.37:0/4202091857 >> 172.21.15.37:0/4202091857 conn(0x7f4fec0fbeb0 msgr2=0x7f4fec0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.505+0000 7f4ff3864640 1 -- 172.21.15.37:0/4202091857 shutdown_connections 2024-06-10T01:03:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.505+0000 7f4ff3864640 1 -- 172.21.15.37:0/4202091857 wait complete. 2024-06-10T01:03:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.506+0000 7f4ff3864640 1 Processor -- start 2024-06-10T01:03:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.506+0000 7f4ff3864640 1 -- start start 2024-06-10T01:03:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.506+0000 7f4ff3864640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec1024a0 0x7f4fec197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.506+0000 7f4ff3864640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fec104dd0 0x7f4fec1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.506+0000 7f4ff3864640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fec107700 0x7f4fec19f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.506+0000 7f4ff3864640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec10d6a0 con 0x7f4fec1024a0 2024-06-10T01:03:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.506+0000 7f4ff3864640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec10d520 con 0x7f4fec107700 2024-06-10T01:03:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.506+0000 7f4ff3864640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec10d820 con 0x7f4fec104dd0 2024-06-10T01:03:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.506+0000 7f4ff1600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec1024a0 0x7f4fec197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.506+0000 7f4ff1600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec1024a0 0x7f4fec197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41066/0 (socket says 172.21.15.37:41066) 2024-06-10T01:03:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.506+0000 7f4ff1600640 1 -- 172.21.15.37:0/2555402351 learned_addr learned my addr 172.21.15.37:0/2555402351 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.506+0000 7f4ff0dff640 1 --2- 172.21.15.37:0/2555402351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fec104dd0 0x7f4fec1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.506+0000 7f4ff1e01640 1 --2- 172.21.15.37:0/2555402351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fec107700 0x7f4fec19f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.507+0000 7f4ff1600640 1 -- 172.21.15.37:0/2555402351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fec104dd0 msgr2=0x7f4fec1981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.507+0000 7f4ff1600640 1 --2- 172.21.15.37:0/2555402351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fec104dd0 0x7f4fec1981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.507+0000 7f4ff1600640 1 -- 172.21.15.37:0/2555402351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fec107700 msgr2=0x7f4fec19f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.507+0000 7f4ff1600640 1 --2- 172.21.15.37:0/2555402351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fec107700 0x7f4fec19f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.507+0000 7f4ff1600640 1 -- 172.21.15.37:0/2555402351 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fec19f8d0 con 0x7f4fec1024a0 2024-06-10T01:03:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.507+0000 7f4ff0dff640 1 --2- 172.21.15.37:0/2555402351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fec104dd0 0x7f4fec1981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:03:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.507+0000 7f4ff1e01640 1 --2- 172.21.15.37:0/2555402351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fec107700 0x7f4fec19f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:03:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.507+0000 7f4ff1600640 1 --2- 172.21.15.37:0/2555402351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec1024a0 0x7f4fec197c90 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009230 tx=0x7f4fe8031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:58.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.507+0000 7f4fe27fc640 1 -- 172.21.15.37:0/2555402351 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe803f030 con 0x7f4fec1024a0 2024-06-10T01:03:58.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.507+0000 7f4fe27fc640 1 -- 172.21.15.37:0/2555402351 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe8003ad0 con 0x7f4fec1024a0 2024-06-10T01:03:58.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.507+0000 7f4ff3864640 1 -- 172.21.15.37:0/2555402351 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fec19fb60 con 0x7f4fec1024a0 2024-06-10T01:03:58.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.507+0000 7f4fe27fc640 1 -- 172.21.15.37:0/2555402351 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8035070 con 0x7f4fec1024a0 2024-06-10T01:03:58.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.507+0000 7f4ff3864640 1 -- 172.21.15.37:0/2555402351 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fec1a0050 con 0x7f4fec1024a0 2024-06-10T01:03:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.510+0000 7f4fe27fc640 1 -- 172.21.15.37:0/2555402351 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4fe8042030 con 0x7f4fec1024a0 2024-06-10T01:03:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.510+0000 7f4ff6571740 1 Processor -- start 2024-06-10T01:03:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.510+0000 7f4ff6571740 1 -- start start 2024-06-10T01:03:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.510+0000 7f4ff6571740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56471b0e0010 0x56471b0e0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.510+0000 7f4ff6571740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56471b0e0a00 0x56471b0e5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.510+0000 7f4ff6571740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471b0e5b70 0x56471b0e8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.510+0000 7f4ff6571740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56471b0434f0 con 0x56471b0e5b70 2024-06-10T01:03:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.510+0000 7f4ff6571740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56471afe1870 con 0x56471b0e0a00 2024-06-10T01:03:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.510+0000 7f4ff6571740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56471b03f4b0 con 0x56471b0e0010 2024-06-10T01:03:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.510+0000 7f4ff1e01640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56471b0e0010 0x56471b0e0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.511+0000 7f4ff0dff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471b0e5b70 0x56471b0e8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.511+0000 7f4ff1e01640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56471b0e0010 0x56471b0e0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:51390/0 (socket says 172.21.15.37:51390) 2024-06-10T01:03:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.511+0000 7f4ff1e01640 1 -- 172.21.15.37:0/4142001070 learned_addr learned my addr 172.21.15.37:0/4142001070 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.511+0000 7f4ff1600640 1 --2- 172.21.15.37:0/4142001070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56471b0e0a00 0x56471b0e5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.511+0000 7f4fe27fc640 1 --2- 172.21.15.37:0/2555402351 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4fd80a2c30 0x7f4fd80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.511+0000 7f4ff1e01640 1 -- 172.21.15.37:0/4142001070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56471b0e0a00 msgr2=0x56471b0e5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.511+0000 7f4ff1e01640 1 --2- 172.21.15.37:0/4142001070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56471b0e0a00 0x56471b0e5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.511+0000 7f4ff1e01640 1 -- 172.21.15.37:0/4142001070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471b0e5b70 msgr2=0x56471b0e8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.511+0000 7f4ff1e01640 1 --2- 172.21.15.37:0/4142001070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471b0e5b70 0x56471b0e8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.511+0000 7f4ff1e01640 1 -- 172.21.15.37:0/4142001070 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56471affd9b0 con 0x56471b0e0010 2024-06-10T01:03:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.511+0000 7f4ff1600640 1 --2- 172.21.15.37:0/4142001070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56471b0e0a00 0x56471b0e5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:03:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.511+0000 7f4ff0dff640 1 --2- 172.21.15.37:0/4142001070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471b0e5b70 0x56471b0e8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:03:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.511+0000 7f4fe27fc640 1 -- 172.21.15.37:0/2555402351 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f4fe80ef9c0 con 0x7f4fec1024a0 2024-06-10T01:03:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.511+0000 7f4ff1e01640 1 --2- 172.21.15.37:0/2555402351 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4fd80a2c30 0x7f4fd80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.511+0000 7f4ff1e01640 1 --2- 172.21.15.37:0/4142001070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56471b0e0010 0x56471b0e0430 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4010540 tx=0x7f4fe4010a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc86cdc67a46dd73 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.511+0000 7f4fd77fe640 1 -- 172.21.15.37:0/4142001070 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe40134c0 con 0x56471b0e0010 2024-06-10T01:03:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.511+0000 7f4ff1e01640 1 --2- 172.21.15.37:0/2555402351 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4fd80a2c30 0x7f4fd80a50f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4021000 tx=0x7f4fe401e000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.511+0000 7f4fd77fe640 1 -- 172.21.15.37:0/4142001070 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe401d8e0 con 0x56471b0e0010 2024-06-10T01:03:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.511+0000 7f4fd77fe640 1 -- 172.21.15.37:0/4142001070 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4011980 con 0x56471b0e0010 2024-06-10T01:03:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.512+0000 7f4ff6571740 1 -- 172.21.15.37:0/4142001070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56471b0e0010 msgr2=0x56471b0e0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.512+0000 7f4ff6571740 1 --2- 172.21.15.37:0/4142001070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56471b0e0010 0x56471b0e0430 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4010540 tx=0x7f4fe4010a00 comp rx=0 tx=0).stop 2024-06-10T01:03:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.512+0000 7f4ff6571740 1 -- 172.21.15.37:0/4142001070 shutdown_connections 2024-06-10T01:03:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.512+0000 7f4ff6571740 1 --2- 172.21.15.37:0/4142001070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471b0e5b70 0x56471b0e8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.512+0000 7f4ff6571740 1 --2- 172.21.15.37:0/4142001070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56471b0e0a00 0x56471b0e5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.512+0000 7f4ff6571740 1 --2- 172.21.15.37:0/4142001070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56471b0e0010 0x56471b0e0430 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.512+0000 7f4ff6571740 1 -- 172.21.15.37:0/4142001070 >> 172.21.15.37:0/4142001070 conn(0x56471b0db680 msgr2=0x56471b0dba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.512+0000 7f4ff6571740 1 -- 172.21.15.37:0/4142001070 shutdown_connections 2024-06-10T01:03:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.512+0000 7f4ff6571740 1 -- 172.21.15.37:0/4142001070 wait complete. 2024-06-10T01:03:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.513+0000 7f4ff6571740 1 -- start start 2024-06-10T01:03:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.513+0000 7f4ff6571740 1 Processor -- start 2024-06-10T01:03:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.513+0000 7f4ff6571740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56471b0e0010 0x56471b0ea370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.513+0000 7f4ff6571740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56471b0e0a00 0x56471b0ea8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.513+0000 7f4ff6571740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471b0e5b70 0x56471b0f1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.513+0000 7f4ff6571740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56471b0e8b20 con 0x56471b0e5b70 2024-06-10T01:03:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.513+0000 7f4ff6571740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56471afccae0 con 0x56471b0e0a00 2024-06-10T01:03:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.513+0000 7f4ff6571740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56471b0e8ca0 con 0x56471b0e0010 2024-06-10T01:03:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.513+0000 7f4ff6571740 10 client.0 _ll_register_callbacks cb 0x56471b0059b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:03:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.513+0000 7f4ff1600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56471b0e0010 0x56471b0ea370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.513+0000 7f4ff1e01640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471b0e5b70 0x56471b0f1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.513+0000 7f4ff1600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56471b0e0010 0x56471b0ea370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:51402/0 (socket says 172.21.15.37:51402) 2024-06-10T01:03:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.513+0000 7f4ff1600640 1 -- 172.21.15.37:0/2491104749 learned_addr learned my addr 172.21.15.37:0/2491104749 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.514+0000 7f4ff0dff640 1 --2- 172.21.15.37:0/2491104749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56471b0e0a00 0x56471b0ea8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.514+0000 7f4ff1600640 1 -- 172.21.15.37:0/2491104749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56471b0e0a00 msgr2=0x56471b0ea8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.514+0000 7f4ff1600640 1 --2- 172.21.15.37:0/2491104749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56471b0e0a00 0x56471b0ea8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.514+0000 7f4ff1600640 1 -- 172.21.15.37:0/2491104749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471b0e5b70 msgr2=0x56471b0f1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.514+0000 7f4ff1600640 1 --2- 172.21.15.37:0/2491104749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471b0e5b70 0x56471b0f1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.514+0000 7f4ff1600640 1 -- 172.21.15.37:0/2491104749 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56471ad57210 con 0x56471b0e0010 2024-06-10T01:03:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.514+0000 7f4ff0dff640 1 --2- 172.21.15.37:0/2491104749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56471b0e0a00 0x56471b0ea8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:03:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.514+0000 7f4ff1600640 1 --2- 172.21.15.37:0/2491104749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56471b0e0010 0x56471b0ea370 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80ed0d0 tx=0x7f4fe80b80d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53e22b737b9e0687 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.514+0000 7f4fd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:03:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.514+0000 7f4fd4ff9640 1 -- 172.21.15.37:0/2491104749 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8049050 con 0x56471b0e0010 2024-06-10T01:03:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.514+0000 7f4fd4ff9640 1 -- 172.21.15.37:0/2491104749 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f4fe80b84a0 con 0x56471b0e0010 2024-06-10T01:03:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.514+0000 7f4ff6571740 1 -- 172.21.15.37:0/2491104749 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fe401d8e0 con 0x56471b0e0010 2024-06-10T01:03:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.514+0000 7f4fd4ff9640 1 -- 172.21.15.37:0/2491104749 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4fe80c4aa0 con 0x56471b0e0010 2024-06-10T01:03:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.514+0000 7f4ff6571740 10 client.26179 fetch_fsmap learned FSMap version 29 2024-06-10T01:03:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.514+0000 7f4ff6571740 1 -- 172.21.15.37:0/2491104749 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56471ae23620 con 0x56471b0e0010 2024-06-10T01:03:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.515+0000 7f4fd4ff9640 1 -- 172.21.15.37:0/2491104749 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe401d8e0 con 0x56471b0e0010 2024-06-10T01:03:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.515+0000 7f4fd4ff9640 1 -- 172.21.15.37:0/2491104749 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f4fe8003c80 con 0x56471b0e0010 2024-06-10T01:03:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.515+0000 7f4ff6571740 10 client.26179 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:03:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.515+0000 7f4ff6571740 10 client.26179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:03:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.515+0000 7f4ff6571740 20 client.26179 populate_metadata read hostname 'smithi037' 2024-06-10T01:03:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.515+0000 7f4ff6571740 1 --2- 172.21.15.37:0/2491104749 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56471b0f6150 0x56471b0f85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.515+0000 7f4ff6571740 4 client.26179 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:03:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.515+0000 7f4ff6571740 1 -- 172.21.15.37:0/2491104749 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56471b056f10 con 0x56471b0f6150 2024-06-10T01:03:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.515+0000 7f4ff0dff640 1 --2- 172.21.15.37:0/2491104749 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56471b0f6150 0x56471b0f85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.515+0000 7f4fd4ff9640 10 client.26179 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:03:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.516+0000 7f4ff0dff640 1 --2- 172.21.15.37:0/2491104749 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56471b0f6150 0x56471b0f85e0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c73fa51129f757e server_cookie=8051de8efc92a3b in_seq=0 out_seq=0 2024-06-10T01:03:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.518+0000 7f4fd4ff9640 1 -- 172.21.15.37:0/2491104749 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56471b056f10 con 0x56471b0f6150 2024-06-10T01:03:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.518+0000 7f4fd4ff9640 10 client.26179 handle_command_reply: tid=0 2024-06-10T01:03:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.518+0000 7f4ff6571740 1 client.26179 shutdown 2024-06-10T01:03:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.519+0000 7f4ff6571740 1 -- 172.21.15.37:0/2491104749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56471b0e0010 msgr2=0x56471b0ea370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.519+0000 7f4ff6571740 1 --2- 172.21.15.37:0/2491104749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56471b0e0010 0x56471b0ea370 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80ed0d0 tx=0x7f4fe80b80d0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.519+0000 7f4ff6571740 1 -- 172.21.15.37:0/2491104749 shutdown_connections 2024-06-10T01:03:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.519+0000 7f4ff6571740 1 --2- 172.21.15.37:0/2491104749 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56471b0f6150 0x56471b0f85e0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.519+0000 7f4ff6571740 1 --2- 172.21.15.37:0/2491104749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471b0e5b70 0x56471b0f1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.519+0000 7f4fd4ff9640 0 client.26179 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:03:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.519+0000 7f4ff6571740 1 --2- 172.21.15.37:0/2491104749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56471b0e0a00 0x56471b0ea8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.519+0000 7f4ff6571740 1 --2- 172.21.15.37:0/2491104749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56471b0e0010 0x56471b0ea370 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.519+0000 7f4ff6571740 1 -- 172.21.15.37:0/2491104749 >> 172.21.15.37:0/2491104749 conn(0x56471b0db680 msgr2=0x56471b0da7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.519+0000 7f4ff6571740 1 -- 172.21.15.37:0/2491104749 shutdown_connections 2024-06-10T01:03:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.519+0000 7f4ff6571740 1 -- 172.21.15.37:0/2491104749 wait complete. 2024-06-10T01:03:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.519+0000 7f4ff6571740 20 client.26179 trim_cache size 0 max 16384 2024-06-10T01:03:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.535+0000 7f4ff6571740 1 Processor -- start 2024-06-10T01:03:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.535+0000 7f4ff6571740 1 -- start start 2024-06-10T01:03:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.536+0000 7f4ff6571740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471b0e0010 0x56471b0e9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.536+0000 7f4ff6571740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56471b0e0a00 0x56471b0ea4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.536+0000 7f4ff6571740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56471b0e5b70 0x56471b0de590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.536+0000 7f4ff6571740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1acf00 con 0x56471b0e0010 2024-06-10T01:03:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.536+0000 7f4ff6571740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1aa7d0 con 0x56471b0e0a00 2024-06-10T01:03:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.536+0000 7f4ff6571740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1a8620 con 0x56471b0e5b70 2024-06-10T01:03:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.536+0000 7f4ff1600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471b0e0010 0x56471b0e9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.536+0000 7f4ff1e01640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56471b0e5b70 0x56471b0de590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.536+0000 7f4ff1600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471b0e0010 0x56471b0e9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41092/0 (socket says 172.21.15.37:41092) 2024-06-10T01:03:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.536+0000 7f4ff1600640 1 -- 172.21.15.37:0/2626171532 learned_addr learned my addr 172.21.15.37:0/2626171532 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.536+0000 7f4ff0dff640 1 --2- 172.21.15.37:0/2626171532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56471b0e0a00 0x56471b0ea4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.536+0000 7f4ff1600640 1 -- 172.21.15.37:0/2626171532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56471b0e5b70 msgr2=0x56471b0de590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.536+0000 7f4ff1600640 1 --2- 172.21.15.37:0/2626171532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56471b0e5b70 0x56471b0de590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.536+0000 7f4ff1600640 1 -- 172.21.15.37:0/2626171532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56471b0e0a00 msgr2=0x56471b0ea4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.536+0000 7f4ff1600640 1 --2- 172.21.15.37:0/2626171532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56471b0e0a00 0x56471b0ea4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.536+0000 7f4ff1600640 1 -- 172.21.15.37:0/2626171532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56471b0f8c60 con 0x56471b0e0010 2024-06-10T01:03:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.536+0000 7f4ff1600640 1 --2- 172.21.15.37:0/2626171532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471b0e0010 0x56471b0e9f80 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80314e0 tx=0x7f4fe802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ffbce72d97051c4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.536+0000 7f4fd6ffd640 1 -- 172.21.15.37:0/2626171532 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8049050 con 0x56471b0e0010 2024-06-10T01:03:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.537+0000 7f4fd6ffd640 1 -- 172.21.15.37:0/2626171532 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe80bc530 con 0x56471b0e0010 2024-06-10T01:03:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.537+0000 7f4fd6ffd640 1 -- 172.21.15.37:0/2626171532 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe80b8600 con 0x56471b0e0010 2024-06-10T01:03:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.537+0000 7f4ff6571740 1 -- 172.21.15.37:0/2626171532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471b0e0010 msgr2=0x56471b0e9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:03:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.537+0000 7f4ff6571740 1 --2- 172.21.15.37:0/2626171532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471b0e0010 0x56471b0e9f80 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80314e0 tx=0x7f4fe802a410 comp rx=0 tx=0).stop 2024-06-10T01:03:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.537+0000 7f4ff6571740 1 -- 172.21.15.37:0/2626171532 shutdown_connections 2024-06-10T01:03:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.537+0000 7f4ff6571740 1 --2- 172.21.15.37:0/2626171532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56471b0e5b70 0x56471b0de590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.537+0000 7f4ff6571740 1 --2- 172.21.15.37:0/2626171532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56471b0e0a00 0x56471b0ea4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.537+0000 7f4ff6571740 1 --2- 172.21.15.37:0/2626171532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471b0e0010 0x56471b0e9f80 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.537+0000 7f4ff6571740 1 -- 172.21.15.37:0/2626171532 >> 172.21.15.37:0/2626171532 conn(0x56471b0db680 msgr2=0x56471b0dba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:03:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.537+0000 7f4ff6571740 1 -- 172.21.15.37:0/2626171532 shutdown_connections 2024-06-10T01:03:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.537+0000 7f4ff6571740 1 -- 172.21.15.37:0/2626171532 wait complete. 2024-06-10T01:03:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.538+0000 7f4ff6571740 1 -- start start 2024-06-10T01:03:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.538+0000 7f4ff6571740 1 Processor -- start 2024-06-10T01:03:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.538+0000 7f4ff6571740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56471b0e0010 0x56471b15bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.538+0000 7f4ff6571740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471b0e0a00 0x56471b15c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.538+0000 7f4ff6571740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56471b0e5b70 0x56471b1631a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.538+0000 7f4ff6571740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1a8c90 con 0x56471b0e0a00 2024-06-10T01:03:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.538+0000 7f4ff6571740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1aa100 con 0x56471b0e5b70 2024-06-10T01:03:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.538+0000 7f4ff6571740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1a9360 con 0x56471b0e0010 2024-06-10T01:03:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.538+0000 7f4ff6571740 10 client.0 _ll_register_callbacks cb 0x7f4fe8005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:03:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.538+0000 7f4ff0dff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471b0e0a00 0x56471b15c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.538+0000 7f4ff1600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56471b0e0010 0x56471b15bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.538+0000 7f4ff1600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56471b0e0010 0x56471b15bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:51424/0 (socket says 172.21.15.37:51424) 2024-06-10T01:03:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.538+0000 7f4ff0dff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471b0e0a00 0x56471b15c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41102/0 (socket says 172.21.15.37:41102) 2024-06-10T01:03:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.538+0000 7f4ff1600640 1 -- 172.21.15.37:0/520129529 learned_addr learned my addr 172.21.15.37:0/520129529 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:03:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.538+0000 7f4ff1e01640 1 --2- 172.21.15.37:0/520129529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56471b0e5b70 0x56471b1631a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.539+0000 7f4ff1600640 1 -- 172.21.15.37:0/520129529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56471b0e5b70 msgr2=0x56471b1631a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.539+0000 7f4ff1600640 1 --2- 172.21.15.37:0/520129529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56471b0e5b70 0x56471b1631a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.539+0000 7f4ff1600640 1 -- 172.21.15.37:0/520129529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471b0e0a00 msgr2=0x56471b15c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:03:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.539+0000 7f4ff1600640 1 --2- 172.21.15.37:0/520129529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471b0e0a00 0x56471b15c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:03:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.539+0000 7f4ff1600640 1 -- 172.21.15.37:0/520129529 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56471b163790 con 0x56471b0e0010 2024-06-10T01:03:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.539+0000 7f4ff0dff640 1 --2- 172.21.15.37:0/520129529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471b0e0a00 0x56471b15c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:03:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.539+0000 7f4ff1600640 1 --2- 172.21.15.37:0/520129529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56471b0e0010 0x56471b15bc20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80d6040 tx=0x7f4fe80c0360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75d0836bc188c75 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:03:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.539+0000 7f4fd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:03:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.539+0000 7f4fd4ff9640 1 -- 172.21.15.37:0/520129529 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8049030 con 0x56471b0e0010 2024-06-10T01:03:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.539+0000 7f4fd4ff9640 1 -- 172.21.15.37:0/520129529 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f4fe8003c80 con 0x56471b0e0010 2024-06-10T01:03:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.539+0000 7f4ff6571740 1 -- 172.21.15.37:0/520129529 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fe80bc530 con 0x56471b0e0010 2024-06-10T01:03:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.539+0000 7f4fd4ff9640 1 -- 172.21.15.37:0/520129529 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4fe8031bd0 con 0x56471b0e0010 2024-06-10T01:03:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.539+0000 7f4ff6571740 10 client.26185 fetch_fsmap learned FSMap version 29 2024-06-10T01:03:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.539+0000 7f4ff6571740 1 -- 172.21.15.37:0/520129529 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56471b163790 con 0x56471b0e0010 2024-06-10T01:03:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.539+0000 7f4fd4ff9640 1 -- 172.21.15.37:0/520129529 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe80bc530 con 0x56471b0e0010 2024-06-10T01:03:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.539+0000 7f4fd4ff9640 1 -- 172.21.15.37:0/520129529 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f4fe80bb280 con 0x56471b0e0010 2024-06-10T01:03:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.540+0000 7f4ff6571740 10 client.26185 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:03:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.540+0000 7f4ff6571740 10 client.26185 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:03:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.540+0000 7f4ff6571740 20 client.26185 populate_metadata read hostname 'smithi037' 2024-06-10T01:03:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.540+0000 7f4ff6571740 1 --2- 172.21.15.37:0/520129529 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56471b164f80 0x56471b167370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:03:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.540+0000 7f4ff6571740 4 client.26185 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:03:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.540+0000 7f4ff6571740 1 -- 172.21.15.37:0/520129529 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56471b050c20 con 0x56471b164f80 2024-06-10T01:03:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.540+0000 7f4ff0dff640 1 --2- 172.21.15.37:0/520129529 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56471b164f80 0x56471b167370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:03:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.540+0000 7f4ff0dff640 1 --2- 172.21.15.37:0/520129529 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56471b164f80 0x56471b167370 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91f9fe6f9f78fc02 server_cookie=7469157e6c61b1b8 in_seq=0 out_seq=0 2024-06-10T01:03:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:03:58.540+0000 7f4fd4ff9640 10 client.26185 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:00.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.108+0000 7f7d1b3eb700 1 -- 172.21.15.37:0/2106387006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d141013a0 msgr2=0x7f7d1410f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:00.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.108+0000 7f7d1b3eb700 1 --2- 172.21.15.37:0/2106387006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d141013a0 0x7f7d1410f330 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009fc0 tx=0x7f7d0c00b330 comp rx=0 tx=0).stop 2024-06-10T01:04:00.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.110+0000 7f7d1b3eb700 1 -- 172.21.15.37:0/2106387006 shutdown_connections 2024-06-10T01:04:00.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.111+0000 7f7d1b3eb700 1 --2- 172.21.15.37:0/2106387006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d141013a0 0x7f7d1410f330 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:00.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.111+0000 7f7d1b3eb700 1 --2- 172.21.15.37:0/2106387006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d14108560 0x7f7d14108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:00.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.111+0000 7f7d1b3eb700 1 --2- 172.21.15.37:0/2106387006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d141009e0 0x7f7d14100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:00.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.111+0000 7f7d1b3eb700 1 -- 172.21.15.37:0/2106387006 >> 172.21.15.37:0/2106387006 conn(0x7f7d14077990 msgr2=0x7f7d140fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:00.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.111+0000 7f7d1b3eb700 1 -- 172.21.15.37:0/2106387006 shutdown_connections 2024-06-10T01:04:00.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.113+0000 7f7d1b3eb700 1 -- 172.21.15.37:0/2106387006 wait complete. 2024-06-10T01:04:00.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.114+0000 7f7d1b3eb700 1 Processor -- start 2024-06-10T01:04:00.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.114+0000 7f7d1b3eb700 1 -- start start 2024-06-10T01:04:00.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.116+0000 7f7d1b3eb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d141009e0 0x7f7d1419e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:00.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.116+0000 7f7d1b3eb700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d141013a0 0x7f7d1419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:00.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.116+0000 7f7d1b3eb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d14108560 0x7f7d141989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:00.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.117+0000 7f7d18986700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d141013a0 0x7f7d1419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:00.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.117+0000 7f7d18986700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d141013a0 0x7f7d1419ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:51438/0 (socket says 172.21.15.37:51438) 2024-06-10T01:04:00.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.117+0000 7f7d18986700 1 -- 172.21.15.37:0/1675054791 learned_addr learned my addr 172.21.15.37:0/1675054791 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:00.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.117+0000 7f7d1b3eb700 1 -- 172.21.15.37:0/1675054791 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7d14112870 con 0x7f7d14108560 2024-06-10T01:04:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.117+0000 7f7d1b3eb700 1 -- 172.21.15.37:0/1675054791 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7d141126f0 con 0x7f7d141009e0 2024-06-10T01:04:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.117+0000 7f7d1b3eb700 1 -- 172.21.15.37:0/1675054791 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7d141129f0 con 0x7f7d141013a0 2024-06-10T01:04:00.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.120+0000 7f7d19988700 1 --2- 172.21.15.37:0/1675054791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d14108560 0x7f7d141989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:00.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.121+0000 7f7d19187700 1 --2- 172.21.15.37:0/1675054791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d141009e0 0x7f7d1419e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:00.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.121+0000 7f7d18986700 1 -- 172.21.15.37:0/1675054791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d141009e0 msgr2=0x7f7d1419e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:00.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.121+0000 7f7d18986700 1 --2- 172.21.15.37:0/1675054791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d141009e0 0x7f7d1419e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:00.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.121+0000 7f7d18986700 1 -- 172.21.15.37:0/1675054791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d14108560 msgr2=0x7f7d141989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:00.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.122+0000 7f7d18986700 1 --2- 172.21.15.37:0/1675054791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d14108560 0x7f7d141989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:00.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.122+0000 7f7d18986700 1 -- 172.21.15.37:0/1675054791 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d14199150 con 0x7f7d141013a0 2024-06-10T01:04:00.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.122+0000 7f7d18986700 1 --2- 172.21.15.37:0/1675054791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d141013a0 0x7f7d1419ed20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400b530 tx=0x7f7d0400b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:00.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.123+0000 7f7d0a7fc700 1 -- 172.21.15.37:0/1675054791 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d04015020 con 0x7f7d141013a0 2024-06-10T01:04:00.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.123+0000 7f7d0a7fc700 1 -- 172.21.15.37:0/1675054791 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d04011dd0 con 0x7f7d141013a0 2024-06-10T01:04:00.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.123+0000 7f7d0a7fc700 1 -- 172.21.15.37:0/1675054791 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0400f2e0 con 0x7f7d141013a0 2024-06-10T01:04:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.123+0000 7f7d1b3eb700 1 -- 172.21.15.37:0/1675054791 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d14199440 con 0x7f7d141013a0 2024-06-10T01:04:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.123+0000 7f7d1b3eb700 1 -- 172.21.15.37:0/1675054791 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d14101af0 con 0x7f7d141013a0 2024-06-10T01:04:00.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.130+0000 7f7d0a7fc700 1 -- 172.21.15.37:0/1675054791 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7d04018020 con 0x7f7d141013a0 2024-06-10T01:04:00.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.131+0000 7f7d0a7fc700 1 --2- 172.21.15.37:0/1675054791 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7d000a2e10 0x7f7d000a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:00.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.131+0000 7f7d0a7fc700 1 -- 172.21.15.37:0/1675054791 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f7d040cec80 con 0x7f7d141013a0 2024-06-10T01:04:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.134+0000 7f7d25c6eb80 1 Processor -- start 2024-06-10T01:04:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.134+0000 7f7d19187700 1 --2- 172.21.15.37:0/1675054791 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7d000a2e10 0x7f7d000a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.135+0000 7f7d25c6eb80 1 -- start start 2024-06-10T01:04:00.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.135+0000 7f7d25c6eb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1d84cbce0 0x55f1d84cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:00.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.135+0000 7f7d25c6eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1d84cc600 0x55f1d84c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:00.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.136+0000 7f7d25c6eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1d84c6470 0x55f1d84c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:00.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.136+0000 7f7d19187700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1d84cbce0 0x55f1d84cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:00.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.136+0000 7f7d19187700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1d84cbce0 0x55f1d84cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:58138/0 (socket says 172.21.15.37:58138) 2024-06-10T01:04:00.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.136+0000 7f7d19187700 1 -- 172.21.15.37:0/359977143 learned_addr learned my addr 172.21.15.37:0/359977143 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:00.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.136+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/359977143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f1d84179b0 con 0x55f1d84cc600 2024-06-10T01:04:00.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.136+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/359977143 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f1d83bf8c0 con 0x55f1d84cbce0 2024-06-10T01:04:00.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.136+0000 7f7d19988700 1 --2- 172.21.15.37:0/359977143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1d84c6470 0x55f1d84c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:00.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.136+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/359977143 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f1d8413970 con 0x55f1d84c6470 2024-06-10T01:04:00.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.141+0000 7f7d19988700 1 -- 172.21.15.37:0/359977143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1d84cbce0 msgr2=0x55f1d84cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:00.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.141+0000 7f7d19988700 1 --2- 172.21.15.37:0/359977143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1d84cbce0 0x55f1d84cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:00.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.141+0000 7f7d19988700 1 -- 172.21.15.37:0/359977143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1d84cc600 msgr2=0x55f1d84c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:00.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.141+0000 7f7d19988700 1 --2- 172.21.15.37:0/359977143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1d84cc600 0x55f1d84c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:00.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.141+0000 7f7d19988700 1 -- 172.21.15.37:0/359977143 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1d83b60f0 con 0x55f1d84c6470 2024-06-10T01:04:00.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.144+0000 7f7d19988700 1 --2- 172.21.15.37:0/359977143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1d84c6470 0x55f1d84c68b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00b770 tx=0x7f7d0c009d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84089b2c91e6f357 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:00.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.144+0000 7f7d19187700 1 --2- 172.21.15.37:0/1675054791 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7d000a2e10 0x7f7d000a52d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009c10 tx=0x7f7d10010e60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:00.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.144+0000 7f7cff7fe700 1 -- 172.21.15.37:0/359977143 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c00bda0 con 0x55f1d84c6470 2024-06-10T01:04:00.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.144+0000 7f7cff7fe700 1 -- 172.21.15.37:0/359977143 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d0c006040 con 0x55f1d84c6470 2024-06-10T01:04:00.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.144+0000 7f7cff7fe700 1 -- 172.21.15.37:0/359977143 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c005040 con 0x55f1d84c6470 2024-06-10T01:04:00.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.145+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/359977143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1d84c6470 msgr2=0x55f1d84c68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:00.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.145+0000 7f7d25c6eb80 1 --2- 172.21.15.37:0/359977143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1d84c6470 0x55f1d84c68b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00b770 tx=0x7f7d0c009d60 comp rx=0 tx=0).stop 2024-06-10T01:04:00.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.150+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/359977143 shutdown_connections 2024-06-10T01:04:00.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.150+0000 7f7d25c6eb80 1 --2- 172.21.15.37:0/359977143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1d84c6470 0x55f1d84c68b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:00.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.150+0000 7f7d25c6eb80 1 --2- 172.21.15.37:0/359977143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1d84cbce0 0x55f1d84cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:00.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.150+0000 7f7d25c6eb80 1 --2- 172.21.15.37:0/359977143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1d84cc600 0x55f1d84c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:00.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.150+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/359977143 >> 172.21.15.37:0/359977143 conn(0x55f1d83f0e60 msgr2=0x55f1d84c11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:00.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.150+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/359977143 shutdown_connections 2024-06-10T01:04:00.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.150+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/359977143 wait complete. 2024-06-10T01:04:00.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.151+0000 7f7d25c6eb80 1 -- start start 2024-06-10T01:04:00.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.151+0000 7f7d25c6eb80 1 Processor -- start 2024-06-10T01:04:00.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.152+0000 7f7d25c6eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1d84c6470 0x55f1d84d2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:00.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.152+0000 7f7d25c6eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1d84cbce0 0x55f1d84cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:00.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.152+0000 7f7d25c6eb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1d84cc600 0x55f1d84cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:00.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.152+0000 7f7d19187700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1d84c6470 0x55f1d84d2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:00.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.152+0000 7f7d19187700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1d84c6470 0x55f1d84d2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41142/0 (socket says 172.21.15.37:41142) 2024-06-10T01:04:00.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.152+0000 7f7d19187700 1 -- 172.21.15.37:0/3202708964 learned_addr learned my addr 172.21.15.37:0/3202708964 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:00.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.152+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/3202708964 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f1d84d3010 con 0x55f1d84c6470 2024-06-10T01:04:00.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.153+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/3202708964 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f1d83b7d90 con 0x55f1d84cc600 2024-06-10T01:04:00.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.153+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/3202708964 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f1d84d3190 con 0x55f1d84cbce0 2024-06-10T01:04:00.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.153+0000 7f7d25c6eb80 10 client.0 _ll_register_callbacks cb 0x55f1d83b1ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:00.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.156+0000 7f7d19187700 1 -- 172.21.15.37:0/3202708964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1d84cbce0 msgr2=0x55f1d84cf730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:04:00.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.156+0000 7f7d19187700 1 --2- 172.21.15.37:0/3202708964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1d84cbce0 0x55f1d84cf730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:00.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.156+0000 7f7d19187700 1 -- 172.21.15.37:0/3202708964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1d84cc600 msgr2=0x55f1d84cfc70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:04:00.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.156+0000 7f7d19187700 1 --2- 172.21.15.37:0/3202708964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1d84cc600 0x55f1d84cfc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:00.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.156+0000 7f7d19187700 1 -- 172.21.15.37:0/3202708964 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1d8366a70 con 0x55f1d84c6470 2024-06-10T01:04:00.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.157+0000 7f7d19187700 1 --2- 172.21.15.37:0/3202708964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1d84c6470 0x55f1d84d2430 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7d100096f0 tx=0x7f7d1001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5951bba4a6f148 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:00.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.160+0000 7f7cfcff9700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:04:00.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.160+0000 7f7cfcff9700 1 -- 172.21.15.37:0/3202708964 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10004a30 con 0x55f1d84c6470 2024-06-10T01:04:00.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.160+0000 7f7cfcff9700 1 -- 172.21.15.37:0/3202708964 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f7d10004bd0 con 0x55f1d84c6470 2024-06-10T01:04:00.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.161+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/3202708964 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d0c006040 con 0x55f1d84c6470 2024-06-10T01:04:00.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.166+0000 7f7cfcff9700 1 -- 172.21.15.37:0/3202708964 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7d10004dd0 con 0x55f1d84c6470 2024-06-10T01:04:00.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.166+0000 7f7d25c6eb80 10 client.16638 fetch_fsmap learned FSMap version 29 2024-06-10T01:04:00.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.166+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/3202708964 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f1d84dd960 con 0x55f1d84c6470 2024-06-10T01:04:00.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.167+0000 7f7cfcff9700 1 -- 172.21.15.37:0/3202708964 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d0c006040 con 0x55f1d84c6470 2024-06-10T01:04:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.167+0000 7f7cfcff9700 1 -- 172.21.15.37:0/3202708964 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f7d1000f990 con 0x55f1d84c6470 2024-06-10T01:04:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.167+0000 7f7d25c6eb80 10 client.16638 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:04:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.167+0000 7f7d25c6eb80 10 client.16638 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:04:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.167+0000 7f7d25c6eb80 20 client.16638 populate_metadata read hostname 'smithi037' 2024-06-10T01:04:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.167+0000 7f7d25c6eb80 1 --2- 172.21.15.37:0/3202708964 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f1d84dff40 0x55f1d84e2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.168+0000 7f7d25c6eb80 4 client.16638 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:04:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.168+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/3202708964 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1d8424eb0 con 0x55f1d84dff40 2024-06-10T01:04:00.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.176+0000 7f7d18986700 1 --2- 172.21.15.37:0/3202708964 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f1d84dff40 0x55f1d84e2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:00.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.180+0000 7f7cfcff9700 10 client.16638 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:00.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.180+0000 7f7d18986700 1 --2- 172.21.15.37:0/3202708964 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f1d84dff40 0x55f1d84e2370 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65746c87e73265fa server_cookie=3260aad5875d5930 in_seq=0 out_seq=0 2024-06-10T01:04:00.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.197+0000 7f7cfcff9700 1 -- 172.21.15.37:0/3202708964 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f1d8424eb0 con 0x55f1d84dff40 2024-06-10T01:04:00.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.197+0000 7f7cfcff9700 10 client.16638 handle_command_reply: tid=0 2024-06-10T01:04:00.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.197+0000 7f7d25c6eb80 1 client.16638 shutdown 2024-06-10T01:04:00.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.199+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/3202708964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1d84c6470 msgr2=0x55f1d84d2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:00.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.199+0000 7f7d25c6eb80 1 --2- 172.21.15.37:0/3202708964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1d84c6470 0x55f1d84d2430 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7d100096f0 tx=0x7f7d1001e000 comp rx=0 tx=0).stop 2024-06-10T01:04:00.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.199+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/3202708964 shutdown_connections 2024-06-10T01:04:00.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.199+0000 7f7d25c6eb80 1 --2- 172.21.15.37:0/3202708964 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f1d84dff40 0x55f1d84e2370 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:00.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.199+0000 7f7d25c6eb80 1 --2- 172.21.15.37:0/3202708964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1d84cc600 0x55f1d84cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:00.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.199+0000 7f7cfcff9700 0 client.16638 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:00.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.199+0000 7f7d25c6eb80 1 --2- 172.21.15.37:0/3202708964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1d84c6470 0x55f1d84d2430 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:00.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.199+0000 7f7d25c6eb80 1 --2- 172.21.15.37:0/3202708964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1d84cbce0 0x55f1d84cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:00.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.199+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/3202708964 >> 172.21.15.37:0/3202708964 conn(0x55f1d83f0e60 msgr2=0x55f1d84c0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:00.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.200+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/3202708964 shutdown_connections 2024-06-10T01:04:00.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.202+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/3202708964 wait complete. 2024-06-10T01:04:00.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.202+0000 7f7d25c6eb80 20 client.16638 trim_cache size 0 max 16384 2024-06-10T01:04:00.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.246+0000 7f7d25c6eb80 1 Processor -- start 2024-06-10T01:04:00.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.246+0000 7f7d25c6eb80 1 -- start start 2024-06-10T01:04:00.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.247+0000 7f7d25c6eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1d84c6470 0x55f1d84c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:00.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.247+0000 7f7d25c6eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1d84cbce0 0x55f1d84c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:00.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.247+0000 7f7d25c6eb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1d84cc600 0x55f1d84cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:00.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.247+0000 7f7d25c6eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf8011630 con 0x55f1d84c6470 2024-06-10T01:04:00.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.247+0000 7f7d25c6eb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf800eec0 con 0x55f1d84cc600 2024-06-10T01:04:00.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.247+0000 7f7d25c6eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf800cd10 con 0x55f1d84cbce0 2024-06-10T01:04:00.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.247+0000 7f7d19187700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1d84c6470 0x55f1d84c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:00.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.247+0000 7f7d19187700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1d84c6470 0x55f1d84c5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41146/0 (socket says 172.21.15.37:41146) 2024-06-10T01:04:00.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.247+0000 7f7d19187700 1 -- 172.21.15.37:0/2852894520 learned_addr learned my addr 172.21.15.37:0/2852894520 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:00.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.247+0000 7f7d19187700 1 -- 172.21.15.37:0/2852894520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1d84cbce0 msgr2=0x55f1d84c1210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:04:00.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.247+0000 7f7d19187700 1 --2- 172.21.15.37:0/2852894520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1d84cbce0 0x55f1d84c1210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:00.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.247+0000 7f7d19187700 1 -- 172.21.15.37:0/2852894520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1d84cc600 msgr2=0x55f1d84cfca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:04:00.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.247+0000 7f7d19187700 1 --2- 172.21.15.37:0/2852894520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1d84cc600 0x55f1d84cfca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:00.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.247+0000 7f7d19187700 1 -- 172.21.15.37:0/2852894520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1d84cdca0 con 0x55f1d84c6470 2024-06-10T01:04:00.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.247+0000 7f7d19187700 1 --2- 172.21.15.37:0/2852894520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1d84c6470 0x55f1d84c5f30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000a010 tx=0x7f7d100109b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b41ba4a4355bfb6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:00.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.248+0000 7f7cfeffd700 1 -- 172.21.15.37:0/2852894520 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10038030 con 0x55f1d84c6470 2024-06-10T01:04:00.246 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:00 smithi037 ceph-mon[32192]: pgmap v650: 129 pgs: 129 active+clean; 12 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 794 KiB/s wr, 2.35k op/s 2024-06-10T01:04:00.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.248+0000 7f7cfeffd700 1 -- 172.21.15.37:0/2852894520 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d10025070 con 0x55f1d84c6470 2024-06-10T01:04:00.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.249+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/2852894520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1d84c6470 msgr2=0x55f1d84c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:00.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.249+0000 7f7d25c6eb80 1 --2- 172.21.15.37:0/2852894520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1d84c6470 0x55f1d84c5f30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000a010 tx=0x7f7d100109b0 comp rx=0 tx=0).stop 2024-06-10T01:04:00.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.250+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/2852894520 shutdown_connections 2024-06-10T01:04:00.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.250+0000 7f7d25c6eb80 1 --2- 172.21.15.37:0/2852894520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1d84cc600 0x55f1d84cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:00.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.250+0000 7f7d25c6eb80 1 --2- 172.21.15.37:0/2852894520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1d84c6470 0x55f1d84c5f30 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:00.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.250+0000 7f7d25c6eb80 1 --2- 172.21.15.37:0/2852894520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1d84cbce0 0x55f1d84c1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:00.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.250+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/2852894520 >> 172.21.15.37:0/2852894520 conn(0x55f1d83f0e60 msgr2=0x55f1d84cca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:00.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.253+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/2852894520 shutdown_connections 2024-06-10T01:04:00.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.253+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/2852894520 wait complete. 2024-06-10T01:04:00.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.254+0000 7f7d25c6eb80 1 -- start start 2024-06-10T01:04:00.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.254+0000 7f7d25c6eb80 1 Processor -- start 2024-06-10T01:04:00.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.254+0000 7f7d25c6eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1d84c6470 0x55f1d84d7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:00.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.254+0000 7f7d25c6eb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1d84cbce0 0x55f1d84d7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:00.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.254+0000 7f7d25c6eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1d84cc600 0x55f1d84d8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:00.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.254+0000 7f7d25c6eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf800d380 con 0x55f1d84c6470 2024-06-10T01:04:00.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.254+0000 7f7d25c6eb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf800e7f0 con 0x55f1d84cbce0 2024-06-10T01:04:00.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.255+0000 7f7d25c6eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf800da50 con 0x55f1d84cc600 2024-06-10T01:04:00.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.255+0000 7f7d25c6eb80 10 client.0 _ll_register_callbacks cb 0x55f1d83b1ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:00.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.256+0000 7f7d19187700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1d84c6470 0x55f1d84d7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:00.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.257+0000 7f7d19187700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1d84c6470 0x55f1d84d7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41148/0 (socket says 172.21.15.37:41148) 2024-06-10T01:04:00.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.257+0000 7f7d19187700 1 -- 172.21.15.37:0/2994816611 learned_addr learned my addr 172.21.15.37:0/2994816611 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:00.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.257+0000 7f7d19988700 1 --2- 172.21.15.37:0/2994816611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1d84cc600 0x55f1d84d8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:00.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.258+0000 7f7d18986700 1 --2- 172.21.15.37:0/2994816611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1d84cbce0 0x55f1d84d7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:00.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.258+0000 7f7d19187700 1 -- 172.21.15.37:0/2994816611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1d84cc600 msgr2=0x55f1d84d8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:00.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.258+0000 7f7d19187700 1 --2- 172.21.15.37:0/2994816611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1d84cc600 0x55f1d84d8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:00.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.258+0000 7f7d19187700 1 -- 172.21.15.37:0/2994816611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1d84cbce0 msgr2=0x55f1d84d7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:00.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.258+0000 7f7d19187700 1 --2- 172.21.15.37:0/2994816611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1d84cbce0 0x55f1d84d7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:00.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.258+0000 7f7d19187700 1 -- 172.21.15.37:0/2994816611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1d84d8a40 con 0x55f1d84c6470 2024-06-10T01:04:00.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.258+0000 7f7d18986700 1 --2- 172.21.15.37:0/2994816611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1d84cbce0 0x55f1d84d7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:04:00.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.258+0000 7f7d19187700 1 --2- 172.21.15.37:0/2994816611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1d84c6470 0x55f1d84d7690 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000fde0 tx=0x7f7d1000fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ca18a3350a4909 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:00.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.259+0000 7f7cfcff9700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:04:00.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.259+0000 7f7cfcff9700 1 -- 172.21.15.37:0/2994816611 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10038040 con 0x55f1d84c6470 2024-06-10T01:04:00.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.259+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/2994816611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d10025070 con 0x55f1d84c6470 2024-06-10T01:04:00.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.259+0000 7f7cfcff9700 1 -- 172.21.15.37:0/2994816611 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f7d10005170 con 0x55f1d84c6470 2024-06-10T01:04:00.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.264+0000 7f7cfcff9700 1 -- 172.21.15.37:0/2994816611 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7d1004b050 con 0x55f1d84c6470 2024-06-10T01:04:00.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.265+0000 7f7d25c6eb80 10 client.16644 fetch_fsmap learned FSMap version 29 2024-06-10T01:04:00.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.265+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/2994816611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f1d8564e40 con 0x55f1d84c6470 2024-06-10T01:04:00.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.266+0000 7f7cfcff9700 1 -- 172.21.15.37:0/2994816611 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d10025070 con 0x55f1d84c6470 2024-06-10T01:04:00.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.266+0000 7f7cfcff9700 1 -- 172.21.15.37:0/2994816611 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f7d100337f0 con 0x55f1d84c6470 2024-06-10T01:04:00.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.266+0000 7f7d25c6eb80 10 client.16644 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:04:00.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.266+0000 7f7d25c6eb80 10 client.16644 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:04:00.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.266+0000 7f7d25c6eb80 20 client.16644 populate_metadata read hostname 'smithi037' 2024-06-10T01:04:00.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.266+0000 7f7d25c6eb80 1 --2- 172.21.15.37:0/2994816611 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f1d84dff40 0x55f1d8567540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:00.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.266+0000 7f7d25c6eb80 4 client.16644 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.45, "await_for": 45.0}] 2024-06-10T01:04:00.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.266+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/2994816611 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.45, "await_for": 45.0}) -- 0x55f1d841f430 con 0x55f1d84dff40 2024-06-10T01:04:00.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.270+0000 7f7d18986700 1 --2- 172.21.15.37:0/2994816611 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f1d84dff40 0x55f1d8567540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:00.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.280+0000 7f7cfcff9700 10 client.16644 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:00.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:00.280+0000 7f7d18986700 1 --2- 172.21.15.37:0/2994816611 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f1d84dff40 0x55f1d8567540 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1beb9fd4a4e690dd server_cookie=5c1f382c1ac66775 in_seq=0 out_seq=0 2024-06-10T01:04:00.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:00 smithi167 ceph-mon[36917]: pgmap v650: 129 pgs: 129 active+clean; 12 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 794 KiB/s wr, 2.35k op/s 2024-06-10T01:04:00.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:00 smithi094 ceph-mon[36708]: pgmap v650: 129 pgs: 129 active+clean; 12 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 794 KiB/s wr, 2.35k op/s 2024-06-10T01:04:02.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:02 smithi167 ceph-mon[36917]: pgmap v651: 129 pgs: 129 active+clean; 15 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 776 KiB/s wr, 799 op/s 2024-06-10T01:04:02.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:02 smithi094 ceph-mon[36708]: pgmap v651: 129 pgs: 129 active+clean; 15 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 776 KiB/s wr, 799 op/s 2024-06-10T01:04:02.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:02 smithi037 ceph-mon[32192]: pgmap v651: 129 pgs: 129 active+clean; 15 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 776 KiB/s wr, 799 op/s 2024-06-10T01:04:03.657 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:04:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:03.856+0000 7f4fd4ff9640 1 -- 172.21.15.37:0/520129529 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56471b050c20 con 0x56471b164f80 2024-06-10T01:04:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:03.856+0000 7f4fd4ff9640 10 client.26185 handle_command_reply: tid=0 2024-06-10T01:04:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:03.856+0000 7f4ff6571740 1 client.26185 shutdown 2024-06-10T01:04:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:03.856+0000 7f4ff6571740 1 -- 172.21.15.37:0/520129529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56471b0e0010 msgr2=0x56471b15bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:03.856+0000 7f4ff6571740 1 --2- 172.21.15.37:0/520129529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56471b0e0010 0x56471b15bc20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80d6040 tx=0x7f4fe80c0360 comp rx=0 tx=0).stop 2024-06-10T01:04:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:03.856+0000 7f4ff6571740 1 -- 172.21.15.37:0/520129529 shutdown_connections 2024-06-10T01:04:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:03.856+0000 7f4ff6571740 1 --2- 172.21.15.37:0/520129529 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56471b164f80 0x56471b167370 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:03.856+0000 7f4ff6571740 1 --2- 172.21.15.37:0/520129529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56471b0e5b70 0x56471b1631a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:03.856+0000 7f4ff6571740 1 --2- 172.21.15.37:0/520129529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471b0e0a00 0x56471b15c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:03.856+0000 7f4fd4ff9640 0 client.26185 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:03.856+0000 7f4ff6571740 1 --2- 172.21.15.37:0/520129529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56471b0e0010 0x56471b15bc20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:03.856+0000 7f4ff6571740 1 -- 172.21.15.37:0/520129529 >> 172.21.15.37:0/520129529 conn(0x56471b0db680 msgr2=0x56471b0e9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:03.856+0000 7f4ff6571740 1 -- 172.21.15.37:0/520129529 shutdown_connections 2024-06-10T01:04:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:03.856+0000 7f4ff6571740 1 -- 172.21.15.37:0/520129529 wait complete. 2024-06-10T01:04:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:03.856+0000 7f4ff6571740 20 client.26185 trim_cache size 0 max 16384 2024-06-10T01:04:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:04:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:04:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:04:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:03.857+0000 7f4ff3864640 1 -- 172.21.15.37:0/2555402351 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4fd80a2c30 msgr2=0x7f4fd80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:03.857+0000 7f4ff3864640 1 --2- 172.21.15.37:0/2555402351 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4fd80a2c30 0x7f4fd80a50f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4021000 tx=0x7f4fe401e000 comp rx=0 tx=0).stop 2024-06-10T01:04:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:03.857+0000 7f4ff3864640 1 -- 172.21.15.37:0/2555402351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec1024a0 msgr2=0x7f4fec197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:03.857+0000 7f4ff3864640 1 --2- 172.21.15.37:0/2555402351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec1024a0 0x7f4fec197c90 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009230 tx=0x7f4fe8031b60 comp rx=0 tx=0).stop 2024-06-10T01:04:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:03.857+0000 7f4ff3864640 1 -- 172.21.15.37:0/2555402351 shutdown_connections 2024-06-10T01:04:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:03.857+0000 7f4ff3864640 1 --2- 172.21.15.37:0/2555402351 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f4fd80a2c30 0x7f4fd80a50f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:03.857+0000 7f4ff3864640 1 --2- 172.21.15.37:0/2555402351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fec107700 0x7f4fec19f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:03.857+0000 7f4ff3864640 1 --2- 172.21.15.37:0/2555402351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fec104dd0 0x7f4fec1981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:03.857+0000 7f4ff3864640 1 --2- 172.21.15.37:0/2555402351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec1024a0 0x7f4fec197c90 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:03.857+0000 7f4ff3864640 1 -- 172.21.15.37:0/2555402351 >> 172.21.15.37:0/2555402351 conn(0x7f4fec0fbeb0 msgr2=0x7f4fec102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:03.857+0000 7f4ff3864640 1 -- 172.21.15.37:0/2555402351 shutdown_connections 2024-06-10T01:04:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:03.857+0000 7f4ff3864640 1 -- 172.21.15.37:0/2555402351 wait complete. 2024-06-10T01:04:03.921 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:04.296 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:04 smithi037 ceph-mon[32192]: pgmap v652: 129 pgs: 129 active+clean; 15 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 956 KiB/s wr, 886 op/s 2024-06-10T01:04:04.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:04 smithi167 ceph-mon[36917]: pgmap v652: 129 pgs: 129 active+clean; 15 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 956 KiB/s wr, 886 op/s 2024-06-10T01:04:04.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:04 smithi094 ceph-mon[36708]: pgmap v652: 129 pgs: 129 active+clean; 15 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 956 KiB/s wr, 886 op/s 2024-06-10T01:04:05.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:05 smithi037 ceph-mon[32192]: pgmap v653: 129 pgs: 129 active+clean; 17 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.3 MiB/s wr, 1.34k op/s 2024-06-10T01:04:05.835 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:05 smithi167 ceph-mon[36917]: pgmap v653: 129 pgs: 129 active+clean; 17 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.3 MiB/s wr, 1.34k op/s 2024-06-10T01:04:05.861 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:05 smithi094 ceph-mon[36708]: pgmap v653: 129 pgs: 129 active+clean; 17 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.3 MiB/s wr, 1.34k op/s 2024-06-10T01:04:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.920+0000 7fc116cfa700 1 -- 172.21.15.37:0/2250687032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc11006b3c0 msgr2=0x7fc11010f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.920+0000 7fc116cfa700 1 --2- 172.21.15.37:0/2250687032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc11006b3c0 0x7fc11010f310 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc10800b0a0 tx=0x7fc10800b3b0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.920+0000 7fc116cfa700 1 -- 172.21.15.37:0/2250687032 shutdown_connections 2024-06-10T01:04:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.920+0000 7fc116cfa700 1 --2- 172.21.15.37:0/2250687032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc11006b3c0 0x7fc11010f310 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.920+0000 7fc116cfa700 1 --2- 172.21.15.37:0/2250687032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1101084f0 0x7fc1101088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.920+0000 7fc116cfa700 1 --2- 172.21.15.37:0/2250687032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc11006aa00 0x7fc11006ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.920+0000 7fc116cfa700 1 -- 172.21.15.37:0/2250687032 >> 172.21.15.37:0/2250687032 conn(0x7fc110078300 msgr2=0x7fc110078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:05.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.920+0000 7fc116cfa700 1 -- 172.21.15.37:0/2250687032 shutdown_connections 2024-06-10T01:04:05.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.920+0000 7fc116cfa700 1 -- 172.21.15.37:0/2250687032 wait complete. 2024-06-10T01:04:05.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.920+0000 7fc116cfa700 1 Processor -- start 2024-06-10T01:04:05.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.921+0000 7fc116cfa700 1 -- start start 2024-06-10T01:04:05.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.921+0000 7fc116cfa700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc11006aa00 0x7fc11019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:05.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.921+0000 7fc116cfa700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc11006b3c0 0x7fc11019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:05.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.921+0000 7fc116cfa700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1101084f0 0x7fc110198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:05.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.921+0000 7fc116cfa700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc110112770 con 0x7fc1101084f0 2024-06-10T01:04:05.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.921+0000 7fc116cfa700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc1101125f0 con 0x7fc11006b3c0 2024-06-10T01:04:05.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.921+0000 7fc116cfa700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc1101128f0 con 0x7fc11006aa00 2024-06-10T01:04:05.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.921+0000 7fc115297700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1101084f0 0x7fc110198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:05.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.921+0000 7fc115297700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1101084f0 0x7fc110198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51408/0 (socket says 172.21.15.37:51408) 2024-06-10T01:04:05.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.921+0000 7fc115297700 1 -- 172.21.15.37:0/1471683013 learned_addr learned my addr 172.21.15.37:0/1471683013 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:05.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.921+0000 7fc114a96700 1 --2- 172.21.15.37:0/1471683013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc11006aa00 0x7fc11019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:05.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.921+0000 7fc10ffff700 1 --2- 172.21.15.37:0/1471683013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc11006b3c0 0x7fc11019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:05.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.921+0000 7fc115297700 1 -- 172.21.15.37:0/1471683013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc11006aa00 msgr2=0x7fc11019e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:05.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.921+0000 7fc115297700 1 --2- 172.21.15.37:0/1471683013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc11006aa00 0x7fc11019e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.921+0000 7fc115297700 1 -- 172.21.15.37:0/1471683013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc11006b3c0 msgr2=0x7fc11019ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:05.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.921+0000 7fc115297700 1 --2- 172.21.15.37:0/1471683013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc11006b3c0 0x7fc11019ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.921+0000 7fc115297700 1 -- 172.21.15.37:0/1471683013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc110199170 con 0x7fc1101084f0 2024-06-10T01:04:05.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.921+0000 7fc115297700 1 --2- 172.21.15.37:0/1471683013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1101084f0 0x7fc110198a10 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc10800b780 tx=0x7fc108004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:05.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.922+0000 7fc10dffb700 1 -- 172.21.15.37:0/1471683013 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc108021070 con 0x7fc1101084f0 2024-06-10T01:04:05.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.922+0000 7fc10dffb700 1 -- 172.21.15.37:0/1471683013 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc108007b80 con 0x7fc1101084f0 2024-06-10T01:04:05.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.922+0000 7fc116cfa700 1 -- 172.21.15.37:0/1471683013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc110199400 con 0x7fc1101084f0 2024-06-10T01:04:05.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.922+0000 7fc10dffb700 1 -- 172.21.15.37:0/1471683013 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1080052a0 con 0x7fc1101084f0 2024-06-10T01:04:05.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.922+0000 7fc116cfa700 1 -- 172.21.15.37:0/1471683013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1101a5720 con 0x7fc1101084f0 2024-06-10T01:04:05.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.924+0000 7fc10dffb700 1 -- 172.21.15.37:0/1471683013 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc108024070 con 0x7fc1101084f0 2024-06-10T01:04:05.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.925+0000 7fc10dffb700 1 --2- 172.21.15.37:0/1471683013 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc1000a2c20 0x7fc1000a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:05.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.925+0000 7fc114a96700 1 --2- 172.21.15.37:0/1471683013 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc1000a2c20 0x7fc1000a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:05.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.925+0000 7fc10dffb700 1 -- 172.21.15.37:0/1471683013 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fc10800f020 con 0x7fc1101084f0 2024-06-10T01:04:05.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.925+0000 7fc114a96700 1 --2- 172.21.15.37:0/1471683013 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc1000a2c20 0x7fc1000a50e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc110199b50 tx=0x7fc104009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:05.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.926+0000 7fc12157db80 1 Processor -- start 2024-06-10T01:04:05.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.926+0000 7fc12157db80 1 -- start start 2024-06-10T01:04:05.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.926+0000 7fc12157db80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b72e1a1640 0x55b72e1a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:05.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.926+0000 7fc12157db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b72e1a1f60 0x55b72e19b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.926+0000 7fc12157db80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b72e19bf20 0x55b72e19c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.926+0000 7fc12157db80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b72e0ed810 con 0x55b72e19bf20 2024-06-10T01:04:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.926+0000 7fc12157db80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b72e095720 con 0x55b72e1a1640 2024-06-10T01:04:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.926+0000 7fc12157db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b72e0e97d0 con 0x55b72e1a1f60 2024-06-10T01:04:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.926+0000 7fc114a96700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b72e1a1640 0x55b72e1a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.926+0000 7fc115297700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b72e19bf20 0x55b72e19c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.926+0000 7fc115297700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b72e19bf20 0x55b72e19c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51420/0 (socket says 172.21.15.37:51420) 2024-06-10T01:04:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.926+0000 7fc10ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b72e1a1f60 0x55b72e19b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.926+0000 7fc115297700 1 -- 172.21.15.37:0/232619770 learned_addr learned my addr 172.21.15.37:0/232619770 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.926+0000 7fc114a96700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b72e1a1640 0x55b72e1a1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:51814/0 (socket says 172.21.15.37:51814) 2024-06-10T01:04:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.926+0000 7fc115297700 1 -- 172.21.15.37:0/232619770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b72e1a1f60 msgr2=0x55b72e19b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.926+0000 7fc115297700 1 --2- 172.21.15.37:0/232619770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b72e1a1f60 0x55b72e19b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.926+0000 7fc115297700 1 -- 172.21.15.37:0/232619770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b72e1a1640 msgr2=0x55b72e1a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.926+0000 7fc115297700 1 --2- 172.21.15.37:0/232619770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b72e1a1640 0x55b72e1a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.926+0000 7fc115297700 1 -- 172.21.15.37:0/232619770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b72e08bf50 con 0x55b72e19bf20 2024-06-10T01:04:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.926+0000 7fc114a96700 1 --2- 172.21.15.37:0/232619770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b72e1a1640 0x55b72e1a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.927+0000 7fc115297700 1 --2- 172.21.15.37:0/232619770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b72e19bf20 0x55b72e19c340 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fc10809ca50 tx=0x7fc1080099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f28de4e0b92e038 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.927+0000 7fc0faffd700 1 -- 172.21.15.37:0/232619770 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10809d450 con 0x55b72e19bf20 2024-06-10T01:04:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.927+0000 7fc0faffd700 1 -- 172.21.15.37:0/232619770 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1080074e0 con 0x55b72e19bf20 2024-06-10T01:04:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.927+0000 7fc0faffd700 1 -- 172.21.15.37:0/232619770 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1080a54a0 con 0x55b72e19bf20 2024-06-10T01:04:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.927+0000 7fc12157db80 1 -- 172.21.15.37:0/232619770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b72e19bf20 msgr2=0x55b72e19c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.927+0000 7fc12157db80 1 --2- 172.21.15.37:0/232619770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b72e19bf20 0x55b72e19c340 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fc10809ca50 tx=0x7fc1080099e0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.927+0000 7fc12157db80 1 -- 172.21.15.37:0/232619770 shutdown_connections 2024-06-10T01:04:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.927+0000 7fc12157db80 1 --2- 172.21.15.37:0/232619770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b72e19bf20 0x55b72e19c340 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.927+0000 7fc12157db80 1 --2- 172.21.15.37:0/232619770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b72e1a1640 0x55b72e1a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.928+0000 7fc12157db80 1 --2- 172.21.15.37:0/232619770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b72e1a1f60 0x55b72e19b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.928+0000 7fc12157db80 1 -- 172.21.15.37:0/232619770 >> 172.21.15.37:0/232619770 conn(0x55b72e0c6cc0 msgr2=0x55b72e196850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.928+0000 7fc12157db80 1 -- 172.21.15.37:0/232619770 shutdown_connections 2024-06-10T01:04:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.928+0000 7fc12157db80 1 -- 172.21.15.37:0/232619770 wait complete. 2024-06-10T01:04:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.928+0000 7fc12157db80 1 -- start start 2024-06-10T01:04:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.928+0000 7fc12157db80 1 Processor -- start 2024-06-10T01:04:05.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.928+0000 7fc12157db80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b72e19bf20 0x55b72e1a7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:05.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.928+0000 7fc12157db80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b72e1a1640 0x55b72e1a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:05.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.928+0000 7fc12157db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b72e1a1f60 0x55b72e1a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:05.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.928+0000 7fc12157db80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b72e1a86f0 con 0x55b72e19bf20 2024-06-10T01:04:05.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.928+0000 7fc12157db80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b72e08dbf0 con 0x55b72e1a1640 2024-06-10T01:04:05.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.928+0000 7fc12157db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b72e1a8870 con 0x55b72e1a1f60 2024-06-10T01:04:05.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.929+0000 7fc12157db80 10 client.0 _ll_register_callbacks cb 0x55b72e087920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:05.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.929+0000 7fc114a96700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b72e19bf20 0x55b72e1a7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:05.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.929+0000 7fc10ffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b72e1a1640 0x55b72e1a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:05.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.929+0000 7fc114a96700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b72e19bf20 0x55b72e1a7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51424/0 (socket says 172.21.15.37:51424) 2024-06-10T01:04:05.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.929+0000 7fc114a96700 1 -- 172.21.15.37:0/1657026045 learned_addr learned my addr 172.21.15.37:0/1657026045 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:05.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.929+0000 7fc115297700 1 --2- 172.21.15.37:0/1657026045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b72e1a1f60 0x55b72e1a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:05.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.929+0000 7fc114a96700 1 -- 172.21.15.37:0/1657026045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b72e1a1f60 msgr2=0x55b72e1a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:05.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.929+0000 7fc114a96700 1 --2- 172.21.15.37:0/1657026045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b72e1a1f60 0x55b72e1a54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.929+0000 7fc114a96700 1 -- 172.21.15.37:0/1657026045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b72e1a1640 msgr2=0x55b72e1a4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:05.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.929+0000 7fc114a96700 1 --2- 172.21.15.37:0/1657026045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b72e1a1640 0x55b72e1a4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.929+0000 7fc114a96700 1 -- 172.21.15.37:0/1657026045 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b72e03c8d0 con 0x55b72e19bf20 2024-06-10T01:04:05.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.929+0000 7fc10ffff700 1 --2- 172.21.15.37:0/1657026045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b72e1a1640 0x55b72e1a4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:05.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.929+0000 7fc114a96700 1 --2- 172.21.15.37:0/1657026045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b72e19bf20 0x55b72e1a7c90 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc104011fd0 tx=0x7fc1040047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a56b0bf9981c495 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:05.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.929+0000 7fc0effff700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:04:05.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.929+0000 7fc0effff700 1 -- 172.21.15.37:0/1657026045 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10400e7c0 con 0x55b72e19bf20 2024-06-10T01:04:05.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.929+0000 7fc0effff700 1 -- 172.21.15.37:0/1657026045 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fc10400e960 con 0x55b72e19bf20 2024-06-10T01:04:05.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.929+0000 7fc12157db80 1 -- 172.21.15.37:0/1657026045 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1080074e0 con 0x55b72e19bf20 2024-06-10T01:04:05.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.929+0000 7fc0effff700 1 -- 172.21.15.37:0/1657026045 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc10400eb60 con 0x55b72e19bf20 2024-06-10T01:04:05.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.929+0000 7fc12157db80 10 client.16656 fetch_fsmap learned FSMap version 29 2024-06-10T01:04:05.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.930+0000 7fc12157db80 1 -- 172.21.15.37:0/1657026045 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b72e1b30d0 con 0x55b72e19bf20 2024-06-10T01:04:05.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.930+0000 7fc0effff700 1 -- 172.21.15.37:0/1657026045 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1080074e0 con 0x55b72e19bf20 2024-06-10T01:04:05.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.930+0000 7fc0effff700 1 -- 172.21.15.37:0/1657026045 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fc104026070 con 0x55b72e19bf20 2024-06-10T01:04:05.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.930+0000 7fc12157db80 10 client.16656 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:04:05.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.930+0000 7fc12157db80 10 client.16656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:04:05.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.930+0000 7fc12157db80 20 client.16656 populate_metadata read hostname 'smithi037' 2024-06-10T01:04:05.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.930+0000 7fc12157db80 1 --2- 172.21.15.37:0/1657026045 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b72e1b57b0 0x55b72e1b7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:05.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.930+0000 7fc12157db80 4 client.16656 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:04:05.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.930+0000 7fc12157db80 1 -- 172.21.15.37:0/1657026045 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b72e0fad10 con 0x55b72e1b57b0 2024-06-10T01:04:05.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.930+0000 7fc10ffff700 1 --2- 172.21.15.37:0/1657026045 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b72e1b57b0 0x55b72e1b7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:05.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.931+0000 7fc10ffff700 1 --2- 172.21.15.37:0/1657026045 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b72e1b57b0 0x55b72e1b7c10 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ebe2415ef6e7404 server_cookie=db722d06dc0f0a9c in_seq=0 out_seq=0 2024-06-10T01:04:05.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.931+0000 7fc0effff700 10 client.16656 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:05.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.936+0000 7fc0effff700 1 -- 172.21.15.37:0/1657026045 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b72e0fad10 con 0x55b72e1b57b0 2024-06-10T01:04:05.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.936+0000 7fc0effff700 10 client.16656 handle_command_reply: tid=0 2024-06-10T01:04:05.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.936+0000 7fc12157db80 1 client.16656 shutdown 2024-06-10T01:04:05.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.936+0000 7fc12157db80 1 -- 172.21.15.37:0/1657026045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b72e19bf20 msgr2=0x55b72e1a7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:05.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.936+0000 7fc12157db80 1 --2- 172.21.15.37:0/1657026045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b72e19bf20 0x55b72e1a7c90 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc104011fd0 tx=0x7fc1040047c0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.936+0000 7fc12157db80 1 -- 172.21.15.37:0/1657026045 shutdown_connections 2024-06-10T01:04:05.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.936+0000 7fc12157db80 1 --2- 172.21.15.37:0/1657026045 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b72e1b57b0 0x55b72e1b7c10 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.936+0000 7fc12157db80 1 --2- 172.21.15.37:0/1657026045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b72e1a1f60 0x55b72e1a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.936+0000 7fc0effff700 0 client.16656 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:05.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.936+0000 7fc12157db80 1 --2- 172.21.15.37:0/1657026045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b72e19bf20 0x55b72e1a7c90 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.936+0000 7fc12157db80 1 --2- 172.21.15.37:0/1657026045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b72e1a1640 0x55b72e1a4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.936+0000 7fc12157db80 1 -- 172.21.15.37:0/1657026045 >> 172.21.15.37:0/1657026045 conn(0x55b72e0c6cc0 msgr2=0x55b72e19a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:05.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.936+0000 7fc12157db80 1 -- 172.21.15.37:0/1657026045 shutdown_connections 2024-06-10T01:04:05.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.936+0000 7fc12157db80 1 -- 172.21.15.37:0/1657026045 wait complete. 2024-06-10T01:04:05.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.936+0000 7fc12157db80 20 client.16656 trim_cache size 0 max 16384 2024-06-10T01:04:05.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.957+0000 7fc12157db80 1 Processor -- start 2024-06-10T01:04:05.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.957+0000 7fc12157db80 1 -- start start 2024-06-10T01:04:05.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.957+0000 7fc12157db80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b72e19bf20 0x55b72e19b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:05.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.957+0000 7fc12157db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b72e1a1640 0x55b72e190690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:05.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.957+0000 7fc12157db80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b72e1a1f60 0x55b72e1a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:05.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.957+0000 7fc12157db80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc1101b0cc0 con 0x55b72e1a1f60 2024-06-10T01:04:05.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.958+0000 7fc12157db80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc1101ae550 con 0x55b72e19bf20 2024-06-10T01:04:05.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.958+0000 7fc12157db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc1101ac3a0 con 0x55b72e1a1640 2024-06-10T01:04:05.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.958+0000 7fc114a96700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b72e19bf20 0x55b72e19b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:05.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.958+0000 7fc114a96700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b72e19bf20 0x55b72e19b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:51834/0 (socket says 172.21.15.37:51834) 2024-06-10T01:04:05.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.958+0000 7fc114a96700 1 -- 172.21.15.37:0/3854555854 learned_addr learned my addr 172.21.15.37:0/3854555854 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:05.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.958+0000 7fc10ffff700 1 --2- 172.21.15.37:0/3854555854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b72e1a1640 0x55b72e190690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:05.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.958+0000 7fc115297700 1 --2- 172.21.15.37:0/3854555854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b72e1a1f60 0x55b72e1a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:05.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.958+0000 7fc114a96700 1 -- 172.21.15.37:0/3854555854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b72e1a1640 msgr2=0x55b72e190690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:05.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.958+0000 7fc114a96700 1 --2- 172.21.15.37:0/3854555854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b72e1a1640 0x55b72e190690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.958+0000 7fc114a96700 1 -- 172.21.15.37:0/3854555854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b72e1a1f60 msgr2=0x55b72e1a3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:05.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.958+0000 7fc114a96700 1 --2- 172.21.15.37:0/3854555854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b72e1a1f60 0x55b72e1a3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.958+0000 7fc114a96700 1 -- 172.21.15.37:0/3854555854 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b72e19c690 con 0x55b72e19bf20 2024-06-10T01:04:05.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.958+0000 7fc115297700 1 --2- 172.21.15.37:0/3854555854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b72e1a1f60 0x55b72e1a3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:05.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.958+0000 7fc10ffff700 1 --2- 172.21.15.37:0/3854555854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b72e1a1640 0x55b72e190690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:05.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.958+0000 7fc114a96700 1 --2- 172.21.15.37:0/3854555854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b72e19bf20 0x55b72e19b9e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc1040117d0 tx=0x7fc104011880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22063e7b07c7a6c1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:05.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.959+0000 7fc0fa7fc700 1 -- 172.21.15.37:0/3854555854 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10403c020 con 0x55b72e19bf20 2024-06-10T01:04:05.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.959+0000 7fc0fa7fc700 1 -- 172.21.15.37:0/3854555854 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc104004030 con 0x55b72e19bf20 2024-06-10T01:04:05.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.959+0000 7fc0fa7fc700 1 -- 172.21.15.37:0/3854555854 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc104027ca0 con 0x55b72e19bf20 2024-06-10T01:04:05.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.959+0000 7fc12157db80 1 -- 172.21.15.37:0/3854555854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b72e19bf20 msgr2=0x55b72e19b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:05.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.959+0000 7fc12157db80 1 --2- 172.21.15.37:0/3854555854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b72e19bf20 0x55b72e19b9e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc1040117d0 tx=0x7fc104011880 comp rx=0 tx=0).stop 2024-06-10T01:04:05.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.959+0000 7fc12157db80 1 -- 172.21.15.37:0/3854555854 shutdown_connections 2024-06-10T01:04:05.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.959+0000 7fc12157db80 1 --2- 172.21.15.37:0/3854555854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b72e1a1f60 0x55b72e1a3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.959+0000 7fc12157db80 1 --2- 172.21.15.37:0/3854555854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b72e19bf20 0x55b72e19b9e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.959+0000 7fc12157db80 1 --2- 172.21.15.37:0/3854555854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b72e1a1640 0x55b72e190690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.959+0000 7fc12157db80 1 -- 172.21.15.37:0/3854555854 >> 172.21.15.37:0/3854555854 conn(0x55b72e0c6cc0 msgr2=0x55b72e1a3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:05.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.959+0000 7fc12157db80 1 -- 172.21.15.37:0/3854555854 shutdown_connections 2024-06-10T01:04:05.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.959+0000 7fc12157db80 1 -- 172.21.15.37:0/3854555854 wait complete. 2024-06-10T01:04:05.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.960+0000 7fc12157db80 1 -- start start 2024-06-10T01:04:05.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.960+0000 7fc12157db80 1 Processor -- start 2024-06-10T01:04:05.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.960+0000 7fc12157db80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b72e19bf20 0x55b72e1addf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:05.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.960+0000 7fc12157db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b72e1a1640 0x55b72e1ae330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:05.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.960+0000 7fc12157db80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b72e1a1f60 0x55b72e1bcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:05.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.960+0000 7fc12157db80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc1101aca10 con 0x55b72e1a1f60 2024-06-10T01:04:05.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.960+0000 7fc12157db80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc1101ade80 con 0x55b72e19bf20 2024-06-10T01:04:05.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.960+0000 7fc12157db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc1101ad0e0 con 0x55b72e1a1640 2024-06-10T01:04:05.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.961+0000 7fc114a96700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b72e19bf20 0x55b72e1addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:05.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.961+0000 7fc10ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b72e1a1640 0x55b72e1ae330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:05.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.961+0000 7fc114a96700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b72e19bf20 0x55b72e1addf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:51836/0 (socket says 172.21.15.37:51836) 2024-06-10T01:04:05.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.961+0000 7fc114a96700 1 -- 172.21.15.37:0/3631790417 learned_addr learned my addr 172.21.15.37:0/3631790417 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:05.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.961+0000 7fc12157db80 10 client.0 _ll_register_callbacks cb 0x55b72e087920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:05.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.961+0000 7fc115297700 1 --2- 172.21.15.37:0/3631790417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b72e1a1f60 0x55b72e1bcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:05.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.961+0000 7fc114a96700 1 -- 172.21.15.37:0/3631790417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b72e1a1640 msgr2=0x55b72e1ae330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:05.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.961+0000 7fc114a96700 1 --2- 172.21.15.37:0/3631790417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b72e1a1640 0x55b72e1ae330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.961+0000 7fc114a96700 1 -- 172.21.15.37:0/3631790417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b72e1a1f60 msgr2=0x55b72e1bcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:05.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.961+0000 7fc114a96700 1 --2- 172.21.15.37:0/3631790417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b72e1a1f60 0x55b72e1bcb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.961+0000 7fc114a96700 1 -- 172.21.15.37:0/3631790417 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b72e1bd400 con 0x55b72e19bf20 2024-06-10T01:04:05.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.961+0000 7fc115297700 1 --2- 172.21.15.37:0/3631790417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b72e1a1f60 0x55b72e1bcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:04:05.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.961+0000 7fc114a96700 1 --2- 172.21.15.37:0/3631790417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b72e19bf20 0x55b72e1addf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc104009aa0 tx=0x7fc10400eac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e696ed43e322e5e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:05.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.961+0000 7fc0effff700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:04:05.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.961+0000 7fc0effff700 1 -- 172.21.15.37:0/3631790417 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10403c070 con 0x55b72e19bf20 2024-06-10T01:04:05.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.961+0000 7fc0effff700 1 -- 172.21.15.37:0/3631790417 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fc104027920 con 0x55b72e19bf20 2024-06-10T01:04:05.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.961+0000 7fc12157db80 1 -- 172.21.15.37:0/3631790417 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc104004030 con 0x55b72e19bf20 2024-06-10T01:04:05.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.962+0000 7fc0effff700 1 -- 172.21.15.37:0/3631790417 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc104005660 con 0x55b72e19bf20 2024-06-10T01:04:05.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.962+0000 7fc12157db80 10 client.26108 fetch_fsmap learned FSMap version 29 2024-06-10T01:04:05.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.962+0000 7fc12157db80 1 -- 172.21.15.37:0/3631790417 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b72e220020 con 0x55b72e19bf20 2024-06-10T01:04:05.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.962+0000 7fc0effff700 1 -- 172.21.15.37:0/3631790417 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc104004030 con 0x55b72e19bf20 2024-06-10T01:04:05.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.962+0000 7fc0effff700 1 -- 172.21.15.37:0/3631790417 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fc104026070 con 0x55b72e19bf20 2024-06-10T01:04:05.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.962+0000 7fc12157db80 10 client.26108 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:04:05.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.962+0000 7fc12157db80 10 client.26108 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:04:05.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.962+0000 7fc12157db80 20 client.26108 populate_metadata read hostname 'smithi037' 2024-06-10T01:04:05.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.962+0000 7fc12157db80 1 --2- 172.21.15.37:0/3631790417 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b72e1b57b0 0x55b72e2224e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:05.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.962+0000 7fc12157db80 4 client.26108 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:04:05.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.962+0000 7fc12157db80 1 -- 172.21.15.37:0/3631790417 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b72e0f5290 con 0x55b72e1b57b0 2024-06-10T01:04:05.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.962+0000 7fc10ffff700 1 --2- 172.21.15.37:0/3631790417 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b72e1b57b0 0x55b72e2224e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:05.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.963+0000 7fc10ffff700 1 --2- 172.21.15.37:0/3631790417 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b72e1b57b0 0x55b72e2224e0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bad79f6258e646d1 server_cookie=526458562a36e5f8 in_seq=0 out_seq=0 2024-06-10T01:04:05.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.963+0000 7fc0effff700 10 client.26108 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:05.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.968+0000 7fc0effff700 1 -- 172.21.15.37:0/3631790417 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b72e0f5290 con 0x55b72e1b57b0 2024-06-10T01:04:05.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.968+0000 7fc0effff700 10 client.26108 handle_command_reply: tid=0 2024-06-10T01:04:05.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.968+0000 7fc12157db80 1 client.26108 shutdown 2024-06-10T01:04:05.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.968+0000 7fc12157db80 1 -- 172.21.15.37:0/3631790417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b72e19bf20 msgr2=0x55b72e1addf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:05.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.968+0000 7fc12157db80 1 --2- 172.21.15.37:0/3631790417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b72e19bf20 0x55b72e1addf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc104009aa0 tx=0x7fc10400eac0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.968+0000 7fc12157db80 1 -- 172.21.15.37:0/3631790417 shutdown_connections 2024-06-10T01:04:05.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.968+0000 7fc12157db80 1 --2- 172.21.15.37:0/3631790417 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b72e1b57b0 0x55b72e2224e0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.968+0000 7fc12157db80 1 --2- 172.21.15.37:0/3631790417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b72e1a1f60 0x55b72e1bcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.968+0000 7fc12157db80 1 --2- 172.21.15.37:0/3631790417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b72e19bf20 0x55b72e1addf0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.968+0000 7fc0effff700 0 client.26108 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:05.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.968+0000 7fc12157db80 1 --2- 172.21.15.37:0/3631790417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b72e1a1640 0x55b72e1ae330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.968+0000 7fc12157db80 1 -- 172.21.15.37:0/3631790417 >> 172.21.15.37:0/3631790417 conn(0x55b72e0c6cc0 msgr2=0x55b72e1a0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:05.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.968+0000 7fc12157db80 1 -- 172.21.15.37:0/3631790417 shutdown_connections 2024-06-10T01:04:05.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.968+0000 7fc12157db80 1 -- 172.21.15.37:0/3631790417 wait complete. 2024-06-10T01:04:05.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.968+0000 7fc12157db80 20 client.26108 trim_cache size 0 max 16384 2024-06-10T01:04:05.968 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:04:05.968 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-06-10T01:04:05.968 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-06-10T01:04:05.968 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:04:05.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.969+0000 7fc116cfa700 1 -- 172.21.15.37:0/1471683013 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc1000a2c20 msgr2=0x7fc1000a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:05.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.969+0000 7fc116cfa700 1 --2- 172.21.15.37:0/1471683013 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc1000a2c20 0x7fc1000a50e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc110199b50 tx=0x7fc104009260 comp rx=0 tx=0).stop 2024-06-10T01:04:05.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.969+0000 7fc116cfa700 1 -- 172.21.15.37:0/1471683013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1101084f0 msgr2=0x7fc110198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:05.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.969+0000 7fc116cfa700 1 --2- 172.21.15.37:0/1471683013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1101084f0 0x7fc110198a10 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc10800b780 tx=0x7fc108004d00 comp rx=0 tx=0).stop 2024-06-10T01:04:05.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.969+0000 7fc116cfa700 1 -- 172.21.15.37:0/1471683013 shutdown_connections 2024-06-10T01:04:05.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.969+0000 7fc116cfa700 1 --2- 172.21.15.37:0/1471683013 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc1000a2c20 0x7fc1000a50e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.969+0000 7fc116cfa700 1 --2- 172.21.15.37:0/1471683013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1101084f0 0x7fc110198a10 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.969+0000 7fc116cfa700 1 --2- 172.21.15.37:0/1471683013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc11006aa00 0x7fc11019e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.969+0000 7fc116cfa700 1 --2- 172.21.15.37:0/1471683013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc11006b3c0 0x7fc11019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:05.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.969+0000 7fc116cfa700 1 -- 172.21.15.37:0/1471683013 >> 172.21.15.37:0/1471683013 conn(0x7fc110078300 msgr2=0x7fc1100fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:05.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.969+0000 7fc116cfa700 1 -- 172.21.15.37:0/1471683013 shutdown_connections 2024-06-10T01:04:05.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:05.969+0000 7fc116cfa700 1 -- 172.21.15.37:0/1471683013 wait complete. 2024-06-10T01:04:06.659 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:04:06.660 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:04:06.921 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:08.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:08 smithi037 ceph-mon[32192]: pgmap v654: 129 pgs: 129 active+clean; 8.6 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 1.3 MiB/s wr, 1.25k op/s 2024-06-10T01:04:08.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:08 smithi167 ceph-mon[36917]: pgmap v654: 129 pgs: 129 active+clean; 8.6 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 1.3 MiB/s wr, 1.25k op/s 2024-06-10T01:04:08.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:08 smithi094 ceph-mon[36708]: pgmap v654: 129 pgs: 129 active+clean; 8.6 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 1.3 MiB/s wr, 1.25k op/s 2024-06-10T01:04:08.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.579+0000 7f9c2f521700 1 -- 172.21.15.37:0/166087375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c28101d50 msgr2=0x7f9c2810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:08.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.579+0000 7f9c2f521700 1 --2- 172.21.15.37:0/166087375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c28101d50 0x7f9c2810e240 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f9c2000b0a0 tx=0x7f9c2000b3b0 comp rx=0 tx=0).stop 2024-06-10T01:04:08.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.579+0000 7f9c2f521700 1 -- 172.21.15.37:0/166087375 shutdown_connections 2024-06-10T01:04:08.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.579+0000 7f9c2f521700 1 --2- 172.21.15.37:0/166087375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c28101d50 0x7f9c2810e240 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:08.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.579+0000 7f9c2f521700 1 --2- 172.21.15.37:0/166087375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c28107400 0x7f9c281077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:08.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.579+0000 7f9c2f521700 1 --2- 172.21.15.37:0/166087375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c28101390 0x7f9c28101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:08.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.579+0000 7f9c2f521700 1 -- 172.21.15.37:0/166087375 >> 172.21.15.37:0/166087375 conn(0x7f9c28078340 msgr2=0x7f9c28078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:08.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.580+0000 7f9c2f521700 1 -- 172.21.15.37:0/166087375 shutdown_connections 2024-06-10T01:04:08.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.580+0000 7f9c2f521700 1 -- 172.21.15.37:0/166087375 wait complete. 2024-06-10T01:04:08.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.580+0000 7f9c2f521700 1 Processor -- start 2024-06-10T01:04:08.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.580+0000 7f9c2f521700 1 -- start start 2024-06-10T01:04:08.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.581+0000 7f9c2f521700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c28101390 0x7f9c28197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:08.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.581+0000 7f9c2f521700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c28101d50 0x7f9c28197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:08.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.581+0000 7f9c2f521700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c28107400 0x7f9c281982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:08.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.581+0000 7f9c2f521700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9c28111690 con 0x7f9c28101d50 2024-06-10T01:04:08.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.581+0000 7f9c2f521700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9c28111510 con 0x7f9c28101390 2024-06-10T01:04:08.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.581+0000 7f9c2f521700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9c28111810 con 0x7f9c28107400 2024-06-10T01:04:08.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.581+0000 7f9c2cabc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c28101d50 0x7f9c28197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:08.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.581+0000 7f9c2cabc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c28101d50 0x7f9c28197d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51464/0 (socket says 172.21.15.37:51464) 2024-06-10T01:04:08.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.581+0000 7f9c2cabc700 1 -- 172.21.15.37:0/2122125450 learned_addr learned my addr 172.21.15.37:0/2122125450 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:08.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.581+0000 7f9c2d2bd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c28101390 0x7f9c28197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:08.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.581+0000 7f9c2dabe700 1 --2- 172.21.15.37:0/2122125450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c28107400 0x7f9c281982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:08.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.581+0000 7f9c2cabc700 1 -- 172.21.15.37:0/2122125450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c28107400 msgr2=0x7f9c281982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:08.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.581+0000 7f9c2cabc700 1 --2- 172.21.15.37:0/2122125450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c28107400 0x7f9c281982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:08.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.581+0000 7f9c2cabc700 1 -- 172.21.15.37:0/2122125450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c28101390 msgr2=0x7f9c28197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:08.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.581+0000 7f9c2cabc700 1 --2- 172.21.15.37:0/2122125450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c28101390 0x7f9c28197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:08.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.581+0000 7f9c2cabc700 1 -- 172.21.15.37:0/2122125450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c281a7050 con 0x7f9c28101d50 2024-06-10T01:04:08.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.581+0000 7f9c2d2bd700 1 --2- 172.21.15.37:0/2122125450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c28101390 0x7f9c28197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:08.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.581+0000 7f9c2cabc700 1 --2- 172.21.15.37:0/2122125450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c28101d50 0x7f9c28197d80 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800e9a0 tx=0x7f9c1800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:08.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.581+0000 7f9c1e7fc700 1 -- 172.21.15.37:0/2122125450 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1800ca10 con 0x7f9c28101d50 2024-06-10T01:04:08.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.581+0000 7f9c2f521700 1 -- 172.21.15.37:0/2122125450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c281a7340 con 0x7f9c28101d50 2024-06-10T01:04:08.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.581+0000 7f9c1e7fc700 1 -- 172.21.15.37:0/2122125450 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c1800cbb0 con 0x7f9c28101d50 2024-06-10T01:04:08.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.581+0000 7f9c1e7fc700 1 -- 172.21.15.37:0/2122125450 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c18018710 con 0x7f9c28101d50 2024-06-10T01:04:08.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.582+0000 7f9c2f521700 1 -- 172.21.15.37:0/2122125450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c281a7880 con 0x7f9c28101d50 2024-06-10T01:04:08.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.582+0000 7f9c2f521700 1 -- 172.21.15.37:0/2122125450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c28069180 con 0x7f9c28101d50 2024-06-10T01:04:08.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.584+0000 7f9c1e7fc700 1 -- 172.21.15.37:0/2122125450 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9c180188b0 con 0x7f9c28101d50 2024-06-10T01:04:08.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.584+0000 7f9c1e7fc700 1 --2- 172.21.15.37:0/2122125450 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9c140a2ca0 0x7f9c140a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:08.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.585+0000 7f9c2d2bd700 1 --2- 172.21.15.37:0/2122125450 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9c140a2ca0 0x7f9c140a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:08.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.585+0000 7f9c1e7fc700 1 -- 172.21.15.37:0/2122125450 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f9c180cf7f0 con 0x7f9c28101d50 2024-06-10T01:04:08.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.585+0000 7f9c2d2bd700 1 --2- 172.21.15.37:0/2122125450 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9c140a2ca0 0x7f9c140a5160 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009fd0 tx=0x7f9c24009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:08.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.588+0000 7f9c1e7fc700 1 -- 172.21.15.37:0/2122125450 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9c1809c300 con 0x7f9c28101d50 2024-06-10T01:04:08.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.838+0000 7f9c2f521700 1 -- 172.21.15.37:0/2122125450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c2806a710 con 0x7f9c28101d50 2024-06-10T01:04:08.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.839+0000 7f9c1e7fc700 1 -- 172.21.15.37:0/2122125450 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11864 (secure 0 0 0) 0x7f9c1809bc00 con 0x7f9c28101d50 2024-06-10T01:04:08.836 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:04:08.837 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T01:03:24.246183+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26110":{"gid":26110,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/317977487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":317977487},{"type":"v1","addr":"172.21.15.167:6835","nonce":317977487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24458,24472,24451]},"id":1}]} 2024-06-10T01:04:08.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.841+0000 7f9c2f521700 1 -- 172.21.15.37:0/2122125450 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9c140a2ca0 msgr2=0x7f9c140a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:08.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.841+0000 7f9c2f521700 1 --2- 172.21.15.37:0/2122125450 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9c140a2ca0 0x7f9c140a5160 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009fd0 tx=0x7f9c24009260 comp rx=0 tx=0).stop 2024-06-10T01:04:08.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.841+0000 7f9c2f521700 1 -- 172.21.15.37:0/2122125450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c28101d50 msgr2=0x7f9c28197d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:08.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.841+0000 7f9c2f521700 1 --2- 172.21.15.37:0/2122125450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c28101d50 0x7f9c28197d80 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800e9a0 tx=0x7f9c1800ecb0 comp rx=0 tx=0).stop 2024-06-10T01:04:08.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.842+0000 7f9c2f521700 1 -- 172.21.15.37:0/2122125450 shutdown_connections 2024-06-10T01:04:08.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.842+0000 7f9c2f521700 1 --2- 172.21.15.37:0/2122125450 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9c140a2ca0 0x7f9c140a5160 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:08.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.842+0000 7f9c2f521700 1 --2- 172.21.15.37:0/2122125450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c28107400 0x7f9c281982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:08.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.842+0000 7f9c2f521700 1 --2- 172.21.15.37:0/2122125450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c28101390 0x7f9c28197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:08.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.842+0000 7f9c2f521700 1 --2- 172.21.15.37:0/2122125450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c28101d50 0x7f9c28197d80 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:08.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.842+0000 7f9c2f521700 1 -- 172.21.15.37:0/2122125450 >> 172.21.15.37:0/2122125450 conn(0x7f9c28078340 msgr2=0x7f9c280ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:08.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.842+0000 7f9c2f521700 1 -- 172.21.15.37:0/2122125450 shutdown_connections 2024-06-10T01:04:08.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:08.842+0000 7f9c2f521700 1 -- 172.21.15.37:0/2122125450 wait complete. 2024-06-10T01:04:08.840 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 29 2024-06-10T01:04:09.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:09 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2122125450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:04:09.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:09 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2122125450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:04:09.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:09 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2122125450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:04:09.462 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:04:09.714 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:10.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:10 smithi037 ceph-mon[32192]: pgmap v655: 129 pgs: 129 active+clean; 7.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 1.5 MiB/s wr, 1.40k op/s 2024-06-10T01:04:10.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:10 smithi167 ceph-mon[36917]: pgmap v655: 129 pgs: 129 active+clean; 7.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 1.5 MiB/s wr, 1.40k op/s 2024-06-10T01:04:10.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:10 smithi094 ceph-mon[36708]: pgmap v655: 129 pgs: 129 active+clean; 7.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 1.5 MiB/s wr, 1.40k op/s 2024-06-10T01:04:11.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.373+0000 7f7c1028e700 1 -- 172.21.15.37:0/3265777012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c08107400 msgr2=0x7f7c081077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:11.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.373+0000 7f7c1028e700 1 --2- 172.21.15.37:0/3265777012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c08107400 0x7f7c081077e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009990 tx=0x7f7c04009ca0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.373+0000 7f7c1028e700 1 -- 172.21.15.37:0/3265777012 shutdown_connections 2024-06-10T01:04:11.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.373+0000 7f7c1028e700 1 --2- 172.21.15.37:0/3265777012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c08101d50 0x7f7c0810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.373+0000 7f7c1028e700 1 --2- 172.21.15.37:0/3265777012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c08107400 0x7f7c081077e0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.373+0000 7f7c1028e700 1 --2- 172.21.15.37:0/3265777012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c08101390 0x7f7c08101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.373+0000 7f7c1028e700 1 -- 172.21.15.37:0/3265777012 >> 172.21.15.37:0/3265777012 conn(0x7f7c08078340 msgr2=0x7f7c08078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:11.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.373+0000 7f7c1028e700 1 -- 172.21.15.37:0/3265777012 shutdown_connections 2024-06-10T01:04:11.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.373+0000 7f7c1028e700 1 -- 172.21.15.37:0/3265777012 wait complete. 2024-06-10T01:04:11.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.374+0000 7f7c1028e700 1 Processor -- start 2024-06-10T01:04:11.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.374+0000 7f7c1028e700 1 -- start start 2024-06-10T01:04:11.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.374+0000 7f7c1028e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c08101390 0x7f7c08103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:11.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.374+0000 7f7c1028e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c08101d50 0x7f7c081035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:11.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.375+0000 7f7c1028e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c08107400 0x7f7c08103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:11.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.375+0000 7f7c1028e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7c081116a0 con 0x7f7c08101d50 2024-06-10T01:04:11.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.375+0000 7f7c1028e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7c08111520 con 0x7f7c08107400 2024-06-10T01:04:11.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.375+0000 7f7c1028e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7c08111820 con 0x7f7c08101390 2024-06-10T01:04:11.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.375+0000 7f7c0d829700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c08101d50 0x7f7c081035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:11.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.375+0000 7f7c0d829700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c08101d50 0x7f7c081035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51484/0 (socket says 172.21.15.37:51484) 2024-06-10T01:04:11.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.375+0000 7f7c0d829700 1 -- 172.21.15.37:0/3845967948 learned_addr learned my addr 172.21.15.37:0/3845967948 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:11.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.375+0000 7f7c0e02a700 1 --2- 172.21.15.37:0/3845967948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c08101390 0x7f7c08103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:11.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.375+0000 7f7c0e82b700 1 --2- 172.21.15.37:0/3845967948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c08107400 0x7f7c08103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:11.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.375+0000 7f7c0d829700 1 -- 172.21.15.37:0/3845967948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c08101390 msgr2=0x7f7c08103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:11.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.375+0000 7f7c0d829700 1 --2- 172.21.15.37:0/3845967948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c08101390 0x7f7c08103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.375+0000 7f7c0d829700 1 -- 172.21.15.37:0/3845967948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c08107400 msgr2=0x7f7c08103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:11.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.375+0000 7f7c0d829700 1 --2- 172.21.15.37:0/3845967948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c08107400 0x7f7c08103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.375+0000 7f7c0d829700 1 -- 172.21.15.37:0/3845967948 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c081a7120 con 0x7f7c08101d50 2024-06-10T01:04:11.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.375+0000 7f7c0e82b700 1 --2- 172.21.15.37:0/3845967948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c08107400 0x7f7c08103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:04:11.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.375+0000 7f7c0d829700 1 --2- 172.21.15.37:0/3845967948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c08101d50 0x7f7c081035b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800e9c0 tx=0x7f7bf800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:11.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.376+0000 7f7bff7fe700 1 -- 172.21.15.37:0/3845967948 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf800cbd0 con 0x7f7c08101d50 2024-06-10T01:04:11.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.376+0000 7f7bff7fe700 1 -- 172.21.15.37:0/3845967948 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bf800cd70 con 0x7f7c08101d50 2024-06-10T01:04:11.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.376+0000 7f7c1028e700 1 -- 172.21.15.37:0/3845967948 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c081a7410 con 0x7f7c08101d50 2024-06-10T01:04:11.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.376+0000 7f7bff7fe700 1 -- 172.21.15.37:0/3845967948 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8010430 con 0x7f7c08101d50 2024-06-10T01:04:11.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.376+0000 7f7c1028e700 1 -- 172.21.15.37:0/3845967948 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c081a7950 con 0x7f7c08101d50 2024-06-10T01:04:11.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.379+0000 7f7bff7fe700 1 -- 172.21.15.37:0/3845967948 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7bf8004750 con 0x7f7c08101d50 2024-06-10T01:04:11.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.380+0000 7f7bff7fe700 1 --2- 172.21.15.37:0/3845967948 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7bf40a2ca0 0x7f7bf40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:11.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.380+0000 7f7c0e02a700 1 --2- 172.21.15.37:0/3845967948 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7bf40a2ca0 0x7f7bf40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:11.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.380+0000 7f7bff7fe700 1 -- 172.21.15.37:0/3845967948 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f7bf80cf670 con 0x7f7c08101d50 2024-06-10T01:04:11.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.380+0000 7f7c0e02a700 1 --2- 172.21.15.37:0/3845967948 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7bf40a2ca0 0x7f7bf40a5160 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400b2e0 tx=0x7f7c04005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:11.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.380+0000 7f7c1ab11b80 1 Processor -- start 2024-06-10T01:04:11.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.381+0000 7f7c1ab11b80 1 -- start start 2024-06-10T01:04:11.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.381+0000 7f7c1ab11b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea825d6f60 0x55ea825d7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:11.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.381+0000 7f7c1ab11b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea825d7880 0x55ea825d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:11.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.381+0000 7f7c1ab11b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea825d17a0 0x55ea825d1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:11.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.381+0000 7f7c1ab11b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ea824c3f60 con 0x55ea825d7880 2024-06-10T01:04:11.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.381+0000 7f7c1ab11b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ea82522850 con 0x55ea825d6f60 2024-06-10T01:04:11.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.381+0000 7f7c1ab11b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ea8248a460 con 0x55ea825d17a0 2024-06-10T01:04:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.381+0000 7f7c0d829700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea825d7880 0x55ea825d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.381+0000 7f7c0d829700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea825d7880 0x55ea825d1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51486/0 (socket says 172.21.15.37:51486) 2024-06-10T01:04:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.381+0000 7f7c0d829700 1 -- 172.21.15.37:0/2423898626 learned_addr learned my addr 172.21.15.37:0/2423898626 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.381+0000 7f7c0e82b700 1 --2- 172.21.15.37:0/2423898626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea825d17a0 0x55ea825d1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.381+0000 7f7c0e02a700 1 --2- 172.21.15.37:0/2423898626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea825d6f60 0x55ea825d7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.381+0000 7f7c0d829700 1 -- 172.21.15.37:0/2423898626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea825d17a0 msgr2=0x55ea825d1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.381+0000 7f7c0d829700 1 --2- 172.21.15.37:0/2423898626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea825d17a0 0x55ea825d1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.381+0000 7f7c0d829700 1 -- 172.21.15.37:0/2423898626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea825d6f60 msgr2=0x55ea825d7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.381+0000 7f7c0d829700 1 --2- 172.21.15.37:0/2423898626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea825d6f60 0x55ea825d7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.381+0000 7f7c0d829700 1 -- 172.21.15.37:0/2423898626 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea824718d0 con 0x55ea825d7880 2024-06-10T01:04:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.382+0000 7f7c0d829700 1 --2- 172.21.15.37:0/2423898626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea825d7880 0x55ea825d1260 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8097f40 tx=0x7f7bf809b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d510d1ff3279829 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.382+0000 7f7bfcd70700 1 -- 172.21.15.37:0/2423898626 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf809c1e0 con 0x55ea825d7880 2024-06-10T01:04:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.382+0000 7f7bfcd70700 1 -- 172.21.15.37:0/2423898626 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bf80b4070 con 0x55ea825d7880 2024-06-10T01:04:11.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.382+0000 7f7bfcd70700 1 -- 172.21.15.37:0/2423898626 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf80cf350 con 0x55ea825d7880 2024-06-10T01:04:11.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.383+0000 7f7c1ab11b80 1 -- 172.21.15.37:0/2423898626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea825d7880 msgr2=0x55ea825d1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:11.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.383+0000 7f7c1ab11b80 1 --2- 172.21.15.37:0/2423898626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea825d7880 0x55ea825d1260 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8097f40 tx=0x7f7bf809b980 comp rx=0 tx=0).stop 2024-06-10T01:04:11.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.383+0000 7f7c1ab11b80 1 -- 172.21.15.37:0/2423898626 shutdown_connections 2024-06-10T01:04:11.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.383+0000 7f7c1ab11b80 1 --2- 172.21.15.37:0/2423898626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea825d17a0 0x55ea825d1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.383+0000 7f7c1ab11b80 1 --2- 172.21.15.37:0/2423898626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea825d6f60 0x55ea825d7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.383+0000 7f7c1ab11b80 1 --2- 172.21.15.37:0/2423898626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea825d7880 0x55ea825d1260 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.383+0000 7f7c1ab11b80 1 -- 172.21.15.37:0/2423898626 >> 172.21.15.37:0/2423898626 conn(0x55ea824fe080 msgr2=0x55ea825cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:11.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.383+0000 7f7c1ab11b80 1 -- 172.21.15.37:0/2423898626 shutdown_connections 2024-06-10T01:04:11.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.383+0000 7f7c1ab11b80 1 -- 172.21.15.37:0/2423898626 wait complete. 2024-06-10T01:04:11.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.383+0000 7f7c1ab11b80 1 -- start start 2024-06-10T01:04:11.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.384+0000 7f7c1ab11b80 1 Processor -- start 2024-06-10T01:04:11.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.384+0000 7f7c1ab11b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea825d17a0 0x55ea825dd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:11.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.384+0000 7f7c1ab11b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea825d6f60 0x55ea825da910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:11.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.384+0000 7f7c1ab11b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea825d7880 0x55ea825dae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:11.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.384+0000 7f7c1ab11b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ea824c2fc0 con 0x55ea825d7880 2024-06-10T01:04:11.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.384+0000 7f7c1ab11b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ea8251e810 con 0x55ea825d17a0 2024-06-10T01:04:11.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.384+0000 7f7c1ab11b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ea825de200 con 0x55ea825d6f60 2024-06-10T01:04:11.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.384+0000 7f7c1ab11b80 10 client.0 _ll_register_callbacks cb 0x55ea824b85b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:11.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.384+0000 7f7c0e82b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea825d7880 0x55ea825dae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:11.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.384+0000 7f7c0d829700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea825d6f60 0x55ea825da910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:11.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.384+0000 7f7c0e82b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea825d7880 0x55ea825dae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51498/0 (socket says 172.21.15.37:51498) 2024-06-10T01:04:11.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.384+0000 7f7c0e82b700 1 -- 172.21.15.37:0/380412110 learned_addr learned my addr 172.21.15.37:0/380412110 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:11.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.384+0000 7f7c0e02a700 1 --2- 172.21.15.37:0/380412110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea825d17a0 0x55ea825dd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:11.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.384+0000 7f7c0e82b700 1 -- 172.21.15.37:0/380412110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea825d6f60 msgr2=0x55ea825da910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:11.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.384+0000 7f7c0e82b700 1 --2- 172.21.15.37:0/380412110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea825d6f60 0x55ea825da910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.384+0000 7f7c0e82b700 1 -- 172.21.15.37:0/380412110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea825d17a0 msgr2=0x55ea825dd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:11.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.384+0000 7f7c0e82b700 1 --2- 172.21.15.37:0/380412110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea825d17a0 0x55ea825dd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.384+0000 7f7c0e82b700 1 -- 172.21.15.37:0/380412110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea824e8750 con 0x55ea825d7880 2024-06-10T01:04:11.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.385+0000 7f7c0e82b700 1 --2- 172.21.15.37:0/380412110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea825d7880 0x55ea825dae50 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f7c00010c70 tx=0x7f7c00010f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f70d5e254111ce0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:11.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.385+0000 7f7bf1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:04:11.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.385+0000 7f7bf1ffb700 1 -- 172.21.15.37:0/380412110 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c00018070 con 0x55ea825d7880 2024-06-10T01:04:11.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.385+0000 7f7bf1ffb700 1 -- 172.21.15.37:0/380412110 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f7c0000ebe0 con 0x55ea825d7880 2024-06-10T01:04:11.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.385+0000 7f7c1ab11b80 1 -- 172.21.15.37:0/380412110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bf80b4070 con 0x55ea825d7880 2024-06-10T01:04:11.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.385+0000 7f7bf1ffb700 1 -- 172.21.15.37:0/380412110 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7c00024050 con 0x55ea825d7880 2024-06-10T01:04:11.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.385+0000 7f7c1ab11b80 10 client.16680 fetch_fsmap learned FSMap version 29 2024-06-10T01:04:11.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.385+0000 7f7c1ab11b80 1 -- 172.21.15.37:0/380412110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ea825e8a10 con 0x55ea825d7880 2024-06-10T01:04:11.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.385+0000 7f7bf1ffb700 1 -- 172.21.15.37:0/380412110 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bf80b4070 con 0x55ea825d7880 2024-06-10T01:04:11.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.385+0000 7f7bf1ffb700 1 -- 172.21.15.37:0/380412110 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f7c00013400 con 0x55ea825d7880 2024-06-10T01:04:11.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.386+0000 7f7c1ab11b80 10 client.16680 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:04:11.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.386+0000 7f7c1ab11b80 10 client.16680 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-10T01:04:11.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.386+0000 7f7c1ab11b80 20 client.16680 populate_metadata read hostname 'smithi037' 2024-06-10T01:04:11.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.386+0000 7f7c1ab11b80 1 --2- 172.21.15.37:0/380412110 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55ea825eaff0 0x55ea825ed3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:11.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.386+0000 7f7c1ab11b80 4 client.16680 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:04:11.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.386+0000 7f7c1ab11b80 1 -- 172.21.15.37:0/380412110 --> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea8252fd50 con 0x55ea825eaff0 2024-06-10T01:04:11.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.386+0000 7f7c0e02a700 1 --2- 172.21.15.37:0/380412110 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55ea825eaff0 0x55ea825ed3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:11.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.386+0000 7f7c0e02a700 1 --2- 172.21.15.37:0/380412110 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55ea825eaff0 0x55ea825ed3e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=17087ec20152435b server_cookie=5586dd72bc4f3a92 in_seq=0 out_seq=0 2024-06-10T01:04:11.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.386+0000 7f7bf1ffb700 10 client.16680 ms_handle_connect on v2:172.21.15.167:6836/2720148065 2024-06-10T01:04:11.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.387+0000 7f7bf1ffb700 1 -- 172.21.15.37:0/380412110 <== mds.2 v2:172.21.15.167:6836/2720148065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ea8252fd50 con 0x55ea825eaff0 2024-06-10T01:04:11.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.387+0000 7f7bf1ffb700 10 client.16680 handle_command_reply: tid=0 2024-06-10T01:04:11.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.387+0000 7f7c1ab11b80 1 client.16680 shutdown 2024-06-10T01:04:11.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.387+0000 7f7c1ab11b80 1 -- 172.21.15.37:0/380412110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea825d7880 msgr2=0x55ea825dae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:11.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.387+0000 7f7c1ab11b80 1 --2- 172.21.15.37:0/380412110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea825d7880 0x55ea825dae50 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f7c00010c70 tx=0x7f7c00010f80 comp rx=0 tx=0).stop 2024-06-10T01:04:11.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.387+0000 7f7c1ab11b80 1 -- 172.21.15.37:0/380412110 shutdown_connections 2024-06-10T01:04:11.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.387+0000 7f7c1ab11b80 1 --2- 172.21.15.37:0/380412110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea825d7880 0x55ea825dae50 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.387+0000 7f7c1ab11b80 1 --2- 172.21.15.37:0/380412110 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55ea825eaff0 0x55ea825ed3e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.387+0000 7f7c1ab11b80 1 --2- 172.21.15.37:0/380412110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea825d17a0 0x55ea825dd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.387+0000 7f7c1ab11b80 1 --2- 172.21.15.37:0/380412110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea825d6f60 0x55ea825da910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.387+0000 7f7c1ab11b80 1 -- 172.21.15.37:0/380412110 >> 172.21.15.37:0/380412110 conn(0x55ea824fe080 msgr2=0x55ea825cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:11.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.387+0000 7f7bf1ffb700 0 client.16680 ms_handle_reset on v2:172.21.15.167:6836/2720148065 2024-06-10T01:04:11.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.387+0000 7f7c1ab11b80 1 -- 172.21.15.37:0/380412110 shutdown_connections 2024-06-10T01:04:11.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.387+0000 7f7c1ab11b80 1 -- 172.21.15.37:0/380412110 wait complete. 2024-06-10T01:04:11.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.387+0000 7f7c1ab11b80 20 client.16680 trim_cache size 0 max 16384 2024-06-10T01:04:11.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.404+0000 7f7c1ab11b80 1 Processor -- start 2024-06-10T01:04:11.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.404+0000 7f7c1ab11b80 1 -- start start 2024-06-10T01:04:11.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.404+0000 7f7c1ab11b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea825d17a0 0x55ea825d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:11.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.404+0000 7f7c1ab11b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea825d6f60 0x55ea825cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:11.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.404+0000 7f7c1ab11b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea825d7880 0x55ea825d7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:11.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.404+0000 7f7c1ab11b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7c081b2f50 con 0x55ea825d7880 2024-06-10T01:04:11.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.404+0000 7f7c1ab11b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7c081b07e0 con 0x55ea825d6f60 2024-06-10T01:04:11.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.404+0000 7f7c1ab11b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7c081ae630 con 0x55ea825d17a0 2024-06-10T01:04:11.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.404+0000 7f7c0e82b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea825d7880 0x55ea825d7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:11.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.404+0000 7f7c0e82b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea825d7880 0x55ea825d7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51512/0 (socket says 172.21.15.37:51512) 2024-06-10T01:04:11.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.404+0000 7f7c0e82b700 1 -- 172.21.15.37:0/1278100365 learned_addr learned my addr 172.21.15.37:0/1278100365 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:11.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.405+0000 7f7c0e02a700 1 --2- 172.21.15.37:0/1278100365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea825d17a0 0x55ea825d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:11.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.405+0000 7f7c0d829700 1 --2- 172.21.15.37:0/1278100365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea825d6f60 0x55ea825cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:11.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.405+0000 7f7c0e82b700 1 -- 172.21.15.37:0/1278100365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea825d17a0 msgr2=0x55ea825d1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:11.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.405+0000 7f7c0e82b700 1 --2- 172.21.15.37:0/1278100365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea825d17a0 0x55ea825d1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.405+0000 7f7c0e82b700 1 -- 172.21.15.37:0/1278100365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea825d6f60 msgr2=0x55ea825cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:11.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.405+0000 7f7c0e82b700 1 --2- 172.21.15.37:0/1278100365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea825d6f60 0x55ea825cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.405+0000 7f7c0e82b700 1 -- 172.21.15.37:0/1278100365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea825d8e80 con 0x55ea825d7880 2024-06-10T01:04:11.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.405+0000 7f7c0e82b700 1 --2- 172.21.15.37:0/1278100365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea825d7880 0x55ea825d7d20 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f7c00000f80 tx=0x7f7c00013e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47ec522ab88fd84 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.405+0000 7f7bf3fff700 1 -- 172.21.15.37:0/1278100365 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c00018040 con 0x55ea825d7880 2024-06-10T01:04:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.405+0000 7f7bf3fff700 1 -- 172.21.15.37:0/1278100365 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c0002d070 con 0x55ea825d7880 2024-06-10T01:04:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.405+0000 7f7bf3fff700 1 -- 172.21.15.37:0/1278100365 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c00014440 con 0x55ea825d7880 2024-06-10T01:04:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.406+0000 7f7c1ab11b80 1 -- 172.21.15.37:0/1278100365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea825d7880 msgr2=0x55ea825d7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.406+0000 7f7c1ab11b80 1 --2- 172.21.15.37:0/1278100365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea825d7880 0x55ea825d7d20 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f7c00000f80 tx=0x7f7c00013e90 comp rx=0 tx=0).stop 2024-06-10T01:04:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.406+0000 7f7c1ab11b80 1 -- 172.21.15.37:0/1278100365 shutdown_connections 2024-06-10T01:04:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.406+0000 7f7c1ab11b80 1 --2- 172.21.15.37:0/1278100365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea825d7880 0x55ea825d7d20 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.406+0000 7f7c1ab11b80 1 --2- 172.21.15.37:0/1278100365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea825d17a0 0x55ea825d1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.406+0000 7f7c1ab11b80 1 --2- 172.21.15.37:0/1278100365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea825d6f60 0x55ea825cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.406+0000 7f7c1ab11b80 1 -- 172.21.15.37:0/1278100365 >> 172.21.15.37:0/1278100365 conn(0x55ea824fe080 msgr2=0x55ea825d9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.406+0000 7f7c1ab11b80 1 -- 172.21.15.37:0/1278100365 shutdown_connections 2024-06-10T01:04:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.406+0000 7f7c1ab11b80 1 -- 172.21.15.37:0/1278100365 wait complete. 2024-06-10T01:04:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.407+0000 7f7c1ab11b80 1 -- start start 2024-06-10T01:04:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.407+0000 7f7c1ab11b80 1 Processor -- start 2024-06-10T01:04:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.407+0000 7f7c1ab11b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea825d17a0 0x55ea825e34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.407+0000 7f7c1ab11b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea825d6f60 0x55ea825e3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.407+0000 7f7c1ab11b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea825d7880 0x55ea825f1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.407+0000 7f7c1ab11b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7c081aeca0 con 0x55ea825d6f60 2024-06-10T01:04:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.407+0000 7f7c1ab11b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7c081b0110 con 0x55ea825d7880 2024-06-10T01:04:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.407+0000 7f7c1ab11b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7c081af370 con 0x55ea825d17a0 2024-06-10T01:04:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.407+0000 7f7c1ab11b80 10 client.0 _ll_register_callbacks cb 0x55ea824b85b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.407+0000 7f7c0e02a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea825d17a0 0x55ea825e34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.407+0000 7f7c0e02a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea825d17a0 0x55ea825e34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:50270/0 (socket says 172.21.15.37:50270) 2024-06-10T01:04:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.407+0000 7f7c0e02a700 1 -- 172.21.15.37:0/3349607706 learned_addr learned my addr 172.21.15.37:0/3349607706 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.407+0000 7f7c0d829700 1 --2- 172.21.15.37:0/3349607706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea825d6f60 0x55ea825e3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.407+0000 7f7c0e82b700 1 --2- 172.21.15.37:0/3349607706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea825d7880 0x55ea825f1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.408+0000 7f7c0e02a700 1 -- 172.21.15.37:0/3349607706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea825d7880 msgr2=0x55ea825f1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.408+0000 7f7c0e02a700 1 --2- 172.21.15.37:0/3349607706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea825d7880 0x55ea825f1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.408+0000 7f7c0e02a700 1 -- 172.21.15.37:0/3349607706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea825d6f60 msgr2=0x55ea825e3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.408+0000 7f7c0e02a700 1 --2- 172.21.15.37:0/3349607706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea825d6f60 0x55ea825e3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.408+0000 7f7c0e02a700 1 -- 172.21.15.37:0/3349607706 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea825f37b0 con 0x55ea825d17a0 2024-06-10T01:04:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.408+0000 7f7c0d829700 1 --2- 172.21.15.37:0/3349607706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea825d6f60 0x55ea825e3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.408+0000 7f7c0e82b700 1 --2- 172.21.15.37:0/3349607706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea825d7880 0x55ea825f1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.408+0000 7f7c0e02a700 1 --2- 172.21.15.37:0/3349607706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea825d17a0 0x55ea825e34c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009990 tx=0x7f7c04009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=205b10ecf3fc8e59 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.408+0000 7f7bf1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:04:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.408+0000 7f7bf1ffb700 1 -- 172.21.15.37:0/3349607706 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04003cd0 con 0x55ea825d17a0 2024-06-10T01:04:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.408+0000 7f7bf1ffb700 1 -- 172.21.15.37:0/3349607706 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f7c0400f460 con 0x55ea825d17a0 2024-06-10T01:04:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.408+0000 7f7c1ab11b80 1 -- 172.21.15.37:0/3349607706 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c0002d070 con 0x55ea825d17a0 2024-06-10T01:04:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.408+0000 7f7bf1ffb700 1 -- 172.21.15.37:0/3349607706 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7c04044050 con 0x55ea825d17a0 2024-06-10T01:04:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.408+0000 7f7c1ab11b80 10 client.26239 fetch_fsmap learned FSMap version 29 2024-06-10T01:04:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.409+0000 7f7c1ab11b80 1 -- 172.21.15.37:0/3349607706 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ea82655150 con 0x55ea825d17a0 2024-06-10T01:04:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.409+0000 7f7bf1ffb700 1 -- 172.21.15.37:0/3349607706 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c0002d070 con 0x55ea825d17a0 2024-06-10T01:04:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.409+0000 7f7bf1ffb700 1 -- 172.21.15.37:0/3349607706 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f7c04032680 con 0x55ea825d17a0 2024-06-10T01:04:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.409+0000 7f7c1ab11b80 10 client.26239 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:04:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.409+0000 7f7c1ab11b80 10 client.26239 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-10T01:04:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.409+0000 7f7c1ab11b80 20 client.26239 populate_metadata read hostname 'smithi037' 2024-06-10T01:04:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.409+0000 7f7c1ab11b80 1 --2- 172.21.15.37:0/3349607706 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55ea826558c0 0x55ea82657ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.409+0000 7f7c1ab11b80 4 client.26239 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:04:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.409+0000 7f7c1ab11b80 1 -- 172.21.15.37:0/3349607706 --> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea8252a2d0 con 0x55ea826558c0 2024-06-10T01:04:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.409+0000 7f7c0d829700 1 --2- 172.21.15.37:0/3349607706 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55ea826558c0 0x55ea82657ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.410+0000 7f7c0d829700 1 --2- 172.21.15.37:0/3349607706 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55ea826558c0 0x55ea82657ce0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=36dfea63f1aba87e server_cookie=4b5f3491816cfa78 in_seq=0 out_seq=0 2024-06-10T01:04:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.410+0000 7f7bf1ffb700 10 client.26239 ms_handle_connect on v2:172.21.15.167:6836/2720148065 2024-06-10T01:04:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.410+0000 7f7bf1ffb700 1 -- 172.21.15.37:0/3349607706 <== mds.2 v2:172.21.15.167:6836/2720148065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea8252a2d0 con 0x55ea826558c0 2024-06-10T01:04:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.410+0000 7f7bf1ffb700 10 client.26239 handle_command_reply: tid=0 2024-06-10T01:04:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.410+0000 7f7c1ab11b80 1 client.26239 shutdown 2024-06-10T01:04:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.410+0000 7f7c1ab11b80 1 -- 172.21.15.37:0/3349607706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea825d17a0 msgr2=0x55ea825e34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.410+0000 7f7c1ab11b80 1 --2- 172.21.15.37:0/3349607706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea825d17a0 0x55ea825e34c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009990 tx=0x7f7c04009ca0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.410+0000 7f7c1ab11b80 1 -- 172.21.15.37:0/3349607706 shutdown_connections 2024-06-10T01:04:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.410+0000 7f7c1ab11b80 1 --2- 172.21.15.37:0/3349607706 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55ea826558c0 0x55ea82657ce0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.410+0000 7f7c1ab11b80 1 --2- 172.21.15.37:0/3349607706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea825d7880 0x55ea825f1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.410+0000 7f7c1ab11b80 1 --2- 172.21.15.37:0/3349607706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ea825d17a0 0x55ea825e34c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.410+0000 7f7c1ab11b80 1 --2- 172.21.15.37:0/3349607706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea825d6f60 0x55ea825e3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.410+0000 7f7c1ab11b80 1 -- 172.21.15.37:0/3349607706 >> 172.21.15.37:0/3349607706 conn(0x55ea824fe080 msgr2=0x55ea825d7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.410+0000 7f7bf1ffb700 0 client.26239 ms_handle_reset on v2:172.21.15.167:6836/2720148065 2024-06-10T01:04:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.410+0000 7f7c1ab11b80 1 -- 172.21.15.37:0/3349607706 shutdown_connections 2024-06-10T01:04:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.411+0000 7f7c1ab11b80 1 -- 172.21.15.37:0/3349607706 wait complete. 2024-06-10T01:04:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.411+0000 7f7c1ab11b80 20 client.26239 trim_cache size 0 max 16384 2024-06-10T01:04:11.414 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T01:04:11.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.411+0000 7f7c1028e700 1 -- 172.21.15.37:0/3845967948 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7bf40a2ca0 msgr2=0x7f7bf40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:11.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.411+0000 7f7c1028e700 1 --2- 172.21.15.37:0/3845967948 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7bf40a2ca0 0x7f7bf40a5160 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400b2e0 tx=0x7f7c04005960 comp rx=0 tx=0).stop 2024-06-10T01:04:11.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.411+0000 7f7c1028e700 1 -- 172.21.15.37:0/3845967948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c08101d50 msgr2=0x7f7c081035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:11.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.411+0000 7f7c1028e700 1 --2- 172.21.15.37:0/3845967948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c08101d50 0x7f7c081035b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800e9c0 tx=0x7f7bf800ecd0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.411+0000 7f7c1028e700 1 -- 172.21.15.37:0/3845967948 shutdown_connections 2024-06-10T01:04:11.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.411+0000 7f7c1028e700 1 --2- 172.21.15.37:0/3845967948 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7bf40a2ca0 0x7f7bf40a5160 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.411+0000 7f7c1028e700 1 --2- 172.21.15.37:0/3845967948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c08107400 0x7f7c08103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.411+0000 7f7c1028e700 1 --2- 172.21.15.37:0/3845967948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c08101390 0x7f7c08103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.411+0000 7f7c1028e700 1 --2- 172.21.15.37:0/3845967948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c08101d50 0x7f7c081035b0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:11.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.411+0000 7f7c1028e700 1 -- 172.21.15.37:0/3845967948 >> 172.21.15.37:0/3845967948 conn(0x7f7c08078340 msgr2=0x7f7c08100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:11.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.411+0000 7f7c1028e700 1 -- 172.21.15.37:0/3845967948 shutdown_connections 2024-06-10T01:04:11.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:11.411+0000 7f7c1028e700 1 -- 172.21.15.37:0/3845967948 wait complete. 2024-06-10T01:04:12.115 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:04:12.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:12 smithi167 ceph-mon[36917]: pgmap v656: 129 pgs: 129 active+clean; 7.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 1.2 MiB/s wr, 1.40k op/s 2024-06-10T01:04:12.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:12 smithi094 ceph-mon[36708]: pgmap v656: 129 pgs: 129 active+clean; 7.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 1.2 MiB/s wr, 1.40k op/s 2024-06-10T01:04:12.368 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:12.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:12 smithi037 ceph-mon[32192]: pgmap v656: 129 pgs: 129 active+clean; 7.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 1.2 MiB/s wr, 1.40k op/s 2024-06-10T01:04:14.309 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:14 smithi037 ceph-mon[32192]: pgmap v657: 129 pgs: 129 active+clean; 8.6 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.1 MiB/s wr, 1.36k op/s 2024-06-10T01:04:14.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:14 smithi167 ceph-mon[36917]: pgmap v657: 129 pgs: 129 active+clean; 8.6 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.1 MiB/s wr, 1.36k op/s 2024-06-10T01:04:14.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:14 smithi094 ceph-mon[36708]: pgmap v657: 129 pgs: 129 active+clean; 8.6 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.1 MiB/s wr, 1.36k op/s 2024-06-10T01:04:14.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.521+0000 7f038fecd700 1 -- 172.21.15.37:0/1375219340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03881085a0 msgr2=0x7f0388108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:14.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.521+0000 7f038fecd700 1 --2- 172.21.15.37:0/1375219340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03881085a0 0x7f0388108980 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f03840099c0 tx=0x7f0384009cd0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.521+0000 7f038fecd700 1 -- 172.21.15.37:0/1375219340 shutdown_connections 2024-06-10T01:04:14.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.521+0000 7f038fecd700 1 --2- 172.21.15.37:0/1375219340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f038806b580 0x7f038810f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.521+0000 7f038fecd700 1 --2- 172.21.15.37:0/1375219340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03881085a0 0x7f0388108980 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.521+0000 7f038fecd700 1 --2- 172.21.15.37:0/1375219340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f038806abc0 0x7f038806b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.521+0000 7f038fecd700 1 -- 172.21.15.37:0/1375219340 >> 172.21.15.37:0/1375219340 conn(0x7f0388077200 msgr2=0x7f0388077610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:14.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.521+0000 7f038fecd700 1 -- 172.21.15.37:0/1375219340 shutdown_connections 2024-06-10T01:04:14.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.521+0000 7f038fecd700 1 -- 172.21.15.37:0/1375219340 wait complete. 2024-06-10T01:04:14.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.522+0000 7f038fecd700 1 Processor -- start 2024-06-10T01:04:14.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.522+0000 7f038fecd700 1 -- start start 2024-06-10T01:04:14.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.522+0000 7f038fecd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f038806abc0 0x7f038819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:14.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.522+0000 7f038fecd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f038806b580 0x7f038819ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:14.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.522+0000 7f038fecd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03881085a0 0x7f0388198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:14.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.522+0000 7f038fecd700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f03881127f0 con 0x7f03881085a0 2024-06-10T01:04:14.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.522+0000 7f038fecd700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0388112670 con 0x7f038806b580 2024-06-10T01:04:14.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.522+0000 7f038fecd700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0388112970 con 0x7f038806abc0 2024-06-10T01:04:14.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.522+0000 7f038d468700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f038806b580 0x7f038819ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:14.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.522+0000 7f038e46a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03881085a0 0x7f0388198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:14.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.522+0000 7f038dc69700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f038806abc0 0x7f038819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:14.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.522+0000 7f038d468700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f038806b580 0x7f038819ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:56960/0 (socket says 172.21.15.37:56960) 2024-06-10T01:04:14.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.522+0000 7f038dc69700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f038806abc0 0x7f038819e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:33526/0 (socket says 172.21.15.37:33526) 2024-06-10T01:04:14.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.522+0000 7f038d468700 1 -- 172.21.15.37:0/2004649636 learned_addr learned my addr 172.21.15.37:0/2004649636 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:14.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.523+0000 7f038e46a700 1 -- 172.21.15.37:0/2004649636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f038806abc0 msgr2=0x7f038819e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:14.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.523+0000 7f038e46a700 1 --2- 172.21.15.37:0/2004649636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f038806abc0 0x7f038819e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.523+0000 7f038e46a700 1 -- 172.21.15.37:0/2004649636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f038806b580 msgr2=0x7f038819ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:14.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.523+0000 7f038e46a700 1 --2- 172.21.15.37:0/2004649636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f038806b580 0x7f038819ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.523+0000 7f038e46a700 1 -- 172.21.15.37:0/2004649636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0388199210 con 0x7f03881085a0 2024-06-10T01:04:14.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.523+0000 7f038dc69700 1 --2- 172.21.15.37:0/2004649636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f038806abc0 0x7f038819e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:14.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.523+0000 7f038d468700 1 --2- 172.21.15.37:0/2004649636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f038806b580 0x7f038819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:14.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.523+0000 7f038e46a700 1 --2- 172.21.15.37:0/2004649636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03881085a0 0x7f0388198ab0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f038000efc0 tx=0x7f038000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:14.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.523+0000 7f037effd700 1 -- 172.21.15.37:0/2004649636 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03800092e0 con 0x7f03881085a0 2024-06-10T01:04:14.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.523+0000 7f038fecd700 1 -- 172.21.15.37:0/2004649636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03881994a0 con 0x7f03881085a0 2024-06-10T01:04:14.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.523+0000 7f038fecd700 1 -- 172.21.15.37:0/2004649636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03881a57c0 con 0x7f03881085a0 2024-06-10T01:04:14.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.523+0000 7f037effd700 1 -- 172.21.15.37:0/2004649636 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0380004bb0 con 0x7f03881085a0 2024-06-10T01:04:14.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.523+0000 7f037effd700 1 -- 172.21.15.37:0/2004649636 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0380004280 con 0x7f03881085a0 2024-06-10T01:04:14.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.525+0000 7f037effd700 1 -- 172.21.15.37:0/2004649636 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f038001b030 con 0x7f03881085a0 2024-06-10T01:04:14.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.526+0000 7f037effd700 1 --2- 172.21.15.37:0/2004649636 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f03740a2ca0 0x7f03740a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:14.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.526+0000 7f038dc69700 1 --2- 172.21.15.37:0/2004649636 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f03740a2ca0 0x7f03740a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:14.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.526+0000 7f037effd700 1 -- 172.21.15.37:0/2004649636 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f03800ced20 con 0x7f03881085a0 2024-06-10T01:04:14.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.527+0000 7f038dc69700 1 --2- 172.21.15.37:0/2004649636 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f03740a2ca0 0x7f03740a5160 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0384000c00 tx=0x7f0384005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:14.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.527+0000 7f039a750b80 1 Processor -- start 2024-06-10T01:04:14.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.527+0000 7f039a750b80 1 -- start start 2024-06-10T01:04:14.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.527+0000 7f039a750b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a7b4b4f60 0x559a7b4b5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:14.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.527+0000 7f039a750b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a7b4b5880 0x559a7b4af260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:14.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.527+0000 7f039a750b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a7b4af7a0 0x559a7b4afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:14.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.527+0000 7f039a750b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559a7b3a1f60 con 0x559a7b4b4f60 2024-06-10T01:04:14.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.527+0000 7f039a750b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559a7b400850 con 0x559a7b4af7a0 2024-06-10T01:04:14.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.528+0000 7f039a750b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559a7b368460 con 0x559a7b4b5880 2024-06-10T01:04:14.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.528+0000 7f038dc69700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a7b4b4f60 0x559a7b4b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:14.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.528+0000 7f038d468700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a7b4b5880 0x559a7b4af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:14.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.528+0000 7f038dc69700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a7b4b4f60 0x559a7b4b5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39964/0 (socket says 172.21.15.37:39964) 2024-06-10T01:04:14.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.528+0000 7f038dc69700 1 -- 172.21.15.37:0/258139554 learned_addr learned my addr 172.21.15.37:0/258139554 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:14.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.528+0000 7f038e46a700 1 --2- 172.21.15.37:0/258139554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a7b4af7a0 0x559a7b4afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:14.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.528+0000 7f038dc69700 1 -- 172.21.15.37:0/258139554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a7b4b5880 msgr2=0x559a7b4af260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:14.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.528+0000 7f038dc69700 1 --2- 172.21.15.37:0/258139554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a7b4b5880 0x559a7b4af260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.528+0000 7f038dc69700 1 -- 172.21.15.37:0/258139554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a7b4af7a0 msgr2=0x559a7b4afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:14.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.528+0000 7f038dc69700 1 --2- 172.21.15.37:0/258139554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a7b4af7a0 0x559a7b4afbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.528+0000 7f038dc69700 1 -- 172.21.15.37:0/258139554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a7b34f8d0 con 0x559a7b4b4f60 2024-06-10T01:04:14.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.528+0000 7f038e46a700 1 --2- 172.21.15.37:0/258139554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a7b4af7a0 0x559a7b4afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:14.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.528+0000 7f038d468700 1 --2- 172.21.15.37:0/258139554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a7b4b5880 0x559a7b4af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:14.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.528+0000 7f038dc69700 1 --2- 172.21.15.37:0/258139554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a7b4b4f60 0x559a7b4b5340 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0384015000 tx=0x7f0384020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae82c94f5ee7581 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:14.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.528+0000 7f0373fff700 1 -- 172.21.15.37:0/258139554 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0384003c40 con 0x559a7b4b4f60 2024-06-10T01:04:14.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.529+0000 7f0373fff700 1 -- 172.21.15.37:0/258139554 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0384003de0 con 0x559a7b4b4f60 2024-06-10T01:04:14.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.529+0000 7f0373fff700 1 -- 172.21.15.37:0/258139554 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038400fcc0 con 0x559a7b4b4f60 2024-06-10T01:04:14.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.529+0000 7f039a750b80 1 -- 172.21.15.37:0/258139554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a7b4b4f60 msgr2=0x559a7b4b5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:14.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.529+0000 7f039a750b80 1 --2- 172.21.15.37:0/258139554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a7b4b4f60 0x559a7b4b5340 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0384015000 tx=0x7f0384020000 comp rx=0 tx=0).stop 2024-06-10T01:04:14.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.529+0000 7f039a750b80 1 -- 172.21.15.37:0/258139554 shutdown_connections 2024-06-10T01:04:14.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.529+0000 7f039a750b80 1 --2- 172.21.15.37:0/258139554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a7b4af7a0 0x559a7b4afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.529+0000 7f039a750b80 1 --2- 172.21.15.37:0/258139554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a7b4b4f60 0x559a7b4b5340 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.529+0000 7f039a750b80 1 --2- 172.21.15.37:0/258139554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a7b4b5880 0x559a7b4af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.529+0000 7f039a750b80 1 -- 172.21.15.37:0/258139554 >> 172.21.15.37:0/258139554 conn(0x559a7b3dc080 msgr2=0x559a7b4aa220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:14.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.529+0000 7f039a750b80 1 -- 172.21.15.37:0/258139554 shutdown_connections 2024-06-10T01:04:14.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.529+0000 7f039a750b80 1 -- 172.21.15.37:0/258139554 wait complete. 2024-06-10T01:04:14.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.530+0000 7f039a750b80 1 -- start start 2024-06-10T01:04:14.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.530+0000 7f039a750b80 1 Processor -- start 2024-06-10T01:04:14.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.530+0000 7f039a750b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a7b4af7a0 0x559a7b4bb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:14.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.530+0000 7f039a750b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a7b4b4f60 0x559a7b4b8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:14.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.530+0000 7f039a750b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a7b4b5880 0x559a7b4b8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:14.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.530+0000 7f039a750b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559a7b3a0fc0 con 0x559a7b4b5880 2024-06-10T01:04:14.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.530+0000 7f039a750b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559a7b3fc810 con 0x559a7b4af7a0 2024-06-10T01:04:14.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.530+0000 7f039a750b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559a7b4bc200 con 0x559a7b4b4f60 2024-06-10T01:04:14.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.530+0000 7f039a750b80 10 client.0 _ll_register_callbacks cb 0x559a7b3965b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:14.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.530+0000 7f038dc69700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a7b4af7a0 0x559a7b4bb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:14.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.531+0000 7f038dc69700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a7b4af7a0 0x559a7b4bb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:56986/0 (socket says 172.21.15.37:56986) 2024-06-10T01:04:14.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.531+0000 7f038dc69700 1 -- 172.21.15.37:0/3420185876 learned_addr learned my addr 172.21.15.37:0/3420185876 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:14.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.531+0000 7f038e46a700 1 --2- 172.21.15.37:0/3420185876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a7b4b5880 0x559a7b4b8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:14.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.531+0000 7f038d468700 1 --2- 172.21.15.37:0/3420185876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a7b4b4f60 0x559a7b4b8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:14.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.531+0000 7f038dc69700 1 -- 172.21.15.37:0/3420185876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a7b4b4f60 msgr2=0x559a7b4b8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:14.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.531+0000 7f038dc69700 1 --2- 172.21.15.37:0/3420185876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a7b4b4f60 0x559a7b4b8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.531+0000 7f038dc69700 1 -- 172.21.15.37:0/3420185876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a7b4b5880 msgr2=0x559a7b4b8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:14.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.531+0000 7f038dc69700 1 --2- 172.21.15.37:0/3420185876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a7b4b5880 0x559a7b4b8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.531+0000 7f038dc69700 1 -- 172.21.15.37:0/3420185876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a7b3c6750 con 0x559a7b4af7a0 2024-06-10T01:04:14.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.531+0000 7f038e46a700 1 --2- 172.21.15.37:0/3420185876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a7b4b5880 0x559a7b4b8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:14.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.531+0000 7f038dc69700 1 --2- 172.21.15.37:0/3420185876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a7b4af7a0 0x559a7b4bb540 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f038400c010 tx=0x7f038400fb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b4c38a0e50425c9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:14.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.531+0000 7f03717fa700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:04:14.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.531+0000 7f03717fa700 1 -- 172.21.15.37:0/3420185876 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0384003c40 con 0x559a7b4af7a0 2024-06-10T01:04:14.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.531+0000 7f039a750b80 1 -- 172.21.15.37:0/3420185876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0384003de0 con 0x559a7b4af7a0 2024-06-10T01:04:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.531+0000 7f03717fa700 1 -- 172.21.15.37:0/3420185876 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f0384005250 con 0x559a7b4af7a0 2024-06-10T01:04:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.531+0000 7f03717fa700 1 -- 172.21.15.37:0/3420185876 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0384040050 con 0x559a7b4af7a0 2024-06-10T01:04:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.532+0000 7f039a750b80 10 client.26138 fetch_fsmap learned FSMap version 29 2024-06-10T01:04:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.532+0000 7f039a750b80 1 -- 172.21.15.37:0/3420185876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559a7b4c6930 con 0x559a7b4af7a0 2024-06-10T01:04:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.532+0000 7f03717fa700 1 -- 172.21.15.37:0/3420185876 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0384003de0 con 0x559a7b4af7a0 2024-06-10T01:04:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.532+0000 7f03717fa700 1 -- 172.21.15.37:0/3420185876 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f0384024410 con 0x559a7b4af7a0 2024-06-10T01:04:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.532+0000 7f039a750b80 10 client.26138 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:04:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.532+0000 7f039a750b80 10 client.26138 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-06-10T01:04:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.532+0000 7f039a750b80 20 client.26138 populate_metadata read hostname 'smithi037' 2024-06-10T01:04:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.532+0000 7f039a750b80 1 --2- 172.21.15.37:0/3420185876 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x559a7b4c8e00 0x559a7b4cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.532+0000 7f039a750b80 4 client.26138 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:04:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.532+0000 7f039a750b80 1 -- 172.21.15.37:0/3420185876 --> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a7b40dd50 con 0x559a7b4c8e00 2024-06-10T01:04:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.532+0000 7f038d468700 1 --2- 172.21.15.37:0/3420185876 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x559a7b4c8e00 0x559a7b4cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:14.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.532+0000 7f03717fa700 10 client.26138 ms_handle_connect on v2:172.21.15.94:6838/946269331 2024-06-10T01:04:14.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.532+0000 7f038d468700 1 --2- 172.21.15.37:0/3420185876 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x559a7b4c8e00 0x559a7b4cb1f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c5f30e1018779c29 server_cookie=9878db0b6b089bc4 in_seq=0 out_seq=0 2024-06-10T01:04:14.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.533+0000 7f03717fa700 1 -- 172.21.15.37:0/3420185876 <== mds.1 v2:172.21.15.94:6838/946269331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559a7b40dd50 con 0x559a7b4c8e00 2024-06-10T01:04:14.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.533+0000 7f03717fa700 10 client.26138 handle_command_reply: tid=0 2024-06-10T01:04:14.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.533+0000 7f039a750b80 1 client.26138 shutdown 2024-06-10T01:04:14.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.533+0000 7f039a750b80 1 -- 172.21.15.37:0/3420185876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a7b4af7a0 msgr2=0x559a7b4bb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:14.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.533+0000 7f039a750b80 1 --2- 172.21.15.37:0/3420185876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a7b4af7a0 0x559a7b4bb540 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f038400c010 tx=0x7f038400fb80 comp rx=0 tx=0).stop 2024-06-10T01:04:14.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.534+0000 7f039a750b80 1 -- 172.21.15.37:0/3420185876 shutdown_connections 2024-06-10T01:04:14.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.534+0000 7f039a750b80 1 --2- 172.21.15.37:0/3420185876 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x559a7b4c8e00 0x559a7b4cb1f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.534+0000 7f039a750b80 1 --2- 172.21.15.37:0/3420185876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a7b4b5880 0x559a7b4b8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.534+0000 7f039a750b80 1 --2- 172.21.15.37:0/3420185876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a7b4af7a0 0x559a7b4bb540 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.534+0000 7f039a750b80 1 --2- 172.21.15.37:0/3420185876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a7b4b4f60 0x559a7b4b8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.534+0000 7f039a750b80 1 -- 172.21.15.37:0/3420185876 >> 172.21.15.37:0/3420185876 conn(0x559a7b3dc080 msgr2=0x559a7b4a9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:14.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.534+0000 7f03717fa700 0 client.26138 ms_handle_reset on v2:172.21.15.94:6838/946269331 2024-06-10T01:04:14.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.534+0000 7f039a750b80 1 -- 172.21.15.37:0/3420185876 shutdown_connections 2024-06-10T01:04:14.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.534+0000 7f039a750b80 1 -- 172.21.15.37:0/3420185876 wait complete. 2024-06-10T01:04:14.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.534+0000 7f039a750b80 20 client.26138 trim_cache size 0 max 16384 2024-06-10T01:04:14.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.549+0000 7f039a750b80 1 Processor -- start 2024-06-10T01:04:14.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.549+0000 7f039a750b80 1 -- start start 2024-06-10T01:04:14.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.549+0000 7f039a750b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a7b4af7a0 0x559a7b4b7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:14.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.549+0000 7f039a750b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a7b4b4f60 0x559a7b4a9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:14.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.549+0000 7f039a750b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a7b4b5880 0x559a7b4aa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:14.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.549+0000 7f039a750b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f036c011500 con 0x559a7b4b4f60 2024-06-10T01:04:14.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.549+0000 7f039a750b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f036c00ed90 con 0x559a7b4b5880 2024-06-10T01:04:14.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.549+0000 7f039a750b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f036c00cbe0 con 0x559a7b4af7a0 2024-06-10T01:04:14.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.549+0000 7f038d468700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a7b4b4f60 0x559a7b4a9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:14.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.549+0000 7f038d468700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a7b4b4f60 0x559a7b4a9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39982/0 (socket says 172.21.15.37:39982) 2024-06-10T01:04:14.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.549+0000 7f038d468700 1 -- 172.21.15.37:0/3725064004 learned_addr learned my addr 172.21.15.37:0/3725064004 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:14.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.550+0000 7f038dc69700 1 --2- 172.21.15.37:0/3725064004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a7b4af7a0 0x559a7b4b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:14.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.550+0000 7f038e46a700 1 --2- 172.21.15.37:0/3725064004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a7b4b5880 0x559a7b4aa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:14.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.550+0000 7f038d468700 1 -- 172.21.15.37:0/3725064004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a7b4af7a0 msgr2=0x559a7b4b7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:14.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.550+0000 7f038d468700 1 --2- 172.21.15.37:0/3725064004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a7b4af7a0 0x559a7b4b7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.550+0000 7f038d468700 1 -- 172.21.15.37:0/3725064004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a7b4b5880 msgr2=0x559a7b4aa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:14.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.550+0000 7f038d468700 1 --2- 172.21.15.37:0/3725064004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a7b4b5880 0x559a7b4aa3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.550+0000 7f038d468700 1 -- 172.21.15.37:0/3725064004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a7b4b6dd0 con 0x559a7b4b4f60 2024-06-10T01:04:14.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.550+0000 7f038d468700 1 --2- 172.21.15.37:0/3725064004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a7b4b4f60 0x559a7b4a9e80 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x559a7b4b9e10 tx=0x7f0378009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd1379341ecfa9d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:14.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.550+0000 7f03737fe700 1 -- 172.21.15.37:0/3725064004 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0378004830 con 0x559a7b4b4f60 2024-06-10T01:04:14.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.550+0000 7f03737fe700 1 -- 172.21.15.37:0/3725064004 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0378005020 con 0x559a7b4b4f60 2024-06-10T01:04:14.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.550+0000 7f03737fe700 1 -- 172.21.15.37:0/3725064004 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0378020840 con 0x559a7b4b4f60 2024-06-10T01:04:14.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.551+0000 7f039a750b80 1 -- 172.21.15.37:0/3725064004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a7b4b4f60 msgr2=0x559a7b4a9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:14.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.551+0000 7f039a750b80 1 --2- 172.21.15.37:0/3725064004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a7b4b4f60 0x559a7b4a9e80 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x559a7b4b9e10 tx=0x7f0378009260 comp rx=0 tx=0).stop 2024-06-10T01:04:14.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.551+0000 7f039a750b80 1 -- 172.21.15.37:0/3725064004 shutdown_connections 2024-06-10T01:04:14.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.551+0000 7f039a750b80 1 --2- 172.21.15.37:0/3725064004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a7b4b5880 0x559a7b4aa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.551+0000 7f039a750b80 1 --2- 172.21.15.37:0/3725064004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a7b4af7a0 0x559a7b4b7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.551+0000 7f039a750b80 1 --2- 172.21.15.37:0/3725064004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a7b4b4f60 0x559a7b4a9e80 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.551+0000 7f039a750b80 1 -- 172.21.15.37:0/3725064004 >> 172.21.15.37:0/3725064004 conn(0x559a7b3dc080 msgr2=0x559a7b4b7150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:14.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.551+0000 7f039a750b80 1 -- 172.21.15.37:0/3725064004 shutdown_connections 2024-06-10T01:04:14.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.551+0000 7f039a750b80 1 -- 172.21.15.37:0/3725064004 wait complete. 2024-06-10T01:04:14.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.551+0000 7f039a750b80 1 -- start start 2024-06-10T01:04:14.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.551+0000 7f039a750b80 1 Processor -- start 2024-06-10T01:04:14.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.552+0000 7f039a750b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a7b4af7a0 0x559a7b4cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:14.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.552+0000 7f039a750b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a7b4b4f60 0x559a7b4cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:14.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.552+0000 7f039a750b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a7b4b5880 0x559a7b531730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:14.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.552+0000 7f039a750b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f036c00d250 con 0x559a7b4b5880 2024-06-10T01:04:14.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.552+0000 7f039a750b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f036c00e6c0 con 0x559a7b4b4f60 2024-06-10T01:04:14.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.552+0000 7f039a750b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f036c00d920 con 0x559a7b4af7a0 2024-06-10T01:04:14.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.552+0000 7f039a750b80 10 client.0 _ll_register_callbacks cb 0x559a7b3965b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:14.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.552+0000 7f038e46a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a7b4b5880 0x559a7b531730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:14.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.552+0000 7f038e46a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a7b4b5880 0x559a7b531730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39994/0 (socket says 172.21.15.37:39994) 2024-06-10T01:04:14.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.552+0000 7f038e46a700 1 -- 172.21.15.37:0/4020683552 learned_addr learned my addr 172.21.15.37:0/4020683552 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:14.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.552+0000 7f038d468700 1 --2- 172.21.15.37:0/4020683552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a7b4b4f60 0x559a7b4cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:14.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.552+0000 7f038dc69700 1 --2- 172.21.15.37:0/4020683552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a7b4af7a0 0x559a7b4cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:14.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.553+0000 7f038d468700 1 -- 172.21.15.37:0/4020683552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a7b4af7a0 msgr2=0x559a7b4cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:14.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.553+0000 7f038d468700 1 --2- 172.21.15.37:0/4020683552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a7b4af7a0 0x559a7b4cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.553+0000 7f038d468700 1 -- 172.21.15.37:0/4020683552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a7b4b5880 msgr2=0x559a7b531730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:14.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.553+0000 7f038d468700 1 --2- 172.21.15.37:0/4020683552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a7b4b5880 0x559a7b531730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.553+0000 7f038d468700 1 -- 172.21.15.37:0/4020683552 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a7b4cbde0 con 0x559a7b4b4f60 2024-06-10T01:04:14.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.553+0000 7f038e46a700 1 --2- 172.21.15.37:0/4020683552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a7b4b5880 0x559a7b531730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:14.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.553+0000 7f038d468700 1 --2- 172.21.15.37:0/4020683552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a7b4b4f60 0x559a7b4cb4f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x559a7b4ab3b0 tx=0x7f0378009e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39525718345f74ae server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:14.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.553+0000 7f03717fa700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:04:14.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.554+0000 7f03717fa700 1 -- 172.21.15.37:0/4020683552 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03780055e0 con 0x559a7b4b4f60 2024-06-10T01:04:14.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.554+0000 7f03717fa700 1 -- 172.21.15.37:0/4020683552 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f0378004710 con 0x559a7b4b4f60 2024-06-10T01:04:14.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.554+0000 7f039a750b80 1 -- 172.21.15.37:0/4020683552 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0378005020 con 0x559a7b4b4f60 2024-06-10T01:04:14.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.554+0000 7f03717fa700 1 -- 172.21.15.37:0/4020683552 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0378004910 con 0x559a7b4b4f60 2024-06-10T01:04:14.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.554+0000 7f039a750b80 10 client.26144 fetch_fsmap learned FSMap version 29 2024-06-10T01:04:14.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.554+0000 7f039a750b80 1 -- 172.21.15.37:0/4020683552 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559a7b532da0 con 0x559a7b4b4f60 2024-06-10T01:04:14.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.554+0000 7f03717fa700 1 -- 172.21.15.37:0/4020683552 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0378005020 con 0x559a7b4b4f60 2024-06-10T01:04:14.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.554+0000 7f03717fa700 1 -- 172.21.15.37:0/4020683552 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f0378030070 con 0x559a7b4b4f60 2024-06-10T01:04:14.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.554+0000 7f039a750b80 10 client.26144 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:04:14.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.554+0000 7f039a750b80 10 client.26144 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-06-10T01:04:14.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.554+0000 7f039a750b80 20 client.26144 populate_metadata read hostname 'smithi037' 2024-06-10T01:04:14.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.554+0000 7f039a750b80 1 --2- 172.21.15.37:0/4020683552 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x559a7b533400 0x559a7b535820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:14.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.554+0000 7f039a750b80 4 client.26144 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:04:14.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.554+0000 7f039a750b80 1 -- 172.21.15.37:0/4020683552 --> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a7b4082d0 con 0x559a7b533400 2024-06-10T01:04:14.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.555+0000 7f038dc69700 1 --2- 172.21.15.37:0/4020683552 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x559a7b533400 0x559a7b535820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:14.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.555+0000 7f038dc69700 1 --2- 172.21.15.37:0/4020683552 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x559a7b533400 0x559a7b535820 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e6166a18fac93460 server_cookie=5c8e32ae850b577a in_seq=0 out_seq=0 2024-06-10T01:04:14.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.555+0000 7f03717fa700 10 client.26144 ms_handle_connect on v2:172.21.15.94:6838/946269331 2024-06-10T01:04:14.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.555+0000 7f03717fa700 1 -- 172.21.15.37:0/4020683552 <== mds.1 v2:172.21.15.94:6838/946269331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a7b4082d0 con 0x559a7b533400 2024-06-10T01:04:14.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.555+0000 7f03717fa700 10 client.26144 handle_command_reply: tid=0 2024-06-10T01:04:14.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.555+0000 7f039a750b80 1 client.26144 shutdown 2024-06-10T01:04:14.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.555+0000 7f039a750b80 1 -- 172.21.15.37:0/4020683552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a7b4b4f60 msgr2=0x559a7b4cb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:14.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.555+0000 7f039a750b80 1 --2- 172.21.15.37:0/4020683552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a7b4b4f60 0x559a7b4cb4f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x559a7b4ab3b0 tx=0x7f0378009e00 comp rx=0 tx=0).stop 2024-06-10T01:04:14.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.555+0000 7f039a750b80 1 -- 172.21.15.37:0/4020683552 shutdown_connections 2024-06-10T01:04:14.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.555+0000 7f039a750b80 1 --2- 172.21.15.37:0/4020683552 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x559a7b533400 0x559a7b535820 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.555+0000 7f039a750b80 1 --2- 172.21.15.37:0/4020683552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559a7b4b5880 0x559a7b531730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.555+0000 7f03717fa700 0 client.26144 ms_handle_reset on v2:172.21.15.94:6838/946269331 2024-06-10T01:04:14.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.555+0000 7f039a750b80 1 --2- 172.21.15.37:0/4020683552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a7b4af7a0 0x559a7b4cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.555+0000 7f039a750b80 1 --2- 172.21.15.37:0/4020683552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a7b4b4f60 0x559a7b4cb4f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.555+0000 7f039a750b80 1 -- 172.21.15.37:0/4020683552 >> 172.21.15.37:0/4020683552 conn(0x559a7b3dc080 msgr2=0x559a7b4b6370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:14.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.555+0000 7f039a750b80 1 -- 172.21.15.37:0/4020683552 shutdown_connections 2024-06-10T01:04:14.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.556+0000 7f039a750b80 1 -- 172.21.15.37:0/4020683552 wait complete. 2024-06-10T01:04:14.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.556+0000 7f039a750b80 20 client.26144 trim_cache size 0 max 16384 2024-06-10T01:04:14.558 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T01:04:14.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.556+0000 7f038fecd700 1 -- 172.21.15.37:0/2004649636 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f03740a2ca0 msgr2=0x7f03740a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:14.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.556+0000 7f038fecd700 1 --2- 172.21.15.37:0/2004649636 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f03740a2ca0 0x7f03740a5160 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0384000c00 tx=0x7f0384005fb0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.556+0000 7f038fecd700 1 -- 172.21.15.37:0/2004649636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03881085a0 msgr2=0x7f0388198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:14.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.556+0000 7f038fecd700 1 --2- 172.21.15.37:0/2004649636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03881085a0 0x7f0388198ab0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f038000efc0 tx=0x7f038000c3a0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.556+0000 7f038fecd700 1 -- 172.21.15.37:0/2004649636 shutdown_connections 2024-06-10T01:04:14.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.556+0000 7f038fecd700 1 --2- 172.21.15.37:0/2004649636 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f03740a2ca0 0x7f03740a5160 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.556+0000 7f038fecd700 1 --2- 172.21.15.37:0/2004649636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03881085a0 0x7f0388198ab0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.556+0000 7f038fecd700 1 --2- 172.21.15.37:0/2004649636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f038806abc0 0x7f038819e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.556+0000 7f038fecd700 1 --2- 172.21.15.37:0/2004649636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f038806b580 0x7f038819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:14.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.556+0000 7f038fecd700 1 -- 172.21.15.37:0/2004649636 >> 172.21.15.37:0/2004649636 conn(0x7f0388077200 msgr2=0x7f03881003f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:14.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.556+0000 7f038fecd700 1 -- 172.21.15.37:0/2004649636 shutdown_connections 2024-06-10T01:04:14.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:14.556+0000 7f038fecd700 1 -- 172.21.15.37:0/2004649636 wait complete. 2024-06-10T01:04:15.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:15.056+0000 7f7cfcff9700 1 -- 172.21.15.37:0/2994816611 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f1d841f430 con 0x55f1d84dff40 2024-06-10T01:04:15.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:15.056+0000 7f7cfcff9700 10 client.16644 handle_command_reply: tid=0 2024-06-10T01:04:15.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:15.056+0000 7f7d25c6eb80 1 client.16644 shutdown 2024-06-10T01:04:15.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:15.056+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/2994816611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1d84c6470 msgr2=0x55f1d84d7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:15.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:15.056+0000 7f7d25c6eb80 1 --2- 172.21.15.37:0/2994816611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1d84c6470 0x55f1d84d7690 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000fde0 tx=0x7f7d1000fac0 comp rx=0 tx=0).stop 2024-06-10T01:04:15.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:15.056+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/2994816611 shutdown_connections 2024-06-10T01:04:15.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:15.056+0000 7f7d25c6eb80 1 --2- 172.21.15.37:0/2994816611 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f1d84dff40 0x55f1d8567540 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:15.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:15.056+0000 7f7d25c6eb80 1 --2- 172.21.15.37:0/2994816611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1d84cc600 0x55f1d84d8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:15.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:15.056+0000 7f7d25c6eb80 1 --2- 172.21.15.37:0/2994816611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1d84c6470 0x55f1d84d7690 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:15.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:15.056+0000 7f7d25c6eb80 1 --2- 172.21.15.37:0/2994816611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f1d84cbce0 0x55f1d84d7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:15.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:15.056+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/2994816611 >> 172.21.15.37:0/2994816611 conn(0x55f1d83f0e60 msgr2=0x55f1d84cd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:15.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:15.056+0000 7f7cfcff9700 0 client.16644 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:15.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:15.056+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/2994816611 shutdown_connections 2024-06-10T01:04:15.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:15.056+0000 7f7d25c6eb80 1 -- 172.21.15.37:0/2994816611 wait complete. 2024-06-10T01:04:15.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:15.056+0000 7f7d25c6eb80 20 client.16644 trim_cache size 0 max 16384 2024-06-10T01:04:15.055 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:04:15.055 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 29, 2024-06-10T01:04:15.055 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24472, 2024-06-10T01:04:15.056 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 18, 2024-06-10T01:04:15.056 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-06-10T01:04:15.056 INFO:teuthology.orchestra.run.smithi037.stdout: "11dd31af": { 2024-06-10T01:04:15.056 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 18, 2024-06-10T01:04:15.056 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-06-10T01:04:15.056 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:04:15.056 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-06-10T01:04:15.056 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:04:15.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:04:15.056 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-06-10T01:04:15.056 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 149.4, 2024-06-10T01:04:15.056 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-06-10T01:04:15.057 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-06-10T01:04:15.057 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-06-10T01:04:15.057 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:04:15.057 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-06-10T01:04:15.057 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:04:15.057 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:04:15.057 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:04:15.057 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:04:15.057 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:04:15.057 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:04:15.057 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:04:15.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:15.057+0000 7f7d1b3eb700 1 -- 172.21.15.37:0/1675054791 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7d000a2e10 msgr2=0x7f7d000a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:15.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:15.057+0000 7f7d1b3eb700 1 --2- 172.21.15.37:0/1675054791 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7d000a2e10 0x7f7d000a52d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009c10 tx=0x7f7d10010e60 comp rx=0 tx=0).stop 2024-06-10T01:04:15.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:15.057+0000 7f7d1b3eb700 1 -- 172.21.15.37:0/1675054791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d141013a0 msgr2=0x7f7d1419ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:15.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:15.057+0000 7f7d1b3eb700 1 --2- 172.21.15.37:0/1675054791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d141013a0 0x7f7d1419ed20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400b530 tx=0x7f7d0400b840 comp rx=0 tx=0).stop 2024-06-10T01:04:15.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:15.057+0000 7f7d1b3eb700 1 -- 172.21.15.37:0/1675054791 shutdown_connections 2024-06-10T01:04:15.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:15.057+0000 7f7d1b3eb700 1 --2- 172.21.15.37:0/1675054791 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7d000a2e10 0x7f7d000a52d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:15.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:15.057+0000 7f7d1b3eb700 1 --2- 172.21.15.37:0/1675054791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d14108560 0x7f7d141989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:15.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:15.057+0000 7f7d1b3eb700 1 --2- 172.21.15.37:0/1675054791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d141009e0 0x7f7d1419e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:15.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:15.057+0000 7f7d1b3eb700 1 --2- 172.21.15.37:0/1675054791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d141013a0 0x7f7d1419ed20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:15.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:15.057+0000 7f7d1b3eb700 1 -- 172.21.15.37:0/1675054791 >> 172.21.15.37:0/1675054791 conn(0x7f7d14077990 msgr2=0x7f7d140fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:15.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:15.057+0000 7f7d1b3eb700 1 -- 172.21.15.37:0/1675054791 shutdown_connections 2024-06-10T01:04:15.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:15.057+0000 7f7d1b3eb700 1 -- 172.21.15.37:0/1675054791 wait complete. 2024-06-10T01:04:15.320 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:04:15.989 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '11dd31af', quiesce took 21.7 seconds. Will release after: 7.75 2024-06-10T01:04:16.008 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:16.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:16 smithi167 ceph-mon[36917]: pgmap v658: 129 pgs: 129 active+clean; 11 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 1.2 MiB/s wr, 1.03k op/s 2024-06-10T01:04:16.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:16 smithi094 ceph-mon[36708]: pgmap v658: 129 pgs: 129 active+clean; 11 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 1.2 MiB/s wr, 1.03k op/s 2024-06-10T01:04:16.376 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:16 smithi037 ceph-mon[32192]: pgmap v658: 129 pgs: 129 active+clean; 11 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 1.2 MiB/s wr, 1.03k op/s 2024-06-10T01:04:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.958+0000 7efd1e99e640 1 -- 172.21.15.37:0/2811426429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd18102480 msgr2=0x7efd18104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.958+0000 7efd1e99e640 1 --2- 172.21.15.37:0/2811426429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd18102480 0x7efd18104870 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7efd08009900 tx=0x7efd0802a410 comp rx=0 tx=0).stop 2024-06-10T01:04:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.958+0000 7efd1e99e640 1 -- 172.21.15.37:0/2811426429 shutdown_connections 2024-06-10T01:04:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.958+0000 7efd1e99e640 1 --2- 172.21.15.37:0/2811426429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd181076e0 0x7efd18109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.958+0000 7efd1e99e640 1 --2- 172.21.15.37:0/2811426429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd18104db0 0x7efd181071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.958+0000 7efd1e99e640 1 --2- 172.21.15.37:0/2811426429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd18102480 0x7efd18104870 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.958+0000 7efd1e99e640 1 -- 172.21.15.37:0/2811426429 >> 172.21.15.37:0/2811426429 conn(0x7efd180fbe70 msgr2=0x7efd180fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.958+0000 7efd1e99e640 1 -- 172.21.15.37:0/2811426429 shutdown_connections 2024-06-10T01:04:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.958+0000 7efd1e99e640 1 -- 172.21.15.37:0/2811426429 wait complete. 2024-06-10T01:04:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.959+0000 7efd1e99e640 1 Processor -- start 2024-06-10T01:04:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.959+0000 7efd1e99e640 1 -- start start 2024-06-10T01:04:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.959+0000 7efd1e99e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd18102480 0x7efd18197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.959+0000 7efd1e99e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd18104db0 0x7efd18198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.959+0000 7efd1e99e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd181076e0 0x7efd1819f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.959+0000 7efd1e99e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efd1810d680 con 0x7efd18104db0 2024-06-10T01:04:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.959+0000 7efd1e99e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efd1810d500 con 0x7efd181076e0 2024-06-10T01:04:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.959+0000 7efd1e99e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efd1810d800 con 0x7efd18102480 2024-06-10T01:04:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.959+0000 7efd0ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd18104db0 0x7efd18198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.959+0000 7efd0ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd18104db0 0x7efd18198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40000/0 (socket says 172.21.15.37:40000) 2024-06-10T01:04:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.959+0000 7efd0ffff640 1 -- 172.21.15.37:0/2959440504 learned_addr learned my addr 172.21.15.37:0/2959440504 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.959+0000 7efd1cf3b640 1 --2- 172.21.15.37:0/2959440504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd181076e0 0x7efd1819f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.960+0000 7efd17fff640 1 --2- 172.21.15.37:0/2959440504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd18102480 0x7efd18197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.960+0000 7efd0ffff640 1 -- 172.21.15.37:0/2959440504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd18102480 msgr2=0x7efd18197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.960+0000 7efd0ffff640 1 --2- 172.21.15.37:0/2959440504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd18102480 0x7efd18197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.960+0000 7efd0ffff640 1 -- 172.21.15.37:0/2959440504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd181076e0 msgr2=0x7efd1819f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.960+0000 7efd0ffff640 1 --2- 172.21.15.37:0/2959440504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd181076e0 0x7efd1819f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.960+0000 7efd0ffff640 1 -- 172.21.15.37:0/2959440504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1819f900 con 0x7efd18104db0 2024-06-10T01:04:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.960+0000 7efd17fff640 1 --2- 172.21.15.37:0/2959440504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd18102480 0x7efd18197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:04:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.960+0000 7efd0ffff640 1 --2- 172.21.15.37:0/2959440504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd18104db0 0x7efd18198200 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7efd00005fd0 tx=0x7efd0000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.960+0000 7efd15ffb640 1 -- 172.21.15.37:0/2959440504 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00017070 con 0x7efd18104db0 2024-06-10T01:04:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.960+0000 7efd15ffb640 1 -- 172.21.15.37:0/2959440504 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd00009c30 con 0x7efd18104db0 2024-06-10T01:04:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.960+0000 7efd1e99e640 1 -- 172.21.15.37:0/2959440504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd1819fbf0 con 0x7efd18104db0 2024-06-10T01:04:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.960+0000 7efd15ffb640 1 -- 172.21.15.37:0/2959440504 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00010460 con 0x7efd18104db0 2024-06-10T01:04:16.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.960+0000 7efd1e99e640 1 -- 172.21.15.37:0/2959440504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd181a0140 con 0x7efd18104db0 2024-06-10T01:04:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.962+0000 7efd15ffb640 1 -- 172.21.15.37:0/2959440504 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7efd00003e70 con 0x7efd18104db0 2024-06-10T01:04:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.963+0000 7efd21786740 1 Processor -- start 2024-06-10T01:04:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.963+0000 7efd21786740 1 -- start start 2024-06-10T01:04:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.963+0000 7efd21786740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ceaaaaa420 0x55ceaaaaa840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.963+0000 7efd21786740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceaaaaae10 0x55ceaaaaf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.963+0000 7efd21786740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ceaaaaff80 0x55ceaaab2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.963+0000 7efd21786740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ceaaa094f0 con 0x55ceaaaaae10 2024-06-10T01:04:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.963+0000 7efd21786740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ceaa9a7870 con 0x55ceaaaaff80 2024-06-10T01:04:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.963+0000 7efd21786740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ceaaa054b0 con 0x55ceaaaaa420 2024-06-10T01:04:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.963+0000 7efd17fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceaaaaae10 0x55ceaaaaf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.963+0000 7efd17fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceaaaaae10 0x55ceaaaaf910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40012/0 (socket says 172.21.15.37:40012) 2024-06-10T01:04:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.963+0000 7efd17fff640 1 -- 172.21.15.37:0/2810003937 learned_addr learned my addr 172.21.15.37:0/2810003937 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.963+0000 7efd0ffff640 1 --2- 172.21.15.37:0/2810003937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ceaaaaff80 0x55ceaaab2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.963+0000 7efd1cf3b640 1 --2- 172.21.15.37:0/2810003937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ceaaaaa420 0x55ceaaaaa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.964+0000 7efd17fff640 1 -- 172.21.15.37:0/2810003937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ceaaaaa420 msgr2=0x55ceaaaaa840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.964+0000 7efd17fff640 1 --2- 172.21.15.37:0/2810003937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ceaaaaa420 0x55ceaaaaa840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.964+0000 7efd17fff640 1 -- 172.21.15.37:0/2810003937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ceaaaaff80 msgr2=0x55ceaaab2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.964+0000 7efd17fff640 1 --2- 172.21.15.37:0/2810003937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ceaaaaff80 0x55ceaaab2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.964+0000 7efd17fff640 1 -- 172.21.15.37:0/2810003937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ceaa9c39b0 con 0x55ceaaaaae10 2024-06-10T01:04:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.964+0000 7efd0ffff640 1 --2- 172.21.15.37:0/2810003937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ceaaaaff80 0x55ceaaab2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:04:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.964+0000 7efd17fff640 1 --2- 172.21.15.37:0/2810003937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceaaaaae10 0x55ceaaaaf910 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7efd08005940 tx=0x7efd08032d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e74da84999aa0ef server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.964+0000 7efd15ffb640 1 --2- 172.21.15.37:0/2959440504 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7efd040a2c30 0x7efd040a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.964+0000 7efd1cf3b640 1 --2- 172.21.15.37:0/2959440504 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7efd040a2c30 0x7efd040a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.964+0000 7efd0e7fc640 1 -- 172.21.15.37:0/2810003937 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0803f070 con 0x55ceaaaaae10 2024-06-10T01:04:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.964+0000 7efd15ffb640 1 -- 172.21.15.37:0/2959440504 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7efd000cf110 con 0x7efd18104db0 2024-06-10T01:04:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.964+0000 7efd0e7fc640 1 -- 172.21.15.37:0/2810003937 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd080318b0 con 0x55ceaaaaae10 2024-06-10T01:04:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.964+0000 7efd0e7fc640 1 -- 172.21.15.37:0/2810003937 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08035070 con 0x55ceaaaaae10 2024-06-10T01:04:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.965+0000 7efd21786740 1 -- 172.21.15.37:0/2810003937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceaaaaae10 msgr2=0x55ceaaaaf910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.965+0000 7efd1cf3b640 1 --2- 172.21.15.37:0/2959440504 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7efd040a2c30 0x7efd040a50f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x55ceaaaab670 tx=0x7efd1000a4d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.965+0000 7efd21786740 1 --2- 172.21.15.37:0/2810003937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceaaaaae10 0x55ceaaaaf910 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7efd08005940 tx=0x7efd08032d60 comp rx=0 tx=0).stop 2024-06-10T01:04:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.965+0000 7efd21786740 1 -- 172.21.15.37:0/2810003937 shutdown_connections 2024-06-10T01:04:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.965+0000 7efd21786740 1 --2- 172.21.15.37:0/2810003937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ceaaaaff80 0x55ceaaab2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.965+0000 7efd21786740 1 --2- 172.21.15.37:0/2810003937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceaaaaae10 0x55ceaaaaf910 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.965+0000 7efd21786740 1 --2- 172.21.15.37:0/2810003937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ceaaaaa420 0x55ceaaaaa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.965+0000 7efd21786740 1 -- 172.21.15.37:0/2810003937 >> 172.21.15.37:0/2810003937 conn(0x55ceaaaa1680 msgr2=0x55ceaaaa1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.965+0000 7efd21786740 1 -- 172.21.15.37:0/2810003937 shutdown_connections 2024-06-10T01:04:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.965+0000 7efd21786740 1 -- 172.21.15.37:0/2810003937 wait complete. 2024-06-10T01:04:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.965+0000 7efd21786740 1 -- start start 2024-06-10T01:04:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.965+0000 7efd21786740 1 Processor -- start 2024-06-10T01:04:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.966+0000 7efd21786740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ceaaaaa420 0x55ceaaab46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.966+0000 7efd21786740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceaaaaae10 0x55ceaaab4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.966+0000 7efd21786740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ceaaaaff80 0x55ceaaabbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.966+0000 7efd21786740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ceaaab2ed0 con 0x55ceaaaaae10 2024-06-10T01:04:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.966+0000 7efd21786740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ceaa992ae0 con 0x55ceaaaaff80 2024-06-10T01:04:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.966+0000 7efd21786740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ceaaab3050 con 0x55ceaaaaa420 2024-06-10T01:04:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.966+0000 7efd21786740 10 client.0 _ll_register_callbacks cb 0x55ceaa9cb9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.966+0000 7efd0ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceaaaaae10 0x55ceaaab4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.966+0000 7efd0ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceaaaaae10 0x55ceaaab4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40016/0 (socket says 172.21.15.37:40016) 2024-06-10T01:04:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.966+0000 7efd0ffff640 1 -- 172.21.15.37:0/1974355428 learned_addr learned my addr 172.21.15.37:0/1974355428 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.966+0000 7efd17fff640 1 --2- 172.21.15.37:0/1974355428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ceaaaaa420 0x55ceaaab46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.966+0000 7efd1cf3b640 1 --2- 172.21.15.37:0/1974355428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ceaaaaff80 0x55ceaaabbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.966+0000 7efd0ffff640 1 -- 172.21.15.37:0/1974355428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ceaaaaa420 msgr2=0x55ceaaab46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.966+0000 7efd0ffff640 1 --2- 172.21.15.37:0/1974355428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ceaaaaa420 0x55ceaaab46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.966+0000 7efd0ffff640 1 -- 172.21.15.37:0/1974355428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ceaaaaff80 msgr2=0x55ceaaabbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.966+0000 7efd0ffff640 1 --2- 172.21.15.37:0/1974355428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ceaaaaff80 0x55ceaaabbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.966+0000 7efd0ffff640 1 -- 172.21.15.37:0/1974355428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ceaa71d210 con 0x55ceaaaaae10 2024-06-10T01:04:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.966+0000 7efd0ffff640 1 --2- 172.21.15.37:0/1974355428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceaaaaae10 0x55ceaaab4c00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7efd000979e0 tx=0x7efd0009b450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8d451a3f360e58 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.966+0000 7efcf77fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:04:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.966+0000 7efcf77fe640 1 -- 172.21.15.37:0/1974355428 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00009de0 con 0x55ceaaaaae10 2024-06-10T01:04:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.967+0000 7efcf77fe640 1 -- 172.21.15.37:0/1974355428 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7efd0009a9d0 con 0x55ceaaaaae10 2024-06-10T01:04:16.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.967+0000 7efd21786740 1 -- 172.21.15.37:0/1974355428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd080318b0 con 0x55ceaaaaae10 2024-06-10T01:04:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.967+0000 7efcf77fe640 1 -- 172.21.15.37:0/1974355428 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efd0009b580 con 0x55ceaaaaae10 2024-06-10T01:04:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.967+0000 7efd21786740 10 client.16716 fetch_fsmap learned FSMap version 29 2024-06-10T01:04:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.967+0000 7efd21786740 1 -- 172.21.15.37:0/1974355428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ceaa7e9620 con 0x55ceaaaaae10 2024-06-10T01:04:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.967+0000 7efcf77fe640 1 -- 172.21.15.37:0/1974355428 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd080318b0 con 0x55ceaaaaae10 2024-06-10T01:04:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.967+0000 7efcf77fe640 1 -- 172.21.15.37:0/1974355428 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7efd000ccce0 con 0x55ceaaaaae10 2024-06-10T01:04:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.967+0000 7efd21786740 10 client.16716 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:04:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.967+0000 7efd21786740 10 client.16716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:04:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.967+0000 7efd21786740 20 client.16716 populate_metadata read hostname 'smithi037' 2024-06-10T01:04:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.967+0000 7efd21786740 1 --2- 172.21.15.37:0/1974355428 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55ceaaac0570 0x55ceaaac2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.967+0000 7efd21786740 4 client.16716 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:04:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.967+0000 7efd21786740 1 -- 172.21.15.37:0/1974355428 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ceaaa1cf10 con 0x55ceaaac0570 2024-06-10T01:04:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.967+0000 7efd17fff640 1 --2- 172.21.15.37:0/1974355428 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55ceaaac0570 0x55ceaaac2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.968+0000 7efcf77fe640 10 client.16716 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.968+0000 7efd17fff640 1 --2- 172.21.15.37:0/1974355428 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55ceaaac0570 0x55ceaaac2960 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=853a4e2e58de777f server_cookie=e7937f8beaeb4225 in_seq=0 out_seq=0 2024-06-10T01:04:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.969+0000 7efcf77fe640 1 -- 172.21.15.37:0/1974355428 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ceaaa1cf10 con 0x55ceaaac0570 2024-06-10T01:04:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.969+0000 7efcf77fe640 10 client.16716 handle_command_reply: tid=0 2024-06-10T01:04:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.969+0000 7efd21786740 1 client.16716 shutdown 2024-06-10T01:04:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.969+0000 7efd21786740 1 -- 172.21.15.37:0/1974355428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceaaaaae10 msgr2=0x55ceaaab4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.969+0000 7efd21786740 1 --2- 172.21.15.37:0/1974355428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceaaaaae10 0x55ceaaab4c00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7efd000979e0 tx=0x7efd0009b450 comp rx=0 tx=0).stop 2024-06-10T01:04:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.969+0000 7efd21786740 1 -- 172.21.15.37:0/1974355428 shutdown_connections 2024-06-10T01:04:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.969+0000 7efd21786740 1 --2- 172.21.15.37:0/1974355428 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55ceaaac0570 0x55ceaaac2960 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.969+0000 7efd21786740 1 --2- 172.21.15.37:0/1974355428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ceaaaaff80 0x55ceaaabbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.969+0000 7efd21786740 1 --2- 172.21.15.37:0/1974355428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceaaaaae10 0x55ceaaab4c00 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.969+0000 7efcf77fe640 0 client.16716 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.969+0000 7efd21786740 1 --2- 172.21.15.37:0/1974355428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ceaaaaa420 0x55ceaaab46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.969+0000 7efd21786740 1 -- 172.21.15.37:0/1974355428 >> 172.21.15.37:0/1974355428 conn(0x55ceaaaa1680 msgr2=0x55ceaaaa07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.969+0000 7efd21786740 1 -- 172.21.15.37:0/1974355428 shutdown_connections 2024-06-10T01:04:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.969+0000 7efd21786740 1 -- 172.21.15.37:0/1974355428 wait complete. 2024-06-10T01:04:16.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.969+0000 7efd21786740 20 client.16716 trim_cache size 0 max 16384 2024-06-10T01:04:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.984+0000 7efd21786740 1 Processor -- start 2024-06-10T01:04:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.984+0000 7efd21786740 1 -- start start 2024-06-10T01:04:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.985+0000 7efd21786740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ceaaaa1680 0x55ceaaab3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.985+0000 7efd21786740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceaaaaa420 0x55ceaaabbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.985+0000 7efd21786740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ceaaaaae10 0x55ceaaaa4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.985+0000 7efd21786740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efd181aae30 con 0x55ceaaaaa420 2024-06-10T01:04:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.985+0000 7efd21786740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efd181a8700 con 0x55ceaaaa1680 2024-06-10T01:04:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.985+0000 7efd21786740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efd181a6550 con 0x55ceaaaaae10 2024-06-10T01:04:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.985+0000 7efd0ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceaaaaa420 0x55ceaaabbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.985+0000 7efd0ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceaaaaa420 0x55ceaaabbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40024/0 (socket says 172.21.15.37:40024) 2024-06-10T01:04:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.985+0000 7efd17fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ceaaaa1680 0x55ceaaab3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.985+0000 7efd0ffff640 1 -- 172.21.15.37:0/4224901120 learned_addr learned my addr 172.21.15.37:0/4224901120 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.985+0000 7efd1cf3b640 1 --2- 172.21.15.37:0/4224901120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ceaaaaae10 0x55ceaaaa4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.985+0000 7efd0ffff640 1 -- 172.21.15.37:0/4224901120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ceaaaaae10 msgr2=0x55ceaaaa4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.985+0000 7efd0ffff640 1 --2- 172.21.15.37:0/4224901120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ceaaaaae10 0x55ceaaaa4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.985+0000 7efd0ffff640 1 -- 172.21.15.37:0/4224901120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ceaaaa1680 msgr2=0x55ceaaab3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.985+0000 7efd0ffff640 1 --2- 172.21.15.37:0/4224901120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ceaaaa1680 0x55ceaaab3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.985+0000 7efd0ffff640 1 -- 172.21.15.37:0/4224901120 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ceaaaab6a0 con 0x55ceaaaaa420 2024-06-10T01:04:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.985+0000 7efd17fff640 1 --2- 172.21.15.37:0/4224901120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ceaaaa1680 0x55ceaaab3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.985+0000 7efd0ffff640 1 --2- 172.21.15.37:0/4224901120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceaaaaa420 0x55ceaaabbb90 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7efd00000c00 tx=0x7efd0000b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b7f68f16592aaf server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.985+0000 7efd0dffb640 1 -- 172.21.15.37:0/4224901120 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00009de0 con 0x55ceaaaaa420 2024-06-10T01:04:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.985+0000 7efd0dffb640 1 -- 172.21.15.37:0/4224901120 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd00015020 con 0x55ceaaaaa420 2024-06-10T01:04:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.985+0000 7efd0dffb640 1 -- 172.21.15.37:0/4224901120 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00097bf0 con 0x55ceaaaaa420 2024-06-10T01:04:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.986+0000 7efd21786740 1 -- 172.21.15.37:0/4224901120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceaaaaa420 msgr2=0x55ceaaabbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.986+0000 7efd21786740 1 --2- 172.21.15.37:0/4224901120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceaaaaa420 0x55ceaaabbb90 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7efd00000c00 tx=0x7efd0000b510 comp rx=0 tx=0).stop 2024-06-10T01:04:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.986+0000 7efd21786740 1 -- 172.21.15.37:0/4224901120 shutdown_connections 2024-06-10T01:04:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.986+0000 7efd21786740 1 --2- 172.21.15.37:0/4224901120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ceaaaaae10 0x55ceaaaa4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.986+0000 7efd21786740 1 --2- 172.21.15.37:0/4224901120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceaaaaa420 0x55ceaaabbb90 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.986+0000 7efd21786740 1 --2- 172.21.15.37:0/4224901120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ceaaaa1680 0x55ceaaab3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.986+0000 7efd21786740 1 -- 172.21.15.37:0/4224901120 >> 172.21.15.37:0/4224901120 conn(0x55ceaaaafa80 msgr2=0x55ceaaaac0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.986+0000 7efd21786740 1 -- 172.21.15.37:0/4224901120 shutdown_connections 2024-06-10T01:04:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.986+0000 7efd21786740 1 -- 172.21.15.37:0/4224901120 wait complete. 2024-06-10T01:04:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.986+0000 7efd21786740 1 -- start start 2024-06-10T01:04:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.987+0000 7efd21786740 1 Processor -- start 2024-06-10T01:04:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.987+0000 7efd21786740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceaaaa1680 0x55ceaaab8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.987+0000 7efd21786740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ceaaaaa420 0x55ceaaab94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.987+0000 7efd21786740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ceaaaaae10 0x55ceaaaba500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.987+0000 7efd21786740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efd181a6bc0 con 0x55ceaaaa1680 2024-06-10T01:04:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.987+0000 7efd21786740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efd181a8030 con 0x55ceaaaaae10 2024-06-10T01:04:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.987+0000 7efd21786740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efd181a7290 con 0x55ceaaaaa420 2024-06-10T01:04:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.987+0000 7efd21786740 10 client.0 _ll_register_callbacks cb 0x7efd0000df10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.987+0000 7efd17fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceaaaa1680 0x55ceaaab8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.987+0000 7efd17fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceaaaa1680 0x55ceaaab8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40030/0 (socket says 172.21.15.37:40030) 2024-06-10T01:04:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.987+0000 7efd17fff640 1 -- 172.21.15.37:0/3764411652 learned_addr learned my addr 172.21.15.37:0/3764411652 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.987+0000 7efd1cf3b640 1 --2- 172.21.15.37:0/3764411652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ceaaaaae10 0x55ceaaaba500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.987+0000 7efd0ffff640 1 --2- 172.21.15.37:0/3764411652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ceaaaaa420 0x55ceaaab94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.987+0000 7efd17fff640 1 -- 172.21.15.37:0/3764411652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ceaaaaa420 msgr2=0x55ceaaab94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.987+0000 7efd17fff640 1 --2- 172.21.15.37:0/3764411652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ceaaaaa420 0x55ceaaab94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.987+0000 7efd17fff640 1 -- 172.21.15.37:0/3764411652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ceaaaaae10 msgr2=0x55ceaaaba500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.987+0000 7efd17fff640 1 --2- 172.21.15.37:0/3764411652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ceaaaaae10 0x55ceaaaba500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.987+0000 7efd17fff640 1 -- 172.21.15.37:0/3764411652 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ceaaabaaf0 con 0x55ceaaaa1680 2024-06-10T01:04:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.987+0000 7efd17fff640 1 --2- 172.21.15.37:0/3764411652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceaaaa1680 0x55ceaaab8fb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7efd0802a930 tx=0x7efd08032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b12e67bdf696773 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.988+0000 7efcf77fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:04:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.988+0000 7efcf77fe640 1 -- 172.21.15.37:0/3764411652 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0803f040 con 0x55ceaaaa1680 2024-06-10T01:04:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.988+0000 7efd21786740 1 -- 172.21.15.37:0/3764411652 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd00015020 con 0x55ceaaaa1680 2024-06-10T01:04:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.988+0000 7efcf77fe640 1 -- 172.21.15.37:0/3764411652 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7efd08004240 con 0x55ceaaaa1680 2024-06-10T01:04:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.988+0000 7efcf77fe640 1 -- 172.21.15.37:0/3764411652 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efd0805c050 con 0x55ceaaaa1680 2024-06-10T01:04:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.988+0000 7efd21786740 10 client.16722 fetch_fsmap learned FSMap version 29 2024-06-10T01:04:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.988+0000 7efd21786740 1 -- 172.21.15.37:0/3764411652 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ceaaabbe70 con 0x55ceaaaa1680 2024-06-10T01:04:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.988+0000 7efcf77fe640 1 -- 172.21.15.37:0/3764411652 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd00015020 con 0x55ceaaaa1680 2024-06-10T01:04:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.988+0000 7efcf77fe640 1 -- 172.21.15.37:0/3764411652 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7efd08004ac0 con 0x55ceaaaa1680 2024-06-10T01:04:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.988+0000 7efd21786740 10 client.16722 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:04:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.988+0000 7efd21786740 10 client.16722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:04:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.988+0000 7efd21786740 20 client.16722 populate_metadata read hostname 'smithi037' 2024-06-10T01:04:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.988+0000 7efd21786740 1 --2- 172.21.15.37:0/3764411652 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55ceaaaaff80 0x55ceaab2d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.988+0000 7efd21786740 4 client.16722 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:04:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.988+0000 7efd21786740 1 -- 172.21.15.37:0/3764411652 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ceaaab3b00 con 0x55ceaaaaff80 2024-06-10T01:04:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.988+0000 7efd0ffff640 1 --2- 172.21.15.37:0/3764411652 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55ceaaaaff80 0x55ceaab2d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.989+0000 7efcf77fe640 10 client.16722 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:16.989+0000 7efd0ffff640 1 --2- 172.21.15.37:0/3764411652 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55ceaaaaff80 0x55ceaab2d4e0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6262437238ec821d server_cookie=15ba519a9f9dfe19 in_seq=0 out_seq=0 2024-06-10T01:04:17.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.785+0000 7f83141c8700 1 -- 172.21.15.37:0/439403917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f830c06b560 msgr2=0x7f830c10f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:17.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.785+0000 7f83141c8700 1 --2- 172.21.15.37:0/439403917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f830c06b560 0x7f830c10f330 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f830400b0d0 tx=0x7f830400b3e0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.785+0000 7f83141c8700 1 -- 172.21.15.37:0/439403917 shutdown_connections 2024-06-10T01:04:17.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.785+0000 7f83141c8700 1 --2- 172.21.15.37:0/439403917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f830c06b560 0x7f830c10f330 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.785+0000 7f83141c8700 1 --2- 172.21.15.37:0/439403917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f830c108560 0x7f830c108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.785+0000 7f83141c8700 1 --2- 172.21.15.37:0/439403917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f830c06aba0 0x7f830c06b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.785+0000 7f83141c8700 1 -- 172.21.15.37:0/439403917 >> 172.21.15.37:0/439403917 conn(0x7f830c0780f0 msgr2=0x7f830c078500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:17.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.785+0000 7f83141c8700 1 -- 172.21.15.37:0/439403917 shutdown_connections 2024-06-10T01:04:17.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.785+0000 7f83141c8700 1 -- 172.21.15.37:0/439403917 wait complete. 2024-06-10T01:04:17.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.786+0000 7f83141c8700 1 Processor -- start 2024-06-10T01:04:17.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.786+0000 7f83141c8700 1 -- start start 2024-06-10T01:04:17.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.786+0000 7f83141c8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f830c06aba0 0x7f830c19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:17.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.786+0000 7f83141c8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f830c06b560 0x7f830c19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:17.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.786+0000 7f83141c8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f830c108560 0x7f830c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:17.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.786+0000 7f83141c8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f830c1127f0 con 0x7f830c06b560 2024-06-10T01:04:17.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.786+0000 7f83141c8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f830c112670 con 0x7f830c108560 2024-06-10T01:04:17.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.786+0000 7f83141c8700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f830c112970 con 0x7f830c06aba0 2024-06-10T01:04:17.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.786+0000 7f8311763700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f830c06b560 0x7f830c19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:17.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.786+0000 7f8311763700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f830c06b560 0x7f830c19ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40058/0 (socket says 172.21.15.37:40058) 2024-06-10T01:04:17.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.786+0000 7f8311f64700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f830c06aba0 0x7f830c19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:17.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.786+0000 7f8311763700 1 -- 172.21.15.37:0/3650872808 learned_addr learned my addr 172.21.15.37:0/3650872808 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:17.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.786+0000 7f8312765700 1 --2- 172.21.15.37:0/3650872808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f830c108560 0x7f830c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:17.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.787+0000 7f8311763700 1 -- 172.21.15.37:0/3650872808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f830c06aba0 msgr2=0x7f830c19e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:17.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.787+0000 7f8311763700 1 --2- 172.21.15.37:0/3650872808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f830c06aba0 0x7f830c19e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.787+0000 7f8311763700 1 -- 172.21.15.37:0/3650872808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f830c108560 msgr2=0x7f830c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:17.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.787+0000 7f8311763700 1 --2- 172.21.15.37:0/3650872808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f830c108560 0x7f830c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.787+0000 7f8311763700 1 -- 172.21.15.37:0/3650872808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830c199110 con 0x7f830c06b560 2024-06-10T01:04:17.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.787+0000 7f8311763700 1 --2- 172.21.15.37:0/3650872808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f830c06b560 0x7f830c19ece0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00d950 tx=0x7f82fc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:17.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.787+0000 7f8302ffd700 1 -- 172.21.15.37:0/3650872808 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc016070 con 0x7f830c06b560 2024-06-10T01:04:17.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.787+0000 7f83141c8700 1 -- 172.21.15.37:0/3650872808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f830c199400 con 0x7f830c06b560 2024-06-10T01:04:17.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.787+0000 7f83141c8700 1 -- 172.21.15.37:0/3650872808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f830c1a5670 con 0x7f830c06b560 2024-06-10T01:04:17.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.787+0000 7f8302ffd700 1 -- 172.21.15.37:0/3650872808 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82fc00fd70 con 0x7f830c06b560 2024-06-10T01:04:17.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.787+0000 7f8302ffd700 1 -- 172.21.15.37:0/3650872808 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc020060 con 0x7f830c06b560 2024-06-10T01:04:17.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.789+0000 7f8302ffd700 1 -- 172.21.15.37:0/3650872808 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f82fc019070 con 0x7f830c06b560 2024-06-10T01:04:17.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.790+0000 7f8302ffd700 1 --2- 172.21.15.37:0/3650872808 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f82f80a2ca0 0x7f82f80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:17.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.790+0000 7f8302ffd700 1 -- 172.21.15.37:0/3650872808 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f82fc0ced70 con 0x7f830c06b560 2024-06-10T01:04:17.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.790+0000 7f8311f64700 1 --2- 172.21.15.37:0/3650872808 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f82f80a2ca0 0x7f82f80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:17.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.790+0000 7f8311f64700 1 --2- 172.21.15.37:0/3650872808 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f82f80a2ca0 0x7f82f80a5160 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f830c0ffe20 tx=0x7f8308009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:17.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.791+0000 7f831ea4bb80 1 Processor -- start 2024-06-10T01:04:17.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.791+0000 7f831ea4bb80 1 -- start start 2024-06-10T01:04:17.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.791+0000 7f831ea4bb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab80bdf640 0x55ab80bdfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:17.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.791+0000 7f831ea4bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab80bdff60 0x55ab80bd99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:17.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.791+0000 7f831ea4bb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab80bd9f20 0x55ab80bda340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:17.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.791+0000 7f831ea4bb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ab80b2b810 con 0x55ab80bd9f20 2024-06-10T01:04:17.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.791+0000 7f831ea4bb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ab80ad3720 con 0x55ab80bdf640 2024-06-10T01:04:17.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.791+0000 7f831ea4bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ab80b277d0 con 0x55ab80bdff60 2024-06-10T01:04:17.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.791+0000 7f8312765700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab80bd9f20 0x55ab80bda340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:17.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.791+0000 7f8312765700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab80bd9f20 0x55ab80bda340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40062/0 (socket says 172.21.15.37:40062) 2024-06-10T01:04:17.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.791+0000 7f8312765700 1 -- 172.21.15.37:0/1786479432 learned_addr learned my addr 172.21.15.37:0/1786479432 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:17.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.791+0000 7f8311763700 1 --2- 172.21.15.37:0/1786479432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab80bdff60 0x55ab80bd99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:17.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.791+0000 7f8311f64700 1 --2- 172.21.15.37:0/1786479432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab80bdf640 0x55ab80bdfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:17.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.791+0000 7f8312765700 1 -- 172.21.15.37:0/1786479432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab80bdff60 msgr2=0x55ab80bd99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:17.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.791+0000 7f8312765700 1 --2- 172.21.15.37:0/1786479432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab80bdff60 0x55ab80bd99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.791+0000 7f8312765700 1 -- 172.21.15.37:0/1786479432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab80bdf640 msgr2=0x55ab80bdfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:17.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.791+0000 7f8312765700 1 --2- 172.21.15.37:0/1786479432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab80bdf640 0x55ab80bdfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.792+0000 7f8312765700 1 -- 172.21.15.37:0/1786479432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab80ac9f50 con 0x55ab80bd9f20 2024-06-10T01:04:17.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.792+0000 7f8311763700 1 --2- 172.21.15.37:0/1786479432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab80bdff60 0x55ab80bd99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:17.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.792+0000 7f8311f64700 1 --2- 172.21.15.37:0/1786479432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab80bdf640 0x55ab80bdfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:17.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.792+0000 7f8312765700 1 --2- 172.21.15.37:0/1786479432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab80bd9f20 0x55ab80bda340 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8304009d50 tx=0x7f8304009d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d7864b4e0578444 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:17.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.792+0000 7f82f7fff700 1 -- 172.21.15.37:0/1786479432 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304009460 con 0x55ab80bd9f20 2024-06-10T01:04:17.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.792+0000 7f82f7fff700 1 -- 172.21.15.37:0/1786479432 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8304003a00 con 0x55ab80bd9f20 2024-06-10T01:04:17.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.792+0000 7f82f7fff700 1 -- 172.21.15.37:0/1786479432 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304005820 con 0x55ab80bd9f20 2024-06-10T01:04:17.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.793+0000 7f831ea4bb80 1 -- 172.21.15.37:0/1786479432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab80bd9f20 msgr2=0x55ab80bda340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:17.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.793+0000 7f831ea4bb80 1 --2- 172.21.15.37:0/1786479432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab80bd9f20 0x55ab80bda340 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8304009d50 tx=0x7f8304009d80 comp rx=0 tx=0).stop 2024-06-10T01:04:17.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.793+0000 7f831ea4bb80 1 -- 172.21.15.37:0/1786479432 shutdown_connections 2024-06-10T01:04:17.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.793+0000 7f831ea4bb80 1 --2- 172.21.15.37:0/1786479432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab80bd9f20 0x55ab80bda340 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.793+0000 7f831ea4bb80 1 --2- 172.21.15.37:0/1786479432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab80bdf640 0x55ab80bdfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.793+0000 7f831ea4bb80 1 --2- 172.21.15.37:0/1786479432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab80bdff60 0x55ab80bd99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.793+0000 7f831ea4bb80 1 -- 172.21.15.37:0/1786479432 >> 172.21.15.37:0/1786479432 conn(0x55ab80b04cc0 msgr2=0x55ab80bd4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:17.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.793+0000 7f831ea4bb80 1 -- 172.21.15.37:0/1786479432 shutdown_connections 2024-06-10T01:04:17.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.793+0000 7f831ea4bb80 1 -- 172.21.15.37:0/1786479432 wait complete. 2024-06-10T01:04:17.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.793+0000 7f831ea4bb80 1 -- start start 2024-06-10T01:04:17.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.793+0000 7f831ea4bb80 1 Processor -- start 2024-06-10T01:04:17.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.794+0000 7f831ea4bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab80bd9f20 0x55ab80be5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:17.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.794+0000 7f831ea4bb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab80bdf640 0x55ab80be2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:17.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.794+0000 7f831ea4bb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab80bdff60 0x55ab80be3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:17.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.794+0000 7f831ea4bb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ab80be6970 con 0x55ab80bdf640 2024-06-10T01:04:17.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.794+0000 7f831ea4bb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ab80acbbf0 con 0x55ab80bdff60 2024-06-10T01:04:17.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.794+0000 7f831ea4bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ab80be6af0 con 0x55ab80bd9f20 2024-06-10T01:04:17.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.794+0000 7f831ea4bb80 10 client.0 _ll_register_callbacks cb 0x55ab80ac5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:17.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.794+0000 7f8311763700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab80bdf640 0x55ab80be2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:17.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.794+0000 7f8311f64700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab80bd9f20 0x55ab80be5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:17.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.794+0000 7f8311f64700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab80bd9f20 0x55ab80be5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:33632/0 (socket says 172.21.15.37:33632) 2024-06-10T01:04:17.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.794+0000 7f8311763700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab80bdf640 0x55ab80be2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40078/0 (socket says 172.21.15.37:40078) 2024-06-10T01:04:17.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.794+0000 7f8311f64700 1 -- 172.21.15.37:0/3220154765 learned_addr learned my addr 172.21.15.37:0/3220154765 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:17.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.794+0000 7f8312765700 1 --2- 172.21.15.37:0/3220154765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab80bdff60 0x55ab80be3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:17.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.794+0000 7f8311763700 1 -- 172.21.15.37:0/3220154765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab80bd9f20 msgr2=0x55ab80be5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:17.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.794+0000 7f8311763700 1 --2- 172.21.15.37:0/3220154765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab80bd9f20 0x55ab80be5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.794+0000 7f8311763700 1 -- 172.21.15.37:0/3220154765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab80bdff60 msgr2=0x55ab80be3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:17.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.794+0000 7f8311763700 1 --2- 172.21.15.37:0/3220154765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab80bdff60 0x55ab80be3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.794+0000 7f8311763700 1 -- 172.21.15.37:0/3220154765 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab80a7a8d0 con 0x55ab80bdf640 2024-06-10T01:04:17.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.794+0000 7f8311763700 1 --2- 172.21.15.37:0/3220154765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab80bdf640 0x55ab80be2fe0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f82fc010c00 tx=0x7f82fc09b250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96293ca201f4d266 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:17.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.794+0000 7f82f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:04:17.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.795+0000 7f82f57fa700 1 -- 172.21.15.37:0/3220154765 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc0b4060 con 0x55ab80bdf640 2024-06-10T01:04:17.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.795+0000 7f82f57fa700 1 -- 172.21.15.37:0/3220154765 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f82fc09b440 con 0x55ab80bdf640 2024-06-10T01:04:17.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.795+0000 7f831ea4bb80 1 -- 172.21.15.37:0/3220154765 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8304003a00 con 0x55ab80bdf640 2024-06-10T01:04:17.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.795+0000 7f82f57fa700 1 -- 172.21.15.37:0/3220154765 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f82fc09f350 con 0x55ab80bdf640 2024-06-10T01:04:17.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.795+0000 7f831ea4bb80 10 client.16734 fetch_fsmap learned FSMap version 29 2024-06-10T01:04:17.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.795+0000 7f831ea4bb80 1 -- 172.21.15.37:0/3220154765 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ab80bf12f0 con 0x55ab80bdf640 2024-06-10T01:04:17.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.795+0000 7f82f57fa700 1 -- 172.21.15.37:0/3220154765 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8304003a00 con 0x55ab80bdf640 2024-06-10T01:04:17.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.795+0000 7f82f57fa700 1 -- 172.21.15.37:0/3220154765 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f82fc09a630 con 0x55ab80bdf640 2024-06-10T01:04:17.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.795+0000 7f831ea4bb80 10 client.16734 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:04:17.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.795+0000 7f831ea4bb80 10 client.16734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:04:17.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.795+0000 7f831ea4bb80 20 client.16734 populate_metadata read hostname 'smithi037' 2024-06-10T01:04:17.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.795+0000 7f831ea4bb80 1 --2- 172.21.15.37:0/3220154765 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55ab80bf3930 0x55ab80bf5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:17.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.795+0000 7f831ea4bb80 4 client.16734 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:04:17.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.795+0000 7f831ea4bb80 1 -- 172.21.15.37:0/3220154765 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab80b38d10 con 0x55ab80bf3930 2024-06-10T01:04:17.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.796+0000 7f8311f64700 1 --2- 172.21.15.37:0/3220154765 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55ab80bf3930 0x55ab80bf5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:17.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.796+0000 7f8311f64700 1 --2- 172.21.15.37:0/3220154765 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55ab80bf3930 0x55ab80bf5d20 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=902a09f42f411bf3 server_cookie=3fbf28e2bf2ebf4e in_seq=0 out_seq=0 2024-06-10T01:04:17.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.796+0000 7f82f57fa700 10 client.16734 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:17.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.826+0000 7f82f57fa700 1 -- 172.21.15.37:0/3220154765 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ab80b38d10 con 0x55ab80bf3930 2024-06-10T01:04:17.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.826+0000 7f82f57fa700 10 client.16734 handle_command_reply: tid=0 2024-06-10T01:04:17.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.826+0000 7f831ea4bb80 1 client.16734 shutdown 2024-06-10T01:04:17.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.826+0000 7f831ea4bb80 1 -- 172.21.15.37:0/3220154765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab80bdf640 msgr2=0x55ab80be2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:17.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.826+0000 7f831ea4bb80 1 --2- 172.21.15.37:0/3220154765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab80bdf640 0x55ab80be2fe0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f82fc010c00 tx=0x7f82fc09b250 comp rx=0 tx=0).stop 2024-06-10T01:04:17.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.826+0000 7f831ea4bb80 1 -- 172.21.15.37:0/3220154765 shutdown_connections 2024-06-10T01:04:17.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.826+0000 7f831ea4bb80 1 --2- 172.21.15.37:0/3220154765 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55ab80bf3930 0x55ab80bf5d20 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.826+0000 7f831ea4bb80 1 --2- 172.21.15.37:0/3220154765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab80bdff60 0x55ab80be3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.826+0000 7f831ea4bb80 1 --2- 172.21.15.37:0/3220154765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab80bd9f20 0x55ab80be5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.826+0000 7f831ea4bb80 1 --2- 172.21.15.37:0/3220154765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab80bdf640 0x55ab80be2fe0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.826+0000 7f82f57fa700 0 client.16734 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:17.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.826+0000 7f831ea4bb80 1 -- 172.21.15.37:0/3220154765 >> 172.21.15.37:0/3220154765 conn(0x55ab80b04cc0 msgr2=0x55ab80bd89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:17.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.826+0000 7f831ea4bb80 1 -- 172.21.15.37:0/3220154765 shutdown_connections 2024-06-10T01:04:17.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.826+0000 7f831ea4bb80 1 -- 172.21.15.37:0/3220154765 wait complete. 2024-06-10T01:04:17.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.826+0000 7f831ea4bb80 20 client.16734 trim_cache size 0 max 16384 2024-06-10T01:04:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:17.836+0000 7efcf77fe640 1 -- 172.21.15.37:0/3764411652 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ceaaab3b00 con 0x55ceaaaaff80 2024-06-10T01:04:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:17.836+0000 7efcf77fe640 10 client.16722 handle_command_reply: tid=0 2024-06-10T01:04:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:17.836+0000 7efd21786740 1 client.16722 shutdown 2024-06-10T01:04:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:17.836+0000 7efd21786740 1 -- 172.21.15.37:0/3764411652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceaaaa1680 msgr2=0x55ceaaab8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:17.836+0000 7efd21786740 1 --2- 172.21.15.37:0/3764411652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceaaaa1680 0x55ceaaab8fb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7efd0802a930 tx=0x7efd08032fc0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:17.836+0000 7efd21786740 1 -- 172.21.15.37:0/3764411652 shutdown_connections 2024-06-10T01:04:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:17.836+0000 7efd21786740 1 --2- 172.21.15.37:0/3764411652 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55ceaaaaff80 0x55ceaab2d4e0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:17.836+0000 7efd21786740 1 --2- 172.21.15.37:0/3764411652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ceaaaaae10 0x55ceaaaba500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:17.836+0000 7efd21786740 1 --2- 172.21.15.37:0/3764411652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ceaaaaa420 0x55ceaaab94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:17.836+0000 7efd21786740 1 --2- 172.21.15.37:0/3764411652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceaaaa1680 0x55ceaaab8fb0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:17.836+0000 7efd21786740 1 -- 172.21.15.37:0/3764411652 >> 172.21.15.37:0/3764411652 conn(0x55ceaaaafa80 msgr2=0x55ceaaaa4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:17.836+0000 7efcf77fe640 0 client.16722 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:17.836+0000 7efd21786740 1 -- 172.21.15.37:0/3764411652 shutdown_connections 2024-06-10T01:04:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:17.836+0000 7efd21786740 1 -- 172.21.15.37:0/3764411652 wait complete. 2024-06-10T01:04:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:17.837+0000 7efd21786740 20 client.16722 trim_cache size 0 max 16384 2024-06-10T01:04:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:04:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:04:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:04:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:17.837+0000 7efd1e99e640 1 -- 172.21.15.37:0/2959440504 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7efd040a2c30 msgr2=0x7efd040a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:17.837+0000 7efd1e99e640 1 --2- 172.21.15.37:0/2959440504 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7efd040a2c30 0x7efd040a50f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x55ceaaaab670 tx=0x7efd1000a4d0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:17.837+0000 7efd1e99e640 1 -- 172.21.15.37:0/2959440504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd18104db0 msgr2=0x7efd18198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:17.837+0000 7efd1e99e640 1 --2- 172.21.15.37:0/2959440504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd18104db0 0x7efd18198200 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7efd00005fd0 tx=0x7efd0000dde0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:17.837+0000 7efd1e99e640 1 -- 172.21.15.37:0/2959440504 shutdown_connections 2024-06-10T01:04:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:17.837+0000 7efd1e99e640 1 --2- 172.21.15.37:0/2959440504 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7efd040a2c30 0x7efd040a50f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:17.837+0000 7efd1e99e640 1 --2- 172.21.15.37:0/2959440504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd181076e0 0x7efd1819f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:17.837+0000 7efd1e99e640 1 --2- 172.21.15.37:0/2959440504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd18104db0 0x7efd18198200 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:17.837+0000 7efd1e99e640 1 --2- 172.21.15.37:0/2959440504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd18102480 0x7efd18197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:17.837+0000 7efd1e99e640 1 -- 172.21.15.37:0/2959440504 >> 172.21.15.37:0/2959440504 conn(0x7efd180fbe70 msgr2=0x7efd18102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:17.838+0000 7efd1e99e640 1 -- 172.21.15.37:0/2959440504 shutdown_connections 2024-06-10T01:04:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:17.838+0000 7efd1e99e640 1 -- 172.21.15.37:0/2959440504 wait complete. 2024-06-10T01:04:17.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.843+0000 7f831ea4bb80 1 Processor -- start 2024-06-10T01:04:17.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.843+0000 7f831ea4bb80 1 -- start start 2024-06-10T01:04:17.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.843+0000 7f831ea4bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab80bd9f20 0x55ab80bd99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:17.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.844+0000 7f831ea4bb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab80bdf640 0x55ab80be0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:17.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.844+0000 7f831ea4bb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab80bdff60 0x55ab80be0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:17.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.844+0000 7f831ea4bb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f82f0011510 con 0x55ab80bdff60 2024-06-10T01:04:17.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.844+0000 7f831ea4bb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f82f000eda0 con 0x55ab80bdf640 2024-06-10T01:04:17.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.844+0000 7f831ea4bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f82f000cbf0 con 0x55ab80bd9f20 2024-06-10T01:04:17.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.844+0000 7f8312765700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab80bdff60 0x55ab80be0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:17.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.844+0000 7f8312765700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab80bdff60 0x55ab80be0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40094/0 (socket says 172.21.15.37:40094) 2024-06-10T01:04:17.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.844+0000 7f8312765700 1 -- 172.21.15.37:0/889139762 learned_addr learned my addr 172.21.15.37:0/889139762 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:17.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.844+0000 7f8311f64700 1 --2- 172.21.15.37:0/889139762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab80bd9f20 0x55ab80bd99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:17.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.844+0000 7f8311763700 1 --2- 172.21.15.37:0/889139762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab80bdf640 0x55ab80be0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:17.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.844+0000 7f8312765700 1 -- 172.21.15.37:0/889139762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab80bd9f20 msgr2=0x55ab80bd99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:17.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.844+0000 7f8312765700 1 --2- 172.21.15.37:0/889139762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab80bd9f20 0x55ab80bd99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.844+0000 7f8312765700 1 -- 172.21.15.37:0/889139762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab80bdf640 msgr2=0x55ab80be0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:17.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.844+0000 7f8312765700 1 --2- 172.21.15.37:0/889139762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab80bdf640 0x55ab80be0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.844+0000 7f8312765700 1 -- 172.21.15.37:0/889139762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab80bf63a0 con 0x55ab80bdff60 2024-06-10T01:04:17.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.844+0000 7f8312765700 1 --2- 172.21.15.37:0/889139762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab80bdff60 0x55ab80be0940 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f8304005390 tx=0x7f83040053c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569d11536947646 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:17.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.845+0000 7f82f77fe700 1 -- 172.21.15.37:0/889139762 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830400baa0 con 0x55ab80bdff60 2024-06-10T01:04:17.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.845+0000 7f82f77fe700 1 -- 172.21.15.37:0/889139762 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f830400bc40 con 0x55ab80bdff60 2024-06-10T01:04:17.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.845+0000 7f82f77fe700 1 -- 172.21.15.37:0/889139762 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304007700 con 0x55ab80bdff60 2024-06-10T01:04:17.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.845+0000 7f831ea4bb80 1 -- 172.21.15.37:0/889139762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab80bdff60 msgr2=0x55ab80be0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:17.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.845+0000 7f831ea4bb80 1 --2- 172.21.15.37:0/889139762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab80bdff60 0x55ab80be0940 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f8304005390 tx=0x7f83040053c0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.845+0000 7f831ea4bb80 1 -- 172.21.15.37:0/889139762 shutdown_connections 2024-06-10T01:04:17.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.845+0000 7f831ea4bb80 1 --2- 172.21.15.37:0/889139762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab80bdff60 0x55ab80be0940 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.845+0000 7f831ea4bb80 1 --2- 172.21.15.37:0/889139762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab80bd9f20 0x55ab80bd99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.845+0000 7f831ea4bb80 1 --2- 172.21.15.37:0/889139762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab80bdf640 0x55ab80be0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.845+0000 7f831ea4bb80 1 -- 172.21.15.37:0/889139762 >> 172.21.15.37:0/889139762 conn(0x55ab80b04cc0 msgr2=0x55ab80be18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:17.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.845+0000 7f831ea4bb80 1 -- 172.21.15.37:0/889139762 shutdown_connections 2024-06-10T01:04:17.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.846+0000 7f831ea4bb80 1 -- 172.21.15.37:0/889139762 wait complete. 2024-06-10T01:04:17.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.846+0000 7f831ea4bb80 1 -- start start 2024-06-10T01:04:17.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.846+0000 7f831ea4bb80 1 Processor -- start 2024-06-10T01:04:17.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.846+0000 7f831ea4bb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab80bd9f20 0x55ab80bebe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:17.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.846+0000 7f831ea4bb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab80bdf640 0x55ab80bec350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:17.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.846+0000 7f831ea4bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab80bdff60 0x55ab80be8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:17.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.846+0000 7f831ea4bb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f82f000d260 con 0x55ab80bdf640 2024-06-10T01:04:17.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.846+0000 7f831ea4bb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f82f000e6d0 con 0x55ab80bd9f20 2024-06-10T01:04:17.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.846+0000 7f831ea4bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f82f000d930 con 0x55ab80bdff60 2024-06-10T01:04:17.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.846+0000 7f831ea4bb80 10 client.0 _ll_register_callbacks cb 0x55ab80ac5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:17.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.847+0000 7f8311f64700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab80bd9f20 0x55ab80bebe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:17.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.847+0000 7f8311763700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab80bdf640 0x55ab80bec350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:17.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.847+0000 7f8311f64700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab80bd9f20 0x55ab80bebe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:57092/0 (socket says 172.21.15.37:57092) 2024-06-10T01:04:17.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.847+0000 7f8311f64700 1 -- 172.21.15.37:0/1688720895 learned_addr learned my addr 172.21.15.37:0/1688720895 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:17.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.847+0000 7f8312765700 1 --2- 172.21.15.37:0/1688720895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab80bdff60 0x55ab80be8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:17.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.847+0000 7f8311f64700 1 -- 172.21.15.37:0/1688720895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab80bdff60 msgr2=0x55ab80be8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:17.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.847+0000 7f8311f64700 1 --2- 172.21.15.37:0/1688720895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab80bdff60 0x55ab80be8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.847+0000 7f8311f64700 1 -- 172.21.15.37:0/1688720895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab80bdf640 msgr2=0x55ab80bec350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:17.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.847+0000 7f8311f64700 1 --2- 172.21.15.37:0/1688720895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab80bdf640 0x55ab80bec350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.847+0000 7f8311f64700 1 -- 172.21.15.37:0/1688720895 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab80becaa0 con 0x55ab80bd9f20 2024-06-10T01:04:17.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.847+0000 7f8311763700 1 --2- 172.21.15.37:0/1688720895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab80bdf640 0x55ab80bec350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:17.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.847+0000 7f8311f64700 1 --2- 172.21.15.37:0/1688720895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab80bd9f20 0x55ab80bebe10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x55ab80bf65a0 tx=0x7f83080047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d65fe05b07aecbb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:17.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.847+0000 7f82f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:04:17.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.847+0000 7f82f57fa700 1 -- 172.21.15.37:0/1688720895 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8308038060 con 0x55ab80bd9f20 2024-06-10T01:04:17.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.847+0000 7f831ea4bb80 1 -- 172.21.15.37:0/1688720895 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f830400bc40 con 0x55ab80bd9f20 2024-06-10T01:04:17.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.847+0000 7f82f57fa700 1 -- 172.21.15.37:0/1688720895 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f830801dbf0 con 0x55ab80bd9f20 2024-06-10T01:04:17.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.848+0000 7f82f57fa700 1 -- 172.21.15.37:0/1688720895 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f830803f050 con 0x55ab80bd9f20 2024-06-10T01:04:17.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.848+0000 7f831ea4bb80 10 client.26180 fetch_fsmap learned FSMap version 29 2024-06-10T01:04:17.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.848+0000 7f831ea4bb80 1 -- 172.21.15.37:0/1688720895 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ab80c5ecd0 con 0x55ab80bd9f20 2024-06-10T01:04:17.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.848+0000 7f82f57fa700 1 -- 172.21.15.37:0/1688720895 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f830400bc40 con 0x55ab80bd9f20 2024-06-10T01:04:17.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.848+0000 7f82f57fa700 1 -- 172.21.15.37:0/1688720895 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f8308033540 con 0x55ab80bd9f20 2024-06-10T01:04:17.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.848+0000 7f831ea4bb80 10 client.26180 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:04:17.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.848+0000 7f831ea4bb80 10 client.26180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:04:17.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.848+0000 7f831ea4bb80 20 client.26180 populate_metadata read hostname 'smithi037' 2024-06-10T01:04:17.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.848+0000 7f831ea4bb80 1 --2- 172.21.15.37:0/1688720895 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55ab80c5f390 0x55ab80c617b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:17.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.848+0000 7f831ea4bb80 4 client.26180 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:04:17.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.848+0000 7f831ea4bb80 1 -- 172.21.15.37:0/1688720895 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab80b33290 con 0x55ab80c5f390 2024-06-10T01:04:17.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.848+0000 7f8311763700 1 --2- 172.21.15.37:0/1688720895 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55ab80c5f390 0x55ab80c617b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:17.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.849+0000 7f82f57fa700 10 client.26180 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:17.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.849+0000 7f8311763700 1 --2- 172.21.15.37:0/1688720895 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55ab80c5f390 0x55ab80c617b0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=949e602380fc58c0 server_cookie=5ffb3b4c4df8b2aa in_seq=0 out_seq=0 2024-06-10T01:04:17.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.849+0000 7f82f57fa700 1 -- 172.21.15.37:0/1688720895 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab80b33290 con 0x55ab80c5f390 2024-06-10T01:04:17.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.849+0000 7f82f57fa700 10 client.26180 handle_command_reply: tid=0 2024-06-10T01:04:17.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.849+0000 7f831ea4bb80 1 client.26180 shutdown 2024-06-10T01:04:17.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.849+0000 7f831ea4bb80 1 -- 172.21.15.37:0/1688720895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab80bd9f20 msgr2=0x55ab80bebe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:17.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.849+0000 7f831ea4bb80 1 --2- 172.21.15.37:0/1688720895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab80bd9f20 0x55ab80bebe10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x55ab80bf65a0 tx=0x7f83080047c0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.849+0000 7f831ea4bb80 1 -- 172.21.15.37:0/1688720895 shutdown_connections 2024-06-10T01:04:17.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.849+0000 7f831ea4bb80 1 --2- 172.21.15.37:0/1688720895 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55ab80c5f390 0x55ab80c617b0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.849+0000 7f831ea4bb80 1 --2- 172.21.15.37:0/1688720895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab80bdff60 0x55ab80be8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.849+0000 7f831ea4bb80 1 --2- 172.21.15.37:0/1688720895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ab80bd9f20 0x55ab80bebe10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.849+0000 7f82f57fa700 0 client.26180 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:17.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.849+0000 7f831ea4bb80 1 --2- 172.21.15.37:0/1688720895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab80bdf640 0x55ab80bec350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.849+0000 7f831ea4bb80 1 -- 172.21.15.37:0/1688720895 >> 172.21.15.37:0/1688720895 conn(0x55ab80b04cc0 msgr2=0x55ab80bcde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:17.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.849+0000 7f831ea4bb80 1 -- 172.21.15.37:0/1688720895 shutdown_connections 2024-06-10T01:04:17.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.849+0000 7f831ea4bb80 1 -- 172.21.15.37:0/1688720895 wait complete. 2024-06-10T01:04:17.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.849+0000 7f831ea4bb80 20 client.26180 trim_cache size 0 max 16384 2024-06-10T01:04:17.852 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T01:04:17.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.850+0000 7f83141c8700 1 -- 172.21.15.37:0/3650872808 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f82f80a2ca0 msgr2=0x7f82f80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:17.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.850+0000 7f83141c8700 1 --2- 172.21.15.37:0/3650872808 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f82f80a2ca0 0x7f82f80a5160 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f830c0ffe20 tx=0x7f8308009260 comp rx=0 tx=0).stop 2024-06-10T01:04:17.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.850+0000 7f83141c8700 1 -- 172.21.15.37:0/3650872808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f830c06b560 msgr2=0x7f830c19ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:17.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.850+0000 7f83141c8700 1 --2- 172.21.15.37:0/3650872808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f830c06b560 0x7f830c19ece0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00d950 tx=0x7f82fc00dc60 comp rx=0 tx=0).stop 2024-06-10T01:04:17.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.850+0000 7f83141c8700 1 -- 172.21.15.37:0/3650872808 shutdown_connections 2024-06-10T01:04:17.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.850+0000 7f83141c8700 1 --2- 172.21.15.37:0/3650872808 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f82f80a2ca0 0x7f82f80a5160 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.850+0000 7f83141c8700 1 --2- 172.21.15.37:0/3650872808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f830c108560 0x7f830c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.850+0000 7f83141c8700 1 --2- 172.21.15.37:0/3650872808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f830c06aba0 0x7f830c19e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.850+0000 7f83141c8700 1 --2- 172.21.15.37:0/3650872808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f830c06b560 0x7f830c19ece0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:17.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.850+0000 7f83141c8700 1 -- 172.21.15.37:0/3650872808 >> 172.21.15.37:0/3650872808 conn(0x7f830c0780f0 msgr2=0x7f830c100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:17.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.850+0000 7f83141c8700 1 -- 172.21.15.37:0/3650872808 shutdown_connections 2024-06-10T01:04:17.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:17.850+0000 7f83141c8700 1 -- 172.21.15.37:0/3650872808 wait complete. 2024-06-10T01:04:18.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:18 smithi167 ceph-mon[36917]: pgmap v659: 129 pgs: 129 active+clean; 11 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 813 KiB/s wr, 293 op/s 2024-06-10T01:04:18.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:18 smithi094 ceph-mon[36708]: pgmap v659: 129 pgs: 129 active+clean; 11 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 813 KiB/s wr, 293 op/s 2024-06-10T01:04:18.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:18 smithi037 ceph-mon[32192]: pgmap v659: 129 pgs: 129 active+clean; 11 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 813 KiB/s wr, 293 op/s 2024-06-10T01:04:19.782 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:04:19.782 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:04:20.050 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:20.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:20 smithi167 ceph-mon[36917]: pgmap v660: 129 pgs: 129 active+clean; 6.6 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 1.2 MiB/s wr, 626 op/s 2024-06-10T01:04:20.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:20 smithi094 ceph-mon[36708]: pgmap v660: 129 pgs: 129 active+clean; 6.6 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 1.2 MiB/s wr, 626 op/s 2024-06-10T01:04:20.374 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:20 smithi037 ceph-mon[32192]: pgmap v660: 129 pgs: 129 active+clean; 6.6 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 1.2 MiB/s wr, 626 op/s 2024-06-10T01:04:21.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.741+0000 7fb96358b700 1 -- 172.21.15.37:0/1881384556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb95c102380 msgr2=0x7fb95c105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:21.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.741+0000 7fb96358b700 1 --2- 172.21.15.37:0/1881384556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb95c102380 0x7fb95c105ff0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0099c0 tx=0x7fb94c009cd0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.741+0000 7fb96358b700 1 -- 172.21.15.37:0/1881384556 shutdown_connections 2024-06-10T01:04:21.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.741+0000 7fb96358b700 1 --2- 172.21.15.37:0/1881384556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb95c106720 0x7fb95c10e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.741+0000 7fb96358b700 1 --2- 172.21.15.37:0/1881384556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb95c1019d0 0x7fb95c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.741+0000 7fb96358b700 1 --2- 172.21.15.37:0/1881384556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb95c102380 0x7fb95c105ff0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.741+0000 7fb96358b700 1 -- 172.21.15.37:0/1881384556 >> 172.21.15.37:0/1881384556 conn(0x7fb95c078340 msgr2=0x7fb95c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:21.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.741+0000 7fb96358b700 1 -- 172.21.15.37:0/1881384556 shutdown_connections 2024-06-10T01:04:21.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.741+0000 7fb96358b700 1 -- 172.21.15.37:0/1881384556 wait complete. 2024-06-10T01:04:21.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.742+0000 7fb96358b700 1 Processor -- start 2024-06-10T01:04:21.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.742+0000 7fb96358b700 1 -- start start 2024-06-10T01:04:21.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.742+0000 7fb96358b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb95c1019d0 0x7fb95c19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:21.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.742+0000 7fb96358b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb95c102380 0x7fb95c19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:21.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.742+0000 7fb96358b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb95c106720 0x7fb95c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:21.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.742+0000 7fb96358b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c1127f0 con 0x7fb95c106720 2024-06-10T01:04:21.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.742+0000 7fb96358b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c112670 con 0x7fb95c102380 2024-06-10T01:04:21.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.742+0000 7fb96358b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c112970 con 0x7fb95c1019d0 2024-06-10T01:04:21.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.742+0000 7fb961b28700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb95c106720 0x7fb95c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:21.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.742+0000 7fb961b28700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb95c106720 0x7fb95c198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40134/0 (socket says 172.21.15.37:40134) 2024-06-10T01:04:21.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.742+0000 7fb961b28700 1 -- 172.21.15.37:0/686984925 learned_addr learned my addr 172.21.15.37:0/686984925 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:21.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.742+0000 7fb961327700 1 --2- 172.21.15.37:0/686984925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb95c1019d0 0x7fb95c19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:21.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.742+0000 7fb960b26700 1 --2- 172.21.15.37:0/686984925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb95c102380 0x7fb95c19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:21.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.743+0000 7fb961b28700 1 -- 172.21.15.37:0/686984925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb95c1019d0 msgr2=0x7fb95c19e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:21.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.743+0000 7fb961b28700 1 --2- 172.21.15.37:0/686984925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb95c1019d0 0x7fb95c19e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.743+0000 7fb961b28700 1 -- 172.21.15.37:0/686984925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb95c102380 msgr2=0x7fb95c19ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:21.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.743+0000 7fb961b28700 1 --2- 172.21.15.37:0/686984925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb95c102380 0x7fb95c19ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.743+0000 7fb961b28700 1 -- 172.21.15.37:0/686984925 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95c199250 con 0x7fb95c106720 2024-06-10T01:04:21.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.743+0000 7fb961b28700 1 --2- 172.21.15.37:0/686984925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb95c106720 0x7fb95c198af0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb954007ee0 tx=0x7fb95400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:21.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.743+0000 7fb9527fc700 1 -- 172.21.15.37:0/686984925 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb954004bb0 con 0x7fb95c106720 2024-06-10T01:04:21.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.743+0000 7fb9527fc700 1 -- 172.21.15.37:0/686984925 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb954004d50 con 0x7fb95c106720 2024-06-10T01:04:21.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.743+0000 7fb9527fc700 1 -- 172.21.15.37:0/686984925 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb954005960 con 0x7fb95c106720 2024-06-10T01:04:21.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.743+0000 7fb96358b700 1 -- 172.21.15.37:0/686984925 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb95c199540 con 0x7fb95c106720 2024-06-10T01:04:21.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.743+0000 7fb96358b700 1 -- 172.21.15.37:0/686984925 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb95c1a57b0 con 0x7fb95c106720 2024-06-10T01:04:21.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.746+0000 7fb9527fc700 1 -- 172.21.15.37:0/686984925 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb95401a020 con 0x7fb95c106720 2024-06-10T01:04:21.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.747+0000 7fb9527fc700 1 --2- 172.21.15.37:0/686984925 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb9480a2ca0 0x7fb9480a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:21.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.747+0000 7fb961327700 1 --2- 172.21.15.37:0/686984925 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb9480a2ca0 0x7fb9480a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:21.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.747+0000 7fb9527fc700 1 -- 172.21.15.37:0/686984925 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fb9540cea80 con 0x7fb95c106720 2024-06-10T01:04:21.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.748+0000 7fb96de0eb80 1 Processor -- start 2024-06-10T01:04:21.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.748+0000 7fb961327700 1 --2- 172.21.15.37:0/686984925 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb9480a2ca0 0x7fb9480a5160 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb95c1062a0 tx=0x7fb95800a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:21.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.748+0000 7fb96de0eb80 1 -- start start 2024-06-10T01:04:21.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.748+0000 7fb96de0eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56393f340af0 0x56393f340ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:21.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.748+0000 7fb96de0eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393f341410 0x56393f33ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:21.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.748+0000 7fb96de0eb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56393f33b3b0 0x56393f33b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:21.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.748+0000 7fb96de0eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56393f288890 con 0x56393f341410 2024-06-10T01:04:21.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.748+0000 7fb96de0eb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56393f2307a0 con 0x56393f33b3b0 2024-06-10T01:04:21.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.748+0000 7fb96de0eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56393f284850 con 0x56393f340af0 2024-06-10T01:04:21.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.748+0000 7fb960b26700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393f341410 0x56393f33ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:21.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.748+0000 7fb960b26700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393f341410 0x56393f33ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40138/0 (socket says 172.21.15.37:40138) 2024-06-10T01:04:21.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.748+0000 7fb960b26700 1 -- 172.21.15.37:0/930289183 learned_addr learned my addr 172.21.15.37:0/930289183 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:21.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.748+0000 7fb961327700 1 --2- 172.21.15.37:0/930289183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56393f340af0 0x56393f340ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:21.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.748+0000 7fb961b28700 1 --2- 172.21.15.37:0/930289183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56393f33b3b0 0x56393f33b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:21.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.749+0000 7fb960b26700 1 -- 172.21.15.37:0/930289183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56393f340af0 msgr2=0x56393f340ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:21.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.749+0000 7fb960b26700 1 --2- 172.21.15.37:0/930289183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56393f340af0 0x56393f340ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.749+0000 7fb960b26700 1 -- 172.21.15.37:0/930289183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56393f33b3b0 msgr2=0x56393f33b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:21.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.749+0000 7fb960b26700 1 --2- 172.21.15.37:0/930289183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56393f33b3b0 0x56393f33b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.749+0000 7fb960b26700 1 -- 172.21.15.37:0/930289183 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56393f226fd0 con 0x56393f341410 2024-06-10T01:04:21.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.749+0000 7fb960b26700 1 --2- 172.21.15.37:0/930289183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393f341410 0x56393f33ae70 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb95c076580 tx=0x7fb94c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c6e5cc471990d9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:21.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.749+0000 7fb9477fe700 1 -- 172.21.15.37:0/930289183 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c028070 con 0x56393f341410 2024-06-10T01:04:21.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.749+0000 7fb9477fe700 1 -- 172.21.15.37:0/930289183 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb94c003780 con 0x56393f341410 2024-06-10T01:04:21.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.749+0000 7fb9477fe700 1 -- 172.21.15.37:0/930289183 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c00bb10 con 0x56393f341410 2024-06-10T01:04:21.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.750+0000 7fb96de0eb80 1 -- 172.21.15.37:0/930289183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393f341410 msgr2=0x56393f33ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:21.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.750+0000 7fb96de0eb80 1 --2- 172.21.15.37:0/930289183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393f341410 0x56393f33ae70 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb95c076580 tx=0x7fb94c005c00 comp rx=0 tx=0).stop 2024-06-10T01:04:21.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.750+0000 7fb96de0eb80 1 -- 172.21.15.37:0/930289183 shutdown_connections 2024-06-10T01:04:21.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.750+0000 7fb96de0eb80 1 --2- 172.21.15.37:0/930289183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56393f33b3b0 0x56393f33b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.750+0000 7fb96de0eb80 1 --2- 172.21.15.37:0/930289183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56393f340af0 0x56393f340ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.750+0000 7fb96de0eb80 1 --2- 172.21.15.37:0/930289183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393f341410 0x56393f33ae70 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.750+0000 7fb96de0eb80 1 -- 172.21.15.37:0/930289183 >> 172.21.15.37:0/930289183 conn(0x56393f261d40 msgr2=0x56393f3318d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:21.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.750+0000 7fb96de0eb80 1 -- 172.21.15.37:0/930289183 shutdown_connections 2024-06-10T01:04:21.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.750+0000 7fb96de0eb80 1 -- 172.21.15.37:0/930289183 wait complete. 2024-06-10T01:04:21.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.750+0000 7fb96de0eb80 1 -- start start 2024-06-10T01:04:21.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.750+0000 7fb96de0eb80 1 Processor -- start 2024-06-10T01:04:21.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.751+0000 7fb96de0eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393f33b3b0 0x56393f33a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:21.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.751+0000 7fb96de0eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56393f340af0 0x56393f33ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:21.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.751+0000 7fb96de0eb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56393f341410 0x56393f333b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:21.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.751+0000 7fb96de0eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56393f347df0 con 0x56393f33b3b0 2024-06-10T01:04:21.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.751+0000 7fb96de0eb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56393f228c70 con 0x56393f341410 2024-06-10T01:04:21.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.751+0000 7fb96de0eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56393f347f70 con 0x56393f340af0 2024-06-10T01:04:21.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.751+0000 7fb96de0eb80 10 client.0 _ll_register_callbacks cb 0x56393f2229a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:21.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.751+0000 7fb961327700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393f33b3b0 0x56393f33a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:21.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.751+0000 7fb961327700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393f33b3b0 0x56393f33a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40140/0 (socket says 172.21.15.37:40140) 2024-06-10T01:04:21.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.751+0000 7fb961327700 1 -- 172.21.15.37:0/690710843 learned_addr learned my addr 172.21.15.37:0/690710843 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:21.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.751+0000 7fb961b28700 1 --2- 172.21.15.37:0/690710843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56393f341410 0x56393f333b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:21.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.751+0000 7fb960b26700 1 --2- 172.21.15.37:0/690710843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56393f340af0 0x56393f33ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:21.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.752+0000 7fb961327700 1 -- 172.21.15.37:0/690710843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56393f340af0 msgr2=0x56393f33ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:21.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.752+0000 7fb961327700 1 --2- 172.21.15.37:0/690710843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56393f340af0 0x56393f33ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.752+0000 7fb961327700 1 -- 172.21.15.37:0/690710843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56393f341410 msgr2=0x56393f333b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:21.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.752+0000 7fb961327700 1 --2- 172.21.15.37:0/690710843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56393f341410 0x56393f333b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.752+0000 7fb961327700 1 -- 172.21.15.37:0/690710843 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56393f1d7950 con 0x56393f33b3b0 2024-06-10T01:04:21.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.752+0000 7fb961b28700 1 --2- 172.21.15.37:0/690710843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56393f341410 0x56393f333b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:21.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.752+0000 7fb961327700 1 --2- 172.21.15.37:0/690710843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393f33b3b0 0x56393f33a7f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb958003930 tx=0x7fb9580047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cec8a7aa73dba59 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:21.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.752+0000 7fb944ff9700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:04:21.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.752+0000 7fb944ff9700 1 -- 172.21.15.37:0/690710843 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb958016070 con 0x56393f33b3b0 2024-06-10T01:04:21.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.752+0000 7fb944ff9700 1 -- 172.21.15.37:0/690710843 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fb958010840 con 0x56393f33b3b0 2024-06-10T01:04:21.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.752+0000 7fb96de0eb80 1 -- 172.21.15.37:0/690710843 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb94c003780 con 0x56393f33b3b0 2024-06-10T01:04:21.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.752+0000 7fb944ff9700 1 -- 172.21.15.37:0/690710843 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb958030050 con 0x56393f33b3b0 2024-06-10T01:04:21.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.752+0000 7fb96de0eb80 10 client.16752 fetch_fsmap learned FSMap version 29 2024-06-10T01:04:21.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.752+0000 7fb96de0eb80 1 -- 172.21.15.37:0/690710843 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56393f356c20 con 0x56393f33b3b0 2024-06-10T01:04:21.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.753+0000 7fb944ff9700 1 -- 172.21.15.37:0/690710843 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb94c003780 con 0x56393f33b3b0 2024-06-10T01:04:21.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.753+0000 7fb944ff9700 1 -- 172.21.15.37:0/690710843 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fb958027c20 con 0x56393f33b3b0 2024-06-10T01:04:21.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.753+0000 7fb96de0eb80 10 client.16752 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:04:21.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.753+0000 7fb96de0eb80 10 client.16752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:04:21.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.753+0000 7fb96de0eb80 20 client.16752 populate_metadata read hostname 'smithi037' 2024-06-10T01:04:21.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.753+0000 7fb96de0eb80 1 --2- 172.21.15.37:0/690710843 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56393f359260 0x56393f35b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:21.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.753+0000 7fb96de0eb80 4 client.16752 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:04:21.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.753+0000 7fb96de0eb80 1 -- 172.21.15.37:0/690710843 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56393f295d90 con 0x56393f359260 2024-06-10T01:04:21.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.753+0000 7fb960b26700 1 --2- 172.21.15.37:0/690710843 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56393f359260 0x56393f35b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:21.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.754+0000 7fb960b26700 1 --2- 172.21.15.37:0/690710843 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56393f359260 0x56393f35b650 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=482c41b3826b2904 server_cookie=4571aacebd445905 in_seq=0 out_seq=0 2024-06-10T01:04:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.754+0000 7fb944ff9700 10 client.16752 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.755+0000 7fb944ff9700 1 -- 172.21.15.37:0/690710843 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56393f295d90 con 0x56393f359260 2024-06-10T01:04:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.755+0000 7fb944ff9700 10 client.16752 handle_command_reply: tid=0 2024-06-10T01:04:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.755+0000 7fb96de0eb80 1 client.16752 shutdown 2024-06-10T01:04:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.755+0000 7fb96de0eb80 1 -- 172.21.15.37:0/690710843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393f33b3b0 msgr2=0x56393f33a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.755+0000 7fb96de0eb80 1 --2- 172.21.15.37:0/690710843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393f33b3b0 0x56393f33a7f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb958003930 tx=0x7fb9580047c0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.755+0000 7fb96de0eb80 1 -- 172.21.15.37:0/690710843 shutdown_connections 2024-06-10T01:04:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.755+0000 7fb96de0eb80 1 --2- 172.21.15.37:0/690710843 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56393f359260 0x56393f35b650 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.755+0000 7fb96de0eb80 1 --2- 172.21.15.37:0/690710843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56393f341410 0x56393f333b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.755+0000 7fb96de0eb80 1 --2- 172.21.15.37:0/690710843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393f33b3b0 0x56393f33a7f0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.755+0000 7fb96de0eb80 1 --2- 172.21.15.37:0/690710843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56393f340af0 0x56393f33ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.755+0000 7fb944ff9700 0 client.16752 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.755+0000 7fb96de0eb80 1 -- 172.21.15.37:0/690710843 >> 172.21.15.37:0/690710843 conn(0x56393f261d40 msgr2=0x56393f33e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.755+0000 7fb96de0eb80 1 -- 172.21.15.37:0/690710843 shutdown_connections 2024-06-10T01:04:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.755+0000 7fb96de0eb80 1 -- 172.21.15.37:0/690710843 wait complete. 2024-06-10T01:04:21.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.755+0000 7fb96de0eb80 20 client.16752 trim_cache size 0 max 16384 2024-06-10T01:04:21.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.777+0000 7fb96de0eb80 1 Processor -- start 2024-06-10T01:04:21.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.777+0000 7fb96de0eb80 1 -- start start 2024-06-10T01:04:21.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.777+0000 7fb96de0eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393f33b3b0 0x56393f32b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:21.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.777+0000 7fb96de0eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56393f340af0 0x56393f360f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:21.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.777+0000 7fb96de0eb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56393f341410 0x56393f361440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:21.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.777+0000 7fb96de0eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c1aea40 con 0x56393f33b3b0 2024-06-10T01:04:21.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.777+0000 7fb96de0eb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c1ac2d0 con 0x56393f341410 2024-06-10T01:04:21.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.777+0000 7fb96de0eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c1aa120 con 0x56393f340af0 2024-06-10T01:04:21.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.777+0000 7fb961327700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393f33b3b0 0x56393f32b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:21.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.777+0000 7fb961327700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393f33b3b0 0x56393f32b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40150/0 (socket says 172.21.15.37:40150) 2024-06-10T01:04:21.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.777+0000 7fb961327700 1 -- 172.21.15.37:0/527741582 learned_addr learned my addr 172.21.15.37:0/527741582 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:21.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.777+0000 7fb960b26700 1 --2- 172.21.15.37:0/527741582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56393f340af0 0x56393f360f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:21.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.777+0000 7fb961b28700 1 --2- 172.21.15.37:0/527741582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56393f341410 0x56393f361440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.777+0000 7fb961327700 1 -- 172.21.15.37:0/527741582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56393f340af0 msgr2=0x56393f360f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.777+0000 7fb961327700 1 --2- 172.21.15.37:0/527741582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56393f340af0 0x56393f360f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.777+0000 7fb961327700 1 -- 172.21.15.37:0/527741582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56393f341410 msgr2=0x56393f361440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.777+0000 7fb961327700 1 --2- 172.21.15.37:0/527741582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56393f341410 0x56393f361440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.777+0000 7fb961327700 1 -- 172.21.15.37:0/527741582 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56393f340730 con 0x56393f33b3b0 2024-06-10T01:04:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.777+0000 7fb960b26700 1 --2- 172.21.15.37:0/527741582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56393f340af0 0x56393f360f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:04:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.777+0000 7fb961b28700 1 --2- 172.21.15.37:0/527741582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56393f341410 0x56393f361440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:04:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.778+0000 7fb961327700 1 --2- 172.21.15.37:0/527741582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393f33b3b0 0x56393f32b590 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb95800a4a0 tx=0x7fb95800fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b191244a9c0b3f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.778+0000 7fb946ffd700 1 -- 172.21.15.37:0/527741582 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95801ebf0 con 0x56393f33b3b0 2024-06-10T01:04:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.778+0000 7fb946ffd700 1 -- 172.21.15.37:0/527741582 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb958022070 con 0x56393f33b3b0 2024-06-10T01:04:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.778+0000 7fb946ffd700 1 -- 172.21.15.37:0/527741582 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9580255d0 con 0x56393f33b3b0 2024-06-10T01:04:21.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.778+0000 7fb96de0eb80 1 -- 172.21.15.37:0/527741582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393f33b3b0 msgr2=0x56393f32b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.778+0000 7fb96de0eb80 1 --2- 172.21.15.37:0/527741582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393f33b3b0 0x56393f32b590 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb95800a4a0 tx=0x7fb95800fc10 comp rx=0 tx=0).stop 2024-06-10T01:04:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.778+0000 7fb96de0eb80 1 -- 172.21.15.37:0/527741582 shutdown_connections 2024-06-10T01:04:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.778+0000 7fb96de0eb80 1 --2- 172.21.15.37:0/527741582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56393f341410 0x56393f361440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.778+0000 7fb96de0eb80 1 --2- 172.21.15.37:0/527741582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393f33b3b0 0x56393f32b590 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.778+0000 7fb96de0eb80 1 --2- 172.21.15.37:0/527741582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56393f340af0 0x56393f360f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.779+0000 7fb96de0eb80 1 -- 172.21.15.37:0/527741582 >> 172.21.15.37:0/527741582 conn(0x56393f261d40 msgr2=0x56393f32ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.779+0000 7fb96de0eb80 1 -- 172.21.15.37:0/527741582 shutdown_connections 2024-06-10T01:04:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.779+0000 7fb96de0eb80 1 -- 172.21.15.37:0/527741582 wait complete. 2024-06-10T01:04:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.779+0000 7fb96de0eb80 1 -- start start 2024-06-10T01:04:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.779+0000 7fb96de0eb80 1 Processor -- start 2024-06-10T01:04:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.779+0000 7fb96de0eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56393f33b3b0 0x56393f351ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.779+0000 7fb96de0eb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56393f340af0 0x56393f352430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.779+0000 7fb96de0eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393f341410 0x56393f3c2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.779+0000 7fb96de0eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c1aa790 con 0x56393f341410 2024-06-10T01:04:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.779+0000 7fb96de0eb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c1abc00 con 0x56393f340af0 2024-06-10T01:04:21.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.779+0000 7fb96de0eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c1aae60 con 0x56393f33b3b0 2024-06-10T01:04:21.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.779+0000 7fb96de0eb80 10 client.0 _ll_register_callbacks cb 0x56393f2229a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:21.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.780+0000 7fb961b28700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393f341410 0x56393f3c2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:21.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.780+0000 7fb960b26700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56393f340af0 0x56393f352430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:21.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.780+0000 7fb961b28700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393f341410 0x56393f3c2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40164/0 (socket says 172.21.15.37:40164) 2024-06-10T01:04:21.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.780+0000 7fb960b26700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56393f340af0 0x56393f352430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:57140/0 (socket says 172.21.15.37:57140) 2024-06-10T01:04:21.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.780+0000 7fb961b28700 1 -- 172.21.15.37:0/697502558 learned_addr learned my addr 172.21.15.37:0/697502558 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:21.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.780+0000 7fb961327700 1 --2- 172.21.15.37:0/697502558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56393f33b3b0 0x56393f351ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:21.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.780+0000 7fb961b28700 1 -- 172.21.15.37:0/697502558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56393f33b3b0 msgr2=0x56393f351ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:21.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.780+0000 7fb961b28700 1 --2- 172.21.15.37:0/697502558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56393f33b3b0 0x56393f351ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.780+0000 7fb961b28700 1 -- 172.21.15.37:0/697502558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56393f340af0 msgr2=0x56393f352430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:21.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.780+0000 7fb961b28700 1 --2- 172.21.15.37:0/697502558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56393f340af0 0x56393f352430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.780+0000 7fb961b28700 1 -- 172.21.15.37:0/697502558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56393f3c3410 con 0x56393f341410 2024-06-10T01:04:21.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.780+0000 7fb960b26700 1 --2- 172.21.15.37:0/697502558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56393f340af0 0x56393f352430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:21.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.780+0000 7fb961b28700 1 --2- 172.21.15.37:0/697502558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393f341410 0x56393f3c2ed0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb95400d700 tx=0x7fb95409a7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17921333100cff92 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:21.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.780+0000 7fb944ff9700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:04:21.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.780+0000 7fb944ff9700 1 -- 172.21.15.37:0/697502558 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95409b2f0 con 0x56393f341410 2024-06-10T01:04:21.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.780+0000 7fb944ff9700 1 -- 172.21.15.37:0/697502558 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fb9540a3670 con 0x56393f341410 2024-06-10T01:04:21.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.780+0000 7fb96de0eb80 1 -- 172.21.15.37:0/697502558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb958022070 con 0x56393f341410 2024-06-10T01:04:21.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.780+0000 7fb944ff9700 1 -- 172.21.15.37:0/697502558 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb95409ae90 con 0x56393f341410 2024-06-10T01:04:21.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.780+0000 7fb96de0eb80 10 client.16758 fetch_fsmap learned FSMap version 29 2024-06-10T01:04:21.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.780+0000 7fb96de0eb80 1 -- 172.21.15.37:0/697502558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56393f3c4580 con 0x56393f341410 2024-06-10T01:04:21.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.781+0000 7fb944ff9700 1 -- 172.21.15.37:0/697502558 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb958022070 con 0x56393f341410 2024-06-10T01:04:21.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.781+0000 7fb944ff9700 1 -- 172.21.15.37:0/697502558 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fb9540b9ad0 con 0x56393f341410 2024-06-10T01:04:21.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.781+0000 7fb96de0eb80 10 client.16758 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:04:21.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.781+0000 7fb96de0eb80 10 client.16758 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:04:21.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.781+0000 7fb96de0eb80 20 client.16758 populate_metadata read hostname 'smithi037' 2024-06-10T01:04:21.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.781+0000 7fb96de0eb80 1 --2- 172.21.15.37:0/697502558 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56393f3c4c30 0x56393f3c7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:21.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.781+0000 7fb96de0eb80 4 client.16758 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:04:21.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.781+0000 7fb96de0eb80 1 -- 172.21.15.37:0/697502558 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56393f290310 con 0x56393f3c4c30 2024-06-10T01:04:21.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.781+0000 7fb961327700 1 --2- 172.21.15.37:0/697502558 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56393f3c4c30 0x56393f3c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:21.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.782+0000 7fb961327700 1 --2- 172.21.15.37:0/697502558 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56393f3c4c30 0x56393f3c7050 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=152f4d965460c863 server_cookie=bdbba52cddc695d1 in_seq=0 out_seq=0 2024-06-10T01:04:21.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.782+0000 7fb944ff9700 10 client.16758 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:21.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.783+0000 7fb944ff9700 1 -- 172.21.15.37:0/697502558 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56393f290310 con 0x56393f3c4c30 2024-06-10T01:04:21.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.783+0000 7fb944ff9700 10 client.16758 handle_command_reply: tid=0 2024-06-10T01:04:21.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.783+0000 7fb96de0eb80 1 client.16758 shutdown 2024-06-10T01:04:21.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.783+0000 7fb96de0eb80 1 -- 172.21.15.37:0/697502558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393f341410 msgr2=0x56393f3c2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:21.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.783+0000 7fb96de0eb80 1 --2- 172.21.15.37:0/697502558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393f341410 0x56393f3c2ed0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb95400d700 tx=0x7fb95409a7f0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.783+0000 7fb96de0eb80 1 -- 172.21.15.37:0/697502558 shutdown_connections 2024-06-10T01:04:21.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.783+0000 7fb96de0eb80 1 --2- 172.21.15.37:0/697502558 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56393f3c4c30 0x56393f3c7050 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.783+0000 7fb96de0eb80 1 --2- 172.21.15.37:0/697502558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56393f341410 0x56393f3c2ed0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.783+0000 7fb96de0eb80 1 --2- 172.21.15.37:0/697502558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56393f33b3b0 0x56393f351ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.783+0000 7fb96de0eb80 1 --2- 172.21.15.37:0/697502558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56393f340af0 0x56393f352430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.783+0000 7fb944ff9700 0 client.16758 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:21.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.783+0000 7fb96de0eb80 1 -- 172.21.15.37:0/697502558 >> 172.21.15.37:0/697502558 conn(0x56393f261d40 msgr2=0x56393f3372f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:21.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.783+0000 7fb96de0eb80 1 -- 172.21.15.37:0/697502558 shutdown_connections 2024-06-10T01:04:21.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.783+0000 7fb96de0eb80 1 -- 172.21.15.37:0/697502558 wait complete. 2024-06-10T01:04:21.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.783+0000 7fb96de0eb80 20 client.16758 trim_cache size 0 max 16384 2024-06-10T01:04:21.784 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:04:21.784 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-06-10T01:04:21.784 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "70db529d-a248-422f-9c99-0f5645aa5c5d", 2024-06-10T01:04:21.784 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-06-10T01:04:21.784 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:04:21.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.784+0000 7fb96358b700 1 -- 172.21.15.37:0/686984925 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb9480a2ca0 msgr2=0x7fb9480a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:21.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.784+0000 7fb96358b700 1 --2- 172.21.15.37:0/686984925 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb9480a2ca0 0x7fb9480a5160 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb95c1062a0 tx=0x7fb95800a380 comp rx=0 tx=0).stop 2024-06-10T01:04:21.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.784+0000 7fb96358b700 1 -- 172.21.15.37:0/686984925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb95c106720 msgr2=0x7fb95c198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:21.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.784+0000 7fb96358b700 1 --2- 172.21.15.37:0/686984925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb95c106720 0x7fb95c198af0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb954007ee0 tx=0x7fb95400d3b0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.784+0000 7fb96358b700 1 -- 172.21.15.37:0/686984925 shutdown_connections 2024-06-10T01:04:21.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.784+0000 7fb96358b700 1 --2- 172.21.15.37:0/686984925 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb9480a2ca0 0x7fb9480a5160 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.784+0000 7fb96358b700 1 --2- 172.21.15.37:0/686984925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb95c106720 0x7fb95c198af0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.784+0000 7fb96358b700 1 --2- 172.21.15.37:0/686984925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb95c1019d0 0x7fb95c19e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.784+0000 7fb96358b700 1 --2- 172.21.15.37:0/686984925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb95c102380 0x7fb95c19ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:21.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.784+0000 7fb96358b700 1 -- 172.21.15.37:0/686984925 >> 172.21.15.37:0/686984925 conn(0x7fb95c078340 msgr2=0x7fb95c0ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:21.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.784+0000 7fb96358b700 1 -- 172.21.15.37:0/686984925 shutdown_connections 2024-06-10T01:04:21.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:21.784+0000 7fb96358b700 1 -- 172.21.15.37:0/686984925 wait complete. 2024-06-10T01:04:22.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:22 smithi167 ceph-mon[36917]: pgmap v661: 129 pgs: 129 active+clean; 6.6 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 891 KiB/s wr, 574 op/s 2024-06-10T01:04:22.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:22 smithi094 ceph-mon[36708]: pgmap v661: 129 pgs: 129 active+clean; 6.6 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 891 KiB/s wr, 574 op/s 2024-06-10T01:04:22.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:22 smithi037 ceph-mon[32192]: pgmap v661: 129 pgs: 129 active+clean; 6.6 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 891 KiB/s wr, 574 op/s 2024-06-10T01:04:22.463 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:04:22.746 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:23.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:23 smithi037 ceph-mon[32192]: scrub queued for path: / 2024-06-10T01:04:23.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:23 smithi037 ceph-mon[32192]: scrub summary: idle+waiting paths [/] 2024-06-10T01:04:23.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:23 smithi037 ceph-mon[32192]: scrub summary: active paths [/] 2024-06-10T01:04:23.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:23 smithi167 ceph-mon[36917]: scrub queued for path: / 2024-06-10T01:04:23.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:23 smithi167 ceph-mon[36917]: scrub summary: idle+waiting paths [/] 2024-06-10T01:04:23.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:23 smithi167 ceph-mon[36917]: scrub summary: active paths [/] 2024-06-10T01:04:23.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:23 smithi094 ceph-mon[36708]: scrub queued for path: / 2024-06-10T01:04:23.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:23 smithi094 ceph-mon[36708]: scrub summary: idle+waiting paths [/] 2024-06-10T01:04:23.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:23 smithi094 ceph-mon[36708]: scrub summary: active paths [/] 2024-06-10T01:04:23.740 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:04:24.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:24 smithi167 ceph-mon[36917]: pgmap v662: 129 pgs: 129 active+clean; 2.6 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 952 KiB/s wr, 803 op/s 2024-06-10T01:04:24.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:24 smithi094 ceph-mon[36708]: pgmap v662: 129 pgs: 129 active+clean; 2.6 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 952 KiB/s wr, 803 op/s 2024-06-10T01:04:24.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:24 smithi037 ceph-mon[32192]: pgmap v662: 129 pgs: 129 active+clean; 2.6 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 952 KiB/s wr, 803 op/s 2024-06-10T01:04:25.157 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:25.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.409+0000 7ff72022e700 1 -- 172.21.15.37:0/1871855535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff718101670 msgr2=0x7ff718101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:25.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.409+0000 7ff72022e700 1 --2- 172.21.15.37:0/1871855535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff718101670 0x7ff718101af0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff708009990 tx=0x7ff708009ca0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.409+0000 7ff72022e700 1 -- 172.21.15.37:0/1871855535 shutdown_connections 2024-06-10T01:04:25.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.409+0000 7ff72022e700 1 --2- 172.21.15.37:0/1871855535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff718102030 0x7ff71810e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.409+0000 7ff72022e700 1 --2- 172.21.15.37:0/1871855535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7181063b0 0x7ff718106790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.409+0000 7ff72022e700 1 --2- 172.21.15.37:0/1871855535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff718101670 0x7ff718101af0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.409+0000 7ff72022e700 1 -- 172.21.15.37:0/1871855535 >> 172.21.15.37:0/1871855535 conn(0x7ff718078250 msgr2=0x7ff718078660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:25.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.409+0000 7ff72022e700 1 -- 172.21.15.37:0/1871855535 shutdown_connections 2024-06-10T01:04:25.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.410+0000 7ff72022e700 1 -- 172.21.15.37:0/1871855535 wait complete. 2024-06-10T01:04:25.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.410+0000 7ff72022e700 1 Processor -- start 2024-06-10T01:04:25.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.410+0000 7ff72022e700 1 -- start start 2024-06-10T01:04:25.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.410+0000 7ff72022e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff718101670 0x7ff718198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:25.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.410+0000 7ff72022e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff718102030 0x7ff7181992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:25.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.410+0000 7ff72022e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7181063b0 0x7ff71819d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:25.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.411+0000 7ff72022e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff7181117a0 con 0x7ff7181063b0 2024-06-10T01:04:25.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.411+0000 7ff72022e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff718111620 con 0x7ff718101670 2024-06-10T01:04:25.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.411+0000 7ff72022e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff718111920 con 0x7ff718102030 2024-06-10T01:04:25.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.411+0000 7ff71dfca700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff718101670 0x7ff718198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:25.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.411+0000 7ff71e7cb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7181063b0 0x7ff71819d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:25.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.411+0000 7ff71dfca700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff718101670 0x7ff718198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:51064/0 (socket says 172.21.15.37:51064) 2024-06-10T01:04:25.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.411+0000 7ff71dfca700 1 -- 172.21.15.37:0/905643027 learned_addr learned my addr 172.21.15.37:0/905643027 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:25.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.411+0000 7ff71d7c9700 1 --2- 172.21.15.37:0/905643027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff718102030 0x7ff7181992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:25.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.411+0000 7ff71dfca700 1 -- 172.21.15.37:0/905643027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff718102030 msgr2=0x7ff7181992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:25.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.411+0000 7ff71dfca700 1 --2- 172.21.15.37:0/905643027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff718102030 0x7ff7181992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.411+0000 7ff71dfca700 1 -- 172.21.15.37:0/905643027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7181063b0 msgr2=0x7ff71819d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:25.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.411+0000 7ff71dfca700 1 --2- 172.21.15.37:0/905643027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7181063b0 0x7ff71819d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.411+0000 7ff71dfca700 1 -- 172.21.15.37:0/905643027 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71819dd50 con 0x7ff718101670 2024-06-10T01:04:25.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.411+0000 7ff71dfca700 1 --2- 172.21.15.37:0/905643027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff718101670 0x7ff718198d90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff71400d980 tx=0x7ff71400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:25.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.411+0000 7ff70effd700 1 -- 172.21.15.37:0/905643027 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714016070 con 0x7ff718101670 2024-06-10T01:04:25.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.411+0000 7ff72022e700 1 -- 172.21.15.37:0/905643027 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff71819e040 con 0x7ff718101670 2024-06-10T01:04:25.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.411+0000 7ff70effd700 1 -- 172.21.15.37:0/905643027 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff714004d10 con 0x7ff718101670 2024-06-10T01:04:25.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.411+0000 7ff70effd700 1 -- 172.21.15.37:0/905643027 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7140055f0 con 0x7ff718101670 2024-06-10T01:04:25.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.412+0000 7ff72022e700 1 -- 172.21.15.37:0/905643027 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7181a58c0 con 0x7ff718101670 2024-06-10T01:04:25.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.415+0000 7ff70effd700 1 -- 172.21.15.37:0/905643027 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff714004750 con 0x7ff718101670 2024-06-10T01:04:25.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.415+0000 7ff70effd700 1 --2- 172.21.15.37:0/905643027 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff7040a2ca0 0x7ff7040a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:25.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.416+0000 7ff71d7c9700 1 --2- 172.21.15.37:0/905643027 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff7040a2ca0 0x7ff7040a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:25.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.416+0000 7ff70effd700 1 -- 172.21.15.37:0/905643027 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7ff7140cf160 con 0x7ff718101670 2024-06-10T01:04:25.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.416+0000 7ff71d7c9700 1 --2- 172.21.15.37:0/905643027 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff7040a2ca0 0x7ff7040a5160 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff708006010 tx=0x7ff70801a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:25.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.416+0000 7ff72aab1b80 1 Processor -- start 2024-06-10T01:04:25.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.416+0000 7ff72aab1b80 1 -- start start 2024-06-10T01:04:25.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.417+0000 7ff72aab1b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570d2438a50 0x5570d2438e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:25.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.417+0000 7ff72aab1b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570d2439370 0x5570d2432df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:25.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.417+0000 7ff72aab1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570d2433330 0x5570d2433750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:25.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.417+0000 7ff72aab1b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5570d2380810 con 0x5570d2439370 2024-06-10T01:04:25.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.417+0000 7ff72aab1b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5570d2328720 con 0x5570d2438a50 2024-06-10T01:04:25.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.417+0000 7ff72aab1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5570d237c7d0 con 0x5570d2433330 2024-06-10T01:04:25.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.417+0000 7ff71e7cb700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570d2433330 0x5570d2433750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:25.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.417+0000 7ff71d7c9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570d2439370 0x5570d2432df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:25.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.417+0000 7ff71e7cb700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570d2433330 0x5570d2433750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:55616/0 (socket says 172.21.15.37:55616) 2024-06-10T01:04:25.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.417+0000 7ff71e7cb700 1 -- 172.21.15.37:0/28261301 learned_addr learned my addr 172.21.15.37:0/28261301 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:25.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.417+0000 7ff71dfca700 1 --2- 172.21.15.37:0/28261301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570d2438a50 0x5570d2438e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:25.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.417+0000 7ff71d7c9700 1 -- 172.21.15.37:0/28261301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570d2433330 msgr2=0x5570d2433750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:25.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.417+0000 7ff71d7c9700 1 --2- 172.21.15.37:0/28261301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570d2433330 0x5570d2433750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.417+0000 7ff71d7c9700 1 -- 172.21.15.37:0/28261301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570d2438a50 msgr2=0x5570d2438e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:25.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.417+0000 7ff71d7c9700 1 --2- 172.21.15.37:0/28261301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570d2438a50 0x5570d2438e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.417+0000 7ff71d7c9700 1 -- 172.21.15.37:0/28261301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570d231ef50 con 0x5570d2439370 2024-06-10T01:04:25.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.417+0000 7ff71d7c9700 1 --2- 172.21.15.37:0/28261301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570d2439370 0x5570d2432df0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff718078a00 tx=0x7ff70801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b98612f93e2801 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:25.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.418+0000 7ff703fff700 1 -- 172.21.15.37:0/28261301 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70800f9c0 con 0x5570d2439370 2024-06-10T01:04:25.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.418+0000 7ff703fff700 1 -- 172.21.15.37:0/28261301 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff708032070 con 0x5570d2439370 2024-06-10T01:04:25.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.418+0000 7ff703fff700 1 -- 172.21.15.37:0/28261301 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff708023db0 con 0x5570d2439370 2024-06-10T01:04:25.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.418+0000 7ff72aab1b80 1 -- 172.21.15.37:0/28261301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570d2439370 msgr2=0x5570d2432df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:25.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.418+0000 7ff72aab1b80 1 --2- 172.21.15.37:0/28261301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570d2439370 0x5570d2432df0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff718078a00 tx=0x7ff70801e000 comp rx=0 tx=0).stop 2024-06-10T01:04:25.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.418+0000 7ff72aab1b80 1 -- 172.21.15.37:0/28261301 shutdown_connections 2024-06-10T01:04:25.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.418+0000 7ff72aab1b80 1 --2- 172.21.15.37:0/28261301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570d2433330 0x5570d2433750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.418+0000 7ff72aab1b80 1 --2- 172.21.15.37:0/28261301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570d2438a50 0x5570d2438e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.418+0000 7ff72aab1b80 1 --2- 172.21.15.37:0/28261301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570d2439370 0x5570d2432df0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.418+0000 7ff72aab1b80 1 -- 172.21.15.37:0/28261301 >> 172.21.15.37:0/28261301 conn(0x5570d2359cc0 msgr2=0x5570d2429850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:25.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.419+0000 7ff72aab1b80 1 -- 172.21.15.37:0/28261301 shutdown_connections 2024-06-10T01:04:25.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.419+0000 7ff72aab1b80 1 -- 172.21.15.37:0/28261301 wait complete. 2024-06-10T01:04:25.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.419+0000 7ff72aab1b80 1 -- start start 2024-06-10T01:04:25.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.419+0000 7ff72aab1b80 1 Processor -- start 2024-06-10T01:04:25.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.419+0000 7ff72aab1b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570d2433330 0x5570d243f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:25.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.419+0000 7ff72aab1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570d2438a50 0x5570d243c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:25.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.419+0000 7ff72aab1b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570d2439370 0x5570d243c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:25.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.420+0000 7ff72aab1b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5570d243fd80 con 0x5570d2439370 2024-06-10T01:04:25.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.420+0000 7ff72aab1b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5570d2320bf0 con 0x5570d2433330 2024-06-10T01:04:25.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.420+0000 7ff72aab1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5570d243ff00 con 0x5570d2438a50 2024-06-10T01:04:25.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.420+0000 7ff72aab1b80 10 client.0 _ll_register_callbacks cb 0x5570d231a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:25.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.420+0000 7ff71dfca700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570d2433330 0x5570d243f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:25.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.420+0000 7ff71dfca700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570d2433330 0x5570d243f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:51080/0 (socket says 172.21.15.37:51080) 2024-06-10T01:04:25.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.420+0000 7ff71dfca700 1 -- 172.21.15.37:0/3537566824 learned_addr learned my addr 172.21.15.37:0/3537566824 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:25.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.420+0000 7ff71e7cb700 1 --2- 172.21.15.37:0/3537566824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570d2439370 0x5570d243c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:25.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.420+0000 7ff71d7c9700 1 --2- 172.21.15.37:0/3537566824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570d2438a50 0x5570d243c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:25.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.420+0000 7ff71dfca700 1 -- 172.21.15.37:0/3537566824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570d2438a50 msgr2=0x5570d243c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:25.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.420+0000 7ff71dfca700 1 --2- 172.21.15.37:0/3537566824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570d2438a50 0x5570d243c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.421+0000 7ff71dfca700 1 -- 172.21.15.37:0/3537566824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570d2439370 msgr2=0x5570d243c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:25.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.421+0000 7ff71dfca700 1 --2- 172.21.15.37:0/3537566824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570d2439370 0x5570d243c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.421+0000 7ff71dfca700 1 -- 172.21.15.37:0/3537566824 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570d22cf8d0 con 0x5570d2433330 2024-06-10T01:04:25.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.421+0000 7ff71e7cb700 1 --2- 172.21.15.37:0/3537566824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570d2439370 0x5570d243c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:25.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.421+0000 7ff71dfca700 1 --2- 172.21.15.37:0/3537566824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570d2433330 0x5570d243f080 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff714097a30 tx=0x7ff71409b190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d07eaed018165e7a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:25.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.421+0000 7ff7017fa700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:04:25.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.421+0000 7ff7017fa700 1 -- 172.21.15.37:0/3537566824 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71409b5d0 con 0x5570d2433330 2024-06-10T01:04:25.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.421+0000 7ff72aab1b80 1 -- 172.21.15.37:0/3537566824 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff708032070 con 0x5570d2433330 2024-06-10T01:04:25.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.421+0000 7ff7017fa700 1 -- 172.21.15.37:0/3537566824 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7ff7140998d0 con 0x5570d2433330 2024-06-10T01:04:25.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.421+0000 7ff7017fa700 1 -- 172.21.15.37:0/3537566824 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff71409f980 con 0x5570d2433330 2024-06-10T01:04:25.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.421+0000 7ff72aab1b80 10 client.26204 fetch_fsmap learned FSMap version 29 2024-06-10T01:04:25.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.421+0000 7ff72aab1b80 1 -- 172.21.15.37:0/3537566824 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5570d244a6a0 con 0x5570d2433330 2024-06-10T01:04:25.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.421+0000 7ff7017fa700 1 -- 172.21.15.37:0/3537566824 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff708032070 con 0x5570d2433330 2024-06-10T01:04:25.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.422+0000 7ff7017fa700 1 -- 172.21.15.37:0/3537566824 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7ff71409b830 con 0x5570d2433330 2024-06-10T01:04:25.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.422+0000 7ff72aab1b80 10 client.26204 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:04:25.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.422+0000 7ff72aab1b80 10 client.26204 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:04:25.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.422+0000 7ff72aab1b80 20 client.26204 populate_metadata read hostname 'smithi037' 2024-06-10T01:04:25.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.422+0000 7ff72aab1b80 1 --2- 172.21.15.37:0/3537566824 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5570d244cc30 0x5570d244f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:25.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.422+0000 7ff72aab1b80 4 client.26204 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:04:25.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.422+0000 7ff72aab1b80 1 -- 172.21.15.37:0/3537566824 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570d238dd10 con 0x5570d244cc30 2024-06-10T01:04:25.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.422+0000 7ff71d7c9700 1 --2- 172.21.15.37:0/3537566824 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5570d244cc30 0x5570d244f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:25.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.422+0000 7ff71d7c9700 1 --2- 172.21.15.37:0/3537566824 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5570d244cc30 0x5570d244f020 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d872a61e3c290ed4 server_cookie=ac6ff0e5a0a3d0e8 in_seq=0 out_seq=0 2024-06-10T01:04:25.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.422+0000 7ff7017fa700 10 client.26204 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:25.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.423+0000 7ff7017fa700 1 -- 172.21.15.37:0/3537566824 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5570d238dd10 con 0x5570d244cc30 2024-06-10T01:04:25.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.423+0000 7ff7017fa700 10 client.26204 handle_command_reply: tid=0 2024-06-10T01:04:25.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.423+0000 7ff72aab1b80 1 client.26204 shutdown 2024-06-10T01:04:25.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.423+0000 7ff72aab1b80 1 -- 172.21.15.37:0/3537566824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570d2433330 msgr2=0x5570d243f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:25.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.423+0000 7ff72aab1b80 1 --2- 172.21.15.37:0/3537566824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570d2433330 0x5570d243f080 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff714097a30 tx=0x7ff71409b190 comp rx=0 tx=0).stop 2024-06-10T01:04:25.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.424+0000 7ff72aab1b80 1 -- 172.21.15.37:0/3537566824 shutdown_connections 2024-06-10T01:04:25.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.424+0000 7ff72aab1b80 1 --2- 172.21.15.37:0/3537566824 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5570d244cc30 0x5570d244f020 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.424+0000 7ff72aab1b80 1 --2- 172.21.15.37:0/3537566824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570d2439370 0x5570d243c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.424+0000 7ff72aab1b80 1 --2- 172.21.15.37:0/3537566824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570d2433330 0x5570d243f080 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.424+0000 7ff72aab1b80 1 --2- 172.21.15.37:0/3537566824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570d2438a50 0x5570d243c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.424+0000 7ff72aab1b80 1 -- 172.21.15.37:0/3537566824 >> 172.21.15.37:0/3537566824 conn(0x5570d2359cc0 msgr2=0x5570d24295f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:25.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.424+0000 7ff7017fa700 0 client.26204 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:25.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.424+0000 7ff72aab1b80 1 -- 172.21.15.37:0/3537566824 shutdown_connections 2024-06-10T01:04:25.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.424+0000 7ff72aab1b80 1 -- 172.21.15.37:0/3537566824 wait complete. 2024-06-10T01:04:25.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.424+0000 7ff72aab1b80 20 client.26204 trim_cache size 0 max 16384 2024-06-10T01:04:25.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.448+0000 7ff72aab1b80 1 Processor -- start 2024-06-10T01:04:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.448+0000 7ff72aab1b80 1 -- start start 2024-06-10T01:04:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.449+0000 7ff72aab1b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570d2433330 0x5570d2432df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.449+0000 7ff72aab1b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570d2438a50 0x5570d2439790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.449+0000 7ff72aab1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570d2439370 0x5570d2439cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.449+0000 7ff72aab1b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff7181aec90 con 0x5570d2433330 2024-06-10T01:04:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.449+0000 7ff72aab1b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff7181ac520 con 0x5570d2438a50 2024-06-10T01:04:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.449+0000 7ff72aab1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff7181aa370 con 0x5570d2439370 2024-06-10T01:04:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.449+0000 7ff71dfca700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570d2433330 0x5570d2432df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.449+0000 7ff71d7c9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570d2438a50 0x5570d2439790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.449+0000 7ff71dfca700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570d2433330 0x5570d2432df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57736/0 (socket says 172.21.15.37:57736) 2024-06-10T01:04:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.449+0000 7ff71d7c9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570d2438a50 0x5570d2439790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:51094/0 (socket says 172.21.15.37:51094) 2024-06-10T01:04:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.449+0000 7ff71dfca700 1 -- 172.21.15.37:0/3223159439 learned_addr learned my addr 172.21.15.37:0/3223159439 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.449+0000 7ff71e7cb700 1 --2- 172.21.15.37:0/3223159439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570d2439370 0x5570d2439cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:25.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.449+0000 7ff71d7c9700 1 -- 172.21.15.37:0/3223159439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570d2439370 msgr2=0x5570d2439cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:25.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.449+0000 7ff71d7c9700 1 --2- 172.21.15.37:0/3223159439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570d2439370 0x5570d2439cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.449+0000 7ff71d7c9700 1 -- 172.21.15.37:0/3223159439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570d2433330 msgr2=0x5570d2432df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:25.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.449+0000 7ff71d7c9700 1 --2- 172.21.15.37:0/3223159439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570d2433330 0x5570d2432df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.449+0000 7ff71d7c9700 1 -- 172.21.15.37:0/3223159439 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570d243a950 con 0x5570d2438a50 2024-06-10T01:04:25.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.449+0000 7ff71dfca700 1 --2- 172.21.15.37:0/3223159439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570d2433330 0x5570d2432df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:25.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.450+0000 7ff71d7c9700 1 --2- 172.21.15.37:0/3223159439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570d2438a50 0x5570d2439790 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff7080239d0 tx=0x7ff70801e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6749ec59af51d090 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:25.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.450+0000 7ff7037fe700 1 -- 172.21.15.37:0/3223159439 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff708005920 con 0x5570d2438a50 2024-06-10T01:04:25.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.450+0000 7ff7037fe700 1 -- 172.21.15.37:0/3223159439 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff708032070 con 0x5570d2438a50 2024-06-10T01:04:25.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.450+0000 7ff7037fe700 1 -- 172.21.15.37:0/3223159439 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70803e370 con 0x5570d2438a50 2024-06-10T01:04:25.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.450+0000 7ff72aab1b80 1 -- 172.21.15.37:0/3223159439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570d2438a50 msgr2=0x5570d2439790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:25.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.450+0000 7ff72aab1b80 1 --2- 172.21.15.37:0/3223159439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570d2438a50 0x5570d2439790 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff7080239d0 tx=0x7ff70801e000 comp rx=0 tx=0).stop 2024-06-10T01:04:25.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.451+0000 7ff72aab1b80 1 -- 172.21.15.37:0/3223159439 shutdown_connections 2024-06-10T01:04:25.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.451+0000 7ff72aab1b80 1 --2- 172.21.15.37:0/3223159439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570d2439370 0x5570d2439cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.451+0000 7ff72aab1b80 1 --2- 172.21.15.37:0/3223159439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570d2433330 0x5570d2432df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.451+0000 7ff72aab1b80 1 --2- 172.21.15.37:0/3223159439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570d2438a50 0x5570d2439790 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.451+0000 7ff72aab1b80 1 -- 172.21.15.37:0/3223159439 >> 172.21.15.37:0/3223159439 conn(0x5570d2359cc0 msgr2=0x5570d243acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:25.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.451+0000 7ff72aab1b80 1 -- 172.21.15.37:0/3223159439 shutdown_connections 2024-06-10T01:04:25.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.451+0000 7ff72aab1b80 1 -- 172.21.15.37:0/3223159439 wait complete. 2024-06-10T01:04:25.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.451+0000 7ff72aab1b80 1 -- start start 2024-06-10T01:04:25.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.451+0000 7ff72aab1b80 1 Processor -- start 2024-06-10T01:04:25.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.452+0000 7ff72aab1b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570d2433330 0x5570d24453d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:25.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.452+0000 7ff72aab1b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570d2438a50 0x5570d2445910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:25.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.452+0000 7ff72aab1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570d2439370 0x5570d2453bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:25.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.452+0000 7ff72aab1b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff7181aa9e0 con 0x5570d2433330 2024-06-10T01:04:25.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.452+0000 7ff72aab1b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff7181abe50 con 0x5570d2438a50 2024-06-10T01:04:25.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.452+0000 7ff72aab1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff7181ab0b0 con 0x5570d2439370 2024-06-10T01:04:25.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.452+0000 7ff71dfca700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570d2433330 0x5570d24453d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:25.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.452+0000 7ff71d7c9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570d2438a50 0x5570d2445910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:25.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.452+0000 7ff71dfca700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570d2433330 0x5570d24453d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57738/0 (socket says 172.21.15.37:57738) 2024-06-10T01:04:25.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.452+0000 7ff71dfca700 1 -- 172.21.15.37:0/2319065460 learned_addr learned my addr 172.21.15.37:0/2319065460 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:25.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.452+0000 7ff71e7cb700 1 --2- 172.21.15.37:0/2319065460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570d2439370 0x5570d2453bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:25.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.452+0000 7ff71d7c9700 1 -- 172.21.15.37:0/2319065460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570d2439370 msgr2=0x5570d2453bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:25.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.452+0000 7ff71d7c9700 1 --2- 172.21.15.37:0/2319065460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570d2439370 0x5570d2453bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.452+0000 7ff71d7c9700 1 -- 172.21.15.37:0/2319065460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570d2433330 msgr2=0x5570d24453d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:25.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.452+0000 7ff71d7c9700 1 --2- 172.21.15.37:0/2319065460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570d2433330 0x5570d24453d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.452+0000 7ff71d7c9700 1 -- 172.21.15.37:0/2319065460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570d2454400 con 0x5570d2438a50 2024-06-10T01:04:25.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.452+0000 7ff71dfca700 1 --2- 172.21.15.37:0/2319065460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570d2433330 0x5570d24453d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:04:25.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.453+0000 7ff71d7c9700 1 --2- 172.21.15.37:0/2319065460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570d2438a50 0x5570d2445910 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff708005f30 tx=0x7ff70800b7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5ccb3900d12036b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:25.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.453+0000 7ff7017fa700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:04:25.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.453+0000 7ff72aab1b80 10 client.0 _ll_register_callbacks cb 0x5570d231a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:25.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.453+0000 7ff72aab1b80 1 -- 172.21.15.37:0/2319065460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff708032070 con 0x5570d2438a50 2024-06-10T01:04:25.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.453+0000 7ff7017fa700 1 -- 172.21.15.37:0/2319065460 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70804c060 con 0x5570d2438a50 2024-06-10T01:04:25.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.453+0000 7ff7017fa700 1 -- 172.21.15.37:0/2319065460 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7ff70800f740 con 0x5570d2438a50 2024-06-10T01:04:25.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.453+0000 7ff7017fa700 1 -- 172.21.15.37:0/2319065460 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff708023830 con 0x5570d2438a50 2024-06-10T01:04:25.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.453+0000 7ff72aab1b80 10 client.26210 fetch_fsmap learned FSMap version 29 2024-06-10T01:04:25.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.453+0000 7ff72aab1b80 1 -- 172.21.15.37:0/2319065460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({fsmap=29}) -- 0x5570d242bc80 con 0x5570d2438a50 2024-06-10T01:04:25.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.454+0000 7ff7017fa700 1 -- 172.21.15.37:0/2319065460 <== mon.2 v2:172.21.15.94:3300/0 4 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7ff708023a00 con 0x5570d2438a50 2024-06-10T01:04:25.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.454+0000 7ff72aab1b80 10 client.26210 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:04:25.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.454+0000 7ff72aab1b80 10 client.26210 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:04:25.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.454+0000 7ff72aab1b80 20 client.26210 populate_metadata read hostname 'smithi037' 2024-06-10T01:04:25.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.454+0000 7ff72aab1b80 1 --2- 172.21.15.37:0/2319065460 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5570d242beb0 0x5570d242e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:25.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.454+0000 7ff72aab1b80 4 client.26210 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:04:25.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.454+0000 7ff72aab1b80 1 -- 172.21.15.37:0/2319065460 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5570d2388290 con 0x5570d242beb0 2024-06-10T01:04:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.454+0000 7ff71dfca700 1 --2- 172.21.15.37:0/2319065460 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5570d242beb0 0x5570d242e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.454+0000 7ff71dfca700 1 --2- 172.21.15.37:0/2319065460 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5570d242beb0 0x5570d242e2d0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d67ed8ede9afeb4e server_cookie=29f5fd8181d53a46 in_seq=0 out_seq=0 2024-06-10T01:04:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.454+0000 7ff7017fa700 10 client.26210 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.455+0000 7ff7017fa700 1 -- 172.21.15.37:0/2319065460 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x5570d2388290 con 0x5570d242beb0 2024-06-10T01:04:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.455+0000 7ff7017fa700 10 client.26210 handle_command_reply: tid=0 2024-06-10T01:04:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.455+0000 7ff72aab1b80 1 client.26210 shutdown 2024-06-10T01:04:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.455+0000 7ff72aab1b80 1 -- 172.21.15.37:0/2319065460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570d2438a50 msgr2=0x5570d2445910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.455+0000 7ff72aab1b80 1 --2- 172.21.15.37:0/2319065460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570d2438a50 0x5570d2445910 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff708005f30 tx=0x7ff70800b7b0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.455+0000 7ff72aab1b80 1 -- 172.21.15.37:0/2319065460 shutdown_connections 2024-06-10T01:04:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.455+0000 7ff72aab1b80 1 --2- 172.21.15.37:0/2319065460 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5570d242beb0 0x5570d242e2d0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.455+0000 7ff72aab1b80 1 --2- 172.21.15.37:0/2319065460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570d2439370 0x5570d2453bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.455+0000 7ff7017fa700 0 client.26210 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.455+0000 7ff72aab1b80 1 --2- 172.21.15.37:0/2319065460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570d2433330 0x5570d24453d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.455+0000 7ff72aab1b80 1 --2- 172.21.15.37:0/2319065460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570d2438a50 0x5570d2445910 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.455+0000 7ff72aab1b80 1 -- 172.21.15.37:0/2319065460 >> 172.21.15.37:0/2319065460 conn(0x5570d2359cc0 msgr2=0x5570d2439790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:25.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.455+0000 7ff72aab1b80 1 -- 172.21.15.37:0/2319065460 shutdown_connections 2024-06-10T01:04:25.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.455+0000 7ff72aab1b80 1 -- 172.21.15.37:0/2319065460 wait complete. 2024-06-10T01:04:25.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.456+0000 7ff72aab1b80 20 client.26210 trim_cache size 0 max 16384 2024-06-10T01:04:25.457 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:04:25.457 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "scrub active (9355 inodes in the stack)", 2024-06-10T01:04:25.457 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-06-10T01:04:25.457 INFO:teuthology.orchestra.run.smithi037.stdout: "70db529d-a248-422f-9c99-0f5645aa5c5d": { 2024-06-10T01:04:25.457 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-06-10T01:04:25.457 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "70db529d-a248-422f-9c99-0f5645aa5c5d", 2024-06-10T01:04:25.457 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-06-10T01:04:25.458 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:04:25.458 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:04:25.458 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:04:25.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.456+0000 7ff72022e700 1 -- 172.21.15.37:0/905643027 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff7040a2ca0 msgr2=0x7ff7040a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:25.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.456+0000 7ff72022e700 1 --2- 172.21.15.37:0/905643027 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff7040a2ca0 0x7ff7040a5160 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff708006010 tx=0x7ff70801a040 comp rx=0 tx=0).stop 2024-06-10T01:04:25.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.456+0000 7ff72022e700 1 -- 172.21.15.37:0/905643027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff718101670 msgr2=0x7ff718198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:25.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.456+0000 7ff72022e700 1 --2- 172.21.15.37:0/905643027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff718101670 0x7ff718198d90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff71400d980 tx=0x7ff71400dc90 comp rx=0 tx=0).stop 2024-06-10T01:04:25.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.456+0000 7ff72022e700 1 -- 172.21.15.37:0/905643027 shutdown_connections 2024-06-10T01:04:25.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.456+0000 7ff72022e700 1 --2- 172.21.15.37:0/905643027 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff7040a2ca0 0x7ff7040a5160 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.456+0000 7ff72022e700 1 --2- 172.21.15.37:0/905643027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7181063b0 0x7ff71819d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.456+0000 7ff72022e700 1 --2- 172.21.15.37:0/905643027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff718101670 0x7ff718198d90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.456+0000 7ff72022e700 1 --2- 172.21.15.37:0/905643027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff718102030 0x7ff7181992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:25.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.456+0000 7ff72022e700 1 -- 172.21.15.37:0/905643027 >> 172.21.15.37:0/905643027 conn(0x7ff718078250 msgr2=0x7ff71806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:25.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.456+0000 7ff72022e700 1 -- 172.21.15.37:0/905643027 shutdown_connections 2024-06-10T01:04:25.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:25.456+0000 7ff72022e700 1 -- 172.21.15.37:0/905643027 wait complete. 2024-06-10T01:04:26.247 INFO:tasks.cephfs.filesystem:scrub status for tag:70db529d-a248-422f-9c99-0f5645aa5c5d - {'path': '/', 'tag': '70db529d-a248-422f-9c99-0f5645aa5c5d', 'options': 'recursive,force'} 2024-06-10T01:04:26.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:26 smithi167 ceph-mon[36917]: pgmap v663: 129 pgs: 129 active+clean; 2.6 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 830 KiB/s wr, 1.52k op/s 2024-06-10T01:04:26.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:26 smithi094 ceph-mon[36708]: pgmap v663: 129 pgs: 129 active+clean; 2.6 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 830 KiB/s wr, 1.52k op/s 2024-06-10T01:04:26.365 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:26 smithi037 ceph-mon[32192]: pgmap v663: 129 pgs: 129 active+clean; 2.6 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 830 KiB/s wr, 1.52k op/s 2024-06-10T01:04:27.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:27 smithi167 ceph-mon[36917]: scrub summary: idle+waiting paths [/] 2024-06-10T01:04:27.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:27 smithi094 ceph-mon[36708]: scrub summary: idle+waiting paths [/] 2024-06-10T01:04:27.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:27 smithi037 ceph-mon[32192]: scrub summary: idle+waiting paths [/] 2024-06-10T01:04:27.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.853+0000 7f4832996700 1 -- 172.21.15.37:0/2630492289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f482c1077b0 msgr2=0x7f482c10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:27.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.853+0000 7f4832996700 1 --2- 172.21.15.37:0/2630492289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f482c1077b0 0x7f482c10f3c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f482400b0d0 tx=0x7f482400b3e0 comp rx=0 tx=0).stop 2024-06-10T01:04:27.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.853+0000 7f4832996700 1 -- 172.21.15.37:0/2630492289 shutdown_connections 2024-06-10T01:04:27.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.853+0000 7f4832996700 1 --2- 172.21.15.37:0/2630492289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f482c1077b0 0x7f482c10f3c0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:27.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.853+0000 7f4832996700 1 --2- 172.21.15.37:0/2630492289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f482c06ae60 0x7f482c06b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:27.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.853+0000 7f4832996700 1 --2- 172.21.15.37:0/2630492289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f482c06b810 0x7f482c1070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:27.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.853+0000 7f4832996700 1 -- 172.21.15.37:0/2630492289 >> 172.21.15.37:0/2630492289 conn(0x7f482c0773e0 msgr2=0x7f482c0777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:27.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.853+0000 7f4832996700 1 -- 172.21.15.37:0/2630492289 shutdown_connections 2024-06-10T01:04:27.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.853+0000 7f4832996700 1 -- 172.21.15.37:0/2630492289 wait complete. 2024-06-10T01:04:27.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.854+0000 7f4832996700 1 Processor -- start 2024-06-10T01:04:27.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.854+0000 7f4832996700 1 -- start start 2024-06-10T01:04:27.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.854+0000 7f4832996700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f482c06ae60 0x7f482c1014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:27.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.854+0000 7f4832996700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f482c06b810 0x7f482c101a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:27.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.854+0000 7f4832996700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f482c1077b0 0x7f482c101f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:27.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.854+0000 7f4832996700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f482c112800 con 0x7f482c06ae60 2024-06-10T01:04:27.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.854+0000 7f4832996700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f482c112680 con 0x7f482c06b810 2024-06-10T01:04:27.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.854+0000 7f4832996700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f482c112980 con 0x7f482c1077b0 2024-06-10T01:04:27.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.854+0000 7f482bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f482c06ae60 0x7f482c1014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:27.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.854+0000 7f482bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f482c06ae60 0x7f482c1014f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57750/0 (socket says 172.21.15.37:57750) 2024-06-10T01:04:27.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.854+0000 7f482bfff700 1 -- 172.21.15.37:0/632604642 learned_addr learned my addr 172.21.15.37:0/632604642 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:27.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.854+0000 7f482b7fe700 1 --2- 172.21.15.37:0/632604642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f482c06b810 0x7f482c101a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:27.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.855+0000 7f4830f33700 1 --2- 172.21.15.37:0/632604642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f482c1077b0 0x7f482c101f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:27.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.855+0000 7f482bfff700 1 -- 172.21.15.37:0/632604642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f482c1077b0 msgr2=0x7f482c101f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:27.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.855+0000 7f482bfff700 1 --2- 172.21.15.37:0/632604642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f482c1077b0 0x7f482c101f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:27.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.855+0000 7f482bfff700 1 -- 172.21.15.37:0/632604642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f482c06b810 msgr2=0x7f482c101a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:27.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.855+0000 7f482bfff700 1 --2- 172.21.15.37:0/632604642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f482c06b810 0x7f482c101a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:27.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.855+0000 7f482bfff700 1 -- 172.21.15.37:0/632604642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482c1a7180 con 0x7f482c06ae60 2024-06-10T01:04:27.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.855+0000 7f482bfff700 1 --2- 172.21.15.37:0/632604642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f482c06ae60 0x7f482c1014f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f481c00d950 tx=0x7f481c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:27.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.855+0000 7f48297fa700 1 -- 172.21.15.37:0/632604642 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481c016070 con 0x7f482c06ae60 2024-06-10T01:04:27.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.855+0000 7f48297fa700 1 -- 172.21.15.37:0/632604642 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f481c009e20 con 0x7f482c06ae60 2024-06-10T01:04:27.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.855+0000 7f48297fa700 1 -- 172.21.15.37:0/632604642 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481c00f620 con 0x7f482c06ae60 2024-06-10T01:04:27.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.855+0000 7f4832996700 1 -- 172.21.15.37:0/632604642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f482c1a7470 con 0x7f482c06ae60 2024-06-10T01:04:27.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.855+0000 7f4832996700 1 -- 172.21.15.37:0/632604642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f482c1a79b0 con 0x7f482c06ae60 2024-06-10T01:04:27.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.856+0000 7f4832996700 1 -- 172.21.15.37:0/632604642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f482c069180 con 0x7f482c06ae60 2024-06-10T01:04:27.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.857+0000 7f48297fa700 1 -- 172.21.15.37:0/632604642 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f481c019070 con 0x7f482c06ae60 2024-06-10T01:04:27.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.858+0000 7f48297fa700 1 --2- 172.21.15.37:0/632604642 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f48140a2c20 0x7f48140a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:27.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.858+0000 7f482b7fe700 1 --2- 172.21.15.37:0/632604642 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f48140a2c20 0x7f48140a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:27.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.858+0000 7f48297fa700 1 -- 172.21.15.37:0/632604642 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f481c0ceff0 con 0x7f482c06ae60 2024-06-10T01:04:27.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.859+0000 7f482b7fe700 1 --2- 172.21.15.37:0/632604642 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f48140a2c20 0x7f48140a50e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f482c0ffc70 tx=0x7f482000b3c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:27.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:27.862+0000 7f48297fa700 1 -- 172.21.15.37:0/632604642 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f481c09bb60 con 0x7f482c06ae60 2024-06-10T01:04:28.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:28.120+0000 7f4832996700 1 -- 172.21.15.37:0/632604642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f482c02dca0 con 0x7f482c06ae60 2024-06-10T01:04:28.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:28.121+0000 7f48297fa700 1 -- 172.21.15.37:0/632604642 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11864 (secure 0 0 0) 0x7f481c09b460 con 0x7f482c06ae60 2024-06-10T01:04:28.119 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:04:28.119 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T01:03:24.246183+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26110":{"gid":26110,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/317977487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":317977487},{"type":"v1","addr":"172.21.15.167:6835","nonce":317977487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24458,24472,24451]},"id":1}]} 2024-06-10T01:04:28.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:28.123+0000 7f4832996700 1 -- 172.21.15.37:0/632604642 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f48140a2c20 msgr2=0x7f48140a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:28.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:28.123+0000 7f4832996700 1 --2- 172.21.15.37:0/632604642 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f48140a2c20 0x7f48140a50e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f482c0ffc70 tx=0x7f482000b3c0 comp rx=0 tx=0).stop 2024-06-10T01:04:28.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:28.123+0000 7f4832996700 1 -- 172.21.15.37:0/632604642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f482c06ae60 msgr2=0x7f482c1014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:28.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:28.123+0000 7f4832996700 1 --2- 172.21.15.37:0/632604642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f482c06ae60 0x7f482c1014f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f481c00d950 tx=0x7f481c00dc60 comp rx=0 tx=0).stop 2024-06-10T01:04:28.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:28.123+0000 7f4832996700 1 -- 172.21.15.37:0/632604642 shutdown_connections 2024-06-10T01:04:28.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:28.123+0000 7f4832996700 1 --2- 172.21.15.37:0/632604642 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f48140a2c20 0x7f48140a50e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:28.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:28.123+0000 7f4832996700 1 --2- 172.21.15.37:0/632604642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f482c1077b0 0x7f482c101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:28.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:28.123+0000 7f4832996700 1 --2- 172.21.15.37:0/632604642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f482c06ae60 0x7f482c1014f0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:28.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:28.123+0000 7f4832996700 1 --2- 172.21.15.37:0/632604642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f482c06b810 0x7f482c101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:28.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:28.123+0000 7f4832996700 1 -- 172.21.15.37:0/632604642 >> 172.21.15.37:0/632604642 conn(0x7f482c0773e0 msgr2=0x7f482c100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:28.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:28.123+0000 7f4832996700 1 -- 172.21.15.37:0/632604642 shutdown_connections 2024-06-10T01:04:28.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:28.123+0000 7f4832996700 1 -- 172.21.15.37:0/632604642 wait complete. 2024-06-10T01:04:28.123 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 29 2024-06-10T01:04:28.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:28 smithi167 ceph-mon[36917]: pgmap v664: 129 pgs: 129 active+clean; 2.6 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 537 KiB/s wr, 1.60k op/s 2024-06-10T01:04:28.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:28 smithi094 ceph-mon[36708]: pgmap v664: 129 pgs: 129 active+clean; 2.6 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 537 KiB/s wr, 1.60k op/s 2024-06-10T01:04:28.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:28 smithi037 ceph-mon[32192]: pgmap v664: 129 pgs: 129 active+clean; 2.6 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 537 KiB/s wr, 1.60k op/s 2024-06-10T01:04:29.027 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24472 quiesce db --set-id 11dd31af --release --await' 2024-06-10T01:04:29.028 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24472 quiesce db --set-id 11dd31af --release --await 2024-06-10T01:04:29.287 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:29.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:29 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/632604642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:04:29.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:29 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/632604642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:04:29.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:29 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/632604642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:04:30.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:30 smithi094 ceph-mon[36708]: pgmap v665: 129 pgs: 129 active+clean; 2.6 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 537 KiB/s wr, 3.55k op/s 2024-06-10T01:04:30.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:30 smithi037 ceph-mon[32192]: pgmap v665: 129 pgs: 129 active+clean; 2.6 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 537 KiB/s wr, 3.55k op/s 2024-06-10T01:04:30.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:30 smithi167 ceph-mon[36917]: pgmap v665: 129 pgs: 129 active+clean; 2.6 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 537 KiB/s wr, 3.55k op/s 2024-06-10T01:04:30.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.933+0000 7fd6a5eee640 1 -- 172.21.15.37:0/3482899326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6a0100900 msgr2=0x7fd6a0100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:30.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.933+0000 7fd6a5eee640 1 --2- 172.21.15.37:0/3482899326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6a0100900 0x7fd6a0100d20 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd690009900 tx=0x7fd69002a410 comp rx=0 tx=0).stop 2024-06-10T01:04:30.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.933+0000 7fd6a5eee640 1 -- 172.21.15.37:0/3482899326 shutdown_connections 2024-06-10T01:04:30.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.933+0000 7fd6a5eee640 1 --2- 172.21.15.37:0/3482899326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6a01024b0 0x7fd6a0108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.933+0000 7fd6a5eee640 1 --2- 172.21.15.37:0/3482899326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6a0101af0 0x7fd6a0101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.933+0000 7fd6a5eee640 1 --2- 172.21.15.37:0/3482899326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6a0100900 0x7fd6a0100d20 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.933+0000 7fd6a5eee640 1 -- 172.21.15.37:0/3482899326 >> 172.21.15.37:0/3482899326 conn(0x7fd6a00fbe70 msgr2=0x7fd6a00fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.933+0000 7fd6a5eee640 1 -- 172.21.15.37:0/3482899326 shutdown_connections 2024-06-10T01:04:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.933+0000 7fd6a5eee640 1 -- 172.21.15.37:0/3482899326 wait complete. 2024-06-10T01:04:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.934+0000 7fd6a5eee640 1 Processor -- start 2024-06-10T01:04:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.934+0000 7fd6a5eee640 1 -- start start 2024-06-10T01:04:30.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.934+0000 7fd6a5eee640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6a0100900 0x7fd6a0197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.934+0000 7fd6a5eee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6a0101af0 0x7fd6a0198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.934+0000 7fd6a5eee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6a01024b0 0x7fd6a019f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.934+0000 7fd6a5eee640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a010c550 con 0x7fd6a01024b0 2024-06-10T01:04:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.934+0000 7fd6a5eee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a010c3d0 con 0x7fd6a0101af0 2024-06-10T01:04:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.934+0000 7fd6a5eee640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a010c6d0 con 0x7fd6a0100900 2024-06-10T01:04:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.935+0000 7fd69ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6a01024b0 0x7fd6a019f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.935+0000 7fd69ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6a01024b0 0x7fd6a019f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57780/0 (socket says 172.21.15.37:57780) 2024-06-10T01:04:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.935+0000 7fd69ffff640 1 -- 172.21.15.37:0/275475133 learned_addr learned my addr 172.21.15.37:0/275475133 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.935+0000 7fd69f7fe640 1 --2- 172.21.15.37:0/275475133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6a0100900 0x7fd6a0197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.935+0000 7fd69effd640 1 --2- 172.21.15.37:0/275475133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6a0101af0 0x7fd6a0198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.935+0000 7fd69ffff640 1 -- 172.21.15.37:0/275475133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6a0100900 msgr2=0x7fd6a0197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.935+0000 7fd69ffff640 1 --2- 172.21.15.37:0/275475133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6a0100900 0x7fd6a0197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.935+0000 7fd69ffff640 1 -- 172.21.15.37:0/275475133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6a0101af0 msgr2=0x7fd6a0198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.935+0000 7fd69ffff640 1 --2- 172.21.15.37:0/275475133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6a0101af0 0x7fd6a0198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.935+0000 7fd69ffff640 1 -- 172.21.15.37:0/275475133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a019f900 con 0x7fd6a01024b0 2024-06-10T01:04:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.935+0000 7fd69f7fe640 1 --2- 172.21.15.37:0/275475133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6a0100900 0x7fd6a0197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:04:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.935+0000 7fd69ffff640 1 --2- 172.21.15.37:0/275475133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6a01024b0 0x7fd6a019f260 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd69800eaa0 tx=0x7fd69800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.935+0000 7fd69cff9640 1 -- 172.21.15.37:0/275475133 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69800cdb0 con 0x7fd6a01024b0 2024-06-10T01:04:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.935+0000 7fd6a5eee640 1 -- 172.21.15.37:0/275475133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a019fbf0 con 0x7fd6a01024b0 2024-06-10T01:04:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.935+0000 7fd69cff9640 1 -- 172.21.15.37:0/275475133 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd698005020 con 0x7fd6a01024b0 2024-06-10T01:04:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.935+0000 7fd69cff9640 1 -- 172.21.15.37:0/275475133 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6980116b0 con 0x7fd6a01024b0 2024-06-10T01:04:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.935+0000 7fd6a5eee640 1 -- 172.21.15.37:0/275475133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a01a00f0 con 0x7fd6a01024b0 2024-06-10T01:04:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.938+0000 7fd69cff9640 1 -- 172.21.15.37:0/275475133 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd698011850 con 0x7fd6a01024b0 2024-06-10T01:04:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.938+0000 7fd6a8bfa740 1 Processor -- start 2024-06-10T01:04:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.938+0000 7fd6a8bfa740 1 -- start start 2024-06-10T01:04:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.938+0000 7fd6a8bfa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc498c4010 0x55dc498c4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.938+0000 7fd6a8bfa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc498c4a00 0x55dc498c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.938+0000 7fd6a8bfa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc498c9b70 0x55dc498cc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.938+0000 7fd6a8bfa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dc498274f0 con 0x55dc498c9b70 2024-06-10T01:04:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.938+0000 7fd6a8bfa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55dc497c5870 con 0x55dc498c4a00 2024-06-10T01:04:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.938+0000 7fd6a8bfa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dc498234b0 con 0x55dc498c4010 2024-06-10T01:04:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.938+0000 7fd69effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc498c9b70 0x55dc498cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:30.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.938+0000 7fd69f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc498c4a00 0x55dc498c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.939+0000 7fd69f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc498c4a00 0x55dc498c9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:51146/0 (socket says 172.21.15.37:51146) 2024-06-10T01:04:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.939+0000 7fd69f7fe640 1 -- 172.21.15.37:0/2103947256 learned_addr learned my addr 172.21.15.37:0/2103947256 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.939+0000 7fd69cff9640 1 --2- 172.21.15.37:0/275475133 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd6880a2c30 0x7fd6880a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.939+0000 7fd69effd640 1 --2- 172.21.15.37:0/2103947256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc498c4010 0x55dc498c4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.939+0000 7fd69cff9640 1 -- 172.21.15.37:0/275475133 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fd6980cf680 con 0x7fd6a01024b0 2024-06-10T01:04:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.939+0000 7fd69ffff640 1 --2- 172.21.15.37:0/275475133 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd6880a2c30 0x7fd6880a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.939+0000 7fd69f7fe640 1 -- 172.21.15.37:0/2103947256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc498c4010 msgr2=0x55dc498c4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.939+0000 7fd69f7fe640 1 --2- 172.21.15.37:0/2103947256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc498c4010 0x55dc498c4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.939+0000 7fd69f7fe640 1 -- 172.21.15.37:0/2103947256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc498c9b70 msgr2=0x55dc498cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.939+0000 7fd69f7fe640 1 --2- 172.21.15.37:0/2103947256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc498c9b70 0x55dc498cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.939+0000 7fd69f7fe640 1 -- 172.21.15.37:0/2103947256 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc497e19b0 con 0x55dc498c4a00 2024-06-10T01:04:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.939+0000 7fd69effd640 1 --2- 172.21.15.37:0/2103947256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc498c9b70 0x55dc498cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.939+0000 7fd69f7fe640 1 --2- 172.21.15.37:0/2103947256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc498c4a00 0x55dc498c9500 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd690006010 tx=0x7fd690032d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a53f72c9bc170170 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.939+0000 7fd69ffff640 1 --2- 172.21.15.37:0/275475133 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd6880a2c30 0x7fd6880a50f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd6980ccd90 tx=0x7fd6980cf980 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.939+0000 7fd685ffb640 1 -- 172.21.15.37:0/2103947256 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69003f070 con 0x55dc498c4a00 2024-06-10T01:04:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.939+0000 7fd685ffb640 1 -- 172.21.15.37:0/2103947256 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd690031970 con 0x55dc498c4a00 2024-06-10T01:04:30.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.939+0000 7fd685ffb640 1 -- 172.21.15.37:0/2103947256 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd690035070 con 0x55dc498c4a00 2024-06-10T01:04:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.940+0000 7fd6a8bfa740 1 -- 172.21.15.37:0/2103947256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc498c4a00 msgr2=0x55dc498c9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.940+0000 7fd6a8bfa740 1 --2- 172.21.15.37:0/2103947256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc498c4a00 0x55dc498c9500 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd690006010 tx=0x7fd690032d80 comp rx=0 tx=0).stop 2024-06-10T01:04:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.940+0000 7fd6a8bfa740 1 -- 172.21.15.37:0/2103947256 shutdown_connections 2024-06-10T01:04:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.940+0000 7fd6a8bfa740 1 --2- 172.21.15.37:0/2103947256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc498c9b70 0x55dc498cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.940+0000 7fd6a8bfa740 1 --2- 172.21.15.37:0/2103947256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc498c4a00 0x55dc498c9500 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.940+0000 7fd6a8bfa740 1 --2- 172.21.15.37:0/2103947256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc498c4010 0x55dc498c4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.940+0000 7fd6a8bfa740 1 -- 172.21.15.37:0/2103947256 >> 172.21.15.37:0/2103947256 conn(0x55dc498bf680 msgr2=0x55dc498bfa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.940+0000 7fd6a8bfa740 1 -- 172.21.15.37:0/2103947256 shutdown_connections 2024-06-10T01:04:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.940+0000 7fd6a8bfa740 1 -- 172.21.15.37:0/2103947256 wait complete. 2024-06-10T01:04:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.941+0000 7fd6a8bfa740 1 -- start start 2024-06-10T01:04:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.941+0000 7fd6a8bfa740 1 Processor -- start 2024-06-10T01:04:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.941+0000 7fd6a8bfa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc498c4010 0x55dc498ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.941+0000 7fd6a8bfa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc498c4a00 0x55dc498ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.941+0000 7fd6a8bfa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc498c9b70 0x55dc498d5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.941+0000 7fd6a8bfa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dc498ccac0 con 0x55dc498c4a00 2024-06-10T01:04:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.941+0000 7fd6a8bfa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55dc497b0ae0 con 0x55dc498c4010 2024-06-10T01:04:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.941+0000 7fd6a8bfa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dc498ccc40 con 0x55dc498c9b70 2024-06-10T01:04:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.941+0000 7fd6a8bfa740 10 client.0 _ll_register_callbacks cb 0x55dc497e99b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.941+0000 7fd69f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc498c4a00 0x55dc498ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.941+0000 7fd69f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc498c4a00 0x55dc498ce7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57800/0 (socket says 172.21.15.37:57800) 2024-06-10T01:04:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.941+0000 7fd69f7fe640 1 -- 172.21.15.37:0/4165210998 learned_addr learned my addr 172.21.15.37:0/4165210998 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.941+0000 7fd69effd640 1 --2- 172.21.15.37:0/4165210998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc498c4010 0x55dc498ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.942+0000 7fd69effd640 1 --2- 172.21.15.37:0/4165210998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc498c9b70 0x55dc498d5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.942+0000 7fd69effd640 1 -- 172.21.15.37:0/4165210998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc498c9b70 msgr2=0x55dc498d5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.942+0000 7fd69effd640 1 --2- 172.21.15.37:0/4165210998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc498c9b70 0x55dc498d5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.942+0000 7fd69effd640 1 -- 172.21.15.37:0/4165210998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc498c4a00 msgr2=0x55dc498ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.942+0000 7fd69effd640 1 --2- 172.21.15.37:0/4165210998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc498c4a00 0x55dc498ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.942+0000 7fd69effd640 1 -- 172.21.15.37:0/4165210998 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc4953b210 con 0x55dc498c4010 2024-06-10T01:04:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.942+0000 7fd69f7fe640 1 --2- 172.21.15.37:0/4165210998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc498c4a00 0x55dc498ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.942+0000 7fd69effd640 1 --2- 172.21.15.37:0/4165210998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc498c4010 0x55dc498ce280 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x55dc498c5260 tx=0x7fd694016040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98d62fc3485d4154 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.942+0000 7fd67effd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:04:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.942+0000 7fd67effd640 1 -- 172.21.15.37:0/4165210998 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6940141d0 con 0x55dc498c4010 2024-06-10T01:04:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.942+0000 7fd6a8bfa740 1 -- 172.21.15.37:0/4165210998 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd690031970 con 0x55dc498c4010 2024-06-10T01:04:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.942+0000 7fd67effd640 1 -- 172.21.15.37:0/4165210998 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fd694004030 con 0x55dc498c4010 2024-06-10T01:04:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.942+0000 7fd67effd640 1 -- 172.21.15.37:0/4165210998 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd694029050 con 0x55dc498c4010 2024-06-10T01:04:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.942+0000 7fd6a8bfa740 10 client.26228 fetch_fsmap learned FSMap version 29 2024-06-10T01:04:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.942+0000 7fd6a8bfa740 1 -- 172.21.15.37:0/4165210998 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dc49607620 con 0x55dc498c4010 2024-06-10T01:04:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.942+0000 7fd67effd640 1 -- 172.21.15.37:0/4165210998 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd690031970 con 0x55dc498c4010 2024-06-10T01:04:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.943+0000 7fd67effd640 1 -- 172.21.15.37:0/4165210998 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fd6940234c0 con 0x55dc498c4010 2024-06-10T01:04:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.943+0000 7fd6a8bfa740 10 client.26228 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:04:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.943+0000 7fd6a8bfa740 10 client.26228 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:04:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.943+0000 7fd6a8bfa740 20 client.26228 populate_metadata read hostname 'smithi037' 2024-06-10T01:04:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.943+0000 7fd6a8bfa740 1 --2- 172.21.15.37:0/4165210998 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55dc498da030 0x55dc498dc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.943+0000 7fd6a8bfa740 4 client.26228 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:04:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.943+0000 7fd6a8bfa740 1 -- 172.21.15.37:0/4165210998 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc4983af10 con 0x55dc498da030 2024-06-10T01:04:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.943+0000 7fd69f7fe640 1 --2- 172.21.15.37:0/4165210998 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55dc498da030 0x55dc498dc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.944+0000 7fd69f7fe640 1 --2- 172.21.15.37:0/4165210998 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55dc498da030 0x55dc498dc440 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0546a0daaaf590e server_cookie=5fd5c617982af3d7 in_seq=0 out_seq=0 2024-06-10T01:04:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.944+0000 7fd67effd640 10 client.26228 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.944+0000 7fd67effd640 1 -- 172.21.15.37:0/4165210998 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55dc4983af10 con 0x55dc498da030 2024-06-10T01:04:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.944+0000 7fd67effd640 10 client.26228 handle_command_reply: tid=0 2024-06-10T01:04:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.945+0000 7fd6a8bfa740 1 client.26228 shutdown 2024-06-10T01:04:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.945+0000 7fd6a8bfa740 1 -- 172.21.15.37:0/4165210998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc498c4010 msgr2=0x55dc498ce280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.945+0000 7fd6a8bfa740 1 --2- 172.21.15.37:0/4165210998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc498c4010 0x55dc498ce280 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x55dc498c5260 tx=0x7fd694016040 comp rx=0 tx=0).stop 2024-06-10T01:04:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.945+0000 7fd6a8bfa740 1 -- 172.21.15.37:0/4165210998 shutdown_connections 2024-06-10T01:04:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.945+0000 7fd6a8bfa740 1 --2- 172.21.15.37:0/4165210998 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55dc498da030 0x55dc498dc440 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.945+0000 7fd6a8bfa740 1 --2- 172.21.15.37:0/4165210998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc498c9b70 0x55dc498d5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.945+0000 7fd6a8bfa740 1 --2- 172.21.15.37:0/4165210998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc498c4a00 0x55dc498ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.945+0000 7fd67effd640 0 client.26228 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.945+0000 7fd6a8bfa740 1 --2- 172.21.15.37:0/4165210998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc498c4010 0x55dc498ce280 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.945+0000 7fd6a8bfa740 1 -- 172.21.15.37:0/4165210998 >> 172.21.15.37:0/4165210998 conn(0x55dc498bf680 msgr2=0x55dc498be7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.945+0000 7fd6a8bfa740 1 -- 172.21.15.37:0/4165210998 shutdown_connections 2024-06-10T01:04:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.945+0000 7fd6a8bfa740 1 -- 172.21.15.37:0/4165210998 wait complete. 2024-06-10T01:04:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.945+0000 7fd6a8bfa740 20 client.26228 trim_cache size 0 max 16384 2024-06-10T01:04:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.960+0000 7fd6a8bfa740 1 Processor -- start 2024-06-10T01:04:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.960+0000 7fd6a8bfa740 1 -- start start 2024-06-10T01:04:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.960+0000 7fd6a8bfa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc498c4010 0x55dc498ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.960+0000 7fd6a8bfa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc498c4a00 0x55dc498d5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.960+0000 7fd6a8bfa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc498c9b70 0x55dc498c2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.960+0000 7fd6a8bfa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a01ad010 con 0x55dc498c9b70 2024-06-10T01:04:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.960+0000 7fd6a8bfa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a01aa8e0 con 0x55dc498c4a00 2024-06-10T01:04:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.960+0000 7fd6a8bfa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a01a8730 con 0x55dc498c4010 2024-06-10T01:04:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.961+0000 7fd69effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc498c9b70 0x55dc498c2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.961+0000 7fd69effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc498c9b70 0x55dc498c2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57814/0 (socket says 172.21.15.37:57814) 2024-06-10T01:04:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.961+0000 7fd69effd640 1 -- 172.21.15.37:0/1681060271 learned_addr learned my addr 172.21.15.37:0/1681060271 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.961+0000 7fd69effd640 1 --2- 172.21.15.37:0/1681060271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc498c4010 0x55dc498ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.961+0000 7fd69f7fe640 1 --2- 172.21.15.37:0/1681060271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc498c4a00 0x55dc498d5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.961+0000 7fd69effd640 1 -- 172.21.15.37:0/1681060271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc498c4010 msgr2=0x55dc498ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.961+0000 7fd69effd640 1 --2- 172.21.15.37:0/1681060271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc498c4010 0x55dc498ce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.961+0000 7fd69effd640 1 -- 172.21.15.37:0/1681060271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc498c4a00 msgr2=0x55dc498d5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.961+0000 7fd69effd640 1 --2- 172.21.15.37:0/1681060271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc498c4a00 0x55dc498d5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.961+0000 7fd69effd640 1 -- 172.21.15.37:0/1681060271 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc498c47a0 con 0x55dc498c9b70 2024-06-10T01:04:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.961+0000 7fd69f7fe640 1 --2- 172.21.15.37:0/1681060271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc498c4a00 0x55dc498d5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:04:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.961+0000 7fd69effd640 1 --2- 172.21.15.37:0/1681060271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc498c9b70 0x55dc498c2530 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fd694014c20 tx=0x7fd69400c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8b5135bed7a0c9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.961+0000 7fd6857fa640 1 -- 172.21.15.37:0/1681060271 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69400c8b0 con 0x55dc498c9b70 2024-06-10T01:04:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.961+0000 7fd6857fa640 1 -- 172.21.15.37:0/1681060271 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6940141d0 con 0x55dc498c9b70 2024-06-10T01:04:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.961+0000 7fd6857fa640 1 -- 172.21.15.37:0/1681060271 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6940233a0 con 0x55dc498c9b70 2024-06-10T01:04:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.962+0000 7fd6a8bfa740 1 -- 172.21.15.37:0/1681060271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc498c9b70 msgr2=0x55dc498c2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.962+0000 7fd6a8bfa740 1 --2- 172.21.15.37:0/1681060271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc498c9b70 0x55dc498c2530 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fd694014c20 tx=0x7fd69400c4d0 comp rx=0 tx=0).stop 2024-06-10T01:04:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.962+0000 7fd6a8bfa740 1 -- 172.21.15.37:0/1681060271 shutdown_connections 2024-06-10T01:04:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.962+0000 7fd6a8bfa740 1 --2- 172.21.15.37:0/1681060271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc498c9b70 0x55dc498c2530 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.962+0000 7fd6a8bfa740 1 --2- 172.21.15.37:0/1681060271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc498c4a00 0x55dc498d5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.962+0000 7fd6a8bfa740 1 --2- 172.21.15.37:0/1681060271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc498c4010 0x55dc498ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.962+0000 7fd6a8bfa740 1 -- 172.21.15.37:0/1681060271 >> 172.21.15.37:0/1681060271 conn(0x55dc498bf680 msgr2=0x55dc498cde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.962+0000 7fd6a8bfa740 1 -- 172.21.15.37:0/1681060271 shutdown_connections 2024-06-10T01:04:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.962+0000 7fd6a8bfa740 1 -- 172.21.15.37:0/1681060271 wait complete. 2024-06-10T01:04:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.963+0000 7fd6a8bfa740 1 -- start start 2024-06-10T01:04:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.963+0000 7fd6a8bfa740 1 Processor -- start 2024-06-10T01:04:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.963+0000 7fd6a8bfa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc498c4010 0x55dc498d28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.963+0000 7fd6a8bfa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc498c4a00 0x55dc498d2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.963+0000 7fd6a8bfa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc498c9b70 0x55dc498d3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.963+0000 7fd6a8bfa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a01a8da0 con 0x55dc498c4010 2024-06-10T01:04:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.963+0000 7fd6a8bfa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a01aa210 con 0x55dc498c9b70 2024-06-10T01:04:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.963+0000 7fd6a8bfa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a01a9470 con 0x55dc498c4a00 2024-06-10T01:04:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.963+0000 7fd6a8bfa740 10 client.0 _ll_register_callbacks cb 0x7fd69400cdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.963+0000 7fd69effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc498c4010 0x55dc498d28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.963+0000 7fd69effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc498c4010 0x55dc498d28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57824/0 (socket says 172.21.15.37:57824) 2024-06-10T01:04:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.963+0000 7fd69effd640 1 -- 172.21.15.37:0/3529567452 learned_addr learned my addr 172.21.15.37:0/3529567452 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.963+0000 7fd69f7fe640 1 --2- 172.21.15.37:0/3529567452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc498c4a00 0x55dc498d2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.963+0000 7fd69effd640 1 --2- 172.21.15.37:0/3529567452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc498c9b70 0x55dc498d3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.963+0000 7fd69effd640 1 -- 172.21.15.37:0/3529567452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc498c4a00 msgr2=0x55dc498d2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.963+0000 7fd69effd640 1 --2- 172.21.15.37:0/3529567452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc498c4a00 0x55dc498d2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.963+0000 7fd69effd640 1 -- 172.21.15.37:0/3529567452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc498c9b70 msgr2=0x55dc498d3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.963+0000 7fd69effd640 1 --2- 172.21.15.37:0/3529567452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc498c9b70 0x55dc498d3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.964+0000 7fd69effd640 1 -- 172.21.15.37:0/3529567452 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc498d4390 con 0x55dc498c4010 2024-06-10T01:04:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.964+0000 7fd69effd640 1 --2- 172.21.15.37:0/3529567452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc498c4010 0x55dc498d28a0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fd694000c00 tx=0x7fd694005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa0f41861727e0c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.964+0000 7fd67effd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:04:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.964+0000 7fd67effd640 1 -- 172.21.15.37:0/3529567452 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd694023320 con 0x55dc498c4010 2024-06-10T01:04:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.964+0000 7fd67effd640 1 -- 172.21.15.37:0/3529567452 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fd6940234c0 con 0x55dc498c4010 2024-06-10T01:04:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.964+0000 7fd6a8bfa740 1 -- 172.21.15.37:0/3529567452 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6940141d0 con 0x55dc498c4010 2024-06-10T01:04:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.964+0000 7fd67effd640 1 -- 172.21.15.37:0/3529567452 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd694012450 con 0x55dc498c4010 2024-06-10T01:04:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.964+0000 7fd6a8bfa740 10 client.16800 fetch_fsmap learned FSMap version 29 2024-06-10T01:04:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.964+0000 7fd6a8bfa740 1 -- 172.21.15.37:0/3529567452 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dc498d56a0 con 0x55dc498c4010 2024-06-10T01:04:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.964+0000 7fd67effd640 1 -- 172.21.15.37:0/3529567452 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6940141d0 con 0x55dc498c4010 2024-06-10T01:04:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.965+0000 7fd67effd640 1 -- 172.21.15.37:0/3529567452 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fd694021530 con 0x55dc498c4010 2024-06-10T01:04:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.965+0000 7fd6a8bfa740 10 client.16800 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:04:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.965+0000 7fd6a8bfa740 10 client.16800 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:04:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.965+0000 7fd6a8bfa740 20 client.16800 populate_metadata read hostname 'smithi037' 2024-06-10T01:04:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.965+0000 7fd6a8bfa740 1 --2- 172.21.15.37:0/3529567452 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55dc49944ff0 0x55dc499473e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.965+0000 7fd6a8bfa740 4 client.16800 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:04:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.965+0000 7fd6a8bfa740 1 -- 172.21.15.37:0/3529567452 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dc49834c20 con 0x55dc49944ff0 2024-06-10T01:04:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.965+0000 7fd69f7fe640 1 --2- 172.21.15.37:0/3529567452 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55dc49944ff0 0x55dc499473e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.965+0000 7fd69f7fe640 1 --2- 172.21.15.37:0/3529567452 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55dc49944ff0 0x55dc499473e0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3a1aae1e73074e6 server_cookie=c24e3da2b42b5045 in_seq=0 out_seq=0 2024-06-10T01:04:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:30.965+0000 7fd67effd640 10 client.16800 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:31.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.050+0000 7fd8b1b6a700 1 -- 172.21.15.37:0/1250741568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8ac1030f0 msgr2=0x7fd8ac107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:31.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.050+0000 7fd8b1b6a700 1 --2- 172.21.15.37:0/1250741568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8ac1030f0 0x7fd8ac107110 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd894009920 tx=0x7fd894009c30 comp rx=0 tx=0).stop 2024-06-10T01:04:31.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.053+0000 7fd8b1b6a700 1 -- 172.21.15.37:0/1250741568 shutdown_connections 2024-06-10T01:04:31.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.053+0000 7fd8b1b6a700 1 --2- 172.21.15.37:0/1250741568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8ac107840 0x7fd8ac10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.053+0000 7fd8b1b6a700 1 --2- 172.21.15.37:0/1250741568 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8ac102740 0x7fd8ac102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.053+0000 7fd8b1b6a700 1 --2- 172.21.15.37:0/1250741568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8ac1030f0 0x7fd8ac107110 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.053+0000 7fd8b1b6a700 1 -- 172.21.15.37:0/1250741568 >> 172.21.15.37:0/1250741568 conn(0x7fd8ac0795f0 msgr2=0x7fd8ac079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:31.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.053+0000 7fd8b1b6a700 1 -- 172.21.15.37:0/1250741568 shutdown_connections 2024-06-10T01:04:31.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.053+0000 7fd8b1b6a700 1 -- 172.21.15.37:0/1250741568 wait complete. 2024-06-10T01:04:31.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.054+0000 7fd8b1b6a700 1 Processor -- start 2024-06-10T01:04:31.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.055+0000 7fd8b1b6a700 1 -- start start 2024-06-10T01:04:31.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.055+0000 7fd8b1b6a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8ac102740 0x7fd8ac19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:31.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.055+0000 7fd8b1b6a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8ac1030f0 0x7fd8ac19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:31.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.055+0000 7fd8b1b6a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8ac107840 0x7fd8ac198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:31.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.056+0000 7fd8b1b6a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac1127a0 con 0x7fd8ac102740 2024-06-10T01:04:31.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.056+0000 7fd8b1b6a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac112620 con 0x7fd8ac1030f0 2024-06-10T01:04:31.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.056+0000 7fd8b1b6a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac112920 con 0x7fd8ac107840 2024-06-10T01:04:31.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.056+0000 7fd8ab7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8ac102740 0x7fd8ac19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:31.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.056+0000 7fd8ab7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8ac102740 0x7fd8ac19e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57846/0 (socket says 172.21.15.37:57846) 2024-06-10T01:04:31.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.056+0000 7fd8ab7fe700 1 -- 172.21.15.37:0/2061907324 learned_addr learned my addr 172.21.15.37:0/2061907324 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:31.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.057+0000 7fd8ab7fe700 1 -- 172.21.15.37:0/2061907324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8ac107840 msgr2=0x7fd8ac198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:31.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.057+0000 7fd8ab7fe700 1 --2- 172.21.15.37:0/2061907324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8ac107840 0x7fd8ac198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.057+0000 7fd8ab7fe700 1 -- 172.21.15.37:0/2061907324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8ac1030f0 msgr2=0x7fd8ac19ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:31.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.057+0000 7fd8ab7fe700 1 --2- 172.21.15.37:0/2061907324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8ac1030f0 0x7fd8ac19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.057+0000 7fd8ab7fe700 1 -- 172.21.15.37:0/2061907324 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ac199200 con 0x7fd8ac102740 2024-06-10T01:04:31.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.057+0000 7fd8ab7fe700 1 --2- 172.21.15.37:0/2061907324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8ac102740 0x7fd8ac19e8e0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00e3c0 tx=0x7fd89c00e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:31.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.057+0000 7fd8a8ff9700 1 -- 172.21.15.37:0/2061907324 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89c0090d0 con 0x7fd8ac102740 2024-06-10T01:04:31.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.058+0000 7fd8b1b6a700 1 -- 172.21.15.37:0/2061907324 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8ac199490 con 0x7fd8ac102740 2024-06-10T01:04:31.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.058+0000 7fd8a8ff9700 1 -- 172.21.15.37:0/2061907324 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd89c009270 con 0x7fd8ac102740 2024-06-10T01:04:31.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.058+0000 7fd8b1b6a700 1 -- 172.21.15.37:0/2061907324 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8ac1a57b0 con 0x7fd8ac102740 2024-06-10T01:04:31.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.059+0000 7fd8a8ff9700 1 -- 172.21.15.37:0/2061907324 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89c0048a0 con 0x7fd8ac102740 2024-06-10T01:04:31.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.065+0000 7fd8bc3edb80 1 Processor -- start 2024-06-10T01:04:31.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.068+0000 7fd8bc3edb80 1 -- start start 2024-06-10T01:04:31.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.069+0000 7fd8bc3edb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56308fc815a0 0x56308fc81980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:31.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.069+0000 7fd8bc3edb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56308fc72e90 0x56308fc73310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:31.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.069+0000 7fd8bc3edb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56308fc73850 0x56308fc88240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:31.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.069+0000 7fd8bc3edb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56308fbc4910 con 0x56308fc73850 2024-06-10T01:04:31.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.069+0000 7fd8bc3edb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56308fb6c820 con 0x56308fc815a0 2024-06-10T01:04:31.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.069+0000 7fd8bc3edb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56308fbc08d0 con 0x56308fc72e90 2024-06-10T01:04:31.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.069+0000 7fd8a8ff9700 1 -- 172.21.15.37:0/2061907324 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd89c01b030 con 0x7fd8ac102740 2024-06-10T01:04:31.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.070+0000 7fd8aaffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56308fc73850 0x56308fc88240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:31.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.070+0000 7fd8aaffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56308fc73850 0x56308fc88240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57862/0 (socket says 172.21.15.37:57862) 2024-06-10T01:04:31.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.070+0000 7fd8aaffd700 1 -- 172.21.15.37:0/1824397534 learned_addr learned my addr 172.21.15.37:0/1824397534 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:31.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.070+0000 7fd8a8ff9700 1 --2- 172.21.15.37:0/2061907324 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd8980a2e40 0x7fd8980a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:31.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.070+0000 7fd8a8ff9700 1 -- 172.21.15.37:0/2061907324 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fd89c0994c0 con 0x7fd8ac102740 2024-06-10T01:04:31.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.072+0000 7fd8aaffd700 1 -- 172.21.15.37:0/1824397534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56308fc72e90 msgr2=0x56308fc73310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:04:31.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.072+0000 7fd8aaffd700 1 --2- 172.21.15.37:0/1824397534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56308fc72e90 0x56308fc73310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.072+0000 7fd8aaffd700 1 -- 172.21.15.37:0/1824397534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56308fc815a0 msgr2=0x56308fc81980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:31.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.073+0000 7fd8aaffd700 1 --2- 172.21.15.37:0/1824397534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56308fc815a0 0x56308fc81980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.073+0000 7fd8abfff700 1 --2- 172.21.15.37:0/2061907324 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd8980a2e40 0x7fd8980a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:31.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.073+0000 7fd8aaffd700 1 -- 172.21.15.37:0/1824397534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56308fb63050 con 0x56308fc73850 2024-06-10T01:04:31.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.074+0000 7fd8aaffd700 1 --2- 172.21.15.37:0/1824397534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56308fc73850 0x56308fc88240 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fd894005950 tx=0x7fd894009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b19d599c5c59c01e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:31.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.074+0000 7fd8abfff700 1 --2- 172.21.15.37:0/2061907324 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd8980a2e40 0x7fd8980a5300 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x56308fc6f760 tx=0x7fd8a4005ce0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:31.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.075+0000 7fd8a1ffb700 1 -- 172.21.15.37:0/1824397534 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd894010e10 con 0x56308fc73850 2024-06-10T01:04:31.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.075+0000 7fd8a1ffb700 1 -- 172.21.15.37:0/1824397534 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd89400f940 con 0x56308fc73850 2024-06-10T01:04:31.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.075+0000 7fd8a1ffb700 1 -- 172.21.15.37:0/1824397534 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8940038e0 con 0x56308fc73850 2024-06-10T01:04:31.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.076+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/1824397534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56308fc73850 msgr2=0x56308fc88240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:31.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.076+0000 7fd8bc3edb80 1 --2- 172.21.15.37:0/1824397534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56308fc73850 0x56308fc88240 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fd894005950 tx=0x7fd894009610 comp rx=0 tx=0).stop 2024-06-10T01:04:31.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.076+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/1824397534 shutdown_connections 2024-06-10T01:04:31.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.076+0000 7fd8bc3edb80 1 --2- 172.21.15.37:0/1824397534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56308fc73850 0x56308fc88240 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.077+0000 7fd8bc3edb80 1 --2- 172.21.15.37:0/1824397534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56308fc815a0 0x56308fc81980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.077+0000 7fd8bc3edb80 1 --2- 172.21.15.37:0/1824397534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56308fc72e90 0x56308fc73310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.077+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/1824397534 >> 172.21.15.37:0/1824397534 conn(0x56308fb9ddc0 msgr2=0x56308fc6ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:31.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.083+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/1824397534 shutdown_connections 2024-06-10T01:04:31.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.084+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/1824397534 wait complete. 2024-06-10T01:04:31.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.085+0000 7fd8bc3edb80 1 -- start start 2024-06-10T01:04:31.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.086+0000 7fd8bc3edb80 1 Processor -- start 2024-06-10T01:04:31.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.086+0000 7fd8bc3edb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56308fc72e90 0x56308fc87e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:31.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.086+0000 7fd8bc3edb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56308fc73850 0x56308fc85100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:31.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.087+0000 7fd8bc3edb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56308fc815a0 0x56308fc85640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:31.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.087+0000 7fd8bc3edb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56308fc88820 con 0x56308fc815a0 2024-06-10T01:04:31.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.087+0000 7fd8bc3edb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56308fb64cf0 con 0x56308fc72e90 2024-06-10T01:04:31.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.087+0000 7fd8bc3edb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56308fc889a0 con 0x56308fc73850 2024-06-10T01:04:31.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.088+0000 7fd8bc3edb80 10 client.0 _ll_register_callbacks cb 0x56308fb5ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:31.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.088+0000 7fd8abfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56308fc815a0 0x56308fc85640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:31.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.088+0000 7fd8abfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56308fc815a0 0x56308fc85640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57878/0 (socket says 172.21.15.37:57878) 2024-06-10T01:04:31.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.088+0000 7fd8abfff700 1 -- 172.21.15.37:0/4152891636 learned_addr learned my addr 172.21.15.37:0/4152891636 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:31.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.088+0000 7fd8ab7fe700 1 --2- 172.21.15.37:0/4152891636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56308fc72e90 0x56308fc87e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:31.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.089+0000 7fd8abfff700 1 -- 172.21.15.37:0/4152891636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56308fc73850 msgr2=0x56308fc85100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:04:31.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.089+0000 7fd8abfff700 1 --2- 172.21.15.37:0/4152891636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56308fc73850 0x56308fc85100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.089+0000 7fd8abfff700 1 -- 172.21.15.37:0/4152891636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56308fc72e90 msgr2=0x56308fc87e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:31.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.089+0000 7fd8abfff700 1 --2- 172.21.15.37:0/4152891636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56308fc72e90 0x56308fc87e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.089+0000 7fd8abfff700 1 -- 172.21.15.37:0/4152891636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56308fb139d0 con 0x56308fc815a0 2024-06-10T01:04:31.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.089+0000 7fd8abfff700 1 --2- 172.21.15.37:0/4152891636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56308fc815a0 0x56308fc85640 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd8a40106b0 tx=0x7fd8a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c4cb85a9097447 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:31.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.092+0000 7fd88affd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:04:31.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.092+0000 7fd88affd700 1 -- 172.21.15.37:0/4152891636 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4020070 con 0x56308fc815a0 2024-06-10T01:04:31.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.092+0000 7fd88affd700 1 -- 172.21.15.37:0/4152891636 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fd8a401c830 con 0x56308fc815a0 2024-06-10T01:04:31.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.092+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/4152891636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd89400f940 con 0x56308fc815a0 2024-06-10T01:04:31.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.096+0000 7fd88affd700 1 -- 172.21.15.37:0/4152891636 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd8a401ca30 con 0x56308fc815a0 2024-06-10T01:04:31.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.100+0000 7fd8bc3edb80 10 client.16812 fetch_fsmap learned FSMap version 29 2024-06-10T01:04:31.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.100+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/4152891636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56308fc93070 con 0x56308fc815a0 2024-06-10T01:04:31.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.100+0000 7fd88affd700 1 -- 172.21.15.37:0/4152891636 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8a401cbd0 con 0x56308fc815a0 2024-06-10T01:04:31.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.101+0000 7fd88affd700 1 -- 172.21.15.37:0/4152891636 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fd8a402f420 con 0x56308fc815a0 2024-06-10T01:04:31.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.101+0000 7fd8bc3edb80 10 client.16812 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:04:31.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.101+0000 7fd8bc3edb80 10 client.16812 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:04:31.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.101+0000 7fd8bc3edb80 20 client.16812 populate_metadata read hostname 'smithi037' 2024-06-10T01:04:31.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.101+0000 7fd8bc3edb80 1 --2- 172.21.15.37:0/4152891636 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56308fc95600 0x56308fc97a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:31.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.101+0000 7fd8bc3edb80 4 client.16812 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:04:31.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.101+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/4152891636 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd89400f940 con 0x56308fc95600 2024-06-10T01:04:31.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.104+0000 7fd8ab7fe700 1 --2- 172.21.15.37:0/4152891636 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56308fc95600 0x56308fc97a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:31.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.109+0000 7fd8ab7fe700 1 --2- 172.21.15.37:0/4152891636 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56308fc95600 0x56308fc97a60 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23ec9b7f75f7d981 server_cookie=23a9b14fa4c9975c in_seq=0 out_seq=0 2024-06-10T01:04:31.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.109+0000 7fd88affd700 10 client.16812 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:31.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.121+0000 7fd88affd700 1 -- 172.21.15.37:0/4152891636 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fd89400f940 con 0x56308fc95600 2024-06-10T01:04:31.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.121+0000 7fd88affd700 10 client.16812 handle_command_reply: tid=0 2024-06-10T01:04:31.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.121+0000 7fd8bc3edb80 1 client.16812 shutdown 2024-06-10T01:04:31.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.125+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/4152891636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56308fc815a0 msgr2=0x56308fc85640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:31.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.125+0000 7fd8bc3edb80 1 --2- 172.21.15.37:0/4152891636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56308fc815a0 0x56308fc85640 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd8a40106b0 tx=0x7fd8a4004750 comp rx=0 tx=0).stop 2024-06-10T01:04:31.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.126+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/4152891636 shutdown_connections 2024-06-10T01:04:31.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.126+0000 7fd8bc3edb80 1 --2- 172.21.15.37:0/4152891636 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56308fc95600 0x56308fc97a60 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.126+0000 7fd8bc3edb80 1 --2- 172.21.15.37:0/4152891636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56308fc815a0 0x56308fc85640 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.126+0000 7fd8bc3edb80 1 --2- 172.21.15.37:0/4152891636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56308fc72e90 0x56308fc87e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.126+0000 7fd8bc3edb80 1 --2- 172.21.15.37:0/4152891636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56308fc73850 0x56308fc85100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.126+0000 7fd88affd700 0 client.16812 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:31.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.126+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/4152891636 >> 172.21.15.37:0/4152891636 conn(0x56308fb9ddc0 msgr2=0x56308fc71e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:31.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.126+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/4152891636 shutdown_connections 2024-06-10T01:04:31.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.126+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/4152891636 wait complete. 2024-06-10T01:04:31.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.126+0000 7fd8bc3edb80 20 client.16812 trim_cache size 0 max 16384 2024-06-10T01:04:31.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.169+0000 7fd8bc3edb80 1 Processor -- start 2024-06-10T01:04:31.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.170+0000 7fd8bc3edb80 1 -- start start 2024-06-10T01:04:31.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.170+0000 7fd8bc3edb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56308fc72e90 0x56308fc83a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:31.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.170+0000 7fd8bc3edb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56308fc73850 0x56308fc6d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:31.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.171+0000 7fd8bc3edb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56308fc815a0 0x56308fc6dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:31.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.171+0000 7fd8bc3edb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c011640 con 0x56308fc73850 2024-06-10T01:04:31.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.171+0000 7fd8aaffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56308fc73850 0x56308fc6d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:31.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.171+0000 7fd8aaffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56308fc73850 0x56308fc6d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57886/0 (socket says 172.21.15.37:57886) 2024-06-10T01:04:31.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.171+0000 7fd8aaffd700 1 -- 172.21.15.37:0/846157274 learned_addr learned my addr 172.21.15.37:0/846157274 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:31.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.171+0000 7fd8abfff700 1 --2- 172.21.15.37:0/846157274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56308fc815a0 0x56308fc6dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:31.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.171+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/846157274 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c00eed0 con 0x56308fc72e90 2024-06-10T01:04:31.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.171+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/846157274 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c00cd20 con 0x56308fc815a0 2024-06-10T01:04:31.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.172+0000 7fd8aaffd700 1 -- 172.21.15.37:0/846157274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56308fc815a0 msgr2=0x56308fc6dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:31.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.172+0000 7fd8aaffd700 1 --2- 172.21.15.37:0/846157274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56308fc815a0 0x56308fc6dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.172+0000 7fd8aaffd700 1 -- 172.21.15.37:0/846157274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56308fc72e90 msgr2=0x56308fc83a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:31.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.172+0000 7fd8aaffd700 1 --2- 172.21.15.37:0/846157274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56308fc72e90 0x56308fc83a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.172+0000 7fd8aaffd700 1 -- 172.21.15.37:0/846157274 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56308fc83690 con 0x56308fc73850 2024-06-10T01:04:31.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.172+0000 7fd8abfff700 1 --2- 172.21.15.37:0/846157274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56308fc815a0 0x56308fc6dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:31.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.172+0000 7fd8aaffd700 1 --2- 172.21.15.37:0/846157274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56308fc73850 0x56308fc6d830 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd894011010 tx=0x7fd894005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe7a0f2a051f628 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:31.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.173+0000 7fd8a17fa700 1 -- 172.21.15.37:0/846157274 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89400f520 con 0x56308fc73850 2024-06-10T01:04:31.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.173+0000 7fd8a17fa700 1 -- 172.21.15.37:0/846157274 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd894005040 con 0x56308fc73850 2024-06-10T01:04:31.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.173+0000 7fd8a17fa700 1 -- 172.21.15.37:0/846157274 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd894003af0 con 0x56308fc73850 2024-06-10T01:04:31.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.174+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/846157274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56308fc73850 msgr2=0x56308fc6d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:31.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.174+0000 7fd8bc3edb80 1 --2- 172.21.15.37:0/846157274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56308fc73850 0x56308fc6d830 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd894011010 tx=0x7fd894005bf0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.174+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/846157274 shutdown_connections 2024-06-10T01:04:31.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.174+0000 7fd8bc3edb80 1 --2- 172.21.15.37:0/846157274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56308fc815a0 0x56308fc6dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.174+0000 7fd8bc3edb80 1 --2- 172.21.15.37:0/846157274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56308fc72e90 0x56308fc83a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.174+0000 7fd8bc3edb80 1 --2- 172.21.15.37:0/846157274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56308fc73850 0x56308fc6d830 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.174+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/846157274 >> 172.21.15.37:0/846157274 conn(0x56308fb9ddc0 msgr2=0x56308fc725f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:31.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.175+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/846157274 shutdown_connections 2024-06-10T01:04:31.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.175+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/846157274 wait complete. 2024-06-10T01:04:31.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.176+0000 7fd8bc3edb80 1 -- start start 2024-06-10T01:04:31.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.177+0000 7fd8bc3edb80 1 Processor -- start 2024-06-10T01:04:31.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.177+0000 7fd8bc3edb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56308fc72e90 0x56308fc8e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:31.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.177+0000 7fd8bc3edb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56308fc73850 0x56308fc8ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:31.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.178+0000 7fd8bc3edb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56308fc815a0 0x56308fc8b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:31.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.178+0000 7fd8bc3edb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c00d390 con 0x56308fc73850 2024-06-10T01:04:31.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.178+0000 7fd8bc3edb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c00e800 con 0x56308fc72e90 2024-06-10T01:04:31.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.178+0000 7fd8abfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56308fc815a0 0x56308fc8b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:31.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.178+0000 7fd8abfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56308fc815a0 0x56308fc8b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:55754/0 (socket says 172.21.15.37:55754) 2024-06-10T01:04:31.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.178+0000 7fd8abfff700 1 -- 172.21.15.37:0/4016684343 learned_addr learned my addr 172.21.15.37:0/4016684343 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:31.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.178+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/4016684343 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c00da60 con 0x56308fc815a0 2024-06-10T01:04:31.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.178+0000 7fd8bc3edb80 10 client.0 _ll_register_callbacks cb 0x56308fb5ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:31.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.178+0000 7fd8abfff700 1 -- 172.21.15.37:0/4016684343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56308fc72e90 msgr2=0x56308fc8e2e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:04:31.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.179+0000 7fd8abfff700 1 --2- 172.21.15.37:0/4016684343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56308fc72e90 0x56308fc8e2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.179+0000 7fd8abfff700 1 -- 172.21.15.37:0/4016684343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56308fc73850 msgr2=0x56308fc8ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:31.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.179+0000 7fd8abfff700 1 --2- 172.21.15.37:0/4016684343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56308fc73850 0x56308fc8ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:31.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.179+0000 7fd8abfff700 1 -- 172.21.15.37:0/4016684343 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56308fc8b980 con 0x56308fc815a0 2024-06-10T01:04:31.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.179+0000 7fd8abfff700 1 --2- 172.21.15.37:0/4016684343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56308fc815a0 0x56308fc8b140 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4010e00 tx=0x7fd8a402fac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff6c2e2e30ed0a5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:31.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.179+0000 7fd88affd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:04:31.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.180+0000 7fd88affd700 1 -- 172.21.15.37:0/4016684343 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4035040 con 0x56308fc815a0 2024-06-10T01:04:31.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.180+0000 7fd88affd700 1 -- 172.21.15.37:0/4016684343 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fd8a401cdd0 con 0x56308fc815a0 2024-06-10T01:04:31.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.180+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/4016684343 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd894005040 con 0x56308fc815a0 2024-06-10T01:04:31.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.180+0000 7fd88affd700 1 -- 172.21.15.37:0/4016684343 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd8a4046050 con 0x56308fc815a0 2024-06-10T01:04:31.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.180+0000 7fd8bc3edb80 10 client.26353 fetch_fsmap learned FSMap version 29 2024-06-10T01:04:31.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.180+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/4016684343 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56308fd012f0 con 0x56308fc815a0 2024-06-10T01:04:31.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.181+0000 7fd88affd700 1 -- 172.21.15.37:0/4016684343 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd894005040 con 0x56308fc815a0 2024-06-10T01:04:31.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.181+0000 7fd88affd700 1 -- 172.21.15.37:0/4016684343 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fd8a4041030 con 0x56308fc815a0 2024-06-10T01:04:31.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.181+0000 7fd8bc3edb80 10 client.26353 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:04:31.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.181+0000 7fd8bc3edb80 10 client.26353 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:04:31.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.181+0000 7fd8bc3edb80 20 client.26353 populate_metadata read hostname 'smithi037' 2024-06-10T01:04:31.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.181+0000 7fd8bc3edb80 1 --2- 172.21.15.37:0/4016684343 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56308fc95600 0x56308fd03ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:31.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.181+0000 7fd8bc3edb80 4 client.26353 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "11dd31af", "release": true, "await": true}] 2024-06-10T01:04:31.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.181+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/4016684343 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "quiesce db", "set_id": "11dd31af", "release": true, "await": true}) -- 0x56308fbd1e10 con 0x56308fc95600 2024-06-10T01:04:31.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.183+0000 7fd8ab7fe700 1 --2- 172.21.15.37:0/4016684343 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56308fc95600 0x56308fd03ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:31.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.184+0000 7fd8ab7fe700 1 --2- 172.21.15.37:0/4016684343 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56308fc95600 0x56308fd03ac0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a0bf96ca81b4a11 server_cookie=97754fcccfc11d9 in_seq=0 out_seq=0 2024-06-10T01:04:31.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:31.184+0000 7fd88affd700 10 client.26353 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:32.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:32 smithi094 ceph-mon[36708]: scrub summary: idle 2024-06-10T01:04:32.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:32 smithi094 ceph-mon[36708]: pgmap v666: 129 pgs: 129 active+clean; 2.6 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 104 KiB/s wr, 3.35k op/s 2024-06-10T01:04:32.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:32 smithi037 ceph-mon[32192]: scrub summary: idle 2024-06-10T01:04:32.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:32 smithi037 ceph-mon[32192]: pgmap v666: 129 pgs: 129 active+clean; 2.6 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 104 KiB/s wr, 3.35k op/s 2024-06-10T01:04:32.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:32 smithi167 ceph-mon[36917]: scrub summary: idle 2024-06-10T01:04:32.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:32 smithi167 ceph-mon[36917]: pgmap v666: 129 pgs: 129 active+clean; 2.6 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 104 KiB/s wr, 3.35k op/s 2024-06-10T01:04:32.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:32.934+0000 7fd88affd700 1 -- 172.21.15.37:0/4016684343 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56308fbd1e10 con 0x56308fc95600 2024-06-10T01:04:32.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:32.934+0000 7fd88affd700 10 client.26353 handle_command_reply: tid=0 2024-06-10T01:04:32.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:32.934+0000 7fd8bc3edb80 1 client.26353 shutdown 2024-06-10T01:04:32.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:32.934+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/4016684343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56308fc815a0 msgr2=0x56308fc8b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:32.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:32.934+0000 7fd8bc3edb80 1 --2- 172.21.15.37:0/4016684343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56308fc815a0 0x56308fc8b140 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4010e00 tx=0x7fd8a402fac0 comp rx=0 tx=0).stop 2024-06-10T01:04:32.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:32.934+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/4016684343 shutdown_connections 2024-06-10T01:04:32.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:32.934+0000 7fd8bc3edb80 1 --2- 172.21.15.37:0/4016684343 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56308fc95600 0x56308fd03ac0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:32.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:32.934+0000 7fd8bc3edb80 1 --2- 172.21.15.37:0/4016684343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56308fc815a0 0x56308fc8b140 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:32.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:32.934+0000 7fd8bc3edb80 1 --2- 172.21.15.37:0/4016684343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56308fc72e90 0x56308fc8e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:32.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:32.934+0000 7fd8bc3edb80 1 --2- 172.21.15.37:0/4016684343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56308fc73850 0x56308fc8ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:32.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:32.934+0000 7fd88affd700 0 client.26353 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:32.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:32.934+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/4016684343 >> 172.21.15.37:0/4016684343 conn(0x56308fb9ddc0 msgr2=0x56308fc679c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:32.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:32.934+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/4016684343 shutdown_connections 2024-06-10T01:04:32.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:32.934+0000 7fd8bc3edb80 1 -- 172.21.15.37:0/4016684343 wait complete. 2024-06-10T01:04:32.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:32.935+0000 7fd8bc3edb80 20 client.26353 trim_cache size 0 max 16384 2024-06-10T01:04:32.934 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:04:32.934 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 29, 2024-06-10T01:04:32.934 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24472, 2024-06-10T01:04:32.934 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 20, 2024-06-10T01:04:32.934 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-06-10T01:04:32.934 INFO:teuthology.orchestra.run.smithi037.stdout: "11dd31af": { 2024-06-10T01:04:32.934 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 20, 2024-06-10T01:04:32.935 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-06-10T01:04:32.935 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:04:32.935 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-06-10T01:04:32.935 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:04:32.935 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:04:32.935 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-06-10T01:04:32.935 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 149.4, 2024-06-10T01:04:32.935 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-06-10T01:04:32.935 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-06-10T01:04:32.935 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-06-10T01:04:32.935 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:04:32.935 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-06-10T01:04:32.935 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:04:32.935 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:04:32.936 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:04:32.936 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:04:32.936 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:04:32.936 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:04:32.936 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:04:32.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:32.935+0000 7fd8b1b6a700 1 -- 172.21.15.37:0/2061907324 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd8980a2e40 msgr2=0x7fd8980a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:32.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:32.935+0000 7fd8b1b6a700 1 --2- 172.21.15.37:0/2061907324 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd8980a2e40 0x7fd8980a5300 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x56308fc6f760 tx=0x7fd8a4005ce0 comp rx=0 tx=0).stop 2024-06-10T01:04:32.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:32.935+0000 7fd8b1b6a700 1 -- 172.21.15.37:0/2061907324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8ac102740 msgr2=0x7fd8ac19e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:32.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:32.935+0000 7fd8b1b6a700 1 --2- 172.21.15.37:0/2061907324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8ac102740 0x7fd8ac19e8e0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00e3c0 tx=0x7fd89c00e780 comp rx=0 tx=0).stop 2024-06-10T01:04:32.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:32.936+0000 7fd8b1b6a700 1 -- 172.21.15.37:0/2061907324 shutdown_connections 2024-06-10T01:04:32.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:32.936+0000 7fd8b1b6a700 1 --2- 172.21.15.37:0/2061907324 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd8980a2e40 0x7fd8980a5300 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:32.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:32.936+0000 7fd8b1b6a700 1 --2- 172.21.15.37:0/2061907324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8ac107840 0x7fd8ac198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:32.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:32.936+0000 7fd8b1b6a700 1 --2- 172.21.15.37:0/2061907324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8ac102740 0x7fd8ac19e8e0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:32.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:32.936+0000 7fd8b1b6a700 1 --2- 172.21.15.37:0/2061907324 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8ac1030f0 0x7fd8ac19ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:32.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:32.936+0000 7fd8b1b6a700 1 -- 172.21.15.37:0/2061907324 >> 172.21.15.37:0/2061907324 conn(0x7fd8ac0795f0 msgr2=0x7fd8ac100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:32.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:32.936+0000 7fd8b1b6a700 1 -- 172.21.15.37:0/2061907324 shutdown_connections 2024-06-10T01:04:32.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:32.936+0000 7fd8b1b6a700 1 -- 172.21.15.37:0/2061907324 wait complete. 2024-06-10T01:04:33.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:33.568+0000 7fd67effd640 1 -- 172.21.15.37:0/3529567452 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55dc49834c20 con 0x55dc49944ff0 2024-06-10T01:04:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:33.568+0000 7fd67effd640 10 client.16800 handle_command_reply: tid=0 2024-06-10T01:04:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:33.568+0000 7fd6a8bfa740 1 client.16800 shutdown 2024-06-10T01:04:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:33.568+0000 7fd6a8bfa740 1 -- 172.21.15.37:0/3529567452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc498c4010 msgr2=0x55dc498d28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:33.568+0000 7fd6a8bfa740 1 --2- 172.21.15.37:0/3529567452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc498c4010 0x55dc498d28a0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fd694000c00 tx=0x7fd694005850 comp rx=0 tx=0).stop 2024-06-10T01:04:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:33.568+0000 7fd6a8bfa740 1 -- 172.21.15.37:0/3529567452 shutdown_connections 2024-06-10T01:04:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:33.568+0000 7fd6a8bfa740 1 --2- 172.21.15.37:0/3529567452 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55dc49944ff0 0x55dc499473e0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:33.568+0000 7fd6a8bfa740 1 --2- 172.21.15.37:0/3529567452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc498c9b70 0x55dc498d3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:33.568+0000 7fd6a8bfa740 1 --2- 172.21.15.37:0/3529567452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc498c4a00 0x55dc498d2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:33.568+0000 7fd67effd640 0 client.16800 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:33.568+0000 7fd6a8bfa740 1 --2- 172.21.15.37:0/3529567452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc498c4010 0x55dc498d28a0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:33.568+0000 7fd6a8bfa740 1 -- 172.21.15.37:0/3529567452 >> 172.21.15.37:0/3529567452 conn(0x55dc498bf680 msgr2=0x55dc498c2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:33.568+0000 7fd6a8bfa740 1 -- 172.21.15.37:0/3529567452 shutdown_connections 2024-06-10T01:04:33.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:33.568+0000 7fd6a8bfa740 1 -- 172.21.15.37:0/3529567452 wait complete. 2024-06-10T01:04:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:33.568+0000 7fd6a8bfa740 20 client.16800 trim_cache size 0 max 16384 2024-06-10T01:04:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:04:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:04:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:04:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:33.569+0000 7fd6a5eee640 1 -- 172.21.15.37:0/275475133 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd6880a2c30 msgr2=0x7fd6880a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:33.569+0000 7fd6a5eee640 1 --2- 172.21.15.37:0/275475133 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd6880a2c30 0x7fd6880a50f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd6980ccd90 tx=0x7fd6980cf980 comp rx=0 tx=0).stop 2024-06-10T01:04:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:33.569+0000 7fd6a5eee640 1 -- 172.21.15.37:0/275475133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6a01024b0 msgr2=0x7fd6a019f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:33.569+0000 7fd6a5eee640 1 --2- 172.21.15.37:0/275475133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6a01024b0 0x7fd6a019f260 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd69800eaa0 tx=0x7fd69800ef60 comp rx=0 tx=0).stop 2024-06-10T01:04:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:33.569+0000 7fd6a5eee640 1 -- 172.21.15.37:0/275475133 shutdown_connections 2024-06-10T01:04:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:33.569+0000 7fd6a5eee640 1 --2- 172.21.15.37:0/275475133 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd6880a2c30 0x7fd6880a50f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:33.569+0000 7fd6a5eee640 1 --2- 172.21.15.37:0/275475133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6a01024b0 0x7fd6a019f260 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:33.569+0000 7fd6a5eee640 1 --2- 172.21.15.37:0/275475133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6a0101af0 0x7fd6a0198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:33.569+0000 7fd6a5eee640 1 --2- 172.21.15.37:0/275475133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6a0100900 0x7fd6a0197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:33.569+0000 7fd6a5eee640 1 -- 172.21.15.37:0/275475133 >> 172.21.15.37:0/275475133 conn(0x7fd6a00fbe70 msgr2=0x7fd6a00fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:33.569+0000 7fd6a5eee640 1 -- 172.21.15.37:0/275475133 shutdown_connections 2024-06-10T01:04:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:33.569+0000 7fd6a5eee640 1 -- 172.21.15.37:0/275475133 wait complete. 2024-06-10T01:04:33.683 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '11dd31af', total seconds elapsed: 39.4 2024-06-10T01:04:34.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:34 smithi094 ceph-mon[36708]: pgmap v667: 129 pgs: 129 active+clean; 2.6 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 70 KiB/s wr, 3.34k op/s 2024-06-10T01:04:34.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:34 smithi037 ceph-mon[32192]: pgmap v667: 129 pgs: 129 active+clean; 2.6 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 70 KiB/s wr, 3.34k op/s 2024-06-10T01:04:34.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:34 smithi167 ceph-mon[36917]: pgmap v667: 129 pgs: 129 active+clean; 2.6 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 70 KiB/s wr, 3.34k op/s 2024-06-10T01:04:36.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:36 smithi094 ceph-mon[36708]: pgmap v668: 129 pgs: 129 active+clean; 3.9 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 129 KiB/s wr, 3.20k op/s 2024-06-10T01:04:36.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:36 smithi037 ceph-mon[32192]: pgmap v668: 129 pgs: 129 active+clean; 3.9 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 129 KiB/s wr, 3.20k op/s 2024-06-10T01:04:36.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:36 smithi167 ceph-mon[36917]: pgmap v668: 129 pgs: 129 active+clean; 3.9 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 129 KiB/s wr, 3.20k op/s 2024-06-10T01:04:38.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:38 smithi094 ceph-mon[36708]: pgmap v669: 129 pgs: 129 active+clean; 3.9 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 143 KiB/s wr, 2.47k op/s 2024-06-10T01:04:38.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:38 smithi167 ceph-mon[36917]: pgmap v669: 129 pgs: 129 active+clean; 3.9 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 143 KiB/s wr, 2.47k op/s 2024-06-10T01:04:38.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:38 smithi037 ceph-mon[32192]: pgmap v669: 129 pgs: 129 active+clean; 3.9 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 143 KiB/s wr, 2.47k op/s 2024-06-10T01:04:40.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:40 smithi094 ceph-mon[36708]: pgmap v670: 129 pgs: 129 active+clean; 9.8 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 779 KiB/s wr, 2.34k op/s 2024-06-10T01:04:40.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:40 smithi037 ceph-mon[32192]: pgmap v670: 129 pgs: 129 active+clean; 9.8 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 779 KiB/s wr, 2.34k op/s 2024-06-10T01:04:40.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:40 smithi167 ceph-mon[36917]: pgmap v670: 129 pgs: 129 active+clean; 9.8 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 779 KiB/s wr, 2.34k op/s 2024-06-10T01:04:42.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:42 smithi094 ceph-mon[36708]: pgmap v671: 129 pgs: 129 active+clean; 9.8 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 830 KiB/s wr, 400 op/s 2024-06-10T01:04:42.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:42 smithi037 ceph-mon[32192]: pgmap v671: 129 pgs: 129 active+clean; 9.8 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 830 KiB/s wr, 400 op/s 2024-06-10T01:04:42.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:42 smithi167 ceph-mon[36917]: pgmap v671: 129 pgs: 129 active+clean; 9.8 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 830 KiB/s wr, 400 op/s 2024-06-10T01:04:44.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:44 smithi094 ceph-mon[36708]: pgmap v672: 129 pgs: 129 active+clean; 11 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 960 KiB/s wr, 265 op/s 2024-06-10T01:04:44.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:44 smithi037 ceph-mon[32192]: pgmap v672: 129 pgs: 129 active+clean; 11 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 960 KiB/s wr, 265 op/s 2024-06-10T01:04:44.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:44 smithi167 ceph-mon[36917]: pgmap v672: 129 pgs: 129 active+clean; 11 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 960 KiB/s wr, 265 op/s 2024-06-10T01:04:46.392 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:46 smithi094 ceph-mon[36708]: pgmap v673: 129 pgs: 129 active+clean; 14 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.3 MiB/s wr, 211 op/s 2024-06-10T01:04:46.392 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:46 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:04:46.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:46 smithi037 ceph-mon[32192]: pgmap v673: 129 pgs: 129 active+clean; 14 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.3 MiB/s wr, 211 op/s 2024-06-10T01:04:46.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:46 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:04:46.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:46 smithi167 ceph-mon[36917]: pgmap v673: 129 pgs: 129 active+clean; 14 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.3 MiB/s wr, 211 op/s 2024-06-10T01:04:46.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:46 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:04:46.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.662+0000 7f335d77e640 1 -- 172.21.15.37:0/2318007926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33581035b0 msgr2=0x7f3358109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:46.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.662+0000 7f335d77e640 1 --2- 172.21.15.37:0/2318007926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33581035b0 0x7f3358109e40 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3350009ca0 tx=0x7f335002a8c0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.662+0000 7f335d77e640 1 -- 172.21.15.37:0/2318007926 shutdown_connections 2024-06-10T01:04:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.662+0000 7f335d77e640 1 --2- 172.21.15.37:0/2318007926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33581035b0 0x7f3358109e40 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.662+0000 7f335d77e640 1 --2- 172.21.15.37:0/2318007926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3358102bf0 0x7f3358103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.662+0000 7f335d77e640 1 --2- 172.21.15.37:0/2318007926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3358101a00 0x7f3358101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.662+0000 7f335d77e640 1 -- 172.21.15.37:0/2318007926 >> 172.21.15.37:0/2318007926 conn(0x7f33580fcf90 msgr2=0x7f33580ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.662+0000 7f335d77e640 1 -- 172.21.15.37:0/2318007926 shutdown_connections 2024-06-10T01:04:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.662+0000 7f335d77e640 1 -- 172.21.15.37:0/2318007926 wait complete. 2024-06-10T01:04:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.662+0000 7f335d77e640 1 Processor -- start 2024-06-10T01:04:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.663+0000 7f335d77e640 1 -- start start 2024-06-10T01:04:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.663+0000 7f335d77e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3358101a00 0x7f3358197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.663+0000 7f335d77e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3358102bf0 0x7f3358198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.663+0000 7f335d77e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33581035b0 0x7f335819f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.663+0000 7f335d77e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f335810d680 con 0x7f3358101a00 2024-06-10T01:04:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.663+0000 7f335d77e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f335810d500 con 0x7f33581035b0 2024-06-10T01:04:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.663+0000 7f335d77e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f335810d800 con 0x7f3358102bf0 2024-06-10T01:04:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.663+0000 7f3356ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3358101a00 0x7f3358197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.663+0000 7f3356ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3358101a00 0x7f3358197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47890/0 (socket says 172.21.15.37:47890) 2024-06-10T01:04:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.663+0000 7f3356ffd640 1 -- 172.21.15.37:0/3240670460 learned_addr learned my addr 172.21.15.37:0/3240670460 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.663+0000 7f33567fc640 1 --2- 172.21.15.37:0/3240670460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3358102bf0 0x7f3358198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.663+0000 7f33577fe640 1 --2- 172.21.15.37:0/3240670460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33581035b0 0x7f335819f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.663+0000 7f3356ffd640 1 -- 172.21.15.37:0/3240670460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3358102bf0 msgr2=0x7f3358198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.663+0000 7f3356ffd640 1 --2- 172.21.15.37:0/3240670460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3358102bf0 0x7f3358198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.663+0000 7f3356ffd640 1 -- 172.21.15.37:0/3240670460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33581035b0 msgr2=0x7f335819f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.663+0000 7f3356ffd640 1 --2- 172.21.15.37:0/3240670460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33581035b0 0x7f335819f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.663+0000 7f3356ffd640 1 -- 172.21.15.37:0/3240670460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f335819f7f0 con 0x7f3358101a00 2024-06-10T01:04:46.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.663+0000 7f3356ffd640 1 --2- 172.21.15.37:0/3240670460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3358101a00 0x7f3358197bd0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f334800e9c0 tx=0x7f334800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:46.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.664+0000 7f333ffff640 1 -- 172.21.15.37:0/3240670460 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334800cd80 con 0x7f3358101a00 2024-06-10T01:04:46.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.664+0000 7f335d77e640 1 -- 172.21.15.37:0/3240670460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f335819fae0 con 0x7f3358101a00 2024-06-10T01:04:46.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.664+0000 7f333ffff640 1 -- 172.21.15.37:0/3240670460 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33480035f0 con 0x7f3358101a00 2024-06-10T01:04:46.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.664+0000 7f333ffff640 1 -- 172.21.15.37:0/3240670460 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3348010690 con 0x7f3358101a00 2024-06-10T01:04:46.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.664+0000 7f335d77e640 1 -- 172.21.15.37:0/3240670460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33581a0030 con 0x7f3358101a00 2024-06-10T01:04:46.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.666+0000 7f333ffff640 1 -- 172.21.15.37:0/3240670460 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f33480037a0 con 0x7f3358101a00 2024-06-10T01:04:46.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.666+0000 7f336056c740 1 Processor -- start 2024-06-10T01:04:46.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.666+0000 7f336056c740 1 -- start start 2024-06-10T01:04:46.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.667+0000 7f336056c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566eecac010 0x5566eecac430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:46.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.667+0000 7f336056c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eecaca00 0x5566eecb1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:46.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.667+0000 7f336056c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566eecb1b70 0x5566eecb4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:46.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.667+0000 7f336056c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5566eec0f4f0 con 0x5566eecb1b70 2024-06-10T01:04:46.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.667+0000 7f336056c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5566eebad870 con 0x5566eecac010 2024-06-10T01:04:46.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.667+0000 7f336056c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5566eec0b4b0 con 0x5566eecaca00 2024-06-10T01:04:46.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.667+0000 7f33577fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566eecac010 0x5566eecac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:46.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.667+0000 7f3356ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eecaca00 0x5566eecb1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:46.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.667+0000 7f33577fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566eecac010 0x5566eecac430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:38524/0 (socket says 172.21.15.37:38524) 2024-06-10T01:04:46.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.667+0000 7f33567fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566eecb1b70 0x5566eecb4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:46.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.667+0000 7f33577fe640 1 -- 172.21.15.37:0/3667975078 learned_addr learned my addr 172.21.15.37:0/3667975078 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:46.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.667+0000 7f33577fe640 1 -- 172.21.15.37:0/3667975078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eecaca00 msgr2=0x5566eecb1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:46.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.667+0000 7f33577fe640 1 --2- 172.21.15.37:0/3667975078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eecaca00 0x5566eecb1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.667+0000 7f333ffff640 1 --2- 172.21.15.37:0/3240670460 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f33400a2c30 0x7f33400a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:46.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.667+0000 7f33577fe640 1 -- 172.21.15.37:0/3667975078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566eecb1b70 msgr2=0x5566eecb4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:46.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.667+0000 7f33577fe640 1 --2- 172.21.15.37:0/3667975078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566eecb1b70 0x5566eecb4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.667+0000 7f33577fe640 1 -- 172.21.15.37:0/3667975078 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566eebc99b0 con 0x5566eecac010 2024-06-10T01:04:46.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.667+0000 7f33567fc640 1 --2- 172.21.15.37:0/3667975078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566eecb1b70 0x5566eecb4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:46.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.667+0000 7f333ffff640 1 -- 172.21.15.37:0/3240670460 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f33480cf7b0 con 0x7f3358101a00 2024-06-10T01:04:46.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.667+0000 7f33577fe640 1 --2- 172.21.15.37:0/3240670460 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f33400a2c30 0x7f33400a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:46.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.667+0000 7f33577fe640 1 --2- 172.21.15.37:0/3667975078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566eecac010 0x5566eecac430 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3350009ca0 tx=0x7f3350009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad6cf86c64a213f2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:46.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.668+0000 7f333d7fa640 1 -- 172.21.15.37:0/3667975078 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3350031cb0 con 0x5566eecac010 2024-06-10T01:04:46.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.668+0000 7f333d7fa640 1 -- 172.21.15.37:0/3667975078 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3350036040 con 0x5566eecac010 2024-06-10T01:04:46.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.668+0000 7f333d7fa640 1 -- 172.21.15.37:0/3667975078 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3350045400 con 0x5566eecac010 2024-06-10T01:04:46.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.669+0000 7f336056c740 1 -- 172.21.15.37:0/3667975078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566eecac010 msgr2=0x5566eecac430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:46.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.669+0000 7f33577fe640 1 --2- 172.21.15.37:0/3240670460 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f33400a2c30 0x7f33400a50f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f335004a000 tx=0x7f335004b000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:46.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.669+0000 7f336056c740 1 --2- 172.21.15.37:0/3667975078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566eecac010 0x5566eecac430 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3350009ca0 tx=0x7f3350009360 comp rx=0 tx=0).stop 2024-06-10T01:04:46.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.669+0000 7f336056c740 1 -- 172.21.15.37:0/3667975078 shutdown_connections 2024-06-10T01:04:46.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.669+0000 7f336056c740 1 --2- 172.21.15.37:0/3667975078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566eecb1b70 0x5566eecb4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.669+0000 7f336056c740 1 --2- 172.21.15.37:0/3667975078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eecaca00 0x5566eecb1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.669+0000 7f336056c740 1 --2- 172.21.15.37:0/3667975078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566eecac010 0x5566eecac430 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.669+0000 7f336056c740 1 -- 172.21.15.37:0/3667975078 >> 172.21.15.37:0/3667975078 conn(0x5566eeca7680 msgr2=0x5566eeca7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:46.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.669+0000 7f336056c740 1 -- 172.21.15.37:0/3667975078 shutdown_connections 2024-06-10T01:04:46.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.669+0000 7f336056c740 1 -- 172.21.15.37:0/3667975078 wait complete. 2024-06-10T01:04:46.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.669+0000 7f336056c740 1 -- start start 2024-06-10T01:04:46.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.669+0000 7f336056c740 1 Processor -- start 2024-06-10T01:04:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.669+0000 7f336056c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eecac010 0x5566eecb6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.669+0000 7f336056c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566eecaca00 0x5566eecb6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.669+0000 7f336056c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566eecb1b70 0x5566eecbd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.669+0000 7f336056c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5566eecb4940 con 0x5566eecaca00 2024-06-10T01:04:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.669+0000 7f336056c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5566eeb98ae0 con 0x5566eecb1b70 2024-06-10T01:04:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.669+0000 7f336056c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5566eecb4ac0 con 0x5566eecac010 2024-06-10T01:04:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.669+0000 7f336056c740 10 client.0 _ll_register_callbacks cb 0x5566eebd19b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.670+0000 7f3356ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eecac010 0x5566eecb6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.670+0000 7f33567fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566eecaca00 0x5566eecb6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.670+0000 7f3356ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eecac010 0x5566eecb6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:36442/0 (socket says 172.21.15.37:36442) 2024-06-10T01:04:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.670+0000 7f33567fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566eecaca00 0x5566eecb6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47904/0 (socket says 172.21.15.37:47904) 2024-06-10T01:04:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.670+0000 7f3356ffd640 1 -- 172.21.15.37:0/700734402 learned_addr learned my addr 172.21.15.37:0/700734402 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.670+0000 7f33577fe640 1 --2- 172.21.15.37:0/700734402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566eecb1b70 0x5566eecbd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.670+0000 7f33567fc640 1 -- 172.21.15.37:0/700734402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eecac010 msgr2=0x5566eecb6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.670+0000 7f33567fc640 1 --2- 172.21.15.37:0/700734402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eecac010 0x5566eecb6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.670+0000 7f33567fc640 1 -- 172.21.15.37:0/700734402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566eecb1b70 msgr2=0x5566eecbd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.670+0000 7f33567fc640 1 --2- 172.21.15.37:0/700734402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566eecb1b70 0x5566eecbd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.670+0000 7f33567fc640 1 -- 172.21.15.37:0/700734402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566ee923210 con 0x5566eecaca00 2024-06-10T01:04:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.670+0000 7f3356ffd640 1 --2- 172.21.15.37:0/700734402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eecac010 0x5566eecb6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.670+0000 7f33567fc640 1 --2- 172.21.15.37:0/700734402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566eecaca00 0x5566eecb6670 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f33581990e0 tx=0x7f334c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27451414b33ef701 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.670+0000 7f333aff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:04:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.670+0000 7f333aff5640 1 -- 172.21.15.37:0/700734402 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c004020 con 0x5566eecaca00 2024-06-10T01:04:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.670+0000 7f333aff5640 1 -- 172.21.15.37:0/700734402 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f334c0041c0 con 0x5566eecaca00 2024-06-10T01:04:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.670+0000 7f336056c740 1 -- 172.21.15.37:0/700734402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3350036040 con 0x5566eecaca00 2024-06-10T01:04:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.670+0000 7f333aff5640 1 -- 172.21.15.37:0/700734402 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f334c0043c0 con 0x5566eecaca00 2024-06-10T01:04:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.670+0000 7f336056c740 10 client.16827 fetch_fsmap learned FSMap version 29 2024-06-10T01:04:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.670+0000 7f336056c740 1 -- 172.21.15.37:0/700734402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5566ee9ef620 con 0x5566eecaca00 2024-06-10T01:04:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.671+0000 7f333aff5640 1 -- 172.21.15.37:0/700734402 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3350036040 con 0x5566eecaca00 2024-06-10T01:04:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.671+0000 7f333aff5640 1 -- 172.21.15.37:0/700734402 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f334c010700 con 0x5566eecaca00 2024-06-10T01:04:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.671+0000 7f336056c740 10 client.16827 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:04:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.671+0000 7f336056c740 10 client.16827 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:04:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.671+0000 7f336056c740 20 client.16827 populate_metadata read hostname 'smithi037' 2024-06-10T01:04:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.671+0000 7f336056c740 1 --2- 172.21.15.37:0/700734402 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5566eecc1f40 0x5566eecc43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.671+0000 7f336056c740 4 client.16827 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:04:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.671+0000 7f336056c740 1 -- 172.21.15.37:0/700734402 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566eec22f10 con 0x5566eecc1f40 2024-06-10T01:04:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.671+0000 7f3356ffd640 1 --2- 172.21.15.37:0/700734402 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5566eecc1f40 0x5566eecc43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.671+0000 7f3356ffd640 1 --2- 172.21.15.37:0/700734402 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5566eecc1f40 0x5566eecc43e0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2ba83787b346509 server_cookie=3b4ccd7771584b2b in_seq=0 out_seq=0 2024-06-10T01:04:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.672+0000 7f333aff5640 10 client.16827 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.679+0000 7f333aff5640 1 -- 172.21.15.37:0/700734402 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5566eec22f10 con 0x5566eecc1f40 2024-06-10T01:04:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.679+0000 7f333aff5640 10 client.16827 handle_command_reply: tid=0 2024-06-10T01:04:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.679+0000 7f336056c740 1 client.16827 shutdown 2024-06-10T01:04:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.679+0000 7f336056c740 1 -- 172.21.15.37:0/700734402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566eecaca00 msgr2=0x5566eecb6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.679+0000 7f336056c740 1 --2- 172.21.15.37:0/700734402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566eecaca00 0x5566eecb6670 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f33581990e0 tx=0x7f334c0118d0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.679+0000 7f336056c740 1 -- 172.21.15.37:0/700734402 shutdown_connections 2024-06-10T01:04:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.679+0000 7f336056c740 1 --2- 172.21.15.37:0/700734402 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5566eecc1f40 0x5566eecc43e0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.679+0000 7f336056c740 1 --2- 172.21.15.37:0/700734402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566eecb1b70 0x5566eecbd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.679+0000 7f336056c740 1 --2- 172.21.15.37:0/700734402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566eecaca00 0x5566eecb6670 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.679+0000 7f333aff5640 0 client.16827 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.679+0000 7f336056c740 1 --2- 172.21.15.37:0/700734402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eecac010 0x5566eecb6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.679+0000 7f336056c740 1 -- 172.21.15.37:0/700734402 >> 172.21.15.37:0/700734402 conn(0x5566eeca7680 msgr2=0x5566eeca67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.679+0000 7f336056c740 1 -- 172.21.15.37:0/700734402 shutdown_connections 2024-06-10T01:04:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.679+0000 7f336056c740 1 -- 172.21.15.37:0/700734402 wait complete. 2024-06-10T01:04:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.679+0000 7f336056c740 20 client.16827 trim_cache size 0 max 16384 2024-06-10T01:04:46.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.696+0000 7f336056c740 1 Processor -- start 2024-06-10T01:04:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.696+0000 7f336056c740 1 -- start start 2024-06-10T01:04:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.696+0000 7f336056c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566eecac010 0x5566eecb3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.696+0000 7f336056c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566eecaca00 0x5566eecbd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.696+0000 7f336056c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eecb1b70 0x5566eecadc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.696+0000 7f336056c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f33581acf30 con 0x5566eecac010 2024-06-10T01:04:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.696+0000 7f336056c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f33581aa800 con 0x5566eecaca00 2024-06-10T01:04:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.696+0000 7f336056c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f33581a8650 con 0x5566eecb1b70 2024-06-10T01:04:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.696+0000 7f3356ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566eecac010 0x5566eecb3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.696+0000 7f3356ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566eecac010 0x5566eecb3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47916/0 (socket says 172.21.15.37:47916) 2024-06-10T01:04:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.696+0000 7f3356ffd640 1 -- 172.21.15.37:0/2185879430 learned_addr learned my addr 172.21.15.37:0/2185879430 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.696+0000 7f33577fe640 1 --2- 172.21.15.37:0/2185879430 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eecb1b70 0x5566eecadc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.696+0000 7f33567fc640 1 --2- 172.21.15.37:0/2185879430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566eecaca00 0x5566eecbd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.696+0000 7f3356ffd640 1 -- 172.21.15.37:0/2185879430 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eecb1b70 msgr2=0x5566eecadc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.696+0000 7f3356ffd640 1 --2- 172.21.15.37:0/2185879430 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eecb1b70 0x5566eecadc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.696+0000 7f3356ffd640 1 -- 172.21.15.37:0/2185879430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566eecaca00 msgr2=0x5566eecbd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.696+0000 7f3356ffd640 1 --2- 172.21.15.37:0/2185879430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566eecaca00 0x5566eecbd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.696+0000 7f3356ffd640 1 -- 172.21.15.37:0/2185879430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566eecbeac0 con 0x5566eecac010 2024-06-10T01:04:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.696+0000 7f3356ffd640 1 --2- 172.21.15.37:0/2185879430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566eecac010 0x5566eecb3f60 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f33480985f0 tx=0x7f33480ccd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4448f77793bafbaf server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.697+0000 7f333cff9640 1 -- 172.21.15.37:0/2185879430 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334809b580 con 0x5566eecac010 2024-06-10T01:04:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.697+0000 7f333cff9640 1 -- 172.21.15.37:0/2185879430 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f334809c320 con 0x5566eecac010 2024-06-10T01:04:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.697+0000 7f333cff9640 1 -- 172.21.15.37:0/2185879430 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334809be80 con 0x5566eecac010 2024-06-10T01:04:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.697+0000 7f336056c740 1 -- 172.21.15.37:0/2185879430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566eecac010 msgr2=0x5566eecb3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.697+0000 7f336056c740 1 --2- 172.21.15.37:0/2185879430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566eecac010 0x5566eecb3f60 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f33480985f0 tx=0x7f33480ccd90 comp rx=0 tx=0).stop 2024-06-10T01:04:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.697+0000 7f336056c740 1 -- 172.21.15.37:0/2185879430 shutdown_connections 2024-06-10T01:04:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.697+0000 7f336056c740 1 --2- 172.21.15.37:0/2185879430 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eecb1b70 0x5566eecadc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.697+0000 7f336056c740 1 --2- 172.21.15.37:0/2185879430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566eecaca00 0x5566eecbd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.697+0000 7f336056c740 1 --2- 172.21.15.37:0/2185879430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566eecac010 0x5566eecb3f60 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.697+0000 7f336056c740 1 -- 172.21.15.37:0/2185879430 >> 172.21.15.37:0/2185879430 conn(0x5566eeca7680 msgr2=0x5566eeca7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.697+0000 7f336056c740 1 -- 172.21.15.37:0/2185879430 shutdown_connections 2024-06-10T01:04:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.697+0000 7f336056c740 1 -- 172.21.15.37:0/2185879430 wait complete. 2024-06-10T01:04:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.698+0000 7f336056c740 1 -- start start 2024-06-10T01:04:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.698+0000 7f336056c740 1 Processor -- start 2024-06-10T01:04:46.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.698+0000 7f336056c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eecac010 0x5566eecbd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:46.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.698+0000 7f336056c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566eecaca00 0x5566eecb9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:46.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.698+0000 7f336056c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566eecb1b70 0x5566eecba3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:46.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.698+0000 7f336056c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f33581a8cc0 con 0x5566eecaca00 2024-06-10T01:04:46.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.698+0000 7f336056c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f33581aa130 con 0x5566eecb1b70 2024-06-10T01:04:46.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.698+0000 7f336056c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f33581a9390 con 0x5566eecac010 2024-06-10T01:04:46.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.698+0000 7f336056c740 10 client.0 _ll_register_callbacks cb 0x7f334c00bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:46.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.698+0000 7f33567fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566eecaca00 0x5566eecb9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:46.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.698+0000 7f3356ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eecac010 0x5566eecbd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:46.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.698+0000 7f3356ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eecac010 0x5566eecbd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:36460/0 (socket says 172.21.15.37:36460) 2024-06-10T01:04:46.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.698+0000 7f33567fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566eecaca00 0x5566eecb9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47920/0 (socket says 172.21.15.37:47920) 2024-06-10T01:04:46.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.698+0000 7f3356ffd640 1 -- 172.21.15.37:0/4100651725 learned_addr learned my addr 172.21.15.37:0/4100651725 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:46.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.698+0000 7f33577fe640 1 --2- 172.21.15.37:0/4100651725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566eecb1b70 0x5566eecba3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:46.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.698+0000 7f3356ffd640 1 -- 172.21.15.37:0/4100651725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566eecb1b70 msgr2=0x5566eecba3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:46.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.698+0000 7f3356ffd640 1 --2- 172.21.15.37:0/4100651725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566eecb1b70 0x5566eecba3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.698+0000 7f3356ffd640 1 -- 172.21.15.37:0/4100651725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566eecaca00 msgr2=0x5566eecb9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:46.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.698+0000 7f3356ffd640 1 --2- 172.21.15.37:0/4100651725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566eecaca00 0x5566eecb9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:46.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.698+0000 7f3356ffd640 1 -- 172.21.15.37:0/4100651725 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566eed2acb0 con 0x5566eecac010 2024-06-10T01:04:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.699+0000 7f3356ffd640 1 --2- 172.21.15.37:0/4100651725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eecac010 0x5566eecbd3a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f33480058c0 tx=0x7f33480cd1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a18b6c2c0aaa7b9d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.699+0000 7f333aff5640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:04:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.699+0000 7f333aff5640 1 -- 172.21.15.37:0/4100651725 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334809bc20 con 0x5566eecac010 2024-06-10T01:04:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.699+0000 7f336056c740 1 -- 172.21.15.37:0/4100651725 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f334809c320 con 0x5566eecac010 2024-06-10T01:04:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.699+0000 7f333aff5640 1 -- 172.21.15.37:0/4100651725 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f334809be80 con 0x5566eecac010 2024-06-10T01:04:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.699+0000 7f333aff5640 1 -- 172.21.15.37:0/4100651725 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f33480a4890 con 0x5566eecac010 2024-06-10T01:04:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.699+0000 7f336056c740 10 client.26371 fetch_fsmap learned FSMap version 29 2024-06-10T01:04:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.699+0000 7f336056c740 1 -- 172.21.15.37:0/4100651725 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5566eed2be50 con 0x5566eecac010 2024-06-10T01:04:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.699+0000 7f333aff5640 1 -- 172.21.15.37:0/4100651725 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f334809c320 con 0x5566eecac010 2024-06-10T01:04:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.699+0000 7f333aff5640 1 -- 172.21.15.37:0/4100651725 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f334809b070 con 0x5566eecac010 2024-06-10T01:04:46.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.699+0000 7f336056c740 10 client.26371 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:04:46.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.699+0000 7f336056c740 10 client.26371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:04:46.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.699+0000 7f336056c740 20 client.26371 populate_metadata read hostname 'smithi037' 2024-06-10T01:04:46.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.699+0000 7f336056c740 1 --2- 172.21.15.37:0/4100651725 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5566eed2c7b0 0x5566eed2ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:46.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.699+0000 7f336056c740 4 client.26371 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:04:46.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.699+0000 7f336056c740 1 -- 172.21.15.37:0/4100651725 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5566eec1cc20 con 0x5566eed2c7b0 2024-06-10T01:04:46.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.700+0000 7f33567fc640 1 --2- 172.21.15.37:0/4100651725 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5566eed2c7b0 0x5566eed2ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:46.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.700+0000 7f33567fc640 1 --2- 172.21.15.37:0/4100651725 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5566eed2c7b0 0x5566eed2ebd0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb59111ebb849d53 server_cookie=1c19531f94dbc5ba in_seq=0 out_seq=0 2024-06-10T01:04:46.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:46.700+0000 7f333aff5640 10 client.26371 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:48.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:48 smithi167 ceph-mon[36917]: pgmap v674: 129 pgs: 129 active+clean; 18 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 130 op/s 2024-06-10T01:04:48.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:48 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:04:48.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:48 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:04:48.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:48 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:04:48.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:48 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:04:48.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:48 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:04:48.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:48 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:04:48.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:48 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:04:48.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:48 smithi094 ceph-mon[36708]: pgmap v674: 129 pgs: 129 active+clean; 18 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 130 op/s 2024-06-10T01:04:48.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:48 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:04:48.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:48 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:04:48.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:48 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:04:48.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:48 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:04:48.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:48 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:04:48.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:48 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:04:48.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:48 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:04:48.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:48 smithi037 ceph-mon[32192]: pgmap v674: 129 pgs: 129 active+clean; 18 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 130 op/s 2024-06-10T01:04:48.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:48 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:04:48.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:48 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:04:48.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:48 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:04:48.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:48 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:04:48.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:48 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:04:48.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:48 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:04:48.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:48 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:04:50.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:50 smithi167 ceph-mon[36917]: pgmap v675: 129 pgs: 129 active+clean; 18 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-06-10T01:04:50.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:50 smithi094 ceph-mon[36708]: pgmap v675: 129 pgs: 129 active+clean; 18 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-06-10T01:04:50.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:50 smithi037 ceph-mon[32192]: pgmap v675: 129 pgs: 129 active+clean; 18 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-06-10T01:04:52.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:52 smithi167 ceph-mon[36917]: pgmap v676: 129 pgs: 129 active+clean; 23 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-06-10T01:04:52.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:52 smithi094 ceph-mon[36708]: pgmap v676: 129 pgs: 129 active+clean; 23 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-06-10T01:04:52.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:52 smithi037 ceph-mon[32192]: pgmap v676: 129 pgs: 129 active+clean; 23 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-06-10T01:04:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:52.987+0000 7f333aff5640 1 -- 172.21.15.37:0/4100651725 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5566eec1cc20 con 0x5566eed2c7b0 2024-06-10T01:04:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:52.987+0000 7f333aff5640 10 client.26371 handle_command_reply: tid=0 2024-06-10T01:04:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:52.987+0000 7f336056c740 1 client.26371 shutdown 2024-06-10T01:04:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:52.987+0000 7f336056c740 1 -- 172.21.15.37:0/4100651725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eecac010 msgr2=0x5566eecbd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:52.987+0000 7f336056c740 1 --2- 172.21.15.37:0/4100651725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eecac010 0x5566eecbd3a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f33480058c0 tx=0x7f33480cd1f0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:52.987+0000 7f336056c740 1 -- 172.21.15.37:0/4100651725 shutdown_connections 2024-06-10T01:04:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:52.987+0000 7f336056c740 1 --2- 172.21.15.37:0/4100651725 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5566eed2c7b0 0x5566eed2ebd0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:52.987+0000 7f336056c740 1 --2- 172.21.15.37:0/4100651725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5566eecb1b70 0x5566eecba3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:52.987+0000 7f336056c740 1 --2- 172.21.15.37:0/4100651725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566eecaca00 0x5566eecb9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:52.987+0000 7f333aff5640 0 client.26371 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:52.987+0000 7f336056c740 1 --2- 172.21.15.37:0/4100651725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5566eecac010 0x5566eecbd3a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:52.987+0000 7f336056c740 1 -- 172.21.15.37:0/4100651725 >> 172.21.15.37:0/4100651725 conn(0x5566eeca7680 msgr2=0x5566eecb60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:52.988+0000 7f336056c740 1 -- 172.21.15.37:0/4100651725 shutdown_connections 2024-06-10T01:04:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:52.988+0000 7f336056c740 1 -- 172.21.15.37:0/4100651725 wait complete. 2024-06-10T01:04:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:52.988+0000 7f336056c740 20 client.26371 trim_cache size 0 max 16384 2024-06-10T01:04:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:04:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:04:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:04:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:52.988+0000 7f335d77e640 1 -- 172.21.15.37:0/3240670460 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f33400a2c30 msgr2=0x7f33400a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:52.988+0000 7f335d77e640 1 --2- 172.21.15.37:0/3240670460 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f33400a2c30 0x7f33400a50f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f335004a000 tx=0x7f335004b000 comp rx=0 tx=0).stop 2024-06-10T01:04:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:52.988+0000 7f335d77e640 1 -- 172.21.15.37:0/3240670460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3358101a00 msgr2=0x7f3358197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:52.988+0000 7f335d77e640 1 --2- 172.21.15.37:0/3240670460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3358101a00 0x7f3358197bd0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f334800e9c0 tx=0x7f334800ee80 comp rx=0 tx=0).stop 2024-06-10T01:04:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:52.988+0000 7f335d77e640 1 -- 172.21.15.37:0/3240670460 shutdown_connections 2024-06-10T01:04:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:52.988+0000 7f335d77e640 1 --2- 172.21.15.37:0/3240670460 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f33400a2c30 0x7f33400a50f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:52.988+0000 7f335d77e640 1 --2- 172.21.15.37:0/3240670460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33581035b0 0x7f335819f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:52.988+0000 7f335d77e640 1 --2- 172.21.15.37:0/3240670460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3358102bf0 0x7f3358198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:52.988+0000 7f335d77e640 1 --2- 172.21.15.37:0/3240670460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3358101a00 0x7f3358197bd0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:52.988+0000 7f335d77e640 1 -- 172.21.15.37:0/3240670460 >> 172.21.15.37:0/3240670460 conn(0x7f33580fcf90 msgr2=0x7f3358105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:52.989+0000 7f335d77e640 1 -- 172.21.15.37:0/3240670460 shutdown_connections 2024-06-10T01:04:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:04:52.989+0000 7f335d77e640 1 -- 172.21.15.37:0/3240670460 wait complete. 2024-06-10T01:04:54.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:54 smithi167 ceph-mon[36917]: pgmap v677: 129 pgs: 129 active+clean; 23 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 962 op/s 2024-06-10T01:04:54.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:54 smithi094 ceph-mon[36708]: pgmap v677: 129 pgs: 129 active+clean; 23 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 962 op/s 2024-06-10T01:04:54.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:54 smithi037 ceph-mon[32192]: pgmap v677: 129 pgs: 129 active+clean; 23 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 962 op/s 2024-06-10T01:04:56.248 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:04:56.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:56 smithi167 ceph-mon[36917]: pgmap v678: 129 pgs: 129 active+clean; 8.6 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 2.23k op/s 2024-06-10T01:04:56.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:56 smithi094 ceph-mon[36708]: pgmap v678: 129 pgs: 129 active+clean; 8.6 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 2.23k op/s 2024-06-10T01:04:56.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:56 smithi037 ceph-mon[32192]: pgmap v678: 129 pgs: 129 active+clean; 8.6 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 2.23k op/s 2024-06-10T01:04:56.517 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:04:58.299 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:04:58 smithi037 ceph-mon[32192]: pgmap v679: 129 pgs: 129 active+clean; 4.6 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 1.4 MiB/s wr, 2.24k op/s 2024-06-10T01:04:58.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:04:58 smithi167 ceph-mon[36917]: pgmap v679: 129 pgs: 129 active+clean; 4.6 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 1.4 MiB/s wr, 2.24k op/s 2024-06-10T01:04:58.340 INFO:tasks.workunit.client.0.smithi037.stdout:created hierarchy, now cleaning up 2024-06-10T01:04:58.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:04:58 smithi094 ceph-mon[36708]: pgmap v679: 129 pgs: 129 active+clean; 4.6 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 1.4 MiB/s wr, 2.24k op/s 2024-06-10T01:04:59.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.430+0000 7ff24a3d0700 1 -- 172.21.15.37:0/1566415569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2441030f0 msgr2=0x7ff244107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:59.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.430+0000 7ff24a3d0700 1 --2- 172.21.15.37:0/1566415569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2441030f0 0x7ff244107110 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff2380099c0 tx=0x7ff238009cd0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.430+0000 7ff24a3d0700 1 -- 172.21.15.37:0/1566415569 shutdown_connections 2024-06-10T01:04:59.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.430+0000 7ff24a3d0700 1 --2- 172.21.15.37:0/1566415569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff244107840 0x7ff24410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.430+0000 7ff24a3d0700 1 --2- 172.21.15.37:0/1566415569 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff244102740 0x7ff244102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.430+0000 7ff24a3d0700 1 --2- 172.21.15.37:0/1566415569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2441030f0 0x7ff244107110 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.430+0000 7ff24a3d0700 1 -- 172.21.15.37:0/1566415569 >> 172.21.15.37:0/1566415569 conn(0x7ff2440795f0 msgr2=0x7ff244079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:59.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.430+0000 7ff24a3d0700 1 -- 172.21.15.37:0/1566415569 shutdown_connections 2024-06-10T01:04:59.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.430+0000 7ff24a3d0700 1 -- 172.21.15.37:0/1566415569 wait complete. 2024-06-10T01:04:59.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.431+0000 7ff24a3d0700 1 Processor -- start 2024-06-10T01:04:59.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.431+0000 7ff24a3d0700 1 -- start start 2024-06-10T01:04:59.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.431+0000 7ff24a3d0700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff244102740 0x7ff24419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:59.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.431+0000 7ff24a3d0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2441030f0 0x7ff24419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:59.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.431+0000 7ff24a3d0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff244107840 0x7ff2441989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:59.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.431+0000 7ff24a3d0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff244112770 con 0x7ff244107840 2024-06-10T01:04:59.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.431+0000 7ff24a3d0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff2441125f0 con 0x7ff2441030f0 2024-06-10T01:04:59.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.431+0000 7ff24a3d0700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff2441128f0 con 0x7ff244102740 2024-06-10T01:04:59.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.431+0000 7ff24896d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff244107840 0x7ff2441989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:59.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.431+0000 7ff24896d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff244107840 0x7ff2441989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58416/0 (socket says 172.21.15.37:58416) 2024-06-10T01:04:59.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.431+0000 7ff24896d700 1 -- 172.21.15.37:0/2727318554 learned_addr learned my addr 172.21.15.37:0/2727318554 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:59.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.431+0000 7ff243fff700 1 --2- 172.21.15.37:0/2727318554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff244102740 0x7ff24419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:59.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.432+0000 7ff2437fe700 1 --2- 172.21.15.37:0/2727318554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2441030f0 0x7ff24419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:59.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.432+0000 7ff243fff700 1 -- 172.21.15.37:0/2727318554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2441030f0 msgr2=0x7ff24419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:59.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.432+0000 7ff243fff700 1 --2- 172.21.15.37:0/2727318554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2441030f0 0x7ff24419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.432+0000 7ff243fff700 1 -- 172.21.15.37:0/2727318554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff244107840 msgr2=0x7ff2441989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:59.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.432+0000 7ff243fff700 1 --2- 172.21.15.37:0/2727318554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff244107840 0x7ff2441989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.432+0000 7ff243fff700 1 -- 172.21.15.37:0/2727318554 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff244199150 con 0x7ff244102740 2024-06-10T01:04:59.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.432+0000 7ff24896d700 1 --2- 172.21.15.37:0/2727318554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff244107840 0x7ff2441989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:59.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.432+0000 7ff243fff700 1 --2- 172.21.15.37:0/2727318554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff244102740 0x7ff24419e830 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff23400e9f0 tx=0x7ff23400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:59.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.432+0000 7ff2417fa700 1 -- 172.21.15.37:0/2727318554 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff234004220 con 0x7ff244102740 2024-06-10T01:04:59.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.432+0000 7ff24a3d0700 1 -- 172.21.15.37:0/2727318554 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff244199440 con 0x7ff244102740 2024-06-10T01:04:59.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.432+0000 7ff24a3d0700 1 -- 172.21.15.37:0/2727318554 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2441a5700 con 0x7ff244102740 2024-06-10T01:04:59.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.432+0000 7ff2417fa700 1 -- 172.21.15.37:0/2727318554 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff234004d10 con 0x7ff244102740 2024-06-10T01:04:59.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.432+0000 7ff2417fa700 1 -- 172.21.15.37:0/2727318554 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff234005020 con 0x7ff244102740 2024-06-10T01:04:59.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.435+0000 7ff2417fa700 1 -- 172.21.15.37:0/2727318554 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff234004750 con 0x7ff244102740 2024-06-10T01:04:59.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.436+0000 7ff2417fa700 1 --2- 172.21.15.37:0/2727318554 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff22c0a2c20 0x7ff22c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:59.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.436+0000 7ff254c53b80 1 Processor -- start 2024-06-10T01:04:59.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.436+0000 7ff254c53b80 1 -- start start 2024-06-10T01:04:59.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.436+0000 7ff2417fa700 1 -- 172.21.15.37:0/2727318554 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7ff2340cf370 con 0x7ff244102740 2024-06-10T01:04:59.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.437+0000 7ff2437fe700 1 --2- 172.21.15.37:0/2727318554 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff22c0a2c20 0x7ff22c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:59.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.437+0000 7ff254c53b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585cdc76470 0x5585cdc76850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:59.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.437+0000 7ff254c53b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585cdc70670 0x5585cdc70af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:59.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.437+0000 7ff254c53b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585cdc71030 0x5585cdc7d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:59.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.437+0000 7ff254c53b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5585cdb5ef60 con 0x5585cdc70670 2024-06-10T01:04:59.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.437+0000 7ff254c53b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5585cdbbd850 con 0x5585cdc71030 2024-06-10T01:04:59.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.437+0000 7ff254c53b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5585cdb25460 con 0x5585cdc76470 2024-06-10T01:04:59.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.437+0000 7ff2437fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585cdc70670 0x5585cdc70af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:59.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.437+0000 7ff243fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585cdc76470 0x5585cdc76850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:59.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.437+0000 7ff2437fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585cdc70670 0x5585cdc70af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58418/0 (socket says 172.21.15.37:58418) 2024-06-10T01:04:59.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.437+0000 7ff2437fe700 1 -- 172.21.15.37:0/455411140 learned_addr learned my addr 172.21.15.37:0/455411140 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:59.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.437+0000 7ff24896d700 1 --2- 172.21.15.37:0/455411140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585cdc71030 0x5585cdc7d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:59.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.437+0000 7ff243fff700 1 -- 172.21.15.37:0/455411140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585cdc71030 msgr2=0x5585cdc7d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:59.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.437+0000 7ff243fff700 1 --2- 172.21.15.37:0/455411140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585cdc71030 0x5585cdc7d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.437+0000 7ff243fff700 1 -- 172.21.15.37:0/455411140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585cdc70670 msgr2=0x5585cdc70af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:59.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.437+0000 7ff243fff700 1 --2- 172.21.15.37:0/455411140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585cdc70670 0x5585cdc70af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.437+0000 7ff243fff700 1 -- 172.21.15.37:0/455411140 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585cdb0c8d0 con 0x5585cdc76470 2024-06-10T01:04:59.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.437+0000 7ff2437fe700 1 --2- 172.21.15.37:0/2727318554 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff22c0a2c20 0x7ff22c0a50e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff2380098e0 tx=0x7ff23800b790 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:59.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.437+0000 7ff243fff700 1 --2- 172.21.15.37:0/455411140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585cdc76470 0x5585cdc76850 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff23409b480 tx=0x7ff234097b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecc0e80f26868652 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:59.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.438+0000 7ff22a7fc700 1 -- 172.21.15.37:0/455411140 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23401c040 con 0x5585cdc76470 2024-06-10T01:04:59.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.438+0000 7ff22a7fc700 1 -- 172.21.15.37:0/455411140 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff23409be80 con 0x5585cdc76470 2024-06-10T01:04:59.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.438+0000 7ff22a7fc700 1 -- 172.21.15.37:0/455411140 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff234004a40 con 0x5585cdc76470 2024-06-10T01:04:59.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.438+0000 7ff254c53b80 1 -- 172.21.15.37:0/455411140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585cdc76470 msgr2=0x5585cdc76850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:59.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.438+0000 7ff254c53b80 1 --2- 172.21.15.37:0/455411140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585cdc76470 0x5585cdc76850 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff23409b480 tx=0x7ff234097b50 comp rx=0 tx=0).stop 2024-06-10T01:04:59.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.439+0000 7ff254c53b80 1 -- 172.21.15.37:0/455411140 shutdown_connections 2024-06-10T01:04:59.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.439+0000 7ff254c53b80 1 --2- 172.21.15.37:0/455411140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585cdc71030 0x5585cdc7d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.439+0000 7ff254c53b80 1 --2- 172.21.15.37:0/455411140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585cdc76470 0x5585cdc76850 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.439+0000 7ff254c53b80 1 --2- 172.21.15.37:0/455411140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585cdc70670 0x5585cdc70af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.439+0000 7ff254c53b80 1 -- 172.21.15.37:0/455411140 >> 172.21.15.37:0/455411140 conn(0x5585cdb99080 msgr2=0x5585cdc67220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:59.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.439+0000 7ff254c53b80 1 -- 172.21.15.37:0/455411140 shutdown_connections 2024-06-10T01:04:59.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.439+0000 7ff254c53b80 1 -- 172.21.15.37:0/455411140 wait complete. 2024-06-10T01:04:59.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.439+0000 7ff254c53b80 1 -- start start 2024-06-10T01:04:59.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.439+0000 7ff254c53b80 1 Processor -- start 2024-06-10T01:04:59.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.440+0000 7ff254c53b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585cdc70670 0x5585cdc7cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:59.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.440+0000 7ff254c53b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585cdc71030 0x5585cdc7a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:59.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.440+0000 7ff254c53b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585cdc76470 0x5585cdc7a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:59.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.440+0000 7ff254c53b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5585cdb5dfc0 con 0x5585cdc70670 2024-06-10T01:04:59.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.440+0000 7ff254c53b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5585cdbb9810 con 0x5585cdc76470 2024-06-10T01:04:59.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.440+0000 7ff254c53b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5585cdc7d750 con 0x5585cdc71030 2024-06-10T01:04:59.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.440+0000 7ff243fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585cdc70670 0x5585cdc7cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:59.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.440+0000 7ff2437fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585cdc71030 0x5585cdc7a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:59.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.440+0000 7ff243fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585cdc70670 0x5585cdc7cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58420/0 (socket says 172.21.15.37:58420) 2024-06-10T01:04:59.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.440+0000 7ff243fff700 1 -- 172.21.15.37:0/2991188691 learned_addr learned my addr 172.21.15.37:0/2991188691 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:59.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.440+0000 7ff254c53b80 10 client.0 _ll_register_callbacks cb 0x5585cdb535b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:59.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.440+0000 7ff24896d700 1 --2- 172.21.15.37:0/2991188691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585cdc76470 0x5585cdc7a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:59.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.440+0000 7ff243fff700 1 -- 172.21.15.37:0/2991188691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585cdc71030 msgr2=0x5585cdc7a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:59.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.440+0000 7ff243fff700 1 --2- 172.21.15.37:0/2991188691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585cdc71030 0x5585cdc7a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.440+0000 7ff243fff700 1 -- 172.21.15.37:0/2991188691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585cdc76470 msgr2=0x5585cdc7a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:59.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.440+0000 7ff243fff700 1 --2- 172.21.15.37:0/2991188691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585cdc76470 0x5585cdc7a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.440+0000 7ff243fff700 1 -- 172.21.15.37:0/2991188691 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585cdb83750 con 0x5585cdc70670 2024-06-10T01:04:59.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.440+0000 7ff2437fe700 1 --2- 172.21.15.37:0/2991188691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585cdc71030 0x5585cdc7a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:59.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.440+0000 7ff243fff700 1 --2- 172.21.15.37:0/2991188691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585cdc70670 0x5585cdc7cef0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff2340a3ec0 tx=0x7ff23409b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9ebc5e2a47e363 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:59.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.440+0000 7ff2237fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:04:59.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.441+0000 7ff2237fe700 1 -- 172.21.15.37:0/2991188691 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2340b9070 con 0x5585cdc70670 2024-06-10T01:04:59.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.441+0000 7ff2237fe700 1 -- 172.21.15.37:0/2991188691 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7ff234004a40 con 0x5585cdc70670 2024-06-10T01:04:59.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.441+0000 7ff254c53b80 1 -- 172.21.15.37:0/2991188691 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff23409be80 con 0x5585cdc70670 2024-06-10T01:04:59.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.441+0000 7ff2237fe700 1 -- 172.21.15.37:0/2991188691 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff23401c070 con 0x5585cdc70670 2024-06-10T01:04:59.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.441+0000 7ff254c53b80 10 client.16845 fetch_fsmap learned FSMap version 29 2024-06-10T01:04:59.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.441+0000 7ff254c53b80 1 -- 172.21.15.37:0/2991188691 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5585cdc87df0 con 0x5585cdc70670 2024-06-10T01:04:59.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.441+0000 7ff2237fe700 1 -- 172.21.15.37:0/2991188691 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2340a43f0 con 0x5585cdc70670 2024-06-10T01:04:59.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.441+0000 7ff2237fe700 1 -- 172.21.15.37:0/2991188691 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7ff234097040 con 0x5585cdc70670 2024-06-10T01:04:59.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.441+0000 7ff254c53b80 10 client.16845 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:04:59.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.441+0000 7ff254c53b80 10 client.16845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:04:59.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.441+0000 7ff254c53b80 20 client.16845 populate_metadata read hostname 'smithi037' 2024-06-10T01:04:59.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.441+0000 7ff254c53b80 1 --2- 172.21.15.37:0/2991188691 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5585cdc8a2c0 0x5585cdc8c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:59.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.441+0000 7ff254c53b80 4 client.16845 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:04:59.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.441+0000 7ff254c53b80 1 -- 172.21.15.37:0/2991188691 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff23409be80 con 0x5585cdc8a2c0 2024-06-10T01:04:59.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.442+0000 7ff2437fe700 1 --2- 172.21.15.37:0/2991188691 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5585cdc8a2c0 0x5585cdc8c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:59.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.442+0000 7ff2437fe700 1 --2- 172.21.15.37:0/2991188691 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5585cdc8a2c0 0x5585cdc8c6b0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f43e5d5283dc679 server_cookie=256f12a928a8898c in_seq=0 out_seq=0 2024-06-10T01:04:59.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.442+0000 7ff2237fe700 10 client.16845 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:59.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.444+0000 7ff2237fe700 1 -- 172.21.15.37:0/2991188691 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7ff23409be80 con 0x5585cdc8a2c0 2024-06-10T01:04:59.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.444+0000 7ff2237fe700 10 client.16845 handle_command_reply: tid=0 2024-06-10T01:04:59.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.444+0000 7ff254c53b80 1 client.16845 shutdown 2024-06-10T01:04:59.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.445+0000 7ff254c53b80 1 -- 172.21.15.37:0/2991188691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585cdc70670 msgr2=0x5585cdc7cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:59.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.445+0000 7ff254c53b80 1 --2- 172.21.15.37:0/2991188691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585cdc70670 0x5585cdc7cef0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff2340a3ec0 tx=0x7ff23409b850 comp rx=0 tx=0).stop 2024-06-10T01:04:59.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.445+0000 7ff254c53b80 1 -- 172.21.15.37:0/2991188691 shutdown_connections 2024-06-10T01:04:59.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.445+0000 7ff254c53b80 1 --2- 172.21.15.37:0/2991188691 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5585cdc8a2c0 0x5585cdc8c6b0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.445+0000 7ff254c53b80 1 --2- 172.21.15.37:0/2991188691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585cdc76470 0x5585cdc7a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.445+0000 7ff254c53b80 1 --2- 172.21.15.37:0/2991188691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585cdc70670 0x5585cdc7cef0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.445+0000 7ff254c53b80 1 --2- 172.21.15.37:0/2991188691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585cdc71030 0x5585cdc7a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.445+0000 7ff2237fe700 0 client.16845 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:59.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.445+0000 7ff254c53b80 1 -- 172.21.15.37:0/2991188691 >> 172.21.15.37:0/2991188691 conn(0x5585cdb99080 msgr2=0x5585cdc66700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:59.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.445+0000 7ff254c53b80 1 -- 172.21.15.37:0/2991188691 shutdown_connections 2024-06-10T01:04:59.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.445+0000 7ff254c53b80 1 -- 172.21.15.37:0/2991188691 wait complete. 2024-06-10T01:04:59.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.445+0000 7ff254c53b80 20 client.16845 trim_cache size 0 max 16384 2024-06-10T01:04:59.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.466+0000 7ff254c53b80 1 Processor -- start 2024-06-10T01:04:59.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.466+0000 7ff254c53b80 1 -- start start 2024-06-10T01:04:59.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.467+0000 7ff254c53b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585cdc70670 0x5585cdc7c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:59.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.467+0000 7ff254c53b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585cdc71030 0x5585cdc66e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:59.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.467+0000 7ff254c53b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585cdc76470 0x5585cdc673c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:59.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.467+0000 7ff254c53b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff2441aebd0 con 0x5585cdc70670 2024-06-10T01:04:59.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.467+0000 7ff254c53b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff2441ac460 con 0x5585cdc76470 2024-06-10T01:04:59.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.467+0000 7ff254c53b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff2441aa2b0 con 0x5585cdc71030 2024-06-10T01:04:59.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.467+0000 7ff243fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585cdc70670 0x5585cdc7c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:59.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.467+0000 7ff2437fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585cdc71030 0x5585cdc66e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:59.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.467+0000 7ff243fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585cdc70670 0x5585cdc7c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58434/0 (socket says 172.21.15.37:58434) 2024-06-10T01:04:59.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.467+0000 7ff243fff700 1 -- 172.21.15.37:0/1211936925 learned_addr learned my addr 172.21.15.37:0/1211936925 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:59.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.467+0000 7ff24896d700 1 --2- 172.21.15.37:0/1211936925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585cdc76470 0x5585cdc673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:59.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.467+0000 7ff243fff700 1 -- 172.21.15.37:0/1211936925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585cdc71030 msgr2=0x5585cdc66e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:59.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.467+0000 7ff243fff700 1 --2- 172.21.15.37:0/1211936925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585cdc71030 0x5585cdc66e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.467+0000 7ff243fff700 1 -- 172.21.15.37:0/1211936925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585cdc76470 msgr2=0x5585cdc673c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.467+0000 7ff243fff700 1 --2- 172.21.15.37:0/1211936925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585cdc76470 0x5585cdc673c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.467+0000 7ff243fff700 1 -- 172.21.15.37:0/1211936925 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585cdc76b90 con 0x5585cdc70670 2024-06-10T01:04:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.467+0000 7ff2437fe700 1 --2- 172.21.15.37:0/1211936925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585cdc71030 0x5585cdc66e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.467+0000 7ff243fff700 1 --2- 172.21.15.37:0/1211936925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585cdc70670 0x5585cdc7c130 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff23409b710 tx=0x7ff23409b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8babcc4d4936d011 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.468+0000 7ff229ffb700 1 -- 172.21.15.37:0/1211936925 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2340b9070 con 0x5585cdc70670 2024-06-10T01:04:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.468+0000 7ff229ffb700 1 -- 172.21.15.37:0/1211936925 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff23409be80 con 0x5585cdc70670 2024-06-10T01:04:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.468+0000 7ff229ffb700 1 -- 172.21.15.37:0/1211936925 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23401ebb0 con 0x5585cdc70670 2024-06-10T01:04:59.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.468+0000 7ff254c53b80 1 -- 172.21.15.37:0/1211936925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585cdc70670 msgr2=0x5585cdc7c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:59.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.468+0000 7ff254c53b80 1 --2- 172.21.15.37:0/1211936925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585cdc70670 0x5585cdc7c130 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff23409b710 tx=0x7ff23409b8c0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.469+0000 7ff254c53b80 1 -- 172.21.15.37:0/1211936925 shutdown_connections 2024-06-10T01:04:59.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.469+0000 7ff254c53b80 1 --2- 172.21.15.37:0/1211936925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585cdc76470 0x5585cdc673c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.469+0000 7ff254c53b80 1 --2- 172.21.15.37:0/1211936925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585cdc70670 0x5585cdc7c130 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.469+0000 7ff254c53b80 1 --2- 172.21.15.37:0/1211936925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585cdc71030 0x5585cdc66e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.469+0000 7ff254c53b80 1 -- 172.21.15.37:0/1211936925 >> 172.21.15.37:0/1211936925 conn(0x5585cdb99080 msgr2=0x5585cdc7ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:59.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.469+0000 7ff254c53b80 1 -- 172.21.15.37:0/1211936925 shutdown_connections 2024-06-10T01:04:59.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.469+0000 7ff254c53b80 1 -- 172.21.15.37:0/1211936925 wait complete. 2024-06-10T01:04:59.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.469+0000 7ff254c53b80 1 -- start start 2024-06-10T01:04:59.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.469+0000 7ff254c53b80 1 Processor -- start 2024-06-10T01:04:59.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.469+0000 7ff254c53b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585cdc70670 0x5585cdc8db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:59.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.469+0000 7ff254c53b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585cdc71030 0x5585cdc8e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:59.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.469+0000 7ff254c53b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585cdc76470 0x5585cdcf3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:59.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.469+0000 7ff254c53b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff2441aa920 con 0x5585cdc71030 2024-06-10T01:04:59.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.469+0000 7ff254c53b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff2441abd90 con 0x5585cdc70670 2024-06-10T01:04:59.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.469+0000 7ff254c53b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff2441aaff0 con 0x5585cdc76470 2024-06-10T01:04:59.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.470+0000 7ff254c53b80 10 client.0 _ll_register_callbacks cb 0x5585cdb535b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:04:59.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.470+0000 7ff2437fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585cdc71030 0x5585cdc8e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:59.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.470+0000 7ff2437fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585cdc71030 0x5585cdc8e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58444/0 (socket says 172.21.15.37:58444) 2024-06-10T01:04:59.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.470+0000 7ff2437fe700 1 -- 172.21.15.37:0/394003510 learned_addr learned my addr 172.21.15.37:0/394003510 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:04:59.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.470+0000 7ff243fff700 1 --2- 172.21.15.37:0/394003510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585cdc70670 0x5585cdc8db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:59.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.470+0000 7ff24896d700 1 --2- 172.21.15.37:0/394003510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585cdc76470 0x5585cdcf3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:59.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.470+0000 7ff2437fe700 1 -- 172.21.15.37:0/394003510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585cdc76470 msgr2=0x5585cdcf3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:59.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.470+0000 7ff2437fe700 1 --2- 172.21.15.37:0/394003510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585cdc76470 0x5585cdcf3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.470+0000 7ff2437fe700 1 -- 172.21.15.37:0/394003510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585cdc70670 msgr2=0x5585cdc8db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:04:59.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.470+0000 7ff2437fe700 1 --2- 172.21.15.37:0/394003510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585cdc70670 0x5585cdc8db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.470+0000 7ff2437fe700 1 -- 172.21.15.37:0/394003510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585cdcf4140 con 0x5585cdc71030 2024-06-10T01:04:59.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.470+0000 7ff243fff700 1 --2- 172.21.15.37:0/394003510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585cdc70670 0x5585cdc8db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:04:59.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.470+0000 7ff2437fe700 1 --2- 172.21.15.37:0/394003510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585cdc71030 0x5585cdc8e0c0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x5585cdc67e10 tx=0x7ff238009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e4d6d9fc190297 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:04:59.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.470+0000 7ff2237fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:04:59.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.471+0000 7ff2237fe700 1 -- 172.21.15.37:0/394003510 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2380050b0 con 0x5585cdc71030 2024-06-10T01:04:59.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.471+0000 7ff2237fe700 1 -- 172.21.15.37:0/394003510 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7ff238005250 con 0x5585cdc71030 2024-06-10T01:04:59.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.471+0000 7ff254c53b80 1 -- 172.21.15.37:0/394003510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff23409be80 con 0x5585cdc71030 2024-06-10T01:04:59.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.471+0000 7ff2237fe700 1 -- 172.21.15.37:0/394003510 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff238005450 con 0x5585cdc71030 2024-06-10T01:04:59.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.471+0000 7ff254c53b80 10 client.16851 fetch_fsmap learned FSMap version 29 2024-06-10T01:04:59.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.471+0000 7ff254c53b80 1 -- 172.21.15.37:0/394003510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5585cdcf5430 con 0x5585cdc71030 2024-06-10T01:04:59.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.471+0000 7ff2237fe700 1 -- 172.21.15.37:0/394003510 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff23409be80 con 0x5585cdc71030 2024-06-10T01:04:59.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.471+0000 7ff2237fe700 1 -- 172.21.15.37:0/394003510 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7ff23800f460 con 0x5585cdc71030 2024-06-10T01:04:59.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.471+0000 7ff254c53b80 10 client.16851 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:04:59.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.471+0000 7ff254c53b80 10 client.16851 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:04:59.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.471+0000 7ff254c53b80 20 client.16851 populate_metadata read hostname 'smithi037' 2024-06-10T01:04:59.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.471+0000 7ff254c53b80 1 --2- 172.21.15.37:0/394003510 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5585cdcf5bc0 0x5585cdcf7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:04:59.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.471+0000 7ff254c53b80 4 client.16851 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:04:59.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.471+0000 7ff254c53b80 1 -- 172.21.15.37:0/394003510 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5585cdbcad50 con 0x5585cdcf5bc0 2024-06-10T01:04:59.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.472+0000 7ff243fff700 1 --2- 172.21.15.37:0/394003510 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5585cdcf5bc0 0x5585cdcf7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:04:59.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.472+0000 7ff243fff700 1 --2- 172.21.15.37:0/394003510 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5585cdcf5bc0 0x5585cdcf7fe0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20ac3b5d142a64b2 server_cookie=cf800f6c82ec6032 in_seq=0 out_seq=0 2024-06-10T01:04:59.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.472+0000 7ff2237fe700 10 client.16851 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:59.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.474+0000 7ff2237fe700 1 -- 172.21.15.37:0/394003510 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5585cdbcad50 con 0x5585cdcf5bc0 2024-06-10T01:04:59.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.474+0000 7ff2237fe700 10 client.16851 handle_command_reply: tid=0 2024-06-10T01:04:59.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.474+0000 7ff254c53b80 1 client.16851 shutdown 2024-06-10T01:04:59.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.474+0000 7ff254c53b80 1 -- 172.21.15.37:0/394003510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585cdc71030 msgr2=0x5585cdc8e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:59.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.474+0000 7ff254c53b80 1 --2- 172.21.15.37:0/394003510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585cdc71030 0x5585cdc8e0c0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x5585cdc67e10 tx=0x7ff238009cd0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.474+0000 7ff254c53b80 1 -- 172.21.15.37:0/394003510 shutdown_connections 2024-06-10T01:04:59.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.474+0000 7ff254c53b80 1 --2- 172.21.15.37:0/394003510 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5585cdcf5bc0 0x5585cdcf7fe0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.474+0000 7ff254c53b80 1 --2- 172.21.15.37:0/394003510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585cdc76470 0x5585cdcf3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.474+0000 7ff254c53b80 1 --2- 172.21.15.37:0/394003510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585cdc70670 0x5585cdc8db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.474+0000 7ff254c53b80 1 --2- 172.21.15.37:0/394003510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585cdc71030 0x5585cdc8e0c0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.474+0000 7ff2237fe700 0 client.16851 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:04:59.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.474+0000 7ff254c53b80 1 -- 172.21.15.37:0/394003510 >> 172.21.15.37:0/394003510 conn(0x5585cdb99080 msgr2=0x5585cdc6bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:59.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.474+0000 7ff254c53b80 1 -- 172.21.15.37:0/394003510 shutdown_connections 2024-06-10T01:04:59.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.474+0000 7ff254c53b80 1 -- 172.21.15.37:0/394003510 wait complete. 2024-06-10T01:04:59.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.474+0000 7ff254c53b80 20 client.16851 trim_cache size 0 max 16384 2024-06-10T01:04:59.474 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:04:59.474 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-06-10T01:04:59.474 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-06-10T01:04:59.475 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:04:59.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.475+0000 7ff24a3d0700 1 -- 172.21.15.37:0/2727318554 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff22c0a2c20 msgr2=0x7ff22c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:59.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.475+0000 7ff24a3d0700 1 --2- 172.21.15.37:0/2727318554 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff22c0a2c20 0x7ff22c0a50e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff2380098e0 tx=0x7ff23800b790 comp rx=0 tx=0).stop 2024-06-10T01:04:59.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.475+0000 7ff24a3d0700 1 -- 172.21.15.37:0/2727318554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff244102740 msgr2=0x7ff24419e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:04:59.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.475+0000 7ff24a3d0700 1 --2- 172.21.15.37:0/2727318554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff244102740 0x7ff24419e830 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff23400e9f0 tx=0x7ff23400ed00 comp rx=0 tx=0).stop 2024-06-10T01:04:59.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.475+0000 7ff24a3d0700 1 -- 172.21.15.37:0/2727318554 shutdown_connections 2024-06-10T01:04:59.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.475+0000 7ff24a3d0700 1 --2- 172.21.15.37:0/2727318554 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff22c0a2c20 0x7ff22c0a50e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.475+0000 7ff24a3d0700 1 --2- 172.21.15.37:0/2727318554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff244107840 0x7ff2441989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.475+0000 7ff24a3d0700 1 --2- 172.21.15.37:0/2727318554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff244102740 0x7ff24419e830 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.475+0000 7ff24a3d0700 1 --2- 172.21.15.37:0/2727318554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2441030f0 0x7ff24419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:04:59.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.475+0000 7ff24a3d0700 1 -- 172.21.15.37:0/2727318554 >> 172.21.15.37:0/2727318554 conn(0x7ff2440795f0 msgr2=0x7ff244100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:04:59.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.475+0000 7ff24a3d0700 1 -- 172.21.15.37:0/2727318554 shutdown_connections 2024-06-10T01:04:59.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:04:59.475+0000 7ff24a3d0700 1 -- 172.21.15.37:0/2727318554 wait complete. 2024-06-10T01:05:00.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:00 smithi167 ceph-mon[36917]: pgmap v680: 129 pgs: 129 active+clean; 8.6 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.5 MiB/s wr, 2.36k op/s 2024-06-10T01:05:00.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:00 smithi094 ceph-mon[36708]: pgmap v680: 129 pgs: 129 active+clean; 8.6 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.5 MiB/s wr, 2.36k op/s 2024-06-10T01:05:00.362 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:00 smithi037 ceph-mon[32192]: pgmap v680: 129 pgs: 129 active+clean; 8.6 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.5 MiB/s wr, 2.36k op/s 2024-06-10T01:05:00.392 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:05:00.392 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:05:00.673 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:05:02.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:02 smithi037 ceph-mon[32192]: pgmap v681: 129 pgs: 129 active+clean; 14 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 2.35k op/s 2024-06-10T01:05:02.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:02 smithi167 ceph-mon[36917]: pgmap v681: 129 pgs: 129 active+clean; 14 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 2.35k op/s 2024-06-10T01:05:02.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:02 smithi094 ceph-mon[36708]: pgmap v681: 129 pgs: 129 active+clean; 14 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 2.35k op/s 2024-06-10T01:05:02.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.437+0000 7fb10483a700 1 -- 172.21.15.37:0/1612125784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb100108520 msgr2=0x7fb100108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:02.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.437+0000 7fb10483a700 1 --2- 172.21.15.37:0/1612125784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb100108520 0x7fb100108900 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009990 tx=0x7fb0f0009ca0 comp rx=0 tx=0).stop 2024-06-10T01:05:02.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.437+0000 7fb10483a700 1 -- 172.21.15.37:0/1612125784 shutdown_connections 2024-06-10T01:05:02.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.437+0000 7fb10483a700 1 --2- 172.21.15.37:0/1612125784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb10006b520 0x7fb10010f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:02.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.437+0000 7fb10483a700 1 --2- 172.21.15.37:0/1612125784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb100108520 0x7fb100108900 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:02.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.437+0000 7fb10483a700 1 --2- 172.21.15.37:0/1612125784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb10006ab60 0x7fb10006afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:02.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.437+0000 7fb10483a700 1 -- 172.21.15.37:0/1612125784 >> 172.21.15.37:0/1612125784 conn(0x7fb100078340 msgr2=0x7fb100078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:02.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.437+0000 7fb10483a700 1 -- 172.21.15.37:0/1612125784 shutdown_connections 2024-06-10T01:05:02.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.437+0000 7fb10483a700 1 -- 172.21.15.37:0/1612125784 wait complete. 2024-06-10T01:05:02.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.438+0000 7fb10483a700 1 Processor -- start 2024-06-10T01:05:02.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.438+0000 7fb10483a700 1 -- start start 2024-06-10T01:05:02.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.438+0000 7fb10483a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb10006ab60 0x7fb10019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:02.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.438+0000 7fb10483a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb10006b520 0x7fb10019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:02.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.438+0000 7fb10483a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb100108520 0x7fb100198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:02.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.438+0000 7fb10483a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb100112780 con 0x7fb100108520 2024-06-10T01:05:02.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.438+0000 7fb10483a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb100112600 con 0x7fb10006b520 2024-06-10T01:05:02.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.438+0000 7fb10483a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb100112900 con 0x7fb10006ab60 2024-06-10T01:05:02.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.439+0000 7fb0fe59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb10006ab60 0x7fb10019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:02.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.439+0000 7fb0fed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb100108520 0x7fb100198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:02.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.439+0000 7fb0fe59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb10006ab60 0x7fb10019e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:41364/0 (socket says 172.21.15.37:41364) 2024-06-10T01:05:02.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.439+0000 7fb0fed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb100108520 0x7fb100198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49058/0 (socket says 172.21.15.37:49058) 2024-06-10T01:05:02.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.439+0000 7fb0fe59c700 1 -- 172.21.15.37:0/2371167078 learned_addr learned my addr 172.21.15.37:0/2371167078 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:02.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.439+0000 7fb0fdd9b700 1 --2- 172.21.15.37:0/2371167078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb10006b520 0x7fb10019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:02.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.439+0000 7fb0fed9d700 1 -- 172.21.15.37:0/2371167078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb10006ab60 msgr2=0x7fb10019e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:02.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.439+0000 7fb0fed9d700 1 --2- 172.21.15.37:0/2371167078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb10006ab60 0x7fb10019e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:02.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.439+0000 7fb0fed9d700 1 -- 172.21.15.37:0/2371167078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb10006b520 msgr2=0x7fb10019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:02.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.439+0000 7fb0fed9d700 1 --2- 172.21.15.37:0/2371167078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb10006b520 0x7fb10019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:02.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.439+0000 7fb0fed9d700 1 -- 172.21.15.37:0/2371167078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb100199160 con 0x7fb100108520 2024-06-10T01:05:02.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.439+0000 7fb0fdd9b700 1 --2- 172.21.15.37:0/2371167078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb10006b520 0x7fb10019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:05:02.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.439+0000 7fb0fed9d700 1 --2- 172.21.15.37:0/2371167078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb100108520 0x7fb100198a00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8007f10 tx=0x7fb0f800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:02.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.439+0000 7fb0ef7fe700 1 -- 172.21.15.37:0/2371167078 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f800ddd0 con 0x7fb100108520 2024-06-10T01:05:02.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.439+0000 7fb0ef7fe700 1 -- 172.21.15.37:0/2371167078 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0f8009e20 con 0x7fb100108520 2024-06-10T01:05:02.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.439+0000 7fb0ef7fe700 1 -- 172.21.15.37:0/2371167078 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8015660 con 0x7fb100108520 2024-06-10T01:05:02.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.439+0000 7fb10483a700 1 -- 172.21.15.37:0/2371167078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb100199450 con 0x7fb100108520 2024-06-10T01:05:02.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.440+0000 7fb10483a700 1 -- 172.21.15.37:0/2371167078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1001a5710 con 0x7fb100108520 2024-06-10T01:05:02.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.440+0000 7fb10483a700 1 -- 172.21.15.37:0/2371167078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb10010aa00 con 0x7fb100108520 2024-06-10T01:05:02.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.443+0000 7fb0ef7fe700 1 -- 172.21.15.37:0/2371167078 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb0f8004ad0 con 0x7fb100108520 2024-06-10T01:05:02.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.443+0000 7fb0ef7fe700 1 --2- 172.21.15.37:0/2371167078 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb0e80a2c20 0x7fb0e80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:02.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.443+0000 7fb0ef7fe700 1 -- 172.21.15.37:0/2371167078 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fb0f80cf5f0 con 0x7fb100108520 2024-06-10T01:05:02.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.443+0000 7fb0fe59c700 1 --2- 172.21.15.37:0/2371167078 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb0e80a2c20 0x7fb0e80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:02.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.444+0000 7fb0fe59c700 1 --2- 172.21.15.37:0/2371167078 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb0e80a2c20 0x7fb0e80a50e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009750 tx=0x7fb0f001a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:02.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.449+0000 7fb0ef7fe700 1 -- 172.21.15.37:0/2371167078 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb0f8010070 con 0x7fb100108520 2024-06-10T01:05:02.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.712+0000 7fb10483a700 1 -- 172.21.15.37:0/2371167078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb100103be0 con 0x7fb100108520 2024-06-10T01:05:02.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.713+0000 7fb0ef7fe700 1 -- 172.21.15.37:0/2371167078 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11864 (secure 0 0 0) 0x7fb0f809c160 con 0x7fb100108520 2024-06-10T01:05:02.710 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:05:02.710 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T01:03:24.246183+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26110":{"gid":26110,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/317977487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":317977487},{"type":"v1","addr":"172.21.15.167:6835","nonce":317977487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24458,24472,24451]},"id":1}]} 2024-06-10T01:05:02.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.715+0000 7fb10483a700 1 -- 172.21.15.37:0/2371167078 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb0e80a2c20 msgr2=0x7fb0e80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:02.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.715+0000 7fb10483a700 1 --2- 172.21.15.37:0/2371167078 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb0e80a2c20 0x7fb0e80a50e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009750 tx=0x7fb0f001a040 comp rx=0 tx=0).stop 2024-06-10T01:05:02.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.716+0000 7fb10483a700 1 -- 172.21.15.37:0/2371167078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb100108520 msgr2=0x7fb100198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:02.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.716+0000 7fb10483a700 1 --2- 172.21.15.37:0/2371167078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb100108520 0x7fb100198a00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8007f10 tx=0x7fb0f800d3b0 comp rx=0 tx=0).stop 2024-06-10T01:05:02.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.716+0000 7fb10483a700 1 -- 172.21.15.37:0/2371167078 shutdown_connections 2024-06-10T01:05:02.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.716+0000 7fb10483a700 1 --2- 172.21.15.37:0/2371167078 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb0e80a2c20 0x7fb0e80a50e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:02.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.716+0000 7fb10483a700 1 --2- 172.21.15.37:0/2371167078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb100108520 0x7fb100198a00 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:02.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.716+0000 7fb10483a700 1 --2- 172.21.15.37:0/2371167078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb10006ab60 0x7fb10019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:02.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.716+0000 7fb10483a700 1 --2- 172.21.15.37:0/2371167078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb10006b520 0x7fb10019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:02.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.716+0000 7fb10483a700 1 -- 172.21.15.37:0/2371167078 >> 172.21.15.37:0/2371167078 conn(0x7fb100078340 msgr2=0x7fb1001003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:02.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.716+0000 7fb10483a700 1 -- 172.21.15.37:0/2371167078 shutdown_connections 2024-06-10T01:05:02.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:02.716+0000 7fb10483a700 1 -- 172.21.15.37:0/2371167078 wait complete. 2024-06-10T01:05:02.714 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 29 2024-06-10T01:05:03.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:03 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2371167078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:05:03.348 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:03 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2371167078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:05:03.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:03 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2371167078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:05:03.406 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:05:03.661 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:05:04.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:04 smithi167 ceph-mon[36917]: pgmap v682: 129 pgs: 129 active+clean; 14 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.4 MiB/s wr, 2.27k op/s 2024-06-10T01:05:04.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:04 smithi094 ceph-mon[36708]: pgmap v682: 129 pgs: 129 active+clean; 14 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.4 MiB/s wr, 2.27k op/s 2024-06-10T01:05:04.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:04 smithi037 ceph-mon[32192]: pgmap v682: 129 pgs: 129 active+clean; 14 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.4 MiB/s wr, 2.27k op/s 2024-06-10T01:05:05.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.400+0000 7f28ad2f6700 1 -- 172.21.15.37:0/4067386798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28a8108500 msgr2=0x7f28a81088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:05.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.400+0000 7f28ad2f6700 1 --2- 172.21.15.37:0/4067386798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28a8108500 0x7f28a81088e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f2898009990 tx=0x7f2898009ca0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.401+0000 7f28ad2f6700 1 -- 172.21.15.37:0/4067386798 shutdown_connections 2024-06-10T01:05:05.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.401+0000 7f28ad2f6700 1 --2- 172.21.15.37:0/4067386798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28a806b5e0 0x7f28a810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.401+0000 7f28ad2f6700 1 --2- 172.21.15.37:0/4067386798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28a8108500 0x7f28a81088e0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.401+0000 7f28ad2f6700 1 --2- 172.21.15.37:0/4067386798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28a806ac20 0x7f28a806b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.401+0000 7f28ad2f6700 1 -- 172.21.15.37:0/4067386798 >> 172.21.15.37:0/4067386798 conn(0x7f28a8078340 msgr2=0x7f28a8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:05.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.401+0000 7f28ad2f6700 1 -- 172.21.15.37:0/4067386798 shutdown_connections 2024-06-10T01:05:05.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.401+0000 7f28ad2f6700 1 -- 172.21.15.37:0/4067386798 wait complete. 2024-06-10T01:05:05.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.401+0000 7f28ad2f6700 1 Processor -- start 2024-06-10T01:05:05.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.402+0000 7f28ad2f6700 1 -- start start 2024-06-10T01:05:05.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.402+0000 7f28ad2f6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28a806ac20 0x7f28a8101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:05.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.402+0000 7f28ad2f6700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28a806b5e0 0x7f28a81019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:05.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.402+0000 7f28ad2f6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28a8108500 0x7f28a8101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:05.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.402+0000 7f28ad2f6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f28a8112780 con 0x7f28a806ac20 2024-06-10T01:05:05.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.402+0000 7f28ad2f6700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f28a8112600 con 0x7f28a8108500 2024-06-10T01:05:05.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.402+0000 7f28ad2f6700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f28a8112900 con 0x7f28a806b5e0 2024-06-10T01:05:05.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.402+0000 7f28a6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28a806ac20 0x7f28a8101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:05.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.402+0000 7f28a6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28a806ac20 0x7f28a8101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49080/0 (socket says 172.21.15.37:49080) 2024-06-10T01:05:05.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.402+0000 7f28a6ffd700 1 -- 172.21.15.37:0/3710419171 learned_addr learned my addr 172.21.15.37:0/3710419171 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:05.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.402+0000 7f28a67fc700 1 --2- 172.21.15.37:0/3710419171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28a806b5e0 0x7f28a81019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:05.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.402+0000 7f28a77fe700 1 --2- 172.21.15.37:0/3710419171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28a8108500 0x7f28a8101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:05.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.402+0000 7f28a6ffd700 1 -- 172.21.15.37:0/3710419171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28a806b5e0 msgr2=0x7f28a81019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:05.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.402+0000 7f28a6ffd700 1 --2- 172.21.15.37:0/3710419171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28a806b5e0 0x7f28a81019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.402+0000 7f28a6ffd700 1 -- 172.21.15.37:0/3710419171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28a8108500 msgr2=0x7f28a8101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:05.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.402+0000 7f28a6ffd700 1 --2- 172.21.15.37:0/3710419171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28a8108500 0x7f28a8101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.402+0000 7f28a6ffd700 1 -- 172.21.15.37:0/3710419171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28a81a71e0 con 0x7f28a806ac20 2024-06-10T01:05:05.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.402+0000 7f28a67fc700 1 --2- 172.21.15.37:0/3710419171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28a806b5e0 0x7f28a81019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:05.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.403+0000 7f28a6ffd700 1 --2- 172.21.15.37:0/3710419171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28a806ac20 0x7f28a8101460 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f28980038b0 tx=0x7f2898009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:05.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.403+0000 7f288ffff700 1 -- 172.21.15.37:0/3710419171 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2898027070 con 0x7f28a806ac20 2024-06-10T01:05:05.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.403+0000 7f288ffff700 1 -- 172.21.15.37:0/3710419171 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2898004050 con 0x7f28a806ac20 2024-06-10T01:05:05.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.403+0000 7f28ad2f6700 1 -- 172.21.15.37:0/3710419171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28a81a7470 con 0x7f28a806ac20 2024-06-10T01:05:05.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.403+0000 7f288ffff700 1 -- 172.21.15.37:0/3710419171 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2898022c00 con 0x7f28a806ac20 2024-06-10T01:05:05.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.403+0000 7f28ad2f6700 1 -- 172.21.15.37:0/3710419171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28a81a7950 con 0x7f28a806ac20 2024-06-10T01:05:05.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.405+0000 7f288ffff700 1 -- 172.21.15.37:0/3710419171 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f289802a070 con 0x7f28a806ac20 2024-06-10T01:05:05.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.406+0000 7f288ffff700 1 --2- 172.21.15.37:0/3710419171 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f28900a2c20 0x7f28900a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:05.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.406+0000 7f28a67fc700 1 --2- 172.21.15.37:0/3710419171 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f28900a2c20 0x7f28900a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:05.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.406+0000 7f288ffff700 1 -- 172.21.15.37:0/3710419171 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f28980d0340 con 0x7f28a806ac20 2024-06-10T01:05:05.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.407+0000 7f28a67fc700 1 --2- 172.21.15.37:0/3710419171 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f28900a2c20 0x7f28900a50e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f28a8078910 tx=0x7f289c00a300 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:05.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.407+0000 7f28b7b79b80 1 Processor -- start 2024-06-10T01:05:05.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.407+0000 7f28b7b79b80 1 -- start start 2024-06-10T01:05:05.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.407+0000 7f28b7b79b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636f3529640 0x5636f3529a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:05.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.407+0000 7f28b7b79b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636f3529f60 0x5636f35239e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:05.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.407+0000 7f28b7b79b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636f3523f20 0x5636f3524340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:05.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.407+0000 7f28b7b79b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5636f3475810 con 0x5636f3529640 2024-06-10T01:05:05.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.407+0000 7f28b7b79b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5636f341d720 con 0x5636f3523f20 2024-06-10T01:05:05.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.407+0000 7f28b7b79b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5636f34717d0 con 0x5636f3529f60 2024-06-10T01:05:05.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.407+0000 7f28a67fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636f3529f60 0x5636f35239e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:05.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.407+0000 7f28a6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636f3529640 0x5636f3529a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:05.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.407+0000 7f28a67fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636f3529f60 0x5636f35239e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:41400/0 (socket says 172.21.15.37:41400) 2024-06-10T01:05:05.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.407+0000 7f28a67fc700 1 -- 172.21.15.37:0/3660419641 learned_addr learned my addr 172.21.15.37:0/3660419641 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:05.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.407+0000 7f28a77fe700 1 --2- 172.21.15.37:0/3660419641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636f3523f20 0x5636f3524340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:05.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.408+0000 7f28a67fc700 1 -- 172.21.15.37:0/3660419641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636f3523f20 msgr2=0x5636f3524340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:05.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.408+0000 7f28a67fc700 1 --2- 172.21.15.37:0/3660419641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636f3523f20 0x5636f3524340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.408+0000 7f28a67fc700 1 -- 172.21.15.37:0/3660419641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636f3529640 msgr2=0x5636f3529a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:05.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.408+0000 7f28a67fc700 1 --2- 172.21.15.37:0/3660419641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636f3529640 0x5636f3529a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.408+0000 7f28a67fc700 1 -- 172.21.15.37:0/3660419641 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636f3413f50 con 0x5636f3529f60 2024-06-10T01:05:05.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.408+0000 7f28a77fe700 1 --2- 172.21.15.37:0/3660419641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636f3523f20 0x5636f3524340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:05:05.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.408+0000 7f28a6ffd700 1 --2- 172.21.15.37:0/3660419641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636f3529640 0x5636f3529a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:05.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.408+0000 7f28a67fc700 1 --2- 172.21.15.37:0/3660419641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636f3529f60 0x5636f35239e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f289c010530 tx=0x7f289c012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32845829d35fb142 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:05.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.408+0000 7f288d7fa700 1 -- 172.21.15.37:0/3660419641 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c016070 con 0x5636f3529f60 2024-06-10T01:05:05.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.408+0000 7f288d7fa700 1 -- 172.21.15.37:0/3660419641 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f289c010bf0 con 0x5636f3529f60 2024-06-10T01:05:05.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.408+0000 7f288d7fa700 1 -- 172.21.15.37:0/3660419641 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c01b660 con 0x5636f3529f60 2024-06-10T01:05:05.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.409+0000 7f28b7b79b80 1 -- 172.21.15.37:0/3660419641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636f3529f60 msgr2=0x5636f35239e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:05.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.409+0000 7f28b7b79b80 1 --2- 172.21.15.37:0/3660419641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636f3529f60 0x5636f35239e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f289c010530 tx=0x7f289c012000 comp rx=0 tx=0).stop 2024-06-10T01:05:05.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.409+0000 7f28b7b79b80 1 -- 172.21.15.37:0/3660419641 shutdown_connections 2024-06-10T01:05:05.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.409+0000 7f28b7b79b80 1 --2- 172.21.15.37:0/3660419641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636f3523f20 0x5636f3524340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.409+0000 7f28b7b79b80 1 --2- 172.21.15.37:0/3660419641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636f3529640 0x5636f3529a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.409+0000 7f28b7b79b80 1 --2- 172.21.15.37:0/3660419641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636f3529f60 0x5636f35239e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.409+0000 7f28b7b79b80 1 -- 172.21.15.37:0/3660419641 >> 172.21.15.37:0/3660419641 conn(0x5636f344ecc0 msgr2=0x5636f351e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:05.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.409+0000 7f28b7b79b80 1 -- 172.21.15.37:0/3660419641 shutdown_connections 2024-06-10T01:05:05.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.409+0000 7f28b7b79b80 1 -- 172.21.15.37:0/3660419641 wait complete. 2024-06-10T01:05:05.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.409+0000 7f28b7b79b80 1 -- start start 2024-06-10T01:05:05.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.409+0000 7f28b7b79b80 1 Processor -- start 2024-06-10T01:05:05.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.410+0000 7f28b7b79b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636f3523f20 0x5636f352fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:05.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.410+0000 7f28b7b79b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636f3529640 0x5636f352d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:05.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.410+0000 7f28b7b79b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636f3529f60 0x5636f352d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:05.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.410+0000 7f28b7b79b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5636f35309d0 con 0x5636f3523f20 2024-06-10T01:05:05.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.410+0000 7f28b7b79b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5636f3415bf0 con 0x5636f3529f60 2024-06-10T01:05:05.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.410+0000 7f28b7b79b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5636f3530b50 con 0x5636f3529640 2024-06-10T01:05:05.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.410+0000 7f28b7b79b80 10 client.0 _ll_register_callbacks cb 0x5636f340f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:05:05.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.410+0000 7f28a6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636f3523f20 0x5636f352fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:05.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.410+0000 7f28a67fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636f3529640 0x5636f352d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:05.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.410+0000 7f28a6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636f3523f20 0x5636f352fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49102/0 (socket says 172.21.15.37:49102) 2024-06-10T01:05:05.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.410+0000 7f28a67fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636f3529640 0x5636f352d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:41402/0 (socket says 172.21.15.37:41402) 2024-06-10T01:05:05.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.410+0000 7f28a6ffd700 1 -- 172.21.15.37:0/1380136918 learned_addr learned my addr 172.21.15.37:0/1380136918 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:05.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.410+0000 7f28a77fe700 1 --2- 172.21.15.37:0/1380136918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636f3529f60 0x5636f352d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:05.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.410+0000 7f28a67fc700 1 -- 172.21.15.37:0/1380136918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636f3529f60 msgr2=0x5636f352d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:05.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.410+0000 7f28a67fc700 1 --2- 172.21.15.37:0/1380136918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636f3529f60 0x5636f352d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.410+0000 7f28a67fc700 1 -- 172.21.15.37:0/1380136918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636f3523f20 msgr2=0x5636f352fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:05.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.410+0000 7f28a67fc700 1 --2- 172.21.15.37:0/1380136918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636f3523f20 0x5636f352fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.410+0000 7f28a67fc700 1 -- 172.21.15.37:0/1380136918 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636f33c48d0 con 0x5636f3529640 2024-06-10T01:05:05.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.410+0000 7f28a6ffd700 1 --2- 172.21.15.37:0/1380136918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636f3523f20 0x5636f352fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:05.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.410+0000 7f28a77fe700 1 --2- 172.21.15.37:0/1380136918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636f3529f60 0x5636f352d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:05:05.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.411+0000 7f28a67fc700 1 --2- 172.21.15.37:0/1380136918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636f3529640 0x5636f352d090 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f289c00adc0 tx=0x7f289c025c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=392b4d9caf3c8453 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:05.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.411+0000 7f288aff5700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:05:05.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.411+0000 7f288aff5700 1 -- 172.21.15.37:0/1380136918 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c033060 con 0x5636f3529640 2024-06-10T01:05:05.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.411+0000 7f28b7b79b80 1 -- 172.21.15.37:0/1380136918 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f289c010bf0 con 0x5636f3529640 2024-06-10T01:05:05.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.411+0000 7f288aff5700 1 -- 172.21.15.37:0/1380136918 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f289c01c420 con 0x5636f3529640 2024-06-10T01:05:05.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.411+0000 7f288aff5700 1 -- 172.21.15.37:0/1380136918 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f289c03a050 con 0x5636f3529640 2024-06-10T01:05:05.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.411+0000 7f28b7b79b80 10 client.26407 fetch_fsmap learned FSMap version 29 2024-06-10T01:05:05.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.411+0000 7f28b7b79b80 1 -- 172.21.15.37:0/1380136918 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5636f353b320 con 0x5636f3529640 2024-06-10T01:05:05.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.411+0000 7f288aff5700 1 -- 172.21.15.37:0/1380136918 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f289c010bf0 con 0x5636f3529640 2024-06-10T01:05:05.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.411+0000 7f288aff5700 1 -- 172.21.15.37:0/1380136918 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f289c01b630 con 0x5636f3529640 2024-06-10T01:05:05.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.412+0000 7f28b7b79b80 10 client.26407 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:05:05.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.412+0000 7f28b7b79b80 10 client.26407 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-10T01:05:05.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.412+0000 7f28b7b79b80 20 client.26407 populate_metadata read hostname 'smithi037' 2024-06-10T01:05:05.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.412+0000 7f28b7b79b80 1 --2- 172.21.15.37:0/1380136918 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5636f353d900 0x5636f353fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:05.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.412+0000 7f28b7b79b80 4 client.26407 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:05:05.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.412+0000 7f28b7b79b80 1 -- 172.21.15.37:0/1380136918 --> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636f3482d10 con 0x5636f353d900 2024-06-10T01:05:05.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.412+0000 7f28a6ffd700 1 --2- 172.21.15.37:0/1380136918 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5636f353d900 0x5636f353fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:05.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.412+0000 7f28a6ffd700 1 --2- 172.21.15.37:0/1380136918 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5636f353d900 0x5636f353fcf0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d78afc666d58d6b6 server_cookie=7b517dd2d8f31ebb in_seq=0 out_seq=0 2024-06-10T01:05:05.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.412+0000 7f288aff5700 10 client.26407 ms_handle_connect on v2:172.21.15.167:6836/2720148065 2024-06-10T01:05:05.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.413+0000 7f288aff5700 1 -- 172.21.15.37:0/1380136918 <== mds.2 v2:172.21.15.167:6836/2720148065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5636f3482d10 con 0x5636f353d900 2024-06-10T01:05:05.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.413+0000 7f288aff5700 10 client.26407 handle_command_reply: tid=0 2024-06-10T01:05:05.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.413+0000 7f28b7b79b80 1 client.26407 shutdown 2024-06-10T01:05:05.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.413+0000 7f28b7b79b80 1 -- 172.21.15.37:0/1380136918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636f3529640 msgr2=0x5636f352d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:05.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.413+0000 7f28b7b79b80 1 --2- 172.21.15.37:0/1380136918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636f3529640 0x5636f352d090 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f289c00adc0 tx=0x7f289c025c50 comp rx=0 tx=0).stop 2024-06-10T01:05:05.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.413+0000 7f28b7b79b80 1 -- 172.21.15.37:0/1380136918 shutdown_connections 2024-06-10T01:05:05.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.413+0000 7f28b7b79b80 1 --2- 172.21.15.37:0/1380136918 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5636f353d900 0x5636f353fcf0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.413+0000 7f28b7b79b80 1 --2- 172.21.15.37:0/1380136918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636f3529f60 0x5636f352d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.413+0000 7f28b7b79b80 1 --2- 172.21.15.37:0/1380136918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636f3523f20 0x5636f352fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.413+0000 7f288aff5700 0 client.26407 ms_handle_reset on v2:172.21.15.167:6836/2720148065 2024-06-10T01:05:05.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.413+0000 7f28b7b79b80 1 --2- 172.21.15.37:0/1380136918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636f3529640 0x5636f352d090 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.413+0000 7f28b7b79b80 1 -- 172.21.15.37:0/1380136918 >> 172.21.15.37:0/1380136918 conn(0x5636f344ecc0 msgr2=0x5636f35229d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:05.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.414+0000 7f28b7b79b80 1 -- 172.21.15.37:0/1380136918 shutdown_connections 2024-06-10T01:05:05.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.414+0000 7f28b7b79b80 1 -- 172.21.15.37:0/1380136918 wait complete. 2024-06-10T01:05:05.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.414+0000 7f28b7b79b80 20 client.26407 trim_cache size 0 max 16384 2024-06-10T01:05:05.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.429+0000 7f28b7b79b80 1 Processor -- start 2024-06-10T01:05:05.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.429+0000 7f28b7b79b80 1 -- start start 2024-06-10T01:05:05.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.429+0000 7f28b7b79b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636f3523f20 0x5636f3539ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:05.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.429+0000 7f28b7b79b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636f3529640 0x5636f35239e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:05.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.429+0000 7f28b7b79b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636f3529f60 0x5636f3518510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:05.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.430+0000 7f28b7b79b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f28a81b2e80 con 0x5636f3529f60 2024-06-10T01:05:05.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.430+0000 7f28b7b79b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f28a81b0710 con 0x5636f3529640 2024-06-10T01:05:05.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.430+0000 7f28b7b79b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f28a81ae560 con 0x5636f3523f20 2024-06-10T01:05:05.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.430+0000 7f28a77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636f3529f60 0x5636f3518510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:05.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.430+0000 7f28a6ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636f3523f20 0x5636f3539ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:05.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.430+0000 7f28a6ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636f3523f20 0x5636f3539ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:41414/0 (socket says 172.21.15.37:41414) 2024-06-10T01:05:05.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.430+0000 7f28a77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636f3529f60 0x5636f3518510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49118/0 (socket says 172.21.15.37:49118) 2024-06-10T01:05:05.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.430+0000 7f28a6ffd700 1 -- 172.21.15.37:0/3813460589 learned_addr learned my addr 172.21.15.37:0/3813460589 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:05.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.430+0000 7f28a67fc700 1 --2- 172.21.15.37:0/3813460589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636f3529640 0x5636f35239e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:05.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.430+0000 7f28a6ffd700 1 -- 172.21.15.37:0/3813460589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636f3529640 msgr2=0x5636f35239e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:05.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.430+0000 7f28a6ffd700 1 --2- 172.21.15.37:0/3813460589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636f3529640 0x5636f35239e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.430+0000 7f28a6ffd700 1 -- 172.21.15.37:0/3813460589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636f3529f60 msgr2=0x5636f3518510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:05.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.430+0000 7f28a6ffd700 1 --2- 172.21.15.37:0/3813460589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636f3529f60 0x5636f3518510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.430+0000 7f28a6ffd700 1 -- 172.21.15.37:0/3813460589 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636f3529a20 con 0x5636f3523f20 2024-06-10T01:05:05.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.430+0000 7f28a77fe700 1 --2- 172.21.15.37:0/3813460589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636f3529f60 0x5636f3518510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:05.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.430+0000 7f28a6ffd700 1 --2- 172.21.15.37:0/3813460589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636f3523f20 0x5636f3539ca0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f28980991b0 tx=0x7f289801a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa42cb36054f3273 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:05.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.430+0000 7f288cff9700 1 -- 172.21.15.37:0/3813460589 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28980a30b0 con 0x5636f3523f20 2024-06-10T01:05:05.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.430+0000 7f288cff9700 1 -- 172.21.15.37:0/3813460589 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f289809c7e0 con 0x5636f3523f20 2024-06-10T01:05:05.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.430+0000 7f288cff9700 1 -- 172.21.15.37:0/3813460589 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2898022470 con 0x5636f3523f20 2024-06-10T01:05:05.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.431+0000 7f28b7b79b80 1 -- 172.21.15.37:0/3813460589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636f3523f20 msgr2=0x5636f3539ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:05.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.431+0000 7f28b7b79b80 1 --2- 172.21.15.37:0/3813460589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636f3523f20 0x5636f3539ca0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f28980991b0 tx=0x7f289801a040 comp rx=0 tx=0).stop 2024-06-10T01:05:05.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.431+0000 7f28b7b79b80 1 -- 172.21.15.37:0/3813460589 shutdown_connections 2024-06-10T01:05:05.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.431+0000 7f28b7b79b80 1 --2- 172.21.15.37:0/3813460589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636f3529f60 0x5636f3518510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.431+0000 7f28b7b79b80 1 --2- 172.21.15.37:0/3813460589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636f3523f20 0x5636f3539ca0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.431+0000 7f28b7b79b80 1 --2- 172.21.15.37:0/3813460589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636f3529640 0x5636f35239e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.431+0000 7f28b7b79b80 1 -- 172.21.15.37:0/3813460589 >> 172.21.15.37:0/3813460589 conn(0x5636f344ecc0 msgr2=0x5636f3517e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:05.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.431+0000 7f28b7b79b80 1 -- 172.21.15.37:0/3813460589 shutdown_connections 2024-06-10T01:05:05.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.431+0000 7f28b7b79b80 1 -- 172.21.15.37:0/3813460589 wait complete. 2024-06-10T01:05:05.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.432+0000 7f28b7b79b80 1 -- start start 2024-06-10T01:05:05.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.432+0000 7f28b7b79b80 1 Processor -- start 2024-06-10T01:05:05.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.432+0000 7f28b7b79b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636f3523f20 0x5636f3536640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:05.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.432+0000 7f28b7b79b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636f3529640 0x5636f3544dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:05.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.432+0000 7f28b7b79b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636f3529f60 0x5636f35453e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:05.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.432+0000 7f28b7b79b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f28a81aebd0 con 0x5636f3523f20 2024-06-10T01:05:05.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.432+0000 7f28b7b79b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f28a81b0040 con 0x5636f3529f60 2024-06-10T01:05:05.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.432+0000 7f28b7b79b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f28a81af2a0 con 0x5636f3529640 2024-06-10T01:05:05.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.432+0000 7f28b7b79b80 10 client.0 _ll_register_callbacks cb 0x5636f340f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:05:05.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.432+0000 7f28a6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636f3523f20 0x5636f3536640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:05.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.432+0000 7f28a67fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636f3529640 0x5636f3544dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:05.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.432+0000 7f28a6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636f3523f20 0x5636f3536640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49120/0 (socket says 172.21.15.37:49120) 2024-06-10T01:05:05.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.432+0000 7f28a6ffd700 1 -- 172.21.15.37:0/3736984331 learned_addr learned my addr 172.21.15.37:0/3736984331 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:05.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.432+0000 7f28a77fe700 1 --2- 172.21.15.37:0/3736984331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636f3529f60 0x5636f35453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:05.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.432+0000 7f28a6ffd700 1 -- 172.21.15.37:0/3736984331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636f3529640 msgr2=0x5636f3544dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:05.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.432+0000 7f28a6ffd700 1 --2- 172.21.15.37:0/3736984331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636f3529640 0x5636f3544dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.432+0000 7f28a6ffd700 1 -- 172.21.15.37:0/3736984331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636f3529f60 msgr2=0x5636f35453e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:05.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.432+0000 7f28a6ffd700 1 --2- 172.21.15.37:0/3736984331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636f3529f60 0x5636f35453e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.432+0000 7f28a6ffd700 1 -- 172.21.15.37:0/3736984331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636f3545be0 con 0x5636f3523f20 2024-06-10T01:05:05.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.432+0000 7f28a67fc700 1 --2- 172.21.15.37:0/3736984331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636f3529640 0x5636f3544dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:05.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.433+0000 7f28a6ffd700 1 --2- 172.21.15.37:0/3736984331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636f3523f20 0x5636f3536640 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f2898009550 tx=0x7f2898005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a88ec1a3c683938 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:05.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.433+0000 7f288aff5700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:05:05.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.433+0000 7f288aff5700 1 -- 172.21.15.37:0/3736984331 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28980b9020 con 0x5636f3523f20 2024-06-10T01:05:05.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.433+0000 7f288aff5700 1 -- 172.21.15.37:0/3736984331 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f289809aae0 con 0x5636f3523f20 2024-06-10T01:05:05.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.433+0000 7f28b7b79b80 1 -- 172.21.15.37:0/3736984331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f289809c7e0 con 0x5636f3523f20 2024-06-10T01:05:05.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.433+0000 7f288aff5700 1 -- 172.21.15.37:0/3736984331 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f28980a30b0 con 0x5636f3523f20 2024-06-10T01:05:05.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.433+0000 7f28b7b79b80 10 client.16875 fetch_fsmap learned FSMap version 29 2024-06-10T01:05:05.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.433+0000 7f28b7b79b80 1 -- 172.21.15.37:0/3736984331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5636f35a8470 con 0x5636f3523f20 2024-06-10T01:05:05.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.433+0000 7f288aff5700 1 -- 172.21.15.37:0/3736984331 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f289809c7e0 con 0x5636f3523f20 2024-06-10T01:05:05.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.434+0000 7f288aff5700 1 -- 172.21.15.37:0/3736984331 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f289809bc30 con 0x5636f3523f20 2024-06-10T01:05:05.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.434+0000 7f28b7b79b80 10 client.16875 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:05:05.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.434+0000 7f28b7b79b80 10 client.16875 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-10T01:05:05.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.434+0000 7f28b7b79b80 20 client.16875 populate_metadata read hostname 'smithi037' 2024-06-10T01:05:05.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.434+0000 7f28b7b79b80 1 --2- 172.21.15.37:0/3736984331 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5636f353d900 0x5636f35aaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:05.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.434+0000 7f28b7b79b80 4 client.16875 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:05:05.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.434+0000 7f28b7b79b80 1 -- 172.21.15.37:0/3736984331 --> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5636f347d290 con 0x5636f353d900 2024-06-10T01:05:05.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.434+0000 7f28a67fc700 1 --2- 172.21.15.37:0/3736984331 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5636f353d900 0x5636f35aaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:05.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.434+0000 7f28a67fc700 1 --2- 172.21.15.37:0/3736984331 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5636f353d900 0x5636f35aaa80 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f025b8223824684b server_cookie=e96baf5b93e04484 in_seq=0 out_seq=0 2024-06-10T01:05:05.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.434+0000 7f288aff5700 10 client.16875 ms_handle_connect on v2:172.21.15.167:6836/2720148065 2024-06-10T01:05:05.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.434+0000 7f288aff5700 1 -- 172.21.15.37:0/3736984331 <== mds.2 v2:172.21.15.167:6836/2720148065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5636f347d290 con 0x5636f353d900 2024-06-10T01:05:05.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.434+0000 7f288aff5700 10 client.16875 handle_command_reply: tid=0 2024-06-10T01:05:05.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.434+0000 7f28b7b79b80 1 client.16875 shutdown 2024-06-10T01:05:05.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.435+0000 7f28b7b79b80 1 -- 172.21.15.37:0/3736984331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636f3523f20 msgr2=0x5636f3536640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:05.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.435+0000 7f28b7b79b80 1 --2- 172.21.15.37:0/3736984331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636f3523f20 0x5636f3536640 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f2898009550 tx=0x7f2898005a00 comp rx=0 tx=0).stop 2024-06-10T01:05:05.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.435+0000 7f28b7b79b80 1 -- 172.21.15.37:0/3736984331 shutdown_connections 2024-06-10T01:05:05.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.435+0000 7f28b7b79b80 1 --2- 172.21.15.37:0/3736984331 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5636f353d900 0x5636f35aaa80 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.435+0000 7f28b7b79b80 1 --2- 172.21.15.37:0/3736984331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636f3529f60 0x5636f35453e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.435+0000 7f28b7b79b80 1 --2- 172.21.15.37:0/3736984331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636f3523f20 0x5636f3536640 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.435+0000 7f28b7b79b80 1 --2- 172.21.15.37:0/3736984331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636f3529640 0x5636f3544dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.435+0000 7f28b7b79b80 1 -- 172.21.15.37:0/3736984331 >> 172.21.15.37:0/3736984331 conn(0x5636f344ecc0 msgr2=0x5636f352a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:05.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.435+0000 7f288aff5700 0 client.16875 ms_handle_reset on v2:172.21.15.167:6836/2720148065 2024-06-10T01:05:05.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.435+0000 7f28b7b79b80 1 -- 172.21.15.37:0/3736984331 shutdown_connections 2024-06-10T01:05:05.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.435+0000 7f28b7b79b80 1 -- 172.21.15.37:0/3736984331 wait complete. 2024-06-10T01:05:05.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.435+0000 7f28b7b79b80 20 client.16875 trim_cache size 0 max 16384 2024-06-10T01:05:05.437 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T01:05:05.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.435+0000 7f28ad2f6700 1 -- 172.21.15.37:0/3710419171 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f28900a2c20 msgr2=0x7f28900a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:05.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.435+0000 7f28ad2f6700 1 --2- 172.21.15.37:0/3710419171 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f28900a2c20 0x7f28900a50e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f28a8078910 tx=0x7f289c00a300 comp rx=0 tx=0).stop 2024-06-10T01:05:05.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.435+0000 7f28ad2f6700 1 -- 172.21.15.37:0/3710419171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28a806ac20 msgr2=0x7f28a8101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:05.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.435+0000 7f28ad2f6700 1 --2- 172.21.15.37:0/3710419171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28a806ac20 0x7f28a8101460 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f28980038b0 tx=0x7f2898009f50 comp rx=0 tx=0).stop 2024-06-10T01:05:05.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.435+0000 7f28ad2f6700 1 -- 172.21.15.37:0/3710419171 shutdown_connections 2024-06-10T01:05:05.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.435+0000 7f28ad2f6700 1 --2- 172.21.15.37:0/3710419171 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f28900a2c20 0x7f28900a50e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.435+0000 7f28ad2f6700 1 --2- 172.21.15.37:0/3710419171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28a8108500 0x7f28a8101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.435+0000 7f28ad2f6700 1 --2- 172.21.15.37:0/3710419171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28a806ac20 0x7f28a8101460 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.435+0000 7f28ad2f6700 1 --2- 172.21.15.37:0/3710419171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28a806b5e0 0x7f28a81019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:05.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.435+0000 7f28ad2f6700 1 -- 172.21.15.37:0/3710419171 >> 172.21.15.37:0/3710419171 conn(0x7f28a8078340 msgr2=0x7f28a80fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:05.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.436+0000 7f28ad2f6700 1 -- 172.21.15.37:0/3710419171 shutdown_connections 2024-06-10T01:05:05.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:05.436+0000 7f28ad2f6700 1 -- 172.21.15.37:0/3710419171 wait complete. 2024-06-10T01:05:06.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.087+0000 7f65a7f12640 1 -- 172.21.15.37:0/1509387443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65a0104e20 msgr2=0x7f65a01052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:06.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.087+0000 7f65a7f12640 1 --2- 172.21.15.37:0/1509387443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65a0104e20 0x7f65a01052a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6590009900 tx=0x7f659002a410 comp rx=0 tx=0).stop 2024-06-10T01:05:06.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.087+0000 7f65a7f12640 1 -- 172.21.15.37:0/1509387443 shutdown_connections 2024-06-10T01:05:06.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.087+0000 7f65a7f12640 1 --2- 172.21.15.37:0/1509387443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65a01057e0 0x7f65a010c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:06.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.087+0000 7f65a7f12640 1 --2- 172.21.15.37:0/1509387443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65a0104e20 0x7f65a01052a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.087+0000 7f65a7f12640 1 --2- 172.21.15.37:0/1509387443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65a0103c30 0x7f65a0104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.087+0000 7f65a7f12640 1 -- 172.21.15.37:0/1509387443 >> 172.21.15.37:0/1509387443 conn(0x7f65a00ff1c0 msgr2=0x7f65a01015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.087+0000 7f65a7f12640 1 -- 172.21.15.37:0/1509387443 shutdown_connections 2024-06-10T01:05:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.087+0000 7f65a7f12640 1 -- 172.21.15.37:0/1509387443 wait complete. 2024-06-10T01:05:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.088+0000 7f65a7f12640 1 Processor -- start 2024-06-10T01:05:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.088+0000 7f65a7f12640 1 -- start start 2024-06-10T01:05:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.088+0000 7f65a7f12640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65a0103c30 0x7f65a0199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.088+0000 7f65a7f12640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65a0104e20 0x7f65a019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.088+0000 7f65a7f12640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65a01057e0 0x7f65a01a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.088+0000 7f65a7f12640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f65a010f890 con 0x7f65a01057e0 2024-06-10T01:05:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.088+0000 7f65a7f12640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f65a010f710 con 0x7f65a0103c30 2024-06-10T01:05:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.088+0000 7f65a7f12640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f65a010fa10 con 0x7f65a0104e20 2024-06-10T01:05:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.088+0000 7f65a64af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65a01057e0 0x7f65a01a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.088+0000 7f65a64af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65a01057e0 0x7f65a01a13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49128/0 (socket says 172.21.15.37:49128) 2024-06-10T01:05:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.088+0000 7f65a64af640 1 -- 172.21.15.37:0/3498463740 learned_addr learned my addr 172.21.15.37:0/3498463740 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.088+0000 7f65a54ad640 1 --2- 172.21.15.37:0/3498463740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65a0104e20 0x7f65a019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.088+0000 7f65a5cae640 1 --2- 172.21.15.37:0/3498463740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65a0103c30 0x7f65a0199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.088+0000 7f65a64af640 1 -- 172.21.15.37:0/3498463740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65a0104e20 msgr2=0x7f65a019a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.088+0000 7f65a64af640 1 --2- 172.21.15.37:0/3498463740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65a0104e20 0x7f65a019a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.088+0000 7f65a64af640 1 -- 172.21.15.37:0/3498463740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65a0103c30 msgr2=0x7f65a0199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.088+0000 7f65a64af640 1 --2- 172.21.15.37:0/3498463740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65a0103c30 0x7f65a0199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.088+0000 7f65a64af640 1 -- 172.21.15.37:0/3498463740 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a01a1a60 con 0x7f65a01057e0 2024-06-10T01:05:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.088+0000 7f65a54ad640 1 --2- 172.21.15.37:0/3498463740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65a0104e20 0x7f65a019a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.088+0000 7f65a5cae640 1 --2- 172.21.15.37:0/3498463740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65a0103c30 0x7f65a0199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:05:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.089+0000 7f65a64af640 1 --2- 172.21.15.37:0/3498463740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65a01057e0 0x7f65a01a13f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f659800eb10 tx=0x7f659800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.089+0000 7f6596ffd640 1 -- 172.21.15.37:0/3498463740 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6598004b20 con 0x7f65a01057e0 2024-06-10T01:05:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.089+0000 7f6596ffd640 1 -- 172.21.15.37:0/3498463740 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6598004cc0 con 0x7f65a01057e0 2024-06-10T01:05:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.089+0000 7f65a7f12640 1 -- 172.21.15.37:0/3498463740 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65a01a1d50 con 0x7f65a01057e0 2024-06-10T01:05:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.089+0000 7f6596ffd640 1 -- 172.21.15.37:0/3498463740 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6598010760 con 0x7f65a01057e0 2024-06-10T01:05:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.089+0000 7f65a7f12640 1 -- 172.21.15.37:0/3498463740 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65a01a21c0 con 0x7f65a01057e0 2024-06-10T01:05:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.092+0000 7f6596ffd640 1 -- 172.21.15.37:0/3498463740 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6598005020 con 0x7f65a01057e0 2024-06-10T01:05:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.092+0000 7f65aac17740 1 Processor -- start 2024-06-10T01:05:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.093+0000 7f65aac17740 1 -- start start 2024-06-10T01:05:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.093+0000 7f65aac17740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56548c5f1010 0x56548c5f1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.093+0000 7f65aac17740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56548c5f1a00 0x56548c5f6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.093+0000 7f65aac17740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56548c5f6b70 0x56548c5f9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.093+0000 7f65aac17740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56548c5544f0 con 0x56548c5f1a00 2024-06-10T01:05:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.093+0000 7f65aac17740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56548c4f2870 con 0x56548c5f6b70 2024-06-10T01:05:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.093+0000 7f65aac17740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56548c5504b0 con 0x56548c5f1010 2024-06-10T01:05:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.093+0000 7f6596ffd640 1 --2- 172.21.15.37:0/3498463740 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f658c0a2c30 0x7f658c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.093+0000 7f65a5cae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56548c5f1a00 0x56548c5f6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.093+0000 7f65a54ad640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56548c5f1010 0x56548c5f1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.093+0000 7f65a54ad640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56548c5f1010 0x56548c5f1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:41458/0 (socket says 172.21.15.37:41458) 2024-06-10T01:05:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.093+0000 7f65a5cae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56548c5f1a00 0x56548c5f6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49136/0 (socket says 172.21.15.37:49136) 2024-06-10T01:05:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.093+0000 7f65a54ad640 1 -- 172.21.15.37:0/227139906 learned_addr learned my addr 172.21.15.37:0/227139906 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.093+0000 7f65a64af640 1 --2- 172.21.15.37:0/3498463740 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f658c0a2c30 0x7f658c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.093+0000 7f65a54ad640 1 --2- 172.21.15.37:0/227139906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56548c5f6b70 0x56548c5f9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.093+0000 7f6596ffd640 1 -- 172.21.15.37:0/3498463740 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f65980cfa30 con 0x7f65a01057e0 2024-06-10T01:05:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.093+0000 7f65a5cae640 1 -- 172.21.15.37:0/227139906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56548c5f1010 msgr2=0x56548c5f1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.093+0000 7f65a5cae640 1 --2- 172.21.15.37:0/227139906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56548c5f1010 0x56548c5f1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.093+0000 7f65a5cae640 1 -- 172.21.15.37:0/227139906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56548c5f6b70 msgr2=0x56548c5f9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.093+0000 7f65a5cae640 1 --2- 172.21.15.37:0/227139906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56548c5f6b70 0x56548c5f9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.093+0000 7f65a5cae640 1 -- 172.21.15.37:0/227139906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56548c50e9b0 con 0x56548c5f1a00 2024-06-10T01:05:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.093+0000 7f65a54ad640 1 --2- 172.21.15.37:0/227139906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56548c5f1010 0x56548c5f1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.094+0000 7f65a64af640 1 --2- 172.21.15.37:0/3498463740 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f658c0a2c30 0x7f658c0a50f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f65980049d0 tx=0x7f659806ef00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.094+0000 7f65a5cae640 1 --2- 172.21.15.37:0/227139906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56548c5f1a00 0x56548c5f6500 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f659c005d00 tx=0x7f659c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2204b13bbc0d53e4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.094+0000 7f658bfff640 1 -- 172.21.15.37:0/227139906 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c016070 con 0x56548c5f1a00 2024-06-10T01:05:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.094+0000 7f658bfff640 1 -- 172.21.15.37:0/227139906 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f659c011950 con 0x56548c5f1a00 2024-06-10T01:05:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.094+0000 7f658bfff640 1 -- 172.21.15.37:0/227139906 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c0107d0 con 0x56548c5f1a00 2024-06-10T01:05:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.095+0000 7f65aac17740 1 -- 172.21.15.37:0/227139906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56548c5f1a00 msgr2=0x56548c5f6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.095+0000 7f65aac17740 1 --2- 172.21.15.37:0/227139906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56548c5f1a00 0x56548c5f6500 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f659c005d00 tx=0x7f659c00e6c0 comp rx=0 tx=0).stop 2024-06-10T01:05:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.095+0000 7f65aac17740 1 -- 172.21.15.37:0/227139906 shutdown_connections 2024-06-10T01:05:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.095+0000 7f65aac17740 1 --2- 172.21.15.37:0/227139906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56548c5f6b70 0x56548c5f9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.095+0000 7f65aac17740 1 --2- 172.21.15.37:0/227139906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56548c5f1a00 0x56548c5f6500 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.095+0000 7f65aac17740 1 --2- 172.21.15.37:0/227139906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56548c5f1010 0x56548c5f1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.095+0000 7f65aac17740 1 -- 172.21.15.37:0/227139906 >> 172.21.15.37:0/227139906 conn(0x56548c5ec680 msgr2=0x56548c5eca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.095+0000 7f65aac17740 1 -- 172.21.15.37:0/227139906 shutdown_connections 2024-06-10T01:05:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.095+0000 7f65aac17740 1 -- 172.21.15.37:0/227139906 wait complete. 2024-06-10T01:05:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.095+0000 7f65aac17740 1 -- start start 2024-06-10T01:05:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.095+0000 7f65aac17740 1 Processor -- start 2024-06-10T01:05:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.096+0000 7f65aac17740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56548c5f1010 0x56548c5fb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.096+0000 7f65aac17740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56548c5f1a00 0x56548c5fba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.096+0000 7f65aac17740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56548c5f6b70 0x56548c602a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.096+0000 7f65aac17740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56548c5f9b20 con 0x56548c5f6b70 2024-06-10T01:05:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.096+0000 7f65aac17740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56548c4ddae0 con 0x56548c5f1a00 2024-06-10T01:05:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.096+0000 7f65aac17740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56548c5f9ca0 con 0x56548c5f1010 2024-06-10T01:05:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.096+0000 7f65aac17740 10 client.0 _ll_register_callbacks cb 0x56548c5169b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:05:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.096+0000 7f65a54ad640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56548c5f1010 0x56548c5fb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.096+0000 7f65a54ad640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56548c5f1010 0x56548c5fb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:41470/0 (socket says 172.21.15.37:41470) 2024-06-10T01:05:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.096+0000 7f65a54ad640 1 -- 172.21.15.37:0/693310007 learned_addr learned my addr 172.21.15.37:0/693310007 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.096+0000 7f65a54ad640 1 --2- 172.21.15.37:0/693310007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56548c5f6b70 0x56548c602a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.096+0000 7f65a5cae640 1 --2- 172.21.15.37:0/693310007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56548c5f1a00 0x56548c5fba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.096+0000 7f65a54ad640 1 -- 172.21.15.37:0/693310007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56548c5f1a00 msgr2=0x56548c5fba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.096+0000 7f65a54ad640 1 --2- 172.21.15.37:0/693310007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56548c5f1a00 0x56548c5fba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.096+0000 7f65a54ad640 1 -- 172.21.15.37:0/693310007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56548c5f6b70 msgr2=0x56548c602a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.096+0000 7f65a54ad640 1 --2- 172.21.15.37:0/693310007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56548c5f6b70 0x56548c602a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.096+0000 7f65a54ad640 1 -- 172.21.15.37:0/693310007 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56548c268210 con 0x56548c5f1010 2024-06-10T01:05:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.096+0000 7f65a54ad640 1 --2- 172.21.15.37:0/693310007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56548c5f1010 0x56548c5fb4c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6590009900 tx=0x7f6590031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8841121faf17b2e9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.096+0000 7f65897fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:05:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.097+0000 7f65897fa640 1 -- 172.21.15.37:0/693310007 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6590035040 con 0x56548c5f1010 2024-06-10T01:05:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.097+0000 7f65897fa640 1 -- 172.21.15.37:0/693310007 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f6590004050 con 0x56548c5f1010 2024-06-10T01:05:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.097+0000 7f65aac17740 1 -- 172.21.15.37:0/693310007 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f659c011950 con 0x56548c5f1010 2024-06-10T01:05:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.097+0000 7f65897fa640 1 -- 172.21.15.37:0/693310007 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6590054050 con 0x56548c5f1010 2024-06-10T01:05:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.097+0000 7f65aac17740 10 client.26425 fetch_fsmap learned FSMap version 29 2024-06-10T01:05:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.097+0000 7f65aac17740 1 -- 172.21.15.37:0/693310007 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56548c334620 con 0x56548c5f1010 2024-06-10T01:05:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.097+0000 7f65897fa640 1 -- 172.21.15.37:0/693310007 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f659c011950 con 0x56548c5f1010 2024-06-10T01:05:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.097+0000 7f65897fa640 1 -- 172.21.15.37:0/693310007 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f65900035f0 con 0x56548c5f1010 2024-06-10T01:05:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.097+0000 7f65aac17740 10 client.26425 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:05:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.097+0000 7f65aac17740 10 client.26425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:05:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.097+0000 7f65aac17740 20 client.26425 populate_metadata read hostname 'smithi037' 2024-06-10T01:05:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.097+0000 7f65aac17740 1 --2- 172.21.15.37:0/693310007 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56548c6072d0 0x56548c609700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.097+0000 7f65aac17740 4 client.26425 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:05:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.097+0000 7f65aac17740 1 -- 172.21.15.37:0/693310007 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56548c567f10 con 0x56548c6072d0 2024-06-10T01:05:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.098+0000 7f65a5cae640 1 --2- 172.21.15.37:0/693310007 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56548c6072d0 0x56548c609700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.098+0000 7f65a5cae640 1 --2- 172.21.15.37:0/693310007 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56548c6072d0 0x56548c609700 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df75d55fe4434374 server_cookie=73581ad66d354410 in_seq=0 out_seq=0 2024-06-10T01:05:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.098+0000 7f65897fa640 10 client.26425 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:05:06.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.102+0000 7f65897fa640 1 -- 172.21.15.37:0/693310007 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56548c567f10 con 0x56548c6072d0 2024-06-10T01:05:06.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.102+0000 7f65897fa640 10 client.26425 handle_command_reply: tid=0 2024-06-10T01:05:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.102+0000 7f65aac17740 1 client.26425 shutdown 2024-06-10T01:05:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.103+0000 7f65aac17740 1 -- 172.21.15.37:0/693310007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56548c5f1010 msgr2=0x56548c5fb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.103+0000 7f65aac17740 1 --2- 172.21.15.37:0/693310007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56548c5f1010 0x56548c5fb4c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6590009900 tx=0x7f6590031400 comp rx=0 tx=0).stop 2024-06-10T01:05:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.103+0000 7f65aac17740 1 -- 172.21.15.37:0/693310007 shutdown_connections 2024-06-10T01:05:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.103+0000 7f65aac17740 1 --2- 172.21.15.37:0/693310007 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56548c6072d0 0x56548c609700 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.103+0000 7f65aac17740 1 --2- 172.21.15.37:0/693310007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56548c5f6b70 0x56548c602a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.103+0000 7f65aac17740 1 --2- 172.21.15.37:0/693310007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56548c5f1a00 0x56548c5fba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.103+0000 7f65897fa640 0 client.26425 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:05:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.103+0000 7f65aac17740 1 --2- 172.21.15.37:0/693310007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56548c5f1010 0x56548c5fb4c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.103+0000 7f65aac17740 1 -- 172.21.15.37:0/693310007 >> 172.21.15.37:0/693310007 conn(0x56548c5ec680 msgr2=0x56548c5eb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.103+0000 7f65aac17740 1 -- 172.21.15.37:0/693310007 shutdown_connections 2024-06-10T01:05:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.103+0000 7f65aac17740 1 -- 172.21.15.37:0/693310007 wait complete. 2024-06-10T01:05:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.103+0000 7f65aac17740 20 client.26425 trim_cache size 0 max 16384 2024-06-10T01:05:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.119+0000 7f65aac17740 1 Processor -- start 2024-06-10T01:05:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.120+0000 7f65aac17740 1 -- start start 2024-06-10T01:05:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.120+0000 7f65aac17740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56548c5f1010 0x56548c5fb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.120+0000 7f65aac17740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56548c5f1a00 0x56548c602a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.120+0000 7f65aac17740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56548c5f6b70 0x56548c5e7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.120+0000 7f65aac17740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f65a01af180 con 0x56548c5f6b70 2024-06-10T01:05:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.120+0000 7f65aac17740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f65a01aca50 con 0x56548c5f1a00 2024-06-10T01:05:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.120+0000 7f65aac17740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f65a01aa8a0 con 0x56548c5f1010 2024-06-10T01:05:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.120+0000 7f65a5cae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56548c5f1a00 0x56548c602a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.120+0000 7f65a54ad640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56548c5f1010 0x56548c5fb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.120+0000 7f65a5cae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56548c5f1a00 0x56548c602a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:43124/0 (socket says 172.21.15.37:43124) 2024-06-10T01:05:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.120+0000 7f65a54ad640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56548c5f1010 0x56548c5fb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:41484/0 (socket says 172.21.15.37:41484) 2024-06-10T01:05:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.120+0000 7f65a5cae640 1 -- 172.21.15.37:0/1474556352 learned_addr learned my addr 172.21.15.37:0/1474556352 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.120+0000 7f65a54ad640 1 --2- 172.21.15.37:0/1474556352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56548c5f6b70 0x56548c5e7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.120+0000 7f65a5cae640 1 -- 172.21.15.37:0/1474556352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56548c5f1010 msgr2=0x56548c5fb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.120+0000 7f65a5cae640 1 --2- 172.21.15.37:0/1474556352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56548c5f1010 0x56548c5fb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.120+0000 7f65a5cae640 1 -- 172.21.15.37:0/1474556352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56548c5f6b70 msgr2=0x56548c5e7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.120+0000 7f65a5cae640 1 --2- 172.21.15.37:0/1474556352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56548c5f6b70 0x56548c5e7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.120+0000 7f65a5cae640 1 -- 172.21.15.37:0/1474556352 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56548c5f1770 con 0x56548c5f1a00 2024-06-10T01:05:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.120+0000 7f65a54ad640 1 --2- 172.21.15.37:0/1474556352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56548c5f1010 0x56548c5fb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.121+0000 7f65a5cae640 1 --2- 172.21.15.37:0/1474556352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56548c5f1a00 0x56548c602a00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f659c000c00 tx=0x7f659c00ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6912770b5242be33 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.121+0000 7f658b7fe640 1 -- 172.21.15.37:0/1474556352 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c016040 con 0x56548c5f1a00 2024-06-10T01:05:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.121+0000 7f658b7fe640 1 -- 172.21.15.37:0/1474556352 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f659c005520 con 0x56548c5f1a00 2024-06-10T01:05:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.121+0000 7f658b7fe640 1 -- 172.21.15.37:0/1474556352 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c004d70 con 0x56548c5f1a00 2024-06-10T01:05:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.122+0000 7f65aac17740 1 -- 172.21.15.37:0/1474556352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56548c5f1a00 msgr2=0x56548c602a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.122+0000 7f65aac17740 1 --2- 172.21.15.37:0/1474556352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56548c5f1a00 0x56548c602a00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f659c000c00 tx=0x7f659c00ec40 comp rx=0 tx=0).stop 2024-06-10T01:05:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.122+0000 7f65aac17740 1 -- 172.21.15.37:0/1474556352 shutdown_connections 2024-06-10T01:05:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.122+0000 7f65aac17740 1 --2- 172.21.15.37:0/1474556352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56548c5f6b70 0x56548c5e7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.122+0000 7f65aac17740 1 --2- 172.21.15.37:0/1474556352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56548c5f1a00 0x56548c602a00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.122+0000 7f65aac17740 1 --2- 172.21.15.37:0/1474556352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56548c5f1010 0x56548c5fb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.122+0000 7f65aac17740 1 -- 172.21.15.37:0/1474556352 >> 172.21.15.37:0/1474556352 conn(0x56548c5ec680 msgr2=0x56548c5eca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.122+0000 7f65aac17740 1 -- 172.21.15.37:0/1474556352 shutdown_connections 2024-06-10T01:05:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.122+0000 7f65aac17740 1 -- 172.21.15.37:0/1474556352 wait complete. 2024-06-10T01:05:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.122+0000 7f65aac17740 1 -- start start 2024-06-10T01:05:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.122+0000 7f65aac17740 1 Processor -- start 2024-06-10T01:05:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.122+0000 7f65aac17740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56548c5f1010 0x56548c5ffc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.122+0000 7f65aac17740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56548c5f1a00 0x56548c600180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.122+0000 7f65aac17740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56548c5f6b70 0x56548c601140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.122+0000 7f65aac17740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f65a01aaf10 con 0x56548c5f1010 2024-06-10T01:05:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.122+0000 7f65aac17740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f65a01ac380 con 0x56548c5f6b70 2024-06-10T01:05:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.122+0000 7f65aac17740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f65a01ab5e0 con 0x56548c5f1a00 2024-06-10T01:05:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.122+0000 7f65aac17740 10 client.0 _ll_register_callbacks cb 0x7f6590031790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:05:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.123+0000 7f65a54ad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56548c5f1010 0x56548c5ffc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.123+0000 7f65a5cae640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56548c5f1a00 0x56548c600180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.123+0000 7f65a5cae640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56548c5f1a00 0x56548c600180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:41496/0 (socket says 172.21.15.37:41496) 2024-06-10T01:05:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.123+0000 7f65a5cae640 1 -- 172.21.15.37:0/1486246607 learned_addr learned my addr 172.21.15.37:0/1486246607 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.123+0000 7f65a54ad640 1 --2- 172.21.15.37:0/1486246607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56548c5f6b70 0x56548c601140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.123+0000 7f65a5cae640 1 -- 172.21.15.37:0/1486246607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56548c5f6b70 msgr2=0x56548c601140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.123+0000 7f65a5cae640 1 --2- 172.21.15.37:0/1486246607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56548c5f6b70 0x56548c601140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.123+0000 7f65a5cae640 1 -- 172.21.15.37:0/1486246607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56548c5f1010 msgr2=0x56548c5ffc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.123+0000 7f65a5cae640 1 --2- 172.21.15.37:0/1486246607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56548c5f1010 0x56548c5ffc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.123+0000 7f65a5cae640 1 -- 172.21.15.37:0/1486246607 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56548c601730 con 0x56548c5f1a00 2024-06-10T01:05:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.123+0000 7f65a54ad640 1 --2- 172.21.15.37:0/1486246607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56548c5f1010 0x56548c5ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.123+0000 7f65a5cae640 1 --2- 172.21.15.37:0/1486246607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56548c5f1a00 0x56548c600180 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f659c009560 tx=0x7f659c009590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab212c06f1210bc0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.123+0000 7f65897fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:05:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.124+0000 7f65897fa640 1 -- 172.21.15.37:0/1486246607 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c016050 con 0x56548c5f1a00 2024-06-10T01:05:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.124+0000 7f65897fa640 1 -- 172.21.15.37:0/1486246607 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f659c009740 con 0x56548c5f1a00 2024-06-10T01:05:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.124+0000 7f65aac17740 1 -- 172.21.15.37:0/1486246607 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f659c005520 con 0x56548c5f1a00 2024-06-10T01:05:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.124+0000 7f65897fa640 1 -- 172.21.15.37:0/1486246607 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f659c004af0 con 0x56548c5f1a00 2024-06-10T01:05:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.124+0000 7f65aac17740 10 client.26431 fetch_fsmap learned FSMap version 29 2024-06-10T01:05:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.124+0000 7f65aac17740 1 -- 172.21.15.37:0/1486246607 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56548c602a40 con 0x56548c5f1a00 2024-06-10T01:05:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.124+0000 7f65897fa640 1 -- 172.21.15.37:0/1486246607 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f659c005520 con 0x56548c5f1a00 2024-06-10T01:05:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.124+0000 7f65897fa640 1 -- 172.21.15.37:0/1486246607 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f659c00f040 con 0x56548c5f1a00 2024-06-10T01:05:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.124+0000 7f65aac17740 10 client.26431 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:05:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.124+0000 7f65aac17740 10 client.26431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:05:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.124+0000 7f65aac17740 20 client.26431 populate_metadata read hostname 'smithi037' 2024-06-10T01:05:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.124+0000 7f65aac17740 1 --2- 172.21.15.37:0/1486246607 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56548c6721d0 0x56548c6745c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.124+0000 7f65aac17740 4 client.26431 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:05:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.124+0000 7f65aac17740 1 -- 172.21.15.37:0/1486246607 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56548c561c20 con 0x56548c6721d0 2024-06-10T01:05:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.124+0000 7f65a54ad640 1 --2- 172.21.15.37:0/1486246607 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56548c6721d0 0x56548c6745c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.125+0000 7f65a54ad640 1 --2- 172.21.15.37:0/1486246607 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56548c6721d0 0x56548c6745c0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f74a60900aed696b server_cookie=9fff25c7d018b379 in_seq=0 out_seq=0 2024-06-10T01:05:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:06.125+0000 7f65897fa640 10 client.26431 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:05:06.143 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:05:06.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:06 smithi167 ceph-mon[36917]: pgmap v683: 129 pgs: 129 active+clean; 17 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 1.48k op/s 2024-06-10T01:05:06.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:06 smithi094 ceph-mon[36708]: pgmap v683: 129 pgs: 129 active+clean; 17 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 1.48k op/s 2024-06-10T01:05:06.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:06 smithi037 ceph-mon[32192]: pgmap v683: 129 pgs: 129 active+clean; 17 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 1.48k op/s 2024-06-10T01:05:06.399 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:05:08.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.130+0000 7fba63d0e700 1 -- 172.21.15.37:0/4133538660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba5c06a170 msgr2=0x7fba5c06a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:08.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.130+0000 7fba63d0e700 1 --2- 172.21.15.37:0/4133538660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba5c06a170 0x7fba5c06a5f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009990 tx=0x7fba4c009ca0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.130+0000 7fba63d0e700 1 -- 172.21.15.37:0/4133538660 shutdown_connections 2024-06-10T01:05:08.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.130+0000 7fba63d0e700 1 --2- 172.21.15.37:0/4133538660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba5c06ab30 0x7fba5c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.130+0000 7fba63d0e700 1 --2- 172.21.15.37:0/4133538660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba5c108500 0x7fba5c1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.130+0000 7fba63d0e700 1 --2- 172.21.15.37:0/4133538660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba5c06a170 0x7fba5c06a5f0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.130+0000 7fba63d0e700 1 -- 172.21.15.37:0/4133538660 >> 172.21.15.37:0/4133538660 conn(0x7fba5c078340 msgr2=0x7fba5c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:08.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.130+0000 7fba63d0e700 1 -- 172.21.15.37:0/4133538660 shutdown_connections 2024-06-10T01:05:08.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.130+0000 7fba63d0e700 1 -- 172.21.15.37:0/4133538660 wait complete. 2024-06-10T01:05:08.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.131+0000 7fba63d0e700 1 Processor -- start 2024-06-10T01:05:08.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.131+0000 7fba63d0e700 1 -- start start 2024-06-10T01:05:08.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.131+0000 7fba63d0e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba5c06a170 0x7fba5c19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:08.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.131+0000 7fba63d0e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba5c06ab30 0x7fba5c19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:08.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.131+0000 7fba63d0e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba5c108500 0x7fba5c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:08.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.131+0000 7fba63d0e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c112780 con 0x7fba5c108500 2024-06-10T01:05:08.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.131+0000 7fba63d0e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c112600 con 0x7fba5c06ab30 2024-06-10T01:05:08.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.131+0000 7fba63d0e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c112900 con 0x7fba5c06a170 2024-06-10T01:05:08.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.131+0000 7fba622ab700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba5c108500 0x7fba5c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:08.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.131+0000 7fba61aaa700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba5c06a170 0x7fba5c19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:08.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.131+0000 7fba622ab700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba5c108500 0x7fba5c198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49170/0 (socket says 172.21.15.37:49170) 2024-06-10T01:05:08.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.131+0000 7fba622ab700 1 -- 172.21.15.37:0/3566639791 learned_addr learned my addr 172.21.15.37:0/3566639791 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:08.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.131+0000 7fba612a9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba5c06ab30 0x7fba5c19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:08.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.132+0000 7fba622ab700 1 -- 172.21.15.37:0/3566639791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba5c06a170 msgr2=0x7fba5c19e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:08.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.132+0000 7fba622ab700 1 --2- 172.21.15.37:0/3566639791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba5c06a170 0x7fba5c19e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.132+0000 7fba622ab700 1 -- 172.21.15.37:0/3566639791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba5c06ab30 msgr2=0x7fba5c19edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:08.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.132+0000 7fba622ab700 1 --2- 172.21.15.37:0/3566639791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba5c06ab30 0x7fba5c19edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.132+0000 7fba622ab700 1 -- 172.21.15.37:0/3566639791 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba5c1991a0 con 0x7fba5c108500 2024-06-10T01:05:08.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.132+0000 7fba61aaa700 1 --2- 172.21.15.37:0/3566639791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba5c06a170 0x7fba5c19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:08.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.132+0000 7fba612a9700 1 --2- 172.21.15.37:0/3566639791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba5c06ab30 0x7fba5c19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:08.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.132+0000 7fba622ab700 1 --2- 172.21.15.37:0/3566639791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba5c108500 0x7fba5c198a40 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fba5400ef30 tx=0x7fba5400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:08.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.132+0000 7fba52ffd700 1 -- 172.21.15.37:0/3566639791 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba540092e0 con 0x7fba5c108500 2024-06-10T01:05:08.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.132+0000 7fba63d0e700 1 -- 172.21.15.37:0/3566639791 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba5c199430 con 0x7fba5c108500 2024-06-10T01:05:08.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.132+0000 7fba52ffd700 1 -- 172.21.15.37:0/3566639791 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba54004bb0 con 0x7fba5c108500 2024-06-10T01:05:08.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.132+0000 7fba52ffd700 1 -- 172.21.15.37:0/3566639791 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba54004280 con 0x7fba5c108500 2024-06-10T01:05:08.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.132+0000 7fba63d0e700 1 -- 172.21.15.37:0/3566639791 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba5c1a5750 con 0x7fba5c108500 2024-06-10T01:05:08.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.135+0000 7fba52ffd700 1 -- 172.21.15.37:0/3566639791 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fba5401b030 con 0x7fba5c108500 2024-06-10T01:05:08.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.135+0000 7fba52ffd700 1 --2- 172.21.15.37:0/3566639791 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fba480a2ca0 0x7fba480a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:08.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.136+0000 7fba61aaa700 1 --2- 172.21.15.37:0/3566639791 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fba480a2ca0 0x7fba480a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:08.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.136+0000 7fba52ffd700 1 -- 172.21.15.37:0/3566639791 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fba540cee90 con 0x7fba5c108500 2024-06-10T01:05:08.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.136+0000 7fba6e591b80 1 Processor -- start 2024-06-10T01:05:08.133 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:08 smithi037 ceph-mon[32192]: pgmap v684: 129 pgs: 129 active+clean; 17 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 199 op/s 2024-06-10T01:05:08.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.136+0000 7fba61aaa700 1 --2- 172.21.15.37:0/3566639791 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fba480a2ca0 0x7fba480a5160 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fba5c199b80 tx=0x7fba58009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:08.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.136+0000 7fba6e591b80 1 -- start start 2024-06-10T01:05:08.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.136+0000 7fba6e591b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9c3dcc640 0x55f9c3dcca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:08.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.136+0000 7fba6e591b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9c3dccf60 0x55f9c3dc69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:08.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.136+0000 7fba6e591b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9c3dc6f20 0x55f9c3dc7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:08.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.136+0000 7fba6e591b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f9c3d18810 con 0x55f9c3dc6f20 2024-06-10T01:05:08.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.136+0000 7fba6e591b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f9c3cc0720 con 0x55f9c3dcc640 2024-06-10T01:05:08.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.136+0000 7fba6e591b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f9c3d147d0 con 0x55f9c3dccf60 2024-06-10T01:05:08.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.137+0000 7fba622ab700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9c3dc6f20 0x55f9c3dc7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:08.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.137+0000 7fba622ab700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9c3dc6f20 0x55f9c3dc7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49186/0 (socket says 172.21.15.37:49186) 2024-06-10T01:05:08.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.137+0000 7fba622ab700 1 -- 172.21.15.37:0/3819112955 learned_addr learned my addr 172.21.15.37:0/3819112955 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:08.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.137+0000 7fba612a9700 1 --2- 172.21.15.37:0/3819112955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9c3dccf60 0x55f9c3dc69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:08.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.137+0000 7fba61aaa700 1 --2- 172.21.15.37:0/3819112955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9c3dcc640 0x55f9c3dcca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:08.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.137+0000 7fba612a9700 1 -- 172.21.15.37:0/3819112955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9c3dcc640 msgr2=0x55f9c3dcca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:08.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.137+0000 7fba612a9700 1 --2- 172.21.15.37:0/3819112955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9c3dcc640 0x55f9c3dcca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.137+0000 7fba612a9700 1 -- 172.21.15.37:0/3819112955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9c3dc6f20 msgr2=0x55f9c3dc7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:08.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.137+0000 7fba612a9700 1 --2- 172.21.15.37:0/3819112955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9c3dc6f20 0x55f9c3dc7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.137+0000 7fba612a9700 1 -- 172.21.15.37:0/3819112955 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9c3cb6f50 con 0x55f9c3dccf60 2024-06-10T01:05:08.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.137+0000 7fba622ab700 1 --2- 172.21.15.37:0/3819112955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9c3dc6f20 0x55f9c3dc7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:08.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.137+0000 7fba612a9700 1 --2- 172.21.15.37:0/3819112955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9c3dccf60 0x55f9c3dc69e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fba4c0059d0 tx=0x7fba4c009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e730872ea951431 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:08.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.137+0000 7fba47fff700 1 -- 172.21.15.37:0/3819112955 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c00b610 con 0x55f9c3dccf60 2024-06-10T01:05:08.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.138+0000 7fba47fff700 1 -- 172.21.15.37:0/3819112955 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba4c00bd60 con 0x55f9c3dccf60 2024-06-10T01:05:08.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.138+0000 7fba47fff700 1 -- 172.21.15.37:0/3819112955 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c005280 con 0x55f9c3dccf60 2024-06-10T01:05:08.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.138+0000 7fba6e591b80 1 -- 172.21.15.37:0/3819112955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9c3dccf60 msgr2=0x55f9c3dc69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:08.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.138+0000 7fba6e591b80 1 --2- 172.21.15.37:0/3819112955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9c3dccf60 0x55f9c3dc69e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fba4c0059d0 tx=0x7fba4c009700 comp rx=0 tx=0).stop 2024-06-10T01:05:08.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.138+0000 7fba6e591b80 1 -- 172.21.15.37:0/3819112955 shutdown_connections 2024-06-10T01:05:08.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.138+0000 7fba6e591b80 1 --2- 172.21.15.37:0/3819112955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9c3dc6f20 0x55f9c3dc7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.138+0000 7fba6e591b80 1 --2- 172.21.15.37:0/3819112955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9c3dcc640 0x55f9c3dcca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.138+0000 7fba6e591b80 1 --2- 172.21.15.37:0/3819112955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9c3dccf60 0x55f9c3dc69e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.138+0000 7fba6e591b80 1 -- 172.21.15.37:0/3819112955 >> 172.21.15.37:0/3819112955 conn(0x55f9c3cf1cc0 msgr2=0x55f9c3dc1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:08.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.139+0000 7fba6e591b80 1 -- 172.21.15.37:0/3819112955 shutdown_connections 2024-06-10T01:05:08.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.139+0000 7fba6e591b80 1 -- 172.21.15.37:0/3819112955 wait complete. 2024-06-10T01:05:08.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.139+0000 7fba6e591b80 1 -- start start 2024-06-10T01:05:08.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.139+0000 7fba6e591b80 1 Processor -- start 2024-06-10T01:05:08.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.140+0000 7fba6e591b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9c3dc6f20 0x55f9c3dd2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:08.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.140+0000 7fba6e591b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9c3dcc640 0x55f9c3dd0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:08.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.140+0000 7fba6e591b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9c3dccf60 0x55f9c3dd05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:08.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.140+0000 7fba6e591b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f9c3dd3970 con 0x55f9c3dccf60 2024-06-10T01:05:08.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.140+0000 7fba6e591b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f9c3cb8bf0 con 0x55f9c3dcc640 2024-06-10T01:05:08.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.140+0000 7fba6e591b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f9c3dd3af0 con 0x55f9c3dc6f20 2024-06-10T01:05:08.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.140+0000 7fba6e591b80 10 client.0 _ll_register_callbacks cb 0x55f9c3cb2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:05:08.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.140+0000 7fba61aaa700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9c3dc6f20 0x55f9c3dd2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:08.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.140+0000 7fba622ab700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9c3dccf60 0x55f9c3dd05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:08.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.140+0000 7fba61aaa700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9c3dc6f20 0x55f9c3dd2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:41544/0 (socket says 172.21.15.37:41544) 2024-06-10T01:05:08.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.140+0000 7fba61aaa700 1 -- 172.21.15.37:0/753315348 learned_addr learned my addr 172.21.15.37:0/753315348 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:08.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.140+0000 7fba612a9700 1 --2- 172.21.15.37:0/753315348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9c3dcc640 0x55f9c3dd0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:08.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.140+0000 7fba622ab700 1 -- 172.21.15.37:0/753315348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9c3dc6f20 msgr2=0x55f9c3dd2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:08.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.140+0000 7fba622ab700 1 --2- 172.21.15.37:0/753315348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9c3dc6f20 0x55f9c3dd2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.140+0000 7fba622ab700 1 -- 172.21.15.37:0/753315348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9c3dcc640 msgr2=0x55f9c3dd0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:08.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.140+0000 7fba622ab700 1 --2- 172.21.15.37:0/753315348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9c3dcc640 0x55f9c3dd0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.140+0000 7fba622ab700 1 -- 172.21.15.37:0/753315348 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9c3c678d0 con 0x55f9c3dccf60 2024-06-10T01:05:08.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.140+0000 7fba61aaa700 1 --2- 172.21.15.37:0/753315348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9c3dc6f20 0x55f9c3dd2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:08.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.140+0000 7fba622ab700 1 --2- 172.21.15.37:0/753315348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9c3dccf60 0x55f9c3dd05a0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fba540090a0 tx=0x7fba5409b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afee1a1f32342b9c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:08.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.140+0000 7fba457fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:05:08.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.141+0000 7fba457fa700 1 -- 172.21.15.37:0/753315348 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5409ba90 con 0x55f9c3dccf60 2024-06-10T01:05:08.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.141+0000 7fba457fa700 1 -- 172.21.15.37:0/753315348 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fba54004d60 con 0x55f9c3dccf60 2024-06-10T01:05:08.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.141+0000 7fba6e591b80 1 -- 172.21.15.37:0/753315348 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba4c00bd60 con 0x55f9c3dccf60 2024-06-10T01:05:08.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.141+0000 7fba457fa700 1 -- 172.21.15.37:0/753315348 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fba5409b390 con 0x55f9c3dccf60 2024-06-10T01:05:08.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.141+0000 7fba6e591b80 10 client.16905 fetch_fsmap learned FSMap version 29 2024-06-10T01:05:08.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.141+0000 7fba6e591b80 1 -- 172.21.15.37:0/753315348 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f9c3dde340 con 0x55f9c3dccf60 2024-06-10T01:05:08.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.141+0000 7fba457fa700 1 -- 172.21.15.37:0/753315348 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba4c00bd60 con 0x55f9c3dccf60 2024-06-10T01:05:08.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.141+0000 7fba457fa700 1 -- 172.21.15.37:0/753315348 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fba5409a970 con 0x55f9c3dccf60 2024-06-10T01:05:08.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.141+0000 7fba6e591b80 10 client.16905 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:05:08.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.141+0000 7fba6e591b80 10 client.16905 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-06-10T01:05:08.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.141+0000 7fba6e591b80 20 client.16905 populate_metadata read hostname 'smithi037' 2024-06-10T01:05:08.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.141+0000 7fba6e591b80 1 --2- 172.21.15.37:0/753315348 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55f9c3de08d0 0x55f9c3de2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:08.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.141+0000 7fba6e591b80 4 client.16905 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:05:08.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.141+0000 7fba6e591b80 1 -- 172.21.15.37:0/753315348 --> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9c3d25d10 con 0x55f9c3de08d0 2024-06-10T01:05:08.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.142+0000 7fba61aaa700 1 --2- 172.21.15.37:0/753315348 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55f9c3de08d0 0x55f9c3de2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:08.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.142+0000 7fba457fa700 10 client.16905 ms_handle_connect on v2:172.21.15.94:6838/946269331 2024-06-10T01:05:08.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.142+0000 7fba61aaa700 1 --2- 172.21.15.37:0/753315348 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55f9c3de08d0 0x55f9c3de2cc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=47b7fb7ae0dfb112 server_cookie=70b283e66d1acff9 in_seq=0 out_seq=0 2024-06-10T01:05:08.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.143+0000 7fba457fa700 1 -- 172.21.15.37:0/753315348 <== mds.1 v2:172.21.15.94:6838/946269331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f9c3d25d10 con 0x55f9c3de08d0 2024-06-10T01:05:08.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.143+0000 7fba457fa700 10 client.16905 handle_command_reply: tid=0 2024-06-10T01:05:08.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.143+0000 7fba6e591b80 1 client.16905 shutdown 2024-06-10T01:05:08.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.143+0000 7fba6e591b80 1 -- 172.21.15.37:0/753315348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9c3dccf60 msgr2=0x55f9c3dd05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:08.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.143+0000 7fba6e591b80 1 --2- 172.21.15.37:0/753315348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9c3dccf60 0x55f9c3dd05a0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fba540090a0 tx=0x7fba5409b8d0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.143+0000 7fba6e591b80 1 -- 172.21.15.37:0/753315348 shutdown_connections 2024-06-10T01:05:08.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.143+0000 7fba6e591b80 1 --2- 172.21.15.37:0/753315348 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55f9c3de08d0 0x55f9c3de2cc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.143+0000 7fba6e591b80 1 --2- 172.21.15.37:0/753315348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9c3dccf60 0x55f9c3dd05a0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.143+0000 7fba6e591b80 1 --2- 172.21.15.37:0/753315348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9c3dc6f20 0x55f9c3dd2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.143+0000 7fba457fa700 0 client.16905 ms_handle_reset on v2:172.21.15.94:6838/946269331 2024-06-10T01:05:08.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.143+0000 7fba6e591b80 1 --2- 172.21.15.37:0/753315348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9c3dcc640 0x55f9c3dd0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.143+0000 7fba6e591b80 1 -- 172.21.15.37:0/753315348 >> 172.21.15.37:0/753315348 conn(0x55f9c3cf1cc0 msgr2=0x55f9c3dc59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:08.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.143+0000 7fba6e591b80 1 -- 172.21.15.37:0/753315348 shutdown_connections 2024-06-10T01:05:08.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.143+0000 7fba6e591b80 1 -- 172.21.15.37:0/753315348 wait complete. 2024-06-10T01:05:08.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.143+0000 7fba6e591b80 20 client.16905 trim_cache size 0 max 16384 2024-06-10T01:05:08.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.159+0000 7fba6e591b80 1 Processor -- start 2024-06-10T01:05:08.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.159+0000 7fba6e591b80 1 -- start start 2024-06-10T01:05:08.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.159+0000 7fba6e591b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9c3dc6f20 0x55f9c3dc69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:08.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.159+0000 7fba6e591b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9c3dcc640 0x55f9c3dca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:08.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.159+0000 7fba6e591b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9c3dccf60 0x55f9c3dbae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:08.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.159+0000 7fba6e591b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c1aea40 con 0x55f9c3dccf60 2024-06-10T01:05:08.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.159+0000 7fba6e591b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c1ac2d0 con 0x55f9c3dc6f20 2024-06-10T01:05:08.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.159+0000 7fba6e591b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c1aa140 con 0x55f9c3dcc640 2024-06-10T01:05:08.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.159+0000 7fba622ab700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9c3dccf60 0x55f9c3dbae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:08.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.159+0000 7fba622ab700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9c3dccf60 0x55f9c3dbae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49212/0 (socket says 172.21.15.37:49212) 2024-06-10T01:05:08.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.159+0000 7fba61aaa700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9c3dc6f20 0x55f9c3dc69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:08.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.159+0000 7fba612a9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9c3dcc640 0x55f9c3dca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:08.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.159+0000 7fba622ab700 1 -- 172.21.15.37:0/4219152325 learned_addr learned my addr 172.21.15.37:0/4219152325 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:08.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.160+0000 7fba61aaa700 1 -- 172.21.15.37:0/4219152325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9c3dcc640 msgr2=0x55f9c3dca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:08.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.160+0000 7fba61aaa700 1 --2- 172.21.15.37:0/4219152325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9c3dcc640 0x55f9c3dca7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.160+0000 7fba61aaa700 1 -- 172.21.15.37:0/4219152325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9c3dccf60 msgr2=0x55f9c3dbae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:08.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.160+0000 7fba61aaa700 1 --2- 172.21.15.37:0/4219152325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9c3dccf60 0x55f9c3dbae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.160+0000 7fba61aaa700 1 -- 172.21.15.37:0/4219152325 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9c3dce5f0 con 0x55f9c3dc6f20 2024-06-10T01:05:08.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.160+0000 7fba622ab700 1 --2- 172.21.15.37:0/4219152325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9c3dccf60 0x55f9c3dbae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:08.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.160+0000 7fba61aaa700 1 --2- 172.21.15.37:0/4219152325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9c3dc6f20 0x55f9c3dc69e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x55f9c3de32a0 tx=0x7fba58014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da7157b924aab8ae server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:08.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.160+0000 7fba477fe700 1 -- 172.21.15.37:0/4219152325 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba58022ba0 con 0x55f9c3dc6f20 2024-06-10T01:05:08.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.160+0000 7fba477fe700 1 -- 172.21.15.37:0/4219152325 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba58022d40 con 0x55f9c3dc6f20 2024-06-10T01:05:08.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.160+0000 7fba477fe700 1 -- 172.21.15.37:0/4219152325 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba580039e0 con 0x55f9c3dc6f20 2024-06-10T01:05:08.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.161+0000 7fba6e591b80 1 -- 172.21.15.37:0/4219152325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9c3dc6f20 msgr2=0x55f9c3dc69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:08.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.161+0000 7fba6e591b80 1 --2- 172.21.15.37:0/4219152325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9c3dc6f20 0x55f9c3dc69e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x55f9c3de32a0 tx=0x7fba58014000 comp rx=0 tx=0).stop 2024-06-10T01:05:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.161+0000 7fba6e591b80 1 -- 172.21.15.37:0/4219152325 shutdown_connections 2024-06-10T01:05:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.161+0000 7fba6e591b80 1 --2- 172.21.15.37:0/4219152325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9c3dccf60 0x55f9c3dbae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.161+0000 7fba6e591b80 1 --2- 172.21.15.37:0/4219152325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9c3dc6f20 0x55f9c3dc69e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.161+0000 7fba6e591b80 1 --2- 172.21.15.37:0/4219152325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9c3dcc640 0x55f9c3dca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.161+0000 7fba6e591b80 1 -- 172.21.15.37:0/4219152325 >> 172.21.15.37:0/4219152325 conn(0x55f9c3cf1cc0 msgr2=0x55f9c3dca0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.161+0000 7fba6e591b80 1 -- 172.21.15.37:0/4219152325 shutdown_connections 2024-06-10T01:05:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.161+0000 7fba6e591b80 1 -- 172.21.15.37:0/4219152325 wait complete. 2024-06-10T01:05:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.161+0000 7fba6e591b80 1 -- start start 2024-06-10T01:05:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.161+0000 7fba6e591b80 1 Processor -- start 2024-06-10T01:05:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.162+0000 7fba6e591b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9c3dc6f20 0x55f9c3dd90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.162+0000 7fba6e591b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9c3dcc640 0x55f9c3dd9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.162+0000 7fba6e591b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9c3dccf60 0x55f9c3de7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.162+0000 7fba6e591b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c1aa790 con 0x55f9c3dccf60 2024-06-10T01:05:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.162+0000 7fba6e591b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c1abc00 con 0x55f9c3dc6f20 2024-06-10T01:05:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.162+0000 7fba6e591b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c1aae60 con 0x55f9c3dcc640 2024-06-10T01:05:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.162+0000 7fba6e591b80 10 client.0 _ll_register_callbacks cb 0x55f9c3cb2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:05:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.162+0000 7fba622ab700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9c3dccf60 0x55f9c3de7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.162+0000 7fba622ab700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9c3dccf60 0x55f9c3de7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49214/0 (socket says 172.21.15.37:49214) 2024-06-10T01:05:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.162+0000 7fba612a9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9c3dcc640 0x55f9c3dd9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.162+0000 7fba622ab700 1 -- 172.21.15.37:0/4052527040 learned_addr learned my addr 172.21.15.37:0/4052527040 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.162+0000 7fba61aaa700 1 --2- 172.21.15.37:0/4052527040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9c3dc6f20 0x55f9c3dd90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.162+0000 7fba622ab700 1 -- 172.21.15.37:0/4052527040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9c3dcc640 msgr2=0x55f9c3dd9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.162+0000 7fba622ab700 1 --2- 172.21.15.37:0/4052527040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9c3dcc640 0x55f9c3dd9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.162+0000 7fba622ab700 1 -- 172.21.15.37:0/4052527040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9c3dc6f20 msgr2=0x55f9c3dd90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.162+0000 7fba622ab700 1 --2- 172.21.15.37:0/4052527040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9c3dc6f20 0x55f9c3dd90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.162+0000 7fba622ab700 1 -- 172.21.15.37:0/4052527040 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9c3de8530 con 0x55f9c3dccf60 2024-06-10T01:05:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.162+0000 7fba612a9700 1 --2- 172.21.15.37:0/4052527040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9c3dcc640 0x55f9c3dd9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.162+0000 7fba622ab700 1 --2- 172.21.15.37:0/4052527040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9c3dccf60 0x55f9c3de7d20 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fba540090a0 tx=0x7fba54097630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e81715b8ecb99 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:08.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.162+0000 7fba457fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:05:08.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.163+0000 7fba457fa700 1 -- 172.21.15.37:0/4052527040 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5409ba90 con 0x55f9c3dccf60 2024-06-10T01:05:08.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.163+0000 7fba457fa700 1 -- 172.21.15.37:0/4052527040 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fba54004d60 con 0x55f9c3dccf60 2024-06-10T01:05:08.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.163+0000 7fba6e591b80 1 -- 172.21.15.37:0/4052527040 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba58022d40 con 0x55f9c3dccf60 2024-06-10T01:05:08.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.163+0000 7fba457fa700 1 -- 172.21.15.37:0/4052527040 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fba540a3f70 con 0x55f9c3dccf60 2024-06-10T01:05:08.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.163+0000 7fba6e591b80 10 client.16911 fetch_fsmap learned FSMap version 29 2024-06-10T01:05:08.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.163+0000 7fba6e591b80 1 -- 172.21.15.37:0/4052527040 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f9c3e4b160 con 0x55f9c3dccf60 2024-06-10T01:05:08.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.163+0000 7fba457fa700 1 -- 172.21.15.37:0/4052527040 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba58022d40 con 0x55f9c3dccf60 2024-06-10T01:05:08.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.163+0000 7fba457fa700 1 -- 172.21.15.37:0/4052527040 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fba5409a870 con 0x55f9c3dccf60 2024-06-10T01:05:08.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.163+0000 7fba6e591b80 10 client.16911 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:05:08.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.163+0000 7fba6e591b80 10 client.16911 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-06-10T01:05:08.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.163+0000 7fba6e591b80 20 client.16911 populate_metadata read hostname 'smithi037' 2024-06-10T01:05:08.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.163+0000 7fba6e591b80 1 --2- 172.21.15.37:0/4052527040 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55f9c3e4b840 0x55f9c3e4dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:08.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.163+0000 7fba6e591b80 4 client.16911 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:05:08.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.163+0000 7fba6e591b80 1 -- 172.21.15.37:0/4052527040 --> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f9c3d20290 con 0x55f9c3e4b840 2024-06-10T01:05:08.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.164+0000 7fba61aaa700 1 --2- 172.21.15.37:0/4052527040 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55f9c3e4b840 0x55f9c3e4dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:08.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.164+0000 7fba61aaa700 1 --2- 172.21.15.37:0/4052527040 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55f9c3e4b840 0x55f9c3e4dc60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=419043b24658f14d server_cookie=f8a8f6d05727daf8 in_seq=0 out_seq=0 2024-06-10T01:05:08.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.164+0000 7fba457fa700 10 client.16911 ms_handle_connect on v2:172.21.15.94:6838/946269331 2024-06-10T01:05:08.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.164+0000 7fba457fa700 1 -- 172.21.15.37:0/4052527040 <== mds.1 v2:172.21.15.94:6838/946269331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f9c3d20290 con 0x55f9c3e4b840 2024-06-10T01:05:08.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.164+0000 7fba457fa700 10 client.16911 handle_command_reply: tid=0 2024-06-10T01:05:08.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.164+0000 7fba6e591b80 1 client.16911 shutdown 2024-06-10T01:05:08.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.164+0000 7fba6e591b80 1 -- 172.21.15.37:0/4052527040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9c3dccf60 msgr2=0x55f9c3de7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:08.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.164+0000 7fba6e591b80 1 --2- 172.21.15.37:0/4052527040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9c3dccf60 0x55f9c3de7d20 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fba540090a0 tx=0x7fba54097630 comp rx=0 tx=0).stop 2024-06-10T01:05:08.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.164+0000 7fba6e591b80 1 -- 172.21.15.37:0/4052527040 shutdown_connections 2024-06-10T01:05:08.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.164+0000 7fba6e591b80 1 --2- 172.21.15.37:0/4052527040 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55f9c3e4b840 0x55f9c3e4dc60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.164+0000 7fba6e591b80 1 --2- 172.21.15.37:0/4052527040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f9c3dccf60 0x55f9c3de7d20 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.164+0000 7fba6e591b80 1 --2- 172.21.15.37:0/4052527040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9c3dc6f20 0x55f9c3dd90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.164+0000 7fba6e591b80 1 --2- 172.21.15.37:0/4052527040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f9c3dcc640 0x55f9c3dd9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.165+0000 7fba457fa700 0 client.16911 ms_handle_reset on v2:172.21.15.94:6838/946269331 2024-06-10T01:05:08.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.165+0000 7fba6e591b80 1 -- 172.21.15.37:0/4052527040 >> 172.21.15.37:0/4052527040 conn(0x55f9c3cf1cc0 msgr2=0x55f9c3dbae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:08.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.165+0000 7fba6e591b80 1 -- 172.21.15.37:0/4052527040 shutdown_connections 2024-06-10T01:05:08.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.165+0000 7fba6e591b80 1 -- 172.21.15.37:0/4052527040 wait complete. 2024-06-10T01:05:08.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.165+0000 7fba6e591b80 20 client.16911 trim_cache size 0 max 16384 2024-06-10T01:05:08.167 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T01:05:08.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.165+0000 7fba63d0e700 1 -- 172.21.15.37:0/3566639791 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fba480a2ca0 msgr2=0x7fba480a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:08.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.165+0000 7fba63d0e700 1 --2- 172.21.15.37:0/3566639791 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fba480a2ca0 0x7fba480a5160 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fba5c199b80 tx=0x7fba58009260 comp rx=0 tx=0).stop 2024-06-10T01:05:08.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.165+0000 7fba63d0e700 1 -- 172.21.15.37:0/3566639791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba5c108500 msgr2=0x7fba5c198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:08.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.165+0000 7fba63d0e700 1 --2- 172.21.15.37:0/3566639791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba5c108500 0x7fba5c198a40 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fba5400ef30 tx=0x7fba5400c3a0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.165+0000 7fba63d0e700 1 -- 172.21.15.37:0/3566639791 shutdown_connections 2024-06-10T01:05:08.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.165+0000 7fba63d0e700 1 --2- 172.21.15.37:0/3566639791 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fba480a2ca0 0x7fba480a5160 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.165+0000 7fba63d0e700 1 --2- 172.21.15.37:0/3566639791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba5c108500 0x7fba5c198a40 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.165+0000 7fba63d0e700 1 --2- 172.21.15.37:0/3566639791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba5c06a170 0x7fba5c19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.165+0000 7fba63d0e700 1 --2- 172.21.15.37:0/3566639791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba5c06ab30 0x7fba5c19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:08.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.165+0000 7fba63d0e700 1 -- 172.21.15.37:0/3566639791 >> 172.21.15.37:0/3566639791 conn(0x7fba5c078340 msgr2=0x7fba5c0fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:08.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.165+0000 7fba63d0e700 1 -- 172.21.15.37:0/3566639791 shutdown_connections 2024-06-10T01:05:08.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:08.165+0000 7fba63d0e700 1 -- 172.21.15.37:0/3566639791 wait complete. 2024-06-10T01:05:08.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:08 smithi167 ceph-mon[36917]: pgmap v684: 129 pgs: 129 active+clean; 17 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 199 op/s 2024-06-10T01:05:08.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:08 smithi094 ceph-mon[36708]: pgmap v684: 129 pgs: 129 active+clean; 17 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 199 op/s 2024-06-10T01:05:08.914 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:05:09.193 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:05:10.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:10 smithi167 ceph-mon[36917]: pgmap v685: 129 pgs: 129 active+clean; 23 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 263 op/s 2024-06-10T01:05:10.336 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:10 smithi037 ceph-mon[32192]: pgmap v685: 129 pgs: 129 active+clean; 23 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 263 op/s 2024-06-10T01:05:10.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:10 smithi094 ceph-mon[36708]: pgmap v685: 129 pgs: 129 active+clean; 23 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 263 op/s 2024-06-10T01:05:11.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.101+0000 7fd48a390700 1 -- 172.21.15.37:0/2792178516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd48406b620 msgr2=0x7fd48410f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:11.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.101+0000 7fd48a390700 1 --2- 172.21.15.37:0/2792178516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd48406b620 0x7fd48410f350 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00b0d0 tx=0x7fd47c00b3e0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.101+0000 7fd48a390700 1 -- 172.21.15.37:0/2792178516 shutdown_connections 2024-06-10T01:05:11.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.101+0000 7fd48a390700 1 --2- 172.21.15.37:0/2792178516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd48406b620 0x7fd48410f350 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.101+0000 7fd48a390700 1 --2- 172.21.15.37:0/2792178516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd484108580 0x7fd484108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.101+0000 7fd48a390700 1 --2- 172.21.15.37:0/2792178516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd48406ac60 0x7fd48406b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.101+0000 7fd48a390700 1 -- 172.21.15.37:0/2792178516 >> 172.21.15.37:0/2792178516 conn(0x7fd484078430 msgr2=0x7fd484078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:11.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.101+0000 7fd48a390700 1 -- 172.21.15.37:0/2792178516 shutdown_connections 2024-06-10T01:05:11.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.101+0000 7fd48a390700 1 -- 172.21.15.37:0/2792178516 wait complete. 2024-06-10T01:05:11.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.101+0000 7fd48a390700 1 Processor -- start 2024-06-10T01:05:11.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.101+0000 7fd48a390700 1 -- start start 2024-06-10T01:05:11.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.102+0000 7fd48a390700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd48406ac60 0x7fd484101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:11.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.102+0000 7fd48a390700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd48406b620 0x7fd484101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:11.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.102+0000 7fd48a390700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd484108580 0x7fd484101e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:11.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.102+0000 7fd48a390700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd484112840 con 0x7fd48406ac60 2024-06-10T01:05:11.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.102+0000 7fd48a390700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd4841126c0 con 0x7fd48406b620 2024-06-10T01:05:11.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.102+0000 7fd48a390700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd4841129c0 con 0x7fd484108580 2024-06-10T01:05:11.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.102+0000 7fd483fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd48406ac60 0x7fd484101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:11.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.102+0000 7fd483fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd48406ac60 0x7fd484101400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49222/0 (socket says 172.21.15.37:49222) 2024-06-10T01:05:11.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.102+0000 7fd483fff700 1 -- 172.21.15.37:0/1925371989 learned_addr learned my addr 172.21.15.37:0/1925371989 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:11.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.102+0000 7fd4837fe700 1 --2- 172.21.15.37:0/1925371989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd48406b620 0x7fd484101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:11.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.102+0000 7fd48892d700 1 --2- 172.21.15.37:0/1925371989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd484108580 0x7fd484101e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:11.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.102+0000 7fd4837fe700 1 -- 172.21.15.37:0/1925371989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd484108580 msgr2=0x7fd484101e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:11.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.102+0000 7fd4837fe700 1 --2- 172.21.15.37:0/1925371989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd484108580 0x7fd484101e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.102+0000 7fd4837fe700 1 -- 172.21.15.37:0/1925371989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd48406ac60 msgr2=0x7fd484101400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:11.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.102+0000 7fd4837fe700 1 --2- 172.21.15.37:0/1925371989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd48406ac60 0x7fd484101400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.102+0000 7fd4837fe700 1 -- 172.21.15.37:0/1925371989 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4841a7240 con 0x7fd48406b620 2024-06-10T01:05:11.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.102+0000 7fd483fff700 1 --2- 172.21.15.37:0/1925371989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd48406ac60 0x7fd484101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:11.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.103+0000 7fd4837fe700 1 --2- 172.21.15.37:0/1925371989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd48406b620 0x7fd484101940 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd47800e9f0 tx=0x7fd47800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:11.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.103+0000 7fd4817fa700 1 -- 172.21.15.37:0/1925371989 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47800cbd0 con 0x7fd48406b620 2024-06-10T01:05:11.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.103+0000 7fd4817fa700 1 -- 172.21.15.37:0/1925371989 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd478004500 con 0x7fd48406b620 2024-06-10T01:05:11.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.103+0000 7fd48a390700 1 -- 172.21.15.37:0/1925371989 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4841a7530 con 0x7fd48406b620 2024-06-10T01:05:11.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.103+0000 7fd4817fa700 1 -- 172.21.15.37:0/1925371989 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478005280 con 0x7fd48406b620 2024-06-10T01:05:11.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.103+0000 7fd48a390700 1 -- 172.21.15.37:0/1925371989 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4841a7a20 con 0x7fd48406b620 2024-06-10T01:05:11.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.105+0000 7fd4817fa700 1 -- 172.21.15.37:0/1925371989 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd478005420 con 0x7fd48406b620 2024-06-10T01:05:11.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.106+0000 7fd4817fa700 1 --2- 172.21.15.37:0/1925371989 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd46c0a2c20 0x7fd46c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:11.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.106+0000 7fd483fff700 1 --2- 172.21.15.37:0/1925371989 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd46c0a2c20 0x7fd46c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:11.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.106+0000 7fd4817fa700 1 -- 172.21.15.37:0/1925371989 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fd4780cf3d0 con 0x7fd48406b620 2024-06-10T01:05:11.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.106+0000 7fd483fff700 1 --2- 172.21.15.37:0/1925371989 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd46c0a2c20 0x7fd46c0a50e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd4740104e0 tx=0x7fd47400b480 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:11.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.107+0000 7fd494c13b80 1 Processor -- start 2024-06-10T01:05:11.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.107+0000 7fd494c13b80 1 -- start start 2024-06-10T01:05:11.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.107+0000 7fd494c13b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56391b830640 0x56391b830a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:11.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.107+0000 7fd494c13b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56391b830f60 0x56391b82a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:11.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.107+0000 7fd494c13b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56391b82af20 0x56391b82b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:11.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.107+0000 7fd494c13b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56391b77c810 con 0x56391b830640 2024-06-10T01:05:11.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.107+0000 7fd494c13b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56391b724720 con 0x56391b830f60 2024-06-10T01:05:11.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.107+0000 7fd494c13b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56391b7787d0 con 0x56391b82af20 2024-06-10T01:05:11.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.107+0000 7fd483fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56391b830640 0x56391b830a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:11.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.107+0000 7fd483fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56391b830640 0x56391b830a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49226/0 (socket says 172.21.15.37:49226) 2024-06-10T01:05:11.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.107+0000 7fd483fff700 1 -- 172.21.15.37:0/1281216338 learned_addr learned my addr 172.21.15.37:0/1281216338 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:11.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.107+0000 7fd48892d700 1 --2- 172.21.15.37:0/1281216338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56391b82af20 0x56391b82b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:11.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.107+0000 7fd4837fe700 1 --2- 172.21.15.37:0/1281216338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56391b830f60 0x56391b82a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:11.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.107+0000 7fd483fff700 1 -- 172.21.15.37:0/1281216338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56391b82af20 msgr2=0x56391b82b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:11.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.108+0000 7fd483fff700 1 --2- 172.21.15.37:0/1281216338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56391b82af20 0x56391b82b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.108+0000 7fd483fff700 1 -- 172.21.15.37:0/1281216338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56391b830f60 msgr2=0x56391b82a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:11.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.108+0000 7fd483fff700 1 --2- 172.21.15.37:0/1281216338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56391b830f60 0x56391b82a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.108+0000 7fd483fff700 1 -- 172.21.15.37:0/1281216338 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56391b71af50 con 0x56391b830640 2024-06-10T01:05:11.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.108+0000 7fd483fff700 1 --2- 172.21.15.37:0/1281216338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56391b830640 0x56391b830a20 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fd4740116e0 tx=0x7fd474013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354130fc40ea186a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:11.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.108+0000 7fd46a7fc700 1 -- 172.21.15.37:0/1281216338 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47401e620 con 0x56391b830640 2024-06-10T01:05:11.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.108+0000 7fd46a7fc700 1 -- 172.21.15.37:0/1281216338 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd47401eca0 con 0x56391b830640 2024-06-10T01:05:11.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.108+0000 7fd46a7fc700 1 -- 172.21.15.37:0/1281216338 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47401cd20 con 0x56391b830640 2024-06-10T01:05:11.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.109+0000 7fd494c13b80 1 -- 172.21.15.37:0/1281216338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56391b830640 msgr2=0x56391b830a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:11.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.109+0000 7fd494c13b80 1 --2- 172.21.15.37:0/1281216338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56391b830640 0x56391b830a20 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fd4740116e0 tx=0x7fd474013000 comp rx=0 tx=0).stop 2024-06-10T01:05:11.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.109+0000 7fd494c13b80 1 -- 172.21.15.37:0/1281216338 shutdown_connections 2024-06-10T01:05:11.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.109+0000 7fd494c13b80 1 --2- 172.21.15.37:0/1281216338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56391b82af20 0x56391b82b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.109+0000 7fd494c13b80 1 --2- 172.21.15.37:0/1281216338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56391b830640 0x56391b830a20 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.109+0000 7fd494c13b80 1 --2- 172.21.15.37:0/1281216338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56391b830f60 0x56391b82a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.109+0000 7fd494c13b80 1 -- 172.21.15.37:0/1281216338 >> 172.21.15.37:0/1281216338 conn(0x56391b755cc0 msgr2=0x56391b825850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:11.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.109+0000 7fd494c13b80 1 -- 172.21.15.37:0/1281216338 shutdown_connections 2024-06-10T01:05:11.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.109+0000 7fd494c13b80 1 -- 172.21.15.37:0/1281216338 wait complete. 2024-06-10T01:05:11.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.109+0000 7fd494c13b80 1 -- start start 2024-06-10T01:05:11.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.110+0000 7fd494c13b80 1 Processor -- start 2024-06-10T01:05:11.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.110+0000 7fd494c13b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56391b82af20 0x56391b836d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:11.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.110+0000 7fd494c13b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56391b830640 0x56391b834030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:11.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.110+0000 7fd494c13b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56391b830f60 0x56391b834570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:11.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.110+0000 7fd494c13b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56391b8379d0 con 0x56391b82af20 2024-06-10T01:05:11.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.110+0000 7fd494c13b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56391b71cbf0 con 0x56391b830f60 2024-06-10T01:05:11.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.110+0000 7fd494c13b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56391b837b50 con 0x56391b830640 2024-06-10T01:05:11.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.110+0000 7fd494c13b80 10 client.0 _ll_register_callbacks cb 0x56391b716920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:05:11.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.110+0000 7fd483fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56391b82af20 0x56391b836d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:11.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.110+0000 7fd4837fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56391b830640 0x56391b834030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:11.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.110+0000 7fd483fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56391b82af20 0x56391b836d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49240/0 (socket says 172.21.15.37:49240) 2024-06-10T01:05:11.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.110+0000 7fd483fff700 1 -- 172.21.15.37:0/2261646905 learned_addr learned my addr 172.21.15.37:0/2261646905 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:11.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.110+0000 7fd48892d700 1 --2- 172.21.15.37:0/2261646905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56391b830f60 0x56391b834570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:11.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.110+0000 7fd483fff700 1 -- 172.21.15.37:0/2261646905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56391b830640 msgr2=0x56391b834030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:11.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.110+0000 7fd483fff700 1 --2- 172.21.15.37:0/2261646905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56391b830640 0x56391b834030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.110+0000 7fd483fff700 1 -- 172.21.15.37:0/2261646905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56391b830f60 msgr2=0x56391b834570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:11.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.110+0000 7fd483fff700 1 --2- 172.21.15.37:0/2261646905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56391b830f60 0x56391b834570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.110+0000 7fd483fff700 1 -- 172.21.15.37:0/2261646905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56391b6cb8d0 con 0x56391b82af20 2024-06-10T01:05:11.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.110+0000 7fd48892d700 1 --2- 172.21.15.37:0/2261646905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56391b830f60 0x56391b834570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:11.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.110+0000 7fd483fff700 1 --2- 172.21.15.37:0/2261646905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56391b82af20 0x56391b836d30 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fd4740116b0 tx=0x7fd47401c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a793fbaaaa91b7de server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:11.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.111+0000 7fd4637fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:05:11.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.111+0000 7fd4637fe700 1 -- 172.21.15.37:0/2261646905 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd474036060 con 0x56391b82af20 2024-06-10T01:05:11.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.111+0000 7fd4637fe700 1 -- 172.21.15.37:0/2261646905 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fd47401c9c0 con 0x56391b82af20 2024-06-10T01:05:11.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.111+0000 7fd494c13b80 1 -- 172.21.15.37:0/2261646905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd47401eca0 con 0x56391b82af20 2024-06-10T01:05:11.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.111+0000 7fd4637fe700 1 -- 172.21.15.37:0/2261646905 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd47403d050 con 0x56391b82af20 2024-06-10T01:05:11.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.111+0000 7fd494c13b80 10 client.16923 fetch_fsmap learned FSMap version 29 2024-06-10T01:05:11.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.111+0000 7fd494c13b80 1 -- 172.21.15.37:0/2261646905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56391b842330 con 0x56391b82af20 2024-06-10T01:05:11.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.111+0000 7fd4637fe700 1 -- 172.21.15.37:0/2261646905 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd47401eca0 con 0x56391b82af20 2024-06-10T01:05:11.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.111+0000 7fd4637fe700 1 -- 172.21.15.37:0/2261646905 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fd47401e5b0 con 0x56391b82af20 2024-06-10T01:05:11.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.111+0000 7fd494c13b80 10 client.16923 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:05:11.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.111+0000 7fd494c13b80 10 client.16923 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:05:11.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.111+0000 7fd494c13b80 20 client.16923 populate_metadata read hostname 'smithi037' 2024-06-10T01:05:11.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.111+0000 7fd494c13b80 1 --2- 172.21.15.37:0/2261646905 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56391b844910 0x56391b846d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:11.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.111+0000 7fd494c13b80 4 client.16923 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:05:11.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.111+0000 7fd494c13b80 1 -- 172.21.15.37:0/2261646905 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56391b789d10 con 0x56391b844910 2024-06-10T01:05:11.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.112+0000 7fd4837fe700 1 --2- 172.21.15.37:0/2261646905 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56391b844910 0x56391b846d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:11.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.112+0000 7fd4837fe700 1 --2- 172.21.15.37:0/2261646905 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56391b844910 0x56391b846d00 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3da3cdd623af0608 server_cookie=5b431b93b88418fe in_seq=0 out_seq=0 2024-06-10T01:05:11.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.112+0000 7fd4637fe700 10 client.16923 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:05:11.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.114+0000 7fd4637fe700 1 -- 172.21.15.37:0/2261646905 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56391b789d10 con 0x56391b844910 2024-06-10T01:05:11.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.114+0000 7fd4637fe700 10 client.16923 handle_command_reply: tid=0 2024-06-10T01:05:11.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.114+0000 7fd494c13b80 1 client.16923 shutdown 2024-06-10T01:05:11.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.114+0000 7fd494c13b80 1 -- 172.21.15.37:0/2261646905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56391b82af20 msgr2=0x56391b836d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:11.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.114+0000 7fd494c13b80 1 --2- 172.21.15.37:0/2261646905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56391b82af20 0x56391b836d30 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fd4740116b0 tx=0x7fd47401c3b0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.114+0000 7fd494c13b80 1 -- 172.21.15.37:0/2261646905 shutdown_connections 2024-06-10T01:05:11.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.114+0000 7fd494c13b80 1 --2- 172.21.15.37:0/2261646905 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56391b844910 0x56391b846d00 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.114+0000 7fd494c13b80 1 --2- 172.21.15.37:0/2261646905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56391b830f60 0x56391b834570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.114+0000 7fd494c13b80 1 --2- 172.21.15.37:0/2261646905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56391b82af20 0x56391b836d30 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.114+0000 7fd494c13b80 1 --2- 172.21.15.37:0/2261646905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56391b830640 0x56391b834030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.114+0000 7fd494c13b80 1 -- 172.21.15.37:0/2261646905 >> 172.21.15.37:0/2261646905 conn(0x56391b755cc0 msgr2=0x56391b8255f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:11.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.114+0000 7fd4637fe700 0 client.16923 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:05:11.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.114+0000 7fd494c13b80 1 -- 172.21.15.37:0/2261646905 shutdown_connections 2024-06-10T01:05:11.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.114+0000 7fd494c13b80 1 -- 172.21.15.37:0/2261646905 wait complete. 2024-06-10T01:05:11.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.114+0000 7fd494c13b80 20 client.16923 trim_cache size 0 max 16384 2024-06-10T01:05:11.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.131+0000 7fd494c13b80 1 Processor -- start 2024-06-10T01:05:11.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.131+0000 7fd494c13b80 1 -- start start 2024-06-10T01:05:11.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.131+0000 7fd494c13b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56391b82af20 0x56391b82a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:11.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.131+0000 7fd494c13b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56391b830640 0x56391b831bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:11.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.131+0000 7fd494c13b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56391b830f60 0x56391b81f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:11.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.131+0000 7fd494c13b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd4841b3030 con 0x56391b82af20 2024-06-10T01:05:11.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.131+0000 7fd494c13b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd4841b08c0 con 0x56391b830f60 2024-06-10T01:05:11.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.131+0000 7fd494c13b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd4841ae710 con 0x56391b830640 2024-06-10T01:05:11.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.131+0000 7fd483fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56391b82af20 0x56391b82a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:11.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.131+0000 7fd483fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56391b82af20 0x56391b82a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49248/0 (socket says 172.21.15.37:49248) 2024-06-10T01:05:11.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.132+0000 7fd483fff700 1 -- 172.21.15.37:0/3689791626 learned_addr learned my addr 172.21.15.37:0/3689791626 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:11.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.132+0000 7fd4837fe700 1 --2- 172.21.15.37:0/3689791626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56391b830640 0x56391b831bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:11.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.132+0000 7fd48892d700 1 --2- 172.21.15.37:0/3689791626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56391b830f60 0x56391b81f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:11.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.132+0000 7fd4837fe700 1 -- 172.21.15.37:0/3689791626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56391b830f60 msgr2=0x56391b81f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:11.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.132+0000 7fd4837fe700 1 --2- 172.21.15.37:0/3689791626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56391b830f60 0x56391b81f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.132+0000 7fd4837fe700 1 -- 172.21.15.37:0/3689791626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56391b82af20 msgr2=0x56391b82a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:11.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.132+0000 7fd4837fe700 1 --2- 172.21.15.37:0/3689791626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56391b82af20 0x56391b82a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.132+0000 7fd4837fe700 1 -- 172.21.15.37:0/3689791626 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56391b836970 con 0x56391b830640 2024-06-10T01:05:11.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.132+0000 7fd483fff700 1 --2- 172.21.15.37:0/3689791626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56391b82af20 0x56391b82a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:11.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.132+0000 7fd48892d700 1 --2- 172.21.15.37:0/3689791626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56391b830f60 0x56391b81f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:11.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.132+0000 7fd4837fe700 1 --2- 172.21.15.37:0/3689791626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56391b830640 0x56391b831bd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd47800e9c0 tx=0x7fd4780cc9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dda78d1b4fa7504d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:11.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.132+0000 7fd469ffb700 1 -- 172.21.15.37:0/3689791626 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4780b9070 con 0x56391b830640 2024-06-10T01:05:11.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.132+0000 7fd469ffb700 1 -- 172.21.15.37:0/3689791626 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd478005710 con 0x56391b830640 2024-06-10T01:05:11.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.132+0000 7fd469ffb700 1 -- 172.21.15.37:0/3689791626 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47801c040 con 0x56391b830640 2024-06-10T01:05:11.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.133+0000 7fd494c13b80 1 -- 172.21.15.37:0/3689791626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56391b830640 msgr2=0x56391b831bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:11.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.133+0000 7fd494c13b80 1 --2- 172.21.15.37:0/3689791626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56391b830640 0x56391b831bd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd47800e9c0 tx=0x7fd4780cc9b0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.133+0000 7fd494c13b80 1 -- 172.21.15.37:0/3689791626 shutdown_connections 2024-06-10T01:05:11.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.133+0000 7fd494c13b80 1 --2- 172.21.15.37:0/3689791626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56391b830f60 0x56391b81f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.133+0000 7fd494c13b80 1 --2- 172.21.15.37:0/3689791626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56391b82af20 0x56391b82a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.133+0000 7fd494c13b80 1 --2- 172.21.15.37:0/3689791626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56391b830640 0x56391b831bd0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.133+0000 7fd494c13b80 1 -- 172.21.15.37:0/3689791626 >> 172.21.15.37:0/3689791626 conn(0x56391b755cc0 msgr2=0x56391b81ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:11.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.133+0000 7fd494c13b80 1 -- 172.21.15.37:0/3689791626 shutdown_connections 2024-06-10T01:05:11.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.133+0000 7fd494c13b80 1 -- 172.21.15.37:0/3689791626 wait complete. 2024-06-10T01:05:11.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.133+0000 7fd494c13b80 1 -- start start 2024-06-10T01:05:11.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.134+0000 7fd494c13b80 1 Processor -- start 2024-06-10T01:05:11.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.134+0000 7fd494c13b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56391b82af20 0x56391b8474c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:11.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.134+0000 7fd494c13b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56391b830640 0x56391b847a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:11.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.134+0000 7fd494c13b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56391b830f60 0x56391b8ad240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:11.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.134+0000 7fd494c13b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd4841aed80 con 0x56391b82af20 2024-06-10T01:05:11.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.134+0000 7fd494c13b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd4841b01f0 con 0x56391b830640 2024-06-10T01:05:11.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.134+0000 7fd494c13b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd4841af450 con 0x56391b830f60 2024-06-10T01:05:11.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.134+0000 7fd494c13b80 10 client.0 _ll_register_callbacks cb 0x56391b716920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:05:11.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.134+0000 7fd483fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56391b82af20 0x56391b8474c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:11.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.134+0000 7fd483fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56391b82af20 0x56391b8474c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49264/0 (socket says 172.21.15.37:49264) 2024-06-10T01:05:11.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.134+0000 7fd483fff700 1 -- 172.21.15.37:0/2902158721 learned_addr learned my addr 172.21.15.37:0/2902158721 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:11.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.134+0000 7fd4837fe700 1 --2- 172.21.15.37:0/2902158721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56391b830640 0x56391b847a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:11.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.134+0000 7fd48892d700 1 --2- 172.21.15.37:0/2902158721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56391b830f60 0x56391b8ad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:11.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.134+0000 7fd483fff700 1 -- 172.21.15.37:0/2902158721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56391b830f60 msgr2=0x56391b8ad240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:11.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.134+0000 7fd483fff700 1 --2- 172.21.15.37:0/2902158721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56391b830f60 0x56391b8ad240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.134+0000 7fd483fff700 1 -- 172.21.15.37:0/2902158721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56391b830640 msgr2=0x56391b847a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:11.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.134+0000 7fd483fff700 1 --2- 172.21.15.37:0/2902158721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56391b830640 0x56391b847a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.134+0000 7fd483fff700 1 -- 172.21.15.37:0/2902158721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56391b8ad850 con 0x56391b82af20 2024-06-10T01:05:11.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.134+0000 7fd4837fe700 1 --2- 172.21.15.37:0/2902158721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56391b830640 0x56391b847a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:11.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.134+0000 7fd483fff700 1 --2- 172.21.15.37:0/2902158721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56391b82af20 0x56391b8474c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fd47401eb70 tx=0x7fd474005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe69360de116a125 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:11.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.135+0000 7fd4637fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:05:11.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.135+0000 7fd4637fe700 1 -- 172.21.15.37:0/2902158721 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd474036040 con 0x56391b82af20 2024-06-10T01:05:11.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.135+0000 7fd4637fe700 1 -- 172.21.15.37:0/2902158721 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fd47401e420 con 0x56391b82af20 2024-06-10T01:05:11.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.135+0000 7fd494c13b80 1 -- 172.21.15.37:0/2902158721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd478005710 con 0x56391b82af20 2024-06-10T01:05:11.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.135+0000 7fd4637fe700 1 -- 172.21.15.37:0/2902158721 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd47404a050 con 0x56391b82af20 2024-06-10T01:05:11.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.135+0000 7fd494c13b80 10 client.16929 fetch_fsmap learned FSMap version 29 2024-06-10T01:05:11.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.135+0000 7fd494c13b80 1 -- 172.21.15.37:0/2902158721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56391b8aeb50 con 0x56391b82af20 2024-06-10T01:05:11.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.135+0000 7fd4637fe700 1 -- 172.21.15.37:0/2902158721 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd478005710 con 0x56391b82af20 2024-06-10T01:05:11.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.135+0000 7fd4637fe700 1 -- 172.21.15.37:0/2902158721 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fd474031570 con 0x56391b82af20 2024-06-10T01:05:11.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.135+0000 7fd494c13b80 10 client.16929 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:05:11.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.135+0000 7fd494c13b80 10 client.16929 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:05:11.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.135+0000 7fd494c13b80 20 client.16929 populate_metadata read hostname 'smithi037' 2024-06-10T01:05:11.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.135+0000 7fd494c13b80 1 --2- 172.21.15.37:0/2902158721 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56391b8af1d0 0x56391b8b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:11.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.135+0000 7fd494c13b80 4 client.16929 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:05:11.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.135+0000 7fd494c13b80 1 -- 172.21.15.37:0/2902158721 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56391b784290 con 0x56391b8af1d0 2024-06-10T01:05:11.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.136+0000 7fd4837fe700 1 --2- 172.21.15.37:0/2902158721 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56391b8af1d0 0x56391b8b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:11.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.136+0000 7fd4637fe700 10 client.16929 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:05:11.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.136+0000 7fd4837fe700 1 --2- 172.21.15.37:0/2902158721 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56391b8af1d0 0x56391b8b15f0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbfdca1bb5cbcfd server_cookie=af9a73ffcc044951 in_seq=0 out_seq=0 2024-06-10T01:05:11.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.149+0000 7fd4637fe700 1 -- 172.21.15.37:0/2902158721 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56391b784290 con 0x56391b8af1d0 2024-06-10T01:05:11.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.149+0000 7fd4637fe700 10 client.16929 handle_command_reply: tid=0 2024-06-10T01:05:11.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.149+0000 7fd494c13b80 1 client.16929 shutdown 2024-06-10T01:05:11.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.150+0000 7fd494c13b80 1 -- 172.21.15.37:0/2902158721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56391b82af20 msgr2=0x56391b8474c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:11.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.150+0000 7fd494c13b80 1 --2- 172.21.15.37:0/2902158721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56391b82af20 0x56391b8474c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fd47401eb70 tx=0x7fd474005ee0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.150+0000 7fd494c13b80 1 -- 172.21.15.37:0/2902158721 shutdown_connections 2024-06-10T01:05:11.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.150+0000 7fd494c13b80 1 --2- 172.21.15.37:0/2902158721 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56391b8af1d0 0x56391b8b15f0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.150+0000 7fd494c13b80 1 --2- 172.21.15.37:0/2902158721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56391b830f60 0x56391b8ad240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.150+0000 7fd494c13b80 1 --2- 172.21.15.37:0/2902158721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56391b82af20 0x56391b8474c0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.150+0000 7fd4637fe700 0 client.16929 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:05:11.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.150+0000 7fd494c13b80 1 --2- 172.21.15.37:0/2902158721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56391b830640 0x56391b847a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.150+0000 7fd494c13b80 1 -- 172.21.15.37:0/2902158721 >> 172.21.15.37:0/2902158721 conn(0x56391b755cc0 msgr2=0x56391b8327f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:11.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.150+0000 7fd494c13b80 1 -- 172.21.15.37:0/2902158721 shutdown_connections 2024-06-10T01:05:11.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.150+0000 7fd494c13b80 1 -- 172.21.15.37:0/2902158721 wait complete. 2024-06-10T01:05:11.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.150+0000 7fd494c13b80 20 client.16929 trim_cache size 0 max 16384 2024-06-10T01:05:11.148 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T01:05:11.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.151+0000 7fd48a390700 1 -- 172.21.15.37:0/1925371989 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd46c0a2c20 msgr2=0x7fd46c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:11.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.151+0000 7fd48a390700 1 --2- 172.21.15.37:0/1925371989 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd46c0a2c20 0x7fd46c0a50e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd4740104e0 tx=0x7fd47400b480 comp rx=0 tx=0).stop 2024-06-10T01:05:11.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.151+0000 7fd48a390700 1 -- 172.21.15.37:0/1925371989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd48406b620 msgr2=0x7fd484101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:11.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.151+0000 7fd48a390700 1 --2- 172.21.15.37:0/1925371989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd48406b620 0x7fd484101940 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd47800e9f0 tx=0x7fd47800ed00 comp rx=0 tx=0).stop 2024-06-10T01:05:11.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.151+0000 7fd48a390700 1 -- 172.21.15.37:0/1925371989 shutdown_connections 2024-06-10T01:05:11.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.151+0000 7fd48a390700 1 --2- 172.21.15.37:0/1925371989 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd46c0a2c20 0x7fd46c0a50e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.151+0000 7fd48a390700 1 --2- 172.21.15.37:0/1925371989 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd484108580 0x7fd484101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.151+0000 7fd48a390700 1 --2- 172.21.15.37:0/1925371989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd48406ac60 0x7fd484101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.151+0000 7fd48a390700 1 --2- 172.21.15.37:0/1925371989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd48406b620 0x7fd484101940 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:11.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.151+0000 7fd48a390700 1 -- 172.21.15.37:0/1925371989 >> 172.21.15.37:0/1925371989 conn(0x7fd484078430 msgr2=0x7fd4840ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:11.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.151+0000 7fd48a390700 1 -- 172.21.15.37:0/1925371989 shutdown_connections 2024-06-10T01:05:11.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:11.151+0000 7fd48a390700 1 -- 172.21.15.37:0/1925371989 wait complete. 2024-06-10T01:05:12.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:12 smithi037 ceph-mon[32192]: pgmap v686: 129 pgs: 129 active+clean; 23 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-06-10T01:05:12.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:12 smithi167 ceph-mon[36917]: pgmap v686: 129 pgs: 129 active+clean; 23 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-06-10T01:05:12.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:12 smithi094 ceph-mon[36708]: pgmap v686: 129 pgs: 129 active+clean; 23 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-06-10T01:05:12.866 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:05:12.867 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:05:13.127 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:05:14.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:14 smithi037 ceph-mon[32192]: pgmap v687: 129 pgs: 129 active+clean; 27 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 265 op/s 2024-06-10T01:05:14.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:14 smithi167 ceph-mon[36917]: pgmap v687: 129 pgs: 129 active+clean; 27 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 265 op/s 2024-06-10T01:05:14.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:14 smithi094 ceph-mon[36708]: pgmap v687: 129 pgs: 129 active+clean; 27 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 265 op/s 2024-06-10T01:05:15.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.153+0000 7ff7dc6cb700 1 -- 172.21.15.37:0/3340937486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7d4110bf0 msgr2=0x7ff7d4109490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:15.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.153+0000 7ff7dc6cb700 1 --2- 172.21.15.37:0/3340937486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7d4110bf0 0x7ff7d4109490 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4009990 tx=0x7ff7c4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:05:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.159+0000 7ff7dc6cb700 1 -- 172.21.15.37:0/3340937486 shutdown_connections 2024-06-10T01:05:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.159+0000 7ff7dc6cb700 1 --2- 172.21.15.37:0/3340937486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7d411bc70 0x7ff7d411e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.159+0000 7ff7dc6cb700 1 --2- 172.21.15.37:0/3340937486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7d4110bf0 0x7ff7d4109490 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.159+0000 7ff7dc6cb700 1 --2- 172.21.15.37:0/3340937486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7d4111060 0x7ff7d41099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.159+0000 7ff7dc6cb700 1 -- 172.21.15.37:0/3340937486 >> 172.21.15.37:0/3340937486 conn(0x7ff7d406d600 msgr2=0x7ff7d406da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.164+0000 7ff7dc6cb700 1 -- 172.21.15.37:0/3340937486 shutdown_connections 2024-06-10T01:05:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.165+0000 7ff7dc6cb700 1 -- 172.21.15.37:0/3340937486 wait complete. 2024-06-10T01:05:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.166+0000 7ff7dc6cb700 1 Processor -- start 2024-06-10T01:05:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.166+0000 7ff7dc6cb700 1 -- start start 2024-06-10T01:05:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.167+0000 7ff7dc6cb700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7d4110bf0 0x7ff7d411a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.167+0000 7ff7dc6cb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7d411bc70 0x7ff7d411a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.168+0000 7ff7dc6cb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7d4113350 0x7ff7d41137d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:15.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.168+0000 7ff7dc6cb700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d41216e0 con 0x7ff7d4113350 2024-06-10T01:05:15.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.168+0000 7ff7dc6cb700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d4121560 con 0x7ff7d411bc70 2024-06-10T01:05:15.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.168+0000 7ff7dc6cb700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d4121860 con 0x7ff7d4110bf0 2024-06-10T01:05:15.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.169+0000 7ff7dac68700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7d4113350 0x7ff7d41137d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:15.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.169+0000 7ff7dac68700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7d4113350 0x7ff7d41137d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58258/0 (socket says 172.21.15.37:58258) 2024-06-10T01:05:15.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.169+0000 7ff7dac68700 1 -- 172.21.15.37:0/447566694 learned_addr learned my addr 172.21.15.37:0/447566694 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:15.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.171+0000 7ff7d9c66700 1 --2- 172.21.15.37:0/447566694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7d411bc70 0x7ff7d411a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:15.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.172+0000 7ff7dac68700 1 -- 172.21.15.37:0/447566694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7d4110bf0 msgr2=0x7ff7d411a160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-10T01:05:15.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.172+0000 7ff7dac68700 1 --2- 172.21.15.37:0/447566694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7d4110bf0 0x7ff7d411a160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:15.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.172+0000 7ff7dac68700 1 -- 172.21.15.37:0/447566694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7d411bc70 msgr2=0x7ff7d411a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:15.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.172+0000 7ff7dac68700 1 --2- 172.21.15.37:0/447566694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7d411bc70 0x7ff7d411a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:15.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.172+0000 7ff7dac68700 1 -- 172.21.15.37:0/447566694 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d4117c90 con 0x7ff7d4113350 2024-06-10T01:05:15.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.177+0000 7ff7dac68700 1 --2- 172.21.15.37:0/447566694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7d4113350 0x7ff7d41137d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000b4a0 tx=0x7ff7d000b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:15.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.177+0000 7ff7cb7fe700 1 -- 172.21.15.37:0/447566694 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0015020 con 0x7ff7d4113350 2024-06-10T01:05:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.178+0000 7ff7dc6cb700 1 -- 172.21.15.37:0/447566694 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7d4117f80 con 0x7ff7d4113350 2024-06-10T01:05:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.178+0000 7ff7cb7fe700 1 -- 172.21.15.37:0/447566694 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7d0011d10 con 0x7ff7d4113350 2024-06-10T01:05:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.178+0000 7ff7dc6cb700 1 -- 172.21.15.37:0/447566694 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7d41184c0 con 0x7ff7d4113350 2024-06-10T01:05:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.178+0000 7ff7cb7fe700 1 -- 172.21.15.37:0/447566694 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0010db0 con 0x7ff7d4113350 2024-06-10T01:05:15.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.189+0000 7ff7e6f4eb80 1 Processor -- start 2024-06-10T01:05:15.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.190+0000 7ff7e6f4eb80 1 -- start start 2024-06-10T01:05:15.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.190+0000 7ff7e6f4eb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ce33d1fe0 0x562ce33d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:15.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.190+0000 7ff7e6f4eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ce33d2900 0x562ce33cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:15.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.190+0000 7ff7e6f4eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ce33cc820 0x562ce33ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:15.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.190+0000 7ff7e6f4eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562ce32befe0 con 0x562ce33d2900 2024-06-10T01:05:15.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.190+0000 7ff7e6f4eb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562ce331d8d0 con 0x562ce33d1fe0 2024-06-10T01:05:15.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.190+0000 7ff7e6f4eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562ce32854e0 con 0x562ce33cc820 2024-06-10T01:05:15.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.190+0000 7ff7cb7fe700 1 -- 172.21.15.37:0/447566694 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff7d0018020 con 0x7ff7d4113350 2024-06-10T01:05:15.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.191+0000 7ff7cb7fe700 1 --2- 172.21.15.37:0/447566694 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff7c00a2cf0 0x7ff7c00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:15.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.192+0000 7ff7dac68700 1 --2- 172.21.15.37:0/447566694 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff7c00a2cf0 0x7ff7c00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:15.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.192+0000 7ff7cb7fe700 1 -- 172.21.15.37:0/447566694 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7ff7d00cebf0 con 0x7ff7d4113350 2024-06-10T01:05:15.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.196+0000 7ff7d9c66700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ce33d1fe0 0x562ce33d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:15.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.196+0000 7ff7d9c66700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ce33d1fe0 0x562ce33d23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:54420/0 (socket says 172.21.15.37:54420) 2024-06-10T01:05:15.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.196+0000 7ff7d9c66700 1 -- 172.21.15.37:0/3723726553 learned_addr learned my addr 172.21.15.37:0/3723726553 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:15.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.196+0000 7ff7d9c66700 1 --2- 172.21.15.37:0/3723726553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ce33cc820 0x562ce33ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:15.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.198+0000 7ff7da467700 1 --2- 172.21.15.37:0/3723726553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ce33d2900 0x562ce33cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:15.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.202+0000 7ff7d9c66700 1 -- 172.21.15.37:0/3723726553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ce33cc820 msgr2=0x562ce33ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:15.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.202+0000 7ff7d9c66700 1 --2- 172.21.15.37:0/3723726553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ce33cc820 0x562ce33ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:15.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.202+0000 7ff7d9c66700 1 -- 172.21.15.37:0/3723726553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ce33d2900 msgr2=0x562ce33cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:15.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.202+0000 7ff7d9c66700 1 --2- 172.21.15.37:0/3723726553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ce33d2900 0x562ce33cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:15.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.202+0000 7ff7d9c66700 1 -- 172.21.15.37:0/3723726553 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ce326c950 con 0x562ce33d1fe0 2024-06-10T01:05:15.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.206+0000 7ff7d9c66700 1 --2- 172.21.15.37:0/3723726553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ce33d1fe0 0x562ce33d23c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc009690 tx=0x7ff7cc009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5235143becb3e844 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:15.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.207+0000 7ff7dac68700 1 --2- 172.21.15.37:0/447566694 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff7c00a2cf0 0x7ff7c00a51b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0096ce0 tx=0x7ff7d0096b70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:15.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.208+0000 7ff7c8ff9700 1 -- 172.21.15.37:0/3723726553 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc007700 con 0x562ce33d1fe0 2024-06-10T01:05:15.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.208+0000 7ff7c8ff9700 1 -- 172.21.15.37:0/3723726553 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7cc007d80 con 0x562ce33d1fe0 2024-06-10T01:05:15.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.208+0000 7ff7c8ff9700 1 -- 172.21.15.37:0/3723726553 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc017ac0 con 0x562ce33d1fe0 2024-06-10T01:05:15.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.211+0000 7ff7e6f4eb80 1 -- 172.21.15.37:0/3723726553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ce33d1fe0 msgr2=0x562ce33d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:15.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.211+0000 7ff7e6f4eb80 1 --2- 172.21.15.37:0/3723726553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ce33d1fe0 0x562ce33d23c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc009690 tx=0x7ff7cc009620 comp rx=0 tx=0).stop 2024-06-10T01:05:15.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.212+0000 7ff7e6f4eb80 1 -- 172.21.15.37:0/3723726553 shutdown_connections 2024-06-10T01:05:15.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.212+0000 7ff7e6f4eb80 1 --2- 172.21.15.37:0/3723726553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ce33cc820 0x562ce33ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:15.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.212+0000 7ff7e6f4eb80 1 --2- 172.21.15.37:0/3723726553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ce33d1fe0 0x562ce33d23c0 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc009690 tx=0x7ff7cc009620 comp rx=0 tx=0).stop 2024-06-10T01:05:15.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.212+0000 7ff7e6f4eb80 1 --2- 172.21.15.37:0/3723726553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ce33d2900 0x562ce33cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:15.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.212+0000 7ff7e6f4eb80 1 -- 172.21.15.37:0/3723726553 >> 172.21.15.37:0/3723726553 conn(0x562ce32f9100 msgr2=0x562ce33c72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:15.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.214+0000 7ff7e6f4eb80 1 -- 172.21.15.37:0/3723726553 shutdown_connections 2024-06-10T01:05:15.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.215+0000 7ff7e6f4eb80 1 -- 172.21.15.37:0/3723726553 wait complete. 2024-06-10T01:05:15.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.215+0000 7ff7e6f4eb80 1 -- start start 2024-06-10T01:05:15.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.215+0000 7ff7e6f4eb80 1 Processor -- start 2024-06-10T01:05:15.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.221+0000 7ff7e6f4eb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ce33cc820 0x562ce33d8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:15.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.221+0000 7ff7e6f4eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ce33d2900 0x562ce33d5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:15.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.221+0000 7ff7e6f4eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ce33d6170 0x562ce33d65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:15.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.221+0000 7ff7e6f4eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562ce32be040 con 0x562ce33d2900 2024-06-10T01:05:15.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.221+0000 7ff7e6f4eb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562ce3319890 con 0x562ce33cc820 2024-06-10T01:05:15.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.221+0000 7ff7e6f4eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562ce33d92e0 con 0x562ce33d6170 2024-06-10T01:05:15.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.221+0000 7ff7e6f4eb80 10 client.0 _ll_register_callbacks cb 0x562ce32b3630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:05:15.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.222+0000 7ff7da467700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ce33d2900 0x562ce33d5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:15.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.222+0000 7ff7da467700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ce33d2900 0x562ce33d5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58270/0 (socket says 172.21.15.37:58270) 2024-06-10T01:05:15.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.222+0000 7ff7da467700 1 -- 172.21.15.37:0/2560825198 learned_addr learned my addr 172.21.15.37:0/2560825198 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:15.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.222+0000 7ff7da467700 1 -- 172.21.15.37:0/2560825198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ce33d6170 msgr2=0x562ce33d65d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:05:15.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.222+0000 7ff7da467700 1 --2- 172.21.15.37:0/2560825198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ce33d6170 0x562ce33d65d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:15.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.222+0000 7ff7da467700 1 -- 172.21.15.37:0/2560825198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ce33cc820 msgr2=0x562ce33d8890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:05:15.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.222+0000 7ff7da467700 1 --2- 172.21.15.37:0/2560825198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ce33cc820 0x562ce33d8890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:15.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.222+0000 7ff7da467700 1 -- 172.21.15.37:0/2560825198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ce32e37d0 con 0x562ce33d2900 2024-06-10T01:05:15.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.223+0000 7ff7da467700 1 --2- 172.21.15.37:0/2560825198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ce33d2900 0x562ce33d5be0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4009740 tx=0x7ff7c4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f532b1c9d535b8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:15.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.223+0000 7ff7bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:05:15.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.223+0000 7ff7bdffb700 1 -- 172.21.15.37:0/2560825198 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4028060 con 0x562ce33d2900 2024-06-10T01:05:15.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.223+0000 7ff7bdffb700 1 -- 172.21.15.37:0/2560825198 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7ff7c40054d0 con 0x562ce33d2900 2024-06-10T01:05:15.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.223+0000 7ff7e6f4eb80 1 -- 172.21.15.37:0/2560825198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7cc007d80 con 0x562ce33d2900 2024-06-10T01:05:15.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.224+0000 7ff7bdffb700 1 -- 172.21.15.37:0/2560825198 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff7c402f050 con 0x562ce33d2900 2024-06-10T01:05:15.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.224+0000 7ff7e6f4eb80 10 client.16941 fetch_fsmap learned FSMap version 29 2024-06-10T01:05:15.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.224+0000 7ff7e6f4eb80 1 -- 172.21.15.37:0/2560825198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562ce33e40a0 con 0x562ce33d2900 2024-06-10T01:05:15.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.224+0000 7ff7bdffb700 1 -- 172.21.15.37:0/2560825198 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7cc007d80 con 0x562ce33d2900 2024-06-10T01:05:15.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.225+0000 7ff7bdffb700 1 -- 172.21.15.37:0/2560825198 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7ff7c400f610 con 0x562ce33d2900 2024-06-10T01:05:15.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.225+0000 7ff7e6f4eb80 10 client.16941 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:05:15.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.225+0000 7ff7e6f4eb80 10 client.16941 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:05:15.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.225+0000 7ff7e6f4eb80 20 client.16941 populate_metadata read hostname 'smithi037' 2024-06-10T01:05:15.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.225+0000 7ff7e6f4eb80 1 --2- 172.21.15.37:0/2560825198 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x562ce33e6a60 0x562ce33e8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:15.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.225+0000 7ff7e6f4eb80 4 client.16941 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:05:15.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.225+0000 7ff7e6f4eb80 1 -- 172.21.15.37:0/2560825198 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ce332add0 con 0x562ce33e6a60 2024-06-10T01:05:15.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.228+0000 7ff7d9c66700 1 --2- 172.21.15.37:0/2560825198 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x562ce33e6a60 0x562ce33e8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:15.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.235+0000 7ff7bdffb700 10 client.16941 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:05:15.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.235+0000 7ff7d9c66700 1 --2- 172.21.15.37:0/2560825198 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x562ce33e6a60 0x562ce33e8ec0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0796a23a46c1e5 server_cookie=936ada564a9fc343 in_seq=0 out_seq=0 2024-06-10T01:05:15.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.246+0000 7ff7bdffb700 1 -- 172.21.15.37:0/2560825198 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562ce332add0 con 0x562ce33e6a60 2024-06-10T01:05:15.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.246+0000 7ff7bdffb700 10 client.16941 handle_command_reply: tid=0 2024-06-10T01:05:15.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.246+0000 7ff7e6f4eb80 1 client.16941 shutdown 2024-06-10T01:05:15.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.246+0000 7ff7e6f4eb80 1 -- 172.21.15.37:0/2560825198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ce33d2900 msgr2=0x562ce33d5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:15.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.246+0000 7ff7e6f4eb80 1 --2- 172.21.15.37:0/2560825198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ce33d2900 0x562ce33d5be0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4009740 tx=0x7ff7c4009f90 comp rx=0 tx=0).stop 2024-06-10T01:05:15.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.246+0000 7ff7e6f4eb80 1 -- 172.21.15.37:0/2560825198 shutdown_connections 2024-06-10T01:05:15.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.246+0000 7ff7e6f4eb80 1 --2- 172.21.15.37:0/2560825198 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x562ce33e6a60 0x562ce33e8ec0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:15.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.246+0000 7ff7e6f4eb80 1 --2- 172.21.15.37:0/2560825198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ce33d6170 0x562ce33d65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:15.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.246+0000 7ff7e6f4eb80 1 --2- 172.21.15.37:0/2560825198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ce33cc820 0x562ce33d8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:15.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.246+0000 7ff7e6f4eb80 1 --2- 172.21.15.37:0/2560825198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ce33d2900 0x562ce33d5be0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:15.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.246+0000 7ff7e6f4eb80 1 -- 172.21.15.37:0/2560825198 >> 172.21.15.37:0/2560825198 conn(0x562ce32f9100 msgr2=0x562ce33c6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:15.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.246+0000 7ff7bdffb700 0 client.16941 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:05:15.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.248+0000 7ff7e6f4eb80 1 -- 172.21.15.37:0/2560825198 shutdown_connections 2024-06-10T01:05:15.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.249+0000 7ff7e6f4eb80 1 -- 172.21.15.37:0/2560825198 wait complete. 2024-06-10T01:05:15.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.249+0000 7ff7e6f4eb80 20 client.16941 trim_cache size 0 max 16384 2024-06-10T01:05:15.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.292+0000 7ff7e6f4eb80 1 Processor -- start 2024-06-10T01:05:15.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.296+0000 7ff7e6f4eb80 1 -- start start 2024-06-10T01:05:15.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.296+0000 7ff7e6f4eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ce33cc820 0x562ce33cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:15.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.296+0000 7ff7e6f4eb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ce33d2900 0x562ce33d4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:15.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.296+0000 7ff7e6f4eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ce33d6170 0x562ce33d13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:15.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.296+0000 7ff7e6f4eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d41c84a0 con 0x562ce33d6170 2024-06-10T01:05:15.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.297+0000 7ff7e6f4eb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d41c5d30 con 0x562ce33d2900 2024-06-10T01:05:15.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.297+0000 7ff7da467700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ce33d2900 0x562ce33d4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:15.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.297+0000 7ff7da467700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ce33d2900 0x562ce33d4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:54434/0 (socket says 172.21.15.37:54434) 2024-06-10T01:05:15.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.297+0000 7ff7da467700 1 -- 172.21.15.37:0/663012201 learned_addr learned my addr 172.21.15.37:0/663012201 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:15.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.297+0000 7ff7d9c66700 1 --2- 172.21.15.37:0/663012201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ce33cc820 0x562ce33cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:15.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.297+0000 7ff7e6f4eb80 1 -- 172.21.15.37:0/663012201 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d41c3b80 con 0x562ce33cc820 2024-06-10T01:05:15.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.300+0000 7ff7d9c66700 1 --2- 172.21.15.37:0/663012201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ce33d6170 0x562ce33d13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:15.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.301+0000 7ff7da467700 1 -- 172.21.15.37:0/663012201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ce33cc820 msgr2=0x562ce33cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:15.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.301+0000 7ff7da467700 1 --2- 172.21.15.37:0/663012201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ce33cc820 0x562ce33cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:15.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.301+0000 7ff7da467700 1 -- 172.21.15.37:0/663012201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ce33d6170 msgr2=0x562ce33d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:15.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.301+0000 7ff7da467700 1 --2- 172.21.15.37:0/663012201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ce33d6170 0x562ce33d13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:15.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.301+0000 7ff7da467700 1 -- 172.21.15.37:0/663012201 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ce33e9540 con 0x562ce33d2900 2024-06-10T01:05:15.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.304+0000 7ff7da467700 1 --2- 172.21.15.37:0/663012201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ce33d2900 0x562ce33d4b40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4005090 tx=0x7ff7c4009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acfa9bee3e3e9b6b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:15.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.308+0000 7ff7bffff700 1 -- 172.21.15.37:0/663012201 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4028070 con 0x562ce33d2900 2024-06-10T01:05:15.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.308+0000 7ff7bffff700 1 -- 172.21.15.37:0/663012201 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7c400f660 con 0x562ce33d2900 2024-06-10T01:05:15.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.308+0000 7ff7bffff700 1 -- 172.21.15.37:0/663012201 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c400bbe0 con 0x562ce33d2900 2024-06-10T01:05:15.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.312+0000 7ff7e6f4eb80 1 -- 172.21.15.37:0/663012201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ce33d2900 msgr2=0x562ce33d4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:15.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.312+0000 7ff7e6f4eb80 1 --2- 172.21.15.37:0/663012201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ce33d2900 0x562ce33d4b40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4005090 tx=0x7ff7c4009290 comp rx=0 tx=0).stop 2024-06-10T01:05:15.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.313+0000 7ff7e6f4eb80 1 -- 172.21.15.37:0/663012201 shutdown_connections 2024-06-10T01:05:15.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.313+0000 7ff7e6f4eb80 1 --2- 172.21.15.37:0/663012201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ce33d6170 0x562ce33d13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:15.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.313+0000 7ff7e6f4eb80 1 --2- 172.21.15.37:0/663012201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ce33cc820 0x562ce33cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:15.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.313+0000 7ff7e6f4eb80 1 --2- 172.21.15.37:0/663012201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ce33d2900 0x562ce33d4b40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:15.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.313+0000 7ff7e6f4eb80 1 -- 172.21.15.37:0/663012201 >> 172.21.15.37:0/663012201 conn(0x562ce32f9100 msgr2=0x562ce33d0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:15.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.316+0000 7ff7e6f4eb80 1 -- 172.21.15.37:0/663012201 shutdown_connections 2024-06-10T01:05:15.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.317+0000 7ff7e6f4eb80 1 -- 172.21.15.37:0/663012201 wait complete. 2024-06-10T01:05:15.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.318+0000 7ff7e6f4eb80 1 -- start start 2024-06-10T01:05:15.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.318+0000 7ff7e6f4eb80 1 Processor -- start 2024-06-10T01:05:15.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.319+0000 7ff7e6f4eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ce33cc820 0x562ce33db560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:15.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.319+0000 7ff7e6f4eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ce33d2900 0x562ce33dbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:15.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.319+0000 7ff7e6f4eb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ce33d6170 0x562ce3450760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:15.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.319+0000 7ff7e6f4eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d41c41f0 con 0x562ce33cc820 2024-06-10T01:05:15.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.319+0000 7ff7e6f4eb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d41c5660 con 0x562ce33d6170 2024-06-10T01:05:15.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.319+0000 7ff7e6f4eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d41c48c0 con 0x562ce33d2900 2024-06-10T01:05:15.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.319+0000 7ff7e6f4eb80 10 client.0 _ll_register_callbacks cb 0x562ce32b3630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:05:15.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.321+0000 7ff7d9c66700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ce33cc820 0x562ce33db560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:15.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.321+0000 7ff7d9c66700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ce33cc820 0x562ce33db560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58276/0 (socket says 172.21.15.37:58276) 2024-06-10T01:05:15.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.321+0000 7ff7d9c66700 1 -- 172.21.15.37:0/2672827919 learned_addr learned my addr 172.21.15.37:0/2672827919 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:15.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.321+0000 7ff7da467700 1 --2- 172.21.15.37:0/2672827919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ce33d2900 0x562ce33dbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:15.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.322+0000 7ff7d9c66700 1 --2- 172.21.15.37:0/2672827919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ce33d6170 0x562ce3450760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:15.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.326+0000 7ff7d9c66700 1 -- 172.21.15.37:0/2672827919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ce33d2900 msgr2=0x562ce33dbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:15.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.326+0000 7ff7d9c66700 1 --2- 172.21.15.37:0/2672827919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ce33d2900 0x562ce33dbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:15.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.326+0000 7ff7d9c66700 1 -- 172.21.15.37:0/2672827919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ce33d6170 msgr2=0x562ce3450760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:15.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.326+0000 7ff7d9c66700 1 --2- 172.21.15.37:0/2672827919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ce33d6170 0x562ce3450760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:15.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.326+0000 7ff7d9c66700 1 -- 172.21.15.37:0/2672827919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ce3450cf0 con 0x562ce33cc820 2024-06-10T01:05:15.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.333+0000 7ff7d9c66700 1 --2- 172.21.15.37:0/2672827919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ce33cc820 0x562ce33db560 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc004330 tx=0x7ff7cc003c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0e4209a7ed02fb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:15.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.334+0000 7ff7bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:05:15.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.336+0000 7ff7bdffb700 1 -- 172.21.15.37:0/2672827919 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc004a10 con 0x562ce33cc820 2024-06-10T01:05:15.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.336+0000 7ff7bdffb700 1 -- 172.21.15.37:0/2672827919 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7ff7cc004bb0 con 0x562ce33cc820 2024-06-10T01:05:15.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.338+0000 7ff7e6f4eb80 1 -- 172.21.15.37:0/2672827919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7c400f660 con 0x562ce33cc820 2024-06-10T01:05:15.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.339+0000 7ff7bdffb700 1 -- 172.21.15.37:0/2672827919 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff7cc004db0 con 0x562ce33cc820 2024-06-10T01:05:15.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.341+0000 7ff7e6f4eb80 10 client.16947 fetch_fsmap learned FSMap version 29 2024-06-10T01:05:15.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.341+0000 7ff7e6f4eb80 1 -- 172.21.15.37:0/2672827919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562ce3452030 con 0x562ce33cc820 2024-06-10T01:05:15.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.348+0000 7ff7bdffb700 1 -- 172.21.15.37:0/2672827919 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7cc029b40 con 0x562ce33cc820 2024-06-10T01:05:15.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.348+0000 7ff7bdffb700 1 -- 172.21.15.37:0/2672827919 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7ff7cc030040 con 0x562ce33cc820 2024-06-10T01:05:15.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.349+0000 7ff7e6f4eb80 10 client.16947 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:05:15.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.349+0000 7ff7e6f4eb80 10 client.16947 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:05:15.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.349+0000 7ff7e6f4eb80 20 client.16947 populate_metadata read hostname 'smithi037' 2024-06-10T01:05:15.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.349+0000 7ff7e6f4eb80 1 --2- 172.21.15.37:0/2672827919 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x562ce3452260 0x562ce3454650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:15.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.349+0000 7ff7e6f4eb80 4 client.16947 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:05:15.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.349+0000 7ff7e6f4eb80 1 -- 172.21.15.37:0/2672827919 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7ff7c400f660 con 0x562ce3452260 2024-06-10T01:05:15.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.360+0000 7ff7da467700 1 --2- 172.21.15.37:0/2672827919 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x562ce3452260 0x562ce3454650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:15.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.366+0000 7ff7da467700 1 --2- 172.21.15.37:0/2672827919 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x562ce3452260 0x562ce3454650 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fde6f4f1941e7a88 server_cookie=2f5c0320c6337e6b in_seq=0 out_seq=0 2024-06-10T01:05:15.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:15.366+0000 7ff7bdffb700 10 client.16947 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:05:16.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:16 smithi094 ceph-mon[36708]: pgmap v688: 129 pgs: 129 active+clean; 31 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 437 op/s 2024-06-10T01:05:16.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:16 smithi037 ceph-mon[32192]: pgmap v688: 129 pgs: 129 active+clean; 31 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 437 op/s 2024-06-10T01:05:16.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:16 smithi167 ceph-mon[36917]: pgmap v688: 129 pgs: 129 active+clean; 31 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 437 op/s 2024-06-10T01:05:17.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:17.275+0000 7ff7bdffb700 1 -- 172.21.15.37:0/2672827919 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7ff7c400f660 con 0x562ce3452260 2024-06-10T01:05:17.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:17.275+0000 7ff7bdffb700 10 client.16947 handle_command_reply: tid=0 2024-06-10T01:05:17.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:17.275+0000 7ff7e6f4eb80 1 client.16947 shutdown 2024-06-10T01:05:17.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:17.276+0000 7ff7e6f4eb80 1 -- 172.21.15.37:0/2672827919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ce33cc820 msgr2=0x562ce33db560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:17.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:17.276+0000 7ff7e6f4eb80 1 --2- 172.21.15.37:0/2672827919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ce33cc820 0x562ce33db560 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc004330 tx=0x7ff7cc003c50 comp rx=0 tx=0).stop 2024-06-10T01:05:17.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:17.276+0000 7ff7e6f4eb80 1 -- 172.21.15.37:0/2672827919 shutdown_connections 2024-06-10T01:05:17.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:17.276+0000 7ff7e6f4eb80 1 --2- 172.21.15.37:0/2672827919 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x562ce3452260 0x562ce3454650 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:17.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:17.276+0000 7ff7e6f4eb80 1 --2- 172.21.15.37:0/2672827919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562ce33d6170 0x562ce3450760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:17.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:17.276+0000 7ff7e6f4eb80 1 --2- 172.21.15.37:0/2672827919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ce33cc820 0x562ce33db560 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:17.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:17.276+0000 7ff7e6f4eb80 1 --2- 172.21.15.37:0/2672827919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ce33d2900 0x562ce33dbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:17.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:17.276+0000 7ff7e6f4eb80 1 -- 172.21.15.37:0/2672827919 >> 172.21.15.37:0/2672827919 conn(0x562ce32f9100 msgr2=0x562ce33c6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:17.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:17.276+0000 7ff7bdffb700 0 client.16947 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:05:17.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:17.277+0000 7ff7e6f4eb80 1 -- 172.21.15.37:0/2672827919 shutdown_connections 2024-06-10T01:05:17.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:17.277+0000 7ff7e6f4eb80 1 -- 172.21.15.37:0/2672827919 wait complete. 2024-06-10T01:05:17.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:17.277+0000 7ff7e6f4eb80 20 client.16947 trim_cache size 0 max 16384 2024-06-10T01:05:17.275 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:05:17.275 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-06-10T01:05:17.275 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "d79c0c89-cbf2-47ac-9a2b-0a728f0b2713", 2024-06-10T01:05:17.275 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-06-10T01:05:17.275 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:05:17.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:17.278+0000 7ff7dc6cb700 1 -- 172.21.15.37:0/447566694 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff7c00a2cf0 msgr2=0x7ff7c00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:17.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:17.278+0000 7ff7dc6cb700 1 --2- 172.21.15.37:0/447566694 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff7c00a2cf0 0x7ff7c00a51b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0096ce0 tx=0x7ff7d0096b70 comp rx=0 tx=0).stop 2024-06-10T01:05:17.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:17.278+0000 7ff7dc6cb700 1 -- 172.21.15.37:0/447566694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7d4113350 msgr2=0x7ff7d41137d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:17.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:17.278+0000 7ff7dc6cb700 1 --2- 172.21.15.37:0/447566694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7d4113350 0x7ff7d41137d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000b4a0 tx=0x7ff7d000b7b0 comp rx=0 tx=0).stop 2024-06-10T01:05:17.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:17.278+0000 7ff7dc6cb700 1 -- 172.21.15.37:0/447566694 shutdown_connections 2024-06-10T01:05:17.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:17.278+0000 7ff7dc6cb700 1 --2- 172.21.15.37:0/447566694 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff7c00a2cf0 0x7ff7c00a51b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:17.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:17.278+0000 7ff7dc6cb700 1 --2- 172.21.15.37:0/447566694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7d4113350 0x7ff7d41137d0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:17.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:17.278+0000 7ff7dc6cb700 1 --2- 172.21.15.37:0/447566694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7d4110bf0 0x7ff7d411a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:17.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:17.278+0000 7ff7dc6cb700 1 --2- 172.21.15.37:0/447566694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7d411bc70 0x7ff7d411a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:17.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:17.278+0000 7ff7dc6cb700 1 -- 172.21.15.37:0/447566694 >> 172.21.15.37:0/447566694 conn(0x7ff7d406d600 msgr2=0x7ff7d4071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:17.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:17.278+0000 7ff7dc6cb700 1 -- 172.21.15.37:0/447566694 shutdown_connections 2024-06-10T01:05:17.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:17.278+0000 7ff7dc6cb700 1 -- 172.21.15.37:0/447566694 wait complete. 2024-06-10T01:05:17.977 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:05:18.239 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:05:18.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:18 smithi094 ceph-mon[36708]: pgmap v689: 129 pgs: 129 active+clean; 33 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 588 op/s 2024-06-10T01:05:18.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:18 smithi094 ceph-mon[36708]: scrub queued for path: / 2024-06-10T01:05:18.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:18 smithi094 ceph-mon[36708]: scrub summary: idle+waiting paths [/] 2024-06-10T01:05:18.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:18 smithi094 ceph-mon[36708]: scrub summary: active paths [/] 2024-06-10T01:05:18.362 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:18 smithi037 ceph-mon[32192]: pgmap v689: 129 pgs: 129 active+clean; 33 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 588 op/s 2024-06-10T01:05:18.362 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:18 smithi037 ceph-mon[32192]: scrub queued for path: / 2024-06-10T01:05:18.362 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:18 smithi037 ceph-mon[32192]: scrub summary: idle+waiting paths [/] 2024-06-10T01:05:18.362 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:18 smithi037 ceph-mon[32192]: scrub summary: active paths [/] 2024-06-10T01:05:18.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:18 smithi167 ceph-mon[36917]: pgmap v689: 129 pgs: 129 active+clean; 33 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 588 op/s 2024-06-10T01:05:18.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:18 smithi167 ceph-mon[36917]: scrub queued for path: / 2024-06-10T01:05:18.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:18 smithi167 ceph-mon[36917]: scrub summary: idle+waiting paths [/] 2024-06-10T01:05:18.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:18 smithi167 ceph-mon[36917]: scrub summary: active paths [/] 2024-06-10T01:05:20.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.219+0000 7f5fe4093700 1 -- 172.21.15.37:0/3752222353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fdc1030f0 msgr2=0x7f5fdc107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:20.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.219+0000 7f5fe4093700 1 --2- 172.21.15.37:0/3752222353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fdc1030f0 0x7f5fdc107110 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0099c0 tx=0x7f5fcc009cd0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.220+0000 7f5fe4093700 1 -- 172.21.15.37:0/3752222353 shutdown_connections 2024-06-10T01:05:20.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.220+0000 7f5fe4093700 1 --2- 172.21.15.37:0/3752222353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fdc107840 0x7f5fdc10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.220+0000 7f5fe4093700 1 --2- 172.21.15.37:0/3752222353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fdc102740 0x7f5fdc102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.220+0000 7f5fe4093700 1 --2- 172.21.15.37:0/3752222353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fdc1030f0 0x7f5fdc107110 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.220+0000 7f5fe4093700 1 -- 172.21.15.37:0/3752222353 >> 172.21.15.37:0/3752222353 conn(0x7f5fdc0795f0 msgr2=0x7f5fdc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:20.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.220+0000 7f5fe4093700 1 -- 172.21.15.37:0/3752222353 shutdown_connections 2024-06-10T01:05:20.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.220+0000 7f5fe4093700 1 -- 172.21.15.37:0/3752222353 wait complete. 2024-06-10T01:05:20.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.220+0000 7f5fe4093700 1 Processor -- start 2024-06-10T01:05:20.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.221+0000 7f5fe4093700 1 -- start start 2024-06-10T01:05:20.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.221+0000 7f5fe4093700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fdc102740 0x7f5fdc19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:20.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.221+0000 7f5fe4093700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fdc1030f0 0x7f5fdc19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:20.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.221+0000 7f5fe4093700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fdc107840 0x7f5fdc198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:20.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.221+0000 7f5fe4093700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc112770 con 0x7f5fdc107840 2024-06-10T01:05:20.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.221+0000 7f5fe4093700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1125f0 con 0x7f5fdc1030f0 2024-06-10T01:05:20.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.221+0000 7f5fe4093700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1128f0 con 0x7f5fdc102740 2024-06-10T01:05:20.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.221+0000 7f5fe2630700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fdc107840 0x7f5fdc198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:20.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.221+0000 7f5fe2630700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fdc107840 0x7f5fdc198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58296/0 (socket says 172.21.15.37:58296) 2024-06-10T01:05:20.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.221+0000 7f5fe2630700 1 -- 172.21.15.37:0/4120391554 learned_addr learned my addr 172.21.15.37:0/4120391554 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:20.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.221+0000 7f5fe1e2f700 1 --2- 172.21.15.37:0/4120391554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fdc102740 0x7f5fdc19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:20.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.221+0000 7f5fe162e700 1 --2- 172.21.15.37:0/4120391554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fdc1030f0 0x7f5fdc19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:20.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.221+0000 7f5fe2630700 1 -- 172.21.15.37:0/4120391554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fdc102740 msgr2=0x7f5fdc19e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:20.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.221+0000 7f5fe2630700 1 --2- 172.21.15.37:0/4120391554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fdc102740 0x7f5fdc19e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.221+0000 7f5fe2630700 1 -- 172.21.15.37:0/4120391554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fdc1030f0 msgr2=0x7f5fdc19ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:20.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.221+0000 7f5fe2630700 1 --2- 172.21.15.37:0/4120391554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fdc1030f0 0x7f5fdc19ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.221+0000 7f5fe2630700 1 -- 172.21.15.37:0/4120391554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fdc1990f0 con 0x7f5fdc107840 2024-06-10T01:05:20.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.221+0000 7f5fe162e700 1 --2- 172.21.15.37:0/4120391554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fdc1030f0 0x7f5fdc19ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:05:20.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.222+0000 7f5fe2630700 1 --2- 172.21.15.37:0/4120391554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fdc107840 0x7f5fdc198990 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400ef30 tx=0x7f5fd400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:20.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.222+0000 7f5fd2ffd700 1 -- 172.21.15.37:0/4120391554 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd40092e0 con 0x7f5fdc107840 2024-06-10T01:05:20.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.222+0000 7f5fd2ffd700 1 -- 172.21.15.37:0/4120391554 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fd4004bb0 con 0x7f5fdc107840 2024-06-10T01:05:20.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.222+0000 7f5fe4093700 1 -- 172.21.15.37:0/4120391554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fdc199380 con 0x7f5fdc107840 2024-06-10T01:05:20.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.222+0000 7f5fd2ffd700 1 -- 172.21.15.37:0/4120391554 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4004280 con 0x7f5fdc107840 2024-06-10T01:05:20.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.222+0000 7f5fe4093700 1 -- 172.21.15.37:0/4120391554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fdc1a56a0 con 0x7f5fdc107840 2024-06-10T01:05:20.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.224+0000 7f5fd2ffd700 1 -- 172.21.15.37:0/4120391554 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5fd401b030 con 0x7f5fdc107840 2024-06-10T01:05:20.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.225+0000 7f5fd2ffd700 1 --2- 172.21.15.37:0/4120391554 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5fc80a2ca0 0x7f5fc80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:20.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.225+0000 7f5fe1e2f700 1 --2- 172.21.15.37:0/4120391554 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5fc80a2ca0 0x7f5fc80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:20.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.225+0000 7f5fd2ffd700 1 -- 172.21.15.37:0/4120391554 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f5fd40ceec0 con 0x7f5fdc107840 2024-06-10T01:05:20.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.225+0000 7f5fe1e2f700 1 --2- 172.21.15.37:0/4120391554 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5fc80a2ca0 0x7f5fc80a5160 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc199ad0 tx=0x7f5fd800a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.226+0000 7f5fee916b80 1 Processor -- start 2024-06-10T01:05:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.226+0000 7f5fee916b80 1 -- start start 2024-06-10T01:05:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.226+0000 7f5fee916b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82ee75a70 0x55b82ee75e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.226+0000 7f5fee916b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82ee76390 0x55b82ee6fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.226+0000 7f5fee916b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b82ee70330 0x55b82ee70750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.226+0000 7f5fee916b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b82edbd810 con 0x55b82ee76390 2024-06-10T01:05:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.226+0000 7f5fee916b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b82ed65720 con 0x55b82ee70330 2024-06-10T01:05:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.226+0000 7f5fee916b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b82edb97d0 con 0x55b82ee75a70 2024-06-10T01:05:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.226+0000 7f5fe162e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82ee76390 0x55b82ee6fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.226+0000 7f5fe162e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82ee76390 0x55b82ee6fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58302/0 (socket says 172.21.15.37:58302) 2024-06-10T01:05:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.226+0000 7f5fe1e2f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82ee75a70 0x55b82ee75e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.226+0000 7f5fe162e700 1 -- 172.21.15.37:0/4260418399 learned_addr learned my addr 172.21.15.37:0/4260418399 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.227+0000 7f5fe2630700 1 --2- 172.21.15.37:0/4260418399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b82ee70330 0x55b82ee70750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.227+0000 7f5fe1e2f700 1 -- 172.21.15.37:0/4260418399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b82ee70330 msgr2=0x55b82ee70750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.227+0000 7f5fe1e2f700 1 --2- 172.21.15.37:0/4260418399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b82ee70330 0x55b82ee70750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.227+0000 7f5fe1e2f700 1 -- 172.21.15.37:0/4260418399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82ee76390 msgr2=0x55b82ee6fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.227+0000 7f5fe1e2f700 1 --2- 172.21.15.37:0/4260418399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82ee76390 0x55b82ee6fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.227+0000 7f5fe1e2f700 1 -- 172.21.15.37:0/4260418399 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b82ed5bf50 con 0x55b82ee75a70 2024-06-10T01:05:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.227+0000 7f5fe162e700 1 --2- 172.21.15.37:0/4260418399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82ee76390 0x55b82ee6fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.227+0000 7f5fe1e2f700 1 --2- 172.21.15.37:0/4260418399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82ee75a70 0x55b82ee75e50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800fdc0 tx=0x7f5fd8012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcca286d852286fe server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.227+0000 7f5fc7fff700 1 -- 172.21.15.37:0/4260418399 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8016070 con 0x55b82ee75a70 2024-06-10T01:05:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.227+0000 7f5fc7fff700 1 -- 172.21.15.37:0/4260418399 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fd8014070 con 0x55b82ee75a70 2024-06-10T01:05:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.227+0000 7f5fc7fff700 1 -- 172.21.15.37:0/4260418399 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8010dc0 con 0x55b82ee75a70 2024-06-10T01:05:20.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.228+0000 7f5fee916b80 1 -- 172.21.15.37:0/4260418399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82ee75a70 msgr2=0x55b82ee75e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:20.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.228+0000 7f5fee916b80 1 --2- 172.21.15.37:0/4260418399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82ee75a70 0x55b82ee75e50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800fdc0 tx=0x7f5fd8012000 comp rx=0 tx=0).stop 2024-06-10T01:05:20.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.228+0000 7f5fee916b80 1 -- 172.21.15.37:0/4260418399 shutdown_connections 2024-06-10T01:05:20.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.228+0000 7f5fee916b80 1 --2- 172.21.15.37:0/4260418399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b82ee70330 0x55b82ee70750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.228+0000 7f5fee916b80 1 --2- 172.21.15.37:0/4260418399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82ee75a70 0x55b82ee75e50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.228+0000 7f5fee916b80 1 --2- 172.21.15.37:0/4260418399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82ee76390 0x55b82ee6fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.228+0000 7f5fee916b80 1 -- 172.21.15.37:0/4260418399 >> 172.21.15.37:0/4260418399 conn(0x55b82ed96cc0 msgr2=0x55b82ee66850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:20.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.228+0000 7f5fee916b80 1 -- 172.21.15.37:0/4260418399 shutdown_connections 2024-06-10T01:05:20.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.228+0000 7f5fee916b80 1 -- 172.21.15.37:0/4260418399 wait complete. 2024-06-10T01:05:20.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.228+0000 7f5fee916b80 1 -- start start 2024-06-10T01:05:20.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.229+0000 7f5fee916b80 1 Processor -- start 2024-06-10T01:05:20.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.229+0000 7f5fee916b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b82ee70330 0x55b82ee7c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:20.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.229+0000 7f5fee916b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82ee75a70 0x55b82ee79440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:20.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.229+0000 7f5fee916b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82ee76390 0x55b82ee79980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:20.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.229+0000 7f5fee916b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b82ee7cdd0 con 0x55b82ee76390 2024-06-10T01:05:20.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.229+0000 7f5fee916b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b82ed5dbf0 con 0x55b82ee70330 2024-06-10T01:05:20.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.229+0000 7f5fee916b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b82ee7cf50 con 0x55b82ee75a70 2024-06-10T01:05:20.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.229+0000 7f5fee916b80 10 client.0 _ll_register_callbacks cb 0x55b82ed57920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:05:20.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.229+0000 7f5fe2630700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82ee76390 0x55b82ee79980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:20.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.229+0000 7f5fe162e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82ee75a70 0x55b82ee79440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:20.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.229+0000 7f5fe162e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82ee75a70 0x55b82ee79440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:33392/0 (socket says 172.21.15.37:33392) 2024-06-10T01:05:20.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.229+0000 7f5fe162e700 1 -- 172.21.15.37:0/4235605430 learned_addr learned my addr 172.21.15.37:0/4235605430 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:20.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.229+0000 7f5fe2630700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82ee76390 0x55b82ee79980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58306/0 (socket says 172.21.15.37:58306) 2024-06-10T01:05:20.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.229+0000 7f5fe1e2f700 1 --2- 172.21.15.37:0/4235605430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b82ee70330 0x55b82ee7c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:20.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.230+0000 7f5fe162e700 1 -- 172.21.15.37:0/4235605430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b82ee70330 msgr2=0x55b82ee7c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:20.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.230+0000 7f5fe162e700 1 --2- 172.21.15.37:0/4235605430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b82ee70330 0x55b82ee7c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.230+0000 7f5fe162e700 1 -- 172.21.15.37:0/4235605430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82ee76390 msgr2=0x55b82ee79980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:20.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.230+0000 7f5fe162e700 1 --2- 172.21.15.37:0/4235605430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82ee76390 0x55b82ee79980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.230+0000 7f5fe162e700 1 -- 172.21.15.37:0/4235605430 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b82ed0c8d0 con 0x55b82ee75a70 2024-06-10T01:05:20.230 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:20 smithi037 ceph-mon[32192]: pgmap v690: 129 pgs: 129 active+clean; 36 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 1.98k op/s 2024-06-10T01:05:20.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.230+0000 7f5fe162e700 1 --2- 172.21.15.37:0/4235605430 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82ee75a70 0x55b82ee79440 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009990 tx=0x7f5fcc00bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15674cf28c3cad34 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:20.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.230+0000 7f5fc57fa700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:05:20.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.230+0000 7f5fc57fa700 1 -- 172.21.15.37:0/4235605430 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc029060 con 0x55b82ee75a70 2024-06-10T01:05:20.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.230+0000 7f5fee916b80 1 -- 172.21.15.37:0/4235605430 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fd8014070 con 0x55b82ee75a70 2024-06-10T01:05:20.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.230+0000 7f5fc57fa700 1 -- 172.21.15.37:0/4235605430 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f5fcc0054e0 con 0x55b82ee75a70 2024-06-10T01:05:20.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.230+0000 7f5fc57fa700 1 -- 172.21.15.37:0/4235605430 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5fcc030050 con 0x55b82ee75a70 2024-06-10T01:05:20.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.230+0000 7f5fee916b80 10 client.26488 fetch_fsmap learned FSMap version 29 2024-06-10T01:05:20.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.230+0000 7f5fee916b80 1 -- 172.21.15.37:0/4235605430 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b82ee87770 con 0x55b82ee75a70 2024-06-10T01:05:20.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.231+0000 7f5fc57fa700 1 -- 172.21.15.37:0/4235605430 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fd8014070 con 0x55b82ee75a70 2024-06-10T01:05:20.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.231+0000 7f5fc57fa700 1 -- 172.21.15.37:0/4235605430 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f5fcc00f760 con 0x55b82ee75a70 2024-06-10T01:05:20.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.231+0000 7f5fee916b80 10 client.26488 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:05:20.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.231+0000 7f5fee916b80 10 client.26488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:05:20.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.231+0000 7f5fee916b80 20 client.26488 populate_metadata read hostname 'smithi037' 2024-06-10T01:05:20.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.231+0000 7f5fee916b80 1 --2- 172.21.15.37:0/4235605430 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b82ee89db0 0x55b82ee8c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:20.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.231+0000 7f5fee916b80 4 client.26488 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:05:20.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.231+0000 7f5fee916b80 1 -- 172.21.15.37:0/4235605430 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b82edcad10 con 0x55b82ee89db0 2024-06-10T01:05:20.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.231+0000 7f5fe1e2f700 1 --2- 172.21.15.37:0/4235605430 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b82ee89db0 0x55b82ee8c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:20.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.232+0000 7f5fc57fa700 10 client.26488 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:05:20.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.232+0000 7f5fe1e2f700 1 --2- 172.21.15.37:0/4235605430 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b82ee89db0 0x55b82ee8c1a0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea5f94a728b62fd1 server_cookie=d83077a54b3fb496 in_seq=0 out_seq=0 2024-06-10T01:05:20.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.234+0000 7f5fc57fa700 1 -- 172.21.15.37:0/4235605430 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b82edcad10 con 0x55b82ee89db0 2024-06-10T01:05:20.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.234+0000 7f5fc57fa700 10 client.26488 handle_command_reply: tid=0 2024-06-10T01:05:20.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.234+0000 7f5fee916b80 1 client.26488 shutdown 2024-06-10T01:05:20.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.234+0000 7f5fee916b80 1 -- 172.21.15.37:0/4235605430 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82ee75a70 msgr2=0x55b82ee79440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:20.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.234+0000 7f5fee916b80 1 --2- 172.21.15.37:0/4235605430 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82ee75a70 0x55b82ee79440 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009990 tx=0x7f5fcc00bf50 comp rx=0 tx=0).stop 2024-06-10T01:05:20.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.234+0000 7f5fee916b80 1 -- 172.21.15.37:0/4235605430 shutdown_connections 2024-06-10T01:05:20.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.234+0000 7f5fee916b80 1 --2- 172.21.15.37:0/4235605430 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b82ee89db0 0x55b82ee8c1a0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.234+0000 7f5fee916b80 1 --2- 172.21.15.37:0/4235605430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82ee76390 0x55b82ee79980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.234+0000 7f5fee916b80 1 --2- 172.21.15.37:0/4235605430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b82ee70330 0x55b82ee7c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.234+0000 7f5fee916b80 1 --2- 172.21.15.37:0/4235605430 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82ee75a70 0x55b82ee79440 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.235+0000 7f5fee916b80 1 -- 172.21.15.37:0/4235605430 >> 172.21.15.37:0/4235605430 conn(0x55b82ed96cc0 msgr2=0x55b82ee6a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:20.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.235+0000 7f5fc57fa700 0 client.26488 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:05:20.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.235+0000 7f5fee916b80 1 -- 172.21.15.37:0/4235605430 shutdown_connections 2024-06-10T01:05:20.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.235+0000 7f5fee916b80 1 -- 172.21.15.37:0/4235605430 wait complete. 2024-06-10T01:05:20.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.235+0000 7f5fee916b80 20 client.26488 trim_cache size 0 max 16384 2024-06-10T01:05:20.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.256+0000 7f5fee916b80 1 Processor -- start 2024-06-10T01:05:20.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.256+0000 7f5fee916b80 1 -- start start 2024-06-10T01:05:20.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.256+0000 7f5fee916b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82ee70330 0x55b82ee77000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:20.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.256+0000 7f5fee916b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82ee75a70 0x55b82ee91a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:20.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.256+0000 7f5fee916b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b82ee76390 0x55b82ee91f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:20.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.256+0000 7f5fee916b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1aeaf0 con 0x55b82ee70330 2024-06-10T01:05:20.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.256+0000 7f5fee916b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1ac380 con 0x55b82ee76390 2024-06-10T01:05:20.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.256+0000 7f5fee916b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1aa1d0 con 0x55b82ee75a70 2024-06-10T01:05:20.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.256+0000 7f5fe1e2f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82ee70330 0x55b82ee77000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:20.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.256+0000 7f5fe1e2f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82ee70330 0x55b82ee77000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58320/0 (socket says 172.21.15.37:58320) 2024-06-10T01:05:20.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.256+0000 7f5fe1e2f700 1 -- 172.21.15.37:0/3347751751 learned_addr learned my addr 172.21.15.37:0/3347751751 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:20.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.256+0000 7f5fe162e700 1 --2- 172.21.15.37:0/3347751751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82ee75a70 0x55b82ee91a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:20.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.257+0000 7f5fe1e2f700 1 -- 172.21.15.37:0/3347751751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82ee75a70 msgr2=0x55b82ee91a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:20.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.257+0000 7f5fe1e2f700 1 --2- 172.21.15.37:0/3347751751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82ee75a70 0x55b82ee91a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.257+0000 7f5fe1e2f700 1 -- 172.21.15.37:0/3347751751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b82ee76390 msgr2=0x55b82ee91f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:20.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.257+0000 7f5fe1e2f700 1 --2- 172.21.15.37:0/3347751751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b82ee76390 0x55b82ee91f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.257+0000 7f5fe1e2f700 1 -- 172.21.15.37:0/3347751751 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b82ee8c800 con 0x55b82ee70330 2024-06-10T01:05:20.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.257+0000 7f5fe162e700 1 --2- 172.21.15.37:0/3347751751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82ee75a70 0x55b82ee91a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:05:20.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.257+0000 7f5fe1e2f700 1 --2- 172.21.15.37:0/3347751751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82ee70330 0x55b82ee77000 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5fd80108f0 tx=0x7f5fd8010920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3779acd64c9e75b5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:20.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.257+0000 7f5fc77fe700 1 -- 172.21.15.37:0/3347751751 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8036020 con 0x55b82ee70330 2024-06-10T01:05:20.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.257+0000 7f5fc77fe700 1 -- 172.21.15.37:0/3347751751 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fd8014040 con 0x55b82ee70330 2024-06-10T01:05:20.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.257+0000 7f5fc77fe700 1 -- 172.21.15.37:0/3347751751 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8005330 con 0x55b82ee70330 2024-06-10T01:05:20.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.258+0000 7f5fee916b80 1 -- 172.21.15.37:0/3347751751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82ee70330 msgr2=0x55b82ee77000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:20.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.258+0000 7f5fee916b80 1 --2- 172.21.15.37:0/3347751751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82ee70330 0x55b82ee77000 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5fd80108f0 tx=0x7f5fd8010920 comp rx=0 tx=0).stop 2024-06-10T01:05:20.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.258+0000 7f5fee916b80 1 -- 172.21.15.37:0/3347751751 shutdown_connections 2024-06-10T01:05:20.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.258+0000 7f5fee916b80 1 --2- 172.21.15.37:0/3347751751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b82ee76390 0x55b82ee91f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.258+0000 7f5fee916b80 1 --2- 172.21.15.37:0/3347751751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82ee70330 0x55b82ee77000 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.258+0000 7f5fee916b80 1 --2- 172.21.15.37:0/3347751751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82ee75a70 0x55b82ee91a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.258+0000 7f5fee916b80 1 -- 172.21.15.37:0/3347751751 >> 172.21.15.37:0/3347751751 conn(0x55b82ed96cc0 msgr2=0x55b82ee60470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:20.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.258+0000 7f5fee916b80 1 -- 172.21.15.37:0/3347751751 shutdown_connections 2024-06-10T01:05:20.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.258+0000 7f5fee916b80 1 -- 172.21.15.37:0/3347751751 wait complete. 2024-06-10T01:05:20.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.258+0000 7f5fee916b80 1 -- start start 2024-06-10T01:05:20.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.258+0000 7f5fee916b80 1 Processor -- start 2024-06-10T01:05:20.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.259+0000 7f5fee916b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b82ee70330 0x55b82ee7e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:20.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.259+0000 7f5fee916b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82ee75a70 0x55b82ee7ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:20.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.259+0000 7f5fee916b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82ee76390 0x55b82eeeda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:20.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.259+0000 7f5fee916b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1aa840 con 0x55b82ee75a70 2024-06-10T01:05:20.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.259+0000 7f5fee916b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1abcb0 con 0x55b82ee70330 2024-06-10T01:05:20.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.259+0000 7f5fee916b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1aaf10 con 0x55b82ee76390 2024-06-10T01:05:20.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.259+0000 7f5fee916b80 10 client.0 _ll_register_callbacks cb 0x55b82ed57920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:05:20.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.259+0000 7f5fe162e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82ee75a70 0x55b82ee7ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:20.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.259+0000 7f5fe1e2f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b82ee70330 0x55b82ee7e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:20.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.259+0000 7f5fe162e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82ee75a70 0x55b82ee7ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58334/0 (socket says 172.21.15.37:58334) 2024-06-10T01:05:20.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.259+0000 7f5fe162e700 1 -- 172.21.15.37:0/2001736101 learned_addr learned my addr 172.21.15.37:0/2001736101 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:20.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.259+0000 7f5fe2630700 1 --2- 172.21.15.37:0/2001736101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82ee76390 0x55b82eeeda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:20.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.259+0000 7f5fe162e700 1 -- 172.21.15.37:0/2001736101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82ee76390 msgr2=0x55b82eeeda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:20.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.259+0000 7f5fe162e700 1 --2- 172.21.15.37:0/2001736101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82ee76390 0x55b82eeeda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.259+0000 7f5fe162e700 1 -- 172.21.15.37:0/2001736101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b82ee70330 msgr2=0x55b82ee7e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:20.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.259+0000 7f5fe162e700 1 --2- 172.21.15.37:0/2001736101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b82ee70330 0x55b82ee7e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.259+0000 7f5fe162e700 1 -- 172.21.15.37:0/2001736101 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b82eeee080 con 0x55b82ee75a70 2024-06-10T01:05:20.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.259+0000 7f5fe1e2f700 1 --2- 172.21.15.37:0/2001736101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b82ee70330 0x55b82ee7e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:20.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.259+0000 7f5fe162e700 1 --2- 172.21.15.37:0/2001736101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82ee75a70 0x55b82ee7ed70 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc00fcf0 tx=0x7f5fcc009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169cca1944bdd0b8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:20.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.259+0000 7f5fc57fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:05:20.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.260+0000 7f5fc57fa700 1 -- 172.21.15.37:0/2001736101 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc039060 con 0x55b82ee75a70 2024-06-10T01:05:20.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.260+0000 7f5fee916b80 1 -- 172.21.15.37:0/2001736101 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fd8014040 con 0x55b82ee75a70 2024-06-10T01:05:20.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.260+0000 7f5fc57fa700 1 -- 172.21.15.37:0/2001736101 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f5fcc0054e0 con 0x55b82ee75a70 2024-06-10T01:05:20.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.260+0000 7f5fc57fa700 1 -- 172.21.15.37:0/2001736101 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5fcc040050 con 0x55b82ee75a70 2024-06-10T01:05:20.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.260+0000 7f5fee916b80 10 client.16965 fetch_fsmap learned FSMap version 29 2024-06-10T01:05:20.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.260+0000 7f5fee916b80 1 -- 172.21.15.37:0/2001736101 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b82eef5440 con 0x55b82ee75a70 2024-06-10T01:05:20.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.260+0000 7f5fc57fa700 1 -- 172.21.15.37:0/2001736101 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fd8014040 con 0x55b82ee75a70 2024-06-10T01:05:20.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.260+0000 7f5fc57fa700 1 -- 172.21.15.37:0/2001736101 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f5fcc016650 con 0x55b82ee75a70 2024-06-10T01:05:20.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.260+0000 7f5fee916b80 10 client.16965 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:05:20.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.260+0000 7f5fee916b80 10 client.16965 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:05:20.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.260+0000 7f5fee916b80 20 client.16965 populate_metadata read hostname 'smithi037' 2024-06-10T01:05:20.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.260+0000 7f5fee916b80 1 --2- 172.21.15.37:0/2001736101 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b82eef5a60 0x55b82eef7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:20.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.260+0000 7f5fee916b80 4 client.16965 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:05:20.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.260+0000 7f5fee916b80 1 -- 172.21.15.37:0/2001736101 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b82edc5290 con 0x55b82eef5a60 2024-06-10T01:05:20.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.261+0000 7f5fe1e2f700 1 --2- 172.21.15.37:0/2001736101 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b82eef5a60 0x55b82eef7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:20.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.261+0000 7f5fc57fa700 10 client.16965 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:05:20.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.261+0000 7f5fe1e2f700 1 --2- 172.21.15.37:0/2001736101 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b82eef5a60 0x55b82eef7e80 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f7121ae7af17070 server_cookie=6fb8107efba77c8d in_seq=0 out_seq=0 2024-06-10T01:05:20.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.264+0000 7f5fc57fa700 1 -- 172.21.15.37:0/2001736101 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x55b82edc5290 con 0x55b82eef5a60 2024-06-10T01:05:20.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.264+0000 7f5fc57fa700 10 client.16965 handle_command_reply: tid=0 2024-06-10T01:05:20.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.264+0000 7f5fee916b80 1 client.16965 shutdown 2024-06-10T01:05:20.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.264+0000 7f5fee916b80 1 -- 172.21.15.37:0/2001736101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82ee75a70 msgr2=0x55b82ee7ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:20.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.264+0000 7f5fee916b80 1 --2- 172.21.15.37:0/2001736101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82ee75a70 0x55b82ee7ed70 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc00fcf0 tx=0x7f5fcc009290 comp rx=0 tx=0).stop 2024-06-10T01:05:20.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.264+0000 7f5fee916b80 1 -- 172.21.15.37:0/2001736101 shutdown_connections 2024-06-10T01:05:20.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.264+0000 7f5fee916b80 1 --2- 172.21.15.37:0/2001736101 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b82eef5a60 0x55b82eef7e80 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.264+0000 7f5fee916b80 1 --2- 172.21.15.37:0/2001736101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b82ee76390 0x55b82eeeda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.264+0000 7f5fee916b80 1 --2- 172.21.15.37:0/2001736101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b82ee70330 0x55b82ee7e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.264+0000 7f5fc57fa700 0 client.16965 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:05:20.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.264+0000 7f5fee916b80 1 --2- 172.21.15.37:0/2001736101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b82ee75a70 0x55b82ee7ed70 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.264+0000 7f5fee916b80 1 -- 172.21.15.37:0/2001736101 >> 172.21.15.37:0/2001736101 conn(0x55b82ed96cc0 msgr2=0x55b82ee6a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:20.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.264+0000 7f5fee916b80 1 -- 172.21.15.37:0/2001736101 shutdown_connections 2024-06-10T01:05:20.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.264+0000 7f5fee916b80 1 -- 172.21.15.37:0/2001736101 wait complete. 2024-06-10T01:05:20.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.264+0000 7f5fee916b80 20 client.16965 trim_cache size 0 max 16384 2024-06-10T01:05:20.263 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:05:20.264 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "scrub active (1 inodes in the stack)", 2024-06-10T01:05:20.264 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-06-10T01:05:20.264 INFO:teuthology.orchestra.run.smithi037.stdout: "d79c0c89-cbf2-47ac-9a2b-0a728f0b2713": { 2024-06-10T01:05:20.264 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-06-10T01:05:20.264 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "d79c0c89-cbf2-47ac-9a2b-0a728f0b2713", 2024-06-10T01:05:20.264 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-06-10T01:05:20.264 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:05:20.264 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:05:20.264 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:05:20.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.265+0000 7f5fe4093700 1 -- 172.21.15.37:0/4120391554 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5fc80a2ca0 msgr2=0x7f5fc80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:20.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.265+0000 7f5fe4093700 1 --2- 172.21.15.37:0/4120391554 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5fc80a2ca0 0x7f5fc80a5160 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc199ad0 tx=0x7f5fd800a380 comp rx=0 tx=0).stop 2024-06-10T01:05:20.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.265+0000 7f5fe4093700 1 -- 172.21.15.37:0/4120391554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fdc107840 msgr2=0x7f5fdc198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:20.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.265+0000 7f5fe4093700 1 --2- 172.21.15.37:0/4120391554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fdc107840 0x7f5fdc198990 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400ef30 tx=0x7f5fd400c3a0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.265+0000 7f5fe4093700 1 -- 172.21.15.37:0/4120391554 shutdown_connections 2024-06-10T01:05:20.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.265+0000 7f5fe4093700 1 --2- 172.21.15.37:0/4120391554 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5fc80a2ca0 0x7f5fc80a5160 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.265+0000 7f5fe4093700 1 --2- 172.21.15.37:0/4120391554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5fdc107840 0x7f5fdc198990 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.265+0000 7f5fe4093700 1 --2- 172.21.15.37:0/4120391554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fdc102740 0x7f5fdc19e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.265+0000 7f5fe4093700 1 --2- 172.21.15.37:0/4120391554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fdc1030f0 0x7f5fdc19ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:20.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.265+0000 7f5fe4093700 1 -- 172.21.15.37:0/4120391554 >> 172.21.15.37:0/4120391554 conn(0x7f5fdc0795f0 msgr2=0x7f5fdc100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:20.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.265+0000 7f5fe4093700 1 -- 172.21.15.37:0/4120391554 shutdown_connections 2024-06-10T01:05:20.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:20.265+0000 7f5fe4093700 1 -- 172.21.15.37:0/4120391554 wait complete. 2024-06-10T01:05:20.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:20 smithi167 ceph-mon[36917]: pgmap v690: 129 pgs: 129 active+clean; 36 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 1.98k op/s 2024-06-10T01:05:20.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:20 smithi094 ceph-mon[36708]: pgmap v690: 129 pgs: 129 active+clean; 36 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 1.98k op/s 2024-06-10T01:05:20.972 INFO:tasks.cephfs.filesystem:scrub status for tag:d79c0c89-cbf2-47ac-9a2b-0a728f0b2713 - {'path': '/', 'tag': 'd79c0c89-cbf2-47ac-9a2b-0a728f0b2713', 'options': 'recursive,force'} 2024-06-10T01:05:22.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:22 smithi094 ceph-mon[36708]: pgmap v691: 129 pgs: 129 active+clean; 36 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 2.10k op/s 2024-06-10T01:05:22.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:22 smithi037 ceph-mon[32192]: pgmap v691: 129 pgs: 129 active+clean; 36 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 2.10k op/s 2024-06-10T01:05:22.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:22 smithi167 ceph-mon[36917]: pgmap v691: 129 pgs: 129 active+clean; 36 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 2.10k op/s 2024-06-10T01:05:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:23.303+0000 7f65897fa640 1 -- 172.21.15.37:0/1486246607 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56548c561c20 con 0x56548c6721d0 2024-06-10T01:05:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:23.303+0000 7f65897fa640 10 client.26431 handle_command_reply: tid=0 2024-06-10T01:05:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:23.303+0000 7f65aac17740 1 client.26431 shutdown 2024-06-10T01:05:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:23.303+0000 7f65aac17740 1 -- 172.21.15.37:0/1486246607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56548c5f1a00 msgr2=0x56548c600180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:23.303+0000 7f65aac17740 1 --2- 172.21.15.37:0/1486246607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56548c5f1a00 0x56548c600180 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f659c009560 tx=0x7f659c009590 comp rx=0 tx=0).stop 2024-06-10T01:05:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:23.303+0000 7f65aac17740 1 -- 172.21.15.37:0/1486246607 shutdown_connections 2024-06-10T01:05:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:23.303+0000 7f65aac17740 1 --2- 172.21.15.37:0/1486246607 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56548c6721d0 0x56548c6745c0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:23.303+0000 7f65aac17740 1 --2- 172.21.15.37:0/1486246607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56548c5f6b70 0x56548c601140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:23.303+0000 7f65aac17740 1 --2- 172.21.15.37:0/1486246607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56548c5f1a00 0x56548c600180 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:23.303+0000 7f65aac17740 1 --2- 172.21.15.37:0/1486246607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56548c5f1010 0x56548c5ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:23.303+0000 7f65897fa640 0 client.26431 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:05:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:23.303+0000 7f65aac17740 1 -- 172.21.15.37:0/1486246607 >> 172.21.15.37:0/1486246607 conn(0x56548c5ec680 msgr2=0x56548c5ef590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:23.303+0000 7f65aac17740 1 -- 172.21.15.37:0/1486246607 shutdown_connections 2024-06-10T01:05:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:23.303+0000 7f65aac17740 1 -- 172.21.15.37:0/1486246607 wait complete. 2024-06-10T01:05:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:23.303+0000 7f65aac17740 20 client.26431 trim_cache size 0 max 16384 2024-06-10T01:05:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:05:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:05:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:05:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:23.304+0000 7f65a7f12640 1 -- 172.21.15.37:0/3498463740 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f658c0a2c30 msgr2=0x7f658c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:23.304+0000 7f65a7f12640 1 --2- 172.21.15.37:0/3498463740 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f658c0a2c30 0x7f658c0a50f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f65980049d0 tx=0x7f659806ef00 comp rx=0 tx=0).stop 2024-06-10T01:05:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:23.304+0000 7f65a7f12640 1 -- 172.21.15.37:0/3498463740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65a01057e0 msgr2=0x7f65a01a13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:23.304+0000 7f65a7f12640 1 --2- 172.21.15.37:0/3498463740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65a01057e0 0x7f65a01a13f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f659800eb10 tx=0x7f659800c2d0 comp rx=0 tx=0).stop 2024-06-10T01:05:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:23.304+0000 7f65a7f12640 1 -- 172.21.15.37:0/3498463740 shutdown_connections 2024-06-10T01:05:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:23.304+0000 7f65a7f12640 1 --2- 172.21.15.37:0/3498463740 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f658c0a2c30 0x7f658c0a50f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:23.304+0000 7f65a7f12640 1 --2- 172.21.15.37:0/3498463740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65a01057e0 0x7f65a01a13f0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:23.304+0000 7f65a7f12640 1 --2- 172.21.15.37:0/3498463740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f65a0104e20 0x7f65a019a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:23.304+0000 7f65a7f12640 1 --2- 172.21.15.37:0/3498463740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65a0103c30 0x7f65a0199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:23.304+0000 7f65a7f12640 1 -- 172.21.15.37:0/3498463740 >> 172.21.15.37:0/3498463740 conn(0x7f65a00ff1c0 msgr2=0x7f65a01080c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:23.304+0000 7f65a7f12640 1 -- 172.21.15.37:0/3498463740 shutdown_connections 2024-06-10T01:05:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:23.304+0000 7f65a7f12640 1 -- 172.21.15.37:0/3498463740 wait complete. 2024-06-10T01:05:24.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:24 smithi094 ceph-mon[36708]: pgmap v692: 129 pgs: 129 active+clean; 42 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 2.20k op/s 2024-06-10T01:05:24.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:24 smithi037 ceph-mon[32192]: pgmap v692: 129 pgs: 129 active+clean; 42 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 2.20k op/s 2024-06-10T01:05:24.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:24 smithi167 ceph-mon[36917]: pgmap v692: 129 pgs: 129 active+clean; 42 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 2.20k op/s 2024-06-10T01:05:26.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:26 smithi094 ceph-mon[36708]: pgmap v693: 129 pgs: 129 active+clean; 26 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 2.15k op/s 2024-06-10T01:05:26.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:26 smithi037 ceph-mon[32192]: pgmap v693: 129 pgs: 129 active+clean; 26 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 2.15k op/s 2024-06-10T01:05:26.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:26 smithi167 ceph-mon[36917]: pgmap v693: 129 pgs: 129 active+clean; 26 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 2.15k op/s 2024-06-10T01:05:28.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:28 smithi094 ceph-mon[36708]: pgmap v694: 129 pgs: 129 active+clean; 26 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 1.99k op/s 2024-06-10T01:05:28.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:28 smithi037 ceph-mon[32192]: pgmap v694: 129 pgs: 129 active+clean; 26 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 1.99k op/s 2024-06-10T01:05:28.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:28 smithi167 ceph-mon[36917]: pgmap v694: 129 pgs: 129 active+clean; 26 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 1.99k op/s 2024-06-10T01:05:30.361 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:30 smithi094 ceph-mon[36708]: pgmap v695: 129 pgs: 129 active+clean; 36 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 1.91k op/s 2024-06-10T01:05:30.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:30 smithi037 ceph-mon[32192]: pgmap v695: 129 pgs: 129 active+clean; 36 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 1.91k op/s 2024-06-10T01:05:30.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:30 smithi167 ceph-mon[36917]: pgmap v695: 129 pgs: 129 active+clean; 36 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 1.91k op/s 2024-06-10T01:05:32.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:32 smithi094 ceph-mon[36708]: pgmap v696: 129 pgs: 129 active+clean; 36 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 529 op/s 2024-06-10T01:05:32.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:32 smithi037 ceph-mon[32192]: pgmap v696: 129 pgs: 129 active+clean; 36 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 529 op/s 2024-06-10T01:05:32.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:32 smithi167 ceph-mon[36917]: pgmap v696: 129 pgs: 129 active+clean; 36 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 529 op/s 2024-06-10T01:05:34.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:34 smithi094 ceph-mon[36708]: pgmap v697: 129 pgs: 129 active+clean; 45 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 357 op/s 2024-06-10T01:05:34.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:34 smithi037 ceph-mon[32192]: pgmap v697: 129 pgs: 129 active+clean; 45 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 357 op/s 2024-06-10T01:05:34.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:34 smithi167 ceph-mon[36917]: pgmap v697: 129 pgs: 129 active+clean; 45 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 357 op/s 2024-06-10T01:05:36.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:36 smithi094 ceph-mon[36708]: pgmap v698: 129 pgs: 129 active+clean; 49 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 271 op/s 2024-06-10T01:05:36.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:36 smithi037 ceph-mon[32192]: pgmap v698: 129 pgs: 129 active+clean; 49 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 271 op/s 2024-06-10T01:05:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.396+0000 7f92b23a5640 1 -- 172.21.15.37:0/638050675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ac100900 msgr2=0x7f92ac100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.396+0000 7f92b23a5640 1 --2- 172.21.15.37:0/638050675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ac100900 0x7f92ac100d20 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f929c009900 tx=0x7f929c02a410 comp rx=0 tx=0).stop 2024-06-10T01:05:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.396+0000 7f92b23a5640 1 -- 172.21.15.37:0/638050675 shutdown_connections 2024-06-10T01:05:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.396+0000 7f92b23a5640 1 --2- 172.21.15.37:0/638050675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92ac1024b0 0x7f92ac108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.396+0000 7f92b23a5640 1 --2- 172.21.15.37:0/638050675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92ac101af0 0x7f92ac101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.396+0000 7f92b23a5640 1 --2- 172.21.15.37:0/638050675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ac100900 0x7f92ac100d20 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.396+0000 7f92b23a5640 1 -- 172.21.15.37:0/638050675 >> 172.21.15.37:0/638050675 conn(0x7f92ac0fbe90 msgr2=0x7f92ac0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.396+0000 7f92b23a5640 1 -- 172.21.15.37:0/638050675 shutdown_connections 2024-06-10T01:05:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.396+0000 7f92b23a5640 1 -- 172.21.15.37:0/638050675 wait complete. 2024-06-10T01:05:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.397+0000 7f92b23a5640 1 Processor -- start 2024-06-10T01:05:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.397+0000 7f92b23a5640 1 -- start start 2024-06-10T01:05:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.397+0000 7f92b23a5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ac100900 0x7f92ac06ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.397+0000 7f92b23a5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92ac101af0 0x7f92ac19c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.397+0000 7f92b23a5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92ac1024b0 0x7f92ac19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.397+0000 7f92b23a5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac10c580 con 0x7f92ac100900 2024-06-10T01:05:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.397+0000 7f92b23a5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac10c400 con 0x7f92ac1024b0 2024-06-10T01:05:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.397+0000 7f92b23a5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac10c700 con 0x7f92ac101af0 2024-06-10T01:05:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.397+0000 7f92abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ac100900 0x7f92ac06ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.397+0000 7f92abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ac100900 0x7f92ac06ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51284/0 (socket says 172.21.15.37:51284) 2024-06-10T01:05:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.397+0000 7f92abfff640 1 -- 172.21.15.37:0/653007281 learned_addr learned my addr 172.21.15.37:0/653007281 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.397+0000 7f92ab7fe640 1 --2- 172.21.15.37:0/653007281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92ac101af0 0x7f92ac19c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.397+0000 7f92b0942640 1 --2- 172.21.15.37:0/653007281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92ac1024b0 0x7f92ac19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.398+0000 7f92b0942640 1 -- 172.21.15.37:0/653007281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92ac101af0 msgr2=0x7f92ac19c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.398+0000 7f92b0942640 1 --2- 172.21.15.37:0/653007281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92ac101af0 0x7f92ac19c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.398+0000 7f92b0942640 1 -- 172.21.15.37:0/653007281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ac100900 msgr2=0x7f92ac06ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.398+0000 7f92b0942640 1 --2- 172.21.15.37:0/653007281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ac100900 0x7f92ac06ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.398+0000 7f92b0942640 1 -- 172.21.15.37:0/653007281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92ac19f860 con 0x7f92ac1024b0 2024-06-10T01:05:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.398+0000 7f92ab7fe640 1 --2- 172.21.15.37:0/653007281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92ac101af0 0x7f92ac19c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.398+0000 7f92abfff640 1 --2- 172.21.15.37:0/653007281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ac100900 0x7f92ac06ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.398+0000 7f92b0942640 1 --2- 172.21.15.37:0/653007281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92ac1024b0 0x7f92ac19efa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f92a4009fd0 tx=0x7f92a400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.398+0000 7f92a97fa640 1 -- 172.21.15.37:0/653007281 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a400ce20 con 0x7f92ac1024b0 2024-06-10T01:05:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.398+0000 7f92b23a5640 1 -- 172.21.15.37:0/653007281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92ac19fb50 con 0x7f92ac1024b0 2024-06-10T01:05:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.398+0000 7f92a97fa640 1 -- 172.21.15.37:0/653007281 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92a4005020 con 0x7f92ac1024b0 2024-06-10T01:05:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.398+0000 7f92a97fa640 1 -- 172.21.15.37:0/653007281 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a4010720 con 0x7f92ac1024b0 2024-06-10T01:05:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.398+0000 7f92b23a5640 1 -- 172.21.15.37:0/653007281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92ac1a00f0 con 0x7f92ac1024b0 2024-06-10T01:05:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.401+0000 7f92a97fa640 1 -- 172.21.15.37:0/653007281 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f92a4005540 con 0x7f92ac1024b0 2024-06-10T01:05:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.401+0000 7f92b518c740 1 Processor -- start 2024-06-10T01:05:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.401+0000 7f92b518c740 1 -- start start 2024-06-10T01:05:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.401+0000 7f92b518c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1faa04420 0x55a1faa04840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.401+0000 7f92b518c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1faa04e10 0x55a1faa09910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.401+0000 7f92b518c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1faa09f80 0x55a1faa0c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.401+0000 7f92b518c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a1fa9634f0 con 0x55a1faa04e10 2024-06-10T01:05:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.401+0000 7f92b518c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a1fa901870 con 0x55a1faa04420 2024-06-10T01:05:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.402+0000 7f92b518c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a1fa95f4b0 con 0x55a1faa09f80 2024-06-10T01:05:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.402+0000 7f92abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1faa04e10 0x55a1faa09910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.402+0000 7f92ab7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1faa04420 0x55a1faa04840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.402+0000 7f92abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1faa04e10 0x55a1faa09910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51298/0 (socket says 172.21.15.37:51298) 2024-06-10T01:05:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.402+0000 7f92abfff640 1 -- 172.21.15.37:0/2165933985 learned_addr learned my addr 172.21.15.37:0/2165933985 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.402+0000 7f92ab7fe640 1 --2- 172.21.15.37:0/2165933985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1faa09f80 0x55a1faa0c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.402+0000 7f92abfff640 1 -- 172.21.15.37:0/2165933985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1faa09f80 msgr2=0x55a1faa0c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.402+0000 7f92abfff640 1 --2- 172.21.15.37:0/2165933985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1faa09f80 0x55a1faa0c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.402+0000 7f92abfff640 1 -- 172.21.15.37:0/2165933985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1faa04420 msgr2=0x55a1faa04840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.402+0000 7f92abfff640 1 --2- 172.21.15.37:0/2165933985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1faa04420 0x55a1faa04840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.402+0000 7f92abfff640 1 -- 172.21.15.37:0/2165933985 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1fa91d9b0 con 0x55a1faa04e10 2024-06-10T01:05:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.402+0000 7f92a97fa640 1 --2- 172.21.15.37:0/653007281 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f92940a2c30 0x7f92940a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.402+0000 7f92ab7fe640 1 --2- 172.21.15.37:0/653007281 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f92940a2c30 0x7f92940a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.402+0000 7f92a97fa640 1 -- 172.21.15.37:0/653007281 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f92a40cfa40 con 0x7f92ac1024b0 2024-06-10T01:05:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.402+0000 7f92abfff640 1 --2- 172.21.15.37:0/2165933985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1faa04e10 0x55a1faa09910 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f929c031440 tx=0x7f929c005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1c0f97fa17fd8f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.402+0000 7f92927fc640 1 -- 172.21.15.37:0/2165933985 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c035040 con 0x55a1faa04e10 2024-06-10T01:05:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.403+0000 7f92927fc640 1 -- 172.21.15.37:0/2165933985 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f929c0318b0 con 0x55a1faa04e10 2024-06-10T01:05:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.403+0000 7f92927fc640 1 -- 172.21.15.37:0/2165933985 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c0049a0 con 0x55a1faa04e10 2024-06-10T01:05:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.403+0000 7f92b518c740 1 -- 172.21.15.37:0/2165933985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1faa04e10 msgr2=0x55a1faa09910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.403+0000 7f92ab7fe640 1 --2- 172.21.15.37:0/653007281 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f92940a2c30 0x7f92940a50f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x55a1faa05670 tx=0x7f92a0005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.403+0000 7f92b518c740 1 --2- 172.21.15.37:0/2165933985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1faa04e10 0x55a1faa09910 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f929c031440 tx=0x7f929c005de0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.403+0000 7f92b518c740 1 -- 172.21.15.37:0/2165933985 shutdown_connections 2024-06-10T01:05:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.403+0000 7f92b518c740 1 --2- 172.21.15.37:0/2165933985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1faa09f80 0x55a1faa0c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.403+0000 7f92b518c740 1 --2- 172.21.15.37:0/2165933985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1faa04e10 0x55a1faa09910 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.403+0000 7f92b518c740 1 --2- 172.21.15.37:0/2165933985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1faa04420 0x55a1faa04840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.403+0000 7f92b518c740 1 -- 172.21.15.37:0/2165933985 >> 172.21.15.37:0/2165933985 conn(0x55a1fa9fb680 msgr2=0x55a1fa9fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.403+0000 7f92b518c740 1 -- 172.21.15.37:0/2165933985 shutdown_connections 2024-06-10T01:05:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.403+0000 7f92b518c740 1 -- 172.21.15.37:0/2165933985 wait complete. 2024-06-10T01:05:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.404+0000 7f92b518c740 1 -- start start 2024-06-10T01:05:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.404+0000 7f92b518c740 1 Processor -- start 2024-06-10T01:05:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.404+0000 7f92b518c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1faa04420 0x55a1faa0e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.404+0000 7f92b518c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1faa04e10 0x55a1faa0eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.404+0000 7f92b518c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1faa09f80 0x55a1faa15b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.404+0000 7f92b518c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a1faa0ced0 con 0x55a1faa09f80 2024-06-10T01:05:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.404+0000 7f92b518c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a1fa8ecae0 con 0x55a1faa04420 2024-06-10T01:05:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.404+0000 7f92b518c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a1faa0d050 con 0x55a1faa04e10 2024-06-10T01:05:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.404+0000 7f92b518c740 10 client.0 _ll_register_callbacks cb 0x55a1fa9259b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:05:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.404+0000 7f92b0942640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1faa09f80 0x55a1faa15b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.404+0000 7f92b0942640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1faa09f80 0x55a1faa15b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51304/0 (socket says 172.21.15.37:51304) 2024-06-10T01:05:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.404+0000 7f92abfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1faa04420 0x55a1faa0e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.404+0000 7f92b0942640 1 -- 172.21.15.37:0/167260253 learned_addr learned my addr 172.21.15.37:0/167260253 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.404+0000 7f92ab7fe640 1 --2- 172.21.15.37:0/167260253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1faa04e10 0x55a1faa0eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.405+0000 7f92abfff640 1 -- 172.21.15.37:0/167260253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1faa04e10 msgr2=0x55a1faa0eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.405+0000 7f92abfff640 1 --2- 172.21.15.37:0/167260253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1faa04e10 0x55a1faa0eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.405+0000 7f92abfff640 1 -- 172.21.15.37:0/167260253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1faa09f80 msgr2=0x55a1faa15b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.405+0000 7f92abfff640 1 --2- 172.21.15.37:0/167260253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1faa09f80 0x55a1faa15b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.405+0000 7f92abfff640 1 -- 172.21.15.37:0/167260253 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1fa677210 con 0x55a1faa04420 2024-06-10T01:05:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.405+0000 7f92b0942640 1 --2- 172.21.15.37:0/167260253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1faa09f80 0x55a1faa15b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.405+0000 7f92abfff640 1 --2- 172.21.15.37:0/167260253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1faa04420 0x55a1faa0e5d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x55a1fa9fc550 tx=0x7f929c0042d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64bb890b95582f65 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.405+0000 7f928b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:05:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.405+0000 7f928b7fe640 1 -- 172.21.15.37:0/167260253 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c04e020 con 0x55a1faa04420 2024-06-10T01:05:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.405+0000 7f92b518c740 1 -- 172.21.15.37:0/167260253 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f929c0318b0 con 0x55a1faa04420 2024-06-10T01:05:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.405+0000 7f928b7fe640 1 -- 172.21.15.37:0/167260253 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f929c003800 con 0x55a1faa04420 2024-06-10T01:05:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.405+0000 7f928b7fe640 1 -- 172.21.15.37:0/167260253 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f929c059050 con 0x55a1faa04420 2024-06-10T01:05:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.405+0000 7f92b518c740 10 client.26381 fetch_fsmap learned FSMap version 29 2024-06-10T01:05:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.405+0000 7f92b518c740 1 -- 172.21.15.37:0/167260253 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a1fa743620 con 0x55a1faa04420 2024-06-10T01:05:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.406+0000 7f928b7fe640 1 -- 172.21.15.37:0/167260253 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f929c0318b0 con 0x55a1faa04420 2024-06-10T01:05:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.406+0000 7f928b7fe640 1 -- 172.21.15.37:0/167260253 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f929c004b30 con 0x55a1faa04420 2024-06-10T01:05:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.406+0000 7f92b518c740 10 client.26381 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:05:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.406+0000 7f92b518c740 10 client.26381 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:05:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.406+0000 7f92b518c740 20 client.26381 populate_metadata read hostname 'smithi037' 2024-06-10T01:05:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.406+0000 7f92b518c740 1 --2- 172.21.15.37:0/167260253 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a1faa1a040 0x55a1faa1c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.406+0000 7f92b518c740 4 client.26381 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:05:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.406+0000 7f92b518c740 1 -- 172.21.15.37:0/167260253 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1fa976f10 con 0x55a1faa1a040 2024-06-10T01:05:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.406+0000 7f92ab7fe640 1 --2- 172.21.15.37:0/167260253 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a1faa1a040 0x55a1faa1c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.406+0000 7f92ab7fe640 1 --2- 172.21.15.37:0/167260253 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a1faa1a040 0x55a1faa1c430 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2ea17d141de7fe5 server_cookie=5a0936ac3da57651 in_seq=0 out_seq=0 2024-06-10T01:05:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.406+0000 7f928b7fe640 10 client.26381 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:05:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.410+0000 7f928b7fe640 1 -- 172.21.15.37:0/167260253 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a1fa976f10 con 0x55a1faa1a040 2024-06-10T01:05:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.410+0000 7f928b7fe640 10 client.26381 handle_command_reply: tid=0 2024-06-10T01:05:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.410+0000 7f92b518c740 1 client.26381 shutdown 2024-06-10T01:05:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.410+0000 7f92b518c740 1 -- 172.21.15.37:0/167260253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1faa04420 msgr2=0x55a1faa0e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.410+0000 7f92b518c740 1 --2- 172.21.15.37:0/167260253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1faa04420 0x55a1faa0e5d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x55a1fa9fc550 tx=0x7f929c0042d0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.410+0000 7f92b518c740 1 -- 172.21.15.37:0/167260253 shutdown_connections 2024-06-10T01:05:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.410+0000 7f92b518c740 1 --2- 172.21.15.37:0/167260253 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a1faa1a040 0x55a1faa1c430 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.410+0000 7f92b518c740 1 --2- 172.21.15.37:0/167260253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1faa09f80 0x55a1faa15b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.410+0000 7f92b518c740 1 --2- 172.21.15.37:0/167260253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1faa04e10 0x55a1faa0eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.410+0000 7f92b518c740 1 --2- 172.21.15.37:0/167260253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1faa04420 0x55a1faa0e5d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.410+0000 7f928b7fe640 0 client.26381 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:05:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.410+0000 7f92b518c740 1 -- 172.21.15.37:0/167260253 >> 172.21.15.37:0/167260253 conn(0x55a1fa9fb680 msgr2=0x55a1fa9fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.410+0000 7f92b518c740 1 -- 172.21.15.37:0/167260253 shutdown_connections 2024-06-10T01:05:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.410+0000 7f92b518c740 1 -- 172.21.15.37:0/167260253 wait complete. 2024-06-10T01:05:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.410+0000 7f92b518c740 20 client.26381 trim_cache size 0 max 16384 2024-06-10T01:05:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.425+0000 7f92b518c740 1 Processor -- start 2024-06-10T01:05:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.425+0000 7f92b518c740 1 -- start start 2024-06-10T01:05:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.426+0000 7f92b518c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1faa04420 0x55a1fa9fe590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.426+0000 7f92b518c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1faa04e10 0x55a1faa15a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.426+0000 7f92b518c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1faa09f80 0x55a1fa9fead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.426+0000 7f92b518c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac1aada0 con 0x55a1faa09f80 2024-06-10T01:05:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.426+0000 7f92b518c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac1a8670 con 0x55a1faa04e10 2024-06-10T01:05:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.426+0000 7f92b518c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac1a64c0 con 0x55a1faa04420 2024-06-10T01:05:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.426+0000 7f92b0942640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1faa09f80 0x55a1fa9fead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.426+0000 7f92b0942640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1faa09f80 0x55a1fa9fead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51314/0 (socket says 172.21.15.37:51314) 2024-06-10T01:05:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.426+0000 7f92b0942640 1 -- 172.21.15.37:0/3345865160 learned_addr learned my addr 172.21.15.37:0/3345865160 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.426+0000 7f92abfff640 1 --2- 172.21.15.37:0/3345865160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1faa04420 0x55a1fa9fe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.426+0000 7f92ab7fe640 1 --2- 172.21.15.37:0/3345865160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1faa04e10 0x55a1faa15a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.426+0000 7f92b0942640 1 -- 172.21.15.37:0/3345865160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1faa04420 msgr2=0x55a1fa9fe590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.426+0000 7f92b0942640 1 --2- 172.21.15.37:0/3345865160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1faa04420 0x55a1fa9fe590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.426+0000 7f92b0942640 1 -- 172.21.15.37:0/3345865160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1faa04e10 msgr2=0x55a1faa15a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.426+0000 7f92b0942640 1 --2- 172.21.15.37:0/3345865160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1faa04e10 0x55a1faa15a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.426+0000 7f92b0942640 1 -- 172.21.15.37:0/3345865160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1faa1c9c0 con 0x55a1faa09f80 2024-06-10T01:05:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.426+0000 7f92b0942640 1 --2- 172.21.15.37:0/3345865160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1faa09f80 0x55a1fa9fead0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f92a40cd4c0 tx=0x7f92a4005940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2470789c18379d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.426+0000 7f9291ffb640 1 -- 172.21.15.37:0/3345865160 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a40a0260 con 0x55a1faa09f80 2024-06-10T01:05:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.426+0000 7f9291ffb640 1 -- 172.21.15.37:0/3345865160 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92a40a4b20 con 0x55a1faa09f80 2024-06-10T01:05:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.426+0000 7f9291ffb640 1 -- 172.21.15.37:0/3345865160 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a409b300 con 0x55a1faa09f80 2024-06-10T01:05:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.427+0000 7f92b518c740 1 -- 172.21.15.37:0/3345865160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1faa09f80 msgr2=0x55a1fa9fead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.427+0000 7f92b518c740 1 --2- 172.21.15.37:0/3345865160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1faa09f80 0x55a1fa9fead0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f92a40cd4c0 tx=0x7f92a4005940 comp rx=0 tx=0).stop 2024-06-10T01:05:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.427+0000 7f92b518c740 1 -- 172.21.15.37:0/3345865160 shutdown_connections 2024-06-10T01:05:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.427+0000 7f92b518c740 1 --2- 172.21.15.37:0/3345865160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1faa09f80 0x55a1fa9fead0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.427+0000 7f92b518c740 1 --2- 172.21.15.37:0/3345865160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1faa04e10 0x55a1faa15a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.427+0000 7f92b518c740 1 --2- 172.21.15.37:0/3345865160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1faa04420 0x55a1fa9fe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.427+0000 7f92b518c740 1 -- 172.21.15.37:0/3345865160 >> 172.21.15.37:0/3345865160 conn(0x55a1fa9fb680 msgr2=0x55a1faa09620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.427+0000 7f92b518c740 1 -- 172.21.15.37:0/3345865160 shutdown_connections 2024-06-10T01:05:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.427+0000 7f92b518c740 1 -- 172.21.15.37:0/3345865160 wait complete. 2024-06-10T01:05:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.427+0000 7f92b518c740 1 -- start start 2024-06-10T01:05:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.428+0000 7f92b518c740 1 Processor -- start 2024-06-10T01:05:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.428+0000 7f92b518c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1faa04420 0x55a1faa12820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.428+0000 7f92b518c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1faa04e10 0x55a1faa12d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.428+0000 7f92b518c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1faa09f80 0x55a1faa82c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.428+0000 7f92b518c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac1a6b30 con 0x55a1faa09f80 2024-06-10T01:05:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.428+0000 7f92b518c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac1a7fa0 con 0x55a1faa04e10 2024-06-10T01:05:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.428+0000 7f92b518c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac1a7200 con 0x55a1faa04420 2024-06-10T01:05:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.428+0000 7f92b518c740 10 client.0 _ll_register_callbacks cb 0x7f929c040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:05:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.428+0000 7f92b0942640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1faa09f80 0x55a1faa82c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.428+0000 7f92b0942640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1faa09f80 0x55a1faa82c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51330/0 (socket says 172.21.15.37:51330) 2024-06-10T01:05:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.428+0000 7f92abfff640 1 --2- 172.21.15.37:0/2425971198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1faa04420 0x55a1faa12820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.428+0000 7f92b0942640 1 -- 172.21.15.37:0/2425971198 learned_addr learned my addr 172.21.15.37:0/2425971198 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.428+0000 7f92ab7fe640 1 --2- 172.21.15.37:0/2425971198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1faa04e10 0x55a1faa12d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.428+0000 7f92b0942640 1 -- 172.21.15.37:0/2425971198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1faa04420 msgr2=0x55a1faa12820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.428+0000 7f92b0942640 1 --2- 172.21.15.37:0/2425971198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1faa04420 0x55a1faa12820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.428+0000 7f92b0942640 1 -- 172.21.15.37:0/2425971198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1faa04e10 msgr2=0x55a1faa12d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.428+0000 7f92b0942640 1 --2- 172.21.15.37:0/2425971198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1faa04e10 0x55a1faa12d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.428+0000 7f92b0942640 1 -- 172.21.15.37:0/2425971198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1fa9fc350 con 0x55a1faa09f80 2024-06-10T01:05:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.428+0000 7f92ab7fe640 1 --2- 172.21.15.37:0/2425971198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1faa04e10 0x55a1faa12d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.429+0000 7f92b0942640 1 --2- 172.21.15.37:0/2425971198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1faa09f80 0x55a1faa82c10 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f92a40a45f0 tx=0x7f92a409a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2137a4a59356edb6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.429+0000 7f928b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:05:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.429+0000 7f928b7fe640 1 -- 172.21.15.37:0/2425971198 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a407e070 con 0x55a1faa09f80 2024-06-10T01:05:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.429+0000 7f928b7fe640 1 -- 172.21.15.37:0/2425971198 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f92a40850b0 con 0x55a1faa09f80 2024-06-10T01:05:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.429+0000 7f92b518c740 1 -- 172.21.15.37:0/2425971198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92a40a4b20 con 0x55a1faa09f80 2024-06-10T01:05:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.429+0000 7f928b7fe640 1 -- 172.21.15.37:0/2425971198 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f92a409b300 con 0x55a1faa09f80 2024-06-10T01:05:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.429+0000 7f92b518c740 10 client.16983 fetch_fsmap learned FSMap version 29 2024-06-10T01:05:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.429+0000 7f92b518c740 1 -- 172.21.15.37:0/2425971198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a1fa9fb4b0 con 0x55a1faa09f80 2024-06-10T01:05:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.429+0000 7f928b7fe640 1 -- 172.21.15.37:0/2425971198 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92a40a4b20 con 0x55a1faa09f80 2024-06-10T01:05:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.429+0000 7f928b7fe640 1 -- 172.21.15.37:0/2425971198 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f92a40ab050 con 0x55a1faa09f80 2024-06-10T01:05:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.429+0000 7f92b518c740 10 client.16983 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:05:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.429+0000 7f92b518c740 10 client.16983 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:05:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.429+0000 7f92b518c740 20 client.16983 populate_metadata read hostname 'smithi037' 2024-06-10T01:05:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.429+0000 7f92b518c740 1 --2- 172.21.15.37:0/2425971198 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a1faa84850 0x55a1faa86c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.429+0000 7f92b518c740 4 client.16983 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:05:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.429+0000 7f92b518c740 1 -- 172.21.15.37:0/2425971198 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a1fa970c20 con 0x55a1faa84850 2024-06-10T01:05:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.430+0000 7f92abfff640 1 --2- 172.21.15.37:0/2425971198 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a1faa84850 0x55a1faa86c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.430+0000 7f92abfff640 1 --2- 172.21.15.37:0/2425971198 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a1faa84850 0x55a1faa86c70 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bf5d1f2df8074c3 server_cookie=d5233ae6a358b42 in_seq=0 out_seq=0 2024-06-10T01:05:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:36.430+0000 7f928b7fe640 10 client.16983 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:05:36.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:36 smithi167 ceph-mon[36917]: pgmap v698: 129 pgs: 129 active+clean; 49 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 271 op/s 2024-06-10T01:05:38.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:38 smithi167 ceph-mon[36917]: pgmap v699: 129 pgs: 129 active+clean; 49 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 228 op/s 2024-06-10T01:05:38.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:38 smithi037 ceph-mon[32192]: pgmap v699: 129 pgs: 129 active+clean; 49 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 228 op/s 2024-06-10T01:05:38.423 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:38 smithi094 ceph-mon[36708]: pgmap v699: 129 pgs: 129 active+clean; 49 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 228 op/s 2024-06-10T01:05:40.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:40 smithi037 ceph-mon[32192]: pgmap v700: 129 pgs: 129 active+clean; 54 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 338 op/s 2024-06-10T01:05:40.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:40 smithi167 ceph-mon[36917]: pgmap v700: 129 pgs: 129 active+clean; 54 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 338 op/s 2024-06-10T01:05:40.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:40 smithi094 ceph-mon[36708]: pgmap v700: 129 pgs: 129 active+clean; 54 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 338 op/s 2024-06-10T01:05:42.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:42 smithi037 ceph-mon[32192]: pgmap v701: 129 pgs: 129 active+clean; 54 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 287 op/s 2024-06-10T01:05:42.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:42 smithi167 ceph-mon[36917]: pgmap v701: 129 pgs: 129 active+clean; 54 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 287 op/s 2024-06-10T01:05:42.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:42 smithi094 ceph-mon[36708]: pgmap v701: 129 pgs: 129 active+clean; 54 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 287 op/s 2024-06-10T01:05:44.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:44 smithi037 ceph-mon[32192]: pgmap v702: 129 pgs: 129 active+clean; 57 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 318 op/s 2024-06-10T01:05:44.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:44 smithi167 ceph-mon[36917]: pgmap v702: 129 pgs: 129 active+clean; 57 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 318 op/s 2024-06-10T01:05:44.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:44 smithi094 ceph-mon[36708]: pgmap v702: 129 pgs: 129 active+clean; 57 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 318 op/s 2024-06-10T01:05:46.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:46 smithi037 ceph-mon[32192]: pgmap v703: 129 pgs: 129 active+clean; 63 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 371 op/s 2024-06-10T01:05:46.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:46 smithi167 ceph-mon[36917]: pgmap v703: 129 pgs: 129 active+clean; 63 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 371 op/s 2024-06-10T01:05:46.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:46 smithi094 ceph-mon[36708]: pgmap v703: 129 pgs: 129 active+clean; 63 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 371 op/s 2024-06-10T01:05:48.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:48 smithi037 ceph-mon[32192]: pgmap v704: 129 pgs: 129 active+clean; 63 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 436 op/s 2024-06-10T01:05:48.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:48 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:05:48.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:48 smithi167 ceph-mon[36917]: pgmap v704: 129 pgs: 129 active+clean; 63 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 436 op/s 2024-06-10T01:05:48.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:48 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:05:48.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:48 smithi094 ceph-mon[36708]: pgmap v704: 129 pgs: 129 active+clean; 63 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 436 op/s 2024-06-10T01:05:48.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:48 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:05:50.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:50 smithi167 ceph-mon[36917]: pgmap v705: 129 pgs: 129 active+clean; 69 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 1.09k op/s 2024-06-10T01:05:50.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:50 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:05:50.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:50 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:05:50.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:50 smithi094 ceph-mon[36708]: pgmap v705: 129 pgs: 129 active+clean; 69 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 1.09k op/s 2024-06-10T01:05:50.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:50 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:05:50.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:50 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:05:50.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:50 smithi037 ceph-mon[32192]: pgmap v705: 129 pgs: 129 active+clean; 69 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 1.09k op/s 2024-06-10T01:05:50.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:50 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:05:50.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:50 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:05:50.973 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:05:51.231 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:05:51.355 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:51 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:05:51.355 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:51 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:05:51.355 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:51 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:05:51.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:51 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:05:51.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:51 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:05:51.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:51 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:05:51.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:51 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:05:51.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:51 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:05:51.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:51 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:05:52.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:52 smithi037 ceph-mon[32192]: pgmap v706: 129 pgs: 129 active+clean; 69 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 1.07k op/s 2024-06-10T01:05:52.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:52 smithi167 ceph-mon[36917]: pgmap v706: 129 pgs: 129 active+clean; 69 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 1.07k op/s 2024-06-10T01:05:52.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:52 smithi094 ceph-mon[36708]: pgmap v706: 129 pgs: 129 active+clean; 69 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 1.07k op/s 2024-06-10T01:05:53.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.270+0000 7f5f7c3e2700 1 -- 172.21.15.37:0/1254251929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f74101390 msgr2=0x7f5f741017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:53.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.270+0000 7f5f7c3e2700 1 --2- 172.21.15.37:0/1254251929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f74101390 0x7f5f741017f0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009990 tx=0x7f5f64009ca0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.270+0000 7f5f7c3e2700 1 -- 172.21.15.37:0/1254251929 shutdown_connections 2024-06-10T01:05:53.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.270+0000 7f5f7c3e2700 1 --2- 172.21.15.37:0/1254251929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f74101d30 0x7f5f7410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.270+0000 7f5f7c3e2700 1 --2- 172.21.15.37:0/1254251929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f741073d0 0x7f5f741077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.270+0000 7f5f7c3e2700 1 --2- 172.21.15.37:0/1254251929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f74101390 0x7f5f741017f0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.270+0000 7f5f7c3e2700 1 -- 172.21.15.37:0/1254251929 >> 172.21.15.37:0/1254251929 conn(0x7f5f740782c0 msgr2=0x7f5f740786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:53.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.270+0000 7f5f7c3e2700 1 -- 172.21.15.37:0/1254251929 shutdown_connections 2024-06-10T01:05:53.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.270+0000 7f5f7c3e2700 1 -- 172.21.15.37:0/1254251929 wait complete. 2024-06-10T01:05:53.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.271+0000 7f5f7c3e2700 1 Processor -- start 2024-06-10T01:05:53.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.271+0000 7f5f7c3e2700 1 -- start start 2024-06-10T01:05:53.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.271+0000 7f5f7c3e2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f74101390 0x7f5f7419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:53.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.271+0000 7f5f7c3e2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f74101d30 0x7f5f7419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:53.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.271+0000 7f5f7c3e2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f741073d0 0x7f5f7419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:53.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.271+0000 7f5f7c3e2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5f74111650 con 0x7f5f74101d30 2024-06-10T01:05:53.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.271+0000 7f5f7c3e2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741114d0 con 0x7f5f74101390 2024-06-10T01:05:53.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.271+0000 7f5f7c3e2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741117d0 con 0x7f5f741073d0 2024-06-10T01:05:53.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.271+0000 7f5f7a17e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f74101390 0x7f5f7419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:53.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.271+0000 7f5f7997d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f74101d30 0x7f5f7419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:53.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.271+0000 7f5f7a97f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f741073d0 0x7f5f7419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:53.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.271+0000 7f5f7a17e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f74101390 0x7f5f7419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:56176/0 (socket says 172.21.15.37:56176) 2024-06-10T01:05:53.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.271+0000 7f5f7a17e700 1 -- 172.21.15.37:0/890295821 learned_addr learned my addr 172.21.15.37:0/890295821 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:53.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.272+0000 7f5f7a97f700 1 -- 172.21.15.37:0/890295821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f74101390 msgr2=0x7f5f7419adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:53.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.272+0000 7f5f7a97f700 1 --2- 172.21.15.37:0/890295821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f74101390 0x7f5f7419adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.272+0000 7f5f7a97f700 1 -- 172.21.15.37:0/890295821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f74101d30 msgr2=0x7f5f7419b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:53.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.272+0000 7f5f7a97f700 1 --2- 172.21.15.37:0/890295821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f74101d30 0x7f5f7419b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.272+0000 7f5f7a97f700 1 -- 172.21.15.37:0/890295821 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f7419fe00 con 0x7f5f741073d0 2024-06-10T01:05:53.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.272+0000 7f5f7997d700 1 --2- 172.21.15.37:0/890295821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f74101d30 0x7f5f7419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:53.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.272+0000 7f5f7a97f700 1 --2- 172.21.15.37:0/890295821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f741073d0 0x7f5f7419f6c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00efc0 tx=0x7f5f6c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:53.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.272+0000 7f5f6b7fe700 1 -- 172.21.15.37:0/890295821 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c0092e0 con 0x7f5f741073d0 2024-06-10T01:05:53.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.272+0000 7f5f6b7fe700 1 -- 172.21.15.37:0/890295821 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f6c004bb0 con 0x7f5f741073d0 2024-06-10T01:05:53.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.272+0000 7f5f7c3e2700 1 -- 172.21.15.37:0/890295821 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f741a0090 con 0x7f5f741073d0 2024-06-10T01:05:53.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.272+0000 7f5f6b7fe700 1 -- 172.21.15.37:0/890295821 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c004280 con 0x7f5f741073d0 2024-06-10T01:05:53.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.272+0000 7f5f7c3e2700 1 -- 172.21.15.37:0/890295821 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f741a7970 con 0x7f5f741073d0 2024-06-10T01:05:53.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.274+0000 7f5f6b7fe700 1 -- 172.21.15.37:0/890295821 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5f6c01b030 con 0x7f5f741073d0 2024-06-10T01:05:53.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.275+0000 7f5f6b7fe700 1 --2- 172.21.15.37:0/890295821 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5f600a2ca0 0x7f5f600a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:53.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.275+0000 7f5f6b7fe700 1 -- 172.21.15.37:0/890295821 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f5f6c0ceef0 con 0x7f5f741073d0 2024-06-10T01:05:53.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.275+0000 7f5f7a17e700 1 --2- 172.21.15.37:0/890295821 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5f600a2ca0 0x7f5f600a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:53.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.276+0000 7f5f86c65b80 1 Processor -- start 2024-06-10T01:05:53.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.276+0000 7f5f7a17e700 1 --2- 172.21.15.37:0/890295821 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5f600a2ca0 0x7f5f600a5160 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5f70009ca0 tx=0x7f5f70009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:53.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.276+0000 7f5f86c65b80 1 -- start start 2024-06-10T01:05:53.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.276+0000 7f5f86c65b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636c5ab1f60 0x5636c5ab2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:53.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.276+0000 7f5f86c65b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636c5ab2880 0x5636c5aac260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:53.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.276+0000 7f5f86c65b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c5aac7a0 0x5636c5aacbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:53.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.276+0000 7f5f86c65b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5636c599ef60 con 0x5636c5ab2880 2024-06-10T01:05:53.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.276+0000 7f5f86c65b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5636c59fd850 con 0x5636c5ab1f60 2024-06-10T01:05:53.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.276+0000 7f5f86c65b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5636c5965460 con 0x5636c5aac7a0 2024-06-10T01:05:53.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.277+0000 7f5f7997d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636c5ab2880 0x5636c5aac260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:53.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.277+0000 7f5f7997d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636c5ab2880 0x5636c5aac260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39970/0 (socket says 172.21.15.37:39970) 2024-06-10T01:05:53.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.277+0000 7f5f7997d700 1 -- 172.21.15.37:0/3260301424 learned_addr learned my addr 172.21.15.37:0/3260301424 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:53.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.277+0000 7f5f7a97f700 1 --2- 172.21.15.37:0/3260301424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c5aac7a0 0x5636c5aacbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:53.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.277+0000 7f5f7a17e700 1 --2- 172.21.15.37:0/3260301424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636c5ab1f60 0x5636c5ab2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:53.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.277+0000 7f5f7997d700 1 -- 172.21.15.37:0/3260301424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c5aac7a0 msgr2=0x5636c5aacbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:53.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.277+0000 7f5f7997d700 1 --2- 172.21.15.37:0/3260301424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c5aac7a0 0x5636c5aacbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.277+0000 7f5f7997d700 1 -- 172.21.15.37:0/3260301424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636c5ab1f60 msgr2=0x5636c5ab2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:53.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.277+0000 7f5f7997d700 1 --2- 172.21.15.37:0/3260301424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636c5ab1f60 0x5636c5ab2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.277+0000 7f5f7997d700 1 -- 172.21.15.37:0/3260301424 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636c594c8d0 con 0x5636c5ab2880 2024-06-10T01:05:53.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.277+0000 7f5f7a97f700 1 --2- 172.21.15.37:0/3260301424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c5aac7a0 0x5636c5aacbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:53.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.277+0000 7f5f7997d700 1 --2- 172.21.15.37:0/3260301424 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636c5ab2880 0x5636c5aac260 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009990 tx=0x7f5f64005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10932e6c1b86318 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:53.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.277+0000 7f5f68d70700 1 -- 172.21.15.37:0/3260301424 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6400b880 con 0x5636c5ab2880 2024-06-10T01:05:53.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.277+0000 7f5f68d70700 1 -- 172.21.15.37:0/3260301424 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f64027070 con 0x5636c5ab2880 2024-06-10T01:05:53.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.277+0000 7f5f68d70700 1 -- 172.21.15.37:0/3260301424 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6400f460 con 0x5636c5ab2880 2024-06-10T01:05:53.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.278+0000 7f5f86c65b80 1 -- 172.21.15.37:0/3260301424 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636c5ab2880 msgr2=0x5636c5aac260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:53.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.278+0000 7f5f86c65b80 1 --2- 172.21.15.37:0/3260301424 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636c5ab2880 0x5636c5aac260 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009990 tx=0x7f5f64005bc0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.278+0000 7f5f86c65b80 1 -- 172.21.15.37:0/3260301424 shutdown_connections 2024-06-10T01:05:53.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.278+0000 7f5f86c65b80 1 --2- 172.21.15.37:0/3260301424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c5aac7a0 0x5636c5aacbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.278+0000 7f5f86c65b80 1 --2- 172.21.15.37:0/3260301424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636c5ab1f60 0x5636c5ab2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.278+0000 7f5f86c65b80 1 --2- 172.21.15.37:0/3260301424 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636c5ab2880 0x5636c5aac260 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.278+0000 7f5f86c65b80 1 -- 172.21.15.37:0/3260301424 >> 172.21.15.37:0/3260301424 conn(0x5636c59d9080 msgr2=0x5636c5aa7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:53.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.278+0000 7f5f86c65b80 1 -- 172.21.15.37:0/3260301424 shutdown_connections 2024-06-10T01:05:53.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.278+0000 7f5f86c65b80 1 -- 172.21.15.37:0/3260301424 wait complete. 2024-06-10T01:05:53.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.279+0000 7f5f86c65b80 1 -- start start 2024-06-10T01:05:53.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.279+0000 7f5f86c65b80 1 Processor -- start 2024-06-10T01:05:53.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.279+0000 7f5f86c65b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c5aac7a0 0x5636c5ab8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:53.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.279+0000 7f5f86c65b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636c5ab1f60 0x5636c5ab5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:53.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.279+0000 7f5f86c65b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636c5ab2880 0x5636c5ab5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:53.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.279+0000 7f5f86c65b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5636c599dfc0 con 0x5636c5ab2880 2024-06-10T01:05:53.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.279+0000 7f5f86c65b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5636c59f9810 con 0x5636c5ab1f60 2024-06-10T01:05:53.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.279+0000 7f5f86c65b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5636c5ab9200 con 0x5636c5aac7a0 2024-06-10T01:05:53.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.279+0000 7f5f86c65b80 10 client.0 _ll_register_callbacks cb 0x5636c59935b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:05:53.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.279+0000 7f5f7a97f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636c5ab2880 0x5636c5ab5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:53.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.279+0000 7f5f7a97f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636c5ab2880 0x5636c5ab5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39982/0 (socket says 172.21.15.37:39982) 2024-06-10T01:05:53.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.279+0000 7f5f7a97f700 1 -- 172.21.15.37:0/3436759399 learned_addr learned my addr 172.21.15.37:0/3436759399 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:53.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.279+0000 7f5f7a17e700 1 --2- 172.21.15.37:0/3436759399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c5aac7a0 0x5636c5ab8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:53.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.280+0000 7f5f7997d700 1 --2- 172.21.15.37:0/3436759399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636c5ab1f60 0x5636c5ab5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:53.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.280+0000 7f5f7a17e700 1 -- 172.21.15.37:0/3436759399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636c5ab1f60 msgr2=0x5636c5ab5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:53.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.280+0000 7f5f7a17e700 1 --2- 172.21.15.37:0/3436759399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636c5ab1f60 0x5636c5ab5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.280+0000 7f5f7a17e700 1 -- 172.21.15.37:0/3436759399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636c5ab2880 msgr2=0x5636c5ab5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:53.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.280+0000 7f5f7a17e700 1 --2- 172.21.15.37:0/3436759399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636c5ab2880 0x5636c5ab5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.280+0000 7f5f7a17e700 1 -- 172.21.15.37:0/3436759399 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636c59c3750 con 0x5636c5aac7a0 2024-06-10T01:05:53.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.280+0000 7f5f7a97f700 1 --2- 172.21.15.37:0/3436759399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636c5ab2880 0x5636c5ab5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:53.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.280+0000 7f5f7a17e700 1 --2- 172.21.15.37:0/3436759399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c5aac7a0 0x5636c5ab8550 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5f70009b60 tx=0x7f5f700047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=548dcd4d010d9ff server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:53.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.280+0000 7f5f5dffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:05:53.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.280+0000 7f5f5dffb700 1 -- 172.21.15.37:0/3436759399 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7000e9a0 con 0x5636c5aac7a0 2024-06-10T01:05:53.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.280+0000 7f5f86c65b80 1 -- 172.21.15.37:0/3436759399 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f64027070 con 0x5636c5aac7a0 2024-06-10T01:05:53.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.280+0000 7f5f5dffb700 1 -- 172.21.15.37:0/3436759399 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f5f70009040 con 0x5636c5aac7a0 2024-06-10T01:05:53.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.280+0000 7f5f5dffb700 1 -- 172.21.15.37:0/3436759399 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5f7000eb40 con 0x5636c5aac7a0 2024-06-10T01:05:53.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.280+0000 7f5f86c65b80 10 client.26524 fetch_fsmap learned FSMap version 29 2024-06-10T01:05:53.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.281+0000 7f5f86c65b80 1 -- 172.21.15.37:0/3436759399 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5636c5ac3940 con 0x5636c5aac7a0 2024-06-10T01:05:53.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.281+0000 7f5f5dffb700 1 -- 172.21.15.37:0/3436759399 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f64027070 con 0x5636c5aac7a0 2024-06-10T01:05:53.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.281+0000 7f5f5dffb700 1 -- 172.21.15.37:0/3436759399 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f5f7001c480 con 0x5636c5aac7a0 2024-06-10T01:05:53.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.281+0000 7f5f86c65b80 10 client.26524 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:05:53.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.281+0000 7f5f86c65b80 10 client.26524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:05:53.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.281+0000 7f5f86c65b80 20 client.26524 populate_metadata read hostname 'smithi037' 2024-06-10T01:05:53.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.281+0000 7f5f86c65b80 1 --2- 172.21.15.37:0/3436759399 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5636c5ac5f80 0x5636c5ac83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:53.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.281+0000 7f5f86c65b80 4 client.26524 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:05:53.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.281+0000 7f5f86c65b80 1 -- 172.21.15.37:0/3436759399 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636c5a0ad50 con 0x5636c5ac5f80 2024-06-10T01:05:53.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.281+0000 7f5f7997d700 1 --2- 172.21.15.37:0/3436759399 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5636c5ac5f80 0x5636c5ac83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:53.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.282+0000 7f5f7997d700 1 --2- 172.21.15.37:0/3436759399 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5636c5ac5f80 0x5636c5ac83b0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=175ad094b8fce3ae server_cookie=4f19dc291923f696 in_seq=0 out_seq=0 2024-06-10T01:05:53.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.282+0000 7f5f5dffb700 10 client.26524 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:05:53.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.289+0000 7f5f5dffb700 1 -- 172.21.15.37:0/3436759399 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5636c5a0ad50 con 0x5636c5ac5f80 2024-06-10T01:05:53.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.289+0000 7f5f5dffb700 10 client.26524 handle_command_reply: tid=0 2024-06-10T01:05:53.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.289+0000 7f5f86c65b80 1 client.26524 shutdown 2024-06-10T01:05:53.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.289+0000 7f5f86c65b80 1 -- 172.21.15.37:0/3436759399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c5aac7a0 msgr2=0x5636c5ab8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:53.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.289+0000 7f5f86c65b80 1 --2- 172.21.15.37:0/3436759399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c5aac7a0 0x5636c5ab8550 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5f70009b60 tx=0x7f5f700047c0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.289+0000 7f5f86c65b80 1 -- 172.21.15.37:0/3436759399 shutdown_connections 2024-06-10T01:05:53.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.289+0000 7f5f86c65b80 1 --2- 172.21.15.37:0/3436759399 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5636c5ac5f80 0x5636c5ac83b0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.289+0000 7f5f86c65b80 1 --2- 172.21.15.37:0/3436759399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636c5ab2880 0x5636c5ab5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.289+0000 7f5f86c65b80 1 --2- 172.21.15.37:0/3436759399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c5aac7a0 0x5636c5ab8550 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.289+0000 7f5f86c65b80 1 --2- 172.21.15.37:0/3436759399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636c5ab1f60 0x5636c5ab5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.289+0000 7f5f86c65b80 1 -- 172.21.15.37:0/3436759399 >> 172.21.15.37:0/3436759399 conn(0x5636c59d9080 msgr2=0x5636c5aa6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:53.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.289+0000 7f5f5dffb700 0 client.26524 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:05:53.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.289+0000 7f5f86c65b80 1 -- 172.21.15.37:0/3436759399 shutdown_connections 2024-06-10T01:05:53.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.289+0000 7f5f86c65b80 1 -- 172.21.15.37:0/3436759399 wait complete. 2024-06-10T01:05:53.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.289+0000 7f5f86c65b80 20 client.26524 trim_cache size 0 max 16384 2024-06-10T01:05:53.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.310+0000 7f5f86c65b80 1 Processor -- start 2024-06-10T01:05:53.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.310+0000 7f5f86c65b80 1 -- start start 2024-06-10T01:05:53.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.310+0000 7f5f86c65b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636c5aac7a0 0x5636c5aac260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:53.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.310+0000 7f5f86c65b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c5ab1f60 0x5636c5ab7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:53.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.310+0000 7f5f86c65b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636c5ab2880 0x5636c5aa6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:53.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.310+0000 7f5f86c65b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741b2e80 con 0x5636c5aac7a0 2024-06-10T01:05:53.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.310+0000 7f5f86c65b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741b0710 con 0x5636c5ab2880 2024-06-10T01:05:53.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.310+0000 7f5f86c65b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741ae560 con 0x5636c5ab1f60 2024-06-10T01:05:53.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.311+0000 7f5f7a17e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636c5aac7a0 0x5636c5aac260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:53.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.311+0000 7f5f7997d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c5ab1f60 0x5636c5ab7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:53.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.311+0000 7f5f7997d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c5ab1f60 0x5636c5ab7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:45204/0 (socket says 172.21.15.37:45204) 2024-06-10T01:05:53.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.311+0000 7f5f7a17e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636c5aac7a0 0x5636c5aac260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39988/0 (socket says 172.21.15.37:39988) 2024-06-10T01:05:53.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.311+0000 7f5f7997d700 1 -- 172.21.15.37:0/1501019138 learned_addr learned my addr 172.21.15.37:0/1501019138 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:53.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.311+0000 7f5f7a97f700 1 --2- 172.21.15.37:0/1501019138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636c5ab2880 0x5636c5aa6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:53.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.311+0000 7f5f7997d700 1 -- 172.21.15.37:0/1501019138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636c5ab2880 msgr2=0x5636c5aa6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:53.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.311+0000 7f5f7997d700 1 --2- 172.21.15.37:0/1501019138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636c5ab2880 0x5636c5aa6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.311+0000 7f5f7997d700 1 -- 172.21.15.37:0/1501019138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636c5aac7a0 msgr2=0x5636c5aac260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:53.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.311+0000 7f5f7997d700 1 --2- 172.21.15.37:0/1501019138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636c5aac7a0 0x5636c5aac260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.311+0000 7f5f7997d700 1 -- 172.21.15.37:0/1501019138 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636c5ab6110 con 0x5636c5ab1f60 2024-06-10T01:05:53.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.311+0000 7f5f7a17e700 1 --2- 172.21.15.37:0/1501019138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636c5aac7a0 0x5636c5aac260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:05:53.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.311+0000 7f5f7a97f700 1 --2- 172.21.15.37:0/1501019138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636c5ab2880 0x5636c5aa6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:05:53.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.311+0000 7f5f7997d700 1 --2- 172.21.15.37:0/1501019138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c5ab1f60 0x5636c5ab7790 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5f64003b60 tx=0x7f5f6401a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc06843daf91c893 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:53.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.311+0000 7f5f5ffff700 1 -- 172.21.15.37:0/1501019138 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f64003e00 con 0x5636c5ab1f60 2024-06-10T01:05:53.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.311+0000 7f5f5ffff700 1 -- 172.21.15.37:0/1501019138 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f64027030 con 0x5636c5ab1f60 2024-06-10T01:05:53.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.311+0000 7f5f5ffff700 1 -- 172.21.15.37:0/1501019138 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f64035980 con 0x5636c5ab1f60 2024-06-10T01:05:53.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.312+0000 7f5f86c65b80 1 -- 172.21.15.37:0/1501019138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c5ab1f60 msgr2=0x5636c5ab7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:53.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.312+0000 7f5f86c65b80 1 --2- 172.21.15.37:0/1501019138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c5ab1f60 0x5636c5ab7790 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5f64003b60 tx=0x7f5f6401a040 comp rx=0 tx=0).stop 2024-06-10T01:05:53.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.312+0000 7f5f86c65b80 1 -- 172.21.15.37:0/1501019138 shutdown_connections 2024-06-10T01:05:53.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.312+0000 7f5f86c65b80 1 --2- 172.21.15.37:0/1501019138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636c5ab2880 0x5636c5aa6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.312+0000 7f5f86c65b80 1 --2- 172.21.15.37:0/1501019138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636c5aac7a0 0x5636c5aac260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.312+0000 7f5f86c65b80 1 --2- 172.21.15.37:0/1501019138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c5ab1f60 0x5636c5ab7790 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.312+0000 7f5f86c65b80 1 -- 172.21.15.37:0/1501019138 >> 172.21.15.37:0/1501019138 conn(0x5636c59d9080 msgr2=0x5636c5ab7080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:53.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.312+0000 7f5f86c65b80 1 -- 172.21.15.37:0/1501019138 shutdown_connections 2024-06-10T01:05:53.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.312+0000 7f5f86c65b80 1 -- 172.21.15.37:0/1501019138 wait complete. 2024-06-10T01:05:53.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.313+0000 7f5f86c65b80 1 -- start start 2024-06-10T01:05:53.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.313+0000 7f5f86c65b80 1 Processor -- start 2024-06-10T01:05:53.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.313+0000 7f5f86c65b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c5aac7a0 0x5636c5abead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:53.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.313+0000 7f5f86c65b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636c5ab1f60 0x5636c5acd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:53.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.313+0000 7f5f86c65b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636c5ab2880 0x5636c5acda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:53.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.313+0000 7f5f86c65b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741aebd0 con 0x5636c5ab1f60 2024-06-10T01:05:53.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.313+0000 7f5f86c65b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741b0040 con 0x5636c5ab2880 2024-06-10T01:05:53.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.313+0000 7f5f86c65b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741af2a0 con 0x5636c5aac7a0 2024-06-10T01:05:53.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.313+0000 7f5f86c65b80 10 client.0 _ll_register_callbacks cb 0x5636c59935b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:05:53.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.313+0000 7f5f7a17e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c5aac7a0 0x5636c5abead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:53.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.313+0000 7f5f7a17e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c5aac7a0 0x5636c5abead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:45206/0 (socket says 172.21.15.37:45206) 2024-06-10T01:05:53.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.313+0000 7f5f7a17e700 1 -- 172.21.15.37:0/166950746 learned_addr learned my addr 172.21.15.37:0/166950746 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:05:53.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.313+0000 7f5f7997d700 1 --2- 172.21.15.37:0/166950746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636c5ab1f60 0x5636c5acd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:53.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.313+0000 7f5f7a97f700 1 --2- 172.21.15.37:0/166950746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636c5ab2880 0x5636c5acda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:53.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.313+0000 7f5f7a17e700 1 -- 172.21.15.37:0/166950746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636c5ab2880 msgr2=0x5636c5acda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:53.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.313+0000 7f5f7a17e700 1 --2- 172.21.15.37:0/166950746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636c5ab2880 0x5636c5acda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.313+0000 7f5f7a17e700 1 -- 172.21.15.37:0/166950746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636c5ab1f60 msgr2=0x5636c5acd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:05:53.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.313+0000 7f5f7a17e700 1 --2- 172.21.15.37:0/166950746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636c5ab1f60 0x5636c5acd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.314+0000 7f5f7a17e700 1 -- 172.21.15.37:0/166950746 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636c5ace0e0 con 0x5636c5aac7a0 2024-06-10T01:05:53.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.314+0000 7f5f7a17e700 1 --2- 172.21.15.37:0/166950746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c5aac7a0 0x5636c5abead0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5f7001c8f0 tx=0x7f5f7001c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f262e55dabf8f04a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:05:53.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.314+0000 7f5f5dffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:05:53.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.314+0000 7f5f5dffb700 1 -- 172.21.15.37:0/166950746 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7003b060 con 0x5636c5aac7a0 2024-06-10T01:05:53.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.314+0000 7f5f5dffb700 1 -- 172.21.15.37:0/166950746 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f5f70009040 con 0x5636c5aac7a0 2024-06-10T01:05:53.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.314+0000 7f5f86c65b80 1 -- 172.21.15.37:0/166950746 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f64027030 con 0x5636c5aac7a0 2024-06-10T01:05:53.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.314+0000 7f5f5dffb700 1 -- 172.21.15.37:0/166950746 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5f70009ea0 con 0x5636c5aac7a0 2024-06-10T01:05:53.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.314+0000 7f5f86c65b80 10 client.26530 fetch_fsmap learned FSMap version 29 2024-06-10T01:05:53.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.314+0000 7f5f86c65b80 1 -- 172.21.15.37:0/166950746 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5636c5b30b90 con 0x5636c5aac7a0 2024-06-10T01:05:53.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.314+0000 7f5f5dffb700 1 -- 172.21.15.37:0/166950746 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f64027030 con 0x5636c5aac7a0 2024-06-10T01:05:53.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.314+0000 7f5f5dffb700 1 -- 172.21.15.37:0/166950746 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f5f700363f0 con 0x5636c5aac7a0 2024-06-10T01:05:53.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.314+0000 7f5f86c65b80 10 client.26530 fetch_fsmap finished waiting for FSMap version 29 2024-06-10T01:05:53.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.314+0000 7f5f86c65b80 10 client.26530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:05:53.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.314+0000 7f5f86c65b80 20 client.26530 populate_metadata read hostname 'smithi037' 2024-06-10T01:05:53.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.315+0000 7f5f86c65b80 1 --2- 172.21.15.37:0/166950746 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5636c5b31270 0x5636c5b33690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:05:53.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.315+0000 7f5f86c65b80 4 client.26530 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:05:53.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.315+0000 7f5f86c65b80 1 -- 172.21.15.37:0/166950746 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5636c5a052d0 con 0x5636c5b31270 2024-06-10T01:05:53.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.315+0000 7f5f7997d700 1 --2- 172.21.15.37:0/166950746 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5636c5b31270 0x5636c5b33690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:05:53.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.315+0000 7f5f7997d700 1 --2- 172.21.15.37:0/166950746 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5636c5b31270 0x5636c5b33690 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e47cbbb76020a467 server_cookie=6b5a6b5a4d13cc60 in_seq=0 out_seq=0 2024-06-10T01:05:53.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.315+0000 7f5f5dffb700 10 client.26530 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:05:53.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.323+0000 7f5f5dffb700 1 -- 172.21.15.37:0/166950746 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x5636c5a052d0 con 0x5636c5b31270 2024-06-10T01:05:53.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.323+0000 7f5f5dffb700 10 client.26530 handle_command_reply: tid=0 2024-06-10T01:05:53.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.323+0000 7f5f86c65b80 1 client.26530 shutdown 2024-06-10T01:05:53.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.323+0000 7f5f86c65b80 1 -- 172.21.15.37:0/166950746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c5aac7a0 msgr2=0x5636c5abead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:53.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.323+0000 7f5f86c65b80 1 --2- 172.21.15.37:0/166950746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c5aac7a0 0x5636c5abead0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5f7001c8f0 tx=0x7f5f7001c6c0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.323+0000 7f5f86c65b80 1 -- 172.21.15.37:0/166950746 shutdown_connections 2024-06-10T01:05:53.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.323+0000 7f5f86c65b80 1 --2- 172.21.15.37:0/166950746 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5636c5b31270 0x5636c5b33690 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.323+0000 7f5f86c65b80 1 --2- 172.21.15.37:0/166950746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636c5ab2880 0x5636c5acda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.323+0000 7f5f86c65b80 1 --2- 172.21.15.37:0/166950746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636c5aac7a0 0x5636c5abead0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.323+0000 7f5f86c65b80 1 --2- 172.21.15.37:0/166950746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636c5ab1f60 0x5636c5acd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.323+0000 7f5f86c65b80 1 -- 172.21.15.37:0/166950746 >> 172.21.15.37:0/166950746 conn(0x5636c59d9080 msgr2=0x5636c5ab3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:53.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.323+0000 7f5f5dffb700 0 client.26530 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:05:53.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.323+0000 7f5f86c65b80 1 -- 172.21.15.37:0/166950746 shutdown_connections 2024-06-10T01:05:53.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.323+0000 7f5f86c65b80 1 -- 172.21.15.37:0/166950746 wait complete. 2024-06-10T01:05:53.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.323+0000 7f5f86c65b80 20 client.26530 trim_cache size 0 max 16384 2024-06-10T01:05:53.321 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:05:53.321 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "scrub active (8957 inodes in the stack)", 2024-06-10T01:05:53.321 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-06-10T01:05:53.321 INFO:teuthology.orchestra.run.smithi037.stdout: "d79c0c89-cbf2-47ac-9a2b-0a728f0b2713": { 2024-06-10T01:05:53.321 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-06-10T01:05:53.321 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "d79c0c89-cbf2-47ac-9a2b-0a728f0b2713", 2024-06-10T01:05:53.321 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-06-10T01:05:53.321 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:05:53.321 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:05:53.322 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:05:53.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.324+0000 7f5f7c3e2700 1 -- 172.21.15.37:0/890295821 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5f600a2ca0 msgr2=0x7f5f600a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:53.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.324+0000 7f5f7c3e2700 1 --2- 172.21.15.37:0/890295821 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5f600a2ca0 0x7f5f600a5160 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5f70009ca0 tx=0x7f5f70009260 comp rx=0 tx=0).stop 2024-06-10T01:05:53.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.324+0000 7f5f7c3e2700 1 -- 172.21.15.37:0/890295821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f741073d0 msgr2=0x7f5f7419f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:53.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.324+0000 7f5f7c3e2700 1 --2- 172.21.15.37:0/890295821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f741073d0 0x7f5f7419f6c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00efc0 tx=0x7f5f6c00c3a0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.324+0000 7f5f7c3e2700 1 -- 172.21.15.37:0/890295821 shutdown_connections 2024-06-10T01:05:53.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.324+0000 7f5f7c3e2700 1 --2- 172.21.15.37:0/890295821 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5f600a2ca0 0x7f5f600a5160 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.324+0000 7f5f7c3e2700 1 --2- 172.21.15.37:0/890295821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f741073d0 0x7f5f7419f6c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.324+0000 7f5f7c3e2700 1 --2- 172.21.15.37:0/890295821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f74101390 0x7f5f7419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.324+0000 7f5f7c3e2700 1 --2- 172.21.15.37:0/890295821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f74101d30 0x7f5f7419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:53.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.324+0000 7f5f7c3e2700 1 -- 172.21.15.37:0/890295821 >> 172.21.15.37:0/890295821 conn(0x7f5f740782c0 msgr2=0x7f5f7406ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:53.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.324+0000 7f5f7c3e2700 1 -- 172.21.15.37:0/890295821 shutdown_connections 2024-06-10T01:05:53.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:05:53.324+0000 7f5f7c3e2700 1 -- 172.21.15.37:0/890295821 wait complete. 2024-06-10T01:05:54.309 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:54 smithi037 ceph-mon[32192]: pgmap v707: 129 pgs: 129 active+clean; 46 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 1.10k op/s 2024-06-10T01:05:54.338 INFO:tasks.cephfs.filesystem:scrub status for tag:d79c0c89-cbf2-47ac-9a2b-0a728f0b2713 - {'path': '/', 'tag': 'd79c0c89-cbf2-47ac-9a2b-0a728f0b2713', 'options': 'recursive,force'} 2024-06-10T01:05:54.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:54 smithi167 ceph-mon[36917]: pgmap v707: 129 pgs: 129 active+clean; 46 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 1.10k op/s 2024-06-10T01:05:54.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:54 smithi094 ceph-mon[36708]: pgmap v707: 129 pgs: 129 active+clean; 46 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 1.10k op/s 2024-06-10T01:05:55.085 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:05:54 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-f[71145]: *** Caught signal (Segmentation fault) ** 2024-06-10T01:05:55.085 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:05:54 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-f[71145]: in thread 7f345b795700 thread_name:MR_Finisher 2024-06-10T01:05:55.085 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:05:54 smithi167 podman[72033]: 2024-06-10 01:05:54.740406312 +0000 UTC m=+0.024498513 container died c384814da1b9fad8ed928ad1a4746a1a016335f498e4bea58ae53d68e0e0e012 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-f, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528) 2024-06-10T01:05:56.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:56 smithi167 ceph-mon[36917]: pgmap v708: 129 pgs: 129 active+clean; 46 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 1.19k op/s 2024-06-10T01:05:56.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:56 smithi094 ceph-mon[36708]: pgmap v708: 129 pgs: 129 active+clean; 46 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 1.19k op/s 2024-06-10T01:05:56.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:56 smithi037 ceph-mon[32192]: pgmap v708: 129 pgs: 129 active+clean; 46 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 1.19k op/s 2024-06-10T01:05:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:57.543+0000 7f928b7fe640 1 -- 172.21.15.37:0/2425971198 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a1fa970c20 con 0x55a1faa84850 2024-06-10T01:05:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:57.543+0000 7f928b7fe640 10 client.16983 handle_command_reply: tid=0 2024-06-10T01:05:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:57.543+0000 7f92b518c740 1 client.16983 shutdown 2024-06-10T01:05:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:57.544+0000 7f92b518c740 1 -- 172.21.15.37:0/2425971198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1faa09f80 msgr2=0x55a1faa82c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:57.544+0000 7f92b518c740 1 --2- 172.21.15.37:0/2425971198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1faa09f80 0x55a1faa82c10 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f92a40a45f0 tx=0x7f92a409a070 comp rx=0 tx=0).stop 2024-06-10T01:05:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:57.544+0000 7f92b518c740 1 -- 172.21.15.37:0/2425971198 shutdown_connections 2024-06-10T01:05:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:57.544+0000 7f92b518c740 1 --2- 172.21.15.37:0/2425971198 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a1faa84850 0x55a1faa86c70 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:57.544+0000 7f92b518c740 1 --2- 172.21.15.37:0/2425971198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1faa09f80 0x55a1faa82c10 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:57.544+0000 7f92b518c740 1 --2- 172.21.15.37:0/2425971198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a1faa04e10 0x55a1faa12d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:57.544+0000 7f928b7fe640 0 client.16983 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:05:57.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:57.544+0000 7f92b518c740 1 --2- 172.21.15.37:0/2425971198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a1faa04420 0x55a1faa12820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:57.544+0000 7f92b518c740 1 -- 172.21.15.37:0/2425971198 >> 172.21.15.37:0/2425971198 conn(0x55a1fa9fb680 msgr2=0x55a1fa9fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:57.544+0000 7f92b518c740 1 -- 172.21.15.37:0/2425971198 shutdown_connections 2024-06-10T01:05:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:57.544+0000 7f92b518c740 1 -- 172.21.15.37:0/2425971198 wait complete. 2024-06-10T01:05:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:57.544+0000 7f92b518c740 20 client.16983 trim_cache size 0 max 16384 2024-06-10T01:05:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:05:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:05:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:05:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:57.544+0000 7f92b23a5640 1 -- 172.21.15.37:0/653007281 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f92940a2c30 msgr2=0x7f92940a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:57.544+0000 7f92b23a5640 1 --2- 172.21.15.37:0/653007281 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f92940a2c30 0x7f92940a50f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x55a1faa05670 tx=0x7f92a0005850 comp rx=0 tx=0).stop 2024-06-10T01:05:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:57.544+0000 7f92b23a5640 1 -- 172.21.15.37:0/653007281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92ac1024b0 msgr2=0x7f92ac19efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:05:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:57.544+0000 7f92b23a5640 1 --2- 172.21.15.37:0/653007281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92ac1024b0 0x7f92ac19efa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f92a4009fd0 tx=0x7f92a400ef10 comp rx=0 tx=0).stop 2024-06-10T01:05:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:57.545+0000 7f92b23a5640 1 -- 172.21.15.37:0/653007281 shutdown_connections 2024-06-10T01:05:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:57.545+0000 7f92b23a5640 1 --2- 172.21.15.37:0/653007281 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f92940a2c30 0x7f92940a50f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:57.545+0000 7f92b23a5640 1 --2- 172.21.15.37:0/653007281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92ac1024b0 0x7f92ac19efa0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:57.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:57.545+0000 7f92b23a5640 1 --2- 172.21.15.37:0/653007281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92ac101af0 0x7f92ac19c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:57.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:57.545+0000 7f92b23a5640 1 --2- 172.21.15.37:0/653007281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ac100900 0x7f92ac06ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:05:57.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:57.545+0000 7f92b23a5640 1 -- 172.21.15.37:0/653007281 >> 172.21.15.37:0/653007281 conn(0x7f92ac0fbe90 msgr2=0x7f92ac104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:05:57.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:57.545+0000 7f92b23a5640 1 -- 172.21.15.37:0/653007281 shutdown_connections 2024-06-10T01:05:57.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:05:57.545+0000 7f92b23a5640 1 -- 172.21.15.37:0/653007281 wait complete. 2024-06-10T01:05:58.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:05:58 smithi167 ceph-mon[36917]: pgmap v709: 129 pgs: 129 active+clean; 46 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.6 MiB/s wr, 1.12k op/s 2024-06-10T01:05:58.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:05:58 smithi094 ceph-mon[36708]: pgmap v709: 129 pgs: 129 active+clean; 46 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.6 MiB/s wr, 1.12k op/s 2024-06-10T01:05:58.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:05:58 smithi037 ceph-mon[32192]: pgmap v709: 129 pgs: 129 active+clean; 46 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.6 MiB/s wr, 1.12k op/s 2024-06-10T01:06:00.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:00 smithi167 ceph-mon[36917]: pgmap v710: 129 pgs: 129 active+clean; 34 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 1.10k op/s 2024-06-10T01:06:00.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:00 smithi094 ceph-mon[36708]: pgmap v710: 129 pgs: 129 active+clean; 34 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 1.10k op/s 2024-06-10T01:06:00.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:00 smithi037 ceph-mon[32192]: pgmap v710: 129 pgs: 129 active+clean; 34 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 1.10k op/s 2024-06-10T01:06:01.921 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:06:01 smithi167 podman[72033]: 2024-06-10 01:06:01.604581376 +0000 UTC m=+6.888673568 container remove c384814da1b9fad8ed928ad1a4746a1a016335f498e4bea58ae53d68e0e0e012 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=) 2024-06-10T01:06:01.921 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:06:01 smithi167 systemd[1]: ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.f.service: Main process exited, code=exited, status=139/n/a 2024-06-10T01:06:02.334 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:02 smithi167 ceph-mon[36917]: pgmap v711: 129 pgs: 129 active+clean; 38 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 1.1 MiB/s wr, 474 op/s 2024-06-10T01:06:02.335 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:06:01 smithi167 systemd[1]: ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.f.service: Failed with result 'exit-code'. 2024-06-10T01:06:02.335 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:06:01 smithi167 systemd[1]: ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.f.service: Consumed 20.297s CPU time. 2024-06-10T01:06:02.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:02 smithi094 ceph-mon[36708]: pgmap v711: 129 pgs: 129 active+clean; 38 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 1.1 MiB/s wr, 474 op/s 2024-06-10T01:06:02.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:02 smithi037 ceph-mon[32192]: pgmap v711: 129 pgs: 129 active+clean; 38 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 1.1 MiB/s wr, 474 op/s 2024-06-10T01:06:04.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:04 smithi037 ceph-mon[32192]: pgmap v712: 129 pgs: 129 active+clean; 33 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.3 MiB/s wr, 437 op/s 2024-06-10T01:06:04.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:04 smithi167 ceph-mon[36917]: pgmap v712: 129 pgs: 129 active+clean; 33 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.3 MiB/s wr, 437 op/s 2024-06-10T01:06:04.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:04 smithi094 ceph-mon[36708]: pgmap v712: 129 pgs: 129 active+clean; 33 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.3 MiB/s wr, 437 op/s 2024-06-10T01:06:06.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:06 smithi167 ceph-mon[36917]: pgmap v713: 129 pgs: 129 active+clean; 33 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.2 MiB/s wr, 561 op/s 2024-06-10T01:06:06.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:06 smithi094 ceph-mon[36708]: pgmap v713: 129 pgs: 129 active+clean; 33 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.2 MiB/s wr, 561 op/s 2024-06-10T01:06:06.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:06 smithi037 ceph-mon[32192]: pgmap v713: 129 pgs: 129 active+clean; 33 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.2 MiB/s wr, 561 op/s 2024-06-10T01:06:08.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:08 smithi167 ceph-mon[36917]: pgmap v714: 129 pgs: 129 active+clean; 33 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 920 KiB/s wr, 448 op/s 2024-06-10T01:06:08.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:08 smithi094 ceph-mon[36708]: pgmap v714: 129 pgs: 129 active+clean; 33 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 920 KiB/s wr, 448 op/s 2024-06-10T01:06:08.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:08 smithi037 ceph-mon[32192]: pgmap v714: 129 pgs: 129 active+clean; 33 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 920 KiB/s wr, 448 op/s 2024-06-10T01:06:10.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:10 smithi167 ceph-mon[36917]: pgmap v715: 129 pgs: 129 active+clean; 38 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 1.4 MiB/s wr, 609 op/s 2024-06-10T01:06:10.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:10 smithi167 ceph-mon[36917]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2024-06-10T01:06:10.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:10 smithi167 ceph-mon[36917]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-06-10T01:06:10.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:10 smithi094 ceph-mon[36708]: pgmap v715: 129 pgs: 129 active+clean; 38 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 1.4 MiB/s wr, 609 op/s 2024-06-10T01:06:10.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:10 smithi094 ceph-mon[36708]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2024-06-10T01:06:10.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:10 smithi094 ceph-mon[36708]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-06-10T01:06:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.637+0000 7f59b7b14640 1 -- 172.21.15.37:0/304418605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b0101af0 msgr2=0x7f59b0101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.637+0000 7f59b7b14640 1 --2- 172.21.15.37:0/304418605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b0101af0 0x7f59b0101f70 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f59a0009900 tx=0x7f59a002a410 comp rx=0 tx=0).stop 2024-06-10T01:06:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.637+0000 7f59b7b14640 1 -- 172.21.15.37:0/304418605 shutdown_connections 2024-06-10T01:06:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.637+0000 7f59b7b14640 1 --2- 172.21.15.37:0/304418605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59b01024b0 0x7f59b0108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.637+0000 7f59b7b14640 1 --2- 172.21.15.37:0/304418605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b0101af0 0x7f59b0101f70 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.637+0000 7f59b7b14640 1 --2- 172.21.15.37:0/304418605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59b0100900 0x7f59b0100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.637+0000 7f59b7b14640 1 -- 172.21.15.37:0/304418605 >> 172.21.15.37:0/304418605 conn(0x7f59b00fbe70 msgr2=0x7f59b00fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.637+0000 7f59b7b14640 1 -- 172.21.15.37:0/304418605 shutdown_connections 2024-06-10T01:06:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.637+0000 7f59b7b14640 1 -- 172.21.15.37:0/304418605 wait complete. 2024-06-10T01:06:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.638+0000 7f59b7b14640 1 Processor -- start 2024-06-10T01:06:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.638+0000 7f59b7b14640 1 -- start start 2024-06-10T01:06:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.638+0000 7f59b7b14640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b0100900 0x7f59b0104840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.638+0000 7f59b7b14640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59b0101af0 0x7f59b0102ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.638+0000 7f59b7b14640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59b01024b0 0x7f59b0103410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.638+0000 7f59b7b14640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f59b010c580 con 0x7f59b0100900 2024-06-10T01:06:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.638+0000 7f59b7b14640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f59b010c400 con 0x7f59b01024b0 2024-06-10T01:06:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.638+0000 7f59b7b14640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f59b010c700 con 0x7f59b0101af0 2024-06-10T01:06:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.638+0000 7f59b58b0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b0100900 0x7f59b0104840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.638+0000 7f59b58b0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b0100900 0x7f59b0104840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35478/0 (socket says 172.21.15.37:35478) 2024-06-10T01:06:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.638+0000 7f59b58b0640 1 -- 172.21.15.37:0/1244586222 learned_addr learned my addr 172.21.15.37:0/1244586222 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:06:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.638+0000 7f59b50af640 1 --2- 172.21.15.37:0/1244586222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59b0101af0 0x7f59b0102ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.638+0000 7f59b60b1640 1 --2- 172.21.15.37:0/1244586222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59b01024b0 0x7f59b0103410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.639+0000 7f59b58b0640 1 -- 172.21.15.37:0/1244586222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59b0101af0 msgr2=0x7f59b0102ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.639+0000 7f59b58b0640 1 --2- 172.21.15.37:0/1244586222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59b0101af0 0x7f59b0102ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.639+0000 7f59b58b0640 1 -- 172.21.15.37:0/1244586222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59b01024b0 msgr2=0x7f59b0103410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.639+0000 7f59b58b0640 1 --2- 172.21.15.37:0/1244586222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59b01024b0 0x7f59b0103410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.639+0000 7f59b58b0640 1 -- 172.21.15.37:0/1244586222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b0103950 con 0x7f59b0100900 2024-06-10T01:06:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.639+0000 7f59b60b1640 1 --2- 172.21.15.37:0/1244586222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59b01024b0 0x7f59b0103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.639+0000 7f59b58b0640 1 --2- 172.21.15.37:0/1244586222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b0100900 0x7f59b0104840 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f59ac00e9c0 tx=0x7f59ac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.639+0000 7f59a6ffd640 1 -- 172.21.15.37:0/1244586222 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac00cd80 con 0x7f59b0100900 2024-06-10T01:06:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.639+0000 7f59a6ffd640 1 -- 172.21.15.37:0/1244586222 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59ac0035f0 con 0x7f59b0100900 2024-06-10T01:06:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.639+0000 7f59b7b14640 1 -- 172.21.15.37:0/1244586222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b0103c40 con 0x7f59b0100900 2024-06-10T01:06:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.639+0000 7f59a6ffd640 1 -- 172.21.15.37:0/1244586222 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac010690 con 0x7f59b0100900 2024-06-10T01:06:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.639+0000 7f59b7b14640 1 -- 172.21.15.37:0/1244586222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b01a6740 con 0x7f59b0100900 2024-06-10T01:06:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.641+0000 7f59a6ffd640 1 -- 172.21.15.37:0/1244586222 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f59ac0037a0 con 0x7f59b0100900 2024-06-10T01:06:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.642+0000 7f59ba824740 1 Processor -- start 2024-06-10T01:06:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.642+0000 7f59ba824740 1 -- start start 2024-06-10T01:06:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.642+0000 7f59ba824740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9b93a010 0x564b9b93a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.642+0000 7f59ba824740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b9b93aa00 0x564b9b93f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.642+0000 7f59ba824740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b9b93fb70 0x564b9b942000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.642+0000 7f59ba824740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564b9b89d4f0 con 0x564b9b93aa00 2024-06-10T01:06:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.642+0000 7f59ba824740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564b9b83b870 con 0x564b9b93a010 2024-06-10T01:06:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.642+0000 7f59ba824740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564b9b8994b0 con 0x564b9b93fb70 2024-06-10T01:06:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.642+0000 7f59b58b0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b9b93aa00 0x564b9b93f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.642+0000 7f59b58b0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b9b93aa00 0x564b9b93f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35494/0 (socket says 172.21.15.37:35494) 2024-06-10T01:06:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.642+0000 7f59b58b0640 1 -- 172.21.15.37:0/196367219 learned_addr learned my addr 172.21.15.37:0/196367219 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:06:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.642+0000 7f59b50af640 1 --2- 172.21.15.37:0/196367219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b9b93fb70 0x564b9b942000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.643+0000 7f59b60b1640 1 --2- 172.21.15.37:0/196367219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9b93a010 0x564b9b93a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.643+0000 7f59a6ffd640 1 --2- 172.21.15.37:0/1244586222 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f599c0a2c30 0x7f599c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.643+0000 7f59b58b0640 1 -- 172.21.15.37:0/196367219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b9b93fb70 msgr2=0x564b9b942000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.643+0000 7f59b58b0640 1 --2- 172.21.15.37:0/196367219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b9b93fb70 0x564b9b942000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.643+0000 7f59b58b0640 1 -- 172.21.15.37:0/196367219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9b93a010 msgr2=0x564b9b93a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.643+0000 7f59b58b0640 1 --2- 172.21.15.37:0/196367219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9b93a010 0x564b9b93a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.643+0000 7f59b58b0640 1 -- 172.21.15.37:0/196367219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b9b8579b0 con 0x564b9b93aa00 2024-06-10T01:06:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.643+0000 7f59b60b1640 1 --2- 172.21.15.37:0/196367219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9b93a010 0x564b9b93a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.643+0000 7f59b50af640 1 --2- 172.21.15.37:0/196367219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b9b93fb70 0x564b9b942000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.643+0000 7f59b60b1640 1 --2- 172.21.15.37:0/1244586222 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f599c0a2c30 0x7f599c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.643+0000 7f59a6ffd640 1 -- 172.21.15.37:0/1244586222 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f59ac0cf830 con 0x7f59b0100900 2024-06-10T01:06:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.643+0000 7f59b58b0640 1 --2- 172.21.15.37:0/196367219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b9b93aa00 0x564b9b93f500 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f59ac056ed0 tx=0x7f59ac05bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9985b1ee4ab4e376 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.643+0000 7f59b60b1640 1 --2- 172.21.15.37:0/1244586222 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f599c0a2c30 0x7f599c0a50f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x564b9b93b260 tx=0x7f59a8004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.643+0000 7f599bfff640 1 -- 172.21.15.37:0/196367219 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac09ba70 con 0x564b9b93aa00 2024-06-10T01:06:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.643+0000 7f599bfff640 1 -- 172.21.15.37:0/196367219 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59ac007380 con 0x564b9b93aa00 2024-06-10T01:06:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.643+0000 7f599bfff640 1 -- 172.21.15.37:0/196367219 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac053bf0 con 0x564b9b93aa00 2024-06-10T01:06:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.644+0000 7f59ba824740 1 -- 172.21.15.37:0/196367219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b9b93aa00 msgr2=0x564b9b93f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.644+0000 7f59ba824740 1 --2- 172.21.15.37:0/196367219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b9b93aa00 0x564b9b93f500 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f59ac056ed0 tx=0x7f59ac05bd10 comp rx=0 tx=0).stop 2024-06-10T01:06:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.644+0000 7f59ba824740 1 -- 172.21.15.37:0/196367219 shutdown_connections 2024-06-10T01:06:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.644+0000 7f59ba824740 1 --2- 172.21.15.37:0/196367219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b9b93fb70 0x564b9b942000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.644+0000 7f59ba824740 1 --2- 172.21.15.37:0/196367219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b9b93aa00 0x564b9b93f500 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.644+0000 7f59ba824740 1 --2- 172.21.15.37:0/196367219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9b93a010 0x564b9b93a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.644+0000 7f59ba824740 1 -- 172.21.15.37:0/196367219 >> 172.21.15.37:0/196367219 conn(0x564b9b935680 msgr2=0x564b9b935a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.644+0000 7f59ba824740 1 -- 172.21.15.37:0/196367219 shutdown_connections 2024-06-10T01:06:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.644+0000 7f59ba824740 1 -- 172.21.15.37:0/196367219 wait complete. 2024-06-10T01:06:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.644+0000 7f59ba824740 1 -- start start 2024-06-10T01:06:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.645+0000 7f59ba824740 1 Processor -- start 2024-06-10T01:06:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.645+0000 7f59ba824740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b9b93a010 0x564b9b9444a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.645+0000 7f59ba824740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9b93aa00 0x564b9b9449e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.645+0000 7f59ba824740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b9b93fb70 0x564b9b94ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.645+0000 7f59ba824740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564b9b942ac0 con 0x564b9b93fb70 2024-06-10T01:06:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.645+0000 7f59ba824740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564b9b826ae0 con 0x564b9b93aa00 2024-06-10T01:06:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.645+0000 7f59ba824740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564b9b942c40 con 0x564b9b93a010 2024-06-10T01:06:10.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:10 smithi037 ceph-mon[32192]: pgmap v715: 129 pgs: 129 active+clean; 38 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 1.4 MiB/s wr, 609 op/s 2024-06-10T01:06:10.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:10 smithi037 ceph-mon[32192]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2024-06-10T01:06:10.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:10 smithi037 ceph-mon[32192]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-06-10T01:06:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.645+0000 7f59ba824740 10 client.0 _ll_register_callbacks cb 0x564b9b85f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.645+0000 7f59b60b1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b9b93fb70 0x564b9b94ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.645+0000 7f59b58b0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b9b93a010 0x564b9b9444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.645+0000 7f59b60b1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b9b93fb70 0x564b9b94ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35504/0 (socket says 172.21.15.37:35504) 2024-06-10T01:06:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.645+0000 7f59b60b1640 1 -- 172.21.15.37:0/4205134762 learned_addr learned my addr 172.21.15.37:0/4205134762 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:06:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.645+0000 7f59b50af640 1 --2- 172.21.15.37:0/4205134762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9b93aa00 0x564b9b9449e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.645+0000 7f59b60b1640 1 -- 172.21.15.37:0/4205134762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b9b93a010 msgr2=0x564b9b9444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.646+0000 7f59b60b1640 1 --2- 172.21.15.37:0/4205134762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b9b93a010 0x564b9b9444a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.646+0000 7f59b60b1640 1 -- 172.21.15.37:0/4205134762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9b93aa00 msgr2=0x564b9b9449e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.646+0000 7f59b60b1640 1 --2- 172.21.15.37:0/4205134762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9b93aa00 0x564b9b9449e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.646+0000 7f59b60b1640 1 -- 172.21.15.37:0/4205134762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b9b5b1210 con 0x564b9b93fb70 2024-06-10T01:06:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.646+0000 7f59b58b0640 1 --2- 172.21.15.37:0/4205134762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b9b93a010 0x564b9b9444a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.646+0000 7f59b60b1640 1 --2- 172.21.15.37:0/4205134762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b9b93fb70 0x564b9b94ba20 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f59a8009f80 tx=0x7f59a8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966a71dcf30b66b7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:10.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.646+0000 7f59997fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:06:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.646+0000 7f59997fa640 1 -- 172.21.15.37:0/4205134762 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a801dd70 con 0x564b9b93fb70 2024-06-10T01:06:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.646+0000 7f59997fa640 1 -- 172.21.15.37:0/4205134762 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f59a8027470 con 0x564b9b93fb70 2024-06-10T01:06:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.646+0000 7f59ba824740 1 -- 172.21.15.37:0/4205134762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59ac007380 con 0x564b9b93fb70 2024-06-10T01:06:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.646+0000 7f59997fa640 1 -- 172.21.15.37:0/4205134762 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f59a8027670 con 0x564b9b93fb70 2024-06-10T01:06:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.646+0000 7f59ba824740 10 client.17013 fetch_fsmap learned FSMap version 30 2024-06-10T01:06:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.646+0000 7f59ba824740 1 -- 172.21.15.37:0/4205134762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564b9b67d620 con 0x564b9b93fb70 2024-06-10T01:06:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.646+0000 7f59997fa640 1 -- 172.21.15.37:0/4205134762 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59ac007380 con 0x564b9b93fb70 2024-06-10T01:06:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.646+0000 7f59997fa640 1 -- 172.21.15.37:0/4205134762 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 30) ==== 7741+0+0 (secure 0 0 0) 0x7f59a801ea00 con 0x564b9b93fb70 2024-06-10T01:06:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.647+0000 7f59ba824740 10 client.17013 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:06:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.647+0000 7f59ba824740 10 client.17013 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:06:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.647+0000 7f59ba824740 20 client.17013 populate_metadata read hostname 'smithi037' 2024-06-10T01:06:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.647+0000 7f59ba824740 1 --2- 172.21.15.37:0/4205134762 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564b9b950380 0x564b9b9527d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.647+0000 7f59ba824740 4 client.17013 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:06:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.647+0000 7f59ba824740 1 -- 172.21.15.37:0/4205134762 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b9b8b0f10 con 0x564b9b950380 2024-06-10T01:06:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.647+0000 7f59b58b0640 1 --2- 172.21.15.37:0/4205134762 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564b9b950380 0x564b9b9527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.647+0000 7f59997fa640 10 client.17013 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:06:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.647+0000 7f59b58b0640 1 --2- 172.21.15.37:0/4205134762 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564b9b950380 0x564b9b9527d0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf4490a27fd85b91 server_cookie=61912799fb3aecd7 in_seq=0 out_seq=0 2024-06-10T01:06:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.653+0000 7f59997fa640 1 -- 172.21.15.37:0/4205134762 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564b9b8b0f10 con 0x564b9b950380 2024-06-10T01:06:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.653+0000 7f59997fa640 10 client.17013 handle_command_reply: tid=0 2024-06-10T01:06:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.653+0000 7f59ba824740 1 client.17013 shutdown 2024-06-10T01:06:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.653+0000 7f59ba824740 1 -- 172.21.15.37:0/4205134762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b9b93fb70 msgr2=0x564b9b94ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.653+0000 7f59ba824740 1 --2- 172.21.15.37:0/4205134762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b9b93fb70 0x564b9b94ba20 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f59a8009f80 tx=0x7f59a8004ab0 comp rx=0 tx=0).stop 2024-06-10T01:06:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.653+0000 7f59ba824740 1 -- 172.21.15.37:0/4205134762 shutdown_connections 2024-06-10T01:06:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.653+0000 7f59ba824740 1 --2- 172.21.15.37:0/4205134762 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564b9b950380 0x564b9b9527d0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.653+0000 7f59ba824740 1 --2- 172.21.15.37:0/4205134762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b9b93fb70 0x564b9b94ba20 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.653+0000 7f59ba824740 1 --2- 172.21.15.37:0/4205134762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9b93aa00 0x564b9b9449e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.653+0000 7f59997fa640 0 client.17013 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:06:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.653+0000 7f59ba824740 1 --2- 172.21.15.37:0/4205134762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b9b93a010 0x564b9b9444a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.653+0000 7f59ba824740 1 -- 172.21.15.37:0/4205134762 >> 172.21.15.37:0/4205134762 conn(0x564b9b935680 msgr2=0x564b9b9347e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.653+0000 7f59ba824740 1 -- 172.21.15.37:0/4205134762 shutdown_connections 2024-06-10T01:06:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.653+0000 7f59ba824740 1 -- 172.21.15.37:0/4205134762 wait complete. 2024-06-10T01:06:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.653+0000 7f59ba824740 20 client.17013 trim_cache size 0 max 16384 2024-06-10T01:06:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.670+0000 7f59ba824740 1 Processor -- start 2024-06-10T01:06:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.670+0000 7f59ba824740 1 -- start start 2024-06-10T01:06:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.670+0000 7f59ba824740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9b93a010 0x564b9b93be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.670+0000 7f59ba824740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b9b93aa00 0x564b9b93c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.670+0000 7f59ba824740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b9b93fb70 0x564b9b94b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.670+0000 7f59ba824740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f59b01b2770 con 0x564b9b93fb70 2024-06-10T01:06:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.670+0000 7f59ba824740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f59b01b2e40 con 0x564b9b93a010 2024-06-10T01:06:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.670+0000 7f59ba824740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f59b01b0750 con 0x564b9b93aa00 2024-06-10T01:06:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.670+0000 7f59b60b1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b9b93fb70 0x564b9b94b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.670+0000 7f59b60b1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b9b93fb70 0x564b9b94b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35506/0 (socket says 172.21.15.37:35506) 2024-06-10T01:06:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.670+0000 7f59b60b1640 1 -- 172.21.15.37:0/3537507950 learned_addr learned my addr 172.21.15.37:0/3537507950 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:06:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.670+0000 7f59b50af640 1 --2- 172.21.15.37:0/3537507950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b9b93aa00 0x564b9b93c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.670+0000 7f59b58b0640 1 --2- 172.21.15.37:0/3537507950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9b93a010 0x564b9b93be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.671+0000 7f59b60b1640 1 -- 172.21.15.37:0/3537507950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b9b93aa00 msgr2=0x564b9b93c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.671+0000 7f59b60b1640 1 --2- 172.21.15.37:0/3537507950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b9b93aa00 0x564b9b93c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.671+0000 7f59b60b1640 1 -- 172.21.15.37:0/3537507950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9b93a010 msgr2=0x564b9b93be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.671+0000 7f59b60b1640 1 --2- 172.21.15.37:0/3537507950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9b93a010 0x564b9b93be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.671+0000 7f59b60b1640 1 -- 172.21.15.37:0/3537507950 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b9b93a770 con 0x564b9b93fb70 2024-06-10T01:06:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.671+0000 7f59b60b1640 1 --2- 172.21.15.37:0/3537507950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b9b93fb70 0x564b9b94b9e0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f59a8011400 tx=0x7f59a8000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b575d3edc095af server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.671+0000 7f599b7fe640 1 -- 172.21.15.37:0/3537507950 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a8038020 con 0x564b9b93fb70 2024-06-10T01:06:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.671+0000 7f599b7fe640 1 -- 172.21.15.37:0/3537507950 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59a8022070 con 0x564b9b93fb70 2024-06-10T01:06:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.671+0000 7f599b7fe640 1 -- 172.21.15.37:0/3537507950 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a801e6b0 con 0x564b9b93fb70 2024-06-10T01:06:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.672+0000 7f59ba824740 1 -- 172.21.15.37:0/3537507950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b9b93fb70 msgr2=0x564b9b94b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.672+0000 7f59ba824740 1 --2- 172.21.15.37:0/3537507950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b9b93fb70 0x564b9b94b9e0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f59a8011400 tx=0x7f59a8000f40 comp rx=0 tx=0).stop 2024-06-10T01:06:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.672+0000 7f59ba824740 1 -- 172.21.15.37:0/3537507950 shutdown_connections 2024-06-10T01:06:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.672+0000 7f59ba824740 1 --2- 172.21.15.37:0/3537507950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b9b93fb70 0x564b9b94b9e0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.672+0000 7f59ba824740 1 --2- 172.21.15.37:0/3537507950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b9b93aa00 0x564b9b93c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.672+0000 7f59ba824740 1 --2- 172.21.15.37:0/3537507950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9b93a010 0x564b9b93be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.672+0000 7f59ba824740 1 -- 172.21.15.37:0/3537507950 >> 172.21.15.37:0/3537507950 conn(0x564b9b935680 msgr2=0x564b9b935a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.672+0000 7f59ba824740 1 -- 172.21.15.37:0/3537507950 shutdown_connections 2024-06-10T01:06:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.672+0000 7f59ba824740 1 -- 172.21.15.37:0/3537507950 wait complete. 2024-06-10T01:06:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.672+0000 7f59ba824740 1 -- start start 2024-06-10T01:06:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.672+0000 7f59ba824740 1 Processor -- start 2024-06-10T01:06:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.673+0000 7f59ba824740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9b93a010 0x564b9b948b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.673+0000 7f59ba824740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b9b93aa00 0x564b9b949080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.673+0000 7f59ba824740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b9b93fb70 0x564b9b94a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.673+0000 7f59ba824740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f59b01b0080 con 0x564b9b93fb70 2024-06-10T01:06:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.673+0000 7f59ba824740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f59b01aec70 con 0x564b9b93a010 2024-06-10T01:06:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.673+0000 7f59ba824740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f59b01af2e0 con 0x564b9b93aa00 2024-06-10T01:06:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.673+0000 7f59ba824740 10 client.0 _ll_register_callbacks cb 0x7f59a80125e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.673+0000 7f59b60b1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b9b93fb70 0x564b9b94a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.673+0000 7f59b50af640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b9b93aa00 0x564b9b949080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.673+0000 7f59b60b1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b9b93fb70 0x564b9b94a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35520/0 (socket says 172.21.15.37:35520) 2024-06-10T01:06:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.673+0000 7f59b60b1640 1 -- 172.21.15.37:0/2705781349 learned_addr learned my addr 172.21.15.37:0/2705781349 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:06:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.673+0000 7f59b58b0640 1 --2- 172.21.15.37:0/2705781349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9b93a010 0x564b9b948b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.673+0000 7f59b60b1640 1 -- 172.21.15.37:0/2705781349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b9b93aa00 msgr2=0x564b9b949080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.673+0000 7f59b60b1640 1 --2- 172.21.15.37:0/2705781349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b9b93aa00 0x564b9b949080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.673+0000 7f59b60b1640 1 -- 172.21.15.37:0/2705781349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9b93a010 msgr2=0x564b9b948b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.673+0000 7f59b60b1640 1 --2- 172.21.15.37:0/2705781349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9b93a010 0x564b9b948b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.673+0000 7f59b60b1640 1 -- 172.21.15.37:0/2705781349 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b9b94a630 con 0x564b9b93fb70 2024-06-10T01:06:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.673+0000 7f59b60b1640 1 --2- 172.21.15.37:0/2705781349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b9b93fb70 0x564b9b94a040 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f59a8011da0 tx=0x7f59a8028da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e21fe699b063664 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.673+0000 7f59997fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:06:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.673+0000 7f59997fa640 1 -- 172.21.15.37:0/2705781349 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a8038030 con 0x564b9b93fb70 2024-06-10T01:06:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.673+0000 7f59997fa640 1 -- 172.21.15.37:0/2705781349 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f59a801e660 con 0x564b9b93fb70 2024-06-10T01:06:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.673+0000 7f59ba824740 1 -- 172.21.15.37:0/2705781349 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59a8022070 con 0x564b9b93fb70 2024-06-10T01:06:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.674+0000 7f59997fa640 1 -- 172.21.15.37:0/2705781349 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f59a804d050 con 0x564b9b93fb70 2024-06-10T01:06:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.674+0000 7f59ba824740 10 client.17019 fetch_fsmap learned FSMap version 30 2024-06-10T01:06:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.674+0000 7f59ba824740 1 -- 172.21.15.37:0/2705781349 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564b9b94b940 con 0x564b9b93fb70 2024-06-10T01:06:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.674+0000 7f59997fa640 1 -- 172.21.15.37:0/2705781349 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59a8022070 con 0x564b9b93fb70 2024-06-10T01:06:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.674+0000 7f59997fa640 1 -- 172.21.15.37:0/2705781349 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 30) ==== 7741+0+0 (secure 0 0 0) 0x7f59a801bab0 con 0x564b9b93fb70 2024-06-10T01:06:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.674+0000 7f59ba824740 10 client.17019 fetch_fsmap finished waiting for FSMap version 30 2024-06-10T01:06:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.674+0000 7f59ba824740 10 client.17019 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:06:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.674+0000 7f59ba824740 20 client.17019 populate_metadata read hostname 'smithi037' 2024-06-10T01:06:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.674+0000 7f59ba824740 1 --2- 172.21.15.37:0/2705781349 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564b9b9bb330 0x564b9b9bd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.674+0000 7f59ba824740 4 client.17019 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:06:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.674+0000 7f59ba824740 1 -- 172.21.15.37:0/2705781349 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564b9b8aac20 con 0x564b9b9bb330 2024-06-10T01:06:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.674+0000 7f59b58b0640 1 --2- 172.21.15.37:0/2705781349 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564b9b9bb330 0x564b9b9bd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.675+0000 7f59b58b0640 1 --2- 172.21.15.37:0/2705781349 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564b9b9bb330 0x564b9b9bd720 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=364af6390dab1277 server_cookie=c540189f5123d5c7 in_seq=0 out_seq=0 2024-06-10T01:06:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:10.675+0000 7f59997fa640 10 client.17019 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:06:12.394 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:12 smithi167 ceph-mon[36917]: pgmap v716: 129 pgs: 129 active+clean; 38 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.1 MiB/s wr, 540 op/s 2024-06-10T01:06:12.395 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:06:12 smithi167 systemd[1]: ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.f.service: Scheduled restart job, restart counter is at 2. 2024-06-10T01:06:12.395 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:06:12 smithi167 systemd[1]: Stopped Ceph mds.f for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T01:06:12.395 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:06:12 smithi167 systemd[1]: ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.f.service: Consumed 20.297s CPU time. 2024-06-10T01:06:12.395 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:06:12 smithi167 systemd[1]: Starting Ceph mds.f for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:06:12.442 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:12 smithi037 ceph-mon[32192]: pgmap v716: 129 pgs: 129 active+clean; 38 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.1 MiB/s wr, 540 op/s 2024-06-10T01:06:12.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:12 smithi094 ceph-mon[36708]: pgmap v716: 129 pgs: 129 active+clean; 38 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.1 MiB/s wr, 540 op/s 2024-06-10T01:06:12.834 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:06:12 smithi167 podman[72130]: 2024-06-10 01:06:12.30424045 +0000 UTC m=+0.021524603 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:06:13.834 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:06:13 smithi167 podman[72130]: 2024-06-10 01:06:13.373179535 +0000 UTC m=+1.090463678 container create b7fd438c57f54b9d65c7146eaae64f591f495d7c99208d55f34f7db38241db64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T01:06:14.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:14 smithi167 ceph-mon[36917]: pgmap v717: 129 pgs: 129 active+clean; 42 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 586 op/s 2024-06-10T01:06:14.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:14 smithi094 ceph-mon[36708]: pgmap v717: 129 pgs: 129 active+clean; 42 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 586 op/s 2024-06-10T01:06:14.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:14 smithi037 ceph-mon[32192]: pgmap v717: 129 pgs: 129 active+clean; 42 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 586 op/s 2024-06-10T01:06:16.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:16 smithi167 ceph-mon[36917]: pgmap v718: 129 pgs: 129 active+clean; 46 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.2 MiB/s wr, 685 op/s 2024-06-10T01:06:16.585 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:06:16 smithi167 podman[72130]: 2024-06-10 01:06:16.127326039 +0000 UTC m=+3.844610183 container init b7fd438c57f54b9d65c7146eaae64f591f495d7c99208d55f34f7db38241db64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-f, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-06-10T01:06:16.585 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:06:16 smithi167 podman[72130]: 2024-06-10 01:06:16.131654196 +0000 UTC m=+3.848938348 container start b7fd438c57f54b9d65c7146eaae64f591f495d7c99208d55f34f7db38241db64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-f, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-06-10T01:06:16.585 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:06:16 smithi167 ceph-mds[72149]: -- 172.21.15.167:0/3410448680 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55791368ba40 con 0x55791365e800 2024-06-10T01:06:16.585 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:06:16 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-f[72143]: starting mds.f at 2024-06-10T01:06:16.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:16 smithi094 ceph-mon[36708]: pgmap v718: 129 pgs: 129 active+clean; 46 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.2 MiB/s wr, 685 op/s 2024-06-10T01:06:16.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:16 smithi037 ceph-mon[32192]: pgmap v718: 129 pgs: 129 active+clean; 46 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.2 MiB/s wr, 685 op/s 2024-06-10T01:06:17.334 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:17 smithi167 ceph-mon[36917]: mds.? [v2:172.21.15.167:6834/2107277223,v1:172.21.15.167:6835/2107277223] up:boot 2024-06-10T01:06:17.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:17 smithi167 ceph-mon[36917]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:06:17.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:17 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T01:06:17.335 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:06:17 smithi167 bash[72130]: b7fd438c57f54b9d65c7146eaae64f591f495d7c99208d55f34f7db38241db64 2024-06-10T01:06:17.335 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:06:17 smithi167 systemd[1]: Started Ceph mds.f for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T01:06:17.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:17 smithi094 ceph-mon[36708]: mds.? [v2:172.21.15.167:6834/2107277223,v1:172.21.15.167:6835/2107277223] up:boot 2024-06-10T01:06:17.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:17 smithi094 ceph-mon[36708]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:06:17.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:17 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T01:06:17.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:17 smithi037 ceph-mon[32192]: mds.? [v2:172.21.15.167:6834/2107277223,v1:172.21.15.167:6835/2107277223] up:boot 2024-06-10T01:06:17.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:17 smithi037 ceph-mon[32192]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:06:17.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:17 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-10T01:06:18.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:18 smithi167 ceph-mon[36917]: pgmap v719: 129 pgs: 129 active+clean; 46 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.2 MiB/s wr, 514 op/s 2024-06-10T01:06:18.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:18 smithi094 ceph-mon[36708]: pgmap v719: 129 pgs: 129 active+clean; 46 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.2 MiB/s wr, 514 op/s 2024-06-10T01:06:18.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:18 smithi037 ceph-mon[32192]: pgmap v719: 129 pgs: 129 active+clean; 46 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.2 MiB/s wr, 514 op/s 2024-06-10T01:06:20.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:20 smithi167 ceph-mon[36917]: pgmap v720: 129 pgs: 129 active+clean; 51 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.5 MiB/s wr, 603 op/s 2024-06-10T01:06:20.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:20 smithi094 ceph-mon[36708]: pgmap v720: 129 pgs: 129 active+clean; 51 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.5 MiB/s wr, 603 op/s 2024-06-10T01:06:20.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:20 smithi037 ceph-mon[32192]: pgmap v720: 129 pgs: 129 active+clean; 51 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.5 MiB/s wr, 603 op/s 2024-06-10T01:06:22.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:22 smithi167 ceph-mon[36917]: pgmap v721: 129 pgs: 129 active+clean; 51 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.1 MiB/s wr, 529 op/s 2024-06-10T01:06:22.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:22 smithi094 ceph-mon[36708]: pgmap v721: 129 pgs: 129 active+clean; 51 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.1 MiB/s wr, 529 op/s 2024-06-10T01:06:22.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:22 smithi037 ceph-mon[32192]: pgmap v721: 129 pgs: 129 active+clean; 51 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.1 MiB/s wr, 529 op/s 2024-06-10T01:06:23.069 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 16.0 2024-06-10T01:06:23.070 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:06:23.486 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:06:24.339 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:06:24.376 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:24 smithi037 ceph-mon[32192]: pgmap v722: 129 pgs: 129 active+clean; 51 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.2 MiB/s wr, 725 op/s 2024-06-10T01:06:24.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:24 smithi167 ceph-mon[36917]: pgmap v722: 129 pgs: 129 active+clean; 51 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.2 MiB/s wr, 725 op/s 2024-06-10T01:06:24.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:24 smithi094 ceph-mon[36708]: pgmap v722: 129 pgs: 129 active+clean; 51 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.2 MiB/s wr, 725 op/s 2024-06-10T01:06:25.860 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:25 smithi094 ceph-mon[36708]: pgmap v723: 129 pgs: 129 active+clean; 58 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 999 op/s 2024-06-10T01:06:25.891 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:25 smithi037 ceph-mon[32192]: pgmap v723: 129 pgs: 129 active+clean; 58 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 999 op/s 2024-06-10T01:06:26.084 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:25 smithi167 ceph-mon[36917]: pgmap v723: 129 pgs: 129 active+clean; 58 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 999 op/s 2024-06-10T01:06:27.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.313+0000 7f193921a700 1 -- 172.21.15.37:0/3130753406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1934108560 msgr2=0x7f1934108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:27.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.313+0000 7f193921a700 1 --2- 172.21.15.37:0/3130753406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1934108560 0x7f1934108940 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1924009990 tx=0x7f1924009ca0 comp rx=0 tx=0).stop 2024-06-10T01:06:27.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.313+0000 7f193921a700 1 -- 172.21.15.37:0/3130753406 shutdown_connections 2024-06-10T01:06:27.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.313+0000 7f193921a700 1 --2- 172.21.15.37:0/3130753406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19341013a0 0x7f193410f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:27.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.313+0000 7f193921a700 1 --2- 172.21.15.37:0/3130753406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1934108560 0x7f1934108940 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:27.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.313+0000 7f193921a700 1 --2- 172.21.15.37:0/3130753406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19341009e0 0x7f1934100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:27.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.313+0000 7f193921a700 1 -- 172.21.15.37:0/3130753406 >> 172.21.15.37:0/3130753406 conn(0x7f1934077990 msgr2=0x7f19340fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:27.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.313+0000 7f193921a700 1 -- 172.21.15.37:0/3130753406 shutdown_connections 2024-06-10T01:06:27.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.313+0000 7f193921a700 1 -- 172.21.15.37:0/3130753406 wait complete. 2024-06-10T01:06:27.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.314+0000 7f193921a700 1 Processor -- start 2024-06-10T01:06:27.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.314+0000 7f193921a700 1 -- start start 2024-06-10T01:06:27.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.314+0000 7f193921a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19341009e0 0x7f193419aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:27.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.314+0000 7f193921a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19341013a0 0x7f193419b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:27.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.314+0000 7f193921a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1934108560 0x7f193419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:27.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.314+0000 7f193921a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f19341127c0 con 0x7f19341013a0 2024-06-10T01:06:27.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.314+0000 7f193921a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1934112640 con 0x7f1934108560 2024-06-10T01:06:27.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.314+0000 7f193921a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1934112940 con 0x7f19341009e0 2024-06-10T01:06:27.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.314+0000 7f193259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19341013a0 0x7f193419b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:27.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.314+0000 7f1932d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19341009e0 0x7f193419aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.314+0000 7f193259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19341013a0 0x7f193419b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56668/0 (socket says 172.21.15.37:56668) 2024-06-10T01:06:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.314+0000 7f193259c700 1 -- 172.21.15.37:0/3168296803 learned_addr learned my addr 172.21.15.37:0/3168296803 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:06:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.314+0000 7f193359e700 1 --2- 172.21.15.37:0/3168296803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1934108560 0x7f193419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.315+0000 7f193259c700 1 -- 172.21.15.37:0/3168296803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19341009e0 msgr2=0x7f193419aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.315+0000 7f193259c700 1 --2- 172.21.15.37:0/3168296803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19341009e0 0x7f193419aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.315+0000 7f193259c700 1 -- 172.21.15.37:0/3168296803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1934108560 msgr2=0x7f193419f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.315+0000 7f193259c700 1 --2- 172.21.15.37:0/3168296803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1934108560 0x7f193419f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.315+0000 7f193259c700 1 -- 172.21.15.37:0/3168296803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193419feb0 con 0x7f19341013a0 2024-06-10T01:06:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.315+0000 7f1932d9d700 1 --2- 172.21.15.37:0/3168296803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19341009e0 0x7f193419aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.315+0000 7f193359e700 1 --2- 172.21.15.37:0/3168296803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1934108560 0x7f193419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.315+0000 7f193259c700 1 --2- 172.21.15.37:0/3168296803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19341013a0 0x7f193419b430 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f192800e9a0 tx=0x7f192800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.315+0000 7f191bfff700 1 -- 172.21.15.37:0/3168296803 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192800c9c0 con 0x7f19341013a0 2024-06-10T01:06:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.315+0000 7f193921a700 1 -- 172.21.15.37:0/3168296803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19341a01a0 con 0x7f19341013a0 2024-06-10T01:06:27.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.315+0000 7f191bfff700 1 -- 172.21.15.37:0/3168296803 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f192800cb60 con 0x7f19341013a0 2024-06-10T01:06:27.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.315+0000 7f191bfff700 1 -- 172.21.15.37:0/3168296803 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928018680 con 0x7f19341013a0 2024-06-10T01:06:27.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.315+0000 7f193921a700 1 -- 172.21.15.37:0/3168296803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19341a7a20 con 0x7f19341013a0 2024-06-10T01:06:27.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.316+0000 7f193921a700 1 -- 172.21.15.37:0/3168296803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1934069180 con 0x7f19341013a0 2024-06-10T01:06:27.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.317+0000 7f191bfff700 1 -- 172.21.15.37:0/3168296803 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f192800cd10 con 0x7f19341013a0 2024-06-10T01:06:27.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.318+0000 7f191bfff700 1 --2- 172.21.15.37:0/3168296803 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f191c0a2c20 0x7f191c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:27.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.318+0000 7f191bfff700 1 -- 172.21.15.37:0/3168296803 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f19280cf5e0 con 0x7f19341013a0 2024-06-10T01:06:27.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.318+0000 7f1932d9d700 1 --2- 172.21.15.37:0/3168296803 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f191c0a2c20 0x7f191c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:27.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.319+0000 7f1932d9d700 1 --2- 172.21.15.37:0/3168296803 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f191c0a2c20 0x7f191c0a50e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f19240098e0 tx=0x7f1924005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:27.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.323+0000 7f191bfff700 1 -- 172.21.15.37:0/3168296803 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f192809c0f0 con 0x7f19341013a0 2024-06-10T01:06:27.342 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:06:27.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.588+0000 7f193921a700 1 -- 172.21.15.37:0/3168296803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19341a7c50 con 0x7f19341013a0 2024-06-10T01:06:27.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.588+0000 7f191bfff700 1 -- 172.21.15.37:0/3168296803 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11868 (secure 0 0 0) 0x7f192809b9f0 con 0x7f19341013a0 2024-06-10T01:06:27.585 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:06:27.585 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":26554,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2107277223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2107277223},{"type":"v1","addr":"172.21.15.167:6835","nonce":2107277223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T01:06:09.604504+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24472,24451,24458]},"id":1}]} 2024-06-10T01:06:27.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.591+0000 7f193921a700 1 -- 172.21.15.37:0/3168296803 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f191c0a2c20 msgr2=0x7f191c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:27.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.591+0000 7f193921a700 1 --2- 172.21.15.37:0/3168296803 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f191c0a2c20 0x7f191c0a50e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f19240098e0 tx=0x7f1924005960 comp rx=0 tx=0).stop 2024-06-10T01:06:27.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.591+0000 7f193921a700 1 -- 172.21.15.37:0/3168296803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19341013a0 msgr2=0x7f193419b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:27.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.591+0000 7f193921a700 1 --2- 172.21.15.37:0/3168296803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19341013a0 0x7f193419b430 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f192800e9a0 tx=0x7f192800ecb0 comp rx=0 tx=0).stop 2024-06-10T01:06:27.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.591+0000 7f193921a700 1 -- 172.21.15.37:0/3168296803 shutdown_connections 2024-06-10T01:06:27.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.591+0000 7f193921a700 1 --2- 172.21.15.37:0/3168296803 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f191c0a2c20 0x7f191c0a50e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:27.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.591+0000 7f193921a700 1 --2- 172.21.15.37:0/3168296803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1934108560 0x7f193419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:27.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.591+0000 7f193921a700 1 --2- 172.21.15.37:0/3168296803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19341009e0 0x7f193419aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:27.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.591+0000 7f193921a700 1 --2- 172.21.15.37:0/3168296803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19341013a0 0x7f193419b430 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:27.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.591+0000 7f193921a700 1 -- 172.21.15.37:0/3168296803 >> 172.21.15.37:0/3168296803 conn(0x7f1934077990 msgr2=0x7f1934107820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:27.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.591+0000 7f193921a700 1 -- 172.21.15.37:0/3168296803 shutdown_connections 2024-06-10T01:06:27.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:27.591+0000 7f193921a700 1 -- 172.21.15.37:0/3168296803 wait complete. 2024-06-10T01:06:27.588 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 31 2024-06-10T01:06:28.141 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:28 smithi037 ceph-mon[32192]: pgmap v724: 129 pgs: 129 active+clean; 61 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 864 op/s 2024-06-10T01:06:28.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:28 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3168296803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:06:28.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:28 smithi094 ceph-mon[36708]: pgmap v724: 129 pgs: 129 active+clean; 61 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 864 op/s 2024-06-10T01:06:28.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:28 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3168296803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:06:28.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:28 smithi167 ceph-mon[36917]: pgmap v724: 129 pgs: 129 active+clean; 61 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 864 op/s 2024-06-10T01:06:28.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:28 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3168296803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:06:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:28.677+0000 7f59997fa640 1 -- 172.21.15.37:0/2705781349 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564b9b8aac20 con 0x564b9b9bb330 2024-06-10T01:06:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:28.677+0000 7f59997fa640 10 client.17019 handle_command_reply: tid=0 2024-06-10T01:06:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:28.677+0000 7f59ba824740 1 client.17019 shutdown 2024-06-10T01:06:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:28.677+0000 7f59ba824740 1 -- 172.21.15.37:0/2705781349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b9b93fb70 msgr2=0x564b9b94a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:28.677+0000 7f59ba824740 1 --2- 172.21.15.37:0/2705781349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b9b93fb70 0x564b9b94a040 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f59a8011da0 tx=0x7f59a8028da0 comp rx=0 tx=0).stop 2024-06-10T01:06:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:28.677+0000 7f59ba824740 1 -- 172.21.15.37:0/2705781349 shutdown_connections 2024-06-10T01:06:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:28.677+0000 7f59ba824740 1 --2- 172.21.15.37:0/2705781349 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564b9b9bb330 0x564b9b9bd720 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:28.677+0000 7f59ba824740 1 --2- 172.21.15.37:0/2705781349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b9b93fb70 0x564b9b94a040 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:28.677+0000 7f59997fa640 0 client.17019 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:06:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:28.677+0000 7f59ba824740 1 --2- 172.21.15.37:0/2705781349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b9b93aa00 0x564b9b949080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:28.677+0000 7f59ba824740 1 --2- 172.21.15.37:0/2705781349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564b9b93a010 0x564b9b948b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:28.677+0000 7f59ba824740 1 -- 172.21.15.37:0/2705781349 >> 172.21.15.37:0/2705781349 conn(0x564b9b935680 msgr2=0x564b9b93be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:28.678+0000 7f59ba824740 1 -- 172.21.15.37:0/2705781349 shutdown_connections 2024-06-10T01:06:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:28.678+0000 7f59ba824740 1 -- 172.21.15.37:0/2705781349 wait complete. 2024-06-10T01:06:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:28.678+0000 7f59ba824740 20 client.17019 trim_cache size 0 max 16384 2024-06-10T01:06:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:06:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:06:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:06:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:28.678+0000 7f59b7b14640 1 -- 172.21.15.37:0/1244586222 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f599c0a2c30 msgr2=0x7f599c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:28.678+0000 7f59b7b14640 1 --2- 172.21.15.37:0/1244586222 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f599c0a2c30 0x7f599c0a50f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x564b9b93b260 tx=0x7f59a8004a40 comp rx=0 tx=0).stop 2024-06-10T01:06:28.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:28.678+0000 7f59b7b14640 1 -- 172.21.15.37:0/1244586222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b0100900 msgr2=0x7f59b0104840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:28.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:28.678+0000 7f59b7b14640 1 --2- 172.21.15.37:0/1244586222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b0100900 0x7f59b0104840 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f59ac00e9c0 tx=0x7f59ac00ee80 comp rx=0 tx=0).stop 2024-06-10T01:06:28.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:28.678+0000 7f59b7b14640 1 -- 172.21.15.37:0/1244586222 shutdown_connections 2024-06-10T01:06:28.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:28.678+0000 7f59b7b14640 1 --2- 172.21.15.37:0/1244586222 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f599c0a2c30 0x7f599c0a50f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:28.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:28.678+0000 7f59b7b14640 1 --2- 172.21.15.37:0/1244586222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59b01024b0 0x7f59b0103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:28.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:28.678+0000 7f59b7b14640 1 --2- 172.21.15.37:0/1244586222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59b0101af0 0x7f59b0102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:28.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:28.678+0000 7f59b7b14640 1 --2- 172.21.15.37:0/1244586222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b0100900 0x7f59b0104840 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:28.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:28.678+0000 7f59b7b14640 1 -- 172.21.15.37:0/1244586222 >> 172.21.15.37:0/1244586222 conn(0x7f59b00fbe70 msgr2=0x7f59b00fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:28.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:28.678+0000 7f59b7b14640 1 -- 172.21.15.37:0/1244586222 shutdown_connections 2024-06-10T01:06:28.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:28.678+0000 7f59b7b14640 1 -- 172.21.15.37:0/1244586222 wait complete. 2024-06-10T01:06:28.811 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24472 quiesce db / --timeout 90 --expiration 136.0 --await-for 45.0' 2024-06-10T01:06:28.812 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24472 quiesce db / --timeout 90 --expiration 136.0 --await-for 45.0 2024-06-10T01:06:29.198 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:06:29.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.830+0000 7f954cfd8700 1 -- 172.21.15.37:0/1409529003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9548101d50 msgr2=0x7f954810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:29.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.830+0000 7f954cfd8700 1 --2- 172.21.15.37:0/1409529003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9548101d50 0x7f954810e240 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f954000b0a0 tx=0x7f954000b3b0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.830+0000 7f954cfd8700 1 -- 172.21.15.37:0/1409529003 shutdown_connections 2024-06-10T01:06:29.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.830+0000 7f954cfd8700 1 --2- 172.21.15.37:0/1409529003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9548101d50 0x7f954810e240 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.830+0000 7f954cfd8700 1 --2- 172.21.15.37:0/1409529003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9548107400 0x7f95481077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.830+0000 7f954cfd8700 1 --2- 172.21.15.37:0/1409529003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9548101390 0x7f9548101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.830+0000 7f954cfd8700 1 -- 172.21.15.37:0/1409529003 >> 172.21.15.37:0/1409529003 conn(0x7f9548078340 msgr2=0x7f9548078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:29.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.830+0000 7f954cfd8700 1 -- 172.21.15.37:0/1409529003 shutdown_connections 2024-06-10T01:06:29.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.830+0000 7f954cfd8700 1 -- 172.21.15.37:0/1409529003 wait complete. 2024-06-10T01:06:29.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.831+0000 7f954cfd8700 1 Processor -- start 2024-06-10T01:06:29.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.832+0000 7f954cfd8700 1 -- start start 2024-06-10T01:06:29.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.832+0000 7f954cfd8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9548101390 0x7f954819e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:29.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.832+0000 7f954cfd8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9548101d50 0x7f954819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:29.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.832+0000 7f954cfd8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9548107400 0x7f9548198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:29.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.832+0000 7f954cfd8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9548111690 con 0x7f9548101390 2024-06-10T01:06:29.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.832+0000 7f954cfd8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9548111510 con 0x7f9548101d50 2024-06-10T01:06:29.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.832+0000 7f954cfd8700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9548111810 con 0x7f9548107400 2024-06-10T01:06:29.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.832+0000 7f954659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9548101390 0x7f954819e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:29.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.832+0000 7f954659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9548101390 0x7f954819e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56686/0 (socket says 172.21.15.37:56686) 2024-06-10T01:06:29.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.832+0000 7f954659c700 1 -- 172.21.15.37:0/1978749467 learned_addr learned my addr 172.21.15.37:0/1978749467 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:06:29.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.832+0000 7f9546d9d700 1 --2- 172.21.15.37:0/1978749467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9548107400 0x7f9548198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:29.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.832+0000 7f953dd9b700 1 --2- 172.21.15.37:0/1978749467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9548101d50 0x7f954819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:29.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.832+0000 7f954659c700 1 -- 172.21.15.37:0/1978749467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9548107400 msgr2=0x7f9548198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:29.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.832+0000 7f954659c700 1 --2- 172.21.15.37:0/1978749467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9548107400 0x7f9548198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.832+0000 7f954659c700 1 -- 172.21.15.37:0/1978749467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9548101d50 msgr2=0x7f954819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:29.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.832+0000 7f954659c700 1 --2- 172.21.15.37:0/1978749467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9548101d50 0x7f954819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.832+0000 7f954659c700 1 -- 172.21.15.37:0/1978749467 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9548199170 con 0x7f9548101390 2024-06-10T01:06:29.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.832+0000 7f9546d9d700 1 --2- 172.21.15.37:0/1978749467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9548107400 0x7f9548198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:29.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.833+0000 7f954659c700 1 --2- 172.21.15.37:0/1978749467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9548101390 0x7f954819e800 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f953800e9c0 tx=0x7f953800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:29.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.833+0000 7f953ffff700 1 -- 172.21.15.37:0/1978749467 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953800cbd0 con 0x7f9548101390 2024-06-10T01:06:29.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.833+0000 7f953ffff700 1 -- 172.21.15.37:0/1978749467 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f953800ee60 con 0x7f9548101390 2024-06-10T01:06:29.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.833+0000 7f954cfd8700 1 -- 172.21.15.37:0/1978749467 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9548199460 con 0x7f9548101390 2024-06-10T01:06:29.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.833+0000 7f953ffff700 1 -- 172.21.15.37:0/1978749467 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9538005630 con 0x7f9548101390 2024-06-10T01:06:29.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.833+0000 7f954cfd8700 1 -- 172.21.15.37:0/1978749467 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95481a56d0 con 0x7f9548101390 2024-06-10T01:06:29.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.835+0000 7f953ffff700 1 -- 172.21.15.37:0/1978749467 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9538004750 con 0x7f9548101390 2024-06-10T01:06:29.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.836+0000 7f953ffff700 1 --2- 172.21.15.37:0/1978749467 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f95340a2c20 0x7f95340a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:29.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.836+0000 7f953dd9b700 1 --2- 172.21.15.37:0/1978749467 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f95340a2c20 0x7f95340a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:29.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.836+0000 7f953ffff700 1 -- 172.21.15.37:0/1978749467 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f95380cf650 con 0x7f9548101390 2024-06-10T01:06:29.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.836+0000 7f953dd9b700 1 --2- 172.21.15.37:0/1978749467 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f95340a2c20 0x7f95340a50e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9548078920 tx=0x7f953000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:29.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.837+0000 7f955785bb80 1 Processor -- start 2024-06-10T01:06:29.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.837+0000 7f955785bb80 1 -- start start 2024-06-10T01:06:29.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.838+0000 7f955785bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fb0e0640 0x55d3fb0e0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:29.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.838+0000 7f955785bb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3fb0e0f60 0x55d3fb0da9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:29.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.838+0000 7f955785bb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fb0daf20 0x55d3fb0db340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:29.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.838+0000 7f955785bb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d3fb02c810 con 0x55d3fb0daf20 2024-06-10T01:06:29.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.838+0000 7f955785bb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d3fafd4720 con 0x55d3fb0e0f60 2024-06-10T01:06:29.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.838+0000 7f955785bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d3fb0287d0 con 0x55d3fb0e0640 2024-06-10T01:06:29.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.838+0000 7f9546d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fb0daf20 0x55d3fb0db340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:29.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.838+0000 7f954659c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fb0e0640 0x55d3fb0e0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:29.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.838+0000 7f9546d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fb0daf20 0x55d3fb0db340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56700/0 (socket says 172.21.15.37:56700) 2024-06-10T01:06:29.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.838+0000 7f9546d9d700 1 -- 172.21.15.37:0/385388168 learned_addr learned my addr 172.21.15.37:0/385388168 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:06:29.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.838+0000 7f953dd9b700 1 --2- 172.21.15.37:0/385388168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3fb0e0f60 0x55d3fb0da9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:29.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.838+0000 7f954659c700 1 --2- 172.21.15.37:0/385388168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fb0e0640 0x55d3fb0e0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:44182/0 (socket says 172.21.15.37:44182) 2024-06-10T01:06:29.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.838+0000 7f953dd9b700 1 -- 172.21.15.37:0/385388168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fb0e0640 msgr2=0x55d3fb0e0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:29.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.838+0000 7f953dd9b700 1 --2- 172.21.15.37:0/385388168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fb0e0640 0x55d3fb0e0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.838+0000 7f953dd9b700 1 -- 172.21.15.37:0/385388168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fb0daf20 msgr2=0x55d3fb0db340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:29.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.838+0000 7f953dd9b700 1 --2- 172.21.15.37:0/385388168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fb0daf20 0x55d3fb0db340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.838+0000 7f953dd9b700 1 -- 172.21.15.37:0/385388168 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3fafcaf50 con 0x55d3fb0e0f60 2024-06-10T01:06:29.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.838+0000 7f9546d9d700 1 --2- 172.21.15.37:0/385388168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fb0daf20 0x55d3fb0db340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:29.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.838+0000 7f954659c700 1 --2- 172.21.15.37:0/385388168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fb0e0640 0x55d3fb0e0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:29.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.838+0000 7f953dd9b700 1 --2- 172.21.15.37:0/385388168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3fb0e0f60 0x55d3fb0da9e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9530005d90 tx=0x7f9530013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a65a79626246f427 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:29.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.838+0000 7f953cd99700 1 -- 172.21.15.37:0/385388168 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530011b80 con 0x55d3fb0e0f60 2024-06-10T01:06:29.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.839+0000 7f953cd99700 1 -- 172.21.15.37:0/385388168 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9530011d20 con 0x55d3fb0e0f60 2024-06-10T01:06:29.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.839+0000 7f953cd99700 1 -- 172.21.15.37:0/385388168 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953001e420 con 0x55d3fb0e0f60 2024-06-10T01:06:29.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.839+0000 7f955785bb80 1 -- 172.21.15.37:0/385388168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3fb0e0f60 msgr2=0x55d3fb0da9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:29.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.839+0000 7f955785bb80 1 --2- 172.21.15.37:0/385388168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3fb0e0f60 0x55d3fb0da9e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9530005d90 tx=0x7f9530013000 comp rx=0 tx=0).stop 2024-06-10T01:06:29.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.839+0000 7f955785bb80 1 -- 172.21.15.37:0/385388168 shutdown_connections 2024-06-10T01:06:29.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.839+0000 7f955785bb80 1 --2- 172.21.15.37:0/385388168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fb0daf20 0x55d3fb0db340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.839+0000 7f955785bb80 1 --2- 172.21.15.37:0/385388168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fb0e0640 0x55d3fb0e0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.839+0000 7f955785bb80 1 --2- 172.21.15.37:0/385388168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3fb0e0f60 0x55d3fb0da9e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.839+0000 7f955785bb80 1 -- 172.21.15.37:0/385388168 >> 172.21.15.37:0/385388168 conn(0x55d3fb005cc0 msgr2=0x55d3fb0d5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:29.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.839+0000 7f955785bb80 1 -- 172.21.15.37:0/385388168 shutdown_connections 2024-06-10T01:06:29.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.839+0000 7f955785bb80 1 -- 172.21.15.37:0/385388168 wait complete. 2024-06-10T01:06:29.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.840+0000 7f955785bb80 1 -- start start 2024-06-10T01:06:29.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.840+0000 7f955785bb80 1 Processor -- start 2024-06-10T01:06:29.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.840+0000 7f955785bb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3fb0daf20 0x55d3fb0e6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:29.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.840+0000 7f955785bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fb0e0640 0x55d3fb0e4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:29.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.840+0000 7f955785bb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fb0e0f60 0x55d3fb0e45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:29.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.840+0000 7f955785bb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d3fb0e79d0 con 0x55d3fb0e0f60 2024-06-10T01:06:29.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.840+0000 7f955785bb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d3fafccbf0 con 0x55d3fb0daf20 2024-06-10T01:06:29.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.840+0000 7f955785bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d3fb0e7b50 con 0x55d3fb0e0640 2024-06-10T01:06:29.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.840+0000 7f955785bb80 10 client.0 _ll_register_callbacks cb 0x55d3fafc6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:29.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.840+0000 7f953dd9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fb0e0640 0x55d3fb0e4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:29.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.840+0000 7f9546d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fb0e0f60 0x55d3fb0e45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:29.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.840+0000 7f954659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3fb0daf20 0x55d3fb0e6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:29.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.840+0000 7f953dd9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fb0e0640 0x55d3fb0e4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:44188/0 (socket says 172.21.15.37:44188) 2024-06-10T01:06:29.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.840+0000 7f953dd9b700 1 -- 172.21.15.37:0/2878879969 learned_addr learned my addr 172.21.15.37:0/2878879969 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:06:29.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.840+0000 7f954659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3fb0daf20 0x55d3fb0e6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:55130/0 (socket says 172.21.15.37:55130) 2024-06-10T01:06:29.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.841+0000 7f954659c700 1 -- 172.21.15.37:0/2878879969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fb0e0640 msgr2=0x55d3fb0e4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:29.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.841+0000 7f954659c700 1 --2- 172.21.15.37:0/2878879969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fb0e0640 0x55d3fb0e4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.841+0000 7f954659c700 1 -- 172.21.15.37:0/2878879969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fb0e0f60 msgr2=0x55d3fb0e45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:29.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.841+0000 7f954659c700 1 --2- 172.21.15.37:0/2878879969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fb0e0f60 0x55d3fb0e45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.841+0000 7f954659c700 1 -- 172.21.15.37:0/2878879969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3faf7b8d0 con 0x55d3fb0daf20 2024-06-10T01:06:29.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.841+0000 7f954659c700 1 --2- 172.21.15.37:0/2878879969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3fb0daf20 0x55d3fb0e6d60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f95380cd0d0 tx=0x7f953809c2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37d83fc84d26c60a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:29.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.841+0000 7f952dffb700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:06:29.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.841+0000 7f952dffb700 1 -- 172.21.15.37:0/2878879969 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95380b9060 con 0x55d3fb0daf20 2024-06-10T01:06:29.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.841+0000 7f952dffb700 1 -- 172.21.15.37:0/2878879969 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f953800cd70 con 0x55d3fb0daf20 2024-06-10T01:06:29.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.841+0000 7f955785bb80 1 -- 172.21.15.37:0/2878879969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9530011d20 con 0x55d3fb0daf20 2024-06-10T01:06:29.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.841+0000 7f952dffb700 1 -- 172.21.15.37:0/2878879969 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f95380b3030 con 0x55d3fb0daf20 2024-06-10T01:06:29.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.841+0000 7f955785bb80 10 client.26444 fetch_fsmap learned FSMap version 31 2024-06-10T01:06:29.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.841+0000 7f955785bb80 1 -- 172.21.15.37:0/2878879969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d3fb0f23e0 con 0x55d3fb0daf20 2024-06-10T01:06:29.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.841+0000 7f952dffb700 1 -- 172.21.15.37:0/2878879969 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9530011d20 con 0x55d3fb0daf20 2024-06-10T01:06:29.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.842+0000 7f952dffb700 1 -- 172.21.15.37:0/2878879969 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f953809bd20 con 0x55d3fb0daf20 2024-06-10T01:06:29.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.842+0000 7f955785bb80 10 client.26444 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:06:29.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.842+0000 7f955785bb80 10 client.26444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:06:29.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.842+0000 7f955785bb80 20 client.26444 populate_metadata read hostname 'smithi037' 2024-06-10T01:06:29.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.842+0000 7f955785bb80 1 --2- 172.21.15.37:0/2878879969 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55d3fb0f4a20 0x55d3fb0f6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:29.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.842+0000 7f955785bb80 4 client.26444 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:06:29.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.842+0000 7f955785bb80 1 -- 172.21.15.37:0/2878879969 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3fb039d10 con 0x55d3fb0f4a20 2024-06-10T01:06:29.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.842+0000 7f953dd9b700 1 --2- 172.21.15.37:0/2878879969 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55d3fb0f4a20 0x55d3fb0f6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:29.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.842+0000 7f953dd9b700 1 --2- 172.21.15.37:0/2878879969 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55d3fb0f4a20 0x55d3fb0f6e10 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66e066cf117b569b server_cookie=f62a31783f86c4f9 in_seq=0 out_seq=0 2024-06-10T01:06:29.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.842+0000 7f952dffb700 10 client.26444 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:06:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.847+0000 7f952dffb700 1 -- 172.21.15.37:0/2878879969 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d3fb039d10 con 0x55d3fb0f4a20 2024-06-10T01:06:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.847+0000 7f952dffb700 10 client.26444 handle_command_reply: tid=0 2024-06-10T01:06:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.847+0000 7f955785bb80 1 client.26444 shutdown 2024-06-10T01:06:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.847+0000 7f955785bb80 1 -- 172.21.15.37:0/2878879969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3fb0daf20 msgr2=0x55d3fb0e6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.847+0000 7f955785bb80 1 --2- 172.21.15.37:0/2878879969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3fb0daf20 0x55d3fb0e6d60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f95380cd0d0 tx=0x7f953809c2b0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.847+0000 7f955785bb80 1 -- 172.21.15.37:0/2878879969 shutdown_connections 2024-06-10T01:06:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.847+0000 7f955785bb80 1 --2- 172.21.15.37:0/2878879969 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55d3fb0f4a20 0x55d3fb0f6e10 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.847+0000 7f955785bb80 1 --2- 172.21.15.37:0/2878879969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fb0e0f60 0x55d3fb0e45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.847+0000 7f955785bb80 1 --2- 172.21.15.37:0/2878879969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3fb0daf20 0x55d3fb0e6d60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.847+0000 7f952dffb700 0 client.26444 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:06:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.847+0000 7f955785bb80 1 --2- 172.21.15.37:0/2878879969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fb0e0640 0x55d3fb0e4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.847+0000 7f955785bb80 1 -- 172.21.15.37:0/2878879969 >> 172.21.15.37:0/2878879969 conn(0x55d3fb005cc0 msgr2=0x55d3fb0d99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:29.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.847+0000 7f955785bb80 1 -- 172.21.15.37:0/2878879969 shutdown_connections 2024-06-10T01:06:29.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.847+0000 7f955785bb80 1 -- 172.21.15.37:0/2878879969 wait complete. 2024-06-10T01:06:29.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.848+0000 7f955785bb80 20 client.26444 trim_cache size 0 max 16384 2024-06-10T01:06:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.869+0000 7f955785bb80 1 Processor -- start 2024-06-10T01:06:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.869+0000 7f955785bb80 1 -- start start 2024-06-10T01:06:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.869+0000 7f955785bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fb0daf20 0x55d3fb0da9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.869+0000 7f955785bb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fb0e0640 0x55d3fb0f0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:29.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.869+0000 7f955785bb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3fb0e0f60 0x55d3fb0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:29.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.869+0000 7f955785bb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f95481b0d70 con 0x55d3fb0e0640 2024-06-10T01:06:29.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.869+0000 7f955785bb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f95481ae600 con 0x55d3fb0e0f60 2024-06-10T01:06:29.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.869+0000 7f955785bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f95481ac450 con 0x55d3fb0daf20 2024-06-10T01:06:29.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.869+0000 7f953dd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fb0e0640 0x55d3fb0f0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:29.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.869+0000 7f954659c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fb0daf20 0x55d3fb0da9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:29.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.869+0000 7f9546d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3fb0e0f60 0x55d3fb0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:29.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.869+0000 7f953dd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fb0e0640 0x55d3fb0f0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56730/0 (socket says 172.21.15.37:56730) 2024-06-10T01:06:29.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.869+0000 7f9546d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3fb0e0f60 0x55d3fb0cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:55132/0 (socket says 172.21.15.37:55132) 2024-06-10T01:06:29.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.869+0000 7f953dd9b700 1 -- 172.21.15.37:0/532221641 learned_addr learned my addr 172.21.15.37:0/532221641 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:06:29.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.869+0000 7f954659c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fb0daf20 0x55d3fb0da9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:44196/0 (socket says 172.21.15.37:44196) 2024-06-10T01:06:29.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.869+0000 7f9546d9d700 1 -- 172.21.15.37:0/532221641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fb0daf20 msgr2=0x55d3fb0da9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:29.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.869+0000 7f9546d9d700 1 --2- 172.21.15.37:0/532221641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fb0daf20 0x55d3fb0da9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.869+0000 7f9546d9d700 1 -- 172.21.15.37:0/532221641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fb0e0640 msgr2=0x55d3fb0f0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:29.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.869+0000 7f9546d9d700 1 --2- 172.21.15.37:0/532221641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fb0e0640 0x55d3fb0f0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.869+0000 7f9546d9d700 1 -- 172.21.15.37:0/532221641 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3fb0d5380 con 0x55d3fb0e0f60 2024-06-10T01:06:29.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.870+0000 7f9546d9d700 1 --2- 172.21.15.37:0/532221641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3fb0e0f60 0x55d3fb0cf690 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9540009fd0 tx=0x7f954000b720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce1a5ff392bcbac9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:29.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.870+0000 7f952ffff700 1 -- 172.21.15.37:0/532221641 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954002a070 con 0x55d3fb0e0f60 2024-06-10T01:06:29.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.870+0000 7f952ffff700 1 -- 172.21.15.37:0/532221641 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95400074e0 con 0x55d3fb0e0f60 2024-06-10T01:06:29.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.870+0000 7f952ffff700 1 -- 172.21.15.37:0/532221641 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9540005280 con 0x55d3fb0e0f60 2024-06-10T01:06:29.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.871+0000 7f955785bb80 1 -- 172.21.15.37:0/532221641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3fb0e0f60 msgr2=0x55d3fb0cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:29.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.871+0000 7f955785bb80 1 --2- 172.21.15.37:0/532221641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3fb0e0f60 0x55d3fb0cf690 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9540009fd0 tx=0x7f954000b720 comp rx=0 tx=0).stop 2024-06-10T01:06:29.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.871+0000 7f955785bb80 1 -- 172.21.15.37:0/532221641 shutdown_connections 2024-06-10T01:06:29.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.871+0000 7f955785bb80 1 --2- 172.21.15.37:0/532221641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3fb0e0f60 0x55d3fb0cf690 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.871+0000 7f955785bb80 1 --2- 172.21.15.37:0/532221641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fb0daf20 0x55d3fb0da9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.871+0000 7f955785bb80 1 --2- 172.21.15.37:0/532221641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fb0e0640 0x55d3fb0f0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.871+0000 7f955785bb80 1 -- 172.21.15.37:0/532221641 >> 172.21.15.37:0/532221641 conn(0x55d3fb005cc0 msgr2=0x55d3fb0e1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:29.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.871+0000 7f955785bb80 1 -- 172.21.15.37:0/532221641 shutdown_connections 2024-06-10T01:06:29.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.871+0000 7f955785bb80 1 -- 172.21.15.37:0/532221641 wait complete. 2024-06-10T01:06:29.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.871+0000 7f955785bb80 1 -- start start 2024-06-10T01:06:29.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.871+0000 7f955785bb80 1 Processor -- start 2024-06-10T01:06:29.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.871+0000 7f955785bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fb0daf20 0x55d3fb0fc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:29.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.871+0000 7f955785bb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3fb0e0640 0x55d3fb0fc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:29.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.871+0000 7f955785bb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fb0e0f60 0x55d3fb0fce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:29.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.871+0000 7f955785bb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f95481acac0 con 0x55d3fb0e0f60 2024-06-10T01:06:29.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.871+0000 7f955785bb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f95481adf30 con 0x55d3fb0e0640 2024-06-10T01:06:29.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.871+0000 7f955785bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f95481ad190 con 0x55d3fb0daf20 2024-06-10T01:06:29.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.871+0000 7f955785bb80 10 client.0 _ll_register_callbacks cb 0x55d3fafc6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:29.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.872+0000 7f954659c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fb0daf20 0x55d3fb0fc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:29.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.872+0000 7f9546d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fb0e0f60 0x55d3fb0fce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:29.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.872+0000 7f953dd9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3fb0e0640 0x55d3fb0fc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:29.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.872+0000 7f954659c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fb0daf20 0x55d3fb0fc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:44202/0 (socket says 172.21.15.37:44202) 2024-06-10T01:06:29.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.872+0000 7f954659c700 1 -- 172.21.15.37:0/33852212 learned_addr learned my addr 172.21.15.37:0/33852212 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:06:29.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.872+0000 7f954659c700 1 -- 172.21.15.37:0/33852212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3fb0e0640 msgr2=0x55d3fb0fc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:29.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.872+0000 7f954659c700 1 --2- 172.21.15.37:0/33852212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3fb0e0640 0x55d3fb0fc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.872+0000 7f954659c700 1 -- 172.21.15.37:0/33852212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fb0e0f60 msgr2=0x55d3fb0fce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:29.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.872+0000 7f954659c700 1 --2- 172.21.15.37:0/33852212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fb0e0f60 0x55d3fb0fce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.872+0000 7f954659c700 1 -- 172.21.15.37:0/33852212 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3fb15e780 con 0x55d3fb0daf20 2024-06-10T01:06:29.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.872+0000 7f953dd9b700 1 --2- 172.21.15.37:0/33852212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3fb0e0640 0x55d3fb0fc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:29.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.872+0000 7f9546d9d700 1 --2- 172.21.15.37:0/33852212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fb0e0f60 0x55d3fb0fce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:29.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.872+0000 7f954659c700 1 --2- 172.21.15.37:0/33852212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fb0daf20 0x55d3fb0fc410 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9538097f20 tx=0x7f953809ffa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f48294387f9b8ec6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:29.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.872+0000 7f952dffb700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:06:29.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.872+0000 7f952dffb700 1 -- 172.21.15.37:0/33852212 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95380ba390 con 0x55d3fb0daf20 2024-06-10T01:06:29.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.872+0000 7f955785bb80 1 -- 172.21.15.37:0/33852212 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95400074e0 con 0x55d3fb0daf20 2024-06-10T01:06:29.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.872+0000 7f952dffb700 1 -- 172.21.15.37:0/33852212 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f953800cd70 con 0x55d3fb0daf20 2024-06-10T01:06:29.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.873+0000 7f952dffb700 1 -- 172.21.15.37:0/33852212 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f95380a4730 con 0x55d3fb0daf20 2024-06-10T01:06:29.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.873+0000 7f955785bb80 10 client.26578 fetch_fsmap learned FSMap version 31 2024-06-10T01:06:29.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.873+0000 7f955785bb80 1 -- 172.21.15.37:0/33852212 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d3fb15fa70 con 0x55d3fb0daf20 2024-06-10T01:06:29.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.873+0000 7f952dffb700 1 -- 172.21.15.37:0/33852212 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95400074e0 con 0x55d3fb0daf20 2024-06-10T01:06:29.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.873+0000 7f952dffb700 1 -- 172.21.15.37:0/33852212 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f953809afa0 con 0x55d3fb0daf20 2024-06-10T01:06:29.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.873+0000 7f955785bb80 10 client.26578 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:06:29.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.873+0000 7f955785bb80 10 client.26578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:06:29.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.873+0000 7f955785bb80 20 client.26578 populate_metadata read hostname 'smithi037' 2024-06-10T01:06:29.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.873+0000 7f955785bb80 1 --2- 172.21.15.37:0/33852212 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55d3fb160150 0x55d3fb162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:29.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.873+0000 7f955785bb80 4 client.26578 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:06:29.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.873+0000 7f955785bb80 1 -- 172.21.15.37:0/33852212 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d3fb034290 con 0x55d3fb160150 2024-06-10T01:06:29.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.873+0000 7f953dd9b700 1 --2- 172.21.15.37:0/33852212 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55d3fb160150 0x55d3fb162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:29.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.874+0000 7f952dffb700 10 client.26578 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:06:29.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.874+0000 7f953dd9b700 1 --2- 172.21.15.37:0/33852212 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55d3fb160150 0x55d3fb162570 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f5a18662a88e989 server_cookie=de177da3b818b1b5 in_seq=0 out_seq=0 2024-06-10T01:06:29.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.882+0000 7f952dffb700 1 -- 172.21.15.37:0/33852212 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55d3fb034290 con 0x55d3fb160150 2024-06-10T01:06:29.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.882+0000 7f952dffb700 10 client.26578 handle_command_reply: tid=0 2024-06-10T01:06:29.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.882+0000 7f955785bb80 1 client.26578 shutdown 2024-06-10T01:06:29.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.882+0000 7f955785bb80 1 -- 172.21.15.37:0/33852212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fb0daf20 msgr2=0x55d3fb0fc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:29.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.882+0000 7f955785bb80 1 --2- 172.21.15.37:0/33852212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fb0daf20 0x55d3fb0fc410 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9538097f20 tx=0x7f953809ffa0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.882+0000 7f955785bb80 1 -- 172.21.15.37:0/33852212 shutdown_connections 2024-06-10T01:06:29.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.882+0000 7f955785bb80 1 --2- 172.21.15.37:0/33852212 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55d3fb160150 0x55d3fb162570 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.882+0000 7f955785bb80 1 --2- 172.21.15.37:0/33852212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fb0e0f60 0x55d3fb0fce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.882+0000 7f955785bb80 1 --2- 172.21.15.37:0/33852212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d3fb0daf20 0x55d3fb0fc410 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.882+0000 7f952dffb700 0 client.26578 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:06:29.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.882+0000 7f955785bb80 1 --2- 172.21.15.37:0/33852212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d3fb0e0640 0x55d3fb0fc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.882+0000 7f955785bb80 1 -- 172.21.15.37:0/33852212 >> 172.21.15.37:0/33852212 conn(0x55d3fb005cc0 msgr2=0x55d3fb0df1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:29.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.882+0000 7f955785bb80 1 -- 172.21.15.37:0/33852212 shutdown_connections 2024-06-10T01:06:29.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.882+0000 7f955785bb80 1 -- 172.21.15.37:0/33852212 wait complete. 2024-06-10T01:06:29.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.882+0000 7f955785bb80 20 client.26578 trim_cache size 0 max 16384 2024-06-10T01:06:29.880 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:06:29.880 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "scrub active (8495 inodes in the stack)", 2024-06-10T01:06:29.880 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-06-10T01:06:29.880 INFO:teuthology.orchestra.run.smithi037.stdout: "d79c0c89-cbf2-47ac-9a2b-0a728f0b2713": { 2024-06-10T01:06:29.880 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-06-10T01:06:29.880 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "d79c0c89-cbf2-47ac-9a2b-0a728f0b2713", 2024-06-10T01:06:29.881 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-06-10T01:06:29.881 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:06:29.881 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:06:29.881 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:06:29.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.883+0000 7f954cfd8700 1 -- 172.21.15.37:0/1978749467 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f95340a2c20 msgr2=0x7f95340a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:29.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.883+0000 7f954cfd8700 1 --2- 172.21.15.37:0/1978749467 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f95340a2c20 0x7f95340a50e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9548078920 tx=0x7f953000b410 comp rx=0 tx=0).stop 2024-06-10T01:06:29.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.883+0000 7f954cfd8700 1 -- 172.21.15.37:0/1978749467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9548101390 msgr2=0x7f954819e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:29.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.883+0000 7f954cfd8700 1 --2- 172.21.15.37:0/1978749467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9548101390 0x7f954819e800 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f953800e9c0 tx=0x7f953800ecd0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.883+0000 7f954cfd8700 1 -- 172.21.15.37:0/1978749467 shutdown_connections 2024-06-10T01:06:29.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.883+0000 7f954cfd8700 1 --2- 172.21.15.37:0/1978749467 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f95340a2c20 0x7f95340a50e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.883+0000 7f954cfd8700 1 --2- 172.21.15.37:0/1978749467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9548107400 0x7f9548198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.883+0000 7f954cfd8700 1 --2- 172.21.15.37:0/1978749467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9548101390 0x7f954819e800 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.883+0000 7f954cfd8700 1 --2- 172.21.15.37:0/1978749467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9548101d50 0x7f954819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:29.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.883+0000 7f954cfd8700 1 -- 172.21.15.37:0/1978749467 >> 172.21.15.37:0/1978749467 conn(0x7f9548078340 msgr2=0x7f95481005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:29.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.883+0000 7f954cfd8700 1 -- 172.21.15.37:0/1978749467 shutdown_connections 2024-06-10T01:06:29.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:29.883+0000 7f954cfd8700 1 -- 172.21.15.37:0/1978749467 wait complete. 2024-06-10T01:06:30.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:30 smithi094 ceph-mon[36708]: pgmap v725: 129 pgs: 129 active+clean; 61 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 1.05k op/s 2024-06-10T01:06:30.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:30 smithi037 ceph-mon[32192]: pgmap v725: 129 pgs: 129 active+clean; 61 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 1.05k op/s 2024-06-10T01:06:30.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:30 smithi167 ceph-mon[36917]: pgmap v725: 129 pgs: 129 active+clean; 61 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 1.05k op/s 2024-06-10T01:06:31.879 INFO:tasks.cephfs.filesystem:scrub status for tag:d79c0c89-cbf2-47ac-9a2b-0a728f0b2713 - {'path': '/', 'tag': 'd79c0c89-cbf2-47ac-9a2b-0a728f0b2713', 'options': 'recursive,force'} 2024-06-10T01:06:32.141 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:32 smithi037 ceph-mon[32192]: pgmap v726: 129 pgs: 129 active+clean; 61 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 1.3 MiB/s wr, 969 op/s 2024-06-10T01:06:32.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:32 smithi094 ceph-mon[36708]: pgmap v726: 129 pgs: 129 active+clean; 61 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 1.3 MiB/s wr, 969 op/s 2024-06-10T01:06:32.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:32 smithi167 ceph-mon[36917]: pgmap v726: 129 pgs: 129 active+clean; 61 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 1.3 MiB/s wr, 969 op/s 2024-06-10T01:06:32.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.959+0000 7fbd1bfbb700 1 -- 172.21.15.37:0/3304916211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd14102770 msgr2=0x7fbd14102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:32.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.959+0000 7fbd1bfbb700 1 --2- 172.21.15.37:0/3304916211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd14102770 0x7fbd14102b50 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fbd100099c0 tx=0x7fbd10009cd0 comp rx=0 tx=0).stop 2024-06-10T01:06:32.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.959+0000 7fbd1bfbb700 1 -- 172.21.15.37:0/3304916211 shutdown_connections 2024-06-10T01:06:32.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.959+0000 7fbd1bfbb700 1 --2- 172.21.15.37:0/3304916211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd14107760 0x7fbd1410f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:32.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.959+0000 7fbd1bfbb700 1 --2- 172.21.15.37:0/3304916211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd14102770 0x7fbd14102b50 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:32.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.959+0000 7fbd1bfbb700 1 --2- 172.21.15.37:0/3304916211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd14103120 0x7fbd14107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:32.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.959+0000 7fbd1bfbb700 1 -- 172.21.15.37:0/3304916211 >> 172.21.15.37:0/3304916211 conn(0x7fbd14079560 msgr2=0x7fbd14079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:32.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.959+0000 7fbd1bfbb700 1 -- 172.21.15.37:0/3304916211 shutdown_connections 2024-06-10T01:06:32.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.959+0000 7fbd1bfbb700 1 -- 172.21.15.37:0/3304916211 wait complete. 2024-06-10T01:06:32.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.960+0000 7fbd1bfbb700 1 Processor -- start 2024-06-10T01:06:32.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.960+0000 7fbd1bfbb700 1 -- start start 2024-06-10T01:06:32.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.960+0000 7fbd1bfbb700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd14102770 0x7fbd1419e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:32.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.960+0000 7fbd1bfbb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd14103120 0x7fbd1419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:32.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.960+0000 7fbd1bfbb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd14107760 0x7fbd14198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:32.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.960+0000 7fbd1bfbb700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbd14112780 con 0x7fbd14107760 2024-06-10T01:06:32.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.960+0000 7fbd1bfbb700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbd14112600 con 0x7fbd14103120 2024-06-10T01:06:32.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.960+0000 7fbd1bfbb700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbd14112900 con 0x7fbd14102770 2024-06-10T01:06:32.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.960+0000 7fbd1a558700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd14107760 0x7fbd14198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:32.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.960+0000 7fbd1a558700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd14107760 0x7fbd14198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59082/0 (socket says 172.21.15.37:59082) 2024-06-10T01:06:32.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.960+0000 7fbd19d57700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd14102770 0x7fbd1419e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:32.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.960+0000 7fbd1a558700 1 -- 172.21.15.37:0/324283275 learned_addr learned my addr 172.21.15.37:0/324283275 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:06:32.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.960+0000 7fbd19556700 1 --2- 172.21.15.37:0/324283275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd14103120 0x7fbd1419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:32.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.961+0000 7fbd1a558700 1 -- 172.21.15.37:0/324283275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd14102770 msgr2=0x7fbd1419e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:32.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.961+0000 7fbd1a558700 1 --2- 172.21.15.37:0/324283275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd14102770 0x7fbd1419e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:32.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.961+0000 7fbd1a558700 1 -- 172.21.15.37:0/324283275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd14103120 msgr2=0x7fbd1419ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:32.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.961+0000 7fbd1a558700 1 --2- 172.21.15.37:0/324283275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd14103120 0x7fbd1419ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:32.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.961+0000 7fbd1a558700 1 -- 172.21.15.37:0/324283275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd14198fe0 con 0x7fbd14107760 2024-06-10T01:06:32.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.961+0000 7fbd19d57700 1 --2- 172.21.15.37:0/324283275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd14102770 0x7fbd1419e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:32.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.961+0000 7fbd19556700 1 --2- 172.21.15.37:0/324283275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd14103120 0x7fbd1419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:32.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.961+0000 7fbd1a558700 1 --2- 172.21.15.37:0/324283275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd14107760 0x7fbd14198880 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c00ef30 tx=0x7fbd0c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:32.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.961+0000 7fbd0affd700 1 -- 172.21.15.37:0/324283275 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c0092e0 con 0x7fbd14107760 2024-06-10T01:06:32.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.961+0000 7fbd0affd700 1 -- 172.21.15.37:0/324283275 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd0c004bb0 con 0x7fbd14107760 2024-06-10T01:06:32.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.961+0000 7fbd0affd700 1 -- 172.21.15.37:0/324283275 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c004280 con 0x7fbd14107760 2024-06-10T01:06:32.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.961+0000 7fbd1bfbb700 1 -- 172.21.15.37:0/324283275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd14199270 con 0x7fbd14107760 2024-06-10T01:06:32.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.961+0000 7fbd1bfbb700 1 -- 172.21.15.37:0/324283275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd141a5590 con 0x7fbd14107760 2024-06-10T01:06:32.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.963+0000 7fbd0affd700 1 -- 172.21.15.37:0/324283275 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fbd0c01b030 con 0x7fbd14107760 2024-06-10T01:06:32.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.964+0000 7fbd0affd700 1 --2- 172.21.15.37:0/324283275 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fbd000a2d40 0x7fbd000a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:32.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.964+0000 7fbd19d57700 1 --2- 172.21.15.37:0/324283275 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fbd000a2d40 0x7fbd000a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:32.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.964+0000 7fbd0affd700 1 -- 172.21.15.37:0/324283275 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fbd0c0cef20 con 0x7fbd14107760 2024-06-10T01:06:32.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.965+0000 7fbd19d57700 1 --2- 172.21.15.37:0/324283275 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fbd000a2d40 0x7fbd000a5200 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbd141999c0 tx=0x7fbd10005a90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:32.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.965+0000 7fbd2683eb80 1 Processor -- start 2024-06-10T01:06:32.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.965+0000 7fbd2683eb80 1 -- start start 2024-06-10T01:06:32.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.966+0000 7fbd2683eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab9222590 0x564ab9222970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:32.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.966+0000 7fbd2683eb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ab9222eb0 0x564ab921c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:32.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.966+0000 7fbd2683eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ab921cce0 0x564ab921d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:32.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.966+0000 7fbd2683eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564ab910f0f0 con 0x564ab9222590 2024-06-10T01:06:32.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.966+0000 7fbd2683eb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564ab916d9e0 con 0x564ab9222eb0 2024-06-10T01:06:32.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.966+0000 7fbd2683eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564ab90d55f0 con 0x564ab921cce0 2024-06-10T01:06:32.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.966+0000 7fbd19d57700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab9222590 0x564ab9222970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:32.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.966+0000 7fbd19d57700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab9222590 0x564ab9222970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59090/0 (socket says 172.21.15.37:59090) 2024-06-10T01:06:32.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.966+0000 7fbd19d57700 1 -- 172.21.15.37:0/2109435876 learned_addr learned my addr 172.21.15.37:0/2109435876 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:06:32.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.966+0000 7fbd1a558700 1 --2- 172.21.15.37:0/2109435876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ab921cce0 0x564ab921d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:32.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.966+0000 7fbd19556700 1 --2- 172.21.15.37:0/2109435876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ab9222eb0 0x564ab921c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:32.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.966+0000 7fbd19d57700 1 -- 172.21.15.37:0/2109435876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ab921cce0 msgr2=0x564ab921d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:32.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.966+0000 7fbd19d57700 1 --2- 172.21.15.37:0/2109435876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ab921cce0 0x564ab921d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:32.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.966+0000 7fbd19d57700 1 -- 172.21.15.37:0/2109435876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ab9222eb0 msgr2=0x564ab921c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:32.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.966+0000 7fbd19d57700 1 --2- 172.21.15.37:0/2109435876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ab9222eb0 0x564ab921c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:32.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.966+0000 7fbd19d57700 1 -- 172.21.15.37:0/2109435876 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ab90bca60 con 0x564ab9222590 2024-06-10T01:06:32.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.966+0000 7fbd19d57700 1 --2- 172.21.15.37:0/2109435876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab9222590 0x564ab9222970 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fbd100099c0 tx=0x7fbd10020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa5e85699dd1eeb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:32.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.967+0000 7fbcfffff700 1 -- 172.21.15.37:0/2109435876 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd10034070 con 0x564ab9222590 2024-06-10T01:06:32.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.967+0000 7fbcfffff700 1 -- 172.21.15.37:0/2109435876 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd100095b0 con 0x564ab9222590 2024-06-10T01:06:32.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.967+0000 7fbcfffff700 1 -- 172.21.15.37:0/2109435876 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd10024410 con 0x564ab9222590 2024-06-10T01:06:32.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.967+0000 7fbd2683eb80 1 -- 172.21.15.37:0/2109435876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab9222590 msgr2=0x564ab9222970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:32.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.967+0000 7fbd2683eb80 1 --2- 172.21.15.37:0/2109435876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab9222590 0x564ab9222970 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fbd100099c0 tx=0x7fbd10020000 comp rx=0 tx=0).stop 2024-06-10T01:06:32.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.967+0000 7fbd2683eb80 1 -- 172.21.15.37:0/2109435876 shutdown_connections 2024-06-10T01:06:32.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.967+0000 7fbd2683eb80 1 --2- 172.21.15.37:0/2109435876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ab921cce0 0x564ab921d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:32.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.967+0000 7fbd2683eb80 1 --2- 172.21.15.37:0/2109435876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab9222590 0x564ab9222970 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:32.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.967+0000 7fbd2683eb80 1 --2- 172.21.15.37:0/2109435876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ab9222eb0 0x564ab921c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:32.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.967+0000 7fbd2683eb80 1 -- 172.21.15.37:0/2109435876 >> 172.21.15.37:0/2109435876 conn(0x564ab9149210 msgr2=0x564ab9217a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:32.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.968+0000 7fbd2683eb80 1 -- 172.21.15.37:0/2109435876 shutdown_connections 2024-06-10T01:06:32.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.968+0000 7fbd2683eb80 1 -- 172.21.15.37:0/2109435876 wait complete. 2024-06-10T01:06:32.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.968+0000 7fbd2683eb80 1 -- start start 2024-06-10T01:06:32.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.968+0000 7fbd2683eb80 1 Processor -- start 2024-06-10T01:06:32.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.969+0000 7fbd2683eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab921cce0 0x564ab9228c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:32.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.969+0000 7fbd2683eb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ab9222590 0x564ab9225f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:32.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.969+0000 7fbd2683eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ab9222eb0 0x564ab9226480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:32.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.969+0000 7fbd2683eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564ab910e150 con 0x564ab921cce0 2024-06-10T01:06:32.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.969+0000 7fbd2683eb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564ab91699a0 con 0x564ab9222590 2024-06-10T01:06:32.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.969+0000 7fbd19d57700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab921cce0 0x564ab9228c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:32.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.969+0000 7fbd2683eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564ab9229830 con 0x564ab9222eb0 2024-06-10T01:06:32.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.969+0000 7fbd2683eb80 10 client.0 _ll_register_callbacks cb 0x564ab9103740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:32.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.969+0000 7fbd19556700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ab9222590 0x564ab9225f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:32.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.969+0000 7fbd19556700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ab9222590 0x564ab9225f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:35776/0 (socket says 172.21.15.37:35776) 2024-06-10T01:06:32.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.969+0000 7fbd19556700 1 -- 172.21.15.37:0/3007899450 learned_addr learned my addr 172.21.15.37:0/3007899450 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:06:32.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.969+0000 7fbd1a558700 1 --2- 172.21.15.37:0/3007899450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ab9222eb0 0x564ab9226480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:32.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.969+0000 7fbd19556700 1 -- 172.21.15.37:0/3007899450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ab9222eb0 msgr2=0x564ab9226480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:32.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.969+0000 7fbd19556700 1 --2- 172.21.15.37:0/3007899450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ab9222eb0 0x564ab9226480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:32.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.969+0000 7fbd19556700 1 -- 172.21.15.37:0/3007899450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab921cce0 msgr2=0x564ab9228c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:32.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.969+0000 7fbd19556700 1 --2- 172.21.15.37:0/3007899450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab921cce0 0x564ab9228c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:32.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.969+0000 7fbd19556700 1 -- 172.21.15.37:0/3007899450 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ab91338e0 con 0x564ab9222590 2024-06-10T01:06:32.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.969+0000 7fbd19d57700 1 --2- 172.21.15.37:0/3007899450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab921cce0 0x564ab9228c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:32.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.969+0000 7fbd19556700 1 --2- 172.21.15.37:0/3007899450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ab9222590 0x564ab9225f40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbd04011a60 tx=0x7fbd04011d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=432d1155fae7505d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:32.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.969+0000 7fbcfd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:06:32.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.970+0000 7fbcfd7fa700 1 -- 172.21.15.37:0/3007899450 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd04004520 con 0x564ab9222590 2024-06-10T01:06:32.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.970+0000 7fbcfd7fa700 1 -- 172.21.15.37:0/3007899450 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fbd0400f8b0 con 0x564ab9222590 2024-06-10T01:06:32.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.970+0000 7fbd2683eb80 1 -- 172.21.15.37:0/3007899450 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd100095b0 con 0x564ab9222590 2024-06-10T01:06:32.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.970+0000 7fbcfd7fa700 1 -- 172.21.15.37:0/3007899450 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbd0400fab0 con 0x564ab9222590 2024-06-10T01:06:32.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.970+0000 7fbd2683eb80 10 client.26459 fetch_fsmap learned FSMap version 31 2024-06-10T01:06:32.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.970+0000 7fbd2683eb80 1 -- 172.21.15.37:0/3007899450 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564ab9234040 con 0x564ab9222590 2024-06-10T01:06:32.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.970+0000 7fbcfd7fa700 1 -- 172.21.15.37:0/3007899450 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd100095b0 con 0x564ab9222590 2024-06-10T01:06:32.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.970+0000 7fbcfd7fa700 1 -- 172.21.15.37:0/3007899450 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fbd04018030 con 0x564ab9222590 2024-06-10T01:06:32.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.970+0000 7fbd2683eb80 10 client.26459 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:06:32.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.970+0000 7fbd2683eb80 10 client.26459 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:06:32.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.970+0000 7fbd2683eb80 20 client.26459 populate_metadata read hostname 'smithi037' 2024-06-10T01:06:32.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.970+0000 7fbd2683eb80 1 --2- 172.21.15.37:0/3007899450 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564ab9236680 0x564ab9238a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:32.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.970+0000 7fbd2683eb80 4 client.26459 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:06:32.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.970+0000 7fbd2683eb80 1 -- 172.21.15.37:0/3007899450 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ab917aee0 con 0x564ab9236680 2024-06-10T01:06:32.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.971+0000 7fbd19d57700 1 --2- 172.21.15.37:0/3007899450 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564ab9236680 0x564ab9238a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:32.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.971+0000 7fbd19d57700 1 --2- 172.21.15.37:0/3007899450 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564ab9236680 0x564ab9238a70 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a26c994285d40885 server_cookie=a5f65ff98c57220c in_seq=0 out_seq=0 2024-06-10T01:06:32.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.971+0000 7fbcfd7fa700 10 client.26459 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:06:32.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.975+0000 7fbcfd7fa700 1 -- 172.21.15.37:0/3007899450 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564ab917aee0 con 0x564ab9236680 2024-06-10T01:06:32.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.975+0000 7fbcfd7fa700 10 client.26459 handle_command_reply: tid=0 2024-06-10T01:06:32.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.975+0000 7fbd2683eb80 1 client.26459 shutdown 2024-06-10T01:06:32.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.975+0000 7fbd2683eb80 1 -- 172.21.15.37:0/3007899450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ab9222590 msgr2=0x564ab9225f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:32.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.975+0000 7fbd2683eb80 1 --2- 172.21.15.37:0/3007899450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ab9222590 0x564ab9225f40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbd04011a60 tx=0x7fbd04011d70 comp rx=0 tx=0).stop 2024-06-10T01:06:32.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.975+0000 7fbd2683eb80 1 -- 172.21.15.37:0/3007899450 shutdown_connections 2024-06-10T01:06:32.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.975+0000 7fbd2683eb80 1 --2- 172.21.15.37:0/3007899450 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564ab9236680 0x564ab9238a70 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:32.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.975+0000 7fbd2683eb80 1 --2- 172.21.15.37:0/3007899450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ab9222eb0 0x564ab9226480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:32.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.975+0000 7fbd2683eb80 1 --2- 172.21.15.37:0/3007899450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab921cce0 0x564ab9228c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:32.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.975+0000 7fbd2683eb80 1 --2- 172.21.15.37:0/3007899450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ab9222590 0x564ab9225f40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:32.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.975+0000 7fbd2683eb80 1 -- 172.21.15.37:0/3007899450 >> 172.21.15.37:0/3007899450 conn(0x564ab9149210 msgr2=0x564ab9216c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:32.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.975+0000 7fbcfd7fa700 0 client.26459 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:06:32.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.975+0000 7fbd2683eb80 1 -- 172.21.15.37:0/3007899450 shutdown_connections 2024-06-10T01:06:32.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.975+0000 7fbd2683eb80 1 -- 172.21.15.37:0/3007899450 wait complete. 2024-06-10T01:06:32.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.975+0000 7fbd2683eb80 20 client.26459 trim_cache size 0 max 16384 2024-06-10T01:06:32.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.996+0000 7fbd2683eb80 1 Processor -- start 2024-06-10T01:06:32.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.996+0000 7fbd2683eb80 1 -- start start 2024-06-10T01:06:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.996+0000 7fbd2683eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab921cce0 0x564ab9228780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.996+0000 7fbd2683eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ab9222590 0x564ab92173c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.996+0000 7fbd2683eb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ab9222eb0 0x564ab9217900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.996+0000 7fbd2683eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbd141b0ba0 con 0x564ab921cce0 2024-06-10T01:06:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.996+0000 7fbd2683eb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbd141ae430 con 0x564ab9222eb0 2024-06-10T01:06:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.996+0000 7fbd2683eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbd141ac280 con 0x564ab9222590 2024-06-10T01:06:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.997+0000 7fbd19d57700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab921cce0 0x564ab9228780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.997+0000 7fbd19556700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ab9222590 0x564ab92173c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.997+0000 7fbd19d57700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab921cce0 0x564ab9228780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59108/0 (socket says 172.21.15.37:59108) 2024-06-10T01:06:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.997+0000 7fbd19d57700 1 -- 172.21.15.37:0/1580235921 learned_addr learned my addr 172.21.15.37:0/1580235921 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:06:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.997+0000 7fbd1a558700 1 --2- 172.21.15.37:0/1580235921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ab9222eb0 0x564ab9217900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.997+0000 7fbd19d57700 1 -- 172.21.15.37:0/1580235921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ab9222590 msgr2=0x564ab92173c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.997+0000 7fbd19d57700 1 --2- 172.21.15.37:0/1580235921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ab9222590 0x564ab92173c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.997+0000 7fbd19d57700 1 -- 172.21.15.37:0/1580235921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ab9222eb0 msgr2=0x564ab9217900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.997+0000 7fbd19d57700 1 --2- 172.21.15.37:0/1580235921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ab9222eb0 0x564ab9217900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.997+0000 7fbd19d57700 1 -- 172.21.15.37:0/1580235921 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ab9222030 con 0x564ab921cce0 2024-06-10T01:06:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.997+0000 7fbd19556700 1 --2- 172.21.15.37:0/1580235921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ab9222590 0x564ab92173c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:32.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.997+0000 7fbd1a558700 1 --2- 172.21.15.37:0/1580235921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ab9222eb0 0x564ab9217900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:06:32.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.997+0000 7fbd19d57700 1 --2- 172.21.15.37:0/1580235921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab921cce0 0x564ab9228780 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fbd10006010 tx=0x7fbd1001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10953c5129013513 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:32.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.997+0000 7fbcff7fe700 1 -- 172.21.15.37:0/1580235921 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd10034040 con 0x564ab921cce0 2024-06-10T01:06:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.998+0000 7fbcff7fe700 1 -- 172.21.15.37:0/1580235921 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd10040070 con 0x564ab921cce0 2024-06-10T01:06:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.998+0000 7fbcff7fe700 1 -- 172.21.15.37:0/1580235921 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd10044ba0 con 0x564ab921cce0 2024-06-10T01:06:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.998+0000 7fbd2683eb80 1 -- 172.21.15.37:0/1580235921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab921cce0 msgr2=0x564ab9228780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.998+0000 7fbd2683eb80 1 --2- 172.21.15.37:0/1580235921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab921cce0 0x564ab9228780 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fbd10006010 tx=0x7fbd1001a040 comp rx=0 tx=0).stop 2024-06-10T01:06:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.998+0000 7fbd2683eb80 1 -- 172.21.15.37:0/1580235921 shutdown_connections 2024-06-10T01:06:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.998+0000 7fbd2683eb80 1 --2- 172.21.15.37:0/1580235921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ab9222eb0 0x564ab9217900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.998+0000 7fbd2683eb80 1 --2- 172.21.15.37:0/1580235921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab921cce0 0x564ab9228780 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.998+0000 7fbd2683eb80 1 --2- 172.21.15.37:0/1580235921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ab9222590 0x564ab92173c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.998+0000 7fbd2683eb80 1 -- 172.21.15.37:0/1580235921 >> 172.21.15.37:0/1580235921 conn(0x564ab9149210 msgr2=0x564ab9221920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.998+0000 7fbd2683eb80 1 -- 172.21.15.37:0/1580235921 shutdown_connections 2024-06-10T01:06:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.998+0000 7fbd2683eb80 1 -- 172.21.15.37:0/1580235921 wait complete. 2024-06-10T01:06:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.999+0000 7fbd2683eb80 1 -- start start 2024-06-10T01:06:32.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.999+0000 7fbd2683eb80 1 Processor -- start 2024-06-10T01:06:32.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.999+0000 7fbd2683eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ab921cce0 0x564ab922f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:32.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.999+0000 7fbd2683eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab9222590 0x564ab922bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:32.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.999+0000 7fbd2683eb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ab9222eb0 0x564ab922c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:32.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.999+0000 7fbd2683eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbd141ac8f0 con 0x564ab9222590 2024-06-10T01:06:32.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.999+0000 7fbd2683eb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbd141add60 con 0x564ab9222eb0 2024-06-10T01:06:32.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.999+0000 7fbd2683eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbd141acfc0 con 0x564ab921cce0 2024-06-10T01:06:32.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.999+0000 7fbd2683eb80 10 client.0 _ll_register_callbacks cb 0x564ab9103740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:32.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:32.999+0000 7fbd19556700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab9222590 0x564ab922bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:32.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.000+0000 7fbd19d57700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ab921cce0 0x564ab922f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:32.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.000+0000 7fbd19556700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab9222590 0x564ab922bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59122/0 (socket says 172.21.15.37:59122) 2024-06-10T01:06:32.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.000+0000 7fbd19d57700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ab921cce0 0x564ab922f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:51536/0 (socket says 172.21.15.37:51536) 2024-06-10T01:06:32.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.000+0000 7fbd19556700 1 -- 172.21.15.37:0/1546530612 learned_addr learned my addr 172.21.15.37:0/1546530612 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:06:33.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.000+0000 7fbd1a558700 1 --2- 172.21.15.37:0/1546530612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ab9222eb0 0x564ab922c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:33.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.000+0000 7fbd19556700 1 -- 172.21.15.37:0/1546530612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ab921cce0 msgr2=0x564ab922f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:33.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.000+0000 7fbd19556700 1 --2- 172.21.15.37:0/1546530612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ab921cce0 0x564ab922f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:33.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.000+0000 7fbd19556700 1 -- 172.21.15.37:0/1546530612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ab9222eb0 msgr2=0x564ab922c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:33.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.000+0000 7fbd19556700 1 --2- 172.21.15.37:0/1546530612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ab9222eb0 0x564ab922c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:33.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.000+0000 7fbd19556700 1 -- 172.21.15.37:0/1546530612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ab922c5c0 con 0x564ab9222590 2024-06-10T01:06:33.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.000+0000 7fbd19d57700 1 --2- 172.21.15.37:0/1546530612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ab921cce0 0x564ab922f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:33.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.000+0000 7fbd19556700 1 --2- 172.21.15.37:0/1546530612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab9222590 0x564ab922bac0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fbd04008e70 tx=0x7fbd040057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca7d6f4ab84e776 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:33.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.000+0000 7fbcfd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:06:33.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.000+0000 7fbcfd7fa700 1 -- 172.21.15.37:0/1546530612 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0400fe60 con 0x564ab9222590 2024-06-10T01:06:33.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.000+0000 7fbcfd7fa700 1 -- 172.21.15.37:0/1546530612 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fbd04003680 con 0x564ab9222590 2024-06-10T01:06:33.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.000+0000 7fbd2683eb80 1 -- 172.21.15.37:0/1546530612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd10040070 con 0x564ab9222590 2024-06-10T01:06:33.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.001+0000 7fbcfd7fa700 1 -- 172.21.15.37:0/1546530612 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fbd040038a0 con 0x564ab9222590 2024-06-10T01:06:33.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.001+0000 7fbd2683eb80 10 client.17067 fetch_fsmap learned FSMap version 31 2024-06-10T01:06:33.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.001+0000 7fbd2683eb80 1 -- 172.21.15.37:0/1546530612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564ab92a6480 con 0x564ab9222590 2024-06-10T01:06:33.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.001+0000 7fbcfd7fa700 1 -- 172.21.15.37:0/1546530612 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd10040070 con 0x564ab9222590 2024-06-10T01:06:33.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.001+0000 7fbcfd7fa700 1 -- 172.21.15.37:0/1546530612 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fbd04018050 con 0x564ab9222590 2024-06-10T01:06:33.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.001+0000 7fbd2683eb80 10 client.17067 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:06:33.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.001+0000 7fbd2683eb80 10 client.17067 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:06:33.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.001+0000 7fbd2683eb80 20 client.17067 populate_metadata read hostname 'smithi037' 2024-06-10T01:06:33.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.001+0000 7fbd2683eb80 1 --2- 172.21.15.37:0/1546530612 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564ab92a6c30 0x564ab92a9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:33.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.001+0000 7fbd2683eb80 4 client.17067 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.0, "await_for": 45.0}] 2024-06-10T01:06:33.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.001+0000 7fbd2683eb80 1 -- 172.21.15.37:0/1546530612 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.0, "await_for": 45.0}) -- 0x564ab9175460 con 0x564ab92a6c30 2024-06-10T01:06:33.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.001+0000 7fbd19d57700 1 --2- 172.21.15.37:0/1546530612 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564ab92a6c30 0x564ab92a9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:33.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.002+0000 7fbd19d57700 1 --2- 172.21.15.37:0/1546530612 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564ab92a6c30 0x564ab92a9050 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=617f0f31b382650f server_cookie=b3abce745a5210aa in_seq=0 out_seq=0 2024-06-10T01:06:33.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:33.002+0000 7fbcfd7fa700 10 client.17067 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:06:34.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:34 smithi094 ceph-mon[36708]: pgmap v727: 129 pgs: 129 active+clean; 45 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 927 op/s 2024-06-10T01:06:34.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:34 smithi037 ceph-mon[32192]: pgmap v727: 129 pgs: 129 active+clean; 45 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 927 op/s 2024-06-10T01:06:34.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:34 smithi167 ceph-mon[36917]: pgmap v727: 129 pgs: 129 active+clean; 45 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 927 op/s 2024-06-10T01:06:36.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:36 smithi094 ceph-mon[36708]: pgmap v728: 129 pgs: 129 active+clean; 29 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 826 op/s 2024-06-10T01:06:36.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:36 smithi037 ceph-mon[32192]: pgmap v728: 129 pgs: 129 active+clean; 29 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 826 op/s 2024-06-10T01:06:36.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:36 smithi167 ceph-mon[36917]: pgmap v728: 129 pgs: 129 active+clean; 29 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 826 op/s 2024-06-10T01:06:38.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:38 smithi094 ceph-mon[36708]: pgmap v729: 129 pgs: 129 active+clean; 29 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 982 KiB/s wr, 489 op/s 2024-06-10T01:06:38.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:38 smithi167 ceph-mon[36917]: pgmap v729: 129 pgs: 129 active+clean; 29 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 982 KiB/s wr, 489 op/s 2024-06-10T01:06:38.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:38 smithi037 ceph-mon[32192]: pgmap v729: 129 pgs: 129 active+clean; 29 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 982 KiB/s wr, 489 op/s 2024-06-10T01:06:40.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:40 smithi094 ceph-mon[36708]: pgmap v730: 129 pgs: 129 active+clean; 35 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 592 op/s 2024-06-10T01:06:40.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:40 smithi037 ceph-mon[32192]: pgmap v730: 129 pgs: 129 active+clean; 35 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 592 op/s 2024-06-10T01:06:40.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:40 smithi167 ceph-mon[36917]: pgmap v730: 129 pgs: 129 active+clean; 35 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 592 op/s 2024-06-10T01:06:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.780+0000 7fdf3a370640 1 -- 172.21.15.37:0/3260248386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf34104e30 msgr2=0x7fdf34107220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.780+0000 7fdf3a370640 1 --2- 172.21.15.37:0/3260248386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf34104e30 0x7fdf34107220 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fdf28009900 tx=0x7fdf2802a410 comp rx=0 tx=0).stop 2024-06-10T01:06:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.780+0000 7fdf3a370640 1 -- 172.21.15.37:0/3260248386 shutdown_connections 2024-06-10T01:06:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.780+0000 7fdf3a370640 1 --2- 172.21.15.37:0/3260248386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf34107760 0x7fdf34109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.780+0000 7fdf3a370640 1 --2- 172.21.15.37:0/3260248386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf34104e30 0x7fdf34107220 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.780+0000 7fdf3a370640 1 --2- 172.21.15.37:0/3260248386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf34102500 0x7fdf341048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.780+0000 7fdf3a370640 1 -- 172.21.15.37:0/3260248386 >> 172.21.15.37:0/3260248386 conn(0x7fdf340fbef0 msgr2=0x7fdf340fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.780+0000 7fdf3a370640 1 -- 172.21.15.37:0/3260248386 shutdown_connections 2024-06-10T01:06:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.780+0000 7fdf3a370640 1 -- 172.21.15.37:0/3260248386 wait complete. 2024-06-10T01:06:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.781+0000 7fdf3a370640 1 Processor -- start 2024-06-10T01:06:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.781+0000 7fdf3a370640 1 -- start start 2024-06-10T01:06:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.781+0000 7fdf3a370640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf34102500 0x7fdf34197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.781+0000 7fdf3a370640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf34104e30 0x7fdf341981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.781+0000 7fdf3a370640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf34107760 0x7fdf3419f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.781+0000 7fdf3a370640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3410d690 con 0x7fdf34104e30 2024-06-10T01:06:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.781+0000 7fdf3a370640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3410d510 con 0x7fdf34102500 2024-06-10T01:06:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.781+0000 7fdf3a370640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3410d810 con 0x7fdf34107760 2024-06-10T01:06:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.781+0000 7fdf337fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf34104e30 0x7fdf341981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.781+0000 7fdf337fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf34104e30 0x7fdf341981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59134/0 (socket says 172.21.15.37:59134) 2024-06-10T01:06:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.781+0000 7fdf337fe640 1 -- 172.21.15.37:0/3876824794 learned_addr learned my addr 172.21.15.37:0/3876824794 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:06:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.781+0000 7fdf3890d640 1 --2- 172.21.15.37:0/3876824794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf34107760 0x7fdf3419f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.781+0000 7fdf33fff640 1 --2- 172.21.15.37:0/3876824794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf34102500 0x7fdf34197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.782+0000 7fdf337fe640 1 -- 172.21.15.37:0/3876824794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf34107760 msgr2=0x7fdf3419f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.782+0000 7fdf337fe640 1 --2- 172.21.15.37:0/3876824794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf34107760 0x7fdf3419f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.782+0000 7fdf337fe640 1 -- 172.21.15.37:0/3876824794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf34102500 msgr2=0x7fdf34197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.782+0000 7fdf337fe640 1 --2- 172.21.15.37:0/3876824794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf34102500 0x7fdf34197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.782+0000 7fdf337fe640 1 -- 172.21.15.37:0/3876824794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf3419f870 con 0x7fdf34104e30 2024-06-10T01:06:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.782+0000 7fdf337fe640 1 --2- 172.21.15.37:0/3876824794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf34104e30 0x7fdf341981a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fdf28031610 tx=0x7fdf28004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.782+0000 7fdf317fa640 1 -- 172.21.15.37:0/3876824794 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2803f030 con 0x7fdf34104e30 2024-06-10T01:06:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.782+0000 7fdf317fa640 1 -- 172.21.15.37:0/3876824794 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf28003ad0 con 0x7fdf34104e30 2024-06-10T01:06:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.782+0000 7fdf317fa640 1 -- 172.21.15.37:0/3876824794 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf28035070 con 0x7fdf34104e30 2024-06-10T01:06:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.782+0000 7fdf3a370640 1 -- 172.21.15.37:0/3876824794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf3419fb00 con 0x7fdf34104e30 2024-06-10T01:06:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.782+0000 7fdf3a370640 1 -- 172.21.15.37:0/3876824794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf3419fff0 con 0x7fdf34104e30 2024-06-10T01:06:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.785+0000 7fdf317fa640 1 -- 172.21.15.37:0/3876824794 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fdf28042030 con 0x7fdf34104e30 2024-06-10T01:06:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.785+0000 7fdf3d07d740 1 Processor -- start 2024-06-10T01:06:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.785+0000 7fdf3d07d740 1 -- start start 2024-06-10T01:06:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.786+0000 7fdf3d07d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb1e5a9010 0x55fb1e5a9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.786+0000 7fdf3d07d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb1e5a9a00 0x55fb1e5ae500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.786+0000 7fdf3d07d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb1e5aeb70 0x55fb1e5b1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.786+0000 7fdf3d07d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55fb1e50c4f0 con 0x55fb1e5a9010 2024-06-10T01:06:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.786+0000 7fdf3d07d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fb1e4aa870 con 0x55fb1e5a9a00 2024-06-10T01:06:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.786+0000 7fdf3d07d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55fb1e5084b0 con 0x55fb1e5aeb70 2024-06-10T01:06:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.786+0000 7fdf3890d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb1e5a9010 0x55fb1e5a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.786+0000 7fdf3890d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb1e5a9010 0x55fb1e5a9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59142/0 (socket says 172.21.15.37:59142) 2024-06-10T01:06:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.786+0000 7fdf3890d640 1 -- 172.21.15.37:0/1052750300 learned_addr learned my addr 172.21.15.37:0/1052750300 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:06:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.786+0000 7fdf337fe640 1 --2- 172.21.15.37:0/1052750300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb1e5aeb70 0x55fb1e5b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.786+0000 7fdf33fff640 1 --2- 172.21.15.37:0/1052750300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb1e5a9a00 0x55fb1e5ae500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.786+0000 7fdf317fa640 1 --2- 172.21.15.37:0/3876824794 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdf1c0a2c30 0x7fdf1c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.786+0000 7fdf3890d640 1 -- 172.21.15.37:0/1052750300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb1e5aeb70 msgr2=0x55fb1e5b1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.786+0000 7fdf3890d640 1 --2- 172.21.15.37:0/1052750300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb1e5aeb70 0x55fb1e5b1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.786+0000 7fdf3890d640 1 -- 172.21.15.37:0/1052750300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb1e5a9a00 msgr2=0x55fb1e5ae500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.786+0000 7fdf3890d640 1 --2- 172.21.15.37:0/1052750300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb1e5a9a00 0x55fb1e5ae500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.786+0000 7fdf3890d640 1 -- 172.21.15.37:0/1052750300 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb1e4c69b0 con 0x55fb1e5a9010 2024-06-10T01:06:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.786+0000 7fdf317fa640 1 -- 172.21.15.37:0/3876824794 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fdf280efba0 con 0x7fdf34104e30 2024-06-10T01:06:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.786+0000 7fdf3890d640 1 --2- 172.21.15.37:0/3876824794 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdf1c0a2c30 0x7fdf1c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.787+0000 7fdf3890d640 1 --2- 172.21.15.37:0/1052750300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb1e5a9010 0x55fb1e5a9430 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c011590 tx=0x7fdf2c011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668c47f76d65395 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.787+0000 7fdf1a7fc640 1 -- 172.21.15.37:0/1052750300 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c017050 con 0x55fb1e5a9010 2024-06-10T01:06:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.787+0000 7fdf1a7fc640 1 -- 172.21.15.37:0/1052750300 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf2c00fd20 con 0x55fb1e5a9010 2024-06-10T01:06:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.787+0000 7fdf1a7fc640 1 -- 172.21.15.37:0/1052750300 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c01ba70 con 0x55fb1e5a9010 2024-06-10T01:06:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.787+0000 7fdf3890d640 1 --2- 172.21.15.37:0/3876824794 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdf1c0a2c30 0x7fdf1c0a50f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c022000 tx=0x7fdf2c023000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.787+0000 7fdf3d07d740 1 -- 172.21.15.37:0/1052750300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb1e5a9010 msgr2=0x55fb1e5a9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.787+0000 7fdf3d07d740 1 --2- 172.21.15.37:0/1052750300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb1e5a9010 0x55fb1e5a9430 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c011590 tx=0x7fdf2c011a50 comp rx=0 tx=0).stop 2024-06-10T01:06:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.788+0000 7fdf3d07d740 1 -- 172.21.15.37:0/1052750300 shutdown_connections 2024-06-10T01:06:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.788+0000 7fdf3d07d740 1 --2- 172.21.15.37:0/1052750300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb1e5aeb70 0x55fb1e5b1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.788+0000 7fdf3d07d740 1 --2- 172.21.15.37:0/1052750300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb1e5a9a00 0x55fb1e5ae500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.788+0000 7fdf3d07d740 1 --2- 172.21.15.37:0/1052750300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb1e5a9010 0x55fb1e5a9430 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.788+0000 7fdf3d07d740 1 -- 172.21.15.37:0/1052750300 >> 172.21.15.37:0/1052750300 conn(0x55fb1e5a4680 msgr2=0x55fb1e5a4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.788+0000 7fdf3d07d740 1 -- 172.21.15.37:0/1052750300 shutdown_connections 2024-06-10T01:06:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.788+0000 7fdf3d07d740 1 -- 172.21.15.37:0/1052750300 wait complete. 2024-06-10T01:06:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.788+0000 7fdf3d07d740 1 -- start start 2024-06-10T01:06:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.788+0000 7fdf3d07d740 1 Processor -- start 2024-06-10T01:06:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.788+0000 7fdf3d07d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb1e5a9010 0x55fb1e5b32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.789+0000 7fdf3d07d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb1e5a9a00 0x55fb1e5b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.789+0000 7fdf3d07d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb1e5aeb70 0x55fb1e5ba830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.789+0000 7fdf3d07d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55fb1e5b1b20 con 0x55fb1e5a9a00 2024-06-10T01:06:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.789+0000 7fdf3d07d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fb1e495ae0 con 0x55fb1e5a9010 2024-06-10T01:06:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.789+0000 7fdf3d07d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55fb1e5b1ca0 con 0x55fb1e5aeb70 2024-06-10T01:06:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.789+0000 7fdf3d07d740 10 client.0 _ll_register_callbacks cb 0x55fb1e4ce9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.789+0000 7fdf337fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb1e5a9a00 0x55fb1e5b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.789+0000 7fdf3890d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb1e5aeb70 0x55fb1e5ba830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.789+0000 7fdf3890d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb1e5aeb70 0x55fb1e5ba830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:51576/0 (socket says 172.21.15.37:51576) 2024-06-10T01:06:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.789+0000 7fdf3890d640 1 -- 172.21.15.37:0/667350624 learned_addr learned my addr 172.21.15.37:0/667350624 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:06:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.789+0000 7fdf337fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb1e5a9a00 0x55fb1e5b37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59146/0 (socket says 172.21.15.37:59146) 2024-06-10T01:06:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.789+0000 7fdf33fff640 1 --2- 172.21.15.37:0/667350624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb1e5a9010 0x55fb1e5b32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.789+0000 7fdf337fe640 1 -- 172.21.15.37:0/667350624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb1e5aeb70 msgr2=0x55fb1e5ba830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.789+0000 7fdf337fe640 1 --2- 172.21.15.37:0/667350624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb1e5aeb70 0x55fb1e5ba830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.789+0000 7fdf337fe640 1 -- 172.21.15.37:0/667350624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb1e5a9010 msgr2=0x55fb1e5b32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.789+0000 7fdf337fe640 1 --2- 172.21.15.37:0/667350624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb1e5a9010 0x55fb1e5b32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.789+0000 7fdf337fe640 1 -- 172.21.15.37:0/667350624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb1e220210 con 0x55fb1e5a9a00 2024-06-10T01:06:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.789+0000 7fdf3890d640 1 --2- 172.21.15.37:0/667350624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb1e5aeb70 0x55fb1e5ba830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.789+0000 7fdf337fe640 1 --2- 172.21.15.37:0/667350624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb1e5a9a00 0x55fb1e5b37f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fdf280b89a0 tx=0x7fdf280ba570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3594c4ca9db78b3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.789+0000 7fdf137fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:06:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.789+0000 7fdf137fe640 1 -- 172.21.15.37:0/667350624 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf28049070 con 0x55fb1e5a9a00 2024-06-10T01:06:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.789+0000 7fdf137fe640 1 -- 172.21.15.37:0/667350624 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fdf2803b460 con 0x55fb1e5a9a00 2024-06-10T01:06:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.789+0000 7fdf3d07d740 1 -- 172.21.15.37:0/667350624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf2c00fd20 con 0x55fb1e5a9a00 2024-06-10T01:06:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.790+0000 7fdf137fe640 1 -- 172.21.15.37:0/667350624 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdf280bbe30 con 0x55fb1e5a9a00 2024-06-10T01:06:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.790+0000 7fdf3d07d740 10 client.17079 fetch_fsmap learned FSMap version 31 2024-06-10T01:06:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.790+0000 7fdf3d07d740 1 -- 172.21.15.37:0/667350624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fb1e2ec620 con 0x55fb1e5a9a00 2024-06-10T01:06:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.790+0000 7fdf137fe640 1 -- 172.21.15.37:0/667350624 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf2c00fd20 con 0x55fb1e5a9a00 2024-06-10T01:06:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.790+0000 7fdf137fe640 1 -- 172.21.15.37:0/667350624 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fdf280bc2c0 con 0x55fb1e5a9a00 2024-06-10T01:06:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.790+0000 7fdf3d07d740 10 client.17079 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:06:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.790+0000 7fdf3d07d740 10 client.17079 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:06:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.790+0000 7fdf3d07d740 20 client.17079 populate_metadata read hostname 'smithi037' 2024-06-10T01:06:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.790+0000 7fdf3d07d740 1 --2- 172.21.15.37:0/667350624 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55fb1e5bf130 0x55fb1e5c1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.790+0000 7fdf3d07d740 4 client.17079 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:06:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.790+0000 7fdf3d07d740 1 -- 172.21.15.37:0/667350624 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb1e51ff10 con 0x55fb1e5bf130 2024-06-10T01:06:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.790+0000 7fdf33fff640 1 --2- 172.21.15.37:0/667350624 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55fb1e5bf130 0x55fb1e5c1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.791+0000 7fdf137fe640 10 client.17079 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:06:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:41.791+0000 7fdf33fff640 1 --2- 172.21.15.37:0/667350624 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55fb1e5bf130 0x55fb1e5c1520 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73f751ac144a5eea server_cookie=8004842750f07721 in_seq=0 out_seq=0 2024-06-10T01:06:42.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:42 smithi094 ceph-mon[36708]: pgmap v731: 129 pgs: 129 active+clean; 35 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 445 op/s 2024-06-10T01:06:42.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:42 smithi037 ceph-mon[32192]: pgmap v731: 129 pgs: 129 active+clean; 35 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 445 op/s 2024-06-10T01:06:42.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:42 smithi167 ceph-mon[36917]: pgmap v731: 129 pgs: 129 active+clean; 35 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 445 op/s 2024-06-10T01:06:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.881+0000 7fdf137fe640 1 -- 172.21.15.37:0/667350624 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fb1e51ff10 con 0x55fb1e5bf130 2024-06-10T01:06:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.881+0000 7fdf137fe640 10 client.17079 handle_command_reply: tid=0 2024-06-10T01:06:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.881+0000 7fdf3d07d740 1 client.17079 shutdown 2024-06-10T01:06:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.881+0000 7fdf3d07d740 1 -- 172.21.15.37:0/667350624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb1e5a9a00 msgr2=0x55fb1e5b37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.881+0000 7fdf3d07d740 1 --2- 172.21.15.37:0/667350624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb1e5a9a00 0x55fb1e5b37f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fdf280b89a0 tx=0x7fdf280ba570 comp rx=0 tx=0).stop 2024-06-10T01:06:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.881+0000 7fdf3d07d740 1 -- 172.21.15.37:0/667350624 shutdown_connections 2024-06-10T01:06:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.881+0000 7fdf3d07d740 1 --2- 172.21.15.37:0/667350624 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55fb1e5bf130 0x55fb1e5c1520 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.881+0000 7fdf3d07d740 1 --2- 172.21.15.37:0/667350624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb1e5aeb70 0x55fb1e5ba830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.881+0000 7fdf3d07d740 1 --2- 172.21.15.37:0/667350624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb1e5a9a00 0x55fb1e5b37f0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.881+0000 7fdf137fe640 0 client.17079 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:06:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.881+0000 7fdf3d07d740 1 --2- 172.21.15.37:0/667350624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb1e5a9010 0x55fb1e5b32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.881+0000 7fdf3d07d740 1 -- 172.21.15.37:0/667350624 >> 172.21.15.37:0/667350624 conn(0x55fb1e5a4680 msgr2=0x55fb1e5a37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.881+0000 7fdf3d07d740 1 -- 172.21.15.37:0/667350624 shutdown_connections 2024-06-10T01:06:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.881+0000 7fdf3d07d740 1 -- 172.21.15.37:0/667350624 wait complete. 2024-06-10T01:06:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.881+0000 7fdf3d07d740 20 client.17079 trim_cache size 0 max 16384 2024-06-10T01:06:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.912+0000 7fdf3d07d740 1 Processor -- start 2024-06-10T01:06:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.912+0000 7fdf3d07d740 1 -- start start 2024-06-10T01:06:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.913+0000 7fdf3d07d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb1e5a4680 0x55fb1e5aac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.913+0000 7fdf3d07d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb1e5a9010 0x55fb1e5ab1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.913+0000 7fdf3d07d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb1e5a9a00 0x55fb1e5ba7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.913+0000 7fdf3d07d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdf341acf70 con 0x55fb1e5a9a00 2024-06-10T01:06:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.913+0000 7fdf3d07d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdf341aa840 con 0x55fb1e5a9010 2024-06-10T01:06:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.913+0000 7fdf3d07d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdf341a8690 con 0x55fb1e5a4680 2024-06-10T01:06:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.913+0000 7fdf3890d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb1e5a9a00 0x55fb1e5ba7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.913+0000 7fdf3890d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb1e5a9a00 0x55fb1e5ba7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42994/0 (socket says 172.21.15.37:42994) 2024-06-10T01:06:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.913+0000 7fdf33fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb1e5a4680 0x55fb1e5aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.913+0000 7fdf3890d640 1 -- 172.21.15.37:0/1473392760 learned_addr learned my addr 172.21.15.37:0/1473392760 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:06:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.913+0000 7fdf337fe640 1 --2- 172.21.15.37:0/1473392760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb1e5a9010 0x55fb1e5ab1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.913+0000 7fdf33fff640 1 -- 172.21.15.37:0/1473392760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb1e5a9010 msgr2=0x55fb1e5ab1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.913+0000 7fdf33fff640 1 --2- 172.21.15.37:0/1473392760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb1e5a9010 0x55fb1e5ab1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.913+0000 7fdf33fff640 1 -- 172.21.15.37:0/1473392760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb1e5a9a00 msgr2=0x55fb1e5ba7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.913+0000 7fdf33fff640 1 --2- 172.21.15.37:0/1473392760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb1e5a9a00 0x55fb1e5ba7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.913+0000 7fdf33fff640 1 -- 172.21.15.37:0/1473392760 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb1e5c1ba0 con 0x55fb1e5a4680 2024-06-10T01:06:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.913+0000 7fdf3890d640 1 --2- 172.21.15.37:0/1473392760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb1e5a9a00 0x55fb1e5ba7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.913+0000 7fdf33fff640 1 --2- 172.21.15.37:0/1473392760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb1e5a4680 0x55fb1e5aac80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55fb1e5c1da0 tx=0x7fdf2400b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=205194eb16cbfea server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.914+0000 7fdf19ffb640 1 -- 172.21.15.37:0/1473392760 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf24003da0 con 0x55fb1e5a4680 2024-06-10T01:06:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.914+0000 7fdf19ffb640 1 -- 172.21.15.37:0/1473392760 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf24005500 con 0x55fb1e5a4680 2024-06-10T01:06:42.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.914+0000 7fdf19ffb640 1 -- 172.21.15.37:0/1473392760 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2401d650 con 0x55fb1e5a4680 2024-06-10T01:06:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.914+0000 7fdf3d07d740 1 -- 172.21.15.37:0/1473392760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb1e5a4680 msgr2=0x55fb1e5aac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.914+0000 7fdf3d07d740 1 --2- 172.21.15.37:0/1473392760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb1e5a4680 0x55fb1e5aac80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55fb1e5c1da0 tx=0x7fdf2400b390 comp rx=0 tx=0).stop 2024-06-10T01:06:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.914+0000 7fdf3d07d740 1 -- 172.21.15.37:0/1473392760 shutdown_connections 2024-06-10T01:06:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.914+0000 7fdf3d07d740 1 --2- 172.21.15.37:0/1473392760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb1e5a9a00 0x55fb1e5ba7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.914+0000 7fdf3d07d740 1 --2- 172.21.15.37:0/1473392760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb1e5a9010 0x55fb1e5ab1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.914+0000 7fdf3d07d740 1 --2- 172.21.15.37:0/1473392760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb1e5a4680 0x55fb1e5aac80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.914+0000 7fdf3d07d740 1 -- 172.21.15.37:0/1473392760 >> 172.21.15.37:0/1473392760 conn(0x55fb1e5ae640 msgr2=0x55fb1e5b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.914+0000 7fdf3d07d740 1 -- 172.21.15.37:0/1473392760 shutdown_connections 2024-06-10T01:06:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.915+0000 7fdf3d07d740 1 -- 172.21.15.37:0/1473392760 wait complete. 2024-06-10T01:06:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.915+0000 7fdf3d07d740 1 -- start start 2024-06-10T01:06:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.915+0000 7fdf3d07d740 1 Processor -- start 2024-06-10T01:06:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.915+0000 7fdf3d07d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb1e5a4680 0x55fb1e5ba8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.915+0000 7fdf3d07d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb1e5a9010 0x55fb1e5b73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.915+0000 7fdf3d07d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb1e5a9a00 0x55fb1e5b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.915+0000 7fdf3d07d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdf341a8d00 con 0x55fb1e5a9a00 2024-06-10T01:06:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.915+0000 7fdf3d07d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdf341aa170 con 0x55fb1e5a4680 2024-06-10T01:06:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.915+0000 7fdf3d07d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdf341a93d0 con 0x55fb1e5a9010 2024-06-10T01:06:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.916+0000 7fdf3d07d740 10 client.0 _ll_register_callbacks cb 0x7fdf28005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.916+0000 7fdf3890d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb1e5a9a00 0x55fb1e5b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.916+0000 7fdf3890d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb1e5a9a00 0x55fb1e5b79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43000/0 (socket says 172.21.15.37:43000) 2024-06-10T01:06:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.916+0000 7fdf3890d640 1 -- 172.21.15.37:0/1205070659 learned_addr learned my addr 172.21.15.37:0/1205070659 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:06:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.916+0000 7fdf337fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb1e5a9010 0x55fb1e5b73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.916+0000 7fdf33fff640 1 --2- 172.21.15.37:0/1205070659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb1e5a4680 0x55fb1e5ba8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.916+0000 7fdf3890d640 1 -- 172.21.15.37:0/1205070659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb1e5a9010 msgr2=0x55fb1e5b73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.916+0000 7fdf3890d640 1 --2- 172.21.15.37:0/1205070659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb1e5a9010 0x55fb1e5b73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.916+0000 7fdf3890d640 1 -- 172.21.15.37:0/1205070659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb1e5a4680 msgr2=0x55fb1e5ba8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.916+0000 7fdf3890d640 1 --2- 172.21.15.37:0/1205070659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb1e5a4680 0x55fb1e5ba8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.916+0000 7fdf3890d640 1 -- 172.21.15.37:0/1205070659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb1e627e70 con 0x55fb1e5a9a00 2024-06-10T01:06:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.916+0000 7fdf337fe640 1 --2- 172.21.15.37:0/1205070659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb1e5a9010 0x55fb1e5b73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:06:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.916+0000 7fdf3890d640 1 --2- 172.21.15.37:0/1205070659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb1e5a9a00 0x55fb1e5b79b0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c01b7c0 tx=0x7fdf2c003bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02b7c4e8d31d245 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.916+0000 7fdf137fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:06:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.916+0000 7fdf137fe640 1 -- 172.21.15.37:0/1205070659 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c004df0 con 0x55fb1e5a9a00 2024-06-10T01:06:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.916+0000 7fdf137fe640 1 -- 172.21.15.37:0/1205070659 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fdf2c00f640 con 0x55fb1e5a9a00 2024-06-10T01:06:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.916+0000 7fdf3d07d740 1 -- 172.21.15.37:0/1205070659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf24005500 con 0x55fb1e5a9a00 2024-06-10T01:06:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.916+0000 7fdf137fe640 1 -- 172.21.15.37:0/1205070659 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdf2c03f050 con 0x55fb1e5a9a00 2024-06-10T01:06:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.917+0000 7fdf3d07d740 10 client.17085 fetch_fsmap learned FSMap version 31 2024-06-10T01:06:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.917+0000 7fdf3d07d740 1 -- 172.21.15.37:0/1205070659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fb1e6291f0 con 0x55fb1e5a9a00 2024-06-10T01:06:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.917+0000 7fdf137fe640 1 -- 172.21.15.37:0/1205070659 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf24005500 con 0x55fb1e5a9a00 2024-06-10T01:06:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.917+0000 7fdf137fe640 1 -- 172.21.15.37:0/1205070659 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fdf2c01a070 con 0x55fb1e5a9a00 2024-06-10T01:06:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.917+0000 7fdf3d07d740 10 client.17085 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:06:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.917+0000 7fdf3d07d740 10 client.17085 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:06:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.917+0000 7fdf3d07d740 20 client.17085 populate_metadata read hostname 'smithi037' 2024-06-10T01:06:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.917+0000 7fdf3d07d740 1 --2- 172.21.15.37:0/1205070659 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55fb1e5aeb70 0x55fb1e62bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.917+0000 7fdf3d07d740 4 client.17085 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:06:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.917+0000 7fdf3d07d740 1 -- 172.21.15.37:0/1205070659 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fb1e519c20 con 0x55fb1e5aeb70 2024-06-10T01:06:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.917+0000 7fdf33fff640 1 --2- 172.21.15.37:0/1205070659 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55fb1e5aeb70 0x55fb1e62bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.918+0000 7fdf137fe640 10 client.17085 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:06:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:42.918+0000 7fdf33fff640 1 --2- 172.21.15.37:0/1205070659 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55fb1e5aeb70 0x55fb1e62bac0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27d444f122a3ecd1 server_cookie=171c1e8e91c41a0c in_seq=0 out_seq=0 2024-06-10T01:06:44.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:44 smithi037 ceph-mon[32192]: pgmap v732: 129 pgs: 129 active+clean; 35 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 902 KiB/s wr, 548 op/s 2024-06-10T01:06:44.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:44 smithi167 ceph-mon[36917]: pgmap v732: 129 pgs: 129 active+clean; 35 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 902 KiB/s wr, 548 op/s 2024-06-10T01:06:44.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:44 smithi094 ceph-mon[36708]: pgmap v732: 129 pgs: 129 active+clean; 35 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 902 KiB/s wr, 548 op/s 2024-06-10T01:06:46.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:46 smithi037 ceph-mon[32192]: pgmap v733: 129 pgs: 129 active+clean; 37 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 664 KiB/s wr, 713 op/s 2024-06-10T01:06:46.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:46 smithi167 ceph-mon[36917]: pgmap v733: 129 pgs: 129 active+clean; 37 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 664 KiB/s wr, 713 op/s 2024-06-10T01:06:46.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:46 smithi094 ceph-mon[36708]: pgmap v733: 129 pgs: 129 active+clean; 37 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 664 KiB/s wr, 713 op/s 2024-06-10T01:06:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:46.716+0000 7fdf137fe640 1 -- 172.21.15.37:0/1205070659 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fb1e519c20 con 0x55fb1e5aeb70 2024-06-10T01:06:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:46.716+0000 7fdf137fe640 10 client.17085 handle_command_reply: tid=0 2024-06-10T01:06:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:46.716+0000 7fdf3d07d740 1 client.17085 shutdown 2024-06-10T01:06:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:46.716+0000 7fdf3d07d740 1 -- 172.21.15.37:0/1205070659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb1e5a9a00 msgr2=0x55fb1e5b79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:46.716+0000 7fdf3d07d740 1 --2- 172.21.15.37:0/1205070659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb1e5a9a00 0x55fb1e5b79b0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c01b7c0 tx=0x7fdf2c003bb0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:46.716+0000 7fdf3d07d740 1 -- 172.21.15.37:0/1205070659 shutdown_connections 2024-06-10T01:06:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:46.716+0000 7fdf3d07d740 1 --2- 172.21.15.37:0/1205070659 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55fb1e5aeb70 0x55fb1e62bac0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:46.716+0000 7fdf3d07d740 1 --2- 172.21.15.37:0/1205070659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb1e5a9a00 0x55fb1e5b79b0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:46.716+0000 7fdf3d07d740 1 --2- 172.21.15.37:0/1205070659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb1e5a9010 0x55fb1e5b73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:46.716+0000 7fdf137fe640 0 client.17085 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:06:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:46.716+0000 7fdf3d07d740 1 --2- 172.21.15.37:0/1205070659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fb1e5a4680 0x55fb1e5ba8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:46.716+0000 7fdf3d07d740 1 -- 172.21.15.37:0/1205070659 >> 172.21.15.37:0/1205070659 conn(0x55fb1e5ae640 msgr2=0x55fb1e5b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:46.716+0000 7fdf3d07d740 1 -- 172.21.15.37:0/1205070659 shutdown_connections 2024-06-10T01:06:46.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:46.716+0000 7fdf3d07d740 1 -- 172.21.15.37:0/1205070659 wait complete. 2024-06-10T01:06:46.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:46.716+0000 7fdf3d07d740 20 client.17085 trim_cache size 0 max 16384 2024-06-10T01:06:46.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:06:46.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:06:46.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:06:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:46.717+0000 7fdf3a370640 1 -- 172.21.15.37:0/3876824794 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdf1c0a2c30 msgr2=0x7fdf1c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:46.717+0000 7fdf3a370640 1 --2- 172.21.15.37:0/3876824794 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdf1c0a2c30 0x7fdf1c0a50f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c022000 tx=0x7fdf2c023000 comp rx=0 tx=0).stop 2024-06-10T01:06:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:46.717+0000 7fdf3a370640 1 -- 172.21.15.37:0/3876824794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf34104e30 msgr2=0x7fdf341981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:46.717+0000 7fdf3a370640 1 --2- 172.21.15.37:0/3876824794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf34104e30 0x7fdf341981a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fdf28031610 tx=0x7fdf28004270 comp rx=0 tx=0).stop 2024-06-10T01:06:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:46.717+0000 7fdf3a370640 1 -- 172.21.15.37:0/3876824794 shutdown_connections 2024-06-10T01:06:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:46.717+0000 7fdf3a370640 1 --2- 172.21.15.37:0/3876824794 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdf1c0a2c30 0x7fdf1c0a50f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:46.717+0000 7fdf3a370640 1 --2- 172.21.15.37:0/3876824794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf34107760 0x7fdf3419f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:46.717+0000 7fdf3a370640 1 --2- 172.21.15.37:0/3876824794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf34104e30 0x7fdf341981a0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:46.717+0000 7fdf3a370640 1 --2- 172.21.15.37:0/3876824794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf34102500 0x7fdf34197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:46.717+0000 7fdf3a370640 1 -- 172.21.15.37:0/3876824794 >> 172.21.15.37:0/3876824794 conn(0x7fdf340fbef0 msgr2=0x7fdf34102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:46.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:46.717+0000 7fdf3a370640 1 -- 172.21.15.37:0/3876824794 shutdown_connections 2024-06-10T01:06:46.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:46.718+0000 7fdf3a370640 1 -- 172.21.15.37:0/3876824794 wait complete. 2024-06-10T01:06:48.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:48 smithi037 ceph-mon[32192]: pgmap v734: 129 pgs: 129 active+clean; 37 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 639 KiB/s wr, 615 op/s 2024-06-10T01:06:48.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:48 smithi167 ceph-mon[36917]: pgmap v734: 129 pgs: 129 active+clean; 37 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 639 KiB/s wr, 615 op/s 2024-06-10T01:06:48.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:48 smithi094 ceph-mon[36708]: pgmap v734: 129 pgs: 129 active+clean; 37 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 639 KiB/s wr, 615 op/s 2024-06-10T01:06:50.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:50 smithi037 ceph-mon[32192]: pgmap v735: 129 pgs: 129 active+clean; 18 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 809 KiB/s wr, 1.14k op/s 2024-06-10T01:06:50.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:50 smithi167 ceph-mon[36917]: pgmap v735: 129 pgs: 129 active+clean; 18 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 809 KiB/s wr, 1.14k op/s 2024-06-10T01:06:50.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:50 smithi094 ceph-mon[36708]: pgmap v735: 129 pgs: 129 active+clean; 18 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 809 KiB/s wr, 1.14k op/s 2024-06-10T01:06:51.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:51 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:06:51.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:51 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:06:51.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:51 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:06:51.685 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:06:51.685 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 30, 2024-06-10T01:06:51.685 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24472, 2024-06-10T01:06:51.685 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 22, 2024-06-10T01:06:51.685 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-06-10T01:06:51.685 INFO:teuthology.orchestra.run.smithi037.stdout: "cf050765": { 2024-06-10T01:06:51.685 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 22, 2024-06-10T01:06:51.686 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-06-10T01:06:51.686 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:06:51.686 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-06-10T01:06:51.686 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:06:51.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:06:51.686 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-06-10T01:06:51.686 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 136.0, 2024-06-10T01:06:51.686 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-06-10T01:06:51.686 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-06-10T01:06:51.686 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-06-10T01:06:51.686 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:06:51.686 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-06-10T01:06:51.687 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:06:51.687 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:06:51.687 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:06:51.687 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:06:51.687 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:06:51.687 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:06:51.687 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:06:51.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:51.687+0000 7fbcfd7fa700 1 -- 172.21.15.37:0/1546530612 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564ab9175460 con 0x564ab92a6c30 2024-06-10T01:06:51.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:51.687+0000 7fbcfd7fa700 10 client.17067 handle_command_reply: tid=0 2024-06-10T01:06:51.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:51.687+0000 7fbd2683eb80 1 client.17067 shutdown 2024-06-10T01:06:51.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:51.688+0000 7fbd2683eb80 1 -- 172.21.15.37:0/1546530612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab9222590 msgr2=0x564ab922bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:51.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:51.688+0000 7fbd2683eb80 1 --2- 172.21.15.37:0/1546530612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab9222590 0x564ab922bac0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fbd04008e70 tx=0x7fbd040057e0 comp rx=0 tx=0).stop 2024-06-10T01:06:51.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:51.688+0000 7fbd2683eb80 1 -- 172.21.15.37:0/1546530612 shutdown_connections 2024-06-10T01:06:51.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:51.688+0000 7fbd2683eb80 1 --2- 172.21.15.37:0/1546530612 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564ab92a6c30 0x564ab92a9050 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:51.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:51.688+0000 7fbd2683eb80 1 --2- 172.21.15.37:0/1546530612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ab9222eb0 0x564ab922c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:51.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:51.688+0000 7fbd2683eb80 1 --2- 172.21.15.37:0/1546530612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564ab921cce0 0x564ab922f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:51.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:51.688+0000 7fbcfd7fa700 0 client.17067 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:06:51.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:51.688+0000 7fbd2683eb80 1 --2- 172.21.15.37:0/1546530612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab9222590 0x564ab922bac0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:51.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:51.688+0000 7fbd2683eb80 1 -- 172.21.15.37:0/1546530612 >> 172.21.15.37:0/1546530612 conn(0x564ab9149210 msgr2=0x564ab921b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:51.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:51.688+0000 7fbd2683eb80 1 -- 172.21.15.37:0/1546530612 shutdown_connections 2024-06-10T01:06:51.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:51.688+0000 7fbd2683eb80 1 -- 172.21.15.37:0/1546530612 wait complete. 2024-06-10T01:06:51.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:51.688+0000 7fbd2683eb80 20 client.17067 trim_cache size 0 max 16384 2024-06-10T01:06:51.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:51.689+0000 7fbd1bfbb700 1 -- 172.21.15.37:0/324283275 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fbd000a2d40 msgr2=0x7fbd000a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:51.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:51.689+0000 7fbd1bfbb700 1 --2- 172.21.15.37:0/324283275 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fbd000a2d40 0x7fbd000a5200 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbd141999c0 tx=0x7fbd10005a90 comp rx=0 tx=0).stop 2024-06-10T01:06:51.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:51.689+0000 7fbd1bfbb700 1 -- 172.21.15.37:0/324283275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd14107760 msgr2=0x7fbd14198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:51.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:51.689+0000 7fbd1bfbb700 1 --2- 172.21.15.37:0/324283275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd14107760 0x7fbd14198880 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c00ef30 tx=0x7fbd0c00c3a0 comp rx=0 tx=0).stop 2024-06-10T01:06:51.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:51.689+0000 7fbd1bfbb700 1 -- 172.21.15.37:0/324283275 shutdown_connections 2024-06-10T01:06:51.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:51.689+0000 7fbd1bfbb700 1 --2- 172.21.15.37:0/324283275 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fbd000a2d40 0x7fbd000a5200 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:51.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:51.689+0000 7fbd1bfbb700 1 --2- 172.21.15.37:0/324283275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd14107760 0x7fbd14198880 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:51.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:51.689+0000 7fbd1bfbb700 1 --2- 172.21.15.37:0/324283275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd14102770 0x7fbd1419e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:51.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:51.689+0000 7fbd1bfbb700 1 --2- 172.21.15.37:0/324283275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd14103120 0x7fbd1419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:51.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:51.689+0000 7fbd1bfbb700 1 -- 172.21.15.37:0/324283275 >> 172.21.15.37:0/324283275 conn(0x7fbd14079560 msgr2=0x7fbd14100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:51.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:51.689+0000 7fbd1bfbb700 1 -- 172.21.15.37:0/324283275 shutdown_connections 2024-06-10T01:06:51.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:51.689+0000 7fbd1bfbb700 1 -- 172.21.15.37:0/324283275 wait complete. 2024-06-10T01:06:52.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:52 smithi037 ceph-mon[32192]: pgmap v736: 129 pgs: 129 active+clean; 10 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 359 KiB/s wr, 1.23k op/s 2024-06-10T01:06:52.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:52 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:06:52.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:52 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:06:52.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:52 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:06:52.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:52 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:06:52.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:52 smithi167 ceph-mon[36917]: pgmap v736: 129 pgs: 129 active+clean; 10 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 359 KiB/s wr, 1.23k op/s 2024-06-10T01:06:52.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:52 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:06:52.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:52 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:06:52.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:52 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:06:52.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:52 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:06:52.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:52 smithi094 ceph-mon[36708]: pgmap v736: 129 pgs: 129 active+clean; 10 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 359 KiB/s wr, 1.23k op/s 2024-06-10T01:06:52.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:52 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:06:52.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:52 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:06:52.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:52 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:06:52.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:52 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:06:53.771 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'cf050765', quiesce took 30.7 seconds. Will release after: -14.7 2024-06-10T01:06:53.772 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:06:53.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:53 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:06:53.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:53 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:06:53.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:53 smithi167 ceph-mon[36917]: pgmap v737: 129 pgs: 129 active+clean; 6.2 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 360 KiB/s wr, 1.59k op/s 2024-06-10T01:06:53.860 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:53 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:06:53.860 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:53 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:06:53.860 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:53 smithi094 ceph-mon[36708]: pgmap v737: 129 pgs: 129 active+clean; 6.2 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 360 KiB/s wr, 1.59k op/s 2024-06-10T01:06:53.891 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:53 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:06:53.891 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:53 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:06:53.891 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:53 smithi037 ceph-mon[32192]: pgmap v737: 129 pgs: 129 active+clean; 6.2 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 360 KiB/s wr, 1.59k op/s 2024-06-10T01:06:54.062 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:06:54.860 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:54 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:06:54.860 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:54 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:06:54.860 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:54 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:06:54.891 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:54 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:06:54.891 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:54 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:06:54.892 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:54 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:06:55.084 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:54 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:06:55.084 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:54 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:06:55.084 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:54 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:06:55.860 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:55 smithi094 ceph-mon[36708]: scrub summary: idle+waiting paths [/] 2024-06-10T01:06:55.860 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:55 smithi094 ceph-mon[36708]: pgmap v738: 129 pgs: 129 active+clean; 6.5 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 357 KiB/s wr, 2.50k op/s 2024-06-10T01:06:55.891 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:55 smithi037 ceph-mon[32192]: scrub summary: idle+waiting paths [/] 2024-06-10T01:06:55.891 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:55 smithi037 ceph-mon[32192]: pgmap v738: 129 pgs: 129 active+clean; 6.5 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 357 KiB/s wr, 2.50k op/s 2024-06-10T01:06:56.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.028+0000 7f9f72ab5700 1 -- 172.21.15.37:0/577863351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f6c103150 msgr2=0x7f9f6c107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:56.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.028+0000 7f9f72ab5700 1 --2- 172.21.15.37:0/577863351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f6c103150 0x7f9f6c107140 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00b0d0 tx=0x7f9f5c00b3e0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.028+0000 7f9f72ab5700 1 -- 172.21.15.37:0/577863351 shutdown_connections 2024-06-10T01:06:56.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.028+0000 7f9f72ab5700 1 --2- 172.21.15.37:0/577863351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f6c107870 0x7f9f6c10f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.028+0000 7f9f72ab5700 1 --2- 172.21.15.37:0/577863351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f6c1027a0 0x7f9f6c102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.028+0000 7f9f72ab5700 1 --2- 172.21.15.37:0/577863351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f6c103150 0x7f9f6c107140 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.028+0000 7f9f72ab5700 1 -- 172.21.15.37:0/577863351 >> 172.21.15.37:0/577863351 conn(0x7f9f6c079570 msgr2=0x7f9f6c079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:56.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.028+0000 7f9f72ab5700 1 -- 172.21.15.37:0/577863351 shutdown_connections 2024-06-10T01:06:56.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.028+0000 7f9f72ab5700 1 -- 172.21.15.37:0/577863351 wait complete. 2024-06-10T01:06:56.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.029+0000 7f9f72ab5700 1 Processor -- start 2024-06-10T01:06:56.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.029+0000 7f9f72ab5700 1 -- start start 2024-06-10T01:06:56.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.029+0000 7f9f72ab5700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f6c1027a0 0x7f9f6c19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:56.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.029+0000 7f9f72ab5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f6c103150 0x7f9f6c19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:56.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.029+0000 7f9f72ab5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f6c107870 0x7f9f6c198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:56.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.029+0000 7f9f72ab5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c1127a0 con 0x7f9f6c107870 2024-06-10T01:06:56.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.029+0000 7f9f72ab5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c112620 con 0x7f9f6c103150 2024-06-10T01:06:56.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.029+0000 7f9f72ab5700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c112920 con 0x7f9f6c1027a0 2024-06-10T01:06:56.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.029+0000 7f9f70851700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f6c1027a0 0x7f9f6c19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:56.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.029+0000 7f9f71052700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f6c107870 0x7f9f6c198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:56.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.029+0000 7f9f70851700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f6c1027a0 0x7f9f6c19e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:46602/0 (socket says 172.21.15.37:46602) 2024-06-10T01:06:56.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.029+0000 7f9f70851700 1 -- 172.21.15.37:0/4211222642 learned_addr learned my addr 172.21.15.37:0/4211222642 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:06:56.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.029+0000 7f9f6bfff700 1 --2- 172.21.15.37:0/4211222642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f6c103150 0x7f9f6c19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:56.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.030+0000 7f9f71052700 1 -- 172.21.15.37:0/4211222642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f6c1027a0 msgr2=0x7f9f6c19e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:56.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.030+0000 7f9f71052700 1 --2- 172.21.15.37:0/4211222642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f6c1027a0 0x7f9f6c19e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.030+0000 7f9f71052700 1 -- 172.21.15.37:0/4211222642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f6c103150 msgr2=0x7f9f6c19edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:56.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.030+0000 7f9f71052700 1 --2- 172.21.15.37:0/4211222642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f6c103150 0x7f9f6c19edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.030+0000 7f9f71052700 1 -- 172.21.15.37:0/4211222642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f6c1991e0 con 0x7f9f6c107870 2024-06-10T01:06:56.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.030+0000 7f9f71052700 1 --2- 172.21.15.37:0/4211222642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f6c107870 0x7f9f6c198a80 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f9f6400e970 tx=0x7f9f6400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:56.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.030+0000 7f9f69ffb700 1 -- 172.21.15.37:0/4211222642 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f64004220 con 0x7f9f6c107870 2024-06-10T01:06:56.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.030+0000 7f9f72ab5700 1 -- 172.21.15.37:0/4211222642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f6c1994d0 con 0x7f9f6c107870 2024-06-10T01:06:56.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.030+0000 7f9f69ffb700 1 -- 172.21.15.37:0/4211222642 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f64004d10 con 0x7f9f6c107870 2024-06-10T01:06:56.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.030+0000 7f9f69ffb700 1 -- 172.21.15.37:0/4211222642 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f64010700 con 0x7f9f6c107870 2024-06-10T01:06:56.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.031+0000 7f9f72ab5700 1 -- 172.21.15.37:0/4211222642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f6c1a5740 con 0x7f9f6c107870 2024-06-10T01:06:56.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.032+0000 7f9f72ab5700 1 -- 172.21.15.37:0/4211222642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f6c069180 con 0x7f9f6c107870 2024-06-10T01:06:56.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.032+0000 7f9f69ffb700 1 -- 172.21.15.37:0/4211222642 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9f64004750 con 0x7f9f6c107870 2024-06-10T01:06:56.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.033+0000 7f9f69ffb700 1 --2- 172.21.15.37:0/4211222642 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9f580a2c20 0x7f9f580a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:56.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.033+0000 7f9f70851700 1 --2- 172.21.15.37:0/4211222642 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9f580a2c20 0x7f9f580a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:56.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.033+0000 7f9f69ffb700 1 -- 172.21.15.37:0/4211222642 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f9f640cf1b0 con 0x7f9f6c107870 2024-06-10T01:06:56.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.034+0000 7f9f70851700 1 --2- 172.21.15.37:0/4211222642 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9f580a2c20 0x7f9f580a50e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c1073d0 tx=0x7f9f54009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:56.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.038+0000 7f9f69ffb700 1 -- 172.21.15.37:0/4211222642 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9f6409bcc0 con 0x7f9f6c107870 2024-06-10T01:06:56.084 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:55 smithi167 ceph-mon[36917]: scrub summary: idle+waiting paths [/] 2024-06-10T01:06:56.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:55 smithi167 ceph-mon[36917]: pgmap v738: 129 pgs: 129 active+clean; 6.5 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 357 KiB/s wr, 2.50k op/s 2024-06-10T01:06:56.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.289+0000 7f9f72ab5700 1 -- 172.21.15.37:0/4211222642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f6c02dca0 con 0x7f9f6c107870 2024-06-10T01:06:56.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.290+0000 7f9f69ffb700 1 -- 172.21.15.37:0/4211222642 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11868 (secure 0 0 0) 0x7f9f6409b5c0 con 0x7f9f6c107870 2024-06-10T01:06:56.287 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:06:56.287 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":26554,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2107277223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2107277223},{"type":"v1","addr":"172.21.15.167:6835","nonce":2107277223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T01:06:09.604504+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24472,24451,24458]},"id":1}]} 2024-06-10T01:06:56.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.292+0000 7f9f72ab5700 1 -- 172.21.15.37:0/4211222642 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9f580a2c20 msgr2=0x7f9f580a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:56.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.292+0000 7f9f72ab5700 1 --2- 172.21.15.37:0/4211222642 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9f580a2c20 0x7f9f580a50e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c1073d0 tx=0x7f9f54009260 comp rx=0 tx=0).stop 2024-06-10T01:06:56.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.292+0000 7f9f72ab5700 1 -- 172.21.15.37:0/4211222642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f6c107870 msgr2=0x7f9f6c198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:56.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.292+0000 7f9f72ab5700 1 --2- 172.21.15.37:0/4211222642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f6c107870 0x7f9f6c198a80 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f9f6400e970 tx=0x7f9f6400ec80 comp rx=0 tx=0).stop 2024-06-10T01:06:56.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.292+0000 7f9f72ab5700 1 -- 172.21.15.37:0/4211222642 shutdown_connections 2024-06-10T01:06:56.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.292+0000 7f9f72ab5700 1 --2- 172.21.15.37:0/4211222642 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9f580a2c20 0x7f9f580a50e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.292+0000 7f9f72ab5700 1 --2- 172.21.15.37:0/4211222642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f6c107870 0x7f9f6c198a80 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.292+0000 7f9f72ab5700 1 --2- 172.21.15.37:0/4211222642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f6c1027a0 0x7f9f6c19e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.292+0000 7f9f72ab5700 1 --2- 172.21.15.37:0/4211222642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f6c103150 0x7f9f6c19edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:56.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.292+0000 7f9f72ab5700 1 -- 172.21.15.37:0/4211222642 >> 172.21.15.37:0/4211222642 conn(0x7f9f6c079570 msgr2=0x7f9f6c1004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:56.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.292+0000 7f9f72ab5700 1 -- 172.21.15.37:0/4211222642 shutdown_connections 2024-06-10T01:06:56.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:06:56.293+0000 7f9f72ab5700 1 -- 172.21.15.37:0/4211222642 wait complete. 2024-06-10T01:06:56.291 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 31 2024-06-10T01:06:56.860 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:56 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/4211222642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:06:56.891 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:56 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/4211222642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:06:56.944 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24472 quiesce db --set-id cf050765 --release --await' 2024-06-10T01:06:56.945 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24472 quiesce db --set-id cf050765 --release --await 2024-06-10T01:06:57.084 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:56 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/4211222642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:06:57.198 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:06:57.860 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:57 smithi094 ceph-mon[36708]: scrub summary: idle 2024-06-10T01:06:57.860 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:06:57 smithi094 ceph-mon[36708]: pgmap v739: 129 pgs: 129 active+clean; 6.5 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 217 KiB/s wr, 2.38k op/s 2024-06-10T01:06:57.891 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:57 smithi037 ceph-mon[32192]: scrub summary: idle 2024-06-10T01:06:57.891 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:06:57 smithi037 ceph-mon[32192]: pgmap v739: 129 pgs: 129 active+clean; 6.5 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 217 KiB/s wr, 2.38k op/s 2024-06-10T01:06:58.084 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:57 smithi167 ceph-mon[36917]: scrub summary: idle 2024-06-10T01:06:58.084 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:06:57 smithi167 ceph-mon[36917]: pgmap v739: 129 pgs: 129 active+clean; 6.5 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 217 KiB/s wr, 2.38k op/s 2024-06-10T01:06:59.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.818+0000 7fcabd16a640 1 -- 172.21.15.37:0/1808234139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab8102c20 msgr2=0x7fcab81030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:59.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.818+0000 7fcabd16a640 1 --2- 172.21.15.37:0/1808234139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab8102c20 0x7fcab81030a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009900 tx=0x7fcaac02a410 comp rx=0 tx=0).stop 2024-06-10T01:06:59.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.818+0000 7fcabd16a640 1 -- 172.21.15.37:0/1808234139 shutdown_connections 2024-06-10T01:06:59.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.818+0000 7fcabd16a640 1 --2- 172.21.15.37:0/1808234139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcab81035e0 0x7fcab8109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:59.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.818+0000 7fcabd16a640 1 --2- 172.21.15.37:0/1808234139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab8102c20 0x7fcab81030a0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:59.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.818+0000 7fcabd16a640 1 --2- 172.21.15.37:0/1808234139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcab8101a30 0x7fcab8101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.818+0000 7fcabd16a640 1 -- 172.21.15.37:0/1808234139 >> 172.21.15.37:0/1808234139 conn(0x7fcab80fcfc0 msgr2=0x7fcab80ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.818+0000 7fcabd16a640 1 -- 172.21.15.37:0/1808234139 shutdown_connections 2024-06-10T01:06:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.818+0000 7fcabd16a640 1 -- 172.21.15.37:0/1808234139 wait complete. 2024-06-10T01:06:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.819+0000 7fcabd16a640 1 Processor -- start 2024-06-10T01:06:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.819+0000 7fcabd16a640 1 -- start start 2024-06-10T01:06:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.820+0000 7fcabd16a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcab8101a30 0x7fcab8197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.820+0000 7fcabd16a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab8102c20 0x7fcab8198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.820+0000 7fcabd16a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcab81035e0 0x7fcab819f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.820+0000 7fcabd16a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcab810d700 con 0x7fcab8102c20 2024-06-10T01:06:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.820+0000 7fcabd16a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcab810d580 con 0x7fcab8101a30 2024-06-10T01:06:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.820+0000 7fcabd16a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcab810d880 con 0x7fcab81035e0 2024-06-10T01:06:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.820+0000 7fcab659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab8102c20 0x7fcab8198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.820+0000 7fcab659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab8102c20 0x7fcab8198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49774/0 (socket says 172.21.15.37:49774) 2024-06-10T01:06:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.820+0000 7fcab659c640 1 -- 172.21.15.37:0/2010769547 learned_addr learned my addr 172.21.15.37:0/2010769547 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:06:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.820+0000 7fcab759e640 1 --2- 172.21.15.37:0/2010769547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcab81035e0 0x7fcab819f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.820+0000 7fcab6d9d640 1 --2- 172.21.15.37:0/2010769547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcab8101a30 0x7fcab8197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.820+0000 7fcab659c640 1 -- 172.21.15.37:0/2010769547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcab81035e0 msgr2=0x7fcab819f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.820+0000 7fcab659c640 1 --2- 172.21.15.37:0/2010769547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcab81035e0 0x7fcab819f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.820+0000 7fcab659c640 1 -- 172.21.15.37:0/2010769547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcab8101a30 msgr2=0x7fcab8197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.820+0000 7fcab659c640 1 --2- 172.21.15.37:0/2010769547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcab8101a30 0x7fcab8197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.820+0000 7fcab659c640 1 -- 172.21.15.37:0/2010769547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab819f850 con 0x7fcab8102c20 2024-06-10T01:06:59.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.820+0000 7fcab659c640 1 --2- 172.21.15.37:0/2010769547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab8102c20 0x7fcab8198180 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fcaac031440 tx=0x7fcaac032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.821+0000 7fca9ffff640 1 -- 172.21.15.37:0/2010769547 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac03f030 con 0x7fcab8102c20 2024-06-10T01:06:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.821+0000 7fcabd16a640 1 -- 172.21.15.37:0/2010769547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcab819fae0 con 0x7fcab8102c20 2024-06-10T01:06:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.821+0000 7fca9ffff640 1 -- 172.21.15.37:0/2010769547 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaac003ce0 con 0x7fcab8102c20 2024-06-10T01:06:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.821+0000 7fca9ffff640 1 -- 172.21.15.37:0/2010769547 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac035070 con 0x7fcab8102c20 2024-06-10T01:06:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.821+0000 7fcabd16a640 1 -- 172.21.15.37:0/2010769547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcab819ffd0 con 0x7fcab8102c20 2024-06-10T01:06:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.823+0000 7fca9ffff640 1 -- 172.21.15.37:0/2010769547 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fcaac042030 con 0x7fcab8102c20 2024-06-10T01:06:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.823+0000 7fcabfe71740 1 Processor -- start 2024-06-10T01:06:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.823+0000 7fcabfe71740 1 -- start start 2024-06-10T01:06:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.823+0000 7fcabfe71740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55617f990010 0x55617f990430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.823+0000 7fcabfe71740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55617f990a00 0x55617f995500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.823+0000 7fcabfe71740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55617f995b70 0x55617f998000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.823+0000 7fcabfe71740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55617f8f34f0 con 0x55617f990a00 2024-06-10T01:06:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.823+0000 7fcabfe71740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55617f891870 con 0x55617f990010 2024-06-10T01:06:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.823+0000 7fcabfe71740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55617f8ef4b0 con 0x55617f995b70 2024-06-10T01:06:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.824+0000 7fcab6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55617f990a00 0x55617f995500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.824+0000 7fcab6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55617f990a00 0x55617f995500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49778/0 (socket says 172.21.15.37:49778) 2024-06-10T01:06:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.824+0000 7fcab6d9d640 1 -- 172.21.15.37:0/651427046 learned_addr learned my addr 172.21.15.37:0/651427046 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:06:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.824+0000 7fcab759e640 1 --2- 172.21.15.37:0/651427046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55617f990010 0x55617f990430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.824+0000 7fca9ffff640 1 --2- 172.21.15.37:0/2010769547 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fcaa00a2c30 0x7fcaa00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.824+0000 7fcab659c640 1 --2- 172.21.15.37:0/651427046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55617f995b70 0x55617f998000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.824+0000 7fcab6d9d640 1 -- 172.21.15.37:0/651427046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55617f995b70 msgr2=0x55617f998000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.824+0000 7fcab6d9d640 1 --2- 172.21.15.37:0/651427046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55617f995b70 0x55617f998000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.824+0000 7fcab6d9d640 1 -- 172.21.15.37:0/651427046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55617f990010 msgr2=0x55617f990430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.824+0000 7fcab6d9d640 1 --2- 172.21.15.37:0/651427046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55617f990010 0x55617f990430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.824+0000 7fcab6d9d640 1 -- 172.21.15.37:0/651427046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55617f8ad9b0 con 0x55617f990a00 2024-06-10T01:06:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.824+0000 7fcab759e640 1 --2- 172.21.15.37:0/651427046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55617f990010 0x55617f990430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.824+0000 7fcab759e640 1 --2- 172.21.15.37:0/2010769547 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fcaa00a2c30 0x7fcaa00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.824+0000 7fca9ffff640 1 -- 172.21.15.37:0/2010769547 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fcaac0efb20 con 0x7fcab8102c20 2024-06-10T01:06:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.824+0000 7fcab6d9d640 1 --2- 172.21.15.37:0/651427046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55617f990a00 0x55617f995500 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fcaa800bc70 tx=0x7fcaa80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9718b249d061cdec server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.824+0000 7fca9d7fa640 1 -- 172.21.15.37:0/651427046 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8004020 con 0x55617f990a00 2024-06-10T01:06:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.824+0000 7fcab759e640 1 --2- 172.21.15.37:0/2010769547 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fcaa00a2c30 0x7fcaa00a50f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcab00098b0 tx=0x7fcab0004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.824+0000 7fca9d7fa640 1 -- 172.21.15.37:0/651427046 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaa80041c0 con 0x55617f990a00 2024-06-10T01:06:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.824+0000 7fca9d7fa640 1 -- 172.21.15.37:0/651427046 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8011750 con 0x55617f990a00 2024-06-10T01:06:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.825+0000 7fcabfe71740 1 -- 172.21.15.37:0/651427046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55617f990a00 msgr2=0x55617f995500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.825+0000 7fcabfe71740 1 --2- 172.21.15.37:0/651427046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55617f990a00 0x55617f995500 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fcaa800bc70 tx=0x7fcaa80107a0 comp rx=0 tx=0).stop 2024-06-10T01:06:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.825+0000 7fcabfe71740 1 -- 172.21.15.37:0/651427046 shutdown_connections 2024-06-10T01:06:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.825+0000 7fcabfe71740 1 --2- 172.21.15.37:0/651427046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55617f995b70 0x55617f998000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.825+0000 7fcabfe71740 1 --2- 172.21.15.37:0/651427046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55617f990a00 0x55617f995500 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.825+0000 7fcabfe71740 1 --2- 172.21.15.37:0/651427046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55617f990010 0x55617f990430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.825+0000 7fcabfe71740 1 -- 172.21.15.37:0/651427046 >> 172.21.15.37:0/651427046 conn(0x55617f98b680 msgr2=0x55617f98ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.825+0000 7fcabfe71740 1 -- 172.21.15.37:0/651427046 shutdown_connections 2024-06-10T01:06:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.825+0000 7fcabfe71740 1 -- 172.21.15.37:0/651427046 wait complete. 2024-06-10T01:06:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.825+0000 7fcabfe71740 1 -- start start 2024-06-10T01:06:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.825+0000 7fcabfe71740 1 Processor -- start 2024-06-10T01:06:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.826+0000 7fcabfe71740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55617f990010 0x55617f99a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.826+0000 7fcabfe71740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55617f990a00 0x55617f99a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.826+0000 7fcabfe71740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55617f995b70 0x55617f9a1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.826+0000 7fcabfe71740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55617f998b20 con 0x55617f990010 2024-06-10T01:06:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.826+0000 7fcabfe71740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55617f87cae0 con 0x55617f990a00 2024-06-10T01:06:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.826+0000 7fcabfe71740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55617f998ca0 con 0x55617f995b70 2024-06-10T01:06:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.826+0000 7fcabfe71740 10 client.0 _ll_register_callbacks cb 0x55617f8b59b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.826+0000 7fcab6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55617f990010 0x55617f99a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.826+0000 7fcab6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55617f990010 0x55617f99a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49792/0 (socket says 172.21.15.37:49792) 2024-06-10T01:06:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.826+0000 7fcab6d9d640 1 -- 172.21.15.37:0/860956733 learned_addr learned my addr 172.21.15.37:0/860956733 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:06:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.826+0000 7fcab759e640 1 --2- 172.21.15.37:0/860956733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55617f995b70 0x55617f9a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.826+0000 7fcab6d9d640 1 -- 172.21.15.37:0/860956733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55617f995b70 msgr2=0x55617f9a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.826+0000 7fcab6d9d640 1 --2- 172.21.15.37:0/860956733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55617f995b70 0x55617f9a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.826+0000 7fcab6d9d640 1 -- 172.21.15.37:0/860956733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55617f990a00 msgr2=0x55617f99a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.826+0000 7fcab6d9d640 1 --2- 172.21.15.37:0/860956733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55617f990a00 0x55617f99a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.826+0000 7fcab6d9d640 1 -- 172.21.15.37:0/860956733 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55617f607210 con 0x55617f990010 2024-06-10T01:06:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.826+0000 7fcab759e640 1 --2- 172.21.15.37:0/860956733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55617f995b70 0x55617f9a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:06:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.826+0000 7fcab6d9d640 1 --2- 172.21.15.37:0/860956733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55617f990010 0x55617f99a320 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8005890 tx=0x7fcaa80045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc07c9f0969a24da server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.826+0000 7fca9aff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:06:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.826+0000 7fca9aff5640 1 -- 172.21.15.37:0/860956733 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa802a060 con 0x55617f990010 2024-06-10T01:06:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.826+0000 7fca9aff5640 1 -- 172.21.15.37:0/860956733 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fcaa8005250 con 0x55617f990010 2024-06-10T01:06:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.826+0000 7fcabfe71740 1 -- 172.21.15.37:0/860956733 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaa80041c0 con 0x55617f990010 2024-06-10T01:06:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.827+0000 7fca9aff5640 1 -- 172.21.15.37:0/860956733 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fcaa8031050 con 0x55617f990010 2024-06-10T01:06:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.827+0000 7fcabfe71740 10 client.17103 fetch_fsmap learned FSMap version 31 2024-06-10T01:06:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.827+0000 7fcabfe71740 1 -- 172.21.15.37:0/860956733 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55617f6d3620 con 0x55617f990010 2024-06-10T01:06:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.827+0000 7fca9aff5640 1 -- 172.21.15.37:0/860956733 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaa80041c0 con 0x55617f990010 2024-06-10T01:06:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.827+0000 7fca9aff5640 1 -- 172.21.15.37:0/860956733 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fcaa80048b0 con 0x55617f990010 2024-06-10T01:06:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.827+0000 7fcabfe71740 10 client.17103 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:06:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.827+0000 7fcabfe71740 10 client.17103 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:06:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.827+0000 7fcabfe71740 20 client.17103 populate_metadata read hostname 'smithi037' 2024-06-10T01:06:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.827+0000 7fcabfe71740 1 --2- 172.21.15.37:0/860956733 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55617f9a5dd0 0x55617f9a81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.827+0000 7fcabfe71740 4 client.17103 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:06:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.827+0000 7fcabfe71740 1 -- 172.21.15.37:0/860956733 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55617f906f10 con 0x55617f9a5dd0 2024-06-10T01:06:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.827+0000 7fcab659c640 1 --2- 172.21.15.37:0/860956733 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55617f9a5dd0 0x55617f9a81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.828+0000 7fcab659c640 1 --2- 172.21.15.37:0/860956733 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55617f9a5dd0 0x55617f9a81c0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5ebcef6c11606ca server_cookie=b41274017eb65b2 in_seq=0 out_seq=0 2024-06-10T01:06:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.828+0000 7fca9aff5640 10 client.17103 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:06:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.829+0000 7fca9aff5640 1 -- 172.21.15.37:0/860956733 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55617f906f10 con 0x55617f9a5dd0 2024-06-10T01:06:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.829+0000 7fca9aff5640 10 client.17103 handle_command_reply: tid=0 2024-06-10T01:06:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.829+0000 7fcabfe71740 1 client.17103 shutdown 2024-06-10T01:06:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.829+0000 7fcabfe71740 1 -- 172.21.15.37:0/860956733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55617f990010 msgr2=0x55617f99a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.829+0000 7fcabfe71740 1 --2- 172.21.15.37:0/860956733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55617f990010 0x55617f99a320 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8005890 tx=0x7fcaa80045a0 comp rx=0 tx=0).stop 2024-06-10T01:06:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.829+0000 7fcabfe71740 1 -- 172.21.15.37:0/860956733 shutdown_connections 2024-06-10T01:06:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.829+0000 7fcabfe71740 1 --2- 172.21.15.37:0/860956733 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55617f9a5dd0 0x55617f9a81c0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.829+0000 7fcabfe71740 1 --2- 172.21.15.37:0/860956733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55617f995b70 0x55617f9a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.829+0000 7fcabfe71740 1 --2- 172.21.15.37:0/860956733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55617f990a00 0x55617f99a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.829+0000 7fcabfe71740 1 --2- 172.21.15.37:0/860956733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55617f990010 0x55617f99a320 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.829+0000 7fca9aff5640 0 client.17103 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:06:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.829+0000 7fcabfe71740 1 -- 172.21.15.37:0/860956733 >> 172.21.15.37:0/860956733 conn(0x55617f98b680 msgr2=0x55617f98a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.829+0000 7fcabfe71740 1 -- 172.21.15.37:0/860956733 shutdown_connections 2024-06-10T01:06:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.829+0000 7fcabfe71740 1 -- 172.21.15.37:0/860956733 wait complete. 2024-06-10T01:06:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.829+0000 7fcabfe71740 20 client.17103 trim_cache size 0 max 16384 2024-06-10T01:06:59.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.844+0000 7fcabfe71740 1 Processor -- start 2024-06-10T01:06:59.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.844+0000 7fcabfe71740 1 -- start start 2024-06-10T01:06:59.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.844+0000 7fcabfe71740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55617f990010 0x55617f999940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:59.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.844+0000 7fcabfe71740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55617f990a00 0x55617f999e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:59.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.844+0000 7fcabfe71740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55617f995b70 0x55617f99a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:59.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.844+0000 7fcabfe71740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcab81ad040 con 0x55617f990a00 2024-06-10T01:06:59.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.844+0000 7fcabfe71740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcab81aa910 con 0x55617f990010 2024-06-10T01:06:59.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.844+0000 7fcabfe71740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcab81a8760 con 0x55617f995b70 2024-06-10T01:06:59.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.844+0000 7fcab659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55617f990a00 0x55617f999e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:59.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.844+0000 7fcab659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55617f990a00 0x55617f999e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49802/0 (socket says 172.21.15.37:49802) 2024-06-10T01:06:59.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.844+0000 7fcab659c640 1 -- 172.21.15.37:0/1887300715 learned_addr learned my addr 172.21.15.37:0/1887300715 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:06:59.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.844+0000 7fcab759e640 1 --2- 172.21.15.37:0/1887300715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55617f995b70 0x55617f99a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:59.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.844+0000 7fcab6d9d640 1 --2- 172.21.15.37:0/1887300715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55617f990010 0x55617f999940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:59.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.845+0000 7fcab659c640 1 -- 172.21.15.37:0/1887300715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55617f995b70 msgr2=0x55617f99a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:59.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.845+0000 7fcab659c640 1 --2- 172.21.15.37:0/1887300715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55617f995b70 0x55617f99a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:59.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.845+0000 7fcab659c640 1 -- 172.21.15.37:0/1887300715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55617f990010 msgr2=0x55617f999940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:59.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.845+0000 7fcab659c640 1 --2- 172.21.15.37:0/1887300715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55617f990010 0x55617f999940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:59.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.845+0000 7fcab659c640 1 -- 172.21.15.37:0/1887300715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55617f986f70 con 0x55617f990a00 2024-06-10T01:06:59.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.845+0000 7fcab659c640 1 --2- 172.21.15.37:0/1887300715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55617f990a00 0x55617f999e80 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fcaac0b8920 tx=0x7fcaac08df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c407aea298bf31b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:59.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.845+0000 7fca9cff9640 1 -- 172.21.15.37:0/1887300715 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac049040 con 0x55617f990a00 2024-06-10T01:06:59.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.845+0000 7fca9cff9640 1 -- 172.21.15.37:0/1887300715 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaac0bbfe0 con 0x55617f990a00 2024-06-10T01:06:59.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.845+0000 7fca9cff9640 1 -- 172.21.15.37:0/1887300715 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac0b85c0 con 0x55617f990a00 2024-06-10T01:06:59.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.845+0000 7fcabfe71740 1 -- 172.21.15.37:0/1887300715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55617f990a00 msgr2=0x55617f999e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:06:59.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.845+0000 7fcabfe71740 1 --2- 172.21.15.37:0/1887300715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55617f990a00 0x55617f999e80 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fcaac0b8920 tx=0x7fcaac08df60 comp rx=0 tx=0).stop 2024-06-10T01:06:59.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.846+0000 7fcabfe71740 1 -- 172.21.15.37:0/1887300715 shutdown_connections 2024-06-10T01:06:59.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.846+0000 7fcabfe71740 1 --2- 172.21.15.37:0/1887300715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55617f995b70 0x55617f99a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.846+0000 7fcabfe71740 1 --2- 172.21.15.37:0/1887300715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55617f990a00 0x55617f999e80 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.846+0000 7fcabfe71740 1 --2- 172.21.15.37:0/1887300715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55617f990010 0x55617f999940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.846+0000 7fcabfe71740 1 -- 172.21.15.37:0/1887300715 >> 172.21.15.37:0/1887300715 conn(0x55617f98b680 msgr2=0x55617f98e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:06:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.846+0000 7fcabfe71740 1 -- 172.21.15.37:0/1887300715 shutdown_connections 2024-06-10T01:06:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.846+0000 7fcabfe71740 1 -- 172.21.15.37:0/1887300715 wait complete. 2024-06-10T01:06:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.846+0000 7fcabfe71740 1 -- start start 2024-06-10T01:06:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.846+0000 7fcabfe71740 1 Processor -- start 2024-06-10T01:06:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.846+0000 7fcabfe71740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55617f990010 0x55617fa0b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.846+0000 7fcabfe71740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55617f990a00 0x55617fa0bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.846+0000 7fcabfe71740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55617f995b70 0x55617fa12d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.846+0000 7fcabfe71740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcab81a8dd0 con 0x55617f990010 2024-06-10T01:06:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.846+0000 7fcabfe71740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcab81aa240 con 0x55617f990a00 2024-06-10T01:06:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.846+0000 7fcabfe71740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcab81a94a0 con 0x55617f995b70 2024-06-10T01:06:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.846+0000 7fcabfe71740 10 client.0 _ll_register_callbacks cb 0x7fcaa800bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:06:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.846+0000 7fcab6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55617f990010 0x55617fa0b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.846+0000 7fcab6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55617f990010 0x55617fa0b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49808/0 (socket says 172.21.15.37:49808) 2024-06-10T01:06:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.846+0000 7fcab6d9d640 1 -- 172.21.15.37:0/225599954 learned_addr learned my addr 172.21.15.37:0/225599954 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:06:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.846+0000 7fcab759e640 1 --2- 172.21.15.37:0/225599954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55617f995b70 0x55617fa12d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.846+0000 7fcab659c640 1 --2- 172.21.15.37:0/225599954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55617f990a00 0x55617fa0bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.847+0000 7fcab6d9d640 1 -- 172.21.15.37:0/225599954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55617f995b70 msgr2=0x55617fa12d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.847+0000 7fcab6d9d640 1 --2- 172.21.15.37:0/225599954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55617f995b70 0x55617fa12d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.847+0000 7fcab6d9d640 1 -- 172.21.15.37:0/225599954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55617f990a00 msgr2=0x55617fa0bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:06:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.847+0000 7fcab6d9d640 1 --2- 172.21.15.37:0/225599954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55617f990a00 0x55617fa0bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:06:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.847+0000 7fcab6d9d640 1 -- 172.21.15.37:0/225599954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55617fa13370 con 0x55617f990010 2024-06-10T01:06:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.847+0000 7fcab6d9d640 1 --2- 172.21.15.37:0/225599954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55617f990010 0x55617fa0b800 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fcaa800c010 tx=0x7fcaa800bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c0c61ab91c8194 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:06:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.847+0000 7fca9aff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:06:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.847+0000 7fca9aff5640 1 -- 172.21.15.37:0/225599954 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa803a060 con 0x55617f990010 2024-06-10T01:06:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.847+0000 7fca9aff5640 1 -- 172.21.15.37:0/225599954 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fcaa8004610 con 0x55617f990010 2024-06-10T01:06:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.847+0000 7fcabfe71740 1 -- 172.21.15.37:0/225599954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaac0bbfe0 con 0x55617f990010 2024-06-10T01:06:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.847+0000 7fca9aff5640 1 -- 172.21.15.37:0/225599954 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fcaa8041050 con 0x55617f990010 2024-06-10T01:06:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.847+0000 7fcabfe71740 10 client.17109 fetch_fsmap learned FSMap version 31 2024-06-10T01:06:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.847+0000 7fcabfe71740 1 -- 172.21.15.37:0/225599954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55617fa14680 con 0x55617f990010 2024-06-10T01:06:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.847+0000 7fca9aff5640 1 -- 172.21.15.37:0/225599954 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaac0bbfe0 con 0x55617f990010 2024-06-10T01:06:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.847+0000 7fca9aff5640 1 -- 172.21.15.37:0/225599954 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fcaa8011a10 con 0x55617f990010 2024-06-10T01:06:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.847+0000 7fcabfe71740 10 client.17109 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:06:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.847+0000 7fcabfe71740 10 client.17109 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:06:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.847+0000 7fcabfe71740 20 client.17109 populate_metadata read hostname 'smithi037' 2024-06-10T01:06:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.847+0000 7fcabfe71740 1 --2- 172.21.15.37:0/225599954 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55617fa15050 0x55617fa17470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:06:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.847+0000 7fcabfe71740 4 client.17109 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:06:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.847+0000 7fcabfe71740 1 -- 172.21.15.37:0/225599954 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55617f900c20 con 0x55617fa15050 2024-06-10T01:06:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.848+0000 7fcab659c640 1 --2- 172.21.15.37:0/225599954 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55617fa15050 0x55617fa17470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:06:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.848+0000 7fcab659c640 1 --2- 172.21.15.37:0/225599954 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55617fa15050 0x55617fa17470 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db72af199ec45d1b server_cookie=357c862d2ea8cbd1 in_seq=0 out_seq=0 2024-06-10T01:06:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:06:59.848+0000 7fca9aff5640 10 client.17109 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:00.322+0000 7fca9aff5640 1 -- 172.21.15.37:0/225599954 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55617f900c20 con 0x55617fa15050 2024-06-10T01:07:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:00.322+0000 7fca9aff5640 10 client.17109 handle_command_reply: tid=0 2024-06-10T01:07:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:00.322+0000 7fcabfe71740 1 client.17109 shutdown 2024-06-10T01:07:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:00.322+0000 7fcabfe71740 1 -- 172.21.15.37:0/225599954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55617f990010 msgr2=0x55617fa0b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:00.322+0000 7fcabfe71740 1 --2- 172.21.15.37:0/225599954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55617f990010 0x55617fa0b800 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fcaa800c010 tx=0x7fcaa800bde0 comp rx=0 tx=0).stop 2024-06-10T01:07:00.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:00.322+0000 7fcabfe71740 1 -- 172.21.15.37:0/225599954 shutdown_connections 2024-06-10T01:07:00.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:00.322+0000 7fcabfe71740 1 --2- 172.21.15.37:0/225599954 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55617fa15050 0x55617fa17470 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:00.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:00.322+0000 7fcabfe71740 1 --2- 172.21.15.37:0/225599954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55617f995b70 0x55617fa12d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:00.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:00.322+0000 7fcabfe71740 1 --2- 172.21.15.37:0/225599954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55617f990a00 0x55617fa0bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:00.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:00.322+0000 7fca9aff5640 0 client.17109 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:00.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:00.322+0000 7fcabfe71740 1 --2- 172.21.15.37:0/225599954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55617f990010 0x55617fa0b800 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:00.322+0000 7fcabfe71740 1 -- 172.21.15.37:0/225599954 >> 172.21.15.37:0/225599954 conn(0x55617f98b680 msgr2=0x55617f989f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:00.322+0000 7fcabfe71740 1 -- 172.21.15.37:0/225599954 shutdown_connections 2024-06-10T01:07:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:00.322+0000 7fcabfe71740 1 -- 172.21.15.37:0/225599954 wait complete. 2024-06-10T01:07:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:00.322+0000 7fcabfe71740 20 client.17109 trim_cache size 0 max 16384 2024-06-10T01:07:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:07:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:07:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:07:00.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:00.322+0000 7fcabd16a640 1 -- 172.21.15.37:0/2010769547 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fcaa00a2c30 msgr2=0x7fcaa00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:00.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:00.322+0000 7fcabd16a640 1 --2- 172.21.15.37:0/2010769547 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fcaa00a2c30 0x7fcaa00a50f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcab00098b0 tx=0x7fcab0004a40 comp rx=0 tx=0).stop 2024-06-10T01:07:00.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:00.322+0000 7fcabd16a640 1 -- 172.21.15.37:0/2010769547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab8102c20 msgr2=0x7fcab8198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:00.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:00.322+0000 7fcabd16a640 1 --2- 172.21.15.37:0/2010769547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab8102c20 0x7fcab8198180 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fcaac031440 tx=0x7fcaac032e20 comp rx=0 tx=0).stop 2024-06-10T01:07:00.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:00.323+0000 7fcabd16a640 1 -- 172.21.15.37:0/2010769547 shutdown_connections 2024-06-10T01:07:00.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:00.323+0000 7fcabd16a640 1 --2- 172.21.15.37:0/2010769547 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fcaa00a2c30 0x7fcaa00a50f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:00.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:00.323+0000 7fcabd16a640 1 --2- 172.21.15.37:0/2010769547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcab81035e0 0x7fcab819f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:00.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:00.323+0000 7fcabd16a640 1 --2- 172.21.15.37:0/2010769547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcab8102c20 0x7fcab8198180 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:00.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:00.323+0000 7fcabd16a640 1 --2- 172.21.15.37:0/2010769547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcab8101a30 0x7fcab8197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:00.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:00.323+0000 7fcabd16a640 1 -- 172.21.15.37:0/2010769547 >> 172.21.15.37:0/2010769547 conn(0x7fcab80fcfc0 msgr2=0x7fcab8105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:00.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:00.323+0000 7fcabd16a640 1 -- 172.21.15.37:0/2010769547 shutdown_connections 2024-06-10T01:07:00.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:00.323+0000 7fcabd16a640 1 -- 172.21.15.37:0/2010769547 wait complete. 2024-06-10T01:07:00.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:00 smithi094 ceph-mon[36708]: pgmap v740: 129 pgs: 129 active+clean; 6.6 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 227 KiB/s wr, 3.03k op/s 2024-06-10T01:07:00.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:00 smithi037 ceph-mon[32192]: pgmap v740: 129 pgs: 129 active+clean; 6.6 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 227 KiB/s wr, 3.03k op/s 2024-06-10T01:07:00.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:00 smithi167 ceph-mon[36917]: pgmap v740: 129 pgs: 129 active+clean; 6.6 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 227 KiB/s wr, 3.03k op/s 2024-06-10T01:07:01.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.482+0000 7f5c33c39700 1 -- 172.21.15.37:0/3398231411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c2c102380 msgr2=0x7f5c2c105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:01.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.482+0000 7f5c33c39700 1 --2- 172.21.15.37:0/3398231411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c2c102380 0x7f5c2c105ff0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c0099c0 tx=0x7f5c1c009cd0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.482+0000 7f5c33c39700 1 -- 172.21.15.37:0/3398231411 shutdown_connections 2024-06-10T01:07:01.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.482+0000 7f5c33c39700 1 --2- 172.21.15.37:0/3398231411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c2c106720 0x7f5c2c10e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.482+0000 7f5c33c39700 1 --2- 172.21.15.37:0/3398231411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c2c1019d0 0x7f5c2c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.482+0000 7f5c33c39700 1 --2- 172.21.15.37:0/3398231411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c2c102380 0x7f5c2c105ff0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.482+0000 7f5c33c39700 1 -- 172.21.15.37:0/3398231411 >> 172.21.15.37:0/3398231411 conn(0x7f5c2c078340 msgr2=0x7f5c2c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:01.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.482+0000 7f5c33c39700 1 -- 172.21.15.37:0/3398231411 shutdown_connections 2024-06-10T01:07:01.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.482+0000 7f5c33c39700 1 -- 172.21.15.37:0/3398231411 wait complete. 2024-06-10T01:07:01.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.483+0000 7f5c33c39700 1 Processor -- start 2024-06-10T01:07:01.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.483+0000 7f5c33c39700 1 -- start start 2024-06-10T01:07:01.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.483+0000 7f5c33c39700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c2c1019d0 0x7f5c2c19e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:01.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.483+0000 7f5c33c39700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c2c102380 0x7f5c2c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:01.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.483+0000 7f5c33c39700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c2c106720 0x7f5c2c198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:01.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.483+0000 7f5c33c39700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c111690 con 0x7f5c2c1019d0 2024-06-10T01:07:01.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.483+0000 7f5c33c39700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c111510 con 0x7f5c2c106720 2024-06-10T01:07:01.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.483+0000 7f5c33c39700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c111810 con 0x7f5c2c102380 2024-06-10T01:07:01.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.484+0000 7f5c319d5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c2c1019d0 0x7f5c2c19e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:01.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.484+0000 7f5c319d5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c2c1019d0 0x7f5c2c19e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49814/0 (socket says 172.21.15.37:49814) 2024-06-10T01:07:01.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.484+0000 7f5c319d5700 1 -- 172.21.15.37:0/3630808026 learned_addr learned my addr 172.21.15.37:0/3630808026 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:01.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.484+0000 7f5c311d4700 1 --2- 172.21.15.37:0/3630808026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c2c102380 0x7f5c2c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:01.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.484+0000 7f5c321d6700 1 --2- 172.21.15.37:0/3630808026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c2c106720 0x7f5c2c198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:01.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.484+0000 7f5c319d5700 1 -- 172.21.15.37:0/3630808026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c2c102380 msgr2=0x7f5c2c19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:01.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.484+0000 7f5c319d5700 1 --2- 172.21.15.37:0/3630808026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c2c102380 0x7f5c2c19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.484+0000 7f5c319d5700 1 -- 172.21.15.37:0/3630808026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c2c106720 msgr2=0x7f5c2c198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:01.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.484+0000 7f5c319d5700 1 --2- 172.21.15.37:0/3630808026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c2c106720 0x7f5c2c198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.484+0000 7f5c319d5700 1 -- 172.21.15.37:0/3630808026 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c2c199640 con 0x7f5c2c1019d0 2024-06-10T01:07:01.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.484+0000 7f5c321d6700 1 --2- 172.21.15.37:0/3630808026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c2c106720 0x7f5c2c198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:07:01.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.484+0000 7f5c319d5700 1 --2- 172.21.15.37:0/3630808026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c2c1019d0 0x7f5c2c19e9a0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f5c2800e9a0 tx=0x7f5c2800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:01.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.484+0000 7f5c22ffd700 1 -- 172.21.15.37:0/3630808026 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2800cbd0 con 0x7f5c2c1019d0 2024-06-10T01:07:01.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.484+0000 7f5c22ffd700 1 -- 172.21.15.37:0/3630808026 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c2800cd70 con 0x7f5c2c1019d0 2024-06-10T01:07:01.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.484+0000 7f5c33c39700 1 -- 172.21.15.37:0/3630808026 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c2c199930 con 0x7f5c2c1019d0 2024-06-10T01:07:01.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.485+0000 7f5c22ffd700 1 -- 172.21.15.37:0/3630808026 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28018750 con 0x7f5c2c1019d0 2024-06-10T01:07:01.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.485+0000 7f5c33c39700 1 -- 172.21.15.37:0/3630808026 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c2c112020 con 0x7f5c2c1019d0 2024-06-10T01:07:01.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.487+0000 7f5c22ffd700 1 -- 172.21.15.37:0/3630808026 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5c28010430 con 0x7f5c2c1019d0 2024-06-10T01:07:01.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.487+0000 7f5c22ffd700 1 --2- 172.21.15.37:0/3630808026 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5c180a2c20 0x7f5c180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:01.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.488+0000 7f5c311d4700 1 --2- 172.21.15.37:0/3630808026 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5c180a2c20 0x7f5c180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:01.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.488+0000 7f5c22ffd700 1 -- 172.21.15.37:0/3630808026 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f5c280cf780 con 0x7f5c2c1019d0 2024-06-10T01:07:01.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.488+0000 7f5c311d4700 1 --2- 172.21.15.37:0/3630808026 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5c180a2c20 0x7f5c180a50e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c009990 tx=0x7f5c1c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:01.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.489+0000 7f5c3e4bcb80 1 Processor -- start 2024-06-10T01:07:01.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.489+0000 7f5c3e4bcb80 1 -- start start 2024-06-10T01:07:01.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.489+0000 7f5c3e4bcb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628931d7c40 0x5628931d8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:01.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.489+0000 7f5c3e4bcb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628931d8560 0x5628931d1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:01.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.489+0000 7f5c3e4bcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628931d23d0 0x5628931d2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:01.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.489+0000 7f5c3e4bcb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562893123910 con 0x5628931d7c40 2024-06-10T01:07:01.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.489+0000 7f5c3e4bcb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5628930cb820 con 0x5628931d8560 2024-06-10T01:07:01.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.489+0000 7f5c3e4bcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56289311f8d0 con 0x5628931d23d0 2024-06-10T01:07:01.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.489+0000 7f5c319d5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628931d7c40 0x5628931d8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:01.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.489+0000 7f5c319d5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628931d7c40 0x5628931d8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49824/0 (socket says 172.21.15.37:49824) 2024-06-10T01:07:01.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.489+0000 7f5c319d5700 1 -- 172.21.15.37:0/2357649023 learned_addr learned my addr 172.21.15.37:0/2357649023 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:01.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.489+0000 7f5c311d4700 1 --2- 172.21.15.37:0/2357649023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628931d8560 0x5628931d1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:01.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.489+0000 7f5c321d6700 1 --2- 172.21.15.37:0/2357649023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628931d23d0 0x5628931d2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:01.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.490+0000 7f5c319d5700 1 -- 172.21.15.37:0/2357649023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628931d23d0 msgr2=0x5628931d2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:01.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.490+0000 7f5c319d5700 1 --2- 172.21.15.37:0/2357649023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628931d23d0 0x5628931d2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.490+0000 7f5c319d5700 1 -- 172.21.15.37:0/2357649023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628931d8560 msgr2=0x5628931d1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:01.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.490+0000 7f5c319d5700 1 --2- 172.21.15.37:0/2357649023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628931d8560 0x5628931d1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.490+0000 7f5c319d5700 1 -- 172.21.15.37:0/2357649023 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628930c2050 con 0x5628931d7c40 2024-06-10T01:07:01.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.490+0000 7f5c319d5700 1 --2- 172.21.15.37:0/2357649023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628931d7c40 0x5628931d8020 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f5c280a42d0 tx=0x7f5c2809c290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eefb0cb30c150c03 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:01.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.490+0000 7f5c17fff700 1 -- 172.21.15.37:0/2357649023 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2809ff40 con 0x5628931d7c40 2024-06-10T01:07:01.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.490+0000 7f5c17fff700 1 -- 172.21.15.37:0/2357649023 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c280a4800 con 0x5628931d7c40 2024-06-10T01:07:01.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.490+0000 7f5c17fff700 1 -- 172.21.15.37:0/2357649023 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c280a4370 con 0x5628931d7c40 2024-06-10T01:07:01.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.491+0000 7f5c3e4bcb80 1 -- 172.21.15.37:0/2357649023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628931d7c40 msgr2=0x5628931d8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:01.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.491+0000 7f5c3e4bcb80 1 --2- 172.21.15.37:0/2357649023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628931d7c40 0x5628931d8020 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f5c280a42d0 tx=0x7f5c2809c290 comp rx=0 tx=0).stop 2024-06-10T01:07:01.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.491+0000 7f5c3e4bcb80 1 -- 172.21.15.37:0/2357649023 shutdown_connections 2024-06-10T01:07:01.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.491+0000 7f5c3e4bcb80 1 --2- 172.21.15.37:0/2357649023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628931d23d0 0x5628931d2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.491+0000 7f5c3e4bcb80 1 --2- 172.21.15.37:0/2357649023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628931d7c40 0x5628931d8020 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.491+0000 7f5c3e4bcb80 1 --2- 172.21.15.37:0/2357649023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628931d8560 0x5628931d1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.491+0000 7f5c3e4bcb80 1 -- 172.21.15.37:0/2357649023 >> 172.21.15.37:0/2357649023 conn(0x5628930fcdc0 msgr2=0x5628931cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:01.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.491+0000 7f5c3e4bcb80 1 -- 172.21.15.37:0/2357649023 shutdown_connections 2024-06-10T01:07:01.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.491+0000 7f5c3e4bcb80 1 -- 172.21.15.37:0/2357649023 wait complete. 2024-06-10T01:07:01.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.491+0000 7f5c3e4bcb80 1 -- start start 2024-06-10T01:07:01.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.492+0000 7f5c3e4bcb80 1 Processor -- start 2024-06-10T01:07:01.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.492+0000 7f5c3e4bcb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628931d23d0 0x5628931de240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:01.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.492+0000 7f5c3e4bcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628931d7c40 0x5628931db540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:01.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.492+0000 7f5c3e4bcb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628931d8560 0x5628931dba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:01.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.492+0000 7f5c3e4bcb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5628931defd0 con 0x5628931d23d0 2024-06-10T01:07:01.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.492+0000 7f5c3e4bcb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5628930c3cf0 con 0x5628931d8560 2024-06-10T01:07:01.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.492+0000 7f5c3e4bcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5628931df150 con 0x5628931d7c40 2024-06-10T01:07:01.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.492+0000 7f5c3e4bcb80 10 client.0 _ll_register_callbacks cb 0x5628930bda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:01.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.492+0000 7f5c319d5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628931d23d0 0x5628931de240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:01.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.492+0000 7f5c319d5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628931d23d0 0x5628931de240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49832/0 (socket says 172.21.15.37:49832) 2024-06-10T01:07:01.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.492+0000 7f5c319d5700 1 -- 172.21.15.37:0/719539655 learned_addr learned my addr 172.21.15.37:0/719539655 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:01.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.492+0000 7f5c321d6700 1 --2- 172.21.15.37:0/719539655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628931d8560 0x5628931dba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:01.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.492+0000 7f5c311d4700 1 --2- 172.21.15.37:0/719539655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628931d7c40 0x5628931db540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:01.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.492+0000 7f5c319d5700 1 -- 172.21.15.37:0/719539655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628931d7c40 msgr2=0x5628931db540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:01.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.492+0000 7f5c319d5700 1 --2- 172.21.15.37:0/719539655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628931d7c40 0x5628931db540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.492+0000 7f5c319d5700 1 -- 172.21.15.37:0/719539655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628931d8560 msgr2=0x5628931dba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:01.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.492+0000 7f5c319d5700 1 --2- 172.21.15.37:0/719539655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628931d8560 0x5628931dba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.493+0000 7f5c319d5700 1 -- 172.21.15.37:0/719539655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628930729d0 con 0x5628931d23d0 2024-06-10T01:07:01.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.493+0000 7f5c321d6700 1 --2- 172.21.15.37:0/719539655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628931d8560 0x5628931dba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:01.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.493+0000 7f5c319d5700 1 --2- 172.21.15.37:0/719539655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628931d23d0 0x5628931de240 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f5c2809bb20 tx=0x7f5c2809ff40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9dc428d6fc5ee4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:01.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.493+0000 7f5c157fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:07:01.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.493+0000 7f5c157fa700 1 -- 172.21.15.37:0/719539655 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c280a4370 con 0x5628931d23d0 2024-06-10T01:07:01.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.493+0000 7f5c157fa700 1 -- 172.21.15.37:0/719539655 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f5c280a4510 con 0x5628931d23d0 2024-06-10T01:07:01.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.493+0000 7f5c3e4bcb80 1 -- 172.21.15.37:0/719539655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c280a4800 con 0x5628931d23d0 2024-06-10T01:07:01.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.493+0000 7f5c157fa700 1 -- 172.21.15.37:0/719539655 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5c2809afe0 con 0x5628931d23d0 2024-06-10T01:07:01.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.493+0000 7f5c3e4bcb80 10 client.17121 fetch_fsmap learned FSMap version 31 2024-06-10T01:07:01.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.493+0000 7f5c3e4bcb80 1 -- 172.21.15.37:0/719539655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5628931defd0 con 0x5628931d23d0 2024-06-10T01:07:01.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.493+0000 7f5c157fa700 1 -- 172.21.15.37:0/719539655 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c280a4800 con 0x5628931d23d0 2024-06-10T01:07:01.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.493+0000 7f5c157fa700 1 -- 172.21.15.37:0/719539655 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f5c280ce040 con 0x5628931d23d0 2024-06-10T01:07:01.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.494+0000 7f5c3e4bcb80 10 client.17121 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:07:01.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.494+0000 7f5c3e4bcb80 10 client.17121 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:07:01.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.494+0000 7f5c3e4bcb80 20 client.17121 populate_metadata read hostname 'smithi037' 2024-06-10T01:07:01.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.494+0000 7f5c3e4bcb80 1 --2- 172.21.15.37:0/719539655 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5628931ebb70 0x5628931edf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:01.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.494+0000 7f5c3e4bcb80 4 client.17121 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:01.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.494+0000 7f5c3e4bcb80 1 -- 172.21.15.37:0/719539655 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562893130e10 con 0x5628931ebb70 2024-06-10T01:07:01.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.494+0000 7f5c311d4700 1 --2- 172.21.15.37:0/719539655 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5628931ebb70 0x5628931edf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:01.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.494+0000 7f5c311d4700 1 --2- 172.21.15.37:0/719539655 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5628931ebb70 0x5628931edf60 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f92a068287a46ef server_cookie=c45ca04928cf0a02 in_seq=0 out_seq=0 2024-06-10T01:07:01.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.494+0000 7f5c157fa700 10 client.17121 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:01.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.495+0000 7f5c157fa700 1 -- 172.21.15.37:0/719539655 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562893130e10 con 0x5628931ebb70 2024-06-10T01:07:01.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.495+0000 7f5c157fa700 10 client.17121 handle_command_reply: tid=0 2024-06-10T01:07:01.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.495+0000 7f5c3e4bcb80 1 client.17121 shutdown 2024-06-10T01:07:01.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.495+0000 7f5c3e4bcb80 1 -- 172.21.15.37:0/719539655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628931d23d0 msgr2=0x5628931de240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:01.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.495+0000 7f5c3e4bcb80 1 --2- 172.21.15.37:0/719539655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628931d23d0 0x5628931de240 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f5c2809bb20 tx=0x7f5c2809ff40 comp rx=0 tx=0).stop 2024-06-10T01:07:01.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.495+0000 7f5c3e4bcb80 1 -- 172.21.15.37:0/719539655 shutdown_connections 2024-06-10T01:07:01.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.495+0000 7f5c3e4bcb80 1 --2- 172.21.15.37:0/719539655 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5628931ebb70 0x5628931edf60 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.495+0000 7f5c3e4bcb80 1 --2- 172.21.15.37:0/719539655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628931d8560 0x5628931dba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.495+0000 7f5c3e4bcb80 1 --2- 172.21.15.37:0/719539655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628931d23d0 0x5628931de240 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.495+0000 7f5c157fa700 0 client.17121 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:01.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.495+0000 7f5c3e4bcb80 1 --2- 172.21.15.37:0/719539655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628931d7c40 0x5628931db540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.495+0000 7f5c3e4bcb80 1 -- 172.21.15.37:0/719539655 >> 172.21.15.37:0/719539655 conn(0x5628930fcdc0 msgr2=0x5628931ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:01.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.495+0000 7f5c3e4bcb80 1 -- 172.21.15.37:0/719539655 shutdown_connections 2024-06-10T01:07:01.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.495+0000 7f5c3e4bcb80 1 -- 172.21.15.37:0/719539655 wait complete. 2024-06-10T01:07:01.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.496+0000 7f5c3e4bcb80 20 client.17121 trim_cache size 0 max 16384 2024-06-10T01:07:01.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.516+0000 7f5c3e4bcb80 1 Processor -- start 2024-06-10T01:07:01.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.516+0000 7f5c3e4bcb80 1 -- start start 2024-06-10T01:07:01.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.516+0000 7f5c3e4bcb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628931d23d0 0x5628931d1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:01.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.516+0000 7f5c3e4bcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628931d7c40 0x5628931da410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:01.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.516+0000 7f5c3e4bcb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628931d8560 0x5628931f0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:01.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.516+0000 7f5c3e4bcb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c1b3090 con 0x5628931d8560 2024-06-10T01:07:01.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.516+0000 7f5c3e4bcb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c1b0920 con 0x5628931d23d0 2024-06-10T01:07:01.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.516+0000 7f5c3e4bcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c1ae770 con 0x5628931d7c40 2024-06-10T01:07:01.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.517+0000 7f5c321d6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628931d8560 0x5628931f0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:01.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.517+0000 7f5c321d6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628931d8560 0x5628931f0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49846/0 (socket says 172.21.15.37:49846) 2024-06-10T01:07:01.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.517+0000 7f5c321d6700 1 -- 172.21.15.37:0/2823395795 learned_addr learned my addr 172.21.15.37:0/2823395795 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:01.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.517+0000 7f5c319d5700 1 --2- 172.21.15.37:0/2823395795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628931d23d0 0x5628931d1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:01.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.517+0000 7f5c311d4700 1 --2- 172.21.15.37:0/2823395795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628931d7c40 0x5628931da410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:01.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.517+0000 7f5c321d6700 1 -- 172.21.15.37:0/2823395795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628931d7c40 msgr2=0x5628931da410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:01.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.517+0000 7f5c321d6700 1 --2- 172.21.15.37:0/2823395795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628931d7c40 0x5628931da410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.517+0000 7f5c321d6700 1 -- 172.21.15.37:0/2823395795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628931d23d0 msgr2=0x5628931d1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:01.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.517+0000 7f5c321d6700 1 --2- 172.21.15.37:0/2823395795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628931d23d0 0x5628931d1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.517+0000 7f5c321d6700 1 -- 172.21.15.37:0/2823395795 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628931c7820 con 0x5628931d8560 2024-06-10T01:07:01.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.517+0000 7f5c321d6700 1 --2- 172.21.15.37:0/2823395795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628931d8560 0x5628931f0f80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009f00 tx=0x7f5c24009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f397cb4d16ecf742 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:01.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.517+0000 7f5c177fe700 1 -- 172.21.15.37:0/2823395795 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2401b020 con 0x5628931d8560 2024-06-10T01:07:01.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.517+0000 7f5c177fe700 1 -- 172.21.15.37:0/2823395795 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c24017950 con 0x5628931d8560 2024-06-10T01:07:01.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.517+0000 7f5c177fe700 1 -- 172.21.15.37:0/2823395795 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c24016a00 con 0x5628931d8560 2024-06-10T01:07:01.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.518+0000 7f5c3e4bcb80 1 -- 172.21.15.37:0/2823395795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628931d8560 msgr2=0x5628931f0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:01.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.518+0000 7f5c3e4bcb80 1 --2- 172.21.15.37:0/2823395795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628931d8560 0x5628931f0f80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009f00 tx=0x7f5c24009040 comp rx=0 tx=0).stop 2024-06-10T01:07:01.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.518+0000 7f5c3e4bcb80 1 -- 172.21.15.37:0/2823395795 shutdown_connections 2024-06-10T01:07:01.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.518+0000 7f5c3e4bcb80 1 --2- 172.21.15.37:0/2823395795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628931d8560 0x5628931f0f80 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.518+0000 7f5c3e4bcb80 1 --2- 172.21.15.37:0/2823395795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628931d23d0 0x5628931d1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.518+0000 7f5c3e4bcb80 1 --2- 172.21.15.37:0/2823395795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628931d7c40 0x5628931da410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.518+0000 7f5c3e4bcb80 1 -- 172.21.15.37:0/2823395795 >> 172.21.15.37:0/2823395795 conn(0x5628930fcdc0 msgr2=0x5628931d9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:01.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.518+0000 7f5c3e4bcb80 1 -- 172.21.15.37:0/2823395795 shutdown_connections 2024-06-10T01:07:01.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.518+0000 7f5c3e4bcb80 1 -- 172.21.15.37:0/2823395795 wait complete. 2024-06-10T01:07:01.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.519+0000 7f5c3e4bcb80 1 -- start start 2024-06-10T01:07:01.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.519+0000 7f5c3e4bcb80 1 Processor -- start 2024-06-10T01:07:01.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.519+0000 7f5c3e4bcb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628931d23d0 0x5628931ef590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:01.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.519+0000 7f5c3e4bcb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628931d7c40 0x5628931efad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:01.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.519+0000 7f5c3e4bcb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628931d8560 0x56289324f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:01.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.519+0000 7f5c3e4bcb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c1aede0 con 0x5628931d23d0 2024-06-10T01:07:01.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.519+0000 7f5c3e4bcb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c1b0250 con 0x5628931d8560 2024-06-10T01:07:01.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.519+0000 7f5c3e4bcb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c1af4b0 con 0x5628931d7c40 2024-06-10T01:07:01.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.519+0000 7f5c3e4bcb80 10 client.0 _ll_register_callbacks cb 0x5628930bda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:01.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.519+0000 7f5c319d5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628931d23d0 0x5628931ef590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:01.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.519+0000 7f5c319d5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628931d23d0 0x5628931ef590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49854/0 (socket says 172.21.15.37:49854) 2024-06-10T01:07:01.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.519+0000 7f5c319d5700 1 -- 172.21.15.37:0/2310348234 learned_addr learned my addr 172.21.15.37:0/2310348234 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:01.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.520+0000 7f5c321d6700 1 --2- 172.21.15.37:0/2310348234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628931d8560 0x56289324f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:01.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.520+0000 7f5c311d4700 1 --2- 172.21.15.37:0/2310348234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628931d7c40 0x5628931efad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:01.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.520+0000 7f5c319d5700 1 -- 172.21.15.37:0/2310348234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628931d7c40 msgr2=0x5628931efad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:01.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.520+0000 7f5c319d5700 1 --2- 172.21.15.37:0/2310348234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628931d7c40 0x5628931efad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.520+0000 7f5c319d5700 1 -- 172.21.15.37:0/2310348234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628931d8560 msgr2=0x56289324f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:01.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.520+0000 7f5c319d5700 1 --2- 172.21.15.37:0/2310348234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628931d8560 0x56289324f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:01.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.520+0000 7f5c319d5700 1 -- 172.21.15.37:0/2310348234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56289324fa70 con 0x5628931d23d0 2024-06-10T01:07:01.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.520+0000 7f5c319d5700 1 --2- 172.21.15.37:0/2310348234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628931d23d0 0x5628931ef590 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f5c2800c5b0 tx=0x7f5c280a0060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859c271c87293855 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:01.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.520+0000 7f5c157fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:07:01.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.520+0000 7f5c157fa700 1 -- 172.21.15.37:0/2310348234 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2809ac50 con 0x5628931d23d0 2024-06-10T01:07:01.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.520+0000 7f5c157fa700 1 -- 172.21.15.37:0/2310348234 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f5c280ba310 con 0x5628931d23d0 2024-06-10T01:07:01.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.520+0000 7f5c3e4bcb80 1 -- 172.21.15.37:0/2310348234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c24017950 con 0x5628931d23d0 2024-06-10T01:07:01.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.520+0000 7f5c157fa700 1 -- 172.21.15.37:0/2310348234 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5c280a4800 con 0x5628931d23d0 2024-06-10T01:07:01.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.520+0000 7f5c3e4bcb80 10 client.17127 fetch_fsmap learned FSMap version 31 2024-06-10T01:07:01.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.520+0000 7f5c3e4bcb80 1 -- 172.21.15.37:0/2310348234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56289324fa70 con 0x5628931d23d0 2024-06-10T01:07:01.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.521+0000 7f5c157fa700 1 -- 172.21.15.37:0/2310348234 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c24017950 con 0x5628931d23d0 2024-06-10T01:07:01.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.521+0000 7f5c157fa700 1 -- 172.21.15.37:0/2310348234 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f5c280ce070 con 0x5628931d23d0 2024-06-10T01:07:01.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.521+0000 7f5c3e4bcb80 10 client.17127 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:07:01.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.521+0000 7f5c3e4bcb80 10 client.17127 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:07:01.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.521+0000 7f5c3e4bcb80 20 client.17127 populate_metadata read hostname 'smithi037' 2024-06-10T01:07:01.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.521+0000 7f5c3e4bcb80 1 --2- 172.21.15.37:0/2310348234 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5628932570e0 0x562893259500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:01.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.521+0000 7f5c3e4bcb80 4 client.17127 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "cf050765", "release": true, "await": true}] 2024-06-10T01:07:01.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.521+0000 7f5c3e4bcb80 1 -- 172.21.15.37:0/2310348234 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "quiesce db", "set_id": "cf050765", "release": true, "await": true}) -- 0x56289312b390 con 0x5628932570e0 2024-06-10T01:07:01.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.521+0000 7f5c311d4700 1 --2- 172.21.15.37:0/2310348234 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5628932570e0 0x562893259500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:01.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.522+0000 7f5c311d4700 1 --2- 172.21.15.37:0/2310348234 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5628932570e0 0x562893259500 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9bd7f4e08ef7871 server_cookie=5a5987ed6e71366d in_seq=0 out_seq=0 2024-06-10T01:07:01.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:01.522+0000 7f5c157fa700 10 client.17127 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:01.880 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:07:02.144 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:02.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:02 smithi094 ceph-mon[36708]: pgmap v741: 129 pgs: 129 active+clean; 6.6 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 56 KiB/s wr, 2.52k op/s 2024-06-10T01:07:02.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:02 smithi037 ceph-mon[32192]: pgmap v741: 129 pgs: 129 active+clean; 6.6 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 56 KiB/s wr, 2.52k op/s 2024-06-10T01:07:02.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:02 smithi167 ceph-mon[36917]: pgmap v741: 129 pgs: 129 active+clean; 6.6 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 56 KiB/s wr, 2.52k op/s 2024-06-10T01:07:02.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:02.795+0000 7f5c157fa700 1 -- 172.21.15.37:0/2310348234 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56289312b390 con 0x5628932570e0 2024-06-10T01:07:02.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:02.795+0000 7f5c157fa700 10 client.17127 handle_command_reply: tid=0 2024-06-10T01:07:02.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:02.795+0000 7f5c3e4bcb80 1 client.17127 shutdown 2024-06-10T01:07:02.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:02.795+0000 7f5c3e4bcb80 1 -- 172.21.15.37:0/2310348234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628931d23d0 msgr2=0x5628931ef590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:02.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:02.795+0000 7f5c3e4bcb80 1 --2- 172.21.15.37:0/2310348234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628931d23d0 0x5628931ef590 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f5c2800c5b0 tx=0x7f5c280a0060 comp rx=0 tx=0).stop 2024-06-10T01:07:02.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:02.795+0000 7f5c3e4bcb80 1 -- 172.21.15.37:0/2310348234 shutdown_connections 2024-06-10T01:07:02.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:02.795+0000 7f5c3e4bcb80 1 --2- 172.21.15.37:0/2310348234 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5628932570e0 0x562893259500 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:02.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:02.795+0000 7f5c3e4bcb80 1 --2- 172.21.15.37:0/2310348234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628931d8560 0x56289324f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:02.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:02.795+0000 7f5c3e4bcb80 1 --2- 172.21.15.37:0/2310348234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628931d23d0 0x5628931ef590 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:02.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:02.795+0000 7f5c3e4bcb80 1 --2- 172.21.15.37:0/2310348234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628931d7c40 0x5628931efad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:02.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:02.795+0000 7f5c3e4bcb80 1 -- 172.21.15.37:0/2310348234 >> 172.21.15.37:0/2310348234 conn(0x5628930fcdc0 msgr2=0x5628931d6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:02.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:02.795+0000 7f5c157fa700 0 client.17127 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:02.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:02.795+0000 7f5c3e4bcb80 1 -- 172.21.15.37:0/2310348234 shutdown_connections 2024-06-10T01:07:02.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:02.795+0000 7f5c3e4bcb80 1 -- 172.21.15.37:0/2310348234 wait complete. 2024-06-10T01:07:02.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:02.795+0000 7f5c3e4bcb80 20 client.17127 trim_cache size 0 max 16384 2024-06-10T01:07:02.794 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:07:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 30, 2024-06-10T01:07:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24472, 2024-06-10T01:07:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 24, 2024-06-10T01:07:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-06-10T01:07:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: "cf050765": { 2024-06-10T01:07:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 24, 2024-06-10T01:07:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-06-10T01:07:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:07:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-06-10T01:07:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:07:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:07:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-06-10T01:07:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 136.0, 2024-06-10T01:07:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-06-10T01:07:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-06-10T01:07:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-06-10T01:07:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:07:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-06-10T01:07:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:07:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:07:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:07:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:07:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:07:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:07:02.796 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:07:02.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:02.796+0000 7f5c33c39700 1 -- 172.21.15.37:0/3630808026 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5c180a2c20 msgr2=0x7f5c180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:02.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:02.796+0000 7f5c33c39700 1 --2- 172.21.15.37:0/3630808026 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5c180a2c20 0x7f5c180a50e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c009990 tx=0x7f5c1c005fb0 comp rx=0 tx=0).stop 2024-06-10T01:07:02.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:02.796+0000 7f5c33c39700 1 -- 172.21.15.37:0/3630808026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c2c1019d0 msgr2=0x7f5c2c19e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:02.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:02.796+0000 7f5c33c39700 1 --2- 172.21.15.37:0/3630808026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c2c1019d0 0x7f5c2c19e9a0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f5c2800e9a0 tx=0x7f5c2800ecb0 comp rx=0 tx=0).stop 2024-06-10T01:07:02.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:02.796+0000 7f5c33c39700 1 -- 172.21.15.37:0/3630808026 shutdown_connections 2024-06-10T01:07:02.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:02.796+0000 7f5c33c39700 1 --2- 172.21.15.37:0/3630808026 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5c180a2c20 0x7f5c180a50e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:02.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:02.796+0000 7f5c33c39700 1 --2- 172.21.15.37:0/3630808026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c2c106720 0x7f5c2c198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:02.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:02.796+0000 7f5c33c39700 1 --2- 172.21.15.37:0/3630808026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c2c1019d0 0x7f5c2c19e9a0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:02.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:02.796+0000 7f5c33c39700 1 --2- 172.21.15.37:0/3630808026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c2c102380 0x7f5c2c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:02.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:02.796+0000 7f5c33c39700 1 -- 172.21.15.37:0/3630808026 >> 172.21.15.37:0/3630808026 conn(0x7f5c2c078340 msgr2=0x7f5c2c104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:02.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:02.796+0000 7f5c33c39700 1 -- 172.21.15.37:0/3630808026 shutdown_connections 2024-06-10T01:07:02.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:02.796+0000 7f5c33c39700 1 -- 172.21.15.37:0/3630808026 wait complete. 2024-06-10T01:07:03.497 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'cf050765', total seconds elapsed: 40.4 2024-06-10T01:07:04.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:04 smithi037 ceph-mon[32192]: pgmap v742: 129 pgs: 129 active+clean; 6.6 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 38 KiB/s wr, 2.41k op/s 2024-06-10T01:07:04.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:04 smithi094 ceph-mon[36708]: pgmap v742: 129 pgs: 129 active+clean; 6.6 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 38 KiB/s wr, 2.41k op/s 2024-06-10T01:07:04.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.524+0000 7f8eb9dfb700 1 -- 172.21.15.37:0/2557538217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8eb4107460 msgr2=0x7f8eb4107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:04.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.524+0000 7f8eb9dfb700 1 --2- 172.21.15.37:0/2557538217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8eb4107460 0x7f8eb4107840 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4009990 tx=0x7f8ea4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.524+0000 7f8eb9dfb700 1 -- 172.21.15.37:0/2557538217 shutdown_connections 2024-06-10T01:07:04.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.524+0000 7f8eb9dfb700 1 --2- 172.21.15.37:0/2557538217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8eb406b520 0x7f8eb410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.524+0000 7f8eb9dfb700 1 --2- 172.21.15.37:0/2557538217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8eb4107460 0x7f8eb4107840 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.524+0000 7f8eb9dfb700 1 --2- 172.21.15.37:0/2557538217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8eb406ab60 0x7f8eb406afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.524+0000 7f8eb9dfb700 1 -- 172.21.15.37:0/2557538217 >> 172.21.15.37:0/2557538217 conn(0x7f8eb4078340 msgr2=0x7f8eb4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:04.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.524+0000 7f8eb9dfb700 1 -- 172.21.15.37:0/2557538217 shutdown_connections 2024-06-10T01:07:04.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.525+0000 7f8eb9dfb700 1 -- 172.21.15.37:0/2557538217 wait complete. 2024-06-10T01:07:04.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.525+0000 7f8eb9dfb700 1 Processor -- start 2024-06-10T01:07:04.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.525+0000 7f8eb9dfb700 1 -- start start 2024-06-10T01:07:04.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.526+0000 7f8eb9dfb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8eb406ab60 0x7f8eb419e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:04.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.526+0000 7f8eb9dfb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8eb406b520 0x7f8eb419ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:04.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.526+0000 7f8eb9dfb700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8eb4107460 0x7f8eb4198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:04.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.526+0000 7f8eb9dfb700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb4112740 con 0x7f8eb406ab60 2024-06-10T01:07:04.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.526+0000 7f8eb9dfb700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb41125c0 con 0x7f8eb406b520 2024-06-10T01:07:04.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.526+0000 7f8eb9dfb700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb41128c0 con 0x7f8eb4107460 2024-06-10T01:07:04.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.526+0000 7f8eb37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8eb406ab60 0x7f8eb419e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:04.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.526+0000 7f8eb3fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8eb4107460 0x7f8eb4198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:04.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.526+0000 7f8eb37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8eb406ab60 0x7f8eb419e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46186/0 (socket says 172.21.15.37:46186) 2024-06-10T01:07:04.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.526+0000 7f8eb37fe700 1 -- 172.21.15.37:0/2085604264 learned_addr learned my addr 172.21.15.37:0/2085604264 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:04.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.526+0000 7f8eaaffd700 1 --2- 172.21.15.37:0/2085604264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8eb406b520 0x7f8eb419ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:04.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.526+0000 7f8eb3fff700 1 -- 172.21.15.37:0/2085604264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8eb406b520 msgr2=0x7f8eb419ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:04.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.526+0000 7f8eb3fff700 1 --2- 172.21.15.37:0/2085604264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8eb406b520 0x7f8eb419ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.526+0000 7f8eb3fff700 1 -- 172.21.15.37:0/2085604264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8eb406ab60 msgr2=0x7f8eb419e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:04.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.526+0000 7f8eb3fff700 1 --2- 172.21.15.37:0/2085604264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8eb406ab60 0x7f8eb419e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.526+0000 7f8eb3fff700 1 -- 172.21.15.37:0/2085604264 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb41990c0 con 0x7f8eb4107460 2024-06-10T01:07:04.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.526+0000 7f8eb37fe700 1 --2- 172.21.15.37:0/2085604264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8eb406ab60 0x7f8eb419e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:04.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.526+0000 7f8eb3fff700 1 --2- 172.21.15.37:0/2085604264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8eb4107460 0x7f8eb4198960 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8eac007f10 tx=0x7f8eac00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:04.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.527+0000 7f8eb17fa700 1 -- 172.21.15.37:0/2085604264 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac00ddd0 con 0x7f8eb4107460 2024-06-10T01:07:04.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.527+0000 7f8eb9dfb700 1 -- 172.21.15.37:0/2085604264 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb41993b0 con 0x7f8eb4107460 2024-06-10T01:07:04.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.527+0000 7f8eb17fa700 1 -- 172.21.15.37:0/2085604264 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8eac004bb0 con 0x7f8eb4107460 2024-06-10T01:07:04.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.527+0000 7f8eb17fa700 1 -- 172.21.15.37:0/2085604264 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac0046a0 con 0x7f8eb4107460 2024-06-10T01:07:04.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.527+0000 7f8eb9dfb700 1 -- 172.21.15.37:0/2085604264 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb41a5670 con 0x7f8eb4107460 2024-06-10T01:07:04.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.529+0000 7f8eb17fa700 1 -- 172.21.15.37:0/2085604264 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8eac01c020 con 0x7f8eb4107460 2024-06-10T01:07:04.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.530+0000 7f8eb17fa700 1 --2- 172.21.15.37:0/2085604264 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8ea00a2c20 0x7f8ea00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:04.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.530+0000 7f8eb17fa700 1 -- 172.21.15.37:0/2085604264 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f8eac0cec80 con 0x7f8eb4107460 2024-06-10T01:07:04.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.530+0000 7f8eb37fe700 1 --2- 172.21.15.37:0/2085604264 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8ea00a2c20 0x7f8ea00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:04.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.530+0000 7f8ec467eb80 1 Processor -- start 2024-06-10T01:07:04.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.531+0000 7f8ec467eb80 1 -- start start 2024-06-10T01:07:04.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.531+0000 7f8eb37fe700 1 --2- 172.21.15.37:0/2085604264 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8ea00a2c20 0x7f8ea00a50e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4006010 tx=0x7f8ea4005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:04.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.531+0000 7f8ec467eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3e91fba70 0x55a3e91fbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:04.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.531+0000 7f8ec467eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a3e91fc390 0x55a3e91f5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:04.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.531+0000 7f8ec467eb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a3e91f6330 0x55a3e91f6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:04.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.531+0000 7f8ec467eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a3e9143810 con 0x55a3e91fba70 2024-06-10T01:07:04.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.531+0000 7f8ec467eb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a3e90eb720 con 0x55a3e91f6330 2024-06-10T01:07:04.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.531+0000 7f8ec467eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a3e913f7d0 con 0x55a3e91fc390 2024-06-10T01:07:04.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.531+0000 7f8eb37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3e91fba70 0x55a3e91fbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:04.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.531+0000 7f8eb37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3e91fba70 0x55a3e91fbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46190/0 (socket says 172.21.15.37:46190) 2024-06-10T01:07:04.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.531+0000 7f8eb37fe700 1 -- 172.21.15.37:0/2546375672 learned_addr learned my addr 172.21.15.37:0/2546375672 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:04.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.531+0000 7f8eaaffd700 1 --2- 172.21.15.37:0/2546375672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a3e91fc390 0x55a3e91f5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:04.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.531+0000 7f8eb3fff700 1 --2- 172.21.15.37:0/2546375672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a3e91f6330 0x55a3e91f6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:04.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.531+0000 7f8eb37fe700 1 -- 172.21.15.37:0/2546375672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a3e91fc390 msgr2=0x55a3e91f5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:04.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.531+0000 7f8eb37fe700 1 --2- 172.21.15.37:0/2546375672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a3e91fc390 0x55a3e91f5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.531+0000 7f8eb37fe700 1 -- 172.21.15.37:0/2546375672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a3e91f6330 msgr2=0x55a3e91f6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:04.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.531+0000 7f8eb37fe700 1 --2- 172.21.15.37:0/2546375672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a3e91f6330 0x55a3e91f6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.531+0000 7f8eb37fe700 1 -- 172.21.15.37:0/2546375672 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3e90e1f50 con 0x55a3e91fba70 2024-06-10T01:07:04.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.531+0000 7f8eb3fff700 1 --2- 172.21.15.37:0/2546375672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a3e91f6330 0x55a3e91f6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:04.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.532+0000 7f8eb37fe700 1 --2- 172.21.15.37:0/2546375672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3e91fba70 0x55a3e91fbe50 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f8ea400bec0 tx=0x7f8ea4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f116484b1773101c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:04.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.532+0000 7f8ea9ffb700 1 -- 172.21.15.37:0/2546375672 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea400fc00 con 0x55a3e91fba70 2024-06-10T01:07:04.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.532+0000 7f8ea9ffb700 1 -- 172.21.15.37:0/2546375672 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ea400fda0 con 0x55a3e91fba70 2024-06-10T01:07:04.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.532+0000 7f8ea9ffb700 1 -- 172.21.15.37:0/2546375672 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea402f660 con 0x55a3e91fba70 2024-06-10T01:07:04.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.533+0000 7f8ec467eb80 1 -- 172.21.15.37:0/2546375672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3e91fba70 msgr2=0x55a3e91fbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:04.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.533+0000 7f8ec467eb80 1 --2- 172.21.15.37:0/2546375672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3e91fba70 0x55a3e91fbe50 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f8ea400bec0 tx=0x7f8ea4020000 comp rx=0 tx=0).stop 2024-06-10T01:07:04.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.533+0000 7f8ec467eb80 1 -- 172.21.15.37:0/2546375672 shutdown_connections 2024-06-10T01:07:04.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.533+0000 7f8ec467eb80 1 --2- 172.21.15.37:0/2546375672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a3e91f6330 0x55a3e91f6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.533+0000 7f8ec467eb80 1 --2- 172.21.15.37:0/2546375672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3e91fba70 0x55a3e91fbe50 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.533+0000 7f8ec467eb80 1 --2- 172.21.15.37:0/2546375672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a3e91fc390 0x55a3e91f5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.533+0000 7f8ec467eb80 1 -- 172.21.15.37:0/2546375672 >> 172.21.15.37:0/2546375672 conn(0x55a3e911ccc0 msgr2=0x55a3e91ec850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:04.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.533+0000 7f8ec467eb80 1 -- 172.21.15.37:0/2546375672 shutdown_connections 2024-06-10T01:07:04.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.533+0000 7f8ec467eb80 1 -- 172.21.15.37:0/2546375672 wait complete. 2024-06-10T01:07:04.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.533+0000 7f8ec467eb80 1 -- start start 2024-06-10T01:07:04.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.534+0000 7f8ec467eb80 1 Processor -- start 2024-06-10T01:07:04.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.534+0000 7f8ec467eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3e91f6330 0x55a3e9202110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:04.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.534+0000 7f8ec467eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a3e91fba70 0x55a3e91ff410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:04.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.534+0000 7f8ec467eb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a3e91fc390 0x55a3e91ff950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:04.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.534+0000 7f8eaaffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a3e91fba70 0x55a3e91ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:04.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.534+0000 7f8eaaffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a3e91fba70 0x55a3e91ff410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:33480/0 (socket says 172.21.15.37:33480) 2024-06-10T01:07:04.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.534+0000 7f8eaaffd700 1 -- 172.21.15.37:0/1154320394 learned_addr learned my addr 172.21.15.37:0/1154320394 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:04.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.534+0000 7f8eb37fe700 1 --2- 172.21.15.37:0/1154320394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3e91f6330 0x55a3e9202110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:04.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.534+0000 7f8ec467eb80 1 -- 172.21.15.37:0/1154320394 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a3e9202d10 con 0x55a3e91f6330 2024-06-10T01:07:04.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.534+0000 7f8ec467eb80 1 -- 172.21.15.37:0/1154320394 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a3e90e3bf0 con 0x55a3e91fc390 2024-06-10T01:07:04.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.534+0000 7f8ec467eb80 1 -- 172.21.15.37:0/1154320394 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a3e9202e90 con 0x55a3e91fba70 2024-06-10T01:07:04.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.534+0000 7f8ec467eb80 10 client.0 _ll_register_callbacks cb 0x55a3e90dd920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:04.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.535+0000 7f8eb3fff700 1 --2- 172.21.15.37:0/1154320394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a3e91fc390 0x55a3e91ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:04.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.535+0000 7f8eb37fe700 1 -- 172.21.15.37:0/1154320394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a3e91fba70 msgr2=0x55a3e91ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:04.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.535+0000 7f8eb37fe700 1 --2- 172.21.15.37:0/1154320394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a3e91fba70 0x55a3e91ff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.535+0000 7f8eb37fe700 1 -- 172.21.15.37:0/1154320394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a3e91fc390 msgr2=0x55a3e91ff950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:04.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.535+0000 7f8eb37fe700 1 --2- 172.21.15.37:0/1154320394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a3e91fc390 0x55a3e91ff950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.535+0000 7f8eb37fe700 1 -- 172.21.15.37:0/1154320394 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3e90928d0 con 0x55a3e91f6330 2024-06-10T01:07:04.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.535+0000 7f8eb3fff700 1 --2- 172.21.15.37:0/1154320394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a3e91fc390 0x55a3e91ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:04.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.535+0000 7f8eb37fe700 1 --2- 172.21.15.37:0/1154320394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3e91f6330 0x55a3e9202110 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8ea40098e0 tx=0x7f8ea402f700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a9d4d19f00092a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:04.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.535+0000 7f8e92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:07:04.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.535+0000 7f8e92ffd700 1 -- 172.21.15.37:0/1154320394 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea403b020 con 0x55a3e91f6330 2024-06-10T01:07:04.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.535+0000 7f8e92ffd700 1 -- 172.21.15.37:0/1154320394 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f8ea402fbb0 con 0x55a3e91f6330 2024-06-10T01:07:04.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.535+0000 7f8ec467eb80 1 -- 172.21.15.37:0/1154320394 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ea400fda0 con 0x55a3e91f6330 2024-06-10T01:07:04.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.536+0000 7f8e92ffd700 1 -- 172.21.15.37:0/1154320394 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8ea400fc00 con 0x55a3e91f6330 2024-06-10T01:07:04.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.536+0000 7f8ec467eb80 10 client.17139 fetch_fsmap learned FSMap version 31 2024-06-10T01:07:04.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.536+0000 7f8ec467eb80 1 -- 172.21.15.37:0/1154320394 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a3e920d590 con 0x55a3e91f6330 2024-06-10T01:07:04.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.536+0000 7f8e92ffd700 1 -- 172.21.15.37:0/1154320394 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ea4024860 con 0x55a3e91f6330 2024-06-10T01:07:04.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.536+0000 7f8e92ffd700 1 -- 172.21.15.37:0/1154320394 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f8ea40178d0 con 0x55a3e91f6330 2024-06-10T01:07:04.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.536+0000 7f8ec467eb80 10 client.17139 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:07:04.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.536+0000 7f8ec467eb80 10 client.17139 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:04.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.536+0000 7f8ec467eb80 20 client.17139 populate_metadata read hostname 'smithi037' 2024-06-10T01:07:04.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.536+0000 7f8ec467eb80 1 --2- 172.21.15.37:0/1154320394 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a3e920fba0 0x55a3e9211fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:04.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.536+0000 7f8ec467eb80 4 client.17139 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:04.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.536+0000 7f8ec467eb80 1 -- 172.21.15.37:0/1154320394 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8ea400fda0 con 0x55a3e920fba0 2024-06-10T01:07:04.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.537+0000 7f8eaaffd700 1 --2- 172.21.15.37:0/1154320394 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a3e920fba0 0x55a3e9211fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:04.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.537+0000 7f8e92ffd700 10 client.17139 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:04.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.537+0000 7f8eaaffd700 1 --2- 172.21.15.37:0/1154320394 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a3e920fba0 0x55a3e9211fc0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=499730c923ac472f server_cookie=749d6420bfbd9467 in_seq=0 out_seq=0 2024-06-10T01:07:04.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.539+0000 7f8e92ffd700 1 -- 172.21.15.37:0/1154320394 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f8ea400fda0 con 0x55a3e920fba0 2024-06-10T01:07:04.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.539+0000 7f8e92ffd700 10 client.17139 handle_command_reply: tid=0 2024-06-10T01:07:04.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.539+0000 7f8ec467eb80 1 client.17139 shutdown 2024-06-10T01:07:04.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.539+0000 7f8ec467eb80 1 -- 172.21.15.37:0/1154320394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3e91f6330 msgr2=0x55a3e9202110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:04.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.539+0000 7f8ec467eb80 1 --2- 172.21.15.37:0/1154320394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3e91f6330 0x55a3e9202110 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8ea40098e0 tx=0x7f8ea402f700 comp rx=0 tx=0).stop 2024-06-10T01:07:04.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.539+0000 7f8ec467eb80 1 -- 172.21.15.37:0/1154320394 shutdown_connections 2024-06-10T01:07:04.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.539+0000 7f8ec467eb80 1 --2- 172.21.15.37:0/1154320394 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a3e920fba0 0x55a3e9211fc0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.539+0000 7f8ec467eb80 1 --2- 172.21.15.37:0/1154320394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a3e91fc390 0x55a3e91ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.539+0000 7f8ec467eb80 1 --2- 172.21.15.37:0/1154320394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3e91f6330 0x55a3e9202110 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.539+0000 7f8ec467eb80 1 --2- 172.21.15.37:0/1154320394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a3e91fba70 0x55a3e91ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.539+0000 7f8ec467eb80 1 -- 172.21.15.37:0/1154320394 >> 172.21.15.37:0/1154320394 conn(0x55a3e911ccc0 msgr2=0x55a3e91f09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:04.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.539+0000 7f8e92ffd700 0 client.17139 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:04.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.539+0000 7f8ec467eb80 1 -- 172.21.15.37:0/1154320394 shutdown_connections 2024-06-10T01:07:04.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.539+0000 7f8ec467eb80 1 -- 172.21.15.37:0/1154320394 wait complete. 2024-06-10T01:07:04.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.540+0000 7f8ec467eb80 20 client.17139 trim_cache size 0 max 16384 2024-06-10T01:07:04.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.560+0000 7f8ec467eb80 1 Processor -- start 2024-06-10T01:07:04.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.560+0000 7f8ec467eb80 1 -- start start 2024-06-10T01:07:04.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.560+0000 7f8ec467eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a3e911ccc0 0x55a3e91fd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:04.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.560+0000 7f8ec467eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3e91fba70 0x55a3e9217890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:04.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.560+0000 7f8ec467eb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a3e91fc390 0x55a3e9217dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:04.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.560+0000 7f8ec467eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb41aea00 con 0x55a3e91fba70 2024-06-10T01:07:04.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.560+0000 7f8ec467eb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb41ac290 con 0x55a3e91fc390 2024-06-10T01:07:04.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.560+0000 7f8ec467eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb41aa0e0 con 0x55a3e911ccc0 2024-06-10T01:07:04.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.561+0000 7f8eaaffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3e91fba70 0x55a3e9217890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:04.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.561+0000 7f8eaaffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3e91fba70 0x55a3e9217890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46206/0 (socket says 172.21.15.37:46206) 2024-06-10T01:07:04.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.561+0000 7f8eb37fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a3e911ccc0 0x55a3e91fd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:04.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.561+0000 7f8eaaffd700 1 -- 172.21.15.37:0/4108155881 learned_addr learned my addr 172.21.15.37:0/4108155881 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:04.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.561+0000 7f8eb3fff700 1 --2- 172.21.15.37:0/4108155881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a3e91fc390 0x55a3e9217dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:04.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.561+0000 7f8eaaffd700 1 -- 172.21.15.37:0/4108155881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a3e911ccc0 msgr2=0x55a3e91fd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:04.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.561+0000 7f8eaaffd700 1 --2- 172.21.15.37:0/4108155881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a3e911ccc0 0x55a3e91fd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.561+0000 7f8eaaffd700 1 -- 172.21.15.37:0/4108155881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a3e91fc390 msgr2=0x55a3e9217dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:04.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.561+0000 7f8eaaffd700 1 --2- 172.21.15.37:0/4108155881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a3e91fc390 0x55a3e9217dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.561+0000 7f8eaaffd700 1 -- 172.21.15.37:0/4108155881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3e9200350 con 0x55a3e91fba70 2024-06-10T01:07:04.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.561+0000 7f8eb37fe700 1 --2- 172.21.15.37:0/4108155881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a3e911ccc0 0x55a3e91fd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:04.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.561+0000 7f8eaaffd700 1 --2- 172.21.15.37:0/4108155881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3e91fba70 0x55a3e9217890 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c005fd0 tx=0x7f8e9c013740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a68a001e41f61ab0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:04.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.561+0000 7f8ea97fa700 1 -- 172.21.15.37:0/4108155881 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c013c30 con 0x55a3e91fba70 2024-06-10T01:07:04.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.561+0000 7f8ea97fa700 1 -- 172.21.15.37:0/4108155881 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e9c004500 con 0x55a3e91fba70 2024-06-10T01:07:04.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.561+0000 7f8ea97fa700 1 -- 172.21.15.37:0/4108155881 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c011440 con 0x55a3e91fba70 2024-06-10T01:07:04.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.562+0000 7f8ec467eb80 1 -- 172.21.15.37:0/4108155881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3e91fba70 msgr2=0x55a3e9217890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:04.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.562+0000 7f8ec467eb80 1 --2- 172.21.15.37:0/4108155881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3e91fba70 0x55a3e9217890 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c005fd0 tx=0x7f8e9c013740 comp rx=0 tx=0).stop 2024-06-10T01:07:04.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.562+0000 7f8ec467eb80 1 -- 172.21.15.37:0/4108155881 shutdown_connections 2024-06-10T01:07:04.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.562+0000 7f8ec467eb80 1 --2- 172.21.15.37:0/4108155881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a3e91fc390 0x55a3e9217dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.562+0000 7f8ec467eb80 1 --2- 172.21.15.37:0/4108155881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a3e911ccc0 0x55a3e91fd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.562+0000 7f8ec467eb80 1 --2- 172.21.15.37:0/4108155881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3e91fba70 0x55a3e9217890 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.562+0000 7f8ec467eb80 1 -- 172.21.15.37:0/4108155881 >> 172.21.15.37:0/4108155881 conn(0x55a3e91f9ca0 msgr2=0x55a3e91f1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:04.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.562+0000 7f8ec467eb80 1 -- 172.21.15.37:0/4108155881 shutdown_connections 2024-06-10T01:07:04.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.562+0000 7f8ec467eb80 1 -- 172.21.15.37:0/4108155881 wait complete. 2024-06-10T01:07:04.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.562+0000 7f8ec467eb80 1 -- start start 2024-06-10T01:07:04.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.563+0000 7f8ec467eb80 1 Processor -- start 2024-06-10T01:07:04.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.563+0000 7f8ec467eb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a3e911ccc0 0x55a3e9217580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:04.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.563+0000 7f8ec467eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a3e91fba70 0x55a3e9217ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:04.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.563+0000 7f8ec467eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3e91fc390 0x55a3e9218000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:04.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.563+0000 7f8ec467eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb41aa750 con 0x55a3e91fc390 2024-06-10T01:07:04.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.563+0000 7f8ec467eb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb41abbc0 con 0x55a3e911ccc0 2024-06-10T01:07:04.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.563+0000 7f8ec467eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb41aae20 con 0x55a3e91fba70 2024-06-10T01:07:04.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.563+0000 7f8ec467eb80 10 client.0 _ll_register_callbacks cb 0x55a3e90dd920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:04.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.563+0000 7f8eb3fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3e91fc390 0x55a3e9218000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:04.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.563+0000 7f8eaaffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a3e91fba70 0x55a3e9217ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:04.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.563+0000 7f8eb37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a3e911ccc0 0x55a3e9217580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:04.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.563+0000 7f8eb3fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3e91fc390 0x55a3e9218000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46210/0 (socket says 172.21.15.37:46210) 2024-06-10T01:07:04.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.563+0000 7f8eb3fff700 1 -- 172.21.15.37:0/3916171829 learned_addr learned my addr 172.21.15.37:0/3916171829 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:04.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.564+0000 7f8eb3fff700 1 -- 172.21.15.37:0/3916171829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a3e91fba70 msgr2=0x55a3e9217ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:04.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.564+0000 7f8eb3fff700 1 --2- 172.21.15.37:0/3916171829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a3e91fba70 0x55a3e9217ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.564+0000 7f8eb3fff700 1 -- 172.21.15.37:0/3916171829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a3e911ccc0 msgr2=0x55a3e9217580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:04.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.564+0000 7f8eb3fff700 1 --2- 172.21.15.37:0/3916171829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a3e911ccc0 0x55a3e9217580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.564+0000 7f8eb3fff700 1 -- 172.21.15.37:0/3916171829 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3e9279960 con 0x55a3e91fc390 2024-06-10T01:07:04.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.564+0000 7f8eaaffd700 1 --2- 172.21.15.37:0/3916171829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a3e91fba70 0x55a3e9217ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:04.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.564+0000 7f8eb3fff700 1 --2- 172.21.15.37:0/3916171829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3e91fc390 0x55a3e9218000 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f8eac009440 tx=0x7f8eac09a9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f076fc8c65c99c5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:04.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.564+0000 7f8e92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:07:04.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.564+0000 7f8e92ffd700 1 -- 172.21.15.37:0/3916171829 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac019040 con 0x55a3e91fc390 2024-06-10T01:07:04.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.564+0000 7f8e92ffd700 1 -- 172.21.15.37:0/3916171829 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f8eac0bb0b0 con 0x55a3e91fc390 2024-06-10T01:07:04.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.564+0000 7f8ec467eb80 1 -- 172.21.15.37:0/3916171829 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e9c004500 con 0x55a3e91fc390 2024-06-10T01:07:04.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.564+0000 7f8e92ffd700 1 -- 172.21.15.37:0/3916171829 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8eac09b090 con 0x55a3e91fc390 2024-06-10T01:07:04.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.564+0000 7f8ec467eb80 10 client.17145 fetch_fsmap learned FSMap version 31 2024-06-10T01:07:04.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.564+0000 7f8ec467eb80 1 -- 172.21.15.37:0/3916171829 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a3e927ac50 con 0x55a3e91fc390 2024-06-10T01:07:04.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.564+0000 7f8e92ffd700 1 -- 172.21.15.37:0/3916171829 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e9c004500 con 0x55a3e91fc390 2024-06-10T01:07:04.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.565+0000 7f8e92ffd700 1 -- 172.21.15.37:0/3916171829 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f8eac0a3870 con 0x55a3e91fc390 2024-06-10T01:07:04.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.565+0000 7f8ec467eb80 10 client.17145 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:07:04.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.565+0000 7f8ec467eb80 10 client.17145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:04.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.565+0000 7f8ec467eb80 20 client.17145 populate_metadata read hostname 'smithi037' 2024-06-10T01:07:04.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.565+0000 7f8ec467eb80 1 --2- 172.21.15.37:0/3916171829 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a3e927b380 0x55a3e927d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:04.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.565+0000 7f8ec467eb80 4 client.17145 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:07:04.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.565+0000 7f8ec467eb80 1 -- 172.21.15.37:0/3916171829 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a3e9150d10 con 0x55a3e927b380 2024-06-10T01:07:04.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.565+0000 7f8eb37fe700 1 --2- 172.21.15.37:0/3916171829 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a3e927b380 0x55a3e927d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:04.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.565+0000 7f8eb37fe700 1 --2- 172.21.15.37:0/3916171829 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a3e927b380 0x55a3e927d7a0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1f649b178eaec23 server_cookie=d62f9daa47753efa in_seq=0 out_seq=0 2024-06-10T01:07:04.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.565+0000 7f8e92ffd700 10 client.17145 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:04.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.568+0000 7f8e92ffd700 1 -- 172.21.15.37:0/3916171829 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a3e9150d10 con 0x55a3e927b380 2024-06-10T01:07:04.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.568+0000 7f8e92ffd700 10 client.17145 handle_command_reply: tid=0 2024-06-10T01:07:04.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.568+0000 7f8ec467eb80 1 client.17145 shutdown 2024-06-10T01:07:04.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.568+0000 7f8ec467eb80 1 -- 172.21.15.37:0/3916171829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3e91fc390 msgr2=0x55a3e9218000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:04.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.568+0000 7f8ec467eb80 1 --2- 172.21.15.37:0/3916171829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3e91fc390 0x55a3e9218000 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f8eac009440 tx=0x7f8eac09a9f0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.569+0000 7f8ec467eb80 1 -- 172.21.15.37:0/3916171829 shutdown_connections 2024-06-10T01:07:04.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.569+0000 7f8ec467eb80 1 --2- 172.21.15.37:0/3916171829 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a3e927b380 0x55a3e927d7a0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.569+0000 7f8ec467eb80 1 --2- 172.21.15.37:0/3916171829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a3e91fc390 0x55a3e9218000 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.569+0000 7f8ec467eb80 1 --2- 172.21.15.37:0/3916171829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a3e911ccc0 0x55a3e9217580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.569+0000 7f8ec467eb80 1 --2- 172.21.15.37:0/3916171829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a3e91fba70 0x55a3e9217ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.569+0000 7f8e92ffd700 0 client.17145 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:04.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.569+0000 7f8ec467eb80 1 -- 172.21.15.37:0/3916171829 >> 172.21.15.37:0/3916171829 conn(0x55a3e91f9ca0 msgr2=0x55a3e91e5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:04.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.569+0000 7f8ec467eb80 1 -- 172.21.15.37:0/3916171829 shutdown_connections 2024-06-10T01:07:04.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.569+0000 7f8ec467eb80 1 -- 172.21.15.37:0/3916171829 wait complete. 2024-06-10T01:07:04.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.569+0000 7f8ec467eb80 20 client.17145 trim_cache size 0 max 16384 2024-06-10T01:07:04.567 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:07:04.567 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-06-10T01:07:04.568 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-06-10T01:07:04.568 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:07:04.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.569+0000 7f8eb9dfb700 1 -- 172.21.15.37:0/2085604264 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8ea00a2c20 msgr2=0x7f8ea00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:04.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.569+0000 7f8eb9dfb700 1 --2- 172.21.15.37:0/2085604264 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8ea00a2c20 0x7f8ea00a50e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4006010 tx=0x7f8ea4005960 comp rx=0 tx=0).stop 2024-06-10T01:07:04.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.569+0000 7f8eb9dfb700 1 -- 172.21.15.37:0/2085604264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8eb4107460 msgr2=0x7f8eb4198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:04.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.569+0000 7f8eb9dfb700 1 --2- 172.21.15.37:0/2085604264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8eb4107460 0x7f8eb4198960 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8eac007f10 tx=0x7f8eac00d3b0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.569+0000 7f8eb9dfb700 1 -- 172.21.15.37:0/2085604264 shutdown_connections 2024-06-10T01:07:04.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.569+0000 7f8eb9dfb700 1 --2- 172.21.15.37:0/2085604264 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8ea00a2c20 0x7f8ea00a50e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.569+0000 7f8eb9dfb700 1 --2- 172.21.15.37:0/2085604264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8eb4107460 0x7f8eb4198960 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.569+0000 7f8eb9dfb700 1 --2- 172.21.15.37:0/2085604264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8eb406ab60 0x7f8eb419e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.569+0000 7f8eb9dfb700 1 --2- 172.21.15.37:0/2085604264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8eb406b520 0x7f8eb419ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:04.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.569+0000 7f8eb9dfb700 1 -- 172.21.15.37:0/2085604264 >> 172.21.15.37:0/2085604264 conn(0x7f8eb4078340 msgr2=0x7f8eb40ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:04.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.569+0000 7f8eb9dfb700 1 -- 172.21.15.37:0/2085604264 shutdown_connections 2024-06-10T01:07:04.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:04.570+0000 7f8eb9dfb700 1 -- 172.21.15.37:0/2085604264 wait complete. 2024-06-10T01:07:04.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:04 smithi167 ceph-mon[36917]: pgmap v742: 129 pgs: 129 active+clean; 6.6 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 38 KiB/s wr, 2.41k op/s 2024-06-10T01:07:05.260 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:07:05.261 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:07:05.518 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:06.360 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:06 smithi094 ceph-mon[36708]: pgmap v743: 129 pgs: 129 active+clean; 6.9 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 64 KiB/s wr, 2.12k op/s 2024-06-10T01:07:06.361 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:06 smithi037 ceph-mon[32192]: pgmap v743: 129 pgs: 129 active+clean; 6.9 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 64 KiB/s wr, 2.12k op/s 2024-06-10T01:07:06.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:06 smithi167 ceph-mon[36917]: pgmap v743: 129 pgs: 129 active+clean; 6.9 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 64 KiB/s wr, 2.12k op/s 2024-06-10T01:07:07.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.390+0000 7f4571b73700 1 -- 172.21.15.37:0/3822304140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f456c10dee0 msgr2=0x7f456c10e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:07.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.390+0000 7f4571b73700 1 --2- 172.21.15.37:0/3822304140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f456c10dee0 0x7f456c10e2c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f455c00b600 tx=0x7f455c00b910 comp rx=0 tx=0).stop 2024-06-10T01:07:07.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.390+0000 7f4571b73700 1 -- 172.21.15.37:0/3822304140 shutdown_connections 2024-06-10T01:07:07.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.390+0000 7f4571b73700 1 --2- 172.21.15.37:0/3822304140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f456c109e50 0x7f456c115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.390+0000 7f4571b73700 1 --2- 172.21.15.37:0/3822304140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f456c10dee0 0x7f456c10e2c0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.390+0000 7f4571b73700 1 --2- 172.21.15.37:0/3822304140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f456c109490 0x7f456c109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.390+0000 7f4571b73700 1 -- 172.21.15.37:0/3822304140 >> 172.21.15.37:0/3822304140 conn(0x7f456c06d9e0 msgr2=0x7f456c06ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:07.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.391+0000 7f4571b73700 1 -- 172.21.15.37:0/3822304140 shutdown_connections 2024-06-10T01:07:07.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.391+0000 7f4571b73700 1 -- 172.21.15.37:0/3822304140 wait complete. 2024-06-10T01:07:07.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.392+0000 7f4571b73700 1 Processor -- start 2024-06-10T01:07:07.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.393+0000 7f4571b73700 1 -- start start 2024-06-10T01:07:07.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.394+0000 7f4571b73700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f456c109490 0x7f456c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:07.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.394+0000 7f4571b73700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f456c109e50 0x7f456c19f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:07.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.394+0000 7f4571b73700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f456c10dee0 0x7f456c1a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:07.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.394+0000 7f4571b73700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f456c118e10 con 0x7f456c10dee0 2024-06-10T01:07:07.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.394+0000 7f4571b73700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f456c118c90 con 0x7f456c109490 2024-06-10T01:07:07.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.394+0000 7f456bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f456c10dee0 0x7f456c1a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:07.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.394+0000 7f456bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f456c10dee0 0x7f456c1a3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46234/0 (socket says 172.21.15.37:46234) 2024-06-10T01:07:07.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.394+0000 7f456bfff700 1 -- 172.21.15.37:0/450301782 learned_addr learned my addr 172.21.15.37:0/450301782 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:07.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.394+0000 7f456b7fe700 1 --2- 172.21.15.37:0/450301782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f456c109490 0x7f456c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:07.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.395+0000 7f4571b73700 1 -- 172.21.15.37:0/450301782 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f456c118f90 con 0x7f456c109e50 2024-06-10T01:07:07.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.395+0000 7f456affd700 1 --2- 172.21.15.37:0/450301782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f456c109e50 0x7f456c19f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:07.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.395+0000 7f456bfff700 1 -- 172.21.15.37:0/450301782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f456c109e50 msgr2=0x7f456c19f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:07.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.395+0000 7f456bfff700 1 --2- 172.21.15.37:0/450301782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f456c109e50 0x7f456c19f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.395+0000 7f456bfff700 1 -- 172.21.15.37:0/450301782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f456c109490 msgr2=0x7f456c19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:07.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.395+0000 7f456bfff700 1 --2- 172.21.15.37:0/450301782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f456c109490 0x7f456c19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.395+0000 7f456bfff700 1 -- 172.21.15.37:0/450301782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456c1a3f90 con 0x7f456c10dee0 2024-06-10T01:07:07.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.396+0000 7f456bfff700 1 --2- 172.21.15.37:0/450301782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f456c10dee0 0x7f456c1a3850 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f4564007d60 tx=0x7f456400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:07.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.397+0000 7f4568ff9700 1 -- 172.21.15.37:0/450301782 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456400ac90 con 0x7f456c10dee0 2024-06-10T01:07:07.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.397+0000 7f4571b73700 1 -- 172.21.15.37:0/450301782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f456c1a4280 con 0x7f456c10dee0 2024-06-10T01:07:07.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.397+0000 7f4571b73700 1 -- 172.21.15.37:0/450301782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f456c1abb00 con 0x7f456c10dee0 2024-06-10T01:07:07.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.398+0000 7f4568ff9700 1 -- 172.21.15.37:0/450301782 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f456400ae30 con 0x7f456c10dee0 2024-06-10T01:07:07.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.398+0000 7f4568ff9700 1 -- 172.21.15.37:0/450301782 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564012420 con 0x7f456c10dee0 2024-06-10T01:07:07.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.399+0000 7f4571b73700 1 -- 172.21.15.37:0/450301782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f456c069180 con 0x7f456c10dee0 2024-06-10T01:07:07.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.402+0000 7f4568ff9700 1 -- 172.21.15.37:0/450301782 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4564003a00 con 0x7f456c10dee0 2024-06-10T01:07:07.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.403+0000 7f4568ff9700 1 --2- 172.21.15.37:0/450301782 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f45540a2d80 0x7f45540a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:07.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.403+0000 7f4568ff9700 1 -- 172.21.15.37:0/450301782 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f45640cfca0 con 0x7f456c10dee0 2024-06-10T01:07:07.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.404+0000 7f456b7fe700 1 --2- 172.21.15.37:0/450301782 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f45540a2d80 0x7f45540a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:07.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.407+0000 7f456b7fe700 1 --2- 172.21.15.37:0/450301782 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f45540a2d80 0x7f45540a5240 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f455c00b3c0 tx=0x7f455c005e80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:07.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.413+0000 7f4568ff9700 1 -- 172.21.15.37:0/450301782 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f456409c890 con 0x7f456c10dee0 2024-06-10T01:07:07.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.828+0000 7f4571b73700 1 -- 172.21.15.37:0/450301782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f456c1a02e0 con 0x7f456c10dee0 2024-06-10T01:07:07.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.829+0000 7f4568ff9700 1 -- 172.21.15.37:0/450301782 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11868 (secure 0 0 0) 0x7f456409c190 con 0x7f456c10dee0 2024-06-10T01:07:07.825 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:07:07.825 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":26554,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2107277223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2107277223},{"type":"v1","addr":"172.21.15.167:6835","nonce":2107277223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T01:06:09.604504+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24472,24451,24458]},"id":1}]} 2024-06-10T01:07:07.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.831+0000 7f4571b73700 1 -- 172.21.15.37:0/450301782 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f45540a2d80 msgr2=0x7f45540a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:07.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.831+0000 7f4571b73700 1 --2- 172.21.15.37:0/450301782 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f45540a2d80 0x7f45540a5240 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f455c00b3c0 tx=0x7f455c005e80 comp rx=0 tx=0).stop 2024-06-10T01:07:07.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.831+0000 7f4571b73700 1 -- 172.21.15.37:0/450301782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f456c10dee0 msgr2=0x7f456c1a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:07.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.831+0000 7f4571b73700 1 --2- 172.21.15.37:0/450301782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f456c10dee0 0x7f456c1a3850 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f4564007d60 tx=0x7f456400a340 comp rx=0 tx=0).stop 2024-06-10T01:07:07.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.831+0000 7f4571b73700 1 -- 172.21.15.37:0/450301782 shutdown_connections 2024-06-10T01:07:07.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.831+0000 7f4571b73700 1 --2- 172.21.15.37:0/450301782 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f45540a2d80 0x7f45540a5240 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.831+0000 7f4571b73700 1 --2- 172.21.15.37:0/450301782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f456c10dee0 0x7f456c1a3850 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.831+0000 7f4571b73700 1 --2- 172.21.15.37:0/450301782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f456c109490 0x7f456c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.831+0000 7f4571b73700 1 --2- 172.21.15.37:0/450301782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f456c109e50 0x7f456c19f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:07.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.831+0000 7f4571b73700 1 -- 172.21.15.37:0/450301782 >> 172.21.15.37:0/450301782 conn(0x7f456c06d9e0 msgr2=0x7f456c071ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:07.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.831+0000 7f4571b73700 1 -- 172.21.15.37:0/450301782 shutdown_connections 2024-06-10T01:07:07.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:07.831+0000 7f4571b73700 1 -- 172.21.15.37:0/450301782 wait complete. 2024-06-10T01:07:07.828 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 31 2024-06-10T01:07:08.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:08 smithi167 ceph-mon[36917]: pgmap v744: 129 pgs: 129 active+clean; 6.9 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 41 KiB/s wr, 1.06k op/s 2024-06-10T01:07:08.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:08 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/450301782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:07:08.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:08 smithi037 ceph-mon[32192]: pgmap v744: 129 pgs: 129 active+clean; 6.9 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 41 KiB/s wr, 1.06k op/s 2024-06-10T01:07:08.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:08 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/450301782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:07:08.523 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:07:08.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:08 smithi094 ceph-mon[36708]: pgmap v744: 129 pgs: 129 active+clean; 6.9 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 41 KiB/s wr, 1.06k op/s 2024-06-10T01:07:08.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:08 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/450301782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:07:08.780 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:10.304 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:10 smithi037 ceph-mon[32192]: pgmap v745: 129 pgs: 129 active+clean; 11 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 414 KiB/s wr, 1.07k op/s 2024-06-10T01:07:10.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:10 smithi167 ceph-mon[36917]: pgmap v745: 129 pgs: 129 active+clean; 11 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 414 KiB/s wr, 1.07k op/s 2024-06-10T01:07:10.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.576+0000 7f5db2834700 1 -- 172.21.15.37:0/3222857422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dac10ef40 msgr2=0x7f5dac111330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:10.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.576+0000 7f5db2834700 1 --2- 172.21.15.37:0/3222857422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dac10ef40 0x7f5dac111330 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5da400b0a0 tx=0x7f5da400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.576+0000 7f5db2834700 1 -- 172.21.15.37:0/3222857422 shutdown_connections 2024-06-10T01:07:10.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.576+0000 7f5db2834700 1 --2- 172.21.15.37:0/3222857422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dac10ef40 0x7f5dac111330 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.576+0000 7f5db2834700 1 --2- 172.21.15.37:0/3222857422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dac06a1a0 0x7f5dac06a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.576+0000 7f5db2834700 1 --2- 172.21.15.37:0/3222857422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dac06aac0 0x7f5dac10ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.576+0000 7f5db2834700 1 -- 172.21.15.37:0/3222857422 >> 172.21.15.37:0/3222857422 conn(0x7f5dac078340 msgr2=0x7f5dac078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:10.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.576+0000 7f5db2834700 1 -- 172.21.15.37:0/3222857422 shutdown_connections 2024-06-10T01:07:10.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.576+0000 7f5db2834700 1 -- 172.21.15.37:0/3222857422 wait complete. 2024-06-10T01:07:10.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.577+0000 7f5db2834700 1 Processor -- start 2024-06-10T01:07:10.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.577+0000 7f5db2834700 1 -- start start 2024-06-10T01:07:10.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.577+0000 7f5db2834700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dac06a1a0 0x7f5dac19aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:10.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.577+0000 7f5db2834700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dac06aac0 0x7f5dac19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:10.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.577+0000 7f5db2834700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dac10ef40 0x7f5dac19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:10.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.577+0000 7f5db2834700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac114a00 con 0x7f5dac06a1a0 2024-06-10T01:07:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.577+0000 7f5db2834700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac114880 con 0x7f5dac10ef40 2024-06-10T01:07:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.577+0000 7f5db2834700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac114b80 con 0x7f5dac06aac0 2024-06-10T01:07:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.577+0000 7f5dabfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dac06a1a0 0x7f5dac19aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.577+0000 7f5dabfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dac06a1a0 0x7f5dac19aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46242/0 (socket says 172.21.15.37:46242) 2024-06-10T01:07:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.577+0000 7f5dab7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dac06aac0 0x7f5dac19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.578+0000 7f5dabfff700 1 -- 172.21.15.37:0/3686247773 learned_addr learned my addr 172.21.15.37:0/3686247773 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.578+0000 7f5db0dd1700 1 --2- 172.21.15.37:0/3686247773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dac10ef40 0x7f5dac19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.578+0000 7f5dabfff700 1 -- 172.21.15.37:0/3686247773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dac06aac0 msgr2=0x7f5dac19b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.578+0000 7f5dabfff700 1 --2- 172.21.15.37:0/3686247773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dac06aac0 0x7f5dac19b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.578+0000 7f5dabfff700 1 -- 172.21.15.37:0/3686247773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dac10ef40 msgr2=0x7f5dac19f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.578+0000 7f5dabfff700 1 --2- 172.21.15.37:0/3686247773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dac10ef40 0x7f5dac19f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.578+0000 7f5dabfff700 1 -- 172.21.15.37:0/3686247773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dac19feb0 con 0x7f5dac06a1a0 2024-06-10T01:07:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.578+0000 7f5dabfff700 1 --2- 172.21.15.37:0/3686247773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dac06a1a0 0x7f5dac19aef0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00e9c0 tx=0x7f5d9c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.578+0000 7f5da97fa700 1 -- 172.21.15.37:0/3686247773 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c00cbd0 con 0x7f5dac06a1a0 2024-06-10T01:07:10.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.578+0000 7f5db2834700 1 -- 172.21.15.37:0/3686247773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dac1a01a0 con 0x7f5dac06a1a0 2024-06-10T01:07:10.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.578+0000 7f5db2834700 1 -- 172.21.15.37:0/3686247773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dac1a7a20 con 0x7f5dac06a1a0 2024-06-10T01:07:10.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.578+0000 7f5da97fa700 1 -- 172.21.15.37:0/3686247773 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d9c00ee60 con 0x7f5dac06a1a0 2024-06-10T01:07:10.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.578+0000 7f5da97fa700 1 -- 172.21.15.37:0/3686247773 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c005610 con 0x7f5dac06a1a0 2024-06-10T01:07:10.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.581+0000 7f5da97fa700 1 -- 172.21.15.37:0/3686247773 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5d9c021440 con 0x7f5dac06a1a0 2024-06-10T01:07:10.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.581+0000 7f5da97fa700 1 --2- 172.21.15.37:0/3686247773 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5d940a2ca0 0x7f5d940a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:10.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.581+0000 7f5da97fa700 1 -- 172.21.15.37:0/3686247773 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f5d9c0cf510 con 0x7f5dac06a1a0 2024-06-10T01:07:10.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.581+0000 7f5dab7fe700 1 --2- 172.21.15.37:0/3686247773 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5d940a2ca0 0x7f5d940a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:10.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.582+0000 7f5dab7fe700 1 --2- 172.21.15.37:0/3686247773 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5d940a2ca0 0x7f5d940a5160 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5dac1027e0 tx=0x7f5da000a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:10.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.582+0000 7f5dbd0b7b80 1 Processor -- start 2024-06-10T01:07:10.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.582+0000 7f5dbd0b7b80 1 -- start start 2024-06-10T01:07:10.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.582+0000 7f5dbd0b7b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594a40c9f60 0x5594a40ca340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:10.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.582+0000 7f5dbd0b7b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594a40ca880 0x5594a40c4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:10.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.582+0000 7f5dbd0b7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594a40c47a0 0x5594a40c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:10.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.582+0000 7f5dbd0b7b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5594a3fb6f60 con 0x5594a40c9f60 2024-06-10T01:07:10.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.582+0000 7f5dbd0b7b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5594a4015850 con 0x5594a40ca880 2024-06-10T01:07:10.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.582+0000 7f5dbd0b7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5594a3f7d460 con 0x5594a40c47a0 2024-06-10T01:07:10.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.583+0000 7f5dabfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594a40c9f60 0x5594a40ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:10.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.583+0000 7f5dabfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594a40c9f60 0x5594a40ca340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46246/0 (socket says 172.21.15.37:46246) 2024-06-10T01:07:10.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.583+0000 7f5dabfff700 1 -- 172.21.15.37:0/788374964 learned_addr learned my addr 172.21.15.37:0/788374964 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:10.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.583+0000 7f5db0dd1700 1 --2- 172.21.15.37:0/788374964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594a40c47a0 0x5594a40c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.583+0000 7f5dab7fe700 1 --2- 172.21.15.37:0/788374964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594a40ca880 0x5594a40c4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.583+0000 7f5dabfff700 1 -- 172.21.15.37:0/788374964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594a40c47a0 msgr2=0x5594a40c4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.583+0000 7f5dabfff700 1 --2- 172.21.15.37:0/788374964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594a40c47a0 0x5594a40c4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.583+0000 7f5dabfff700 1 -- 172.21.15.37:0/788374964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594a40ca880 msgr2=0x5594a40c4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.583+0000 7f5dabfff700 1 --2- 172.21.15.37:0/788374964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594a40ca880 0x5594a40c4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.583+0000 7f5dabfff700 1 -- 172.21.15.37:0/788374964 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594a3f648d0 con 0x5594a40c9f60 2024-06-10T01:07:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.583+0000 7f5dabfff700 1 --2- 172.21.15.37:0/788374964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594a40c9f60 0x5594a40ca340 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c09b680 tx=0x7f5d9c09b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270839457dafc9ed server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.583+0000 7f5d927fc700 1 -- 172.21.15.37:0/788374964 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c09fd30 con 0x5594a40c9f60 2024-06-10T01:07:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.583+0000 7f5d927fc700 1 -- 172.21.15.37:0/788374964 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d9c0b4070 con 0x5594a40c9f60 2024-06-10T01:07:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.583+0000 7f5d927fc700 1 -- 172.21.15.37:0/788374964 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c09bbe0 con 0x5594a40c9f60 2024-06-10T01:07:10.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.584+0000 7f5dbd0b7b80 1 -- 172.21.15.37:0/788374964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594a40c9f60 msgr2=0x5594a40ca340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:10.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.584+0000 7f5dbd0b7b80 1 --2- 172.21.15.37:0/788374964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594a40c9f60 0x5594a40ca340 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c09b680 tx=0x7f5d9c09b3b0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.584+0000 7f5dbd0b7b80 1 -- 172.21.15.37:0/788374964 shutdown_connections 2024-06-10T01:07:10.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.584+0000 7f5dbd0b7b80 1 --2- 172.21.15.37:0/788374964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594a40c47a0 0x5594a40c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.584+0000 7f5dbd0b7b80 1 --2- 172.21.15.37:0/788374964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594a40c9f60 0x5594a40ca340 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.584+0000 7f5dbd0b7b80 1 --2- 172.21.15.37:0/788374964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594a40ca880 0x5594a40c4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.584+0000 7f5dbd0b7b80 1 -- 172.21.15.37:0/788374964 >> 172.21.15.37:0/788374964 conn(0x5594a3ff1080 msgr2=0x5594a40bf220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:10.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.584+0000 7f5dbd0b7b80 1 -- 172.21.15.37:0/788374964 shutdown_connections 2024-06-10T01:07:10.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.584+0000 7f5dbd0b7b80 1 -- 172.21.15.37:0/788374964 wait complete. 2024-06-10T01:07:10.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.585+0000 7f5dbd0b7b80 1 -- start start 2024-06-10T01:07:10.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.585+0000 7f5dbd0b7b80 1 Processor -- start 2024-06-10T01:07:10.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.585+0000 7f5dbd0b7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594a40c47a0 0x5594a40d0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:10.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.585+0000 7f5dbd0b7b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594a40c9f60 0x5594a40cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:10.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.585+0000 7f5dbd0b7b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594a40ca880 0x5594a40cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:10.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.585+0000 7f5dbd0b7b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5594a3fb5fc0 con 0x5594a40ca880 2024-06-10T01:07:10.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.585+0000 7f5dbd0b7b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5594a4011810 con 0x5594a40c9f60 2024-06-10T01:07:10.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.585+0000 7f5dbd0b7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5594a40d1200 con 0x5594a40c47a0 2024-06-10T01:07:10.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.585+0000 7f5dbd0b7b80 10 client.0 _ll_register_callbacks cb 0x5594a3fab5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:10.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.585+0000 7f5db0dd1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594a40ca880 0x5594a40cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:10.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.585+0000 7f5db0dd1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594a40ca880 0x5594a40cde70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46260/0 (socket says 172.21.15.37:46260) 2024-06-10T01:07:10.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.585+0000 7f5db0dd1700 1 -- 172.21.15.37:0/2898325310 learned_addr learned my addr 172.21.15.37:0/2898325310 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:10.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.585+0000 7f5dabfff700 1 --2- 172.21.15.37:0/2898325310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594a40c47a0 0x5594a40d0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:10.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.585+0000 7f5dab7fe700 1 --2- 172.21.15.37:0/2898325310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594a40c9f60 0x5594a40cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:10.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.586+0000 7f5db0dd1700 1 -- 172.21.15.37:0/2898325310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594a40c47a0 msgr2=0x5594a40d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:10.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.586+0000 7f5db0dd1700 1 --2- 172.21.15.37:0/2898325310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594a40c47a0 0x5594a40d0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.586+0000 7f5db0dd1700 1 -- 172.21.15.37:0/2898325310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594a40c9f60 msgr2=0x5594a40cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:10.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.586+0000 7f5db0dd1700 1 --2- 172.21.15.37:0/2898325310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594a40c9f60 0x5594a40cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.586+0000 7f5db0dd1700 1 -- 172.21.15.37:0/2898325310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594a3fdb750 con 0x5594a40ca880 2024-06-10T01:07:10.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.586+0000 7f5db0dd1700 1 --2- 172.21.15.37:0/2898325310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594a40ca880 0x5594a40cde70 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f5da400b0a0 tx=0x7f5da4004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cdc65393f821eca server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:10.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.586+0000 7f5d8b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:07:10.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.586+0000 7f5d8b7fe700 1 -- 172.21.15.37:0/2898325310 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da40076e0 con 0x5594a40ca880 2024-06-10T01:07:10.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.586+0000 7f5d8b7fe700 1 -- 172.21.15.37:0/2898325310 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f5da4007880 con 0x5594a40ca880 2024-06-10T01:07:10.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.586+0000 7f5dbd0b7b80 1 -- 172.21.15.37:0/2898325310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d9c0b4070 con 0x5594a40ca880 2024-06-10T01:07:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.586+0000 7f5d8b7fe700 1 -- 172.21.15.37:0/2898325310 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5da4004e40 con 0x5594a40ca880 2024-06-10T01:07:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.586+0000 7f5dbd0b7b80 10 client.17163 fetch_fsmap learned FSMap version 31 2024-06-10T01:07:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.586+0000 7f5dbd0b7b80 1 -- 172.21.15.37:0/2898325310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5594a40dba60 con 0x5594a40ca880 2024-06-10T01:07:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.586+0000 7f5d8b7fe700 1 -- 172.21.15.37:0/2898325310 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d9c0b4070 con 0x5594a40ca880 2024-06-10T01:07:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.587+0000 7f5d8b7fe700 1 -- 172.21.15.37:0/2898325310 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f5da4011070 con 0x5594a40ca880 2024-06-10T01:07:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.587+0000 7f5dbd0b7b80 10 client.17163 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:07:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.587+0000 7f5dbd0b7b80 10 client.17163 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-10T01:07:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.587+0000 7f5dbd0b7b80 20 client.17163 populate_metadata read hostname 'smithi037' 2024-06-10T01:07:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.587+0000 7f5dbd0b7b80 1 --2- 172.21.15.37:0/2898325310 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5594a40de0f0 0x5594a40e0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.587+0000 7f5dbd0b7b80 4 client.17163 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.587+0000 7f5dbd0b7b80 1 -- 172.21.15.37:0/2898325310 --> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594a4022d50 con 0x5594a40de0f0 2024-06-10T01:07:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.587+0000 7f5dabfff700 1 --2- 172.21.15.37:0/2898325310 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5594a40de0f0 0x5594a40e0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.587+0000 7f5dabfff700 1 --2- 172.21.15.37:0/2898325310 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5594a40de0f0 0x5594a40e0500 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9bc2cb09b438828 server_cookie=e4d38be565c05c0c in_seq=0 out_seq=0 2024-06-10T01:07:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.587+0000 7f5d8b7fe700 10 client.17163 ms_handle_connect on v2:172.21.15.167:6836/2720148065 2024-06-10T01:07:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.588+0000 7f5d8b7fe700 1 -- 172.21.15.37:0/2898325310 <== mds.2 v2:172.21.15.167:6836/2720148065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5594a4022d50 con 0x5594a40de0f0 2024-06-10T01:07:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.588+0000 7f5d8b7fe700 10 client.17163 handle_command_reply: tid=0 2024-06-10T01:07:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.588+0000 7f5dbd0b7b80 1 client.17163 shutdown 2024-06-10T01:07:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.588+0000 7f5dbd0b7b80 1 -- 172.21.15.37:0/2898325310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594a40ca880 msgr2=0x5594a40cde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.588+0000 7f5dbd0b7b80 1 --2- 172.21.15.37:0/2898325310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594a40ca880 0x5594a40cde70 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f5da400b0a0 tx=0x7f5da4004a10 comp rx=0 tx=0).stop 2024-06-10T01:07:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.588+0000 7f5dbd0b7b80 1 -- 172.21.15.37:0/2898325310 shutdown_connections 2024-06-10T01:07:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.588+0000 7f5dbd0b7b80 1 --2- 172.21.15.37:0/2898325310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594a40ca880 0x5594a40cde70 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.588+0000 7f5dbd0b7b80 1 --2- 172.21.15.37:0/2898325310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594a40c47a0 0x5594a40d0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.588+0000 7f5dbd0b7b80 1 --2- 172.21.15.37:0/2898325310 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5594a40de0f0 0x5594a40e0500 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.588+0000 7f5dbd0b7b80 1 --2- 172.21.15.37:0/2898325310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594a40c9f60 0x5594a40cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.588+0000 7f5dbd0b7b80 1 -- 172.21.15.37:0/2898325310 >> 172.21.15.37:0/2898325310 conn(0x5594a3ff1080 msgr2=0x5594a40be700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.588+0000 7f5d8b7fe700 0 client.17163 ms_handle_reset on v2:172.21.15.167:6836/2720148065 2024-06-10T01:07:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.588+0000 7f5dbd0b7b80 1 -- 172.21.15.37:0/2898325310 shutdown_connections 2024-06-10T01:07:10.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.589+0000 7f5dbd0b7b80 1 -- 172.21.15.37:0/2898325310 wait complete. 2024-06-10T01:07:10.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.589+0000 7f5dbd0b7b80 20 client.17163 trim_cache size 0 max 16384 2024-06-10T01:07:10.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.604+0000 7f5dbd0b7b80 1 Processor -- start 2024-06-10T01:07:10.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.605+0000 7f5dbd0b7b80 1 -- start start 2024-06-10T01:07:10.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.605+0000 7f5dbd0b7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594a40c47a0 0x5594a40c4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:10.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.605+0000 7f5dbd0b7b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594a40c9f60 0x5594a40c8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:10.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.605+0000 7f5dbd0b7b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594a40ca880 0x5594a40ce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:10.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.605+0000 7f5dbd0b7b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5d98011500 con 0x5594a40c9f60 2024-06-10T01:07:10.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.605+0000 7f5dbd0b7b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9800ed90 con 0x5594a40ca880 2024-06-10T01:07:10.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.605+0000 7f5dbd0b7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9800cbe0 con 0x5594a40c47a0 2024-06-10T01:07:10.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.605+0000 7f5dab7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594a40c9f60 0x5594a40c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:10.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.605+0000 7f5dab7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594a40c9f60 0x5594a40c8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46262/0 (socket says 172.21.15.37:46262) 2024-06-10T01:07:10.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.605+0000 7f5dab7fe700 1 -- 172.21.15.37:0/761260899 learned_addr learned my addr 172.21.15.37:0/761260899 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:10.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.605+0000 7f5dabfff700 1 --2- 172.21.15.37:0/761260899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594a40c47a0 0x5594a40c4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:10.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.605+0000 7f5db0dd1700 1 --2- 172.21.15.37:0/761260899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594a40ca880 0x5594a40ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:10.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.605+0000 7f5dabfff700 1 -- 172.21.15.37:0/761260899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594a40ca880 msgr2=0x5594a40ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:10.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.605+0000 7f5dabfff700 1 --2- 172.21.15.37:0/761260899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594a40ca880 0x5594a40ce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.605+0000 7f5dabfff700 1 -- 172.21.15.37:0/761260899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594a40c9f60 msgr2=0x5594a40c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:10.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.605+0000 7f5dabfff700 1 --2- 172.21.15.37:0/761260899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594a40c9f60 0x5594a40c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.605+0000 7f5dabfff700 1 -- 172.21.15.37:0/761260899 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594a40cbea0 con 0x5594a40c47a0 2024-06-10T01:07:10.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.605+0000 7f5dab7fe700 1 --2- 172.21.15.37:0/761260899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594a40c9f60 0x5594a40c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:10.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.606+0000 7f5dabfff700 1 --2- 172.21.15.37:0/761260899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594a40c47a0 0x5594a40c4260 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00f010 tx=0x7f5d9c09b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd376a098d4a942c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:10.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.606+0000 7f5d91ffb700 1 -- 172.21.15.37:0/761260899 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c0ba2d0 con 0x5594a40c47a0 2024-06-10T01:07:10.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.606+0000 7f5d91ffb700 1 -- 172.21.15.37:0/761260899 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d9c0b4040 con 0x5594a40c47a0 2024-06-10T01:07:10.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.606+0000 7f5d91ffb700 1 -- 172.21.15.37:0/761260899 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c00cd70 con 0x5594a40c47a0 2024-06-10T01:07:10.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.606+0000 7f5dbd0b7b80 1 -- 172.21.15.37:0/761260899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594a40c47a0 msgr2=0x5594a40c4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:10.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.606+0000 7f5dbd0b7b80 1 --2- 172.21.15.37:0/761260899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594a40c47a0 0x5594a40c4260 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00f010 tx=0x7f5d9c09b880 comp rx=0 tx=0).stop 2024-06-10T01:07:10.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.606+0000 7f5dbd0b7b80 1 -- 172.21.15.37:0/761260899 shutdown_connections 2024-06-10T01:07:10.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.606+0000 7f5dbd0b7b80 1 --2- 172.21.15.37:0/761260899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594a40ca880 0x5594a40ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.606+0000 7f5dbd0b7b80 1 --2- 172.21.15.37:0/761260899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594a40c47a0 0x5594a40c4260 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.606+0000 7f5dbd0b7b80 1 --2- 172.21.15.37:0/761260899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594a40c9f60 0x5594a40c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.606+0000 7f5dbd0b7b80 1 -- 172.21.15.37:0/761260899 >> 172.21.15.37:0/761260899 conn(0x5594a3ff1080 msgr2=0x5594a40cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:10.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.607+0000 7f5dbd0b7b80 1 -- 172.21.15.37:0/761260899 shutdown_connections 2024-06-10T01:07:10.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.607+0000 7f5dbd0b7b80 1 -- 172.21.15.37:0/761260899 wait complete. 2024-06-10T01:07:10.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.607+0000 7f5dbd0b7b80 1 -- start start 2024-06-10T01:07:10.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.607+0000 7f5dbd0b7b80 1 Processor -- start 2024-06-10T01:07:10.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.607+0000 7f5dbd0b7b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594a40c47a0 0x5594a40d6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:10.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.607+0000 7f5dbd0b7b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594a40c9f60 0x5594a40d6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:10.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.608+0000 7f5dbd0b7b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594a40ca880 0x5594a40e5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:10.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.608+0000 7f5dbd0b7b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9800d250 con 0x5594a40ca880 2024-06-10T01:07:10.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.608+0000 7f5dbd0b7b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9800e6c0 con 0x5594a40c47a0 2024-06-10T01:07:10.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.608+0000 7f5dbd0b7b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9800d920 con 0x5594a40c9f60 2024-06-10T01:07:10.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.608+0000 7f5dbd0b7b80 10 client.0 _ll_register_callbacks cb 0x5594a3fab5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:10.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.608+0000 7f5dabfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594a40c47a0 0x5594a40d6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:10.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.608+0000 7f5dab7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594a40c9f60 0x5594a40d6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:10.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.608+0000 7f5dabfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594a40c47a0 0x5594a40d6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:41330/0 (socket says 172.21.15.37:41330) 2024-06-10T01:07:10.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.608+0000 7f5dabfff700 1 -- 172.21.15.37:0/659969889 learned_addr learned my addr 172.21.15.37:0/659969889 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:10.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.608+0000 7f5db0dd1700 1 --2- 172.21.15.37:0/659969889 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594a40ca880 0x5594a40e5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:10.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.608+0000 7f5dabfff700 1 -- 172.21.15.37:0/659969889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594a40c9f60 msgr2=0x5594a40d6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:10.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.608+0000 7f5dabfff700 1 --2- 172.21.15.37:0/659969889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594a40c9f60 0x5594a40d6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.608+0000 7f5dabfff700 1 -- 172.21.15.37:0/659969889 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594a40ca880 msgr2=0x5594a40e5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:10.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.608+0000 7f5dabfff700 1 --2- 172.21.15.37:0/659969889 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594a40ca880 0x5594a40e5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.608+0000 7f5dabfff700 1 -- 172.21.15.37:0/659969889 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594a40e5a40 con 0x5594a40c47a0 2024-06-10T01:07:10.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.608+0000 7f5dab7fe700 1 --2- 172.21.15.37:0/659969889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594a40c9f60 0x5594a40d6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:10.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.608+0000 7f5dabfff700 1 --2- 172.21.15.37:0/659969889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594a40c47a0 0x5594a40d6740 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c09b680 tx=0x7f5d9c005bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b15d0820e00fffa9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:10.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.608+0000 7f5d8b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:07:10.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.608+0000 7f5d8b7fe700 1 -- 172.21.15.37:0/659969889 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c0a45f0 con 0x5594a40c47a0 2024-06-10T01:07:10.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.608+0000 7f5d8b7fe700 1 -- 172.21.15.37:0/659969889 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f5d9c016df0 con 0x5594a40c47a0 2024-06-10T01:07:10.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.608+0000 7f5dbd0b7b80 1 -- 172.21.15.37:0/659969889 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d9c0b4040 con 0x5594a40c47a0 2024-06-10T01:07:10.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.609+0000 7f5d8b7fe700 1 -- 172.21.15.37:0/659969889 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5d9c099c80 con 0x5594a40c47a0 2024-06-10T01:07:10.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.609+0000 7f5dbd0b7b80 10 client.26549 fetch_fsmap learned FSMap version 31 2024-06-10T01:07:10.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.609+0000 7f5dbd0b7b80 1 -- 172.21.15.37:0/659969889 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5594a41488c0 con 0x5594a40c47a0 2024-06-10T01:07:10.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.609+0000 7f5d8b7fe700 1 -- 172.21.15.37:0/659969889 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d9c0b4040 con 0x5594a40c47a0 2024-06-10T01:07:10.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.609+0000 7f5d8b7fe700 1 -- 172.21.15.37:0/659969889 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f5d9c0a4160 con 0x5594a40c47a0 2024-06-10T01:07:10.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.609+0000 7f5dbd0b7b80 10 client.26549 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:07:10.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.609+0000 7f5dbd0b7b80 10 client.26549 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-10T01:07:10.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.609+0000 7f5dbd0b7b80 20 client.26549 populate_metadata read hostname 'smithi037' 2024-06-10T01:07:10.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.609+0000 7f5dbd0b7b80 1 --2- 172.21.15.37:0/659969889 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5594a4148b40 0x5594a414af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:10.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.609+0000 7f5dbd0b7b80 4 client.26549 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:07:10.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.609+0000 7f5dbd0b7b80 1 -- 172.21.15.37:0/659969889 --> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5594a401d2d0 con 0x5594a4148b40 2024-06-10T01:07:10.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.609+0000 7f5dab7fe700 1 --2- 172.21.15.37:0/659969889 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5594a4148b40 0x5594a414af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:10.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.610+0000 7f5dab7fe700 1 --2- 172.21.15.37:0/659969889 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5594a4148b40 0x5594a414af30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7789cf8493088ca9 server_cookie=7adbac266c993ce5 in_seq=0 out_seq=0 2024-06-10T01:07:10.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.610+0000 7f5d8b7fe700 10 client.26549 ms_handle_connect on v2:172.21.15.167:6836/2720148065 2024-06-10T01:07:10.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.610+0000 7f5d8b7fe700 1 -- 172.21.15.37:0/659969889 <== mds.2 v2:172.21.15.167:6836/2720148065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5594a401d2d0 con 0x5594a4148b40 2024-06-10T01:07:10.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.610+0000 7f5d8b7fe700 10 client.26549 handle_command_reply: tid=0 2024-06-10T01:07:10.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.610+0000 7f5dbd0b7b80 1 client.26549 shutdown 2024-06-10T01:07:10.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.610+0000 7f5dbd0b7b80 1 -- 172.21.15.37:0/659969889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594a40c47a0 msgr2=0x5594a40d6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:10.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.610+0000 7f5dbd0b7b80 1 --2- 172.21.15.37:0/659969889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594a40c47a0 0x5594a40d6740 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c09b680 tx=0x7f5d9c005bc0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.610+0000 7f5dbd0b7b80 1 -- 172.21.15.37:0/659969889 shutdown_connections 2024-06-10T01:07:10.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.610+0000 7f5dbd0b7b80 1 --2- 172.21.15.37:0/659969889 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5594a40ca880 0x5594a40e5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.610+0000 7f5dbd0b7b80 1 --2- 172.21.15.37:0/659969889 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5594a4148b40 0x5594a414af30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.610+0000 7f5d8b7fe700 0 client.26549 ms_handle_reset on v2:172.21.15.167:6836/2720148065 2024-06-10T01:07:10.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.610+0000 7f5dbd0b7b80 1 --2- 172.21.15.37:0/659969889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5594a40c47a0 0x5594a40d6740 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.610+0000 7f5dbd0b7b80 1 --2- 172.21.15.37:0/659969889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594a40c9f60 0x5594a40d6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.610+0000 7f5dbd0b7b80 1 -- 172.21.15.37:0/659969889 >> 172.21.15.37:0/659969889 conn(0x5594a3ff1080 msgr2=0x5594a40cac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:10.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.610+0000 7f5dbd0b7b80 1 -- 172.21.15.37:0/659969889 shutdown_connections 2024-06-10T01:07:10.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.610+0000 7f5dbd0b7b80 1 -- 172.21.15.37:0/659969889 wait complete. 2024-06-10T01:07:10.612 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T01:07:10.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.610+0000 7f5dbd0b7b80 20 client.26549 trim_cache size 0 max 16384 2024-06-10T01:07:10.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.611+0000 7f5db2834700 1 -- 172.21.15.37:0/3686247773 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5d940a2ca0 msgr2=0x7f5d940a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:10.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.611+0000 7f5db2834700 1 --2- 172.21.15.37:0/3686247773 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5d940a2ca0 0x7f5d940a5160 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5dac1027e0 tx=0x7f5da000a380 comp rx=0 tx=0).stop 2024-06-10T01:07:10.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.611+0000 7f5db2834700 1 -- 172.21.15.37:0/3686247773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dac06a1a0 msgr2=0x7f5dac19aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:10.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.611+0000 7f5db2834700 1 --2- 172.21.15.37:0/3686247773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dac06a1a0 0x7f5dac19aef0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00e9c0 tx=0x7f5d9c00ecd0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.611+0000 7f5db2834700 1 -- 172.21.15.37:0/3686247773 shutdown_connections 2024-06-10T01:07:10.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.611+0000 7f5db2834700 1 --2- 172.21.15.37:0/3686247773 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f5d940a2ca0 0x7f5d940a5160 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.611+0000 7f5db2834700 1 --2- 172.21.15.37:0/3686247773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dac10ef40 0x7f5dac19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.611+0000 7f5db2834700 1 --2- 172.21.15.37:0/3686247773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dac06a1a0 0x7f5dac19aef0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.611+0000 7f5db2834700 1 --2- 172.21.15.37:0/3686247773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dac06aac0 0x7f5dac19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:10.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.611+0000 7f5db2834700 1 -- 172.21.15.37:0/3686247773 >> 172.21.15.37:0/3686247773 conn(0x7f5dac078340 msgr2=0x7f5dac110390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:10.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.611+0000 7f5db2834700 1 -- 172.21.15.37:0/3686247773 shutdown_connections 2024-06-10T01:07:10.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:10.611+0000 7f5db2834700 1 -- 172.21.15.37:0/3686247773 wait complete. 2024-06-10T01:07:10.614 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:10 smithi094 ceph-mon[36708]: pgmap v745: 129 pgs: 129 active+clean; 11 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 414 KiB/s wr, 1.07k op/s 2024-06-10T01:07:11.357 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:07:11.734 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:12.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:12 smithi037 ceph-mon[32192]: pgmap v746: 129 pgs: 129 active+clean; 11 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 401 KiB/s wr, 445 op/s 2024-06-10T01:07:12.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:12 smithi167 ceph-mon[36917]: pgmap v746: 129 pgs: 129 active+clean; 11 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 401 KiB/s wr, 445 op/s 2024-06-10T01:07:12.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:12 smithi094 ceph-mon[36708]: pgmap v746: 129 pgs: 129 active+clean; 11 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 401 KiB/s wr, 445 op/s 2024-06-10T01:07:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.475+0000 7f08eaca5640 1 -- 172.21.15.37:0/2055895669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08e4109ab0 msgr2=0x7f08e410bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.475+0000 7f08eaca5640 1 --2- 172.21.15.37:0/2055895669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08e4109ab0 0x7f08e410bea0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009900 tx=0x7f08d802a3f0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.476+0000 7f08eaca5640 1 -- 172.21.15.37:0/2055895669 shutdown_connections 2024-06-10T01:07:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.476+0000 7f08eaca5640 1 --2- 172.21.15.37:0/2055895669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08e410ed60 0x7f08e4111220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.476+0000 7f08eaca5640 1 --2- 172.21.15.37:0/2055895669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08e410c3e0 0x7f08e410e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.476+0000 7f08eaca5640 1 --2- 172.21.15.37:0/2055895669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08e4109ab0 0x7f08e410bea0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.476+0000 7f08eaca5640 1 -- 172.21.15.37:0/2055895669 >> 172.21.15.37:0/2055895669 conn(0x7f08e406f010 msgr2=0x7f08e4071450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.476+0000 7f08eaca5640 1 -- 172.21.15.37:0/2055895669 shutdown_connections 2024-06-10T01:07:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.476+0000 7f08eaca5640 1 -- 172.21.15.37:0/2055895669 wait complete. 2024-06-10T01:07:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.476+0000 7f08eaca5640 1 Processor -- start 2024-06-10T01:07:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.477+0000 7f08eaca5640 1 -- start start 2024-06-10T01:07:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.477+0000 7f08eaca5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08e4109ab0 0x7f08e41a6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.477+0000 7f08eaca5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08e410c3e0 0x7f08e41a6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.477+0000 7f08eaca5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08e410ed60 0x7f08e41ae050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.477+0000 7f08eaca5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f08e41134b0 con 0x7f08e410ed60 2024-06-10T01:07:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.477+0000 7f08eaca5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f08e4113330 con 0x7f08e410c3e0 2024-06-10T01:07:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.477+0000 7f08eaca5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f08e4113630 con 0x7f08e4109ab0 2024-06-10T01:07:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.477+0000 7f08e9242640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08e410ed60 0x7f08e41ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.477+0000 7f08e9242640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08e410ed60 0x7f08e41ae050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57830/0 (socket says 172.21.15.37:57830) 2024-06-10T01:07:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.477+0000 7f08e9242640 1 -- 172.21.15.37:0/1853080265 learned_addr learned my addr 172.21.15.37:0/1853080265 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.477+0000 7f08e8a41640 1 --2- 172.21.15.37:0/1853080265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08e4109ab0 0x7f08e41a6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.478+0000 7f08e3fff640 1 --2- 172.21.15.37:0/1853080265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08e410c3e0 0x7f08e41a6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.478+0000 7f08e9242640 1 -- 172.21.15.37:0/1853080265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08e4109ab0 msgr2=0x7f08e41a6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.478+0000 7f08e9242640 1 --2- 172.21.15.37:0/1853080265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08e4109ab0 0x7f08e41a6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.478+0000 7f08e9242640 1 -- 172.21.15.37:0/1853080265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08e410c3e0 msgr2=0x7f08e41a6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.478+0000 7f08e9242640 1 --2- 172.21.15.37:0/1853080265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08e410c3e0 0x7f08e41a6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.478+0000 7f08e9242640 1 -- 172.21.15.37:0/1853080265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e41ae6c0 con 0x7f08e410ed60 2024-06-10T01:07:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.478+0000 7f08e9242640 1 --2- 172.21.15.37:0/1853080265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08e410ed60 0x7f08e41ae050 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00eb10 tx=0x7f08dc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.479+0000 7f08e1ffb640 1 -- 172.21.15.37:0/1853080265 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08dc004b20 con 0x7f08e410ed60 2024-06-10T01:07:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.479+0000 7f08e1ffb640 1 -- 172.21.15.37:0/1853080265 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08dc004cc0 con 0x7f08e410ed60 2024-06-10T01:07:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.479+0000 7f08e1ffb640 1 -- 172.21.15.37:0/1853080265 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08dc010760 con 0x7f08e410ed60 2024-06-10T01:07:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.480+0000 7f08eaca5640 1 -- 172.21.15.37:0/1853080265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08e41ae9b0 con 0x7f08e410ed60 2024-06-10T01:07:13.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.480+0000 7f08eaca5640 1 -- 172.21.15.37:0/1853080265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08e41aef00 con 0x7f08e410ed60 2024-06-10T01:07:13.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.485+0000 7f08e1ffb640 1 -- 172.21.15.37:0/1853080265 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f08dc010900 con 0x7f08e410ed60 2024-06-10T01:07:13.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.485+0000 7f08ed9b9740 1 Processor -- start 2024-06-10T01:07:13.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.485+0000 7f08ed9b9740 1 -- start start 2024-06-10T01:07:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.486+0000 7f08ed9b9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cae9a71010 0x55cae9a71430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.486+0000 7f08ed9b9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cae9a71a00 0x55cae9a76500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.486+0000 7f08ed9b9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cae9a76b70 0x55cae9a79000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.486+0000 7f08ed9b9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cae99d44f0 con 0x55cae9a76b70 2024-06-10T01:07:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.486+0000 7f08ed9b9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cae9972870 con 0x55cae9a71010 2024-06-10T01:07:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.486+0000 7f08ed9b9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cae99d04b0 con 0x55cae9a71a00 2024-06-10T01:07:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.486+0000 7f08e1ffb640 1 --2- 172.21.15.37:0/1853080265 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f08d40a2dd0 0x7f08d40a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.486+0000 7f08e1ffb640 1 -- 172.21.15.37:0/1853080265 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f08dc0cfa00 con 0x7f08e410ed60 2024-06-10T01:07:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.486+0000 7f08e8a41640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cae9a71a00 0x55cae9a76500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.486+0000 7f08e8a41640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cae9a71a00 0x55cae9a76500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:55012/0 (socket says 172.21.15.37:55012) 2024-06-10T01:07:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.486+0000 7f08e8a41640 1 -- 172.21.15.37:0/1667346079 learned_addr learned my addr 172.21.15.37:0/1667346079 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.487+0000 7f08e3fff640 1 --2- 172.21.15.37:0/1667346079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cae9a76b70 0x55cae9a79000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.487+0000 7f08e3fff640 1 --2- 172.21.15.37:0/1667346079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cae9a71010 0x55cae9a71430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.487+0000 7f08e9242640 1 --2- 172.21.15.37:0/1853080265 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f08d40a2dd0 0x7f08d40a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.487+0000 7f08e3fff640 1 -- 172.21.15.37:0/1667346079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cae9a71a00 msgr2=0x55cae9a76500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.487+0000 7f08e3fff640 1 --2- 172.21.15.37:0/1667346079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cae9a71a00 0x55cae9a76500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.487+0000 7f08e3fff640 1 -- 172.21.15.37:0/1667346079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cae9a76b70 msgr2=0x55cae9a79000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.487+0000 7f08e3fff640 1 --2- 172.21.15.37:0/1667346079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cae9a76b70 0x55cae9a79000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.487+0000 7f08e3fff640 1 -- 172.21.15.37:0/1667346079 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cae998e9b0 con 0x55cae9a71010 2024-06-10T01:07:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.487+0000 7f08e3fff640 1 --2- 172.21.15.37:0/1667346079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cae9a71010 0x55cae9a71430 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f08e41a7fb0 tx=0x7f08d0006b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c85529557cebb66 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.488+0000 7f08ceffd640 1 -- 172.21.15.37:0/1667346079 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d001c070 con 0x55cae9a71010 2024-06-10T01:07:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.488+0000 7f08ceffd640 1 -- 172.21.15.37:0/1667346079 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08d0013be0 con 0x55cae9a71010 2024-06-10T01:07:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.488+0000 7f08ceffd640 1 -- 172.21.15.37:0/1667346079 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d000f040 con 0x55cae9a71010 2024-06-10T01:07:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.489+0000 7f08ed9b9740 1 -- 172.21.15.37:0/1667346079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cae9a71010 msgr2=0x55cae9a71430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.489+0000 7f08ed9b9740 1 --2- 172.21.15.37:0/1667346079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cae9a71010 0x55cae9a71430 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f08e41a7fb0 tx=0x7f08d0006b20 comp rx=0 tx=0).stop 2024-06-10T01:07:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.489+0000 7f08e9242640 1 --2- 172.21.15.37:0/1853080265 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f08d40a2dd0 0x7f08d40a5290 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f08dc0049d0 tx=0x7f08dc005b40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.489+0000 7f08ed9b9740 1 -- 172.21.15.37:0/1667346079 shutdown_connections 2024-06-10T01:07:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.489+0000 7f08ed9b9740 1 --2- 172.21.15.37:0/1667346079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cae9a76b70 0x55cae9a79000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.489+0000 7f08ed9b9740 1 --2- 172.21.15.37:0/1667346079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cae9a71a00 0x55cae9a76500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.489+0000 7f08ed9b9740 1 --2- 172.21.15.37:0/1667346079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cae9a71010 0x55cae9a71430 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.489+0000 7f08ed9b9740 1 -- 172.21.15.37:0/1667346079 >> 172.21.15.37:0/1667346079 conn(0x55cae9a6c680 msgr2=0x55cae9a6ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.489+0000 7f08ed9b9740 1 -- 172.21.15.37:0/1667346079 shutdown_connections 2024-06-10T01:07:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.490+0000 7f08ed9b9740 1 -- 172.21.15.37:0/1667346079 wait complete. 2024-06-10T01:07:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.490+0000 7f08ed9b9740 1 -- start start 2024-06-10T01:07:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.490+0000 7f08ed9b9740 1 Processor -- start 2024-06-10T01:07:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.491+0000 7f08ed9b9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cae9a71010 0x55cae9a7b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.491+0000 7f08ed9b9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cae9a71a00 0x55cae9a7b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.491+0000 7f08ed9b9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cae9a76b70 0x55cae9a82860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.491+0000 7f08ed9b9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cae9a79ac0 con 0x55cae9a71010 2024-06-10T01:07:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.491+0000 7f08ed9b9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cae995dae0 con 0x55cae9a71a00 2024-06-10T01:07:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.491+0000 7f08ed9b9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cae9a79c40 con 0x55cae9a76b70 2024-06-10T01:07:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.491+0000 7f08ed9b9740 10 client.0 _ll_register_callbacks cb 0x55cae99969b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.491+0000 7f08e8a41640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cae9a71a00 0x55cae9a7b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.491+0000 7f08e8a41640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cae9a71a00 0x55cae9a7b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:56916/0 (socket says 172.21.15.37:56916) 2024-06-10T01:07:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.491+0000 7f08e8a41640 1 -- 172.21.15.37:0/4276005195 learned_addr learned my addr 172.21.15.37:0/4276005195 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.492+0000 7f08e8a41640 1 -- 172.21.15.37:0/4276005195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cae9a76b70 msgr2=0x55cae9a82860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:07:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.492+0000 7f08e8a41640 1 --2- 172.21.15.37:0/4276005195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cae9a76b70 0x55cae9a82860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.492+0000 7f08e8a41640 1 -- 172.21.15.37:0/4276005195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cae9a71010 msgr2=0x55cae9a7b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.492+0000 7f08e8a41640 1 --2- 172.21.15.37:0/4276005195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cae9a71010 0x55cae9a7b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.492+0000 7f08e8a41640 1 -- 172.21.15.37:0/4276005195 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cae96e8210 con 0x55cae9a71a00 2024-06-10T01:07:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.492+0000 7f08e8a41640 1 --2- 172.21.15.37:0/4276005195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cae9a71a00 0x55cae9a7b820 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009230 tx=0x7f08d8005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db9912e34eed5201 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.492+0000 7f08c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:07:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.492+0000 7f08c3fff640 1 -- 172.21.15.37:0/4276005195 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d8035040 con 0x55cae9a71a00 2024-06-10T01:07:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.492+0000 7f08ed9b9740 1 -- 172.21.15.37:0/4276005195 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08d0013be0 con 0x55cae9a71a00 2024-06-10T01:07:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.492+0000 7f08c3fff640 1 -- 172.21.15.37:0/4276005195 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f08d80316b0 con 0x55cae9a71a00 2024-06-10T01:07:13.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.493+0000 7f08c3fff640 1 -- 172.21.15.37:0/4276005195 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f08d80318b0 con 0x55cae9a71a00 2024-06-10T01:07:13.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.493+0000 7f08ed9b9740 10 client.26558 fetch_fsmap learned FSMap version 31 2024-06-10T01:07:13.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.493+0000 7f08ed9b9740 1 -- 172.21.15.37:0/4276005195 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55cae97b4620 con 0x55cae9a71a00 2024-06-10T01:07:13.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.493+0000 7f08c3fff640 1 -- 172.21.15.37:0/4276005195 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08d0013be0 con 0x55cae9a71a00 2024-06-10T01:07:13.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.493+0000 7f08c3fff640 1 -- 172.21.15.37:0/4276005195 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f08d80035f0 con 0x55cae9a71a00 2024-06-10T01:07:13.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.494+0000 7f08ed9b9740 10 client.26558 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:07:13.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.494+0000 7f08ed9b9740 10 client.26558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:13.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.494+0000 7f08ed9b9740 20 client.26558 populate_metadata read hostname 'smithi037' 2024-06-10T01:07:13.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.494+0000 7f08ed9b9740 1 --2- 172.21.15.37:0/4276005195 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55cae9a98550 0x55cae9a9a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.494+0000 7f08ed9b9740 4 client.26558 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:13.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.494+0000 7f08ed9b9740 1 -- 172.21.15.37:0/4276005195 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cae99e7f10 con 0x55cae9a98550 2024-06-10T01:07:13.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.495+0000 7f08e3fff640 1 --2- 172.21.15.37:0/4276005195 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55cae9a98550 0x55cae9a9a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.495+0000 7f08e3fff640 1 --2- 172.21.15.37:0/4276005195 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55cae9a98550 0x55cae9a9a9f0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c60e71fba48400d8 server_cookie=769a235328a3b28a in_seq=0 out_seq=0 2024-06-10T01:07:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.495+0000 7f08c3fff640 10 client.26558 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:13.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.504+0000 7f08c3fff640 1 -- 172.21.15.37:0/4276005195 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cae99e7f10 con 0x55cae9a98550 2024-06-10T01:07:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.504+0000 7f08c3fff640 10 client.26558 handle_command_reply: tid=0 2024-06-10T01:07:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.505+0000 7f08ed9b9740 1 client.26558 shutdown 2024-06-10T01:07:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.505+0000 7f08ed9b9740 1 -- 172.21.15.37:0/4276005195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cae9a71a00 msgr2=0x55cae9a7b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.505+0000 7f08ed9b9740 1 --2- 172.21.15.37:0/4276005195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cae9a71a00 0x55cae9a7b820 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009230 tx=0x7f08d8005040 comp rx=0 tx=0).stop 2024-06-10T01:07:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.508+0000 7f08ed9b9740 1 -- 172.21.15.37:0/4276005195 shutdown_connections 2024-06-10T01:07:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.508+0000 7f08ed9b9740 1 --2- 172.21.15.37:0/4276005195 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55cae9a98550 0x55cae9a9a9f0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.508+0000 7f08ed9b9740 1 --2- 172.21.15.37:0/4276005195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cae9a76b70 0x55cae9a82860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.508+0000 7f08ed9b9740 1 --2- 172.21.15.37:0/4276005195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cae9a71a00 0x55cae9a7b820 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.508+0000 7f08ed9b9740 1 --2- 172.21.15.37:0/4276005195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cae9a71010 0x55cae9a7b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.508+0000 7f08ed9b9740 1 -- 172.21.15.37:0/4276005195 >> 172.21.15.37:0/4276005195 conn(0x55cae9a6c680 msgr2=0x55cae9a6b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.508+0000 7f08c3fff640 0 client.26558 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:13.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.508+0000 7f08ed9b9740 1 -- 172.21.15.37:0/4276005195 shutdown_connections 2024-06-10T01:07:13.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.509+0000 7f08ed9b9740 1 -- 172.21.15.37:0/4276005195 wait complete. 2024-06-10T01:07:13.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.510+0000 7f08ed9b9740 20 client.26558 trim_cache size 0 max 16384 2024-06-10T01:07:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.538+0000 7f08ed9b9740 1 Processor -- start 2024-06-10T01:07:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.538+0000 7f08ed9b9740 1 -- start start 2024-06-10T01:07:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.539+0000 7f08ed9b9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cae9a71010 0x55cae9a7b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.539+0000 7f08ed9b9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cae9a71a00 0x55cae9a7b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.539+0000 7f08ed9b9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cae9a76b70 0x55cae9a82820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.539+0000 7f08e8a41640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cae9a71a00 0x55cae9a7b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.539+0000 7f08e8a41640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cae9a71a00 0x55cae9a7b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:56930/0 (socket says 172.21.15.37:56930) 2024-06-10T01:07:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.539+0000 7f08e8a41640 1 -- 172.21.15.37:0/2261707432 learned_addr learned my addr 172.21.15.37:0/2261707432 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.539+0000 7f08ed9b9740 1 -- 172.21.15.37:0/2261707432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f08e41b36f0 con 0x55cae9a71010 2024-06-10T01:07:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.539+0000 7f08e3fff640 1 --2- 172.21.15.37:0/2261707432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cae9a71010 0x55cae9a7b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.539+0000 7f08ed9b9740 1 -- 172.21.15.37:0/2261707432 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f08e41b0fc0 con 0x55cae9a71a00 2024-06-10T01:07:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.539+0000 7f08ed9b9740 1 -- 172.21.15.37:0/2261707432 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f08e411f200 con 0x55cae9a76b70 2024-06-10T01:07:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.540+0000 7f08e3fff640 1 --2- 172.21.15.37:0/2261707432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cae9a76b70 0x55cae9a82820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.540+0000 7f08e8a41640 1 -- 172.21.15.37:0/2261707432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cae9a76b70 msgr2=0x55cae9a82820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.540+0000 7f08e8a41640 1 --2- 172.21.15.37:0/2261707432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cae9a76b70 0x55cae9a82820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.540+0000 7f08e8a41640 1 -- 172.21.15.37:0/2261707432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cae9a71010 msgr2=0x55cae9a7b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.540+0000 7f08e8a41640 1 --2- 172.21.15.37:0/2261707432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cae9a71010 0x55cae9a7b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.540+0000 7f08e8a41640 1 -- 172.21.15.37:0/2261707432 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cae9a6c4b0 con 0x55cae9a71a00 2024-06-10T01:07:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.540+0000 7f08e8a41640 1 --2- 172.21.15.37:0/2261707432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cae9a71a00 0x55cae9a7b820 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009850 tx=0x7f08d8005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3939014a36243d08 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.541+0000 7f08e3fff640 1 --2- 172.21.15.37:0/2261707432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cae9a76b70 0x55cae9a82820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:07:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.541+0000 7f08ce7fc640 1 -- 172.21.15.37:0/2261707432 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d8035030 con 0x55cae9a71a00 2024-06-10T01:07:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.541+0000 7f08ce7fc640 1 -- 172.21.15.37:0/2261707432 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08d8041070 con 0x55cae9a71a00 2024-06-10T01:07:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.541+0000 7f08ce7fc640 1 -- 172.21.15.37:0/2261707432 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d80046e0 con 0x55cae9a71a00 2024-06-10T01:07:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.542+0000 7f08ed9b9740 1 -- 172.21.15.37:0/2261707432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cae9a71a00 msgr2=0x55cae9a7b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.542+0000 7f08ed9b9740 1 --2- 172.21.15.37:0/2261707432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cae9a71a00 0x55cae9a7b820 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009850 tx=0x7f08d8005970 comp rx=0 tx=0).stop 2024-06-10T01:07:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.543+0000 7f08ed9b9740 1 -- 172.21.15.37:0/2261707432 shutdown_connections 2024-06-10T01:07:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.543+0000 7f08ed9b9740 1 --2- 172.21.15.37:0/2261707432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cae9a76b70 0x55cae9a82820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.543+0000 7f08ed9b9740 1 --2- 172.21.15.37:0/2261707432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cae9a71a00 0x55cae9a7b820 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.543+0000 7f08ed9b9740 1 --2- 172.21.15.37:0/2261707432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cae9a71010 0x55cae9a7b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.543+0000 7f08ed9b9740 1 -- 172.21.15.37:0/2261707432 >> 172.21.15.37:0/2261707432 conn(0x55cae9a6c680 msgr2=0x55cae9a6ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.545+0000 7f08ed9b9740 1 -- 172.21.15.37:0/2261707432 shutdown_connections 2024-06-10T01:07:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.545+0000 7f08ed9b9740 1 -- 172.21.15.37:0/2261707432 wait complete. 2024-06-10T01:07:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.546+0000 7f08ed9b9740 1 -- start start 2024-06-10T01:07:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.546+0000 7f08ed9b9740 1 Processor -- start 2024-06-10T01:07:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.546+0000 7f08ed9b9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cae9a71010 0x55cae9a82540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.546+0000 7f08ed9b9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cae9a71a00 0x55cae9a7f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.546+0000 7f08ed9b9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cae9a76b70 0x55cae9a7f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.546+0000 7f08ed9b9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f08e41af480 con 0x55cae9a71010 2024-06-10T01:07:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.546+0000 7f08ed9b9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f08e41b08f0 con 0x55cae9a71a00 2024-06-10T01:07:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.546+0000 7f08ed9b9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f08e41afb50 con 0x55cae9a76b70 2024-06-10T01:07:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.547+0000 7f08ed9b9740 10 client.0 _ll_register_callbacks cb 0x7f08d8004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.547+0000 7f08e3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cae9a76b70 0x55cae9a7f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.547+0000 7f08e3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cae9a71010 0x55cae9a82540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.547+0000 7f08e3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cae9a71010 0x55cae9a82540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57858/0 (socket says 172.21.15.37:57858) 2024-06-10T01:07:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.547+0000 7f08e3fff640 1 -- 172.21.15.37:0/2912096407 learned_addr learned my addr 172.21.15.37:0/2912096407 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.547+0000 7f08e8a41640 1 --2- 172.21.15.37:0/2912096407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cae9a71a00 0x55cae9a7f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.548+0000 7f08e8a41640 1 -- 172.21.15.37:0/2912096407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cae9a76b70 msgr2=0x55cae9a7f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.548+0000 7f08e8a41640 1 --2- 172.21.15.37:0/2912096407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cae9a76b70 0x55cae9a7f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.548+0000 7f08e8a41640 1 -- 172.21.15.37:0/2912096407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cae9a71010 msgr2=0x55cae9a82540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.548+0000 7f08e8a41640 1 --2- 172.21.15.37:0/2912096407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cae9a71010 0x55cae9a82540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.548+0000 7f08e8a41640 1 -- 172.21.15.37:0/2912096407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cae9a7fbd0 con 0x55cae9a71a00 2024-06-10T01:07:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.548+0000 7f08e8a41640 1 --2- 172.21.15.37:0/2912096407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cae9a71a00 0x55cae9a7f050 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f08d80096c0 tx=0x7f08d8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc84e3d87f057b3d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.548+0000 7f08c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:07:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.548+0000 7f08c3fff640 1 -- 172.21.15.37:0/2912096407 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d8035040 con 0x55cae9a71a00 2024-06-10T01:07:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.548+0000 7f08c3fff640 1 -- 172.21.15.37:0/2912096407 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f08d8031a50 con 0x55cae9a71a00 2024-06-10T01:07:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.548+0000 7f08ed9b9740 1 -- 172.21.15.37:0/2912096407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08d8041070 con 0x55cae9a71a00 2024-06-10T01:07:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.549+0000 7f08c3fff640 1 -- 172.21.15.37:0/2912096407 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f08d8067050 con 0x55cae9a71a00 2024-06-10T01:07:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.549+0000 7f08ed9b9740 10 client.26564 fetch_fsmap learned FSMap version 31 2024-06-10T01:07:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.549+0000 7f08ed9b9740 1 -- 172.21.15.37:0/2912096407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55cae9b02310 con 0x55cae9a71a00 2024-06-10T01:07:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.549+0000 7f08c3fff640 1 -- 172.21.15.37:0/2912096407 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08d8041070 con 0x55cae9a71a00 2024-06-10T01:07:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.549+0000 7f08c3fff640 1 -- 172.21.15.37:0/2912096407 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f08d8059470 con 0x55cae9a71a00 2024-06-10T01:07:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.549+0000 7f08ed9b9740 10 client.26564 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:07:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.549+0000 7f08ed9b9740 10 client.26564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.549+0000 7f08ed9b9740 20 client.26564 populate_metadata read hostname 'smithi037' 2024-06-10T01:07:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.549+0000 7f08ed9b9740 1 --2- 172.21.15.37:0/2912096407 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55cae9b02c60 0x55cae9b05080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.549+0000 7f08ed9b9740 4 client.26564 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:07:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.549+0000 7f08ed9b9740 1 -- 172.21.15.37:0/2912096407 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cae99e1c20 con 0x55cae9b02c60 2024-06-10T01:07:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.553+0000 7f08e3fff640 1 --2- 172.21.15.37:0/2912096407 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55cae9b02c60 0x55cae9b05080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.555+0000 7f08e3fff640 1 --2- 172.21.15.37:0/2912096407 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55cae9b02c60 0x55cae9b05080 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5a1a64718d31c9c server_cookie=347d86ec6854bf01 in_seq=0 out_seq=0 2024-06-10T01:07:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:13.555+0000 7f08c3fff640 10 client.26564 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:13.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.635+0000 7f260e48d700 1 -- 172.21.15.37:0/4099272452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f260806b140 msgr2=0x7f260806b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.635+0000 7f260e48d700 1 --2- 172.21.15.37:0/4099272452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f260806b140 0x7f260806b5c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f25f0009990 tx=0x7f25f0009ca0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.636+0000 7f260e48d700 1 -- 172.21.15.37:0/4099272452 shutdown_connections 2024-06-10T01:07:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.636+0000 7f260e48d700 1 --2- 172.21.15.37:0/4099272452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2608104f00 0x7f2608105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.636+0000 7f260e48d700 1 --2- 172.21.15.37:0/4099272452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26081073e0 0x7f260806ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.636+0000 7f260e48d700 1 --2- 172.21.15.37:0/4099272452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f260806b140 0x7f260806b5c0 secure :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f25f0009990 tx=0x7f25f0009ca0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.636+0000 7f260e48d700 1 -- 172.21.15.37:0/4099272452 >> 172.21.15.37:0/4099272452 conn(0x7f2608077400 msgr2=0x7f2608077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.636+0000 7f260e48d700 1 -- 172.21.15.37:0/4099272452 shutdown_connections 2024-06-10T01:07:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.636+0000 7f260e48d700 1 -- 172.21.15.37:0/4099272452 wait complete. 2024-06-10T01:07:13.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.637+0000 7f260e48d700 1 Processor -- start 2024-06-10T01:07:13.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.637+0000 7f260e48d700 1 -- start start 2024-06-10T01:07:13.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.638+0000 7f260e48d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2608104f00 0x7f260819f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.638+0000 7f260e48d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26081073e0 0x7f260819fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.638+0000 7f260e48d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26081a3e90 0x7f26081a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.638+0000 7f260e48d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2608118ee0 con 0x7f26081a3e90 2024-06-10T01:07:13.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.638+0000 7f260e48d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2608118d60 con 0x7f2608104f00 2024-06-10T01:07:13.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.638+0000 7f260e48d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2608119060 con 0x7f26081073e0 2024-06-10T01:07:13.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.639+0000 7f2607fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2608104f00 0x7f260819f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.639+0000 7f2607fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2608104f00 0x7f260819f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:56974/0 (socket says 172.21.15.37:56974) 2024-06-10T01:07:13.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.639+0000 7f2607fff700 1 -- 172.21.15.37:0/1118366775 learned_addr learned my addr 172.21.15.37:0/1118366775 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:13.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.639+0000 7f25ff7fe700 1 --2- 172.21.15.37:0/1118366775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26081073e0 0x7f260819fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.639+0000 7f2607fff700 1 -- 172.21.15.37:0/1118366775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26081073e0 msgr2=0x7f260819fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:13.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.639+0000 7f2607fff700 1 --2- 172.21.15.37:0/1118366775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26081073e0 0x7f260819fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.639+0000 7f2607fff700 1 -- 172.21.15.37:0/1118366775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26081a3e90 msgr2=0x7f26081a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:13.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.639+0000 7f2607fff700 1 --2- 172.21.15.37:0/1118366775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26081a3e90 0x7f26081a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.639+0000 7f2607fff700 1 -- 172.21.15.37:0/1118366775 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26081a4a40 con 0x7f2608104f00 2024-06-10T01:07:13.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.640+0000 7f2607fff700 1 --2- 172.21.15.37:0/1118366775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2608104f00 0x7f260819f5c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f25f800b530 tx=0x7f25f800b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:13.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.641+0000 7f2605ffb700 1 -- 172.21.15.37:0/1118366775 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f8015020 con 0x7f2608104f00 2024-06-10T01:07:13.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.641+0000 7f2605ffb700 1 -- 172.21.15.37:0/1118366775 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25f8011dd0 con 0x7f2608104f00 2024-06-10T01:07:13.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.641+0000 7f260e48d700 1 -- 172.21.15.37:0/1118366775 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26081ac140 con 0x7f2608104f00 2024-06-10T01:07:13.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.641+0000 7f2605ffb700 1 -- 172.21.15.37:0/1118366775 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f800f2e0 con 0x7f2608104f00 2024-06-10T01:07:13.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.641+0000 7f260e48d700 1 -- 172.21.15.37:0/1118366775 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26081ac4e0 con 0x7f2608104f00 2024-06-10T01:07:13.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.646+0000 7f2605ffb700 1 -- 172.21.15.37:0/1118366775 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f25f8018020 con 0x7f2608104f00 2024-06-10T01:07:13.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.647+0000 7f2605ffb700 1 --2- 172.21.15.37:0/1118366775 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f25f40a2db0 0x7f25f40a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.647+0000 7f2618d10b80 1 Processor -- start 2024-06-10T01:07:13.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.647+0000 7f25ff7fe700 1 --2- 172.21.15.37:0/1118366775 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f25f40a2db0 0x7f25f40a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.647+0000 7f2618d10b80 1 -- start start 2024-06-10T01:07:13.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.647+0000 7f2605ffb700 1 -- 172.21.15.37:0/1118366775 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f25f80cecc0 con 0x7f2608104f00 2024-06-10T01:07:13.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.648+0000 7f2618d10b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f04b04640 0x563f04b04a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.648+0000 7f2618d10b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f04b04f60 0x563f04afe9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.648+0000 7f2618d10b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f04afef20 0x563f04aff340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.648+0000 7f2618d10b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563f04a50810 con 0x563f04b04f60 2024-06-10T01:07:13.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.648+0000 7f2618d10b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563f049f8720 con 0x563f04b04640 2024-06-10T01:07:13.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.648+0000 7f2618d10b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563f04a4c7d0 con 0x563f04afef20 2024-06-10T01:07:13.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.648+0000 7f2607fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f04b04640 0x563f04b04a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.648+0000 7f2607fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f04b04640 0x563f04b04a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:56988/0 (socket says 172.21.15.37:56988) 2024-06-10T01:07:13.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.648+0000 7f2607fff700 1 -- 172.21.15.37:0/1146940551 learned_addr learned my addr 172.21.15.37:0/1146940551 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:13.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.648+0000 7f25ff7fe700 1 --2- 172.21.15.37:0/1146940551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f04b04f60 0x563f04afe9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.648+0000 7f25ff7fe700 1 --2- 172.21.15.37:0/1118366775 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f25f40a2db0 0x7f25f40a5270 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f25f0005950 tx=0x7f25f001a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:13.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.648+0000 7f2607fff700 1 -- 172.21.15.37:0/1146940551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f04afef20 msgr2=0x563f04aff340 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-10T01:07:13.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.648+0000 7f2607fff700 1 --2- 172.21.15.37:0/1146940551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f04afef20 0x563f04aff340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.648+0000 7f2607fff700 1 -- 172.21.15.37:0/1146940551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f04b04f60 msgr2=0x563f04afe9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:13.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.648+0000 7f2607fff700 1 --2- 172.21.15.37:0/1146940551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f04b04f60 0x563f04afe9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.648+0000 7f2607fff700 1 -- 172.21.15.37:0/1146940551 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f049eef50 con 0x563f04b04640 2024-06-10T01:07:13.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.649+0000 7f2607fff700 1 --2- 172.21.15.37:0/1146940551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f04b04640 0x563f04b04a20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f25f80109c0 tx=0x7f25f8010a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7f4ae3281e18fb1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:13.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.650+0000 7f25fe7fc700 1 -- 172.21.15.37:0/1146940551 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f80b4070 con 0x563f04b04640 2024-06-10T01:07:13.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.650+0000 7f25fe7fc700 1 -- 172.21.15.37:0/1146940551 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25f809f480 con 0x563f04b04640 2024-06-10T01:07:13.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.650+0000 7f25fe7fc700 1 -- 172.21.15.37:0/1146940551 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f8010610 con 0x563f04b04640 2024-06-10T01:07:13.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.651+0000 7f2618d10b80 1 -- 172.21.15.37:0/1146940551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f04b04640 msgr2=0x563f04b04a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:13.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.651+0000 7f2618d10b80 1 --2- 172.21.15.37:0/1146940551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f04b04640 0x563f04b04a20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f25f80109c0 tx=0x7f25f8010a10 comp rx=0 tx=0).stop 2024-06-10T01:07:13.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.651+0000 7f2618d10b80 1 -- 172.21.15.37:0/1146940551 shutdown_connections 2024-06-10T01:07:13.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.651+0000 7f2618d10b80 1 --2- 172.21.15.37:0/1146940551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f04afef20 0x563f04aff340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.651+0000 7f2618d10b80 1 --2- 172.21.15.37:0/1146940551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f04b04640 0x563f04b04a20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.651+0000 7f2618d10b80 1 --2- 172.21.15.37:0/1146940551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f04b04f60 0x563f04afe9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.651+0000 7f2618d10b80 1 -- 172.21.15.37:0/1146940551 >> 172.21.15.37:0/1146940551 conn(0x563f04a29cc0 msgr2=0x563f04af9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:13.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.652+0000 7f2618d10b80 1 -- 172.21.15.37:0/1146940551 shutdown_connections 2024-06-10T01:07:13.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.652+0000 7f2618d10b80 1 -- 172.21.15.37:0/1146940551 wait complete. 2024-06-10T01:07:13.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.652+0000 7f2618d10b80 1 -- start start 2024-06-10T01:07:13.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.652+0000 7f2618d10b80 1 Processor -- start 2024-06-10T01:07:13.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.653+0000 7f2618d10b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f04afef20 0x563f04b0ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.653+0000 7f2618d10b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f04b04640 0x563f04b08070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.654+0000 7f2618d10b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f04b04f60 0x563f04b085b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.654+0000 7f2618d10b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563f04b0b9d0 con 0x563f04afef20 2024-06-10T01:07:13.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.654+0000 7f2618d10b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563f049f0bf0 con 0x563f04b04f60 2024-06-10T01:07:13.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.654+0000 7f2618d10b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563f04b0bb50 con 0x563f04b04640 2024-06-10T01:07:13.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.654+0000 7f25ff7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f04b04640 0x563f04b08070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.654+0000 7f25ff7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f04b04640 0x563f04b08070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:55080/0 (socket says 172.21.15.37:55080) 2024-06-10T01:07:13.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.654+0000 7f25ff7fe700 1 -- 172.21.15.37:0/2192260231 learned_addr learned my addr 172.21.15.37:0/2192260231 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:13.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.654+0000 7f260ca2a700 1 --2- 172.21.15.37:0/2192260231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f04b04f60 0x563f04b085b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.654+0000 7f2618d10b80 10 client.0 _ll_register_callbacks cb 0x563f049ea920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:13.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.655+0000 7f25ff7fe700 1 -- 172.21.15.37:0/2192260231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f04b04f60 msgr2=0x563f04b085b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:13.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.655+0000 7f25ff7fe700 1 --2- 172.21.15.37:0/2192260231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f04b04f60 0x563f04b085b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.655+0000 7f25ff7fe700 1 -- 172.21.15.37:0/2192260231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f04afef20 msgr2=0x563f04b0ad20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:07:13.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.655+0000 7f25ff7fe700 1 --2- 172.21.15.37:0/2192260231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f04afef20 0x563f04b0ad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.655+0000 7f25ff7fe700 1 -- 172.21.15.37:0/2192260231 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f0499f8d0 con 0x563f04b04640 2024-06-10T01:07:13.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.657+0000 7f25ff7fe700 1 --2- 172.21.15.37:0/2192260231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f04b04640 0x563f04b08070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f25f0009990 tx=0x7f25f0009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32a613091fbfd2da server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:13.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.657+0000 7f25e77fe700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:07:13.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.657+0000 7f25e77fe700 1 -- 172.21.15.37:0/2192260231 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f0035060 con 0x563f04b04640 2024-06-10T01:07:13.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.657+0000 7f2618d10b80 1 -- 172.21.15.37:0/2192260231 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25f809f480 con 0x563f04b04640 2024-06-10T01:07:13.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.657+0000 7f25e77fe700 1 -- 172.21.15.37:0/2192260231 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f25f000bbe0 con 0x563f04b04640 2024-06-10T01:07:13.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.657+0000 7f25e77fe700 1 -- 172.21.15.37:0/2192260231 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f25f003c050 con 0x563f04b04640 2024-06-10T01:07:13.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.659+0000 7f2618d10b80 10 client.26710 fetch_fsmap learned FSMap version 31 2024-06-10T01:07:13.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.659+0000 7f2618d10b80 1 -- 172.21.15.37:0/2192260231 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563f04b16150 con 0x563f04b04640 2024-06-10T01:07:13.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.660+0000 7f25e77fe700 1 -- 172.21.15.37:0/2192260231 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25f000bde0 con 0x563f04b04640 2024-06-10T01:07:13.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.660+0000 7f25e77fe700 1 -- 172.21.15.37:0/2192260231 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f25f0024410 con 0x563f04b04640 2024-06-10T01:07:13.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.661+0000 7f2618d10b80 10 client.26710 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:07:13.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.661+0000 7f2618d10b80 10 client.26710 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-06-10T01:07:13.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.661+0000 7f2618d10b80 20 client.26710 populate_metadata read hostname 'smithi037' 2024-06-10T01:07:13.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.661+0000 7f2618d10b80 1 --2- 172.21.15.37:0/2192260231 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x563f04b186e0 0x563f04b1ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.661+0000 7f2618d10b80 4 client.26710 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:13.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.661+0000 7f2618d10b80 1 -- 172.21.15.37:0/2192260231 --> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f25f809f480 con 0x563f04b186e0 2024-06-10T01:07:13.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.661+0000 7f2607fff700 1 --2- 172.21.15.37:0/2192260231 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x563f04b186e0 0x563f04b1ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.662+0000 7f2607fff700 1 --2- 172.21.15.37:0/2192260231 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x563f04b186e0 0x563f04b1ab30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9437e78673a523f5 server_cookie=e9b0aaf3f4900860 in_seq=0 out_seq=0 2024-06-10T01:07:13.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.662+0000 7f25e77fe700 10 client.26710 ms_handle_connect on v2:172.21.15.94:6838/946269331 2024-06-10T01:07:13.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.663+0000 7f25e77fe700 1 -- 172.21.15.37:0/2192260231 <== mds.1 v2:172.21.15.94:6838/946269331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f25f809f480 con 0x563f04b186e0 2024-06-10T01:07:13.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.663+0000 7f25e77fe700 10 client.26710 handle_command_reply: tid=0 2024-06-10T01:07:13.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.663+0000 7f2618d10b80 1 client.26710 shutdown 2024-06-10T01:07:13.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.665+0000 7f2618d10b80 1 -- 172.21.15.37:0/2192260231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f04b04640 msgr2=0x563f04b08070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:13.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.665+0000 7f2618d10b80 1 --2- 172.21.15.37:0/2192260231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f04b04640 0x563f04b08070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f25f0009990 tx=0x7f25f0009ca0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.666+0000 7f2618d10b80 1 -- 172.21.15.37:0/2192260231 shutdown_connections 2024-06-10T01:07:13.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.666+0000 7f2618d10b80 1 --2- 172.21.15.37:0/2192260231 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x563f04b186e0 0x563f04b1ab30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.666+0000 7f2618d10b80 1 --2- 172.21.15.37:0/2192260231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f04b04f60 0x563f04b085b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.666+0000 7f2618d10b80 1 --2- 172.21.15.37:0/2192260231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f04afef20 0x563f04b0ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.666+0000 7f2618d10b80 1 --2- 172.21.15.37:0/2192260231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f04b04640 0x563f04b08070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.666+0000 7f25e77fe700 0 client.26710 ms_handle_reset on v2:172.21.15.94:6838/946269331 2024-06-10T01:07:13.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.666+0000 7f2618d10b80 1 -- 172.21.15.37:0/2192260231 >> 172.21.15.37:0/2192260231 conn(0x563f04a29cc0 msgr2=0x563f04af95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:13.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.667+0000 7f2618d10b80 1 -- 172.21.15.37:0/2192260231 shutdown_connections 2024-06-10T01:07:13.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.667+0000 7f2618d10b80 1 -- 172.21.15.37:0/2192260231 wait complete. 2024-06-10T01:07:13.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.669+0000 7f2618d10b80 20 client.26710 trim_cache size 0 max 16384 2024-06-10T01:07:13.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.704+0000 7f2618d10b80 1 Processor -- start 2024-06-10T01:07:13.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.705+0000 7f2618d10b80 1 -- start start 2024-06-10T01:07:13.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.705+0000 7f2618d10b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f04afef20 0x563f04afe9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.705+0000 7f2618d10b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f04b04640 0x563f04b06960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.705+0000 7f2618d10b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f04b04f60 0x563f04b06ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.705+0000 7f2618d10b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f26081b7b10 con 0x563f04afef20 2024-06-10T01:07:13.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.705+0000 7f2618d10b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f26081b53a0 con 0x563f04b04f60 2024-06-10T01:07:13.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.705+0000 7f2618d10b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f26081b31f0 con 0x563f04b04640 2024-06-10T01:07:13.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.705+0000 7f25ff7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f04b04640 0x563f04b06960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.705+0000 7f2607fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f04afef20 0x563f04afe9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.705+0000 7f25ff7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f04b04640 0x563f04b06960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:55092/0 (socket says 172.21.15.37:55092) 2024-06-10T01:07:13.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.705+0000 7f25ff7fe700 1 -- 172.21.15.37:0/1340158113 learned_addr learned my addr 172.21.15.37:0/1340158113 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:13.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.708+0000 7f2607fff700 1 -- 172.21.15.37:0/1340158113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f04b04640 msgr2=0x563f04b06960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:13.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.708+0000 7f2607fff700 1 --2- 172.21.15.37:0/1340158113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f04b04640 0x563f04b06960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.708+0000 7f2607fff700 1 -- 172.21.15.37:0/1340158113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f04b04f60 msgr2=0x563f04b06ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:13.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.708+0000 7f2607fff700 1 --2- 172.21.15.37:0/1340158113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f04b04f60 0x563f04b06ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.708+0000 7f2607fff700 1 -- 172.21.15.37:0/1340158113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f04b065e0 con 0x563f04afef20 2024-06-10T01:07:13.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.712+0000 7f2607fff700 1 --2- 172.21.15.37:0/1340158113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f04afef20 0x563f04afe9e0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f25f809add0 tx=0x7f25f8010a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ac5c0e96f0bed2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:13.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.714+0000 7f25fdffb700 1 -- 172.21.15.37:0/1340158113 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f80b4040 con 0x563f04afef20 2024-06-10T01:07:13.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.714+0000 7f25fdffb700 1 -- 172.21.15.37:0/1340158113 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25f8099170 con 0x563f04afef20 2024-06-10T01:07:13.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.714+0000 7f25fdffb700 1 -- 172.21.15.37:0/1340158113 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f8010610 con 0x563f04afef20 2024-06-10T01:07:13.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.715+0000 7f2618d10b80 1 -- 172.21.15.37:0/1340158113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f04afef20 msgr2=0x563f04afe9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:13.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.715+0000 7f2618d10b80 1 --2- 172.21.15.37:0/1340158113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f04afef20 0x563f04afe9e0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f25f809add0 tx=0x7f25f8010a10 comp rx=0 tx=0).stop 2024-06-10T01:07:13.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.717+0000 7f2618d10b80 1 -- 172.21.15.37:0/1340158113 shutdown_connections 2024-06-10T01:07:13.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.717+0000 7f2618d10b80 1 --2- 172.21.15.37:0/1340158113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f04b04f60 0x563f04b06ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.717+0000 7f2618d10b80 1 --2- 172.21.15.37:0/1340158113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f04afef20 0x563f04afe9e0 secure :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f25f809add0 tx=0x7f25f8010a10 comp rx=0 tx=0).stop 2024-06-10T01:07:13.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.717+0000 7f2618d10b80 1 --2- 172.21.15.37:0/1340158113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f04b04640 0x563f04b06960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.717+0000 7f2618d10b80 1 -- 172.21.15.37:0/1340158113 >> 172.21.15.37:0/1340158113 conn(0x563f04a29cc0 msgr2=0x563f04af9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:13.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.719+0000 7f2618d10b80 1 -- 172.21.15.37:0/1340158113 shutdown_connections 2024-06-10T01:07:13.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.721+0000 7f2618d10b80 1 -- 172.21.15.37:0/1340158113 wait complete. 2024-06-10T01:07:13.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.722+0000 7f2618d10b80 1 -- start start 2024-06-10T01:07:13.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.722+0000 7f2618d10b80 1 Processor -- start 2024-06-10T01:07:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.723+0000 7f2618d10b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f04b04640 0x563f04b10ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.723+0000 7f2618d10b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f04b04f60 0x563f04b11530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.723+0000 7f2618d10b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f04b1f5f0 0x563f04b1fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.723+0000 7f2618d10b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f26081b3860 con 0x563f04b04640 2024-06-10T01:07:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.723+0000 7f2618d10b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f26081b4cd0 con 0x563f04b04f60 2024-06-10T01:07:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.723+0000 7f2618d10b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f26081b3f30 con 0x563f04b1f5f0 2024-06-10T01:07:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.723+0000 7f2618d10b80 10 client.0 _ll_register_callbacks cb 0x563f049ea920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.723+0000 7f260ca2a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f04b1f5f0 0x563f04b1fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.723+0000 7f2607fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f04b04640 0x563f04b10ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.723+0000 7f260ca2a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f04b1f5f0 0x563f04b1fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:55106/0 (socket says 172.21.15.37:55106) 2024-06-10T01:07:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.723+0000 7f2607fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f04b04640 0x563f04b10ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57886/0 (socket says 172.21.15.37:57886) 2024-06-10T01:07:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.723+0000 7f260ca2a700 1 -- 172.21.15.37:0/2643215941 learned_addr learned my addr 172.21.15.37:0/2643215941 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:13.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.724+0000 7f2607fff700 1 -- 172.21.15.37:0/2643215941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f04b1f5f0 msgr2=0x563f04b1fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:13.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.724+0000 7f2607fff700 1 --2- 172.21.15.37:0/2643215941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f04b1f5f0 0x563f04b1fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.724+0000 7f2607fff700 1 -- 172.21.15.37:0/2643215941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f04b04f60 msgr2=0x563f04b11530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:13.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.724+0000 7f2607fff700 1 --2- 172.21.15.37:0/2643215941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f04b04f60 0x563f04b11530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.724+0000 7f2607fff700 1 -- 172.21.15.37:0/2643215941 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f04b200b0 con 0x563f04b04640 2024-06-10T01:07:13.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.724+0000 7f2607fff700 1 --2- 172.21.15.37:0/2643215941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f04b04640 0x563f04b10ff0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f25f8004790 tx=0x7f25f80b41e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f444c1fc4b28152f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:13.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.724+0000 7f25e77fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:07:13.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.725+0000 7f25e77fe700 1 -- 172.21.15.37:0/2643215941 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f80b4040 con 0x563f04b04640 2024-06-10T01:07:13.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.725+0000 7f25e77fe700 1 -- 172.21.15.37:0/2643215941 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f25f80bb0b0 con 0x563f04b04640 2024-06-10T01:07:13.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.725+0000 7f2618d10b80 1 -- 172.21.15.37:0/2643215941 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25f8099170 con 0x563f04b04640 2024-06-10T01:07:13.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.730+0000 7f25e77fe700 1 -- 172.21.15.37:0/2643215941 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f25f8010610 con 0x563f04b04640 2024-06-10T01:07:13.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.730+0000 7f2618d10b80 10 client.17193 fetch_fsmap learned FSMap version 31 2024-06-10T01:07:13.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.730+0000 7f2618d10b80 1 -- 172.21.15.37:0/2643215941 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563f04b94190 con 0x563f04b04640 2024-06-10T01:07:13.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.731+0000 7f25e77fe700 1 -- 172.21.15.37:0/2643215941 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25f8099170 con 0x563f04b04640 2024-06-10T01:07:13.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.732+0000 7f25e77fe700 1 -- 172.21.15.37:0/2643215941 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f25f80a3940 con 0x563f04b04640 2024-06-10T01:07:13.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.732+0000 7f2618d10b80 10 client.17193 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:07:13.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.732+0000 7f2618d10b80 10 client.17193 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-06-10T01:07:13.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.732+0000 7f2618d10b80 20 client.17193 populate_metadata read hostname 'smithi037' 2024-06-10T01:07:13.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.732+0000 7f2618d10b80 1 --2- 172.21.15.37:0/2643215941 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x563f04b94780 0x563f04b96b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:13.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.732+0000 7f2618d10b80 4 client.17193 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:07:13.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.732+0000 7f2618d10b80 1 -- 172.21.15.37:0/2643215941 --> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563f04a5dd10 con 0x563f04b94780 2024-06-10T01:07:13.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.734+0000 7f25ff7fe700 1 --2- 172.21.15.37:0/2643215941 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x563f04b94780 0x563f04b96b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:13.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.735+0000 7f25ff7fe700 1 --2- 172.21.15.37:0/2643215941 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x563f04b94780 0x563f04b96b70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=53486e52a365775f server_cookie=c28ee5d9dcf0156d in_seq=0 out_seq=0 2024-06-10T01:07:13.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.735+0000 7f25e77fe700 10 client.17193 ms_handle_connect on v2:172.21.15.94:6838/946269331 2024-06-10T01:07:13.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.735+0000 7f25e77fe700 1 -- 172.21.15.37:0/2643215941 <== mds.1 v2:172.21.15.94:6838/946269331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563f04a5dd10 con 0x563f04b94780 2024-06-10T01:07:13.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.735+0000 7f25e77fe700 10 client.17193 handle_command_reply: tid=0 2024-06-10T01:07:13.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.735+0000 7f2618d10b80 1 client.17193 shutdown 2024-06-10T01:07:13.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.736+0000 7f2618d10b80 1 -- 172.21.15.37:0/2643215941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f04b04640 msgr2=0x563f04b10ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:13.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.736+0000 7f2618d10b80 1 --2- 172.21.15.37:0/2643215941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f04b04640 0x563f04b10ff0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f25f8004790 tx=0x7f25f80b41e0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.736+0000 7f2618d10b80 1 -- 172.21.15.37:0/2643215941 shutdown_connections 2024-06-10T01:07:13.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.736+0000 7f2618d10b80 1 --2- 172.21.15.37:0/2643215941 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x563f04b94780 0x563f04b96b70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.736+0000 7f2618d10b80 1 --2- 172.21.15.37:0/2643215941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f04b1f5f0 0x563f04b1fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.736+0000 7f2618d10b80 1 --2- 172.21.15.37:0/2643215941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563f04b04640 0x563f04b10ff0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.736+0000 7f2618d10b80 1 --2- 172.21.15.37:0/2643215941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563f04b04f60 0x563f04b11530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.736+0000 7f25e77fe700 0 client.17193 ms_handle_reset on v2:172.21.15.94:6838/946269331 2024-06-10T01:07:13.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.736+0000 7f2618d10b80 1 -- 172.21.15.37:0/2643215941 >> 172.21.15.37:0/2643215941 conn(0x563f04a29cc0 msgr2=0x563f04b067f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:13.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.736+0000 7f2618d10b80 1 -- 172.21.15.37:0/2643215941 shutdown_connections 2024-06-10T01:07:13.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.736+0000 7f2618d10b80 1 -- 172.21.15.37:0/2643215941 wait complete. 2024-06-10T01:07:13.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.737+0000 7f2618d10b80 20 client.17193 trim_cache size 0 max 16384 2024-06-10T01:07:13.734 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T01:07:13.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.737+0000 7f260e48d700 1 -- 172.21.15.37:0/1118366775 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f25f40a2db0 msgr2=0x7f25f40a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:13.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.737+0000 7f260e48d700 1 --2- 172.21.15.37:0/1118366775 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f25f40a2db0 0x7f25f40a5270 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f25f0005950 tx=0x7f25f001a040 comp rx=0 tx=0).stop 2024-06-10T01:07:13.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.737+0000 7f260e48d700 1 -- 172.21.15.37:0/1118366775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2608104f00 msgr2=0x7f260819f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:13.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.737+0000 7f260e48d700 1 --2- 172.21.15.37:0/1118366775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2608104f00 0x7f260819f5c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f25f800b530 tx=0x7f25f800b840 comp rx=0 tx=0).stop 2024-06-10T01:07:13.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.737+0000 7f260e48d700 1 -- 172.21.15.37:0/1118366775 shutdown_connections 2024-06-10T01:07:13.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.737+0000 7f260e48d700 1 --2- 172.21.15.37:0/1118366775 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f25f40a2db0 0x7f25f40a5270 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.738+0000 7f260e48d700 1 --2- 172.21.15.37:0/1118366775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26081a3e90 0x7f26081a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.738+0000 7f260e48d700 1 --2- 172.21.15.37:0/1118366775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2608104f00 0x7f260819f5c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.738+0000 7f260e48d700 1 --2- 172.21.15.37:0/1118366775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26081073e0 0x7f260819fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:13.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.738+0000 7f260e48d700 1 -- 172.21.15.37:0/1118366775 >> 172.21.15.37:0/1118366775 conn(0x7f2608077400 msgr2=0x7f2608101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:13.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.738+0000 7f260e48d700 1 -- 172.21.15.37:0/1118366775 shutdown_connections 2024-06-10T01:07:13.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:13.738+0000 7f260e48d700 1 -- 172.21.15.37:0/1118366775 wait complete. 2024-06-10T01:07:14.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:14 smithi037 ceph-mon[32192]: pgmap v747: 129 pgs: 129 active+clean; 15 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 732 KiB/s wr, 478 op/s 2024-06-10T01:07:14.507 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:07:14.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:14 smithi167 ceph-mon[36917]: pgmap v747: 129 pgs: 129 active+clean; 15 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 732 KiB/s wr, 478 op/s 2024-06-10T01:07:14.613 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:14 smithi094 ceph-mon[36708]: pgmap v747: 129 pgs: 129 active+clean; 15 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 732 KiB/s wr, 478 op/s 2024-06-10T01:07:14.776 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:16.264 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:16 smithi037 ceph-mon[32192]: pgmap v748: 129 pgs: 129 active+clean; 19 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 502 op/s 2024-06-10T01:07:16.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:16 smithi167 ceph-mon[36917]: pgmap v748: 129 pgs: 129 active+clean; 19 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 502 op/s 2024-06-10T01:07:16.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:16 smithi094 ceph-mon[36708]: pgmap v748: 129 pgs: 129 active+clean; 19 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 502 op/s 2024-06-10T01:07:16.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.667+0000 7f46b5960700 1 -- 172.21.15.37:0/3755378416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46b0102740 msgr2=0x7f46b0102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:16.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.667+0000 7f46b5960700 1 --2- 172.21.15.37:0/3755378416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46b0102740 0x7f46b0102b20 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f46a00099c0 tx=0x7f46a0009cd0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.667+0000 7f46b5960700 1 -- 172.21.15.37:0/3755378416 shutdown_connections 2024-06-10T01:07:16.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.667+0000 7f46b5960700 1 --2- 172.21.15.37:0/3755378416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46b0107840 0x7f46b010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.667+0000 7f46b5960700 1 --2- 172.21.15.37:0/3755378416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46b0102740 0x7f46b0102b20 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.667+0000 7f46b5960700 1 --2- 172.21.15.37:0/3755378416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46b01030f0 0x7f46b0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.667+0000 7f46b5960700 1 -- 172.21.15.37:0/3755378416 >> 172.21.15.37:0/3755378416 conn(0x7f46b00795f0 msgr2=0x7f46b0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:16.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.667+0000 7f46b5960700 1 -- 172.21.15.37:0/3755378416 shutdown_connections 2024-06-10T01:07:16.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.667+0000 7f46b5960700 1 -- 172.21.15.37:0/3755378416 wait complete. 2024-06-10T01:07:16.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.668+0000 7f46b5960700 1 Processor -- start 2024-06-10T01:07:16.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.668+0000 7f46b5960700 1 -- start start 2024-06-10T01:07:16.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.668+0000 7f46b5960700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46b0102740 0x7f46b0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.668+0000 7f46b5960700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46b01030f0 0x7f46b0074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.668+0000 7f46b5960700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46b0107840 0x7f46b0075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.668+0000 7f46b5960700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f46b0112760 con 0x7f46b0102740 2024-06-10T01:07:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.668+0000 7f46b5960700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f46b01125e0 con 0x7f46b01030f0 2024-06-10T01:07:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.668+0000 7f46b5960700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f46b01128e0 con 0x7f46b0107840 2024-06-10T01:07:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.668+0000 7f46aeffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46b0102740 0x7f46b0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.668+0000 7f46aeffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46b0102740 0x7f46b0074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57898/0 (socket says 172.21.15.37:57898) 2024-06-10T01:07:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.668+0000 7f46aeffd700 1 -- 172.21.15.37:0/4168504030 learned_addr learned my addr 172.21.15.37:0/4168504030 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.668+0000 7f46af7fe700 1 --2- 172.21.15.37:0/4168504030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46b0107840 0x7f46b0075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.668+0000 7f46ae7fc700 1 --2- 172.21.15.37:0/4168504030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46b01030f0 0x7f46b0074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.669+0000 7f46aeffd700 1 -- 172.21.15.37:0/4168504030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46b0107840 msgr2=0x7f46b0075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.669+0000 7f46aeffd700 1 --2- 172.21.15.37:0/4168504030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46b0107840 0x7f46b0075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.669+0000 7f46aeffd700 1 -- 172.21.15.37:0/4168504030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46b01030f0 msgr2=0x7f46b0074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.669+0000 7f46aeffd700 1 --2- 172.21.15.37:0/4168504030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46b01030f0 0x7f46b0074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.669+0000 7f46aeffd700 1 -- 172.21.15.37:0/4168504030 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b01a70c0 con 0x7f46b0102740 2024-06-10T01:07:16.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.669+0000 7f46af7fe700 1 --2- 172.21.15.37:0/4168504030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46b0107840 0x7f46b0075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:16.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.669+0000 7f46aeffd700 1 --2- 172.21.15.37:0/4168504030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46b0102740 0x7f46b0074790 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f46a0006010 tx=0x7f46a000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:16.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.669+0000 7f46b495e700 1 -- 172.21.15.37:0/4168504030 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0003680 con 0x7f46b0102740 2024-06-10T01:07:16.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.669+0000 7f46b495e700 1 -- 172.21.15.37:0/4168504030 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46a00050d0 con 0x7f46b0102740 2024-06-10T01:07:16.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.669+0000 7f46b5960700 1 -- 172.21.15.37:0/4168504030 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b01a7350 con 0x7f46b0102740 2024-06-10T01:07:16.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.669+0000 7f46b495e700 1 -- 172.21.15.37:0/4168504030 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0003db0 con 0x7f46b0102740 2024-06-10T01:07:16.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.669+0000 7f46b5960700 1 -- 172.21.15.37:0/4168504030 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b01a77a0 con 0x7f46b0102740 2024-06-10T01:07:16.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.671+0000 7f46b495e700 1 -- 172.21.15.37:0/4168504030 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f46a0005280 con 0x7f46b0102740 2024-06-10T01:07:16.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.672+0000 7f46b495e700 1 --2- 172.21.15.37:0/4168504030 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f46980a2dc0 0x7f46980a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:16.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.673+0000 7f46ae7fc700 1 --2- 172.21.15.37:0/4168504030 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f46980a2dc0 0x7f46980a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:16.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.673+0000 7f46b495e700 1 -- 172.21.15.37:0/4168504030 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f46a00d00c0 con 0x7f46b0102740 2024-06-10T01:07:16.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.673+0000 7f46ae7fc700 1 --2- 172.21.15.37:0/4168504030 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f46980a2dc0 0x7f46980a5280 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f46b0101f60 tx=0x7f46a4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:16.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.673+0000 7f46c01e3b80 1 Processor -- start 2024-06-10T01:07:16.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.674+0000 7f46c01e3b80 1 -- start start 2024-06-10T01:07:16.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.674+0000 7f46c01e3b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609148a7f60 0x5609148a8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:16.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.674+0000 7f46c01e3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609148a8880 0x5609148a2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:16.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.674+0000 7f46c01e3b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5609148a27a0 0x5609148a2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:16.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.674+0000 7f46c01e3b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560914794f60 con 0x5609148a7f60 2024-06-10T01:07:16.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.674+0000 7f46c01e3b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5609147f3850 con 0x5609148a27a0 2024-06-10T01:07:16.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.674+0000 7f46c01e3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56091475b460 con 0x5609148a8880 2024-06-10T01:07:16.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.674+0000 7f46aeffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609148a7f60 0x5609148a8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:16.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.674+0000 7f46aeffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609148a7f60 0x5609148a8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57910/0 (socket says 172.21.15.37:57910) 2024-06-10T01:07:16.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.674+0000 7f46aeffd700 1 -- 172.21.15.37:0/3776520706 learned_addr learned my addr 172.21.15.37:0/3776520706 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:16.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.674+0000 7f46ae7fc700 1 --2- 172.21.15.37:0/3776520706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609148a8880 0x5609148a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:16.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.674+0000 7f46af7fe700 1 --2- 172.21.15.37:0/3776520706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5609148a27a0 0x5609148a2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:16.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.674+0000 7f46ae7fc700 1 -- 172.21.15.37:0/3776520706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5609148a27a0 msgr2=0x5609148a2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:16.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.674+0000 7f46ae7fc700 1 --2- 172.21.15.37:0/3776520706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5609148a27a0 0x5609148a2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.674+0000 7f46ae7fc700 1 -- 172.21.15.37:0/3776520706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609148a7f60 msgr2=0x5609148a8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:16.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.674+0000 7f46ae7fc700 1 --2- 172.21.15.37:0/3776520706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609148a7f60 0x5609148a8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.674+0000 7f46ae7fc700 1 -- 172.21.15.37:0/3776520706 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609147428d0 con 0x5609148a8880 2024-06-10T01:07:16.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.674+0000 7f46aeffd700 1 --2- 172.21.15.37:0/3776520706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609148a7f60 0x5609148a8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:16.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.674+0000 7f46ae7fc700 1 --2- 172.21.15.37:0/3776520706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609148a8880 0x5609148a2260 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f46a4011c60 tx=0x7f46a4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e3a1a6e585344b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:16.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.675+0000 7f4695ffb700 1 -- 172.21.15.37:0/3776520706 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a4004230 con 0x5609148a8880 2024-06-10T01:07:16.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.675+0000 7f4695ffb700 1 -- 172.21.15.37:0/3776520706 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46a400ed20 con 0x5609148a8880 2024-06-10T01:07:16.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.675+0000 7f4695ffb700 1 -- 172.21.15.37:0/3776520706 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a4003e00 con 0x5609148a8880 2024-06-10T01:07:16.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.675+0000 7f46c01e3b80 1 -- 172.21.15.37:0/3776520706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609148a8880 msgr2=0x5609148a2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:16.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.675+0000 7f46c01e3b80 1 --2- 172.21.15.37:0/3776520706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609148a8880 0x5609148a2260 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f46a4011c60 tx=0x7f46a4014000 comp rx=0 tx=0).stop 2024-06-10T01:07:16.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.675+0000 7f46c01e3b80 1 -- 172.21.15.37:0/3776520706 shutdown_connections 2024-06-10T01:07:16.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.675+0000 7f46c01e3b80 1 --2- 172.21.15.37:0/3776520706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5609148a27a0 0x5609148a2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.675+0000 7f46c01e3b80 1 --2- 172.21.15.37:0/3776520706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609148a7f60 0x5609148a8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.675+0000 7f46c01e3b80 1 --2- 172.21.15.37:0/3776520706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609148a8880 0x5609148a2260 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.676+0000 7f46c01e3b80 1 -- 172.21.15.37:0/3776520706 >> 172.21.15.37:0/3776520706 conn(0x5609147cf080 msgr2=0x56091489d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:16.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.676+0000 7f46c01e3b80 1 -- 172.21.15.37:0/3776520706 shutdown_connections 2024-06-10T01:07:16.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.676+0000 7f46c01e3b80 1 -- 172.21.15.37:0/3776520706 wait complete. 2024-06-10T01:07:16.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.676+0000 7f46c01e3b80 1 -- start start 2024-06-10T01:07:16.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.676+0000 7f46c01e3b80 1 Processor -- start 2024-06-10T01:07:16.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.676+0000 7f46c01e3b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5609148a27a0 0x5609148ae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:16.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.676+0000 7f46c01e3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609148a7f60 0x5609148ab900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:16.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.676+0000 7f46c01e3b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609148a8880 0x5609148abe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:16.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.676+0000 7f46c01e3b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560914793fc0 con 0x5609148a8880 2024-06-10T01:07:16.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.676+0000 7f46c01e3b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5609147ef810 con 0x5609148a27a0 2024-06-10T01:07:16.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.676+0000 7f46c01e3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5609148af260 con 0x5609148a7f60 2024-06-10T01:07:16.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.677+0000 7f46c01e3b80 10 client.0 _ll_register_callbacks cb 0x5609147895b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:16.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.677+0000 7f46af7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609148a8880 0x5609148abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:16.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.677+0000 7f46af7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609148a8880 0x5609148abe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57926/0 (socket says 172.21.15.37:57926) 2024-06-10T01:07:16.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.677+0000 7f46ae7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609148a7f60 0x5609148ab900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:16.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.677+0000 7f46af7fe700 1 -- 172.21.15.37:0/1216610455 learned_addr learned my addr 172.21.15.37:0/1216610455 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:16.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.677+0000 7f46aeffd700 1 --2- 172.21.15.37:0/1216610455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5609148a27a0 0x5609148ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:16.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.677+0000 7f46af7fe700 1 -- 172.21.15.37:0/1216610455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609148a7f60 msgr2=0x5609148ab900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:16.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.677+0000 7f46af7fe700 1 --2- 172.21.15.37:0/1216610455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609148a7f60 0x5609148ab900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.677+0000 7f46af7fe700 1 -- 172.21.15.37:0/1216610455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5609148a27a0 msgr2=0x5609148ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:16.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.677+0000 7f46af7fe700 1 --2- 172.21.15.37:0/1216610455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5609148a27a0 0x5609148ae600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.677+0000 7f46af7fe700 1 -- 172.21.15.37:0/1216610455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609147b9750 con 0x5609148a8880 2024-06-10T01:07:16.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.677+0000 7f46ae7fc700 1 --2- 172.21.15.37:0/1216610455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609148a7f60 0x5609148ab900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:16.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.677+0000 7f46af7fe700 1 --2- 172.21.15.37:0/1216610455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609148a8880 0x5609148abe40 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f46a8003d50 tx=0x7f46a800e7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229ab19129952db5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:16.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.677+0000 7f468effd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:07:16.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.677+0000 7f468effd700 1 -- 172.21.15.37:0/1216610455 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a80092d0 con 0x5609148a8880 2024-06-10T01:07:16.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.677+0000 7f468effd700 1 -- 172.21.15.37:0/1216610455 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f46a8007500 con 0x5609148a8880 2024-06-10T01:07:16.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.677+0000 7f46c01e3b80 1 -- 172.21.15.37:0/1216610455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46a400ed20 con 0x5609148a8880 2024-06-10T01:07:16.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.678+0000 7f468effd700 1 -- 172.21.15.37:0/1216610455 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f46a8027050 con 0x5609148a8880 2024-06-10T01:07:16.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.678+0000 7f46c01e3b80 10 client.17205 fetch_fsmap learned FSMap version 31 2024-06-10T01:07:16.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.678+0000 7f46c01e3b80 1 -- 172.21.15.37:0/1216610455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5609148b9aa0 con 0x5609148a8880 2024-06-10T01:07:16.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.678+0000 7f468effd700 1 -- 172.21.15.37:0/1216610455 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46a400ed20 con 0x5609148a8880 2024-06-10T01:07:16.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.678+0000 7f468effd700 1 -- 172.21.15.37:0/1216610455 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f46a801e660 con 0x5609148a8880 2024-06-10T01:07:16.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.678+0000 7f46c01e3b80 10 client.17205 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:07:16.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.678+0000 7f46c01e3b80 10 client.17205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:16.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.678+0000 7f46c01e3b80 20 client.17205 populate_metadata read hostname 'smithi037' 2024-06-10T01:07:16.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.678+0000 7f46c01e3b80 1 --2- 172.21.15.37:0/1216610455 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5609148bc0e0 0x5609148be4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:16.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.678+0000 7f46c01e3b80 4 client.17205 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:16.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.678+0000 7f46c01e3b80 1 -- 172.21.15.37:0/1216610455 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560914800d50 con 0x5609148bc0e0 2024-06-10T01:07:16.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.678+0000 7f46aeffd700 1 --2- 172.21.15.37:0/1216610455 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5609148bc0e0 0x5609148be4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:16.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.679+0000 7f46aeffd700 1 --2- 172.21.15.37:0/1216610455 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5609148bc0e0 0x5609148be4d0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bec255cc9617851f server_cookie=850d3b86bb4b5c66 in_seq=0 out_seq=0 2024-06-10T01:07:16.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.679+0000 7f468effd700 10 client.17205 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:16.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.683+0000 7f468effd700 1 -- 172.21.15.37:0/1216610455 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560914800d50 con 0x5609148bc0e0 2024-06-10T01:07:16.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.683+0000 7f468effd700 10 client.17205 handle_command_reply: tid=0 2024-06-10T01:07:16.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.683+0000 7f46c01e3b80 1 client.17205 shutdown 2024-06-10T01:07:16.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.684+0000 7f46c01e3b80 1 -- 172.21.15.37:0/1216610455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609148a8880 msgr2=0x5609148abe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:16.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.684+0000 7f46c01e3b80 1 --2- 172.21.15.37:0/1216610455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609148a8880 0x5609148abe40 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f46a8003d50 tx=0x7f46a800e7d0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.684+0000 7f46c01e3b80 1 -- 172.21.15.37:0/1216610455 shutdown_connections 2024-06-10T01:07:16.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.684+0000 7f46c01e3b80 1 --2- 172.21.15.37:0/1216610455 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5609148bc0e0 0x5609148be4d0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.684+0000 7f46c01e3b80 1 --2- 172.21.15.37:0/1216610455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609148a8880 0x5609148abe40 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.684+0000 7f46c01e3b80 1 --2- 172.21.15.37:0/1216610455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5609148a27a0 0x5609148ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.684+0000 7f46c01e3b80 1 --2- 172.21.15.37:0/1216610455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609148a7f60 0x5609148ab900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.684+0000 7f46c01e3b80 1 -- 172.21.15.37:0/1216610455 >> 172.21.15.37:0/1216610455 conn(0x5609147cf080 msgr2=0x56091489c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:16.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.684+0000 7f468effd700 0 client.17205 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:16.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.684+0000 7f46c01e3b80 1 -- 172.21.15.37:0/1216610455 shutdown_connections 2024-06-10T01:07:16.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.684+0000 7f46c01e3b80 1 -- 172.21.15.37:0/1216610455 wait complete. 2024-06-10T01:07:16.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.684+0000 7f46c01e3b80 20 client.17205 trim_cache size 0 max 16384 2024-06-10T01:07:16.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.699+0000 7f46c01e3b80 1 Processor -- start 2024-06-10T01:07:16.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.699+0000 7f46c01e3b80 1 -- start start 2024-06-10T01:07:16.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.699+0000 7f46c01e3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609148a27a0 0x5609148a2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:16.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.699+0000 7f46c01e3b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609148a7f60 0x5609148abe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:16.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.699+0000 7f46c01e3b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5609148a8880 0x5609148ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:16.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.699+0000 7f46c01e3b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f46b01b2e30 con 0x5609148a7f60 2024-06-10T01:07:16.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.699+0000 7f46c01e3b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f46b01b06c0 con 0x5609148a8880 2024-06-10T01:07:16.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.699+0000 7f46c01e3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f46b01ae510 con 0x5609148a27a0 2024-06-10T01:07:16.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.700+0000 7f46ae7fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609148a7f60 0x5609148abe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:16.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.700+0000 7f46ae7fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609148a7f60 0x5609148abe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57940/0 (socket says 172.21.15.37:57940) 2024-06-10T01:07:16.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.700+0000 7f46ae7fc700 1 -- 172.21.15.37:0/601440929 learned_addr learned my addr 172.21.15.37:0/601440929 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:16.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.700+0000 7f46aeffd700 1 --2- 172.21.15.37:0/601440929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609148a27a0 0x5609148a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:16.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.700+0000 7f46af7fe700 1 --2- 172.21.15.37:0/601440929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5609148a8880 0x5609148ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:16.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.700+0000 7f46ae7fc700 1 -- 172.21.15.37:0/601440929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609148a27a0 msgr2=0x5609148a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:16.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.700+0000 7f46ae7fc700 1 --2- 172.21.15.37:0/601440929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609148a27a0 0x5609148a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.700+0000 7f46ae7fc700 1 -- 172.21.15.37:0/601440929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5609148a8880 msgr2=0x5609148ad9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:16.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.700+0000 7f46ae7fc700 1 --2- 172.21.15.37:0/601440929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5609148a8880 0x5609148ad9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.700+0000 7f46ae7fc700 1 -- 172.21.15.37:0/601440929 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560914899810 con 0x5609148a7f60 2024-06-10T01:07:16.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.700+0000 7f46ae7fc700 1 --2- 172.21.15.37:0/601440929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609148a7f60 0x5609148abe70 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f46a4005950 tx=0x7f46a4003810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519535486f66dabd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:16.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.700+0000 7f46957fa700 1 -- 172.21.15.37:0/601440929 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a400ed50 con 0x5609148a7f60 2024-06-10T01:07:16.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.700+0000 7f46957fa700 1 -- 172.21.15.37:0/601440929 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46a40050d0 con 0x5609148a7f60 2024-06-10T01:07:16.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.700+0000 7f46957fa700 1 -- 172.21.15.37:0/601440929 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a4026760 con 0x5609148a7f60 2024-06-10T01:07:16.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.701+0000 7f46c01e3b80 1 -- 172.21.15.37:0/601440929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609148a7f60 msgr2=0x5609148abe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:16.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.701+0000 7f46c01e3b80 1 --2- 172.21.15.37:0/601440929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609148a7f60 0x5609148abe70 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f46a4005950 tx=0x7f46a4003810 comp rx=0 tx=0).stop 2024-06-10T01:07:16.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.701+0000 7f46c01e3b80 1 -- 172.21.15.37:0/601440929 shutdown_connections 2024-06-10T01:07:16.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.701+0000 7f46c01e3b80 1 --2- 172.21.15.37:0/601440929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5609148a8880 0x5609148ad9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.701+0000 7f46c01e3b80 1 --2- 172.21.15.37:0/601440929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609148a27a0 0x5609148a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.701+0000 7f46c01e3b80 1 --2- 172.21.15.37:0/601440929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609148a7f60 0x5609148abe70 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.701+0000 7f46c01e3b80 1 -- 172.21.15.37:0/601440929 >> 172.21.15.37:0/601440929 conn(0x5609147cf080 msgr2=0x5609148a9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:16.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.701+0000 7f46c01e3b80 1 -- 172.21.15.37:0/601440929 shutdown_connections 2024-06-10T01:07:16.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.701+0000 7f46c01e3b80 1 -- 172.21.15.37:0/601440929 wait complete. 2024-06-10T01:07:16.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.701+0000 7f46c01e3b80 1 -- start start 2024-06-10T01:07:16.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.702+0000 7f46c01e3b80 1 Processor -- start 2024-06-10T01:07:16.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.702+0000 7f46c01e3b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5609148a27a0 0x5609148c3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:16.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.702+0000 7f46c01e3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609148a7f60 0x5609148c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:16.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.702+0000 7f46c01e3b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609148a8880 0x5609148c4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:16.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.702+0000 7f46c01e3b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f46b01aeb80 con 0x5609148a8880 2024-06-10T01:07:16.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.702+0000 7f46c01e3b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f46b01afff0 con 0x5609148a27a0 2024-06-10T01:07:16.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.702+0000 7f46c01e3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f46b01af250 con 0x5609148a7f60 2024-06-10T01:07:16.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.702+0000 7f46c01e3b80 10 client.0 _ll_register_callbacks cb 0x5609147895b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:16.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.702+0000 7f46af7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609148a8880 0x5609148c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:16.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.702+0000 7f46ae7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609148a7f60 0x5609148c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:16.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.702+0000 7f46ae7fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609148a7f60 0x5609148c3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:55170/0 (socket says 172.21.15.37:55170) 2024-06-10T01:07:16.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.702+0000 7f46ae7fc700 1 -- 172.21.15.37:0/3808356223 learned_addr learned my addr 172.21.15.37:0/3808356223 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:16.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.702+0000 7f46aeffd700 1 --2- 172.21.15.37:0/3808356223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5609148a27a0 0x5609148c3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:16.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.702+0000 7f46ae7fc700 1 -- 172.21.15.37:0/3808356223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5609148a27a0 msgr2=0x5609148c3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:16.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.703+0000 7f46ae7fc700 1 --2- 172.21.15.37:0/3808356223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5609148a27a0 0x5609148c3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.703+0000 7f46ae7fc700 1 -- 172.21.15.37:0/3808356223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609148a8880 msgr2=0x5609148c4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:16.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.703+0000 7f46ae7fc700 1 --2- 172.21.15.37:0/3808356223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609148a8880 0x5609148c4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.703+0000 7f46ae7fc700 1 -- 172.21.15.37:0/3808356223 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560914925e20 con 0x5609148a7f60 2024-06-10T01:07:16.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.703+0000 7f46aeffd700 1 --2- 172.21.15.37:0/3808356223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5609148a27a0 0x5609148c3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:16.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.703+0000 7f46af7fe700 1 --2- 172.21.15.37:0/3808356223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609148a8880 0x5609148c4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:16.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.703+0000 7f46ae7fc700 1 --2- 172.21.15.37:0/3808356223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609148a7f60 0x5609148c3ff0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f46a4011c30 tx=0x7f46a4003710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daf139823a2bd84b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:16.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.703+0000 7f468effd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:07:16.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.703+0000 7f468effd700 1 -- 172.21.15.37:0/3808356223 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a4005490 con 0x5609148a7f60 2024-06-10T01:07:16.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.703+0000 7f46c01e3b80 1 -- 172.21.15.37:0/3808356223 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46a40050d0 con 0x5609148a7f60 2024-06-10T01:07:16.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.703+0000 7f468effd700 1 -- 172.21.15.37:0/3808356223 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f46a4005630 con 0x5609148a7f60 2024-06-10T01:07:16.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.703+0000 7f468effd700 1 -- 172.21.15.37:0/3808356223 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f46a4003900 con 0x5609148a7f60 2024-06-10T01:07:16.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.703+0000 7f46c01e3b80 10 client.26734 fetch_fsmap learned FSMap version 31 2024-06-10T01:07:16.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.703+0000 7f46c01e3b80 1 -- 172.21.15.37:0/3808356223 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560914927100 con 0x5609148a7f60 2024-06-10T01:07:16.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.703+0000 7f468effd700 1 -- 172.21.15.37:0/3808356223 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46a40050d0 con 0x5609148a7f60 2024-06-10T01:07:16.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.704+0000 7f468effd700 1 -- 172.21.15.37:0/3808356223 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f46a4026c50 con 0x5609148a7f60 2024-06-10T01:07:16.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.704+0000 7f46c01e3b80 10 client.26734 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:07:16.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.704+0000 7f46c01e3b80 10 client.26734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:16.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.704+0000 7f46c01e3b80 20 client.26734 populate_metadata read hostname 'smithi037' 2024-06-10T01:07:16.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.704+0000 7f46c01e3b80 1 --2- 172.21.15.37:0/3808356223 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560914927420 0x560914929840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:16.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.704+0000 7f46c01e3b80 4 client.26734 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:07:16.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.704+0000 7f46c01e3b80 1 -- 172.21.15.37:0/3808356223 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5609147fb2d0 con 0x560914927420 2024-06-10T01:07:16.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.704+0000 7f46aeffd700 1 --2- 172.21.15.37:0/3808356223 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560914927420 0x560914929840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:16.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.704+0000 7f46aeffd700 1 --2- 172.21.15.37:0/3808356223 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560914927420 0x560914929840 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d91bf3280fa78db server_cookie=1a949785b87dbba7 in_seq=0 out_seq=0 2024-06-10T01:07:16.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.704+0000 7f468effd700 10 client.26734 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:16.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.711+0000 7f468effd700 1 -- 172.21.15.37:0/3808356223 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5609147fb2d0 con 0x560914927420 2024-06-10T01:07:16.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.711+0000 7f468effd700 10 client.26734 handle_command_reply: tid=0 2024-06-10T01:07:16.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.711+0000 7f46c01e3b80 1 client.26734 shutdown 2024-06-10T01:07:16.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.712+0000 7f46c01e3b80 1 -- 172.21.15.37:0/3808356223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609148a7f60 msgr2=0x5609148c3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:16.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.712+0000 7f46c01e3b80 1 --2- 172.21.15.37:0/3808356223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609148a7f60 0x5609148c3ff0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f46a4011c30 tx=0x7f46a4003710 comp rx=0 tx=0).stop 2024-06-10T01:07:16.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.712+0000 7f46c01e3b80 1 -- 172.21.15.37:0/3808356223 shutdown_connections 2024-06-10T01:07:16.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.712+0000 7f46c01e3b80 1 --2- 172.21.15.37:0/3808356223 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560914927420 0x560914929840 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.712+0000 7f46c01e3b80 1 --2- 172.21.15.37:0/3808356223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5609148a8880 0x5609148c4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.712+0000 7f468effd700 0 client.26734 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:16.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.712+0000 7f46c01e3b80 1 --2- 172.21.15.37:0/3808356223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5609148a27a0 0x5609148c3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.712+0000 7f46c01e3b80 1 --2- 172.21.15.37:0/3808356223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609148a7f60 0x5609148c3ff0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.712+0000 7f46c01e3b80 1 -- 172.21.15.37:0/3808356223 >> 172.21.15.37:0/3808356223 conn(0x5609147cf080 msgr2=0x5609148a72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:16.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.712+0000 7f46c01e3b80 1 -- 172.21.15.37:0/3808356223 shutdown_connections 2024-06-10T01:07:16.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.712+0000 7f46c01e3b80 1 -- 172.21.15.37:0/3808356223 wait complete. 2024-06-10T01:07:16.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.712+0000 7f46c01e3b80 20 client.26734 trim_cache size 0 max 16384 2024-06-10T01:07:16.712 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T01:07:16.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.713+0000 7f46b5960700 1 -- 172.21.15.37:0/4168504030 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f46980a2dc0 msgr2=0x7f46980a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:16.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.713+0000 7f46b5960700 1 --2- 172.21.15.37:0/4168504030 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f46980a2dc0 0x7f46980a5280 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f46b0101f60 tx=0x7f46a4009260 comp rx=0 tx=0).stop 2024-06-10T01:07:16.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.713+0000 7f46b5960700 1 -- 172.21.15.37:0/4168504030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46b0102740 msgr2=0x7f46b0074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:16.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.713+0000 7f46b5960700 1 --2- 172.21.15.37:0/4168504030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46b0102740 0x7f46b0074790 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f46a0006010 tx=0x7f46a000f690 comp rx=0 tx=0).stop 2024-06-10T01:07:16.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.713+0000 7f46b5960700 1 -- 172.21.15.37:0/4168504030 shutdown_connections 2024-06-10T01:07:16.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.713+0000 7f46b5960700 1 --2- 172.21.15.37:0/4168504030 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f46980a2dc0 0x7f46980a5280 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.713+0000 7f46b5960700 1 --2- 172.21.15.37:0/4168504030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46b0107840 0x7f46b0075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.713+0000 7f46b5960700 1 --2- 172.21.15.37:0/4168504030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46b0102740 0x7f46b0074790 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.713+0000 7f46b5960700 1 --2- 172.21.15.37:0/4168504030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46b01030f0 0x7f46b0074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:16.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.713+0000 7f46b5960700 1 -- 172.21.15.37:0/4168504030 >> 172.21.15.37:0/4168504030 conn(0x7f46b00795f0 msgr2=0x7f46b0108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:16.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.713+0000 7f46b5960700 1 -- 172.21.15.37:0/4168504030 shutdown_connections 2024-06-10T01:07:16.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:16.713+0000 7f46b5960700 1 -- 172.21.15.37:0/4168504030 wait complete. 2024-06-10T01:07:18.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:18 smithi037 ceph-mon[32192]: pgmap v749: 129 pgs: 129 active+clean; 19 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 395 op/s 2024-06-10T01:07:18.522 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:07:18.523 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:07:18.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:18 smithi167 ceph-mon[36917]: pgmap v749: 129 pgs: 129 active+clean; 19 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 395 op/s 2024-06-10T01:07:18.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:18 smithi094 ceph-mon[36708]: pgmap v749: 129 pgs: 129 active+clean; 19 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 395 op/s 2024-06-10T01:07:18.893 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:20.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:20 smithi037 ceph-mon[32192]: pgmap v750: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 669 op/s 2024-06-10T01:07:20.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:20 smithi167 ceph-mon[36917]: pgmap v750: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 669 op/s 2024-06-10T01:07:20.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:20 smithi094 ceph-mon[36708]: pgmap v750: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 669 op/s 2024-06-10T01:07:21.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:21.446+0000 7f08c3fff640 1 -- 172.21.15.37:0/2912096407 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55cae99e1c20 con 0x55cae9b02c60 2024-06-10T01:07:21.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:21.446+0000 7f08c3fff640 10 client.26564 handle_command_reply: tid=0 2024-06-10T01:07:21.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:21.446+0000 7f08ed9b9740 1 client.26564 shutdown 2024-06-10T01:07:21.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:21.446+0000 7f08ed9b9740 1 -- 172.21.15.37:0/2912096407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cae9a71a00 msgr2=0x55cae9a7f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:21.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:21.446+0000 7f08ed9b9740 1 --2- 172.21.15.37:0/2912096407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cae9a71a00 0x55cae9a7f050 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f08d80096c0 tx=0x7f08d8000bc0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:21.446+0000 7f08ed9b9740 1 -- 172.21.15.37:0/2912096407 shutdown_connections 2024-06-10T01:07:21.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:21.446+0000 7f08ed9b9740 1 --2- 172.21.15.37:0/2912096407 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55cae9b02c60 0x55cae9b05080 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:21.446+0000 7f08ed9b9740 1 --2- 172.21.15.37:0/2912096407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cae9a76b70 0x55cae9a7f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:21.446+0000 7f08ed9b9740 1 --2- 172.21.15.37:0/2912096407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cae9a71a00 0x55cae9a7f050 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:21.446+0000 7f08c3fff640 0 client.26564 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:21.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:21.446+0000 7f08ed9b9740 1 --2- 172.21.15.37:0/2912096407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cae9a71010 0x55cae9a82540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:21.446+0000 7f08ed9b9740 1 -- 172.21.15.37:0/2912096407 >> 172.21.15.37:0/2912096407 conn(0x55cae9a6c680 msgr2=0x55cae9a6ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:21.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:21.446+0000 7f08ed9b9740 1 -- 172.21.15.37:0/2912096407 shutdown_connections 2024-06-10T01:07:21.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:21.446+0000 7f08ed9b9740 1 -- 172.21.15.37:0/2912096407 wait complete. 2024-06-10T01:07:21.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:21.446+0000 7f08ed9b9740 20 client.26564 trim_cache size 0 max 16384 2024-06-10T01:07:21.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:07:21.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:07:21.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:07:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:21.447+0000 7f08eaca5640 1 -- 172.21.15.37:0/1853080265 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f08d40a2dd0 msgr2=0x7f08d40a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:21.447+0000 7f08eaca5640 1 --2- 172.21.15.37:0/1853080265 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f08d40a2dd0 0x7f08d40a5290 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f08dc0049d0 tx=0x7f08dc005b40 comp rx=0 tx=0).stop 2024-06-10T01:07:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:21.447+0000 7f08eaca5640 1 -- 172.21.15.37:0/1853080265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08e410ed60 msgr2=0x7f08e41ae050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:21.447+0000 7f08eaca5640 1 --2- 172.21.15.37:0/1853080265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08e410ed60 0x7f08e41ae050 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00eb10 tx=0x7f08dc00c2d0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:21.447+0000 7f08eaca5640 1 -- 172.21.15.37:0/1853080265 shutdown_connections 2024-06-10T01:07:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:21.447+0000 7f08eaca5640 1 --2- 172.21.15.37:0/1853080265 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f08d40a2dd0 0x7f08d40a5290 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:21.447+0000 7f08eaca5640 1 --2- 172.21.15.37:0/1853080265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08e410ed60 0x7f08e41ae050 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:21.447+0000 7f08eaca5640 1 --2- 172.21.15.37:0/1853080265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08e410c3e0 0x7f08e41a6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:21.447+0000 7f08eaca5640 1 --2- 172.21.15.37:0/1853080265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08e4109ab0 0x7f08e41a6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:21.447+0000 7f08eaca5640 1 -- 172.21.15.37:0/1853080265 >> 172.21.15.37:0/1853080265 conn(0x7f08e406f010 msgr2=0x7f08e410cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:21.447+0000 7f08eaca5640 1 -- 172.21.15.37:0/1853080265 shutdown_connections 2024-06-10T01:07:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:21.447+0000 7f08eaca5640 1 -- 172.21.15.37:0/1853080265 wait complete. 2024-06-10T01:07:21.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.809+0000 7f2e79c62700 1 -- 172.21.15.37:0/2481482303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e741073d0 msgr2=0x7f2e741077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:21.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.809+0000 7f2e79c62700 1 --2- 172.21.15.37:0/2481482303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e741073d0 0x7f2e741077b0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f2e64009990 tx=0x7f2e64009ca0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.809+0000 7f2e79c62700 1 -- 172.21.15.37:0/2481482303 shutdown_connections 2024-06-10T01:07:21.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.809+0000 7f2e79c62700 1 --2- 172.21.15.37:0/2481482303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e74101d30 0x7f2e7410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.809+0000 7f2e79c62700 1 --2- 172.21.15.37:0/2481482303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e741073d0 0x7f2e741077b0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.809+0000 7f2e79c62700 1 --2- 172.21.15.37:0/2481482303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e74101390 0x7f2e741017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.809+0000 7f2e79c62700 1 -- 172.21.15.37:0/2481482303 >> 172.21.15.37:0/2481482303 conn(0x7f2e740782c0 msgr2=0x7f2e740786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:21.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.809+0000 7f2e79c62700 1 -- 172.21.15.37:0/2481482303 shutdown_connections 2024-06-10T01:07:21.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.809+0000 7f2e79c62700 1 -- 172.21.15.37:0/2481482303 wait complete. 2024-06-10T01:07:21.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.809+0000 7f2e79c62700 1 Processor -- start 2024-06-10T01:07:21.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.810+0000 7f2e79c62700 1 -- start start 2024-06-10T01:07:21.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.810+0000 7f2e79c62700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e74101390 0x7f2e74103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:21.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.810+0000 7f2e79c62700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e74101d30 0x7f2e74103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:21.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.810+0000 7f2e79c62700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e741073d0 0x7f2e74103ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:21.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.810+0000 7f2e79c62700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2e74111680 con 0x7f2e74101390 2024-06-10T01:07:21.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.810+0000 7f2e737fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e74101390 0x7f2e74103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:21.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.810+0000 7f2e737fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e74101390 0x7f2e74103040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60394/0 (socket says 172.21.15.37:60394) 2024-06-10T01:07:21.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.810+0000 7f2e737fe700 1 -- 172.21.15.37:0/1697265450 learned_addr learned my addr 172.21.15.37:0/1697265450 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:21.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.810+0000 7f2e72ffd700 1 --2- 172.21.15.37:0/1697265450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e74101d30 0x7f2e74103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:21.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.810+0000 7f2e79c62700 1 -- 172.21.15.37:0/1697265450 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2e74111500 con 0x7f2e74101d30 2024-06-10T01:07:21.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.810+0000 7f2e79c62700 1 -- 172.21.15.37:0/1697265450 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2e74111800 con 0x7f2e741073d0 2024-06-10T01:07:21.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.810+0000 7f2e73fff700 1 --2- 172.21.15.37:0/1697265450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e741073d0 0x7f2e74103ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:21.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.810+0000 7f2e737fe700 1 -- 172.21.15.37:0/1697265450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e741073d0 msgr2=0x7f2e74103ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:21.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.810+0000 7f2e737fe700 1 --2- 172.21.15.37:0/1697265450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e741073d0 0x7f2e74103ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.810+0000 7f2e737fe700 1 -- 172.21.15.37:0/1697265450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e74101d30 msgr2=0x7f2e74103580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:21.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.810+0000 7f2e737fe700 1 --2- 172.21.15.37:0/1697265450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e74101d30 0x7f2e74103580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.810+0000 7f2e737fe700 1 -- 172.21.15.37:0/1697265450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e741a7150 con 0x7f2e74101390 2024-06-10T01:07:21.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.810+0000 7f2e72ffd700 1 --2- 172.21.15.37:0/1697265450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e74101d30 0x7f2e74103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:21.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.811+0000 7f2e737fe700 1 --2- 172.21.15.37:0/1697265450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e74101390 0x7f2e74103040 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f2e64005f50 tx=0x7f2e640047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:21.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.811+0000 7f2e70ff9700 1 -- 172.21.15.37:0/1697265450 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e64027070 con 0x7f2e74101390 2024-06-10T01:07:21.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.811+0000 7f2e70ff9700 1 -- 172.21.15.37:0/1697265450 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e6400fde0 con 0x7f2e74101390 2024-06-10T01:07:21.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.812+0000 7f2e70ff9700 1 -- 172.21.15.37:0/1697265450 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e640173f0 con 0x7f2e74101390 2024-06-10T01:07:21.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.812+0000 7f2e79c62700 1 -- 172.21.15.37:0/1697265450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e741a73e0 con 0x7f2e74101390 2024-06-10T01:07:21.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.812+0000 7f2e79c62700 1 -- 172.21.15.37:0/1697265450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e741a77e0 con 0x7f2e74101390 2024-06-10T01:07:21.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.814+0000 7f2e70ff9700 1 -- 172.21.15.37:0/1697265450 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2e6402a070 con 0x7f2e74101390 2024-06-10T01:07:21.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.815+0000 7f2e70ff9700 1 --2- 172.21.15.37:0/1697265450 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2e5c0a2c20 0x7f2e5c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:21.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.815+0000 7f2e72ffd700 1 --2- 172.21.15.37:0/1697265450 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2e5c0a2c20 0x7f2e5c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:21.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.815+0000 7f2e70ff9700 1 -- 172.21.15.37:0/1697265450 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f2e640d01a0 con 0x7f2e74101390 2024-06-10T01:07:21.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.815+0000 7f2e72ffd700 1 --2- 172.21.15.37:0/1697265450 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2e5c0a2c20 0x7f2e5c0a50e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2e74078830 tx=0x7f2e68009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:21.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.816+0000 7f2e844e5b80 1 Processor -- start 2024-06-10T01:07:21.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.816+0000 7f2e844e5b80 1 -- start start 2024-06-10T01:07:21.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.816+0000 7f2e844e5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d0cf01fe0 0x561d0cf023c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:21.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.816+0000 7f2e844e5b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d0cf02900 0x561d0cefc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:21.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.816+0000 7f2e844e5b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d0cefc820 0x561d0cefcc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:21.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.816+0000 7f2e844e5b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561d0cdeefe0 con 0x561d0cf02900 2024-06-10T01:07:21.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.816+0000 7f2e844e5b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561d0ce4d8d0 con 0x561d0cefc820 2024-06-10T01:07:21.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.816+0000 7f2e844e5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561d0cdb54e0 con 0x561d0cf01fe0 2024-06-10T01:07:21.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.816+0000 7f2e737fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d0cf01fe0 0x561d0cf023c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:21.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.816+0000 7f2e737fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d0cf01fe0 0x561d0cf023c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:39312/0 (socket says 172.21.15.37:39312) 2024-06-10T01:07:21.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.816+0000 7f2e737fe700 1 -- 172.21.15.37:0/1030402363 learned_addr learned my addr 172.21.15.37:0/1030402363 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:21.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.816+0000 7f2e72ffd700 1 --2- 172.21.15.37:0/1030402363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d0cf02900 0x561d0cefc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:21.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.816+0000 7f2e73fff700 1 --2- 172.21.15.37:0/1030402363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d0cefc820 0x561d0cefcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:21.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.817+0000 7f2e737fe700 1 -- 172.21.15.37:0/1030402363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d0cefc820 msgr2=0x561d0cefcc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:21.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.817+0000 7f2e737fe700 1 --2- 172.21.15.37:0/1030402363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d0cefc820 0x561d0cefcc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.817+0000 7f2e737fe700 1 -- 172.21.15.37:0/1030402363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d0cf02900 msgr2=0x561d0cefc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:21.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.817+0000 7f2e737fe700 1 --2- 172.21.15.37:0/1030402363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d0cf02900 0x561d0cefc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.817+0000 7f2e737fe700 1 -- 172.21.15.37:0/1030402363 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d0cd9c950 con 0x561d0cf01fe0 2024-06-10T01:07:21.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.817+0000 7f2e73fff700 1 --2- 172.21.15.37:0/1030402363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d0cefc820 0x561d0cefcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:07:21.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.817+0000 7f2e72ffd700 1 --2- 172.21.15.37:0/1030402363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d0cf02900 0x561d0cefc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:21.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.817+0000 7f2e737fe700 1 --2- 172.21.15.37:0/1030402363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d0cf01fe0 0x561d0cf023c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2e6409c6c0 tx=0x7f2e64009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4404a7468af2ba3f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:21.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.817+0000 7f2e59ffb700 1 -- 172.21.15.37:0/1030402363 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6409ce30 con 0x561d0cf01fe0 2024-06-10T01:07:21.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.817+0000 7f2e59ffb700 1 -- 172.21.15.37:0/1030402363 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e6409c730 con 0x561d0cf01fe0 2024-06-10T01:07:21.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.817+0000 7f2e59ffb700 1 -- 172.21.15.37:0/1030402363 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6409c990 con 0x561d0cf01fe0 2024-06-10T01:07:21.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.818+0000 7f2e844e5b80 1 -- 172.21.15.37:0/1030402363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d0cf01fe0 msgr2=0x561d0cf023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:21.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.818+0000 7f2e844e5b80 1 --2- 172.21.15.37:0/1030402363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d0cf01fe0 0x561d0cf023c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2e6409c6c0 tx=0x7f2e64009ca0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.818+0000 7f2e844e5b80 1 -- 172.21.15.37:0/1030402363 shutdown_connections 2024-06-10T01:07:21.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.818+0000 7f2e844e5b80 1 --2- 172.21.15.37:0/1030402363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d0cefc820 0x561d0cefcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.818+0000 7f2e844e5b80 1 --2- 172.21.15.37:0/1030402363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d0cf01fe0 0x561d0cf023c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.818+0000 7f2e844e5b80 1 --2- 172.21.15.37:0/1030402363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d0cf02900 0x561d0cefc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.818+0000 7f2e844e5b80 1 -- 172.21.15.37:0/1030402363 >> 172.21.15.37:0/1030402363 conn(0x561d0ce29100 msgr2=0x561d0cef72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:21.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.818+0000 7f2e844e5b80 1 -- 172.21.15.37:0/1030402363 shutdown_connections 2024-06-10T01:07:21.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.818+0000 7f2e844e5b80 1 -- 172.21.15.37:0/1030402363 wait complete. 2024-06-10T01:07:21.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.818+0000 7f2e844e5b80 1 -- start start 2024-06-10T01:07:21.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.819+0000 7f2e844e5b80 1 Processor -- start 2024-06-10T01:07:21.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.819+0000 7f2e844e5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d0cefc820 0x561d0cf08670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:21.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.819+0000 7f2e844e5b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d0cf01fe0 0x561d0cf05970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:21.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.819+0000 7f2e844e5b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d0cf02900 0x561d0cf05eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:21.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.819+0000 7f2e844e5b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561d0cdee040 con 0x561d0cf02900 2024-06-10T01:07:21.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.819+0000 7f2e844e5b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561d0ce49890 con 0x561d0cf01fe0 2024-06-10T01:07:21.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.819+0000 7f2e844e5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561d0cf09090 con 0x561d0cefc820 2024-06-10T01:07:21.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.819+0000 7f2e844e5b80 10 client.0 _ll_register_callbacks cb 0x561d0cde3630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:21.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.819+0000 7f2e737fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d0cefc820 0x561d0cf08670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:21.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.819+0000 7f2e737fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d0cefc820 0x561d0cf08670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:39314/0 (socket says 172.21.15.37:39314) 2024-06-10T01:07:21.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.819+0000 7f2e737fe700 1 -- 172.21.15.37:0/1518511413 learned_addr learned my addr 172.21.15.37:0/1518511413 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:21.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.819+0000 7f2e73fff700 1 --2- 172.21.15.37:0/1518511413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d0cf02900 0x561d0cf05eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:21.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.819+0000 7f2e72ffd700 1 --2- 172.21.15.37:0/1518511413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d0cf01fe0 0x561d0cf05970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:21.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.819+0000 7f2e737fe700 1 -- 172.21.15.37:0/1518511413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d0cf01fe0 msgr2=0x561d0cf05970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:21.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.819+0000 7f2e737fe700 1 --2- 172.21.15.37:0/1518511413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d0cf01fe0 0x561d0cf05970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.819+0000 7f2e737fe700 1 -- 172.21.15.37:0/1518511413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d0cf02900 msgr2=0x561d0cf05eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:21.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.819+0000 7f2e737fe700 1 --2- 172.21.15.37:0/1518511413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d0cf02900 0x561d0cf05eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.819+0000 7f2e737fe700 1 -- 172.21.15.37:0/1518511413 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d0ce137d0 con 0x561d0cefc820 2024-06-10T01:07:21.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.820+0000 7f2e72ffd700 1 --2- 172.21.15.37:0/1518511413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d0cf01fe0 0x561d0cf05970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:21.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.820+0000 7f2e737fe700 1 --2- 172.21.15.37:0/1518511413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d0cefc820 0x561d0cf08670 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2e6400c010 tx=0x7f2e6400b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=832f3e4f188f942 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:21.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.820+0000 7f2e52ffd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:07:21.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.820+0000 7f2e52ffd700 1 -- 172.21.15.37:0/1518511413 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e640b4030 con 0x561d0cefc820 2024-06-10T01:07:21.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.820+0000 7f2e844e5b80 1 -- 172.21.15.37:0/1518511413 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e6409c730 con 0x561d0cefc820 2024-06-10T01:07:21.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.820+0000 7f2e52ffd700 1 -- 172.21.15.37:0/1518511413 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f2e64022470 con 0x561d0cefc820 2024-06-10T01:07:21.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.820+0000 7f2e52ffd700 1 -- 172.21.15.37:0/1518511413 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2e6409ce30 con 0x561d0cefc820 2024-06-10T01:07:21.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.820+0000 7f2e844e5b80 10 client.26746 fetch_fsmap learned FSMap version 31 2024-06-10T01:07:21.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.820+0000 7f2e844e5b80 1 -- 172.21.15.37:0/1518511413 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561d0cf136d0 con 0x561d0cefc820 2024-06-10T01:07:21.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.820+0000 7f2e52ffd700 1 -- 172.21.15.37:0/1518511413 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e6409c730 con 0x561d0cefc820 2024-06-10T01:07:21.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.820+0000 7f2e52ffd700 1 -- 172.21.15.37:0/1518511413 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f2e640b0bc0 con 0x561d0cefc820 2024-06-10T01:07:21.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.820+0000 7f2e844e5b80 10 client.26746 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:07:21.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.820+0000 7f2e844e5b80 10 client.26746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:21.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.820+0000 7f2e844e5b80 20 client.26746 populate_metadata read hostname 'smithi037' 2024-06-10T01:07:21.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.821+0000 7f2e844e5b80 1 --2- 172.21.15.37:0/1518511413 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561d0cf15ba0 0x561d0cf17f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:21.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.821+0000 7f2e844e5b80 4 client.26746 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:21.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.821+0000 7f2e844e5b80 1 -- 172.21.15.37:0/1518511413 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d0ce5add0 con 0x561d0cf15ba0 2024-06-10T01:07:21.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.821+0000 7f2e72ffd700 1 --2- 172.21.15.37:0/1518511413 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561d0cf15ba0 0x561d0cf17f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:21.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.821+0000 7f2e52ffd700 10 client.26746 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:21.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.821+0000 7f2e72ffd700 1 --2- 172.21.15.37:0/1518511413 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561d0cf15ba0 0x561d0cf17f90 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbd80570d73d3389 server_cookie=7ee8dc15d825c1 in_seq=0 out_seq=0 2024-06-10T01:07:21.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.824+0000 7f2e52ffd700 1 -- 172.21.15.37:0/1518511413 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561d0ce5add0 con 0x561d0cf15ba0 2024-06-10T01:07:21.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.824+0000 7f2e52ffd700 10 client.26746 handle_command_reply: tid=0 2024-06-10T01:07:21.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.824+0000 7f2e844e5b80 1 client.26746 shutdown 2024-06-10T01:07:21.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.824+0000 7f2e844e5b80 1 -- 172.21.15.37:0/1518511413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d0cefc820 msgr2=0x561d0cf08670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:21.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.824+0000 7f2e844e5b80 1 --2- 172.21.15.37:0/1518511413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d0cefc820 0x561d0cf08670 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2e6400c010 tx=0x7f2e6400b820 comp rx=0 tx=0).stop 2024-06-10T01:07:21.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.824+0000 7f2e844e5b80 1 -- 172.21.15.37:0/1518511413 shutdown_connections 2024-06-10T01:07:21.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.824+0000 7f2e844e5b80 1 --2- 172.21.15.37:0/1518511413 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561d0cf15ba0 0x561d0cf17f90 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.824+0000 7f2e52ffd700 0 client.26746 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:21.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.824+0000 7f2e844e5b80 1 --2- 172.21.15.37:0/1518511413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d0cf02900 0x561d0cf05eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.824+0000 7f2e844e5b80 1 --2- 172.21.15.37:0/1518511413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d0cefc820 0x561d0cf08670 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.824+0000 7f2e844e5b80 1 --2- 172.21.15.37:0/1518511413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d0cf01fe0 0x561d0cf05970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.824+0000 7f2e844e5b80 1 -- 172.21.15.37:0/1518511413 >> 172.21.15.37:0/1518511413 conn(0x561d0ce29100 msgr2=0x561d0cef6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:21.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.824+0000 7f2e844e5b80 1 -- 172.21.15.37:0/1518511413 shutdown_connections 2024-06-10T01:07:21.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.825+0000 7f2e844e5b80 1 -- 172.21.15.37:0/1518511413 wait complete. 2024-06-10T01:07:21.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.825+0000 7f2e844e5b80 20 client.26746 trim_cache size 0 max 16384 2024-06-10T01:07:21.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.846+0000 7f2e844e5b80 1 Processor -- start 2024-06-10T01:07:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.846+0000 7f2e844e5b80 1 -- start start 2024-06-10T01:07:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.846+0000 7f2e844e5b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d0cefc820 0x561d0cf081b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.846+0000 7f2e844e5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d0cf01fe0 0x561d0cef6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.847+0000 7f2e844e5b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d0cf02900 0x561d0cef7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.847+0000 7f2e844e5b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2e741b2e20 con 0x561d0cefc820 2024-06-10T01:07:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.847+0000 7f2e844e5b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2e741b06b0 con 0x561d0cf02900 2024-06-10T01:07:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.847+0000 7f2e844e5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2e741ae500 con 0x561d0cf01fe0 2024-06-10T01:07:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.847+0000 7f2e737fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d0cefc820 0x561d0cf081b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.847+0000 7f2e737fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d0cefc820 0x561d0cf081b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60412/0 (socket says 172.21.15.37:60412) 2024-06-10T01:07:21.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.847+0000 7f2e737fe700 1 -- 172.21.15.37:0/1928924915 learned_addr learned my addr 172.21.15.37:0/1928924915 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:21.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.847+0000 7f2e72ffd700 1 --2- 172.21.15.37:0/1928924915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d0cf01fe0 0x561d0cef6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:21.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.847+0000 7f2e73fff700 1 --2- 172.21.15.37:0/1928924915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d0cf02900 0x561d0cef7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:21.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.847+0000 7f2e737fe700 1 -- 172.21.15.37:0/1928924915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d0cf01fe0 msgr2=0x561d0cef6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:21.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.847+0000 7f2e737fe700 1 --2- 172.21.15.37:0/1928924915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d0cf01fe0 0x561d0cef6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.847+0000 7f2e737fe700 1 -- 172.21.15.37:0/1928924915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d0cf02900 msgr2=0x561d0cef7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:21.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.847+0000 7f2e737fe700 1 --2- 172.21.15.37:0/1928924915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d0cf02900 0x561d0cef7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.847+0000 7f2e737fe700 1 -- 172.21.15.37:0/1928924915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d0cefc2e0 con 0x561d0cefc820 2024-06-10T01:07:21.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.847+0000 7f2e72ffd700 1 --2- 172.21.15.37:0/1928924915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d0cf01fe0 0x561d0cef6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:21.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.847+0000 7f2e737fe700 1 --2- 172.21.15.37:0/1928924915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d0cefc820 0x561d0cf081b0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f2e6409c6c0 tx=0x7f2e64022a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77fd76dbe318b2eb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:21.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.847+0000 7f2e597fa700 1 -- 172.21.15.37:0/1928924915 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e640a5310 con 0x561d0cefc820 2024-06-10T01:07:21.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.848+0000 7f2e597fa700 1 -- 172.21.15.37:0/1928924915 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e640095f0 con 0x561d0cefc820 2024-06-10T01:07:21.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.848+0000 7f2e597fa700 1 -- 172.21.15.37:0/1928924915 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6409c990 con 0x561d0cefc820 2024-06-10T01:07:21.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.848+0000 7f2e844e5b80 1 -- 172.21.15.37:0/1928924915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d0cefc820 msgr2=0x561d0cf081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:21.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.848+0000 7f2e844e5b80 1 --2- 172.21.15.37:0/1928924915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d0cefc820 0x561d0cf081b0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f2e6409c6c0 tx=0x7f2e64022a30 comp rx=0 tx=0).stop 2024-06-10T01:07:21.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.848+0000 7f2e844e5b80 1 -- 172.21.15.37:0/1928924915 shutdown_connections 2024-06-10T01:07:21.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.848+0000 7f2e844e5b80 1 --2- 172.21.15.37:0/1928924915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d0cf02900 0x561d0cef7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.848+0000 7f2e844e5b80 1 --2- 172.21.15.37:0/1928924915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d0cefc820 0x561d0cf081b0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.848+0000 7f2e844e5b80 1 --2- 172.21.15.37:0/1928924915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d0cf01fe0 0x561d0cef6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.848+0000 7f2e844e5b80 1 -- 172.21.15.37:0/1928924915 >> 172.21.15.37:0/1928924915 conn(0x561d0ce29100 msgr2=0x561d0cf00a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:21.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.848+0000 7f2e844e5b80 1 -- 172.21.15.37:0/1928924915 shutdown_connections 2024-06-10T01:07:21.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.849+0000 7f2e844e5b80 1 -- 172.21.15.37:0/1928924915 wait complete. 2024-06-10T01:07:21.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.849+0000 7f2e844e5b80 1 -- start start 2024-06-10T01:07:21.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.849+0000 7f2e844e5b80 1 Processor -- start 2024-06-10T01:07:21.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.849+0000 7f2e844e5b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d0cefc820 0x561d0cf0e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:21.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.849+0000 7f2e844e5b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d0cf01fe0 0x561d0cf0e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:21.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.849+0000 7f2e844e5b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d0cf02900 0x561d0cf0b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:21.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.849+0000 7f2e844e5b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2e741aeb70 con 0x561d0cf01fe0 2024-06-10T01:07:21.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.849+0000 7f2e844e5b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2e741affe0 con 0x561d0cefc820 2024-06-10T01:07:21.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.849+0000 7f2e844e5b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2e741af240 con 0x561d0cf02900 2024-06-10T01:07:21.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.850+0000 7f2e844e5b80 10 client.0 _ll_register_callbacks cb 0x561d0cde3630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:21.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.850+0000 7f2e72ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d0cf01fe0 0x561d0cf0e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:21.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.850+0000 7f2e737fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d0cefc820 0x561d0cf0e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:21.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.850+0000 7f2e72ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d0cf01fe0 0x561d0cf0e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60418/0 (socket says 172.21.15.37:60418) 2024-06-10T01:07:21.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.850+0000 7f2e72ffd700 1 -- 172.21.15.37:0/762292602 learned_addr learned my addr 172.21.15.37:0/762292602 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:21.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.850+0000 7f2e737fe700 1 --2- 172.21.15.37:0/762292602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d0cefc820 0x561d0cf0e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:59178/0 (socket says 172.21.15.37:59178) 2024-06-10T01:07:21.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.850+0000 7f2e73fff700 1 --2- 172.21.15.37:0/762292602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d0cf02900 0x561d0cf0b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:21.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.850+0000 7f2e737fe700 1 -- 172.21.15.37:0/762292602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d0cf02900 msgr2=0x561d0cf0b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:21.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.850+0000 7f2e737fe700 1 --2- 172.21.15.37:0/762292602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d0cf02900 0x561d0cf0b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.850+0000 7f2e737fe700 1 -- 172.21.15.37:0/762292602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d0cf01fe0 msgr2=0x561d0cf0e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:21.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.850+0000 7f2e737fe700 1 --2- 172.21.15.37:0/762292602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d0cf01fe0 0x561d0cf0e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:21.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.850+0000 7f2e737fe700 1 -- 172.21.15.37:0/762292602 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d0cf0ee60 con 0x561d0cefc820 2024-06-10T01:07:21.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.850+0000 7f2e72ffd700 1 --2- 172.21.15.37:0/762292602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d0cf01fe0 0x561d0cf0e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:21.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.850+0000 7f2e737fe700 1 --2- 172.21.15.37:0/762292602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d0cefc820 0x561d0cf0e2e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2e640b0f60 tx=0x7f2e6400ff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5149a0362c454b96 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:21.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.850+0000 7f2e52ffd700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:07:21.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.850+0000 7f2e52ffd700 1 -- 172.21.15.37:0/762292602 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e640a5310 con 0x561d0cefc820 2024-06-10T01:07:21.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.850+0000 7f2e52ffd700 1 -- 172.21.15.37:0/762292602 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f2e640b55a0 con 0x561d0cefc820 2024-06-10T01:07:21.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.850+0000 7f2e844e5b80 1 -- 172.21.15.37:0/762292602 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e640095f0 con 0x561d0cefc820 2024-06-10T01:07:21.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.850+0000 7f2e52ffd700 1 -- 172.21.15.37:0/762292602 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2e6409c990 con 0x561d0cefc820 2024-06-10T01:07:21.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.851+0000 7f2e844e5b80 10 client.26609 fetch_fsmap learned FSMap version 31 2024-06-10T01:07:21.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.851+0000 7f2e844e5b80 1 -- 172.21.15.37:0/762292602 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561d0cf0ee60 con 0x561d0cefc820 2024-06-10T01:07:21.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.851+0000 7f2e52ffd700 1 -- 172.21.15.37:0/762292602 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e640095f0 con 0x561d0cefc820 2024-06-10T01:07:21.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.851+0000 7f2e52ffd700 1 -- 172.21.15.37:0/762292602 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f2e6409b820 con 0x561d0cefc820 2024-06-10T01:07:21.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.851+0000 7f2e844e5b80 10 client.26609 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:07:21.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.851+0000 7f2e844e5b80 10 client.26609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:21.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.851+0000 7f2e844e5b80 20 client.26609 populate_metadata read hostname 'smithi037' 2024-06-10T01:07:21.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.851+0000 7f2e844e5b80 1 --2- 172.21.15.37:0/762292602 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561d0cf15ba0 0x561d0cf82f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:21.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.851+0000 7f2e844e5b80 4 client.26609 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:07:21.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.851+0000 7f2e844e5b80 1 -- 172.21.15.37:0/762292602 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561d0ce55350 con 0x561d0cf15ba0 2024-06-10T01:07:21.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.851+0000 7f2e72ffd700 1 --2- 172.21.15.37:0/762292602 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561d0cf15ba0 0x561d0cf82f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:21.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.852+0000 7f2e72ffd700 1 --2- 172.21.15.37:0/762292602 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561d0cf15ba0 0x561d0cf82f70 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce32edba2a6249c3 server_cookie=d606eda3699ec7ca in_seq=0 out_seq=0 2024-06-10T01:07:21.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:21.852+0000 7f2e52ffd700 10 client.26609 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:22.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:22.458+0000 7f2e52ffd700 1 -- 172.21.15.37:0/762292602 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561d0ce55350 con 0x561d0cf15ba0 2024-06-10T01:07:22.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:22.458+0000 7f2e52ffd700 10 client.26609 handle_command_reply: tid=0 2024-06-10T01:07:22.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:22.458+0000 7f2e844e5b80 1 client.26609 shutdown 2024-06-10T01:07:22.455 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:22 smithi037 ceph-mon[32192]: pgmap v751: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 621 op/s 2024-06-10T01:07:22.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:22.458+0000 7f2e844e5b80 1 -- 172.21.15.37:0/762292602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d0cefc820 msgr2=0x561d0cf0e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:22.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:22.458+0000 7f2e844e5b80 1 --2- 172.21.15.37:0/762292602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d0cefc820 0x561d0cf0e2e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2e640b0f60 tx=0x7f2e6400ff90 comp rx=0 tx=0).stop 2024-06-10T01:07:22.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:22.458+0000 7f2e844e5b80 1 -- 172.21.15.37:0/762292602 shutdown_connections 2024-06-10T01:07:22.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:22.458+0000 7f2e844e5b80 1 --2- 172.21.15.37:0/762292602 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561d0cf15ba0 0x561d0cf82f70 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:22.458+0000 7f2e844e5b80 1 --2- 172.21.15.37:0/762292602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561d0cf02900 0x561d0cf0b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:22.458+0000 7f2e52ffd700 0 client.26609 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:22.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:22.458+0000 7f2e844e5b80 1 --2- 172.21.15.37:0/762292602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561d0cefc820 0x561d0cf0e2e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:22.458+0000 7f2e844e5b80 1 --2- 172.21.15.37:0/762292602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d0cf01fe0 0x561d0cf0e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:22.458+0000 7f2e844e5b80 1 -- 172.21.15.37:0/762292602 >> 172.21.15.37:0/762292602 conn(0x561d0ce29100 msgr2=0x561d0cf00a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:22.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:22.458+0000 7f2e844e5b80 1 -- 172.21.15.37:0/762292602 shutdown_connections 2024-06-10T01:07:22.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:22.458+0000 7f2e844e5b80 1 -- 172.21.15.37:0/762292602 wait complete. 2024-06-10T01:07:22.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:22.459+0000 7f2e844e5b80 20 client.26609 trim_cache size 0 max 16384 2024-06-10T01:07:22.457 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:07:22.457 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-06-10T01:07:22.457 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "ea4883dd-fc0f-4a1f-9901-cf51e85dbb26", 2024-06-10T01:07:22.457 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-06-10T01:07:22.457 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:07:22.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:22.459+0000 7f2e79c62700 1 -- 172.21.15.37:0/1697265450 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2e5c0a2c20 msgr2=0x7f2e5c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:22.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:22.459+0000 7f2e79c62700 1 --2- 172.21.15.37:0/1697265450 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2e5c0a2c20 0x7f2e5c0a50e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2e74078830 tx=0x7f2e68009260 comp rx=0 tx=0).stop 2024-06-10T01:07:22.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:22.459+0000 7f2e79c62700 1 -- 172.21.15.37:0/1697265450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e74101390 msgr2=0x7f2e74103040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:22.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:22.459+0000 7f2e79c62700 1 --2- 172.21.15.37:0/1697265450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e74101390 0x7f2e74103040 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f2e64005f50 tx=0x7f2e640047b0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:22.459+0000 7f2e79c62700 1 -- 172.21.15.37:0/1697265450 shutdown_connections 2024-06-10T01:07:22.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:22.459+0000 7f2e79c62700 1 --2- 172.21.15.37:0/1697265450 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2e5c0a2c20 0x7f2e5c0a50e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:22.459+0000 7f2e79c62700 1 --2- 172.21.15.37:0/1697265450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e741073d0 0x7f2e74103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:22.459+0000 7f2e79c62700 1 --2- 172.21.15.37:0/1697265450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e74101390 0x7f2e74103040 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:22.459+0000 7f2e79c62700 1 --2- 172.21.15.37:0/1697265450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e74101d30 0x7f2e74103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:22.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:22.459+0000 7f2e79c62700 1 -- 172.21.15.37:0/1697265450 >> 172.21.15.37:0/1697265450 conn(0x7f2e740782c0 msgr2=0x7f2e741006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:22.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:22.459+0000 7f2e79c62700 1 -- 172.21.15.37:0/1697265450 shutdown_connections 2024-06-10T01:07:22.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:22.459+0000 7f2e79c62700 1 -- 172.21.15.37:0/1697265450 wait complete. 2024-06-10T01:07:22.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:22 smithi167 ceph-mon[36917]: pgmap v751: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 621 op/s 2024-06-10T01:07:22.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:22 smithi094 ceph-mon[36708]: pgmap v751: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 621 op/s 2024-06-10T01:07:23.193 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:07:23.448 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:23.573 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:23 smithi037 ceph-mon[32192]: scrub queued for path: / 2024-06-10T01:07:23.574 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:23 smithi037 ceph-mon[32192]: scrub summary: idle+waiting paths [/] 2024-06-10T01:07:23.574 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:23 smithi037 ceph-mon[32192]: scrub summary: active paths [/] 2024-06-10T01:07:23.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:23 smithi167 ceph-mon[36917]: scrub queued for path: / 2024-06-10T01:07:23.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:23 smithi167 ceph-mon[36917]: scrub summary: idle+waiting paths [/] 2024-06-10T01:07:23.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:23 smithi167 ceph-mon[36917]: scrub summary: active paths [/] 2024-06-10T01:07:23.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:23 smithi094 ceph-mon[36708]: scrub queued for path: / 2024-06-10T01:07:23.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:23 smithi094 ceph-mon[36708]: scrub summary: idle+waiting paths [/] 2024-06-10T01:07:23.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:23 smithi094 ceph-mon[36708]: scrub summary: active paths [/] 2024-06-10T01:07:24.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:24 smithi167 ceph-mon[36917]: pgmap v752: 129 pgs: 129 active+clean; 25 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 706 op/s 2024-06-10T01:07:24.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:24 smithi094 ceph-mon[36708]: pgmap v752: 129 pgs: 129 active+clean; 25 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 706 op/s 2024-06-10T01:07:24.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:24 smithi037 ceph-mon[32192]: pgmap v752: 129 pgs: 129 active+clean; 25 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 706 op/s 2024-06-10T01:07:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.721+0000 7fdb532e3700 1 -- 172.21.15.37:0/1121995103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb4c101390 msgr2=0x7fdb4c101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.721+0000 7fdb532e3700 1 --2- 172.21.15.37:0/1121995103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb4c101390 0x7fdb4c101810 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c009990 tx=0x7fdb3c009ca0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.721+0000 7fdb532e3700 1 -- 172.21.15.37:0/1121995103 shutdown_connections 2024-06-10T01:07:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.721+0000 7fdb532e3700 1 --2- 172.21.15.37:0/1121995103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb4c101d50 0x7fdb4c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.721+0000 7fdb532e3700 1 --2- 172.21.15.37:0/1121995103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb4c107400 0x7fdb4c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.721+0000 7fdb532e3700 1 --2- 172.21.15.37:0/1121995103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb4c101390 0x7fdb4c101810 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.721+0000 7fdb532e3700 1 -- 172.21.15.37:0/1121995103 >> 172.21.15.37:0/1121995103 conn(0x7fdb4c078340 msgr2=0x7fdb4c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.721+0000 7fdb532e3700 1 -- 172.21.15.37:0/1121995103 shutdown_connections 2024-06-10T01:07:25.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.721+0000 7fdb532e3700 1 -- 172.21.15.37:0/1121995103 wait complete. 2024-06-10T01:07:25.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.722+0000 7fdb532e3700 1 Processor -- start 2024-06-10T01:07:25.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.722+0000 7fdb532e3700 1 -- start start 2024-06-10T01:07:25.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.722+0000 7fdb532e3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb4c101390 0x7fdb4c19f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:25.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.722+0000 7fdb532e3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb4c101d50 0x7fdb4c19fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:25.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.723+0000 7fdb532e3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb4c107400 0x7fdb4c1a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:25.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.723+0000 7fdb532e3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c111670 con 0x7fdb4c107400 2024-06-10T01:07:25.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.723+0000 7fdb532e3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c1114f0 con 0x7fdb4c101390 2024-06-10T01:07:25.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.723+0000 7fdb532e3700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c1117f0 con 0x7fdb4c101d50 2024-06-10T01:07:25.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.723+0000 7fdb5087e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb4c101d50 0x7fdb4c19fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:25.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.723+0000 7fdb5087e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb4c101d50 0x7fdb4c19fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:39362/0 (socket says 172.21.15.37:39362) 2024-06-10T01:07:25.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.723+0000 7fdb5087e700 1 -- 172.21.15.37:0/3800901234 learned_addr learned my addr 172.21.15.37:0/3800901234 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:25.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.723+0000 7fdb5107f700 1 --2- 172.21.15.37:0/3800901234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb4c101390 0x7fdb4c19f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:25.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.723+0000 7fdb51880700 1 --2- 172.21.15.37:0/3800901234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb4c107400 0x7fdb4c1a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:25.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.723+0000 7fdb5107f700 1 -- 172.21.15.37:0/3800901234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb4c101d50 msgr2=0x7fdb4c19fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:25.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.723+0000 7fdb5107f700 1 --2- 172.21.15.37:0/3800901234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb4c101d50 0x7fdb4c19fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.723+0000 7fdb5107f700 1 -- 172.21.15.37:0/3800901234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb4c107400 msgr2=0x7fdb4c1a3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:25.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.723+0000 7fdb5107f700 1 --2- 172.21.15.37:0/3800901234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb4c107400 0x7fdb4c1a3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.723+0000 7fdb5107f700 1 -- 172.21.15.37:0/3800901234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4c1a4260 con 0x7fdb4c101390 2024-06-10T01:07:25.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.723+0000 7fdb5107f700 1 --2- 172.21.15.37:0/3800901234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb4c101390 0x7fdb4c19f520 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800d950 tx=0x7fdb4800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:25.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.723+0000 7fdb427fc700 1 -- 172.21.15.37:0/3800901234 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb48016070 con 0x7fdb4c101390 2024-06-10T01:07:25.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.723+0000 7fdb532e3700 1 -- 172.21.15.37:0/3800901234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb4c1a4550 con 0x7fdb4c101390 2024-06-10T01:07:25.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.723+0000 7fdb427fc700 1 -- 172.21.15.37:0/3800901234 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb4800fd70 con 0x7fdb4c101390 2024-06-10T01:07:25.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.723+0000 7fdb427fc700 1 -- 172.21.15.37:0/3800901234 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4800e3b0 con 0x7fdb4c101390 2024-06-10T01:07:25.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.723+0000 7fdb532e3700 1 -- 172.21.15.37:0/3800901234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb4c1abdd0 con 0x7fdb4c101390 2024-06-10T01:07:25.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.726+0000 7fdb427fc700 1 -- 172.21.15.37:0/3800901234 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fdb48019070 con 0x7fdb4c101390 2024-06-10T01:07:25.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.726+0000 7fdb427fc700 1 --2- 172.21.15.37:0/3800901234 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdb380a2d20 0x7fdb380a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:25.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.727+0000 7fdb5087e700 1 --2- 172.21.15.37:0/3800901234 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdb380a2d20 0x7fdb380a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:25.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.727+0000 7fdb427fc700 1 -- 172.21.15.37:0/3800901234 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fdb480ced90 con 0x7fdb4c101390 2024-06-10T01:07:25.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.727+0000 7fdb5087e700 1 --2- 172.21.15.37:0/3800901234 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdb380a2d20 0x7fdb380a51e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c009750 tx=0x7fdb3c005bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:25.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.727+0000 7fdb5db66b80 1 Processor -- start 2024-06-10T01:07:25.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.727+0000 7fdb5db66b80 1 -- start start 2024-06-10T01:07:25.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.727+0000 7fdb5db66b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636dffe0640 0x5636dffe0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:25.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.727+0000 7fdb5db66b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636dffe0f60 0x5636dffda9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:25.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.728+0000 7fdb5db66b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636dffdaf20 0x5636dffdb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:25.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.728+0000 7fdb5087e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636dffe0f60 0x5636dffda9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:25.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.728+0000 7fdb51880700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636dffdaf20 0x5636dffdb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:25.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.728+0000 7fdb5087e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636dffe0f60 0x5636dffda9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:39372/0 (socket says 172.21.15.37:39372) 2024-06-10T01:07:25.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.728+0000 7fdb51880700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636dffdaf20 0x5636dffdb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60446/0 (socket says 172.21.15.37:60446) 2024-06-10T01:07:25.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.728+0000 7fdb5087e700 1 -- 172.21.15.37:0/1861434841 learned_addr learned my addr 172.21.15.37:0/1861434841 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:25.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.728+0000 7fdb5db66b80 1 -- 172.21.15.37:0/1861434841 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5636dff2c810 con 0x5636dffdaf20 2024-06-10T01:07:25.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.728+0000 7fdb5db66b80 1 -- 172.21.15.37:0/1861434841 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5636dfed4720 con 0x5636dffe0640 2024-06-10T01:07:25.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.728+0000 7fdb5db66b80 1 -- 172.21.15.37:0/1861434841 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5636dff287d0 con 0x5636dffe0f60 2024-06-10T01:07:25.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.728+0000 7fdb5107f700 1 --2- 172.21.15.37:0/1861434841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636dffe0640 0x5636dffe0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:25.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.728+0000 7fdb5087e700 1 -- 172.21.15.37:0/1861434841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636dffe0640 msgr2=0x5636dffe0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:25.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.728+0000 7fdb5087e700 1 --2- 172.21.15.37:0/1861434841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636dffe0640 0x5636dffe0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.728+0000 7fdb5087e700 1 -- 172.21.15.37:0/1861434841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636dffdaf20 msgr2=0x5636dffdb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:25.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.728+0000 7fdb5087e700 1 --2- 172.21.15.37:0/1861434841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636dffdaf20 0x5636dffdb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.728+0000 7fdb5087e700 1 -- 172.21.15.37:0/1861434841 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636dfecaf50 con 0x5636dffe0f60 2024-06-10T01:07:25.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.728+0000 7fdb5087e700 1 --2- 172.21.15.37:0/1861434841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636dffe0f60 0x5636dffda9e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00ba60 tx=0x7fdb3c020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30dbd8db1f8575b8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:25.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.729+0000 7fdb377fe700 1 -- 172.21.15.37:0/1861434841 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c00bd70 con 0x5636dffe0f60 2024-06-10T01:07:25.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.729+0000 7fdb377fe700 1 -- 172.21.15.37:0/1861434841 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb3c00fcd0 con 0x5636dffe0f60 2024-06-10T01:07:25.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.729+0000 7fdb377fe700 1 -- 172.21.15.37:0/1861434841 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c024c60 con 0x5636dffe0f60 2024-06-10T01:07:25.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.729+0000 7fdb5db66b80 1 -- 172.21.15.37:0/1861434841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636dffe0f60 msgr2=0x5636dffda9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:25.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.729+0000 7fdb5db66b80 1 --2- 172.21.15.37:0/1861434841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636dffe0f60 0x5636dffda9e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00ba60 tx=0x7fdb3c020000 comp rx=0 tx=0).stop 2024-06-10T01:07:25.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.729+0000 7fdb5db66b80 1 -- 172.21.15.37:0/1861434841 shutdown_connections 2024-06-10T01:07:25.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.729+0000 7fdb5db66b80 1 --2- 172.21.15.37:0/1861434841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636dffdaf20 0x5636dffdb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.729+0000 7fdb5db66b80 1 --2- 172.21.15.37:0/1861434841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636dffe0640 0x5636dffe0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.730+0000 7fdb5db66b80 1 --2- 172.21.15.37:0/1861434841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636dffe0f60 0x5636dffda9e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.730+0000 7fdb5db66b80 1 -- 172.21.15.37:0/1861434841 >> 172.21.15.37:0/1861434841 conn(0x5636dff05cc0 msgr2=0x5636dffd5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:25.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.730+0000 7fdb5db66b80 1 -- 172.21.15.37:0/1861434841 shutdown_connections 2024-06-10T01:07:25.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.730+0000 7fdb5db66b80 1 -- 172.21.15.37:0/1861434841 wait complete. 2024-06-10T01:07:25.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.730+0000 7fdb5db66b80 1 -- start start 2024-06-10T01:07:25.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.730+0000 7fdb5db66b80 1 Processor -- start 2024-06-10T01:07:25.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.731+0000 7fdb5db66b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636dffdaf20 0x5636dffe6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:25.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.731+0000 7fdb5db66b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636dffe0640 0x5636dffe3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:25.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.731+0000 7fdb5db66b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636dffe0f60 0x5636dffe44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:25.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.731+0000 7fdb5db66b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5636dffe7910 con 0x5636dffe0f60 2024-06-10T01:07:25.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.731+0000 7fdb5db66b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5636dfeccbf0 con 0x5636dffe0640 2024-06-10T01:07:25.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.731+0000 7fdb5db66b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5636dffe7a90 con 0x5636dffdaf20 2024-06-10T01:07:25.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.731+0000 7fdb5db66b80 10 client.0 _ll_register_callbacks cb 0x5636dfec6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:25.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.731+0000 7fdb5107f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636dffdaf20 0x5636dffe6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:25.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.731+0000 7fdb5107f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636dffdaf20 0x5636dffe6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:39386/0 (socket says 172.21.15.37:39386) 2024-06-10T01:07:25.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.731+0000 7fdb51880700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636dffe0f60 0x5636dffe44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:25.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.731+0000 7fdb5107f700 1 -- 172.21.15.37:0/526634965 learned_addr learned my addr 172.21.15.37:0/526634965 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:25.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.731+0000 7fdb5087e700 1 --2- 172.21.15.37:0/526634965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636dffe0640 0x5636dffe3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:25.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.731+0000 7fdb5107f700 1 -- 172.21.15.37:0/526634965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636dffe0640 msgr2=0x5636dffe3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:25.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.731+0000 7fdb5107f700 1 --2- 172.21.15.37:0/526634965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636dffe0640 0x5636dffe3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.731+0000 7fdb5107f700 1 -- 172.21.15.37:0/526634965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636dffe0f60 msgr2=0x5636dffe44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:25.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.731+0000 7fdb5107f700 1 --2- 172.21.15.37:0/526634965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636dffe0f60 0x5636dffe44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.731+0000 7fdb5107f700 1 -- 172.21.15.37:0/526634965 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636dfe7b8d0 con 0x5636dffdaf20 2024-06-10T01:07:25.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.731+0000 7fdb5107f700 1 --2- 172.21.15.37:0/526634965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636dffdaf20 0x5636dffe6c90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdb480cc440 tx=0x7fdb480996b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4003f20a863ad50 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:25.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.731+0000 7fdb34ff9700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:07:25.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.731+0000 7fdb34ff9700 1 -- 172.21.15.37:0/526634965 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb48020040 con 0x5636dffdaf20 2024-06-10T01:07:25.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.732+0000 7fdb34ff9700 1 -- 172.21.15.37:0/526634965 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fdb48010680 con 0x5636dffdaf20 2024-06-10T01:07:25.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.732+0000 7fdb5db66b80 1 -- 172.21.15.37:0/526634965 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb3c00fcd0 con 0x5636dffdaf20 2024-06-10T01:07:25.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.732+0000 7fdb34ff9700 1 -- 172.21.15.37:0/526634965 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdb4809b1a0 con 0x5636dffdaf20 2024-06-10T01:07:25.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.732+0000 7fdb5db66b80 10 client.26764 fetch_fsmap learned FSMap version 31 2024-06-10T01:07:25.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.732+0000 7fdb5db66b80 1 -- 172.21.15.37:0/526634965 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5636dfff2230 con 0x5636dffdaf20 2024-06-10T01:07:25.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.732+0000 7fdb34ff9700 1 -- 172.21.15.37:0/526634965 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb4809f550 con 0x5636dffdaf20 2024-06-10T01:07:25.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.732+0000 7fdb34ff9700 1 -- 172.21.15.37:0/526634965 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fdb480a3980 con 0x5636dffdaf20 2024-06-10T01:07:25.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.732+0000 7fdb5db66b80 10 client.26764 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:07:25.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.732+0000 7fdb5db66b80 10 client.26764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:25.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.732+0000 7fdb5db66b80 20 client.26764 populate_metadata read hostname 'smithi037' 2024-06-10T01:07:25.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.732+0000 7fdb5db66b80 1 --2- 172.21.15.37:0/526634965 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5636dfff47c0 0x5636dfff6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:25.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.732+0000 7fdb5db66b80 4 client.26764 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:25.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.732+0000 7fdb5db66b80 1 -- 172.21.15.37:0/526634965 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdb3c00fcd0 con 0x5636dfff47c0 2024-06-10T01:07:25.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.732+0000 7fdb5087e700 1 --2- 172.21.15.37:0/526634965 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5636dfff47c0 0x5636dfff6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:25.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.733+0000 7fdb5087e700 1 --2- 172.21.15.37:0/526634965 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5636dfff47c0 0x5636dfff6bb0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=124779205ba4f94c server_cookie=88b87afb6c4496fc in_seq=0 out_seq=0 2024-06-10T01:07:25.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.733+0000 7fdb34ff9700 10 client.26764 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:25.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.737+0000 7fdb34ff9700 1 -- 172.21.15.37:0/526634965 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fdb3c00fcd0 con 0x5636dfff47c0 2024-06-10T01:07:25.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.737+0000 7fdb34ff9700 10 client.26764 handle_command_reply: tid=0 2024-06-10T01:07:25.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.737+0000 7fdb5db66b80 1 client.26764 shutdown 2024-06-10T01:07:25.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.737+0000 7fdb5db66b80 1 -- 172.21.15.37:0/526634965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636dffdaf20 msgr2=0x5636dffe6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:25.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.737+0000 7fdb5db66b80 1 --2- 172.21.15.37:0/526634965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636dffdaf20 0x5636dffe6c90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdb480cc440 tx=0x7fdb480996b0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.737+0000 7fdb5db66b80 1 -- 172.21.15.37:0/526634965 shutdown_connections 2024-06-10T01:07:25.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.737+0000 7fdb5db66b80 1 --2- 172.21.15.37:0/526634965 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5636dfff47c0 0x5636dfff6bb0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.737+0000 7fdb5db66b80 1 --2- 172.21.15.37:0/526634965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636dffe0f60 0x5636dffe44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.737+0000 7fdb5db66b80 1 --2- 172.21.15.37:0/526634965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636dffdaf20 0x5636dffe6c90 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.737+0000 7fdb5db66b80 1 --2- 172.21.15.37:0/526634965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636dffe0640 0x5636dffe3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.737+0000 7fdb5db66b80 1 -- 172.21.15.37:0/526634965 >> 172.21.15.37:0/526634965 conn(0x5636dff05cc0 msgr2=0x5636dffd55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:25.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.737+0000 7fdb34ff9700 0 client.26764 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:25.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.738+0000 7fdb5db66b80 1 -- 172.21.15.37:0/526634965 shutdown_connections 2024-06-10T01:07:25.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.738+0000 7fdb5db66b80 1 -- 172.21.15.37:0/526634965 wait complete. 2024-06-10T01:07:25.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.738+0000 7fdb5db66b80 20 client.26764 trim_cache size 0 max 16384 2024-06-10T01:07:25.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.760+0000 7fdb5db66b80 1 Processor -- start 2024-06-10T01:07:25.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.760+0000 7fdb5db66b80 1 -- start start 2024-06-10T01:07:25.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.760+0000 7fdb5db66b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636dffdaf20 0x5636dffda9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:25.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.760+0000 7fdb5db66b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636dffe0640 0x5636dffd59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:25.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.760+0000 7fdb5db66b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636dffe0f60 0x5636dffcf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:25.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.760+0000 7fdb5db66b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c1b7280 con 0x5636dffdaf20 2024-06-10T01:07:25.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.760+0000 7fdb5db66b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c1b4b10 con 0x5636dffe0640 2024-06-10T01:07:25.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.760+0000 7fdb5db66b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c1b2960 con 0x5636dffe0f60 2024-06-10T01:07:25.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.760+0000 7fdb5107f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636dffdaf20 0x5636dffda9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:25.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.760+0000 7fdb5107f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636dffdaf20 0x5636dffda9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60462/0 (socket says 172.21.15.37:60462) 2024-06-10T01:07:25.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.760+0000 7fdb5107f700 1 -- 172.21.15.37:0/1877680489 learned_addr learned my addr 172.21.15.37:0/1877680489 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:25.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.760+0000 7fdb51880700 1 --2- 172.21.15.37:0/1877680489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636dffe0f60 0x5636dffcf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:25.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.761+0000 7fdb5087e700 1 --2- 172.21.15.37:0/1877680489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636dffe0640 0x5636dffd59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:25.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.761+0000 7fdb5107f700 1 -- 172.21.15.37:0/1877680489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636dffe0f60 msgr2=0x5636dffcf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:25.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.761+0000 7fdb5107f700 1 --2- 172.21.15.37:0/1877680489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636dffe0f60 0x5636dffcf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.761+0000 7fdb5107f700 1 -- 172.21.15.37:0/1877680489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636dffe0640 msgr2=0x5636dffd59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:25.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.761+0000 7fdb5107f700 1 --2- 172.21.15.37:0/1877680489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636dffe0640 0x5636dffd59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.761+0000 7fdb5107f700 1 -- 172.21.15.37:0/1877680489 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636dffe2500 con 0x5636dffdaf20 2024-06-10T01:07:25.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.761+0000 7fdb5107f700 1 --2- 172.21.15.37:0/1877680489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636dffdaf20 0x5636dffda9e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fdb480a38e0 tx=0x7fdb4800fff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e48d09023c6ac66 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:25.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.761+0000 7fdb36ffd700 1 -- 172.21.15.37:0/1877680489 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb48020070 con 0x5636dffdaf20 2024-06-10T01:07:25.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.761+0000 7fdb36ffd700 1 -- 172.21.15.37:0/1877680489 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb4809a910 con 0x5636dffdaf20 2024-06-10T01:07:25.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.761+0000 7fdb36ffd700 1 -- 172.21.15.37:0/1877680489 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb48010770 con 0x5636dffdaf20 2024-06-10T01:07:25.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.762+0000 7fdb5db66b80 1 -- 172.21.15.37:0/1877680489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636dffdaf20 msgr2=0x5636dffda9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:25.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.762+0000 7fdb5db66b80 1 --2- 172.21.15.37:0/1877680489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636dffdaf20 0x5636dffda9e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fdb480a38e0 tx=0x7fdb4800fff0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.762+0000 7fdb5db66b80 1 -- 172.21.15.37:0/1877680489 shutdown_connections 2024-06-10T01:07:25.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.762+0000 7fdb5db66b80 1 --2- 172.21.15.37:0/1877680489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636dffe0f60 0x5636dffcf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.762+0000 7fdb5db66b80 1 --2- 172.21.15.37:0/1877680489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636dffdaf20 0x5636dffda9e0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.762+0000 7fdb5db66b80 1 --2- 172.21.15.37:0/1877680489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636dffe0640 0x5636dffd59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.762+0000 7fdb5db66b80 1 -- 172.21.15.37:0/1877680489 >> 172.21.15.37:0/1877680489 conn(0x5636dff05cc0 msgr2=0x5636dffdf0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:25.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.762+0000 7fdb5db66b80 1 -- 172.21.15.37:0/1877680489 shutdown_connections 2024-06-10T01:07:25.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.762+0000 7fdb5db66b80 1 -- 172.21.15.37:0/1877680489 wait complete. 2024-06-10T01:07:25.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.762+0000 7fdb5db66b80 1 -- start start 2024-06-10T01:07:25.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.763+0000 7fdb5db66b80 1 Processor -- start 2024-06-10T01:07:25.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.763+0000 7fdb5db66b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636dffdaf20 0x5636dffeadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:25.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.763+0000 7fdb5db66b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636dffe0640 0x5636dffeb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:25.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.763+0000 7fdb5db66b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636dffe0f60 0x5636dffeb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:25.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.763+0000 7fdb5db66b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c1b2fd0 con 0x5636dffdaf20 2024-06-10T01:07:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.763+0000 7fdb5db66b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c1b4440 con 0x5636dffe0f60 2024-06-10T01:07:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.763+0000 7fdb5db66b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c1b36a0 con 0x5636dffe0640 2024-06-10T01:07:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.763+0000 7fdb5107f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636dffdaf20 0x5636dffeadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.763+0000 7fdb5107f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636dffdaf20 0x5636dffeadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60466/0 (socket says 172.21.15.37:60466) 2024-06-10T01:07:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.763+0000 7fdb5107f700 1 -- 172.21.15.37:0/246031240 learned_addr learned my addr 172.21.15.37:0/246031240 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.763+0000 7fdb5db66b80 10 client.0 _ll_register_callbacks cb 0x5636dfec6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.763+0000 7fdb5087e700 1 --2- 172.21.15.37:0/246031240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636dffe0640 0x5636dffeb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.763+0000 7fdb51880700 1 --2- 172.21.15.37:0/246031240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636dffe0f60 0x5636dffeb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.764+0000 7fdb5107f700 1 -- 172.21.15.37:0/246031240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636dffe0640 msgr2=0x5636dffeb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.764+0000 7fdb5107f700 1 --2- 172.21.15.37:0/246031240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636dffe0640 0x5636dffeb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.764+0000 7fdb5107f700 1 -- 172.21.15.37:0/246031240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636dffe0f60 msgr2=0x5636dffeb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.764+0000 7fdb5107f700 1 --2- 172.21.15.37:0/246031240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636dffe0f60 0x5636dffeb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.764+0000 7fdb5107f700 1 -- 172.21.15.37:0/246031240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636e005e6c0 con 0x5636dffdaf20 2024-06-10T01:07:25.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.764+0000 7fdb5087e700 1 --2- 172.21.15.37:0/246031240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636dffe0640 0x5636dffeb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:25.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.764+0000 7fdb5107f700 1 --2- 172.21.15.37:0/246031240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636dffdaf20 0x5636dffeadc0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fdb4809aea0 tx=0x7fdb4809b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7406d36a882a5b8b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:25.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.764+0000 7fdb34ff9700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:07:25.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.764+0000 7fdb34ff9700 1 -- 172.21.15.37:0/246031240 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb48020070 con 0x5636dffdaf20 2024-06-10T01:07:25.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.764+0000 7fdb34ff9700 1 -- 172.21.15.37:0/246031240 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fdb4809f170 con 0x5636dffdaf20 2024-06-10T01:07:25.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.764+0000 7fdb5db66b80 1 -- 172.21.15.37:0/246031240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb4809a910 con 0x5636dffdaf20 2024-06-10T01:07:25.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.764+0000 7fdb34ff9700 1 -- 172.21.15.37:0/246031240 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdb48097440 con 0x5636dffdaf20 2024-06-10T01:07:25.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.764+0000 7fdb5db66b80 10 client.17247 fetch_fsmap learned FSMap version 31 2024-06-10T01:07:25.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.764+0000 7fdb5db66b80 1 -- 172.21.15.37:0/246031240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5636e005e6c0 con 0x5636dffdaf20 2024-06-10T01:07:25.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.764+0000 7fdb34ff9700 1 -- 172.21.15.37:0/246031240 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb4809f550 con 0x5636dffdaf20 2024-06-10T01:07:25.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.765+0000 7fdb34ff9700 1 -- 172.21.15.37:0/246031240 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fdb480af180 con 0x5636dffdaf20 2024-06-10T01:07:25.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.765+0000 7fdb5db66b80 10 client.17247 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:07:25.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.765+0000 7fdb5db66b80 10 client.17247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:25.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.765+0000 7fdb5db66b80 20 client.17247 populate_metadata read hostname 'smithi037' 2024-06-10T01:07:25.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.765+0000 7fdb5db66b80 1 --2- 172.21.15.37:0/246031240 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5636e005fb70 0x5636e0061f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:25.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.765+0000 7fdb5db66b80 4 client.17247 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:07:25.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.765+0000 7fdb5db66b80 1 -- 172.21.15.37:0/246031240 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fdb4809a910 con 0x5636e005fb70 2024-06-10T01:07:25.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.765+0000 7fdb5087e700 1 --2- 172.21.15.37:0/246031240 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5636e005fb70 0x5636e0061f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:25.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.765+0000 7fdb34ff9700 10 client.17247 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:25.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.765+0000 7fdb5087e700 1 --2- 172.21.15.37:0/246031240 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5636e005fb70 0x5636e0061f60 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fc80aedcd6bb785 server_cookie=4a9b22c3f3a3abc8 in_seq=0 out_seq=0 2024-06-10T01:07:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.772+0000 7fdb34ff9700 1 -- 172.21.15.37:0/246031240 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x7fdb4809a910 con 0x5636e005fb70 2024-06-10T01:07:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.772+0000 7fdb34ff9700 10 client.17247 handle_command_reply: tid=0 2024-06-10T01:07:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.772+0000 7fdb5db66b80 1 client.17247 shutdown 2024-06-10T01:07:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.772+0000 7fdb5db66b80 1 -- 172.21.15.37:0/246031240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636dffdaf20 msgr2=0x5636dffeadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.772+0000 7fdb5db66b80 1 --2- 172.21.15.37:0/246031240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636dffdaf20 0x5636dffeadc0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fdb4809aea0 tx=0x7fdb4809b630 comp rx=0 tx=0).stop 2024-06-10T01:07:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.772+0000 7fdb5db66b80 1 -- 172.21.15.37:0/246031240 shutdown_connections 2024-06-10T01:07:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.773+0000 7fdb5db66b80 1 --2- 172.21.15.37:0/246031240 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5636e005fb70 0x5636e0061f60 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.773+0000 7fdb5db66b80 1 --2- 172.21.15.37:0/246031240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5636dffe0f60 0x5636dffeb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.773+0000 7fdb5db66b80 1 --2- 172.21.15.37:0/246031240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5636dffdaf20 0x5636dffeadc0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.773+0000 7fdb5db66b80 1 --2- 172.21.15.37:0/246031240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636dffe0640 0x5636dffeb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.773+0000 7fdb34ff9700 0 client.17247 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.773+0000 7fdb5db66b80 1 -- 172.21.15.37:0/246031240 >> 172.21.15.37:0/246031240 conn(0x5636dff05cc0 msgr2=0x5636dffcf510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.773+0000 7fdb5db66b80 1 -- 172.21.15.37:0/246031240 shutdown_connections 2024-06-10T01:07:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.773+0000 7fdb5db66b80 1 -- 172.21.15.37:0/246031240 wait complete. 2024-06-10T01:07:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.773+0000 7fdb5db66b80 20 client.17247 trim_cache size 0 max 16384 2024-06-10T01:07:25.770 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:07:25.770 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "scrub active (11168 inodes in the stack)", 2024-06-10T01:07:25.770 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-06-10T01:07:25.770 INFO:teuthology.orchestra.run.smithi037.stdout: "ea4883dd-fc0f-4a1f-9901-cf51e85dbb26": { 2024-06-10T01:07:25.771 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-06-10T01:07:25.771 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "ea4883dd-fc0f-4a1f-9901-cf51e85dbb26", 2024-06-10T01:07:25.771 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-06-10T01:07:25.771 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:07:25.771 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:07:25.771 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:07:25.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.773+0000 7fdb532e3700 1 -- 172.21.15.37:0/3800901234 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdb380a2d20 msgr2=0x7fdb380a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:25.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.773+0000 7fdb532e3700 1 --2- 172.21.15.37:0/3800901234 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdb380a2d20 0x7fdb380a51e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c009750 tx=0x7fdb3c005bc0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.773+0000 7fdb532e3700 1 -- 172.21.15.37:0/3800901234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb4c101390 msgr2=0x7fdb4c19f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.773+0000 7fdb532e3700 1 --2- 172.21.15.37:0/3800901234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb4c101390 0x7fdb4c19f520 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800d950 tx=0x7fdb4800dc60 comp rx=0 tx=0).stop 2024-06-10T01:07:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.774+0000 7fdb532e3700 1 -- 172.21.15.37:0/3800901234 shutdown_connections 2024-06-10T01:07:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.774+0000 7fdb532e3700 1 --2- 172.21.15.37:0/3800901234 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdb380a2d20 0x7fdb380a51e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.774+0000 7fdb532e3700 1 --2- 172.21.15.37:0/3800901234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb4c107400 0x7fdb4c1a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.774+0000 7fdb532e3700 1 --2- 172.21.15.37:0/3800901234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb4c101390 0x7fdb4c19f520 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.774+0000 7fdb532e3700 1 --2- 172.21.15.37:0/3800901234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb4c101d50 0x7fdb4c19fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.774+0000 7fdb532e3700 1 -- 172.21.15.37:0/3800901234 >> 172.21.15.37:0/3800901234 conn(0x7fdb4c078340 msgr2=0x7fdb4c06a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.774+0000 7fdb532e3700 1 -- 172.21.15.37:0/3800901234 shutdown_connections 2024-06-10T01:07:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:25.774+0000 7fdb532e3700 1 -- 172.21.15.37:0/3800901234 wait complete. 2024-06-10T01:07:26.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:26 smithi037 ceph-mon[32192]: pgmap v753: 129 pgs: 129 active+clean; 13 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 754 op/s 2024-06-10T01:07:26.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:26 smithi167 ceph-mon[36917]: pgmap v753: 129 pgs: 129 active+clean; 13 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 754 op/s 2024-06-10T01:07:26.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:26 smithi094 ceph-mon[36708]: pgmap v753: 129 pgs: 129 active+clean; 13 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 754 op/s 2024-06-10T01:07:28.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:28 smithi037 ceph-mon[32192]: pgmap v754: 129 pgs: 129 active+clean; 13 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 828 KiB/s wr, 648 op/s 2024-06-10T01:07:28.563 INFO:tasks.cephfs.filesystem:scrub status for tag:ea4883dd-fc0f-4a1f-9901-cf51e85dbb26 - {'path': '/', 'tag': 'ea4883dd-fc0f-4a1f-9901-cf51e85dbb26', 'options': 'recursive,force'} 2024-06-10T01:07:28.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:28 smithi167 ceph-mon[36917]: pgmap v754: 129 pgs: 129 active+clean; 13 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 828 KiB/s wr, 648 op/s 2024-06-10T01:07:28.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:28 smithi094 ceph-mon[36708]: pgmap v754: 129 pgs: 129 active+clean; 13 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 828 KiB/s wr, 648 op/s 2024-06-10T01:07:30.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:30 smithi167 ceph-mon[36917]: pgmap v755: 129 pgs: 129 active+clean; 19 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 769 op/s 2024-06-10T01:07:30.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:30 smithi094 ceph-mon[36708]: pgmap v755: 129 pgs: 129 active+clean; 19 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 769 op/s 2024-06-10T01:07:30.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:30 smithi037 ceph-mon[32192]: pgmap v755: 129 pgs: 129 active+clean; 19 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 769 op/s 2024-06-10T01:07:32.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:32 smithi167 ceph-mon[36917]: pgmap v756: 129 pgs: 129 active+clean; 19 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 532 op/s 2024-06-10T01:07:32.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:32 smithi094 ceph-mon[36708]: pgmap v756: 129 pgs: 129 active+clean; 19 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 532 op/s 2024-06-10T01:07:32.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:32 smithi037 ceph-mon[32192]: pgmap v756: 129 pgs: 129 active+clean; 19 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 532 op/s 2024-06-10T01:07:33.212 INFO:tasks.quiescer.fs.[cephfs]:Total duration (35.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-10T01:07:33.213 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.8 2024-06-10T01:07:33.213 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:07:33.522 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:34.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:34 smithi037 ceph-mon[32192]: pgmap v757: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 534 op/s 2024-06-10T01:07:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.539+0000 7fab357f4640 1 -- 172.21.15.37:0/1321870483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab30101af0 msgr2=0x7fab30101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.539+0000 7fab357f4640 1 --2- 172.21.15.37:0/1321870483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab30101af0 0x7fab30101f70 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fab24009900 tx=0x7fab2402a410 comp rx=0 tx=0).stop 2024-06-10T01:07:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.539+0000 7fab357f4640 1 -- 172.21.15.37:0/1321870483 shutdown_connections 2024-06-10T01:07:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.539+0000 7fab357f4640 1 --2- 172.21.15.37:0/1321870483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab301024b0 0x7fab30108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.539+0000 7fab357f4640 1 --2- 172.21.15.37:0/1321870483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab30101af0 0x7fab30101f70 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.539+0000 7fab357f4640 1 --2- 172.21.15.37:0/1321870483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab30100900 0x7fab30100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.539+0000 7fab357f4640 1 -- 172.21.15.37:0/1321870483 >> 172.21.15.37:0/1321870483 conn(0x7fab300fbe70 msgr2=0x7fab300fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.539+0000 7fab357f4640 1 -- 172.21.15.37:0/1321870483 shutdown_connections 2024-06-10T01:07:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.539+0000 7fab357f4640 1 -- 172.21.15.37:0/1321870483 wait complete. 2024-06-10T01:07:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.540+0000 7fab357f4640 1 Processor -- start 2024-06-10T01:07:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.540+0000 7fab357f4640 1 -- start start 2024-06-10T01:07:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.540+0000 7fab357f4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab30100900 0x7fab30197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.540+0000 7fab357f4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab30101af0 0x7fab30198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.540+0000 7fab357f4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab301024b0 0x7fab3019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.540+0000 7fab357f4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fab3010d6a0 con 0x7fab30100900 2024-06-10T01:07:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.540+0000 7fab357f4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab3010d520 con 0x7fab301024b0 2024-06-10T01:07:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.540+0000 7fab357f4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fab3010d820 con 0x7fab30101af0 2024-06-10T01:07:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.540+0000 7fab2effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab30100900 0x7fab30197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.540+0000 7fab2effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab30100900 0x7fab30197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42760/0 (socket says 172.21.15.37:42760) 2024-06-10T01:07:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.540+0000 7fab2effd640 1 -- 172.21.15.37:0/1923416601 learned_addr learned my addr 172.21.15.37:0/1923416601 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.540+0000 7fab2e7fc640 1 --2- 172.21.15.37:0/1923416601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab30101af0 0x7fab30198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.540+0000 7fab2f7fe640 1 --2- 172.21.15.37:0/1923416601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab301024b0 0x7fab3019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.540+0000 7fab2effd640 1 -- 172.21.15.37:0/1923416601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab30101af0 msgr2=0x7fab30198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.540+0000 7fab2effd640 1 --2- 172.21.15.37:0/1923416601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab30101af0 0x7fab30198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.540+0000 7fab2effd640 1 -- 172.21.15.37:0/1923416601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab301024b0 msgr2=0x7fab3019f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.540+0000 7fab2effd640 1 --2- 172.21.15.37:0/1923416601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab301024b0 0x7fab3019f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.540+0000 7fab2effd640 1 -- 172.21.15.37:0/1923416601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3019f830 con 0x7fab30100900 2024-06-10T01:07:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.540+0000 7fab2e7fc640 1 --2- 172.21.15.37:0/1923416601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab30101af0 0x7fab30198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.540+0000 7fab2f7fe640 1 --2- 172.21.15.37:0/1923416601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab301024b0 0x7fab3019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.541+0000 7fab2effd640 1 --2- 172.21.15.37:0/1923416601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab30100900 0x7fab30197c20 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fab2000ccb0 tx=0x7fab2000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.541+0000 7fab17fff640 1 -- 172.21.15.37:0/1923416601 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab20003b70 con 0x7fab30100900 2024-06-10T01:07:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.541+0000 7fab357f4640 1 -- 172.21.15.37:0/1923416601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab3019fb20 con 0x7fab30100900 2024-06-10T01:07:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.541+0000 7fab17fff640 1 -- 172.21.15.37:0/1923416601 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab2000ae50 con 0x7fab30100900 2024-06-10T01:07:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.541+0000 7fab17fff640 1 -- 172.21.15.37:0/1923416601 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2000f810 con 0x7fab30100900 2024-06-10T01:07:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.541+0000 7fab357f4640 1 -- 172.21.15.37:0/1923416601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab301a00c0 con 0x7fab30100900 2024-06-10T01:07:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.543+0000 7fab17fff640 1 -- 172.21.15.37:0/1923416601 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fab2000f9b0 con 0x7fab30100900 2024-06-10T01:07:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.544+0000 7fab385d7740 1 Processor -- start 2024-06-10T01:07:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.544+0000 7fab385d7740 1 -- start start 2024-06-10T01:07:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.544+0000 7fab385d7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7c720c010 0x55f7c720c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.544+0000 7fab385d7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7c720ca00 0x55f7c7211500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.544+0000 7fab385d7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7c7211b70 0x55f7c7214000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.544+0000 7fab385d7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f7c716f4f0 con 0x55f7c7211b70 2024-06-10T01:07:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.544+0000 7fab385d7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f7c710d870 con 0x55f7c720ca00 2024-06-10T01:07:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.544+0000 7fab385d7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f7c716b4b0 con 0x55f7c720c010 2024-06-10T01:07:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.544+0000 7fab2f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7c720c010 0x55f7c720c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.544+0000 7fab2e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7c7211b70 0x55f7c7214000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.544+0000 7fab2f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7c720c010 0x55f7c720c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:35228/0 (socket says 172.21.15.37:35228) 2024-06-10T01:07:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.544+0000 7fab17fff640 1 --2- 172.21.15.37:0/1923416601 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fab180a2c30 0x7fab180a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.544+0000 7fab2f7fe640 1 -- 172.21.15.37:0/227150210 learned_addr learned my addr 172.21.15.37:0/227150210 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.544+0000 7fab2effd640 1 --2- 172.21.15.37:0/227150210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7c720ca00 0x55f7c7211500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.545+0000 7fab2f7fe640 1 --2- 172.21.15.37:0/1923416601 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fab180a2c30 0x7fab180a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.545+0000 7fab2f7fe640 1 -- 172.21.15.37:0/227150210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7c720ca00 msgr2=0x55f7c7211500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.545+0000 7fab2f7fe640 1 --2- 172.21.15.37:0/227150210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7c720ca00 0x55f7c7211500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.545+0000 7fab2f7fe640 1 -- 172.21.15.37:0/227150210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7c7211b70 msgr2=0x55f7c7214000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.545+0000 7fab2f7fe640 1 --2- 172.21.15.37:0/227150210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7c7211b70 0x55f7c7214000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.545+0000 7fab17fff640 1 -- 172.21.15.37:0/1923416601 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fab200cf6a0 con 0x7fab30100900 2024-06-10T01:07:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.545+0000 7fab2f7fe640 1 -- 172.21.15.37:0/227150210 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7c71299b0 con 0x55f7c720c010 2024-06-10T01:07:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.545+0000 7fab2f7fe640 1 --2- 172.21.15.37:0/227150210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7c720c010 0x55f7c720c430 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fab280137a0 tx=0x7fab28013c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9396143ff5c0fd6e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.545+0000 7fab2f7fe640 1 --2- 172.21.15.37:0/1923416601 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fab180a2c30 0x7fab180a50f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fab2801e000 tx=0x7fab28020000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.545+0000 7fab157fa640 1 -- 172.21.15.37:0/227150210 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2801f850 con 0x55f7c720c010 2024-06-10T01:07:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.545+0000 7fab157fa640 1 -- 172.21.15.37:0/227150210 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab28023070 con 0x55f7c720c010 2024-06-10T01:07:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.545+0000 7fab157fa640 1 -- 172.21.15.37:0/227150210 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab280119e0 con 0x55f7c720c010 2024-06-10T01:07:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.546+0000 7fab385d7740 1 -- 172.21.15.37:0/227150210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7c720c010 msgr2=0x55f7c720c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.546+0000 7fab385d7740 1 --2- 172.21.15.37:0/227150210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7c720c010 0x55f7c720c430 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fab280137a0 tx=0x7fab28013c60 comp rx=0 tx=0).stop 2024-06-10T01:07:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.546+0000 7fab385d7740 1 -- 172.21.15.37:0/227150210 shutdown_connections 2024-06-10T01:07:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.546+0000 7fab385d7740 1 --2- 172.21.15.37:0/227150210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7c7211b70 0x55f7c7214000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.546+0000 7fab385d7740 1 --2- 172.21.15.37:0/227150210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7c720ca00 0x55f7c7211500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.546+0000 7fab385d7740 1 --2- 172.21.15.37:0/227150210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7c720c010 0x55f7c720c430 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.546+0000 7fab385d7740 1 -- 172.21.15.37:0/227150210 >> 172.21.15.37:0/227150210 conn(0x55f7c7207680 msgr2=0x55f7c7207a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.546+0000 7fab385d7740 1 -- 172.21.15.37:0/227150210 shutdown_connections 2024-06-10T01:07:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.546+0000 7fab385d7740 1 -- 172.21.15.37:0/227150210 wait complete. 2024-06-10T01:07:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.546+0000 7fab385d7740 1 -- start start 2024-06-10T01:07:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.547+0000 7fab385d7740 1 Processor -- start 2024-06-10T01:07:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.547+0000 7fab385d7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7c720c010 0x55f7c72161f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.547+0000 7fab385d7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7c720ca00 0x55f7c7216730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.547+0000 7fab385d7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7c7211b70 0x55f7c721d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.547+0000 7fab385d7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f7c7214ac0 con 0x55f7c720c010 2024-06-10T01:07:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.547+0000 7fab385d7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f7c70f8ae0 con 0x55f7c7211b70 2024-06-10T01:07:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.547+0000 7fab385d7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f7c7214c40 con 0x55f7c720ca00 2024-06-10T01:07:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.547+0000 7fab385d7740 10 client.0 _ll_register_callbacks cb 0x55f7c71319b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.547+0000 7fab2effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7c720c010 0x55f7c72161f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.547+0000 7fab2effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7c720c010 0x55f7c72161f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42778/0 (socket says 172.21.15.37:42778) 2024-06-10T01:07:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.547+0000 7fab2e7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7c720ca00 0x55f7c7216730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.547+0000 7fab2effd640 1 -- 172.21.15.37:0/3893971028 learned_addr learned my addr 172.21.15.37:0/3893971028 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.547+0000 7fab2f7fe640 1 --2- 172.21.15.37:0/3893971028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7c7211b70 0x55f7c721d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.547+0000 7fab2effd640 1 -- 172.21.15.37:0/3893971028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7c720ca00 msgr2=0x55f7c7216730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.547+0000 7fab2effd640 1 --2- 172.21.15.37:0/3893971028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7c720ca00 0x55f7c7216730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.547+0000 7fab2effd640 1 -- 172.21.15.37:0/3893971028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7c7211b70 msgr2=0x55f7c721d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.547+0000 7fab2effd640 1 --2- 172.21.15.37:0/3893971028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7c7211b70 0x55f7c721d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.547+0000 7fab2effd640 1 -- 172.21.15.37:0/3893971028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7c6e83210 con 0x55f7c720c010 2024-06-10T01:07:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.547+0000 7fab2e7fc640 1 --2- 172.21.15.37:0/3893971028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7c720ca00 0x55f7c7216730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.547+0000 7fab2f7fe640 1 --2- 172.21.15.37:0/3893971028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7c7211b70 0x55f7c721d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.547+0000 7fab2effd640 1 --2- 172.21.15.37:0/3893971028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7c720c010 0x55f7c72161f0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fab200ccd50 tx=0x7fab20007920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=737272a16fbf85d9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.547+0000 7fab12ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:07:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.548+0000 7fab12ff5640 1 -- 172.21.15.37:0/3893971028 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2001d040 con 0x55f7c720c010 2024-06-10T01:07:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.548+0000 7fab12ff5640 1 -- 172.21.15.37:0/3893971028 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fab200a4290 con 0x55f7c720c010 2024-06-10T01:07:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.548+0000 7fab385d7740 1 -- 172.21.15.37:0/3893971028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab28023070 con 0x55f7c720c010 2024-06-10T01:07:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.548+0000 7fab12ff5640 1 -- 172.21.15.37:0/3893971028 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fab20099bf0 con 0x55f7c720c010 2024-06-10T01:07:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.548+0000 7fab385d7740 10 client.17259 fetch_fsmap learned FSMap version 31 2024-06-10T01:07:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.548+0000 7fab385d7740 1 -- 172.21.15.37:0/3893971028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f7c6f4f620 con 0x55f7c720c010 2024-06-10T01:07:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.548+0000 7fab12ff5640 1 -- 172.21.15.37:0/3893971028 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab28023070 con 0x55f7c720c010 2024-06-10T01:07:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.548+0000 7fab12ff5640 1 -- 172.21.15.37:0/3893971028 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fab2009af00 con 0x55f7c720c010 2024-06-10T01:07:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.548+0000 7fab385d7740 10 client.17259 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:07:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.548+0000 7fab385d7740 10 client.17259 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.548+0000 7fab385d7740 20 client.17259 populate_metadata read hostname 'smithi037' 2024-06-10T01:07:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.548+0000 7fab385d7740 1 --2- 172.21.15.37:0/3893971028 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f7c7222000 0x55f7c7224450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.548+0000 7fab385d7740 4 client.17259 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.548+0000 7fab385d7740 1 -- 172.21.15.37:0/3893971028 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7c7182f10 con 0x55f7c7222000 2024-06-10T01:07:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.548+0000 7fab2e7fc640 1 --2- 172.21.15.37:0/3893971028 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f7c7222000 0x55f7c7224450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.549+0000 7fab2e7fc640 1 --2- 172.21.15.37:0/3893971028 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f7c7222000 0x55f7c7224450 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f387d7ff68cffa4b server_cookie=8cc6ea70fb79f39c in_seq=0 out_seq=0 2024-06-10T01:07:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.549+0000 7fab12ff5640 10 client.17259 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.555+0000 7fab12ff5640 1 -- 172.21.15.37:0/3893971028 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f7c7182f10 con 0x55f7c7222000 2024-06-10T01:07:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.555+0000 7fab12ff5640 10 client.17259 handle_command_reply: tid=0 2024-06-10T01:07:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.555+0000 7fab385d7740 1 client.17259 shutdown 2024-06-10T01:07:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.555+0000 7fab385d7740 1 -- 172.21.15.37:0/3893971028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7c720c010 msgr2=0x55f7c72161f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.555+0000 7fab385d7740 1 --2- 172.21.15.37:0/3893971028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7c720c010 0x55f7c72161f0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fab200ccd50 tx=0x7fab20007920 comp rx=0 tx=0).stop 2024-06-10T01:07:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.555+0000 7fab385d7740 1 -- 172.21.15.37:0/3893971028 shutdown_connections 2024-06-10T01:07:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.555+0000 7fab385d7740 1 --2- 172.21.15.37:0/3893971028 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f7c7222000 0x55f7c7224450 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.555+0000 7fab385d7740 1 --2- 172.21.15.37:0/3893971028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7c7211b70 0x55f7c721d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.555+0000 7fab385d7740 1 --2- 172.21.15.37:0/3893971028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7c720ca00 0x55f7c7216730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.555+0000 7fab385d7740 1 --2- 172.21.15.37:0/3893971028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7c720c010 0x55f7c72161f0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.555+0000 7fab12ff5640 0 client.17259 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.555+0000 7fab385d7740 1 -- 172.21.15.37:0/3893971028 >> 172.21.15.37:0/3893971028 conn(0x55f7c7207680 msgr2=0x55f7c72067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.556+0000 7fab385d7740 1 -- 172.21.15.37:0/3893971028 shutdown_connections 2024-06-10T01:07:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.556+0000 7fab385d7740 1 -- 172.21.15.37:0/3893971028 wait complete. 2024-06-10T01:07:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.556+0000 7fab385d7740 20 client.17259 trim_cache size 0 max 16384 2024-06-10T01:07:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.571+0000 7fab385d7740 1 Processor -- start 2024-06-10T01:07:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.571+0000 7fab385d7740 1 -- start start 2024-06-10T01:07:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.571+0000 7fab385d7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7c720c010 0x55f7c720dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.571+0000 7fab385d7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7c720ca00 0x55f7c721d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.571+0000 7fab385d7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7c7211b70 0x55f7c720e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.571+0000 7fab385d7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fab301acec0 con 0x55f7c7211b70 2024-06-10T01:07:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.571+0000 7fab385d7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab301aa790 con 0x55f7c720c010 2024-06-10T01:07:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.571+0000 7fab385d7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fab301a85e0 con 0x55f7c720ca00 2024-06-10T01:07:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.572+0000 7fab2f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7c7211b70 0x55f7c720e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.572+0000 7fab2f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7c7211b70 0x55f7c720e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42794/0 (socket says 172.21.15.37:42794) 2024-06-10T01:07:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.572+0000 7fab2e7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7c720ca00 0x55f7c721d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.572+0000 7fab2f7fe640 1 -- 172.21.15.37:0/397761605 learned_addr learned my addr 172.21.15.37:0/397761605 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.572+0000 7fab2effd640 1 --2- 172.21.15.37:0/397761605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7c720c010 0x55f7c720dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.572+0000 7fab2f7fe640 1 -- 172.21.15.37:0/397761605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7c720ca00 msgr2=0x55f7c721d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.572+0000 7fab2f7fe640 1 --2- 172.21.15.37:0/397761605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7c720ca00 0x55f7c721d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.572+0000 7fab2f7fe640 1 -- 172.21.15.37:0/397761605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7c720c010 msgr2=0x55f7c720dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.572+0000 7fab2f7fe640 1 --2- 172.21.15.37:0/397761605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7c720c010 0x55f7c720dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.572+0000 7fab2f7fe640 1 -- 172.21.15.37:0/397761605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7c72074b0 con 0x55f7c7211b70 2024-06-10T01:07:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.572+0000 7fab2f7fe640 1 --2- 172.21.15.37:0/397761605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7c7211b70 0x55f7c720e120 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fab28000c00 tx=0x7fab28011430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56670b7334a6c91d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.572+0000 7fab14ff9640 1 -- 172.21.15.37:0/397761605 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28013080 con 0x55f7c7211b70 2024-06-10T01:07:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.572+0000 7fab14ff9640 1 -- 172.21.15.37:0/397761605 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab28023040 con 0x55f7c7211b70 2024-06-10T01:07:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.572+0000 7fab14ff9640 1 -- 172.21.15.37:0/397761605 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28011810 con 0x55f7c7211b70 2024-06-10T01:07:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.573+0000 7fab385d7740 1 -- 172.21.15.37:0/397761605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7c7211b70 msgr2=0x55f7c720e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.573+0000 7fab385d7740 1 --2- 172.21.15.37:0/397761605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7c7211b70 0x55f7c720e120 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fab28000c00 tx=0x7fab28011430 comp rx=0 tx=0).stop 2024-06-10T01:07:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.573+0000 7fab385d7740 1 -- 172.21.15.37:0/397761605 shutdown_connections 2024-06-10T01:07:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.573+0000 7fab385d7740 1 --2- 172.21.15.37:0/397761605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7c7211b70 0x55f7c720e120 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.573+0000 7fab385d7740 1 --2- 172.21.15.37:0/397761605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7c720ca00 0x55f7c721d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.573+0000 7fab385d7740 1 --2- 172.21.15.37:0/397761605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7c720c010 0x55f7c720dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.573+0000 7fab385d7740 1 -- 172.21.15.37:0/397761605 >> 172.21.15.37:0/397761605 conn(0x55f7c7207680 msgr2=0x55f7c7207a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.573+0000 7fab385d7740 1 -- 172.21.15.37:0/397761605 shutdown_connections 2024-06-10T01:07:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.573+0000 7fab385d7740 1 -- 172.21.15.37:0/397761605 wait complete. 2024-06-10T01:07:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.573+0000 7fab385d7740 1 -- start start 2024-06-10T01:07:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.574+0000 7fab385d7740 1 Processor -- start 2024-06-10T01:07:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.574+0000 7fab385d7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7c720c010 0x55f7c721a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.574+0000 7fab385d7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7c720ca00 0x55f7c721a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.574+0000 7fab385d7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7c7211b70 0x55f7c728acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.574+0000 7fab385d7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fab301a8c50 con 0x55f7c7211b70 2024-06-10T01:07:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.574+0000 7fab385d7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab301aa0c0 con 0x55f7c720c010 2024-06-10T01:07:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.574+0000 7fab385d7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fab301a9320 con 0x55f7c720ca00 2024-06-10T01:07:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.574+0000 7fab385d7740 10 client.0 _ll_register_callbacks cb 0x7fab20004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.574+0000 7fab2f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7c7211b70 0x55f7c728acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.574+0000 7fab2e7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7c720ca00 0x55f7c721a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.574+0000 7fab2f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7c7211b70 0x55f7c728acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42808/0 (socket says 172.21.15.37:42808) 2024-06-10T01:07:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.574+0000 7fab2e7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7c720ca00 0x55f7c721a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:35268/0 (socket says 172.21.15.37:35268) 2024-06-10T01:07:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.574+0000 7fab2f7fe640 1 -- 172.21.15.37:0/2414553369 learned_addr learned my addr 172.21.15.37:0/2414553369 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.574+0000 7fab2effd640 1 --2- 172.21.15.37:0/2414553369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7c720c010 0x55f7c721a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.574+0000 7fab2f7fe640 1 -- 172.21.15.37:0/2414553369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7c720ca00 msgr2=0x55f7c721a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.574+0000 7fab2f7fe640 1 --2- 172.21.15.37:0/2414553369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7c720ca00 0x55f7c721a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.574+0000 7fab2f7fe640 1 -- 172.21.15.37:0/2414553369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7c720c010 msgr2=0x55f7c721a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.574+0000 7fab2f7fe640 1 --2- 172.21.15.37:0/2414553369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7c720c010 0x55f7c721a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.574+0000 7fab2f7fe640 1 -- 172.21.15.37:0/2414553369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7c721d820 con 0x55f7c7211b70 2024-06-10T01:07:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.574+0000 7fab2e7fc640 1 --2- 172.21.15.37:0/2414553369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7c720ca00 0x55f7c721a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.575+0000 7fab2f7fe640 1 --2- 172.21.15.37:0/2414553369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7c7211b70 0x55f7c728acf0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fab28009530 tx=0x7fab28004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ef2dcb53952308 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.575+0000 7fab12ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:07:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.575+0000 7fab12ff5640 1 -- 172.21.15.37:0/2414553369 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28013080 con 0x55f7c7211b70 2024-06-10T01:07:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.575+0000 7fab12ff5640 1 -- 172.21.15.37:0/2414553369 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fab28009290 con 0x55f7c7211b70 2024-06-10T01:07:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.575+0000 7fab385d7740 1 -- 172.21.15.37:0/2414553369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab28023040 con 0x55f7c7211b70 2024-06-10T01:07:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.575+0000 7fab12ff5640 1 -- 172.21.15.37:0/2414553369 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fab2800fe90 con 0x55f7c7211b70 2024-06-10T01:07:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.575+0000 7fab385d7740 10 client.17265 fetch_fsmap learned FSMap version 31 2024-06-10T01:07:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.575+0000 7fab385d7740 1 -- 172.21.15.37:0/2414553369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f7c728c0f0 con 0x55f7c7211b70 2024-06-10T01:07:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.575+0000 7fab12ff5640 1 -- 172.21.15.37:0/2414553369 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab28023040 con 0x55f7c7211b70 2024-06-10T01:07:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.575+0000 7fab12ff5640 1 -- 172.21.15.37:0/2414553369 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fab28011c20 con 0x55f7c7211b70 2024-06-10T01:07:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.575+0000 7fab385d7740 10 client.17265 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:07:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.575+0000 7fab385d7740 10 client.17265 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.575+0000 7fab385d7740 20 client.17265 populate_metadata read hostname 'smithi037' 2024-06-10T01:07:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.575+0000 7fab385d7740 1 --2- 172.21.15.37:0/2414553369 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f7c728c9a0 0x55f7c728edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.575+0000 7fab385d7740 4 client.17265 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:07:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.575+0000 7fab385d7740 1 -- 172.21.15.37:0/2414553369 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f7c717cc20 con 0x55f7c728c9a0 2024-06-10T01:07:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.576+0000 7fab2effd640 1 --2- 172.21.15.37:0/2414553369 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f7c728c9a0 0x55f7c728edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.576+0000 7fab2effd640 1 --2- 172.21.15.37:0/2414553369 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f7c728c9a0 0x55f7c728edc0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=321857d76ffd617a server_cookie=edc9b50e73f887bd in_seq=0 out_seq=0 2024-06-10T01:07:34.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:34.576+0000 7fab12ff5640 10 client.17265 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:34.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:34 smithi167 ceph-mon[36917]: pgmap v757: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 534 op/s 2024-06-10T01:07:34.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:34 smithi094 ceph-mon[36708]: pgmap v757: 129 pgs: 129 active+clean; 23 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 534 op/s 2024-06-10T01:07:35.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.749+0000 7f055fadd700 1 -- 172.21.15.37:0/647154335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f055806a6e0 msgr2=0x7f055806ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:35.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.749+0000 7f055fadd700 1 --2- 172.21.15.37:0/647154335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f055806a6e0 0x7f055806ab40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f05480098f0 tx=0x7f0548009c00 comp rx=0 tx=0).stop 2024-06-10T01:07:35.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.750+0000 7f055fadd700 1 -- 172.21.15.37:0/647154335 shutdown_connections 2024-06-10T01:07:35.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.750+0000 7f055fadd700 1 --2- 172.21.15.37:0/647154335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0558104ed0 0x7f0558105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.750+0000 7f055fadd700 1 --2- 172.21.15.37:0/647154335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05581073c0 0x7f055806a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.750+0000 7f055fadd700 1 --2- 172.21.15.37:0/647154335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f055806a6e0 0x7f055806ab40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.750+0000 7f055fadd700 1 -- 172.21.15.37:0/647154335 >> 172.21.15.37:0/647154335 conn(0x7f0558078340 msgr2=0x7f0558078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:35.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.750+0000 7f055fadd700 1 -- 172.21.15.37:0/647154335 shutdown_connections 2024-06-10T01:07:35.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.750+0000 7f055fadd700 1 -- 172.21.15.37:0/647154335 wait complete. 2024-06-10T01:07:35.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.751+0000 7f055fadd700 1 Processor -- start 2024-06-10T01:07:35.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.751+0000 7f055fadd700 1 -- start start 2024-06-10T01:07:35.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.751+0000 7f055fadd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f055806a6e0 0x7f05581156a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:35.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.751+0000 7f055fadd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0558104ed0 0x7f0558115be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:35.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.751+0000 7f055fadd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05581073c0 0x7f055810e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:35.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.751+0000 7f055fadd700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0558118e70 con 0x7f05581073c0 2024-06-10T01:07:35.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.751+0000 7f055fadd700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0558118cf0 con 0x7f055806a6e0 2024-06-10T01:07:35.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.751+0000 7f055fadd700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0558118ff0 con 0x7f0558104ed0 2024-06-10T01:07:35.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.752+0000 7f055e07a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05581073c0 0x7f055810e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:35.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.752+0000 7f055e07a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05581073c0 0x7f055810e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42828/0 (socket says 172.21.15.37:42828) 2024-06-10T01:07:35.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.752+0000 7f055e07a700 1 -- 172.21.15.37:0/111630822 learned_addr learned my addr 172.21.15.37:0/111630822 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:35.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.752+0000 7f055d078700 1 --2- 172.21.15.37:0/111630822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0558104ed0 0x7f0558115be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:35.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.753+0000 7f055d879700 1 --2- 172.21.15.37:0/111630822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f055806a6e0 0x7f05581156a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:35.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.753+0000 7f055e07a700 1 -- 172.21.15.37:0/111630822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0558104ed0 msgr2=0x7f0558115be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:35.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.753+0000 7f055e07a700 1 --2- 172.21.15.37:0/111630822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0558104ed0 0x7f0558115be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.753+0000 7f055e07a700 1 -- 172.21.15.37:0/111630822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f055806a6e0 msgr2=0x7f05581156a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:35.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.753+0000 7f055e07a700 1 --2- 172.21.15.37:0/111630822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f055806a6e0 0x7f05581156a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:35.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.753+0000 7f055e07a700 1 -- 172.21.15.37:0/111630822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055810f1c0 con 0x7f05581073c0 2024-06-10T01:07:35.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.753+0000 7f055e07a700 1 --2- 172.21.15.37:0/111630822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05581073c0 0x7f055810e8c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f055000c110 tx=0x7f055000c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:35.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.754+0000 7f055d078700 1 --2- 172.21.15.37:0/111630822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0558104ed0 0x7f0558115be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:35.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.754+0000 7f054effd700 1 -- 172.21.15.37:0/111630822 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055000ce50 con 0x7f05581073c0 2024-06-10T01:07:35.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.754+0000 7f055fadd700 1 -- 172.21.15.37:0/111630822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055810f450 con 0x7f05581073c0 2024-06-10T01:07:35.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.754+0000 7f054effd700 1 -- 172.21.15.37:0/111630822 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0550004bb0 con 0x7f05581073c0 2024-06-10T01:07:35.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.754+0000 7f054effd700 1 -- 172.21.15.37:0/111630822 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0550013400 con 0x7f05581073c0 2024-06-10T01:07:35.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.754+0000 7f055fadd700 1 -- 172.21.15.37:0/111630822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05581133a0 con 0x7f05581073c0 2024-06-10T01:07:35.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.754+0000 7f055d879700 1 --2- 172.21.15.37:0/111630822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f055806a6e0 0x7f05581156a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:07:35.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.755+0000 7f055fadd700 1 -- 172.21.15.37:0/111630822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f053c005330 con 0x7f05581073c0 2024-06-10T01:07:35.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.757+0000 7f054effd700 1 -- 172.21.15.37:0/111630822 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f055001a020 con 0x7f05581073c0 2024-06-10T01:07:35.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.758+0000 7f054effd700 1 --2- 172.21.15.37:0/111630822 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f05440a2ce0 0x7f05440a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:35.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.759+0000 7f054effd700 1 -- 172.21.15.37:0/111630822 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f05500ceea0 con 0x7f05581073c0 2024-06-10T01:07:35.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.762+0000 7f055d879700 1 --2- 172.21.15.37:0/111630822 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f05440a2ce0 0x7f05440a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:35.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.764+0000 7f055d879700 1 --2- 172.21.15.37:0/111630822 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f05440a2ce0 0x7f05440a51a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0554007900 tx=0x7f0554008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:35.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:35.766+0000 7f054effd700 1 -- 172.21.15.37:0/111630822 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0550010040 con 0x7f05581073c0 2024-06-10T01:07:36.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:36.033+0000 7f055fadd700 1 -- 172.21.15.37:0/111630822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f053c005620 con 0x7f05581073c0 2024-06-10T01:07:36.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:36.034+0000 7f054effd700 1 -- 172.21.15.37:0/111630822 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11868 (secure 0 0 0) 0x7f0550095e20 con 0x7f05581073c0 2024-06-10T01:07:36.030 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:07:36.030 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":26554,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2107277223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2107277223},{"type":"v1","addr":"172.21.15.167:6835","nonce":2107277223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T01:06:09.604504+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24472,24451,24458]},"id":1}]} 2024-06-10T01:07:36.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:36.036+0000 7f055fadd700 1 -- 172.21.15.37:0/111630822 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f05440a2ce0 msgr2=0x7f05440a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:36.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:36.036+0000 7f055fadd700 1 --2- 172.21.15.37:0/111630822 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f05440a2ce0 0x7f05440a51a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0554007900 tx=0x7f0554008040 comp rx=0 tx=0).stop 2024-06-10T01:07:36.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:36.036+0000 7f055fadd700 1 -- 172.21.15.37:0/111630822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05581073c0 msgr2=0x7f055810e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:36.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:36.036+0000 7f055fadd700 1 --2- 172.21.15.37:0/111630822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05581073c0 0x7f055810e8c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f055000c110 tx=0x7f055000c4d0 comp rx=0 tx=0).stop 2024-06-10T01:07:36.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:36.036+0000 7f055fadd700 1 -- 172.21.15.37:0/111630822 shutdown_connections 2024-06-10T01:07:36.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:36.036+0000 7f055fadd700 1 --2- 172.21.15.37:0/111630822 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f05440a2ce0 0x7f05440a51a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:36.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:36.036+0000 7f055fadd700 1 --2- 172.21.15.37:0/111630822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05581073c0 0x7f055810e8c0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:36.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:36.036+0000 7f055fadd700 1 --2- 172.21.15.37:0/111630822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f055806a6e0 0x7f05581156a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:36.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:36.036+0000 7f055fadd700 1 --2- 172.21.15.37:0/111630822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0558104ed0 0x7f0558115be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:36.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:36.036+0000 7f055fadd700 1 -- 172.21.15.37:0/111630822 >> 172.21.15.37:0/111630822 conn(0x7f0558078340 msgr2=0x7f0558103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:36.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:36.037+0000 7f055fadd700 1 -- 172.21.15.37:0/111630822 shutdown_connections 2024-06-10T01:07:36.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:36.037+0000 7f055fadd700 1 -- 172.21.15.37:0/111630822 wait complete. 2024-06-10T01:07:36.034 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 31 2024-06-10T01:07:36.343 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:36 smithi037 ceph-mon[32192]: pgmap v758: 129 pgs: 129 active+clean; 25 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 1.0 MiB/s wr, 541 op/s 2024-06-10T01:07:36.343 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:36 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/111630822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:07:36.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:36 smithi167 ceph-mon[36917]: pgmap v758: 129 pgs: 129 active+clean; 25 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 1.0 MiB/s wr, 541 op/s 2024-06-10T01:07:36.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:36 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/111630822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:07:36.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:36 smithi094 ceph-mon[36708]: pgmap v758: 129 pgs: 129 active+clean; 25 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 1.0 MiB/s wr, 541 op/s 2024-06-10T01:07:36.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:36 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/111630822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:07:36.803 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24472 quiesce db / --timeout 90 --expiration 137.8 --await-for 45.0' 2024-06-10T01:07:36.803 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24472 quiesce db / --timeout 90 --expiration 137.8 --await-for 45.0 2024-06-10T01:07:37.370 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:38.555 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:38 smithi037 ceph-mon[32192]: pgmap v759: 129 pgs: 129 active+clean; 26 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 446 op/s 2024-06-10T01:07:38.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:38 smithi167 ceph-mon[36917]: pgmap v759: 129 pgs: 129 active+clean; 26 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 446 op/s 2024-06-10T01:07:38.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:38 smithi094 ceph-mon[36708]: pgmap v759: 129 pgs: 129 active+clean; 26 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 446 op/s 2024-06-10T01:07:39.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.532+0000 7f58d00f8700 1 -- 172.21.15.37:0/914295436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58c8107920 msgr2=0x7f58c806b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:39.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.532+0000 7f58d00f8700 1 --2- 172.21.15.37:0/914295436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58c8107920 0x7f58c806b0a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f58b8009990 tx=0x7f58b8009ca0 comp rx=0 tx=0).stop 2024-06-10T01:07:39.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.532+0000 7f58d00f8700 1 -- 172.21.15.37:0/914295436 shutdown_connections 2024-06-10T01:07:39.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.532+0000 7f58d00f8700 1 --2- 172.21.15.37:0/914295436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58c8104f20 0x7f58c81053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:39.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.532+0000 7f58d00f8700 1 --2- 172.21.15.37:0/914295436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58c81074b0 0x7f58c806ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:39.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.532+0000 7f58d00f8700 1 --2- 172.21.15.37:0/914295436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58c8107920 0x7f58c806b0a0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:39.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.532+0000 7f58d00f8700 1 -- 172.21.15.37:0/914295436 >> 172.21.15.37:0/914295436 conn(0x7f58c8078340 msgr2=0x7f58c8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:39.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.532+0000 7f58d00f8700 1 -- 172.21.15.37:0/914295436 shutdown_connections 2024-06-10T01:07:39.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.532+0000 7f58d00f8700 1 -- 172.21.15.37:0/914295436 wait complete. 2024-06-10T01:07:39.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.533+0000 7f58d00f8700 1 Processor -- start 2024-06-10T01:07:39.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.533+0000 7f58d00f8700 1 -- start start 2024-06-10T01:07:39.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.533+0000 7f58d00f8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58c8104f20 0x7f58c819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:39.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.533+0000 7f58d00f8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58c81074b0 0x7f58c819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:39.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.533+0000 7f58d00f8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58c8107920 0x7f58c81a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:39.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.533+0000 7f58d00f8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f58c8118d50 con 0x7f58c8107920 2024-06-10T01:07:39.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.533+0000 7f58d00f8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f58c8118bd0 con 0x7f58c81074b0 2024-06-10T01:07:39.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.533+0000 7f58d00f8700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f58c8118ed0 con 0x7f58c8104f20 2024-06-10T01:07:39.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.533+0000 7f58cde94700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58c8104f20 0x7f58c819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:39.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.533+0000 7f58ce695700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58c8107920 0x7f58c81a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:39.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.533+0000 7f58cde94700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58c8104f20 0x7f58c819f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:35312/0 (socket says 172.21.15.37:35312) 2024-06-10T01:07:39.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.533+0000 7f58cde94700 1 -- 172.21.15.37:0/453100000 learned_addr learned my addr 172.21.15.37:0/453100000 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:39.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.534+0000 7f58cd693700 1 --2- 172.21.15.37:0/453100000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58c81074b0 0x7f58c819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:39.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.534+0000 7f58ce695700 1 -- 172.21.15.37:0/453100000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58c8104f20 msgr2=0x7f58c819f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:39.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.534+0000 7f58ce695700 1 --2- 172.21.15.37:0/453100000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58c8104f20 0x7f58c819f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.534+0000 7f58ce695700 1 -- 172.21.15.37:0/453100000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58c81074b0 msgr2=0x7f58c819f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.534+0000 7f58ce695700 1 --2- 172.21.15.37:0/453100000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58c81074b0 0x7f58c819f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.534+0000 7f58ce695700 1 -- 172.21.15.37:0/453100000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c81a4060 con 0x7f58c8107920 2024-06-10T01:07:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.534+0000 7f58cd693700 1 --2- 172.21.15.37:0/453100000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58c81074b0 0x7f58c819f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.534+0000 7f58ce695700 1 --2- 172.21.15.37:0/453100000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58c8107920 0x7f58c81a3920 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f58c000efc0 tx=0x7f58c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.534+0000 7f58beffd700 1 -- 172.21.15.37:0/453100000 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c00092e0 con 0x7f58c8107920 2024-06-10T01:07:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.534+0000 7f58beffd700 1 -- 172.21.15.37:0/453100000 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58c0004bb0 con 0x7f58c8107920 2024-06-10T01:07:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.534+0000 7f58beffd700 1 -- 172.21.15.37:0/453100000 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c0004280 con 0x7f58c8107920 2024-06-10T01:07:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.534+0000 7f58d00f8700 1 -- 172.21.15.37:0/453100000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c81a42f0 con 0x7f58c8107920 2024-06-10T01:07:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.534+0000 7f58d00f8700 1 -- 172.21.15.37:0/453100000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c81abbd0 con 0x7f58c8107920 2024-06-10T01:07:39.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.537+0000 7f58beffd700 1 -- 172.21.15.37:0/453100000 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f58c001b030 con 0x7f58c8107920 2024-06-10T01:07:39.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.537+0000 7f58beffd700 1 --2- 172.21.15.37:0/453100000 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f58b40a2d40 0x7f58b40a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:39.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.537+0000 7f58beffd700 1 -- 172.21.15.37:0/453100000 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f58c00cef70 con 0x7f58c8107920 2024-06-10T01:07:39.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.537+0000 7f58cde94700 1 --2- 172.21.15.37:0/453100000 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f58b40a2d40 0x7f58b40a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:39.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.538+0000 7f58cde94700 1 --2- 172.21.15.37:0/453100000 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f58b40a2d40 0x7f58b40a5200 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f58c4007c90 tx=0x7f58c4007480 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:39.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.538+0000 7f58da97bb80 1 Processor -- start 2024-06-10T01:07:39.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.538+0000 7f58da97bb80 1 -- start start 2024-06-10T01:07:39.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.539+0000 7f58da97bb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557714039cd0 0x55771403a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:39.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.539+0000 7f58da97bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55771403a5f0 0x557714033f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:39.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.539+0000 7f58da97bb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557714034460 0x5577140348a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:39.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.539+0000 7f58da97bb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557713f859a0 con 0x557714039cd0 2024-06-10T01:07:39.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.539+0000 7f58da97bb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557713f2d8b0 con 0x557714034460 2024-06-10T01:07:39.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.539+0000 7f58da97bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557713f81960 con 0x55771403a5f0 2024-06-10T01:07:39.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.539+0000 7f58cde94700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557714039cd0 0x55771403a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:39.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.539+0000 7f58cd693700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55771403a5f0 0x557714033f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:39.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.539+0000 7f58cd693700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55771403a5f0 0x557714033f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:35316/0 (socket says 172.21.15.37:35316) 2024-06-10T01:07:39.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.539+0000 7f58cde94700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557714039cd0 0x55771403a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42858/0 (socket says 172.21.15.37:42858) 2024-06-10T01:07:39.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.539+0000 7f58cd693700 1 -- 172.21.15.37:0/266068837 learned_addr learned my addr 172.21.15.37:0/266068837 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:39.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.539+0000 7f58cd693700 1 -- 172.21.15.37:0/266068837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557714034460 msgr2=0x5577140348a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:39.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.539+0000 7f58cd693700 1 --2- 172.21.15.37:0/266068837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557714034460 0x5577140348a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:39.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.539+0000 7f58cd693700 1 -- 172.21.15.37:0/266068837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557714039cd0 msgr2=0x55771403a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:39.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.539+0000 7f58cd693700 1 --2- 172.21.15.37:0/266068837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557714039cd0 0x55771403a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:39.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.539+0000 7f58cd693700 1 -- 172.21.15.37:0/266068837 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557713f240e0 con 0x55771403a5f0 2024-06-10T01:07:39.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.539+0000 7f58cde94700 1 --2- 172.21.15.37:0/266068837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557714039cd0 0x55771403a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:39.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.539+0000 7f58cd693700 1 --2- 172.21.15.37:0/266068837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55771403a5f0 0x557714033f20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f58c80795c0 tx=0x7f58b800b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2c6254bb25f810 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:39.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.540+0000 7f58b3fff700 1 -- 172.21.15.37:0/266068837 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b8026020 con 0x55771403a5f0 2024-06-10T01:07:39.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.540+0000 7f58b3fff700 1 -- 172.21.15.37:0/266068837 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58b800b680 con 0x55771403a5f0 2024-06-10T01:07:39.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.540+0000 7f58b3fff700 1 -- 172.21.15.37:0/266068837 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b8005580 con 0x55771403a5f0 2024-06-10T01:07:39.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.540+0000 7f58da97bb80 1 -- 172.21.15.37:0/266068837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55771403a5f0 msgr2=0x557714033f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:39.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.540+0000 7f58da97bb80 1 --2- 172.21.15.37:0/266068837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55771403a5f0 0x557714033f20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f58c80795c0 tx=0x7f58b800b610 comp rx=0 tx=0).stop 2024-06-10T01:07:39.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.540+0000 7f58da97bb80 1 -- 172.21.15.37:0/266068837 shutdown_connections 2024-06-10T01:07:39.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.540+0000 7f58da97bb80 1 --2- 172.21.15.37:0/266068837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557714034460 0x5577140348a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:39.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.540+0000 7f58da97bb80 1 --2- 172.21.15.37:0/266068837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557714039cd0 0x55771403a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:39.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.540+0000 7f58da97bb80 1 --2- 172.21.15.37:0/266068837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55771403a5f0 0x557714033f20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:39.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.540+0000 7f58da97bb80 1 -- 172.21.15.37:0/266068837 >> 172.21.15.37:0/266068837 conn(0x557713f5ee50 msgr2=0x55771402f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:39.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.541+0000 7f58da97bb80 1 -- 172.21.15.37:0/266068837 shutdown_connections 2024-06-10T01:07:39.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.541+0000 7f58da97bb80 1 -- 172.21.15.37:0/266068837 wait complete. 2024-06-10T01:07:39.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.541+0000 7f58da97bb80 1 -- start start 2024-06-10T01:07:39.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.541+0000 7f58da97bb80 1 Processor -- start 2024-06-10T01:07:39.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.542+0000 7f58da97bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557714034460 0x557714040350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:39.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.542+0000 7f58da97bb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557714039cd0 0x55771403d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:39.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.542+0000 7f58da97bb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55771403a5f0 0x55771403db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:39.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.542+0000 7f58da97bb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557714041060 con 0x55771403a5f0 2024-06-10T01:07:39.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.542+0000 7f58da97bb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557713f25d80 con 0x557714039cd0 2024-06-10T01:07:39.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.542+0000 7f58da97bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5577140411e0 con 0x557714034460 2024-06-10T01:07:39.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.542+0000 7f58da97bb80 10 client.0 _ll_register_callbacks cb 0x557713f1fab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:39.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.542+0000 7f58cde94700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557714034460 0x557714040350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:39.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.542+0000 7f58cde94700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557714034460 0x557714040350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:35326/0 (socket says 172.21.15.37:35326) 2024-06-10T01:07:39.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.542+0000 7f58cde94700 1 -- 172.21.15.37:0/1565447440 learned_addr learned my addr 172.21.15.37:0/1565447440 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:39.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.542+0000 7f58ce695700 1 --2- 172.21.15.37:0/1565447440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55771403a5f0 0x55771403db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:39.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.542+0000 7f58cd693700 1 --2- 172.21.15.37:0/1565447440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557714039cd0 0x55771403d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:39.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.542+0000 7f58cde94700 1 -- 172.21.15.37:0/1565447440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557714039cd0 msgr2=0x55771403d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:39.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.542+0000 7f58cde94700 1 --2- 172.21.15.37:0/1565447440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557714039cd0 0x55771403d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.542+0000 7f58cde94700 1 -- 172.21.15.37:0/1565447440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55771403a5f0 msgr2=0x55771403db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.542+0000 7f58cde94700 1 --2- 172.21.15.37:0/1565447440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55771403a5f0 0x55771403db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.542+0000 7f58cde94700 1 -- 172.21.15.37:0/1565447440 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557713ed4a60 con 0x557714034460 2024-06-10T01:07:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.542+0000 7f58cde94700 1 --2- 172.21.15.37:0/1565447440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557714034460 0x557714040350 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f58c40118f0 tx=0x7f58c40047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a300fba69f2c0fd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.542+0000 7f58b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:07:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.543+0000 7f58b17fa700 1 -- 172.21.15.37:0/1565447440 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4028670 con 0x557714034460 2024-06-10T01:07:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.543+0000 7f58da97bb80 1 -- 172.21.15.37:0/1565447440 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58b800b680 con 0x557714034460 2024-06-10T01:07:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.543+0000 7f58b17fa700 1 -- 172.21.15.37:0/1565447440 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f58c4028810 con 0x557714034460 2024-06-10T01:07:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.543+0000 7f58b17fa700 1 -- 172.21.15.37:0/1565447440 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f58c4031050 con 0x557714034460 2024-06-10T01:07:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.543+0000 7f58da97bb80 10 client.26806 fetch_fsmap learned FSMap version 31 2024-06-10T01:07:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.543+0000 7f58da97bb80 1 -- 172.21.15.37:0/1565447440 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55771404ba10 con 0x557714034460 2024-06-10T01:07:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.543+0000 7f58b17fa700 1 -- 172.21.15.37:0/1565447440 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58b800b680 con 0x557714034460 2024-06-10T01:07:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.543+0000 7f58b17fa700 1 -- 172.21.15.37:0/1565447440 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f58c401fc00 con 0x557714034460 2024-06-10T01:07:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.543+0000 7f58da97bb80 10 client.26806 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:07:39.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.543+0000 7f58da97bb80 10 client.26806 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:07:39.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.543+0000 7f58da97bb80 20 client.26806 populate_metadata read hostname 'smithi037' 2024-06-10T01:07:39.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.543+0000 7f58da97bb80 1 --2- 172.21.15.37:0/1565447440 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55771404e3f0 0x557714050810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:39.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.543+0000 7f58da97bb80 4 client.26806 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:39.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.543+0000 7f58da97bb80 1 -- 172.21.15.37:0/1565447440 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557713f92ea0 con 0x55771404e3f0 2024-06-10T01:07:39.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.543+0000 7f58cd693700 1 --2- 172.21.15.37:0/1565447440 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55771404e3f0 0x557714050810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:39.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.544+0000 7f58cd693700 1 --2- 172.21.15.37:0/1565447440 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55771404e3f0 0x557714050810 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b1190ca702d301b server_cookie=32fde4fba9919acd in_seq=0 out_seq=0 2024-06-10T01:07:39.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.544+0000 7f58b17fa700 10 client.26806 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:39.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.577+0000 7f58b17fa700 1 -- 172.21.15.37:0/1565447440 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557713f92ea0 con 0x55771404e3f0 2024-06-10T01:07:39.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.577+0000 7f58b17fa700 10 client.26806 handle_command_reply: tid=0 2024-06-10T01:07:39.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.577+0000 7f58da97bb80 1 client.26806 shutdown 2024-06-10T01:07:39.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.577+0000 7f58da97bb80 1 -- 172.21.15.37:0/1565447440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557714034460 msgr2=0x557714040350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:39.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.577+0000 7f58da97bb80 1 --2- 172.21.15.37:0/1565447440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557714034460 0x557714040350 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f58c40118f0 tx=0x7f58c40047c0 comp rx=0 tx=0).stop 2024-06-10T01:07:39.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.577+0000 7f58da97bb80 1 -- 172.21.15.37:0/1565447440 shutdown_connections 2024-06-10T01:07:39.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.577+0000 7f58da97bb80 1 --2- 172.21.15.37:0/1565447440 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55771404e3f0 0x557714050810 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:39.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.577+0000 7f58da97bb80 1 --2- 172.21.15.37:0/1565447440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55771403a5f0 0x55771403db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:39.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.577+0000 7f58da97bb80 1 --2- 172.21.15.37:0/1565447440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557714034460 0x557714040350 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:39.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.577+0000 7f58da97bb80 1 --2- 172.21.15.37:0/1565447440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557714039cd0 0x55771403d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:39.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.577+0000 7f58b17fa700 0 client.26806 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:39.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.577+0000 7f58da97bb80 1 -- 172.21.15.37:0/1565447440 >> 172.21.15.37:0/1565447440 conn(0x557713f5ee50 msgr2=0x55771402eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:39.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.577+0000 7f58da97bb80 1 -- 172.21.15.37:0/1565447440 shutdown_connections 2024-06-10T01:07:39.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.577+0000 7f58da97bb80 1 -- 172.21.15.37:0/1565447440 wait complete. 2024-06-10T01:07:39.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.577+0000 7f58da97bb80 20 client.26806 trim_cache size 0 max 16384 2024-06-10T01:07:39.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.598+0000 7f58da97bb80 1 Processor -- start 2024-06-10T01:07:39.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.598+0000 7f58da97bb80 1 -- start start 2024-06-10T01:07:39.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.598+0000 7f58da97bb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557714034460 0x557714033f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:39.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.598+0000 7f58da97bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557714039cd0 0x55771403b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:39.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.598+0000 7f58da97bb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55771403a5f0 0x55771403dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:39.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.598+0000 7f58da97bb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f58c81b7160 con 0x557714034460 2024-06-10T01:07:39.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.598+0000 7f58da97bb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f58c81b49f0 con 0x55771403a5f0 2024-06-10T01:07:39.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.598+0000 7f58da97bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f58c81b2840 con 0x557714039cd0 2024-06-10T01:07:39.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.598+0000 7f58cde94700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557714034460 0x557714033f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:39.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.598+0000 7f58cde94700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557714034460 0x557714033f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42876/0 (socket says 172.21.15.37:42876) 2024-06-10T01:07:39.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.598+0000 7f58cde94700 1 -- 172.21.15.37:0/415284412 learned_addr learned my addr 172.21.15.37:0/415284412 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:39.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.598+0000 7f58cd693700 1 --2- 172.21.15.37:0/415284412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557714039cd0 0x55771403b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:39.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.598+0000 7f58ce695700 1 --2- 172.21.15.37:0/415284412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55771403a5f0 0x55771403dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:39.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.598+0000 7f58cde94700 1 -- 172.21.15.37:0/415284412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557714039cd0 msgr2=0x55771403b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:39.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.598+0000 7f58cde94700 1 --2- 172.21.15.37:0/415284412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557714039cd0 0x55771403b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:39.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.598+0000 7f58cde94700 1 -- 172.21.15.37:0/415284412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55771403a5f0 msgr2=0x55771403dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:39.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.598+0000 7f58cde94700 1 --2- 172.21.15.37:0/415284412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55771403a5f0 0x55771403dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:39.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.598+0000 7f58cde94700 1 -- 172.21.15.37:0/415284412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557714050e90 con 0x557714034460 2024-06-10T01:07:39.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.599+0000 7f58cde94700 1 --2- 172.21.15.37:0/415284412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557714034460 0x557714033f20 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f58c4000c00 tx=0x7f58c40046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88224c2e66e82820 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:39.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.599+0000 7f58b37fe700 1 -- 172.21.15.37:0/415284412 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c403a020 con 0x557714034460 2024-06-10T01:07:39.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.599+0000 7f58b37fe700 1 -- 172.21.15.37:0/415284412 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58c4023070 con 0x557714034460 2024-06-10T01:07:39.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.599+0000 7f58b37fe700 1 -- 172.21.15.37:0/415284412 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c401f890 con 0x557714034460 2024-06-10T01:07:39.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.600+0000 7f58da97bb80 1 -- 172.21.15.37:0/415284412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557714034460 msgr2=0x557714033f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:39.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.600+0000 7f58da97bb80 1 --2- 172.21.15.37:0/415284412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557714034460 0x557714033f20 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f58c4000c00 tx=0x7f58c40046d0 comp rx=0 tx=0).stop 2024-06-10T01:07:39.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.600+0000 7f58da97bb80 1 -- 172.21.15.37:0/415284412 shutdown_connections 2024-06-10T01:07:39.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.600+0000 7f58da97bb80 1 --2- 172.21.15.37:0/415284412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55771403a5f0 0x55771403dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:39.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.600+0000 7f58da97bb80 1 --2- 172.21.15.37:0/415284412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557714034460 0x557714033f20 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:39.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.600+0000 7f58da97bb80 1 --2- 172.21.15.37:0/415284412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557714039cd0 0x55771403b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:39.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.600+0000 7f58da97bb80 1 -- 172.21.15.37:0/415284412 >> 172.21.15.37:0/415284412 conn(0x557713f5ee50 msgr2=0x557714038790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:39.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.600+0000 7f58da97bb80 1 -- 172.21.15.37:0/415284412 shutdown_connections 2024-06-10T01:07:39.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.600+0000 7f58da97bb80 1 -- 172.21.15.37:0/415284412 wait complete. 2024-06-10T01:07:39.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.600+0000 7f58da97bb80 1 -- start start 2024-06-10T01:07:39.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.600+0000 7f58da97bb80 1 Processor -- start 2024-06-10T01:07:39.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.601+0000 7f58da97bb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557714034460 0x557714046680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:39.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.601+0000 7f58da97bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557714039cd0 0x557714046bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:39.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.601+0000 7f58da97bb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55771403a5f0 0x557714043470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:39.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.601+0000 7f58da97bb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f58c81b2eb0 con 0x557714034460 2024-06-10T01:07:39.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.601+0000 7f58da97bb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f58c81b4320 con 0x55771403a5f0 2024-06-10T01:07:39.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.601+0000 7f58da97bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f58c81b3580 con 0x557714039cd0 2024-06-10T01:07:39.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.601+0000 7f58da97bb80 10 client.0 _ll_register_callbacks cb 0x557713f1fab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:39.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.601+0000 7f58cde94700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557714034460 0x557714046680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:39.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.601+0000 7f58cde94700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557714034460 0x557714046680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42888/0 (socket says 172.21.15.37:42888) 2024-06-10T01:07:39.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.601+0000 7f58cde94700 1 -- 172.21.15.37:0/806985071 learned_addr learned my addr 172.21.15.37:0/806985071 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:39.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.601+0000 7f58ce695700 1 --2- 172.21.15.37:0/806985071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55771403a5f0 0x557714043470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:39.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.601+0000 7f58cd693700 1 --2- 172.21.15.37:0/806985071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557714039cd0 0x557714046bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:39.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.601+0000 7f58cde94700 1 -- 172.21.15.37:0/806985071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557714039cd0 msgr2=0x557714046bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:39.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.601+0000 7f58cde94700 1 --2- 172.21.15.37:0/806985071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557714039cd0 0x557714046bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:39.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.601+0000 7f58cde94700 1 -- 172.21.15.37:0/806985071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55771403a5f0 msgr2=0x557714043470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:39.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.601+0000 7f58cde94700 1 --2- 172.21.15.37:0/806985071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55771403a5f0 0x557714043470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:39.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.601+0000 7f58cde94700 1 -- 172.21.15.37:0/806985071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557714043cb0 con 0x557714034460 2024-06-10T01:07:39.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.601+0000 7f58ce695700 1 --2- 172.21.15.37:0/806985071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55771403a5f0 0x557714043470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:07:39.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.601+0000 7f58cde94700 1 --2- 172.21.15.37:0/806985071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557714034460 0x557714046680 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f58c4012fd0 tx=0x7f58c40075b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea8e6046a877df6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:39.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.601+0000 7f58b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:07:39.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.602+0000 7f58b17fa700 1 -- 172.21.15.37:0/806985071 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c403a030 con 0x557714034460 2024-06-10T01:07:39.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.602+0000 7f58b17fa700 1 -- 172.21.15.37:0/806985071 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f58c401ea40 con 0x557714034460 2024-06-10T01:07:39.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.602+0000 7f58da97bb80 1 -- 172.21.15.37:0/806985071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58c4023070 con 0x557714034460 2024-06-10T01:07:39.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.602+0000 7f58b17fa700 1 -- 172.21.15.37:0/806985071 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f58c404f050 con 0x557714034460 2024-06-10T01:07:39.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.602+0000 7f58da97bb80 10 client.17289 fetch_fsmap learned FSMap version 31 2024-06-10T01:07:39.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.602+0000 7f58da97bb80 1 -- 172.21.15.37:0/806985071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5577140b9bc0 con 0x557714034460 2024-06-10T01:07:39.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.602+0000 7f58b17fa700 1 -- 172.21.15.37:0/806985071 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58c4023070 con 0x557714034460 2024-06-10T01:07:39.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.602+0000 7f58b17fa700 1 -- 172.21.15.37:0/806985071 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f58c4004a40 con 0x557714034460 2024-06-10T01:07:39.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.602+0000 7f58da97bb80 10 client.17289 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:07:39.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.602+0000 7f58da97bb80 10 client.17289 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:07:39.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.602+0000 7f58da97bb80 20 client.17289 populate_metadata read hostname 'smithi037' 2024-06-10T01:07:39.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.602+0000 7f58da97bb80 1 --2- 172.21.15.37:0/806985071 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5577140ba220 0x5577140bc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:39.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.602+0000 7f58da97bb80 4 client.17289 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.8, "await_for": 45.0}] 2024-06-10T01:07:39.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.602+0000 7f58da97bb80 1 -- 172.21.15.37:0/806985071 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.8, "await_for": 45.0}) -- 0x557713f8d420 con 0x5577140ba220 2024-06-10T01:07:39.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.602+0000 7f58cd693700 1 --2- 172.21.15.37:0/806985071 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5577140ba220 0x5577140bc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:39.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.603+0000 7f58b17fa700 10 client.17289 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:39.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:39.603+0000 7f58cd693700 1 --2- 172.21.15.37:0/806985071 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5577140ba220 0x5577140bc610 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1880c0d1f7e38d7d server_cookie=1617d79e963ac7a1 in_seq=0 out_seq=0 2024-06-10T01:07:40.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:40.557+0000 7fab12ff5640 1 -- 172.21.15.37:0/2414553369 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f7c717cc20 con 0x55f7c728c9a0 2024-06-10T01:07:40.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:40.557+0000 7fab12ff5640 10 client.17265 handle_command_reply: tid=0 2024-06-10T01:07:40.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:40.557+0000 7fab385d7740 1 client.17265 shutdown 2024-06-10T01:07:40.554 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:40.559+0000 7fab385d7740 1 -- 172.21.15.37:0/2414553369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7c7211b70 msgr2=0x55f7c728acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:40.559+0000 7fab385d7740 1 --2- 172.21.15.37:0/2414553369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7c7211b70 0x55f7c728acf0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fab28009530 tx=0x7fab28004660 comp rx=0 tx=0).stop 2024-06-10T01:07:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:40.559+0000 7fab385d7740 1 -- 172.21.15.37:0/2414553369 shutdown_connections 2024-06-10T01:07:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:40.559+0000 7fab385d7740 1 --2- 172.21.15.37:0/2414553369 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f7c728c9a0 0x55f7c728edc0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:40.559+0000 7fab385d7740 1 --2- 172.21.15.37:0/2414553369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7c7211b70 0x55f7c728acf0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:40.559+0000 7fab385d7740 1 --2- 172.21.15.37:0/2414553369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7c720ca00 0x55f7c721a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:40.559+0000 7fab385d7740 1 --2- 172.21.15.37:0/2414553369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7c720c010 0x55f7c721a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:40.559+0000 7fab385d7740 1 -- 172.21.15.37:0/2414553369 >> 172.21.15.37:0/2414553369 conn(0x55f7c7207680 msgr2=0x55f7c720dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:40.559+0000 7fab12ff5640 0 client.17265 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:40.559+0000 7fab385d7740 1 -- 172.21.15.37:0/2414553369 shutdown_connections 2024-06-10T01:07:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:40.559+0000 7fab385d7740 1 -- 172.21.15.37:0/2414553369 wait complete. 2024-06-10T01:07:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:40.559+0000 7fab385d7740 20 client.17265 trim_cache size 0 max 16384 2024-06-10T01:07:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:07:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:07:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:07:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:40.560+0000 7fab357f4640 1 -- 172.21.15.37:0/1923416601 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fab180a2c30 msgr2=0x7fab180a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:40.560+0000 7fab357f4640 1 --2- 172.21.15.37:0/1923416601 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fab180a2c30 0x7fab180a50f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fab2801e000 tx=0x7fab28020000 comp rx=0 tx=0).stop 2024-06-10T01:07:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:40.560+0000 7fab357f4640 1 -- 172.21.15.37:0/1923416601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab30100900 msgr2=0x7fab30197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:40.560+0000 7fab357f4640 1 --2- 172.21.15.37:0/1923416601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab30100900 0x7fab30197c20 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fab2000ccb0 tx=0x7fab2000a620 comp rx=0 tx=0).stop 2024-06-10T01:07:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:40.560+0000 7fab357f4640 1 -- 172.21.15.37:0/1923416601 shutdown_connections 2024-06-10T01:07:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:40.560+0000 7fab357f4640 1 --2- 172.21.15.37:0/1923416601 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fab180a2c30 0x7fab180a50f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:40.560+0000 7fab357f4640 1 --2- 172.21.15.37:0/1923416601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab301024b0 0x7fab3019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:40.560+0000 7fab357f4640 1 --2- 172.21.15.37:0/1923416601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab30101af0 0x7fab30198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:40.560+0000 7fab357f4640 1 --2- 172.21.15.37:0/1923416601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab30100900 0x7fab30197c20 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:40.560+0000 7fab357f4640 1 -- 172.21.15.37:0/1923416601 >> 172.21.15.37:0/1923416601 conn(0x7fab300fbe70 msgr2=0x7fab30104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:40.561+0000 7fab357f4640 1 -- 172.21.15.37:0/1923416601 shutdown_connections 2024-06-10T01:07:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:40.561+0000 7fab357f4640 1 -- 172.21.15.37:0/1923416601 wait complete. 2024-06-10T01:07:40.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:40 smithi167 ceph-mon[36917]: pgmap v760: 129 pgs: 129 active+clean; 29 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 761 op/s 2024-06-10T01:07:40.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:40 smithi094 ceph-mon[36708]: pgmap v760: 129 pgs: 129 active+clean; 29 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 761 op/s 2024-06-10T01:07:40.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:40 smithi037 ceph-mon[32192]: pgmap v760: 129 pgs: 129 active+clean; 29 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 761 op/s 2024-06-10T01:07:42.443 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:42 smithi037 ceph-mon[32192]: pgmap v761: 129 pgs: 129 active+clean; 29 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 869 KiB/s wr, 693 op/s 2024-06-10T01:07:42.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:42 smithi167 ceph-mon[36917]: pgmap v761: 129 pgs: 129 active+clean; 29 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 869 KiB/s wr, 693 op/s 2024-06-10T01:07:42.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:42 smithi094 ceph-mon[36708]: pgmap v761: 129 pgs: 129 active+clean; 29 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 869 KiB/s wr, 693 op/s 2024-06-10T01:07:44.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:44 smithi167 ceph-mon[36917]: pgmap v762: 129 pgs: 129 active+clean; 20 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 737 op/s 2024-06-10T01:07:44.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:44 smithi094 ceph-mon[36708]: pgmap v762: 129 pgs: 129 active+clean; 20 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 737 op/s 2024-06-10T01:07:44.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:44 smithi037 ceph-mon[32192]: pgmap v762: 129 pgs: 129 active+clean; 20 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 737 op/s 2024-06-10T01:07:46.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:46 smithi167 ceph-mon[36917]: pgmap v763: 129 pgs: 129 active+clean; 15 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 746 op/s 2024-06-10T01:07:46.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:46 smithi094 ceph-mon[36708]: pgmap v763: 129 pgs: 129 active+clean; 15 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 746 op/s 2024-06-10T01:07:46.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:46 smithi037 ceph-mon[32192]: pgmap v763: 129 pgs: 129 active+clean; 15 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 746 op/s 2024-06-10T01:07:46.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:46.972+0000 7f58b17fa700 1 -- 172.21.15.37:0/806985071 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557713f8d420 con 0x5577140ba220 2024-06-10T01:07:46.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:46.972+0000 7f58b17fa700 10 client.17289 handle_command_reply: tid=0 2024-06-10T01:07:46.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:46.972+0000 7f58da97bb80 1 client.17289 shutdown 2024-06-10T01:07:46.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:46.972+0000 7f58da97bb80 1 -- 172.21.15.37:0/806985071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557714034460 msgr2=0x557714046680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:46.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:46.972+0000 7f58da97bb80 1 --2- 172.21.15.37:0/806985071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557714034460 0x557714046680 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f58c4012fd0 tx=0x7f58c40075b0 comp rx=0 tx=0).stop 2024-06-10T01:07:46.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:46.972+0000 7f58da97bb80 1 -- 172.21.15.37:0/806985071 shutdown_connections 2024-06-10T01:07:46.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:46.973+0000 7f58da97bb80 1 --2- 172.21.15.37:0/806985071 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5577140ba220 0x5577140bc610 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:46.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:46.973+0000 7f58da97bb80 1 --2- 172.21.15.37:0/806985071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55771403a5f0 0x557714043470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:46.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:46.973+0000 7f58da97bb80 1 --2- 172.21.15.37:0/806985071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557714034460 0x557714046680 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:46.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:46.973+0000 7f58da97bb80 1 --2- 172.21.15.37:0/806985071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557714039cd0 0x557714046bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:46.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:46.973+0000 7f58b17fa700 0 client.17289 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:46.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:46.973+0000 7f58da97bb80 1 -- 172.21.15.37:0/806985071 >> 172.21.15.37:0/806985071 conn(0x557713f5ee50 msgr2=0x557714039020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:46.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:46.973+0000 7f58da97bb80 1 -- 172.21.15.37:0/806985071 shutdown_connections 2024-06-10T01:07:46.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:46.973+0000 7f58da97bb80 1 -- 172.21.15.37:0/806985071 wait complete. 2024-06-10T01:07:46.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:46.973+0000 7f58da97bb80 20 client.17289 trim_cache size 0 max 16384 2024-06-10T01:07:46.970 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:07:46.970 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 30, 2024-06-10T01:07:46.970 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24472, 2024-06-10T01:07:46.970 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 26, 2024-06-10T01:07:46.970 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-06-10T01:07:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "2e2e5b63": { 2024-06-10T01:07:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 26, 2024-06-10T01:07:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-06-10T01:07:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:07:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-06-10T01:07:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:07:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:07:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-06-10T01:07:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 137.8, 2024-06-10T01:07:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-06-10T01:07:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-06-10T01:07:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-06-10T01:07:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:07:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-06-10T01:07:46.972 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:07:46.972 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:07:46.972 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:07:46.972 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:07:46.972 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:07:46.972 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:07:46.972 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:07:46.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:46.974+0000 7f58d00f8700 1 -- 172.21.15.37:0/453100000 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f58b40a2d40 msgr2=0x7f58b40a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:46.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:46.974+0000 7f58d00f8700 1 --2- 172.21.15.37:0/453100000 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f58b40a2d40 0x7f58b40a5200 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f58c4007c90 tx=0x7f58c4007480 comp rx=0 tx=0).stop 2024-06-10T01:07:46.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:46.974+0000 7f58d00f8700 1 -- 172.21.15.37:0/453100000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58c8107920 msgr2=0x7f58c81a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:46.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:46.974+0000 7f58d00f8700 1 --2- 172.21.15.37:0/453100000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58c8107920 0x7f58c81a3920 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f58c000efc0 tx=0x7f58c000c3a0 comp rx=0 tx=0).stop 2024-06-10T01:07:46.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:46.974+0000 7f58d00f8700 1 -- 172.21.15.37:0/453100000 shutdown_connections 2024-06-10T01:07:46.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:46.974+0000 7f58d00f8700 1 --2- 172.21.15.37:0/453100000 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f58b40a2d40 0x7f58b40a5200 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:46.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:46.974+0000 7f58d00f8700 1 --2- 172.21.15.37:0/453100000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58c8107920 0x7f58c81a3920 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:46.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:46.974+0000 7f58d00f8700 1 --2- 172.21.15.37:0/453100000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58c8104f20 0x7f58c819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:46.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:46.974+0000 7f58d00f8700 1 --2- 172.21.15.37:0/453100000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58c81074b0 0x7f58c819f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:46.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:46.974+0000 7f58d00f8700 1 -- 172.21.15.37:0/453100000 >> 172.21.15.37:0/453100000 conn(0x7f58c8078340 msgr2=0x7f58c80feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:46.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:46.975+0000 7f58d00f8700 1 -- 172.21.15.37:0/453100000 shutdown_connections 2024-06-10T01:07:46.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:46.975+0000 7f58d00f8700 1 -- 172.21.15.37:0/453100000 wait complete. 2024-06-10T01:07:47.818 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2e2e5b63', quiesce took 14.6 seconds. Will release after: 3.200000000000001 2024-06-10T01:07:48.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:48 smithi167 ceph-mon[36917]: pgmap v764: 129 pgs: 129 active+clean; 15 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 933 KiB/s wr, 620 op/s 2024-06-10T01:07:48.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:48 smithi094 ceph-mon[36708]: pgmap v764: 129 pgs: 129 active+clean; 15 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 933 KiB/s wr, 620 op/s 2024-06-10T01:07:48.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:48 smithi037 ceph-mon[32192]: pgmap v764: 129 pgs: 129 active+clean; 15 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 933 KiB/s wr, 620 op/s 2024-06-10T01:07:50.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:50 smithi167 ceph-mon[36917]: pgmap v765: 129 pgs: 129 active+clean; 16 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 882 KiB/s wr, 1.42k op/s 2024-06-10T01:07:50.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:50 smithi094 ceph-mon[36708]: pgmap v765: 129 pgs: 129 active+clean; 16 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 882 KiB/s wr, 1.42k op/s 2024-06-10T01:07:50.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:50 smithi037 ceph-mon[32192]: pgmap v765: 129 pgs: 129 active+clean; 16 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 882 KiB/s wr, 1.42k op/s 2024-06-10T01:07:51.019 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:07:51.287 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:51.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:51 smithi037 ceph-mon[32192]: scrub summary: idle+waiting paths [/] 2024-06-10T01:07:51.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:51 smithi167 ceph-mon[36917]: scrub summary: idle+waiting paths [/] 2024-06-10T01:07:51.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:51 smithi094 ceph-mon[36708]: scrub summary: idle+waiting paths [/] 2024-06-10T01:07:52.445 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:52 smithi037 ceph-mon[32192]: scrub summary: idle 2024-06-10T01:07:52.445 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:52 smithi037 ceph-mon[32192]: pgmap v766: 129 pgs: 129 active+clean; 16 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 610 KiB/s wr, 1.37k op/s 2024-06-10T01:07:52.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:52 smithi167 ceph-mon[36917]: scrub summary: idle 2024-06-10T01:07:52.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:52 smithi167 ceph-mon[36917]: pgmap v766: 129 pgs: 129 active+clean; 16 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 610 KiB/s wr, 1.37k op/s 2024-06-10T01:07:52.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:52 smithi094 ceph-mon[36708]: scrub summary: idle 2024-06-10T01:07:52.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:52 smithi094 ceph-mon[36708]: pgmap v766: 129 pgs: 129 active+clean; 16 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 610 KiB/s wr, 1.37k op/s 2024-06-10T01:07:53.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.563+0000 7fa33ccb4700 1 -- 172.21.15.37:0/2794084570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa338102740 msgr2=0x7fa338102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:53.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.563+0000 7fa33ccb4700 1 --2- 172.21.15.37:0/2794084570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa338102740 0x7fa338102b20 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fa3280099c0 tx=0x7fa328009cd0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.563+0000 7fa33ccb4700 1 -- 172.21.15.37:0/2794084570 shutdown_connections 2024-06-10T01:07:53.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.563+0000 7fa33ccb4700 1 --2- 172.21.15.37:0/2794084570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa338107840 0x7fa33810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.563+0000 7fa33ccb4700 1 --2- 172.21.15.37:0/2794084570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa338102740 0x7fa338102b20 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.563+0000 7fa33ccb4700 1 --2- 172.21.15.37:0/2794084570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3381030f0 0x7fa338107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.563+0000 7fa33ccb4700 1 -- 172.21.15.37:0/2794084570 >> 172.21.15.37:0/2794084570 conn(0x7fa3380795f0 msgr2=0x7fa338079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:53.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.563+0000 7fa33ccb4700 1 -- 172.21.15.37:0/2794084570 shutdown_connections 2024-06-10T01:07:53.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.563+0000 7fa33ccb4700 1 -- 172.21.15.37:0/2794084570 wait complete. 2024-06-10T01:07:53.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.564+0000 7fa33ccb4700 1 Processor -- start 2024-06-10T01:07:53.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.564+0000 7fa33ccb4700 1 -- start start 2024-06-10T01:07:53.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.564+0000 7fa33ccb4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa338102740 0x7fa33819e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:53.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.564+0000 7fa33ccb4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3381030f0 0x7fa33819ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:53.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.564+0000 7fa33ccb4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa338107840 0x7fa3381988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:53.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.564+0000 7fa33ccb4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa338112760 con 0x7fa3381030f0 2024-06-10T01:07:53.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.564+0000 7fa33ccb4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa3381125e0 con 0x7fa338107840 2024-06-10T01:07:53.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.564+0000 7fa33ccb4700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa3381128e0 con 0x7fa338102740 2024-06-10T01:07:53.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.564+0000 7fa335d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3381030f0 0x7fa33819ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:53.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.565+0000 7fa335d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3381030f0 0x7fa33819ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37416/0 (socket says 172.21.15.37:37416) 2024-06-10T01:07:53.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.565+0000 7fa335d9b700 1 -- 172.21.15.37:0/1000262699 learned_addr learned my addr 172.21.15.37:0/1000262699 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:53.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.565+0000 7fa33659c700 1 --2- 172.21.15.37:0/1000262699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa338102740 0x7fa33819e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:53.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.565+0000 7fa336d9d700 1 --2- 172.21.15.37:0/1000262699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa338107840 0x7fa3381988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:53.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.565+0000 7fa335d9b700 1 -- 172.21.15.37:0/1000262699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa338102740 msgr2=0x7fa33819e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:53.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.565+0000 7fa335d9b700 1 --2- 172.21.15.37:0/1000262699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa338102740 0x7fa33819e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.565+0000 7fa335d9b700 1 -- 172.21.15.37:0/1000262699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa338107840 msgr2=0x7fa3381988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:53.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.565+0000 7fa335d9b700 1 --2- 172.21.15.37:0/1000262699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa338107840 0x7fa3381988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.565+0000 7fa335d9b700 1 -- 172.21.15.37:0/1000262699 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa338199020 con 0x7fa3381030f0 2024-06-10T01:07:53.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.565+0000 7fa335d9b700 1 --2- 172.21.15.37:0/1000262699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3381030f0 0x7fa33819ec40 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00c9b0 tx=0x7fa32c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:53.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.565+0000 7fa3277fe700 1 -- 172.21.15.37:0/1000262699 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c007b00 con 0x7fa3381030f0 2024-06-10T01:07:53.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.565+0000 7fa3277fe700 1 -- 172.21.15.37:0/1000262699 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa32c00ce50 con 0x7fa3381030f0 2024-06-10T01:07:53.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.565+0000 7fa33ccb4700 1 -- 172.21.15.37:0/1000262699 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa338199310 con 0x7fa3381030f0 2024-06-10T01:07:53.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.566+0000 7fa3277fe700 1 -- 172.21.15.37:0/1000262699 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c018660 con 0x7fa3381030f0 2024-06-10T01:07:53.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.566+0000 7fa33ccb4700 1 -- 172.21.15.37:0/1000262699 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3381a55d0 con 0x7fa3381030f0 2024-06-10T01:07:53.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.567+0000 7fa33ccb4700 1 -- 172.21.15.37:0/1000262699 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa338069180 con 0x7fa3381030f0 2024-06-10T01:07:53.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.568+0000 7fa3277fe700 1 -- 172.21.15.37:0/1000262699 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa32c00f450 con 0x7fa3381030f0 2024-06-10T01:07:53.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.569+0000 7fa3277fe700 1 --2- 172.21.15.37:0/1000262699 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa3200a2ca0 0x7fa3200a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:53.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.569+0000 7fa33659c700 1 --2- 172.21.15.37:0/1000262699 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa3200a2ca0 0x7fa3200a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:53.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.569+0000 7fa3277fe700 1 -- 172.21.15.37:0/1000262699 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fa32c0cf350 con 0x7fa3381030f0 2024-06-10T01:07:53.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.569+0000 7fa33659c700 1 --2- 172.21.15.37:0/1000262699 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa3200a2ca0 0x7fa3200a5160 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa328000c00 tx=0x7fa328005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:53.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.573+0000 7fa3277fe700 1 -- 172.21.15.37:0/1000262699 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa32c014030 con 0x7fa3381030f0 2024-06-10T01:07:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.660+0000 7f292ffff640 1 -- 172.21.15.37:0/690711542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2930100720 msgr2=0x7f2930100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.660+0000 7f292ffff640 1 --2- 172.21.15.37:0/690711542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2930100720 0x7f2930100b40 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f2920009900 tx=0x7f292002a410 comp rx=0 tx=0).stop 2024-06-10T01:07:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.660+0000 7f292ffff640 1 -- 172.21.15.37:0/690711542 shutdown_connections 2024-06-10T01:07:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.660+0000 7f292ffff640 1 --2- 172.21.15.37:0/690711542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29301022d0 0x7f2930108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.660+0000 7f292ffff640 1 --2- 172.21.15.37:0/690711542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2930101910 0x7f2930101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.660+0000 7f292ffff640 1 --2- 172.21.15.37:0/690711542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2930100720 0x7f2930100b40 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.660+0000 7f292ffff640 1 -- 172.21.15.37:0/690711542 >> 172.21.15.37:0/690711542 conn(0x7f29300fbc90 msgr2=0x7f29300fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.661+0000 7f292ffff640 1 -- 172.21.15.37:0/690711542 shutdown_connections 2024-06-10T01:07:53.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.661+0000 7f292ffff640 1 -- 172.21.15.37:0/690711542 wait complete. 2024-06-10T01:07:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.661+0000 7f292ffff640 1 Processor -- start 2024-06-10T01:07:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.661+0000 7f292ffff640 1 -- start start 2024-06-10T01:07:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.662+0000 7f292ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2930100720 0x7f2930197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.662+0000 7f292ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2930101910 0x7f29301980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.662+0000 7f292ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29301022d0 0x7f293019f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.662+0000 7f292ffff640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f293010c3a0 con 0x7f29301022d0 2024-06-10T01:07:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.662+0000 7f292ffff640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f293010c220 con 0x7f2930101910 2024-06-10T01:07:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.662+0000 7f292ffff640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f293010c520 con 0x7f2930100720 2024-06-10T01:07:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.662+0000 7f292f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29301022d0 0x7f293019f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.662+0000 7f292f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29301022d0 0x7f293019f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37438/0 (socket says 172.21.15.37:37438) 2024-06-10T01:07:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.662+0000 7f292f7fe640 1 -- 172.21.15.37:0/3561094220 learned_addr learned my addr 172.21.15.37:0/3561094220 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.662+0000 7f292effd640 1 --2- 172.21.15.37:0/3561094220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2930100720 0x7f2930197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.662+0000 7f292e7fc640 1 --2- 172.21.15.37:0/3561094220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2930101910 0x7f29301980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.662+0000 7f292f7fe640 1 -- 172.21.15.37:0/3561094220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2930100720 msgr2=0x7f2930197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.662+0000 7f292f7fe640 1 --2- 172.21.15.37:0/3561094220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2930100720 0x7f2930197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.662+0000 7f292f7fe640 1 -- 172.21.15.37:0/3561094220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2930101910 msgr2=0x7f29301980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.662+0000 7f292f7fe640 1 --2- 172.21.15.37:0/3561094220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2930101910 0x7f29301980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.662+0000 7f292f7fe640 1 -- 172.21.15.37:0/3561094220 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f293019f7c0 con 0x7f29301022d0 2024-06-10T01:07:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.662+0000 7f292f7fe640 1 --2- 172.21.15.37:0/3561094220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29301022d0 0x7f293019f150 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f292800cad0 tx=0x7f292800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.663+0000 7f2917fff640 1 -- 172.21.15.37:0/3561094220 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292800ae20 con 0x7f29301022d0 2024-06-10T01:07:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.663+0000 7f292ffff640 1 -- 172.21.15.37:0/3561094220 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f293019fab0 con 0x7f29301022d0 2024-06-10T01:07:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.663+0000 7f2917fff640 1 -- 172.21.15.37:0/3561094220 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2928005020 con 0x7f29301022d0 2024-06-10T01:07:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.663+0000 7f2917fff640 1 -- 172.21.15.37:0/3561094220 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292800f6b0 con 0x7f29301022d0 2024-06-10T01:07:53.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.663+0000 7f292ffff640 1 -- 172.21.15.37:0/3561094220 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29301a0000 con 0x7f29301022d0 2024-06-10T01:07:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.665+0000 7f2917fff640 1 -- 172.21.15.37:0/3561094220 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f29280051d0 con 0x7f29301022d0 2024-06-10T01:07:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.665+0000 7f29381ef740 1 Processor -- start 2024-06-10T01:07:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.665+0000 7f29381ef740 1 -- start start 2024-06-10T01:07:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.666+0000 7f2917fff640 1 --2- 172.21.15.37:0/3561094220 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f29180a2b80 0x7f29180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.666+0000 7f29381ef740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56024f472200 0x56024f472620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.666+0000 7f29381ef740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56024f472bf0 0x56024f4776f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.666+0000 7f29381ef740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56024f477d60 0x56024f47a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.666+0000 7f29381ef740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56024f3d54f0 con 0x56024f472200 2024-06-10T01:07:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.666+0000 7f29381ef740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56024f373870 con 0x56024f472bf0 2024-06-10T01:07:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.666+0000 7f29381ef740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56024f3d14b0 con 0x56024f477d60 2024-06-10T01:07:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.666+0000 7f292effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56024f472200 0x56024f472620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.666+0000 7f292effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56024f472200 0x56024f472620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37452/0 (socket says 172.21.15.37:37452) 2024-06-10T01:07:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.666+0000 7f292effd640 1 -- 172.21.15.37:0/312452672 learned_addr learned my addr 172.21.15.37:0/312452672 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.666+0000 7f292e7fc640 1 --2- 172.21.15.37:0/3561094220 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f29180a2b80 0x7f29180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.666+0000 7f2917fff640 1 -- 172.21.15.37:0/3561094220 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f29280cf770 con 0x7f29301022d0 2024-06-10T01:07:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.666+0000 7f292f7fe640 1 --2- 172.21.15.37:0/312452672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56024f477d60 0x56024f47a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.666+0000 7f292e7fc640 1 --2- 172.21.15.37:0/312452672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56024f472bf0 0x56024f4776f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.666+0000 7f292effd640 1 -- 172.21.15.37:0/312452672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56024f477d60 msgr2=0x56024f47a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.666+0000 7f292effd640 1 --2- 172.21.15.37:0/312452672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56024f477d60 0x56024f47a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.666+0000 7f292effd640 1 -- 172.21.15.37:0/312452672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56024f472bf0 msgr2=0x56024f4776f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.666+0000 7f292effd640 1 --2- 172.21.15.37:0/312452672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56024f472bf0 0x56024f4776f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.666+0000 7f292effd640 1 -- 172.21.15.37:0/312452672 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56024f38f9b0 con 0x56024f472200 2024-06-10T01:07:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.666+0000 7f292e7fc640 1 --2- 172.21.15.37:0/3561094220 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f29180a2b80 0x7f29180a5040 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f29301990b0 tx=0x7f2924013830 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.666+0000 7f292effd640 1 --2- 172.21.15.37:0/312452672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56024f472200 0x56024f472620 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f29200314e0 tx=0x7f2920005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c2ba06597b2050a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.667+0000 7f29157fa640 1 -- 172.21.15.37:0/312452672 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920040070 con 0x56024f472200 2024-06-10T01:07:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.667+0000 7f29157fa640 1 -- 172.21.15.37:0/312452672 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2920005500 con 0x56024f472200 2024-06-10T01:07:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.667+0000 7f29157fa640 1 -- 172.21.15.37:0/312452672 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920035040 con 0x56024f472200 2024-06-10T01:07:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.668+0000 7f29381ef740 1 -- 172.21.15.37:0/312452672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56024f472200 msgr2=0x56024f472620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.668+0000 7f29381ef740 1 --2- 172.21.15.37:0/312452672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56024f472200 0x56024f472620 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f29200314e0 tx=0x7f2920005c00 comp rx=0 tx=0).stop 2024-06-10T01:07:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.668+0000 7f29381ef740 1 -- 172.21.15.37:0/312452672 shutdown_connections 2024-06-10T01:07:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.668+0000 7f29381ef740 1 --2- 172.21.15.37:0/312452672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56024f477d60 0x56024f47a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.668+0000 7f29381ef740 1 --2- 172.21.15.37:0/312452672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56024f472bf0 0x56024f4776f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.668+0000 7f29381ef740 1 --2- 172.21.15.37:0/312452672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56024f472200 0x56024f472620 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.668+0000 7f29381ef740 1 -- 172.21.15.37:0/312452672 >> 172.21.15.37:0/312452672 conn(0x56024f46d870 msgr2=0x56024f46dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.668+0000 7f29381ef740 1 -- 172.21.15.37:0/312452672 shutdown_connections 2024-06-10T01:07:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.668+0000 7f29381ef740 1 -- 172.21.15.37:0/312452672 wait complete. 2024-06-10T01:07:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.668+0000 7f29381ef740 1 -- start start 2024-06-10T01:07:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.668+0000 7f29381ef740 1 Processor -- start 2024-06-10T01:07:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.669+0000 7f29381ef740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56024f472200 0x56024f47c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.669+0000 7f29381ef740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56024f472bf0 0x56024f47c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.669+0000 7f29381ef740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56024f477d60 0x56024f483900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.669+0000 7f29381ef740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56024f47acb0 con 0x56024f472bf0 2024-06-10T01:07:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.669+0000 7f29381ef740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56024f35eae0 con 0x56024f472200 2024-06-10T01:07:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.669+0000 7f29381ef740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56024f47ae30 con 0x56024f477d60 2024-06-10T01:07:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.669+0000 7f292e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56024f472bf0 0x56024f47c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.669+0000 7f29381ef740 10 client.0 _ll_register_callbacks cb 0x56024f3979b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.669+0000 7f292e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56024f472bf0 0x56024f47c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37460/0 (socket says 172.21.15.37:37460) 2024-06-10T01:07:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.669+0000 7f292e7fc640 1 -- 172.21.15.37:0/2307234644 learned_addr learned my addr 172.21.15.37:0/2307234644 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.669+0000 7f292effd640 1 --2- 172.21.15.37:0/2307234644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56024f472200 0x56024f47c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.669+0000 7f292f7fe640 1 --2- 172.21.15.37:0/2307234644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56024f477d60 0x56024f483900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.669+0000 7f292e7fc640 1 -- 172.21.15.37:0/2307234644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56024f477d60 msgr2=0x56024f483900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.669+0000 7f292e7fc640 1 --2- 172.21.15.37:0/2307234644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56024f477d60 0x56024f483900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.669+0000 7f292e7fc640 1 -- 172.21.15.37:0/2307234644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56024f472200 msgr2=0x56024f47c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.669+0000 7f292e7fc640 1 --2- 172.21.15.37:0/2307234644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56024f472200 0x56024f47c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.669+0000 7f292e7fc640 1 -- 172.21.15.37:0/2307234644 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56024f0e9210 con 0x56024f472bf0 2024-06-10T01:07:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.669+0000 7f292f7fe640 1 --2- 172.21.15.37:0/2307234644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56024f477d60 0x56024f483900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:07:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.669+0000 7f292e7fc640 1 --2- 172.21.15.37:0/2307234644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56024f472bf0 0x56024f47c8c0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f292400e990 tx=0x7f2924004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4363fe45a6e742 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.669+0000 7f2912ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:07:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.669+0000 7f2912ff5640 1 -- 172.21.15.37:0/2307234644 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2924011d70 con 0x56024f472bf0 2024-06-10T01:07:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.670+0000 7f2912ff5640 1 -- 172.21.15.37:0/2307234644 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f292401e440 con 0x56024f472bf0 2024-06-10T01:07:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.670+0000 7f29381ef740 1 -- 172.21.15.37:0/2307234644 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2920005500 con 0x56024f472bf0 2024-06-10T01:07:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.670+0000 7f2912ff5640 1 -- 172.21.15.37:0/2307234644 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f292401e640 con 0x56024f472bf0 2024-06-10T01:07:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.670+0000 7f29381ef740 10 client.17307 fetch_fsmap learned FSMap version 31 2024-06-10T01:07:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.670+0000 7f29381ef740 1 -- 172.21.15.37:0/2307234644 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56024f1b5620 con 0x56024f472bf0 2024-06-10T01:07:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.670+0000 7f2912ff5640 1 -- 172.21.15.37:0/2307234644 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2920005500 con 0x56024f472bf0 2024-06-10T01:07:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.670+0000 7f2912ff5640 1 -- 172.21.15.37:0/2307234644 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f292401c040 con 0x56024f472bf0 2024-06-10T01:07:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.670+0000 7f29381ef740 10 client.17307 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:07:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.670+0000 7f29381ef740 10 client.17307 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.670+0000 7f29381ef740 20 client.17307 populate_metadata read hostname 'smithi037' 2024-06-10T01:07:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.670+0000 7f29381ef740 1 --2- 172.21.15.37:0/2307234644 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56024f488200 0x56024f48a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.670+0000 7f29381ef740 4 client.17307 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.670+0000 7f29381ef740 1 -- 172.21.15.37:0/2307234644 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56024f3e8f10 con 0x56024f488200 2024-06-10T01:07:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.670+0000 7f292effd640 1 --2- 172.21.15.37:0/2307234644 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56024f488200 0x56024f48a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.671+0000 7f292effd640 1 --2- 172.21.15.37:0/2307234644 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56024f488200 0x56024f48a610 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcd4c8e27850c7db server_cookie=3a9c6d3dfd1f0d57 in_seq=0 out_seq=0 2024-06-10T01:07:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.671+0000 7f2912ff5640 10 client.17307 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:53.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.672+0000 7f2912ff5640 1 -- 172.21.15.37:0/2307234644 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56024f3e8f10 con 0x56024f488200 2024-06-10T01:07:53.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.672+0000 7f2912ff5640 10 client.17307 handle_command_reply: tid=0 2024-06-10T01:07:53.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.672+0000 7f29381ef740 1 client.17307 shutdown 2024-06-10T01:07:53.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.672+0000 7f29381ef740 1 -- 172.21.15.37:0/2307234644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56024f472bf0 msgr2=0x56024f47c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:53.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.672+0000 7f29381ef740 1 --2- 172.21.15.37:0/2307234644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56024f472bf0 0x56024f47c8c0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f292400e990 tx=0x7f2924004730 comp rx=0 tx=0).stop 2024-06-10T01:07:53.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.672+0000 7f29381ef740 1 -- 172.21.15.37:0/2307234644 shutdown_connections 2024-06-10T01:07:53.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.672+0000 7f29381ef740 1 --2- 172.21.15.37:0/2307234644 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56024f488200 0x56024f48a610 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.672+0000 7f29381ef740 1 --2- 172.21.15.37:0/2307234644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56024f477d60 0x56024f483900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.672+0000 7f29381ef740 1 --2- 172.21.15.37:0/2307234644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56024f472bf0 0x56024f47c8c0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.672+0000 7f2912ff5640 0 client.17307 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:53.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.672+0000 7f29381ef740 1 --2- 172.21.15.37:0/2307234644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56024f472200 0x56024f47c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.672+0000 7f29381ef740 1 -- 172.21.15.37:0/2307234644 >> 172.21.15.37:0/2307234644 conn(0x56024f46d870 msgr2=0x56024f46c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:53.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.672+0000 7f29381ef740 1 -- 172.21.15.37:0/2307234644 shutdown_connections 2024-06-10T01:07:53.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.672+0000 7f29381ef740 1 -- 172.21.15.37:0/2307234644 wait complete. 2024-06-10T01:07:53.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.672+0000 7f29381ef740 20 client.17307 trim_cache size 0 max 16384 2024-06-10T01:07:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.687+0000 7f29381ef740 1 Processor -- start 2024-06-10T01:07:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.687+0000 7f29381ef740 1 -- start start 2024-06-10T01:07:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.687+0000 7f29381ef740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56024f472200 0x56024f473d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.687+0000 7f29381ef740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56024f472bf0 0x56024f4742b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.687+0000 7f29381ef740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56024f477d60 0x56024f4838c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.687+0000 7f29381ef740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f29301acf70 con 0x56024f472200 2024-06-10T01:07:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.687+0000 7f29381ef740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f29301aa840 con 0x56024f477d60 2024-06-10T01:07:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.687+0000 7f29381ef740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f29301a8690 con 0x56024f472bf0 2024-06-10T01:07:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.687+0000 7f292effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56024f472200 0x56024f473d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.687+0000 7f292effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56024f472200 0x56024f473d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37466/0 (socket says 172.21.15.37:37466) 2024-06-10T01:07:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.687+0000 7f292effd640 1 -- 172.21.15.37:0/4253495690 learned_addr learned my addr 172.21.15.37:0/4253495690 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.688+0000 7f292f7fe640 1 --2- 172.21.15.37:0/4253495690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56024f477d60 0x56024f4838c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.688+0000 7f292effd640 1 -- 172.21.15.37:0/4253495690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56024f472bf0 msgr2=0x56024f4742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.688+0000 7f292effd640 1 --2- 172.21.15.37:0/4253495690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56024f472bf0 0x56024f4742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.688+0000 7f292effd640 1 -- 172.21.15.37:0/4253495690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56024f477d60 msgr2=0x56024f4838c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.688+0000 7f292effd640 1 --2- 172.21.15.37:0/4253495690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56024f477d60 0x56024f4838c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.688+0000 7f292effd640 1 -- 172.21.15.37:0/4253495690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56024f469240 con 0x56024f472200 2024-06-10T01:07:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.688+0000 7f292effd640 1 --2- 172.21.15.37:0/4253495690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56024f472200 0x56024f473d70 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f2920031bf0 tx=0x7f2920031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e10e5bf36747ce2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.688+0000 7f2914ff9640 1 -- 172.21.15.37:0/4253495690 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920054070 con 0x56024f472200 2024-06-10T01:07:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.688+0000 7f2914ff9640 1 -- 172.21.15.37:0/4253495690 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29200043c0 con 0x56024f472200 2024-06-10T01:07:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.688+0000 7f2914ff9640 1 -- 172.21.15.37:0/4253495690 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920040070 con 0x56024f472200 2024-06-10T01:07:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.689+0000 7f29381ef740 1 -- 172.21.15.37:0/4253495690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56024f472200 msgr2=0x56024f473d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.689+0000 7f29381ef740 1 --2- 172.21.15.37:0/4253495690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56024f472200 0x56024f473d70 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f2920031bf0 tx=0x7f2920031400 comp rx=0 tx=0).stop 2024-06-10T01:07:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.689+0000 7f29381ef740 1 -- 172.21.15.37:0/4253495690 shutdown_connections 2024-06-10T01:07:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.689+0000 7f29381ef740 1 --2- 172.21.15.37:0/4253495690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56024f477d60 0x56024f4838c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.689+0000 7f29381ef740 1 --2- 172.21.15.37:0/4253495690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56024f472bf0 0x56024f4742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.689+0000 7f29381ef740 1 --2- 172.21.15.37:0/4253495690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56024f472200 0x56024f473d70 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.689+0000 7f29381ef740 1 -- 172.21.15.37:0/4253495690 >> 172.21.15.37:0/4253495690 conn(0x56024f46d870 msgr2=0x56024f47c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.689+0000 7f29381ef740 1 -- 172.21.15.37:0/4253495690 shutdown_connections 2024-06-10T01:07:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.689+0000 7f29381ef740 1 -- 172.21.15.37:0/4253495690 wait complete. 2024-06-10T01:07:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.689+0000 7f29381ef740 1 -- start start 2024-06-10T01:07:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.689+0000 7f29381ef740 1 Processor -- start 2024-06-10T01:07:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.689+0000 7f29381ef740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56024f472200 0x56024f480a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.689+0000 7f29381ef740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56024f472bf0 0x56024f480f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.689+0000 7f29381ef740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56024f477d60 0x56024f481fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.689+0000 7f29381ef740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f29301a8d00 con 0x56024f472bf0 2024-06-10T01:07:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.689+0000 7f29381ef740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f29301aa170 con 0x56024f472200 2024-06-10T01:07:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.689+0000 7f29381ef740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f29301a93d0 con 0x56024f477d60 2024-06-10T01:07:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.689+0000 7f29381ef740 10 client.0 _ll_register_callbacks cb 0x7f292400a9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.689+0000 7f292e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56024f472bf0 0x56024f480f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.689+0000 7f292e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56024f472bf0 0x56024f480f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37474/0 (socket says 172.21.15.37:37474) 2024-06-10T01:07:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.690+0000 7f292e7fc640 1 -- 172.21.15.37:0/2690351610 learned_addr learned my addr 172.21.15.37:0/2690351610 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.690+0000 7f292effd640 1 --2- 172.21.15.37:0/2690351610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56024f472200 0x56024f480a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.690+0000 7f292e7fc640 1 -- 172.21.15.37:0/2690351610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56024f477d60 msgr2=0x56024f481fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.690+0000 7f292e7fc640 1 --2- 172.21.15.37:0/2690351610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56024f477d60 0x56024f481fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.690+0000 7f292e7fc640 1 -- 172.21.15.37:0/2690351610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56024f472200 msgr2=0x56024f480a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.690+0000 7f292e7fc640 1 --2- 172.21.15.37:0/2690351610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56024f472200 0x56024f480a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.690+0000 7f292e7fc640 1 -- 172.21.15.37:0/2690351610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56024f482590 con 0x56024f472bf0 2024-06-10T01:07:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.690+0000 7f292e7fc640 1 --2- 172.21.15.37:0/2690351610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56024f472bf0 0x56024f480f90 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f2924028940 tx=0x7f2924028de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a4d3bf4e03b1f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.690+0000 7f2912ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:07:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.690+0000 7f2912ff5640 1 -- 172.21.15.37:0/2690351610 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292400e750 con 0x56024f472bf0 2024-06-10T01:07:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.690+0000 7f29381ef740 1 -- 172.21.15.37:0/2690351610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29200043c0 con 0x56024f472bf0 2024-06-10T01:07:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.690+0000 7f2912ff5640 1 -- 172.21.15.37:0/2690351610 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f2924028640 con 0x56024f472bf0 2024-06-10T01:07:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.690+0000 7f2912ff5640 1 -- 172.21.15.37:0/2690351610 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2924011b70 con 0x56024f472bf0 2024-06-10T01:07:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.690+0000 7f29381ef740 10 client.17313 fetch_fsmap learned FSMap version 31 2024-06-10T01:07:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.690+0000 7f29381ef740 1 -- 172.21.15.37:0/2690351610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56024f4838d0 con 0x56024f472bf0 2024-06-10T01:07:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.691+0000 7f2912ff5640 1 -- 172.21.15.37:0/2690351610 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29200043c0 con 0x56024f472bf0 2024-06-10T01:07:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.691+0000 7f2912ff5640 1 -- 172.21.15.37:0/2690351610 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f292401c030 con 0x56024f472bf0 2024-06-10T01:07:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.691+0000 7f29381ef740 10 client.17313 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:07:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.691+0000 7f29381ef740 10 client.17313 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:07:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.691+0000 7f29381ef740 20 client.17313 populate_metadata read hostname 'smithi037' 2024-06-10T01:07:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.691+0000 7f29381ef740 1 --2- 172.21.15.37:0/2690351610 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56024f4f3450 0x56024f4f5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.691+0000 7f29381ef740 4 client.17313 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:07:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.691+0000 7f29381ef740 1 -- 172.21.15.37:0/2690351610 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56024f4729b0 con 0x56024f4f3450 2024-06-10T01:07:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.691+0000 7f292effd640 1 --2- 172.21.15.37:0/2690351610 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56024f4f3450 0x56024f4f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.692+0000 7f2912ff5640 10 client.17313 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.692+0000 7f292effd640 1 --2- 172.21.15.37:0/2690351610 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56024f4f3450 0x56024f4f5870 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d7bc92e5dbb52d6 server_cookie=ec515b18f125e4c4 in_seq=0 out_seq=0 2024-06-10T01:07:53.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.917+0000 7fa33ccb4700 1 -- 172.21.15.37:0/1000262699 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3381a58b0 con 0x7fa3381030f0 2024-06-10T01:07:53.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.918+0000 7fa3277fe700 1 -- 172.21.15.37:0/1000262699 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11868 (secure 0 0 0) 0x7fa32c09be60 con 0x7fa3381030f0 2024-06-10T01:07:53.914 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:07:53.914 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":26554,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2107277223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2107277223},{"type":"v1","addr":"172.21.15.167:6835","nonce":2107277223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T01:06:09.604504+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24472,24451,24458]},"id":1}]} 2024-06-10T01:07:53.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.921+0000 7fa33ccb4700 1 -- 172.21.15.37:0/1000262699 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa3200a2ca0 msgr2=0x7fa3200a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:53.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.921+0000 7fa33ccb4700 1 --2- 172.21.15.37:0/1000262699 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa3200a2ca0 0x7fa3200a5160 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa328000c00 tx=0x7fa328005fb0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.921+0000 7fa33ccb4700 1 -- 172.21.15.37:0/1000262699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3381030f0 msgr2=0x7fa33819ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:53.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.921+0000 7fa33ccb4700 1 --2- 172.21.15.37:0/1000262699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3381030f0 0x7fa33819ec40 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00c9b0 tx=0x7fa32c00ccc0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.921+0000 7fa33ccb4700 1 -- 172.21.15.37:0/1000262699 shutdown_connections 2024-06-10T01:07:53.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.921+0000 7fa33ccb4700 1 --2- 172.21.15.37:0/1000262699 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa3200a2ca0 0x7fa3200a5160 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.921+0000 7fa33ccb4700 1 --2- 172.21.15.37:0/1000262699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa338107840 0x7fa3381988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.921+0000 7fa33ccb4700 1 --2- 172.21.15.37:0/1000262699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa338102740 0x7fa33819e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.921+0000 7fa33ccb4700 1 --2- 172.21.15.37:0/1000262699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3381030f0 0x7fa33819ec40 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.921+0000 7fa33ccb4700 1 -- 172.21.15.37:0/1000262699 >> 172.21.15.37:0/1000262699 conn(0x7fa3380795f0 msgr2=0x7fa3381003f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:53.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.921+0000 7fa33ccb4700 1 -- 172.21.15.37:0/1000262699 shutdown_connections 2024-06-10T01:07:53.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:53.921+0000 7fa33ccb4700 1 -- 172.21.15.37:0/1000262699 wait complete. 2024-06-10T01:07:53.920 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 31 2024-06-10T01:07:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.967+0000 7f2912ff5640 1 -- 172.21.15.37:0/2690351610 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56024f4729b0 con 0x56024f4f3450 2024-06-10T01:07:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.967+0000 7f2912ff5640 10 client.17313 handle_command_reply: tid=0 2024-06-10T01:07:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.967+0000 7f29381ef740 1 client.17313 shutdown 2024-06-10T01:07:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.967+0000 7f29381ef740 1 -- 172.21.15.37:0/2690351610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56024f472bf0 msgr2=0x56024f480f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.967+0000 7f29381ef740 1 --2- 172.21.15.37:0/2690351610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56024f472bf0 0x56024f480f90 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f2924028940 tx=0x7f2924028de0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.968+0000 7f29381ef740 1 -- 172.21.15.37:0/2690351610 shutdown_connections 2024-06-10T01:07:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.968+0000 7f29381ef740 1 --2- 172.21.15.37:0/2690351610 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56024f4f3450 0x56024f4f5870 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.968+0000 7f29381ef740 1 --2- 172.21.15.37:0/2690351610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56024f477d60 0x56024f481fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.968+0000 7f29381ef740 1 --2- 172.21.15.37:0/2690351610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56024f472bf0 0x56024f480f90 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.968+0000 7f2912ff5640 0 client.17313 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.968+0000 7f29381ef740 1 --2- 172.21.15.37:0/2690351610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56024f472200 0x56024f480a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.968+0000 7f29381ef740 1 -- 172.21.15.37:0/2690351610 >> 172.21.15.37:0/2690351610 conn(0x56024f46d870 msgr2=0x56024f473d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.968+0000 7f29381ef740 1 -- 172.21.15.37:0/2690351610 shutdown_connections 2024-06-10T01:07:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.968+0000 7f29381ef740 1 -- 172.21.15.37:0/2690351610 wait complete. 2024-06-10T01:07:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.968+0000 7f29381ef740 20 client.17313 trim_cache size 0 max 16384 2024-06-10T01:07:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:07:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:07:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:07:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.968+0000 7f292ffff640 1 -- 172.21.15.37:0/3561094220 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f29180a2b80 msgr2=0x7f29180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.968+0000 7f292ffff640 1 --2- 172.21.15.37:0/3561094220 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f29180a2b80 0x7f29180a5040 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f29301990b0 tx=0x7f2924013830 comp rx=0 tx=0).stop 2024-06-10T01:07:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.968+0000 7f292ffff640 1 -- 172.21.15.37:0/3561094220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29301022d0 msgr2=0x7f293019f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.968+0000 7f292ffff640 1 --2- 172.21.15.37:0/3561094220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29301022d0 0x7f293019f150 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f292800cad0 tx=0x7f292800cf90 comp rx=0 tx=0).stop 2024-06-10T01:07:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.969+0000 7f292ffff640 1 -- 172.21.15.37:0/3561094220 shutdown_connections 2024-06-10T01:07:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.969+0000 7f292ffff640 1 --2- 172.21.15.37:0/3561094220 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f29180a2b80 0x7f29180a5040 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.969+0000 7f292ffff640 1 --2- 172.21.15.37:0/3561094220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29301022d0 0x7f293019f150 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.969+0000 7f292ffff640 1 --2- 172.21.15.37:0/3561094220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2930101910 0x7f29301980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.969+0000 7f292ffff640 1 --2- 172.21.15.37:0/3561094220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2930100720 0x7f2930197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.969+0000 7f292ffff640 1 -- 172.21.15.37:0/3561094220 >> 172.21.15.37:0/3561094220 conn(0x7f29300fbc90 msgr2=0x7f29300fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.969+0000 7f292ffff640 1 -- 172.21.15.37:0/3561094220 shutdown_connections 2024-06-10T01:07:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:07:53.969+0000 7f292ffff640 1 -- 172.21.15.37:0/3561094220 wait complete. 2024-06-10T01:07:54.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:54 smithi167 ceph-mon[36917]: pgmap v767: 129 pgs: 129 active+clean; 16 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 617 KiB/s wr, 1.68k op/s 2024-06-10T01:07:54.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:54 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1000262699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:07:54.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:54 smithi094 ceph-mon[36708]: pgmap v767: 129 pgs: 129 active+clean; 16 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 617 KiB/s wr, 1.68k op/s 2024-06-10T01:07:54.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:54 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1000262699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:07:54.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:54 smithi037 ceph-mon[32192]: pgmap v767: 129 pgs: 129 active+clean; 16 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 617 KiB/s wr, 1.68k op/s 2024-06-10T01:07:54.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:54 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1000262699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:07:54.706 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24472 quiesce db --set-id 2e2e5b63 --release --await' 2024-06-10T01:07:54.706 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24472 quiesce db --set-id 2e2e5b63 --release --await 2024-06-10T01:07:54.991 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:07:55.374 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:55 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:07:55.375 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:55 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:07:55.375 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:55 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:07:55.375 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:55 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:07:55.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:55 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:07:55.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:55 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:07:55.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:55 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:07:55.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:55 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:07:55.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:55 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:07:55.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:55 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:07:55.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:55 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:07:55.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:55 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:07:56.503 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:56 smithi037 ceph-mon[32192]: pgmap v768: 129 pgs: 129 active+clean; 16 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 415 KiB/s wr, 1.98k op/s 2024-06-10T01:07:56.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:56 smithi167 ceph-mon[36917]: pgmap v768: 129 pgs: 129 active+clean; 16 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 415 KiB/s wr, 1.98k op/s 2024-06-10T01:07:56.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:56 smithi094 ceph-mon[36708]: pgmap v768: 129 pgs: 129 active+clean; 16 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 415 KiB/s wr, 1.98k op/s 2024-06-10T01:07:56.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.773+0000 7f7acee7e700 1 -- 172.21.15.37:0/2866784643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ac8101680 msgr2=0x7f7ac8101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:56.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.773+0000 7f7acee7e700 1 --2- 172.21.15.37:0/2866784643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ac8101680 0x7f7ac8101a60 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f7abc0099c0 tx=0x7f7abc009cd0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.773+0000 7f7acee7e700 1 -- 172.21.15.37:0/2866784643 shutdown_connections 2024-06-10T01:07:56.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.773+0000 7f7acee7e700 1 --2- 172.21.15.37:0/2866784643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ac81066d0 0x7f7ac810e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.773+0000 7f7acee7e700 1 --2- 172.21.15.37:0/2866784643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ac8101680 0x7f7ac8101a60 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.773+0000 7f7acee7e700 1 --2- 172.21.15.37:0/2866784643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ac8102030 0x7f7ac8105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.773+0000 7f7acee7e700 1 -- 172.21.15.37:0/2866784643 >> 172.21.15.37:0/2866784643 conn(0x7f7ac80773e0 msgr2=0x7f7ac80777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:56.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.773+0000 7f7acee7e700 1 -- 172.21.15.37:0/2866784643 shutdown_connections 2024-06-10T01:07:56.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.773+0000 7f7acee7e700 1 -- 172.21.15.37:0/2866784643 wait complete. 2024-06-10T01:07:56.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.773+0000 7f7acee7e700 1 Processor -- start 2024-06-10T01:07:56.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.773+0000 7f7acee7e700 1 -- start start 2024-06-10T01:07:56.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.774+0000 7f7acee7e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ac8101680 0x7f7ac8197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.774+0000 7f7acee7e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ac8102030 0x7f7ac8197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.774+0000 7f7acee7e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ac81066d0 0x7f7ac81982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.774+0000 7f7acee7e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81116b0 con 0x7f7ac8102030 2024-06-10T01:07:56.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.774+0000 7f7acee7e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac8111530 con 0x7f7ac81066d0 2024-06-10T01:07:56.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.774+0000 7f7acee7e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac8111830 con 0x7f7ac8101680 2024-06-10T01:07:56.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.774+0000 7f7ac7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ac8102030 0x7f7ac8197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.774+0000 7f7ac7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ac8102030 0x7f7ac8197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37490/0 (socket says 172.21.15.37:37490) 2024-06-10T01:07:56.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.774+0000 7f7ac7fff700 1 -- 172.21.15.37:0/3373637424 learned_addr learned my addr 172.21.15.37:0/3373637424 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:56.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.774+0000 7f7accc1a700 1 --2- 172.21.15.37:0/3373637424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ac8101680 0x7f7ac8197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.774+0000 7f7acd41b700 1 --2- 172.21.15.37:0/3373637424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ac81066d0 0x7f7ac81982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.774+0000 7f7ac7fff700 1 -- 172.21.15.37:0/3373637424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ac8101680 msgr2=0x7f7ac8197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:56.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.774+0000 7f7ac7fff700 1 --2- 172.21.15.37:0/3373637424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ac8101680 0x7f7ac8197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.774+0000 7f7ac7fff700 1 -- 172.21.15.37:0/3373637424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ac81066d0 msgr2=0x7f7ac81982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:56.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.774+0000 7f7ac7fff700 1 --2- 172.21.15.37:0/3373637424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ac81066d0 0x7f7ac81982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.774+0000 7f7ac7fff700 1 -- 172.21.15.37:0/3373637424 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac81a7070 con 0x7f7ac8102030 2024-06-10T01:07:56.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.774+0000 7f7acd41b700 1 --2- 172.21.15.37:0/3373637424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ac81066d0 0x7f7ac81982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:56.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.775+0000 7f7ac7fff700 1 --2- 172.21.15.37:0/3373637424 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ac8102030 0x7f7ac8197da0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400c9e0 tx=0x7f7ab400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:56.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.775+0000 7f7ac5ffb700 1 -- 172.21.15.37:0/3373637424 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4007aa0 con 0x7f7ac8102030 2024-06-10T01:07:56.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.775+0000 7f7acee7e700 1 -- 172.21.15.37:0/3373637424 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ac81a7360 con 0x7f7ac8102030 2024-06-10T01:07:56.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.775+0000 7f7ac5ffb700 1 -- 172.21.15.37:0/3373637424 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ab4007c40 con 0x7f7ac8102030 2024-06-10T01:07:56.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.775+0000 7f7ac5ffb700 1 -- 172.21.15.37:0/3373637424 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab40186f0 con 0x7f7ac8102030 2024-06-10T01:07:56.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.775+0000 7f7acee7e700 1 -- 172.21.15.37:0/3373637424 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ac81a78a0 con 0x7f7ac8102030 2024-06-10T01:07:56.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.777+0000 7f7ac5ffb700 1 -- 172.21.15.37:0/3373637424 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7ab4018890 con 0x7f7ac8102030 2024-06-10T01:07:56.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.778+0000 7f7ac5ffb700 1 --2- 172.21.15.37:0/3373637424 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7ab80a2c20 0x7f7ab80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.778+0000 7f7ac5ffb700 1 -- 172.21.15.37:0/3373637424 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f7ab40cf3f0 con 0x7f7ac8102030 2024-06-10T01:07:56.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.778+0000 7f7accc1a700 1 --2- 172.21.15.37:0/3373637424 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7ab80a2c20 0x7f7ab80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.779+0000 7f7accc1a700 1 --2- 172.21.15.37:0/3373637424 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7ab80a2c20 0x7f7ab80a50e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7abc000c00 tx=0x7f7abc005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:56.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.779+0000 7f7ad9701b80 1 Processor -- start 2024-06-10T01:07:56.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.779+0000 7f7ad9701b80 1 -- start start 2024-06-10T01:07:56.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.779+0000 7f7ad9701b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557889b29c40 0x557889b2a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.779+0000 7f7ad9701b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557889b2a560 0x557889b23e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.779+0000 7f7ad9701b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557889b243d0 0x557889b24810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.779+0000 7f7ad9701b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557889a75910 con 0x557889b243d0 2024-06-10T01:07:56.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.779+0000 7f7ad9701b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557889a1d820 con 0x557889b29c40 2024-06-10T01:07:56.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.779+0000 7f7ad9701b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557889a718d0 con 0x557889b2a560 2024-06-10T01:07:56.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.779+0000 7f7acd41b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557889b243d0 0x557889b24810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.779+0000 7f7acd41b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557889b243d0 0x557889b24810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37498/0 (socket says 172.21.15.37:37498) 2024-06-10T01:07:56.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.779+0000 7f7acd41b700 1 -- 172.21.15.37:0/1731567203 learned_addr learned my addr 172.21.15.37:0/1731567203 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:56.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.779+0000 7f7accc1a700 1 --2- 172.21.15.37:0/1731567203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557889b29c40 0x557889b2a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.779+0000 7f7ac7fff700 1 --2- 172.21.15.37:0/1731567203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557889b2a560 0x557889b23e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.780+0000 7f7acd41b700 1 -- 172.21.15.37:0/1731567203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557889b2a560 msgr2=0x557889b23e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:56.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.780+0000 7f7acd41b700 1 --2- 172.21.15.37:0/1731567203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557889b2a560 0x557889b23e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.780+0000 7f7acd41b700 1 -- 172.21.15.37:0/1731567203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557889b29c40 msgr2=0x557889b2a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:56.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.780+0000 7f7acd41b700 1 --2- 172.21.15.37:0/1731567203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557889b29c40 0x557889b2a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.780+0000 7f7acd41b700 1 -- 172.21.15.37:0/1731567203 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557889a14050 con 0x557889b243d0 2024-06-10T01:07:56.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.780+0000 7f7acd41b700 1 --2- 172.21.15.37:0/1731567203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557889b243d0 0x557889b24810 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f7ac000e160 tx=0x7f7ac000e470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6b202db271f8d9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:56.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.780+0000 7f7ab2ffd700 1 -- 172.21.15.37:0/1731567203 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac000ed00 con 0x557889b243d0 2024-06-10T01:07:56.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.780+0000 7f7ab2ffd700 1 -- 172.21.15.37:0/1731567203 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ac0004bb0 con 0x557889b243d0 2024-06-10T01:07:56.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.780+0000 7f7ab2ffd700 1 -- 172.21.15.37:0/1731567203 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0004410 con 0x557889b243d0 2024-06-10T01:07:56.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.781+0000 7f7ad9701b80 1 -- 172.21.15.37:0/1731567203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557889b243d0 msgr2=0x557889b24810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:56.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.781+0000 7f7ad9701b80 1 --2- 172.21.15.37:0/1731567203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557889b243d0 0x557889b24810 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f7ac000e160 tx=0x7f7ac000e470 comp rx=0 tx=0).stop 2024-06-10T01:07:56.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.781+0000 7f7ad9701b80 1 -- 172.21.15.37:0/1731567203 shutdown_connections 2024-06-10T01:07:56.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.781+0000 7f7ad9701b80 1 --2- 172.21.15.37:0/1731567203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557889b243d0 0x557889b24810 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.781+0000 7f7ad9701b80 1 --2- 172.21.15.37:0/1731567203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557889b29c40 0x557889b2a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.781+0000 7f7ad9701b80 1 --2- 172.21.15.37:0/1731567203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557889b2a560 0x557889b23e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.781+0000 7f7ad9701b80 1 -- 172.21.15.37:0/1731567203 >> 172.21.15.37:0/1731567203 conn(0x557889a4edc0 msgr2=0x557889b1f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:56.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.781+0000 7f7ad9701b80 1 -- 172.21.15.37:0/1731567203 shutdown_connections 2024-06-10T01:07:56.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.781+0000 7f7ad9701b80 1 -- 172.21.15.37:0/1731567203 wait complete. 2024-06-10T01:07:56.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.781+0000 7f7ad9701b80 1 -- start start 2024-06-10T01:07:56.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.781+0000 7f7ad9701b80 1 Processor -- start 2024-06-10T01:07:56.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.782+0000 7f7ad9701b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557889b243d0 0x557889b30260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.782+0000 7f7ad9701b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557889b29c40 0x557889b2d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.782+0000 7f7ad9701b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557889b2a560 0x557889b2daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.782+0000 7f7ad9701b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557889b30fd0 con 0x557889b2a560 2024-06-10T01:07:56.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.782+0000 7f7ad9701b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557889a15cf0 con 0x557889b243d0 2024-06-10T01:07:56.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.782+0000 7f7ad9701b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557889b31150 con 0x557889b29c40 2024-06-10T01:07:56.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.782+0000 7f7ad9701b80 10 client.0 _ll_register_callbacks cb 0x557889a0fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:56.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.782+0000 7f7acd41b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557889b2a560 0x557889b2daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.782+0000 7f7acd41b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557889b2a560 0x557889b2daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37510/0 (socket says 172.21.15.37:37510) 2024-06-10T01:07:56.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.782+0000 7f7acd41b700 1 -- 172.21.15.37:0/1603511010 learned_addr learned my addr 172.21.15.37:0/1603511010 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:56.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.782+0000 7f7ac7fff700 1 --2- 172.21.15.37:0/1603511010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557889b29c40 0x557889b2d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.782+0000 7f7accc1a700 1 --2- 172.21.15.37:0/1603511010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557889b243d0 0x557889b30260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.782+0000 7f7acd41b700 1 -- 172.21.15.37:0/1603511010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557889b29c40 msgr2=0x557889b2d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:56.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.782+0000 7f7acd41b700 1 --2- 172.21.15.37:0/1603511010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557889b29c40 0x557889b2d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.782+0000 7f7acd41b700 1 -- 172.21.15.37:0/1603511010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557889b243d0 msgr2=0x557889b30260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:56.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.782+0000 7f7acd41b700 1 --2- 172.21.15.37:0/1603511010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557889b243d0 0x557889b30260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.782+0000 7f7acd41b700 1 -- 172.21.15.37:0/1603511010 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578899c49d0 con 0x557889b2a560 2024-06-10T01:07:56.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.783+0000 7f7acd41b700 1 --2- 172.21.15.37:0/1603511010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557889b2a560 0x557889b2daa0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0004170 tx=0x7f7ac0004000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a945979b0ab4dea0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:56.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.783+0000 7f7aa7fff700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:07:56.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.783+0000 7f7aa7fff700 1 -- 172.21.15.37:0/1603511010 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0023020 con 0x557889b2a560 2024-06-10T01:07:56.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.783+0000 7f7ad9701b80 1 -- 172.21.15.37:0/1603511010 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ac0004bb0 con 0x557889b2a560 2024-06-10T01:07:56.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.783+0000 7f7aa7fff700 1 -- 172.21.15.37:0/1603511010 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f7ac0010040 con 0x557889b2a560 2024-06-10T01:07:56.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.783+0000 7f7aa7fff700 1 -- 172.21.15.37:0/1603511010 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7ac002e050 con 0x557889b2a560 2024-06-10T01:07:56.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.783+0000 7f7ad9701b80 10 client.17325 fetch_fsmap learned FSMap version 31 2024-06-10T01:07:56.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.783+0000 7f7ad9701b80 1 -- 172.21.15.37:0/1603511010 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557889b3b860 con 0x557889b2a560 2024-06-10T01:07:56.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.783+0000 7f7aa7fff700 1 -- 172.21.15.37:0/1603511010 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ac0004bb0 con 0x557889b2a560 2024-06-10T01:07:56.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.783+0000 7f7aa7fff700 1 -- 172.21.15.37:0/1603511010 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f7ac0014680 con 0x557889b2a560 2024-06-10T01:07:56.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.783+0000 7f7ad9701b80 10 client.17325 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:07:56.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.784+0000 7f7ad9701b80 10 client.17325 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:07:56.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.784+0000 7f7ad9701b80 20 client.17325 populate_metadata read hostname 'smithi037' 2024-06-10T01:07:56.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.784+0000 7f7ad9701b80 1 --2- 172.21.15.37:0/1603511010 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557889b3de40 0x557889b40230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.784+0000 7f7ad9701b80 4 client.17325 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:07:56.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.784+0000 7f7ad9701b80 1 -- 172.21.15.37:0/1603511010 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557889a82e10 con 0x557889b3de40 2024-06-10T01:07:56.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.784+0000 7f7accc1a700 1 --2- 172.21.15.37:0/1603511010 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557889b3de40 0x557889b40230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.784+0000 7f7accc1a700 1 --2- 172.21.15.37:0/1603511010 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557889b3de40 0x557889b40230 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e5f75a7a0ce61f7 server_cookie=bf8c5a9b30f2951f in_seq=0 out_seq=0 2024-06-10T01:07:56.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.784+0000 7f7aa7fff700 10 client.17325 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:56.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.785+0000 7f7aa7fff700 1 -- 172.21.15.37:0/1603511010 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557889a82e10 con 0x557889b3de40 2024-06-10T01:07:56.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.785+0000 7f7aa7fff700 10 client.17325 handle_command_reply: tid=0 2024-06-10T01:07:56.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.785+0000 7f7ad9701b80 1 client.17325 shutdown 2024-06-10T01:07:56.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.785+0000 7f7ad9701b80 1 -- 172.21.15.37:0/1603511010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557889b2a560 msgr2=0x557889b2daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:56.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.785+0000 7f7ad9701b80 1 --2- 172.21.15.37:0/1603511010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557889b2a560 0x557889b2daa0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0004170 tx=0x7f7ac0004000 comp rx=0 tx=0).stop 2024-06-10T01:07:56.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.785+0000 7f7ad9701b80 1 -- 172.21.15.37:0/1603511010 shutdown_connections 2024-06-10T01:07:56.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.785+0000 7f7ad9701b80 1 --2- 172.21.15.37:0/1603511010 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557889b3de40 0x557889b40230 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.785+0000 7f7ad9701b80 1 --2- 172.21.15.37:0/1603511010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557889b2a560 0x557889b2daa0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.785+0000 7f7ad9701b80 1 --2- 172.21.15.37:0/1603511010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557889b243d0 0x557889b30260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.785+0000 7f7aa7fff700 0 client.17325 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:56.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.785+0000 7f7ad9701b80 1 --2- 172.21.15.37:0/1603511010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557889b29c40 0x557889b2d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.785+0000 7f7ad9701b80 1 -- 172.21.15.37:0/1603511010 >> 172.21.15.37:0/1603511010 conn(0x557889a4edc0 msgr2=0x557889b1ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:56.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.786+0000 7f7ad9701b80 1 -- 172.21.15.37:0/1603511010 shutdown_connections 2024-06-10T01:07:56.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.786+0000 7f7ad9701b80 1 -- 172.21.15.37:0/1603511010 wait complete. 2024-06-10T01:07:56.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.786+0000 7f7ad9701b80 20 client.17325 trim_cache size 0 max 16384 2024-06-10T01:07:56.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.806+0000 7f7ad9701b80 1 Processor -- start 2024-06-10T01:07:56.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.806+0000 7f7ad9701b80 1 -- start start 2024-06-10T01:07:56.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.806+0000 7f7ad9701b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557889b243d0 0x557889b23e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.806+0000 7f7ad9701b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557889b29c40 0x557889b27a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.806+0000 7f7ad9701b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557889b2a560 0x557889b45800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.806+0000 7f7ad9701b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81b2f70 con 0x557889b243d0 2024-06-10T01:07:56.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.806+0000 7f7ad9701b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81b0800 con 0x557889b2a560 2024-06-10T01:07:56.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.806+0000 7f7ad9701b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81ae650 con 0x557889b29c40 2024-06-10T01:07:56.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.806+0000 7f7accc1a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557889b243d0 0x557889b23e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.806+0000 7f7accc1a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557889b243d0 0x557889b23e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37518/0 (socket says 172.21.15.37:37518) 2024-06-10T01:07:56.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.806+0000 7f7accc1a700 1 -- 172.21.15.37:0/1071499702 learned_addr learned my addr 172.21.15.37:0/1071499702 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:56.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.807+0000 7f7acd41b700 1 --2- 172.21.15.37:0/1071499702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557889b2a560 0x557889b45800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.807+0000 7f7ac7fff700 1 --2- 172.21.15.37:0/1071499702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557889b29c40 0x557889b27a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.807+0000 7f7accc1a700 1 -- 172.21.15.37:0/1071499702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557889b29c40 msgr2=0x557889b27a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:56.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.807+0000 7f7accc1a700 1 --2- 172.21.15.37:0/1071499702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557889b29c40 0x557889b27a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.807+0000 7f7accc1a700 1 -- 172.21.15.37:0/1071499702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557889b2a560 msgr2=0x557889b45800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:56.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.807+0000 7f7accc1a700 1 --2- 172.21.15.37:0/1071499702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557889b2a560 0x557889b45800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.807+0000 7f7accc1a700 1 -- 172.21.15.37:0/1071499702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557889b30a40 con 0x557889b243d0 2024-06-10T01:07:56.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.807+0000 7f7ac7fff700 1 --2- 172.21.15.37:0/1071499702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557889b29c40 0x557889b27a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:07:56.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.807+0000 7f7acd41b700 1 --2- 172.21.15.37:0/1071499702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557889b2a560 0x557889b45800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:07:56.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.807+0000 7f7accc1a700 1 --2- 172.21.15.37:0/1071499702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557889b243d0 0x557889b23e90 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009750 tx=0x7f7abc020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336ca0537371caaf server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:56.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.807+0000 7f7ab27fc700 1 -- 172.21.15.37:0/1071499702 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc00fdd0 con 0x557889b243d0 2024-06-10T01:07:56.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.807+0000 7f7ab27fc700 1 -- 172.21.15.37:0/1071499702 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7abc03c8f0 con 0x557889b243d0 2024-06-10T01:07:56.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.807+0000 7f7ab27fc700 1 -- 172.21.15.37:0/1071499702 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc024910 con 0x557889b243d0 2024-06-10T01:07:56.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.808+0000 7f7ad9701b80 1 -- 172.21.15.37:0/1071499702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557889b243d0 msgr2=0x557889b23e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:56.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.808+0000 7f7ad9701b80 1 --2- 172.21.15.37:0/1071499702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557889b243d0 0x557889b23e90 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009750 tx=0x7f7abc020000 comp rx=0 tx=0).stop 2024-06-10T01:07:56.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.808+0000 7f7ad9701b80 1 -- 172.21.15.37:0/1071499702 shutdown_connections 2024-06-10T01:07:56.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.808+0000 7f7ad9701b80 1 --2- 172.21.15.37:0/1071499702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557889b2a560 0x557889b45800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.808+0000 7f7ad9701b80 1 --2- 172.21.15.37:0/1071499702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557889b243d0 0x557889b23e90 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.808+0000 7f7ad9701b80 1 --2- 172.21.15.37:0/1071499702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557889b29c40 0x557889b27a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.808+0000 7f7ad9701b80 1 -- 172.21.15.37:0/1071499702 >> 172.21.15.37:0/1071499702 conn(0x557889a4edc0 msgr2=0x557889b1e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:56.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.808+0000 7f7ad9701b80 1 -- 172.21.15.37:0/1071499702 shutdown_connections 2024-06-10T01:07:56.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.808+0000 7f7ad9701b80 1 -- 172.21.15.37:0/1071499702 wait complete. 2024-06-10T01:07:56.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.808+0000 7f7ad9701b80 1 -- start start 2024-06-10T01:07:56.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.808+0000 7f7ad9701b80 1 Processor -- start 2024-06-10T01:07:56.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.809+0000 7f7ad9701b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557889b243d0 0x557889b366d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.809+0000 7f7ad9701b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557889b29c40 0x557889b36c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.809+0000 7f7ad9701b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557889b2a560 0x557889b333e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.809+0000 7f7ad9701b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81aecc0 con 0x557889b2a560 2024-06-10T01:07:56.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.809+0000 7f7ad9701b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81b0130 con 0x557889b243d0 2024-06-10T01:07:56.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.809+0000 7f7ad9701b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81af390 con 0x557889b29c40 2024-06-10T01:07:56.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.809+0000 7f7ad9701b80 10 client.0 _ll_register_callbacks cb 0x557889a0fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:07:56.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.809+0000 7f7acd41b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557889b2a560 0x557889b333e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.809+0000 7f7acd41b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557889b2a560 0x557889b333e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37522/0 (socket says 172.21.15.37:37522) 2024-06-10T01:07:56.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.809+0000 7f7acd41b700 1 -- 172.21.15.37:0/2780510559 learned_addr learned my addr 172.21.15.37:0/2780510559 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:07:56.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.809+0000 7f7ac7fff700 1 --2- 172.21.15.37:0/2780510559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557889b29c40 0x557889b36c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.809+0000 7f7accc1a700 1 --2- 172.21.15.37:0/2780510559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557889b243d0 0x557889b366d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.809+0000 7f7acd41b700 1 -- 172.21.15.37:0/2780510559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557889b29c40 msgr2=0x557889b36c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:56.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.809+0000 7f7acd41b700 1 --2- 172.21.15.37:0/2780510559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557889b29c40 0x557889b36c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.809+0000 7f7acd41b700 1 -- 172.21.15.37:0/2780510559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557889b243d0 msgr2=0x557889b366d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:07:56.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.809+0000 7f7acd41b700 1 --2- 172.21.15.37:0/2780510559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557889b243d0 0x557889b366d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:56.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.809+0000 7f7acd41b700 1 -- 172.21.15.37:0/2780510559 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557889b33c20 con 0x557889b2a560 2024-06-10T01:07:56.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.810+0000 7f7acd41b700 1 --2- 172.21.15.37:0/2780510559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557889b2a560 0x557889b333e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0002230 tx=0x7f7ac00040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584623f668aee9e6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:07:56.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.810+0000 7f7aa7fff700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:07:56.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.810+0000 7f7aa7fff700 1 -- 172.21.15.37:0/2780510559 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0023040 con 0x557889b2a560 2024-06-10T01:07:56.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.810+0000 7f7ad9701b80 1 -- 172.21.15.37:0/2780510559 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7abc03c8f0 con 0x557889b2a560 2024-06-10T01:07:56.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.810+0000 7f7aa7fff700 1 -- 172.21.15.37:0/2780510559 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f7ac0019050 con 0x557889b2a560 2024-06-10T01:07:56.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.810+0000 7f7aa7fff700 1 -- 172.21.15.37:0/2780510559 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7ac0005e70 con 0x557889b2a560 2024-06-10T01:07:56.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.810+0000 7f7ad9701b80 10 client.17331 fetch_fsmap learned FSMap version 31 2024-06-10T01:07:56.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.810+0000 7f7ad9701b80 1 -- 172.21.15.37:0/2780510559 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557889ba92d0 con 0x557889b2a560 2024-06-10T01:07:56.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.810+0000 7f7aa7fff700 1 -- 172.21.15.37:0/2780510559 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7abc03c8f0 con 0x557889b2a560 2024-06-10T01:07:56.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.810+0000 7f7aa7fff700 1 -- 172.21.15.37:0/2780510559 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f7ac00329b0 con 0x557889b2a560 2024-06-10T01:07:56.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.810+0000 7f7ad9701b80 10 client.17331 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:07:56.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.810+0000 7f7ad9701b80 10 client.17331 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:07:56.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.810+0000 7f7ad9701b80 20 client.17331 populate_metadata read hostname 'smithi037' 2024-06-10T01:07:56.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.810+0000 7f7ad9701b80 1 --2- 172.21.15.37:0/2780510559 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557889ba9970 0x557889babd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:07:56.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.810+0000 7f7ad9701b80 4 client.17331 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2e2e5b63", "release": true, "await": true}] 2024-06-10T01:07:56.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.810+0000 7f7ad9701b80 1 -- 172.21.15.37:0/2780510559 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2e2e5b63", "release": true, "await": true}) -- 0x557889a7d390 con 0x557889ba9970 2024-06-10T01:07:56.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.811+0000 7f7accc1a700 1 --2- 172.21.15.37:0/2780510559 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557889ba9970 0x557889babd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:07:56.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.811+0000 7f7aa7fff700 10 client.17331 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:56.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:56.811+0000 7f7accc1a700 1 --2- 172.21.15.37:0/2780510559 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557889ba9970 0x557889babd60 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38a2bb11a4c50699 server_cookie=d39c9c784f5a4de2 in_seq=0 out_seq=0 2024-06-10T01:07:57.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:57.651+0000 7f7aa7fff700 1 -- 172.21.15.37:0/2780510559 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557889a7d390 con 0x557889ba9970 2024-06-10T01:07:57.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:57.651+0000 7f7aa7fff700 10 client.17331 handle_command_reply: tid=0 2024-06-10T01:07:57.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:57.651+0000 7f7ad9701b80 1 client.17331 shutdown 2024-06-10T01:07:57.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:57.651+0000 7f7ad9701b80 1 -- 172.21.15.37:0/2780510559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557889b2a560 msgr2=0x557889b333e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:57.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:57.651+0000 7f7ad9701b80 1 --2- 172.21.15.37:0/2780510559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557889b2a560 0x557889b333e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0002230 tx=0x7f7ac00040e0 comp rx=0 tx=0).stop 2024-06-10T01:07:57.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:57.651+0000 7f7ad9701b80 1 -- 172.21.15.37:0/2780510559 shutdown_connections 2024-06-10T01:07:57.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:57.651+0000 7f7ad9701b80 1 --2- 172.21.15.37:0/2780510559 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557889ba9970 0x557889babd60 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:57.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:57.651+0000 7f7ad9701b80 1 --2- 172.21.15.37:0/2780510559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557889b2a560 0x557889b333e0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:57.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:57.651+0000 7f7ad9701b80 1 --2- 172.21.15.37:0/2780510559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557889b243d0 0x557889b366d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:57.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:57.651+0000 7f7ad9701b80 1 --2- 172.21.15.37:0/2780510559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557889b29c40 0x557889b36c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:57.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:57.651+0000 7f7aa7fff700 0 client.17331 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:07:57.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:57.651+0000 7f7ad9701b80 1 -- 172.21.15.37:0/2780510559 >> 172.21.15.37:0/2780510559 conn(0x557889a4edc0 msgr2=0x557889b2a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:57.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:57.651+0000 7f7ad9701b80 1 -- 172.21.15.37:0/2780510559 shutdown_connections 2024-06-10T01:07:57.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:57.652+0000 7f7ad9701b80 1 -- 172.21.15.37:0/2780510559 wait complete. 2024-06-10T01:07:57.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:57.652+0000 7f7ad9701b80 20 client.17331 trim_cache size 0 max 16384 2024-06-10T01:07:57.649 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:07:57.649 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 30, 2024-06-10T01:07:57.649 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24472, 2024-06-10T01:07:57.650 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 28, 2024-06-10T01:07:57.650 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-06-10T01:07:57.650 INFO:teuthology.orchestra.run.smithi037.stdout: "2e2e5b63": { 2024-06-10T01:07:57.650 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 28, 2024-06-10T01:07:57.650 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-06-10T01:07:57.650 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:07:57.650 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-06-10T01:07:57.650 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:07:57.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:07:57.650 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-06-10T01:07:57.650 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 137.8, 2024-06-10T01:07:57.650 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-06-10T01:07:57.650 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-06-10T01:07:57.651 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-06-10T01:07:57.651 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:07:57.651 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-06-10T01:07:57.651 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:07:57.651 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:07:57.651 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:07:57.651 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:07:57.651 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:07:57.651 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:07:57.651 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:07:57.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:57.652+0000 7f7acee7e700 1 -- 172.21.15.37:0/3373637424 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7ab80a2c20 msgr2=0x7f7ab80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:57.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:57.652+0000 7f7acee7e700 1 --2- 172.21.15.37:0/3373637424 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7ab80a2c20 0x7f7ab80a50e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7abc000c00 tx=0x7f7abc005fb0 comp rx=0 tx=0).stop 2024-06-10T01:07:57.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:57.652+0000 7f7acee7e700 1 -- 172.21.15.37:0/3373637424 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ac8102030 msgr2=0x7f7ac8197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:07:57.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:57.652+0000 7f7acee7e700 1 --2- 172.21.15.37:0/3373637424 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ac8102030 0x7f7ac8197da0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400c9e0 tx=0x7f7ab400ccf0 comp rx=0 tx=0).stop 2024-06-10T01:07:57.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:57.653+0000 7f7acee7e700 1 -- 172.21.15.37:0/3373637424 shutdown_connections 2024-06-10T01:07:57.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:57.653+0000 7f7acee7e700 1 --2- 172.21.15.37:0/3373637424 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f7ab80a2c20 0x7f7ab80a50e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:57.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:57.653+0000 7f7acee7e700 1 --2- 172.21.15.37:0/3373637424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ac81066d0 0x7f7ac81982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:57.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:57.653+0000 7f7acee7e700 1 --2- 172.21.15.37:0/3373637424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ac8101680 0x7f7ac8197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:57.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:57.653+0000 7f7acee7e700 1 --2- 172.21.15.37:0/3373637424 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ac8102030 0x7f7ac8197da0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:07:57.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:57.653+0000 7f7acee7e700 1 -- 172.21.15.37:0/3373637424 >> 172.21.15.37:0/3373637424 conn(0x7f7ac80773e0 msgr2=0x7f7ac81005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:07:57.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:57.653+0000 7f7acee7e700 1 -- 172.21.15.37:0/3373637424 shutdown_connections 2024-06-10T01:07:57.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:07:57.653+0000 7f7acee7e700 1 -- 172.21.15.37:0/3373637424 wait complete. 2024-06-10T01:07:58.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:07:58 smithi037 ceph-mon[32192]: pgmap v769: 129 pgs: 129 active+clean; 12 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 150 KiB/s wr, 1.92k op/s 2024-06-10T01:07:58.460 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2e2e5b63', total seconds elapsed: 25.2 2024-06-10T01:07:58.461 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.8 2024-06-10T01:07:58.461 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:07:58.564 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:07:58.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:07:58 smithi167 ceph-mon[36917]: pgmap v769: 129 pgs: 129 active+clean; 12 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 150 KiB/s wr, 1.92k op/s 2024-06-10T01:07:58.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:07:58 smithi094 ceph-mon[36708]: pgmap v769: 129 pgs: 129 active+clean; 12 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 150 KiB/s wr, 1.92k op/s 2024-06-10T01:07:58.734 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:08:00.151 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:08:00.557 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:00 smithi037 ceph-mon[32192]: pgmap v770: 129 pgs: 129 active+clean; 4.6 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 162 KiB/s wr, 2.02k op/s 2024-06-10T01:08:00.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:00 smithi167 ceph-mon[36917]: pgmap v770: 129 pgs: 129 active+clean; 4.6 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 162 KiB/s wr, 2.02k op/s 2024-06-10T01:08:00.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:00 smithi094 ceph-mon[36708]: pgmap v770: 129 pgs: 129 active+clean; 4.6 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 162 KiB/s wr, 2.02k op/s 2024-06-10T01:08:02.259 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:02 smithi037 ceph-mon[32192]: pgmap v771: 129 pgs: 129 active+clean; 4.6 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 73 KiB/s wr, 1.20k op/s 2024-06-10T01:08:02.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:02 smithi167 ceph-mon[36917]: pgmap v771: 129 pgs: 129 active+clean; 4.6 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 73 KiB/s wr, 1.20k op/s 2024-06-10T01:08:02.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:02 smithi094 ceph-mon[36708]: pgmap v771: 129 pgs: 129 active+clean; 4.6 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 73 KiB/s wr, 1.20k op/s 2024-06-10T01:08:03.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.222+0000 7effab194700 1 -- 172.21.15.37:0/3854574215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effa406aa00 msgr2=0x7effa406ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:03.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.222+0000 7effab194700 1 --2- 172.21.15.37:0/3854574215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effa406aa00 0x7effa406ae80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7eff90009990 tx=0x7eff90009ca0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.222+0000 7effab194700 1 -- 172.21.15.37:0/3854574215 shutdown_connections 2024-06-10T01:08:03.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.222+0000 7effab194700 1 --2- 172.21.15.37:0/3854574215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7effa406b3c0 0x7effa410f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.222+0000 7effab194700 1 --2- 172.21.15.37:0/3854574215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effa41084f0 0x7effa41088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.222+0000 7effab194700 1 --2- 172.21.15.37:0/3854574215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effa406aa00 0x7effa406ae80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.222+0000 7effab194700 1 -- 172.21.15.37:0/3854574215 >> 172.21.15.37:0/3854574215 conn(0x7effa4078300 msgr2=0x7effa4078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:03.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.222+0000 7effab194700 1 -- 172.21.15.37:0/3854574215 shutdown_connections 2024-06-10T01:08:03.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.222+0000 7effab194700 1 -- 172.21.15.37:0/3854574215 wait complete. 2024-06-10T01:08:03.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.223+0000 7effab194700 1 Processor -- start 2024-06-10T01:08:03.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.223+0000 7effab194700 1 -- start start 2024-06-10T01:08:03.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.223+0000 7effab194700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effa406aa00 0x7effa4101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:03.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.223+0000 7effab194700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7effa406b3c0 0x7effa4101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:03.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.223+0000 7effab194700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effa41084f0 0x7effa4101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:03.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.223+0000 7effab194700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7effa41127a0 con 0x7effa406aa00 2024-06-10T01:08:03.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.223+0000 7effab194700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7effa4112620 con 0x7effa406b3c0 2024-06-10T01:08:03.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.223+0000 7effab194700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7effa4112920 con 0x7effa41084f0 2024-06-10T01:08:03.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.223+0000 7effa8f30700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effa406aa00 0x7effa4101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:03.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.223+0000 7effa8f30700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effa406aa00 0x7effa4101450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39836/0 (socket says 172.21.15.37:39836) 2024-06-10T01:08:03.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.223+0000 7effa8f30700 1 -- 172.21.15.37:0/774178656 learned_addr learned my addr 172.21.15.37:0/774178656 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:03.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.223+0000 7effa3fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7effa406b3c0 0x7effa4101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:03.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.223+0000 7effa9731700 1 --2- 172.21.15.37:0/774178656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effa41084f0 0x7effa4101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:03.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.224+0000 7effa8f30700 1 -- 172.21.15.37:0/774178656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effa41084f0 msgr2=0x7effa4101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:03.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.224+0000 7effa8f30700 1 --2- 172.21.15.37:0/774178656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effa41084f0 0x7effa4101ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.224+0000 7effa8f30700 1 -- 172.21.15.37:0/774178656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7effa406b3c0 msgr2=0x7effa4101990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:03.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.224+0000 7effa8f30700 1 --2- 172.21.15.37:0/774178656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7effa406b3c0 0x7effa4101990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.224+0000 7effa8f30700 1 -- 172.21.15.37:0/774178656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa41a7190 con 0x7effa406aa00 2024-06-10T01:08:03.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.224+0000 7effa3fff700 1 --2- 172.21.15.37:0/774178656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7effa406b3c0 0x7effa4101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:03.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.224+0000 7effa8f30700 1 --2- 172.21.15.37:0/774178656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effa406aa00 0x7effa4101450 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7eff9800eb30 tx=0x7eff9800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:03.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.224+0000 7effa1ffb700 1 -- 172.21.15.37:0/774178656 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9800cbd0 con 0x7effa406aa00 2024-06-10T01:08:03.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.224+0000 7effab194700 1 -- 172.21.15.37:0/774178656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa41a7480 con 0x7effa406aa00 2024-06-10T01:08:03.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.224+0000 7effa1ffb700 1 -- 172.21.15.37:0/774178656 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff98004500 con 0x7effa406aa00 2024-06-10T01:08:03.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.224+0000 7effa1ffb700 1 -- 172.21.15.37:0/774178656 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98010430 con 0x7effa406aa00 2024-06-10T01:08:03.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.225+0000 7effab194700 1 -- 172.21.15.37:0/774178656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa41a79c0 con 0x7effa406aa00 2024-06-10T01:08:03.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.226+0000 7effab194700 1 -- 172.21.15.37:0/774178656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effa4069180 con 0x7effa406aa00 2024-06-10T01:08:03.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.227+0000 7effa1ffb700 1 -- 172.21.15.37:0/774178656 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7eff98003f00 con 0x7effa406aa00 2024-06-10T01:08:03.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.227+0000 7effa1ffb700 1 --2- 172.21.15.37:0/774178656 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7eff940a2c20 0x7eff940a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:03.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.228+0000 7effa3fff700 1 --2- 172.21.15.37:0/774178656 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7eff940a2c20 0x7eff940a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:03.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.228+0000 7effa1ffb700 1 -- 172.21.15.37:0/774178656 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7eff980cf730 con 0x7effa406aa00 2024-06-10T01:08:03.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.228+0000 7effa3fff700 1 --2- 172.21.15.37:0/774178656 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7eff940a2c20 0x7eff940a50e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7effa4078920 tx=0x7eff90005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:03.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.236+0000 7effa1ffb700 1 -- 172.21.15.37:0/774178656 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7eff9809c2a0 con 0x7effa406aa00 2024-06-10T01:08:03.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.167+0000 7fe3e3325700 1 -- 172.21.15.37:0/2111166023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3dc108500 msgr2=0x7fe3dc1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:03.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.168+0000 7fe3e3325700 1 --2- 172.21.15.37:0/2111166023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3dc108500 0x7fe3dc1088e0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8009990 tx=0x7fe3d8009ca0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.168+0000 7fe3e3325700 1 -- 172.21.15.37:0/2111166023 shutdown_connections 2024-06-10T01:08:03.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.168+0000 7fe3e3325700 1 --2- 172.21.15.37:0/2111166023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3dc06ab30 0x7fe3dc10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.168+0000 7fe3e3325700 1 --2- 172.21.15.37:0/2111166023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3dc108500 0x7fe3dc1088e0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.168+0000 7fe3e3325700 1 --2- 172.21.15.37:0/2111166023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3dc06a170 0x7fe3dc06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.168+0000 7fe3e3325700 1 -- 172.21.15.37:0/2111166023 >> 172.21.15.37:0/2111166023 conn(0x7fe3dc078340 msgr2=0x7fe3dc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:03.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.168+0000 7fe3e3325700 1 -- 172.21.15.37:0/2111166023 shutdown_connections 2024-06-10T01:08:03.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.168+0000 7fe3e3325700 1 -- 172.21.15.37:0/2111166023 wait complete. 2024-06-10T01:08:03.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.168+0000 7fe3e3325700 1 Processor -- start 2024-06-10T01:08:03.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.169+0000 7fe3e3325700 1 -- start start 2024-06-10T01:08:03.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.169+0000 7fe3e3325700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3dc06a170 0x7fe3dc19aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:03.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.169+0000 7fe3e3325700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3dc06ab30 0x7fe3dc19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:03.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.169+0000 7fe3e3325700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3dc108500 0x7fe3dc19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:03.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.169+0000 7fe3e3325700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc1127d0 con 0x7fe3dc108500 2024-06-10T01:08:03.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.169+0000 7fe3e3325700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc112650 con 0x7fe3dc06ab30 2024-06-10T01:08:03.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.169+0000 7fe3e3325700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc112950 con 0x7fe3dc06a170 2024-06-10T01:08:03.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.169+0000 7fe3e10c1700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3dc06a170 0x7fe3dc19aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:03.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.169+0000 7fe3e10c1700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3dc06a170 0x7fe3dc19aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:50256/0 (socket says 172.21.15.37:50256) 2024-06-10T01:08:03.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.169+0000 7fe3e10c1700 1 -- 172.21.15.37:0/1464674261 learned_addr learned my addr 172.21.15.37:0/1464674261 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:03.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.169+0000 7fe3e18c2700 1 --2- 172.21.15.37:0/1464674261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3dc108500 0x7fe3dc19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:03.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.169+0000 7fe3e08c0700 1 --2- 172.21.15.37:0/1464674261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3dc06ab30 0x7fe3dc19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:03.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.169+0000 7fe3e10c1700 1 -- 172.21.15.37:0/1464674261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3dc06ab30 msgr2=0x7fe3dc19b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:03.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.169+0000 7fe3e10c1700 1 --2- 172.21.15.37:0/1464674261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3dc06ab30 0x7fe3dc19b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.169+0000 7fe3e10c1700 1 -- 172.21.15.37:0/1464674261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3dc108500 msgr2=0x7fe3dc19f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:03.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.169+0000 7fe3e10c1700 1 --2- 172.21.15.37:0/1464674261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3dc108500 0x7fe3dc19f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.169+0000 7fe3e10c1700 1 -- 172.21.15.37:0/1464674261 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3dc19fee0 con 0x7fe3dc06a170 2024-06-10T01:08:03.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.170+0000 7fe3e10c1700 1 --2- 172.21.15.37:0/1464674261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3dc06a170 0x7fe3dc19aed0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8005910 tx=0x7fe3d800bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:03.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.170+0000 7fe3d27fc700 1 -- 172.21.15.37:0/1464674261 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8027070 con 0x7fe3dc06a170 2024-06-10T01:08:03.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.170+0000 7fe3d27fc700 1 -- 172.21.15.37:0/1464674261 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3d8003d70 con 0x7fe3dc06a170 2024-06-10T01:08:03.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.170+0000 7fe3d27fc700 1 -- 172.21.15.37:0/1464674261 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d800f460 con 0x7fe3dc06a170 2024-06-10T01:08:03.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.170+0000 7fe3e3325700 1 -- 172.21.15.37:0/1464674261 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3dc1a0170 con 0x7fe3dc06a170 2024-06-10T01:08:03.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.170+0000 7fe3e3325700 1 -- 172.21.15.37:0/1464674261 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3dc1a7a50 con 0x7fe3dc06a170 2024-06-10T01:08:03.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.173+0000 7fe3d27fc700 1 -- 172.21.15.37:0/1464674261 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe3d802a070 con 0x7fe3dc06a170 2024-06-10T01:08:03.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.173+0000 7fe3d27fc700 1 --2- 172.21.15.37:0/1464674261 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe3c80a2c20 0x7fe3c80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:03.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.173+0000 7fe3d27fc700 1 -- 172.21.15.37:0/1464674261 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fe3d80d02a0 con 0x7fe3dc06a170 2024-06-10T01:08:03.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.174+0000 7fe3e08c0700 1 --2- 172.21.15.37:0/1464674261 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe3c80a2c20 0x7fe3c80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:03.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.174+0000 7fe3edba8b80 1 Processor -- start 2024-06-10T01:08:03.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.174+0000 7fe3edba8b80 1 -- start start 2024-06-10T01:08:03.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.174+0000 7fe3edba8b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fe80ec640 0x562fe80eca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:03.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.174+0000 7fe3e08c0700 1 --2- 172.21.15.37:0/1464674261 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe3c80a2c20 0x7fe3c80a50e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc103420 tx=0x7fe3cc00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:03.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.174+0000 7fe3edba8b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fe80ecf60 0x562fe80e69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:03.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.174+0000 7fe3edba8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562fe80e6f20 0x562fe80e7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:03.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.174+0000 7fe3edba8b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562fe8038810 con 0x562fe80ecf60 2024-06-10T01:08:03.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.174+0000 7fe3edba8b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562fe7fe0720 con 0x562fe80ec640 2024-06-10T01:08:03.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.174+0000 7fe3edba8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562fe80347d0 con 0x562fe80e6f20 2024-06-10T01:08:03.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.174+0000 7fe3e08c0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fe80ecf60 0x562fe80e69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:03.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.174+0000 7fe3e08c0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fe80ecf60 0x562fe80e69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39800/0 (socket says 172.21.15.37:39800) 2024-06-10T01:08:03.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.174+0000 7fe3e08c0700 1 -- 172.21.15.37:0/280037508 learned_addr learned my addr 172.21.15.37:0/280037508 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:03.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.174+0000 7fe3e10c1700 1 --2- 172.21.15.37:0/280037508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fe80ec640 0x562fe80eca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:03.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.174+0000 7fe3e18c2700 1 --2- 172.21.15.37:0/280037508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562fe80e6f20 0x562fe80e7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:03.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.175+0000 7fe3e10c1700 1 -- 172.21.15.37:0/280037508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562fe80e6f20 msgr2=0x562fe80e7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:03.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.175+0000 7fe3e10c1700 1 --2- 172.21.15.37:0/280037508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562fe80e6f20 0x562fe80e7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.175+0000 7fe3e10c1700 1 -- 172.21.15.37:0/280037508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fe80ecf60 msgr2=0x562fe80e69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:03.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.175+0000 7fe3e10c1700 1 --2- 172.21.15.37:0/280037508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fe80ecf60 0x562fe80e69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.175+0000 7fe3e10c1700 1 -- 172.21.15.37:0/280037508 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fe7fd6f50 con 0x562fe80ec640 2024-06-10T01:08:03.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.175+0000 7fe3e08c0700 1 --2- 172.21.15.37:0/280037508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fe80ecf60 0x562fe80e69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:03.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.175+0000 7fe3e10c1700 1 --2- 172.21.15.37:0/280037508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fe80ec640 0x562fe80eca20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80cda00 tx=0x7fe3d8009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa9b8df46a9de25e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:03.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.175+0000 7fe3c77fe700 1 -- 172.21.15.37:0/280037508 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d809c100 con 0x562fe80ec640 2024-06-10T01:08:03.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.175+0000 7fe3c77fe700 1 -- 172.21.15.37:0/280037508 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3d80043d0 con 0x562fe80ec640 2024-06-10T01:08:03.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.175+0000 7fe3c77fe700 1 -- 172.21.15.37:0/280037508 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d80a5380 con 0x562fe80ec640 2024-06-10T01:08:03.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.176+0000 7fe3edba8b80 1 -- 172.21.15.37:0/280037508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fe80ec640 msgr2=0x562fe80eca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:03.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.176+0000 7fe3edba8b80 1 --2- 172.21.15.37:0/280037508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fe80ec640 0x562fe80eca20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80cda00 tx=0x7fe3d8009ca0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.176+0000 7fe3edba8b80 1 -- 172.21.15.37:0/280037508 shutdown_connections 2024-06-10T01:08:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.176+0000 7fe3edba8b80 1 --2- 172.21.15.37:0/280037508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562fe80e6f20 0x562fe80e7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.176+0000 7fe3edba8b80 1 --2- 172.21.15.37:0/280037508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fe80ec640 0x562fe80eca20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.176+0000 7fe3edba8b80 1 --2- 172.21.15.37:0/280037508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fe80ecf60 0x562fe80e69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.176+0000 7fe3edba8b80 1 -- 172.21.15.37:0/280037508 >> 172.21.15.37:0/280037508 conn(0x562fe8011cc0 msgr2=0x562fe80e1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.176+0000 7fe3edba8b80 1 -- 172.21.15.37:0/280037508 shutdown_connections 2024-06-10T01:08:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.176+0000 7fe3edba8b80 1 -- 172.21.15.37:0/280037508 wait complete. 2024-06-10T01:08:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.177+0000 7fe3edba8b80 1 -- start start 2024-06-10T01:08:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.177+0000 7fe3edba8b80 1 Processor -- start 2024-06-10T01:08:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.177+0000 7fe3edba8b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fe80e6f20 0x562fe80f2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.177+0000 7fe3edba8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562fe80ec640 0x562fe80f0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.177+0000 7fe3edba8b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fe80ecf60 0x562fe80f0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.177+0000 7fe3edba8b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562fe80f36f0 con 0x562fe80e6f20 2024-06-10T01:08:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.177+0000 7fe3edba8b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562fe7fd8bf0 con 0x562fe80ecf60 2024-06-10T01:08:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.177+0000 7fe3edba8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562fe80f3870 con 0x562fe80ec640 2024-06-10T01:08:03.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.177+0000 7fe3edba8b80 10 client.0 _ll_register_callbacks cb 0x562fe7fd2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:03.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.177+0000 7fe3e18c2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fe80ecf60 0x562fe80f0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:03.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.177+0000 7fe3e10c1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fe80e6f20 0x562fe80f2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:03.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.177+0000 7fe3e18c2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fe80ecf60 0x562fe80f0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:53582/0 (socket says 172.21.15.37:53582) 2024-06-10T01:08:03.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.177+0000 7fe3e18c2700 1 -- 172.21.15.37:0/3173854307 learned_addr learned my addr 172.21.15.37:0/3173854307 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:03.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.177+0000 7fe3e10c1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fe80e6f20 0x562fe80f2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39802/0 (socket says 172.21.15.37:39802) 2024-06-10T01:08:03.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.178+0000 7fe3e08c0700 1 --2- 172.21.15.37:0/3173854307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562fe80ec640 0x562fe80f0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:03.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.178+0000 7fe3e18c2700 1 -- 172.21.15.37:0/3173854307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562fe80ec640 msgr2=0x562fe80f0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:03.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.178+0000 7fe3e18c2700 1 --2- 172.21.15.37:0/3173854307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562fe80ec640 0x562fe80f0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.178+0000 7fe3e18c2700 1 -- 172.21.15.37:0/3173854307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fe80e6f20 msgr2=0x562fe80f2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:03.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.178+0000 7fe3e18c2700 1 --2- 172.21.15.37:0/3173854307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fe80e6f20 0x562fe80f2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.178+0000 7fe3e18c2700 1 -- 172.21.15.37:0/3173854307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fe7f878d0 con 0x562fe80ecf60 2024-06-10T01:08:03.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.178+0000 7fe3e18c2700 1 --2- 172.21.15.37:0/3173854307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fe80ecf60 0x562fe80f0590 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4009fd0 tx=0x7fe3d4007b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de8f78d54051a2ff server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:03.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.178+0000 7fe3c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:08:03.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.178+0000 7fe3c4ff9700 1 -- 172.21.15.37:0/3173854307 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d401a070 con 0x562fe80ecf60 2024-06-10T01:08:03.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.178+0000 7fe3edba8b80 1 -- 172.21.15.37:0/3173854307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3d80043d0 con 0x562fe80ecf60 2024-06-10T01:08:03.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.178+0000 7fe3c4ff9700 1 -- 172.21.15.37:0/3173854307 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fe3d40138c0 con 0x562fe80ecf60 2024-06-10T01:08:03.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.178+0000 7fe3c4ff9700 1 -- 172.21.15.37:0/3173854307 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe3d4026050 con 0x562fe80ecf60 2024-06-10T01:08:03.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.178+0000 7fe3edba8b80 10 client.26699 fetch_fsmap learned FSMap version 31 2024-06-10T01:08:03.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.178+0000 7fe3edba8b80 1 -- 172.21.15.37:0/3173854307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562fe80fe070 con 0x562fe80ecf60 2024-06-10T01:08:03.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.178+0000 7fe3c4ff9700 1 -- 172.21.15.37:0/3173854307 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3d80043d0 con 0x562fe80ecf60 2024-06-10T01:08:03.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.179+0000 7fe3c4ff9700 1 -- 172.21.15.37:0/3173854307 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fe3d401e3b0 con 0x562fe80ecf60 2024-06-10T01:08:03.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.179+0000 7fe3edba8b80 10 client.26699 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:08:03.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.179+0000 7fe3edba8b80 10 client.26699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:03.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.179+0000 7fe3edba8b80 20 client.26699 populate_metadata read hostname 'smithi037' 2024-06-10T01:08:03.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.179+0000 7fe3edba8b80 1 --2- 172.21.15.37:0/3173854307 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x562fe8100a80 0x562fe8102ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:03.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.179+0000 7fe3edba8b80 4 client.26699 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:08:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.179+0000 7fe3edba8b80 1 -- 172.21.15.37:0/3173854307 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fe8045d10 con 0x562fe8100a80 2024-06-10T01:08:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.179+0000 7fe3e10c1700 1 --2- 172.21.15.37:0/3173854307 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x562fe8100a80 0x562fe8102ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.179+0000 7fe3c4ff9700 10 client.26699 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.179+0000 7fe3e10c1700 1 --2- 172.21.15.37:0/3173854307 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x562fe8100a80 0x562fe8102ea0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a79ab1139b56e33f server_cookie=242812b1f52725d7 in_seq=0 out_seq=0 2024-06-10T01:08:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.183+0000 7fe3c4ff9700 1 -- 172.21.15.37:0/3173854307 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562fe8045d10 con 0x562fe8100a80 2024-06-10T01:08:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.183+0000 7fe3c4ff9700 10 client.26699 handle_command_reply: tid=0 2024-06-10T01:08:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.183+0000 7fe3edba8b80 1 client.26699 shutdown 2024-06-10T01:08:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.183+0000 7fe3edba8b80 1 -- 172.21.15.37:0/3173854307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fe80ecf60 msgr2=0x562fe80f0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.183+0000 7fe3edba8b80 1 --2- 172.21.15.37:0/3173854307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fe80ecf60 0x562fe80f0590 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4009fd0 tx=0x7fe3d4007b10 comp rx=0 tx=0).stop 2024-06-10T01:08:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.183+0000 7fe3edba8b80 1 -- 172.21.15.37:0/3173854307 shutdown_connections 2024-06-10T01:08:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.183+0000 7fe3edba8b80 1 --2- 172.21.15.37:0/3173854307 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x562fe8100a80 0x562fe8102ea0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.183+0000 7fe3edba8b80 1 --2- 172.21.15.37:0/3173854307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fe80ecf60 0x562fe80f0590 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.183+0000 7fe3edba8b80 1 --2- 172.21.15.37:0/3173854307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fe80e6f20 0x562fe80f2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.183+0000 7fe3edba8b80 1 --2- 172.21.15.37:0/3173854307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562fe80ec640 0x562fe80f0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.183+0000 7fe3c4ff9700 0 client.26699 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.183+0000 7fe3edba8b80 1 -- 172.21.15.37:0/3173854307 >> 172.21.15.37:0/3173854307 conn(0x562fe8011cc0 msgr2=0x562fe80e15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.184+0000 7fe3edba8b80 1 -- 172.21.15.37:0/3173854307 shutdown_connections 2024-06-10T01:08:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.184+0000 7fe3edba8b80 1 -- 172.21.15.37:0/3173854307 wait complete. 2024-06-10T01:08:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.184+0000 7fe3edba8b80 20 client.26699 trim_cache size 0 max 16384 2024-06-10T01:08:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.206+0000 7fe3edba8b80 1 Processor -- start 2024-06-10T01:08:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.206+0000 7fe3edba8b80 1 -- start start 2024-06-10T01:08:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.206+0000 7fe3edba8b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fe80e6f20 0x562fe80e69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.206+0000 7fe3edba8b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fe80ec640 0x562fe80f0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.206+0000 7fe3edba8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562fe80ecf60 0x562fe80f0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.206+0000 7fe3edba8b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc1b2e30 con 0x562fe80e6f20 2024-06-10T01:08:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.206+0000 7fe3edba8b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc1b06c0 con 0x562fe80ec640 2024-06-10T01:08:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.206+0000 7fe3edba8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc1ae510 con 0x562fe80ecf60 2024-06-10T01:08:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.206+0000 7fe3e10c1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fe80e6f20 0x562fe80e69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:03.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.206+0000 7fe3e10c1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fe80e6f20 0x562fe80e69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39818/0 (socket says 172.21.15.37:39818) 2024-06-10T01:08:03.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.206+0000 7fe3e10c1700 1 -- 172.21.15.37:0/372728087 learned_addr learned my addr 172.21.15.37:0/372728087 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:03.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.206+0000 7fe3e18c2700 1 --2- 172.21.15.37:0/372728087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562fe80ecf60 0x562fe80f0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:03.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.206+0000 7fe3e08c0700 1 --2- 172.21.15.37:0/372728087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fe80ec640 0x562fe80f0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:03.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.206+0000 7fe3e10c1700 1 -- 172.21.15.37:0/372728087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562fe80ecf60 msgr2=0x562fe80f0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:03.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.206+0000 7fe3e10c1700 1 --2- 172.21.15.37:0/372728087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562fe80ecf60 0x562fe80f0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.206+0000 7fe3e10c1700 1 -- 172.21.15.37:0/372728087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fe80ec640 msgr2=0x562fe80f0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:03.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.206+0000 7fe3e10c1700 1 --2- 172.21.15.37:0/372728087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fe80ec640 0x562fe80f0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.206+0000 7fe3e10c1700 1 -- 172.21.15.37:0/372728087 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fe80f0a90 con 0x562fe80e6f20 2024-06-10T01:08:03.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.207+0000 7fe3e10c1700 1 --2- 172.21.15.37:0/372728087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fe80e6f20 0x562fe80e69e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8098c00 tx=0x7fe3d8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e82dcc6290412f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:03.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.207+0000 7fe3c6ffd700 1 -- 172.21.15.37:0/372728087 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d80a30b0 con 0x562fe80e6f20 2024-06-10T01:08:03.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.207+0000 7fe3c6ffd700 1 -- 172.21.15.37:0/372728087 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3d80043d0 con 0x562fe80e6f20 2024-06-10T01:08:03.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.207+0000 7fe3c6ffd700 1 -- 172.21.15.37:0/372728087 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8022650 con 0x562fe80e6f20 2024-06-10T01:08:03.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.208+0000 7fe3edba8b80 1 -- 172.21.15.37:0/372728087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fe80e6f20 msgr2=0x562fe80e69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:03.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.208+0000 7fe3edba8b80 1 --2- 172.21.15.37:0/372728087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fe80e6f20 0x562fe80e69e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8098c00 tx=0x7fe3d8009ca0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.208+0000 7fe3edba8b80 1 -- 172.21.15.37:0/372728087 shutdown_connections 2024-06-10T01:08:03.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.208+0000 7fe3edba8b80 1 --2- 172.21.15.37:0/372728087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562fe80ecf60 0x562fe80f0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.208+0000 7fe3edba8b80 1 --2- 172.21.15.37:0/372728087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fe80e6f20 0x562fe80e69e0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.208+0000 7fe3edba8b80 1 --2- 172.21.15.37:0/372728087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fe80ec640 0x562fe80f0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.208+0000 7fe3edba8b80 1 -- 172.21.15.37:0/372728087 >> 172.21.15.37:0/372728087 conn(0x562fe8011cc0 msgr2=0x562fe80e1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:03.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.208+0000 7fe3edba8b80 1 -- 172.21.15.37:0/372728087 shutdown_connections 2024-06-10T01:08:03.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.208+0000 7fe3edba8b80 1 -- 172.21.15.37:0/372728087 wait complete. 2024-06-10T01:08:03.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.208+0000 7fe3edba8b80 1 -- start start 2024-06-10T01:08:03.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.208+0000 7fe3edba8b80 1 Processor -- start 2024-06-10T01:08:03.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.209+0000 7fe3edba8b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fe80e6f20 0x562fe80f8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:03.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.209+0000 7fe3edba8b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fe80ec640 0x562fe80f92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:03.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.209+0000 7fe3e10c1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fe80e6f20 0x562fe80f8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:03.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.209+0000 7fe3e10c1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fe80e6f20 0x562fe80f8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39822/0 (socket says 172.21.15.37:39822) 2024-06-10T01:08:03.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.209+0000 7fe3edba8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562fe80ecf60 0x562fe8107dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:03.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.209+0000 7fe3edba8b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc1aeb80 con 0x562fe80e6f20 2024-06-10T01:08:03.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.209+0000 7fe3e10c1700 1 -- 172.21.15.37:0/4202872103 learned_addr learned my addr 172.21.15.37:0/4202872103 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:03.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.209+0000 7fe3edba8b80 1 -- 172.21.15.37:0/4202872103 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc1afff0 con 0x562fe80ec640 2024-06-10T01:08:03.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.209+0000 7fe3edba8b80 1 -- 172.21.15.37:0/4202872103 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc1af250 con 0x562fe80ecf60 2024-06-10T01:08:03.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.209+0000 7fe3e08c0700 1 --2- 172.21.15.37:0/4202872103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fe80ec640 0x562fe80f92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:03.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.209+0000 7fe3e18c2700 1 --2- 172.21.15.37:0/4202872103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562fe80ecf60 0x562fe8107dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:03.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.209+0000 7fe3edba8b80 10 client.0 _ll_register_callbacks cb 0x562fe7fd2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:03.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.209+0000 7fe3e10c1700 1 -- 172.21.15.37:0/4202872103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562fe80ecf60 msgr2=0x562fe8107dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:03.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.210+0000 7fe3e10c1700 1 --2- 172.21.15.37:0/4202872103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562fe80ecf60 0x562fe8107dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.210+0000 7fe3e10c1700 1 -- 172.21.15.37:0/4202872103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fe80ec640 msgr2=0x562fe80f92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:03.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.210+0000 7fe3e10c1700 1 --2- 172.21.15.37:0/4202872103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fe80ec640 0x562fe80f92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.210+0000 7fe3e10c1700 1 -- 172.21.15.37:0/4202872103 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fe8108600 con 0x562fe80e6f20 2024-06-10T01:08:03.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.210+0000 7fe3e08c0700 1 --2- 172.21.15.37:0/4202872103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fe80ec640 0x562fe80f92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:03.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.210+0000 7fe3e10c1700 1 --2- 172.21.15.37:0/4202872103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fe80e6f20 0x562fe80f8d80 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe3d809c4a0 tx=0x7fe3d8022a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c4d132a1f40753 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:03.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.210+0000 7fe3c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:08:03.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.210+0000 7fe3c4ff9700 1 -- 172.21.15.37:0/4202872103 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d80b9070 con 0x562fe80e6f20 2024-06-10T01:08:03.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.210+0000 7fe3c4ff9700 1 -- 172.21.15.37:0/4202872103 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fe3d80870b0 con 0x562fe80e6f20 2024-06-10T01:08:03.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.210+0000 7fe3edba8b80 1 -- 172.21.15.37:0/4202872103 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3d80043d0 con 0x562fe80e6f20 2024-06-10T01:08:03.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.210+0000 7fe3c4ff9700 1 -- 172.21.15.37:0/4202872103 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe3d809cea0 con 0x562fe80e6f20 2024-06-10T01:08:03.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.210+0000 7fe3edba8b80 10 client.17349 fetch_fsmap learned FSMap version 31 2024-06-10T01:08:03.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.210+0000 7fe3edba8b80 1 -- 172.21.15.37:0/4202872103 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562fe816b2d0 con 0x562fe80e6f20 2024-06-10T01:08:03.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.211+0000 7fe3c4ff9700 1 -- 172.21.15.37:0/4202872103 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3d80043d0 con 0x562fe80e6f20 2024-06-10T01:08:03.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.211+0000 7fe3c4ff9700 1 -- 172.21.15.37:0/4202872103 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fe3d80a1ce0 con 0x562fe80e6f20 2024-06-10T01:08:03.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.211+0000 7fe3edba8b80 10 client.17349 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:08:03.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.211+0000 7fe3edba8b80 10 client.17349 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:03.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.211+0000 7fe3edba8b80 20 client.17349 populate_metadata read hostname 'smithi037' 2024-06-10T01:08:03.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.211+0000 7fe3edba8b80 1 --2- 172.21.15.37:0/4202872103 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x562fe816b500 0x562fe816d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:03.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.211+0000 7fe3edba8b80 4 client.17349 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:08:03.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.211+0000 7fe3edba8b80 1 -- 172.21.15.37:0/4202872103 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562fe8040290 con 0x562fe816b500 2024-06-10T01:08:03.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.211+0000 7fe3e08c0700 1 --2- 172.21.15.37:0/4202872103 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x562fe816b500 0x562fe816d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:03.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.212+0000 7fe3e08c0700 1 --2- 172.21.15.37:0/4202872103 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x562fe816b500 0x562fe816d8f0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39d89d8c113ced1a server_cookie=f53dc8fb01b8573f in_seq=0 out_seq=0 2024-06-10T01:08:03.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.212+0000 7fe3c4ff9700 10 client.17349 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:03.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.219+0000 7fe3c4ff9700 1 -- 172.21.15.37:0/4202872103 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562fe8040290 con 0x562fe816b500 2024-06-10T01:08:03.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.219+0000 7fe3c4ff9700 10 client.17349 handle_command_reply: tid=0 2024-06-10T01:08:03.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.219+0000 7fe3edba8b80 1 client.17349 shutdown 2024-06-10T01:08:03.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.219+0000 7fe3edba8b80 1 -- 172.21.15.37:0/4202872103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fe80e6f20 msgr2=0x562fe80f8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:03.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.219+0000 7fe3edba8b80 1 --2- 172.21.15.37:0/4202872103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fe80e6f20 0x562fe80f8d80 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe3d809c4a0 tx=0x7fe3d8022a20 comp rx=0 tx=0).stop 2024-06-10T01:08:03.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.219+0000 7fe3edba8b80 1 -- 172.21.15.37:0/4202872103 shutdown_connections 2024-06-10T01:08:03.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.219+0000 7fe3edba8b80 1 --2- 172.21.15.37:0/4202872103 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x562fe816b500 0x562fe816d8f0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.220+0000 7fe3edba8b80 1 --2- 172.21.15.37:0/4202872103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562fe80ecf60 0x562fe8107dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.370 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:08:03.370 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-06-10T01:08:03.370 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-06-10T01:08:03.370 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:08:03.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.220+0000 7fe3edba8b80 1 --2- 172.21.15.37:0/4202872103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562fe80e6f20 0x562fe80f8d80 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.220+0000 7fe3c4ff9700 0 client.17349 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:03.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.220+0000 7fe3edba8b80 1 --2- 172.21.15.37:0/4202872103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562fe80ec640 0x562fe80f92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.220+0000 7fe3edba8b80 1 -- 172.21.15.37:0/4202872103 >> 172.21.15.37:0/4202872103 conn(0x562fe8011cc0 msgr2=0x562fe80dae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:03.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.220+0000 7fe3edba8b80 1 -- 172.21.15.37:0/4202872103 shutdown_connections 2024-06-10T01:08:03.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.220+0000 7fe3edba8b80 1 -- 172.21.15.37:0/4202872103 wait complete. 2024-06-10T01:08:03.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.220+0000 7fe3edba8b80 20 client.17349 trim_cache size 0 max 16384 2024-06-10T01:08:03.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.221+0000 7fe3e3325700 1 -- 172.21.15.37:0/1464674261 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe3c80a2c20 msgr2=0x7fe3c80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:03.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.221+0000 7fe3e3325700 1 --2- 172.21.15.37:0/1464674261 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe3c80a2c20 0x7fe3c80a50e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc103420 tx=0x7fe3cc00b410 comp rx=0 tx=0).stop 2024-06-10T01:08:03.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.221+0000 7fe3e3325700 1 -- 172.21.15.37:0/1464674261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3dc06a170 msgr2=0x7fe3dc19aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:03.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.221+0000 7fe3e3325700 1 --2- 172.21.15.37:0/1464674261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3dc06a170 0x7fe3dc19aed0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8005910 tx=0x7fe3d800bab0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.221+0000 7fe3e3325700 1 -- 172.21.15.37:0/1464674261 shutdown_connections 2024-06-10T01:08:03.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.221+0000 7fe3e3325700 1 --2- 172.21.15.37:0/1464674261 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe3c80a2c20 0x7fe3c80a50e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.221+0000 7fe3e3325700 1 --2- 172.21.15.37:0/1464674261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3dc108500 0x7fe3dc19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.221+0000 7fe3e3325700 1 --2- 172.21.15.37:0/1464674261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3dc06a170 0x7fe3dc19aed0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.221+0000 7fe3e3325700 1 --2- 172.21.15.37:0/1464674261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3dc06ab30 0x7fe3dc19b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.221+0000 7fe3e3325700 1 -- 172.21.15.37:0/1464674261 >> 172.21.15.37:0/1464674261 conn(0x7fe3dc078340 msgr2=0x7fe3dc10d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:03.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.221+0000 7fe3e3325700 1 -- 172.21.15.37:0/1464674261 shutdown_connections 2024-06-10T01:08:03.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.221+0000 7fe3e3325700 1 -- 172.21.15.37:0/1464674261 wait complete. 2024-06-10T01:08:03.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.506+0000 7effab194700 1 -- 172.21.15.37:0/774178656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effa402dca0 con 0x7effa406aa00 2024-06-10T01:08:03.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.507+0000 7effa1ffb700 1 -- 172.21.15.37:0/774178656 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11868 (secure 0 0 0) 0x7eff9809bba0 con 0x7effa406aa00 2024-06-10T01:08:03.523 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:08:03.524 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":26554,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2107277223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2107277223},{"type":"v1","addr":"172.21.15.167:6835","nonce":2107277223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T01:06:09.604504+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24472,24451,24458]},"id":1}]} 2024-06-10T01:08:03.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.509+0000 7effab194700 1 -- 172.21.15.37:0/774178656 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7eff940a2c20 msgr2=0x7eff940a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:03.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.509+0000 7effab194700 1 --2- 172.21.15.37:0/774178656 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7eff940a2c20 0x7eff940a50e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7effa4078920 tx=0x7eff90005fd0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.509+0000 7effab194700 1 -- 172.21.15.37:0/774178656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effa406aa00 msgr2=0x7effa4101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:03.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.509+0000 7effab194700 1 --2- 172.21.15.37:0/774178656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effa406aa00 0x7effa4101450 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7eff9800eb30 tx=0x7eff9800ee40 comp rx=0 tx=0).stop 2024-06-10T01:08:03.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.509+0000 7effab194700 1 -- 172.21.15.37:0/774178656 shutdown_connections 2024-06-10T01:08:03.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.509+0000 7effab194700 1 --2- 172.21.15.37:0/774178656 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7eff940a2c20 0x7eff940a50e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.509+0000 7effab194700 1 --2- 172.21.15.37:0/774178656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effa41084f0 0x7effa4101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.509+0000 7effab194700 1 --2- 172.21.15.37:0/774178656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effa406aa00 0x7effa4101450 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.509+0000 7effab194700 1 --2- 172.21.15.37:0/774178656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7effa406b3c0 0x7effa4101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:03.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.509+0000 7effab194700 1 -- 172.21.15.37:0/774178656 >> 172.21.15.37:0/774178656 conn(0x7effa4078300 msgr2=0x7effa40fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:03.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.510+0000 7effab194700 1 -- 172.21.15.37:0/774178656 shutdown_connections 2024-06-10T01:08:03.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:03.510+0000 7effab194700 1 -- 172.21.15.37:0/774178656 wait complete. 2024-06-10T01:08:03.525 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 31 2024-06-10T01:08:04.394 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24472 quiesce db / --timeout 90 --expiration 137.8 --await-for 45.0' 2024-06-10T01:08:04.395 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24472 quiesce db / --timeout 90 --expiration 137.8 --await-for 45.0 2024-06-10T01:08:04.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:04 smithi167 ceph-mon[36917]: pgmap v772: 129 pgs: 129 active+clean; 5.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 185 KiB/s wr, 956 op/s 2024-06-10T01:08:04.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:04 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/774178656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:08:04.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:04 smithi094 ceph-mon[36708]: pgmap v772: 129 pgs: 129 active+clean; 5.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 185 KiB/s wr, 956 op/s 2024-06-10T01:08:04.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:04 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/774178656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:08:04.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:04 smithi037 ceph-mon[32192]: pgmap v772: 129 pgs: 129 active+clean; 5.9 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 185 KiB/s wr, 956 op/s 2024-06-10T01:08:04.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:04 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/774178656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:08:05.278 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:08:05.279 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:08:05.389 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:08:06.095 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:08:06.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:06 smithi037 ceph-mon[32192]: pgmap v773: 129 pgs: 129 active+clean; 10 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 529 KiB/s wr, 690 op/s 2024-06-10T01:08:06.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:06 smithi167 ceph-mon[36917]: pgmap v773: 129 pgs: 129 active+clean; 10 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 529 KiB/s wr, 690 op/s 2024-06-10T01:08:06.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:06 smithi094 ceph-mon[36708]: pgmap v773: 129 pgs: 129 active+clean; 10 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 529 KiB/s wr, 690 op/s 2024-06-10T01:08:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.062+0000 7fa812b89640 1 -- 172.21.15.37:0/984845931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa80c100bd0 msgr2=0x7fa80c101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.062+0000 7fa812b89640 1 --2- 172.21.15.37:0/984845931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa80c100bd0 0x7fa80c101030 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8009900 tx=0x7fa7f802a410 comp rx=0 tx=0).stop 2024-06-10T01:08:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.062+0000 7fa812b89640 1 -- 172.21.15.37:0/984845931 shutdown_connections 2024-06-10T01:08:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.062+0000 7fa812b89640 1 --2- 172.21.15.37:0/984845931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa80c101570 0x7fa80c1019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.062+0000 7fa812b89640 1 --2- 172.21.15.37:0/984845931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa80c100bd0 0x7fa80c101030 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.062+0000 7fa812b89640 1 --2- 172.21.15.37:0/984845931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa80c100270 0x7fa80c100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.062+0000 7fa812b89640 1 -- 172.21.15.37:0/984845931 >> 172.21.15.37:0/984845931 conn(0x7fa80c0fbe90 msgr2=0x7fa80c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.062+0000 7fa812b89640 1 -- 172.21.15.37:0/984845931 shutdown_connections 2024-06-10T01:08:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.062+0000 7fa812b89640 1 -- 172.21.15.37:0/984845931 wait complete. 2024-06-10T01:08:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.063+0000 7fa812b89640 1 Processor -- start 2024-06-10T01:08:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.063+0000 7fa812b89640 1 -- start start 2024-06-10T01:08:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.063+0000 7fa812b89640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa80c100270 0x7fa80c199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.063+0000 7fa812b89640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa80c100bd0 0x7fa80c19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.063+0000 7fa812b89640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa80c101570 0x7fa80c1a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.063+0000 7fa812b89640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c10f870 con 0x7fa80c100bd0 2024-06-10T01:08:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.063+0000 7fa812b89640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c10f6f0 con 0x7fa80c100270 2024-06-10T01:08:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.063+0000 7fa812b89640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c10f9f0 con 0x7fa80c101570 2024-06-10T01:08:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.063+0000 7fa80bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa80c100bd0 0x7fa80c19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.063+0000 7fa810925640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa80c100270 0x7fa80c199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.063+0000 7fa80bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa80c100bd0 0x7fa80c19a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39868/0 (socket says 172.21.15.37:39868) 2024-06-10T01:08:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.063+0000 7fa80bfff640 1 -- 172.21.15.37:0/1586789630 learned_addr learned my addr 172.21.15.37:0/1586789630 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.063+0000 7fa811126640 1 --2- 172.21.15.37:0/1586789630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa80c101570 0x7fa80c1a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.064+0000 7fa80bfff640 1 -- 172.21.15.37:0/1586789630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa80c101570 msgr2=0x7fa80c1a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.064+0000 7fa80bfff640 1 --2- 172.21.15.37:0/1586789630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa80c101570 0x7fa80c1a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.064+0000 7fa80bfff640 1 -- 172.21.15.37:0/1586789630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa80c100270 msgr2=0x7fa80c199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.064+0000 7fa80bfff640 1 --2- 172.21.15.37:0/1586789630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa80c100270 0x7fa80c199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.064+0000 7fa80bfff640 1 -- 172.21.15.37:0/1586789630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80c1a1a10 con 0x7fa80c100bd0 2024-06-10T01:08:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.064+0000 7fa810925640 1 --2- 172.21.15.37:0/1586789630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa80c100270 0x7fa80c199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.064+0000 7fa80bfff640 1 --2- 172.21.15.37:0/1586789630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa80c100bd0 0x7fa80c19a330 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8009230 tx=0x7fa7f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.064+0000 7fa809ffb640 1 -- 172.21.15.37:0/1586789630 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f803f030 con 0x7fa80c100bd0 2024-06-10T01:08:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.064+0000 7fa809ffb640 1 -- 172.21.15.37:0/1586789630 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7f8003ce0 con 0x7fa80c100bd0 2024-06-10T01:08:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.064+0000 7fa812b89640 1 -- 172.21.15.37:0/1586789630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa80c1a1ca0 con 0x7fa80c100bd0 2024-06-10T01:08:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.064+0000 7fa809ffb640 1 -- 172.21.15.37:0/1586789630 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8035070 con 0x7fa80c100bd0 2024-06-10T01:08:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.064+0000 7fa812b89640 1 -- 172.21.15.37:0/1586789630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa80c1a2190 con 0x7fa80c100bd0 2024-06-10T01:08:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.066+0000 7fa809ffb640 1 -- 172.21.15.37:0/1586789630 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa7f8042030 con 0x7fa80c100bd0 2024-06-10T01:08:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.067+0000 7fa815972740 1 Processor -- start 2024-06-10T01:08:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.067+0000 7fa815972740 1 -- start start 2024-06-10T01:08:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.067+0000 7fa809ffb640 1 --2- 172.21.15.37:0/1586789630 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa7fc0a2c30 0x7fa7fc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.067+0000 7fa815972740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625d76f3010 0x5625d76f3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.068+0000 7fa815972740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625d76f3a00 0x5625d76f8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.068+0000 7fa815972740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625d76f8b70 0x5625d76fb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.068+0000 7fa815972740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5625d76564f0 con 0x5625d76f3010 2024-06-10T01:08:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.068+0000 7fa815972740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5625d75f4870 con 0x5625d76f8b70 2024-06-10T01:08:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.068+0000 7fa815972740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5625d76524b0 con 0x5625d76f3a00 2024-06-10T01:08:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.068+0000 7fa811126640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625d76f3010 0x5625d76f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.068+0000 7fa810925640 1 --2- 172.21.15.37:0/1586789630 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa7fc0a2c30 0x7fa7fc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.068+0000 7fa811126640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625d76f3010 0x5625d76f3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39872/0 (socket says 172.21.15.37:39872) 2024-06-10T01:08:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.068+0000 7fa811126640 1 -- 172.21.15.37:0/686369548 learned_addr learned my addr 172.21.15.37:0/686369548 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.068+0000 7fa809ffb640 1 -- 172.21.15.37:0/1586789630 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fa7f80ef900 con 0x7fa80c100bd0 2024-06-10T01:08:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.068+0000 7fa80bfff640 1 --2- 172.21.15.37:0/686369548 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625d76f3a00 0x5625d76f8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.068+0000 7fa811126640 1 --2- 172.21.15.37:0/686369548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625d76f8b70 0x5625d76fb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.068+0000 7fa811126640 1 -- 172.21.15.37:0/686369548 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625d76f3a00 msgr2=0x5625d76f8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.068+0000 7fa811126640 1 --2- 172.21.15.37:0/686369548 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625d76f3a00 0x5625d76f8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.068+0000 7fa811126640 1 -- 172.21.15.37:0/686369548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625d76f8b70 msgr2=0x5625d76fb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.068+0000 7fa811126640 1 --2- 172.21.15.37:0/686369548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625d76f8b70 0x5625d76fb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.068+0000 7fa811126640 1 -- 172.21.15.37:0/686369548 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625d76109b0 con 0x5625d76f3010 2024-06-10T01:08:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.068+0000 7fa811126640 1 --2- 172.21.15.37:0/686369548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625d76f3010 0x5625d76f3430 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fa804013960 tx=0x7fa804009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a02fbe68bc5270e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.068+0000 7fa810925640 1 --2- 172.21.15.37:0/1586789630 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa7fc0a2c30 0x7fa7fc0a50f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa800009fc0 tx=0x7fa800005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.068+0000 7fa7f6ffd640 1 -- 172.21.15.37:0/686369548 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa804004b20 con 0x5625d76f3010 2024-06-10T01:08:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.068+0000 7fa7f6ffd640 1 -- 172.21.15.37:0/686369548 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa804003e50 con 0x5625d76f3010 2024-06-10T01:08:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.068+0000 7fa7f6ffd640 1 -- 172.21.15.37:0/686369548 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8040042b0 con 0x5625d76f3010 2024-06-10T01:08:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.069+0000 7fa815972740 1 -- 172.21.15.37:0/686369548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625d76f3010 msgr2=0x5625d76f3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.069+0000 7fa815972740 1 --2- 172.21.15.37:0/686369548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625d76f3010 0x5625d76f3430 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fa804013960 tx=0x7fa804009be0 comp rx=0 tx=0).stop 2024-06-10T01:08:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.069+0000 7fa815972740 1 -- 172.21.15.37:0/686369548 shutdown_connections 2024-06-10T01:08:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.069+0000 7fa815972740 1 --2- 172.21.15.37:0/686369548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625d76f8b70 0x5625d76fb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.069+0000 7fa815972740 1 --2- 172.21.15.37:0/686369548 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625d76f3a00 0x5625d76f8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.069+0000 7fa815972740 1 --2- 172.21.15.37:0/686369548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625d76f3010 0x5625d76f3430 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.069+0000 7fa815972740 1 -- 172.21.15.37:0/686369548 >> 172.21.15.37:0/686369548 conn(0x5625d76ee680 msgr2=0x5625d76eea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.069+0000 7fa815972740 1 -- 172.21.15.37:0/686369548 shutdown_connections 2024-06-10T01:08:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.069+0000 7fa815972740 1 -- 172.21.15.37:0/686369548 wait complete. 2024-06-10T01:08:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.070+0000 7fa815972740 1 -- start start 2024-06-10T01:08:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.070+0000 7fa815972740 1 Processor -- start 2024-06-10T01:08:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.070+0000 7fa815972740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625d76f3010 0x5625d76fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.070+0000 7fa815972740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625d76f3a00 0x5625d76fd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.070+0000 7fa815972740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625d76f8b70 0x5625d7704740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.070+0000 7fa815972740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5625d76fbac0 con 0x5625d76f3010 2024-06-10T01:08:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.070+0000 7fa815972740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5625d75dfae0 con 0x5625d76f8b70 2024-06-10T01:08:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.070+0000 7fa815972740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5625d76fbc40 con 0x5625d76f3a00 2024-06-10T01:08:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.070+0000 7fa810925640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625d76f3010 0x5625d76fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.070+0000 7fa815972740 10 client.0 _ll_register_callbacks cb 0x5625d76189b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.070+0000 7fa810925640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625d76f3010 0x5625d76fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39888/0 (socket says 172.21.15.37:39888) 2024-06-10T01:08:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.070+0000 7fa810925640 1 -- 172.21.15.37:0/2019258948 learned_addr learned my addr 172.21.15.37:0/2019258948 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.070+0000 7fa811126640 1 --2- 172.21.15.37:0/2019258948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625d76f8b70 0x5625d7704740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.070+0000 7fa80bfff640 1 --2- 172.21.15.37:0/2019258948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625d76f3a00 0x5625d76fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.071+0000 7fa810925640 1 -- 172.21.15.37:0/2019258948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625d76f3a00 msgr2=0x5625d76fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.071+0000 7fa810925640 1 --2- 172.21.15.37:0/2019258948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625d76f3a00 0x5625d76fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.071+0000 7fa810925640 1 -- 172.21.15.37:0/2019258948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625d76f8b70 msgr2=0x5625d7704740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.071+0000 7fa810925640 1 --2- 172.21.15.37:0/2019258948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625d76f8b70 0x5625d7704740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.071+0000 7fa810925640 1 -- 172.21.15.37:0/2019258948 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625d736a210 con 0x5625d76f3010 2024-06-10T01:08:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.071+0000 7fa811126640 1 --2- 172.21.15.37:0/2019258948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625d76f8b70 0x5625d7704740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.071+0000 7fa810925640 1 --2- 172.21.15.37:0/2019258948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625d76f3010 0x5625d76fd1c0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fa800009e80 tx=0x7fa800004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414dabd41f6270dd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.071+0000 7fa7ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:08:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.071+0000 7fa7ebfff640 1 -- 172.21.15.37:0/2019258948 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80000f7c0 con 0x5625d76f3010 2024-06-10T01:08:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.071+0000 7fa7ebfff640 1 -- 172.21.15.37:0/2019258948 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fa80000f960 con 0x5625d76f3010 2024-06-10T01:08:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.071+0000 7fa815972740 1 -- 172.21.15.37:0/2019258948 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa804003e50 con 0x5625d76f3010 2024-06-10T01:08:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.071+0000 7fa7ebfff640 1 -- 172.21.15.37:0/2019258948 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa80000fb60 con 0x5625d76f3010 2024-06-10T01:08:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.071+0000 7fa815972740 10 client.17367 fetch_fsmap learned FSMap version 31 2024-06-10T01:08:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.071+0000 7fa815972740 1 -- 172.21.15.37:0/2019258948 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5625d7436620 con 0x5625d76f3010 2024-06-10T01:08:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.071+0000 7fa7ebfff640 1 -- 172.21.15.37:0/2019258948 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa804003e50 con 0x5625d76f3010 2024-06-10T01:08:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.071+0000 7fa7ebfff640 1 -- 172.21.15.37:0/2019258948 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fa80001a070 con 0x5625d76f3010 2024-06-10T01:08:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.072+0000 7fa815972740 10 client.17367 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:08:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.072+0000 7fa815972740 10 client.17367 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.072+0000 7fa815972740 20 client.17367 populate_metadata read hostname 'smithi037' 2024-06-10T01:08:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.072+0000 7fa815972740 1 --2- 172.21.15.37:0/2019258948 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5625d7709140 0x5625d770b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:07.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.072+0000 7fa815972740 4 client.17367 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:08:07.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.072+0000 7fa815972740 1 -- 172.21.15.37:0/2019258948 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625d7669f10 con 0x5625d7709140 2024-06-10T01:08:07.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.072+0000 7fa80bfff640 1 --2- 172.21.15.37:0/2019258948 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5625d7709140 0x5625d770b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:07.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.072+0000 7fa80bfff640 1 --2- 172.21.15.37:0/2019258948 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5625d7709140 0x5625d770b5b0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f829788174aaea9f server_cookie=7d1577aaa472e0fb in_seq=0 out_seq=0 2024-06-10T01:08:07.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.072+0000 7fa7ebfff640 10 client.17367 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:07.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.078+0000 7fa7ebfff640 1 -- 172.21.15.37:0/2019258948 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5625d7669f10 con 0x5625d7709140 2024-06-10T01:08:07.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.078+0000 7fa7ebfff640 10 client.17367 handle_command_reply: tid=0 2024-06-10T01:08:07.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.078+0000 7fa815972740 1 client.17367 shutdown 2024-06-10T01:08:07.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.078+0000 7fa815972740 1 -- 172.21.15.37:0/2019258948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625d76f3010 msgr2=0x5625d76fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:07.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.078+0000 7fa815972740 1 --2- 172.21.15.37:0/2019258948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625d76f3010 0x5625d76fd1c0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fa800009e80 tx=0x7fa800004730 comp rx=0 tx=0).stop 2024-06-10T01:08:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.078+0000 7fa815972740 1 -- 172.21.15.37:0/2019258948 shutdown_connections 2024-06-10T01:08:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.078+0000 7fa815972740 1 --2- 172.21.15.37:0/2019258948 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5625d7709140 0x5625d770b5b0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.078+0000 7fa815972740 1 --2- 172.21.15.37:0/2019258948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625d76f8b70 0x5625d7704740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.078+0000 7fa815972740 1 --2- 172.21.15.37:0/2019258948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625d76f3a00 0x5625d76fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.078+0000 7fa7ebfff640 0 client.17367 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.078+0000 7fa815972740 1 --2- 172.21.15.37:0/2019258948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625d76f3010 0x5625d76fd1c0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.078+0000 7fa815972740 1 -- 172.21.15.37:0/2019258948 >> 172.21.15.37:0/2019258948 conn(0x5625d76ee680 msgr2=0x5625d76ed7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.079+0000 7fa815972740 1 -- 172.21.15.37:0/2019258948 shutdown_connections 2024-06-10T01:08:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.079+0000 7fa815972740 1 -- 172.21.15.37:0/2019258948 wait complete. 2024-06-10T01:08:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.079+0000 7fa815972740 20 client.17367 trim_cache size 0 max 16384 2024-06-10T01:08:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.094+0000 7fa815972740 1 Processor -- start 2024-06-10T01:08:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.094+0000 7fa815972740 1 -- start start 2024-06-10T01:08:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.095+0000 7fa815972740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625d76f3010 0x5625d76fd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.095+0000 7fa815972740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625d76f3a00 0x5625d7704700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.095+0000 7fa815972740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625d76f8b70 0x5625d76ecf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.095+0000 7fa815972740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c1af0d0 con 0x5625d76f8b70 2024-06-10T01:08:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.095+0000 7fa815972740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c1ac9a0 con 0x5625d76f3a00 2024-06-10T01:08:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.095+0000 7fa815972740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c1aa7f0 con 0x5625d76f3010 2024-06-10T01:08:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.095+0000 7fa811126640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625d76f8b70 0x5625d76ecf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.095+0000 7fa811126640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625d76f8b70 0x5625d76ecf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39900/0 (socket says 172.21.15.37:39900) 2024-06-10T01:08:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.095+0000 7fa811126640 1 -- 172.21.15.37:0/3622988098 learned_addr learned my addr 172.21.15.37:0/3622988098 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.095+0000 7fa810925640 1 --2- 172.21.15.37:0/3622988098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625d76f3010 0x5625d76fd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.095+0000 7fa80bfff640 1 --2- 172.21.15.37:0/3622988098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625d76f3a00 0x5625d7704700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.095+0000 7fa811126640 1 -- 172.21.15.37:0/3622988098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625d76f3010 msgr2=0x5625d76fd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.095+0000 7fa811126640 1 --2- 172.21.15.37:0/3622988098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625d76f3010 0x5625d76fd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.095+0000 7fa811126640 1 -- 172.21.15.37:0/3622988098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625d76f3a00 msgr2=0x5625d7704700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.095+0000 7fa811126640 1 --2- 172.21.15.37:0/3622988098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625d76f3a00 0x5625d7704700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.095+0000 7fa811126640 1 -- 172.21.15.37:0/3622988098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625d77079f0 con 0x5625d76f8b70 2024-06-10T01:08:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.095+0000 7fa811126640 1 --2- 172.21.15.37:0/3622988098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625d76f8b70 0x5625d76ecf50 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fa804009560 tx=0x7fa804013260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50de217e0b5a81f8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.096+0000 7fa7f67fc640 1 -- 172.21.15.37:0/3622988098 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80400fdd0 con 0x5625d76f8b70 2024-06-10T01:08:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.096+0000 7fa7f67fc640 1 -- 172.21.15.37:0/3622988098 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa804003e50 con 0x5625d76f8b70 2024-06-10T01:08:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.096+0000 7fa7f67fc640 1 -- 172.21.15.37:0/3622988098 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8040059c0 con 0x5625d76f8b70 2024-06-10T01:08:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.096+0000 7fa815972740 1 -- 172.21.15.37:0/3622988098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625d76f8b70 msgr2=0x5625d76ecf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.096+0000 7fa815972740 1 --2- 172.21.15.37:0/3622988098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625d76f8b70 0x5625d76ecf50 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fa804009560 tx=0x7fa804013260 comp rx=0 tx=0).stop 2024-06-10T01:08:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.096+0000 7fa815972740 1 -- 172.21.15.37:0/3622988098 shutdown_connections 2024-06-10T01:08:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.096+0000 7fa815972740 1 --2- 172.21.15.37:0/3622988098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625d76f8b70 0x5625d76ecf50 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.096+0000 7fa815972740 1 --2- 172.21.15.37:0/3622988098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625d76f3a00 0x5625d7704700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.096+0000 7fa815972740 1 --2- 172.21.15.37:0/3622988098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625d76f3010 0x5625d76fd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.096+0000 7fa815972740 1 -- 172.21.15.37:0/3622988098 >> 172.21.15.37:0/3622988098 conn(0x5625d76ee680 msgr2=0x5625d76fc680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.096+0000 7fa815972740 1 -- 172.21.15.37:0/3622988098 shutdown_connections 2024-06-10T01:08:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.096+0000 7fa815972740 1 -- 172.21.15.37:0/3622988098 wait complete. 2024-06-10T01:08:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.097+0000 7fa815972740 1 -- start start 2024-06-10T01:08:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.097+0000 7fa815972740 1 Processor -- start 2024-06-10T01:08:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.097+0000 7fa815972740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625d76f3010 0x5625d7701900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.097+0000 7fa815972740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625d76f3a00 0x5625d7701e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.097+0000 7fa815972740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625d76f8b70 0x5625d7702e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.097+0000 7fa815972740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c1aae60 con 0x5625d76f3a00 2024-06-10T01:08:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.097+0000 7fa815972740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c1ac2d0 con 0x5625d76f3010 2024-06-10T01:08:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.097+0000 7fa815972740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c1ab530 con 0x5625d76f8b70 2024-06-10T01:08:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.097+0000 7fa815972740 10 client.0 _ll_register_callbacks cb 0x7fa800009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.097+0000 7fa80bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625d76f3a00 0x5625d7701e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.097+0000 7fa810925640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625d76f3010 0x5625d7701900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.097+0000 7fa80bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625d76f3a00 0x5625d7701e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39910/0 (socket says 172.21.15.37:39910) 2024-06-10T01:08:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.097+0000 7fa80bfff640 1 -- 172.21.15.37:0/1181819730 learned_addr learned my addr 172.21.15.37:0/1181819730 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.097+0000 7fa811126640 1 --2- 172.21.15.37:0/1181819730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625d76f8b70 0x5625d7702e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.098+0000 7fa810925640 1 -- 172.21.15.37:0/1181819730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625d76f8b70 msgr2=0x5625d7702e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.098+0000 7fa810925640 1 --2- 172.21.15.37:0/1181819730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625d76f8b70 0x5625d7702e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.098+0000 7fa810925640 1 -- 172.21.15.37:0/1181819730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625d76f3a00 msgr2=0x5625d7701e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.098+0000 7fa810925640 1 --2- 172.21.15.37:0/1181819730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625d76f3a00 0x5625d7701e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.098+0000 7fa810925640 1 -- 172.21.15.37:0/1181819730 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625d7703440 con 0x5625d76f3010 2024-06-10T01:08:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.098+0000 7fa80bfff640 1 --2- 172.21.15.37:0/1181819730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625d76f3a00 0x5625d7701e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.098+0000 7fa810925640 1 --2- 172.21.15.37:0/1181819730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625d76f3010 0x5625d7701900 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa80001c5c0 tx=0x7fa800011d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=465e8404b4e1c594 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.098+0000 7fa7ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:08:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.098+0000 7fa7ebfff640 1 -- 172.21.15.37:0/1181819730 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80001c9c0 con 0x5625d76f3010 2024-06-10T01:08:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.098+0000 7fa815972740 1 -- 172.21.15.37:0/1181819730 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa804003e50 con 0x5625d76f3010 2024-06-10T01:08:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.098+0000 7fa7ebfff640 1 -- 172.21.15.37:0/1181819730 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fa800009040 con 0x5625d76f3010 2024-06-10T01:08:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.098+0000 7fa7ebfff640 1 -- 172.21.15.37:0/1181819730 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa80001cb80 con 0x5625d76f3010 2024-06-10T01:08:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.098+0000 7fa815972740 10 client.26726 fetch_fsmap learned FSMap version 31 2024-06-10T01:08:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.098+0000 7fa815972740 1 -- 172.21.15.37:0/1181819730 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5625d77047c0 con 0x5625d76f3010 2024-06-10T01:08:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.098+0000 7fa7ebfff640 1 -- 172.21.15.37:0/1181819730 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa804003e50 con 0x5625d76f3010 2024-06-10T01:08:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.099+0000 7fa7ebfff640 1 -- 172.21.15.37:0/1181819730 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fa80001a040 con 0x5625d76f3010 2024-06-10T01:08:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.099+0000 7fa815972740 10 client.26726 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:08:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.099+0000 7fa815972740 10 client.26726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.099+0000 7fa815972740 20 client.26726 populate_metadata read hostname 'smithi037' 2024-06-10T01:08:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.099+0000 7fa815972740 1 --2- 172.21.15.37:0/1181819730 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5625d77743e0 0x5625d7776800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.099+0000 7fa815972740 4 client.26726 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:08:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.099+0000 7fa815972740 1 -- 172.21.15.37:0/1181819730 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5625d7663c20 con 0x5625d77743e0 2024-06-10T01:08:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.099+0000 7fa80bfff640 1 --2- 172.21.15.37:0/1181819730 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5625d77743e0 0x5625d7776800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.099+0000 7fa80bfff640 1 --2- 172.21.15.37:0/1181819730 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5625d77743e0 0x5625d7776800 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ab3abab68c92196 server_cookie=6f251b6b1dc3682e in_seq=0 out_seq=0 2024-06-10T01:08:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:07.099+0000 7fa7ebfff640 10 client.26726 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:08.235+0000 7fa7ebfff640 1 -- 172.21.15.37:0/1181819730 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5625d7663c20 con 0x5625d77743e0 2024-06-10T01:08:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:08.235+0000 7fa7ebfff640 10 client.26726 handle_command_reply: tid=0 2024-06-10T01:08:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:08.235+0000 7fa815972740 1 client.26726 shutdown 2024-06-10T01:08:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:08.235+0000 7fa815972740 1 -- 172.21.15.37:0/1181819730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625d76f3010 msgr2=0x5625d7701900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:08.235+0000 7fa815972740 1 --2- 172.21.15.37:0/1181819730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625d76f3010 0x5625d7701900 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa80001c5c0 tx=0x7fa800011d90 comp rx=0 tx=0).stop 2024-06-10T01:08:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:08.235+0000 7fa815972740 1 -- 172.21.15.37:0/1181819730 shutdown_connections 2024-06-10T01:08:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:08.235+0000 7fa815972740 1 --2- 172.21.15.37:0/1181819730 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5625d77743e0 0x5625d7776800 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:08.236+0000 7fa815972740 1 --2- 172.21.15.37:0/1181819730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625d76f8b70 0x5625d7702e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:08.236+0000 7fa815972740 1 --2- 172.21.15.37:0/1181819730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625d76f3a00 0x5625d7701e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:08.236+0000 7fa7ebfff640 0 client.26726 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:08.236+0000 7fa815972740 1 --2- 172.21.15.37:0/1181819730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5625d76f3010 0x5625d7701900 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:08.236+0000 7fa815972740 1 -- 172.21.15.37:0/1181819730 >> 172.21.15.37:0/1181819730 conn(0x5625d76ee680 msgr2=0x5625d76ecf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:08.236+0000 7fa815972740 1 -- 172.21.15.37:0/1181819730 shutdown_connections 2024-06-10T01:08:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:08.236+0000 7fa815972740 1 -- 172.21.15.37:0/1181819730 wait complete. 2024-06-10T01:08:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:08.236+0000 7fa815972740 20 client.26726 trim_cache size 0 max 16384 2024-06-10T01:08:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:08:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:08:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:08:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:08.236+0000 7fa812b89640 1 -- 172.21.15.37:0/1586789630 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa7fc0a2c30 msgr2=0x7fa7fc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:08.236+0000 7fa812b89640 1 --2- 172.21.15.37:0/1586789630 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa7fc0a2c30 0x7fa7fc0a50f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa800009fc0 tx=0x7fa800005f90 comp rx=0 tx=0).stop 2024-06-10T01:08:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:08.236+0000 7fa812b89640 1 -- 172.21.15.37:0/1586789630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa80c100bd0 msgr2=0x7fa80c19a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:08.236+0000 7fa812b89640 1 --2- 172.21.15.37:0/1586789630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa80c100bd0 0x7fa80c19a330 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8009230 tx=0x7fa7f80042f0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:08.236+0000 7fa812b89640 1 -- 172.21.15.37:0/1586789630 shutdown_connections 2024-06-10T01:08:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:08.236+0000 7fa812b89640 1 --2- 172.21.15.37:0/1586789630 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa7fc0a2c30 0x7fa7fc0a50f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:08.236+0000 7fa812b89640 1 --2- 172.21.15.37:0/1586789630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa80c101570 0x7fa80c1a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:08.236+0000 7fa812b89640 1 --2- 172.21.15.37:0/1586789630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa80c100bd0 0x7fa80c19a330 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:08.236+0000 7fa812b89640 1 --2- 172.21.15.37:0/1586789630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa80c100270 0x7fa80c199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:08.236+0000 7fa812b89640 1 -- 172.21.15.37:0/1586789630 >> 172.21.15.37:0/1586789630 conn(0x7fa80c0fbe90 msgr2=0x7fa80c0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:08.237+0000 7fa812b89640 1 -- 172.21.15.37:0/1586789630 shutdown_connections 2024-06-10T01:08:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:08.237+0000 7fa812b89640 1 -- 172.21.15.37:0/1586789630 wait complete. 2024-06-10T01:08:08.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.414+0000 7f841521c700 1 -- 172.21.15.37:0/3176273938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f841006aa60 msgr2=0x7f841006aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:08.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.414+0000 7f841521c700 1 --2- 172.21.15.37:0/3176273938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f841006aa60 0x7f841006aee0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f8404009990 tx=0x7f8404009ca0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.414+0000 7f841521c700 1 -- 172.21.15.37:0/3176273938 shutdown_connections 2024-06-10T01:08:08.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.414+0000 7f841521c700 1 --2- 172.21.15.37:0/3176273938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f841006b420 0x7f841010f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.414+0000 7f841521c700 1 --2- 172.21.15.37:0/3176273938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8410108520 0x7f8410108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.414+0000 7f841521c700 1 --2- 172.21.15.37:0/3176273938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f841006aa60 0x7f841006aee0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.414+0000 7f841521c700 1 -- 172.21.15.37:0/3176273938 >> 172.21.15.37:0/3176273938 conn(0x7f84100783d0 msgr2=0x7f84100787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:08.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.414+0000 7f841521c700 1 -- 172.21.15.37:0/3176273938 shutdown_connections 2024-06-10T01:08:08.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.414+0000 7f841521c700 1 -- 172.21.15.37:0/3176273938 wait complete. 2024-06-10T01:08:08.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.415+0000 7f841521c700 1 Processor -- start 2024-06-10T01:08:08.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.415+0000 7f841521c700 1 -- start start 2024-06-10T01:08:08.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.415+0000 7f841521c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f841006aa60 0x7f841019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.415+0000 7f841521c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f841006b420 0x7f841019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.415+0000 7f841521c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8410108520 0x7f8410198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.415+0000 7f841521c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f84101127d0 con 0x7f8410108520 2024-06-10T01:08:08.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.415+0000 7f841521c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8410112650 con 0x7f841006b420 2024-06-10T01:08:08.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.415+0000 7f841521c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8410112950 con 0x7f841006aa60 2024-06-10T01:08:08.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.416+0000 7f840f59e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8410108520 0x7f8410198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.416+0000 7f840ed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f841006aa60 0x7f841019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.416+0000 7f840e59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f841006b420 0x7f841019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.416+0000 7f840ed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f841006aa60 0x7f841019e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:50360/0 (socket says 172.21.15.37:50360) 2024-06-10T01:08:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.416+0000 7f840ed9d700 1 -- 172.21.15.37:0/1650437957 learned_addr learned my addr 172.21.15.37:0/1650437957 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.416+0000 7f840f59e700 1 -- 172.21.15.37:0/1650437957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f841006aa60 msgr2=0x7f841019e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.416+0000 7f840f59e700 1 --2- 172.21.15.37:0/1650437957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f841006aa60 0x7f841019e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.416+0000 7f840f59e700 1 -- 172.21.15.37:0/1650437957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f841006b420 msgr2=0x7f841019edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.416+0000 7f840f59e700 1 --2- 172.21.15.37:0/1650437957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f841006b420 0x7f841019edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.416+0000 7f840f59e700 1 -- 172.21.15.37:0/1650437957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8410199190 con 0x7f8410108520 2024-06-10T01:08:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.416+0000 7f840e59c700 1 --2- 172.21.15.37:0/1650437957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f841006b420 0x7f841019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.416+0000 7f840ed9d700 1 --2- 172.21.15.37:0/1650437957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f841006aa60 0x7f841019e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.416+0000 7f840f59e700 1 --2- 172.21.15.37:0/1650437957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8410108520 0x7f8410198a30 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f840800efc0 tx=0x7f840800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.416+0000 7f83f7fff700 1 -- 172.21.15.37:0/1650437957 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84080092e0 con 0x7f8410108520 2024-06-10T01:08:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.416+0000 7f841521c700 1 -- 172.21.15.37:0/1650437957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8410199420 con 0x7f8410108520 2024-06-10T01:08:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.416+0000 7f841521c700 1 -- 172.21.15.37:0/1650437957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84101a5740 con 0x7f8410108520 2024-06-10T01:08:08.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.416+0000 7f83f7fff700 1 -- 172.21.15.37:0/1650437957 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8408004bb0 con 0x7f8410108520 2024-06-10T01:08:08.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.417+0000 7f83f7fff700 1 -- 172.21.15.37:0/1650437957 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8408004280 con 0x7f8410108520 2024-06-10T01:08:08.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.419+0000 7f83f7fff700 1 -- 172.21.15.37:0/1650437957 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f840801b030 con 0x7f8410108520 2024-06-10T01:08:08.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.420+0000 7f83f7fff700 1 --2- 172.21.15.37:0/1650437957 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f83f80a2d40 0x7f83f80a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.420+0000 7f840ed9d700 1 --2- 172.21.15.37:0/1650437957 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f83f80a2d40 0x7f83f80a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.420+0000 7f83f7fff700 1 -- 172.21.15.37:0/1650437957 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f84080cee00 con 0x7f8410108520 2024-06-10T01:08:08.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.420+0000 7f841fa9fb80 1 Processor -- start 2024-06-10T01:08:08.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.420+0000 7f841fa9fb80 1 -- start start 2024-06-10T01:08:08.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.420+0000 7f840ed9d700 1 --2- 172.21.15.37:0/1650437957 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f83f80a2d40 0x7f83f80a5200 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8410199b70 tx=0x7f840000b500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:08.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.420+0000 7f841fa9fb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f325b5d590 0x55f325b5d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.420+0000 7f841fa9fb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f325b5deb0 0x55f325b577a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.420+0000 7f841fa9fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f325b57ce0 0x55f325b58100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.420+0000 7f841fa9fb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f325a4a0f0 con 0x55f325b5d590 2024-06-10T01:08:08.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.420+0000 7f841fa9fb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f325aa89e0 con 0x55f325b5deb0 2024-06-10T01:08:08.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.420+0000 7f841fa9fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f325a105f0 con 0x55f325b57ce0 2024-06-10T01:08:08.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.421+0000 7f840ed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f325b5d590 0x55f325b5d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.421+0000 7f840ed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f325b5d590 0x55f325b5d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39940/0 (socket says 172.21.15.37:39940) 2024-06-10T01:08:08.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.421+0000 7f840ed9d700 1 -- 172.21.15.37:0/3101975880 learned_addr learned my addr 172.21.15.37:0/3101975880 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:08.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.421+0000 7f840f59e700 1 --2- 172.21.15.37:0/3101975880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f325b57ce0 0x55f325b58100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.421+0000 7f840e59c700 1 --2- 172.21.15.37:0/3101975880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f325b5deb0 0x55f325b577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.421+0000 7f840f59e700 1 -- 172.21.15.37:0/3101975880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f325b5deb0 msgr2=0x55f325b577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:08.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.421+0000 7f840f59e700 1 --2- 172.21.15.37:0/3101975880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f325b5deb0 0x55f325b577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.421+0000 7f840f59e700 1 -- 172.21.15.37:0/3101975880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f325b5d590 msgr2=0x55f325b5d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:08.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.421+0000 7f840f59e700 1 --2- 172.21.15.37:0/3101975880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f325b5d590 0x55f325b5d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.421+0000 7f840f59e700 1 -- 172.21.15.37:0/3101975880 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3259f7a60 con 0x55f325b57ce0 2024-06-10T01:08:08.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.421+0000 7f840ed9d700 1 --2- 172.21.15.37:0/3101975880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f325b5d590 0x55f325b5d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:08.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.421+0000 7f840f59e700 1 --2- 172.21.15.37:0/3101975880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f325b57ce0 0x55f325b58100 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f840800cc40 tx=0x7f84080976b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b217fd0dc193fac server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:08.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.421+0000 7f83f57fa700 1 -- 172.21.15.37:0/3101975880 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840809b300 con 0x55f325b57ce0 2024-06-10T01:08:08.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.421+0000 7f83f57fa700 1 -- 172.21.15.37:0/3101975880 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f840809f6b0 con 0x55f325b57ce0 2024-06-10T01:08:08.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.421+0000 7f83f57fa700 1 -- 172.21.15.37:0/3101975880 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84080bb0c0 con 0x55f325b57ce0 2024-06-10T01:08:08.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.422+0000 7f841fa9fb80 1 -- 172.21.15.37:0/3101975880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f325b57ce0 msgr2=0x55f325b58100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:08.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.422+0000 7f841fa9fb80 1 --2- 172.21.15.37:0/3101975880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f325b57ce0 0x55f325b58100 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f840800cc40 tx=0x7f84080976b0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.422+0000 7f841fa9fb80 1 -- 172.21.15.37:0/3101975880 shutdown_connections 2024-06-10T01:08:08.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.422+0000 7f841fa9fb80 1 --2- 172.21.15.37:0/3101975880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f325b57ce0 0x55f325b58100 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.422+0000 7f841fa9fb80 1 --2- 172.21.15.37:0/3101975880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f325b5d590 0x55f325b5d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.422+0000 7f841fa9fb80 1 --2- 172.21.15.37:0/3101975880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f325b5deb0 0x55f325b577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.422+0000 7f841fa9fb80 1 -- 172.21.15.37:0/3101975880 >> 172.21.15.37:0/3101975880 conn(0x55f325a84210 msgr2=0x55f325b52a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:08.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.422+0000 7f841fa9fb80 1 -- 172.21.15.37:0/3101975880 shutdown_connections 2024-06-10T01:08:08.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.422+0000 7f841fa9fb80 1 -- 172.21.15.37:0/3101975880 wait complete. 2024-06-10T01:08:08.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.422+0000 7f841fa9fb80 1 -- start start 2024-06-10T01:08:08.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.423+0000 7f841fa9fb80 1 Processor -- start 2024-06-10T01:08:08.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.423+0000 7f841fa9fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f325b57ce0 0x55f325b63c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.423+0000 7f841fa9fb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f325b5d590 0x55f325b60f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.423+0000 7f841fa9fb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f325b5deb0 0x55f325b61480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.423+0000 7f841fa9fb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f325a49150 con 0x55f325b5d590 2024-06-10T01:08:08.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.423+0000 7f841fa9fb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f325aa49a0 con 0x55f325b5deb0 2024-06-10T01:08:08.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.423+0000 7f841fa9fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f325b64890 con 0x55f325b57ce0 2024-06-10T01:08:08.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.423+0000 7f841fa9fb80 10 client.0 _ll_register_callbacks cb 0x55f325a3e740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:08.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.423+0000 7f840e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f325b5d590 0x55f325b60f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.423+0000 7f840ed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f325b57ce0 0x55f325b63c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.423+0000 7f840e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f325b5d590 0x55f325b60f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39944/0 (socket says 172.21.15.37:39944) 2024-06-10T01:08:08.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.423+0000 7f840e59c700 1 -- 172.21.15.37:0/341313701 learned_addr learned my addr 172.21.15.37:0/341313701 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:08.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.423+0000 7f840f59e700 1 --2- 172.21.15.37:0/341313701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f325b5deb0 0x55f325b61480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.423+0000 7f840ed9d700 1 -- 172.21.15.37:0/341313701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f325b5deb0 msgr2=0x55f325b61480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:08.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.423+0000 7f840ed9d700 1 --2- 172.21.15.37:0/341313701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f325b5deb0 0x55f325b61480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.423+0000 7f840ed9d700 1 -- 172.21.15.37:0/341313701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f325b5d590 msgr2=0x55f325b60f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:08.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.423+0000 7f840ed9d700 1 --2- 172.21.15.37:0/341313701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f325b5d590 0x55f325b60f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.423+0000 7f840ed9d700 1 -- 172.21.15.37:0/341313701 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f325a6e8e0 con 0x55f325b57ce0 2024-06-10T01:08:08.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.424+0000 7f840ed9d700 1 --2- 172.21.15.37:0/341313701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f325b57ce0 0x55f325b63c40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8400005950 tx=0x7f84000047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39cd6affd322ac97 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:08.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.424+0000 7f83f2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:08:08.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.424+0000 7f83f2ff5700 1 -- 172.21.15.37:0/341313701 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840001e6c0 con 0x55f325b57ce0 2024-06-10T01:08:08.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.424+0000 7f841fa9fb80 1 -- 172.21.15.37:0/341313701 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f840809f6b0 con 0x55f325b57ce0 2024-06-10T01:08:08.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.424+0000 7f83f2ff5700 1 -- 172.21.15.37:0/341313701 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f840001e860 con 0x55f325b57ce0 2024-06-10T01:08:08.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.424+0000 7f83f2ff5700 1 -- 172.21.15.37:0/341313701 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f840001ea60 con 0x55f325b57ce0 2024-06-10T01:08:08.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.424+0000 7f841fa9fb80 10 client.26884 fetch_fsmap learned FSMap version 31 2024-06-10T01:08:08.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.424+0000 7f841fa9fb80 1 -- 172.21.15.37:0/341313701 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f325b6f120 con 0x55f325b57ce0 2024-06-10T01:08:08.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.424+0000 7f83f2ff5700 1 -- 172.21.15.37:0/341313701 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f840809f6b0 con 0x55f325b57ce0 2024-06-10T01:08:08.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.424+0000 7f83f2ff5700 1 -- 172.21.15.37:0/341313701 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f840001f430 con 0x55f325b57ce0 2024-06-10T01:08:08.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.424+0000 7f841fa9fb80 10 client.26884 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:08:08.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.424+0000 7f841fa9fb80 10 client.26884 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:08:08.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.424+0000 7f841fa9fb80 20 client.26884 populate_metadata read hostname 'smithi037' 2024-06-10T01:08:08.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.424+0000 7f841fa9fb80 1 --2- 172.21.15.37:0/341313701 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f325b71a90 0x55f325b73f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.424+0000 7f841fa9fb80 4 client.26884 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:08:08.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.424+0000 7f841fa9fb80 1 -- 172.21.15.37:0/341313701 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f325ab5ee0 con 0x55f325b71a90 2024-06-10T01:08:08.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.425+0000 7f840e59c700 1 --2- 172.21.15.37:0/341313701 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f325b71a90 0x55f325b73f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.425+0000 7f840e59c700 1 --2- 172.21.15.37:0/341313701 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f325b71a90 0x55f325b73f10 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=348c465d788ee8ba server_cookie=a423a382aefbc0c7 in_seq=0 out_seq=0 2024-06-10T01:08:08.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.425+0000 7f83f2ff5700 10 client.26884 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:08.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.428+0000 7f83f2ff5700 1 -- 172.21.15.37:0/341313701 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f325ab5ee0 con 0x55f325b71a90 2024-06-10T01:08:08.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.428+0000 7f83f2ff5700 10 client.26884 handle_command_reply: tid=0 2024-06-10T01:08:08.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.428+0000 7f841fa9fb80 1 client.26884 shutdown 2024-06-10T01:08:08.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.428+0000 7f841fa9fb80 1 -- 172.21.15.37:0/341313701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f325b57ce0 msgr2=0x55f325b63c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:08.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.428+0000 7f841fa9fb80 1 --2- 172.21.15.37:0/341313701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f325b57ce0 0x55f325b63c40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8400005950 tx=0x7f84000047c0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.428+0000 7f841fa9fb80 1 -- 172.21.15.37:0/341313701 shutdown_connections 2024-06-10T01:08:08.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.428+0000 7f841fa9fb80 1 --2- 172.21.15.37:0/341313701 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f325b71a90 0x55f325b73f10 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.428+0000 7f841fa9fb80 1 --2- 172.21.15.37:0/341313701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f325b5deb0 0x55f325b61480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.428+0000 7f841fa9fb80 1 --2- 172.21.15.37:0/341313701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f325b57ce0 0x55f325b63c40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.428+0000 7f83f2ff5700 0 client.26884 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:08.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.428+0000 7f841fa9fb80 1 --2- 172.21.15.37:0/341313701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f325b5d590 0x55f325b60f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.428+0000 7f841fa9fb80 1 -- 172.21.15.37:0/341313701 >> 172.21.15.37:0/341313701 conn(0x55f325a84210 msgr2=0x55f325b51c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:08.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.428+0000 7f841fa9fb80 1 -- 172.21.15.37:0/341313701 shutdown_connections 2024-06-10T01:08:08.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.428+0000 7f841fa9fb80 1 -- 172.21.15.37:0/341313701 wait complete. 2024-06-10T01:08:08.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.428+0000 7f841fa9fb80 20 client.26884 trim_cache size 0 max 16384 2024-06-10T01:08:08.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.449+0000 7f841fa9fb80 1 Processor -- start 2024-06-10T01:08:08.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.449+0000 7f841fa9fb80 1 -- start start 2024-06-10T01:08:08.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.449+0000 7f841fa9fb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f325b57ce0 0x55f325b6d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.449+0000 7f841fa9fb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f325b5d590 0x55f325b614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.449+0000 7f841fa9fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f325b5deb0 0x55f325b523c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.449+0000 7f841fa9fb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc011500 con 0x55f325b5d590 2024-06-10T01:08:08.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.449+0000 7f841fa9fb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc00ed90 con 0x55f325b57ce0 2024-06-10T01:08:08.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.450+0000 7f841fa9fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc00cbe0 con 0x55f325b5deb0 2024-06-10T01:08:08.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.450+0000 7f840e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f325b5d590 0x55f325b614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.450+0000 7f840e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f325b5d590 0x55f325b614b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39948/0 (socket says 172.21.15.37:39948) 2024-06-10T01:08:08.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.450+0000 7f840f59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f325b5deb0 0x55f325b523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.450+0000 7f840e59c700 1 -- 172.21.15.37:0/409772205 learned_addr learned my addr 172.21.15.37:0/409772205 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:08.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.450+0000 7f840ed9d700 1 --2- 172.21.15.37:0/409772205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f325b57ce0 0x55f325b6d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.450+0000 7f840e59c700 1 -- 172.21.15.37:0/409772205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f325b5deb0 msgr2=0x55f325b523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:08.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.450+0000 7f840e59c700 1 --2- 172.21.15.37:0/409772205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f325b5deb0 0x55f325b523c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.450+0000 7f840e59c700 1 -- 172.21.15.37:0/409772205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f325b57ce0 msgr2=0x55f325b6d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:08.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.450+0000 7f840e59c700 1 --2- 172.21.15.37:0/409772205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f325b57ce0 0x55f325b6d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.450+0000 7f840e59c700 1 -- 172.21.15.37:0/409772205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f325b74590 con 0x55f325b5d590 2024-06-10T01:08:08.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.450+0000 7f840f59e700 1 --2- 172.21.15.37:0/409772205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f325b5deb0 0x55f325b523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:08.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.450+0000 7f840e59c700 1 --2- 172.21.15.37:0/409772205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f325b5d590 0x55f325b614b0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f8404000c00 tx=0x7f8404005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893bf298d87a81e4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:08.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.450+0000 7f83f4ff9700 1 -- 172.21.15.37:0/409772205 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840400ba80 con 0x55f325b5d590 2024-06-10T01:08:08.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.450+0000 7f83f4ff9700 1 -- 172.21.15.37:0/409772205 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f840400bc20 con 0x55f325b5d590 2024-06-10T01:08:08.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.450+0000 7f83f4ff9700 1 -- 172.21.15.37:0/409772205 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840400f6c0 con 0x55f325b5d590 2024-06-10T01:08:08.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.451+0000 7f841fa9fb80 1 -- 172.21.15.37:0/409772205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f325b5d590 msgr2=0x55f325b614b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:08.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.451+0000 7f841fa9fb80 1 --2- 172.21.15.37:0/409772205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f325b5d590 0x55f325b614b0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f8404000c00 tx=0x7f8404005fd0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.451+0000 7f841fa9fb80 1 -- 172.21.15.37:0/409772205 shutdown_connections 2024-06-10T01:08:08.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.451+0000 7f841fa9fb80 1 --2- 172.21.15.37:0/409772205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f325b5deb0 0x55f325b523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.451+0000 7f841fa9fb80 1 --2- 172.21.15.37:0/409772205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f325b57ce0 0x55f325b6d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.451+0000 7f841fa9fb80 1 --2- 172.21.15.37:0/409772205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f325b5d590 0x55f325b614b0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.451+0000 7f841fa9fb80 1 -- 172.21.15.37:0/409772205 >> 172.21.15.37:0/409772205 conn(0x55f325a84210 msgr2=0x55f325b56790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:08.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.451+0000 7f841fa9fb80 1 -- 172.21.15.37:0/409772205 shutdown_connections 2024-06-10T01:08:08.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.451+0000 7f841fa9fb80 1 -- 172.21.15.37:0/409772205 wait complete. 2024-06-10T01:08:08.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.452+0000 7f841fa9fb80 1 -- start start 2024-06-10T01:08:08.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.452+0000 7f841fa9fb80 1 Processor -- start 2024-06-10T01:08:08.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.452+0000 7f841fa9fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f325b57ce0 0x55f325b69b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.452+0000 7f841fa9fb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f325b5d590 0x55f325b6a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.452+0000 7f841fa9fb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f325b5deb0 0x55f325b66b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.452+0000 7f841fa9fb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc00d250 con 0x55f325b5deb0 2024-06-10T01:08:08.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.452+0000 7f841fa9fb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc00e6c0 con 0x55f325b5d590 2024-06-10T01:08:08.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.452+0000 7f841fa9fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc00d920 con 0x55f325b57ce0 2024-06-10T01:08:08.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.452+0000 7f841fa9fb80 10 client.0 _ll_register_callbacks cb 0x55f325b5d0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:08.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.452+0000 7f840ed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f325b57ce0 0x55f325b69b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.452+0000 7f840f59e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f325b5deb0 0x55f325b66b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.452+0000 7f840ed9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f325b57ce0 0x55f325b69b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:50382/0 (socket says 172.21.15.37:50382) 2024-06-10T01:08:08.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.452+0000 7f840ed9d700 1 -- 172.21.15.37:0/2506976174 learned_addr learned my addr 172.21.15.37:0/2506976174 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:08.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.452+0000 7f840e59c700 1 --2- 172.21.15.37:0/2506976174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f325b5d590 0x55f325b6a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.452+0000 7f840f59e700 1 -- 172.21.15.37:0/2506976174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f325b57ce0 msgr2=0x55f325b69b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:08.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.452+0000 7f840f59e700 1 --2- 172.21.15.37:0/2506976174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f325b57ce0 0x55f325b69b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.452+0000 7f840f59e700 1 -- 172.21.15.37:0/2506976174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f325b5d590 msgr2=0x55f325b6a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:08.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.452+0000 7f840f59e700 1 --2- 172.21.15.37:0/2506976174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f325b5d590 0x55f325b6a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:08.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.452+0000 7f840f59e700 1 -- 172.21.15.37:0/2506976174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f325b6a790 con 0x55f325b5deb0 2024-06-10T01:08:08.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.452+0000 7f840e59c700 1 --2- 172.21.15.37:0/2506976174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f325b5d590 0x55f325b6a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:08.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.452+0000 7f840ed9d700 1 --2- 172.21.15.37:0/2506976174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f325b57ce0 0x55f325b69b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:08.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.453+0000 7f840f59e700 1 --2- 172.21.15.37:0/2506976174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f325b5deb0 0x55f325b66b20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f84080a40a0 tx=0x7f840800c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88478f7c031e452d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:08.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.453+0000 7f83f2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:08:08.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.453+0000 7f83f2ff5700 1 -- 172.21.15.37:0/2506976174 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840807b060 con 0x55f325b5deb0 2024-06-10T01:08:08.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.453+0000 7f83f2ff5700 1 -- 172.21.15.37:0/2506976174 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f840800c940 con 0x55f325b5deb0 2024-06-10T01:08:08.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.453+0000 7f841fa9fb80 1 -- 172.21.15.37:0/2506976174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f840400bc20 con 0x55f325b5deb0 2024-06-10T01:08:08.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.453+0000 7f83f2ff5700 1 -- 172.21.15.37:0/2506976174 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f84080b8070 con 0x55f325b5deb0 2024-06-10T01:08:08.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.453+0000 7f841fa9fb80 10 client.17391 fetch_fsmap learned FSMap version 31 2024-06-10T01:08:08.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.453+0000 7f841fa9fb80 1 -- 172.21.15.37:0/2506976174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f325be11e0 con 0x55f325b5deb0 2024-06-10T01:08:08.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.453+0000 7f83f2ff5700 1 -- 172.21.15.37:0/2506976174 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f840400bc20 con 0x55f325b5deb0 2024-06-10T01:08:08.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.453+0000 7f83f2ff5700 1 -- 172.21.15.37:0/2506976174 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f840809a750 con 0x55f325b5deb0 2024-06-10T01:08:08.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.453+0000 7f841fa9fb80 10 client.17391 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:08:08.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.453+0000 7f841fa9fb80 10 client.17391 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:08:08.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.453+0000 7f841fa9fb80 20 client.17391 populate_metadata read hostname 'smithi037' 2024-06-10T01:08:08.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.453+0000 7f841fa9fb80 1 --2- 172.21.15.37:0/2506976174 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f325be14b0 0x55f325be38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:08.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.454+0000 7f841fa9fb80 4 client.17391 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.8, "await_for": 45.0}] 2024-06-10T01:08:08.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.454+0000 7f841fa9fb80 1 -- 172.21.15.37:0/2506976174 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.8, "await_for": 45.0}) -- 0x55f325ab0460 con 0x55f325be14b0 2024-06-10T01:08:08.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.454+0000 7f840ed9d700 1 --2- 172.21.15.37:0/2506976174 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f325be14b0 0x55f325be38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:08.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.454+0000 7f83f2ff5700 10 client.17391 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:08.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:08.454+0000 7f840ed9d700 1 --2- 172.21.15.37:0/2506976174 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f325be14b0 0x55f325be38a0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42c0f7c30a91b270 server_cookie=253fbe4162c8eb4b in_seq=0 out_seq=0 2024-06-10T01:08:08.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:08 smithi167 ceph-mon[36917]: pgmap v774: 129 pgs: 129 active+clean; 10 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 480 KiB/s wr, 330 op/s 2024-06-10T01:08:08.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:08 smithi094 ceph-mon[36708]: pgmap v774: 129 pgs: 129 active+clean; 10 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 480 KiB/s wr, 330 op/s 2024-06-10T01:08:08.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:08 smithi037 ceph-mon[32192]: pgmap v774: 129 pgs: 129 active+clean; 10 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 480 KiB/s wr, 330 op/s 2024-06-10T01:08:09.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.132+0000 7f6b98791700 1 -- 172.21.15.37:0/111668756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b901019d0 msgr2=0x7f6b90101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:09.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.132+0000 7f6b98791700 1 --2- 172.21.15.37:0/111668756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b901019d0 0x7f6b90101db0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0099c0 tx=0x7f6b8c009cd0 comp rx=0 tx=0).stop 2024-06-10T01:08:09.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.132+0000 7f6b98791700 1 -- 172.21.15.37:0/111668756 shutdown_connections 2024-06-10T01:08:09.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.132+0000 7f6b98791700 1 --2- 172.21.15.37:0/111668756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b90106720 0x7f6b9010e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:09.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.132+0000 7f6b98791700 1 --2- 172.21.15.37:0/111668756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b901019d0 0x7f6b90101db0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:09.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.132+0000 7f6b98791700 1 --2- 172.21.15.37:0/111668756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b90102380 0x7f6b90105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:09.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.132+0000 7f6b98791700 1 -- 172.21.15.37:0/111668756 >> 172.21.15.37:0/111668756 conn(0x7f6b90078340 msgr2=0x7f6b90078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:09.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.132+0000 7f6b98791700 1 -- 172.21.15.37:0/111668756 shutdown_connections 2024-06-10T01:08:09.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.132+0000 7f6b98791700 1 -- 172.21.15.37:0/111668756 wait complete. 2024-06-10T01:08:09.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.133+0000 7f6b98791700 1 Processor -- start 2024-06-10T01:08:09.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.133+0000 7f6b98791700 1 -- start start 2024-06-10T01:08:09.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.133+0000 7f6b98791700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b901019d0 0x7f6b9019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:09.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.133+0000 7f6b98791700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b90102380 0x7f6b9019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:09.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.133+0000 7f6b98791700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b90106720 0x7f6b9019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:09.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.133+0000 7f6b98791700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6b901116d0 con 0x7f6b90106720 2024-06-10T01:08:09.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.133+0000 7f6b98791700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6b90111550 con 0x7f6b901019d0 2024-06-10T01:08:09.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.133+0000 7f6b98791700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6b90111850 con 0x7f6b90102380 2024-06-10T01:08:09.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.133+0000 7f6b95d2c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b90102380 0x7f6b9019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:09.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.133+0000 7f6b96d2e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b90106720 0x7f6b9019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:09.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.133+0000 7f6b96d2e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b90106720 0x7f6b9019f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39986/0 (socket says 172.21.15.37:39986) 2024-06-10T01:08:09.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.133+0000 7f6b95d2c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b90102380 0x7f6b9019b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:50412/0 (socket says 172.21.15.37:50412) 2024-06-10T01:08:09.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.133+0000 7f6b96d2e700 1 -- 172.21.15.37:0/3962725608 learned_addr learned my addr 172.21.15.37:0/3962725608 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:09.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.133+0000 7f6b9652d700 1 --2- 172.21.15.37:0/3962725608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b901019d0 0x7f6b9019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:09.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.133+0000 7f6b96d2e700 1 -- 172.21.15.37:0/3962725608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b90102380 msgr2=0x7f6b9019b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:09.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.133+0000 7f6b96d2e700 1 --2- 172.21.15.37:0/3962725608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b90102380 0x7f6b9019b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:09.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.133+0000 7f6b96d2e700 1 -- 172.21.15.37:0/3962725608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b901019d0 msgr2=0x7f6b9019ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:09.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.133+0000 7f6b96d2e700 1 --2- 172.21.15.37:0/3962725608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b901019d0 0x7f6b9019ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:09.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.133+0000 7f6b96d2e700 1 -- 172.21.15.37:0/3962725608 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9019fe30 con 0x7f6b90106720 2024-06-10T01:08:09.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.133+0000 7f6b95d2c700 1 --2- 172.21.15.37:0/3962725608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b90102380 0x7f6b9019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:09.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.134+0000 7f6b96d2e700 1 --2- 172.21.15.37:0/3962725608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b90106720 0x7f6b9019f6f0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f6b88007d70 tx=0x7f6b8800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:09.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.134+0000 7f6b877fe700 1 -- 172.21.15.37:0/3962725608 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b88016020 con 0x7f6b90106720 2024-06-10T01:08:09.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.134+0000 7f6b98791700 1 -- 172.21.15.37:0/3962725608 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b901a0120 con 0x7f6b90106720 2024-06-10T01:08:09.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.134+0000 7f6b877fe700 1 -- 172.21.15.37:0/3962725608 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b8800ac90 con 0x7f6b90106720 2024-06-10T01:08:09.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.134+0000 7f6b877fe700 1 -- 172.21.15.37:0/3962725608 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b880115c0 con 0x7f6b90106720 2024-06-10T01:08:09.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.134+0000 7f6b98791700 1 -- 172.21.15.37:0/3962725608 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b901a79a0 con 0x7f6b90106720 2024-06-10T01:08:09.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.135+0000 7f6b98791700 1 -- 172.21.15.37:0/3962725608 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b90102ad0 con 0x7f6b90106720 2024-06-10T01:08:09.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.137+0000 7f6b877fe700 1 -- 172.21.15.37:0/3962725608 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6b88019020 con 0x7f6b90106720 2024-06-10T01:08:09.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.137+0000 7f6b877fe700 1 --2- 172.21.15.37:0/3962725608 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f6b7c0a2c20 0x7f6b7c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:09.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.138+0000 7f6b9652d700 1 --2- 172.21.15.37:0/3962725608 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f6b7c0a2c20 0x7f6b7c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:09.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.138+0000 7f6b877fe700 1 -- 172.21.15.37:0/3962725608 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f6b880cf750 con 0x7f6b90106720 2024-06-10T01:08:09.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.138+0000 7f6b9652d700 1 --2- 172.21.15.37:0/3962725608 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f6b7c0a2c20 0x7f6b7c0a50e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c000c00 tx=0x7f6b8c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:09.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.141+0000 7f6b877fe700 1 -- 172.21.15.37:0/3962725608 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6b88020020 con 0x7f6b90106720 2024-06-10T01:08:09.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.406+0000 7f6b98791700 1 -- 172.21.15.37:0/3962725608 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b90069180 con 0x7f6b90106720 2024-06-10T01:08:09.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.407+0000 7f6b877fe700 1 -- 172.21.15.37:0/3962725608 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11868 (secure 0 0 0) 0x7f6b90069180 con 0x7f6b90106720 2024-06-10T01:08:09.404 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:08:09.404 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":26554,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2107277223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2107277223},{"type":"v1","addr":"172.21.15.167:6835","nonce":2107277223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T01:06:09.604504+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24472,24451,24458]},"id":1}]} 2024-06-10T01:08:09.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.409+0000 7f6b98791700 1 -- 172.21.15.37:0/3962725608 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f6b7c0a2c20 msgr2=0x7f6b7c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:09.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.409+0000 7f6b98791700 1 --2- 172.21.15.37:0/3962725608 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f6b7c0a2c20 0x7f6b7c0a50e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c000c00 tx=0x7f6b8c005fb0 comp rx=0 tx=0).stop 2024-06-10T01:08:09.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.409+0000 7f6b98791700 1 -- 172.21.15.37:0/3962725608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b90106720 msgr2=0x7f6b9019f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:09.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.409+0000 7f6b98791700 1 --2- 172.21.15.37:0/3962725608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b90106720 0x7f6b9019f6f0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f6b88007d70 tx=0x7f6b8800a340 comp rx=0 tx=0).stop 2024-06-10T01:08:09.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.409+0000 7f6b98791700 1 -- 172.21.15.37:0/3962725608 shutdown_connections 2024-06-10T01:08:09.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.409+0000 7f6b98791700 1 --2- 172.21.15.37:0/3962725608 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f6b7c0a2c20 0x7f6b7c0a50e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:09.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.409+0000 7f6b98791700 1 --2- 172.21.15.37:0/3962725608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b90106720 0x7f6b9019f6f0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:09.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.409+0000 7f6b98791700 1 --2- 172.21.15.37:0/3962725608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b901019d0 0x7f6b9019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:09.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.409+0000 7f6b98791700 1 --2- 172.21.15.37:0/3962725608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b90102380 0x7f6b9019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:09.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.409+0000 7f6b98791700 1 -- 172.21.15.37:0/3962725608 >> 172.21.15.37:0/3962725608 conn(0x7f6b90078340 msgr2=0x7f6b9006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:09.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.409+0000 7f6b98791700 1 -- 172.21.15.37:0/3962725608 shutdown_connections 2024-06-10T01:08:09.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:09.409+0000 7f6b98791700 1 -- 172.21.15.37:0/3962725608 wait complete. 2024-06-10T01:08:09.407 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 31 2024-06-10T01:08:10.105 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:08:10.371 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:08:10.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:10 smithi037 ceph-mon[32192]: pgmap v775: 129 pgs: 129 active+clean; 9.0 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 750 KiB/s wr, 545 op/s 2024-06-10T01:08:10.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:10 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3962725608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:08:10.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:10 smithi167 ceph-mon[36917]: pgmap v775: 129 pgs: 129 active+clean; 9.0 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 750 KiB/s wr, 545 op/s 2024-06-10T01:08:10.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:10 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3962725608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:08:10.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:10 smithi094 ceph-mon[36708]: pgmap v775: 129 pgs: 129 active+clean; 9.0 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 750 KiB/s wr, 545 op/s 2024-06-10T01:08:10.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:10 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3962725608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:08:12.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.110+0000 7f2ee2cbb700 1 -- 172.21.15.37:0/1687541564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2edc06aa00 msgr2=0x7f2edc06ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:12.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.110+0000 7f2ee2cbb700 1 --2- 172.21.15.37:0/1687541564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2edc06aa00 0x7f2edc06ae80 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009990 tx=0x7f2ec4009ca0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.110+0000 7f2ee2cbb700 1 -- 172.21.15.37:0/1687541564 shutdown_connections 2024-06-10T01:08:12.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.110+0000 7f2ee2cbb700 1 --2- 172.21.15.37:0/1687541564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2edc06b3c0 0x7f2edc10f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.110+0000 7f2ee2cbb700 1 --2- 172.21.15.37:0/1687541564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2edc1084f0 0x7f2edc1088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.110+0000 7f2ee2cbb700 1 --2- 172.21.15.37:0/1687541564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2edc06aa00 0x7f2edc06ae80 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.110+0000 7f2ee2cbb700 1 -- 172.21.15.37:0/1687541564 >> 172.21.15.37:0/1687541564 conn(0x7f2edc078300 msgr2=0x7f2edc078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:12.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.110+0000 7f2ee2cbb700 1 -- 172.21.15.37:0/1687541564 shutdown_connections 2024-06-10T01:08:12.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.110+0000 7f2ee2cbb700 1 -- 172.21.15.37:0/1687541564 wait complete. 2024-06-10T01:08:12.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.111+0000 7f2ee2cbb700 1 Processor -- start 2024-06-10T01:08:12.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.111+0000 7f2ee2cbb700 1 -- start start 2024-06-10T01:08:12.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.111+0000 7f2ee2cbb700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2edc06aa00 0x7f2edc101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:12.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.111+0000 7f2ee2cbb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2edc06b3c0 0x7f2edc101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:12.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.111+0000 7f2ee2cbb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2edc1084f0 0x7f2edc101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:12.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.111+0000 7f2ee2cbb700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc112750 con 0x7f2edc1084f0 2024-06-10T01:08:12.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.111+0000 7f2ee2cbb700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc1125d0 con 0x7f2edc06b3c0 2024-06-10T01:08:12.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.111+0000 7f2ee2cbb700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc1128d0 con 0x7f2edc06aa00 2024-06-10T01:08:12.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.111+0000 7f2ee1258700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2edc1084f0 0x7f2edc101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:12.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.111+0000 7f2ee0a57700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2edc06aa00 0x7f2edc101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:12.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.111+0000 7f2ee1258700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2edc1084f0 0x7f2edc101ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48972/0 (socket says 172.21.15.37:48972) 2024-06-10T01:08:12.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.111+0000 7f2ee1258700 1 -- 172.21.15.37:0/1920909168 learned_addr learned my addr 172.21.15.37:0/1920909168 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:12.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.111+0000 7f2edbfff700 1 --2- 172.21.15.37:0/1920909168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2edc06b3c0 0x7f2edc101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:12.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.112+0000 7f2ee0a57700 1 -- 172.21.15.37:0/1920909168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2edc06b3c0 msgr2=0x7f2edc101990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:12.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.112+0000 7f2ee0a57700 1 --2- 172.21.15.37:0/1920909168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2edc06b3c0 0x7f2edc101990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.112+0000 7f2ee0a57700 1 -- 172.21.15.37:0/1920909168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2edc1084f0 msgr2=0x7f2edc101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:12.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.112+0000 7f2ee0a57700 1 --2- 172.21.15.37:0/1920909168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2edc1084f0 0x7f2edc101ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.112+0000 7f2ee0a57700 1 -- 172.21.15.37:0/1920909168 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2edc1a7150 con 0x7f2edc06aa00 2024-06-10T01:08:12.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.112+0000 7f2ee0a57700 1 --2- 172.21.15.37:0/1920909168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2edc06aa00 0x7f2edc101450 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc00ada0 tx=0x7f2ecc00c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:12.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.112+0000 7f2ed9ffb700 1 -- 172.21.15.37:0/1920909168 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc017020 con 0x7f2edc06aa00 2024-06-10T01:08:12.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.112+0000 7f2ee2cbb700 1 -- 172.21.15.37:0/1920909168 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2edc1a7440 con 0x7f2edc06aa00 2024-06-10T01:08:12.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.112+0000 7f2ed9ffb700 1 -- 172.21.15.37:0/1920909168 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ecc00cdb0 con 0x7f2edc06aa00 2024-06-10T01:08:12.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.112+0000 7f2ed9ffb700 1 -- 172.21.15.37:0/1920909168 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc005350 con 0x7f2edc06aa00 2024-06-10T01:08:12.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.112+0000 7f2ee2cbb700 1 -- 172.21.15.37:0/1920909168 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2edc1a7950 con 0x7f2edc06aa00 2024-06-10T01:08:12.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.114+0000 7f2ed9ffb700 1 -- 172.21.15.37:0/1920909168 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2ecc00c8d0 con 0x7f2edc06aa00 2024-06-10T01:08:12.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.115+0000 7f2ed9ffb700 1 --2- 172.21.15.37:0/1920909168 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2ec80a2c20 0x7f2ec80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:12.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.115+0000 7f2ed9ffb700 1 -- 172.21.15.37:0/1920909168 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f2ecc0cead0 con 0x7f2edc06aa00 2024-06-10T01:08:12.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.115+0000 7f2edbfff700 1 --2- 172.21.15.37:0/1920909168 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2ec80a2c20 0x7f2ec80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:12.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.116+0000 7f2edbfff700 1 --2- 172.21.15.37:0/1920909168 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2ec80a2c20 0x7f2ec80a50e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40098e0 tx=0x7f2ec4005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:12.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.116+0000 7f2eed53eb80 1 Processor -- start 2024-06-10T01:08:12.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.116+0000 7f2eed53eb80 1 -- start start 2024-06-10T01:08:12.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.116+0000 7f2eed53eb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628e6b1af60 0x5628e6b1b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:12.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.116+0000 7f2eed53eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628e6b1b880 0x5628e6b15260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:12.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.116+0000 7f2eed53eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e6b157a0 0x5628e6b15bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:12.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.116+0000 7f2eed53eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5628e6a07f60 con 0x5628e6b157a0 2024-06-10T01:08:12.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.116+0000 7f2eed53eb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5628e6a66850 con 0x5628e6b1af60 2024-06-10T01:08:12.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.116+0000 7f2eed53eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5628e69ce460 con 0x5628e6b1b880 2024-06-10T01:08:12.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.116+0000 7f2ee1258700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e6b157a0 0x5628e6b15bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:12.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.116+0000 7f2edbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628e6b1b880 0x5628e6b15260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:12.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.116+0000 7f2ee1258700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e6b157a0 0x5628e6b15bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48982/0 (socket says 172.21.15.37:48982) 2024-06-10T01:08:12.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.116+0000 7f2ee1258700 1 -- 172.21.15.37:0/3923933679 learned_addr learned my addr 172.21.15.37:0/3923933679 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:12.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.116+0000 7f2ee0a57700 1 --2- 172.21.15.37:0/3923933679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628e6b1af60 0x5628e6b1b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:12.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.117+0000 7f2ee1258700 1 -- 172.21.15.37:0/3923933679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628e6b1b880 msgr2=0x5628e6b15260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:12.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.117+0000 7f2ee1258700 1 --2- 172.21.15.37:0/3923933679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628e6b1b880 0x5628e6b15260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.117+0000 7f2ee1258700 1 -- 172.21.15.37:0/3923933679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628e6b1af60 msgr2=0x5628e6b1b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:12.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.117+0000 7f2ee1258700 1 --2- 172.21.15.37:0/3923933679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628e6b1af60 0x5628e6b1b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.117+0000 7f2ee1258700 1 -- 172.21.15.37:0/3923933679 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628e69b58d0 con 0x5628e6b157a0 2024-06-10T01:08:12.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.117+0000 7f2ee1258700 1 --2- 172.21.15.37:0/3923933679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e6b157a0 0x5628e6b15bc0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4007d40 tx=0x7f2ed40106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce977ed2bdb3c4bf server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:12.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.117+0000 7f2ed2ffd700 1 -- 172.21.15.37:0/3923933679 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4010e10 con 0x5628e6b157a0 2024-06-10T01:08:12.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.117+0000 7f2ed2ffd700 1 -- 172.21.15.37:0/3923933679 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed4011920 con 0x5628e6b157a0 2024-06-10T01:08:12.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.117+0000 7f2ed2ffd700 1 -- 172.21.15.37:0/3923933679 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed40197f0 con 0x5628e6b157a0 2024-06-10T01:08:12.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.118+0000 7f2eed53eb80 1 -- 172.21.15.37:0/3923933679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e6b157a0 msgr2=0x5628e6b15bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:12.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.118+0000 7f2eed53eb80 1 --2- 172.21.15.37:0/3923933679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e6b157a0 0x5628e6b15bc0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4007d40 tx=0x7f2ed40106b0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.118+0000 7f2eed53eb80 1 -- 172.21.15.37:0/3923933679 shutdown_connections 2024-06-10T01:08:12.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.118+0000 7f2eed53eb80 1 --2- 172.21.15.37:0/3923933679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e6b157a0 0x5628e6b15bc0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.118+0000 7f2eed53eb80 1 --2- 172.21.15.37:0/3923933679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628e6b1af60 0x5628e6b1b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.118+0000 7f2eed53eb80 1 --2- 172.21.15.37:0/3923933679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628e6b1b880 0x5628e6b15260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.118+0000 7f2eed53eb80 1 -- 172.21.15.37:0/3923933679 >> 172.21.15.37:0/3923933679 conn(0x5628e6a42080 msgr2=0x5628e6b10220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.118+0000 7f2eed53eb80 1 -- 172.21.15.37:0/3923933679 shutdown_connections 2024-06-10T01:08:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.118+0000 7f2eed53eb80 1 -- 172.21.15.37:0/3923933679 wait complete. 2024-06-10T01:08:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.118+0000 7f2eed53eb80 1 -- start start 2024-06-10T01:08:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.118+0000 7f2eed53eb80 1 Processor -- start 2024-06-10T01:08:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.119+0000 7f2eed53eb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628e6b157a0 0x5628e6b21570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.119+0000 7f2eed53eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628e6b1af60 0x5628e6b1e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.119+0000 7f2eed53eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e6b1b880 0x5628e6b1edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.119+0000 7f2eed53eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5628e6a06fc0 con 0x5628e6b1b880 2024-06-10T01:08:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.119+0000 7f2eed53eb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5628e6a62810 con 0x5628e6b157a0 2024-06-10T01:08:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.119+0000 7f2eed53eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5628e6b22260 con 0x5628e6b1af60 2024-06-10T01:08:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.119+0000 7f2eed53eb80 10 client.0 _ll_register_callbacks cb 0x5628e69fc5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.119+0000 7f2edbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628e6b1af60 0x5628e6b1e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:12.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.119+0000 7f2edbfff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628e6b1af60 0x5628e6b1e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:37716/0 (socket says 172.21.15.37:37716) 2024-06-10T01:08:12.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.119+0000 7f2edbfff700 1 -- 172.21.15.37:0/984958966 learned_addr learned my addr 172.21.15.37:0/984958966 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:12.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.119+0000 7f2ee1258700 1 --2- 172.21.15.37:0/984958966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e6b1b880 0x5628e6b1edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:12.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.119+0000 7f2ee0a57700 1 --2- 172.21.15.37:0/984958966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628e6b157a0 0x5628e6b21570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:12.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.119+0000 7f2edbfff700 1 -- 172.21.15.37:0/984958966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628e6b157a0 msgr2=0x5628e6b21570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:12.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.119+0000 7f2edbfff700 1 --2- 172.21.15.37:0/984958966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628e6b157a0 0x5628e6b21570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.119+0000 7f2edbfff700 1 -- 172.21.15.37:0/984958966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e6b1b880 msgr2=0x5628e6b1edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:12.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.119+0000 7f2edbfff700 1 --2- 172.21.15.37:0/984958966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e6b1b880 0x5628e6b1edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.119+0000 7f2edbfff700 1 -- 172.21.15.37:0/984958966 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628e6a2c750 con 0x5628e6b1af60 2024-06-10T01:08:12.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.120+0000 7f2edbfff700 1 --2- 172.21.15.37:0/984958966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628e6b1af60 0x5628e6b1e870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009990 tx=0x7f2ec401a050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37da8c35faece633 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:12.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.120+0000 7f2ebbfff700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:08:12.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.120+0000 7f2ebbfff700 1 -- 172.21.15.37:0/984958966 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4024610 con 0x5628e6b1af60 2024-06-10T01:08:12.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.120+0000 7f2eed53eb80 1 -- 172.21.15.37:0/984958966 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ed4011920 con 0x5628e6b1af60 2024-06-10T01:08:12.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.120+0000 7f2ebbfff700 1 -- 172.21.15.37:0/984958966 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f2ec40247b0 con 0x5628e6b1af60 2024-06-10T01:08:12.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.120+0000 7f2ebbfff700 1 -- 172.21.15.37:0/984958966 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2ec403e050 con 0x5628e6b1af60 2024-06-10T01:08:12.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.120+0000 7f2eed53eb80 10 client.26908 fetch_fsmap learned FSMap version 31 2024-06-10T01:08:12.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.120+0000 7f2eed53eb80 1 -- 172.21.15.37:0/984958966 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5628e6b2c9f0 con 0x5628e6b1af60 2024-06-10T01:08:12.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.120+0000 7f2ebbfff700 1 -- 172.21.15.37:0/984958966 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed4011920 con 0x5628e6b1af60 2024-06-10T01:08:12.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.121+0000 7f2ebbfff700 1 -- 172.21.15.37:0/984958966 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f2ec4030ac0 con 0x5628e6b1af60 2024-06-10T01:08:12.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.121+0000 7f2eed53eb80 10 client.26908 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:08:12.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.121+0000 7f2eed53eb80 10 client.26908 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-10T01:08:12.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.121+0000 7f2eed53eb80 20 client.26908 populate_metadata read hostname 'smithi037' 2024-06-10T01:08:12.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.121+0000 7f2eed53eb80 1 --2- 172.21.15.37:0/984958966 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5628e6b2f080 0x5628e6b314a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:12.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.121+0000 7f2eed53eb80 4 client.26908 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:08:12.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.121+0000 7f2eed53eb80 1 -- 172.21.15.37:0/984958966 --> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628e6a73d50 con 0x5628e6b2f080 2024-06-10T01:08:12.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.122+0000 7f2ee0a57700 1 --2- 172.21.15.37:0/984958966 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5628e6b2f080 0x5628e6b314a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:12.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.122+0000 7f2ee0a57700 1 --2- 172.21.15.37:0/984958966 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5628e6b2f080 0x5628e6b314a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=145a88a8361da7e server_cookie=c4ec277cc47b79f8 in_seq=0 out_seq=0 2024-06-10T01:08:12.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.122+0000 7f2ebbfff700 10 client.26908 ms_handle_connect on v2:172.21.15.167:6836/2720148065 2024-06-10T01:08:12.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.123+0000 7f2ebbfff700 1 -- 172.21.15.37:0/984958966 <== mds.2 v2:172.21.15.167:6836/2720148065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5628e6a73d50 con 0x5628e6b2f080 2024-06-10T01:08:12.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.123+0000 7f2ebbfff700 10 client.26908 handle_command_reply: tid=0 2024-06-10T01:08:12.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.123+0000 7f2eed53eb80 1 client.26908 shutdown 2024-06-10T01:08:12.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.123+0000 7f2eed53eb80 1 -- 172.21.15.37:0/984958966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628e6b1af60 msgr2=0x5628e6b1e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:12.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.123+0000 7f2eed53eb80 1 --2- 172.21.15.37:0/984958966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628e6b1af60 0x5628e6b1e870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009990 tx=0x7f2ec401a050 comp rx=0 tx=0).stop 2024-06-10T01:08:12.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.123+0000 7f2eed53eb80 1 -- 172.21.15.37:0/984958966 shutdown_connections 2024-06-10T01:08:12.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.123+0000 7f2eed53eb80 1 --2- 172.21.15.37:0/984958966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e6b1b880 0x5628e6b1edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.123+0000 7f2eed53eb80 1 --2- 172.21.15.37:0/984958966 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5628e6b2f080 0x5628e6b314a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.123+0000 7f2eed53eb80 1 --2- 172.21.15.37:0/984958966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628e6b157a0 0x5628e6b21570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.123+0000 7f2eed53eb80 1 --2- 172.21.15.37:0/984958966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628e6b1af60 0x5628e6b1e870 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.123+0000 7f2eed53eb80 1 -- 172.21.15.37:0/984958966 >> 172.21.15.37:0/984958966 conn(0x5628e6a42080 msgr2=0x5628e6b0f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:12.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.124+0000 7f2ebbfff700 0 client.26908 ms_handle_reset on v2:172.21.15.167:6836/2720148065 2024-06-10T01:08:12.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.124+0000 7f2eed53eb80 1 -- 172.21.15.37:0/984958966 shutdown_connections 2024-06-10T01:08:12.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.124+0000 7f2eed53eb80 1 -- 172.21.15.37:0/984958966 wait complete. 2024-06-10T01:08:12.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.124+0000 7f2eed53eb80 20 client.26908 trim_cache size 0 max 16384 2024-06-10T01:08:12.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.142+0000 7f2eed53eb80 1 Processor -- start 2024-06-10T01:08:12.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.142+0000 7f2eed53eb80 1 -- start start 2024-06-10T01:08:12.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.142+0000 7f2eed53eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628e6b157a0 0x5628e6b15260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:12.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.142+0000 7f2eed53eb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628e6b1af60 0x5628e6b1ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:12.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.142+0000 7f2eed53eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e6b1b880 0x5628e6b1c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:12.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.142+0000 7f2eed53eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc1b2f60 con 0x5628e6b1b880 2024-06-10T01:08:12.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.142+0000 7f2eed53eb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc1b07f0 con 0x5628e6b1af60 2024-06-10T01:08:12.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.142+0000 7f2eed53eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc1ae640 con 0x5628e6b157a0 2024-06-10T01:08:12.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.142+0000 7f2ee1258700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e6b1b880 0x5628e6b1c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:12.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.142+0000 7f2ee1258700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e6b1b880 0x5628e6b1c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49000/0 (socket says 172.21.15.37:49000) 2024-06-10T01:08:12.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.142+0000 7f2ee1258700 1 -- 172.21.15.37:0/2737702923 learned_addr learned my addr 172.21.15.37:0/2737702923 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:12.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.142+0000 7f2edbfff700 1 --2- 172.21.15.37:0/2737702923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628e6b1af60 0x5628e6b1ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:12.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.142+0000 7f2ee0a57700 1 --2- 172.21.15.37:0/2737702923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628e6b157a0 0x5628e6b15260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:12.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.143+0000 7f2ee1258700 1 -- 172.21.15.37:0/2737702923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628e6b157a0 msgr2=0x5628e6b15260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:12.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.143+0000 7f2ee1258700 1 --2- 172.21.15.37:0/2737702923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628e6b157a0 0x5628e6b15260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.143+0000 7f2ee1258700 1 -- 172.21.15.37:0/2737702923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628e6b1af60 msgr2=0x5628e6b1ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:12.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.143+0000 7f2ee1258700 1 --2- 172.21.15.37:0/2737702923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628e6b1af60 0x5628e6b1ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.143+0000 7f2ee1258700 1 -- 172.21.15.37:0/2737702923 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628e6b1ce00 con 0x5628e6b1b880 2024-06-10T01:08:12.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.143+0000 7f2edbfff700 1 --2- 172.21.15.37:0/2737702923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628e6b1af60 0x5628e6b1ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:12.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.143+0000 7f2ee0a57700 1 --2- 172.21.15.37:0/2737702923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628e6b157a0 0x5628e6b15260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:12.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.143+0000 7f2ee1258700 1 --2- 172.21.15.37:0/2737702923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e6b1b880 0x5628e6b1c370 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4007c60 tx=0x7f2ed4007f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d282bacf3f6596e7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:12.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.143+0000 7f2ed27fc700 1 -- 172.21.15.37:0/2737702923 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4028070 con 0x5628e6b1b880 2024-06-10T01:08:12.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.143+0000 7f2ed27fc700 1 -- 172.21.15.37:0/2737702923 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed4005520 con 0x5628e6b1b880 2024-06-10T01:08:12.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.143+0000 7f2ed27fc700 1 -- 172.21.15.37:0/2737702923 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed401f040 con 0x5628e6b1b880 2024-06-10T01:08:12.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.144+0000 7f2eed53eb80 1 -- 172.21.15.37:0/2737702923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e6b1b880 msgr2=0x5628e6b1c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:12.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.144+0000 7f2eed53eb80 1 --2- 172.21.15.37:0/2737702923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e6b1b880 0x5628e6b1c370 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4007c60 tx=0x7f2ed4007f80 comp rx=0 tx=0).stop 2024-06-10T01:08:12.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.144+0000 7f2eed53eb80 1 -- 172.21.15.37:0/2737702923 shutdown_connections 2024-06-10T01:08:12.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.144+0000 7f2eed53eb80 1 --2- 172.21.15.37:0/2737702923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e6b1b880 0x5628e6b1c370 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.144+0000 7f2eed53eb80 1 --2- 172.21.15.37:0/2737702923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628e6b157a0 0x5628e6b15260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.144+0000 7f2eed53eb80 1 --2- 172.21.15.37:0/2737702923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628e6b1af60 0x5628e6b1ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.144+0000 7f2eed53eb80 1 -- 172.21.15.37:0/2737702923 >> 172.21.15.37:0/2737702923 conn(0x5628e6a42080 msgr2=0x5628e6b1bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:12.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.144+0000 7f2eed53eb80 1 -- 172.21.15.37:0/2737702923 shutdown_connections 2024-06-10T01:08:12.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.144+0000 7f2eed53eb80 1 -- 172.21.15.37:0/2737702923 wait complete. 2024-06-10T01:08:12.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.144+0000 7f2eed53eb80 1 -- start start 2024-06-10T01:08:12.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.144+0000 7f2eed53eb80 1 Processor -- start 2024-06-10T01:08:12.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.145+0000 7f2eed53eb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628e6b157a0 0x5628e6b27730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:12.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.145+0000 7f2eed53eb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628e6b1af60 0x5628e6b27c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:12.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.145+0000 7f2eed53eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e6b1b880 0x5628e6b36120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:12.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.145+0000 7f2eed53eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc1aecb0 con 0x5628e6b1b880 2024-06-10T01:08:12.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.145+0000 7f2eed53eb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc1b0120 con 0x5628e6b157a0 2024-06-10T01:08:12.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.145+0000 7f2eed53eb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc1af380 con 0x5628e6b1af60 2024-06-10T01:08:12.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.145+0000 7f2eed53eb80 10 client.0 _ll_register_callbacks cb 0x5628e69fc5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:12.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.145+0000 7f2ee1258700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e6b1b880 0x5628e6b36120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:12.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.145+0000 7f2ee1258700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e6b1b880 0x5628e6b36120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49010/0 (socket says 172.21.15.37:49010) 2024-06-10T01:08:12.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.145+0000 7f2ee1258700 1 -- 172.21.15.37:0/1495697880 learned_addr learned my addr 172.21.15.37:0/1495697880 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:12.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.145+0000 7f2edbfff700 1 --2- 172.21.15.37:0/1495697880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628e6b1af60 0x5628e6b27c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:12.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.145+0000 7f2ee0a57700 1 --2- 172.21.15.37:0/1495697880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628e6b157a0 0x5628e6b27730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:12.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.145+0000 7f2ee1258700 1 -- 172.21.15.37:0/1495697880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628e6b1af60 msgr2=0x5628e6b27c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:12.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.145+0000 7f2ee1258700 1 --2- 172.21.15.37:0/1495697880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628e6b1af60 0x5628e6b27c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.145+0000 7f2ee1258700 1 -- 172.21.15.37:0/1495697880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628e6b157a0 msgr2=0x5628e6b27730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:12.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.145+0000 7f2ee1258700 1 --2- 172.21.15.37:0/1495697880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628e6b157a0 0x5628e6b27730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.145+0000 7f2ee1258700 1 -- 172.21.15.37:0/1495697880 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628e6b367f0 con 0x5628e6b1b880 2024-06-10T01:08:12.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.145+0000 7f2ee0a57700 1 --2- 172.21.15.37:0/1495697880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628e6b157a0 0x5628e6b27730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:08:12.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.145+0000 7f2ee1258700 1 --2- 172.21.15.37:0/1495697880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e6b1b880 0x5628e6b36120 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4005740 tx=0x7f2ed4007760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4845db2da35af462 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:12.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.145+0000 7f2ebbfff700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:08:12.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.146+0000 7f2ebbfff700 1 -- 172.21.15.37:0/1495697880 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4031060 con 0x5628e6b1b880 2024-06-10T01:08:12.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.146+0000 7f2ebbfff700 1 -- 172.21.15.37:0/1495697880 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f2ed4003680 con 0x5628e6b1b880 2024-06-10T01:08:12.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.146+0000 7f2eed53eb80 1 -- 172.21.15.37:0/1495697880 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ed4005520 con 0x5628e6b1b880 2024-06-10T01:08:12.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.146+0000 7f2ebbfff700 1 -- 172.21.15.37:0/1495697880 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2ed4038050 con 0x5628e6b1b880 2024-06-10T01:08:12.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.146+0000 7f2eed53eb80 10 client.17415 fetch_fsmap learned FSMap version 31 2024-06-10T01:08:12.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.146+0000 7f2eed53eb80 1 -- 172.21.15.37:0/1495697880 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5628e6ba1ed0 con 0x5628e6b1b880 2024-06-10T01:08:12.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.146+0000 7f2ebbfff700 1 -- 172.21.15.37:0/1495697880 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed4005520 con 0x5628e6b1b880 2024-06-10T01:08:12.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.146+0000 7f2ebbfff700 1 -- 172.21.15.37:0/1495697880 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f2ed402b070 con 0x5628e6b1b880 2024-06-10T01:08:12.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.146+0000 7f2eed53eb80 10 client.17415 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:08:12.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.146+0000 7f2eed53eb80 10 client.17415 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-10T01:08:12.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.146+0000 7f2eed53eb80 20 client.17415 populate_metadata read hostname 'smithi037' 2024-06-10T01:08:12.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.146+0000 7f2eed53eb80 1 --2- 172.21.15.37:0/1495697880 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5628e6ba22d0 0x5628e6ba46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:12.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.146+0000 7f2eed53eb80 4 client.17415 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:08:12.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.146+0000 7f2eed53eb80 1 -- 172.21.15.37:0/1495697880 --> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5628e6a6e2d0 con 0x5628e6ba22d0 2024-06-10T01:08:12.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.146+0000 7f2ee0a57700 1 --2- 172.21.15.37:0/1495697880 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5628e6ba22d0 0x5628e6ba46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:12.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.147+0000 7f2ee0a57700 1 --2- 172.21.15.37:0/1495697880 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5628e6ba22d0 0x5628e6ba46c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b77b58e12825df15 server_cookie=b916afed9c986fd8 in_seq=0 out_seq=0 2024-06-10T01:08:12.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.147+0000 7f2ebbfff700 10 client.17415 ms_handle_connect on v2:172.21.15.167:6836/2720148065 2024-06-10T01:08:12.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.147+0000 7f2ebbfff700 1 -- 172.21.15.37:0/1495697880 <== mds.2 v2:172.21.15.167:6836/2720148065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5628e6a6e2d0 con 0x5628e6ba22d0 2024-06-10T01:08:12.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.147+0000 7f2ebbfff700 10 client.17415 handle_command_reply: tid=0 2024-06-10T01:08:12.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.147+0000 7f2eed53eb80 1 client.17415 shutdown 2024-06-10T01:08:12.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.147+0000 7f2eed53eb80 1 -- 172.21.15.37:0/1495697880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e6b1b880 msgr2=0x5628e6b36120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:12.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.147+0000 7f2eed53eb80 1 --2- 172.21.15.37:0/1495697880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e6b1b880 0x5628e6b36120 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4005740 tx=0x7f2ed4007760 comp rx=0 tx=0).stop 2024-06-10T01:08:12.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.147+0000 7f2eed53eb80 1 -- 172.21.15.37:0/1495697880 shutdown_connections 2024-06-10T01:08:12.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.147+0000 7f2eed53eb80 1 --2- 172.21.15.37:0/1495697880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5628e6b1b880 0x5628e6b36120 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.147+0000 7f2eed53eb80 1 --2- 172.21.15.37:0/1495697880 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x5628e6ba22d0 0x5628e6ba46c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.147+0000 7f2eed53eb80 1 --2- 172.21.15.37:0/1495697880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5628e6b157a0 0x5628e6b27730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.147+0000 7f2eed53eb80 1 --2- 172.21.15.37:0/1495697880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5628e6b1af60 0x5628e6b27c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.147+0000 7f2ebbfff700 0 client.17415 ms_handle_reset on v2:172.21.15.167:6836/2720148065 2024-06-10T01:08:12.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.147+0000 7f2eed53eb80 1 -- 172.21.15.37:0/1495697880 >> 172.21.15.37:0/1495697880 conn(0x5628e6a42080 msgr2=0x5628e6b200a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:12.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.147+0000 7f2eed53eb80 1 -- 172.21.15.37:0/1495697880 shutdown_connections 2024-06-10T01:08:12.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.147+0000 7f2eed53eb80 1 -- 172.21.15.37:0/1495697880 wait complete. 2024-06-10T01:08:12.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.147+0000 7f2eed53eb80 20 client.17415 trim_cache size 0 max 16384 2024-06-10T01:08:12.150 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T01:08:12.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.148+0000 7f2ee2cbb700 1 -- 172.21.15.37:0/1920909168 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2ec80a2c20 msgr2=0x7f2ec80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:12.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.148+0000 7f2ee2cbb700 1 --2- 172.21.15.37:0/1920909168 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2ec80a2c20 0x7f2ec80a50e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40098e0 tx=0x7f2ec4005960 comp rx=0 tx=0).stop 2024-06-10T01:08:12.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.148+0000 7f2ee2cbb700 1 -- 172.21.15.37:0/1920909168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2edc06aa00 msgr2=0x7f2edc101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:12.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.148+0000 7f2ee2cbb700 1 --2- 172.21.15.37:0/1920909168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2edc06aa00 0x7f2edc101450 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc00ada0 tx=0x7f2ecc00c330 comp rx=0 tx=0).stop 2024-06-10T01:08:12.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.148+0000 7f2ee2cbb700 1 -- 172.21.15.37:0/1920909168 shutdown_connections 2024-06-10T01:08:12.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.148+0000 7f2ee2cbb700 1 --2- 172.21.15.37:0/1920909168 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2ec80a2c20 0x7f2ec80a50e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.148+0000 7f2ee2cbb700 1 --2- 172.21.15.37:0/1920909168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2edc1084f0 0x7f2edc101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.148+0000 7f2ee2cbb700 1 --2- 172.21.15.37:0/1920909168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2edc06aa00 0x7f2edc101450 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.148+0000 7f2ee2cbb700 1 --2- 172.21.15.37:0/1920909168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2edc06b3c0 0x7f2edc101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:12.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.148+0000 7f2ee2cbb700 1 -- 172.21.15.37:0/1920909168 >> 172.21.15.37:0/1920909168 conn(0x7f2edc078300 msgr2=0x7f2edc0fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:12.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.148+0000 7f2ee2cbb700 1 -- 172.21.15.37:0/1920909168 shutdown_connections 2024-06-10T01:08:12.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:12.148+0000 7f2ee2cbb700 1 -- 172.21.15.37:0/1920909168 wait complete. 2024-06-10T01:08:12.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:12 smithi037 ceph-mon[32192]: pgmap v776: 129 pgs: 129 active+clean; 11 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 515 op/s 2024-06-10T01:08:12.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:12 smithi167 ceph-mon[36917]: pgmap v776: 129 pgs: 129 active+clean; 11 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 515 op/s 2024-06-10T01:08:12.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:12 smithi094 ceph-mon[36708]: pgmap v776: 129 pgs: 129 active+clean; 11 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 515 op/s 2024-06-10T01:08:12.848 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:08:13.098 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:08:14.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:14.415+0000 7f83f2ff5700 1 -- 172.21.15.37:0/2506976174 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f325ab0460 con 0x55f325be14b0 2024-06-10T01:08:14.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:14.415+0000 7f83f2ff5700 10 client.17391 handle_command_reply: tid=0 2024-06-10T01:08:14.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:14.415+0000 7f841fa9fb80 1 client.17391 shutdown 2024-06-10T01:08:14.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:14.415+0000 7f841fa9fb80 1 -- 172.21.15.37:0/2506976174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f325b5deb0 msgr2=0x55f325b66b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:14.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:14.415+0000 7f841fa9fb80 1 --2- 172.21.15.37:0/2506976174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f325b5deb0 0x55f325b66b20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f84080a40a0 tx=0x7f840800c7a0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:14.415+0000 7f841fa9fb80 1 -- 172.21.15.37:0/2506976174 shutdown_connections 2024-06-10T01:08:14.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:14.415+0000 7f841fa9fb80 1 --2- 172.21.15.37:0/2506976174 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55f325be14b0 0x55f325be38a0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:14.415+0000 7f841fa9fb80 1 --2- 172.21.15.37:0/2506976174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f325b5deb0 0x55f325b66b20 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:14.415+0000 7f841fa9fb80 1 --2- 172.21.15.37:0/2506976174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f325b57ce0 0x55f325b69b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:14.415+0000 7f83f2ff5700 0 client.17391 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:14.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:14.415+0000 7f841fa9fb80 1 --2- 172.21.15.37:0/2506976174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f325b5d590 0x55f325b6a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:14.415+0000 7f841fa9fb80 1 -- 172.21.15.37:0/2506976174 >> 172.21.15.37:0/2506976174 conn(0x55f325a84210 msgr2=0x55f325b56790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:14.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:14.415+0000 7f841fa9fb80 1 -- 172.21.15.37:0/2506976174 shutdown_connections 2024-06-10T01:08:14.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:14.415+0000 7f841fa9fb80 1 -- 172.21.15.37:0/2506976174 wait complete. 2024-06-10T01:08:14.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:14.415+0000 7f841fa9fb80 20 client.17391 trim_cache size 0 max 16384 2024-06-10T01:08:14.414 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:08:14.414 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 30, 2024-06-10T01:08:14.414 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24472, 2024-06-10T01:08:14.414 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 30, 2024-06-10T01:08:14.414 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-06-10T01:08:14.414 INFO:teuthology.orchestra.run.smithi037.stdout: "96a2bbd8": { 2024-06-10T01:08:14.414 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 30, 2024-06-10T01:08:14.414 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-06-10T01:08:14.414 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:08:14.415 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-06-10T01:08:14.415 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:08:14.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:08:14.415 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-06-10T01:08:14.415 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 137.8, 2024-06-10T01:08:14.415 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-06-10T01:08:14.415 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-06-10T01:08:14.415 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-06-10T01:08:14.415 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:08:14.416 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-06-10T01:08:14.416 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:08:14.416 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:08:14.416 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:08:14.416 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:08:14.416 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:08:14.416 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:08:14.416 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:08:14.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:14.416+0000 7f841521c700 1 -- 172.21.15.37:0/1650437957 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f83f80a2d40 msgr2=0x7f83f80a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:14.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:14.416+0000 7f841521c700 1 --2- 172.21.15.37:0/1650437957 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f83f80a2d40 0x7f83f80a5200 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8410199b70 tx=0x7f840000b500 comp rx=0 tx=0).stop 2024-06-10T01:08:14.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:14.416+0000 7f841521c700 1 -- 172.21.15.37:0/1650437957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8410108520 msgr2=0x7f8410198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:14.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:14.416+0000 7f841521c700 1 --2- 172.21.15.37:0/1650437957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8410108520 0x7f8410198a30 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f840800efc0 tx=0x7f840800c3a0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:14.416+0000 7f841521c700 1 -- 172.21.15.37:0/1650437957 shutdown_connections 2024-06-10T01:08:14.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:14.416+0000 7f841521c700 1 --2- 172.21.15.37:0/1650437957 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f83f80a2d40 0x7f83f80a5200 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:14.416+0000 7f841521c700 1 --2- 172.21.15.37:0/1650437957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8410108520 0x7f8410198a30 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:14.416+0000 7f841521c700 1 --2- 172.21.15.37:0/1650437957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f841006aa60 0x7f841019e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:14.416+0000 7f841521c700 1 --2- 172.21.15.37:0/1650437957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f841006b420 0x7f841019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:14.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:14.416+0000 7f841521c700 1 -- 172.21.15.37:0/1650437957 >> 172.21.15.37:0/1650437957 conn(0x7f84100783d0 msgr2=0x7f84100ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:14.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:14.416+0000 7f841521c700 1 -- 172.21.15.37:0/1650437957 shutdown_connections 2024-06-10T01:08:14.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:14.416+0000 7f841521c700 1 -- 172.21.15.37:0/1650437957 wait complete. 2024-06-10T01:08:14.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:14 smithi167 ceph-mon[36917]: pgmap v777: 129 pgs: 129 active+clean; 6.8 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 615 op/s 2024-06-10T01:08:14.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:14 smithi094 ceph-mon[36708]: pgmap v777: 129 pgs: 129 active+clean; 6.8 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 615 op/s 2024-06-10T01:08:14.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:14 smithi037 ceph-mon[32192]: pgmap v777: 129 pgs: 129 active+clean; 6.8 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 615 op/s 2024-06-10T01:08:15.268 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '96a2bbd8', quiesce took 16.8 seconds. Will release after: 1.0 2024-06-10T01:08:15.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.527+0000 7fe1ec73e700 1 -- 172.21.15.37:0/1983748477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1e406a2d0 msgr2=0x7fe1e4106c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:15.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.527+0000 7fe1ec73e700 1 --2- 172.21.15.37:0/1983748477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1e406a2d0 0x7fe1e4106c10 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0009990 tx=0x7fe1e0009ca0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.529+0000 7fe1ec73e700 1 -- 172.21.15.37:0/1983748477 shutdown_connections 2024-06-10T01:08:15.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.529+0000 7fe1ec73e700 1 --2- 172.21.15.37:0/1983748477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1e4100e30 0x7fe1e41012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.529+0000 7fe1ec73e700 1 --2- 172.21.15.37:0/1983748477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1e406a2d0 0x7fe1e4106c10 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.529+0000 7fe1ec73e700 1 --2- 172.21.15.37:0/1983748477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e406a740 0x7fe1e4107150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.529+0000 7fe1ec73e700 1 -- 172.21.15.37:0/1983748477 >> 172.21.15.37:0/1983748477 conn(0x7fe1e4076730 msgr2=0x7fe1e4076b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:15.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.529+0000 7fe1ec73e700 1 -- 172.21.15.37:0/1983748477 shutdown_connections 2024-06-10T01:08:15.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.531+0000 7fe1ec73e700 1 -- 172.21.15.37:0/1983748477 wait complete. 2024-06-10T01:08:15.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.532+0000 7fe1ec73e700 1 Processor -- start 2024-06-10T01:08:15.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.532+0000 7fe1ec73e700 1 -- start start 2024-06-10T01:08:15.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.533+0000 7fe1ec73e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e406a2d0 0x7fe1e4110db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.533+0000 7fe1ec73e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1e406a740 0x7fe1e41112f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.533+0000 7fe1ec73e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1e4100e30 0x7fe1e4111830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.533+0000 7fe1ec73e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e410e6c0 con 0x7fe1e4100e30 2024-06-10T01:08:15.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.533+0000 7fe1ec73e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e410e540 con 0x7fe1e406a2d0 2024-06-10T01:08:15.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.533+0000 7fe1e9cd9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1e406a740 0x7fe1e41112f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.533+0000 7fe1eacdb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1e4100e30 0x7fe1e4111830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.533+0000 7fe1e9cd9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1e406a740 0x7fe1e41112f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:37748/0 (socket says 172.21.15.37:37748) 2024-06-10T01:08:15.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.533+0000 7fe1e9cd9700 1 -- 172.21.15.37:0/1157530089 learned_addr learned my addr 172.21.15.37:0/1157530089 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:15.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.533+0000 7fe1eacdb700 1 --2- 172.21.15.37:0/1157530089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1e4100e30 0x7fe1e4111830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49020/0 (socket says 172.21.15.37:49020) 2024-06-10T01:08:15.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.533+0000 7fe1ea4da700 1 --2- 172.21.15.37:0/1157530089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e406a2d0 0x7fe1e4110db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.534+0000 7fe1ec73e700 1 -- 172.21.15.37:0/1157530089 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e410e840 con 0x7fe1e406a740 2024-06-10T01:08:15.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.534+0000 7fe1e9cd9700 1 -- 172.21.15.37:0/1157530089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e406a2d0 msgr2=0x7fe1e4110db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:15.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.535+0000 7fe1e9cd9700 1 --2- 172.21.15.37:0/1157530089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e406a2d0 0x7fe1e4110db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.535+0000 7fe1e9cd9700 1 -- 172.21.15.37:0/1157530089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1e4100e30 msgr2=0x7fe1e4111830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:15.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.535+0000 7fe1e9cd9700 1 --2- 172.21.15.37:0/1157530089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1e4100e30 0x7fe1e4111830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.535+0000 7fe1e9cd9700 1 -- 172.21.15.37:0/1157530089 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e41ab4f0 con 0x7fe1e406a740 2024-06-10T01:08:15.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.535+0000 7fe1eacdb700 1 --2- 172.21.15.37:0/1157530089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1e4100e30 0x7fe1e4111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:15.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.535+0000 7fe1e9cd9700 1 --2- 172.21.15.37:0/1157530089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1e406a740 0x7fe1e41112f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe1d400e970 tx=0x7fe1d400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:15.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.536+0000 7fe1db7fe700 1 -- 172.21.15.37:0/1157530089 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d400cbd0 con 0x7fe1e406a740 2024-06-10T01:08:15.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.536+0000 7fe1ec73e700 1 -- 172.21.15.37:0/1157530089 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1e41ab7e0 con 0x7fe1e406a740 2024-06-10T01:08:15.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.536+0000 7fe1db7fe700 1 -- 172.21.15.37:0/1157530089 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1d400cd70 con 0x7fe1e406a740 2024-06-10T01:08:15.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.536+0000 7fe1ec73e700 1 -- 172.21.15.37:0/1157530089 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1e41abd70 con 0x7fe1e406a740 2024-06-10T01:08:15.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.537+0000 7fe1db7fe700 1 -- 172.21.15.37:0/1157530089 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4018680 con 0x7fe1e406a740 2024-06-10T01:08:15.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.542+0000 7fe1f6fc1b80 1 Processor -- start 2024-06-10T01:08:15.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.542+0000 7fe1f6fc1b80 1 -- start start 2024-06-10T01:08:15.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.542+0000 7fe1f6fc1b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558129f60f60 0x558129f61340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.542+0000 7fe1f6fc1b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558129f61880 0x558129f5b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.543+0000 7fe1f6fc1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558129f5b7a0 0x558129f5bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.543+0000 7fe1f6fc1b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558129e4df60 con 0x558129f60f60 2024-06-10T01:08:15.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.543+0000 7fe1f6fc1b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558129eac850 con 0x558129f61880 2024-06-10T01:08:15.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.543+0000 7fe1f6fc1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558129e14460 con 0x558129f5b7a0 2024-06-10T01:08:15.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.543+0000 7fe1ea4da700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558129f61880 0x558129f5b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.543+0000 7fe1ea4da700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558129f61880 0x558129f5b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:55066/0 (socket says 172.21.15.37:55066) 2024-06-10T01:08:15.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.543+0000 7fe1ea4da700 1 -- 172.21.15.37:0/3238334374 learned_addr learned my addr 172.21.15.37:0/3238334374 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:15.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.543+0000 7fe1eacdb700 1 --2- 172.21.15.37:0/3238334374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558129f60f60 0x558129f61340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.543+0000 7fe1ea4da700 1 -- 172.21.15.37:0/3238334374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558129f5b7a0 msgr2=0x558129f5bbc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:08:15.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.543+0000 7fe1ea4da700 1 --2- 172.21.15.37:0/3238334374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558129f5b7a0 0x558129f5bbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.543+0000 7fe1ea4da700 1 -- 172.21.15.37:0/3238334374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558129f60f60 msgr2=0x558129f61340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:15.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.543+0000 7fe1ea4da700 1 --2- 172.21.15.37:0/3238334374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558129f60f60 0x558129f61340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.543+0000 7fe1ea4da700 1 -- 172.21.15.37:0/3238334374 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558129dfb8d0 con 0x558129f61880 2024-06-10T01:08:15.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.543+0000 7fe1ea4da700 1 --2- 172.21.15.37:0/3238334374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558129f61880 0x558129f5b260 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0015000 tx=0x7fe1e0005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaaa72658dd2c4af server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:15.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.543+0000 7fe1db7fe700 1 -- 172.21.15.37:0/1157530089 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe1d401f020 con 0x7fe1e406a740 2024-06-10T01:08:15.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.544+0000 7fe1d8ff9700 1 -- 172.21.15.37:0/3238334374 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e000bd10 con 0x558129f61880 2024-06-10T01:08:15.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.544+0000 7fe1d8ff9700 1 -- 172.21.15.37:0/3238334374 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1e0005550 con 0x558129f61880 2024-06-10T01:08:15.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.544+0000 7fe1d8ff9700 1 -- 172.21.15.37:0/3238334374 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e000f460 con 0x558129f61880 2024-06-10T01:08:15.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.545+0000 7fe1db7fe700 1 --2- 172.21.15.37:0/1157530089 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe1d00a2e90 0x7fe1d00a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.545+0000 7fe1f6fc1b80 1 -- 172.21.15.37:0/3238334374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558129f61880 msgr2=0x558129f5b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:15.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.545+0000 7fe1f6fc1b80 1 --2- 172.21.15.37:0/3238334374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558129f61880 0x558129f5b260 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0015000 tx=0x7fe1e0005c00 comp rx=0 tx=0).stop 2024-06-10T01:08:15.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.545+0000 7fe1eacdb700 1 --2- 172.21.15.37:0/1157530089 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe1d00a2e90 0x7fe1d00a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.545+0000 7fe1f6fc1b80 1 -- 172.21.15.37:0/3238334374 shutdown_connections 2024-06-10T01:08:15.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.545+0000 7fe1f6fc1b80 1 --2- 172.21.15.37:0/3238334374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558129f5b7a0 0x558129f5bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.545+0000 7fe1f6fc1b80 1 --2- 172.21.15.37:0/3238334374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558129f60f60 0x558129f61340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.545+0000 7fe1f6fc1b80 1 --2- 172.21.15.37:0/3238334374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558129f61880 0x558129f5b260 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.545+0000 7fe1f6fc1b80 1 -- 172.21.15.37:0/3238334374 >> 172.21.15.37:0/3238334374 conn(0x558129e88080 msgr2=0x558129f56220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:15.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.545+0000 7fe1db7fe700 1 -- 172.21.15.37:0/1157530089 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fe1d4018420 con 0x7fe1e406a740 2024-06-10T01:08:15.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.546+0000 7fe1f6fc1b80 1 -- 172.21.15.37:0/3238334374 shutdown_connections 2024-06-10T01:08:15.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.549+0000 7fe1f6fc1b80 1 -- 172.21.15.37:0/3238334374 wait complete. 2024-06-10T01:08:15.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.549+0000 7fe1f6fc1b80 1 -- start start 2024-06-10T01:08:15.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.549+0000 7fe1eacdb700 1 --2- 172.21.15.37:0/1157530089 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe1d00a2e90 0x7fe1d00a5350 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009fd0 tx=0x7fe1dc009050 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:15.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.549+0000 7fe1f6fc1b80 1 Processor -- start 2024-06-10T01:08:15.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.550+0000 7fe1f6fc1b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558129f60f60 0x558129f67770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.550+0000 7fe1f6fc1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558129f61880 0x558129f64a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.550+0000 7fe1f6fc1b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558129f64fb0 0x558129f65430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.551+0000 7fe1f6fc1b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558129e4cfc0 con 0x558129f64fb0 2024-06-10T01:08:15.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.551+0000 7fe1f6fc1b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558129ea8810 con 0x558129f60f60 2024-06-10T01:08:15.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.551+0000 7fe1f6fc1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558129f681a0 con 0x558129f61880 2024-06-10T01:08:15.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.551+0000 7fe1f6fc1b80 10 client.0 _ll_register_callbacks cb 0x558129e425b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:15.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.551+0000 7fe1e9cd9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558129f61880 0x558129f64a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.552+0000 7fe1e9cd9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558129f61880 0x558129f64a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:37762/0 (socket says 172.21.15.37:37762) 2024-06-10T01:08:15.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.552+0000 7fe1eacdb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558129f64fb0 0x558129f65430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.552+0000 7fe1e9cd9700 1 -- 172.21.15.37:0/994385275 learned_addr learned my addr 172.21.15.37:0/994385275 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:15.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.552+0000 7fe1eacdb700 1 -- 172.21.15.37:0/994385275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558129f61880 msgr2=0x558129f64a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:15.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.552+0000 7fe1eacdb700 1 --2- 172.21.15.37:0/994385275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558129f61880 0x558129f64a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.552+0000 7fe1eacdb700 1 -- 172.21.15.37:0/994385275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558129f60f60 msgr2=0x558129f67770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:15.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.552+0000 7fe1eacdb700 1 --2- 172.21.15.37:0/994385275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558129f60f60 0x558129f67770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.552+0000 7fe1eacdb700 1 -- 172.21.15.37:0/994385275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558129e72750 con 0x558129f64fb0 2024-06-10T01:08:15.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.552+0000 7fe1eacdb700 1 --2- 172.21.15.37:0/994385275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558129f64fb0 0x558129f65430 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc004320 tx=0x7fe1dc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e2b59fca7c7c20 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:15.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.552+0000 7fe1cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:08:15.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.553+0000 7fe1cdffb700 1 -- 172.21.15.37:0/994385275 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc027020 con 0x558129f64fb0 2024-06-10T01:08:15.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.553+0000 7fe1cdffb700 1 -- 172.21.15.37:0/994385275 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fe1dc007870 con 0x558129f64fb0 2024-06-10T01:08:15.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.553+0000 7fe1f6fc1b80 1 -- 172.21.15.37:0/994385275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1e0005550 con 0x558129f64fb0 2024-06-10T01:08:15.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.553+0000 7fe1cdffb700 1 -- 172.21.15.37:0/994385275 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe1dc032050 con 0x558129f64fb0 2024-06-10T01:08:15.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.553+0000 7fe1f6fc1b80 10 client.17427 fetch_fsmap learned FSMap version 31 2024-06-10T01:08:15.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.553+0000 7fe1f6fc1b80 1 -- 172.21.15.37:0/994385275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558129f72ce0 con 0x558129f64fb0 2024-06-10T01:08:15.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.554+0000 7fe1cdffb700 1 -- 172.21.15.37:0/994385275 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1e0005550 con 0x558129f64fb0 2024-06-10T01:08:15.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.554+0000 7fe1cdffb700 1 -- 172.21.15.37:0/994385275 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fe1dc023c40 con 0x558129f64fb0 2024-06-10T01:08:15.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.554+0000 7fe1f6fc1b80 10 client.17427 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:08:15.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.554+0000 7fe1f6fc1b80 10 client.17427 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-06-10T01:08:15.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.554+0000 7fe1f6fc1b80 20 client.17427 populate_metadata read hostname 'smithi037' 2024-06-10T01:08:15.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.554+0000 7fe1f6fc1b80 1 --2- 172.21.15.37:0/994385275 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x558129f75370 0x558129f777c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.554+0000 7fe1f6fc1b80 4 client.17427 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:08:15.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.554+0000 7fe1f6fc1b80 1 -- 172.21.15.37:0/994385275 --> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558129eb9d50 con 0x558129f75370 2024-06-10T01:08:15.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.555+0000 7fe1ea4da700 1 --2- 172.21.15.37:0/994385275 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x558129f75370 0x558129f777c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.555+0000 7fe1ea4da700 1 --2- 172.21.15.37:0/994385275 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x558129f75370 0x558129f777c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=11d949ccd9939038 server_cookie=c2a23302c3005e5b in_seq=0 out_seq=0 2024-06-10T01:08:15.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.555+0000 7fe1cdffb700 10 client.17427 ms_handle_connect on v2:172.21.15.94:6838/946269331 2024-06-10T01:08:15.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.557+0000 7fe1cdffb700 1 -- 172.21.15.37:0/994385275 <== mds.1 v2:172.21.15.94:6838/946269331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558129eb9d50 con 0x558129f75370 2024-06-10T01:08:15.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.557+0000 7fe1cdffb700 10 client.17427 handle_command_reply: tid=0 2024-06-10T01:08:15.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.557+0000 7fe1f6fc1b80 1 client.17427 shutdown 2024-06-10T01:08:15.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.557+0000 7fe1f6fc1b80 1 -- 172.21.15.37:0/994385275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558129f64fb0 msgr2=0x558129f65430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:15.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.558+0000 7fe1f6fc1b80 1 --2- 172.21.15.37:0/994385275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558129f64fb0 0x558129f65430 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc004320 tx=0x7fe1dc004ad0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.558+0000 7fe1f6fc1b80 1 -- 172.21.15.37:0/994385275 shutdown_connections 2024-06-10T01:08:15.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.558+0000 7fe1f6fc1b80 1 --2- 172.21.15.37:0/994385275 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x558129f75370 0x558129f777c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.558+0000 7fe1f6fc1b80 1 --2- 172.21.15.37:0/994385275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558129f64fb0 0x558129f65430 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.558+0000 7fe1f6fc1b80 1 --2- 172.21.15.37:0/994385275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558129f60f60 0x558129f67770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.558+0000 7fe1f6fc1b80 1 --2- 172.21.15.37:0/994385275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558129f61880 0x558129f64a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.558+0000 7fe1f6fc1b80 1 -- 172.21.15.37:0/994385275 >> 172.21.15.37:0/994385275 conn(0x558129e88080 msgr2=0x558129f55700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:15.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.558+0000 7fe1cdffb700 0 client.17427 ms_handle_reset on v2:172.21.15.94:6838/946269331 2024-06-10T01:08:15.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.558+0000 7fe1f6fc1b80 1 -- 172.21.15.37:0/994385275 shutdown_connections 2024-06-10T01:08:15.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.558+0000 7fe1f6fc1b80 1 -- 172.21.15.37:0/994385275 wait complete. 2024-06-10T01:08:15.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.558+0000 7fe1f6fc1b80 20 client.17427 trim_cache size 0 max 16384 2024-06-10T01:08:15.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.575+0000 7fe1f6fc1b80 1 Processor -- start 2024-06-10T01:08:15.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.575+0000 7fe1f6fc1b80 1 -- start start 2024-06-10T01:08:15.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.575+0000 7fe1f6fc1b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558129f60f60 0x558129f5b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.575+0000 7fe1f6fc1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558129f61880 0x558129f669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.575+0000 7fe1f6fc1b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558129f64fb0 0x558129f55e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.575+0000 7fe1f6fc1b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e41b73c0 con 0x558129f64fb0 2024-06-10T01:08:15.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.575+0000 7fe1f6fc1b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e41b4c50 con 0x558129f60f60 2024-06-10T01:08:15.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.576+0000 7fe1f6fc1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e41b2aa0 con 0x558129f61880 2024-06-10T01:08:15.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.576+0000 7fe1eacdb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558129f64fb0 0x558129f55e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.576+0000 7fe1eacdb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558129f64fb0 0x558129f55e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49040/0 (socket says 172.21.15.37:49040) 2024-06-10T01:08:15.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.576+0000 7fe1eacdb700 1 -- 172.21.15.37:0/915577205 learned_addr learned my addr 172.21.15.37:0/915577205 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:15.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.576+0000 7fe1e9cd9700 1 --2- 172.21.15.37:0/915577205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558129f61880 0x558129f669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.576+0000 7fe1ea4da700 1 --2- 172.21.15.37:0/915577205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558129f60f60 0x558129f5b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.576+0000 7fe1eacdb700 1 -- 172.21.15.37:0/915577205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558129f61880 msgr2=0x558129f669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:15.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.576+0000 7fe1eacdb700 1 --2- 172.21.15.37:0/915577205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558129f61880 0x558129f669b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.576+0000 7fe1eacdb700 1 -- 172.21.15.37:0/915577205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558129f60f60 msgr2=0x558129f5b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:15.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.576+0000 7fe1eacdb700 1 --2- 172.21.15.37:0/915577205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558129f60f60 0x558129f5b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.576+0000 7fe1eacdb700 1 -- 172.21.15.37:0/915577205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558129f61590 con 0x558129f64fb0 2024-06-10T01:08:15.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.576+0000 7fe1eacdb700 1 --2- 172.21.15.37:0/915577205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558129f64fb0 0x558129f55e80 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc022470 tx=0x7fe1dc004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36b498023a37a69 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:15.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.576+0000 7fe1cffff700 1 -- 172.21.15.37:0/915577205 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc027040 con 0x558129f64fb0 2024-06-10T01:08:15.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.577+0000 7fe1cffff700 1 -- 172.21.15.37:0/915577205 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1dc01c030 con 0x558129f64fb0 2024-06-10T01:08:15.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.577+0000 7fe1cffff700 1 -- 172.21.15.37:0/915577205 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc0076c0 con 0x558129f64fb0 2024-06-10T01:08:15.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.577+0000 7fe1f6fc1b80 1 -- 172.21.15.37:0/915577205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558129f64fb0 msgr2=0x558129f55e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:15.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.577+0000 7fe1f6fc1b80 1 --2- 172.21.15.37:0/915577205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558129f64fb0 0x558129f55e80 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc022470 tx=0x7fe1dc004980 comp rx=0 tx=0).stop 2024-06-10T01:08:15.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.577+0000 7fe1f6fc1b80 1 -- 172.21.15.37:0/915577205 shutdown_connections 2024-06-10T01:08:15.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.577+0000 7fe1f6fc1b80 1 --2- 172.21.15.37:0/915577205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558129f64fb0 0x558129f55e80 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.577+0000 7fe1f6fc1b80 1 --2- 172.21.15.37:0/915577205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558129f60f60 0x558129f5b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.577+0000 7fe1f6fc1b80 1 --2- 172.21.15.37:0/915577205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558129f61880 0x558129f669b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.577+0000 7fe1f6fc1b80 1 -- 172.21.15.37:0/915577205 >> 172.21.15.37:0/915577205 conn(0x558129e88080 msgr2=0x558129f662a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:15.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.577+0000 7fe1f6fc1b80 1 -- 172.21.15.37:0/915577205 shutdown_connections 2024-06-10T01:08:15.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.577+0000 7fe1f6fc1b80 1 -- 172.21.15.37:0/915577205 wait complete. 2024-06-10T01:08:15.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.578+0000 7fe1f6fc1b80 1 -- start start 2024-06-10T01:08:15.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.578+0000 7fe1f6fc1b80 1 Processor -- start 2024-06-10T01:08:15.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.578+0000 7fe1f6fc1b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558129f60f60 0x558129f6d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.578+0000 7fe1f6fc1b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558129f61880 0x558129f7c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.578+0000 7fe1f6fc1b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558129f64fb0 0x558129f7cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.578+0000 7fe1f6fc1b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e41b3110 con 0x558129f60f60 2024-06-10T01:08:15.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.578+0000 7fe1f6fc1b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e41b4580 con 0x558129f61880 2024-06-10T01:08:15.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.578+0000 7fe1f6fc1b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e41b37e0 con 0x558129f64fb0 2024-06-10T01:08:15.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.578+0000 7fe1f6fc1b80 10 client.0 _ll_register_callbacks cb 0x558129e425b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:15.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.578+0000 7fe1ea4da700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558129f60f60 0x558129f6d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.578+0000 7fe1ea4da700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558129f60f60 0x558129f6d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49048/0 (socket says 172.21.15.37:49048) 2024-06-10T01:08:15.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.578+0000 7fe1ea4da700 1 -- 172.21.15.37:0/1431930169 learned_addr learned my addr 172.21.15.37:0/1431930169 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:15.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.578+0000 7fe1e9cd9700 1 --2- 172.21.15.37:0/1431930169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558129f61880 0x558129f7c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.578+0000 7fe1eacdb700 1 --2- 172.21.15.37:0/1431930169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558129f64fb0 0x558129f7cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.578+0000 7fe1ea4da700 1 -- 172.21.15.37:0/1431930169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558129f64fb0 msgr2=0x558129f7cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:15.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.578+0000 7fe1ea4da700 1 --2- 172.21.15.37:0/1431930169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558129f64fb0 0x558129f7cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.578+0000 7fe1ea4da700 1 -- 172.21.15.37:0/1431930169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558129f61880 msgr2=0x558129f7c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:15.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.578+0000 7fe1ea4da700 1 --2- 172.21.15.37:0/1431930169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558129f61880 0x558129f7c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.578+0000 7fe1ea4da700 1 -- 172.21.15.37:0/1431930169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558129f7d270 con 0x558129f60f60 2024-06-10T01:08:15.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.578+0000 7fe1e9cd9700 1 --2- 172.21.15.37:0/1431930169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558129f61880 0x558129f7c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:15.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.579+0000 7fe1ea4da700 1 --2- 172.21.15.37:0/1431930169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558129f60f60 0x558129f6d990 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0003ad0 tx=0x7fe1e0009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39b71f07640cd09 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:15.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.579+0000 7fe1cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:08:15.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.579+0000 7fe1cdffb700 1 -- 172.21.15.37:0/1431930169 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0003680 con 0x558129f60f60 2024-06-10T01:08:15.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.579+0000 7fe1cdffb700 1 -- 172.21.15.37:0/1431930169 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fe1e0003820 con 0x558129f60f60 2024-06-10T01:08:15.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.579+0000 7fe1f6fc1b80 1 -- 172.21.15.37:0/1431930169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1dc01c030 con 0x558129f60f60 2024-06-10T01:08:15.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.579+0000 7fe1cdffb700 1 -- 172.21.15.37:0/1431930169 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe1e003f050 con 0x558129f60f60 2024-06-10T01:08:15.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.579+0000 7fe1f6fc1b80 10 client.17433 fetch_fsmap learned FSMap version 31 2024-06-10T01:08:15.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.579+0000 7fe1f6fc1b80 1 -- 172.21.15.37:0/1431930169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558129fdfe40 con 0x558129f60f60 2024-06-10T01:08:15.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.579+0000 7fe1cdffb700 1 -- 172.21.15.37:0/1431930169 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1dc01c030 con 0x558129f60f60 2024-06-10T01:08:15.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.579+0000 7fe1cdffb700 1 -- 172.21.15.37:0/1431930169 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fe1e000f5f0 con 0x558129f60f60 2024-06-10T01:08:15.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.579+0000 7fe1f6fc1b80 10 client.17433 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:08:15.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.579+0000 7fe1f6fc1b80 10 client.17433 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-06-10T01:08:15.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.579+0000 7fe1f6fc1b80 20 client.17433 populate_metadata read hostname 'smithi037' 2024-06-10T01:08:15.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.580+0000 7fe1f6fc1b80 1 --2- 172.21.15.37:0/1431930169 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x558129fe00d0 0x558129fe24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:15.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.580+0000 7fe1f6fc1b80 4 client.17433 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:08:15.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.580+0000 7fe1f6fc1b80 1 -- 172.21.15.37:0/1431930169 --> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558129eb42d0 con 0x558129fe00d0 2024-06-10T01:08:15.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.580+0000 7fe1e9cd9700 1 --2- 172.21.15.37:0/1431930169 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x558129fe00d0 0x558129fe24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:15.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.580+0000 7fe1e9cd9700 1 --2- 172.21.15.37:0/1431930169 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x558129fe00d0 0x558129fe24c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f2537cecc51d2e8 server_cookie=f2f98a856e082675 in_seq=0 out_seq=0 2024-06-10T01:08:15.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.580+0000 7fe1cdffb700 10 client.17433 ms_handle_connect on v2:172.21.15.94:6838/946269331 2024-06-10T01:08:15.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.580+0000 7fe1cdffb700 1 -- 172.21.15.37:0/1431930169 <== mds.1 v2:172.21.15.94:6838/946269331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558129eb42d0 con 0x558129fe00d0 2024-06-10T01:08:15.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.580+0000 7fe1cdffb700 10 client.17433 handle_command_reply: tid=0 2024-06-10T01:08:15.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.580+0000 7fe1f6fc1b80 1 client.17433 shutdown 2024-06-10T01:08:15.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.581+0000 7fe1f6fc1b80 1 -- 172.21.15.37:0/1431930169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558129f60f60 msgr2=0x558129f6d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:15.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.581+0000 7fe1f6fc1b80 1 --2- 172.21.15.37:0/1431930169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558129f60f60 0x558129f6d990 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0003ad0 tx=0x7fe1e0009360 comp rx=0 tx=0).stop 2024-06-10T01:08:15.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.581+0000 7fe1f6fc1b80 1 -- 172.21.15.37:0/1431930169 shutdown_connections 2024-06-10T01:08:15.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.581+0000 7fe1f6fc1b80 1 --2- 172.21.15.37:0/1431930169 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x558129fe00d0 0x558129fe24c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.581+0000 7fe1f6fc1b80 1 --2- 172.21.15.37:0/1431930169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558129f64fb0 0x558129f7cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.581+0000 7fe1f6fc1b80 1 --2- 172.21.15.37:0/1431930169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558129f60f60 0x558129f6d990 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.581+0000 7fe1cdffb700 0 client.17433 ms_handle_reset on v2:172.21.15.94:6838/946269331 2024-06-10T01:08:15.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.581+0000 7fe1f6fc1b80 1 --2- 172.21.15.37:0/1431930169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558129f61880 0x558129f7c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.581+0000 7fe1f6fc1b80 1 -- 172.21.15.37:0/1431930169 >> 172.21.15.37:0/1431930169 conn(0x558129e88080 msgr2=0x558129f60440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:15.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.581+0000 7fe1f6fc1b80 1 -- 172.21.15.37:0/1431930169 shutdown_connections 2024-06-10T01:08:15.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.581+0000 7fe1f6fc1b80 1 -- 172.21.15.37:0/1431930169 wait complete. 2024-06-10T01:08:15.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.581+0000 7fe1f6fc1b80 20 client.17433 trim_cache size 0 max 16384 2024-06-10T01:08:15.584 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T01:08:15.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.581+0000 7fe1ec73e700 1 -- 172.21.15.37:0/1157530089 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe1d00a2e90 msgr2=0x7fe1d00a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:15.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.581+0000 7fe1ec73e700 1 --2- 172.21.15.37:0/1157530089 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe1d00a2e90 0x7fe1d00a5350 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009fd0 tx=0x7fe1dc009050 comp rx=0 tx=0).stop 2024-06-10T01:08:15.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.581+0000 7fe1ec73e700 1 -- 172.21.15.37:0/1157530089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1e406a740 msgr2=0x7fe1e41112f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:15.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.581+0000 7fe1ec73e700 1 --2- 172.21.15.37:0/1157530089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1e406a740 0x7fe1e41112f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe1d400e970 tx=0x7fe1d400ec80 comp rx=0 tx=0).stop 2024-06-10T01:08:15.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.581+0000 7fe1ec73e700 1 -- 172.21.15.37:0/1157530089 shutdown_connections 2024-06-10T01:08:15.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.582+0000 7fe1ec73e700 1 --2- 172.21.15.37:0/1157530089 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe1d00a2e90 0x7fe1d00a5350 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.582+0000 7fe1ec73e700 1 --2- 172.21.15.37:0/1157530089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1e4100e30 0x7fe1e4111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.582+0000 7fe1ec73e700 1 --2- 172.21.15.37:0/1157530089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e406a2d0 0x7fe1e4110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.582+0000 7fe1ec73e700 1 --2- 172.21.15.37:0/1157530089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1e406a740 0x7fe1e41112f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:15.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.582+0000 7fe1ec73e700 1 -- 172.21.15.37:0/1157530089 >> 172.21.15.37:0/1157530089 conn(0x7fe1e4076730 msgr2=0x7fe1e4102720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:15.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.582+0000 7fe1ec73e700 1 -- 172.21.15.37:0/1157530089 shutdown_connections 2024-06-10T01:08:15.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:15.582+0000 7fe1ec73e700 1 -- 172.21.15.37:0/1157530089 wait complete. 2024-06-10T01:08:16.270 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:08:16.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:16 smithi167 ceph-mon[36917]: pgmap v778: 129 pgs: 129 active+clean; 6.8 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 665 op/s 2024-06-10T01:08:16.597 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:16 smithi037 ceph-mon[32192]: pgmap v778: 129 pgs: 129 active+clean; 6.8 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 665 op/s 2024-06-10T01:08:16.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:16 smithi094 ceph-mon[36708]: pgmap v778: 129 pgs: 129 active+clean; 6.8 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 665 op/s 2024-06-10T01:08:16.624 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:08:16.712 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:08:17.448 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:08:18.384 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:18 smithi037 ceph-mon[32192]: pgmap v779: 129 pgs: 129 active+clean; 8.2 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 892 KiB/s wr, 555 op/s 2024-06-10T01:08:18.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:18 smithi167 ceph-mon[36917]: pgmap v779: 129 pgs: 129 active+clean; 8.2 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 892 KiB/s wr, 555 op/s 2024-06-10T01:08:18.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:18 smithi094 ceph-mon[36708]: pgmap v779: 129 pgs: 129 active+clean; 8.2 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 892 KiB/s wr, 555 op/s 2024-06-10T01:08:18.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.882+0000 7fc0dc99b700 1 -- 172.21.15.37:0/149981329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0d8108520 msgr2=0x7fc0d8108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:18.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.882+0000 7fc0dc99b700 1 --2- 172.21.15.37:0/149981329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0d8108520 0x7fc0d8108900 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8009990 tx=0x7fc0c8009ca0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.882+0000 7fc0dc99b700 1 -- 172.21.15.37:0/149981329 shutdown_connections 2024-06-10T01:08:18.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.882+0000 7fc0dc99b700 1 --2- 172.21.15.37:0/149981329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d8102ed0 0x7fc0d810f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.882+0000 7fc0dc99b700 1 --2- 172.21.15.37:0/149981329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0d8108520 0x7fc0d8108900 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.882+0000 7fc0dc99b700 1 --2- 172.21.15.37:0/149981329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0d8102510 0x7fc0d8102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.882+0000 7fc0dc99b700 1 -- 172.21.15.37:0/149981329 >> 172.21.15.37:0/149981329 conn(0x7fc0d8079640 msgr2=0x7fc0d8079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:18.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.882+0000 7fc0dc99b700 1 -- 172.21.15.37:0/149981329 shutdown_connections 2024-06-10T01:08:18.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.882+0000 7fc0dc99b700 1 -- 172.21.15.37:0/149981329 wait complete. 2024-06-10T01:08:18.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.883+0000 7fc0dc99b700 1 Processor -- start 2024-06-10T01:08:18.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.883+0000 7fc0dc99b700 1 -- start start 2024-06-10T01:08:18.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.884+0000 7fc0dc99b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0d8102510 0x7fc0d819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:18.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.884+0000 7fc0dc99b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0d8102ed0 0x7fc0d819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:18.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.884+0000 7fc0dc99b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d8108520 0x7fc0d8198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:18.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.884+0000 7fc0dc99b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d81127b0 con 0x7fc0d8102510 2024-06-10T01:08:18.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.884+0000 7fc0dc99b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d8112630 con 0x7fc0d8102ed0 2024-06-10T01:08:18.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.884+0000 7fc0dc99b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d8112930 con 0x7fc0d8108520 2024-06-10T01:08:18.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.884+0000 7fc0d659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0d8102510 0x7fc0d819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:18.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.884+0000 7fc0d659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0d8102510 0x7fc0d819e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49074/0 (socket says 172.21.15.37:49074) 2024-06-10T01:08:18.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.884+0000 7fc0d659c700 1 -- 172.21.15.37:0/1335383460 learned_addr learned my addr 172.21.15.37:0/1335383460 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:18.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.884+0000 7fc0d6d9d700 1 --2- 172.21.15.37:0/1335383460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d8108520 0x7fc0d8198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:18.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.884+0000 7fc0d5d9b700 1 --2- 172.21.15.37:0/1335383460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0d8102ed0 0x7fc0d819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:18.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.884+0000 7fc0d659c700 1 -- 172.21.15.37:0/1335383460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d8108520 msgr2=0x7fc0d8198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:18.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.884+0000 7fc0d659c700 1 --2- 172.21.15.37:0/1335383460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d8108520 0x7fc0d8198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.884+0000 7fc0d659c700 1 -- 172.21.15.37:0/1335383460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0d8102ed0 msgr2=0x7fc0d819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:18.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.884+0000 7fc0d659c700 1 --2- 172.21.15.37:0/1335383460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0d8102ed0 0x7fc0d819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:18.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.884+0000 7fc0d659c700 1 -- 172.21.15.37:0/1335383460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d8199190 con 0x7fc0d8102510 2024-06-10T01:08:18.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.884+0000 7fc0d5d9b700 1 --2- 172.21.15.37:0/1335383460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0d8102ed0 0x7fc0d819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:08:18.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.884+0000 7fc0d659c700 1 --2- 172.21.15.37:0/1335383460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0d8102510 0x7fc0d819e870 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8000c00 tx=0x7fc0c80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:18.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.885+0000 7fc0c77fe700 1 -- 172.21.15.37:0/1335383460 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8027070 con 0x7fc0d8102510 2024-06-10T01:08:18.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.885+0000 7fc0dc99b700 1 -- 172.21.15.37:0/1335383460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d8199420 con 0x7fc0d8102510 2024-06-10T01:08:18.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.885+0000 7fc0c77fe700 1 -- 172.21.15.37:0/1335383460 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0c8003780 con 0x7fc0d8102510 2024-06-10T01:08:18.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.885+0000 7fc0c77fe700 1 -- 172.21.15.37:0/1335383460 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8022dd0 con 0x7fc0d8102510 2024-06-10T01:08:18.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.885+0000 7fc0dc99b700 1 -- 172.21.15.37:0/1335383460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d81a5740 con 0x7fc0d8102510 2024-06-10T01:08:18.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.886+0000 7fc0dc99b700 1 -- 172.21.15.37:0/1335383460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0d8069180 con 0x7fc0d8102510 2024-06-10T01:08:18.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.887+0000 7fc0c77fe700 1 -- 172.21.15.37:0/1335383460 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc0c802a070 con 0x7fc0d8102510 2024-06-10T01:08:18.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.888+0000 7fc0c77fe700 1 --2- 172.21.15.37:0/1335383460 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc0c00a2c20 0x7fc0c00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:18.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.888+0000 7fc0c77fe700 1 -- 172.21.15.37:0/1335383460 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fc0c80d04b0 con 0x7fc0d8102510 2024-06-10T01:08:18.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.888+0000 7fc0d5d9b700 1 --2- 172.21.15.37:0/1335383460 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc0c00a2c20 0x7fc0c00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:18.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.889+0000 7fc0d5d9b700 1 --2- 172.21.15.37:0/1335383460 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc0c00a2c20 0x7fc0c00a50e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8079c90 tx=0x7fc0cc00b5f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:18.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:18.892+0000 7fc0c77fe700 1 -- 172.21.15.37:0/1335383460 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc0c8013070 con 0x7fc0d8102510 2024-06-10T01:08:19.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:19.146+0000 7fc0dc99b700 1 -- 172.21.15.37:0/1335383460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0d8199f60 con 0x7fc0d8102510 2024-06-10T01:08:19.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:19.147+0000 7fc0c77fe700 1 -- 172.21.15.37:0/1335383460 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11868 (secure 0 0 0) 0x7fc0c809d140 con 0x7fc0d8102510 2024-06-10T01:08:19.143 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:08:19.143 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":26554,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2107277223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2107277223},{"type":"v1","addr":"172.21.15.167:6835","nonce":2107277223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T01:06:09.604504+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24472,24451,24458]},"id":1}]} 2024-06-10T01:08:19.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:19.149+0000 7fc0dc99b700 1 -- 172.21.15.37:0/1335383460 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc0c00a2c20 msgr2=0x7fc0c00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:19.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:19.149+0000 7fc0dc99b700 1 --2- 172.21.15.37:0/1335383460 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc0c00a2c20 0x7fc0c00a50e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8079c90 tx=0x7fc0cc00b5f0 comp rx=0 tx=0).stop 2024-06-10T01:08:19.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:19.149+0000 7fc0dc99b700 1 -- 172.21.15.37:0/1335383460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0d8102510 msgr2=0x7fc0d819e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:19.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:19.149+0000 7fc0dc99b700 1 --2- 172.21.15.37:0/1335383460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0d8102510 0x7fc0d819e870 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8000c00 tx=0x7fc0c80047b0 comp rx=0 tx=0).stop 2024-06-10T01:08:19.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:19.149+0000 7fc0dc99b700 1 -- 172.21.15.37:0/1335383460 shutdown_connections 2024-06-10T01:08:19.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:19.149+0000 7fc0dc99b700 1 --2- 172.21.15.37:0/1335383460 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc0c00a2c20 0x7fc0c00a50e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:19.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:19.149+0000 7fc0dc99b700 1 --2- 172.21.15.37:0/1335383460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d8108520 0x7fc0d8198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:19.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:19.149+0000 7fc0dc99b700 1 --2- 172.21.15.37:0/1335383460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0d8102510 0x7fc0d819e870 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:19.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:19.149+0000 7fc0dc99b700 1 --2- 172.21.15.37:0/1335383460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0d8102ed0 0x7fc0d819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:19.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:19.149+0000 7fc0dc99b700 1 -- 172.21.15.37:0/1335383460 >> 172.21.15.37:0/1335383460 conn(0x7fc0d8079640 msgr2=0x7fc0d81011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:19.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:19.149+0000 7fc0dc99b700 1 -- 172.21.15.37:0/1335383460 shutdown_connections 2024-06-10T01:08:19.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:19.149+0000 7fc0dc99b700 1 -- 172.21.15.37:0/1335383460 wait complete. 2024-06-10T01:08:19.147 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 31 2024-06-10T01:08:19.285 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:19 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1335383460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:08:19.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:19 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1335383460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:08:19.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:19 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1335383460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:08:19.916 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24472 quiesce db --set-id 96a2bbd8 --release --await' 2024-06-10T01:08:19.916 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24472 quiesce db --set-id 96a2bbd8 --release --await 2024-06-10T01:08:20.294 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:08:20.496 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:20 smithi037 ceph-mon[32192]: pgmap v780: 129 pgs: 129 active+clean; 8.7 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 932 KiB/s wr, 602 op/s 2024-06-10T01:08:20.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:20 smithi167 ceph-mon[36917]: pgmap v780: 129 pgs: 129 active+clean; 8.7 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 932 KiB/s wr, 602 op/s 2024-06-10T01:08:20.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:20 smithi094 ceph-mon[36708]: pgmap v780: 129 pgs: 129 active+clean; 8.7 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 932 KiB/s wr, 602 op/s 2024-06-10T01:08:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.333+0000 7f561f942640 1 -- 172.21.15.37:0/2786202382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56181024e0 msgr2=0x7f56181048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.333+0000 7f561f942640 1 --2- 172.21.15.37:0/2786202382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56181024e0 0x7f56181048d0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f5614009900 tx=0x7f561402a410 comp rx=0 tx=0).stop 2024-06-10T01:08:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.334+0000 7f561f942640 1 -- 172.21.15.37:0/2786202382 shutdown_connections 2024-06-10T01:08:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.334+0000 7f561f942640 1 --2- 172.21.15.37:0/2786202382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5618107740 0x7f5618109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.334+0000 7f561f942640 1 --2- 172.21.15.37:0/2786202382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5618104e10 0x7f5618107200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.334+0000 7f561f942640 1 --2- 172.21.15.37:0/2786202382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56181024e0 0x7f56181048d0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.334+0000 7f561f942640 1 -- 172.21.15.37:0/2786202382 >> 172.21.15.37:0/2786202382 conn(0x7f56180fbe70 msgr2=0x7f56180fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.334+0000 7f561f942640 1 -- 172.21.15.37:0/2786202382 shutdown_connections 2024-06-10T01:08:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.334+0000 7f561f942640 1 -- 172.21.15.37:0/2786202382 wait complete. 2024-06-10T01:08:21.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.334+0000 7f561f942640 1 Processor -- start 2024-06-10T01:08:21.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.334+0000 7f561f942640 1 -- start start 2024-06-10T01:08:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.335+0000 7f561f942640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56181024e0 0x7f5618197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.335+0000 7f561f942640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5618104e10 0x7f56181981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.335+0000 7f561f942640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5618107740 0x7f561819f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.335+0000 7f561f942640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f561810d6a0 con 0x7f56181024e0 2024-06-10T01:08:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.335+0000 7f561f942640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f561810d520 con 0x7f5618107740 2024-06-10T01:08:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.335+0000 7f561f942640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f561810d820 con 0x7f5618104e10 2024-06-10T01:08:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.335+0000 7f561d6de640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56181024e0 0x7f5618197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.335+0000 7f561d6de640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56181024e0 0x7f5618197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49084/0 (socket says 172.21.15.37:49084) 2024-06-10T01:08:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.335+0000 7f561d6de640 1 -- 172.21.15.37:0/515440112 learned_addr learned my addr 172.21.15.37:0/515440112 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.335+0000 7f561cedd640 1 --2- 172.21.15.37:0/515440112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5618104e10 0x7f56181981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.335+0000 7f561dedf640 1 --2- 172.21.15.37:0/515440112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5618107740 0x7f561819f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.335+0000 7f561d6de640 1 -- 172.21.15.37:0/515440112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5618104e10 msgr2=0x7f56181981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.335+0000 7f561d6de640 1 --2- 172.21.15.37:0/515440112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5618104e10 0x7f56181981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.335+0000 7f561d6de640 1 -- 172.21.15.37:0/515440112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5618107740 msgr2=0x7f561819f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.335+0000 7f561d6de640 1 --2- 172.21.15.37:0/515440112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5618107740 0x7f561819f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.335+0000 7f561d6de640 1 -- 172.21.15.37:0/515440112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561819f8d0 con 0x7f56181024e0 2024-06-10T01:08:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.335+0000 7f561d6de640 1 --2- 172.21.15.37:0/515440112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56181024e0 0x7f5618197c90 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f5614009230 tx=0x7f5614031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.336+0000 7f560e7fc640 1 -- 172.21.15.37:0/515440112 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561403f030 con 0x7f56181024e0 2024-06-10T01:08:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.336+0000 7f561f942640 1 -- 172.21.15.37:0/515440112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f561819fb60 con 0x7f56181024e0 2024-06-10T01:08:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.336+0000 7f560e7fc640 1 -- 172.21.15.37:0/515440112 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5614003ad0 con 0x7f56181024e0 2024-06-10T01:08:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.336+0000 7f560e7fc640 1 -- 172.21.15.37:0/515440112 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5614035070 con 0x7f56181024e0 2024-06-10T01:08:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.336+0000 7f561f942640 1 -- 172.21.15.37:0/515440112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56181a00a0 con 0x7f56181024e0 2024-06-10T01:08:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.338+0000 7f560e7fc640 1 -- 172.21.15.37:0/515440112 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5614042030 con 0x7f56181024e0 2024-06-10T01:08:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.339+0000 7f562264e740 1 Processor -- start 2024-06-10T01:08:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.339+0000 7f562264e740 1 -- start start 2024-06-10T01:08:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.339+0000 7f562264e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1a5f58420 0x55c1a5f58840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.339+0000 7f562264e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c1a5f58e10 0x55c1a5f5d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.339+0000 7f562264e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c1a5f5df80 0x55c1a5f60410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.339+0000 7f562264e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c1a5eb74f0 con 0x55c1a5f58420 2024-06-10T01:08:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.339+0000 7f562264e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c1a5e55870 con 0x55c1a5f58e10 2024-06-10T01:08:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.339+0000 7f562264e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c1a5eb34b0 con 0x55c1a5f5df80 2024-06-10T01:08:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.339+0000 7f561dedf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1a5f58420 0x55c1a5f58840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.339+0000 7f561dedf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1a5f58420 0x55c1a5f58840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49090/0 (socket says 172.21.15.37:49090) 2024-06-10T01:08:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.339+0000 7f561dedf640 1 -- 172.21.15.37:0/4237328211 learned_addr learned my addr 172.21.15.37:0/4237328211 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.339+0000 7f561cedd640 1 --2- 172.21.15.37:0/4237328211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c1a5f5df80 0x55c1a5f60410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.339+0000 7f561d6de640 1 --2- 172.21.15.37:0/4237328211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c1a5f58e10 0x55c1a5f5d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.339+0000 7f561dedf640 1 -- 172.21.15.37:0/4237328211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c1a5f5df80 msgr2=0x55c1a5f60410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.339+0000 7f561dedf640 1 --2- 172.21.15.37:0/4237328211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c1a5f5df80 0x55c1a5f60410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.339+0000 7f561dedf640 1 -- 172.21.15.37:0/4237328211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c1a5f58e10 msgr2=0x55c1a5f5d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.339+0000 7f561dedf640 1 --2- 172.21.15.37:0/4237328211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c1a5f58e10 0x55c1a5f5d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.339+0000 7f561dedf640 1 -- 172.21.15.37:0/4237328211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1a5e719b0 con 0x55c1a5f58420 2024-06-10T01:08:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.340+0000 7f560e7fc640 1 --2- 172.21.15.37:0/515440112 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f56040a2c30 0x7f56040a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.340+0000 7f561dedf640 1 --2- 172.21.15.37:0/4237328211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1a5f58420 0x55c1a5f58840 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f5610009fc0 tx=0x7f5610010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48afb2fb6b61c05b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.340+0000 7f561d6de640 1 --2- 172.21.15.37:0/515440112 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f56040a2c30 0x7f56040a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.340+0000 7f56037fe640 1 -- 172.21.15.37:0/4237328211 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610003b70 con 0x55c1a5f58420 2024-06-10T01:08:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.340+0000 7f56037fe640 1 -- 172.21.15.37:0/4237328211 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5610003d10 con 0x55c1a5f58420 2024-06-10T01:08:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.340+0000 7f560e7fc640 1 -- 172.21.15.37:0/515440112 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f56140ef6c0 con 0x7f56181024e0 2024-06-10T01:08:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.340+0000 7f56037fe640 1 -- 172.21.15.37:0/4237328211 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610011aa0 con 0x55c1a5f58420 2024-06-10T01:08:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.341+0000 7f562264e740 1 -- 172.21.15.37:0/4237328211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1a5f58420 msgr2=0x55c1a5f58840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.341+0000 7f562264e740 1 --2- 172.21.15.37:0/4237328211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1a5f58420 0x55c1a5f58840 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f5610009fc0 tx=0x7f5610010860 comp rx=0 tx=0).stop 2024-06-10T01:08:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.341+0000 7f561d6de640 1 --2- 172.21.15.37:0/515440112 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f56040a2c30 0x7f56040a50f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x55c1a5f59210 tx=0x7f5614009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.341+0000 7f562264e740 1 -- 172.21.15.37:0/4237328211 shutdown_connections 2024-06-10T01:08:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.341+0000 7f562264e740 1 --2- 172.21.15.37:0/4237328211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c1a5f5df80 0x55c1a5f60410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.341+0000 7f562264e740 1 --2- 172.21.15.37:0/4237328211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c1a5f58e10 0x55c1a5f5d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.341+0000 7f562264e740 1 --2- 172.21.15.37:0/4237328211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1a5f58420 0x55c1a5f58840 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.341+0000 7f562264e740 1 -- 172.21.15.37:0/4237328211 >> 172.21.15.37:0/4237328211 conn(0x55c1a5f4f680 msgr2=0x55c1a5f4fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.341+0000 7f562264e740 1 -- 172.21.15.37:0/4237328211 shutdown_connections 2024-06-10T01:08:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.341+0000 7f562264e740 1 -- 172.21.15.37:0/4237328211 wait complete. 2024-06-10T01:08:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.341+0000 7f562264e740 1 -- start start 2024-06-10T01:08:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.341+0000 7f562264e740 1 Processor -- start 2024-06-10T01:08:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.342+0000 7f562264e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1a5f58420 0x55c1a5f62690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.342+0000 7f562264e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c1a5f58e10 0x55c1a5f62bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.342+0000 7f562264e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c1a5f5df80 0x55c1a5f69c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.342+0000 7f562264e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c1a5f60f30 con 0x55c1a5f58420 2024-06-10T01:08:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.342+0000 7f562264e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c1a5e40ae0 con 0x55c1a5f5df80 2024-06-10T01:08:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.342+0000 7f562264e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c1a5f610b0 con 0x55c1a5f58e10 2024-06-10T01:08:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.342+0000 7f562264e740 10 client.0 _ll_register_callbacks cb 0x55c1a5e799b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.342+0000 7f561dedf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1a5f58420 0x55c1a5f62690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.342+0000 7f561dedf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1a5f58420 0x55c1a5f62690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49104/0 (socket says 172.21.15.37:49104) 2024-06-10T01:08:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.342+0000 7f561dedf640 1 -- 172.21.15.37:0/1266827297 learned_addr learned my addr 172.21.15.37:0/1266827297 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.342+0000 7f561cedd640 1 --2- 172.21.15.37:0/1266827297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c1a5f58e10 0x55c1a5f62bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.342+0000 7f561dedf640 1 --2- 172.21.15.37:0/1266827297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c1a5f5df80 0x55c1a5f69c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.342+0000 7f561dedf640 1 -- 172.21.15.37:0/1266827297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c1a5f58e10 msgr2=0x55c1a5f62bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.342+0000 7f561dedf640 1 --2- 172.21.15.37:0/1266827297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c1a5f58e10 0x55c1a5f62bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.342+0000 7f561dedf640 1 -- 172.21.15.37:0/1266827297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c1a5f5df80 msgr2=0x55c1a5f69c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.342+0000 7f561dedf640 1 --2- 172.21.15.37:0/1266827297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c1a5f5df80 0x55c1a5f69c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.342+0000 7f561dedf640 1 -- 172.21.15.37:0/1266827297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1a5bcb210 con 0x55c1a5f58420 2024-06-10T01:08:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.342+0000 7f561dedf640 1 --2- 172.21.15.37:0/1266827297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1a5f58420 0x55c1a5f62690 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f5610009c10 tx=0x7f5610004900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=207b48357996a2ef server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.342+0000 7f5600ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:08:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.343+0000 7f5600ff9640 1 -- 172.21.15.37:0/1266827297 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56100119f0 con 0x55c1a5f58420 2024-06-10T01:08:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.343+0000 7f5600ff9640 1 -- 172.21.15.37:0/1266827297 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f5610023040 con 0x55c1a5f58420 2024-06-10T01:08:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.343+0000 7f562264e740 1 -- 172.21.15.37:0/1266827297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5610003d10 con 0x55c1a5f58420 2024-06-10T01:08:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.343+0000 7f5600ff9640 1 -- 172.21.15.37:0/1266827297 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5610011b90 con 0x55c1a5f58420 2024-06-10T01:08:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.343+0000 7f562264e740 10 client.17451 fetch_fsmap learned FSMap version 31 2024-06-10T01:08:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.343+0000 7f562264e740 1 -- 172.21.15.37:0/1266827297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c1a5c97620 con 0x55c1a5f58420 2024-06-10T01:08:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.343+0000 7f5600ff9640 1 -- 172.21.15.37:0/1266827297 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5610003d10 con 0x55c1a5f58420 2024-06-10T01:08:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.343+0000 7f5600ff9640 1 -- 172.21.15.37:0/1266827297 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f5610026070 con 0x55c1a5f58420 2024-06-10T01:08:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.343+0000 7f562264e740 10 client.17451 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:08:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.343+0000 7f562264e740 10 client.17451 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.343+0000 7f562264e740 20 client.17451 populate_metadata read hostname 'smithi037' 2024-06-10T01:08:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.343+0000 7f562264e740 1 --2- 172.21.15.37:0/1266827297 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c1a5f6e1d0 0x55c1a5f705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.343+0000 7f562264e740 4 client.17451 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:08:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.343+0000 7f562264e740 1 -- 172.21.15.37:0/1266827297 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1a5ecaf10 con 0x55c1a5f6e1d0 2024-06-10T01:08:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.344+0000 7f561cedd640 1 --2- 172.21.15.37:0/1266827297 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c1a5f6e1d0 0x55c1a5f705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.344+0000 7f5600ff9640 10 client.17451 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.344+0000 7f561cedd640 1 --2- 172.21.15.37:0/1266827297 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c1a5f6e1d0 0x55c1a5f705c0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fae61607c3cc49cf server_cookie=c42fe00c35804132 in_seq=0 out_seq=0 2024-06-10T01:08:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.345+0000 7f5600ff9640 1 -- 172.21.15.37:0/1266827297 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c1a5ecaf10 con 0x55c1a5f6e1d0 2024-06-10T01:08:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.345+0000 7f5600ff9640 10 client.17451 handle_command_reply: tid=0 2024-06-10T01:08:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.345+0000 7f562264e740 1 client.17451 shutdown 2024-06-10T01:08:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.345+0000 7f562264e740 1 -- 172.21.15.37:0/1266827297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1a5f58420 msgr2=0x55c1a5f62690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.345+0000 7f562264e740 1 --2- 172.21.15.37:0/1266827297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1a5f58420 0x55c1a5f62690 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f5610009c10 tx=0x7f5610004900 comp rx=0 tx=0).stop 2024-06-10T01:08:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.345+0000 7f562264e740 1 -- 172.21.15.37:0/1266827297 shutdown_connections 2024-06-10T01:08:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.345+0000 7f562264e740 1 --2- 172.21.15.37:0/1266827297 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c1a5f6e1d0 0x55c1a5f705c0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.345+0000 7f562264e740 1 --2- 172.21.15.37:0/1266827297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c1a5f5df80 0x55c1a5f69c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.345+0000 7f562264e740 1 --2- 172.21.15.37:0/1266827297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c1a5f58e10 0x55c1a5f62bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.345+0000 7f5600ff9640 0 client.17451 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.345+0000 7f562264e740 1 --2- 172.21.15.37:0/1266827297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1a5f58420 0x55c1a5f62690 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.345+0000 7f562264e740 1 -- 172.21.15.37:0/1266827297 >> 172.21.15.37:0/1266827297 conn(0x55c1a5f4f680 msgr2=0x55c1a5f4e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.345+0000 7f562264e740 1 -- 172.21.15.37:0/1266827297 shutdown_connections 2024-06-10T01:08:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.345+0000 7f562264e740 1 -- 172.21.15.37:0/1266827297 wait complete. 2024-06-10T01:08:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.345+0000 7f562264e740 20 client.17451 trim_cache size 0 max 16384 2024-06-10T01:08:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.361+0000 7f562264e740 1 Processor -- start 2024-06-10T01:08:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.361+0000 7f562264e740 1 -- start start 2024-06-10T01:08:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.361+0000 7f562264e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c1a5f58420 0x55c1a5f52670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.361+0000 7f562264e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1a5f58e10 0x55c1a5f69ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.361+0000 7f562264e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c1a5f5df80 0x55c1a5f52bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.361+0000 7f562264e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f56181aae60 con 0x55c1a5f58e10 2024-06-10T01:08:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.361+0000 7f562264e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f56181a8730 con 0x55c1a5f5df80 2024-06-10T01:08:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.361+0000 7f562264e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f56181a6580 con 0x55c1a5f58420 2024-06-10T01:08:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.361+0000 7f561cedd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1a5f58e10 0x55c1a5f69ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.361+0000 7f561cedd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1a5f58e10 0x55c1a5f69ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49120/0 (socket says 172.21.15.37:49120) 2024-06-10T01:08:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.361+0000 7f561cedd640 1 -- 172.21.15.37:0/805395550 learned_addr learned my addr 172.21.15.37:0/805395550 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.361+0000 7f561dedf640 1 --2- 172.21.15.37:0/805395550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c1a5f58420 0x55c1a5f52670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.361+0000 7f561dedf640 1 --2- 172.21.15.37:0/805395550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c1a5f5df80 0x55c1a5f52bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.361+0000 7f561cedd640 1 -- 172.21.15.37:0/805395550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c1a5f58420 msgr2=0x55c1a5f52670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.362+0000 7f561cedd640 1 --2- 172.21.15.37:0/805395550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c1a5f58420 0x55c1a5f52670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.362+0000 7f561cedd640 1 -- 172.21.15.37:0/805395550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c1a5f5df80 msgr2=0x55c1a5f52bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.362+0000 7f561cedd640 1 --2- 172.21.15.37:0/805395550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c1a5f5df80 0x55c1a5f52bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.362+0000 7f561cedd640 1 -- 172.21.15.37:0/805395550 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1a5f70c20 con 0x55c1a5f58e10 2024-06-10T01:08:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.362+0000 7f561dedf640 1 --2- 172.21.15.37:0/805395550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c1a5f5df80 0x55c1a5f52bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:08:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.362+0000 7f561cedd640 1 --2- 172.21.15.37:0/805395550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1a5f58e10 0x55c1a5f69ba0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f5608005940 tx=0x7f560800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47764eb432a74e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.362+0000 7f5602ffd640 1 -- 172.21.15.37:0/805395550 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5608015070 con 0x55c1a5f58e10 2024-06-10T01:08:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.362+0000 7f5602ffd640 1 -- 172.21.15.37:0/805395550 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5608003ea0 con 0x55c1a5f58e10 2024-06-10T01:08:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.362+0000 7f5602ffd640 1 -- 172.21.15.37:0/805395550 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560801d3b0 con 0x55c1a5f58e10 2024-06-10T01:08:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.363+0000 7f562264e740 1 -- 172.21.15.37:0/805395550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1a5f58e10 msgr2=0x55c1a5f69ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.363+0000 7f562264e740 1 --2- 172.21.15.37:0/805395550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1a5f58e10 0x55c1a5f69ba0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f5608005940 tx=0x7f560800be70 comp rx=0 tx=0).stop 2024-06-10T01:08:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.363+0000 7f562264e740 1 -- 172.21.15.37:0/805395550 shutdown_connections 2024-06-10T01:08:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.363+0000 7f562264e740 1 --2- 172.21.15.37:0/805395550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c1a5f5df80 0x55c1a5f52bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.363+0000 7f562264e740 1 --2- 172.21.15.37:0/805395550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1a5f58e10 0x55c1a5f69ba0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.363+0000 7f562264e740 1 --2- 172.21.15.37:0/805395550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c1a5f58420 0x55c1a5f52670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.363+0000 7f562264e740 1 -- 172.21.15.37:0/805395550 >> 172.21.15.37:0/805395550 conn(0x55c1a5f4f680 msgr2=0x55c1a5f61480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.363+0000 7f562264e740 1 -- 172.21.15.37:0/805395550 shutdown_connections 2024-06-10T01:08:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.363+0000 7f562264e740 1 -- 172.21.15.37:0/805395550 wait complete. 2024-06-10T01:08:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.363+0000 7f562264e740 1 -- start start 2024-06-10T01:08:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.363+0000 7f562264e740 1 Processor -- start 2024-06-10T01:08:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.363+0000 7f562264e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c1a5f58420 0x55c1a5f66c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.363+0000 7f562264e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1a5f58e10 0x55c1a5f67160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.363+0000 7f562264e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c1a5f5df80 0x55c1a5f68170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.363+0000 7f562264e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f56181a6bf0 con 0x55c1a5f58e10 2024-06-10T01:08:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.363+0000 7f562264e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f56181a8060 con 0x55c1a5f5df80 2024-06-10T01:08:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.364+0000 7f562264e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f56181a72c0 con 0x55c1a5f58420 2024-06-10T01:08:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.364+0000 7f562264e740 10 client.0 _ll_register_callbacks cb 0x7f5610009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.364+0000 7f561cedd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1a5f58e10 0x55c1a5f67160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.364+0000 7f561cedd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1a5f58e10 0x55c1a5f67160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49136/0 (socket says 172.21.15.37:49136) 2024-06-10T01:08:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.364+0000 7f561cedd640 1 -- 172.21.15.37:0/3164792787 learned_addr learned my addr 172.21.15.37:0/3164792787 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.364+0000 7f561dedf640 1 --2- 172.21.15.37:0/3164792787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c1a5f58420 0x55c1a5f66c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.364+0000 7f561dedf640 1 --2- 172.21.15.37:0/3164792787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c1a5f5df80 0x55c1a5f68170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.364+0000 7f561cedd640 1 -- 172.21.15.37:0/3164792787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c1a5f58420 msgr2=0x55c1a5f66c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.364+0000 7f561cedd640 1 --2- 172.21.15.37:0/3164792787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c1a5f58420 0x55c1a5f66c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.364+0000 7f561cedd640 1 -- 172.21.15.37:0/3164792787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c1a5f5df80 msgr2=0x55c1a5f68170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.364+0000 7f561cedd640 1 --2- 172.21.15.37:0/3164792787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c1a5f5df80 0x55c1a5f68170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.364+0000 7f561cedd640 1 -- 172.21.15.37:0/3164792787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1a5f59ae0 con 0x55c1a5f58e10 2024-06-10T01:08:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.364+0000 7f561cedd640 1 --2- 172.21.15.37:0/3164792787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1a5f58e10 0x55c1a5f67160 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f5608004140 tx=0x7f5608003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=605b963bb85b8ac8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.364+0000 7f5600ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:08:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.364+0000 7f5600ff9640 1 -- 172.21.15.37:0/3164792787 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560802c070 con 0x55c1a5f58e10 2024-06-10T01:08:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.364+0000 7f5600ff9640 1 -- 172.21.15.37:0/3164792787 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f560801d5b0 con 0x55c1a5f58e10 2024-06-10T01:08:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.364+0000 7f562264e740 1 -- 172.21.15.37:0/3164792787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5608003ea0 con 0x55c1a5f58e10 2024-06-10T01:08:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.364+0000 7f5600ff9640 1 -- 172.21.15.37:0/3164792787 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5608038050 con 0x55c1a5f58e10 2024-06-10T01:08:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.364+0000 7f562264e740 10 client.17457 fetch_fsmap learned FSMap version 31 2024-06-10T01:08:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.364+0000 7f562264e740 1 -- 172.21.15.37:0/3164792787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c1a5f4f4b0 con 0x55c1a5f58e10 2024-06-10T01:08:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.365+0000 7f5600ff9640 1 -- 172.21.15.37:0/3164792787 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5608003ea0 con 0x55c1a5f58e10 2024-06-10T01:08:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.365+0000 7f5600ff9640 1 -- 172.21.15.37:0/3164792787 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f5608005450 con 0x55c1a5f58e10 2024-06-10T01:08:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.365+0000 7f562264e740 10 client.17457 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:08:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.365+0000 7f562264e740 10 client.17457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.365+0000 7f562264e740 20 client.17457 populate_metadata read hostname 'smithi037' 2024-06-10T01:08:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.365+0000 7f562264e740 1 --2- 172.21.15.37:0/3164792787 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c1a5fd8e80 0x55c1a5fdb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.365+0000 7f562264e740 4 client.17457 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:08:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.365+0000 7f562264e740 1 -- 172.21.15.37:0/3164792787 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c1a5ec4c20 con 0x55c1a5fd8e80 2024-06-10T01:08:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.365+0000 7f561dedf640 1 --2- 172.21.15.37:0/3164792787 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c1a5fd8e80 0x55c1a5fdb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.365+0000 7f561dedf640 1 --2- 172.21.15.37:0/3164792787 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c1a5fd8e80 0x55c1a5fdb270 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1429900c18ba1dcd server_cookie=1784c7d53ea45dfe in_seq=0 out_seq=0 2024-06-10T01:08:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.365+0000 7f5600ff9640 10 client.17457 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:21.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.441+0000 7f2f24a40700 1 -- 172.21.15.37:0/1570734040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f20106650 msgr2=0x7f2f2010e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:21.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.441+0000 7f2f24a40700 1 --2- 172.21.15.37:0/1570734040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f20106650 0x7f2f2010e220 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2f100099c0 tx=0x7f2f10009cd0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.441+0000 7f2f24a40700 1 -- 172.21.15.37:0/1570734040 shutdown_connections 2024-06-10T01:08:21.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.441+0000 7f2f24a40700 1 --2- 172.21.15.37:0/1570734040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f20106650 0x7f2f2010e220 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.441+0000 7f2f24a40700 1 --2- 172.21.15.37:0/1570734040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f20101570 0x7f2f20101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.441+0000 7f2f24a40700 1 --2- 172.21.15.37:0/1570734040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f20101f20 0x7f2f20105f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.441+0000 7f2f24a40700 1 -- 172.21.15.37:0/1570734040 >> 172.21.15.37:0/1570734040 conn(0x7f2f200fce20 msgr2=0x7f2f200ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:21.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.442+0000 7f2f24a40700 1 -- 172.21.15.37:0/1570734040 shutdown_connections 2024-06-10T01:08:21.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.442+0000 7f2f24a40700 1 -- 172.21.15.37:0/1570734040 wait complete. 2024-06-10T01:08:21.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.443+0000 7f2f24a40700 1 Processor -- start 2024-06-10T01:08:21.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.443+0000 7f2f24a40700 1 -- start start 2024-06-10T01:08:21.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.443+0000 7f2f24a40700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f20101570 0x7f2f20116970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.443+0000 7f2f24a40700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f20101f20 0x7f2f20116eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.444+0000 7f2f24a40700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f20106650 0x7f2f20111ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.444+0000 7f2f24a40700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201105f0 con 0x7f2f20101570 2024-06-10T01:08:21.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.444+0000 7f2f24a40700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2f20110470 con 0x7f2f20101f20 2024-06-10T01:08:21.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.444+0000 7f2f24a40700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2f20110770 con 0x7f2f20106650 2024-06-10T01:08:21.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.446+0000 7f2f1f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f20101570 0x7f2f20116970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.446+0000 7f2f1f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f20101570 0x7f2f20116970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49168/0 (socket says 172.21.15.37:49168) 2024-06-10T01:08:21.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.446+0000 7f2f1f7fe700 1 -- 172.21.15.37:0/3910252028 learned_addr learned my addr 172.21.15.37:0/3910252028 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:21.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.447+0000 7f2f1f7fe700 1 -- 172.21.15.37:0/3910252028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f20106650 msgr2=0x7f2f20111ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:21.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.447+0000 7f2f1ffff700 1 --2- 172.21.15.37:0/3910252028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f20106650 0x7f2f20111ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.447+0000 7f2f1f7fe700 1 --2- 172.21.15.37:0/3910252028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f20106650 0x7f2f20111ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.447+0000 7f2f1f7fe700 1 -- 172.21.15.37:0/3910252028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f20101f20 msgr2=0x7f2f20116eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:21.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.447+0000 7f2f1f7fe700 1 --2- 172.21.15.37:0/3910252028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f20101f20 0x7f2f20116eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.447+0000 7f2f1f7fe700 1 -- 172.21.15.37:0/3910252028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f201123e0 con 0x7f2f20101570 2024-06-10T01:08:21.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.447+0000 7f2f1f7fe700 1 --2- 172.21.15.37:0/3910252028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f20101570 0x7f2f20116970 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f2f180060b0 tx=0x7f2f18007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:21.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.448+0000 7f2f1cff9700 1 -- 172.21.15.37:0/3910252028 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1801a070 con 0x7f2f20101570 2024-06-10T01:08:21.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.448+0000 7f2f1cff9700 1 -- 172.21.15.37:0/3910252028 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f1800de20 con 0x7f2f20101570 2024-06-10T01:08:21.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.448+0000 7f2f1cff9700 1 -- 172.21.15.37:0/3910252028 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f18014790 con 0x7f2f20101570 2024-06-10T01:08:21.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.449+0000 7f2f24a40700 1 -- 172.21.15.37:0/3910252028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f201126d0 con 0x7f2f20101570 2024-06-10T01:08:21.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.449+0000 7f2f24a40700 1 -- 172.21.15.37:0/3910252028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f2011c860 con 0x7f2f20101570 2024-06-10T01:08:21.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.455+0000 7f2f30525b80 1 Processor -- start 2024-06-10T01:08:21.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.456+0000 7f2f30525b80 1 -- start start 2024-06-10T01:08:21.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.456+0000 7f2f30525b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586531c5160 0x5586531c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.457+0000 7f2f30525b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586531c5a80 0x5586531bf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.457+0000 7f2f30525b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586531bfa40 0x5586531bfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.457+0000 7f2f30525b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5586530b1f60 con 0x5586531bfa40 2024-06-10T01:08:21.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.457+0000 7f2f30525b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558653110850 con 0x5586531c5a80 2024-06-10T01:08:21.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.457+0000 7f2f30525b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558653078460 con 0x5586531c5160 2024-06-10T01:08:21.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.458+0000 7f2f1ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586531c5160 0x5586531c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.458+0000 7f2f1ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586531c5160 0x5586531c5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:37892/0 (socket says 172.21.15.37:37892) 2024-06-10T01:08:21.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.458+0000 7f2f1ffff700 1 -- 172.21.15.37:0/2169775819 learned_addr learned my addr 172.21.15.37:0/2169775819 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:21.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.460+0000 7f2f1effd700 1 --2- 172.21.15.37:0/2169775819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586531bfa40 0x5586531bfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.460+0000 7f2f1effd700 1 -- 172.21.15.37:0/2169775819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586531c5160 msgr2=0x5586531c5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:21.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.460+0000 7f2f1effd700 1 --2- 172.21.15.37:0/2169775819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586531c5160 0x5586531c5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.461+0000 7f2f1effd700 1 -- 172.21.15.37:0/2169775819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586531c5a80 msgr2=0x5586531bf4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-10T01:08:21.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.461+0000 7f2f1effd700 1 --2- 172.21.15.37:0/2169775819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586531c5a80 0x5586531bf4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.461+0000 7f2f1effd700 1 -- 172.21.15.37:0/2169775819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55865305f8d0 con 0x5586531bfa40 2024-06-10T01:08:21.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.462+0000 7f2f1effd700 1 --2- 172.21.15.37:0/2169775819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586531bfa40 0x5586531bfea0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f2f201132e0 tx=0x7f2f14009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53545cde41ea0fe4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:21.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.462+0000 7f2f1cff9700 1 -- 172.21.15.37:0/3910252028 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2f1801d070 con 0x7f2f20101570 2024-06-10T01:08:21.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.462+0000 7f2f24a40700 1 -- 172.21.15.37:0/2169775819 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f14004230 con 0x5586531bfa40 2024-06-10T01:08:21.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.462+0000 7f2f24a40700 1 -- 172.21.15.37:0/2169775819 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f14004d10 con 0x5586531bfa40 2024-06-10T01:08:21.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.463+0000 7f2f1cff9700 1 --2- 172.21.15.37:0/3910252028 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2f080a2d70 0x7f2f080a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.463+0000 7f2f30525b80 1 -- 172.21.15.37:0/2169775819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586531bfa40 msgr2=0x5586531bfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:21.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.463+0000 7f2f30525b80 1 --2- 172.21.15.37:0/2169775819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586531bfa40 0x5586531bfea0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f2f201132e0 tx=0x7f2f14009e00 comp rx=0 tx=0).stop 2024-06-10T01:08:21.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.463+0000 7f2f1cff9700 1 -- 172.21.15.37:0/3910252028 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f2f180ceda0 con 0x7f2f20101570 2024-06-10T01:08:21.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.463+0000 7f2f1ffff700 1 --2- 172.21.15.37:0/3910252028 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2f080a2d70 0x7f2f080a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.469+0000 7f2f30525b80 1 -- 172.21.15.37:0/2169775819 shutdown_connections 2024-06-10T01:08:21.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.469+0000 7f2f30525b80 1 --2- 172.21.15.37:0/2169775819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586531bfa40 0x5586531bfea0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.469+0000 7f2f30525b80 1 --2- 172.21.15.37:0/2169775819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586531c5160 0x5586531c5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.469+0000 7f2f30525b80 1 --2- 172.21.15.37:0/2169775819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586531c5a80 0x5586531bf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.469+0000 7f2f30525b80 1 -- 172.21.15.37:0/2169775819 >> 172.21.15.37:0/2169775819 conn(0x5586530ec080 msgr2=0x5586531ba830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:21.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.470+0000 7f2f1ffff700 1 --2- 172.21.15.37:0/3910252028 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2f080a2d70 0x7f2f080a5230 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009750 tx=0x7f2f10005040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:21.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.473+0000 7f2f30525b80 1 -- 172.21.15.37:0/2169775819 shutdown_connections 2024-06-10T01:08:21.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.474+0000 7f2f30525b80 1 -- 172.21.15.37:0/2169775819 wait complete. 2024-06-10T01:08:21.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.475+0000 7f2f30525b80 1 -- start start 2024-06-10T01:08:21.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.475+0000 7f2f30525b80 1 Processor -- start 2024-06-10T01:08:21.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.476+0000 7f2f30525b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586531bfa40 0x5586531cb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.476+0000 7f2f30525b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586531c5160 0x5586531c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.476+0000 7f2f30525b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586531c5a80 0x5586531c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.476+0000 7f2f30525b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5586530b0fc0 con 0x5586531bfa40 2024-06-10T01:08:21.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.476+0000 7f2f30525b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55865310c810 con 0x5586531c5a80 2024-06-10T01:08:21.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.476+0000 7f2f30525b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5586531cc4d0 con 0x5586531c5160 2024-06-10T01:08:21.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.477+0000 7f2f30525b80 10 client.0 _ll_register_callbacks cb 0x5586530a65b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:21.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.477+0000 7f2f1f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586531bfa40 0x5586531cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.477+0000 7f2f1f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586531bfa40 0x5586531cb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49192/0 (socket says 172.21.15.37:49192) 2024-06-10T01:08:21.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.477+0000 7f2f1f7fe700 1 -- 172.21.15.37:0/442126416 learned_addr learned my addr 172.21.15.37:0/442126416 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:21.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.478+0000 7f2f1ffff700 1 --2- 172.21.15.37:0/442126416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586531c5a80 0x5586531c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.478+0000 7f2f1effd700 1 --2- 172.21.15.37:0/442126416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586531c5160 0x5586531c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.479+0000 7f2f1f7fe700 1 -- 172.21.15.37:0/442126416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586531c5160 msgr2=0x5586531c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:21.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.479+0000 7f2f1f7fe700 1 --2- 172.21.15.37:0/442126416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586531c5160 0x5586531c8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.479+0000 7f2f1f7fe700 1 -- 172.21.15.37:0/442126416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586531c5a80 msgr2=0x5586531c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:21.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.479+0000 7f2f1f7fe700 1 --2- 172.21.15.37:0/442126416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586531c5a80 0x5586531c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.479+0000 7f2f1f7fe700 1 -- 172.21.15.37:0/442126416 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586530d6750 con 0x5586531bfa40 2024-06-10T01:08:21.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.479+0000 7f2f1f7fe700 1 --2- 172.21.15.37:0/442126416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586531bfa40 0x5586531cb820 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2f18097b80 tx=0x7f2f1809b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1dd964dcdfa202f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:21.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.479+0000 7f2efeffd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:08:21.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.480+0000 7f2f1effd700 1 --2- 172.21.15.37:0/442126416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586531c5160 0x5586531c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:21.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.480+0000 7f2efeffd700 1 -- 172.21.15.37:0/442126416 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f18015ae0 con 0x5586531bfa40 2024-06-10T01:08:21.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.480+0000 7f2efeffd700 1 -- 172.21.15.37:0/442126416 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f2f180bb0b0 con 0x5586531bfa40 2024-06-10T01:08:21.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.480+0000 7f2f30525b80 1 -- 172.21.15.37:0/442126416 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f14004d10 con 0x5586531bfa40 2024-06-10T01:08:21.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.480+0000 7f2efeffd700 1 -- 172.21.15.37:0/442126416 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2f180bb2b0 con 0x5586531bfa40 2024-06-10T01:08:21.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.481+0000 7f2f30525b80 10 client.17469 fetch_fsmap learned FSMap version 31 2024-06-10T01:08:21.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.481+0000 7f2f30525b80 1 -- 172.21.15.37:0/442126416 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5586531d6db0 con 0x5586531bfa40 2024-06-10T01:08:21.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.482+0000 7f2efeffd700 1 -- 172.21.15.37:0/442126416 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f14004d10 con 0x5586531bfa40 2024-06-10T01:08:21.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.483+0000 7f2efeffd700 1 -- 172.21.15.37:0/442126416 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f2f1809b500 con 0x5586531bfa40 2024-06-10T01:08:21.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.483+0000 7f2f30525b80 10 client.17469 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:08:21.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.483+0000 7f2f30525b80 10 client.17469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:21.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.483+0000 7f2f30525b80 20 client.17469 populate_metadata read hostname 'smithi037' 2024-06-10T01:08:21.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.483+0000 7f2f30525b80 1 --2- 172.21.15.37:0/442126416 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5586531d9340 0x5586531db730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.483+0000 7f2f30525b80 4 client.17469 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:08:21.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.483+0000 7f2f30525b80 1 -- 172.21.15.37:0/442126416 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55865311dd50 con 0x5586531d9340 2024-06-10T01:08:21.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.490+0000 7f2f1effd700 1 --2- 172.21.15.37:0/442126416 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5586531d9340 0x5586531db730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.493+0000 7f2f1effd700 1 --2- 172.21.15.37:0/442126416 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5586531d9340 0x5586531db730 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd5c550ebc3ab6e3 server_cookie=926f2f6539670597 in_seq=0 out_seq=0 2024-06-10T01:08:21.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.493+0000 7f2efeffd700 10 client.17469 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:21.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.495+0000 7f2efeffd700 1 -- 172.21.15.37:0/442126416 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55865311dd50 con 0x5586531d9340 2024-06-10T01:08:21.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.495+0000 7f2efeffd700 10 client.17469 handle_command_reply: tid=0 2024-06-10T01:08:21.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.496+0000 7f2f30525b80 1 client.17469 shutdown 2024-06-10T01:08:21.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.503+0000 7f2f30525b80 1 -- 172.21.15.37:0/442126416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586531bfa40 msgr2=0x5586531cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:21.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.503+0000 7f2f30525b80 1 --2- 172.21.15.37:0/442126416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586531bfa40 0x5586531cb820 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2f18097b80 tx=0x7f2f1809b9f0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.503+0000 7f2f30525b80 1 -- 172.21.15.37:0/442126416 shutdown_connections 2024-06-10T01:08:21.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.503+0000 7f2f30525b80 1 --2- 172.21.15.37:0/442126416 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5586531d9340 0x5586531db730 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.503+0000 7f2f30525b80 1 --2- 172.21.15.37:0/442126416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586531c5a80 0x5586531c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.503+0000 7f2f30525b80 1 --2- 172.21.15.37:0/442126416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586531bfa40 0x5586531cb820 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.503+0000 7f2f30525b80 1 --2- 172.21.15.37:0/442126416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586531c5160 0x5586531c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.503+0000 7f2f30525b80 1 -- 172.21.15.37:0/442126416 >> 172.21.15.37:0/442126416 conn(0x5586530ec080 msgr2=0x5586531b47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:21.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.503+0000 7f2efeffd700 0 client.17469 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:21.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.503+0000 7f2f30525b80 1 -- 172.21.15.37:0/442126416 shutdown_connections 2024-06-10T01:08:21.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.503+0000 7f2f30525b80 1 -- 172.21.15.37:0/442126416 wait complete. 2024-06-10T01:08:21.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.503+0000 7f2f30525b80 20 client.17469 trim_cache size 0 max 16384 2024-06-10T01:08:21.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.532+0000 7f2f30525b80 1 Processor -- start 2024-06-10T01:08:21.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.532+0000 7f2f30525b80 1 -- start start 2024-06-10T01:08:21.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.532+0000 7f2f30525b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586530ec080 0x5586531bf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.532+0000 7f2f30525b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586531bfa40 0x5586531c6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.532+0000 7f2f30525b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586531c5160 0x5586531b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.532+0000 7f2f30525b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201bf9b0 con 0x5586531c5160 2024-06-10T01:08:21.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.532+0000 7f2f30525b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201bd240 con 0x5586530ec080 2024-06-10T01:08:21.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.532+0000 7f2f30525b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201bb140 con 0x5586531bfa40 2024-06-10T01:08:21.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.532+0000 7f2f1ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586531c5160 0x5586531b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.532+0000 7f2f1ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586531c5160 0x5586531b48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49194/0 (socket says 172.21.15.37:49194) 2024-06-10T01:08:21.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.532+0000 7f2f1ffff700 1 -- 172.21.15.37:0/980459593 learned_addr learned my addr 172.21.15.37:0/980459593 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:21.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.532+0000 7f2f1f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586530ec080 0x5586531bf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.532+0000 7f2f1effd700 1 --2- 172.21.15.37:0/980459593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586531bfa40 0x5586531c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.532+0000 7f2f1ffff700 1 -- 172.21.15.37:0/980459593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586531bfa40 msgr2=0x5586531c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:21.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.532+0000 7f2f1ffff700 1 --2- 172.21.15.37:0/980459593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586531bfa40 0x5586531c6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.532+0000 7f2f1ffff700 1 -- 172.21.15.37:0/980459593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586530ec080 msgr2=0x5586531bf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:21.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.532+0000 7f2f1ffff700 1 --2- 172.21.15.37:0/980459593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586530ec080 0x5586531bf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.532+0000 7f2f1ffff700 1 -- 172.21.15.37:0/980459593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586531bbe10 con 0x5586531c5160 2024-06-10T01:08:21.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.533+0000 7f2f1effd700 1 --2- 172.21.15.37:0/980459593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586531bfa40 0x5586531c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:08:21.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.533+0000 7f2f1ffff700 1 --2- 172.21.15.37:0/980459593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586531c5160 0x5586531b48e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f2f10005f50 tx=0x7f2f10023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13e95fc02dab5f7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:21.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.533+0000 7f2f057fa700 1 -- 172.21.15.37:0/980459593 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10035070 con 0x5586531c5160 2024-06-10T01:08:21.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.533+0000 7f2f057fa700 1 -- 172.21.15.37:0/980459593 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f1000bc70 con 0x5586531c5160 2024-06-10T01:08:21.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.533+0000 7f2f057fa700 1 -- 172.21.15.37:0/980459593 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1000fb80 con 0x5586531c5160 2024-06-10T01:08:21.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.534+0000 7f2f30525b80 1 -- 172.21.15.37:0/980459593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586531c5160 msgr2=0x5586531b48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:21.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.534+0000 7f2f30525b80 1 --2- 172.21.15.37:0/980459593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586531c5160 0x5586531b48e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f2f10005f50 tx=0x7f2f10023000 comp rx=0 tx=0).stop 2024-06-10T01:08:21.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.534+0000 7f2f30525b80 1 -- 172.21.15.37:0/980459593 shutdown_connections 2024-06-10T01:08:21.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.534+0000 7f2f30525b80 1 --2- 172.21.15.37:0/980459593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586531c5160 0x5586531b48e0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.534+0000 7f2f30525b80 1 --2- 172.21.15.37:0/980459593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586530ec080 0x5586531bf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.534+0000 7f2f30525b80 1 --2- 172.21.15.37:0/980459593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586531bfa40 0x5586531c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.534+0000 7f2f30525b80 1 -- 172.21.15.37:0/980459593 >> 172.21.15.37:0/980459593 conn(0x5586531baaf0 msgr2=0x5586531c83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:21.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.534+0000 7f2f30525b80 1 -- 172.21.15.37:0/980459593 shutdown_connections 2024-06-10T01:08:21.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.534+0000 7f2f30525b80 1 -- 172.21.15.37:0/980459593 wait complete. 2024-06-10T01:08:21.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.535+0000 7f2f30525b80 1 -- start start 2024-06-10T01:08:21.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.535+0000 7f2f30525b80 1 Processor -- start 2024-06-10T01:08:21.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.535+0000 7f2f30525b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586530ec080 0x5586531d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.535+0000 7f2f30525b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586531bfa40 0x5586531ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.535+0000 7f2f30525b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586531c5160 0x5586531cd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.535+0000 7f2f30525b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201bb700 con 0x5586531bfa40 2024-06-10T01:08:21.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.536+0000 7f2f30525b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201bcb70 con 0x5586530ec080 2024-06-10T01:08:21.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.536+0000 7f2f30525b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201bbdd0 con 0x5586531c5160 2024-06-10T01:08:21.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.536+0000 7f2f30525b80 10 client.0 _ll_register_callbacks cb 0x5586530a65b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:21.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.536+0000 7f2f1f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586530ec080 0x5586531d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.536+0000 7f2f1ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586531c5160 0x5586531cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.536+0000 7f2f1ffff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586531c5160 0x5586531cd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:37932/0 (socket says 172.21.15.37:37932) 2024-06-10T01:08:21.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.536+0000 7f2f1ffff700 1 -- 172.21.15.37:0/240352659 learned_addr learned my addr 172.21.15.37:0/240352659 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:21.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.536+0000 7f2f1effd700 1 --2- 172.21.15.37:0/240352659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586531bfa40 0x5586531ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.536+0000 7f2f1ffff700 1 -- 172.21.15.37:0/240352659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586530ec080 msgr2=0x5586531d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:21.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.536+0000 7f2f1ffff700 1 --2- 172.21.15.37:0/240352659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586530ec080 0x5586531d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.536+0000 7f2f1ffff700 1 -- 172.21.15.37:0/240352659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586531bfa40 msgr2=0x5586531ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:21.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.536+0000 7f2f1ffff700 1 --2- 172.21.15.37:0/240352659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586531bfa40 0x5586531ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.536+0000 7f2f1ffff700 1 -- 172.21.15.37:0/240352659 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586531cda50 con 0x5586531c5160 2024-06-10T01:08:21.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.537+0000 7f2f1ffff700 1 --2- 172.21.15.37:0/240352659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586531c5160 0x5586531cd510 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2f10005010 tx=0x7f2f10005a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2250551486c2bbec server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:21.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.537+0000 7f2efeffd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:08:21.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.538+0000 7f2efeffd700 1 -- 172.21.15.37:0/240352659 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10035040 con 0x5586531c5160 2024-06-10T01:08:21.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.538+0000 7f2f30525b80 1 -- 172.21.15.37:0/240352659 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f1000bc70 con 0x5586531c5160 2024-06-10T01:08:21.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.538+0000 7f2efeffd700 1 -- 172.21.15.37:0/240352659 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f2f1000f7c0 con 0x5586531c5160 2024-06-10T01:08:21.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.538+0000 7f2efeffd700 1 -- 172.21.15.37:0/240352659 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2f10047050 con 0x5586531c5160 2024-06-10T01:08:21.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.538+0000 7f2f30525b80 10 client.26965 fetch_fsmap learned FSMap version 31 2024-06-10T01:08:21.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.538+0000 7f2f30525b80 1 -- 172.21.15.37:0/240352659 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558653243c90 con 0x5586531c5160 2024-06-10T01:08:21.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.538+0000 7f2efeffd700 1 -- 172.21.15.37:0/240352659 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f1000bc70 con 0x5586531c5160 2024-06-10T01:08:21.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.539+0000 7f2efeffd700 1 -- 172.21.15.37:0/240352659 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f2f10025990 con 0x5586531c5160 2024-06-10T01:08:21.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.539+0000 7f2f30525b80 10 client.26965 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:08:21.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.539+0000 7f2f30525b80 10 client.26965 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:21.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.539+0000 7f2f30525b80 20 client.26965 populate_metadata read hostname 'smithi037' 2024-06-10T01:08:21.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.539+0000 7f2f30525b80 1 --2- 172.21.15.37:0/240352659 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5586531c5a80 0x558653246210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:21.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.539+0000 7f2f30525b80 4 client.26965 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:08:21.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.539+0000 7f2f30525b80 1 -- 172.21.15.37:0/240352659 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5586531182d0 con 0x5586531c5a80 2024-06-10T01:08:21.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.541+0000 7f2f1f7fe700 1 --2- 172.21.15.37:0/240352659 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5586531c5a80 0x558653246210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:21.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.542+0000 7f2f1f7fe700 1 --2- 172.21.15.37:0/240352659 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5586531c5a80 0x558653246210 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=304032cc4c8e0968 server_cookie=46ce4041828166a9 in_seq=0 out_seq=0 2024-06-10T01:08:21.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.542+0000 7f2efeffd700 10 client.26965 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:21.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.546+0000 7f2efeffd700 1 -- 172.21.15.37:0/240352659 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5586531182d0 con 0x5586531c5a80 2024-06-10T01:08:21.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.546+0000 7f2efeffd700 10 client.26965 handle_command_reply: tid=0 2024-06-10T01:08:21.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.546+0000 7f2f30525b80 1 client.26965 shutdown 2024-06-10T01:08:21.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.546+0000 7f2f30525b80 1 -- 172.21.15.37:0/240352659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586531c5160 msgr2=0x5586531cd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:21.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.546+0000 7f2f30525b80 1 --2- 172.21.15.37:0/240352659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586531c5160 0x5586531cd510 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2f10005010 tx=0x7f2f10005a00 comp rx=0 tx=0).stop 2024-06-10T01:08:21.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.547+0000 7f2f30525b80 1 -- 172.21.15.37:0/240352659 shutdown_connections 2024-06-10T01:08:21.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.547+0000 7f2f30525b80 1 --2- 172.21.15.37:0/240352659 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5586531c5a80 0x558653246210 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.547+0000 7f2f30525b80 1 --2- 172.21.15.37:0/240352659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5586531c5160 0x5586531cd510 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.547+0000 7f2efeffd700 0 client.26965 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:21.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.547+0000 7f2f30525b80 1 --2- 172.21.15.37:0/240352659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5586530ec080 0x5586531d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.547+0000 7f2f30525b80 1 --2- 172.21.15.37:0/240352659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586531bfa40 0x5586531ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.547+0000 7f2f30525b80 1 -- 172.21.15.37:0/240352659 >> 172.21.15.37:0/240352659 conn(0x5586531baaf0 msgr2=0x5586531bed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:21.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.547+0000 7f2f30525b80 1 -- 172.21.15.37:0/240352659 shutdown_connections 2024-06-10T01:08:21.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.547+0000 7f2f30525b80 1 -- 172.21.15.37:0/240352659 wait complete. 2024-06-10T01:08:21.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.547+0000 7f2f30525b80 20 client.26965 trim_cache size 0 max 16384 2024-06-10T01:08:21.544 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T01:08:21.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.548+0000 7f2f067fc700 1 -- 172.21.15.37:0/3910252028 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2f080a2d70 msgr2=0x7f2f080a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:21.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.548+0000 7f2f067fc700 1 --2- 172.21.15.37:0/3910252028 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2f080a2d70 0x7f2f080a5230 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009750 tx=0x7f2f10005040 comp rx=0 tx=0).stop 2024-06-10T01:08:21.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.548+0000 7f2f067fc700 1 -- 172.21.15.37:0/3910252028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f20101570 msgr2=0x7f2f20116970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:21.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.548+0000 7f2f067fc700 1 --2- 172.21.15.37:0/3910252028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f20101570 0x7f2f20116970 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f2f180060b0 tx=0x7f2f18007f90 comp rx=0 tx=0).stop 2024-06-10T01:08:21.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.548+0000 7f2f067fc700 1 -- 172.21.15.37:0/3910252028 shutdown_connections 2024-06-10T01:08:21.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.548+0000 7f2f067fc700 1 --2- 172.21.15.37:0/3910252028 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2f080a2d70 0x7f2f080a5230 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.548+0000 7f2f067fc700 1 --2- 172.21.15.37:0/3910252028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f20106650 0x7f2f20111ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.548+0000 7f2f067fc700 1 --2- 172.21.15.37:0/3910252028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f20101570 0x7f2f20116970 secure :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f2f180060b0 tx=0x7f2f18007f90 comp rx=0 tx=0).stop 2024-06-10T01:08:21.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.548+0000 7f2f067fc700 1 --2- 172.21.15.37:0/3910252028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f20101f20 0x7f2f20116eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.548+0000 7f2f067fc700 1 -- 172.21.15.37:0/3910252028 >> 172.21.15.37:0/3910252028 conn(0x7f2f200fce20 msgr2=0x7f2f200ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:21.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.548+0000 7f2f067fc700 1 -- 172.21.15.37:0/3910252028 shutdown_connections 2024-06-10T01:08:21.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:21.548+0000 7f2f067fc700 1 -- 172.21.15.37:0/3910252028 wait complete. 2024-06-10T01:08:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.589+0000 7f5600ff9640 1 -- 172.21.15.37:0/3164792787 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c1a5ec4c20 con 0x55c1a5fd8e80 2024-06-10T01:08:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.589+0000 7f5600ff9640 10 client.17457 handle_command_reply: tid=0 2024-06-10T01:08:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.589+0000 7f562264e740 1 client.17457 shutdown 2024-06-10T01:08:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.590+0000 7f562264e740 1 -- 172.21.15.37:0/3164792787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1a5f58e10 msgr2=0x55c1a5f67160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.590+0000 7f562264e740 1 --2- 172.21.15.37:0/3164792787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1a5f58e10 0x55c1a5f67160 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f5608004140 tx=0x7f5608003e00 comp rx=0 tx=0).stop 2024-06-10T01:08:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.590+0000 7f562264e740 1 -- 172.21.15.37:0/3164792787 shutdown_connections 2024-06-10T01:08:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.590+0000 7f562264e740 1 --2- 172.21.15.37:0/3164792787 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c1a5fd8e80 0x55c1a5fdb270 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.590+0000 7f562264e740 1 --2- 172.21.15.37:0/3164792787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c1a5f5df80 0x55c1a5f68170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.590+0000 7f562264e740 1 --2- 172.21.15.37:0/3164792787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c1a5f58e10 0x55c1a5f67160 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.590+0000 7f5600ff9640 0 client.17457 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.590+0000 7f562264e740 1 --2- 172.21.15.37:0/3164792787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c1a5f58420 0x55c1a5f66c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.590+0000 7f562264e740 1 -- 172.21.15.37:0/3164792787 >> 172.21.15.37:0/3164792787 conn(0x55c1a5f4f680 msgr2=0x55c1a5f52670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.590+0000 7f562264e740 1 -- 172.21.15.37:0/3164792787 shutdown_connections 2024-06-10T01:08:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.590+0000 7f562264e740 1 -- 172.21.15.37:0/3164792787 wait complete. 2024-06-10T01:08:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.590+0000 7f562264e740 20 client.17457 trim_cache size 0 max 16384 2024-06-10T01:08:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:08:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:08:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:08:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.591+0000 7f561f942640 1 -- 172.21.15.37:0/515440112 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f56040a2c30 msgr2=0x7f56040a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.591+0000 7f561f942640 1 --2- 172.21.15.37:0/515440112 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f56040a2c30 0x7f56040a50f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x55c1a5f59210 tx=0x7f5614009260 comp rx=0 tx=0).stop 2024-06-10T01:08:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.591+0000 7f561f942640 1 -- 172.21.15.37:0/515440112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56181024e0 msgr2=0x7f5618197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.591+0000 7f561f942640 1 --2- 172.21.15.37:0/515440112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56181024e0 0x7f5618197c90 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f5614009230 tx=0x7f5614031b60 comp rx=0 tx=0).stop 2024-06-10T01:08:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.591+0000 7f561f942640 1 -- 172.21.15.37:0/515440112 shutdown_connections 2024-06-10T01:08:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.591+0000 7f561f942640 1 --2- 172.21.15.37:0/515440112 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f56040a2c30 0x7f56040a50f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.591+0000 7f561f942640 1 --2- 172.21.15.37:0/515440112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5618107740 0x7f561819f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.591+0000 7f561f942640 1 --2- 172.21.15.37:0/515440112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5618104e10 0x7f56181981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.591+0000 7f561f942640 1 --2- 172.21.15.37:0/515440112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56181024e0 0x7f5618197c90 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.591+0000 7f561f942640 1 -- 172.21.15.37:0/515440112 >> 172.21.15.37:0/515440112 conn(0x7f56180fbe70 msgr2=0x7f5618102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.591+0000 7f561f942640 1 -- 172.21.15.37:0/515440112 shutdown_connections 2024-06-10T01:08:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:21.591+0000 7f561f942640 1 -- 172.21.15.37:0/515440112 wait complete. 2024-06-10T01:08:22.523 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:22 smithi037 ceph-mon[32192]: pgmap v781: 129 pgs: 129 active+clean; 8.7 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 661 KiB/s wr, 362 op/s 2024-06-10T01:08:22.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:22 smithi167 ceph-mon[36917]: pgmap v781: 129 pgs: 129 active+clean; 8.7 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 661 KiB/s wr, 362 op/s 2024-06-10T01:08:22.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:22 smithi094 ceph-mon[36708]: pgmap v781: 129 pgs: 129 active+clean; 8.7 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 661 KiB/s wr, 362 op/s 2024-06-10T01:08:23.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.555+0000 7f271676a700 1 -- 172.21.15.37:0/4216955845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2710102770 msgr2=0x7f2710102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:23.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.555+0000 7f271676a700 1 --2- 172.21.15.37:0/4216955845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2710102770 0x7f2710102b50 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f27000099c0 tx=0x7f2700009cd0 comp rx=0 tx=0).stop 2024-06-10T01:08:23.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.555+0000 7f271676a700 1 -- 172.21.15.37:0/4216955845 shutdown_connections 2024-06-10T01:08:23.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.555+0000 7f271676a700 1 --2- 172.21.15.37:0/4216955845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2710107860 0x7f271010f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:23.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.555+0000 7f271676a700 1 --2- 172.21.15.37:0/4216955845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2710102770 0x7f2710102b50 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:23.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.555+0000 7f271676a700 1 --2- 172.21.15.37:0/4216955845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2710103120 0x7f2710107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:23.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.555+0000 7f271676a700 1 -- 172.21.15.37:0/4216955845 >> 172.21.15.37:0/4216955845 conn(0x7f2710079600 msgr2=0x7f2710079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:23.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.555+0000 7f271676a700 1 -- 172.21.15.37:0/4216955845 shutdown_connections 2024-06-10T01:08:23.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.555+0000 7f271676a700 1 -- 172.21.15.37:0/4216955845 wait complete. 2024-06-10T01:08:23.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.556+0000 7f271676a700 1 Processor -- start 2024-06-10T01:08:23.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.556+0000 7f271676a700 1 -- start start 2024-06-10T01:08:23.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.556+0000 7f271676a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2710102770 0x7f2710074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:23.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.556+0000 7f271676a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2710103120 0x7f2710074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:23.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.556+0000 7f271676a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2710107860 0x7f2710075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:23.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.556+0000 7f271676a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2710112810 con 0x7f2710107860 2024-06-10T01:08:23.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.556+0000 7f271676a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2710112690 con 0x7f2710102770 2024-06-10T01:08:23.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.556+0000 7f271676a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2710112990 con 0x7f2710103120 2024-06-10T01:08:23.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.556+0000 7f2714d07700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2710107860 0x7f2710075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:23.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.556+0000 7f2714d07700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2710107860 0x7f2710075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50552/0 (socket says 172.21.15.37:50552) 2024-06-10T01:08:23.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.556+0000 7f2714d07700 1 -- 172.21.15.37:0/3084952224 learned_addr learned my addr 172.21.15.37:0/3084952224 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:23.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.556+0000 7f270ffff700 1 --2- 172.21.15.37:0/3084952224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2710102770 0x7f2710074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:23.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.556+0000 7f2707fff700 1 --2- 172.21.15.37:0/3084952224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2710103120 0x7f2710074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:23.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.557+0000 7f2714d07700 1 -- 172.21.15.37:0/3084952224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2710103120 msgr2=0x7f2710074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:23.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.557+0000 7f2714d07700 1 --2- 172.21.15.37:0/3084952224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2710103120 0x7f2710074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:23.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.557+0000 7f2714d07700 1 -- 172.21.15.37:0/3084952224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2710102770 msgr2=0x7f2710074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:23.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.557+0000 7f2714d07700 1 --2- 172.21.15.37:0/3084952224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2710102770 0x7f2710074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:23.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.557+0000 7f2714d07700 1 -- 172.21.15.37:0/3084952224 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27101a7160 con 0x7f2710107860 2024-06-10T01:08:23.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.557+0000 7f2707fff700 1 --2- 172.21.15.37:0/3084952224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2710103120 0x7f2710074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:08:23.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.557+0000 7f2714d07700 1 --2- 172.21.15.37:0/3084952224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2710107860 0x7f2710075210 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f2708007ee0 tx=0x7f270800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:23.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.557+0000 7f270dffb700 1 -- 172.21.15.37:0/3084952224 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270800dbd0 con 0x7f2710107860 2024-06-10T01:08:23.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.557+0000 7f270dffb700 1 -- 172.21.15.37:0/3084952224 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f270800dd70 con 0x7f2710107860 2024-06-10T01:08:23.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.557+0000 7f271676a700 1 -- 172.21.15.37:0/3084952224 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27101a7450 con 0x7f2710107860 2024-06-10T01:08:23.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.557+0000 7f270dffb700 1 -- 172.21.15.37:0/3084952224 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2708005a60 con 0x7f2710107860 2024-06-10T01:08:23.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.557+0000 7f271676a700 1 -- 172.21.15.37:0/3084952224 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27101a7990 con 0x7f2710107860 2024-06-10T01:08:23.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.559+0000 7f270dffb700 1 -- 172.21.15.37:0/3084952224 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f270801a020 con 0x7f2710107860 2024-06-10T01:08:23.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.560+0000 7f270dffb700 1 --2- 172.21.15.37:0/3084952224 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f26fc0a2c20 0x7f26fc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:23.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.560+0000 7f270ffff700 1 --2- 172.21.15.37:0/3084952224 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f26fc0a2c20 0x7f26fc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:23.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.560+0000 7f270dffb700 1 -- 172.21.15.37:0/3084952224 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f27080ceb50 con 0x7f2710107860 2024-06-10T01:08:23.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.561+0000 7f270ffff700 1 --2- 172.21.15.37:0/3084952224 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f26fc0a2c20 0x7f26fc0a50e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2700000c00 tx=0x7f2700005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:23.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.561+0000 7f2720fedb80 1 Processor -- start 2024-06-10T01:08:23.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.561+0000 7f2720fedb80 1 -- start start 2024-06-10T01:08:23.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.561+0000 7f2720fedb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e35352c40 0x561e35353020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:23.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.561+0000 7f2720fedb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e35353560 0x561e3534ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:23.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.561+0000 7f2720fedb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e3534d3d0 0x561e3534d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:23.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.561+0000 7f2720fedb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561e3529e910 con 0x561e35352c40 2024-06-10T01:08:23.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.561+0000 7f2720fedb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561e35246820 con 0x561e3534d3d0 2024-06-10T01:08:23.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.561+0000 7f2720fedb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561e3529a8d0 con 0x561e35353560 2024-06-10T01:08:23.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.561+0000 7f270ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e35352c40 0x561e35353020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:23.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.561+0000 7f270ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e35352c40 0x561e35353020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50568/0 (socket says 172.21.15.37:50568) 2024-06-10T01:08:23.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.561+0000 7f270ffff700 1 -- 172.21.15.37:0/3032491934 learned_addr learned my addr 172.21.15.37:0/3032491934 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:23.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.562+0000 7f2707fff700 1 --2- 172.21.15.37:0/3032491934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e35353560 0x561e3534ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:23.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.562+0000 7f2714d07700 1 --2- 172.21.15.37:0/3032491934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e3534d3d0 0x561e3534d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.562+0000 7f270ffff700 1 -- 172.21.15.37:0/3032491934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e35353560 msgr2=0x561e3534ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.562+0000 7f270ffff700 1 --2- 172.21.15.37:0/3032491934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e35353560 0x561e3534ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.562+0000 7f270ffff700 1 -- 172.21.15.37:0/3032491934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e3534d3d0 msgr2=0x561e3534d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.562+0000 7f270ffff700 1 --2- 172.21.15.37:0/3032491934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e3534d3d0 0x561e3534d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.562+0000 7f270ffff700 1 -- 172.21.15.37:0/3032491934 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e3523d050 con 0x561e35352c40 2024-06-10T01:08:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.562+0000 7f2707fff700 1 --2- 172.21.15.37:0/3032491934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e35353560 0x561e3534ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:08:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.562+0000 7f270ffff700 1 --2- 172.21.15.37:0/3032491934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e35352c40 0x561e35353020 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f2700015000 tx=0x7f2700020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d54bcedf76541a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.562+0000 7f27067fc700 1 -- 172.21.15.37:0/3032491934 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270000bdb0 con 0x561e35352c40 2024-06-10T01:08:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.562+0000 7f27067fc700 1 -- 172.21.15.37:0/3032491934 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f270000fd50 con 0x561e35352c40 2024-06-10T01:08:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.562+0000 7f27067fc700 1 -- 172.21.15.37:0/3032491934 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2700024e00 con 0x561e35352c40 2024-06-10T01:08:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.563+0000 7f2720fedb80 1 -- 172.21.15.37:0/3032491934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e35352c40 msgr2=0x561e35353020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.563+0000 7f2720fedb80 1 --2- 172.21.15.37:0/3032491934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e35352c40 0x561e35353020 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f2700015000 tx=0x7f2700020000 comp rx=0 tx=0).stop 2024-06-10T01:08:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.563+0000 7f2720fedb80 1 -- 172.21.15.37:0/3032491934 shutdown_connections 2024-06-10T01:08:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.563+0000 7f2720fedb80 1 --2- 172.21.15.37:0/3032491934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e3534d3d0 0x561e3534d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:23.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.563+0000 7f2720fedb80 1 --2- 172.21.15.37:0/3032491934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e35352c40 0x561e35353020 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:23.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.563+0000 7f2720fedb80 1 --2- 172.21.15.37:0/3032491934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e35353560 0x561e3534ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:23.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.563+0000 7f2720fedb80 1 -- 172.21.15.37:0/3032491934 >> 172.21.15.37:0/3032491934 conn(0x561e35277dc0 msgr2=0x561e35348120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:23.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.563+0000 7f2720fedb80 1 -- 172.21.15.37:0/3032491934 shutdown_connections 2024-06-10T01:08:23.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.563+0000 7f2720fedb80 1 -- 172.21.15.37:0/3032491934 wait complete. 2024-06-10T01:08:23.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.563+0000 7f2720fedb80 1 -- start start 2024-06-10T01:08:23.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.564+0000 7f2720fedb80 1 Processor -- start 2024-06-10T01:08:23.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.564+0000 7f2720fedb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e3534d3d0 0x561e353592e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:23.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.564+0000 7f2720fedb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e35352c40 0x561e35356630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:23.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.564+0000 7f2720fedb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e35353560 0x561e35356b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:23.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.564+0000 7f2720fedb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561e35359f10 con 0x561e3534d3d0 2024-06-10T01:08:23.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.564+0000 7f2720fedb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561e3523ecf0 con 0x561e35352c40 2024-06-10T01:08:23.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.564+0000 7f2720fedb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561e3535a090 con 0x561e35353560 2024-06-10T01:08:23.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.564+0000 7f2720fedb80 10 client.0 _ll_register_callbacks cb 0x561e35238a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:23.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.564+0000 7f270ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e3534d3d0 0x561e353592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:23.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.564+0000 7f270ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e3534d3d0 0x561e353592e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50584/0 (socket says 172.21.15.37:50584) 2024-06-10T01:08:23.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.564+0000 7f270ffff700 1 -- 172.21.15.37:0/3290483737 learned_addr learned my addr 172.21.15.37:0/3290483737 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:23.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.564+0000 7f2707fff700 1 --2- 172.21.15.37:0/3290483737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e35352c40 0x561e35356630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:23.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.564+0000 7f2714d07700 1 --2- 172.21.15.37:0/3290483737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e35353560 0x561e35356b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:23.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.564+0000 7f270ffff700 1 -- 172.21.15.37:0/3290483737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e35353560 msgr2=0x561e35356b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:23.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.564+0000 7f270ffff700 1 --2- 172.21.15.37:0/3290483737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e35353560 0x561e35356b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:23.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.564+0000 7f270ffff700 1 -- 172.21.15.37:0/3290483737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e35352c40 msgr2=0x561e35356630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:23.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.564+0000 7f270ffff700 1 --2- 172.21.15.37:0/3290483737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e35352c40 0x561e35356630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:23.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.564+0000 7f270ffff700 1 -- 172.21.15.37:0/3290483737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e351ed9d0 con 0x561e3534d3d0 2024-06-10T01:08:23.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.565+0000 7f270ffff700 1 --2- 172.21.15.37:0/3290483737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e3534d3d0 0x561e353592e0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f2700009350 tx=0x7f270000ff00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b8cac1192c05b4 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:23.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.565+0000 7f26ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:08:23.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.565+0000 7f26ef7fe700 1 -- 172.21.15.37:0/3290483737 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270003c060 con 0x561e3534d3d0 2024-06-10T01:08:23.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.565+0000 7f26ef7fe700 1 -- 172.21.15.37:0/3290483737 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f2700024a80 con 0x561e3534d3d0 2024-06-10T01:08:23.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.565+0000 7f2720fedb80 1 -- 172.21.15.37:0/3290483737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f270000fd50 con 0x561e3534d3d0 2024-06-10T01:08:23.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.565+0000 7f26ef7fe700 1 -- 172.21.15.37:0/3290483737 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2700043050 con 0x561e3534d3d0 2024-06-10T01:08:23.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.565+0000 7f2720fedb80 10 client.17487 fetch_fsmap learned FSMap version 31 2024-06-10T01:08:23.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.565+0000 7f2720fedb80 1 -- 172.21.15.37:0/3290483737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561e35364790 con 0x561e3534d3d0 2024-06-10T01:08:23.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.565+0000 7f26ef7fe700 1 -- 172.21.15.37:0/3290483737 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f270000fd50 con 0x561e3534d3d0 2024-06-10T01:08:23.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.565+0000 7f26ef7fe700 1 -- 172.21.15.37:0/3290483737 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f270002fcb0 con 0x561e3534d3d0 2024-06-10T01:08:23.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.565+0000 7f2720fedb80 10 client.17487 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:08:23.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.565+0000 7f2720fedb80 10 client.17487 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:08:23.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.565+0000 7f2720fedb80 20 client.17487 populate_metadata read hostname 'smithi037' 2024-06-10T01:08:23.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.565+0000 7f2720fedb80 1 --2- 172.21.15.37:0/3290483737 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561e35366c60 0x561e35369050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:23.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.565+0000 7f2720fedb80 4 client.17487 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:08:23.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.565+0000 7f2720fedb80 1 -- 172.21.15.37:0/3290483737 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e352abe10 con 0x561e35366c60 2024-06-10T01:08:23.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.566+0000 7f2707fff700 1 --2- 172.21.15.37:0/3290483737 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561e35366c60 0x561e35369050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:23.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.566+0000 7f2707fff700 1 --2- 172.21.15.37:0/3290483737 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561e35366c60 0x561e35369050 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87f577d8694ac516 server_cookie=d19081d8063a77e3 in_seq=0 out_seq=0 2024-06-10T01:08:23.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.566+0000 7f26ef7fe700 10 client.17487 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:23.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.567+0000 7f26ef7fe700 1 -- 172.21.15.37:0/3290483737 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561e352abe10 con 0x561e35366c60 2024-06-10T01:08:23.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.567+0000 7f26ef7fe700 10 client.17487 handle_command_reply: tid=0 2024-06-10T01:08:23.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.567+0000 7f2720fedb80 1 client.17487 shutdown 2024-06-10T01:08:23.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.567+0000 7f2720fedb80 1 -- 172.21.15.37:0/3290483737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e3534d3d0 msgr2=0x561e353592e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:23.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.567+0000 7f2720fedb80 1 --2- 172.21.15.37:0/3290483737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e3534d3d0 0x561e353592e0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f2700009350 tx=0x7f270000ff00 comp rx=0 tx=0).stop 2024-06-10T01:08:23.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.567+0000 7f2720fedb80 1 -- 172.21.15.37:0/3290483737 shutdown_connections 2024-06-10T01:08:23.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.567+0000 7f2720fedb80 1 --2- 172.21.15.37:0/3290483737 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561e35366c60 0x561e35369050 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:23.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.567+0000 7f2720fedb80 1 --2- 172.21.15.37:0/3290483737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e35353560 0x561e35356b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:23.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.567+0000 7f2720fedb80 1 --2- 172.21.15.37:0/3290483737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e3534d3d0 0x561e353592e0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:23.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.567+0000 7f26ef7fe700 0 client.17487 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:23.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.567+0000 7f2720fedb80 1 --2- 172.21.15.37:0/3290483737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e35352c40 0x561e35356630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:23.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.567+0000 7f2720fedb80 1 -- 172.21.15.37:0/3290483737 >> 172.21.15.37:0/3290483737 conn(0x561e35277dc0 msgr2=0x561e3534be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:23.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.567+0000 7f2720fedb80 1 -- 172.21.15.37:0/3290483737 shutdown_connections 2024-06-10T01:08:23.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.567+0000 7f2720fedb80 1 -- 172.21.15.37:0/3290483737 wait complete. 2024-06-10T01:08:23.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.567+0000 7f2720fedb80 20 client.17487 trim_cache size 0 max 16384 2024-06-10T01:08:23.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.588+0000 7f2720fedb80 1 Processor -- start 2024-06-10T01:08:23.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.588+0000 7f2720fedb80 1 -- start start 2024-06-10T01:08:23.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.588+0000 7f2720fedb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e3534d3d0 0x561e35354110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:23.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.588+0000 7f2720fedb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e35352c40 0x561e35347830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:23.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.588+0000 7f2720fedb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e35353560 0x561e35347d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:23.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.588+0000 7f2720fedb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f27101b2fc0 con 0x561e35353560 2024-06-10T01:08:23.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.588+0000 7f2720fedb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f27101b0850 con 0x561e3534d3d0 2024-06-10T01:08:23.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.588+0000 7f2720fedb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f27101ae6a0 con 0x561e35352c40 2024-06-10T01:08:23.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.588+0000 7f2714d07700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e35353560 0x561e35347d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:23.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.588+0000 7f2714d07700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e35353560 0x561e35347d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50590/0 (socket says 172.21.15.37:50590) 2024-06-10T01:08:23.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.588+0000 7f2714d07700 1 -- 172.21.15.37:0/14143353 learned_addr learned my addr 172.21.15.37:0/14143353 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:23.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.588+0000 7f270ffff700 1 --2- 172.21.15.37:0/14143353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e3534d3d0 0x561e35354110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:23.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.588+0000 7f2707fff700 1 --2- 172.21.15.37:0/14143353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e35352c40 0x561e35347830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:23.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.588+0000 7f2714d07700 1 -- 172.21.15.37:0/14143353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e35352c40 msgr2=0x561e35347830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:23.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.588+0000 7f2714d07700 1 --2- 172.21.15.37:0/14143353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e35352c40 0x561e35347830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:23.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.588+0000 7f2714d07700 1 -- 172.21.15.37:0/14143353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e3534d3d0 msgr2=0x561e35354110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:23.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.588+0000 7f2714d07700 1 --2- 172.21.15.37:0/14143353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e3534d3d0 0x561e35354110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:23.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.588+0000 7f2714d07700 1 -- 172.21.15.37:0/14143353 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e35343fa0 con 0x561e35353560 2024-06-10T01:08:23.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.588+0000 7f270ffff700 1 --2- 172.21.15.37:0/14143353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e3534d3d0 0x561e35354110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:08:23.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.589+0000 7f2714d07700 1 --2- 172.21.15.37:0/14143353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e35353560 0x561e35347d70 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f27080cc5d0 tx=0x7f270800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2930b59dd531a61 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:23.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.589+0000 7f2705ffb700 1 -- 172.21.15.37:0/14143353 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270809f370 con 0x561e35353560 2024-06-10T01:08:23.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.589+0000 7f2705ffb700 1 -- 172.21.15.37:0/14143353 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27080a3c30 con 0x561e35353560 2024-06-10T01:08:23.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.589+0000 7f2705ffb700 1 -- 172.21.15.37:0/14143353 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270809b220 con 0x561e35353560 2024-06-10T01:08:23.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.589+0000 7f2720fedb80 1 -- 172.21.15.37:0/14143353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e35353560 msgr2=0x561e35347d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:23.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.589+0000 7f2720fedb80 1 --2- 172.21.15.37:0/14143353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e35353560 0x561e35347d70 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f27080cc5d0 tx=0x7f270800b040 comp rx=0 tx=0).stop 2024-06-10T01:08:23.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.589+0000 7f2720fedb80 1 -- 172.21.15.37:0/14143353 shutdown_connections 2024-06-10T01:08:23.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.589+0000 7f2720fedb80 1 --2- 172.21.15.37:0/14143353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e35353560 0x561e35347d70 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:23.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.589+0000 7f2720fedb80 1 --2- 172.21.15.37:0/14143353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e3534d3d0 0x561e35354110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:23.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.589+0000 7f2720fedb80 1 --2- 172.21.15.37:0/14143353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e35352c40 0x561e35347830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:23.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.589+0000 7f2720fedb80 1 -- 172.21.15.37:0/14143353 >> 172.21.15.37:0/14143353 conn(0x561e35277dc0 msgr2=0x561e35353a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:23.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.589+0000 7f2720fedb80 1 -- 172.21.15.37:0/14143353 shutdown_connections 2024-06-10T01:08:23.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.589+0000 7f2720fedb80 1 -- 172.21.15.37:0/14143353 wait complete. 2024-06-10T01:08:23.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.590+0000 7f2720fedb80 1 -- start start 2024-06-10T01:08:23.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.590+0000 7f2720fedb80 1 Processor -- start 2024-06-10T01:08:23.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.590+0000 7f2720fedb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e3534d3d0 0x561e3536c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:23.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.590+0000 7f2720fedb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e35352c40 0x561e3536a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:23.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.590+0000 7f2720fedb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e35353560 0x561e3536add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:23.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.590+0000 7f2720fedb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f27101aed10 con 0x561e35352c40 2024-06-10T01:08:23.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.590+0000 7f2720fedb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f27101b0180 con 0x561e35353560 2024-06-10T01:08:23.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.590+0000 7f2720fedb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f27101af3e0 con 0x561e3534d3d0 2024-06-10T01:08:23.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.590+0000 7f2720fedb80 10 client.0 _ll_register_callbacks cb 0x561e35238a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:23.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.590+0000 7f2707fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e35352c40 0x561e3536a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:23.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.590+0000 7f2707fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e35352c40 0x561e3536a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50600/0 (socket says 172.21.15.37:50600) 2024-06-10T01:08:23.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.590+0000 7f2707fff700 1 -- 172.21.15.37:0/948584178 learned_addr learned my addr 172.21.15.37:0/948584178 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:23.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.591+0000 7f270ffff700 1 --2- 172.21.15.37:0/948584178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e3534d3d0 0x561e3536c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:23.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.591+0000 7f2714d07700 1 --2- 172.21.15.37:0/948584178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e35353560 0x561e3536add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:23.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.591+0000 7f2707fff700 1 -- 172.21.15.37:0/948584178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e3534d3d0 msgr2=0x561e3536c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:23.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.591+0000 7f2707fff700 1 --2- 172.21.15.37:0/948584178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e3534d3d0 0x561e3536c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:23.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.591+0000 7f2707fff700 1 -- 172.21.15.37:0/948584178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e35353560 msgr2=0x561e3536add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:23.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.591+0000 7f2707fff700 1 --2- 172.21.15.37:0/948584178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e35353560 0x561e3536add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:23.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.591+0000 7f2707fff700 1 -- 172.21.15.37:0/948584178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e3536b310 con 0x561e35352c40 2024-06-10T01:08:23.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.591+0000 7f270ffff700 1 --2- 172.21.15.37:0/948584178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e3534d3d0 0x561e3536c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:08:23.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.591+0000 7f2707fff700 1 --2- 172.21.15.37:0/948584178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e35352c40 0x561e3536a890 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f26f8006fd0 tx=0x7f26f8009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc3f5f4d7854d3b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:23.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.591+0000 7f26ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:08:23.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.591+0000 7f26ef7fe700 1 -- 172.21.15.37:0/948584178 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f800f040 con 0x561e35352c40 2024-06-10T01:08:23.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.591+0000 7f26ef7fe700 1 -- 172.21.15.37:0/948584178 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f26f8015c90 con 0x561e35352c40 2024-06-10T01:08:23.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.591+0000 7f2720fedb80 1 -- 172.21.15.37:0/948584178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27080a3c30 con 0x561e35352c40 2024-06-10T01:08:23.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.591+0000 7f26ef7fe700 1 -- 172.21.15.37:0/948584178 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f26f8024440 con 0x561e35352c40 2024-06-10T01:08:23.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.591+0000 7f2720fedb80 10 client.17493 fetch_fsmap learned FSMap version 31 2024-06-10T01:08:23.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.591+0000 7f2720fedb80 1 -- 172.21.15.37:0/948584178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561e353d23a0 con 0x561e35352c40 2024-06-10T01:08:23.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.592+0000 7f26ef7fe700 1 -- 172.21.15.37:0/948584178 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27080a3c30 con 0x561e35352c40 2024-06-10T01:08:23.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.592+0000 7f26ef7fe700 1 -- 172.21.15.37:0/948584178 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f26f8022070 con 0x561e35352c40 2024-06-10T01:08:23.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.592+0000 7f2720fedb80 10 client.17493 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:08:23.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.592+0000 7f2720fedb80 10 client.17493 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:08:23.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.592+0000 7f2720fedb80 20 client.17493 populate_metadata read hostname 'smithi037' 2024-06-10T01:08:23.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.592+0000 7f2720fedb80 1 --2- 172.21.15.37:0/948584178 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561e353d2ae0 0x561e353d4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:23.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.592+0000 7f2720fedb80 4 client.17493 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "96a2bbd8", "release": true, "await": true}] 2024-06-10T01:08:23.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.592+0000 7f2720fedb80 1 -- 172.21.15.37:0/948584178 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "quiesce db", "set_id": "96a2bbd8", "release": true, "await": true}) -- 0x561e352a6390 con 0x561e353d2ae0 2024-06-10T01:08:23.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.592+0000 7f270ffff700 1 --2- 172.21.15.37:0/948584178 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561e353d2ae0 0x561e353d4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:23.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.592+0000 7f270ffff700 1 --2- 172.21.15.37:0/948584178 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561e353d2ae0 0x561e353d4f00 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1c8a191eeafe24a server_cookie=e12a9529e0a796f in_seq=0 out_seq=0 2024-06-10T01:08:23.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:23.592+0000 7f26ef7fe700 10 client.17493 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:23.849 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:08:23.849 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:08:24.103 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:08:24.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:24.277+0000 7f26ef7fe700 1 -- 172.21.15.37:0/948584178 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561e352a6390 con 0x561e353d2ae0 2024-06-10T01:08:24.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:24.277+0000 7f26ef7fe700 10 client.17493 handle_command_reply: tid=0 2024-06-10T01:08:24.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:24.277+0000 7f2720fedb80 1 client.17493 shutdown 2024-06-10T01:08:24.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:24.277+0000 7f2720fedb80 1 -- 172.21.15.37:0/948584178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e35352c40 msgr2=0x561e3536a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:24.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:24.277+0000 7f2720fedb80 1 --2- 172.21.15.37:0/948584178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e35352c40 0x561e3536a890 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f26f8006fd0 tx=0x7f26f8009ee0 comp rx=0 tx=0).stop 2024-06-10T01:08:24.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:24.277+0000 7f2720fedb80 1 -- 172.21.15.37:0/948584178 shutdown_connections 2024-06-10T01:08:24.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:24.277+0000 7f2720fedb80 1 --2- 172.21.15.37:0/948584178 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561e353d2ae0 0x561e353d4f00 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:24.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:24.277+0000 7f2720fedb80 1 --2- 172.21.15.37:0/948584178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561e35353560 0x561e3536add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:24.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:24.277+0000 7f2720fedb80 1 --2- 172.21.15.37:0/948584178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561e3534d3d0 0x561e3536c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:24.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:24.277+0000 7f26ef7fe700 0 client.17493 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:24.277+0000 7f2720fedb80 1 --2- 172.21.15.37:0/948584178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e35352c40 0x561e3536a890 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:24.277+0000 7f2720fedb80 1 -- 172.21.15.37:0/948584178 >> 172.21.15.37:0/948584178 conn(0x561e35277dc0 msgr2=0x561e35347830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:24.277+0000 7f2720fedb80 1 -- 172.21.15.37:0/948584178 shutdown_connections 2024-06-10T01:08:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:24.277+0000 7f2720fedb80 1 -- 172.21.15.37:0/948584178 wait complete. 2024-06-10T01:08:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:24.278+0000 7f2720fedb80 20 client.17493 trim_cache size 0 max 16384 2024-06-10T01:08:24.275 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:08:24.275 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 30, 2024-06-10T01:08:24.275 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24472, 2024-06-10T01:08:24.275 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 32, 2024-06-10T01:08:24.275 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-06-10T01:08:24.276 INFO:teuthology.orchestra.run.smithi037.stdout: "96a2bbd8": { 2024-06-10T01:08:24.276 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 32, 2024-06-10T01:08:24.276 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-06-10T01:08:24.276 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:08:24.276 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-06-10T01:08:24.276 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:08:24.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:08:24.276 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-06-10T01:08:24.276 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 137.8, 2024-06-10T01:08:24.276 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-06-10T01:08:24.276 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-06-10T01:08:24.276 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-06-10T01:08:24.276 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:08:24.276 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-06-10T01:08:24.277 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:08:24.277 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:08:24.277 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:08:24.277 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:08:24.277 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:08:24.277 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:08:24.277 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:08:24.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:24.278+0000 7f271676a700 1 -- 172.21.15.37:0/3084952224 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f26fc0a2c20 msgr2=0x7f26fc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:24.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:24.278+0000 7f271676a700 1 --2- 172.21.15.37:0/3084952224 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f26fc0a2c20 0x7f26fc0a50e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2700000c00 tx=0x7f2700005fb0 comp rx=0 tx=0).stop 2024-06-10T01:08:24.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:24.278+0000 7f271676a700 1 -- 172.21.15.37:0/3084952224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2710107860 msgr2=0x7f2710075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:24.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:24.278+0000 7f271676a700 1 --2- 172.21.15.37:0/3084952224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2710107860 0x7f2710075210 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f2708007ee0 tx=0x7f270800d3b0 comp rx=0 tx=0).stop 2024-06-10T01:08:24.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:24.278+0000 7f271676a700 1 -- 172.21.15.37:0/3084952224 shutdown_connections 2024-06-10T01:08:24.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:24.278+0000 7f271676a700 1 --2- 172.21.15.37:0/3084952224 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f26fc0a2c20 0x7f26fc0a50e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:24.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:24.278+0000 7f271676a700 1 --2- 172.21.15.37:0/3084952224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2710107860 0x7f2710075210 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:24.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:24.278+0000 7f271676a700 1 --2- 172.21.15.37:0/3084952224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2710102770 0x7f2710074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:24.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:24.278+0000 7f271676a700 1 --2- 172.21.15.37:0/3084952224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2710103120 0x7f2710074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:24.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:24.278+0000 7f271676a700 1 -- 172.21.15.37:0/3084952224 >> 172.21.15.37:0/3084952224 conn(0x7f2710079600 msgr2=0x7f27101006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:24.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:24.278+0000 7f271676a700 1 -- 172.21.15.37:0/3084952224 shutdown_connections 2024-06-10T01:08:24.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:24.278+0000 7f271676a700 1 -- 172.21.15.37:0/3084952224 wait complete. 2024-06-10T01:08:24.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:24 smithi167 ceph-mon[36917]: pgmap v782: 129 pgs: 129 active+clean; 8.7 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 169 KiB/s wr, 321 op/s 2024-06-10T01:08:24.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:24 smithi094 ceph-mon[36708]: pgmap v782: 129 pgs: 129 active+clean; 8.7 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 169 KiB/s wr, 321 op/s 2024-06-10T01:08:24.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:24 smithi037 ceph-mon[32192]: pgmap v782: 129 pgs: 129 active+clean; 8.7 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 169 KiB/s wr, 321 op/s 2024-06-10T01:08:25.489 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '96a2bbd8', total seconds elapsed: 27.0 2024-06-10T01:08:26.345 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:26 smithi037 ceph-mon[32192]: pgmap v783: 129 pgs: 129 active+clean; 8.8 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 176 KiB/s wr, 274 op/s 2024-06-10T01:08:26.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.466+0000 7fc40775c700 1 -- 172.21.15.37:0/4055095040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc40010ef40 msgr2=0x7fc400111330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:26.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.466+0000 7fc40775c700 1 --2- 172.21.15.37:0/4055095040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc40010ef40 0x7fc400111330 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fc3f800b0a0 tx=0x7fc3f800b3b0 comp rx=0 tx=0).stop 2024-06-10T01:08:26.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.466+0000 7fc40775c700 1 -- 172.21.15.37:0/4055095040 shutdown_connections 2024-06-10T01:08:26.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.466+0000 7fc40775c700 1 --2- 172.21.15.37:0/4055095040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc40010ef40 0x7fc400111330 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:26.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.466+0000 7fc40775c700 1 --2- 172.21.15.37:0/4055095040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc40006a1a0 0x7fc40006a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:26.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.466+0000 7fc40775c700 1 --2- 172.21.15.37:0/4055095040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc40006aac0 0x7fc40010ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:26.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.466+0000 7fc40775c700 1 -- 172.21.15.37:0/4055095040 >> 172.21.15.37:0/4055095040 conn(0x7fc400078340 msgr2=0x7fc400078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:26.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.467+0000 7fc40775c700 1 -- 172.21.15.37:0/4055095040 shutdown_connections 2024-06-10T01:08:26.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.467+0000 7fc40775c700 1 -- 172.21.15.37:0/4055095040 wait complete. 2024-06-10T01:08:26.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.467+0000 7fc40775c700 1 Processor -- start 2024-06-10T01:08:26.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.467+0000 7fc40775c700 1 -- start start 2024-06-10T01:08:26.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.468+0000 7fc40775c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc40006a1a0 0x7fc40019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:26.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.468+0000 7fc40775c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc40006aac0 0x7fc40019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:26.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.468+0000 7fc40775c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc40010ef40 0x7fc40019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:26.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.468+0000 7fc40775c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc4001149a0 con 0x7fc40006aac0 2024-06-10T01:08:26.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.468+0000 7fc40775c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc400114820 con 0x7fc40010ef40 2024-06-10T01:08:26.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.468+0000 7fc40775c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc400114b20 con 0x7fc40006a1a0 2024-06-10T01:08:26.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.468+0000 7fc404cf7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc40006aac0 0x7fc40019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:26.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.468+0000 7fc4054f8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc40006a1a0 0x7fc40019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:26.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.468+0000 7fc404cf7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc40006aac0 0x7fc40019b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50632/0 (socket says 172.21.15.37:50632) 2024-06-10T01:08:26.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.468+0000 7fc404cf7700 1 -- 172.21.15.37:0/4169219031 learned_addr learned my addr 172.21.15.37:0/4169219031 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:26.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.468+0000 7fc405cf9700 1 --2- 172.21.15.37:0/4169219031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc40010ef40 0x7fc40019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:26.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.468+0000 7fc404cf7700 1 -- 172.21.15.37:0/4169219031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc40006a1a0 msgr2=0x7fc40019ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:26.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.468+0000 7fc404cf7700 1 --2- 172.21.15.37:0/4169219031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc40006a1a0 0x7fc40019ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:26.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.468+0000 7fc404cf7700 1 -- 172.21.15.37:0/4169219031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc40010ef40 msgr2=0x7fc40019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:26.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.468+0000 7fc404cf7700 1 --2- 172.21.15.37:0/4169219031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc40010ef40 0x7fc40019f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:26.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.468+0000 7fc404cf7700 1 -- 172.21.15.37:0/4169219031 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40019fe60 con 0x7fc40006aac0 2024-06-10T01:08:26.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.468+0000 7fc4054f8700 1 --2- 172.21.15.37:0/4169219031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc40006a1a0 0x7fc40019ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:26.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.468+0000 7fc404cf7700 1 --2- 172.21.15.37:0/4169219031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc40006aac0 0x7fc40019b390 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fc3f000e9c0 tx=0x7fc3f000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:26.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.469+0000 7fc3f67fc700 1 -- 172.21.15.37:0/4169219031 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f000cbd0 con 0x7fc40006aac0 2024-06-10T01:08:26.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.469+0000 7fc3f67fc700 1 -- 172.21.15.37:0/4169219031 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3f000ee60 con 0x7fc40006aac0 2024-06-10T01:08:26.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.469+0000 7fc40775c700 1 -- 172.21.15.37:0/4169219031 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4001a0150 con 0x7fc40006aac0 2024-06-10T01:08:26.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.469+0000 7fc3f67fc700 1 -- 172.21.15.37:0/4169219031 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0005630 con 0x7fc40006aac0 2024-06-10T01:08:26.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.469+0000 7fc40775c700 1 -- 172.21.15.37:0/4169219031 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4001a79d0 con 0x7fc40006aac0 2024-06-10T01:08:26.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.471+0000 7fc3f67fc700 1 -- 172.21.15.37:0/4169219031 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc3f0004750 con 0x7fc40006aac0 2024-06-10T01:08:26.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.472+0000 7fc3f67fc700 1 --2- 172.21.15.37:0/4169219031 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3ec0a2ca0 0x7fc3ec0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:26.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.472+0000 7fc4054f8700 1 --2- 172.21.15.37:0/4169219031 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3ec0a2ca0 0x7fc3ec0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:26.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.472+0000 7fc3f67fc700 1 -- 172.21.15.37:0/4169219031 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fc3f00cf5b0 con 0x7fc40006aac0 2024-06-10T01:08:26.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.473+0000 7fc4054f8700 1 --2- 172.21.15.37:0/4169219031 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3ec0a2ca0 0x7fc3ec0a5160 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc005950 tx=0x7fc3fc00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:26.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.474+0000 7fc411fdfb80 1 Processor -- start 2024-06-10T01:08:26.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.474+0000 7fc411fdfb80 1 -- start start 2024-06-10T01:08:26.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.474+0000 7fc411fdfb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ef30ac6c0 0x557ef30acaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:26.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.474+0000 7fc411fdfb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ef30acfe0 0x557ef30a6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:26.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.474+0000 7fc411fdfb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ef30a6fa0 0x557ef30a73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:26.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.474+0000 7fc411fdfb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557ef2ff8890 con 0x557ef30acfe0 2024-06-10T01:08:26.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.474+0000 7fc411fdfb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557ef2fa07a0 con 0x557ef30ac6c0 2024-06-10T01:08:26.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.474+0000 7fc411fdfb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557ef2ff4850 con 0x557ef30a6fa0 2024-06-10T01:08:26.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.474+0000 7fc404cf7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ef30acfe0 0x557ef30a6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:26.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.474+0000 7fc404cf7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ef30acfe0 0x557ef30a6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50638/0 (socket says 172.21.15.37:50638) 2024-06-10T01:08:26.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.474+0000 7fc404cf7700 1 -- 172.21.15.37:0/1818507292 learned_addr learned my addr 172.21.15.37:0/1818507292 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:26.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.474+0000 7fc405cf9700 1 --2- 172.21.15.37:0/1818507292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ef30a6fa0 0x557ef30a73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:26.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.474+0000 7fc4054f8700 1 --2- 172.21.15.37:0/1818507292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ef30ac6c0 0x557ef30acaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:26.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.474+0000 7fc404cf7700 1 -- 172.21.15.37:0/1818507292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ef30a6fa0 msgr2=0x557ef30a73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:26.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.474+0000 7fc404cf7700 1 --2- 172.21.15.37:0/1818507292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ef30a6fa0 0x557ef30a73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:26.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.474+0000 7fc404cf7700 1 -- 172.21.15.37:0/1818507292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ef30ac6c0 msgr2=0x557ef30acaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:26.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.474+0000 7fc404cf7700 1 --2- 172.21.15.37:0/1818507292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ef30ac6c0 0x557ef30acaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:26.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.474+0000 7fc404cf7700 1 -- 172.21.15.37:0/1818507292 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ef2f96fd0 con 0x557ef30acfe0 2024-06-10T01:08:26.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.474+0000 7fc405cf9700 1 --2- 172.21.15.37:0/1818507292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ef30a6fa0 0x557ef30a73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:26.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.475+0000 7fc404cf7700 1 --2- 172.21.15.37:0/1818507292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ef30acfe0 0x557ef30a6a60 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0097e80 tx=0x7fc3f009b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aaa9264fe385d33 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:26.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.475+0000 7fc3eb7fe700 1 -- 172.21.15.37:0/1818507292 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f009fdd0 con 0x557ef30acfe0 2024-06-10T01:08:26.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.475+0000 7fc3eb7fe700 1 -- 172.21.15.37:0/1818507292 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3f00ce070 con 0x557ef30acfe0 2024-06-10T01:08:26.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.475+0000 7fc3eb7fe700 1 -- 172.21.15.37:0/1818507292 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f009bee0 con 0x557ef30acfe0 2024-06-10T01:08:26.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.475+0000 7fc411fdfb80 1 -- 172.21.15.37:0/1818507292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ef30acfe0 msgr2=0x557ef30a6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:26.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.475+0000 7fc411fdfb80 1 --2- 172.21.15.37:0/1818507292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ef30acfe0 0x557ef30a6a60 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0097e80 tx=0x7fc3f009b400 comp rx=0 tx=0).stop 2024-06-10T01:08:26.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.476+0000 7fc411fdfb80 1 -- 172.21.15.37:0/1818507292 shutdown_connections 2024-06-10T01:08:26.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.476+0000 7fc411fdfb80 1 --2- 172.21.15.37:0/1818507292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ef30a6fa0 0x557ef30a73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:26.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.476+0000 7fc411fdfb80 1 --2- 172.21.15.37:0/1818507292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ef30ac6c0 0x557ef30acaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:26.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.476+0000 7fc411fdfb80 1 --2- 172.21.15.37:0/1818507292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ef30acfe0 0x557ef30a6a60 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:26.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.476+0000 7fc411fdfb80 1 -- 172.21.15.37:0/1818507292 >> 172.21.15.37:0/1818507292 conn(0x557ef2fd1d40 msgr2=0x557ef30a18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:26.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.476+0000 7fc411fdfb80 1 -- 172.21.15.37:0/1818507292 shutdown_connections 2024-06-10T01:08:26.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.476+0000 7fc411fdfb80 1 -- 172.21.15.37:0/1818507292 wait complete. 2024-06-10T01:08:26.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.476+0000 7fc411fdfb80 1 -- start start 2024-06-10T01:08:26.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.476+0000 7fc411fdfb80 1 Processor -- start 2024-06-10T01:08:26.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.477+0000 7fc411fdfb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ef30a6fa0 0x557ef30b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:26.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.477+0000 7fc411fdfb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ef30ac6c0 0x557ef30b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:26.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.477+0000 7fc411fdfb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ef30acfe0 0x557ef30b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:26.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.477+0000 7fc411fdfb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557ef30b3a50 con 0x557ef30ac6c0 2024-06-10T01:08:26.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.477+0000 7fc411fdfb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557ef2f98c70 con 0x557ef30acfe0 2024-06-10T01:08:26.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.477+0000 7fc411fdfb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557ef30b3bd0 con 0x557ef30a6fa0 2024-06-10T01:08:26.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.477+0000 7fc411fdfb80 10 client.0 _ll_register_callbacks cb 0x557ef2f929a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:26.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.477+0000 7fc4054f8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ef30a6fa0 0x557ef30b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:26.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.477+0000 7fc404cf7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ef30ac6c0 0x557ef30b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:26.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.477+0000 7fc404cf7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ef30ac6c0 0x557ef30b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50644/0 (socket says 172.21.15.37:50644) 2024-06-10T01:08:26.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.477+0000 7fc4054f8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ef30a6fa0 0x557ef30b2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:58920/0 (socket says 172.21.15.37:58920) 2024-06-10T01:08:26.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.477+0000 7fc404cf7700 1 -- 172.21.15.37:0/3666190225 learned_addr learned my addr 172.21.15.37:0/3666190225 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:26.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.477+0000 7fc405cf9700 1 --2- 172.21.15.37:0/3666190225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ef30acfe0 0x557ef30b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:26.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.477+0000 7fc404cf7700 1 -- 172.21.15.37:0/3666190225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ef30a6fa0 msgr2=0x557ef30b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:26.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.477+0000 7fc404cf7700 1 --2- 172.21.15.37:0/3666190225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ef30a6fa0 0x557ef30b2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:26.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.477+0000 7fc404cf7700 1 -- 172.21.15.37:0/3666190225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ef30acfe0 msgr2=0x557ef30b05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:26.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.477+0000 7fc404cf7700 1 --2- 172.21.15.37:0/3666190225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ef30acfe0 0x557ef30b05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:26.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.477+0000 7fc404cf7700 1 -- 172.21.15.37:0/3666190225 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ef2f47950 con 0x557ef30ac6c0 2024-06-10T01:08:26.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.477+0000 7fc4054f8700 1 --2- 172.21.15.37:0/3666190225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ef30a6fa0 0x557ef30b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:26.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.477+0000 7fc404cf7700 1 --2- 172.21.15.37:0/3666190225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ef30ac6c0 0x557ef30b0090 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc3f00a4160 tx=0x7fc3f0099f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9448a3cb88d2def8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:26.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.477+0000 7fc3e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:08:26.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.477+0000 7fc3e8ff9700 1 -- 172.21.15.37:0/3666190225 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0080060 con 0x557ef30ac6c0 2024-06-10T01:08:26.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.477+0000 7fc3e8ff9700 1 -- 172.21.15.37:0/3666190225 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fc3f009be90 con 0x557ef30ac6c0 2024-06-10T01:08:26.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.478+0000 7fc411fdfb80 1 -- 172.21.15.37:0/3666190225 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3f00ce070 con 0x557ef30ac6c0 2024-06-10T01:08:26.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.478+0000 7fc3e8ff9700 1 -- 172.21.15.37:0/3666190225 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc3f009fdd0 con 0x557ef30ac6c0 2024-06-10T01:08:26.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.478+0000 7fc411fdfb80 10 client.17505 fetch_fsmap learned FSMap version 31 2024-06-10T01:08:26.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.478+0000 7fc411fdfb80 1 -- 172.21.15.37:0/3666190225 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557ef30be400 con 0x557ef30ac6c0 2024-06-10T01:08:26.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.478+0000 7fc3e8ff9700 1 -- 172.21.15.37:0/3666190225 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3f00ce070 con 0x557ef30ac6c0 2024-06-10T01:08:26.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.478+0000 7fc3e8ff9700 1 -- 172.21.15.37:0/3666190225 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fc3f0016be0 con 0x557ef30ac6c0 2024-06-10T01:08:26.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.478+0000 7fc411fdfb80 10 client.17505 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:08:26.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.478+0000 7fc411fdfb80 10 client.17505 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:26.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.478+0000 7fc411fdfb80 20 client.17505 populate_metadata read hostname 'smithi037' 2024-06-10T01:08:26.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.478+0000 7fc411fdfb80 1 --2- 172.21.15.37:0/3666190225 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557ef30c08d0 0x557ef30c2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:26.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.478+0000 7fc411fdfb80 4 client.17505 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:08:26.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.478+0000 7fc411fdfb80 1 -- 172.21.15.37:0/3666190225 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ef3005d90 con 0x557ef30c08d0 2024-06-10T01:08:26.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.479+0000 7fc4054f8700 1 --2- 172.21.15.37:0/3666190225 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557ef30c08d0 0x557ef30c2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:26.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.479+0000 7fc4054f8700 1 --2- 172.21.15.37:0/3666190225 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557ef30c08d0 0x557ef30c2cc0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7a512090df666c server_cookie=faf3645f16bbb5a8 in_seq=0 out_seq=0 2024-06-10T01:08:26.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.479+0000 7fc3e8ff9700 10 client.17505 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:26.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.489+0000 7fc3e8ff9700 1 -- 172.21.15.37:0/3666190225 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557ef3005d90 con 0x557ef30c08d0 2024-06-10T01:08:26.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.489+0000 7fc3e8ff9700 10 client.17505 handle_command_reply: tid=0 2024-06-10T01:08:26.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.489+0000 7fc411fdfb80 1 client.17505 shutdown 2024-06-10T01:08:26.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.489+0000 7fc411fdfb80 1 -- 172.21.15.37:0/3666190225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ef30ac6c0 msgr2=0x557ef30b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:26.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.489+0000 7fc411fdfb80 1 --2- 172.21.15.37:0/3666190225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ef30ac6c0 0x557ef30b0090 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc3f00a4160 tx=0x7fc3f0099f30 comp rx=0 tx=0).stop 2024-06-10T01:08:26.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.489+0000 7fc411fdfb80 1 -- 172.21.15.37:0/3666190225 shutdown_connections 2024-06-10T01:08:26.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.489+0000 7fc411fdfb80 1 --2- 172.21.15.37:0/3666190225 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557ef30c08d0 0x557ef30c2cc0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:26.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.489+0000 7fc411fdfb80 1 --2- 172.21.15.37:0/3666190225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ef30acfe0 0x557ef30b05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:26.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.489+0000 7fc411fdfb80 1 --2- 172.21.15.37:0/3666190225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ef30a6fa0 0x557ef30b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:26.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.489+0000 7fc411fdfb80 1 --2- 172.21.15.37:0/3666190225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ef30ac6c0 0x557ef30b0090 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:26.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.489+0000 7fc411fdfb80 1 -- 172.21.15.37:0/3666190225 >> 172.21.15.37:0/3666190225 conn(0x557ef2fd1d40 msgr2=0x557ef30a5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:26.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.489+0000 7fc3e8ff9700 0 client.17505 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:26.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.490+0000 7fc411fdfb80 1 -- 172.21.15.37:0/3666190225 shutdown_connections 2024-06-10T01:08:26.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.490+0000 7fc411fdfb80 1 -- 172.21.15.37:0/3666190225 wait complete. 2024-06-10T01:08:26.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.490+0000 7fc411fdfb80 20 client.17505 trim_cache size 0 max 16384 2024-06-10T01:08:26.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.511+0000 7fc411fdfb80 1 Processor -- start 2024-06-10T01:08:26.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.511+0000 7fc411fdfb80 1 -- start start 2024-06-10T01:08:26.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.511+0000 7fc411fdfb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ef30a6fa0 0x557ef30a6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:26.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.511+0000 7fc411fdfb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ef30ac6c0 0x557ef309b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:26.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.511+0000 7fc411fdfb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ef30acfe0 0x557ef30ad480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:26.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.511+0000 7fc411fdfb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc4001b2f10 con 0x557ef30acfe0 2024-06-10T01:08:26.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.511+0000 7fc411fdfb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc4001b07a0 con 0x557ef30ac6c0 2024-06-10T01:08:26.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.511+0000 7fc411fdfb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc4001ae5f0 con 0x557ef30a6fa0 2024-06-10T01:08:26.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.512+0000 7fc405cf9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ef30acfe0 0x557ef30ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:26.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.512+0000 7fc4054f8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ef30a6fa0 0x557ef30a6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:26.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.512+0000 7fc405cf9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ef30acfe0 0x557ef30ad480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50654/0 (socket says 172.21.15.37:50654) 2024-06-10T01:08:26.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.512+0000 7fc4054f8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ef30a6fa0 0x557ef30a6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:58928/0 (socket says 172.21.15.37:58928) 2024-06-10T01:08:26.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.512+0000 7fc405cf9700 1 -- 172.21.15.37:0/1981247799 learned_addr learned my addr 172.21.15.37:0/1981247799 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:26.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.512+0000 7fc404cf7700 1 --2- 172.21.15.37:0/1981247799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ef30ac6c0 0x557ef309b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:26.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.512+0000 7fc405cf9700 1 -- 172.21.15.37:0/1981247799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ef30a6fa0 msgr2=0x557ef30a6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:26.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.512+0000 7fc405cf9700 1 --2- 172.21.15.37:0/1981247799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ef30a6fa0 0x557ef30a6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:26.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.512+0000 7fc405cf9700 1 -- 172.21.15.37:0/1981247799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ef30ac6c0 msgr2=0x557ef309b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:26.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.512+0000 7fc405cf9700 1 --2- 172.21.15.37:0/1981247799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ef30ac6c0 0x557ef309b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:26.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.512+0000 7fc405cf9700 1 -- 172.21.15.37:0/1981247799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ef30b3c20 con 0x557ef30acfe0 2024-06-10T01:08:26.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.512+0000 7fc404cf7700 1 --2- 172.21.15.37:0/1981247799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ef30ac6c0 0x557ef309b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:08:26.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.512+0000 7fc4054f8700 1 --2- 172.21.15.37:0/1981247799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ef30a6fa0 0x557ef30a6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:26.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.512+0000 7fc405cf9700 1 --2- 172.21.15.37:0/1981247799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ef30acfe0 0x557ef30ad480 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8015000 tx=0x7fc3f800bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae889a3a5a6c09a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:26.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.512+0000 7fc3eaffd700 1 -- 172.21.15.37:0/1981247799 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8003a00 con 0x557ef30acfe0 2024-06-10T01:08:26.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.512+0000 7fc3eaffd700 1 -- 172.21.15.37:0/1981247799 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3f8003ba0 con 0x557ef30acfe0 2024-06-10T01:08:26.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.512+0000 7fc3eaffd700 1 -- 172.21.15.37:0/1981247799 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f80276c0 con 0x557ef30acfe0 2024-06-10T01:08:26.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.513+0000 7fc411fdfb80 1 -- 172.21.15.37:0/1981247799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ef30acfe0 msgr2=0x557ef30ad480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:26.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.513+0000 7fc411fdfb80 1 --2- 172.21.15.37:0/1981247799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ef30acfe0 0x557ef30ad480 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8015000 tx=0x7fc3f800bcf0 comp rx=0 tx=0).stop 2024-06-10T01:08:26.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.513+0000 7fc411fdfb80 1 -- 172.21.15.37:0/1981247799 shutdown_connections 2024-06-10T01:08:26.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.513+0000 7fc411fdfb80 1 --2- 172.21.15.37:0/1981247799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ef30acfe0 0x557ef30ad480 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:26.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.513+0000 7fc411fdfb80 1 --2- 172.21.15.37:0/1981247799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ef30a6fa0 0x557ef30a6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:26.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.513+0000 7fc411fdfb80 1 --2- 172.21.15.37:0/1981247799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ef30ac6c0 0x557ef309b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:26.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.513+0000 7fc411fdfb80 1 -- 172.21.15.37:0/1981247799 >> 172.21.15.37:0/1981247799 conn(0x557ef2fd1d40 msgr2=0x557ef309ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:26.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.513+0000 7fc411fdfb80 1 -- 172.21.15.37:0/1981247799 shutdown_connections 2024-06-10T01:08:26.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.513+0000 7fc411fdfb80 1 -- 172.21.15.37:0/1981247799 wait complete. 2024-06-10T01:08:26.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.514+0000 7fc411fdfb80 1 -- start start 2024-06-10T01:08:26.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.514+0000 7fc411fdfb80 1 Processor -- start 2024-06-10T01:08:26.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.514+0000 7fc411fdfb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ef30a6fa0 0x557ef30c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:26.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.514+0000 7fc411fdfb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ef30ac6c0 0x557ef30c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:26.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.514+0000 7fc411fdfb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ef30acfe0 0x557ef3124250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:26.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.514+0000 7fc411fdfb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc4001aec60 con 0x557ef30ac6c0 2024-06-10T01:08:26.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.514+0000 7fc411fdfb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc4001b00d0 con 0x557ef30acfe0 2024-06-10T01:08:26.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.514+0000 7fc411fdfb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc4001af330 con 0x557ef30a6fa0 2024-06-10T01:08:26.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.514+0000 7fc411fdfb80 10 client.0 _ll_register_callbacks cb 0x557ef2f929a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:26.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.514+0000 7fc4054f8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ef30a6fa0 0x557ef30c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:26.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.514+0000 7fc404cf7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ef30ac6c0 0x557ef30c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:26.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.514+0000 7fc404cf7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ef30ac6c0 0x557ef30c44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50664/0 (socket says 172.21.15.37:50664) 2024-06-10T01:08:26.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.514+0000 7fc4054f8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ef30a6fa0 0x557ef30c3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:58930/0 (socket says 172.21.15.37:58930) 2024-06-10T01:08:26.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.514+0000 7fc404cf7700 1 -- 172.21.15.37:0/4151345570 learned_addr learned my addr 172.21.15.37:0/4151345570 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:26.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.514+0000 7fc405cf9700 1 --2- 172.21.15.37:0/4151345570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ef30acfe0 0x557ef3124250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:26.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.515+0000 7fc404cf7700 1 -- 172.21.15.37:0/4151345570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ef30a6fa0 msgr2=0x557ef30c3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:26.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.515+0000 7fc404cf7700 1 --2- 172.21.15.37:0/4151345570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ef30a6fa0 0x557ef30c3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:26.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.515+0000 7fc404cf7700 1 -- 172.21.15.37:0/4151345570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ef30acfe0 msgr2=0x557ef3124250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:26.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.515+0000 7fc404cf7700 1 --2- 172.21.15.37:0/4151345570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ef30acfe0 0x557ef3124250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:26.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.515+0000 7fc404cf7700 1 -- 172.21.15.37:0/4151345570 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ef3124840 con 0x557ef30ac6c0 2024-06-10T01:08:26.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.515+0000 7fc405cf9700 1 --2- 172.21.15.37:0/4151345570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ef30acfe0 0x557ef3124250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:08:26.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.515+0000 7fc4054f8700 1 --2- 172.21.15.37:0/4151345570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ef30a6fa0 0x557ef30c3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:26.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.515+0000 7fc404cf7700 1 --2- 172.21.15.37:0/4151345570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ef30ac6c0 0x557ef30c44d0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fc3f00ba5b0 tx=0x7fc3f0097d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9014643a176fb5 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:26.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.515+0000 7fc3e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:08:26.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.515+0000 7fc3e8ff9700 1 -- 172.21.15.37:0/4151345570 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f00b8070 con 0x557ef30ac6c0 2024-06-10T01:08:26.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.515+0000 7fc3e8ff9700 1 -- 172.21.15.37:0/4151345570 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fc3f00ba190 con 0x557ef30ac6c0 2024-06-10T01:08:26.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.515+0000 7fc411fdfb80 1 -- 172.21.15.37:0/4151345570 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3f8003ba0 con 0x557ef30ac6c0 2024-06-10T01:08:26.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.516+0000 7fc3e8ff9700 1 -- 172.21.15.37:0/4151345570 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc3f0080040 con 0x557ef30ac6c0 2024-06-10T01:08:26.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.516+0000 7fc411fdfb80 10 client.17511 fetch_fsmap learned FSMap version 31 2024-06-10T01:08:26.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.516+0000 7fc411fdfb80 1 -- 172.21.15.37:0/4151345570 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557ef3124840 con 0x557ef30ac6c0 2024-06-10T01:08:26.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.516+0000 7fc3e8ff9700 1 -- 172.21.15.37:0/4151345570 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3f00ce040 con 0x557ef30ac6c0 2024-06-10T01:08:26.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.516+0000 7fc3e8ff9700 1 -- 172.21.15.37:0/4151345570 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fc3f00977f0 con 0x557ef30ac6c0 2024-06-10T01:08:26.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.516+0000 7fc411fdfb80 10 client.17511 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:08:26.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.516+0000 7fc411fdfb80 10 client.17511 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:26.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.516+0000 7fc411fdfb80 20 client.17511 populate_metadata read hostname 'smithi037' 2024-06-10T01:08:26.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.516+0000 7fc411fdfb80 1 --2- 172.21.15.37:0/4151345570 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557ef312beb0 0x557ef312e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:26.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.516+0000 7fc411fdfb80 4 client.17511 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:08:26.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.516+0000 7fc411fdfb80 1 -- 172.21.15.37:0/4151345570 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fc3f8003ba0 con 0x557ef312beb0 2024-06-10T01:08:26.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.517+0000 7fc4054f8700 1 --2- 172.21.15.37:0/4151345570 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557ef312beb0 0x557ef312e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:26.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.517+0000 7fc4054f8700 1 --2- 172.21.15.37:0/4151345570 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557ef312beb0 0x557ef312e2d0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4bee10d4c596aef server_cookie=21959bc343f4751d in_seq=0 out_seq=0 2024-06-10T01:08:26.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:26.517+0000 7fc3e8ff9700 10 client.17511 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:26.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:26 smithi167 ceph-mon[36917]: pgmap v783: 129 pgs: 129 active+clean; 8.8 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 176 KiB/s wr, 274 op/s 2024-06-10T01:08:26.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:26 smithi094 ceph-mon[36708]: pgmap v783: 129 pgs: 129 active+clean; 8.8 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 176 KiB/s wr, 274 op/s 2024-06-10T01:08:27.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:27.680+0000 7fc3e8ff9700 1 -- 172.21.15.37:0/4151345570 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fc3f8003ba0 con 0x557ef312beb0 2024-06-10T01:08:27.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:27.681+0000 7fc3e8ff9700 10 client.17511 handle_command_reply: tid=0 2024-06-10T01:08:27.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:27.681+0000 7fc411fdfb80 1 client.17511 shutdown 2024-06-10T01:08:27.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:27.681+0000 7fc411fdfb80 1 -- 172.21.15.37:0/4151345570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ef30ac6c0 msgr2=0x557ef30c44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:27.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:27.681+0000 7fc411fdfb80 1 --2- 172.21.15.37:0/4151345570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ef30ac6c0 0x557ef30c44d0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fc3f00ba5b0 tx=0x7fc3f0097d20 comp rx=0 tx=0).stop 2024-06-10T01:08:27.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:27.681+0000 7fc411fdfb80 1 -- 172.21.15.37:0/4151345570 shutdown_connections 2024-06-10T01:08:27.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:27.681+0000 7fc411fdfb80 1 --2- 172.21.15.37:0/4151345570 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x557ef312beb0 0x557ef312e2d0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:27.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:27.681+0000 7fc411fdfb80 1 --2- 172.21.15.37:0/4151345570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ef30acfe0 0x557ef3124250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:27.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:27.681+0000 7fc411fdfb80 1 --2- 172.21.15.37:0/4151345570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557ef30a6fa0 0x557ef30c3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:27.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:27.681+0000 7fc411fdfb80 1 --2- 172.21.15.37:0/4151345570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ef30ac6c0 0x557ef30c44d0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:27.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:27.681+0000 7fc3e8ff9700 0 client.17511 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:27.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:27.681+0000 7fc411fdfb80 1 -- 172.21.15.37:0/4151345570 >> 172.21.15.37:0/4151345570 conn(0x557ef2fd1d40 msgr2=0x557ef30ad400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:27.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:27.681+0000 7fc411fdfb80 1 -- 172.21.15.37:0/4151345570 shutdown_connections 2024-06-10T01:08:27.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:27.681+0000 7fc411fdfb80 1 -- 172.21.15.37:0/4151345570 wait complete. 2024-06-10T01:08:27.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:27.681+0000 7fc411fdfb80 20 client.17511 trim_cache size 0 max 16384 2024-06-10T01:08:27.678 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:08:27.678 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-06-10T01:08:27.679 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "e78a272e-e18b-41f0-b3ca-e304eb51fcfe", 2024-06-10T01:08:27.679 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-06-10T01:08:27.679 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:08:27.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:27.682+0000 7fc40775c700 1 -- 172.21.15.37:0/4169219031 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3ec0a2ca0 msgr2=0x7fc3ec0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:27.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:27.682+0000 7fc40775c700 1 --2- 172.21.15.37:0/4169219031 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3ec0a2ca0 0x7fc3ec0a5160 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc005950 tx=0x7fc3fc00b410 comp rx=0 tx=0).stop 2024-06-10T01:08:27.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:27.682+0000 7fc40775c700 1 -- 172.21.15.37:0/4169219031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc40006aac0 msgr2=0x7fc40019b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:27.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:27.682+0000 7fc40775c700 1 --2- 172.21.15.37:0/4169219031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc40006aac0 0x7fc40019b390 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fc3f000e9c0 tx=0x7fc3f000ecd0 comp rx=0 tx=0).stop 2024-06-10T01:08:27.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:27.682+0000 7fc40775c700 1 -- 172.21.15.37:0/4169219031 shutdown_connections 2024-06-10T01:08:27.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:27.682+0000 7fc40775c700 1 --2- 172.21.15.37:0/4169219031 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc3ec0a2ca0 0x7fc3ec0a5160 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:27.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:27.682+0000 7fc40775c700 1 --2- 172.21.15.37:0/4169219031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc40010ef40 0x7fc40019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:27.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:27.682+0000 7fc40775c700 1 --2- 172.21.15.37:0/4169219031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc40006a1a0 0x7fc40019ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:27.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:27.682+0000 7fc40775c700 1 --2- 172.21.15.37:0/4169219031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc40006aac0 0x7fc40019b390 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:27.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:27.682+0000 7fc40775c700 1 -- 172.21.15.37:0/4169219031 >> 172.21.15.37:0/4169219031 conn(0x7fc400078340 msgr2=0x7fc4001103b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:27.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:27.682+0000 7fc40775c700 1 -- 172.21.15.37:0/4169219031 shutdown_connections 2024-06-10T01:08:27.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:27.682+0000 7fc40775c700 1 -- 172.21.15.37:0/4169219031 wait complete. 2024-06-10T01:08:28.314 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:08:28.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:28 smithi167 ceph-mon[36917]: pgmap v784: 129 pgs: 129 active+clean; 4.8 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 177 KiB/s wr, 205 op/s 2024-06-10T01:08:28.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:28 smithi167 ceph-mon[36917]: scrub queued for path: / 2024-06-10T01:08:28.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:28 smithi167 ceph-mon[36917]: scrub summary: idle+waiting paths [/] 2024-06-10T01:08:28.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:28 smithi167 ceph-mon[36917]: scrub summary: active paths [/] 2024-06-10T01:08:28.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:28 smithi094 ceph-mon[36708]: pgmap v784: 129 pgs: 129 active+clean; 4.8 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 177 KiB/s wr, 205 op/s 2024-06-10T01:08:28.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:28 smithi094 ceph-mon[36708]: scrub queued for path: / 2024-06-10T01:08:28.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:28 smithi094 ceph-mon[36708]: scrub summary: idle+waiting paths [/] 2024-06-10T01:08:28.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:28 smithi094 ceph-mon[36708]: scrub summary: active paths [/] 2024-06-10T01:08:28.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:28 smithi037 ceph-mon[32192]: pgmap v784: 129 pgs: 129 active+clean; 4.8 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 177 KiB/s wr, 205 op/s 2024-06-10T01:08:28.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:28 smithi037 ceph-mon[32192]: scrub queued for path: / 2024-06-10T01:08:28.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:28 smithi037 ceph-mon[32192]: scrub summary: idle+waiting paths [/] 2024-06-10T01:08:28.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:28 smithi037 ceph-mon[32192]: scrub summary: active paths [/] 2024-06-10T01:08:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:08:30.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:30 smithi037 ceph-mon[32192]: pgmap v785: 129 pgs: 129 active+clean; 8.3 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 356 KiB/s wr, 222 op/s 2024-06-10T01:08:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.554+0000 7fa28f6d0700 1 -- 172.21.15.37:0/483041958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa288102530 msgr2=0x7fa2881029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.554+0000 7fa28f6d0700 1 --2- 172.21.15.37:0/483041958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa288102530 0x7fa2881029b0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa2780099c0 tx=0x7fa278009cd0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.554+0000 7fa28f6d0700 1 -- 172.21.15.37:0/483041958 shutdown_connections 2024-06-10T01:08:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.554+0000 7fa28f6d0700 1 --2- 172.21.15.37:0/483041958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa288102ef0 0x7fa28810f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.554+0000 7fa28f6d0700 1 --2- 172.21.15.37:0/483041958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2881085d0 0x7fa2881089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.554+0000 7fa28f6d0700 1 --2- 172.21.15.37:0/483041958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa288102530 0x7fa2881029b0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.554+0000 7fa28f6d0700 1 -- 172.21.15.37:0/483041958 >> 172.21.15.37:0/483041958 conn(0x7fa288079640 msgr2=0x7fa288079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:30.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.554+0000 7fa28f6d0700 1 -- 172.21.15.37:0/483041958 shutdown_connections 2024-06-10T01:08:30.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.554+0000 7fa28f6d0700 1 -- 172.21.15.37:0/483041958 wait complete. 2024-06-10T01:08:30.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.555+0000 7fa28f6d0700 1 Processor -- start 2024-06-10T01:08:30.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.555+0000 7fa28f6d0700 1 -- start start 2024-06-10T01:08:30.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.555+0000 7fa28f6d0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa288102530 0x7fa288074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:30.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.555+0000 7fa28f6d0700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa288102ef0 0x7fa288074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:30.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.555+0000 7fa28f6d0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2881085d0 0x7fa288075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:30.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.555+0000 7fa28f6d0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa288112820 con 0x7fa2881085d0 2024-06-10T01:08:30.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.555+0000 7fa28f6d0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa2881126a0 con 0x7fa288102530 2024-06-10T01:08:30.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.555+0000 7fa28f6d0700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa2881129a0 con 0x7fa288102ef0 2024-06-10T01:08:30.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.555+0000 7fa28dc6d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2881085d0 0x7fa288075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:30.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.555+0000 7fa28dc6d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2881085d0 0x7fa288075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50688/0 (socket says 172.21.15.37:50688) 2024-06-10T01:08:30.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.555+0000 7fa28dc6d700 1 -- 172.21.15.37:0/1042475579 learned_addr learned my addr 172.21.15.37:0/1042475579 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:30.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.555+0000 7fa28cc6b700 1 --2- 172.21.15.37:0/1042475579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa288102ef0 0x7fa288074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:30.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.555+0000 7fa28d46c700 1 --2- 172.21.15.37:0/1042475579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa288102530 0x7fa288074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:30.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.556+0000 7fa28dc6d700 1 -- 172.21.15.37:0/1042475579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa288102ef0 msgr2=0x7fa288074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:30.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.556+0000 7fa28dc6d700 1 --2- 172.21.15.37:0/1042475579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa288102ef0 0x7fa288074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.556+0000 7fa28dc6d700 1 -- 172.21.15.37:0/1042475579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa288102530 msgr2=0x7fa288074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:30.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.556+0000 7fa28dc6d700 1 --2- 172.21.15.37:0/1042475579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa288102530 0x7fa288074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.556+0000 7fa28dc6d700 1 -- 172.21.15.37:0/1042475579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2881a7120 con 0x7fa2881085d0 2024-06-10T01:08:30.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.556+0000 7fa28dc6d700 1 --2- 172.21.15.37:0/1042475579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2881085d0 0x7fa288075210 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fa28000ef30 tx=0x7fa28000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:30.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.556+0000 7fa27e7fc700 1 -- 172.21.15.37:0/1042475579 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2800092e0 con 0x7fa2881085d0 2024-06-10T01:08:30.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.556+0000 7fa27e7fc700 1 -- 172.21.15.37:0/1042475579 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa280004bb0 con 0x7fa2881085d0 2024-06-10T01:08:30.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.556+0000 7fa27e7fc700 1 -- 172.21.15.37:0/1042475579 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa280004280 con 0x7fa2881085d0 2024-06-10T01:08:30.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.556+0000 7fa28f6d0700 1 -- 172.21.15.37:0/1042475579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2881a73b0 con 0x7fa2881085d0 2024-06-10T01:08:30.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.556+0000 7fa28f6d0700 1 -- 172.21.15.37:0/1042475579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2881a7910 con 0x7fa2881085d0 2024-06-10T01:08:30.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.559+0000 7fa27e7fc700 1 -- 172.21.15.37:0/1042475579 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa28001b030 con 0x7fa2881085d0 2024-06-10T01:08:30.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.559+0000 7fa27e7fc700 1 --2- 172.21.15.37:0/1042475579 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa2740a2ca0 0x7fa2740a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:30.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.560+0000 7fa27e7fc700 1 -- 172.21.15.37:0/1042475579 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fa2800ceed0 con 0x7fa2881085d0 2024-06-10T01:08:30.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.560+0000 7fa28d46c700 1 --2- 172.21.15.37:0/1042475579 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa2740a2ca0 0x7fa2740a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:30.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.560+0000 7fa28d46c700 1 --2- 172.21.15.37:0/1042475579 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa2740a2ca0 0x7fa2740a5160 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa284007b60 tx=0x7fa2840073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:30.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.560+0000 7fa299f53b80 1 Processor -- start 2024-06-10T01:08:30.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.560+0000 7fa299f53b80 1 -- start start 2024-06-10T01:08:30.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.561+0000 7fa299f53b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b18cc45f60 0x55b18cc46340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:30.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.561+0000 7fa299f53b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b18cc46880 0x55b18cc40260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:30.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.561+0000 7fa299f53b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b18cc407a0 0x55b18cc40bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:30.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.561+0000 7fa299f53b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b18cb32f60 con 0x55b18cc45f60 2024-06-10T01:08:30.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.561+0000 7fa299f53b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b18cb91850 con 0x55b18cc46880 2024-06-10T01:08:30.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.561+0000 7fa299f53b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b18caf9460 con 0x55b18cc407a0 2024-06-10T01:08:30.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.561+0000 7fa28dc6d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b18cc407a0 0x55b18cc40bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:30.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.561+0000 7fa28d46c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b18cc45f60 0x55b18cc46340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:30.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.561+0000 7fa28dc6d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b18cc407a0 0x55b18cc40bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:58962/0 (socket says 172.21.15.37:58962) 2024-06-10T01:08:30.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.561+0000 7fa28dc6d700 1 -- 172.21.15.37:0/2990800816 learned_addr learned my addr 172.21.15.37:0/2990800816 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:30.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.561+0000 7fa28cc6b700 1 --2- 172.21.15.37:0/2990800816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b18cc46880 0x55b18cc40260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:30.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.561+0000 7fa28dc6d700 1 -- 172.21.15.37:0/2990800816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b18cc46880 msgr2=0x55b18cc40260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:30.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.561+0000 7fa28dc6d700 1 --2- 172.21.15.37:0/2990800816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b18cc46880 0x55b18cc40260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.561+0000 7fa28dc6d700 1 -- 172.21.15.37:0/2990800816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b18cc45f60 msgr2=0x55b18cc46340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:30.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.561+0000 7fa28dc6d700 1 --2- 172.21.15.37:0/2990800816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b18cc45f60 0x55b18cc46340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.561+0000 7fa28dc6d700 1 -- 172.21.15.37:0/2990800816 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b18cae08d0 con 0x55b18cc407a0 2024-06-10T01:08:30.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.561+0000 7fa28dc6d700 1 --2- 172.21.15.37:0/2990800816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b18cc407a0 0x55b18cc40bc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa2800090a0 tx=0x7fa28000cf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=471aeaead3ff36d1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:30.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.562+0000 7fa2737fe700 1 -- 172.21.15.37:0/2990800816 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28009b370 con 0x55b18cc407a0 2024-06-10T01:08:30.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.562+0000 7fa2737fe700 1 -- 172.21.15.37:0/2990800816 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa28009f720 con 0x55b18cc407a0 2024-06-10T01:08:30.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.562+0000 7fa2737fe700 1 -- 172.21.15.37:0/2990800816 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2800a3d20 con 0x55b18cc407a0 2024-06-10T01:08:30.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.562+0000 7fa299f53b80 1 -- 172.21.15.37:0/2990800816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b18cc407a0 msgr2=0x55b18cc40bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:30.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.562+0000 7fa299f53b80 1 --2- 172.21.15.37:0/2990800816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b18cc407a0 0x55b18cc40bc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa2800090a0 tx=0x7fa28000cf00 comp rx=0 tx=0).stop 2024-06-10T01:08:30.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.562+0000 7fa299f53b80 1 -- 172.21.15.37:0/2990800816 shutdown_connections 2024-06-10T01:08:30.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.562+0000 7fa299f53b80 1 --2- 172.21.15.37:0/2990800816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b18cc407a0 0x55b18cc40bc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.562+0000 7fa299f53b80 1 --2- 172.21.15.37:0/2990800816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b18cc45f60 0x55b18cc46340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.562+0000 7fa299f53b80 1 --2- 172.21.15.37:0/2990800816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b18cc46880 0x55b18cc40260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.562+0000 7fa299f53b80 1 -- 172.21.15.37:0/2990800816 >> 172.21.15.37:0/2990800816 conn(0x55b18cb6d080 msgr2=0x55b18cc3b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:30.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.562+0000 7fa299f53b80 1 -- 172.21.15.37:0/2990800816 shutdown_connections 2024-06-10T01:08:30.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.563+0000 7fa299f53b80 1 -- 172.21.15.37:0/2990800816 wait complete. 2024-06-10T01:08:30.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.563+0000 7fa299f53b80 1 -- start start 2024-06-10T01:08:30.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.563+0000 7fa299f53b80 1 Processor -- start 2024-06-10T01:08:30.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.563+0000 7fa299f53b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b18cc407a0 0x55b18cc4c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:30.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.563+0000 7fa299f53b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b18cc45f60 0x55b18cc49910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:30.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.563+0000 7fa299f53b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b18cc46880 0x55b18cc49e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:30.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.563+0000 7fa299f53b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b18cb31fc0 con 0x55b18cc45f60 2024-06-10T01:08:30.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.563+0000 7fa299f53b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b18cb8d810 con 0x55b18cc46880 2024-06-10T01:08:30.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.563+0000 7fa299f53b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b18cc4d200 con 0x55b18cc407a0 2024-06-10T01:08:30.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.564+0000 7fa299f53b80 10 client.0 _ll_register_callbacks cb 0x55b18cb275b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:30.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.564+0000 7fa28cc6b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b18cc45f60 0x55b18cc49910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:30.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.564+0000 7fa28d46c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b18cc407a0 0x55b18cc4c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:30.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.564+0000 7fa28cc6b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b18cc45f60 0x55b18cc49910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50706/0 (socket says 172.21.15.37:50706) 2024-06-10T01:08:30.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.564+0000 7fa28cc6b700 1 -- 172.21.15.37:0/1944784896 learned_addr learned my addr 172.21.15.37:0/1944784896 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:30.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.564+0000 7fa28dc6d700 1 --2- 172.21.15.37:0/1944784896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b18cc46880 0x55b18cc49e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:30.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.564+0000 7fa28cc6b700 1 -- 172.21.15.37:0/1944784896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b18cc407a0 msgr2=0x55b18cc4c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:30.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.564+0000 7fa28cc6b700 1 --2- 172.21.15.37:0/1944784896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b18cc407a0 0x55b18cc4c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.564+0000 7fa28cc6b700 1 -- 172.21.15.37:0/1944784896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b18cc46880 msgr2=0x55b18cc49e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:30.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.564+0000 7fa28cc6b700 1 --2- 172.21.15.37:0/1944784896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b18cc46880 0x55b18cc49e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.564+0000 7fa28cc6b700 1 -- 172.21.15.37:0/1944784896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b18cb57750 con 0x55b18cc45f60 2024-06-10T01:08:30.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.564+0000 7fa28d46c700 1 --2- 172.21.15.37:0/1944784896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b18cc407a0 0x55b18cc4c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:30.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.564+0000 7fa28cc6b700 1 --2- 172.21.15.37:0/1944784896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b18cc45f60 0x55b18cc49910 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fa2780099c0 tx=0x7fa27800bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38e006cb146fdf1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:30.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.564+0000 7fa270ff9700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:08:30.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.564+0000 7fa270ff9700 1 -- 172.21.15.37:0/1944784896 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278003a60 con 0x55b18cc45f60 2024-06-10T01:08:30.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.564+0000 7fa299f53b80 1 -- 172.21.15.37:0/1944784896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa28009f720 con 0x55b18cc45f60 2024-06-10T01:08:30.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.564+0000 7fa270ff9700 1 -- 172.21.15.37:0/1944784896 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fa278003c00 con 0x55b18cc45f60 2024-06-10T01:08:30.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.565+0000 7fa270ff9700 1 -- 172.21.15.37:0/1944784896 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa278030050 con 0x55b18cc45f60 2024-06-10T01:08:30.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.565+0000 7fa299f53b80 10 client.17523 fetch_fsmap learned FSMap version 31 2024-06-10T01:08:30.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.565+0000 7fa299f53b80 1 -- 172.21.15.37:0/1944784896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b18cc60210 con 0x55b18cc45f60 2024-06-10T01:08:30.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.565+0000 7fa270ff9700 1 -- 172.21.15.37:0/1944784896 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa28009f720 con 0x55b18cc45f60 2024-06-10T01:08:30.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.565+0000 7fa270ff9700 1 -- 172.21.15.37:0/1944784896 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fa27802b030 con 0x55b18cc45f60 2024-06-10T01:08:30.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.565+0000 7fa299f53b80 10 client.17523 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:08:30.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.565+0000 7fa299f53b80 10 client.17523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:30.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.565+0000 7fa299f53b80 20 client.17523 populate_metadata read hostname 'smithi037' 2024-06-10T01:08:30.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.565+0000 7fa299f53b80 1 --2- 172.21.15.37:0/1944784896 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b18cc627a0 0x55b18cc64c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:30.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.565+0000 7fa299f53b80 4 client.17523 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:08:30.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.565+0000 7fa299f53b80 1 -- 172.21.15.37:0/1944784896 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b18cb9ed50 con 0x55b18cc627a0 2024-06-10T01:08:30.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.565+0000 7fa28d46c700 1 --2- 172.21.15.37:0/1944784896 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b18cc627a0 0x55b18cc64c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:30.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.566+0000 7fa28d46c700 1 --2- 172.21.15.37:0/1944784896 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b18cc627a0 0x55b18cc64c00 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6271896a8e24e582 server_cookie=b2babbae38075077 in_seq=0 out_seq=0 2024-06-10T01:08:30.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.566+0000 7fa270ff9700 10 client.17523 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:30.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.568+0000 7fa270ff9700 1 -- 172.21.15.37:0/1944784896 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b18cb9ed50 con 0x55b18cc627a0 2024-06-10T01:08:30.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.568+0000 7fa270ff9700 10 client.17523 handle_command_reply: tid=0 2024-06-10T01:08:30.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.568+0000 7fa299f53b80 1 client.17523 shutdown 2024-06-10T01:08:30.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.568+0000 7fa299f53b80 1 -- 172.21.15.37:0/1944784896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b18cc45f60 msgr2=0x55b18cc49910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:30.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.568+0000 7fa299f53b80 1 --2- 172.21.15.37:0/1944784896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b18cc45f60 0x55b18cc49910 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fa2780099c0 tx=0x7fa27800bee0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.568+0000 7fa299f53b80 1 -- 172.21.15.37:0/1944784896 shutdown_connections 2024-06-10T01:08:30.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.568+0000 7fa299f53b80 1 --2- 172.21.15.37:0/1944784896 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b18cc627a0 0x55b18cc64c00 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.568+0000 7fa270ff9700 0 client.17523 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:30.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.568+0000 7fa299f53b80 1 --2- 172.21.15.37:0/1944784896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b18cc46880 0x55b18cc49e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.568+0000 7fa299f53b80 1 --2- 172.21.15.37:0/1944784896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b18cc407a0 0x55b18cc4c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.568+0000 7fa299f53b80 1 --2- 172.21.15.37:0/1944784896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b18cc45f60 0x55b18cc49910 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.568+0000 7fa299f53b80 1 -- 172.21.15.37:0/1944784896 >> 172.21.15.37:0/1944784896 conn(0x55b18cb6d080 msgr2=0x55b18cc3a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:30.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.568+0000 7fa299f53b80 1 -- 172.21.15.37:0/1944784896 shutdown_connections 2024-06-10T01:08:30.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.568+0000 7fa299f53b80 1 -- 172.21.15.37:0/1944784896 wait complete. 2024-06-10T01:08:30.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.568+0000 7fa299f53b80 20 client.17523 trim_cache size 0 max 16384 2024-06-10T01:08:30.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:30 smithi167 ceph-mon[36917]: pgmap v785: 129 pgs: 129 active+clean; 8.3 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 356 KiB/s wr, 222 op/s 2024-06-10T01:08:30.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.589+0000 7fa299f53b80 1 Processor -- start 2024-06-10T01:08:30.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.589+0000 7fa299f53b80 1 -- start start 2024-06-10T01:08:30.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.590+0000 7fa299f53b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b18cc407a0 0x55b18cc40260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:30.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.590+0000 7fa299f53b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b18cc45f60 0x55b18cc44050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:30.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.590+0000 7fa299f53b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b18cc46880 0x55b18cc47450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:30.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.590+0000 7fa299f53b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa2881aeb80 con 0x55b18cc45f60 2024-06-10T01:08:30.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.590+0000 7fa299f53b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa2881ac410 con 0x55b18cc407a0 2024-06-10T01:08:30.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.590+0000 7fa299f53b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa2881aa260 con 0x55b18cc46880 2024-06-10T01:08:30.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.590+0000 7fa28cc6b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b18cc45f60 0x55b18cc44050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:30.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.590+0000 7fa28cc6b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b18cc45f60 0x55b18cc44050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50710/0 (socket says 172.21.15.37:50710) 2024-06-10T01:08:30.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.590+0000 7fa28cc6b700 1 -- 172.21.15.37:0/3453227482 learned_addr learned my addr 172.21.15.37:0/3453227482 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:30.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.590+0000 7fa28dc6d700 1 --2- 172.21.15.37:0/3453227482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b18cc46880 0x55b18cc47450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:30.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.590+0000 7fa28d46c700 1 --2- 172.21.15.37:0/3453227482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b18cc407a0 0x55b18cc40260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:30.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.590+0000 7fa28cc6b700 1 -- 172.21.15.37:0/3453227482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b18cc46880 msgr2=0x55b18cc47450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:30.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.590+0000 7fa28cc6b700 1 --2- 172.21.15.37:0/3453227482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b18cc46880 0x55b18cc47450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.590+0000 7fa28cc6b700 1 -- 172.21.15.37:0/3453227482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b18cc407a0 msgr2=0x55b18cc40260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:30.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.590+0000 7fa28cc6b700 1 --2- 172.21.15.37:0/3453227482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b18cc407a0 0x55b18cc40260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.590+0000 7fa28cc6b700 1 -- 172.21.15.37:0/3453227482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b18cc46680 con 0x55b18cc45f60 2024-06-10T01:08:30.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.590+0000 7fa28d46c700 1 --2- 172.21.15.37:0/3453227482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b18cc407a0 0x55b18cc40260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:30.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.590+0000 7fa28cc6b700 1 --2- 172.21.15.37:0/3453227482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b18cc45f60 0x55b18cc44050 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fa278009990 tx=0x7fa278009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb6810ec6cd0df3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:30.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.590+0000 7fa272ffd700 1 -- 172.21.15.37:0/3453227482 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278009e60 con 0x55b18cc45f60 2024-06-10T01:08:30.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.591+0000 7fa272ffd700 1 -- 172.21.15.37:0/3453227482 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2780055f0 con 0x55b18cc45f60 2024-06-10T01:08:30.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.591+0000 7fa272ffd700 1 -- 172.21.15.37:0/3453227482 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27800f460 con 0x55b18cc45f60 2024-06-10T01:08:30.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.591+0000 7fa299f53b80 1 -- 172.21.15.37:0/3453227482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b18cc45f60 msgr2=0x55b18cc44050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:30.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.591+0000 7fa299f53b80 1 --2- 172.21.15.37:0/3453227482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b18cc45f60 0x55b18cc44050 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fa278009990 tx=0x7fa278009290 comp rx=0 tx=0).stop 2024-06-10T01:08:30.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.591+0000 7fa299f53b80 1 -- 172.21.15.37:0/3453227482 shutdown_connections 2024-06-10T01:08:30.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.591+0000 7fa299f53b80 1 --2- 172.21.15.37:0/3453227482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b18cc46880 0x55b18cc47450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.591+0000 7fa299f53b80 1 --2- 172.21.15.37:0/3453227482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b18cc407a0 0x55b18cc40260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.591+0000 7fa299f53b80 1 --2- 172.21.15.37:0/3453227482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b18cc45f60 0x55b18cc44050 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.591+0000 7fa299f53b80 1 -- 172.21.15.37:0/3453227482 >> 172.21.15.37:0/3453227482 conn(0x55b18cb6d080 msgr2=0x55b18cc46d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:30.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.591+0000 7fa299f53b80 1 -- 172.21.15.37:0/3453227482 shutdown_connections 2024-06-10T01:08:30.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.591+0000 7fa299f53b80 1 -- 172.21.15.37:0/3453227482 wait complete. 2024-06-10T01:08:30.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.592+0000 7fa299f53b80 1 -- start start 2024-06-10T01:08:30.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.592+0000 7fa299f53b80 1 Processor -- start 2024-06-10T01:08:30.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.592+0000 7fa299f53b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b18cc407a0 0x55b18cc5b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:30.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.592+0000 7fa299f53b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b18cc45f60 0x55b18cc69db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:30.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.592+0000 7fa299f53b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b18cc46880 0x55b18cc6a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:30.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.592+0000 7fa299f53b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa2881aa8d0 con 0x55b18cc46880 2024-06-10T01:08:30.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.592+0000 7fa299f53b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa2881abd40 con 0x55b18cc45f60 2024-06-10T01:08:30.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.592+0000 7fa299f53b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa2881aafa0 con 0x55b18cc407a0 2024-06-10T01:08:30.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.592+0000 7fa299f53b80 10 client.0 _ll_register_callbacks cb 0x55b18cb275b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:30.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.592+0000 7fa28d46c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b18cc407a0 0x55b18cc5b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:30.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.592+0000 7fa28dc6d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b18cc46880 0x55b18cc6a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:30.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.592+0000 7fa28d46c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b18cc407a0 0x55b18cc5b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:58980/0 (socket says 172.21.15.37:58980) 2024-06-10T01:08:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.592+0000 7fa28d46c700 1 -- 172.21.15.37:0/3457131637 learned_addr learned my addr 172.21.15.37:0/3457131637 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.592+0000 7fa28cc6b700 1 --2- 172.21.15.37:0/3457131637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b18cc45f60 0x55b18cc69db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.593+0000 7fa28d46c700 1 -- 172.21.15.37:0/3457131637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b18cc45f60 msgr2=0x55b18cc69db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.593+0000 7fa28d46c700 1 --2- 172.21.15.37:0/3457131637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b18cc45f60 0x55b18cc69db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.593+0000 7fa28d46c700 1 -- 172.21.15.37:0/3457131637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b18cc46880 msgr2=0x55b18cc6a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.593+0000 7fa28d46c700 1 --2- 172.21.15.37:0/3457131637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b18cc46880 0x55b18cc6a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.593+0000 7fa28d46c700 1 -- 172.21.15.37:0/3457131637 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b18cc6ab40 con 0x55b18cc407a0 2024-06-10T01:08:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.593+0000 7fa28cc6b700 1 --2- 172.21.15.37:0/3457131637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b18cc45f60 0x55b18cc69db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:08:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.593+0000 7fa28d46c700 1 --2- 172.21.15.37:0/3457131637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b18cc407a0 0x55b18cc5b490 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa284011610 tx=0x7fa2840047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef21282c1727e824 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.593+0000 7fa270ff9700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:08:30.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.593+0000 7fa270ff9700 1 -- 172.21.15.37:0/3457131637 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa284004250 con 0x55b18cc407a0 2024-06-10T01:08:30.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.593+0000 7fa270ff9700 1 -- 172.21.15.37:0/3457131637 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fa2840043f0 con 0x55b18cc407a0 2024-06-10T01:08:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.593+0000 7fa299f53b80 1 -- 172.21.15.37:0/3457131637 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2780055f0 con 0x55b18cc407a0 2024-06-10T01:08:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.593+0000 7fa270ff9700 1 -- 172.21.15.37:0/3457131637 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa28403b050 con 0x55b18cc407a0 2024-06-10T01:08:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.593+0000 7fa299f53b80 10 client.27016 fetch_fsmap learned FSMap version 31 2024-06-10T01:08:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.593+0000 7fa299f53b80 1 -- 172.21.15.37:0/3457131637 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b18cccd4c0 con 0x55b18cc407a0 2024-06-10T01:08:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.594+0000 7fa270ff9700 1 -- 172.21.15.37:0/3457131637 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2780055f0 con 0x55b18cc407a0 2024-06-10T01:08:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.594+0000 7fa270ff9700 1 -- 172.21.15.37:0/3457131637 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fa284031030 con 0x55b18cc407a0 2024-06-10T01:08:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.594+0000 7fa299f53b80 10 client.27016 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:08:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.594+0000 7fa299f53b80 10 client.27016 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.594+0000 7fa299f53b80 20 client.27016 populate_metadata read hostname 'smithi037' 2024-06-10T01:08:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.594+0000 7fa299f53b80 1 --2- 172.21.15.37:0/3457131637 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b18cc627a0 0x55b18cccfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.594+0000 7fa299f53b80 4 client.27016 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:08:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.594+0000 7fa299f53b80 1 -- 172.21.15.37:0/3457131637 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b18cb992d0 con 0x55b18cc627a0 2024-06-10T01:08:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.594+0000 7fa28cc6b700 1 --2- 172.21.15.37:0/3457131637 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b18cc627a0 0x55b18cccfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.594+0000 7fa28cc6b700 1 --2- 172.21.15.37:0/3457131637 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b18cc627a0 0x55b18cccfbe0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=786b82efd3166d24 server_cookie=356309939c87f932 in_seq=0 out_seq=0 2024-06-10T01:08:30.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.594+0000 7fa270ff9700 10 client.27016 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:30.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.599+0000 7fa270ff9700 1 -- 172.21.15.37:0/3457131637 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55b18cb992d0 con 0x55b18cc627a0 2024-06-10T01:08:30.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.599+0000 7fa270ff9700 10 client.27016 handle_command_reply: tid=0 2024-06-10T01:08:30.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.599+0000 7fa299f53b80 1 client.27016 shutdown 2024-06-10T01:08:30.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.599+0000 7fa299f53b80 1 -- 172.21.15.37:0/3457131637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b18cc407a0 msgr2=0x55b18cc5b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:30.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.599+0000 7fa299f53b80 1 --2- 172.21.15.37:0/3457131637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b18cc407a0 0x55b18cc5b490 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa284011610 tx=0x7fa2840047c0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.599+0000 7fa299f53b80 1 -- 172.21.15.37:0/3457131637 shutdown_connections 2024-06-10T01:08:30.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.599+0000 7fa299f53b80 1 --2- 172.21.15.37:0/3457131637 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b18cc627a0 0x55b18cccfbe0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.599+0000 7fa299f53b80 1 --2- 172.21.15.37:0/3457131637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b18cc46880 0x55b18cc6a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.599+0000 7fa299f53b80 1 --2- 172.21.15.37:0/3457131637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b18cc407a0 0x55b18cc5b490 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.599+0000 7fa299f53b80 1 --2- 172.21.15.37:0/3457131637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b18cc45f60 0x55b18cc69db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.599+0000 7fa299f53b80 1 -- 172.21.15.37:0/3457131637 >> 172.21.15.37:0/3457131637 conn(0x55b18cb6d080 msgr2=0x55b18cc4b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:30.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.599+0000 7fa270ff9700 0 client.27016 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:30.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.599+0000 7fa299f53b80 1 -- 172.21.15.37:0/3457131637 shutdown_connections 2024-06-10T01:08:30.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.599+0000 7fa299f53b80 1 -- 172.21.15.37:0/3457131637 wait complete. 2024-06-10T01:08:30.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.599+0000 7fa299f53b80 20 client.27016 trim_cache size 0 max 16384 2024-06-10T01:08:30.597 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:08:30.597 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "scrub active (7019 inodes in the stack)", 2024-06-10T01:08:30.597 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-06-10T01:08:30.597 INFO:teuthology.orchestra.run.smithi037.stdout: "e78a272e-e18b-41f0-b3ca-e304eb51fcfe": { 2024-06-10T01:08:30.597 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-06-10T01:08:30.597 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "e78a272e-e18b-41f0-b3ca-e304eb51fcfe", 2024-06-10T01:08:30.597 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-06-10T01:08:30.597 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:08:30.598 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:08:30.598 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:08:30.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.600+0000 7fa28f6d0700 1 -- 172.21.15.37:0/1042475579 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa2740a2ca0 msgr2=0x7fa2740a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:30.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.600+0000 7fa28f6d0700 1 --2- 172.21.15.37:0/1042475579 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa2740a2ca0 0x7fa2740a5160 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa284007b60 tx=0x7fa2840073d0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.600+0000 7fa28f6d0700 1 -- 172.21.15.37:0/1042475579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2881085d0 msgr2=0x7fa288075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:30.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.600+0000 7fa28f6d0700 1 --2- 172.21.15.37:0/1042475579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2881085d0 0x7fa288075210 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fa28000ef30 tx=0x7fa28000c3a0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.600+0000 7fa28f6d0700 1 -- 172.21.15.37:0/1042475579 shutdown_connections 2024-06-10T01:08:30.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.600+0000 7fa28f6d0700 1 --2- 172.21.15.37:0/1042475579 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa2740a2ca0 0x7fa2740a5160 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.600+0000 7fa28f6d0700 1 --2- 172.21.15.37:0/1042475579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2881085d0 0x7fa288075210 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.600+0000 7fa28f6d0700 1 --2- 172.21.15.37:0/1042475579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa288102530 0x7fa288074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.600+0000 7fa28f6d0700 1 --2- 172.21.15.37:0/1042475579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa288102ef0 0x7fa288074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:30.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.600+0000 7fa28f6d0700 1 -- 172.21.15.37:0/1042475579 >> 172.21.15.37:0/1042475579 conn(0x7fa288079640 msgr2=0x7fa288101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:30.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.600+0000 7fa28f6d0700 1 -- 172.21.15.37:0/1042475579 shutdown_connections 2024-06-10T01:08:30.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:08:30.600+0000 7fa28f6d0700 1 -- 172.21.15.37:0/1042475579 wait complete. 2024-06-10T01:08:30.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:30 smithi094 ceph-mon[36708]: pgmap v785: 129 pgs: 129 active+clean; 8.3 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 356 KiB/s wr, 222 op/s 2024-06-10T01:08:31.866 INFO:tasks.cephfs.filesystem:scrub status for tag:e78a272e-e18b-41f0-b3ca-e304eb51fcfe - {'path': '/', 'tag': 'e78a272e-e18b-41f0-b3ca-e304eb51fcfe', 'options': 'recursive,force'} 2024-06-10T01:08:32.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:32 smithi167 ceph-mon[36917]: pgmap v786: 129 pgs: 129 active+clean; 11 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 513 KiB/s wr, 186 op/s 2024-06-10T01:08:32.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:32 smithi094 ceph-mon[36708]: pgmap v786: 129 pgs: 129 active+clean; 11 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 513 KiB/s wr, 186 op/s 2024-06-10T01:08:32.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:32 smithi037 ceph-mon[32192]: pgmap v786: 129 pgs: 129 active+clean; 11 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 513 KiB/s wr, 186 op/s 2024-06-10T01:08:34.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:34 smithi167 ceph-mon[36917]: pgmap v787: 129 pgs: 129 active+clean; 11 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 523 KiB/s wr, 252 op/s 2024-06-10T01:08:34.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:34 smithi094 ceph-mon[36708]: pgmap v787: 129 pgs: 129 active+clean; 11 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 523 KiB/s wr, 252 op/s 2024-06-10T01:08:34.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:34 smithi037 ceph-mon[32192]: pgmap v787: 129 pgs: 129 active+clean; 11 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 523 KiB/s wr, 252 op/s 2024-06-10T01:08:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.686+0000 7fc662153640 1 -- 172.21.15.37:0/707320546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc65c102480 msgr2=0x7fc65c104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.686+0000 7fc662153640 1 --2- 172.21.15.37:0/707320546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc65c102480 0x7fc65c104870 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fc64c009900 tx=0x7fc64c02a410 comp rx=0 tx=0).stop 2024-06-10T01:08:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.686+0000 7fc662153640 1 -- 172.21.15.37:0/707320546 shutdown_connections 2024-06-10T01:08:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.686+0000 7fc662153640 1 --2- 172.21.15.37:0/707320546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc65c1076e0 0x7fc65c109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.686+0000 7fc662153640 1 --2- 172.21.15.37:0/707320546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc65c104db0 0x7fc65c1071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.686+0000 7fc662153640 1 --2- 172.21.15.37:0/707320546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc65c102480 0x7fc65c104870 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.686+0000 7fc662153640 1 -- 172.21.15.37:0/707320546 >> 172.21.15.37:0/707320546 conn(0x7fc65c0fbe70 msgr2=0x7fc65c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.686+0000 7fc662153640 1 -- 172.21.15.37:0/707320546 shutdown_connections 2024-06-10T01:08:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.686+0000 7fc662153640 1 -- 172.21.15.37:0/707320546 wait complete. 2024-06-10T01:08:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.686+0000 7fc662153640 1 Processor -- start 2024-06-10T01:08:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.687+0000 7fc662153640 1 -- start start 2024-06-10T01:08:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.687+0000 7fc662153640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc65c102480 0x7fc65c101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.687+0000 7fc662153640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc65c104db0 0x7fc65c100430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.687+0000 7fc662153640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc65c1076e0 0x7fc65c100970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.687+0000 7fc662153640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c10d6b0 con 0x7fc65c104db0 2024-06-10T01:08:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.687+0000 7fc662153640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c10d530 con 0x7fc65c102480 2024-06-10T01:08:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.687+0000 7fc662153640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c10d830 con 0x7fc65c1076e0 2024-06-10T01:08:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.687+0000 7fc65affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc65c104db0 0x7fc65c100430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.687+0000 7fc65affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc65c104db0 0x7fc65c100430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59504/0 (socket says 172.21.15.37:59504) 2024-06-10T01:08:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.687+0000 7fc65affd640 1 -- 172.21.15.37:0/1168845676 learned_addr learned my addr 172.21.15.37:0/1168845676 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.687+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc65c1076e0 0x7fc65c100970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.687+0000 7fc65b7fe640 1 --2- 172.21.15.37:0/1168845676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc65c102480 0x7fc65c101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.687+0000 7fc65affd640 1 -- 172.21.15.37:0/1168845676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc65c1076e0 msgr2=0x7fc65c100970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.687+0000 7fc65affd640 1 --2- 172.21.15.37:0/1168845676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc65c1076e0 0x7fc65c100970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.687+0000 7fc65affd640 1 -- 172.21.15.37:0/1168845676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc65c102480 msgr2=0x7fc65c101df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.687+0000 7fc65affd640 1 --2- 172.21.15.37:0/1168845676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc65c102480 0x7fc65c101df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.687+0000 7fc65affd640 1 -- 172.21.15.37:0/1168845676 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65c101200 con 0x7fc65c104db0 2024-06-10T01:08:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.687+0000 7fc65bfff640 1 --2- 172.21.15.37:0/1168845676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc65c1076e0 0x7fc65c100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.688+0000 7fc65affd640 1 --2- 172.21.15.37:0/1168845676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc65c104db0 0x7fc65c100430 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fc65000e9f0 tx=0x7fc65000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.688+0000 7fc658ff9640 1 -- 172.21.15.37:0/1168845676 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65000ccc0 con 0x7fc65c104db0 2024-06-10T01:08:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.688+0000 7fc662153640 1 -- 172.21.15.37:0/1168845676 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc65c1a1de0 con 0x7fc65c104db0 2024-06-10T01:08:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.688+0000 7fc658ff9640 1 -- 172.21.15.37:0/1168845676 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc650005500 con 0x7fc65c104db0 2024-06-10T01:08:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.688+0000 7fc658ff9640 1 -- 172.21.15.37:0/1168845676 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc650010690 con 0x7fc65c104db0 2024-06-10T01:08:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.688+0000 7fc662153640 1 -- 172.21.15.37:0/1168845676 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc65c1a21f0 con 0x7fc65c104db0 2024-06-10T01:08:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.690+0000 7fc658ff9640 1 -- 172.21.15.37:0/1168845676 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc6500046c0 con 0x7fc65c104db0 2024-06-10T01:08:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.691+0000 7fc664e5e740 1 Processor -- start 2024-06-10T01:08:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.691+0000 7fc664e5e740 1 -- start start 2024-06-10T01:08:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.691+0000 7fc664e5e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557f6c4c010 0x5557f6c4c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.691+0000 7fc664e5e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557f6c4ca00 0x5557f6c51500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.691+0000 7fc664e5e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557f6c51b70 0x5557f6c54000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.691+0000 7fc664e5e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5557f6baf4f0 con 0x5557f6c4ca00 2024-06-10T01:08:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.691+0000 7fc664e5e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5557f6b4d870 con 0x5557f6c51b70 2024-06-10T01:08:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.691+0000 7fc664e5e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5557f6bab4b0 con 0x5557f6c4c010 2024-06-10T01:08:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.691+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557f6c4c010 0x5557f6c4c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.691+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557f6c4c010 0x5557f6c4c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:33816/0 (socket says 172.21.15.37:33816) 2024-06-10T01:08:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.691+0000 7fc65bfff640 1 -- 172.21.15.37:0/1886989261 learned_addr learned my addr 172.21.15.37:0/1886989261 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.691+0000 7fc65b7fe640 1 --2- 172.21.15.37:0/1886989261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557f6c4ca00 0x5557f6c51500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.691+0000 7fc65affd640 1 --2- 172.21.15.37:0/1886989261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557f6c51b70 0x5557f6c54000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.691+0000 7fc65bfff640 1 -- 172.21.15.37:0/1886989261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557f6c51b70 msgr2=0x5557f6c54000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.691+0000 7fc65bfff640 1 --2- 172.21.15.37:0/1886989261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557f6c51b70 0x5557f6c54000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.691+0000 7fc65bfff640 1 -- 172.21.15.37:0/1886989261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557f6c4ca00 msgr2=0x5557f6c51500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.691+0000 7fc65bfff640 1 --2- 172.21.15.37:0/1886989261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557f6c4ca00 0x5557f6c51500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.691+0000 7fc658ff9640 1 --2- 172.21.15.37:0/1168845676 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc6440a2c30 0x7fc6440a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.691+0000 7fc65bfff640 1 -- 172.21.15.37:0/1886989261 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557f6b699b0 con 0x5557f6c4c010 2024-06-10T01:08:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.691+0000 7fc65affd640 1 --2- 172.21.15.37:0/1886989261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557f6c51b70 0x5557f6c54000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:08:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.692+0000 7fc658ff9640 1 -- 172.21.15.37:0/1168845676 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fc6500cf850 con 0x7fc65c104db0 2024-06-10T01:08:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.692+0000 7fc65bfff640 1 --2- 172.21.15.37:0/1168845676 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc6440a2c30 0x7fc6440a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.692+0000 7fc65bfff640 1 --2- 172.21.15.37:0/1886989261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557f6c4c010 0x5557f6c4c430 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc654010750 tx=0x7fc654010c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd4854e33a5bc39c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.692+0000 7fc641ffb640 1 -- 172.21.15.37:0/1886989261 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc654012ca0 con 0x5557f6c4c010 2024-06-10T01:08:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.692+0000 7fc641ffb640 1 -- 172.21.15.37:0/1886989261 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc654012e40 con 0x5557f6c4c010 2024-06-10T01:08:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.692+0000 7fc641ffb640 1 -- 172.21.15.37:0/1886989261 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc654011640 con 0x5557f6c4c010 2024-06-10T01:08:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.693+0000 7fc664e5e740 1 -- 172.21.15.37:0/1886989261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557f6c4c010 msgr2=0x5557f6c4c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.693+0000 7fc65bfff640 1 --2- 172.21.15.37:0/1168845676 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc6440a2c30 0x7fc6440a50f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc65401f000 tx=0x7fc654022000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.693+0000 7fc664e5e740 1 --2- 172.21.15.37:0/1886989261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557f6c4c010 0x5557f6c4c430 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc654010750 tx=0x7fc654010c10 comp rx=0 tx=0).stop 2024-06-10T01:08:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.693+0000 7fc664e5e740 1 -- 172.21.15.37:0/1886989261 shutdown_connections 2024-06-10T01:08:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.693+0000 7fc664e5e740 1 --2- 172.21.15.37:0/1886989261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557f6c51b70 0x5557f6c54000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.693+0000 7fc664e5e740 1 --2- 172.21.15.37:0/1886989261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557f6c4ca00 0x5557f6c51500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.693+0000 7fc664e5e740 1 --2- 172.21.15.37:0/1886989261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557f6c4c010 0x5557f6c4c430 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.693+0000 7fc664e5e740 1 -- 172.21.15.37:0/1886989261 >> 172.21.15.37:0/1886989261 conn(0x5557f6c47680 msgr2=0x5557f6c47a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.693+0000 7fc664e5e740 1 -- 172.21.15.37:0/1886989261 shutdown_connections 2024-06-10T01:08:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.693+0000 7fc664e5e740 1 -- 172.21.15.37:0/1886989261 wait complete. 2024-06-10T01:08:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.693+0000 7fc664e5e740 1 -- start start 2024-06-10T01:08:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.693+0000 7fc664e5e740 1 Processor -- start 2024-06-10T01:08:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.694+0000 7fc664e5e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557f6c4c010 0x5557f6c5ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.694+0000 7fc664e5e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557f6c4ca00 0x5557f6c5efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.694+0000 7fc664e5e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557f6c51b70 0x5557f6c65fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.694+0000 7fc664e5e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5557f6c54b20 con 0x5557f6c4ca00 2024-06-10T01:08:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.694+0000 7fc664e5e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5557f6b38ae0 con 0x5557f6c4c010 2024-06-10T01:08:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.694+0000 7fc664e5e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5557f6c54ca0 con 0x5557f6c51b70 2024-06-10T01:08:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.694+0000 7fc664e5e740 10 client.0 _ll_register_callbacks cb 0x5557f6b719b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.694+0000 7fc65affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557f6c4ca00 0x5557f6c5efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.694+0000 7fc65affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557f6c4ca00 0x5557f6c5efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59518/0 (socket says 172.21.15.37:59518) 2024-06-10T01:08:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.694+0000 7fc65affd640 1 -- 172.21.15.37:0/2678417619 learned_addr learned my addr 172.21.15.37:0/2678417619 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.694+0000 7fc65bfff640 1 --2- 172.21.15.37:0/2678417619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557f6c51b70 0x5557f6c65fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.694+0000 7fc65b7fe640 1 --2- 172.21.15.37:0/2678417619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557f6c4c010 0x5557f6c5ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.694+0000 7fc65affd640 1 -- 172.21.15.37:0/2678417619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557f6c51b70 msgr2=0x5557f6c65fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.694+0000 7fc65affd640 1 --2- 172.21.15.37:0/2678417619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557f6c51b70 0x5557f6c65fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.694+0000 7fc65affd640 1 -- 172.21.15.37:0/2678417619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557f6c4c010 msgr2=0x5557f6c5ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.694+0000 7fc65affd640 1 --2- 172.21.15.37:0/2678417619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557f6c4c010 0x5557f6c5ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.694+0000 7fc65affd640 1 -- 172.21.15.37:0/2678417619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557f68c3210 con 0x5557f6c4ca00 2024-06-10T01:08:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.694+0000 7fc65affd640 1 --2- 172.21.15.37:0/2678417619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557f6c4ca00 0x5557f6c5efa0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc6500cfaf0 tx=0x7fc65009b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4f5f044a9178a1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.694+0000 7fc63affd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:08:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.695+0000 7fc63affd640 1 -- 172.21.15.37:0/2678417619 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65009c360 con 0x5557f6c4ca00 2024-06-10T01:08:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.695+0000 7fc63affd640 1 -- 172.21.15.37:0/2678417619 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fc6500a4440 con 0x5557f6c4ca00 2024-06-10T01:08:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.695+0000 7fc664e5e740 1 -- 172.21.15.37:0/2678417619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5557f6bc2f10 con 0x5557f6c4ca00 2024-06-10T01:08:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.695+0000 7fc63affd640 1 -- 172.21.15.37:0/2678417619 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc6500b4e60 con 0x5557f6c4ca00 2024-06-10T01:08:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.695+0000 7fc664e5e740 10 client.17541 fetch_fsmap learned FSMap version 31 2024-06-10T01:08:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.695+0000 7fc664e5e740 1 -- 172.21.15.37:0/2678417619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5557f698f620 con 0x5557f6c4ca00 2024-06-10T01:08:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.695+0000 7fc63affd640 1 -- 172.21.15.37:0/2678417619 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5557f6bc2f10 con 0x5557f6c4ca00 2024-06-10T01:08:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.695+0000 7fc63affd640 1 -- 172.21.15.37:0/2678417619 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fc65009b140 con 0x5557f6c4ca00 2024-06-10T01:08:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.695+0000 7fc664e5e740 10 client.17541 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:08:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.695+0000 7fc664e5e740 10 client.17541 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.695+0000 7fc664e5e740 20 client.17541 populate_metadata read hostname 'smithi037' 2024-06-10T01:08:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.695+0000 7fc664e5e740 1 --2- 172.21.15.37:0/2678417619 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5557f6c6a8e0 0x5557f6c6ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.695+0000 7fc664e5e740 4 client.17541 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:08:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.695+0000 7fc664e5e740 1 -- 172.21.15.37:0/2678417619 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557f6bbcc20 con 0x5557f6c6a8e0 2024-06-10T01:08:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.695+0000 7fc65b7fe640 1 --2- 172.21.15.37:0/2678417619 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5557f6c6a8e0 0x5557f6c6ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.696+0000 7fc65b7fe640 1 --2- 172.21.15.37:0/2678417619 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5557f6c6a8e0 0x5557f6c6ccf0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff995a995bb49530 server_cookie=3ce147bde0d5b8ab in_seq=0 out_seq=0 2024-06-10T01:08:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.696+0000 7fc63affd640 10 client.17541 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.699+0000 7fc63affd640 1 -- 172.21.15.37:0/2678417619 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5557f6bbcc20 con 0x5557f6c6a8e0 2024-06-10T01:08:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.699+0000 7fc63affd640 10 client.17541 handle_command_reply: tid=0 2024-06-10T01:08:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.699+0000 7fc664e5e740 1 client.17541 shutdown 2024-06-10T01:08:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.699+0000 7fc664e5e740 1 -- 172.21.15.37:0/2678417619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557f6c4ca00 msgr2=0x5557f6c5efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.699+0000 7fc664e5e740 1 --2- 172.21.15.37:0/2678417619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557f6c4ca00 0x5557f6c5efa0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc6500cfaf0 tx=0x7fc65009b860 comp rx=0 tx=0).stop 2024-06-10T01:08:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.699+0000 7fc664e5e740 1 -- 172.21.15.37:0/2678417619 shutdown_connections 2024-06-10T01:08:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.699+0000 7fc664e5e740 1 --2- 172.21.15.37:0/2678417619 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5557f6c6a8e0 0x5557f6c6ccf0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.699+0000 7fc664e5e740 1 --2- 172.21.15.37:0/2678417619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557f6c51b70 0x5557f6c65fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.699+0000 7fc664e5e740 1 --2- 172.21.15.37:0/2678417619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557f6c4ca00 0x5557f6c5efa0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.699+0000 7fc63affd640 0 client.17541 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.699+0000 7fc664e5e740 1 --2- 172.21.15.37:0/2678417619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557f6c4c010 0x5557f6c5ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.699+0000 7fc664e5e740 1 -- 172.21.15.37:0/2678417619 >> 172.21.15.37:0/2678417619 conn(0x5557f6c47680 msgr2=0x5557f6c467e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.699+0000 7fc664e5e740 1 -- 172.21.15.37:0/2678417619 shutdown_connections 2024-06-10T01:08:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.699+0000 7fc664e5e740 1 -- 172.21.15.37:0/2678417619 wait complete. 2024-06-10T01:08:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.699+0000 7fc664e5e740 20 client.17541 trim_cache size 0 max 16384 2024-06-10T01:08:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.716+0000 7fc664e5e740 1 Processor -- start 2024-06-10T01:08:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.716+0000 7fc664e5e740 1 -- start start 2024-06-10T01:08:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.716+0000 7fc664e5e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557f6c4c010 0x5557f6c47120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.716+0000 7fc664e5e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557f6c4ca00 0x5557f6c65fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.716+0000 7fc664e5e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557f6c51b70 0x5557f6c4dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.716+0000 7fc664e5e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c1aaeb0 con 0x5557f6c4c010 2024-06-10T01:08:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.716+0000 7fc664e5e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c1a8780 con 0x5557f6c4ca00 2024-06-10T01:08:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.716+0000 7fc664e5e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c1a65d0 con 0x5557f6c51b70 2024-06-10T01:08:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.716+0000 7fc65b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557f6c4c010 0x5557f6c47120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.716+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557f6c51b70 0x5557f6c4dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.716+0000 7fc65b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557f6c4c010 0x5557f6c47120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59524/0 (socket says 172.21.15.37:59524) 2024-06-10T01:08:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.716+0000 7fc65b7fe640 1 -- 172.21.15.37:0/1429558837 learned_addr learned my addr 172.21.15.37:0/1429558837 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.717+0000 7fc65affd640 1 --2- 172.21.15.37:0/1429558837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557f6c4ca00 0x5557f6c65fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.717+0000 7fc65bfff640 1 -- 172.21.15.37:0/1429558837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557f6c4ca00 msgr2=0x5557f6c65fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.717+0000 7fc65bfff640 1 --2- 172.21.15.37:0/1429558837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557f6c4ca00 0x5557f6c65fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.717+0000 7fc65bfff640 1 -- 172.21.15.37:0/1429558837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557f6c4c010 msgr2=0x5557f6c47120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.717+0000 7fc65bfff640 1 --2- 172.21.15.37:0/1429558837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557f6c4c010 0x5557f6c47120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.717+0000 7fc65bfff640 1 -- 172.21.15.37:0/1429558837 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557f6c4cde0 con 0x5557f6c51b70 2024-06-10T01:08:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.717+0000 7fc65bfff640 1 --2- 172.21.15.37:0/1429558837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557f6c51b70 0x5557f6c4dbf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc654020040 tx=0x7fc654026000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3957f809175b0bb8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.717+0000 7fc6417fa640 1 -- 172.21.15.37:0/1429558837 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc654038020 con 0x5557f6c51b70 2024-06-10T01:08:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.717+0000 7fc6417fa640 1 -- 172.21.15.37:0/1429558837 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc654010390 con 0x5557f6c51b70 2024-06-10T01:08:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.717+0000 7fc6417fa640 1 -- 172.21.15.37:0/1429558837 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc654011640 con 0x5557f6c51b70 2024-06-10T01:08:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.718+0000 7fc664e5e740 1 -- 172.21.15.37:0/1429558837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557f6c51b70 msgr2=0x5557f6c4dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.718+0000 7fc664e5e740 1 --2- 172.21.15.37:0/1429558837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557f6c51b70 0x5557f6c4dbf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc654020040 tx=0x7fc654026000 comp rx=0 tx=0).stop 2024-06-10T01:08:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.718+0000 7fc664e5e740 1 -- 172.21.15.37:0/1429558837 shutdown_connections 2024-06-10T01:08:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.718+0000 7fc664e5e740 1 --2- 172.21.15.37:0/1429558837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557f6c51b70 0x5557f6c4dbf0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.718+0000 7fc664e5e740 1 --2- 172.21.15.37:0/1429558837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557f6c4ca00 0x5557f6c65fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.718+0000 7fc664e5e740 1 --2- 172.21.15.37:0/1429558837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557f6c4c010 0x5557f6c47120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.718+0000 7fc664e5e740 1 -- 172.21.15.37:0/1429558837 >> 172.21.15.37:0/1429558837 conn(0x5557f6c47680 msgr2=0x5557f6c51280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.718+0000 7fc664e5e740 1 -- 172.21.15.37:0/1429558837 shutdown_connections 2024-06-10T01:08:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.718+0000 7fc664e5e740 1 -- 172.21.15.37:0/1429558837 wait complete. 2024-06-10T01:08:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.718+0000 7fc664e5e740 1 -- start start 2024-06-10T01:08:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.719+0000 7fc664e5e740 1 Processor -- start 2024-06-10T01:08:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.719+0000 7fc664e5e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557f6c4c010 0x5557f6c62b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.719+0000 7fc664e5e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557f6c4ca00 0x5557f6c63050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.719+0000 7fc664e5e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557f6c51b70 0x5557f6cd3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.719+0000 7fc664e5e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c1a6c40 con 0x5557f6c51b70 2024-06-10T01:08:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.719+0000 7fc664e5e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c1a80b0 con 0x5557f6c4ca00 2024-06-10T01:08:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.719+0000 7fc664e5e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c1a7310 con 0x5557f6c4c010 2024-06-10T01:08:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.719+0000 7fc664e5e740 10 client.0 _ll_register_callbacks cb 0x7fc650097f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.719+0000 7fc65b7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557f6c4c010 0x5557f6c62b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.719+0000 7fc65bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557f6c51b70 0x5557f6cd3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.719+0000 7fc65b7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557f6c4c010 0x5557f6c62b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:33838/0 (socket says 172.21.15.37:33838) 2024-06-10T01:08:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.719+0000 7fc65b7fe640 1 -- 172.21.15.37:0/2350442446 learned_addr learned my addr 172.21.15.37:0/2350442446 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.719+0000 7fc65affd640 1 --2- 172.21.15.37:0/2350442446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557f6c4ca00 0x5557f6c63050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.719+0000 7fc65bfff640 1 -- 172.21.15.37:0/2350442446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557f6c4c010 msgr2=0x5557f6c62b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.719+0000 7fc65bfff640 1 --2- 172.21.15.37:0/2350442446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557f6c4c010 0x5557f6c62b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.719+0000 7fc65bfff640 1 -- 172.21.15.37:0/2350442446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557f6c4ca00 msgr2=0x5557f6c63050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.719+0000 7fc65bfff640 1 --2- 172.21.15.37:0/2350442446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557f6c4ca00 0x5557f6c63050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.719+0000 7fc65bfff640 1 -- 172.21.15.37:0/2350442446 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557f6c66120 con 0x5557f6c51b70 2024-06-10T01:08:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.720+0000 7fc65bfff640 1 --2- 172.21.15.37:0/2350442446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557f6c51b70 0x5557f6cd3590 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fc65400e6c0 tx=0x7fc654012b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abc9c33de74722b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.720+0000 7fc63affd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:08:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.720+0000 7fc63affd640 1 -- 172.21.15.37:0/2350442446 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc654038040 con 0x5557f6c51b70 2024-06-10T01:08:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.720+0000 7fc63affd640 1 -- 172.21.15.37:0/2350442446 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fc6540115e0 con 0x5557f6c51b70 2024-06-10T01:08:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.720+0000 7fc664e5e740 1 -- 172.21.15.37:0/2350442446 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc654010390 con 0x5557f6c51b70 2024-06-10T01:08:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.720+0000 7fc63affd640 1 -- 172.21.15.37:0/2350442446 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc6540049e0 con 0x5557f6c51b70 2024-06-10T01:08:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.720+0000 7fc664e5e740 10 client.17547 fetch_fsmap learned FSMap version 31 2024-06-10T01:08:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.720+0000 7fc664e5e740 1 -- 172.21.15.37:0/2350442446 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5557f6cd4a20 con 0x5557f6c51b70 2024-06-10T01:08:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.720+0000 7fc63affd640 1 -- 172.21.15.37:0/2350442446 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc654010390 con 0x5557f6c51b70 2024-06-10T01:08:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.720+0000 7fc63affd640 1 -- 172.21.15.37:0/2350442446 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fc65400f040 con 0x5557f6c51b70 2024-06-10T01:08:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.720+0000 7fc664e5e740 10 client.17547 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:08:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.720+0000 7fc664e5e740 10 client.17547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.720+0000 7fc664e5e740 20 client.17547 populate_metadata read hostname 'smithi037' 2024-06-10T01:08:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.720+0000 7fc664e5e740 1 --2- 172.21.15.37:0/2350442446 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5557f6cd5280 0x5557f6cd76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.720+0000 7fc664e5e740 4 client.17547 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:08:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.720+0000 7fc664e5e740 1 -- 172.21.15.37:0/2350442446 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5557f6bb6f70 con 0x5557f6cd5280 2024-06-10T01:08:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.721+0000 7fc65b7fe640 1 --2- 172.21.15.37:0/2350442446 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5557f6cd5280 0x5557f6cd76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.721+0000 7fc65b7fe640 1 --2- 172.21.15.37:0/2350442446 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5557f6cd5280 0x5557f6cd76a0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9538cfe334f764c server_cookie=eb3770095c5f860 in_seq=0 out_seq=0 2024-06-10T01:08:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:34.721+0000 7fc63affd640 10 client.17547 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:36.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:36 smithi167 ceph-mon[36917]: pgmap v788: 129 pgs: 129 active+clean; 15 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 917 KiB/s wr, 389 op/s 2024-06-10T01:08:36.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:36 smithi094 ceph-mon[36708]: pgmap v788: 129 pgs: 129 active+clean; 15 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 917 KiB/s wr, 389 op/s 2024-06-10T01:08:36.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:36 smithi037 ceph-mon[32192]: pgmap v788: 129 pgs: 129 active+clean; 15 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 917 KiB/s wr, 389 op/s 2024-06-10T01:08:38.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:38 smithi167 ceph-mon[36917]: pgmap v789: 129 pgs: 129 active+clean; 17 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 1.0 MiB/s wr, 356 op/s 2024-06-10T01:08:38.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:38 smithi094 ceph-mon[36708]: pgmap v789: 129 pgs: 129 active+clean; 17 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 1.0 MiB/s wr, 356 op/s 2024-06-10T01:08:38.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:38 smithi037 ceph-mon[32192]: pgmap v789: 129 pgs: 129 active+clean; 17 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 1.0 MiB/s wr, 356 op/s 2024-06-10T01:08:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:39.009+0000 7fc63affd640 1 -- 172.21.15.37:0/2350442446 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5557f6bb6f70 con 0x5557f6cd5280 2024-06-10T01:08:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:39.009+0000 7fc63affd640 10 client.17547 handle_command_reply: tid=0 2024-06-10T01:08:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:39.009+0000 7fc664e5e740 1 client.17547 shutdown 2024-06-10T01:08:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:39.009+0000 7fc664e5e740 1 -- 172.21.15.37:0/2350442446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557f6c51b70 msgr2=0x5557f6cd3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:39.009+0000 7fc664e5e740 1 --2- 172.21.15.37:0/2350442446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557f6c51b70 0x5557f6cd3590 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fc65400e6c0 tx=0x7fc654012b30 comp rx=0 tx=0).stop 2024-06-10T01:08:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:39.009+0000 7fc664e5e740 1 -- 172.21.15.37:0/2350442446 shutdown_connections 2024-06-10T01:08:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:39.009+0000 7fc664e5e740 1 --2- 172.21.15.37:0/2350442446 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5557f6cd5280 0x5557f6cd76a0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:39.009+0000 7fc664e5e740 1 --2- 172.21.15.37:0/2350442446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5557f6c51b70 0x5557f6cd3590 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:39.009+0000 7fc664e5e740 1 --2- 172.21.15.37:0/2350442446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557f6c4ca00 0x5557f6c63050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:39.009+0000 7fc664e5e740 1 --2- 172.21.15.37:0/2350442446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557f6c4c010 0x5557f6c62b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:39.009+0000 7fc664e5e740 1 -- 172.21.15.37:0/2350442446 >> 172.21.15.37:0/2350442446 conn(0x5557f6c47680 msgr2=0x5557f6c5e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:39.009+0000 7fc63affd640 0 client.17547 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:39.009+0000 7fc664e5e740 1 -- 172.21.15.37:0/2350442446 shutdown_connections 2024-06-10T01:08:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:39.010+0000 7fc664e5e740 1 -- 172.21.15.37:0/2350442446 wait complete. 2024-06-10T01:08:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:39.010+0000 7fc664e5e740 20 client.17547 trim_cache size 0 max 16384 2024-06-10T01:08:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:08:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:08:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:08:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:39.011+0000 7fc662153640 1 -- 172.21.15.37:0/1168845676 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc6440a2c30 msgr2=0x7fc6440a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:39.011+0000 7fc662153640 1 --2- 172.21.15.37:0/1168845676 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc6440a2c30 0x7fc6440a50f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc65401f000 tx=0x7fc654022000 comp rx=0 tx=0).stop 2024-06-10T01:08:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:39.011+0000 7fc662153640 1 -- 172.21.15.37:0/1168845676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc65c104db0 msgr2=0x7fc65c100430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:39.011+0000 7fc662153640 1 --2- 172.21.15.37:0/1168845676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc65c104db0 0x7fc65c100430 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fc65000e9f0 tx=0x7fc65000eeb0 comp rx=0 tx=0).stop 2024-06-10T01:08:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:39.011+0000 7fc662153640 1 -- 172.21.15.37:0/1168845676 shutdown_connections 2024-06-10T01:08:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:39.011+0000 7fc662153640 1 --2- 172.21.15.37:0/1168845676 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc6440a2c30 0x7fc6440a50f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:39.011+0000 7fc662153640 1 --2- 172.21.15.37:0/1168845676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc65c1076e0 0x7fc65c100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:39.011+0000 7fc662153640 1 --2- 172.21.15.37:0/1168845676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc65c104db0 0x7fc65c100430 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:39.011+0000 7fc662153640 1 --2- 172.21.15.37:0/1168845676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc65c102480 0x7fc65c101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:39.011+0000 7fc662153640 1 -- 172.21.15.37:0/1168845676 >> 172.21.15.37:0/1168845676 conn(0x7fc65c0fbe70 msgr2=0x7fc65c102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:39.011+0000 7fc662153640 1 -- 172.21.15.37:0/1168845676 shutdown_connections 2024-06-10T01:08:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:39.011+0000 7fc662153640 1 -- 172.21.15.37:0/1168845676 wait complete. 2024-06-10T01:08:40.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:40 smithi167 ceph-mon[36917]: pgmap v790: 129 pgs: 129 active+clean; 18 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 607 op/s 2024-06-10T01:08:40.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:40 smithi094 ceph-mon[36708]: pgmap v790: 129 pgs: 129 active+clean; 18 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 607 op/s 2024-06-10T01:08:40.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:40 smithi037 ceph-mon[32192]: pgmap v790: 129 pgs: 129 active+clean; 18 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 607 op/s 2024-06-10T01:08:42.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:42 smithi167 ceph-mon[36917]: pgmap v791: 129 pgs: 129 active+clean; 17 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 639 op/s 2024-06-10T01:08:42.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:42 smithi094 ceph-mon[36708]: pgmap v791: 129 pgs: 129 active+clean; 17 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 639 op/s 2024-06-10T01:08:42.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:42 smithi037 ceph-mon[32192]: pgmap v791: 129 pgs: 129 active+clean; 17 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 639 op/s 2024-06-10T01:08:44.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:44 smithi167 ceph-mon[36917]: pgmap v792: 129 pgs: 129 active+clean; 17 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 698 op/s 2024-06-10T01:08:44.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:44 smithi094 ceph-mon[36708]: pgmap v792: 129 pgs: 129 active+clean; 17 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 698 op/s 2024-06-10T01:08:44.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:44 smithi037 ceph-mon[32192]: pgmap v792: 129 pgs: 129 active+clean; 17 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 698 op/s 2024-06-10T01:08:46.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:46 smithi167 ceph-mon[36917]: pgmap v793: 129 pgs: 129 active+clean; 11 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 790 op/s 2024-06-10T01:08:46.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:46 smithi094 ceph-mon[36708]: pgmap v793: 129 pgs: 129 active+clean; 11 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 790 op/s 2024-06-10T01:08:46.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:46 smithi037 ceph-mon[32192]: pgmap v793: 129 pgs: 129 active+clean; 11 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 790 op/s 2024-06-10T01:08:48.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:48 smithi167 ceph-mon[36917]: pgmap v794: 129 pgs: 129 active+clean; 11 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 640 op/s 2024-06-10T01:08:48.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:48 smithi094 ceph-mon[36708]: pgmap v794: 129 pgs: 129 active+clean; 11 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 640 op/s 2024-06-10T01:08:48.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:48 smithi037 ceph-mon[32192]: pgmap v794: 129 pgs: 129 active+clean; 11 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 640 op/s 2024-06-10T01:08:50.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:50 smithi167 ceph-mon[36917]: pgmap v795: 129 pgs: 129 active+clean; 17 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 844 op/s 2024-06-10T01:08:50.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:50 smithi094 ceph-mon[36708]: pgmap v795: 129 pgs: 129 active+clean; 17 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 844 op/s 2024-06-10T01:08:50.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:50 smithi037 ceph-mon[32192]: pgmap v795: 129 pgs: 129 active+clean; 17 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 844 op/s 2024-06-10T01:08:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.114+0000 7fa9756df640 1 -- 172.21.15.37:0/1377833457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa970102d40 msgr2=0x7fa9701031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.114+0000 7fa9756df640 1 --2- 172.21.15.37:0/1377833457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa970102d40 0x7fa9701031c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fa964009930 tx=0x7fa96402a420 comp rx=0 tx=0).stop 2024-06-10T01:08:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.114+0000 7fa9756df640 1 -- 172.21.15.37:0/1377833457 shutdown_connections 2024-06-10T01:08:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.114+0000 7fa9756df640 1 --2- 172.21.15.37:0/1377833457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa970103700 0x7fa970109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.114+0000 7fa9756df640 1 --2- 172.21.15.37:0/1377833457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa970102d40 0x7fa9701031c0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.114+0000 7fa9756df640 1 --2- 172.21.15.37:0/1377833457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9700ffcd0 0x7fa9701000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.114+0000 7fa9756df640 1 -- 172.21.15.37:0/1377833457 >> 172.21.15.37:0/1377833457 conn(0x7fa9700fb600 msgr2=0x7fa9700fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.114+0000 7fa9756df640 1 -- 172.21.15.37:0/1377833457 shutdown_connections 2024-06-10T01:08:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.114+0000 7fa9756df640 1 -- 172.21.15.37:0/1377833457 wait complete. 2024-06-10T01:08:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.115+0000 7fa9756df640 1 Processor -- start 2024-06-10T01:08:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.115+0000 7fa9756df640 1 -- start start 2024-06-10T01:08:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.115+0000 7fa9756df640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9700ffcd0 0x7fa9701026b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.115+0000 7fa9756df640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa970102d40 0x7fa970100cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.115+0000 7fa9756df640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa970103700 0x7fa970101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.115+0000 7fa9756df640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa97010d660 con 0x7fa970103700 2024-06-10T01:08:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.115+0000 7fa9756df640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa97010d4e0 con 0x7fa9700ffcd0 2024-06-10T01:08:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.115+0000 7fa9756df640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa97010d7e0 con 0x7fa970102d40 2024-06-10T01:08:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.115+0000 7fa96f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa970103700 0x7fa970101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.115+0000 7fa96f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa970103700 0x7fa970101230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36600/0 (socket says 172.21.15.37:36600) 2024-06-10T01:08:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.115+0000 7fa96e7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa970102d40 0x7fa970100cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.115+0000 7fa96f7fe640 1 -- 172.21.15.37:0/1447295590 learned_addr learned my addr 172.21.15.37:0/1447295590 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.115+0000 7fa96effd640 1 --2- 172.21.15.37:0/1447295590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9700ffcd0 0x7fa9701026b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.116+0000 7fa96f7fe640 1 -- 172.21.15.37:0/1447295590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa970102d40 msgr2=0x7fa970100cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.116+0000 7fa96f7fe640 1 --2- 172.21.15.37:0/1447295590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa970102d40 0x7fa970100cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.116+0000 7fa96f7fe640 1 -- 172.21.15.37:0/1447295590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9700ffcd0 msgr2=0x7fa9701026b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.116+0000 7fa96f7fe640 1 --2- 172.21.15.37:0/1447295590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9700ffcd0 0x7fa9701026b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.116+0000 7fa96f7fe640 1 -- 172.21.15.37:0/1447295590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa970101ac0 con 0x7fa970103700 2024-06-10T01:08:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.116+0000 7fa96e7fc640 1 --2- 172.21.15.37:0/1447295590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa970102d40 0x7fa970100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.116+0000 7fa96effd640 1 --2- 172.21.15.37:0/1447295590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9700ffcd0 0x7fa9701026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:08:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.116+0000 7fa96f7fe640 1 --2- 172.21.15.37:0/1447295590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa970103700 0x7fa970101230 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa96800eb10 tx=0x7fa96800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.116+0000 7fa957fff640 1 -- 172.21.15.37:0/1447295590 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968004b20 con 0x7fa970103700 2024-06-10T01:08:52.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.116+0000 7fa957fff640 1 -- 172.21.15.37:0/1447295590 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa968004cc0 con 0x7fa970103700 2024-06-10T01:08:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.116+0000 7fa9756df640 1 -- 172.21.15.37:0/1447295590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9701a1df0 con 0x7fa970103700 2024-06-10T01:08:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.116+0000 7fa957fff640 1 -- 172.21.15.37:0/1447295590 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968010760 con 0x7fa970103700 2024-06-10T01:08:52.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.116+0000 7fa9756df640 1 -- 172.21.15.37:0/1447295590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9701a2250 con 0x7fa970103700 2024-06-10T01:08:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.118+0000 7fa957fff640 1 -- 172.21.15.37:0/1447295590 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa968005020 con 0x7fa970103700 2024-06-10T01:08:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.119+0000 7fa9783df740 1 Processor -- start 2024-06-10T01:08:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.119+0000 7fa9783df740 1 -- start start 2024-06-10T01:08:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.119+0000 7fa9783df740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b627211010 0x55b627211430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.119+0000 7fa9783df740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b627211a00 0x55b627216500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.119+0000 7fa9783df740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b627216b70 0x55b627219000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.119+0000 7fa9783df740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b6271744f0 con 0x55b627211010 2024-06-10T01:08:52.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.119+0000 7fa9783df740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b627112870 con 0x55b627211a00 2024-06-10T01:08:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.119+0000 7fa9783df740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b6271704b0 con 0x55b627216b70 2024-06-10T01:08:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.119+0000 7fa96e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b627211010 0x55b627211430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.119+0000 7fa96e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b627211010 0x55b627211430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36610/0 (socket says 172.21.15.37:36610) 2024-06-10T01:08:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.119+0000 7fa96e7fc640 1 -- 172.21.15.37:0/2103777693 learned_addr learned my addr 172.21.15.37:0/2103777693 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.119+0000 7fa96e7fc640 1 --2- 172.21.15.37:0/2103777693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b627216b70 0x55b627219000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.119+0000 7fa957fff640 1 --2- 172.21.15.37:0/1447295590 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa9580a2cb0 0x7fa9580a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.119+0000 7fa96effd640 1 --2- 172.21.15.37:0/2103777693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b627211a00 0x55b627216500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.119+0000 7fa96e7fc640 1 -- 172.21.15.37:0/2103777693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b627216b70 msgr2=0x55b627219000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.120+0000 7fa96f7fe640 1 --2- 172.21.15.37:0/1447295590 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa9580a2cb0 0x7fa9580a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.120+0000 7fa96e7fc640 1 --2- 172.21.15.37:0/2103777693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b627216b70 0x55b627219000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.120+0000 7fa96e7fc640 1 -- 172.21.15.37:0/2103777693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b627211a00 msgr2=0x55b627216500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.120+0000 7fa96e7fc640 1 --2- 172.21.15.37:0/2103777693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b627211a00 0x55b627216500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.120+0000 7fa96e7fc640 1 -- 172.21.15.37:0/2103777693 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b62712e9b0 con 0x55b627211010 2024-06-10T01:08:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.120+0000 7fa957fff640 1 -- 172.21.15.37:0/1447295590 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fa9680cfa10 con 0x7fa970103700 2024-06-10T01:08:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.120+0000 7fa96e7fc640 1 --2- 172.21.15.37:0/2103777693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b627211010 0x55b627211430 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa964009230 tx=0x7fa964005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946a007c24e5fb9f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.120+0000 7fa9557fa640 1 -- 172.21.15.37:0/2103777693 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964035040 con 0x55b627211010 2024-06-10T01:08:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.120+0000 7fa96f7fe640 1 --2- 172.21.15.37:0/1447295590 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa9580a2cb0 0x7fa9580a5170 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa9680982e0 tx=0x7fa96806ef40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.120+0000 7fa9557fa640 1 -- 172.21.15.37:0/2103777693 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa964045070 con 0x55b627211010 2024-06-10T01:08:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.120+0000 7fa9557fa640 1 -- 172.21.15.37:0/2103777693 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964004af0 con 0x55b627211010 2024-06-10T01:08:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.121+0000 7fa9783df740 1 -- 172.21.15.37:0/2103777693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b627211010 msgr2=0x55b627211430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.121+0000 7fa9783df740 1 --2- 172.21.15.37:0/2103777693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b627211010 0x55b627211430 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa964009230 tx=0x7fa964005fd0 comp rx=0 tx=0).stop 2024-06-10T01:08:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.121+0000 7fa9783df740 1 -- 172.21.15.37:0/2103777693 shutdown_connections 2024-06-10T01:08:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.121+0000 7fa9783df740 1 --2- 172.21.15.37:0/2103777693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b627216b70 0x55b627219000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.121+0000 7fa9783df740 1 --2- 172.21.15.37:0/2103777693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b627211a00 0x55b627216500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.121+0000 7fa9783df740 1 --2- 172.21.15.37:0/2103777693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b627211010 0x55b627211430 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.121+0000 7fa9783df740 1 -- 172.21.15.37:0/2103777693 >> 172.21.15.37:0/2103777693 conn(0x55b62720c680 msgr2=0x55b62720ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.121+0000 7fa9783df740 1 -- 172.21.15.37:0/2103777693 shutdown_connections 2024-06-10T01:08:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.121+0000 7fa9783df740 1 -- 172.21.15.37:0/2103777693 wait complete. 2024-06-10T01:08:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.121+0000 7fa9783df740 1 -- start start 2024-06-10T01:08:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.121+0000 7fa9783df740 1 Processor -- start 2024-06-10T01:08:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.121+0000 7fa9783df740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b627211010 0x55b62721b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.121+0000 7fa9783df740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b627211a00 0x55b62721b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.122+0000 7fa9783df740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b627216b70 0x55b627222870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.122+0000 7fa9783df740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b627219ac0 con 0x55b627211010 2024-06-10T01:08:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.122+0000 7fa9783df740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b6270fdae0 con 0x55b627211a00 2024-06-10T01:08:52.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.122+0000 7fa9783df740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b627219c40 con 0x55b627216b70 2024-06-10T01:08:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.122+0000 7fa9783df740 10 client.0 _ll_register_callbacks cb 0x55b6271369b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.122+0000 7fa96e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b627211010 0x55b62721b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.122+0000 7fa96e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b627211010 0x55b62721b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36616/0 (socket says 172.21.15.37:36616) 2024-06-10T01:08:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.122+0000 7fa96e7fc640 1 -- 172.21.15.37:0/2222168960 learned_addr learned my addr 172.21.15.37:0/2222168960 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.122+0000 7fa96effd640 1 --2- 172.21.15.37:0/2222168960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b627211a00 0x55b62721b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.122+0000 7fa96e7fc640 1 --2- 172.21.15.37:0/2222168960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b627216b70 0x55b627222870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.122+0000 7fa96e7fc640 1 -- 172.21.15.37:0/2222168960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b627216b70 msgr2=0x55b627222870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.122+0000 7fa96e7fc640 1 --2- 172.21.15.37:0/2222168960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b627216b70 0x55b627222870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.122+0000 7fa96e7fc640 1 -- 172.21.15.37:0/2222168960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b627211a00 msgr2=0x55b62721b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.122+0000 7fa96e7fc640 1 --2- 172.21.15.37:0/2222168960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b627211a00 0x55b62721b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.122+0000 7fa96e7fc640 1 -- 172.21.15.37:0/2222168960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b626e88210 con 0x55b627211010 2024-06-10T01:08:52.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.122+0000 7fa96e7fc640 1 --2- 172.21.15.37:0/2222168960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b627211010 0x55b62721b2f0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa964031440 tx=0x7fa964009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb91955e57bb758 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.122+0000 7fa952ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:08:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.122+0000 7fa952ff5640 1 -- 172.21.15.37:0/2222168960 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964035040 con 0x55b627211010 2024-06-10T01:08:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.122+0000 7fa952ff5640 1 -- 172.21.15.37:0/2222168960 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fa96404cc70 con 0x55b627211010 2024-06-10T01:08:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.122+0000 7fa9783df740 1 -- 172.21.15.37:0/2222168960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa964045070 con 0x55b627211010 2024-06-10T01:08:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.123+0000 7fa952ff5640 1 -- 172.21.15.37:0/2222168960 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa96405c050 con 0x55b627211010 2024-06-10T01:08:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.123+0000 7fa9783df740 10 client.17559 fetch_fsmap learned FSMap version 31 2024-06-10T01:08:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.123+0000 7fa9783df740 1 -- 172.21.15.37:0/2222168960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b626f54620 con 0x55b627211010 2024-06-10T01:08:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.123+0000 7fa952ff5640 1 -- 172.21.15.37:0/2222168960 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa964045070 con 0x55b627211010 2024-06-10T01:08:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.123+0000 7fa952ff5640 1 -- 172.21.15.37:0/2222168960 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fa96403a3f0 con 0x55b627211010 2024-06-10T01:08:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.123+0000 7fa9783df740 10 client.17559 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:08:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.123+0000 7fa9783df740 10 client.17559 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.123+0000 7fa9783df740 20 client.17559 populate_metadata read hostname 'smithi037' 2024-06-10T01:08:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.123+0000 7fa9783df740 1 --2- 172.21.15.37:0/2222168960 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b627226d60 0x55b627229150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.123+0000 7fa9783df740 4 client.17559 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:08:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.123+0000 7fa9783df740 1 -- 172.21.15.37:0/2222168960 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b627187f10 con 0x55b627226d60 2024-06-10T01:08:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.123+0000 7fa96effd640 1 --2- 172.21.15.37:0/2222168960 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b627226d60 0x55b627229150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.124+0000 7fa96effd640 1 --2- 172.21.15.37:0/2222168960 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b627226d60 0x55b627229150 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cab8f32d3e045de server_cookie=6e73d87d3546866 in_seq=0 out_seq=0 2024-06-10T01:08:52.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.124+0000 7fa952ff5640 10 client.17559 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.127+0000 7fa952ff5640 1 -- 172.21.15.37:0/2222168960 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b627187f10 con 0x55b627226d60 2024-06-10T01:08:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.127+0000 7fa952ff5640 10 client.17559 handle_command_reply: tid=0 2024-06-10T01:08:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.127+0000 7fa9783df740 1 client.17559 shutdown 2024-06-10T01:08:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.127+0000 7fa9783df740 1 -- 172.21.15.37:0/2222168960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b627211010 msgr2=0x55b62721b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.127+0000 7fa9783df740 1 --2- 172.21.15.37:0/2222168960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b627211010 0x55b62721b2f0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa964031440 tx=0x7fa964009260 comp rx=0 tx=0).stop 2024-06-10T01:08:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.127+0000 7fa9783df740 1 -- 172.21.15.37:0/2222168960 shutdown_connections 2024-06-10T01:08:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.127+0000 7fa9783df740 1 --2- 172.21.15.37:0/2222168960 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b627226d60 0x55b627229150 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.127+0000 7fa9783df740 1 --2- 172.21.15.37:0/2222168960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b627216b70 0x55b627222870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.127+0000 7fa9783df740 1 --2- 172.21.15.37:0/2222168960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b627211a00 0x55b62721b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.127+0000 7fa952ff5640 0 client.17559 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.127+0000 7fa9783df740 1 --2- 172.21.15.37:0/2222168960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b627211010 0x55b62721b2f0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.127+0000 7fa9783df740 1 -- 172.21.15.37:0/2222168960 >> 172.21.15.37:0/2222168960 conn(0x55b62720c680 msgr2=0x55b62720b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.127+0000 7fa9783df740 1 -- 172.21.15.37:0/2222168960 shutdown_connections 2024-06-10T01:08:52.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.127+0000 7fa9783df740 1 -- 172.21.15.37:0/2222168960 wait complete. 2024-06-10T01:08:52.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.127+0000 7fa9783df740 20 client.17559 trim_cache size 0 max 16384 2024-06-10T01:08:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.143+0000 7fa9783df740 1 Processor -- start 2024-06-10T01:08:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.143+0000 7fa9783df740 1 -- start start 2024-06-10T01:08:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.143+0000 7fa9783df740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b627211010 0x55b62720f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.143+0000 7fa9783df740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b627211a00 0x55b627222800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.143+0000 7fa9783df740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b627216b70 0x55b62720fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.143+0000 7fa9783df740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa9701af0f0 con 0x55b627216b70 2024-06-10T01:08:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.143+0000 7fa9783df740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa9701ac9c0 con 0x55b627211010 2024-06-10T01:08:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.143+0000 7fa9783df740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa9701aa810 con 0x55b627211a00 2024-06-10T01:08:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.143+0000 7fa96e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b627216b70 0x55b62720fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.143+0000 7fa96effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b627211a00 0x55b627222800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.143+0000 7fa96effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b627211a00 0x55b627222800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:42758/0 (socket says 172.21.15.37:42758) 2024-06-10T01:08:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.143+0000 7fa96e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b627216b70 0x55b62720fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36624/0 (socket says 172.21.15.37:36624) 2024-06-10T01:08:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.143+0000 7fa96effd640 1 -- 172.21.15.37:0/3084964813 learned_addr learned my addr 172.21.15.37:0/3084964813 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.143+0000 7fa96e7fc640 1 --2- 172.21.15.37:0/3084964813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b627211010 0x55b62720f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.144+0000 7fa96effd640 1 -- 172.21.15.37:0/3084964813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b627211010 msgr2=0x55b62720f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.144+0000 7fa96effd640 1 --2- 172.21.15.37:0/3084964813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b627211010 0x55b62720f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.144+0000 7fa96effd640 1 -- 172.21.15.37:0/3084964813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b627216b70 msgr2=0x55b62720fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.144+0000 7fa96effd640 1 --2- 172.21.15.37:0/3084964813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b627216b70 0x55b62720fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.144+0000 7fa96effd640 1 -- 172.21.15.37:0/3084964813 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b627211800 con 0x55b627211a00 2024-06-10T01:08:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.144+0000 7fa96e7fc640 1 --2- 172.21.15.37:0/3084964813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b627216b70 0x55b62720fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:08:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.144+0000 7fa96effd640 1 --2- 172.21.15.37:0/3084964813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b627211a00 0x55b627222800 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa96000c8f0 tx=0x7fa9600138f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e88a53f2010585d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.144+0000 7fa954ff9640 1 -- 172.21.15.37:0/3084964813 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9600130d0 con 0x55b627211a00 2024-06-10T01:08:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.144+0000 7fa954ff9640 1 -- 172.21.15.37:0/3084964813 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa960013270 con 0x55b627211a00 2024-06-10T01:08:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.144+0000 7fa954ff9640 1 -- 172.21.15.37:0/3084964813 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9600258e0 con 0x55b627211a00 2024-06-10T01:08:52.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.145+0000 7fa9783df740 1 -- 172.21.15.37:0/3084964813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b627211a00 msgr2=0x55b627222800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:52.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.145+0000 7fa9783df740 1 --2- 172.21.15.37:0/3084964813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b627211a00 0x55b627222800 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa96000c8f0 tx=0x7fa9600138f0 comp rx=0 tx=0).stop 2024-06-10T01:08:52.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.145+0000 7fa9783df740 1 -- 172.21.15.37:0/3084964813 shutdown_connections 2024-06-10T01:08:52.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.145+0000 7fa9783df740 1 --2- 172.21.15.37:0/3084964813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b627216b70 0x55b62720fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:52.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.145+0000 7fa9783df740 1 --2- 172.21.15.37:0/3084964813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b627211a00 0x55b627222800 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:52.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.145+0000 7fa9783df740 1 --2- 172.21.15.37:0/3084964813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b627211010 0x55b62720f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:52.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.145+0000 7fa9783df740 1 -- 172.21.15.37:0/3084964813 >> 172.21.15.37:0/3084964813 conn(0x55b62720c680 msgr2=0x55b62721af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:52.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.145+0000 7fa9783df740 1 -- 172.21.15.37:0/3084964813 shutdown_connections 2024-06-10T01:08:52.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.145+0000 7fa9783df740 1 -- 172.21.15.37:0/3084964813 wait complete. 2024-06-10T01:08:52.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.145+0000 7fa9783df740 1 -- start start 2024-06-10T01:08:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.145+0000 7fa9783df740 1 Processor -- start 2024-06-10T01:08:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.145+0000 7fa9783df740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b627211010 0x55b62721fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.145+0000 7fa9783df740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b627211a00 0x55b6272203a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.145+0000 7fa9783df740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b627216b70 0x55b627221360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.145+0000 7fa9783df740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa9701aae80 con 0x55b627216b70 2024-06-10T01:08:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.145+0000 7fa9783df740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa9701ac2f0 con 0x55b627211010 2024-06-10T01:08:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.145+0000 7fa9783df740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa9701ab550 con 0x55b627211a00 2024-06-10T01:08:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.145+0000 7fa9783df740 10 client.0 _ll_register_callbacks cb 0x7fa9640319c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:08:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.146+0000 7fa96e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b627216b70 0x55b627221360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.146+0000 7fa96e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b627216b70 0x55b627221360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36640/0 (socket says 172.21.15.37:36640) 2024-06-10T01:08:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.146+0000 7fa96e7fc640 1 -- 172.21.15.37:0/376984067 learned_addr learned my addr 172.21.15.37:0/376984067 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:08:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.146+0000 7fa96effd640 1 --2- 172.21.15.37:0/376984067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b627211a00 0x55b6272203a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:52.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.146+0000 7fa96e7fc640 1 --2- 172.21.15.37:0/376984067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b627211010 0x55b62721fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.146+0000 7fa96e7fc640 1 -- 172.21.15.37:0/376984067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b627211a00 msgr2=0x55b6272203a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.146+0000 7fa96e7fc640 1 --2- 172.21.15.37:0/376984067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b627211a00 0x55b6272203a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.146+0000 7fa96e7fc640 1 -- 172.21.15.37:0/376984067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b627211010 msgr2=0x55b62721fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:08:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.146+0000 7fa96e7fc640 1 --2- 172.21.15.37:0/376984067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b627211010 0x55b62721fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.146+0000 7fa96e7fc640 1 -- 172.21.15.37:0/376984067 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b627221950 con 0x55b627216b70 2024-06-10T01:08:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.146+0000 7fa96e7fc640 1 --2- 172.21.15.37:0/376984067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b627216b70 0x55b627221360 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa964000c00 tx=0x7fa964031890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5624dfcec530fdcc server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:08:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.146+0000 7fa952ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:08:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.146+0000 7fa952ff5640 1 -- 172.21.15.37:0/376984067 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964035040 con 0x55b627216b70 2024-06-10T01:08:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.146+0000 7fa952ff5640 1 -- 172.21.15.37:0/376984067 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fa96404c8d0 con 0x55b627216b70 2024-06-10T01:08:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.146+0000 7fa9783df740 1 -- 172.21.15.37:0/376984067 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa960013270 con 0x55b627216b70 2024-06-10T01:08:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.147+0000 7fa952ff5640 1 -- 172.21.15.37:0/376984067 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa964062080 con 0x55b627216b70 2024-06-10T01:08:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.147+0000 7fa9783df740 10 client.17565 fetch_fsmap learned FSMap version 31 2024-06-10T01:08:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.147+0000 7fa9783df740 1 -- 172.21.15.37:0/376984067 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b627291840 con 0x55b627216b70 2024-06-10T01:08:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.147+0000 7fa952ff5640 1 -- 172.21.15.37:0/376984067 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa960013270 con 0x55b627216b70 2024-06-10T01:08:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.147+0000 7fa952ff5640 1 -- 172.21.15.37:0/376984067 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fa964060950 con 0x55b627216b70 2024-06-10T01:08:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.147+0000 7fa9783df740 10 client.17565 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:08:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.147+0000 7fa9783df740 10 client.17565 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:08:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.147+0000 7fa9783df740 20 client.17565 populate_metadata read hostname 'smithi037' 2024-06-10T01:08:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.147+0000 7fa9783df740 1 --2- 172.21.15.37:0/376984067 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b627292070 0x55b627294490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:08:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.147+0000 7fa9783df740 4 client.17565 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:08:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.147+0000 7fa9783df740 1 -- 172.21.15.37:0/376984067 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b627181c20 con 0x55b627292070 2024-06-10T01:08:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.147+0000 7fa96effd640 1 --2- 172.21.15.37:0/376984067 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b627292070 0x55b627294490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:08:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.148+0000 7fa96effd640 1 --2- 172.21.15.37:0/376984067 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b627292070 0x55b627294490 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84328d8d150e64e7 server_cookie=78c9de56b7228e4b in_seq=0 out_seq=0 2024-06-10T01:08:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:52.148+0000 7fa952ff5640 10 client.17565 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:52.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:52 smithi167 ceph-mon[36917]: pgmap v796: 129 pgs: 129 active+clean; 17 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 617 op/s 2024-06-10T01:08:52.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:52 smithi094 ceph-mon[36708]: pgmap v796: 129 pgs: 129 active+clean; 17 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 617 op/s 2024-06-10T01:08:52.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:52 smithi037 ceph-mon[32192]: pgmap v796: 129 pgs: 129 active+clean; 17 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 617 op/s 2024-06-10T01:08:54.322 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:54 smithi037 ceph-mon[32192]: pgmap v797: 129 pgs: 129 active+clean; 17 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.1 MiB/s wr, 626 op/s 2024-06-10T01:08:54.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:54 smithi167 ceph-mon[36917]: pgmap v797: 129 pgs: 129 active+clean; 17 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.1 MiB/s wr, 626 op/s 2024-06-10T01:08:54.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:54 smithi094 ceph-mon[36708]: pgmap v797: 129 pgs: 129 active+clean; 17 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.1 MiB/s wr, 626 op/s 2024-06-10T01:08:55.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:55 smithi037 ceph-mon[32192]: pgmap v798: 129 pgs: 129 active+clean; 21 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 693 op/s 2024-06-10T01:08:55.643 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:55 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:08:55.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:55 smithi167 ceph-mon[36917]: pgmap v798: 129 pgs: 129 active+clean; 21 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 693 op/s 2024-06-10T01:08:55.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:55 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:08:55.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:55 smithi094 ceph-mon[36708]: pgmap v798: 129 pgs: 129 active+clean; 21 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 693 op/s 2024-06-10T01:08:55.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:55 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:08:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:56.637+0000 7fa952ff5640 1 -- 172.21.15.37:0/376984067 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b627181c20 con 0x55b627292070 2024-06-10T01:08:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:56.637+0000 7fa952ff5640 10 client.17565 handle_command_reply: tid=0 2024-06-10T01:08:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:56.637+0000 7fa9783df740 1 client.17565 shutdown 2024-06-10T01:08:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:56.638+0000 7fa9783df740 1 -- 172.21.15.37:0/376984067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b627216b70 msgr2=0x55b627221360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:56.638+0000 7fa9783df740 1 --2- 172.21.15.37:0/376984067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b627216b70 0x55b627221360 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa964000c00 tx=0x7fa964031890 comp rx=0 tx=0).stop 2024-06-10T01:08:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:56.638+0000 7fa9783df740 1 -- 172.21.15.37:0/376984067 shutdown_connections 2024-06-10T01:08:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:56.638+0000 7fa9783df740 1 --2- 172.21.15.37:0/376984067 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55b627292070 0x55b627294490 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:56.638+0000 7fa9783df740 1 --2- 172.21.15.37:0/376984067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b627216b70 0x55b627221360 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:56.638+0000 7fa952ff5640 0 client.17565 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:08:56.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:56.638+0000 7fa9783df740 1 --2- 172.21.15.37:0/376984067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b627211a00 0x55b6272203a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:56.638+0000 7fa9783df740 1 --2- 172.21.15.37:0/376984067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b627211010 0x55b62721fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:56.638+0000 7fa9783df740 1 -- 172.21.15.37:0/376984067 >> 172.21.15.37:0/376984067 conn(0x55b62720c680 msgr2=0x55b62720f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:56.638+0000 7fa9783df740 1 -- 172.21.15.37:0/376984067 shutdown_connections 2024-06-10T01:08:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:56.638+0000 7fa9783df740 1 -- 172.21.15.37:0/376984067 wait complete. 2024-06-10T01:08:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:56.638+0000 7fa9783df740 20 client.17565 trim_cache size 0 max 16384 2024-06-10T01:08:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:08:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:08:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:08:56.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:56.638+0000 7fa9756df640 1 -- 172.21.15.37:0/1447295590 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa9580a2cb0 msgr2=0x7fa9580a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:56.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:56.638+0000 7fa9756df640 1 --2- 172.21.15.37:0/1447295590 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa9580a2cb0 0x7fa9580a5170 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa9680982e0 tx=0x7fa96806ef40 comp rx=0 tx=0).stop 2024-06-10T01:08:56.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:56.638+0000 7fa9756df640 1 -- 172.21.15.37:0/1447295590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa970103700 msgr2=0x7fa970101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:08:56.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:56.638+0000 7fa9756df640 1 --2- 172.21.15.37:0/1447295590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa970103700 0x7fa970101230 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa96800eb10 tx=0x7fa96800c2d0 comp rx=0 tx=0).stop 2024-06-10T01:08:56.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:56.638+0000 7fa9756df640 1 -- 172.21.15.37:0/1447295590 shutdown_connections 2024-06-10T01:08:56.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:56.638+0000 7fa9756df640 1 --2- 172.21.15.37:0/1447295590 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa9580a2cb0 0x7fa9580a5170 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:56.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:56.638+0000 7fa9756df640 1 --2- 172.21.15.37:0/1447295590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa970103700 0x7fa970101230 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:56.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:56.638+0000 7fa9756df640 1 --2- 172.21.15.37:0/1447295590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa970102d40 0x7fa970100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:56.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:56.638+0000 7fa9756df640 1 --2- 172.21.15.37:0/1447295590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9700ffcd0 0x7fa9701026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:08:56.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:56.638+0000 7fa9756df640 1 -- 172.21.15.37:0/1447295590 >> 172.21.15.37:0/1447295590 conn(0x7fa9700fb600 msgr2=0x7fa970106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:08:56.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:56.639+0000 7fa9756df640 1 -- 172.21.15.37:0/1447295590 shutdown_connections 2024-06-10T01:08:56.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:08:56.639+0000 7fa9756df640 1 -- 172.21.15.37:0/1447295590 wait complete. 2024-06-10T01:08:56.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:56 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:08:56.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:56 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:08:56.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:56 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:08:56.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:56 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:08:56.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:56 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:08:56.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:56 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:08:56.891 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:56 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:08:56.891 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:56 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:08:56.891 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:56 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:08:57.833 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:08:57 smithi167 ceph-mon[36917]: pgmap v799: 129 pgs: 129 active+clean; 21 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 910 KiB/s wr, 582 op/s 2024-06-10T01:08:57.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:08:57 smithi094 ceph-mon[36708]: pgmap v799: 129 pgs: 129 active+clean; 21 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 910 KiB/s wr, 582 op/s 2024-06-10T01:08:57.890 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:08:57 smithi037 ceph-mon[32192]: pgmap v799: 129 pgs: 129 active+clean; 21 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 910 KiB/s wr, 582 op/s 2024-06-10T01:09:00.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:00 smithi037 ceph-mon[32192]: pgmap v800: 129 pgs: 129 active+clean; 11 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 956 op/s 2024-06-10T01:09:00.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:00 smithi167 ceph-mon[36917]: pgmap v800: 129 pgs: 129 active+clean; 11 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 956 op/s 2024-06-10T01:09:00.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:00 smithi094 ceph-mon[36708]: pgmap v800: 129 pgs: 129 active+clean; 11 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 956 op/s 2024-06-10T01:09:01.867 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:09:02.125 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:09:02.246 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:02 smithi037 ceph-mon[32192]: pgmap v801: 129 pgs: 129 active+clean; 7.4 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 997 KiB/s wr, 799 op/s 2024-06-10T01:09:02.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:02 smithi167 ceph-mon[36917]: pgmap v801: 129 pgs: 129 active+clean; 7.4 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 997 KiB/s wr, 799 op/s 2024-06-10T01:09:02.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:02 smithi094 ceph-mon[36708]: pgmap v801: 129 pgs: 129 active+clean; 7.4 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 997 KiB/s wr, 799 op/s 2024-06-10T01:09:04.158 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:04 smithi037 ceph-mon[32192]: pgmap v802: 129 pgs: 129 active+clean; 7.6 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1011 KiB/s wr, 833 op/s 2024-06-10T01:09:04.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:04 smithi167 ceph-mon[36917]: pgmap v802: 129 pgs: 129 active+clean; 7.6 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1011 KiB/s wr, 833 op/s 2024-06-10T01:09:04.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:04 smithi094 ceph-mon[36708]: pgmap v802: 129 pgs: 129 active+clean; 7.6 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1011 KiB/s wr, 833 op/s 2024-06-10T01:09:04.885 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:09:04.885 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "scrub active (3140 inodes in the stack)", 2024-06-10T01:09:04.886 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-06-10T01:09:04.886 INFO:teuthology.orchestra.run.smithi037.stdout: "e78a272e-e18b-41f0-b3ca-e304eb51fcfe": { 2024-06-10T01:09:04.886 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-06-10T01:09:04.886 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "e78a272e-e18b-41f0-b3ca-e304eb51fcfe", 2024-06-10T01:09:04.886 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-06-10T01:09:04.886 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:09:04.886 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:09:04.886 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:09:04.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.078+0000 7f8e6de1f700 1 -- 172.21.15.37:0/163989839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e6806abe0 msgr2=0x7f8e6806b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:04.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.078+0000 7f8e6de1f700 1 --2- 172.21.15.37:0/163989839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e6806abe0 0x7f8e6806b060 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c009990 tx=0x7f8e5c009ca0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.078+0000 7f8e6de1f700 1 -- 172.21.15.37:0/163989839 shutdown_connections 2024-06-10T01:09:04.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.078+0000 7f8e6de1f700 1 --2- 172.21.15.37:0/163989839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e6806b5a0 0x7f8e6810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.078+0000 7f8e6de1f700 1 --2- 172.21.15.37:0/163989839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e68108500 0x7f8e681088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.078+0000 7f8e6de1f700 1 --2- 172.21.15.37:0/163989839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e6806abe0 0x7f8e6806b060 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.078+0000 7f8e6de1f700 1 -- 172.21.15.37:0/163989839 >> 172.21.15.37:0/163989839 conn(0x7f8e68078360 msgr2=0x7f8e68078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:04.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.078+0000 7f8e6de1f700 1 -- 172.21.15.37:0/163989839 shutdown_connections 2024-06-10T01:09:04.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.078+0000 7f8e6de1f700 1 -- 172.21.15.37:0/163989839 wait complete. 2024-06-10T01:09:04.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.079+0000 7f8e6de1f700 1 Processor -- start 2024-06-10T01:09:04.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.079+0000 7f8e6de1f700 1 -- start start 2024-06-10T01:09:04.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.079+0000 7f8e6de1f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e6806abe0 0x7f8e68101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:04.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.079+0000 7f8e6de1f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e6806b5a0 0x7f8e681019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:04.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.079+0000 7f8e6de1f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e68108500 0x7f8e68101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:04.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.079+0000 7f8e6de1f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8e681127b0 con 0x7f8e68108500 2024-06-10T01:09:04.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.079+0000 7f8e6de1f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8e68112630 con 0x7f8e6806abe0 2024-06-10T01:09:04.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.079+0000 7f8e6de1f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8e68112930 con 0x7f8e6806b5a0 2024-06-10T01:09:04.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.079+0000 7f8e67fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e68108500 0x7f8e68101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:04.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.079+0000 7f8e66ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e6806b5a0 0x7f8e681019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:04.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.079+0000 7f8e67fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e68108500 0x7f8e68101f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44878/0 (socket says 172.21.15.37:44878) 2024-06-10T01:09:04.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.079+0000 7f8e66ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e6806b5a0 0x7f8e681019d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:56394/0 (socket says 172.21.15.37:56394) 2024-06-10T01:09:04.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.079+0000 7f8e67fff700 1 -- 172.21.15.37:0/1289939090 learned_addr learned my addr 172.21.15.37:0/1289939090 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:04.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.079+0000 7f8e677fe700 1 --2- 172.21.15.37:0/1289939090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e6806abe0 0x7f8e68101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:04.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.080+0000 7f8e67fff700 1 -- 172.21.15.37:0/1289939090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e6806b5a0 msgr2=0x7f8e681019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:04.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.080+0000 7f8e67fff700 1 --2- 172.21.15.37:0/1289939090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e6806b5a0 0x7f8e681019d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.080+0000 7f8e67fff700 1 -- 172.21.15.37:0/1289939090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e6806abe0 msgr2=0x7f8e68101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:04.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.080+0000 7f8e67fff700 1 --2- 172.21.15.37:0/1289939090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e6806abe0 0x7f8e68101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.080+0000 7f8e67fff700 1 -- 172.21.15.37:0/1289939090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e681a71b0 con 0x7f8e68108500 2024-06-10T01:09:04.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.080+0000 7f8e66ffd700 1 --2- 172.21.15.37:0/1289939090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e6806b5a0 0x7f8e681019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:04.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.080+0000 7f8e67fff700 1 --2- 172.21.15.37:0/1289939090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e68108500 0x7f8e68101f10 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000ef30 tx=0x7f8e6000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:04.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.080+0000 7f8e64ff9700 1 -- 172.21.15.37:0/1289939090 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e600092e0 con 0x7f8e68108500 2024-06-10T01:09:04.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.080+0000 7f8e64ff9700 1 -- 172.21.15.37:0/1289939090 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e60004bb0 con 0x7f8e68108500 2024-06-10T01:09:04.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.080+0000 7f8e6de1f700 1 -- 172.21.15.37:0/1289939090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e681a7440 con 0x7f8e68108500 2024-06-10T01:09:04.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.080+0000 7f8e64ff9700 1 -- 172.21.15.37:0/1289939090 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e60004280 con 0x7f8e68108500 2024-06-10T01:09:04.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.080+0000 7f8e6de1f700 1 -- 172.21.15.37:0/1289939090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e681a79d0 con 0x7f8e68108500 2024-06-10T01:09:04.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.082+0000 7f8e64ff9700 1 -- 172.21.15.37:0/1289939090 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8e6001b030 con 0x7f8e68108500 2024-06-10T01:09:04.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.083+0000 7f8e64ff9700 1 --2- 172.21.15.37:0/1289939090 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8e500a2d20 0x7f8e500a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:04.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.083+0000 7f8e64ff9700 1 -- 172.21.15.37:0/1289939090 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f8e600ceed0 con 0x7f8e68108500 2024-06-10T01:09:04.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.083+0000 7f8e677fe700 1 --2- 172.21.15.37:0/1289939090 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8e500a2d20 0x7f8e500a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:04.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.084+0000 7f8e677fe700 1 --2- 172.21.15.37:0/1289939090 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8e500a2d20 0x7f8e500a51e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009c30 tx=0x7f8e58009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:04.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.085+0000 7f8e786a2b80 1 Processor -- start 2024-06-10T01:09:04.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.085+0000 7f8e786a2b80 1 -- start start 2024-06-10T01:09:04.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.085+0000 7f8e786a2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556717f8a640 0x556717f8aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:04.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.085+0000 7f8e786a2b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556717f8af60 0x556717f849e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:04.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.085+0000 7f8e786a2b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556717f84f20 0x556717f85340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:04.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.085+0000 7f8e786a2b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556717ed6810 con 0x556717f84f20 2024-06-10T01:09:04.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.085+0000 7f8e786a2b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556717e7e720 con 0x556717f8af60 2024-06-10T01:09:04.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.085+0000 7f8e786a2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556717ed27d0 con 0x556717f8a640 2024-06-10T01:09:04.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.085+0000 7f8e677fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556717f8a640 0x556717f8aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:04.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.086+0000 7f8e67fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556717f84f20 0x556717f85340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:04.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.086+0000 7f8e677fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556717f8a640 0x556717f8aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:56406/0 (socket says 172.21.15.37:56406) 2024-06-10T01:09:04.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.086+0000 7f8e677fe700 1 -- 172.21.15.37:0/2316018370 learned_addr learned my addr 172.21.15.37:0/2316018370 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:04.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.086+0000 7f8e67fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556717f84f20 0x556717f85340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44894/0 (socket says 172.21.15.37:44894) 2024-06-10T01:09:04.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.086+0000 7f8e66ffd700 1 --2- 172.21.15.37:0/2316018370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556717f8af60 0x556717f849e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:04.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.086+0000 7f8e677fe700 1 -- 172.21.15.37:0/2316018370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556717f8af60 msgr2=0x556717f849e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:04.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.086+0000 7f8e677fe700 1 --2- 172.21.15.37:0/2316018370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556717f8af60 0x556717f849e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.086+0000 7f8e677fe700 1 -- 172.21.15.37:0/2316018370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556717f84f20 msgr2=0x556717f85340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:04.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.086+0000 7f8e677fe700 1 --2- 172.21.15.37:0/2316018370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556717f84f20 0x556717f85340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.086+0000 7f8e677fe700 1 -- 172.21.15.37:0/2316018370 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556717e74f50 con 0x556717f8a640 2024-06-10T01:09:04.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.086+0000 7f8e677fe700 1 --2- 172.21.15.37:0/2316018370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556717f8a640 0x556717f8aa20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8e58011d80 tx=0x7f8e58014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14725fc17010090f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:04.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.086+0000 7f8e4dffb700 1 -- 172.21.15.37:0/2316018370 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58011db0 con 0x556717f8a640 2024-06-10T01:09:04.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.086+0000 7f8e4dffb700 1 -- 172.21.15.37:0/2316018370 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e58004020 con 0x556717f8a640 2024-06-10T01:09:04.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.086+0000 7f8e4dffb700 1 -- 172.21.15.37:0/2316018370 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58003c70 con 0x556717f8a640 2024-06-10T01:09:04.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.087+0000 7f8e786a2b80 1 -- 172.21.15.37:0/2316018370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556717f8a640 msgr2=0x556717f8aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:04.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.087+0000 7f8e786a2b80 1 --2- 172.21.15.37:0/2316018370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556717f8a640 0x556717f8aa20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8e58011d80 tx=0x7f8e58014000 comp rx=0 tx=0).stop 2024-06-10T01:09:04.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.087+0000 7f8e786a2b80 1 -- 172.21.15.37:0/2316018370 shutdown_connections 2024-06-10T01:09:04.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.087+0000 7f8e786a2b80 1 --2- 172.21.15.37:0/2316018370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556717f84f20 0x556717f85340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.087+0000 7f8e786a2b80 1 --2- 172.21.15.37:0/2316018370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556717f8a640 0x556717f8aa20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.087+0000 7f8e786a2b80 1 --2- 172.21.15.37:0/2316018370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556717f8af60 0x556717f849e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.087+0000 7f8e786a2b80 1 -- 172.21.15.37:0/2316018370 >> 172.21.15.37:0/2316018370 conn(0x556717eafcc0 msgr2=0x556717f7f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:04.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.087+0000 7f8e786a2b80 1 -- 172.21.15.37:0/2316018370 shutdown_connections 2024-06-10T01:09:04.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.087+0000 7f8e786a2b80 1 -- 172.21.15.37:0/2316018370 wait complete. 2024-06-10T01:09:04.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.088+0000 7f8e786a2b80 1 -- start start 2024-06-10T01:09:04.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.088+0000 7f8e786a2b80 1 Processor -- start 2024-06-10T01:09:04.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.088+0000 7f8e786a2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556717f84f20 0x556717f90c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:04.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.088+0000 7f8e786a2b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556717f8a640 0x556717f8df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:04.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.088+0000 7f8e786a2b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556717f8af60 0x556717f8e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:04.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.088+0000 7f8e677fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556717f84f20 0x556717f90c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:04.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.088+0000 7f8e786a2b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556717f919d0 con 0x556717f8af60 2024-06-10T01:09:04.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.088+0000 7f8e786a2b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556717e76bf0 con 0x556717f8a640 2024-06-10T01:09:04.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.088+0000 7f8e786a2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556717f91b50 con 0x556717f84f20 2024-06-10T01:09:04.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.088+0000 7f8e677fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556717f84f20 0x556717f90c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:56410/0 (socket says 172.21.15.37:56410) 2024-06-10T01:09:04.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.088+0000 7f8e677fe700 1 -- 172.21.15.37:0/203991982 learned_addr learned my addr 172.21.15.37:0/203991982 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:04.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.088+0000 7f8e66ffd700 1 --2- 172.21.15.37:0/203991982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556717f8a640 0x556717f8df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:04.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.088+0000 7f8e67fff700 1 --2- 172.21.15.37:0/203991982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556717f8af60 0x556717f8e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:04.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.089+0000 7f8e786a2b80 10 client.0 _ll_register_callbacks cb 0x556717e70920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:04.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.089+0000 7f8e677fe700 1 -- 172.21.15.37:0/203991982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556717f8a640 msgr2=0x556717f8df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:04.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.089+0000 7f8e677fe700 1 --2- 172.21.15.37:0/203991982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556717f8a640 0x556717f8df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.089+0000 7f8e677fe700 1 -- 172.21.15.37:0/203991982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556717f8af60 msgr2=0x556717f8e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:04.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.089+0000 7f8e677fe700 1 --2- 172.21.15.37:0/203991982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556717f8af60 0x556717f8e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.089+0000 7f8e677fe700 1 -- 172.21.15.37:0/203991982 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556717e258d0 con 0x556717f84f20 2024-06-10T01:09:04.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.089+0000 7f8e786a2b80 1 -- 172.21.15.37:0/203991982 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e58004020 con 0x556717f84f20 2024-06-10T01:09:04.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.089+0000 7f8e677fe700 1 --2- 172.21.15.37:0/203991982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556717f84f20 0x556717f90c60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8e580099f0 tx=0x7f8e58003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19ec20d0d30ccba8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:04.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.089+0000 7f8e46ffd700 10 client.27064 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:09:04.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.089+0000 7f8e46ffd700 1 -- 172.21.15.37:0/203991982 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58034060 con 0x556717f84f20 2024-06-10T01:09:04.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.089+0000 7f8e46ffd700 1 -- 172.21.15.37:0/203991982 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f8e5800eb20 con 0x556717f84f20 2024-06-10T01:09:04.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.089+0000 7f8e46ffd700 1 -- 172.21.15.37:0/203991982 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8e5800ed20 con 0x556717f84f20 2024-06-10T01:09:04.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.089+0000 7f8e786a2b80 10 client.27064 fetch_fsmap learned FSMap version 31 2024-06-10T01:09:04.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.089+0000 7f8e786a2b80 1 -- 172.21.15.37:0/203991982 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=31}) -- 0x556717f9c1f0 con 0x556717f84f20 2024-06-10T01:09:04.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.090+0000 7f8e46ffd700 1 -- 172.21.15.37:0/203991982 <== mon.1 v2:172.21.15.167:3300/0 4 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f8e580038f0 con 0x556717f84f20 2024-06-10T01:09:04.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.090+0000 7f8e786a2b80 10 client.27064 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:09:04.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.090+0000 7f8e786a2b80 10 client.27064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:04.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.090+0000 7f8e786a2b80 20 client.27064 populate_metadata read hostname 'smithi037' 2024-06-10T01:09:04.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.090+0000 7f8e786a2b80 1 --2- 172.21.15.37:0/203991982 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x556717f9e6c0 0x556717fa0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:04.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.090+0000 7f8e786a2b80 4 client.27064 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:09:04.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.090+0000 7f8e786a2b80 1 -- 172.21.15.37:0/203991982 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556717ee3d10 con 0x556717f9e6c0 2024-06-10T01:09:04.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.090+0000 7f8e66ffd700 1 --2- 172.21.15.37:0/203991982 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x556717f9e6c0 0x556717fa0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:04.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.091+0000 7f8e46ffd700 10 client.27064 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:04.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.091+0000 7f8e66ffd700 1 --2- 172.21.15.37:0/203991982 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x556717f9e6c0 0x556717fa0ab0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d69a66ec0b4ff0d6 server_cookie=9a17759a0bf8361e in_seq=0 out_seq=0 2024-06-10T01:09:04.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.094+0000 7f8e46ffd700 1 -- 172.21.15.37:0/203991982 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556717ee3d10 con 0x556717f9e6c0 2024-06-10T01:09:04.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.094+0000 7f8e46ffd700 10 client.27064 handle_command_reply: tid=0 2024-06-10T01:09:04.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.094+0000 7f8e786a2b80 1 client.27064 shutdown 2024-06-10T01:09:04.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.094+0000 7f8e786a2b80 1 -- 172.21.15.37:0/203991982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556717f84f20 msgr2=0x556717f90c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:04.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.094+0000 7f8e786a2b80 1 --2- 172.21.15.37:0/203991982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556717f84f20 0x556717f90c60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8e580099f0 tx=0x7f8e58003680 comp rx=0 tx=0).stop 2024-06-10T01:09:04.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.094+0000 7f8e786a2b80 1 -- 172.21.15.37:0/203991982 shutdown_connections 2024-06-10T01:09:04.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.094+0000 7f8e786a2b80 1 --2- 172.21.15.37:0/203991982 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x556717f9e6c0 0x556717fa0ab0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.094+0000 7f8e786a2b80 1 --2- 172.21.15.37:0/203991982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556717f8af60 0x556717f8e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.094+0000 7f8e786a2b80 1 --2- 172.21.15.37:0/203991982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556717f84f20 0x556717f90c60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.094+0000 7f8e786a2b80 1 --2- 172.21.15.37:0/203991982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556717f8a640 0x556717f8df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.094+0000 7f8e46ffd700 0 client.27064 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:04.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.094+0000 7f8e786a2b80 1 -- 172.21.15.37:0/203991982 >> 172.21.15.37:0/203991982 conn(0x556717eafcc0 msgr2=0x556717f839d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:04.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.094+0000 7f8e786a2b80 1 -- 172.21.15.37:0/203991982 shutdown_connections 2024-06-10T01:09:04.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.094+0000 7f8e786a2b80 1 -- 172.21.15.37:0/203991982 wait complete. 2024-06-10T01:09:04.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.094+0000 7f8e786a2b80 20 client.27064 trim_cache size 0 max 16384 2024-06-10T01:09:04.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.120+0000 7f8e786a2b80 1 Processor -- start 2024-06-10T01:09:04.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.120+0000 7f8e786a2b80 1 -- start start 2024-06-10T01:09:04.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.120+0000 7f8e786a2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556717f84f20 0x556717f849e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:04.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.120+0000 7f8e786a2b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556717f8a640 0x556717f8bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:04.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.120+0000 7f8e786a2b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556717f8af60 0x556717f8cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:04.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.120+0000 7f8e786a2b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8e681b3010 con 0x556717f8a640 2024-06-10T01:09:04.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.120+0000 7f8e786a2b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8e681b08a0 con 0x556717f8af60 2024-06-10T01:09:04.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.120+0000 7f8e786a2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8e681ae6f0 con 0x556717f84f20 2024-06-10T01:09:04.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.121+0000 7f8e677fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556717f84f20 0x556717f849e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:04.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.121+0000 7f8e66ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556717f8a640 0x556717f8bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:04.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.121+0000 7f8e677fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556717f84f20 0x556717f849e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:56420/0 (socket says 172.21.15.37:56420) 2024-06-10T01:09:04.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.121+0000 7f8e66ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556717f8a640 0x556717f8bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44906/0 (socket says 172.21.15.37:44906) 2024-06-10T01:09:04.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.121+0000 7f8e677fe700 1 -- 172.21.15.37:0/3759559864 learned_addr learned my addr 172.21.15.37:0/3759559864 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:04.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.121+0000 7f8e67fff700 1 --2- 172.21.15.37:0/3759559864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556717f8af60 0x556717f8cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:04.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.121+0000 7f8e677fe700 1 -- 172.21.15.37:0/3759559864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556717f8af60 msgr2=0x556717f8cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:04.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.121+0000 7f8e677fe700 1 --2- 172.21.15.37:0/3759559864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556717f8af60 0x556717f8cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.121+0000 7f8e677fe700 1 -- 172.21.15.37:0/3759559864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556717f8a640 msgr2=0x556717f8bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:04.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.121+0000 7f8e677fe700 1 --2- 172.21.15.37:0/3759559864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556717f8a640 0x556717f8bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.121+0000 7f8e677fe700 1 -- 172.21.15.37:0/3759559864 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556717f8ad70 con 0x556717f84f20 2024-06-10T01:09:04.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.121+0000 7f8e66ffd700 1 --2- 172.21.15.37:0/3759559864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556717f8a640 0x556717f8bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:04.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.121+0000 7f8e677fe700 1 --2- 172.21.15.37:0/3759559864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556717f84f20 0x556717f849e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8e58021040 tx=0x7f8e58013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9021533ab76da0f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:04.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.121+0000 7f8e4d7fa700 1 -- 172.21.15.37:0/3759559864 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58034070 con 0x556717f84f20 2024-06-10T01:09:04.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.121+0000 7f8e4d7fa700 1 -- 172.21.15.37:0/3759559864 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e58004020 con 0x556717f84f20 2024-06-10T01:09:04.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.121+0000 7f8e4d7fa700 1 -- 172.21.15.37:0/3759559864 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58030c70 con 0x556717f84f20 2024-06-10T01:09:04.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.122+0000 7f8e786a2b80 1 -- 172.21.15.37:0/3759559864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556717f84f20 msgr2=0x556717f849e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:04.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.122+0000 7f8e786a2b80 1 --2- 172.21.15.37:0/3759559864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556717f84f20 0x556717f849e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8e58021040 tx=0x7f8e58013000 comp rx=0 tx=0).stop 2024-06-10T01:09:04.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.122+0000 7f8e786a2b80 1 -- 172.21.15.37:0/3759559864 shutdown_connections 2024-06-10T01:09:04.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.122+0000 7f8e786a2b80 1 --2- 172.21.15.37:0/3759559864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556717f8af60 0x556717f8cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.122+0000 7f8e786a2b80 1 --2- 172.21.15.37:0/3759559864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556717f84f20 0x556717f849e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.122+0000 7f8e786a2b80 1 --2- 172.21.15.37:0/3759559864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556717f8a640 0x556717f8bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.122+0000 7f8e786a2b80 1 -- 172.21.15.37:0/3759559864 >> 172.21.15.37:0/3759559864 conn(0x556717eafcc0 msgr2=0x556717f78e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:04.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.122+0000 7f8e786a2b80 1 -- 172.21.15.37:0/3759559864 shutdown_connections 2024-06-10T01:09:04.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.122+0000 7f8e786a2b80 1 -- 172.21.15.37:0/3759559864 wait complete. 2024-06-10T01:09:04.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.123+0000 7f8e786a2b80 1 -- start start 2024-06-10T01:09:04.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.123+0000 7f8e786a2b80 1 Processor -- start 2024-06-10T01:09:04.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.123+0000 7f8e786a2b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556717f84f20 0x556717fa17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:04.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.123+0000 7f8e786a2b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556717f8a640 0x556717fa1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:04.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.123+0000 7f8e786a2b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556717f8af60 0x556717fa2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:04.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.123+0000 7f8e786a2b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8e681aed60 con 0x556717f84f20 2024-06-10T01:09:04.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.123+0000 7f8e786a2b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8e681b01d0 con 0x556717f8af60 2024-06-10T01:09:04.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.123+0000 7f8e786a2b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8e681af430 con 0x556717f8a640 2024-06-10T01:09:04.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.123+0000 7f8e786a2b80 10 client.0 _ll_register_callbacks cb 0x556717e70920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:04.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.123+0000 7f8e677fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556717f84f20 0x556717fa17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:04.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.123+0000 7f8e677fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556717f84f20 0x556717fa17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44920/0 (socket says 172.21.15.37:44920) 2024-06-10T01:09:04.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.123+0000 7f8e677fe700 1 -- 172.21.15.37:0/176994186 learned_addr learned my addr 172.21.15.37:0/176994186 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:04.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.123+0000 7f8e66ffd700 1 --2- 172.21.15.37:0/176994186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556717f8a640 0x556717fa1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:04.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.123+0000 7f8e67fff700 1 --2- 172.21.15.37:0/176994186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556717f8af60 0x556717fa2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:04.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.123+0000 7f8e677fe700 1 -- 172.21.15.37:0/176994186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556717f8a640 msgr2=0x556717fa1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:04.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.123+0000 7f8e677fe700 1 --2- 172.21.15.37:0/176994186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556717f8a640 0x556717fa1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.124+0000 7f8e677fe700 1 -- 172.21.15.37:0/176994186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556717f8af60 msgr2=0x556717fa2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:04.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.124+0000 7f8e677fe700 1 --2- 172.21.15.37:0/176994186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556717f8af60 0x556717fa2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.124+0000 7f8e677fe700 1 -- 172.21.15.37:0/176994186 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556718008000 con 0x556717f84f20 2024-06-10T01:09:04.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.124+0000 7f8e677fe700 1 --2- 172.21.15.37:0/176994186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556717f84f20 0x556717fa17d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8e58005830 tx=0x7f8e580038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6908e96fec097e5c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:04.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.124+0000 7f8e46ffd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:09:04.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.124+0000 7f8e46ffd700 1 -- 172.21.15.37:0/176994186 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58049070 con 0x556717f84f20 2024-06-10T01:09:04.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.124+0000 7f8e786a2b80 1 -- 172.21.15.37:0/176994186 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e58004020 con 0x556717f84f20 2024-06-10T01:09:04.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.124+0000 7f8e46ffd700 1 -- 172.21.15.37:0/176994186 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f8e58030660 con 0x556717f84f20 2024-06-10T01:09:04.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.124+0000 7f8e46ffd700 1 -- 172.21.15.37:0/176994186 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8e58034070 con 0x556717f84f20 2024-06-10T01:09:04.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.124+0000 7f8e786a2b80 10 client.17583 fetch_fsmap learned FSMap version 31 2024-06-10T01:09:04.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.124+0000 7f8e786a2b80 1 -- 172.21.15.37:0/176994186 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556718009250 con 0x556717f84f20 2024-06-10T01:09:04.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.124+0000 7f8e46ffd700 1 -- 172.21.15.37:0/176994186 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e58004020 con 0x556717f84f20 2024-06-10T01:09:04.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.125+0000 7f8e46ffd700 1 -- 172.21.15.37:0/176994186 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f8e5804c020 con 0x556717f84f20 2024-06-10T01:09:04.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.125+0000 7f8e786a2b80 10 client.17583 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:09:04.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.125+0000 7f8e786a2b80 10 client.17583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:04.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.125+0000 7f8e786a2b80 20 client.17583 populate_metadata read hostname 'smithi037' 2024-06-10T01:09:04.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.125+0000 7f8e786a2b80 1 --2- 172.21.15.37:0/176994186 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x556718009890 0x55671800bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:04.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.125+0000 7f8e786a2b80 4 client.17583 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:09:04.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.125+0000 7f8e786a2b80 1 -- 172.21.15.37:0/176994186 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556717ede290 con 0x556718009890 2024-06-10T01:09:04.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.125+0000 7f8e66ffd700 1 --2- 172.21.15.37:0/176994186 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x556718009890 0x55671800bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:04.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.125+0000 7f8e66ffd700 1 --2- 172.21.15.37:0/176994186 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x556718009890 0x55671800bc80 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=998e264f20b42a89 server_cookie=b8edba521b129c95 in_seq=0 out_seq=0 2024-06-10T01:09:04.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.125+0000 7f8e46ffd700 10 client.17583 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:04.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.129+0000 7f8e46ffd700 1 -- 172.21.15.37:0/176994186 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x556717ede290 con 0x556718009890 2024-06-10T01:09:04.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.129+0000 7f8e46ffd700 10 client.17583 handle_command_reply: tid=0 2024-06-10T01:09:04.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.129+0000 7f8e786a2b80 1 client.17583 shutdown 2024-06-10T01:09:04.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.129+0000 7f8e786a2b80 1 -- 172.21.15.37:0/176994186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556717f84f20 msgr2=0x556717fa17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:04.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.129+0000 7f8e786a2b80 1 --2- 172.21.15.37:0/176994186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556717f84f20 0x556717fa17d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8e58005830 tx=0x7f8e580038b0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.130+0000 7f8e786a2b80 1 -- 172.21.15.37:0/176994186 shutdown_connections 2024-06-10T01:09:04.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.130+0000 7f8e786a2b80 1 --2- 172.21.15.37:0/176994186 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x556718009890 0x55671800bc80 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.130+0000 7f8e786a2b80 1 --2- 172.21.15.37:0/176994186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556717f8af60 0x556717fa2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.130+0000 7f8e786a2b80 1 --2- 172.21.15.37:0/176994186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556717f84f20 0x556717fa17d0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.130+0000 7f8e786a2b80 1 --2- 172.21.15.37:0/176994186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556717f8a640 0x556717fa1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.130+0000 7f8e46ffd700 0 client.17583 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:04.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.130+0000 7f8e786a2b80 1 -- 172.21.15.37:0/176994186 >> 172.21.15.37:0/176994186 conn(0x556717eafcc0 msgr2=0x556717f8ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:04.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.130+0000 7f8e786a2b80 1 -- 172.21.15.37:0/176994186 shutdown_connections 2024-06-10T01:09:04.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.130+0000 7f8e786a2b80 1 -- 172.21.15.37:0/176994186 wait complete. 2024-06-10T01:09:04.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.130+0000 7f8e786a2b80 20 client.17583 trim_cache size 0 max 16384 2024-06-10T01:09:04.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.130+0000 7f8e6de1f700 1 -- 172.21.15.37:0/1289939090 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8e500a2d20 msgr2=0x7f8e500a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:04.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.130+0000 7f8e6de1f700 1 --2- 172.21.15.37:0/1289939090 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8e500a2d20 0x7f8e500a51e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009c30 tx=0x7f8e58009260 comp rx=0 tx=0).stop 2024-06-10T01:09:04.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.130+0000 7f8e6de1f700 1 -- 172.21.15.37:0/1289939090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e68108500 msgr2=0x7f8e68101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:04.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.130+0000 7f8e6de1f700 1 --2- 172.21.15.37:0/1289939090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e68108500 0x7f8e68101f10 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000ef30 tx=0x7f8e6000c3a0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.131+0000 7f8e6de1f700 1 -- 172.21.15.37:0/1289939090 shutdown_connections 2024-06-10T01:09:04.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.131+0000 7f8e6de1f700 1 --2- 172.21.15.37:0/1289939090 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8e500a2d20 0x7f8e500a51e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.131+0000 7f8e6de1f700 1 --2- 172.21.15.37:0/1289939090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e68108500 0x7f8e68101f10 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.131+0000 7f8e6de1f700 1 --2- 172.21.15.37:0/1289939090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e6806abe0 0x7f8e68101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.131+0000 7f8e6de1f700 1 --2- 172.21.15.37:0/1289939090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e6806b5a0 0x7f8e681019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:04.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.131+0000 7f8e6de1f700 1 -- 172.21.15.37:0/1289939090 >> 172.21.15.37:0/1289939090 conn(0x7f8e68078360 msgr2=0x7f8e68100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:04.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.131+0000 7f8e6de1f700 1 -- 172.21.15.37:0/1289939090 shutdown_connections 2024-06-10T01:09:04.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:04.131+0000 7f8e6de1f700 1 -- 172.21.15.37:0/1289939090 wait complete. 2024-06-10T01:09:06.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:06 smithi037 ceph-mon[32192]: pgmap v803: 129 pgs: 129 active+clean; 13 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 942 op/s 2024-06-10T01:09:06.495 INFO:tasks.cephfs.filesystem:scrub status for tag:e78a272e-e18b-41f0-b3ca-e304eb51fcfe - {'path': '/', 'tag': 'e78a272e-e18b-41f0-b3ca-e304eb51fcfe', 'options': 'recursive,force'} 2024-06-10T01:09:06.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:06 smithi167 ceph-mon[36917]: pgmap v803: 129 pgs: 129 active+clean; 13 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 942 op/s 2024-06-10T01:09:06.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:06 smithi094 ceph-mon[36708]: pgmap v803: 129 pgs: 129 active+clean; 13 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 942 op/s 2024-06-10T01:09:08.411 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:08 smithi094 ceph-mon[36708]: pgmap v804: 129 pgs: 129 active+clean; 13 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 815 op/s 2024-06-10T01:09:08.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:08 smithi167 ceph-mon[36917]: pgmap v804: 129 pgs: 129 active+clean; 13 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 815 op/s 2024-06-10T01:09:08.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:08 smithi037 ceph-mon[32192]: pgmap v804: 129 pgs: 129 active+clean; 13 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 815 op/s 2024-06-10T01:09:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.764+0000 7f2269da2640 1 -- 172.21.15.37:0/982181293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2264101af0 msgr2=0x7f2264101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.764+0000 7f2269da2640 1 --2- 172.21.15.37:0/982181293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2264101af0 0x7f2264101f70 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f224c009900 tx=0x7f224c02a410 comp rx=0 tx=0).stop 2024-06-10T01:09:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.764+0000 7f2269da2640 1 -- 172.21.15.37:0/982181293 shutdown_connections 2024-06-10T01:09:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.764+0000 7f2269da2640 1 --2- 172.21.15.37:0/982181293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22641024b0 0x7f2264108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.764+0000 7f2269da2640 1 --2- 172.21.15.37:0/982181293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2264101af0 0x7f2264101f70 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.764+0000 7f2269da2640 1 --2- 172.21.15.37:0/982181293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2264100900 0x7f2264100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.764+0000 7f2269da2640 1 -- 172.21.15.37:0/982181293 >> 172.21.15.37:0/982181293 conn(0x7f22640fbe70 msgr2=0x7f22640fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.765+0000 7f2269da2640 1 -- 172.21.15.37:0/982181293 shutdown_connections 2024-06-10T01:09:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.765+0000 7f2269da2640 1 -- 172.21.15.37:0/982181293 wait complete. 2024-06-10T01:09:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.765+0000 7f2269da2640 1 Processor -- start 2024-06-10T01:09:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.765+0000 7f2269da2640 1 -- start start 2024-06-10T01:09:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.765+0000 7f2269da2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2264100900 0x7f2264197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.765+0000 7f2269da2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2264101af0 0x7f22641981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.765+0000 7f2269da2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22641024b0 0x7f226419f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.765+0000 7f2269da2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f226410d6a0 con 0x7f2264101af0 2024-06-10T01:09:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.765+0000 7f2269da2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f226410d520 con 0x7f2264100900 2024-06-10T01:09:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.765+0000 7f2269da2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f226410d820 con 0x7f22641024b0 2024-06-10T01:09:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.766+0000 7f225affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2264101af0 0x7f22641981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.766+0000 7f225affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2264101af0 0x7f22641981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44940/0 (socket says 172.21.15.37:44940) 2024-06-10T01:09:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.766+0000 7f225affd640 1 -- 172.21.15.37:0/1097837938 learned_addr learned my addr 172.21.15.37:0/1097837938 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.766+0000 7f22637fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2264100900 0x7f2264197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.766+0000 7f2263fff640 1 --2- 172.21.15.37:0/1097837938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22641024b0 0x7f226419f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.766+0000 7f225affd640 1 -- 172.21.15.37:0/1097837938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22641024b0 msgr2=0x7f226419f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.766+0000 7f225affd640 1 --2- 172.21.15.37:0/1097837938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22641024b0 0x7f226419f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.766+0000 7f225affd640 1 -- 172.21.15.37:0/1097837938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2264100900 msgr2=0x7f2264197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.766+0000 7f225affd640 1 --2- 172.21.15.37:0/1097837938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2264100900 0x7f2264197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.766+0000 7f225affd640 1 -- 172.21.15.37:0/1097837938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f226419f8d0 con 0x7f2264101af0 2024-06-10T01:09:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.766+0000 7f225affd640 1 --2- 172.21.15.37:0/1097837938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2264101af0 0x7f22641981d0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f224c009230 tx=0x7f224c031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.766+0000 7f22617fa640 1 -- 172.21.15.37:0/1097837938 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c03f030 con 0x7f2264101af0 2024-06-10T01:09:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.766+0000 7f2269da2640 1 -- 172.21.15.37:0/1097837938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f226419fb60 con 0x7f2264101af0 2024-06-10T01:09:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.766+0000 7f22617fa640 1 -- 172.21.15.37:0/1097837938 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f224c003ad0 con 0x7f2264101af0 2024-06-10T01:09:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.766+0000 7f22617fa640 1 -- 172.21.15.37:0/1097837938 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c035070 con 0x7f2264101af0 2024-06-10T01:09:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.766+0000 7f2269da2640 1 -- 172.21.15.37:0/1097837938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22641098e0 con 0x7f2264101af0 2024-06-10T01:09:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.769+0000 7f226cb90740 1 Processor -- start 2024-06-10T01:09:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.769+0000 7f226cb90740 1 -- start start 2024-06-10T01:09:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.770+0000 7f226cb90740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56549703a820 0x56549703ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.770+0000 7f226cb90740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56549703b210 0x56549703fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.770+0000 7f226cb90740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565497040380 0x565497042810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.770+0000 7f226cb90740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x565496f954f0 con 0x565497040380 2024-06-10T01:09:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.770+0000 7f226cb90740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x565496f33870 con 0x56549703b210 2024-06-10T01:09:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.770+0000 7f226cb90740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x565496f914b0 con 0x56549703a820 2024-06-10T01:09:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.770+0000 7f22617fa640 1 -- 172.21.15.37:0/1097837938 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f224c042030 con 0x7f2264101af0 2024-06-10T01:09:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.770+0000 7f2263fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56549703a820 0x56549703ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.770+0000 7f2263fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56549703a820 0x56549703ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:56454/0 (socket says 172.21.15.37:56454) 2024-06-10T01:09:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.770+0000 7f2263fff640 1 -- 172.21.15.37:0/2566630690 learned_addr learned my addr 172.21.15.37:0/2566630690 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.770+0000 7f22637fe640 1 --2- 172.21.15.37:0/2566630690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56549703b210 0x56549703fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.770+0000 7f225affd640 1 --2- 172.21.15.37:0/2566630690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565497040380 0x565497042810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.770+0000 7f2263fff640 1 -- 172.21.15.37:0/2566630690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56549703b210 msgr2=0x56549703fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.770+0000 7f2263fff640 1 --2- 172.21.15.37:0/2566630690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56549703b210 0x56549703fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.770+0000 7f2263fff640 1 -- 172.21.15.37:0/2566630690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565497040380 msgr2=0x565497042810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.770+0000 7f2263fff640 1 --2- 172.21.15.37:0/2566630690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565497040380 0x565497042810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.770+0000 7f2263fff640 1 -- 172.21.15.37:0/2566630690 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565496f4f9b0 con 0x56549703a820 2024-06-10T01:09:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.770+0000 7f22637fe640 1 --2- 172.21.15.37:0/2566630690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56549703b210 0x56549703fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:09:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.770+0000 7f2263fff640 1 --2- 172.21.15.37:0/2566630690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56549703a820 0x56549703ac40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f225c00ae90 tx=0x7f225c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b30714625f82f6c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.770+0000 7f2259ffb640 1 -- 172.21.15.37:0/2566630690 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c003b70 con 0x56549703a820 2024-06-10T01:09:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.771+0000 7f2259ffb640 1 -- 172.21.15.37:0/2566630690 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f225c003d10 con 0x56549703a820 2024-06-10T01:09:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.771+0000 7f2259ffb640 1 -- 172.21.15.37:0/2566630690 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c010c70 con 0x56549703a820 2024-06-10T01:09:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.771+0000 7f22617fa640 1 --2- 172.21.15.37:0/1097837938 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f22500a2cb0 0x7f22500a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.771+0000 7f22617fa640 1 -- 172.21.15.37:0/1097837938 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f224c0ef890 con 0x7f2264101af0 2024-06-10T01:09:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.771+0000 7f22637fe640 1 --2- 172.21.15.37:0/1097837938 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f22500a2cb0 0x7f22500a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.771+0000 7f226cb90740 1 -- 172.21.15.37:0/2566630690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56549703a820 msgr2=0x56549703ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.771+0000 7f226cb90740 1 --2- 172.21.15.37:0/2566630690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56549703a820 0x56549703ac40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f225c00ae90 tx=0x7f225c00f790 comp rx=0 tx=0).stop 2024-06-10T01:09:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.771+0000 7f226cb90740 1 -- 172.21.15.37:0/2566630690 shutdown_connections 2024-06-10T01:09:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.771+0000 7f226cb90740 1 --2- 172.21.15.37:0/2566630690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565497040380 0x565497042810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.771+0000 7f226cb90740 1 --2- 172.21.15.37:0/2566630690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56549703b210 0x56549703fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.771+0000 7f226cb90740 1 --2- 172.21.15.37:0/2566630690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56549703a820 0x56549703ac40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.771+0000 7f226cb90740 1 -- 172.21.15.37:0/2566630690 >> 172.21.15.37:0/2566630690 conn(0x56549702d680 msgr2=0x56549702da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.771+0000 7f22637fe640 1 --2- 172.21.15.37:0/1097837938 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f22500a2cb0 0x7f22500a5170 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x56549703b610 tx=0x7f2254005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.771+0000 7f226cb90740 1 -- 172.21.15.37:0/2566630690 shutdown_connections 2024-06-10T01:09:09.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.771+0000 7f226cb90740 1 -- 172.21.15.37:0/2566630690 wait complete. 2024-06-10T01:09:09.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.772+0000 7f226cb90740 1 -- start start 2024-06-10T01:09:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.772+0000 7f226cb90740 1 Processor -- start 2024-06-10T01:09:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.772+0000 7f226cb90740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56549703a820 0x565497044af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.772+0000 7f226cb90740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56549703b210 0x565497045030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.772+0000 7f226cb90740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565497040380 0x56549704c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.772+0000 7f226cb90740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x565497043330 con 0x565497040380 2024-06-10T01:09:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.772+0000 7f226cb90740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x565496f1eae0 con 0x56549703a820 2024-06-10T01:09:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.772+0000 7f226cb90740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5654970434b0 con 0x56549703b210 2024-06-10T01:09:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.772+0000 7f226cb90740 10 client.0 _ll_register_callbacks cb 0x565496f579b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.772+0000 7f22637fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56549703a820 0x565497044af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.772+0000 7f22637fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56549703a820 0x565497044af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:37940/0 (socket says 172.21.15.37:37940) 2024-06-10T01:09:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.772+0000 7f22637fe640 1 -- 172.21.15.37:0/4213147758 learned_addr learned my addr 172.21.15.37:0/4213147758 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.772+0000 7f225affd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56549703b210 0x565497045030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.772+0000 7f2263fff640 1 --2- 172.21.15.37:0/4213147758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565497040380 0x56549704c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.773+0000 7f22637fe640 1 -- 172.21.15.37:0/4213147758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56549703b210 msgr2=0x565497045030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.773+0000 7f22637fe640 1 --2- 172.21.15.37:0/4213147758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56549703b210 0x565497045030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.773+0000 7f22637fe640 1 -- 172.21.15.37:0/4213147758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565497040380 msgr2=0x56549704c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.773+0000 7f22637fe640 1 --2- 172.21.15.37:0/4213147758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565497040380 0x56549704c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.773+0000 7f22637fe640 1 -- 172.21.15.37:0/4213147758 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565496ca9210 con 0x56549703a820 2024-06-10T01:09:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.773+0000 7f2263fff640 1 --2- 172.21.15.37:0/4213147758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565497040380 0x56549704c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.773+0000 7f22637fe640 1 --2- 172.21.15.37:0/4213147758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56549703a820 0x565497044af0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f225401e9b0 tx=0x7f2254004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87208523b389fce9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.773+0000 7f2242ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:09:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.773+0000 7f2242ffd640 1 -- 172.21.15.37:0/4213147758 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2254009db0 con 0x56549703a820 2024-06-10T01:09:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.773+0000 7f226cb90740 1 -- 172.21.15.37:0/4213147758 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f225c003d10 con 0x56549703a820 2024-06-10T01:09:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.773+0000 7f2242ffd640 1 -- 172.21.15.37:0/4213147758 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f2254018030 con 0x56549703a820 2024-06-10T01:09:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.773+0000 7f2242ffd640 1 -- 172.21.15.37:0/4213147758 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2254034050 con 0x56549703a820 2024-06-10T01:09:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.773+0000 7f226cb90740 10 client.26915 fetch_fsmap learned FSMap version 31 2024-06-10T01:09:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.773+0000 7f226cb90740 1 -- 172.21.15.37:0/4213147758 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x565496d75620 con 0x56549703a820 2024-06-10T01:09:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.774+0000 7f2242ffd640 1 -- 172.21.15.37:0/4213147758 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f225c003d10 con 0x56549703a820 2024-06-10T01:09:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.774+0000 7f2242ffd640 1 -- 172.21.15.37:0/4213147758 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f225400f620 con 0x56549703a820 2024-06-10T01:09:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.774+0000 7f226cb90740 10 client.26915 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:09:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.774+0000 7f226cb90740 10 client.26915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.774+0000 7f226cb90740 20 client.26915 populate_metadata read hostname 'smithi037' 2024-06-10T01:09:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.774+0000 7f226cb90740 1 --2- 172.21.15.37:0/4213147758 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x565497050960 0x565497052d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.774+0000 7f226cb90740 4 client.26915 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:09:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.774+0000 7f226cb90740 1 -- 172.21.15.37:0/4213147758 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565496fa8f10 con 0x565497050960 2024-06-10T01:09:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.774+0000 7f225affd640 1 --2- 172.21.15.37:0/4213147758 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x565497050960 0x565497052d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.775+0000 7f2242ffd640 10 client.26915 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.775+0000 7f225affd640 1 --2- 172.21.15.37:0/4213147758 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x565497050960 0x565497052d70 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9576c08af921a7 server_cookie=f9ea64392f0e379c in_seq=0 out_seq=0 2024-06-10T01:09:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.779+0000 7f2242ffd640 1 -- 172.21.15.37:0/4213147758 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x565496fa8f10 con 0x565497050960 2024-06-10T01:09:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.779+0000 7f2242ffd640 10 client.26915 handle_command_reply: tid=0 2024-06-10T01:09:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.779+0000 7f226cb90740 1 client.26915 shutdown 2024-06-10T01:09:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.780+0000 7f226cb90740 1 -- 172.21.15.37:0/4213147758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56549703a820 msgr2=0x565497044af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.780+0000 7f226cb90740 1 --2- 172.21.15.37:0/4213147758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56549703a820 0x565497044af0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f225401e9b0 tx=0x7f2254004730 comp rx=0 tx=0).stop 2024-06-10T01:09:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.780+0000 7f226cb90740 1 -- 172.21.15.37:0/4213147758 shutdown_connections 2024-06-10T01:09:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.780+0000 7f226cb90740 1 --2- 172.21.15.37:0/4213147758 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x565497050960 0x565497052d70 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.780+0000 7f226cb90740 1 --2- 172.21.15.37:0/4213147758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565497040380 0x56549704c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.780+0000 7f226cb90740 1 --2- 172.21.15.37:0/4213147758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56549703b210 0x565497045030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.780+0000 7f226cb90740 1 --2- 172.21.15.37:0/4213147758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56549703a820 0x565497044af0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.780+0000 7f226cb90740 1 -- 172.21.15.37:0/4213147758 >> 172.21.15.37:0/4213147758 conn(0x56549702d680 msgr2=0x56549702c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.780+0000 7f2242ffd640 0 client.26915 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.780+0000 7f226cb90740 1 -- 172.21.15.37:0/4213147758 shutdown_connections 2024-06-10T01:09:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.780+0000 7f226cb90740 1 -- 172.21.15.37:0/4213147758 wait complete. 2024-06-10T01:09:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.780+0000 7f226cb90740 20 client.26915 trim_cache size 0 max 16384 2024-06-10T01:09:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.795+0000 7f226cb90740 1 Processor -- start 2024-06-10T01:09:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.795+0000 7f226cb90740 1 -- start start 2024-06-10T01:09:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.795+0000 7f226cb90740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56549703a820 0x565497044e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.795+0000 7f226cb90740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56549703b210 0x56549704c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.795+0000 7f226cb90740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565497040380 0x56549702bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.795+0000 7f226cb90740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f22641aade0 con 0x565497040380 2024-06-10T01:09:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.795+0000 7f226cb90740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f22641a86b0 con 0x56549703a820 2024-06-10T01:09:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.795+0000 7f226cb90740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f22641a6500 con 0x56549703b210 2024-06-10T01:09:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.796+0000 7f2263fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565497040380 0x56549702bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.796+0000 7f2263fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565497040380 0x56549702bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44960/0 (socket says 172.21.15.37:44960) 2024-06-10T01:09:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.796+0000 7f2263fff640 1 -- 172.21.15.37:0/2253111341 learned_addr learned my addr 172.21.15.37:0/2253111341 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.796+0000 7f225affd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56549703b210 0x56549704c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.796+0000 7f22637fe640 1 --2- 172.21.15.37:0/2253111341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56549703a820 0x565497044e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.796+0000 7f2263fff640 1 -- 172.21.15.37:0/2253111341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56549703b210 msgr2=0x56549704c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.796+0000 7f2263fff640 1 --2- 172.21.15.37:0/2253111341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56549703b210 0x56549704c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.796+0000 7f2263fff640 1 -- 172.21.15.37:0/2253111341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56549703a820 msgr2=0x565497044e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.796+0000 7f2263fff640 1 --2- 172.21.15.37:0/2253111341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56549703a820 0x565497044e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.796+0000 7f2263fff640 1 -- 172.21.15.37:0/2253111341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565497040050 con 0x565497040380 2024-06-10T01:09:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.796+0000 7f2263fff640 1 --2- 172.21.15.37:0/2253111341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565497040380 0x56549702bf50 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f225c00ae90 tx=0x7f225c0106f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec31e7d5d02c2a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.796+0000 7f22597fa640 1 -- 172.21.15.37:0/2253111341 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c01d040 con 0x565497040380 2024-06-10T01:09:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.796+0000 7f22597fa640 1 -- 172.21.15.37:0/2253111341 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f225c028070 con 0x565497040380 2024-06-10T01:09:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.796+0000 7f22597fa640 1 -- 172.21.15.37:0/2253111341 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c0052b0 con 0x565497040380 2024-06-10T01:09:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.797+0000 7f226cb90740 1 -- 172.21.15.37:0/2253111341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565497040380 msgr2=0x56549702bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.797+0000 7f226cb90740 1 --2- 172.21.15.37:0/2253111341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565497040380 0x56549702bf50 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f225c00ae90 tx=0x7f225c0106f0 comp rx=0 tx=0).stop 2024-06-10T01:09:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.797+0000 7f226cb90740 1 -- 172.21.15.37:0/2253111341 shutdown_connections 2024-06-10T01:09:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.797+0000 7f226cb90740 1 --2- 172.21.15.37:0/2253111341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x565497040380 0x56549702bf50 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.797+0000 7f226cb90740 1 --2- 172.21.15.37:0/2253111341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56549703b210 0x56549704c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.797+0000 7f226cb90740 1 --2- 172.21.15.37:0/2253111341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56549703a820 0x565497044e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.797+0000 7f226cb90740 1 -- 172.21.15.37:0/2253111341 >> 172.21.15.37:0/2253111341 conn(0x56549702d680 msgr2=0x565497044700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.797+0000 7f226cb90740 1 -- 172.21.15.37:0/2253111341 shutdown_connections 2024-06-10T01:09:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.797+0000 7f226cb90740 1 -- 172.21.15.37:0/2253111341 wait complete. 2024-06-10T01:09:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.797+0000 7f226cb90740 1 -- start start 2024-06-10T01:09:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.798+0000 7f226cb90740 1 Processor -- start 2024-06-10T01:09:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.798+0000 7f226cb90740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56549703a820 0x56549704c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.798+0000 7f226cb90740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56549703b210 0x565497048b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.798+0000 7f226cb90740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565497040380 0x5654970490b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.798+0000 7f226cb90740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f22641a6b70 con 0x56549703b210 2024-06-10T01:09:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.798+0000 7f226cb90740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f22641a7fe0 con 0x565497040380 2024-06-10T01:09:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.798+0000 7f226cb90740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f22641a7240 con 0x56549703a820 2024-06-10T01:09:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.798+0000 7f226cb90740 10 client.0 _ll_register_callbacks cb 0x7f2254009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.798+0000 7f225affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56549703b210 0x565497048b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.798+0000 7f22637fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56549703a820 0x56549704c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.798+0000 7f225affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56549703b210 0x565497048b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44968/0 (socket says 172.21.15.37:44968) 2024-06-10T01:09:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.798+0000 7f2263fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565497040380 0x5654970490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.798+0000 7f225affd640 1 -- 172.21.15.37:0/1005776387 learned_addr learned my addr 172.21.15.37:0/1005776387 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.798+0000 7f22637fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56549703a820 0x56549704c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:56498/0 (socket says 172.21.15.37:56498) 2024-06-10T01:09:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.798+0000 7f225affd640 1 -- 172.21.15.37:0/1005776387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56549703a820 msgr2=0x56549704c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.798+0000 7f225affd640 1 --2- 172.21.15.37:0/1005776387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56549703a820 0x56549704c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.798+0000 7f225affd640 1 -- 172.21.15.37:0/1005776387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565497040380 msgr2=0x5654970490b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.798+0000 7f225affd640 1 --2- 172.21.15.37:0/1005776387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565497040380 0x5654970490b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.798+0000 7f225affd640 1 -- 172.21.15.37:0/1005776387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654970b9670 con 0x56549703b210 2024-06-10T01:09:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.798+0000 7f225affd640 1 --2- 172.21.15.37:0/1005776387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56549703b210 0x565497048b10 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f224c0b8780 tx=0x7f224c0ece70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e63058ed4940c7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.798+0000 7f2242ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:09:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.798+0000 7f2242ffd640 1 -- 172.21.15.37:0/1005776387 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c049040 con 0x56549703b210 2024-06-10T01:09:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.799+0000 7f2242ffd640 1 -- 172.21.15.37:0/1005776387 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f224c03b460 con 0x56549703b210 2024-06-10T01:09:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.799+0000 7f226cb90740 1 -- 172.21.15.37:0/1005776387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f225c028070 con 0x56549703b210 2024-06-10T01:09:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.799+0000 7f2242ffd640 1 -- 172.21.15.37:0/1005776387 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f224c0c00b0 con 0x56549703b210 2024-06-10T01:09:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.799+0000 7f226cb90740 10 client.17601 fetch_fsmap learned FSMap version 31 2024-06-10T01:09:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.799+0000 7f226cb90740 1 -- 172.21.15.37:0/1005776387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5654970baa30 con 0x56549703b210 2024-06-10T01:09:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.799+0000 7f2242ffd640 1 -- 172.21.15.37:0/1005776387 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f225c028070 con 0x56549703b210 2024-06-10T01:09:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.799+0000 7f2242ffd640 1 -- 172.21.15.37:0/1005776387 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f224c03bc70 con 0x56549703b210 2024-06-10T01:09:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.799+0000 7f226cb90740 10 client.17601 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:09:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.799+0000 7f226cb90740 10 client.17601 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.799+0000 7f226cb90740 20 client.17601 populate_metadata read hostname 'smithi037' 2024-06-10T01:09:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.799+0000 7f226cb90740 1 --2- 172.21.15.37:0/1005776387 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5654970bb3b0 0x5654970bd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.799+0000 7f226cb90740 4 client.17601 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:09:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.799+0000 7f226cb90740 1 -- 172.21.15.37:0/1005776387 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565496fa2c20 con 0x5654970bb3b0 2024-06-10T01:09:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.799+0000 7f22637fe640 1 --2- 172.21.15.37:0/1005776387 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5654970bb3b0 0x5654970bd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.800+0000 7f2242ffd640 10 client.17601 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:09.800+0000 7f22637fe640 1 --2- 172.21.15.37:0/1005776387 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5654970bb3b0 0x5654970bd7d0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=668a84cc06007865 server_cookie=67e4884279cb1d89 in_seq=0 out_seq=0 2024-06-10T01:09:10.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:10 smithi167 ceph-mon[36917]: pgmap v805: 129 pgs: 129 active+clean; 17 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 1.01k op/s 2024-06-10T01:09:10.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:10 smithi094 ceph-mon[36708]: pgmap v805: 129 pgs: 129 active+clean; 17 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 1.01k op/s 2024-06-10T01:09:10.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:10 smithi037 ceph-mon[32192]: pgmap v805: 129 pgs: 129 active+clean; 17 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 1.01k op/s 2024-06-10T01:09:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:11.799+0000 7f2242ffd640 1 -- 172.21.15.37:0/1005776387 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x565496fa2c20 con 0x5654970bb3b0 2024-06-10T01:09:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:11.799+0000 7f2242ffd640 10 client.17601 handle_command_reply: tid=0 2024-06-10T01:09:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:11.799+0000 7f226cb90740 1 client.17601 shutdown 2024-06-10T01:09:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:11.799+0000 7f226cb90740 1 -- 172.21.15.37:0/1005776387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56549703b210 msgr2=0x565497048b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:11.799+0000 7f226cb90740 1 --2- 172.21.15.37:0/1005776387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56549703b210 0x565497048b10 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f224c0b8780 tx=0x7f224c0ece70 comp rx=0 tx=0).stop 2024-06-10T01:09:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:11.799+0000 7f226cb90740 1 -- 172.21.15.37:0/1005776387 shutdown_connections 2024-06-10T01:09:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:11.799+0000 7f226cb90740 1 --2- 172.21.15.37:0/1005776387 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5654970bb3b0 0x5654970bd7d0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:11.799+0000 7f226cb90740 1 --2- 172.21.15.37:0/1005776387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x565497040380 0x5654970490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:11.799+0000 7f226cb90740 1 --2- 172.21.15.37:0/1005776387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56549703b210 0x565497048b10 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:11.799+0000 7f2242ffd640 0 client.17601 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:11.799+0000 7f226cb90740 1 --2- 172.21.15.37:0/1005776387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56549703a820 0x56549704c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:11.799+0000 7f226cb90740 1 -- 172.21.15.37:0/1005776387 >> 172.21.15.37:0/1005776387 conn(0x56549702d680 msgr2=0x565497030360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:11.799+0000 7f226cb90740 1 -- 172.21.15.37:0/1005776387 shutdown_connections 2024-06-10T01:09:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:11.799+0000 7f226cb90740 1 -- 172.21.15.37:0/1005776387 wait complete. 2024-06-10T01:09:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:11.799+0000 7f226cb90740 20 client.17601 trim_cache size 0 max 16384 2024-06-10T01:09:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:09:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:09:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:09:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:11.800+0000 7f2269da2640 1 -- 172.21.15.37:0/1097837938 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f22500a2cb0 msgr2=0x7f22500a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:11.800+0000 7f2269da2640 1 --2- 172.21.15.37:0/1097837938 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f22500a2cb0 0x7f22500a5170 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x56549703b610 tx=0x7f2254005f90 comp rx=0 tx=0).stop 2024-06-10T01:09:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:11.800+0000 7f2269da2640 1 -- 172.21.15.37:0/1097837938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2264101af0 msgr2=0x7f22641981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:11.800+0000 7f2269da2640 1 --2- 172.21.15.37:0/1097837938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2264101af0 0x7f22641981d0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f224c009230 tx=0x7f224c031b60 comp rx=0 tx=0).stop 2024-06-10T01:09:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:11.800+0000 7f2269da2640 1 -- 172.21.15.37:0/1097837938 shutdown_connections 2024-06-10T01:09:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:11.800+0000 7f2269da2640 1 --2- 172.21.15.37:0/1097837938 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f22500a2cb0 0x7f22500a5170 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:11.800+0000 7f2269da2640 1 --2- 172.21.15.37:0/1097837938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22641024b0 0x7f226419f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:11.800+0000 7f2269da2640 1 --2- 172.21.15.37:0/1097837938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2264101af0 0x7f22641981d0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:11.800+0000 7f2269da2640 1 --2- 172.21.15.37:0/1097837938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2264100900 0x7f2264197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:11.800+0000 7f2269da2640 1 -- 172.21.15.37:0/1097837938 >> 172.21.15.37:0/1097837938 conn(0x7f22640fbe70 msgr2=0x7f2264104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:11.800+0000 7f2269da2640 1 -- 172.21.15.37:0/1097837938 shutdown_connections 2024-06-10T01:09:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:11.800+0000 7f2269da2640 1 -- 172.21.15.37:0/1097837938 wait complete. 2024-06-10T01:09:12.441 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:12 smithi037 ceph-mon[32192]: pgmap v806: 129 pgs: 129 active+clean; 17 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 863 KiB/s wr, 679 op/s 2024-06-10T01:09:12.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:12 smithi167 ceph-mon[36917]: pgmap v806: 129 pgs: 129 active+clean; 17 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 863 KiB/s wr, 679 op/s 2024-06-10T01:09:12.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:12 smithi094 ceph-mon[36708]: pgmap v806: 129 pgs: 129 active+clean; 17 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 863 KiB/s wr, 679 op/s 2024-06-10T01:09:14.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:14 smithi167 ceph-mon[36917]: pgmap v807: 129 pgs: 129 active+clean; 17 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 767 op/s 2024-06-10T01:09:14.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:14 smithi094 ceph-mon[36708]: pgmap v807: 129 pgs: 129 active+clean; 17 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 767 op/s 2024-06-10T01:09:14.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:14 smithi037 ceph-mon[32192]: pgmap v807: 129 pgs: 129 active+clean; 17 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 767 op/s 2024-06-10T01:09:16.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:16 smithi167 ceph-mon[36917]: pgmap v808: 129 pgs: 129 active+clean; 5.4 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 1.07k op/s 2024-06-10T01:09:16.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:16 smithi094 ceph-mon[36708]: pgmap v808: 129 pgs: 129 active+clean; 5.4 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 1.07k op/s 2024-06-10T01:09:16.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:16 smithi037 ceph-mon[32192]: pgmap v808: 129 pgs: 129 active+clean; 5.4 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 1.07k op/s 2024-06-10T01:09:18.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:18 smithi167 ceph-mon[36917]: scrub summary: idle+waiting paths [/] 2024-06-10T01:09:18.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:18 smithi167 ceph-mon[36917]: pgmap v809: 129 pgs: 129 active+clean; 9.7 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 936 op/s 2024-06-10T01:09:18.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:18 smithi094 ceph-mon[36708]: scrub summary: idle+waiting paths [/] 2024-06-10T01:09:18.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:18 smithi094 ceph-mon[36708]: pgmap v809: 129 pgs: 129 active+clean; 9.7 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 936 op/s 2024-06-10T01:09:18.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:18 smithi037 ceph-mon[32192]: scrub summary: idle+waiting paths [/] 2024-06-10T01:09:18.642 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:18 smithi037 ceph-mon[32192]: pgmap v809: 129 pgs: 129 active+clean; 9.7 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 936 op/s 2024-06-10T01:09:20.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:20 smithi167 ceph-mon[36917]: pgmap v810: 129 pgs: 129 active+clean; 9.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 1.31k op/s 2024-06-10T01:09:20.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:20 smithi094 ceph-mon[36708]: pgmap v810: 129 pgs: 129 active+clean; 9.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 1.31k op/s 2024-06-10T01:09:20.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:20 smithi037 ceph-mon[32192]: pgmap v810: 129 pgs: 129 active+clean; 9.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 1.31k op/s 2024-06-10T01:09:20.686 INFO:tasks.workunit.client.0.smithi037.stdout:cleaned up hierarchy 2024-06-10T01:09:21.165 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-10T01:09:21.165 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-10T01:09:21.204 INFO:tasks.workunit:Running workunit fs/misc/filelock_deadlock.py... 2024-06-10T01:09:21.205 DEBUG:teuthology.orchestra.run.smithi037:workunit test fs/misc/filelock_deadlock.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b7a5d375cf4e4afab96b9d4d229b699ae8304bb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/filelock_deadlock.py 2024-06-10T01:09:22.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:22 smithi167 ceph-mon[36917]: scrub summary: idle 2024-06-10T01:09:22.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:22 smithi167 ceph-mon[36917]: pgmap v811: 129 pgs: 129 active+clean; 9.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 806 KiB/s wr, 1.13k op/s 2024-06-10T01:09:22.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:22 smithi094 ceph-mon[36708]: scrub summary: idle 2024-06-10T01:09:22.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:22 smithi094 ceph-mon[36708]: pgmap v811: 129 pgs: 129 active+clean; 9.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 806 KiB/s wr, 1.13k op/s 2024-06-10T01:09:22.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:22 smithi037 ceph-mon[32192]: scrub summary: idle 2024-06-10T01:09:22.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:22 smithi037 ceph-mon[32192]: pgmap v811: 129 pgs: 129 active+clean; 9.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 806 KiB/s wr, 1.13k op/s 2024-06-10T01:09:24.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:24 smithi167 ceph-mon[36917]: pgmap v812: 129 pgs: 129 active+clean; 11 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 859 KiB/s wr, 1.13k op/s 2024-06-10T01:09:24.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:24 smithi094 ceph-mon[36708]: pgmap v812: 129 pgs: 129 active+clean; 11 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 859 KiB/s wr, 1.13k op/s 2024-06-10T01:09:24.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:24 smithi037 ceph-mon[32192]: pgmap v812: 129 pgs: 129 active+clean; 11 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 859 KiB/s wr, 1.13k op/s 2024-06-10T01:09:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.899+0000 7f685841c640 1 -- 172.21.15.37:0/2446230972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6850107790 msgr2=0x7f6850109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.899+0000 7f685841c640 1 --2- 172.21.15.37:0/2446230972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6850107790 0x7f6850109be0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f6848009ca0 tx=0x7f684802a8c0 comp rx=0 tx=0).stop 2024-06-10T01:09:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.900+0000 7f685841c640 1 -- 172.21.15.37:0/2446230972 shutdown_connections 2024-06-10T01:09:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.900+0000 7f685841c640 1 --2- 172.21.15.37:0/2446230972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6850107790 0x7f6850109be0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.900+0000 7f685841c640 1 --2- 172.21.15.37:0/2446230972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6850104e60 0x7f6850107250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.900+0000 7f685841c640 1 --2- 172.21.15.37:0/2446230972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6850102530 0x7f6850104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.900+0000 7f685841c640 1 -- 172.21.15.37:0/2446230972 >> 172.21.15.37:0/2446230972 conn(0x7f68500fbf20 msgr2=0x7f68500fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.900+0000 7f685841c640 1 -- 172.21.15.37:0/2446230972 shutdown_connections 2024-06-10T01:09:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.900+0000 7f685841c640 1 -- 172.21.15.37:0/2446230972 wait complete. 2024-06-10T01:09:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.900+0000 7f685841c640 1 Processor -- start 2024-06-10T01:09:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.900+0000 7f685841c640 1 -- start start 2024-06-10T01:09:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.900+0000 7f685841c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6850102530 0x7f6850101de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.900+0000 7f685841c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6850104e60 0x7f68501004e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.900+0000 7f685841c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6850107790 0x7f6850100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.900+0000 7f685841c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f685010d6e0 con 0x7f6850107790 2024-06-10T01:09:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.900+0000 7f685841c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f685010d560 con 0x7f6850102530 2024-06-10T01:09:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.900+0000 7f685841c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f685010d860 con 0x7f6850104e60 2024-06-10T01:09:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.901+0000 7f68569b9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6850107790 0x7f6850100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.901+0000 7f68569b9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6850107790 0x7f6850100a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36504/0 (socket says 172.21.15.37:36504) 2024-06-10T01:09:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.901+0000 7f68569b9640 1 -- 172.21.15.37:0/352371714 learned_addr learned my addr 172.21.15.37:0/352371714 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.901+0000 7f68561b8640 1 --2- 172.21.15.37:0/352371714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6850102530 0x7f6850101de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.901+0000 7f68559b7640 1 --2- 172.21.15.37:0/352371714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6850104e60 0x7f68501004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.901+0000 7f68569b9640 1 -- 172.21.15.37:0/352371714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6850104e60 msgr2=0x7f68501004e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.901+0000 7f68569b9640 1 --2- 172.21.15.37:0/352371714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6850104e60 0x7f68501004e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.901+0000 7f68569b9640 1 -- 172.21.15.37:0/352371714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6850102530 msgr2=0x7f6850101de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.901+0000 7f68569b9640 1 --2- 172.21.15.37:0/352371714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6850102530 0x7f6850101de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.901+0000 7f68569b9640 1 -- 172.21.15.37:0/352371714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68501011f0 con 0x7f6850107790 2024-06-10T01:09:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.901+0000 7f68569b9640 1 --2- 172.21.15.37:0/352371714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6850107790 0x7f6850100a20 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f6848005e30 tx=0x7f6848031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.901+0000 7f68477fe640 1 -- 172.21.15.37:0/352371714 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6848003c30 con 0x7f6850107790 2024-06-10T01:09:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.901+0000 7f68477fe640 1 -- 172.21.15.37:0/352371714 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6848003dd0 con 0x7f6850107790 2024-06-10T01:09:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.901+0000 7f685841c640 1 -- 172.21.15.37:0/352371714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68501a1d30 con 0x7f6850107790 2024-06-10T01:09:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.901+0000 7f68477fe640 1 -- 172.21.15.37:0/352371714 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684802fde0 con 0x7f6850107790 2024-06-10T01:09:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.901+0000 7f685841c640 1 -- 172.21.15.37:0/352371714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68501a2210 con 0x7f6850107790 2024-06-10T01:09:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.904+0000 7f68477fe640 1 -- 172.21.15.37:0/352371714 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6848004050 con 0x7f6850107790 2024-06-10T01:09:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.904+0000 7f685b20a740 1 Processor -- start 2024-06-10T01:09:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.905+0000 7f685b20a740 1 -- start start 2024-06-10T01:09:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.905+0000 7f685b20a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e65fad6010 0x55e65fad6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.905+0000 7f685b20a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e65fad6a00 0x55e65fadb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.905+0000 7f685b20a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e65fadbb70 0x55e65fade000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.905+0000 7f685b20a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e65fa394f0 con 0x55e65fadbb70 2024-06-10T01:09:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.905+0000 7f685b20a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e65f9d7870 con 0x55e65fad6a00 2024-06-10T01:09:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.905+0000 7f685b20a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e65fa354b0 con 0x55e65fad6010 2024-06-10T01:09:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.905+0000 7f68559b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e65fadbb70 0x55e65fade000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.905+0000 7f68559b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e65fadbb70 0x55e65fade000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36508/0 (socket says 172.21.15.37:36508) 2024-06-10T01:09:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.905+0000 7f68559b7640 1 -- 172.21.15.37:0/196871635 learned_addr learned my addr 172.21.15.37:0/196871635 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.905+0000 7f68559b7640 1 --2- 172.21.15.37:0/196871635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e65fad6010 0x55e65fad6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.905+0000 7f68561b8640 1 --2- 172.21.15.37:0/196871635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e65fad6a00 0x55e65fadb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.905+0000 7f68477fe640 1 --2- 172.21.15.37:0/352371714 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f683c0a2cb0 0x7f683c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.905+0000 7f68569b9640 1 --2- 172.21.15.37:0/352371714 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f683c0a2cb0 0x7f683c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.905+0000 7f68559b7640 1 -- 172.21.15.37:0/196871635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e65fad6010 msgr2=0x55e65fad6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.905+0000 7f68559b7640 1 --2- 172.21.15.37:0/196871635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e65fad6010 0x55e65fad6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.905+0000 7f68559b7640 1 -- 172.21.15.37:0/196871635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e65fad6a00 msgr2=0x55e65fadb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.905+0000 7f68559b7640 1 --2- 172.21.15.37:0/196871635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e65fad6a00 0x55e65fadb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.906+0000 7f68559b7640 1 -- 172.21.15.37:0/196871635 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e65f9f39b0 con 0x55e65fadbb70 2024-06-10T01:09:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.906+0000 7f68477fe640 1 -- 172.21.15.37:0/352371714 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f68480f0730 con 0x7f6850107790 2024-06-10T01:09:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.906+0000 7f68559b7640 1 --2- 172.21.15.37:0/196871635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e65fadbb70 0x55e65fade000 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f6840005d70 tx=0x7f684000e3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ed0fca679da2b3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.906+0000 7f6844bfe640 1 -- 172.21.15.37:0/196871635 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840004020 con 0x55e65fadbb70 2024-06-10T01:09:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.906+0000 7f6844bfe640 1 -- 172.21.15.37:0/196871635 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f684000bdb0 con 0x55e65fadbb70 2024-06-10T01:09:24.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.906+0000 7f6844bfe640 1 -- 172.21.15.37:0/196871635 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840005280 con 0x55e65fadbb70 2024-06-10T01:09:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.907+0000 7f685b20a740 1 -- 172.21.15.37:0/196871635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e65fadbb70 msgr2=0x55e65fade000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:24.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.907+0000 7f68569b9640 1 --2- 172.21.15.37:0/352371714 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f683c0a2cb0 0x7f683c0a5170 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f68480c52e0 tx=0x7f684808bb20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.907+0000 7f685b20a740 1 --2- 172.21.15.37:0/196871635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e65fadbb70 0x55e65fade000 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f6840005d70 tx=0x7f684000e3d0 comp rx=0 tx=0).stop 2024-06-10T01:09:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.907+0000 7f685b20a740 1 -- 172.21.15.37:0/196871635 shutdown_connections 2024-06-10T01:09:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.907+0000 7f685b20a740 1 --2- 172.21.15.37:0/196871635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e65fadbb70 0x55e65fade000 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.907+0000 7f685b20a740 1 --2- 172.21.15.37:0/196871635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e65fad6a00 0x55e65fadb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.907+0000 7f685b20a740 1 --2- 172.21.15.37:0/196871635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e65fad6010 0x55e65fad6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.907+0000 7f685b20a740 1 -- 172.21.15.37:0/196871635 >> 172.21.15.37:0/196871635 conn(0x55e65fad1680 msgr2=0x55e65fad1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.907+0000 7f685b20a740 1 -- 172.21.15.37:0/196871635 shutdown_connections 2024-06-10T01:09:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.907+0000 7f685b20a740 1 -- 172.21.15.37:0/196871635 wait complete. 2024-06-10T01:09:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.907+0000 7f685b20a740 1 -- start start 2024-06-10T01:09:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.907+0000 7f685b20a740 1 Processor -- start 2024-06-10T01:09:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.908+0000 7f685b20a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e65fad6010 0x55e65fae0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:24.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.908+0000 7f685b20a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e65fad6a00 0x55e65fae07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.908+0000 7f685b20a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e65fadbb70 0x55e65fae7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.908+0000 7f685b20a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e65fadeb20 con 0x55e65fad6a00 2024-06-10T01:09:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.908+0000 7f685b20a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e65f9c2ae0 con 0x55e65fadbb70 2024-06-10T01:09:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.908+0000 7f685b20a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e65fadeca0 con 0x55e65fad6010 2024-06-10T01:09:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.908+0000 7f685b20a740 10 client.0 _ll_register_callbacks cb 0x55e65f9fb9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.908+0000 7f68561b8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e65fad6a00 0x55e65fae07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.908+0000 7f68561b8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e65fad6a00 0x55e65fae07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36516/0 (socket says 172.21.15.37:36516) 2024-06-10T01:09:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.908+0000 7f68561b8640 1 -- 172.21.15.37:0/3045140760 learned_addr learned my addr 172.21.15.37:0/3045140760 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.908+0000 7f68559b7640 1 --2- 172.21.15.37:0/3045140760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e65fadbb70 0x55e65fae7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.908+0000 7f68559b7640 1 --2- 172.21.15.37:0/3045140760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e65fad6010 0x55e65fae0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.908+0000 7f68561b8640 1 -- 172.21.15.37:0/3045140760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e65fad6010 msgr2=0x55e65fae0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.908+0000 7f68561b8640 1 --2- 172.21.15.37:0/3045140760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e65fad6010 0x55e65fae0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:24.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.908+0000 7f68561b8640 1 -- 172.21.15.37:0/3045140760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e65fadbb70 msgr2=0x55e65fae7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.908+0000 7f68561b8640 1 --2- 172.21.15.37:0/3045140760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e65fadbb70 0x55e65fae7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.908+0000 7f68561b8640 1 -- 172.21.15.37:0/3045140760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e65f74d210 con 0x55e65fad6a00 2024-06-10T01:09:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.908+0000 7f68559b7640 1 --2- 172.21.15.37:0/3045140760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e65fad6010 0x55e65fae0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:09:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.908+0000 7f68561b8640 1 --2- 172.21.15.37:0/3045140760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e65fad6a00 0x55e65fae07c0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f684c00bfd0 tx=0x7f684c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fae9ac3d1ab315ca server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.908+0000 7f6839ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:09:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.909+0000 7f6839ffb640 1 -- 172.21.15.37:0/3045140760 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c004020 con 0x55e65fad6a00 2024-06-10T01:09:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.909+0000 7f6839ffb640 1 -- 172.21.15.37:0/3045140760 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f684c0041c0 con 0x55e65fad6a00 2024-06-10T01:09:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.909+0000 7f685b20a740 1 -- 172.21.15.37:0/3045140760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f684000bdb0 con 0x55e65fad6a00 2024-06-10T01:09:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.909+0000 7f6839ffb640 1 -- 172.21.15.37:0/3045140760 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f684c024050 con 0x55e65fad6a00 2024-06-10T01:09:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.909+0000 7f685b20a740 10 client.17613 fetch_fsmap learned FSMap version 31 2024-06-10T01:09:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.909+0000 7f685b20a740 1 -- 172.21.15.37:0/3045140760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e65f819620 con 0x55e65fad6a00 2024-06-10T01:09:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.909+0000 7f6839ffb640 1 -- 172.21.15.37:0/3045140760 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f684000bdb0 con 0x55e65fad6a00 2024-06-10T01:09:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.909+0000 7f6839ffb640 1 -- 172.21.15.37:0/3045140760 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f684c003a00 con 0x55e65fad6a00 2024-06-10T01:09:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.909+0000 7f685b20a740 10 client.17613 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:09:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.909+0000 7f685b20a740 10 client.17613 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.909+0000 7f685b20a740 20 client.17613 populate_metadata read hostname 'smithi037' 2024-06-10T01:09:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.909+0000 7f685b20a740 1 --2- 172.21.15.37:0/3045140760 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55e65faec1a0 0x55e65faee5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.909+0000 7f685b20a740 4 client.17613 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:09:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.909+0000 7f685b20a740 1 -- 172.21.15.37:0/3045140760 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e65fa4cf10 con 0x55e65faec1a0 2024-06-10T01:09:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.910+0000 7f68559b7640 1 --2- 172.21.15.37:0/3045140760 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55e65faec1a0 0x55e65faee5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.910+0000 7f68559b7640 1 --2- 172.21.15.37:0/3045140760 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55e65faec1a0 0x55e65faee5e0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=332a59faf64a2b56 server_cookie=b41959b2083cc961 in_seq=0 out_seq=0 2024-06-10T01:09:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.910+0000 7f6839ffb640 10 client.17613 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.911+0000 7f6839ffb640 1 -- 172.21.15.37:0/3045140760 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e65fa4cf10 con 0x55e65faec1a0 2024-06-10T01:09:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.911+0000 7f6839ffb640 10 client.17613 handle_command_reply: tid=0 2024-06-10T01:09:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.911+0000 7f685b20a740 1 client.17613 shutdown 2024-06-10T01:09:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.911+0000 7f685b20a740 1 -- 172.21.15.37:0/3045140760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e65fad6a00 msgr2=0x55e65fae07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.911+0000 7f685b20a740 1 --2- 172.21.15.37:0/3045140760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e65fad6a00 0x55e65fae07c0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f684c00bfd0 tx=0x7f684c0118d0 comp rx=0 tx=0).stop 2024-06-10T01:09:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.911+0000 7f685b20a740 1 -- 172.21.15.37:0/3045140760 shutdown_connections 2024-06-10T01:09:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.911+0000 7f685b20a740 1 --2- 172.21.15.37:0/3045140760 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55e65faec1a0 0x55e65faee5e0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.911+0000 7f685b20a740 1 --2- 172.21.15.37:0/3045140760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e65fadbb70 0x55e65fae7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.911+0000 7f685b20a740 1 --2- 172.21.15.37:0/3045140760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e65fad6a00 0x55e65fae07c0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.911+0000 7f6839ffb640 0 client.17613 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.911+0000 7f685b20a740 1 --2- 172.21.15.37:0/3045140760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e65fad6010 0x55e65fae0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.911+0000 7f685b20a740 1 -- 172.21.15.37:0/3045140760 >> 172.21.15.37:0/3045140760 conn(0x55e65fad1680 msgr2=0x55e65fad07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.911+0000 7f685b20a740 1 -- 172.21.15.37:0/3045140760 shutdown_connections 2024-06-10T01:09:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.911+0000 7f685b20a740 1 -- 172.21.15.37:0/3045140760 wait complete. 2024-06-10T01:09:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.911+0000 7f685b20a740 20 client.17613 trim_cache size 0 max 16384 2024-06-10T01:09:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.927+0000 7f685b20a740 1 Processor -- start 2024-06-10T01:09:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.927+0000 7f685b20a740 1 -- start start 2024-06-10T01:09:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.927+0000 7f685b20a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e65fad6010 0x55e65fae0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.927+0000 7f685b20a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e65fad6a00 0x55e65fae07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.927+0000 7f685b20a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e65fadbb70 0x55e65fae77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.927+0000 7f685b20a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f68501af190 con 0x55e65fad6010 2024-06-10T01:09:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.927+0000 7f685b20a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f68501aca60 con 0x55e65fadbb70 2024-06-10T01:09:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.927+0000 7f685b20a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f68501aa8b0 con 0x55e65fad6a00 2024-06-10T01:09:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.927+0000 7f68559b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e65fad6010 0x55e65fae0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.927+0000 7f68559b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e65fad6010 0x55e65fae0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36524/0 (socket says 172.21.15.37:36524) 2024-06-10T01:09:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.927+0000 7f68559b7640 1 -- 172.21.15.37:0/1669388951 learned_addr learned my addr 172.21.15.37:0/1669388951 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.928+0000 7f68561b8640 1 --2- 172.21.15.37:0/1669388951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e65fad6a00 0x55e65fae07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.928+0000 7f68559b7640 1 -- 172.21.15.37:0/1669388951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e65fad6a00 msgr2=0x55e65fae07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.928+0000 7f68559b7640 1 --2- 172.21.15.37:0/1669388951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e65fad6a00 0x55e65fae07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.928+0000 7f68559b7640 1 -- 172.21.15.37:0/1669388951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e65fadbb70 msgr2=0x55e65fae77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.928+0000 7f68559b7640 1 --2- 172.21.15.37:0/1669388951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e65fadbb70 0x55e65fae77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.928+0000 7f68559b7640 1 -- 172.21.15.37:0/1669388951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e65fad14b0 con 0x55e65fad6010 2024-06-10T01:09:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.928+0000 7f68561b8640 1 --2- 172.21.15.37:0/1669388951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e65fad6a00 0x55e65fae07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:09:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.928+0000 7f68559b7640 1 --2- 172.21.15.37:0/1669388951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e65fad6010 0x55e65fae0280 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f6840013fd0 tx=0x7f6840013550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e5ff28a83b35 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.928+0000 7f683bfff640 1 -- 172.21.15.37:0/1669388951 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840004220 con 0x55e65fad6010 2024-06-10T01:09:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.928+0000 7f683bfff640 1 -- 172.21.15.37:0/1669388951 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f684002a040 con 0x55e65fad6010 2024-06-10T01:09:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.928+0000 7f683bfff640 1 -- 172.21.15.37:0/1669388951 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840003ac0 con 0x55e65fad6010 2024-06-10T01:09:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.929+0000 7f685b20a740 1 -- 172.21.15.37:0/1669388951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e65fad6010 msgr2=0x55e65fae0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.929+0000 7f685b20a740 1 --2- 172.21.15.37:0/1669388951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e65fad6010 0x55e65fae0280 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f6840013fd0 tx=0x7f6840013550 comp rx=0 tx=0).stop 2024-06-10T01:09:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.929+0000 7f685b20a740 1 -- 172.21.15.37:0/1669388951 shutdown_connections 2024-06-10T01:09:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.929+0000 7f685b20a740 1 --2- 172.21.15.37:0/1669388951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e65fadbb70 0x55e65fae77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.929+0000 7f685b20a740 1 --2- 172.21.15.37:0/1669388951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e65fad6a00 0x55e65fae07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.929+0000 7f685b20a740 1 --2- 172.21.15.37:0/1669388951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e65fad6010 0x55e65fae0280 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.929+0000 7f685b20a740 1 -- 172.21.15.37:0/1669388951 >> 172.21.15.37:0/1669388951 conn(0x55e65fad1680 msgr2=0x55e65fadf070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.929+0000 7f685b20a740 1 -- 172.21.15.37:0/1669388951 shutdown_connections 2024-06-10T01:09:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.929+0000 7f685b20a740 1 -- 172.21.15.37:0/1669388951 wait complete. 2024-06-10T01:09:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.929+0000 7f685b20a740 1 -- start start 2024-06-10T01:09:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.929+0000 7f685b20a740 1 Processor -- start 2024-06-10T01:09:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.929+0000 7f685b20a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e65fad6010 0x55e65fae4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.929+0000 7f685b20a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e65fad6a00 0x55e65fae4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.929+0000 7f685b20a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e65fadbb70 0x55e65fb54eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.929+0000 7f685b20a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f68501aaf20 con 0x55e65fad6010 2024-06-10T01:09:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.929+0000 7f685b20a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f68501ac390 con 0x55e65fad6a00 2024-06-10T01:09:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.929+0000 7f685b20a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f68501ab5f0 con 0x55e65fadbb70 2024-06-10T01:09:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.929+0000 7f685b20a740 10 client.0 _ll_register_callbacks cb 0x7f684c00bc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.929+0000 7f68559b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e65fad6010 0x55e65fae4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.930+0000 7f68559b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e65fad6010 0x55e65fae4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36528/0 (socket says 172.21.15.37:36528) 2024-06-10T01:09:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.930+0000 7f68559b7640 1 -- 172.21.15.37:0/3555270006 learned_addr learned my addr 172.21.15.37:0/3555270006 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.930+0000 7f68561b8640 1 --2- 172.21.15.37:0/3555270006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e65fad6a00 0x55e65fae4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.930+0000 7f68559b7640 1 -- 172.21.15.37:0/3555270006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e65fadbb70 msgr2=0x55e65fb54eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.930+0000 7f68559b7640 1 --2- 172.21.15.37:0/3555270006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e65fadbb70 0x55e65fb54eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.930+0000 7f68559b7640 1 -- 172.21.15.37:0/3555270006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e65fad6a00 msgr2=0x55e65fae4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.930+0000 7f68559b7640 1 --2- 172.21.15.37:0/3555270006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e65fad6a00 0x55e65fae4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.930+0000 7f68559b7640 1 -- 172.21.15.37:0/3555270006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e65fad2350 con 0x55e65fad6010 2024-06-10T01:09:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.930+0000 7f68561b8640 1 --2- 172.21.15.37:0/3555270006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e65fad6a00 0x55e65fae4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:09:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.930+0000 7f68559b7640 1 --2- 172.21.15.37:0/3555270006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e65fad6010 0x55e65fae4420 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f6840016040 tx=0x7f6840005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ca703485f0a832 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.930+0000 7f6839ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:09:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.930+0000 7f6839ffb640 1 -- 172.21.15.37:0/3555270006 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840004230 con 0x55e65fad6010 2024-06-10T01:09:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.930+0000 7f6839ffb640 1 -- 172.21.15.37:0/3555270006 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f68400043d0 con 0x55e65fad6010 2024-06-10T01:09:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.930+0000 7f685b20a740 1 -- 172.21.15.37:0/3555270006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f684002a040 con 0x55e65fad6010 2024-06-10T01:09:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.930+0000 7f6839ffb640 1 -- 172.21.15.37:0/3555270006 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f684000b8d0 con 0x55e65fad6010 2024-06-10T01:09:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.930+0000 7f685b20a740 10 client.17619 fetch_fsmap learned FSMap version 31 2024-06-10T01:09:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.930+0000 7f685b20a740 1 -- 172.21.15.37:0/3555270006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e65fb56260 con 0x55e65fad6010 2024-06-10T01:09:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.930+0000 7f6839ffb640 1 -- 172.21.15.37:0/3555270006 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f684002a040 con 0x55e65fad6010 2024-06-10T01:09:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.931+0000 7f6839ffb640 1 -- 172.21.15.37:0/3555270006 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f684002c330 con 0x55e65fad6010 2024-06-10T01:09:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.931+0000 7f685b20a740 10 client.17619 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:09:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.931+0000 7f685b20a740 10 client.17619 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.931+0000 7f685b20a740 20 client.17619 populate_metadata read hostname 'smithi037' 2024-06-10T01:09:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.931+0000 7f685b20a740 1 --2- 172.21.15.37:0/3555270006 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55e65faec1a0 0x55e65fb58ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.931+0000 7f685b20a740 4 client.17619 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:09:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.931+0000 7f685b20a740 1 -- 172.21.15.37:0/3555270006 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e65fa46c20 con 0x55e65faec1a0 2024-06-10T01:09:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.931+0000 7f68561b8640 1 --2- 172.21.15.37:0/3555270006 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55e65faec1a0 0x55e65fb58ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.931+0000 7f6839ffb640 10 client.17619 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:24.931+0000 7f68561b8640 1 --2- 172.21.15.37:0/3555270006 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55e65faec1a0 0x55e65fb58ab0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f4df7b5b24c80d8 server_cookie=1af2c600857fdaf0 in_seq=0 out_seq=0 2024-06-10T01:09:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:25.039+0000 7f6839ffb640 1 -- 172.21.15.37:0/3555270006 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e65fa46c20 con 0x55e65faec1a0 2024-06-10T01:09:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:25.039+0000 7f6839ffb640 10 client.17619 handle_command_reply: tid=0 2024-06-10T01:09:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:25.039+0000 7f685b20a740 1 client.17619 shutdown 2024-06-10T01:09:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:25.039+0000 7f685b20a740 1 -- 172.21.15.37:0/3555270006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e65fad6010 msgr2=0x55e65fae4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:25.039+0000 7f685b20a740 1 --2- 172.21.15.37:0/3555270006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e65fad6010 0x55e65fae4420 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f6840016040 tx=0x7f6840005d00 comp rx=0 tx=0).stop 2024-06-10T01:09:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:25.039+0000 7f685b20a740 1 -- 172.21.15.37:0/3555270006 shutdown_connections 2024-06-10T01:09:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:25.039+0000 7f685b20a740 1 --2- 172.21.15.37:0/3555270006 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55e65faec1a0 0x55e65fb58ab0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:25.039+0000 7f6839ffb640 0 client.17619 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:25.039+0000 7f685b20a740 1 --2- 172.21.15.37:0/3555270006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e65fadbb70 0x55e65fb54eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:25.039+0000 7f685b20a740 1 --2- 172.21.15.37:0/3555270006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e65fad6a00 0x55e65fae4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:25.039+0000 7f685b20a740 1 --2- 172.21.15.37:0/3555270006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e65fad6010 0x55e65fae4420 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:25.039+0000 7f685b20a740 1 -- 172.21.15.37:0/3555270006 >> 172.21.15.37:0/3555270006 conn(0x55e65fad1680 msgr2=0x55e65fad7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:25.039+0000 7f685b20a740 1 -- 172.21.15.37:0/3555270006 shutdown_connections 2024-06-10T01:09:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:25.039+0000 7f685b20a740 1 -- 172.21.15.37:0/3555270006 wait complete. 2024-06-10T01:09:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:25.040+0000 7f685b20a740 20 client.17619 trim_cache size 0 max 16384 2024-06-10T01:09:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:09:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:09:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:09:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:25.040+0000 7f685841c640 1 -- 172.21.15.37:0/352371714 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f683c0a2cb0 msgr2=0x7f683c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:25.040+0000 7f685841c640 1 --2- 172.21.15.37:0/352371714 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f683c0a2cb0 0x7f683c0a5170 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f68480c52e0 tx=0x7f684808bb20 comp rx=0 tx=0).stop 2024-06-10T01:09:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:25.040+0000 7f685841c640 1 -- 172.21.15.37:0/352371714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6850107790 msgr2=0x7f6850100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:25.040+0000 7f685841c640 1 --2- 172.21.15.37:0/352371714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6850107790 0x7f6850100a20 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f6848005e30 tx=0x7f6848031ee0 comp rx=0 tx=0).stop 2024-06-10T01:09:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:25.040+0000 7f685841c640 1 -- 172.21.15.37:0/352371714 shutdown_connections 2024-06-10T01:09:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:25.040+0000 7f685841c640 1 --2- 172.21.15.37:0/352371714 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f683c0a2cb0 0x7f683c0a5170 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:25.040+0000 7f685841c640 1 --2- 172.21.15.37:0/352371714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6850107790 0x7f6850100a20 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:25.040+0000 7f685841c640 1 --2- 172.21.15.37:0/352371714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6850104e60 0x7f68501004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:25.040+0000 7f685841c640 1 --2- 172.21.15.37:0/352371714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6850102530 0x7f6850101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:25.040+0000 7f685841c640 1 -- 172.21.15.37:0/352371714 >> 172.21.15.37:0/352371714 conn(0x7f68500fbf20 msgr2=0x7f6850102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:25.041+0000 7f685841c640 1 -- 172.21.15.37:0/352371714 shutdown_connections 2024-06-10T01:09:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:25.041+0000 7f685841c640 1 -- 172.21.15.37:0/352371714 wait complete. 2024-06-10T01:09:26.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:26 smithi167 ceph-mon[36917]: pgmap v813: 129 pgs: 129 active+clean; 14 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 809 KiB/s wr, 1.09k op/s 2024-06-10T01:09:26.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:26 smithi094 ceph-mon[36708]: pgmap v813: 129 pgs: 129 active+clean; 14 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 809 KiB/s wr, 1.09k op/s 2024-06-10T01:09:26.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:26 smithi037 ceph-mon[32192]: pgmap v813: 129 pgs: 129 active+clean; 14 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 809 KiB/s wr, 1.09k op/s 2024-06-10T01:09:28.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:28 smithi167 ceph-mon[36917]: pgmap v814: 129 pgs: 129 active+clean; 10 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 768 KiB/s wr, 721 op/s 2024-06-10T01:09:28.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:28 smithi094 ceph-mon[36708]: pgmap v814: 129 pgs: 129 active+clean; 10 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 768 KiB/s wr, 721 op/s 2024-06-10T01:09:28.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:28 smithi037 ceph-mon[32192]: pgmap v814: 129 pgs: 129 active+clean; 10 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 768 KiB/s wr, 721 op/s 2024-06-10T01:09:30.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:30 smithi167 ceph-mon[36917]: pgmap v815: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 424 KiB/s wr, 682 op/s 2024-06-10T01:09:30.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:30 smithi094 ceph-mon[36708]: pgmap v815: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 424 KiB/s wr, 682 op/s 2024-06-10T01:09:30.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:30 smithi037 ceph-mon[32192]: pgmap v815: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 424 KiB/s wr, 682 op/s 2024-06-10T01:09:31.381 INFO:tasks.workunit.client.0.smithi037.stdout:ok 2024-06-10T01:09:31.386 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-10T01:09:31.387 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-10T01:09:31.442 INFO:tasks.workunit:Running workunit fs/misc/filelock_interrupt.py... 2024-06-10T01:09:31.442 DEBUG:teuthology.orchestra.run.smithi037:workunit test fs/misc/filelock_interrupt.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b7a5d375cf4e4afab96b9d4d229b699ae8304bb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/filelock_interrupt.py 2024-06-10T01:09:31.604 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 15.8 2024-06-10T01:09:31.605 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:09:31.866 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:09:32.255 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:32 smithi037 ceph-mon[32192]: pgmap v816: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 405 KiB/s wr, 285 op/s 2024-06-10T01:09:32.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:32 smithi167 ceph-mon[36917]: pgmap v816: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 405 KiB/s wr, 285 op/s 2024-06-10T01:09:32.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:32 smithi094 ceph-mon[36708]: pgmap v816: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 405 KiB/s wr, 285 op/s 2024-06-10T01:09:33.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.827+0000 7f8b31eb5700 1 -- 172.21.15.37:0/346554649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b2c106be0 msgr2=0x7f8b2c106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:33.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.827+0000 7f8b31eb5700 1 --2- 172.21.15.37:0/346554649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b2c106be0 0x7f8b2c106fc0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0099c0 tx=0x7f8b1c009cd0 comp rx=0 tx=0).stop 2024-06-10T01:09:33.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.827+0000 7f8b31eb5700 1 -- 172.21.15.37:0/346554649 shutdown_connections 2024-06-10T01:09:33.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.827+0000 7f8b31eb5700 1 --2- 172.21.15.37:0/346554649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b2c1014f0 0x7f8b2c10d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:33.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.827+0000 7f8b31eb5700 1 --2- 172.21.15.37:0/346554649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b2c106be0 0x7f8b2c106fc0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:33.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.827+0000 7f8b31eb5700 1 --2- 172.21.15.37:0/346554649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b2c100b30 0x7f8b2c100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:33.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.827+0000 7f8b31eb5700 1 -- 172.21.15.37:0/346554649 >> 172.21.15.37:0/346554649 conn(0x7f8b2c077990 msgr2=0x7f8b2c0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:33.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.827+0000 7f8b31eb5700 1 -- 172.21.15.37:0/346554649 shutdown_connections 2024-06-10T01:09:33.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.827+0000 7f8b31eb5700 1 -- 172.21.15.37:0/346554649 wait complete. 2024-06-10T01:09:33.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.828+0000 7f8b31eb5700 1 Processor -- start 2024-06-10T01:09:33.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.828+0000 7f8b31eb5700 1 -- start start 2024-06-10T01:09:33.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.828+0000 7f8b31eb5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b2c100b30 0x7f8b2c199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:33.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.828+0000 7f8b31eb5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b2c1014f0 0x7f8b2c199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:33.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.828+0000 7f8b31eb5700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b2c106be0 0x7f8b2c1a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:33.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.828+0000 7f8b31eb5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c111700 con 0x7f8b2c100b30 2024-06-10T01:09:33.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.828+0000 7f8b31eb5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c111580 con 0x7f8b2c1014f0 2024-06-10T01:09:33.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.828+0000 7f8b31eb5700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c111880 con 0x7f8b2c106be0 2024-06-10T01:09:33.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.828+0000 7f8b2b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b2c100b30 0x7f8b2c199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:33.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.828+0000 7f8b2b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b2c100b30 0x7f8b2c199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34412/0 (socket says 172.21.15.37:34412) 2024-06-10T01:09:33.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.828+0000 7f8b2b7fe700 1 -- 172.21.15.37:0/2932306721 learned_addr learned my addr 172.21.15.37:0/2932306721 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:33.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.828+0000 7f8b2affd700 1 --2- 172.21.15.37:0/2932306721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b2c1014f0 0x7f8b2c199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:33.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.829+0000 7f8b2bfff700 1 --2- 172.21.15.37:0/2932306721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b2c106be0 0x7f8b2c1a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:33.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.829+0000 7f8b2b7fe700 1 -- 172.21.15.37:0/2932306721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b2c106be0 msgr2=0x7f8b2c1a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:33.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.829+0000 7f8b2b7fe700 1 --2- 172.21.15.37:0/2932306721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b2c106be0 0x7f8b2c1a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:33.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.829+0000 7f8b2b7fe700 1 -- 172.21.15.37:0/2932306721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b2c1014f0 msgr2=0x7f8b2c199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:33.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.829+0000 7f8b2b7fe700 1 --2- 172.21.15.37:0/2932306721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b2c1014f0 0x7f8b2c199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:33.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.829+0000 7f8b2b7fe700 1 -- 172.21.15.37:0/2932306721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2c1a5040 con 0x7f8b2c100b30 2024-06-10T01:09:33.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.829+0000 7f8b2bfff700 1 --2- 172.21.15.37:0/2932306721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b2c106be0 0x7f8b2c1a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:09:33.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.829+0000 7f8b2b7fe700 1 --2- 172.21.15.37:0/2932306721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b2c100b30 0x7f8b2c199a80 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c009990 tx=0x7f8b1c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:33.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.829+0000 7f8b28ff9700 1 -- 172.21.15.37:0/2932306721 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c027070 con 0x7f8b2c100b30 2024-06-10T01:09:33.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.829+0000 7f8b31eb5700 1 -- 172.21.15.37:0/2932306721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b2c1a5210 con 0x7f8b2c100b30 2024-06-10T01:09:33.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.829+0000 7f8b28ff9700 1 -- 172.21.15.37:0/2932306721 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b1c004050 con 0x7f8b2c100b30 2024-06-10T01:09:33.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.829+0000 7f8b28ff9700 1 -- 172.21.15.37:0/2932306721 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c022e40 con 0x7f8b2c100b30 2024-06-10T01:09:33.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.829+0000 7f8b31eb5700 1 -- 172.21.15.37:0/2932306721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b2c1a56c0 con 0x7f8b2c100b30 2024-06-10T01:09:33.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.830+0000 7f8b31eb5700 1 -- 172.21.15.37:0/2932306721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b2c109070 con 0x7f8b2c100b30 2024-06-10T01:09:33.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.831+0000 7f8b28ff9700 1 -- 172.21.15.37:0/2932306721 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8b1c02a070 con 0x7f8b2c100b30 2024-06-10T01:09:33.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.832+0000 7f8b28ff9700 1 --2- 172.21.15.37:0/2932306721 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8b140a2c20 0x7f8b140a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:33.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.832+0000 7f8b2affd700 1 --2- 172.21.15.37:0/2932306721 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8b140a2c20 0x7f8b140a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:33.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.832+0000 7f8b28ff9700 1 -- 172.21.15.37:0/2932306721 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f8b1c0d0580 con 0x7f8b2c100b30 2024-06-10T01:09:33.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.833+0000 7f8b2affd700 1 --2- 172.21.15.37:0/2932306721 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8b140a2c20 0x7f8b140a50e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c1058b0 tx=0x7f8b2000a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:33.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:33.836+0000 7f8b28ff9700 1 -- 172.21.15.37:0/2932306721 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8b1c013070 con 0x7f8b2c100b30 2024-06-10T01:09:34.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:34.088+0000 7f8b31eb5700 1 -- 172.21.15.37:0/2932306721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b2c10e960 con 0x7f8b2c100b30 2024-06-10T01:09:34.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:34.088+0000 7f8b28ff9700 1 -- 172.21.15.37:0/2932306721 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11868 (secure 0 0 0) 0x7f8b1c09d120 con 0x7f8b2c100b30 2024-06-10T01:09:34.084 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:09:34.084 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":26554,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2107277223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2107277223},{"type":"v1","addr":"172.21.15.167:6835","nonce":2107277223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T01:06:09.604504+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24472,24451,24458]},"id":1}]} 2024-06-10T01:09:34.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:34.091+0000 7f8b31eb5700 1 -- 172.21.15.37:0/2932306721 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8b140a2c20 msgr2=0x7f8b140a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:34.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:34.091+0000 7f8b31eb5700 1 --2- 172.21.15.37:0/2932306721 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8b140a2c20 0x7f8b140a50e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c1058b0 tx=0x7f8b2000a380 comp rx=0 tx=0).stop 2024-06-10T01:09:34.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:34.091+0000 7f8b31eb5700 1 -- 172.21.15.37:0/2932306721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b2c100b30 msgr2=0x7f8b2c199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:34.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:34.091+0000 7f8b31eb5700 1 --2- 172.21.15.37:0/2932306721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b2c100b30 0x7f8b2c199a80 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c009990 tx=0x7f8b1c0047b0 comp rx=0 tx=0).stop 2024-06-10T01:09:34.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:34.091+0000 7f8b31eb5700 1 -- 172.21.15.37:0/2932306721 shutdown_connections 2024-06-10T01:09:34.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:34.091+0000 7f8b31eb5700 1 --2- 172.21.15.37:0/2932306721 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8b140a2c20 0x7f8b140a50e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:34.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:34.091+0000 7f8b31eb5700 1 --2- 172.21.15.37:0/2932306721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b2c106be0 0x7f8b2c1a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:34.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:34.091+0000 7f8b31eb5700 1 --2- 172.21.15.37:0/2932306721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b2c100b30 0x7f8b2c199a80 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:34.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:34.091+0000 7f8b31eb5700 1 --2- 172.21.15.37:0/2932306721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b2c1014f0 0x7f8b2c199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:34.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:34.091+0000 7f8b31eb5700 1 -- 172.21.15.37:0/2932306721 >> 172.21.15.37:0/2932306721 conn(0x7f8b2c077990 msgr2=0x7f8b2c0feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:34.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:34.091+0000 7f8b31eb5700 1 -- 172.21.15.37:0/2932306721 shutdown_connections 2024-06-10T01:09:34.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:34.092+0000 7f8b31eb5700 1 -- 172.21.15.37:0/2932306721 wait complete. 2024-06-10T01:09:34.088 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 31 2024-06-10T01:09:34.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:34 smithi037 ceph-mon[32192]: pgmap v817: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 408 KiB/s wr, 223 op/s 2024-06-10T01:09:34.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:34 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2932306721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:09:34.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:34 smithi167 ceph-mon[36917]: pgmap v817: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 408 KiB/s wr, 223 op/s 2024-06-10T01:09:34.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:34 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2932306721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:09:34.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:34 smithi094 ceph-mon[36708]: pgmap v817: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 408 KiB/s wr, 223 op/s 2024-06-10T01:09:34.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:34 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2932306721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:09:34.806 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24472 quiesce db / --timeout 90 --expiration 135.8 --await-for 45.0' 2024-06-10T01:09:34.807 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24472 quiesce db / --timeout 90 --expiration 135.8 --await-for 45.0 2024-06-10T01:09:35.064 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:09:36.437 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:36 smithi037 ceph-mon[32192]: pgmap v818: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 352 KiB/s wr, 153 op/s 2024-06-10T01:09:36.496 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:09:36.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:36 smithi167 ceph-mon[36917]: pgmap v818: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 352 KiB/s wr, 153 op/s 2024-06-10T01:09:36.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:36 smithi094 ceph-mon[36708]: pgmap v818: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 352 KiB/s wr, 153 op/s 2024-06-10T01:09:36.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.846+0000 7f8a59128700 1 -- 172.21.15.37:0/2737149484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a54101fa0 msgr2=0x7f8a54105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:36.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.846+0000 7f8a59128700 1 --2- 172.21.15.37:0/2737149484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a54101fa0 0x7f8a54105fd0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009990 tx=0x7f8a48009ca0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.846+0000 7f8a59128700 1 -- 172.21.15.37:0/2737149484 shutdown_connections 2024-06-10T01:09:36.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.846+0000 7f8a59128700 1 --2- 172.21.15.37:0/2737149484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a541066b0 0x7f8a5410e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.846+0000 7f8a59128700 1 --2- 172.21.15.37:0/2737149484 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a541015f0 0x7f8a541019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.846+0000 7f8a59128700 1 --2- 172.21.15.37:0/2737149484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a54101fa0 0x7f8a54105fd0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.846+0000 7f8a59128700 1 -- 172.21.15.37:0/2737149484 >> 172.21.15.37:0/2737149484 conn(0x7f8a54078300 msgr2=0x7f8a54078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:36.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.846+0000 7f8a59128700 1 -- 172.21.15.37:0/2737149484 shutdown_connections 2024-06-10T01:09:36.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.846+0000 7f8a59128700 1 -- 172.21.15.37:0/2737149484 wait complete. 2024-06-10T01:09:36.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.847+0000 7f8a59128700 1 Processor -- start 2024-06-10T01:09:36.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.847+0000 7f8a59128700 1 -- start start 2024-06-10T01:09:36.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.848+0000 7f8a59128700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a541015f0 0x7f8a5419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:36.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.848+0000 7f8a59128700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a54101fa0 0x7f8a5419dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:36.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.848+0000 7f8a59128700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a541066b0 0x7f8a54197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:36.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.848+0000 7f8a59128700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8a54111670 con 0x7f8a541015f0 2024-06-10T01:09:36.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.848+0000 7f8a59128700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541114f0 con 0x7f8a541066b0 2024-06-10T01:09:36.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.848+0000 7f8a59128700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541117f0 con 0x7f8a54101fa0 2024-06-10T01:09:36.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.848+0000 7f8a52d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a541015f0 0x7f8a5419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:36.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.848+0000 7f8a52d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a541015f0 0x7f8a5419d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34432/0 (socket says 172.21.15.37:34432) 2024-06-10T01:09:36.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.848+0000 7f8a52d9d700 1 -- 172.21.15.37:0/2523968542 learned_addr learned my addr 172.21.15.37:0/2523968542 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:36.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.848+0000 7f8a5259c700 1 --2- 172.21.15.37:0/2523968542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a54101fa0 0x7f8a5419dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:36.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.848+0000 7f8a5359e700 1 --2- 172.21.15.37:0/2523968542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a541066b0 0x7f8a54197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:36.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.848+0000 7f8a52d9d700 1 -- 172.21.15.37:0/2523968542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a54101fa0 msgr2=0x7f8a5419dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:36.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.848+0000 7f8a52d9d700 1 --2- 172.21.15.37:0/2523968542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a54101fa0 0x7f8a5419dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.848+0000 7f8a52d9d700 1 -- 172.21.15.37:0/2523968542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a541066b0 msgr2=0x7f8a54197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:36.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.848+0000 7f8a52d9d700 1 --2- 172.21.15.37:0/2523968542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a541066b0 0x7f8a54197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.848+0000 7f8a52d9d700 1 -- 172.21.15.37:0/2523968542 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a54197fd0 con 0x7f8a541015f0 2024-06-10T01:09:36.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.848+0000 7f8a5259c700 1 --2- 172.21.15.37:0/2523968542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a54101fa0 0x7f8a5419dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:36.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.848+0000 7f8a52d9d700 1 --2- 172.21.15.37:0/2523968542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a541015f0 0x7f8a5419d6b0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400d980 tx=0x7f8a4400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:36.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.849+0000 7f8a3bfff700 1 -- 172.21.15.37:0/2523968542 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a44017070 con 0x7f8a541015f0 2024-06-10T01:09:36.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.849+0000 7f8a59128700 1 -- 172.21.15.37:0/2523968542 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a541982c0 con 0x7f8a541015f0 2024-06-10T01:09:36.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.849+0000 7f8a3bfff700 1 -- 172.21.15.37:0/2523968542 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a44005020 con 0x7f8a541015f0 2024-06-10T01:09:36.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.849+0000 7f8a3bfff700 1 -- 172.21.15.37:0/2523968542 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4400fbf0 con 0x7f8a541015f0 2024-06-10T01:09:36.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.849+0000 7f8a59128700 1 -- 172.21.15.37:0/2523968542 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a541a4580 con 0x7f8a541015f0 2024-06-10T01:09:36.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.852+0000 7f8a3bfff700 1 -- 172.21.15.37:0/2523968542 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8a440051d0 con 0x7f8a541015f0 2024-06-10T01:09:36.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.852+0000 7f8a3bfff700 1 --2- 172.21.15.37:0/2523968542 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8a3c0a2cc0 0x7f8a3c0a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:36.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.852+0000 7f8a3bfff700 1 -- 172.21.15.37:0/2523968542 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f8a440cf260 con 0x7f8a541015f0 2024-06-10T01:09:36.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.852+0000 7f8a5259c700 1 --2- 172.21.15.37:0/2523968542 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8a3c0a2cc0 0x7f8a3c0a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:36.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.853+0000 7f8a639abb80 1 Processor -- start 2024-06-10T01:09:36.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.853+0000 7f8a5259c700 1 --2- 172.21.15.37:0/2523968542 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8a3c0a2cc0 0x7f8a3c0a5180 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8a48006010 tx=0x7f8a48005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:36.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.853+0000 7f8a639abb80 1 -- start start 2024-06-10T01:09:36.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.853+0000 7f8a639abb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56080082b780 0x56080082bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:36.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.853+0000 7f8a639abb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56080082c0a0 0x56080083ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:36.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.853+0000 7f8a639abb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56080083d360 0x56080083f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:36.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.853+0000 7f8a639abb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5608007210f0 con 0x56080082c0a0 2024-06-10T01:09:36.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.853+0000 7f8a639abb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56080077f9e0 con 0x56080083d360 2024-06-10T01:09:36.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.853+0000 7f8a639abb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5608006e75f0 con 0x56080082b780 2024-06-10T01:09:36.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.853+0000 7f8a5259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56080082c0a0 0x56080083ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:36.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.853+0000 7f8a5259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56080082c0a0 0x56080083ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34434/0 (socket says 172.21.15.37:34434) 2024-06-10T01:09:36.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.853+0000 7f8a5259c700 1 -- 172.21.15.37:0/486905668 learned_addr learned my addr 172.21.15.37:0/486905668 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:36.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.853+0000 7f8a5359e700 1 --2- 172.21.15.37:0/486905668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56080083d360 0x56080083f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:36.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.853+0000 7f8a52d9d700 1 --2- 172.21.15.37:0/486905668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56080082b780 0x56080082bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:36.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.854+0000 7f8a5259c700 1 -- 172.21.15.37:0/486905668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56080082b780 msgr2=0x56080082bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:36.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.854+0000 7f8a5259c700 1 --2- 172.21.15.37:0/486905668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56080082b780 0x56080082bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.854+0000 7f8a5259c700 1 -- 172.21.15.37:0/486905668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56080083d360 msgr2=0x56080083f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:36.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.854+0000 7f8a5259c700 1 --2- 172.21.15.37:0/486905668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56080083d360 0x56080083f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.854+0000 7f8a5259c700 1 -- 172.21.15.37:0/486905668 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608006cea60 con 0x56080082c0a0 2024-06-10T01:09:36.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.854+0000 7f8a5359e700 1 --2- 172.21.15.37:0/486905668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56080083d360 0x56080083f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:36.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.854+0000 7f8a5259c700 1 --2- 172.21.15.37:0/486905668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56080082c0a0 0x56080083ce20 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f8a48015000 tx=0x7f8a48020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f804472ce1ac6a8c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:36.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.854+0000 7f8a397fa700 1 -- 172.21.15.37:0/486905668 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4800be10 con 0x56080082c0a0 2024-06-10T01:09:36.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.854+0000 7f8a397fa700 1 -- 172.21.15.37:0/486905668 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a480055a0 con 0x56080082c0a0 2024-06-10T01:09:36.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.854+0000 7f8a397fa700 1 -- 172.21.15.37:0/486905668 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4800fe30 con 0x56080082c0a0 2024-06-10T01:09:36.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.855+0000 7f8a639abb80 1 -- 172.21.15.37:0/486905668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56080082c0a0 msgr2=0x56080083ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:36.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.855+0000 7f8a639abb80 1 --2- 172.21.15.37:0/486905668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56080082c0a0 0x56080083ce20 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f8a48015000 tx=0x7f8a48020000 comp rx=0 tx=0).stop 2024-06-10T01:09:36.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.855+0000 7f8a639abb80 1 -- 172.21.15.37:0/486905668 shutdown_connections 2024-06-10T01:09:36.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.855+0000 7f8a639abb80 1 --2- 172.21.15.37:0/486905668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56080083d360 0x56080083f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.855+0000 7f8a639abb80 1 --2- 172.21.15.37:0/486905668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56080082b780 0x56080082bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.855+0000 7f8a639abb80 1 --2- 172.21.15.37:0/486905668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56080082c0a0 0x56080083ce20 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.855+0000 7f8a639abb80 1 -- 172.21.15.37:0/486905668 >> 172.21.15.37:0/486905668 conn(0x56080075b210 msgr2=0x560800829a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:36.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.855+0000 7f8a639abb80 1 -- 172.21.15.37:0/486905668 shutdown_connections 2024-06-10T01:09:36.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.855+0000 7f8a639abb80 1 -- 172.21.15.37:0/486905668 wait complete. 2024-06-10T01:09:36.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.855+0000 7f8a639abb80 1 -- start start 2024-06-10T01:09:36.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.855+0000 7f8a639abb80 1 Processor -- start 2024-06-10T01:09:36.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.856+0000 7f8a639abb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56080082b780 0x56080083cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:36.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.856+0000 7f8a639abb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56080082c0a0 0x560800839e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:36.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.856+0000 7f8a639abb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56080083d360 0x56080083a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:36.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.856+0000 7f8a639abb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560800720150 con 0x56080083d360 2024-06-10T01:09:36.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.856+0000 7f8a639abb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56080077b9a0 con 0x56080082b780 2024-06-10T01:09:36.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.856+0000 7f8a639abb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56080083fd30 con 0x56080082c0a0 2024-06-10T01:09:36.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.856+0000 7f8a639abb80 10 client.0 _ll_register_callbacks cb 0x560800715740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:36.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.856+0000 7f8a5359e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56080083d360 0x56080083a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:36.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.856+0000 7f8a5359e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56080083d360 0x56080083a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34436/0 (socket says 172.21.15.37:34436) 2024-06-10T01:09:36.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.856+0000 7f8a5359e700 1 -- 172.21.15.37:0/3369945952 learned_addr learned my addr 172.21.15.37:0/3369945952 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:36.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.856+0000 7f8a5259c700 1 --2- 172.21.15.37:0/3369945952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56080082c0a0 0x560800839e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:36.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.856+0000 7f8a52d9d700 1 --2- 172.21.15.37:0/3369945952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56080082b780 0x56080083cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:36.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.856+0000 7f8a5359e700 1 -- 172.21.15.37:0/3369945952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56080082c0a0 msgr2=0x560800839e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:36.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.856+0000 7f8a5359e700 1 --2- 172.21.15.37:0/3369945952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56080082c0a0 0x560800839e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.856+0000 7f8a5359e700 1 -- 172.21.15.37:0/3369945952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56080082b780 msgr2=0x56080083cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:36.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.856+0000 7f8a5359e700 1 --2- 172.21.15.37:0/3369945952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56080082b780 0x56080083cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.856+0000 7f8a5359e700 1 -- 172.21.15.37:0/3369945952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608007458e0 con 0x56080083d360 2024-06-10T01:09:36.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.856+0000 7f8a52d9d700 1 --2- 172.21.15.37:0/3369945952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56080082b780 0x56080083cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:09:36.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.856+0000 7f8a5359e700 1 --2- 172.21.15.37:0/3369945952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56080083d360 0x56080083a360 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c0042f0 tx=0x7f8a4c004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25fdfb4f7304fdc0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:36.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.856+0000 7f8a36ff5700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:09:36.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.856+0000 7f8a36ff5700 1 -- 172.21.15.37:0/3369945952 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c01a070 con 0x56080083d360 2024-06-10T01:09:36.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.856+0000 7f8a36ff5700 1 -- 172.21.15.37:0/3369945952 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f8a4c004db0 con 0x56080083d360 2024-06-10T01:09:36.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.856+0000 7f8a639abb80 1 -- 172.21.15.37:0/3369945952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a480055a0 con 0x56080083d360 2024-06-10T01:09:36.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.857+0000 7f8a36ff5700 1 -- 172.21.15.37:0/3369945952 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8a4c025050 con 0x56080083d360 2024-06-10T01:09:36.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.857+0000 7f8a639abb80 10 client.17637 fetch_fsmap learned FSMap version 31 2024-06-10T01:09:36.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.857+0000 7f8a639abb80 1 -- 172.21.15.37:0/3369945952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56080084a4e0 con 0x56080083d360 2024-06-10T01:09:36.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.857+0000 7f8a36ff5700 1 -- 172.21.15.37:0/3369945952 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a480055a0 con 0x56080083d360 2024-06-10T01:09:36.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.857+0000 7f8a36ff5700 1 -- 172.21.15.37:0/3369945952 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f8a4c016720 con 0x56080083d360 2024-06-10T01:09:36.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.857+0000 7f8a639abb80 10 client.17637 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:09:36.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.857+0000 7f8a639abb80 10 client.17637 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:09:36.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.857+0000 7f8a639abb80 20 client.17637 populate_metadata read hostname 'smithi037' 2024-06-10T01:09:36.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.857+0000 7f8a639abb80 1 --2- 172.21.15.37:0/3369945952 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56080084cac0 0x56080084eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:36.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.857+0000 7f8a639abb80 4 client.17637 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:09:36.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.857+0000 7f8a639abb80 1 -- 172.21.15.37:0/3369945952 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56080078cee0 con 0x56080084cac0 2024-06-10T01:09:36.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.857+0000 7f8a52d9d700 1 --2- 172.21.15.37:0/3369945952 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56080084cac0 0x56080084eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:36.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.858+0000 7f8a52d9d700 1 --2- 172.21.15.37:0/3369945952 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56080084cac0 0x56080084eeb0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe6463c468163b61 server_cookie=735fb44152e95e06 in_seq=0 out_seq=0 2024-06-10T01:09:36.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.858+0000 7f8a36ff5700 10 client.17637 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:36.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.859+0000 7f8a36ff5700 1 -- 172.21.15.37:0/3369945952 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56080078cee0 con 0x56080084cac0 2024-06-10T01:09:36.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.859+0000 7f8a36ff5700 10 client.17637 handle_command_reply: tid=0 2024-06-10T01:09:36.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.859+0000 7f8a639abb80 1 client.17637 shutdown 2024-06-10T01:09:36.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.859+0000 7f8a639abb80 1 -- 172.21.15.37:0/3369945952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56080083d360 msgr2=0x56080083a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:36.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.859+0000 7f8a639abb80 1 --2- 172.21.15.37:0/3369945952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56080083d360 0x56080083a360 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c0042f0 tx=0x7f8a4c004320 comp rx=0 tx=0).stop 2024-06-10T01:09:36.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.859+0000 7f8a639abb80 1 -- 172.21.15.37:0/3369945952 shutdown_connections 2024-06-10T01:09:36.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.859+0000 7f8a639abb80 1 --2- 172.21.15.37:0/3369945952 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56080084cac0 0x56080084eeb0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.859+0000 7f8a36ff5700 0 client.17637 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:36.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.859+0000 7f8a639abb80 1 --2- 172.21.15.37:0/3369945952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56080083d360 0x56080083a360 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.859+0000 7f8a639abb80 1 --2- 172.21.15.37:0/3369945952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56080082b780 0x56080083cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.859+0000 7f8a639abb80 1 --2- 172.21.15.37:0/3369945952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56080082c0a0 0x560800839e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.859+0000 7f8a639abb80 1 -- 172.21.15.37:0/3369945952 >> 172.21.15.37:0/3369945952 conn(0x56080075b210 msgr2=0x560800828c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:36.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.859+0000 7f8a639abb80 1 -- 172.21.15.37:0/3369945952 shutdown_connections 2024-06-10T01:09:36.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.859+0000 7f8a639abb80 1 -- 172.21.15.37:0/3369945952 wait complete. 2024-06-10T01:09:36.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.859+0000 7f8a639abb80 20 client.17637 trim_cache size 0 max 16384 2024-06-10T01:09:36.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.881+0000 7f8a639abb80 1 Processor -- start 2024-06-10T01:09:36.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.881+0000 7f8a639abb80 1 -- start start 2024-06-10T01:09:36.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.882+0000 7f8a639abb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56080082b780 0x5608008354d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:36.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.882+0000 7f8a639abb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56080082c0a0 0x560800854780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:36.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.882+0000 7f8a639abb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56080083d360 0x560800854cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:36.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.882+0000 7f8a639abb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541ae920 con 0x56080083d360 2024-06-10T01:09:36.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.882+0000 7f8a639abb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541ac1b0 con 0x56080082c0a0 2024-06-10T01:09:36.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.882+0000 7f8a639abb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541aa000 con 0x56080082b780 2024-06-10T01:09:36.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.882+0000 7f8a5359e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56080083d360 0x560800854cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:36.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.882+0000 7f8a5359e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56080083d360 0x560800854cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34452/0 (socket says 172.21.15.37:34452) 2024-06-10T01:09:36.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.882+0000 7f8a5359e700 1 -- 172.21.15.37:0/3904955695 learned_addr learned my addr 172.21.15.37:0/3904955695 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:36.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.882+0000 7f8a5259c700 1 --2- 172.21.15.37:0/3904955695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56080082c0a0 0x560800854780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:36.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.882+0000 7f8a52d9d700 1 --2- 172.21.15.37:0/3904955695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56080082b780 0x5608008354d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:36.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.882+0000 7f8a52d9d700 1 -- 172.21.15.37:0/3904955695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56080082c0a0 msgr2=0x560800854780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:36.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.882+0000 7f8a52d9d700 1 --2- 172.21.15.37:0/3904955695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56080082c0a0 0x560800854780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.882+0000 7f8a52d9d700 1 -- 172.21.15.37:0/3904955695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56080083d360 msgr2=0x560800854cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:36.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.882+0000 7f8a52d9d700 1 --2- 172.21.15.37:0/3904955695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56080083d360 0x560800854cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.882+0000 7f8a52d9d700 1 -- 172.21.15.37:0/3904955695 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560800835a10 con 0x56080082b780 2024-06-10T01:09:36.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.882+0000 7f8a5259c700 1 --2- 172.21.15.37:0/3904955695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56080082c0a0 0x560800854780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:36.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.882+0000 7f8a5359e700 1 --2- 172.21.15.37:0/3904955695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56080083d360 0x560800854cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:36.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.883+0000 7f8a52d9d700 1 --2- 172.21.15.37:0/3904955695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56080082b780 0x5608008354d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400bfd0 tx=0x7f8a440cf040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b81c2eb9a42e327 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:36.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.883+0000 7f8a38ff9700 1 -- 172.21.15.37:0/3904955695 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a440b3070 con 0x56080082b780 2024-06-10T01:09:36.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.883+0000 7f8a38ff9700 1 -- 172.21.15.37:0/3904955695 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a440a4340 con 0x56080082b780 2024-06-10T01:09:36.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.883+0000 7f8a38ff9700 1 -- 172.21.15.37:0/3904955695 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4409bb90 con 0x56080082b780 2024-06-10T01:09:36.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.884+0000 7f8a639abb80 1 -- 172.21.15.37:0/3904955695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56080082b780 msgr2=0x5608008354d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:36.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.884+0000 7f8a639abb80 1 --2- 172.21.15.37:0/3904955695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56080082b780 0x5608008354d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400bfd0 tx=0x7f8a440cf040 comp rx=0 tx=0).stop 2024-06-10T01:09:36.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.884+0000 7f8a639abb80 1 -- 172.21.15.37:0/3904955695 shutdown_connections 2024-06-10T01:09:36.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.884+0000 7f8a639abb80 1 --2- 172.21.15.37:0/3904955695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56080083d360 0x560800854cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.884+0000 7f8a639abb80 1 --2- 172.21.15.37:0/3904955695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56080082b780 0x5608008354d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.884+0000 7f8a639abb80 1 --2- 172.21.15.37:0/3904955695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56080082c0a0 0x560800854780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.884+0000 7f8a639abb80 1 -- 172.21.15.37:0/3904955695 >> 172.21.15.37:0/3904955695 conn(0x56080075b210 msgr2=0x560800834dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:36.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.884+0000 7f8a639abb80 1 -- 172.21.15.37:0/3904955695 shutdown_connections 2024-06-10T01:09:36.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.884+0000 7f8a639abb80 1 -- 172.21.15.37:0/3904955695 wait complete. 2024-06-10T01:09:36.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.884+0000 7f8a639abb80 1 -- start start 2024-06-10T01:09:36.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.884+0000 7f8a639abb80 1 Processor -- start 2024-06-10T01:09:36.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.884+0000 7f8a639abb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56080082b780 0x560800845750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:36.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.884+0000 7f8a639abb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56080082c0a0 0x560800841ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:36.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.884+0000 7f8a639abb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56080083d360 0x560800842060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:36.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.884+0000 7f8a639abb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541aa670 con 0x56080082c0a0 2024-06-10T01:09:36.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.884+0000 7f8a639abb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541abae0 con 0x56080083d360 2024-06-10T01:09:36.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.884+0000 7f8a639abb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541aad40 con 0x56080082b780 2024-06-10T01:09:36.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.884+0000 7f8a639abb80 10 client.0 _ll_register_callbacks cb 0x560800715740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:36.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.885+0000 7f8a52d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56080082b780 0x560800845750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:36.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.885+0000 7f8a5259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56080082c0a0 0x560800841ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:36.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.885+0000 7f8a52d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56080082b780 0x560800845750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:40404/0 (socket says 172.21.15.37:40404) 2024-06-10T01:09:36.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.885+0000 7f8a52d9d700 1 -- 172.21.15.37:0/3974044556 learned_addr learned my addr 172.21.15.37:0/3974044556 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:36.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.885+0000 7f8a5359e700 1 --2- 172.21.15.37:0/3974044556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56080083d360 0x560800842060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:36.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.885+0000 7f8a52d9d700 1 -- 172.21.15.37:0/3974044556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56080083d360 msgr2=0x560800842060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:36.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.885+0000 7f8a52d9d700 1 --2- 172.21.15.37:0/3974044556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56080083d360 0x560800842060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.885+0000 7f8a52d9d700 1 -- 172.21.15.37:0/3974044556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56080082c0a0 msgr2=0x560800841ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:36.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.885+0000 7f8a52d9d700 1 --2- 172.21.15.37:0/3974044556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56080082c0a0 0x560800841ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.885+0000 7f8a52d9d700 1 -- 172.21.15.37:0/3974044556 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608008428a0 con 0x56080082b780 2024-06-10T01:09:36.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.885+0000 7f8a52d9d700 1 --2- 172.21.15.37:0/3974044556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56080082b780 0x560800845750 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8a4409b610 tx=0x7f8a4409b030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=800163233623dff6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:36.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.885+0000 7f8a36ff5700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:09:36.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.885+0000 7f8a36ff5700 1 -- 172.21.15.37:0/3974044556 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a440b3070 con 0x56080082b780 2024-06-10T01:09:36.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.885+0000 7f8a639abb80 1 -- 172.21.15.37:0/3974044556 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a440a4340 con 0x56080082b780 2024-06-10T01:09:36.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.885+0000 7f8a36ff5700 1 -- 172.21.15.37:0/3974044556 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f8a44005540 con 0x56080082b780 2024-06-10T01:09:36.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.885+0000 7f8a36ff5700 1 -- 172.21.15.37:0/3974044556 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8a44099770 con 0x56080082b780 2024-06-10T01:09:36.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.885+0000 7f8a639abb80 10 client.27118 fetch_fsmap learned FSMap version 31 2024-06-10T01:09:36.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.885+0000 7f8a639abb80 1 -- 172.21.15.37:0/3974044556 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5608008b87f0 con 0x56080082b780 2024-06-10T01:09:36.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.886+0000 7f8a36ff5700 1 -- 172.21.15.37:0/3974044556 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a440a4340 con 0x56080082b780 2024-06-10T01:09:36.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.886+0000 7f8a36ff5700 1 -- 172.21.15.37:0/3974044556 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f8a4409ab20 con 0x56080082b780 2024-06-10T01:09:36.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.886+0000 7f8a639abb80 10 client.27118 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:09:36.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.886+0000 7f8a639abb80 10 client.27118 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:09:36.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.886+0000 7f8a639abb80 20 client.27118 populate_metadata read hostname 'smithi037' 2024-06-10T01:09:36.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.886+0000 7f8a639abb80 1 --2- 172.21.15.37:0/3974044556 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56080084cac0 0x5608008bae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:36.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.886+0000 7f8a639abb80 4 client.27118 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 135.8, "await_for": 45.0}] 2024-06-10T01:09:36.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.886+0000 7f8a639abb80 1 -- 172.21.15.37:0/3974044556 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 135.8, "await_for": 45.0}) -- 0x560800787460 con 0x56080084cac0 2024-06-10T01:09:36.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.886+0000 7f8a5259c700 1 --2- 172.21.15.37:0/3974044556 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56080084cac0 0x5608008bae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:36.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.887+0000 7f8a36ff5700 10 client.27118 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:36.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.887+0000 7f8a5259c700 1 --2- 172.21.15.37:0/3974044556 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56080084cac0 0x5608008bae70 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38901df698fe8a12 server_cookie=d06d377422f5469 in_seq=0 out_seq=0 2024-06-10T01:09:36.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.901+0000 7f8a36ff5700 1 -- 172.21.15.37:0/3974044556 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560800787460 con 0x56080084cac0 2024-06-10T01:09:36.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.901+0000 7f8a36ff5700 10 client.27118 handle_command_reply: tid=0 2024-06-10T01:09:36.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.901+0000 7f8a639abb80 1 client.27118 shutdown 2024-06-10T01:09:36.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.901+0000 7f8a639abb80 1 -- 172.21.15.37:0/3974044556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56080082b780 msgr2=0x560800845750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:36.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.901+0000 7f8a639abb80 1 --2- 172.21.15.37:0/3974044556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56080082b780 0x560800845750 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8a4409b610 tx=0x7f8a4409b030 comp rx=0 tx=0).stop 2024-06-10T01:09:36.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.901+0000 7f8a639abb80 1 -- 172.21.15.37:0/3974044556 shutdown_connections 2024-06-10T01:09:36.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.901+0000 7f8a639abb80 1 --2- 172.21.15.37:0/3974044556 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x56080084cac0 0x5608008bae70 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.901+0000 7f8a36ff5700 0 client.27118 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:36.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.901+0000 7f8a639abb80 1 --2- 172.21.15.37:0/3974044556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56080083d360 0x560800842060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.901+0000 7f8a639abb80 1 --2- 172.21.15.37:0/3974044556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56080082b780 0x560800845750 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.901+0000 7f8a639abb80 1 --2- 172.21.15.37:0/3974044556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56080082c0a0 0x560800841ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.901+0000 7f8a639abb80 1 -- 172.21.15.37:0/3974044556 >> 172.21.15.37:0/3974044556 conn(0x56080075b210 msgr2=0x56080082f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:36.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.901+0000 7f8a639abb80 1 -- 172.21.15.37:0/3974044556 shutdown_connections 2024-06-10T01:09:36.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.902+0000 7f8a639abb80 1 -- 172.21.15.37:0/3974044556 wait complete. 2024-06-10T01:09:36.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.902+0000 7f8a639abb80 20 client.27118 trim_cache size 0 max 16384 2024-06-10T01:09:36.898 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:09:36.898 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 30, 2024-06-10T01:09:36.899 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24472, 2024-06-10T01:09:36.899 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 34, 2024-06-10T01:09:36.899 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-06-10T01:09:36.899 INFO:teuthology.orchestra.run.smithi037.stdout: "9e1aa52e": { 2024-06-10T01:09:36.899 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 34, 2024-06-10T01:09:36.899 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-06-10T01:09:36.899 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:09:36.899 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-06-10T01:09:36.899 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:09:36.899 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:09:36.899 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-06-10T01:09:36.899 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 135.8, 2024-06-10T01:09:36.899 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-06-10T01:09:36.899 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-06-10T01:09:36.900 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-06-10T01:09:36.900 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:09:36.900 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-06-10T01:09:36.900 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:09:36.900 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:09:36.900 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:09:36.900 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:09:36.900 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:09:36.900 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:09:36.900 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:09:36.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.902+0000 7f8a59128700 1 -- 172.21.15.37:0/2523968542 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8a3c0a2cc0 msgr2=0x7f8a3c0a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:36.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.902+0000 7f8a59128700 1 --2- 172.21.15.37:0/2523968542 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8a3c0a2cc0 0x7f8a3c0a5180 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8a48006010 tx=0x7f8a48005960 comp rx=0 tx=0).stop 2024-06-10T01:09:36.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.902+0000 7f8a59128700 1 -- 172.21.15.37:0/2523968542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a541015f0 msgr2=0x7f8a5419d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:36.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.902+0000 7f8a59128700 1 --2- 172.21.15.37:0/2523968542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a541015f0 0x7f8a5419d6b0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400d980 tx=0x7f8a4400dc90 comp rx=0 tx=0).stop 2024-06-10T01:09:36.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.902+0000 7f8a59128700 1 -- 172.21.15.37:0/2523968542 shutdown_connections 2024-06-10T01:09:36.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.902+0000 7f8a59128700 1 --2- 172.21.15.37:0/2523968542 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8a3c0a2cc0 0x7f8a3c0a5180 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.902+0000 7f8a59128700 1 --2- 172.21.15.37:0/2523968542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a541066b0 0x7f8a54197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.902+0000 7f8a59128700 1 --2- 172.21.15.37:0/2523968542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a541015f0 0x7f8a5419d6b0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.902+0000 7f8a59128700 1 --2- 172.21.15.37:0/2523968542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a54101fa0 0x7f8a5419dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:36.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.902+0000 7f8a59128700 1 -- 172.21.15.37:0/2523968542 >> 172.21.15.37:0/2523968542 conn(0x7f8a54078300 msgr2=0x7f8a540ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:36.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.903+0000 7f8a59128700 1 -- 172.21.15.37:0/2523968542 shutdown_connections 2024-06-10T01:09:36.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:36.903+0000 7f8a59128700 1 -- 172.21.15.37:0/2523968542 wait complete. 2024-06-10T01:09:37.626 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9e1aa52e', quiesce took 6.0 seconds. Will release after: 9.8 2024-06-10T01:09:37.666 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:09:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.145+0000 7ff231184640 1 -- 172.21.15.37:0/303289624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff22c107700 msgr2=0x7ff22c109b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.145+0000 7ff231184640 1 --2- 172.21.15.37:0/303289624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff22c107700 0x7ff22c109b50 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff224009ca0 tx=0x7ff22402a8c0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.145+0000 7ff231184640 1 -- 172.21.15.37:0/303289624 shutdown_connections 2024-06-10T01:09:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.145+0000 7ff231184640 1 --2- 172.21.15.37:0/303289624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff22c107700 0x7ff22c109b50 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.145+0000 7ff231184640 1 --2- 172.21.15.37:0/303289624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff22c104dd0 0x7ff22c1071c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.145+0000 7ff231184640 1 --2- 172.21.15.37:0/303289624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff22c1024a0 0x7ff22c104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.145+0000 7ff231184640 1 -- 172.21.15.37:0/303289624 >> 172.21.15.37:0/303289624 conn(0x7ff22c0fbeb0 msgr2=0x7ff22c0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.146+0000 7ff231184640 1 -- 172.21.15.37:0/303289624 shutdown_connections 2024-06-10T01:09:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.146+0000 7ff231184640 1 -- 172.21.15.37:0/303289624 wait complete. 2024-06-10T01:09:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.146+0000 7ff231184640 1 Processor -- start 2024-06-10T01:09:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.146+0000 7ff231184640 1 -- start start 2024-06-10T01:09:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.147+0000 7ff231184640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff22c1024a0 0x7ff22c197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.147+0000 7ff231184640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff22c104dd0 0x7ff22c198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.147+0000 7ff231184640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff22c107700 0x7ff22c19f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.147+0000 7ff231184640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c10d6c0 con 0x7ff22c1024a0 2024-06-10T01:09:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.147+0000 7ff231184640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c10d540 con 0x7ff22c104dd0 2024-06-10T01:09:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.147+0000 7ff231184640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c10d840 con 0x7ff22c107700 2024-06-10T01:09:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.147+0000 7ff22ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff22c1024a0 0x7ff22c197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.147+0000 7ff22ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff22c1024a0 0x7ff22c197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34486/0 (socket says 172.21.15.37:34486) 2024-06-10T01:09:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.147+0000 7ff22ad9d640 1 -- 172.21.15.37:0/3000184688 learned_addr learned my addr 172.21.15.37:0/3000184688 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.147+0000 7ff22b59e640 1 --2- 172.21.15.37:0/3000184688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff22c107700 0x7ff22c19f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.147+0000 7ff22a59c640 1 --2- 172.21.15.37:0/3000184688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff22c104dd0 0x7ff22c198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.147+0000 7ff22ad9d640 1 -- 172.21.15.37:0/3000184688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff22c107700 msgr2=0x7ff22c19f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.147+0000 7ff22ad9d640 1 --2- 172.21.15.37:0/3000184688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff22c107700 0x7ff22c19f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.147+0000 7ff22ad9d640 1 -- 172.21.15.37:0/3000184688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff22c104dd0 msgr2=0x7ff22c198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.147+0000 7ff22ad9d640 1 --2- 172.21.15.37:0/3000184688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff22c104dd0 0x7ff22c198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.147+0000 7ff22ad9d640 1 -- 172.21.15.37:0/3000184688 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff22c19f850 con 0x7ff22c1024a0 2024-06-10T01:09:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.147+0000 7ff22a59c640 1 --2- 172.21.15.37:0/3000184688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff22c104dd0 0x7ff22c198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:09:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.147+0000 7ff22ad9d640 1 --2- 172.21.15.37:0/3000184688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff22c1024a0 0x7ff22c197c40 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ff21c00e9f0 tx=0x7ff21c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.148+0000 7ff213fff640 1 -- 172.21.15.37:0/3000184688 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c00cd80 con 0x7ff22c1024a0 2024-06-10T01:09:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.148+0000 7ff231184640 1 -- 172.21.15.37:0/3000184688 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff22c19fb40 con 0x7ff22c1024a0 2024-06-10T01:09:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.148+0000 7ff213fff640 1 -- 172.21.15.37:0/3000184688 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff21c005500 con 0x7ff22c1024a0 2024-06-10T01:09:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.148+0000 7ff213fff640 1 -- 172.21.15.37:0/3000184688 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c010690 con 0x7ff22c1024a0 2024-06-10T01:09:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.148+0000 7ff231184640 1 -- 172.21.15.37:0/3000184688 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff22c1a0040 con 0x7ff22c1024a0 2024-06-10T01:09:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.150+0000 7ff213fff640 1 -- 172.21.15.37:0/3000184688 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff21c0046c0 con 0x7ff22c1024a0 2024-06-10T01:09:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.150+0000 7ff233f68740 1 Processor -- start 2024-06-10T01:09:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.150+0000 7ff233f68740 1 -- start start 2024-06-10T01:09:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.150+0000 7ff233f68740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d13858010 0x563d13858430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.150+0000 7ff233f68740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d13858a00 0x563d1385d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.151+0000 7ff233f68740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d1385db70 0x563d13860000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.151+0000 7ff233f68740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563d137bb4f0 con 0x563d1385db70 2024-06-10T01:09:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.151+0000 7ff233f68740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563d13759870 con 0x563d13858a00 2024-06-10T01:09:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.151+0000 7ff233f68740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563d137b74b0 con 0x563d13858010 2024-06-10T01:09:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.151+0000 7ff213fff640 1 --2- 172.21.15.37:0/3000184688 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff2140a2c30 0x7ff2140a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.151+0000 7ff22a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d1385db70 0x563d13860000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.151+0000 7ff22a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d1385db70 0x563d13860000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34496/0 (socket says 172.21.15.37:34496) 2024-06-10T01:09:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.151+0000 7ff22a59c640 1 -- 172.21.15.37:0/1596390856 learned_addr learned my addr 172.21.15.37:0/1596390856 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.151+0000 7ff22b59e640 1 --2- 172.21.15.37:0/1596390856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d13858010 0x563d13858430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.151+0000 7ff22b59e640 1 --2- 172.21.15.37:0/3000184688 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff2140a2c30 0x7ff2140a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.151+0000 7ff213fff640 1 -- 172.21.15.37:0/3000184688 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7ff21c0cf9b0 con 0x7ff22c1024a0 2024-06-10T01:09:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.151+0000 7ff22ad9d640 1 --2- 172.21.15.37:0/1596390856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d13858a00 0x563d1385d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.151+0000 7ff22a59c640 1 -- 172.21.15.37:0/1596390856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d13858010 msgr2=0x563d13858430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.151+0000 7ff22a59c640 1 --2- 172.21.15.37:0/1596390856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d13858010 0x563d13858430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.151+0000 7ff22a59c640 1 -- 172.21.15.37:0/1596390856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d13858a00 msgr2=0x563d1385d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.151+0000 7ff22a59c640 1 --2- 172.21.15.37:0/1596390856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d13858a00 0x563d1385d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.151+0000 7ff22a59c640 1 -- 172.21.15.37:0/1596390856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d137759b0 con 0x563d1385db70 2024-06-10T01:09:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.151+0000 7ff22a59c640 1 --2- 172.21.15.37:0/1596390856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d1385db70 0x563d13860000 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7ff220009c00 tx=0x7ff220010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84dcb13b56b0c16 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.151+0000 7ff22b59e640 1 --2- 172.21.15.37:0/3000184688 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff2140a2c30 0x7ff2140a50f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff224004af0 tx=0x7ff224005d80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.151+0000 7ff2117fa640 1 -- 172.21.15.37:0/1596390856 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220004030 con 0x563d1385db70 2024-06-10T01:09:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.151+0000 7ff2117fa640 1 -- 172.21.15.37:0/1596390856 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2200041d0 con 0x563d1385db70 2024-06-10T01:09:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.151+0000 7ff2117fa640 1 -- 172.21.15.37:0/1596390856 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2200126b0 con 0x563d1385db70 2024-06-10T01:09:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.152+0000 7ff233f68740 1 -- 172.21.15.37:0/1596390856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d1385db70 msgr2=0x563d13860000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.152+0000 7ff233f68740 1 --2- 172.21.15.37:0/1596390856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d1385db70 0x563d13860000 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7ff220009c00 tx=0x7ff220010740 comp rx=0 tx=0).stop 2024-06-10T01:09:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.152+0000 7ff233f68740 1 -- 172.21.15.37:0/1596390856 shutdown_connections 2024-06-10T01:09:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.152+0000 7ff233f68740 1 --2- 172.21.15.37:0/1596390856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d1385db70 0x563d13860000 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.152+0000 7ff233f68740 1 --2- 172.21.15.37:0/1596390856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d13858a00 0x563d1385d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.152+0000 7ff233f68740 1 --2- 172.21.15.37:0/1596390856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d13858010 0x563d13858430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.152+0000 7ff233f68740 1 -- 172.21.15.37:0/1596390856 >> 172.21.15.37:0/1596390856 conn(0x563d13853680 msgr2=0x563d13853a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.152+0000 7ff233f68740 1 -- 172.21.15.37:0/1596390856 shutdown_connections 2024-06-10T01:09:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.152+0000 7ff233f68740 1 -- 172.21.15.37:0/1596390856 wait complete. 2024-06-10T01:09:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.152+0000 7ff233f68740 1 -- start start 2024-06-10T01:09:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.153+0000 7ff233f68740 1 Processor -- start 2024-06-10T01:09:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.153+0000 7ff233f68740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d13858010 0x563d138621f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.153+0000 7ff233f68740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d13858a00 0x563d13862730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.153+0000 7ff233f68740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d1385db70 0x563d13869770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.153+0000 7ff233f68740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563d13860b20 con 0x563d13858010 2024-06-10T01:09:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.153+0000 7ff233f68740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563d13744ae0 con 0x563d1385db70 2024-06-10T01:09:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.153+0000 7ff233f68740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563d13860ca0 con 0x563d13858a00 2024-06-10T01:09:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.153+0000 7ff233f68740 10 client.0 _ll_register_callbacks cb 0x563d1377d9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.153+0000 7ff22ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d13858010 0x563d138621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.153+0000 7ff22ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d13858010 0x563d138621f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34500/0 (socket says 172.21.15.37:34500) 2024-06-10T01:09:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.153+0000 7ff22ad9d640 1 -- 172.21.15.37:0/3786024551 learned_addr learned my addr 172.21.15.37:0/3786024551 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.153+0000 7ff22b59e640 1 --2- 172.21.15.37:0/3786024551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d1385db70 0x563d13869770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.153+0000 7ff22a59c640 1 --2- 172.21.15.37:0/3786024551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d13858a00 0x563d13862730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.153+0000 7ff22ad9d640 1 -- 172.21.15.37:0/3786024551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d13858a00 msgr2=0x563d13862730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.153+0000 7ff22ad9d640 1 --2- 172.21.15.37:0/3786024551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d13858a00 0x563d13862730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.153+0000 7ff22ad9d640 1 -- 172.21.15.37:0/3786024551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d1385db70 msgr2=0x563d13869770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.153+0000 7ff22ad9d640 1 --2- 172.21.15.37:0/3786024551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d1385db70 0x563d13869770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.153+0000 7ff22ad9d640 1 -- 172.21.15.37:0/3786024551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d134cf210 con 0x563d13858010 2024-06-10T01:09:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.153+0000 7ff22ad9d640 1 --2- 172.21.15.37:0/3786024551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d13858010 0x563d138621f0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7ff21c098190 tx=0x7ff21c099f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcaf615109820745 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.153+0000 7ff20eff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:09:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.154+0000 7ff20eff5640 1 -- 172.21.15.37:0/3786024551 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c0a0170 con 0x563d13858010 2024-06-10T01:09:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.154+0000 7ff20eff5640 1 -- 172.21.15.37:0/3786024551 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7ff21c0ba5e0 con 0x563d13858010 2024-06-10T01:09:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.154+0000 7ff233f68740 1 -- 172.21.15.37:0/3786024551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2200041d0 con 0x563d13858010 2024-06-10T01:09:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.154+0000 7ff20eff5640 1 -- 172.21.15.37:0/3786024551 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff21c0ba7e0 con 0x563d13858010 2024-06-10T01:09:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.154+0000 7ff233f68740 10 client.17655 fetch_fsmap learned FSMap version 31 2024-06-10T01:09:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.154+0000 7ff233f68740 1 -- 172.21.15.37:0/3786024551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563d1359b620 con 0x563d13858010 2024-06-10T01:09:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.154+0000 7ff20eff5640 1 -- 172.21.15.37:0/3786024551 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2200041d0 con 0x563d13858010 2024-06-10T01:09:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.154+0000 7ff20eff5640 1 -- 172.21.15.37:0/3786024551 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7ff21c09ac80 con 0x563d13858010 2024-06-10T01:09:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.154+0000 7ff233f68740 10 client.17655 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:09:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.154+0000 7ff233f68740 10 client.17655 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.154+0000 7ff233f68740 20 client.17655 populate_metadata read hostname 'smithi037' 2024-06-10T01:09:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.154+0000 7ff233f68740 1 --2- 172.21.15.37:0/3786024551 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x563d1386e0b0 0x563d13870540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.154+0000 7ff233f68740 4 client.17655 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:09:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.154+0000 7ff233f68740 1 -- 172.21.15.37:0/3786024551 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d137cef10 con 0x563d1386e0b0 2024-06-10T01:09:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.155+0000 7ff22a59c640 1 --2- 172.21.15.37:0/3786024551 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x563d1386e0b0 0x563d13870540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.155+0000 7ff22a59c640 1 --2- 172.21.15.37:0/3786024551 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x563d1386e0b0 0x563d13870540 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8442e3c7f5e78306 server_cookie=6521da3b8e7737ae in_seq=0 out_seq=0 2024-06-10T01:09:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.155+0000 7ff20eff5640 10 client.17655 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.156+0000 7ff20eff5640 1 -- 172.21.15.37:0/3786024551 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563d137cef10 con 0x563d1386e0b0 2024-06-10T01:09:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.156+0000 7ff20eff5640 10 client.17655 handle_command_reply: tid=0 2024-06-10T01:09:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.156+0000 7ff233f68740 1 client.17655 shutdown 2024-06-10T01:09:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.156+0000 7ff233f68740 1 -- 172.21.15.37:0/3786024551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d13858010 msgr2=0x563d138621f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.156+0000 7ff233f68740 1 --2- 172.21.15.37:0/3786024551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d13858010 0x563d138621f0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7ff21c098190 tx=0x7ff21c099f00 comp rx=0 tx=0).stop 2024-06-10T01:09:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.156+0000 7ff233f68740 1 -- 172.21.15.37:0/3786024551 shutdown_connections 2024-06-10T01:09:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.156+0000 7ff233f68740 1 --2- 172.21.15.37:0/3786024551 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x563d1386e0b0 0x563d13870540 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.156+0000 7ff20eff5640 0 client.17655 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.156+0000 7ff233f68740 1 --2- 172.21.15.37:0/3786024551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d1385db70 0x563d13869770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.156+0000 7ff233f68740 1 --2- 172.21.15.37:0/3786024551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d13858a00 0x563d13862730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.156+0000 7ff233f68740 1 --2- 172.21.15.37:0/3786024551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d13858010 0x563d138621f0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.156+0000 7ff233f68740 1 -- 172.21.15.37:0/3786024551 >> 172.21.15.37:0/3786024551 conn(0x563d13853680 msgr2=0x563d138527e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.156+0000 7ff233f68740 1 -- 172.21.15.37:0/3786024551 shutdown_connections 2024-06-10T01:09:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.156+0000 7ff233f68740 1 -- 172.21.15.37:0/3786024551 wait complete. 2024-06-10T01:09:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.156+0000 7ff233f68740 20 client.17655 trim_cache size 0 max 16384 2024-06-10T01:09:38.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.171+0000 7ff233f68740 1 Processor -- start 2024-06-10T01:09:38.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.172+0000 7ff233f68740 1 -- start start 2024-06-10T01:09:38.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.172+0000 7ff233f68740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d13858010 0x563d13856590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:38.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.172+0000 7ff233f68740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d13858a00 0x563d13869700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:38.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.172+0000 7ff233f68740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d1385db70 0x563d13856ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:38.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.172+0000 7ff233f68740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c1acf00 con 0x563d1385db70 2024-06-10T01:09:38.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.172+0000 7ff233f68740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c1aa7d0 con 0x563d13858a00 2024-06-10T01:09:38.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.172+0000 7ff233f68740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c1a8620 con 0x563d13858010 2024-06-10T01:09:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.172+0000 7ff22b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d1385db70 0x563d13856ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.172+0000 7ff22b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d1385db70 0x563d13856ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34510/0 (socket says 172.21.15.37:34510) 2024-06-10T01:09:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.172+0000 7ff22b59e640 1 -- 172.21.15.37:0/3450954374 learned_addr learned my addr 172.21.15.37:0/3450954374 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.172+0000 7ff22ad9d640 1 --2- 172.21.15.37:0/3450954374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d13858010 0x563d13856590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.172+0000 7ff22a59c640 1 --2- 172.21.15.37:0/3450954374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d13858a00 0x563d13869700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.172+0000 7ff22b59e640 1 -- 172.21.15.37:0/3450954374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d13858010 msgr2=0x563d13856590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.172+0000 7ff22b59e640 1 --2- 172.21.15.37:0/3450954374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d13858010 0x563d13856590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.172+0000 7ff22b59e640 1 -- 172.21.15.37:0/3450954374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d13858a00 msgr2=0x563d13869700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.172+0000 7ff22b59e640 1 --2- 172.21.15.37:0/3450954374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d13858a00 0x563d13869700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.172+0000 7ff22b59e640 1 -- 172.21.15.37:0/3450954374 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d138534b0 con 0x563d1385db70 2024-06-10T01:09:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.172+0000 7ff22ad9d640 1 --2- 172.21.15.37:0/3450954374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d13858010 0x563d13856590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:09:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.172+0000 7ff22a59c640 1 --2- 172.21.15.37:0/3450954374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d13858a00 0x563d13869700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:09:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.172+0000 7ff22b59e640 1 --2- 172.21.15.37:0/3450954374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d1385db70 0x563d13856ad0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ff224009ca0 tx=0x7ff224036000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af829f35133ce58a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.173+0000 7ff210ff9640 1 -- 172.21.15.37:0/3450954374 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224059070 con 0x563d1385db70 2024-06-10T01:09:38.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.173+0000 7ff210ff9640 1 -- 172.21.15.37:0/3450954374 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff224036070 con 0x563d1385db70 2024-06-10T01:09:38.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.173+0000 7ff210ff9640 1 -- 172.21.15.37:0/3450954374 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22404a6a0 con 0x563d1385db70 2024-06-10T01:09:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.173+0000 7ff233f68740 1 -- 172.21.15.37:0/3450954374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d1385db70 msgr2=0x563d13856ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.173+0000 7ff233f68740 1 --2- 172.21.15.37:0/3450954374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d1385db70 0x563d13856ad0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ff224009ca0 tx=0x7ff224036000 comp rx=0 tx=0).stop 2024-06-10T01:09:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.173+0000 7ff233f68740 1 -- 172.21.15.37:0/3450954374 shutdown_connections 2024-06-10T01:09:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.173+0000 7ff233f68740 1 --2- 172.21.15.37:0/3450954374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d1385db70 0x563d13856ad0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.173+0000 7ff233f68740 1 --2- 172.21.15.37:0/3450954374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d13858a00 0x563d13869700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.173+0000 7ff233f68740 1 --2- 172.21.15.37:0/3450954374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d13858010 0x563d13856590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.173+0000 7ff233f68740 1 -- 172.21.15.37:0/3450954374 >> 172.21.15.37:0/3450954374 conn(0x563d13853680 msgr2=0x563d1385d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.173+0000 7ff233f68740 1 -- 172.21.15.37:0/3450954374 shutdown_connections 2024-06-10T01:09:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.173+0000 7ff233f68740 1 -- 172.21.15.37:0/3450954374 wait complete. 2024-06-10T01:09:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.174+0000 7ff233f68740 1 -- start start 2024-06-10T01:09:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.174+0000 7ff233f68740 1 Processor -- start 2024-06-10T01:09:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.174+0000 7ff233f68740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d13858010 0x563d13866a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.174+0000 7ff233f68740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d13858a00 0x563d13866fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.174+0000 7ff233f68740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d1385db70 0x563d13867fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.174+0000 7ff233f68740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c1a8c90 con 0x563d1385db70 2024-06-10T01:09:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.174+0000 7ff233f68740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c1aa100 con 0x563d13858010 2024-06-10T01:09:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.174+0000 7ff233f68740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c1a9360 con 0x563d13858a00 2024-06-10T01:09:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.174+0000 7ff233f68740 10 client.0 _ll_register_callbacks cb 0x7ff21c00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.174+0000 7ff22b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d1385db70 0x563d13867fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.174+0000 7ff22b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d1385db70 0x563d13867fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34526/0 (socket says 172.21.15.37:34526) 2024-06-10T01:09:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.174+0000 7ff22b59e640 1 -- 172.21.15.37:0/1292326925 learned_addr learned my addr 172.21.15.37:0/1292326925 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.174+0000 7ff22ad9d640 1 --2- 172.21.15.37:0/1292326925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d13858010 0x563d13866a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.174+0000 7ff22a59c640 1 --2- 172.21.15.37:0/1292326925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d13858a00 0x563d13866fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.174+0000 7ff22b59e640 1 -- 172.21.15.37:0/1292326925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d13858a00 msgr2=0x563d13866fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.174+0000 7ff22b59e640 1 --2- 172.21.15.37:0/1292326925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d13858a00 0x563d13866fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.174+0000 7ff22b59e640 1 -- 172.21.15.37:0/1292326925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d13858010 msgr2=0x563d13866a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.174+0000 7ff22b59e640 1 --2- 172.21.15.37:0/1292326925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d13858010 0x563d13866a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.174+0000 7ff22b59e640 1 -- 172.21.15.37:0/1292326925 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d138685c0 con 0x563d1385db70 2024-06-10T01:09:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.174+0000 7ff22ad9d640 1 --2- 172.21.15.37:0/1292326925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d13858010 0x563d13866a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:09:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.175+0000 7ff22b59e640 1 --2- 172.21.15.37:0/1292326925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d1385db70 0x563d13867fd0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ff224030d70 tx=0x7ff22402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655cbed3c2781a8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.175+0000 7ff20eff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:09:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.175+0000 7ff20eff5640 1 -- 172.21.15.37:0/1292326925 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224059070 con 0x563d1385db70 2024-06-10T01:09:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.175+0000 7ff20eff5640 1 -- 172.21.15.37:0/1292326925 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7ff22404a440 con 0x563d1385db70 2024-06-10T01:09:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.175+0000 7ff233f68740 1 -- 172.21.15.37:0/1292326925 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff224036070 con 0x563d1385db70 2024-06-10T01:09:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.175+0000 7ff20eff5640 1 -- 172.21.15.37:0/1292326925 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff2240309d0 con 0x563d1385db70 2024-06-10T01:09:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.175+0000 7ff233f68740 10 client.17661 fetch_fsmap learned FSMap version 31 2024-06-10T01:09:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.175+0000 7ff233f68740 1 -- 172.21.15.37:0/1292326925 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563d138698d0 con 0x563d1385db70 2024-06-10T01:09:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.175+0000 7ff20eff5640 1 -- 172.21.15.37:0/1292326925 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff224036070 con 0x563d1385db70 2024-06-10T01:09:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.175+0000 7ff20eff5640 1 -- 172.21.15.37:0/1292326925 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7ff224048040 con 0x563d1385db70 2024-06-10T01:09:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.175+0000 7ff233f68740 10 client.17661 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:09:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.175+0000 7ff233f68740 10 client.17661 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.175+0000 7ff233f68740 20 client.17661 populate_metadata read hostname 'smithi037' 2024-06-10T01:09:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.175+0000 7ff233f68740 1 --2- 172.21.15.37:0/1292326925 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x563d138d94e0 0x563d138db900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.175+0000 7ff233f68740 4 client.17661 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:09:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.175+0000 7ff233f68740 1 -- 172.21.15.37:0/1292326925 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563d137c8c20 con 0x563d138d94e0 2024-06-10T01:09:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.176+0000 7ff22ad9d640 1 --2- 172.21.15.37:0/1292326925 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x563d138d94e0 0x563d138db900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.176+0000 7ff20eff5640 10 client.17661 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.176+0000 7ff22ad9d640 1 --2- 172.21.15.37:0/1292326925 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x563d138d94e0 0x563d138db900 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61497fa3bb913bcc server_cookie=579682b6d666ce57 in_seq=0 out_seq=0 2024-06-10T01:09:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.203+0000 7ff20eff5640 1 -- 172.21.15.37:0/1292326925 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563d137c8c20 con 0x563d138d94e0 2024-06-10T01:09:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.203+0000 7ff20eff5640 10 client.17661 handle_command_reply: tid=0 2024-06-10T01:09:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.203+0000 7ff233f68740 1 client.17661 shutdown 2024-06-10T01:09:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.203+0000 7ff233f68740 1 -- 172.21.15.37:0/1292326925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d1385db70 msgr2=0x563d13867fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.203+0000 7ff233f68740 1 --2- 172.21.15.37:0/1292326925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d1385db70 0x563d13867fd0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ff224030d70 tx=0x7ff22402a8c0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.203+0000 7ff233f68740 1 -- 172.21.15.37:0/1292326925 shutdown_connections 2024-06-10T01:09:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.203+0000 7ff233f68740 1 --2- 172.21.15.37:0/1292326925 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x563d138d94e0 0x563d138db900 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.203+0000 7ff233f68740 1 --2- 172.21.15.37:0/1292326925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d1385db70 0x563d13867fd0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.203+0000 7ff20eff5640 0 client.17661 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.203+0000 7ff233f68740 1 --2- 172.21.15.37:0/1292326925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d13858a00 0x563d13866fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.203+0000 7ff233f68740 1 --2- 172.21.15.37:0/1292326925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d13858010 0x563d13866a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.203+0000 7ff233f68740 1 -- 172.21.15.37:0/1292326925 >> 172.21.15.37:0/1292326925 conn(0x563d13853680 msgr2=0x563d13856590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.203+0000 7ff233f68740 1 -- 172.21.15.37:0/1292326925 shutdown_connections 2024-06-10T01:09:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.203+0000 7ff233f68740 1 -- 172.21.15.37:0/1292326925 wait complete. 2024-06-10T01:09:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.204+0000 7ff233f68740 20 client.17661 trim_cache size 0 max 16384 2024-06-10T01:09:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:09:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:09:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:09:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.204+0000 7ff231184640 1 -- 172.21.15.37:0/3000184688 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff2140a2c30 msgr2=0x7ff2140a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.204+0000 7ff231184640 1 --2- 172.21.15.37:0/3000184688 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff2140a2c30 0x7ff2140a50f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff224004af0 tx=0x7ff224005d80 comp rx=0 tx=0).stop 2024-06-10T01:09:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.204+0000 7ff231184640 1 -- 172.21.15.37:0/3000184688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff22c1024a0 msgr2=0x7ff22c197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.204+0000 7ff231184640 1 --2- 172.21.15.37:0/3000184688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff22c1024a0 0x7ff22c197c40 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ff21c00e9f0 tx=0x7ff21c00edf0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.204+0000 7ff231184640 1 -- 172.21.15.37:0/3000184688 shutdown_connections 2024-06-10T01:09:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.204+0000 7ff231184640 1 --2- 172.21.15.37:0/3000184688 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7ff2140a2c30 0x7ff2140a50f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.204+0000 7ff231184640 1 --2- 172.21.15.37:0/3000184688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff22c107700 0x7ff22c19f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.204+0000 7ff231184640 1 --2- 172.21.15.37:0/3000184688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff22c104dd0 0x7ff22c198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.204+0000 7ff231184640 1 --2- 172.21.15.37:0/3000184688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff22c1024a0 0x7ff22c197c40 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.204+0000 7ff231184640 1 -- 172.21.15.37:0/3000184688 >> 172.21.15.37:0/3000184688 conn(0x7ff22c0fbeb0 msgr2=0x7ff22c102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.204+0000 7ff231184640 1 -- 172.21.15.37:0/3000184688 shutdown_connections 2024-06-10T01:09:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:38.204+0000 7ff231184640 1 -- 172.21.15.37:0/3000184688 wait complete. 2024-06-10T01:09:38.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:38 smithi037 ceph-mon[32192]: pgmap v819: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 40 op/s 2024-06-10T01:09:38.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:38 smithi167 ceph-mon[36917]: pgmap v819: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 40 op/s 2024-06-10T01:09:38.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:38 smithi094 ceph-mon[36708]: pgmap v819: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 40 op/s 2024-06-10T01:09:39.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.422+0000 7fdd793d6700 1 -- 172.21.15.37:0/1325837109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd74101380 msgr2=0x7fdd74101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:39.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.422+0000 7fdd793d6700 1 --2- 172.21.15.37:0/1325837109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd74101380 0x7fdd74101800 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fdd68009990 tx=0x7fdd68009ca0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.422+0000 7fdd793d6700 1 -- 172.21.15.37:0/1325837109 shutdown_connections 2024-06-10T01:09:39.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.422+0000 7fdd793d6700 1 --2- 172.21.15.37:0/1325837109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd74101d40 0x7fdd7410e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.422+0000 7fdd793d6700 1 --2- 172.21.15.37:0/1325837109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd741073c0 0x7fdd741077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.422+0000 7fdd793d6700 1 --2- 172.21.15.37:0/1325837109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd74101380 0x7fdd74101800 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.422+0000 7fdd793d6700 1 -- 172.21.15.37:0/1325837109 >> 172.21.15.37:0/1325837109 conn(0x7fdd74078330 msgr2=0x7fdd74078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:39.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.422+0000 7fdd793d6700 1 -- 172.21.15.37:0/1325837109 shutdown_connections 2024-06-10T01:09:39.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.422+0000 7fdd793d6700 1 -- 172.21.15.37:0/1325837109 wait complete. 2024-06-10T01:09:39.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.423+0000 7fdd793d6700 1 Processor -- start 2024-06-10T01:09:39.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.423+0000 7fdd793d6700 1 -- start start 2024-06-10T01:09:39.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.423+0000 7fdd793d6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd74101380 0x7fdd74199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:39.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.423+0000 7fdd793d6700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd74101d40 0x7fdd74199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:39.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.423+0000 7fdd793d6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd741073c0 0x7fdd741a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:39.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.423+0000 7fdd793d6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdd74111630 con 0x7fdd74101380 2024-06-10T01:09:39.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.423+0000 7fdd793d6700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdd741114b0 con 0x7fdd741073c0 2024-06-10T01:09:39.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.423+0000 7fdd793d6700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdd741117b0 con 0x7fdd74101d40 2024-06-10T01:09:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.423+0000 7fdd72ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd74101380 0x7fdd74199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.423+0000 7fdd72ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd74101380 0x7fdd74199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34552/0 (socket says 172.21.15.37:34552) 2024-06-10T01:09:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.423+0000 7fdd72ffd700 1 -- 172.21.15.37:0/2364824460 learned_addr learned my addr 172.21.15.37:0/2364824460 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.424+0000 7fdd727fc700 1 --2- 172.21.15.37:0/2364824460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd74101d40 0x7fdd74199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.424+0000 7fdd737fe700 1 --2- 172.21.15.37:0/2364824460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd741073c0 0x7fdd741a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.424+0000 7fdd72ffd700 1 -- 172.21.15.37:0/2364824460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd74101d40 msgr2=0x7fdd74199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.424+0000 7fdd72ffd700 1 --2- 172.21.15.37:0/2364824460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd74101d40 0x7fdd74199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.424+0000 7fdd72ffd700 1 -- 172.21.15.37:0/2364824460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd741073c0 msgr2=0x7fdd741a49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.424+0000 7fdd72ffd700 1 --2- 172.21.15.37:0/2364824460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd741073c0 0x7fdd741a49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.424+0000 7fdd72ffd700 1 -- 172.21.15.37:0/2364824460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd741a4f10 con 0x7fdd74101380 2024-06-10T01:09:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.424+0000 7fdd727fc700 1 --2- 172.21.15.37:0/2364824460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd74101d40 0x7fdd74199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:09:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.424+0000 7fdd72ffd700 1 --2- 172.21.15.37:0/2364824460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd74101380 0x7fdd74199950 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400c9b0 tx=0x7fdd6400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.424+0000 7fdd5bfff700 1 -- 172.21.15.37:0/2364824460 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64007b00 con 0x7fdd74101380 2024-06-10T01:09:39.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.424+0000 7fdd5bfff700 1 -- 172.21.15.37:0/2364824460 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd64007ca0 con 0x7fdd74101380 2024-06-10T01:09:39.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.424+0000 7fdd793d6700 1 -- 172.21.15.37:0/2364824460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd741a5140 con 0x7fdd74101380 2024-06-10T01:09:39.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.424+0000 7fdd5bfff700 1 -- 172.21.15.37:0/2364824460 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64018780 con 0x7fdd74101380 2024-06-10T01:09:39.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.424+0000 7fdd793d6700 1 -- 172.21.15.37:0/2364824460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd741a56a0 con 0x7fdd74101380 2024-06-10T01:09:39.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.426+0000 7fdd5bfff700 1 -- 172.21.15.37:0/2364824460 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fdd6400f450 con 0x7fdd74101380 2024-06-10T01:09:39.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.427+0000 7fdd5bfff700 1 --2- 172.21.15.37:0/2364824460 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdd5c0a2ca0 0x7fdd5c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:39.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.427+0000 7fdd727fc700 1 --2- 172.21.15.37:0/2364824460 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdd5c0a2ca0 0x7fdd5c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:39.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.427+0000 7fdd5bfff700 1 -- 172.21.15.37:0/2364824460 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fdd640cf3e0 con 0x7fdd74101380 2024-06-10T01:09:39.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.428+0000 7fdd727fc700 1 --2- 172.21.15.37:0/2364824460 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdd5c0a2ca0 0x7fdd5c0a5160 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fdd68006010 tx=0x7fdd6800b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:39.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.428+0000 7fdd83c59b80 1 Processor -- start 2024-06-10T01:09:39.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.428+0000 7fdd83c59b80 1 -- start start 2024-06-10T01:09:39.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.428+0000 7fdd83c59b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560150e65a70 0x560150e65e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:39.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.428+0000 7fdd83c59b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560150e66390 0x560150e5fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:39.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.428+0000 7fdd83c59b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560150e60330 0x560150e60750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:39.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.428+0000 7fdd83c59b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560150dad810 con 0x560150e66390 2024-06-10T01:09:39.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.428+0000 7fdd83c59b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560150d55720 con 0x560150e65a70 2024-06-10T01:09:39.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.428+0000 7fdd83c59b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560150da97d0 con 0x560150e60330 2024-06-10T01:09:39.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.428+0000 7fdd727fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560150e66390 0x560150e5fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.428+0000 7fdd727fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560150e66390 0x560150e5fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34562/0 (socket says 172.21.15.37:34562) 2024-06-10T01:09:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.428+0000 7fdd727fc700 1 -- 172.21.15.37:0/1802663390 learned_addr learned my addr 172.21.15.37:0/1802663390 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.429+0000 7fdd737fe700 1 --2- 172.21.15.37:0/1802663390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560150e60330 0x560150e60750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.429+0000 7fdd72ffd700 1 --2- 172.21.15.37:0/1802663390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560150e65a70 0x560150e65e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.429+0000 7fdd727fc700 1 -- 172.21.15.37:0/1802663390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560150e60330 msgr2=0x560150e60750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.429+0000 7fdd727fc700 1 --2- 172.21.15.37:0/1802663390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560150e60330 0x560150e60750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.429+0000 7fdd727fc700 1 -- 172.21.15.37:0/1802663390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560150e65a70 msgr2=0x560150e65e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.429+0000 7fdd727fc700 1 --2- 172.21.15.37:0/1802663390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560150e65a70 0x560150e65e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.429+0000 7fdd727fc700 1 -- 172.21.15.37:0/1802663390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560150d4bf50 con 0x560150e66390 2024-06-10T01:09:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.429+0000 7fdd72ffd700 1 --2- 172.21.15.37:0/1802663390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560150e65a70 0x560150e65e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:09:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.429+0000 7fdd727fc700 1 --2- 172.21.15.37:0/1802663390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560150e66390 0x560150e5fdf0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fdd68015000 tx=0x7fdd6801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1289f1a8728968e0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.429+0000 7fdd597fa700 1 -- 172.21.15.37:0/1802663390 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6800be90 con 0x560150e66390 2024-06-10T01:09:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.429+0000 7fdd597fa700 1 -- 172.21.15.37:0/1802663390 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd6800f940 con 0x560150e66390 2024-06-10T01:09:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.429+0000 7fdd597fa700 1 -- 172.21.15.37:0/1802663390 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd68024990 con 0x560150e66390 2024-06-10T01:09:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.430+0000 7fdd83c59b80 1 -- 172.21.15.37:0/1802663390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560150e66390 msgr2=0x560150e5fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.430+0000 7fdd83c59b80 1 --2- 172.21.15.37:0/1802663390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560150e66390 0x560150e5fdf0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fdd68015000 tx=0x7fdd6801f000 comp rx=0 tx=0).stop 2024-06-10T01:09:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.430+0000 7fdd83c59b80 1 -- 172.21.15.37:0/1802663390 shutdown_connections 2024-06-10T01:09:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.430+0000 7fdd83c59b80 1 --2- 172.21.15.37:0/1802663390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560150e60330 0x560150e60750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.430+0000 7fdd83c59b80 1 --2- 172.21.15.37:0/1802663390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560150e65a70 0x560150e65e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.430+0000 7fdd83c59b80 1 --2- 172.21.15.37:0/1802663390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560150e66390 0x560150e5fdf0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.430+0000 7fdd83c59b80 1 -- 172.21.15.37:0/1802663390 >> 172.21.15.37:0/1802663390 conn(0x560150d86cc0 msgr2=0x560150e56850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.430+0000 7fdd83c59b80 1 -- 172.21.15.37:0/1802663390 shutdown_connections 2024-06-10T01:09:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.430+0000 7fdd83c59b80 1 -- 172.21.15.37:0/1802663390 wait complete. 2024-06-10T01:09:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.430+0000 7fdd83c59b80 1 -- start start 2024-06-10T01:09:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.430+0000 7fdd83c59b80 1 Processor -- start 2024-06-10T01:09:39.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.431+0000 7fdd83c59b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560150e60330 0x560150e6c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:39.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.431+0000 7fdd83c59b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560150e65a70 0x560150e69400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:39.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.431+0000 7fdd83c59b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560150e66390 0x560150e69940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:39.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.431+0000 7fdd83c59b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560150e6cd10 con 0x560150e66390 2024-06-10T01:09:39.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.431+0000 7fdd83c59b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560150d4dbf0 con 0x560150e60330 2024-06-10T01:09:39.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.431+0000 7fdd83c59b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560150e6ce90 con 0x560150e65a70 2024-06-10T01:09:39.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.431+0000 7fdd83c59b80 10 client.0 _ll_register_callbacks cb 0x560150d47920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:39.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.431+0000 7fdd737fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560150e66390 0x560150e69940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:39.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.431+0000 7fdd737fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560150e66390 0x560150e69940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34564/0 (socket says 172.21.15.37:34564) 2024-06-10T01:09:39.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.431+0000 7fdd737fe700 1 -- 172.21.15.37:0/164301632 learned_addr learned my addr 172.21.15.37:0/164301632 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:39.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.431+0000 7fdd72ffd700 1 --2- 172.21.15.37:0/164301632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560150e60330 0x560150e6c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:39.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.431+0000 7fdd727fc700 1 --2- 172.21.15.37:0/164301632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560150e65a70 0x560150e69400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:39.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.431+0000 7fdd737fe700 1 -- 172.21.15.37:0/164301632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560150e65a70 msgr2=0x560150e69400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:39.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.431+0000 7fdd737fe700 1 --2- 172.21.15.37:0/164301632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560150e65a70 0x560150e69400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.431+0000 7fdd737fe700 1 -- 172.21.15.37:0/164301632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560150e60330 msgr2=0x560150e6c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:39.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.431+0000 7fdd737fe700 1 --2- 172.21.15.37:0/164301632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560150e60330 0x560150e6c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.431+0000 7fdd737fe700 1 -- 172.21.15.37:0/164301632 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560150cfc8d0 con 0x560150e66390 2024-06-10T01:09:39.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.431+0000 7fdd737fe700 1 --2- 172.21.15.37:0/164301632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560150e66390 0x560150e69940 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c011d00 tx=0x7fdd6c00f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818d8bf0e18ae2b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:39.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.431+0000 7fdd56ff5700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:09:39.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.432+0000 7fdd56ff5700 1 -- 172.21.15.37:0/164301632 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c019070 con 0x560150e66390 2024-06-10T01:09:39.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.432+0000 7fdd56ff5700 1 -- 172.21.15.37:0/164301632 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fdd6c00fa10 con 0x560150e66390 2024-06-10T01:09:39.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.432+0000 7fdd83c59b80 1 -- 172.21.15.37:0/164301632 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd6800f940 con 0x560150e66390 2024-06-10T01:09:39.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.432+0000 7fdd56ff5700 1 -- 172.21.15.37:0/164301632 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdd6c025050 con 0x560150e66390 2024-06-10T01:09:39.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.432+0000 7fdd83c59b80 10 client.17673 fetch_fsmap learned FSMap version 31 2024-06-10T01:09:39.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.432+0000 7fdd83c59b80 1 -- 172.21.15.37:0/164301632 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560150e77670 con 0x560150e66390 2024-06-10T01:09:39.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.432+0000 7fdd56ff5700 1 -- 172.21.15.37:0/164301632 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd6800f940 con 0x560150e66390 2024-06-10T01:09:39.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.432+0000 7fdd56ff5700 1 -- 172.21.15.37:0/164301632 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fdd6c0143d0 con 0x560150e66390 2024-06-10T01:09:39.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.432+0000 7fdd83c59b80 10 client.17673 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:09:39.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.432+0000 7fdd83c59b80 10 client.17673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:39.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.432+0000 7fdd83c59b80 20 client.17673 populate_metadata read hostname 'smithi037' 2024-06-10T01:09:39.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.432+0000 7fdd83c59b80 1 --2- 172.21.15.37:0/164301632 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560150e79c00 0x560150e7c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:39.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.432+0000 7fdd83c59b80 4 client.17673 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:09:39.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.432+0000 7fdd83c59b80 1 -- 172.21.15.37:0/164301632 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560150dbad10 con 0x560150e79c00 2024-06-10T01:09:39.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.433+0000 7fdd72ffd700 1 --2- 172.21.15.37:0/164301632 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560150e79c00 0x560150e7c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:39.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.433+0000 7fdd56ff5700 10 client.17673 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:39.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.433+0000 7fdd72ffd700 1 --2- 172.21.15.37:0/164301632 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560150e79c00 0x560150e7c050 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dacece75138ba93 server_cookie=f2d6ac4a8c3bb843 in_seq=0 out_seq=0 2024-06-10T01:09:39.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.434+0000 7fdd56ff5700 1 -- 172.21.15.37:0/164301632 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560150dbad10 con 0x560150e79c00 2024-06-10T01:09:39.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.434+0000 7fdd56ff5700 10 client.17673 handle_command_reply: tid=0 2024-06-10T01:09:39.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.434+0000 7fdd83c59b80 1 client.17673 shutdown 2024-06-10T01:09:39.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.434+0000 7fdd83c59b80 1 -- 172.21.15.37:0/164301632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560150e66390 msgr2=0x560150e69940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:39.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.434+0000 7fdd83c59b80 1 --2- 172.21.15.37:0/164301632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560150e66390 0x560150e69940 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c011d00 tx=0x7fdd6c00f4b0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.434+0000 7fdd83c59b80 1 -- 172.21.15.37:0/164301632 shutdown_connections 2024-06-10T01:09:39.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.434+0000 7fdd83c59b80 1 --2- 172.21.15.37:0/164301632 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560150e79c00 0x560150e7c050 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.434+0000 7fdd83c59b80 1 --2- 172.21.15.37:0/164301632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560150e66390 0x560150e69940 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.434+0000 7fdd83c59b80 1 --2- 172.21.15.37:0/164301632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560150e60330 0x560150e6c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.434+0000 7fdd83c59b80 1 --2- 172.21.15.37:0/164301632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560150e65a70 0x560150e69400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.434+0000 7fdd83c59b80 1 -- 172.21.15.37:0/164301632 >> 172.21.15.37:0/164301632 conn(0x560150d86cc0 msgr2=0x560150e5a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:39.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.434+0000 7fdd56ff5700 0 client.17673 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:39.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.434+0000 7fdd83c59b80 1 -- 172.21.15.37:0/164301632 shutdown_connections 2024-06-10T01:09:39.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.434+0000 7fdd83c59b80 1 -- 172.21.15.37:0/164301632 wait complete. 2024-06-10T01:09:39.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.434+0000 7fdd83c59b80 20 client.17673 trim_cache size 0 max 16384 2024-06-10T01:09:39.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.455+0000 7fdd83c59b80 1 Processor -- start 2024-06-10T01:09:39.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.455+0000 7fdd83c59b80 1 -- start start 2024-06-10T01:09:39.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.455+0000 7fdd83c59b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560150e60330 0x560150e68d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:39.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.456+0000 7fdd83c59b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560150e65a70 0x560150e66f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:39.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.456+0000 7fdd83c59b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560150e66390 0x560150e81920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:39.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.456+0000 7fdd83c59b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdd741aea00 con 0x560150e66390 2024-06-10T01:09:39.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.456+0000 7fdd83c59b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdd741ac290 con 0x560150e65a70 2024-06-10T01:09:39.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.456+0000 7fdd83c59b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdd741aa100 con 0x560150e60330 2024-06-10T01:09:39.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.456+0000 7fdd737fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560150e66390 0x560150e81920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:39.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.456+0000 7fdd737fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560150e66390 0x560150e81920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34566/0 (socket says 172.21.15.37:34566) 2024-06-10T01:09:39.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.456+0000 7fdd737fe700 1 -- 172.21.15.37:0/1692315310 learned_addr learned my addr 172.21.15.37:0/1692315310 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:39.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.456+0000 7fdd72ffd700 1 --2- 172.21.15.37:0/1692315310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560150e60330 0x560150e68d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:39.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.456+0000 7fdd727fc700 1 --2- 172.21.15.37:0/1692315310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560150e65a70 0x560150e66f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:39.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.456+0000 7fdd737fe700 1 -- 172.21.15.37:0/1692315310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560150e60330 msgr2=0x560150e68d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:39.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.456+0000 7fdd737fe700 1 --2- 172.21.15.37:0/1692315310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560150e60330 0x560150e68d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.456+0000 7fdd737fe700 1 -- 172.21.15.37:0/1692315310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560150e65a70 msgr2=0x560150e66f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:39.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.456+0000 7fdd737fe700 1 --2- 172.21.15.37:0/1692315310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560150e65a70 0x560150e66f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.456+0000 7fdd737fe700 1 -- 172.21.15.37:0/1692315310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560150e679f0 con 0x560150e66390 2024-06-10T01:09:39.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.456+0000 7fdd737fe700 1 --2- 172.21.15.37:0/1692315310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560150e66390 0x560150e81920 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c007ab0 tx=0x7fdd6c007ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66127833f4db0bfa server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:39.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.456+0000 7fdd58ff9700 1 -- 172.21.15.37:0/1692315310 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c019070 con 0x560150e66390 2024-06-10T01:09:39.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.456+0000 7fdd58ff9700 1 -- 172.21.15.37:0/1692315310 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd6c00f800 con 0x560150e66390 2024-06-10T01:09:39.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.456+0000 7fdd58ff9700 1 -- 172.21.15.37:0/1692315310 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c003d70 con 0x560150e66390 2024-06-10T01:09:39.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.457+0000 7fdd83c59b80 1 -- 172.21.15.37:0/1692315310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560150e66390 msgr2=0x560150e81920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:39.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.457+0000 7fdd83c59b80 1 --2- 172.21.15.37:0/1692315310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560150e66390 0x560150e81920 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c007ab0 tx=0x7fdd6c007ae0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.457+0000 7fdd83c59b80 1 -- 172.21.15.37:0/1692315310 shutdown_connections 2024-06-10T01:09:39.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.457+0000 7fdd83c59b80 1 --2- 172.21.15.37:0/1692315310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560150e66390 0x560150e81920 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.457+0000 7fdd83c59b80 1 --2- 172.21.15.37:0/1692315310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560150e60330 0x560150e68d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.457+0000 7fdd83c59b80 1 --2- 172.21.15.37:0/1692315310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560150e65a70 0x560150e66f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.457+0000 7fdd83c59b80 1 -- 172.21.15.37:0/1692315310 >> 172.21.15.37:0/1692315310 conn(0x560150d86cc0 msgr2=0x560150e66830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:39.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.457+0000 7fdd83c59b80 1 -- 172.21.15.37:0/1692315310 shutdown_connections 2024-06-10T01:09:39.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.457+0000 7fdd83c59b80 1 -- 172.21.15.37:0/1692315310 wait complete. 2024-06-10T01:09:39.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.457+0000 7fdd83c59b80 1 -- start start 2024-06-10T01:09:39.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.457+0000 7fdd83c59b80 1 Processor -- start 2024-06-10T01:09:39.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.458+0000 7fdd83c59b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560150e60330 0x560150e727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:39.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.458+0000 7fdd83c59b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560150e65a70 0x560150e811a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:39.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.458+0000 7fdd83c59b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560150e66390 0x560150e816e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:39.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.458+0000 7fdd83c59b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdd741aa750 con 0x560150e60330 2024-06-10T01:09:39.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.458+0000 7fdd83c59b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdd741abbc0 con 0x560150e65a70 2024-06-10T01:09:39.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.458+0000 7fdd83c59b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdd741aae20 con 0x560150e66390 2024-06-10T01:09:39.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.458+0000 7fdd83c59b80 10 client.0 _ll_register_callbacks cb 0x560150d47920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:39.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.458+0000 7fdd72ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560150e60330 0x560150e727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:39.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.458+0000 7fdd72ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560150e60330 0x560150e727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34570/0 (socket says 172.21.15.37:34570) 2024-06-10T01:09:39.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.458+0000 7fdd72ffd700 1 -- 172.21.15.37:0/1672675630 learned_addr learned my addr 172.21.15.37:0/1672675630 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:39.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.458+0000 7fdd737fe700 1 --2- 172.21.15.37:0/1672675630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560150e66390 0x560150e816e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:39.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.458+0000 7fdd727fc700 1 --2- 172.21.15.37:0/1672675630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560150e65a70 0x560150e811a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:39.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.458+0000 7fdd72ffd700 1 -- 172.21.15.37:0/1672675630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560150e66390 msgr2=0x560150e816e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:39.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.458+0000 7fdd72ffd700 1 --2- 172.21.15.37:0/1672675630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560150e66390 0x560150e816e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.458+0000 7fdd72ffd700 1 -- 172.21.15.37:0/1672675630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560150e65a70 msgr2=0x560150e811a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:39.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.458+0000 7fdd72ffd700 1 --2- 172.21.15.37:0/1672675630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560150e65a70 0x560150e811a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.458+0000 7fdd72ffd700 1 -- 172.21.15.37:0/1672675630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560150e81e30 con 0x560150e60330 2024-06-10T01:09:39.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.458+0000 7fdd72ffd700 1 --2- 172.21.15.37:0/1672675630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560150e60330 0x560150e727b0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fdd64097cb0 tx=0x7fdd6409c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9890a6981f12999f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:39.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.458+0000 7fdd56ff5700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:09:39.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.458+0000 7fdd56ff5700 1 -- 172.21.15.37:0/1672675630 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6409fc00 con 0x560150e60330 2024-06-10T01:09:39.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.459+0000 7fdd56ff5700 1 -- 172.21.15.37:0/1672675630 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fdd64099b50 con 0x560150e60330 2024-06-10T01:09:39.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.459+0000 7fdd83c59b80 1 -- 172.21.15.37:0/1672675630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd6c00f800 con 0x560150e60330 2024-06-10T01:09:39.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.459+0000 7fdd56ff5700 1 -- 172.21.15.37:0/1672675630 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdd640a44c0 con 0x560150e60330 2024-06-10T01:09:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.459+0000 7fdd83c59b80 10 client.17679 fetch_fsmap learned FSMap version 31 2024-06-10T01:09:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.459+0000 7fdd83c59b80 1 -- 172.21.15.37:0/1672675630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560150ee48e0 con 0x560150e60330 2024-06-10T01:09:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.459+0000 7fdd56ff5700 1 -- 172.21.15.37:0/1672675630 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd6c00f800 con 0x560150e60330 2024-06-10T01:09:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.459+0000 7fdd56ff5700 1 -- 172.21.15.37:0/1672675630 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fdd6409bab0 con 0x560150e60330 2024-06-10T01:09:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.459+0000 7fdd83c59b80 10 client.17679 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:09:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.459+0000 7fdd83c59b80 10 client.17679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.459+0000 7fdd83c59b80 20 client.17679 populate_metadata read hostname 'smithi037' 2024-06-10T01:09:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.459+0000 7fdd83c59b80 1 --2- 172.21.15.37:0/1672675630 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560150ee4fc0 0x560150ee73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.459+0000 7fdd83c59b80 4 client.17679 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:09:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.459+0000 7fdd83c59b80 1 -- 172.21.15.37:0/1672675630 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560150db5290 con 0x560150ee4fc0 2024-06-10T01:09:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.460+0000 7fdd727fc700 1 --2- 172.21.15.37:0/1672675630 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560150ee4fc0 0x560150ee73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.460+0000 7fdd727fc700 1 --2- 172.21.15.37:0/1672675630 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560150ee4fc0 0x560150ee73b0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce05e8a393c44d94 server_cookie=f86b47dc34cc6c31 in_seq=0 out_seq=0 2024-06-10T01:09:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.460+0000 7fdd56ff5700 10 client.17679 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.460+0000 7fdd56ff5700 1 -- 172.21.15.37:0/1672675630 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560150db5290 con 0x560150ee4fc0 2024-06-10T01:09:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.460+0000 7fdd56ff5700 10 client.17679 handle_command_reply: tid=0 2024-06-10T01:09:39.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.460+0000 7fdd83c59b80 1 client.17679 shutdown 2024-06-10T01:09:39.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.460+0000 7fdd83c59b80 1 -- 172.21.15.37:0/1672675630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560150e60330 msgr2=0x560150e727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:39.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.460+0000 7fdd83c59b80 1 --2- 172.21.15.37:0/1672675630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560150e60330 0x560150e727b0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fdd64097cb0 tx=0x7fdd6409c070 comp rx=0 tx=0).stop 2024-06-10T01:09:39.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.460+0000 7fdd83c59b80 1 -- 172.21.15.37:0/1672675630 shutdown_connections 2024-06-10T01:09:39.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.460+0000 7fdd83c59b80 1 --2- 172.21.15.37:0/1672675630 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560150ee4fc0 0x560150ee73b0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.460+0000 7fdd83c59b80 1 --2- 172.21.15.37:0/1672675630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560150e66390 0x560150e816e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.460+0000 7fdd83c59b80 1 --2- 172.21.15.37:0/1672675630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560150e60330 0x560150e727b0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.460+0000 7fdd83c59b80 1 --2- 172.21.15.37:0/1672675630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560150e65a70 0x560150e811a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.460+0000 7fdd56ff5700 0 client.17679 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:39.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.460+0000 7fdd83c59b80 1 -- 172.21.15.37:0/1672675630 >> 172.21.15.37:0/1672675630 conn(0x560150d86cc0 msgr2=0x560150e4fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:39.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.460+0000 7fdd83c59b80 1 -- 172.21.15.37:0/1672675630 shutdown_connections 2024-06-10T01:09:39.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.461+0000 7fdd83c59b80 1 -- 172.21.15.37:0/1672675630 wait complete. 2024-06-10T01:09:39.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.461+0000 7fdd83c59b80 20 client.17679 trim_cache size 0 max 16384 2024-06-10T01:09:39.462 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:09:39.462 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-06-10T01:09:39.462 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-06-10T01:09:39.462 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:09:39.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.461+0000 7fdd793d6700 1 -- 172.21.15.37:0/2364824460 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdd5c0a2ca0 msgr2=0x7fdd5c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:39.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.461+0000 7fdd793d6700 1 --2- 172.21.15.37:0/2364824460 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdd5c0a2ca0 0x7fdd5c0a5160 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fdd68006010 tx=0x7fdd6800b540 comp rx=0 tx=0).stop 2024-06-10T01:09:39.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.461+0000 7fdd793d6700 1 -- 172.21.15.37:0/2364824460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd74101380 msgr2=0x7fdd74199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:39.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.461+0000 7fdd793d6700 1 --2- 172.21.15.37:0/2364824460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd74101380 0x7fdd74199950 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400c9b0 tx=0x7fdd6400cd70 comp rx=0 tx=0).stop 2024-06-10T01:09:39.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.461+0000 7fdd793d6700 1 -- 172.21.15.37:0/2364824460 shutdown_connections 2024-06-10T01:09:39.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.461+0000 7fdd793d6700 1 --2- 172.21.15.37:0/2364824460 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fdd5c0a2ca0 0x7fdd5c0a5160 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.461+0000 7fdd793d6700 1 --2- 172.21.15.37:0/2364824460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd741073c0 0x7fdd741a49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.461+0000 7fdd793d6700 1 --2- 172.21.15.37:0/2364824460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd74101380 0x7fdd74199950 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.461+0000 7fdd793d6700 1 --2- 172.21.15.37:0/2364824460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd74101d40 0x7fdd74199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:39.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.461+0000 7fdd793d6700 1 -- 172.21.15.37:0/2364824460 >> 172.21.15.37:0/2364824460 conn(0x7fdd74078330 msgr2=0x7fdd740ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:39.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.461+0000 7fdd793d6700 1 -- 172.21.15.37:0/2364824460 shutdown_connections 2024-06-10T01:09:39.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:39.461+0000 7fdd793d6700 1 -- 172.21.15.37:0/2364824460 wait complete. 2024-06-10T01:09:40.136 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:09:40.136 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:09:40.388 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:09:40.390 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:40 smithi037 ceph-mon[32192]: pgmap v820: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 28 op/s 2024-06-10T01:09:40.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:40 smithi167 ceph-mon[36917]: pgmap v820: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 28 op/s 2024-06-10T01:09:40.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:40 smithi094 ceph-mon[36708]: pgmap v820: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 28 op/s 2024-06-10T01:09:41.497 INFO:tasks.workunit.client.0.smithi037.stdout:ok 2024-06-10T01:09:41.502 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-10T01:09:41.502 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-10T01:09:42.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.118+0000 7f2cc9466700 1 -- 172.21.15.37:0/1287707385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cc4107870 msgr2=0x7f2cc410f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:42.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.118+0000 7f2cc9466700 1 --2- 172.21.15.37:0/1287707385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cc4107870 0x7f2cc410f3f0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc0099c0 tx=0x7f2cbc009cd0 comp rx=0 tx=0).stop 2024-06-10T01:09:42.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.118+0000 7f2cc9466700 1 -- 172.21.15.37:0/1287707385 shutdown_connections 2024-06-10T01:09:42.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.118+0000 7f2cc9466700 1 --2- 172.21.15.37:0/1287707385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cc4107870 0x7f2cc410f3f0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:42.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.118+0000 7f2cc9466700 1 --2- 172.21.15.37:0/1287707385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cc41027a0 0x7f2cc4102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:42.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.118+0000 7f2cc9466700 1 --2- 172.21.15.37:0/1287707385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cc4103150 0x7f2cc4107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:42.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.118+0000 7f2cc9466700 1 -- 172.21.15.37:0/1287707385 >> 172.21.15.37:0/1287707385 conn(0x7f2cc4079520 msgr2=0x7f2cc4079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:42.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.118+0000 7f2cc9466700 1 -- 172.21.15.37:0/1287707385 shutdown_connections 2024-06-10T01:09:42.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.118+0000 7f2cc9466700 1 -- 172.21.15.37:0/1287707385 wait complete. 2024-06-10T01:09:42.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.119+0000 7f2cc9466700 1 Processor -- start 2024-06-10T01:09:42.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.119+0000 7f2cc9466700 1 -- start start 2024-06-10T01:09:42.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.119+0000 7f2cc9466700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cc41027a0 0x7f2cc4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:42.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.119+0000 7f2cc9466700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cc4103150 0x7f2cc4074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:42.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.119+0000 7f2cc9466700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cc4107870 0x7f2cc4075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:42.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.119+0000 7f2cc9466700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc4112790 con 0x7f2cc4103150 2024-06-10T01:09:42.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.119+0000 7f2cc9466700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc4112610 con 0x7f2cc41027a0 2024-06-10T01:09:42.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.119+0000 7f2cc9466700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc4112910 con 0x7f2cc4107870 2024-06-10T01:09:42.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.120+0000 7f2cc27fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cc4103150 0x7f2cc4074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:42.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.120+0000 7f2cc27fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cc4103150 0x7f2cc4074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53224/0 (socket says 172.21.15.37:53224) 2024-06-10T01:09:42.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.120+0000 7f2cc27fc700 1 -- 172.21.15.37:0/2104094774 learned_addr learned my addr 172.21.15.37:0/2104094774 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:42.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.120+0000 7f2cc2ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cc41027a0 0x7f2cc4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:42.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.120+0000 7f2cc37fe700 1 --2- 172.21.15.37:0/2104094774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cc4107870 0x7f2cc4075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:42.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.120+0000 7f2cc2ffd700 1 -- 172.21.15.37:0/2104094774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cc4107870 msgr2=0x7f2cc4075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:42.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.120+0000 7f2cc2ffd700 1 --2- 172.21.15.37:0/2104094774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cc4107870 0x7f2cc4075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:42.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.120+0000 7f2cc2ffd700 1 -- 172.21.15.37:0/2104094774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cc4103150 msgr2=0x7f2cc4074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:42.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.120+0000 7f2cc2ffd700 1 --2- 172.21.15.37:0/2104094774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cc4103150 0x7f2cc4074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:42.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.120+0000 7f2cc2ffd700 1 -- 172.21.15.37:0/2104094774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc41a7060 con 0x7f2cc41027a0 2024-06-10T01:09:42.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.120+0000 7f2cc27fc700 1 --2- 172.21.15.37:0/2104094774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cc4103150 0x7f2cc4074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:42.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.120+0000 7f2cc2ffd700 1 --2- 172.21.15.37:0/2104094774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cc41027a0 0x7f2cc4074790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4007ec0 tx=0x7f2cb400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:42.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.121+0000 7f2cbbfff700 1 -- 172.21.15.37:0/2104094774 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4004db0 con 0x7f2cc41027a0 2024-06-10T01:09:42.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.121+0000 7f2cc9466700 1 -- 172.21.15.37:0/2104094774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc41a7350 con 0x7f2cc41027a0 2024-06-10T01:09:42.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.121+0000 7f2cbbfff700 1 -- 172.21.15.37:0/2104094774 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cb4005020 con 0x7f2cc41027a0 2024-06-10T01:09:42.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.121+0000 7f2cbbfff700 1 -- 172.21.15.37:0/2104094774 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4015660 con 0x7f2cc41027a0 2024-06-10T01:09:42.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.121+0000 7f2cc9466700 1 -- 172.21.15.37:0/2104094774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc41a78e0 con 0x7f2cc41027a0 2024-06-10T01:09:42.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.122+0000 7f2cc9466700 1 -- 172.21.15.37:0/2104094774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cc4069180 con 0x7f2cc41027a0 2024-06-10T01:09:42.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.123+0000 7f2cbbfff700 1 -- 172.21.15.37:0/2104094774 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2cb4005540 con 0x7f2cc41027a0 2024-06-10T01:09:42.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.124+0000 7f2cbbfff700 1 --2- 172.21.15.37:0/2104094774 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2cb00a2ca0 0x7f2cb00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:42.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.124+0000 7f2cc27fc700 1 --2- 172.21.15.37:0/2104094774 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2cb00a2ca0 0x7f2cb00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:42.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.124+0000 7f2cbbfff700 1 -- 172.21.15.37:0/2104094774 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f2cb40cf530 con 0x7f2cc41027a0 2024-06-10T01:09:42.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.124+0000 7f2cc27fc700 1 --2- 172.21.15.37:0/2104094774 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2cb00a2ca0 0x7f2cb00a5160 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4101f80 tx=0x7f2cac00a400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:42.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.128+0000 7f2cbbfff700 1 -- 172.21.15.37:0/2104094774 <== mon.2 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2cb4010070 con 0x7f2cc41027a0 2024-06-10T01:09:42.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.382+0000 7f2cc9466700 1 -- 172.21.15.37:0/2104094774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2cc402dca0 con 0x7f2cc41027a0 2024-06-10T01:09:42.378 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:42 smithi037 ceph-mon[32192]: pgmap v821: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.1 KiB/s wr, 13 op/s 2024-06-10T01:09:42.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.383+0000 7f2cbbfff700 1 -- 172.21.15.37:0/2104094774 <== mon.2 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11868 (secure 0 0 0) 0x7f2cb409c040 con 0x7f2cc41027a0 2024-06-10T01:09:42.380 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:09:42.380 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":26554,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2107277223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2107277223},{"type":"v1","addr":"172.21.15.167:6835","nonce":2107277223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T01:06:09.604504+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24472,24451,24458]},"id":1}]} 2024-06-10T01:09:42.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.385+0000 7f2cc9466700 1 -- 172.21.15.37:0/2104094774 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2cb00a2ca0 msgr2=0x7f2cb00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:42.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.385+0000 7f2cc9466700 1 --2- 172.21.15.37:0/2104094774 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2cb00a2ca0 0x7f2cb00a5160 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4101f80 tx=0x7f2cac00a400 comp rx=0 tx=0).stop 2024-06-10T01:09:42.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.385+0000 7f2cc9466700 1 -- 172.21.15.37:0/2104094774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cc41027a0 msgr2=0x7f2cc4074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:42.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.385+0000 7f2cc9466700 1 --2- 172.21.15.37:0/2104094774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cc41027a0 0x7f2cc4074790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4007ec0 tx=0x7f2cb400d3b0 comp rx=0 tx=0).stop 2024-06-10T01:09:42.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.385+0000 7f2cc9466700 1 -- 172.21.15.37:0/2104094774 shutdown_connections 2024-06-10T01:09:42.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.385+0000 7f2cc9466700 1 --2- 172.21.15.37:0/2104094774 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2cb00a2ca0 0x7f2cb00a5160 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:42.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.385+0000 7f2cc9466700 1 --2- 172.21.15.37:0/2104094774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cc4107870 0x7f2cc4075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:42.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.385+0000 7f2cc9466700 1 --2- 172.21.15.37:0/2104094774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cc41027a0 0x7f2cc4074790 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:42.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.385+0000 7f2cc9466700 1 --2- 172.21.15.37:0/2104094774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cc4103150 0x7f2cc4074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:42.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.385+0000 7f2cc9466700 1 -- 172.21.15.37:0/2104094774 >> 172.21.15.37:0/2104094774 conn(0x7f2cc4079520 msgr2=0x7f2cc41004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:42.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.385+0000 7f2cc9466700 1 -- 172.21.15.37:0/2104094774 shutdown_connections 2024-06-10T01:09:42.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:42.385+0000 7f2cc9466700 1 -- 172.21.15.37:0/2104094774 wait complete. 2024-06-10T01:09:42.384 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 31 2024-06-10T01:09:42.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:42 smithi167 ceph-mon[36917]: pgmap v821: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.1 KiB/s wr, 13 op/s 2024-06-10T01:09:42.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:42 smithi094 ceph-mon[36708]: pgmap v821: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.1 KiB/s wr, 13 op/s 2024-06-10T01:09:43.033 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:09:43.292 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:09:43.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:43 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2104094774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:09:43.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:43 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2104094774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:09:43.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:43 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2104094774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:09:44.390 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:44 smithi037 ceph-mon[32192]: pgmap v822: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.7 KiB/s wr, 15 op/s 2024-06-10T01:09:44.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:44 smithi167 ceph-mon[36917]: pgmap v822: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.7 KiB/s wr, 15 op/s 2024-06-10T01:09:44.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:44 smithi094 ceph-mon[36708]: pgmap v822: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.7 KiB/s wr, 15 op/s 2024-06-10T01:09:45.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.092+0000 7f80bcd55700 1 -- 172.21.15.37:0/2508146068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b806b620 msgr2=0x7f80b810f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:45.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.092+0000 7f80bcd55700 1 --2- 172.21.15.37:0/2508146068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b806b620 0x7f80b810f350 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f80b000b0d0 tx=0x7f80b000b3e0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.092+0000 7f80bcd55700 1 -- 172.21.15.37:0/2508146068 shutdown_connections 2024-06-10T01:09:45.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.092+0000 7f80bcd55700 1 --2- 172.21.15.37:0/2508146068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b806b620 0x7f80b810f350 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.092+0000 7f80bcd55700 1 --2- 172.21.15.37:0/2508146068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80b8108580 0x7f80b8108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.092+0000 7f80bcd55700 1 --2- 172.21.15.37:0/2508146068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80b806ac60 0x7f80b806b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.092+0000 7f80bcd55700 1 -- 172.21.15.37:0/2508146068 >> 172.21.15.37:0/2508146068 conn(0x7f80b8078430 msgr2=0x7f80b8078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:45.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.092+0000 7f80bcd55700 1 -- 172.21.15.37:0/2508146068 shutdown_connections 2024-06-10T01:09:45.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.092+0000 7f80bcd55700 1 -- 172.21.15.37:0/2508146068 wait complete. 2024-06-10T01:09:45.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.093+0000 7f80bcd55700 1 Processor -- start 2024-06-10T01:09:45.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.093+0000 7f80bcd55700 1 -- start start 2024-06-10T01:09:45.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.093+0000 7f80bcd55700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80b806ac60 0x7f80b819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:45.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.093+0000 7f80bcd55700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80b806b620 0x7f80b819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:45.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.093+0000 7f80bcd55700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b8108580 0x7f80b81989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:45.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.093+0000 7f80bcd55700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f80b81127f0 con 0x7f80b8108580 2024-06-10T01:09:45.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.093+0000 7f80bcd55700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f80b8112670 con 0x7f80b806b620 2024-06-10T01:09:45.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.093+0000 7f80bcd55700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f80b8112970 con 0x7f80b806ac60 2024-06-10T01:09:45.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.093+0000 7f80b6d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b8108580 0x7f80b81989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:45.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.093+0000 7f80b6d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b8108580 0x7f80b81989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53246/0 (socket says 172.21.15.37:53246) 2024-06-10T01:09:45.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.093+0000 7f80b6d9d700 1 -- 172.21.15.37:0/574006265 learned_addr learned my addr 172.21.15.37:0/574006265 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:45.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.093+0000 7f80b5d9b700 1 --2- 172.21.15.37:0/574006265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80b806b620 0x7f80b819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:45.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.093+0000 7f80b659c700 1 --2- 172.21.15.37:0/574006265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80b806ac60 0x7f80b819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:45.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.094+0000 7f80b6d9d700 1 -- 172.21.15.37:0/574006265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80b806ac60 msgr2=0x7f80b819e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:45.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.094+0000 7f80b6d9d700 1 --2- 172.21.15.37:0/574006265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80b806ac60 0x7f80b819e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.094+0000 7f80b6d9d700 1 -- 172.21.15.37:0/574006265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80b806b620 msgr2=0x7f80b819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:45.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.094+0000 7f80b6d9d700 1 --2- 172.21.15.37:0/574006265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80b806b620 0x7f80b819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.094+0000 7f80b6d9d700 1 -- 172.21.15.37:0/574006265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b8199110 con 0x7f80b8108580 2024-06-10T01:09:45.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.094+0000 7f80b659c700 1 --2- 172.21.15.37:0/574006265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80b806ac60 0x7f80b819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:09:45.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.094+0000 7f80b6d9d700 1 --2- 172.21.15.37:0/574006265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b8108580 0x7f80b81989b0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f80b0006070 tx=0x7f80b0007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:45.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.094+0000 7f80a77fe700 1 -- 172.21.15.37:0/574006265 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b0003c00 con 0x7f80b8108580 2024-06-10T01:09:45.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.094+0000 7f80bcd55700 1 -- 172.21.15.37:0/574006265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b81993a0 con 0x7f80b8108580 2024-06-10T01:09:45.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.094+0000 7f80a77fe700 1 -- 172.21.15.37:0/574006265 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80b0003da0 con 0x7f80b8108580 2024-06-10T01:09:45.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.094+0000 7f80a77fe700 1 -- 172.21.15.37:0/574006265 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b001b960 con 0x7f80b8108580 2024-06-10T01:09:45.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.094+0000 7f80bcd55700 1 -- 172.21.15.37:0/574006265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b81a56c0 con 0x7f80b8108580 2024-06-10T01:09:45.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.096+0000 7f80a77fe700 1 -- 172.21.15.37:0/574006265 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f80b0005670 con 0x7f80b8108580 2024-06-10T01:09:45.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.097+0000 7f80a77fe700 1 --2- 172.21.15.37:0/574006265 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f80a00a2c20 0x7f80a00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:45.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.097+0000 7f80b659c700 1 --2- 172.21.15.37:0/574006265 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f80a00a2c20 0x7f80a00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:45.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.097+0000 7f80a77fe700 1 -- 172.21.15.37:0/574006265 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f80b000f020 con 0x7f80b8108580 2024-06-10T01:09:45.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.098+0000 7f80b659c700 1 --2- 172.21.15.37:0/574006265 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f80a00a2c20 0x7f80a00a50e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f80a8005fd0 tx=0x7f80a8005f60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:45.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.098+0000 7f80c75d8b80 1 Processor -- start 2024-06-10T01:09:45.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.098+0000 7f80c75d8b80 1 -- start start 2024-06-10T01:09:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.098+0000 7f80c75d8b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f165e1f60 0x562f165e2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.098+0000 7f80c75d8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f165e2880 0x562f165dc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.098+0000 7f80c75d8b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f165dc7a0 0x562f165dcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.098+0000 7f80c75d8b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562f164cef60 con 0x562f165dc7a0 2024-06-10T01:09:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.098+0000 7f80c75d8b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562f1652d850 con 0x562f165e1f60 2024-06-10T01:09:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.098+0000 7f80c75d8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562f16495460 con 0x562f165e2880 2024-06-10T01:09:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.099+0000 7f80b6d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f165dc7a0 0x562f165dcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.099+0000 7f80b6d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f165dc7a0 0x562f165dcbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53256/0 (socket says 172.21.15.37:53256) 2024-06-10T01:09:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.099+0000 7f80b6d9d700 1 -- 172.21.15.37:0/2963901311 learned_addr learned my addr 172.21.15.37:0/2963901311 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.099+0000 7f80b659c700 1 --2- 172.21.15.37:0/2963901311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f165e1f60 0x562f165e2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.099+0000 7f80b5d9b700 1 --2- 172.21.15.37:0/2963901311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f165e2880 0x562f165dc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.099+0000 7f80b6d9d700 1 -- 172.21.15.37:0/2963901311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f165e2880 msgr2=0x562f165dc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.099+0000 7f80b6d9d700 1 --2- 172.21.15.37:0/2963901311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f165e2880 0x562f165dc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.099+0000 7f80b6d9d700 1 -- 172.21.15.37:0/2963901311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f165e1f60 msgr2=0x562f165e2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.099+0000 7f80b6d9d700 1 --2- 172.21.15.37:0/2963901311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f165e1f60 0x562f165e2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.099+0000 7f80b6d9d700 1 -- 172.21.15.37:0/2963901311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f1647c8d0 con 0x562f165dc7a0 2024-06-10T01:09:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.099+0000 7f80b5d9b700 1 --2- 172.21.15.37:0/2963901311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f165e2880 0x562f165dc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:09:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.099+0000 7f80b6d9d700 1 --2- 172.21.15.37:0/2963901311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f165dc7a0 0x562f165dcbc0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f80b000d010 tx=0x7f80b00099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b9919a872992e7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.099+0000 7f80a4ff9700 1 -- 172.21.15.37:0/2963901311 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b00a0cb0 con 0x562f165dc7a0 2024-06-10T01:09:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.099+0000 7f80a4ff9700 1 -- 172.21.15.37:0/2963901311 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80b00a54e0 con 0x562f165dc7a0 2024-06-10T01:09:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.099+0000 7f80a4ff9700 1 -- 172.21.15.37:0/2963901311 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b00a30c0 con 0x562f165dc7a0 2024-06-10T01:09:45.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.100+0000 7f80c75d8b80 1 -- 172.21.15.37:0/2963901311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f165dc7a0 msgr2=0x562f165dcbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:45.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.100+0000 7f80c75d8b80 1 --2- 172.21.15.37:0/2963901311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f165dc7a0 0x562f165dcbc0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f80b000d010 tx=0x7f80b00099e0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.100+0000 7f80c75d8b80 1 -- 172.21.15.37:0/2963901311 shutdown_connections 2024-06-10T01:09:45.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.100+0000 7f80c75d8b80 1 --2- 172.21.15.37:0/2963901311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f165dc7a0 0x562f165dcbc0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.100+0000 7f80c75d8b80 1 --2- 172.21.15.37:0/2963901311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f165e1f60 0x562f165e2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.100+0000 7f80c75d8b80 1 --2- 172.21.15.37:0/2963901311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f165e2880 0x562f165dc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.100+0000 7f80c75d8b80 1 -- 172.21.15.37:0/2963901311 >> 172.21.15.37:0/2963901311 conn(0x562f16509080 msgr2=0x562f165d7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:45.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.100+0000 7f80c75d8b80 1 -- 172.21.15.37:0/2963901311 shutdown_connections 2024-06-10T01:09:45.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.100+0000 7f80c75d8b80 1 -- 172.21.15.37:0/2963901311 wait complete. 2024-06-10T01:09:45.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.101+0000 7f80c75d8b80 1 -- start start 2024-06-10T01:09:45.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.101+0000 7f80c75d8b80 1 Processor -- start 2024-06-10T01:09:45.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.101+0000 7f80c75d8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f165dc7a0 0x562f165e8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:45.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.101+0000 7f80c75d8b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f165e1f60 0x562f165e5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:45.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.101+0000 7f80c75d8b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f165e2880 0x562f165e5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:45.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.101+0000 7f80c75d8b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562f164cdfc0 con 0x562f165e1f60 2024-06-10T01:09:45.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.101+0000 7f80c75d8b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562f16529810 con 0x562f165e2880 2024-06-10T01:09:45.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.101+0000 7f80c75d8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562f165e9010 con 0x562f165dc7a0 2024-06-10T01:09:45.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.101+0000 7f80c75d8b80 10 client.0 _ll_register_callbacks cb 0x562f164c35b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:45.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.101+0000 7f80b5d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f165e1f60 0x562f165e5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:45.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.101+0000 7f80b5d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f165e1f60 0x562f165e5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53258/0 (socket says 172.21.15.37:53258) 2024-06-10T01:09:45.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.101+0000 7f80b5d9b700 1 -- 172.21.15.37:0/2987879029 learned_addr learned my addr 172.21.15.37:0/2987879029 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:45.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.101+0000 7f80b659c700 1 --2- 172.21.15.37:0/2987879029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f165dc7a0 0x562f165e8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:45.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.101+0000 7f80b6d9d700 1 --2- 172.21.15.37:0/2987879029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f165e2880 0x562f165e5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:45.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.101+0000 7f80b5d9b700 1 -- 172.21.15.37:0/2987879029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f165dc7a0 msgr2=0x562f165e8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:45.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.101+0000 7f80b5d9b700 1 --2- 172.21.15.37:0/2987879029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f165dc7a0 0x562f165e8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.101+0000 7f80b5d9b700 1 -- 172.21.15.37:0/2987879029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f165e2880 msgr2=0x562f165e5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:45.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.101+0000 7f80b5d9b700 1 --2- 172.21.15.37:0/2987879029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f165e2880 0x562f165e5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.102+0000 7f80b5d9b700 1 -- 172.21.15.37:0/2987879029 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f164f3750 con 0x562f165e1f60 2024-06-10T01:09:45.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.102+0000 7f80b5d9b700 1 --2- 172.21.15.37:0/2987879029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f165e1f60 0x562f165e5880 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f80ac012740 tx=0x7f80ac012a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95c4fa9d176d70f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:45.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.102+0000 7f809dffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:09:45.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.102+0000 7f809dffb700 1 -- 172.21.15.37:0/2987879029 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac004050 con 0x562f165e1f60 2024-06-10T01:09:45.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.102+0000 7f809dffb700 1 -- 172.21.15.37:0/2987879029 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f80ac0041f0 con 0x562f165e1f60 2024-06-10T01:09:45.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.102+0000 7f80c75d8b80 1 -- 172.21.15.37:0/2987879029 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80b00a54e0 con 0x562f165e1f60 2024-06-10T01:09:45.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.102+0000 7f809dffb700 1 -- 172.21.15.37:0/2987879029 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f80ac025050 con 0x562f165e1f60 2024-06-10T01:09:45.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.102+0000 7f80c75d8b80 10 client.17697 fetch_fsmap learned FSMap version 31 2024-06-10T01:09:45.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.102+0000 7f80c75d8b80 1 -- 172.21.15.37:0/2987879029 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562f165f3780 con 0x562f165e1f60 2024-06-10T01:09:45.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.102+0000 7f809dffb700 1 -- 172.21.15.37:0/2987879029 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80ac0043f0 con 0x562f165e1f60 2024-06-10T01:09:45.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.102+0000 7f809dffb700 1 -- 172.21.15.37:0/2987879029 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f80ac00f8a0 con 0x562f165e1f60 2024-06-10T01:09:45.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.102+0000 7f80c75d8b80 10 client.17697 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:09:45.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.102+0000 7f80c75d8b80 10 client.17697 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-10T01:09:45.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.103+0000 7f80c75d8b80 20 client.17697 populate_metadata read hostname 'smithi037' 2024-06-10T01:09:45.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.103+0000 7f80c75d8b80 1 --2- 172.21.15.37:0/2987879029 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x562f165f5e10 0x562f165f8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:45.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.103+0000 7f80c75d8b80 4 client.17697 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:09:45.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.103+0000 7f80c75d8b80 1 -- 172.21.15.37:0/2987879029 --> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f80b00a54e0 con 0x562f165f5e10 2024-06-10T01:09:45.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.103+0000 7f80b659c700 1 --2- 172.21.15.37:0/2987879029 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x562f165f5e10 0x562f165f8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:45.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.103+0000 7f80b659c700 1 --2- 172.21.15.37:0/2987879029 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x562f165f5e10 0x562f165f8200 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a0d966ff3e8ef4e4 server_cookie=f44a587128c33061 in_seq=0 out_seq=0 2024-06-10T01:09:45.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.103+0000 7f809dffb700 10 client.17697 ms_handle_connect on v2:172.21.15.167:6836/2720148065 2024-06-10T01:09:45.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.104+0000 7f809dffb700 1 -- 172.21.15.37:0/2987879029 <== mds.2 v2:172.21.15.167:6836/2720148065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f80b00a54e0 con 0x562f165f5e10 2024-06-10T01:09:45.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.104+0000 7f809dffb700 10 client.17697 handle_command_reply: tid=0 2024-06-10T01:09:45.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.104+0000 7f80c75d8b80 1 client.17697 shutdown 2024-06-10T01:09:45.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.104+0000 7f80c75d8b80 1 -- 172.21.15.37:0/2987879029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f165e1f60 msgr2=0x562f165e5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:45.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.104+0000 7f80c75d8b80 1 --2- 172.21.15.37:0/2987879029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f165e1f60 0x562f165e5880 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f80ac012740 tx=0x7f80ac012a50 comp rx=0 tx=0).stop 2024-06-10T01:09:45.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.104+0000 7f80c75d8b80 1 -- 172.21.15.37:0/2987879029 shutdown_connections 2024-06-10T01:09:45.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.104+0000 7f80c75d8b80 1 --2- 172.21.15.37:0/2987879029 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x562f165f5e10 0x562f165f8200 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.104+0000 7f80c75d8b80 1 --2- 172.21.15.37:0/2987879029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f165e2880 0x562f165e5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.104+0000 7f80c75d8b80 1 --2- 172.21.15.37:0/2987879029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f165dc7a0 0x562f165e8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.104+0000 7f809dffb700 0 client.17697 ms_handle_reset on v2:172.21.15.167:6836/2720148065 2024-06-10T01:09:45.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.104+0000 7f80c75d8b80 1 --2- 172.21.15.37:0/2987879029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f165e1f60 0x562f165e5880 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.104+0000 7f80c75d8b80 1 -- 172.21.15.37:0/2987879029 >> 172.21.15.37:0/2987879029 conn(0x562f16509080 msgr2=0x562f165d6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:45.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.104+0000 7f80c75d8b80 1 -- 172.21.15.37:0/2987879029 shutdown_connections 2024-06-10T01:09:45.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.104+0000 7f80c75d8b80 1 -- 172.21.15.37:0/2987879029 wait complete. 2024-06-10T01:09:45.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.104+0000 7f80c75d8b80 20 client.17697 trim_cache size 0 max 16384 2024-06-10T01:09:45.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.120+0000 7f80c75d8b80 1 Processor -- start 2024-06-10T01:09:45.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.120+0000 7f80c75d8b80 1 -- start start 2024-06-10T01:09:45.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.120+0000 7f80c75d8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f165dc7a0 0x562f165dc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:45.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.120+0000 7f80c75d8b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f165e1f60 0x562f165e34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:45.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.120+0000 7f80c75d8b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f165e2880 0x562f165e47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:45.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.120+0000 7f80c75d8b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f80b81b0c50 con 0x562f165e1f60 2024-06-10T01:09:45.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.120+0000 7f80c75d8b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f80b81ae4e0 con 0x562f165e2880 2024-06-10T01:09:45.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.120+0000 7f80c75d8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f80b81ac330 con 0x562f165dc7a0 2024-06-10T01:09:45.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.120+0000 7f80b5d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f165e1f60 0x562f165e34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:45.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.120+0000 7f80b5d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f165e1f60 0x562f165e34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53268/0 (socket says 172.21.15.37:53268) 2024-06-10T01:09:45.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.120+0000 7f80b5d9b700 1 -- 172.21.15.37:0/2261833288 learned_addr learned my addr 172.21.15.37:0/2261833288 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:45.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.120+0000 7f80b659c700 1 --2- 172.21.15.37:0/2261833288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f165dc7a0 0x562f165dc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:45.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.120+0000 7f80b6d9d700 1 --2- 172.21.15.37:0/2261833288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f165e2880 0x562f165e47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:45.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.120+0000 7f80b5d9b700 1 -- 172.21.15.37:0/2261833288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f165dc7a0 msgr2=0x562f165dc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:45.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.120+0000 7f80b5d9b700 1 --2- 172.21.15.37:0/2261833288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f165dc7a0 0x562f165dc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.120+0000 7f80b5d9b700 1 -- 172.21.15.37:0/2261833288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f165e2880 msgr2=0x562f165e47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:45.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.120+0000 7f80b5d9b700 1 --2- 172.21.15.37:0/2261833288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f165e2880 0x562f165e47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.120+0000 7f80b5d9b700 1 -- 172.21.15.37:0/2261833288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f165dcb80 con 0x562f165e1f60 2024-06-10T01:09:45.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.120+0000 7f80b659c700 1 --2- 172.21.15.37:0/2261833288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f165dc7a0 0x562f165dc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:09:45.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.120+0000 7f80b6d9d700 1 --2- 172.21.15.37:0/2261833288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f165e2880 0x562f165e47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:09:45.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.121+0000 7f80b5d9b700 1 --2- 172.21.15.37:0/2261833288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f165e1f60 0x562f165e34f0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00a010 tx=0x7f80ac0093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b04013d7bd7913 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:45.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.121+0000 7f809ffff700 1 -- 172.21.15.37:0/2261833288 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac02d020 con 0x562f165e1f60 2024-06-10T01:09:45.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.121+0000 7f809ffff700 1 -- 172.21.15.37:0/2261833288 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80ac00fa20 con 0x562f165e1f60 2024-06-10T01:09:45.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.121+0000 7f809ffff700 1 -- 172.21.15.37:0/2261833288 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac003680 con 0x562f165e1f60 2024-06-10T01:09:45.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.121+0000 7f80c75d8b80 1 -- 172.21.15.37:0/2261833288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f165e1f60 msgr2=0x562f165e34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:45.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.121+0000 7f80c75d8b80 1 --2- 172.21.15.37:0/2261833288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f165e1f60 0x562f165e34f0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00a010 tx=0x7f80ac0093d0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.121+0000 7f80c75d8b80 1 -- 172.21.15.37:0/2261833288 shutdown_connections 2024-06-10T01:09:45.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.121+0000 7f80c75d8b80 1 --2- 172.21.15.37:0/2261833288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f165e2880 0x562f165e47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.121+0000 7f80c75d8b80 1 --2- 172.21.15.37:0/2261833288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f165dc7a0 0x562f165dc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.121+0000 7f80c75d8b80 1 --2- 172.21.15.37:0/2261833288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f165e1f60 0x562f165e34f0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.121+0000 7f80c75d8b80 1 -- 172.21.15.37:0/2261833288 >> 172.21.15.37:0/2261833288 conn(0x562f16509080 msgr2=0x562f165e70b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:45.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.121+0000 7f80c75d8b80 1 -- 172.21.15.37:0/2261833288 shutdown_connections 2024-06-10T01:09:45.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.122+0000 7f80c75d8b80 1 -- 172.21.15.37:0/2261833288 wait complete. 2024-06-10T01:09:45.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.122+0000 7f80c75d8b80 1 -- start start 2024-06-10T01:09:45.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.122+0000 7f80c75d8b80 1 Processor -- start 2024-06-10T01:09:45.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.122+0000 7f80c75d8b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f165dc7a0 0x562f165fd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:45.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.122+0000 7f80c75d8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f165e1f60 0x562f165fddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:45.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.122+0000 7f80c75d8b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f165e2880 0x562f165fe310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:45.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.122+0000 7f80c75d8b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f80b81ac9a0 con 0x562f165e2880 2024-06-10T01:09:45.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.122+0000 7f80c75d8b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f80b81ade10 con 0x562f165dc7a0 2024-06-10T01:09:45.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.122+0000 7f80c75d8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f80b81ad070 con 0x562f165e1f60 2024-06-10T01:09:45.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.122+0000 7f80c75d8b80 10 client.0 _ll_register_callbacks cb 0x562f164c35b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:45.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.122+0000 7f80b6d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f165e2880 0x562f165fe310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:45.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.122+0000 7f80b6d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f165e2880 0x562f165fe310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53276/0 (socket says 172.21.15.37:53276) 2024-06-10T01:09:45.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.122+0000 7f80b6d9d700 1 -- 172.21.15.37:0/1615373431 learned_addr learned my addr 172.21.15.37:0/1615373431 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:45.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.123+0000 7f80b659c700 1 --2- 172.21.15.37:0/1615373431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f165dc7a0 0x562f165fd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.123+0000 7f80b5d9b700 1 --2- 172.21.15.37:0/1615373431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f165e1f60 0x562f165fddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.123+0000 7f80b6d9d700 1 -- 172.21.15.37:0/1615373431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f165e1f60 msgr2=0x562f165fddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.123+0000 7f80b6d9d700 1 --2- 172.21.15.37:0/1615373431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f165e1f60 0x562f165fddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.123+0000 7f80b6d9d700 1 -- 172.21.15.37:0/1615373431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f165dc7a0 msgr2=0x562f165fd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.123+0000 7f80b6d9d700 1 --2- 172.21.15.37:0/1615373431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f165dc7a0 0x562f165fd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.123+0000 7f80b6d9d700 1 -- 172.21.15.37:0/1615373431 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f1665fc40 con 0x562f165e2880 2024-06-10T01:09:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.123+0000 7f80b6d9d700 1 --2- 172.21.15.37:0/1615373431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f165e2880 0x562f165fe310 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f80b00992d0 tx=0x7f80b000b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b378f55146826358 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.123+0000 7f809dffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:09:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.123+0000 7f809dffb700 1 -- 172.21.15.37:0/1615373431 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b00a0cb0 con 0x562f165e2880 2024-06-10T01:09:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.123+0000 7f809dffb700 1 -- 172.21.15.37:0/1615373431 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f80b00074e0 con 0x562f165e2880 2024-06-10T01:09:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.123+0000 7f80c75d8b80 1 -- 172.21.15.37:0/1615373431 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80ac00fa20 con 0x562f165e2880 2024-06-10T01:09:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.123+0000 7f809dffb700 1 -- 172.21.15.37:0/1615373431 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f80b00d0120 con 0x562f165e2880 2024-06-10T01:09:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.123+0000 7f80c75d8b80 10 client.17703 fetch_fsmap learned FSMap version 31 2024-06-10T01:09:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.123+0000 7f80c75d8b80 1 -- 172.21.15.37:0/1615373431 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562f16660e90 con 0x562f165e2880 2024-06-10T01:09:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.123+0000 7f809dffb700 1 -- 172.21.15.37:0/1615373431 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80ac00fa20 con 0x562f165e2880 2024-06-10T01:09:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.123+0000 7f809dffb700 1 -- 172.21.15.37:0/1615373431 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f80b0011070 con 0x562f165e2880 2024-06-10T01:09:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.123+0000 7f80c75d8b80 10 client.17703 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:09:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.123+0000 7f80c75d8b80 10 client.17703 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-10T01:09:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.123+0000 7f80c75d8b80 20 client.17703 populate_metadata read hostname 'smithi037' 2024-06-10T01:09:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.124+0000 7f80c75d8b80 1 --2- 172.21.15.37:0/1615373431 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x562f16661430 0x562f16663850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.124+0000 7f80c75d8b80 4 client.17703 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:09:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.124+0000 7f80c75d8b80 1 -- 172.21.15.37:0/1615373431 --> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562f1653ad50 con 0x562f16661430 2024-06-10T01:09:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.124+0000 7f80b659c700 1 --2- 172.21.15.37:0/1615373431 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x562f16661430 0x562f16663850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.124+0000 7f80b659c700 1 --2- 172.21.15.37:0/1615373431 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x562f16661430 0x562f16663850 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=662d32d12a135477 server_cookie=6bc068f61b55efe1 in_seq=0 out_seq=0 2024-06-10T01:09:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.124+0000 7f809dffb700 10 client.17703 ms_handle_connect on v2:172.21.15.167:6836/2720148065 2024-06-10T01:09:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.125+0000 7f809dffb700 1 -- 172.21.15.37:0/1615373431 <== mds.2 v2:172.21.15.167:6836/2720148065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562f1653ad50 con 0x562f16661430 2024-06-10T01:09:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.125+0000 7f809dffb700 10 client.17703 handle_command_reply: tid=0 2024-06-10T01:09:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.125+0000 7f80c75d8b80 1 client.17703 shutdown 2024-06-10T01:09:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.125+0000 7f80c75d8b80 1 -- 172.21.15.37:0/1615373431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f165e2880 msgr2=0x562f165fe310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:45.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.125+0000 7f80c75d8b80 1 --2- 172.21.15.37:0/1615373431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f165e2880 0x562f165fe310 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f80b00992d0 tx=0x7f80b000b970 comp rx=0 tx=0).stop 2024-06-10T01:09:45.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.125+0000 7f80c75d8b80 1 -- 172.21.15.37:0/1615373431 shutdown_connections 2024-06-10T01:09:45.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.125+0000 7f80c75d8b80 1 --2- 172.21.15.37:0/1615373431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f165e2880 0x562f165fe310 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.125+0000 7f80c75d8b80 1 --2- 172.21.15.37:0/1615373431 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x562f16661430 0x562f16663850 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.125+0000 7f80c75d8b80 1 --2- 172.21.15.37:0/1615373431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f165dc7a0 0x562f165fd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.125+0000 7f80c75d8b80 1 --2- 172.21.15.37:0/1615373431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f165e1f60 0x562f165fddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.125+0000 7f80c75d8b80 1 -- 172.21.15.37:0/1615373431 >> 172.21.15.37:0/1615373431 conn(0x562f16509080 msgr2=0x562f165e3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:45.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.125+0000 7f809dffb700 0 client.17703 ms_handle_reset on v2:172.21.15.167:6836/2720148065 2024-06-10T01:09:45.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.125+0000 7f80c75d8b80 1 -- 172.21.15.37:0/1615373431 shutdown_connections 2024-06-10T01:09:45.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.125+0000 7f80c75d8b80 1 -- 172.21.15.37:0/1615373431 wait complete. 2024-06-10T01:09:45.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.125+0000 7f80c75d8b80 20 client.17703 trim_cache size 0 max 16384 2024-06-10T01:09:45.127 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T01:09:45.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.126+0000 7f80bcd55700 1 -- 172.21.15.37:0/574006265 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f80a00a2c20 msgr2=0x7f80a00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:45.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.126+0000 7f80bcd55700 1 --2- 172.21.15.37:0/574006265 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f80a00a2c20 0x7f80a00a50e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f80a8005fd0 tx=0x7f80a8005f60 comp rx=0 tx=0).stop 2024-06-10T01:09:45.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.126+0000 7f80bcd55700 1 -- 172.21.15.37:0/574006265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b8108580 msgr2=0x7f80b81989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:45.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.126+0000 7f80bcd55700 1 --2- 172.21.15.37:0/574006265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b8108580 0x7f80b81989b0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f80b0006070 tx=0x7f80b0007a20 comp rx=0 tx=0).stop 2024-06-10T01:09:45.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.126+0000 7f80bcd55700 1 -- 172.21.15.37:0/574006265 shutdown_connections 2024-06-10T01:09:45.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.126+0000 7f80bcd55700 1 --2- 172.21.15.37:0/574006265 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f80a00a2c20 0x7f80a00a50e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.126+0000 7f80bcd55700 1 --2- 172.21.15.37:0/574006265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b8108580 0x7f80b81989b0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.126+0000 7f80bcd55700 1 --2- 172.21.15.37:0/574006265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80b806ac60 0x7f80b819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.126+0000 7f80bcd55700 1 --2- 172.21.15.37:0/574006265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80b806b620 0x7f80b819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:45.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.126+0000 7f80bcd55700 1 -- 172.21.15.37:0/574006265 >> 172.21.15.37:0/574006265 conn(0x7f80b8078430 msgr2=0x7f80b80ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:45.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.126+0000 7f80bcd55700 1 -- 172.21.15.37:0/574006265 shutdown_connections 2024-06-10T01:09:45.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:45.126+0000 7f80bcd55700 1 -- 172.21.15.37:0/574006265 wait complete. 2024-06-10T01:09:45.876 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:09:46.131 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:09:46.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:46 smithi037 ceph-mon[32192]: pgmap v823: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.4 KiB/s wr, 12 op/s 2024-06-10T01:09:46.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:46 smithi167 ceph-mon[36917]: pgmap v823: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.4 KiB/s wr, 12 op/s 2024-06-10T01:09:46.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:46 smithi094 ceph-mon[36708]: pgmap v823: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.4 KiB/s wr, 12 op/s 2024-06-10T01:09:47.428 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:09:47.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.821+0000 7f7138d0f700 1 -- 172.21.15.37:0/2288423240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71341073d0 msgr2=0x7f71341077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:47.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.821+0000 7f7138d0f700 1 --2- 172.21.15.37:0/2288423240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71341073d0 0x7f71341077b0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f7124009990 tx=0x7f7124009ca0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.821+0000 7f7138d0f700 1 -- 172.21.15.37:0/2288423240 shutdown_connections 2024-06-10T01:09:47.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.821+0000 7f7138d0f700 1 --2- 172.21.15.37:0/2288423240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7134101d30 0x7f713410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.821+0000 7f7138d0f700 1 --2- 172.21.15.37:0/2288423240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71341073d0 0x7f71341077b0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.821+0000 7f7138d0f700 1 --2- 172.21.15.37:0/2288423240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7134101390 0x7f71341017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.821+0000 7f7138d0f700 1 -- 172.21.15.37:0/2288423240 >> 172.21.15.37:0/2288423240 conn(0x7f71340782c0 msgr2=0x7f71340786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:47.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.821+0000 7f7138d0f700 1 -- 172.21.15.37:0/2288423240 shutdown_connections 2024-06-10T01:09:47.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.821+0000 7f7138d0f700 1 -- 172.21.15.37:0/2288423240 wait complete. 2024-06-10T01:09:47.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.822+0000 7f7138d0f700 1 Processor -- start 2024-06-10T01:09:47.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.822+0000 7f7138d0f700 1 -- start start 2024-06-10T01:09:47.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.822+0000 7f7138d0f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7134101390 0x7f71341999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:47.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.822+0000 7f7138d0f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7134101d30 0x7f7134199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:47.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.822+0000 7f7138d0f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71341073d0 0x7f71341a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:47.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.822+0000 7f7138d0f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7134111610 con 0x7f7134101390 2024-06-10T01:09:47.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.822+0000 7f7138d0f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7134111490 con 0x7f7134101d30 2024-06-10T01:09:47.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.822+0000 7f7138d0f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7134111790 con 0x7f71341073d0 2024-06-10T01:09:47.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.822+0000 7f713259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7134101390 0x7f71341999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:47.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.822+0000 7f713259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7134101390 0x7f71341999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53292/0 (socket says 172.21.15.37:53292) 2024-06-10T01:09:47.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.822+0000 7f713259c700 1 -- 172.21.15.37:0/2769318881 learned_addr learned my addr 172.21.15.37:0/2769318881 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:47.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.822+0000 7f7132d9d700 1 --2- 172.21.15.37:0/2769318881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71341073d0 0x7f71341a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:47.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.822+0000 7f7131d9b700 1 --2- 172.21.15.37:0/2769318881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7134101d30 0x7f7134199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:47.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.822+0000 7f713259c700 1 -- 172.21.15.37:0/2769318881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71341073d0 msgr2=0x7f71341a4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:47.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.822+0000 7f713259c700 1 --2- 172.21.15.37:0/2769318881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71341073d0 0x7f71341a4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.822+0000 7f713259c700 1 -- 172.21.15.37:0/2769318881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7134101d30 msgr2=0x7f7134199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:47.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.822+0000 7f713259c700 1 --2- 172.21.15.37:0/2769318881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7134101d30 0x7f7134199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.822+0000 7f713259c700 1 -- 172.21.15.37:0/2769318881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71341a4f90 con 0x7f7134101390 2024-06-10T01:09:47.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.823+0000 7f713259c700 1 --2- 172.21.15.37:0/2769318881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7134101390 0x7f71341999d0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f7124000c00 tx=0x7f7124003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:47.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.823+0000 7f71237fe700 1 -- 172.21.15.37:0/2769318881 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7124027070 con 0x7f7134101390 2024-06-10T01:09:47.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.823+0000 7f71237fe700 1 -- 172.21.15.37:0/2769318881 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7124004050 con 0x7f7134101390 2024-06-10T01:09:47.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.823+0000 7f7138d0f700 1 -- 172.21.15.37:0/2769318881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71341a5160 con 0x7f7134101390 2024-06-10T01:09:47.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.823+0000 7f71237fe700 1 -- 172.21.15.37:0/2769318881 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712400f460 con 0x7f7134101390 2024-06-10T01:09:47.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.823+0000 7f7138d0f700 1 -- 172.21.15.37:0/2769318881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71341a55b0 con 0x7f7134101390 2024-06-10T01:09:47.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.825+0000 7f71237fe700 1 -- 172.21.15.37:0/2769318881 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f712402a070 con 0x7f7134101390 2024-06-10T01:09:47.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.826+0000 7f71237fe700 1 --2- 172.21.15.37:0/2769318881 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f711c0a2c20 0x7f711c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:47.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.826+0000 7f7131d9b700 1 --2- 172.21.15.37:0/2769318881 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f711c0a2c20 0x7f711c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:47.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.826+0000 7f71237fe700 1 -- 172.21.15.37:0/2769318881 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f71240d0440 con 0x7f7134101390 2024-06-10T01:09:47.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.826+0000 7f7131d9b700 1 --2- 172.21.15.37:0/2769318881 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f711c0a2c20 0x7f711c0a50e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f71340787e0 tx=0x7f712800a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:47.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.827+0000 7f7143592b80 1 Processor -- start 2024-06-10T01:09:47.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.827+0000 7f7143592b80 1 -- start start 2024-06-10T01:09:47.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.827+0000 7f7143592b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a352c08f60 0x55a352c09340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:47.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.827+0000 7f7143592b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a352c09880 0x55a352c03260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:47.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.827+0000 7f7143592b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a352c037a0 0x55a352c03bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:47.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.827+0000 7f7143592b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a352af5f60 con 0x55a352c09880 2024-06-10T01:09:47.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.827+0000 7f7143592b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a352b54850 con 0x55a352c037a0 2024-06-10T01:09:47.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.827+0000 7f7143592b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a352abc460 con 0x55a352c08f60 2024-06-10T01:09:47.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.827+0000 7f7131d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a352c09880 0x55a352c03260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:47.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.827+0000 7f7131d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a352c09880 0x55a352c03260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53296/0 (socket says 172.21.15.37:53296) 2024-06-10T01:09:47.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.827+0000 7f7131d9b700 1 -- 172.21.15.37:0/3934138291 learned_addr learned my addr 172.21.15.37:0/3934138291 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:47.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.827+0000 7f713259c700 1 --2- 172.21.15.37:0/3934138291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a352c08f60 0x55a352c09340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:47.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.827+0000 7f7132d9d700 1 --2- 172.21.15.37:0/3934138291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a352c037a0 0x55a352c03bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:47.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.827+0000 7f7131d9b700 1 -- 172.21.15.37:0/3934138291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a352c08f60 msgr2=0x55a352c09340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:47.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.827+0000 7f7131d9b700 1 --2- 172.21.15.37:0/3934138291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a352c08f60 0x55a352c09340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.827+0000 7f7131d9b700 1 -- 172.21.15.37:0/3934138291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a352c037a0 msgr2=0x55a352c03bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:47.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.827+0000 7f7131d9b700 1 --2- 172.21.15.37:0/3934138291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a352c037a0 0x55a352c03bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.827+0000 7f7131d9b700 1 -- 172.21.15.37:0/3934138291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a352aa38d0 con 0x55a352c09880 2024-06-10T01:09:47.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.828+0000 7f713259c700 1 --2- 172.21.15.37:0/3934138291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a352c08f60 0x55a352c09340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:09:47.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.828+0000 7f7131d9b700 1 --2- 172.21.15.37:0/3934138291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a352c09880 0x55a352c03260 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f712800fe70 tx=0x7f7128012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb922b55247066df server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:47.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.828+0000 7f7120ff9700 1 -- 172.21.15.37:0/3934138291 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128016070 con 0x55a352c09880 2024-06-10T01:09:47.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.828+0000 7f7120ff9700 1 -- 172.21.15.37:0/3934138291 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7128014070 con 0x55a352c09880 2024-06-10T01:09:47.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.828+0000 7f7120ff9700 1 -- 172.21.15.37:0/3934138291 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712801d400 con 0x55a352c09880 2024-06-10T01:09:47.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.829+0000 7f7143592b80 1 -- 172.21.15.37:0/3934138291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a352c09880 msgr2=0x55a352c03260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:47.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.829+0000 7f7143592b80 1 --2- 172.21.15.37:0/3934138291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a352c09880 0x55a352c03260 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f712800fe70 tx=0x7f7128012000 comp rx=0 tx=0).stop 2024-06-10T01:09:47.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.829+0000 7f7143592b80 1 -- 172.21.15.37:0/3934138291 shutdown_connections 2024-06-10T01:09:47.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.829+0000 7f7143592b80 1 --2- 172.21.15.37:0/3934138291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a352c037a0 0x55a352c03bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.829+0000 7f7143592b80 1 --2- 172.21.15.37:0/3934138291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a352c08f60 0x55a352c09340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.829+0000 7f7143592b80 1 --2- 172.21.15.37:0/3934138291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a352c09880 0x55a352c03260 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.829+0000 7f7143592b80 1 -- 172.21.15.37:0/3934138291 >> 172.21.15.37:0/3934138291 conn(0x55a352b30080 msgr2=0x55a352bfe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.829+0000 7f7143592b80 1 -- 172.21.15.37:0/3934138291 shutdown_connections 2024-06-10T01:09:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.829+0000 7f7143592b80 1 -- 172.21.15.37:0/3934138291 wait complete. 2024-06-10T01:09:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.829+0000 7f7143592b80 1 -- start start 2024-06-10T01:09:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.829+0000 7f7143592b80 1 Processor -- start 2024-06-10T01:09:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.829+0000 7f7143592b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a352c037a0 0x55a352c0f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.829+0000 7f7143592b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a352c08f60 0x55a352c0c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.830+0000 7f7143592b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a352c09880 0x55a352c0cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.830+0000 7f7143592b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a352af4fc0 con 0x55a352c037a0 2024-06-10T01:09:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.830+0000 7f7143592b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a352b50810 con 0x55a352c08f60 2024-06-10T01:09:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.830+0000 7f7143592b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a352c10260 con 0x55a352c09880 2024-06-10T01:09:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.830+0000 7f713259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a352c037a0 0x55a352c0f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.830+0000 7f7143592b80 10 client.0 _ll_register_callbacks cb 0x55a352aea5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:47.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.830+0000 7f713259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a352c037a0 0x55a352c0f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53298/0 (socket says 172.21.15.37:53298) 2024-06-10T01:09:47.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.830+0000 7f713259c700 1 -- 172.21.15.37:0/1329751612 learned_addr learned my addr 172.21.15.37:0/1329751612 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:47.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.830+0000 7f7131d9b700 1 --2- 172.21.15.37:0/1329751612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a352c08f60 0x55a352c0c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:47.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.830+0000 7f7132d9d700 1 --2- 172.21.15.37:0/1329751612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a352c09880 0x55a352c0cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:47.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.830+0000 7f713259c700 1 -- 172.21.15.37:0/1329751612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a352c09880 msgr2=0x55a352c0cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:47.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.830+0000 7f713259c700 1 --2- 172.21.15.37:0/1329751612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a352c09880 0x55a352c0cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.830+0000 7f713259c700 1 -- 172.21.15.37:0/1329751612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a352c08f60 msgr2=0x55a352c0c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:47.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.830+0000 7f713259c700 1 --2- 172.21.15.37:0/1329751612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a352c08f60 0x55a352c0c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.830+0000 7f713259c700 1 -- 172.21.15.37:0/1329751612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a352b1a750 con 0x55a352c037a0 2024-06-10T01:09:47.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.830+0000 7f713259c700 1 --2- 172.21.15.37:0/1329751612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a352c037a0 0x55a352c0f580 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f712409c870 tx=0x7f712409c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09eb6158513db6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:47.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.830+0000 7f7119ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:09:47.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.830+0000 7f7119ffb700 1 -- 172.21.15.37:0/1329751612 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712409c8e0 con 0x55a352c037a0 2024-06-10T01:09:47.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.830+0000 7f7119ffb700 1 -- 172.21.15.37:0/1329751612 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f71240870b0 con 0x55a352c037a0 2024-06-10T01:09:47.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.830+0000 7f7143592b80 1 -- 172.21.15.37:0/1329751612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7128014070 con 0x55a352c037a0 2024-06-10T01:09:47.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.831+0000 7f7119ffb700 1 -- 172.21.15.37:0/1329751612 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f71240a0c90 con 0x55a352c037a0 2024-06-10T01:09:47.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.831+0000 7f7143592b80 10 client.17715 fetch_fsmap learned FSMap version 31 2024-06-10T01:09:47.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.831+0000 7f7143592b80 1 -- 172.21.15.37:0/1329751612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a352c1a910 con 0x55a352c037a0 2024-06-10T01:09:47.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.831+0000 7f7119ffb700 1 -- 172.21.15.37:0/1329751612 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7128014070 con 0x55a352c037a0 2024-06-10T01:09:47.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.831+0000 7f7119ffb700 1 -- 172.21.15.37:0/1329751612 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f71240a5030 con 0x55a352c037a0 2024-06-10T01:09:47.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.831+0000 7f7143592b80 10 client.17715 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:09:47.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.831+0000 7f7143592b80 10 client.17715 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-06-10T01:09:47.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.831+0000 7f7143592b80 20 client.17715 populate_metadata read hostname 'smithi037' 2024-06-10T01:09:47.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.831+0000 7f7143592b80 1 --2- 172.21.15.37:0/1329751612 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55a352c1cde0 0x55a352c1f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:47.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.831+0000 7f7143592b80 4 client.17715 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:09:47.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.831+0000 7f7143592b80 1 -- 172.21.15.37:0/1329751612 --> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a352b61d50 con 0x55a352c1cde0 2024-06-10T01:09:47.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.831+0000 7f7131d9b700 1 --2- 172.21.15.37:0/1329751612 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55a352c1cde0 0x55a352c1f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:47.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.832+0000 7f7131d9b700 1 --2- 172.21.15.37:0/1329751612 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55a352c1cde0 0x55a352c1f210 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c41a5cfe2d5156a server_cookie=ff90a494640b68b5 in_seq=0 out_seq=0 2024-06-10T01:09:47.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.832+0000 7f7119ffb700 10 client.17715 ms_handle_connect on v2:172.21.15.94:6838/946269331 2024-06-10T01:09:47.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.832+0000 7f7119ffb700 1 -- 172.21.15.37:0/1329751612 <== mds.1 v2:172.21.15.94:6838/946269331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a352b61d50 con 0x55a352c1cde0 2024-06-10T01:09:47.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.832+0000 7f7119ffb700 10 client.17715 handle_command_reply: tid=0 2024-06-10T01:09:47.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.833+0000 7f7143592b80 1 client.17715 shutdown 2024-06-10T01:09:47.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.833+0000 7f7143592b80 1 -- 172.21.15.37:0/1329751612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a352c037a0 msgr2=0x55a352c0f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:47.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.833+0000 7f7143592b80 1 --2- 172.21.15.37:0/1329751612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a352c037a0 0x55a352c0f580 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f712409c870 tx=0x7f712409c420 comp rx=0 tx=0).stop 2024-06-10T01:09:47.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.833+0000 7f7143592b80 1 -- 172.21.15.37:0/1329751612 shutdown_connections 2024-06-10T01:09:47.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.833+0000 7f7143592b80 1 --2- 172.21.15.37:0/1329751612 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55a352c1cde0 0x55a352c1f210 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.833+0000 7f7143592b80 1 --2- 172.21.15.37:0/1329751612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a352c09880 0x55a352c0cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.833+0000 7f7143592b80 1 --2- 172.21.15.37:0/1329751612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a352c037a0 0x55a352c0f580 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.833+0000 7f7143592b80 1 --2- 172.21.15.37:0/1329751612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a352c08f60 0x55a352c0c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.833+0000 7f7119ffb700 0 client.17715 ms_handle_reset on v2:172.21.15.94:6838/946269331 2024-06-10T01:09:47.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.833+0000 7f7143592b80 1 -- 172.21.15.37:0/1329751612 >> 172.21.15.37:0/1329751612 conn(0x55a352b30080 msgr2=0x55a352bfd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:47.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.833+0000 7f7143592b80 1 -- 172.21.15.37:0/1329751612 shutdown_connections 2024-06-10T01:09:47.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.833+0000 7f7143592b80 1 -- 172.21.15.37:0/1329751612 wait complete. 2024-06-10T01:09:47.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.833+0000 7f7143592b80 20 client.17715 trim_cache size 0 max 16384 2024-06-10T01:09:47.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.848+0000 7f7143592b80 1 Processor -- start 2024-06-10T01:09:47.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.848+0000 7f7143592b80 1 -- start start 2024-06-10T01:09:47.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.848+0000 7f7143592b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a352c037a0 0x55a352c03260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:47.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.848+0000 7f7143592b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a352c08f60 0x55a352c0a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:47.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.848+0000 7f7143592b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a352c09880 0x55a352bfde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:47.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.848+0000 7f7143592b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f71341b0a80 con 0x55a352c037a0 2024-06-10T01:09:47.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.848+0000 7f7143592b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f71341ae310 con 0x55a352c08f60 2024-06-10T01:09:47.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.848+0000 7f7143592b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f71341ac160 con 0x55a352c09880 2024-06-10T01:09:47.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.848+0000 7f713259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a352c037a0 0x55a352c03260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:47.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.848+0000 7f713259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a352c037a0 0x55a352c03260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53314/0 (socket says 172.21.15.37:53314) 2024-06-10T01:09:47.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.848+0000 7f713259c700 1 -- 172.21.15.37:0/1483895922 learned_addr learned my addr 172.21.15.37:0/1483895922 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:47.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.849+0000 7f7132d9d700 1 --2- 172.21.15.37:0/1483895922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a352c09880 0x55a352bfde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:47.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.849+0000 7f7131d9b700 1 --2- 172.21.15.37:0/1483895922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a352c08f60 0x55a352c0a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:47.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.849+0000 7f713259c700 1 -- 172.21.15.37:0/1483895922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a352c09880 msgr2=0x55a352bfde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:47.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.849+0000 7f713259c700 1 --2- 172.21.15.37:0/1483895922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a352c09880 0x55a352bfde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.849+0000 7f713259c700 1 -- 172.21.15.37:0/1483895922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a352c08f60 msgr2=0x55a352c0a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:47.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.849+0000 7f713259c700 1 --2- 172.21.15.37:0/1483895922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a352c08f60 0x55a352c0a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.849+0000 7f713259c700 1 -- 172.21.15.37:0/1483895922 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a352c0a8f0 con 0x55a352c037a0 2024-06-10T01:09:47.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.849+0000 7f7131d9b700 1 --2- 172.21.15.37:0/1483895922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a352c08f60 0x55a352c0a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:09:47.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.849+0000 7f7132d9d700 1 --2- 172.21.15.37:0/1483895922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a352c09880 0x55a352bfde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:09:47.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.849+0000 7f713259c700 1 --2- 172.21.15.37:0/1483895922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a352c037a0 0x55a352c03260 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f7124099260 tx=0x7f712409adf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615836cac78b2221 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:47.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.849+0000 7f711bfff700 1 -- 172.21.15.37:0/1483895922 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712409c8e0 con 0x55a352c037a0 2024-06-10T01:09:47.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.849+0000 7f711bfff700 1 -- 172.21.15.37:0/1483895922 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71240d0170 con 0x55a352c037a0 2024-06-10T01:09:47.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.849+0000 7f711bfff700 1 -- 172.21.15.37:0/1483895922 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712409cc60 con 0x55a352c037a0 2024-06-10T01:09:47.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.850+0000 7f7143592b80 1 -- 172.21.15.37:0/1483895922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a352c037a0 msgr2=0x55a352c03260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:47.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.850+0000 7f7143592b80 1 --2- 172.21.15.37:0/1483895922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a352c037a0 0x55a352c03260 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f7124099260 tx=0x7f712409adf0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.850+0000 7f7143592b80 1 -- 172.21.15.37:0/1483895922 shutdown_connections 2024-06-10T01:09:47.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.850+0000 7f7143592b80 1 --2- 172.21.15.37:0/1483895922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a352c09880 0x55a352bfde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.850+0000 7f7143592b80 1 --2- 172.21.15.37:0/1483895922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a352c037a0 0x55a352c03260 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.850+0000 7f7143592b80 1 --2- 172.21.15.37:0/1483895922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a352c08f60 0x55a352c0a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.850+0000 7f7143592b80 1 -- 172.21.15.37:0/1483895922 >> 172.21.15.37:0/1483895922 conn(0x55a352b30080 msgr2=0x55a352c09ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:47.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.850+0000 7f7143592b80 1 -- 172.21.15.37:0/1483895922 shutdown_connections 2024-06-10T01:09:47.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.850+0000 7f7143592b80 1 -- 172.21.15.37:0/1483895922 wait complete. 2024-06-10T01:09:47.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.850+0000 7f7143592b80 1 -- start start 2024-06-10T01:09:47.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.850+0000 7f7143592b80 1 Processor -- start 2024-06-10T01:09:47.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.851+0000 7f7143592b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a352c037a0 0x55a352c15e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:47.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.851+0000 7f7143592b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a352c08f60 0x55a352c24530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:47.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.851+0000 7f7143592b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a352c09880 0x55a352c24bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:47.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.851+0000 7f7143592b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f71341ac7d0 con 0x55a352c037a0 2024-06-10T01:09:47.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.851+0000 7f7143592b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f71341adc40 con 0x55a352c09880 2024-06-10T01:09:47.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.851+0000 7f7143592b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f71341acea0 con 0x55a352c08f60 2024-06-10T01:09:47.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.851+0000 7f7143592b80 10 client.0 _ll_register_callbacks cb 0x55a352aea5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:47.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.851+0000 7f713259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a352c037a0 0x55a352c15e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:47.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.851+0000 7f713259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a352c037a0 0x55a352c15e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53316/0 (socket says 172.21.15.37:53316) 2024-06-10T01:09:47.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.851+0000 7f713259c700 1 -- 172.21.15.37:0/1435015975 learned_addr learned my addr 172.21.15.37:0/1435015975 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:47.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.851+0000 7f7131d9b700 1 --2- 172.21.15.37:0/1435015975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a352c08f60 0x55a352c24530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:47.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.851+0000 7f7132d9d700 1 --2- 172.21.15.37:0/1435015975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a352c09880 0x55a352c24bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:47.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.851+0000 7f713259c700 1 -- 172.21.15.37:0/1435015975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a352c08f60 msgr2=0x55a352c24530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:47.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.851+0000 7f713259c700 1 --2- 172.21.15.37:0/1435015975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a352c08f60 0x55a352c24530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.851+0000 7f713259c700 1 -- 172.21.15.37:0/1435015975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a352c09880 msgr2=0x55a352c24bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:47.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.851+0000 7f713259c700 1 --2- 172.21.15.37:0/1435015975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a352c09880 0x55a352c24bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.851+0000 7f713259c700 1 -- 172.21.15.37:0/1435015975 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a352c86b00 con 0x55a352c037a0 2024-06-10T01:09:47.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.851+0000 7f7132d9d700 1 --2- 172.21.15.37:0/1435015975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a352c09880 0x55a352c24bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:09:47.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.851+0000 7f713259c700 1 --2- 172.21.15.37:0/1435015975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a352c037a0 0x55a352c15e50 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f71240a3080 tx=0x7f71240d0340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b3e4a494030c9f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:47.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.851+0000 7f7119ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:09:47.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.851+0000 7f7119ffb700 1 -- 172.21.15.37:0/1435015975 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712409c8e0 con 0x55a352c037a0 2024-06-10T01:09:47.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.851+0000 7f7119ffb700 1 -- 172.21.15.37:0/1435015975 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f7124087070 con 0x55a352c037a0 2024-06-10T01:09:47.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.851+0000 7f7143592b80 1 -- 172.21.15.37:0/1435015975 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71240d0170 con 0x55a352c037a0 2024-06-10T01:09:47.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.852+0000 7f7119ffb700 1 -- 172.21.15.37:0/1435015975 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f712409c6b0 con 0x55a352c037a0 2024-06-10T01:09:47.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.852+0000 7f7143592b80 10 client.17721 fetch_fsmap learned FSMap version 31 2024-06-10T01:09:47.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.852+0000 7f7143592b80 1 -- 172.21.15.37:0/1435015975 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a352c86b00 con 0x55a352c037a0 2024-06-10T01:09:47.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.852+0000 7f7119ffb700 1 -- 172.21.15.37:0/1435015975 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71240d0170 con 0x55a352c037a0 2024-06-10T01:09:47.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.852+0000 7f7119ffb700 1 -- 172.21.15.37:0/1435015975 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f712409b7a0 con 0x55a352c037a0 2024-06-10T01:09:47.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.852+0000 7f7143592b80 10 client.17721 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:09:47.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.852+0000 7f7143592b80 10 client.17721 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-06-10T01:09:47.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.852+0000 7f7143592b80 20 client.17721 populate_metadata read hostname 'smithi037' 2024-06-10T01:09:47.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.852+0000 7f7143592b80 1 --2- 172.21.15.37:0/1435015975 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55a352c8c1e0 0x55a352c8e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:47.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.853+0000 7f7143592b80 4 client.17721 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:09:47.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.853+0000 7f7143592b80 1 -- 172.21.15.37:0/1435015975 --> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a352b5c2d0 con 0x55a352c8c1e0 2024-06-10T01:09:47.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.853+0000 7f7131d9b700 1 --2- 172.21.15.37:0/1435015975 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55a352c8c1e0 0x55a352c8e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:47.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.853+0000 7f7119ffb700 10 client.17721 ms_handle_connect on v2:172.21.15.94:6838/946269331 2024-06-10T01:09:47.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.853+0000 7f7131d9b700 1 --2- 172.21.15.37:0/1435015975 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55a352c8c1e0 0x55a352c8e5d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa5511d2c4b89a41 server_cookie=debfc763067b023d in_seq=0 out_seq=0 2024-06-10T01:09:47.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.853+0000 7f7119ffb700 1 -- 172.21.15.37:0/1435015975 <== mds.1 v2:172.21.15.94:6838/946269331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a352b5c2d0 con 0x55a352c8c1e0 2024-06-10T01:09:47.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.853+0000 7f7119ffb700 10 client.17721 handle_command_reply: tid=0 2024-06-10T01:09:47.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.853+0000 7f7143592b80 1 client.17721 shutdown 2024-06-10T01:09:47.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.854+0000 7f7143592b80 1 -- 172.21.15.37:0/1435015975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a352c037a0 msgr2=0x55a352c15e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:47.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.854+0000 7f7143592b80 1 --2- 172.21.15.37:0/1435015975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a352c037a0 0x55a352c15e50 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f71240a3080 tx=0x7f71240d0340 comp rx=0 tx=0).stop 2024-06-10T01:09:47.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.854+0000 7f7143592b80 1 -- 172.21.15.37:0/1435015975 shutdown_connections 2024-06-10T01:09:47.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.854+0000 7f7143592b80 1 --2- 172.21.15.37:0/1435015975 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55a352c8c1e0 0x55a352c8e5d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.854+0000 7f7143592b80 1 --2- 172.21.15.37:0/1435015975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a352c09880 0x55a352c24bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.854+0000 7f7143592b80 1 --2- 172.21.15.37:0/1435015975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a352c037a0 0x55a352c15e50 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.854+0000 7f7119ffb700 0 client.17721 ms_handle_reset on v2:172.21.15.94:6838/946269331 2024-06-10T01:09:47.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.854+0000 7f7143592b80 1 --2- 172.21.15.37:0/1435015975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a352c08f60 0x55a352c24530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.854+0000 7f7143592b80 1 -- 172.21.15.37:0/1435015975 >> 172.21.15.37:0/1435015975 conn(0x55a352b30080 msgr2=0x55a352c0e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:47.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.854+0000 7f7143592b80 1 -- 172.21.15.37:0/1435015975 shutdown_connections 2024-06-10T01:09:47.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.854+0000 7f7143592b80 1 -- 172.21.15.37:0/1435015975 wait complete. 2024-06-10T01:09:47.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.854+0000 7f7143592b80 20 client.17721 trim_cache size 0 max 16384 2024-06-10T01:09:47.854 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T01:09:47.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.855+0000 7f7138d0f700 1 -- 172.21.15.37:0/2769318881 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f711c0a2c20 msgr2=0x7f711c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:47.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.855+0000 7f7138d0f700 1 --2- 172.21.15.37:0/2769318881 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f711c0a2c20 0x7f711c0a50e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f71340787e0 tx=0x7f712800a380 comp rx=0 tx=0).stop 2024-06-10T01:09:47.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.855+0000 7f7138d0f700 1 -- 172.21.15.37:0/2769318881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7134101390 msgr2=0x7f71341999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:47.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.855+0000 7f7138d0f700 1 --2- 172.21.15.37:0/2769318881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7134101390 0x7f71341999d0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f7124000c00 tx=0x7f7124003680 comp rx=0 tx=0).stop 2024-06-10T01:09:47.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.855+0000 7f7138d0f700 1 -- 172.21.15.37:0/2769318881 shutdown_connections 2024-06-10T01:09:47.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.855+0000 7f7138d0f700 1 --2- 172.21.15.37:0/2769318881 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f711c0a2c20 0x7f711c0a50e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.855+0000 7f7138d0f700 1 --2- 172.21.15.37:0/2769318881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71341073d0 0x7f71341a4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.855+0000 7f7138d0f700 1 --2- 172.21.15.37:0/2769318881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7134101390 0x7f71341999d0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.855+0000 7f7138d0f700 1 --2- 172.21.15.37:0/2769318881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7134101d30 0x7f7134199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:47.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.855+0000 7f7138d0f700 1 -- 172.21.15.37:0/2769318881 >> 172.21.15.37:0/2769318881 conn(0x7f71340782c0 msgr2=0x7f71340fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:47.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.855+0000 7f7138d0f700 1 -- 172.21.15.37:0/2769318881 shutdown_connections 2024-06-10T01:09:47.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:47.855+0000 7f7138d0f700 1 -- 172.21.15.37:0/2769318881 wait complete. 2024-06-10T01:09:48.522 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:48 smithi037 ceph-mon[32192]: pgmap v824: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2024-06-10T01:09:48.551 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:09:48.580 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:09:48.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:48 smithi167 ceph-mon[36917]: pgmap v824: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2024-06-10T01:09:48.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:48 smithi094 ceph-mon[36708]: pgmap v824: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2024-06-10T01:09:49.778 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:09:50.252 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:50 smithi037 ceph-mon[32192]: pgmap v825: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.4 KiB/s wr, 13 op/s 2024-06-10T01:09:50.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.262+0000 7f71630d3700 1 -- 172.21.15.37:0/2751168701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f715c1066b0 msgr2=0x7f715c10e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:50.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.262+0000 7f71630d3700 1 --2- 172.21.15.37:0/2751168701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f715c1066b0 0x7f715c10e280 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f715400b0d0 tx=0x7f715400b3e0 comp rx=0 tx=0).stop 2024-06-10T01:09:50.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.262+0000 7f71630d3700 1 -- 172.21.15.37:0/2751168701 shutdown_connections 2024-06-10T01:09:50.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.262+0000 7f71630d3700 1 --2- 172.21.15.37:0/2751168701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f715c1066b0 0x7f715c10e280 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:50.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.262+0000 7f71630d3700 1 --2- 172.21.15.37:0/2751168701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f715c101620 0x7f715c101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:50.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.262+0000 7f71630d3700 1 --2- 172.21.15.37:0/2751168701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f715c101fd0 0x7f715c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:50.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.262+0000 7f71630d3700 1 -- 172.21.15.37:0/2751168701 >> 172.21.15.37:0/2751168701 conn(0x7f715c078340 msgr2=0x7f715c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:50.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.262+0000 7f71630d3700 1 -- 172.21.15.37:0/2751168701 shutdown_connections 2024-06-10T01:09:50.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.262+0000 7f71630d3700 1 -- 172.21.15.37:0/2751168701 wait complete. 2024-06-10T01:09:50.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.262+0000 7f71630d3700 1 Processor -- start 2024-06-10T01:09:50.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.262+0000 7f71630d3700 1 -- start start 2024-06-10T01:09:50.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.263+0000 7f71630d3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f715c101620 0x7f715c103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:50.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.263+0000 7f71630d3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f715c101fd0 0x7f715c1035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:50.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.263+0000 7f71630d3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f715c1066b0 0x7f715c103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:50.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.263+0000 7f71630d3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f715c1116a0 con 0x7f715c101620 2024-06-10T01:09:50.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.263+0000 7f71630d3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f715c111520 con 0x7f715c1066b0 2024-06-10T01:09:50.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.263+0000 7f71630d3700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f715c111820 con 0x7f715c101fd0 2024-06-10T01:09:50.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.263+0000 7f7160e6f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f715c101620 0x7f715c103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:50.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.263+0000 7f7160e6f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f715c101620 0x7f715c103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53336/0 (socket says 172.21.15.37:53336) 2024-06-10T01:09:50.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.263+0000 7f7160e6f700 1 -- 172.21.15.37:0/2525317965 learned_addr learned my addr 172.21.15.37:0/2525317965 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:50.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.263+0000 7f715bfff700 1 --2- 172.21.15.37:0/2525317965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f715c101fd0 0x7f715c1035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:50.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.263+0000 7f7161670700 1 --2- 172.21.15.37:0/2525317965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f715c1066b0 0x7f715c103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:50.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.263+0000 7f7160e6f700 1 -- 172.21.15.37:0/2525317965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f715c101fd0 msgr2=0x7f715c1035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:50.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.263+0000 7f7160e6f700 1 --2- 172.21.15.37:0/2525317965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f715c101fd0 0x7f715c1035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:50.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.263+0000 7f7160e6f700 1 -- 172.21.15.37:0/2525317965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f715c1066b0 msgr2=0x7f715c103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:50.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.263+0000 7f7160e6f700 1 --2- 172.21.15.37:0/2525317965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f715c1066b0 0x7f715c103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:50.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.263+0000 7f7160e6f700 1 -- 172.21.15.37:0/2525317965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f715c1a7070 con 0x7f715c101620 2024-06-10T01:09:50.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.263+0000 7f7161670700 1 --2- 172.21.15.37:0/2525317965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f715c1066b0 0x7f715c103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:09:50.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.263+0000 7f7160e6f700 1 --2- 172.21.15.37:0/2525317965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f715c101620 0x7f715c103070 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f715000d980 tx=0x7f715000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:50.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.263+0000 7f7159ffb700 1 -- 172.21.15.37:0/2525317965 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7150016070 con 0x7f715c101620 2024-06-10T01:09:50.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.263+0000 7f7159ffb700 1 -- 172.21.15.37:0/2525317965 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7150004d10 con 0x7f715c101620 2024-06-10T01:09:50.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.264+0000 7f71630d3700 1 -- 172.21.15.37:0/2525317965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f715c1a7360 con 0x7f715c101620 2024-06-10T01:09:50.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.264+0000 7f7159ffb700 1 -- 172.21.15.37:0/2525317965 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71500055f0 con 0x7f715c101620 2024-06-10T01:09:50.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.264+0000 7f71630d3700 1 -- 172.21.15.37:0/2525317965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f715c1a78f0 con 0x7f715c101620 2024-06-10T01:09:50.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.265+0000 7f71630d3700 1 -- 172.21.15.37:0/2525317965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f715c069180 con 0x7f715c101620 2024-06-10T01:09:50.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.266+0000 7f7159ffb700 1 -- 172.21.15.37:0/2525317965 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7150004750 con 0x7f715c101620 2024-06-10T01:09:50.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.267+0000 7f7159ffb700 1 --2- 172.21.15.37:0/2525317965 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f714c0a2ca0 0x7f714c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:50.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.267+0000 7f715bfff700 1 --2- 172.21.15.37:0/2525317965 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f714c0a2ca0 0x7f714c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:50.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.267+0000 7f7159ffb700 1 -- 172.21.15.37:0/2525317965 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f71500cf1e0 con 0x7f715c101620 2024-06-10T01:09:50.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.267+0000 7f715bfff700 1 --2- 172.21.15.37:0/2525317965 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f714c0a2ca0 0x7f714c0a5160 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f715c06bca0 tx=0x7f714800a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:50.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.272+0000 7f7159ffb700 1 -- 172.21.15.37:0/2525317965 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f715001f020 con 0x7f715c101620 2024-06-10T01:09:50.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.530+0000 7f71630d3700 1 -- 172.21.15.37:0/2525317965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f715c1a7b50 con 0x7f715c101620 2024-06-10T01:09:50.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.531+0000 7f7159ffb700 1 -- 172.21.15.37:0/2525317965 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11868 (secure 0 0 0) 0x7f715009bd50 con 0x7f715c101620 2024-06-10T01:09:50.526 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:09:50.526 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":26554,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2107277223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2107277223},{"type":"v1","addr":"172.21.15.167:6835","nonce":2107277223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T01:06:09.604504+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24472,24451,24458]},"id":1}]} 2024-06-10T01:09:50.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.533+0000 7f71630d3700 1 -- 172.21.15.37:0/2525317965 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f714c0a2ca0 msgr2=0x7f714c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:50.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.533+0000 7f71630d3700 1 --2- 172.21.15.37:0/2525317965 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f714c0a2ca0 0x7f714c0a5160 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f715c06bca0 tx=0x7f714800a380 comp rx=0 tx=0).stop 2024-06-10T01:09:50.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.533+0000 7f71630d3700 1 -- 172.21.15.37:0/2525317965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f715c101620 msgr2=0x7f715c103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:50.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.533+0000 7f71630d3700 1 --2- 172.21.15.37:0/2525317965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f715c101620 0x7f715c103070 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f715000d980 tx=0x7f715000dc90 comp rx=0 tx=0).stop 2024-06-10T01:09:50.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.533+0000 7f71630d3700 1 -- 172.21.15.37:0/2525317965 shutdown_connections 2024-06-10T01:09:50.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.533+0000 7f71630d3700 1 --2- 172.21.15.37:0/2525317965 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f714c0a2ca0 0x7f714c0a5160 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:50.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.533+0000 7f71630d3700 1 --2- 172.21.15.37:0/2525317965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f715c1066b0 0x7f715c103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:50.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.533+0000 7f71630d3700 1 --2- 172.21.15.37:0/2525317965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f715c101620 0x7f715c103070 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:50.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.533+0000 7f71630d3700 1 --2- 172.21.15.37:0/2525317965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f715c101fd0 0x7f715c1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:50.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.533+0000 7f71630d3700 1 -- 172.21.15.37:0/2525317965 >> 172.21.15.37:0/2525317965 conn(0x7f715c078340 msgr2=0x7f715c1051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:50.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.533+0000 7f71630d3700 1 -- 172.21.15.37:0/2525317965 shutdown_connections 2024-06-10T01:09:50.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:50.533+0000 7f71630d3700 1 -- 172.21.15.37:0/2525317965 wait complete. 2024-06-10T01:09:50.530 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 31 2024-06-10T01:09:50.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:50 smithi167 ceph-mon[36917]: pgmap v825: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.4 KiB/s wr, 13 op/s 2024-06-10T01:09:50.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:50 smithi094 ceph-mon[36708]: pgmap v825: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.4 KiB/s wr, 13 op/s 2024-06-10T01:09:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.297+0000 7fa6e5909640 1 -- 172.21.15.37:0/218488388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6e00ffcf0 msgr2=0x7fa6e0100110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.297+0000 7fa6e5909640 1 --2- 172.21.15.37:0/218488388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6e00ffcf0 0x7fa6e0100110 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0009930 tx=0x7fa6d002a420 comp rx=0 tx=0).stop 2024-06-10T01:09:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.297+0000 7fa6e5909640 1 -- 172.21.15.37:0/218488388 shutdown_connections 2024-06-10T01:09:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.297+0000 7fa6e5909640 1 --2- 172.21.15.37:0/218488388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6e0103680 0x7fa6e0109f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.297+0000 7fa6e5909640 1 --2- 172.21.15.37:0/218488388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6e0102cc0 0x7fa6e0103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.297+0000 7fa6e5909640 1 --2- 172.21.15.37:0/218488388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6e00ffcf0 0x7fa6e0100110 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.297+0000 7fa6e5909640 1 -- 172.21.15.37:0/218488388 >> 172.21.15.37:0/218488388 conn(0x7fa6e00fb620 msgr2=0x7fa6e00fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.298+0000 7fa6e5909640 1 -- 172.21.15.37:0/218488388 shutdown_connections 2024-06-10T01:09:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.298+0000 7fa6e5909640 1 -- 172.21.15.37:0/218488388 wait complete. 2024-06-10T01:09:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.298+0000 7fa6e5909640 1 Processor -- start 2024-06-10T01:09:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.298+0000 7fa6e5909640 1 -- start start 2024-06-10T01:09:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.298+0000 7fa6e5909640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6e00ffcf0 0x7fa6e0102630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.298+0000 7fa6e5909640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6e0102cc0 0x7fa6e0100c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.299+0000 7fa6e5909640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6e0103680 0x7fa6e01011b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.299+0000 7fa6e5909640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e010d6c0 con 0x7fa6e0103680 2024-06-10T01:09:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.299+0000 7fa6e5909640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e010d540 con 0x7fa6e0102cc0 2024-06-10T01:09:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.299+0000 7fa6e5909640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e010d840 con 0x7fa6e00ffcf0 2024-06-10T01:09:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.299+0000 7fa6df7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6e0103680 0x7fa6e01011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.299+0000 7fa6df7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6e0103680 0x7fa6e01011b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53366/0 (socket says 172.21.15.37:53366) 2024-06-10T01:09:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.299+0000 7fa6df7fe640 1 -- 172.21.15.37:0/1752053896 learned_addr learned my addr 172.21.15.37:0/1752053896 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.299+0000 7fa6deffd640 1 --2- 172.21.15.37:0/1752053896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6e00ffcf0 0x7fa6e0102630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.299+0000 7fa6de7fc640 1 --2- 172.21.15.37:0/1752053896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6e0102cc0 0x7fa6e0100c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.299+0000 7fa6df7fe640 1 -- 172.21.15.37:0/1752053896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6e00ffcf0 msgr2=0x7fa6e0102630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.299+0000 7fa6df7fe640 1 --2- 172.21.15.37:0/1752053896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6e00ffcf0 0x7fa6e0102630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.299+0000 7fa6df7fe640 1 -- 172.21.15.37:0/1752053896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6e0102cc0 msgr2=0x7fa6e0100c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.299+0000 7fa6df7fe640 1 --2- 172.21.15.37:0/1752053896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6e0102cc0 0x7fa6e0100c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.299+0000 7fa6df7fe640 1 -- 172.21.15.37:0/1752053896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e0101a40 con 0x7fa6e0103680 2024-06-10T01:09:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.299+0000 7fa6deffd640 1 --2- 172.21.15.37:0/1752053896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6e00ffcf0 0x7fa6e0102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:09:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.299+0000 7fa6df7fe640 1 --2- 172.21.15.37:0/1752053896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6e0103680 0x7fa6e01011b0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800cad0 tx=0x7fa6d800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.299+0000 7fa6e4907640 1 -- 172.21.15.37:0/1752053896 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d800ae20 con 0x7fa6e0103680 2024-06-10T01:09:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.299+0000 7fa6e4907640 1 -- 172.21.15.37:0/1752053896 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6d8005020 con 0x7fa6e0103680 2024-06-10T01:09:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.299+0000 7fa6e5909640 1 -- 172.21.15.37:0/1752053896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6e01a1d60 con 0x7fa6e0103680 2024-06-10T01:09:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.299+0000 7fa6e4907640 1 -- 172.21.15.37:0/1752053896 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d800f6f0 con 0x7fa6e0103680 2024-06-10T01:09:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.299+0000 7fa6e5909640 1 -- 172.21.15.37:0/1752053896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6e01a21f0 con 0x7fa6e0103680 2024-06-10T01:09:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.302+0000 7fa6e4907640 1 -- 172.21.15.37:0/1752053896 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa6d80051d0 con 0x7fa6e0103680 2024-06-10T01:09:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.302+0000 7fa6e860f740 1 Processor -- start 2024-06-10T01:09:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.302+0000 7fa6e860f740 1 -- start start 2024-06-10T01:09:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.302+0000 7fa6e860f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558887881010 0x558887881430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.302+0000 7fa6e860f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558887881a00 0x558887886500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.302+0000 7fa6e860f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558887886b70 0x558887889000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.302+0000 7fa6e860f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5588877e44f0 con 0x558887886b70 2024-06-10T01:09:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.302+0000 7fa6e860f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558887782870 con 0x558887881010 2024-06-10T01:09:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.302+0000 7fa6e860f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5588877e04b0 con 0x558887881a00 2024-06-10T01:09:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.303+0000 7fa6de7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558887886b70 0x558887889000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.303+0000 7fa6de7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558887886b70 0x558887889000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53376/0 (socket says 172.21.15.37:53376) 2024-06-10T01:09:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.303+0000 7fa6de7fc640 1 -- 172.21.15.37:0/2668401446 learned_addr learned my addr 172.21.15.37:0/2668401446 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.303+0000 7fa6de7fc640 1 --2- 172.21.15.37:0/2668401446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558887881010 0x558887881430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.303+0000 7fa6deffd640 1 --2- 172.21.15.37:0/2668401446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558887881a00 0x558887886500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.303+0000 7fa6e4907640 1 --2- 172.21.15.37:0/1752053896 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa6c80a2c30 0x7fa6c80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.303+0000 7fa6de7fc640 1 -- 172.21.15.37:0/2668401446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558887881a00 msgr2=0x558887886500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.303+0000 7fa6de7fc640 1 --2- 172.21.15.37:0/2668401446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558887881a00 0x558887886500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.303+0000 7fa6de7fc640 1 -- 172.21.15.37:0/2668401446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558887881010 msgr2=0x558887881430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.303+0000 7fa6de7fc640 1 --2- 172.21.15.37:0/2668401446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558887881010 0x558887881430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.303+0000 7fa6df7fe640 1 --2- 172.21.15.37:0/1752053896 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa6c80a2c30 0x7fa6c80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.303+0000 7fa6de7fc640 1 -- 172.21.15.37:0/2668401446 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55888779e9b0 con 0x558887886b70 2024-06-10T01:09:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.303+0000 7fa6deffd640 1 --2- 172.21.15.37:0/2668401446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558887881a00 0x558887886500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:09:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.303+0000 7fa6e4907640 1 -- 172.21.15.37:0/1752053896 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fa6d80cf8e0 con 0x7fa6e0103680 2024-06-10T01:09:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.303+0000 7fa6de7fc640 1 --2- 172.21.15.37:0/2668401446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558887886b70 0x558887889000 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fa6d400bfd0 tx=0x7fa6d400b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d3ae5c4f6c2cc0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.303+0000 7fa6c5ffb640 1 -- 172.21.15.37:0/2668401446 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d400e950 con 0x558887886b70 2024-06-10T01:09:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.303+0000 7fa6c5ffb640 1 -- 172.21.15.37:0/2668401446 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6d4010430 con 0x558887886b70 2024-06-10T01:09:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.303+0000 7fa6c5ffb640 1 -- 172.21.15.37:0/2668401446 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d400f620 con 0x558887886b70 2024-06-10T01:09:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.304+0000 7fa6e860f740 1 -- 172.21.15.37:0/2668401446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558887886b70 msgr2=0x558887889000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.304+0000 7fa6df7fe640 1 --2- 172.21.15.37:0/1752053896 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa6c80a2c30 0x7fa6c80a50f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa6d80986c0 tx=0x7fa6d806ac70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.304+0000 7fa6e860f740 1 --2- 172.21.15.37:0/2668401446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558887886b70 0x558887889000 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fa6d400bfd0 tx=0x7fa6d400b460 comp rx=0 tx=0).stop 2024-06-10T01:09:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.304+0000 7fa6e860f740 1 -- 172.21.15.37:0/2668401446 shutdown_connections 2024-06-10T01:09:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.304+0000 7fa6e860f740 1 --2- 172.21.15.37:0/2668401446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558887886b70 0x558887889000 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.304+0000 7fa6e860f740 1 --2- 172.21.15.37:0/2668401446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558887881a00 0x558887886500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.304+0000 7fa6e860f740 1 --2- 172.21.15.37:0/2668401446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558887881010 0x558887881430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.304+0000 7fa6e860f740 1 -- 172.21.15.37:0/2668401446 >> 172.21.15.37:0/2668401446 conn(0x55888787c680 msgr2=0x55888787ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.304+0000 7fa6e860f740 1 -- 172.21.15.37:0/2668401446 shutdown_connections 2024-06-10T01:09:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.304+0000 7fa6e860f740 1 -- 172.21.15.37:0/2668401446 wait complete. 2024-06-10T01:09:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.304+0000 7fa6e860f740 1 -- start start 2024-06-10T01:09:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.305+0000 7fa6e860f740 1 Processor -- start 2024-06-10T01:09:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.305+0000 7fa6e860f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558887881010 0x55888788b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.305+0000 7fa6e860f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558887881a00 0x55888788b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.305+0000 7fa6e860f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558887886b70 0x558887892780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.305+0000 7fa6e860f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558887889b20 con 0x558887886b70 2024-06-10T01:09:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.305+0000 7fa6e860f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55888776dae0 con 0x558887881a00 2024-06-10T01:09:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.305+0000 7fa6e860f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558887889ca0 con 0x558887881010 2024-06-10T01:09:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.305+0000 7fa6e860f740 10 client.0 _ll_register_callbacks cb 0x5588877a69b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.305+0000 7fa6de7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558887886b70 0x558887892780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.305+0000 7fa6de7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558887886b70 0x558887892780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53384/0 (socket says 172.21.15.37:53384) 2024-06-10T01:09:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.305+0000 7fa6de7fc640 1 -- 172.21.15.37:0/2230943483 learned_addr learned my addr 172.21.15.37:0/2230943483 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.305+0000 7fa6deffd640 1 --2- 172.21.15.37:0/2230943483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558887881a00 0x55888788b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.305+0000 7fa6de7fc640 1 --2- 172.21.15.37:0/2230943483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558887881010 0x55888788b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.305+0000 7fa6de7fc640 1 -- 172.21.15.37:0/2230943483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558887881010 msgr2=0x55888788b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.305+0000 7fa6de7fc640 1 --2- 172.21.15.37:0/2230943483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558887881010 0x55888788b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.305+0000 7fa6de7fc640 1 -- 172.21.15.37:0/2230943483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558887881a00 msgr2=0x55888788b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.305+0000 7fa6de7fc640 1 --2- 172.21.15.37:0/2230943483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558887881a00 0x55888788b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.305+0000 7fa6de7fc640 1 -- 172.21.15.37:0/2230943483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588874f8210 con 0x558887886b70 2024-06-10T01:09:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.306+0000 7fa6de7fc640 1 --2- 172.21.15.37:0/2230943483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558887886b70 0x558887892780 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4001eb0 tx=0x7fa6d4005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3663ac1cba845e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.306+0000 7fa6beffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:09:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.306+0000 7fa6beffd640 1 -- 172.21.15.37:0/2230943483 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4022030 con 0x558887886b70 2024-06-10T01:09:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.306+0000 7fa6beffd640 1 -- 172.21.15.37:0/2230943483 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fa6d4023070 con 0x558887886b70 2024-06-10T01:09:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.306+0000 7fa6e860f740 1 -- 172.21.15.37:0/2230943483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6d4010430 con 0x558887886b70 2024-06-10T01:09:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.307+0000 7fa6beffd640 1 -- 172.21.15.37:0/2230943483 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa6d40105e0 con 0x558887886b70 2024-06-10T01:09:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.307+0000 7fa6e860f740 10 client.17739 fetch_fsmap learned FSMap version 31 2024-06-10T01:09:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.307+0000 7fa6e860f740 1 -- 172.21.15.37:0/2230943483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5588875c4620 con 0x558887886b70 2024-06-10T01:09:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.307+0000 7fa6beffd640 1 -- 172.21.15.37:0/2230943483 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6d4010430 con 0x558887886b70 2024-06-10T01:09:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.307+0000 7fa6beffd640 1 -- 172.21.15.37:0/2230943483 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fa6d400f5d0 con 0x558887886b70 2024-06-10T01:09:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.307+0000 7fa6e860f740 10 client.17739 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:09:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.307+0000 7fa6e860f740 10 client.17739 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.307+0000 7fa6e860f740 20 client.17739 populate_metadata read hostname 'smithi037' 2024-06-10T01:09:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.307+0000 7fa6e860f740 1 --2- 172.21.15.37:0/2230943483 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558887896d20 0x558887899110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.307+0000 7fa6e860f740 4 client.17739 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:09:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.307+0000 7fa6e860f740 1 -- 172.21.15.37:0/2230943483 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588877f7f10 con 0x558887896d20 2024-06-10T01:09:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.308+0000 7fa6deffd640 1 --2- 172.21.15.37:0/2230943483 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558887896d20 0x558887899110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.308+0000 7fa6beffd640 10 client.17739 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.308+0000 7fa6deffd640 1 --2- 172.21.15.37:0/2230943483 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558887896d20 0x558887899110 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33cb4c0500ab40df server_cookie=57eb5526d69e8a3 in_seq=0 out_seq=0 2024-06-10T01:09:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.309+0000 7fa6beffd640 1 -- 172.21.15.37:0/2230943483 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5588877f7f10 con 0x558887896d20 2024-06-10T01:09:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.309+0000 7fa6beffd640 10 client.17739 handle_command_reply: tid=0 2024-06-10T01:09:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.309+0000 7fa6e860f740 1 client.17739 shutdown 2024-06-10T01:09:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.309+0000 7fa6e860f740 1 -- 172.21.15.37:0/2230943483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558887886b70 msgr2=0x558887892780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.309+0000 7fa6e860f740 1 --2- 172.21.15.37:0/2230943483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558887886b70 0x558887892780 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4001eb0 tx=0x7fa6d4005710 comp rx=0 tx=0).stop 2024-06-10T01:09:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.309+0000 7fa6e860f740 1 -- 172.21.15.37:0/2230943483 shutdown_connections 2024-06-10T01:09:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.309+0000 7fa6e860f740 1 --2- 172.21.15.37:0/2230943483 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558887896d20 0x558887899110 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.309+0000 7fa6beffd640 0 client.17739 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.309+0000 7fa6e860f740 1 --2- 172.21.15.37:0/2230943483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558887886b70 0x558887892780 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.309+0000 7fa6e860f740 1 --2- 172.21.15.37:0/2230943483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558887881a00 0x55888788b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.309+0000 7fa6e860f740 1 --2- 172.21.15.37:0/2230943483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558887881010 0x55888788b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.309+0000 7fa6e860f740 1 -- 172.21.15.37:0/2230943483 >> 172.21.15.37:0/2230943483 conn(0x55888787c680 msgr2=0x55888787b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.309+0000 7fa6e860f740 1 -- 172.21.15.37:0/2230943483 shutdown_connections 2024-06-10T01:09:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.310+0000 7fa6e860f740 1 -- 172.21.15.37:0/2230943483 wait complete. 2024-06-10T01:09:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.310+0000 7fa6e860f740 20 client.17739 trim_cache size 0 max 16384 2024-06-10T01:09:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.325+0000 7fa6e860f740 1 Processor -- start 2024-06-10T01:09:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.325+0000 7fa6e860f740 1 -- start start 2024-06-10T01:09:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.325+0000 7fa6e860f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558887881010 0x558887882290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.325+0000 7fa6e860f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558887881a00 0x558887892740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.325+0000 7fa6e860f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558887886b70 0x558887886230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.325+0000 7fa6e860f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e01af2a0 con 0x558887881a00 2024-06-10T01:09:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.325+0000 7fa6e860f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e01acb70 con 0x558887886b70 2024-06-10T01:09:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.325+0000 7fa6e860f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e01aa9c0 con 0x558887881010 2024-06-10T01:09:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.325+0000 7fa6deffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558887881a00 0x558887892740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.325+0000 7fa6deffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558887881a00 0x558887892740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53390/0 (socket says 172.21.15.37:53390) 2024-06-10T01:09:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.325+0000 7fa6deffd640 1 -- 172.21.15.37:0/1422259089 learned_addr learned my addr 172.21.15.37:0/1422259089 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.325+0000 7fa6de7fc640 1 --2- 172.21.15.37:0/1422259089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558887881010 0x558887882290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.325+0000 7fa6de7fc640 1 --2- 172.21.15.37:0/1422259089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558887886b70 0x558887886230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.325+0000 7fa6deffd640 1 -- 172.21.15.37:0/1422259089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558887881010 msgr2=0x558887882290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.325+0000 7fa6deffd640 1 --2- 172.21.15.37:0/1422259089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558887881010 0x558887882290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.325+0000 7fa6deffd640 1 -- 172.21.15.37:0/1422259089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558887886b70 msgr2=0x558887886230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.325+0000 7fa6deffd640 1 --2- 172.21.15.37:0/1422259089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558887886b70 0x558887886230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.325+0000 7fa6deffd640 1 -- 172.21.15.37:0/1422259089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588878996f0 con 0x558887881a00 2024-06-10T01:09:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.326+0000 7fa6de7fc640 1 --2- 172.21.15.37:0/1422259089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558887886b70 0x558887886230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:09:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.326+0000 7fa6deffd640 1 --2- 172.21.15.37:0/1422259089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558887881a00 0x558887892740 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fa6d00096c0 tx=0x7fa6d00315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910bbf5b70538346 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.326+0000 7fa6c57fa640 1 -- 172.21.15.37:0/1422259089 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0048070 con 0x558887881a00 2024-06-10T01:09:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.326+0000 7fa6c57fa640 1 -- 172.21.15.37:0/1422259089 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6d0005500 con 0x558887881a00 2024-06-10T01:09:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.326+0000 7fa6c57fa640 1 -- 172.21.15.37:0/1422259089 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0035040 con 0x558887881a00 2024-06-10T01:09:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.326+0000 7fa6e860f740 1 -- 172.21.15.37:0/1422259089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558887881a00 msgr2=0x558887892740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.326+0000 7fa6e860f740 1 --2- 172.21.15.37:0/1422259089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558887881a00 0x558887892740 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fa6d00096c0 tx=0x7fa6d00315b0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.327+0000 7fa6e860f740 1 -- 172.21.15.37:0/1422259089 shutdown_connections 2024-06-10T01:09:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.327+0000 7fa6e860f740 1 --2- 172.21.15.37:0/1422259089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558887886b70 0x558887886230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.327+0000 7fa6e860f740 1 --2- 172.21.15.37:0/1422259089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558887881a00 0x558887892740 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.327+0000 7fa6e860f740 1 --2- 172.21.15.37:0/1422259089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558887881010 0x558887882290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.327+0000 7fa6e860f740 1 -- 172.21.15.37:0/1422259089 >> 172.21.15.37:0/1422259089 conn(0x55888787c680 msgr2=0x55888787ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.327+0000 7fa6e860f740 1 -- 172.21.15.37:0/1422259089 shutdown_connections 2024-06-10T01:09:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.327+0000 7fa6e860f740 1 -- 172.21.15.37:0/1422259089 wait complete. 2024-06-10T01:09:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.327+0000 7fa6e860f740 1 -- start start 2024-06-10T01:09:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.327+0000 7fa6e860f740 1 Processor -- start 2024-06-10T01:09:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.327+0000 7fa6e860f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558887881010 0x55888788fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.327+0000 7fa6e860f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558887881a00 0x5588878900b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.327+0000 7fa6e860f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558887886b70 0x5588878910c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.327+0000 7fa6e860f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e01ab030 con 0x558887881010 2024-06-10T01:09:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.327+0000 7fa6e860f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e01ac4a0 con 0x558887886b70 2024-06-10T01:09:51.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.327+0000 7fa6e860f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e01ab700 con 0x558887881a00 2024-06-10T01:09:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.327+0000 7fa6e860f740 10 client.0 _ll_register_callbacks cb 0x7fa6d400bea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.327+0000 7fa6de7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558887881010 0x55888788fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.327+0000 7fa6de7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558887881010 0x55888788fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53404/0 (socket says 172.21.15.37:53404) 2024-06-10T01:09:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.327+0000 7fa6de7fc640 1 -- 172.21.15.37:0/4078978084 learned_addr learned my addr 172.21.15.37:0/4078978084 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.328+0000 7fa6deffd640 1 --2- 172.21.15.37:0/4078978084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558887881a00 0x5588878900b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.328+0000 7fa6de7fc640 1 --2- 172.21.15.37:0/4078978084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558887886b70 0x5588878910c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.328+0000 7fa6de7fc640 1 -- 172.21.15.37:0/4078978084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558887881a00 msgr2=0x5588878900b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.328+0000 7fa6de7fc640 1 --2- 172.21.15.37:0/4078978084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558887881a00 0x5588878900b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.328+0000 7fa6de7fc640 1 -- 172.21.15.37:0/4078978084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558887886b70 msgr2=0x5588878910c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.328+0000 7fa6de7fc640 1 --2- 172.21.15.37:0/4078978084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558887886b70 0x5588878910c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.328+0000 7fa6de7fc640 1 -- 172.21.15.37:0/4078978084 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588878916b0 con 0x558887881010 2024-06-10T01:09:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.328+0000 7fa6de7fc640 1 --2- 172.21.15.37:0/4078978084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558887881010 0x55888788fb70 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4013fd0 tx=0x7fa6d4005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d511e6d94051726b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.328+0000 7fa6beffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:09:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.328+0000 7fa6beffd640 1 -- 172.21.15.37:0/4078978084 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4022040 con 0x558887881010 2024-06-10T01:09:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.328+0000 7fa6beffd640 1 -- 172.21.15.37:0/4078978084 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fa6d400f5d0 con 0x558887881010 2024-06-10T01:09:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.328+0000 7fa6e860f740 1 -- 172.21.15.37:0/4078978084 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6d0005500 con 0x558887881010 2024-06-10T01:09:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.328+0000 7fa6beffd640 1 -- 172.21.15.37:0/4078978084 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa6d4043050 con 0x558887881010 2024-06-10T01:09:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.328+0000 7fa6e860f740 10 client.17745 fetch_fsmap learned FSMap version 31 2024-06-10T01:09:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.328+0000 7fa6e860f740 1 -- 172.21.15.37:0/4078978084 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5588878929a0 con 0x558887881010 2024-06-10T01:09:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.328+0000 7fa6beffd640 1 -- 172.21.15.37:0/4078978084 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6d0005500 con 0x558887881010 2024-06-10T01:09:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.329+0000 7fa6beffd640 1 -- 172.21.15.37:0/4078978084 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fa6d4010780 con 0x558887881010 2024-06-10T01:09:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.329+0000 7fa6e860f740 10 client.17745 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:09:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.329+0000 7fa6e860f740 10 client.17745 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.329+0000 7fa6e860f740 20 client.17745 populate_metadata read hostname 'smithi037' 2024-06-10T01:09:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.329+0000 7fa6e860f740 1 --2- 172.21.15.37:0/4078978084 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558887896d20 0x558887903fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.329+0000 7fa6e860f740 4 client.17745 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:09:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.329+0000 7fa6e860f740 1 -- 172.21.15.37:0/4078978084 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5588877f1c20 con 0x558887896d20 2024-06-10T01:09:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.329+0000 7fa6deffd640 1 --2- 172.21.15.37:0/4078978084 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558887896d20 0x558887903fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.329+0000 7fa6beffd640 10 client.17745 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.329+0000 7fa6deffd640 1 --2- 172.21.15.37:0/4078978084 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558887896d20 0x558887903fc0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fd4a628e48161ee server_cookie=8cb761e7841e1f4d in_seq=0 out_seq=0 2024-06-10T01:09:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.349+0000 7fa6beffd640 1 -- 172.21.15.37:0/4078978084 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5588877f1c20 con 0x558887896d20 2024-06-10T01:09:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.349+0000 7fa6beffd640 10 client.17745 handle_command_reply: tid=0 2024-06-10T01:09:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.349+0000 7fa6e860f740 1 client.17745 shutdown 2024-06-10T01:09:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.349+0000 7fa6e860f740 1 -- 172.21.15.37:0/4078978084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558887881010 msgr2=0x55888788fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.349+0000 7fa6e860f740 1 --2- 172.21.15.37:0/4078978084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558887881010 0x55888788fb70 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4013fd0 tx=0x7fa6d4005710 comp rx=0 tx=0).stop 2024-06-10T01:09:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.349+0000 7fa6e860f740 1 -- 172.21.15.37:0/4078978084 shutdown_connections 2024-06-10T01:09:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.349+0000 7fa6e860f740 1 --2- 172.21.15.37:0/4078978084 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x558887896d20 0x558887903fc0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.349+0000 7fa6e860f740 1 --2- 172.21.15.37:0/4078978084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558887886b70 0x5588878910c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.349+0000 7fa6e860f740 1 --2- 172.21.15.37:0/4078978084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558887881a00 0x5588878900b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.349+0000 7fa6e860f740 1 --2- 172.21.15.37:0/4078978084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558887881010 0x55888788fb70 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.349+0000 7fa6beffd640 0 client.17745 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.349+0000 7fa6e860f740 1 -- 172.21.15.37:0/4078978084 >> 172.21.15.37:0/4078978084 conn(0x55888787c680 msgr2=0x55888787f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.349+0000 7fa6e860f740 1 -- 172.21.15.37:0/4078978084 shutdown_connections 2024-06-10T01:09:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.349+0000 7fa6e860f740 1 -- 172.21.15.37:0/4078978084 wait complete. 2024-06-10T01:09:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.349+0000 7fa6e860f740 20 client.17745 trim_cache size 0 max 16384 2024-06-10T01:09:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-06-10T01:09:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-06-10T01:09:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-06-10T01:09:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.350+0000 7fa6e5909640 1 -- 172.21.15.37:0/1752053896 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa6c80a2c30 msgr2=0x7fa6c80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.350+0000 7fa6e5909640 1 --2- 172.21.15.37:0/1752053896 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa6c80a2c30 0x7fa6c80a50f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa6d80986c0 tx=0x7fa6d806ac70 comp rx=0 tx=0).stop 2024-06-10T01:09:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.350+0000 7fa6e5909640 1 -- 172.21.15.37:0/1752053896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6e0103680 msgr2=0x7fa6e01011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.350+0000 7fa6e5909640 1 --2- 172.21.15.37:0/1752053896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6e0103680 0x7fa6e01011b0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800cad0 tx=0x7fa6d800cf90 comp rx=0 tx=0).stop 2024-06-10T01:09:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.350+0000 7fa6e5909640 1 -- 172.21.15.37:0/1752053896 shutdown_connections 2024-06-10T01:09:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.350+0000 7fa6e5909640 1 --2- 172.21.15.37:0/1752053896 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa6c80a2c30 0x7fa6c80a50f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.350+0000 7fa6e5909640 1 --2- 172.21.15.37:0/1752053896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6e0103680 0x7fa6e01011b0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.350+0000 7fa6e5909640 1 --2- 172.21.15.37:0/1752053896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6e0102cc0 0x7fa6e0100c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.350+0000 7fa6e5909640 1 --2- 172.21.15.37:0/1752053896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6e00ffcf0 0x7fa6e0102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.350+0000 7fa6e5909640 1 -- 172.21.15.37:0/1752053896 >> 172.21.15.37:0/1752053896 conn(0x7fa6e00fb620 msgr2=0x7fa6e0105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.350+0000 7fa6e5909640 1 -- 172.21.15.37:0/1752053896 shutdown_connections 2024-06-10T01:09:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:09:51.350+0000 7fa6e5909640 1 -- 172.21.15.37:0/1752053896 wait complete. 2024-06-10T01:09:51.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:51 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2525317965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:09:51.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:51 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2525317965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:09:51.629 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:51 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2525317965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:09:51.657 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24472 quiesce db --set-id 9e1aa52e --release --await' 2024-06-10T01:09:51.657 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24472 quiesce db --set-id 9e1aa52e --release --await 2024-06-10T01:09:52.390 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:52 smithi037 ceph-mon[32192]: pgmap v826: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-06-10T01:09:52.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:52 smithi167 ceph-mon[36917]: pgmap v826: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-06-10T01:09:52.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:52 smithi094 ceph-mon[36708]: pgmap v826: 129 pgs: 129 active+clean; 2.6 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-06-10T01:09:52.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.719+0000 7fe49d0d7700 1 -- 172.21.15.37:0/4245213254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe498107840 msgr2=0x7fe49810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:52.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.719+0000 7fe49d0d7700 1 --2- 172.21.15.37:0/4245213254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe498107840 0x7fe49810f3c0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fe49000b0d0 tx=0x7fe49000b3e0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.719+0000 7fe49d0d7700 1 -- 172.21.15.37:0/4245213254 shutdown_connections 2024-06-10T01:09:52.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.719+0000 7fe49d0d7700 1 --2- 172.21.15.37:0/4245213254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe498107840 0x7fe49810f3c0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.719+0000 7fe49d0d7700 1 --2- 172.21.15.37:0/4245213254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe498102740 0x7fe498102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.719+0000 7fe49d0d7700 1 --2- 172.21.15.37:0/4245213254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4981030f0 0x7fe498107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.719+0000 7fe49d0d7700 1 -- 172.21.15.37:0/4245213254 >> 172.21.15.37:0/4245213254 conn(0x7fe4980795f0 msgr2=0x7fe498079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:52.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.719+0000 7fe49d0d7700 1 -- 172.21.15.37:0/4245213254 shutdown_connections 2024-06-10T01:09:52.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.719+0000 7fe49d0d7700 1 -- 172.21.15.37:0/4245213254 wait complete. 2024-06-10T01:09:52.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.720+0000 7fe49d0d7700 1 Processor -- start 2024-06-10T01:09:52.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.720+0000 7fe49d0d7700 1 -- start start 2024-06-10T01:09:52.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.720+0000 7fe49d0d7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe498102740 0x7fe49819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:52.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.720+0000 7fe49d0d7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4981030f0 0x7fe49819ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:52.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.720+0000 7fe49d0d7700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe498107840 0x7fe498198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:52.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.720+0000 7fe49d0d7700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe498112760 con 0x7fe4981030f0 2024-06-10T01:09:52.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.720+0000 7fe49d0d7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe4981125e0 con 0x7fe498102740 2024-06-10T01:09:52.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.720+0000 7fe49d0d7700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe4981128e0 con 0x7fe498107840 2024-06-10T01:09:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.721+0000 7fe49659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4981030f0 0x7fe49819ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.721+0000 7fe496d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe498102740 0x7fe49819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.721+0000 7fe496d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe498102740 0x7fe49819e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:46012/0 (socket says 172.21.15.37:46012) 2024-06-10T01:09:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.721+0000 7fe496d9d700 1 -- 172.21.15.37:0/2063310291 learned_addr learned my addr 172.21.15.37:0/2063310291 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.721+0000 7fe496d9d700 1 -- 172.21.15.37:0/2063310291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe498107840 msgr2=0x7fe498198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.721+0000 7fe49759e700 1 --2- 172.21.15.37:0/2063310291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe498107840 0x7fe498198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.721+0000 7fe496d9d700 1 --2- 172.21.15.37:0/2063310291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe498107840 0x7fe498198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.721+0000 7fe496d9d700 1 -- 172.21.15.37:0/2063310291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4981030f0 msgr2=0x7fe49819ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:52.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.721+0000 7fe496d9d700 1 --2- 172.21.15.37:0/2063310291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4981030f0 0x7fe49819ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.721+0000 7fe496d9d700 1 -- 172.21.15.37:0/2063310291 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe498199060 con 0x7fe498102740 2024-06-10T01:09:52.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.721+0000 7fe496d9d700 1 --2- 172.21.15.37:0/2063310291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe498102740 0x7fe49819e740 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe48800e9a0 tx=0x7fe48800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:52.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.722+0000 7fe47ffff700 1 -- 172.21.15.37:0/2063310291 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48800cbd0 con 0x7fe498102740 2024-06-10T01:09:52.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.722+0000 7fe47ffff700 1 -- 172.21.15.37:0/2063310291 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe48800cd70 con 0x7fe498102740 2024-06-10T01:09:52.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.722+0000 7fe47ffff700 1 -- 172.21.15.37:0/2063310291 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe488018750 con 0x7fe498102740 2024-06-10T01:09:52.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.722+0000 7fe49d0d7700 1 -- 172.21.15.37:0/2063310291 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe498199350 con 0x7fe498102740 2024-06-10T01:09:52.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.722+0000 7fe49d0d7700 1 -- 172.21.15.37:0/2063310291 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4981a5610 con 0x7fe498102740 2024-06-10T01:09:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.725+0000 7fe47ffff700 1 -- 172.21.15.37:0/2063310291 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe488010430 con 0x7fe498102740 2024-06-10T01:09:52.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.726+0000 7fe47ffff700 1 --2- 172.21.15.37:0/2063310291 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe4800a2c20 0x7fe4800a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:52.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.726+0000 7fe49659c700 1 --2- 172.21.15.37:0/2063310291 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe4800a2c20 0x7fe4800a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:52.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.726+0000 7fe47ffff700 1 -- 172.21.15.37:0/2063310291 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fe4880cf750 con 0x7fe498102740 2024-06-10T01:09:52.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.726+0000 7fe4a795ab80 1 Processor -- start 2024-06-10T01:09:52.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.726+0000 7fe4a795ab80 1 -- start start 2024-06-10T01:09:52.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.726+0000 7fe4a795ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561746555370 0x561746555750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:52.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.726+0000 7fe4a795ab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561746555c90 0x56174654f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:52.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.727+0000 7fe4a795ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56174654fbb0 0x56174654fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:52.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.727+0000 7fe49659c700 1 --2- 172.21.15.37:0/2063310291 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe4800a2c20 0x7fe4800a50e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe498108c90 tx=0x7fe48c0073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:52.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.727+0000 7fe4a795ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56174643df60 con 0x56174654fbb0 2024-06-10T01:09:52.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.727+0000 7fe4a795ab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56174649c850 con 0x561746555c90 2024-06-10T01:09:52.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.727+0000 7fe4a795ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561746404460 con 0x561746555370 2024-06-10T01:09:52.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.727+0000 7fe49759e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56174654fbb0 0x56174654fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:52.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.727+0000 7fe49759e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56174654fbb0 0x56174654fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36772/0 (socket says 172.21.15.37:36772) 2024-06-10T01:09:52.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.727+0000 7fe49659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561746555c90 0x56174654f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:52.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.727+0000 7fe49759e700 1 -- 172.21.15.37:0/601115513 learned_addr learned my addr 172.21.15.37:0/601115513 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:52.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.727+0000 7fe496d9d700 1 --2- 172.21.15.37:0/601115513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561746555370 0x561746555750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:52.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.727+0000 7fe49759e700 1 -- 172.21.15.37:0/601115513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561746555370 msgr2=0x561746555750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:52.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.727+0000 7fe49759e700 1 --2- 172.21.15.37:0/601115513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561746555370 0x561746555750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.727+0000 7fe49759e700 1 -- 172.21.15.37:0/601115513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561746555c90 msgr2=0x56174654f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:52.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.727+0000 7fe49759e700 1 --2- 172.21.15.37:0/601115513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561746555c90 0x56174654f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.727+0000 7fe49759e700 1 -- 172.21.15.37:0/601115513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617463eb8d0 con 0x56174654fbb0 2024-06-10T01:09:52.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.727+0000 7fe49659c700 1 --2- 172.21.15.37:0/601115513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561746555c90 0x56174654f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:52.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.727+0000 7fe49759e700 1 --2- 172.21.15.37:0/601115513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56174654fbb0 0x56174654fff0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fe490009d90 tx=0x7fe490009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3e82e0930d2027 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:52.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.727+0000 7fe47d7fa700 1 -- 172.21.15.37:0/601115513 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490021070 con 0x56174654fbb0 2024-06-10T01:09:52.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.728+0000 7fe47d7fa700 1 -- 172.21.15.37:0/601115513 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4900074e0 con 0x56174654fbb0 2024-06-10T01:09:52.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.728+0000 7fe47d7fa700 1 -- 172.21.15.37:0/601115513 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49001b630 con 0x56174654fbb0 2024-06-10T01:09:52.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.728+0000 7fe4a795ab80 1 -- 172.21.15.37:0/601115513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56174654fbb0 msgr2=0x56174654fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:52.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.728+0000 7fe4a795ab80 1 --2- 172.21.15.37:0/601115513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56174654fbb0 0x56174654fff0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fe490009d90 tx=0x7fe490009dc0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.728+0000 7fe4a795ab80 1 -- 172.21.15.37:0/601115513 shutdown_connections 2024-06-10T01:09:52.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.728+0000 7fe4a795ab80 1 --2- 172.21.15.37:0/601115513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56174654fbb0 0x56174654fff0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.728+0000 7fe4a795ab80 1 --2- 172.21.15.37:0/601115513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561746555370 0x561746555750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.728+0000 7fe4a795ab80 1 --2- 172.21.15.37:0/601115513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561746555c90 0x56174654f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.728+0000 7fe4a795ab80 1 -- 172.21.15.37:0/601115513 >> 172.21.15.37:0/601115513 conn(0x561746478080 msgr2=0x561746546220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:52.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.728+0000 7fe4a795ab80 1 -- 172.21.15.37:0/601115513 shutdown_connections 2024-06-10T01:09:52.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.728+0000 7fe4a795ab80 1 -- 172.21.15.37:0/601115513 wait complete. 2024-06-10T01:09:52.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.729+0000 7fe4a795ab80 1 -- start start 2024-06-10T01:09:52.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.729+0000 7fe4a795ab80 1 Processor -- start 2024-06-10T01:09:52.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.729+0000 7fe4a795ab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56174654fbb0 0x56174655b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:52.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.729+0000 7fe4a795ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561746555370 0x561746558c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:52.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.729+0000 7fe4a795ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561746555c90 0x561746559190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:52.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.729+0000 7fe4a795ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56174643cfc0 con 0x561746555370 2024-06-10T01:09:52.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.729+0000 7fe4a795ab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561746498810 con 0x56174654fbb0 2024-06-10T01:09:52.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.729+0000 7fe4a795ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56174655c620 con 0x561746555c90 2024-06-10T01:09:52.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.729+0000 7fe49659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561746555370 0x561746558c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:52.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.729+0000 7fe49659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561746555370 0x561746558c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36774/0 (socket says 172.21.15.37:36774) 2024-06-10T01:09:52.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.729+0000 7fe49659c700 1 -- 172.21.15.37:0/1261776415 learned_addr learned my addr 172.21.15.37:0/1261776415 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:52.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.729+0000 7fe496d9d700 1 --2- 172.21.15.37:0/1261776415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56174654fbb0 0x56174655b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:52.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.730+0000 7fe49759e700 1 --2- 172.21.15.37:0/1261776415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561746555c90 0x561746559190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:52.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.730+0000 7fe4a795ab80 10 client.0 _ll_register_callbacks cb 0x5617464325b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:52.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.730+0000 7fe49659c700 1 -- 172.21.15.37:0/1261776415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561746555c90 msgr2=0x561746559190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:52.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.730+0000 7fe49659c700 1 --2- 172.21.15.37:0/1261776415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561746555c90 0x561746559190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.730+0000 7fe49659c700 1 -- 172.21.15.37:0/1261776415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56174654fbb0 msgr2=0x56174655b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:52.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.730+0000 7fe49659c700 1 --2- 172.21.15.37:0/1261776415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56174654fbb0 0x56174655b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.730+0000 7fe49659c700 1 -- 172.21.15.37:0/1261776415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561746462750 con 0x561746555370 2024-06-10T01:09:52.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.730+0000 7fe496d9d700 1 --2- 172.21.15.37:0/1261776415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56174654fbb0 0x56174655b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:52.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.730+0000 7fe49759e700 1 --2- 172.21.15.37:0/1261776415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561746555c90 0x561746559190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:09:52.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.730+0000 7fe4a795ab80 1 -- 172.21.15.37:0/1261776415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4900074e0 con 0x561746555370 2024-06-10T01:09:52.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.730+0000 7fe49659c700 1 --2- 172.21.15.37:0/1261776415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561746555370 0x561746558c50 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0042b0 tx=0x7fe48c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144fb204a7221949 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:52.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.730+0000 7fe47aff5700 10 client.17757 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:09:52.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.730+0000 7fe47aff5700 1 -- 172.21.15.37:0/1261776415 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c02e050 con 0x561746555370 2024-06-10T01:09:52.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.730+0000 7fe47aff5700 1 -- 172.21.15.37:0/1261776415 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fe48c003a90 con 0x561746555370 2024-06-10T01:09:52.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.730+0000 7fe47aff5700 1 -- 172.21.15.37:0/1261776415 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe48c024070 con 0x561746555370 2024-06-10T01:09:52.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.731+0000 7fe4a795ab80 10 client.17757 fetch_fsmap learned FSMap version 31 2024-06-10T01:09:52.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.731+0000 7fe4a795ab80 1 -- 172.21.15.37:0/1261776415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=31}) -- 0x561746566cd0 con 0x561746555370 2024-06-10T01:09:52.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.731+0000 7fe47aff5700 1 -- 172.21.15.37:0/1261776415 <== mon.0 v2:172.21.15.37:3300/0 4 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fe48c003c90 con 0x561746555370 2024-06-10T01:09:52.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.731+0000 7fe4a795ab80 10 client.17757 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:09:52.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.731+0000 7fe4a795ab80 10 client.17757 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:52.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.731+0000 7fe4a795ab80 20 client.17757 populate_metadata read hostname 'smithi037' 2024-06-10T01:09:52.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.731+0000 7fe4a795ab80 1 --2- 172.21.15.37:0/1261776415 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561746569200 0x56174656b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:52.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.731+0000 7fe4a795ab80 4 client.17757 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:09:52.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.731+0000 7fe4a795ab80 1 -- 172.21.15.37:0/1261776415 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617464a9d50 con 0x561746569200 2024-06-10T01:09:52.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.731+0000 7fe496d9d700 1 --2- 172.21.15.37:0/1261776415 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561746569200 0x56174656b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:52.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.732+0000 7fe496d9d700 1 --2- 172.21.15.37:0/1261776415 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561746569200 0x56174656b5f0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dce3705a3c66726 server_cookie=868c59d2b68b03e7 in_seq=0 out_seq=0 2024-06-10T01:09:52.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.732+0000 7fe47aff5700 10 client.17757 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:52.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.733+0000 7fe47aff5700 1 -- 172.21.15.37:0/1261776415 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5617464a9d50 con 0x561746569200 2024-06-10T01:09:52.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.733+0000 7fe47aff5700 10 client.17757 handle_command_reply: tid=0 2024-06-10T01:09:52.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.733+0000 7fe4a795ab80 1 client.17757 shutdown 2024-06-10T01:09:52.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.733+0000 7fe4a795ab80 1 -- 172.21.15.37:0/1261776415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561746555370 msgr2=0x561746558c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:52.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.733+0000 7fe4a795ab80 1 --2- 172.21.15.37:0/1261776415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561746555370 0x561746558c50 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0042b0 tx=0x7fe48c0047c0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.733+0000 7fe4a795ab80 1 -- 172.21.15.37:0/1261776415 shutdown_connections 2024-06-10T01:09:52.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.733+0000 7fe4a795ab80 1 --2- 172.21.15.37:0/1261776415 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x561746569200 0x56174656b5f0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.733+0000 7fe4a795ab80 1 --2- 172.21.15.37:0/1261776415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561746555c90 0x561746559190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.733+0000 7fe4a795ab80 1 --2- 172.21.15.37:0/1261776415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56174654fbb0 0x56174655b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.733+0000 7fe47aff5700 0 client.17757 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:52.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.733+0000 7fe4a795ab80 1 --2- 172.21.15.37:0/1261776415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561746555370 0x561746558c50 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.733+0000 7fe4a795ab80 1 -- 172.21.15.37:0/1261776415 >> 172.21.15.37:0/1261776415 conn(0x561746478080 msgr2=0x561746545700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:52.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.733+0000 7fe4a795ab80 1 -- 172.21.15.37:0/1261776415 shutdown_connections 2024-06-10T01:09:52.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.733+0000 7fe4a795ab80 1 -- 172.21.15.37:0/1261776415 wait complete. 2024-06-10T01:09:52.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.733+0000 7fe4a795ab80 20 client.17757 trim_cache size 0 max 16384 2024-06-10T01:09:52.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.750+0000 7fe4a795ab80 1 Processor -- start 2024-06-10T01:09:52.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.750+0000 7fe4a795ab80 1 -- start start 2024-06-10T01:09:52.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.750+0000 7fe4a795ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174654fbb0 0x561746557df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:52.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.750+0000 7fe4a795ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561746555370 0x561746545e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:52.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.750+0000 7fe4a795ab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561746555c90 0x5617465463c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:52.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.750+0000 7fe4a795ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe4981ae990 con 0x561746555370 2024-06-10T01:09:52.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.750+0000 7fe4a795ab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe4981ac220 con 0x561746555c90 2024-06-10T01:09:52.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.750+0000 7fe4a795ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe4981aa070 con 0x56174654fbb0 2024-06-10T01:09:52.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.750+0000 7fe49659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561746555370 0x561746545e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:52.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.750+0000 7fe49659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561746555370 0x561746545e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36790/0 (socket says 172.21.15.37:36790) 2024-06-10T01:09:52.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.750+0000 7fe49659c700 1 -- 172.21.15.37:0/2325635271 learned_addr learned my addr 172.21.15.37:0/2325635271 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:52.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.750+0000 7fe496d9d700 1 --2- 172.21.15.37:0/2325635271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174654fbb0 0x561746557df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:52.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.750+0000 7fe49759e700 1 --2- 172.21.15.37:0/2325635271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561746555c90 0x5617465463c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:52.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.750+0000 7fe49659c700 1 -- 172.21.15.37:0/2325635271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174654fbb0 msgr2=0x561746557df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:52.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.750+0000 7fe49659c700 1 --2- 172.21.15.37:0/2325635271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174654fbb0 0x561746557df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.750+0000 7fe49659c700 1 -- 172.21.15.37:0/2325635271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561746555c90 msgr2=0x5617465463c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:52.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.750+0000 7fe49659c700 1 --2- 172.21.15.37:0/2325635271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561746555c90 0x5617465463c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.750+0000 7fe49659c700 1 -- 172.21.15.37:0/2325635271 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561746546e40 con 0x561746555370 2024-06-10T01:09:52.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.751+0000 7fe49659c700 1 --2- 172.21.15.37:0/2325635271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561746555370 0x561746545e80 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fe48c000c00 tx=0x7fe48c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee97b1b497cc828c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:52.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.751+0000 7fe47cff9700 1 -- 172.21.15.37:0/2325635271 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c02e040 con 0x561746555370 2024-06-10T01:09:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.751+0000 7fe47cff9700 1 -- 172.21.15.37:0/2325635271 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4900074e0 con 0x561746555370 2024-06-10T01:09:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.751+0000 7fe47cff9700 1 -- 172.21.15.37:0/2325635271 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c029e30 con 0x561746555370 2024-06-10T01:09:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.752+0000 7fe4a795ab80 1 -- 172.21.15.37:0/2325635271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561746555370 msgr2=0x561746545e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.752+0000 7fe4a795ab80 1 --2- 172.21.15.37:0/2325635271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561746555370 0x561746545e80 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fe48c000c00 tx=0x7fe48c021000 comp rx=0 tx=0).stop 2024-06-10T01:09:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.752+0000 7fe4a795ab80 1 -- 172.21.15.37:0/2325635271 shutdown_connections 2024-06-10T01:09:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.752+0000 7fe4a795ab80 1 --2- 172.21.15.37:0/2325635271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561746555c90 0x5617465463c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.752+0000 7fe4a795ab80 1 --2- 172.21.15.37:0/2325635271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174654fbb0 0x561746557df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.752+0000 7fe4a795ab80 1 --2- 172.21.15.37:0/2325635271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561746555370 0x561746545e80 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.752+0000 7fe4a795ab80 1 -- 172.21.15.37:0/2325635271 >> 172.21.15.37:0/2325635271 conn(0x561746478080 msgr2=0x5617465566e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.752+0000 7fe4a795ab80 1 -- 172.21.15.37:0/2325635271 shutdown_connections 2024-06-10T01:09:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.752+0000 7fe4a795ab80 1 -- 172.21.15.37:0/2325635271 wait complete. 2024-06-10T01:09:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.752+0000 7fe4a795ab80 1 -- start start 2024-06-10T01:09:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.752+0000 7fe4a795ab80 1 Processor -- start 2024-06-10T01:09:52.750 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:09:52.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.753+0000 7fe4a795ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174654fbb0 0x561746561ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:52.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.753+0000 7fe4a795ab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561746555370 0x561746570630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:52.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.753+0000 7fe4a795ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561746555c90 0x561746570b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:52.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.753+0000 7fe4a795ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe4981aa6e0 con 0x561746555c90 2024-06-10T01:09:52.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.753+0000 7fe4a795ab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe4981abb50 con 0x561746555370 2024-06-10T01:09:52.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.753+0000 7fe4a795ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe4981aadb0 con 0x56174654fbb0 2024-06-10T01:09:52.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.753+0000 7fe4a795ab80 10 client.0 _ll_register_callbacks cb 0x5617464325b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:52.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.753+0000 7fe49759e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561746555c90 0x561746570b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:52.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.753+0000 7fe49759e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561746555c90 0x561746570b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36806/0 (socket says 172.21.15.37:36806) 2024-06-10T01:09:52.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.753+0000 7fe49759e700 1 -- 172.21.15.37:0/3690357203 learned_addr learned my addr 172.21.15.37:0/3690357203 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:52.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.753+0000 7fe49659c700 1 --2- 172.21.15.37:0/3690357203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561746555370 0x561746570630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:52.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.753+0000 7fe496d9d700 1 --2- 172.21.15.37:0/3690357203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174654fbb0 0x561746561ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:52.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.753+0000 7fe49759e700 1 -- 172.21.15.37:0/3690357203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174654fbb0 msgr2=0x561746561ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:52.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.753+0000 7fe49759e700 1 --2- 172.21.15.37:0/3690357203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174654fbb0 0x561746561ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.753+0000 7fe49759e700 1 -- 172.21.15.37:0/3690357203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561746555370 msgr2=0x561746570630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:52.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.753+0000 7fe49759e700 1 --2- 172.21.15.37:0/3690357203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561746555370 0x561746570630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.753+0000 7fe49759e700 1 -- 172.21.15.37:0/3690357203 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617465711b0 con 0x561746555c90 2024-06-10T01:09:52.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.753+0000 7fe496d9d700 1 --2- 172.21.15.37:0/3690357203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174654fbb0 0x561746561ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:09:52.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.753+0000 7fe49759e700 1 --2- 172.21.15.37:0/3690357203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561746555c90 0x561746570b70 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fe490000f80 tx=0x7fe49000bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5797293b2bccbe25 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:52.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.753+0000 7fe47aff5700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:09:52.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.753+0000 7fe47aff5700 1 -- 172.21.15.37:0/3690357203 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49002d070 con 0x561746555c90 2024-06-10T01:09:52.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.753+0000 7fe47aff5700 1 -- 172.21.15.37:0/3690357203 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fe490007aa0 con 0x561746555c90 2024-06-10T01:09:52.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.754+0000 7fe4a795ab80 1 -- 172.21.15.37:0/3690357203 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4900074e0 con 0x561746555c90 2024-06-10T01:09:52.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.754+0000 7fe47aff5700 1 -- 172.21.15.37:0/3690357203 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe490039050 con 0x561746555c90 2024-06-10T01:09:52.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.754+0000 7fe4a795ab80 10 client.17763 fetch_fsmap learned FSMap version 31 2024-06-10T01:09:52.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.754+0000 7fe4a795ab80 1 -- 172.21.15.37:0/3690357203 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5617465d3ca0 con 0x561746555c90 2024-06-10T01:09:52.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.754+0000 7fe47aff5700 1 -- 172.21.15.37:0/3690357203 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4900074e0 con 0x561746555c90 2024-06-10T01:09:52.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.754+0000 7fe47aff5700 1 -- 172.21.15.37:0/3690357203 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fe490011070 con 0x561746555c90 2024-06-10T01:09:52.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.754+0000 7fe4a795ab80 10 client.17763 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:09:52.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.754+0000 7fe4a795ab80 10 client.17763 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:52.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.754+0000 7fe4a795ab80 20 client.17763 populate_metadata read hostname 'smithi037' 2024-06-10T01:09:52.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.754+0000 7fe4a795ab80 1 --2- 172.21.15.37:0/3690357203 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5617465d3e70 0x5617465d6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:52.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.754+0000 7fe4a795ab80 4 client.17763 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:09:52.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.754+0000 7fe4a795ab80 1 -- 172.21.15.37:0/3690357203 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5617464a42d0 con 0x5617465d3e70 2024-06-10T01:09:52.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.755+0000 7fe496d9d700 1 --2- 172.21.15.37:0/3690357203 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5617465d3e70 0x5617465d6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:52.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.755+0000 7fe496d9d700 1 --2- 172.21.15.37:0/3690357203 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5617465d3e70 0x5617465d6260 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa2a5d22ddea90db server_cookie=36f8441eeea049a5 in_seq=0 out_seq=0 2024-06-10T01:09:52.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.755+0000 7fe47aff5700 10 client.17763 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:52.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.755+0000 7fe47aff5700 1 -- 172.21.15.37:0/3690357203 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5617464a42d0 con 0x5617465d3e70 2024-06-10T01:09:52.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.755+0000 7fe47aff5700 10 client.17763 handle_command_reply: tid=0 2024-06-10T01:09:52.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.755+0000 7fe4a795ab80 1 client.17763 shutdown 2024-06-10T01:09:52.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.755+0000 7fe4a795ab80 1 -- 172.21.15.37:0/3690357203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561746555c90 msgr2=0x561746570b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:52.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.755+0000 7fe4a795ab80 1 --2- 172.21.15.37:0/3690357203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561746555c90 0x561746570b70 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fe490000f80 tx=0x7fe49000bcd0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.756+0000 7fe4a795ab80 1 -- 172.21.15.37:0/3690357203 shutdown_connections 2024-06-10T01:09:52.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.756+0000 7fe4a795ab80 1 --2- 172.21.15.37:0/3690357203 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x5617465d3e70 0x5617465d6260 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.756+0000 7fe4a795ab80 1 --2- 172.21.15.37:0/3690357203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561746555c90 0x561746570b70 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.756+0000 7fe4a795ab80 1 --2- 172.21.15.37:0/3690357203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56174654fbb0 0x561746561ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.756+0000 7fe47aff5700 0 client.17763 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:52.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.756+0000 7fe4a795ab80 1 --2- 172.21.15.37:0/3690357203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561746555370 0x561746570630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.756+0000 7fe4a795ab80 1 -- 172.21.15.37:0/3690357203 >> 172.21.15.37:0/3690357203 conn(0x561746478080 msgr2=0x56174654a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:52.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.756+0000 7fe4a795ab80 1 -- 172.21.15.37:0/3690357203 shutdown_connections 2024-06-10T01:09:52.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.756+0000 7fe4a795ab80 1 -- 172.21.15.37:0/3690357203 wait complete. 2024-06-10T01:09:52.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.756+0000 7fe4a795ab80 20 client.17763 trim_cache size 0 max 16384 2024-06-10T01:09:52.756 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T01:09:52.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.756+0000 7fe49d0d7700 1 -- 172.21.15.37:0/2063310291 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe4800a2c20 msgr2=0x7fe4800a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:52.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.756+0000 7fe49d0d7700 1 --2- 172.21.15.37:0/2063310291 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe4800a2c20 0x7fe4800a50e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe498108c90 tx=0x7fe48c0073d0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.756+0000 7fe49d0d7700 1 -- 172.21.15.37:0/2063310291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe498102740 msgr2=0x7fe49819e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:52.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.756+0000 7fe49d0d7700 1 --2- 172.21.15.37:0/2063310291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe498102740 0x7fe49819e740 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe48800e9a0 tx=0x7fe48800ecb0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.756+0000 7fe49d0d7700 1 -- 172.21.15.37:0/2063310291 shutdown_connections 2024-06-10T01:09:52.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.756+0000 7fe49d0d7700 1 --2- 172.21.15.37:0/2063310291 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe4800a2c20 0x7fe4800a50e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.756+0000 7fe49d0d7700 1 --2- 172.21.15.37:0/2063310291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe498107840 0x7fe498198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.756+0000 7fe49d0d7700 1 --2- 172.21.15.37:0/2063310291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe498102740 0x7fe49819e740 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.756+0000 7fe49d0d7700 1 --2- 172.21.15.37:0/2063310291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4981030f0 0x7fe49819ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:52.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.756+0000 7fe49d0d7700 1 -- 172.21.15.37:0/2063310291 >> 172.21.15.37:0/2063310291 conn(0x7fe4980795f0 msgr2=0x7fe498100410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:52.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.757+0000 7fe49d0d7700 1 -- 172.21.15.37:0/2063310291 shutdown_connections 2024-06-10T01:09:52.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:52.757+0000 7fe49d0d7700 1 -- 172.21.15.37:0/2063310291 wait complete. 2024-06-10T01:09:54.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:54 smithi037 ceph-mon[32192]: pgmap v827: 129 pgs: 129 active+clean; 2.7 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-06-10T01:09:54.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:54 smithi167 ceph-mon[36917]: pgmap v827: 129 pgs: 129 active+clean; 2.7 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-06-10T01:09:54.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:54 smithi094 ceph-mon[36708]: pgmap v827: 129 pgs: 129 active+clean; 2.7 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-06-10T01:09:55.177 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-10T01:09:55.177 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-10T01:09:55.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.242+0000 7fea1be66700 1 -- 172.21.15.37:0/3910581763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea141084d0 msgr2=0x7fea141088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:55.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.242+0000 7fea1be66700 1 --2- 172.21.15.37:0/3910581763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea141084d0 0x7fea141088b0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fea1000b0a0 tx=0x7fea1000b3b0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.242+0000 7fea1be66700 1 -- 172.21.15.37:0/3910581763 shutdown_connections 2024-06-10T01:09:55.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.242+0000 7fea1be66700 1 --2- 172.21.15.37:0/3910581763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea1406b3a0 0x7fea1410f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.242+0000 7fea1be66700 1 --2- 172.21.15.37:0/3910581763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea141084d0 0x7fea141088b0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.242+0000 7fea1be66700 1 --2- 172.21.15.37:0/3910581763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea1406aa00 0x7fea1406ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.242+0000 7fea1be66700 1 -- 172.21.15.37:0/3910581763 >> 172.21.15.37:0/3910581763 conn(0x7fea14078330 msgr2=0x7fea14078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:55.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.242+0000 7fea1be66700 1 -- 172.21.15.37:0/3910581763 shutdown_connections 2024-06-10T01:09:55.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.242+0000 7fea1be66700 1 -- 172.21.15.37:0/3910581763 wait complete. 2024-06-10T01:09:55.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.243+0000 7fea1be66700 1 Processor -- start 2024-06-10T01:09:55.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.243+0000 7fea1be66700 1 -- start start 2024-06-10T01:09:55.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.243+0000 7fea1be66700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea1406aa00 0x7fea14198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:55.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.243+0000 7fea1be66700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea1406b3a0 0x7fea141991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:55.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.243+0000 7fea1be66700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea141084d0 0x7fea1419d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:55.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.243+0000 7fea1be66700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fea14112760 con 0x7fea1406b3a0 2024-06-10T01:09:55.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.243+0000 7fea1be66700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fea141125e0 con 0x7fea1406aa00 2024-06-10T01:09:55.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.243+0000 7fea1be66700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fea141128e0 con 0x7fea141084d0 2024-06-10T01:09:55.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.243+0000 7fea19401700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea1406b3a0 0x7fea141991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:55.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.243+0000 7fea19401700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea1406b3a0 0x7fea141991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36812/0 (socket says 172.21.15.37:36812) 2024-06-10T01:09:55.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.243+0000 7fea19401700 1 -- 172.21.15.37:0/3677348453 learned_addr learned my addr 172.21.15.37:0/3677348453 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:55.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.243+0000 7fea19c02700 1 --2- 172.21.15.37:0/3677348453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea1406aa00 0x7fea14198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:55.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.244+0000 7fea1a403700 1 --2- 172.21.15.37:0/3677348453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea141084d0 0x7fea1419d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:55.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.244+0000 7fea19401700 1 -- 172.21.15.37:0/3677348453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea141084d0 msgr2=0x7fea1419d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:55.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.244+0000 7fea19401700 1 --2- 172.21.15.37:0/3677348453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea141084d0 0x7fea1419d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.244+0000 7fea19401700 1 -- 172.21.15.37:0/3677348453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea1406aa00 msgr2=0x7fea14198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:55.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.244+0000 7fea19401700 1 --2- 172.21.15.37:0/3677348453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea1406aa00 0x7fea14198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.244+0000 7fea19401700 1 -- 172.21.15.37:0/3677348453 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea1419dc70 con 0x7fea1406b3a0 2024-06-10T01:09:55.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.244+0000 7fea1a403700 1 --2- 172.21.15.37:0/3677348453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea141084d0 0x7fea1419d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:09:55.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.244+0000 7fea19401700 1 --2- 172.21.15.37:0/3677348453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea1406b3a0 0x7fea141991b0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fea0400e9a0 tx=0x7fea0400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:55.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.244+0000 7fea0affd700 1 -- 172.21.15.37:0/3677348453 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0400cbd0 con 0x7fea1406b3a0 2024-06-10T01:09:55.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.244+0000 7fea1be66700 1 -- 172.21.15.37:0/3677348453 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea1419df60 con 0x7fea1406b3a0 2024-06-10T01:09:55.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.244+0000 7fea0affd700 1 -- 172.21.15.37:0/3677348453 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea0400cd70 con 0x7fea1406b3a0 2024-06-10T01:09:55.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.244+0000 7fea0affd700 1 -- 172.21.15.37:0/3677348453 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea04018750 con 0x7fea1406b3a0 2024-06-10T01:09:55.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.245+0000 7fea1be66700 1 -- 172.21.15.37:0/3677348453 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea141a5820 con 0x7fea1406b3a0 2024-06-10T01:09:55.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.247+0000 7fea0affd700 1 -- 172.21.15.37:0/3677348453 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fea04010430 con 0x7fea1406b3a0 2024-06-10T01:09:55.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.248+0000 7fea0affd700 1 --2- 172.21.15.37:0/3677348453 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fea000a2ca0 0x7fea000a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:55.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.248+0000 7fea19c02700 1 --2- 172.21.15.37:0/3677348453 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fea000a2ca0 0x7fea000a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:55.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.248+0000 7fea0affd700 1 -- 172.21.15.37:0/3677348453 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fea040cf770 con 0x7fea1406b3a0 2024-06-10T01:09:55.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.249+0000 7fea19c02700 1 --2- 172.21.15.37:0/3677348453 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fea000a2ca0 0x7fea000a5160 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fea10009fd0 tx=0x7fea10006210 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:55.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.249+0000 7fea266e9b80 1 Processor -- start 2024-06-10T01:09:55.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.249+0000 7fea266e9b80 1 -- start start 2024-06-10T01:09:55.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.249+0000 7fea266e9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a005754910 0x55a005754cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:55.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.249+0000 7fea266e9b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a005755230 0x55a00574eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:55.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.249+0000 7fea266e9b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a00574f060 0x55a00574f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:55.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.249+0000 7fea266e9b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a00563d060 con 0x55a00574f060 2024-06-10T01:09:55.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.249+0000 7fea266e9b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a00569b950 con 0x55a005755230 2024-06-10T01:09:55.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.249+0000 7fea266e9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a005603560 con 0x55a005754910 2024-06-10T01:09:55.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.250+0000 7fea19401700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a005755230 0x55a00574eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:55.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.250+0000 7fea1a403700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a00574f060 0x55a00574f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:55.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.250+0000 7fea19401700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a005755230 0x55a00574eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:46076/0 (socket says 172.21.15.37:46076) 2024-06-10T01:09:55.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.250+0000 7fea1a403700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a00574f060 0x55a00574f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36826/0 (socket says 172.21.15.37:36826) 2024-06-10T01:09:55.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.250+0000 7fea19401700 1 -- 172.21.15.37:0/38276358 learned_addr learned my addr 172.21.15.37:0/38276358 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:55.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.250+0000 7fea19c02700 1 --2- 172.21.15.37:0/38276358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a005754910 0x55a005754cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:55.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.250+0000 7fea19401700 1 -- 172.21.15.37:0/38276358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a005754910 msgr2=0x55a005754cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:55.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.250+0000 7fea19401700 1 --2- 172.21.15.37:0/38276358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a005754910 0x55a005754cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.250+0000 7fea19401700 1 -- 172.21.15.37:0/38276358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a00574f060 msgr2=0x55a00574f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:55.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.250+0000 7fea19401700 1 --2- 172.21.15.37:0/38276358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a00574f060 0x55a00574f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.250+0000 7fea19401700 1 -- 172.21.15.37:0/38276358 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0055ea9d0 con 0x55a005755230 2024-06-10T01:09:55.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.250+0000 7fea19401700 1 --2- 172.21.15.37:0/38276358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a005755230 0x55a00574eb20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fea04097fe0 tx=0x7fea0409b740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2ff4d50c25f3f27 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:55.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.250+0000 7fe9fffff700 1 -- 172.21.15.37:0/38276358 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0409bb80 con 0x55a005755230 2024-06-10T01:09:55.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.250+0000 7fe9fffff700 1 -- 172.21.15.37:0/38276358 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea0409ff30 con 0x55a005755230 2024-06-10T01:09:55.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.250+0000 7fe9fffff700 1 -- 172.21.15.37:0/38276358 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea040a4360 con 0x55a005755230 2024-06-10T01:09:55.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.251+0000 7fea266e9b80 1 -- 172.21.15.37:0/38276358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a005755230 msgr2=0x55a00574eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:55.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.251+0000 7fea266e9b80 1 --2- 172.21.15.37:0/38276358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a005755230 0x55a00574eb20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fea04097fe0 tx=0x7fea0409b740 comp rx=0 tx=0).stop 2024-06-10T01:09:55.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.251+0000 7fea266e9b80 1 -- 172.21.15.37:0/38276358 shutdown_connections 2024-06-10T01:09:55.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.251+0000 7fea266e9b80 1 --2- 172.21.15.37:0/38276358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a00574f060 0x55a00574f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.251+0000 7fea266e9b80 1 --2- 172.21.15.37:0/38276358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a005754910 0x55a005754cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.251+0000 7fea266e9b80 1 --2- 172.21.15.37:0/38276358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a005755230 0x55a00574eb20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.251+0000 7fea266e9b80 1 -- 172.21.15.37:0/38276358 >> 172.21.15.37:0/38276358 conn(0x55a005677180 msgr2=0x55a005745a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:55.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.251+0000 7fea266e9b80 1 -- 172.21.15.37:0/38276358 shutdown_connections 2024-06-10T01:09:55.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.251+0000 7fea266e9b80 1 -- 172.21.15.37:0/38276358 wait complete. 2024-06-10T01:09:55.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.252+0000 7fea266e9b80 1 -- start start 2024-06-10T01:09:55.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.252+0000 7fea266e9b80 1 Processor -- start 2024-06-10T01:09:55.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.252+0000 7fea266e9b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a00574f060 0x55a00575afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:55.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.252+0000 7fea266e9b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a005754910 0x55a0057582a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:55.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.252+0000 7fea266e9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a005755230 0x55a0057587e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:55.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.252+0000 7fea266e9b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a00563c0c0 con 0x55a00574f060 2024-06-10T01:09:55.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.252+0000 7fea266e9b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a005697910 con 0x55a005754910 2024-06-10T01:09:55.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.252+0000 7fea266e9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a00575bc10 con 0x55a005755230 2024-06-10T01:09:55.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.253+0000 7fea266e9b80 10 client.0 _ll_register_callbacks cb 0x55a0056316b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:55.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.253+0000 7fea19c02700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a00574f060 0x55a00575afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:55.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.253+0000 7fea19c02700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a00574f060 0x55a00575afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36828/0 (socket says 172.21.15.37:36828) 2024-06-10T01:09:55.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.253+0000 7fea19c02700 1 -- 172.21.15.37:0/516615699 learned_addr learned my addr 172.21.15.37:0/516615699 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:55.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.253+0000 7fea1a403700 1 --2- 172.21.15.37:0/516615699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a005755230 0x55a0057587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:55.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.253+0000 7fea19401700 1 --2- 172.21.15.37:0/516615699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a005754910 0x55a0057582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:55.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.253+0000 7fea19c02700 1 -- 172.21.15.37:0/516615699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a005755230 msgr2=0x55a0057587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:55.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.253+0000 7fea19c02700 1 --2- 172.21.15.37:0/516615699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a005755230 0x55a0057587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.253+0000 7fea19c02700 1 -- 172.21.15.37:0/516615699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a005754910 msgr2=0x55a0057582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:55.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.253+0000 7fea19c02700 1 --2- 172.21.15.37:0/516615699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a005754910 0x55a0057582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.253+0000 7fea19c02700 1 -- 172.21.15.37:0/516615699 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a005661850 con 0x55a00574f060 2024-06-10T01:09:55.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.253+0000 7fea19c02700 1 --2- 172.21.15.37:0/516615699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a00574f060 0x55a00575afa0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fea1000b0a0 tx=0x7fea100099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=979c5f1ad7e0b5dd server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:55.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.253+0000 7fe9fd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:09:55.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.253+0000 7fe9fd7fa700 1 -- 172.21.15.37:0/516615699 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea10003e60 con 0x55a00574f060 2024-06-10T01:09:55.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.253+0000 7fe9fd7fa700 1 -- 172.21.15.37:0/516615699 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fea100076e0 con 0x55a00574f060 2024-06-10T01:09:55.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.254+0000 7fea266e9b80 1 -- 172.21.15.37:0/516615699 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea0409ff30 con 0x55a00574f060 2024-06-10T01:09:55.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.254+0000 7fe9fd7fa700 1 -- 172.21.15.37:0/516615699 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fea10039050 con 0x55a00574f060 2024-06-10T01:09:55.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.254+0000 7fea266e9b80 10 client.17775 fetch_fsmap learned FSMap version 31 2024-06-10T01:09:55.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.254+0000 7fea266e9b80 1 -- 172.21.15.37:0/516615699 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a0057663e0 con 0x55a00574f060 2024-06-10T01:09:55.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.254+0000 7fe9fd7fa700 1 -- 172.21.15.37:0/516615699 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea10007dc0 con 0x55a00574f060 2024-06-10T01:09:55.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.254+0000 7fe9fd7fa700 1 -- 172.21.15.37:0/516615699 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fea10011070 con 0x55a00574f060 2024-06-10T01:09:55.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.254+0000 7fea266e9b80 10 client.17775 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:09:55.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.254+0000 7fea266e9b80 10 client.17775 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:09:55.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.254+0000 7fea266e9b80 20 client.17775 populate_metadata read hostname 'smithi037' 2024-06-10T01:09:55.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.254+0000 7fea266e9b80 1 --2- 172.21.15.37:0/516615699 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a005768a70 0x55a00576ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:55.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.254+0000 7fea266e9b80 4 client.17775 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:09:55.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.254+0000 7fea266e9b80 1 -- 172.21.15.37:0/516615699 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fea0409ff30 con 0x55a005768a70 2024-06-10T01:09:55.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.255+0000 7fea19401700 1 --2- 172.21.15.37:0/516615699 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a005768a70 0x55a00576ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:55.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.255+0000 7fea19401700 1 --2- 172.21.15.37:0/516615699 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a005768a70 0x55a00576ae60 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0f9f27a8f5789a2 server_cookie=ca2eb64358c899e4 in_seq=0 out_seq=0 2024-06-10T01:09:55.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.255+0000 7fe9fd7fa700 10 client.17775 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:55.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.256+0000 7fe9fd7fa700 1 -- 172.21.15.37:0/516615699 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fea0409ff30 con 0x55a005768a70 2024-06-10T01:09:55.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.256+0000 7fe9fd7fa700 10 client.17775 handle_command_reply: tid=0 2024-06-10T01:09:55.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.256+0000 7fea266e9b80 1 client.17775 shutdown 2024-06-10T01:09:55.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.256+0000 7fea266e9b80 1 -- 172.21.15.37:0/516615699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a00574f060 msgr2=0x55a00575afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:55.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.256+0000 7fea266e9b80 1 --2- 172.21.15.37:0/516615699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a00574f060 0x55a00575afa0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fea1000b0a0 tx=0x7fea100099e0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.256+0000 7fea266e9b80 1 -- 172.21.15.37:0/516615699 shutdown_connections 2024-06-10T01:09:55.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.256+0000 7fea266e9b80 1 --2- 172.21.15.37:0/516615699 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a005768a70 0x55a00576ae60 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.256+0000 7fea266e9b80 1 --2- 172.21.15.37:0/516615699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a005755230 0x55a0057587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.256+0000 7fea266e9b80 1 --2- 172.21.15.37:0/516615699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a00574f060 0x55a00575afa0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.256+0000 7fe9fd7fa700 0 client.17775 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:55.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.256+0000 7fea266e9b80 1 --2- 172.21.15.37:0/516615699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a005754910 0x55a0057582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.256+0000 7fea266e9b80 1 -- 172.21.15.37:0/516615699 >> 172.21.15.37:0/516615699 conn(0x55a005677180 msgr2=0x55a005749700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:55.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.256+0000 7fea266e9b80 1 -- 172.21.15.37:0/516615699 shutdown_connections 2024-06-10T01:09:55.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.256+0000 7fea266e9b80 1 -- 172.21.15.37:0/516615699 wait complete. 2024-06-10T01:09:55.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.256+0000 7fea266e9b80 20 client.17775 trim_cache size 0 max 16384 2024-06-10T01:09:55.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.300+0000 7fea266e9b80 1 Processor -- start 2024-06-10T01:09:55.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.300+0000 7fea266e9b80 1 -- start start 2024-06-10T01:09:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.301+0000 7fea266e9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a00574f060 0x55a005755c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.301+0000 7fea266e9b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a005754910 0x55a00575aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.301+0000 7fea266e9b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a005755230 0x55a005770720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.301+0000 7fea266e9b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fea141aead0 con 0x55a005754910 2024-06-10T01:09:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.301+0000 7fea266e9b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fea141ac360 con 0x55a005755230 2024-06-10T01:09:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.301+0000 7fea266e9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fea141aa1b0 con 0x55a00574f060 2024-06-10T01:09:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.301+0000 7fea19401700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a005754910 0x55a00575aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.301+0000 7fea19401700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a005754910 0x55a00575aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36830/0 (socket says 172.21.15.37:36830) 2024-06-10T01:09:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.301+0000 7fea19401700 1 -- 172.21.15.37:0/2443523051 learned_addr learned my addr 172.21.15.37:0/2443523051 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:55.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.301+0000 7fea19c02700 1 --2- 172.21.15.37:0/2443523051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a00574f060 0x55a005755c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:55.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.301+0000 7fea19401700 1 -- 172.21.15.37:0/2443523051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a00574f060 msgr2=0x55a005755c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:55.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.301+0000 7fea1a403700 1 --2- 172.21.15.37:0/2443523051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a005755230 0x55a005770720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:55.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.301+0000 7fea19401700 1 --2- 172.21.15.37:0/2443523051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a00574f060 0x55a005755c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.301+0000 7fea19401700 1 -- 172.21.15.37:0/2443523051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a005755230 msgr2=0x55a005770720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:55.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.301+0000 7fea19401700 1 --2- 172.21.15.37:0/2443523051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a005755230 0x55a005770720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.301+0000 7fea19401700 1 -- 172.21.15.37:0/2443523051 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a00576b490 con 0x55a005754910 2024-06-10T01:09:55.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.302+0000 7fea19401700 1 --2- 172.21.15.37:0/2443523051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a005754910 0x55a00575aae0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fea04098550 tx=0x7fea0401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77c530f48740c72 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:55.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.302+0000 7fe9ff7fe700 1 -- 172.21.15.37:0/2443523051 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0409bb80 con 0x55a005754910 2024-06-10T01:09:55.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.302+0000 7fe9ff7fe700 1 -- 172.21.15.37:0/2443523051 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea0407f070 con 0x55a005754910 2024-06-10T01:09:55.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.302+0000 7fe9ff7fe700 1 -- 172.21.15.37:0/2443523051 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea040cf3c0 con 0x55a005754910 2024-06-10T01:09:55.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.303+0000 7fea266e9b80 1 -- 172.21.15.37:0/2443523051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a005754910 msgr2=0x55a00575aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:55.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.303+0000 7fea266e9b80 1 --2- 172.21.15.37:0/2443523051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a005754910 0x55a00575aae0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fea04098550 tx=0x7fea0401f000 comp rx=0 tx=0).stop 2024-06-10T01:09:55.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.303+0000 7fea266e9b80 1 -- 172.21.15.37:0/2443523051 shutdown_connections 2024-06-10T01:09:55.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.303+0000 7fea266e9b80 1 --2- 172.21.15.37:0/2443523051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a005755230 0x55a005770720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.303+0000 7fea266e9b80 1 --2- 172.21.15.37:0/2443523051 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a00574f060 0x55a005755c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.303+0000 7fea266e9b80 1 --2- 172.21.15.37:0/2443523051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a005754910 0x55a00575aae0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.303+0000 7fea266e9b80 1 -- 172.21.15.37:0/2443523051 >> 172.21.15.37:0/2443523051 conn(0x55a005677180 msgr2=0x55a005749f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:55.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.303+0000 7fea266e9b80 1 -- 172.21.15.37:0/2443523051 shutdown_connections 2024-06-10T01:09:55.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.303+0000 7fea266e9b80 1 -- 172.21.15.37:0/2443523051 wait complete. 2024-06-10T01:09:55.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.303+0000 7fea266e9b80 1 -- start start 2024-06-10T01:09:55.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.303+0000 7fea266e9b80 1 Processor -- start 2024-06-10T01:09:55.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.304+0000 7fea266e9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a00574f060 0x55a005761380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:55.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.304+0000 7fea266e9b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a005754910 0x55a00575dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:55.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.304+0000 7fea266e9b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a005755230 0x55a00575e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:55.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.304+0000 7fea266e9b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fea141aa820 con 0x55a005755230 2024-06-10T01:09:55.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.304+0000 7fea266e9b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fea141abc90 con 0x55a005754910 2024-06-10T01:09:55.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.304+0000 7fea266e9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fea141aaef0 con 0x55a00574f060 2024-06-10T01:09:55.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.304+0000 7fea266e9b80 10 client.0 _ll_register_callbacks cb 0x55a0056316b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:55.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.304+0000 7fea1a403700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a005755230 0x55a00575e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:55.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.304+0000 7fea19c02700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a00574f060 0x55a005761380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:55.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.304+0000 7fea1a403700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a005755230 0x55a00575e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36846/0 (socket says 172.21.15.37:36846) 2024-06-10T01:09:55.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.304+0000 7fea1a403700 1 -- 172.21.15.37:0/4096895808 learned_addr learned my addr 172.21.15.37:0/4096895808 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:55.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.304+0000 7fea19401700 1 --2- 172.21.15.37:0/4096895808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a005754910 0x55a00575dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:55.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.305+0000 7fea1a403700 1 -- 172.21.15.37:0/4096895808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a00574f060 msgr2=0x55a005761380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:55.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.305+0000 7fea1a403700 1 --2- 172.21.15.37:0/4096895808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a00574f060 0x55a005761380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.305+0000 7fea1a403700 1 -- 172.21.15.37:0/4096895808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a005754910 msgr2=0x55a00575dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:55.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.305+0000 7fea1a403700 1 --2- 172.21.15.37:0/4096895808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a005754910 0x55a00575dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.305+0000 7fea1a403700 1 -- 172.21.15.37:0/4096895808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a00575e920 con 0x55a005755230 2024-06-10T01:09:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.305+0000 7fea19401700 1 --2- 172.21.15.37:0/4096895808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a005754910 0x55a00575dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:09:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.305+0000 7fea19c02700 1 --2- 172.21.15.37:0/4096895808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a00574f060 0x55a005761380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.305+0000 7fea1a403700 1 --2- 172.21.15.37:0/4096895808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a005755230 0x55a00575e3e0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fea0c013710 tx=0x7fea0c013a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1e6c35722c6f86 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.305+0000 7fe9fd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:09:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.305+0000 7fe9fd7fa700 1 -- 172.21.15.37:0/4096895808 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c011440 con 0x55a005755230 2024-06-10T01:09:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.305+0000 7fe9fd7fa700 1 -- 172.21.15.37:0/4096895808 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fea0c0115e0 con 0x55a005755230 2024-06-10T01:09:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.305+0000 7fea266e9b80 1 -- 172.21.15.37:0/4096895808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea0407f070 con 0x55a005755230 2024-06-10T01:09:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.305+0000 7fe9fd7fa700 1 -- 172.21.15.37:0/4096895808 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fea0c0117e0 con 0x55a005755230 2024-06-10T01:09:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.305+0000 7fea266e9b80 10 client.17781 fetch_fsmap learned FSMap version 31 2024-06-10T01:09:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.305+0000 7fea266e9b80 1 -- 172.21.15.37:0/4096895808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a0057d4410 con 0x55a005755230 2024-06-10T01:09:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.305+0000 7fe9fd7fa700 1 -- 172.21.15.37:0/4096895808 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea0407f070 con 0x55a005755230 2024-06-10T01:09:55.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.306+0000 7fe9fd7fa700 1 -- 172.21.15.37:0/4096895808 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fea0c01db20 con 0x55a005755230 2024-06-10T01:09:55.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.306+0000 7fea266e9b80 10 client.17781 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:09:55.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.306+0000 7fea266e9b80 10 client.17781 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:09:55.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.306+0000 7fea266e9b80 20 client.17781 populate_metadata read hostname 'smithi037' 2024-06-10T01:09:55.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.306+0000 7fea266e9b80 1 --2- 172.21.15.37:0/4096895808 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a0057d4b70 0x55a0057d6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:55.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.306+0000 7fea266e9b80 4 client.17781 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9e1aa52e", "release": true, "await": true}] 2024-06-10T01:09:55.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.306+0000 7fea266e9b80 1 -- 172.21.15.37:0/4096895808 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9e1aa52e", "release": true, "await": true}) -- 0x55a0056a8e50 con 0x55a0057d4b70 2024-06-10T01:09:55.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.306+0000 7fea19c02700 1 --2- 172.21.15.37:0/4096895808 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a0057d4b70 0x55a0057d6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:55.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.306+0000 7fea19c02700 1 --2- 172.21.15.37:0/4096895808 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a0057d4b70 0x55a0057d6f90 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d35419359f391339 server_cookie=daa9e1f323be5bac in_seq=0 out_seq=0 2024-06-10T01:09:55.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.306+0000 7fe9fd7fa700 10 client.17781 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:55.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.309+0000 7fe9fd7fa700 1 -- 172.21.15.37:0/4096895808 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a0056a8e50 con 0x55a0057d4b70 2024-06-10T01:09:55.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.309+0000 7fe9fd7fa700 10 client.17781 handle_command_reply: tid=0 2024-06-10T01:09:55.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.309+0000 7fea266e9b80 1 client.17781 shutdown 2024-06-10T01:09:55.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.309+0000 7fea266e9b80 1 -- 172.21.15.37:0/4096895808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a005755230 msgr2=0x55a00575e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:55.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.309+0000 7fea266e9b80 1 --2- 172.21.15.37:0/4096895808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a005755230 0x55a00575e3e0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fea0c013710 tx=0x7fea0c013a20 comp rx=0 tx=0).stop 2024-06-10T01:09:55.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.309+0000 7fea266e9b80 1 -- 172.21.15.37:0/4096895808 shutdown_connections 2024-06-10T01:09:55.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.309+0000 7fea266e9b80 1 --2- 172.21.15.37:0/4096895808 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55a0057d4b70 0x55a0057d6f90 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.309+0000 7fea266e9b80 1 --2- 172.21.15.37:0/4096895808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a005755230 0x55a00575e3e0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.309+0000 7fea266e9b80 1 --2- 172.21.15.37:0/4096895808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a00574f060 0x55a005761380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.309+0000 7fea266e9b80 1 --2- 172.21.15.37:0/4096895808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a005754910 0x55a00575dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.309+0000 7fe9fd7fa700 0 client.17781 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:55.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.309+0000 7fea266e9b80 1 -- 172.21.15.37:0/4096895808 >> 172.21.15.37:0/4096895808 conn(0x55a005677180 msgr2=0x55a005749f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:55.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.309+0000 7fea266e9b80 1 -- 172.21.15.37:0/4096895808 shutdown_connections 2024-06-10T01:09:55.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.309+0000 7fea266e9b80 1 -- 172.21.15.37:0/4096895808 wait complete. 2024-06-10T01:09:55.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.309+0000 7fea266e9b80 20 client.17781 trim_cache size 0 max 16384 2024-06-10T01:09:55.308 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:09:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 30, 2024-06-10T01:09:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24472, 2024-06-10T01:09:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 36, 2024-06-10T01:09:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-06-10T01:09:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "9e1aa52e": { 2024-06-10T01:09:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 36, 2024-06-10T01:09:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-06-10T01:09:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:09:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-06-10T01:09:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:09:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:09:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-06-10T01:09:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 135.8, 2024-06-10T01:09:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-06-10T01:09:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-06-10T01:09:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-06-10T01:09:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:09:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-06-10T01:09:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:09:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:09:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:09:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:09:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:09:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:09:55.309 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:09:55.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.310+0000 7fea1be66700 1 -- 172.21.15.37:0/3677348453 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fea000a2ca0 msgr2=0x7fea000a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:55.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.310+0000 7fea1be66700 1 --2- 172.21.15.37:0/3677348453 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fea000a2ca0 0x7fea000a5160 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fea10009fd0 tx=0x7fea10006210 comp rx=0 tx=0).stop 2024-06-10T01:09:55.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.310+0000 7fea1be66700 1 -- 172.21.15.37:0/3677348453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea1406b3a0 msgr2=0x7fea141991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:55.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.310+0000 7fea1be66700 1 --2- 172.21.15.37:0/3677348453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea1406b3a0 0x7fea141991b0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fea0400e9a0 tx=0x7fea0400ecb0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.310+0000 7fea1be66700 1 -- 172.21.15.37:0/3677348453 shutdown_connections 2024-06-10T01:09:55.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.310+0000 7fea1be66700 1 --2- 172.21.15.37:0/3677348453 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fea000a2ca0 0x7fea000a5160 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.310+0000 7fea1be66700 1 --2- 172.21.15.37:0/3677348453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea141084d0 0x7fea1419d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.310+0000 7fea1be66700 1 --2- 172.21.15.37:0/3677348453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea1406aa00 0x7fea14198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.310+0000 7fea1be66700 1 --2- 172.21.15.37:0/3677348453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea1406b3a0 0x7fea141991b0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:55.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.310+0000 7fea1be66700 1 -- 172.21.15.37:0/3677348453 >> 172.21.15.37:0/3677348453 conn(0x7fea14078330 msgr2=0x7fea140ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:55.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.310+0000 7fea1be66700 1 -- 172.21.15.37:0/3677348453 shutdown_connections 2024-06-10T01:09:55.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:55.310+0000 7fea1be66700 1 -- 172.21.15.37:0/3677348453 wait complete. 2024-06-10T01:09:55.317 INFO:tasks.workunit:Running workunit fs/misc/i_complete_vs_rename.sh... 2024-06-10T01:09:55.317 DEBUG:teuthology.orchestra.run.smithi037:workunit test fs/misc/i_complete_vs_rename.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b7a5d375cf4e4afab96b9d4d229b699ae8304bb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/i_complete_vs_rename.sh 2024-06-10T01:09:55.365 INFO:tasks.workunit.client.0.smithi037.stdout:a 2024-06-10T01:09:55.365 INFO:tasks.workunit.client.0.smithi037.stdout:b 2024-06-10T01:09:55.366 INFO:tasks.workunit.client.0.smithi037.stdout:c 2024-06-10T01:09:55.366 INFO:tasks.workunit.client.0.smithi037.stdout:d 2024-06-10T01:09:55.387 INFO:tasks.workunit.client.0.smithi037.stderr:stat: cannot statx 'e': No such file or directory 2024-06-10T01:09:55.391 INFO:tasks.workunit.client.0.smithi037.stdout:attempting rename over existing file... 2024-06-10T01:09:55.443 INFO:tasks.workunit.client.0.smithi037.stdout:f 2024-06-10T01:09:55.443 INFO:tasks.workunit.client.0.smithi037.stdout:rename over existing file is okay 2024-06-10T01:09:55.443 INFO:tasks.workunit.client.0.smithi037.stdout:attempting rename over negative dentry... 2024-06-10T01:09:55.454 INFO:tasks.workunit.client.0.smithi037.stdout:e 2024-06-10T01:09:55.454 INFO:tasks.workunit.client.0.smithi037.stdout:rename over negative dentry is ok 2024-06-10T01:09:55.454 INFO:tasks.workunit.client.0.smithi037.stdout:OK 2024-06-10T01:09:55.454 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-10T01:09:55.454 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-10T01:09:55.515 INFO:tasks.workunit:Running workunit fs/misc/layout_vxattrs.sh... 2024-06-10T01:09:55.516 DEBUG:teuthology.orchestra.run.smithi037:workunit test fs/misc/layout_vxattrs.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b7a5d375cf4e4afab96b9d4d229b699ae8304bb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/layout_vxattrs.sh 2024-06-10T01:09:55.638 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:09:56.080 INFO:tasks.workunit.client.0.smithi037.stderr:+ datapool= 2024-06-10T01:09:56.080 INFO:tasks.workunit.client.0.smithi037.stderr:+ dir=. 2024-06-10T01:09:56.080 INFO:tasks.workunit.client.0.smithi037.stderr:+ true 2024-06-10T01:09:56.080 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo . 2024-06-10T01:09:56.080 INFO:tasks.workunit.client.0.smithi037.stdout:. 2024-06-10T01:09:56.081 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfattr -n ceph.dir.layout.pool . --only-values 2024-06-10T01:09:56.081 INFO:tasks.workunit.client.0.smithi037.stderr:.: ceph.dir.layout.pool: No such attribute 2024-06-10T01:09:56.081 INFO:tasks.workunit.client.0.smithi037.stderr:+ datapool= 2024-06-10T01:09:56.081 INFO:tasks.workunit.client.0.smithi037.stderr:+ dir=./.. 2024-06-10T01:09:56.082 INFO:tasks.workunit.client.0.smithi037.stderr:+ true 2024-06-10T01:09:56.082 INFO:tasks.workunit.client.0.smithi037.stdout:./.. 2024-06-10T01:09:56.083 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo ./.. 2024-06-10T01:09:56.083 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfattr -n ceph.dir.layout.pool ./.. --only-values 2024-06-10T01:09:56.083 INFO:tasks.workunit.client.0.smithi037.stderr:./..: ceph.dir.layout.pool: No such attribute 2024-06-10T01:09:56.083 INFO:tasks.workunit.client.0.smithi037.stderr:+ datapool= 2024-06-10T01:09:56.083 INFO:tasks.workunit.client.0.smithi037.stderr:+ dir=./../.. 2024-06-10T01:09:56.083 INFO:tasks.workunit.client.0.smithi037.stderr:+ true 2024-06-10T01:09:56.083 INFO:tasks.workunit.client.0.smithi037.stdout:./../.. 2024-06-10T01:09:56.084 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo ./../.. 2024-06-10T01:09:56.084 INFO:tasks.workunit.client.0.smithi037.stderr:++ getfattr -n ceph.dir.layout.pool ./../.. --only-values 2024-06-10T01:09:56.084 INFO:tasks.workunit.client.0.smithi037.stderr:+ datapool=cephfs_data_ec 2024-06-10T01:09:56.084 INFO:tasks.workunit.client.0.smithi037.stderr:+ break 2024-06-10T01:09:56.084 INFO:tasks.workunit.client.0.smithi037.stderr:+ rm -f file file2 2024-06-10T01:09:56.085 INFO:tasks.workunit.client.0.smithi037.stderr:+ touch file file2 2024-06-10T01:09:56.087 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout file 2024-06-10T01:09:56.088 INFO:tasks.workunit.client.0.smithi037.stdout:# file: file 2024-06-10T01:09:56.088 INFO:tasks.workunit.client.0.smithi037.stdout:ceph.file.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-06-10T01:09:56.089 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-06-10T01:09:56.089 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout file 2024-06-10T01:09:56.089 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q object_size= 2024-06-10T01:09:56.091 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout file 2024-06-10T01:09:56.091 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q stripe_count= 2024-06-10T01:09:56.092 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout file 2024-06-10T01:09:56.092 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q stripe_unit= 2024-06-10T01:09:56.094 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout file 2024-06-10T01:09:56.094 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q pool= 2024-06-10T01:09:56.095 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.pool file 2024-06-10T01:09:56.096 INFO:tasks.workunit.client.0.smithi037.stdout:# file: file 2024-06-10T01:09:56.097 INFO:tasks.workunit.client.0.smithi037.stdout:ceph.file.layout.pool="cephfs_data_ec" 2024-06-10T01:09:56.097 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-06-10T01:09:56.097 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.pool_namespace file 2024-06-10T01:09:56.098 INFO:tasks.workunit.client.0.smithi037.stdout:# file: file 2024-06-10T01:09:56.098 INFO:tasks.workunit.client.0.smithi037.stdout:ceph.file.layout.pool_namespace="" 2024-06-10T01:09:56.098 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-06-10T01:09:56.098 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.stripe_unit file 2024-06-10T01:09:56.099 INFO:tasks.workunit.client.0.smithi037.stdout:# file: file 2024-06-10T01:09:56.099 INFO:tasks.workunit.client.0.smithi037.stdout:ceph.file.layout.stripe_unit="4194304" 2024-06-10T01:09:56.099 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-06-10T01:09:56.099 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.stripe_count file 2024-06-10T01:09:56.100 INFO:tasks.workunit.client.0.smithi037.stdout:# file: file 2024-06-10T01:09:56.100 INFO:tasks.workunit.client.0.smithi037.stdout:ceph.file.layout.stripe_count="1" 2024-06-10T01:09:56.101 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-06-10T01:09:56.101 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.object_size file 2024-06-10T01:09:56.102 INFO:tasks.workunit.client.0.smithi037.stdout:# file: file 2024-06-10T01:09:56.102 INFO:tasks.workunit.client.0.smithi037.stdout:ceph.file.layout.object_size="4194304" 2024-06-10T01:09:56.102 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-06-10T01:09:56.102 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.bogus file 2024-06-10T01:09:56.102 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q 'No such attribute' 2024-06-10T01:09:56.103 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.dir.layout file 2024-06-10T01:09:56.103 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q 'No such attribute' 2024-06-10T01:09:56.104 INFO:tasks.workunit.client.0.smithi037.stderr:+ setfattr -n ceph.file.layout.stripe_unit -v 1048576 file2 2024-06-10T01:09:56.111 INFO:tasks.workunit.client.0.smithi037.stderr:+ setfattr -n ceph.file.layout.stripe_count -v 8 file2 2024-06-10T01:09:56.115 INFO:tasks.workunit.client.0.smithi037.stderr:+ setfattr -n ceph.file.layout.object_size -v 10485760 file2 2024-06-10T01:09:56.123 INFO:tasks.workunit.client.0.smithi037.stderr:+ setfattr -n ceph.file.layout.pool -v cephfs_data_ec file2 2024-06-10T01:09:56.130 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.pool file2 2024-06-10T01:09:56.130 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q cephfs_data_ec 2024-06-10T01:09:56.131 INFO:tasks.workunit.client.0.smithi037.stderr:+ setfattr -n ceph.file.layout.pool_namespace -v foons file2 2024-06-10T01:09:56.136 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.pool_namespace file2 2024-06-10T01:09:56.136 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q foons 2024-06-10T01:09:56.137 INFO:tasks.workunit.client.0.smithi037.stderr:+ setfattr -x ceph.file.layout.pool_namespace file2 2024-06-10T01:09:56.142 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.pool_namespace file2 2024-06-10T01:09:56.142 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q -v foons 2024-06-10T01:09:56.143 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.stripe_unit file2 2024-06-10T01:09:56.143 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q 1048576 2024-06-10T01:09:56.145 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2024-06-10T01:09:56.145 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q 8 2024-06-10T01:09:56.146 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.object_size file2 2024-06-10T01:09:56.146 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q 10485760 2024-06-10T01:09:56.148 INFO:tasks.workunit.client.0.smithi037.stderr:+ setfattr -n ceph.file.layout -v 'stripe_unit=4194304 stripe_count=16 object_size=41943040 pool=cephfs_data_ec pool_namespace=foons' file2 2024-06-10T01:09:56.152 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.stripe_unit file2 2024-06-10T01:09:56.153 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q 4194304 2024-06-10T01:09:56.154 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2024-06-10T01:09:56.154 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q 16 2024-06-10T01:09:56.156 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.object_size file2 2024-06-10T01:09:56.156 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q 41943040 2024-06-10T01:09:56.158 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.pool file2 2024-06-10T01:09:56.158 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q cephfs_data_ec 2024-06-10T01:09:56.159 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.pool_namespace file2 2024-06-10T01:09:56.159 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q foons 2024-06-10T01:09:56.161 INFO:tasks.workunit.client.0.smithi037.stderr:+ setfattr -n ceph.file.layout -v stripe_unit=1048576 file2 2024-06-10T01:09:56.168 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.stripe_unit file2 2024-06-10T01:09:56.168 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q 1048576 2024-06-10T01:09:56.170 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2024-06-10T01:09:56.170 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q 16 2024-06-10T01:09:56.171 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.object_size file2 2024-06-10T01:09:56.171 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q 41943040 2024-06-10T01:09:56.172 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.pool file2 2024-06-10T01:09:56.173 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q cephfs_data_ec 2024-06-10T01:09:56.174 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.pool_namespace file2 2024-06-10T01:09:56.175 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q foons 2024-06-10T01:09:56.176 INFO:tasks.workunit.client.0.smithi037.stderr:+ setfattr -n ceph.file.layout -v 'stripe_unit=2097152 stripe_count=4 object_size=2097152 pool=cephfs_data_ec pool_namespace=barns' file2 2024-06-10T01:09:56.181 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.stripe_unit file2 2024-06-10T01:09:56.181 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q 2097152 2024-06-10T01:09:56.182 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2024-06-10T01:09:56.182 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q 4 2024-06-10T01:09:56.184 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.object_size file2 2024-06-10T01:09:56.184 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q 2097152 2024-06-10T01:09:56.186 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.pool file2 2024-06-10T01:09:56.186 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q cephfs_data_ec 2024-06-10T01:09:56.191 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.pool_namespace file2 2024-06-10T01:09:56.191 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q barns 2024-06-10T01:09:56.191 INFO:tasks.workunit.client.0.smithi037.stderr:+ rm -f dir/file 2024-06-10T01:09:56.191 INFO:tasks.workunit.client.0.smithi037.stderr:+ rmdir dir 2024-06-10T01:09:56.191 INFO:tasks.workunit.client.0.smithi037.stderr:rmdir: failed to remove 'dir': No such file or directory 2024-06-10T01:09:56.191 INFO:tasks.workunit.client.0.smithi037.stderr:+ true 2024-06-10T01:09:56.191 INFO:tasks.workunit.client.0.smithi037.stderr:+ mkdir -p dir 2024-06-10T01:09:56.193 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -d -m - dir 2024-06-10T01:09:56.193 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q ceph.dir.layout 2024-06-10T01:09:56.194 INFO:tasks.workunit.client.0.smithi037.stderr:+ true 2024-06-10T01:09:56.194 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -d -m - dir 2024-06-10T01:09:56.194 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q ceph.file.layout 2024-06-10T01:09:56.195 INFO:tasks.workunit.client.0.smithi037.stderr:+ true 2024-06-10T01:09:56.195 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.dir.layout dir 2024-06-10T01:09:56.196 INFO:tasks.workunit.client.0.smithi037.stderr:dir: ceph.dir.layout: No such attribute 2024-06-10T01:09:56.196 INFO:tasks.workunit.client.0.smithi037.stderr:+ true 2024-06-10T01:09:56.197 INFO:tasks.workunit.client.0.smithi037.stderr:+ setfattr -n ceph.dir.layout.stripe_unit -v 1048576 dir 2024-06-10T01:09:56.202 INFO:tasks.workunit.client.0.smithi037.stderr:+ setfattr -n ceph.dir.layout.stripe_count -v 8 dir 2024-06-10T01:09:56.206 INFO:tasks.workunit.client.0.smithi037.stderr:+ setfattr -n ceph.dir.layout.object_size -v 10485760 dir 2024-06-10T01:09:56.211 INFO:tasks.workunit.client.0.smithi037.stderr:+ setfattr -n ceph.dir.layout.pool -v cephfs_data_ec dir 2024-06-10T01:09:56.215 INFO:tasks.workunit.client.0.smithi037.stderr:+ setfattr -n ceph.dir.layout.pool_namespace -v dirns dir 2024-06-10T01:09:56.219 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.dir.layout dir 2024-06-10T01:09:56.220 INFO:tasks.workunit.client.0.smithi037.stdout:# file: dir 2024-06-10T01:09:56.221 INFO:tasks.workunit.client.0.smithi037.stdout:ceph.dir.layout="stripe_unit=1048576 stripe_count=8 object_size=10485760 pool=cephfs_data_ec pool_namespace=dirns" 2024-06-10T01:09:56.221 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-06-10T01:09:56.221 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.dir.layout dir 2024-06-10T01:09:56.221 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q object_size=10485760 2024-06-10T01:09:56.222 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.dir.layout dir 2024-06-10T01:09:56.222 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q stripe_count=8 2024-06-10T01:09:56.224 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.dir.layout dir 2024-06-10T01:09:56.224 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q stripe_unit=1048576 2024-06-10T01:09:56.225 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.dir.layout dir 2024-06-10T01:09:56.225 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q pool=cephfs_data_ec 2024-06-10T01:09:56.227 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.dir.layout dir 2024-06-10T01:09:56.227 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q pool_namespace=dirns 2024-06-10T01:09:56.228 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.dir.layout.pool dir 2024-06-10T01:09:56.228 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q cephfs_data_ec 2024-06-10T01:09:56.230 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.dir.layout.stripe_unit dir 2024-06-10T01:09:56.230 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q 1048576 2024-06-10T01:09:56.232 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.dir.layout.stripe_count dir 2024-06-10T01:09:56.232 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q 8 2024-06-10T01:09:56.233 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.dir.layout.object_size dir 2024-06-10T01:09:56.233 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q 10485760 2024-06-10T01:09:56.235 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.dir.layout.pool_namespace dir 2024-06-10T01:09:56.235 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q dirns 2024-06-10T01:09:56.236 INFO:tasks.workunit.client.0.smithi037.stderr:+ setfattr -n ceph.file.layout -v stripe_count=16 file2 2024-06-10T01:09:56.238 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2024-06-10T01:09:56.238 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q 16 2024-06-10T01:09:56.239 INFO:tasks.workunit.client.0.smithi037.stderr:+ setfattr -n ceph.file.layout -v 'object_size=10485760 stripe_count=8 stripe_unit=1048576 pool=cephfs_data_ec pool_namespace=dirns' file2 2024-06-10T01:09:56.245 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2024-06-10T01:09:56.245 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q 8 2024-06-10T01:09:56.246 INFO:tasks.workunit.client.0.smithi037.stderr:+ touch dir/file 2024-06-10T01:09:56.248 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.pool dir/file 2024-06-10T01:09:56.249 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q cephfs_data_ec 2024-06-10T01:09:56.250 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.stripe_unit dir/file 2024-06-10T01:09:56.250 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q 1048576 2024-06-10T01:09:56.251 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.stripe_count dir/file 2024-06-10T01:09:56.251 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q 8 2024-06-10T01:09:56.253 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.object_size dir/file 2024-06-10T01:09:56.253 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q 10485760 2024-06-10T01:09:56.254 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.file.layout.pool_namespace dir/file 2024-06-10T01:09:56.254 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q dirns 2024-06-10T01:09:56.255 INFO:tasks.workunit.client.0.smithi037.stderr:+ setfattr -x ceph.dir.layout.pool_namespace dir 2024-06-10T01:09:56.260 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9e1aa52e', total seconds elapsed: 24.7 2024-06-10T01:09:56.263 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.dir.layout dir 2024-06-10T01:09:56.264 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q -v pool_namespace=dirns 2024-06-10T01:09:56.265 INFO:tasks.workunit.client.0.smithi037.stderr:+ setfattr -x ceph.dir.layout dir 2024-06-10T01:09:56.269 INFO:tasks.workunit.client.0.smithi037.stderr:+ getfattr -n ceph.dir.layout dir 2024-06-10T01:09:56.269 INFO:tasks.workunit.client.0.smithi037.stderr:+ grep -q 'No such attribute' 2024-06-10T01:09:56.269 INFO:tasks.workunit.client.0.smithi037.stdout:OK 2024-06-10T01:09:56.270 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo OK 2024-06-10T01:09:56.270 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-10T01:09:56.271 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-10T01:09:56.361 INFO:tasks.workunit:Running workunit fs/misc/mkpool_layout_vxattrs.sh... 2024-06-10T01:09:56.362 DEBUG:teuthology.orchestra.run.smithi037:workunit test fs/misc/mkpool_layout_vxattrs.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b7a5d375cf4e4afab96b9d4d229b699ae8304bb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/mkpool_layout_vxattrs.sh 2024-06-10T01:09:56.498 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:56 smithi037 ceph-mon[32192]: pgmap v828: 129 pgs: 129 active+clean; 2.7 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-06-10T01:09:56.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:56 smithi167 ceph-mon[36917]: pgmap v828: 129 pgs: 129 active+clean; 2.7 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-06-10T01:09:56.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:56 smithi094 ceph-mon[36708]: pgmap v828: 129 pgs: 129 active+clean; 2.7 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-06-10T01:09:57.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:57 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:09:57.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:57 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:09:57.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:57 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:09:57.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:57 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:09:57.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:57 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:09:57.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:57 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:09:57.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:57 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:09:57.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:57 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:09:57.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:57 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:09:57.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.734+0000 7fab75426700 1 -- 172.21.15.37:0/2119237277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab701030f0 msgr2=0x7fab70107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:57.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.734+0000 7fab75426700 1 --2- 172.21.15.37:0/2119237277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab701030f0 0x7fab70107110 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fab580099c0 tx=0x7fab58009cd0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.734+0000 7fab75426700 1 -- 172.21.15.37:0/2119237277 shutdown_connections 2024-06-10T01:09:57.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.734+0000 7fab75426700 1 --2- 172.21.15.37:0/2119237277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab70107840 0x7fab7010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.734+0000 7fab75426700 1 --2- 172.21.15.37:0/2119237277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab70102740 0x7fab70102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.734+0000 7fab75426700 1 --2- 172.21.15.37:0/2119237277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab701030f0 0x7fab70107110 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.734+0000 7fab75426700 1 -- 172.21.15.37:0/2119237277 >> 172.21.15.37:0/2119237277 conn(0x7fab700795f0 msgr2=0x7fab70079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:57.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.734+0000 7fab75426700 1 -- 172.21.15.37:0/2119237277 shutdown_connections 2024-06-10T01:09:57.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.734+0000 7fab75426700 1 -- 172.21.15.37:0/2119237277 wait complete. 2024-06-10T01:09:57.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.735+0000 7fab75426700 1 Processor -- start 2024-06-10T01:09:57.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.735+0000 7fab75426700 1 -- start start 2024-06-10T01:09:57.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.735+0000 7fab75426700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab70102740 0x7fab7019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.735+0000 7fab75426700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab701030f0 0x7fab7019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.735+0000 7fab75426700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab70107840 0x7fab701989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.735+0000 7fab75426700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fab701127a0 con 0x7fab70102740 2024-06-10T01:09:57.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.735+0000 7fab75426700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab70112620 con 0x7fab70107840 2024-06-10T01:09:57.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.735+0000 7fab75426700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fab70112920 con 0x7fab701030f0 2024-06-10T01:09:57.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.735+0000 7fab6effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab70102740 0x7fab7019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.735+0000 7fab6f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab70107840 0x7fab701989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.735+0000 7fab6effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab70102740 0x7fab7019e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36862/0 (socket says 172.21.15.37:36862) 2024-06-10T01:09:57.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.735+0000 7fab6effd700 1 -- 172.21.15.37:0/2113337028 learned_addr learned my addr 172.21.15.37:0/2113337028 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:57.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.735+0000 7fab6f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab70107840 0x7fab701989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:46130/0 (socket says 172.21.15.37:46130) 2024-06-10T01:09:57.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.736+0000 7fab667fc700 1 --2- 172.21.15.37:0/2113337028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab701030f0 0x7fab7019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.736+0000 7fab6f7fe700 1 -- 172.21.15.37:0/2113337028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab701030f0 msgr2=0x7fab7019ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:57.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.736+0000 7fab6f7fe700 1 --2- 172.21.15.37:0/2113337028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab701030f0 0x7fab7019ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.736+0000 7fab6f7fe700 1 -- 172.21.15.37:0/2113337028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab70102740 msgr2=0x7fab7019e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:57.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.736+0000 7fab6f7fe700 1 --2- 172.21.15.37:0/2113337028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab70102740 0x7fab7019e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.736+0000 7fab6f7fe700 1 -- 172.21.15.37:0/2113337028 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab70199100 con 0x7fab70107840 2024-06-10T01:09:57.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.736+0000 7fab6f7fe700 1 --2- 172.21.15.37:0/2113337028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab70107840 0x7fab701989a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fab6800efc0 tx=0x7fab6800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:57.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.736+0000 7fab6cff9700 1 -- 172.21.15.37:0/2113337028 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab680092e0 con 0x7fab70107840 2024-06-10T01:09:57.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.736+0000 7fab6cff9700 1 -- 172.21.15.37:0/2113337028 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab68004bb0 con 0x7fab70107840 2024-06-10T01:09:57.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.736+0000 7fab6cff9700 1 -- 172.21.15.37:0/2113337028 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab68004280 con 0x7fab70107840 2024-06-10T01:09:57.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.736+0000 7fab75426700 1 -- 172.21.15.37:0/2113337028 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab70199390 con 0x7fab70107840 2024-06-10T01:09:57.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.736+0000 7fab75426700 1 -- 172.21.15.37:0/2113337028 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab700746c0 con 0x7fab70107840 2024-06-10T01:09:57.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.739+0000 7fab6cff9700 1 -- 172.21.15.37:0/2113337028 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fab6801b030 con 0x7fab70107840 2024-06-10T01:09:57.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.740+0000 7fab6cff9700 1 --2- 172.21.15.37:0/2113337028 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fab5c0a2c20 0x7fab5c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.740+0000 7fab6effd700 1 --2- 172.21.15.37:0/2113337028 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fab5c0a2c20 0x7fab5c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.740+0000 7fab6cff9700 1 -- 172.21.15.37:0/2113337028 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fab680ceff0 con 0x7fab70107840 2024-06-10T01:09:57.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.740+0000 7fab6effd700 1 --2- 172.21.15.37:0/2113337028 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fab5c0a2c20 0x7fab5c0a50e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fab70199ae0 tx=0x7fab60005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:57.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.740+0000 7fab7fca9b80 1 Processor -- start 2024-06-10T01:09:57.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.740+0000 7fab7fca9b80 1 -- start start 2024-06-10T01:09:57.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.741+0000 7fab7fca9b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e3d335fe0 0x560e3d3363c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.741+0000 7fab7fca9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3d336900 0x560e3d3302e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.741+0000 7fab7fca9b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e3d330820 0x560e3d330c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.741+0000 7fab7fca9b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560e3d222fe0 con 0x560e3d330820 2024-06-10T01:09:57.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.741+0000 7fab7fca9b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560e3d2818d0 con 0x560e3d335fe0 2024-06-10T01:09:57.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.741+0000 7fab7fca9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560e3d1e94e0 con 0x560e3d336900 2024-06-10T01:09:57.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.741+0000 7fab6f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e3d330820 0x560e3d330c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.741+0000 7fab667fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3d336900 0x560e3d3302e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.741+0000 7fab6effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e3d335fe0 0x560e3d3363c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.741+0000 7fab667fc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3d336900 0x560e3d3302e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.37:51952/0 (socket says 172.21.15.37:51952) 2024-06-10T01:09:57.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.741+0000 7fab6f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e3d330820 0x560e3d330c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36878/0 (socket says 172.21.15.37:36878) 2024-06-10T01:09:57.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.741+0000 7fab667fc700 1 -- 172.21.15.37:0/246883711 learned_addr learned my addr 172.21.15.37:0/246883711 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:57.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.741+0000 7fab667fc700 1 -- 172.21.15.37:0/246883711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e3d335fe0 msgr2=0x560e3d3363c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:57.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.741+0000 7fab667fc700 1 --2- 172.21.15.37:0/246883711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e3d335fe0 0x560e3d3363c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.741+0000 7fab667fc700 1 -- 172.21.15.37:0/246883711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e3d330820 msgr2=0x560e3d330c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:57.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.741+0000 7fab667fc700 1 --2- 172.21.15.37:0/246883711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e3d330820 0x560e3d330c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.741+0000 7fab667fc700 1 -- 172.21.15.37:0/246883711 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e3d1d0950 con 0x560e3d336900 2024-06-10T01:09:57.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.741+0000 7fab6f7fe700 1 --2- 172.21.15.37:0/246883711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e3d330820 0x560e3d330c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.742+0000 7fab667fc700 1 --2- 172.21.15.37:0/246883711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3d336900 0x560e3d3302e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fab580099c0 tx=0x7fab58003730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=190c4d3589110d4f server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.742+0000 7fab657fa700 1 -- 172.21.15.37:0/246883711 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab58028070 con 0x560e3d336900 2024-06-10T01:09:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.742+0000 7fab657fa700 1 -- 172.21.15.37:0/246883711 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab580050c0 con 0x560e3d336900 2024-06-10T01:09:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.742+0000 7fab657fa700 1 -- 172.21.15.37:0/246883711 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5800ba80 con 0x560e3d336900 2024-06-10T01:09:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.743+0000 7fab7fca9b80 1 -- 172.21.15.37:0/246883711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3d336900 msgr2=0x560e3d3302e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:57.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.743+0000 7fab7fca9b80 1 --2- 172.21.15.37:0/246883711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3d336900 0x560e3d3302e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fab580099c0 tx=0x7fab58003730 comp rx=0 tx=0).stop 2024-06-10T01:09:57.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.743+0000 7fab7fca9b80 1 -- 172.21.15.37:0/246883711 shutdown_connections 2024-06-10T01:09:57.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.743+0000 7fab7fca9b80 1 --2- 172.21.15.37:0/246883711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e3d330820 0x560e3d330c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.743+0000 7fab7fca9b80 1 --2- 172.21.15.37:0/246883711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e3d335fe0 0x560e3d3363c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.743+0000 7fab7fca9b80 1 --2- 172.21.15.37:0/246883711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3d336900 0x560e3d3302e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.743+0000 7fab7fca9b80 1 -- 172.21.15.37:0/246883711 >> 172.21.15.37:0/246883711 conn(0x560e3d25d100 msgr2=0x560e3d32b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:57.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.743+0000 7fab7fca9b80 1 -- 172.21.15.37:0/246883711 shutdown_connections 2024-06-10T01:09:57.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.743+0000 7fab7fca9b80 1 -- 172.21.15.37:0/246883711 wait complete. 2024-06-10T01:09:57.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.743+0000 7fab7fca9b80 1 -- start start 2024-06-10T01:09:57.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.743+0000 7fab7fca9b80 1 Processor -- start 2024-06-10T01:09:57.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.743+0000 7fab7fca9b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e3d330820 0x560e3d33c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.743+0000 7fab7fca9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3d335fe0 0x560e3d3399f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.743+0000 7fab7fca9b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e3d336900 0x560e3d339f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.744+0000 7fab7fca9b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560e3d222040 con 0x560e3d336900 2024-06-10T01:09:57.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.744+0000 7fab7fca9b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560e3d27d890 con 0x560e3d330820 2024-06-10T01:09:57.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.744+0000 7fab7fca9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560e3d33d280 con 0x560e3d335fe0 2024-06-10T01:09:57.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.744+0000 7fab7fca9b80 10 client.0 _ll_register_callbacks cb 0x560e3d217630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:57.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.744+0000 7fab6f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e3d336900 0x560e3d339f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.744+0000 7fab6f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e3d336900 0x560e3d339f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36888/0 (socket says 172.21.15.37:36888) 2024-06-10T01:09:57.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.744+0000 7fab6f7fe700 1 -- 172.21.15.37:0/2661280169 learned_addr learned my addr 172.21.15.37:0/2661280169 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:57.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.744+0000 7fab667fc700 1 --2- 172.21.15.37:0/2661280169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3d335fe0 0x560e3d3399f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.744+0000 7fab6effd700 1 --2- 172.21.15.37:0/2661280169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e3d330820 0x560e3d33c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.744+0000 7fab667fc700 1 -- 172.21.15.37:0/2661280169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e3d330820 msgr2=0x560e3d33c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:57.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.744+0000 7fab667fc700 1 --2- 172.21.15.37:0/2661280169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e3d330820 0x560e3d33c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.744+0000 7fab667fc700 1 -- 172.21.15.37:0/2661280169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e3d336900 msgr2=0x560e3d339f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:57.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.744+0000 7fab667fc700 1 --2- 172.21.15.37:0/2661280169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e3d336900 0x560e3d339f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.744+0000 7fab667fc700 1 -- 172.21.15.37:0/2661280169 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e3d2477d0 con 0x560e3d335fe0 2024-06-10T01:09:57.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.744+0000 7fab6f7fe700 1 --2- 172.21.15.37:0/2661280169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e3d336900 0x560e3d339f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:57.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.744+0000 7fab667fc700 1 --2- 172.21.15.37:0/2661280169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3d335fe0 0x560e3d3399f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fab58009750 tx=0x7fab58003b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c81c567158702d6a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:57.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.744+0000 7fab567fc700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-06-10T01:09:57.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.744+0000 7fab567fc700 1 -- 172.21.15.37:0/2661280169 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab58028070 con 0x560e3d335fe0 2024-06-10T01:09:57.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.744+0000 7fab7fca9b80 1 -- 172.21.15.37:0/2661280169 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab580050c0 con 0x560e3d335fe0 2024-06-10T01:09:57.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.744+0000 7fab567fc700 1 -- 172.21.15.37:0/2661280169 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fab5800b990 con 0x560e3d335fe0 2024-06-10T01:09:57.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.745+0000 7fab567fc700 1 -- 172.21.15.37:0/2661280169 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fab580056c0 con 0x560e3d335fe0 2024-06-10T01:09:57.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.745+0000 7fab7fca9b80 10 client.27244 fetch_fsmap learned FSMap version 31 2024-06-10T01:09:57.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.745+0000 7fab7fca9b80 1 -- 172.21.15.37:0/2661280169 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560e3d347ab0 con 0x560e3d335fe0 2024-06-10T01:09:57.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.745+0000 7fab567fc700 1 -- 172.21.15.37:0/2661280169 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab580050c0 con 0x560e3d335fe0 2024-06-10T01:09:57.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.745+0000 7fab567fc700 1 -- 172.21.15.37:0/2661280169 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fab5800f9d0 con 0x560e3d335fe0 2024-06-10T01:09:57.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.745+0000 7fab7fca9b80 10 client.27244 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:09:57.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.745+0000 7fab7fca9b80 10 client.27244 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:57.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.745+0000 7fab7fca9b80 20 client.27244 populate_metadata read hostname 'smithi037' 2024-06-10T01:09:57.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.745+0000 7fab7fca9b80 1 --2- 172.21.15.37:0/2661280169 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560e3d349f80 0x560e3d34c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.745+0000 7fab7fca9b80 4 client.27244 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:09:57.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.745+0000 7fab7fca9b80 1 -- 172.21.15.37:0/2661280169 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e3d28edd0 con 0x560e3d349f80 2024-06-10T01:09:57.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.745+0000 7fab6effd700 1 --2- 172.21.15.37:0/2661280169 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560e3d349f80 0x560e3d34c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.746+0000 7fab567fc700 10 client.27244 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:57.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.746+0000 7fab6effd700 1 --2- 172.21.15.37:0/2661280169 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560e3d349f80 0x560e3d34c370 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec5e195c7a316ed4 server_cookie=9bb031432a263341 in_seq=0 out_seq=0 2024-06-10T01:09:57.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.746+0000 7fab567fc700 1 -- 172.21.15.37:0/2661280169 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560e3d28edd0 con 0x560e3d349f80 2024-06-10T01:09:57.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.746+0000 7fab567fc700 10 client.27244 handle_command_reply: tid=0 2024-06-10T01:09:57.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.746+0000 7fab7fca9b80 1 client.27244 shutdown 2024-06-10T01:09:57.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.747+0000 7fab7fca9b80 1 -- 172.21.15.37:0/2661280169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3d335fe0 msgr2=0x560e3d3399f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:57.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.747+0000 7fab7fca9b80 1 --2- 172.21.15.37:0/2661280169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3d335fe0 0x560e3d3399f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fab58009750 tx=0x7fab58003b10 comp rx=0 tx=0).stop 2024-06-10T01:09:57.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.747+0000 7fab7fca9b80 1 -- 172.21.15.37:0/2661280169 shutdown_connections 2024-06-10T01:09:57.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.747+0000 7fab7fca9b80 1 --2- 172.21.15.37:0/2661280169 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560e3d349f80 0x560e3d34c370 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.747+0000 7fab7fca9b80 1 --2- 172.21.15.37:0/2661280169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e3d336900 0x560e3d339f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.747+0000 7fab7fca9b80 1 --2- 172.21.15.37:0/2661280169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e3d330820 0x560e3d33c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.747+0000 7fab567fc700 0 client.27244 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:57.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.747+0000 7fab7fca9b80 1 --2- 172.21.15.37:0/2661280169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3d335fe0 0x560e3d3399f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.747+0000 7fab7fca9b80 1 -- 172.21.15.37:0/2661280169 >> 172.21.15.37:0/2661280169 conn(0x560e3d25d100 msgr2=0x560e3d32a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:57.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.747+0000 7fab7fca9b80 1 -- 172.21.15.37:0/2661280169 shutdown_connections 2024-06-10T01:09:57.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.747+0000 7fab7fca9b80 1 -- 172.21.15.37:0/2661280169 wait complete. 2024-06-10T01:09:57.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.747+0000 7fab7fca9b80 20 client.27244 trim_cache size 0 max 16384 2024-06-10T01:09:57.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.767+0000 7fab7fca9b80 1 Processor -- start 2024-06-10T01:09:57.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.767+0000 7fab7fca9b80 1 -- start start 2024-06-10T01:09:57.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.768+0000 7fab7fca9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3d330820 0x560e3d32af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.768+0000 7fab7fca9b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e3d335fe0 0x560e3d32b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.768+0000 7fab7fca9b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e3d336900 0x560e3d34f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.768+0000 7fab7fca9b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fab701b2dd0 con 0x560e3d336900 2024-06-10T01:09:57.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.768+0000 7fab7fca9b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab701b0660 con 0x560e3d335fe0 2024-06-10T01:09:57.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.768+0000 7fab7fca9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fab701ae4b0 con 0x560e3d330820 2024-06-10T01:09:57.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.768+0000 7fab6f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e3d336900 0x560e3d34f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.768+0000 7fab6f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e3d336900 0x560e3d34f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36896/0 (socket says 172.21.15.37:36896) 2024-06-10T01:09:57.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.768+0000 7fab6f7fe700 1 -- 172.21.15.37:0/556757090 learned_addr learned my addr 172.21.15.37:0/556757090 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:57.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.768+0000 7fab6effd700 1 --2- 172.21.15.37:0/556757090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3d330820 0x560e3d32af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.768+0000 7fab667fc700 1 --2- 172.21.15.37:0/556757090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e3d335fe0 0x560e3d32b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.768+0000 7fab6f7fe700 1 -- 172.21.15.37:0/556757090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3d330820 msgr2=0x560e3d32af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:57.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.768+0000 7fab6f7fe700 1 --2- 172.21.15.37:0/556757090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3d330820 0x560e3d32af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.768+0000 7fab6f7fe700 1 -- 172.21.15.37:0/556757090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e3d335fe0 msgr2=0x560e3d32b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:57.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.768+0000 7fab6f7fe700 1 --2- 172.21.15.37:0/556757090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e3d335fe0 0x560e3d32b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.768+0000 7fab6f7fe700 1 -- 172.21.15.37:0/556757090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e3d3302e0 con 0x560e3d336900 2024-06-10T01:09:57.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.768+0000 7fab6effd700 1 --2- 172.21.15.37:0/556757090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3d330820 0x560e3d32af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:09:57.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.769+0000 7fab6f7fe700 1 --2- 172.21.15.37:0/556757090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e3d336900 0x560e3d34f390 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fab6809b420 tx=0x7fab680ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372a24da830d817 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:57.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.769+0000 7fab64ff9700 1 -- 172.21.15.37:0/556757090 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab680b3070 con 0x560e3d336900 2024-06-10T01:09:57.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.769+0000 7fab64ff9700 1 -- 172.21.15.37:0/556757090 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab6809bb90 con 0x560e3d336900 2024-06-10T01:09:57.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.769+0000 7fab64ff9700 1 -- 172.21.15.37:0/556757090 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab680b4260 con 0x560e3d336900 2024-06-10T01:09:57.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.769+0000 7fab7fca9b80 1 -- 172.21.15.37:0/556757090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e3d336900 msgr2=0x560e3d34f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:57.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.769+0000 7fab7fca9b80 1 --2- 172.21.15.37:0/556757090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e3d336900 0x560e3d34f390 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fab6809b420 tx=0x7fab680ce040 comp rx=0 tx=0).stop 2024-06-10T01:09:57.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.769+0000 7fab7fca9b80 1 -- 172.21.15.37:0/556757090 shutdown_connections 2024-06-10T01:09:57.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.769+0000 7fab7fca9b80 1 --2- 172.21.15.37:0/556757090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e3d336900 0x560e3d34f390 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.769+0000 7fab7fca9b80 1 --2- 172.21.15.37:0/556757090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3d330820 0x560e3d32af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.769+0000 7fab7fca9b80 1 --2- 172.21.15.37:0/556757090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e3d335fe0 0x560e3d32b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.769+0000 7fab7fca9b80 1 -- 172.21.15.37:0/556757090 >> 172.21.15.37:0/556757090 conn(0x560e3d25d100 msgr2=0x560e3d335360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:57.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.769+0000 7fab7fca9b80 1 -- 172.21.15.37:0/556757090 shutdown_connections 2024-06-10T01:09:57.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.770+0000 7fab7fca9b80 1 -- 172.21.15.37:0/556757090 wait complete. 2024-06-10T01:09:57.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.770+0000 7fab7fca9b80 1 -- start start 2024-06-10T01:09:57.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.770+0000 7fab7fca9b80 1 Processor -- start 2024-06-10T01:09:57.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.770+0000 7fab7fca9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3d330820 0x560e3d34dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.770+0000 7fab7fca9b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e3d335fe0 0x560e3d34e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.770+0000 7fab7fca9b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e3d336900 0x560e3d34f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.770+0000 7fab7fca9b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fab701aeb20 con 0x560e3d335fe0 2024-06-10T01:09:57.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.770+0000 7fab7fca9b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab701aff90 con 0x560e3d336900 2024-06-10T01:09:57.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.770+0000 7fab7fca9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fab701af1f0 con 0x560e3d330820 2024-06-10T01:09:57.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.770+0000 7fab7fca9b80 10 client.0 _ll_register_callbacks cb 0x560e3d217630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:09:57.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.770+0000 7fab667fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e3d335fe0 0x560e3d34e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.770+0000 7fab667fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e3d335fe0 0x560e3d34e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36902/0 (socket says 172.21.15.37:36902) 2024-06-10T01:09:57.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.770+0000 7fab667fc700 1 -- 172.21.15.37:0/3999921240 learned_addr learned my addr 172.21.15.37:0/3999921240 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:57.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.770+0000 7fab6effd700 1 --2- 172.21.15.37:0/3999921240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3d330820 0x560e3d34dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.771+0000 7fab6f7fe700 1 --2- 172.21.15.37:0/3999921240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e3d336900 0x560e3d34f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.771+0000 7fab667fc700 1 -- 172.21.15.37:0/3999921240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3d330820 msgr2=0x560e3d34dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:57.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.771+0000 7fab667fc700 1 --2- 172.21.15.37:0/3999921240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3d330820 0x560e3d34dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.771+0000 7fab667fc700 1 -- 172.21.15.37:0/3999921240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e3d336900 msgr2=0x560e3d34f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:09:57.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.771+0000 7fab667fc700 1 --2- 172.21.15.37:0/3999921240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e3d336900 0x560e3d34f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.771+0000 7fab667fc700 1 -- 172.21.15.37:0/3999921240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e3d34fa30 con 0x560e3d335fe0 2024-06-10T01:09:57.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.771+0000 7fab667fc700 1 --2- 172.21.15.37:0/3999921240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e3d335fe0 0x560e3d34e2a0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fab58009290 tx=0x7fab58003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513b84a6f0172732 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:57.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.771+0000 7fab567fc700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:09:57.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.771+0000 7fab567fc700 1 -- 172.21.15.37:0/3999921240 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab58028040 con 0x560e3d335fe0 2024-06-10T01:09:57.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.771+0000 7fab7fca9b80 1 -- 172.21.15.37:0/3999921240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab6809bb90 con 0x560e3d335fe0 2024-06-10T01:09:57.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.771+0000 7fab567fc700 1 -- 172.21.15.37:0/3999921240 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7fab58005bc0 con 0x560e3d335fe0 2024-06-10T01:09:57.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.771+0000 7fab567fc700 1 -- 172.21.15.37:0/3999921240 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fab58049050 con 0x560e3d335fe0 2024-06-10T01:09:57.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.771+0000 7fab7fca9b80 10 client.17799 fetch_fsmap learned FSMap version 31 2024-06-10T01:09:57.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.771+0000 7fab7fca9b80 1 -- 172.21.15.37:0/3999921240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560e3d3b54e0 con 0x560e3d335fe0 2024-06-10T01:09:57.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.771+0000 7fab567fc700 1 -- 172.21.15.37:0/3999921240 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab6809bb90 con 0x560e3d335fe0 2024-06-10T01:09:57.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.772+0000 7fab567fc700 1 -- 172.21.15.37:0/3999921240 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fab5800f9b0 con 0x560e3d335fe0 2024-06-10T01:09:57.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.772+0000 7fab7fca9b80 10 client.17799 fetch_fsmap finished waiting for FSMap version 31 2024-06-10T01:09:57.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.772+0000 7fab7fca9b80 10 client.17799 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:09:57.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.772+0000 7fab7fca9b80 20 client.17799 populate_metadata read hostname 'smithi037' 2024-06-10T01:09:57.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.772+0000 7fab7fca9b80 1 --2- 172.21.15.37:0/3999921240 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560e3d3b5b20 0x560e3d3b7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.772+0000 7fab7fca9b80 4 client.17799 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-10T01:09:57.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.772+0000 7fab7fca9b80 1 -- 172.21.15.37:0/3999921240 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560e3d33c110 con 0x560e3d3b5b20 2024-06-10T01:09:57.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.772+0000 7fab6effd700 1 --2- 172.21.15.37:0/3999921240 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560e3d3b5b20 0x560e3d3b7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.772+0000 7fab6effd700 1 --2- 172.21.15.37:0/3999921240 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560e3d3b5b20 0x560e3d3b7f40 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47dfed9d5d5356b4 server_cookie=e8da0640e03cd2c6 in_seq=0 out_seq=0 2024-06-10T01:09:57.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.772+0000 7fab567fc700 10 client.17799 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:57.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.773+0000 7fab567fc700 1 -- 172.21.15.37:0/3999921240 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560e3d33c110 con 0x560e3d3b5b20 2024-06-10T01:09:57.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.773+0000 7fab567fc700 10 client.17799 handle_command_reply: tid=0 2024-06-10T01:09:57.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.773+0000 7fab7fca9b80 1 client.17799 shutdown 2024-06-10T01:09:57.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.773+0000 7fab7fca9b80 1 -- 172.21.15.37:0/3999921240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e3d335fe0 msgr2=0x560e3d34e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:57.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.774+0000 7fab7fca9b80 1 --2- 172.21.15.37:0/3999921240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e3d335fe0 0x560e3d34e2a0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fab58009290 tx=0x7fab58003730 comp rx=0 tx=0).stop 2024-06-10T01:09:57.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.774+0000 7fab7fca9b80 1 -- 172.21.15.37:0/3999921240 shutdown_connections 2024-06-10T01:09:57.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.774+0000 7fab7fca9b80 1 --2- 172.21.15.37:0/3999921240 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x560e3d3b5b20 0x560e3d3b7f40 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.774+0000 7fab7fca9b80 1 --2- 172.21.15.37:0/3999921240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e3d336900 0x560e3d34f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.774+0000 7fab567fc700 0 client.17799 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:09:57.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.774+0000 7fab7fca9b80 1 --2- 172.21.15.37:0/3999921240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e3d330820 0x560e3d34dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.774+0000 7fab7fca9b80 1 --2- 172.21.15.37:0/3999921240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e3d335fe0 0x560e3d34e2a0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.774+0000 7fab7fca9b80 1 -- 172.21.15.37:0/3999921240 >> 172.21.15.37:0/3999921240 conn(0x560e3d25d100 msgr2=0x560e3d32f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:57.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.774+0000 7fab7fca9b80 1 -- 172.21.15.37:0/3999921240 shutdown_connections 2024-06-10T01:09:57.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.774+0000 7fab7fca9b80 1 -- 172.21.15.37:0/3999921240 wait complete. 2024-06-10T01:09:57.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.774+0000 7fab7fca9b80 20 client.17799 trim_cache size 0 max 16384 2024-06-10T01:09:57.831 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:09:57.831 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-06-10T01:09:57.831 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "af6360d9-ca22-467c-a010-3496836b3e01", 2024-06-10T01:09:57.832 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-06-10T01:09:57.832 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:09:57.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.774+0000 7fab75426700 1 -- 172.21.15.37:0/2113337028 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fab5c0a2c20 msgr2=0x7fab5c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:57.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.774+0000 7fab75426700 1 --2- 172.21.15.37:0/2113337028 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fab5c0a2c20 0x7fab5c0a50e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fab70199ae0 tx=0x7fab60005f90 comp rx=0 tx=0).stop 2024-06-10T01:09:57.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.774+0000 7fab75426700 1 -- 172.21.15.37:0/2113337028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab70107840 msgr2=0x7fab701989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:57.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.774+0000 7fab75426700 1 --2- 172.21.15.37:0/2113337028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab70107840 0x7fab701989a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fab6800efc0 tx=0x7fab6800c3a0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.775+0000 7fab75426700 1 -- 172.21.15.37:0/2113337028 shutdown_connections 2024-06-10T01:09:57.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.775+0000 7fab75426700 1 --2- 172.21.15.37:0/2113337028 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fab5c0a2c20 0x7fab5c0a50e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.775+0000 7fab75426700 1 --2- 172.21.15.37:0/2113337028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab70107840 0x7fab701989a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.775+0000 7fab75426700 1 --2- 172.21.15.37:0/2113337028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab70102740 0x7fab7019e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.775+0000 7fab75426700 1 --2- 172.21.15.37:0/2113337028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab701030f0 0x7fab7019ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.775+0000 7fab75426700 1 -- 172.21.15.37:0/2113337028 >> 172.21.15.37:0/2113337028 conn(0x7fab700795f0 msgr2=0x7fab701004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:57.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.775+0000 7fab75426700 1 -- 172.21.15.37:0/2113337028 shutdown_connections 2024-06-10T01:09:57.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:09:57.775+0000 7fab75426700 1 -- 172.21.15.37:0/2113337028 wait complete. 2024-06-10T01:09:57.879 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.877+0000 7f3235fdb640 1 -- 172.21.15.37:0/3131084219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32301035d0 msgr2=0x7f3230109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:57.879 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.877+0000 7f3235fdb640 1 --2- 172.21.15.37:0/3131084219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32301035d0 0x7f3230109e60 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f3228009ca0 tx=0x7f322802a8c0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.879 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.877+0000 7f3235fdb640 1 -- 172.21.15.37:0/3131084219 shutdown_connections 2024-06-10T01:09:57.879 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.877+0000 7f3235fdb640 1 --2- 172.21.15.37:0/3131084219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32301035d0 0x7f3230109e60 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.879 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.877+0000 7f3235fdb640 1 --2- 172.21.15.37:0/3131084219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3230102c10 0x7f3230103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.879 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.877+0000 7f3235fdb640 1 --2- 172.21.15.37:0/3131084219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3230101a20 0x7f3230101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.879 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.877+0000 7f3235fdb640 1 -- 172.21.15.37:0/3131084219 >> 172.21.15.37:0/3131084219 conn(0x7f32300fcf90 msgr2=0x7f32300ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:57.879 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.878+0000 7f3235fdb640 1 -- 172.21.15.37:0/3131084219 shutdown_connections 2024-06-10T01:09:57.879 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.878+0000 7f3235fdb640 1 -- 172.21.15.37:0/3131084219 wait complete. 2024-06-10T01:09:57.879 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.878+0000 7f3235fdb640 1 Processor -- start 2024-06-10T01:09:57.880 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.878+0000 7f3235fdb640 1 -- start start 2024-06-10T01:09:57.880 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.878+0000 7f3235fdb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3230101a20 0x7f3230197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.880 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.878+0000 7f3235fdb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3230102c10 0x7f3230198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.880 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.878+0000 7f3235fdb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32301035d0 0x7f323019f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.880 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.878+0000 7f3235fdb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f323010d6a0 con 0x7f3230101a20 2024-06-10T01:09:57.880 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.878+0000 7f3235fdb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f323010d520 con 0x7f3230102c10 2024-06-10T01:09:57.880 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.878+0000 7f3235fdb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f323010d820 con 0x7f32301035d0 2024-06-10T01:09:57.880 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.879+0000 7f322f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3230101a20 0x7f3230197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.880 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.879+0000 7f322f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3230101a20 0x7f3230197d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36914/0 (socket says 172.21.15.37:36914) 2024-06-10T01:09:57.880 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.879+0000 7f322f7fe640 1 -- 172.21.15.37:0/3034791700 learned_addr learned my addr 172.21.15.37:0/3034791700 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:57.880 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.879+0000 7f3226ffd640 1 --2- 172.21.15.37:0/3034791700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3230102c10 0x7f3230198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.880 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.879+0000 7f322ffff640 1 --2- 172.21.15.37:0/3034791700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32301035d0 0x7f323019f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.880 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.879+0000 7f322f7fe640 1 -- 172.21.15.37:0/3034791700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32301035d0 msgr2=0x7f323019f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:57.880 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.879+0000 7f322f7fe640 1 --2- 172.21.15.37:0/3034791700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32301035d0 0x7f323019f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.881 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.879+0000 7f322f7fe640 1 -- 172.21.15.37:0/3034791700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3230102c10 msgr2=0x7f3230198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:57.881 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.879+0000 7f322f7fe640 1 --2- 172.21.15.37:0/3034791700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3230102c10 0x7f3230198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:57.881 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.879+0000 7f322f7fe640 1 -- 172.21.15.37:0/3034791700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f323019f920 con 0x7f3230101a20 2024-06-10T01:09:57.881 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.879+0000 7f322f7fe640 1 --2- 172.21.15.37:0/3034791700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3230101a20 0x7f3230197d10 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f322000d770 tx=0x7f322000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:57.881 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.879+0000 7f322d7fa640 1 -- 172.21.15.37:0/3034791700 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3220017070 con 0x7f3230101a20 2024-06-10T01:09:57.881 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.879+0000 7f322d7fa640 1 -- 172.21.15.37:0/3034791700 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3220009c30 con 0x7f3230101a20 2024-06-10T01:09:57.881 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.879+0000 7f3235fdb640 1 -- 172.21.15.37:0/3034791700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f323019fc10 con 0x7f3230101a20 2024-06-10T01:09:57.881 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.879+0000 7f322d7fa640 1 -- 172.21.15.37:0/3034791700 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3220003dd0 con 0x7f3230101a20 2024-06-10T01:09:57.881 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.879+0000 7f3235fdb640 1 -- 172.21.15.37:0/3034791700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32301a0160 con 0x7f3230101a20 2024-06-10T01:09:57.881 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.881+0000 7f322d7fa640 1 -- 172.21.15.37:0/3034791700 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3220003f70 con 0x7f3230101a20 2024-06-10T01:09:57.881 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.882+0000 7f322d7fa640 1 --2- 172.21.15.37:0/3034791700 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f321c0a2b90 0x7f321c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:57.881 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.882+0000 7f3226ffd640 1 --2- 172.21.15.37:0/3034791700 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f321c0a2b90 0x7f321c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:57.881 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.882+0000 7f322d7fa640 1 -- 172.21.15.37:0/3034791700 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9237+0+0 (secure 0 0 0) 0x7f32200cedc0 con 0x7f3230101a20 2024-06-10T01:09:57.883 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.880+0000 7f3235fdb640 1 -- 172.21.15.37:0/3034791700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3230108d40 con 0x7f3230101a20 2024-06-10T01:09:57.884 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.889+0000 7f3226ffd640 1 --2- 172.21.15.37:0/3034791700 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f321c0a2b90 0x7f321c0a5050 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3218009bd0 tx=0x7f3218005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:57.889 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:57.894+0000 7f322d7fa640 1 -- 172.21.15.37:0/3034791700 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3220009de0 con 0x7f3230101a20 2024-06-10T01:09:58.077 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.082+0000 7f3235fdb640 1 -- 172.21.15.37:0/3034791700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "foo.217144", "pg_num": 8} v 0) -- 0x7f323002d2e0 con 0x7f3230101a20 2024-06-10T01:09:58.570 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.575+0000 7f322d7fa640 1 -- 172.21.15.37:0/3034791700 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foo.217144", "pg_num": 8}]=0 pool 'foo.217144' created v102) ==== 123+0+0 (secure 0 0 0) 0x7f322009b2e0 con 0x7f3230101a20 2024-06-10T01:09:58.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:58 smithi037 ceph-mon[32192]: pgmap v829: 129 pgs: 129 active+clean; 2.7 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-06-10T01:09:58.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:58 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:09:58.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:58 smithi037 ceph-mon[32192]: scrub queued for path: / 2024-06-10T01:09:58.603 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:58 smithi037 ceph-mon[32192]: scrub summary: idle+waiting paths [/] 2024-06-10T01:09:58.603 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:58 smithi037 ceph-mon[32192]: scrub summary: active paths [/] 2024-06-10T01:09:58.603 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:58 smithi037 ceph-mon[32192]: scrub summary: idle+waiting paths [/] 2024-06-10T01:09:58.628 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:09:58.687 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.692+0000 7f3235fdb640 1 -- 172.21.15.37:0/3034791700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "foo.217144", "pg_num": 8} v 0) -- 0x7f323002c920 con 0x7f3230101a20 2024-06-10T01:09:58.688 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.693+0000 7f322d7fa640 1 -- 172.21.15.37:0/3034791700 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foo.217144", "pg_num": 8}]=0 pool 'foo.217144' already exists v102) ==== 130+0+0 (secure 0 0 0) 0x7f322009f5e0 con 0x7f3230101a20 2024-06-10T01:09:58.688 INFO:tasks.workunit.client.0.smithi037.stderr:pool 'foo.217144' already exists 2024-06-10T01:09:58.690 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.695+0000 7f3235fdb640 1 -- 172.21.15.37:0/3034791700 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f321c0a2b90 msgr2=0x7f321c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:58.690 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.695+0000 7f3235fdb640 1 --2- 172.21.15.37:0/3034791700 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f321c0a2b90 0x7f321c0a5050 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3218009bd0 tx=0x7f3218005f90 comp rx=0 tx=0).stop 2024-06-10T01:09:58.690 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.695+0000 7f3235fdb640 1 -- 172.21.15.37:0/3034791700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3230101a20 msgr2=0x7f3230197d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:58.690 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.695+0000 7f3235fdb640 1 --2- 172.21.15.37:0/3034791700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3230101a20 0x7f3230197d10 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f322000d770 tx=0x7f322000dc30 comp rx=0 tx=0).stop 2024-06-10T01:09:58.690 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.695+0000 7f3235fdb640 1 -- 172.21.15.37:0/3034791700 shutdown_connections 2024-06-10T01:09:58.690 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.695+0000 7f3235fdb640 1 --2- 172.21.15.37:0/3034791700 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f321c0a2b90 0x7f321c0a5050 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:58.691 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.695+0000 7f3235fdb640 1 --2- 172.21.15.37:0/3034791700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32301035d0 0x7f323019f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:58.691 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.695+0000 7f3235fdb640 1 --2- 172.21.15.37:0/3034791700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3230102c10 0x7f3230198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:58.691 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.695+0000 7f3235fdb640 1 --2- 172.21.15.37:0/3034791700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3230101a20 0x7f3230197d10 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:58.691 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.695+0000 7f3235fdb640 1 -- 172.21.15.37:0/3034791700 >> 172.21.15.37:0/3034791700 conn(0x7f32300fcf90 msgr2=0x7f3230105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:58.691 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.695+0000 7f3235fdb640 1 -- 172.21.15.37:0/3034791700 shutdown_connections 2024-06-10T01:09:58.691 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.695+0000 7f3235fdb640 1 -- 172.21.15.37:0/3034791700 wait complete. 2024-06-10T01:09:58.787 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.790+0000 7fb5b6a4d640 1 -- 172.21.15.37:0/444803535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b0101b00 msgr2=0x7fb5b0101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:58.788 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.790+0000 7fb5b6a4d640 1 --2- 172.21.15.37:0/444803535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b0101b00 0x7fb5b0101f80 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fb598009900 tx=0x7fb59802a410 comp rx=0 tx=0).stop 2024-06-10T01:09:58.788 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.790+0000 7fb5b6a4d640 1 -- 172.21.15.37:0/444803535 shutdown_connections 2024-06-10T01:09:58.788 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.790+0000 7fb5b6a4d640 1 --2- 172.21.15.37:0/444803535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5b01024c0 0x7fb5b0108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:58.788 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.790+0000 7fb5b6a4d640 1 --2- 172.21.15.37:0/444803535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b0101b00 0x7fb5b0101f80 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:58.788 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.790+0000 7fb5b6a4d640 1 --2- 172.21.15.37:0/444803535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5b0100910 0x7fb5b0100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:58.788 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.790+0000 7fb5b6a4d640 1 -- 172.21.15.37:0/444803535 >> 172.21.15.37:0/444803535 conn(0x7fb5b00fbe80 msgr2=0x7fb5b00fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:58.788 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.790+0000 7fb5b6a4d640 1 -- 172.21.15.37:0/444803535 shutdown_connections 2024-06-10T01:09:58.788 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.790+0000 7fb5b6a4d640 1 -- 172.21.15.37:0/444803535 wait complete. 2024-06-10T01:09:58.788 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.791+0000 7fb5b6a4d640 1 Processor -- start 2024-06-10T01:09:58.788 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.791+0000 7fb5b6a4d640 1 -- start start 2024-06-10T01:09:58.789 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.791+0000 7fb5b6a4d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b0100910 0x7fb5b0069890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:58.789 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.791+0000 7fb5b6a4d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5b0101b00 0x7fb5b019c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:58.789 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.791+0000 7fb5b6a4d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5b01024c0 0x7fb5b019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:58.789 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.791+0000 7fb5b6a4d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b010c590 con 0x7fb5b0100910 2024-06-10T01:09:58.789 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.791+0000 7fb5b6a4d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b010c410 con 0x7fb5b0101b00 2024-06-10T01:09:58.789 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.791+0000 7fb5b6a4d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b010c710 con 0x7fb5b01024c0 2024-06-10T01:09:58.789 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.791+0000 7fb5affff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b0100910 0x7fb5b0069890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:58.789 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.791+0000 7fb5affff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b0100910 0x7fb5b0069890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36932/0 (socket says 172.21.15.37:36932) 2024-06-10T01:09:58.789 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.791+0000 7fb5affff640 1 -- 172.21.15.37:0/3223013298 learned_addr learned my addr 172.21.15.37:0/3223013298 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:58.789 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.791+0000 7fb5a7fff640 1 --2- 172.21.15.37:0/3223013298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5b0101b00 0x7fb5b019c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:58.789 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.791+0000 7fb5affff640 1 -- 172.21.15.37:0/3223013298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5b01024c0 msgr2=0x7fb5b019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:58.789 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.791+0000 7fb5affff640 1 --2- 172.21.15.37:0/3223013298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5b01024c0 0x7fb5b019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:58.789 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.791+0000 7fb5affff640 1 -- 172.21.15.37:0/3223013298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5b0101b00 msgr2=0x7fb5b019c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:58.789 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.791+0000 7fb5affff640 1 --2- 172.21.15.37:0/3223013298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5b0101b00 0x7fb5b019c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:58.790 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.791+0000 7fb5affff640 1 -- 172.21.15.37:0/3223013298 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b019f8e0 con 0x7fb5b0100910 2024-06-10T01:09:58.790 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.791+0000 7fb5a7fff640 1 --2- 172.21.15.37:0/3223013298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5b0101b00 0x7fb5b019c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:09:58.790 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.792+0000 7fb5affff640 1 --2- 172.21.15.37:0/3223013298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b0100910 0x7fb5b0069890 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fb5a000e9c0 tx=0x7fb5a000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:58.790 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.792+0000 7fb5adffb640 1 -- 172.21.15.37:0/3223013298 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a000cd80 con 0x7fb5b0100910 2024-06-10T01:09:58.790 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.792+0000 7fb5b6a4d640 1 -- 172.21.15.37:0/3223013298 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5b019fbd0 con 0x7fb5b0100910 2024-06-10T01:09:58.790 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.792+0000 7fb5adffb640 1 -- 172.21.15.37:0/3223013298 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5a00035f0 con 0x7fb5b0100910 2024-06-10T01:09:58.790 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.792+0000 7fb5adffb640 1 -- 172.21.15.37:0/3223013298 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0010690 con 0x7fb5b0100910 2024-06-10T01:09:58.790 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.792+0000 7fb5b6a4d640 1 -- 172.21.15.37:0/3223013298 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b01a0140 con 0x7fb5b0100910 2024-06-10T01:09:58.791 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.793+0000 7fb5b6a4d640 1 -- 172.21.15.37:0/3223013298 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5b0107bd0 con 0x7fb5b0100910 2024-06-10T01:09:58.791 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.794+0000 7fb5adffb640 1 -- 172.21.15.37:0/3223013298 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb5a00037a0 con 0x7fb5b0100910 2024-06-10T01:09:58.791 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.795+0000 7fb5adffb640 1 --2- 172.21.15.37:0/3223013298 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb59c0a2b90 0x7fb59c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:58.792 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.795+0000 7fb5a7fff640 1 --2- 172.21.15.37:0/3223013298 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb59c0a2b90 0x7fb59c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:58.792 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.795+0000 7fb5adffb640 1 -- 172.21.15.37:0/3223013298 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9586+0+0 (secure 0 0 0) 0x7fb5a00cfb40 con 0x7fb5b0100910 2024-06-10T01:09:58.792 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.796+0000 7fb5a7fff640 1 --2- 172.21.15.37:0/3223013298 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb59c0a2b90 0x7fb59c0a5050 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb598000c00 tx=0x7fb5980095f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:58.798 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:58.802+0000 7fb5adffb640 1 -- 172.21.15.37:0/3223013298 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb5a009c5e0 con 0x7fb5b0100910 2024-06-10T01:09:58.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:58 smithi167 ceph-mon[36917]: pgmap v829: 129 pgs: 129 active+clean; 2.7 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-06-10T01:09:58.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:58 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:09:58.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:58 smithi167 ceph-mon[36917]: scrub queued for path: / 2024-06-10T01:09:58.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:58 smithi167 ceph-mon[36917]: scrub summary: idle+waiting paths [/] 2024-06-10T01:09:58.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:58 smithi167 ceph-mon[36917]: scrub summary: active paths [/] 2024-06-10T01:09:58.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:58 smithi167 ceph-mon[36917]: scrub summary: idle+waiting paths [/] 2024-06-10T01:09:58.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:58 smithi094 ceph-mon[36708]: pgmap v829: 129 pgs: 129 active+clean; 2.7 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-06-10T01:09:58.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:58 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:09:58.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:58 smithi094 ceph-mon[36708]: scrub queued for path: / 2024-06-10T01:09:58.860 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:58 smithi094 ceph-mon[36708]: scrub summary: idle+waiting paths [/] 2024-06-10T01:09:58.860 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:58 smithi094 ceph-mon[36708]: scrub summary: active paths [/] 2024-06-10T01:09:58.860 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:58 smithi094 ceph-mon[36708]: scrub summary: idle+waiting paths [/] 2024-06-10T01:09:58.895 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:09:59.041 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.046+0000 7fb5b6a4d640 1 -- 172.21.15.37:0/3223013298 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.217144"} v 0) -- 0x7fb5b002d540 con 0x7fb5b0100910 2024-06-10T01:09:59.577 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.581+0000 7fb5adffb640 1 -- 172.21.15.37:0/3223013298 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.217144"}]=0 added data pool 5 to fsmap v32) ==== 133+0+0 (secure 0 0 0) 0x7fb5a009bee0 con 0x7fb5b0100910 2024-06-10T01:09:59.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:59 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3034791700' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo.217144", "pg_num": 8}]: dispatch 2024-06-10T01:09:59.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:59 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3034791700' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "foo.217144", "pg_num": 8}]': finished 2024-06-10T01:09:59.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:59 smithi037 ceph-mon[32192]: osdmap e102: 12 total, 12 up, 12 in 2024-06-10T01:09:59.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:59 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3034791700' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo.217144", "pg_num": 8}]: dispatch 2024-06-10T01:09:59.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:59 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3223013298' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.217144"}]: dispatch 2024-06-10T01:09:59.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:09:59 smithi037 ceph-mon[32192]: pgmap v831: 137 pgs: 8 unknown, 129 active+clean; 2.8 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-06-10T01:09:59.729 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.734+0000 7fb5b6a4d640 1 -- 172.21.15.37:0/3223013298 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.217144"} v 0) -- 0x7fb5b0062640 con 0x7fb5b0100910 2024-06-10T01:09:59.729 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.734+0000 7fb5adffb640 1 -- 172.21.15.37:0/3223013298 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.217144"}]=-22 RADOS pool 'foo.217144' is already used by filesystem 'cephfs' as a 'data' pool for application 'cephfs' v32) ==== 211+0+0 (secure 0 0 0) 0x7fb5a00a0290 con 0x7fb5b0100910 2024-06-10T01:09:59.730 INFO:tasks.workunit.client.0.smithi037.stderr:Second attempt of previously successful command failed with EINVAL: RADOS pool 'foo.217144' is already used by filesystem 'cephfs' as a 'data' pool for application 'cephfs' 2024-06-10T01:09:59.730 INFO:tasks.workunit.client.0.smithi037.stderr:RADOS pool 'foo.217144' is already used by filesystem 'cephfs' as a 'data' pool for application 'cephfs' 2024-06-10T01:09:59.732 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.736+0000 7fb5b6a4d640 1 -- 172.21.15.37:0/3223013298 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb59c0a2b90 msgr2=0x7fb59c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:59.732 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.736+0000 7fb5b6a4d640 1 --2- 172.21.15.37:0/3223013298 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb59c0a2b90 0x7fb59c0a5050 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb598000c00 tx=0x7fb5980095f0 comp rx=0 tx=0).stop 2024-06-10T01:09:59.732 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.736+0000 7fb5b6a4d640 1 -- 172.21.15.37:0/3223013298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b0100910 msgr2=0x7fb5b0069890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:59.732 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.736+0000 7fb5b6a4d640 1 --2- 172.21.15.37:0/3223013298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b0100910 0x7fb5b0069890 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fb5a000e9c0 tx=0x7fb5a000ee80 comp rx=0 tx=0).stop 2024-06-10T01:09:59.732 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.736+0000 7fb5b6a4d640 1 -- 172.21.15.37:0/3223013298 shutdown_connections 2024-06-10T01:09:59.732 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.736+0000 7fb5b6a4d640 1 --2- 172.21.15.37:0/3223013298 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb59c0a2b90 0x7fb59c0a5050 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:59.732 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.736+0000 7fb5b6a4d640 1 --2- 172.21.15.37:0/3223013298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5b01024c0 0x7fb5b019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:59.732 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.736+0000 7fb5b6a4d640 1 --2- 172.21.15.37:0/3223013298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5b0101b00 0x7fb5b019c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:59.732 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.736+0000 7fb5b6a4d640 1 --2- 172.21.15.37:0/3223013298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5b0100910 0x7fb5b0069890 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:59.732 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.736+0000 7fb5b6a4d640 1 -- 172.21.15.37:0/3223013298 >> 172.21.15.37:0/3223013298 conn(0x7fb5b00fbe80 msgr2=0x7fb5b0104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:59.732 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.736+0000 7fb5b6a4d640 1 -- 172.21.15.37:0/3223013298 shutdown_connections 2024-06-10T01:09:59.732 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.736+0000 7fb5b6a4d640 1 -- 172.21.15.37:0/3223013298 wait complete. 2024-06-10T01:09:59.833 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:59 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3034791700' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo.217144", "pg_num": 8}]: dispatch 2024-06-10T01:09:59.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:59 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3034791700' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "foo.217144", "pg_num": 8}]': finished 2024-06-10T01:09:59.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:59 smithi167 ceph-mon[36917]: osdmap e102: 12 total, 12 up, 12 in 2024-06-10T01:09:59.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:59 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3034791700' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo.217144", "pg_num": 8}]: dispatch 2024-06-10T01:09:59.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:59 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3223013298' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.217144"}]: dispatch 2024-06-10T01:09:59.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:09:59 smithi167 ceph-mon[36917]: pgmap v831: 137 pgs: 8 unknown, 129 active+clean; 2.8 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-06-10T01:09:59.837 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.842+0000 7f0ba91ee640 1 -- 172.21.15.37:0/2431241874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ba41035d0 msgr2=0x7f0ba4109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:59.838 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.842+0000 7f0ba91ee640 1 --2- 172.21.15.37:0/2431241874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ba41035d0 0x7f0ba4109e60 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c009ca0 tx=0x7f0b9c02a8c0 comp rx=0 tx=0).stop 2024-06-10T01:09:59.838 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.842+0000 7f0ba91ee640 1 -- 172.21.15.37:0/2431241874 shutdown_connections 2024-06-10T01:09:59.838 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.842+0000 7f0ba91ee640 1 --2- 172.21.15.37:0/2431241874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ba41035d0 0x7f0ba4109e60 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:59.838 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.842+0000 7f0ba91ee640 1 --2- 172.21.15.37:0/2431241874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ba4102c10 0x7f0ba4103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:59.838 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.842+0000 7f0ba91ee640 1 --2- 172.21.15.37:0/2431241874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ba4101a20 0x7f0ba4101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:59.838 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.842+0000 7f0ba91ee640 1 -- 172.21.15.37:0/2431241874 >> 172.21.15.37:0/2431241874 conn(0x7f0ba40fcf90 msgr2=0x7f0ba40ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:09:59.838 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.842+0000 7f0ba91ee640 1 -- 172.21.15.37:0/2431241874 shutdown_connections 2024-06-10T01:09:59.838 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.842+0000 7f0ba91ee640 1 -- 172.21.15.37:0/2431241874 wait complete. 2024-06-10T01:09:59.840 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.843+0000 7f0ba91ee640 1 Processor -- start 2024-06-10T01:09:59.841 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.843+0000 7f0ba91ee640 1 -- start start 2024-06-10T01:09:59.841 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.843+0000 7f0ba91ee640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ba4101a20 0x7f0ba4197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:59.841 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.843+0000 7f0ba91ee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ba4102c10 0x7f0ba41981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:59.841 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.843+0000 7f0ba91ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ba41035d0 0x7f0ba419f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:59.841 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.843+0000 7f0ba91ee640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba410d6a0 con 0x7f0ba4102c10 2024-06-10T01:09:59.841 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.843+0000 7f0ba91ee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba410d520 con 0x7f0ba41035d0 2024-06-10T01:09:59.841 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.843+0000 7f0ba91ee640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba410d820 con 0x7f0ba4101a20 2024-06-10T01:09:59.841 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.843+0000 7f0ba259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ba4102c10 0x7f0ba41981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:59.841 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.843+0000 7f0ba259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ba4102c10 0x7f0ba41981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36950/0 (socket says 172.21.15.37:36950) 2024-06-10T01:09:59.841 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.843+0000 7f0ba259c640 1 -- 172.21.15.37:0/2788632844 learned_addr learned my addr 172.21.15.37:0/2788632844 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:09:59.841 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.844+0000 7f0ba2d9d640 1 --2- 172.21.15.37:0/2788632844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ba4101a20 0x7f0ba4197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:59.841 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.844+0000 7f0ba359e640 1 --2- 172.21.15.37:0/2788632844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ba41035d0 0x7f0ba419f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:59.841 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.844+0000 7f0ba259c640 1 -- 172.21.15.37:0/2788632844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ba4101a20 msgr2=0x7f0ba4197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:59.842 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.844+0000 7f0ba259c640 1 --2- 172.21.15.37:0/2788632844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ba4101a20 0x7f0ba4197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:59.842 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.844+0000 7f0ba259c640 1 -- 172.21.15.37:0/2788632844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ba41035d0 msgr2=0x7f0ba419f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:09:59.842 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.844+0000 7f0ba259c640 1 --2- 172.21.15.37:0/2788632844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ba41035d0 0x7f0ba419f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:09:59.842 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.844+0000 7f0ba259c640 1 -- 172.21.15.37:0/2788632844 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba419f8c0 con 0x7f0ba4102c10 2024-06-10T01:09:59.842 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.844+0000 7f0ba259c640 1 --2- 172.21.15.37:0/2788632844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ba4102c10 0x7f0ba41981c0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f0b9800d950 tx=0x7f0b9800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:59.842 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.844+0000 7f0b8bfff640 1 -- 172.21.15.37:0/2788632844 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b98017070 con 0x7f0ba4102c10 2024-06-10T01:09:59.842 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.844+0000 7f0b8bfff640 1 -- 172.21.15.37:0/2788632844 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b98009c30 con 0x7f0ba4102c10 2024-06-10T01:09:59.842 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.844+0000 7f0ba91ee640 1 -- 172.21.15.37:0/2788632844 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba419fbb0 con 0x7f0ba4102c10 2024-06-10T01:09:59.842 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.844+0000 7f0b8bfff640 1 -- 172.21.15.37:0/2788632844 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9800f6b0 con 0x7f0ba4102c10 2024-06-10T01:09:59.842 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.844+0000 7f0ba91ee640 1 -- 172.21.15.37:0/2788632844 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba41a0100 con 0x7f0ba4102c10 2024-06-10T01:09:59.842 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.845+0000 7f0ba91ee640 1 -- 172.21.15.37:0/2788632844 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ba4108d40 con 0x7f0ba4102c10 2024-06-10T01:09:59.843 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.847+0000 7f0b8bfff640 1 -- 172.21.15.37:0/2788632844 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f0b980109d0 con 0x7f0ba4102c10 2024-06-10T01:09:59.843 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.847+0000 7f0b8bfff640 1 --2- 172.21.15.37:0/2788632844 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0b8c0a2b90 0x7f0b8c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:09:59.843 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.847+0000 7f0ba2d9d640 1 --2- 172.21.15.37:0/2788632844 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0b8c0a2b90 0x7f0b8c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:09:59.843 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.847+0000 7f0b8bfff640 1 -- 172.21.15.37:0/2788632844 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9618+0+0 (secure 0 0 0) 0x7f0b980cf110 con 0x7f0ba4102c10 2024-06-10T01:09:59.844 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.848+0000 7f0ba2d9d640 1 --2- 172.21.15.37:0/2788632844 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0b8c0a2b90 0x7f0b8c0a5050 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4198b50 tx=0x7f0b9400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:09:59.846 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:09:59.851+0000 7f0b8bfff640 1 -- 172.21.15.37:0/2788632844 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0b98009de0 con 0x7f0ba4102c10 2024-06-10T01:09:59.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:59 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3034791700' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo.217144", "pg_num": 8}]: dispatch 2024-06-10T01:09:59.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:59 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3034791700' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "foo.217144", "pg_num": 8}]': finished 2024-06-10T01:09:59.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:59 smithi094 ceph-mon[36708]: osdmap e102: 12 total, 12 up, 12 in 2024-06-10T01:09:59.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:59 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3034791700' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo.217144", "pg_num": 8}]: dispatch 2024-06-10T01:09:59.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:59 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3223013298' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.217144"}]: dispatch 2024-06-10T01:09:59.860 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:09:59 smithi094 ceph-mon[36708]: pgmap v831: 137 pgs: 8 unknown, 129 active+clean; 2.8 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-06-10T01:10:00.075 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:00.080+0000 7f0ba91ee640 1 -- 172.21.15.37:0/2788632844 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.217144"} v 0) -- 0x7f0ba402d2e0 con 0x7f0ba4102c10 2024-06-10T01:10:00.579 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:00.583+0000 7f0b8bfff640 1 -- 172.21.15.37:0/2788632844 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.217144"}]=0 removed data pool 5 from fsmap v33) ==== 136+0+0 (secure 0 0 0) 0x7f0b9809b4b0 con 0x7f0ba4102c10 2024-06-10T01:10:00.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:00 smithi037 ceph-mon[32192]: osdmap e103: 12 total, 12 up, 12 in 2024-06-10T01:10:00.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:00 smithi037 ceph-mon[32192]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-10T01:10:00.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:00 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3223013298' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.217144"}]': finished 2024-06-10T01:10:00.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:00 smithi037 ceph-mon[32192]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:10:00.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:00 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3223013298' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.217144"}]: dispatch 2024-06-10T01:10:00.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:00 smithi037 ceph-mon[32192]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-06-10T01:10:00.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:00 smithi037 ceph-mon[32192]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-06-10T01:10:00.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:00 smithi037 ceph-mon[32192]: application not enabled on pool 'foo.217144' 2024-06-10T01:10:00.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:00 smithi037 ceph-mon[32192]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-06-10T01:10:00.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:00 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2788632844' entity='client.admin' cmd=[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.217144"}]: dispatch 2024-06-10T01:10:00.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.665+0000 7f1c2c3a7700 1 -- 172.21.15.37:0/1117930540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c24101380 msgr2=0x7f1c24101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:00.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.665+0000 7f1c2c3a7700 1 --2- 172.21.15.37:0/1117930540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c24101380 0x7f1c24101800 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009990 tx=0x7f1c18009ca0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.665+0000 7f1c2c3a7700 1 -- 172.21.15.37:0/1117930540 shutdown_connections 2024-06-10T01:10:00.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.665+0000 7f1c2c3a7700 1 --2- 172.21.15.37:0/1117930540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c24101d40 0x7f1c2410e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.665+0000 7f1c2c3a7700 1 --2- 172.21.15.37:0/1117930540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c241073f0 0x7f1c241077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.665+0000 7f1c2c3a7700 1 --2- 172.21.15.37:0/1117930540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c24101380 0x7f1c24101800 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.665+0000 7f1c2c3a7700 1 -- 172.21.15.37:0/1117930540 >> 172.21.15.37:0/1117930540 conn(0x7f1c24078300 msgr2=0x7f1c24078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:00.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.665+0000 7f1c2c3a7700 1 -- 172.21.15.37:0/1117930540 shutdown_connections 2024-06-10T01:10:00.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.665+0000 7f1c2c3a7700 1 -- 172.21.15.37:0/1117930540 wait complete. 2024-06-10T01:10:00.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.666+0000 7f1c2c3a7700 1 Processor -- start 2024-06-10T01:10:00.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.666+0000 7f1c2c3a7700 1 -- start start 2024-06-10T01:10:00.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.666+0000 7f1c2c3a7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c24101380 0x7f1c24199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:00.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.666+0000 7f1c2c3a7700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c24101d40 0x7f1c2419a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:00.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.666+0000 7f1c2c3a7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c241073f0 0x7f1c241a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:00.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.666+0000 7f1c2c3a7700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1c24111660 con 0x7f1c241073f0 2024-06-10T01:10:00.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.666+0000 7f1c2c3a7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1c241114e0 con 0x7f1c24101380 2024-06-10T01:10:00.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.666+0000 7f1c2c3a7700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1c241117e0 con 0x7f1c24101d40 2024-06-10T01:10:00.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.666+0000 7f1c2a944700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c241073f0 0x7f1c241a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:00.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.666+0000 7f1c2a944700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c241073f0 0x7f1c241a4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36966/0 (socket says 172.21.15.37:36966) 2024-06-10T01:10:00.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.666+0000 7f1c2a944700 1 -- 172.21.15.37:0/2293339852 learned_addr learned my addr 172.21.15.37:0/2293339852 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:00.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.666+0000 7f1c29942700 1 --2- 172.21.15.37:0/2293339852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c24101d40 0x7f1c2419a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:00.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.666+0000 7f1c2a143700 1 --2- 172.21.15.37:0/2293339852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c24101380 0x7f1c24199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:00.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.667+0000 7f1c2a944700 1 -- 172.21.15.37:0/2293339852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c24101d40 msgr2=0x7f1c2419a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:00.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.667+0000 7f1c2a944700 1 --2- 172.21.15.37:0/2293339852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c24101d40 0x7f1c2419a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.667+0000 7f1c2a944700 1 -- 172.21.15.37:0/2293339852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c24101380 msgr2=0x7f1c24199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:00.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.667+0000 7f1c2a944700 1 --2- 172.21.15.37:0/2293339852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c24101380 0x7f1c24199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.667+0000 7f1c2a944700 1 -- 172.21.15.37:0/2293339852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c241a50a0 con 0x7f1c241073f0 2024-06-10T01:10:00.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.667+0000 7f1c2a944700 1 --2- 172.21.15.37:0/2293339852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c241073f0 0x7f1c241a4b60 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c007ee0 tx=0x7f1c1c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:00.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.667+0000 7f1c177fe700 1 -- 172.21.15.37:0/2293339852 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c004bb0 con 0x7f1c241073f0 2024-06-10T01:10:00.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.667+0000 7f1c177fe700 1 -- 172.21.15.37:0/2293339852 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c1c004d50 con 0x7f1c241073f0 2024-06-10T01:10:00.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.667+0000 7f1c2c3a7700 1 -- 172.21.15.37:0/2293339852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c241a5270 con 0x7f1c241073f0 2024-06-10T01:10:00.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.667+0000 7f1c177fe700 1 -- 172.21.15.37:0/2293339852 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c005960 con 0x7f1c241073f0 2024-06-10T01:10:00.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.667+0000 7f1c2c3a7700 1 -- 172.21.15.37:0/2293339852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c241a5730 con 0x7f1c241073f0 2024-06-10T01:10:00.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.669+0000 7f1c177fe700 1 -- 172.21.15.37:0/2293339852 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f1c1c01a020 con 0x7f1c241073f0 2024-06-10T01:10:00.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.670+0000 7f1c177fe700 1 --2- 172.21.15.37:0/2293339852 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1c100a2c20 0x7f1c100a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:00.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.670+0000 7f1c2a143700 1 --2- 172.21.15.37:0/2293339852 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1c100a2c20 0x7f1c100a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:00.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.670+0000 7f1c177fe700 1 -- 172.21.15.37:0/2293339852 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9618+0+0 (secure 0 0 0) 0x7f1c1c0ced50 con 0x7f1c241073f0 2024-06-10T01:10:00.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.671+0000 7f1c2a143700 1 --2- 172.21.15.37:0/2293339852 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1c100a2c20 0x7f1c100a50e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1c20005950 tx=0x7f1c2000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:00.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.672+0000 7f1c36c2ab80 1 Processor -- start 2024-06-10T01:10:00.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.672+0000 7f1c36c2ab80 1 -- start start 2024-06-10T01:10:00.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.673+0000 7f1c36c2ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca91d2a640 0x55ca91d2aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:00.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.673+0000 7f1c36c2ab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca91d2af60 0x55ca91d249e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:00.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.673+0000 7f1c36c2ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca91d24f20 0x55ca91d25340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:00.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.673+0000 7f1c36c2ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ca91c76810 con 0x55ca91d2a640 2024-06-10T01:10:00.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.673+0000 7f1c36c2ab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ca91c1e720 con 0x55ca91d2af60 2024-06-10T01:10:00.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.673+0000 7f1c36c2ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ca91c727d0 con 0x55ca91d24f20 2024-06-10T01:10:00.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.673+0000 7f1c29942700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca91d2af60 0x55ca91d249e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:00.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.673+0000 7f1c2a143700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca91d2a640 0x55ca91d2aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:00.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.673+0000 7f1c29942700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca91d2af60 0x55ca91d249e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:46240/0 (socket says 172.21.15.37:46240) 2024-06-10T01:10:00.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.673+0000 7f1c29942700 1 -- 172.21.15.37:0/652432392 learned_addr learned my addr 172.21.15.37:0/652432392 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:00.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.673+0000 7f1c2a143700 1 --2- 172.21.15.37:0/652432392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca91d2a640 0x55ca91d2aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36972/0 (socket says 172.21.15.37:36972) 2024-06-10T01:10:00.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.673+0000 7f1c2a944700 1 --2- 172.21.15.37:0/652432392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca91d24f20 0x55ca91d25340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:00.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.673+0000 7f1c2a143700 1 -- 172.21.15.37:0/652432392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca91d24f20 msgr2=0x55ca91d25340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:00.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.673+0000 7f1c2a143700 1 --2- 172.21.15.37:0/652432392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca91d24f20 0x55ca91d25340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.673+0000 7f1c2a143700 1 -- 172.21.15.37:0/652432392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca91d2af60 msgr2=0x55ca91d249e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:00.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.673+0000 7f1c2a143700 1 --2- 172.21.15.37:0/652432392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca91d2af60 0x55ca91d249e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.673+0000 7f1c2a143700 1 -- 172.21.15.37:0/652432392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca91c14f50 con 0x55ca91d2a640 2024-06-10T01:10:00.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.673+0000 7f1c29942700 1 --2- 172.21.15.37:0/652432392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca91d2af60 0x55ca91d249e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:00.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.673+0000 7f1c2a944700 1 --2- 172.21.15.37:0/652432392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca91d24f20 0x55ca91d25340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:00.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.674+0000 7f1c2a143700 1 --2- 172.21.15.37:0/652432392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca91d2a640 0x55ca91d2aa20 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f1c20010d70 tx=0x7f1c20013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a07f5fe1930c09c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:00.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.674+0000 7f1c14d70700 1 -- 172.21.15.37:0/652432392 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c20011db0 con 0x55ca91d2a640 2024-06-10T01:10:00.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.674+0000 7f1c14d70700 1 -- 172.21.15.37:0/652432392 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c20010e60 con 0x55ca91d2a640 2024-06-10T01:10:00.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.674+0000 7f1c14d70700 1 -- 172.21.15.37:0/652432392 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2001c710 con 0x55ca91d2a640 2024-06-10T01:10:00.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.675+0000 7f1c36c2ab80 1 -- 172.21.15.37:0/652432392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca91d2a640 msgr2=0x55ca91d2aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:00.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.675+0000 7f1c36c2ab80 1 --2- 172.21.15.37:0/652432392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca91d2a640 0x55ca91d2aa20 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f1c20010d70 tx=0x7f1c20013000 comp rx=0 tx=0).stop 2024-06-10T01:10:00.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.675+0000 7f1c36c2ab80 1 -- 172.21.15.37:0/652432392 shutdown_connections 2024-06-10T01:10:00.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.675+0000 7f1c36c2ab80 1 --2- 172.21.15.37:0/652432392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca91d24f20 0x55ca91d25340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.675+0000 7f1c36c2ab80 1 --2- 172.21.15.37:0/652432392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca91d2a640 0x55ca91d2aa20 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.675+0000 7f1c36c2ab80 1 --2- 172.21.15.37:0/652432392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca91d2af60 0x55ca91d249e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.675+0000 7f1c36c2ab80 1 -- 172.21.15.37:0/652432392 >> 172.21.15.37:0/652432392 conn(0x55ca91c4fcc0 msgr2=0x55ca91d1f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:00.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.675+0000 7f1c36c2ab80 1 -- 172.21.15.37:0/652432392 shutdown_connections 2024-06-10T01:10:00.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.675+0000 7f1c36c2ab80 1 -- 172.21.15.37:0/652432392 wait complete. 2024-06-10T01:10:00.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.676+0000 7f1c36c2ab80 1 -- start start 2024-06-10T01:10:00.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.676+0000 7f1c36c2ab80 1 Processor -- start 2024-06-10T01:10:00.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.676+0000 7f1c36c2ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca91d24f20 0x55ca91d30cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:00.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.676+0000 7f1c36c2ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca91d2a640 0x55ca91d2dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:00.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.676+0000 7f1c36c2ab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca91d2af60 0x55ca91d2e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:00.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.676+0000 7f1c36c2ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ca91d319d0 con 0x55ca91d2a640 2024-06-10T01:10:00.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.676+0000 7f1c36c2ab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ca91c16bf0 con 0x55ca91d2af60 2024-06-10T01:10:00.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.676+0000 7f1c36c2ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ca91d31b50 con 0x55ca91d24f20 2024-06-10T01:10:00.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.676+0000 7f1c29942700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca91d2a640 0x55ca91d2dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:00.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.676+0000 7f1c29942700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca91d2a640 0x55ca91d2dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36982/0 (socket says 172.21.15.37:36982) 2024-06-10T01:10:00.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.676+0000 7f1c2a143700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca91d24f20 0x55ca91d30cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:00.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.676+0000 7f1c29942700 1 -- 172.21.15.37:0/3331361794 learned_addr learned my addr 172.21.15.37:0/3331361794 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:00.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.677+0000 7f1c2a944700 1 --2- 172.21.15.37:0/3331361794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca91d2af60 0x55ca91d2e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:00.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.677+0000 7f1c36c2ab80 10 client.0 _ll_register_callbacks cb 0x55ca91c10920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:00.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.677+0000 7f1c29942700 1 -- 172.21.15.37:0/3331361794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca91d24f20 msgr2=0x55ca91d30cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:00.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.677+0000 7f1c29942700 1 --2- 172.21.15.37:0/3331361794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca91d24f20 0x55ca91d30cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.677+0000 7f1c29942700 1 -- 172.21.15.37:0/3331361794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca91d2af60 msgr2=0x55ca91d2e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:00.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.677+0000 7f1c29942700 1 --2- 172.21.15.37:0/3331361794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca91d2af60 0x55ca91d2e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.677+0000 7f1c29942700 1 -- 172.21.15.37:0/3331361794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca91bc58d0 con 0x55ca91d2a640 2024-06-10T01:10:00.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.677+0000 7f1c2a143700 1 --2- 172.21.15.37:0/3331361794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca91d24f20 0x55ca91d30cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:00.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.677+0000 7f1c2a944700 1 --2- 172.21.15.37:0/3331361794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca91d2af60 0x55ca91d2e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:00.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.677+0000 7f1c29942700 1 --2- 172.21.15.37:0/3331361794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca91d2a640 0x55ca91d2dff0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f1c180098e0 tx=0x7f1c1800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f3a6d9bed1d836 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:00.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.677+0000 7f1c0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:10:00.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.677+0000 7f1c0dffb700 1 -- 172.21.15.37:0/3331361794 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18029060 con 0x55ca91d2a640 2024-06-10T01:10:00.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.677+0000 7f1c36c2ab80 1 -- 172.21.15.37:0/3331361794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c20010e60 con 0x55ca91d2a640 2024-06-10T01:10:00.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.677+0000 7f1c0dffb700 1 -- 172.21.15.37:0/3331361794 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9618+0+0 (secure 0 0 0) 0x7f1c18003d70 con 0x55ca91d2a640 2024-06-10T01:10:00.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.677+0000 7f1c0dffb700 1 -- 172.21.15.37:0/3331361794 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1c18030050 con 0x55ca91d2a640 2024-06-10T01:10:00.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.678+0000 7f1c36c2ab80 10 client.17829 fetch_fsmap learned FSMap version 33 2024-06-10T01:10:00.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.678+0000 7f1c36c2ab80 1 -- 172.21.15.37:0/3331361794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ca91d40770 con 0x55ca91d2a640 2024-06-10T01:10:00.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.678+0000 7f1c0dffb700 1 -- 172.21.15.37:0/3331361794 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c20010e60 con 0x55ca91d2a640 2024-06-10T01:10:00.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.678+0000 7f1c0dffb700 1 -- 172.21.15.37:0/3331361794 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1c1800f980 con 0x55ca91d2a640 2024-06-10T01:10:00.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.678+0000 7f1c36c2ab80 10 client.17829 fetch_fsmap finished waiting for FSMap version 33 2024-06-10T01:10:00.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.678+0000 7f1c36c2ab80 10 client.17829 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:10:00.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.678+0000 7f1c36c2ab80 20 client.17829 populate_metadata read hostname 'smithi037' 2024-06-10T01:10:00.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.678+0000 7f1c36c2ab80 1 --2- 172.21.15.37:0/3331361794 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55ca91d42d00 0x55ca91d450f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:00.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.678+0000 7f1c36c2ab80 4 client.17829 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:10:00.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.678+0000 7f1c36c2ab80 1 -- 172.21.15.37:0/3331361794 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca91c83d10 con 0x55ca91d42d00 2024-06-10T01:10:00.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.678+0000 7f1c2a143700 1 --2- 172.21.15.37:0/3331361794 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55ca91d42d00 0x55ca91d450f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:00.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.679+0000 7f1c2a143700 1 --2- 172.21.15.37:0/3331361794 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55ca91d42d00 0x55ca91d450f0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b7be30296fbeec7 server_cookie=b95dd69819f29b44 in_seq=0 out_seq=0 2024-06-10T01:10:00.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.679+0000 7f1c0dffb700 10 client.17829 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:10:00.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.680+0000 7f1c0dffb700 1 -- 172.21.15.37:0/3331361794 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ca91c83d10 con 0x55ca91d42d00 2024-06-10T01:10:00.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.680+0000 7f1c0dffb700 10 client.17829 handle_command_reply: tid=0 2024-06-10T01:10:00.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.680+0000 7f1c36c2ab80 1 client.17829 shutdown 2024-06-10T01:10:00.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.680+0000 7f1c36c2ab80 1 -- 172.21.15.37:0/3331361794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca91d2a640 msgr2=0x55ca91d2dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:00.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.680+0000 7f1c36c2ab80 1 --2- 172.21.15.37:0/3331361794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca91d2a640 0x55ca91d2dff0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f1c180098e0 tx=0x7f1c1800bfd0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.680+0000 7f1c36c2ab80 1 -- 172.21.15.37:0/3331361794 shutdown_connections 2024-06-10T01:10:00.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.680+0000 7f1c36c2ab80 1 --2- 172.21.15.37:0/3331361794 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55ca91d42d00 0x55ca91d450f0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.680+0000 7f1c36c2ab80 1 --2- 172.21.15.37:0/3331361794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca91d2af60 0x55ca91d2e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.680+0000 7f1c36c2ab80 1 --2- 172.21.15.37:0/3331361794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca91d24f20 0x55ca91d30cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.680+0000 7f1c36c2ab80 1 --2- 172.21.15.37:0/3331361794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca91d2a640 0x55ca91d2dff0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.680+0000 7f1c36c2ab80 1 -- 172.21.15.37:0/3331361794 >> 172.21.15.37:0/3331361794 conn(0x55ca91c4fcc0 msgr2=0x55ca91d239d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:00.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.680+0000 7f1c0dffb700 0 client.17829 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:10:00.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.680+0000 7f1c36c2ab80 1 -- 172.21.15.37:0/3331361794 shutdown_connections 2024-06-10T01:10:00.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.680+0000 7f1c36c2ab80 1 -- 172.21.15.37:0/3331361794 wait complete. 2024-06-10T01:10:00.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.680+0000 7f1c36c2ab80 20 client.17829 trim_cache size 0 max 16384 2024-06-10T01:10:00.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.715+0000 7f1c36c2ab80 1 Processor -- start 2024-06-10T01:10:00.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.715+0000 7f1c36c2ab80 1 -- start start 2024-06-10T01:10:00.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.715+0000 7f1c36c2ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca91d24f20 0x55ca91d249e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:00.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.715+0000 7f1c36c2ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca91d2a640 0x55ca91d2cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:00.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.715+0000 7f1c36c2ab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca91d2af60 0x55ca91d1f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:00.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.715+0000 7f1c36c2ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1c241af230 con 0x55ca91d2a640 2024-06-10T01:10:00.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.715+0000 7f1c36c2ab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1c241acac0 con 0x55ca91d2af60 2024-06-10T01:10:00.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.715+0000 7f1c36c2ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1c241aa910 con 0x55ca91d24f20 2024-06-10T01:10:00.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.715+0000 7f1c29942700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca91d2a640 0x55ca91d2cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:00.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.715+0000 7f1c29942700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca91d2a640 0x55ca91d2cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36984/0 (socket says 172.21.15.37:36984) 2024-06-10T01:10:00.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.715+0000 7f1c29942700 1 -- 172.21.15.37:0/841674802 learned_addr learned my addr 172.21.15.37:0/841674802 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:00.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.715+0000 7f1c2a143700 1 --2- 172.21.15.37:0/841674802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca91d24f20 0x55ca91d249e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:00.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.715+0000 7f1c2a944700 1 --2- 172.21.15.37:0/841674802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca91d2af60 0x55ca91d1f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:00.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.715+0000 7f1c29942700 1 -- 172.21.15.37:0/841674802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca91d24f20 msgr2=0x55ca91d249e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:00.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.715+0000 7f1c29942700 1 --2- 172.21.15.37:0/841674802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca91d24f20 0x55ca91d249e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.715+0000 7f1c29942700 1 -- 172.21.15.37:0/841674802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca91d2af60 msgr2=0x55ca91d1f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:00.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.715+0000 7f1c29942700 1 --2- 172.21.15.37:0/841674802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca91d2af60 0x55ca91d1f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.715+0000 7f1c29942700 1 -- 172.21.15.37:0/841674802 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca91d21080 con 0x55ca91d2a640 2024-06-10T01:10:00.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.715+0000 7f1c2a944700 1 --2- 172.21.15.37:0/841674802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca91d2af60 0x55ca91d1f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:10:00.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.716+0000 7f1c29942700 1 --2- 172.21.15.37:0/841674802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca91d2a640 0x55ca91d2cc50 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800ff10 tx=0x7f1c18005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7414d0aeb4cf9ee7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:00.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.716+0000 7f1c0ffff700 1 -- 172.21.15.37:0/841674802 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18039070 con 0x55ca91d2a640 2024-06-10T01:10:00.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.716+0000 7f1c0ffff700 1 -- 172.21.15.37:0/841674802 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c18003740 con 0x55ca91d2a640 2024-06-10T01:10:00.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.716+0000 7f1c0ffff700 1 -- 172.21.15.37:0/841674802 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18029070 con 0x55ca91d2a640 2024-06-10T01:10:00.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.717+0000 7f1c36c2ab80 1 -- 172.21.15.37:0/841674802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca91d2a640 msgr2=0x55ca91d2cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:00.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.717+0000 7f1c36c2ab80 1 --2- 172.21.15.37:0/841674802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca91d2a640 0x55ca91d2cc50 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800ff10 tx=0x7f1c18005c70 comp rx=0 tx=0).stop 2024-06-10T01:10:00.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.717+0000 7f1c36c2ab80 1 -- 172.21.15.37:0/841674802 shutdown_connections 2024-06-10T01:10:00.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.717+0000 7f1c36c2ab80 1 --2- 172.21.15.37:0/841674802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca91d2af60 0x55ca91d1f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.717+0000 7f1c36c2ab80 1 --2- 172.21.15.37:0/841674802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca91d24f20 0x55ca91d249e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.717+0000 7f1c36c2ab80 1 --2- 172.21.15.37:0/841674802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca91d2a640 0x55ca91d2cc50 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.717+0000 7f1c36c2ab80 1 -- 172.21.15.37:0/841674802 >> 172.21.15.37:0/841674802 conn(0x55ca91c4fcc0 msgr2=0x55ca91d2b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:00.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.717+0000 7f1c36c2ab80 1 -- 172.21.15.37:0/841674802 shutdown_connections 2024-06-10T01:10:00.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.717+0000 7f1c36c2ab80 1 -- 172.21.15.37:0/841674802 wait complete. 2024-06-10T01:10:00.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.717+0000 7f1c36c2ab80 1 -- start start 2024-06-10T01:10:00.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.717+0000 7f1c36c2ab80 1 Processor -- start 2024-06-10T01:10:00.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.717+0000 7f1c36c2ab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca91d24f20 0x55ca91d4a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:00.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.717+0000 7f1c36c2ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca91d2a640 0x55ca91d4ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:00.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.717+0000 7f1c36c2ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca91d2af60 0x55ca91d4b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:00.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.717+0000 7f1c36c2ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1c241aaf80 con 0x55ca91d2a640 2024-06-10T01:10:00.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.717+0000 7f1c36c2ab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1c241ac3f0 con 0x55ca91d24f20 2024-06-10T01:10:00.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.717+0000 7f1c36c2ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1c241ab650 con 0x55ca91d2af60 2024-06-10T01:10:00.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.717+0000 7f1c36c2ab80 10 client.0 _ll_register_callbacks cb 0x55ca91c10920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:00.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.718+0000 7f1c29942700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca91d2a640 0x55ca91d4ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:00.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.718+0000 7f1c29942700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca91d2a640 0x55ca91d4ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36992/0 (socket says 172.21.15.37:36992) 2024-06-10T01:10:00.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.718+0000 7f1c2a944700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca91d2af60 0x55ca91d4b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:00.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.718+0000 7f1c29942700 1 -- 172.21.15.37:0/2942431170 learned_addr learned my addr 172.21.15.37:0/2942431170 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:00.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.718+0000 7f1c2a143700 1 --2- 172.21.15.37:0/2942431170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca91d24f20 0x55ca91d4a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:00.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.718+0000 7f1c29942700 1 -- 172.21.15.37:0/2942431170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca91d2af60 msgr2=0x55ca91d4b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:00.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.718+0000 7f1c29942700 1 --2- 172.21.15.37:0/2942431170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca91d2af60 0x55ca91d4b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.718+0000 7f1c29942700 1 -- 172.21.15.37:0/2942431170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca91d24f20 msgr2=0x55ca91d4a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:00.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.718+0000 7f1c29942700 1 --2- 172.21.15.37:0/2942431170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca91d24f20 0x55ca91d4a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.718+0000 7f1c29942700 1 -- 172.21.15.37:0/2942431170 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca91dac970 con 0x55ca91d2a640 2024-06-10T01:10:00.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.718+0000 7f1c2a143700 1 --2- 172.21.15.37:0/2942431170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca91d24f20 0x55ca91d4a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:00.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.718+0000 7f1c29942700 1 --2- 172.21.15.37:0/2942431170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca91d2a640 0x55ca91d4ab60 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f1c18011040 tx=0x7f1c180058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18df6af38484306 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:00.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.718+0000 7f1c0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:10:00.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.718+0000 7f1c0dffb700 1 -- 172.21.15.37:0/2942431170 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18029070 con 0x55ca91d2a640 2024-06-10T01:10:00.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.718+0000 7f1c0dffb700 1 -- 172.21.15.37:0/2942431170 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9618+0+0 (secure 0 0 0) 0x7f1c18003d70 con 0x55ca91d2a640 2024-06-10T01:10:00.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.718+0000 7f1c36c2ab80 1 -- 172.21.15.37:0/2942431170 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c18003740 con 0x55ca91d2a640 2024-06-10T01:10:00.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.719+0000 7f1c0dffb700 1 -- 172.21.15.37:0/2942431170 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1c18039040 con 0x55ca91d2a640 2024-06-10T01:10:00.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.719+0000 7f1c36c2ab80 10 client.17835 fetch_fsmap learned FSMap version 33 2024-06-10T01:10:00.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.719+0000 7f1c36c2ab80 1 -- 172.21.15.37:0/2942431170 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ca91dadc40 con 0x55ca91d2a640 2024-06-10T01:10:00.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.719+0000 7f1c0dffb700 1 -- 172.21.15.37:0/2942431170 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c18003740 con 0x55ca91d2a640 2024-06-10T01:10:00.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.719+0000 7f1c0dffb700 1 -- 172.21.15.37:0/2942431170 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f1c180246f0 con 0x55ca91d2a640 2024-06-10T01:10:00.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.719+0000 7f1c36c2ab80 10 client.17835 fetch_fsmap finished waiting for FSMap version 33 2024-06-10T01:10:00.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.719+0000 7f1c36c2ab80 10 client.17835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:10:00.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.719+0000 7f1c36c2ab80 20 client.17835 populate_metadata read hostname 'smithi037' 2024-06-10T01:10:00.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.719+0000 7f1c36c2ab80 1 --2- 172.21.15.37:0/2942431170 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55ca91dae220 0x55ca91db0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:00.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.719+0000 7f1c36c2ab80 4 client.17835 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:10:00.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.719+0000 7f1c36c2ab80 1 -- 172.21.15.37:0/2942431170 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ca91c7e290 con 0x55ca91dae220 2024-06-10T01:10:00.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.719+0000 7f1c2a143700 1 --2- 172.21.15.37:0/2942431170 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55ca91dae220 0x55ca91db0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:00.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.720+0000 7f1c2a143700 1 --2- 172.21.15.37:0/2942431170 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55ca91dae220 0x55ca91db0610 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80d8d0c1b181ac24 server_cookie=225019aa684a0fa2 in_seq=0 out_seq=0 2024-06-10T01:10:00.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.720+0000 7f1c0dffb700 10 client.17835 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:10:00.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.720+0000 7f1c0dffb700 1 -- 172.21.15.37:0/2942431170 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ca91c7e290 con 0x55ca91dae220 2024-06-10T01:10:00.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.720+0000 7f1c0dffb700 10 client.17835 handle_command_reply: tid=0 2024-06-10T01:10:00.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.720+0000 7f1c36c2ab80 1 client.17835 shutdown 2024-06-10T01:10:00.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.720+0000 7f1c36c2ab80 1 -- 172.21.15.37:0/2942431170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca91d2a640 msgr2=0x55ca91d4ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:00.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.720+0000 7f1c36c2ab80 1 --2- 172.21.15.37:0/2942431170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca91d2a640 0x55ca91d4ab60 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f1c18011040 tx=0x7f1c180058e0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.720+0000 7f1c36c2ab80 1 -- 172.21.15.37:0/2942431170 shutdown_connections 2024-06-10T01:10:00.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.720+0000 7f1c36c2ab80 1 --2- 172.21.15.37:0/2942431170 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55ca91dae220 0x55ca91db0610 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.720+0000 7f1c36c2ab80 1 --2- 172.21.15.37:0/2942431170 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ca91d2af60 0x55ca91d4b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.720+0000 7f1c36c2ab80 1 --2- 172.21.15.37:0/2942431170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ca91d24f20 0x55ca91d4a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.720+0000 7f1c36c2ab80 1 --2- 172.21.15.37:0/2942431170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca91d2a640 0x55ca91d4ab60 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.720+0000 7f1c36c2ab80 1 -- 172.21.15.37:0/2942431170 >> 172.21.15.37:0/2942431170 conn(0x55ca91c4fcc0 msgr2=0x55ca91d2b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:00.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.720+0000 7f1c0dffb700 0 client.17835 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:10:00.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.720+0000 7f1c36c2ab80 1 -- 172.21.15.37:0/2942431170 shutdown_connections 2024-06-10T01:10:00.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.720+0000 7f1c36c2ab80 1 -- 172.21.15.37:0/2942431170 wait complete. 2024-06-10T01:10:00.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.720+0000 7f1c36c2ab80 20 client.17835 trim_cache size 0 max 16384 2024-06-10T01:10:00.721 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:10:00.721 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-10T01:10:00.721 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-06-10T01:10:00.721 INFO:teuthology.orchestra.run.smithi037.stdout: "af6360d9-ca22-467c-a010-3496836b3e01": { 2024-06-10T01:10:00.721 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-06-10T01:10:00.721 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "af6360d9-ca22-467c-a010-3496836b3e01", 2024-06-10T01:10:00.721 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-06-10T01:10:00.721 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:00.722 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:00.722 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:10:00.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.721+0000 7f1c2c3a7700 1 -- 172.21.15.37:0/2293339852 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1c100a2c20 msgr2=0x7f1c100a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:00.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.721+0000 7f1c2c3a7700 1 --2- 172.21.15.37:0/2293339852 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1c100a2c20 0x7f1c100a50e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1c20005950 tx=0x7f1c2000b410 comp rx=0 tx=0).stop 2024-06-10T01:10:00.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.721+0000 7f1c2c3a7700 1 -- 172.21.15.37:0/2293339852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c241073f0 msgr2=0x7f1c241a4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:00.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.721+0000 7f1c2c3a7700 1 --2- 172.21.15.37:0/2293339852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c241073f0 0x7f1c241a4b60 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c007ee0 tx=0x7f1c1c00d3b0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.721+0000 7f1c2c3a7700 1 -- 172.21.15.37:0/2293339852 shutdown_connections 2024-06-10T01:10:00.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.721+0000 7f1c2c3a7700 1 --2- 172.21.15.37:0/2293339852 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f1c100a2c20 0x7f1c100a50e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.721+0000 7f1c2c3a7700 1 --2- 172.21.15.37:0/2293339852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c241073f0 0x7f1c241a4b60 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.721+0000 7f1c2c3a7700 1 --2- 172.21.15.37:0/2293339852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c24101380 0x7f1c24199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.721+0000 7f1c2c3a7700 1 --2- 172.21.15.37:0/2293339852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c24101d40 0x7f1c2419a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:00.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.721+0000 7f1c2c3a7700 1 -- 172.21.15.37:0/2293339852 >> 172.21.15.37:0/2293339852 conn(0x7f1c24078300 msgr2=0x7f1c240fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:00.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.721+0000 7f1c2c3a7700 1 -- 172.21.15.37:0/2293339852 shutdown_connections 2024-06-10T01:10:00.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:00.721+0000 7f1c2c3a7700 1 -- 172.21.15.37:0/2293339852 wait complete. 2024-06-10T01:10:00.758 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:00.762+0000 7f0ba91ee640 1 -- 172.21.15.37:0/2788632844 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.217144"} v 0) -- 0x7f0ba402c920 con 0x7f0ba4102c10 2024-06-10T01:10:00.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:00 smithi167 ceph-mon[36917]: osdmap e103: 12 total, 12 up, 12 in 2024-06-10T01:10:00.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:00 smithi167 ceph-mon[36917]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-10T01:10:00.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:00 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3223013298' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.217144"}]': finished 2024-06-10T01:10:00.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:00 smithi167 ceph-mon[36917]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:10:00.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:00 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3223013298' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.217144"}]: dispatch 2024-06-10T01:10:00.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:00 smithi167 ceph-mon[36917]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-06-10T01:10:00.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:00 smithi167 ceph-mon[36917]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-06-10T01:10:00.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:00 smithi167 ceph-mon[36917]: application not enabled on pool 'foo.217144' 2024-06-10T01:10:00.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:00 smithi167 ceph-mon[36917]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-06-10T01:10:00.835 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:00 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2788632844' entity='client.admin' cmd=[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.217144"}]: dispatch 2024-06-10T01:10:00.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:00 smithi094 ceph-mon[36708]: osdmap e103: 12 total, 12 up, 12 in 2024-06-10T01:10:00.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:00 smithi094 ceph-mon[36708]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-10T01:10:00.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:00 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3223013298' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.217144"}]': finished 2024-06-10T01:10:00.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:00 smithi094 ceph-mon[36708]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:10:00.860 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:00 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3223013298' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.217144"}]: dispatch 2024-06-10T01:10:00.860 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:00 smithi094 ceph-mon[36708]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-06-10T01:10:00.860 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:00 smithi094 ceph-mon[36708]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-06-10T01:10:00.860 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:00 smithi094 ceph-mon[36708]: application not enabled on pool 'foo.217144' 2024-06-10T01:10:00.860 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:00 smithi094 ceph-mon[36708]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-06-10T01:10:00.860 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:00 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2788632844' entity='client.admin' cmd=[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.217144"}]: dispatch 2024-06-10T01:10:01.536 INFO:tasks.cephfs.filesystem:scrub status for tag:af6360d9-ca22-467c-a010-3496836b3e01 - {'path': '/', 'tag': 'af6360d9-ca22-467c-a010-3496836b3e01', 'options': 'recursive,force'} 2024-06-10T01:10:01.583 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.588+0000 7f0b8bfff640 1 -- 172.21.15.37:0/2788632844 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.217144"}]=0 v34) ==== 106+0+0 (secure 0 0 0) 0x7f0b9809f7b0 con 0x7f0ba4102c10 2024-06-10T01:10:01.586 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.591+0000 7f0ba91ee640 1 -- 172.21.15.37:0/2788632844 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0b8c0a2b90 msgr2=0x7f0b8c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:01.587 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.591+0000 7f0ba91ee640 1 --2- 172.21.15.37:0/2788632844 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0b8c0a2b90 0x7f0b8c0a5050 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4198b50 tx=0x7f0b9400b390 comp rx=0 tx=0).stop 2024-06-10T01:10:01.587 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.591+0000 7f0ba91ee640 1 -- 172.21.15.37:0/2788632844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ba4102c10 msgr2=0x7f0ba41981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:01.587 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.591+0000 7f0ba91ee640 1 --2- 172.21.15.37:0/2788632844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ba4102c10 0x7f0ba41981c0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f0b9800d950 tx=0x7f0b9800de10 comp rx=0 tx=0).stop 2024-06-10T01:10:01.587 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.591+0000 7f0ba91ee640 1 -- 172.21.15.37:0/2788632844 shutdown_connections 2024-06-10T01:10:01.587 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.591+0000 7f0ba91ee640 1 --2- 172.21.15.37:0/2788632844 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f0b8c0a2b90 0x7f0b8c0a5050 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.587 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.591+0000 7f0ba91ee640 1 --2- 172.21.15.37:0/2788632844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ba41035d0 0x7f0ba419f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.587 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.591+0000 7f0ba91ee640 1 --2- 172.21.15.37:0/2788632844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ba4102c10 0x7f0ba41981c0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.587 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.591+0000 7f0ba91ee640 1 --2- 172.21.15.37:0/2788632844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ba4101a20 0x7f0ba4197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.587 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.591+0000 7f0ba91ee640 1 -- 172.21.15.37:0/2788632844 >> 172.21.15.37:0/2788632844 conn(0x7f0ba40fcf90 msgr2=0x7f0ba4105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:01.587 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.591+0000 7f0ba91ee640 1 -- 172.21.15.37:0/2788632844 shutdown_connections 2024-06-10T01:10:01.587 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.591+0000 7f0ba91ee640 1 -- 172.21.15.37:0/2788632844 wait complete. 2024-06-10T01:10:01.692 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.694+0000 7f0306837640 1 -- 172.21.15.37:0/3853554738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f030010bdc0 msgr2=0x7f0300112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:01.692 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.694+0000 7f0306837640 1 --2- 172.21.15.37:0/3853554738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f030010bdc0 0x7f0300112650 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f02f8009ca0 tx=0x7f02f802a8c0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.692 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.695+0000 7f0306837640 1 -- 172.21.15.37:0/3853554738 shutdown_connections 2024-06-10T01:10:01.692 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.695+0000 7f0306837640 1 --2- 172.21.15.37:0/3853554738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f030010bdc0 0x7f0300112650 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.692 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.695+0000 7f0306837640 1 --2- 172.21.15.37:0/3853554738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f030010b400 0x7f030010b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.692 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.695+0000 7f0306837640 1 --2- 172.21.15.37:0/3853554738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f030010a210 0x7f030010a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.692 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.695+0000 7f0306837640 1 -- 172.21.15.37:0/3853554738 >> 172.21.15.37:0/3853554738 conn(0x7f03000709b0 msgr2=0x7f0300072df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:01.693 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.695+0000 7f0306837640 1 -- 172.21.15.37:0/3853554738 shutdown_connections 2024-06-10T01:10:01.693 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.695+0000 7f0306837640 1 -- 172.21.15.37:0/3853554738 wait complete. 2024-06-10T01:10:01.693 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.695+0000 7f0306837640 1 Processor -- start 2024-06-10T01:10:01.693 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.695+0000 7f0306837640 1 -- start start 2024-06-10T01:10:01.693 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.695+0000 7f0306837640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f030010a210 0x7f03001a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.693 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.695+0000 7f0306837640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f030010b400 0x7f03001a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.693 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.695+0000 7f0306837640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f030010bdc0 0x7f03001a7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.693 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.695+0000 7f0306837640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0300115ea0 con 0x7f030010bdc0 2024-06-10T01:10:01.693 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.695+0000 7f0306837640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0300115d20 con 0x7f030010a210 2024-06-10T01:10:01.693 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.696+0000 7f0306837640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0300116020 con 0x7f030010b400 2024-06-10T01:10:01.693 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.696+0000 7f0304dd4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f030010bdc0 0x7f03001a7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.693 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.696+0000 7f0304dd4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f030010bdc0 0x7f03001a7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37010/0 (socket says 172.21.15.37:37010) 2024-06-10T01:10:01.694 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.696+0000 7f0304dd4640 1 -- 172.21.15.37:0/3690875121 learned_addr learned my addr 172.21.15.37:0/3690875121 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:01.694 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.696+0000 7f02fffff640 1 --2- 172.21.15.37:0/3690875121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f030010a210 0x7f03001a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.694 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.696+0000 7f02ff7fe640 1 --2- 172.21.15.37:0/3690875121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f030010b400 0x7f03001a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.694 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.696+0000 7f0304dd4640 1 -- 172.21.15.37:0/3690875121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f030010b400 msgr2=0x7f03001a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:01.694 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.696+0000 7f0304dd4640 1 --2- 172.21.15.37:0/3690875121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f030010b400 0x7f03001a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.694 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.696+0000 7f0304dd4640 1 -- 172.21.15.37:0/3690875121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f030010a210 msgr2=0x7f03001a04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:01.694 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.696+0000 7f0304dd4640 1 --2- 172.21.15.37:0/3690875121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f030010a210 0x7f03001a04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:01.694 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.696+0000 7f0304dd4640 1 -- 172.21.15.37:0/3690875121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03001a80d0 con 0x7f030010bdc0 2024-06-10T01:10:01.694 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.696+0000 7f02ff7fe640 1 --2- 172.21.15.37:0/3690875121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f030010b400 0x7f03001a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:10:01.694 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.696+0000 7f0304dd4640 1 --2- 172.21.15.37:0/3690875121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f030010bdc0 0x7f03001a7a60 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f02f8031ae0 tx=0x7f02f8031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:01.694 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.696+0000 7f02fd7fa640 1 -- 172.21.15.37:0/3690875121 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f8003c30 con 0x7f030010bdc0 2024-06-10T01:10:01.694 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.696+0000 7f0306837640 1 -- 172.21.15.37:0/3690875121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03001a8360 con 0x7f030010bdc0 2024-06-10T01:10:01.694 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.696+0000 7f02fd7fa640 1 -- 172.21.15.37:0/3690875121 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02f8003dd0 con 0x7f030010bdc0 2024-06-10T01:10:01.695 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.696+0000 7f02fd7fa640 1 -- 172.21.15.37:0/3690875121 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f802fcb0 con 0x7f030010bdc0 2024-06-10T01:10:01.695 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.696+0000 7f0306837640 1 -- 172.21.15.37:0/3690875121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03001a8900 con 0x7f030010bdc0 2024-06-10T01:10:01.696 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.698+0000 7f0306837640 1 -- 172.21.15.37:0/3690875121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0300111460 con 0x7f030010bdc0 2024-06-10T01:10:01.696 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.698+0000 7f02fd7fa640 1 -- 172.21.15.37:0/3690875121 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f02f8004050 con 0x7f030010bdc0 2024-06-10T01:10:01.696 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.699+0000 7f02fd7fa640 1 --2- 172.21.15.37:0/3690875121 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f02e80a2b90 0x7f02e80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:01.696 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.699+0000 7f02fffff640 1 --2- 172.21.15.37:0/3690875121 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f02e80a2b90 0x7f02e80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:01.696 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.699+0000 7f02fd7fa640 1 -- 172.21.15.37:0/3690875121 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9618+0+0 (secure 0 0 0) 0x7f02f80f0910 con 0x7f030010bdc0 2024-06-10T01:10:01.696 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.700+0000 7f02fffff640 1 --2- 172.21.15.37:0/3690875121 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f02e80a2b90 0x7f02e80a5050 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f030010b260 tx=0x7f02f000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:01.699 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.704+0000 7f02fd7fa640 1 -- 172.21.15.37:0/3690875121 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f02f80bd3c0 con 0x7f030010bdc0 2024-06-10T01:10:01.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:01 smithi167 ceph-mon[36917]: osdmap e104: 12 total, 12 up, 12 in 2024-06-10T01:10:01.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:01 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2788632844' entity='client.admin' cmd='[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.217144"}]': finished 2024-06-10T01:10:01.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:01 smithi167 ceph-mon[36917]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:10:01.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:01 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2788632844' entity='client.admin' cmd=[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.217144"}]: dispatch 2024-06-10T01:10:01.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:01 smithi167 ceph-mon[36917]: pgmap v834: 137 pgs: 4 unknown, 133 active+clean; 2.8 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 KiB/s wr, 15 op/s 2024-06-10T01:10:01.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:01 smithi094 ceph-mon[36708]: osdmap e104: 12 total, 12 up, 12 in 2024-06-10T01:10:01.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:01 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2788632844' entity='client.admin' cmd='[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.217144"}]': finished 2024-06-10T01:10:01.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:01 smithi094 ceph-mon[36708]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:10:01.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:01 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2788632844' entity='client.admin' cmd=[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.217144"}]: dispatch 2024-06-10T01:10:01.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:01 smithi094 ceph-mon[36708]: pgmap v834: 137 pgs: 4 unknown, 133 active+clean; 2.8 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 KiB/s wr, 15 op/s 2024-06-10T01:10:01.885 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:01.890+0000 7f0306837640 1 -- 172.21.15.37:0/3690875121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "foo.217144", "pool2": "foo.217144", "yes_i_really_really_mean_it": true} v 0) -- 0x7f030002d2e0 con 0x7f030010bdc0 2024-06-10T01:10:01.890 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:01 smithi037 ceph-mon[32192]: osdmap e104: 12 total, 12 up, 12 in 2024-06-10T01:10:01.890 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:01 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2788632844' entity='client.admin' cmd='[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.217144"}]': finished 2024-06-10T01:10:01.891 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:01 smithi037 ceph-mon[32192]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:10:01.891 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:01 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2788632844' entity='client.admin' cmd=[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.217144"}]: dispatch 2024-06-10T01:10:01.891 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:01 smithi037 ceph-mon[32192]: pgmap v834: 137 pgs: 4 unknown, 133 active+clean; 2.8 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 34 KiB/s wr, 15 op/s 2024-06-10T01:10:02.585 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:02.590+0000 7f02fd7fa640 1 -- 172.21.15.37:0/3690875121 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "foo.217144", "pool2": "foo.217144", "yes_i_really_really_mean_it": true}]=0 pool 'foo.217144' removed v105) ==== 166+0+0 (secure 0 0 0) 0x7f02f80bccc0 con 0x7f030010bdc0 2024-06-10T01:10:02.703 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:02.708+0000 7f0306837640 1 -- 172.21.15.37:0/3690875121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "foo.217144", "pool2": "foo.217144", "yes_i_really_really_mean_it": true} v 0) -- 0x7f030002c920 con 0x7f030010bdc0 2024-06-10T01:10:02.704 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:02.708+0000 7f02fd7fa640 1 -- 172.21.15.37:0/3690875121 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "foo.217144", "pool2": "foo.217144", "yes_i_really_really_mean_it": true}]=0 pool 'foo.217144' does not exist v105) ==== 173+0+0 (secure 0 0 0) 0x7f02f80c1070 con 0x7f030010bdc0 2024-06-10T01:10:02.704 INFO:tasks.workunit.client.0.smithi037.stderr:pool 'foo.217144' does not exist 2024-06-10T01:10:02.706 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:02.711+0000 7f0306837640 1 -- 172.21.15.37:0/3690875121 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f02e80a2b90 msgr2=0x7f02e80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:02.706 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:02.711+0000 7f0306837640 1 --2- 172.21.15.37:0/3690875121 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f02e80a2b90 0x7f02e80a5050 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f030010b260 tx=0x7f02f000b390 comp rx=0 tx=0).stop 2024-06-10T01:10:02.706 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:02.711+0000 7f0306837640 1 -- 172.21.15.37:0/3690875121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f030010bdc0 msgr2=0x7f03001a7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:02.706 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:02.711+0000 7f0306837640 1 --2- 172.21.15.37:0/3690875121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f030010bdc0 0x7f03001a7a60 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f02f8031ae0 tx=0x7f02f8031ee0 comp rx=0 tx=0).stop 2024-06-10T01:10:02.706 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:02.711+0000 7f0306837640 1 -- 172.21.15.37:0/3690875121 shutdown_connections 2024-06-10T01:10:02.707 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:02.711+0000 7f0306837640 1 --2- 172.21.15.37:0/3690875121 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f02e80a2b90 0x7f02e80a5050 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:02.707 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:02.711+0000 7f0306837640 1 --2- 172.21.15.37:0/3690875121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f030010bdc0 0x7f03001a7a60 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:02.707 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:02.711+0000 7f0306837640 1 --2- 172.21.15.37:0/3690875121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f030010b400 0x7f03001a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:02.707 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:02.711+0000 7f0306837640 1 --2- 172.21.15.37:0/3690875121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f030010a210 0x7f03001a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:02.707 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:02.711+0000 7f0306837640 1 -- 172.21.15.37:0/3690875121 >> 172.21.15.37:0/3690875121 conn(0x7f03000709b0 msgr2=0x7f030010e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:02.707 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:02.711+0000 7f0306837640 1 -- 172.21.15.37:0/3690875121 shutdown_connections 2024-06-10T01:10:02.707 INFO:tasks.workunit.client.0.smithi037.stderr:2024-06-10T01:10:02.711+0000 7f0306837640 1 -- 172.21.15.37:0/3690875121 wait complete. 2024-06-10T01:10:02.717 INFO:tasks.workunit.client.0.smithi037.stdout:OK 2024-06-10T01:10:02.717 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-10T01:10:02.717 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-10T01:10:02.753 INFO:tasks.workunit:Running workunit fs/misc/multiple_rsync.sh... 2024-06-10T01:10:02.753 DEBUG:teuthology.orchestra.run.smithi037:workunit test fs/misc/multiple_rsync.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b7a5d375cf4e4afab96b9d4d229b699ae8304bb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/multiple_rsync.sh 2024-06-10T01:10:02.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:02 smithi167 ceph-mon[36917]: scrub summary: idle 2024-06-10T01:10:02.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:02 smithi167 ceph-mon[36917]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-10T01:10:02.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:02 smithi167 ceph-mon[36917]: Cluster is now healthy 2024-06-10T01:10:02.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:02 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2788632844' entity='client.admin' cmd='[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.217144"}]': finished 2024-06-10T01:10:02.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:02 smithi167 ceph-mon[36917]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:10:02.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:02 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3690875121' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "foo.217144", "pool2": "foo.217144", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T01:10:02.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:02 smithi094 ceph-mon[36708]: scrub summary: idle 2024-06-10T01:10:02.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:02 smithi094 ceph-mon[36708]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-10T01:10:02.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:02 smithi094 ceph-mon[36708]: Cluster is now healthy 2024-06-10T01:10:02.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:02 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2788632844' entity='client.admin' cmd='[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.217144"}]': finished 2024-06-10T01:10:02.860 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:02 smithi094 ceph-mon[36708]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:10:02.860 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:02 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3690875121' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "foo.217144", "pool2": "foo.217144", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T01:10:02.890 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:02 smithi037 ceph-mon[32192]: scrub summary: idle 2024-06-10T01:10:02.890 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:02 smithi037 ceph-mon[32192]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-10T01:10:02.891 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:02 smithi037 ceph-mon[32192]: Cluster is now healthy 2024-06-10T01:10:02.891 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:02 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2788632844' entity='client.admin' cmd='[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.217144"}]': finished 2024-06-10T01:10:02.891 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:02 smithi037 ceph-mon[32192]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:10:02.891 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:02 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3690875121' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "foo.217144", "pool2": "foo.217144", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T01:10:03.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:03 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3690875121' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "foo.217144", "pool2": "foo.217144", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T01:10:03.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:03 smithi094 ceph-mon[36708]: osdmap e105: 12 total, 12 up, 12 in 2024-06-10T01:10:03.859 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:03 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3690875121' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "foo.217144", "pool2": "foo.217144", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T01:10:03.860 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:03 smithi094 ceph-mon[36708]: pgmap v836: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-06-10T01:10:03.890 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:03 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3690875121' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "foo.217144", "pool2": "foo.217144", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T01:10:03.891 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:03 smithi037 ceph-mon[32192]: osdmap e105: 12 total, 12 up, 12 in 2024-06-10T01:10:03.891 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:03 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3690875121' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "foo.217144", "pool2": "foo.217144", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T01:10:03.891 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:03 smithi037 ceph-mon[32192]: pgmap v836: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-06-10T01:10:04.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:03 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3690875121' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "foo.217144", "pool2": "foo.217144", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T01:10:04.084 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:03 smithi167 ceph-mon[36917]: osdmap e105: 12 total, 12 up, 12 in 2024-06-10T01:10:04.084 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:03 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3690875121' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "foo.217144", "pool2": "foo.217144", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T01:10:04.084 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:03 smithi167 ceph-mon[36917]: pgmap v836: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-06-10T01:10:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.450+0000 7f04e2e9a640 1 -- 172.21.15.37:0/450989516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04dc101af0 msgr2=0x7f04dc101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.450+0000 7f04e2e9a640 1 --2- 172.21.15.37:0/450989516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04dc101af0 0x7f04dc101f70 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009ca0 tx=0x7f04cc02a8c0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.450+0000 7f04e2e9a640 1 -- 172.21.15.37:0/450989516 shutdown_connections 2024-06-10T01:10:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.450+0000 7f04e2e9a640 1 --2- 172.21.15.37:0/450989516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04dc1024b0 0x7f04dc108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.450+0000 7f04e2e9a640 1 --2- 172.21.15.37:0/450989516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04dc101af0 0x7f04dc101f70 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.450+0000 7f04e2e9a640 1 --2- 172.21.15.37:0/450989516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04dc100900 0x7f04dc100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.450+0000 7f04e2e9a640 1 -- 172.21.15.37:0/450989516 >> 172.21.15.37:0/450989516 conn(0x7f04dc0fbe70 msgr2=0x7f04dc0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.450+0000 7f04e2e9a640 1 -- 172.21.15.37:0/450989516 shutdown_connections 2024-06-10T01:10:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.450+0000 7f04e2e9a640 1 -- 172.21.15.37:0/450989516 wait complete. 2024-06-10T01:10:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.451+0000 7f04e2e9a640 1 Processor -- start 2024-06-10T01:10:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.451+0000 7f04e2e9a640 1 -- start start 2024-06-10T01:10:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.451+0000 7f04e2e9a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04dc100900 0x7f04dc197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.451+0000 7f04e2e9a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04dc101af0 0x7f04dc1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.451+0000 7f04e2e9a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04dc1024b0 0x7f04dc19f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.451+0000 7f04e2e9a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc10c580 con 0x7f04dc101af0 2024-06-10T01:10:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.451+0000 7f04e2e9a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc10c400 con 0x7f04dc1024b0 2024-06-10T01:10:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.451+0000 7f04e2e9a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc10c700 con 0x7f04dc100900 2024-06-10T01:10:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.451+0000 7f04dbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04dc101af0 0x7f04dc1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.451+0000 7f04dbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04dc101af0 0x7f04dc1981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52224/0 (socket says 172.21.15.37:52224) 2024-06-10T01:10:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.451+0000 7f04dbfff640 1 -- 172.21.15.37:0/2597772886 learned_addr learned my addr 172.21.15.37:0/2597772886 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.451+0000 7f04e0c36640 1 --2- 172.21.15.37:0/2597772886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04dc100900 0x7f04dc197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.451+0000 7f04e1437640 1 --2- 172.21.15.37:0/2597772886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04dc1024b0 0x7f04dc19f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.452+0000 7f04dbfff640 1 -- 172.21.15.37:0/2597772886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04dc100900 msgr2=0x7f04dc197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.452+0000 7f04dbfff640 1 --2- 172.21.15.37:0/2597772886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04dc100900 0x7f04dc197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.452+0000 7f04dbfff640 1 -- 172.21.15.37:0/2597772886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04dc1024b0 msgr2=0x7f04dc19f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.452+0000 7f04dbfff640 1 --2- 172.21.15.37:0/2597772886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04dc1024b0 0x7f04dc19f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.452+0000 7f04dbfff640 1 -- 172.21.15.37:0/2597772886 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04dc19f890 con 0x7f04dc101af0 2024-06-10T01:10:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.452+0000 7f04dbfff640 1 --2- 172.21.15.37:0/2597772886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04dc101af0 0x7f04dc1981c0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f04cc000f80 tx=0x7f04cc031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.452+0000 7f04d9ffb640 1 -- 172.21.15.37:0/2597772886 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc02fa20 con 0x7f04dc101af0 2024-06-10T01:10:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.452+0000 7f04d9ffb640 1 -- 172.21.15.37:0/2597772886 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04cc02fbc0 con 0x7f04dc101af0 2024-06-10T01:10:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.452+0000 7f04e2e9a640 1 -- 172.21.15.37:0/2597772886 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04dc19fb20 con 0x7f04dc101af0 2024-06-10T01:10:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.452+0000 7f04d9ffb640 1 -- 172.21.15.37:0/2597772886 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc03a6f0 con 0x7f04dc101af0 2024-06-10T01:10:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.452+0000 7f04e2e9a640 1 -- 172.21.15.37:0/2597772886 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04dc1a00c0 con 0x7f04dc101af0 2024-06-10T01:10:04.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.455+0000 7f04d9ffb640 1 -- 172.21.15.37:0/2597772886 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f04cc03a890 con 0x7f04dc101af0 2024-06-10T01:10:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.455+0000 7f04e5ba2740 1 Processor -- start 2024-06-10T01:10:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.455+0000 7f04e5ba2740 1 -- start start 2024-06-10T01:10:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.455+0000 7f04e5ba2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c983e77010 0x55c983e77430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.455+0000 7f04e5ba2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c983e77a00 0x55c983e7c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.455+0000 7f04e5ba2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c983e7cb70 0x55c983e7f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.455+0000 7f04e5ba2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c983dda4f0 con 0x55c983e77a00 2024-06-10T01:10:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.455+0000 7f04e5ba2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c983d78870 con 0x55c983e7cb70 2024-06-10T01:10:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.455+0000 7f04e5ba2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c983dd64b0 con 0x55c983e77010 2024-06-10T01:10:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.456+0000 7f04d9ffb640 1 --2- 172.21.15.37:0/2597772886 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f04c80a2cb0 0x7f04c80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.456+0000 7f04e0c36640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c983e77a00 0x55c983e7c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.456+0000 7f04e0c36640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c983e77a00 0x55c983e7c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52230/0 (socket says 172.21.15.37:52230) 2024-06-10T01:10:04.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.456+0000 7f04e0c36640 1 -- 172.21.15.37:0/1210423271 learned_addr learned my addr 172.21.15.37:0/1210423271 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.456+0000 7f04e1437640 1 --2- 172.21.15.37:0/1210423271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c983e77010 0x55c983e77430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.456+0000 7f04e1437640 1 --2- 172.21.15.37:0/2597772886 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f04c80a2cb0 0x7f04c80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.456+0000 7f04dbfff640 1 --2- 172.21.15.37:0/1210423271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c983e7cb70 0x55c983e7f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.456+0000 7f04d9ffb640 1 -- 172.21.15.37:0/2597772886 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7f04cc0f0900 con 0x7f04dc101af0 2024-06-10T01:10:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.456+0000 7f04e1437640 1 -- 172.21.15.37:0/1210423271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c983e7cb70 msgr2=0x55c983e7f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.456+0000 7f04e1437640 1 --2- 172.21.15.37:0/1210423271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c983e7cb70 0x55c983e7f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.456+0000 7f04e1437640 1 -- 172.21.15.37:0/1210423271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c983e77a00 msgr2=0x55c983e7c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.456+0000 7f04e1437640 1 --2- 172.21.15.37:0/1210423271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c983e77a00 0x55c983e7c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.456+0000 7f04e1437640 1 -- 172.21.15.37:0/1210423271 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c983d949b0 con 0x55c983e77010 2024-06-10T01:10:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.456+0000 7f04e0c36640 1 --2- 172.21.15.37:0/1210423271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c983e77a00 0x55c983e7c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.456+0000 7f04e1437640 1 --2- 172.21.15.37:0/1210423271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c983e77010 0x55c983e77430 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f04d40119b0 tx=0x7f04d4011e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7d5cbc60b170996 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.456+0000 7f04e1437640 1 --2- 172.21.15.37:0/2597772886 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f04c80a2cb0 0x7f04c80a5170 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f04d4009710 tx=0x7f04d401e000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:04.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.456+0000 7f04d2ffd640 1 -- 172.21.15.37:0/1210423271 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4004250 con 0x55c983e77010 2024-06-10T01:10:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.457+0000 7f04d2ffd640 1 -- 172.21.15.37:0/1210423271 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d4011560 con 0x55c983e77010 2024-06-10T01:10:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.457+0000 7f04d2ffd640 1 -- 172.21.15.37:0/1210423271 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4004c60 con 0x55c983e77010 2024-06-10T01:10:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.457+0000 7f04e5ba2740 1 -- 172.21.15.37:0/1210423271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c983e77010 msgr2=0x55c983e77430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.457+0000 7f04e5ba2740 1 --2- 172.21.15.37:0/1210423271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c983e77010 0x55c983e77430 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f04d40119b0 tx=0x7f04d4011e70 comp rx=0 tx=0).stop 2024-06-10T01:10:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.457+0000 7f04e5ba2740 1 -- 172.21.15.37:0/1210423271 shutdown_connections 2024-06-10T01:10:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.457+0000 7f04e5ba2740 1 --2- 172.21.15.37:0/1210423271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c983e7cb70 0x55c983e7f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.457+0000 7f04e5ba2740 1 --2- 172.21.15.37:0/1210423271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c983e77a00 0x55c983e7c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.457+0000 7f04e5ba2740 1 --2- 172.21.15.37:0/1210423271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c983e77010 0x55c983e77430 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.457+0000 7f04e5ba2740 1 -- 172.21.15.37:0/1210423271 >> 172.21.15.37:0/1210423271 conn(0x55c983e72680 msgr2=0x55c983e72a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.457+0000 7f04e5ba2740 1 -- 172.21.15.37:0/1210423271 shutdown_connections 2024-06-10T01:10:04.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.457+0000 7f04e5ba2740 1 -- 172.21.15.37:0/1210423271 wait complete. 2024-06-10T01:10:04.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.458+0000 7f04e5ba2740 1 -- start start 2024-06-10T01:10:04.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.458+0000 7f04e5ba2740 1 Processor -- start 2024-06-10T01:10:04.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.458+0000 7f04e5ba2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c983e77010 0x55c983e812c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:04.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.458+0000 7f04e5ba2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c983e77a00 0x55c983e81800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:04.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.458+0000 7f04e5ba2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c983e7cb70 0x55c983e88840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:04.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.458+0000 7f04e5ba2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c983e7fac0 con 0x55c983e7cb70 2024-06-10T01:10:04.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.458+0000 7f04e5ba2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55c983d63ae0 con 0x55c983e77a00 2024-06-10T01:10:04.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.458+0000 7f04e5ba2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c983e7fc40 con 0x55c983e77010 2024-06-10T01:10:04.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.458+0000 7f04e5ba2740 10 client.0 _ll_register_callbacks cb 0x55c983d9c9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.458+0000 7f04e1437640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c983e7cb70 0x55c983e88840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.458+0000 7f04e1437640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c983e7cb70 0x55c983e88840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52242/0 (socket says 172.21.15.37:52242) 2024-06-10T01:10:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.458+0000 7f04e1437640 1 -- 172.21.15.37:0/2882440999 learned_addr learned my addr 172.21.15.37:0/2882440999 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.459+0000 7f04e0c36640 1 --2- 172.21.15.37:0/2882440999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c983e77010 0x55c983e812c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.459+0000 7f04dbfff640 1 --2- 172.21.15.37:0/2882440999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c983e77a00 0x55c983e81800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.459+0000 7f04e1437640 1 -- 172.21.15.37:0/2882440999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c983e77010 msgr2=0x55c983e812c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.459+0000 7f04e1437640 1 --2- 172.21.15.37:0/2882440999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c983e77010 0x55c983e812c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.459+0000 7f04e1437640 1 -- 172.21.15.37:0/2882440999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c983e77a00 msgr2=0x55c983e81800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.459+0000 7f04e1437640 1 --2- 172.21.15.37:0/2882440999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c983e77a00 0x55c983e81800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.459+0000 7f04e1437640 1 -- 172.21.15.37:0/2882440999 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c983aee210 con 0x55c983e7cb70 2024-06-10T01:10:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.459+0000 7f04dbfff640 1 --2- 172.21.15.37:0/2882440999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c983e77a00 0x55c983e81800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:10:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.459+0000 7f04e0c36640 1 --2- 172.21.15.37:0/2882440999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c983e77010 0x55c983e812c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:10:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.459+0000 7f04e1437640 1 --2- 172.21.15.37:0/2882440999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c983e7cb70 0x55c983e88840 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f04d400ff90 tx=0x7f04d4004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7101cfec17ed9a61 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:04.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.459+0000 7f04bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:10:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.459+0000 7f04bbfff640 1 -- 172.21.15.37:0/2882440999 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d401d730 con 0x55c983e7cb70 2024-06-10T01:10:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.459+0000 7f04bbfff640 1 -- 172.21.15.37:0/2882440999 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7f04d401d8d0 con 0x55c983e7cb70 2024-06-10T01:10:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.459+0000 7f04e5ba2740 1 -- 172.21.15.37:0/2882440999 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c983dedf10 con 0x55c983e7cb70 2024-06-10T01:10:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.459+0000 7f04bbfff640 1 -- 172.21.15.37:0/2882440999 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f04d403b050 con 0x55c983e7cb70 2024-06-10T01:10:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.459+0000 7f04e5ba2740 10 client.17853 fetch_fsmap learned FSMap version 34 2024-06-10T01:10:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.459+0000 7f04e5ba2740 1 -- 172.21.15.37:0/2882440999 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c983bba620 con 0x55c983e7cb70 2024-06-10T01:10:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.460+0000 7f04bbfff640 1 -- 172.21.15.37:0/2882440999 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c983dedf10 con 0x55c983e7cb70 2024-06-10T01:10:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.460+0000 7f04bbfff640 1 -- 172.21.15.37:0/2882440999 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f04d4026450 con 0x55c983e7cb70 2024-06-10T01:10:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.460+0000 7f04e5ba2740 10 client.17853 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T01:10:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.460+0000 7f04e5ba2740 10 client.17853 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:10:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.460+0000 7f04e5ba2740 20 client.17853 populate_metadata read hostname 'smithi037' 2024-06-10T01:10:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.460+0000 7f04e5ba2740 1 --2- 172.21.15.37:0/2882440999 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c983e8cfd0 0x55c983e8f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.460+0000 7f04e5ba2740 4 client.17853 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:10:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.460+0000 7f04e5ba2740 1 -- 172.21.15.37:0/2882440999 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c983de7c20 con 0x55c983e8cfd0 2024-06-10T01:10:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.460+0000 7f04e0c36640 1 --2- 172.21.15.37:0/2882440999 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c983e8cfd0 0x55c983e8f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.461+0000 7f04bbfff640 10 client.17853 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:10:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.461+0000 7f04e0c36640 1 --2- 172.21.15.37:0/2882440999 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c983e8cfd0 0x55c983e8f3c0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9198a6f533c81e9 server_cookie=ba4ef60842bc4441 in_seq=0 out_seq=0 2024-06-10T01:10:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.461+0000 7f04bbfff640 1 -- 172.21.15.37:0/2882440999 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c983de7c20 con 0x55c983e8cfd0 2024-06-10T01:10:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.461+0000 7f04bbfff640 10 client.17853 handle_command_reply: tid=0 2024-06-10T01:10:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.461+0000 7f04e5ba2740 1 client.17853 shutdown 2024-06-10T01:10:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.461+0000 7f04e5ba2740 1 -- 172.21.15.37:0/2882440999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c983e7cb70 msgr2=0x55c983e88840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.461+0000 7f04e5ba2740 1 --2- 172.21.15.37:0/2882440999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c983e7cb70 0x55c983e88840 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f04d400ff90 tx=0x7f04d4004550 comp rx=0 tx=0).stop 2024-06-10T01:10:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.462+0000 7f04e5ba2740 1 -- 172.21.15.37:0/2882440999 shutdown_connections 2024-06-10T01:10:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.462+0000 7f04e5ba2740 1 --2- 172.21.15.37:0/2882440999 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c983e8cfd0 0x55c983e8f3c0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.462+0000 7f04e5ba2740 1 --2- 172.21.15.37:0/2882440999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c983e7cb70 0x55c983e88840 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.462+0000 7f04e5ba2740 1 --2- 172.21.15.37:0/2882440999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c983e77a00 0x55c983e81800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.462+0000 7f04e5ba2740 1 --2- 172.21.15.37:0/2882440999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c983e77010 0x55c983e812c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.462+0000 7f04e5ba2740 1 -- 172.21.15.37:0/2882440999 >> 172.21.15.37:0/2882440999 conn(0x55c983e72680 msgr2=0x55c983e717e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.462+0000 7f04bbfff640 0 client.17853 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:10:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.462+0000 7f04e5ba2740 1 -- 172.21.15.37:0/2882440999 shutdown_connections 2024-06-10T01:10:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.462+0000 7f04e5ba2740 1 -- 172.21.15.37:0/2882440999 wait complete. 2024-06-10T01:10:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.462+0000 7f04e5ba2740 20 client.17853 trim_cache size 0 max 16384 2024-06-10T01:10:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.477+0000 7f04e5ba2740 1 Processor -- start 2024-06-10T01:10:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.477+0000 7f04e5ba2740 1 -- start start 2024-06-10T01:10:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.477+0000 7f04e5ba2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c983e77010 0x55c983e812c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.477+0000 7f04e5ba2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c983e77a00 0x55c983e81800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.478+0000 7f04e5ba2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c983e7cb70 0x55c983e88800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.478+0000 7f04e5ba2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc1acf60 con 0x55c983e77a00 2024-06-10T01:10:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.478+0000 7f04e5ba2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc1aa830 con 0x55c983e7cb70 2024-06-10T01:10:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.478+0000 7f04e5ba2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc1a8680 con 0x55c983e77010 2024-06-10T01:10:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.478+0000 7f04dbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c983e77a00 0x55c983e81800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.478+0000 7f04dbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c983e77a00 0x55c983e81800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52248/0 (socket says 172.21.15.37:52248) 2024-06-10T01:10:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.478+0000 7f04dbfff640 1 -- 172.21.15.37:0/2676533017 learned_addr learned my addr 172.21.15.37:0/2676533017 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.478+0000 7f04e0c36640 1 --2- 172.21.15.37:0/2676533017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c983e77010 0x55c983e812c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.478+0000 7f04dbfff640 1 -- 172.21.15.37:0/2676533017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c983e77010 msgr2=0x55c983e812c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.478+0000 7f04e1437640 1 --2- 172.21.15.37:0/2676533017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c983e7cb70 0x55c983e88800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.478+0000 7f04dbfff640 1 --2- 172.21.15.37:0/2676533017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c983e77010 0x55c983e812c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.478+0000 7f04dbfff640 1 -- 172.21.15.37:0/2676533017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c983e7cb70 msgr2=0x55c983e88800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.478+0000 7f04dbfff640 1 --2- 172.21.15.37:0/2676533017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c983e7cb70 0x55c983e88800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.478+0000 7f04dbfff640 1 -- 172.21.15.37:0/2676533017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c983e73350 con 0x55c983e77a00 2024-06-10T01:10:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.478+0000 7f04e1437640 1 --2- 172.21.15.37:0/2676533017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c983e7cb70 0x55c983e88800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:10:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.478+0000 7f04dbfff640 1 --2- 172.21.15.37:0/2676533017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c983e77a00 0x55c983e81800 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f04cc0bca70 tx=0x7f04cc041c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0d5856d0136ff2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.478+0000 7f04d27fc640 1 -- 172.21.15.37:0/2676533017 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc0bcd70 con 0x55c983e77a00 2024-06-10T01:10:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.478+0000 7f04d27fc640 1 -- 172.21.15.37:0/2676533017 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04cc036040 con 0x55c983e77a00 2024-06-10T01:10:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.478+0000 7f04d27fc640 1 -- 172.21.15.37:0/2676533017 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc0b93e0 con 0x55c983e77a00 2024-06-10T01:10:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.479+0000 7f04e5ba2740 1 -- 172.21.15.37:0/2676533017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c983e77a00 msgr2=0x55c983e81800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.479+0000 7f04e5ba2740 1 --2- 172.21.15.37:0/2676533017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c983e77a00 0x55c983e81800 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f04cc0bca70 tx=0x7f04cc041c00 comp rx=0 tx=0).stop 2024-06-10T01:10:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.479+0000 7f04e5ba2740 1 -- 172.21.15.37:0/2676533017 shutdown_connections 2024-06-10T01:10:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.479+0000 7f04e5ba2740 1 --2- 172.21.15.37:0/2676533017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c983e7cb70 0x55c983e88800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.479+0000 7f04e5ba2740 1 --2- 172.21.15.37:0/2676533017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c983e77a00 0x55c983e81800 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.479+0000 7f04e5ba2740 1 --2- 172.21.15.37:0/2676533017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c983e77010 0x55c983e812c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.479+0000 7f04e5ba2740 1 -- 172.21.15.37:0/2676533017 >> 172.21.15.37:0/2676533017 conn(0x55c983e72680 msgr2=0x55c983e72a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.479+0000 7f04e5ba2740 1 -- 172.21.15.37:0/2676533017 shutdown_connections 2024-06-10T01:10:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.479+0000 7f04e5ba2740 1 -- 172.21.15.37:0/2676533017 wait complete. 2024-06-10T01:10:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.479+0000 7f04e5ba2740 1 -- start start 2024-06-10T01:10:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.479+0000 7f04e5ba2740 1 Processor -- start 2024-06-10T01:10:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.480+0000 7f04e5ba2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c983e77010 0x55c983e88460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.480+0000 7f04e5ba2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c983e77a00 0x55c983e84f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.480+0000 7f04e5ba2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c983e7cb70 0x55c983e85440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.480+0000 7f04e5ba2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc1a8cf0 con 0x55c983e77010 2024-06-10T01:10:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.480+0000 7f04e5ba2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc1aa160 con 0x55c983e77a00 2024-06-10T01:10:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.480+0000 7f04e5ba2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc1a93c0 con 0x55c983e7cb70 2024-06-10T01:10:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.480+0000 7f04e5ba2740 10 client.0 _ll_register_callbacks cb 0x7f04c40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.480+0000 7f04e0c36640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c983e77010 0x55c983e88460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.480+0000 7f04e0c36640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c983e77010 0x55c983e88460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52264/0 (socket says 172.21.15.37:52264) 2024-06-10T01:10:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.480+0000 7f04e0c36640 1 -- 172.21.15.37:0/734116240 learned_addr learned my addr 172.21.15.37:0/734116240 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.480+0000 7f04e1437640 1 --2- 172.21.15.37:0/734116240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c983e7cb70 0x55c983e85440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.480+0000 7f04e0c36640 1 -- 172.21.15.37:0/734116240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c983e7cb70 msgr2=0x55c983e85440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.480+0000 7f04dbfff640 1 --2- 172.21.15.37:0/734116240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c983e77a00 0x55c983e84f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.480+0000 7f04e0c36640 1 --2- 172.21.15.37:0/734116240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c983e7cb70 0x55c983e85440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.480+0000 7f04e0c36640 1 -- 172.21.15.37:0/734116240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c983e77a00 msgr2=0x55c983e84f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.480+0000 7f04e0c36640 1 --2- 172.21.15.37:0/734116240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c983e77a00 0x55c983e84f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.480+0000 7f04e0c36640 1 -- 172.21.15.37:0/734116240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c983e7fa00 con 0x55c983e77010 2024-06-10T01:10:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.480+0000 7f04dbfff640 1 --2- 172.21.15.37:0/734116240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c983e77a00 0x55c983e84f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:10:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.480+0000 7f04e0c36640 1 --2- 172.21.15.37:0/734116240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c983e77010 0x55c983e88460 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f04c4009730 tx=0x7f04c40108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c81f968c832ec7e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.480+0000 7f04bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:10:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.480+0000 7f04bbfff640 1 -- 172.21.15.37:0/734116240 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c40035f0 con 0x55c983e77010 2024-06-10T01:10:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.481+0000 7f04bbfff640 1 -- 172.21.15.37:0/734116240 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7f04c4003790 con 0x55c983e77010 2024-06-10T01:10:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.481+0000 7f04e5ba2740 1 -- 172.21.15.37:0/734116240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04cc036040 con 0x55c983e77010 2024-06-10T01:10:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.481+0000 7f04bbfff640 1 -- 172.21.15.37:0/734116240 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f04c402f050 con 0x55c983e77010 2024-06-10T01:10:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.481+0000 7f04e5ba2740 10 client.17859 fetch_fsmap learned FSMap version 34 2024-06-10T01:10:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.481+0000 7f04e5ba2740 1 -- 172.21.15.37:0/734116240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c983ef6a70 con 0x55c983e77010 2024-06-10T01:10:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.481+0000 7f04bbfff640 1 -- 172.21.15.37:0/734116240 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04cc036040 con 0x55c983e77010 2024-06-10T01:10:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.481+0000 7f04bbfff640 1 -- 172.21.15.37:0/734116240 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f04c4027a60 con 0x55c983e77010 2024-06-10T01:10:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.481+0000 7f04e5ba2740 10 client.17859 fetch_fsmap finished waiting for FSMap version 34 2024-06-10T01:10:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.481+0000 7f04e5ba2740 10 client.17859 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-10T01:10:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.481+0000 7f04e5ba2740 20 client.17859 populate_metadata read hostname 'smithi037' 2024-06-10T01:10:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.481+0000 7f04e5ba2740 1 --2- 172.21.15.37:0/734116240 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c983ef7450 0x55c983ef9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.481+0000 7f04e5ba2740 4 client.17859 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-10T01:10:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.481+0000 7f04e5ba2740 1 -- 172.21.15.37:0/734116240 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c983de1f70 con 0x55c983ef7450 2024-06-10T01:10:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.481+0000 7f04dbfff640 1 --2- 172.21.15.37:0/734116240 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c983ef7450 0x55c983ef9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.482+0000 7f04dbfff640 1 --2- 172.21.15.37:0/734116240 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55c983ef7450 0x55c983ef9870 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72a086dec76cd749 server_cookie=59f50799e3b96363 in_seq=0 out_seq=0 2024-06-10T01:10:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-06-10T01:10:04.482+0000 7f04bbfff640 10 client.17859 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:10:04.505 INFO:tasks.workunit.client.0.smithi037.stderr:+ export PAYLOAD=/tmp/multiple_rsync_payload.217519 2024-06-10T01:10:04.506 INFO:tasks.workunit.client.0.smithi037.stderr:+ PAYLOAD=/tmp/multiple_rsync_payload.217519 2024-06-10T01:10:04.506 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo cp -r /usr/lib/ /tmp/multiple_rsync_payload.217519 2024-06-10T01:10:04.833 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jun 10 01:10:04 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-i[67988]: 2024-06-10T01:10:04.500+0000 7fe3cab4f700 -1 log_channel(cluster) log [ERR] : failed to store backtrace on ino 0x1000000c5da object, pool 2, errno -2 2024-06-10T01:10:04.834 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jun 10 01:10:04 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-i[67988]: 2024-06-10T01:10:04.500+0000 7fe3cab4f700 -1 mds.0.14 unhandled write error (2) No such file or directory, force readonly... 2024-06-10T01:10:04.834 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jun 10 01:10:04 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-i[67988]: 2024-06-10T01:10:04.509+0000 7fe3cab4f700 -1 mds.0.14 unhandled write error (2) No such file or directory, force readonly... 2024-06-10T01:10:05.140 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:04 smithi037 ceph-mon[32192]: failed to store backtrace on ino 0x1000000c5da object, pool 2, errno -2 2024-06-10T01:10:05.140 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:04 smithi037 ceph-mon[32192]: force file system read-only 2024-06-10T01:10:05.333 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:04 smithi167 ceph-mon[36917]: failed to store backtrace on ino 0x1000000c5da object, pool 2, errno -2 2024-06-10T01:10:05.334 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:04 smithi167 ceph-mon[36917]: force file system read-only 2024-06-10T01:10:05.359 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:04 smithi094 ceph-mon[36708]: failed to store backtrace on ino 0x1000000c5da object, pool 2, errno -2 2024-06-10T01:10:05.359 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:04 smithi094 ceph-mon[36708]: force file system read-only 2024-06-10T01:10:06.140 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:05 smithi037 ceph-mon[32192]: pgmap v837: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-06-10T01:10:06.333 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:05 smithi167 ceph-mon[36917]: pgmap v837: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-06-10T01:10:06.359 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:05 smithi094 ceph-mon[36708]: pgmap v837: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-06-10T01:10:08.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:08 smithi167 ceph-mon[36917]: pgmap v838: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2024-06-10T01:10:08.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:08 smithi094 ceph-mon[36708]: pgmap v838: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2024-06-10T01:10:08.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:08 smithi037 ceph-mon[32192]: pgmap v838: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2024-06-10T01:10:10.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:10 smithi167 ceph-mon[36917]: pgmap v839: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-06-10T01:10:10.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:10 smithi094 ceph-mon[36708]: pgmap v839: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-06-10T01:10:10.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:10 smithi037 ceph-mon[32192]: pgmap v839: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-06-10T01:10:11.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:11 smithi167 ceph-mon[36917]: Health check failed: 1 MDSs are read only (MDS_READ_ONLY) 2024-06-10T01:10:11.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:11 smithi167 ceph-mon[36917]: mds.? [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] up:active 2024-06-10T01:10:11.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:11 smithi167 ceph-mon[36917]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:10:11.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:11 smithi094 ceph-mon[36708]: Health check failed: 1 MDSs are read only (MDS_READ_ONLY) 2024-06-10T01:10:11.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:11 smithi094 ceph-mon[36708]: mds.? [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] up:active 2024-06-10T01:10:11.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:11 smithi094 ceph-mon[36708]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:10:11.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:11 smithi037 ceph-mon[32192]: Health check failed: 1 MDSs are read only (MDS_READ_ONLY) 2024-06-10T01:10:11.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:11 smithi037 ceph-mon[32192]: mds.? [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] up:active 2024-06-10T01:10:11.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:11 smithi037 ceph-mon[32192]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-06-10T01:10:12.542 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:12 smithi037 ceph-mon[32192]: pgmap v840: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-06-10T01:10:12.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:12 smithi167 ceph-mon[36917]: pgmap v840: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-06-10T01:10:12.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:12 smithi094 ceph-mon[36708]: pgmap v840: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-06-10T01:10:14.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:14 smithi167 ceph-mon[36917]: pgmap v841: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-06-10T01:10:14.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:14 smithi094 ceph-mon[36708]: pgmap v841: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-06-10T01:10:14.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:14 smithi037 ceph-mon[32192]: pgmap v841: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-06-10T01:10:16.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:16 smithi167 ceph-mon[36917]: pgmap v842: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-06-10T01:10:16.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:16 smithi094 ceph-mon[36708]: pgmap v842: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-06-10T01:10:16.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:16 smithi037 ceph-mon[32192]: pgmap v842: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-06-10T01:10:18.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:18 smithi167 ceph-mon[36917]: pgmap v843: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-06-10T01:10:18.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:18 smithi094 ceph-mon[36708]: pgmap v843: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-06-10T01:10:18.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:18 smithi037 ceph-mon[32192]: pgmap v843: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-06-10T01:10:20.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:20 smithi167 ceph-mon[36917]: pgmap v844: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-06-10T01:10:20.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:20 smithi094 ceph-mon[36708]: pgmap v844: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-06-10T01:10:20.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:20 smithi037 ceph-mon[32192]: pgmap v844: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-06-10T01:10:22.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:22 smithi167 ceph-mon[36917]: pgmap v845: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:10:22.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:22 smithi094 ceph-mon[36708]: pgmap v845: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:10:22.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:22 smithi037 ceph-mon[32192]: pgmap v845: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:10:24.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:24 smithi167 ceph-mon[36917]: pgmap v846: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:10:24.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:24 smithi094 ceph-mon[36708]: pgmap v846: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:10:24.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:24 smithi037 ceph-mon[32192]: pgmap v846: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:10:25.604 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.2 2024-06-10T01:10:25.605 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:10:25.863 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:10:26.238 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:26 smithi037 ceph-mon[32192]: pgmap v847: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:10:26.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:26 smithi167 ceph-mon[36917]: pgmap v847: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:10:26.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:26 smithi094 ceph-mon[36708]: pgmap v847: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:10:28.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.110+0000 7fe21ed69700 1 -- 172.21.15.37:0/1639455954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe218102ef0 msgr2=0x7fe21810f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:28.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.110+0000 7fe21ed69700 1 --2- 172.21.15.37:0/1639455954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe218102ef0 0x7fe21810f3a0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fe2100099c0 tx=0x7fe210009cd0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.110+0000 7fe21ed69700 1 -- 172.21.15.37:0/1639455954 shutdown_connections 2024-06-10T01:10:28.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.110+0000 7fe21ed69700 1 --2- 172.21.15.37:0/1639455954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe218102ef0 0x7fe21810f3a0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.110+0000 7fe21ed69700 1 --2- 172.21.15.37:0/1639455954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2181085d0 0x7fe2181089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.110+0000 7fe21ed69700 1 --2- 172.21.15.37:0/1639455954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe218102530 0x7fe2181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.110+0000 7fe21ed69700 1 -- 172.21.15.37:0/1639455954 >> 172.21.15.37:0/1639455954 conn(0x7fe218079640 msgr2=0x7fe218079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:28.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.111+0000 7fe21ed69700 1 -- 172.21.15.37:0/1639455954 shutdown_connections 2024-06-10T01:10:28.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.111+0000 7fe21ed69700 1 -- 172.21.15.37:0/1639455954 wait complete. 2024-06-10T01:10:28.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.111+0000 7fe21ed69700 1 Processor -- start 2024-06-10T01:10:28.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.111+0000 7fe21ed69700 1 -- start start 2024-06-10T01:10:28.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.112+0000 7fe21ed69700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe218102530 0x7fe21819e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:28.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.112+0000 7fe21ed69700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe218102ef0 0x7fe21819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:28.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.112+0000 7fe21ed69700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2181085d0 0x7fe2181987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:28.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.112+0000 7fe21ed69700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe218112810 con 0x7fe218102ef0 2024-06-10T01:10:28.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.112+0000 7fe21ed69700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe218112690 con 0x7fe218102530 2024-06-10T01:10:28.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.112+0000 7fe21ed69700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe218112990 con 0x7fe2181085d0 2024-06-10T01:10:28.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.112+0000 7fe20ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe218102ef0 0x7fe21819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:28.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.112+0000 7fe20ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe218102ef0 0x7fe21819eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36362/0 (socket says 172.21.15.37:36362) 2024-06-10T01:10:28.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.112+0000 7fe20ffff700 1 -- 172.21.15.37:0/3845461074 learned_addr learned my addr 172.21.15.37:0/3845461074 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:28.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.112+0000 7fe21cb05700 1 --2- 172.21.15.37:0/3845461074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe218102530 0x7fe21819e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:28.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.112+0000 7fe21d306700 1 --2- 172.21.15.37:0/3845461074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2181085d0 0x7fe2181987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:28.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.112+0000 7fe20ffff700 1 -- 172.21.15.37:0/3845461074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2181085d0 msgr2=0x7fe2181987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:28.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.112+0000 7fe20ffff700 1 --2- 172.21.15.37:0/3845461074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2181085d0 0x7fe2181987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.112+0000 7fe20ffff700 1 -- 172.21.15.37:0/3845461074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe218102530 msgr2=0x7fe21819e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:28.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.112+0000 7fe20ffff700 1 --2- 172.21.15.37:0/3845461074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe218102530 0x7fe21819e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.112+0000 7fe20ffff700 1 -- 172.21.15.37:0/3845461074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe218198fb0 con 0x7fe218102ef0 2024-06-10T01:10:28.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.112+0000 7fe20ffff700 1 --2- 172.21.15.37:0/3845461074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe218102ef0 0x7fe21819eb90 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fe20800d950 tx=0x7fe20800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:28.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.113+0000 7fe20dffb700 1 -- 172.21.15.37:0/3845461074 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe208016070 con 0x7fe218102ef0 2024-06-10T01:10:28.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.113+0000 7fe20dffb700 1 -- 172.21.15.37:0/3845461074 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe20800fd70 con 0x7fe218102ef0 2024-06-10T01:10:28.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.113+0000 7fe21ed69700 1 -- 172.21.15.37:0/3845461074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2181992a0 con 0x7fe218102ef0 2024-06-10T01:10:28.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.113+0000 7fe20dffb700 1 -- 172.21.15.37:0/3845461074 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20800e610 con 0x7fe218102ef0 2024-06-10T01:10:28.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.113+0000 7fe21ed69700 1 -- 172.21.15.37:0/3845461074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2181a54b0 con 0x7fe218102ef0 2024-06-10T01:10:28.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.114+0000 7fe21ed69700 1 -- 172.21.15.37:0/3845461074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe218103c40 con 0x7fe218102ef0 2024-06-10T01:10:28.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.115+0000 7fe20dffb700 1 -- 172.21.15.37:0/3845461074 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe208019070 con 0x7fe218102ef0 2024-06-10T01:10:28.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.116+0000 7fe20dffb700 1 --2- 172.21.15.37:0/3845461074 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe2000a2ca0 0x7fe2000a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:28.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.116+0000 7fe20dffb700 1 -- 172.21.15.37:0/3845461074 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7fe2080cee60 con 0x7fe218102ef0 2024-06-10T01:10:28.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.116+0000 7fe21cb05700 1 --2- 172.21.15.37:0/3845461074 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe2000a2ca0 0x7fe2000a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:28.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.116+0000 7fe21cb05700 1 --2- 172.21.15.37:0/3845461074 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe2000a2ca0 0x7fe2000a5160 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe214009550 tx=0x7fe2140091b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:28.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.121+0000 7fe20dffb700 1 -- 172.21.15.37:0/3845461074 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe20809b9d0 con 0x7fe218102ef0 2024-06-10T01:10:28.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.376+0000 7fe21ed69700 1 -- 172.21.15.37:0/3845461074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2181a5710 con 0x7fe218102ef0 2024-06-10T01:10:28.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:28 smithi037 ceph-mon[32192]: pgmap v848: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-06-10T01:10:28.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.377+0000 7fe20dffb700 1 -- 172.21.15.37:0/3845461074 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11870 (secure 0 0 0) 0x7fe20809b2d0 con 0x7fe218102ef0 2024-06-10T01:10:28.372 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:10:28.373 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":26554,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2107277223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2107277223},{"type":"v1","addr":"172.21.15.167:6835","nonce":2107277223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T01:10:10.062065+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":271,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24458,24472,24451]},"id":1}]} 2024-06-10T01:10:28.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.380+0000 7fe21ed69700 1 -- 172.21.15.37:0/3845461074 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe2000a2ca0 msgr2=0x7fe2000a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:28.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.380+0000 7fe21ed69700 1 --2- 172.21.15.37:0/3845461074 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe2000a2ca0 0x7fe2000a5160 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe214009550 tx=0x7fe2140091b0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.380+0000 7fe21ed69700 1 -- 172.21.15.37:0/3845461074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe218102ef0 msgr2=0x7fe21819eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:28.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.380+0000 7fe21ed69700 1 --2- 172.21.15.37:0/3845461074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe218102ef0 0x7fe21819eb90 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fe20800d950 tx=0x7fe20800dc60 comp rx=0 tx=0).stop 2024-06-10T01:10:28.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.380+0000 7fe21ed69700 1 -- 172.21.15.37:0/3845461074 shutdown_connections 2024-06-10T01:10:28.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.380+0000 7fe21ed69700 1 --2- 172.21.15.37:0/3845461074 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe2000a2ca0 0x7fe2000a5160 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.380+0000 7fe21ed69700 1 --2- 172.21.15.37:0/3845461074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2181085d0 0x7fe2181987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.380+0000 7fe21ed69700 1 --2- 172.21.15.37:0/3845461074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe218102530 0x7fe21819e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.380+0000 7fe21ed69700 1 --2- 172.21.15.37:0/3845461074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe218102ef0 0x7fe21819eb90 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:28.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.380+0000 7fe21ed69700 1 -- 172.21.15.37:0/3845461074 >> 172.21.15.37:0/3845461074 conn(0x7fe218079640 msgr2=0x7fe218109350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:28.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.380+0000 7fe21ed69700 1 -- 172.21.15.37:0/3845461074 shutdown_connections 2024-06-10T01:10:28.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:28.380+0000 7fe21ed69700 1 -- 172.21.15.37:0/3845461074 wait complete. 2024-06-10T01:10:28.376 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 35 2024-06-10T01:10:28.387 INFO:tasks.workunit.client.0.smithi037.stderr:+ set -e 2024-06-10T01:10:28.388 INFO:tasks.workunit.client.0.smithi037.stderr:+ sudo rsync -av /tmp/multiple_rsync_payload.217519 payload.1 2024-06-10T01:10:28.502 INFO:tasks.workunit.client.0.smithi037.stdout:sending incremental file list 2024-06-10T01:10:28.502 INFO:tasks.workunit.client.0.smithi037.stderr:rsync: [Receiver] mkdir "/home/ubuntu/cephtest/mnt.0/client.0/tmp/payload.1" failed: Read-only file system (30) 2024-06-10T01:10:28.502 INFO:tasks.workunit.client.0.smithi037.stderr:rsync error: error in file IO (code 11) at main.c(783) [Receiver=3.2.3] 2024-06-10T01:10:28.506 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-06-10T01:10:28.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:28 smithi167 ceph-mon[36917]: pgmap v848: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-06-10T01:10:28.601 INFO:tasks.workunit:Stopping ['fs/misc'] on client.0... 2024-06-10T01:10:28.602 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-06-10T01:10:28.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:28 smithi094 ceph-mon[36708]: pgmap v848: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-06-10T01:10:29.172 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24472 quiesce db / --timeout 90 --expiration 147.2 --await-for 45.0' 2024-06-10T01:10:29.172 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24472 quiesce db / --timeout 90 --expiration 147.2 --await-for 45.0 2024-06-10T01:10:29.433 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:10:29.557 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:29 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3845461074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:10:29.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:29 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3845461074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:10:29.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:29 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3845461074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:10:30.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:30 smithi167 ceph-mon[36917]: pgmap v849: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T01:10:30.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:30 smithi094 ceph-mon[36708]: pgmap v849: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T01:10:30.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:30 smithi037 ceph-mon[32192]: pgmap v849: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T01:10:31.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.294+0000 7f4531a82700 1 -- 172.21.15.37:0/2994837684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f452c1066c0 msgr2=0x7f452c10e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:31.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.294+0000 7f4531a82700 1 --2- 172.21.15.37:0/2994837684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f452c1066c0 0x7f452c10e290 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f45240099c0 tx=0x7f4524009cd0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.294+0000 7f4531a82700 1 -- 172.21.15.37:0/2994837684 shutdown_connections 2024-06-10T01:10:31.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.294+0000 7f4531a82700 1 --2- 172.21.15.37:0/2994837684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f452c1066c0 0x7f452c10e290 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.294+0000 7f4531a82700 1 --2- 172.21.15.37:0/2994837684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f452c1015f0 0x7f452c1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.294+0000 7f4531a82700 1 --2- 172.21.15.37:0/2994837684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f452c101fa0 0x7f452c105fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.294+0000 7f4531a82700 1 -- 172.21.15.37:0/2994837684 >> 172.21.15.37:0/2994837684 conn(0x7f452c078330 msgr2=0x7f452c078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:31.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.294+0000 7f4531a82700 1 -- 172.21.15.37:0/2994837684 shutdown_connections 2024-06-10T01:10:31.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.294+0000 7f4531a82700 1 -- 172.21.15.37:0/2994837684 wait complete. 2024-06-10T01:10:31.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.295+0000 7f4531a82700 1 Processor -- start 2024-06-10T01:10:31.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.295+0000 7f4531a82700 1 -- start start 2024-06-10T01:10:31.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.296+0000 7f4531a82700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f452c1015f0 0x7f452c19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:31.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.296+0000 7f4531a82700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f452c101fa0 0x7f452c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:31.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.296+0000 7f4531a82700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f452c1066c0 0x7f452c198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:31.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.296+0000 7f4531a82700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f452c1116d0 con 0x7f452c1066c0 2024-06-10T01:10:31.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.296+0000 7f4531a82700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f452c111550 con 0x7f452c101fa0 2024-06-10T01:10:31.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.296+0000 7f4531a82700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f452c111850 con 0x7f452c1015f0 2024-06-10T01:10:31.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.296+0000 7f452bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f452c1066c0 0x7f452c198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:31.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.296+0000 7f452bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f452c1066c0 0x7f452c198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36382/0 (socket says 172.21.15.37:36382) 2024-06-10T01:10:31.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.296+0000 7f452bfff700 1 -- 172.21.15.37:0/4024021746 learned_addr learned my addr 172.21.15.37:0/4024021746 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:31.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.296+0000 7f452b7fe700 1 --2- 172.21.15.37:0/4024021746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f452c1015f0 0x7f452c19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:31.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.296+0000 7f452affd700 1 --2- 172.21.15.37:0/4024021746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f452c101fa0 0x7f452c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:31.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.296+0000 7f452bfff700 1 -- 172.21.15.37:0/4024021746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f452c1015f0 msgr2=0x7f452c19e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:31.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.296+0000 7f452bfff700 1 --2- 172.21.15.37:0/4024021746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f452c1015f0 0x7f452c19e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.296+0000 7f452bfff700 1 -- 172.21.15.37:0/4024021746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f452c101fa0 msgr2=0x7f452c19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:31.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.296+0000 7f452bfff700 1 --2- 172.21.15.37:0/4024021746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f452c101fa0 0x7f452c19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.296+0000 7f452bfff700 1 -- 172.21.15.37:0/4024021746 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452c199670 con 0x7f452c1066c0 2024-06-10T01:10:31.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.296+0000 7f452bfff700 1 --2- 172.21.15.37:0/4024021746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f452c1066c0 0x7f452c198db0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f4524006010 tx=0x7f452400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:31.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.297+0000 7f4528ff9700 1 -- 172.21.15.37:0/4024021746 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4524027070 con 0x7f452c1066c0 2024-06-10T01:10:31.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.297+0000 7f4531a82700 1 -- 172.21.15.37:0/4024021746 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452c199900 con 0x7f452c1066c0 2024-06-10T01:10:31.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.297+0000 7f4528ff9700 1 -- 172.21.15.37:0/4024021746 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4524003680 con 0x7f452c1066c0 2024-06-10T01:10:31.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.297+0000 7f4528ff9700 1 -- 172.21.15.37:0/4024021746 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45240173f0 con 0x7f452c1066c0 2024-06-10T01:10:31.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.297+0000 7f4531a82700 1 -- 172.21.15.37:0/4024021746 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f452c1a57d0 con 0x7f452c1066c0 2024-06-10T01:10:31.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.299+0000 7f4528ff9700 1 -- 172.21.15.37:0/4024021746 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f452402a070 con 0x7f452c1066c0 2024-06-10T01:10:31.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.300+0000 7f4528ff9700 1 --2- 172.21.15.37:0/4024021746 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f45180a2cc0 0x7f45180a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:31.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.300+0000 7f452b7fe700 1 --2- 172.21.15.37:0/4024021746 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f45180a2cc0 0x7f45180a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:31.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.300+0000 7f4528ff9700 1 -- 172.21.15.37:0/4024021746 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7f45240cfff0 con 0x7f452c1066c0 2024-06-10T01:10:31.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.300+0000 7f452b7fe700 1 --2- 172.21.15.37:0/4024021746 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f45180a2cc0 0x7f45180a5180 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f451c009550 tx=0x7f451c0091b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:31.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.301+0000 7f453c305b80 1 Processor -- start 2024-06-10T01:10:31.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.301+0000 7f453c305b80 1 -- start start 2024-06-10T01:10:31.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.301+0000 7f453c305b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a0b765cd0 0x564a0b7660b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:31.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.301+0000 7f453c305b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a0b7665f0 0x564a0b75ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:31.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.301+0000 7f453c305b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a0b760460 0x564a0b7608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:31.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.301+0000 7f453c305b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564a0b6b19a0 con 0x564a0b760460 2024-06-10T01:10:31.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.301+0000 7f453c305b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564a0b6598b0 con 0x564a0b765cd0 2024-06-10T01:10:31.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.301+0000 7f453c305b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564a0b6ad960 con 0x564a0b7665f0 2024-06-10T01:10:31.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.301+0000 7f452bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a0b760460 0x564a0b7608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:31.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.301+0000 7f452bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a0b760460 0x564a0b7608a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36396/0 (socket says 172.21.15.37:36396) 2024-06-10T01:10:31.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.301+0000 7f452bfff700 1 -- 172.21.15.37:0/2134450122 learned_addr learned my addr 172.21.15.37:0/2134450122 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:31.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.301+0000 7f452affd700 1 --2- 172.21.15.37:0/2134450122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a0b7665f0 0x564a0b75ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:31.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.301+0000 7f452b7fe700 1 --2- 172.21.15.37:0/2134450122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a0b765cd0 0x564a0b7660b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:31.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.301+0000 7f452bfff700 1 -- 172.21.15.37:0/2134450122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a0b7665f0 msgr2=0x564a0b75ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:31.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.301+0000 7f452bfff700 1 --2- 172.21.15.37:0/2134450122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a0b7665f0 0x564a0b75ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.301+0000 7f452bfff700 1 -- 172.21.15.37:0/2134450122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a0b765cd0 msgr2=0x564a0b7660b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:31.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.301+0000 7f452bfff700 1 --2- 172.21.15.37:0/2134450122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a0b765cd0 0x564a0b7660b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.301+0000 7f452bfff700 1 -- 172.21.15.37:0/2134450122 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a0b6500e0 con 0x564a0b760460 2024-06-10T01:10:31.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.301+0000 7f452b7fe700 1 --2- 172.21.15.37:0/2134450122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a0b765cd0 0x564a0b7660b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:10:31.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.302+0000 7f452bfff700 1 --2- 172.21.15.37:0/2134450122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a0b760460 0x564a0b7608a0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f452409c460 tx=0x7f4524009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4ed070037502d8 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:31.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.302+0000 7f4521ffb700 1 -- 172.21.15.37:0/2134450122 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452400fe00 con 0x564a0b760460 2024-06-10T01:10:31.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.302+0000 7f4521ffb700 1 -- 172.21.15.37:0/2134450122 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f452409cbd0 con 0x564a0b760460 2024-06-10T01:10:31.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.302+0000 7f4521ffb700 1 -- 172.21.15.37:0/2134450122 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45240a4e80 con 0x564a0b760460 2024-06-10T01:10:31.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.303+0000 7f453c305b80 1 -- 172.21.15.37:0/2134450122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a0b760460 msgr2=0x564a0b7608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:31.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.303+0000 7f453c305b80 1 --2- 172.21.15.37:0/2134450122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a0b760460 0x564a0b7608a0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f452409c460 tx=0x7f4524009310 comp rx=0 tx=0).stop 2024-06-10T01:10:31.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.303+0000 7f453c305b80 1 -- 172.21.15.37:0/2134450122 shutdown_connections 2024-06-10T01:10:31.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.303+0000 7f453c305b80 1 --2- 172.21.15.37:0/2134450122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a0b760460 0x564a0b7608a0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.303+0000 7f453c305b80 1 --2- 172.21.15.37:0/2134450122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a0b765cd0 0x564a0b7660b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.303+0000 7f453c305b80 1 --2- 172.21.15.37:0/2134450122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a0b7665f0 0x564a0b75ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.303+0000 7f453c305b80 1 -- 172.21.15.37:0/2134450122 >> 172.21.15.37:0/2134450122 conn(0x564a0b68ae50 msgr2=0x564a0b75b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:31.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.303+0000 7f453c305b80 1 -- 172.21.15.37:0/2134450122 shutdown_connections 2024-06-10T01:10:31.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.303+0000 7f453c305b80 1 -- 172.21.15.37:0/2134450122 wait complete. 2024-06-10T01:10:31.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.303+0000 7f453c305b80 1 -- start start 2024-06-10T01:10:31.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.303+0000 7f453c305b80 1 Processor -- start 2024-06-10T01:10:31.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.304+0000 7f453c305b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a0b760460 0x564a0b76c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:31.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.304+0000 7f453c305b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a0b765cd0 0x564a0b7696e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:31.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.304+0000 7f453c305b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a0b7665f0 0x564a0b769c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:31.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.304+0000 7f453c305b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564a0b76cd80 con 0x564a0b765cd0 2024-06-10T01:10:31.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.304+0000 7f453c305b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564a0b651d80 con 0x564a0b760460 2024-06-10T01:10:31.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.304+0000 7f453c305b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564a0b76cf00 con 0x564a0b7665f0 2024-06-10T01:10:31.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.304+0000 7f453c305b80 10 client.0 _ll_register_callbacks cb 0x564a0b64bab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:31.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.304+0000 7f452affd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a0b765cd0 0x564a0b7696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:31.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.304+0000 7f452affd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a0b765cd0 0x564a0b7696e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36398/0 (socket says 172.21.15.37:36398) 2024-06-10T01:10:31.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.304+0000 7f452affd700 1 -- 172.21.15.37:0/3408023822 learned_addr learned my addr 172.21.15.37:0/3408023822 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:31.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.304+0000 7f452bfff700 1 --2- 172.21.15.37:0/3408023822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a0b7665f0 0x564a0b769c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:31.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.304+0000 7f452b7fe700 1 --2- 172.21.15.37:0/3408023822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a0b760460 0x564a0b76c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:31.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.304+0000 7f452affd700 1 -- 172.21.15.37:0/3408023822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a0b7665f0 msgr2=0x564a0b769c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:31.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.304+0000 7f452affd700 1 --2- 172.21.15.37:0/3408023822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a0b7665f0 0x564a0b769c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.304+0000 7f452affd700 1 -- 172.21.15.37:0/3408023822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a0b760460 msgr2=0x564a0b76c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:31.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.304+0000 7f452affd700 1 --2- 172.21.15.37:0/3408023822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a0b760460 0x564a0b76c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.304+0000 7f452affd700 1 -- 172.21.15.37:0/3408023822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a0b600a60 con 0x564a0b765cd0 2024-06-10T01:10:31.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.304+0000 7f452affd700 1 --2- 172.21.15.37:0/3408023822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a0b765cd0 0x564a0b7696e0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f4514012700 tx=0x7f4514012a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6cdb9341d567ed server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:31.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.304+0000 7f450affd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:10:31.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.305+0000 7f450affd700 1 -- 172.21.15.37:0/3408023822 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4514004020 con 0x564a0b765cd0 2024-06-10T01:10:31.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.305+0000 7f450affd700 1 -- 172.21.15.37:0/3408023822 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7f45140041c0 con 0x564a0b765cd0 2024-06-10T01:10:31.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.305+0000 7f453c305b80 1 -- 172.21.15.37:0/3408023822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f452409cbd0 con 0x564a0b765cd0 2024-06-10T01:10:31.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.305+0000 7f450affd700 1 -- 172.21.15.37:0/3408023822 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4514025050 con 0x564a0b765cd0 2024-06-10T01:10:31.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.305+0000 7f453c305b80 10 client.17877 fetch_fsmap learned FSMap version 35 2024-06-10T01:10:31.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.305+0000 7f453c305b80 1 -- 172.21.15.37:0/3408023822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564a0b777690 con 0x564a0b765cd0 2024-06-10T01:10:31.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.305+0000 7f450affd700 1 -- 172.21.15.37:0/3408023822 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f452409cbd0 con 0x564a0b765cd0 2024-06-10T01:10:31.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.305+0000 7f450affd700 1 -- 172.21.15.37:0/3408023822 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f451401c6a0 con 0x564a0b765cd0 2024-06-10T01:10:31.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.305+0000 7f453c305b80 10 client.17877 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:10:31.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.305+0000 7f453c305b80 10 client.17877 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:10:31.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.305+0000 7f453c305b80 20 client.17877 populate_metadata read hostname 'smithi037' 2024-06-10T01:10:31.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.305+0000 7f453c305b80 1 --2- 172.21.15.37:0/3408023822 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564a0b779cd0 0x564a0b77c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:31.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.305+0000 7f453c305b80 4 client.17877 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:10:31.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.305+0000 7f453c305b80 1 -- 172.21.15.37:0/3408023822 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a0b6beea0 con 0x564a0b779cd0 2024-06-10T01:10:31.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.306+0000 7f452b7fe700 1 --2- 172.21.15.37:0/3408023822 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564a0b779cd0 0x564a0b77c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:31.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.306+0000 7f450affd700 10 client.17877 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:10:31.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.306+0000 7f452b7fe700 1 --2- 172.21.15.37:0/3408023822 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564a0b779cd0 0x564a0b77c0c0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa1e8b374e141e96 server_cookie=8dc29de3110f2992 in_seq=0 out_seq=0 2024-06-10T01:10:31.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.307+0000 7f450affd700 1 -- 172.21.15.37:0/3408023822 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564a0b6beea0 con 0x564a0b779cd0 2024-06-10T01:10:31.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.307+0000 7f450affd700 10 client.17877 handle_command_reply: tid=0 2024-06-10T01:10:31.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.307+0000 7f453c305b80 1 client.17877 shutdown 2024-06-10T01:10:31.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.307+0000 7f453c305b80 1 -- 172.21.15.37:0/3408023822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a0b765cd0 msgr2=0x564a0b7696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:31.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.307+0000 7f453c305b80 1 --2- 172.21.15.37:0/3408023822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a0b765cd0 0x564a0b7696e0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f4514012700 tx=0x7f4514012a10 comp rx=0 tx=0).stop 2024-06-10T01:10:31.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.307+0000 7f453c305b80 1 -- 172.21.15.37:0/3408023822 shutdown_connections 2024-06-10T01:10:31.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.307+0000 7f453c305b80 1 --2- 172.21.15.37:0/3408023822 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564a0b779cd0 0x564a0b77c0c0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.307+0000 7f453c305b80 1 --2- 172.21.15.37:0/3408023822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a0b7665f0 0x564a0b769c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.307+0000 7f453c305b80 1 --2- 172.21.15.37:0/3408023822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a0b760460 0x564a0b76c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.307+0000 7f450affd700 0 client.17877 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:10:31.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.307+0000 7f453c305b80 1 --2- 172.21.15.37:0/3408023822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a0b765cd0 0x564a0b7696e0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.307+0000 7f453c305b80 1 -- 172.21.15.37:0/3408023822 >> 172.21.15.37:0/3408023822 conn(0x564a0b68ae50 msgr2=0x564a0b75ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:31.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.307+0000 7f453c305b80 1 -- 172.21.15.37:0/3408023822 shutdown_connections 2024-06-10T01:10:31.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.307+0000 7f453c305b80 1 -- 172.21.15.37:0/3408023822 wait complete. 2024-06-10T01:10:31.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.307+0000 7f453c305b80 20 client.17877 trim_cache size 0 max 16384 2024-06-10T01:10:31.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.341+0000 7f453c305b80 1 Processor -- start 2024-06-10T01:10:31.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.341+0000 7f453c305b80 1 -- start start 2024-06-10T01:10:31.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.341+0000 7f453c305b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a0b760460 0x564a0b75ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:31.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.341+0000 7f453c305b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a0b765cd0 0x564a0b769c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:31.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.341+0000 7f453c305b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a0b7665f0 0x564a0b767260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:31.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.341+0000 7f453c305b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f452c1b0c00 con 0x564a0b765cd0 2024-06-10T01:10:31.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.341+0000 7f453c305b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f452c1ae490 con 0x564a0b760460 2024-06-10T01:10:31.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.341+0000 7f453c305b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f452c1ac2e0 con 0x564a0b7665f0 2024-06-10T01:10:31.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.342+0000 7f452affd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a0b765cd0 0x564a0b769c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:31.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.342+0000 7f452affd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a0b765cd0 0x564a0b769c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36404/0 (socket says 172.21.15.37:36404) 2024-06-10T01:10:31.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.342+0000 7f452affd700 1 -- 172.21.15.37:0/4192815101 learned_addr learned my addr 172.21.15.37:0/4192815101 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:31.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.342+0000 7f452b7fe700 1 --2- 172.21.15.37:0/4192815101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a0b760460 0x564a0b75ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:31.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.342+0000 7f452bfff700 1 --2- 172.21.15.37:0/4192815101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a0b7665f0 0x564a0b767260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:31.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.342+0000 7f452affd700 1 -- 172.21.15.37:0/4192815101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a0b7665f0 msgr2=0x564a0b767260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:31.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.342+0000 7f452affd700 1 --2- 172.21.15.37:0/4192815101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a0b7665f0 0x564a0b767260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.342+0000 7f452affd700 1 -- 172.21.15.37:0/4192815101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a0b760460 msgr2=0x564a0b75ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:31.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.342+0000 7f452affd700 1 --2- 172.21.15.37:0/4192815101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a0b760460 0x564a0b75ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.342+0000 7f452affd700 1 -- 172.21.15.37:0/4192815101 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a0b76bf60 con 0x564a0b765cd0 2024-06-10T01:10:31.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.342+0000 7f452bfff700 1 --2- 172.21.15.37:0/4192815101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a0b7665f0 0x564a0b767260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:10:31.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.342+0000 7f452affd700 1 --2- 172.21.15.37:0/4192815101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a0b765cd0 0x564a0b769c50 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f451401caf0 tx=0x7f45140123c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770f6c88cf013cf9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:31.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.342+0000 7f45217fa700 1 -- 172.21.15.37:0/4192815101 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451401c530 con 0x564a0b765cd0 2024-06-10T01:10:31.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.342+0000 7f45217fa700 1 -- 172.21.15.37:0/4192815101 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f451401c6d0 con 0x564a0b765cd0 2024-06-10T01:10:31.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.342+0000 7f45217fa700 1 -- 172.21.15.37:0/4192815101 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45140107f0 con 0x564a0b765cd0 2024-06-10T01:10:31.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.343+0000 7f453c305b80 1 -- 172.21.15.37:0/4192815101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a0b765cd0 msgr2=0x564a0b769c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:31.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.343+0000 7f453c305b80 1 --2- 172.21.15.37:0/4192815101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a0b765cd0 0x564a0b769c50 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f451401caf0 tx=0x7f45140123c0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.343+0000 7f453c305b80 1 -- 172.21.15.37:0/4192815101 shutdown_connections 2024-06-10T01:10:31.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.343+0000 7f453c305b80 1 --2- 172.21.15.37:0/4192815101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a0b7665f0 0x564a0b767260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.343+0000 7f453c305b80 1 --2- 172.21.15.37:0/4192815101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a0b760460 0x564a0b75ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.343+0000 7f453c305b80 1 --2- 172.21.15.37:0/4192815101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a0b765cd0 0x564a0b769c50 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.343+0000 7f453c305b80 1 -- 172.21.15.37:0/4192815101 >> 172.21.15.37:0/4192815101 conn(0x564a0b68ae50 msgr2=0x564a0b75af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:31.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.343+0000 7f453c305b80 1 -- 172.21.15.37:0/4192815101 shutdown_connections 2024-06-10T01:10:31.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.343+0000 7f453c305b80 1 -- 172.21.15.37:0/4192815101 wait complete. 2024-06-10T01:10:31.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.344+0000 7f453c305b80 1 -- start start 2024-06-10T01:10:31.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.344+0000 7f453c305b80 1 Processor -- start 2024-06-10T01:10:31.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.344+0000 7f453c305b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a0b760460 0x564a0b7726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:31.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.344+0000 7f453c305b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a0b765cd0 0x564a0b76f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:31.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.344+0000 7f453c305b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a0b7665f0 0x564a0b76f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:31.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.344+0000 7f453c305b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f452c1ac950 con 0x564a0b760460 2024-06-10T01:10:31.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.344+0000 7f453c305b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f452c1addc0 con 0x564a0b7665f0 2024-06-10T01:10:31.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.344+0000 7f453c305b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f452c1ad020 con 0x564a0b765cd0 2024-06-10T01:10:31.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.344+0000 7f453c305b80 10 client.0 _ll_register_callbacks cb 0x564a0b64bab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:31.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.344+0000 7f452b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a0b760460 0x564a0b7726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:31.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.344+0000 7f452b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a0b760460 0x564a0b7726d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36410/0 (socket says 172.21.15.37:36410) 2024-06-10T01:10:31.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.344+0000 7f452b7fe700 1 -- 172.21.15.37:0/537865651 learned_addr learned my addr 172.21.15.37:0/537865651 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:31.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.345+0000 7f452b7fe700 1 -- 172.21.15.37:0/537865651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a0b765cd0 msgr2=0x564a0b76f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:31.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.345+0000 7f452b7fe700 1 --2- 172.21.15.37:0/537865651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a0b765cd0 0x564a0b76f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.345+0000 7f452b7fe700 1 -- 172.21.15.37:0/537865651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a0b7665f0 msgr2=0x564a0b76f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:31.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.345+0000 7f452b7fe700 1 --2- 172.21.15.37:0/537865651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a0b7665f0 0x564a0b76f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.345+0000 7f452b7fe700 1 -- 172.21.15.37:0/537865651 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a0b76fc60 con 0x564a0b760460 2024-06-10T01:10:31.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.345+0000 7f452b7fe700 1 --2- 172.21.15.37:0/537865651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a0b760460 0x564a0b7726d0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f451c003fb0 tx=0x7f451c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632b2f07c38e1ca2 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:31.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.345+0000 7f450affd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:10:31.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.345+0000 7f450affd700 1 -- 172.21.15.37:0/537865651 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c01c040 con 0x564a0b760460 2024-06-10T01:10:31.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.345+0000 7f450affd700 1 -- 172.21.15.37:0/537865651 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7f451c007db0 con 0x564a0b760460 2024-06-10T01:10:31.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.345+0000 7f453c305b80 1 -- 172.21.15.37:0/537865651 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f451401c6d0 con 0x564a0b760460 2024-06-10T01:10:31.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.345+0000 7f450affd700 1 -- 172.21.15.37:0/537865651 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f451c037050 con 0x564a0b760460 2024-06-10T01:10:31.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.345+0000 7f453c305b80 10 client.17883 fetch_fsmap learned FSMap version 35 2024-06-10T01:10:31.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.345+0000 7f453c305b80 1 -- 172.21.15.37:0/537865651 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564a0b7e56a0 con 0x564a0b760460 2024-06-10T01:10:31.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.345+0000 7f450affd700 1 -- 172.21.15.37:0/537865651 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f451401c6d0 con 0x564a0b760460 2024-06-10T01:10:31.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.346+0000 7f450affd700 1 -- 172.21.15.37:0/537865651 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f451c0225e0 con 0x564a0b760460 2024-06-10T01:10:31.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.346+0000 7f453c305b80 10 client.17883 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:10:31.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.346+0000 7f453c305b80 10 client.17883 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:10:31.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.346+0000 7f453c305b80 20 client.17883 populate_metadata read hostname 'smithi037' 2024-06-10T01:10:31.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.346+0000 7f453c305b80 1 --2- 172.21.15.37:0/537865651 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564a0b7e5e80 0x564a0b7e8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:31.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.346+0000 7f453c305b80 4 client.17883 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.2, "await_for": 45.0}] 2024-06-10T01:10:31.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.346+0000 7f453c305b80 1 -- 172.21.15.37:0/537865651 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.2, "await_for": 45.0}) -- 0x564a0b6b9420 con 0x564a0b7e5e80 2024-06-10T01:10:31.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.346+0000 7f452affd700 1 --2- 172.21.15.37:0/537865651 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564a0b7e5e80 0x564a0b7e8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:31.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.346+0000 7f452affd700 1 --2- 172.21.15.37:0/537865651 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564a0b7e5e80 0x564a0b7e8300 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4aafa679f8f7e079 server_cookie=26f1987ae2dd6e27 in_seq=0 out_seq=0 2024-06-10T01:10:31.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.346+0000 7f450affd700 10 client.17883 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:10:31.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.355+0000 7f450affd700 1 -- 172.21.15.37:0/537865651 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564a0b6b9420 con 0x564a0b7e5e80 2024-06-10T01:10:31.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.355+0000 7f450affd700 10 client.17883 handle_command_reply: tid=0 2024-06-10T01:10:31.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.355+0000 7f453c305b80 1 client.17883 shutdown 2024-06-10T01:10:31.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.355+0000 7f453c305b80 1 -- 172.21.15.37:0/537865651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a0b760460 msgr2=0x564a0b7726d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:31.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.355+0000 7f453c305b80 1 --2- 172.21.15.37:0/537865651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a0b760460 0x564a0b7726d0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f451c003fb0 tx=0x7f451c004b40 comp rx=0 tx=0).stop 2024-06-10T01:10:31.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.355+0000 7f453c305b80 1 -- 172.21.15.37:0/537865651 shutdown_connections 2024-06-10T01:10:31.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.356+0000 7f453c305b80 1 --2- 172.21.15.37:0/537865651 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x564a0b7e5e80 0x564a0b7e8300 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.356+0000 7f453c305b80 1 --2- 172.21.15.37:0/537865651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a0b7665f0 0x564a0b76f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.356+0000 7f453c305b80 1 --2- 172.21.15.37:0/537865651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a0b760460 0x564a0b7726d0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.356+0000 7f450affd700 0 client.17883 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:10:31.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.356+0000 7f453c305b80 1 --2- 172.21.15.37:0/537865651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564a0b765cd0 0x564a0b76f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.356+0000 7f453c305b80 1 -- 172.21.15.37:0/537865651 >> 172.21.15.37:0/537865651 conn(0x564a0b68ae50 msgr2=0x564a0b7669d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:31.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.356+0000 7f453c305b80 1 -- 172.21.15.37:0/537865651 shutdown_connections 2024-06-10T01:10:31.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.356+0000 7f453c305b80 1 -- 172.21.15.37:0/537865651 wait complete. 2024-06-10T01:10:31.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.356+0000 7f453c305b80 20 client.17883 trim_cache size 0 max 16384 2024-06-10T01:10:31.352 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:10:31.352 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 35, 2024-06-10T01:10:31.352 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24472, 2024-06-10T01:10:31.352 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 38, 2024-06-10T01:10:31.353 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-06-10T01:10:31.353 INFO:teuthology.orchestra.run.smithi037.stdout: "a00f46e9": { 2024-06-10T01:10:31.353 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 38, 2024-06-10T01:10:31.353 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-06-10T01:10:31.353 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:10:31.353 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-06-10T01:10:31.353 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:10:31.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:31.353 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-06-10T01:10:31.353 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 147.2, 2024-06-10T01:10:31.353 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-06-10T01:10:31.353 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-06-10T01:10:31.353 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-06-10T01:10:31.353 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-06-10T01:10:31.353 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-06-10T01:10:31.354 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-06-10T01:10:31.354 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:31.354 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:31.354 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:31.354 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:31.354 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:31.354 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:10:31.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.356+0000 7f4531a82700 1 -- 172.21.15.37:0/4024021746 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f45180a2cc0 msgr2=0x7f45180a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:31.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.356+0000 7f4531a82700 1 --2- 172.21.15.37:0/4024021746 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f45180a2cc0 0x7f45180a5180 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f451c009550 tx=0x7f451c0091b0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.356+0000 7f4531a82700 1 -- 172.21.15.37:0/4024021746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f452c1066c0 msgr2=0x7f452c198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:31.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.356+0000 7f4531a82700 1 --2- 172.21.15.37:0/4024021746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f452c1066c0 0x7f452c198db0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f4524006010 tx=0x7f452400fa00 comp rx=0 tx=0).stop 2024-06-10T01:10:31.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.356+0000 7f4531a82700 1 -- 172.21.15.37:0/4024021746 shutdown_connections 2024-06-10T01:10:31.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.356+0000 7f4531a82700 1 --2- 172.21.15.37:0/4024021746 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f45180a2cc0 0x7f45180a5180 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.356+0000 7f4531a82700 1 --2- 172.21.15.37:0/4024021746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f452c1066c0 0x7f452c198db0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.356+0000 7f4531a82700 1 --2- 172.21.15.37:0/4024021746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f452c1015f0 0x7f452c19e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.356+0000 7f4531a82700 1 --2- 172.21.15.37:0/4024021746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f452c101fa0 0x7f452c19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:31.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.356+0000 7f4531a82700 1 -- 172.21.15.37:0/4024021746 >> 172.21.15.37:0/4024021746 conn(0x7f452c078330 msgr2=0x7f452c0fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:31.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.357+0000 7f4531a82700 1 -- 172.21.15.37:0/4024021746 shutdown_connections 2024-06-10T01:10:31.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:31.357+0000 7f4531a82700 1 -- 172.21.15.37:0/4024021746 wait complete. 2024-06-10T01:10:31.537 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-10T01:10:31.897 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi037 with status 11: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b7a5d375cf4e4afab96b9d4d229b699ae8304bb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/multiple_rsync.sh' 2024-06-10T01:10:32.112 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=20d7eaa86a65426c83a27ddfa1f5d0ab Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi037 with status 11: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b7a5d375cf4e4afab96b9d4d229b699ae8304bb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/multiple_rsync.sh' 2024-06-10T01:10:32.115 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-06-10T01:10:32.125 INFO:tasks.quiescer:joining Quiescers 2024-06-10T01:10:32.125 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-06-10T01:10:32.125 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:10:32.131 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:10:32.139 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a00f46e9', quiesce took 6.5 seconds. Will release after: 20.7 2024-06-10T01:10:32.500 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:32 smithi037 ceph-mon[32192]: pgmap v850: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:10:32.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:32 smithi167 ceph-mon[36917]: pgmap v850: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:10:32.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:32 smithi094 ceph-mon[36708]: pgmap v850: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:10:33.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.921+0000 7fc1949f9700 1 -- 172.21.15.37:0/2892168334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc190101fc0 msgr2=0x7fc190106010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:33.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.921+0000 7fc1949f9700 1 --2- 172.21.15.37:0/2892168334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc190101fc0 0x7fc190106010 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fc1840099c0 tx=0x7fc184009cd0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.922+0000 7fc1949f9700 1 -- 172.21.15.37:0/2892168334 shutdown_connections 2024-06-10T01:10:33.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.922+0000 7fc1949f9700 1 --2- 172.21.15.37:0/2892168334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1901066f0 0x7fc19010e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.922+0000 7fc1949f9700 1 --2- 172.21.15.37:0/2892168334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc190101610 0x7fc1901019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.922+0000 7fc1949f9700 1 --2- 172.21.15.37:0/2892168334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc190101fc0 0x7fc190106010 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.922+0000 7fc1949f9700 1 -- 172.21.15.37:0/2892168334 >> 172.21.15.37:0/2892168334 conn(0x7fc190078430 msgr2=0x7fc190078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:33.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.922+0000 7fc1949f9700 1 -- 172.21.15.37:0/2892168334 shutdown_connections 2024-06-10T01:10:33.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.922+0000 7fc1949f9700 1 -- 172.21.15.37:0/2892168334 wait complete. 2024-06-10T01:10:33.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.922+0000 7fc1949f9700 1 Processor -- start 2024-06-10T01:10:33.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.923+0000 7fc1949f9700 1 -- start start 2024-06-10T01:10:33.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.923+0000 7fc1949f9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc190101610 0x7fc19010eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:33.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.923+0000 7fc1949f9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc190101fc0 0x7fc19010f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:33.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.923+0000 7fc1949f9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1901066f0 0x7fc19010f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:33.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.923+0000 7fc1949f9700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc190112830 con 0x7fc190101610 2024-06-10T01:10:33.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.923+0000 7fc1949f9700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901126b0 con 0x7fc190101fc0 2024-06-10T01:10:33.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.923+0000 7fc1949f9700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901129b0 con 0x7fc1901066f0 2024-06-10T01:10:33.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.923+0000 7fc18e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc190101610 0x7fc19010eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:33.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.923+0000 7fc18e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc190101610 0x7fc19010eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34950/0 (socket says 172.21.15.37:34950) 2024-06-10T01:10:33.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.923+0000 7fc18e59c700 1 -- 172.21.15.37:0/4246422584 learned_addr learned my addr 172.21.15.37:0/4246422584 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:33.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.923+0000 7fc18dd9b700 1 --2- 172.21.15.37:0/4246422584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc190101fc0 0x7fc19010f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:33.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.923+0000 7fc18e59c700 1 -- 172.21.15.37:0/4246422584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1901066f0 msgr2=0x7fc19010f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:33.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.923+0000 7fc18ed9d700 1 --2- 172.21.15.37:0/4246422584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1901066f0 0x7fc19010f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:33.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.923+0000 7fc18e59c700 1 --2- 172.21.15.37:0/4246422584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1901066f0 0x7fc19010f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.923+0000 7fc18e59c700 1 -- 172.21.15.37:0/4246422584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc190101fc0 msgr2=0x7fc19010f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:33.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.923+0000 7fc18e59c700 1 --2- 172.21.15.37:0/4246422584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc190101fc0 0x7fc19010f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.923+0000 7fc18e59c700 1 -- 172.21.15.37:0/4246422584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1901a70b0 con 0x7fc190101610 2024-06-10T01:10:33.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.923+0000 7fc18dd9b700 1 --2- 172.21.15.37:0/4246422584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc190101fc0 0x7fc19010f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:10:33.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.923+0000 7fc18ed9d700 1 --2- 172.21.15.37:0/4246422584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1901066f0 0x7fc19010f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:10:33.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.924+0000 7fc18e59c700 1 --2- 172.21.15.37:0/4246422584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc190101610 0x7fc19010eca0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc18000c9b0 tx=0x7fc18000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:33.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.924+0000 7fc17f7fe700 1 -- 172.21.15.37:0/4246422584 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc180007b00 con 0x7fc190101610 2024-06-10T01:10:33.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.924+0000 7fc1949f9700 1 -- 172.21.15.37:0/4246422584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1901a73a0 con 0x7fc190101610 2024-06-10T01:10:33.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.924+0000 7fc17f7fe700 1 -- 172.21.15.37:0/4246422584 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc180007ca0 con 0x7fc190101610 2024-06-10T01:10:33.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.924+0000 7fc17f7fe700 1 -- 172.21.15.37:0/4246422584 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc180018780 con 0x7fc190101610 2024-06-10T01:10:33.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.924+0000 7fc1949f9700 1 -- 172.21.15.37:0/4246422584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1901a7900 con 0x7fc190101610 2024-06-10T01:10:33.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.926+0000 7fc17f7fe700 1 -- 172.21.15.37:0/4246422584 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc18000f450 con 0x7fc190101610 2024-06-10T01:10:33.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.927+0000 7fc17f7fe700 1 --2- 172.21.15.37:0/4246422584 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc1780a2c20 0x7fc1780a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:33.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.927+0000 7fc18dd9b700 1 --2- 172.21.15.37:0/4246422584 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc1780a2c20 0x7fc1780a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:33.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.927+0000 7fc17f7fe700 1 -- 172.21.15.37:0/4246422584 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7fc1800cf4a0 con 0x7fc190101610 2024-06-10T01:10:33.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.927+0000 7fc18dd9b700 1 --2- 172.21.15.37:0/4246422584 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc1780a2c20 0x7fc1780a50e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc184000c00 tx=0x7fc184005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:33.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.927+0000 7fc19f27cb80 1 Processor -- start 2024-06-10T01:10:33.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.928+0000 7fc19f27cb80 1 -- start start 2024-06-10T01:10:33.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.928+0000 7fc19f27cb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aaa490ff60 0x55aaa4910340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:33.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.928+0000 7fc19f27cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aaa4910880 0x55aaa490a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:33.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.928+0000 7fc19f27cb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa490a7a0 0x55aaa490abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:33.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.928+0000 7fc19f27cb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55aaa47fcf60 con 0x55aaa490ff60 2024-06-10T01:10:33.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.928+0000 7fc19f27cb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55aaa485b850 con 0x55aaa490a7a0 2024-06-10T01:10:33.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.928+0000 7fc19f27cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55aaa47c3460 con 0x55aaa4910880 2024-06-10T01:10:33.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.928+0000 7fc18e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aaa490ff60 0x55aaa4910340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:33.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.928+0000 7fc18e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aaa490ff60 0x55aaa4910340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34954/0 (socket says 172.21.15.37:34954) 2024-06-10T01:10:33.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.928+0000 7fc18e59c700 1 -- 172.21.15.37:0/144541009 learned_addr learned my addr 172.21.15.37:0/144541009 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:33.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.928+0000 7fc18dd9b700 1 --2- 172.21.15.37:0/144541009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aaa4910880 0x55aaa490a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:33.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.928+0000 7fc18ed9d700 1 --2- 172.21.15.37:0/144541009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa490a7a0 0x55aaa490abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:33.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.928+0000 7fc18e59c700 1 -- 172.21.15.37:0/144541009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aaa4910880 msgr2=0x55aaa490a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:33.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.928+0000 7fc18e59c700 1 --2- 172.21.15.37:0/144541009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aaa4910880 0x55aaa490a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.928+0000 7fc18e59c700 1 -- 172.21.15.37:0/144541009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa490a7a0 msgr2=0x55aaa490abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:33.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.928+0000 7fc18e59c700 1 --2- 172.21.15.37:0/144541009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa490a7a0 0x55aaa490abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.928+0000 7fc18e59c700 1 -- 172.21.15.37:0/144541009 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaa47aa8d0 con 0x55aaa490ff60 2024-06-10T01:10:33.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.929+0000 7fc18e59c700 1 --2- 172.21.15.37:0/144541009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aaa490ff60 0x55aaa4910340 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc1800a4050 tx=0x7fc18009c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54344eef1b847b0e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:33.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.929+0000 7fc17cff9700 1 -- 172.21.15.37:0/144541009 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1800a4580 con 0x55aaa490ff60 2024-06-10T01:10:33.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.929+0000 7fc17cff9700 1 -- 172.21.15.37:0/144541009 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc18000dda0 con 0x55aaa490ff60 2024-06-10T01:10:33.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.929+0000 7fc17cff9700 1 -- 172.21.15.37:0/144541009 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18009bb70 con 0x55aaa490ff60 2024-06-10T01:10:33.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.929+0000 7fc19f27cb80 1 -- 172.21.15.37:0/144541009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aaa490ff60 msgr2=0x55aaa4910340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:33.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.929+0000 7fc19f27cb80 1 --2- 172.21.15.37:0/144541009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aaa490ff60 0x55aaa4910340 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc1800a4050 tx=0x7fc18009c010 comp rx=0 tx=0).stop 2024-06-10T01:10:33.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.930+0000 7fc19f27cb80 1 -- 172.21.15.37:0/144541009 shutdown_connections 2024-06-10T01:10:33.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.930+0000 7fc19f27cb80 1 --2- 172.21.15.37:0/144541009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa490a7a0 0x55aaa490abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.930+0000 7fc19f27cb80 1 --2- 172.21.15.37:0/144541009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aaa490ff60 0x55aaa4910340 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.930+0000 7fc19f27cb80 1 --2- 172.21.15.37:0/144541009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aaa4910880 0x55aaa490a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.930+0000 7fc19f27cb80 1 -- 172.21.15.37:0/144541009 >> 172.21.15.37:0/144541009 conn(0x55aaa4837080 msgr2=0x55aaa4905220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:33.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.930+0000 7fc19f27cb80 1 -- 172.21.15.37:0/144541009 shutdown_connections 2024-06-10T01:10:33.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.930+0000 7fc19f27cb80 1 -- 172.21.15.37:0/144541009 wait complete. 2024-06-10T01:10:33.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.930+0000 7fc19f27cb80 1 -- start start 2024-06-10T01:10:33.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.930+0000 7fc19f27cb80 1 Processor -- start 2024-06-10T01:10:33.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.930+0000 7fc19f27cb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aaa490a7a0 0x55aaa49165f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:33.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.930+0000 7fc19f27cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aaa490ff60 0x55aaa49138f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:33.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.930+0000 7fc19f27cb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa4910880 0x55aaa4913e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:33.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.930+0000 7fc19f27cb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55aaa47fbfc0 con 0x55aaa490a7a0 2024-06-10T01:10:33.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.930+0000 7fc19f27cb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55aaa4857810 con 0x55aaa4910880 2024-06-10T01:10:33.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.930+0000 7fc19f27cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55aaa4917260 con 0x55aaa490ff60 2024-06-10T01:10:33.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.930+0000 7fc19f27cb80 10 client.0 _ll_register_callbacks cb 0x55aaa47f15b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:33.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.931+0000 7fc18e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aaa490a7a0 0x55aaa49165f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:33.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.931+0000 7fc18e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aaa490a7a0 0x55aaa49165f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34968/0 (socket says 172.21.15.37:34968) 2024-06-10T01:10:33.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.931+0000 7fc18e59c700 1 -- 172.21.15.37:0/1831741307 learned_addr learned my addr 172.21.15.37:0/1831741307 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:33.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.931+0000 7fc18dd9b700 1 --2- 172.21.15.37:0/1831741307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aaa490ff60 0x55aaa49138f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:33.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.931+0000 7fc18ed9d700 1 --2- 172.21.15.37:0/1831741307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa4910880 0x55aaa4913e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:33.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.931+0000 7fc18e59c700 1 -- 172.21.15.37:0/1831741307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aaa490ff60 msgr2=0x55aaa49138f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:33.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.931+0000 7fc18e59c700 1 --2- 172.21.15.37:0/1831741307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aaa490ff60 0x55aaa49138f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.931+0000 7fc18e59c700 1 -- 172.21.15.37:0/1831741307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa4910880 msgr2=0x55aaa4913e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:33.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.931+0000 7fc18e59c700 1 --2- 172.21.15.37:0/1831741307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa4910880 0x55aaa4913e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.931+0000 7fc18e59c700 1 -- 172.21.15.37:0/1831741307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaa4821750 con 0x55aaa490a7a0 2024-06-10T01:10:33.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.931+0000 7fc18dd9b700 1 --2- 172.21.15.37:0/1831741307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aaa490ff60 0x55aaa49138f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:10:33.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.931+0000 7fc18ed9d700 1 --2- 172.21.15.37:0/1831741307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa4910880 0x55aaa4913e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:10:33.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.931+0000 7fc18e59c700 1 --2- 172.21.15.37:0/1831741307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aaa490a7a0 0x55aaa49165f0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc180097d70 tx=0x7fc1800a4630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6b9d4ec69b99c7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:33.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.931+0000 7fc175ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:10:33.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.931+0000 7fc175ffb700 1 -- 172.21.15.37:0/1831741307 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc180060e60 con 0x55aaa490a7a0 2024-06-10T01:10:33.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.931+0000 7fc175ffb700 1 -- 172.21.15.37:0/1831741307 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7fc18000fa30 con 0x55aaa490a7a0 2024-06-10T01:10:33.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.931+0000 7fc19f27cb80 1 -- 172.21.15.37:0/1831741307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc18000dda0 con 0x55aaa490a7a0 2024-06-10T01:10:33.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.931+0000 7fc175ffb700 1 -- 172.21.15.37:0/1831741307 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc1800b9070 con 0x55aaa490a7a0 2024-06-10T01:10:33.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.931+0000 7fc19f27cb80 10 client.17895 fetch_fsmap learned FSMap version 35 2024-06-10T01:10:33.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.931+0000 7fc19f27cb80 1 -- 172.21.15.37:0/1831741307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55aaa4921a10 con 0x55aaa490a7a0 2024-06-10T01:10:33.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.932+0000 7fc175ffb700 1 -- 172.21.15.37:0/1831741307 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc18000dda0 con 0x55aaa490a7a0 2024-06-10T01:10:33.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.932+0000 7fc175ffb700 1 -- 172.21.15.37:0/1831741307 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc1800860c0 con 0x55aaa490a7a0 2024-06-10T01:10:33.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.932+0000 7fc19f27cb80 10 client.17895 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:10:33.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.932+0000 7fc19f27cb80 10 client.17895 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:10:33.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.932+0000 7fc19f27cb80 20 client.17895 populate_metadata read hostname 'smithi037' 2024-06-10T01:10:33.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.932+0000 7fc19f27cb80 1 --2- 172.21.15.37:0/1831741307 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55aaa4923d10 0x55aaa4926100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:33.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.932+0000 7fc19f27cb80 4 client.17895 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:10:33.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.932+0000 7fc19f27cb80 1 -- 172.21.15.37:0/1831741307 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aaa4868d50 con 0x55aaa4923d10 2024-06-10T01:10:33.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.932+0000 7fc18dd9b700 1 --2- 172.21.15.37:0/1831741307 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55aaa4923d10 0x55aaa4926100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:33.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.933+0000 7fc175ffb700 10 client.17895 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:10:33.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.933+0000 7fc18dd9b700 1 --2- 172.21.15.37:0/1831741307 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55aaa4923d10 0x55aaa4926100 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee019af49ec5ae72 server_cookie=82f2cf069a227d3e in_seq=0 out_seq=0 2024-06-10T01:10:33.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.933+0000 7fc175ffb700 1 -- 172.21.15.37:0/1831741307 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55aaa4868d50 con 0x55aaa4923d10 2024-06-10T01:10:33.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.933+0000 7fc175ffb700 10 client.17895 handle_command_reply: tid=0 2024-06-10T01:10:33.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.933+0000 7fc19f27cb80 1 client.17895 shutdown 2024-06-10T01:10:33.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.934+0000 7fc19f27cb80 1 -- 172.21.15.37:0/1831741307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aaa490a7a0 msgr2=0x55aaa49165f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:33.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.934+0000 7fc19f27cb80 1 --2- 172.21.15.37:0/1831741307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aaa490a7a0 0x55aaa49165f0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc180097d70 tx=0x7fc1800a4630 comp rx=0 tx=0).stop 2024-06-10T01:10:33.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.934+0000 7fc19f27cb80 1 -- 172.21.15.37:0/1831741307 shutdown_connections 2024-06-10T01:10:33.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.934+0000 7fc19f27cb80 1 --2- 172.21.15.37:0/1831741307 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55aaa4923d10 0x55aaa4926100 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.934+0000 7fc19f27cb80 1 --2- 172.21.15.37:0/1831741307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa4910880 0x55aaa4913e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.934+0000 7fc19f27cb80 1 --2- 172.21.15.37:0/1831741307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aaa490a7a0 0x55aaa49165f0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.934+0000 7fc19f27cb80 1 --2- 172.21.15.37:0/1831741307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aaa490ff60 0x55aaa49138f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.934+0000 7fc175ffb700 0 client.17895 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:10:33.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.934+0000 7fc19f27cb80 1 -- 172.21.15.37:0/1831741307 >> 172.21.15.37:0/1831741307 conn(0x55aaa4837080 msgr2=0x55aaa4904700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:33.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.934+0000 7fc19f27cb80 1 -- 172.21.15.37:0/1831741307 shutdown_connections 2024-06-10T01:10:33.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.934+0000 7fc19f27cb80 1 -- 172.21.15.37:0/1831741307 wait complete. 2024-06-10T01:10:33.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.934+0000 7fc19f27cb80 20 client.17895 trim_cache size 0 max 16384 2024-06-10T01:10:33.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.954+0000 7fc19f27cb80 1 Processor -- start 2024-06-10T01:10:33.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.954+0000 7fc19f27cb80 1 -- start start 2024-06-10T01:10:33.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.955+0000 7fc19f27cb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa490a7a0 0x55aaa490a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:33.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.955+0000 7fc19f27cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aaa490ff60 0x55aaa4916130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:33.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.955+0000 7fc19f27cb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aaa4910880 0x55aaa4904e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:33.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.955+0000 7fc19f27cb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901b2e60 con 0x55aaa4910880 2024-06-10T01:10:33.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.955+0000 7fc19f27cb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901b06f0 con 0x55aaa490a7a0 2024-06-10T01:10:33.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.955+0000 7fc19f27cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901ae540 con 0x55aaa490ff60 2024-06-10T01:10:33.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.955+0000 7fc18ed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aaa4910880 0x55aaa4904e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:33.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.955+0000 7fc18ed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aaa4910880 0x55aaa4904e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34974/0 (socket says 172.21.15.37:34974) 2024-06-10T01:10:33.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.955+0000 7fc18ed9d700 1 -- 172.21.15.37:0/1879502960 learned_addr learned my addr 172.21.15.37:0/1879502960 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:33.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.955+0000 7fc18e59c700 1 --2- 172.21.15.37:0/1879502960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa490a7a0 0x55aaa490a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:33.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.955+0000 7fc18dd9b700 1 --2- 172.21.15.37:0/1879502960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aaa490ff60 0x55aaa4916130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:33.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.955+0000 7fc18ed9d700 1 -- 172.21.15.37:0/1879502960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aaa490ff60 msgr2=0x55aaa4916130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:33.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.955+0000 7fc18ed9d700 1 --2- 172.21.15.37:0/1879502960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aaa490ff60 0x55aaa4916130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.955+0000 7fc18ed9d700 1 -- 172.21.15.37:0/1879502960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa490a7a0 msgr2=0x55aaa490a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:33.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.955+0000 7fc18ed9d700 1 --2- 172.21.15.37:0/1879502960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa490a7a0 0x55aaa490a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.955+0000 7fc18ed9d700 1 -- 172.21.15.37:0/1879502960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaa4911e80 con 0x55aaa4910880 2024-06-10T01:10:33.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.955+0000 7fc18ed9d700 1 --2- 172.21.15.37:0/1879502960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aaa4910880 0x55aaa4904e80 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc188011400 tx=0x7fc188011710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff854e6513be72b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:33.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.956+0000 7fc177fff700 1 -- 172.21.15.37:0/1879502960 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188019070 con 0x55aaa4910880 2024-06-10T01:10:33.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.956+0000 7fc177fff700 1 -- 172.21.15.37:0/1879502960 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc188011e80 con 0x55aaa4910880 2024-06-10T01:10:33.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.956+0000 7fc177fff700 1 -- 172.21.15.37:0/1879502960 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188014870 con 0x55aaa4910880 2024-06-10T01:10:33.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.956+0000 7fc19f27cb80 1 -- 172.21.15.37:0/1879502960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aaa4910880 msgr2=0x55aaa4904e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:33.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.956+0000 7fc19f27cb80 1 --2- 172.21.15.37:0/1879502960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aaa4910880 0x55aaa4904e80 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc188011400 tx=0x7fc188011710 comp rx=0 tx=0).stop 2024-06-10T01:10:33.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.956+0000 7fc19f27cb80 1 -- 172.21.15.37:0/1879502960 shutdown_connections 2024-06-10T01:10:33.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.956+0000 7fc19f27cb80 1 --2- 172.21.15.37:0/1879502960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aaa4910880 0x55aaa4904e80 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.956+0000 7fc19f27cb80 1 --2- 172.21.15.37:0/1879502960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa490a7a0 0x55aaa490a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.956+0000 7fc19f27cb80 1 --2- 172.21.15.37:0/1879502960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aaa490ff60 0x55aaa4916130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.956+0000 7fc19f27cb80 1 -- 172.21.15.37:0/1879502960 >> 172.21.15.37:0/1879502960 conn(0x55aaa4837080 msgr2=0x55aaa490ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:33.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.956+0000 7fc19f27cb80 1 -- 172.21.15.37:0/1879502960 shutdown_connections 2024-06-10T01:10:33.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.956+0000 7fc19f27cb80 1 -- 172.21.15.37:0/1879502960 wait complete. 2024-06-10T01:10:33.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.957+0000 7fc19f27cb80 1 -- start start 2024-06-10T01:10:33.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.957+0000 7fc19f27cb80 1 Processor -- start 2024-06-10T01:10:33.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.957+0000 7fc19f27cb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aaa490a7a0 0x55aaa4926b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:33.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.957+0000 7fc19f27cb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa490ff60 0x55aaa4927080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:33.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.957+0000 7fc19f27cb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aaa4910880 0x55aaa49275c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:33.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.957+0000 7fc19f27cb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901aebb0 con 0x55aaa4910880 2024-06-10T01:10:33.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.957+0000 7fc19f27cb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901b0020 con 0x55aaa490ff60 2024-06-10T01:10:33.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.957+0000 7fc19f27cb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901af280 con 0x55aaa490a7a0 2024-06-10T01:10:33.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.957+0000 7fc19f27cb80 10 client.0 _ll_register_callbacks cb 0x55aaa47f15b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:33.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.957+0000 7fc18ed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aaa4910880 0x55aaa49275c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:33.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.957+0000 7fc18ed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aaa4910880 0x55aaa49275c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34986/0 (socket says 172.21.15.37:34986) 2024-06-10T01:10:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.957+0000 7fc18ed9d700 1 -- 172.21.15.37:0/533549762 learned_addr learned my addr 172.21.15.37:0/533549762 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.957+0000 7fc18e59c700 1 --2- 172.21.15.37:0/533549762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aaa490a7a0 0x55aaa4926b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.957+0000 7fc18dd9b700 1 --2- 172.21.15.37:0/533549762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa490ff60 0x55aaa4927080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.958+0000 7fc18ed9d700 1 -- 172.21.15.37:0/533549762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aaa490a7a0 msgr2=0x55aaa4926b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.958+0000 7fc18ed9d700 1 --2- 172.21.15.37:0/533549762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aaa490a7a0 0x55aaa4926b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.958+0000 7fc18ed9d700 1 -- 172.21.15.37:0/533549762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa490ff60 msgr2=0x55aaa4927080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.958+0000 7fc18ed9d700 1 --2- 172.21.15.37:0/533549762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa490ff60 0x55aaa4927080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.958+0000 7fc18ed9d700 1 -- 172.21.15.37:0/533549762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaa498d6a0 con 0x55aaa4910880 2024-06-10T01:10:33.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.958+0000 7fc18ed9d700 1 --2- 172.21.15.37:0/533549762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aaa4910880 0x55aaa49275c0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fc188015bf0 tx=0x7fc188015c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43817dff46e9348d server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:33.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.958+0000 7fc175ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:10:33.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.958+0000 7fc175ffb700 1 -- 172.21.15.37:0/533549762 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18802c060 con 0x55aaa4910880 2024-06-10T01:10:33.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.958+0000 7fc19f27cb80 1 -- 172.21.15.37:0/533549762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55aaa48632d0 con 0x55aaa4910880 2024-06-10T01:10:33.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.958+0000 7fc175ffb700 1 -- 172.21.15.37:0/533549762 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7fc188014920 con 0x55aaa4910880 2024-06-10T01:10:33.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.958+0000 7fc175ffb700 1 -- 172.21.15.37:0/533549762 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc188033050 con 0x55aaa4910880 2024-06-10T01:10:33.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.958+0000 7fc19f27cb80 10 client.17901 fetch_fsmap learned FSMap version 35 2024-06-10T01:10:33.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.958+0000 7fc19f27cb80 1 -- 172.21.15.37:0/533549762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55aaa498e990 con 0x55aaa4910880 2024-06-10T01:10:33.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.958+0000 7fc175ffb700 1 -- 172.21.15.37:0/533549762 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55aaa48632d0 con 0x55aaa4910880 2024-06-10T01:10:33.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.959+0000 7fc175ffb700 1 -- 172.21.15.37:0/533549762 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc188026030 con 0x55aaa4910880 2024-06-10T01:10:33.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.959+0000 7fc19f27cb80 10 client.17901 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:10:33.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.959+0000 7fc19f27cb80 10 client.17901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:10:33.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.959+0000 7fc19f27cb80 20 client.17901 populate_metadata read hostname 'smithi037' 2024-06-10T01:10:33.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.959+0000 7fc19f27cb80 1 --2- 172.21.15.37:0/533549762 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55aaa498efb0 0x55aaa49913d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:33.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.959+0000 7fc19f27cb80 4 client.17901 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-10T01:10:33.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.959+0000 7fc19f27cb80 1 -- 172.21.15.37:0/533549762 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55aaa4816f50 con 0x55aaa498efb0 2024-06-10T01:10:33.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.959+0000 7fc18e59c700 1 --2- 172.21.15.37:0/533549762 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55aaa498efb0 0x55aaa49913d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:33.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.959+0000 7fc18e59c700 1 --2- 172.21.15.37:0/533549762 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55aaa498efb0 0x55aaa49913d0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=745c78bc74851b97 server_cookie=46043ab867466f9a in_seq=0 out_seq=0 2024-06-10T01:10:33.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.959+0000 7fc175ffb700 10 client.17901 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:10:33.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.960+0000 7fc175ffb700 1 -- 172.21.15.37:0/533549762 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55aaa4816f50 con 0x55aaa498efb0 2024-06-10T01:10:33.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.960+0000 7fc175ffb700 10 client.17901 handle_command_reply: tid=0 2024-06-10T01:10:33.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.960+0000 7fc19f27cb80 1 client.17901 shutdown 2024-06-10T01:10:33.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.960+0000 7fc19f27cb80 1 -- 172.21.15.37:0/533549762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aaa4910880 msgr2=0x55aaa49275c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:33.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.960+0000 7fc19f27cb80 1 --2- 172.21.15.37:0/533549762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aaa4910880 0x55aaa49275c0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fc188015bf0 tx=0x7fc188015c20 comp rx=0 tx=0).stop 2024-06-10T01:10:33.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.960+0000 7fc19f27cb80 1 -- 172.21.15.37:0/533549762 shutdown_connections 2024-06-10T01:10:33.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.960+0000 7fc19f27cb80 1 --2- 172.21.15.37:0/533549762 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55aaa498efb0 0x55aaa49913d0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.960+0000 7fc19f27cb80 1 --2- 172.21.15.37:0/533549762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aaa4910880 0x55aaa49275c0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.960+0000 7fc19f27cb80 1 --2- 172.21.15.37:0/533549762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aaa490a7a0 0x55aaa4926b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.960+0000 7fc175ffb700 0 client.17901 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:10:33.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.960+0000 7fc19f27cb80 1 --2- 172.21.15.37:0/533549762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa490ff60 0x55aaa4927080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.960+0000 7fc19f27cb80 1 -- 172.21.15.37:0/533549762 >> 172.21.15.37:0/533549762 conn(0x55aaa4837080 msgr2=0x55aaa490f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:33.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.960+0000 7fc19f27cb80 1 -- 172.21.15.37:0/533549762 shutdown_connections 2024-06-10T01:10:33.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.960+0000 7fc19f27cb80 1 -- 172.21.15.37:0/533549762 wait complete. 2024-06-10T01:10:33.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.960+0000 7fc19f27cb80 20 client.17901 trim_cache size 0 max 16384 2024-06-10T01:10:33.965 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:10:33.965 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-06-10T01:10:33.965 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-06-10T01:10:33.966 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:10:33.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.960+0000 7fc1949f9700 1 -- 172.21.15.37:0/4246422584 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc1780a2c20 msgr2=0x7fc1780a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:33.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.960+0000 7fc1949f9700 1 --2- 172.21.15.37:0/4246422584 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc1780a2c20 0x7fc1780a50e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc184000c00 tx=0x7fc184005fd0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.960+0000 7fc1949f9700 1 -- 172.21.15.37:0/4246422584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc190101610 msgr2=0x7fc19010eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:33.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.960+0000 7fc1949f9700 1 --2- 172.21.15.37:0/4246422584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc190101610 0x7fc19010eca0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc18000c9b0 tx=0x7fc18000cd70 comp rx=0 tx=0).stop 2024-06-10T01:10:33.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.961+0000 7fc1949f9700 1 -- 172.21.15.37:0/4246422584 shutdown_connections 2024-06-10T01:10:33.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.961+0000 7fc1949f9700 1 --2- 172.21.15.37:0/4246422584 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fc1780a2c20 0x7fc1780a50e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.961+0000 7fc1949f9700 1 --2- 172.21.15.37:0/4246422584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1901066f0 0x7fc19010f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.961+0000 7fc1949f9700 1 --2- 172.21.15.37:0/4246422584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc190101610 0x7fc19010eca0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.961+0000 7fc1949f9700 1 --2- 172.21.15.37:0/4246422584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc190101fc0 0x7fc19010f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:33.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.961+0000 7fc1949f9700 1 -- 172.21.15.37:0/4246422584 >> 172.21.15.37:0/4246422584 conn(0x7fc190078430 msgr2=0x7fc1900ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:33.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.961+0000 7fc1949f9700 1 -- 172.21.15.37:0/4246422584 shutdown_connections 2024-06-10T01:10:33.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:33.961+0000 7fc1949f9700 1 -- 172.21.15.37:0/4246422584 wait complete. 2024-06-10T01:10:34.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:34 smithi167 ceph-mon[36917]: pgmap v851: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:10:34.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:34 smithi094 ceph-mon[36708]: pgmap v851: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:10:34.616 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:34 smithi037 ceph-mon[32192]: pgmap v851: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:10:34.645 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-10T01:10:34.645 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:10:34.706 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:10:35.463 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:10:36.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:36 smithi167 ceph-mon[36917]: pgmap v852: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:10:36.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:36 smithi094 ceph-mon[36708]: pgmap v852: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:10:36.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:36 smithi037 ceph-mon[32192]: pgmap v852: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:10:37.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.133+0000 7f2accdf8700 1 -- 172.21.15.37:0/972665306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ac8101390 msgr2=0x7f2ac8101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:37.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.133+0000 7f2accdf8700 1 --2- 172.21.15.37:0/972665306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ac8101390 0x7f2ac8101810 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f2abc009990 tx=0x7f2abc009ca0 comp rx=0 tx=0).stop 2024-06-10T01:10:37.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.133+0000 7f2accdf8700 1 -- 172.21.15.37:0/972665306 shutdown_connections 2024-06-10T01:10:37.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.133+0000 7f2accdf8700 1 --2- 172.21.15.37:0/972665306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ac8101d50 0x7f2ac810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:37.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.133+0000 7f2accdf8700 1 --2- 172.21.15.37:0/972665306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ac8107400 0x7f2ac81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:37.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.133+0000 7f2accdf8700 1 --2- 172.21.15.37:0/972665306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ac8101390 0x7f2ac8101810 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:37.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.133+0000 7f2accdf8700 1 -- 172.21.15.37:0/972665306 >> 172.21.15.37:0/972665306 conn(0x7f2ac8078340 msgr2=0x7f2ac8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:37.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.133+0000 7f2accdf8700 1 -- 172.21.15.37:0/972665306 shutdown_connections 2024-06-10T01:10:37.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.133+0000 7f2accdf8700 1 -- 172.21.15.37:0/972665306 wait complete. 2024-06-10T01:10:37.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.134+0000 7f2accdf8700 1 Processor -- start 2024-06-10T01:10:37.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.134+0000 7f2accdf8700 1 -- start start 2024-06-10T01:10:37.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.134+0000 7f2accdf8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ac8101390 0x7f2ac8199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:37.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.135+0000 7f2accdf8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ac8101d50 0x7f2ac819a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:37.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.135+0000 7f2accdf8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ac8107400 0x7f2ac81a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:37.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.135+0000 7f2accdf8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac8111690 con 0x7f2ac8107400 2024-06-10T01:10:37.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.135+0000 7f2accdf8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac8111510 con 0x7f2ac8101d50 2024-06-10T01:10:37.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.135+0000 7f2accdf8700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac8111810 con 0x7f2ac8101390 2024-06-10T01:10:37.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.135+0000 7f2ac6d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ac8107400 0x7f2ac81a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:37.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.135+0000 7f2ac6d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ac8107400 0x7f2ac81a4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35010/0 (socket says 172.21.15.37:35010) 2024-06-10T01:10:37.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.135+0000 7f2ac6d9d700 1 -- 172.21.15.37:0/3042662083 learned_addr learned my addr 172.21.15.37:0/3042662083 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:37.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.135+0000 7f2ac659c700 1 --2- 172.21.15.37:0/3042662083 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ac8101390 0x7f2ac8199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:37.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.135+0000 7f2ac5d9b700 1 --2- 172.21.15.37:0/3042662083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ac8101d50 0x7f2ac819a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:37.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.135+0000 7f2ac6d9d700 1 -- 172.21.15.37:0/3042662083 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ac8101390 msgr2=0x7f2ac8199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:37.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.135+0000 7f2ac6d9d700 1 --2- 172.21.15.37:0/3042662083 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ac8101390 0x7f2ac8199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:37.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.135+0000 7f2ac6d9d700 1 -- 172.21.15.37:0/3042662083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ac8101d50 msgr2=0x7f2ac819a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:37.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.135+0000 7f2ac6d9d700 1 --2- 172.21.15.37:0/3042662083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ac8101d50 0x7f2ac819a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:37.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.135+0000 7f2ac6d9d700 1 -- 172.21.15.37:0/3042662083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac81a50d0 con 0x7f2ac8107400 2024-06-10T01:10:37.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.135+0000 7f2ac5d9b700 1 --2- 172.21.15.37:0/3042662083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ac8101d50 0x7f2ac819a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:10:37.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.135+0000 7f2ac6d9d700 1 --2- 172.21.15.37:0/3042662083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ac8107400 0x7f2ac81a4b90 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000d980 tx=0x7f2ac000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:37.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.135+0000 7f2ab77fe700 1 -- 172.21.15.37:0/3042662083 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0016070 con 0x7f2ac8107400 2024-06-10T01:10:37.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.136+0000 7f2accdf8700 1 -- 172.21.15.37:0/3042662083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac81a5300 con 0x7f2ac8107400 2024-06-10T01:10:37.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.136+0000 7f2ab77fe700 1 -- 172.21.15.37:0/3042662083 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ac0004d10 con 0x7f2ac8107400 2024-06-10T01:10:37.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.136+0000 7f2ab77fe700 1 -- 172.21.15.37:0/3042662083 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac00055e0 con 0x7f2ac8107400 2024-06-10T01:10:37.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.136+0000 7f2accdf8700 1 -- 172.21.15.37:0/3042662083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac81a5810 con 0x7f2ac8107400 2024-06-10T01:10:37.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.137+0000 7f2accdf8700 1 -- 172.21.15.37:0/3042662083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ac8069180 con 0x7f2ac8107400 2024-06-10T01:10:37.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.138+0000 7f2ab77fe700 1 -- 172.21.15.37:0/3042662083 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2ac0004750 con 0x7f2ac8107400 2024-06-10T01:10:37.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.138+0000 7f2ab77fe700 1 --2- 172.21.15.37:0/3042662083 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2ab00a2c20 0x7f2ab00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:37.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.139+0000 7f2ac659c700 1 --2- 172.21.15.37:0/3042662083 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2ab00a2c20 0x7f2ab00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:37.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.139+0000 7f2ab77fe700 1 -- 172.21.15.37:0/3042662083 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7f2ac00cf2a0 con 0x7f2ac8107400 2024-06-10T01:10:37.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.139+0000 7f2ac659c700 1 --2- 172.21.15.37:0/3042662083 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2ab00a2c20 0x7f2ab00a50e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8003f10 tx=0x7f2ab8009180 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:37.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.143+0000 7f2ab77fe700 1 -- 172.21.15.37:0/3042662083 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2ac001f020 con 0x7f2ac8107400 2024-06-10T01:10:37.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.396+0000 7f2accdf8700 1 -- 172.21.15.37:0/3042662083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ac802dca0 con 0x7f2ac8107400 2024-06-10T01:10:37.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.397+0000 7f2ab77fe700 1 -- 172.21.15.37:0/3042662083 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11870 (secure 0 0 0) 0x7f2ac009be10 con 0x7f2ac8107400 2024-06-10T01:10:37.392 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:10:37.392 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":26554,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2107277223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2107277223},{"type":"v1","addr":"172.21.15.167:6835","nonce":2107277223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T01:10:10.062065+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":271,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24458,24472,24451]},"id":1}]} 2024-06-10T01:10:37.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.399+0000 7f2accdf8700 1 -- 172.21.15.37:0/3042662083 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2ab00a2c20 msgr2=0x7f2ab00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:37.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.399+0000 7f2accdf8700 1 --2- 172.21.15.37:0/3042662083 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2ab00a2c20 0x7f2ab00a50e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8003f10 tx=0x7f2ab8009180 comp rx=0 tx=0).stop 2024-06-10T01:10:37.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.399+0000 7f2accdf8700 1 -- 172.21.15.37:0/3042662083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ac8107400 msgr2=0x7f2ac81a4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:37.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.399+0000 7f2accdf8700 1 --2- 172.21.15.37:0/3042662083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ac8107400 0x7f2ac81a4b90 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000d980 tx=0x7f2ac000dc90 comp rx=0 tx=0).stop 2024-06-10T01:10:37.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.399+0000 7f2accdf8700 1 -- 172.21.15.37:0/3042662083 shutdown_connections 2024-06-10T01:10:37.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.399+0000 7f2accdf8700 1 --2- 172.21.15.37:0/3042662083 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2ab00a2c20 0x7f2ab00a50e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:37.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.399+0000 7f2accdf8700 1 --2- 172.21.15.37:0/3042662083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ac8107400 0x7f2ac81a4b90 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:37.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.399+0000 7f2accdf8700 1 --2- 172.21.15.37:0/3042662083 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ac8101390 0x7f2ac8199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:37.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.399+0000 7f2accdf8700 1 --2- 172.21.15.37:0/3042662083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ac8101d50 0x7f2ac819a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:37.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.399+0000 7f2accdf8700 1 -- 172.21.15.37:0/3042662083 >> 172.21.15.37:0/3042662083 conn(0x7f2ac8078340 msgr2=0x7f2ac80fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:37.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.399+0000 7f2accdf8700 1 -- 172.21.15.37:0/3042662083 shutdown_connections 2024-06-10T01:10:37.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:37.399+0000 7f2accdf8700 1 -- 172.21.15.37:0/3042662083 wait complete. 2024-06-10T01:10:37.396 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 35 2024-06-10T01:10:38.532 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:38 smithi037 ceph-mon[32192]: pgmap v853: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-06-10T01:10:38.532 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:38 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3042662083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:10:38.561 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24472 quiesce db --cancel --all' 2024-06-10T01:10:38.561 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24472 quiesce db --cancel --all 2024-06-10T01:10:38.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:38 smithi167 ceph-mon[36917]: pgmap v853: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-06-10T01:10:38.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:38 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3042662083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:10:38.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:38 smithi094 ceph-mon[36708]: pgmap v853: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-06-10T01:10:38.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:38 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3042662083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:10:38.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.841+0000 7f2ed0c34700 1 -- 172.21.15.37:0/469908095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ecc101510 msgr2=0x7f2ecc10d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:38.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.841+0000 7f2ed0c34700 1 --2- 172.21.15.37:0/469908095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ecc101510 0x7f2ecc10d9d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f2ec400b0a0 tx=0x7f2ec400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:10:38.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.842+0000 7f2ed0c34700 1 -- 172.21.15.37:0/469908095 shutdown_connections 2024-06-10T01:10:38.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.842+0000 7f2ed0c34700 1 --2- 172.21.15.37:0/469908095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ecc101510 0x7f2ecc10d9d0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:38.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.842+0000 7f2ed0c34700 1 --2- 172.21.15.37:0/469908095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ecc106b90 0x7f2ecc106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:38.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.842+0000 7f2ed0c34700 1 --2- 172.21.15.37:0/469908095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ecc100b50 0x7f2ecc100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:38.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.842+0000 7f2ed0c34700 1 -- 172.21.15.37:0/469908095 >> 172.21.15.37:0/469908095 conn(0x7f2ecc077a00 msgr2=0x7f2ecc0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:38.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.842+0000 7f2ed0c34700 1 -- 172.21.15.37:0/469908095 shutdown_connections 2024-06-10T01:10:38.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.842+0000 7f2ed0c34700 1 -- 172.21.15.37:0/469908095 wait complete. 2024-06-10T01:10:38.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.842+0000 7f2ed0c34700 1 Processor -- start 2024-06-10T01:10:38.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.842+0000 7f2ed0c34700 1 -- start start 2024-06-10T01:10:38.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.843+0000 7f2ed0c34700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ecc100b50 0x7f2ecc19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:38.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.843+0000 7f2ed0c34700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ecc101510 0x7f2ecc19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:38.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.843+0000 7f2ed0c34700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ecc106b90 0x7f2ecc198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:38.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.843+0000 7f2ed0c34700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc110600 con 0x7f2ecc106b90 2024-06-10T01:10:38.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.843+0000 7f2ed0c34700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc110480 con 0x7f2ecc100b50 2024-06-10T01:10:38.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.843+0000 7f2ed0c34700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc110780 con 0x7f2ecc101510 2024-06-10T01:10:38.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.843+0000 7f2ecad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ecc106b90 0x7f2ecc198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:38.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.843+0000 7f2ecad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ecc106b90 0x7f2ecc198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35034/0 (socket says 172.21.15.37:35034) 2024-06-10T01:10:38.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.843+0000 7f2ecad9d700 1 -- 172.21.15.37:0/607937160 learned_addr learned my addr 172.21.15.37:0/607937160 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:38.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.843+0000 7f2ec9d9b700 1 --2- 172.21.15.37:0/607937160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ecc101510 0x7f2ecc19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:38.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.843+0000 7f2eca59c700 1 --2- 172.21.15.37:0/607937160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ecc100b50 0x7f2ecc19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:38.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.843+0000 7f2ecad9d700 1 -- 172.21.15.37:0/607937160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ecc101510 msgr2=0x7f2ecc19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:38.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.843+0000 7f2ecad9d700 1 --2- 172.21.15.37:0/607937160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ecc101510 0x7f2ecc19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:38.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.843+0000 7f2ecad9d700 1 -- 172.21.15.37:0/607937160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ecc100b50 msgr2=0x7f2ecc19e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.843+0000 7f2ecad9d700 1 --2- 172.21.15.37:0/607937160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ecc100b50 0x7f2ecc19e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.843+0000 7f2ecad9d700 1 -- 172.21.15.37:0/607937160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ecc199160 con 0x7f2ecc106b90 2024-06-10T01:10:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.843+0000 7f2eca59c700 1 --2- 172.21.15.37:0/607937160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ecc100b50 0x7f2ecc19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:10:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.844+0000 7f2ecad9d700 1 --2- 172.21.15.37:0/607937160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ecc106b90 0x7f2ecc198a00 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f2ec400d010 tx=0x7f2ec4004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.844+0000 7f2ebb7fe700 1 -- 172.21.15.37:0/607937160 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4021070 con 0x7f2ecc106b90 2024-06-10T01:10:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.844+0000 7f2ed0c34700 1 -- 172.21.15.37:0/607937160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ecc1993f0 con 0x7f2ecc106b90 2024-06-10T01:10:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.844+0000 7f2ed0c34700 1 -- 172.21.15.37:0/607937160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ecc1a5710 con 0x7f2ecc106b90 2024-06-10T01:10:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.844+0000 7f2ebb7fe700 1 -- 172.21.15.37:0/607937160 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ec4007c00 con 0x7f2ecc106b90 2024-06-10T01:10:38.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.844+0000 7f2ebb7fe700 1 -- 172.21.15.37:0/607937160 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec401d450 con 0x7f2ecc106b90 2024-06-10T01:10:38.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.845+0000 7f2ed0c34700 1 -- 172.21.15.37:0/607937160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ecc069180 con 0x7f2ecc106b90 2024-06-10T01:10:38.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.846+0000 7f2ebb7fe700 1 -- 172.21.15.37:0/607937160 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2ec4004000 con 0x7f2ecc106b90 2024-06-10T01:10:38.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.847+0000 7f2ebb7fe700 1 --2- 172.21.15.37:0/607937160 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2eb40a2ca0 0x7f2eb40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:38.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.847+0000 7f2eca59c700 1 --2- 172.21.15.37:0/607937160 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2eb40a2ca0 0x7f2eb40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:38.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.847+0000 7f2ebb7fe700 1 -- 172.21.15.37:0/607937160 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7f2ec40d0420 con 0x7f2ecc106b90 2024-06-10T01:10:38.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.847+0000 7f2eca59c700 1 --2- 172.21.15.37:0/607937160 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2eb40a2ca0 0x7f2eb40a5160 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc005e10 tx=0x7f2ebc005d60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:38.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:38.855+0000 7f2ebb7fe700 1 -- 172.21.15.37:0/607937160 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2ec409d0a0 con 0x7f2ecc106b90 2024-06-10T01:10:38.905 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:10:39.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:39.114+0000 7f2ed0c34700 1 -- 172.21.15.37:0/607937160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ecc02dca0 con 0x7f2ecc106b90 2024-06-10T01:10:39.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:39.115+0000 7f2ebb7fe700 1 -- 172.21.15.37:0/607937160 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11870 (secure 0 0 0) 0x7f2ec4007db0 con 0x7f2ecc106b90 2024-06-10T01:10:39.113 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:10:39.114 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":26554,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2107277223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2107277223},{"type":"v1","addr":"172.21.15.167:6835","nonce":2107277223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T01:10:10.062065+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":271,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24458,24472,24451]},"id":1}]} 2024-06-10T01:10:39.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:39.117+0000 7f2ed0c34700 1 -- 172.21.15.37:0/607937160 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2eb40a2ca0 msgr2=0x7f2eb40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:39.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:39.117+0000 7f2ed0c34700 1 --2- 172.21.15.37:0/607937160 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2eb40a2ca0 0x7f2eb40a5160 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc005e10 tx=0x7f2ebc005d60 comp rx=0 tx=0).stop 2024-06-10T01:10:39.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:39.117+0000 7f2ed0c34700 1 -- 172.21.15.37:0/607937160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ecc106b90 msgr2=0x7f2ecc198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:39.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:39.117+0000 7f2ed0c34700 1 --2- 172.21.15.37:0/607937160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ecc106b90 0x7f2ecc198a00 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f2ec400d010 tx=0x7f2ec4004d50 comp rx=0 tx=0).stop 2024-06-10T01:10:39.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:39.117+0000 7f2ed0c34700 1 -- 172.21.15.37:0/607937160 shutdown_connections 2024-06-10T01:10:39.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:39.117+0000 7f2ed0c34700 1 --2- 172.21.15.37:0/607937160 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f2eb40a2ca0 0x7f2eb40a5160 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:39.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:39.117+0000 7f2ed0c34700 1 --2- 172.21.15.37:0/607937160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ecc106b90 0x7f2ecc198a00 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:39.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:39.117+0000 7f2ed0c34700 1 --2- 172.21.15.37:0/607937160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ecc100b50 0x7f2ecc19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:39.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:39.117+0000 7f2ed0c34700 1 --2- 172.21.15.37:0/607937160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ecc101510 0x7f2ecc19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:39.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:39.118+0000 7f2ed0c34700 1 -- 172.21.15.37:0/607937160 >> 172.21.15.37:0/607937160 conn(0x7f2ecc077a00 msgr2=0x7f2ecc0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:39.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:39.118+0000 7f2ed0c34700 1 -- 172.21.15.37:0/607937160 shutdown_connections 2024-06-10T01:10:39.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:39.118+0000 7f2ed0c34700 1 -- 172.21.15.37:0/607937160 wait complete. 2024-06-10T01:10:39.116 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 35 2024-06-10T01:10:39.390 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:39 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/607937160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:10:39.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:39 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/607937160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:10:39.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:39 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/607937160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:10:40.353 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:40 smithi037 ceph-mon[32192]: pgmap v854: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T01:10:40.388 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-10T01:10:40.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:40 smithi167 ceph-mon[36917]: pgmap v854: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T01:10:40.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:40 smithi094 ceph-mon[36708]: pgmap v854: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T01:10:41.326 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:10:41.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.354+0000 7f403a8f7700 1 -- 172.21.15.37:0/2108221726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f403406b3a0 msgr2=0x7f403410f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:41.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.354+0000 7f403a8f7700 1 --2- 172.21.15.37:0/2108221726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f403406b3a0 0x7f403410f310 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f402c00b0a0 tx=0x7f402c00b3b0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.354+0000 7f403a8f7700 1 -- 172.21.15.37:0/2108221726 shutdown_connections 2024-06-10T01:10:41.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.354+0000 7f403a8f7700 1 --2- 172.21.15.37:0/2108221726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f403406b3a0 0x7f403410f310 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.354+0000 7f403a8f7700 1 --2- 172.21.15.37:0/2108221726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40341084d0 0x7f40341088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.354+0000 7f403a8f7700 1 --2- 172.21.15.37:0/2108221726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f403406aa00 0x7f403406ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.354+0000 7f403a8f7700 1 -- 172.21.15.37:0/2108221726 >> 172.21.15.37:0/2108221726 conn(0x7f4034078330 msgr2=0x7f4034078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:41.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.354+0000 7f403a8f7700 1 -- 172.21.15.37:0/2108221726 shutdown_connections 2024-06-10T01:10:41.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.354+0000 7f403a8f7700 1 -- 172.21.15.37:0/2108221726 wait complete. 2024-06-10T01:10:41.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.355+0000 7f403a8f7700 1 Processor -- start 2024-06-10T01:10:41.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.355+0000 7f403a8f7700 1 -- start start 2024-06-10T01:10:41.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.356+0000 7f403a8f7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f403406aa00 0x7f403419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.356+0000 7f403a8f7700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f403406b3a0 0x7f403419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.356+0000 7f403a8f7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40341084d0 0x7f40341989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.356+0000 7f403a8f7700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4034112720 con 0x7f40341084d0 2024-06-10T01:10:41.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.356+0000 7f403a8f7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f40341125a0 con 0x7f403406aa00 2024-06-10T01:10:41.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.356+0000 7f403a8f7700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f40341128a0 con 0x7f403406b3a0 2024-06-10T01:10:41.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.356+0000 7f4038e94700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40341084d0 0x7f40341989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.356+0000 7f4038e94700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40341084d0 0x7f40341989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35058/0 (socket says 172.21.15.37:35058) 2024-06-10T01:10:41.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.356+0000 7f4038e94700 1 -- 172.21.15.37:0/2097545722 learned_addr learned my addr 172.21.15.37:0/2097545722 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:41.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.356+0000 7f4033fff700 1 --2- 172.21.15.37:0/2097545722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f403406aa00 0x7f403419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.356+0000 7f40337fe700 1 --2- 172.21.15.37:0/2097545722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f403406b3a0 0x7f403419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.356+0000 7f4038e94700 1 -- 172.21.15.37:0/2097545722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f403406b3a0 msgr2=0x7f403419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:41.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.356+0000 7f4038e94700 1 --2- 172.21.15.37:0/2097545722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f403406b3a0 0x7f403419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.356+0000 7f4038e94700 1 -- 172.21.15.37:0/2097545722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f403406aa00 msgr2=0x7f403419e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:41.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.356+0000 7f4038e94700 1 --2- 172.21.15.37:0/2097545722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f403406aa00 0x7f403419e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.356+0000 7f4038e94700 1 -- 172.21.15.37:0/2097545722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4034199120 con 0x7f40341084d0 2024-06-10T01:10:41.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.356+0000 7f4033fff700 1 --2- 172.21.15.37:0/2097545722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f403406aa00 0x7f403419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:41.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.357+0000 7f4038e94700 1 --2- 172.21.15.37:0/2097545722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40341084d0 0x7f40341989c0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f402c00b780 tx=0x7f402c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:41.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.357+0000 7f40317fa700 1 -- 172.21.15.37:0/2097545722 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c021070 con 0x7f40341084d0 2024-06-10T01:10:41.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.357+0000 7f40317fa700 1 -- 172.21.15.37:0/2097545722 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f402c007b80 con 0x7f40341084d0 2024-06-10T01:10:41.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.357+0000 7f403a8f7700 1 -- 172.21.15.37:0/2097545722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40341993b0 con 0x7f40341084d0 2024-06-10T01:10:41.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.357+0000 7f40317fa700 1 -- 172.21.15.37:0/2097545722 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c0052a0 con 0x7f40341084d0 2024-06-10T01:10:41.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.357+0000 7f403a8f7700 1 -- 172.21.15.37:0/2097545722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4034101380 con 0x7f40341084d0 2024-06-10T01:10:41.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.360+0000 7f40317fa700 1 -- 172.21.15.37:0/2097545722 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f402c024070 con 0x7f40341084d0 2024-06-10T01:10:41.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.360+0000 7f40317fa700 1 --2- 172.21.15.37:0/2097545722 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f401c0a2ca0 0x7f401c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.361+0000 7f4033fff700 1 --2- 172.21.15.37:0/2097545722 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f401c0a2ca0 0x7f401c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.361+0000 7f40317fa700 1 -- 172.21.15.37:0/2097545722 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7f402c00f020 con 0x7f40341084d0 2024-06-10T01:10:41.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.361+0000 7f404517ab80 1 Processor -- start 2024-06-10T01:10:41.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.361+0000 7f404517ab80 1 -- start start 2024-06-10T01:10:41.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.361+0000 7f4033fff700 1 --2- 172.21.15.37:0/2097545722 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f401c0a2ca0 0x7f401c0a5160 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4034199b00 tx=0x7f4024012040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:41.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.361+0000 7f404517ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf976c6e00 0x55bf976c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.361+0000 7f404517ab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf976c7720 0x55bf976d9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.361+0000 7f404517ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf976d9c50 0x55bf976dc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.361+0000 7f404517ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bf97618880 con 0x55bf976d9c50 2024-06-10T01:10:41.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.361+0000 7f404517ab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bf975c0790 con 0x55bf976c7720 2024-06-10T01:10:41.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.361+0000 7f404517ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bf97614840 con 0x55bf976c6e00 2024-06-10T01:10:41.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.362+0000 7f4038e94700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf976d9c50 0x55bf976dc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.362+0000 7f4038e94700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf976d9c50 0x55bf976dc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35068/0 (socket says 172.21.15.37:35068) 2024-06-10T01:10:41.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.362+0000 7f4038e94700 1 -- 172.21.15.37:0/3597533089 learned_addr learned my addr 172.21.15.37:0/3597533089 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:41.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.362+0000 7f40337fe700 1 --2- 172.21.15.37:0/3597533089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf976c7720 0x55bf976d9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.362+0000 7f4033fff700 1 --2- 172.21.15.37:0/3597533089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf976c6e00 0x55bf976c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.362+0000 7f4038e94700 1 -- 172.21.15.37:0/3597533089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf976c6e00 msgr2=0x55bf976c71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:41.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.362+0000 7f4038e94700 1 --2- 172.21.15.37:0/3597533089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf976c6e00 0x55bf976c71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.362+0000 7f4038e94700 1 -- 172.21.15.37:0/3597533089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf976c7720 msgr2=0x55bf976d9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:41.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.362+0000 7f4038e94700 1 --2- 172.21.15.37:0/3597533089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf976c7720 0x55bf976d9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.362+0000 7f4038e94700 1 -- 172.21.15.37:0/3597533089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf975b6fc0 con 0x55bf976d9c50 2024-06-10T01:10:41.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.362+0000 7f4033fff700 1 --2- 172.21.15.37:0/3597533089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf976c6e00 0x55bf976c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:10:41.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.362+0000 7f4038e94700 1 --2- 172.21.15.37:0/3597533089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf976d9c50 0x55bf976dc040 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f402c0a5400 tx=0x7f402c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f45569a855ec8b9 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:41.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.362+0000 7f401a7fc700 1 -- 172.21.15.37:0/3597533089 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c0074e0 con 0x55bf976d9c50 2024-06-10T01:10:41.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.362+0000 7f401a7fc700 1 -- 172.21.15.37:0/3597533089 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f402c09cd50 con 0x55bf976d9c50 2024-06-10T01:10:41.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.363+0000 7f401a7fc700 1 -- 172.21.15.37:0/3597533089 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c09d210 con 0x55bf976d9c50 2024-06-10T01:10:41.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.363+0000 7f404517ab80 1 -- 172.21.15.37:0/3597533089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf976d9c50 msgr2=0x55bf976dc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:41.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.363+0000 7f404517ab80 1 --2- 172.21.15.37:0/3597533089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf976d9c50 0x55bf976dc040 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f402c0a5400 tx=0x7f402c0099e0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.363+0000 7f404517ab80 1 -- 172.21.15.37:0/3597533089 shutdown_connections 2024-06-10T01:10:41.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.363+0000 7f404517ab80 1 --2- 172.21.15.37:0/3597533089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf976d9c50 0x55bf976dc040 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.363+0000 7f404517ab80 1 --2- 172.21.15.37:0/3597533089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf976c6e00 0x55bf976c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.363+0000 7f404517ab80 1 --2- 172.21.15.37:0/3597533089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf976c7720 0x55bf976d9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.363+0000 7f404517ab80 1 -- 172.21.15.37:0/3597533089 >> 172.21.15.37:0/3597533089 conn(0x55bf975f1d30 msgr2=0x55bf976c1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:41.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.364+0000 7f404517ab80 1 -- 172.21.15.37:0/3597533089 shutdown_connections 2024-06-10T01:10:41.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.364+0000 7f404517ab80 1 -- 172.21.15.37:0/3597533089 wait complete. 2024-06-10T01:10:41.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.364+0000 7f404517ab80 1 -- start start 2024-06-10T01:10:41.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.364+0000 7f404517ab80 1 Processor -- start 2024-06-10T01:10:41.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.364+0000 7f404517ab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf976c6e00 0x55bf976d9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.365+0000 7f404517ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf976c7720 0x55bf976d6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.365+0000 7f404517ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf976d9c50 0x55bf976d6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.365+0000 7f404517ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bf97618880 con 0x55bf976c7720 2024-06-10T01:10:41.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.365+0000 7f404517ab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bf975b8c60 con 0x55bf976c6e00 2024-06-10T01:10:41.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.365+0000 7f404517ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bf976dc580 con 0x55bf976d9c50 2024-06-10T01:10:41.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.365+0000 7f40337fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf976c7720 0x55bf976d6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.365+0000 7f404517ab80 10 client.0 _ll_register_callbacks cb 0x55bf975b2990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:41.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.365+0000 7f40337fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf976c7720 0x55bf976d6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35078/0 (socket says 172.21.15.37:35078) 2024-06-10T01:10:41.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.365+0000 7f40337fe700 1 -- 172.21.15.37:0/3241582763 learned_addr learned my addr 172.21.15.37:0/3241582763 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:41.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.365+0000 7f4033fff700 1 --2- 172.21.15.37:0/3241582763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf976c6e00 0x55bf976d9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.365+0000 7f4038e94700 1 --2- 172.21.15.37:0/3241582763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf976d9c50 0x55bf976d6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.365+0000 7f40337fe700 1 -- 172.21.15.37:0/3241582763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf976d9c50 msgr2=0x55bf976d6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:41.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.365+0000 7f40337fe700 1 --2- 172.21.15.37:0/3241582763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf976d9c50 0x55bf976d6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.365+0000 7f40337fe700 1 -- 172.21.15.37:0/3241582763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf976c6e00 msgr2=0x55bf976d9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:41.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.365+0000 7f40337fe700 1 --2- 172.21.15.37:0/3241582763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf976c6e00 0x55bf976d9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.365+0000 7f40337fe700 1 -- 172.21.15.37:0/3241582763 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf97567940 con 0x55bf976c7720 2024-06-10T01:10:41.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.365+0000 7f4038e94700 1 --2- 172.21.15.37:0/3241582763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf976d9c50 0x55bf976d6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:10:41.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.365+0000 7f40337fe700 1 --2- 172.21.15.37:0/3241582763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf976c7720 0x55bf976d6700 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f4028012700 tx=0x7f4028012a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c93eaef33fb392b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:41.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.365+0000 7f40137fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:10:41.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.365+0000 7f40137fe700 1 -- 172.21.15.37:0/3241582763 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4028004050 con 0x55bf976c7720 2024-06-10T01:10:41.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.365+0000 7f404517ab80 1 -- 172.21.15.37:0/3241582763 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f402c09cd50 con 0x55bf976c7720 2024-06-10T01:10:41.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.365+0000 7f40137fe700 1 -- 172.21.15.37:0/3241582763 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7f40280041f0 con 0x55bf976c7720 2024-06-10T01:10:41.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.366+0000 7f40137fe700 1 -- 172.21.15.37:0/3241582763 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4028025050 con 0x55bf976c7720 2024-06-10T01:10:41.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.366+0000 7f404517ab80 10 client.17925 fetch_fsmap learned FSMap version 35 2024-06-10T01:10:41.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.366+0000 7f404517ab80 1 -- 172.21.15.37:0/3241582763 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55bf976e6de0 con 0x55bf976c7720 2024-06-10T01:10:41.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.366+0000 7f40137fe700 1 -- 172.21.15.37:0/3241582763 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f402c09cd50 con 0x55bf976c7720 2024-06-10T01:10:41.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.366+0000 7f40137fe700 1 -- 172.21.15.37:0/3241582763 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f402800f640 con 0x55bf976c7720 2024-06-10T01:10:41.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.366+0000 7f404517ab80 10 client.17925 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:10:41.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.366+0000 7f404517ab80 10 client.17925 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:10:41.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.366+0000 7f404517ab80 20 client.17925 populate_metadata read hostname 'smithi037' 2024-06-10T01:10:41.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.366+0000 7f404517ab80 1 --2- 172.21.15.37:0/3241582763 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bf976e93c0 0x55bf976eb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.366+0000 7f404517ab80 4 client.17925 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:10:41.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.366+0000 7f404517ab80 1 -- 172.21.15.37:0/3241582763 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf97625d80 con 0x55bf976e93c0 2024-06-10T01:10:41.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.367+0000 7f4033fff700 1 --2- 172.21.15.37:0/3241582763 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bf976e93c0 0x55bf976eb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.367+0000 7f4033fff700 1 --2- 172.21.15.37:0/3241582763 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bf976e93c0 0x55bf976eb7b0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=397ed429853b4ec5 server_cookie=a0e864496e810a57 in_seq=0 out_seq=0 2024-06-10T01:10:41.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.367+0000 7f40137fe700 10 client.17925 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:10:41.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.368+0000 7f40137fe700 1 -- 172.21.15.37:0/3241582763 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bf97625d80 con 0x55bf976e93c0 2024-06-10T01:10:41.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.368+0000 7f40137fe700 10 client.17925 handle_command_reply: tid=0 2024-06-10T01:10:41.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.368+0000 7f404517ab80 1 client.17925 shutdown 2024-06-10T01:10:41.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.368+0000 7f404517ab80 1 -- 172.21.15.37:0/3241582763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf976c7720 msgr2=0x55bf976d6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:41.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.368+0000 7f404517ab80 1 --2- 172.21.15.37:0/3241582763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf976c7720 0x55bf976d6700 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f4028012700 tx=0x7f4028012a10 comp rx=0 tx=0).stop 2024-06-10T01:10:41.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.368+0000 7f404517ab80 1 -- 172.21.15.37:0/3241582763 shutdown_connections 2024-06-10T01:10:41.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.368+0000 7f404517ab80 1 --2- 172.21.15.37:0/3241582763 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bf976e93c0 0x55bf976eb7b0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.368+0000 7f40137fe700 0 client.17925 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:10:41.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.368+0000 7f404517ab80 1 --2- 172.21.15.37:0/3241582763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf976d9c50 0x55bf976d6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.368+0000 7f404517ab80 1 --2- 172.21.15.37:0/3241582763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf976c6e00 0x55bf976d9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.368+0000 7f404517ab80 1 --2- 172.21.15.37:0/3241582763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf976c7720 0x55bf976d6700 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.368+0000 7f404517ab80 1 -- 172.21.15.37:0/3241582763 >> 172.21.15.37:0/3241582763 conn(0x55bf975f1d30 msgr2=0x55bf976c3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:41.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.369+0000 7f404517ab80 1 -- 172.21.15.37:0/3241582763 shutdown_connections 2024-06-10T01:10:41.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.369+0000 7f404517ab80 1 -- 172.21.15.37:0/3241582763 wait complete. 2024-06-10T01:10:41.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.369+0000 7f404517ab80 20 client.17925 trim_cache size 0 max 16384 2024-06-10T01:10:41.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.389+0000 7f404517ab80 1 Processor -- start 2024-06-10T01:10:41.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.389+0000 7f404517ab80 1 -- start start 2024-06-10T01:10:41.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.389+0000 7f404517ab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf976c6e00 0x55bf976c6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.389+0000 7f404517ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf976c7720 0x55bf976f1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.389+0000 7f404517ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf976d9c50 0x55bf976f15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.389+0000 7f404517ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f40341aeb10 con 0x55bf976d9c50 2024-06-10T01:10:41.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.389+0000 7f404517ab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f40341ac3a0 con 0x55bf976c6e00 2024-06-10T01:10:41.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.389+0000 7f404517ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f40341aa1f0 con 0x55bf976c7720 2024-06-10T01:10:41.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.389+0000 7f4038e94700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf976d9c50 0x55bf976f15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.390+0000 7f4038e94700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf976d9c50 0x55bf976f15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35080/0 (socket says 172.21.15.37:35080) 2024-06-10T01:10:41.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.390+0000 7f4038e94700 1 -- 172.21.15.37:0/1175254214 learned_addr learned my addr 172.21.15.37:0/1175254214 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:41.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.390+0000 7f4033fff700 1 --2- 172.21.15.37:0/1175254214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf976c6e00 0x55bf976c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.390+0000 7f40337fe700 1 --2- 172.21.15.37:0/1175254214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf976c7720 0x55bf976f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.390+0000 7f4038e94700 1 -- 172.21.15.37:0/1175254214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf976c7720 msgr2=0x55bf976f1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:41.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.390+0000 7f4038e94700 1 --2- 172.21.15.37:0/1175254214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf976c7720 0x55bf976f1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.390+0000 7f4038e94700 1 -- 172.21.15.37:0/1175254214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf976c6e00 msgr2=0x55bf976c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:41.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.390+0000 7f4038e94700 1 --2- 172.21.15.37:0/1175254214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf976c6e00 0x55bf976c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.390+0000 7f4038e94700 1 -- 172.21.15.37:0/1175254214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf976d4c70 con 0x55bf976d9c50 2024-06-10T01:10:41.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.390+0000 7f4038e94700 1 --2- 172.21.15.37:0/1175254214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf976d9c50 0x55bf976f15c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f402c028040 tx=0x7f402c00b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dbf98712464857a server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:41.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.390+0000 7f4019ffb700 1 -- 172.21.15.37:0/1175254214 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c0bb070 con 0x55bf976d9c50 2024-06-10T01:10:41.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.390+0000 7f4019ffb700 1 -- 172.21.15.37:0/1175254214 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f402c0b9070 con 0x55bf976d9c50 2024-06-10T01:10:41.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.390+0000 7f4019ffb700 1 -- 172.21.15.37:0/1175254214 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c0ce070 con 0x55bf976d9c50 2024-06-10T01:10:41.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.391+0000 7f404517ab80 1 -- 172.21.15.37:0/1175254214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf976d9c50 msgr2=0x55bf976f15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:41.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.391+0000 7f404517ab80 1 --2- 172.21.15.37:0/1175254214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf976d9c50 0x55bf976f15c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f402c028040 tx=0x7f402c00b3b0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.391+0000 7f404517ab80 1 -- 172.21.15.37:0/1175254214 shutdown_connections 2024-06-10T01:10:41.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.391+0000 7f404517ab80 1 --2- 172.21.15.37:0/1175254214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf976d9c50 0x55bf976f15c0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.391+0000 7f404517ab80 1 --2- 172.21.15.37:0/1175254214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf976c6e00 0x55bf976c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.391+0000 7f404517ab80 1 --2- 172.21.15.37:0/1175254214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf976c7720 0x55bf976f1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.391+0000 7f404517ab80 1 -- 172.21.15.37:0/1175254214 >> 172.21.15.37:0/1175254214 conn(0x55bf975f1d30 msgr2=0x55bf976c5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:41.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.391+0000 7f404517ab80 1 -- 172.21.15.37:0/1175254214 shutdown_connections 2024-06-10T01:10:41.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.391+0000 7f404517ab80 1 -- 172.21.15.37:0/1175254214 wait complete. 2024-06-10T01:10:41.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.391+0000 7f404517ab80 1 -- start start 2024-06-10T01:10:41.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.392+0000 7f404517ab80 1 Processor -- start 2024-06-10T01:10:41.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.392+0000 7f404517ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf976c6e00 0x55bf976de6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.392+0000 7f404517ab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf976c7720 0x55bf976dec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.392+0000 7f404517ab80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf976d9c50 0x55bf976df170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.392+0000 7f404517ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f40341aa860 con 0x55bf976c6e00 2024-06-10T01:10:41.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.392+0000 7f404517ab80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f40341abcd0 con 0x55bf976d9c50 2024-06-10T01:10:41.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.392+0000 7f404517ab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f40341aaf30 con 0x55bf976c7720 2024-06-10T01:10:41.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.392+0000 7f404517ab80 10 client.0 _ll_register_callbacks cb 0x55bf975b2990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:41.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.392+0000 7f4033fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf976c6e00 0x55bf976de6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.392+0000 7f4033fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf976c6e00 0x55bf976de6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35090/0 (socket says 172.21.15.37:35090) 2024-06-10T01:10:41.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.392+0000 7f4033fff700 1 -- 172.21.15.37:0/3787569665 learned_addr learned my addr 172.21.15.37:0/3787569665 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:41.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.392+0000 7f40337fe700 1 --2- 172.21.15.37:0/3787569665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf976c7720 0x55bf976dec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.392+0000 7f4038e94700 1 --2- 172.21.15.37:0/3787569665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf976d9c50 0x55bf976df170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.392+0000 7f4033fff700 1 -- 172.21.15.37:0/3787569665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf976c7720 msgr2=0x55bf976dec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:41.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.392+0000 7f4033fff700 1 --2- 172.21.15.37:0/3787569665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf976c7720 0x55bf976dec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.392+0000 7f4033fff700 1 -- 172.21.15.37:0/3787569665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf976d9c50 msgr2=0x55bf976df170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:41.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.392+0000 7f4033fff700 1 --2- 172.21.15.37:0/3787569665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf976d9c50 0x55bf976df170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.392+0000 7f4033fff700 1 -- 172.21.15.37:0/3787569665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf97754130 con 0x55bf976c6e00 2024-06-10T01:10:41.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.392+0000 7f4038e94700 1 --2- 172.21.15.37:0/3787569665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf976d9c50 0x55bf976df170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:10:41.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.393+0000 7f4033fff700 1 --2- 172.21.15.37:0/3787569665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf976c6e00 0x55bf976de6f0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x55bf976ec020 tx=0x7f4024004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57ae8c4baa793d3 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:41.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.393+0000 7f40137fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:10:41.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.393+0000 7f40137fe700 1 -- 172.21.15.37:0/3787569665 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4024003c30 con 0x55bf976c6e00 2024-06-10T01:10:41.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.393+0000 7f40137fe700 1 -- 172.21.15.37:0/3787569665 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7f4024003dd0 con 0x55bf976c6e00 2024-06-10T01:10:41.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.393+0000 7f404517ab80 1 -- 172.21.15.37:0/3787569665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f402c0b9070 con 0x55bf976c6e00 2024-06-10T01:10:41.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.393+0000 7f40137fe700 1 -- 172.21.15.37:0/3787569665 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f402403e050 con 0x55bf976c6e00 2024-06-10T01:10:41.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.393+0000 7f404517ab80 10 client.17931 fetch_fsmap learned FSMap version 35 2024-06-10T01:10:41.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.393+0000 7f404517ab80 1 -- 172.21.15.37:0/3787569665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55bf97755450 con 0x55bf976c6e00 2024-06-10T01:10:41.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.393+0000 7f40137fe700 1 -- 172.21.15.37:0/3787569665 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f402c0b9070 con 0x55bf976c6e00 2024-06-10T01:10:41.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.393+0000 7f40137fe700 1 -- 172.21.15.37:0/3787569665 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f4024028650 con 0x55bf976c6e00 2024-06-10T01:10:41.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.393+0000 7f404517ab80 10 client.17931 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:10:41.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.393+0000 7f404517ab80 10 client.17931 resolve_mds: validated gid 24472 aka daemon mds.i 2024-06-10T01:10:41.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.393+0000 7f404517ab80 20 client.17931 populate_metadata read hostname 'smithi037' 2024-06-10T01:10:41.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.393+0000 7f404517ab80 1 --2- 172.21.15.37:0/3787569665 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bf97755b90 0x55bf97757fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:41.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.393+0000 7f404517ab80 4 client.17931 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-06-10T01:10:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.394+0000 7f404517ab80 1 -- 172.21.15.37:0/3787569665 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55bf97620300 con 0x55bf97755b90 2024-06-10T01:10:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.394+0000 7f40337fe700 1 --2- 172.21.15.37:0/3787569665 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bf97755b90 0x55bf97757fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.394+0000 7f40337fe700 1 --2- 172.21.15.37:0/3787569665 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bf97755b90 0x55bf97757fb0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab47d650c571e1a6 server_cookie=9500f2038e2023e5 in_seq=0 out_seq=0 2024-06-10T01:10:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.394+0000 7f40137fe700 10 client.17931 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:10:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.394+0000 7f40137fe700 1 -- 172.21.15.37:0/3787569665 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55bf97620300 con 0x55bf97755b90 2024-06-10T01:10:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.394+0000 7f40137fe700 10 client.17931 handle_command_reply: tid=0 2024-06-10T01:10:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.394+0000 7f404517ab80 1 client.17931 shutdown 2024-06-10T01:10:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.395+0000 7f404517ab80 1 -- 172.21.15.37:0/3787569665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf976c6e00 msgr2=0x55bf976de6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:41.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.395+0000 7f404517ab80 1 --2- 172.21.15.37:0/3787569665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf976c6e00 0x55bf976de6f0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x55bf976ec020 tx=0x7f4024004750 comp rx=0 tx=0).stop 2024-06-10T01:10:41.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.395+0000 7f404517ab80 1 -- 172.21.15.37:0/3787569665 shutdown_connections 2024-06-10T01:10:41.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.395+0000 7f404517ab80 1 --2- 172.21.15.37:0/3787569665 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55bf97755b90 0x55bf97757fb0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.395+0000 7f404517ab80 1 --2- 172.21.15.37:0/3787569665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bf976d9c50 0x55bf976df170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.395+0000 7f404517ab80 1 --2- 172.21.15.37:0/3787569665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf976c6e00 0x55bf976de6f0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.395+0000 7f404517ab80 1 --2- 172.21.15.37:0/3787569665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf976c7720 0x55bf976dec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.395+0000 7f404517ab80 1 -- 172.21.15.37:0/3787569665 >> 172.21.15.37:0/3787569665 conn(0x55bf975f1d30 msgr2=0x55bf976c6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:41.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.395+0000 7f40137fe700 0 client.17931 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:10:41.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.395+0000 7f404517ab80 1 -- 172.21.15.37:0/3787569665 shutdown_connections 2024-06-10T01:10:41.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.395+0000 7f404517ab80 1 -- 172.21.15.37:0/3787569665 wait complete. 2024-06-10T01:10:41.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.395+0000 7f404517ab80 20 client.17931 trim_cache size 0 max 16384 2024-06-10T01:10:41.400 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:10:41.400 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 35, 2024-06-10T01:10:41.400 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24472, 2024-06-10T01:10:41.400 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 39, 2024-06-10T01:10:41.401 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": {} 2024-06-10T01:10:41.401 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:10:41.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.395+0000 7f403a8f7700 1 -- 172.21.15.37:0/2097545722 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f401c0a2ca0 msgr2=0x7f401c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:41.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.395+0000 7f403a8f7700 1 --2- 172.21.15.37:0/2097545722 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f401c0a2ca0 0x7f401c0a5160 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4034199b00 tx=0x7f4024012040 comp rx=0 tx=0).stop 2024-06-10T01:10:41.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.395+0000 7f403a8f7700 1 -- 172.21.15.37:0/2097545722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40341084d0 msgr2=0x7f40341989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:41.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.395+0000 7f403a8f7700 1 --2- 172.21.15.37:0/2097545722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40341084d0 0x7f40341989c0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f402c00b780 tx=0x7f402c004d00 comp rx=0 tx=0).stop 2024-06-10T01:10:41.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.396+0000 7f403a8f7700 1 -- 172.21.15.37:0/2097545722 shutdown_connections 2024-06-10T01:10:41.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.396+0000 7f403a8f7700 1 --2- 172.21.15.37:0/2097545722 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f401c0a2ca0 0x7f401c0a5160 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.396+0000 7f403a8f7700 1 --2- 172.21.15.37:0/2097545722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40341084d0 0x7f40341989c0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.396+0000 7f403a8f7700 1 --2- 172.21.15.37:0/2097545722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f403406aa00 0x7f403419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.396+0000 7f403a8f7700 1 --2- 172.21.15.37:0/2097545722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f403406b3a0 0x7f403419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:41.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.396+0000 7f403a8f7700 1 -- 172.21.15.37:0/2097545722 >> 172.21.15.37:0/2097545722 conn(0x7f4034078330 msgr2=0x7f40340ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:41.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.396+0000 7f403a8f7700 1 -- 172.21.15.37:0/2097545722 shutdown_connections 2024-06-10T01:10:41.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:41.396+0000 7f403a8f7700 1 -- 172.21.15.37:0/2097545722 wait complete. 2024-06-10T01:10:42.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:42 smithi167 ceph-mon[36917]: pgmap v855: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:10:42.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:42 smithi094 ceph-mon[36708]: pgmap v855: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:10:42.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:42 smithi037 ceph-mon[32192]: pgmap v855: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:10:43.200 INFO:tasks.quiescer:done joining Quiescers 2024-06-10T01:10:43.200 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-06-10T01:10:43.215 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi037.front.sepia.ceph.com 2024-06-10T01:10:43.226 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-06-10T01:10:43.247 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-06-10T01:10:43.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.745+0000 7f8e17560700 1 -- 172.21.15.37:0/788788181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e10100a80 msgr2=0x7f8e10100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:43.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.745+0000 7f8e17560700 1 --2- 172.21.15.37:0/788788181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e10100a80 0x7f8e10100f00 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f8e04009990 tx=0x7f8e04009ca0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.745+0000 7f8e17560700 1 -- 172.21.15.37:0/788788181 shutdown_connections 2024-06-10T01:10:43.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.745+0000 7f8e17560700 1 --2- 172.21.15.37:0/788788181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e10101440 0x7f8e1010f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.745+0000 7f8e17560700 1 --2- 172.21.15.37:0/788788181 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e10108510 0x7f8e101088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.745+0000 7f8e17560700 1 --2- 172.21.15.37:0/788788181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e10100a80 0x7f8e10100f00 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.745+0000 7f8e17560700 1 -- 172.21.15.37:0/788788181 >> 172.21.15.37:0/788788181 conn(0x7f8e10077a00 msgr2=0x7f8e100fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:43.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.745+0000 7f8e17560700 1 -- 172.21.15.37:0/788788181 shutdown_connections 2024-06-10T01:10:43.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.745+0000 7f8e17560700 1 -- 172.21.15.37:0/788788181 wait complete. 2024-06-10T01:10:43.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.746+0000 7f8e17560700 1 Processor -- start 2024-06-10T01:10:43.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.746+0000 7f8e17560700 1 -- start start 2024-06-10T01:10:43.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.746+0000 7f8e17560700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e10100a80 0x7f8e10101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:43.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.746+0000 7f8e17560700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e10101440 0x7f8e101021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:43.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.746+0000 7f8e17560700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e10108510 0x7f8e101026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:43.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.746+0000 7f8e17560700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8e10112790 con 0x7f8e10100a80 2024-06-10T01:10:43.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.746+0000 7f8e17560700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8e10112610 con 0x7f8e10108510 2024-06-10T01:10:43.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.746+0000 7f8e17560700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8e10112910 con 0x7f8e10101440 2024-06-10T01:10:43.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.746+0000 7f8e152fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e10100a80 0x7f8e10101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:43.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.746+0000 7f8e152fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e10100a80 0x7f8e10101c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59294/0 (socket says 172.21.15.37:59294) 2024-06-10T01:10:43.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.746+0000 7f8e152fc700 1 -- 172.21.15.37:0/197753112 learned_addr learned my addr 172.21.15.37:0/197753112 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:43.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.747+0000 7f8e14afb700 1 --2- 172.21.15.37:0/197753112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e10101440 0x7f8e101021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:43.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.747+0000 7f8e152fc700 1 -- 172.21.15.37:0/197753112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e10101440 msgr2=0x7f8e101021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:43.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.747+0000 7f8e152fc700 1 --2- 172.21.15.37:0/197753112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e10101440 0x7f8e101021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.747+0000 7f8e152fc700 1 -- 172.21.15.37:0/197753112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e10108510 msgr2=0x7f8e101026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:43.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.747+0000 7f8e152fc700 1 --2- 172.21.15.37:0/197753112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e10108510 0x7f8e101026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.747+0000 7f8e152fc700 1 -- 172.21.15.37:0/197753112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e101a7160 con 0x7f8e10100a80 2024-06-10T01:10:43.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.747+0000 7f8e152fc700 1 --2- 172.21.15.37:0/197753112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e10100a80 0x7f8e10101c60 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c00d950 tx=0x7f8e0c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:43.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.747+0000 7f8e027fc700 1 -- 172.21.15.37:0/197753112 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c016070 con 0x7f8e10100a80 2024-06-10T01:10:43.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.747+0000 7f8e17560700 1 -- 172.21.15.37:0/197753112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e101a7450 con 0x7f8e10100a80 2024-06-10T01:10:43.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.747+0000 7f8e027fc700 1 -- 172.21.15.37:0/197753112 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e0c009e20 con 0x7f8e10100a80 2024-06-10T01:10:43.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.747+0000 7f8e17560700 1 -- 172.21.15.37:0/197753112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e101a79b0 con 0x7f8e10100a80 2024-06-10T01:10:43.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.747+0000 7f8e027fc700 1 -- 172.21.15.37:0/197753112 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c00f620 con 0x7f8e10100a80 2024-06-10T01:10:43.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.750+0000 7f8e027fc700 1 -- 172.21.15.37:0/197753112 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8e0c019070 con 0x7f8e10100a80 2024-06-10T01:10:43.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.752+0000 7f8e21de3b80 1 Processor -- start 2024-06-10T01:10:43.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.752+0000 7f8e027fc700 1 --2- 172.21.15.37:0/197753112 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8dfc0a2ca0 0x7f8dfc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:43.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.752+0000 7f8e21de3b80 1 -- start start 2024-06-10T01:10:43.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.752+0000 7f8e15afd700 1 --2- 172.21.15.37:0/197753112 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8dfc0a2ca0 0x7f8dfc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:43.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.752+0000 7f8e21de3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df939e4640 0x55df939e4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:43.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.752+0000 7f8e21de3b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df939e4f60 0x55df939de9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:43.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.752+0000 7f8e21de3b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df939def20 0x55df939df340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:43.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.752+0000 7f8e21de3b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55df93930810 con 0x55df939e4f60 2024-06-10T01:10:43.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.752+0000 7f8e21de3b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55df938d8720 con 0x55df939def20 2024-06-10T01:10:43.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.752+0000 7f8e21de3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55df9392c7d0 con 0x55df939e4640 2024-06-10T01:10:43.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.752+0000 7f8e15afd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df939def20 0x55df939df340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:43.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.752+0000 7f8e14afb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df939e4f60 0x55df939de9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:43.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.752+0000 7f8e14afb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df939e4f60 0x55df939de9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59302/0 (socket says 172.21.15.37:59302) 2024-06-10T01:10:43.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.752+0000 7f8e14afb700 1 -- 172.21.15.37:0/3685293986 learned_addr learned my addr 172.21.15.37:0/3685293986 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:43.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.752+0000 7f8e152fc700 1 --2- 172.21.15.37:0/3685293986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df939e4640 0x55df939e4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:43.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.753+0000 7f8e027fc700 1 -- 172.21.15.37:0/197753112 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7f8e0c0cef60 con 0x7f8e10100a80 2024-06-10T01:10:43.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.753+0000 7f8e15afd700 1 -- 172.21.15.37:0/3685293986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df939e4640 msgr2=0x55df939e4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:43.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.753+0000 7f8e15afd700 1 --2- 172.21.15.37:0/3685293986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df939e4640 0x55df939e4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.753+0000 7f8e15afd700 1 -- 172.21.15.37:0/3685293986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df939e4f60 msgr2=0x55df939de9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:43.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.753+0000 7f8e15afd700 1 --2- 172.21.15.37:0/3685293986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df939e4f60 0x55df939de9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.753+0000 7f8e15afd700 1 -- 172.21.15.37:0/3685293986 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df938cef50 con 0x55df939def20 2024-06-10T01:10:43.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.753+0000 7f8e14afb700 1 --2- 172.21.15.37:0/3685293986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df939e4f60 0x55df939de9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:10:43.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.753+0000 7f8e15afd700 1 --2- 172.21.15.37:0/197753112 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8dfc0a2ca0 0x7f8dfc0a5160 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8e08009510 tx=0x7f8e08007580 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:43.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.753+0000 7f8e15afd700 1 --2- 172.21.15.37:0/3685293986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df939def20 0x55df939df340 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8e080060b0 tx=0x7f8e08019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fee4a5b32ee697fb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:43.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.753+0000 7f8dfb7fe700 1 -- 172.21.15.37:0/3685293986 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e08012040 con 0x55df939def20 2024-06-10T01:10:43.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.753+0000 7f8dfb7fe700 1 -- 172.21.15.37:0/3685293986 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e080198e0 con 0x55df939def20 2024-06-10T01:10:43.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.753+0000 7f8dfb7fe700 1 -- 172.21.15.37:0/3685293986 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e080185a0 con 0x55df939def20 2024-06-10T01:10:43.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.754+0000 7f8e21de3b80 1 -- 172.21.15.37:0/3685293986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df939def20 msgr2=0x55df939df340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:43.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.754+0000 7f8e21de3b80 1 --2- 172.21.15.37:0/3685293986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df939def20 0x55df939df340 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8e080060b0 tx=0x7f8e08019000 comp rx=0 tx=0).stop 2024-06-10T01:10:43.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.754+0000 7f8e21de3b80 1 -- 172.21.15.37:0/3685293986 shutdown_connections 2024-06-10T01:10:43.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.754+0000 7f8e21de3b80 1 --2- 172.21.15.37:0/3685293986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df939def20 0x55df939df340 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.754+0000 7f8e21de3b80 1 --2- 172.21.15.37:0/3685293986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df939e4640 0x55df939e4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.754+0000 7f8e21de3b80 1 --2- 172.21.15.37:0/3685293986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df939e4f60 0x55df939de9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.754+0000 7f8e21de3b80 1 -- 172.21.15.37:0/3685293986 >> 172.21.15.37:0/3685293986 conn(0x55df93909cc0 msgr2=0x55df939d9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:43.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.754+0000 7f8e21de3b80 1 -- 172.21.15.37:0/3685293986 shutdown_connections 2024-06-10T01:10:43.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.754+0000 7f8e21de3b80 1 -- 172.21.15.37:0/3685293986 wait complete. 2024-06-10T01:10:43.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.755+0000 7f8e21de3b80 1 -- start start 2024-06-10T01:10:43.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.755+0000 7f8e21de3b80 1 Processor -- start 2024-06-10T01:10:43.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.755+0000 7f8e21de3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df939def20 0x55df939ead10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:43.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.755+0000 7f8e21de3b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df939e4640 0x55df939e8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:43.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.755+0000 7f8e21de3b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df939e4f60 0x55df939e8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:43.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.755+0000 7f8e21de3b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55df939eb970 con 0x55df939e4f60 2024-06-10T01:10:43.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.755+0000 7f8e21de3b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55df938d0bf0 con 0x55df939e4640 2024-06-10T01:10:43.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.755+0000 7f8e21de3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55df939ebaf0 con 0x55df939def20 2024-06-10T01:10:43.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.756+0000 7f8e21de3b80 10 client.0 _ll_register_callbacks cb 0x55df938ca920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:43.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.756+0000 7f8e14afb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df939e4640 0x55df939e8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:43.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.756+0000 7f8e14afb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df939e4640 0x55df939e8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.37:36270/0 (socket says 172.21.15.37:36270) 2024-06-10T01:10:43.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.756+0000 7f8e14afb700 1 -- 172.21.15.37:0/2951930425 learned_addr learned my addr 172.21.15.37:0/2951930425 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:43.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.756+0000 7f8e152fc700 1 --2- 172.21.15.37:0/2951930425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df939def20 0x55df939ead10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:43.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.756+0000 7f8e15afd700 1 --2- 172.21.15.37:0/2951930425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df939e4f60 0x55df939e8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:43.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.756+0000 7f8e14afb700 1 -- 172.21.15.37:0/2951930425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df939def20 msgr2=0x55df939ead10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:43.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.756+0000 7f8e14afb700 1 --2- 172.21.15.37:0/2951930425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df939def20 0x55df939ead10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.756+0000 7f8e14afb700 1 -- 172.21.15.37:0/2951930425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df939e4f60 msgr2=0x55df939e8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:43.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.756+0000 7f8e14afb700 1 --2- 172.21.15.37:0/2951930425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df939e4f60 0x55df939e8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.756+0000 7f8e14afb700 1 -- 172.21.15.37:0/2951930425 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df9387f8d0 con 0x55df939e4640 2024-06-10T01:10:43.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.756+0000 7f8e14afb700 1 --2- 172.21.15.37:0/2951930425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df939e4640 0x55df939e8010 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400bfb0 tx=0x7f8e04005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b26696a210d6ccb server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:43.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.756+0000 7f8df8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-10T01:10:43.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.757+0000 7f8df8ff9700 1 -- 172.21.15.37:0/2951930425 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04029060 con 0x55df939e4640 2024-06-10T01:10:43.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.757+0000 7f8df8ff9700 1 -- 172.21.15.37:0/2951930425 <== mon.2 v2:172.21.15.94:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7f8e04003d70 con 0x55df939e4640 2024-06-10T01:10:43.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.757+0000 7f8e21de3b80 1 -- 172.21.15.37:0/2951930425 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e080198e0 con 0x55df939e4640 2024-06-10T01:10:43.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.757+0000 7f8df8ff9700 1 -- 172.21.15.37:0/2951930425 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8e04030050 con 0x55df939e4640 2024-06-10T01:10:43.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.757+0000 7f8e21de3b80 10 client.27200 fetch_fsmap learned FSMap version 35 2024-06-10T01:10:43.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.757+0000 7f8e21de3b80 1 -- 172.21.15.37:0/2951930425 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55df939f6330 con 0x55df939e4640 2024-06-10T01:10:43.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.757+0000 7f8df8ff9700 1 -- 172.21.15.37:0/2951930425 <== mon.2 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e080198e0 con 0x55df939e4640 2024-06-10T01:10:43.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.757+0000 7f8df8ff9700 1 -- 172.21.15.37:0/2951930425 <== mon.2 v2:172.21.15.94:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f8e0400f860 con 0x55df939e4640 2024-06-10T01:10:43.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.757+0000 7f8e21de3b80 10 client.27200 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:10:43.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.757+0000 7f8e21de3b80 10 client.27200 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-10T01:10:43.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.757+0000 7f8e21de3b80 20 client.27200 populate_metadata read hostname 'smithi037' 2024-06-10T01:10:43.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.757+0000 7f8e21de3b80 1 --2- 172.21.15.37:0/2951930425 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55df939f8970 0x55df939fad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.757+0000 7f8e21de3b80 4 client.27200 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:10:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.757+0000 7f8e21de3b80 1 -- 172.21.15.37:0/2951930425 --> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df9393dd10 con 0x55df939f8970 2024-06-10T01:10:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.758+0000 7f8e152fc700 1 --2- 172.21.15.37:0/2951930425 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55df939f8970 0x55df939fad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.758+0000 7f8df8ff9700 10 client.27200 ms_handle_connect on v2:172.21.15.167:6836/2720148065 2024-06-10T01:10:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.758+0000 7f8e152fc700 1 --2- 172.21.15.37:0/2951930425 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55df939f8970 0x55df939fad60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a7336f5fa6e117b server_cookie=a7ae5df1b27c757f in_seq=0 out_seq=0 2024-06-10T01:10:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.759+0000 7f8df8ff9700 1 -- 172.21.15.37:0/2951930425 <== mds.2 v2:172.21.15.167:6836/2720148065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55df9393dd10 con 0x55df939f8970 2024-06-10T01:10:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.759+0000 7f8df8ff9700 10 client.27200 handle_command_reply: tid=0 2024-06-10T01:10:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.759+0000 7f8e21de3b80 1 client.27200 shutdown 2024-06-10T01:10:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.759+0000 7f8e21de3b80 1 -- 172.21.15.37:0/2951930425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df939e4640 msgr2=0x55df939e8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.759+0000 7f8e21de3b80 1 --2- 172.21.15.37:0/2951930425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df939e4640 0x55df939e8010 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400bfb0 tx=0x7f8e04005c70 comp rx=0 tx=0).stop 2024-06-10T01:10:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.759+0000 7f8e21de3b80 1 -- 172.21.15.37:0/2951930425 shutdown_connections 2024-06-10T01:10:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.759+0000 7f8e21de3b80 1 --2- 172.21.15.37:0/2951930425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df939e4f60 0x55df939e8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.759+0000 7f8e21de3b80 1 --2- 172.21.15.37:0/2951930425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df939def20 0x55df939ead10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.759+0000 7f8e21de3b80 1 --2- 172.21.15.37:0/2951930425 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55df939f8970 0x55df939fad60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.759+0000 7f8e21de3b80 1 --2- 172.21.15.37:0/2951930425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df939e4640 0x55df939e8010 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.759+0000 7f8e21de3b80 1 -- 172.21.15.37:0/2951930425 >> 172.21.15.37:0/2951930425 conn(0x55df93909cc0 msgr2=0x55df939dd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:43.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.759+0000 7f8df8ff9700 0 client.27200 ms_handle_reset on v2:172.21.15.167:6836/2720148065 2024-06-10T01:10:43.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.759+0000 7f8e21de3b80 1 -- 172.21.15.37:0/2951930425 shutdown_connections 2024-06-10T01:10:43.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.759+0000 7f8e21de3b80 1 -- 172.21.15.37:0/2951930425 wait complete. 2024-06-10T01:10:43.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.760+0000 7f8e21de3b80 20 client.27200 trim_cache size 0 max 16384 2024-06-10T01:10:43.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.775+0000 7f8e21de3b80 1 Processor -- start 2024-06-10T01:10:43.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.775+0000 7f8e21de3b80 1 -- start start 2024-06-10T01:10:43.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.775+0000 7f8e21de3b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df939def20 0x55df939de9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:43.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.775+0000 7f8e21de3b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df939e4640 0x55df939d3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:43.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.775+0000 7f8e21de3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df939e4f60 0x55df939e9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:43.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.775+0000 7f8e21de3b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8e101b2e30 con 0x55df939e4640 2024-06-10T01:10:43.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.775+0000 7f8e21de3b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8e101b06c0 con 0x55df939def20 2024-06-10T01:10:43.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.775+0000 7f8e21de3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8e101ae510 con 0x55df939e4f60 2024-06-10T01:10:43.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.775+0000 7f8e14afb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df939e4640 0x55df939d3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:43.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.775+0000 7f8e14afb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df939e4640 0x55df939d3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59320/0 (socket says 172.21.15.37:59320) 2024-06-10T01:10:43.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.775+0000 7f8e14afb700 1 -- 172.21.15.37:0/3551341436 learned_addr learned my addr 172.21.15.37:0/3551341436 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:43.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.776+0000 7f8e152fc700 1 --2- 172.21.15.37:0/3551341436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df939def20 0x55df939de9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:43.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.776+0000 7f8e15afd700 1 --2- 172.21.15.37:0/3551341436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df939e4f60 0x55df939e9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:43.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.776+0000 7f8e14afb700 1 -- 172.21.15.37:0/3551341436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df939e4f60 msgr2=0x55df939e9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:43.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.776+0000 7f8e14afb700 1 --2- 172.21.15.37:0/3551341436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df939e4f60 0x55df939e9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.776+0000 7f8e14afb700 1 -- 172.21.15.37:0/3551341436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df939def20 msgr2=0x55df939de9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:43.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.776+0000 7f8e14afb700 1 --2- 172.21.15.37:0/3551341436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df939def20 0x55df939de9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.776+0000 7f8e14afb700 1 -- 172.21.15.37:0/3551341436 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df939e8f50 con 0x55df939e4640 2024-06-10T01:10:43.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.776+0000 7f8e14afb700 1 --2- 172.21.15.37:0/3551341436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df939e4640 0x55df939d3510 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f8e040098e0 tx=0x7f8e04005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c9e5c2c4f09611 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:43.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.776+0000 7f8dfaffd700 1 -- 172.21.15.37:0/3551341436 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04029070 con 0x55df939e4640 2024-06-10T01:10:43.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.776+0000 7f8dfaffd700 1 -- 172.21.15.37:0/3551341436 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e04003740 con 0x55df939e4640 2024-06-10T01:10:43.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.776+0000 7f8dfaffd700 1 -- 172.21.15.37:0/3551341436 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04005400 con 0x55df939e4640 2024-06-10T01:10:43.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.777+0000 7f8e21de3b80 1 -- 172.21.15.37:0/3551341436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df939e4640 msgr2=0x55df939d3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:43.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.777+0000 7f8e21de3b80 1 --2- 172.21.15.37:0/3551341436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df939e4640 0x55df939d3510 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f8e040098e0 tx=0x7f8e04005f00 comp rx=0 tx=0).stop 2024-06-10T01:10:43.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.777+0000 7f8e21de3b80 1 -- 172.21.15.37:0/3551341436 shutdown_connections 2024-06-10T01:10:43.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.777+0000 7f8e21de3b80 1 --2- 172.21.15.37:0/3551341436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df939e4f60 0x55df939e9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.777+0000 7f8e21de3b80 1 --2- 172.21.15.37:0/3551341436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df939def20 0x55df939de9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.777+0000 7f8e21de3b80 1 --2- 172.21.15.37:0/3551341436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df939e4640 0x55df939d3510 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.777+0000 7f8e21de3b80 1 -- 172.21.15.37:0/3551341436 >> 172.21.15.37:0/3551341436 conn(0x55df93909cc0 msgr2=0x55df939d2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:43.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.777+0000 7f8e21de3b80 1 -- 172.21.15.37:0/3551341436 shutdown_connections 2024-06-10T01:10:43.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.777+0000 7f8e21de3b80 1 -- 172.21.15.37:0/3551341436 wait complete. 2024-06-10T01:10:43.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.778+0000 7f8e21de3b80 1 -- start start 2024-06-10T01:10:43.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.778+0000 7f8e21de3b80 1 Processor -- start 2024-06-10T01:10:43.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.778+0000 7f8e21de3b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df939def20 0x55df939f0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:43.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.778+0000 7f8e21de3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df939e4640 0x55df939f1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:43.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.778+0000 7f8e21de3b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df939e4f60 0x55df939ffa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:43.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.779+0000 7f8e21de3b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8e101aeb80 con 0x55df939def20 2024-06-10T01:10:43.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.779+0000 7f8e21de3b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8e101afff0 con 0x55df939e4f60 2024-06-10T01:10:43.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.779+0000 7f8e21de3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8e101af250 con 0x55df939e4640 2024-06-10T01:10:43.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.779+0000 7f8e21de3b80 10 client.0 _ll_register_callbacks cb 0x55df938ca920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:43.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.779+0000 7f8e152fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df939def20 0x55df939f0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:43.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.779+0000 7f8e152fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df939def20 0x55df939f0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59324/0 (socket says 172.21.15.37:59324) 2024-06-10T01:10:43.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.779+0000 7f8e152fc700 1 -- 172.21.15.37:0/2042108108 learned_addr learned my addr 172.21.15.37:0/2042108108 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:43.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.779+0000 7f8e14afb700 1 --2- 172.21.15.37:0/2042108108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df939e4640 0x55df939f1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:43.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.779+0000 7f8e152fc700 1 -- 172.21.15.37:0/2042108108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df939e4640 msgr2=0x55df939f1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:43.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.779+0000 7f8e152fc700 1 --2- 172.21.15.37:0/2042108108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df939e4640 0x55df939f1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.779+0000 7f8e152fc700 1 -- 172.21.15.37:0/2042108108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df939e4f60 msgr2=0x55df939ffa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:43.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.779+0000 7f8e152fc700 1 --2- 172.21.15.37:0/2042108108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df939e4f60 0x55df939ffa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.779+0000 7f8e152fc700 1 -- 172.21.15.37:0/2042108108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df93a00010 con 0x55df939def20 2024-06-10T01:10:43.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.779+0000 7f8e152fc700 1 --2- 172.21.15.37:0/2042108108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df939def20 0x55df939f0de0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c010b00 tx=0x7f8e0c020e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e553677ff816285e server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:43.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.779+0000 7f8df8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:10:43.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.779+0000 7f8df8ff9700 1 -- 172.21.15.37:0/2042108108 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c0a0650 con 0x55df939def20 2024-06-10T01:10:43.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.779+0000 7f8e21de3b80 1 -- 172.21.15.37:0/2042108108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e04003740 con 0x55df939def20 2024-06-10T01:10:43.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.779+0000 7f8df8ff9700 1 -- 172.21.15.37:0/2042108108 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7f8e0c09f4c0 con 0x55df939def20 2024-06-10T01:10:43.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.780+0000 7f8df8ff9700 1 -- 172.21.15.37:0/2042108108 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8e0c0a0c30 con 0x55df939def20 2024-06-10T01:10:43.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.780+0000 7f8e21de3b80 10 client.17949 fetch_fsmap learned FSMap version 35 2024-06-10T01:10:43.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.780+0000 7f8e21de3b80 1 -- 172.21.15.37:0/2042108108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55df93a62ea0 con 0x55df939def20 2024-06-10T01:10:43.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.780+0000 7f8df8ff9700 1 -- 172.21.15.37:0/2042108108 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e04003740 con 0x55df939def20 2024-06-10T01:10:43.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.780+0000 7f8df8ff9700 1 -- 172.21.15.37:0/2042108108 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f8e0c0b41f0 con 0x55df939def20 2024-06-10T01:10:43.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.780+0000 7f8e21de3b80 10 client.17949 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:10:43.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.780+0000 7f8e21de3b80 10 client.17949 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-06-10T01:10:43.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.780+0000 7f8e21de3b80 20 client.17949 populate_metadata read hostname 'smithi037' 2024-06-10T01:10:43.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.780+0000 7f8e21de3b80 1 --2- 172.21.15.37:0/2042108108 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55df93a635b0 0x55df93a659f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:43.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.780+0000 7f8e21de3b80 4 client.17949 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:10:43.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.780+0000 7f8e21de3b80 1 -- 172.21.15.37:0/2042108108 --> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55df93938290 con 0x55df93a635b0 2024-06-10T01:10:43.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.780+0000 7f8e14afb700 1 --2- 172.21.15.37:0/2042108108 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55df93a635b0 0x55df93a659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:43.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.781+0000 7f8df8ff9700 10 client.17949 ms_handle_connect on v2:172.21.15.167:6836/2720148065 2024-06-10T01:10:43.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.781+0000 7f8e14afb700 1 --2- 172.21.15.37:0/2042108108 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55df93a635b0 0x55df93a659f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea51a3178211edc8 server_cookie=43207b49067b498e in_seq=0 out_seq=0 2024-06-10T01:10:43.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.781+0000 7f8df8ff9700 1 -- 172.21.15.37:0/2042108108 <== mds.2 v2:172.21.15.167:6836/2720148065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55df93938290 con 0x55df93a635b0 2024-06-10T01:10:43.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.781+0000 7f8df8ff9700 10 client.17949 handle_command_reply: tid=0 2024-06-10T01:10:43.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.781+0000 7f8e21de3b80 1 client.17949 shutdown 2024-06-10T01:10:43.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.781+0000 7f8e21de3b80 1 -- 172.21.15.37:0/2042108108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df939def20 msgr2=0x55df939f0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:43.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.781+0000 7f8e21de3b80 1 --2- 172.21.15.37:0/2042108108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df939def20 0x55df939f0de0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c010b00 tx=0x7f8e0c020e50 comp rx=0 tx=0).stop 2024-06-10T01:10:43.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.781+0000 7f8e21de3b80 1 -- 172.21.15.37:0/2042108108 shutdown_connections 2024-06-10T01:10:43.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.781+0000 7f8e21de3b80 1 --2- 172.21.15.37:0/2042108108 >> [v2:172.21.15.167:6836/2720148065,v1:172.21.15.167:6837/2720148065] conn(0x55df93a635b0 0x55df93a659f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.781+0000 7f8e21de3b80 1 --2- 172.21.15.37:0/2042108108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55df939e4f60 0x55df939ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.781+0000 7f8e21de3b80 1 --2- 172.21.15.37:0/2042108108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df939def20 0x55df939f0de0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.781+0000 7f8e21de3b80 1 --2- 172.21.15.37:0/2042108108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df939e4640 0x55df939f1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.781+0000 7f8df8ff9700 0 client.17949 ms_handle_reset on v2:172.21.15.167:6836/2720148065 2024-06-10T01:10:43.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.781+0000 7f8e21de3b80 1 -- 172.21.15.37:0/2042108108 >> 172.21.15.37:0/2042108108 conn(0x55df93909cc0 msgr2=0x55df939e9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:43.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.781+0000 7f8e21de3b80 1 -- 172.21.15.37:0/2042108108 shutdown_connections 2024-06-10T01:10:43.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.782+0000 7f8e21de3b80 1 -- 172.21.15.37:0/2042108108 wait complete. 2024-06-10T01:10:43.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.782+0000 7f8e21de3b80 20 client.17949 trim_cache size 0 max 16384 2024-06-10T01:10:43.786 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T01:10:43.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.782+0000 7f8e17560700 1 -- 172.21.15.37:0/197753112 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8dfc0a2ca0 msgr2=0x7f8dfc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:43.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.782+0000 7f8e17560700 1 --2- 172.21.15.37:0/197753112 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8dfc0a2ca0 0x7f8dfc0a5160 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8e08009510 tx=0x7f8e08007580 comp rx=0 tx=0).stop 2024-06-10T01:10:43.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.782+0000 7f8e17560700 1 -- 172.21.15.37:0/197753112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e10100a80 msgr2=0x7f8e10101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:43.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.782+0000 7f8e17560700 1 --2- 172.21.15.37:0/197753112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e10100a80 0x7f8e10101c60 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c00d950 tx=0x7f8e0c00dc60 comp rx=0 tx=0).stop 2024-06-10T01:10:43.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.782+0000 7f8e17560700 1 -- 172.21.15.37:0/197753112 shutdown_connections 2024-06-10T01:10:43.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.782+0000 7f8e17560700 1 --2- 172.21.15.37:0/197753112 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8dfc0a2ca0 0x7f8dfc0a5160 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.782+0000 7f8e17560700 1 --2- 172.21.15.37:0/197753112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e10108510 0x7f8e101026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.782+0000 7f8e17560700 1 --2- 172.21.15.37:0/197753112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e10100a80 0x7f8e10101c60 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.782+0000 7f8e17560700 1 --2- 172.21.15.37:0/197753112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e10101440 0x7f8e101021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:43.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.782+0000 7f8e17560700 1 -- 172.21.15.37:0/197753112 >> 172.21.15.37:0/197753112 conn(0x7f8e10077a00 msgr2=0x7f8e100ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:43.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.782+0000 7f8e17560700 1 -- 172.21.15.37:0/197753112 shutdown_connections 2024-06-10T01:10:43.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:43.783+0000 7f8e17560700 1 -- 172.21.15.37:0/197753112 wait complete. 2024-06-10T01:10:44.465 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-10T01:10:44.489 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:44 smithi037 ceph-mon[32192]: pgmap v856: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:10:44.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:44 smithi167 ceph-mon[36917]: pgmap v856: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:10:44.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:44 smithi094 ceph-mon[36708]: pgmap v856: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:10:44.718 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:10:46.451 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:46 smithi037 ceph-mon[32192]: pgmap v857: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-10T01:10:46.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.464+0000 7fee5cb6b700 1 -- 172.21.15.37:0/560425972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee58106720 msgr2=0x7fee5810e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:46.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.464+0000 7fee5cb6b700 1 --2- 172.21.15.37:0/560425972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee58106720 0x7fee5810e2c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fee5000b0d0 tx=0x7fee5000b3e0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.464+0000 7fee5cb6b700 1 -- 172.21.15.37:0/560425972 shutdown_connections 2024-06-10T01:10:46.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.464+0000 7fee5cb6b700 1 --2- 172.21.15.37:0/560425972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee58106720 0x7fee5810e2c0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.464+0000 7fee5cb6b700 1 --2- 172.21.15.37:0/560425972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee581019d0 0x7fee58101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.464+0000 7fee5cb6b700 1 --2- 172.21.15.37:0/560425972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee58102380 0x7fee58105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.464+0000 7fee5cb6b700 1 -- 172.21.15.37:0/560425972 >> 172.21.15.37:0/560425972 conn(0x7fee58078340 msgr2=0x7fee58078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:46.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.465+0000 7fee5cb6b700 1 -- 172.21.15.37:0/560425972 shutdown_connections 2024-06-10T01:10:46.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.465+0000 7fee5cb6b700 1 -- 172.21.15.37:0/560425972 wait complete. 2024-06-10T01:10:46.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.465+0000 7fee5cb6b700 1 Processor -- start 2024-06-10T01:10:46.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.466+0000 7fee5cb6b700 1 -- start start 2024-06-10T01:10:46.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.466+0000 7fee5cb6b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee581019d0 0x7fee58199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:46.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.466+0000 7fee5cb6b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee58102380 0x7fee58199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:46.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.466+0000 7fee5cb6b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee58106720 0x7fee581a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:46.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.466+0000 7fee5cb6b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fee581116e0 con 0x7fee58102380 2024-06-10T01:10:46.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.466+0000 7fee5cb6b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fee58111560 con 0x7fee58106720 2024-06-10T01:10:46.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.466+0000 7fee5cb6b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fee58111860 con 0x7fee581019d0 2024-06-10T01:10:46.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.466+0000 7fee55d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee58102380 0x7fee58199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:46.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.466+0000 7fee55d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee58102380 0x7fee58199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59352/0 (socket says 172.21.15.37:59352) 2024-06-10T01:10:46.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.466+0000 7fee55d9b700 1 -- 172.21.15.37:0/3877163265 learned_addr learned my addr 172.21.15.37:0/3877163265 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:46.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.466+0000 7fee5659c700 1 --2- 172.21.15.37:0/3877163265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee581019d0 0x7fee58199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:46.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.466+0000 7fee56d9d700 1 --2- 172.21.15.37:0/3877163265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee58106720 0x7fee581a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:46.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.466+0000 7fee55d9b700 1 -- 172.21.15.37:0/3877163265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee581019d0 msgr2=0x7fee58199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:46.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.466+0000 7fee55d9b700 1 --2- 172.21.15.37:0/3877163265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee581019d0 0x7fee58199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.466+0000 7fee55d9b700 1 -- 172.21.15.37:0/3877163265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee58106720 msgr2=0x7fee581a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:46.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.466+0000 7fee55d9b700 1 --2- 172.21.15.37:0/3877163265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee58106720 0x7fee581a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.466+0000 7fee55d9b700 1 -- 172.21.15.37:0/3877163265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee581a5040 con 0x7fee58102380 2024-06-10T01:10:46.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.467+0000 7fee55d9b700 1 --2- 172.21.15.37:0/3877163265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee58102380 0x7fee58199fc0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fee4c00e940 tx=0x7fee4c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:46.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.467+0000 7fee477fe700 1 -- 172.21.15.37:0/3877163265 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c00cbd0 con 0x7fee58102380 2024-06-10T01:10:46.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.467+0000 7fee5cb6b700 1 -- 172.21.15.37:0/3877163265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee581a5210 con 0x7fee58102380 2024-06-10T01:10:46.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.467+0000 7fee477fe700 1 -- 172.21.15.37:0/3877163265 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee4c00ee90 con 0x7fee58102380 2024-06-10T01:10:46.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.467+0000 7fee477fe700 1 -- 172.21.15.37:0/3877163265 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c018660 con 0x7fee58102380 2024-06-10T01:10:46.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.467+0000 7fee5cb6b700 1 -- 172.21.15.37:0/3877163265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee581a5750 con 0x7fee58102380 2024-06-10T01:10:46.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.469+0000 7fee477fe700 1 -- 172.21.15.37:0/3877163265 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fee4c010430 con 0x7fee58102380 2024-06-10T01:10:46.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.470+0000 7fee477fe700 1 --2- 172.21.15.37:0/3877163265 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fee400a2c20 0x7fee400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:46.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.470+0000 7fee5659c700 1 --2- 172.21.15.37:0/3877163265 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fee400a2c20 0x7fee400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:46.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.470+0000 7fee477fe700 1 -- 172.21.15.37:0/3877163265 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7fee4c0cf680 con 0x7fee58102380 2024-06-10T01:10:46.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.470+0000 7fee5659c700 1 --2- 172.21.15.37:0/3877163265 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fee400a2c20 0x7fee400a50e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fee48007be0 tx=0x7fee480073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:46.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.471+0000 7fee673eeb80 1 Processor -- start 2024-06-10T01:10:46.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.471+0000 7fee673eeb80 1 -- start start 2024-06-10T01:10:46.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.471+0000 7fee673eeb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9dfc7ff60 0x55d9dfc80340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:46.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.471+0000 7fee673eeb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9dfc80880 0x55d9dfc7a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:46.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.471+0000 7fee673eeb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9dfc7a7a0 0x55d9dfc7abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:46.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.471+0000 7fee673eeb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d9dfb6cf60 con 0x55d9dfc7ff60 2024-06-10T01:10:46.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.471+0000 7fee673eeb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d9dfbcb850 con 0x55d9dfc80880 2024-06-10T01:10:46.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.471+0000 7fee673eeb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d9dfb33460 con 0x55d9dfc7a7a0 2024-06-10T01:10:46.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.471+0000 7fee5659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9dfc7ff60 0x55d9dfc80340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:46.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.471+0000 7fee5659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9dfc7ff60 0x55d9dfc80340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59362/0 (socket says 172.21.15.37:59362) 2024-06-10T01:10:46.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.471+0000 7fee5659c700 1 -- 172.21.15.37:0/306246616 learned_addr learned my addr 172.21.15.37:0/306246616 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:46.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.471+0000 7fee56d9d700 1 --2- 172.21.15.37:0/306246616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9dfc7a7a0 0x55d9dfc7abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:46.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.471+0000 7fee55d9b700 1 --2- 172.21.15.37:0/306246616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9dfc80880 0x55d9dfc7a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:46.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.472+0000 7fee5659c700 1 -- 172.21.15.37:0/306246616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9dfc7a7a0 msgr2=0x55d9dfc7abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:46.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.472+0000 7fee5659c700 1 --2- 172.21.15.37:0/306246616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9dfc7a7a0 0x55d9dfc7abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.472+0000 7fee5659c700 1 -- 172.21.15.37:0/306246616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9dfc80880 msgr2=0x55d9dfc7a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:46.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.472+0000 7fee5659c700 1 --2- 172.21.15.37:0/306246616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9dfc80880 0x55d9dfc7a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.472+0000 7fee5659c700 1 -- 172.21.15.37:0/306246616 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9dfb1a8d0 con 0x55d9dfc7ff60 2024-06-10T01:10:46.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.472+0000 7fee55d9b700 1 --2- 172.21.15.37:0/306246616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9dfc80880 0x55d9dfc7a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:10:46.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.472+0000 7fee5659c700 1 --2- 172.21.15.37:0/306246616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9dfc7ff60 0x55d9dfc80340 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fee48010e80 tx=0x7fee48013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237088a554749658 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:46.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.472+0000 7fee44ff9700 1 -- 172.21.15.37:0/306246616 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48024070 con 0x55d9dfc7ff60 2024-06-10T01:10:46.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.472+0000 7fee44ff9700 1 -- 172.21.15.37:0/306246616 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee48015070 con 0x55d9dfc7ff60 2024-06-10T01:10:46.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.472+0000 7fee44ff9700 1 -- 172.21.15.37:0/306246616 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48011a30 con 0x55d9dfc7ff60 2024-06-10T01:10:46.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.473+0000 7fee673eeb80 1 -- 172.21.15.37:0/306246616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9dfc7ff60 msgr2=0x55d9dfc80340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:46.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.473+0000 7fee673eeb80 1 --2- 172.21.15.37:0/306246616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9dfc7ff60 0x55d9dfc80340 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fee48010e80 tx=0x7fee48013000 comp rx=0 tx=0).stop 2024-06-10T01:10:46.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.473+0000 7fee673eeb80 1 -- 172.21.15.37:0/306246616 shutdown_connections 2024-06-10T01:10:46.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.473+0000 7fee673eeb80 1 --2- 172.21.15.37:0/306246616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9dfc7a7a0 0x55d9dfc7abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.473+0000 7fee673eeb80 1 --2- 172.21.15.37:0/306246616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9dfc7ff60 0x55d9dfc80340 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.473+0000 7fee673eeb80 1 --2- 172.21.15.37:0/306246616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9dfc80880 0x55d9dfc7a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.473+0000 7fee673eeb80 1 -- 172.21.15.37:0/306246616 >> 172.21.15.37:0/306246616 conn(0x55d9dfba7080 msgr2=0x55d9dfc75220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.473+0000 7fee673eeb80 1 -- 172.21.15.37:0/306246616 shutdown_connections 2024-06-10T01:10:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.473+0000 7fee673eeb80 1 -- 172.21.15.37:0/306246616 wait complete. 2024-06-10T01:10:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.473+0000 7fee673eeb80 1 -- start start 2024-06-10T01:10:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.473+0000 7fee673eeb80 1 Processor -- start 2024-06-10T01:10:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.474+0000 7fee673eeb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9dfc7a7a0 0x55d9dfc86610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.474+0000 7fee673eeb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9dfc7ff60 0x55d9dfc83910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:46.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.474+0000 7fee673eeb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9dfc80880 0x55d9dfc83e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:46.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.474+0000 7fee673eeb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d9dfb6bfc0 con 0x55d9dfc7a7a0 2024-06-10T01:10:46.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.474+0000 7fee673eeb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d9dfbc7810 con 0x55d9dfc80880 2024-06-10T01:10:46.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.474+0000 7fee673eeb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d9dfc87260 con 0x55d9dfc7ff60 2024-06-10T01:10:46.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.474+0000 7fee673eeb80 10 client.0 _ll_register_callbacks cb 0x55d9dfb615b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:46.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.474+0000 7fee5659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9dfc7a7a0 0x55d9dfc86610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:46.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.474+0000 7fee5659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9dfc7a7a0 0x55d9dfc86610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59368/0 (socket says 172.21.15.37:59368) 2024-06-10T01:10:46.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.474+0000 7fee5659c700 1 -- 172.21.15.37:0/806045334 learned_addr learned my addr 172.21.15.37:0/806045334 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:46.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.474+0000 7fee55d9b700 1 --2- 172.21.15.37:0/806045334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9dfc7ff60 0x55d9dfc83910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:46.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.474+0000 7fee56d9d700 1 --2- 172.21.15.37:0/806045334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9dfc80880 0x55d9dfc83e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:46.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.474+0000 7fee5659c700 1 -- 172.21.15.37:0/806045334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9dfc7ff60 msgr2=0x55d9dfc83910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:46.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.474+0000 7fee5659c700 1 --2- 172.21.15.37:0/806045334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9dfc7ff60 0x55d9dfc83910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.474+0000 7fee5659c700 1 -- 172.21.15.37:0/806045334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9dfc80880 msgr2=0x55d9dfc83e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:46.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.474+0000 7fee5659c700 1 --2- 172.21.15.37:0/806045334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9dfc80880 0x55d9dfc83e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.474+0000 7fee5659c700 1 -- 172.21.15.37:0/806045334 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9dfb91750 con 0x55d9dfc7a7a0 2024-06-10T01:10:46.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.474+0000 7fee55d9b700 1 --2- 172.21.15.37:0/806045334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9dfc7ff60 0x55d9dfc83910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:10:46.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.474+0000 7fee5659c700 1 --2- 172.21.15.37:0/806045334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9dfc7a7a0 0x55d9dfc86610 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fee480115b0 tx=0x7fee480114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7027096a6549990 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:46.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.474+0000 7fee3dffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:10:46.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.474+0000 7fee3dffb700 1 -- 172.21.15.37:0/806045334 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48024070 con 0x55d9dfc7a7a0 2024-06-10T01:10:46.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.474+0000 7fee3dffb700 1 -- 172.21.15.37:0/806045334 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7fee48003a90 con 0x55d9dfc7a7a0 2024-06-10T01:10:46.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.474+0000 7fee673eeb80 1 -- 172.21.15.37:0/806045334 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee48015070 con 0x55d9dfc7a7a0 2024-06-10T01:10:46.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.475+0000 7fee3dffb700 1 -- 172.21.15.37:0/806045334 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fee48038050 con 0x55d9dfc7a7a0 2024-06-10T01:10:46.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.475+0000 7fee673eeb80 10 client.17961 fetch_fsmap learned FSMap version 35 2024-06-10T01:10:46.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.475+0000 7fee673eeb80 1 -- 172.21.15.37:0/806045334 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d9dfc91a10 con 0x55d9dfc7a7a0 2024-06-10T01:10:46.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.475+0000 7fee3dffb700 1 -- 172.21.15.37:0/806045334 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee48015070 con 0x55d9dfc7a7a0 2024-06-10T01:10:46.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.475+0000 7fee3dffb700 1 -- 172.21.15.37:0/806045334 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fee48033030 con 0x55d9dfc7a7a0 2024-06-10T01:10:46.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.475+0000 7fee673eeb80 10 client.17961 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:10:46.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.475+0000 7fee673eeb80 10 client.17961 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-06-10T01:10:46.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.475+0000 7fee673eeb80 20 client.17961 populate_metadata read hostname 'smithi037' 2024-06-10T01:10:46.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.475+0000 7fee673eeb80 1 --2- 172.21.15.37:0/806045334 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55d9dfc93f90 0x55d9dfc96380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:46.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.475+0000 7fee673eeb80 4 client.17961 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:10:46.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.475+0000 7fee673eeb80 1 -- 172.21.15.37:0/806045334 --> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9dfbd8d50 con 0x55d9dfc93f90 2024-06-10T01:10:46.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.475+0000 7fee55d9b700 1 --2- 172.21.15.37:0/806045334 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55d9dfc93f90 0x55d9dfc96380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:46.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.476+0000 7fee55d9b700 1 --2- 172.21.15.37:0/806045334 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55d9dfc93f90 0x55d9dfc96380 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9cea9040e3a1624a server_cookie=589bdd415f47daec in_seq=0 out_seq=0 2024-06-10T01:10:46.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.476+0000 7fee3dffb700 10 client.17961 ms_handle_connect on v2:172.21.15.94:6838/946269331 2024-06-10T01:10:46.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.477+0000 7fee3dffb700 1 -- 172.21.15.37:0/806045334 <== mds.1 v2:172.21.15.94:6838/946269331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d9dfbd8d50 con 0x55d9dfc93f90 2024-06-10T01:10:46.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.477+0000 7fee3dffb700 10 client.17961 handle_command_reply: tid=0 2024-06-10T01:10:46.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.477+0000 7fee673eeb80 1 client.17961 shutdown 2024-06-10T01:10:46.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.477+0000 7fee673eeb80 1 -- 172.21.15.37:0/806045334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9dfc7a7a0 msgr2=0x55d9dfc86610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:46.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.477+0000 7fee673eeb80 1 --2- 172.21.15.37:0/806045334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9dfc7a7a0 0x55d9dfc86610 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fee480115b0 tx=0x7fee480114b0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.477+0000 7fee673eeb80 1 -- 172.21.15.37:0/806045334 shutdown_connections 2024-06-10T01:10:46.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.477+0000 7fee673eeb80 1 --2- 172.21.15.37:0/806045334 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55d9dfc93f90 0x55d9dfc96380 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.477+0000 7fee673eeb80 1 --2- 172.21.15.37:0/806045334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9dfc80880 0x55d9dfc83e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.477+0000 7fee673eeb80 1 --2- 172.21.15.37:0/806045334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9dfc7a7a0 0x55d9dfc86610 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.477+0000 7fee673eeb80 1 --2- 172.21.15.37:0/806045334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9dfc7ff60 0x55d9dfc83910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.477+0000 7fee673eeb80 1 -- 172.21.15.37:0/806045334 >> 172.21.15.37:0/806045334 conn(0x55d9dfba7080 msgr2=0x55d9dfc74700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:46.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.477+0000 7fee3dffb700 0 client.17961 ms_handle_reset on v2:172.21.15.94:6838/946269331 2024-06-10T01:10:46.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.477+0000 7fee673eeb80 1 -- 172.21.15.37:0/806045334 shutdown_connections 2024-06-10T01:10:46.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.477+0000 7fee673eeb80 1 -- 172.21.15.37:0/806045334 wait complete. 2024-06-10T01:10:46.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.477+0000 7fee673eeb80 20 client.17961 trim_cache size 0 max 16384 2024-06-10T01:10:46.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.492+0000 7fee673eeb80 1 Processor -- start 2024-06-10T01:10:46.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.492+0000 7fee673eeb80 1 -- start start 2024-06-10T01:10:46.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.492+0000 7fee673eeb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9dfba7080 0x55d9dfc7a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:46.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.492+0000 7fee673eeb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9dfc7a7a0 0x55d9dfc85850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:46.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.493+0000 7fee673eeb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9dfc7ff60 0x55d9dfc74e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:46.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.493+0000 7fee673eeb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fee581b0e10 con 0x55d9dfc7ff60 2024-06-10T01:10:46.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.493+0000 7fee673eeb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fee581ae6a0 con 0x55d9dfba7080 2024-06-10T01:10:46.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.493+0000 7fee673eeb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fee581ac4f0 con 0x55d9dfc7a7a0 2024-06-10T01:10:46.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.493+0000 7fee56d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9dfc7ff60 0x55d9dfc74e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:46.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.493+0000 7fee56d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9dfc7ff60 0x55d9dfc74e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59376/0 (socket says 172.21.15.37:59376) 2024-06-10T01:10:46.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.493+0000 7fee56d9d700 1 -- 172.21.15.37:0/1629112798 learned_addr learned my addr 172.21.15.37:0/1629112798 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:46.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.493+0000 7fee5659c700 1 --2- 172.21.15.37:0/1629112798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9dfba7080 0x55d9dfc7a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:46.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.493+0000 7fee56d9d700 1 -- 172.21.15.37:0/1629112798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9dfc7a7a0 msgr2=0x55d9dfc85850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:46.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.493+0000 7fee55d9b700 1 --2- 172.21.15.37:0/1629112798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9dfc7a7a0 0x55d9dfc85850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:46.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.493+0000 7fee56d9d700 1 --2- 172.21.15.37:0/1629112798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9dfc7a7a0 0x55d9dfc85850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.493+0000 7fee56d9d700 1 -- 172.21.15.37:0/1629112798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9dfba7080 msgr2=0x55d9dfc7a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:46.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.493+0000 7fee56d9d700 1 --2- 172.21.15.37:0/1629112798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9dfba7080 0x55d9dfc7a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.493+0000 7fee56d9d700 1 -- 172.21.15.37:0/1629112798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9dfc71810 con 0x55d9dfc7ff60 2024-06-10T01:10:46.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.493+0000 7fee5659c700 1 --2- 172.21.15.37:0/1629112798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9dfba7080 0x55d9dfc7a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:10:46.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.493+0000 7fee56d9d700 1 --2- 172.21.15.37:0/1629112798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9dfc7ff60 0x55d9dfc74e80 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fee50009520 tx=0x7fee50009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39203f2583a32f3c server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:46.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.493+0000 7fee3ffff700 1 -- 172.21.15.37:0/1629112798 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5000bca0 con 0x55d9dfc7ff60 2024-06-10T01:10:46.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.493+0000 7fee3ffff700 1 -- 172.21.15.37:0/1629112798 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee5000be40 con 0x55d9dfc7ff60 2024-06-10T01:10:46.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.493+0000 7fee3ffff700 1 -- 172.21.15.37:0/1629112798 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee50026480 con 0x55d9dfc7ff60 2024-06-10T01:10:46.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.494+0000 7fee673eeb80 1 -- 172.21.15.37:0/1629112798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9dfc7ff60 msgr2=0x55d9dfc74e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:46.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.494+0000 7fee673eeb80 1 --2- 172.21.15.37:0/1629112798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9dfc7ff60 0x55d9dfc74e80 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fee50009520 tx=0x7fee50009550 comp rx=0 tx=0).stop 2024-06-10T01:10:46.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.494+0000 7fee673eeb80 1 -- 172.21.15.37:0/1629112798 shutdown_connections 2024-06-10T01:10:46.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.494+0000 7fee673eeb80 1 --2- 172.21.15.37:0/1629112798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9dfc7ff60 0x55d9dfc74e80 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.494+0000 7fee673eeb80 1 --2- 172.21.15.37:0/1629112798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9dfba7080 0x55d9dfc7a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.494+0000 7fee673eeb80 1 --2- 172.21.15.37:0/1629112798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9dfc7a7a0 0x55d9dfc85850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.494+0000 7fee673eeb80 1 -- 172.21.15.37:0/1629112798 >> 172.21.15.37:0/1629112798 conn(0x55d9dfc7e1e0 msgr2=0x55d9dfc85140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:46.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.494+0000 7fee673eeb80 1 -- 172.21.15.37:0/1629112798 shutdown_connections 2024-06-10T01:10:46.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.494+0000 7fee673eeb80 1 -- 172.21.15.37:0/1629112798 wait complete. 2024-06-10T01:10:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.494+0000 7fee673eeb80 1 -- start start 2024-06-10T01:10:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.494+0000 7fee673eeb80 1 Processor -- start 2024-06-10T01:10:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.495+0000 7fee673eeb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9dfba7080 0x55d9dfcf6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.495+0000 7fee673eeb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9dfc7a7a0 0x55d9dfcf7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.495+0000 7fee673eeb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9dfc7ff60 0x55d9dfc97fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.495+0000 7fee673eeb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fee581acb60 con 0x55d9dfc7ff60 2024-06-10T01:10:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.495+0000 7fee673eeb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fee581adfd0 con 0x55d9dfba7080 2024-06-10T01:10:46.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.495+0000 7fee673eeb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fee581ad230 con 0x55d9dfc7a7a0 2024-06-10T01:10:46.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.495+0000 7fee673eeb80 10 client.0 _ll_register_callbacks cb 0x55d9dfb615b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:46.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.495+0000 7fee56d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9dfc7ff60 0x55d9dfc97fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:46.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.495+0000 7fee56d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9dfc7ff60 0x55d9dfc97fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59392/0 (socket says 172.21.15.37:59392) 2024-06-10T01:10:46.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.495+0000 7fee56d9d700 1 -- 172.21.15.37:0/2297270556 learned_addr learned my addr 172.21.15.37:0/2297270556 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:46.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.495+0000 7fee55d9b700 1 --2- 172.21.15.37:0/2297270556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9dfc7a7a0 0x55d9dfcf7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:46.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.495+0000 7fee5659c700 1 --2- 172.21.15.37:0/2297270556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9dfba7080 0x55d9dfcf6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:46.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.495+0000 7fee56d9d700 1 -- 172.21.15.37:0/2297270556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9dfc7a7a0 msgr2=0x55d9dfcf7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:46.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.495+0000 7fee56d9d700 1 --2- 172.21.15.37:0/2297270556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9dfc7a7a0 0x55d9dfcf7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.495+0000 7fee56d9d700 1 -- 172.21.15.37:0/2297270556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9dfba7080 msgr2=0x55d9dfcf6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:46.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.495+0000 7fee56d9d700 1 --2- 172.21.15.37:0/2297270556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9dfba7080 0x55d9dfcf6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.495+0000 7fee56d9d700 1 -- 172.21.15.37:0/2297270556 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9dfc98670 con 0x55d9dfc7ff60 2024-06-10T01:10:46.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.495+0000 7fee5659c700 1 --2- 172.21.15.37:0/2297270556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9dfba7080 0x55d9dfcf6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:10:46.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.495+0000 7fee56d9d700 1 --2- 172.21.15.37:0/2297270556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9dfc7ff60 0x55d9dfc97fd0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fee50009dd0 tx=0x7fee5000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3ec68167bc774b server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:46.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.495+0000 7fee3dffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:10:46.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.495+0000 7fee3dffb700 1 -- 172.21.15.37:0/2297270556 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5002c070 con 0x55d9dfc7ff60 2024-06-10T01:10:46.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.495+0000 7fee3dffb700 1 -- 172.21.15.37:0/2297270556 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7fee50004c50 con 0x55d9dfc7ff60 2024-06-10T01:10:46.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.495+0000 7fee673eeb80 1 -- 172.21.15.37:0/2297270556 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d9dfbd32d0 con 0x55d9dfc7ff60 2024-06-10T01:10:46.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.496+0000 7fee3dffb700 1 -- 172.21.15.37:0/2297270556 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fee50038050 con 0x55d9dfc7ff60 2024-06-10T01:10:46.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.496+0000 7fee673eeb80 10 client.17967 fetch_fsmap learned FSMap version 35 2024-06-10T01:10:46.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.496+0000 7fee673eeb80 1 -- 172.21.15.37:0/2297270556 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d9dfc998c0 con 0x55d9dfc7ff60 2024-06-10T01:10:46.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.496+0000 7fee3dffb700 1 -- 172.21.15.37:0/2297270556 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d9dfbd32d0 con 0x55d9dfc7ff60 2024-06-10T01:10:46.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.496+0000 7fee3dffb700 1 -- 172.21.15.37:0/2297270556 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fee50011070 con 0x55d9dfc7ff60 2024-06-10T01:10:46.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.496+0000 7fee673eeb80 10 client.17967 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:10:46.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.496+0000 7fee673eeb80 10 client.17967 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-06-10T01:10:46.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.496+0000 7fee673eeb80 20 client.17967 populate_metadata read hostname 'smithi037' 2024-06-10T01:10:46.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.496+0000 7fee673eeb80 1 --2- 172.21.15.37:0/2297270556 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55d9dfc80880 0x55d9dfd04d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:46.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.496+0000 7fee673eeb80 4 client.17967 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:10:46.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.496+0000 7fee673eeb80 1 -- 172.21.15.37:0/2297270556 --> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d9dfb86f50 con 0x55d9dfc80880 2024-06-10T01:10:46.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.496+0000 7fee5659c700 1 --2- 172.21.15.37:0/2297270556 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55d9dfc80880 0x55d9dfd04d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:46.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.497+0000 7fee3dffb700 10 client.17967 ms_handle_connect on v2:172.21.15.94:6838/946269331 2024-06-10T01:10:46.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.497+0000 7fee5659c700 1 --2- 172.21.15.37:0/2297270556 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55d9dfc80880 0x55d9dfd04d80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2dcf0875af2d44c0 server_cookie=63bac949690038a4 in_seq=0 out_seq=0 2024-06-10T01:10:46.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.497+0000 7fee3dffb700 1 -- 172.21.15.37:0/2297270556 <== mds.1 v2:172.21.15.94:6838/946269331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d9dfb86f50 con 0x55d9dfc80880 2024-06-10T01:10:46.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.497+0000 7fee3dffb700 10 client.17967 handle_command_reply: tid=0 2024-06-10T01:10:46.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.497+0000 7fee673eeb80 1 client.17967 shutdown 2024-06-10T01:10:46.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.497+0000 7fee673eeb80 1 -- 172.21.15.37:0/2297270556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9dfc7ff60 msgr2=0x55d9dfc97fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:46.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.497+0000 7fee673eeb80 1 --2- 172.21.15.37:0/2297270556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9dfc7ff60 0x55d9dfc97fd0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fee50009dd0 tx=0x7fee5000bdd0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.497+0000 7fee673eeb80 1 -- 172.21.15.37:0/2297270556 shutdown_connections 2024-06-10T01:10:46.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.497+0000 7fee673eeb80 1 --2- 172.21.15.37:0/2297270556 >> [v2:172.21.15.94:6838/946269331,v1:172.21.15.94:6839/946269331] conn(0x55d9dfc80880 0x55d9dfd04d80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.497+0000 7fee673eeb80 1 --2- 172.21.15.37:0/2297270556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9dfc7ff60 0x55d9dfc97fd0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.497+0000 7fee673eeb80 1 --2- 172.21.15.37:0/2297270556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d9dfba7080 0x55d9dfcf6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.497+0000 7fee673eeb80 1 --2- 172.21.15.37:0/2297270556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9dfc7a7a0 0x55d9dfcf7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.497+0000 7fee3dffb700 0 client.17967 ms_handle_reset on v2:172.21.15.94:6838/946269331 2024-06-10T01:10:46.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.497+0000 7fee673eeb80 1 -- 172.21.15.37:0/2297270556 >> 172.21.15.37:0/2297270556 conn(0x55d9dfc7e1e0 msgr2=0x55d9dfc7ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:46.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.497+0000 7fee673eeb80 1 -- 172.21.15.37:0/2297270556 shutdown_connections 2024-06-10T01:10:46.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.497+0000 7fee673eeb80 1 -- 172.21.15.37:0/2297270556 wait complete. 2024-06-10T01:10:46.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.497+0000 7fee673eeb80 20 client.17967 trim_cache size 0 max 16384 2024-06-10T01:10:46.509 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T01:10:46.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.498+0000 7fee5cb6b700 1 -- 172.21.15.37:0/3877163265 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fee400a2c20 msgr2=0x7fee400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:46.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.498+0000 7fee5cb6b700 1 --2- 172.21.15.37:0/3877163265 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fee400a2c20 0x7fee400a50e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fee48007be0 tx=0x7fee480073d0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.498+0000 7fee5cb6b700 1 -- 172.21.15.37:0/3877163265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee58102380 msgr2=0x7fee58199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:46.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.498+0000 7fee5cb6b700 1 --2- 172.21.15.37:0/3877163265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee58102380 0x7fee58199fc0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fee4c00e940 tx=0x7fee4c00ec50 comp rx=0 tx=0).stop 2024-06-10T01:10:46.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.498+0000 7fee5cb6b700 1 -- 172.21.15.37:0/3877163265 shutdown_connections 2024-06-10T01:10:46.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.498+0000 7fee5cb6b700 1 --2- 172.21.15.37:0/3877163265 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fee400a2c20 0x7fee400a50e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.498+0000 7fee5cb6b700 1 --2- 172.21.15.37:0/3877163265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee58106720 0x7fee581a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.498+0000 7fee5cb6b700 1 --2- 172.21.15.37:0/3877163265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee581019d0 0x7fee58199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.498+0000 7fee5cb6b700 1 --2- 172.21.15.37:0/3877163265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee58102380 0x7fee58199fc0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:46.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.498+0000 7fee5cb6b700 1 -- 172.21.15.37:0/3877163265 >> 172.21.15.37:0/3877163265 conn(0x7fee58078340 msgr2=0x7fee580ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:46.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.498+0000 7fee5cb6b700 1 -- 172.21.15.37:0/3877163265 shutdown_connections 2024-06-10T01:10:46.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:46.498+0000 7fee5cb6b700 1 -- 172.21.15.37:0/3877163265 wait complete. 2024-06-10T01:10:46.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:46 smithi167 ceph-mon[36917]: pgmap v857: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-10T01:10:46.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:46 smithi094 ceph-mon[36708]: pgmap v857: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-10T01:10:47.212 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-10T01:10:47.510 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:10:48.475 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:48 smithi037 ceph-mon[32192]: pgmap v858: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-10T01:10:48.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:48 smithi167 ceph-mon[36917]: pgmap v858: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-10T01:10:48.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:48 smithi094 ceph-mon[36708]: pgmap v858: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-10T01:10:50.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:50 smithi167 ceph-mon[36917]: pgmap v859: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-06-10T01:10:50.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:50 smithi094 ceph-mon[36708]: pgmap v859: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-06-10T01:10:50.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:50 smithi037 ceph-mon[32192]: pgmap v859: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-06-10T01:10:52.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.237+0000 7f9f7d0c2700 1 -- 172.21.15.37:0/2525473226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f781085d0 msgr2=0x7f9f781089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:52.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.237+0000 7f9f7d0c2700 1 --2- 172.21.15.37:0/2525473226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f781085d0 0x7f9f781089b0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f9f680099c0 tx=0x7f9f68009cd0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.237+0000 7f9f7d0c2700 1 -- 172.21.15.37:0/2525473226 shutdown_connections 2024-06-10T01:10:52.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.237+0000 7f9f7d0c2700 1 --2- 172.21.15.37:0/2525473226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f78102ef0 0x7f9f7810f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.237+0000 7f9f7d0c2700 1 --2- 172.21.15.37:0/2525473226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f781085d0 0x7f9f781089b0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.237+0000 7f9f7d0c2700 1 --2- 172.21.15.37:0/2525473226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f78102530 0x7f9f781029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.237+0000 7f9f7d0c2700 1 -- 172.21.15.37:0/2525473226 >> 172.21.15.37:0/2525473226 conn(0x7f9f78079640 msgr2=0x7f9f78079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:52.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.237+0000 7f9f7d0c2700 1 -- 172.21.15.37:0/2525473226 shutdown_connections 2024-06-10T01:10:52.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.237+0000 7f9f7d0c2700 1 -- 172.21.15.37:0/2525473226 wait complete. 2024-06-10T01:10:52.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.238+0000 7f9f7d0c2700 1 Processor -- start 2024-06-10T01:10:52.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.238+0000 7f9f7d0c2700 1 -- start start 2024-06-10T01:10:52.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.239+0000 7f9f7d0c2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f78102530 0x7f9f7819e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:52.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.239+0000 7f9f7d0c2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f78102ef0 0x7f9f7819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:52.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.239+0000 7f9f7d0c2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f781085d0 0x7f9f78198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:52.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.239+0000 7f9f7d0c2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9f78112860 con 0x7f9f78102ef0 2024-06-10T01:10:52.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.239+0000 7f9f7d0c2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9f781126e0 con 0x7f9f78102530 2024-06-10T01:10:52.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.239+0000 7f9f7d0c2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9f781129e0 con 0x7f9f781085d0 2024-06-10T01:10:52.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.239+0000 7f9f7659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f78102ef0 0x7f9f7819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:52.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.239+0000 7f9f7659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f78102ef0 0x7f9f7819ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60094/0 (socket says 172.21.15.37:60094) 2024-06-10T01:10:52.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.239+0000 7f9f7659c700 1 -- 172.21.15.37:0/1962755571 learned_addr learned my addr 172.21.15.37:0/1962755571 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:52.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.239+0000 7f9f76d9d700 1 --2- 172.21.15.37:0/1962755571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f78102530 0x7f9f7819e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:52.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.239+0000 7f9f7759e700 1 --2- 172.21.15.37:0/1962755571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f781085d0 0x7f9f78198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:52.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.239+0000 7f9f7659c700 1 -- 172.21.15.37:0/1962755571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f781085d0 msgr2=0x7f9f78198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:52.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.239+0000 7f9f7659c700 1 --2- 172.21.15.37:0/1962755571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f781085d0 0x7f9f78198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.239+0000 7f9f7659c700 1 -- 172.21.15.37:0/1962755571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f78102530 msgr2=0x7f9f7819e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:52.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.239+0000 7f9f7659c700 1 --2- 172.21.15.37:0/1962755571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f78102530 0x7f9f7819e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.239+0000 7f9f7659c700 1 -- 172.21.15.37:0/1962755571 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f781991e0 con 0x7f9f78102ef0 2024-06-10T01:10:52.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.239+0000 7f9f7659c700 1 --2- 172.21.15.37:0/1962755571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f78102ef0 0x7f9f7819ee00 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f9f6000e9a0 tx=0x7f9f6000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:52.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.239+0000 7f9f6ffff700 1 -- 172.21.15.37:0/1962755571 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6000ca10 con 0x7f9f78102ef0 2024-06-10T01:10:52.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.239+0000 7f9f6ffff700 1 -- 172.21.15.37:0/1962755571 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f6000cbb0 con 0x7f9f78102ef0 2024-06-10T01:10:52.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.239+0000 7f9f7d0c2700 1 -- 172.21.15.37:0/1962755571 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f781994d0 con 0x7f9f78102ef0 2024-06-10T01:10:52.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.239+0000 7f9f6ffff700 1 -- 172.21.15.37:0/1962755571 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f60018680 con 0x7f9f78102ef0 2024-06-10T01:10:52.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.240+0000 7f9f7d0c2700 1 -- 172.21.15.37:0/1962755571 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f781a5790 con 0x7f9f78102ef0 2024-06-10T01:10:52.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.242+0000 7f9f6ffff700 1 -- 172.21.15.37:0/1962755571 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9f60010430 con 0x7f9f78102ef0 2024-06-10T01:10:52.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.242+0000 7f9f6ffff700 1 --2- 172.21.15.37:0/1962755571 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9f640a2c20 0x7f9f640a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:52.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.243+0000 7f9f76d9d700 1 --2- 172.21.15.37:0/1962755571 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9f640a2c20 0x7f9f640a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:52.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.243+0000 7f9f6ffff700 1 -- 172.21.15.37:0/1962755571 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7f9f600cf650 con 0x7f9f78102ef0 2024-06-10T01:10:52.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.243+0000 7f9f76d9d700 1 --2- 172.21.15.37:0/1962755571 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9f640a2c20 0x7f9f640a50e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9f68000c00 tx=0x7f9f68005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:52.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.243+0000 7f9f87945b80 1 Processor -- start 2024-06-10T01:10:52.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.243+0000 7f9f87945b80 1 -- start start 2024-06-10T01:10:52.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.244+0000 7f9f87945b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6782c640 0x55dc6782ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:52.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.244+0000 7f9f87945b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6782cf60 0x55dc678269e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:52.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.244+0000 7f9f87945b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc67826f20 0x55dc67827340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:52.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.244+0000 7f9f87945b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dc67778810 con 0x55dc6782c640 2024-06-10T01:10:52.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.244+0000 7f9f87945b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55dc67720720 con 0x55dc67826f20 2024-06-10T01:10:52.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.244+0000 7f9f87945b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dc677747d0 con 0x55dc6782cf60 2024-06-10T01:10:52.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.244+0000 7f9f76d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6782c640 0x55dc6782ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:52.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.244+0000 7f9f76d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6782c640 0x55dc6782ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60106/0 (socket says 172.21.15.37:60106) 2024-06-10T01:10:52.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.244+0000 7f9f76d9d700 1 -- 172.21.15.37:0/3870275758 learned_addr learned my addr 172.21.15.37:0/3870275758 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:52.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.244+0000 7f9f7659c700 1 --2- 172.21.15.37:0/3870275758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6782cf60 0x55dc678269e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:52.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.244+0000 7f9f7759e700 1 --2- 172.21.15.37:0/3870275758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc67826f20 0x55dc67827340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:52.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.244+0000 7f9f76d9d700 1 -- 172.21.15.37:0/3870275758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6782cf60 msgr2=0x55dc678269e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:52.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.244+0000 7f9f76d9d700 1 --2- 172.21.15.37:0/3870275758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6782cf60 0x55dc678269e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.244+0000 7f9f76d9d700 1 -- 172.21.15.37:0/3870275758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc67826f20 msgr2=0x55dc67827340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:52.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.244+0000 7f9f76d9d700 1 --2- 172.21.15.37:0/3870275758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc67826f20 0x55dc67827340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.244+0000 7f9f76d9d700 1 -- 172.21.15.37:0/3870275758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc67716f50 con 0x55dc6782c640 2024-06-10T01:10:52.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.245+0000 7f9f76d9d700 1 --2- 172.21.15.37:0/3870275758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6782c640 0x55dc6782ca20 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f9f680099c0 tx=0x7f9f68020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce80c5419664103 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:52.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.245+0000 7f9f6d7fa700 1 -- 172.21.15.37:0/3870275758 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6800f460 con 0x55dc6782c640 2024-06-10T01:10:52.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.245+0000 7f9f6d7fa700 1 -- 172.21.15.37:0/3870275758 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f6800fae0 con 0x55dc6782c640 2024-06-10T01:10:52.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.245+0000 7f9f6d7fa700 1 -- 172.21.15.37:0/3870275758 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f68024b70 con 0x55dc6782c640 2024-06-10T01:10:52.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.245+0000 7f9f87945b80 1 -- 172.21.15.37:0/3870275758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6782c640 msgr2=0x55dc6782ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:52.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.245+0000 7f9f87945b80 1 --2- 172.21.15.37:0/3870275758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6782c640 0x55dc6782ca20 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f9f680099c0 tx=0x7f9f68020000 comp rx=0 tx=0).stop 2024-06-10T01:10:52.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.246+0000 7f9f87945b80 1 -- 172.21.15.37:0/3870275758 shutdown_connections 2024-06-10T01:10:52.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.246+0000 7f9f87945b80 1 --2- 172.21.15.37:0/3870275758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc67826f20 0x55dc67827340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.246+0000 7f9f87945b80 1 --2- 172.21.15.37:0/3870275758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6782c640 0x55dc6782ca20 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.246+0000 7f9f87945b80 1 --2- 172.21.15.37:0/3870275758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6782cf60 0x55dc678269e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.246+0000 7f9f87945b80 1 -- 172.21.15.37:0/3870275758 >> 172.21.15.37:0/3870275758 conn(0x55dc67751cc0 msgr2=0x55dc67821850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:52.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.246+0000 7f9f87945b80 1 -- 172.21.15.37:0/3870275758 shutdown_connections 2024-06-10T01:10:52.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.246+0000 7f9f87945b80 1 -- 172.21.15.37:0/3870275758 wait complete. 2024-06-10T01:10:52.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.246+0000 7f9f87945b80 1 -- start start 2024-06-10T01:10:52.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.246+0000 7f9f87945b80 1 Processor -- start 2024-06-10T01:10:52.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.246+0000 7f9f87945b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc67826f20 0x55dc67832c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:52.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.246+0000 7f9f87945b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6782c640 0x55dc6782ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:52.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.246+0000 7f9f87945b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6782cf60 0x55dc678304e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:52.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.246+0000 7f9f87945b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dc67833910 con 0x55dc6782c640 2024-06-10T01:10:52.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.246+0000 7f9f87945b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55dc67718bf0 con 0x55dc6782cf60 2024-06-10T01:10:52.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.246+0000 7f9f87945b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dc67833a90 con 0x55dc67826f20 2024-06-10T01:10:52.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.246+0000 7f9f87945b80 10 client.0 _ll_register_callbacks cb 0x55dc67712920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:52.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.247+0000 7f9f7659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6782c640 0x55dc6782ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:52.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.247+0000 7f9f7659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6782c640 0x55dc6782ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60112/0 (socket says 172.21.15.37:60112) 2024-06-10T01:10:52.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.247+0000 7f9f7659c700 1 -- 172.21.15.37:0/1334533587 learned_addr learned my addr 172.21.15.37:0/1334533587 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:52.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.247+0000 7f9f76d9d700 1 --2- 172.21.15.37:0/1334533587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc67826f20 0x55dc67832c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:52.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.247+0000 7f9f7759e700 1 --2- 172.21.15.37:0/1334533587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6782cf60 0x55dc678304e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:52.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.247+0000 7f9f7659c700 1 -- 172.21.15.37:0/1334533587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc67826f20 msgr2=0x55dc67832c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:52.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.247+0000 7f9f7659c700 1 --2- 172.21.15.37:0/1334533587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc67826f20 0x55dc67832c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.247+0000 7f9f7659c700 1 -- 172.21.15.37:0/1334533587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6782cf60 msgr2=0x55dc678304e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:52.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.247+0000 7f9f7659c700 1 --2- 172.21.15.37:0/1334533587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6782cf60 0x55dc678304e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.247+0000 7f9f7659c700 1 -- 172.21.15.37:0/1334533587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc676c78d0 con 0x55dc6782c640 2024-06-10T01:10:52.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.247+0000 7f9f7659c700 1 --2- 172.21.15.37:0/1334533587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6782c640 0x55dc6782ffa0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f9f600ccd60 tx=0x7f9f600cf5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b9fbc88f9a861a7 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:52.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.247+0000 7f9f5e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:10:52.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.247+0000 7f9f5e7fc700 1 -- 172.21.15.37:0/1334533587 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f600b3020 con 0x55dc6782c640 2024-06-10T01:10:52.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.247+0000 7f9f5e7fc700 1 -- 172.21.15.37:0/1334533587 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7f9f60099dc0 con 0x55dc6782c640 2024-06-10T01:10:52.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.247+0000 7f9f87945b80 1 -- 172.21.15.37:0/1334533587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f6800fae0 con 0x55dc6782c640 2024-06-10T01:10:52.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.247+0000 7f9f5e7fc700 1 -- 172.21.15.37:0/1334533587 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9f600a4730 con 0x55dc6782c640 2024-06-10T01:10:52.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.247+0000 7f9f87945b80 10 client.17979 fetch_fsmap learned FSMap version 35 2024-06-10T01:10:52.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.247+0000 7f9f87945b80 1 -- 172.21.15.37:0/1334533587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55dc6783e280 con 0x55dc6782c640 2024-06-10T01:10:52.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.248+0000 7f9f5e7fc700 1 -- 172.21.15.37:0/1334533587 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f6800fae0 con 0x55dc6782c640 2024-06-10T01:10:52.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.248+0000 7f9f5e7fc700 1 -- 172.21.15.37:0/1334533587 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f9f600107b0 con 0x55dc6782c640 2024-06-10T01:10:52.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.248+0000 7f9f87945b80 10 client.17979 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:10:52.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.248+0000 7f9f87945b80 10 client.17979 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:10:52.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.248+0000 7f9f87945b80 20 client.17979 populate_metadata read hostname 'smithi037' 2024-06-10T01:10:52.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.248+0000 7f9f87945b80 1 --2- 172.21.15.37:0/1334533587 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55dc67840810 0x55dc67842c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:52.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.248+0000 7f9f87945b80 4 client.17979 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-10T01:10:52.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.248+0000 7f9f87945b80 1 -- 172.21.15.37:0/1334533587 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc67785d10 con 0x55dc67840810 2024-06-10T01:10:52.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.248+0000 7f9f76d9d700 1 --2- 172.21.15.37:0/1334533587 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55dc67840810 0x55dc67842c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:52.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.249+0000 7f9f76d9d700 1 --2- 172.21.15.37:0/1334533587 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55dc67840810 0x55dc67842c00 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f445740da67a4119 server_cookie=964d1eb769052d42 in_seq=0 out_seq=0 2024-06-10T01:10:52.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.249+0000 7f9f5e7fc700 10 client.17979 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:10:52.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.249+0000 7f9f5e7fc700 1 -- 172.21.15.37:0/1334533587 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55dc67785d10 con 0x55dc67840810 2024-06-10T01:10:52.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.249+0000 7f9f5e7fc700 10 client.17979 handle_command_reply: tid=0 2024-06-10T01:10:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.249+0000 7f9f87945b80 1 client.17979 shutdown 2024-06-10T01:10:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.250+0000 7f9f87945b80 1 -- 172.21.15.37:0/1334533587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6782c640 msgr2=0x55dc6782ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.250+0000 7f9f87945b80 1 --2- 172.21.15.37:0/1334533587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6782c640 0x55dc6782ffa0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f9f600ccd60 tx=0x7f9f600cf5e0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.250+0000 7f9f87945b80 1 -- 172.21.15.37:0/1334533587 shutdown_connections 2024-06-10T01:10:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.250+0000 7f9f87945b80 1 --2- 172.21.15.37:0/1334533587 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55dc67840810 0x55dc67842c00 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.250+0000 7f9f5e7fc700 0 client.17979 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:10:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.250+0000 7f9f87945b80 1 --2- 172.21.15.37:0/1334533587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6782cf60 0x55dc678304e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.250+0000 7f9f87945b80 1 --2- 172.21.15.37:0/1334533587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc67826f20 0x55dc67832c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.250+0000 7f9f87945b80 1 --2- 172.21.15.37:0/1334533587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6782c640 0x55dc6782ffa0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.250+0000 7f9f87945b80 1 -- 172.21.15.37:0/1334533587 >> 172.21.15.37:0/1334533587 conn(0x55dc67751cc0 msgr2=0x55dc678215f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:52.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.250+0000 7f9f87945b80 1 -- 172.21.15.37:0/1334533587 shutdown_connections 2024-06-10T01:10:52.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.250+0000 7f9f87945b80 1 -- 172.21.15.37:0/1334533587 wait complete. 2024-06-10T01:10:52.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.250+0000 7f9f87945b80 20 client.17979 trim_cache size 0 max 16384 2024-06-10T01:10:52.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.265+0000 7f9f87945b80 1 Processor -- start 2024-06-10T01:10:52.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.265+0000 7f9f87945b80 1 -- start start 2024-06-10T01:10:52.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.266+0000 7f9f87945b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc67826f20 0x55dc678269e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:52.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.266+0000 7f9f87945b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6782c640 0x55dc6782da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:52.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.266+0000 7f9f87945b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6782cf60 0x55dc678484b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:52.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.266+0000 7f9f87945b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9f781b0ce0 con 0x55dc6782cf60 2024-06-10T01:10:52.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.266+0000 7f9f87945b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9f781ae570 con 0x55dc67826f20 2024-06-10T01:10:52.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.266+0000 7f9f87945b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9f781ac3c0 con 0x55dc6782c640 2024-06-10T01:10:52.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.266+0000 7f9f7759e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6782cf60 0x55dc678484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:52.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.266+0000 7f9f7759e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6782cf60 0x55dc678484b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60120/0 (socket says 172.21.15.37:60120) 2024-06-10T01:10:52.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.266+0000 7f9f7759e700 1 -- 172.21.15.37:0/3165429046 learned_addr learned my addr 172.21.15.37:0/3165429046 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:52.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.266+0000 7f9f7659c700 1 --2- 172.21.15.37:0/3165429046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6782c640 0x55dc6782da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:52.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.266+0000 7f9f76d9d700 1 --2- 172.21.15.37:0/3165429046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc67826f20 0x55dc678269e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:52.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.266+0000 7f9f7759e700 1 -- 172.21.15.37:0/3165429046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6782c640 msgr2=0x55dc6782da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:52.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.266+0000 7f9f7759e700 1 --2- 172.21.15.37:0/3165429046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6782c640 0x55dc6782da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.266+0000 7f9f7759e700 1 -- 172.21.15.37:0/3165429046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc67826f20 msgr2=0x55dc678269e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:52.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.266+0000 7f9f7759e700 1 --2- 172.21.15.37:0/3165429046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc67826f20 0x55dc678269e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.266+0000 7f9f7759e700 1 -- 172.21.15.37:0/3165429046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc67822420 con 0x55dc6782cf60 2024-06-10T01:10:52.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.266+0000 7f9f76d9d700 1 --2- 172.21.15.37:0/3165429046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc67826f20 0x55dc678269e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:10:52.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.267+0000 7f9f7759e700 1 --2- 172.21.15.37:0/3165429046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6782cf60 0x55dc678484b0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f9f70009e20 tx=0x7f9f70009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649675b911bd61e6 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:52.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.267+0000 7f9f6cff9700 1 -- 172.21.15.37:0/3165429046 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f70003c00 con 0x55dc6782cf60 2024-06-10T01:10:52.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.267+0000 7f9f6cff9700 1 -- 172.21.15.37:0/3165429046 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f70003da0 con 0x55dc6782cf60 2024-06-10T01:10:52.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.267+0000 7f9f6cff9700 1 -- 172.21.15.37:0/3165429046 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f70017a70 con 0x55dc6782cf60 2024-06-10T01:10:52.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.267+0000 7f9f87945b80 1 -- 172.21.15.37:0/3165429046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6782cf60 msgr2=0x55dc678484b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:52.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.267+0000 7f9f87945b80 1 --2- 172.21.15.37:0/3165429046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6782cf60 0x55dc678484b0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f9f70009e20 tx=0x7f9f70009920 comp rx=0 tx=0).stop 2024-06-10T01:10:52.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.268+0000 7f9f87945b80 1 -- 172.21.15.37:0/3165429046 shutdown_connections 2024-06-10T01:10:52.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.268+0000 7f9f87945b80 1 --2- 172.21.15.37:0/3165429046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6782cf60 0x55dc678484b0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.268+0000 7f9f87945b80 1 --2- 172.21.15.37:0/3165429046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc67826f20 0x55dc678269e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.268+0000 7f9f87945b80 1 --2- 172.21.15.37:0/3165429046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc6782c640 0x55dc6782da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.268+0000 7f9f87945b80 1 -- 172.21.15.37:0/3165429046 >> 172.21.15.37:0/3165429046 conn(0x55dc67751cc0 msgr2=0x55dc6782d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:52.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.268+0000 7f9f87945b80 1 -- 172.21.15.37:0/3165429046 shutdown_connections 2024-06-10T01:10:52.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.268+0000 7f9f87945b80 1 -- 172.21.15.37:0/3165429046 wait complete. 2024-06-10T01:10:52.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.268+0000 7f9f87945b80 1 -- start start 2024-06-10T01:10:52.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.268+0000 7f9f87945b80 1 Processor -- start 2024-06-10T01:10:52.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.268+0000 7f9f87945b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc67826f20 0x55dc67839510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:52.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.268+0000 7f9f87945b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6782c640 0x55dc67847e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:52.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.268+0000 7f9f87945b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6782cf60 0x55dc67848380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:52.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.268+0000 7f9f87945b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9f781aca30 con 0x55dc6782cf60 2024-06-10T01:10:52.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.268+0000 7f9f87945b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9f781adea0 con 0x55dc6782c640 2024-06-10T01:10:52.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.268+0000 7f9f87945b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9f781ad100 con 0x55dc67826f20 2024-06-10T01:10:52.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.268+0000 7f9f87945b80 10 client.0 _ll_register_callbacks cb 0x55dc67712920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-10T01:10:52.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.268+0000 7f9f7759e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6782cf60 0x55dc67848380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:52.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.268+0000 7f9f7759e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6782cf60 0x55dc67848380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60130/0 (socket says 172.21.15.37:60130) 2024-06-10T01:10:52.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.268+0000 7f9f7759e700 1 -- 172.21.15.37:0/1930822741 learned_addr learned my addr 172.21.15.37:0/1930822741 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:52.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.268+0000 7f9f7659c700 1 --2- 172.21.15.37:0/1930822741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6782c640 0x55dc67847e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:52.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.269+0000 7f9f76d9d700 1 --2- 172.21.15.37:0/1930822741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc67826f20 0x55dc67839510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:52.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.269+0000 7f9f7759e700 1 -- 172.21.15.37:0/1930822741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc67826f20 msgr2=0x55dc67839510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:52.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.269+0000 7f9f7759e700 1 --2- 172.21.15.37:0/1930822741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc67826f20 0x55dc67839510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.269+0000 7f9f7759e700 1 -- 172.21.15.37:0/1930822741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6782c640 msgr2=0x55dc67847e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-10T01:10:52.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.269+0000 7f9f7759e700 1 --2- 172.21.15.37:0/1930822741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6782c640 0x55dc67847e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.269+0000 7f9f7759e700 1 -- 172.21.15.37:0/1930822741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc67848bc0 con 0x55dc6782cf60 2024-06-10T01:10:52.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.269+0000 7f9f76d9d700 1 --2- 172.21.15.37:0/1930822741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc67826f20 0x55dc67839510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:10:52.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.269+0000 7f9f7759e700 1 --2- 172.21.15.37:0/1930822741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6782cf60 0x55dc67848380 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f9f700175f0 tx=0x7f9f70009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601f9570729b2d37 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:52.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.269+0000 7f9f5e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-06-10T01:10:52.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.269+0000 7f9f5e7fc700 1 -- 172.21.15.37:0/1930822741 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7002e060 con 0x55dc6782cf60 2024-06-10T01:10:52.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.269+0000 7f9f5e7fc700 1 -- 172.21.15.37:0/1930822741 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7f9f70017b00 con 0x55dc6782cf60 2024-06-10T01:10:52.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.269+0000 7f9f87945b80 1 -- 172.21.15.37:0/1930822741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f70003da0 con 0x55dc6782cf60 2024-06-10T01:10:52.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.269+0000 7f9f5e7fc700 1 -- 172.21.15.37:0/1930822741 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9f70035050 con 0x55dc6782cf60 2024-06-10T01:10:52.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.269+0000 7f9f87945b80 10 client.17985 fetch_fsmap learned FSMap version 35 2024-06-10T01:10:52.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.269+0000 7f9f87945b80 1 -- 172.21.15.37:0/1930822741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55dc678ab460 con 0x55dc6782cf60 2024-06-10T01:10:52.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.270+0000 7f9f5e7fc700 1 -- 172.21.15.37:0/1930822741 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f70003da0 con 0x55dc6782cf60 2024-06-10T01:10:52.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.270+0000 7f9f5e7fc700 1 -- 172.21.15.37:0/1930822741 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f9f700050d0 con 0x55dc6782cf60 2024-06-10T01:10:52.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.270+0000 7f9f87945b80 10 client.17985 fetch_fsmap finished waiting for FSMap version 35 2024-06-10T01:10:52.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.270+0000 7f9f87945b80 10 client.17985 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-10T01:10:52.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.270+0000 7f9f87945b80 20 client.17985 populate_metadata read hostname 'smithi037' 2024-06-10T01:10:52.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.270+0000 7f9f87945b80 1 --2- 172.21.15.37:0/1930822741 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55dc67840810 0x55dc678ada70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:52.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.270+0000 7f9f87945b80 4 client.17985 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-10T01:10:52.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.270+0000 7f9f87945b80 1 -- 172.21.15.37:0/1930822741 --> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc67780290 con 0x55dc67840810 2024-06-10T01:10:52.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.270+0000 7f9f76d9d700 1 --2- 172.21.15.37:0/1930822741 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55dc67840810 0x55dc678ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:52.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.271+0000 7f9f76d9d700 1 --2- 172.21.15.37:0/1930822741 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55dc67840810 0x55dc678ada70 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b03c4519c8cc102 server_cookie=c14334c2416fdcce in_seq=0 out_seq=0 2024-06-10T01:10:52.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.271+0000 7f9f5e7fc700 10 client.17985 ms_handle_connect on v2:172.21.15.167:6838/3945638144 2024-06-10T01:10:52.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.271+0000 7f9f5e7fc700 1 -- 172.21.15.37:0/1930822741 <== mds.0 v2:172.21.15.167:6838/3945638144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc67780290 con 0x55dc67840810 2024-06-10T01:10:52.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.271+0000 7f9f5e7fc700 10 client.17985 handle_command_reply: tid=0 2024-06-10T01:10:52.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.271+0000 7f9f87945b80 1 client.17985 shutdown 2024-06-10T01:10:52.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.271+0000 7f9f87945b80 1 -- 172.21.15.37:0/1930822741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6782cf60 msgr2=0x55dc67848380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:52.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.271+0000 7f9f87945b80 1 --2- 172.21.15.37:0/1930822741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6782cf60 0x55dc67848380 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f9f700175f0 tx=0x7f9f70009b90 comp rx=0 tx=0).stop 2024-06-10T01:10:52.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.271+0000 7f9f87945b80 1 -- 172.21.15.37:0/1930822741 shutdown_connections 2024-06-10T01:10:52.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.271+0000 7f9f87945b80 1 --2- 172.21.15.37:0/1930822741 >> [v2:172.21.15.167:6838/3945638144,v1:172.21.15.167:6839/3945638144] conn(0x55dc67840810 0x55dc678ada70 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.271+0000 7f9f87945b80 1 --2- 172.21.15.37:0/1930822741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc6782cf60 0x55dc67848380 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.271+0000 7f9f5e7fc700 0 client.17985 ms_handle_reset on v2:172.21.15.167:6838/3945638144 2024-06-10T01:10:52.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.271+0000 7f9f87945b80 1 --2- 172.21.15.37:0/1930822741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc67826f20 0x55dc67839510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.271+0000 7f9f87945b80 1 --2- 172.21.15.37:0/1930822741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dc6782c640 0x55dc67847e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.271+0000 7f9f87945b80 1 -- 172.21.15.37:0/1930822741 >> 172.21.15.37:0/1930822741 conn(0x55dc67751cc0 msgr2=0x55dc6782d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:52.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.271+0000 7f9f87945b80 1 -- 172.21.15.37:0/1930822741 shutdown_connections 2024-06-10T01:10:52.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.271+0000 7f9f87945b80 1 -- 172.21.15.37:0/1930822741 wait complete. 2024-06-10T01:10:52.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.271+0000 7f9f87945b80 20 client.17985 trim_cache size 0 max 16384 2024-06-10T01:10:52.277 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-06-10T01:10:52.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.272+0000 7f9f7d0c2700 1 -- 172.21.15.37:0/1962755571 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9f640a2c20 msgr2=0x7f9f640a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:52.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.272+0000 7f9f7d0c2700 1 --2- 172.21.15.37:0/1962755571 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9f640a2c20 0x7f9f640a50e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9f68000c00 tx=0x7f9f68005fb0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.272+0000 7f9f7d0c2700 1 -- 172.21.15.37:0/1962755571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f78102ef0 msgr2=0x7f9f7819ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:52.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.272+0000 7f9f7d0c2700 1 --2- 172.21.15.37:0/1962755571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f78102ef0 0x7f9f7819ee00 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f9f6000e9a0 tx=0x7f9f6000ecb0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.272+0000 7f9f7d0c2700 1 -- 172.21.15.37:0/1962755571 shutdown_connections 2024-06-10T01:10:52.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.272+0000 7f9f7d0c2700 1 --2- 172.21.15.37:0/1962755571 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9f640a2c20 0x7f9f640a50e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.272+0000 7f9f7d0c2700 1 --2- 172.21.15.37:0/1962755571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f781085d0 0x7f9f78198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.272+0000 7f9f7d0c2700 1 --2- 172.21.15.37:0/1962755571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f78102530 0x7f9f7819e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.272+0000 7f9f7d0c2700 1 --2- 172.21.15.37:0/1962755571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f78102ef0 0x7f9f7819ee00 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:52.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.272+0000 7f9f7d0c2700 1 -- 172.21.15.37:0/1962755571 >> 172.21.15.37:0/1962755571 conn(0x7f9f78079640 msgr2=0x7f9f78101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:52.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.272+0000 7f9f7d0c2700 1 -- 172.21.15.37:0/1962755571 shutdown_connections 2024-06-10T01:10:52.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:52.272+0000 7f9f7d0c2700 1 -- 172.21.15.37:0/1962755571 wait complete. 2024-06-10T01:10:52.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:52 smithi167 ceph-mon[36917]: pgmap v860: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-10T01:10:52.584 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:52 smithi037 ceph-mon[32192]: pgmap v860: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-10T01:10:52.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:52 smithi094 ceph-mon[36708]: pgmap v860: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-10T01:10:53.019 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-06-10T01:10:53.024 INFO:tasks.fwd_scrub:done joining 2024-06-10T01:10:53.024 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-06-10T01:10:53.036 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-10T01:10:53.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.155+0000 7f74bdf9e640 1 -- 172.21.15.37:0/3888901651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74b81033d0 msgr2=0x7f74b8109c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:53.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.155+0000 7f74bdf9e640 1 --2- 172.21.15.37:0/3888901651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74b81033d0 0x7f74b8109c60 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f74b0009ca0 tx=0x7f74b002a8c0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.155+0000 7f74bdf9e640 1 -- 172.21.15.37:0/3888901651 shutdown_connections 2024-06-10T01:10:53.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.155+0000 7f74bdf9e640 1 --2- 172.21.15.37:0/3888901651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74b81033d0 0x7f74b8109c60 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.155+0000 7f74bdf9e640 1 --2- 172.21.15.37:0/3888901651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74b8102a10 0x7f74b8102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.155+0000 7f74bdf9e640 1 --2- 172.21.15.37:0/3888901651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74b8101820 0x7f74b8101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.155+0000 7f74bdf9e640 1 -- 172.21.15.37:0/3888901651 >> 172.21.15.37:0/3888901651 conn(0x7f74b80fcdb0 msgr2=0x7f74b80ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:53.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.155+0000 7f74bdf9e640 1 -- 172.21.15.37:0/3888901651 shutdown_connections 2024-06-10T01:10:53.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.155+0000 7f74bdf9e640 1 -- 172.21.15.37:0/3888901651 wait complete. 2024-06-10T01:10:53.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.156+0000 7f74bdf9e640 1 Processor -- start 2024-06-10T01:10:53.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.156+0000 7f74bdf9e640 1 -- start start 2024-06-10T01:10:53.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.156+0000 7f74bdf9e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74b8101820 0x7f74b8197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:53.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.156+0000 7f74bdf9e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74b8102a10 0x7f74b8197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:53.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.156+0000 7f74bdf9e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74b81033d0 0x7f74b819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:53.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.156+0000 7f74bdf9e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f74b810d480 con 0x7f74b8102a10 2024-06-10T01:10:53.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.156+0000 7f74bdf9e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f74b810d300 con 0x7f74b8101820 2024-06-10T01:10:53.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.156+0000 7f74bdf9e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f74b810d600 con 0x7f74b81033d0 2024-06-10T01:10:53.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.156+0000 7f74b7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74b8102a10 0x7f74b8197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:53.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.156+0000 7f74b7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74b8102a10 0x7f74b8197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60142/0 (socket says 172.21.15.37:60142) 2024-06-10T01:10:53.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.156+0000 7f74b7fff640 1 -- 172.21.15.37:0/2026814576 learned_addr learned my addr 172.21.15.37:0/2026814576 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:53.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.156+0000 7f74bcf9c640 1 --2- 172.21.15.37:0/2026814576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74b8101820 0x7f74b8197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:53.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.157+0000 7f74bd79d640 1 --2- 172.21.15.37:0/2026814576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74b81033d0 0x7f74b819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:53.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.157+0000 7f74b7fff640 1 -- 172.21.15.37:0/2026814576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74b81033d0 msgr2=0x7f74b819f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:53.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.157+0000 7f74b7fff640 1 --2- 172.21.15.37:0/2026814576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74b81033d0 0x7f74b819f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.157+0000 7f74b7fff640 1 -- 172.21.15.37:0/2026814576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74b8101820 msgr2=0x7f74b8197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:53.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.157+0000 7f74b7fff640 1 --2- 172.21.15.37:0/2026814576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74b8101820 0x7f74b8197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.157+0000 7f74b7fff640 1 -- 172.21.15.37:0/2026814576 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74b819f6b0 con 0x7f74b8102a10 2024-06-10T01:10:53.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.157+0000 7f74b7fff640 1 --2- 172.21.15.37:0/2026814576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74b8102a10 0x7f74b8197fd0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f74a000e9f0 tx=0x7f74a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:53.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.157+0000 7f74b5ffb640 1 -- 172.21.15.37:0/2026814576 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a000cd80 con 0x7f74b8102a10 2024-06-10T01:10:53.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.157+0000 7f74bdf9e640 1 -- 172.21.15.37:0/2026814576 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74b819f9a0 con 0x7f74b8102a10 2024-06-10T01:10:53.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.157+0000 7f74b5ffb640 1 -- 172.21.15.37:0/2026814576 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74a0010910 con 0x7f74b8102a10 2024-06-10T01:10:53.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.157+0000 7f74b5ffb640 1 -- 172.21.15.37:0/2026814576 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a0018990 con 0x7f74b8102a10 2024-06-10T01:10:53.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.157+0000 7f74bdf9e640 1 -- 172.21.15.37:0/2026814576 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74b819fef0 con 0x7f74b8102a10 2024-06-10T01:10:53.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.158+0000 7f74bdf9e640 1 -- 172.21.15.37:0/2026814576 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74b8068b30 con 0x7f74b8102a10 2024-06-10T01:10:53.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.160+0000 7f74b5ffb640 1 -- 172.21.15.37:0/2026814576 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f74a0010ac0 con 0x7f74b8102a10 2024-06-10T01:10:53.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.161+0000 7f74b5ffb640 1 --2- 172.21.15.37:0/2026814576 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f74a40a2b90 0x7f74a40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:53.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.161+0000 7f74bcf9c640 1 --2- 172.21.15.37:0/2026814576 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f74a40a2b90 0x7f74a40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:53.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.161+0000 7f74b5ffb640 1 -- 172.21.15.37:0/2026814576 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7f74a00cf7f0 con 0x7f74b8102a10 2024-06-10T01:10:53.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.161+0000 7f74bcf9c640 1 --2- 172.21.15.37:0/2026814576 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f74a40a2b90 0x7f74a40a5050 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f74b8102870 tx=0x7f74a8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:53.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.165+0000 7f74b5ffb640 1 -- 172.21.15.37:0/2026814576 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f74a009c360 con 0x7f74b8102a10 2024-06-10T01:10:53.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.353+0000 7f74bdf9e640 1 -- 172.21.15.37:0/2026814576 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f74b80625a0 con 0x7f74b8102a10 2024-06-10T01:10:53.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.355+0000 7f74b5ffb640 1 -- 172.21.15.37:0/2026814576 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+24861 (secure 0 0 0) 0x7f74a009bc60 con 0x7f74b8102a10 2024-06-10T01:10:53.350 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:10:53.350 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":105,"fsid":"68793514-26c2-11ef-bc9b-c7b262605968","created":"2024-06-10T00:44:24.106713+0000","modified":"2024-06-10T01:10:02.589262+0000","last_up_change":"2024-06-10T00:51:31.562156+0000","last_in_change":"2024-06-10T00:51:11.294301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:47:44.773911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:52:31.684780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:52:34.885503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:52:44.415437+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1df6046a-6801-4950-9193-54303fed169b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6803","nonce":3594704094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6805","nonce":3594704094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6809","nonce":3594704094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6807","nonce":3594704094}]},"public_addr":"172.21.15.37:6803/3594704094","cluster_addr":"172.21.15.37:6805/3594704094","heartbeat_back_addr":"172.21.15.37:6809/3594704094","heartbeat_front_addr":"172.21.15.37:6807/3594704094","state":["exists","up"]},{"osd":1,"uuid":"4e1c8944-94e4-4e46-9efe-56229fe6df1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6801","nonce":3427727155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6803","nonce":3427727155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6807","nonce":3427727155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6805","nonce":3427727155}]},"public_addr":"172.21.15.94:6801/3427727155","cluster_addr":"172.21.15.94:6803/3427727155","heartbeat_back_addr":"172.21.15.94:6807/3427727155","heartbeat_front_addr":"172.21.15.94:6805/3427727155","state":["exists","up"]},{"osd":2,"uuid":"fd2fcedc-4c88-478f-8025-13e5944a27ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6801","nonce":4002498292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6803","nonce":4002498292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6807","nonce":4002498292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6805","nonce":4002498292}]},"public_addr":"172.21.15.167:6801/4002498292","cluster_addr":"172.21.15.167:6803/4002498292","heartbeat_back_addr":"172.21.15.167:6807/4002498292","heartbeat_front_addr":"172.21.15.167:6805/4002498292","state":["exists","up"]},{"osd":3,"uuid":"d0153676-818c-4dcc-83b1-287435a0c8a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6811","nonce":3425469324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6813","nonce":3425469324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6817","nonce":3425469324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6815","nonce":3425469324}]},"public_addr":"172.21.15.37:6811/3425469324","cluster_addr":"172.21.15.37:6813/3425469324","heartbeat_back_addr":"172.21.15.37:6817/3425469324","heartbeat_front_addr":"172.21.15.37:6815/3425469324","state":["exists","up"]},{"osd":4,"uuid":"172c333c-00c0-4e94-8870-acb3b32b2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6809","nonce":1740627431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6811","nonce":1740627431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6815","nonce":1740627431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6813","nonce":1740627431}]},"public_addr":"172.21.15.94:6809/1740627431","cluster_addr":"172.21.15.94:6811/1740627431","heartbeat_back_addr":"172.21.15.94:6815/1740627431","heartbeat_front_addr":"172.21.15.94:6813/1740627431","state":["exists","up"]},{"osd":5,"uuid":"dbce656e-5f93-4bda-baf2-fc3cc03d8282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6809","nonce":221805236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6811","nonce":221805236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6815","nonce":221805236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6813","nonce":221805236}]},"public_addr":"172.21.15.167:6809/221805236","cluster_addr":"172.21.15.167:6811/221805236","heartbeat_back_addr":"172.21.15.167:6815/221805236","heartbeat_front_addr":"172.21.15.167:6813/221805236","state":["exists","up"]},{"osd":6,"uuid":"fcbda266-6d60-43b2-a51c-4e562476d1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6819","nonce":472670875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6821","nonce":472670875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6825","nonce":472670875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6823","nonce":472670875}]},"public_addr":"172.21.15.37:6819/472670875","cluster_addr":"172.21.15.37:6821/472670875","heartbeat_back_addr":"172.21.15.37:6825/472670875","heartbeat_front_addr":"172.21.15.37:6823/472670875","state":["exists","up"]},{"osd":7,"uuid":"ed829ed3-ba85-4e82-ac37-e44e9534d37c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6817","nonce":3932281552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6819","nonce":3932281552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6823","nonce":3932281552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6821","nonce":3932281552}]},"public_addr":"172.21.15.94:6817/3932281552","cluster_addr":"172.21.15.94:6819/3932281552","heartbeat_back_addr":"172.21.15.94:6823/3932281552","heartbeat_front_addr":"172.21.15.94:6821/3932281552","state":["exists","up"]},{"osd":8,"uuid":"7d6267d8-72f4-4704-b554-bafc52d11a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6817","nonce":2290216047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6819","nonce":2290216047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6823","nonce":2290216047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6821","nonce":2290216047}]},"public_addr":"172.21.15.167:6817/2290216047","cluster_addr":"172.21.15.167:6819/2290216047","heartbeat_back_addr":"172.21.15.167:6823/2290216047","heartbeat_front_addr":"172.21.15.167:6821/2290216047","state":["exists","up"]},{"osd":9,"uuid":"210fdc20-0a36-4eeb-bdf2-c0a7f130e19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6827","nonce":2063666428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6829","nonce":2063666428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6833","nonce":2063666428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6831","nonce":2063666428}]},"public_addr":"172.21.15.37:6827/2063666428","cluster_addr":"172.21.15.37:6829/2063666428","heartbeat_back_addr":"172.21.15.37:6833/2063666428","heartbeat_front_addr":"172.21.15.37:6831/2063666428","state":["exists","up"]},{"osd":10,"uuid":"e200b371-b3cf-4b92-a659-f155ca86f0a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6825","nonce":1706618899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6827","nonce":1706618899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6831","nonce":1706618899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6829","nonce":1706618899}]},"public_addr":"172.21.15.94:6825/1706618899","cluster_addr":"172.21.15.94:6827/1706618899","heartbeat_back_addr":"172.21.15.94:6831/1706618899","heartbeat_front_addr":"172.21.15.94:6829/1706618899","state":["exists","up"]},{"osd":11,"uuid":"cfa427df-37a5-4d28-ac2e-af777d607008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6825","nonce":1349892516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6827","nonce":1349892516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6831","nonce":1349892516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6829","nonce":1349892516}]},"public_addr":"172.21.15.167:6825/1349892516","cluster_addr":"172.21.15.167:6827/1349892516","heartbeat_back_addr":"172.21.15.167:6831/1349892516","heartbeat_front_addr":"172.21.15.167:6829/1349892516","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:46:51.846147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:15.763723+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:40.106800+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:05.923204+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:30.501883+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:55.565575+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:20.411360+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:46.071776+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:11.854599+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:37.608818+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:03.176767+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:29.768705+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/4084722397":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/3571803585":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/416587443":"2024-06-11T00:45:20.254342+0000","172.21.15.37:6801/2766692263":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/2026344134":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/2781321886":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6801/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/1110143997":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2766692263":"2024-06-11T00:45:20.254342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T01:10:53.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.357+0000 7f74bdf9e640 1 -- 172.21.15.37:0/2026814576 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f74a40a2b90 msgr2=0x7f74a40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:53.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.357+0000 7f74bdf9e640 1 --2- 172.21.15.37:0/2026814576 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f74a40a2b90 0x7f74a40a5050 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f74b8102870 tx=0x7f74a8005f90 comp rx=0 tx=0).stop 2024-06-10T01:10:53.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.357+0000 7f74bdf9e640 1 -- 172.21.15.37:0/2026814576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74b8102a10 msgr2=0x7f74b8197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:53.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.357+0000 7f74bdf9e640 1 --2- 172.21.15.37:0/2026814576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74b8102a10 0x7f74b8197fd0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f74a000e9f0 tx=0x7f74a000eeb0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.357+0000 7f74bdf9e640 1 -- 172.21.15.37:0/2026814576 shutdown_connections 2024-06-10T01:10:53.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.357+0000 7f74bdf9e640 1 --2- 172.21.15.37:0/2026814576 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f74a40a2b90 0x7f74a40a5050 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.357+0000 7f74bdf9e640 1 --2- 172.21.15.37:0/2026814576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74b81033d0 0x7f74b819f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.357+0000 7f74bdf9e640 1 --2- 172.21.15.37:0/2026814576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74b8102a10 0x7f74b8197fd0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.357+0000 7f74bdf9e640 1 --2- 172.21.15.37:0/2026814576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74b8101820 0x7f74b8197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.357+0000 7f74bdf9e640 1 -- 172.21.15.37:0/2026814576 >> 172.21.15.37:0/2026814576 conn(0x7f74b80fcdb0 msgr2=0x7f74b8105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:53.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.357+0000 7f74bdf9e640 1 -- 172.21.15.37:0/2026814576 shutdown_connections 2024-06-10T01:10:53.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.357+0000 7f74bdf9e640 1 -- 172.21.15.37:0/2026814576 wait complete. 2024-06-10T01:10:53.364 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-10T00:47:44.773911+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-10T00:52:31.684780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-10T00:52:34.885503+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-06-10T00:52:44.415437+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-06-10T01:10:53.364 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-10T01:10:53.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.477+0000 7fb251a9e640 1 -- 172.21.15.37:0/23028733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb24c102460 msgr2=0x7fb24c104850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:53.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.477+0000 7fb251a9e640 1 --2- 172.21.15.37:0/23028733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb24c102460 0x7fb24c104850 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fb23c009900 tx=0x7fb23c02a410 comp rx=0 tx=0).stop 2024-06-10T01:10:53.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.477+0000 7fb251a9e640 1 -- 172.21.15.37:0/23028733 shutdown_connections 2024-06-10T01:10:53.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.477+0000 7fb251a9e640 1 --2- 172.21.15.37:0/23028733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb24c1076c0 0x7fb24c109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.477+0000 7fb251a9e640 1 --2- 172.21.15.37:0/23028733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb24c104d90 0x7fb24c107180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.477+0000 7fb251a9e640 1 --2- 172.21.15.37:0/23028733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb24c102460 0x7fb24c104850 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.477+0000 7fb251a9e640 1 -- 172.21.15.37:0/23028733 >> 172.21.15.37:0/23028733 conn(0x7fb24c0fbe90 msgr2=0x7fb24c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:53.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.477+0000 7fb251a9e640 1 -- 172.21.15.37:0/23028733 shutdown_connections 2024-06-10T01:10:53.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.477+0000 7fb251a9e640 1 -- 172.21.15.37:0/23028733 wait complete. 2024-06-10T01:10:53.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.478+0000 7fb251a9e640 1 Processor -- start 2024-06-10T01:10:53.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.478+0000 7fb251a9e640 1 -- start start 2024-06-10T01:10:53.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.478+0000 7fb251a9e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb24c102460 0x7fb24c101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:53.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.478+0000 7fb251a9e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb24c104d90 0x7fb24c100410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:53.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.478+0000 7fb251a9e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb24c1076c0 0x7fb24c100950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:53.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.478+0000 7fb251a9e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c10d660 con 0x7fb24c104d90 2024-06-10T01:10:53.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.478+0000 7fb251a9e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c10d4e0 con 0x7fb24c102460 2024-06-10T01:10:53.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.478+0000 7fb251a9e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c10d7e0 con 0x7fb24c1076c0 2024-06-10T01:10:53.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.478+0000 7fb24affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb24c104d90 0x7fb24c100410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:53.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.478+0000 7fb24affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb24c104d90 0x7fb24c100410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60162/0 (socket says 172.21.15.37:60162) 2024-06-10T01:10:53.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.478+0000 7fb24affd640 1 -- 172.21.15.37:0/3617600409 learned_addr learned my addr 172.21.15.37:0/3617600409 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:53.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.478+0000 7fb24bfff640 1 --2- 172.21.15.37:0/3617600409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb24c1076c0 0x7fb24c100950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:53.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.478+0000 7fb24b7fe640 1 --2- 172.21.15.37:0/3617600409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb24c102460 0x7fb24c101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:53.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.479+0000 7fb24affd640 1 -- 172.21.15.37:0/3617600409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb24c1076c0 msgr2=0x7fb24c100950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:53.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.479+0000 7fb24affd640 1 --2- 172.21.15.37:0/3617600409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb24c1076c0 0x7fb24c100950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.479+0000 7fb24affd640 1 -- 172.21.15.37:0/3617600409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb24c102460 msgr2=0x7fb24c101dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:53.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.479+0000 7fb24affd640 1 --2- 172.21.15.37:0/3617600409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb24c102460 0x7fb24c101dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.479+0000 7fb24affd640 1 -- 172.21.15.37:0/3617600409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb24c1011e0 con 0x7fb24c104d90 2024-06-10T01:10:53.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.479+0000 7fb24b7fe640 1 --2- 172.21.15.37:0/3617600409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb24c102460 0x7fb24c101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:10:53.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.479+0000 7fb24affd640 1 --2- 172.21.15.37:0/3617600409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb24c104d90 0x7fb24c100410 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fb240005fd0 tx=0x7fb24000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:53.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.479+0000 7fb248ff9640 1 -- 172.21.15.37:0/3617600409 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb240017070 con 0x7fb24c104d90 2024-06-10T01:10:53.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.479+0000 7fb251a9e640 1 -- 172.21.15.37:0/3617600409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb24c1a1e50 con 0x7fb24c104d90 2024-06-10T01:10:53.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.479+0000 7fb248ff9640 1 -- 172.21.15.37:0/3617600409 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb240009c30 con 0x7fb24c104d90 2024-06-10T01:10:53.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.479+0000 7fb248ff9640 1 -- 172.21.15.37:0/3617600409 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb240010460 con 0x7fb24c104d90 2024-06-10T01:10:53.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.479+0000 7fb251a9e640 1 -- 172.21.15.37:0/3617600409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb24c1a22b0 con 0x7fb24c104d90 2024-06-10T01:10:53.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.480+0000 7fb251a9e640 1 -- 172.21.15.37:0/3617600409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb24c0fd980 con 0x7fb24c104d90 2024-06-10T01:10:53.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.481+0000 7fb248ff9640 1 -- 172.21.15.37:0/3617600409 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb240003e70 con 0x7fb24c104d90 2024-06-10T01:10:53.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.482+0000 7fb248ff9640 1 --2- 172.21.15.37:0/3617600409 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb2340a2b90 0x7fb2340a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:53.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.482+0000 7fb24b7fe640 1 --2- 172.21.15.37:0/3617600409 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb2340a2b90 0x7fb2340a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:53.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.482+0000 7fb248ff9640 1 -- 172.21.15.37:0/3617600409 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7fb2400cf920 con 0x7fb24c104d90 2024-06-10T01:10:53.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.483+0000 7fb24b7fe640 1 --2- 172.21.15.37:0/3617600409 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb2340a2b90 0x7fb2340a5050 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb23c000c00 tx=0x7fb23c0095f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:53.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.485+0000 7fb248ff9640 1 -- 172.21.15.37:0/3617600409 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb240020030 con 0x7fb24c104d90 2024-06-10T01:10:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.671+0000 7fb251a9e640 1 -- 172.21.15.37:0/3617600409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb24c02d530 con 0x7fb24c104d90 2024-06-10T01:10:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.671+0000 7fb248ff9640 1 -- 172.21.15.37:0/3617600409 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7fb24009c430 con 0x7fb24c104d90 2024-06-10T01:10:53.667 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-06-10T01:10:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.673+0000 7fb251a9e640 1 -- 172.21.15.37:0/3617600409 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb2340a2b90 msgr2=0x7fb2340a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.673+0000 7fb251a9e640 1 --2- 172.21.15.37:0/3617600409 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb2340a2b90 0x7fb2340a5050 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb23c000c00 tx=0x7fb23c0095f0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.673+0000 7fb251a9e640 1 -- 172.21.15.37:0/3617600409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb24c104d90 msgr2=0x7fb24c100410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.673+0000 7fb251a9e640 1 --2- 172.21.15.37:0/3617600409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb24c104d90 0x7fb24c100410 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fb240005fd0 tx=0x7fb24000dde0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.673+0000 7fb251a9e640 1 -- 172.21.15.37:0/3617600409 shutdown_connections 2024-06-10T01:10:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.673+0000 7fb251a9e640 1 --2- 172.21.15.37:0/3617600409 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fb2340a2b90 0x7fb2340a5050 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.673+0000 7fb251a9e640 1 --2- 172.21.15.37:0/3617600409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb24c1076c0 0x7fb24c100950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.673+0000 7fb251a9e640 1 --2- 172.21.15.37:0/3617600409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb24c104d90 0x7fb24c100410 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.673+0000 7fb251a9e640 1 --2- 172.21.15.37:0/3617600409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb24c102460 0x7fb24c101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.673+0000 7fb251a9e640 1 -- 172.21.15.37:0/3617600409 >> 172.21.15.37:0/3617600409 conn(0x7fb24c0fbe90 msgr2=0x7fb24c102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.674+0000 7fb251a9e640 1 -- 172.21.15.37:0/3617600409 shutdown_connections 2024-06-10T01:10:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.674+0000 7fb251a9e640 1 -- 172.21.15.37:0/3617600409 wait complete. 2024-06-10T01:10:53.680 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-10T01:10:53.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.793+0000 7fce2c2d4640 1 -- 172.21.15.37:0/686907389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce241035b0 msgr2=0x7fce24109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:53.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.793+0000 7fce2c2d4640 1 --2- 172.21.15.37:0/686907389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce241035b0 0x7fce24109e40 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fce1c009ca0 tx=0x7fce1c02a8c0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.793+0000 7fce2c2d4640 1 -- 172.21.15.37:0/686907389 shutdown_connections 2024-06-10T01:10:53.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.793+0000 7fce2c2d4640 1 --2- 172.21.15.37:0/686907389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce241035b0 0x7fce24109e40 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.793+0000 7fce2c2d4640 1 --2- 172.21.15.37:0/686907389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce24102bf0 0x7fce24103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.793+0000 7fce2c2d4640 1 --2- 172.21.15.37:0/686907389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce24101a00 0x7fce24101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.793+0000 7fce2c2d4640 1 -- 172.21.15.37:0/686907389 >> 172.21.15.37:0/686907389 conn(0x7fce240fcf70 msgr2=0x7fce240ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:53.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.793+0000 7fce2c2d4640 1 -- 172.21.15.37:0/686907389 shutdown_connections 2024-06-10T01:10:53.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.793+0000 7fce2c2d4640 1 -- 172.21.15.37:0/686907389 wait complete. 2024-06-10T01:10:53.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.794+0000 7fce2c2d4640 1 Processor -- start 2024-06-10T01:10:53.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.794+0000 7fce2c2d4640 1 -- start start 2024-06-10T01:10:53.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.794+0000 7fce2c2d4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce24101a00 0x7fce24075750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:53.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.794+0000 7fce2c2d4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce24102bf0 0x7fce24073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:53.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.794+0000 7fce2c2d4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce241035b0 0x7fce240742d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:53.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.794+0000 7fce2c2d4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fce2410d680 con 0x7fce24101a00 2024-06-10T01:10:53.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.794+0000 7fce2c2d4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fce2410d500 con 0x7fce24102bf0 2024-06-10T01:10:53.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.794+0000 7fce2c2d4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fce2410d800 con 0x7fce241035b0 2024-06-10T01:10:53.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.794+0000 7fce2a070640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce24101a00 0x7fce24075750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:53.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.794+0000 7fce2a070640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce24101a00 0x7fce24075750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60176/0 (socket says 172.21.15.37:60176) 2024-06-10T01:10:53.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.794+0000 7fce2a070640 1 -- 172.21.15.37:0/489387263 learned_addr learned my addr 172.21.15.37:0/489387263 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:53.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.794+0000 7fce2986f640 1 --2- 172.21.15.37:0/489387263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce24102bf0 0x7fce24073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:53.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.794+0000 7fce2a871640 1 --2- 172.21.15.37:0/489387263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce241035b0 0x7fce240742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:53.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.794+0000 7fce2a070640 1 -- 172.21.15.37:0/489387263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce241035b0 msgr2=0x7fce240742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:53.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.794+0000 7fce2a070640 1 --2- 172.21.15.37:0/489387263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce241035b0 0x7fce240742d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.794+0000 7fce2a070640 1 -- 172.21.15.37:0/489387263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce24102bf0 msgr2=0x7fce24073d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:53.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.795+0000 7fce2a070640 1 --2- 172.21.15.37:0/489387263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce24102bf0 0x7fce24073d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.795+0000 7fce2a070640 1 -- 172.21.15.37:0/489387263 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce24074b60 con 0x7fce24101a00 2024-06-10T01:10:53.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.795+0000 7fce2a871640 1 --2- 172.21.15.37:0/489387263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce241035b0 0x7fce240742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:10:53.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.795+0000 7fce2986f640 1 --2- 172.21.15.37:0/489387263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce24102bf0 0x7fce24073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-10T01:10:53.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.795+0000 7fce2a070640 1 --2- 172.21.15.37:0/489387263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce24101a00 0x7fce24075750 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fce2000c970 tx=0x7fce2000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:53.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.795+0000 7fce1b7fe640 1 -- 172.21.15.37:0/489387263 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce20004220 con 0x7fce24101a00 2024-06-10T01:10:53.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.795+0000 7fce1b7fe640 1 -- 172.21.15.37:0/489387263 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce200043c0 con 0x7fce24101a00 2024-06-10T01:10:53.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.795+0000 7fce2c2d4640 1 -- 172.21.15.37:0/489387263 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce241a1dc0 con 0x7fce24101a00 2024-06-10T01:10:53.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.795+0000 7fce1b7fe640 1 -- 172.21.15.37:0/489387263 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2000f450 con 0x7fce24101a00 2024-06-10T01:10:53.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.795+0000 7fce2c2d4640 1 -- 172.21.15.37:0/489387263 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce241a2220 con 0x7fce24101a00 2024-06-10T01:10:53.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.796+0000 7fce2c2d4640 1 -- 172.21.15.37:0/489387263 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce24108d20 con 0x7fce24101a00 2024-06-10T01:10:53.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.797+0000 7fce1b7fe640 1 -- 172.21.15.37:0/489387263 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fce200079b0 con 0x7fce24101a00 2024-06-10T01:10:53.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.798+0000 7fce1b7fe640 1 --2- 172.21.15.37:0/489387263 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fce100a2b90 0x7fce100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:53.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.798+0000 7fce1b7fe640 1 -- 172.21.15.37:0/489387263 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7fce200cf6c0 con 0x7fce24101a00 2024-06-10T01:10:53.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.798+0000 7fce2986f640 1 --2- 172.21.15.37:0/489387263 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fce100a2b90 0x7fce100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:53.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.798+0000 7fce2986f640 1 --2- 172.21.15.37:0/489387263 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fce100a2b90 0x7fce100a5050 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fce240754c0 tx=0x7fce14005e50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:53.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.801+0000 7fce1b7fe640 1 -- 172.21.15.37:0/489387263 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fce20014030 con 0x7fce24101a00 2024-06-10T01:10:53.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.987+0000 7fce2c2d4640 1 -- 172.21.15.37:0/489387263 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fce2402d590 con 0x7fce24101a00 2024-06-10T01:10:53.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.987+0000 7fce1b7fe640 1 -- 172.21.15.37:0/489387263 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) ==== 104+0+11 (secure 0 0 0) 0x7fce2009c1d0 con 0x7fce24101a00 2024-06-10T01:10:53.982 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-06-10T01:10:53.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.989+0000 7fce2c2d4640 1 -- 172.21.15.37:0/489387263 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fce100a2b90 msgr2=0x7fce100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:53.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.989+0000 7fce2c2d4640 1 --2- 172.21.15.37:0/489387263 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fce100a2b90 0x7fce100a5050 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fce240754c0 tx=0x7fce14005e50 comp rx=0 tx=0).stop 2024-06-10T01:10:53.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.989+0000 7fce2c2d4640 1 -- 172.21.15.37:0/489387263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce24101a00 msgr2=0x7fce24075750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:53.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.989+0000 7fce2c2d4640 1 --2- 172.21.15.37:0/489387263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce24101a00 0x7fce24075750 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fce2000c970 tx=0x7fce2000ce30 comp rx=0 tx=0).stop 2024-06-10T01:10:53.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.989+0000 7fce2c2d4640 1 -- 172.21.15.37:0/489387263 shutdown_connections 2024-06-10T01:10:53.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.989+0000 7fce2c2d4640 1 --2- 172.21.15.37:0/489387263 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fce100a2b90 0x7fce100a5050 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.989+0000 7fce2c2d4640 1 --2- 172.21.15.37:0/489387263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce241035b0 0x7fce240742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.989+0000 7fce2c2d4640 1 --2- 172.21.15.37:0/489387263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce24102bf0 0x7fce24073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.989+0000 7fce2c2d4640 1 --2- 172.21.15.37:0/489387263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce24101a00 0x7fce24075750 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:53.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.989+0000 7fce2c2d4640 1 -- 172.21.15.37:0/489387263 >> 172.21.15.37:0/489387263 conn(0x7fce240fcf70 msgr2=0x7fce24105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:53.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.990+0000 7fce2c2d4640 1 -- 172.21.15.37:0/489387263 shutdown_connections 2024-06-10T01:10:53.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:53.990+0000 7fce2c2d4640 1 -- 172.21.15.37:0/489387263 wait complete. 2024-06-10T01:10:53.996 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-10T01:10:54.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.110+0000 7f04cd246640 1 -- 172.21.15.37:0/3244444572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04c8107720 msgr2=0x7f04c8109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:54.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.110+0000 7f04cd246640 1 --2- 172.21.15.37:0/3244444572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04c8107720 0x7f04c8109b10 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f04c0009900 tx=0x7f04c002a410 comp rx=0 tx=0).stop 2024-06-10T01:10:54.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.110+0000 7f04cd246640 1 -- 172.21.15.37:0/3244444572 shutdown_connections 2024-06-10T01:10:54.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.110+0000 7f04cd246640 1 --2- 172.21.15.37:0/3244444572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04c8107720 0x7f04c8109b10 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.110+0000 7f04cd246640 1 --2- 172.21.15.37:0/3244444572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04c8104df0 0x7f04c81071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.110+0000 7f04cd246640 1 --2- 172.21.15.37:0/3244444572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04c81024c0 0x7f04c81048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.110+0000 7f04cd246640 1 -- 172.21.15.37:0/3244444572 >> 172.21.15.37:0/3244444572 conn(0x7f04c80fbe50 msgr2=0x7f04c80fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:54.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.110+0000 7f04cd246640 1 -- 172.21.15.37:0/3244444572 shutdown_connections 2024-06-10T01:10:54.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.110+0000 7f04cd246640 1 -- 172.21.15.37:0/3244444572 wait complete. 2024-06-10T01:10:54.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.111+0000 7f04cd246640 1 Processor -- start 2024-06-10T01:10:54.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.111+0000 7f04cd246640 1 -- start start 2024-06-10T01:10:54.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.111+0000 7f04cd246640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04c81024c0 0x7f04c8197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.111+0000 7f04cd246640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04c8104df0 0x7f04c8198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.111+0000 7f04cd246640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04c8107720 0x7f04c819f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.111+0000 7f04cd246640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f04c810d660 con 0x7f04c81024c0 2024-06-10T01:10:54.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.111+0000 7f04cd246640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f04c810d4e0 con 0x7f04c8107720 2024-06-10T01:10:54.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.111+0000 7f04cd246640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f04c810d7e0 con 0x7f04c8104df0 2024-06-10T01:10:54.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.112+0000 7f04c6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04c81024c0 0x7f04c8197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.112+0000 7f04c6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04c81024c0 0x7f04c8197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60192/0 (socket says 172.21.15.37:60192) 2024-06-10T01:10:54.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.112+0000 7f04c6d9d640 1 -- 172.21.15.37:0/4087027999 learned_addr learned my addr 172.21.15.37:0/4087027999 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:54.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.112+0000 7f04c659c640 1 --2- 172.21.15.37:0/4087027999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04c8104df0 0x7f04c8198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.112+0000 7f04c759e640 1 --2- 172.21.15.37:0/4087027999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04c8107720 0x7f04c819f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.112+0000 7f04c6d9d640 1 -- 172.21.15.37:0/4087027999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04c8104df0 msgr2=0x7f04c8198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:54.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.112+0000 7f04c6d9d640 1 --2- 172.21.15.37:0/4087027999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04c8104df0 0x7f04c8198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.112+0000 7f04c6d9d640 1 -- 172.21.15.37:0/4087027999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04c8107720 msgr2=0x7f04c819f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:54.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.112+0000 7f04c6d9d640 1 --2- 172.21.15.37:0/4087027999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04c8107720 0x7f04c819f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.112+0000 7f04c6d9d640 1 -- 172.21.15.37:0/4087027999 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c819f940 con 0x7f04c81024c0 2024-06-10T01:10:54.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.112+0000 7f04c659c640 1 --2- 172.21.15.37:0/4087027999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04c8104df0 0x7f04c8198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:10:54.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.112+0000 7f04c6d9d640 1 --2- 172.21.15.37:0/4087027999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04c81024c0 0x7f04c8197d30 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f04b800db80 tx=0x7f04b800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:54.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.112+0000 7f04affff640 1 -- 172.21.15.37:0/4087027999 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b8017070 con 0x7f04c81024c0 2024-06-10T01:10:54.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.112+0000 7f04cd246640 1 -- 172.21.15.37:0/4087027999 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c819fc30 con 0x7f04c81024c0 2024-06-10T01:10:54.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.112+0000 7f04affff640 1 -- 172.21.15.37:0/4087027999 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04b8003b70 con 0x7f04c81024c0 2024-06-10T01:10:54.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.112+0000 7f04cd246640 1 -- 172.21.15.37:0/4087027999 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c81a01d0 con 0x7f04c81024c0 2024-06-10T01:10:54.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.112+0000 7f04affff640 1 -- 172.21.15.37:0/4087027999 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b8004d20 con 0x7f04c81024c0 2024-06-10T01:10:54.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.113+0000 7f04cd246640 1 -- 172.21.15.37:0/4087027999 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04c80fd940 con 0x7f04c81024c0 2024-06-10T01:10:54.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.115+0000 7f04affff640 1 -- 172.21.15.37:0/4087027999 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f04b8004020 con 0x7f04c81024c0 2024-06-10T01:10:54.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.115+0000 7f04affff640 1 --2- 172.21.15.37:0/4087027999 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f04b00a2b90 0x7f04b00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.116+0000 7f04affff640 1 -- 172.21.15.37:0/4087027999 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7f04b80cfca0 con 0x7f04c81024c0 2024-06-10T01:10:54.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.116+0000 7f04c659c640 1 --2- 172.21.15.37:0/4087027999 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f04b00a2b90 0x7f04b00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.116+0000 7f04c659c640 1 --2- 172.21.15.37:0/4087027999 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f04b00a2b90 0x7f04b00a5050 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f04bc005d90 tx=0x7f04bc005d20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:54.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.119+0000 7f04affff640 1 -- 172.21.15.37:0/4087027999 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f04b809c810 con 0x7f04c81024c0 2024-06-10T01:10:54.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.306+0000 7f04cd246640 1 -- 172.21.15.37:0/4087027999 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f04c80625a0 con 0x7f04c81024c0 2024-06-10T01:10:54.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.306+0000 7f04affff640 1 -- 172.21.15.37:0/4087027999 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) ==== 100+0+11 (secure 0 0 0) 0x7f04b800fe80 con 0x7f04c81024c0 2024-06-10T01:10:54.302 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-06-10T01:10:54.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.308+0000 7f04cd246640 1 -- 172.21.15.37:0/4087027999 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f04b00a2b90 msgr2=0x7f04b00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:54.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.308+0000 7f04cd246640 1 --2- 172.21.15.37:0/4087027999 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f04b00a2b90 0x7f04b00a5050 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f04bc005d90 tx=0x7f04bc005d20 comp rx=0 tx=0).stop 2024-06-10T01:10:54.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.308+0000 7f04cd246640 1 -- 172.21.15.37:0/4087027999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04c81024c0 msgr2=0x7f04c8197d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:54.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.308+0000 7f04cd246640 1 --2- 172.21.15.37:0/4087027999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04c81024c0 0x7f04c8197d30 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f04b800db80 tx=0x7f04b800b630 comp rx=0 tx=0).stop 2024-06-10T01:10:54.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.308+0000 7f04cd246640 1 -- 172.21.15.37:0/4087027999 shutdown_connections 2024-06-10T01:10:54.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.308+0000 7f04cd246640 1 --2- 172.21.15.37:0/4087027999 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f04b00a2b90 0x7f04b00a5050 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.308+0000 7f04cd246640 1 --2- 172.21.15.37:0/4087027999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04c8107720 0x7f04c819f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.308+0000 7f04cd246640 1 --2- 172.21.15.37:0/4087027999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04c8104df0 0x7f04c8198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.308+0000 7f04cd246640 1 --2- 172.21.15.37:0/4087027999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04c81024c0 0x7f04c8197d30 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.308+0000 7f04cd246640 1 -- 172.21.15.37:0/4087027999 >> 172.21.15.37:0/4087027999 conn(0x7f04c80fbe50 msgr2=0x7f04c8102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:54.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.309+0000 7f04cd246640 1 -- 172.21.15.37:0/4087027999 shutdown_connections 2024-06-10T01:10:54.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.309+0000 7f04cd246640 1 -- 172.21.15.37:0/4087027999 wait complete. 2024-06-10T01:10:54.315 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-06-10T01:10:54.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.428+0000 7fd8394a5640 1 -- 172.21.15.37:0/3270509650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd834107780 msgr2=0x7fd834109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:54.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.428+0000 7fd8394a5640 1 --2- 172.21.15.37:0/3270509650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd834107780 0x7fd834109b70 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009900 tx=0x7fd82c02a410 comp rx=0 tx=0).stop 2024-06-10T01:10:54.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.428+0000 7fd8394a5640 1 -- 172.21.15.37:0/3270509650 shutdown_connections 2024-06-10T01:10:54.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.428+0000 7fd8394a5640 1 --2- 172.21.15.37:0/3270509650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd834107780 0x7fd834109b70 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.428+0000 7fd8394a5640 1 --2- 172.21.15.37:0/3270509650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd834104e50 0x7fd834107240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.428+0000 7fd8394a5640 1 --2- 172.21.15.37:0/3270509650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd834102520 0x7fd834104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.428+0000 7fd8394a5640 1 -- 172.21.15.37:0/3270509650 >> 172.21.15.37:0/3270509650 conn(0x7fd8340fbe50 msgr2=0x7fd8340fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:54.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.429+0000 7fd8394a5640 1 -- 172.21.15.37:0/3270509650 shutdown_connections 2024-06-10T01:10:54.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.429+0000 7fd8394a5640 1 -- 172.21.15.37:0/3270509650 wait complete. 2024-06-10T01:10:54.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.429+0000 7fd8394a5640 1 Processor -- start 2024-06-10T01:10:54.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.429+0000 7fd8394a5640 1 -- start start 2024-06-10T01:10:54.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.429+0000 7fd8394a5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd834102520 0x7fd834197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.429+0000 7fd8394a5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd834104e50 0x7fd834198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.430+0000 7fd8394a5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd834107780 0x7fd83419f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.430+0000 7fd8394a5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd83410d690 con 0x7fd834104e50 2024-06-10T01:10:54.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.430+0000 7fd8394a5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd83410d510 con 0x7fd834102520 2024-06-10T01:10:54.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.430+0000 7fd8394a5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd83410d810 con 0x7fd834107780 2024-06-10T01:10:54.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.430+0000 7fd8327fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd834104e50 0x7fd834198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.430+0000 7fd8327fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd834104e50 0x7fd834198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60204/0 (socket says 172.21.15.37:60204) 2024-06-10T01:10:54.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.430+0000 7fd8327fc640 1 -- 172.21.15.37:0/1161741618 learned_addr learned my addr 172.21.15.37:0/1161741618 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:54.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.430+0000 7fd832ffd640 1 --2- 172.21.15.37:0/1161741618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd834102520 0x7fd834197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.430+0000 7fd8327fc640 1 -- 172.21.15.37:0/1161741618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd834107780 msgr2=0x7fd83419f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:54.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.430+0000 7fd8327fc640 1 --2- 172.21.15.37:0/1161741618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd834107780 0x7fd83419f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.430+0000 7fd8327fc640 1 -- 172.21.15.37:0/1161741618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd834102520 msgr2=0x7fd834197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:54.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.430+0000 7fd8327fc640 1 --2- 172.21.15.37:0/1161741618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd834102520 0x7fd834197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.430+0000 7fd8327fc640 1 -- 172.21.15.37:0/1161741618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd83419f940 con 0x7fd834104e50 2024-06-10T01:10:54.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.430+0000 7fd8327fc640 1 --2- 172.21.15.37:0/1161741618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd834104e50 0x7fd834198270 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fd82800d770 tx=0x7fd82800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:54.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.430+0000 7fd81bfff640 1 -- 172.21.15.37:0/1161741618 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd828017070 con 0x7fd834104e50 2024-06-10T01:10:54.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.430+0000 7fd81bfff640 1 -- 172.21.15.37:0/1161741618 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd828009c30 con 0x7fd834104e50 2024-06-10T01:10:54.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.430+0000 7fd8394a5640 1 -- 172.21.15.37:0/1161741618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd83419fc30 con 0x7fd834104e50 2024-06-10T01:10:54.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.430+0000 7fd81bfff640 1 -- 172.21.15.37:0/1161741618 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82800f620 con 0x7fd834104e50 2024-06-10T01:10:54.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.430+0000 7fd8394a5640 1 -- 172.21.15.37:0/1161741618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8341a0180 con 0x7fd834104e50 2024-06-10T01:10:54.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.431+0000 7fd8394a5640 1 -- 172.21.15.37:0/1161741618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8340fd940 con 0x7fd834104e50 2024-06-10T01:10:54.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.432+0000 7fd81bfff640 1 -- 172.21.15.37:0/1161741618 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd82801a070 con 0x7fd834104e50 2024-06-10T01:10:54.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.433+0000 7fd81bfff640 1 --2- 172.21.15.37:0/1161741618 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd81c0a2b90 0x7fd81c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.433+0000 7fd81bfff640 1 -- 172.21.15.37:0/1161741618 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7fd8280cf080 con 0x7fd834104e50 2024-06-10T01:10:54.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.433+0000 7fd832ffd640 1 --2- 172.21.15.37:0/1161741618 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd81c0a2b90 0x7fd81c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.433+0000 7fd832ffd640 1 --2- 172.21.15.37:0/1161741618 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd81c0a2b90 0x7fd81c0a5050 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd834199200 tx=0x7fd8240094a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:54.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.437+0000 7fd81bfff640 1 -- 172.21.15.37:0/1161741618 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd828009de0 con 0x7fd834104e50 2024-06-10T01:10:54.562 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:54 smithi037 ceph-mon[32192]: pgmap v861: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-10T01:10:54.563 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:54 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2026814576' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:10:54.563 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:54 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3617600409' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T01:10:54.563 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:54 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/489387263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-10T01:10:54.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:54 smithi167 ceph-mon[36917]: pgmap v861: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-10T01:10:54.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:54 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2026814576' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:10:54.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:54 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3617600409' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T01:10:54.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:54 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/489387263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-10T01:10:54.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:54 smithi094 ceph-mon[36708]: pgmap v861: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-06-10T01:10:54.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:54 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2026814576' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:10:54.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:54 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3617600409' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-10T01:10:54.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:54 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/489387263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-10T01:10:54.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.624+0000 7fd8394a5640 1 -- 172.21.15.37:0/1161741618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd834062640 con 0x7fd834104e50 2024-06-10T01:10:54.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.624+0000 7fd81bfff640 1 -- 172.21.15.37:0/1161741618 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v105) ==== 103+0+11 (secure 0 0 0) 0x7fd82809bbf0 con 0x7fd834104e50 2024-06-10T01:10:54.620 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-06-10T01:10:54.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.627+0000 7fd8394a5640 1 -- 172.21.15.37:0/1161741618 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd81c0a2b90 msgr2=0x7fd81c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:54.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.627+0000 7fd8394a5640 1 --2- 172.21.15.37:0/1161741618 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd81c0a2b90 0x7fd81c0a5050 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd834199200 tx=0x7fd8240094a0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.627+0000 7fd8394a5640 1 -- 172.21.15.37:0/1161741618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd834104e50 msgr2=0x7fd834198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:54.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.627+0000 7fd8394a5640 1 --2- 172.21.15.37:0/1161741618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd834104e50 0x7fd834198270 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fd82800d770 tx=0x7fd82800dc30 comp rx=0 tx=0).stop 2024-06-10T01:10:54.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.627+0000 7fd8394a5640 1 -- 172.21.15.37:0/1161741618 shutdown_connections 2024-06-10T01:10:54.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.627+0000 7fd8394a5640 1 --2- 172.21.15.37:0/1161741618 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd81c0a2b90 0x7fd81c0a5050 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.627+0000 7fd8394a5640 1 --2- 172.21.15.37:0/1161741618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd834107780 0x7fd83419f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.627+0000 7fd8394a5640 1 --2- 172.21.15.37:0/1161741618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd834104e50 0x7fd834198270 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.627+0000 7fd8394a5640 1 --2- 172.21.15.37:0/1161741618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd834102520 0x7fd834197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.627+0000 7fd8394a5640 1 -- 172.21.15.37:0/1161741618 >> 172.21.15.37:0/1161741618 conn(0x7fd8340fbe50 msgr2=0x7fd834102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:54.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.627+0000 7fd8394a5640 1 -- 172.21.15.37:0/1161741618 shutdown_connections 2024-06-10T01:10:54.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.627+0000 7fd8394a5640 1 -- 172.21.15.37:0/1161741618 wait complete. 2024-06-10T01:10:54.633 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-06-10T01:10:54.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.743+0000 7fa8913f9640 1 -- 172.21.15.37:0/3556581794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa88c1024b0 msgr2=0x7fa88c108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:54.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.743+0000 7fa8913f9640 1 --2- 172.21.15.37:0/3556581794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa88c1024b0 0x7fa88c108d40 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fa884009ca0 tx=0x7fa88402a8c0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.743+0000 7fa8913f9640 1 -- 172.21.15.37:0/3556581794 shutdown_connections 2024-06-10T01:10:54.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.743+0000 7fa8913f9640 1 --2- 172.21.15.37:0/3556581794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa88c1024b0 0x7fa88c108d40 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.743+0000 7fa8913f9640 1 --2- 172.21.15.37:0/3556581794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa88c101af0 0x7fa88c101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.743+0000 7fa8913f9640 1 --2- 172.21.15.37:0/3556581794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa88c100900 0x7fa88c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.743+0000 7fa8913f9640 1 -- 172.21.15.37:0/3556581794 >> 172.21.15.37:0/3556581794 conn(0x7fa88c0fbe90 msgr2=0x7fa88c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:54.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.743+0000 7fa8913f9640 1 -- 172.21.15.37:0/3556581794 shutdown_connections 2024-06-10T01:10:54.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.743+0000 7fa8913f9640 1 -- 172.21.15.37:0/3556581794 wait complete. 2024-06-10T01:10:54.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.744+0000 7fa8913f9640 1 Processor -- start 2024-06-10T01:10:54.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.744+0000 7fa8913f9640 1 -- start start 2024-06-10T01:10:54.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.744+0000 7fa8913f9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa88c100900 0x7fa88c197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.744+0000 7fa8913f9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa88c101af0 0x7fa88c198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.744+0000 7fa8913f9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa88c1024b0 0x7fa88c19f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.744+0000 7fa8913f9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c10c670 con 0x7fa88c1024b0 2024-06-10T01:10:54.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.744+0000 7fa8913f9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c10c4f0 con 0x7fa88c100900 2024-06-10T01:10:54.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.744+0000 7fa8913f9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c10c7f0 con 0x7fa88c101af0 2024-06-10T01:10:54.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.744+0000 7fa88b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa88c1024b0 0x7fa88c19f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.744+0000 7fa88b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa88c1024b0 0x7fa88c19f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60222/0 (socket says 172.21.15.37:60222) 2024-06-10T01:10:54.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.744+0000 7fa88b7fe640 1 -- 172.21.15.37:0/218062647 learned_addr learned my addr 172.21.15.37:0/218062647 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:10:54.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.744+0000 7fa88affd640 1 --2- 172.21.15.37:0/218062647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa88c100900 0x7fa88c197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.745+0000 7fa88a7fc640 1 --2- 172.21.15.37:0/218062647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa88c101af0 0x7fa88c198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.745+0000 7fa88b7fe640 1 -- 172.21.15.37:0/218062647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa88c101af0 msgr2=0x7fa88c198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:54.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.745+0000 7fa88b7fe640 1 --2- 172.21.15.37:0/218062647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa88c101af0 0x7fa88c198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.745+0000 7fa88b7fe640 1 -- 172.21.15.37:0/218062647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa88c100900 msgr2=0x7fa88c197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:54.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.745+0000 7fa88b7fe640 1 --2- 172.21.15.37:0/218062647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa88c100900 0x7fa88c197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:54.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.745+0000 7fa88b7fe640 1 -- 172.21.15.37:0/218062647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88c19f970 con 0x7fa88c1024b0 2024-06-10T01:10:54.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.745+0000 7fa88b7fe640 1 --2- 172.21.15.37:0/218062647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa88c1024b0 0x7fa88c19f300 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fa8840319f0 tx=0x7fa88402f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:54.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.745+0000 7fa873fff640 1 -- 172.21.15.37:0/218062647 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa884003c70 con 0x7fa88c1024b0 2024-06-10T01:10:54.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.745+0000 7fa873fff640 1 -- 172.21.15.37:0/218062647 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa884003e10 con 0x7fa88c1024b0 2024-06-10T01:10:54.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.745+0000 7fa8913f9640 1 -- 172.21.15.37:0/218062647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa88c19fc00 con 0x7fa88c1024b0 2024-06-10T01:10:54.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.745+0000 7fa873fff640 1 -- 172.21.15.37:0/218062647 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88402fcc0 con 0x7fa88c1024b0 2024-06-10T01:10:54.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.745+0000 7fa8913f9640 1 -- 172.21.15.37:0/218062647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa88c1a0150 con 0x7fa88c1024b0 2024-06-10T01:10:54.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.746+0000 7fa8913f9640 1 -- 172.21.15.37:0/218062647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa88c1a0380 con 0x7fa88c1024b0 2024-06-10T01:10:54.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.747+0000 7fa873fff640 1 -- 172.21.15.37:0/218062647 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa884004040 con 0x7fa88c1024b0 2024-06-10T01:10:54.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.748+0000 7fa873fff640 1 --2- 172.21.15.37:0/218062647 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa8740a2b90 0x7fa8740a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:10:54.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.748+0000 7fa88affd640 1 --2- 172.21.15.37:0/218062647 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa8740a2b90 0x7fa8740a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:10:54.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.748+0000 7fa873fff640 1 -- 172.21.15.37:0/218062647 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7fa8840f0990 con 0x7fa88c1024b0 2024-06-10T01:10:54.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.749+0000 7fa88affd640 1 --2- 172.21.15.37:0/218062647 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa8740a2b90 0x7fa8740a5050 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa88c101950 tx=0x7fa87c009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:10:54.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.753+0000 7fa873fff640 1 -- 172.21.15.37:0/218062647 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa8840bd4a0 con 0x7fa88c1024b0 2024-06-10T01:10:54.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:54.991+0000 7fa8913f9640 1 -- 172.21.15.37:0/218062647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fa88c1a0710 con 0x7fa88c1024b0 2024-06-10T01:10:54.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:55.000+0000 7fa873fff640 1 -- 172.21.15.37:0/218062647 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+740370 (secure 0 0 0) 0x7fa8840bcda0 con 0x7fa88c1024b0 2024-06-10T01:10:55.005 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:10:55.005 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:10:55.005 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 20, 2024-06-10T01:10:55.005 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.005 INFO:teuthology.orchestra.run.smithi037.stdout: "active_gid": 14148, 2024-06-10T01:10:55.006 INFO:teuthology.orchestra.run.smithi037.stdout: "active_name": "x", 2024-06-10T01:10:55.006 INFO:teuthology.orchestra.run.smithi037.stdout: "active_addrs": { 2024-06-10T01:10:55.006 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-06-10T01:10:55.006 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.006 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-06-10T01:10:55.006 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:6800", 2024-06-10T01:10:55.006 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 2387001309 2024-06-10T01:10:55.006 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.007 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.007 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-06-10T01:10:55.007 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:6801", 2024-06-10T01:10:55.007 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 2387001309 2024-06-10T01:10:55.007 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.007 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-06-10T01:10:55.007 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.007 INFO:teuthology.orchestra.run.smithi037.stdout: "active_addr": "172.21.15.37:6801/2387001309", 2024-06-10T01:10:55.008 INFO:teuthology.orchestra.run.smithi037.stdout: "active_change": "2024-06-10T00:45:20.254495+0000", 2024-06-10T01:10:55.008 INFO:teuthology.orchestra.run.smithi037.stdout: "active_mgr_features": 4540701547738038271, 2024-06-10T01:10:55.008 INFO:teuthology.orchestra.run.smithi037.stdout: "available": true, 2024-06-10T01:10:55.008 INFO:teuthology.orchestra.run.smithi037.stdout: "standbys": [ 2024-06-10T01:10:55.008 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.008 INFO:teuthology.orchestra.run.smithi037.stdout: "gid": 24107, 2024-06-10T01:10:55.008 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "y", 2024-06-10T01:10:55.008 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr_features": 4540701547738038271, 2024-06-10T01:10:55.009 INFO:teuthology.orchestra.run.smithi037.stdout: "available_modules": [ 2024-06-10T01:10:55.009 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.009 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "alerts", 2024-06-10T01:10:55.009 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.009 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.009 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.009 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-06-10T01:10:55.009 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-06-10T01:10:55.010 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.010 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.010 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.010 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-06-10T01:10:55.010 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.010 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.010 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.010 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How frequently to reexamine health status", 2024-06-10T01:10:55.011 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.011 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.011 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.011 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.011 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.011 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.011 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.011 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.012 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.012 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.012 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.012 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.012 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.012 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.012 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.012 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.013 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.013 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.013 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.013 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.013 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.013 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.013 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.013 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.014 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.014 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.014 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.014 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.014 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.014 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.014 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.014 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.015 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.015 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.015 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.015 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.015 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.015 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.016 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.016 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.016 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.016 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.016 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.016 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.016 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.016 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.016 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.017 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.017 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.017 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.017 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.017 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.017 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.017 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.017 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.018 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.018 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.018 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.018 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.018 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.018 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.019 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.019 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.019 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.019 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.019 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.019 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.019 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.019 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.019 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.020 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.020 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.020 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_destination": { 2024-06-10T01:10:55.020 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_destination", 2024-06-10T01:10:55.020 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.020 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.020 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.020 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.021 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.021 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.021 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.021 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email address to send alerts to", 2024-06-10T01:10:55.021 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.021 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.021 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.021 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.022 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_from_name": { 2024-06-10T01:10:55.022 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_from_name", 2024-06-10T01:10:55.022 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.022 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.022 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.022 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "Ceph", 2024-06-10T01:10:55.022 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.022 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.023 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.023 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email From: name", 2024-06-10T01:10:55.023 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.023 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.023 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.023 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_host": { 2024-06-10T01:10:55.023 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_host", 2024-06-10T01:10:55.024 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.024 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.024 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.024 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.024 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.024 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.024 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.024 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP server", 2024-06-10T01:10:55.024 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.025 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.025 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.025 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_password": { 2024-06-10T01:10:55.025 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_password", 2024-06-10T01:10:55.025 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.025 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.025 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.026 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.026 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.026 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.026 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.026 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Password to authenticate with", 2024-06-10T01:10:55.026 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.026 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.026 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.027 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_port": { 2024-06-10T01:10:55.027 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_port", 2024-06-10T01:10:55.027 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.027 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.027 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.027 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "465", 2024-06-10T01:10:55.027 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.028 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.028 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.028 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP port", 2024-06-10T01:10:55.028 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.028 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.028 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.028 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.028 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_sender": { 2024-06-10T01:10:55.028 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_sender", 2024-06-10T01:10:55.029 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.029 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.029 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.029 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.029 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.029 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.029 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.029 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP envelope sender", 2024-06-10T01:10:55.030 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.030 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.030 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.030 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.030 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_ssl": { 2024-06-10T01:10:55.030 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_ssl", 2024-06-10T01:10:55.031 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.031 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.031 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.031 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.031 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.031 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.031 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.031 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-10T01:10:55.031 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.032 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.032 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.032 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.032 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_user": { 2024-06-10T01:10:55.032 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_user", 2024-06-10T01:10:55.032 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.032 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.032 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.033 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.033 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.033 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.033 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.033 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "User to authenticate as", 2024-06-10T01:10:55.033 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.033 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.033 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.033 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.034 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.034 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.034 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "balancer", 2024-06-10T01:10:55.034 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.034 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.034 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.034 INFO:teuthology.orchestra.run.smithi037.stdout: "active": { 2024-06-10T01:10:55.035 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "active", 2024-06-10T01:10:55.035 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.035 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.035 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.035 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.035 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.035 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.035 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.036 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically balance PGs across cluster", 2024-06-10T01:10:55.036 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.036 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.036 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.036 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_time": { 2024-06-10T01:10:55.036 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_time", 2024-06-10T01:10:55.036 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.037 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.037 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.037 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0000", 2024-06-10T01:10:55.037 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.037 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.037 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.037 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "beginning time of day to automatically balance", 2024-06-10T01:10:55.037 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-10T01:10:55.037 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.038 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.038 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_weekday": { 2024-06-10T01:10:55.038 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_weekday", 2024-06-10T01:10:55.038 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-06-10T01:10:55.038 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.038 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.038 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-06-10T01:10:55.039 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-06-10T01:10:55.039 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-06-10T01:10:55.039 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.039 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-10T01:10:55.039 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-10T01:10:55.039 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.039 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.039 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.040 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_max_iterations": { 2024-06-10T01:10:55.040 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_max_iterations", 2024-06-10T01:10:55.040 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-06-10T01:10:55.040 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.040 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.040 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "25", 2024-06-10T01:10:55.040 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-06-10T01:10:55.040 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "250", 2024-06-10T01:10:55.041 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.041 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-10T01:10:55.041 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.041 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.041 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.041 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.041 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_metrics": { 2024-06-10T01:10:55.041 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_metrics", 2024-06-10T01:10:55.041 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.042 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.042 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.042 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "pgs,objects,bytes", 2024-06-10T01:10:55.042 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.042 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.042 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.042 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-10T01:10:55.043 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-10T01:10:55.043 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.043 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.043 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.043 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_step": { 2024-06-10T01:10:55.043 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_step", 2024-06-10T01:10:55.043 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-06-10T01:10:55.043 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.044 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.044 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.5", 2024-06-10T01:10:55.044 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0.001", 2024-06-10T01:10:55.044 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "0.999", 2024-06-10T01:10:55.044 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.044 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "aggressiveness of optimization", 2024-06-10T01:10:55.044 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-10T01:10:55.044 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.044 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.045 INFO:teuthology.orchestra.run.smithi037.stdout: "end_time": { 2024-06-10T01:10:55.045 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_time", 2024-06-10T01:10:55.045 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.045 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.045 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.045 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2359", 2024-06-10T01:10:55.045 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.046 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.046 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.046 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ending time of day to automatically balance", 2024-06-10T01:10:55.046 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-10T01:10:55.046 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.046 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.046 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.046 INFO:teuthology.orchestra.run.smithi037.stdout: "end_weekday": { 2024-06-10T01:10:55.047 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_weekday", 2024-06-10T01:10:55.047 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-06-10T01:10:55.047 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.047 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.047 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-06-10T01:10:55.047 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-06-10T01:10:55.047 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-06-10T01:10:55.047 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.047 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-10T01:10:55.048 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-10T01:10:55.048 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.048 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.048 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.048 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.048 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.048 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.049 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.049 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.049 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.049 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.049 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.049 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.049 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.049 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.049 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.050 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.050 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.050 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.050 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.050 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.050 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.050 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.051 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.051 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.051 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.051 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.051 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.051 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.051 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.052 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.052 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.052 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.052 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.052 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.052 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.052 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.052 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.053 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.053 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.053 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.053 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.053 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.053 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.053 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.053 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.054 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.054 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.054 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.054 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.054 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.054 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.054 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.054 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.055 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.055 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.055 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.055 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.055 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.055 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.055 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.056 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.056 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.056 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.056 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.056 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.056 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.056 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.056 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.056 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.057 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.057 INFO:teuthology.orchestra.run.smithi037.stdout: "min_score": { 2024-06-10T01:10:55.057 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_score", 2024-06-10T01:10:55.057 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-06-10T01:10:55.057 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.057 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.057 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-06-10T01:10:55.057 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.058 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.058 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.058 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-10T01:10:55.058 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.058 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.058 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.058 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-06-10T01:10:55.059 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-06-10T01:10:55.059 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.059 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.059 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.059 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "upmap", 2024-06-10T01:10:55.059 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.059 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.059 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.060 INFO:teuthology.orchestra.run.smithi037.stdout: "crush-compat", 2024-06-10T01:10:55.060 INFO:teuthology.orchestra.run.smithi037.stdout: "none", 2024-06-10T01:10:55.060 INFO:teuthology.orchestra.run.smithi037.stdout: "read", 2024-06-10T01:10:55.060 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap", 2024-06-10T01:10:55.060 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap-read" 2024-06-10T01:10:55.060 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.060 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Balancer mode", 2024-06-10T01:10:55.060 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.060 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.061 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.061 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.061 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_ids": { 2024-06-10T01:10:55.061 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_ids", 2024-06-10T01:10:55.061 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.061 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.061 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.061 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.062 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.062 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.062 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.062 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-10T01:10:55.062 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.062 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.062 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.063 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-06-10T01:10:55.063 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-06-10T01:10:55.063 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.063 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.063 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.063 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-06-10T01:10:55.063 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.063 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.063 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.064 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-10T01:10:55.064 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.064 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.064 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.064 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_deviation": { 2024-06-10T01:10:55.064 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_deviation", 2024-06-10T01:10:55.064 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.065 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.065 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.065 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-06-10T01:10:55.065 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-06-10T01:10:55.065 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.065 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.065 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-10T01:10:55.065 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-10T01:10:55.065 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.066 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.066 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.066 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_optimizations": { 2024-06-10T01:10:55.066 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_optimizations", 2024-06-10T01:10:55.066 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-06-10T01:10:55.066 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.066 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.066 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-06-10T01:10:55.067 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.067 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.067 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.067 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-10T01:10:55.067 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.067 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.067 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.067 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.068 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.068 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.068 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm", 2024-06-10T01:10:55.068 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.068 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.068 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.068 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_down_multiplier": { 2024-06-10T01:10:55.069 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_down_multiplier", 2024-06-10T01:10:55.069 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-06-10T01:10:55.069 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.069 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.069 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-06-10T01:10:55.069 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.069 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.069 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.070 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-10T01:10:55.070 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.070 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.070 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.070 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_refresh_rate": { 2024-06-10T01:10:55.070 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_refresh_rate", 2024-06-10T01:10:55.070 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.071 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.071 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.071 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "20", 2024-06-10T01:10:55.071 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.071 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.071 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.071 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-10T01:10:55.071 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.072 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.072 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.072 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_starting_port": { 2024-06-10T01:10:55.072 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_starting_port", 2024-06-10T01:10:55.072 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.072 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.072 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.073 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4721", 2024-06-10T01:10:55.073 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.073 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.073 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.073 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-10T01:10:55.073 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.073 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.073 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_ptrace": { 2024-06-10T01:10:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_ptrace", 2024-06-10T01:10:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.075 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.075 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.075 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.075 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-10T01:10:55.075 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-10T01:10:55.075 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.075 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.075 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_interval": { 2024-06-10T01:10:55.076 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_interval", 2024-06-10T01:10:55.076 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.076 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.076 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.076 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-06-10T01:10:55.076 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.076 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.076 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.077 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-10T01:10:55.077 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.077 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.077 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.077 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_memory_target_ratio": { 2024-06-10T01:10:55.077 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_memory_target_ratio", 2024-06-10T01:10:55.077 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-06-10T01:10:55.077 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.078 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.078 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.7", 2024-06-10T01:10:55.078 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.078 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.078 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.078 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-10T01:10:55.078 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.078 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.079 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.079 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm_log_destination": { 2024-06-10T01:10:55.079 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm_log_destination", 2024-06-10T01:10:55.079 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.079 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.079 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.079 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.080 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.080 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.080 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.080 INFO:teuthology.orchestra.run.smithi037.stdout: "file", 2024-06-10T01:10:55.080 INFO:teuthology.orchestra.run.smithi037.stdout: "file,syslog", 2024-06-10T01:10:55.080 INFO:teuthology.orchestra.run.smithi037.stdout: "syslog" 2024-06-10T01:10:55.080 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.080 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-10T01:10:55.081 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.081 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.081 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.081 INFO:teuthology.orchestra.run.smithi037.stdout: "cgroups_split": { 2024-06-10T01:10:55.081 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cgroups_split", 2024-06-10T01:10:55.081 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.082 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.082 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.082 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.082 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.082 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.082 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.082 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-10T01:10:55.082 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.083 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.083 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.083 INFO:teuthology.orchestra.run.smithi037.stdout: "config_checks_enabled": { 2024-06-10T01:10:55.083 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_checks_enabled", 2024-06-10T01:10:55.083 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.083 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.083 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.084 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.084 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.084 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.084 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.084 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-10T01:10:55.084 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.084 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.084 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.085 INFO:teuthology.orchestra.run.smithi037.stdout: "config_dashboard": { 2024-06-10T01:10:55.085 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_dashboard", 2024-06-10T01:10:55.085 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.085 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.085 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.085 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.085 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.086 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.086 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.086 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-10T01:10:55.086 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.086 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.086 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.086 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_alertmanager": { 2024-06-10T01:10:55.087 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_alertmanager", 2024-06-10T01:10:55.087 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.087 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.087 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.087 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-10T01:10:55.087 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.087 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.087 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.088 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-06-10T01:10:55.088 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.088 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.088 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.088 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_base": { 2024-06-10T01:10:55.088 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_base", 2024-06-10T01:10:55.088 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.088 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.089 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.089 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-10T01:10:55.089 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.089 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.089 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.089 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Container image name, without the tag", 2024-06-10T01:10:55.089 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.089 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.090 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.090 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_elasticsearch": { 2024-06-10T01:10:55.090 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_elasticsearch", 2024-06-10T01:10:55.090 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.090 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.090 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.090 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-10T01:10:55.091 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.091 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.091 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.091 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "elasticsearch container image", 2024-06-10T01:10:55.091 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.091 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.091 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.092 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_grafana": { 2024-06-10T01:10:55.092 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_grafana", 2024-06-10T01:10:55.092 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.092 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.092 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.092 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-10T01:10:55.092 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.092 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.093 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.093 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-06-10T01:10:55.093 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.093 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.093 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.093 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.093 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_haproxy": { 2024-06-10T01:10:55.093 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_haproxy", 2024-06-10T01:10:55.094 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.094 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.094 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.094 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-10T01:10:55.094 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.094 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.094 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.095 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "HAproxy container image", 2024-06-10T01:10:55.095 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.095 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.095 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.095 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_agent": { 2024-06-10T01:10:55.095 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_agent", 2024-06-10T01:10:55.095 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.096 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.096 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.096 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-10T01:10:55.096 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.096 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.096 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.096 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger agent container image", 2024-06-10T01:10:55.096 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.097 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.097 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.097 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_collector": { 2024-06-10T01:10:55.097 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_collector", 2024-06-10T01:10:55.097 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.097 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.097 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.097 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-10T01:10:55.098 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.098 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.098 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.098 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger collector container image", 2024-06-10T01:10:55.098 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.098 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.098 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.099 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_query": { 2024-06-10T01:10:55.099 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_query", 2024-06-10T01:10:55.099 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.099 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.099 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.099 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-10T01:10:55.099 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.099 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.100 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.100 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger query container image", 2024-06-10T01:10:55.100 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.100 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.100 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.100 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_keepalived": { 2024-06-10T01:10:55.100 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_keepalived", 2024-06-10T01:10:55.101 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.101 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.101 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.101 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-10T01:10:55.101 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.101 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.101 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.101 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Keepalived container image", 2024-06-10T01:10:55.102 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.102 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.102 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.102 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_loki": { 2024-06-10T01:10:55.102 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_loki", 2024-06-10T01:10:55.102 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.102 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.103 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.103 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-10T01:10:55.103 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.103 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.103 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.103 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Loki container image", 2024-06-10T01:10:55.103 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.103 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.104 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.104 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_node_exporter": { 2024-06-10T01:10:55.104 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_node_exporter", 2024-06-10T01:10:55.104 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.104 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.104 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.104 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-10T01:10:55.105 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.105 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.105 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.105 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-06-10T01:10:55.105 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.105 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.105 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.106 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_nvmeof": { 2024-06-10T01:10:55.106 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_nvmeof", 2024-06-10T01:10:55.106 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.106 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.106 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.106 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-10T01:10:55.106 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.107 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.107 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.107 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Nvme-of container image", 2024-06-10T01:10:55.107 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.107 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.107 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.107 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_prometheus": { 2024-06-10T01:10:55.108 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_prometheus", 2024-06-10T01:10:55.108 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.108 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.108 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.108 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-10T01:10:55.108 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.108 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.108 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.109 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-06-10T01:10:55.109 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.109 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.109 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.109 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_promtail": { 2024-06-10T01:10:55.109 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_promtail", 2024-06-10T01:10:55.109 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.110 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.110 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.110 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-10T01:10:55.110 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.110 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.110 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.110 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Promtail container image", 2024-06-10T01:10:55.110 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.111 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.111 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.111 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.111 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_samba": { 2024-06-10T01:10:55.111 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_samba", 2024-06-10T01:10:55.111 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.111 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.111 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.111 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-10T01:10:55.112 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.112 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.112 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.112 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Samba/SMB container image", 2024-06-10T01:10:55.112 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.112 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.112 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.113 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_snmp_gateway": { 2024-06-10T01:10:55.113 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_snmp_gateway", 2024-06-10T01:10:55.113 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.113 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.113 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.113 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-10T01:10:55.113 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.113 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.114 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.114 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SNMP Gateway container image", 2024-06-10T01:10:55.114 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.114 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.114 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.114 INFO:teuthology.orchestra.run.smithi037.stdout: "container_init": { 2024-06-10T01:10:55.114 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_init", 2024-06-10T01:10:55.115 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.115 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.115 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.115 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.115 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.115 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.115 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Run podman/docker with `--init`", 2024-06-10T01:10:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: "daemon_cache_timeout": { 2024-06-10T01:10:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "daemon_cache_timeout", 2024-06-10T01:10:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.117 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.117 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.117 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-06-10T01:10:55.117 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.117 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.117 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.117 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-10T01:10:55.117 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.118 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.118 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.118 INFO:teuthology.orchestra.run.smithi037.stdout: "default_cephadm_command_timeout": { 2024-06-10T01:10:55.118 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_cephadm_command_timeout", 2024-06-10T01:10:55.118 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.118 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.118 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.118 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "900", 2024-06-10T01:10:55.119 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.119 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.119 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.119 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-10T01:10:55.119 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.119 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.119 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.120 INFO:teuthology.orchestra.run.smithi037.stdout: "default_registry": { 2024-06-10T01:10:55.120 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_registry", 2024-06-10T01:10:55.120 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.120 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.120 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.120 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io", 2024-06-10T01:10:55.120 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.120 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.121 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.121 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-10T01:10:55.121 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.121 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.121 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.121 INFO:teuthology.orchestra.run.smithi037.stdout: "device_cache_timeout": { 2024-06-10T01:10:55.121 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_cache_timeout", 2024-06-10T01:10:55.122 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.122 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.122 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.122 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1800", 2024-06-10T01:10:55.122 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.122 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.122 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.122 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache device inventory", 2024-06-10T01:10:55.123 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.123 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.123 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.123 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.123 INFO:teuthology.orchestra.run.smithi037.stdout: "device_enhanced_scan": { 2024-06-10T01:10:55.123 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_enhanced_scan", 2024-06-10T01:10:55.123 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.123 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.124 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.124 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.124 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.124 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.124 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.124 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-10T01:10:55.124 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.124 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: "facts_cache_timeout": { 2024-06-10T01:10:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "facts_cache_timeout", 2024-06-10T01:10:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-06-10T01:10:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache host facts data", 2024-06-10T01:10:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "grafana_dashboards_path": { 2024-06-10T01:10:55.127 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "grafana_dashboards_path", 2024-06-10T01:10:55.127 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.127 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.127 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.127 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-10T01:10:55.127 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.127 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.127 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.127 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-10T01:10:55.128 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.128 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.128 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.128 INFO:teuthology.orchestra.run.smithi037.stdout: "host_check_interval": { 2024-06-10T01:10:55.128 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "host_check_interval", 2024-06-10T01:10:55.128 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.128 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.129 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.129 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-06-10T01:10:55.129 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.129 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.129 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.129 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to perform a host check", 2024-06-10T01:10:55.129 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.129 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.130 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.130 INFO:teuthology.orchestra.run.smithi037.stdout: "hw_monitoring": { 2024-06-10T01:10:55.130 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hw_monitoring", 2024-06-10T01:10:55.130 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.130 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.130 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.130 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.131 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.131 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.131 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.131 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-10T01:10:55.131 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.131 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.131 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.131 INFO:teuthology.orchestra.run.smithi037.stdout: "inventory_list_all": { 2024-06-10T01:10:55.132 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "inventory_list_all", 2024-06-10T01:10:55.132 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.132 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.132 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.132 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.132 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.132 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.132 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.133 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-10T01:10:55.133 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.133 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.133 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.133 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.133 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.133 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.134 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.134 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.134 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.134 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.134 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.134 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.134 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.134 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.135 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.135 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.135 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.135 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.135 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.135 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.135 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.135 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.136 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.136 INFO:teuthology.orchestra.run.smithi037.stdout: "log_refresh_metadata": { 2024-06-10T01:10:55.136 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_refresh_metadata", 2024-06-10T01:10:55.136 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.136 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.136 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.137 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.137 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.137 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.137 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.137 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-10T01:10:55.137 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.137 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.137 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.138 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.138 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.138 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.138 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.138 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.138 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.138 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.139 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.139 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.139 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-10T01:10:55.139 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.139 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.139 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.139 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.140 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.140 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.140 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.140 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.140 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.140 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.140 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.140 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.141 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.141 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.141 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.141 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.141 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.141 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.141 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.141 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.142 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.142 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.142 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.142 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.142 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.142 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.143 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.143 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.143 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.143 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.143 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.143 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.143 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.143 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.143 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.144 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.144 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.144 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-10T01:10:55.144 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-10T01:10:55.144 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.144 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.144 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.144 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.145 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.145 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.145 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.145 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-10T01:10:55.145 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.145 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.145 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.145 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.146 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-10T01:10:55.146 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-10T01:10:55.146 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.146 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.146 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.146 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "*", 2024-06-10T01:10:55.146 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.146 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.146 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.147 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-10T01:10:55.147 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.147 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.147 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.147 INFO:teuthology.orchestra.run.smithi037.stdout: "max_count_per_host": { 2024-06-10T01:10:55.147 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_count_per_host", 2024-06-10T01:10:55.147 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.148 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.148 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.148 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-06-10T01:10:55.148 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.148 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.148 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.148 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of daemons per service per host", 2024-06-10T01:10:55.148 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.149 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.149 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.149 INFO:teuthology.orchestra.run.smithi037.stdout: "max_osd_draining_count": { 2024-06-10T01:10:55.149 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_osd_draining_count", 2024-06-10T01:10:55.149 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.149 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.149 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.149 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-06-10T01:10:55.150 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.150 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.150 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.150 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-10T01:10:55.150 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.150 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.150 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.151 INFO:teuthology.orchestra.run.smithi037.stdout: "migration_current": { 2024-06-10T01:10:55.151 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "migration_current", 2024-06-10T01:10:55.151 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.151 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.151 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.151 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.151 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.151 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.152 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.152 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "internal - do not modify", 2024-06-10T01:10:55.152 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.152 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.152 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.152 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.152 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-06-10T01:10:55.152 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-06-10T01:10:55.153 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.153 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.153 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.153 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "root", 2024-06-10T01:10:55.153 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.153 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.153 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.153 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm-package", 2024-06-10T01:10:55.154 INFO:teuthology.orchestra.run.smithi037.stdout: "root" 2024-06-10T01:10:55.154 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.154 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "mode for remote execution of cephadm", 2024-06-10T01:10:55.154 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.154 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.154 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.154 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.154 INFO:teuthology.orchestra.run.smithi037.stdout: "oob_default_addr": { 2024-06-10T01:10:55.155 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "oob_default_addr", 2024-06-10T01:10:55.155 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.155 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.155 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.155 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "169.254.1.1", 2024-06-10T01:10:55.155 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.155 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.155 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.155 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-10T01:10:55.156 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.156 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.156 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.156 INFO:teuthology.orchestra.run.smithi037.stdout: "prometheus_alerts_path": { 2024-06-10T01:10:55.156 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus_alerts_path", 2024-06-10T01:10:55.156 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.156 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.157 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.157 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-10T01:10:55.157 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.157 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.157 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.157 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-10T01:10:55.157 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.157 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.157 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.158 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.158 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_insecure": { 2024-06-10T01:10:55.158 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_insecure", 2024-06-10T01:10:55.158 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.158 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.158 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.158 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.159 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.159 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.159 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.159 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-10T01:10:55.159 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.159 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.159 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.159 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.160 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_password": { 2024-06-10T01:10:55.160 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_password", 2024-06-10T01:10:55.160 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.160 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.160 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.160 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.160 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.160 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.160 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.161 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-10T01:10:55.161 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.161 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.161 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.161 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.161 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_url": { 2024-06-10T01:10:55.161 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_url", 2024-06-10T01:10:55.161 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.162 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.162 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.162 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.162 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.162 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.162 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.162 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-10T01:10:55.163 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.163 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.163 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.163 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.163 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_username": { 2024-06-10T01:10:55.163 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_username", 2024-06-10T01:10:55.163 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.163 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.163 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.164 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.164 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.164 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.164 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.164 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-10T01:10:55.164 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.164 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.164 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.165 INFO:teuthology.orchestra.run.smithi037.stdout: "secure_monitoring_stack": { 2024-06-10T01:10:55.165 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secure_monitoring_stack", 2024-06-10T01:10:55.165 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.165 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.165 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.165 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.165 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.165 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.166 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.166 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-10T01:10:55.166 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.166 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.166 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.166 INFO:teuthology.orchestra.run.smithi037.stdout: "service_discovery_port": { 2024-06-10T01:10:55.166 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "service_discovery_port", 2024-06-10T01:10:55.167 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.167 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.167 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.167 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8765", 2024-06-10T01:10:55.167 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.167 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.167 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.167 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "cephadm service discovery port", 2024-06-10T01:10:55.168 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.168 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.168 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.168 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.168 INFO:teuthology.orchestra.run.smithi037.stdout: "ssh_config_file": { 2024-06-10T01:10:55.168 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssh_config_file", 2024-06-10T01:10:55.168 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.168 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.169 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.169 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.169 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.169 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.169 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.169 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-10T01:10:55.169 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.169 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.170 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.170 INFO:teuthology.orchestra.run.smithi037.stdout: "use_agent": { 2024-06-10T01:10:55.170 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_agent", 2024-06-10T01:10:55.170 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.170 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.170 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.170 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.171 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.171 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.171 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.171 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-10T01:10:55.171 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.171 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.171 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.172 INFO:teuthology.orchestra.run.smithi037.stdout: "use_repo_digest": { 2024-06-10T01:10:55.172 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_repo_digest", 2024-06-10T01:10:55.172 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.172 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.172 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.172 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.172 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.172 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.173 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.173 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-10T01:10:55.173 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.173 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.173 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.173 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.173 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_failed_host_check": { 2024-06-10T01:10:55.173 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_failed_host_check", 2024-06-10T01:10:55.174 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.174 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.174 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.174 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.174 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.174 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.174 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.175 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if the host check fails", 2024-06-10T01:10:55.175 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.175 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.175 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.175 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.175 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_daemons": { 2024-06-10T01:10:55.175 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_daemons", 2024-06-10T01:10:55.175 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.175 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.176 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.176 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.176 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.176 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.176 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.176 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-10T01:10:55.176 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.176 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.176 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.177 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_hosts": { 2024-06-10T01:10:55.177 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_hosts", 2024-06-10T01:10:55.177 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.177 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.177 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.177 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.177 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.177 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.178 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.178 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-10T01:10:55.178 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.178 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.178 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.178 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.178 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.178 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.179 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.179 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crash", 2024-06-10T01:10:55.179 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.179 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.179 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.179 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.179 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.179 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.180 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.180 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.180 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.180 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.180 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.180 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.180 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.180 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.181 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.181 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.181 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.181 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.181 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.181 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.181 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.181 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.182 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.182 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.182 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.182 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.182 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.182 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.182 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.182 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.183 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.183 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.183 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.183 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.183 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.183 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.183 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.183 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.184 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.184 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.184 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.184 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.184 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.184 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.184 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.184 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.184 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.185 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.185 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.185 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.185 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.185 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.185 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.185 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.185 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.186 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.186 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.186 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.186 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.186 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.186 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.186 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.187 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.187 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.187 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.187 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.187 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.187 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.187 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.187 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.187 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.188 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.188 INFO:teuthology.orchestra.run.smithi037.stdout: "retain_interval": { 2024-06-10T01:10:55.188 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retain_interval", 2024-06-10T01:10:55.188 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.188 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.188 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.188 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "31536000", 2024-06-10T01:10:55.188 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.189 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.189 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.189 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-10T01:10:55.189 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.189 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.189 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.189 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.189 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_recent_interval": { 2024-06-10T01:10:55.190 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_recent_interval", 2024-06-10T01:10:55.190 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.190 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.190 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.190 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1209600", 2024-06-10T01:10:55.190 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.190 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.190 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.191 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-10T01:10:55.191 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.191 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.191 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.191 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.191 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.191 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.191 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.191 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dashboard", 2024-06-10T01:10:55.192 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.192 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.192 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.192 INFO:teuthology.orchestra.run.smithi037.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-10T01:10:55.192 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-10T01:10:55.192 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.192 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.192 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.193 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-06-10T01:10:55.193 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.193 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.193 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.193 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.193 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.193 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.193 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.193 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.194 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_HOST": { 2024-06-10T01:10:55.194 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-10T01:10:55.194 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.194 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.194 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.194 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.194 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.194 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.195 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.195 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.195 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.195 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.195 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.195 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.195 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-10T01:10:55.195 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-10T01:10:55.196 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.196 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.196 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.196 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.196 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.196 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.196 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.196 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.197 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.197 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.197 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.197 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_ENABLED": { 2024-06-10T01:10:55.197 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_ENABLED", 2024-06-10T01:10:55.197 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.197 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.197 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.198 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.198 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.198 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.198 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.198 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.198 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.198 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.198 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.199 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.199 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-10T01:10:55.199 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-10T01:10:55.199 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.199 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.199 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.199 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.199 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.200 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.200 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.200 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.200 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.200 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.200 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.200 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.200 INFO:teuthology.orchestra.run.smithi037.stdout: "ENABLE_BROWSABLE_API": { 2024-06-10T01:10:55.201 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-10T01:10:55.201 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.201 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.201 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.202 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.202 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.202 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.202 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.202 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.202 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.202 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.203 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.203 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-10T01:10:55.203 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-10T01:10:55.203 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.203 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.203 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.203 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.203 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.204 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.204 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.204 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.204 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.204 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.204 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.204 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.204 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-10T01:10:55.204 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-10T01:10:55.205 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.205 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.205 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.205 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.205 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.205 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.205 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.205 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.206 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.206 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.206 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.206 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-10T01:10:55.206 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-10T01:10:55.206 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.206 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.207 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.207 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.207 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.207 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.207 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.207 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.207 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.207 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.207 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.208 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.208 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-10T01:10:55.208 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-10T01:10:55.208 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.208 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.208 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.208 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.208 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.208 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.209 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.209 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.209 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.209 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.209 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.209 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-10T01:10:55.209 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-10T01:10:55.209 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.210 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.210 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.210 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.210 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.210 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.210 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.210 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.211 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.211 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.211 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.211 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.211 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-10T01:10:55.211 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-10T01:10:55.211 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.211 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.212 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.212 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.212 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.212 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.212 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.212 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.212 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.212 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.212 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.213 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-10T01:10:55.213 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-10T01:10:55.213 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.213 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.213 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.213 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.213 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.214 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.214 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.214 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.214 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.214 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.214 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.214 INFO:teuthology.orchestra.run.smithi037.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-10T01:10:55.215 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-10T01:10:55.215 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.215 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.215 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.215 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.215 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.215 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.215 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.215 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.216 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.216 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.216 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.216 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_PASSWORD": { 2024-06-10T01:10:55.216 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-10T01:10:55.216 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.216 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.217 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.217 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-06-10T01:10:55.217 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.217 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.217 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.217 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.217 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.217 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.217 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.218 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-10T01:10:55.218 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-10T01:10:55.218 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.218 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.218 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.218 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.218 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.219 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.219 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.219 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.219 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.219 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.219 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.219 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_URL": { 2024-06-10T01:10:55.220 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_URL", 2024-06-10T01:10:55.220 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.220 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.220 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.220 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.220 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.220 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.220 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.220 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.221 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.221 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.221 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.221 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_USERNAME": { 2024-06-10T01:10:55.221 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-10T01:10:55.221 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.221 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.222 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.222 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-06-10T01:10:55.222 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.222 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.222 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.222 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.222 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.222 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.223 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.223 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-10T01:10:55.223 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-10T01:10:55.223 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.223 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.223 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.223 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.224 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.224 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.224 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.224 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.224 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.224 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.224 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.225 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-10T01:10:55.225 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-10T01:10:55.225 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.225 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.225 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.225 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.225 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.225 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.226 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.226 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.226 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.226 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.226 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.226 INFO:teuthology.orchestra.run.smithi037.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-10T01:10:55.226 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-10T01:10:55.227 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.227 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.227 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.227 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.227 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.227 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.227 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.227 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.228 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.228 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.228 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.228 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.228 INFO:teuthology.orchestra.run.smithi037.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-10T01:10:55.228 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-10T01:10:55.228 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.228 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.229 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.229 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.229 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.229 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.229 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.229 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.229 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.229 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.229 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.230 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_HOST": { 2024-06-10T01:10:55.230 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-10T01:10:55.230 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.230 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.230 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.230 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.231 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.231 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.231 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.231 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.231 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.231 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.231 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.232 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-10T01:10:55.232 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-10T01:10:55.232 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.232 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.232 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.232 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.232 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.232 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.233 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.233 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.233 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.233 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.233 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.233 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-10T01:10:55.233 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-10T01:10:55.234 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.234 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.234 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.234 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.234 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.234 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.234 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.234 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.235 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.235 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.235 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.235 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-10T01:10:55.235 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-10T01:10:55.235 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.235 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.236 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.236 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.236 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.236 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.236 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.236 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.236 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.236 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.236 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.237 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-10T01:10:55.237 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-10T01:10:55.237 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.237 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.237 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.237 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.237 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.237 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.237 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.238 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.238 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.238 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.238 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.238 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.238 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-10T01:10:55.238 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-10T01:10:55.238 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.239 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.239 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.239 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.239 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.239 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.239 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.239 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.239 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.239 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.240 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.240 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-10T01:10:55.240 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-10T01:10:55.240 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.240 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.240 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.240 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.241 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.241 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.241 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.241 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.241 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.241 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.241 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.242 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-10T01:10:55.242 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-10T01:10:55.242 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.242 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.242 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.242 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.242 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.242 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.243 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.243 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.243 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.243 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.243 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.243 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-10T01:10:55.243 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-10T01:10:55.244 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.244 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.244 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.244 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.244 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.244 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.244 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.244 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.245 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.245 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.245 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.245 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_ENABLED": { 2024-06-10T01:10:55.245 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-10T01:10:55.245 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.245 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.245 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.246 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.246 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.246 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.246 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.246 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.246 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.246 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.246 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.247 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-10T01:10:55.247 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-10T01:10:55.247 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.247 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.247 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.247 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-10T01:10:55.248 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.248 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.248 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.248 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.248 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.248 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.248 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.249 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-10T01:10:55.249 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-10T01:10:55.249 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.249 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.249 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.249 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-06-10T01:10:55.249 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.249 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.250 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.250 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.250 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.250 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.250 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.250 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-10T01:10:55.250 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-10T01:10:55.251 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.251 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.251 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.251 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8", 2024-06-10T01:10:55.251 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.251 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.251 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.251 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.251 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.252 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.252 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.252 INFO:teuthology.orchestra.run.smithi037.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-10T01:10:55.252 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-10T01:10:55.252 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.252 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.252 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.253 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "45", 2024-06-10T01:10:55.253 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.253 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.253 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.253 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.253 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.253 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.253 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.254 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ACCESS_KEY": { 2024-06-10T01:10:55.254 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-10T01:10:55.254 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.254 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.254 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.254 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.254 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.255 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.255 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.255 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.255 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.255 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.255 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.255 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-10T01:10:55.256 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-10T01:10:55.256 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.256 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.256 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.256 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-06-10T01:10:55.256 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.256 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.256 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.257 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.257 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.257 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.257 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.257 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SECRET_KEY": { 2024-06-10T01:10:55.257 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-10T01:10:55.257 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.258 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.258 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.258 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.258 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.258 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.258 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.258 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.258 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.259 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.259 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.259 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SSL_VERIFY": { 2024-06-10T01:10:55.259 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-10T01:10:55.259 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.259 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.259 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.260 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.260 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.260 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.260 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.260 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.260 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.260 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.260 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.261 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-10T01:10:55.261 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-10T01:10:55.261 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.261 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.261 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.261 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-06-10T01:10:55.261 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.261 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.262 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.262 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.262 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.262 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.262 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.262 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-10T01:10:55.262 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-10T01:10:55.263 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.263 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.263 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.263 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-06-10T01:10:55.263 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.263 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.263 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.263 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.264 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.264 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.264 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.264 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-10T01:10:55.264 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-10T01:10:55.264 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.264 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.265 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.265 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-06-10T01:10:55.265 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.265 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.265 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.265 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.265 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.265 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.265 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.266 INFO:teuthology.orchestra.run.smithi037.stdout: "cross_origin_url": { 2024-06-10T01:10:55.266 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cross_origin_url", 2024-06-10T01:10:55.266 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.266 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.266 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.266 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.266 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.267 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.267 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.267 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.267 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.267 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.267 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.267 INFO:teuthology.orchestra.run.smithi037.stdout: "crt_file": { 2024-06-10T01:10:55.268 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crt_file", 2024-06-10T01:10:55.268 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.268 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.268 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.268 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.268 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.268 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.268 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.268 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.269 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.269 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.269 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.269 INFO:teuthology.orchestra.run.smithi037.stdout: "debug": { 2024-06-10T01:10:55.269 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "debug", 2024-06-10T01:10:55.269 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.270 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.270 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.270 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.270 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.270 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.270 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.270 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable/disable debug options", 2024-06-10T01:10:55.270 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.271 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.271 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.271 INFO:teuthology.orchestra.run.smithi037.stdout: "jwt_token_ttl": { 2024-06-10T01:10:55.271 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "jwt_token_ttl", 2024-06-10T01:10:55.271 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.271 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.271 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.272 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "28800", 2024-06-10T01:10:55.272 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.272 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.272 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.272 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.272 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.272 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.272 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.273 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-06-10T01:10:55.273 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-06-10T01:10:55.273 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.273 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.273 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.273 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.273 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.274 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.274 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.274 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.274 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.274 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.274 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.274 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.275 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.275 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.275 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.275 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.275 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.275 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.275 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.275 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.276 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.276 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.276 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.276 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.276 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.276 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.276 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.276 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.276 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.277 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.277 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.277 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.277 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.277 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.277 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.277 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.277 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.278 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.278 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.278 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.278 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.278 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.278 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.278 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.278 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.279 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.279 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.279 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.279 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.279 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.279 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.279 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.279 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.279 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.280 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.280 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.280 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.280 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.280 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.280 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.280 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.280 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.281 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.281 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.281 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.281 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.281 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.281 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.281 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.282 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.282 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.282 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.282 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.282 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.282 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.282 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.282 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.282 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.283 INFO:teuthology.orchestra.run.smithi037.stdout: "motd": { 2024-06-10T01:10:55.283 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "motd", 2024-06-10T01:10:55.283 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.283 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.283 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.283 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.283 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.284 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.284 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.284 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "The message of the day", 2024-06-10T01:10:55.284 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.284 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.284 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.284 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect_resolve_ip_addr": { 2024-06-10T01:10:55.284 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "redirect_resolve_ip_addr", 2024-06-10T01:10:55.285 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.285 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.285 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.285 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.285 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.285 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.285 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.285 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.286 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.286 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.286 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.286 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-06-10T01:10:55.286 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-06-10T01:10:55.286 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.286 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.287 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.287 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-06-10T01:10:55.287 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.287 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.287 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.287 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.287 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.287 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.288 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.288 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-06-10T01:10:55.288 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-06-10T01:10:55.288 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.288 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.288 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.288 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8080", 2024-06-10T01:10:55.289 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.289 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.289 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.289 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.289 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.289 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.289 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.289 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.289 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-06-10T01:10:55.290 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-06-10T01:10:55.290 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.290 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.290 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.290 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.290 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.290 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.291 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.291 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.291 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.291 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.291 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.291 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl_server_port": { 2024-06-10T01:10:55.291 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl_server_port", 2024-06-10T01:10:55.291 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.292 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.292 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.292 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8443", 2024-06-10T01:10:55.292 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.292 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.292 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.292 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.292 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.293 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.293 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.293 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-06-10T01:10:55.293 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-06-10T01:10:55.293 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.293 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.294 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.294 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "redirect", 2024-06-10T01:10:55.294 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.294 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.294 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.294 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.294 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect" 2024-06-10T01:10:55.294 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.295 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.295 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.295 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.295 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.295 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-06-10T01:10:55.295 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-06-10T01:10:55.295 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.295 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.296 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.296 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-06-10T01:10:55.296 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-06-10T01:10:55.296 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-06-10T01:10:55.296 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.296 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.296 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.296 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.297 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.297 INFO:teuthology.orchestra.run.smithi037.stdout: "url_prefix": { 2024-06-10T01:10:55.297 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url_prefix", 2024-06-10T01:10:55.297 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.297 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.297 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.297 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.297 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.298 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.298 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.298 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.298 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.298 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.298 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.298 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.298 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.299 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.299 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-06-10T01:10:55.299 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.299 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.299 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.299 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_monitoring": { 2024-06-10T01:10:55.299 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_monitoring", 2024-06-10T01:10:55.299 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.300 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.300 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.300 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.300 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.300 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.300 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.300 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "monitor device health metrics", 2024-06-10T01:10:55.300 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.301 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.301 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.301 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.301 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.301 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.301 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.301 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.302 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.302 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.302 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.302 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.302 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.302 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.302 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.302 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.302 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.303 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.303 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.303 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.303 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.303 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.303 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.303 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.304 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.304 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.304 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.304 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.304 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.304 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.304 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.304 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.304 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.305 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.305 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.305 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.305 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.305 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.305 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.305 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.305 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.306 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.306 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.306 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.306 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.306 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.306 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.306 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.306 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.307 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.307 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.307 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.307 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.307 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.307 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.307 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: "mark_out_threshold": { 2024-06-10T01:10:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mark_out_threshold", 2024-06-10T01:10:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.310 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.310 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.310 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2419200", 2024-06-10T01:10:55.310 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.310 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.310 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.310 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-10T01:10:55.310 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.311 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.311 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.311 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_name": { 2024-06-10T01:10:55.311 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_name", 2024-06-10T01:10:55.311 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.311 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.311 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.312 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "device_health_metrics", 2024-06-10T01:10:55.312 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.312 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.312 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.312 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-10T01:10:55.312 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.312 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.312 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.313 INFO:teuthology.orchestra.run.smithi037.stdout: "retention_period": { 2024-06-10T01:10:55.313 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retention_period", 2024-06-10T01:10:55.313 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.313 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.313 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.313 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15552000", 2024-06-10T01:10:55.313 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.313 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.314 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.314 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain device health metrics", 2024-06-10T01:10:55.314 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.314 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.314 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.314 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_frequency": { 2024-06-10T01:10:55.314 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_frequency", 2024-06-10T01:10:55.315 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.315 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.315 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.315 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-06-10T01:10:55.315 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.315 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.315 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.315 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-10T01:10:55.316 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.316 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.316 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.316 INFO:teuthology.orchestra.run.smithi037.stdout: "self_heal": { 2024-06-10T01:10:55.316 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "self_heal", 2024-06-10T01:10:55.316 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.316 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.317 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.317 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.317 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.317 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.317 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.317 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-10T01:10:55.317 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.317 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.318 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.318 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-06-10T01:10:55.318 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-06-10T01:10:55.318 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.318 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.318 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.318 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-06-10T01:10:55.319 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.319 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.319 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.319 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and check device health", 2024-06-10T01:10:55.319 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.319 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.319 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.319 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_threshold": { 2024-06-10T01:10:55.320 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_threshold", 2024-06-10T01:10:55.320 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.320 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.320 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.320 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7257600", 2024-06-10T01:10:55.320 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.320 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.320 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.321 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-10T01:10:55.321 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.321 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.321 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.321 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.321 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.321 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.322 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "diskprediction_local", 2024-06-10T01:10:55.322 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.322 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.322 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.322 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.322 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.322 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.323 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.323 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.323 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.323 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.323 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.323 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.323 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.323 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.323 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.324 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.324 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.324 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.324 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.324 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.324 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.324 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.324 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.325 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.325 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.325 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.325 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.325 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.325 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.325 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.325 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.326 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.326 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.326 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.326 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.326 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.326 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.326 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.327 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.327 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.327 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.327 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.327 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.327 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.327 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.327 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.327 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.328 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.328 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.328 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.328 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.328 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.328 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.328 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.328 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.329 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.329 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.329 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.329 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.329 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.329 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.329 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.330 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.330 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.330 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.330 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.330 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.330 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.330 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.330 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.331 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.331 INFO:teuthology.orchestra.run.smithi037.stdout: "predict_interval": { 2024-06-10T01:10:55.331 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predict_interval", 2024-06-10T01:10:55.331 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.331 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.331 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.331 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-06-10T01:10:55.331 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.332 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.332 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.332 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.332 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.332 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.332 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.332 INFO:teuthology.orchestra.run.smithi037.stdout: "predictor_model": { 2024-06-10T01:10:55.332 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predictor_model", 2024-06-10T01:10:55.333 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.333 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.333 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.333 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "prophetstor", 2024-06-10T01:10:55.333 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.333 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.333 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.333 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.334 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.334 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.334 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.334 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-06-10T01:10:55.334 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-06-10T01:10:55.334 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.334 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.334 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.335 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-06-10T01:10:55.335 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.335 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.335 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.335 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.335 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.335 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.335 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "influx", 2024-06-10T01:10:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": false, 2024-06-10T01:10:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "influxdb python module not found", 2024-06-10T01:10:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: "batch_size": { 2024-06-10T01:10:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "batch_size", 2024-06-10T01:10:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5000", 2024-06-10T01:10:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-10T01:10:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: "database": { 2024-06-10T01:10:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "database", 2024-06-10T01:10:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "ceph", 2024-06-10T01:10:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-10T01:10:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": { 2024-06-10T01:10:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hostname", 2024-06-10T01:10:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server hostname", 2024-06-10T01:10:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.342 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-06-10T01:10:55.342 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-06-10T01:10:55.342 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.342 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.342 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.342 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "30", 2024-06-10T01:10:55.342 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "5", 2024-06-10T01:10:55.343 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.343 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.343 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-10T01:10:55.343 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.343 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.343 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.343 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.343 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.344 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.344 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.344 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.344 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.344 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.344 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.344 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.345 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.345 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.345 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.345 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.345 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.345 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.345 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.345 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.346 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.346 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.346 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.346 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.346 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.346 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.346 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.347 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.347 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.347 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.347 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.347 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.347 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.347 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.347 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.348 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.348 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.348 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.348 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.348 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.348 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.348 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.349 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.349 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.349 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.349 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.349 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.349 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.349 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.349 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.349 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.350 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.350 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.350 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.350 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.350 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.350 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.350 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.351 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.351 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.351 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.351 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.351 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.351 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.351 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.351 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.352 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.352 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.352 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.352 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.352 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.352 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.352 INFO:teuthology.orchestra.run.smithi037.stdout: "password": { 2024-06-10T01:10:55.352 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "password", 2024-06-10T01:10:55.353 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.353 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.353 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.353 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.353 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.353 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.353 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.353 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "password of InfluxDB server user", 2024-06-10T01:10:55.354 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.354 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.354 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.354 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.354 INFO:teuthology.orchestra.run.smithi037.stdout: "port": { 2024-06-10T01:10:55.354 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "port", 2024-06-10T01:10:55.354 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.354 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.355 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.355 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8086", 2024-06-10T01:10:55.355 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.355 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.355 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.355 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server port", 2024-06-10T01:10:55.355 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.355 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.355 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.356 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.356 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-06-10T01:10:55.356 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-06-10T01:10:55.356 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.356 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.356 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.356 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "false", 2024-06-10T01:10:55.356 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.357 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.357 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.357 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-10T01:10:55.357 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.357 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.357 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.357 INFO:teuthology.orchestra.run.smithi037.stdout: "threads": { 2024-06-10T01:10:55.357 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threads", 2024-06-10T01:10:55.358 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.358 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.358 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.358 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-06-10T01:10:55.358 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-06-10T01:10:55.358 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "32", 2024-06-10T01:10:55.358 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.358 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-10T01:10:55.359 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.359 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.359 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.359 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.359 INFO:teuthology.orchestra.run.smithi037.stdout: "username": { 2024-06-10T01:10:55.359 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "username", 2024-06-10T01:10:55.359 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.359 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.360 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.360 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.360 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.360 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.360 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.360 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "username of InfluxDB server user", 2024-06-10T01:10:55.360 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.360 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.360 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.361 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.361 INFO:teuthology.orchestra.run.smithi037.stdout: "verify_ssl": { 2024-06-10T01:10:55.361 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "verify_ssl", 2024-06-10T01:10:55.361 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.361 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.361 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.361 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "true", 2024-06-10T01:10:55.361 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.362 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.362 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.362 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-10T01:10:55.362 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.362 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.362 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.362 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.362 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.363 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.363 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "insights", 2024-06-10T01:10:55.363 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.363 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.363 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.363 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.363 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.364 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.364 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.364 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.364 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.364 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.364 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.364 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.364 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.365 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.365 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.365 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.365 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.365 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.365 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.365 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.365 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.366 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.366 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.366 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.366 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.366 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.366 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.366 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.366 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.367 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.367 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.367 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.367 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.367 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.367 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.367 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.367 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.368 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.368 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.368 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.368 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.368 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.368 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.368 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.368 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.369 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.369 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.369 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.369 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.369 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.369 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.369 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.369 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.370 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.370 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.370 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.370 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.370 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.370 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.370 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.370 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.371 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.371 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.371 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.371 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.371 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.371 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.371 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.371 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.371 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.372 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.372 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.372 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.372 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.372 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.372 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "iostat", 2024-06-10T01:10:55.372 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.372 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.373 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.373 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.373 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.373 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.373 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.373 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.373 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.373 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.374 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.374 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.374 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.374 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.374 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.374 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.374 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.374 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.375 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.375 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.375 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.375 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.375 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.375 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.375 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.375 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.376 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.376 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.376 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.376 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.376 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.376 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.376 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.376 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.377 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.377 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.377 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.377 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.377 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.377 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.377 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.377 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.378 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.378 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.378 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.378 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.378 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.378 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.378 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.378 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.379 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.379 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.379 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.379 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.379 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.379 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.379 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.379 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.380 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.380 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.380 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.380 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.380 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.380 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.380 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.380 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.381 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.381 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.381 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.381 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.381 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.381 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.381 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.381 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.382 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.382 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "k8sevents", 2024-06-10T01:10:55.382 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.382 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.382 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.382 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_event_retention_days": { 2024-06-10T01:10:55.382 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ceph_event_retention_days", 2024-06-10T01:10:55.382 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.383 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.383 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.383 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7", 2024-06-10T01:10:55.383 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.383 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.383 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.383 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-10T01:10:55.383 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.384 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.384 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.384 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.384 INFO:teuthology.orchestra.run.smithi037.stdout: "config_check_secs": { 2024-06-10T01:10:55.384 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_check_secs", 2024-06-10T01:10:55.384 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.384 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.384 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.385 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-06-10T01:10:55.385 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "10", 2024-06-10T01:10:55.385 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.385 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.385 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-10T01:10:55.385 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.385 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.385 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.386 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.386 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.386 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.386 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.386 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.386 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.386 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.387 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.387 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.387 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.387 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.387 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.387 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.387 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.387 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.388 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.388 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.388 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.388 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.388 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.388 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.388 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.388 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.388 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.389 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.389 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.389 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.389 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.389 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.389 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.389 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.389 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.390 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.390 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.390 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.390 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.390 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.390 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.390 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.390 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.391 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.391 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.391 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.391 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.391 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.391 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.391 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.391 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.391 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.392 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.392 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.392 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.392 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.392 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.392 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.392 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.392 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.393 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.393 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.393 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.393 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.393 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.393 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.393 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.393 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.394 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.394 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.394 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.394 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.394 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.394 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.394 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.394 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.395 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "localpool", 2024-06-10T01:10:55.395 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.395 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.395 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.395 INFO:teuthology.orchestra.run.smithi037.stdout: "failure_domain": { 2024-06-10T01:10:55.395 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "failure_domain", 2024-06-10T01:10:55.395 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.395 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.395 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.396 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "host", 2024-06-10T01:10:55.396 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.396 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.396 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.396 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "failure domain for any created local pool", 2024-06-10T01:10:55.396 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-10T01:10:55.396 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.396 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.397 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.397 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.397 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.397 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.397 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.397 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.397 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.397 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.398 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.398 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.398 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.398 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.398 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.398 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.398 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.398 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.399 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.399 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.399 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.399 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.399 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.399 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.399 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.399 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.400 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.400 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.400 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.400 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.400 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.400 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.400 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.400 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.401 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.401 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.401 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.401 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.401 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.401 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.401 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.401 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.402 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.402 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.402 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.402 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.402 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.402 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.402 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.402 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.403 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.403 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.403 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.403 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.403 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.403 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.403 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.404 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.404 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.404 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.404 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.404 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.404 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.404 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.404 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.405 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.405 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.405 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.405 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.405 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.405 INFO:teuthology.orchestra.run.smithi037.stdout: "min_size": { 2024-06-10T01:10:55.405 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_size", 2024-06-10T01:10:55.406 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.406 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.406 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.406 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.406 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.406 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.406 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.407 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default min_size for any created local pool", 2024-06-10T01:10:55.407 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-10T01:10:55.407 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.407 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.407 INFO:teuthology.orchestra.run.smithi037.stdout: "num_rep": { 2024-06-10T01:10:55.407 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "num_rep", 2024-06-10T01:10:55.407 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.408 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.408 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.408 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3", 2024-06-10T01:10:55.408 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.408 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.408 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.408 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default replica count for any created local pool", 2024-06-10T01:10:55.409 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.409 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.409 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.409 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_num": { 2024-06-10T01:10:55.409 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_num", 2024-06-10T01:10:55.409 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.409 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.410 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.410 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "128", 2024-06-10T01:10:55.410 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.410 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.410 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.410 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default pg_num for any created local pool", 2024-06-10T01:10:55.410 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.411 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.411 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.411 INFO:teuthology.orchestra.run.smithi037.stdout: "prefix": { 2024-06-10T01:10:55.411 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prefix", 2024-06-10T01:10:55.411 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.411 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.411 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.412 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.412 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.412 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.412 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.412 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name prefix for any created local pool", 2024-06-10T01:10:55.412 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.412 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.412 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.413 INFO:teuthology.orchestra.run.smithi037.stdout: "subtree": { 2024-06-10T01:10:55.413 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "subtree", 2024-06-10T01:10:55.413 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.413 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.413 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.413 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "rack", 2024-06-10T01:10:55.413 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.413 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.414 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.414 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-10T01:10:55.414 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-10T01:10:55.414 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.414 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.414 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.414 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.415 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.415 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mds_autoscaler", 2024-06-10T01:10:55.415 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.415 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.415 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.415 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.415 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.415 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.416 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.416 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.416 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.416 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.416 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.416 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.416 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.416 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.417 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.417 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.417 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.417 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.417 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.417 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.417 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.417 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.417 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.418 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.418 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.418 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.418 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.418 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.418 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.418 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.419 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.419 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.419 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.419 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.419 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.419 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.419 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.419 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.420 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.420 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.420 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.420 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.420 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.420 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.420 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.421 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.421 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.421 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.421 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.421 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.421 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.421 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.421 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.421 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.422 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.422 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.422 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.422 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.422 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.422 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.422 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.422 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.423 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.423 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.423 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.423 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.423 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.423 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.423 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.423 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.424 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.424 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.424 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.424 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirroring", 2024-06-10T01:10:55.424 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.424 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.424 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.424 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.425 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.425 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.425 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.425 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.425 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.425 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.425 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.425 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.426 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.426 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.426 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.426 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.426 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.426 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.426 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.426 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.427 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.427 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.427 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.427 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.427 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.427 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.427 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.427 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.428 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.428 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.428 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.428 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.428 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.428 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.428 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.428 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.429 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.429 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.429 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.429 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.429 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.429 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.429 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.430 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.430 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.430 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.430 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.430 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.430 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.430 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.430 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.431 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.431 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.431 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.431 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.431 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.431 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.431 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.432 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.432 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.432 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.432 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.432 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.432 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.432 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.432 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.433 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.433 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.433 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.433 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.433 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.433 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.433 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "nfs", 2024-06-10T01:10:55.433 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.434 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.434 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.434 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.434 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.434 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.434 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.434 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.434 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.435 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.435 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.435 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.435 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.435 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.435 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.435 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.435 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.435 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.436 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.436 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.436 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.436 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.436 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.436 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.436 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.437 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.437 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.437 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.437 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.437 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.437 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.437 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.437 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.438 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.438 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.438 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.438 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.438 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.438 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.438 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.438 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.439 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.439 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.439 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.439 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.439 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.439 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.439 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.439 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.439 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.440 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.440 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.440 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.440 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.440 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.440 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.440 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.441 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.441 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.441 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.441 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.441 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.441 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.441 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.441 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.442 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.442 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.442 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.442 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.442 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.442 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.442 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.443 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-06-10T01:10:55.443 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.443 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.443 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.443 INFO:teuthology.orchestra.run.smithi037.stdout: "fail_fs": { 2024-06-10T01:10:55.443 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "fail_fs", 2024-06-10T01:10:55.443 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.443 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.443 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.444 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.444 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.444 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.444 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.444 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-10T01:10:55.444 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.444 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.444 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.444 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.445 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.445 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.445 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.445 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.445 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.445 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.445 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.445 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.446 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.446 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.446 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.446 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.446 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.446 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.446 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.446 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.447 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.447 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.447 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.447 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.447 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.447 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.447 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.447 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.447 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.448 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.448 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.448 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.448 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.448 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.448 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.448 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.448 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.449 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.449 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.449 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.449 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.449 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.449 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.449 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.449 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.450 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.450 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.450 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.450 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.450 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.450 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.450 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.450 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.451 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.451 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.451 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.451 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.451 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.451 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.451 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.451 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.452 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.452 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.452 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.452 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.452 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.452 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.452 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.452 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.453 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.453 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.453 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.453 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.453 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.453 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator": { 2024-06-10T01:10:55.453 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-06-10T01:10:55.453 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.454 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.454 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.454 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.454 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.454 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.454 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.454 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-06-10T01:10:55.454 INFO:teuthology.orchestra.run.smithi037.stdout: "rook", 2024-06-10T01:10:55.454 INFO:teuthology.orchestra.run.smithi037.stdout: "test_orchestrator" 2024-06-10T01:10:55.455 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.455 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Orchestrator backend", 2024-06-10T01:10:55.455 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.455 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.455 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.455 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.455 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.456 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.456 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_perf_query", 2024-06-10T01:10:55.456 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.456 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.456 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.456 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.456 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.456 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.457 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.457 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.457 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.457 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.457 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.457 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.457 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.457 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.458 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.458 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.458 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.458 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.458 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.458 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.458 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.459 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.459 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.459 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.459 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.459 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.459 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.459 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.459 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.460 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.460 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.460 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.460 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.460 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.460 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.460 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.460 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.461 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.461 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.461 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.461 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.461 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.461 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.461 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.461 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.462 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.462 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.462 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.462 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.462 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.462 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.462 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.463 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.463 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.463 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.463 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.463 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.463 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.463 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.463 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.463 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.464 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.464 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.464 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.464 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.464 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.464 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.464 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.464 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.465 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.465 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.465 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.465 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.465 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.465 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_support", 2024-06-10T01:10:55.465 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.465 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.466 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.466 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.466 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.466 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.466 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.466 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.466 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.466 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.467 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.467 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.467 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.467 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.467 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.467 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.467 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.467 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.468 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.468 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.468 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.468 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.468 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.468 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.468 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.468 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.469 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.469 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.469 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.469 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.469 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.469 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.469 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.469 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.469 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.470 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.470 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.470 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.470 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.470 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.470 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.470 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.470 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.471 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.471 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.471 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.471 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.471 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.471 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.471 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.471 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.472 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.472 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.472 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.472 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.472 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.472 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.472 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.473 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.473 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.473 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.473 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.473 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.473 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.473 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.473 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.474 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.474 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.474 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.474 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.474 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.474 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.474 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.474 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.475 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.475 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_autoscaler", 2024-06-10T01:10:55.475 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.475 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.475 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.475 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.475 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.475 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.476 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.476 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.476 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.476 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.476 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.476 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.476 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.476 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.476 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.477 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.477 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.477 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.477 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.477 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.477 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.477 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.477 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.478 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.478 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.478 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.478 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.478 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.478 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.478 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.478 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.479 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.479 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.479 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.479 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.479 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.479 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.479 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.479 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.480 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.480 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.480 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.480 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.480 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.480 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.480 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.480 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.481 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.481 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.481 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.481 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.481 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.481 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.481 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.481 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.482 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.482 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.482 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.482 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.482 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.482 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.482 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.483 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.483 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.483 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.483 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.483 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.483 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.483 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.483 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.484 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.484 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.484 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-06-10T01:10:55.484 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-06-10T01:10:55.484 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.484 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.484 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.484 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-06-10T01:10:55.485 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.485 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.485 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.485 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.485 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.485 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.485 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.485 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.486 INFO:teuthology.orchestra.run.smithi037.stdout: "threshold": { 2024-06-10T01:10:55.486 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threshold", 2024-06-10T01:10:55.486 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-06-10T01:10:55.486 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.486 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.486 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-06-10T01:10:55.486 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1.0", 2024-06-10T01:10:55.486 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.487 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.487 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "scaling threshold", 2024-06-10T01:10:55.487 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-10T01:10:55.487 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.487 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.487 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.487 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.487 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.487 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.488 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "progress", 2024-06-10T01:10:55.488 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.488 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.488 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.488 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_pg_recovery_event": { 2024-06-10T01:10:55.488 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_pg_recovery_event", 2024-06-10T01:10:55.488 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.488 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.489 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.489 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.489 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.489 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.489 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.489 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-10T01:10:55.489 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.489 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.489 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.490 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.490 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-06-10T01:10:55.490 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-06-10T01:10:55.490 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.490 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.490 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.490 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.491 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.491 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.491 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.491 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.491 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.491 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.491 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.491 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.491 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.492 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.492 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.492 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.492 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.492 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.492 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.492 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.492 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.493 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.493 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.493 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.493 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.493 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.493 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.493 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.493 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.494 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.494 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.494 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.494 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.494 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.494 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.494 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.494 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.495 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.495 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.495 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.495 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.495 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.495 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.495 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.495 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.495 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.496 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.496 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.496 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.496 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.496 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.496 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.496 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.497 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.497 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.497 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.497 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.497 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.497 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.497 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.497 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.497 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.498 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.498 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.498 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.498 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.498 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.498 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.498 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.498 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.499 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.499 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.499 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.499 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.499 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.499 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.499 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.499 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.500 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.500 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.500 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.500 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.500 INFO:teuthology.orchestra.run.smithi037.stdout: "max_completed_events": { 2024-06-10T01:10:55.500 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_completed_events", 2024-06-10T01:10:55.500 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.500 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.501 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.501 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "50", 2024-06-10T01:10:55.501 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.501 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.501 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.501 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "number of past completed events to remember", 2024-06-10T01:10:55.501 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.501 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.501 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.502 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.502 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-06-10T01:10:55.502 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-06-10T01:10:55.502 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.502 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.502 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.502 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-06-10T01:10:55.502 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.503 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.503 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.503 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long the module is going to sleep", 2024-06-10T01:10:55.503 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.503 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.503 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.503 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.503 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.504 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.504 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus", 2024-06-10T01:10:55.504 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.504 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.504 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.504 INFO:teuthology.orchestra.run.smithi037.stdout: "cache": { 2024-06-10T01:10:55.504 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cache", 2024-06-10T01:10:55.505 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.505 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.505 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.505 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.505 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.505 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.505 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.505 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.505 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.506 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.506 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.506 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.506 INFO:teuthology.orchestra.run.smithi037.stdout: "exclude_perf_counters": { 2024-06-10T01:10:55.506 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "exclude_perf_counters", 2024-06-10T01:10:55.506 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.506 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.506 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.507 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.507 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.507 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.507 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.507 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-10T01:10:55.507 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-10T01:10:55.507 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.507 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.508 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.508 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.508 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.508 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.508 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.508 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.508 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.508 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.509 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.509 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.509 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.509 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.509 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.509 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.509 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.509 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.510 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.510 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.510 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.510 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.510 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.510 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.510 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.510 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.511 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.511 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.511 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.511 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.511 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.511 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.511 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.511 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.512 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.512 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.512 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.512 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.512 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.512 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.512 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.512 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.513 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.513 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.513 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.513 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.513 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.513 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.513 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.513 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.513 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.513 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.514 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.514 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.514 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.514 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.514 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.514 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.514 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.514 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.515 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.515 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.515 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.515 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.515 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.515 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.515 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.515 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.516 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.516 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.516 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.516 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.516 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.516 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools": { 2024-06-10T01:10:55.516 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools", 2024-06-10T01:10:55.517 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.517 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.517 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.517 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.517 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.517 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.517 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.517 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.517 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.518 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.518 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.518 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.518 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-10T01:10:55.518 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-10T01:10:55.518 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.518 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.519 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.519 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "300", 2024-06-10T01:10:55.519 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.519 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.519 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.519 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.519 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.519 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.520 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.520 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.520 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_interval": { 2024-06-10T01:10:55.520 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_interval", 2024-06-10T01:10:55.520 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-06-10T01:10:55.520 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.520 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.521 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15.0", 2024-06-10T01:10:55.521 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.521 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.521 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.521 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.521 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.521 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.521 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.522 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.522 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-06-10T01:10:55.522 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-06-10T01:10:55.522 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.522 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.522 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.522 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-06-10T01:10:55.522 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.523 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.523 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.523 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-10T01:10:55.523 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.523 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.523 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.523 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.523 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-06-10T01:10:55.524 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-06-10T01:10:55.524 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.524 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.524 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.524 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "9283", 2024-06-10T01:10:55.524 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.524 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.524 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.525 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-10T01:10:55.525 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.525 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.525 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.525 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.525 INFO:teuthology.orchestra.run.smithi037.stdout: "stale_cache_strategy": { 2024-06-10T01:10:55.525 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stale_cache_strategy", 2024-06-10T01:10:55.526 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.526 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.526 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.526 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "log", 2024-06-10T01:10:55.526 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.526 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.526 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.526 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.527 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.527 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.527 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.527 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.527 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-06-10T01:10:55.527 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-06-10T01:10:55.527 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.527 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.528 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.528 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "default", 2024-06-10T01:10:55.528 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.528 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.528 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.528 INFO:teuthology.orchestra.run.smithi037.stdout: "default", 2024-06-10T01:10:55.528 INFO:teuthology.orchestra.run.smithi037.stdout: "error" 2024-06-10T01:10:55.528 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.529 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.529 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.529 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.529 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.529 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-06-10T01:10:55.529 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-06-10T01:10:55.529 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.530 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.530 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.530 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-06-10T01:10:55.530 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-06-10T01:10:55.530 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-06-10T01:10:55.530 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.530 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.531 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.531 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.531 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.531 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.531 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.531 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.531 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.531 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-06-10T01:10:55.531 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.532 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.532 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.532 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.532 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.532 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.532 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.532 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.532 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.533 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.533 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.533 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.533 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.533 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.533 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.533 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.533 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.534 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.534 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.534 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.534 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.534 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.534 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.534 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.535 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.535 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.535 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.535 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.535 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.535 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.535 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.535 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.536 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.536 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.536 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.536 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.536 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.536 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.536 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.536 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.537 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.537 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.537 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.537 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.537 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.537 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.537 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.537 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.538 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.538 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.538 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.538 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.538 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.538 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.538 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.538 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.539 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.539 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.539 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.539 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.539 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.539 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.539 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.540 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.540 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.540 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.540 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.540 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.540 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.540 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.540 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.541 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.541 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_snap_create": { 2024-06-10T01:10:55.541 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_snap_create", 2024-06-10T01:10:55.541 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.541 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.541 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.541 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-06-10T01:10:55.541 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.542 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.542 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.542 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.542 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.542 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.542 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.542 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.543 INFO:teuthology.orchestra.run.smithi037.stdout: "mirror_snapshot_schedule": { 2024-06-10T01:10:55.543 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirror_snapshot_schedule", 2024-06-10T01:10:55.543 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.543 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.543 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.543 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.543 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.543 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.544 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.544 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.544 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.544 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.544 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.544 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.544 INFO:teuthology.orchestra.run.smithi037.stdout: "trash_purge_schedule": { 2024-06-10T01:10:55.544 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "trash_purge_schedule", 2024-06-10T01:10:55.545 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.545 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.545 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.545 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.545 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.545 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.545 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.545 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.546 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.546 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.546 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.546 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.546 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.546 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.546 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.547 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "restful", 2024-06-10T01:10:55.547 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.547 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.547 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.547 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_auth": { 2024-06-10T01:10:55.547 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_auth", 2024-06-10T01:10:55.547 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.547 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.548 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.548 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.548 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.548 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.548 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.548 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.548 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.548 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.549 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.549 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.549 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-06-10T01:10:55.549 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-06-10T01:10:55.549 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.549 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.549 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.549 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.549 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.550 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.550 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.550 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.550 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.550 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.551 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.551 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.551 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.551 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.551 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.551 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.551 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.551 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.552 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.552 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.552 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.552 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.552 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.552 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.552 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.552 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.553 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.553 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.553 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.553 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.553 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.553 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.553 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.553 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.554 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.554 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.554 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.554 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.554 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.554 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.554 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.555 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.555 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.555 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.555 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.555 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.555 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.555 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.555 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.556 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.556 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.556 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.556 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.556 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.556 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.556 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.556 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.557 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.557 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.557 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.557 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.557 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.557 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.557 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.557 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.558 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.558 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.558 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.558 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.558 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.558 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.558 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.559 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.559 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.559 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.559 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.559 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.559 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.559 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.560 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.560 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.560 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.560 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-06-10T01:10:55.560 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-06-10T01:10:55.560 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.560 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.560 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.561 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.561 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.561 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.561 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.561 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.561 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.561 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.561 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.562 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.562 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-06-10T01:10:55.562 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-06-10T01:10:55.562 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.562 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.562 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.562 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.563 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.563 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.563 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.563 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.563 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.563 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.563 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.563 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.563 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.564 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.564 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.564 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rgw", 2024-06-10T01:10:55.564 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.564 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.564 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.564 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.564 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.565 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.565 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.565 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.565 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.565 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.565 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.565 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.566 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.566 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.566 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.566 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.566 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.566 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.566 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.566 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.566 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.567 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.567 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.567 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.567 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.567 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.567 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.567 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.567 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.568 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.568 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.568 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.568 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.568 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.568 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.568 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.568 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.568 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.569 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.569 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.569 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.569 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.569 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.569 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.569 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.569 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.570 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.570 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.570 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.570 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.570 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.570 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.570 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.570 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.571 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.571 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.571 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.571 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.571 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.571 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.571 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.572 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.572 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.572 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.572 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.572 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.572 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.572 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.572 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.572 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.573 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.573 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.573 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.573 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.573 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.573 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rook", 2024-06-10T01:10:55.573 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.573 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.574 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.574 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.574 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.574 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.574 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.574 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.574 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.574 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.575 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.575 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.575 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.575 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.575 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.575 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.575 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.575 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.576 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.576 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.576 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.576 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.576 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.576 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.576 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.576 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.577 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.577 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.577 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.577 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.577 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.577 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.577 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.577 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.578 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.578 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.578 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.578 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.578 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.578 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.578 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.578 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.579 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.579 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.579 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.579 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.579 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.579 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.579 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.579 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.579 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.580 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.580 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.580 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.580 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.580 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.580 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.580 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.581 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.581 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.581 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.581 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.581 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.581 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.581 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.581 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.582 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.582 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.582 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.582 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.582 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.582 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.582 INFO:teuthology.orchestra.run.smithi037.stdout: "storage_class": { 2024-06-10T01:10:55.582 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "storage_class", 2024-06-10T01:10:55.583 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.583 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.583 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.583 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "local", 2024-06-10T01:10:55.583 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.584 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.584 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.584 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-10T01:10:55.584 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.584 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.584 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.584 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.585 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.585 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.585 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "selftest", 2024-06-10T01:10:55.585 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.585 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.586 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.586 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.586 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.586 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.586 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.586 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.587 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.587 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.587 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.587 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.587 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.587 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.588 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.588 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.588 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.588 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.588 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.589 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.589 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.589 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.589 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.589 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.589 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.589 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.589 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.590 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.590 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.590 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.590 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.590 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.590 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.591 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.591 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.591 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.591 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.591 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.591 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.591 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.592 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.592 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.592 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.592 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.592 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.592 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.592 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.593 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.593 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.593 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.593 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.593 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.593 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.593 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.593 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.594 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.594 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.594 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.594 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.594 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.594 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.594 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.595 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.595 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.595 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.595 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.595 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.595 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.595 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.595 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.596 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.596 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.596 INFO:teuthology.orchestra.run.smithi037.stdout: "roption1": { 2024-06-10T01:10:55.596 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption1", 2024-06-10T01:10:55.596 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.596 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.596 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.597 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.597 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.597 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.597 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.597 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.597 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.597 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.597 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.597 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.598 INFO:teuthology.orchestra.run.smithi037.stdout: "roption2": { 2024-06-10T01:10:55.598 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption2", 2024-06-10T01:10:55.598 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.598 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.598 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.598 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "xyz", 2024-06-10T01:10:55.598 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.599 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.599 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.599 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.599 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.599 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.599 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.599 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.599 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption1": { 2024-06-10T01:10:55.599 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption1", 2024-06-10T01:10:55.600 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.600 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.600 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.600 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.600 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.600 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.600 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.600 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.601 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.601 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.601 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.601 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption2": { 2024-06-10T01:10:55.601 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption2", 2024-06-10T01:10:55.601 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.601 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.602 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.602 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.602 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.602 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.602 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.602 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.603 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.603 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.603 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.603 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption3": { 2024-06-10T01:10:55.603 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption3", 2024-06-10T01:10:55.603 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-06-10T01:10:55.603 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.603 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.604 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.604 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.604 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.604 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.604 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.604 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.604 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.605 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.605 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.605 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption4": { 2024-06-10T01:10:55.605 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption4", 2024-06-10T01:10:55.605 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.605 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.605 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.605 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.605 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.606 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.606 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.606 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.606 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.606 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.606 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.606 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.607 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption5": { 2024-06-10T01:10:55.607 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption5", 2024-06-10T01:10:55.607 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.607 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.607 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.607 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.607 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.607 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.608 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.608 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.608 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.608 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.608 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.608 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.608 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption6": { 2024-06-10T01:10:55.608 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption6", 2024-06-10T01:10:55.609 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.609 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.609 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.609 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.609 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.609 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.609 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.609 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.610 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.610 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.610 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.610 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.610 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption7": { 2024-06-10T01:10:55.610 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption7", 2024-06-10T01:10:55.610 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.610 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.611 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.611 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.611 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-06-10T01:10:55.611 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "42", 2024-06-10T01:10:55.611 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.611 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.611 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.612 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.612 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.612 INFO:teuthology.orchestra.run.smithi037.stdout: "testkey": { 2024-06-10T01:10:55.612 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testkey", 2024-06-10T01:10:55.612 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.612 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.612 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.612 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.613 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.613 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.613 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.613 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.613 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.613 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.613 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.613 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.613 INFO:teuthology.orchestra.run.smithi037.stdout: "testlkey": { 2024-06-10T01:10:55.614 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testlkey", 2024-06-10T01:10:55.614 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.614 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.614 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.614 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.614 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.614 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.615 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.615 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.615 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.615 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.615 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.615 INFO:teuthology.orchestra.run.smithi037.stdout: "testnewline": { 2024-06-10T01:10:55.615 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testnewline", 2024-06-10T01:10:55.615 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.616 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.616 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.616 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.616 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.616 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.616 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.616 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.616 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.616 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.617 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.617 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.617 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.617 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.617 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-06-10T01:10:55.617 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.617 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.618 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.618 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_m_granularity": { 2024-06-10T01:10:55.618 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_m_granularity", 2024-06-10T01:10:55.618 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.618 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.618 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.618 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.618 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.618 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.619 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.619 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow minute scheduled snapshots", 2024-06-10T01:10:55.619 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.619 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.619 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.619 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.619 INFO:teuthology.orchestra.run.smithi037.stdout: "dump_on_update": { 2024-06-10T01:10:55.619 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dump_on_update", 2024-06-10T01:10:55.619 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.620 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.620 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.620 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.620 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.620 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.620 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.620 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "dump database to debug log on update", 2024-06-10T01:10:55.620 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.621 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.621 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.621 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.621 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.621 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.621 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.621 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.621 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.622 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.622 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.622 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.622 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.622 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.622 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.622 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.622 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.623 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.623 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.623 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.623 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.623 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.623 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.623 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.624 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.624 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.624 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.624 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.624 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.624 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.624 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.624 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.625 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.625 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.625 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.625 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.625 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.625 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.625 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.626 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.626 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.626 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.626 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.626 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.626 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.626 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.627 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.627 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.627 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.627 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.627 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.627 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.627 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.627 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.628 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.628 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.628 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.628 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.628 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.628 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.628 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.628 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.628 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.629 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.629 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.629 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.629 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.629 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.629 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.629 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.629 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.630 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.630 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.630 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.630 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.630 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stats", 2024-06-10T01:10:55.630 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.630 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.630 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.631 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.631 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.631 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.631 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.631 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.631 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.631 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.631 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.632 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.632 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.632 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.632 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.632 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.632 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.632 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.632 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.632 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.633 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.633 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.633 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.633 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.633 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.633 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.634 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.634 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.634 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.634 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.634 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.634 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.634 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.634 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.635 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.635 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.635 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.635 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.635 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.635 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.635 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.636 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.636 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.636 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.636 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.636 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.636 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.636 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.636 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.637 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.637 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.637 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.637 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.637 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.637 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.637 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.638 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.638 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.638 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.638 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.638 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.638 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.638 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.638 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.639 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.639 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.639 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.639 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.639 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.639 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.639 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.640 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.640 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "status", 2024-06-10T01:10:55.640 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.640 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.640 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.640 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.640 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.640 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.641 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.641 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.641 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.641 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.641 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.641 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.641 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.641 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.642 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.642 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.642 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.642 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.642 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.642 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.642 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.642 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.643 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.643 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.643 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.643 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.643 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.643 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.644 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.644 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.644 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.644 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.644 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.644 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.644 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.645 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.645 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.645 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.645 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.645 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.645 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.645 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.646 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.646 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.646 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.646 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.646 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.646 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.646 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.646 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.647 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.647 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.647 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.647 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.647 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.647 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.647 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.648 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.648 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.648 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.648 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.648 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.648 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.648 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.649 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.649 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.649 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.649 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.649 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.649 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.649 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.650 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telegraf", 2024-06-10T01:10:55.650 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.650 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.650 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.650 INFO:teuthology.orchestra.run.smithi037.stdout: "address": { 2024-06-10T01:10:55.650 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "address", 2024-06-10T01:10:55.651 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.651 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.651 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.651 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-10T01:10:55.651 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.651 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.651 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.652 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.652 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.652 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.652 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.652 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-06-10T01:10:55.652 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-06-10T01:10:55.652 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.653 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.653 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.653 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15", 2024-06-10T01:10:55.653 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.653 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.653 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.653 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.653 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.654 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.654 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.654 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.654 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.654 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.654 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.654 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.655 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.655 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.655 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.655 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.655 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.655 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.655 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.656 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.656 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.656 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.656 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.656 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.656 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.656 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.656 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.657 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.657 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.657 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.657 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.657 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.657 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.657 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.658 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.658 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.658 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.658 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.658 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.658 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.658 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.659 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.659 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.659 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.659 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.659 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.659 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.659 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.659 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.660 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.660 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.660 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.660 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.660 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.660 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.660 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.660 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.661 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.661 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.661 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.661 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.661 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.661 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.661 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.662 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.662 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.662 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.662 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.662 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.662 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.662 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.662 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.663 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.663 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.663 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.663 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.663 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telemetry", 2024-06-10T01:10:55.663 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.663 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.663 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.664 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_basic": { 2024-06-10T01:10:55.664 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_basic", 2024-06-10T01:10:55.664 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.664 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.664 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.664 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.664 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.664 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.665 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.665 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-10T01:10:55.665 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.665 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.665 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.665 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_crash": { 2024-06-10T01:10:55.665 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_crash", 2024-06-10T01:10:55.666 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.666 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.666 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.666 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.666 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.666 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.666 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.666 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-10T01:10:55.667 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.667 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.667 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.667 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_device": { 2024-06-10T01:10:55.668 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_device", 2024-06-10T01:10:55.668 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.668 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.668 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.668 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.668 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.668 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.668 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.668 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-10T01:10:55.669 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.669 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.669 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.669 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_ident": { 2024-06-10T01:10:55.669 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_ident", 2024-06-10T01:10:55.669 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.669 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.670 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.670 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.670 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.670 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.670 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.670 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-10T01:10:55.670 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.670 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.671 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.671 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_perf": { 2024-06-10T01:10:55.671 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_perf", 2024-06-10T01:10:55.671 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.671 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.671 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.671 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.672 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.672 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.672 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.672 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-10T01:10:55.672 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.672 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.672 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.672 INFO:teuthology.orchestra.run.smithi037.stdout: "contact": { 2024-06-10T01:10:55.673 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "contact", 2024-06-10T01:10:55.673 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.673 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.673 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.673 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.673 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.673 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.673 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.674 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.674 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.674 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.674 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.674 INFO:teuthology.orchestra.run.smithi037.stdout: "description": { 2024-06-10T01:10:55.674 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "description", 2024-06-10T01:10:55.674 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.675 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.675 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.675 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.675 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.675 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.675 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.675 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.675 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.676 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.676 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.676 INFO:teuthology.orchestra.run.smithi037.stdout: "device_url": { 2024-06-10T01:10:55.676 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_url", 2024-06-10T01:10:55.676 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.676 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.676 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.676 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-10T01:10:55.677 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.677 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.677 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.677 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.677 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.677 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.677 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.678 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-06-10T01:10:55.678 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-06-10T01:10:55.678 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.678 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.678 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.678 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.678 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.678 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.679 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.679 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.679 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.679 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.679 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.679 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-06-10T01:10:55.679 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-06-10T01:10:55.680 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.680 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.680 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.680 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "24", 2024-06-10T01:10:55.680 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "8", 2024-06-10T01:10:55.680 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.680 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.680 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.681 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.681 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.681 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.681 INFO:teuthology.orchestra.run.smithi037.stdout: "last_opt_revision": { 2024-06-10T01:10:55.681 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "last_opt_revision", 2024-06-10T01:10:55.681 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.681 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.682 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.682 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1", 2024-06-10T01:10:55.682 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.682 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.682 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.682 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.682 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.683 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.683 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.683 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard": { 2024-06-10T01:10:55.683 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard", 2024-06-10T01:10:55.683 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.683 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.683 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.684 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.684 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.684 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.684 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.684 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.684 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.684 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.684 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.685 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard_description": { 2024-06-10T01:10:55.685 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard_description", 2024-06-10T01:10:55.685 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.685 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.685 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.685 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.685 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.685 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.685 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.686 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.686 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.686 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.686 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.686 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.686 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.686 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.687 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.687 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.687 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.687 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.687 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.687 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.687 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.688 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.688 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.688 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.688 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.688 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.688 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.688 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.688 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.688 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.689 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.689 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.689 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.689 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.689 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.689 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.690 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.690 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.690 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.690 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.690 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.690 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.690 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.690 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.691 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.691 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.691 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.691 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.691 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.693 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.693 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.693 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.693 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.693 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.693 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.694 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.694 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.694 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.694 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.694 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.694 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.694 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.695 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.695 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.695 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.695 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.695 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.695 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.695 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.696 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.696 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.696 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.696 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.696 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.696 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.696 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.696 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.697 INFO:teuthology.orchestra.run.smithi037.stdout: "organization": { 2024-06-10T01:10:55.697 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "organization", 2024-06-10T01:10:55.697 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.697 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.697 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.697 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.697 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.698 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.698 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.698 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.698 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.698 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.698 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.698 INFO:teuthology.orchestra.run.smithi037.stdout: "proxy": { 2024-06-10T01:10:55.699 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "proxy", 2024-06-10T01:10:55.699 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.699 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.699 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.699 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.699 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.699 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.699 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.699 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.700 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.700 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.700 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.700 INFO:teuthology.orchestra.run.smithi037.stdout: "url": { 2024-06-10T01:10:55.700 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url", 2024-06-10T01:10:55.700 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.701 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.701 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.701 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-10T01:10:55.701 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.701 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.701 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.701 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.701 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.702 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.702 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.702 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.702 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.702 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.702 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "test_orchestrator", 2024-06-10T01:10:55.702 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.703 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.703 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.703 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.703 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.703 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.703 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.703 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.703 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.704 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.704 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.704 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.704 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.704 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.704 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.704 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.704 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.705 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.705 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.705 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.705 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.705 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.705 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.705 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.706 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.706 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.706 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.706 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.706 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.706 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.706 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.706 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.707 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.707 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.707 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.707 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.707 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.707 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.708 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.708 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.708 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.708 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.708 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.708 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.708 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.708 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.709 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.709 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.709 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.709 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.709 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.709 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.709 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.709 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.710 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.710 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.710 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.710 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.710 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.710 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.710 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.711 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.711 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.711 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.711 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.711 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.711 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.711 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.711 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.712 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.712 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.712 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.712 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-06-10T01:10:55.712 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.712 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.712 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.713 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.713 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.713 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.713 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.713 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.713 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.713 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.714 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.714 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.714 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.714 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.714 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.714 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.714 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.714 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.714 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.715 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.715 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.715 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.715 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.715 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.715 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.715 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.716 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.716 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.716 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.716 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.716 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.716 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.716 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.716 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.717 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.717 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.717 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.717 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.717 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.717 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.717 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.717 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.718 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.718 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.718 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.718 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.718 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.718 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.718 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.718 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.719 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.719 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.719 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.719 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.719 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.719 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.719 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.719 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.720 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.720 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.720 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.720 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.720 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.720 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.720 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.720 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.720 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.721 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.721 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.721 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_clones": { 2024-06-10T01:10:55.721 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_clones", 2024-06-10T01:10:55.721 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.721 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.721 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.722 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4", 2024-06-10T01:10:55.722 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.722 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.722 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.722 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-10T01:10:55.722 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.722 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.722 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.723 INFO:teuthology.orchestra.run.smithi037.stdout: "periodic_async_work": { 2024-06-10T01:10:55.723 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "periodic_async_work", 2024-06-10T01:10:55.723 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.723 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.723 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.723 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.723 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.724 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.724 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.724 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Periodically check for async work", 2024-06-10T01:10:55.724 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.724 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.724 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.725 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_delay": { 2024-06-10T01:10:55.725 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_delay", 2024-06-10T01:10:55.725 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.725 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.725 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.725 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-06-10T01:10:55.725 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.725 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.725 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.726 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-10T01:10:55.726 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.726 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.726 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.726 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_no_wait": { 2024-06-10T01:10:55.726 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_no_wait", 2024-06-10T01:10:55.726 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.727 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.727 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.727 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.727 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.727 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.727 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.727 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-10T01:10:55.727 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.728 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.728 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.728 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.728 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.728 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix", 2024-06-10T01:10:55.728 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.728 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.729 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.729 INFO:teuthology.orchestra.run.smithi037.stdout: "discovery_interval": { 2024-06-10T01:10:55.729 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "discovery_interval", 2024-06-10T01:10:55.729 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-06-10T01:10:55.729 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.729 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.729 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "100", 2024-06-10T01:10:55.729 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.730 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.730 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.730 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.730 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.730 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.730 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.730 INFO:teuthology.orchestra.run.smithi037.stdout: "identifier": { 2024-06-10T01:10:55.731 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "identifier", 2024-06-10T01:10:55.731 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.731 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.731 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.731 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.731 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.731 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.731 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.731 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.732 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.732 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.732 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.732 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-06-10T01:10:55.732 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-06-10T01:10:55.732 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.732 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.733 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.733 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-06-10T01:10:55.733 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.733 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.733 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.733 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.733 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.733 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.734 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.734 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.734 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.734 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.734 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.734 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.734 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.734 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.735 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.735 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.735 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.735 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.735 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.735 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.735 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.735 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.736 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.736 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.736 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.736 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.736 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.736 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.736 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.737 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.737 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.737 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.737 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.737 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.737 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.737 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.737 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.737 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.738 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.738 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.738 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.738 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.738 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.738 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.738 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.739 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.739 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.739 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.739 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.739 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.739 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.739 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.739 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.740 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.740 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.740 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.740 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.740 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.740 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.740 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.741 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.741 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.741 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.741 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.741 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.741 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.741 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.741 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.742 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.742 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.742 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.742 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.742 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.742 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_host": { 2024-06-10T01:10:55.743 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_host", 2024-06-10T01:10:55.743 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.743 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.743 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.743 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.743 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.743 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.743 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.744 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.744 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.744 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.744 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.744 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_port": { 2024-06-10T01:10:55.744 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_port", 2024-06-10T01:10:55.744 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.744 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.745 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.745 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10051", 2024-06-10T01:10:55.745 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.745 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.745 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.745 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.745 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.745 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.746 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.746 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_sender": { 2024-06-10T01:10:55.746 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_sender", 2024-06-10T01:10:55.746 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.746 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.746 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.747 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-10T01:10:55.747 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.747 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.747 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.747 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.747 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.747 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.747 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.747 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.748 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.748 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.748 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-06-10T01:10:55.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.748 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.748 INFO:teuthology.orchestra.run.smithi037.stdout: "gid": 24109, 2024-06-10T01:10:55.748 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "z", 2024-06-10T01:10:55.748 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr_features": 4540701547738038271, 2024-06-10T01:10:55.748 INFO:teuthology.orchestra.run.smithi037.stdout: "available_modules": [ 2024-06-10T01:10:55.749 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.749 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "alerts", 2024-06-10T01:10:55.749 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.749 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.749 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.749 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-06-10T01:10:55.749 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-06-10T01:10:55.749 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.750 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.750 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.750 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-06-10T01:10:55.750 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.750 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.750 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.750 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How frequently to reexamine health status", 2024-06-10T01:10:55.750 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.751 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.751 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.751 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.751 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.751 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.751 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.751 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.752 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.752 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.752 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.752 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.752 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.752 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.752 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.752 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.752 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.753 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.753 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.753 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.753 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.753 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.753 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.753 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.754 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.754 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.754 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.754 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.754 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.754 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.754 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.754 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.755 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.755 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.755 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.755 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.755 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.755 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.755 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.756 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.756 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.756 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.756 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.756 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.756 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.756 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.756 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.757 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.757 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.757 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.757 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.757 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.757 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.757 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.757 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.757 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.758 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.758 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.758 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.758 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.758 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.758 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.758 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.759 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.759 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.759 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.759 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.759 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.759 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.759 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_destination": { 2024-06-10T01:10:55.760 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_destination", 2024-06-10T01:10:55.760 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.760 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.760 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.760 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.760 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.760 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.760 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.761 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email address to send alerts to", 2024-06-10T01:10:55.761 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.761 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.761 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.761 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.761 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_from_name": { 2024-06-10T01:10:55.761 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_from_name", 2024-06-10T01:10:55.761 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.762 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.762 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.762 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "Ceph", 2024-06-10T01:10:55.762 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.762 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.762 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.762 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email From: name", 2024-06-10T01:10:55.763 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.763 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.763 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.763 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.763 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_host": { 2024-06-10T01:10:55.763 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_host", 2024-06-10T01:10:55.763 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.763 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.763 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.764 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.764 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.764 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.764 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.764 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP server", 2024-06-10T01:10:55.764 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.764 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.764 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.765 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_password": { 2024-06-10T01:10:55.765 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_password", 2024-06-10T01:10:55.765 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.765 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.765 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.765 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.765 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.766 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.766 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.766 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Password to authenticate with", 2024-06-10T01:10:55.766 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.766 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.766 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.766 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_port": { 2024-06-10T01:10:55.767 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_port", 2024-06-10T01:10:55.767 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.767 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.767 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.767 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "465", 2024-06-10T01:10:55.767 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.767 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.767 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.767 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP port", 2024-06-10T01:10:55.768 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.768 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.768 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.768 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_sender": { 2024-06-10T01:10:55.768 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_sender", 2024-06-10T01:10:55.768 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.768 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.769 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.769 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.769 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.769 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.769 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.769 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP envelope sender", 2024-06-10T01:10:55.769 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.769 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.770 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.770 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_ssl": { 2024-06-10T01:10:55.770 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_ssl", 2024-06-10T01:10:55.770 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.770 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.770 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.770 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.771 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.771 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.771 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.771 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-10T01:10:55.771 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.771 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.771 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.772 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_user": { 2024-06-10T01:10:55.772 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_user", 2024-06-10T01:10:55.772 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.772 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.772 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.772 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.772 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.772 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.773 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.773 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "User to authenticate as", 2024-06-10T01:10:55.773 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.773 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.773 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.773 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.773 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.773 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.774 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "balancer", 2024-06-10T01:10:55.774 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.774 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.774 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.774 INFO:teuthology.orchestra.run.smithi037.stdout: "active": { 2024-06-10T01:10:55.774 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "active", 2024-06-10T01:10:55.774 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.774 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.775 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.775 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.775 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.775 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.775 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.775 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically balance PGs across cluster", 2024-06-10T01:10:55.775 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.775 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.776 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.776 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.776 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_time": { 2024-06-10T01:10:55.776 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_time", 2024-06-10T01:10:55.776 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.776 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.776 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.776 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0000", 2024-06-10T01:10:55.777 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.777 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.777 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.777 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "beginning time of day to automatically balance", 2024-06-10T01:10:55.777 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-10T01:10:55.777 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.777 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.777 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.778 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_weekday": { 2024-06-10T01:10:55.778 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_weekday", 2024-06-10T01:10:55.778 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-06-10T01:10:55.778 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.778 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.778 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-06-10T01:10:55.778 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-06-10T01:10:55.778 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-06-10T01:10:55.779 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.779 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-10T01:10:55.779 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-10T01:10:55.779 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.779 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.779 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_max_iterations": { 2024-06-10T01:10:55.779 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_max_iterations", 2024-06-10T01:10:55.779 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-06-10T01:10:55.780 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.780 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.780 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "25", 2024-06-10T01:10:55.780 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-06-10T01:10:55.780 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "250", 2024-06-10T01:10:55.780 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.780 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-10T01:10:55.780 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.781 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.781 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.781 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_metrics": { 2024-06-10T01:10:55.781 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_metrics", 2024-06-10T01:10:55.781 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.781 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.781 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.782 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "pgs,objects,bytes", 2024-06-10T01:10:55.782 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.782 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.782 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.782 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-10T01:10:55.782 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-10T01:10:55.782 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.783 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.783 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_step": { 2024-06-10T01:10:55.783 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_step", 2024-06-10T01:10:55.783 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-06-10T01:10:55.783 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.783 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.783 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.5", 2024-06-10T01:10:55.783 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0.001", 2024-06-10T01:10:55.784 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "0.999", 2024-06-10T01:10:55.784 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.784 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "aggressiveness of optimization", 2024-06-10T01:10:55.784 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-10T01:10:55.784 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.784 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.784 INFO:teuthology.orchestra.run.smithi037.stdout: "end_time": { 2024-06-10T01:10:55.785 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_time", 2024-06-10T01:10:55.785 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.785 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.785 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.785 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2359", 2024-06-10T01:10:55.785 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.785 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.785 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.786 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ending time of day to automatically balance", 2024-06-10T01:10:55.786 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-10T01:10:55.786 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.786 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.786 INFO:teuthology.orchestra.run.smithi037.stdout: "end_weekday": { 2024-06-10T01:10:55.786 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_weekday", 2024-06-10T01:10:55.786 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-06-10T01:10:55.787 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.787 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.787 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-06-10T01:10:55.787 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-06-10T01:10:55.787 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-06-10T01:10:55.787 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.787 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-10T01:10:55.788 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-10T01:10:55.788 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.788 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.788 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.788 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.788 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.788 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.789 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.789 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.789 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.789 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.789 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.789 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.789 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.789 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.789 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.790 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.790 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.790 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.790 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.790 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.790 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.790 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.790 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.791 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.791 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.791 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.791 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.791 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.791 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.791 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.791 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.792 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.792 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.792 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.792 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.792 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.792 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.792 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.793 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.793 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.793 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.793 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.793 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.793 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.793 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.794 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.794 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.794 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.794 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.794 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.794 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.794 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.794 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.795 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.795 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.795 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.795 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.795 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.795 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.795 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.796 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.796 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.796 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.796 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.796 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.796 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.796 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.796 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.797 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.797 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.797 INFO:teuthology.orchestra.run.smithi037.stdout: "min_score": { 2024-06-10T01:10:55.797 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_score", 2024-06-10T01:10:55.797 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-06-10T01:10:55.797 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.797 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.797 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-06-10T01:10:55.798 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.798 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.798 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.798 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-10T01:10:55.798 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.798 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.798 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.799 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-06-10T01:10:55.799 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-06-10T01:10:55.799 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.799 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.799 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.799 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "upmap", 2024-06-10T01:10:55.799 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.799 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.800 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.800 INFO:teuthology.orchestra.run.smithi037.stdout: "crush-compat", 2024-06-10T01:10:55.800 INFO:teuthology.orchestra.run.smithi037.stdout: "none", 2024-06-10T01:10:55.800 INFO:teuthology.orchestra.run.smithi037.stdout: "read", 2024-06-10T01:10:55.800 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap", 2024-06-10T01:10:55.800 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap-read" 2024-06-10T01:10:55.800 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.800 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Balancer mode", 2024-06-10T01:10:55.800 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.801 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.801 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.801 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_ids": { 2024-06-10T01:10:55.801 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_ids", 2024-06-10T01:10:55.801 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.801 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.802 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.802 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.802 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.802 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.802 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.802 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-10T01:10:55.802 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.802 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.803 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.803 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-06-10T01:10:55.803 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-06-10T01:10:55.803 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.803 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.803 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.803 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-06-10T01:10:55.804 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.804 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.804 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.804 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-10T01:10:55.804 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.804 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.804 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.804 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_deviation": { 2024-06-10T01:10:55.805 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_deviation", 2024-06-10T01:10:55.805 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.805 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.805 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.805 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-06-10T01:10:55.805 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-06-10T01:10:55.805 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.805 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.806 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-10T01:10:55.806 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-10T01:10:55.806 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.806 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.806 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.806 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_optimizations": { 2024-06-10T01:10:55.806 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_optimizations", 2024-06-10T01:10:55.806 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-06-10T01:10:55.807 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.807 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.807 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-06-10T01:10:55.807 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.807 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.807 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.807 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-10T01:10:55.807 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.808 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.808 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.808 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.808 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.808 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.808 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.808 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm", 2024-06-10T01:10:55.808 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.809 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.809 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.809 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_down_multiplier": { 2024-06-10T01:10:55.809 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_down_multiplier", 2024-06-10T01:10:55.809 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-06-10T01:10:55.809 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.809 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.809 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-06-10T01:10:55.810 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.810 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.810 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.810 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-10T01:10:55.810 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.810 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.810 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.810 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.811 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_refresh_rate": { 2024-06-10T01:10:55.811 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_refresh_rate", 2024-06-10T01:10:55.811 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.811 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.811 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.811 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "20", 2024-06-10T01:10:55.811 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.811 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.812 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.812 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-10T01:10:55.812 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.812 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.812 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.812 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.812 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_starting_port": { 2024-06-10T01:10:55.813 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_starting_port", 2024-06-10T01:10:55.813 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.813 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.813 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.813 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4721", 2024-06-10T01:10:55.813 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.813 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.813 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.814 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-10T01:10:55.814 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.814 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.814 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.814 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.814 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_ptrace": { 2024-06-10T01:10:55.814 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_ptrace", 2024-06-10T01:10:55.814 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.815 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.815 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.815 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.815 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.815 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.815 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.815 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-10T01:10:55.815 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-10T01:10:55.816 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.816 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.816 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.816 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_interval": { 2024-06-10T01:10:55.816 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_interval", 2024-06-10T01:10:55.816 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.816 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.816 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.817 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-06-10T01:10:55.817 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.817 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.817 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.817 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-10T01:10:55.817 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.817 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.817 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.817 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.818 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_memory_target_ratio": { 2024-06-10T01:10:55.818 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_memory_target_ratio", 2024-06-10T01:10:55.818 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-06-10T01:10:55.818 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.818 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.818 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.7", 2024-06-10T01:10:55.818 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.818 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.819 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.819 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-10T01:10:55.819 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.819 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.819 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.819 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm_log_destination": { 2024-06-10T01:10:55.819 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm_log_destination", 2024-06-10T01:10:55.820 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.820 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.820 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.820 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.820 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.820 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.820 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.820 INFO:teuthology.orchestra.run.smithi037.stdout: "file", 2024-06-10T01:10:55.821 INFO:teuthology.orchestra.run.smithi037.stdout: "file,syslog", 2024-06-10T01:10:55.821 INFO:teuthology.orchestra.run.smithi037.stdout: "syslog" 2024-06-10T01:10:55.821 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.821 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-10T01:10:55.821 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.821 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.821 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.821 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.822 INFO:teuthology.orchestra.run.smithi037.stdout: "cgroups_split": { 2024-06-10T01:10:55.822 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cgroups_split", 2024-06-10T01:10:55.822 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.822 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.822 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.822 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.822 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.822 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.823 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.823 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-10T01:10:55.823 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.823 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.823 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.823 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.823 INFO:teuthology.orchestra.run.smithi037.stdout: "config_checks_enabled": { 2024-06-10T01:10:55.823 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_checks_enabled", 2024-06-10T01:10:55.824 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.824 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.824 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.824 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.824 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.824 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.824 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.825 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-10T01:10:55.825 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.825 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.825 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.825 INFO:teuthology.orchestra.run.smithi037.stdout: "config_dashboard": { 2024-06-10T01:10:55.825 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_dashboard", 2024-06-10T01:10:55.825 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.825 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.826 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.826 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.826 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.826 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.826 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.826 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-10T01:10:55.826 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.826 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.827 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.827 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_alertmanager": { 2024-06-10T01:10:55.827 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_alertmanager", 2024-06-10T01:10:55.827 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.827 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.827 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.827 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-10T01:10:55.828 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.828 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.828 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.828 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-06-10T01:10:55.828 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.828 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.828 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.829 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_base": { 2024-06-10T01:10:55.829 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_base", 2024-06-10T01:10:55.829 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.829 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.829 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.829 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-10T01:10:55.829 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.829 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.830 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.830 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Container image name, without the tag", 2024-06-10T01:10:55.830 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.830 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.830 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.830 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_elasticsearch": { 2024-06-10T01:10:55.830 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_elasticsearch", 2024-06-10T01:10:55.831 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.831 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.831 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.831 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-10T01:10:55.831 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.831 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.831 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.831 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "elasticsearch container image", 2024-06-10T01:10:55.832 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.832 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.832 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.832 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_grafana": { 2024-06-10T01:10:55.832 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_grafana", 2024-06-10T01:10:55.832 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.832 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.832 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.833 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-10T01:10:55.833 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.833 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.833 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.833 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-06-10T01:10:55.833 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.833 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.833 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.833 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.834 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_haproxy": { 2024-06-10T01:10:55.834 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_haproxy", 2024-06-10T01:10:55.834 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.834 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.834 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.834 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-10T01:10:55.834 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.834 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.835 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.835 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "HAproxy container image", 2024-06-10T01:10:55.835 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.835 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.835 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.835 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_agent": { 2024-06-10T01:10:55.835 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_agent", 2024-06-10T01:10:55.835 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.836 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.836 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.836 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-10T01:10:55.836 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.836 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.836 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.836 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger agent container image", 2024-06-10T01:10:55.836 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.836 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.837 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.837 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_collector": { 2024-06-10T01:10:55.837 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_collector", 2024-06-10T01:10:55.837 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.837 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.837 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.837 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-10T01:10:55.838 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.838 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.838 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.838 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger collector container image", 2024-06-10T01:10:55.838 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.838 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.838 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.838 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.839 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_query": { 2024-06-10T01:10:55.839 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_query", 2024-06-10T01:10:55.839 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.839 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.839 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.839 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-10T01:10:55.839 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.839 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.840 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.840 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger query container image", 2024-06-10T01:10:55.840 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.840 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.840 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.840 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.840 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_keepalived": { 2024-06-10T01:10:55.840 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_keepalived", 2024-06-10T01:10:55.841 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.841 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.841 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.841 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-10T01:10:55.841 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.841 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.841 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.841 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Keepalived container image", 2024-06-10T01:10:55.841 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.842 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.842 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.842 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_loki": { 2024-06-10T01:10:55.842 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_loki", 2024-06-10T01:10:55.842 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.842 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.843 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.843 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-10T01:10:55.843 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.843 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.843 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.843 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Loki container image", 2024-06-10T01:10:55.843 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.843 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.844 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.844 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_node_exporter": { 2024-06-10T01:10:55.844 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_node_exporter", 2024-06-10T01:10:55.844 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.844 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.844 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.844 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-10T01:10:55.845 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.845 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.845 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.845 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-06-10T01:10:55.845 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.845 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.845 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.846 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_nvmeof": { 2024-06-10T01:10:55.846 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_nvmeof", 2024-06-10T01:10:55.846 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.846 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.846 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.846 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-10T01:10:55.846 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.847 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.847 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.847 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Nvme-of container image", 2024-06-10T01:10:55.847 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.847 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.847 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.847 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.847 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_prometheus": { 2024-06-10T01:10:55.847 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_prometheus", 2024-06-10T01:10:55.848 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.848 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.848 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.848 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-10T01:10:55.848 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.848 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.848 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.848 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-06-10T01:10:55.848 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.849 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.849 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.849 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.849 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_promtail": { 2024-06-10T01:10:55.849 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_promtail", 2024-06-10T01:10:55.849 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.849 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.849 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.849 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-10T01:10:55.850 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.850 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.850 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.850 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Promtail container image", 2024-06-10T01:10:55.850 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.850 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.850 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.851 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_samba": { 2024-06-10T01:10:55.851 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_samba", 2024-06-10T01:10:55.851 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.851 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.851 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.851 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-10T01:10:55.851 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.852 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.852 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.852 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Samba/SMB container image", 2024-06-10T01:10:55.852 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.852 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.852 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.852 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_snmp_gateway": { 2024-06-10T01:10:55.852 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_snmp_gateway", 2024-06-10T01:10:55.853 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.853 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.853 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.853 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-10T01:10:55.853 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.853 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.853 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.853 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SNMP Gateway container image", 2024-06-10T01:10:55.854 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.854 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.854 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.854 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.854 INFO:teuthology.orchestra.run.smithi037.stdout: "container_init": { 2024-06-10T01:10:55.854 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_init", 2024-06-10T01:10:55.854 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.854 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.855 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.855 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.855 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.855 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.855 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.855 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Run podman/docker with `--init`", 2024-06-10T01:10:55.855 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.855 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.856 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.856 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.856 INFO:teuthology.orchestra.run.smithi037.stdout: "daemon_cache_timeout": { 2024-06-10T01:10:55.856 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "daemon_cache_timeout", 2024-06-10T01:10:55.856 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.856 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.856 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.856 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-06-10T01:10:55.856 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.857 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.857 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.857 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-10T01:10:55.857 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.857 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.857 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.857 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.857 INFO:teuthology.orchestra.run.smithi037.stdout: "default_cephadm_command_timeout": { 2024-06-10T01:10:55.858 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_cephadm_command_timeout", 2024-06-10T01:10:55.858 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.858 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.858 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.858 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "900", 2024-06-10T01:10:55.858 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.858 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.858 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.859 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-10T01:10:55.859 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.859 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.859 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.859 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.859 INFO:teuthology.orchestra.run.smithi037.stdout: "default_registry": { 2024-06-10T01:10:55.859 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_registry", 2024-06-10T01:10:55.859 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.859 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.860 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.860 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io", 2024-06-10T01:10:55.860 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.860 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.860 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.860 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-10T01:10:55.860 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.861 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.861 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.861 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.861 INFO:teuthology.orchestra.run.smithi037.stdout: "device_cache_timeout": { 2024-06-10T01:10:55.861 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_cache_timeout", 2024-06-10T01:10:55.861 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.861 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.861 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.862 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1800", 2024-06-10T01:10:55.862 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.862 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.862 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.862 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache device inventory", 2024-06-10T01:10:55.862 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.862 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.862 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.863 INFO:teuthology.orchestra.run.smithi037.stdout: "device_enhanced_scan": { 2024-06-10T01:10:55.863 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_enhanced_scan", 2024-06-10T01:10:55.863 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.863 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.863 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.863 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.863 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.864 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.864 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.864 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-10T01:10:55.864 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.864 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.864 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.864 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.864 INFO:teuthology.orchestra.run.smithi037.stdout: "facts_cache_timeout": { 2024-06-10T01:10:55.864 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "facts_cache_timeout", 2024-06-10T01:10:55.865 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.865 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.865 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.865 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-06-10T01:10:55.865 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.865 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.865 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.865 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache host facts data", 2024-06-10T01:10:55.866 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.866 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.866 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.866 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.866 INFO:teuthology.orchestra.run.smithi037.stdout: "grafana_dashboards_path": { 2024-06-10T01:10:55.866 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "grafana_dashboards_path", 2024-06-10T01:10:55.866 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.866 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.867 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.867 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-10T01:10:55.867 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.867 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.867 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.867 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-10T01:10:55.867 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.867 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.867 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.868 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.868 INFO:teuthology.orchestra.run.smithi037.stdout: "host_check_interval": { 2024-06-10T01:10:55.868 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "host_check_interval", 2024-06-10T01:10:55.868 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.868 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.868 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.868 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-06-10T01:10:55.868 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.869 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.869 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.869 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to perform a host check", 2024-06-10T01:10:55.869 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.869 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.869 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.869 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.869 INFO:teuthology.orchestra.run.smithi037.stdout: "hw_monitoring": { 2024-06-10T01:10:55.870 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hw_monitoring", 2024-06-10T01:10:55.870 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.870 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.870 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.870 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.870 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.870 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.870 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.871 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-10T01:10:55.871 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.871 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.871 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.871 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.871 INFO:teuthology.orchestra.run.smithi037.stdout: "inventory_list_all": { 2024-06-10T01:10:55.871 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "inventory_list_all", 2024-06-10T01:10:55.871 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.872 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.872 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.872 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.872 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.872 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.872 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.872 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-10T01:10:55.872 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.872 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.873 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.873 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.873 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.873 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.873 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.873 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.873 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.873 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.874 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.874 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.874 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.874 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.874 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.874 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.874 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.874 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.875 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.875 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.875 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.875 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.875 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.875 INFO:teuthology.orchestra.run.smithi037.stdout: "log_refresh_metadata": { 2024-06-10T01:10:55.875 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_refresh_metadata", 2024-06-10T01:10:55.876 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.876 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.876 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.876 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.876 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.876 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.876 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.876 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-10T01:10:55.877 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.877 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.877 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.877 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.877 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.877 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.877 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.877 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.878 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.878 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.878 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.878 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.878 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.878 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-10T01:10:55.878 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.879 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.879 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.879 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.879 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.879 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.879 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.879 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.879 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.880 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.880 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.880 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.880 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.880 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.880 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.880 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.880 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.881 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.881 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.881 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.881 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.881 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.881 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.881 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.881 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.882 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.882 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.882 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.882 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.882 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.882 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.882 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.882 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.883 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.883 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.883 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.883 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.883 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.883 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.883 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-10T01:10:55.884 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-10T01:10:55.884 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.884 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.884 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.884 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.884 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.884 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.884 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.885 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-10T01:10:55.885 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.885 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.885 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.885 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.885 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-10T01:10:55.885 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-10T01:10:55.885 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.885 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.886 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.886 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "*", 2024-06-10T01:10:55.886 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.886 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.886 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.886 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-10T01:10:55.886 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.887 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.887 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.887 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.887 INFO:teuthology.orchestra.run.smithi037.stdout: "max_count_per_host": { 2024-06-10T01:10:55.887 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_count_per_host", 2024-06-10T01:10:55.887 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.887 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.887 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.888 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-06-10T01:10:55.888 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.888 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.888 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.888 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of daemons per service per host", 2024-06-10T01:10:55.888 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.888 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.888 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.888 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.889 INFO:teuthology.orchestra.run.smithi037.stdout: "max_osd_draining_count": { 2024-06-10T01:10:55.889 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_osd_draining_count", 2024-06-10T01:10:55.889 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.889 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.889 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.889 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-06-10T01:10:55.889 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.890 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.890 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.890 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-10T01:10:55.890 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.890 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.890 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.890 INFO:teuthology.orchestra.run.smithi037.stdout: "migration_current": { 2024-06-10T01:10:55.891 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "migration_current", 2024-06-10T01:10:55.891 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.891 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.891 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.891 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.891 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.891 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.891 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.892 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "internal - do not modify", 2024-06-10T01:10:55.892 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.892 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.892 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.892 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.892 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-06-10T01:10:55.892 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-06-10T01:10:55.892 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.893 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.893 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.893 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "root", 2024-06-10T01:10:55.893 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.893 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.893 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.893 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm-package", 2024-06-10T01:10:55.893 INFO:teuthology.orchestra.run.smithi037.stdout: "root" 2024-06-10T01:10:55.894 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.894 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "mode for remote execution of cephadm", 2024-06-10T01:10:55.894 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.894 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.894 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.894 INFO:teuthology.orchestra.run.smithi037.stdout: "oob_default_addr": { 2024-06-10T01:10:55.894 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "oob_default_addr", 2024-06-10T01:10:55.895 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.895 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.895 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.896 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "169.254.1.1", 2024-06-10T01:10:55.896 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.896 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.897 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.897 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-10T01:10:55.897 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.897 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.897 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.897 INFO:teuthology.orchestra.run.smithi037.stdout: "prometheus_alerts_path": { 2024-06-10T01:10:55.897 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus_alerts_path", 2024-06-10T01:10:55.898 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.898 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.898 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.898 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-10T01:10:55.898 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.898 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.898 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.898 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-10T01:10:55.899 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.899 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.899 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.899 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.899 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_insecure": { 2024-06-10T01:10:55.899 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_insecure", 2024-06-10T01:10:55.899 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.899 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.899 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.900 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.900 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.900 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.900 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.900 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-10T01:10:55.900 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.900 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.901 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.901 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_password": { 2024-06-10T01:10:55.901 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_password", 2024-06-10T01:10:55.901 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.901 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.901 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.901 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.901 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.902 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.902 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.902 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-10T01:10:55.902 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.902 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.902 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.902 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_url": { 2024-06-10T01:10:55.903 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_url", 2024-06-10T01:10:55.903 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.903 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.903 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.903 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.903 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.903 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.903 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.904 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-10T01:10:55.904 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.904 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.904 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.904 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_username": { 2024-06-10T01:10:55.904 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_username", 2024-06-10T01:10:55.904 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.905 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.905 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.905 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.905 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.905 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.905 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.905 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-10T01:10:55.905 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.905 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.906 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.906 INFO:teuthology.orchestra.run.smithi037.stdout: "secure_monitoring_stack": { 2024-06-10T01:10:55.906 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secure_monitoring_stack", 2024-06-10T01:10:55.906 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.906 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.906 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.906 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.907 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.907 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.907 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.907 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-10T01:10:55.907 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.907 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.907 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.907 INFO:teuthology.orchestra.run.smithi037.stdout: "service_discovery_port": { 2024-06-10T01:10:55.908 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "service_discovery_port", 2024-06-10T01:10:55.908 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.908 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.908 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.908 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8765", 2024-06-10T01:10:55.908 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.908 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.908 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.909 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "cephadm service discovery port", 2024-06-10T01:10:55.909 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.909 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.909 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.909 INFO:teuthology.orchestra.run.smithi037.stdout: "ssh_config_file": { 2024-06-10T01:10:55.909 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssh_config_file", 2024-06-10T01:10:55.909 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.910 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.910 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.910 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.910 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.910 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.910 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.910 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-10T01:10:55.910 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.911 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.911 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.911 INFO:teuthology.orchestra.run.smithi037.stdout: "use_agent": { 2024-06-10T01:10:55.911 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_agent", 2024-06-10T01:10:55.911 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.911 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.911 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.912 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.912 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.912 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.912 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.912 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-10T01:10:55.912 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.912 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.912 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.913 INFO:teuthology.orchestra.run.smithi037.stdout: "use_repo_digest": { 2024-06-10T01:10:55.913 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_repo_digest", 2024-06-10T01:10:55.913 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.913 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.913 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.913 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.913 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.913 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.914 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.914 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-10T01:10:55.914 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.914 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.914 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.914 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_failed_host_check": { 2024-06-10T01:10:55.914 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_failed_host_check", 2024-06-10T01:10:55.915 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.915 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.915 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.915 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.915 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.915 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.915 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.915 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if the host check fails", 2024-06-10T01:10:55.915 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.916 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.916 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.916 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.916 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_daemons": { 2024-06-10T01:10:55.916 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_daemons", 2024-06-10T01:10:55.916 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.916 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.916 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.917 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.917 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.917 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.917 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.917 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-10T01:10:55.917 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.917 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.917 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.918 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_hosts": { 2024-06-10T01:10:55.918 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_hosts", 2024-06-10T01:10:55.918 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.918 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.918 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.918 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.918 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.918 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.919 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.919 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-10T01:10:55.919 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.919 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.919 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.919 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.919 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.919 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.919 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.920 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crash", 2024-06-10T01:10:55.920 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.920 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.920 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.920 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:55.920 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:55.920 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.920 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.921 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.921 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.921 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.921 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.921 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.921 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.921 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.921 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.922 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.922 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.922 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.922 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.922 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.922 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.922 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.922 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.923 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:55.923 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:55.923 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.923 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.923 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.923 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.923 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.924 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.924 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.924 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.924 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.924 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.924 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.924 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.924 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:55.925 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:55.925 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.925 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.925 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.925 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:55.925 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.925 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.925 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:55.925 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:55.926 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:55.926 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:55.926 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:55.926 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:55.926 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:55.926 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:55.926 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.927 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.927 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.927 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.927 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:55.927 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:55.927 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.927 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.927 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.928 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.928 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.928 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.928 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.928 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.928 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.928 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.928 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.929 INFO:teuthology.orchestra.run.smithi037.stdout: "retain_interval": { 2024-06-10T01:10:55.929 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retain_interval", 2024-06-10T01:10:55.929 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.929 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.929 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.929 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "31536000", 2024-06-10T01:10:55.929 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.930 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.930 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.930 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-10T01:10:55.930 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.930 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.930 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.930 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_recent_interval": { 2024-06-10T01:10:55.931 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_recent_interval", 2024-06-10T01:10:55.931 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:55.931 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.931 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:55.931 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1209600", 2024-06-10T01:10:55.931 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.931 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.931 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.931 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-10T01:10:55.932 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.932 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.932 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.932 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.932 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:55.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.932 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:55.932 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dashboard", 2024-06-10T01:10:55.933 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:55.933 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:55.933 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:55.933 INFO:teuthology.orchestra.run.smithi037.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-10T01:10:55.933 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-10T01:10:55.933 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.933 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.933 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.933 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-06-10T01:10:55.933 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.934 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.934 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.934 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.934 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.934 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.934 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.934 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.934 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_HOST": { 2024-06-10T01:10:55.935 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-10T01:10:55.935 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.935 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.935 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.935 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.935 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.935 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.935 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.935 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.936 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.936 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.936 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.936 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.936 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-10T01:10:55.936 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-10T01:10:55.936 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.936 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.936 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.937 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.937 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.937 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.937 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.937 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.937 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.937 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.937 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.938 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_ENABLED": { 2024-06-10T01:10:55.938 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_ENABLED", 2024-06-10T01:10:55.938 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.938 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.938 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.938 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.938 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.938 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.939 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.939 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.939 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.939 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.939 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.939 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-10T01:10:55.939 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-10T01:10:55.939 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.939 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.940 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.940 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.940 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.940 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.940 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.940 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.940 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.940 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.940 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.941 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.941 INFO:teuthology.orchestra.run.smithi037.stdout: "ENABLE_BROWSABLE_API": { 2024-06-10T01:10:55.941 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-10T01:10:55.941 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.941 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.941 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.941 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.941 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.942 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.942 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.942 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.942 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.942 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.942 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.942 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.942 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-10T01:10:55.943 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-10T01:10:55.943 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.943 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.943 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.943 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.943 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.943 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.943 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.944 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.944 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.944 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.944 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.944 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-10T01:10:55.944 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-10T01:10:55.944 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.944 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.945 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.945 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.945 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.945 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.945 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.945 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.945 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.945 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.946 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.946 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.946 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-10T01:10:55.946 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-10T01:10:55.946 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.946 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.946 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.946 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.947 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.947 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.947 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.947 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.947 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.947 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.947 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.948 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-10T01:10:55.948 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-10T01:10:55.948 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.948 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.948 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.948 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.948 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.948 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.949 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.949 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.949 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.949 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.949 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.949 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.949 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-10T01:10:55.949 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-10T01:10:55.950 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.950 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.950 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.950 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.950 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.950 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.950 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.950 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.951 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.951 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.951 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.951 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.951 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-10T01:10:55.951 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-10T01:10:55.951 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.951 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.952 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.952 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.952 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.952 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.952 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.952 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.952 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.952 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.953 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.953 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.953 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-10T01:10:55.953 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-10T01:10:55.953 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.953 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.953 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.953 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.954 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.954 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.954 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.954 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.954 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.954 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.954 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.954 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.955 INFO:teuthology.orchestra.run.smithi037.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-10T01:10:55.955 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-10T01:10:55.955 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.955 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.955 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.955 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.955 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.955 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.955 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.956 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.956 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.956 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.956 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.956 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.956 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_PASSWORD": { 2024-06-10T01:10:55.956 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-10T01:10:55.956 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.957 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.957 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.957 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-06-10T01:10:55.957 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.957 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.957 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.957 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.957 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.958 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.958 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.958 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-10T01:10:55.958 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-10T01:10:55.958 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.958 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.958 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.959 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.959 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.959 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.959 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.959 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.959 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.959 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.959 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.960 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_URL": { 2024-06-10T01:10:55.960 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_URL", 2024-06-10T01:10:55.960 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.960 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.960 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.960 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.960 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.960 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.961 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.961 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.961 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.961 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.961 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.961 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_USERNAME": { 2024-06-10T01:10:55.961 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-10T01:10:55.962 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.962 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.962 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.962 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-06-10T01:10:55.962 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.962 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.962 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.962 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.963 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.963 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.963 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.963 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-10T01:10:55.963 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-10T01:10:55.963 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.963 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.964 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.964 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.964 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.964 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.964 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.964 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.964 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.964 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.965 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.965 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-10T01:10:55.965 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-10T01:10:55.965 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.965 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.965 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.965 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.965 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.966 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.966 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.966 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.966 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.966 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.966 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.966 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.967 INFO:teuthology.orchestra.run.smithi037.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-10T01:10:55.967 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-10T01:10:55.967 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.967 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.967 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.967 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.967 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.967 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.968 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.968 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.968 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.968 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.968 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.969 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.969 INFO:teuthology.orchestra.run.smithi037.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-10T01:10:55.969 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-10T01:10:55.969 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.969 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.969 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.969 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.970 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.970 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.970 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.970 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.970 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.970 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.970 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.970 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.971 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_HOST": { 2024-06-10T01:10:55.971 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-10T01:10:55.971 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.971 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.971 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.971 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.971 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.971 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.972 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.972 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.972 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.972 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.972 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.972 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.972 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-10T01:10:55.972 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-10T01:10:55.973 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.973 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.973 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.973 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.973 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.973 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.973 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.973 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.974 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.974 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.974 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.974 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.974 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-10T01:10:55.974 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-10T01:10:55.974 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.974 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.975 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.975 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.975 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.975 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.975 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.975 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.975 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.975 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.976 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.976 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-10T01:10:55.976 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-10T01:10:55.976 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.976 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.976 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.976 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.977 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.977 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.977 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.977 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.977 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.977 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.977 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.977 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.978 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-10T01:10:55.978 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-10T01:10:55.978 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.978 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.978 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.978 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.978 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.978 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.979 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.979 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.979 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.979 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.979 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.979 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.979 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-10T01:10:55.979 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-10T01:10:55.979 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.980 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.980 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.980 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.980 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.980 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.980 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.980 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.980 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.981 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.981 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.981 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.981 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-10T01:10:55.981 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-10T01:10:55.981 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.981 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.982 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.982 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.982 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.982 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.982 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.982 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.982 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.983 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.983 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.983 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-10T01:10:55.983 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-10T01:10:55.983 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.983 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.983 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.983 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.984 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.984 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.984 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.984 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.984 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.984 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.984 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.984 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-10T01:10:55.984 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-10T01:10:55.985 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.985 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.985 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.985 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:55.985 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.985 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.985 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.985 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.986 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.986 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.986 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.986 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.986 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_ENABLED": { 2024-06-10T01:10:55.986 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-10T01:10:55.986 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:55.986 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.986 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.987 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:55.987 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.987 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.987 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.987 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.987 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.987 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.987 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.988 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-10T01:10:55.988 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-10T01:10:55.988 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.988 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.988 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.988 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-10T01:10:55.988 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.989 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.989 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.989 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.989 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.989 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.989 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.989 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.990 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-10T01:10:55.990 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-10T01:10:55.990 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.990 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.990 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.991 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-06-10T01:10:55.991 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.991 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.991 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.991 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.991 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.991 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.991 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.992 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.992 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-10T01:10:55.992 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-10T01:10:55.992 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.992 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.992 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.993 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8", 2024-06-10T01:10:55.993 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.993 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.993 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.993 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.993 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.993 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.993 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.993 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.994 INFO:teuthology.orchestra.run.smithi037.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-10T01:10:55.994 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-10T01:10:55.994 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:55.994 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.994 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.995 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "45", 2024-06-10T01:10:55.995 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.995 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.995 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.995 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.995 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.995 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.996 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.996 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.996 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ACCESS_KEY": { 2024-06-10T01:10:55.996 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-10T01:10:55.996 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.996 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.997 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.997 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:55.997 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.997 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.997 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.997 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:55.997 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:55.998 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:55.998 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:55.998 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:55.998 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-10T01:10:55.998 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-10T01:10:55.998 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:55.998 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:55.999 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:55.999 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-06-10T01:10:55.999 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:55.999 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:55.999 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:55.999 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.000 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.000 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.000 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.000 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SECRET_KEY": { 2024-06-10T01:10:56.000 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-10T01:10:56.001 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.001 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.001 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.001 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.001 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.001 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.002 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.002 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.002 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.002 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.002 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.002 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.002 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SSL_VERIFY": { 2024-06-10T01:10:56.002 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-10T01:10:56.002 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.003 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.003 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.003 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.003 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.003 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.003 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.003 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.003 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.004 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.004 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.004 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.004 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-10T01:10:56.004 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-10T01:10:56.004 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.004 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.004 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.004 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-06-10T01:10:56.005 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.005 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.005 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.005 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.005 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.005 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.005 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.005 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.006 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-10T01:10:56.006 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-10T01:10:56.006 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.006 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.006 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.006 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-06-10T01:10:56.006 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.006 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.007 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.007 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.007 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.007 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.007 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.007 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.007 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-10T01:10:56.007 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-10T01:10:56.007 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.008 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.008 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.008 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-06-10T01:10:56.008 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.008 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.008 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.008 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.008 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.008 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.009 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.009 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.009 INFO:teuthology.orchestra.run.smithi037.stdout: "cross_origin_url": { 2024-06-10T01:10:56.009 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cross_origin_url", 2024-06-10T01:10:56.009 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.009 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.009 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.009 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.009 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.010 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.010 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.010 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.010 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.010 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.010 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.010 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.010 INFO:teuthology.orchestra.run.smithi037.stdout: "crt_file": { 2024-06-10T01:10:56.011 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crt_file", 2024-06-10T01:10:56.011 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.011 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.011 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.011 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.011 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.011 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.011 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.011 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.012 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.012 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.012 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.012 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.012 INFO:teuthology.orchestra.run.smithi037.stdout: "debug": { 2024-06-10T01:10:56.012 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "debug", 2024-06-10T01:10:56.012 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.012 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.012 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.013 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.013 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.013 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.013 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.013 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable/disable debug options", 2024-06-10T01:10:56.013 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.013 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.013 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.014 INFO:teuthology.orchestra.run.smithi037.stdout: "jwt_token_ttl": { 2024-06-10T01:10:56.014 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "jwt_token_ttl", 2024-06-10T01:10:56.014 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.014 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.014 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.014 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "28800", 2024-06-10T01:10:56.015 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.015 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.015 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.015 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.015 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.015 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.015 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.015 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.016 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-06-10T01:10:56.016 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-06-10T01:10:56.016 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.016 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.016 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.016 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.016 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.016 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.016 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.017 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.017 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.017 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.017 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.017 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.017 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.017 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.017 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.018 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.018 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.018 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.018 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.018 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.018 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.018 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.018 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.018 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.019 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.019 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.019 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.019 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.019 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.019 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.019 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.019 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.019 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.020 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.020 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.020 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.020 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.020 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.020 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.020 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.020 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.020 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.021 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.021 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.021 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.021 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.021 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.021 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.021 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.021 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.021 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.022 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.022 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.022 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.022 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.022 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.022 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.022 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.023 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.023 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.023 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.023 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.023 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.023 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.023 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.023 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.023 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.024 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.024 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.024 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.024 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.024 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.024 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.024 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.025 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.025 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.025 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.025 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.025 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.025 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.025 INFO:teuthology.orchestra.run.smithi037.stdout: "motd": { 2024-06-10T01:10:56.026 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "motd", 2024-06-10T01:10:56.026 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.026 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.026 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.026 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.026 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.026 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.026 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.027 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "The message of the day", 2024-06-10T01:10:56.027 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.027 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.027 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.027 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect_resolve_ip_addr": { 2024-06-10T01:10:56.027 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "redirect_resolve_ip_addr", 2024-06-10T01:10:56.027 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.027 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.028 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.028 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.028 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.028 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.028 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.028 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.028 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.028 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.029 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.029 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-06-10T01:10:56.029 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-06-10T01:10:56.029 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.029 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.029 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.029 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-06-10T01:10:56.030 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.030 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.030 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.030 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.030 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.030 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.030 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.030 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.031 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-06-10T01:10:56.031 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-06-10T01:10:56.031 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.031 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.031 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.031 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8080", 2024-06-10T01:10:56.031 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.031 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.032 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.032 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.032 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.032 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.032 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.032 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.032 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-06-10T01:10:56.032 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-06-10T01:10:56.033 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.033 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.033 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.033 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.033 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.033 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.033 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.033 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.033 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.034 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.034 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.034 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl_server_port": { 2024-06-10T01:10:56.034 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl_server_port", 2024-06-10T01:10:56.034 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.034 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.035 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.035 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8443", 2024-06-10T01:10:56.035 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.035 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.035 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.035 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.035 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.035 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.035 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.036 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-06-10T01:10:56.036 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-06-10T01:10:56.036 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.036 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.036 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.036 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "redirect", 2024-06-10T01:10:56.036 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.037 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.037 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.037 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.037 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect" 2024-06-10T01:10:56.037 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.037 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.037 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.037 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.038 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.038 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-06-10T01:10:56.038 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-06-10T01:10:56.038 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.038 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.038 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.038 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-06-10T01:10:56.039 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-06-10T01:10:56.039 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-06-10T01:10:56.039 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.039 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.039 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.039 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.039 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.039 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.039 INFO:teuthology.orchestra.run.smithi037.stdout: "url_prefix": { 2024-06-10T01:10:56.040 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url_prefix", 2024-06-10T01:10:56.040 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.040 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.040 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.040 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.040 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.040 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.040 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.041 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.041 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.041 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.041 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.041 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.041 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.041 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.041 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.042 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-06-10T01:10:56.042 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.042 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.042 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.042 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_monitoring": { 2024-06-10T01:10:56.042 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_monitoring", 2024-06-10T01:10:56.042 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.042 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.043 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.043 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.043 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.043 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.043 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.043 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "monitor device health metrics", 2024-06-10T01:10:56.043 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.043 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.043 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.044 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.044 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.044 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.044 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.044 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.044 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.044 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.044 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.045 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.045 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.045 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.045 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.045 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.045 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.045 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.045 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.045 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.046 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.046 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.046 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.046 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.046 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.046 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.046 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.046 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.047 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.047 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.047 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.047 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.047 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.047 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.047 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.047 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.047 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.048 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.048 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.048 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.048 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.048 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.048 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.048 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.049 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.049 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.049 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.049 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.049 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.049 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.049 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.049 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.049 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.050 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.050 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.050 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.050 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.050 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.050 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.050 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.050 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.051 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.051 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.051 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.051 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.051 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.051 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.051 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.052 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.052 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.052 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.052 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.052 INFO:teuthology.orchestra.run.smithi037.stdout: "mark_out_threshold": { 2024-06-10T01:10:56.052 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mark_out_threshold", 2024-06-10T01:10:56.052 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:56.052 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.052 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.053 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2419200", 2024-06-10T01:10:56.053 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.053 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.053 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.053 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-10T01:10:56.053 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.053 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.053 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.054 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_name": { 2024-06-10T01:10:56.054 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_name", 2024-06-10T01:10:56.054 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.054 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.054 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.054 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "device_health_metrics", 2024-06-10T01:10:56.054 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.054 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.054 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.055 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-10T01:10:56.055 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.055 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.055 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.055 INFO:teuthology.orchestra.run.smithi037.stdout: "retention_period": { 2024-06-10T01:10:56.055 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retention_period", 2024-06-10T01:10:56.055 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:56.055 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.056 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.056 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15552000", 2024-06-10T01:10:56.056 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.056 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.056 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.056 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain device health metrics", 2024-06-10T01:10:56.056 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.056 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.057 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.057 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.057 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_frequency": { 2024-06-10T01:10:56.057 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_frequency", 2024-06-10T01:10:56.057 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:56.057 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.057 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.057 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-06-10T01:10:56.058 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.058 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.058 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.058 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-10T01:10:56.058 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.058 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.058 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.059 INFO:teuthology.orchestra.run.smithi037.stdout: "self_heal": { 2024-06-10T01:10:56.059 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "self_heal", 2024-06-10T01:10:56.059 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.059 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.059 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.059 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.059 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.059 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.060 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.060 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-10T01:10:56.060 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.060 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.060 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.060 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-06-10T01:10:56.060 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-06-10T01:10:56.061 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:56.061 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.061 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.061 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-06-10T01:10:56.061 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.061 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.061 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.061 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and check device health", 2024-06-10T01:10:56.062 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.062 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.062 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.062 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_threshold": { 2024-06-10T01:10:56.062 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_threshold", 2024-06-10T01:10:56.062 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:56.063 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.063 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.063 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7257600", 2024-06-10T01:10:56.063 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.063 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.063 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.063 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-10T01:10:56.063 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.064 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.064 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.064 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.064 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.064 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.064 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "diskprediction_local", 2024-06-10T01:10:56.064 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.065 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.065 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.065 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.065 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.065 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.065 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.065 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.065 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.066 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.066 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.066 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.066 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.066 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.066 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.066 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.066 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.067 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.067 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.067 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.067 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.067 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.067 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.068 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.068 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.068 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.068 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.068 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.068 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.068 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.068 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.069 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.069 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.069 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.069 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.069 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.069 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.069 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.069 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.069 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.070 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.070 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.070 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.070 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.070 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.070 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.070 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.070 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.071 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.071 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.071 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.071 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.071 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.071 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.071 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.071 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.072 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.072 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.072 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.072 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.072 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.072 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.072 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.072 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.073 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.073 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.073 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.073 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.073 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.073 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.073 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.073 INFO:teuthology.orchestra.run.smithi037.stdout: "predict_interval": { 2024-06-10T01:10:56.074 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predict_interval", 2024-06-10T01:10:56.074 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.074 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.074 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.074 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-06-10T01:10:56.074 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.074 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.074 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.074 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.075 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.075 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.075 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.075 INFO:teuthology.orchestra.run.smithi037.stdout: "predictor_model": { 2024-06-10T01:10:56.075 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predictor_model", 2024-06-10T01:10:56.075 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.075 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.076 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.076 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "prophetstor", 2024-06-10T01:10:56.076 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.076 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.076 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.076 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.076 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.076 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.076 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.077 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-06-10T01:10:56.077 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-06-10T01:10:56.077 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.077 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.077 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.077 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-06-10T01:10:56.077 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.078 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.078 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.078 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.078 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.078 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.078 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.078 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.078 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.079 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.079 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "influx", 2024-06-10T01:10:56.079 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": false, 2024-06-10T01:10:56.079 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "influxdb python module not found", 2024-06-10T01:10:56.079 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.079 INFO:teuthology.orchestra.run.smithi037.stdout: "batch_size": { 2024-06-10T01:10:56.079 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "batch_size", 2024-06-10T01:10:56.079 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.080 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.080 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.080 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5000", 2024-06-10T01:10:56.080 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.080 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.080 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.080 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-10T01:10:56.080 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.080 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.081 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.081 INFO:teuthology.orchestra.run.smithi037.stdout: "database": { 2024-06-10T01:10:56.081 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "database", 2024-06-10T01:10:56.081 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.081 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.081 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.081 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "ceph", 2024-06-10T01:10:56.082 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.082 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.082 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.082 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-10T01:10:56.082 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.082 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.082 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.082 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.083 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": { 2024-06-10T01:10:56.083 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hostname", 2024-06-10T01:10:56.083 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.083 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.083 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.083 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.083 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.083 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.083 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.084 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server hostname", 2024-06-10T01:10:56.084 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.084 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.084 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.084 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-06-10T01:10:56.084 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-06-10T01:10:56.084 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:56.085 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.085 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.085 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "30", 2024-06-10T01:10:56.085 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "5", 2024-06-10T01:10:56.085 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.085 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.085 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-10T01:10:56.085 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.085 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.085 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.086 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.086 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.086 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.086 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.086 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.086 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.087 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.087 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.087 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.087 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.087 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.087 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.087 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.087 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.088 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.088 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.088 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.088 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.088 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.088 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.088 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.088 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.089 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.089 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.089 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.089 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.089 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.089 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.089 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.089 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.089 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.090 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.090 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.090 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.090 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.090 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.090 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.090 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.091 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.091 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.091 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.091 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.091 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.091 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.091 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.091 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.092 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.092 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.092 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.092 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.092 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.092 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.092 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.092 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.093 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.093 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.093 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.093 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.093 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.093 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.093 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.093 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.094 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.094 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.094 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.094 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.094 INFO:teuthology.orchestra.run.smithi037.stdout: "password": { 2024-06-10T01:10:56.094 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "password", 2024-06-10T01:10:56.094 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.095 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.095 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.095 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.095 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.095 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.095 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.095 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "password of InfluxDB server user", 2024-06-10T01:10:56.095 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.095 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.096 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.096 INFO:teuthology.orchestra.run.smithi037.stdout: "port": { 2024-06-10T01:10:56.096 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "port", 2024-06-10T01:10:56.096 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.096 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.096 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.096 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8086", 2024-06-10T01:10:56.097 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.097 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.097 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.097 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server port", 2024-06-10T01:10:56.097 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.097 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.097 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.098 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-06-10T01:10:56.098 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-06-10T01:10:56.098 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.098 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.098 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.098 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "false", 2024-06-10T01:10:56.098 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.098 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.099 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.099 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-10T01:10:56.099 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.099 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.099 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.099 INFO:teuthology.orchestra.run.smithi037.stdout: "threads": { 2024-06-10T01:10:56.099 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threads", 2024-06-10T01:10:56.100 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.100 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.100 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.100 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-06-10T01:10:56.100 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-06-10T01:10:56.100 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "32", 2024-06-10T01:10:56.100 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.100 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-10T01:10:56.101 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.101 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.101 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.101 INFO:teuthology.orchestra.run.smithi037.stdout: "username": { 2024-06-10T01:10:56.101 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "username", 2024-06-10T01:10:56.101 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.101 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.102 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.102 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.102 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.102 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.102 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.102 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "username of InfluxDB server user", 2024-06-10T01:10:56.102 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.102 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.103 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.103 INFO:teuthology.orchestra.run.smithi037.stdout: "verify_ssl": { 2024-06-10T01:10:56.103 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "verify_ssl", 2024-06-10T01:10:56.103 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.103 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.103 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.103 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "true", 2024-06-10T01:10:56.103 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.104 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.104 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.104 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-10T01:10:56.104 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.104 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.104 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.104 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.105 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.105 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.105 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "insights", 2024-06-10T01:10:56.105 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.105 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.105 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.105 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.105 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.106 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.106 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.106 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.106 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.106 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.106 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.106 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.106 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.107 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.107 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.107 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.107 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.107 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.107 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.107 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.107 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.107 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.108 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.108 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.108 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.108 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.108 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.108 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.108 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.108 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.109 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.109 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.109 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.109 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.109 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.109 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.109 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.110 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.110 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.110 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.110 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.110 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.110 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.110 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.110 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.111 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.111 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.111 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.111 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.111 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.111 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.111 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.111 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.112 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.112 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.112 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.112 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.112 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.112 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.112 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.113 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.113 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.113 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.113 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.113 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.113 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.113 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.113 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.113 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.114 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.114 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.114 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.114 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "iostat", 2024-06-10T01:10:56.114 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.114 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.114 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.115 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.115 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.115 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.115 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.115 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.115 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.115 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.115 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.116 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.116 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.116 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.116 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.116 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.116 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.116 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.117 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.117 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.117 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.117 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.117 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.117 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.117 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.117 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.118 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.118 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.118 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.118 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.118 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.118 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.118 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.118 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.119 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.119 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.119 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.119 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.119 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.119 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.119 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.120 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.120 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.120 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.120 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.120 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.120 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.120 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.120 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.121 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.121 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.121 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.121 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.121 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.121 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.121 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.121 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.122 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.122 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.122 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.122 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.122 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.122 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.122 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.123 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.123 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.123 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.123 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.123 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.123 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.123 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.123 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.123 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.124 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.124 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.124 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "k8sevents", 2024-06-10T01:10:56.124 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.124 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.124 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.124 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_event_retention_days": { 2024-06-10T01:10:56.124 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ceph_event_retention_days", 2024-06-10T01:10:56.124 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.125 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.125 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.125 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7", 2024-06-10T01:10:56.125 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.125 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.125 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.125 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-10T01:10:56.125 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.126 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.126 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.126 INFO:teuthology.orchestra.run.smithi037.stdout: "config_check_secs": { 2024-06-10T01:10:56.126 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_check_secs", 2024-06-10T01:10:56.126 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.126 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.126 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.127 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-06-10T01:10:56.127 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "10", 2024-06-10T01:10:56.127 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.127 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.127 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-10T01:10:56.127 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.127 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.127 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.128 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.128 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.128 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.128 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.128 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.128 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.128 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.129 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.129 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.129 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.129 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.129 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.129 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.129 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.129 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.130 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.130 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.130 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.130 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.130 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.130 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.130 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.131 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.131 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.131 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.131 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.131 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.131 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.131 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.131 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.132 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.132 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.132 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.132 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.132 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.132 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.132 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.133 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.133 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.133 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.133 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.133 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.133 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.133 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.133 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.134 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.134 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.134 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.134 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.134 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.134 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.134 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.134 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.135 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.135 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.135 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.135 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.135 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.135 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.135 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.136 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.136 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.136 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.136 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.136 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.136 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.136 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.136 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.137 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "localpool", 2024-06-10T01:10:56.137 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.137 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.137 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.137 INFO:teuthology.orchestra.run.smithi037.stdout: "failure_domain": { 2024-06-10T01:10:56.137 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "failure_domain", 2024-06-10T01:10:56.138 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.138 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.138 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.138 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "host", 2024-06-10T01:10:56.138 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.138 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.138 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.138 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "failure domain for any created local pool", 2024-06-10T01:10:56.138 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-10T01:10:56.139 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.139 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.139 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.139 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.139 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.139 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.139 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.140 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.140 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.140 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.140 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.140 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.140 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.140 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.140 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.141 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.141 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.141 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.141 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.141 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.141 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.141 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.142 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.142 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.142 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.142 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.142 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.142 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.142 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.142 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.143 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.143 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.143 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.143 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.143 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.143 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.143 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.143 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.144 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.144 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.144 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.144 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.144 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.144 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.144 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.144 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.145 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.145 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.145 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.145 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.145 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.145 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.145 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.145 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.145 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.146 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.146 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.146 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.146 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.146 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.146 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.146 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.147 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.147 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.147 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.147 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.147 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.147 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.147 INFO:teuthology.orchestra.run.smithi037.stdout: "min_size": { 2024-06-10T01:10:56.148 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_size", 2024-06-10T01:10:56.148 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.148 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.148 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.148 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.148 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.148 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.148 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.149 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default min_size for any created local pool", 2024-06-10T01:10:56.149 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-10T01:10:56.149 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.149 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.149 INFO:teuthology.orchestra.run.smithi037.stdout: "num_rep": { 2024-06-10T01:10:56.149 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "num_rep", 2024-06-10T01:10:56.149 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.149 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.149 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.150 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3", 2024-06-10T01:10:56.150 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.150 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.150 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.150 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default replica count for any created local pool", 2024-06-10T01:10:56.150 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.150 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.150 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.151 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_num": { 2024-06-10T01:10:56.151 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_num", 2024-06-10T01:10:56.151 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.151 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.151 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.152 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "128", 2024-06-10T01:10:56.152 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.152 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.152 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.152 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default pg_num for any created local pool", 2024-06-10T01:10:56.152 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.152 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.152 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.153 INFO:teuthology.orchestra.run.smithi037.stdout: "prefix": { 2024-06-10T01:10:56.153 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prefix", 2024-06-10T01:10:56.153 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.153 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.153 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.153 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.153 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.154 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.154 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.154 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name prefix for any created local pool", 2024-06-10T01:10:56.154 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.154 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.154 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.154 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.154 INFO:teuthology.orchestra.run.smithi037.stdout: "subtree": { 2024-06-10T01:10:56.154 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "subtree", 2024-06-10T01:10:56.155 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.155 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.155 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.155 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "rack", 2024-06-10T01:10:56.155 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.155 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.155 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.155 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-10T01:10:56.156 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-10T01:10:56.156 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.156 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.156 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.156 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.156 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.156 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mds_autoscaler", 2024-06-10T01:10:56.156 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.157 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.157 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.157 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.157 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.157 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.157 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.157 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.157 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.157 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.158 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.158 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.158 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.158 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.158 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.158 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.158 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.158 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.159 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.159 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.159 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.159 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.159 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.159 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.159 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.159 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.160 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.160 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.160 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.160 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.160 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.160 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.160 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.160 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.161 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.161 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.161 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.161 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.161 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.161 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.161 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.161 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.162 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.162 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.162 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.162 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.162 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.162 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.162 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.162 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.163 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.163 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.163 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.163 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.163 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.163 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.163 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.163 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.164 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.164 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.164 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.164 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.164 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.164 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.164 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.164 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.165 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.165 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.165 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.165 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.165 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.165 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.165 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.165 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.166 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.166 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirroring", 2024-06-10T01:10:56.166 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.166 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.166 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.166 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.166 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.166 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.167 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.167 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.167 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.167 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.167 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.167 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.167 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.167 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.167 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.168 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.168 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.168 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.168 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.168 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.168 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.168 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.168 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.169 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.169 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.169 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.169 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.169 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.169 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.169 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.169 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.169 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.170 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.170 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.170 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.170 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.170 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.170 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.170 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.171 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.171 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.171 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.171 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.171 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.171 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.171 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.171 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.172 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.172 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.172 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.172 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.172 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.172 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.172 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.172 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.172 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.173 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.173 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.173 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.173 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.173 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.173 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.173 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.173 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.174 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.174 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.174 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.174 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.174 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.174 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.174 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.174 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.174 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.175 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.175 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.175 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "nfs", 2024-06-10T01:10:56.175 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.175 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.175 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.175 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.175 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.176 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.176 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.176 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.176 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.176 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.176 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.176 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.176 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.177 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.177 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.177 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.177 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.177 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.177 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.177 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.177 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.177 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.178 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.178 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.178 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.178 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.178 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.178 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.178 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.179 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.179 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.179 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.179 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.179 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.179 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.179 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.179 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.179 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.180 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.180 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.180 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.180 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.180 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.180 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.180 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.180 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.180 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.181 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.181 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.181 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.181 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.181 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.181 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.181 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.181 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.181 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.182 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.182 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.182 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.182 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.182 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.182 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.182 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.182 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.183 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.183 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.183 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.183 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.183 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.183 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.183 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.183 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.183 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.184 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.184 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.184 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-06-10T01:10:56.184 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.184 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.184 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.184 INFO:teuthology.orchestra.run.smithi037.stdout: "fail_fs": { 2024-06-10T01:10:56.185 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "fail_fs", 2024-06-10T01:10:56.185 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.185 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.185 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.185 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.185 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.185 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.185 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.185 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-10T01:10:56.186 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.186 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.186 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.186 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.186 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.186 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.186 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.187 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.187 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.187 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.187 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.187 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.187 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.187 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.187 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.187 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.188 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.188 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.188 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.188 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.188 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.188 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.188 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.188 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.188 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.189 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.189 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.189 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.189 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.189 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.189 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.189 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.189 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.189 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.190 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.190 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.190 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.190 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.190 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.190 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.190 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.190 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.191 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.191 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.191 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.191 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.191 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.191 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.191 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.191 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.192 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.192 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.192 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.192 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.192 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.192 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.192 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.192 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.192 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.192 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.193 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.193 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.193 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.193 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.193 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.193 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.193 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.193 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.193 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.194 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.194 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.194 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.194 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator": { 2024-06-10T01:10:56.194 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-06-10T01:10:56.194 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.194 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.194 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.195 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.195 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.195 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.195 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.195 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-06-10T01:10:56.195 INFO:teuthology.orchestra.run.smithi037.stdout: "rook", 2024-06-10T01:10:56.195 INFO:teuthology.orchestra.run.smithi037.stdout: "test_orchestrator" 2024-06-10T01:10:56.195 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.195 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Orchestrator backend", 2024-06-10T01:10:56.196 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.196 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.196 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.196 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.196 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.196 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.196 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.196 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_perf_query", 2024-06-10T01:10:56.196 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.197 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.197 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.197 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.197 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.197 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.197 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.197 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.197 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.198 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.198 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.198 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.198 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.198 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.198 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.198 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.198 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.199 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.199 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.199 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.199 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.199 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.199 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.199 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.199 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.200 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.200 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.200 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.200 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.200 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.200 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.200 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.200 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.201 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.201 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.201 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.201 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.201 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.201 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.201 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.201 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.202 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.202 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.202 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.202 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.202 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.202 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.202 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.202 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.203 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.203 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.203 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.203 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.203 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.203 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.203 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.203 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.203 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.204 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.204 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.204 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.204 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.204 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.204 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.204 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.204 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.205 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.205 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.205 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.205 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.205 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.205 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.205 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.205 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.205 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.206 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.206 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_support", 2024-06-10T01:10:56.206 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.206 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.206 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.206 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.206 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.207 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.207 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.207 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.207 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.207 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.207 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.207 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.207 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.208 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.208 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.208 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.208 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.208 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.208 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.208 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.208 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.209 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.209 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.209 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.209 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.209 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.209 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.209 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.209 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.210 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.210 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.210 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.210 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.210 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.210 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.210 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.211 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.211 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.211 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.211 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.211 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.211 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.211 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.211 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.212 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.212 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.212 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.212 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.212 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.212 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.212 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.212 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.212 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.213 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.213 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.213 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.213 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.213 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.213 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.213 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.213 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.214 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.214 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.214 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.214 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.214 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.214 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.214 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.214 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.215 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.215 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.215 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.215 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.215 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_autoscaler", 2024-06-10T01:10:56.215 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.215 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.216 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.216 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.216 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.216 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.216 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.216 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.216 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.216 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.217 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.217 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.217 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.217 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.217 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.217 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.217 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.217 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.217 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.218 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.218 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.218 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.218 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.218 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.218 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.218 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.219 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.219 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.219 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.219 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.219 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.219 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.219 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.219 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.219 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.220 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.220 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.220 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.220 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.220 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.220 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.220 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.221 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.221 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.221 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.221 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.221 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.221 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.221 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.221 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.222 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.222 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.222 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.222 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.222 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.222 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.222 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.223 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.223 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.223 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.223 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.223 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.223 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.223 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.223 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.224 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.224 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.224 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.224 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.224 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-06-10T01:10:56.224 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-06-10T01:10:56.224 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:56.225 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.225 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.225 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-06-10T01:10:56.225 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.225 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.225 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.225 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.225 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.225 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.226 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.226 INFO:teuthology.orchestra.run.smithi037.stdout: "threshold": { 2024-06-10T01:10:56.226 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threshold", 2024-06-10T01:10:56.226 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-06-10T01:10:56.226 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.226 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.226 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-06-10T01:10:56.226 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1.0", 2024-06-10T01:10:56.227 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.227 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.227 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "scaling threshold", 2024-06-10T01:10:56.227 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-10T01:10:56.227 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.227 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.227 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.227 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.228 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.228 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "progress", 2024-06-10T01:10:56.228 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.228 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.228 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.228 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_pg_recovery_event": { 2024-06-10T01:10:56.228 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_pg_recovery_event", 2024-06-10T01:10:56.228 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.228 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.229 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.229 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.229 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.229 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.229 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.229 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-10T01:10:56.229 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.229 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.229 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.230 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-06-10T01:10:56.230 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-06-10T01:10:56.230 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.230 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.230 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.230 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.230 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.231 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.231 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.231 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.231 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.231 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.231 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.231 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.231 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.232 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.232 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.232 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.232 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.232 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.232 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.232 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.232 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.233 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.233 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.233 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.233 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.233 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.233 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.233 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.233 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.234 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.234 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.234 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.234 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.234 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.234 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.234 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.235 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.235 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.235 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.235 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.235 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.235 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.235 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.235 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.236 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.236 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.236 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.236 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.236 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.236 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.236 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.236 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.237 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.237 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.237 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.237 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.237 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.237 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.237 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.237 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.238 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.238 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.238 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.238 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.238 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.238 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.238 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.238 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.239 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.239 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.239 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.239 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.239 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.239 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.239 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.239 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.239 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.240 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.240 INFO:teuthology.orchestra.run.smithi037.stdout: "max_completed_events": { 2024-06-10T01:10:56.240 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_completed_events", 2024-06-10T01:10:56.240 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.240 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.240 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.240 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "50", 2024-06-10T01:10:56.241 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.241 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.241 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.241 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "number of past completed events to remember", 2024-06-10T01:10:56.241 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.241 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.241 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.242 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-06-10T01:10:56.242 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-06-10T01:10:56.242 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:56.242 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.242 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.242 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-06-10T01:10:56.242 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.242 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.243 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.243 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long the module is going to sleep", 2024-06-10T01:10:56.243 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.243 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.243 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.243 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.243 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.243 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.244 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus", 2024-06-10T01:10:56.244 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.244 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.244 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.244 INFO:teuthology.orchestra.run.smithi037.stdout: "cache": { 2024-06-10T01:10:56.244 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cache", 2024-06-10T01:10:56.244 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.244 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.245 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.245 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.245 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.245 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.245 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.245 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.245 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.245 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.245 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.246 INFO:teuthology.orchestra.run.smithi037.stdout: "exclude_perf_counters": { 2024-06-10T01:10:56.246 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "exclude_perf_counters", 2024-06-10T01:10:56.246 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.246 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.246 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.246 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.247 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.247 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.247 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.247 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-10T01:10:56.247 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-10T01:10:56.247 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.247 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.248 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.248 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.248 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.248 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.248 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.248 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.248 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.248 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.249 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.249 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.249 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.249 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.249 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.249 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.249 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.249 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.250 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.250 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.250 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.250 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.250 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.250 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.250 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.251 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.251 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.251 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.251 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.251 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.251 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.251 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.251 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.251 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.252 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.252 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.252 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.252 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.252 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.252 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.252 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.253 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.253 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.253 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.253 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.253 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.253 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.253 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.253 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.253 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.254 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.254 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.254 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.254 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.254 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.254 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.254 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.255 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.255 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.255 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.255 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.255 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.255 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.255 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.255 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.256 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.256 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.256 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.256 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools": { 2024-06-10T01:10:56.256 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools", 2024-06-10T01:10:56.256 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.256 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.257 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.257 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.257 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.257 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.257 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.257 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.257 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.257 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.257 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.258 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-10T01:10:56.258 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-10T01:10:56.258 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.258 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.258 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.258 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "300", 2024-06-10T01:10:56.258 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.259 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.259 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.259 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.259 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.259 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.259 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.259 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_interval": { 2024-06-10T01:10:56.260 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_interval", 2024-06-10T01:10:56.260 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-06-10T01:10:56.260 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.260 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.260 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15.0", 2024-06-10T01:10:56.260 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.260 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.260 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.260 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.261 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.261 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.261 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.261 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-06-10T01:10:56.261 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-06-10T01:10:56.261 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.261 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.262 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.262 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-06-10T01:10:56.262 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.262 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.262 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.262 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-10T01:10:56.262 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.263 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.263 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.263 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-06-10T01:10:56.263 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-06-10T01:10:56.263 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.263 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.263 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.263 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "9283", 2024-06-10T01:10:56.264 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.264 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.264 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.264 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-10T01:10:56.264 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.264 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.264 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.265 INFO:teuthology.orchestra.run.smithi037.stdout: "stale_cache_strategy": { 2024-06-10T01:10:56.265 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stale_cache_strategy", 2024-06-10T01:10:56.265 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.265 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.265 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.265 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "log", 2024-06-10T01:10:56.265 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.265 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.265 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.266 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.266 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.266 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.266 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.266 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-06-10T01:10:56.266 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-06-10T01:10:56.267 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.267 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.267 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.267 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "default", 2024-06-10T01:10:56.267 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.267 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.267 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.267 INFO:teuthology.orchestra.run.smithi037.stdout: "default", 2024-06-10T01:10:56.268 INFO:teuthology.orchestra.run.smithi037.stdout: "error" 2024-06-10T01:10:56.268 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.268 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.268 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.268 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.268 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.268 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-06-10T01:10:56.269 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-06-10T01:10:56.269 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.269 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.269 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.269 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-06-10T01:10:56.269 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-06-10T01:10:56.269 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-06-10T01:10:56.269 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.269 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.270 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.270 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.270 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.270 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.270 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.270 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.270 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-06-10T01:10:56.271 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.271 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.271 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.271 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.271 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.271 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.271 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.271 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.272 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.272 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.272 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.272 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.272 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.272 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.272 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.272 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.273 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.273 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.273 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.273 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.273 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.273 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.273 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.274 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.274 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.274 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.274 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.274 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.274 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.274 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.274 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.275 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.275 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.275 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.275 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.275 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.275 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.275 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.275 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.276 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.276 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.276 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.276 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.276 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.276 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.276 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.276 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.276 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.277 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.277 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.277 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.277 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.277 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.277 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.277 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.277 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.278 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.278 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.278 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.278 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.278 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.278 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.278 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.279 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.279 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.279 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.279 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.279 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.279 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.279 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.280 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_snap_create": { 2024-06-10T01:10:56.280 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_snap_create", 2024-06-10T01:10:56.280 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.280 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.280 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.280 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-06-10T01:10:56.280 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.281 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.281 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.281 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.281 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.281 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.281 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.281 INFO:teuthology.orchestra.run.smithi037.stdout: "mirror_snapshot_schedule": { 2024-06-10T01:10:56.282 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirror_snapshot_schedule", 2024-06-10T01:10:56.282 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.282 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.282 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.282 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.282 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.282 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.282 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.283 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.283 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.283 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.283 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.283 INFO:teuthology.orchestra.run.smithi037.stdout: "trash_purge_schedule": { 2024-06-10T01:10:56.283 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "trash_purge_schedule", 2024-06-10T01:10:56.283 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.283 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.284 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.284 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.284 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.284 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.284 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.284 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.284 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.284 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.285 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.285 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.285 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.285 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "restful", 2024-06-10T01:10:56.285 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.285 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.285 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.286 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_auth": { 2024-06-10T01:10:56.286 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_auth", 2024-06-10T01:10:56.286 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.286 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.286 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.286 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.286 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.286 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.287 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.287 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.287 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.287 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.287 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.287 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-06-10T01:10:56.287 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-06-10T01:10:56.288 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.288 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.288 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.288 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.288 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.288 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.288 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.288 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.289 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.289 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.289 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.289 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.289 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.289 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.289 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.289 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.289 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.290 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.290 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.290 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.290 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.290 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.290 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.290 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.290 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.291 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.291 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.291 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.291 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.291 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.291 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.291 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.292 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.292 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.292 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.292 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.292 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.292 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.292 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.292 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.293 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.293 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.293 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.293 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.293 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.293 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.293 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.293 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.294 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.294 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.294 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.294 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.294 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.294 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.294 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.294 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.295 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.295 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.295 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.295 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.295 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.295 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.295 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.295 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.295 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.296 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.296 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.296 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.296 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.296 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.296 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.296 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.297 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.297 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.297 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.297 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.297 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.297 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.297 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-06-10T01:10:56.298 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-06-10T01:10:56.298 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.298 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.298 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.298 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.298 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.298 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.298 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.299 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.299 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.299 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.299 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.299 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-06-10T01:10:56.299 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-06-10T01:10:56.299 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.299 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.300 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.300 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.300 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.300 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.300 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.300 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.300 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.300 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.301 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.301 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.301 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.301 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.301 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rgw", 2024-06-10T01:10:56.301 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.301 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.302 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.302 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.302 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.302 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.302 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.302 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.302 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.302 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.303 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.303 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.303 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.303 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.303 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.303 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.303 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.303 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.304 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.304 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.304 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.304 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.304 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.304 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.305 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.305 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.305 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.305 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.305 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.305 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.305 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.305 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.305 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.306 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.306 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.306 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.306 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.306 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.306 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.307 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.307 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.307 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.307 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.307 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.307 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.307 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.307 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.308 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.308 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.308 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.308 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.308 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.308 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.308 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.308 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.309 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.309 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.309 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.309 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.309 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.309 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.309 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.310 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.310 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.310 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.310 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.310 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.310 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.310 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.311 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.311 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.311 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rook", 2024-06-10T01:10:56.311 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.311 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.311 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.312 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.312 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.312 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.312 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.312 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.312 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.312 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.312 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.312 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.313 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.313 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.313 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.313 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.313 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.313 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.313 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.313 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.314 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.314 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.314 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.314 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.314 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.314 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.314 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.315 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.315 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.315 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.315 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.315 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.315 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.315 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.315 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.316 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.316 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.316 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.316 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.316 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.316 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.316 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.316 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.317 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.317 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.317 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.317 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.317 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.317 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.317 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.317 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.318 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.318 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.318 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.318 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.318 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.318 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.318 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.319 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.319 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.319 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.319 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.319 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.319 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.319 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.319 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.320 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.320 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.320 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.320 INFO:teuthology.orchestra.run.smithi037.stdout: "storage_class": { 2024-06-10T01:10:56.320 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "storage_class", 2024-06-10T01:10:56.320 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.320 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.321 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.321 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "local", 2024-06-10T01:10:56.321 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.321 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.321 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.321 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-10T01:10:56.321 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.321 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.322 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.322 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.322 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.322 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.322 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "selftest", 2024-06-10T01:10:56.322 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.322 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.323 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.323 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.323 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.323 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.323 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.323 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.323 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.323 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.323 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.324 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.324 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.324 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.324 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.324 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.324 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.324 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.324 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.325 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.325 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.325 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.325 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.325 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.325 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.325 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.326 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.326 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.326 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.326 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.326 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.326 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.326 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.326 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.327 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.327 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.327 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.327 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.327 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.327 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.327 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.327 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.328 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.328 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.328 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.328 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.328 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.328 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.328 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.328 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.328 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.329 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.329 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.329 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.329 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.329 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.329 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.329 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.330 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.330 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.330 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.330 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.330 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.330 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.330 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.330 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.331 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.331 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.331 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.331 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.331 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.331 INFO:teuthology.orchestra.run.smithi037.stdout: "roption1": { 2024-06-10T01:10:56.331 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption1", 2024-06-10T01:10:56.331 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.331 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.332 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.332 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.332 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.332 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.332 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.332 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.332 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.332 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.332 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.333 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.333 INFO:teuthology.orchestra.run.smithi037.stdout: "roption2": { 2024-06-10T01:10:56.333 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption2", 2024-06-10T01:10:56.333 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.333 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.333 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.333 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "xyz", 2024-06-10T01:10:56.333 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.334 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.334 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.334 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.334 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.334 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.334 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.334 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption1": { 2024-06-10T01:10:56.335 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption1", 2024-06-10T01:10:56.335 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.335 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.335 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.335 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.335 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.335 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.335 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.336 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.336 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.336 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.336 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.336 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption2": { 2024-06-10T01:10:56.336 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption2", 2024-06-10T01:10:56.336 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.337 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.337 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.337 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.337 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.337 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.337 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.337 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.337 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.337 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.338 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.338 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.338 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption3": { 2024-06-10T01:10:56.338 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption3", 2024-06-10T01:10:56.338 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-06-10T01:10:56.338 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.338 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.339 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.339 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.339 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.339 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.339 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.339 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.339 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.339 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.339 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.340 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption4": { 2024-06-10T01:10:56.340 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption4", 2024-06-10T01:10:56.340 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.340 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.340 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.340 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.340 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.340 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.341 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.341 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.341 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.341 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.341 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.341 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption5": { 2024-06-10T01:10:56.341 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption5", 2024-06-10T01:10:56.342 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.342 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.342 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.342 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.342 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.342 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.342 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.342 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.343 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.343 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.343 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.343 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.343 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption6": { 2024-06-10T01:10:56.343 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption6", 2024-06-10T01:10:56.343 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.343 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.344 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.344 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.344 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.344 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.344 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.344 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.344 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.344 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.345 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.345 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption7": { 2024-06-10T01:10:56.345 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption7", 2024-06-10T01:10:56.345 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.345 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.345 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.345 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.345 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-06-10T01:10:56.346 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "42", 2024-06-10T01:10:56.346 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.346 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.346 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.346 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.346 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.347 INFO:teuthology.orchestra.run.smithi037.stdout: "testkey": { 2024-06-10T01:10:56.347 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testkey", 2024-06-10T01:10:56.347 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.347 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.347 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.347 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.347 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.347 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.348 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.348 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.348 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.348 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.348 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.348 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.348 INFO:teuthology.orchestra.run.smithi037.stdout: "testlkey": { 2024-06-10T01:10:56.348 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testlkey", 2024-06-10T01:10:56.348 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.349 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.349 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.349 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.349 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.349 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.349 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.349 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.350 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.350 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.350 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.350 INFO:teuthology.orchestra.run.smithi037.stdout: "testnewline": { 2024-06-10T01:10:56.350 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testnewline", 2024-06-10T01:10:56.350 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.350 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.351 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.351 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.351 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.351 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.351 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.351 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.351 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.351 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.351 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.352 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.352 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.352 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.352 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.352 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-06-10T01:10:56.352 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.352 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.352 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.353 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_m_granularity": { 2024-06-10T01:10:56.353 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_m_granularity", 2024-06-10T01:10:56.353 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.353 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.353 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.353 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.353 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.353 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.353 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.354 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow minute scheduled snapshots", 2024-06-10T01:10:56.354 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.354 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.354 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.354 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.354 INFO:teuthology.orchestra.run.smithi037.stdout: "dump_on_update": { 2024-06-10T01:10:56.354 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dump_on_update", 2024-06-10T01:10:56.354 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.355 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.355 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.355 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.355 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.355 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.355 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.355 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "dump database to debug log on update", 2024-06-10T01:10:56.355 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.356 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.356 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.356 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.356 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.356 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.356 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.356 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.356 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.357 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.357 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.357 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.357 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.357 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.357 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.357 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.357 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.357 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.358 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.358 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.358 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.358 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.358 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.358 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.358 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.359 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.359 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.359 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.359 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.359 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.359 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.359 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.359 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.360 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.360 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.360 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.360 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.360 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.360 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.360 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.361 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.361 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.361 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.361 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.361 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.361 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.361 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.361 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.362 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.362 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.362 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.362 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.362 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.362 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.362 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.362 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.363 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.363 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.363 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.363 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.363 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.363 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.363 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.364 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.364 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.364 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.364 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.364 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.364 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.364 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.364 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.364 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.365 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.365 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.365 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stats", 2024-06-10T01:10:56.365 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.365 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.365 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.365 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.365 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.366 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.366 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.366 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.366 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.366 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.366 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.366 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.366 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.367 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.367 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.367 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.367 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.367 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.367 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.367 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.367 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.368 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.368 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.368 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.368 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.368 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.368 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.368 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.369 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.369 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.369 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.369 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.369 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.369 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.369 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.369 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.370 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.370 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.370 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.370 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.370 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.370 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.370 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.371 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.371 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.371 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.371 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.371 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.371 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.371 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.371 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.372 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.372 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.372 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.372 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.372 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.372 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.372 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.372 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.373 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.373 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.373 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.373 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.373 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.373 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.373 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.373 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.374 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.374 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.374 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.374 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.374 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.374 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.374 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.374 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.375 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "status", 2024-06-10T01:10:56.375 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.375 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.375 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.375 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.375 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.375 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.375 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.376 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.376 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.376 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.376 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.376 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.376 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.376 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.376 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.376 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.377 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.377 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.377 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.377 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.377 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.377 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.377 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.378 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.378 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.378 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.378 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.378 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.378 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.378 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.378 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.379 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.379 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.379 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.379 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.379 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.379 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.379 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.379 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.379 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.380 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.380 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.380 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.380 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.380 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.380 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.380 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.380 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.381 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.381 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.381 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.381 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.381 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.381 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.381 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.382 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.382 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.382 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.382 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.382 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.382 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.382 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.383 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.383 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.383 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.383 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.383 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.383 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.383 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.383 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.383 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.384 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.384 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.384 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.384 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telegraf", 2024-06-10T01:10:56.384 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.384 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.384 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.384 INFO:teuthology.orchestra.run.smithi037.stdout: "address": { 2024-06-10T01:10:56.385 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "address", 2024-06-10T01:10:56.385 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.385 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.385 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.385 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-10T01:10:56.385 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.385 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.385 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.385 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.386 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.386 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.386 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.386 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-06-10T01:10:56.386 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-06-10T01:10:56.386 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:56.386 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.387 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.387 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15", 2024-06-10T01:10:56.387 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.387 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.387 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.387 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.387 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.387 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.387 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.388 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.388 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.388 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.388 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.388 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.388 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.388 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.388 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.389 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.389 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.389 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.389 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.389 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.389 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.389 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.389 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.390 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.390 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.390 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.390 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.390 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.390 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.390 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.390 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.391 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.391 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.391 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.391 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.391 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.391 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.391 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.391 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.392 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.392 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.392 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.392 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.392 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.392 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.392 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.392 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.393 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.393 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.393 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.393 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.393 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.393 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.393 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.393 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.393 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.394 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.394 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.394 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.394 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.394 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.394 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.394 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.394 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.395 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.395 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.395 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.395 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.395 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.395 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.395 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.395 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.396 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.396 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.396 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.396 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.396 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.396 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.396 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.396 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.397 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telemetry", 2024-06-10T01:10:56.397 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.397 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.397 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.397 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_basic": { 2024-06-10T01:10:56.397 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_basic", 2024-06-10T01:10:56.397 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.397 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.398 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.398 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.398 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.398 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.398 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.398 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-10T01:10:56.398 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.399 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.399 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.399 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.399 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_crash": { 2024-06-10T01:10:56.399 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_crash", 2024-06-10T01:10:56.399 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.399 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.399 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.400 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.400 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.400 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.400 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.400 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-10T01:10:56.400 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.400 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.400 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.401 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_device": { 2024-06-10T01:10:56.401 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_device", 2024-06-10T01:10:56.401 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.401 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.401 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.401 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.401 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.402 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.402 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.402 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-10T01:10:56.402 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.402 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.402 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.402 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_ident": { 2024-06-10T01:10:56.403 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_ident", 2024-06-10T01:10:56.403 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.403 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.403 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.403 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.403 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.403 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.403 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.404 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-10T01:10:56.404 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.404 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.404 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.404 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_perf": { 2024-06-10T01:10:56.404 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_perf", 2024-06-10T01:10:56.404 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.405 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.405 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.405 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.405 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.405 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.405 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.405 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-10T01:10:56.405 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.406 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.406 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.406 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.406 INFO:teuthology.orchestra.run.smithi037.stdout: "contact": { 2024-06-10T01:10:56.406 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "contact", 2024-06-10T01:10:56.406 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.406 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.406 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.407 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.407 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.407 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.407 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.407 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.407 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.407 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.407 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.408 INFO:teuthology.orchestra.run.smithi037.stdout: "description": { 2024-06-10T01:10:56.408 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "description", 2024-06-10T01:10:56.408 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.408 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.408 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.408 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.408 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.409 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.409 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.409 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.409 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.409 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.409 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.409 INFO:teuthology.orchestra.run.smithi037.stdout: "device_url": { 2024-06-10T01:10:56.410 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_url", 2024-06-10T01:10:56.410 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.410 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.410 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.410 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-10T01:10:56.410 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.410 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.410 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.411 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.411 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.411 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.411 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.411 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-06-10T01:10:56.411 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-06-10T01:10:56.411 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.412 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.412 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.412 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.412 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.412 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.412 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.412 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.412 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.413 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.413 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.413 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-06-10T01:10:56.413 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-06-10T01:10:56.413 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.413 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.413 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.413 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "24", 2024-06-10T01:10:56.414 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "8", 2024-06-10T01:10:56.414 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.414 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.414 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.414 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.414 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.414 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.415 INFO:teuthology.orchestra.run.smithi037.stdout: "last_opt_revision": { 2024-06-10T01:10:56.415 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "last_opt_revision", 2024-06-10T01:10:56.415 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.415 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.415 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.415 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1", 2024-06-10T01:10:56.415 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.416 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.416 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.416 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.416 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.416 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.416 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.416 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.416 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard": { 2024-06-10T01:10:56.417 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard", 2024-06-10T01:10:56.417 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.417 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.417 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.417 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.417 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.417 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.417 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.417 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.418 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.418 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.418 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.418 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard_description": { 2024-06-10T01:10:56.418 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard_description", 2024-06-10T01:10:56.418 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.418 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.419 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.419 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.419 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.419 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.419 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.419 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.419 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.419 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.419 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.420 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.420 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.420 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.420 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.420 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.420 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.420 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.421 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.421 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.421 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.421 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.421 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.421 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.421 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.421 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.422 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.422 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.422 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.422 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.422 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.422 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.422 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.422 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.423 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.423 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.423 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.423 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.423 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.423 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.423 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.423 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.424 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.424 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.424 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.424 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.424 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.424 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.424 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.425 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.425 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.425 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.425 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.425 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.425 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.425 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.425 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.425 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.426 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.426 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.426 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.426 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.426 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.426 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.426 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.427 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.427 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.427 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.427 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.427 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.427 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.427 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.427 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.428 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.428 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.428 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.428 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.428 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.428 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.428 INFO:teuthology.orchestra.run.smithi037.stdout: "organization": { 2024-06-10T01:10:56.428 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "organization", 2024-06-10T01:10:56.429 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.429 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.429 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.429 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.429 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.429 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.429 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.429 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.430 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.430 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.430 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.430 INFO:teuthology.orchestra.run.smithi037.stdout: "proxy": { 2024-06-10T01:10:56.430 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "proxy", 2024-06-10T01:10:56.430 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.430 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.431 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.431 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.431 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.431 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.431 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.431 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.431 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.431 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.432 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.432 INFO:teuthology.orchestra.run.smithi037.stdout: "url": { 2024-06-10T01:10:56.432 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url", 2024-06-10T01:10:56.432 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.432 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.432 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.432 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-10T01:10:56.433 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.433 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.433 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.433 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.433 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.433 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.433 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.434 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.434 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.434 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.434 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.434 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "test_orchestrator", 2024-06-10T01:10:56.434 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.434 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.435 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.435 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.435 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.435 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.435 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.435 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.435 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.435 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.435 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.436 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.436 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.436 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.436 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.436 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.436 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.436 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.436 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.437 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.437 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.437 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.437 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.437 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.437 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.437 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.438 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.438 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.438 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.438 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.438 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.438 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.438 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.438 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.439 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.439 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.439 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.439 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.439 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.439 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.439 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.440 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.440 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.440 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.440 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.440 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.440 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.440 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.440 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.441 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.441 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.441 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.441 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.441 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.441 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.441 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.441 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.442 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.442 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.442 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.442 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.442 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.442 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.442 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.442 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.443 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.443 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.443 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.443 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.443 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.443 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.443 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.443 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.444 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.444 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-06-10T01:10:56.444 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.444 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.444 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.444 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.444 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.444 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.444 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.445 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.445 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.445 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.445 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.445 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.445 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.445 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.445 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.446 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.446 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.446 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.446 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.446 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.446 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.446 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.446 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.447 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.447 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.447 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.447 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.447 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.447 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.447 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.448 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.448 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.448 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.448 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.448 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.448 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.448 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.448 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.449 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.449 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.449 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.449 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.449 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.449 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.449 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.449 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.450 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.450 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.450 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.450 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.450 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.450 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.450 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.451 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.451 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.451 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.451 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.451 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.451 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.451 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.451 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.452 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.452 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.452 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.452 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.452 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.452 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.452 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.452 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.452 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.453 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.453 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.453 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_clones": { 2024-06-10T01:10:56.453 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_clones", 2024-06-10T01:10:56.453 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.453 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.453 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.453 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4", 2024-06-10T01:10:56.454 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.454 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.454 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.454 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-10T01:10:56.454 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.454 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.454 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.454 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.455 INFO:teuthology.orchestra.run.smithi037.stdout: "periodic_async_work": { 2024-06-10T01:10:56.455 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "periodic_async_work", 2024-06-10T01:10:56.455 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.455 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.455 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.455 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.455 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.455 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.455 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.456 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Periodically check for async work", 2024-06-10T01:10:56.456 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.456 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.456 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.456 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_delay": { 2024-06-10T01:10:56.456 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_delay", 2024-06-10T01:10:56.456 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.457 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.457 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.457 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-06-10T01:10:56.457 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.457 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.457 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.457 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-10T01:10:56.457 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.458 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.458 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.458 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.458 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_no_wait": { 2024-06-10T01:10:56.458 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_no_wait", 2024-06-10T01:10:56.458 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.458 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.458 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.459 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.459 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.459 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.459 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.459 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-10T01:10:56.459 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.459 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.459 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.460 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.460 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.460 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.460 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix", 2024-06-10T01:10:56.460 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.460 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.460 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.460 INFO:teuthology.orchestra.run.smithi037.stdout: "discovery_interval": { 2024-06-10T01:10:56.461 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "discovery_interval", 2024-06-10T01:10:56.461 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-06-10T01:10:56.461 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.461 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.461 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "100", 2024-06-10T01:10:56.461 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.461 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.461 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.462 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.462 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.462 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.462 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.462 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.462 INFO:teuthology.orchestra.run.smithi037.stdout: "identifier": { 2024-06-10T01:10:56.462 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "identifier", 2024-06-10T01:10:56.462 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.463 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.463 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.463 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.463 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.463 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.463 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.463 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.463 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.463 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.464 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.464 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.464 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-06-10T01:10:56.464 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-06-10T01:10:56.464 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:56.464 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.464 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.464 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-06-10T01:10:56.464 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.464 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.465 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.465 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.465 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.465 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.465 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.465 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.465 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.465 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.466 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.466 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.466 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.466 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.466 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.466 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.466 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.466 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.467 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.467 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.467 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.467 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.467 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.467 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.467 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.467 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.467 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.468 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.468 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.468 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.468 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.468 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.468 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.468 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.468 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.468 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.468 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.469 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.469 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.469 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.469 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.469 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.469 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.469 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.469 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.470 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.470 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.470 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.470 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.470 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.470 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.470 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.470 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.471 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.471 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.471 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.471 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.471 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.471 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.471 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.471 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.472 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.472 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.472 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.472 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.472 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.472 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.472 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.472 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.473 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.473 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.473 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.473 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.473 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.473 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.473 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.473 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_host": { 2024-06-10T01:10:56.473 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_host", 2024-06-10T01:10:56.473 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.474 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.474 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.474 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.474 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.474 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.474 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.474 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.475 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.475 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.475 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.475 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.475 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_port": { 2024-06-10T01:10:56.475 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_port", 2024-06-10T01:10:56.475 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.475 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.475 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.476 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10051", 2024-06-10T01:10:56.476 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.476 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.476 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.476 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.476 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.476 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.476 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.477 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.477 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_sender": { 2024-06-10T01:10:56.477 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_sender", 2024-06-10T01:10:56.477 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.477 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.477 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.477 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-10T01:10:56.477 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.478 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.478 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.478 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.478 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.478 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.478 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.478 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.479 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.479 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.479 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-06-10T01:10:56.479 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.479 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.479 INFO:teuthology.orchestra.run.smithi037.stdout: "modules": [ 2024-06-10T01:10:56.479 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-06-10T01:10:56.479 INFO:teuthology.orchestra.run.smithi037.stdout: "dashboard", 2024-06-10T01:10:56.480 INFO:teuthology.orchestra.run.smithi037.stdout: "iostat", 2024-06-10T01:10:56.480 INFO:teuthology.orchestra.run.smithi037.stdout: "nfs", 2024-06-10T01:10:56.480 INFO:teuthology.orchestra.run.smithi037.stdout: "restful" 2024-06-10T01:10:56.480 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.480 INFO:teuthology.orchestra.run.smithi037.stdout: "available_modules": [ 2024-06-10T01:10:56.480 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.480 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "alerts", 2024-06-10T01:10:56.480 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.481 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.481 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.481 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-06-10T01:10:56.481 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-06-10T01:10:56.481 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:56.481 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.481 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.481 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-06-10T01:10:56.482 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.482 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.482 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.482 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How frequently to reexamine health status", 2024-06-10T01:10:56.482 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.482 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.482 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.483 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.483 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.483 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.483 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.483 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.483 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.483 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.483 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.484 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.484 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.484 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.484 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.484 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.484 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.484 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.484 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.485 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.485 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.485 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.485 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.485 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.485 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.485 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.485 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.486 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.486 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.486 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.486 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.486 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.486 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.486 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.486 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.487 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.487 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.487 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.487 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.487 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.487 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.487 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.487 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.488 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.488 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.488 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.488 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.488 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.488 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.488 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.488 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.489 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.489 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.489 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.489 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.489 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.489 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.489 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.489 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.490 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.490 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.490 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.490 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.490 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.490 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.490 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.491 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.491 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.491 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.491 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.491 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.491 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.491 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.491 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_destination": { 2024-06-10T01:10:56.492 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_destination", 2024-06-10T01:10:56.492 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.492 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.492 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.492 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.492 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.492 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.492 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.492 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email address to send alerts to", 2024-06-10T01:10:56.493 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.493 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.493 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.493 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.493 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_from_name": { 2024-06-10T01:10:56.493 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_from_name", 2024-06-10T01:10:56.493 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.493 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.494 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.494 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "Ceph", 2024-06-10T01:10:56.494 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.494 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.494 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.494 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email From: name", 2024-06-10T01:10:56.494 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.494 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.494 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.495 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.495 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_host": { 2024-06-10T01:10:56.495 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_host", 2024-06-10T01:10:56.495 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.495 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.495 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.495 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.495 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.495 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.496 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.496 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP server", 2024-06-10T01:10:56.496 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.496 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.496 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.496 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.496 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_password": { 2024-06-10T01:10:56.496 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_password", 2024-06-10T01:10:56.496 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.497 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.497 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.497 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.497 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.497 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.497 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.497 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Password to authenticate with", 2024-06-10T01:10:56.497 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.497 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.498 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.498 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.498 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_port": { 2024-06-10T01:10:56.498 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_port", 2024-06-10T01:10:56.498 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.498 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.498 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.498 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "465", 2024-06-10T01:10:56.499 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.499 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.499 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.499 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP port", 2024-06-10T01:10:56.499 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.499 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.499 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.499 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.499 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_sender": { 2024-06-10T01:10:56.500 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_sender", 2024-06-10T01:10:56.500 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.500 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.500 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.500 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.500 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.500 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.500 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.501 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP envelope sender", 2024-06-10T01:10:56.501 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.501 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.501 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.501 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.501 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_ssl": { 2024-06-10T01:10:56.501 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_ssl", 2024-06-10T01:10:56.501 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.502 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.502 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.502 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.502 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.502 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.502 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.502 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-10T01:10:56.502 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.503 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.503 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.503 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_user": { 2024-06-10T01:10:56.503 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_user", 2024-06-10T01:10:56.503 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.503 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.503 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.503 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.504 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.504 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.504 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.504 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "User to authenticate as", 2024-06-10T01:10:56.504 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.504 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.504 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.504 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.504 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.505 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.505 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.505 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "balancer", 2024-06-10T01:10:56.505 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.505 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.505 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.505 INFO:teuthology.orchestra.run.smithi037.stdout: "active": { 2024-06-10T01:10:56.505 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "active", 2024-06-10T01:10:56.505 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.506 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.506 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.506 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.506 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.506 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.506 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.506 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically balance PGs across cluster", 2024-06-10T01:10:56.507 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.507 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.507 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.507 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.507 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_time": { 2024-06-10T01:10:56.507 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_time", 2024-06-10T01:10:56.507 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.507 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.507 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.508 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0000", 2024-06-10T01:10:56.508 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.508 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.508 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.508 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "beginning time of day to automatically balance", 2024-06-10T01:10:56.508 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-10T01:10:56.508 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.508 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.508 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.509 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_weekday": { 2024-06-10T01:10:56.509 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_weekday", 2024-06-10T01:10:56.509 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-06-10T01:10:56.509 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.509 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.509 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-06-10T01:10:56.509 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-06-10T01:10:56.509 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-06-10T01:10:56.510 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.510 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-10T01:10:56.510 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-10T01:10:56.510 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.510 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.510 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.510 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_max_iterations": { 2024-06-10T01:10:56.511 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_max_iterations", 2024-06-10T01:10:56.511 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-06-10T01:10:56.511 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.511 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.511 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "25", 2024-06-10T01:10:56.511 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-06-10T01:10:56.511 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "250", 2024-06-10T01:10:56.511 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.512 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-10T01:10:56.512 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.512 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.512 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.512 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.512 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_metrics": { 2024-06-10T01:10:56.512 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_metrics", 2024-06-10T01:10:56.512 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.513 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.513 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.513 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "pgs,objects,bytes", 2024-06-10T01:10:56.513 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.513 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.513 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.513 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-10T01:10:56.513 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-10T01:10:56.514 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.514 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.514 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.514 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_step": { 2024-06-10T01:10:56.514 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_step", 2024-06-10T01:10:56.514 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-06-10T01:10:56.514 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.514 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.515 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.5", 2024-06-10T01:10:56.515 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0.001", 2024-06-10T01:10:56.515 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "0.999", 2024-06-10T01:10:56.515 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.515 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "aggressiveness of optimization", 2024-06-10T01:10:56.515 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-10T01:10:56.515 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.515 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.516 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.516 INFO:teuthology.orchestra.run.smithi037.stdout: "end_time": { 2024-06-10T01:10:56.516 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_time", 2024-06-10T01:10:56.516 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.516 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.516 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.516 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2359", 2024-06-10T01:10:56.516 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.517 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.517 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.517 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ending time of day to automatically balance", 2024-06-10T01:10:56.517 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-10T01:10:56.517 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.517 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.517 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.517 INFO:teuthology.orchestra.run.smithi037.stdout: "end_weekday": { 2024-06-10T01:10:56.518 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_weekday", 2024-06-10T01:10:56.518 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-06-10T01:10:56.518 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.518 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.518 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-06-10T01:10:56.518 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-06-10T01:10:56.518 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-06-10T01:10:56.518 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.519 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-10T01:10:56.519 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-10T01:10:56.519 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.519 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.519 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.519 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.519 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.519 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.520 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.520 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.520 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.520 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.520 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.520 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.520 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.520 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.521 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.521 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.521 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.521 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.521 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.521 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.521 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.521 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.522 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.522 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.522 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.522 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.522 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.522 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.522 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.522 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.523 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.523 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.523 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.523 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.523 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.523 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.523 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.523 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.523 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.524 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.524 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.524 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.524 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.524 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.524 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.524 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.524 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.525 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.525 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.525 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.525 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.525 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.525 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.525 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.525 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.526 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.526 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.526 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.526 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.526 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.526 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.526 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.526 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.527 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.527 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.527 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.527 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.527 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.527 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.527 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.527 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.528 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.528 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.528 INFO:teuthology.orchestra.run.smithi037.stdout: "min_score": { 2024-06-10T01:10:56.528 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_score", 2024-06-10T01:10:56.528 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-06-10T01:10:56.528 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.528 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.528 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-06-10T01:10:56.528 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.529 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.529 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.529 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-10T01:10:56.529 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.529 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.529 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.529 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-06-10T01:10:56.530 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-06-10T01:10:56.530 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.530 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.530 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.530 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "upmap", 2024-06-10T01:10:56.530 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.530 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.530 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.531 INFO:teuthology.orchestra.run.smithi037.stdout: "crush-compat", 2024-06-10T01:10:56.531 INFO:teuthology.orchestra.run.smithi037.stdout: "none", 2024-06-10T01:10:56.531 INFO:teuthology.orchestra.run.smithi037.stdout: "read", 2024-06-10T01:10:56.531 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap", 2024-06-10T01:10:56.531 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap-read" 2024-06-10T01:10:56.531 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.531 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Balancer mode", 2024-06-10T01:10:56.531 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.531 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.532 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.532 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.532 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_ids": { 2024-06-10T01:10:56.532 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_ids", 2024-06-10T01:10:56.532 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.532 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.532 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.532 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.533 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.533 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.533 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.533 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-10T01:10:56.533 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.533 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.533 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.533 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.534 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-06-10T01:10:56.534 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-06-10T01:10:56.534 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:56.534 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.534 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.534 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-06-10T01:10:56.534 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.534 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.535 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.535 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-10T01:10:56.535 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.535 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.535 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.535 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.535 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_deviation": { 2024-06-10T01:10:56.536 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_deviation", 2024-06-10T01:10:56.536 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.536 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.536 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.536 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-06-10T01:10:56.536 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-06-10T01:10:56.536 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.536 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.537 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-10T01:10:56.537 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-10T01:10:56.537 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.537 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.537 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.537 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_optimizations": { 2024-06-10T01:10:56.537 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_optimizations", 2024-06-10T01:10:56.537 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-06-10T01:10:56.538 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.538 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.538 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-06-10T01:10:56.538 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.538 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.538 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.538 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-10T01:10:56.538 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.538 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.539 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.539 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.539 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.539 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.539 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.539 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm", 2024-06-10T01:10:56.539 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.539 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.540 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.540 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_down_multiplier": { 2024-06-10T01:10:56.540 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_down_multiplier", 2024-06-10T01:10:56.540 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-06-10T01:10:56.540 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.540 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.540 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-06-10T01:10:56.540 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.541 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.541 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.541 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-10T01:10:56.541 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.541 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.541 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.541 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.542 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_refresh_rate": { 2024-06-10T01:10:56.542 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_refresh_rate", 2024-06-10T01:10:56.542 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:56.542 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.542 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.542 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "20", 2024-06-10T01:10:56.542 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.542 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.543 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.543 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-10T01:10:56.543 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.543 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.543 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.543 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.543 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_starting_port": { 2024-06-10T01:10:56.543 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_starting_port", 2024-06-10T01:10:56.544 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.544 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.544 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.544 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4721", 2024-06-10T01:10:56.544 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.544 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.544 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.544 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-10T01:10:56.544 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.545 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.545 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.545 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.545 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_ptrace": { 2024-06-10T01:10:56.545 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_ptrace", 2024-06-10T01:10:56.545 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.545 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.545 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.545 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.546 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.546 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.546 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.546 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-10T01:10:56.546 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-10T01:10:56.546 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.546 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.546 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.547 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_interval": { 2024-06-10T01:10:56.547 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_interval", 2024-06-10T01:10:56.547 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:56.547 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.547 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.547 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-06-10T01:10:56.547 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.547 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.547 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.548 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-10T01:10:56.548 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.548 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.548 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.548 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.548 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_memory_target_ratio": { 2024-06-10T01:10:56.548 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_memory_target_ratio", 2024-06-10T01:10:56.548 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-06-10T01:10:56.549 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.549 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.549 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.7", 2024-06-10T01:10:56.549 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.549 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.549 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.549 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-10T01:10:56.549 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.550 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.550 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.550 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.550 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm_log_destination": { 2024-06-10T01:10:56.550 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm_log_destination", 2024-06-10T01:10:56.550 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.550 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.550 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.551 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.551 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.551 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.551 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.551 INFO:teuthology.orchestra.run.smithi037.stdout: "file", 2024-06-10T01:10:56.551 INFO:teuthology.orchestra.run.smithi037.stdout: "file,syslog", 2024-06-10T01:10:56.551 INFO:teuthology.orchestra.run.smithi037.stdout: "syslog" 2024-06-10T01:10:56.551 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.552 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-10T01:10:56.552 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.552 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.552 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.552 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.552 INFO:teuthology.orchestra.run.smithi037.stdout: "cgroups_split": { 2024-06-10T01:10:56.552 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cgroups_split", 2024-06-10T01:10:56.552 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.553 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.553 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.553 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.553 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.553 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.553 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.553 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-10T01:10:56.553 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.554 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.554 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.554 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.554 INFO:teuthology.orchestra.run.smithi037.stdout: "config_checks_enabled": { 2024-06-10T01:10:56.554 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_checks_enabled", 2024-06-10T01:10:56.554 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.554 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.554 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.555 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.555 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.555 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.555 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.555 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-10T01:10:56.555 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.555 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.555 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.556 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.556 INFO:teuthology.orchestra.run.smithi037.stdout: "config_dashboard": { 2024-06-10T01:10:56.556 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_dashboard", 2024-06-10T01:10:56.556 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.556 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.556 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.556 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.556 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.557 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.557 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.557 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-10T01:10:56.557 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.557 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.557 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.557 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.557 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_alertmanager": { 2024-06-10T01:10:56.557 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_alertmanager", 2024-06-10T01:10:56.558 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.558 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.558 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.558 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-10T01:10:56.558 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.558 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.558 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.558 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-06-10T01:10:56.559 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.559 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.559 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.559 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.559 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_base": { 2024-06-10T01:10:56.559 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_base", 2024-06-10T01:10:56.559 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.559 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.560 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.560 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-10T01:10:56.560 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.560 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.560 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.560 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Container image name, without the tag", 2024-06-10T01:10:56.560 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.560 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.561 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.561 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.561 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_elasticsearch": { 2024-06-10T01:10:56.561 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_elasticsearch", 2024-06-10T01:10:56.561 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.561 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.561 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.561 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-10T01:10:56.561 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.562 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.562 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.562 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "elasticsearch container image", 2024-06-10T01:10:56.562 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.562 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.562 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.562 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.562 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_grafana": { 2024-06-10T01:10:56.563 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_grafana", 2024-06-10T01:10:56.563 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.563 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.563 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.563 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-10T01:10:56.563 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.563 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.563 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.564 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-06-10T01:10:56.564 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.564 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.564 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.564 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.564 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_haproxy": { 2024-06-10T01:10:56.564 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_haproxy", 2024-06-10T01:10:56.564 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.564 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.565 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.565 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-10T01:10:56.565 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.565 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.565 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.565 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "HAproxy container image", 2024-06-10T01:10:56.565 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.565 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.566 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.566 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.566 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_agent": { 2024-06-10T01:10:56.566 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_agent", 2024-06-10T01:10:56.566 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.566 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.566 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.566 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-10T01:10:56.566 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.567 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.567 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.567 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger agent container image", 2024-06-10T01:10:56.567 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.567 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.567 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.567 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.567 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_collector": { 2024-06-10T01:10:56.568 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_collector", 2024-06-10T01:10:56.568 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.568 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.568 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.568 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-10T01:10:56.568 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.568 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.568 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.568 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger collector container image", 2024-06-10T01:10:56.569 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.569 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.569 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.569 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.569 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_query": { 2024-06-10T01:10:56.569 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_query", 2024-06-10T01:10:56.569 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.569 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.570 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.570 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-10T01:10:56.570 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.570 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.570 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.570 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger query container image", 2024-06-10T01:10:56.570 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.570 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.571 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.571 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_keepalived": { 2024-06-10T01:10:56.571 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_keepalived", 2024-06-10T01:10:56.571 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.571 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.571 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.571 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-10T01:10:56.572 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.572 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.572 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.572 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Keepalived container image", 2024-06-10T01:10:56.572 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.572 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.572 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.572 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.572 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_loki": { 2024-06-10T01:10:56.573 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_loki", 2024-06-10T01:10:56.573 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.573 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.573 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.573 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-10T01:10:56.573 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.573 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.573 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.574 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Loki container image", 2024-06-10T01:10:56.574 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.574 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.574 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.574 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.574 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_node_exporter": { 2024-06-10T01:10:56.574 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_node_exporter", 2024-06-10T01:10:56.574 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.575 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.575 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.575 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-10T01:10:56.575 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.575 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.575 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.575 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-06-10T01:10:56.575 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.576 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.576 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.576 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.576 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_nvmeof": { 2024-06-10T01:10:56.576 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_nvmeof", 2024-06-10T01:10:56.576 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.576 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.576 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.577 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-10T01:10:56.577 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.577 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.577 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.577 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Nvme-of container image", 2024-06-10T01:10:56.577 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.577 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.577 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.577 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.578 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_prometheus": { 2024-06-10T01:10:56.578 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_prometheus", 2024-06-10T01:10:56.578 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.578 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.578 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.578 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-10T01:10:56.578 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.578 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.579 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.579 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-06-10T01:10:56.579 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.579 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.579 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.579 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.579 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_promtail": { 2024-06-10T01:10:56.579 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_promtail", 2024-06-10T01:10:56.580 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.580 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.580 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.580 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-10T01:10:56.580 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.580 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.580 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.580 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Promtail container image", 2024-06-10T01:10:56.580 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.581 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.581 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.581 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.581 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_samba": { 2024-06-10T01:10:56.581 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_samba", 2024-06-10T01:10:56.581 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.581 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.581 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.582 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-10T01:10:56.582 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.582 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.582 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.582 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Samba/SMB container image", 2024-06-10T01:10:56.582 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.582 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.582 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.583 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.583 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_snmp_gateway": { 2024-06-10T01:10:56.583 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_snmp_gateway", 2024-06-10T01:10:56.583 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.583 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.583 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.583 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-10T01:10:56.583 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.583 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.584 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.584 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SNMP Gateway container image", 2024-06-10T01:10:56.584 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.584 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.584 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.584 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.584 INFO:teuthology.orchestra.run.smithi037.stdout: "container_init": { 2024-06-10T01:10:56.585 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_init", 2024-06-10T01:10:56.585 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.585 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.585 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.585 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.585 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.585 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.585 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.585 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Run podman/docker with `--init`", 2024-06-10T01:10:56.586 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.586 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.586 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.586 INFO:teuthology.orchestra.run.smithi037.stdout: "daemon_cache_timeout": { 2024-06-10T01:10:56.586 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "daemon_cache_timeout", 2024-06-10T01:10:56.586 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:56.586 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.587 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.587 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-06-10T01:10:56.587 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.587 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.587 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.587 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-10T01:10:56.587 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.587 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.588 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.588 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.588 INFO:teuthology.orchestra.run.smithi037.stdout: "default_cephadm_command_timeout": { 2024-06-10T01:10:56.588 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_cephadm_command_timeout", 2024-06-10T01:10:56.588 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.588 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.588 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.588 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "900", 2024-06-10T01:10:56.589 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.589 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.589 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.589 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-10T01:10:56.589 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.589 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.589 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.589 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.590 INFO:teuthology.orchestra.run.smithi037.stdout: "default_registry": { 2024-06-10T01:10:56.590 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_registry", 2024-06-10T01:10:56.590 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.590 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.590 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.590 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io", 2024-06-10T01:10:56.590 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.590 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.591 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.591 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-10T01:10:56.591 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.591 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.591 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.591 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.591 INFO:teuthology.orchestra.run.smithi037.stdout: "device_cache_timeout": { 2024-06-10T01:10:56.591 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_cache_timeout", 2024-06-10T01:10:56.592 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:56.592 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.592 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.592 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1800", 2024-06-10T01:10:56.592 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.592 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.592 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.592 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache device inventory", 2024-06-10T01:10:56.593 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.593 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.593 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.593 INFO:teuthology.orchestra.run.smithi037.stdout: "device_enhanced_scan": { 2024-06-10T01:10:56.593 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_enhanced_scan", 2024-06-10T01:10:56.593 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.593 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.593 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.594 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.594 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.594 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.594 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.594 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-10T01:10:56.594 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.594 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.595 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.595 INFO:teuthology.orchestra.run.smithi037.stdout: "facts_cache_timeout": { 2024-06-10T01:10:56.595 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "facts_cache_timeout", 2024-06-10T01:10:56.595 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:56.595 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.595 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.596 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-06-10T01:10:56.596 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.596 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.596 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.596 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache host facts data", 2024-06-10T01:10:56.596 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.596 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.596 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.597 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.597 INFO:teuthology.orchestra.run.smithi037.stdout: "grafana_dashboards_path": { 2024-06-10T01:10:56.597 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "grafana_dashboards_path", 2024-06-10T01:10:56.597 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.597 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.597 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.597 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-10T01:10:56.597 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.598 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.598 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.598 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-10T01:10:56.598 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.598 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.598 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.598 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.598 INFO:teuthology.orchestra.run.smithi037.stdout: "host_check_interval": { 2024-06-10T01:10:56.599 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "host_check_interval", 2024-06-10T01:10:56.599 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:56.599 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.599 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.599 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-06-10T01:10:56.599 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.599 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.599 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.600 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to perform a host check", 2024-06-10T01:10:56.600 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.600 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.600 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.600 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.600 INFO:teuthology.orchestra.run.smithi037.stdout: "hw_monitoring": { 2024-06-10T01:10:56.600 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hw_monitoring", 2024-06-10T01:10:56.600 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.600 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.601 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.601 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.601 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.601 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.601 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.601 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-10T01:10:56.601 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.601 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.601 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.602 INFO:teuthology.orchestra.run.smithi037.stdout: "inventory_list_all": { 2024-06-10T01:10:56.602 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "inventory_list_all", 2024-06-10T01:10:56.602 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.602 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.602 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.602 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.603 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.603 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.603 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.603 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-10T01:10:56.603 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.603 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.603 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.603 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.604 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.604 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.604 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.604 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.604 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.604 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.604 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.604 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.604 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.604 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.605 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.605 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.605 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.605 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.605 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.605 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.605 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.605 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.606 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.606 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.606 INFO:teuthology.orchestra.run.smithi037.stdout: "log_refresh_metadata": { 2024-06-10T01:10:56.606 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_refresh_metadata", 2024-06-10T01:10:56.606 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.606 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.606 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.606 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.607 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.607 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.607 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.607 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-10T01:10:56.607 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.607 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.607 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.607 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.608 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.608 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.608 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.608 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.608 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.608 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.608 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.608 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.608 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-10T01:10:56.609 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.609 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.609 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.609 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.609 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.609 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.609 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.610 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.610 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.610 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.610 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.610 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.610 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.610 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.610 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.611 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.611 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.611 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.611 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.611 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.611 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.611 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.611 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.612 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.612 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.612 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.612 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.612 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.612 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.612 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.613 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.613 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.613 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.613 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.613 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.613 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.613 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.613 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-10T01:10:56.614 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-10T01:10:56.614 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.614 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.614 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.614 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.614 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.614 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.614 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.614 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-10T01:10:56.615 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.615 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.615 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.615 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-10T01:10:56.615 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-10T01:10:56.615 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.615 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.616 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.616 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "*", 2024-06-10T01:10:56.616 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.616 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.616 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.616 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-10T01:10:56.616 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.616 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.617 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.617 INFO:teuthology.orchestra.run.smithi037.stdout: "max_count_per_host": { 2024-06-10T01:10:56.617 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_count_per_host", 2024-06-10T01:10:56.617 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.617 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.617 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.618 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-06-10T01:10:56.618 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.618 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.618 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.618 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of daemons per service per host", 2024-06-10T01:10:56.618 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.618 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.618 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.619 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.619 INFO:teuthology.orchestra.run.smithi037.stdout: "max_osd_draining_count": { 2024-06-10T01:10:56.619 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_osd_draining_count", 2024-06-10T01:10:56.619 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.619 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.619 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.619 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-06-10T01:10:56.619 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.620 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.620 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.620 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-10T01:10:56.620 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.620 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.620 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.620 INFO:teuthology.orchestra.run.smithi037.stdout: "migration_current": { 2024-06-10T01:10:56.621 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "migration_current", 2024-06-10T01:10:56.621 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.621 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.621 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.621 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.621 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.621 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.621 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.622 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "internal - do not modify", 2024-06-10T01:10:56.622 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.622 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.622 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.622 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-06-10T01:10:56.622 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-06-10T01:10:56.622 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.623 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.623 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.623 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "root", 2024-06-10T01:10:56.623 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.623 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.623 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.623 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm-package", 2024-06-10T01:10:56.623 INFO:teuthology.orchestra.run.smithi037.stdout: "root" 2024-06-10T01:10:56.623 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.624 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "mode for remote execution of cephadm", 2024-06-10T01:10:56.624 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.624 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.624 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.624 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.624 INFO:teuthology.orchestra.run.smithi037.stdout: "oob_default_addr": { 2024-06-10T01:10:56.624 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "oob_default_addr", 2024-06-10T01:10:56.624 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.625 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.625 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.625 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "169.254.1.1", 2024-06-10T01:10:56.625 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.625 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.625 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.625 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-10T01:10:56.625 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.626 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.626 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.626 INFO:teuthology.orchestra.run.smithi037.stdout: "prometheus_alerts_path": { 2024-06-10T01:10:56.626 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus_alerts_path", 2024-06-10T01:10:56.626 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.626 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.626 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.627 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-10T01:10:56.627 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.627 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.627 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.627 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-10T01:10:56.627 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.627 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.627 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.627 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.628 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_insecure": { 2024-06-10T01:10:56.628 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_insecure", 2024-06-10T01:10:56.628 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.628 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.628 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.628 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.628 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.628 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.628 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.629 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-10T01:10:56.629 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.629 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.629 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.629 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_password": { 2024-06-10T01:10:56.629 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_password", 2024-06-10T01:10:56.629 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.629 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.630 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.630 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.630 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.630 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.630 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.630 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-10T01:10:56.630 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.630 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.631 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.631 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_url": { 2024-06-10T01:10:56.631 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_url", 2024-06-10T01:10:56.631 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.631 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.631 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.631 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.631 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.632 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.632 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.632 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-10T01:10:56.632 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.632 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.632 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.632 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_username": { 2024-06-10T01:10:56.632 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_username", 2024-06-10T01:10:56.633 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.633 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.633 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.633 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.633 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.633 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.633 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.633 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-10T01:10:56.634 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.634 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.634 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.634 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.634 INFO:teuthology.orchestra.run.smithi037.stdout: "secure_monitoring_stack": { 2024-06-10T01:10:56.634 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secure_monitoring_stack", 2024-06-10T01:10:56.634 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.634 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.634 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.635 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.635 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.635 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.635 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.635 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-10T01:10:56.635 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.635 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.635 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.636 INFO:teuthology.orchestra.run.smithi037.stdout: "service_discovery_port": { 2024-06-10T01:10:56.636 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "service_discovery_port", 2024-06-10T01:10:56.636 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.636 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.636 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.636 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8765", 2024-06-10T01:10:56.636 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.637 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.637 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.637 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "cephadm service discovery port", 2024-06-10T01:10:56.637 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.637 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.637 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.637 INFO:teuthology.orchestra.run.smithi037.stdout: "ssh_config_file": { 2024-06-10T01:10:56.637 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssh_config_file", 2024-06-10T01:10:56.638 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.638 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.638 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.638 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.638 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.638 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.638 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.638 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-10T01:10:56.638 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.639 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.639 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.639 INFO:teuthology.orchestra.run.smithi037.stdout: "use_agent": { 2024-06-10T01:10:56.639 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_agent", 2024-06-10T01:10:56.639 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.639 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.639 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.640 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.640 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.640 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.640 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.640 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-10T01:10:56.640 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.640 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.640 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.641 INFO:teuthology.orchestra.run.smithi037.stdout: "use_repo_digest": { 2024-06-10T01:10:56.641 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_repo_digest", 2024-06-10T01:10:56.641 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.641 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.641 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.641 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.642 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.642 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.642 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.642 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-10T01:10:56.642 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.642 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.642 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.643 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_failed_host_check": { 2024-06-10T01:10:56.643 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_failed_host_check", 2024-06-10T01:10:56.643 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.643 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.643 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.643 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.643 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.643 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.644 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.644 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if the host check fails", 2024-06-10T01:10:56.644 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.644 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.644 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.644 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_daemons": { 2024-06-10T01:10:56.644 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_daemons", 2024-06-10T01:10:56.645 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.645 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.645 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.645 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.645 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.645 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.645 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.645 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-10T01:10:56.646 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.646 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.646 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.646 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_hosts": { 2024-06-10T01:10:56.646 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_hosts", 2024-06-10T01:10:56.646 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.646 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.647 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.647 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.647 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.647 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.647 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.647 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-10T01:10:56.647 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.647 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.647 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.648 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.648 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.648 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.648 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crash", 2024-06-10T01:10:56.648 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.648 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.648 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.648 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.649 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.649 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.649 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.649 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.649 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.649 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.649 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.649 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.650 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.650 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.650 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.650 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.650 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.650 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.650 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.650 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.650 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.651 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.651 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.651 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.651 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.651 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.651 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.651 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.652 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.652 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.652 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.652 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.652 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.652 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.652 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.652 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.653 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.653 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.653 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.653 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.653 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.653 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.653 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.654 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.654 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.654 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.654 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.654 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.654 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.654 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.654 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.655 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.655 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.655 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.655 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.655 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.655 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.655 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.656 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.656 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.656 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.656 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.656 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.656 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.656 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.656 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.657 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.657 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.657 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.657 INFO:teuthology.orchestra.run.smithi037.stdout: "retain_interval": { 2024-06-10T01:10:56.657 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retain_interval", 2024-06-10T01:10:56.657 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:56.657 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.658 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.658 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "31536000", 2024-06-10T01:10:56.658 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.658 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.658 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.658 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-10T01:10:56.658 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.658 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.658 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.659 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_recent_interval": { 2024-06-10T01:10:56.659 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_recent_interval", 2024-06-10T01:10:56.659 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:56.659 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.659 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.659 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1209600", 2024-06-10T01:10:56.660 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.660 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.660 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.660 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-10T01:10:56.660 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.660 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.660 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.660 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.660 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.661 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.661 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dashboard", 2024-06-10T01:10:56.661 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.661 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.661 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.661 INFO:teuthology.orchestra.run.smithi037.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-10T01:10:56.661 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-10T01:10:56.661 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.662 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.662 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.662 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-06-10T01:10:56.662 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.662 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.662 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.662 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.663 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.663 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.663 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.663 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_HOST": { 2024-06-10T01:10:56.663 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-10T01:10:56.663 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.663 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.664 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.664 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.664 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.664 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.664 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.664 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.664 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.664 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.665 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.665 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-10T01:10:56.665 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-10T01:10:56.665 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.665 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.665 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.665 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.666 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.666 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.666 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.666 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.666 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.666 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.666 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.667 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_ENABLED": { 2024-06-10T01:10:56.667 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_ENABLED", 2024-06-10T01:10:56.667 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.667 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.667 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.667 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.667 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.667 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.668 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.668 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.668 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.668 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.668 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.668 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-10T01:10:56.668 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-10T01:10:56.669 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.669 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.669 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.669 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.669 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.669 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.669 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.669 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.669 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.670 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.670 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.670 INFO:teuthology.orchestra.run.smithi037.stdout: "ENABLE_BROWSABLE_API": { 2024-06-10T01:10:56.670 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-10T01:10:56.670 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.670 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.670 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.671 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.671 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.671 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.671 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.671 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.671 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.671 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.671 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.672 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-10T01:10:56.672 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-10T01:10:56.672 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.672 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.672 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.672 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.672 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.672 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.672 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.673 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.673 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.673 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.673 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.673 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-10T01:10:56.673 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-10T01:10:56.673 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.673 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.674 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.674 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.674 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.674 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.674 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.674 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.674 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.674 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.674 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.675 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-10T01:10:56.675 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-10T01:10:56.675 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.675 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.675 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.675 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.675 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.676 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.676 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.676 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.676 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.676 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.676 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.676 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-10T01:10:56.676 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-10T01:10:56.677 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.677 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.677 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.677 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.677 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.677 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.677 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.677 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.677 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.678 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.678 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.678 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-10T01:10:56.678 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-10T01:10:56.678 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.678 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.678 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.678 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.679 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.679 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.679 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.679 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.679 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.679 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.679 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.679 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-10T01:10:56.680 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-10T01:10:56.680 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.680 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.680 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.680 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.680 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.680 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.680 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.680 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.681 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.681 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.681 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.681 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-10T01:10:56.681 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-10T01:10:56.681 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.681 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.682 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.682 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.682 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.682 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.682 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.682 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.682 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.683 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.683 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.683 INFO:teuthology.orchestra.run.smithi037.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-10T01:10:56.683 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-10T01:10:56.683 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.683 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.683 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.683 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.684 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.684 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.684 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.684 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.684 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.684 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.684 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.685 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_PASSWORD": { 2024-06-10T01:10:56.685 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-10T01:10:56.685 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.685 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.685 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.685 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-06-10T01:10:56.685 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.685 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.686 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.686 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.686 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.686 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.686 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.686 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-10T01:10:56.686 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-10T01:10:56.687 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.687 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.687 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.687 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.687 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.687 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.687 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.687 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.687 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.688 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.688 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.688 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_URL": { 2024-06-10T01:10:56.688 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_URL", 2024-06-10T01:10:56.688 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.688 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.688 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.688 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.689 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.689 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.689 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.689 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.689 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.689 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.689 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.690 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_USERNAME": { 2024-06-10T01:10:56.690 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-10T01:10:56.690 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.690 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.690 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.690 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-06-10T01:10:56.690 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.691 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.691 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.691 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.691 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.691 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.691 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.691 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-10T01:10:56.692 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-10T01:10:56.692 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.692 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.692 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.692 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.692 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.692 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.692 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.693 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.693 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.693 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.693 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.693 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-10T01:10:56.693 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-10T01:10:56.693 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.694 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.694 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.694 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.694 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.694 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.694 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.694 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.694 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.695 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.695 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.695 INFO:teuthology.orchestra.run.smithi037.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-10T01:10:56.695 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-10T01:10:56.695 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.695 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.695 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.696 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.696 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.696 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.696 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.696 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.696 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.696 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.696 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.697 INFO:teuthology.orchestra.run.smithi037.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-10T01:10:56.697 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-10T01:10:56.697 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.697 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.697 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.697 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.697 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.698 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.698 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.698 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.698 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.698 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.698 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.699 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_HOST": { 2024-06-10T01:10:56.699 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-10T01:10:56.699 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.699 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.699 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.700 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.700 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.700 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.700 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.700 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.700 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.700 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.700 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.701 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-10T01:10:56.701 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-10T01:10:56.701 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.701 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.701 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.701 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.702 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.702 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.702 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.702 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.702 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.702 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.702 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.703 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-10T01:10:56.703 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-10T01:10:56.703 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.703 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.703 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.703 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.703 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.703 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.703 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.704 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.704 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.704 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.704 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.704 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-10T01:10:56.704 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-10T01:10:56.704 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.705 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.705 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.705 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.705 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.705 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.705 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.705 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.705 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.706 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.706 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.706 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-10T01:10:56.706 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-10T01:10:56.706 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.706 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.707 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.707 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.707 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.707 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.707 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.707 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.707 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.707 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.708 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.708 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-10T01:10:56.708 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-10T01:10:56.708 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.708 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.708 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.708 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.708 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.709 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.709 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.709 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.709 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.709 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.709 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.709 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-10T01:10:56.710 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-10T01:10:56.710 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.710 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.710 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.710 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.710 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.710 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.710 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.711 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.711 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.711 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.711 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.711 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-10T01:10:56.711 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-10T01:10:56.711 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.711 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.712 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.712 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.712 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.712 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.712 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.712 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.712 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.712 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.712 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.713 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-10T01:10:56.713 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-10T01:10:56.713 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.713 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.713 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.713 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.713 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.714 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.714 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.714 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.714 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.714 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.714 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.715 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_ENABLED": { 2024-06-10T01:10:56.715 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-10T01:10:56.715 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.715 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.715 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.715 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.715 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.715 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.715 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.716 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.716 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.716 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.716 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.716 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-10T01:10:56.716 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-10T01:10:56.716 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.717 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.717 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.717 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-10T01:10:56.717 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.717 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.717 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.717 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.717 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.718 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.718 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.718 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-10T01:10:56.718 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-10T01:10:56.718 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.718 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.718 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.719 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-06-10T01:10:56.719 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.719 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.719 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.719 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.719 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.719 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.720 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.720 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-10T01:10:56.720 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-10T01:10:56.720 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.720 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.720 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.720 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8", 2024-06-10T01:10:56.720 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.721 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.721 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.721 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.721 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.721 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.721 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.721 INFO:teuthology.orchestra.run.smithi037.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-10T01:10:56.722 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-10T01:10:56.722 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.722 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.722 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.722 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "45", 2024-06-10T01:10:56.722 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.722 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.722 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.723 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.723 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.723 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.723 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.723 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ACCESS_KEY": { 2024-06-10T01:10:56.723 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-10T01:10:56.723 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.723 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.724 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.724 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.724 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.724 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.724 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.724 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.724 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.724 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.725 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.725 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-10T01:10:56.725 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-10T01:10:56.725 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.725 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.725 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.725 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-06-10T01:10:56.726 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.726 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.726 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.726 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.726 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.726 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.726 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.727 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SECRET_KEY": { 2024-06-10T01:10:56.727 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-10T01:10:56.727 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.727 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.727 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.727 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.727 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.727 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.728 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.728 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.728 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.728 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.728 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.728 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SSL_VERIFY": { 2024-06-10T01:10:56.728 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-10T01:10:56.728 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.729 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.729 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.729 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.729 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.729 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.729 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.729 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.729 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.730 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.730 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.730 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-10T01:10:56.730 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-10T01:10:56.730 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.730 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.730 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.731 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-06-10T01:10:56.731 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.731 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.731 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.731 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.731 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.731 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.731 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.732 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-10T01:10:56.732 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-10T01:10:56.732 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.732 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.732 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.732 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-06-10T01:10:56.732 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.732 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.733 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.733 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.733 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.733 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.733 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.733 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-10T01:10:56.733 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-10T01:10:56.733 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.734 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.734 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.734 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-06-10T01:10:56.734 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.734 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.734 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.734 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.734 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.735 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.735 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.735 INFO:teuthology.orchestra.run.smithi037.stdout: "cross_origin_url": { 2024-06-10T01:10:56.735 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cross_origin_url", 2024-06-10T01:10:56.735 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.735 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.735 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.736 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.736 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.736 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.736 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.736 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.736 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.736 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.736 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.737 INFO:teuthology.orchestra.run.smithi037.stdout: "crt_file": { 2024-06-10T01:10:56.737 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crt_file", 2024-06-10T01:10:56.737 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.737 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.737 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.737 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.737 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.737 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.738 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.738 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.738 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.738 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.738 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.738 INFO:teuthology.orchestra.run.smithi037.stdout: "debug": { 2024-06-10T01:10:56.738 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "debug", 2024-06-10T01:10:56.738 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.739 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.739 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.739 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.739 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.739 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.739 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.739 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable/disable debug options", 2024-06-10T01:10:56.739 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.740 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.740 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.740 INFO:teuthology.orchestra.run.smithi037.stdout: "jwt_token_ttl": { 2024-06-10T01:10:56.740 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "jwt_token_ttl", 2024-06-10T01:10:56.740 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.740 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.740 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.741 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "28800", 2024-06-10T01:10:56.741 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.741 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.741 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.741 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.741 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.741 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.741 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.742 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-06-10T01:10:56.742 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-06-10T01:10:56.742 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.742 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.742 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.742 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.742 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.743 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.743 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.743 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.743 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.743 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.743 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.743 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.743 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.743 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.744 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.744 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.744 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.744 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.744 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.744 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.744 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.744 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.745 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.745 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.745 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.745 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.745 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.745 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.745 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.745 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.746 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.746 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.746 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.746 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.746 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.746 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.746 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.747 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.747 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.747 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.747 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.747 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.747 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.747 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.747 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.748 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.748 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.748 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.748 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.748 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.748 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.748 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.748 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.749 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.749 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.749 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.749 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.749 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.749 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.749 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.749 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.749 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.750 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.750 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.750 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.750 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.750 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.750 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.751 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.751 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.751 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.751 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.751 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.751 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.751 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.751 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.752 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.752 INFO:teuthology.orchestra.run.smithi037.stdout: "motd": { 2024-06-10T01:10:56.752 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "motd", 2024-06-10T01:10:56.752 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.752 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.752 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.752 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.752 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.753 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.753 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.753 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "The message of the day", 2024-06-10T01:10:56.753 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.753 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.753 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.753 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect_resolve_ip_addr": { 2024-06-10T01:10:56.754 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "redirect_resolve_ip_addr", 2024-06-10T01:10:56.754 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.754 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.754 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.754 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.754 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.754 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.754 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.755 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.755 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.755 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.755 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.755 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-06-10T01:10:56.755 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-06-10T01:10:56.755 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.755 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.756 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.756 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-06-10T01:10:56.756 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.756 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.756 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.756 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.756 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.756 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.757 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.757 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-06-10T01:10:56.757 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-06-10T01:10:56.757 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.757 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.757 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.757 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8080", 2024-06-10T01:10:56.757 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.758 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.758 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.758 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.758 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.758 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.758 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.758 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-06-10T01:10:56.759 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-06-10T01:10:56.759 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.759 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.759 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.759 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.759 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.759 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.759 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.760 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.760 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.760 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.760 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.760 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl_server_port": { 2024-06-10T01:10:56.760 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl_server_port", 2024-06-10T01:10:56.760 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.760 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.761 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.761 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8443", 2024-06-10T01:10:56.761 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.761 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.761 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.761 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.761 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.761 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.762 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.762 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-06-10T01:10:56.762 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-06-10T01:10:56.762 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.762 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.762 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.762 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "redirect", 2024-06-10T01:10:56.763 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.763 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.763 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.763 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.763 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect" 2024-06-10T01:10:56.763 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.763 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.763 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.763 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.764 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.764 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-06-10T01:10:56.764 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-06-10T01:10:56.764 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.764 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.764 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.764 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-06-10T01:10:56.765 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-06-10T01:10:56.765 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-06-10T01:10:56.765 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.765 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.765 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.765 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.765 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.766 INFO:teuthology.orchestra.run.smithi037.stdout: "url_prefix": { 2024-06-10T01:10:56.766 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url_prefix", 2024-06-10T01:10:56.766 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.766 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.766 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.766 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.766 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.766 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.767 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.767 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.767 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.767 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.767 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.767 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.767 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.768 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.768 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-06-10T01:10:56.768 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.768 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.768 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.768 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_monitoring": { 2024-06-10T01:10:56.768 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_monitoring", 2024-06-10T01:10:56.768 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.768 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.769 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.769 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.769 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.769 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.769 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.769 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "monitor device health metrics", 2024-06-10T01:10:56.769 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.769 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.770 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.770 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.770 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.770 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.770 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.770 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.770 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.771 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.771 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.771 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.771 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.771 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.771 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.771 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.771 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.771 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.772 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.772 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.772 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.772 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.772 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.772 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.772 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.772 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.773 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.773 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.773 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.773 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.773 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.773 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.773 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.773 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.774 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.774 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.774 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.774 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.774 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.774 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.774 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.775 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.775 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.775 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.775 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.775 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.775 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.775 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.775 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.775 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.776 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.776 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.776 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.776 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.776 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.776 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.776 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.776 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.777 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.777 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.777 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.777 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.777 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.777 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.777 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.777 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.778 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.778 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.778 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.778 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.778 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.778 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.778 INFO:teuthology.orchestra.run.smithi037.stdout: "mark_out_threshold": { 2024-06-10T01:10:56.778 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mark_out_threshold", 2024-06-10T01:10:56.779 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:56.779 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.779 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.779 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2419200", 2024-06-10T01:10:56.779 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.779 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.779 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.779 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-10T01:10:56.780 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.780 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.780 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.780 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_name": { 2024-06-10T01:10:56.780 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_name", 2024-06-10T01:10:56.780 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.780 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.781 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.781 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "device_health_metrics", 2024-06-10T01:10:56.781 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.781 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.781 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.781 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-10T01:10:56.781 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.781 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.781 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.782 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.782 INFO:teuthology.orchestra.run.smithi037.stdout: "retention_period": { 2024-06-10T01:10:56.782 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retention_period", 2024-06-10T01:10:56.782 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:56.782 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.782 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.782 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15552000", 2024-06-10T01:10:56.783 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.783 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.783 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.783 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain device health metrics", 2024-06-10T01:10:56.783 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.783 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.783 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.783 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_frequency": { 2024-06-10T01:10:56.784 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_frequency", 2024-06-10T01:10:56.784 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:56.784 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.784 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.784 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-06-10T01:10:56.784 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.784 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.785 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.785 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-10T01:10:56.785 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.785 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.785 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.785 INFO:teuthology.orchestra.run.smithi037.stdout: "self_heal": { 2024-06-10T01:10:56.785 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "self_heal", 2024-06-10T01:10:56.785 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.786 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.786 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.786 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.786 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.786 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.786 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.786 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-10T01:10:56.786 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.787 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.787 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.787 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.787 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-06-10T01:10:56.787 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-06-10T01:10:56.787 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:56.787 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.788 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.788 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-06-10T01:10:56.788 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.788 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.788 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.788 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and check device health", 2024-06-10T01:10:56.788 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.788 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.788 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.789 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_threshold": { 2024-06-10T01:10:56.789 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_threshold", 2024-06-10T01:10:56.789 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:56.789 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.789 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.789 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7257600", 2024-06-10T01:10:56.789 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.790 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.790 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.790 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-10T01:10:56.790 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.790 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.790 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.790 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.790 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.791 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.791 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.791 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "diskprediction_local", 2024-06-10T01:10:56.791 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.791 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.791 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.791 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.791 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.792 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.792 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.792 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.792 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.792 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.792 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.792 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.792 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.793 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.793 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.793 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.793 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.793 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.793 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.793 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.793 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.794 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.794 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.794 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.794 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.794 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.794 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.794 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.795 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.795 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.795 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.795 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.795 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.795 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.795 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.795 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.796 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.796 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.796 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.796 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.796 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.796 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.796 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.796 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.797 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.797 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.797 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.797 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.797 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.797 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.797 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.797 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.798 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.798 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.798 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.798 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.798 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.798 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.798 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.799 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.799 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.799 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.799 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.799 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.799 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.799 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.799 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.800 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.800 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.800 INFO:teuthology.orchestra.run.smithi037.stdout: "predict_interval": { 2024-06-10T01:10:56.800 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predict_interval", 2024-06-10T01:10:56.800 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.800 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.800 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.801 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-06-10T01:10:56.801 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.801 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.801 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.801 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.801 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.802 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.802 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.802 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.802 INFO:teuthology.orchestra.run.smithi037.stdout: "predictor_model": { 2024-06-10T01:10:56.802 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predictor_model", 2024-06-10T01:10:56.802 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.802 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.802 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.802 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "prophetstor", 2024-06-10T01:10:56.803 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.803 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.803 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.803 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.803 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.803 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.803 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.803 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-06-10T01:10:56.804 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-06-10T01:10:56.804 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.804 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.804 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.804 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-06-10T01:10:56.804 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.804 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.804 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.805 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.805 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.805 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.805 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.805 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.805 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.805 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.806 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "influx", 2024-06-10T01:10:56.806 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": false, 2024-06-10T01:10:56.806 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "influxdb python module not found", 2024-06-10T01:10:56.806 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.806 INFO:teuthology.orchestra.run.smithi037.stdout: "batch_size": { 2024-06-10T01:10:56.806 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "batch_size", 2024-06-10T01:10:56.806 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.806 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.807 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.807 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5000", 2024-06-10T01:10:56.807 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.807 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.807 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.807 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-10T01:10:56.807 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.807 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.808 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.808 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.808 INFO:teuthology.orchestra.run.smithi037.stdout: "database": { 2024-06-10T01:10:56.808 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "database", 2024-06-10T01:10:56.808 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.808 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.808 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.808 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "ceph", 2024-06-10T01:10:56.809 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.809 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.809 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.809 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-10T01:10:56.809 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.809 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.809 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.809 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.810 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": { 2024-06-10T01:10:56.810 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hostname", 2024-06-10T01:10:56.810 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.810 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.810 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.810 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.810 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.810 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.811 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.811 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server hostname", 2024-06-10T01:10:56.811 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.811 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.811 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.811 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.811 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-06-10T01:10:56.811 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-06-10T01:10:56.812 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:56.812 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.812 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.812 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "30", 2024-06-10T01:10:56.812 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "5", 2024-06-10T01:10:56.812 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.812 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.812 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-10T01:10:56.813 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.813 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.813 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.813 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.813 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.813 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.813 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.813 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.814 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.814 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.814 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.814 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.814 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.814 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.814 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.814 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.815 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.815 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.815 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.815 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.815 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.815 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.815 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.815 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.815 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.816 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.816 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.816 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.816 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.816 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.816 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.816 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.816 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.817 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.817 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.817 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.817 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.817 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.817 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.817 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.817 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.818 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.818 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.818 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.818 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.818 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.818 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.818 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.818 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.819 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.819 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.819 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.819 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.819 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.819 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.819 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.819 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.820 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.820 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.820 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.820 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.820 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.820 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.820 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.820 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.821 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.821 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.821 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.821 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.821 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.821 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.821 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.821 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.822 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.822 INFO:teuthology.orchestra.run.smithi037.stdout: "password": { 2024-06-10T01:10:56.822 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "password", 2024-06-10T01:10:56.822 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.822 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.822 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.822 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.822 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.823 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.823 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.823 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "password of InfluxDB server user", 2024-06-10T01:10:56.823 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.823 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.823 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.823 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.823 INFO:teuthology.orchestra.run.smithi037.stdout: "port": { 2024-06-10T01:10:56.823 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "port", 2024-06-10T01:10:56.824 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.824 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.824 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.824 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8086", 2024-06-10T01:10:56.824 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.824 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.824 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.824 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server port", 2024-06-10T01:10:56.824 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.825 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.825 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.825 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-06-10T01:10:56.825 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-06-10T01:10:56.825 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.825 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.825 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.826 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "false", 2024-06-10T01:10:56.826 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.826 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.826 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.826 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-10T01:10:56.826 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.826 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.826 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.827 INFO:teuthology.orchestra.run.smithi037.stdout: "threads": { 2024-06-10T01:10:56.827 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threads", 2024-06-10T01:10:56.827 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.827 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.827 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.827 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-06-10T01:10:56.827 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-06-10T01:10:56.828 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "32", 2024-06-10T01:10:56.828 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.828 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-10T01:10:56.828 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.828 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.828 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.828 INFO:teuthology.orchestra.run.smithi037.stdout: "username": { 2024-06-10T01:10:56.829 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "username", 2024-06-10T01:10:56.829 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.829 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.829 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.829 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.829 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.829 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.829 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.830 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "username of InfluxDB server user", 2024-06-10T01:10:56.830 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.830 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.830 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.830 INFO:teuthology.orchestra.run.smithi037.stdout: "verify_ssl": { 2024-06-10T01:10:56.830 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "verify_ssl", 2024-06-10T01:10:56.830 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.831 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.831 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.831 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "true", 2024-06-10T01:10:56.831 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.831 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.831 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.831 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-10T01:10:56.831 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.831 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.832 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.832 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.832 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.832 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.832 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "insights", 2024-06-10T01:10:56.832 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.832 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.833 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.833 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.833 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.833 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.833 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.833 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.833 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.833 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.834 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.834 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.834 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.834 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.834 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.834 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.834 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.834 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.835 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.835 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.835 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.835 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.835 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.835 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.835 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.835 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.836 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.836 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.836 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.836 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.836 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.836 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.836 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.836 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.837 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.837 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.837 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.837 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.837 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.837 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.837 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.838 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.838 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.838 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.838 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.838 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.838 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.838 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.838 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.839 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.839 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.839 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.839 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.839 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.839 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.839 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.839 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.840 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.840 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.840 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.840 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.840 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.840 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.840 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.841 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.841 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.841 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.841 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.841 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.841 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.841 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.841 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.842 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.842 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "iostat", 2024-06-10T01:10:56.842 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.842 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.842 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.842 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.842 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.842 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.843 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.843 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.843 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.843 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.843 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.843 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.843 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.843 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.844 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.844 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.844 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.844 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.844 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.844 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.844 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.844 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.844 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.845 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.845 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.845 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.845 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.845 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.845 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.845 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.846 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.846 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.846 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.846 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.846 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.846 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.847 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.847 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.847 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.847 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.847 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.847 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.847 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.847 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.848 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.848 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.848 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.848 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.848 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.848 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.848 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.848 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.849 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.849 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.849 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.849 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.849 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.849 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.849 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.849 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.850 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.850 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.850 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.850 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.850 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.850 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.850 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.850 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.851 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.851 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.851 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.851 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.851 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.851 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "k8sevents", 2024-06-10T01:10:56.851 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.852 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.852 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.852 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_event_retention_days": { 2024-06-10T01:10:56.852 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ceph_event_retention_days", 2024-06-10T01:10:56.852 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.852 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.852 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.852 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7", 2024-06-10T01:10:56.853 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.853 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.853 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.853 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-10T01:10:56.853 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.853 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.853 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.853 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.854 INFO:teuthology.orchestra.run.smithi037.stdout: "config_check_secs": { 2024-06-10T01:10:56.854 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_check_secs", 2024-06-10T01:10:56.854 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.854 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.854 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.854 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-06-10T01:10:56.854 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "10", 2024-06-10T01:10:56.854 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.855 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.855 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-10T01:10:56.855 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.855 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.855 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.855 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.855 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.856 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.856 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.856 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.856 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.856 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.856 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.856 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.856 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.857 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.857 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.857 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.857 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.857 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.857 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.857 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.857 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.858 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.858 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.858 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.858 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.858 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.858 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.858 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.859 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.859 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.859 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.859 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.859 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.859 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.859 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.859 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.860 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.860 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.860 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.860 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.860 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.860 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.860 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.860 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.860 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.861 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.861 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.861 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.861 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.861 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.861 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.861 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.861 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.862 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.862 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.862 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.862 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.862 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.862 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.862 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.862 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.863 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.863 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.863 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.863 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.863 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.863 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.863 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.863 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.864 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.864 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.864 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.864 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.864 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.864 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.864 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "localpool", 2024-06-10T01:10:56.864 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.865 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.865 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.865 INFO:teuthology.orchestra.run.smithi037.stdout: "failure_domain": { 2024-06-10T01:10:56.865 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "failure_domain", 2024-06-10T01:10:56.865 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.865 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.865 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.865 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "host", 2024-06-10T01:10:56.866 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.866 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.866 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.866 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "failure domain for any created local pool", 2024-06-10T01:10:56.866 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-10T01:10:56.866 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.866 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.866 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.867 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.867 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.867 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.867 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.867 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.867 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.867 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.867 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.868 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.868 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.868 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.868 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.868 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.868 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.868 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.868 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.868 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.869 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.869 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.869 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.869 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.869 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.869 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.869 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.869 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.870 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.870 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.870 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.870 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.870 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.870 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.870 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.870 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.871 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.871 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.871 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.871 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.871 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.871 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.871 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.871 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.872 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.872 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.872 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.872 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.872 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.872 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.872 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.872 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.873 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.873 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.873 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.873 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.873 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.873 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.873 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.874 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.874 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.874 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.874 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.874 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.874 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.874 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.874 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.875 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.875 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.875 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.875 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.875 INFO:teuthology.orchestra.run.smithi037.stdout: "min_size": { 2024-06-10T01:10:56.875 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_size", 2024-06-10T01:10:56.875 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.876 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.876 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.876 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.876 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.876 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.876 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.876 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default min_size for any created local pool", 2024-06-10T01:10:56.876 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-10T01:10:56.877 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.877 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.877 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.877 INFO:teuthology.orchestra.run.smithi037.stdout: "num_rep": { 2024-06-10T01:10:56.877 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "num_rep", 2024-06-10T01:10:56.877 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.877 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.877 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.877 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3", 2024-06-10T01:10:56.878 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.878 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.878 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.878 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default replica count for any created local pool", 2024-06-10T01:10:56.878 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.878 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.878 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.878 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.878 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_num": { 2024-06-10T01:10:56.879 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_num", 2024-06-10T01:10:56.879 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.879 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.879 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.879 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "128", 2024-06-10T01:10:56.879 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.879 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.879 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.880 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default pg_num for any created local pool", 2024-06-10T01:10:56.880 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.880 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.880 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.880 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.880 INFO:teuthology.orchestra.run.smithi037.stdout: "prefix": { 2024-06-10T01:10:56.880 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prefix", 2024-06-10T01:10:56.880 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.880 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.881 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.881 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.881 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.881 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.881 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.881 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name prefix for any created local pool", 2024-06-10T01:10:56.881 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.881 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.881 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.882 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.882 INFO:teuthology.orchestra.run.smithi037.stdout: "subtree": { 2024-06-10T01:10:56.882 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "subtree", 2024-06-10T01:10:56.882 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.882 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.882 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.882 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "rack", 2024-06-10T01:10:56.882 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.882 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.883 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.883 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-10T01:10:56.883 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-10T01:10:56.883 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.883 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.883 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.883 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.884 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.884 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.884 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mds_autoscaler", 2024-06-10T01:10:56.884 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.884 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.884 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.884 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.884 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.884 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.884 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.885 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.885 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.885 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.885 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.885 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.885 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.885 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.885 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.885 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.886 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.886 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.886 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.886 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.886 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.886 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.886 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.887 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.887 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.887 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.887 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.887 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.887 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.887 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.887 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.887 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.888 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.888 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.888 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.888 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.888 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.888 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.888 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.888 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.888 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.889 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.889 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.889 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.889 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.889 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.889 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.889 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.889 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.890 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.890 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.890 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.890 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.890 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.890 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.890 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.891 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.891 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.891 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.891 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.891 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.891 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.891 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.891 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.892 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.892 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.892 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.892 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.892 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.892 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.892 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.892 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.893 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.893 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirroring", 2024-06-10T01:10:56.893 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.893 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.893 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.893 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.893 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.894 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.894 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.894 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.894 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.894 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.894 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.894 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.894 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.895 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.895 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.895 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.895 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.895 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.895 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.895 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.895 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.896 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.896 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.896 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.896 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.896 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.896 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.896 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.897 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.897 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.897 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.897 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.897 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.897 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.897 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.897 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.898 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.898 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.898 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.898 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.898 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.898 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.898 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.898 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.899 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.899 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.899 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.899 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.899 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.899 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.899 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.899 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.900 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.900 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.900 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.900 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.900 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.900 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.900 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.900 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.901 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.901 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.901 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.901 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.901 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.901 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.901 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.901 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.902 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.902 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.902 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.902 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.902 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.902 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "nfs", 2024-06-10T01:10:56.903 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.903 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.903 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.903 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.903 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.903 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.903 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.903 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.903 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.904 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.904 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.904 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.904 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.904 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.904 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.904 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.905 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.905 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.905 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.905 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.905 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.905 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.905 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.905 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.906 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.906 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.906 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.906 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.906 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.906 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.906 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.906 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.907 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.907 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.907 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.907 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.907 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.907 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.907 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.908 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.908 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.908 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.908 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.908 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.908 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.908 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.908 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.908 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.909 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.909 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.909 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.909 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.909 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.909 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.909 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.909 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.910 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.910 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.910 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.910 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.910 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.910 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.910 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.910 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.911 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.911 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.911 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.911 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.911 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.911 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.911 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.912 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.912 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-06-10T01:10:56.912 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.912 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.912 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.912 INFO:teuthology.orchestra.run.smithi037.stdout: "fail_fs": { 2024-06-10T01:10:56.912 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "fail_fs", 2024-06-10T01:10:56.912 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.912 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.913 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.913 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.913 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.913 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.913 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.913 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-10T01:10:56.913 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.913 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.914 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.914 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.914 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.914 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.914 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.914 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.914 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.915 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.915 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.915 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.915 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.915 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.915 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.915 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.915 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.916 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.916 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.916 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.916 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.916 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.916 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.916 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.916 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.917 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.917 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.917 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.917 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.917 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.917 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.917 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.917 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.918 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.918 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.918 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.918 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.918 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.918 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.918 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.919 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.919 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.919 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.919 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.919 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.919 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.919 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.919 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.919 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.920 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.920 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.920 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.920 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.920 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.920 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.920 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.920 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.921 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.921 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.921 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.921 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.921 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.921 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.921 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.922 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.922 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.922 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.922 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.922 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.922 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.923 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator": { 2024-06-10T01:10:56.923 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-06-10T01:10:56.923 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.923 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.923 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.923 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.923 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.923 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.923 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.924 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-06-10T01:10:56.924 INFO:teuthology.orchestra.run.smithi037.stdout: "rook", 2024-06-10T01:10:56.924 INFO:teuthology.orchestra.run.smithi037.stdout: "test_orchestrator" 2024-06-10T01:10:56.924 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.924 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Orchestrator backend", 2024-06-10T01:10:56.924 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.924 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.924 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.925 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.925 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.925 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.925 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_perf_query", 2024-06-10T01:10:56.925 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.925 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.925 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.926 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.926 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.926 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.926 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.926 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.926 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.926 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.926 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.927 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.927 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.927 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.927 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.927 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.927 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.927 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.927 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.928 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.928 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.928 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.928 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.928 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.928 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.928 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.928 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.929 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.929 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.929 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.929 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.929 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.929 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.929 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.929 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.930 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.930 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.930 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.930 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.930 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.930 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.930 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.931 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.931 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.931 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.931 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.931 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.931 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.931 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.931 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.932 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.932 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.932 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.932 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.932 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.932 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.932 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.933 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.933 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.933 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.933 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.933 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.933 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.933 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.933 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.933 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.934 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.934 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.934 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.934 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.934 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.934 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.934 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.934 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_support", 2024-06-10T01:10:56.935 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.935 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.935 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.935 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.935 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.935 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.935 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.936 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.936 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.936 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.936 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.936 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.936 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.936 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.936 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.937 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.937 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.937 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.937 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.937 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.937 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.937 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.937 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.938 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.938 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.938 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.938 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.938 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.938 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.938 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.938 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.939 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.939 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.939 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.939 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.939 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.939 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.939 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.939 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.940 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.940 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.940 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.940 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.940 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.940 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.940 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.941 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.941 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.941 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.941 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.941 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.941 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.941 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.941 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.942 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.942 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.942 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.942 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.942 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.942 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.942 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.942 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.943 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.943 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.943 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.943 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.943 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.943 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.943 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.943 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.944 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.944 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.944 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.944 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_autoscaler", 2024-06-10T01:10:56.944 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.944 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.944 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.945 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.945 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.945 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.945 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.945 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.945 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.945 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.945 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.945 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.946 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.946 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.946 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.946 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.946 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.946 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.946 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.947 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.947 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.947 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.947 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.947 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.947 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.947 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.948 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.948 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.948 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.948 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.948 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.948 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.948 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.948 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.948 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.949 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.949 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.949 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.949 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.949 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.949 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.949 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.949 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.950 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.950 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.950 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.950 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.950 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.950 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.950 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.950 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.951 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.951 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.951 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.951 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.951 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.951 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.951 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.951 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.952 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.952 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.952 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.952 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.952 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.952 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.952 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.952 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.952 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.953 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.953 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.953 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.953 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.953 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-06-10T01:10:56.953 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-06-10T01:10:56.953 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:56.953 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.954 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.954 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-06-10T01:10:56.954 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.954 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.954 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.954 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.954 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.954 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.955 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.955 INFO:teuthology.orchestra.run.smithi037.stdout: "threshold": { 2024-06-10T01:10:56.955 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threshold", 2024-06-10T01:10:56.955 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-06-10T01:10:56.955 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.955 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.955 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-06-10T01:10:56.956 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1.0", 2024-06-10T01:10:56.956 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.956 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.956 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "scaling threshold", 2024-06-10T01:10:56.956 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-10T01:10:56.956 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.956 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.956 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.956 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.957 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.957 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "progress", 2024-06-10T01:10:56.957 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.957 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.957 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.957 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_pg_recovery_event": { 2024-06-10T01:10:56.957 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_pg_recovery_event", 2024-06-10T01:10:56.958 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.958 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.958 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.958 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.958 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.958 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.958 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.958 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-10T01:10:56.959 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.959 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.959 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.959 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-06-10T01:10:56.959 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-06-10T01:10:56.959 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.959 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.960 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.960 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.960 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.960 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.960 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.960 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.960 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.960 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.960 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.961 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.961 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.961 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.961 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.961 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.961 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.961 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.962 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.962 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.962 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.962 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.962 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.962 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.962 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.962 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.963 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.963 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.963 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.963 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.963 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.963 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.963 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.964 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.964 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.964 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.964 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.964 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.964 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.964 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.964 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.965 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.965 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.965 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.965 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.965 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.965 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.965 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.966 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.966 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.966 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.966 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.966 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.966 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.966 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.966 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.967 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.967 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.967 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.967 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.967 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.967 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.967 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.967 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.967 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.968 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.968 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.968 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.968 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.968 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.968 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.968 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.968 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.969 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.969 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.969 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.969 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.969 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.969 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.969 INFO:teuthology.orchestra.run.smithi037.stdout: "max_completed_events": { 2024-06-10T01:10:56.969 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_completed_events", 2024-06-10T01:10:56.970 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.970 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.970 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.970 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "50", 2024-06-10T01:10:56.970 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.970 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.970 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.970 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "number of past completed events to remember", 2024-06-10T01:10:56.971 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.971 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.971 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.971 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.971 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-06-10T01:10:56.971 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-06-10T01:10:56.971 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:56.971 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.972 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.972 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-06-10T01:10:56.972 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.972 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.972 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.972 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long the module is going to sleep", 2024-06-10T01:10:56.972 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.972 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.973 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.973 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.973 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.973 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:56.973 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus", 2024-06-10T01:10:56.973 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:56.973 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:56.974 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:56.974 INFO:teuthology.orchestra.run.smithi037.stdout: "cache": { 2024-06-10T01:10:56.974 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cache", 2024-06-10T01:10:56.974 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.974 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.974 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.974 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.974 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.975 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.975 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.975 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.975 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.975 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.975 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.975 INFO:teuthology.orchestra.run.smithi037.stdout: "exclude_perf_counters": { 2024-06-10T01:10:56.976 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "exclude_perf_counters", 2024-06-10T01:10:56.976 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.976 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.976 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.976 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:56.976 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.976 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.976 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.976 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-10T01:10:56.977 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-10T01:10:56.977 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.977 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.977 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.977 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:56.977 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:56.977 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.977 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.978 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.978 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.978 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.978 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.978 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.978 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.978 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.978 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.979 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.979 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.979 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.979 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.979 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.979 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.979 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.979 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.979 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.980 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:56.980 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:56.980 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.980 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.980 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.980 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.980 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.980 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.981 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.981 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.981 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.981 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.981 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.981 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.981 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:56.981 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:56.982 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.982 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.982 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.982 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:56.982 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.982 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.982 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.982 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:56.983 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:56.983 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:56.983 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:56.983 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:56.983 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:56.983 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.983 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.983 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.984 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.984 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.984 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:56.984 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:56.984 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:56.984 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.984 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.985 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:56.985 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.985 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.985 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.985 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.985 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.985 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.985 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.985 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.986 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools": { 2024-06-10T01:10:56.986 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools", 2024-06-10T01:10:56.986 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.986 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.986 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.986 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:56.986 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.986 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.987 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.987 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.987 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.987 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.987 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.987 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-10T01:10:56.987 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-10T01:10:56.987 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.988 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.988 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.988 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "300", 2024-06-10T01:10:56.988 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.988 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.988 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.988 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.988 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.989 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.989 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.989 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.989 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_interval": { 2024-06-10T01:10:56.989 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_interval", 2024-06-10T01:10:56.989 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-06-10T01:10:56.989 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.989 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.989 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15.0", 2024-06-10T01:10:56.989 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.990 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.990 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.990 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.990 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.990 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.990 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.990 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.990 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-06-10T01:10:56.991 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-06-10T01:10:56.991 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.991 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.991 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.991 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-06-10T01:10:56.991 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.991 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.991 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.992 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-10T01:10:56.992 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.992 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.992 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.992 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.992 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-06-10T01:10:56.992 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-06-10T01:10:56.992 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.993 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.993 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.993 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "9283", 2024-06-10T01:10:56.993 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.993 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.993 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.993 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-10T01:10:56.993 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.994 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.994 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.994 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.994 INFO:teuthology.orchestra.run.smithi037.stdout: "stale_cache_strategy": { 2024-06-10T01:10:56.994 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stale_cache_strategy", 2024-06-10T01:10:56.994 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.994 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.995 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:56.995 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "log", 2024-06-10T01:10:56.995 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.995 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.995 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.995 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.995 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.995 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.995 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.996 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.996 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-06-10T01:10:56.996 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-06-10T01:10:56.996 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:56.996 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.996 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.996 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "default", 2024-06-10T01:10:56.996 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:56.996 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:56.997 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:56.997 INFO:teuthology.orchestra.run.smithi037.stdout: "default", 2024-06-10T01:10:56.997 INFO:teuthology.orchestra.run.smithi037.stdout: "error" 2024-06-10T01:10:56.997 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:56.997 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.997 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.997 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.997 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.997 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:56.998 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-06-10T01:10:56.998 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-06-10T01:10:56.998 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:56.998 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:56.998 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:56.998 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-06-10T01:10:56.998 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-06-10T01:10:56.999 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-06-10T01:10:56.999 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:56.999 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:56.999 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:56.999 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:56.999 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:56.999 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:56.999 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.000 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:57.000 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-06-10T01:10:57.000 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:57.000 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:57.000 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:57.000 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:57.000 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:57.001 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.001 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.001 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.001 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.001 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.001 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.001 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:57.001 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:57.002 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:57.002 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:57.002 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:57.002 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:57.002 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:57.002 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.002 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.003 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.003 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.003 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.003 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:57.003 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:57.003 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.003 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.003 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.004 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.004 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.004 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.004 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.004 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.004 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.004 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.005 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.005 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.005 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:57.005 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:57.005 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.005 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.005 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.005 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:57.005 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.006 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.006 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:57.006 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:57.006 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:57.006 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:57.006 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:57.006 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:57.006 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:57.007 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.007 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.007 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.007 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.007 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.007 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.007 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:57.007 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:57.008 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.008 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.008 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.008 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.008 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.008 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.008 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.008 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.009 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.009 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.009 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.009 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.009 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_snap_create": { 2024-06-10T01:10:57.009 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_snap_create", 2024-06-10T01:10:57.009 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:57.009 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.009 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.010 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-06-10T01:10:57.010 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.010 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.010 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.010 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.010 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.010 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.010 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.011 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.011 INFO:teuthology.orchestra.run.smithi037.stdout: "mirror_snapshot_schedule": { 2024-06-10T01:10:57.011 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirror_snapshot_schedule", 2024-06-10T01:10:57.011 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.011 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.011 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.011 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.011 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.012 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.012 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.012 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.012 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.012 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.012 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.012 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.012 INFO:teuthology.orchestra.run.smithi037.stdout: "trash_purge_schedule": { 2024-06-10T01:10:57.013 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "trash_purge_schedule", 2024-06-10T01:10:57.013 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.013 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.013 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.013 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.013 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.013 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.013 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.013 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.014 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.014 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.014 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.014 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.014 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.014 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:57.014 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "restful", 2024-06-10T01:10:57.015 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:57.015 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:57.015 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:57.015 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_auth": { 2024-06-10T01:10:57.015 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_auth", 2024-06-10T01:10:57.015 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.015 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.015 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.015 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:57.016 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.016 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.016 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.016 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.016 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.016 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.016 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.016 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.016 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-06-10T01:10:57.017 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-06-10T01:10:57.017 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.017 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.017 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.017 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.017 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.017 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.017 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.017 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.018 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.018 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.018 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.018 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:57.018 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:57.018 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.019 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.019 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.019 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.019 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.019 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.019 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:57.019 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:57.019 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:57.019 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:57.020 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:57.020 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:57.020 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:57.020 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.020 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.020 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.020 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.021 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.021 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.021 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:57.021 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:57.021 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.021 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.021 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.021 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.022 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.022 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.022 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.022 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.022 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.022 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.022 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.023 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:57.023 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:57.023 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.023 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.023 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.023 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:57.023 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.023 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.024 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:57.024 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:57.024 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:57.024 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:57.024 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:57.024 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:57.024 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:57.024 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.024 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.025 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.025 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.025 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.025 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:57.025 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:57.025 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.025 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.026 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.026 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.026 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.026 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.026 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.026 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.026 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.026 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.026 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.027 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-06-10T01:10:57.027 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-06-10T01:10:57.027 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.027 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.027 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.027 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.027 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.028 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.028 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.028 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.028 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.028 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.028 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.028 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.028 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-06-10T01:10:57.029 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-06-10T01:10:57.029 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.029 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.029 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.029 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.029 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.029 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.029 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.029 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.030 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.030 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.030 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.030 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.030 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.030 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.030 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:57.030 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rgw", 2024-06-10T01:10:57.031 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:57.031 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:57.031 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:57.031 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:57.031 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:57.031 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.031 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.031 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.032 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.032 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.032 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.032 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:57.032 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:57.032 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:57.032 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:57.033 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:57.033 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:57.033 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:57.033 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.033 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.033 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.033 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.033 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.034 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:57.034 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:57.034 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.034 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.034 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.034 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.034 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.035 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.035 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.035 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.035 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.035 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.035 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.035 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:57.035 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:57.036 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.036 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.036 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.036 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:57.036 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.036 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.036 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:57.036 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:57.037 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:57.037 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:57.037 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:57.037 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:57.037 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:57.037 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.037 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.037 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.037 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.038 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.038 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:57.038 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:57.038 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.038 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.038 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.038 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.039 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.039 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.039 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.039 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.039 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.039 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.039 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.039 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.040 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.040 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:57.040 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rook", 2024-06-10T01:10:57.040 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:57.040 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:57.040 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:57.040 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:57.041 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:57.041 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.041 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.041 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.041 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.041 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.041 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.041 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:57.042 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:57.042 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:57.042 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:57.042 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:57.042 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:57.042 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:57.042 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.042 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.043 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.043 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.043 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.043 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.043 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:57.043 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:57.043 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.043 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.044 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.044 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.044 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.044 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.044 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.044 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.044 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.044 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.045 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.045 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:57.045 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:57.045 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.045 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.045 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.045 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:57.046 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.046 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.046 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:57.046 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:57.046 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:57.046 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:57.046 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:57.046 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:57.047 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:57.047 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.047 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.047 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.047 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.047 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.047 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:57.048 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:57.048 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.048 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.048 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.048 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.048 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.048 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.048 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.048 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.049 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.049 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.049 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.049 INFO:teuthology.orchestra.run.smithi037.stdout: "storage_class": { 2024-06-10T01:10:57.049 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "storage_class", 2024-06-10T01:10:57.049 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.049 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.050 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.050 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "local", 2024-06-10T01:10:57.050 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.050 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.050 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.050 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-10T01:10:57.050 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.051 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.051 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.051 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.051 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.051 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.051 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:57.051 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "selftest", 2024-06-10T01:10:57.051 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:57.052 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:57.052 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:57.052 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:57.052 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:57.052 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.052 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.052 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.052 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.053 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.053 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.053 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:57.053 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:57.053 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:57.053 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:57.053 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:57.053 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:57.053 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:57.054 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.054 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.054 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.054 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.054 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.054 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:57.055 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:57.055 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.055 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.055 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.055 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.055 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.055 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.055 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.055 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.056 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.056 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.056 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.056 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:57.056 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:57.056 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.056 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.056 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.057 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:57.057 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.057 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.057 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:57.057 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:57.057 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:57.057 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:57.057 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:57.057 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:57.058 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:57.058 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.058 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.058 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.058 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.058 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.058 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:57.059 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:57.059 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.059 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.059 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.059 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.059 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.059 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.059 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.059 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.060 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.060 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.060 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.060 INFO:teuthology.orchestra.run.smithi037.stdout: "roption1": { 2024-06-10T01:10:57.060 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption1", 2024-06-10T01:10:57.060 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.060 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.060 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.061 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.061 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.061 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.061 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.061 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.061 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.061 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.061 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.061 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.062 INFO:teuthology.orchestra.run.smithi037.stdout: "roption2": { 2024-06-10T01:10:57.062 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption2", 2024-06-10T01:10:57.062 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.062 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.062 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.062 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "xyz", 2024-06-10T01:10:57.062 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.062 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.063 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.063 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.063 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.063 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.063 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.063 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption1": { 2024-06-10T01:10:57.063 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption1", 2024-06-10T01:10:57.063 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.064 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.064 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.064 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.064 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.064 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.064 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.064 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.064 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.064 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.065 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.065 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption2": { 2024-06-10T01:10:57.065 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption2", 2024-06-10T01:10:57.065 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:57.065 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.065 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.065 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.066 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.066 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.066 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.066 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.066 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.066 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.066 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.066 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.067 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption3": { 2024-06-10T01:10:57.067 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption3", 2024-06-10T01:10:57.067 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-06-10T01:10:57.067 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.067 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.067 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.067 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.067 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.068 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.068 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.068 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.068 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.068 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.068 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption4": { 2024-06-10T01:10:57.068 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption4", 2024-06-10T01:10:57.069 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.069 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.069 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.069 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.069 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.069 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.069 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.069 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.070 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.070 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.070 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.070 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption5": { 2024-06-10T01:10:57.070 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption5", 2024-06-10T01:10:57.070 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.071 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.071 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.071 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.071 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.071 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.071 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.071 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.071 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.072 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.072 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.072 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption6": { 2024-06-10T01:10:57.072 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption6", 2024-06-10T01:10:57.072 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.072 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.072 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.073 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:57.073 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.073 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.073 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.073 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.073 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.073 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.073 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.074 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption7": { 2024-06-10T01:10:57.074 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption7", 2024-06-10T01:10:57.074 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:57.074 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.074 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.074 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.074 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-06-10T01:10:57.075 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "42", 2024-06-10T01:10:57.075 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.075 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.075 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.075 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.075 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.076 INFO:teuthology.orchestra.run.smithi037.stdout: "testkey": { 2024-06-10T01:10:57.076 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testkey", 2024-06-10T01:10:57.076 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.076 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.076 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.076 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.076 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.076 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.076 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.077 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.077 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.077 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.077 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.077 INFO:teuthology.orchestra.run.smithi037.stdout: "testlkey": { 2024-06-10T01:10:57.077 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testlkey", 2024-06-10T01:10:57.077 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.078 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.078 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.078 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.078 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.078 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.078 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.078 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.078 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.079 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.079 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.079 INFO:teuthology.orchestra.run.smithi037.stdout: "testnewline": { 2024-06-10T01:10:57.079 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testnewline", 2024-06-10T01:10:57.079 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.079 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.079 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.080 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.080 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.080 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.080 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.080 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.080 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.080 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.080 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.080 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.081 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.081 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:57.081 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-06-10T01:10:57.081 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:57.081 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:57.081 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:57.081 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_m_granularity": { 2024-06-10T01:10:57.082 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_m_granularity", 2024-06-10T01:10:57.082 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.082 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.082 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.082 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.082 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.082 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.082 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.083 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow minute scheduled snapshots", 2024-06-10T01:10:57.083 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.083 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.083 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.083 INFO:teuthology.orchestra.run.smithi037.stdout: "dump_on_update": { 2024-06-10T01:10:57.083 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dump_on_update", 2024-06-10T01:10:57.083 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.083 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "dump database to debug log on update", 2024-06-10T01:10:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.085 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:57.085 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:57.085 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.085 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.085 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.085 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.085 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.086 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.086 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:57.086 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:57.086 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:57.086 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:57.086 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:57.086 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:57.086 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:57.087 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.087 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.087 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.087 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.087 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.087 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:57.087 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:57.088 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.088 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.088 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.088 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.088 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.088 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.088 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.088 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.089 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.089 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.089 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.089 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:57.089 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:57.089 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.089 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.090 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.090 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:57.090 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.090 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.090 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:57.090 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:57.090 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:57.090 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:57.091 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:57.091 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:57.091 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:57.091 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.091 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.091 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.091 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.091 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.092 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:57.092 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:57.092 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.092 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.092 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.092 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.092 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.093 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.093 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.093 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.093 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.093 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.093 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.093 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.093 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.094 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:57.094 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stats", 2024-06-10T01:10:57.094 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:57.094 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:57.094 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:57.094 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:57.094 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:57.095 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.095 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.095 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.095 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.095 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.095 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.095 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:57.095 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:57.096 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:57.096 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:57.096 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:57.096 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:57.096 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:57.096 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.096 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.096 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.097 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.097 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.097 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:57.097 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:57.097 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.097 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.097 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.098 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.098 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.098 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.098 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.098 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.098 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.098 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.099 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.099 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:57.099 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:57.099 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.099 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.099 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.099 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:57.099 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.100 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.100 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:57.100 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:57.100 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:57.100 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:57.100 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:57.100 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:57.100 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:57.101 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.101 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.101 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.101 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.101 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.101 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:57.101 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:57.102 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.102 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.102 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.102 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.102 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.102 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.102 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.102 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.103 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.103 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.103 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.103 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.103 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.103 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:57.103 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "status", 2024-06-10T01:10:57.104 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:57.104 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:57.104 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:57.104 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:57.104 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:57.104 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.104 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.104 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.104 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.105 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.105 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.105 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:57.105 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:57.105 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:57.105 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:57.105 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:57.105 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:57.105 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:57.106 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.106 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.106 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.106 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.106 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.106 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:57.107 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:57.107 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.107 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.107 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.107 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.107 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.107 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.107 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.107 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.108 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.108 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.108 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.108 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:57.108 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:57.108 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.108 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.108 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.109 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:57.109 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.109 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.109 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:57.109 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:57.109 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:57.109 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:57.109 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:57.110 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:57.110 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:57.110 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.110 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.110 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.110 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.110 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.111 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:57.111 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:57.111 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.111 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.111 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.111 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.111 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.111 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.111 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.112 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.112 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.112 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.112 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.112 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.112 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.112 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:57.113 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telegraf", 2024-06-10T01:10:57.113 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:57.113 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:57.113 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:57.113 INFO:teuthology.orchestra.run.smithi037.stdout: "address": { 2024-06-10T01:10:57.113 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "address", 2024-06-10T01:10:57.113 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.113 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.113 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.114 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-10T01:10:57.114 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.114 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.114 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.114 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.114 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.114 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.114 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.115 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-06-10T01:10:57.115 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-06-10T01:10:57.115 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:57.115 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.115 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.115 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15", 2024-06-10T01:10:57.115 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.115 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.116 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.116 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.116 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.116 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.116 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.116 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.116 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:57.116 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:57.117 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.117 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.117 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.117 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.117 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.117 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.117 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:57.117 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:57.118 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:57.118 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:57.118 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:57.118 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:57.118 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:57.118 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.118 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.118 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.119 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.119 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.119 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:57.119 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:57.119 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.119 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.119 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.120 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.120 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.120 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.120 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.120 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.120 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.120 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.120 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.121 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:57.121 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:57.121 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.121 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.121 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.121 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:57.121 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.121 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.122 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:57.122 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:57.122 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:57.122 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:57.122 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:57.122 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:57.122 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:57.123 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.123 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.123 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.123 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.123 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.123 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.123 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:57.124 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:57.124 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.124 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.124 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.124 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.124 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.124 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.124 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.125 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.125 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.125 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.125 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.125 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.125 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.125 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.125 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:57.126 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telemetry", 2024-06-10T01:10:57.126 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:57.126 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:57.126 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:57.126 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_basic": { 2024-06-10T01:10:57.126 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_basic", 2024-06-10T01:10:57.126 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.126 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.127 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.127 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:57.127 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.127 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.127 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.127 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-10T01:10:57.127 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.127 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.127 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.128 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_crash": { 2024-06-10T01:10:57.128 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_crash", 2024-06-10T01:10:57.128 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.128 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.128 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.128 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:57.128 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.129 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.129 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.129 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-10T01:10:57.129 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.129 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.129 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.129 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_device": { 2024-06-10T01:10:57.129 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_device", 2024-06-10T01:10:57.130 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.130 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.130 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.130 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:57.130 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.130 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.130 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.131 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-10T01:10:57.131 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.131 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.131 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.131 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_ident": { 2024-06-10T01:10:57.131 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_ident", 2024-06-10T01:10:57.131 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.132 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.132 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.132 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.132 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.132 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.132 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.132 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-10T01:10:57.132 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.132 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.133 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.133 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_perf": { 2024-06-10T01:10:57.133 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_perf", 2024-06-10T01:10:57.133 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.133 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.133 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.133 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.133 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.134 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.134 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.134 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-10T01:10:57.134 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.134 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.134 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.134 INFO:teuthology.orchestra.run.smithi037.stdout: "contact": { 2024-06-10T01:10:57.135 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "contact", 2024-06-10T01:10:57.135 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.135 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.135 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.135 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.135 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.135 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.135 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.136 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.136 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.136 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.136 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.136 INFO:teuthology.orchestra.run.smithi037.stdout: "description": { 2024-06-10T01:10:57.136 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "description", 2024-06-10T01:10:57.136 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.137 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.137 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.137 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.137 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.137 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.137 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.137 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.137 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.138 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.138 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.138 INFO:teuthology.orchestra.run.smithi037.stdout: "device_url": { 2024-06-10T01:10:57.138 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_url", 2024-06-10T01:10:57.138 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.138 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.138 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.139 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-10T01:10:57.139 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.139 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.139 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.139 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.139 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.139 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.139 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.140 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-06-10T01:10:57.140 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-06-10T01:10:57.140 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.140 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.140 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.140 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.140 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.140 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.141 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.141 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.141 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.141 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.141 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.141 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-06-10T01:10:57.141 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-06-10T01:10:57.141 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:57.142 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.142 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.142 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "24", 2024-06-10T01:10:57.142 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "8", 2024-06-10T01:10:57.142 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.142 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.142 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.142 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.143 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.143 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.143 INFO:teuthology.orchestra.run.smithi037.stdout: "last_opt_revision": { 2024-06-10T01:10:57.143 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "last_opt_revision", 2024-06-10T01:10:57.143 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:57.143 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.143 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.144 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1", 2024-06-10T01:10:57.144 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.144 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.144 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.144 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.144 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.144 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.144 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.145 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.145 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard": { 2024-06-10T01:10:57.145 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard", 2024-06-10T01:10:57.145 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.145 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.145 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.145 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.145 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.146 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.146 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.146 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.146 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.146 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.146 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.146 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard_description": { 2024-06-10T01:10:57.147 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard_description", 2024-06-10T01:10:57.147 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.147 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.147 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.147 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.147 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.147 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.147 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.148 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.148 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.148 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.148 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.148 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:57.148 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:57.148 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.149 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.149 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.149 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.149 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.149 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.149 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:57.149 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:57.149 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:57.149 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:57.150 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:57.150 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:57.150 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:57.150 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.150 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.150 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.150 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.150 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.151 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:57.151 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:57.151 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.151 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.151 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.151 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.151 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.151 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.152 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.152 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.152 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.152 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.152 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.152 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.152 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:57.152 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:57.153 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.153 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.153 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.153 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:57.153 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.153 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.153 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:57.153 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:57.153 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:57.154 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:57.154 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:57.154 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:57.154 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:57.154 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.154 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.154 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.154 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.155 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.155 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.155 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:57.155 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:57.155 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.155 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.155 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.155 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.156 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.156 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.156 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.156 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.156 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.156 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.156 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.156 INFO:teuthology.orchestra.run.smithi037.stdout: "organization": { 2024-06-10T01:10:57.157 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "organization", 2024-06-10T01:10:57.157 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.157 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.157 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.157 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.157 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.157 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.157 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.158 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.158 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.158 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.158 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.158 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.158 INFO:teuthology.orchestra.run.smithi037.stdout: "proxy": { 2024-06-10T01:10:57.158 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "proxy", 2024-06-10T01:10:57.158 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.159 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.159 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.159 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.159 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.159 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.159 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.159 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.159 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.160 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.160 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.160 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.160 INFO:teuthology.orchestra.run.smithi037.stdout: "url": { 2024-06-10T01:10:57.160 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url", 2024-06-10T01:10:57.160 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.160 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.160 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.160 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-10T01:10:57.161 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.161 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.161 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.161 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.161 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.161 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.161 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.161 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.162 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.162 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:57.162 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "test_orchestrator", 2024-06-10T01:10:57.162 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:57.162 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:57.162 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:57.162 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:57.163 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:57.163 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.163 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.163 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.163 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.163 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.163 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.163 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:57.163 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:57.164 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:57.164 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:57.164 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:57.164 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:57.164 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:57.164 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.164 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.164 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.164 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.165 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.165 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.165 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:57.165 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:57.165 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.165 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.165 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.165 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.166 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.166 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.166 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.166 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.166 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.166 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.166 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.166 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:57.167 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:57.167 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.167 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.167 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.167 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:57.167 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.167 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.167 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:57.168 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:57.168 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:57.168 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:57.168 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:57.168 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:57.168 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:57.168 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.168 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.169 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.169 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.169 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.169 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.169 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:57.169 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:57.169 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.169 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.170 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.170 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.170 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.170 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.170 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.170 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.170 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.170 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.171 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.171 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.171 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.171 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:57.171 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-06-10T01:10:57.171 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:57.171 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:57.172 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:57.172 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:57.172 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:57.172 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.172 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.172 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.172 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.172 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.172 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.173 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:57.173 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:57.173 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:57.173 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:57.173 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:57.173 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:57.173 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:57.173 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.174 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.174 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.174 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.174 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.174 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.174 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:57.174 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:57.174 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.175 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.175 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.175 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.175 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.175 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.175 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.175 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.175 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.176 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.176 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.176 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.176 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:57.176 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:57.176 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.176 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.176 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.176 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:57.177 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.177 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.177 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:57.177 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:57.177 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:57.177 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:57.177 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:57.177 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:57.178 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:57.178 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.178 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.178 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.178 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.178 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.178 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.178 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:57.179 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:57.179 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.179 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.179 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.179 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.179 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.179 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.179 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.179 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.180 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.180 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.180 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.180 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.180 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_clones": { 2024-06-10T01:10:57.180 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_clones", 2024-06-10T01:10:57.180 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:57.180 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.181 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.181 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4", 2024-06-10T01:10:57.181 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.181 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.181 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.181 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-10T01:10:57.181 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.181 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.182 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.182 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.182 INFO:teuthology.orchestra.run.smithi037.stdout: "periodic_async_work": { 2024-06-10T01:10:57.182 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "periodic_async_work", 2024-06-10T01:10:57.182 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.182 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.182 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.182 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.183 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.183 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.183 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.183 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Periodically check for async work", 2024-06-10T01:10:57.183 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.183 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.183 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.183 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.184 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_delay": { 2024-06-10T01:10:57.184 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_delay", 2024-06-10T01:10:57.184 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:57.184 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.184 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.184 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-06-10T01:10:57.184 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.184 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.185 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.185 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-10T01:10:57.185 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.185 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.185 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.185 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.185 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_no_wait": { 2024-06-10T01:10:57.185 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_no_wait", 2024-06-10T01:10:57.185 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.186 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.186 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.186 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-06-10T01:10:57.186 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.186 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.186 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.186 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-10T01:10:57.186 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.187 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.187 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.187 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.187 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.187 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:57.187 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix", 2024-06-10T01:10:57.187 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-06-10T01:10:57.187 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-06-10T01:10:57.187 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-06-10T01:10:57.188 INFO:teuthology.orchestra.run.smithi037.stdout: "discovery_interval": { 2024-06-10T01:10:57.188 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "discovery_interval", 2024-06-10T01:10:57.188 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-06-10T01:10:57.188 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.188 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.188 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "100", 2024-06-10T01:10:57.188 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.188 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.189 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.189 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.189 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.189 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.189 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.189 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.189 INFO:teuthology.orchestra.run.smithi037.stdout: "identifier": { 2024-06-10T01:10:57.189 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "identifier", 2024-06-10T01:10:57.189 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.190 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.190 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.190 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.190 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.190 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.190 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.190 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.191 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.191 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.191 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.191 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.191 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-06-10T01:10:57.191 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-06-10T01:10:57.191 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-06-10T01:10:57.191 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.192 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.192 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-06-10T01:10:57.192 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.192 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.192 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.192 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.192 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.192 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.193 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.193 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.193 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-06-10T01:10:57.193 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-06-10T01:10:57.193 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.193 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.193 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.193 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.193 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.194 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.194 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:57.194 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:57.194 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:57.194 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:57.194 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:57.194 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:57.194 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:57.195 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.195 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.195 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.195 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.195 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.195 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.195 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-06-10T01:10:57.195 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-06-10T01:10:57.196 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.196 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.196 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.196 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.196 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.196 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.196 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.196 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.197 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.197 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.197 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.197 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-06-10T01:10:57.197 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-06-10T01:10:57.197 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.197 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.198 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.198 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-06-10T01:10:57.198 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.198 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.198 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-06-10T01:10:57.198 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-06-10T01:10:57.198 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-06-10T01:10:57.199 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-06-10T01:10:57.199 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-06-10T01:10:57.199 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-06-10T01:10:57.199 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-06-10T01:10:57.199 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.199 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.199 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.199 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.200 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.200 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.200 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-06-10T01:10:57.200 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-06-10T01:10:57.200 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-06-10T01:10:57.200 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.200 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-06-10T01:10:57.200 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-06-10T01:10:57.201 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.201 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.201 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.201 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.201 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.201 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.201 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.201 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.202 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_host": { 2024-06-10T01:10:57.202 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_host", 2024-06-10T01:10:57.202 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.202 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.202 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.202 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-06-10T01:10:57.202 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.202 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.203 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.203 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.203 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.203 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.203 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.203 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_port": { 2024-06-10T01:10:57.203 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_port", 2024-06-10T01:10:57.203 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-06-10T01:10:57.204 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.204 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.204 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10051", 2024-06-10T01:10:57.204 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.204 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.204 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.204 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.204 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.205 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.205 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.205 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.205 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_sender": { 2024-06-10T01:10:57.205 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_sender", 2024-06-10T01:10:57.205 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-06-10T01:10:57.205 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-06-10T01:10:57.205 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-06-10T01:10:57.206 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-10T01:10:57.206 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-06-10T01:10:57.206 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-06-10T01:10:57.206 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-06-10T01:10:57.206 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-06-10T01:10:57.206 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-06-10T01:10:57.206 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-06-10T01:10:57.206 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-06-10T01:10:57.207 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.207 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.207 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.207 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.207 INFO:teuthology.orchestra.run.smithi037.stdout: "services": { 2024-06-10T01:10:57.207 INFO:teuthology.orchestra.run.smithi037.stdout: "dashboard": "https://172.21.15.37:8443/" 2024-06-10T01:10:57.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.207 INFO:teuthology.orchestra.run.smithi037.stdout: "always_on_modules": { 2024-06-10T01:10:57.208 INFO:teuthology.orchestra.run.smithi037.stdout: "octopus": [ 2024-06-10T01:10:57.208 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-06-10T01:10:57.208 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-06-10T01:10:57.208 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-06-10T01:10:57.208 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-06-10T01:10:57.208 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-06-10T01:10:57.208 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-06-10T01:10:57.208 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-06-10T01:10:57.209 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-06-10T01:10:57.209 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-06-10T01:10:57.209 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-06-10T01:10:57.209 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.209 INFO:teuthology.orchestra.run.smithi037.stdout: "pacific": [ 2024-06-10T01:10:57.209 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-06-10T01:10:57.209 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-06-10T01:10:57.209 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-06-10T01:10:57.210 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-06-10T01:10:57.210 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-06-10T01:10:57.210 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-06-10T01:10:57.210 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-06-10T01:10:57.210 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-06-10T01:10:57.210 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-06-10T01:10:57.210 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-06-10T01:10:57.210 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.211 INFO:teuthology.orchestra.run.smithi037.stdout: "quincy": [ 2024-06-10T01:10:57.211 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-06-10T01:10:57.211 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-06-10T01:10:57.211 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-06-10T01:10:57.211 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-06-10T01:10:57.211 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-06-10T01:10:57.211 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-06-10T01:10:57.211 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-06-10T01:10:57.212 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-06-10T01:10:57.212 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-06-10T01:10:57.212 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-06-10T01:10:57.212 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.212 INFO:teuthology.orchestra.run.smithi037.stdout: "reef": [ 2024-06-10T01:10:57.212 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-06-10T01:10:57.212 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-06-10T01:10:57.212 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-06-10T01:10:57.213 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-06-10T01:10:57.213 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-06-10T01:10:57.213 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-06-10T01:10:57.213 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-06-10T01:10:57.213 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-06-10T01:10:57.213 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-06-10T01:10:57.213 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-06-10T01:10:57.214 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-06-10T01:10:57.214 INFO:teuthology.orchestra.run.smithi037.stdout: "squid": [ 2024-06-10T01:10:57.214 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-06-10T01:10:57.214 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-06-10T01:10:57.214 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-06-10T01:10:57.214 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-06-10T01:10:57.214 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-06-10T01:10:57.214 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-06-10T01:10:57.215 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-06-10T01:10:57.215 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-06-10T01:10:57.215 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-06-10T01:10:57.215 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-06-10T01:10:57.215 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-06-10T01:10:57.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.215 INFO:teuthology.orchestra.run.smithi037.stdout: "last_failure_osd_epoch": 3, 2024-06-10T01:10:57.215 INFO:teuthology.orchestra.run.smithi037.stdout: "active_clients": [ 2024-06-10T01:10:57.216 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:57.216 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-06-10T01:10:57.216 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-06-10T01:10:57.216 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:57.216 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-06-10T01:10:57.216 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-06-10T01:10:57.216 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 2549243596 2024-06-10T01:10:57.216 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.217 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-06-10T01:10:57.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.217 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:57.217 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "libcephsqlite", 2024-06-10T01:10:57.217 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-06-10T01:10:57.217 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:57.217 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-06-10T01:10:57.217 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-06-10T01:10:57.218 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 2247424744 2024-06-10T01:10:57.218 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.218 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-06-10T01:10:57.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.218 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:57.218 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-06-10T01:10:57.218 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-06-10T01:10:57.218 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:57.219 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-06-10T01:10:57.219 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-06-10T01:10:57.219 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1093170904 2024-06-10T01:10:57.219 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.219 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-06-10T01:10:57.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:10:57.219 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:57.219 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-06-10T01:10:57.219 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-06-10T01:10:57.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-06-10T01:10:57.220 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-06-10T01:10:57.220 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-06-10T01:10:57.220 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 880157111 2024-06-10T01:10:57.220 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.220 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-06-10T01:10:57.220 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:10:57.220 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-06-10T01:10:57.221 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:10:57.222 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:55 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/4087027999' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-10T01:10:57.222 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:55 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1161741618' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-10T01:10:57.222 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:55 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/218062647' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-10T01:10:57.223 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:56 smithi094 ceph-mon[36708]: pgmap v862: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-10T01:10:57.223 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:55 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/4087027999' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-10T01:10:57.223 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:55 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1161741618' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-10T01:10:57.223 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:55 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/218062647' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-10T01:10:57.223 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:56 smithi167 ceph-mon[36917]: pgmap v862: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-10T01:10:57.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:55.005+0000 7fa8913f9640 1 -- 172.21.15.37:0/218062647 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa8740a2b90 msgr2=0x7fa8740a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:57.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:55.005+0000 7fa8913f9640 1 --2- 172.21.15.37:0/218062647 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa8740a2b90 0x7fa8740a5050 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa88c101950 tx=0x7fa87c009110 comp rx=0 tx=0).stop 2024-06-10T01:10:57.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:55.005+0000 7fa8913f9640 1 -- 172.21.15.37:0/218062647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa88c1024b0 msgr2=0x7fa88c19f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:10:57.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:55.005+0000 7fa8913f9640 1 --2- 172.21.15.37:0/218062647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa88c1024b0 0x7fa88c19f300 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fa8840319f0 tx=0x7fa88402f450 comp rx=0 tx=0).stop 2024-06-10T01:10:57.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:55.005+0000 7fa8913f9640 1 -- 172.21.15.37:0/218062647 shutdown_connections 2024-06-10T01:10:57.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:55.005+0000 7fa8913f9640 1 --2- 172.21.15.37:0/218062647 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa8740a2b90 0x7fa8740a5050 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:57.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:55.005+0000 7fa8913f9640 1 --2- 172.21.15.37:0/218062647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa88c1024b0 0x7fa88c19f300 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:57.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:55.005+0000 7fa8913f9640 1 --2- 172.21.15.37:0/218062647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa88c101af0 0x7fa88c198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:57.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:55.005+0000 7fa8913f9640 1 --2- 172.21.15.37:0/218062647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa88c100900 0x7fa88c197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:10:57.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:55.005+0000 7fa8913f9640 1 -- 172.21.15.37:0/218062647 >> 172.21.15.37:0/218062647 conn(0x7fa88c0fbe90 msgr2=0x7fa88c0fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:10:57.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:55.005+0000 7fa8913f9640 1 -- 172.21.15.37:0/218062647 shutdown_connections 2024-06-10T01:10:57.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:10:55.005+0000 7fa8913f9640 1 -- 172.21.15.37:0/218062647 wait complete. 2024-06-10T01:10:57.231 DEBUG:tasks.check_counter:Getting stats from a 2024-06-10T01:10:57.231 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph daemon mds.a perf dump 2024-06-10T01:10:57.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:55 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/4087027999' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-10T01:10:57.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:55 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1161741618' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-10T01:10:57.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:55 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/218062647' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-10T01:10:57.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:56 smithi037 ceph-mon[32192]: pgmap v862: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-10T01:10:57.500 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:10:58.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:58 smithi167 ceph-mon[36917]: pgmap v863: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-06-10T01:10:58.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:10:58 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:10:58.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:58 smithi094 ceph-mon[36708]: pgmap v863: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-06-10T01:10:58.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:10:58 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:10:58.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:58 smithi037 ceph-mon[32192]: pgmap v863: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-06-10T01:10:58.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:10:58 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:11:00.435 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:00 smithi037 ceph-mon[32192]: pgmap v864: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T01:11:00.435 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:00 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:11:00.435 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:00 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:11:00.436 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:00 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:11:00.436 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:00 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:11:00.436 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:00 smithi037 ceph-mon[32192]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-06-10T01:11:00.436 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:00 smithi037 ceph-mon[32192]: Unable to set osd_memory_target on smithi037 to 185844019: error parsing value: Value '185844019' is below minimum 939524096 2024-06-10T01:11:00.436 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:00 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:11:00.436 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:00 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:11:00.436 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:00 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:11:00.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:00 smithi167 ceph-mon[36917]: pgmap v864: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T01:11:00.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:00 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:11:00.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:00 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:11:00.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:00 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:11:00.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:00 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:11:00.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:00 smithi167 ceph-mon[36917]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-06-10T01:11:00.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:00 smithi167 ceph-mon[36917]: Unable to set osd_memory_target on smithi037 to 185844019: error parsing value: Value '185844019' is below minimum 939524096 2024-06-10T01:11:00.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:00 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:11:00.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:00 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:11:00.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:00 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:11:00.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:00 smithi094 ceph-mon[36708]: pgmap v864: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T01:11:00.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:00 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:11:00.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:00 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:11:00.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:00 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:11:00.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:00 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:11:00.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:00 smithi094 ceph-mon[36708]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-06-10T01:11:00.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:00 smithi094 ceph-mon[36708]: Unable to set osd_memory_target on smithi037 to 185844019: error parsing value: Value '185844019' is below minimum 939524096 2024-06-10T01:11:00.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:00 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:11:00.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:00 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:11:00.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:00 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:11:00.841 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:11:00.841 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T01:11:00.841 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 0, 2024-06-10T01:11:00.841 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 2, 2024-06-10T01:11:00.842 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 0, 2024-06-10T01:11:00.842 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 0, 2024-06-10T01:11:00.842 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-06-10T01:11:00.842 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-06-10T01:11:00.842 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.001470084, 2024-06-10T01:11:00.842 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000000000, 2024-06-10T01:11:00.842 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000322337, 2024-06-10T01:11:00.842 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T01:11:00.843 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:00.843 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-06-10T01:11:00.843 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-06-10T01:11:00.843 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-06-10T01:11:00.843 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:00.843 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:00.843 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-06-10T01:11:00.843 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-06-10T01:11:00.844 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-06-10T01:11:00.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:00.844 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-10T01:11:00.844 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 0 2024-06-10T01:11:00.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:00.845 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T01:11:00.845 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 2, 2024-06-10T01:11:00.845 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 229, 2024-06-10T01:11:00.845 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 320, 2024-06-10T01:11:00.845 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 196304, 2024-06-10T01:11:00.845 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-06-10T01:11:00.845 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 2, 2024-06-10T01:11:00.846 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.015858161, 2024-06-10T01:11:00.846 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.012147963, 2024-06-10T01:11:00.846 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.001592103, 2024-06-10T01:11:00.846 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T01:11:00.846 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:00.846 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 227, 2024-06-10T01:11:00.846 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.012262727, 2024-06-10T01:11:00.847 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000054020 2024-06-10T01:11:00.847 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:00.847 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:00.847 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-06-10T01:11:00.847 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-06-10T01:11:00.847 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-06-10T01:11:00.847 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:00.847 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-10T01:11:00.848 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 196304 2024-06-10T01:11:00.848 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:00.848 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T01:11:00.848 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 309, 2024-06-10T01:11:00.848 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 302, 2024-06-10T01:11:00.848 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 781536, 2024-06-10T01:11:00.848 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 185328, 2024-06-10T01:11:00.849 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-06-10T01:11:00.849 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-06-10T01:11:00.849 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.045331257, 2024-06-10T01:11:00.849 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.019014815, 2024-06-10T01:11:00.849 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.021509195, 2024-06-10T01:11:00.849 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.005627846, 2024-06-10T01:11:00.849 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:00.850 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 302, 2024-06-10T01:11:00.850 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.014666776, 2024-06-10T01:11:00.850 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000048565 2024-06-10T01:11:00.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:00.850 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:00.850 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-06-10T01:11:00.850 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-06-10T01:11:00.851 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-06-10T01:11:00.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:00.851 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 781536, 2024-06-10T01:11:00.851 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 185328 2024-06-10T01:11:00.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:00.851 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-06-10T01:11:00.851 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-06-10T01:11:00.852 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-06-10T01:11:00.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:00.852 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-06-10T01:11:00.852 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-06-10T01:11:00.852 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-06-10T01:11:00.852 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T01:11:00.852 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-06-10T01:11:00.853 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T01:11:00.853 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-06-10T01:11:00.853 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T01:11:00.853 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T01:11:00.853 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T01:11:00.853 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T01:11:00.853 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T01:11:00.854 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-06-10T01:11:00.854 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T01:11:00.854 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T01:11:00.854 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-06-10T01:11:00.854 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-06-10T01:11:00.854 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-06-10T01:11:00.854 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-06-10T01:11:00.855 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T01:11:00.855 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T01:11:00.855 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T01:11:00.855 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T01:11:00.855 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T01:11:00.855 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-06-10T01:11:00.855 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-06-10T01:11:00.856 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-06-10T01:11:00.856 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T01:11:00.856 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T01:11:00.856 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-06-10T01:11:00.856 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-06-10T01:11:00.856 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-06-10T01:11:00.856 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-06-10T01:11:00.857 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T01:11:00.857 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-06-10T01:11:00.857 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T01:11:00.857 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-06-10T01:11:00.857 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148744, 2024-06-10T01:11:00.857 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 30, 2024-06-10T01:11:00.857 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-06-10T01:11:00.857 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-06-10T01:11:00.858 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-06-10T01:11:00.858 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-06-10T01:11:00.858 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-06-10T01:11:00.858 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-06-10T01:11:00.858 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-06-10T01:11:00.858 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-06-10T01:11:00.859 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-06-10T01:11:00.859 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-06-10T01:11:00.859 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T01:11:00.859 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-06-10T01:11:00.859 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-06-10T01:11:00.859 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-06-10T01:11:00.859 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-06-10T01:11:00.859 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-06-10T01:11:00.860 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-06-10T01:11:00.860 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-06-10T01:11:00.860 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-06-10T01:11:00.860 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-06-10T01:11:00.860 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:00.860 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T01:11:00.860 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-06-10T01:11:00.861 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-06-10T01:11:00.861 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-06-10T01:11:00.861 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 307, 2024-06-10T01:11:00.861 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 735440, 2024-06-10T01:11:00.861 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:00.861 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 307, 2024-06-10T01:11:00.861 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-06-10T01:11:00.862 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-06-10T01:11:00.862 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 307, 2024-06-10T01:11:00.862 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 735440, 2024-06-10T01:11:00.862 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-06-10T01:11:00.862 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-06-10T01:11:00.862 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-06-10T01:11:00.862 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-06-10T01:11:00.863 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:11:00.863 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:11:00.863 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:11:01.545 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-06-10T01:11:01.546 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-06-10T01:11:01.546 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-06-10T01:11:01.546 DEBUG:tasks.check_counter:Getting stats from d 2024-06-10T01:11:01.546 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph daemon mds.d perf dump 2024-06-10T01:11:01.811 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:11:02.390 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:02 smithi037 ceph-mon[32192]: pgmap v865: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:11:02.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:02 smithi167 ceph-mon[36917]: pgmap v865: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:11:02.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:02 smithi094 ceph-mon[36708]: pgmap v865: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:11:03.578 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:11:03.578 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T01:11:03.579 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 0, 2024-06-10T01:11:03.579 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 2, 2024-06-10T01:11:03.579 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 0, 2024-06-10T01:11:03.579 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 0, 2024-06-10T01:11:03.579 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-06-10T01:11:03.579 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-06-10T01:11:03.579 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.001297396, 2024-06-10T01:11:03.579 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000000000, 2024-06-10T01:11:03.580 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000254689, 2024-06-10T01:11:03.580 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T01:11:03.580 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:03.580 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-06-10T01:11:03.580 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-06-10T01:11:03.580 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-06-10T01:11:03.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:03.580 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:03.581 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-06-10T01:11:03.581 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-06-10T01:11:03.581 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-06-10T01:11:03.581 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:03.581 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-10T01:11:03.581 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 0 2024-06-10T01:11:03.581 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:03.581 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T01:11:03.582 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 2, 2024-06-10T01:11:03.582 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 234, 2024-06-10T01:11:03.582 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 320, 2024-06-10T01:11:03.582 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 197184, 2024-06-10T01:11:03.582 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-06-10T01:11:03.582 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 2, 2024-06-10T01:11:03.582 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.016865457, 2024-06-10T01:11:03.582 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.013326942, 2024-06-10T01:11:03.583 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.001332652, 2024-06-10T01:11:03.583 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T01:11:03.583 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:03.583 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 232, 2024-06-10T01:11:03.583 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.010722701, 2024-06-10T01:11:03.583 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000046218 2024-06-10T01:11:03.583 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:03.583 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:03.584 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-06-10T01:11:03.584 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-06-10T01:11:03.584 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-06-10T01:11:03.584 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:03.584 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-10T01:11:03.584 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 197184 2024-06-10T01:11:03.584 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:03.584 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T01:11:03.584 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 315, 2024-06-10T01:11:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 308, 2024-06-10T01:11:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 783168, 2024-06-10T01:11:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 189168, 2024-06-10T01:11:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-06-10T01:11:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-06-10T01:11:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.045132662, 2024-06-10T01:11:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.018892455, 2024-06-10T01:11:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.021313635, 2024-06-10T01:11:03.586 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.005913030, 2024-06-10T01:11:03.586 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:03.586 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 308, 2024-06-10T01:11:03.586 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.013667865, 2024-06-10T01:11:03.586 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000044376 2024-06-10T01:11:03.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:03.586 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:03.586 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-06-10T01:11:03.587 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-06-10T01:11:03.587 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-06-10T01:11:03.588 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:03.588 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 783168, 2024-06-10T01:11:03.588 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 189168 2024-06-10T01:11:03.588 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:03.588 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-06-10T01:11:03.588 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-06-10T01:11:03.589 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-06-10T01:11:03.589 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:03.589 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-06-10T01:11:03.589 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-06-10T01:11:03.589 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-06-10T01:11:03.589 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T01:11:03.589 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-06-10T01:11:03.589 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T01:11:03.590 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-06-10T01:11:03.590 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T01:11:03.590 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T01:11:03.590 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T01:11:03.590 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T01:11:03.590 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T01:11:03.590 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-06-10T01:11:03.591 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T01:11:03.591 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T01:11:03.591 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-06-10T01:11:03.591 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-06-10T01:11:03.591 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-06-10T01:11:03.591 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-06-10T01:11:03.591 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T01:11:03.591 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T01:11:03.592 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T01:11:03.592 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T01:11:03.592 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T01:11:03.592 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-06-10T01:11:03.592 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-06-10T01:11:03.592 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-06-10T01:11:03.592 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T01:11:03.592 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T01:11:03.593 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-06-10T01:11:03.593 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-06-10T01:11:03.593 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-06-10T01:11:03.593 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-06-10T01:11:03.593 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T01:11:03.593 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-06-10T01:11:03.593 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T01:11:03.594 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-06-10T01:11:03.594 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148744, 2024-06-10T01:11:03.594 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 30, 2024-06-10T01:11:03.594 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-06-10T01:11:03.594 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-06-10T01:11:03.594 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-06-10T01:11:03.594 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-06-10T01:11:03.595 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-06-10T01:11:03.595 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-06-10T01:11:03.595 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-06-10T01:11:03.595 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-06-10T01:11:03.595 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-06-10T01:11:03.595 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-06-10T01:11:03.595 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T01:11:03.595 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-06-10T01:11:03.595 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-06-10T01:11:03.596 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-06-10T01:11:03.596 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-06-10T01:11:03.596 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-06-10T01:11:03.596 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-06-10T01:11:03.596 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-06-10T01:11:03.596 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-06-10T01:11:03.596 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-06-10T01:11:03.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T01:11:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-06-10T01:11:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-06-10T01:11:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-06-10T01:11:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 313, 2024-06-10T01:11:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 736220, 2024-06-10T01:11:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 313, 2024-06-10T01:11:03.598 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-06-10T01:11:03.598 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-06-10T01:11:03.598 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 313, 2024-06-10T01:11:03.598 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 736220, 2024-06-10T01:11:03.598 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-06-10T01:11:03.598 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-06-10T01:11:03.598 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-06-10T01:11:03.599 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-06-10T01:11:03.599 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:11:03.599 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:11:03.599 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:11:04.275 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-06-10T01:11:04.275 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-06-10T01:11:04.275 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-06-10T01:11:04.275 DEBUG:tasks.check_counter:Getting stats from g 2024-06-10T01:11:04.276 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph daemon mds.g perf dump 2024-06-10T01:11:04.530 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:11:04.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:04 smithi167 ceph-mon[36917]: pgmap v866: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:11:04.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:04 smithi094 ceph-mon[36708]: pgmap v866: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:11:04.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:04 smithi037 ceph-mon[32192]: pgmap v866: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:11:06.293 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:11:06.293 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T01:11:06.293 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 308, 2024-06-10T01:11:06.293 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 303, 2024-06-10T01:11:06.294 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 779328, 2024-06-10T01:11:06.294 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 186800, 2024-06-10T01:11:06.294 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-06-10T01:11:06.294 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-06-10T01:11:06.294 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.044640054, 2024-06-10T01:11:06.294 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.018100058, 2024-06-10T01:11:06.294 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.020601926, 2024-06-10T01:11:06.295 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.005829196, 2024-06-10T01:11:06.295 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:06.295 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 302, 2024-06-10T01:11:06.295 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.013192004, 2024-06-10T01:11:06.295 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000043682 2024-06-10T01:11:06.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:06.295 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:06.296 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-06-10T01:11:06.296 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-06-10T01:11:06.296 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-06-10T01:11:06.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:06.296 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 779328, 2024-06-10T01:11:06.296 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 186800 2024-06-10T01:11:06.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:06.296 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T01:11:06.296 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 6, 2024-06-10T01:11:06.297 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 233, 2024-06-10T01:11:06.297 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 3344, 2024-06-10T01:11:06.297 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 197104, 2024-06-10T01:11:06.297 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-06-10T01:11:06.297 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 2, 2024-06-10T01:11:06.297 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.018459864, 2024-06-10T01:11:06.297 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.013495898, 2024-06-10T01:11:06.297 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.002668736, 2024-06-10T01:11:06.298 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T01:11:06.298 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:06.298 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 232, 2024-06-10T01:11:06.298 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.012634355, 2024-06-10T01:11:06.298 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000054458 2024-06-10T01:11:06.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:06.299 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:06.299 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-06-10T01:11:06.299 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-06-10T01:11:06.299 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-06-10T01:11:06.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:06.299 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-06-10T01:11:06.299 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 197104 2024-06-10T01:11:06.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:06.300 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T01:11:06.300 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 0, 2024-06-10T01:11:06.300 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 2, 2024-06-10T01:11:06.300 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 0, 2024-06-10T01:11:06.300 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 0, 2024-06-10T01:11:06.300 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-06-10T01:11:06.301 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-06-10T01:11:06.301 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.000244762, 2024-06-10T01:11:06.301 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000000000, 2024-06-10T01:11:06.301 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000181371, 2024-06-10T01:11:06.301 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T01:11:06.301 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:06.301 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-06-10T01:11:06.301 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-06-10T01:11:06.302 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-06-10T01:11:06.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:06.302 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:06.302 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-06-10T01:11:06.302 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-06-10T01:11:06.302 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-06-10T01:11:06.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:06.302 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-10T01:11:06.303 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 0 2024-06-10T01:11:06.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:06.303 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-06-10T01:11:06.303 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-06-10T01:11:06.303 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-06-10T01:11:06.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:06.303 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-06-10T01:11:06.303 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-06-10T01:11:06.304 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-06-10T01:11:06.304 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T01:11:06.304 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-06-10T01:11:06.304 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T01:11:06.304 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-06-10T01:11:06.304 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T01:11:06.304 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T01:11:06.304 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T01:11:06.304 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T01:11:06.305 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T01:11:06.305 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-06-10T01:11:06.305 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T01:11:06.305 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T01:11:06.305 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-06-10T01:11:06.305 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-06-10T01:11:06.305 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-06-10T01:11:06.305 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-06-10T01:11:06.306 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T01:11:06.306 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T01:11:06.306 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T01:11:06.306 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T01:11:06.307 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T01:11:06.307 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-06-10T01:11:06.307 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-06-10T01:11:06.307 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-06-10T01:11:06.308 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T01:11:06.308 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T01:11:06.308 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-06-10T01:11:06.308 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-06-10T01:11:06.309 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-06-10T01:11:06.309 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-06-10T01:11:06.309 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T01:11:06.309 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-06-10T01:11:06.310 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T01:11:06.310 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-06-10T01:11:06.310 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148744, 2024-06-10T01:11:06.310 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 30, 2024-06-10T01:11:06.310 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-06-10T01:11:06.311 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-06-10T01:11:06.311 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-06-10T01:11:06.311 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-06-10T01:11:06.311 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-06-10T01:11:06.311 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-06-10T01:11:06.312 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-06-10T01:11:06.312 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-06-10T01:11:06.312 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-06-10T01:11:06.312 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-06-10T01:11:06.312 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T01:11:06.312 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-06-10T01:11:06.313 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-06-10T01:11:06.313 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-06-10T01:11:06.313 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-06-10T01:11:06.313 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-06-10T01:11:06.313 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-06-10T01:11:06.313 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-06-10T01:11:06.313 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-06-10T01:11:06.314 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-06-10T01:11:06.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:06.314 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T01:11:06.314 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-06-10T01:11:06.314 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-06-10T01:11:06.314 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-06-10T01:11:06.314 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 310, 2024-06-10T01:11:06.314 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 735830, 2024-06-10T01:11:06.315 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:06.315 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 310, 2024-06-10T01:11:06.315 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-06-10T01:11:06.315 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-06-10T01:11:06.315 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 310, 2024-06-10T01:11:06.315 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 735830, 2024-06-10T01:11:06.315 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-06-10T01:11:06.315 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-06-10T01:11:06.316 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-06-10T01:11:06.316 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-06-10T01:11:06.316 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:11:06.316 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:11:06.316 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:11:06.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:06 smithi167 ceph-mon[36917]: pgmap v867: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:11:06.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:06 smithi094 ceph-mon[36708]: pgmap v867: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:11:06.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:06 smithi037 ceph-mon[32192]: pgmap v867: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:11:06.948 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-06-10T01:11:06.948 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-06-10T01:11:06.948 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-06-10T01:11:06.948 DEBUG:tasks.check_counter:Getting stats from j 2024-06-10T01:11:06.949 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph daemon mds.j perf dump 2024-06-10T01:11:07.201 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:11:08.320 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:08 smithi037 ceph-mon[32192]: pgmap v868: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-06-10T01:11:08.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:08 smithi167 ceph-mon[36917]: pgmap v868: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-06-10T01:11:08.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:08 smithi094 ceph-mon[36708]: pgmap v868: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-06-10T01:11:08.955 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-06-10T01:11:08.955 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T01:11:08.956 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 0, 2024-06-10T01:11:08.956 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 2, 2024-06-10T01:11:08.956 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 0, 2024-06-10T01:11:08.956 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 0, 2024-06-10T01:11:08.956 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-06-10T01:11:08.956 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-06-10T01:11:08.956 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.001312292, 2024-06-10T01:11:08.957 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000000000, 2024-06-10T01:11:08.957 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000292586, 2024-06-10T01:11:08.957 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T01:11:08.957 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:08.957 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-06-10T01:11:08.957 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-06-10T01:11:08.957 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-06-10T01:11:08.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:08.958 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:08.958 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-06-10T01:11:08.958 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-06-10T01:11:08.958 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-06-10T01:11:08.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:08.958 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-10T01:11:08.958 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 0 2024-06-10T01:11:08.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:08.959 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T01:11:08.959 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 2, 2024-06-10T01:11:08.959 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 236, 2024-06-10T01:11:08.959 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 320, 2024-06-10T01:11:08.959 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 197536, 2024-06-10T01:11:08.959 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-06-10T01:11:08.959 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 2, 2024-06-10T01:11:08.959 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.017301939, 2024-06-10T01:11:08.960 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.012771633, 2024-06-10T01:11:08.960 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.001667769, 2024-06-10T01:11:08.960 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T01:11:08.960 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:08.960 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 234, 2024-06-10T01:11:08.960 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.011174678, 2024-06-10T01:11:08.960 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000047755 2024-06-10T01:11:08.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:08.961 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:08.961 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-06-10T01:11:08.961 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-06-10T01:11:08.961 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-06-10T01:11:08.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:08.961 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-10T01:11:08.961 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 197536 2024-06-10T01:11:08.962 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:08.962 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T01:11:08.962 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 318, 2024-06-10T01:11:08.962 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 311, 2024-06-10T01:11:08.962 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 783984, 2024-06-10T01:11:08.962 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 191088, 2024-06-10T01:11:08.962 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-06-10T01:11:08.962 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-06-10T01:11:08.963 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.046387719, 2024-06-10T01:11:08.963 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.018689052, 2024-06-10T01:11:08.963 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.022524346, 2024-06-10T01:11:08.963 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.006132109, 2024-06-10T01:11:08.963 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:08.963 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 311, 2024-06-10T01:11:08.963 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.013505658, 2024-06-10T01:11:08.964 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000043426 2024-06-10T01:11:08.964 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:08.964 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:08.964 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-06-10T01:11:08.964 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-06-10T01:11:08.964 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-06-10T01:11:08.964 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:08.964 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 783984, 2024-06-10T01:11:08.965 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 191088 2024-06-10T01:11:08.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:08.965 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-06-10T01:11:08.965 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-06-10T01:11:08.965 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-06-10T01:11:08.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:08.965 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-06-10T01:11:08.965 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-06-10T01:11:08.965 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-06-10T01:11:08.966 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T01:11:08.966 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-06-10T01:11:08.966 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T01:11:08.966 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-06-10T01:11:08.966 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T01:11:08.966 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T01:11:08.966 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T01:11:08.967 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T01:11:08.967 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T01:11:08.967 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-06-10T01:11:08.967 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T01:11:08.967 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T01:11:08.967 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-06-10T01:11:08.967 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-06-10T01:11:08.967 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-06-10T01:11:08.968 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-06-10T01:11:08.968 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T01:11:08.968 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T01:11:08.968 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T01:11:08.968 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T01:11:08.968 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T01:11:08.968 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-06-10T01:11:08.968 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-06-10T01:11:08.969 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-06-10T01:11:08.969 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T01:11:08.969 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T01:11:08.969 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-06-10T01:11:08.969 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-06-10T01:11:08.969 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-06-10T01:11:08.969 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-06-10T01:11:08.969 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T01:11:08.970 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-06-10T01:11:08.970 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T01:11:08.970 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-06-10T01:11:08.970 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148761, 2024-06-10T01:11:08.970 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 31, 2024-06-10T01:11:08.970 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-06-10T01:11:08.970 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-06-10T01:11:08.970 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-06-10T01:11:08.971 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-06-10T01:11:08.971 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-06-10T01:11:08.971 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-06-10T01:11:08.971 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-06-10T01:11:08.971 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-06-10T01:11:08.971 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-06-10T01:11:08.971 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-06-10T01:11:08.971 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T01:11:08.972 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-06-10T01:11:08.972 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-06-10T01:11:08.972 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-06-10T01:11:08.972 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-06-10T01:11:08.972 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-06-10T01:11:08.972 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-06-10T01:11:08.972 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-06-10T01:11:08.972 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-06-10T01:11:08.973 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-06-10T01:11:08.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-06-10T01:11:08.973 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T01:11:08.973 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-06-10T01:11:08.973 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-06-10T01:11:08.973 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-06-10T01:11:08.973 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 316, 2024-06-10T01:11:08.973 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 736610, 2024-06-10T01:11:08.974 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:08.974 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 316, 2024-06-10T01:11:08.974 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-06-10T01:11:08.974 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-06-10T01:11:08.974 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 316, 2024-06-10T01:11:08.974 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 736610, 2024-06-10T01:11:08.974 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-06-10T01:11:08.974 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-06-10T01:11:08.975 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-06-10T01:11:08.975 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-06-10T01:11:08.975 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:11:08.975 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-06-10T01:11:08.975 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-06-10T01:11:09.669 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-06-10T01:11:09.669 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-06-10T01:11:09.669 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-06-10T01:11:09.669 DEBUG:tasks.check_counter:Getting stats from b 2024-06-10T01:11:09.670 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph daemon mds.b perf dump 2024-06-10T01:11:09.919 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T01:11:10.359 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:10 smithi094 ceph-mon[36708]: pgmap v869: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T01:11:10.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:10 smithi167 ceph-mon[36917]: pgmap v869: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T01:11:10.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:10 smithi037 ceph-mon[32192]: pgmap v869: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T01:11:11.670 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-06-10T01:11:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T01:11:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 47, 2024-06-10T01:11:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 49, 2024-06-10T01:11:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 9466, 2024-06-10T01:11:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 304335, 2024-06-10T01:11:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 34, 2024-06-10T01:11:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 7, 2024-06-10T01:11:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.014701248, 2024-06-10T01:11:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.000949609, 2024-06-10T01:11:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.009678074, 2024-06-10T01:11:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000617319, 2024-06-10T01:11:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 47, 2024-06-10T01:11:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.029244512, 2024-06-10T01:11:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000622223 2024-06-10T01:11:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 24, 2024-06-10T01:11:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000002662, 2024-06-10T01:11:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000110 2024-06-10T01:11:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-10T01:11:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 0 2024-06-10T01:11:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T01:11:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 428, 2024-06-10T01:11:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 414, 2024-06-10T01:11:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 870290, 2024-06-10T01:11:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 261993, 2024-06-10T01:11:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 13, 2024-06-10T01:11:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 6, 2024-06-10T01:11:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.071034892, 2024-06-10T01:11:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.025954330, 2024-06-10T01:11:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.038140282, 2024-06-10T01:11:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.008110727, 2024-06-10T01:11:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 411, 2024-06-10T01:11:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.035384980, 2024-06-10T01:11:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000086094 2024-06-10T01:11:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 98, 2024-06-10T01:11:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000016866, 2024-06-10T01:11:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000172 2024-06-10T01:11:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 867168, 2024-06-10T01:11:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 207904 2024-06-10T01:11:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T01:11:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 2770, 2024-06-10T01:11:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 1869, 2024-06-10T01:11:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 10480881, 2024-06-10T01:11:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 1042057, 2024-06-10T01:11:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 9, 2024-06-10T01:11:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 4, 2024-06-10T01:11:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.194506035, 2024-06-10T01:11:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.074566867, 2024-06-10T01:11:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.163560493, 2024-06-10T01:11:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.006798253, 2024-06-10T01:11:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 1870, 2024-06-10T01:11:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.074517219, 2024-06-10T01:11:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000039848 2024-06-10T01:11:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 4269, 2024-06-10T01:11:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000754611, 2024-06-10T01:11:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000176 2024-06-10T01:11:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-06-10T01:11:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 556640 2024-06-10T01:11:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-06-10T01:11:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 1, 2024-06-10T01:11:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-06-10T01:11:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-MDSRank": { 2024-06-10T01:11:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-06-10T01:11:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-06-10T01:11:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 35, 2024-06-10T01:11:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.002044943, 2024-06-10T01:11:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000058426 2024-06-10T01:11:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-PurgeQueue": { 2024-06-10T01:11:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-06-10T01:11:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-06-10T01:11:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2, 2024-06-10T01:11:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000017438, 2024-06-10T01:11:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000008719 2024-06-10T01:11:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-06-10T01:11:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "request": 23, 2024-06-10T01:11:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "reply": 6, 2024-06-10T01:11:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "reply_latency": { 2024-06-10T01:11:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 6, 2024-06-10T01:11:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 3.609262306, 2024-06-10T01:11:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.601543717 2024-06-10T01:11:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_reply": 0, 2024-06-10T01:11:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "forward": 17, 2024-06-10T01:11:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_complete": 0, 2024-06-10T01:11:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_keys": 0, 2024-06-10T01:11:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_commit": 11, 2024-06-10T01:11:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_split": 0, 2024-06-10T01:11:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_merge": 0, 2024-06-10T01:11:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes": 19, 2024-06-10T01:11:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_top": 2, 2024-06-10T01:11:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_bottom": 17, 2024-06-10T01:11:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pin_tail": 0, 2024-06-10T01:11:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pinned": 15, 2024-06-10T01:11:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_expired": 14, 2024-06-10T01:11:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_with_caps": 2, 2024-06-10T01:11:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": 2, 2024-06-10T01:11:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: "subtrees": 3, 2024-06-10T01:11:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse": 57, 2024-06-10T01:11:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_hit": 27, 2024-06-10T01:11:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_forward": 15, 2024-06-10T01:11:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_discover": 4, 2024-06-10T01:11:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_dir_fetch": 0, 2024-06-10T01:11:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_remote_ino": 0, 2024-06-10T01:11:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_lock": 3, 2024-06-10T01:11:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "load_cent": 0, 2024-06-10T01:11:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "q": 0, 2024-06-10T01:11:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "exported": 0, 2024-06-10T01:11:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "exported_inodes": 0, 2024-06-10T01:11:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "imported": 0, 2024-06-10T01:11:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "imported_inodes": 0, 2024-06-10T01:11:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_dir_fetch": 0, 2024-06-10T01:11:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_backtrace_fetch": 0, 2024-06-10T01:11:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_peer_discover": 0, 2024-06-10T01:11:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rfiles": 4, 2024-06-10T01:11:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rbytes": 234, 2024-06-10T01:11:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rsnaps": 0, 2024-06-10T01:11:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_fetch": 0, 2024-06-10T01:11:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_set_tag": 0, 2024-06-10T01:11:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T01:11:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_inotable_repaired": 0, 2024-06-10T01:11:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_inodes": 0, 2024-06-10T01:11:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_base_inodes": 0, 2024-06-10T01:11:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dirfrag_rstats": 0, 2024-06-10T01:11:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_file_inodes": 0, 2024-06-10T01:11:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_inode_file_caps": 0, 2024-06-10T01:11:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_revoke": 6, 2024-06-10T01:11:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_grant": 6, 2024-06-10T01:11:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_trunc": 0, 2024-06-10T01:11:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T01:11:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T01:11:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps": 8, 2024-06-10T01:11:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps_dirty": 0, 2024-06-10T01:11:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_cap_release": 1, 2024-06-10T01:11:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "process_request_cap_release": 2 2024-06-10T01:11:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache": { 2024-06-10T01:11:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update": 0, 2024-06-10T01:11:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update_receipt": 6, 2024-06-10T01:11:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_try_discover": 4, 2024-06-10T01:11:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_send_discover": 7, 2024-06-10T01:11:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_handle_discover": 0, 2024-06-10T01:11:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays": 0, 2024-06-10T01:11:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_delayed": 0, 2024-06-10T01:11:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_enqueuing": 0, 2024-06-10T01:11:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_created": 0, 2024-06-10T01:11:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_enqueued": 0, 2024-06-10T01:11:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_reintegrated": 0, 2024-06-10T01:11:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_migrated": 0, 2024-06-10T01:11:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_processing": 0, 2024-06-10T01:11:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_enqueued": 0, 2024-06-10T01:11:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_prioritized": 0, 2024-06-10T01:11:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_started": 0, 2024-06-10T01:11:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_completed": 0, 2024-06-10T01:11:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_path": 10, 2024-06-10T01:11:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_inode": 72, 2024-06-10T01:11:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_enqueue_scrub": 0, 2024-06-10T01:11:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_exportdir": 0, 2024-06-10T01:11:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_flush": 0, 2024-06-10T01:11:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragmentdir": 0, 2024-06-10T01:11:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragstats": 0, 2024-06-10T01:11:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_inodestats": 0 2024-06-10T01:11:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log": { 2024-06-10T01:11:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "evlrg": 0, 2024-06-10T01:11:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "evadd": 5, 2024-06-10T01:11:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "evex": 0, 2024-06-10T01:11:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "evtrm": 0, 2024-06-10T01:11:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "ev": 5, 2024-06-10T01:11:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "evexg": 0, 2024-06-10T01:11:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "evexd": 0, 2024-06-10T01:11:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "segadd": 2, 2024-06-10T01:11:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "segex": 0, 2024-06-10T01:11:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "segtrm": 0, 2024-06-10T01:11:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "seg": 2, 2024-06-10T01:11:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "segmjr": 2, 2024-06-10T01:11:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "segexg": 0, 2024-06-10T01:11:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "segexd": 0, 2024-06-10T01:11:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "expos": 4194304, 2024-06-10T01:11:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "wrpos": 4195731, 2024-06-10T01:11:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "rdpos": 0, 2024-06-10T01:11:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "jlat": { 2024-06-10T01:11:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 4, 2024-06-10T01:11:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.007106405, 2024-06-10T01:11:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.001776601 2024-06-10T01:11:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "replayed": 0 2024-06-10T01:11:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_mem": { 2024-06-10T01:11:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: "ino": 23, 2024-06-10T01:11:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: "ino+": 33, 2024-06-10T01:11:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: "ino-": 10, 2024-06-10T01:11:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: "dir": 24, 2024-06-10T01:11:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: "dir+": 37, 2024-06-10T01:11:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: "dir-": 13, 2024-06-10T01:11:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: "dn": 19, 2024-06-10T01:11:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: "dn+": 33, 2024-06-10T01:11:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: "dn-": 14, 2024-06-10T01:11:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: "cap": 2, 2024-06-10T01:11:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: "cap+": 4, 2024-06-10T01:11:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: "cap-": 2, 2024-06-10T01:11:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: "rss": 57140, 2024-06-10T01:11:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: "heap": 223612 2024-06-10T01:11:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_server": { 2024-06-10T01:11:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_client_request": 28, 2024-06-10T01:11:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_server_request": 2, 2024-06-10T01:11:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_request": 23, 2024-06-10T01:11:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_session": 16, 2024-06-10T01:11:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_peer_request": 4, 2024-06-10T01:11:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: "req_create_latency": { 2024-06-10T01:11:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getattr_latency": { 2024-06-10T01:11:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2, 2024-06-10T01:11:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.001129079, 2024-06-10T01:11:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000564539 2024-06-10T01:11:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.702 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getfilelock_latency": { 2024-06-10T01:11:11.702 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.702 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.702 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.703 INFO:teuthology.orchestra.run.smithi094.stdout: "req_link_latency": { 2024-06-10T01:11:11.703 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.703 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.703 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookup_latency": { 2024-06-10T01:11:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 3, 2024-06-10T01:11:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 3.607932022, 2024-06-10T01:11:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 1.202644007 2024-06-10T01:11:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookuphash_latency": { 2024-06-10T01:11:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupino_latency": { 2024-06-10T01:11:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 1, 2024-06-10T01:11:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000201205, 2024-06-10T01:11:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000201205 2024-06-10T01:11:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupname_latency": { 2024-06-10T01:11:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupparent_latency": { 2024-06-10T01:11:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupsnap_latency": { 2024-06-10T01:11:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.708 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lssnap_latency": { 2024-06-10T01:11:11.708 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.708 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.708 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.708 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mkdir_latency": { 2024-06-10T01:11:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mknod_latency": { 2024-06-10T01:11:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mksnap_latency": { 2024-06-10T01:11:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: "req_open_latency": { 2024-06-10T01:11:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: "req_readdir_latency": { 2024-06-10T01:11:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rename_latency": { 2024-06-10T01:11:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: "req_renamesnap_latency": { 2024-06-10T01:11:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: "req_snapdiff_latency": { 2024-06-10T01:11:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.714 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.714 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmdir_latency": { 2024-06-10T01:11:11.714 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.714 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.714 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmsnap_latency": { 2024-06-10T01:11:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmxattr_latency": { 2024-06-10T01:11:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setattr_latency": { 2024-06-10T01:11:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setdirlayout_latency": { 2024-06-10T01:11:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setfilelock_latency": { 2024-06-10T01:11:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setlayout_latency": { 2024-06-10T01:11:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setxattr_latency": { 2024-06-10T01:11:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: "req_symlink_latency": { 2024-06-10T01:11:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: "req_unlink_latency": { 2024-06-10T01:11:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_revoke_eviction": 0, 2024-06-10T01:11:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_acquisition_throttle": 0, 2024-06-10T01:11:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getvxattr_latency": { 2024-06-10T01:11:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_sessions": { 2024-06-10T01:11:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: "session_count": 1, 2024-06-10T01:11:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: "session_add": 2, 2024-06-10T01:11:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: "session_remove": 1, 2024-06-10T01:11:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_open": 1, 2024-06-10T01:11:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_stale": 0, 2024-06-10T01:11:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: "total_load": 2, 2024-06-10T01:11:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: "average_load": 2, 2024-06-10T01:11:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: "avg_session_uptime": 881, 2024-06-10T01:11:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: "mdthresh_evicted": 0 2024-06-10T01:11:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-06-10T01:11:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-06-10T01:11:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-06-10T01:11:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T01:11:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-06-10T01:11:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T01:11:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-06-10T01:11:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T01:11:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T01:11:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T01:11:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T01:11:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T01:11:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-06-10T01:11:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T01:11:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T01:11:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-06-10T01:11:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-06-10T01:11:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-06-10T01:11:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-06-10T01:11:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T01:11:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T01:11:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T01:11:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T01:11:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T01:11:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-06-10T01:11:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-06-10T01:11:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-06-10T01:11:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T01:11:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T01:11:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-06-10T01:11:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-06-10T01:11:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-06-10T01:11:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-06-10T01:11:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T01:11:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-06-10T01:11:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T01:11:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-06-10T01:11:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 255457, 2024-06-10T01:11:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 90, 2024-06-10T01:11:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-06-10T01:11:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-06-10T01:11:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-06-10T01:11:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-06-10T01:11:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-06-10T01:11:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-06-10T01:11:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-06-10T01:11:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-06-10T01:11:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 9176, 2024-06-10T01:11:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 176, 2024-06-10T01:11:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T01:11:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-06-10T01:11:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-06-10T01:11:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-06-10T01:11:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 97135, 2024-06-10T01:11:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 415, 2024-06-10T01:11:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-06-10T01:11:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-06-10T01:11:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-06-10T01:11:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-06-10T01:11:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter": { 2024-06-10T01:11:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: "op_active": 0, 2024-06-10T01:11:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: "op_laggy": 0, 2024-06-10T01:11:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send": 45, 2024-06-10T01:11:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send_bytes": 12456, 2024-06-10T01:11:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: "op_resend": 33, 2024-06-10T01:11:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: "op_reply": 45, 2024-06-10T01:11:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: "op_latency": { 2024-06-10T01:11:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 45, 2024-06-10T01:11:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.251995694, 2024-06-10T01:11:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.005599904 2024-06-10T01:11:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: "op_inflight": 0, 2024-06-10T01:11:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: "oplen_avg": { 2024-06-10T01:11:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 45, 2024-06-10T01:11:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 72 2024-06-10T01:11:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: "op": 45, 2024-06-10T01:11:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: "op_r": 0, 2024-06-10T01:11:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: "op_w": 45, 2024-06-10T01:11:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: "op_rmw": 0, 2024-06-10T01:11:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: "op_pg": 0, 2024-06-10T01:11:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_stat": 0, 2024-06-10T01:11:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_create": 10, 2024-06-10T01:11:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_read": 0, 2024-06-10T01:11:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_write": 4, 2024-06-10T01:11:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writefull": 7, 2024-06-10T01:11:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writesame": 0, 2024-06-10T01:11:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_append": 0, 2024-06-10T01:11:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_zero": 0, 2024-06-10T01:11:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_truncate": 0, 2024-06-10T01:11:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_delete": 6, 2024-06-10T01:11:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_mapext": 0, 2024-06-10T01:11:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_sparse_read": 0, 2024-06-10T01:11:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_clonerange": 0, 2024-06-10T01:11:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_getxattr": 0, 2024-06-10T01:11:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_setxattr": 20, 2024-06-10T01:11:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_cmpxattr": 0, 2024-06-10T01:11:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_rmxattr": 0, 2024-06-10T01:11:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_resetxattrs": 0, 2024-06-10T01:11:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_call": 0, 2024-06-10T01:11:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_watch": 0, 2024-06-10T01:11:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_notify": 0, 2024-06-10T01:11:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T01:11:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls": 0, 2024-06-10T01:11:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls_filter": 0, 2024-06-10T01:11:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_other": 0, 2024-06-10T01:11:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_active": 0, 2024-06-10T01:11:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_send": 0, 2024-06-10T01:11:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_resend": 0, 2024-06-10T01:11:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_ping": 0, 2024-06-10T01:11:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_active": 0, 2024-06-10T01:11:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_send": 0, 2024-06-10T01:11:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_resend": 0, 2024-06-10T01:11:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_active": 0, 2024-06-10T01:11:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_send": 0, 2024-06-10T01:11:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_resend": 0, 2024-06-10T01:11:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_active": 0, 2024-06-10T01:11:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_send": 0, 2024-06-10T01:11:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_resend": 0, 2024-06-10T01:11:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "command_active": 0, 2024-06-10T01:11:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "command_send": 0, 2024-06-10T01:11:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "command_resend": 0, 2024-06-10T01:11:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "map_epoch": 105, 2024-06-10T01:11:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "map_full": 0, 2024-06-10T01:11:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "map_inc": 7, 2024-06-10T01:11:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_sessions": 10, 2024-06-10T01:11:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_open": 14, 2024-06-10T01:11:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_close": 4, 2024-06-10T01:11:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_laggy": 0, 2024-06-10T01:11:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_wr": 22, 2024-06-10T01:11:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_rd": 0, 2024-06-10T01:11:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_del": 3 2024-06-10T01:11:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "oft": { 2024-06-10T01:11:11.740 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_objs": 1, 2024-06-10T01:11:11.740 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_kv_pairs": 0, 2024-06-10T01:11:11.740 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_updates": 0, 2024-06-10T01:11:11.740 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_removes": 0 2024-06-10T01:11:11.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.740 INFO:teuthology.orchestra.run.smithi094.stdout: "osd-slow-ops": { 2024-06-10T01:11:11.740 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_ops_count": 82 2024-06-10T01:11:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: "purge_queue": { 2024-06-10T01:11:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops": 0, 2024-06-10T01:11:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops_high_water": 0, 2024-06-10T01:11:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing": 0, 2024-06-10T01:11:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_high_water": 0, 2024-06-10T01:11:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed_ops": 0, 2024-06-10T01:11:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed": 0, 2024-06-10T01:11:11.742 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_item_in_journal": 0 2024-06-10T01:11:11.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.742 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T01:11:11.742 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-06-10T01:11:11.742 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-06-10T01:11:11.742 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-06-10T01:11:11.742 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 3241, 2024-06-10T01:11:11.742 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 11045193, 2024-06-10T01:11:11.743 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:11.743 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 3241, 2024-06-10T01:11:11.743 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-06-10T01:11:11.743 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-06-10T01:11:11.743 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 3241, 2024-06-10T01:11:11.743 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 11045193, 2024-06-10T01:11:11.743 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-06-10T01:11:11.743 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.744 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.744 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.744 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:11.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.744 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_bytes": { 2024-06-10T01:11:11.744 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-06-10T01:11:11.744 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-06-10T01:11:11.744 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-06-10T01:11:11.744 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-06-10T01:11:11.745 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-06-10T01:11:11.745 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:11.745 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-06-10T01:11:11.745 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 45, 2024-06-10T01:11:11.745 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 12456, 2024-06-10T01:11:11.745 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 45, 2024-06-10T01:11:11.745 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 12456, 2024-06-10T01:11:11.745 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-06-10T01:11:11.746 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.746 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.746 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.746 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:11.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.746 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_ops": { 2024-06-10T01:11:11.746 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-06-10T01:11:11.746 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 1024, 2024-06-10T01:11:11.747 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-06-10T01:11:11.747 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-06-10T01:11:11.747 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-06-10T01:11:11.747 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:11.747 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-06-10T01:11:11.747 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 45, 2024-06-10T01:11:11.747 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 45, 2024-06-10T01:11:11.747 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 45, 2024-06-10T01:11:11.748 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 45, 2024-06-10T01:11:11.748 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-06-10T01:11:11.748 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.748 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.748 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.748 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:11.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.748 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle": { 2024-06-10T01:11:11.749 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-06-10T01:11:11.749 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-06-10T01:11:11.749 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-06-10T01:11:11.749 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-06-10T01:11:11.749 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-06-10T01:11:11.749 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:11.749 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-06-10T01:11:11.749 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-06-10T01:11:11.749 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-06-10T01:11:11.750 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 0, 2024-06-10T01:11:11.750 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 0, 2024-06-10T01:11:11.750 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-06-10T01:11:11.750 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.750 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.750 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.750 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:11.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:11.751 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle-0x5563863d2500": { 2024-06-10T01:11:11.751 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-06-10T01:11:11.751 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-06-10T01:11:11.751 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-06-10T01:11:11.751 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 5, 2024-06-10T01:11:11.751 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 1427, 2024-06-10T01:11:11.751 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:11.752 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 5, 2024-06-10T01:11:11.752 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-06-10T01:11:11.752 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-06-10T01:11:11.752 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 4, 2024-06-10T01:11:11.752 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1427, 2024-06-10T01:11:11.752 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-06-10T01:11:11.752 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:11.752 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:11.753 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:11.753 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:11.753 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:11.753 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-06-10T01:11:12.426 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:12 smithi094 ceph-mon[36708]: pgmap v870: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:11:12.453 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-06-10T01:11:12.454 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-06-10T01:11:12.454 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=0 2024-06-10T01:11:12.454 DEBUG:tasks.check_counter:Getting stats from e 2024-06-10T01:11:12.454 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph daemon mds.e perf dump 2024-06-10T01:11:12.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:12 smithi167 ceph-mon[36917]: pgmap v870: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:11:12.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:12 smithi037 ceph-mon[32192]: pgmap v870: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:11:12.704 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T01:11:14.449 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:14 smithi094 ceph-mon[36708]: pgmap v871: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:11:14.485 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-06-10T01:11:14.485 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T01:11:14.485 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 1058, 2024-06-10T01:11:14.486 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 1058, 2024-06-10T01:11:14.486 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 280675, 2024-06-10T01:11:14.486 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 335684, 2024-06-10T01:11:14.486 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 7, 2024-06-10T01:11:14.486 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 2, 2024-06-10T01:11:14.486 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.093325960, 2024-06-10T01:11:14.486 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.037222253, 2024-06-10T01:11:14.487 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.022251541, 2024-06-10T01:11:14.487 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.024425684, 2024-06-10T01:11:14.487 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:14.487 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 1057, 2024-06-10T01:11:14.488 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.038458222, 2024-06-10T01:11:14.488 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000036384 2024-06-10T01:11:14.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.488 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:14.488 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.488 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.488 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.489 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-06-10T01:11:14.489 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 448 2024-06-10T01:11:14.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.489 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T01:11:14.489 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 1177, 2024-06-10T01:11:14.489 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 1163, 2024-06-10T01:11:14.489 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 1045016, 2024-06-10T01:11:14.490 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 460127, 2024-06-10T01:11:14.490 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 4, 2024-06-10T01:11:14.490 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 3, 2024-06-10T01:11:14.490 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.121638754, 2024-06-10T01:11:14.490 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.041206852, 2024-06-10T01:11:14.490 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.039222578, 2024-06-10T01:11:14.490 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.035485667, 2024-06-10T01:11:14.490 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:14.491 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 1162, 2024-06-10T01:11:14.491 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.044982144, 2024-06-10T01:11:14.491 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000038710 2024-06-10T01:11:14.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.491 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:14.491 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.491 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.492 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.492 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 837616, 2024-06-10T01:11:14.492 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 192048 2024-06-10T01:11:14.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.492 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T01:11:14.493 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 1051, 2024-06-10T01:11:14.493 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 1285, 2024-06-10T01:11:14.493 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 352784, 2024-06-10T01:11:14.493 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 890654, 2024-06-10T01:11:14.493 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 5, 2024-06-10T01:11:14.493 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 3, 2024-06-10T01:11:14.494 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.135223967, 2024-06-10T01:11:14.494 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.047264922, 2024-06-10T01:11:14.494 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.032170245, 2024-06-10T01:11:14.494 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.041472124, 2024-06-10T01:11:14.494 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:14.495 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 1283, 2024-06-10T01:11:14.495 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.077893022, 2024-06-10T01:11:14.495 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000060711 2024-06-10T01:11:14.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.496 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:14.496 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.496 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.496 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.497 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-10T01:11:14.497 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 557072 2024-06-10T01:11:14.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.497 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-06-10T01:11:14.497 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 1, 2024-06-10T01:11:14.497 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-06-10T01:11:14.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.498 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-MDSRank": { 2024-06-10T01:11:14.498 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-06-10T01:11:14.498 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-06-10T01:11:14.498 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 4962, 2024-06-10T01:11:14.499 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.139434050, 2024-06-10T01:11:14.499 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000028100 2024-06-10T01:11:14.499 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:14.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.499 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-PurgeQueue": { 2024-06-10T01:11:14.499 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-06-10T01:11:14.499 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-06-10T01:11:14.499 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.500 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.500 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.500 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:14.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.500 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-06-10T01:11:14.500 INFO:teuthology.orchestra.run.smithi094.stdout: "request": 0, 2024-06-10T01:11:14.500 INFO:teuthology.orchestra.run.smithi094.stdout: "reply": 0, 2024-06-10T01:11:14.500 INFO:teuthology.orchestra.run.smithi094.stdout: "reply_latency": { 2024-06-10T01:11:14.501 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.501 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.501 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.501 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_reply": 0, 2024-06-10T01:11:14.501 INFO:teuthology.orchestra.run.smithi094.stdout: "forward": 0, 2024-06-10T01:11:14.501 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_complete": 0, 2024-06-10T01:11:14.501 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_keys": 0, 2024-06-10T01:11:14.502 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_commit": 0, 2024-06-10T01:11:14.502 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_split": 0, 2024-06-10T01:11:14.502 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_merge": 0, 2024-06-10T01:11:14.502 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes": 0, 2024-06-10T01:11:14.502 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_top": 0, 2024-06-10T01:11:14.502 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_bottom": 0, 2024-06-10T01:11:14.502 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pin_tail": 0, 2024-06-10T01:11:14.502 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pinned": 0, 2024-06-10T01:11:14.503 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_expired": 0, 2024-06-10T01:11:14.503 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_with_caps": 0, 2024-06-10T01:11:14.503 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": 0, 2024-06-10T01:11:14.503 INFO:teuthology.orchestra.run.smithi094.stdout: "subtrees": 1, 2024-06-10T01:11:14.503 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse": 0, 2024-06-10T01:11:14.503 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_hit": 0, 2024-06-10T01:11:14.503 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_forward": 0, 2024-06-10T01:11:14.503 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_discover": 0, 2024-06-10T01:11:14.504 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_dir_fetch": 0, 2024-06-10T01:11:14.504 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_remote_ino": 0, 2024-06-10T01:11:14.504 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_lock": 0, 2024-06-10T01:11:14.504 INFO:teuthology.orchestra.run.smithi094.stdout: "load_cent": 0, 2024-06-10T01:11:14.504 INFO:teuthology.orchestra.run.smithi094.stdout: "q": 0, 2024-06-10T01:11:14.504 INFO:teuthology.orchestra.run.smithi094.stdout: "exported": 0, 2024-06-10T01:11:14.504 INFO:teuthology.orchestra.run.smithi094.stdout: "exported_inodes": 0, 2024-06-10T01:11:14.504 INFO:teuthology.orchestra.run.smithi094.stdout: "imported": 0, 2024-06-10T01:11:14.505 INFO:teuthology.orchestra.run.smithi094.stdout: "imported_inodes": 0, 2024-06-10T01:11:14.505 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_dir_fetch": 0, 2024-06-10T01:11:14.505 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_backtrace_fetch": 0, 2024-06-10T01:11:14.505 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_peer_discover": 0, 2024-06-10T01:11:14.505 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rfiles": 0, 2024-06-10T01:11:14.505 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rbytes": 0, 2024-06-10T01:11:14.505 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rsnaps": 0, 2024-06-10T01:11:14.505 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_fetch": 0, 2024-06-10T01:11:14.506 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_set_tag": 0, 2024-06-10T01:11:14.506 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T01:11:14.506 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_inotable_repaired": 0, 2024-06-10T01:11:14.506 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_inodes": 0, 2024-06-10T01:11:14.506 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_base_inodes": 0, 2024-06-10T01:11:14.506 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dirfrag_rstats": 0, 2024-06-10T01:11:14.506 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_file_inodes": 0, 2024-06-10T01:11:14.507 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_inode_file_caps": 0, 2024-06-10T01:11:14.507 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_revoke": 0, 2024-06-10T01:11:14.507 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_grant": 0, 2024-06-10T01:11:14.507 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_trunc": 0, 2024-06-10T01:11:14.507 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T01:11:14.507 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T01:11:14.507 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps": 0, 2024-06-10T01:11:14.507 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps_dirty": 0, 2024-06-10T01:11:14.508 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_cap_release": 0, 2024-06-10T01:11:14.508 INFO:teuthology.orchestra.run.smithi094.stdout: "process_request_cap_release": 0 2024-06-10T01:11:14.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.508 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache": { 2024-06-10T01:11:14.508 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update": 0, 2024-06-10T01:11:14.508 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update_receipt": 0, 2024-06-10T01:11:14.508 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_try_discover": 0, 2024-06-10T01:11:14.508 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_send_discover": 0, 2024-06-10T01:11:14.509 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_handle_discover": 0, 2024-06-10T01:11:14.509 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays": 0, 2024-06-10T01:11:14.509 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_delayed": 0, 2024-06-10T01:11:14.509 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_enqueuing": 0, 2024-06-10T01:11:14.509 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_created": 0, 2024-06-10T01:11:14.509 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_enqueued": 0, 2024-06-10T01:11:14.509 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_reintegrated": 0, 2024-06-10T01:11:14.509 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_migrated": 0, 2024-06-10T01:11:14.510 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_processing": 0, 2024-06-10T01:11:14.510 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_enqueued": 0, 2024-06-10T01:11:14.510 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_prioritized": 0, 2024-06-10T01:11:14.510 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_started": 0, 2024-06-10T01:11:14.510 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_completed": 0, 2024-06-10T01:11:14.510 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_path": 0, 2024-06-10T01:11:14.510 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_inode": 0, 2024-06-10T01:11:14.511 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_enqueue_scrub": 0, 2024-06-10T01:11:14.511 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_exportdir": 0, 2024-06-10T01:11:14.511 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_flush": 0, 2024-06-10T01:11:14.511 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragmentdir": 0, 2024-06-10T01:11:14.511 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragstats": 0, 2024-06-10T01:11:14.511 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_inodestats": 0 2024-06-10T01:11:14.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.512 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log": { 2024-06-10T01:11:14.512 INFO:teuthology.orchestra.run.smithi094.stdout: "evlrg": 0, 2024-06-10T01:11:14.512 INFO:teuthology.orchestra.run.smithi094.stdout: "evadd": 0, 2024-06-10T01:11:14.512 INFO:teuthology.orchestra.run.smithi094.stdout: "evex": 0, 2024-06-10T01:11:14.512 INFO:teuthology.orchestra.run.smithi094.stdout: "evtrm": 0, 2024-06-10T01:11:14.512 INFO:teuthology.orchestra.run.smithi094.stdout: "ev": 3, 2024-06-10T01:11:14.512 INFO:teuthology.orchestra.run.smithi094.stdout: "evexg": 0, 2024-06-10T01:11:14.513 INFO:teuthology.orchestra.run.smithi094.stdout: "evexd": 0, 2024-06-10T01:11:14.513 INFO:teuthology.orchestra.run.smithi094.stdout: "segadd": 0, 2024-06-10T01:11:14.513 INFO:teuthology.orchestra.run.smithi094.stdout: "segex": 0, 2024-06-10T01:11:14.513 INFO:teuthology.orchestra.run.smithi094.stdout: "segtrm": 0, 2024-06-10T01:11:14.513 INFO:teuthology.orchestra.run.smithi094.stdout: "seg": 2, 2024-06-10T01:11:14.513 INFO:teuthology.orchestra.run.smithi094.stdout: "segmjr": 2, 2024-06-10T01:11:14.513 INFO:teuthology.orchestra.run.smithi094.stdout: "segexg": 0, 2024-06-10T01:11:14.513 INFO:teuthology.orchestra.run.smithi094.stdout: "segexd": 0, 2024-06-10T01:11:14.514 INFO:teuthology.orchestra.run.smithi094.stdout: "expos": 4194304, 2024-06-10T01:11:14.514 INFO:teuthology.orchestra.run.smithi094.stdout: "wrpos": 4195200, 2024-06-10T01:11:14.514 INFO:teuthology.orchestra.run.smithi094.stdout: "rdpos": 4194892, 2024-06-10T01:11:14.514 INFO:teuthology.orchestra.run.smithi094.stdout: "jlat": { 2024-06-10T01:11:14.514 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.514 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.514 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.515 INFO:teuthology.orchestra.run.smithi094.stdout: "replayed": 3 2024-06-10T01:11:14.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.515 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_mem": { 2024-06-10T01:11:14.515 INFO:teuthology.orchestra.run.smithi094.stdout: "ino": 3, 2024-06-10T01:11:14.515 INFO:teuthology.orchestra.run.smithi094.stdout: "ino+": 3, 2024-06-10T01:11:14.515 INFO:teuthology.orchestra.run.smithi094.stdout: "ino-": 0, 2024-06-10T01:11:14.515 INFO:teuthology.orchestra.run.smithi094.stdout: "dir": 1, 2024-06-10T01:11:14.515 INFO:teuthology.orchestra.run.smithi094.stdout: "dir+": 1, 2024-06-10T01:11:14.516 INFO:teuthology.orchestra.run.smithi094.stdout: "dir-": 0, 2024-06-10T01:11:14.516 INFO:teuthology.orchestra.run.smithi094.stdout: "dn": 0, 2024-06-10T01:11:14.516 INFO:teuthology.orchestra.run.smithi094.stdout: "dn+": 0, 2024-06-10T01:11:14.516 INFO:teuthology.orchestra.run.smithi094.stdout: "dn-": 0, 2024-06-10T01:11:14.516 INFO:teuthology.orchestra.run.smithi094.stdout: "cap": 0, 2024-06-10T01:11:14.516 INFO:teuthology.orchestra.run.smithi094.stdout: "cap+": 0, 2024-06-10T01:11:14.516 INFO:teuthology.orchestra.run.smithi094.stdout: "cap-": 0, 2024-06-10T01:11:14.517 INFO:teuthology.orchestra.run.smithi094.stdout: "rss": 50376, 2024-06-10T01:11:14.517 INFO:teuthology.orchestra.run.smithi094.stdout: "heap": 215420 2024-06-10T01:11:14.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.517 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_server": { 2024-06-10T01:11:14.517 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_client_request": 0, 2024-06-10T01:11:14.517 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_server_request": 0, 2024-06-10T01:11:14.517 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_request": 0, 2024-06-10T01:11:14.517 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_session": 0, 2024-06-10T01:11:14.517 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_peer_request": 0, 2024-06-10T01:11:14.518 INFO:teuthology.orchestra.run.smithi094.stdout: "req_create_latency": { 2024-06-10T01:11:14.518 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.518 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.518 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.518 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getattr_latency": { 2024-06-10T01:11:14.519 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.519 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.519 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.519 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getfilelock_latency": { 2024-06-10T01:11:14.519 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.520 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.520 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.520 INFO:teuthology.orchestra.run.smithi094.stdout: "req_link_latency": { 2024-06-10T01:11:14.520 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.520 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.520 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.521 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookup_latency": { 2024-06-10T01:11:14.521 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.521 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.521 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.522 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookuphash_latency": { 2024-06-10T01:11:14.522 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.522 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.522 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.522 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupino_latency": { 2024-06-10T01:11:14.523 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.523 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.523 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.523 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupname_latency": { 2024-06-10T01:11:14.523 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.523 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.523 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.524 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupparent_latency": { 2024-06-10T01:11:14.524 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.524 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.524 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.525 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupsnap_latency": { 2024-06-10T01:11:14.525 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.525 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.525 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.525 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lssnap_latency": { 2024-06-10T01:11:14.525 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.525 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.526 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.526 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mkdir_latency": { 2024-06-10T01:11:14.526 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.527 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.527 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.527 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mknod_latency": { 2024-06-10T01:11:14.528 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.528 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.528 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.529 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mksnap_latency": { 2024-06-10T01:11:14.529 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.529 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.530 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.530 INFO:teuthology.orchestra.run.smithi094.stdout: "req_open_latency": { 2024-06-10T01:11:14.530 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.530 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.531 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.531 INFO:teuthology.orchestra.run.smithi094.stdout: "req_readdir_latency": { 2024-06-10T01:11:14.531 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.531 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.531 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.532 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rename_latency": { 2024-06-10T01:11:14.532 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.532 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.532 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.533 INFO:teuthology.orchestra.run.smithi094.stdout: "req_renamesnap_latency": { 2024-06-10T01:11:14.533 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.533 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.533 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.534 INFO:teuthology.orchestra.run.smithi094.stdout: "req_snapdiff_latency": { 2024-06-10T01:11:14.534 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.534 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.534 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.535 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmdir_latency": { 2024-06-10T01:11:14.535 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.535 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.535 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.535 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmsnap_latency": { 2024-06-10T01:11:14.536 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.536 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.536 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.536 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmxattr_latency": { 2024-06-10T01:11:14.536 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.536 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.536 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.537 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setattr_latency": { 2024-06-10T01:11:14.537 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.537 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.537 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.537 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setdirlayout_latency": { 2024-06-10T01:11:14.538 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.538 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.538 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.538 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setfilelock_latency": { 2024-06-10T01:11:14.541 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.541 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.542 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.542 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setlayout_latency": { 2024-06-10T01:11:14.543 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.543 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.543 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.544 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setxattr_latency": { 2024-06-10T01:11:14.544 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.544 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.545 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.545 INFO:teuthology.orchestra.run.smithi094.stdout: "req_symlink_latency": { 2024-06-10T01:11:14.546 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.546 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.546 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.547 INFO:teuthology.orchestra.run.smithi094.stdout: "req_unlink_latency": { 2024-06-10T01:11:14.548 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.548 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.548 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.549 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_revoke_eviction": 0, 2024-06-10T01:11:14.549 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_acquisition_throttle": 0, 2024-06-10T01:11:14.549 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getvxattr_latency": { 2024-06-10T01:11:14.550 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.550 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.550 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.551 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:14.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.551 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_sessions": { 2024-06-10T01:11:14.551 INFO:teuthology.orchestra.run.smithi094.stdout: "session_count": 1, 2024-06-10T01:11:14.552 INFO:teuthology.orchestra.run.smithi094.stdout: "session_add": 1, 2024-06-10T01:11:14.552 INFO:teuthology.orchestra.run.smithi094.stdout: "session_remove": 0, 2024-06-10T01:11:14.552 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_open": 1, 2024-06-10T01:11:14.552 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_stale": 0, 2024-06-10T01:11:14.553 INFO:teuthology.orchestra.run.smithi094.stdout: "total_load": 0, 2024-06-10T01:11:14.553 INFO:teuthology.orchestra.run.smithi094.stdout: "average_load": 0, 2024-06-10T01:11:14.553 INFO:teuthology.orchestra.run.smithi094.stdout: "avg_session_uptime": 2437, 2024-06-10T01:11:14.554 INFO:teuthology.orchestra.run.smithi094.stdout: "mdthresh_evicted": 0 2024-06-10T01:11:14.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.554 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-06-10T01:11:14.554 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-06-10T01:11:14.555 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-06-10T01:11:14.555 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T01:11:14.555 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-06-10T01:11:14.556 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T01:11:14.556 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-06-10T01:11:14.556 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T01:11:14.556 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T01:11:14.557 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T01:11:14.557 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T01:11:14.557 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T01:11:14.558 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-06-10T01:11:14.558 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T01:11:14.558 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T01:11:14.558 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-06-10T01:11:14.559 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-06-10T01:11:14.559 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-06-10T01:11:14.560 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-06-10T01:11:14.560 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T01:11:14.561 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T01:11:14.561 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T01:11:14.561 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T01:11:14.562 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T01:11:14.562 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-06-10T01:11:14.563 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-06-10T01:11:14.563 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-06-10T01:11:14.563 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T01:11:14.563 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T01:11:14.563 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-06-10T01:11:14.563 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-06-10T01:11:14.564 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-06-10T01:11:14.564 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-06-10T01:11:14.564 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T01:11:14.564 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-06-10T01:11:14.564 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T01:11:14.564 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-06-10T01:11:14.564 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 190991, 2024-06-10T01:11:14.565 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 55, 2024-06-10T01:11:14.565 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-06-10T01:11:14.565 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-06-10T01:11:14.565 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-06-10T01:11:14.565 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-06-10T01:11:14.566 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-06-10T01:11:14.566 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-06-10T01:11:14.566 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-06-10T01:11:14.566 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-06-10T01:11:14.566 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 9176, 2024-06-10T01:11:14.567 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 176, 2024-06-10T01:11:14.567 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T01:11:14.567 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-06-10T01:11:14.567 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-06-10T01:11:14.567 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-06-10T01:11:14.567 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 8464, 2024-06-10T01:11:14.567 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 13, 2024-06-10T01:11:14.568 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-06-10T01:11:14.568 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-06-10T01:11:14.568 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-06-10T01:11:14.568 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-06-10T01:11:14.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.568 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter": { 2024-06-10T01:11:14.568 INFO:teuthology.orchestra.run.smithi094.stdout: "op_active": 0, 2024-06-10T01:11:14.569 INFO:teuthology.orchestra.run.smithi094.stdout: "op_laggy": 0, 2024-06-10T01:11:14.569 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send": 2946, 2024-06-10T01:11:14.569 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send_bytes": 21, 2024-06-10T01:11:14.569 INFO:teuthology.orchestra.run.smithi094.stdout: "op_resend": 3, 2024-06-10T01:11:14.569 INFO:teuthology.orchestra.run.smithi094.stdout: "op_reply": 2946, 2024-06-10T01:11:14.569 INFO:teuthology.orchestra.run.smithi094.stdout: "op_latency": { 2024-06-10T01:11:14.569 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2946, 2024-06-10T01:11:14.570 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 5.159911275, 2024-06-10T01:11:14.570 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.001751497 2024-06-10T01:11:14.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.570 INFO:teuthology.orchestra.run.smithi094.stdout: "op_inflight": 0, 2024-06-10T01:11:14.570 INFO:teuthology.orchestra.run.smithi094.stdout: "oplen_avg": { 2024-06-10T01:11:14.570 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2946, 2024-06-10T01:11:14.570 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 2947 2024-06-10T01:11:14.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.571 INFO:teuthology.orchestra.run.smithi094.stdout: "op": 2946, 2024-06-10T01:11:14.571 INFO:teuthology.orchestra.run.smithi094.stdout: "op_r": 2946, 2024-06-10T01:11:14.571 INFO:teuthology.orchestra.run.smithi094.stdout: "op_w": 0, 2024-06-10T01:11:14.571 INFO:teuthology.orchestra.run.smithi094.stdout: "op_rmw": 0, 2024-06-10T01:11:14.571 INFO:teuthology.orchestra.run.smithi094.stdout: "op_pg": 0, 2024-06-10T01:11:14.572 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_stat": 1890, 2024-06-10T01:11:14.572 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_create": 0, 2024-06-10T01:11:14.572 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_read": 1054, 2024-06-10T01:11:14.572 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_write": 0, 2024-06-10T01:11:14.572 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writefull": 0, 2024-06-10T01:11:14.572 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writesame": 0, 2024-06-10T01:11:14.573 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_append": 0, 2024-06-10T01:11:14.573 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_zero": 0, 2024-06-10T01:11:14.573 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_truncate": 0, 2024-06-10T01:11:14.573 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_delete": 0, 2024-06-10T01:11:14.573 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_mapext": 0, 2024-06-10T01:11:14.573 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_sparse_read": 0, 2024-06-10T01:11:14.574 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_clonerange": 0, 2024-06-10T01:11:14.574 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_getxattr": 1, 2024-06-10T01:11:14.574 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_setxattr": 0, 2024-06-10T01:11:14.574 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_cmpxattr": 0, 2024-06-10T01:11:14.575 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_rmxattr": 0, 2024-06-10T01:11:14.575 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_resetxattrs": 0, 2024-06-10T01:11:14.575 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_call": 0, 2024-06-10T01:11:14.575 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_watch": 0, 2024-06-10T01:11:14.575 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_notify": 0, 2024-06-10T01:11:14.576 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T01:11:14.576 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls": 0, 2024-06-10T01:11:14.576 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls_filter": 0, 2024-06-10T01:11:14.576 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_other": 0, 2024-06-10T01:11:14.576 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_active": 0, 2024-06-10T01:11:14.576 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_send": 0, 2024-06-10T01:11:14.576 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_resend": 0, 2024-06-10T01:11:14.577 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_ping": 0, 2024-06-10T01:11:14.577 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_active": 0, 2024-06-10T01:11:14.577 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_send": 0, 2024-06-10T01:11:14.577 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_resend": 0, 2024-06-10T01:11:14.577 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_active": 0, 2024-06-10T01:11:14.577 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_send": 0, 2024-06-10T01:11:14.577 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_resend": 0, 2024-06-10T01:11:14.577 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_active": 0, 2024-06-10T01:11:14.578 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_send": 0, 2024-06-10T01:11:14.578 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_resend": 0, 2024-06-10T01:11:14.578 INFO:teuthology.orchestra.run.smithi094.stdout: "command_active": 0, 2024-06-10T01:11:14.578 INFO:teuthology.orchestra.run.smithi094.stdout: "command_send": 0, 2024-06-10T01:11:14.578 INFO:teuthology.orchestra.run.smithi094.stdout: "command_resend": 0, 2024-06-10T01:11:14.578 INFO:teuthology.orchestra.run.smithi094.stdout: "map_epoch": 105, 2024-06-10T01:11:14.578 INFO:teuthology.orchestra.run.smithi094.stdout: "map_full": 0, 2024-06-10T01:11:14.578 INFO:teuthology.orchestra.run.smithi094.stdout: "map_inc": 7, 2024-06-10T01:11:14.579 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_sessions": 5, 2024-06-10T01:11:14.579 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_open": 6, 2024-06-10T01:11:14.579 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_close": 1, 2024-06-10T01:11:14.579 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_laggy": 0, 2024-06-10T01:11:14.579 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_wr": 0, 2024-06-10T01:11:14.579 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_rd": 2, 2024-06-10T01:11:14.579 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_del": 0 2024-06-10T01:11:14.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.580 INFO:teuthology.orchestra.run.smithi094.stdout: "oft": { 2024-06-10T01:11:14.580 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_objs": 0, 2024-06-10T01:11:14.580 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_kv_pairs": 0, 2024-06-10T01:11:14.580 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_updates": 0, 2024-06-10T01:11:14.580 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_removes": 0 2024-06-10T01:11:14.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.580 INFO:teuthology.orchestra.run.smithi094.stdout: "osd-slow-ops": { 2024-06-10T01:11:14.580 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_ops_count": 0 2024-06-10T01:11:14.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.581 INFO:teuthology.orchestra.run.smithi094.stdout: "purge_queue": { 2024-06-10T01:11:14.581 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops": 0, 2024-06-10T01:11:14.581 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops_high_water": 0, 2024-06-10T01:11:14.581 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing": 0, 2024-06-10T01:11:14.581 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_high_water": 0, 2024-06-10T01:11:14.581 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed_ops": 0, 2024-06-10T01:11:14.581 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed": 0, 2024-06-10T01:11:14.582 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_item_in_journal": 0 2024-06-10T01:11:14.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.582 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T01:11:14.582 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-06-10T01:11:14.582 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-06-10T01:11:14.582 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-06-10T01:11:14.582 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 3282, 2024-06-10T01:11:14.582 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 1362932, 2024-06-10T01:11:14.583 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:14.583 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 3282, 2024-06-10T01:11:14.583 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-06-10T01:11:14.583 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-06-10T01:11:14.583 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 3282, 2024-06-10T01:11:14.583 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1362932, 2024-06-10T01:11:14.583 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-06-10T01:11:14.583 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.584 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.584 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.584 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:14.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.584 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_bytes": { 2024-06-10T01:11:14.584 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-06-10T01:11:14.584 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-06-10T01:11:14.584 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-06-10T01:11:14.585 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-06-10T01:11:14.585 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-06-10T01:11:14.585 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:14.585 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-06-10T01:11:14.585 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 2946, 2024-06-10T01:11:14.585 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 901, 2024-06-10T01:11:14.585 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 2946, 2024-06-10T01:11:14.585 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 901, 2024-06-10T01:11:14.586 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-06-10T01:11:14.586 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.586 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.586 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.586 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:14.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.586 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_ops": { 2024-06-10T01:11:14.586 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-06-10T01:11:14.587 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 1024, 2024-06-10T01:11:14.587 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-06-10T01:11:14.587 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-06-10T01:11:14.587 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-06-10T01:11:14.587 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:14.587 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-06-10T01:11:14.587 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 2946, 2024-06-10T01:11:14.587 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 2946, 2024-06-10T01:11:14.588 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 2946, 2024-06-10T01:11:14.588 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 2946, 2024-06-10T01:11:14.588 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-06-10T01:11:14.588 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.588 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.588 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.588 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:14.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.589 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle": { 2024-06-10T01:11:14.589 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-06-10T01:11:14.589 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-06-10T01:11:14.589 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-06-10T01:11:14.589 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-06-10T01:11:14.589 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-06-10T01:11:14.589 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:14.590 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-06-10T01:11:14.590 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-06-10T01:11:14.590 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-06-10T01:11:14.590 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 0, 2024-06-10T01:11:14.590 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 0, 2024-06-10T01:11:14.590 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-06-10T01:11:14.590 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.591 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.591 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.591 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:14.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:14.591 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle-0x5598868ec6e0": { 2024-06-10T01:11:14.591 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-06-10T01:11:14.592 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-06-10T01:11:14.592 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-06-10T01:11:14.592 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-06-10T01:11:14.592 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-06-10T01:11:14.592 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:14.592 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-06-10T01:11:14.592 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-06-10T01:11:14.592 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-06-10T01:11:14.593 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 0, 2024-06-10T01:11:14.593 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 0, 2024-06-10T01:11:14.593 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-06-10T01:11:14.593 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:14.593 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:14.593 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:14.593 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:14.594 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:14.594 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-06-10T01:11:14.595 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:14 smithi167 ceph-mon[36917]: pgmap v871: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:11:14.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:14 smithi037 ceph-mon[32192]: pgmap v871: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:11:15.260 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-06-10T01:11:15.260 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-06-10T01:11:15.260 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-06-10T01:11:15.261 DEBUG:tasks.check_counter:Getting stats from h 2024-06-10T01:11:15.261 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph daemon mds.h perf dump 2024-06-10T01:11:15.499 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T01:11:16.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:16 smithi167 ceph-mon[36917]: pgmap v872: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:11:16.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:16 smithi094 ceph-mon[36708]: pgmap v872: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:11:16.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:16 smithi037 ceph-mon[32192]: pgmap v872: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:11:17.171 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-06-10T01:11:17.171 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T01:11:17.172 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 1386, 2024-06-10T01:11:17.172 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 1372, 2024-06-10T01:11:17.172 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 1191152, 2024-06-10T01:11:17.172 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 526278, 2024-06-10T01:11:17.172 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 5, 2024-06-10T01:11:17.172 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 1, 2024-06-10T01:11:17.172 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.162962654, 2024-06-10T01:11:17.172 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.065279500, 2024-06-10T01:11:17.173 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.047807414, 2024-06-10T01:11:17.173 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.038271025, 2024-06-10T01:11:17.173 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:17.173 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 1371, 2024-06-10T01:11:17.173 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.071985401, 2024-06-10T01:11:17.173 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000052505 2024-06-10T01:11:17.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.173 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:17.174 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.174 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.174 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.174 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 837344, 2024-06-10T01:11:17.174 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 191424 2024-06-10T01:11:17.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.175 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T01:11:17.175 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 4, 2024-06-10T01:11:17.175 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 236, 2024-06-10T01:11:17.175 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 876, 2024-06-10T01:11:17.175 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 557418, 2024-06-10T01:11:17.175 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 6, 2024-06-10T01:11:17.175 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 3, 2024-06-10T01:11:17.175 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.013214096, 2024-06-10T01:11:17.175 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.009388850, 2024-06-10T01:11:17.176 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.001554582, 2024-06-10T01:11:17.176 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000062130, 2024-06-10T01:11:17.176 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:17.176 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 234, 2024-06-10T01:11:17.176 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.014674246, 2024-06-10T01:11:17.176 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000062710 2024-06-10T01:11:17.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.176 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:17.177 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.177 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.177 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.177 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-10T01:11:17.177 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 556720 2024-06-10T01:11:17.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.177 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T01:11:17.178 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 2018, 2024-06-10T01:11:17.178 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 2018, 2024-06-10T01:11:17.178 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 517388, 2024-06-10T01:11:17.178 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 640907, 2024-06-10T01:11:17.178 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 5, 2024-06-10T01:11:17.178 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 3, 2024-06-10T01:11:17.178 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.120272878, 2024-06-10T01:11:17.179 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.028583807, 2024-06-10T01:11:17.179 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.033642869, 2024-06-10T01:11:17.179 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.050574213, 2024-06-10T01:11:17.179 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:17.179 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2017, 2024-06-10T01:11:17.179 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.051182553, 2024-06-10T01:11:17.179 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000025375 2024-06-10T01:11:17.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.180 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:17.180 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.180 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.180 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.180 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-06-10T01:11:17.180 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 448 2024-06-10T01:11:17.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.181 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-06-10T01:11:17.181 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 1, 2024-06-10T01:11:17.181 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-06-10T01:11:17.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.181 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-MDSRank": { 2024-06-10T01:11:17.181 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-06-10T01:11:17.181 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-06-10T01:11:17.181 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 5133, 2024-06-10T01:11:17.182 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.119961500, 2024-06-10T01:11:17.182 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000023370 2024-06-10T01:11:17.182 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:17.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.182 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-PurgeQueue": { 2024-06-10T01:11:17.182 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-06-10T01:11:17.182 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-06-10T01:11:17.182 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.183 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.183 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.183 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:17.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.183 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-06-10T01:11:17.183 INFO:teuthology.orchestra.run.smithi094.stdout: "request": 0, 2024-06-10T01:11:17.183 INFO:teuthology.orchestra.run.smithi094.stdout: "reply": 0, 2024-06-10T01:11:17.183 INFO:teuthology.orchestra.run.smithi094.stdout: "reply_latency": { 2024-06-10T01:11:17.184 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.184 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.184 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.184 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_reply": 0, 2024-06-10T01:11:17.184 INFO:teuthology.orchestra.run.smithi094.stdout: "forward": 0, 2024-06-10T01:11:17.184 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_complete": 0, 2024-06-10T01:11:17.185 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_keys": 0, 2024-06-10T01:11:17.185 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_commit": 0, 2024-06-10T01:11:17.185 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_split": 0, 2024-06-10T01:11:17.185 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_merge": 0, 2024-06-10T01:11:17.185 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes": 0, 2024-06-10T01:11:17.185 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_top": 0, 2024-06-10T01:11:17.185 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_bottom": 0, 2024-06-10T01:11:17.185 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pin_tail": 0, 2024-06-10T01:11:17.185 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pinned": 0, 2024-06-10T01:11:17.186 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_expired": 0, 2024-06-10T01:11:17.186 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_with_caps": 0, 2024-06-10T01:11:17.186 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": 0, 2024-06-10T01:11:17.186 INFO:teuthology.orchestra.run.smithi094.stdout: "subtrees": 1, 2024-06-10T01:11:17.186 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse": 0, 2024-06-10T01:11:17.186 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_hit": 0, 2024-06-10T01:11:17.186 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_forward": 0, 2024-06-10T01:11:17.187 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_discover": 0, 2024-06-10T01:11:17.187 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_dir_fetch": 0, 2024-06-10T01:11:17.187 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_remote_ino": 0, 2024-06-10T01:11:17.187 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_lock": 0, 2024-06-10T01:11:17.187 INFO:teuthology.orchestra.run.smithi094.stdout: "load_cent": 0, 2024-06-10T01:11:17.187 INFO:teuthology.orchestra.run.smithi094.stdout: "q": 0, 2024-06-10T01:11:17.187 INFO:teuthology.orchestra.run.smithi094.stdout: "exported": 0, 2024-06-10T01:11:17.187 INFO:teuthology.orchestra.run.smithi094.stdout: "exported_inodes": 0, 2024-06-10T01:11:17.187 INFO:teuthology.orchestra.run.smithi094.stdout: "imported": 0, 2024-06-10T01:11:17.188 INFO:teuthology.orchestra.run.smithi094.stdout: "imported_inodes": 0, 2024-06-10T01:11:17.188 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_dir_fetch": 0, 2024-06-10T01:11:17.188 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_backtrace_fetch": 0, 2024-06-10T01:11:17.188 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_peer_discover": 0, 2024-06-10T01:11:17.188 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rfiles": 0, 2024-06-10T01:11:17.188 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rbytes": 0, 2024-06-10T01:11:17.188 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rsnaps": 0, 2024-06-10T01:11:17.188 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_fetch": 0, 2024-06-10T01:11:17.189 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_set_tag": 0, 2024-06-10T01:11:17.189 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T01:11:17.189 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_inotable_repaired": 0, 2024-06-10T01:11:17.189 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_inodes": 0, 2024-06-10T01:11:17.189 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_base_inodes": 0, 2024-06-10T01:11:17.189 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dirfrag_rstats": 0, 2024-06-10T01:11:17.189 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_file_inodes": 0, 2024-06-10T01:11:17.189 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_inode_file_caps": 0, 2024-06-10T01:11:17.190 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_revoke": 0, 2024-06-10T01:11:17.190 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_grant": 0, 2024-06-10T01:11:17.190 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_trunc": 0, 2024-06-10T01:11:17.190 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T01:11:17.190 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T01:11:17.190 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps": 0, 2024-06-10T01:11:17.190 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps_dirty": 0, 2024-06-10T01:11:17.191 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_cap_release": 0, 2024-06-10T01:11:17.191 INFO:teuthology.orchestra.run.smithi094.stdout: "process_request_cap_release": 0 2024-06-10T01:11:17.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.191 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache": { 2024-06-10T01:11:17.191 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update": 0, 2024-06-10T01:11:17.191 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update_receipt": 0, 2024-06-10T01:11:17.191 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_try_discover": 0, 2024-06-10T01:11:17.191 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_send_discover": 0, 2024-06-10T01:11:17.192 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_handle_discover": 0, 2024-06-10T01:11:17.192 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays": 0, 2024-06-10T01:11:17.192 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_delayed": 0, 2024-06-10T01:11:17.192 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_enqueuing": 0, 2024-06-10T01:11:17.192 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_created": 0, 2024-06-10T01:11:17.192 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_enqueued": 0, 2024-06-10T01:11:17.192 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_reintegrated": 0, 2024-06-10T01:11:17.192 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_migrated": 0, 2024-06-10T01:11:17.193 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_processing": 0, 2024-06-10T01:11:17.193 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_enqueued": 0, 2024-06-10T01:11:17.193 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_prioritized": 0, 2024-06-10T01:11:17.193 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_started": 0, 2024-06-10T01:11:17.193 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_completed": 0, 2024-06-10T01:11:17.193 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_path": 0, 2024-06-10T01:11:17.193 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_inode": 0, 2024-06-10T01:11:17.193 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_enqueue_scrub": 0, 2024-06-10T01:11:17.193 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_exportdir": 0, 2024-06-10T01:11:17.194 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_flush": 0, 2024-06-10T01:11:17.194 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragmentdir": 0, 2024-06-10T01:11:17.194 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragstats": 0, 2024-06-10T01:11:17.194 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_inodestats": 0 2024-06-10T01:11:17.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.194 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log": { 2024-06-10T01:11:17.194 INFO:teuthology.orchestra.run.smithi094.stdout: "evlrg": 0, 2024-06-10T01:11:17.194 INFO:teuthology.orchestra.run.smithi094.stdout: "evadd": 0, 2024-06-10T01:11:17.195 INFO:teuthology.orchestra.run.smithi094.stdout: "evex": 0, 2024-06-10T01:11:17.195 INFO:teuthology.orchestra.run.smithi094.stdout: "evtrm": 0, 2024-06-10T01:11:17.195 INFO:teuthology.orchestra.run.smithi094.stdout: "ev": 5, 2024-06-10T01:11:17.195 INFO:teuthology.orchestra.run.smithi094.stdout: "evexg": 0, 2024-06-10T01:11:17.195 INFO:teuthology.orchestra.run.smithi094.stdout: "evexd": 0, 2024-06-10T01:11:17.195 INFO:teuthology.orchestra.run.smithi094.stdout: "segadd": 0, 2024-06-10T01:11:17.195 INFO:teuthology.orchestra.run.smithi094.stdout: "segex": 0, 2024-06-10T01:11:17.195 INFO:teuthology.orchestra.run.smithi094.stdout: "segtrm": 0, 2024-06-10T01:11:17.196 INFO:teuthology.orchestra.run.smithi094.stdout: "seg": 2, 2024-06-10T01:11:17.196 INFO:teuthology.orchestra.run.smithi094.stdout: "segmjr": 2, 2024-06-10T01:11:17.196 INFO:teuthology.orchestra.run.smithi094.stdout: "segexg": 0, 2024-06-10T01:11:17.196 INFO:teuthology.orchestra.run.smithi094.stdout: "segexd": 0, 2024-06-10T01:11:17.196 INFO:teuthology.orchestra.run.smithi094.stdout: "expos": 4194304, 2024-06-10T01:11:17.196 INFO:teuthology.orchestra.run.smithi094.stdout: "wrpos": 4195731, 2024-06-10T01:11:17.196 INFO:teuthology.orchestra.run.smithi094.stdout: "rdpos": 4195423, 2024-06-10T01:11:17.196 INFO:teuthology.orchestra.run.smithi094.stdout: "jlat": { 2024-06-10T01:11:17.197 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.197 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.197 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.197 INFO:teuthology.orchestra.run.smithi094.stdout: "replayed": 5 2024-06-10T01:11:17.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.197 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_mem": { 2024-06-10T01:11:17.197 INFO:teuthology.orchestra.run.smithi094.stdout: "ino": 3, 2024-06-10T01:11:17.197 INFO:teuthology.orchestra.run.smithi094.stdout: "ino+": 3, 2024-06-10T01:11:17.198 INFO:teuthology.orchestra.run.smithi094.stdout: "ino-": 0, 2024-06-10T01:11:17.198 INFO:teuthology.orchestra.run.smithi094.stdout: "dir": 1, 2024-06-10T01:11:17.198 INFO:teuthology.orchestra.run.smithi094.stdout: "dir+": 1, 2024-06-10T01:11:17.198 INFO:teuthology.orchestra.run.smithi094.stdout: "dir-": 0, 2024-06-10T01:11:17.198 INFO:teuthology.orchestra.run.smithi094.stdout: "dn": 0, 2024-06-10T01:11:17.198 INFO:teuthology.orchestra.run.smithi094.stdout: "dn+": 0, 2024-06-10T01:11:17.198 INFO:teuthology.orchestra.run.smithi094.stdout: "dn-": 0, 2024-06-10T01:11:17.198 INFO:teuthology.orchestra.run.smithi094.stdout: "cap": 0, 2024-06-10T01:11:17.199 INFO:teuthology.orchestra.run.smithi094.stdout: "cap+": 0, 2024-06-10T01:11:17.199 INFO:teuthology.orchestra.run.smithi094.stdout: "cap-": 0, 2024-06-10T01:11:17.199 INFO:teuthology.orchestra.run.smithi094.stdout: "rss": 50436, 2024-06-10T01:11:17.199 INFO:teuthology.orchestra.run.smithi094.stdout: "heap": 215420 2024-06-10T01:11:17.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.199 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_server": { 2024-06-10T01:11:17.199 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_client_request": 0, 2024-06-10T01:11:17.199 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_server_request": 0, 2024-06-10T01:11:17.200 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_request": 0, 2024-06-10T01:11:17.200 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_session": 0, 2024-06-10T01:11:17.200 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_peer_request": 0, 2024-06-10T01:11:17.200 INFO:teuthology.orchestra.run.smithi094.stdout: "req_create_latency": { 2024-06-10T01:11:17.200 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.200 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.200 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.201 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getattr_latency": { 2024-06-10T01:11:17.201 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.201 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.201 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.201 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getfilelock_latency": { 2024-06-10T01:11:17.201 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.201 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.202 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.202 INFO:teuthology.orchestra.run.smithi094.stdout: "req_link_latency": { 2024-06-10T01:11:17.202 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.202 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.202 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.202 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookup_latency": { 2024-06-10T01:11:17.203 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.203 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.203 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.203 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookuphash_latency": { 2024-06-10T01:11:17.203 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.203 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.203 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.204 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupino_latency": { 2024-06-10T01:11:17.204 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.204 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.204 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.204 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupname_latency": { 2024-06-10T01:11:17.204 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.205 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.205 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.205 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupparent_latency": { 2024-06-10T01:11:17.205 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.205 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.205 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.206 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupsnap_latency": { 2024-06-10T01:11:17.206 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.206 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.206 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.206 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lssnap_latency": { 2024-06-10T01:11:17.206 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.206 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.207 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.207 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mkdir_latency": { 2024-06-10T01:11:17.207 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.208 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.208 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.208 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mknod_latency": { 2024-06-10T01:11:17.208 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.208 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.208 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.209 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mksnap_latency": { 2024-06-10T01:11:17.209 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.209 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.209 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.209 INFO:teuthology.orchestra.run.smithi094.stdout: "req_open_latency": { 2024-06-10T01:11:17.210 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.210 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.210 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.210 INFO:teuthology.orchestra.run.smithi094.stdout: "req_readdir_latency": { 2024-06-10T01:11:17.210 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.210 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.210 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.211 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rename_latency": { 2024-06-10T01:11:17.211 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.211 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.211 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.211 INFO:teuthology.orchestra.run.smithi094.stdout: "req_renamesnap_latency": { 2024-06-10T01:11:17.211 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.212 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.212 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.212 INFO:teuthology.orchestra.run.smithi094.stdout: "req_snapdiff_latency": { 2024-06-10T01:11:17.212 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.212 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.212 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.213 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmdir_latency": { 2024-06-10T01:11:17.213 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.213 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.213 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.213 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmsnap_latency": { 2024-06-10T01:11:17.213 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.213 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.213 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.214 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmxattr_latency": { 2024-06-10T01:11:17.214 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.214 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.214 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.214 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setattr_latency": { 2024-06-10T01:11:17.215 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.215 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.215 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.215 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setdirlayout_latency": { 2024-06-10T01:11:17.215 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.215 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.215 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.216 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setfilelock_latency": { 2024-06-10T01:11:17.216 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.216 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.216 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.216 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setlayout_latency": { 2024-06-10T01:11:17.216 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.217 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.217 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.217 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setxattr_latency": { 2024-06-10T01:11:17.217 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.217 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.217 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.217 INFO:teuthology.orchestra.run.smithi094.stdout: "req_symlink_latency": { 2024-06-10T01:11:17.218 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.218 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.218 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.218 INFO:teuthology.orchestra.run.smithi094.stdout: "req_unlink_latency": { 2024-06-10T01:11:17.218 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.218 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.219 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.219 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_revoke_eviction": 0, 2024-06-10T01:11:17.219 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_acquisition_throttle": 0, 2024-06-10T01:11:17.219 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getvxattr_latency": { 2024-06-10T01:11:17.219 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.219 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.219 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.219 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:17.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.220 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_sessions": { 2024-06-10T01:11:17.220 INFO:teuthology.orchestra.run.smithi094.stdout: "session_count": 1, 2024-06-10T01:11:17.220 INFO:teuthology.orchestra.run.smithi094.stdout: "session_add": 2, 2024-06-10T01:11:17.220 INFO:teuthology.orchestra.run.smithi094.stdout: "session_remove": 1, 2024-06-10T01:11:17.220 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_open": 1, 2024-06-10T01:11:17.220 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_stale": 0, 2024-06-10T01:11:17.220 INFO:teuthology.orchestra.run.smithi094.stdout: "total_load": 0, 2024-06-10T01:11:17.221 INFO:teuthology.orchestra.run.smithi094.stdout: "average_load": 0, 2024-06-10T01:11:17.221 INFO:teuthology.orchestra.run.smithi094.stdout: "avg_session_uptime": 2436, 2024-06-10T01:11:17.221 INFO:teuthology.orchestra.run.smithi094.stdout: "mdthresh_evicted": 0 2024-06-10T01:11:17.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.221 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-06-10T01:11:17.221 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-06-10T01:11:17.221 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-06-10T01:11:17.221 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T01:11:17.222 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-06-10T01:11:17.222 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T01:11:17.222 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-06-10T01:11:17.222 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T01:11:17.222 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T01:11:17.222 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T01:11:17.222 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T01:11:17.222 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T01:11:17.223 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-06-10T01:11:17.223 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T01:11:17.223 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T01:11:17.223 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-06-10T01:11:17.223 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-06-10T01:11:17.223 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-06-10T01:11:17.223 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-06-10T01:11:17.223 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T01:11:17.224 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T01:11:17.224 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T01:11:17.224 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T01:11:17.224 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T01:11:17.224 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-06-10T01:11:17.224 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-06-10T01:11:17.224 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-06-10T01:11:17.224 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T01:11:17.224 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T01:11:17.225 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-06-10T01:11:17.225 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-06-10T01:11:17.225 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-06-10T01:11:17.225 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-06-10T01:11:17.225 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T01:11:17.225 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-06-10T01:11:17.225 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T01:11:17.225 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-06-10T01:11:17.226 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 182087, 2024-06-10T01:11:17.226 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 48, 2024-06-10T01:11:17.226 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-06-10T01:11:17.226 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-06-10T01:11:17.226 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-06-10T01:11:17.226 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-06-10T01:11:17.226 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-06-10T01:11:17.226 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-06-10T01:11:17.227 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-06-10T01:11:17.227 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-06-10T01:11:17.227 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 9176, 2024-06-10T01:11:17.227 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 176, 2024-06-10T01:11:17.227 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T01:11:17.227 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-06-10T01:11:17.227 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-06-10T01:11:17.227 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-06-10T01:11:17.227 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 8464, 2024-06-10T01:11:17.228 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 13, 2024-06-10T01:11:17.228 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-06-10T01:11:17.228 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-06-10T01:11:17.228 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-06-10T01:11:17.228 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-06-10T01:11:17.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.228 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter": { 2024-06-10T01:11:17.228 INFO:teuthology.orchestra.run.smithi094.stdout: "op_active": 0, 2024-06-10T01:11:17.229 INFO:teuthology.orchestra.run.smithi094.stdout: "op_laggy": 0, 2024-06-10T01:11:17.229 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send": 3069, 2024-06-10T01:11:17.229 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send_bytes": 21, 2024-06-10T01:11:17.229 INFO:teuthology.orchestra.run.smithi094.stdout: "op_resend": 3, 2024-06-10T01:11:17.229 INFO:teuthology.orchestra.run.smithi094.stdout: "op_reply": 3069, 2024-06-10T01:11:17.229 INFO:teuthology.orchestra.run.smithi094.stdout: "op_latency": { 2024-06-10T01:11:17.229 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 3069, 2024-06-10T01:11:17.229 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 5.172911000, 2024-06-10T01:11:17.229 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.001685536 2024-06-10T01:11:17.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.230 INFO:teuthology.orchestra.run.smithi094.stdout: "op_inflight": 0, 2024-06-10T01:11:17.230 INFO:teuthology.orchestra.run.smithi094.stdout: "oplen_avg": { 2024-06-10T01:11:17.230 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 3069, 2024-06-10T01:11:17.230 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 3070 2024-06-10T01:11:17.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.230 INFO:teuthology.orchestra.run.smithi094.stdout: "op": 3069, 2024-06-10T01:11:17.231 INFO:teuthology.orchestra.run.smithi094.stdout: "op_r": 3069, 2024-06-10T01:11:17.231 INFO:teuthology.orchestra.run.smithi094.stdout: "op_w": 0, 2024-06-10T01:11:17.231 INFO:teuthology.orchestra.run.smithi094.stdout: "op_rmw": 0, 2024-06-10T01:11:17.231 INFO:teuthology.orchestra.run.smithi094.stdout: "op_pg": 0, 2024-06-10T01:11:17.231 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_stat": 2007, 2024-06-10T01:11:17.231 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_create": 0, 2024-06-10T01:11:17.231 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_read": 1060, 2024-06-10T01:11:17.232 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_write": 0, 2024-06-10T01:11:17.232 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writefull": 0, 2024-06-10T01:11:17.232 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writesame": 0, 2024-06-10T01:11:17.232 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_append": 0, 2024-06-10T01:11:17.232 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_zero": 0, 2024-06-10T01:11:17.232 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_truncate": 0, 2024-06-10T01:11:17.232 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_delete": 0, 2024-06-10T01:11:17.232 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_mapext": 0, 2024-06-10T01:11:17.233 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_sparse_read": 0, 2024-06-10T01:11:17.233 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_clonerange": 0, 2024-06-10T01:11:17.233 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_getxattr": 1, 2024-06-10T01:11:17.233 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_setxattr": 0, 2024-06-10T01:11:17.233 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_cmpxattr": 0, 2024-06-10T01:11:17.233 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_rmxattr": 0, 2024-06-10T01:11:17.233 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_resetxattrs": 0, 2024-06-10T01:11:17.233 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_call": 0, 2024-06-10T01:11:17.233 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_watch": 0, 2024-06-10T01:11:17.234 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_notify": 0, 2024-06-10T01:11:17.234 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T01:11:17.234 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls": 0, 2024-06-10T01:11:17.234 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls_filter": 0, 2024-06-10T01:11:17.234 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_other": 0, 2024-06-10T01:11:17.234 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_active": 0, 2024-06-10T01:11:17.234 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_send": 0, 2024-06-10T01:11:17.234 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_resend": 0, 2024-06-10T01:11:17.235 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_ping": 0, 2024-06-10T01:11:17.235 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_active": 0, 2024-06-10T01:11:17.235 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_send": 0, 2024-06-10T01:11:17.235 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_resend": 0, 2024-06-10T01:11:17.235 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_active": 0, 2024-06-10T01:11:17.235 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_send": 0, 2024-06-10T01:11:17.235 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_resend": 0, 2024-06-10T01:11:17.235 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_active": 0, 2024-06-10T01:11:17.235 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_send": 0, 2024-06-10T01:11:17.235 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_resend": 0, 2024-06-10T01:11:17.236 INFO:teuthology.orchestra.run.smithi094.stdout: "command_active": 0, 2024-06-10T01:11:17.236 INFO:teuthology.orchestra.run.smithi094.stdout: "command_send": 0, 2024-06-10T01:11:17.236 INFO:teuthology.orchestra.run.smithi094.stdout: "command_resend": 0, 2024-06-10T01:11:17.236 INFO:teuthology.orchestra.run.smithi094.stdout: "map_epoch": 105, 2024-06-10T01:11:17.236 INFO:teuthology.orchestra.run.smithi094.stdout: "map_full": 0, 2024-06-10T01:11:17.236 INFO:teuthology.orchestra.run.smithi094.stdout: "map_inc": 7, 2024-06-10T01:11:17.236 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_sessions": 4, 2024-06-10T01:11:17.236 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_open": 6, 2024-06-10T01:11:17.237 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_close": 2, 2024-06-10T01:11:17.237 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_laggy": 0, 2024-06-10T01:11:17.237 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_wr": 0, 2024-06-10T01:11:17.237 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_rd": 2, 2024-06-10T01:11:17.237 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_del": 0 2024-06-10T01:11:17.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.237 INFO:teuthology.orchestra.run.smithi094.stdout: "oft": { 2024-06-10T01:11:17.237 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_objs": 0, 2024-06-10T01:11:17.237 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_kv_pairs": 0, 2024-06-10T01:11:17.238 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_updates": 0, 2024-06-10T01:11:17.238 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_removes": 0 2024-06-10T01:11:17.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.238 INFO:teuthology.orchestra.run.smithi094.stdout: "osd-slow-ops": { 2024-06-10T01:11:17.238 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_ops_count": 0 2024-06-10T01:11:17.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.238 INFO:teuthology.orchestra.run.smithi094.stdout: "purge_queue": { 2024-06-10T01:11:17.238 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops": 0, 2024-06-10T01:11:17.239 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops_high_water": 0, 2024-06-10T01:11:17.239 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing": 0, 2024-06-10T01:11:17.239 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_high_water": 0, 2024-06-10T01:11:17.239 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed_ops": 0, 2024-06-10T01:11:17.239 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed": 0, 2024-06-10T01:11:17.239 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_item_in_journal": 0 2024-06-10T01:11:17.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.239 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T01:11:17.239 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-06-10T01:11:17.240 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-06-10T01:11:17.240 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-06-10T01:11:17.240 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 3404, 2024-06-10T01:11:17.240 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 1382945, 2024-06-10T01:11:17.240 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:17.240 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 3404, 2024-06-10T01:11:17.240 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-06-10T01:11:17.240 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-06-10T01:11:17.241 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 3404, 2024-06-10T01:11:17.241 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1382945, 2024-06-10T01:11:17.241 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-06-10T01:11:17.241 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.241 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.241 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.241 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:17.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.242 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_bytes": { 2024-06-10T01:11:17.242 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-06-10T01:11:17.242 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-06-10T01:11:17.242 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-06-10T01:11:17.242 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-06-10T01:11:17.242 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-06-10T01:11:17.242 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:17.242 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-06-10T01:11:17.243 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 3069, 2024-06-10T01:11:17.243 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 1432, 2024-06-10T01:11:17.243 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 3069, 2024-06-10T01:11:17.243 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1432, 2024-06-10T01:11:17.243 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-06-10T01:11:17.243 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.243 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.243 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.244 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:17.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.244 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_ops": { 2024-06-10T01:11:17.244 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-06-10T01:11:17.244 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 1024, 2024-06-10T01:11:17.244 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-06-10T01:11:17.244 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-06-10T01:11:17.244 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-06-10T01:11:17.244 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:17.245 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-06-10T01:11:17.245 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 3069, 2024-06-10T01:11:17.245 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 3069, 2024-06-10T01:11:17.245 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 3069, 2024-06-10T01:11:17.245 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 3069, 2024-06-10T01:11:17.245 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-06-10T01:11:17.245 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.245 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.246 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.246 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:17.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.246 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle": { 2024-06-10T01:11:17.246 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-06-10T01:11:17.246 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-06-10T01:11:17.246 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-06-10T01:11:17.246 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-06-10T01:11:17.247 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-06-10T01:11:17.247 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:17.247 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-06-10T01:11:17.247 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-06-10T01:11:17.247 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-06-10T01:11:17.247 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 0, 2024-06-10T01:11:17.247 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 0, 2024-06-10T01:11:17.247 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-06-10T01:11:17.248 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.248 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.248 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.248 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:17.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:17.248 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle-0x55a3f9f9c6e0": { 2024-06-10T01:11:17.248 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-06-10T01:11:17.248 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-06-10T01:11:17.249 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-06-10T01:11:17.249 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-06-10T01:11:17.249 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-06-10T01:11:17.249 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:17.249 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-06-10T01:11:17.249 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-06-10T01:11:17.249 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-06-10T01:11:17.249 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 0, 2024-06-10T01:11:17.249 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 0, 2024-06-10T01:11:17.250 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-06-10T01:11:17.250 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:17.250 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:17.250 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:17.250 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:17.250 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:17.250 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-06-10T01:11:17.971 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-06-10T01:11:17.971 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-06-10T01:11:17.971 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-06-10T01:11:17.971 DEBUG:tasks.check_counter:Getting stats from k 2024-06-10T01:11:17.971 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph daemon mds.k perf dump 2024-06-10T01:11:18.209 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.b/config 2024-06-10T01:11:18.580 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:18 smithi094 ceph-mon[36708]: pgmap v873: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-06-10T01:11:18.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:18 smithi167 ceph-mon[36917]: pgmap v873: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-06-10T01:11:18.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:18 smithi037 ceph-mon[32192]: pgmap v873: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-06-10T01:11:19.908 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-06-10T01:11:19.908 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T01:11:19.908 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 318, 2024-06-10T01:11:19.908 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 313, 2024-06-10T01:11:19.909 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 782048, 2024-06-10T01:11:19.909 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 193200, 2024-06-10T01:11:19.909 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 4, 2024-06-10T01:11:19.909 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 0, 2024-06-10T01:11:19.909 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.052110510, 2024-06-10T01:11:19.909 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.019243217, 2024-06-10T01:11:19.909 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.024327465, 2024-06-10T01:11:19.909 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.008119661, 2024-06-10T01:11:19.910 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:19.910 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 312, 2024-06-10T01:11:19.910 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.018445493, 2024-06-10T01:11:19.910 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000059120 2024-06-10T01:11:19.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:19.910 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:19.910 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:19.910 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:19.911 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:19.911 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:19.911 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 782048, 2024-06-10T01:11:19.911 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 193200 2024-06-10T01:11:19.911 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:19.911 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T01:11:19.911 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 2, 2024-06-10T01:11:19.912 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 239, 2024-06-10T01:11:19.912 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 320, 2024-06-10T01:11:19.912 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 198064, 2024-06-10T01:11:19.912 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 4, 2024-06-10T01:11:19.912 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 2, 2024-06-10T01:11:19.912 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.014393579, 2024-06-10T01:11:19.912 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.010441655, 2024-06-10T01:11:19.912 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.001195327, 2024-06-10T01:11:19.913 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T01:11:19.913 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:19.913 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 237, 2024-06-10T01:11:19.913 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.015139365, 2024-06-10T01:11:19.913 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000063879 2024-06-10T01:11:19.913 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:19.913 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:19.913 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:19.914 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:19.914 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:19.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:19.914 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-10T01:11:19.914 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 198064 2024-06-10T01:11:19.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:19.914 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T01:11:19.915 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 4, 2024-06-10T01:11:19.915 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 4, 2024-06-10T01:11:19.915 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 3024, 2024-06-10T01:11:19.915 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 448, 2024-06-10T01:11:19.915 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 2, 2024-06-10T01:11:19.915 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 1, 2024-06-10T01:11:19.915 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.000709469, 2024-06-10T01:11:19.915 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.000040181, 2024-06-10T01:11:19.916 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.000643669, 2024-06-10T01:11:19.916 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T01:11:19.916 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:19.916 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 3, 2024-06-10T01:11:19.916 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000745700, 2024-06-10T01:11:19.916 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000248566 2024-06-10T01:11:19.916 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:19.916 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:19.916 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:19.917 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:19.917 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:19.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:19.917 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-06-10T01:11:19.917 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 448 2024-06-10T01:11:19.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:19.917 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-06-10T01:11:19.918 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 0, 2024-06-10T01:11:19.918 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-06-10T01:11:19.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:19.918 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-06-10T01:11:19.918 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-06-10T01:11:19.918 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-06-10T01:11:19.918 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T01:11:19.919 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-06-10T01:11:19.919 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T01:11:19.919 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-06-10T01:11:19.919 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T01:11:19.919 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T01:11:19.919 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T01:11:19.919 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T01:11:19.919 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T01:11:19.920 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-06-10T01:11:19.920 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T01:11:19.920 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T01:11:19.920 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-06-10T01:11:19.920 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-06-10T01:11:19.920 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-06-10T01:11:19.920 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-06-10T01:11:19.920 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T01:11:19.920 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T01:11:19.921 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T01:11:19.921 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T01:11:19.921 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T01:11:19.921 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-06-10T01:11:19.921 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-06-10T01:11:19.921 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-06-10T01:11:19.921 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T01:11:19.921 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T01:11:19.922 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-06-10T01:11:19.922 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-06-10T01:11:19.922 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-06-10T01:11:19.922 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-06-10T01:11:19.922 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T01:11:19.922 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-06-10T01:11:19.922 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T01:11:19.923 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-06-10T01:11:19.923 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 149752, 2024-06-10T01:11:19.923 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 31, 2024-06-10T01:11:19.923 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-06-10T01:11:19.923 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-06-10T01:11:19.923 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-06-10T01:11:19.923 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-06-10T01:11:19.923 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-06-10T01:11:19.924 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-06-10T01:11:19.924 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-06-10T01:11:19.924 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-06-10T01:11:19.924 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 0, 2024-06-10T01:11:19.924 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 0, 2024-06-10T01:11:19.924 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T01:11:19.924 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-06-10T01:11:19.924 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-06-10T01:11:19.925 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-06-10T01:11:19.925 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 928, 2024-06-10T01:11:19.925 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 2, 2024-06-10T01:11:19.925 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-06-10T01:11:19.925 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-06-10T01:11:19.925 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-06-10T01:11:19.925 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-06-10T01:11:19.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-10T01:11:19.926 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T01:11:19.926 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-06-10T01:11:19.926 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-06-10T01:11:19.926 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-06-10T01:11:19.926 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 320, 2024-06-10T01:11:19.926 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 737130, 2024-06-10T01:11:19.926 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:19.926 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 320, 2024-06-10T01:11:19.927 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-06-10T01:11:19.927 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-06-10T01:11:19.927 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 320, 2024-06-10T01:11:19.927 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 737130, 2024-06-10T01:11:19.927 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-06-10T01:11:19.927 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-10T01:11:19.927 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-10T01:11:19.927 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-10T01:11:19.928 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:19.928 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-10T01:11:19.928 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-06-10T01:11:20.571 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:20 smithi094 ceph-mon[36708]: pgmap v874: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T01:11:20.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:20 smithi167 ceph-mon[36917]: pgmap v874: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T01:11:20.598 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-06-10T01:11:20.598 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-06-10T01:11:20.599 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-06-10T01:11:20.599 DEBUG:tasks.check_counter:Getting stats from c 2024-06-10T01:11:20.599 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph daemon mds.c perf dump 2024-06-10T01:11:20.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:20 smithi037 ceph-mon[32192]: pgmap v874: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T01:11:20.973 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T01:11:22.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:22 smithi167 ceph-mon[36917]: pgmap v875: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:11:22.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:22 smithi094 ceph-mon[36708]: pgmap v875: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:11:22.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:22 smithi037 ceph-mon[32192]: pgmap v875: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:11:23.421 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-06-10T01:11:23.422 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T01:11:23.422 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 47, 2024-06-10T01:11:23.422 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 49, 2024-06-10T01:11:23.422 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 13488, 2024-06-10T01:11:23.422 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 62925, 2024-06-10T01:11:23.423 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 14, 2024-06-10T01:11:23.423 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 4, 2024-06-10T01:11:23.423 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.016482614, 2024-06-10T01:11:23.423 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.003208395, 2024-06-10T01:11:23.423 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.009703970, 2024-06-10T01:11:23.423 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000420607, 2024-06-10T01:11:23.423 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:23.424 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 46, 2024-06-10T01:11:23.424 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.013578439, 2024-06-10T01:11:23.424 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000295183 2024-06-10T01:11:23.424 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.424 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:23.424 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 44, 2024-06-10T01:11:23.424 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000008677, 2024-06-10T01:11:23.425 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000197 2024-06-10T01:11:23.425 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.425 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 8064, 2024-06-10T01:11:23.425 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 4416 2024-06-10T01:11:23.425 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.425 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T01:11:23.425 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 2657, 2024-06-10T01:11:23.425 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 1814, 2024-06-10T01:11:23.426 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 11213032, 2024-06-10T01:11:23.426 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 621649, 2024-06-10T01:11:23.426 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 7, 2024-06-10T01:11:23.426 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 4, 2024-06-10T01:11:23.426 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.199233740, 2024-06-10T01:11:23.426 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.085162972, 2024-06-10T01:11:23.426 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.127969250, 2024-06-10T01:11:23.427 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.012663949, 2024-06-10T01:11:23.427 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:23.427 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1815, 2024-06-10T01:11:23.427 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.071658571, 2024-06-10T01:11:23.427 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000039481 2024-06-10T01:11:23.427 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.427 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:23.427 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 3633, 2024-06-10T01:11:23.428 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000646494, 2024-06-10T01:11:23.428 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000177 2024-06-10T01:11:23.428 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.428 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 849984, 2024-06-10T01:11:23.428 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 197344 2024-06-10T01:11:23.428 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.428 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T01:11:23.428 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 31, 2024-06-10T01:11:23.429 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 265, 2024-06-10T01:11:23.429 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 5183, 2024-06-10T01:11:23.429 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 855592, 2024-06-10T01:11:23.429 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 33, 2024-06-10T01:11:23.429 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 5, 2024-06-10T01:11:23.429 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.022119955, 2024-06-10T01:11:23.430 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.009642789, 2024-06-10T01:11:23.430 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.009432380, 2024-06-10T01:11:23.430 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000098887, 2024-06-10T01:11:23.430 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:23.430 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 263, 2024-06-10T01:11:23.431 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.028173456, 2024-06-10T01:11:23.431 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000107123 2024-06-10T01:11:23.431 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.431 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:23.432 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 24, 2024-06-10T01:11:23.432 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000002512, 2024-06-10T01:11:23.432 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000104 2024-06-10T01:11:23.432 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.433 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-10T01:11:23.433 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 560304 2024-06-10T01:11:23.433 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.433 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-06-10T01:11:23.433 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2024-06-10T01:11:23.433 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-06-10T01:11:23.433 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.434 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2024-06-10T01:11:23.434 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-06-10T01:11:23.434 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-06-10T01:11:23.434 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 38, 2024-06-10T01:11:23.434 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.001882226, 2024-06-10T01:11:23.434 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000049532 2024-06-10T01:11:23.434 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:23.434 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.435 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2024-06-10T01:11:23.435 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-06-10T01:11:23.435 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-06-10T01:11:23.435 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2, 2024-06-10T01:11:23.435 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000006208, 2024-06-10T01:11:23.435 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000003104 2024-06-10T01:11:23.435 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:23.435 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.436 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2024-06-10T01:11:23.436 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 8, 2024-06-10T01:11:23.436 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 1, 2024-06-10T01:11:23.436 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2024-06-10T01:11:23.436 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1, 2024-06-10T01:11:23.436 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000157430, 2024-06-10T01:11:23.436 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000157430 2024-06-10T01:11:23.437 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.437 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2024-06-10T01:11:23.437 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 7, 2024-06-10T01:11:23.437 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2024-06-10T01:11:23.437 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2024-06-10T01:11:23.437 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 11, 2024-06-10T01:11:23.438 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 0, 2024-06-10T01:11:23.438 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2024-06-10T01:11:23.438 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 19, 2024-06-10T01:11:23.438 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 2, 2024-06-10T01:11:23.438 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 17, 2024-06-10T01:11:23.438 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 0, 2024-06-10T01:11:23.438 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 15, 2024-06-10T01:11:23.439 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 14, 2024-06-10T01:11:23.439 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 1, 2024-06-10T01:11:23.439 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 1, 2024-06-10T01:11:23.439 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 3, 2024-06-10T01:11:23.439 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 33, 2024-06-10T01:11:23.439 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 24, 2024-06-10T01:11:23.439 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 7, 2024-06-10T01:11:23.439 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 2, 2024-06-10T01:11:23.440 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2024-06-10T01:11:23.440 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2024-06-10T01:11:23.440 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 0, 2024-06-10T01:11:23.440 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 0, 2024-06-10T01:11:23.440 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2024-06-10T01:11:23.440 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 0, 2024-06-10T01:11:23.440 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 0, 2024-06-10T01:11:23.440 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 0, 2024-06-10T01:11:23.441 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 0, 2024-06-10T01:11:23.441 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2024-06-10T01:11:23.441 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 0, 2024-06-10T01:11:23.441 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2024-06-10T01:11:23.441 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 4, 2024-06-10T01:11:23.441 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 234, 2024-06-10T01:11:23.441 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 0, 2024-06-10T01:11:23.441 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 0, 2024-06-10T01:11:23.442 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2024-06-10T01:11:23.442 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T01:11:23.442 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2024-06-10T01:11:23.442 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 0, 2024-06-10T01:11:23.442 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2024-06-10T01:11:23.442 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 0, 2024-06-10T01:11:23.442 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 0, 2024-06-10T01:11:23.442 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 0, 2024-06-10T01:11:23.443 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 1, 2024-06-10T01:11:23.443 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 0, 2024-06-10T01:11:23.443 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2024-06-10T01:11:23.443 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T01:11:23.443 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T01:11:23.443 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 1, 2024-06-10T01:11:23.443 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 0, 2024-06-10T01:11:23.443 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 0, 2024-06-10T01:11:23.444 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 0 2024-06-10T01:11:23.444 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.444 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2024-06-10T01:11:23.444 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 0, 2024-06-10T01:11:23.444 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 6, 2024-06-10T01:11:23.444 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 4, 2024-06-10T01:11:23.444 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 5, 2024-06-10T01:11:23.444 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 0, 2024-06-10T01:11:23.445 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2024-06-10T01:11:23.445 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2024-06-10T01:11:23.445 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2024-06-10T01:11:23.445 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 0, 2024-06-10T01:11:23.445 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 0, 2024-06-10T01:11:23.445 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2024-06-10T01:11:23.446 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2024-06-10T01:11:23.446 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2024-06-10T01:11:23.446 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2024-06-10T01:11:23.446 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2024-06-10T01:11:23.446 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2024-06-10T01:11:23.446 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2024-06-10T01:11:23.447 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_path": 10, 2024-06-10T01:11:23.447 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_inode": 52, 2024-06-10T01:11:23.447 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2024-06-10T01:11:23.447 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 0, 2024-06-10T01:11:23.448 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2024-06-10T01:11:23.448 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 0, 2024-06-10T01:11:23.448 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2024-06-10T01:11:23.448 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2024-06-10T01:11:23.448 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.449 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2024-06-10T01:11:23.449 INFO:teuthology.orchestra.run.smithi167.stdout: "evlrg": 0, 2024-06-10T01:11:23.449 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 3, 2024-06-10T01:11:23.449 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2024-06-10T01:11:23.449 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2024-06-10T01:11:23.449 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 3, 2024-06-10T01:11:23.449 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2024-06-10T01:11:23.449 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2024-06-10T01:11:23.450 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 2, 2024-06-10T01:11:23.450 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2024-06-10T01:11:23.450 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2024-06-10T01:11:23.450 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 2, 2024-06-10T01:11:23.450 INFO:teuthology.orchestra.run.smithi167.stdout: "segmjr": 2, 2024-06-10T01:11:23.450 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2024-06-10T01:11:23.450 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2024-06-10T01:11:23.450 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 4194304, 2024-06-10T01:11:23.451 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 4195200, 2024-06-10T01:11:23.451 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 0, 2024-06-10T01:11:23.451 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2024-06-10T01:11:23.451 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2, 2024-06-10T01:11:23.451 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.003676058, 2024-06-10T01:11:23.451 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.001838029 2024-06-10T01:11:23.451 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.451 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 0 2024-06-10T01:11:23.452 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.452 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2024-06-10T01:11:23.452 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 23, 2024-06-10T01:11:23.452 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 33, 2024-06-10T01:11:23.452 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 10, 2024-06-10T01:11:23.452 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 24, 2024-06-10T01:11:23.452 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 37, 2024-06-10T01:11:23.452 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 13, 2024-06-10T01:11:23.452 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 19, 2024-06-10T01:11:23.453 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 33, 2024-06-10T01:11:23.453 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 14, 2024-06-10T01:11:23.453 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 1, 2024-06-10T01:11:23.453 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 1, 2024-06-10T01:11:23.453 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 0, 2024-06-10T01:11:23.453 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 57208, 2024-06-10T01:11:23.453 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 223612 2024-06-10T01:11:23.453 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.454 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2024-06-10T01:11:23.454 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 8, 2024-06-10T01:11:23.454 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 2, 2024-06-10T01:11:23.454 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 8, 2024-06-10T01:11:23.454 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 12, 2024-06-10T01:11:23.454 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 4, 2024-06-10T01:11:23.454 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2024-06-10T01:11:23.454 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.455 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.455 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.455 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.455 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2024-06-10T01:11:23.455 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.455 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.456 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.456 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.456 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2024-06-10T01:11:23.456 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.456 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.456 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.456 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.456 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2024-06-10T01:11:23.457 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.457 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.457 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.457 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.457 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2024-06-10T01:11:23.457 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.457 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.457 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.458 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.458 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2024-06-10T01:11:23.458 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.458 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.459 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.459 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.459 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2024-06-10T01:11:23.459 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1, 2024-06-10T01:11:23.459 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000157430, 2024-06-10T01:11:23.459 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000157430 2024-06-10T01:11:23.460 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.460 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2024-06-10T01:11:23.460 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.460 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.460 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.460 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.460 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2024-06-10T01:11:23.461 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.461 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.461 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.461 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.461 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2024-06-10T01:11:23.461 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.461 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.461 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.462 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.462 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2024-06-10T01:11:23.462 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.462 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.462 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.462 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.463 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2024-06-10T01:11:23.463 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.463 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.464 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.464 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.464 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2024-06-10T01:11:23.464 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.464 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.464 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.464 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.465 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2024-06-10T01:11:23.465 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.465 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.465 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.465 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.465 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2024-06-10T01:11:23.465 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.466 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.466 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.466 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.466 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2024-06-10T01:11:23.466 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.466 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.466 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.467 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.467 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2024-06-10T01:11:23.467 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.467 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.467 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.467 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.468 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2024-06-10T01:11:23.468 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.468 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.468 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.468 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.468 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2024-06-10T01:11:23.468 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.469 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.469 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.469 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.469 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2024-06-10T01:11:23.469 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.469 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.470 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.470 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.470 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2024-06-10T01:11:23.470 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.470 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.470 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.470 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.471 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2024-06-10T01:11:23.471 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.471 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.471 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.471 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.471 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2024-06-10T01:11:23.471 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.471 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.472 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.472 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.472 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2024-06-10T01:11:23.472 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.472 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.472 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.472 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.472 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2024-06-10T01:11:23.473 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.473 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.473 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.473 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.473 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2024-06-10T01:11:23.473 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.473 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.473 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.474 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.474 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2024-06-10T01:11:23.474 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.474 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.474 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.474 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.475 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2024-06-10T01:11:23.475 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.475 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.475 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.475 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.475 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2024-06-10T01:11:23.475 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.476 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.476 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.476 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.476 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2024-06-10T01:11:23.476 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2024-06-10T01:11:23.476 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2024-06-10T01:11:23.476 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.476 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.477 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.477 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:23.477 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.477 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2024-06-10T01:11:23.477 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 1, 2024-06-10T01:11:23.477 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 1, 2024-06-10T01:11:23.477 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 0, 2024-06-10T01:11:23.477 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 1, 2024-06-10T01:11:23.478 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2024-06-10T01:11:23.478 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 1, 2024-06-10T01:11:23.478 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 1, 2024-06-10T01:11:23.478 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 850, 2024-06-10T01:11:23.478 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2024-06-10T01:11:23.478 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.478 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-06-10T01:11:23.479 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-06-10T01:11:23.479 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-06-10T01:11:23.479 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T01:11:23.479 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-06-10T01:11:23.479 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T01:11:23.479 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-06-10T01:11:23.479 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T01:11:23.479 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T01:11:23.480 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T01:11:23.480 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T01:11:23.480 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T01:11:23.480 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-06-10T01:11:23.480 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T01:11:23.480 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T01:11:23.480 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-06-10T01:11:23.481 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-06-10T01:11:23.481 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-06-10T01:11:23.481 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-06-10T01:11:23.481 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T01:11:23.481 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T01:11:23.481 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T01:11:23.481 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T01:11:23.481 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T01:11:23.481 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-06-10T01:11:23.482 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-06-10T01:11:23.482 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-06-10T01:11:23.482 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T01:11:23.482 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T01:11:23.482 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-06-10T01:11:23.482 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-06-10T01:11:23.482 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-06-10T01:11:23.483 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-06-10T01:11:23.483 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T01:11:23.483 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-06-10T01:11:23.483 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T01:11:23.483 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-06-10T01:11:23.483 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 238929, 2024-06-10T01:11:23.483 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 78, 2024-06-10T01:11:23.483 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2024-06-10T01:11:23.484 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2024-06-10T01:11:23.484 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-06-10T01:11:23.484 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-06-10T01:11:23.484 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-06-10T01:11:23.484 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-06-10T01:11:23.484 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-06-10T01:11:23.484 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-06-10T01:11:23.484 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 9176, 2024-06-10T01:11:23.484 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 176, 2024-06-10T01:11:23.485 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T01:11:23.485 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-06-10T01:11:23.485 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-06-10T01:11:23.485 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-06-10T01:11:23.485 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 96647, 2024-06-10T01:11:23.485 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 398, 2024-06-10T01:11:23.485 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-06-10T01:11:23.485 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-06-10T01:11:23.486 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-06-10T01:11:23.486 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-06-10T01:11:23.486 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.486 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2024-06-10T01:11:23.486 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2024-06-10T01:11:23.486 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2024-06-10T01:11:23.487 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 37, 2024-06-10T01:11:23.487 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 11108, 2024-06-10T01:11:23.487 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 28, 2024-06-10T01:11:23.487 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 37, 2024-06-10T01:11:23.487 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2024-06-10T01:11:23.487 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 37, 2024-06-10T01:11:23.487 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.196996612, 2024-06-10T01:11:23.488 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.005324232 2024-06-10T01:11:23.488 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.488 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2024-06-10T01:11:23.488 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2024-06-10T01:11:23.488 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 37, 2024-06-10T01:11:23.488 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 59 2024-06-10T01:11:23.488 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.488 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 37, 2024-06-10T01:11:23.489 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 0, 2024-06-10T01:11:23.489 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 37, 2024-06-10T01:11:23.489 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2024-06-10T01:11:23.489 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2024-06-10T01:11:23.489 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 0, 2024-06-10T01:11:23.489 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 10, 2024-06-10T01:11:23.489 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 0, 2024-06-10T01:11:23.489 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 2, 2024-06-10T01:11:23.489 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 6, 2024-06-10T01:11:23.490 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2024-06-10T01:11:23.490 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2024-06-10T01:11:23.490 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2024-06-10T01:11:23.490 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 0, 2024-06-10T01:11:23.490 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 6, 2024-06-10T01:11:23.490 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2024-06-10T01:11:23.490 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2024-06-10T01:11:23.490 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2024-06-10T01:11:23.491 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 0, 2024-06-10T01:11:23.491 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 20, 2024-06-10T01:11:23.491 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2024-06-10T01:11:23.491 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2024-06-10T01:11:23.491 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2024-06-10T01:11:23.491 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2024-06-10T01:11:23.491 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2024-06-10T01:11:23.491 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2024-06-10T01:11:23.491 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T01:11:23.492 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2024-06-10T01:11:23.492 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2024-06-10T01:11:23.492 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2024-06-10T01:11:23.492 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2024-06-10T01:11:23.492 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2024-06-10T01:11:23.492 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2024-06-10T01:11:23.492 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2024-06-10T01:11:23.492 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2024-06-10T01:11:23.493 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2024-06-10T01:11:23.493 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2024-06-10T01:11:23.493 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2024-06-10T01:11:23.493 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2024-06-10T01:11:23.493 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2024-06-10T01:11:23.493 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2024-06-10T01:11:23.493 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2024-06-10T01:11:23.493 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2024-06-10T01:11:23.493 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2024-06-10T01:11:23.494 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2024-06-10T01:11:23.494 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2024-06-10T01:11:23.494 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 105, 2024-06-10T01:11:23.494 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2024-06-10T01:11:23.494 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 7, 2024-06-10T01:11:23.494 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 9, 2024-06-10T01:11:23.494 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 13, 2024-06-10T01:11:23.494 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 4, 2024-06-10T01:11:23.495 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2024-06-10T01:11:23.495 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 14, 2024-06-10T01:11:23.495 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 0, 2024-06-10T01:11:23.495 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 1 2024-06-10T01:11:23.495 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.495 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2024-06-10T01:11:23.495 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 1, 2024-06-10T01:11:23.495 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2024-06-10T01:11:23.496 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2024-06-10T01:11:23.496 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2024-06-10T01:11:23.496 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.496 INFO:teuthology.orchestra.run.smithi167.stdout: "osd-slow-ops": { 2024-06-10T01:11:23.496 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 62 2024-06-10T01:11:23.496 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.496 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2024-06-10T01:11:23.496 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2024-06-10T01:11:23.496 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 0, 2024-06-10T01:11:23.497 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2024-06-10T01:11:23.497 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 0, 2024-06-10T01:11:23.497 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed_ops": 0, 2024-06-10T01:11:23.497 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 0, 2024-06-10T01:11:23.497 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2024-06-10T01:11:23.497 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.497 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T01:11:23.498 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-06-10T01:11:23.498 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-06-10T01:11:23.498 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-06-10T01:11:23.498 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 2731, 2024-06-10T01:11:23.498 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 10963459, 2024-06-10T01:11:23.498 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:23.498 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 2731, 2024-06-10T01:11:23.499 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-06-10T01:11:23.499 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-06-10T01:11:23.499 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 2731, 2024-06-10T01:11:23.499 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 10963459, 2024-06-10T01:11:23.499 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-06-10T01:11:23.499 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.499 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.499 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.500 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:23.500 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.500 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2024-06-10T01:11:23.500 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-06-10T01:11:23.500 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-06-10T01:11:23.500 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-06-10T01:11:23.500 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-06-10T01:11:23.500 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-06-10T01:11:23.500 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:23.501 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-06-10T01:11:23.501 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 37, 2024-06-10T01:11:23.501 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 11108, 2024-06-10T01:11:23.501 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 37, 2024-06-10T01:11:23.501 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 11108, 2024-06-10T01:11:23.501 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-06-10T01:11:23.501 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.501 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.502 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.502 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:23.502 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.502 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2024-06-10T01:11:23.502 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-06-10T01:11:23.502 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2024-06-10T01:11:23.502 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-06-10T01:11:23.503 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-06-10T01:11:23.503 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-06-10T01:11:23.503 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:23.503 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-06-10T01:11:23.503 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 37, 2024-06-10T01:11:23.503 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 37, 2024-06-10T01:11:23.503 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 37, 2024-06-10T01:11:23.503 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 37, 2024-06-10T01:11:23.504 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-06-10T01:11:23.504 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.504 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.504 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.504 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:23.504 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.504 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2024-06-10T01:11:23.504 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-06-10T01:11:23.504 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-06-10T01:11:23.505 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-06-10T01:11:23.505 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-06-10T01:11:23.505 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-06-10T01:11:23.505 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:23.505 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-06-10T01:11:23.505 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-06-10T01:11:23.505 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-06-10T01:11:23.505 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-06-10T01:11:23.505 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-06-10T01:11:23.506 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-06-10T01:11:23.506 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.506 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.506 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.506 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:23.506 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:23.506 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x55d02d788500": { 2024-06-10T01:11:23.507 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-06-10T01:11:23.507 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-06-10T01:11:23.507 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-06-10T01:11:23.507 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 3, 2024-06-10T01:11:23.507 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 896, 2024-06-10T01:11:23.507 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:23.507 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 3, 2024-06-10T01:11:23.508 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-06-10T01:11:23.508 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-06-10T01:11:23.508 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 2, 2024-06-10T01:11:23.508 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 896, 2024-06-10T01:11:23.508 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-06-10T01:11:23.508 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:23.508 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:23.508 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:23.508 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:23.509 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:23.509 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-06-10T01:11:24.438 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-06-10T01:11:24.439 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-06-10T01:11:24.439 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-06-10T01:11:24.439 DEBUG:tasks.check_counter:Getting stats from f 2024-06-10T01:11:24.439 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph daemon mds.f perf dump 2024-06-10T01:11:24.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:24 smithi167 ceph-mon[36917]: pgmap v876: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:11:24.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:24 smithi094 ceph-mon[36708]: pgmap v876: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:11:24.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:24 smithi037 ceph-mon[32192]: pgmap v876: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:11:24.685 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T01:11:26.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:26 smithi167 ceph-mon[36917]: pgmap v877: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:11:26.603 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-06-10T01:11:26.603 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T01:11:26.603 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 0, 2024-06-10T01:11:26.604 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 2, 2024-06-10T01:11:26.604 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 0, 2024-06-10T01:11:26.604 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 0, 2024-06-10T01:11:26.604 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 4, 2024-06-10T01:11:26.604 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 0, 2024-06-10T01:11:26.604 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.001612484, 2024-06-10T01:11:26.604 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.000000000, 2024-06-10T01:11:26.604 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.000308328, 2024-06-10T01:11:26.605 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T01:11:26.605 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:26.605 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:26.605 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:26.605 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:26.605 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:26.605 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:26.605 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:26.605 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:26.605 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:26.606 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:26.606 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-10T01:11:26.606 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 0 2024-06-10T01:11:26.606 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:26.606 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T01:11:26.606 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 91, 2024-06-10T01:11:26.606 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 88, 2024-06-10T01:11:26.607 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 167792, 2024-06-10T01:11:26.607 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 52480, 2024-06-10T01:11:26.607 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 2, 2024-06-10T01:11:26.607 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 1, 2024-06-10T01:11:26.607 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.011880170, 2024-06-10T01:11:26.607 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.004254927, 2024-06-10T01:11:26.607 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.005480400, 2024-06-10T01:11:26.607 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.001860066, 2024-06-10T01:11:26.607 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:26.608 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 88, 2024-06-10T01:11:26.608 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.004950878, 2024-06-10T01:11:26.608 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000056259 2024-06-10T01:11:26.608 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:26.608 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:26.608 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:26.608 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:26.608 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:26.608 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:26.609 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 167792, 2024-06-10T01:11:26.609 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 52480 2024-06-10T01:11:26.609 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:26.609 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T01:11:26.609 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 1, 2024-06-10T01:11:26.609 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 66, 2024-06-10T01:11:26.609 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 160, 2024-06-10T01:11:26.609 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 89440, 2024-06-10T01:11:26.609 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 3, 2024-06-10T01:11:26.610 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 2, 2024-06-10T01:11:26.610 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.004546477, 2024-06-10T01:11:26.610 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.002433195, 2024-06-10T01:11:26.610 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.001432044, 2024-06-10T01:11:26.610 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-10T01:11:26.610 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:26.610 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 64, 2024-06-10T01:11:26.610 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.003163191, 2024-06-10T01:11:26.611 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000049424 2024-06-10T01:11:26.611 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:26.611 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:26.611 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:26.611 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:26.611 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:26.611 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:26.611 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-10T01:11:26.611 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 89440 2024-06-10T01:11:26.612 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:26.612 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-06-10T01:11:26.612 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 0, 2024-06-10T01:11:26.612 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-06-10T01:11:26.612 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:26.612 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-06-10T01:11:26.612 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-06-10T01:11:26.612 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-06-10T01:11:26.612 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T01:11:26.613 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-06-10T01:11:26.613 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T01:11:26.613 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-06-10T01:11:26.613 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T01:11:26.613 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T01:11:26.613 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T01:11:26.613 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T01:11:26.613 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T01:11:26.613 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-06-10T01:11:26.614 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T01:11:26.614 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T01:11:26.614 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-06-10T01:11:26.614 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-06-10T01:11:26.614 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-06-10T01:11:26.614 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-06-10T01:11:26.614 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T01:11:26.614 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T01:11:26.614 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T01:11:26.615 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T01:11:26.615 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T01:11:26.615 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-06-10T01:11:26.615 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-06-10T01:11:26.615 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-06-10T01:11:26.615 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T01:11:26.615 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T01:11:26.615 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-06-10T01:11:26.615 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-06-10T01:11:26.616 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-06-10T01:11:26.616 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-06-10T01:11:26.616 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T01:11:26.616 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-06-10T01:11:26.616 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T01:11:26.616 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-06-10T01:11:26.616 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 148863, 2024-06-10T01:11:26.616 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 28, 2024-06-10T01:11:26.616 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2024-06-10T01:11:26.616 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2024-06-10T01:11:26.617 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-06-10T01:11:26.617 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-06-10T01:11:26.617 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-06-10T01:11:26.617 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-06-10T01:11:26.617 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-06-10T01:11:26.617 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-06-10T01:11:26.617 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 0, 2024-06-10T01:11:26.617 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 0, 2024-06-10T01:11:26.617 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T01:11:26.618 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-06-10T01:11:26.618 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-06-10T01:11:26.618 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-06-10T01:11:26.618 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 928, 2024-06-10T01:11:26.618 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 2, 2024-06-10T01:11:26.618 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-06-10T01:11:26.618 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-06-10T01:11:26.618 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-06-10T01:11:26.618 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-06-10T01:11:26.619 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:26.619 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T01:11:26.619 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-06-10T01:11:26.619 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-06-10T01:11:26.619 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-06-10T01:11:26.619 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 88, 2024-06-10T01:11:26.619 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 152512, 2024-06-10T01:11:26.619 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:26.620 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 88, 2024-06-10T01:11:26.620 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-06-10T01:11:26.620 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-06-10T01:11:26.620 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 88, 2024-06-10T01:11:26.620 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 152512, 2024-06-10T01:11:26.620 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-06-10T01:11:26.620 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:26.620 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:26.620 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:26.621 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:26.621 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:26.621 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-06-10T01:11:26.621 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:26 smithi094 ceph-mon[36708]: pgmap v877: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:11:26.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:26 smithi037 ceph-mon[32192]: pgmap v877: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:11:27.358 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.f 2024-06-10T01:11:27.358 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.f 2024-06-10T01:11:27.358 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.f 2024-06-10T01:11:27.359 DEBUG:tasks.check_counter:Getting stats from i 2024-06-10T01:11:27.359 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph daemon mds.i perf dump 2024-06-10T01:11:27.605 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T01:11:28.581 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:28 smithi167 ceph-mon[36917]: pgmap v878: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-06-10T01:11:28.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:28 smithi094 ceph-mon[36708]: pgmap v878: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-06-10T01:11:28.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:28 smithi037 ceph-mon[32192]: pgmap v878: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2024-06-10T01:11:29.490 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-06-10T01:11:29.491 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T01:11:29.491 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 130600, 2024-06-10T01:11:29.491 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 131347, 2024-06-10T01:11:29.491 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 52615371, 2024-06-10T01:11:29.491 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 346580677, 2024-06-10T01:11:29.492 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 174, 2024-06-10T01:11:29.492 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 6, 2024-06-10T01:11:29.492 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 10.606012338, 2024-06-10T01:11:29.492 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 3.635646170, 2024-06-10T01:11:29.492 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 1.864047088, 2024-06-10T01:11:29.492 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 4.604653193, 2024-06-10T01:11:29.492 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:29.493 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 131345, 2024-06-10T01:11:29.493 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 83.070056835, 2024-06-10T01:11:29.493 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000632456 2024-06-10T01:11:29.493 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.493 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:29.493 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 3766, 2024-06-10T01:11:29.493 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000737359, 2024-06-10T01:11:29.493 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000195 2024-06-10T01:11:29.493 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.494 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 39568, 2024-06-10T01:11:29.494 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 5120 2024-06-10T01:11:29.494 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.494 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T01:11:29.494 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 353916, 2024-06-10T01:11:29.494 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 581456, 2024-06-10T01:11:29.495 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 143571922, 2024-06-10T01:11:29.495 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 539034408, 2024-06-10T01:11:29.495 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 72, 2024-06-10T01:11:29.495 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 5, 2024-06-10T01:11:29.495 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 25.505944310, 2024-06-10T01:11:29.495 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 10.557750616, 2024-06-10T01:11:29.495 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 123.981848546, 2024-06-10T01:11:29.495 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 5.380285924, 2024-06-10T01:11:29.496 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:29.496 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 581455, 2024-06-10T01:11:29.496 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 134.763000255, 2024-06-10T01:11:29.496 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000231768 2024-06-10T01:11:29.496 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.496 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:29.496 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 548041, 2024-06-10T01:11:29.496 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.080341498, 2024-06-10T01:11:29.497 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000146 2024-06-10T01:11:29.497 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.497 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 84232032, 2024-06-10T01:11:29.497 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 283237568 2024-06-10T01:11:29.497 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.497 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T01:11:29.497 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 147257, 2024-06-10T01:11:29.497 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 148604, 2024-06-10T01:11:29.498 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 62624872, 2024-06-10T01:11:29.498 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 175773225, 2024-06-10T01:11:29.498 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 14, 2024-06-10T01:11:29.498 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 6, 2024-06-10T01:11:29.498 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 10.878189098, 2024-06-10T01:11:29.498 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 3.002553196, 2024-06-10T01:11:29.498 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 1.941873771, 2024-06-10T01:11:29.498 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 5.338780338, 2024-06-10T01:11:29.499 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:29.499 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 148603, 2024-06-10T01:11:29.499 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 88.998477839, 2024-06-10T01:11:29.499 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000598900 2024-06-10T01:11:29.499 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.499 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:29.499 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 4000, 2024-06-10T01:11:29.499 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000985961, 2024-06-10T01:11:29.500 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000246 2024-06-10T01:11:29.500 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.500 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-06-10T01:11:29.500 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 588384 2024-06-10T01:11:29.500 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.500 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-06-10T01:11:29.500 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2024-06-10T01:11:29.500 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-06-10T01:11:29.500 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.501 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2024-06-10T01:11:29.501 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-06-10T01:11:29.501 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-06-10T01:11:29.501 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 93353, 2024-06-10T01:11:29.501 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 365.838214821, 2024-06-10T01:11:29.501 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.003918869 2024-06-10T01:11:29.501 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:29.501 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.502 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2024-06-10T01:11:29.502 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-06-10T01:11:29.502 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-06-10T01:11:29.502 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 78256, 2024-06-10T01:11:29.502 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 84.346642294, 2024-06-10T01:11:29.502 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.001077829 2024-06-10T01:11:29.502 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:29.503 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.503 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2024-06-10T01:11:29.503 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 136619, 2024-06-10T01:11:29.503 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 136619, 2024-06-10T01:11:29.503 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2024-06-10T01:11:29.503 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 136619, 2024-06-10T01:11:29.503 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 2029.422689891, 2024-06-10T01:11:29.503 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.014854615 2024-06-10T01:11:29.503 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.504 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2024-06-10T01:11:29.504 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 0, 2024-06-10T01:11:29.504 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2024-06-10T01:11:29.504 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2024-06-10T01:11:29.504 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 30644, 2024-06-10T01:11:29.504 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 382, 2024-06-10T01:11:29.504 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 210, 2024-06-10T01:11:29.504 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 2852, 2024-06-10T01:11:29.505 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 1983, 2024-06-10T01:11:29.505 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 869, 2024-06-10T01:11:29.505 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 0, 2024-06-10T01:11:29.505 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 18, 2024-06-10T01:11:29.505 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 0, 2024-06-10T01:11:29.505 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 1, 2024-06-10T01:11:29.505 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 1, 2024-06-10T01:11:29.505 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 2, 2024-06-10T01:11:29.506 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 170783, 2024-06-10T01:11:29.506 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 170098, 2024-06-10T01:11:29.506 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 0, 2024-06-10T01:11:29.506 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 0, 2024-06-10T01:11:29.506 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2024-06-10T01:11:29.506 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2024-06-10T01:11:29.506 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 0, 2024-06-10T01:11:29.506 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 0, 2024-06-10T01:11:29.507 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2024-06-10T01:11:29.507 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 0, 2024-06-10T01:11:29.507 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 0, 2024-06-10T01:11:29.507 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 0, 2024-06-10T01:11:29.507 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 0, 2024-06-10T01:11:29.507 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2024-06-10T01:11:29.507 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 139172, 2024-06-10T01:11:29.507 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2024-06-10T01:11:29.508 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 4, 2024-06-10T01:11:29.508 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 234, 2024-06-10T01:11:29.508 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 0, 2024-06-10T01:11:29.508 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 139172, 2024-06-10T01:11:29.508 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2024-06-10T01:11:29.508 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T01:11:29.508 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2024-06-10T01:11:29.508 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 133, 2024-06-10T01:11:29.509 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 11, 2024-06-10T01:11:29.509 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 122, 2024-06-10T01:11:29.509 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 139039, 2024-06-10T01:11:29.509 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 5, 2024-06-10T01:11:29.509 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 52843, 2024-06-10T01:11:29.509 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 102645, 2024-06-10T01:11:29.509 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 1, 2024-06-10T01:11:29.509 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T01:11:29.510 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T01:11:29.510 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 73314, 2024-06-10T01:11:29.510 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 50529, 2024-06-10T01:11:29.510 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 4268, 2024-06-10T01:11:29.510 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 52843 2024-06-10T01:11:29.510 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.510 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2024-06-10T01:11:29.510 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 4, 2024-06-10T01:11:29.511 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 0, 2024-06-10T01:11:29.511 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 0, 2024-06-10T01:11:29.511 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 0, 2024-06-10T01:11:29.511 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 12, 2024-06-10T01:11:29.511 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 2, 2024-06-10T01:11:29.511 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2024-06-10T01:11:29.511 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2024-06-10T01:11:29.511 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 50141, 2024-06-10T01:11:29.512 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 50139, 2024-06-10T01:11:29.512 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2024-06-10T01:11:29.512 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2024-06-10T01:11:29.512 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2024-06-10T01:11:29.512 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2024-06-10T01:11:29.512 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2024-06-10T01:11:29.512 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2024-06-10T01:11:29.512 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2024-06-10T01:11:29.512 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_path": 10, 2024-06-10T01:11:29.513 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_inode": 141637, 2024-06-10T01:11:29.513 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 11, 2024-06-10T01:11:29.513 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 0, 2024-06-10T01:11:29.513 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2024-06-10T01:11:29.513 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 736, 2024-06-10T01:11:29.513 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2024-06-10T01:11:29.513 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2024-06-10T01:11:29.513 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.513 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2024-06-10T01:11:29.514 INFO:teuthology.orchestra.run.smithi167.stdout: "evlrg": 0, 2024-06-10T01:11:29.514 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 206089, 2024-06-10T01:11:29.514 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 205972, 2024-06-10T01:11:29.514 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 205972, 2024-06-10T01:11:29.514 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 117, 2024-06-10T01:11:29.514 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 106, 2024-06-10T01:11:29.514 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 15, 2024-06-10T01:11:29.514 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 287, 2024-06-10T01:11:29.515 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 285, 2024-06-10T01:11:29.515 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 285, 2024-06-10T01:11:29.515 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 2, 2024-06-10T01:11:29.515 INFO:teuthology.orchestra.run.smithi167.stdout: "segmjr": 2, 2024-06-10T01:11:29.515 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 1, 2024-06-10T01:11:29.515 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 1, 2024-06-10T01:11:29.515 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 487447318, 2024-06-10T01:11:29.515 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 487708449, 2024-06-10T01:11:29.516 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 0, 2024-06-10T01:11:29.516 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2024-06-10T01:11:29.516 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 31417, 2024-06-10T01:11:29.516 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 9181.768716247, 2024-06-10T01:11:29.516 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.292254789 2024-06-10T01:11:29.516 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.516 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 0 2024-06-10T01:11:29.516 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.517 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2024-06-10T01:11:29.517 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 27, 2024-06-10T01:11:29.517 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 50177, 2024-06-10T01:11:29.517 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 50150, 2024-06-10T01:11:29.517 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 42, 2024-06-10T01:11:29.517 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 3407, 2024-06-10T01:11:29.517 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 3365, 2024-06-10T01:11:29.517 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 2852, 2024-06-10T01:11:29.518 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 100197, 2024-06-10T01:11:29.518 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 97345, 2024-06-10T01:11:29.518 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 1, 2024-06-10T01:11:29.518 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 117942, 2024-06-10T01:11:29.518 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 117941, 2024-06-10T01:11:29.518 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 267584, 2024-06-10T01:11:29.518 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 231804 2024-06-10T01:11:29.518 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.519 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2024-06-10T01:11:29.519 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 139243, 2024-06-10T01:11:29.519 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 0, 2024-06-10T01:11:29.519 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 136619, 2024-06-10T01:11:29.519 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 26, 2024-06-10T01:11:29.519 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 8, 2024-06-10T01:11:29.519 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2024-06-10T01:11:29.519 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 50024, 2024-06-10T01:11:29.520 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 919.633430712, 2024-06-10T01:11:29.520 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.018383844 2024-06-10T01:11:29.520 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.520 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2024-06-10T01:11:29.520 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 138, 2024-06-10T01:11:29.520 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.029018090, 2024-06-10T01:11:29.520 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000210276 2024-06-10T01:11:29.520 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.520 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2024-06-10T01:11:29.521 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:29.521 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:29.521 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:29.521 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.521 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2024-06-10T01:11:29.521 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:29.521 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:29.521 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:29.522 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.522 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2024-06-10T01:11:29.522 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 33508, 2024-06-10T01:11:29.522 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 210.698151941, 2024-06-10T01:11:29.522 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.006287995 2024-06-10T01:11:29.522 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.522 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2024-06-10T01:11:29.522 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:29.523 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:29.523 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:29.523 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.523 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2024-06-10T01:11:29.523 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:29.523 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:29.523 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:29.523 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.524 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2024-06-10T01:11:29.524 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:29.524 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:29.524 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:29.524 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.524 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2024-06-10T01:11:29.524 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:29.524 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:29.525 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:29.525 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.525 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2024-06-10T01:11:29.525 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:29.526 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:29.526 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:29.526 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.526 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2024-06-10T01:11:29.527 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:29.527 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:29.527 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:29.527 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.527 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2024-06-10T01:11:29.527 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 130, 2024-06-10T01:11:29.527 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 6.088287352, 2024-06-10T01:11:29.527 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.046832979 2024-06-10T01:11:29.528 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.528 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2024-06-10T01:11:29.528 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:29.528 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:29.528 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:29.528 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.528 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2024-06-10T01:11:29.528 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:29.529 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:29.529 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:29.529 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.529 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2024-06-10T01:11:29.529 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1, 2024-06-10T01:11:29.529 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000316540, 2024-06-10T01:11:29.529 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000316540 2024-06-10T01:11:29.530 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.530 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2024-06-10T01:11:29.530 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2616, 2024-06-10T01:11:29.530 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 18.626620614, 2024-06-10T01:11:29.530 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.007120267 2024-06-10T01:11:29.530 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.530 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2024-06-10T01:11:29.530 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 4, 2024-06-10T01:11:29.531 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.006576372, 2024-06-10T01:11:29.531 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.001644093 2024-06-10T01:11:29.531 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.531 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2024-06-10T01:11:29.531 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:29.531 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:29.531 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:29.531 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.531 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2024-06-10T01:11:29.532 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:29.532 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:29.532 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:29.532 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.532 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2024-06-10T01:11:29.532 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 121, 2024-06-10T01:11:29.532 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 14.350419921, 2024-06-10T01:11:29.532 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.118598511 2024-06-10T01:11:29.533 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.533 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2024-06-10T01:11:29.533 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:29.533 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:29.533 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:29.533 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.533 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2024-06-10T01:11:29.533 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 3, 2024-06-10T01:11:29.533 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.012669786, 2024-06-10T01:11:29.534 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.004223262 2024-06-10T01:11:29.534 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.534 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2024-06-10T01:11:29.534 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 5, 2024-06-10T01:11:29.534 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.007317787, 2024-06-10T01:11:29.534 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.001463557 2024-06-10T01:11:29.534 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.535 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2024-06-10T01:11:29.535 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:29.535 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:29.535 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:29.535 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.535 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2024-06-10T01:11:29.535 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 22, 2024-06-10T01:11:29.535 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 10.023471875, 2024-06-10T01:11:29.536 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.455612357 2024-06-10T01:11:29.536 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.536 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2024-06-10T01:11:29.536 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:29.536 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:29.536 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:29.536 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.536 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2024-06-10T01:11:29.537 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 26, 2024-06-10T01:11:29.537 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.094724602, 2024-06-10T01:11:29.537 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.003643253 2024-06-10T01:11:29.537 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.537 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2024-06-10T01:11:29.537 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:29.537 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:29.537 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:29.538 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.538 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2024-06-10T01:11:29.538 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 50019, 2024-06-10T01:11:29.538 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 849.851302859, 2024-06-10T01:11:29.538 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.016990569 2024-06-10T01:11:29.538 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.538 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2024-06-10T01:11:29.538 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2024-06-10T01:11:29.539 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2024-06-10T01:11:29.539 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2, 2024-06-10T01:11:29.539 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000381440, 2024-06-10T01:11:29.539 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000190720 2024-06-10T01:11:29.539 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:29.539 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.539 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2024-06-10T01:11:29.539 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 1, 2024-06-10T01:11:29.540 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 4, 2024-06-10T01:11:29.540 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 3, 2024-06-10T01:11:29.540 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 1, 2024-06-10T01:11:29.540 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2024-06-10T01:11:29.540 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 14990, 2024-06-10T01:11:29.540 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 14990, 2024-06-10T01:11:29.540 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 896, 2024-06-10T01:11:29.540 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2024-06-10T01:11:29.541 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.541 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-06-10T01:11:29.541 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-06-10T01:11:29.541 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-06-10T01:11:29.541 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T01:11:29.541 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-06-10T01:11:29.541 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T01:11:29.541 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-06-10T01:11:29.542 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T01:11:29.542 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T01:11:29.542 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T01:11:29.542 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T01:11:29.542 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T01:11:29.542 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-06-10T01:11:29.542 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T01:11:29.543 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T01:11:29.543 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-06-10T01:11:29.543 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-06-10T01:11:29.543 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-06-10T01:11:29.543 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-06-10T01:11:29.543 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T01:11:29.543 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T01:11:29.544 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T01:11:29.544 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T01:11:29.544 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T01:11:29.544 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-06-10T01:11:29.544 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-06-10T01:11:29.544 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-06-10T01:11:29.544 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T01:11:29.544 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T01:11:29.544 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-06-10T01:11:29.545 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-06-10T01:11:29.545 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-06-10T01:11:29.545 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-06-10T01:11:29.545 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T01:11:29.545 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-06-10T01:11:29.545 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T01:11:29.545 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-06-10T01:11:29.545 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 305342, 2024-06-10T01:11:29.546 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 117, 2024-06-10T01:11:29.546 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 176, 2024-06-10T01:11:29.546 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 2, 2024-06-10T01:11:29.546 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-06-10T01:11:29.546 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-06-10T01:11:29.546 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-06-10T01:11:29.546 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-06-10T01:11:29.546 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-06-10T01:11:29.547 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-06-10T01:11:29.547 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 9176, 2024-06-10T01:11:29.547 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 176, 2024-06-10T01:11:29.547 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T01:11:29.547 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-06-10T01:11:29.547 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-06-10T01:11:29.547 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-06-10T01:11:29.547 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 2290159, 2024-06-10T01:11:29.548 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 32341, 2024-06-10T01:11:29.548 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-06-10T01:11:29.548 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-06-10T01:11:29.548 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-06-10T01:11:29.548 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-06-10T01:11:29.548 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.548 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2024-06-10T01:11:29.548 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2024-06-10T01:11:29.548 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2024-06-10T01:11:29.549 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 413226, 2024-06-10T01:11:29.549 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 636212052, 2024-06-10T01:11:29.549 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 37, 2024-06-10T01:11:29.549 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 413226, 2024-06-10T01:11:29.549 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2024-06-10T01:11:29.549 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 413226, 2024-06-10T01:11:29.549 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 127397.447497506, 2024-06-10T01:11:29.549 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.308299689 2024-06-10T01:11:29.549 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.550 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2024-06-10T01:11:29.550 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2024-06-10T01:11:29.550 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 413227, 2024-06-10T01:11:29.550 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 621876 2024-06-10T01:11:29.550 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.550 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 413227, 2024-06-10T01:11:29.550 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 139796, 2024-06-10T01:11:29.550 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 273431, 2024-06-10T01:11:29.551 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2024-06-10T01:11:29.551 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2024-06-10T01:11:29.551 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 27336, 2024-06-10T01:11:29.551 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 105665, 2024-06-10T01:11:29.551 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 613, 2024-06-10T01:11:29.551 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 32020, 2024-06-10T01:11:29.551 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 396, 2024-06-10T01:11:29.551 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2024-06-10T01:11:29.551 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2024-06-10T01:11:29.552 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2024-06-10T01:11:29.552 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 22, 2024-06-10T01:11:29.552 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 104198, 2024-06-10T01:11:29.552 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2024-06-10T01:11:29.552 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2024-06-10T01:11:29.552 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2024-06-10T01:11:29.552 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 139183, 2024-06-10T01:11:29.552 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 161610, 2024-06-10T01:11:29.553 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2024-06-10T01:11:29.553 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2024-06-10T01:11:29.553 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2024-06-10T01:11:29.553 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2024-06-10T01:11:29.553 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2024-06-10T01:11:29.553 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2024-06-10T01:11:29.553 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T01:11:29.554 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2024-06-10T01:11:29.554 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2024-06-10T01:11:29.554 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 1, 2024-06-10T01:11:29.554 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2024-06-10T01:11:29.554 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2024-06-10T01:11:29.554 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2024-06-10T01:11:29.554 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2024-06-10T01:11:29.554 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2024-06-10T01:11:29.555 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2024-06-10T01:11:29.555 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2024-06-10T01:11:29.555 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2024-06-10T01:11:29.555 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2024-06-10T01:11:29.555 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2024-06-10T01:11:29.555 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2024-06-10T01:11:29.555 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2024-06-10T01:11:29.555 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2024-06-10T01:11:29.555 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2024-06-10T01:11:29.556 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2024-06-10T01:11:29.556 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2024-06-10T01:11:29.556 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 105, 2024-06-10T01:11:29.556 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2024-06-10T01:11:29.556 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 16, 2024-06-10T01:11:29.556 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 12, 2024-06-10T01:11:29.556 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 12, 2024-06-10T01:11:29.556 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 0, 2024-06-10T01:11:29.557 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2024-06-10T01:11:29.557 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 49844, 2024-06-10T01:11:29.557 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 0, 2024-06-10T01:11:29.557 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 988 2024-06-10T01:11:29.557 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.557 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2024-06-10T01:11:29.557 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 1, 2024-06-10T01:11:29.558 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2024-06-10T01:11:29.558 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2024-06-10T01:11:29.558 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2024-06-10T01:11:29.558 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.558 INFO:teuthology.orchestra.run.smithi167.stdout: "osd-slow-ops": { 2024-06-10T01:11:29.558 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 141438 2024-06-10T01:11:29.558 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.558 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2024-06-10T01:11:29.559 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2024-06-10T01:11:29.559 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 79, 2024-06-10T01:11:29.559 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2024-06-10T01:11:29.559 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 10, 2024-06-10T01:11:29.559 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed_ops": 100264, 2024-06-10T01:11:29.559 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 50139, 2024-06-10T01:11:29.559 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2024-06-10T01:11:29.559 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.559 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T01:11:29.560 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-06-10T01:11:29.560 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-06-10T01:11:29.560 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-06-10T01:11:29.560 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 631769, 2024-06-10T01:11:29.560 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 191352204, 2024-06-10T01:11:29.560 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:29.560 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 631769, 2024-06-10T01:11:29.560 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-06-10T01:11:29.561 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-06-10T01:11:29.561 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 631769, 2024-06-10T01:11:29.561 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 191352204, 2024-06-10T01:11:29.561 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-06-10T01:11:29.561 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:29.561 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:29.561 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:29.561 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:29.562 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.562 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2024-06-10T01:11:29.562 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-06-10T01:11:29.562 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-06-10T01:11:29.562 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-06-10T01:11:29.562 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-06-10T01:11:29.562 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-06-10T01:11:29.562 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:29.563 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-06-10T01:11:29.563 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 413227, 2024-06-10T01:11:29.563 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 641676498, 2024-06-10T01:11:29.563 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 413227, 2024-06-10T01:11:29.563 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 641676498, 2024-06-10T01:11:29.563 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-06-10T01:11:29.563 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:29.563 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:29.564 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:29.564 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:29.564 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.564 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2024-06-10T01:11:29.564 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-06-10T01:11:29.564 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2024-06-10T01:11:29.564 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-06-10T01:11:29.564 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-06-10T01:11:29.565 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-06-10T01:11:29.565 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:29.565 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-06-10T01:11:29.565 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 413227, 2024-06-10T01:11:29.565 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 413227, 2024-06-10T01:11:29.565 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 413227, 2024-06-10T01:11:29.565 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 413227, 2024-06-10T01:11:29.565 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-06-10T01:11:29.566 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:29.566 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:29.566 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:29.566 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:29.566 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.566 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2024-06-10T01:11:29.566 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-06-10T01:11:29.566 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-06-10T01:11:29.567 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-06-10T01:11:29.567 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 50139, 2024-06-10T01:11:29.567 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 5464289, 2024-06-10T01:11:29.567 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:29.567 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 50139, 2024-06-10T01:11:29.567 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-06-10T01:11:29.567 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-06-10T01:11:29.567 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 603, 2024-06-10T01:11:29.568 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 5464289, 2024-06-10T01:11:29.568 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-06-10T01:11:29.568 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:29.568 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:29.568 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:29.568 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:29.568 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:29.568 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x55702a170500": { 2024-06-10T01:11:29.568 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-06-10T01:11:29.569 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-06-10T01:11:29.569 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-06-10T01:11:29.569 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 206089, 2024-06-10T01:11:29.569 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 483514145, 2024-06-10T01:11:29.569 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:29.569 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 206089, 2024-06-10T01:11:29.569 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-06-10T01:11:29.569 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-06-10T01:11:29.570 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 31417, 2024-06-10T01:11:29.570 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 483514145, 2024-06-10T01:11:29.570 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-06-10T01:11:29.570 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:29.570 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:29.570 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:29.570 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:29.570 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:29.571 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-06-10T01:11:30.263 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-06-10T01:11:30.264 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-06-10T01:11:30.264 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=382 2024-06-10T01:11:30.264 DEBUG:tasks.check_counter:Getting stats from l 2024-06-10T01:11:30.264 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph daemon mds.l perf dump 2024-06-10T01:11:30.502 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.c/config 2024-06-10T01:11:30.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:30 smithi167 ceph-mon[36917]: pgmap v879: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T01:11:30.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:30 smithi094 ceph-mon[36708]: pgmap v879: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T01:11:30.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:30 smithi037 ceph-mon[32192]: pgmap v879: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-10T01:11:32.210 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-06-10T01:11:32.211 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-06-10T01:11:32.211 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 743, 2024-06-10T01:11:32.211 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 732, 2024-06-10T01:11:32.211 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 38703513, 2024-06-10T01:11:32.211 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 325740, 2024-06-10T01:11:32.211 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 7, 2024-06-10T01:11:32.212 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 3, 2024-06-10T01:11:32.212 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.111477699, 2024-06-10T01:11:32.212 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.029706658, 2024-06-10T01:11:32.212 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.057521860, 2024-06-10T01:11:32.212 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.020872508, 2024-06-10T01:11:32.212 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:32.212 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 732, 2024-06-10T01:11:32.212 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.037392495, 2024-06-10T01:11:32.212 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000051082 2024-06-10T01:11:32.213 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.213 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:32.213 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.213 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.213 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.213 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.213 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 820496, 2024-06-10T01:11:32.213 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 197264 2024-06-10T01:11:32.214 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.214 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-06-10T01:11:32.214 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 362, 2024-06-10T01:11:32.214 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 602, 2024-06-10T01:11:32.214 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 67213662, 2024-06-10T01:11:32.214 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 347808, 2024-06-10T01:11:32.214 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 7, 2024-06-10T01:11:32.214 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 6, 2024-06-10T01:11:32.215 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.078693068, 2024-06-10T01:11:32.215 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.018479336, 2024-06-10T01:11:32.215 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.045381775, 2024-06-10T01:11:32.215 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.009802666, 2024-06-10T01:11:32.215 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:32.215 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 600, 2024-06-10T01:11:32.216 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.032200198, 2024-06-10T01:11:32.216 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000053666 2024-06-10T01:11:32.216 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.216 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:32.216 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.216 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.216 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.216 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.216 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-10T01:11:32.217 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 232944 2024-06-10T01:11:32.217 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.217 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-06-10T01:11:32.217 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 513, 2024-06-10T01:11:32.217 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 515, 2024-06-10T01:11:32.217 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 100804667, 2024-06-10T01:11:32.217 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 163141, 2024-06-10T01:11:32.217 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 6, 2024-06-10T01:11:32.217 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 5, 2024-06-10T01:11:32.218 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.089076813, 2024-06-10T01:11:32.218 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.013195544, 2024-06-10T01:11:32.218 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.060423442, 2024-06-10T01:11:32.218 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.012195515, 2024-06-10T01:11:32.218 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-06-10T01:11:32.218 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 513, 2024-06-10T01:11:32.218 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.024458552, 2024-06-10T01:11:32.218 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000047677 2024-06-10T01:11:32.219 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.219 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-06-10T01:11:32.219 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.219 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.219 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.219 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.219 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-10T01:11:32.219 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 0 2024-06-10T01:11:32.220 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.220 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-06-10T01:11:32.220 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2024-06-10T01:11:32.220 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-06-10T01:11:32.220 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.220 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2024-06-10T01:11:32.220 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-06-10T01:11:32.220 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-06-10T01:11:32.220 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1998, 2024-06-10T01:11:32.221 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.340671518, 2024-06-10T01:11:32.221 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000170506 2024-06-10T01:11:32.221 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:32.221 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.221 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2024-06-10T01:11:32.221 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-06-10T01:11:32.221 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-06-10T01:11:32.221 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.222 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.222 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.222 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:32.222 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.222 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2024-06-10T01:11:32.222 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 0, 2024-06-10T01:11:32.222 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 0, 2024-06-10T01:11:32.222 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2024-06-10T01:11:32.223 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.223 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.223 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.223 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.223 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2024-06-10T01:11:32.223 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 0, 2024-06-10T01:11:32.223 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2024-06-10T01:11:32.223 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2024-06-10T01:11:32.223 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 0, 2024-06-10T01:11:32.224 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 0, 2024-06-10T01:11:32.224 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2024-06-10T01:11:32.224 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 44770, 2024-06-10T01:11:32.224 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 31314, 2024-06-10T01:11:32.224 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 13456, 2024-06-10T01:11:32.224 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 0, 2024-06-10T01:11:32.224 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 36, 2024-06-10T01:11:32.224 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 0, 2024-06-10T01:11:32.225 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 0, 2024-06-10T01:11:32.225 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 0, 2024-06-10T01:11:32.225 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 2, 2024-06-10T01:11:32.225 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 0, 2024-06-10T01:11:32.225 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 0, 2024-06-10T01:11:32.225 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 0, 2024-06-10T01:11:32.225 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 0, 2024-06-10T01:11:32.225 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2024-06-10T01:11:32.225 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2024-06-10T01:11:32.226 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 0, 2024-06-10T01:11:32.226 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 0, 2024-06-10T01:11:32.226 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2024-06-10T01:11:32.226 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 0, 2024-06-10T01:11:32.226 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 0, 2024-06-10T01:11:32.226 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 0, 2024-06-10T01:11:32.226 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 0, 2024-06-10T01:11:32.226 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2024-06-10T01:11:32.227 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 0, 2024-06-10T01:11:32.227 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2024-06-10T01:11:32.227 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 4, 2024-06-10T01:11:32.227 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 234, 2024-06-10T01:11:32.227 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 0, 2024-06-10T01:11:32.227 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 0, 2024-06-10T01:11:32.227 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2024-06-10T01:11:32.227 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2024-06-10T01:11:32.228 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2024-06-10T01:11:32.228 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 0, 2024-06-10T01:11:32.228 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2024-06-10T01:11:32.228 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 0, 2024-06-10T01:11:32.228 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 0, 2024-06-10T01:11:32.228 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 0, 2024-06-10T01:11:32.228 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 0, 2024-06-10T01:11:32.228 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 0, 2024-06-10T01:11:32.229 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2024-06-10T01:11:32.229 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-10T01:11:32.229 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-10T01:11:32.229 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 0, 2024-06-10T01:11:32.229 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 0, 2024-06-10T01:11:32.229 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 0, 2024-06-10T01:11:32.229 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 0 2024-06-10T01:11:32.229 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.230 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2024-06-10T01:11:32.230 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 0, 2024-06-10T01:11:32.230 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 0, 2024-06-10T01:11:32.230 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 0, 2024-06-10T01:11:32.230 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 0, 2024-06-10T01:11:32.230 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 0, 2024-06-10T01:11:32.230 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2024-06-10T01:11:32.230 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2024-06-10T01:11:32.231 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2024-06-10T01:11:32.231 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 0, 2024-06-10T01:11:32.231 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 0, 2024-06-10T01:11:32.231 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2024-06-10T01:11:32.231 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2024-06-10T01:11:32.231 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2024-06-10T01:11:32.231 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2024-06-10T01:11:32.231 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2024-06-10T01:11:32.232 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2024-06-10T01:11:32.232 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2024-06-10T01:11:32.232 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_path": 0, 2024-06-10T01:11:32.232 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_inode": 0, 2024-06-10T01:11:32.232 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2024-06-10T01:11:32.232 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 0, 2024-06-10T01:11:32.232 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2024-06-10T01:11:32.232 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 0, 2024-06-10T01:11:32.233 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2024-06-10T01:11:32.233 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2024-06-10T01:11:32.233 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.233 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2024-06-10T01:11:32.233 INFO:teuthology.orchestra.run.smithi167.stdout: "evlrg": 0, 2024-06-10T01:11:32.233 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 0, 2024-06-10T01:11:32.233 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2024-06-10T01:11:32.233 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2024-06-10T01:11:32.233 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 89228, 2024-06-10T01:11:32.234 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2024-06-10T01:11:32.234 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2024-06-10T01:11:32.234 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 0, 2024-06-10T01:11:32.234 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2024-06-10T01:11:32.234 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2024-06-10T01:11:32.234 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 2, 2024-06-10T01:11:32.234 INFO:teuthology.orchestra.run.smithi167.stdout: "segmjr": 18, 2024-06-10T01:11:32.235 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2024-06-10T01:11:32.235 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2024-06-10T01:11:32.235 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 487447318, 2024-06-10T01:11:32.235 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 487708449, 2024-06-10T01:11:32.235 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 487706092, 2024-06-10T01:11:32.235 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2024-06-10T01:11:32.235 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.235 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.235 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.236 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.236 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 89228 2024-06-10T01:11:32.236 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.236 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2024-06-10T01:11:32.236 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 21, 2024-06-10T01:11:32.236 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 43362, 2024-06-10T01:11:32.237 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 43341, 2024-06-10T01:11:32.237 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 40, 2024-06-10T01:11:32.237 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 2581, 2024-06-10T01:11:32.237 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 2541, 2024-06-10T01:11:32.237 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 44770, 2024-06-10T01:11:32.237 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 88106, 2024-06-10T01:11:32.237 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 43336, 2024-06-10T01:11:32.237 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 0, 2024-06-10T01:11:32.238 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 0, 2024-06-10T01:11:32.238 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 0, 2024-06-10T01:11:32.238 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 163640, 2024-06-10T01:11:32.238 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 215420 2024-06-10T01:11:32.238 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.238 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2024-06-10T01:11:32.238 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 0, 2024-06-10T01:11:32.239 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 0, 2024-06-10T01:11:32.239 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 0, 2024-06-10T01:11:32.239 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 0, 2024-06-10T01:11:32.239 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 0, 2024-06-10T01:11:32.239 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2024-06-10T01:11:32.239 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.239 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.239 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.239 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.240 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2024-06-10T01:11:32.240 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.240 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.240 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.240 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.240 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2024-06-10T01:11:32.240 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.240 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.240 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.241 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.241 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2024-06-10T01:11:32.241 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.241 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.241 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.241 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.241 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2024-06-10T01:11:32.241 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.242 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.242 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.242 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.242 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2024-06-10T01:11:32.242 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.242 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.242 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.242 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.243 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2024-06-10T01:11:32.243 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.243 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.243 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.243 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.243 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2024-06-10T01:11:32.243 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.243 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.243 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.244 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.244 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2024-06-10T01:11:32.244 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.244 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.244 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.244 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.244 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2024-06-10T01:11:32.244 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.245 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.245 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.245 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.245 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2024-06-10T01:11:32.245 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.245 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.245 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.245 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.246 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2024-06-10T01:11:32.246 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.247 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.247 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.247 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.247 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2024-06-10T01:11:32.247 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.247 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.247 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.247 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.248 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2024-06-10T01:11:32.248 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.248 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.248 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.248 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.248 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2024-06-10T01:11:32.248 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.248 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.249 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.249 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.249 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2024-06-10T01:11:32.249 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.249 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.249 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.249 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.249 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2024-06-10T01:11:32.249 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.250 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.250 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.250 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.250 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2024-06-10T01:11:32.250 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.250 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.250 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.250 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.251 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2024-06-10T01:11:32.251 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.251 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.251 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.251 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.251 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2024-06-10T01:11:32.251 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.251 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.252 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.252 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.252 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2024-06-10T01:11:32.252 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.252 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.252 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.252 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.252 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2024-06-10T01:11:32.252 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.253 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.253 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.253 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.253 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2024-06-10T01:11:32.253 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.253 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.253 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.253 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.254 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2024-06-10T01:11:32.254 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.254 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.254 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.254 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.254 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2024-06-10T01:11:32.254 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.254 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.255 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.255 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.255 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2024-06-10T01:11:32.255 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.255 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.255 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.255 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.255 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2024-06-10T01:11:32.256 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.256 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.256 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.256 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.256 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2024-06-10T01:11:32.256 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.256 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.256 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.257 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.257 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2024-06-10T01:11:32.257 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.257 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.257 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.257 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.257 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2024-06-10T01:11:32.257 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2024-06-10T01:11:32.257 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2024-06-10T01:11:32.258 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.258 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.258 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.258 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:32.258 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.258 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2024-06-10T01:11:32.258 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 1, 2024-06-10T01:11:32.258 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 1, 2024-06-10T01:11:32.259 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 0, 2024-06-10T01:11:32.259 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 0, 2024-06-10T01:11:32.259 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2024-06-10T01:11:32.259 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 0, 2024-06-10T01:11:32.259 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 0, 2024-06-10T01:11:32.259 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 2450, 2024-06-10T01:11:32.259 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2024-06-10T01:11:32.259 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.260 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-06-10T01:11:32.260 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-06-10T01:11:32.260 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-06-10T01:11:32.260 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-06-10T01:11:32.260 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-06-10T01:11:32.260 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-06-10T01:11:32.260 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-06-10T01:11:32.260 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-10T01:11:32.260 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-06-10T01:11:32.261 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-10T01:11:32.261 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-06-10T01:11:32.261 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-06-10T01:11:32.261 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-06-10T01:11:32.261 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-10T01:11:32.261 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-06-10T01:11:32.261 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-06-10T01:11:32.261 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-06-10T01:11:32.262 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-06-10T01:11:32.262 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-06-10T01:11:32.262 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-10T01:11:32.262 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-06-10T01:11:32.262 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-10T01:11:32.262 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-06-10T01:11:32.262 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-06-10T01:11:32.262 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-06-10T01:11:32.263 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-06-10T01:11:32.263 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-06-10T01:11:32.263 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-10T01:11:32.263 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-06-10T01:11:32.263 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-06-10T01:11:32.263 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-06-10T01:11:32.263 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-06-10T01:11:32.263 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-06-10T01:11:32.263 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-06-10T01:11:32.264 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-06-10T01:11:32.264 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-06-10T01:11:32.264 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-06-10T01:11:32.264 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 239603, 2024-06-10T01:11:32.264 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 82, 2024-06-10T01:11:32.264 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2024-06-10T01:11:32.264 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2024-06-10T01:11:32.264 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-06-10T01:11:32.265 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-06-10T01:11:32.265 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-06-10T01:11:32.265 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-06-10T01:11:32.265 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-06-10T01:11:32.265 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-06-10T01:11:32.265 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 9176, 2024-06-10T01:11:32.265 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 176, 2024-06-10T01:11:32.265 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-06-10T01:11:32.265 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-06-10T01:11:32.266 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-06-10T01:11:32.266 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-06-10T01:11:32.266 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 31608826, 2024-06-10T01:11:32.266 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 178232, 2024-06-10T01:11:32.266 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-06-10T01:11:32.266 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-06-10T01:11:32.266 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-06-10T01:11:32.267 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-06-10T01:11:32.267 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.267 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2024-06-10T01:11:32.267 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2024-06-10T01:11:32.267 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2024-06-10T01:11:32.267 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 1278, 2024-06-10T01:11:32.267 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 26, 2024-06-10T01:11:32.267 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 4, 2024-06-10T01:11:32.267 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 1278, 2024-06-10T01:11:32.268 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2024-06-10T01:11:32.268 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1278, 2024-06-10T01:11:32.268 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 2.100963892, 2024-06-10T01:11:32.268 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.001643946 2024-06-10T01:11:32.268 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.268 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2024-06-10T01:11:32.268 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2024-06-10T01:11:32.268 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1278, 2024-06-10T01:11:32.269 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 1279 2024-06-10T01:11:32.269 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.269 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 1278, 2024-06-10T01:11:32.269 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 1278, 2024-06-10T01:11:32.269 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 0, 2024-06-10T01:11:32.269 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2024-06-10T01:11:32.269 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2024-06-10T01:11:32.269 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 759, 2024-06-10T01:11:32.270 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 0, 2024-06-10T01:11:32.270 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 516, 2024-06-10T01:11:32.270 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 0, 2024-06-10T01:11:32.270 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 0, 2024-06-10T01:11:32.270 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2024-06-10T01:11:32.270 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2024-06-10T01:11:32.270 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2024-06-10T01:11:32.270 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 0, 2024-06-10T01:11:32.271 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 0, 2024-06-10T01:11:32.271 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2024-06-10T01:11:32.271 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2024-06-10T01:11:32.271 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2024-06-10T01:11:32.271 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 2, 2024-06-10T01:11:32.271 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 0, 2024-06-10T01:11:32.271 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2024-06-10T01:11:32.271 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2024-06-10T01:11:32.272 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2024-06-10T01:11:32.272 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2024-06-10T01:11:32.272 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2024-06-10T01:11:32.272 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2024-06-10T01:11:32.272 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2024-06-10T01:11:32.272 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2024-06-10T01:11:32.272 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2024-06-10T01:11:32.272 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2024-06-10T01:11:32.272 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2024-06-10T01:11:32.273 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2024-06-10T01:11:32.273 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2024-06-10T01:11:32.273 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2024-06-10T01:11:32.273 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2024-06-10T01:11:32.273 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2024-06-10T01:11:32.273 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2024-06-10T01:11:32.273 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2024-06-10T01:11:32.273 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2024-06-10T01:11:32.274 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2024-06-10T01:11:32.274 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2024-06-10T01:11:32.274 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2024-06-10T01:11:32.274 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2024-06-10T01:11:32.274 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2024-06-10T01:11:32.274 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2024-06-10T01:11:32.274 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2024-06-10T01:11:32.274 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 105, 2024-06-10T01:11:32.275 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2024-06-10T01:11:32.275 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 4, 2024-06-10T01:11:32.275 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 11, 2024-06-10T01:11:32.275 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 11, 2024-06-10T01:11:32.275 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 0, 2024-06-10T01:11:32.275 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2024-06-10T01:11:32.275 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 0, 2024-06-10T01:11:32.275 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 2, 2024-06-10T01:11:32.275 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 0 2024-06-10T01:11:32.276 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.276 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2024-06-10T01:11:32.276 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 0, 2024-06-10T01:11:32.276 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2024-06-10T01:11:32.276 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2024-06-10T01:11:32.276 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2024-06-10T01:11:32.276 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.276 INFO:teuthology.orchestra.run.smithi167.stdout: "osd-slow-ops": { 2024-06-10T01:11:32.277 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 0 2024-06-10T01:11:32.277 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.277 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2024-06-10T01:11:32.277 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2024-06-10T01:11:32.277 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 0, 2024-06-10T01:11:32.277 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2024-06-10T01:11:32.277 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 0, 2024-06-10T01:11:32.278 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed_ops": 0, 2024-06-10T01:11:32.278 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 0, 2024-06-10T01:11:32.278 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2024-06-10T01:11:32.278 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.278 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-10T01:11:32.278 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-06-10T01:11:32.278 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-06-10T01:11:32.278 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-06-10T01:11:32.279 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 1614, 2024-06-10T01:11:32.279 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 206556368, 2024-06-10T01:11:32.279 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:32.279 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 1614, 2024-06-10T01:11:32.279 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-06-10T01:11:32.279 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-06-10T01:11:32.279 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 1614, 2024-06-10T01:11:32.279 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 206556368, 2024-06-10T01:11:32.279 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-06-10T01:11:32.280 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.280 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.280 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.280 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:32.280 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.280 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2024-06-10T01:11:32.280 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-06-10T01:11:32.280 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-06-10T01:11:32.281 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-06-10T01:11:32.281 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-06-10T01:11:32.281 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-06-10T01:11:32.281 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:32.281 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-06-10T01:11:32.281 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 1278, 2024-06-10T01:11:32.281 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 205545298, 2024-06-10T01:11:32.281 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 1278, 2024-06-10T01:11:32.281 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 205545298, 2024-06-10T01:11:32.282 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-06-10T01:11:32.282 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.282 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.282 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.282 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:32.282 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.282 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2024-06-10T01:11:32.282 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-06-10T01:11:32.283 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2024-06-10T01:11:32.283 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-06-10T01:11:32.283 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-06-10T01:11:32.283 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-06-10T01:11:32.283 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:32.283 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-06-10T01:11:32.283 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 1278, 2024-06-10T01:11:32.284 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 1278, 2024-06-10T01:11:32.284 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 1278, 2024-06-10T01:11:32.284 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 1278, 2024-06-10T01:11:32.284 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-06-10T01:11:32.284 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.284 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.284 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.284 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:32.285 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.285 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2024-06-10T01:11:32.285 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-06-10T01:11:32.285 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-06-10T01:11:32.285 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-06-10T01:11:32.285 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-06-10T01:11:32.285 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-06-10T01:11:32.285 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:32.286 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-06-10T01:11:32.286 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-06-10T01:11:32.286 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-06-10T01:11:32.286 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-06-10T01:11:32.286 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-06-10T01:11:32.286 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-06-10T01:11:32.286 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.286 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.287 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.287 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:32.287 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-06-10T01:11:32.287 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x564643ed46e0": { 2024-06-10T01:11:32.287 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-06-10T01:11:32.287 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-06-10T01:11:32.287 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-06-10T01:11:32.287 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-06-10T01:11:32.287 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-06-10T01:11:32.288 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-06-10T01:11:32.288 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-06-10T01:11:32.288 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-06-10T01:11:32.288 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-06-10T01:11:32.288 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-06-10T01:11:32.288 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-06-10T01:11:32.288 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-06-10T01:11:32.288 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-06-10T01:11:32.289 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-06-10T01:11:32.289 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-06-10T01:11:32.289 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:32.289 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-06-10T01:11:32.289 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-06-10T01:11:32.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:32 smithi167 ceph-mon[36917]: pgmap v880: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:11:32.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:32 smithi094 ceph-mon[36708]: pgmap v880: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:11:32.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:32 smithi037 ceph-mon[32192]: pgmap v880: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-10T01:11:33.036 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-06-10T01:11:33.036 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-06-10T01:11:33.036 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-06-10T01:11:33.037 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph_7b7a5d375cf4e4afab96b9d4d229b699ae8304bb/qa/tasks/check_counter.py", line 125, in end raise RuntimeError("The following counters failed to be set " RuntimeError: The following counters failed to be set on mds daemons: {'mds.imported', 'mds.exported'} 2024-06-10T01:11:33.045 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-06-10T01:11:33.057 INFO:tasks.kclient:Unmounting kernel clients... 2024-06-10T01:11:33.057 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-06-10T01:11:33.057 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-10T01:11:33.093 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-06-10T01:11:33.093 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-10T01:11:33.154 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-10T01:11:33.459 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:11:34.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:34 smithi167 ceph-mon[36917]: pgmap v881: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:11:34.592 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:34 smithi037 ceph-mon[32192]: pgmap v881: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:11:34.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:34 smithi094 ceph-mon[36708]: pgmap v881: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-10T01:11:35.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.235+0000 7f7901da7700 1 -- 172.21.15.37:0/889914149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78fc102ef0 msgr2=0x7f78fc10f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:35.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.235+0000 7f7901da7700 1 --2- 172.21.15.37:0/889914149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78fc102ef0 0x7f78fc10f3a0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f78f400b0d0 tx=0x7f78f400b3e0 comp rx=0 tx=0).stop 2024-06-10T01:11:35.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.235+0000 7f7901da7700 1 -- 172.21.15.37:0/889914149 shutdown_connections 2024-06-10T01:11:35.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.235+0000 7f7901da7700 1 --2- 172.21.15.37:0/889914149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78fc102ef0 0x7f78fc10f3a0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:35.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.235+0000 7f7901da7700 1 --2- 172.21.15.37:0/889914149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78fc1085d0 0x7f78fc1089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.235+0000 7f7901da7700 1 --2- 172.21.15.37:0/889914149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78fc102530 0x7f78fc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.235+0000 7f7901da7700 1 -- 172.21.15.37:0/889914149 >> 172.21.15.37:0/889914149 conn(0x7f78fc079640 msgr2=0x7f78fc079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.235+0000 7f7901da7700 1 -- 172.21.15.37:0/889914149 shutdown_connections 2024-06-10T01:11:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.235+0000 7f7901da7700 1 -- 172.21.15.37:0/889914149 wait complete. 2024-06-10T01:11:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.236+0000 7f7901da7700 1 Processor -- start 2024-06-10T01:11:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.236+0000 7f7901da7700 1 -- start start 2024-06-10T01:11:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.237+0000 7f7901da7700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78fc102530 0x7f78fc19e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.237+0000 7f7901da7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78fc102ef0 0x7f78fc19ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.237+0000 7f7901da7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78fc1085d0 0x7f78fc1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:35.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.237+0000 7f7901da7700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f78fc112860 con 0x7f78fc102ef0 2024-06-10T01:11:35.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.237+0000 7f7901da7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f78fc1126e0 con 0x7f78fc1085d0 2024-06-10T01:11:35.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.237+0000 7f7901da7700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f78fc1129e0 con 0x7f78fc102530 2024-06-10T01:11:35.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.237+0000 7f78f2ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78fc102ef0 0x7f78fc19ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:35.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.237+0000 7f78f2ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78fc102ef0 0x7f78fc19ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42830/0 (socket says 172.21.15.37:42830) 2024-06-10T01:11:35.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.237+0000 7f78f2ffd700 1 -- 172.21.15.37:0/2499594299 learned_addr learned my addr 172.21.15.37:0/2499594299 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:11:35.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.237+0000 7f78fbfff700 1 --2- 172.21.15.37:0/2499594299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78fc1085d0 0x7f78fc1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:35.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.237+0000 7f78fb7fe700 1 --2- 172.21.15.37:0/2499594299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78fc102530 0x7f78fc19e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:35.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.237+0000 7f78f2ffd700 1 -- 172.21.15.37:0/2499594299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78fc102530 msgr2=0x7f78fc19e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:35.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.237+0000 7f78f2ffd700 1 --2- 172.21.15.37:0/2499594299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78fc102530 0x7f78fc19e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:35.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.237+0000 7f78f2ffd700 1 -- 172.21.15.37:0/2499594299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78fc1085d0 msgr2=0x7f78fc1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:35.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.237+0000 7f78f2ffd700 1 --2- 172.21.15.37:0/2499594299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78fc1085d0 0x7f78fc1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:35.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.237+0000 7f78f2ffd700 1 -- 172.21.15.37:0/2499594299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78fc199020 con 0x7f78fc102ef0 2024-06-10T01:11:35.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.237+0000 7f78fb7fe700 1 --2- 172.21.15.37:0/2499594299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78fc102530 0x7f78fc19e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:11:35.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.237+0000 7f78f2ffd700 1 --2- 172.21.15.37:0/2499594299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78fc102ef0 0x7f78fc19ec40 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f78e400c9b0 tx=0x7f78e400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:35.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.238+0000 7f78f97fa700 1 -- 172.21.15.37:0/2499594299 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e4007b00 con 0x7f78fc102ef0 2024-06-10T01:11:35.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.238+0000 7f7901da7700 1 -- 172.21.15.37:0/2499594299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78fc199310 con 0x7f78fc102ef0 2024-06-10T01:11:35.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.238+0000 7f78f97fa700 1 -- 172.21.15.37:0/2499594299 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78e4007ca0 con 0x7f78fc102ef0 2024-06-10T01:11:35.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.238+0000 7f7901da7700 1 -- 172.21.15.37:0/2499594299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78fc1a55d0 con 0x7f78fc102ef0 2024-06-10T01:11:35.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.238+0000 7f78f97fa700 1 -- 172.21.15.37:0/2499594299 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e4018730 con 0x7f78fc102ef0 2024-06-10T01:11:35.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.239+0000 7f7901da7700 1 -- 172.21.15.37:0/2499594299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78fc069180 con 0x7f78fc102ef0 2024-06-10T01:11:35.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.240+0000 7f78f97fa700 1 -- 172.21.15.37:0/2499594299 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f78e40188d0 con 0x7f78fc102ef0 2024-06-10T01:11:35.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.241+0000 7f78f97fa700 1 --2- 172.21.15.37:0/2499594299 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f78e80a2c20 0x7f78e80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:35.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.241+0000 7f78fb7fe700 1 --2- 172.21.15.37:0/2499594299 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f78e80a2c20 0x7f78e80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:35.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.241+0000 7f78f97fa700 1 -- 172.21.15.37:0/2499594299 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7f78e40cf6d0 con 0x7f78fc102ef0 2024-06-10T01:11:35.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.241+0000 7f78fb7fe700 1 --2- 172.21.15.37:0/2499594299 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f78e80a2c20 0x7f78e80a50e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f78fc199a00 tx=0x7f78ec00a680 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:35.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.244+0000 7f78f97fa700 1 -- 172.21.15.37:0/2499594299 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f78e4014030 con 0x7f78fc102ef0 2024-06-10T01:11:35.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.472+0000 7f7901da7700 1 -- 172.21.15.37:0/2499594299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f78fc199df0 con 0x7f78fc102ef0 2024-06-10T01:11:35.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.472+0000 7f78f97fa700 1 -- 172.21.15.37:0/2499594299 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v105) ==== 81+0+591 (secure 0 0 0) 0x7f78fc199df0 con 0x7f78fc102ef0 2024-06-10T01:11:35.467 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/4084722397 2024-06-11T00:45:20.254342+0000 2024-06-10T01:11:35.468 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3571803585 2024-06-11T00:45:20.254342+0000 2024-06-10T01:11:35.468 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/416587443 2024-06-11T00:45:20.254342+0000 2024-06-10T01:11:35.468 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/2766692263 2024-06-11T00:45:20.254342+0000 2024-06-10T01:11:35.468 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2026344134 2024-06-11T00:44:53.913969+0000 2024-06-10T01:11:35.468 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2781321886 2024-06-11T00:44:53.913969+0000 2024-06-10T01:11:35.468 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/2803174880 2024-06-11T00:44:53.913969+0000 2024-06-10T01:11:35.468 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/2803174880 2024-06-11T00:44:53.913969+0000 2024-06-10T01:11:35.469 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1110143997 2024-06-11T00:44:53.913969+0000 2024-06-10T01:11:35.469 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/2766692263 2024-06-11T00:45:20.254342+0000 2024-06-10T01:11:35.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.474+0000 7f7901da7700 1 -- 172.21.15.37:0/2499594299 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f78e80a2c20 msgr2=0x7f78e80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:35.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.474+0000 7f7901da7700 1 --2- 172.21.15.37:0/2499594299 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f78e80a2c20 0x7f78e80a50e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f78fc199a00 tx=0x7f78ec00a680 comp rx=0 tx=0).stop 2024-06-10T01:11:35.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.474+0000 7f7901da7700 1 -- 172.21.15.37:0/2499594299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78fc102ef0 msgr2=0x7f78fc19ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:35.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.474+0000 7f7901da7700 1 --2- 172.21.15.37:0/2499594299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78fc102ef0 0x7f78fc19ec40 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f78e400c9b0 tx=0x7f78e400ccc0 comp rx=0 tx=0).stop 2024-06-10T01:11:35.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.474+0000 7f7901da7700 1 -- 172.21.15.37:0/2499594299 shutdown_connections 2024-06-10T01:11:35.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.474+0000 7f7901da7700 1 --2- 172.21.15.37:0/2499594299 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f78e80a2c20 0x7f78e80a50e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:35.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.474+0000 7f7901da7700 1 --2- 172.21.15.37:0/2499594299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78fc1085d0 0x7f78fc1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:35.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.474+0000 7f7901da7700 1 --2- 172.21.15.37:0/2499594299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78fc102530 0x7f78fc19e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:35.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.474+0000 7f7901da7700 1 --2- 172.21.15.37:0/2499594299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78fc102ef0 0x7f78fc19ec40 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:35.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.474+0000 7f7901da7700 1 -- 172.21.15.37:0/2499594299 >> 172.21.15.37:0/2499594299 conn(0x7f78fc079640 msgr2=0x7f78fc100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:35.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.475+0000 7f7901da7700 1 -- 172.21.15.37:0/2499594299 shutdown_connections 2024-06-10T01:11:35.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:35.475+0000 7f7901da7700 1 -- 172.21.15.37:0/2499594299 wait complete. 2024-06-10T01:11:35.471 INFO:teuthology.orchestra.run.smithi037.stderr:listed 10 entries 2024-06-10T01:11:36.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:35 smithi167 ceph-mon[36917]: pgmap v882: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:11:36.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:35 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2499594299' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T01:11:36.107 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:35 smithi037 ceph-mon[32192]: pgmap v882: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:11:36.107 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:35 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2499594299' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T01:11:36.108 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:35 smithi094 ceph-mon[36708]: pgmap v882: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-10T01:11:36.109 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:35 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2499594299' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-10T01:11:36.136 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-06-10T01:11:36.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:11:36.137 DEBUG:teuthology.orchestra.run.smithi037:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-06-10T01:11:36.187 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-06-10T01:11:36.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:11:36.187 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-06-10T01:11:36.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:11:36.232 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-06-10T01:11:36.309 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-06-10T01:11:36.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:11:36.310 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-10T01:11:36.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:11:36.337 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-06-10T01:11:36.443 INFO:tasks.kclient:Unmounting kernel clients... 2024-06-10T01:11:36.444 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-06-10T01:11:36.444 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-10T01:11:36.464 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498872k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20808 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249432k,nr_inodes=812358,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/72917fc4fee6157b8b6a0eba1bea40127e1343f851b3aa47a0130217bf6d0133/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/TRJMER24OEVUG6GOY77ZIVLA57:/var/lib/containers/storage/overlay/l/OUQYFINT4JZQ6O5ND75IXNTGSJ,upperdir=/var/lib/containers/storage/overlay/72917fc4fee6157b8b6a0eba1bea40127e1343f851b3aa47a0130217bf6d0133/diff,workdir=/var/lib/containers/storage/overlay/72917fc4fee6157b8b6a0eba1bea40127e1343f851b3aa47a0130217bf6d0133/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b82c3a83afe005e7dcbc5a3602cc27fdbe2a32a1be1362e2bf0a61254a6da392/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/TRJMER24OEVUG6GOY77ZIVLA57:/var/lib/containers/storage/overlay/l/OUQYFINT4JZQ6O5ND75IXNTGSJ,upperdir=/var/lib/containers/storage/overlay/b82c3a83afe005e7dcbc5a3602cc27fdbe2a32a1be1362e2bf0a61254a6da392/diff,workdir=/var/lib/containers/storage/overlay/b82c3a83afe005e7dcbc5a3602cc27fdbe2a32a1be1362e2bf0a61254a6da392/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249432k,nr_inodes=812358,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/aa501ada4d39d1720ec9dd405b503da8def30043a3ac4ae8418e40934a917bdd/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/TRJMER24OEVUG6GOY77ZIVLA57:/var/lib/containers/storage/overlay/l/OUQYFINT4JZQ6O5ND75IXNTGSJ,upperdir=/var/lib/containers/storage/overlay/aa501ada4d39d1720ec9dd405b503da8def30043a3ac4ae8418e40934a917bdd/diff,workdir=/var/lib/containers/storage/overlay/aa501ada4d39d1720ec9dd405b503da8def30043a3ac4ae8418e40934a917bdd/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/78504ec97d528fb1ec24a1049be338f6803877526d40795823c55652465dacd5/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/TRJMER24OEVUG6GOY77ZIVLA57:/var/lib/containers/storage/overlay/l/OUQYFINT4JZQ6O5ND75IXNTGSJ,upperdir=/var/lib/containers/storage/overlay/78504ec97d528fb1ec24a1049be338f6803877526d40795823c55652465dacd5/diff,workdir=/var/lib/containers/storage/overlay/78504ec97d528fb1ec24a1049be338f6803877526d40795823c55652465dacd5/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/9eb4b48744bd472142e43a97d10ba70b9d5f62ecb8c7bd6c68a0968f9156c735/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/TRJMER24OEVUG6GOY77ZIVLA57:/var/lib/containers/storage/overlay/l/OUQYFINT4JZQ6O5ND75IXNTGSJ,upperdir=/var/lib/containers/storage/overlay/9eb4b48744bd472142e43a97d10ba70b9d5f62ecb8c7bd6c68a0968f9156c735/diff,workdir=/var/lib/containers/storage/overlay/9eb4b48744bd472142e43a97d10ba70b9d5f62ecb8c7bd6c68a0968f9156c735/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/9a576aba0a8fd3f77e135cbd271e5d92716de545f053cc548581ffbd189cb41e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/TRJMER24OEVUG6GOY77ZIVLA57:/var/lib/containers/storage/overlay/l/OUQYFINT4JZQ6O5ND75IXNTGSJ,upperdir=/var/lib/containers/storage/overlay/9a576aba0a8fd3f77e135cbd271e5d92716de545f053cc548581ffbd189cb41e/diff,workdir=/var/lib/containers/storage/overlay/9a576aba0a8fd3f77e135cbd271e5d92716de545f053cc548581ffbd189cb41e/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/aa368ccca5bf4ed7574503daef34fe807e89f2001244e2e14262f12196c33c7e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/TRJMER24OEVUG6GOY77ZIVLA57:/var/lib/containers/storage/overlay/l/OUQYFINT4JZQ6O5ND75IXNTGSJ,upperdir=/var/lib/containers/storage/overlay/aa368ccca5bf4ed7574503daef34fe807e89f2001244e2e14262f12196c33c7e/diff,workdir=/var/lib/containers/storage/overlay/aa368ccca5bf4ed7574503daef34fe807e89f2001244e2e14262f12196c33c7e/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/66d66b5fdf853912e3ea32ae06dad5909d6c06fe121e6e997bb6347eacd097c2/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/TRJMER24OEVUG6GOY77ZIVLA57:/var/lib/containers/storage/overlay/l/OUQYFINT4JZQ6O5ND75IXNTGSJ,upperdir=/var/lib/containers/storage/overlay/66d66b5fdf853912e3ea32ae06dad5909d6c06fe121e6e997bb6347eacd097c2/diff,workdir=/var/lib/containers/storage/overlay/66d66b5fdf853912e3ea32ae06dad5909d6c06fe121e6e997bb6347eacd097c2/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/00e2e5150eb6e25db344ee6a8c032b156500eb01d194d65a47c30ba656fb5328/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/TRJMER24OEVUG6GOY77ZIVLA57:/var/lib/containers/storage/overlay/l/OUQYFINT4JZQ6O5ND75IXNTGSJ,upperdir=/var/lib/containers/storage/overlay/00e2e5150eb6e25db344ee6a8c032b156500eb01d194d65a47c30ba656fb5328/diff,workdir=/var/lib/containers/storage/overlay/00e2e5150eb6e25db344ee6a8c032b156500eb01d194d65a47c30ba656fb5328/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ee65ab3b1ea21a5b86cbd1f9ade498928da7c0ee71a7f7ba9d13f95142e95646/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/TRJMER24OEVUG6GOY77ZIVLA57:/var/lib/containers/storage/overlay/l/OUQYFINT4JZQ6O5ND75IXNTGSJ,upperdir=/var/lib/containers/storage/overlay/ee65ab3b1ea21a5b86cbd1f9ade498928da7c0ee71a7f7ba9d13f95142e95646/diff,workdir=/var/lib/containers/storage/overlay/ee65ab3b1ea21a5b86cbd1f9ade498928da7c0ee71a7f7ba9d13f95142e95646/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498872k,nr_inodes=819200,mode=755,inode64 0 0 2024-06-10T01:11:36.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:11:36.465 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-06-10T01:11:36.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-10T01:11:36.521 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-06-10T01:11:36.614 INFO:teuthology.orchestra.run.smithi037.stderr:Cannot find device "ceph-brx" 2024-06-10T01:11:36.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T01:11:36.615 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-06-10T01:11:36.627 INFO:tasks.cephadm:Teardown begin 2024-06-10T01:11:36.627 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-06-10T01:11:36.627 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:11:36.878 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:11:38.376 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:38 smithi037 ceph-mon[32192]: pgmap v883: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 5 op/s 2024-06-10T01:11:38.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:38 smithi167 ceph-mon[36917]: pgmap v883: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 5 op/s 2024-06-10T01:11:38.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:38 smithi094 ceph-mon[36708]: pgmap v883: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 5 op/s 2024-06-10T01:11:38.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.668+0000 7fed3a539700 1 -- 172.21.15.37:0/2367062362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed34102550 msgr2=0x7fed341029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:38.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.668+0000 7fed3a539700 1 --2- 172.21.15.37:0/2367062362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed34102550 0x7fed341029d0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fed28009990 tx=0x7fed28009ca0 comp rx=0 tx=0).stop 2024-06-10T01:11:38.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.668+0000 7fed3a539700 1 -- 172.21.15.37:0/2367062362 shutdown_connections 2024-06-10T01:11:38.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.668+0000 7fed3a539700 1 --2- 172.21.15.37:0/2367062362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed34102f10 0x7fed3410f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:38.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.668+0000 7fed3a539700 1 --2- 172.21.15.37:0/2367062362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed341084f0 0x7fed341088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:38.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.668+0000 7fed3a539700 1 --2- 172.21.15.37:0/2367062362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed34102550 0x7fed341029d0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:38.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.668+0000 7fed3a539700 1 -- 172.21.15.37:0/2367062362 >> 172.21.15.37:0/2367062362 conn(0x7fed34079680 msgr2=0x7fed34079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:38.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.668+0000 7fed3a539700 1 -- 172.21.15.37:0/2367062362 shutdown_connections 2024-06-10T01:11:38.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.668+0000 7fed3a539700 1 -- 172.21.15.37:0/2367062362 wait complete. 2024-06-10T01:11:38.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.669+0000 7fed3a539700 1 Processor -- start 2024-06-10T01:11:38.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.669+0000 7fed3a539700 1 -- start start 2024-06-10T01:11:38.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.670+0000 7fed3a539700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed34102550 0x7fed34074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:38.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.670+0000 7fed3a539700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed34102f10 0x7fed34074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:38.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.670+0000 7fed3a539700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed341084f0 0x7fed34075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:38.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.670+0000 7fed3a539700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fed341127d0 con 0x7fed34102f10 2024-06-10T01:11:38.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.670+0000 7fed3a539700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fed34112650 con 0x7fed341084f0 2024-06-10T01:11:38.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.670+0000 7fed3a539700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fed34112950 con 0x7fed34102550 2024-06-10T01:11:38.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.670+0000 7fed337fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed34102f10 0x7fed34074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:38.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.670+0000 7fed33fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed34102550 0x7fed34074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:38.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.670+0000 7fed337fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed34102f10 0x7fed34074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42846/0 (socket says 172.21.15.37:42846) 2024-06-10T01:11:38.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.670+0000 7fed337fe700 1 -- 172.21.15.37:0/297841285 learned_addr learned my addr 172.21.15.37:0/297841285 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:11:38.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.670+0000 7fed38ad6700 1 --2- 172.21.15.37:0/297841285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed341084f0 0x7fed34075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:38.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.670+0000 7fed337fe700 1 -- 172.21.15.37:0/297841285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed34102550 msgr2=0x7fed34074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.670+0000 7fed337fe700 1 --2- 172.21.15.37:0/297841285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed34102550 0x7fed34074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.670+0000 7fed337fe700 1 -- 172.21.15.37:0/297841285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed341084f0 msgr2=0x7fed34075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.670+0000 7fed337fe700 1 --2- 172.21.15.37:0/297841285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed341084f0 0x7fed34075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.670+0000 7fed337fe700 1 -- 172.21.15.37:0/297841285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed341a7230 con 0x7fed34102f10 2024-06-10T01:11:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.670+0000 7fed38ad6700 1 --2- 172.21.15.37:0/297841285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed341084f0 0x7fed34075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:11:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.670+0000 7fed337fe700 1 --2- 172.21.15.37:0/297841285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed34102f10 0x7fed34074cd0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fed28000c00 tx=0x7fed280047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.671+0000 7fed317fa700 1 -- 172.21.15.37:0/297841285 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28027070 con 0x7fed34102f10 2024-06-10T01:11:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.671+0000 7fed317fa700 1 -- 172.21.15.37:0/297841285 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed28003780 con 0x7fed34102f10 2024-06-10T01:11:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.671+0000 7fed3a539700 1 -- 172.21.15.37:0/297841285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed341a74c0 con 0x7fed34102f10 2024-06-10T01:11:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.671+0000 7fed317fa700 1 -- 172.21.15.37:0/297841285 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28022dd0 con 0x7fed34102f10 2024-06-10T01:11:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.671+0000 7fed3a539700 1 -- 172.21.15.37:0/297841285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed341a7970 con 0x7fed34102f10 2024-06-10T01:11:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.672+0000 7fed3a539700 1 -- 172.21.15.37:0/297841285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed3410a9d0 con 0x7fed34102f10 2024-06-10T01:11:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.673+0000 7fed317fa700 1 -- 172.21.15.37:0/297841285 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fed2802a070 con 0x7fed34102f10 2024-06-10T01:11:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.674+0000 7fed317fa700 1 --2- 172.21.15.37:0/297841285 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fed1c0a2ca0 0x7fed1c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.674+0000 7fed33fff700 1 --2- 172.21.15.37:0/297841285 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fed1c0a2ca0 0x7fed1c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.674+0000 7fed317fa700 1 -- 172.21.15.37:0/297841285 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7fed280d0480 con 0x7fed34102f10 2024-06-10T01:11:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.674+0000 7fed33fff700 1 --2- 172.21.15.37:0/297841285 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fed1c0a2ca0 0x7fed1c0a5160 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fed24009a10 tx=0x7fed24009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:38.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.677+0000 7fed317fa700 1 -- 172.21.15.37:0/297841285 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fed28013070 con 0x7fed34102f10 2024-06-10T01:11:38.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.928+0000 7fed3a539700 1 -- 172.21.15.37:0/297841285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed34069180 con 0x7fed34102f10 2024-06-10T01:11:38.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.929+0000 7fed317fa700 1 -- 172.21.15.37:0/297841285 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11870 (secure 0 0 0) 0x7fed2809d0b0 con 0x7fed34102f10 2024-06-10T01:11:38.926 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:11:38.926 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":26554,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2107277223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2107277223},{"type":"v1","addr":"172.21.15.167:6835","nonce":2107277223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T01:10:10.062065+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":271,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24458,24472,24451]},"id":1}]} 2024-06-10T01:11:38.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.931+0000 7fed3a539700 1 -- 172.21.15.37:0/297841285 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fed1c0a2ca0 msgr2=0x7fed1c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:38.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.931+0000 7fed3a539700 1 --2- 172.21.15.37:0/297841285 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fed1c0a2ca0 0x7fed1c0a5160 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fed24009a10 tx=0x7fed24009190 comp rx=0 tx=0).stop 2024-06-10T01:11:38.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.931+0000 7fed3a539700 1 -- 172.21.15.37:0/297841285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed34102f10 msgr2=0x7fed34074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:38.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.931+0000 7fed3a539700 1 --2- 172.21.15.37:0/297841285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed34102f10 0x7fed34074cd0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fed28000c00 tx=0x7fed280047b0 comp rx=0 tx=0).stop 2024-06-10T01:11:38.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.931+0000 7fed3a539700 1 -- 172.21.15.37:0/297841285 shutdown_connections 2024-06-10T01:11:38.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.931+0000 7fed3a539700 1 --2- 172.21.15.37:0/297841285 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fed1c0a2ca0 0x7fed1c0a5160 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:38.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.931+0000 7fed3a539700 1 --2- 172.21.15.37:0/297841285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed341084f0 0x7fed34075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:38.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.931+0000 7fed3a539700 1 --2- 172.21.15.37:0/297841285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed34102550 0x7fed34074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:38.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.931+0000 7fed3a539700 1 --2- 172.21.15.37:0/297841285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed34102f10 0x7fed34074cd0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:38.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.931+0000 7fed3a539700 1 -- 172.21.15.37:0/297841285 >> 172.21.15.37:0/297841285 conn(0x7fed34079680 msgr2=0x7fed341012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:38.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.931+0000 7fed3a539700 1 -- 172.21.15.37:0/297841285 shutdown_connections 2024-06-10T01:11:38.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:38.931+0000 7fed3a539700 1 -- 172.21.15.37:0/297841285 wait complete. 2024-06-10T01:11:38.929 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 35 2024-06-10T01:11:39.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:39 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/297841285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:11:39.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:39 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/297841285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:11:39.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:39 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/297841285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:11:39.687 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-10T01:11:39.942 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:11:40.318 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:40 smithi037 ceph-mon[32192]: pgmap v884: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-10T01:11:40.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:40 smithi167 ceph-mon[36917]: pgmap v884: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-10T01:11:40.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:40 smithi094 ceph-mon[36708]: pgmap v884: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-10T01:11:41.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.710+0000 7f71c0b78700 1 -- 172.21.15.37:0/3671809948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71bc101610 msgr2=0x7f71bc1019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:41.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.710+0000 7f71c0b78700 1 --2- 172.21.15.37:0/3671809948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71bc101610 0x7f71bc1019f0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0099c0 tx=0x7f71ac009cd0 comp rx=0 tx=0).stop 2024-06-10T01:11:41.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.710+0000 7f71c0b78700 1 -- 172.21.15.37:0/3671809948 shutdown_connections 2024-06-10T01:11:41.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.710+0000 7f71c0b78700 1 --2- 172.21.15.37:0/3671809948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71bc1066f0 0x7f71bc10e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:41.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.710+0000 7f71c0b78700 1 --2- 172.21.15.37:0/3671809948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71bc101610 0x7f71bc1019f0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:41.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.710+0000 7f71c0b78700 1 --2- 172.21.15.37:0/3671809948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71bc101fc0 0x7f71bc106010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:41.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.710+0000 7f71c0b78700 1 -- 172.21.15.37:0/3671809948 >> 172.21.15.37:0/3671809948 conn(0x7f71bc078430 msgr2=0x7f71bc078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:41.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.710+0000 7f71c0b78700 1 -- 172.21.15.37:0/3671809948 shutdown_connections 2024-06-10T01:11:41.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.710+0000 7f71c0b78700 1 -- 172.21.15.37:0/3671809948 wait complete. 2024-06-10T01:11:41.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.711+0000 7f71c0b78700 1 Processor -- start 2024-06-10T01:11:41.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.711+0000 7f71c0b78700 1 -- start start 2024-06-10T01:11:41.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.711+0000 7f71c0b78700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71bc101610 0x7f71bc19d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:41.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.711+0000 7f71c0b78700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71bc101fc0 0x7f71bc19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:41.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.711+0000 7f71c0b78700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71bc1066f0 0x7f71bc197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:41.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.711+0000 7f71c0b78700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc1117d0 con 0x7f71bc101fc0 2024-06-10T01:11:41.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.711+0000 7f71c0b78700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc111650 con 0x7f71bc1066f0 2024-06-10T01:11:41.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.711+0000 7f71c0b78700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc111950 con 0x7f71bc101610 2024-06-10T01:11:41.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.711+0000 7f71b9d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71bc101fc0 0x7f71bc19de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:41.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.711+0000 7f71b9d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71bc101fc0 0x7f71bc19de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42864/0 (socket says 172.21.15.37:42864) 2024-06-10T01:11:41.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.711+0000 7f71b9d9b700 1 -- 172.21.15.37:0/3763734878 learned_addr learned my addr 172.21.15.37:0/3763734878 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:11:41.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.711+0000 7f71bad9d700 1 --2- 172.21.15.37:0/3763734878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71bc1066f0 0x7f71bc197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:41.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.711+0000 7f71ba59c700 1 --2- 172.21.15.37:0/3763734878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71bc101610 0x7f71bc19d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:41.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.712+0000 7f71b9d9b700 1 -- 172.21.15.37:0/3763734878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71bc101610 msgr2=0x7f71bc19d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:41.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.712+0000 7f71b9d9b700 1 --2- 172.21.15.37:0/3763734878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71bc101610 0x7f71bc19d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:41.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.712+0000 7f71b9d9b700 1 -- 172.21.15.37:0/3763734878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71bc1066f0 msgr2=0x7f71bc197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:41.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.712+0000 7f71b9d9b700 1 --2- 172.21.15.37:0/3763734878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71bc1066f0 0x7f71bc197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:41.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.712+0000 7f71b9d9b700 1 -- 172.21.15.37:0/3763734878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71bc198250 con 0x7f71bc101fc0 2024-06-10T01:11:41.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.712+0000 7f71b9d9b700 1 --2- 172.21.15.37:0/3763734878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71bc101fc0 0x7f71bc19de70 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f71b000b500 tx=0x7f71b000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:41.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.712+0000 7f71ab7fe700 1 -- 172.21.15.37:0/3763734878 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0015020 con 0x7f71bc101fc0 2024-06-10T01:11:41.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.712+0000 7f71ab7fe700 1 -- 172.21.15.37:0/3763734878 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71b0011d10 con 0x7f71bc101fc0 2024-06-10T01:11:41.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.712+0000 7f71c0b78700 1 -- 172.21.15.37:0/3763734878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71bc198540 con 0x7f71bc101fc0 2024-06-10T01:11:41.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.712+0000 7f71ab7fe700 1 -- 172.21.15.37:0/3763734878 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0010de0 con 0x7f71bc101fc0 2024-06-10T01:11:41.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.712+0000 7f71c0b78700 1 -- 172.21.15.37:0/3763734878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71bc1a4800 con 0x7f71bc101fc0 2024-06-10T01:11:41.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.713+0000 7f71c0b78700 1 -- 172.21.15.37:0/3763734878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71bc069180 con 0x7f71bc101fc0 2024-06-10T01:11:41.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.714+0000 7f71ab7fe700 1 -- 172.21.15.37:0/3763734878 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f71b0018020 con 0x7f71bc101fc0 2024-06-10T01:11:41.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.715+0000 7f71ab7fe700 1 --2- 172.21.15.37:0/3763734878 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f71a40a2ca0 0x7f71a40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:41.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.715+0000 7f71ab7fe700 1 -- 172.21.15.37:0/3763734878 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7f71b00cefe0 con 0x7f71bc101fc0 2024-06-10T01:11:41.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.715+0000 7f71ba59c700 1 --2- 172.21.15.37:0/3763734878 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f71a40a2ca0 0x7f71a40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:41.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.715+0000 7f71ba59c700 1 --2- 172.21.15.37:0/3763734878 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f71a40a2ca0 0x7f71a40a5160 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f71ac000c00 tx=0x7f71ac005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:41.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.718+0000 7f71ab7fe700 1 -- 172.21.15.37:0/3763734878 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f71b001f020 con 0x7f71bc101fc0 2024-06-10T01:11:41.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.970+0000 7f71c0b78700 1 -- 172.21.15.37:0/3763734878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71bc02dca0 con 0x7f71bc101fc0 2024-06-10T01:11:41.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.971+0000 7f71ab7fe700 1 -- 172.21.15.37:0/3763734878 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11870 (secure 0 0 0) 0x7f71b009baf0 con 0x7f71bc101fc0 2024-06-10T01:11:41.967 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:11:41.967 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1021220889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1021220889},{"type":"v1","addr":"172.21.15.37:6835","nonce":1021220889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/4093234668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":4093234668},{"type":"v1","addr":"172.21.15.37:6837","nonce":4093234668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/595511083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":595511083},{"type":"v1","addr":"172.21.15.37:6839","nonce":595511083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2289348383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2289348383},{"type":"v1","addr":"172.21.15.37:6841","nonce":2289348383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2915505496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2915505496},{"type":"v1","addr":"172.21.15.94:6833","nonce":2915505496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":26554,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2107277223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2107277223},{"type":"v1","addr":"172.21.15.167:6835","nonce":2107277223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-10T00:52:37.869277+0000","modified":"2024-06-10T01:10:10.062065+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24472,"mds_1":24458,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/2828145715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2828145715},{"type":"v1","addr":"172.21.15.167:6833","nonce":2828145715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/570288048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":570288048},{"type":"v1","addr":"172.21.15.94:6835","nonce":570288048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6837/3801637791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3801637791},{"type":"v1","addr":"172.21.15.94:6837","nonce":3801637791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/2720148065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2720148065},{"type":"v1","addr":"172.21.15.167:6837","nonce":2720148065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6839/946269331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":946269331},{"type":"v1","addr":"172.21.15.94:6839","nonce":946269331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":271,"addr":"172.21.15.167:6839/3945638144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":3945638144},{"type":"v1","addr":"172.21.15.167:6839","nonce":3945638144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24472,"qdb_cluster":[24458,24472,24451]},"id":1}]} 2024-06-10T01:11:41.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.973+0000 7f71c0b78700 1 -- 172.21.15.37:0/3763734878 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f71a40a2ca0 msgr2=0x7f71a40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:41.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.973+0000 7f71c0b78700 1 --2- 172.21.15.37:0/3763734878 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f71a40a2ca0 0x7f71a40a5160 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f71ac000c00 tx=0x7f71ac005fb0 comp rx=0 tx=0).stop 2024-06-10T01:11:41.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.973+0000 7f71c0b78700 1 -- 172.21.15.37:0/3763734878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71bc101fc0 msgr2=0x7f71bc19de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:41.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.973+0000 7f71c0b78700 1 --2- 172.21.15.37:0/3763734878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71bc101fc0 0x7f71bc19de70 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f71b000b500 tx=0x7f71b000b810 comp rx=0 tx=0).stop 2024-06-10T01:11:41.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.973+0000 7f71c0b78700 1 -- 172.21.15.37:0/3763734878 shutdown_connections 2024-06-10T01:11:41.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.973+0000 7f71c0b78700 1 --2- 172.21.15.37:0/3763734878 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f71a40a2ca0 0x7f71a40a5160 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:41.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.973+0000 7f71c0b78700 1 --2- 172.21.15.37:0/3763734878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71bc1066f0 0x7f71bc197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:41.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.973+0000 7f71c0b78700 1 --2- 172.21.15.37:0/3763734878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71bc101610 0x7f71bc19d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:41.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.973+0000 7f71c0b78700 1 --2- 172.21.15.37:0/3763734878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71bc101fc0 0x7f71bc19de70 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:41.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.973+0000 7f71c0b78700 1 -- 172.21.15.37:0/3763734878 >> 172.21.15.37:0/3763734878 conn(0x7f71bc078430 msgr2=0x7f71bc1052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:41.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.973+0000 7f71c0b78700 1 -- 172.21.15.37:0/3763734878 shutdown_connections 2024-06-10T01:11:41.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:41.973+0000 7f71c0b78700 1 -- 172.21.15.37:0/3763734878 wait complete. 2024-06-10T01:11:41.970 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 35 2024-06-10T01:11:42.333 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:42 smithi037 ceph-mon[32192]: pgmap v885: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-06-10T01:11:42.333 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:42 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3763734878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:11:42.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:42 smithi167 ceph-mon[36917]: pgmap v885: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-06-10T01:11:42.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:42 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3763734878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:11:42.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:42 smithi094 ceph-mon[36708]: pgmap v885: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-06-10T01:11:42.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:42 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3763734878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-10T01:11:42.640 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-10T01:11:42.893 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:11:44.342 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:44 smithi037 ceph-mon[32192]: pgmap v886: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-06-10T01:11:44.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:44 smithi167 ceph-mon[36917]: pgmap v886: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-06-10T01:11:44.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.599+0000 7f8d596ce700 1 -- 172.21.15.37:0/627888547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d5406ac00 msgr2=0x7f8d5406b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:44.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.599+0000 7f8d596ce700 1 --2- 172.21.15.37:0/627888547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d5406ac00 0x7f8d5406b080 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009990 tx=0x7f8d48009ca0 comp rx=0 tx=0).stop 2024-06-10T01:11:44.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.599+0000 7f8d596ce700 1 -- 172.21.15.37:0/627888547 shutdown_connections 2024-06-10T01:11:44.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.599+0000 7f8d596ce700 1 --2- 172.21.15.37:0/627888547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d5406b5c0 0x7f8d5410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:44.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.599+0000 7f8d596ce700 1 --2- 172.21.15.37:0/627888547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d54108520 0x7f8d54108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:44.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.599+0000 7f8d596ce700 1 --2- 172.21.15.37:0/627888547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d5406ac00 0x7f8d5406b080 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:44.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.599+0000 7f8d596ce700 1 -- 172.21.15.37:0/627888547 >> 172.21.15.37:0/627888547 conn(0x7f8d540773b0 msgr2=0x7f8d540777c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:44.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.599+0000 7f8d596ce700 1 -- 172.21.15.37:0/627888547 shutdown_connections 2024-06-10T01:11:44.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.599+0000 7f8d596ce700 1 -- 172.21.15.37:0/627888547 wait complete. 2024-06-10T01:11:44.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.600+0000 7f8d596ce700 1 Processor -- start 2024-06-10T01:11:44.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.600+0000 7f8d596ce700 1 -- start start 2024-06-10T01:11:44.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.601+0000 7f8d596ce700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d5406ac00 0x7f8d5419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:44.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.601+0000 7f8d596ce700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d5406b5c0 0x7f8d5419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:44.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.601+0000 7f8d596ce700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d54108520 0x7f8d54198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:44.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.601+0000 7f8d596ce700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8d54112770 con 0x7f8d5406ac00 2024-06-10T01:11:44.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.601+0000 7f8d596ce700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8d541125f0 con 0x7f8d5406b5c0 2024-06-10T01:11:44.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.601+0000 7f8d596ce700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8d541128f0 con 0x7f8d54108520 2024-06-10T01:11:44.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.601+0000 7f8d52ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d5406ac00 0x7f8d5419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:44.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.601+0000 7f8d52ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d5406ac00 0x7f8d5419e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59704/0 (socket says 172.21.15.37:59704) 2024-06-10T01:11:44.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.601+0000 7f8d52ffd700 1 -- 172.21.15.37:0/4131962090 learned_addr learned my addr 172.21.15.37:0/4131962090 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:11:44.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.601+0000 7f8d527fc700 1 --2- 172.21.15.37:0/4131962090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d5406b5c0 0x7f8d5419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:44.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.601+0000 7f8d537fe700 1 --2- 172.21.15.37:0/4131962090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d54108520 0x7f8d54198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:44.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.601+0000 7f8d52ffd700 1 -- 172.21.15.37:0/4131962090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d54108520 msgr2=0x7f8d54198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:44.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.601+0000 7f8d52ffd700 1 --2- 172.21.15.37:0/4131962090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d54108520 0x7f8d54198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:44.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.601+0000 7f8d52ffd700 1 -- 172.21.15.37:0/4131962090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d5406b5c0 msgr2=0x7f8d5419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:44.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.601+0000 7f8d52ffd700 1 --2- 172.21.15.37:0/4131962090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d5406b5c0 0x7f8d5419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:44.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.601+0000 7f8d52ffd700 1 -- 172.21.15.37:0/4131962090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d541991f0 con 0x7f8d5406ac00 2024-06-10T01:11:44.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.601+0000 7f8d52ffd700 1 --2- 172.21.15.37:0/4131962090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d5406ac00 0x7f8d5419e880 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f8d440060b0 tx=0x7f8d4400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:44.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.602+0000 7f8d3bfff700 1 -- 172.21.15.37:0/4131962090 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4401a070 con 0x7f8d5406ac00 2024-06-10T01:11:44.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.602+0000 7f8d596ce700 1 -- 172.21.15.37:0/4131962090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d541994e0 con 0x7f8d5406ac00 2024-06-10T01:11:44.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.602+0000 7f8d3bfff700 1 -- 172.21.15.37:0/4131962090 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d4400ddd0 con 0x7f8d5406ac00 2024-06-10T01:11:44.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.602+0000 7f8d3bfff700 1 -- 172.21.15.37:0/4131962090 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d440147a0 con 0x7f8d5406ac00 2024-06-10T01:11:44.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.602+0000 7f8d596ce700 1 -- 172.21.15.37:0/4131962090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d541a5750 con 0x7f8d5406ac00 2024-06-10T01:11:44.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.603+0000 7f8d596ce700 1 -- 172.21.15.37:0/4131962090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d54103bc0 con 0x7f8d5406ac00 2024-06-10T01:11:44.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.604+0000 7f8d3bfff700 1 -- 172.21.15.37:0/4131962090 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8d4401d070 con 0x7f8d5406ac00 2024-06-10T01:11:44.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.605+0000 7f8d3bfff700 1 --2- 172.21.15.37:0/4131962090 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8d3c0a2c20 0x7f8d3c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:44.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.605+0000 7f8d527fc700 1 --2- 172.21.15.37:0/4131962090 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8d3c0a2c20 0x7f8d3c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:44.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.605+0000 7f8d3bfff700 1 -- 172.21.15.37:0/4131962090 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7f8d440cf110 con 0x7f8d5406ac00 2024-06-10T01:11:44.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.605+0000 7f8d527fc700 1 --2- 172.21.15.37:0/4131962090 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8d3c0a2c20 0x7f8d3c0a50e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800b5c0 tx=0x7f8d48005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:44.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.609+0000 7f8d3bfff700 1 -- 172.21.15.37:0/4131962090 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8d44010070 con 0x7f8d5406ac00 2024-06-10T01:11:44.608 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:44 smithi094 ceph-mon[36708]: pgmap v886: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-06-10T01:11:44.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.817+0000 7f8d596ce700 1 -- 172.21.15.37:0/4131962090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8d54199f70 con 0x7f8d5406ac00 2024-06-10T01:11:44.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.819+0000 7f8d3bfff700 1 -- 172.21.15.37:0/4131962090 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+24861 (secure 0 0 0) 0x7f8d4409bc80 con 0x7f8d5406ac00 2024-06-10T01:11:44.815 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-06-10T01:11:44.815 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":105,"fsid":"68793514-26c2-11ef-bc9b-c7b262605968","created":"2024-06-10T00:44:24.106713+0000","modified":"2024-06-10T01:10:02.589262+0000","last_up_change":"2024-06-10T00:51:31.562156+0000","last_in_change":"2024-06-10T00:51:11.294301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-10T00:47:44.773911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-10T00:52:31.684780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-10T00:52:34.885503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-10T00:52:44.415437+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1df6046a-6801-4950-9193-54303fed169b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6803","nonce":3594704094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6805","nonce":3594704094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6809","nonce":3594704094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3594704094},{"type":"v1","addr":"172.21.15.37:6807","nonce":3594704094}]},"public_addr":"172.21.15.37:6803/3594704094","cluster_addr":"172.21.15.37:6805/3594704094","heartbeat_back_addr":"172.21.15.37:6809/3594704094","heartbeat_front_addr":"172.21.15.37:6807/3594704094","state":["exists","up"]},{"osd":1,"uuid":"4e1c8944-94e4-4e46-9efe-56229fe6df1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6801","nonce":3427727155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6803","nonce":3427727155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6807","nonce":3427727155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3427727155},{"type":"v1","addr":"172.21.15.94:6805","nonce":3427727155}]},"public_addr":"172.21.15.94:6801/3427727155","cluster_addr":"172.21.15.94:6803/3427727155","heartbeat_back_addr":"172.21.15.94:6807/3427727155","heartbeat_front_addr":"172.21.15.94:6805/3427727155","state":["exists","up"]},{"osd":2,"uuid":"fd2fcedc-4c88-478f-8025-13e5944a27ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6801","nonce":4002498292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6803","nonce":4002498292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6807","nonce":4002498292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":4002498292},{"type":"v1","addr":"172.21.15.167:6805","nonce":4002498292}]},"public_addr":"172.21.15.167:6801/4002498292","cluster_addr":"172.21.15.167:6803/4002498292","heartbeat_back_addr":"172.21.15.167:6807/4002498292","heartbeat_front_addr":"172.21.15.167:6805/4002498292","state":["exists","up"]},{"osd":3,"uuid":"d0153676-818c-4dcc-83b1-287435a0c8a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6811","nonce":3425469324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6813","nonce":3425469324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6817","nonce":3425469324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3425469324},{"type":"v1","addr":"172.21.15.37:6815","nonce":3425469324}]},"public_addr":"172.21.15.37:6811/3425469324","cluster_addr":"172.21.15.37:6813/3425469324","heartbeat_back_addr":"172.21.15.37:6817/3425469324","heartbeat_front_addr":"172.21.15.37:6815/3425469324","state":["exists","up"]},{"osd":4,"uuid":"172c333c-00c0-4e94-8870-acb3b32b2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6809","nonce":1740627431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6811","nonce":1740627431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6815","nonce":1740627431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1740627431},{"type":"v1","addr":"172.21.15.94:6813","nonce":1740627431}]},"public_addr":"172.21.15.94:6809/1740627431","cluster_addr":"172.21.15.94:6811/1740627431","heartbeat_back_addr":"172.21.15.94:6815/1740627431","heartbeat_front_addr":"172.21.15.94:6813/1740627431","state":["exists","up"]},{"osd":5,"uuid":"dbce656e-5f93-4bda-baf2-fc3cc03d8282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6809","nonce":221805236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6811","nonce":221805236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6815","nonce":221805236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":221805236},{"type":"v1","addr":"172.21.15.167:6813","nonce":221805236}]},"public_addr":"172.21.15.167:6809/221805236","cluster_addr":"172.21.15.167:6811/221805236","heartbeat_back_addr":"172.21.15.167:6815/221805236","heartbeat_front_addr":"172.21.15.167:6813/221805236","state":["exists","up"]},{"osd":6,"uuid":"fcbda266-6d60-43b2-a51c-4e562476d1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6819","nonce":472670875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6821","nonce":472670875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6825","nonce":472670875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":472670875},{"type":"v1","addr":"172.21.15.37:6823","nonce":472670875}]},"public_addr":"172.21.15.37:6819/472670875","cluster_addr":"172.21.15.37:6821/472670875","heartbeat_back_addr":"172.21.15.37:6825/472670875","heartbeat_front_addr":"172.21.15.37:6823/472670875","state":["exists","up"]},{"osd":7,"uuid":"ed829ed3-ba85-4e82-ac37-e44e9534d37c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6817","nonce":3932281552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6819","nonce":3932281552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6823","nonce":3932281552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3932281552},{"type":"v1","addr":"172.21.15.94:6821","nonce":3932281552}]},"public_addr":"172.21.15.94:6817/3932281552","cluster_addr":"172.21.15.94:6819/3932281552","heartbeat_back_addr":"172.21.15.94:6823/3932281552","heartbeat_front_addr":"172.21.15.94:6821/3932281552","state":["exists","up"]},{"osd":8,"uuid":"7d6267d8-72f4-4704-b554-bafc52d11a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6817","nonce":2290216047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6819","nonce":2290216047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6823","nonce":2290216047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2290216047},{"type":"v1","addr":"172.21.15.167:6821","nonce":2290216047}]},"public_addr":"172.21.15.167:6817/2290216047","cluster_addr":"172.21.15.167:6819/2290216047","heartbeat_back_addr":"172.21.15.167:6823/2290216047","heartbeat_front_addr":"172.21.15.167:6821/2290216047","state":["exists","up"]},{"osd":9,"uuid":"210fdc20-0a36-4eeb-bdf2-c0a7f130e19d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6827","nonce":2063666428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6829","nonce":2063666428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6833","nonce":2063666428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2063666428},{"type":"v1","addr":"172.21.15.37:6831","nonce":2063666428}]},"public_addr":"172.21.15.37:6827/2063666428","cluster_addr":"172.21.15.37:6829/2063666428","heartbeat_back_addr":"172.21.15.37:6833/2063666428","heartbeat_front_addr":"172.21.15.37:6831/2063666428","state":["exists","up"]},{"osd":10,"uuid":"e200b371-b3cf-4b92-a659-f155ca86f0a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6825","nonce":1706618899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6827","nonce":1706618899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6831","nonce":1706618899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1706618899},{"type":"v1","addr":"172.21.15.94:6829","nonce":1706618899}]},"public_addr":"172.21.15.94:6825/1706618899","cluster_addr":"172.21.15.94:6827/1706618899","heartbeat_back_addr":"172.21.15.94:6831/1706618899","heartbeat_front_addr":"172.21.15.94:6829/1706618899","state":["exists","up"]},{"osd":11,"uuid":"cfa427df-37a5-4d28-ac2e-af777d607008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6825","nonce":1349892516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6827","nonce":1349892516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6831","nonce":1349892516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1349892516},{"type":"v1","addr":"172.21.15.167:6829","nonce":1349892516}]},"public_addr":"172.21.15.167:6825/1349892516","cluster_addr":"172.21.15.167:6827/1349892516","heartbeat_back_addr":"172.21.15.167:6831/1349892516","heartbeat_front_addr":"172.21.15.167:6829/1349892516","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:46:51.846147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:15.763723+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:47:40.106800+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:05.923204+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:30.501883+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:48:55.565575+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:20.411360+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:49:46.071776+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:11.854599+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:50:37.608818+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:03.176767+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-10T00:51:29.768705+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/4084722397":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/3571803585":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/416587443":"2024-06-11T00:45:20.254342+0000","172.21.15.37:6801/2766692263":"2024-06-11T00:45:20.254342+0000","172.21.15.37:0/2026344134":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/2781321886":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6801/2803174880":"2024-06-11T00:44:53.913969+0000","172.21.15.37:0/1110143997":"2024-06-11T00:44:53.913969+0000","172.21.15.37:6800/2766692263":"2024-06-11T00:45:20.254342+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-10T01:11:44.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.822+0000 7f8d596ce700 1 -- 172.21.15.37:0/4131962090 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8d3c0a2c20 msgr2=0x7f8d3c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:44.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.822+0000 7f8d596ce700 1 --2- 172.21.15.37:0/4131962090 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8d3c0a2c20 0x7f8d3c0a50e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800b5c0 tx=0x7f8d48005960 comp rx=0 tx=0).stop 2024-06-10T01:11:44.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.822+0000 7f8d596ce700 1 -- 172.21.15.37:0/4131962090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d5406ac00 msgr2=0x7f8d5419e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:44.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.822+0000 7f8d596ce700 1 --2- 172.21.15.37:0/4131962090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d5406ac00 0x7f8d5419e880 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f8d440060b0 tx=0x7f8d4400d3b0 comp rx=0 tx=0).stop 2024-06-10T01:11:44.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.822+0000 7f8d596ce700 1 -- 172.21.15.37:0/4131962090 shutdown_connections 2024-06-10T01:11:44.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.822+0000 7f8d596ce700 1 --2- 172.21.15.37:0/4131962090 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f8d3c0a2c20 0x7f8d3c0a50e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:44.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.822+0000 7f8d596ce700 1 --2- 172.21.15.37:0/4131962090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d54108520 0x7f8d54198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:44.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.822+0000 7f8d596ce700 1 --2- 172.21.15.37:0/4131962090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d5406ac00 0x7f8d5419e880 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:44.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.822+0000 7f8d596ce700 1 --2- 172.21.15.37:0/4131962090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d5406b5c0 0x7f8d5419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:44.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.822+0000 7f8d596ce700 1 -- 172.21.15.37:0/4131962090 >> 172.21.15.37:0/4131962090 conn(0x7f8d540773b0 msgr2=0x7f8d54100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:44.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.822+0000 7f8d596ce700 1 -- 172.21.15.37:0/4131962090 shutdown_connections 2024-06-10T01:11:44.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:44.822+0000 7f8d596ce700 1 -- 172.21.15.37:0/4131962090 wait complete. 2024-06-10T01:11:45.470 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:45 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/4131962090' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:11:45.501 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs fail cephfs 2024-06-10T01:11:45.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:45 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/4131962090' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:11:45.608 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:45 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/4131962090' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-10T01:11:45.756 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:11:46.390 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:46 smithi037 ceph-mon[32192]: pgmap v887: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-06-10T01:11:46.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:46 smithi167 ceph-mon[36917]: pgmap v887: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-06-10T01:11:46.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:46 smithi094 ceph-mon[36708]: pgmap v887: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-06-10T01:11:47.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.489+0000 7f9b179c3700 1 -- 172.21.15.37:0/1796980159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b10101620 msgr2=0x7f9b10101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:47.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.489+0000 7f9b179c3700 1 --2- 172.21.15.37:0/1796980159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b10101620 0x7f9b10101a00 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c0099c0 tx=0x7f9b0c009cd0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.489+0000 7f9b179c3700 1 -- 172.21.15.37:0/1796980159 shutdown_connections 2024-06-10T01:11:47.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.489+0000 7f9b179c3700 1 --2- 172.21.15.37:0/1796980159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b101066b0 0x7f9b1010e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.489+0000 7f9b179c3700 1 --2- 172.21.15.37:0/1796980159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b10101620 0x7f9b10101a00 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.489+0000 7f9b179c3700 1 --2- 172.21.15.37:0/1796980159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b10101fd0 0x7f9b10105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.489+0000 7f9b179c3700 1 -- 172.21.15.37:0/1796980159 >> 172.21.15.37:0/1796980159 conn(0x7f9b10078340 msgr2=0x7f9b10078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:47.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.489+0000 7f9b179c3700 1 -- 172.21.15.37:0/1796980159 shutdown_connections 2024-06-10T01:11:47.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.489+0000 7f9b179c3700 1 -- 172.21.15.37:0/1796980159 wait complete. 2024-06-10T01:11:47.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.490+0000 7f9b179c3700 1 Processor -- start 2024-06-10T01:11:47.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.490+0000 7f9b179c3700 1 -- start start 2024-06-10T01:11:47.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.490+0000 7f9b179c3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b10101620 0x7f9b1019d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:47.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.490+0000 7f9b179c3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b10101fd0 0x7f9b1019dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:47.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.490+0000 7f9b179c3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b101066b0 0x7f9b101979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:47.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.490+0000 7f9b179c3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9b10111710 con 0x7f9b10101fd0 2024-06-10T01:11:47.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.490+0000 7f9b179c3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9b10111590 con 0x7f9b101066b0 2024-06-10T01:11:47.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.490+0000 7f9b179c3700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9b10111890 con 0x7f9b10101620 2024-06-10T01:11:47.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.490+0000 7f9b14f5e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b10101fd0 0x7f9b1019dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:47.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.490+0000 7f9b14f5e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b10101fd0 0x7f9b1019dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59724/0 (socket says 172.21.15.37:59724) 2024-06-10T01:11:47.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.490+0000 7f9b14f5e700 1 -- 172.21.15.37:0/3142249219 learned_addr learned my addr 172.21.15.37:0/3142249219 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:11:47.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.490+0000 7f9b1575f700 1 --2- 172.21.15.37:0/3142249219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b10101620 0x7f9b1019d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:47.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.490+0000 7f9b15f60700 1 --2- 172.21.15.37:0/3142249219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b101066b0 0x7f9b101979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:47.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.490+0000 7f9b14f5e700 1 -- 172.21.15.37:0/3142249219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b10101620 msgr2=0x7f9b1019d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:47.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.490+0000 7f9b14f5e700 1 --2- 172.21.15.37:0/3142249219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b10101620 0x7f9b1019d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.490+0000 7f9b14f5e700 1 -- 172.21.15.37:0/3142249219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b101066b0 msgr2=0x7f9b101979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:47.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.490+0000 7f9b14f5e700 1 --2- 172.21.15.37:0/3142249219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b101066b0 0x7f9b101979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.490+0000 7f9b14f5e700 1 -- 172.21.15.37:0/3142249219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b10198130 con 0x7f9b10101fd0 2024-06-10T01:11:47.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.490+0000 7f9b1575f700 1 --2- 172.21.15.37:0/3142249219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b10101620 0x7f9b1019d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:11:47.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.490+0000 7f9b15f60700 1 --2- 172.21.15.37:0/3142249219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b101066b0 0x7f9b101979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:11:47.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.491+0000 7f9b14f5e700 1 --2- 172.21.15.37:0/3142249219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b10101fd0 0x7f9b1019dd00 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000d9b0 tx=0x7f9b0000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:47.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.491+0000 7f9b067fc700 1 -- 172.21.15.37:0/3142249219 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0001b070 con 0x7f9b10101fd0 2024-06-10T01:11:47.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.491+0000 7f9b067fc700 1 -- 172.21.15.37:0/3142249219 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b0000ecd0 con 0x7f9b10101fd0 2024-06-10T01:11:47.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.491+0000 7f9b179c3700 1 -- 172.21.15.37:0/3142249219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b10198420 con 0x7f9b10101fd0 2024-06-10T01:11:47.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.491+0000 7f9b067fc700 1 -- 172.21.15.37:0/3142249219 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b000146c0 con 0x7f9b10101fd0 2024-06-10T01:11:47.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.491+0000 7f9b179c3700 1 -- 172.21.15.37:0/3142249219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b1006abc0 con 0x7f9b10101fd0 2024-06-10T01:11:47.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.492+0000 7f9b179c3700 1 -- 172.21.15.37:0/3142249219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b10069180 con 0x7f9b10101fd0 2024-06-10T01:11:47.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.493+0000 7f9b067fc700 1 -- 172.21.15.37:0/3142249219 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f9b0001e070 con 0x7f9b10101fd0 2024-06-10T01:11:47.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.494+0000 7f9b067fc700 1 --2- 172.21.15.37:0/3142249219 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9afc0a2c20 0x7f9afc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:47.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.494+0000 7f9b067fc700 1 -- 172.21.15.37:0/3142249219 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9237+0+0 (secure 0 0 0) 0x7f9b000d0f50 con 0x7f9b10101fd0 2024-06-10T01:11:47.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.494+0000 7f9b1575f700 1 --2- 172.21.15.37:0/3142249219 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9afc0a2c20 0x7f9afc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:47.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.495+0000 7f9b1575f700 1 --2- 172.21.15.37:0/3142249219 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9afc0a2c20 0x7f9afc0a50e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c000c00 tx=0x7f9b0c005940 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:47.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.497+0000 7f9b067fc700 1 -- 172.21.15.37:0/3142249219 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9b0009da60 con 0x7f9b10101fd0 2024-06-10T01:11:47.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.749+0000 7f9b179c3700 1 -- 172.21.15.37:0/3142249219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9b10198eb0 con 0x7f9b10101fd0 2024-06-10T01:11:47.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.751+0000 7f9b067fc700 1 -- 172.21.15.37:0/3142249219 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) ==== 161+0+0 (secure 0 0 0) 0x7f9b10198eb0 con 0x7f9b10101fd0 2024-06-10T01:11:47.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.753+0000 7f9b179c3700 1 -- 172.21.15.37:0/3142249219 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9afc0a2c20 msgr2=0x7f9afc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:47.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.753+0000 7f9b179c3700 1 --2- 172.21.15.37:0/3142249219 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9afc0a2c20 0x7f9afc0a50e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c000c00 tx=0x7f9b0c005940 comp rx=0 tx=0).stop 2024-06-10T01:11:47.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.753+0000 7f9b179c3700 1 -- 172.21.15.37:0/3142249219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b10101fd0 msgr2=0x7f9b1019dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:47.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.753+0000 7f9b179c3700 1 --2- 172.21.15.37:0/3142249219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b10101fd0 0x7f9b1019dd00 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000d9b0 tx=0x7f9b0000dd70 comp rx=0 tx=0).stop 2024-06-10T01:11:47.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.753+0000 7f9b179c3700 1 -- 172.21.15.37:0/3142249219 shutdown_connections 2024-06-10T01:11:47.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.753+0000 7f9b179c3700 1 --2- 172.21.15.37:0/3142249219 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f9afc0a2c20 0x7f9afc0a50e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.753+0000 7f9b179c3700 1 --2- 172.21.15.37:0/3142249219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b101066b0 0x7f9b101979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.753+0000 7f9b179c3700 1 --2- 172.21.15.37:0/3142249219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b10101620 0x7f9b1019d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.753+0000 7f9b179c3700 1 --2- 172.21.15.37:0/3142249219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b10101fd0 0x7f9b1019dd00 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:47.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.753+0000 7f9b179c3700 1 -- 172.21.15.37:0/3142249219 >> 172.21.15.37:0/3142249219 conn(0x7f9b10078340 msgr2=0x7f9b10100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:47.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.754+0000 7f9b179c3700 1 -- 172.21.15.37:0/3142249219 shutdown_connections 2024-06-10T01:11:47.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:47.754+0000 7f9b179c3700 1 -- 172.21.15.37:0/3142249219 wait complete. 2024-06-10T01:11:47.750 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-06-10T01:11:48.065 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:47 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-a[32187]: 2024-06-10T01:11:47.750+0000 7fd24a5dc700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-10T01:11:48.083 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jun 10 01:11:47 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-i[67988]: -8733> 2024-06-10T01:10:04.500+0000 7fe3cab4f700 -1 log_channel(cluster) log [ERR] : failed to store backtrace on ino 0x1000000c5da object, pool 2, errno -2 2024-06-10T01:11:48.084 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jun 10 01:11:47 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-i[67988]: -8732> 2024-06-10T01:10:04.500+0000 7fe3cab4f700 -1 mds.0.14 unhandled write error (2) No such file or directory, force readonly... 2024-06-10T01:11:48.084 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jun 10 01:11:47 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-i[67988]: -7857> 2024-06-10T01:10:04.509+0000 7fe3cab4f700 -1 mds.0.14 unhandled write error (2) No such file or directory, force readonly... 2024-06-10T01:11:48.084 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jun 10 01:11:47 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-i[67988]: ignoring --setuser ceph since I am not root 2024-06-10T01:11:48.084 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jun 10 01:11:47 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-i[67988]: ignoring --setgroup ceph since I am not root 2024-06-10T01:11:48.085 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jun 10 01:11:47 smithi167 ceph-mds[67992]: -- 172.21.15.167:0/413450312 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5638b7cd9a40 con 0x5638b7cac800 2024-06-10T01:11:48.085 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jun 10 01:11:47 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-i[67988]: starting mds.i at 2024-06-10T01:11:48.085 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jun 10 01:11:47 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-c[67682]: ignoring --setuser ceph since I am not root 2024-06-10T01:11:48.086 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jun 10 01:11:47 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-c[67682]: ignoring --setgroup ceph since I am not root 2024-06-10T01:11:48.086 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jun 10 01:11:47 smithi167 ceph-mds[67688]: -- 172.21.15.167:0/278240427 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564dd5b9da40 con 0x564dd5b70800 2024-06-10T01:11:48.086 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jun 10 01:11:47 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-c[67682]: starting mds.c at 2024-06-10T01:11:48.086 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jun 10 01:11:47 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-c[67682]: 2024-06-10T01:11:47.907+0000 7fc68be7f700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-06-10T01:11:48.087 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jun 10 01:11:47 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-l[67064]: ignoring --setuser ceph since I am not root 2024-06-10T01:11:48.087 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jun 10 01:11:47 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-l[67064]: ignoring --setgroup ceph since I am not root 2024-06-10T01:11:48.087 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jun 10 01:11:47 smithi167 ceph-mds[67069]: -- 172.21.15.167:0/272244283 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56095fe59a40 con 0x56095fe2d000 2024-06-10T01:11:48.087 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jun 10 01:11:47 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-l[67064]: starting mds.l at 2024-06-10T01:11:48.088 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jun 10 01:11:47 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-l[67064]: 2024-06-10T01:11:47.907+0000 7f92b8ce8700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-06-10T01:11:48.109 INFO:journalctl@ceph.mds.b.smithi094.stdout:Jun 10 01:11:47 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-b[67261]: ignoring --setuser ceph since I am not root 2024-06-10T01:11:48.109 INFO:journalctl@ceph.mds.b.smithi094.stdout:Jun 10 01:11:47 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-b[67261]: ignoring --setgroup ceph since I am not root 2024-06-10T01:11:48.110 INFO:journalctl@ceph.mds.b.smithi094.stdout:Jun 10 01:11:47 smithi094 ceph-mds[67265]: -- 172.21.15.94:0/1124105520 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d9b39b9a40 con 0x55d9b398c800 2024-06-10T01:11:48.110 INFO:journalctl@ceph.mds.b.smithi094.stdout:Jun 10 01:11:47 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-b[67261]: starting mds.b at 2024-06-10T01:11:48.111 INFO:journalctl@ceph.mds.e.smithi094.stdout:Jun 10 01:11:47 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-e[66642]: ignoring --setuser ceph since I am not root 2024-06-10T01:11:48.111 INFO:journalctl@ceph.mds.e.smithi094.stdout:Jun 10 01:11:47 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-e[66642]: ignoring --setgroup ceph since I am not root 2024-06-10T01:11:48.111 INFO:journalctl@ceph.mds.e.smithi094.stdout:Jun 10 01:11:47 smithi094 ceph-mds[66646]: -- 172.21.15.94:0/2192036446 <== mon.2 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5627c3c67a40 con 0x5627c3c3a800 2024-06-10T01:11:48.112 INFO:journalctl@ceph.mds.e.smithi094.stdout:Jun 10 01:11:47 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-e[66642]: starting mds.e at 2024-06-10T01:11:48.112 INFO:journalctl@ceph.mds.h.smithi094.stdout:Jun 10 01:11:47 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-h[66959]: ignoring --setuser ceph since I am not root 2024-06-10T01:11:48.112 INFO:journalctl@ceph.mds.h.smithi094.stdout:Jun 10 01:11:47 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-h[66959]: ignoring --setgroup ceph since I am not root 2024-06-10T01:11:48.113 INFO:journalctl@ceph.mds.h.smithi094.stdout:Jun 10 01:11:47 smithi094 ceph-mds[66963]: -- 172.21.15.94:0/624016420 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e924887a40 con 0x55e92485b000 2024-06-10T01:11:48.113 INFO:journalctl@ceph.mds.h.smithi094.stdout:Jun 10 01:11:47 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-h[66959]: starting mds.h at 2024-06-10T01:11:48.366 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:48 smithi037 ceph-mon[32192]: pgmap v888: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 6 op/s 2024-06-10T01:11:48.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:48 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3142249219' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-10T01:11:48.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:48 smithi037 ceph-mon[32192]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-10T01:11:48.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:48 smithi037 ceph-mon[32192]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-10T01:11:48.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:48 smithi037 ceph-mon[32192]: Health check cleared: MDS_READ_ONLY (was: 1 MDSs are read only) 2024-06-10T01:11:48.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:48 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3142249219' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-10T01:11:48.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:48 smithi037 ceph-mon[32192]: osdmap e106: 12 total, 12 up, 12 in 2024-06-10T01:11:48.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:48 smithi037 ceph-mon[32192]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-06-10T01:11:48.395 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-06-10T01:11:48.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:48 smithi167 ceph-mon[36917]: pgmap v888: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 6 op/s 2024-06-10T01:11:48.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:48 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3142249219' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-10T01:11:48.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:48 smithi167 ceph-mon[36917]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-10T01:11:48.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:48 smithi167 ceph-mon[36917]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-10T01:11:48.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:48 smithi167 ceph-mon[36917]: Health check cleared: MDS_READ_ONLY (was: 1 MDSs are read only) 2024-06-10T01:11:48.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:48 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3142249219' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-10T01:11:48.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:48 smithi167 ceph-mon[36917]: osdmap e106: 12 total, 12 up, 12 in 2024-06-10T01:11:48.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:48 smithi167 ceph-mon[36917]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-06-10T01:11:48.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:48 smithi094 ceph-mon[36708]: pgmap v888: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 6 op/s 2024-06-10T01:11:48.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:48 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3142249219' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-10T01:11:48.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:48 smithi094 ceph-mon[36708]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-10T01:11:48.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:48 smithi094 ceph-mon[36708]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-10T01:11:48.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:48 smithi094 ceph-mon[36708]: Health check cleared: MDS_READ_ONLY (was: 1 MDSs are read only) 2024-06-10T01:11:48.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:48 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3142249219' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-10T01:11:48.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:48 smithi094 ceph-mon[36708]: osdmap e106: 12 total, 12 up, 12 in 2024-06-10T01:11:48.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:48 smithi094 ceph-mon[36708]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-06-10T01:11:48.649 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:11:50.082 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:49 smithi037 ceph-mon[32192]: mds.? [v2:172.21.15.94:6834/2907691579,v1:172.21.15.94:6837/2907691579] up:boot 2024-06-10T01:11:50.083 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:49 smithi037 ceph-mon[32192]: mds.? [v2:172.21.15.94:6835/1522578197,v1:172.21.15.94:6838/1522578197] up:boot 2024-06-10T01:11:50.083 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:49 smithi037 ceph-mon[32192]: mds.? [v2:172.21.15.94:6836/1144690292,v1:172.21.15.94:6839/1144690292] up:boot 2024-06-10T01:11:50.083 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:49 smithi037 ceph-mon[32192]: mds.? [v2:172.21.15.167:6832/3306976383,v1:172.21.15.167:6837/3306976383] up:boot 2024-06-10T01:11:50.083 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:49 smithi037 ceph-mon[32192]: mds.? [v2:172.21.15.167:6836/1943805233,v1:172.21.15.167:6838/1943805233] up:boot 2024-06-10T01:11:50.083 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:49 smithi037 ceph-mon[32192]: mds.? [v2:172.21.15.167:6833/1510106763,v1:172.21.15.167:6839/1510106763] up:boot 2024-06-10T01:11:50.083 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:49 smithi037 ceph-mon[32192]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-06-10T01:11:50.083 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:49 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-10T01:11:50.083 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:49 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T01:11:50.084 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:49 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-10T01:11:50.084 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:49 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-10T01:11:50.084 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:49 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-10T01:11:50.084 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:49 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-10T01:11:50.084 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:49 smithi037 ceph-mon[32192]: pgmap v890: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-10T01:11:50.084 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:49 smithi167 ceph-mon[36917]: mds.? [v2:172.21.15.94:6834/2907691579,v1:172.21.15.94:6837/2907691579] up:boot 2024-06-10T01:11:50.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:49 smithi167 ceph-mon[36917]: mds.? [v2:172.21.15.94:6835/1522578197,v1:172.21.15.94:6838/1522578197] up:boot 2024-06-10T01:11:50.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:49 smithi167 ceph-mon[36917]: mds.? [v2:172.21.15.94:6836/1144690292,v1:172.21.15.94:6839/1144690292] up:boot 2024-06-10T01:11:50.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:49 smithi167 ceph-mon[36917]: mds.? [v2:172.21.15.167:6832/3306976383,v1:172.21.15.167:6837/3306976383] up:boot 2024-06-10T01:11:50.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:49 smithi167 ceph-mon[36917]: mds.? [v2:172.21.15.167:6836/1943805233,v1:172.21.15.167:6838/1943805233] up:boot 2024-06-10T01:11:50.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:49 smithi167 ceph-mon[36917]: mds.? [v2:172.21.15.167:6833/1510106763,v1:172.21.15.167:6839/1510106763] up:boot 2024-06-10T01:11:50.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:49 smithi167 ceph-mon[36917]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-06-10T01:11:50.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:49 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-10T01:11:50.086 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:49 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T01:11:50.086 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:49 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-10T01:11:50.086 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:49 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-10T01:11:50.086 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:49 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-10T01:11:50.086 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:49 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-10T01:11:50.086 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:49 smithi167 ceph-mon[36917]: pgmap v890: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-10T01:11:50.109 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:49 smithi094 ceph-mon[36708]: mds.? [v2:172.21.15.94:6834/2907691579,v1:172.21.15.94:6837/2907691579] up:boot 2024-06-10T01:11:50.109 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:49 smithi094 ceph-mon[36708]: mds.? [v2:172.21.15.94:6835/1522578197,v1:172.21.15.94:6838/1522578197] up:boot 2024-06-10T01:11:50.109 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:49 smithi094 ceph-mon[36708]: mds.? [v2:172.21.15.94:6836/1144690292,v1:172.21.15.94:6839/1144690292] up:boot 2024-06-10T01:11:50.109 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:49 smithi094 ceph-mon[36708]: mds.? [v2:172.21.15.167:6832/3306976383,v1:172.21.15.167:6837/3306976383] up:boot 2024-06-10T01:11:50.109 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:49 smithi094 ceph-mon[36708]: mds.? [v2:172.21.15.167:6836/1943805233,v1:172.21.15.167:6838/1943805233] up:boot 2024-06-10T01:11:50.110 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:49 smithi094 ceph-mon[36708]: mds.? [v2:172.21.15.167:6833/1510106763,v1:172.21.15.167:6839/1510106763] up:boot 2024-06-10T01:11:50.110 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:49 smithi094 ceph-mon[36708]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-06-10T01:11:50.110 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:49 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-10T01:11:50.110 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:49 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-10T01:11:50.110 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:49 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-10T01:11:50.110 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:49 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-10T01:11:50.110 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:49 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-10T01:11:50.111 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:49 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-10T01:11:50.111 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:49 smithi094 ceph-mon[36708]: pgmap v890: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-10T01:11:50.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.342+0000 7f6f0432d700 1 -- 172.21.15.37:0/4229444563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc107400 msgr2=0x7f6efc1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:50.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.342+0000 7f6f0432d700 1 --2- 172.21.15.37:0/4229444563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc107400 0x7f6efc1077e0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800b0a0 tx=0x7f6ef800b3b0 comp rx=0 tx=0).stop 2024-06-10T01:11:50.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.342+0000 7f6f0432d700 1 -- 172.21.15.37:0/4229444563 shutdown_connections 2024-06-10T01:11:50.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.342+0000 7f6f0432d700 1 --2- 172.21.15.37:0/4229444563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6efc101d50 0x7f6efc10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:50.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.342+0000 7f6f0432d700 1 --2- 172.21.15.37:0/4229444563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc107400 0x7f6efc1077e0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:50.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.342+0000 7f6f0432d700 1 --2- 172.21.15.37:0/4229444563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6efc101390 0x7f6efc101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:50.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.342+0000 7f6f0432d700 1 -- 172.21.15.37:0/4229444563 >> 172.21.15.37:0/4229444563 conn(0x7f6efc078340 msgr2=0x7f6efc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:50.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.342+0000 7f6f0432d700 1 -- 172.21.15.37:0/4229444563 shutdown_connections 2024-06-10T01:11:50.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.342+0000 7f6f0432d700 1 -- 172.21.15.37:0/4229444563 wait complete. 2024-06-10T01:11:50.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.342+0000 7f6f0432d700 1 Processor -- start 2024-06-10T01:11:50.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.343+0000 7f6f0432d700 1 -- start start 2024-06-10T01:11:50.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.343+0000 7f6f0432d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc101390 0x7f6efc103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:50.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.343+0000 7f6f0432d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6efc101d50 0x7f6efc1035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:50.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.343+0000 7f6f0432d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6efc107400 0x7f6efc103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:50.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.343+0000 7f6f0432d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc1116f0 con 0x7f6efc101390 2024-06-10T01:11:50.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.343+0000 7f6f0432d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc111570 con 0x7f6efc107400 2024-06-10T01:11:50.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.343+0000 7f6f0432d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc111870 con 0x7f6efc101d50 2024-06-10T01:11:50.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.343+0000 7f6f020c9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc101390 0x7f6efc103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:50.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.343+0000 7f6f020c9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc101390 0x7f6efc103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59754/0 (socket says 172.21.15.37:59754) 2024-06-10T01:11:50.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.343+0000 7f6f018c8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6efc101d50 0x7f6efc1035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:50.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.343+0000 7f6f020c9700 1 -- 172.21.15.37:0/3670533214 learned_addr learned my addr 172.21.15.37:0/3670533214 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:11:50.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.343+0000 7f6f028ca700 1 --2- 172.21.15.37:0/3670533214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6efc107400 0x7f6efc103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:50.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.343+0000 7f6f020c9700 1 -- 172.21.15.37:0/3670533214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6efc101d50 msgr2=0x7f6efc1035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:50.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.343+0000 7f6f020c9700 1 --2- 172.21.15.37:0/3670533214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6efc101d50 0x7f6efc1035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:50.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.344+0000 7f6f020c9700 1 -- 172.21.15.37:0/3670533214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6efc107400 msgr2=0x7f6efc103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:50.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.344+0000 7f6f020c9700 1 --2- 172.21.15.37:0/3670533214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6efc107400 0x7f6efc103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:50.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.344+0000 7f6f020c9700 1 -- 172.21.15.37:0/3670533214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6efc1a7150 con 0x7f6efc101390 2024-06-10T01:11:50.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.344+0000 7f6f028ca700 1 --2- 172.21.15.37:0/3670533214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6efc107400 0x7f6efc103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-10T01:11:50.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.344+0000 7f6f020c9700 1 --2- 172.21.15.37:0/3670533214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc101390 0x7f6efc103070 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800b070 tx=0x7f6ef8004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:50.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.344+0000 7f6ef37fe700 1 -- 172.21.15.37:0/3670533214 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8021070 con 0x7f6efc101390 2024-06-10T01:11:50.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.344+0000 7f6f0432d700 1 -- 172.21.15.37:0/3670533214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6efc1a73e0 con 0x7f6efc101390 2024-06-10T01:11:50.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.344+0000 7f6ef37fe700 1 -- 172.21.15.37:0/3670533214 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ef8007b80 con 0x7f6efc101390 2024-06-10T01:11:50.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.344+0000 7f6ef37fe700 1 -- 172.21.15.37:0/3670533214 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef80052a0 con 0x7f6efc101390 2024-06-10T01:11:50.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.344+0000 7f6f0432d700 1 -- 172.21.15.37:0/3670533214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6efc1a78e0 con 0x7f6efc101390 2024-06-10T01:11:50.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.345+0000 7f6f0432d700 1 -- 172.21.15.37:0/3670533214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6efc069180 con 0x7f6efc101390 2024-06-10T01:11:50.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.346+0000 7f6ef37fe700 1 -- 172.21.15.37:0/3670533214 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f6ef8024070 con 0x7f6efc101390 2024-06-10T01:11:50.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.347+0000 7f6ef37fe700 1 --2- 172.21.15.37:0/3670533214 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f6ee80a2ca0 0x7f6ee80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:50.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.347+0000 7f6f018c8700 1 --2- 172.21.15.37:0/3670533214 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f6ee80a2ca0 0x7f6ee80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:50.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.347+0000 7f6ef37fe700 1 -- 172.21.15.37:0/3670533214 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9495+0+0 (secure 0 0 0) 0x7f6ef800f020 con 0x7f6efc101390 2024-06-10T01:11:50.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.348+0000 7f6f018c8700 1 --2- 172.21.15.37:0/3670533214 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f6ee80a2ca0 0x7f6ee80a5160 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0ffb70 tx=0x7f6eec00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:50.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.352+0000 7f6ef37fe700 1 -- 172.21.15.37:0/3670533214 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6ef802b020 con 0x7f6efc101390 2024-06-10T01:11:50.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.603+0000 7f6f0432d700 1 -- 172.21.15.37:0/3670533214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f6efc1a7ab0 con 0x7f6efc101390 2024-06-10T01:11:50.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.764+0000 7f6ef37fe700 1 -- 172.21.15.37:0/3670533214 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) ==== 104+0+0 (secure 0 0 0) 0x7f6ef809d630 con 0x7f6efc101390 2024-06-10T01:11:50.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.766+0000 7f6f0432d700 1 -- 172.21.15.37:0/3670533214 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f6ee80a2ca0 msgr2=0x7f6ee80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:50.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.766+0000 7f6f0432d700 1 --2- 172.21.15.37:0/3670533214 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f6ee80a2ca0 0x7f6ee80a5160 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0ffb70 tx=0x7f6eec00b410 comp rx=0 tx=0).stop 2024-06-10T01:11:50.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.766+0000 7f6f0432d700 1 -- 172.21.15.37:0/3670533214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc101390 msgr2=0x7f6efc103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:50.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.766+0000 7f6f0432d700 1 --2- 172.21.15.37:0/3670533214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc101390 0x7f6efc103070 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800b070 tx=0x7f6ef8004e40 comp rx=0 tx=0).stop 2024-06-10T01:11:50.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.767+0000 7f6f0432d700 1 -- 172.21.15.37:0/3670533214 shutdown_connections 2024-06-10T01:11:50.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.767+0000 7f6f0432d700 1 --2- 172.21.15.37:0/3670533214 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7f6ee80a2ca0 0x7f6ee80a5160 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:50.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.767+0000 7f6f0432d700 1 --2- 172.21.15.37:0/3670533214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6efc107400 0x7f6efc103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:50.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.767+0000 7f6f0432d700 1 --2- 172.21.15.37:0/3670533214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc101390 0x7f6efc103070 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:50.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.767+0000 7f6f0432d700 1 --2- 172.21.15.37:0/3670533214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6efc101d50 0x7f6efc1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:50.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.767+0000 7f6f0432d700 1 -- 172.21.15.37:0/3670533214 >> 172.21.15.37:0/3670533214 conn(0x7f6efc078340 msgr2=0x7f6efc1005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:50.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.767+0000 7f6f0432d700 1 -- 172.21.15.37:0/3670533214 shutdown_connections 2024-06-10T01:11:50.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:50.767+0000 7f6f0432d700 1 -- 172.21.15.37:0/3670533214 wait complete. 2024-06-10T01:11:50.890 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:50 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3670533214' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-10T01:11:51.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:50 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3670533214' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-10T01:11:51.108 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:50 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3670533214' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-10T01:11:51.514 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-06-10T01:11:51.773 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:11:52.062 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:51 smithi037 ceph-mon[32192]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-10T01:11:52.062 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:51 smithi037 ceph-mon[32192]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-10T01:11:52.062 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:51 smithi037 ceph-mon[32192]: Cluster is now healthy 2024-06-10T01:11:52.063 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:51 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3670533214' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-10T01:11:52.063 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:51 smithi037 ceph-mon[32192]: fsmap 12 up:standby 2024-06-10T01:11:52.063 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:51 smithi037 ceph-mon[32192]: pgmap v891: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-10T01:11:52.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:51 smithi167 ceph-mon[36917]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-10T01:11:52.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:51 smithi167 ceph-mon[36917]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-10T01:11:52.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:51 smithi167 ceph-mon[36917]: Cluster is now healthy 2024-06-10T01:11:52.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:51 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3670533214' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-10T01:11:52.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:51 smithi167 ceph-mon[36917]: fsmap 12 up:standby 2024-06-10T01:11:52.084 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:51 smithi167 ceph-mon[36917]: pgmap v891: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-10T01:11:52.109 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:51 smithi094 ceph-mon[36708]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-10T01:11:52.109 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:51 smithi094 ceph-mon[36708]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-10T01:11:52.109 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:51 smithi094 ceph-mon[36708]: Cluster is now healthy 2024-06-10T01:11:52.109 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:51 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3670533214' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-10T01:11:52.109 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:51 smithi094 ceph-mon[36708]: fsmap 12 up:standby 2024-06-10T01:11:52.109 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:51 smithi094 ceph-mon[36708]: pgmap v891: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-10T01:11:53.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.513+0000 7fe6e2175700 1 -- 172.21.15.37:0/3908641564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6dc06ab30 msgr2=0x7fe6dc10f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.513+0000 7fe6e2175700 1 --2- 172.21.15.37:0/3908641564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6dc06ab30 0x7fe6dc10f320 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fe6d400b0a0 tx=0x7fe6d400b3b0 comp rx=0 tx=0).stop 2024-06-10T01:11:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.513+0000 7fe6e2175700 1 -- 172.21.15.37:0/3908641564 shutdown_connections 2024-06-10T01:11:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.513+0000 7fe6e2175700 1 --2- 172.21.15.37:0/3908641564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6dc06ab30 0x7fe6dc10f320 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.513+0000 7fe6e2175700 1 --2- 172.21.15.37:0/3908641564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6dc108500 0x7fe6dc1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.513+0000 7fe6e2175700 1 --2- 172.21.15.37:0/3908641564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6dc06a170 0x7fe6dc06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.513+0000 7fe6e2175700 1 -- 172.21.15.37:0/3908641564 >> 172.21.15.37:0/3908641564 conn(0x7fe6dc078340 msgr2=0x7fe6dc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.513+0000 7fe6e2175700 1 -- 172.21.15.37:0/3908641564 shutdown_connections 2024-06-10T01:11:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.513+0000 7fe6e2175700 1 -- 172.21.15.37:0/3908641564 wait complete. 2024-06-10T01:11:53.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.514+0000 7fe6e2175700 1 Processor -- start 2024-06-10T01:11:53.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.514+0000 7fe6e2175700 1 -- start start 2024-06-10T01:11:53.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.514+0000 7fe6e2175700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6dc06a170 0x7fe6dc19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:53.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.514+0000 7fe6e2175700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6dc06ab30 0x7fe6dc19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:53.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.514+0000 7fe6e2175700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6dc108500 0x7fe6dc198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:53.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.514+0000 7fe6e2175700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc1127d0 con 0x7fe6dc108500 2024-06-10T01:11:53.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.514+0000 7fe6e2175700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc112650 con 0x7fe6dc06ab30 2024-06-10T01:11:53.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.514+0000 7fe6e2175700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc112950 con 0x7fe6dc06a170 2024-06-10T01:11:53.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.514+0000 7fe6dbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6dc108500 0x7fe6dc198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:53.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.514+0000 7fe6dbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6dc108500 0x7fe6dc198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40482/0 (socket says 172.21.15.37:40482) 2024-06-10T01:11:53.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.514+0000 7fe6dbfff700 1 -- 172.21.15.37:0/2146324099 learned_addr learned my addr 172.21.15.37:0/2146324099 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:11:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.514+0000 7fe6daffd700 1 --2- 172.21.15.37:0/2146324099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6dc06ab30 0x7fe6dc19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.514+0000 7fe6db7fe700 1 --2- 172.21.15.37:0/2146324099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6dc06a170 0x7fe6dc19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.515+0000 7fe6dbfff700 1 -- 172.21.15.37:0/2146324099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6dc06a170 msgr2=0x7fe6dc19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.515+0000 7fe6dbfff700 1 --2- 172.21.15.37:0/2146324099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6dc06a170 0x7fe6dc19e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.515+0000 7fe6dbfff700 1 -- 172.21.15.37:0/2146324099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6dc06ab30 msgr2=0x7fe6dc19edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.515+0000 7fe6dbfff700 1 --2- 172.21.15.37:0/2146324099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6dc06ab30 0x7fe6dc19edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.515+0000 7fe6dbfff700 1 -- 172.21.15.37:0/2146324099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6dc1991b0 con 0x7fe6dc108500 2024-06-10T01:11:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.515+0000 7fe6daffd700 1 --2- 172.21.15.37:0/2146324099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6dc06ab30 0x7fe6dc19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:11:53.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.515+0000 7fe6dbfff700 1 --2- 172.21.15.37:0/2146324099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6dc108500 0x7fe6dc198a50 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fe6d400b070 tx=0x7fe6d4004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:53.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.515+0000 7fe6d8ff9700 1 -- 172.21.15.37:0/2146324099 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4021070 con 0x7fe6dc108500 2024-06-10T01:11:53.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.515+0000 7fe6d8ff9700 1 -- 172.21.15.37:0/2146324099 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6d4003c00 con 0x7fe6dc108500 2024-06-10T01:11:53.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.515+0000 7fe6e2175700 1 -- 172.21.15.37:0/2146324099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6dc199440 con 0x7fe6dc108500 2024-06-10T01:11:53.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.515+0000 7fe6d8ff9700 1 -- 172.21.15.37:0/2146324099 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d40047f0 con 0x7fe6dc108500 2024-06-10T01:11:53.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.515+0000 7fe6e2175700 1 -- 172.21.15.37:0/2146324099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6dc1a5760 con 0x7fe6dc108500 2024-06-10T01:11:53.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.516+0000 7fe6e2175700 1 -- 172.21.15.37:0/2146324099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6dc10a9e0 con 0x7fe6dc108500 2024-06-10T01:11:53.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.517+0000 7fe6d8ff9700 1 -- 172.21.15.37:0/2146324099 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fe6d4007980 con 0x7fe6dc108500 2024-06-10T01:11:53.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.518+0000 7fe6d8ff9700 1 --2- 172.21.15.37:0/2146324099 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe6c40a2c20 0x7fe6c40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:53.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.518+0000 7fe6d8ff9700 1 -- 172.21.15.37:0/2146324099 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9495+0+0 (secure 0 0 0) 0x7fe6d400f020 con 0x7fe6dc108500 2024-06-10T01:11:53.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.518+0000 7fe6db7fe700 1 --2- 172.21.15.37:0/2146324099 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe6c40a2c20 0x7fe6c40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:53.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.518+0000 7fe6db7fe700 1 --2- 172.21.15.37:0/2146324099 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe6c40a2c20 0x7fe6c40a50e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc199b90 tx=0x7fe6cc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:53.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.522+0000 7fe6d8ff9700 1 -- 172.21.15.37:0/2146324099 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe6d402a020 con 0x7fe6dc108500 2024-06-10T01:11:53.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:53.730+0000 7fe6e2175700 1 -- 172.21.15.37:0/2146324099 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe6dc199f80 con 0x7fe6dc108500 2024-06-10T01:11:54.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:54.190+0000 7fe6d8ff9700 1 -- 172.21.15.37:0/2146324099 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v107) ==== 181+0+0 (secure 0 0 0) 0x7fe6d409d400 con 0x7fe6dc108500 2024-06-10T01:11:54.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:54.193+0000 7fe6e2175700 1 -- 172.21.15.37:0/2146324099 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe6c40a2c20 msgr2=0x7fe6c40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:54.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:54.193+0000 7fe6e2175700 1 --2- 172.21.15.37:0/2146324099 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe6c40a2c20 0x7fe6c40a50e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc199b90 tx=0x7fe6cc009260 comp rx=0 tx=0).stop 2024-06-10T01:11:54.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:54.193+0000 7fe6e2175700 1 -- 172.21.15.37:0/2146324099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6dc108500 msgr2=0x7fe6dc198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:54.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:54.193+0000 7fe6e2175700 1 --2- 172.21.15.37:0/2146324099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6dc108500 0x7fe6dc198a50 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fe6d400b070 tx=0x7fe6d4004c60 comp rx=0 tx=0).stop 2024-06-10T01:11:54.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:54.195+0000 7fe6e2175700 1 -- 172.21.15.37:0/2146324099 shutdown_connections 2024-06-10T01:11:54.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:54.195+0000 7fe6e2175700 1 --2- 172.21.15.37:0/2146324099 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fe6c40a2c20 0x7fe6c40a50e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:54.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:54.195+0000 7fe6e2175700 1 --2- 172.21.15.37:0/2146324099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6dc108500 0x7fe6dc198a50 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:54.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:54.195+0000 7fe6e2175700 1 --2- 172.21.15.37:0/2146324099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6dc06a170 0x7fe6dc19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:54.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:54.195+0000 7fe6e2175700 1 --2- 172.21.15.37:0/2146324099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6dc06ab30 0x7fe6dc19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:54.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:54.195+0000 7fe6e2175700 1 -- 172.21.15.37:0/2146324099 >> 172.21.15.37:0/2146324099 conn(0x7fe6dc078340 msgr2=0x7fe6dc0ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:54.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:54.196+0000 7fe6e2175700 1 -- 172.21.15.37:0/2146324099 shutdown_connections 2024-06-10T01:11:54.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:54.196+0000 7fe6e2175700 1 -- 172.21.15.37:0/2146324099 wait complete. 2024-06-10T01:11:54.193 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-06-10T01:11:54.538 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:54 smithi037 ceph-mon[32192]: pgmap v892: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-06-10T01:11:54.538 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:54 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2146324099' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T01:11:54.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:54 smithi167 ceph-mon[36917]: pgmap v892: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-06-10T01:11:54.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:54 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2146324099' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T01:11:54.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:54 smithi094 ceph-mon[36708]: pgmap v892: 129 pgs: 129 active+clean; 2.9 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3 op/s 2024-06-10T01:11:54.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:54 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2146324099' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T01:11:54.958 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-06-10T01:11:55.220 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:11:55.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:55 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/2146324099' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T01:11:55.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:55 smithi037 ceph-mon[32192]: osdmap e107: 12 total, 12 up, 12 in 2024-06-10T01:11:55.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:55 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/2146324099' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T01:11:55.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:55 smithi167 ceph-mon[36917]: osdmap e107: 12 total, 12 up, 12 in 2024-06-10T01:11:55.608 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:55 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/2146324099' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T01:11:55.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:55 smithi094 ceph-mon[36708]: osdmap e107: 12 total, 12 up, 12 in 2024-06-10T01:11:56.476 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:56 smithi037 ceph-mon[32192]: pgmap v894: 97 pgs: 97 active+clean; 577 KiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:11:56.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:56 smithi167 ceph-mon[36917]: pgmap v894: 97 pgs: 97 active+clean; 577 KiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:11:56.608 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:56 smithi094 ceph-mon[36708]: pgmap v894: 97 pgs: 97 active+clean; 577 KiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:11:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.078+0000 7fd51a778700 1 -- 172.21.15.37:0/1885697971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd514102740 msgr2=0x7fd514102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.078+0000 7fd51a778700 1 --2- 172.21.15.37:0/1885697971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd514102740 0x7fd514102b20 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fd5040099c0 tx=0x7fd504009cd0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.078+0000 7fd51a778700 1 -- 172.21.15.37:0/1885697971 shutdown_connections 2024-06-10T01:11:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.078+0000 7fd51a778700 1 --2- 172.21.15.37:0/1885697971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd514107830 0x7fd51410f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.078+0000 7fd51a778700 1 --2- 172.21.15.37:0/1885697971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd514102740 0x7fd514102b20 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.078+0000 7fd51a778700 1 --2- 172.21.15.37:0/1885697971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5141030f0 0x7fd514107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.078+0000 7fd51a778700 1 -- 172.21.15.37:0/1885697971 >> 172.21.15.37:0/1885697971 conn(0x7fd5140795b0 msgr2=0x7fd5140799c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:57.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.078+0000 7fd51a778700 1 -- 172.21.15.37:0/1885697971 shutdown_connections 2024-06-10T01:11:57.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.078+0000 7fd51a778700 1 -- 172.21.15.37:0/1885697971 wait complete. 2024-06-10T01:11:57.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.079+0000 7fd51a778700 1 Processor -- start 2024-06-10T01:11:57.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.079+0000 7fd51a778700 1 -- start start 2024-06-10T01:11:57.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.080+0000 7fd51a778700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd514102740 0x7fd51419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:57.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.080+0000 7fd51a778700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5141030f0 0x7fd51419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:57.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.080+0000 7fd51a778700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd514107830 0x7fd514198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:57.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.080+0000 7fd51a778700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd514112840 con 0x7fd5141030f0 2024-06-10T01:11:57.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.080+0000 7fd51a778700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd5141126c0 con 0x7fd514107830 2024-06-10T01:11:57.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.080+0000 7fd51a778700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd5141129c0 con 0x7fd514102740 2024-06-10T01:11:57.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.080+0000 7fd50bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5141030f0 0x7fd51419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:57.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.080+0000 7fd50bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5141030f0 0x7fd51419ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40508/0 (socket says 172.21.15.37:40508) 2024-06-10T01:11:57.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.080+0000 7fd50bfff700 1 -- 172.21.15.37:0/3615300671 learned_addr learned my addr 172.21.15.37:0/3615300671 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:11:57.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.080+0000 7fd513fff700 1 --2- 172.21.15.37:0/3615300671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd514102740 0x7fd51419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.080+0000 7fd518d15700 1 --2- 172.21.15.37:0/3615300671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd514107830 0x7fd514198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.080+0000 7fd50bfff700 1 -- 172.21.15.37:0/3615300671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd514102740 msgr2=0x7fd51419e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.080+0000 7fd50bfff700 1 --2- 172.21.15.37:0/3615300671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd514102740 0x7fd51419e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.080+0000 7fd50bfff700 1 -- 172.21.15.37:0/3615300671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd514107830 msgr2=0x7fd514198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.080+0000 7fd50bfff700 1 --2- 172.21.15.37:0/3615300671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd514107830 0x7fd514198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.080+0000 7fd50bfff700 1 -- 172.21.15.37:0/3615300671 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd514199240 con 0x7fd5141030f0 2024-06-10T01:11:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.080+0000 7fd518d15700 1 --2- 172.21.15.37:0/3615300671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd514107830 0x7fd514198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-10T01:11:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.080+0000 7fd50bfff700 1 --2- 172.21.15.37:0/3615300671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5141030f0 0x7fd51419ee60 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00b530 tx=0x7fd4fc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:57.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.081+0000 7fd511ffb700 1 -- 172.21.15.37:0/3615300671 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc015020 con 0x7fd5141030f0 2024-06-10T01:11:57.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.081+0000 7fd511ffb700 1 -- 172.21.15.37:0/3615300671 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4fc011d10 con 0x7fd5141030f0 2024-06-10T01:11:57.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.081+0000 7fd51a778700 1 -- 172.21.15.37:0/3615300671 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd514199530 con 0x7fd5141030f0 2024-06-10T01:11:57.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.081+0000 7fd511ffb700 1 -- 172.21.15.37:0/3615300671 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc00f2e0 con 0x7fd5141030f0 2024-06-10T01:11:57.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.081+0000 7fd51a778700 1 -- 172.21.15.37:0/3615300671 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5141a5870 con 0x7fd5141030f0 2024-06-10T01:11:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.082+0000 7fd51a778700 1 -- 172.21.15.37:0/3615300671 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd51410aa70 con 0x7fd5141030f0 2024-06-10T01:11:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.083+0000 7fd511ffb700 1 -- 172.21.15.37:0/3615300671 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fd4fc018020 con 0x7fd5141030f0 2024-06-10T01:11:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.084+0000 7fd511ffb700 1 --2- 172.21.15.37:0/3615300671 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd5000a2ca0 0x7fd5000a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:11:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.084+0000 7fd511ffb700 1 -- 172.21.15.37:0/3615300671 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8912+0+0 (secure 0 0 0) 0x7fd4fc0ceb00 con 0x7fd5141030f0 2024-06-10T01:11:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.084+0000 7fd513fff700 1 --2- 172.21.15.37:0/3615300671 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd5000a2ca0 0x7fd5000a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:11:57.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.084+0000 7fd513fff700 1 --2- 172.21.15.37:0/3615300671 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd5000a2ca0 0x7fd5000a5160 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd504000c00 tx=0x7fd504005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:11:57.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.088+0000 7fd511ffb700 1 -- 172.21.15.37:0/3615300671 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd4fc01f020 con 0x7fd5141030f0 2024-06-10T01:11:57.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:57.295+0000 7fd51a778700 1 -- 172.21.15.37:0/3615300671 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd514199fd0 con 0x7fd5141030f0 2024-06-10T01:11:58.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:58.214+0000 7fd511ffb700 1 -- 172.21.15.37:0/3615300671 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v108) ==== 169+0+0 (secure 0 0 0) 0x7fd4fc09b7c0 con 0x7fd5141030f0 2024-06-10T01:11:58.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:58.218+0000 7fd51a778700 1 -- 172.21.15.37:0/3615300671 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd5000a2ca0 msgr2=0x7fd5000a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:58.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:58.218+0000 7fd51a778700 1 --2- 172.21.15.37:0/3615300671 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd5000a2ca0 0x7fd5000a5160 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd504000c00 tx=0x7fd504005fb0 comp rx=0 tx=0).stop 2024-06-10T01:11:58.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:58.218+0000 7fd51a778700 1 -- 172.21.15.37:0/3615300671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5141030f0 msgr2=0x7fd51419ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:11:58.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:58.218+0000 7fd51a778700 1 --2- 172.21.15.37:0/3615300671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5141030f0 0x7fd51419ee60 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00b530 tx=0x7fd4fc00b840 comp rx=0 tx=0).stop 2024-06-10T01:11:58.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:58.218+0000 7fd51a778700 1 -- 172.21.15.37:0/3615300671 shutdown_connections 2024-06-10T01:11:58.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:58.218+0000 7fd51a778700 1 --2- 172.21.15.37:0/3615300671 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fd5000a2ca0 0x7fd5000a5160 secure :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd504000c00 tx=0x7fd504005fb0 comp rx=0 tx=0).stop 2024-06-10T01:11:58.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:58.218+0000 7fd51a778700 1 --2- 172.21.15.37:0/3615300671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd514107830 0x7fd514198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:58.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:58.218+0000 7fd51a778700 1 --2- 172.21.15.37:0/3615300671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd514102740 0x7fd51419e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:58.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:58.218+0000 7fd51a778700 1 --2- 172.21.15.37:0/3615300671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5141030f0 0x7fd51419ee60 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:11:58.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:58.218+0000 7fd51a778700 1 -- 172.21.15.37:0/3615300671 >> 172.21.15.37:0/3615300671 conn(0x7fd5140795b0 msgr2=0x7fd5141007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:11:58.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:58.220+0000 7fd51a778700 1 -- 172.21.15.37:0/3615300671 shutdown_connections 2024-06-10T01:11:58.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:11:58.220+0000 7fd51a778700 1 -- 172.21.15.37:0/3615300671 wait complete. 2024-06-10T01:11:58.217 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-06-10T01:11:58.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:58 smithi167 ceph-mon[36917]: pgmap v895: 97 pgs: 97 active+clean; 577 KiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:11:58.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:58 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3615300671' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T01:11:58.608 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:58 smithi094 ceph-mon[36708]: pgmap v895: 97 pgs: 97 active+clean; 577 KiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:11:58.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:58 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3615300671' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T01:11:58.609 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:58 smithi037 ceph-mon[32192]: pgmap v895: 97 pgs: 97 active+clean; 577 KiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:11:58.609 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:58 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3615300671' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T01:11:58.930 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 68793514-26c2-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-06-10T01:11:59.181 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/mon.a/config 2024-06-10T01:11:59.572 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:59 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/3615300671' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T01:11:59.572 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:11:59 smithi037 ceph-mon[32192]: osdmap e108: 12 total, 12 up, 12 in 2024-06-10T01:11:59.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:59 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/3615300671' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T01:11:59.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:11:59 smithi167 ceph-mon[36917]: osdmap e108: 12 total, 12 up, 12 in 2024-06-10T01:11:59.608 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:59 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/3615300671' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T01:11:59.608 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:11:59 smithi094 ceph-mon[36708]: osdmap e108: 12 total, 12 up, 12 in 2024-06-10T01:12:00.390 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:12:00 smithi037 ceph-mon[32192]: pgmap v897: 33 pgs: 33 active+clean; 577 KiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:12:00.390 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:12:00 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:12:00.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:12:00 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:12:00.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:12:00 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:12:00.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:12:00 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:12:00.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:12:00 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:12:00.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:12:00 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:12:00.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:12:00 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:12:00.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:12:00 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:12:00.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:12:00 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:12:00.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:12:00 smithi167 ceph-mon[36917]: pgmap v897: 33 pgs: 33 active+clean; 577 KiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:12:00.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:12:00 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:12:00.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:12:00 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:12:00.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:12:00 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:12:00.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:12:00 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:12:00.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:12:00 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:12:00.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:12:00 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:12:00.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:12:00 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:12:00.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:12:00 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:12:00.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:12:00 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:12:00.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:12:00 smithi094 ceph-mon[36708]: pgmap v897: 33 pgs: 33 active+clean; 577 KiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:12:00.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:12:00 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-10T01:12:00.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:12:00 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:12:00.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:12:00 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:12:00.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:12:00 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:12:00.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:12:00 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:12:00.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:12:00 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:12:00.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:12:00 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:12:00.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:12:00 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:12:00.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:12:00 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-10T01:12:00.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.980+0000 7fa325e2b700 1 -- 172.21.15.37:0/3172346870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3201085d0 msgr2=0x7fa3201089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:00.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.980+0000 7fa325e2b700 1 --2- 172.21.15.37:0/3172346870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3201085d0 0x7fa3201089b0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fa31000b0d0 tx=0x7fa31000b3e0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.980+0000 7fa325e2b700 1 -- 172.21.15.37:0/3172346870 shutdown_connections 2024-06-10T01:12:00.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.980+0000 7fa325e2b700 1 --2- 172.21.15.37:0/3172346870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa320102ef0 0x7fa32010f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.980+0000 7fa325e2b700 1 --2- 172.21.15.37:0/3172346870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3201085d0 0x7fa3201089b0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.980+0000 7fa325e2b700 1 --2- 172.21.15.37:0/3172346870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa320102530 0x7fa3201029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.980+0000 7fa325e2b700 1 -- 172.21.15.37:0/3172346870 >> 172.21.15.37:0/3172346870 conn(0x7fa320079640 msgr2=0x7fa320079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:00.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.980+0000 7fa325e2b700 1 -- 172.21.15.37:0/3172346870 shutdown_connections 2024-06-10T01:12:00.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.980+0000 7fa325e2b700 1 -- 172.21.15.37:0/3172346870 wait complete. 2024-06-10T01:12:00.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.981+0000 7fa325e2b700 1 Processor -- start 2024-06-10T01:12:00.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.981+0000 7fa325e2b700 1 -- start start 2024-06-10T01:12:00.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.981+0000 7fa325e2b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa320102530 0x7fa32019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:00.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.981+0000 7fa325e2b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa320102ef0 0x7fa32019edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:00.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.982+0000 7fa325e2b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3201085d0 0x7fa320198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:00.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.982+0000 7fa325e2b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa320112810 con 0x7fa320102ef0 2024-06-10T01:12:00.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.982+0000 7fa325e2b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa320112690 con 0x7fa3201085d0 2024-06-10T01:12:00.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.982+0000 7fa325e2b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa320112990 con 0x7fa320102530 2024-06-10T01:12:00.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.982+0000 7fa31effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa320102ef0 0x7fa32019edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:00.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.982+0000 7fa31effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa320102ef0 0x7fa32019edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40530/0 (socket says 172.21.15.37:40530) 2024-06-10T01:12:00.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.982+0000 7fa31effd700 1 -- 172.21.15.37:0/1045127376 learned_addr learned my addr 172.21.15.37:0/1045127376 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-06-10T01:12:00.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.982+0000 7fa31f7fe700 1 --2- 172.21.15.37:0/1045127376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa320102530 0x7fa32019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:00.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.982+0000 7fa31ffff700 1 --2- 172.21.15.37:0/1045127376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3201085d0 0x7fa320198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:00.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.982+0000 7fa31effd700 1 -- 172.21.15.37:0/1045127376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa320102530 msgr2=0x7fa32019e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:00.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.982+0000 7fa31effd700 1 --2- 172.21.15.37:0/1045127376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa320102530 0x7fa32019e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.982+0000 7fa31effd700 1 -- 172.21.15.37:0/1045127376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3201085d0 msgr2=0x7fa320198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:00.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.982+0000 7fa31effd700 1 --2- 172.21.15.37:0/1045127376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3201085d0 0x7fa320198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:00.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.982+0000 7fa31effd700 1 -- 172.21.15.37:0/1045127376 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3201991f0 con 0x7fa320102ef0 2024-06-10T01:12:00.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.982+0000 7fa31effd700 1 --2- 172.21.15.37:0/1045127376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa320102ef0 0x7fa32019edc0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fa30800d950 tx=0x7fa30800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:00.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.983+0000 7fa31cff9700 1 -- 172.21.15.37:0/1045127376 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa308016070 con 0x7fa320102ef0 2024-06-10T01:12:00.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.983+0000 7fa31cff9700 1 -- 172.21.15.37:0/1045127376 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa30800fd70 con 0x7fa320102ef0 2024-06-10T01:12:00.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.983+0000 7fa325e2b700 1 -- 172.21.15.37:0/1045127376 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3201994e0 con 0x7fa320102ef0 2024-06-10T01:12:00.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.983+0000 7fa31cff9700 1 -- 172.21.15.37:0/1045127376 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30800e610 con 0x7fa320102ef0 2024-06-10T01:12:00.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.983+0000 7fa325e2b700 1 -- 172.21.15.37:0/1045127376 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3201a5750 con 0x7fa320102ef0 2024-06-10T01:12:00.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.984+0000 7fa325e2b700 1 -- 172.21.15.37:0/1045127376 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa32010aa60 con 0x7fa320102ef0 2024-06-10T01:12:00.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.985+0000 7fa31cff9700 1 -- 172.21.15.37:0/1045127376 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fa308019070 con 0x7fa320102ef0 2024-06-10T01:12:00.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.986+0000 7fa31cff9700 1 --2- 172.21.15.37:0/1045127376 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa30c0a2c20 0x7fa30c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-10T01:12:00.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.986+0000 7fa31f7fe700 1 --2- 172.21.15.37:0/1045127376 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa30c0a2c20 0x7fa30c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-10T01:12:00.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.986+0000 7fa31cff9700 1 -- 172.21.15.37:0/1045127376 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8324+0+0 (secure 0 0 0) 0x7fa3080ceb30 con 0x7fa320102ef0 2024-06-10T01:12:00.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.987+0000 7fa31f7fe700 1 --2- 172.21.15.37:0/1045127376 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa30c0a2c20 0x7fa30c0a50e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa310009fd0 tx=0x7fa31000b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-10T01:12:00.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:00.990+0000 7fa31cff9700 1 -- 172.21.15.37:0/1045127376 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa30809ba30 con 0x7fa320102ef0 2024-06-10T01:12:01.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:01.197+0000 7fa325e2b700 1 -- 172.21.15.37:0/1045127376 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa320069180 con 0x7fa320102ef0 2024-06-10T01:12:01.390 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:12:01 smithi037 ceph-mon[32192]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-06-10T01:12:01.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:12:01 smithi037 ceph-mon[32192]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-06-10T01:12:01.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:12:01 smithi037 ceph-mon[32192]: Unable to set osd_memory_target on smithi094 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-06-10T01:12:01.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:12:01 smithi037 ceph-mon[32192]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-06-10T01:12:01.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:12:01 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:12:01.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:12:01 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:12:01.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:12:01 smithi037 ceph-mon[32192]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:12:01.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:12:01 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1045127376' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T01:12:01.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:12:01 smithi167 ceph-mon[36917]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-06-10T01:12:01.625 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:12:01 smithi167 ceph-mon[36917]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-06-10T01:12:01.626 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:12:01 smithi167 ceph-mon[36917]: Unable to set osd_memory_target on smithi094 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-06-10T01:12:01.626 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:12:01 smithi167 ceph-mon[36917]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-06-10T01:12:01.626 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:12:01 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:12:01.626 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:12:01 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:12:01.626 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:12:01 smithi167 ceph-mon[36917]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:12:01.626 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:12:01 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1045127376' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T01:12:01.627 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:12:01 smithi094 ceph-mon[36708]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-06-10T01:12:01.627 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:12:01 smithi094 ceph-mon[36708]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-06-10T01:12:01.627 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:12:01 smithi094 ceph-mon[36708]: Unable to set osd_memory_target on smithi094 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-06-10T01:12:01.627 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:12:01 smithi094 ceph-mon[36708]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-06-10T01:12:01.628 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:12:01 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-10T01:12:01.628 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:12:01 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-10T01:12:01.628 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:12:01 smithi094 ceph-mon[36708]: from='mgr.14148 172.21.15.37:0/1605869799' entity='mgr.x' 2024-06-10T01:12:01.628 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:12:01 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1045127376' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-10T01:12:01.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:01.642+0000 7fa31cff9700 1 -- 172.21.15.37:0/1045127376 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v109) ==== 178+0+0 (secure 0 0 0) 0x7fa30809b330 con 0x7fa320102ef0 2024-06-10T01:12:01.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:01.647+0000 7fa325e2b700 1 -- 172.21.15.37:0/1045127376 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa30c0a2c20 msgr2=0x7fa30c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:01.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:01.647+0000 7fa325e2b700 1 --2- 172.21.15.37:0/1045127376 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa30c0a2c20 0x7fa30c0a50e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa310009fd0 tx=0x7fa31000b040 comp rx=0 tx=0).stop 2024-06-10T01:12:01.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:01.647+0000 7fa325e2b700 1 -- 172.21.15.37:0/1045127376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa320102ef0 msgr2=0x7fa32019edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-10T01:12:01.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:01.647+0000 7fa325e2b700 1 --2- 172.21.15.37:0/1045127376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa320102ef0 0x7fa32019edc0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fa30800d950 tx=0x7fa30800dc60 comp rx=0 tx=0).stop 2024-06-10T01:12:01.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:01.647+0000 7fa325e2b700 1 -- 172.21.15.37:0/1045127376 shutdown_connections 2024-06-10T01:12:01.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:01.647+0000 7fa325e2b700 1 --2- 172.21.15.37:0/1045127376 >> [v2:172.21.15.37:6800/2387001309,v1:172.21.15.37:6801/2387001309] conn(0x7fa30c0a2c20 0x7fa30c0a50e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:01.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:01.647+0000 7fa325e2b700 1 --2- 172.21.15.37:0/1045127376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3201085d0 0x7fa320198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:01.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:01.647+0000 7fa325e2b700 1 --2- 172.21.15.37:0/1045127376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa320102530 0x7fa32019e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:01.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:01.647+0000 7fa325e2b700 1 --2- 172.21.15.37:0/1045127376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa320102ef0 0x7fa32019edc0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-10T01:12:01.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:01.647+0000 7fa325e2b700 1 -- 172.21.15.37:0/1045127376 >> 172.21.15.37:0/1045127376 conn(0x7fa320079640 msgr2=0x7fa320101010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-10T01:12:01.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:01.648+0000 7fa325e2b700 1 -- 172.21.15.37:0/1045127376 shutdown_connections 2024-06-10T01:12:01.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-06-10T01:12:01.648+0000 7fa325e2b700 1 -- 172.21.15.37:0/1045127376 wait complete. 2024-06-10T01:12:01.647 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data_ec' removed 2024-06-10T01:12:02.390 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:12:02 smithi037 ceph-mon[32192]: pgmap v898: 33 pgs: 33 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:12:02.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:12:02 smithi037 ceph-mon[32192]: from='client.? 172.21.15.37:0/1045127376' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T01:12:02.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:12:02 smithi037 ceph-mon[32192]: osdmap e109: 12 total, 12 up, 12 in 2024-06-10T01:12:02.451 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-10T01:12:02.480 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-10T01:12:02.503 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:12:02 smithi094 ceph-mon[36708]: pgmap v898: 33 pgs: 33 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:12:02.503 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:12:02 smithi094 ceph-mon[36708]: from='client.? 172.21.15.37:0/1045127376' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T01:12:02.503 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:12:02 smithi094 ceph-mon[36708]: osdmap e109: 12 total, 12 up, 12 in 2024-06-10T01:12:02.508 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-10T01:12:02.530 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:12:02 smithi167 ceph-mon[36917]: pgmap v898: 33 pgs: 33 active+clean; 577 KiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-10T01:12:02.531 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:12:02 smithi167 ceph-mon[36917]: from='client.? 172.21.15.37:0/1045127376' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-10T01:12:02.531 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:12:02 smithi167 ceph-mon[36917]: osdmap e109: 12 total, 12 up, 12 in 2024-06-10T01:12:02.831 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-06-10T01:12:02.832 DEBUG:teuthology.orchestra.run.smithi037:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-10T01:12:02.853 DEBUG:teuthology.orchestra.run.smithi094:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-10T01:12:02.868 DEBUG:teuthology.orchestra.run.smithi167:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-10T01:12:02.886 INFO:tasks.cephadm:Stopping all daemons... 2024-06-10T01:12:02.887 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-06-10T01:12:02.887 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@mon.a 2024-06-10T01:12:03.251 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:12:02 smithi037 systemd[1]: Stopping Ceph mon.a for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:12:03.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:12:03 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-a[32187]: 2024-06-10T01:12:03.255+0000 7fd24fde7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:12:03.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:12:03 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-a[32187]: 2024-06-10T01:12:03.255+0000 7fd24fde7700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-06-10T01:12:03.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:12:03 smithi037 podman[221913]: 2024-06-10 01:12:03.43072598 +0000 UTC m=+0.286281131 container died b72c961cf1e0d476a1b01095215bfac013f9985e0afb1213b4b8989ac35eb135 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-a, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-06-10T01:12:04.140 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:12:03 smithi037 podman[221913]: 2024-06-10 01:12:03.823549286 +0000 UTC m=+0.679104482 container cleanup b72c961cf1e0d476a1b01095215bfac013f9985e0afb1213b4b8989ac35eb135 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-a, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-06-10T01:12:04.140 INFO:journalctl@ceph.mon.a.smithi037.stdout:Jun 10 01:12:03 smithi037 bash[221913]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-a 2024-06-10T01:12:04.280 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mon.a.service' 2024-06-10T01:12:04.328 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:12:04.328 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-06-10T01:12:04.329 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-06-10T01:12:04.329 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@mon.b 2024-06-10T01:12:04.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:12:04 smithi094 systemd[1]: Stopping Ceph mon.b for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:12:05.109 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:12:04 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-b[36704]: 2024-06-10T01:12:04.662+0000 7fc4397e0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:12:05.109 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:12:04 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-b[36704]: 2024-06-10T01:12:04.662+0000 7fc4397e0700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-06-10T01:12:05.109 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:12:04 smithi094 podman[72394]: 2024-06-10 01:12:04.867944967 +0000 UTC m=+0.334133809 container died 17bea145dd2ed513c1ed172bbd06da593f2a3f5d0fe606e50a2fb19989eabc7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240528) 2024-06-10T01:12:05.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:12:05 smithi094 podman[72394]: 2024-06-10 01:12:05.272907822 +0000 UTC m=+0.739096679 container cleanup 17bea145dd2ed513c1ed172bbd06da593f2a3f5d0fe606e50a2fb19989eabc7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-b, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528) 2024-06-10T01:12:05.527 INFO:journalctl@ceph.mon.b.smithi094.stdout:Jun 10 01:12:05 smithi094 bash[72394]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-b 2024-06-10T01:12:05.687 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mon.b.service' 2024-06-10T01:12:05.733 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:12:05.734 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-06-10T01:12:05.734 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-06-10T01:12:05.734 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@mon.c 2024-06-10T01:12:06.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:12:05 smithi167 systemd[1]: Stopping Ceph mon.c for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:12:06.333 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:12:06 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-c[36913]: 2024-06-10T01:12:06.078+0000 7f1d35362700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:12:06.333 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:12:06 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-c[36913]: 2024-06-10T01:12:06.078+0000 7f1d35362700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-06-10T01:12:06.334 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:12:06 smithi167 podman[73908]: 2024-06-10 01:12:06.099746989 +0000 UTC m=+0.139208193 container died 97970bf31e2765491684ae603a65dff70862fb7fade8e8d0a27eba77e93a2b1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-c, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux ) 2024-06-10T01:12:06.768 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:12:06 smithi167 podman[73908]: 2024-06-10 01:12:06.496717718 +0000 UTC m=+0.536178921 container cleanup 97970bf31e2765491684ae603a65dff70862fb7fade8e8d0a27eba77e93a2b1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-c, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2) 2024-06-10T01:12:06.768 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jun 10 01:12:06 smithi167 bash[73908]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-mon-c 2024-06-10T01:12:06.921 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mon.c.service' 2024-06-10T01:12:06.970 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:12:06.971 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-06-10T01:12:06.971 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-06-10T01:12:06.971 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@mgr.x 2024-06-10T01:12:07.306 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 01:12:07 smithi037 systemd[1]: Stopping Ceph mgr.x for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:12:07.624 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 01:12:07 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T01:12:07.311+0000 7fcbc431e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:12:07.625 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 01:12:07 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x[32423]: 2024-06-10T01:12:07.311+0000 7fcbc431e700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-06-10T01:12:07.625 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 01:12:07 smithi037 podman[222033]: 2024-06-10 01:12:07.342813154 +0000 UTC m=+0.156138173 container died 8038bdd9f3fd672ed68a5b12abe68d9eda1020817d5aaa89eb0e3d36c41062c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-06-10T01:12:07.890 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 01:12:07 smithi037 podman[222033]: 2024-06-10 01:12:07.821518087 +0000 UTC m=+0.634843143 container cleanup 8038bdd9f3fd672ed68a5b12abe68d9eda1020817d5aaa89eb0e3d36c41062c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5) 2024-06-10T01:12:07.890 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 01:12:07 smithi037 bash[222033]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x 2024-06-10T01:12:08.268 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mgr.x.service' 2024-06-10T01:12:08.293 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 01:12:08 smithi037 podman[222045]: 2024-06-10 01:12:08.048402981 +0000 UTC m=+0.702341888 container remove 8038bdd9f3fd672ed68a5b12abe68d9eda1020817d5aaa89eb0e3d36c41062c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T01:12:08.293 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 01:12:08 smithi037 systemd[1]: ceph-68793514-26c2-11ef-bc9b-c7b262605968@mgr.x.service: Deactivated successfully. 2024-06-10T01:12:08.294 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 01:12:08 smithi037 systemd[1]: Stopped Ceph mgr.x for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T01:12:08.294 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Jun 10 01:12:08 smithi037 systemd[1]: ceph-68793514-26c2-11ef-bc9b-c7b262605968@mgr.x.service: Consumed 1min 29.407s CPU time. 2024-06-10T01:12:08.305 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:12:08.305 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-06-10T01:12:08.306 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-06-10T01:12:08.306 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@mgr.y 2024-06-10T01:12:08.608 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 01:12:08 smithi094 systemd[1]: Stopping Ceph mgr.y for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:12:08.920 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 01:12:08 smithi094 podman[72516]: 2024-06-10 01:12:08.673997009 +0000 UTC m=+0.158183094 container died 166781b6b467f5fbf396e4ad2c29e72e70c7cf8c2e3fa8b6f169ed780d6b7c52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-06-10T01:12:09.180 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 01:12:09 smithi094 podman[72516]: 2024-06-10 01:12:09.083331124 +0000 UTC m=+0.567517217 container cleanup 166781b6b467f5fbf396e4ad2c29e72e70c7cf8c2e3fa8b6f169ed780d6b7c52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-3d89acc) 2024-06-10T01:12:09.181 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 01:12:09 smithi094 bash[72516]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y 2024-06-10T01:12:09.454 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 01:12:09 smithi094 podman[72530]: 2024-06-10 01:12:09.331306443 +0000 UTC m=+0.649300782 container remove 166781b6b467f5fbf396e4ad2c29e72e70c7cf8c2e3fa8b6f169ed780d6b7c52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T01:12:09.454 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Jun 10 01:12:09 smithi094 systemd[1]: ceph-68793514-26c2-11ef-bc9b-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-06-10T01:12:09.468 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mgr.y.service' 2024-06-10T01:12:09.502 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:12:09.503 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-06-10T01:12:09.503 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-06-10T01:12:09.503 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@mgr.z 2024-06-10T01:12:09.833 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 01:12:09 smithi167 systemd[1]: Stopping Ceph mgr.z for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:12:10.230 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 01:12:09 smithi167 podman[74032]: 2024-06-10 01:12:09.868500061 +0000 UTC m=+0.166586242 container died fbbed67eb8043fdc5144732a1a4db9b47c44ed66e7ce9ef1d0e5206406b0ab9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, io.buildah.version=1.33.5) 2024-06-10T01:12:10.583 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 01:12:10 smithi167 podman[74032]: 2024-06-10 01:12:10.344895035 +0000 UTC m=+0.642981212 container cleanup fbbed67eb8043fdc5144732a1a4db9b47c44ed66e7ce9ef1d0e5206406b0ab9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T01:12:10.583 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jun 10 01:12:10 smithi167 bash[74032]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-mgr-z 2024-06-10T01:12:10.848 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mgr.z.service' 2024-06-10T01:12:10.884 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:12:10.885 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-06-10T01:12:10.885 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-06-10T01:12:10.885 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.0 2024-06-10T01:12:11.171 INFO:journalctl@ceph.osd.0.smithi037.stdout:Jun 10 01:12:10 smithi037 systemd[1]: Stopping Ceph osd.0 for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:12:11.640 INFO:journalctl@ceph.osd.0.smithi037.stdout:Jun 10 01:12:11 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-0[44549]: 2024-06-10T01:12:11.176+0000 7fdd77a3c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:12:11.640 INFO:journalctl@ceph.osd.0.smithi037.stdout:Jun 10 01:12:11 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-0[44549]: 2024-06-10T01:12:11.176+0000 7fdd77a3c700 -1 osd.0 109 *** Got signal Terminated *** 2024-06-10T01:12:11.641 INFO:journalctl@ceph.osd.0.smithi037.stdout:Jun 10 01:12:11 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-0[44549]: 2024-06-10T01:12:11.176+0000 7fdd77a3c700 -1 osd.0 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T01:12:17.046 INFO:journalctl@ceph.osd.0.smithi037.stdout:Jun 10 01:12:16 smithi037 podman[222154]: 2024-06-10 01:12:16.746569664 +0000 UTC m=+5.657672197 container died b53f47f0c6fa4351369a00a8da634310a0e5e6f2d7539a6e7929a33097991259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-06-10T01:12:17.390 INFO:journalctl@ceph.osd.0.smithi037.stdout:Jun 10 01:12:17 smithi037 podman[222154]: 2024-06-10 01:12:17.277716678 +0000 UTC m=+6.188819208 container cleanup b53f47f0c6fa4351369a00a8da634310a0e5e6f2d7539a6e7929a33097991259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, ceph=True) 2024-06-10T01:12:17.390 INFO:journalctl@ceph.osd.0.smithi037.stdout:Jun 10 01:12:17 smithi037 bash[222154]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-0 2024-06-10T01:12:17.890 INFO:journalctl@ceph.osd.0.smithi037.stdout:Jun 10 01:12:17 smithi037 podman[222171]: 2024-06-10 01:12:17.529485233 +0000 UTC m=+0.781985295 container remove b53f47f0c6fa4351369a00a8da634310a0e5e6f2d7539a6e7929a33097991259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc) 2024-06-10T01:12:18.244 INFO:journalctl@ceph.osd.0.smithi037.stdout:Jun 10 01:12:17 smithi037 podman[222259]: 2024-06-10 01:12:17.867410121 +0000 UTC m=+0.018394532 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:12:18.245 INFO:journalctl@ceph.osd.0.smithi037.stdout:Jun 10 01:12:17 smithi037 podman[222259]: 2024-06-10 01:12:17.980487584 +0000 UTC m=+0.131471982 container create 430a3e4eb977efb9bc3a4ed149b66592af0b4d5cccea9809d62c12a142673fb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528) 2024-06-10T01:12:18.640 INFO:journalctl@ceph.osd.0.smithi037.stdout:Jun 10 01:12:18 smithi037 podman[222259]: 2024-06-10 01:12:18.250510151 +0000 UTC m=+0.401494564 container init 430a3e4eb977efb9bc3a4ed149b66592af0b4d5cccea9809d62c12a142673fb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240528, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T01:12:18.640 INFO:journalctl@ceph.osd.0.smithi037.stdout:Jun 10 01:12:18 smithi037 podman[222259]: 2024-06-10 01:12:18.254459625 +0000 UTC m=+0.405444029 container start 430a3e4eb977efb9bc3a4ed149b66592af0b4d5cccea9809d62c12a142673fb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T01:12:18.640 INFO:journalctl@ceph.osd.0.smithi037.stdout:Jun 10 01:12:18 smithi037 podman[222259]: 2024-06-10 01:12:18.425483277 +0000 UTC m=+0.576467691 container attach 430a3e4eb977efb9bc3a4ed149b66592af0b4d5cccea9809d62c12a142673fb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-0-deactivate, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-10T01:12:20.390 INFO:journalctl@ceph.osd.0.smithi037.stdout:Jun 10 01:12:20 smithi037 conmon[222274]: conmon 430a3e4eb977efb9bc3a : Failed to open cgroups file: /sys/fs/cgroup/machine.slice/libpod-430a3e4eb977efb9bc3a4ed149b66592af0b4d5cccea9809d62c12a142673fb4.scope/container/memory.events 2024-06-10T01:12:20.391 INFO:journalctl@ceph.osd.0.smithi037.stdout:Jun 10 01:12:20 smithi037 podman[222259]: 2024-06-10 01:12:20.062964424 +0000 UTC m=+2.213948830 container died 430a3e4eb977efb9bc3a4ed149b66592af0b4d5cccea9809d62c12a142673fb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-06-10T01:12:20.820 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.0.service' 2024-06-10T01:12:20.861 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:12:20.861 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-06-10T01:12:20.861 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-06-10T01:12:20.861 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.1 2024-06-10T01:12:21.187 INFO:journalctl@ceph.osd.1.smithi094.stdout:Jun 10 01:12:20 smithi094 systemd[1]: Stopping Ceph osd.1 for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:12:21.609 INFO:journalctl@ceph.osd.1.smithi094.stdout:Jun 10 01:12:21 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-1[41257]: 2024-06-10T01:12:21.192+0000 7f828fd17700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:12:21.609 INFO:journalctl@ceph.osd.1.smithi094.stdout:Jun 10 01:12:21 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-1[41257]: 2024-06-10T01:12:21.192+0000 7f828fd17700 -1 osd.1 109 *** Got signal Terminated *** 2024-06-10T01:12:21.609 INFO:journalctl@ceph.osd.1.smithi094.stdout:Jun 10 01:12:21 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-1[41257]: 2024-06-10T01:12:21.192+0000 7f828fd17700 -1 osd.1 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T01:12:26.846 INFO:journalctl@ceph.osd.1.smithi094.stdout:Jun 10 01:12:26 smithi094 podman[72651]: 2024-06-10 01:12:26.546658575 +0000 UTC m=+5.478999161 container died c9a33d71ab043842ccff6925df961b1d67a91cfe5350280bde634e9abe041416 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-1, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T01:12:27.108 INFO:journalctl@ceph.osd.1.smithi094.stdout:Jun 10 01:12:26 smithi094 podman[72651]: 2024-06-10 01:12:26.943760549 +0000 UTC m=+5.876101131 container cleanup c9a33d71ab043842ccff6925df961b1d67a91cfe5350280bde634e9abe041416 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-1, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-06-10T01:12:27.109 INFO:journalctl@ceph.osd.1.smithi094.stdout:Jun 10 01:12:26 smithi094 bash[72651]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-1 2024-06-10T01:12:27.564 INFO:journalctl@ceph.osd.1.smithi094.stdout:Jun 10 01:12:27 smithi094 podman[72669]: 2024-06-10 01:12:27.212080595 +0000 UTC m=+0.664535211 container remove c9a33d71ab043842ccff6925df961b1d67a91cfe5350280bde634e9abe041416 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-1, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T01:12:27.844 INFO:journalctl@ceph.osd.1.smithi094.stdout:Jun 10 01:12:27 smithi094 podman[72760]: 2024-06-10 01:12:27.473303832 +0000 UTC m=+0.016934817 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:12:27.844 INFO:journalctl@ceph.osd.1.smithi094.stdout:Jun 10 01:12:27 smithi094 podman[72760]: 2024-06-10 01:12:27.57144349 +0000 UTC m=+0.115074520 container create 39037cfac8fe9866bc3043b5a75cb6c334febf531600689de4303d38c6ee5e16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc) 2024-06-10T01:12:28.109 INFO:journalctl@ceph.osd.1.smithi094.stdout:Jun 10 01:12:27 smithi094 podman[72760]: 2024-06-10 01:12:27.849726879 +0000 UTC m=+0.393357853 container init 39037cfac8fe9866bc3043b5a75cb6c334febf531600689de4303d38c6ee5e16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528) 2024-06-10T01:12:28.109 INFO:journalctl@ceph.osd.1.smithi094.stdout:Jun 10 01:12:27 smithi094 podman[72760]: 2024-06-10 01:12:27.853539232 +0000 UTC m=+0.397170213 container start 39037cfac8fe9866bc3043b5a75cb6c334febf531600689de4303d38c6ee5e16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-06-10T01:12:28.109 INFO:journalctl@ceph.osd.1.smithi094.stdout:Jun 10 01:12:27 smithi094 podman[72760]: 2024-06-10 01:12:27.949539325 +0000 UTC m=+0.493170304 container attach 39037cfac8fe9866bc3043b5a75cb6c334febf531600689de4303d38c6ee5e16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240528, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T01:12:28.413 INFO:journalctl@ceph.osd.1.smithi094.stdout:Jun 10 01:12:28 smithi094 podman[72760]: 2024-06-10 01:12:28.144535487 +0000 UTC m=+0.688166459 container died 39037cfac8fe9866bc3043b5a75cb6c334febf531600689de4303d38c6ee5e16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-06-10T01:12:28.778 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.1.service' 2024-06-10T01:12:28.815 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:12:28.816 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-06-10T01:12:28.816 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-06-10T01:12:28.816 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.2 2024-06-10T01:12:29.144 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 10 01:12:28 smithi167 systemd[1]: Stopping Ceph osd.2 for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:12:29.583 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 10 01:12:29 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-2[42020]: 2024-06-10T01:12:29.149+0000 7f6482d6b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:12:29.583 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 10 01:12:29 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-2[42020]: 2024-06-10T01:12:29.149+0000 7f6482d6b700 -1 osd.2 109 *** Got signal Terminated *** 2024-06-10T01:12:29.583 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 10 01:12:29 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-2[42020]: 2024-06-10T01:12:29.149+0000 7f6482d6b700 -1 osd.2 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T01:12:34.996 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 10 01:12:34 smithi167 podman[74168]: 2024-06-10 01:12:34.71129082 +0000 UTC m=+5.695295530 container died d35e34d3bfbf4bd8fa8d71b32b2ecf9576a33e080dabc109f6d0fc17db2c6a0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, ceph=True) 2024-06-10T01:12:35.333 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 10 01:12:35 smithi167 podman[74168]: 2024-06-10 01:12:35.100719106 +0000 UTC m=+6.084723838 container cleanup d35e34d3bfbf4bd8fa8d71b32b2ecf9576a33e080dabc109f6d0fc17db2c6a0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T01:12:35.333 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 10 01:12:35 smithi167 bash[74168]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-2 2024-06-10T01:12:35.719 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 10 01:12:35 smithi167 podman[74185]: 2024-06-10 01:12:35.41106958 +0000 UTC m=+0.696049689 container remove d35e34d3bfbf4bd8fa8d71b32b2ecf9576a33e080dabc109f6d0fc17db2c6a0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, ceph=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-10T01:12:35.984 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 10 01:12:35 smithi167 podman[74273]: 2024-06-10 01:12:35.627982208 +0000 UTC m=+0.017275217 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:12:35.985 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 10 01:12:35 smithi167 podman[74273]: 2024-06-10 01:12:35.762077558 +0000 UTC m=+0.151370544 container create 877ac5544053348e2a02d167c2e85a0f91b04e1da7aa424edeefca578883c3ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, ceph=True, org.label-schema.license=GPLv2) 2024-06-10T01:12:36.333 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 10 01:12:35 smithi167 podman[74273]: 2024-06-10 01:12:35.990472152 +0000 UTC m=+0.379765150 container init 877ac5544053348e2a02d167c2e85a0f91b04e1da7aa424edeefca578883c3ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-2-deactivate, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux ) 2024-06-10T01:12:36.333 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 10 01:12:35 smithi167 podman[74273]: 2024-06-10 01:12:35.994246845 +0000 UTC m=+0.383539840 container start 877ac5544053348e2a02d167c2e85a0f91b04e1da7aa424edeefca578883c3ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528) 2024-06-10T01:12:36.334 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 10 01:12:36 smithi167 podman[74273]: 2024-06-10 01:12:36.132086652 +0000 UTC m=+0.521379664 container attach 877ac5544053348e2a02d167c2e85a0f91b04e1da7aa424edeefca578883c3ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2) 2024-06-10T01:12:36.726 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jun 10 01:12:36 smithi167 podman[74273]: 2024-06-10 01:12:36.45708639 +0000 UTC m=+0.846379386 container died 877ac5544053348e2a02d167c2e85a0f91b04e1da7aa424edeefca578883c3ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-2-deactivate, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc) 2024-06-10T01:12:37.152 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.2.service' 2024-06-10T01:12:37.190 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:12:37.190 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-06-10T01:12:37.191 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-06-10T01:12:37.191 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.3 2024-06-10T01:12:37.536 INFO:journalctl@ceph.osd.3.smithi037.stdout:Jun 10 01:12:37 smithi037 systemd[1]: Stopping Ceph osd.3 for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:12:37.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:37 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:37.330+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:12:37.890 INFO:journalctl@ceph.osd.3.smithi037.stdout:Jun 10 01:12:37 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-3[51301]: 2024-06-10T01:12:37.541+0000 7f33540a8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:12:37.890 INFO:journalctl@ceph.osd.3.smithi037.stdout:Jun 10 01:12:37 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-3[51301]: 2024-06-10T01:12:37.541+0000 7f33540a8700 -1 osd.3 109 *** Got signal Terminated *** 2024-06-10T01:12:37.890 INFO:journalctl@ceph.osd.3.smithi037.stdout:Jun 10 01:12:37 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-3[51301]: 2024-06-10T01:12:37.541+0000 7f33540a8700 -1 osd.3 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T01:12:37.999 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:37 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:37.778+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:12:38.335 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:38 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:38.004+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:12:38.608 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:38 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:38.356+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:12:39.040 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:38 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:38.760+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:12:39.041 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:38 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:38.635+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.172216+0000 front 2024-06-10T01:12:14.172227+0000 (oldest deadline 2024-06-10T01:12:37.672261+0000) 2024-06-10T01:12:39.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:39 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:39.045+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:12:39.608 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:39 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:39.329+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:12:40.052 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:39 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:39.776+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:12:40.053 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:39 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:39.676+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.172216+0000 front 2024-06-10T01:12:14.172227+0000 (oldest deadline 2024-06-10T01:12:37.672261+0000) 2024-06-10T01:12:40.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:40 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:40.071+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:12:40.390 INFO:journalctl@ceph.osd.3.smithi037.stdout:Jun 10 01:12:40 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-3[51301]: 2024-06-10T01:12:40.117+0000 7f334c694700 -1 osd.3 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.200033+0000 front 2024-06-10T01:12:14.200239+0000 (oldest deadline 2024-06-10T01:12:39.400737+0000) 2024-06-10T01:12:40.608 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:40 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:40.332+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:12:40.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:40 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:40.621+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:12:41.076 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:40 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:40.708+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.172216+0000 front 2024-06-10T01:12:14.172227+0000 (oldest deadline 2024-06-10T01:12:37.672261+0000) 2024-06-10T01:12:41.076 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:40 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:40.825+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:12:41.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:41 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:41.080+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:12:41.390 INFO:journalctl@ceph.osd.3.smithi037.stdout:Jun 10 01:12:41 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-3[51301]: 2024-06-10T01:12:41.130+0000 7f334c694700 -1 osd.3 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.200033+0000 front 2024-06-10T01:12:14.200239+0000 (oldest deadline 2024-06-10T01:12:39.400737+0000) 2024-06-10T01:12:41.608 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:41 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:41.351+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:12:41.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:41 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:41.614+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:12:42.026 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:41 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:41.698+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.172216+0000 front 2024-06-10T01:12:14.172227+0000 (oldest deadline 2024-06-10T01:12:37.672261+0000) 2024-06-10T01:12:42.027 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:41 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:41.780+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:12:42.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:42 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:42.031+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:12:42.390 INFO:journalctl@ceph.osd.3.smithi037.stdout:Jun 10 01:12:42 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-3[51301]: 2024-06-10T01:12:42.125+0000 7f334c694700 -1 osd.3 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.200033+0000 front 2024-06-10T01:12:14.200239+0000 (oldest deadline 2024-06-10T01:12:39.400737+0000) 2024-06-10T01:12:42.608 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:42 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:42.304+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:12:42.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:42 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:42.589+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:12:43.037 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:42 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:42.710+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.172216+0000 front 2024-06-10T01:12:14.172227+0000 (oldest deadline 2024-06-10T01:12:37.672261+0000) 2024-06-10T01:12:43.038 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:42 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:42.744+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:12:43.221 INFO:journalctl@ceph.osd.3.smithi037.stdout:Jun 10 01:12:42 smithi037 podman[222377]: 2024-06-10 01:12:42.950941436 +0000 UTC m=+5.543394941 container died 3d40a4974a8a05261168c1510353f4368192fc22364284aad15c2ffd30acb74b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-3, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-10T01:12:43.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:43 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:43.042+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:12:43.601 INFO:journalctl@ceph.osd.3.smithi037.stdout:Jun 10 01:12:43 smithi037 podman[222377]: 2024-06-10 01:12:43.416606592 +0000 UTC m=+6.009060090 container cleanup 3d40a4974a8a05261168c1510353f4368192fc22364284aad15c2ffd30acb74b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T01:12:43.602 INFO:journalctl@ceph.osd.3.smithi037.stdout:Jun 10 01:12:43 smithi037 bash[222377]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-3 2024-06-10T01:12:43.608 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:43 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:43.265+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:12:43.890 INFO:journalctl@ceph.osd.3.smithi037.stdout:Jun 10 01:12:43 smithi037 podman[222394]: 2024-06-10 01:12:43.72652981 +0000 UTC m=+0.771365468 container remove 3d40a4974a8a05261168c1510353f4368192fc22364284aad15c2ffd30acb74b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-3, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-06-10T01:12:43.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:43 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:43.606+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:12:43.994 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:43 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:43.733+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.172216+0000 front 2024-06-10T01:12:14.172227+0000 (oldest deadline 2024-06-10T01:12:37.672261+0000) 2024-06-10T01:12:43.994 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:43 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:43.732+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:12:44.324 INFO:journalctl@ceph.osd.3.smithi037.stdout:Jun 10 01:12:44 smithi037 podman[222483]: 2024-06-10 01:12:43.958691477 +0000 UTC m=+0.018055216 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:12:44.324 INFO:journalctl@ceph.osd.3.smithi037.stdout:Jun 10 01:12:44 smithi037 podman[222483]: 2024-06-10 01:12:44.077563834 +0000 UTC m=+0.136927572 container create b1a4ad0cf751edb065ff34ac5b6f8cc0d69de2f70ac93c2c30750878800b4992 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-3-deactivate, ceph=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-06-10T01:12:44.332 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:44 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:43.999+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:12:44.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:44 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:44.222+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:12:44.636 INFO:journalctl@ceph.osd.3.smithi037.stdout:Jun 10 01:12:44 smithi037 podman[222483]: 2024-06-10 01:12:44.330377864 +0000 UTC m=+0.389741571 container init b1a4ad0cf751edb065ff34ac5b6f8cc0d69de2f70ac93c2c30750878800b4992 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD) 2024-06-10T01:12:44.637 INFO:journalctl@ceph.osd.3.smithi037.stdout:Jun 10 01:12:44 smithi037 podman[222483]: 2024-06-10 01:12:44.333931494 +0000 UTC m=+0.393295221 container start b1a4ad0cf751edb065ff34ac5b6f8cc0d69de2f70ac93c2c30750878800b4992 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T01:12:44.637 INFO:journalctl@ceph.osd.3.smithi037.stdout:Jun 10 01:12:44 smithi037 podman[222483]: 2024-06-10 01:12:44.447123068 +0000 UTC m=+0.506486785 container attach b1a4ad0cf751edb065ff34ac5b6f8cc0d69de2f70ac93c2c30750878800b4992 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240528, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T01:12:44.890 INFO:journalctl@ceph.osd.3.smithi037.stdout:Jun 10 01:12:44 smithi037 podman[222483]: 2024-06-10 01:12:44.647080339 +0000 UTC m=+0.706444064 container died b1a4ad0cf751edb065ff34ac5b6f8cc0d69de2f70ac93c2c30750878800b4992 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-3-deactivate, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-06-10T01:12:44.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:44 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:44.642+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:12:45.083 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:44 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:44.739+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.172216+0000 front 2024-06-10T01:12:14.172227+0000 (oldest deadline 2024-06-10T01:12:37.672261+0000) 2024-06-10T01:12:45.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:44 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:44.775+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:12:45.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:44 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:44.973+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:12:45.220 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.3.service' 2024-06-10T01:12:45.260 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:12:45.260 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-06-10T01:12:45.261 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-06-10T01:12:45.261 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.4 2024-06-10T01:12:45.554 INFO:journalctl@ceph.osd.4.smithi094.stdout:Jun 10 01:12:45 smithi094 systemd[1]: Stopping Ceph osd.4 for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:12:45.554 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:45 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:45.269+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:12:45.858 INFO:journalctl@ceph.osd.4.smithi094.stdout:Jun 10 01:12:45 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-4[48025]: 2024-06-10T01:12:45.559+0000 7f5f39f5e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:12:45.859 INFO:journalctl@ceph.osd.4.smithi094.stdout:Jun 10 01:12:45 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-4[48025]: 2024-06-10T01:12:45.559+0000 7f5f39f5e700 -1 osd.4 109 *** Got signal Terminated *** 2024-06-10T01:12:45.859 INFO:journalctl@ceph.osd.4.smithi094.stdout:Jun 10 01:12:45 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-4[48025]: 2024-06-10T01:12:45.559+0000 7f5f39f5e700 -1 osd.4 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T01:12:46.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:45 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:45.733+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:12:46.083 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:45 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:45.695+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.172216+0000 front 2024-06-10T01:12:14.172227+0000 (oldest deadline 2024-06-10T01:12:37.672261+0000) 2024-06-10T01:12:46.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:45 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:45.937+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:12:46.140 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:45 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:45.679+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:12:46.608 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:46 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:46.314+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:12:46.935 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:46 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:46.728+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.172216+0000 front 2024-06-10T01:12:14.172227+0000 (oldest deadline 2024-06-10T01:12:37.672261+0000) 2024-06-10T01:12:46.935 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:46 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:46.684+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:12:47.041 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:46 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:46.708+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:12:47.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:46 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:46.940+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:12:47.608 INFO:journalctl@ceph.osd.4.smithi094.stdout:Jun 10 01:12:47 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-4[48025]: 2024-06-10T01:12:47.574+0000 7f5f36552700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:22.773898+0000 front 2024-06-10T01:12:22.773870+0000 (oldest deadline 2024-06-10T01:12:47.473999+0000) 2024-06-10T01:12:47.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:47 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:47.330+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:12:48.083 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:47 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:47.692+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.172216+0000 front 2024-06-10T01:12:14.172227+0000 (oldest deadline 2024-06-10T01:12:37.672261+0000) 2024-06-10T01:12:48.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:47 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:47.917+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:12:48.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:47 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:47.723+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:12:48.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:47 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:47.723+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:12:48.140 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:47 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:47.738+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:12:48.858 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:48 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:48.378+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:12:48.859 INFO:journalctl@ceph.osd.4.smithi094.stdout:Jun 10 01:12:48 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-4[48025]: 2024-06-10T01:12:48.565+0000 7f5f36552700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:22.773898+0000 front 2024-06-10T01:12:22.773870+0000 (oldest deadline 2024-06-10T01:12:47.473999+0000) 2024-06-10T01:12:49.083 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:48 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:48.691+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.172216+0000 front 2024-06-10T01:12:14.172227+0000 (oldest deadline 2024-06-10T01:12:37.672261+0000) 2024-06-10T01:12:49.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:48 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:48.717+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:12:49.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:48 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:48.717+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:12:49.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:48 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:48.872+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:12:49.140 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:48 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:48.784+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:12:49.555 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:49 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:49.394+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:12:49.858 INFO:journalctl@ceph.osd.4.smithi094.stdout:Jun 10 01:12:49 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-4[48025]: 2024-06-10T01:12:49.560+0000 7f5f36552700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:22.773898+0000 front 2024-06-10T01:12:22.773870+0000 (oldest deadline 2024-06-10T01:12:47.473999+0000) 2024-06-10T01:12:50.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:49 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:49.822+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:12:50.083 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:49 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:49.663+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.172216+0000 front 2024-06-10T01:12:14.172227+0000 (oldest deadline 2024-06-10T01:12:37.672261+0000) 2024-06-10T01:12:50.084 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:49 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:49.663+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.474125+0000 front 2024-06-10T01:12:23.474173+0000 (oldest deadline 2024-06-10T01:12:48.774174+0000) 2024-06-10T01:12:50.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:49 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:49.694+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:12:50.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:49 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:49.694+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:12:50.140 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:49 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:49.783+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:12:50.140 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:49 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:49.783+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:12:50.608 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:50 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:50.434+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:12:50.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:12:50 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:12:50.292+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:24.283412+0000 front 2024-06-10T01:12:24.283445+0000 (oldest deadline 2024-06-10T01:12:50.183651+0000) 2024-06-10T01:12:51.083 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:50 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:50.652+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.172216+0000 front 2024-06-10T01:12:14.172227+0000 (oldest deadline 2024-06-10T01:12:37.672261+0000) 2024-06-10T01:12:51.083 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:50 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:50.652+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.474125+0000 front 2024-06-10T01:12:23.474173+0000 (oldest deadline 2024-06-10T01:12:48.774174+0000) 2024-06-10T01:12:51.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:50 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:50.727+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:12:51.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:50 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:50.727+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:12:51.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:50 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:50.827+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:12:51.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:50 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:50.827+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:12:51.140 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:50 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:50.734+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:12:51.140 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:50 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:50.734+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:12:51.301 INFO:journalctl@ceph.osd.4.smithi094.stdout:Jun 10 01:12:51 smithi094 podman[72883]: 2024-06-10 01:12:51.050473932 +0000 UTC m=+5.588630958 container died af25f95213d272f7006469ebb95999344bcf1423f67df80dcb94449496da1490 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-4, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-06-10T01:12:51.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:12:51 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:12:51.306+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:24.283412+0000 front 2024-06-10T01:12:24.283445+0000 (oldest deadline 2024-06-10T01:12:50.183651+0000) 2024-06-10T01:12:51.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:51 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:51.389+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:12:51.610 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:51 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:51.389+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:12:51.610 INFO:journalctl@ceph.osd.4.smithi094.stdout:Jun 10 01:12:51 smithi094 podman[72883]: 2024-06-10 01:12:51.410206881 +0000 UTC m=+5.948363908 container cleanup af25f95213d272f7006469ebb95999344bcf1423f67df80dcb94449496da1490 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-06-10T01:12:51.611 INFO:journalctl@ceph.osd.4.smithi094.stdout:Jun 10 01:12:51 smithi094 bash[72883]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-4 2024-06-10T01:12:52.027 INFO:journalctl@ceph.osd.4.smithi094.stdout:Jun 10 01:12:51 smithi094 podman[72900]: 2024-06-10 01:12:51.686911762 +0000 UTC m=+0.632128677 container remove af25f95213d272f7006469ebb95999344bcf1423f67df80dcb94449496da1490 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-4, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS) 2024-06-10T01:12:52.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:51 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:51.736+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:12:52.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:51 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:51.736+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:12:52.083 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:51 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:51.687+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.172216+0000 front 2024-06-10T01:12:14.172227+0000 (oldest deadline 2024-06-10T01:12:37.672261+0000) 2024-06-10T01:12:52.084 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:51 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:51.687+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.474125+0000 front 2024-06-10T01:12:23.474173+0000 (oldest deadline 2024-06-10T01:12:48.774174+0000) 2024-06-10T01:12:52.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:51 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:51.861+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:12:52.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:51 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:51.861+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:12:52.140 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:51 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:51.696+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:12:52.141 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:51 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:51.696+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:12:52.141 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:12:51 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:12:51.934+0000 7fc774f40700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.106926+0000 front 2024-06-10T01:12:25.106948+0000 (oldest deadline 2024-06-10T01:12:51.007025+0000) 2024-06-10T01:12:52.326 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:12:52 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:12:52.270+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:24.283412+0000 front 2024-06-10T01:12:24.283445+0000 (oldest deadline 2024-06-10T01:12:50.183651+0000) 2024-06-10T01:12:52.327 INFO:journalctl@ceph.osd.4.smithi094.stdout:Jun 10 01:12:52 smithi094 podman[72989]: 2024-06-10 01:12:51.936078452 +0000 UTC m=+0.016842806 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:12:52.327 INFO:journalctl@ceph.osd.4.smithi094.stdout:Jun 10 01:12:52 smithi094 podman[72989]: 2024-06-10 01:12:52.087446354 +0000 UTC m=+0.168210697 container create e6268210ae7659041d63857a1bbb34e2704e72d18ad7a3ced8cb27b6beacb0f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-06-10T01:12:52.609 INFO:journalctl@ceph.osd.4.smithi094.stdout:Jun 10 01:12:52 smithi094 podman[72989]: 2024-06-10 01:12:52.33246781 +0000 UTC m=+0.413232146 container init e6268210ae7659041d63857a1bbb34e2704e72d18ad7a3ced8cb27b6beacb0f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-06-10T01:12:52.609 INFO:journalctl@ceph.osd.4.smithi094.stdout:Jun 10 01:12:52 smithi094 podman[72989]: 2024-06-10 01:12:52.336136078 +0000 UTC m=+0.416900423 container start e6268210ae7659041d63857a1bbb34e2704e72d18ad7a3ced8cb27b6beacb0f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528) 2024-06-10T01:12:52.609 INFO:journalctl@ceph.osd.4.smithi094.stdout:Jun 10 01:12:52 smithi094 podman[72989]: 2024-06-10 01:12:52.432939607 +0000 UTC m=+0.513703956 container attach e6268210ae7659041d63857a1bbb34e2704e72d18ad7a3ced8cb27b6beacb0f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-06-10T01:12:52.610 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:52 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:52.347+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:12:52.610 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:52 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:52.347+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:12:52.921 INFO:journalctl@ceph.osd.4.smithi094.stdout:Jun 10 01:12:52 smithi094 podman[72989]: 2024-06-10 01:12:52.624696319 +0000 UTC m=+0.705460666 container died e6268210ae7659041d63857a1bbb34e2704e72d18ad7a3ced8cb27b6beacb0f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-06-10T01:12:52.932 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:52 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:52.687+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:12:52.932 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:52 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:52.687+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:12:53.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:52 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:52.694+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:12:53.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:52 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:52.694+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:12:53.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:52 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:52.833+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:12:53.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:52 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:52.833+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:12:53.084 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:52 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:52.663+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.172216+0000 front 2024-06-10T01:12:14.172227+0000 (oldest deadline 2024-06-10T01:12:37.672261+0000) 2024-06-10T01:12:53.084 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:52 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:52.663+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.474125+0000 front 2024-06-10T01:12:23.474173+0000 (oldest deadline 2024-06-10T01:12:48.774174+0000) 2024-06-10T01:12:53.233 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.4.service' 2024-06-10T01:12:53.271 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:12:53.272 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-06-10T01:12:53.272 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-06-10T01:12:53.272 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.5 2024-06-10T01:12:53.390 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:12:52 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:12:52.938+0000 7fc774f40700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.106926+0000 front 2024-06-10T01:12:25.106948+0000 (oldest deadline 2024-06-10T01:12:51.007025+0000) 2024-06-10T01:12:53.583 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:53 smithi167 systemd[1]: Stopping Ceph osd.5 for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:12:53.608 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:12:53 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:12:53.265+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:24.283412+0000 front 2024-06-10T01:12:24.283445+0000 (oldest deadline 2024-06-10T01:12:50.183651+0000) 2024-06-10T01:12:53.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:53 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:53.311+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:12:53.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:53 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:53.311+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:12:54.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:53 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:53.724+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:12:54.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:53 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:53.724+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:12:54.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:53 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:53.844+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:12:54.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:53 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:53.844+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:12:54.084 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:53 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:53.630+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.172216+0000 front 2024-06-10T01:12:14.172227+0000 (oldest deadline 2024-06-10T01:12:37.672261+0000) 2024-06-10T01:12:54.084 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:53 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:53.630+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.474125+0000 front 2024-06-10T01:12:23.474173+0000 (oldest deadline 2024-06-10T01:12:48.774174+0000) 2024-06-10T01:12:54.084 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:53 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:53.731+0000 7f94b3553700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:12:54.084 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:53 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:53.731+0000 7f94b3553700 -1 osd.5 109 *** Got signal Terminated *** 2024-06-10T01:12:54.084 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:53 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:53.731+0000 7f94b3553700 -1 osd.5 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T01:12:54.140 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:12:53 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:12:53.921+0000 7fc774f40700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.106926+0000 front 2024-06-10T01:12:25.106948+0000 (oldest deadline 2024-06-10T01:12:51.007025+0000) 2024-06-10T01:12:54.140 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:53 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:53.702+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:12:54.140 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:53 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:53.702+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:12:54.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:12:54 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:12:54.267+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:24.283412+0000 front 2024-06-10T01:12:24.283445+0000 (oldest deadline 2024-06-10T01:12:50.183651+0000) 2024-06-10T01:12:54.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:54 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:54.323+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:12:54.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:54 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:54.323+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:12:54.878 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:54 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:54.615+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.172216+0000 front 2024-06-10T01:12:14.172227+0000 (oldest deadline 2024-06-10T01:12:37.672261+0000) 2024-06-10T01:12:54.878 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:54 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:54.615+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.474125+0000 front 2024-06-10T01:12:23.474173+0000 (oldest deadline 2024-06-10T01:12:48.774174+0000) 2024-06-10T01:12:54.879 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:54 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:54.762+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:12:54.879 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:54 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:54.762+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:12:55.140 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:12:54 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:12:54.936+0000 7fc774f40700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.106926+0000 front 2024-06-10T01:12:25.106948+0000 (oldest deadline 2024-06-10T01:12:51.007025+0000) 2024-06-10T01:12:55.140 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:54 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:54.730+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:12:55.140 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:54 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:54.730+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:12:55.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:54 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:54.883+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:12:55.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:54 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:54.883+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:12:55.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:12:55 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:12:55.265+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:24.283412+0000 front 2024-06-10T01:12:24.283445+0000 (oldest deadline 2024-06-10T01:12:50.183651+0000) 2024-06-10T01:12:55.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:55 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:55.365+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:12:55.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:55 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:55.365+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:12:56.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:55 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:55.769+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:12:56.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:55 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:55.769+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:12:56.083 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:55 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:55.647+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.172216+0000 front 2024-06-10T01:12:14.172227+0000 (oldest deadline 2024-06-10T01:12:37.672261+0000) 2024-06-10T01:12:56.084 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:55 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:55.647+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.474125+0000 front 2024-06-10T01:12:23.474173+0000 (oldest deadline 2024-06-10T01:12:48.774174+0000) 2024-06-10T01:12:56.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:55 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:55.892+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:12:56.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:55 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:55.892+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:12:56.140 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:12:55 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:12:55.963+0000 7fc774f40700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.106926+0000 front 2024-06-10T01:12:25.106948+0000 (oldest deadline 2024-06-10T01:12:51.007025+0000) 2024-06-10T01:12:56.140 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:55 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:55.737+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:12:56.140 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:55 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:55.737+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:12:56.140 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:55 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:55.737+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:12:56.608 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:12:56 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:12:56.219+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:24.283412+0000 front 2024-06-10T01:12:24.283445+0000 (oldest deadline 2024-06-10T01:12:50.183651+0000) 2024-06-10T01:12:56.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:12:56 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:12:56.219+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.284997+0000 front 2024-06-10T01:12:31.284852+0000 (oldest deadline 2024-06-10T01:12:55.384713+0000) 2024-06-10T01:12:56.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:56 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:56.375+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:12:56.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:56 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:56.375+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:12:56.904 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:56 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:56.649+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.172216+0000 front 2024-06-10T01:12:14.172227+0000 (oldest deadline 2024-06-10T01:12:37.672261+0000) 2024-06-10T01:12:56.904 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:56 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:56.649+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.474125+0000 front 2024-06-10T01:12:23.474173+0000 (oldest deadline 2024-06-10T01:12:48.774174+0000) 2024-06-10T01:12:56.904 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:56 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:56.779+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:12:56.904 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:56 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:56.779+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:12:56.905 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:56 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:56.779+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.475054+0000 front 2024-06-10T01:12:31.475199+0000 (oldest deadline 2024-06-10T01:12:56.175181+0000) 2024-06-10T01:12:57.140 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:12:56 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:12:56.939+0000 7fc774f40700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.106926+0000 front 2024-06-10T01:12:25.106948+0000 (oldest deadline 2024-06-10T01:12:51.007025+0000) 2024-06-10T01:12:57.140 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:12:56 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:12:56.939+0000 7fc774f40700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.007521+0000 front 2024-06-10T01:12:31.007592+0000 (oldest deadline 2024-06-10T01:12:56.907625+0000) 2024-06-10T01:12:57.140 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:56 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:56.696+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:12:57.141 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:56 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:56.696+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:12:57.141 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:56 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:56.696+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:12:57.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:56 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:56.908+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:12:57.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:56 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:56.908+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:12:57.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:56 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:56.908+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:12:57.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:12:57 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:12:57.199+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:24.283412+0000 front 2024-06-10T01:12:24.283445+0000 (oldest deadline 2024-06-10T01:12:50.183651+0000) 2024-06-10T01:12:57.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:12:57 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:12:57.199+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.284997+0000 front 2024-06-10T01:12:31.284852+0000 (oldest deadline 2024-06-10T01:12:55.384713+0000) 2024-06-10T01:12:57.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:57 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:57.335+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:12:57.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:57 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:57.335+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:12:57.916 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:57 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:57.662+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:12:57.916 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:57 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:57.662+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:12:57.916 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:57 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:57.662+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:12:57.946 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:57 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:57.687+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.172216+0000 front 2024-06-10T01:12:14.172227+0000 (oldest deadline 2024-06-10T01:12:37.672261+0000) 2024-06-10T01:12:57.946 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:57 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:57.687+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.474125+0000 front 2024-06-10T01:12:23.474173+0000 (oldest deadline 2024-06-10T01:12:48.774174+0000) 2024-06-10T01:12:57.946 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:57 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:57.687+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:32.175849+0000 front 2024-06-10T01:12:32.175902+0000 (oldest deadline 2024-06-10T01:12:56.876045+0000) 2024-06-10T01:12:57.947 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:57 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:57.743+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:12:57.947 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:57 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:57.743+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:12:57.947 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:57 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:57.743+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.475054+0000 front 2024-06-10T01:12:31.475199+0000 (oldest deadline 2024-06-10T01:12:56.175181+0000) 2024-06-10T01:12:58.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:57 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:57.951+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:12:58.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:57 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:57.951+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:12:58.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:57 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:57.951+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:12:58.384 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:12:57 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:12:57.921+0000 7fc774f40700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.106926+0000 front 2024-06-10T01:12:25.106948+0000 (oldest deadline 2024-06-10T01:12:51.007025+0000) 2024-06-10T01:12:58.385 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:12:57 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:12:57.921+0000 7fc774f40700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.007521+0000 front 2024-06-10T01:12:31.007592+0000 (oldest deadline 2024-06-10T01:12:56.907625+0000) 2024-06-10T01:12:58.608 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:12:58 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:12:58.221+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:24.283412+0000 front 2024-06-10T01:12:24.283445+0000 (oldest deadline 2024-06-10T01:12:50.183651+0000) 2024-06-10T01:12:58.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:12:58 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:12:58.221+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.284997+0000 front 2024-06-10T01:12:31.284852+0000 (oldest deadline 2024-06-10T01:12:55.384713+0000) 2024-06-10T01:12:58.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:58 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:58.286+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:12:58.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:58 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:58.286+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:12:58.640 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:58 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:58.624+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:12:58.640 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:58 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:58.624+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:12:58.640 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:58 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:58.624+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:12:59.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:58 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:58.907+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:12:59.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:58 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:58.907+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:12:59.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:58 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:58.907+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:12:59.084 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:58 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:58.688+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.172216+0000 front 2024-06-10T01:12:14.172227+0000 (oldest deadline 2024-06-10T01:12:37.672261+0000) 2024-06-10T01:12:59.084 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:58 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:58.689+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.474125+0000 front 2024-06-10T01:12:23.474173+0000 (oldest deadline 2024-06-10T01:12:48.774174+0000) 2024-06-10T01:12:59.084 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:58 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5[48964]: 2024-06-10T01:12:58.689+0000 7f94afb47700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:32.175849+0000 front 2024-06-10T01:12:32.175902+0000 (oldest deadline 2024-06-10T01:12:56.876045+0000) 2024-06-10T01:12:59.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:58 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:58.708+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:12:59.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:58 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:58.708+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:12:59.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:58 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:58.708+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.475054+0000 front 2024-06-10T01:12:31.475199+0000 (oldest deadline 2024-06-10T01:12:56.175181+0000) 2024-06-10T01:12:59.140 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:12:58 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:12:58.873+0000 7fc774f40700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.106926+0000 front 2024-06-10T01:12:25.106948+0000 (oldest deadline 2024-06-10T01:12:51.007025+0000) 2024-06-10T01:12:59.140 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:12:58 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:12:58.873+0000 7fc774f40700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.007521+0000 front 2024-06-10T01:12:31.007592+0000 (oldest deadline 2024-06-10T01:12:56.907625+0000) 2024-06-10T01:12:59.410 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:59 smithi167 podman[74390]: 2024-06-10 01:12:59.119231782 +0000 UTC m=+5.639726978 container died 328b54319753320fc50f4036055bb9b5144fcf3fd974ad9494e980dc6dba8c85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0) 2024-06-10T01:12:59.608 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:12:59 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:12:59.232+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:24.283412+0000 front 2024-06-10T01:12:24.283445+0000 (oldest deadline 2024-06-10T01:12:50.183651+0000) 2024-06-10T01:12:59.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:12:59 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:12:59.232+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.284997+0000 front 2024-06-10T01:12:31.284852+0000 (oldest deadline 2024-06-10T01:12:55.384713+0000) 2024-06-10T01:12:59.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:59 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:59.328+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:12:59.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:12:59 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:12:59.328+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:12:59.769 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:59 smithi167 podman[74390]: 2024-06-10 01:12:59.507250963 +0000 UTC m=+6.027746151 container cleanup 328b54319753320fc50f4036055bb9b5144fcf3fd974ad9494e980dc6dba8c85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-10T01:12:59.770 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:59 smithi167 bash[74390]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5 2024-06-10T01:12:59.770 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:59 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:59.664+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:12:59.770 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:59 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:59.664+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:12:59.771 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:12:59 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:12:59.664+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.475054+0000 front 2024-06-10T01:12:31.475199+0000 (oldest deadline 2024-06-10T01:12:56.175181+0000) 2024-06-10T01:12:59.889 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:59 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:59.615+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:12:59.889 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:59 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:59.615+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:12:59.889 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:12:59 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:12:59.615+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:00.083 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:12:59 smithi167 podman[74408]: 2024-06-10 01:12:59.775599294 +0000 UTC m=+0.651957262 container remove 328b54319753320fc50f4036055bb9b5144fcf3fd974ad9494e980dc6dba8c85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-06-10T01:13:00.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:59 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:59.859+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:00.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:59 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:59.859+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:00.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:12:59 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:12:59.859+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:00.140 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:12:59 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:12:59.894+0000 7fc774f40700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.106926+0000 front 2024-06-10T01:12:25.106948+0000 (oldest deadline 2024-06-10T01:12:51.007025+0000) 2024-06-10T01:13:00.140 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:12:59 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:12:59.894+0000 7fc774f40700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.007521+0000 front 2024-06-10T01:12:31.007592+0000 (oldest deadline 2024-06-10T01:12:56.907625+0000) 2024-06-10T01:13:00.341 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:13:00 smithi167 podman[74497]: 2024-06-10 01:12:59.994881064 +0000 UTC m=+0.017990701 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:13:00.341 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:13:00 smithi167 podman[74497]: 2024-06-10 01:13:00.093208598 +0000 UTC m=+0.116318257 container create 1ddf5fa7d164c31b287fba5eb4a629edc8dbe9ed51ae62b0587ba8957f2769c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-06-10T01:13:00.608 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:00 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:00.195+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:24.283412+0000 front 2024-06-10T01:12:24.283445+0000 (oldest deadline 2024-06-10T01:12:50.183651+0000) 2024-06-10T01:13:00.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:00 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:00.195+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.284997+0000 front 2024-06-10T01:12:31.284852+0000 (oldest deadline 2024-06-10T01:12:55.384713+0000) 2024-06-10T01:13:00.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:00 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:00.325+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:00.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:00 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:00.325+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:00.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:00 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:00.325+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:00.640 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:13:00 smithi167 podman[74497]: 2024-06-10 01:13:00.346922998 +0000 UTC m=+0.370032636 container init 1ddf5fa7d164c31b287fba5eb4a629edc8dbe9ed51ae62b0587ba8957f2769c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5-deactivate, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528) 2024-06-10T01:13:00.640 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:13:00 smithi167 podman[74497]: 2024-06-10 01:13:00.350289036 +0000 UTC m=+0.373398674 container start 1ddf5fa7d164c31b287fba5eb4a629edc8dbe9ed51ae62b0587ba8957f2769c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T01:13:00.641 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:13:00 smithi167 podman[74497]: 2024-06-10 01:13:00.454676349 +0000 UTC m=+0.477785999 container attach 1ddf5fa7d164c31b287fba5eb4a629edc8dbe9ed51ae62b0587ba8957f2769c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, RELEASE=squid-3d89acc) 2024-06-10T01:13:00.860 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:00 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:00.583+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:00.860 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:00 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:00.583+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:00.860 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:00 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:00.583+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:01.002 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 10 01:13:00 smithi167 podman[74497]: 2024-06-10 01:13:00.646993692 +0000 UTC m=+0.670103329 container died 1ddf5fa7d164c31b287fba5eb4a629edc8dbe9ed51ae62b0587ba8957f2769c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-06-10T01:13:01.002 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:00 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:00.688+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:13:01.002 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:00 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:00.688+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:13:01.002 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:00 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:00.688+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.475054+0000 front 2024-06-10T01:12:31.475199+0000 (oldest deadline 2024-06-10T01:12:56.175181+0000) 2024-06-10T01:13:01.003 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:00 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:00.876+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:01.003 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:00 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:00.876+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:01.003 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:00 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:00.876+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:01.140 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:00 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:13:00.866+0000 7fc774f40700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.106926+0000 front 2024-06-10T01:12:25.106948+0000 (oldest deadline 2024-06-10T01:12:51.007025+0000) 2024-06-10T01:13:01.140 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:00 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:13:00.866+0000 7fc774f40700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.007521+0000 front 2024-06-10T01:12:31.007592+0000 (oldest deadline 2024-06-10T01:12:56.907625+0000) 2024-06-10T01:13:01.338 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.5.service' 2024-06-10T01:13:01.378 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:13:01.379 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-06-10T01:13:01.379 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-06-10T01:13:01.379 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.6 2024-06-10T01:13:01.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:01 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:01.243+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:24.283412+0000 front 2024-06-10T01:12:24.283445+0000 (oldest deadline 2024-06-10T01:12:50.183651+0000) 2024-06-10T01:13:01.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:01 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:01.243+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.284997+0000 front 2024-06-10T01:12:31.284852+0000 (oldest deadline 2024-06-10T01:12:55.384713+0000) 2024-06-10T01:13:01.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:01 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:01.285+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:01.610 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:01 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:01.285+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:01.610 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:01 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:01.285+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:01.723 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:01 smithi037 systemd[1]: Stopping Ceph osd.6 for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:13:01.723 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:01 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:01.547+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:01.723 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:01 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:01.547+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:01.723 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:01 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:01.547+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:02.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:01 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:01.829+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:02.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:01 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:01.829+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:02.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:01 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:01.829+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:02.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:01 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:01.716+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:13:02.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:01 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:01.716+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:13:02.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:01 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:01.716+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.475054+0000 front 2024-06-10T01:12:31.475199+0000 (oldest deadline 2024-06-10T01:12:56.175181+0000) 2024-06-10T01:13:02.140 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:01 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:13:01.728+0000 7fc77894c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:13:02.140 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:01 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:13:01.728+0000 7fc77894c700 -1 osd.6 109 *** Got signal Terminated *** 2024-06-10T01:13:02.140 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:01 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:13:01.728+0000 7fc77894c700 -1 osd.6 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T01:13:02.141 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:01 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:13:01.873+0000 7fc774f40700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.106926+0000 front 2024-06-10T01:12:25.106948+0000 (oldest deadline 2024-06-10T01:12:51.007025+0000) 2024-06-10T01:13:02.141 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:01 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:13:01.873+0000 7fc774f40700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.007521+0000 front 2024-06-10T01:12:31.007592+0000 (oldest deadline 2024-06-10T01:12:56.907625+0000) 2024-06-10T01:13:02.608 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:02 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:02.256+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:24.283412+0000 front 2024-06-10T01:12:24.283445+0000 (oldest deadline 2024-06-10T01:12:50.183651+0000) 2024-06-10T01:13:02.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:02 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:02.256+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.284997+0000 front 2024-06-10T01:12:31.284852+0000 (oldest deadline 2024-06-10T01:12:55.384713+0000) 2024-06-10T01:13:02.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:02 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:02.261+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:02.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:02 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:02.261+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:02.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:02 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:02.261+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:02.825 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:02 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:02.572+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:02.826 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:02 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:02.572+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:02.826 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:02 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:02.572+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:03.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:02 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:02.675+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:13:03.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:02 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:02.675+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:13:03.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:02 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:02.675+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.475054+0000 front 2024-06-10T01:12:31.475199+0000 (oldest deadline 2024-06-10T01:12:56.175181+0000) 2024-06-10T01:13:03.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:02 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:02.824+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:03.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:02 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:02.824+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:03.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:02 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:02.824+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:03.140 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:02 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:13:02.831+0000 7fc774f40700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.106926+0000 front 2024-06-10T01:12:25.106948+0000 (oldest deadline 2024-06-10T01:12:51.007025+0000) 2024-06-10T01:13:03.140 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:02 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:13:02.831+0000 7fc774f40700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.007521+0000 front 2024-06-10T01:12:31.007592+0000 (oldest deadline 2024-06-10T01:12:56.907625+0000) 2024-06-10T01:13:03.610 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:03 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:03.299+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:24.283412+0000 front 2024-06-10T01:12:24.283445+0000 (oldest deadline 2024-06-10T01:12:50.183651+0000) 2024-06-10T01:13:03.610 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:03 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:03.299+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.284997+0000 front 2024-06-10T01:12:31.284852+0000 (oldest deadline 2024-06-10T01:12:55.384713+0000) 2024-06-10T01:13:03.611 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:03 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:03.270+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:03.611 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:03 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:03.270+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:03.611 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:03 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:03.270+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:03.891 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:03 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:13:03.822+0000 7fc774f40700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.106926+0000 front 2024-06-10T01:12:25.106948+0000 (oldest deadline 2024-06-10T01:12:51.007025+0000) 2024-06-10T01:13:03.891 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:03 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:13:03.822+0000 7fc774f40700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.007521+0000 front 2024-06-10T01:12:31.007592+0000 (oldest deadline 2024-06-10T01:12:56.907625+0000) 2024-06-10T01:13:03.892 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:03 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:13:03.822+0000 7fc774f40700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:39.109362+0000 front 2024-06-10T01:12:39.109419+0000 (oldest deadline 2024-06-10T01:13:03.209589+0000) 2024-06-10T01:13:03.892 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:03 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:03.576+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:03.892 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:03 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:03.576+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:03.892 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:03 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:03.576+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:04.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:03 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:03.671+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:13:04.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:03 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:03.671+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:13:04.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:03 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:03.671+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.475054+0000 front 2024-06-10T01:12:31.475199+0000 (oldest deadline 2024-06-10T01:12:56.175181+0000) 2024-06-10T01:13:04.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:03 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:03.856+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:04.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:03 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:03.856+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:04.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:03 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:03.856+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:04.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:04 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:04.276+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:24.283412+0000 front 2024-06-10T01:12:24.283445+0000 (oldest deadline 2024-06-10T01:12:50.183651+0000) 2024-06-10T01:13:04.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:04 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:04.276+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.284997+0000 front 2024-06-10T01:12:31.284852+0000 (oldest deadline 2024-06-10T01:12:55.384713+0000) 2024-06-10T01:13:04.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:04 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:04.229+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:04.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:04 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:04.229+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:04.610 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:04 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:04.229+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:04.610 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:04 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:04.229+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:04.891 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:04 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:13:04.782+0000 7fc774f40700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.106926+0000 front 2024-06-10T01:12:25.106948+0000 (oldest deadline 2024-06-10T01:12:51.007025+0000) 2024-06-10T01:13:04.891 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:04 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:13:04.782+0000 7fc774f40700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.007521+0000 front 2024-06-10T01:12:31.007592+0000 (oldest deadline 2024-06-10T01:12:56.907625+0000) 2024-06-10T01:13:04.891 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:04 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:13:04.782+0000 7fc774f40700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:39.109362+0000 front 2024-06-10T01:12:39.109419+0000 (oldest deadline 2024-06-10T01:13:03.209589+0000) 2024-06-10T01:13:04.892 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:04 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:04.546+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:04.892 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:04 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:04.546+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:04.892 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:04 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:04.546+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:05.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:04 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:04.718+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:13:05.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:04 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:04.718+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:13:05.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:04 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:04.718+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.475054+0000 front 2024-06-10T01:12:31.475199+0000 (oldest deadline 2024-06-10T01:12:56.175181+0000) 2024-06-10T01:13:05.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:04 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:04.850+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:05.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:04 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:04.850+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:05.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:04 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:04.850+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:05.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:05 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:05.234+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:24.283412+0000 front 2024-06-10T01:12:24.283445+0000 (oldest deadline 2024-06-10T01:12:50.183651+0000) 2024-06-10T01:13:05.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:05 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:05.234+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.284997+0000 front 2024-06-10T01:12:31.284852+0000 (oldest deadline 2024-06-10T01:12:55.384713+0000) 2024-06-10T01:13:05.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:05 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:05.234+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:05.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:05 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:05.234+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:05.610 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:05 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:05.234+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:05.610 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:05 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:05.234+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:05.890 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:05 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:13:05.749+0000 7fc774f40700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.106926+0000 front 2024-06-10T01:12:25.106948+0000 (oldest deadline 2024-06-10T01:12:51.007025+0000) 2024-06-10T01:13:05.890 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:05 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:13:05.749+0000 7fc774f40700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.007521+0000 front 2024-06-10T01:12:31.007592+0000 (oldest deadline 2024-06-10T01:12:56.907625+0000) 2024-06-10T01:13:05.890 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:05 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6[58232]: 2024-06-10T01:13:05.749+0000 7fc774f40700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:39.109362+0000 front 2024-06-10T01:12:39.109419+0000 (oldest deadline 2024-06-10T01:13:03.209589+0000) 2024-06-10T01:13:05.891 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:05 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:05.588+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:05.891 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:05 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:05.588+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:05.891 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:05 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:05.588+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:06.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:05 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:05.875+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:06.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:05 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:05.875+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:06.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:05 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:05.875+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:06.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:05 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:05.673+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:13:06.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:05 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:05.673+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:13:06.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:05 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:05.673+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.475054+0000 front 2024-06-10T01:12:31.475199+0000 (oldest deadline 2024-06-10T01:12:56.175181+0000) 2024-06-10T01:13:06.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:05 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:05.673+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.876285+0000 front 2024-06-10T01:12:40.876394+0000 (oldest deadline 2024-06-10T01:13:04.976389+0000) 2024-06-10T01:13:06.608 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:06 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:06.239+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:24.283412+0000 front 2024-06-10T01:12:24.283445+0000 (oldest deadline 2024-06-10T01:12:50.183651+0000) 2024-06-10T01:13:06.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:06 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:06.239+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.284997+0000 front 2024-06-10T01:12:31.284852+0000 (oldest deadline 2024-06-10T01:12:55.384713+0000) 2024-06-10T01:13:06.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:06 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:06.267+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:06.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:06 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:06.267+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:06.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:06 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:06.267+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:06.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:06 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:06.267+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:06.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:06 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:06.572+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:06.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:06 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:06.572+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:06.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:06 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:06.572+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:06.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:06 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:06.572+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:41.102653+0000 front 2024-06-10T01:12:41.102799+0000 (oldest deadline 2024-06-10T01:13:05.802845+0000) 2024-06-10T01:13:07.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:06 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:06.916+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:07.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:06 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:06.916+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:07.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:06 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:06.916+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:07.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:06 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:06.720+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:13:07.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:06 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:06.720+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:13:07.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:06 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:06.720+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.475054+0000 front 2024-06-10T01:12:31.475199+0000 (oldest deadline 2024-06-10T01:12:56.175181+0000) 2024-06-10T01:13:07.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:06 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:06.720+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.876285+0000 front 2024-06-10T01:12:40.876394+0000 (oldest deadline 2024-06-10T01:13:04.976389+0000) 2024-06-10T01:13:07.574 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:07 smithi037 podman[222600]: 2024-06-10 01:13:07.285994283 +0000 UTC m=+5.695896922 container died 1db8efd002c5e1b44017263627cff0855fa2fb4c9d0f04cfe80c1332e76d2580 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-06-10T01:13:07.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:07 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:07.228+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:24.283412+0000 front 2024-06-10T01:12:24.283445+0000 (oldest deadline 2024-06-10T01:12:50.183651+0000) 2024-06-10T01:13:07.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:07 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:07.228+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.284997+0000 front 2024-06-10T01:12:31.284852+0000 (oldest deadline 2024-06-10T01:12:55.384713+0000) 2024-06-10T01:13:07.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:07 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:07.228+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:42.386439+0000 front 2024-06-10T01:12:42.386500+0000 (oldest deadline 2024-06-10T01:13:07.086225+0000) 2024-06-10T01:13:07.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:07 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:07.296+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:07.610 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:07 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:07.296+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:07.610 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:07 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:07.296+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:07.610 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:07 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:07.296+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:07.890 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:07 smithi037 podman[222600]: 2024-06-10 01:13:07.745962372 +0000 UTC m=+6.155864999 container cleanup 1db8efd002c5e1b44017263627cff0855fa2fb4c9d0f04cfe80c1332e76d2580 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-06-10T01:13:07.890 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:07 smithi037 bash[222600]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6 2024-06-10T01:13:07.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:07 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:07.592+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:07.891 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:07 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:07.592+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:07.891 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:07 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:07.592+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:07.891 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:07 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:07.592+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:41.102653+0000 front 2024-06-10T01:12:41.102799+0000 (oldest deadline 2024-06-10T01:13:05.802845+0000) 2024-06-10T01:13:08.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:07 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:07.897+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:08.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:07 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:07.897+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:08.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:07 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:07.897+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:08.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:07 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:07.748+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:13:08.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:07 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:07.748+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:13:08.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:07 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:07.748+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.475054+0000 front 2024-06-10T01:12:31.475199+0000 (oldest deadline 2024-06-10T01:12:56.175181+0000) 2024-06-10T01:13:08.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:07 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:07.748+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.876285+0000 front 2024-06-10T01:12:40.876394+0000 (oldest deadline 2024-06-10T01:13:04.976389+0000) 2024-06-10T01:13:08.390 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:07 smithi037 podman[222619]: 2024-06-10 01:13:07.964124437 +0000 UTC m=+0.679525046 container remove 1db8efd002c5e1b44017263627cff0855fa2fb4c9d0f04cfe80c1332e76d2580 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2) 2024-06-10T01:13:08.510 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:08 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:08.205+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:24.283412+0000 front 2024-06-10T01:12:24.283445+0000 (oldest deadline 2024-06-10T01:12:50.183651+0000) 2024-06-10T01:13:08.510 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:08 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:08.205+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.284997+0000 front 2024-06-10T01:12:31.284852+0000 (oldest deadline 2024-06-10T01:12:55.384713+0000) 2024-06-10T01:13:08.510 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:08 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:08.205+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:42.386439+0000 front 2024-06-10T01:12:42.386500+0000 (oldest deadline 2024-06-10T01:13:07.086225+0000) 2024-06-10T01:13:08.511 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:08 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:08.268+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:08.511 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:08 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:08.268+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:08.511 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:08 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:08.268+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:08.511 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:08 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:08.268+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:08.669 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:08 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:08.618+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:08.722 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:08 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:08.618+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:08.722 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:08 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:08.618+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:08.722 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:08 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:08.618+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:41.102653+0000 front 2024-06-10T01:12:41.102799+0000 (oldest deadline 2024-06-10T01:13:05.802845+0000) 2024-06-10T01:13:08.723 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:08 smithi037 podman[222706]: 2024-06-10 01:13:08.30127959 +0000 UTC m=+0.017845383 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:13:08.723 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:08 smithi037 podman[222706]: 2024-06-10 01:13:08.398230532 +0000 UTC m=+0.114796335 container create c97ac21bc60ba47c646a1279ccaf81e4b1974d2e67d0564189b8e569a1be2648 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-06-10T01:13:08.723 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:08 smithi037 podman[222706]: 2024-06-10 01:13:08.675819368 +0000 UTC m=+0.392385155 container init c97ac21bc60ba47c646a1279ccaf81e4b1974d2e67d0564189b8e569a1be2648 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-06-10T01:13:08.981 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:08 smithi037 podman[222706]: 2024-06-10 01:13:08.679567018 +0000 UTC m=+0.396132799 container start c97ac21bc60ba47c646a1279ccaf81e4b1974d2e67d0564189b8e569a1be2648 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528) 2024-06-10T01:13:08.981 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:08 smithi037 podman[222706]: 2024-06-10 01:13:08.77589912 +0000 UTC m=+0.492464908 container attach c97ac21bc60ba47c646a1279ccaf81e4b1974d2e67d0564189b8e569a1be2648 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6-deactivate, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True) 2024-06-10T01:13:09.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:08 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:08.918+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:09.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:08 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:08.918+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:09.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:08 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:08.918+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:09.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:08 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:08.780+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:13:09.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:08 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:08.780+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:13:09.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:08 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:08.780+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.475054+0000 front 2024-06-10T01:12:31.475199+0000 (oldest deadline 2024-06-10T01:12:56.175181+0000) 2024-06-10T01:13:09.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:08 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:08.780+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.876285+0000 front 2024-06-10T01:12:40.876394+0000 (oldest deadline 2024-06-10T01:13:04.976389+0000) 2024-06-10T01:13:09.281 INFO:journalctl@ceph.osd.6.smithi037.stdout:Jun 10 01:13:08 smithi037 podman[222706]: 2024-06-10 01:13:08.988084461 +0000 UTC m=+0.704650242 container died c97ac21bc60ba47c646a1279ccaf81e4b1974d2e67d0564189b8e569a1be2648 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T01:13:09.526 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:09 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:09.167+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:24.283412+0000 front 2024-06-10T01:12:24.283445+0000 (oldest deadline 2024-06-10T01:12:50.183651+0000) 2024-06-10T01:13:09.527 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:09 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:09.167+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.284997+0000 front 2024-06-10T01:12:31.284852+0000 (oldest deadline 2024-06-10T01:12:55.384713+0000) 2024-06-10T01:13:09.527 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:09 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:09.167+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:42.386439+0000 front 2024-06-10T01:12:42.386500+0000 (oldest deadline 2024-06-10T01:13:07.086225+0000) 2024-06-10T01:13:09.527 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:09 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:09.240+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:09.527 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:09 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:09.240+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:09.527 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:09 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:09.240+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:09.527 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:09 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:09.240+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:09.691 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.6.service' 2024-06-10T01:13:09.739 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:13:09.739 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-06-10T01:13:09.739 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-06-10T01:13:09.740 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.7 2024-06-10T01:13:09.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:09 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:09.591+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:09.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:09 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:09.591+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:09.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:09 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:09.591+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:09.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:09 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:09.591+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:41.102653+0000 front 2024-06-10T01:12:41.102799+0000 (oldest deadline 2024-06-10T01:13:05.802845+0000) 2024-06-10T01:13:10.076 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:09 smithi094 systemd[1]: Stopping Ceph osd.7 for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:13:10.082 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:09 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:09.785+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:13:10.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:09 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:09.785+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:13:10.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:09 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:09.785+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.475054+0000 front 2024-06-10T01:12:31.475199+0000 (oldest deadline 2024-06-10T01:12:56.175181+0000) 2024-06-10T01:13:10.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:09 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:09.785+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.876285+0000 front 2024-06-10T01:12:40.876394+0000 (oldest deadline 2024-06-10T01:13:04.976389+0000) 2024-06-10T01:13:10.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:09 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:09.928+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:10.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:09 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:09.928+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:10.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:09 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:09.928+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:10.358 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:10 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:10.236+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:10.359 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:10 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:10.236+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:10.359 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:10 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:10.236+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:10.359 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:10 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:10.236+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:10.359 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:10 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:10.080+0000 7f5d47ea4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:13:10.359 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:10 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:10.080+0000 7f5d47ea4700 -1 osd.7 109 *** Got signal Terminated *** 2024-06-10T01:13:10.360 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:10 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:10.080+0000 7f5d47ea4700 -1 osd.7 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T01:13:10.360 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:10 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:10.149+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:24.283412+0000 front 2024-06-10T01:12:24.283445+0000 (oldest deadline 2024-06-10T01:12:50.183651+0000) 2024-06-10T01:13:10.360 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:10 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:10.149+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.284997+0000 front 2024-06-10T01:12:31.284852+0000 (oldest deadline 2024-06-10T01:12:55.384713+0000) 2024-06-10T01:13:10.360 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:10 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:10.149+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:42.386439+0000 front 2024-06-10T01:12:42.386500+0000 (oldest deadline 2024-06-10T01:13:07.086225+0000) 2024-06-10T01:13:10.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:10 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:10.544+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:10.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:10 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:10.544+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:10.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:10 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:10.544+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:10.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:10 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:10.544+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:41.102653+0000 front 2024-06-10T01:12:41.102799+0000 (oldest deadline 2024-06-10T01:13:05.802845+0000) 2024-06-10T01:13:11.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:10 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:10.949+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:11.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:10 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:10.949+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:11.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:10 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:10.949+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:11.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:10 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:10.825+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:13:11.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:10 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:10.825+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:13:11.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:10 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:10.825+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.475054+0000 front 2024-06-10T01:12:31.475199+0000 (oldest deadline 2024-06-10T01:12:56.175181+0000) 2024-06-10T01:13:11.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:10 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:10.825+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.876285+0000 front 2024-06-10T01:12:40.876394+0000 (oldest deadline 2024-06-10T01:13:04.976389+0000) 2024-06-10T01:13:11.608 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:11 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:11.149+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:24.283412+0000 front 2024-06-10T01:12:24.283445+0000 (oldest deadline 2024-06-10T01:12:50.183651+0000) 2024-06-10T01:13:11.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:11 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:11.149+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.284997+0000 front 2024-06-10T01:12:31.284852+0000 (oldest deadline 2024-06-10T01:12:55.384713+0000) 2024-06-10T01:13:11.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:11 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:11.149+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:42.386439+0000 front 2024-06-10T01:12:42.386500+0000 (oldest deadline 2024-06-10T01:13:07.086225+0000) 2024-06-10T01:13:11.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:11 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:11.253+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:11.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:11 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:11.253+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:11.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:11 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:11.253+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:11.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:11 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:11.253+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:11.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:11 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:11.553+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:11.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:11 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:11.553+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:11.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:11 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:11.553+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:11.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:11 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:11.553+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:41.102653+0000 front 2024-06-10T01:12:41.102799+0000 (oldest deadline 2024-06-10T01:13:05.802845+0000) 2024-06-10T01:13:12.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:11 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:11.986+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:12.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:11 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:11.986+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:12.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:11 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:11.986+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:12.334 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:11 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:11.859+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:13:12.334 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:11 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:11.859+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:13:12.334 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:11 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:11.859+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.475054+0000 front 2024-06-10T01:12:31.475199+0000 (oldest deadline 2024-06-10T01:12:56.175181+0000) 2024-06-10T01:13:12.334 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:11 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:11.859+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.876285+0000 front 2024-06-10T01:12:40.876394+0000 (oldest deadline 2024-06-10T01:13:04.976389+0000) 2024-06-10T01:13:12.608 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:12 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:12.183+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:24.283412+0000 front 2024-06-10T01:12:24.283445+0000 (oldest deadline 2024-06-10T01:12:50.183651+0000) 2024-06-10T01:13:12.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:12 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:12.183+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.284997+0000 front 2024-06-10T01:12:31.284852+0000 (oldest deadline 2024-06-10T01:12:55.384713+0000) 2024-06-10T01:13:12.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:12 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:12.183+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:42.386439+0000 front 2024-06-10T01:12:42.386500+0000 (oldest deadline 2024-06-10T01:13:07.086225+0000) 2024-06-10T01:13:12.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:12 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:12.294+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:12.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:12 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:12.294+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:12.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:12 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:12.294+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:12.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:12 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:12.294+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:12.610 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:12 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:12.294+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.389036+0000 front 2024-06-10T01:12:49.389016+0000 (oldest deadline 2024-06-10T01:13:12.289127+0000) 2024-06-10T01:13:12.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:12 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:12.565+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:12.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:12 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:12.565+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:12.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:12 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:12.565+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:12.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:12 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:12.565+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:41.102653+0000 front 2024-06-10T01:12:41.102799+0000 (oldest deadline 2024-06-10T01:13:05.802845+0000) 2024-06-10T01:13:13.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:12 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:12.820+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:13:13.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:12 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:12.820+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:13:13.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:12 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:12.820+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.475054+0000 front 2024-06-10T01:12:31.475199+0000 (oldest deadline 2024-06-10T01:12:56.175181+0000) 2024-06-10T01:13:13.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:12 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:12.820+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.876285+0000 front 2024-06-10T01:12:40.876394+0000 (oldest deadline 2024-06-10T01:13:04.976389+0000) 2024-06-10T01:13:13.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:12 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:12.949+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:13.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:12 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:12.949+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:13.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:12 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:12.949+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:13.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:12 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:12.949+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:13.608 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:13 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:13.156+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:24.283412+0000 front 2024-06-10T01:12:24.283445+0000 (oldest deadline 2024-06-10T01:12:50.183651+0000) 2024-06-10T01:13:13.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:13 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:13.157+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.284997+0000 front 2024-06-10T01:12:31.284852+0000 (oldest deadline 2024-06-10T01:12:55.384713+0000) 2024-06-10T01:13:13.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:13 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:13.157+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:42.386439+0000 front 2024-06-10T01:12:42.386500+0000 (oldest deadline 2024-06-10T01:13:07.086225+0000) 2024-06-10T01:13:13.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:13 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:13.157+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:47.086637+0000 front 2024-06-10T01:12:47.086620+0000 (oldest deadline 2024-06-10T01:13:12.386662+0000) 2024-06-10T01:13:13.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:13 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:13.292+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:13.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:13 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:13.292+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:13.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:13 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:13.292+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:13.610 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:13 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:13.292+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:13.610 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:13 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:13.292+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.389036+0000 front 2024-06-10T01:12:49.389016+0000 (oldest deadline 2024-06-10T01:13:12.289127+0000) 2024-06-10T01:13:13.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:13 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:13.535+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:13.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:13 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:13.535+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:13.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:13 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:13.535+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:13.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:13 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:13.535+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:41.102653+0000 front 2024-06-10T01:12:41.102799+0000 (oldest deadline 2024-06-10T01:13:05.802845+0000) 2024-06-10T01:13:14.333 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:13 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:13.851+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:13:14.333 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:13 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:13.851+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:13:14.333 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:13 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:13.851+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.475054+0000 front 2024-06-10T01:12:31.475199+0000 (oldest deadline 2024-06-10T01:12:56.175181+0000) 2024-06-10T01:13:14.333 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:13 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:13.851+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.876285+0000 front 2024-06-10T01:12:40.876394+0000 (oldest deadline 2024-06-10T01:13:04.976389+0000) 2024-06-10T01:13:14.334 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:13 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:13.991+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:14.334 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:13 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:13.991+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:14.334 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:13 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:13.991+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:14.334 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:13 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:13.991+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:14.608 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:14 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:14.179+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:24.283412+0000 front 2024-06-10T01:12:24.283445+0000 (oldest deadline 2024-06-10T01:12:50.183651+0000) 2024-06-10T01:13:14.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:14 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:14.179+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.284997+0000 front 2024-06-10T01:12:31.284852+0000 (oldest deadline 2024-06-10T01:12:55.384713+0000) 2024-06-10T01:13:14.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:14 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:14.179+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:42.386439+0000 front 2024-06-10T01:12:42.386500+0000 (oldest deadline 2024-06-10T01:13:07.086225+0000) 2024-06-10T01:13:14.609 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:14 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7[54862]: 2024-06-10T01:13:14.179+0000 7f5d40490700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:47.086637+0000 front 2024-06-10T01:12:47.086620+0000 (oldest deadline 2024-06-10T01:13:12.386662+0000) 2024-06-10T01:13:14.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:14 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:14.319+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:14.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:14 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:14.319+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:14.610 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:14 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:14.319+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:14.610 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:14 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:14.319+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:14.610 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:14 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:14.319+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.389036+0000 front 2024-06-10T01:12:49.389016+0000 (oldest deadline 2024-06-10T01:13:12.289127+0000) 2024-06-10T01:13:14.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:14 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:14.553+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:14.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:14 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:14.553+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:14.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:14 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:14.553+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:14.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:14 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:14.553+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:41.102653+0000 front 2024-06-10T01:12:41.102799+0000 (oldest deadline 2024-06-10T01:13:05.802845+0000) 2024-06-10T01:13:14.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:14 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:14.553+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:48.604216+0000 front 2024-06-10T01:12:48.604270+0000 (oldest deadline 2024-06-10T01:13:13.904217+0000) 2024-06-10T01:13:15.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:15 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:15.013+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:15.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:15 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:15.013+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:15.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:15 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:15.013+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:15.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:15 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:15.013+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:15.334 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:14 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:14.864+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:13:15.334 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:14 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:14.864+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:13:15.334 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:14 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:14.864+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.475054+0000 front 2024-06-10T01:12:31.475199+0000 (oldest deadline 2024-06-10T01:12:56.175181+0000) 2024-06-10T01:13:15.334 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:14 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:14.864+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.876285+0000 front 2024-06-10T01:12:40.876394+0000 (oldest deadline 2024-06-10T01:13:04.976389+0000) 2024-06-10T01:13:15.334 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:14 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:14.864+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:50.178151+0000 front 2024-06-10T01:12:50.178163+0000 (oldest deadline 2024-06-10T01:13:14.278058+0000) 2024-06-10T01:13:15.606 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:15 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:15.297+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:15.607 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:15 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:15.297+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:15.607 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:15 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:15.297+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:15.607 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:15 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:15.297+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:15.607 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:15 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:15.297+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.389036+0000 front 2024-06-10T01:12:49.389016+0000 (oldest deadline 2024-06-10T01:13:12.289127+0000) 2024-06-10T01:13:15.858 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:15 smithi094 podman[73098]: 2024-06-10 01:13:15.612889938 +0000 UTC m=+5.678640774 container died e405938ba63c47eb90e1d325672c551d4e3764cc3441fa28a03cb4f2786d40cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T01:13:15.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:15 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:15.583+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:15.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:15 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:15.583+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:15.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:15 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:15.583+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:15.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:15 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:15.583+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:41.102653+0000 front 2024-06-10T01:12:41.102799+0000 (oldest deadline 2024-06-10T01:13:05.802845+0000) 2024-06-10T01:13:15.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:15 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:15.583+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:48.604216+0000 front 2024-06-10T01:12:48.604270+0000 (oldest deadline 2024-06-10T01:13:13.904217+0000) 2024-06-10T01:13:16.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:16 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:16.015+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:16.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:16 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:16.015+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:16.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:16 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:16.015+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:16.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:16 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:16.015+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:16.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:15 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:15.822+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:13:16.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:15 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:15.822+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:13:16.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:15 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:15.822+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.475054+0000 front 2024-06-10T01:12:31.475199+0000 (oldest deadline 2024-06-10T01:12:56.175181+0000) 2024-06-10T01:13:16.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:15 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:15.822+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.876285+0000 front 2024-06-10T01:12:40.876394+0000 (oldest deadline 2024-06-10T01:13:04.976389+0000) 2024-06-10T01:13:16.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:15 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:15.822+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:50.178151+0000 front 2024-06-10T01:12:50.178163+0000 (oldest deadline 2024-06-10T01:13:14.278058+0000) 2024-06-10T01:13:16.323 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:15 smithi094 podman[73098]: 2024-06-10 01:13:15.949016498 +0000 UTC m=+6.014767327 container cleanup e405938ba63c47eb90e1d325672c551d4e3764cc3441fa28a03cb4f2786d40cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True) 2024-06-10T01:13:16.323 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:15 smithi094 bash[73098]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7 2024-06-10T01:13:16.323 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:16 smithi094 podman[73121]: 2024-06-10 01:13:16.175557416 +0000 UTC m=+0.565482738 container remove e405938ba63c47eb90e1d325672c551d4e3764cc3441fa28a03cb4f2786d40cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-06-10T01:13:16.595 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:16 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:16.328+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:16.595 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:16 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:16.328+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:16.596 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:16 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:16.328+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:16.596 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:16 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:16.328+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:16.596 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:16 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:16.328+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.389036+0000 front 2024-06-10T01:12:49.389016+0000 (oldest deadline 2024-06-10T01:13:12.289127+0000) 2024-06-10T01:13:16.858 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:16 smithi094 podman[73212]: 2024-06-10 01:13:16.601534235 +0000 UTC m=+0.103537121 container create a432e360a69ead36b2e8fce31e9517782c1a5f68f8d485e8e7fe97b59de8440c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-06-10T01:13:16.859 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:16 smithi094 podman[73212]: 2024-06-10 01:13:16.514779069 +0000 UTC m=+0.016781922 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:13:16.859 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:16 smithi094 podman[73212]: 2024-06-10 01:13:16.845725185 +0000 UTC m=+0.347728047 container init a432e360a69ead36b2e8fce31e9517782c1a5f68f8d485e8e7fe97b59de8440c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-06-10T01:13:16.859 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:16 smithi094 podman[73212]: 2024-06-10 01:13:16.848792849 +0000 UTC m=+0.350795703 container start a432e360a69ead36b2e8fce31e9517782c1a5f68f8d485e8e7fe97b59de8440c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2) 2024-06-10T01:13:16.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:16 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:16.536+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:16.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:16 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:16.536+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:16.891 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:16 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:16.536+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:16.891 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:16 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:16.536+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:41.102653+0000 front 2024-06-10T01:12:41.102799+0000 (oldest deadline 2024-06-10T01:13:05.802845+0000) 2024-06-10T01:13:16.891 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:16 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:16.536+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:48.604216+0000 front 2024-06-10T01:12:48.604270+0000 (oldest deadline 2024-06-10T01:13:13.904217+0000) 2024-06-10T01:13:17.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:16 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:16.973+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:17.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:16 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:16.973+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:17.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:16 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:16.973+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:17.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:16 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:16.973+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:17.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:16 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:16.815+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:13:17.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:16 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:16.815+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:13:17.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:16 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:16.815+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.475054+0000 front 2024-06-10T01:12:31.475199+0000 (oldest deadline 2024-06-10T01:12:56.175181+0000) 2024-06-10T01:13:17.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:16 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:16.815+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.876285+0000 front 2024-06-10T01:12:40.876394+0000 (oldest deadline 2024-06-10T01:13:04.976389+0000) 2024-06-10T01:13:17.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:16 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:16.815+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:50.178151+0000 front 2024-06-10T01:12:50.178163+0000 (oldest deadline 2024-06-10T01:13:14.278058+0000) 2024-06-10T01:13:17.129 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:16 smithi094 podman[73212]: 2024-06-10 01:13:16.945854262 +0000 UTC m=+0.447857121 container attach a432e360a69ead36b2e8fce31e9517782c1a5f68f8d485e8e7fe97b59de8440c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-06-10T01:13:17.484 INFO:journalctl@ceph.osd.7.smithi094.stdout:Jun 10 01:13:17 smithi094 podman[73212]: 2024-06-10 01:13:17.135259294 +0000 UTC m=+0.637262153 container died a432e360a69ead36b2e8fce31e9517782c1a5f68f8d485e8e7fe97b59de8440c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-10T01:13:17.485 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:17 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:17.334+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:17.485 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:17 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:17.334+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:17.485 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:17 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:17.334+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:17.485 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:17 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:17.334+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:17.485 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:17 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:17.334+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.389036+0000 front 2024-06-10T01:12:49.389016+0000 (oldest deadline 2024-06-10T01:13:12.289127+0000) 2024-06-10T01:13:17.768 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.7.service' 2024-06-10T01:13:17.806 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:13:17.807 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-06-10T01:13:17.807 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-06-10T01:13:17.807 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.8 2024-06-10T01:13:17.814 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:17 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:17.561+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:17.814 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:17 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:17.561+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:17.814 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:17 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:17.561+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:17.814 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:17 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:17.561+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:41.102653+0000 front 2024-06-10T01:12:41.102799+0000 (oldest deadline 2024-06-10T01:13:05.802845+0000) 2024-06-10T01:13:17.814 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:17 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:17.561+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:48.604216+0000 front 2024-06-10T01:12:48.604270+0000 (oldest deadline 2024-06-10T01:13:13.904217+0000) 2024-06-10T01:13:18.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:17 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:17.989+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:18.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:17 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:17.989+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:18.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:17 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:17.989+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:18.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:17 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:17.989+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:18.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:17 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:17.783+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:13:18.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:17 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:17.783+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:13:18.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:17 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:17.783+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.475054+0000 front 2024-06-10T01:12:31.475199+0000 (oldest deadline 2024-06-10T01:12:56.175181+0000) 2024-06-10T01:13:18.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:17 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:17.783+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.876285+0000 front 2024-06-10T01:12:40.876394+0000 (oldest deadline 2024-06-10T01:13:04.976389+0000) 2024-06-10T01:13:18.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:17 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:17.783+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:50.178151+0000 front 2024-06-10T01:12:50.178163+0000 (oldest deadline 2024-06-10T01:13:14.278058+0000) 2024-06-10T01:13:18.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:17 smithi167 systemd[1]: Stopping Ceph osd.8 for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:13:18.583 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:18 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:18.137+0000 7f74b01ee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:13:18.583 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:18 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:18.137+0000 7f74b01ee700 -1 osd.8 109 *** Got signal Terminated *** 2024-06-10T01:13:18.583 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:18 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:18.137+0000 7f74b01ee700 -1 osd.8 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T01:13:18.608 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:18 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:18.344+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:18.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:18 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:18.344+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:18.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:18 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:18.344+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:18.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:18 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:18.344+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:18.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:18 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:18.344+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.389036+0000 front 2024-06-10T01:12:49.389016+0000 (oldest deadline 2024-06-10T01:13:12.289127+0000) 2024-06-10T01:13:18.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:18 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:18.523+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:18.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:18 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:18.523+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:18.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:18 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:18.523+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:18.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:18 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:18.523+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:41.102653+0000 front 2024-06-10T01:12:41.102799+0000 (oldest deadline 2024-06-10T01:13:05.802845+0000) 2024-06-10T01:13:18.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:18 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:18.523+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:48.604216+0000 front 2024-06-10T01:12:48.604270+0000 (oldest deadline 2024-06-10T01:13:13.904217+0000) 2024-06-10T01:13:19.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:18 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:18.989+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:19.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:18 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:18.989+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:19.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:18 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:18.989+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:19.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:18 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:18.989+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:19.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:18 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:18.825+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:13:19.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:18 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:18.825+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:13:19.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:18 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:18.825+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.475054+0000 front 2024-06-10T01:12:31.475199+0000 (oldest deadline 2024-06-10T01:12:56.175181+0000) 2024-06-10T01:13:19.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:18 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:18.825+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.876285+0000 front 2024-06-10T01:12:40.876394+0000 (oldest deadline 2024-06-10T01:13:04.976389+0000) 2024-06-10T01:13:19.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:18 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:18.825+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:50.178151+0000 front 2024-06-10T01:12:50.178163+0000 (oldest deadline 2024-06-10T01:13:14.278058+0000) 2024-06-10T01:13:19.858 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:19 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:19.387+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:19.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:19 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:19.387+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:19.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:19 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:19.387+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:19.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:19 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:19.387+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:19.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:19 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:19.387+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.389036+0000 front 2024-06-10T01:12:49.389016+0000 (oldest deadline 2024-06-10T01:13:12.289127+0000) 2024-06-10T01:13:19.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:19 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:19.536+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:19.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:19 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:19.536+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:19.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:19 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:19.536+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:19.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:19 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:19.536+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:41.102653+0000 front 2024-06-10T01:12:41.102799+0000 (oldest deadline 2024-06-10T01:13:05.802845+0000) 2024-06-10T01:13:19.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:19 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:19.536+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:48.604216+0000 front 2024-06-10T01:12:48.604270+0000 (oldest deadline 2024-06-10T01:13:13.904217+0000) 2024-06-10T01:13:20.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:19 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:19.805+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:13:20.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:19 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:19.805+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:13:20.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:19 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:19.805+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.475054+0000 front 2024-06-10T01:12:31.475199+0000 (oldest deadline 2024-06-10T01:12:56.175181+0000) 2024-06-10T01:13:20.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:19 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:19.805+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.876285+0000 front 2024-06-10T01:12:40.876394+0000 (oldest deadline 2024-06-10T01:13:04.976389+0000) 2024-06-10T01:13:20.083 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:19 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:19.805+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:50.178151+0000 front 2024-06-10T01:12:50.178163+0000 (oldest deadline 2024-06-10T01:13:14.278058+0000) 2024-06-10T01:13:20.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:19 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:19.995+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:20.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:19 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:19.995+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:20.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:19 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:19.995+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:20.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:19 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:19.995+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:20.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:20 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:20.380+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:20.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:20 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:20.380+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:20.860 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:20 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:20.380+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:20.860 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:20 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:20.380+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:20.860 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:20 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:20.380+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.389036+0000 front 2024-06-10T01:12:49.389016+0000 (oldest deadline 2024-06-10T01:13:12.289127+0000) 2024-06-10T01:13:20.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:20 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:20.575+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:20.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:20 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:20.575+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:20.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:20 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:20.575+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:20.891 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:20 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:20.575+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:41.102653+0000 front 2024-06-10T01:12:41.102799+0000 (oldest deadline 2024-06-10T01:13:05.802845+0000) 2024-06-10T01:13:20.891 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:20 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:20.575+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:48.604216+0000 front 2024-06-10T01:12:48.604270+0000 (oldest deadline 2024-06-10T01:13:13.904217+0000) 2024-06-10T01:13:21.035 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:20 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:20.805+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:13:21.036 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:20 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:20.805+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:13:21.036 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:20 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:20.805+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.475054+0000 front 2024-06-10T01:12:31.475199+0000 (oldest deadline 2024-06-10T01:12:56.175181+0000) 2024-06-10T01:13:21.036 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:20 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:20.805+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.876285+0000 front 2024-06-10T01:12:40.876394+0000 (oldest deadline 2024-06-10T01:13:04.976389+0000) 2024-06-10T01:13:21.036 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:20 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:20.805+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:50.178151+0000 front 2024-06-10T01:12:50.178163+0000 (oldest deadline 2024-06-10T01:13:14.278058+0000) 2024-06-10T01:13:21.036 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:20 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:20.805+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:55.978851+0000 front 2024-06-10T01:12:55.978897+0000 (oldest deadline 2024-06-10T01:13:20.079040+0000) 2024-06-10T01:13:21.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:21 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:21.041+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:21.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:21 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:21.041+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:21.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:21 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:21.041+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:21.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:21 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:21.041+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:21.334 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:21 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:21.041+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:57.307803+0000 front 2024-06-10T01:12:57.307782+0000 (oldest deadline 2024-06-10T01:13:20.207892+0000) 2024-06-10T01:13:21.608 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:21 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:21.340+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:21.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:21 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:21.340+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:21.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:21 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:21.340+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:21.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:21 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:21.340+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:21.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:21 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:21.340+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.389036+0000 front 2024-06-10T01:12:49.389016+0000 (oldest deadline 2024-06-10T01:13:12.289127+0000) 2024-06-10T01:13:21.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:21 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:21.577+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:21.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:21 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:21.577+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:21.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:21 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:21.577+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:21.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:21 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:21.577+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:41.102653+0000 front 2024-06-10T01:12:41.102799+0000 (oldest deadline 2024-06-10T01:13:05.802845+0000) 2024-06-10T01:13:21.891 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:21 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:21.577+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:48.604216+0000 front 2024-06-10T01:12:48.604270+0000 (oldest deadline 2024-06-10T01:13:13.904217+0000) 2024-06-10T01:13:22.061 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:21 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:21.815+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:13:22.061 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:21 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:21.815+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:13:22.061 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:21 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:21.815+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.475054+0000 front 2024-06-10T01:12:31.475199+0000 (oldest deadline 2024-06-10T01:12:56.175181+0000) 2024-06-10T01:13:22.061 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:21 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:21.815+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.876285+0000 front 2024-06-10T01:12:40.876394+0000 (oldest deadline 2024-06-10T01:13:04.976389+0000) 2024-06-10T01:13:22.061 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:21 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:21.816+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:50.178151+0000 front 2024-06-10T01:12:50.178163+0000 (oldest deadline 2024-06-10T01:13:14.278058+0000) 2024-06-10T01:13:22.061 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:21 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:21.816+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:55.978851+0000 front 2024-06-10T01:12:55.978897+0000 (oldest deadline 2024-06-10T01:13:20.079040+0000) 2024-06-10T01:13:22.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:22 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:22.066+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:22.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:22 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:22.066+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:22.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:22 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:22.066+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:22.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:22 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:22.066+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:22.334 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:22 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:22.066+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:57.307803+0000 front 2024-06-10T01:12:57.307782+0000 (oldest deadline 2024-06-10T01:13:20.207892+0000) 2024-06-10T01:13:22.858 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:22 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:22.381+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:22.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:22 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:22.381+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:22.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:22 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:22.381+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:22.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:22 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:22.381+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:22.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:22 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:22.381+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.389036+0000 front 2024-06-10T01:12:49.389016+0000 (oldest deadline 2024-06-10T01:13:12.289127+0000) 2024-06-10T01:13:22.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:22 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:22.381+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:56.390413+0000 front 2024-06-10T01:12:56.390285+0000 (oldest deadline 2024-06-10T01:13:22.290215+0000) 2024-06-10T01:13:22.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:22 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:22.538+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:22.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:22 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:22.538+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:22.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:22 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:22.538+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:22.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:22 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:22.538+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:41.102653+0000 front 2024-06-10T01:12:41.102799+0000 (oldest deadline 2024-06-10T01:13:05.802845+0000) 2024-06-10T01:13:22.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:22 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:22.538+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:48.604216+0000 front 2024-06-10T01:12:48.604270+0000 (oldest deadline 2024-06-10T01:13:13.904217+0000) 2024-06-10T01:13:23.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:23 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:23.041+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:23.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:23 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:23.041+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:23.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:23 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:23.041+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:23.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:23 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:23.041+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:23.084 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:23 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:23.041+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:57.307803+0000 front 2024-06-10T01:12:57.307782+0000 (oldest deadline 2024-06-10T01:13:20.207892+0000) 2024-06-10T01:13:23.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:22 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:22.821+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.571013+0000 front 2024-06-10T01:12:13.571002+0000 (oldest deadline 2024-06-10T01:12:37.070822+0000) 2024-06-10T01:13:23.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:22 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:22.821+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.673941+0000 front 2024-06-10T01:12:25.673848+0000 (oldest deadline 2024-06-10T01:12:46.773950+0000) 2024-06-10T01:13:23.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:22 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:22.821+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:31.475054+0000 front 2024-06-10T01:12:31.475199+0000 (oldest deadline 2024-06-10T01:12:56.175181+0000) 2024-06-10T01:13:23.084 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:22 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:22.822+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.876285+0000 front 2024-06-10T01:12:40.876394+0000 (oldest deadline 2024-06-10T01:13:04.976389+0000) 2024-06-10T01:13:23.085 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:22 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:22.822+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:50.178151+0000 front 2024-06-10T01:12:50.178163+0000 (oldest deadline 2024-06-10T01:13:14.278058+0000) 2024-06-10T01:13:23.085 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:22 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8[55506]: 2024-06-10T01:13:22.822+0000 7f74ac7e2700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:55.978851+0000 front 2024-06-10T01:12:55.978897+0000 (oldest deadline 2024-06-10T01:13:20.079040+0000) 2024-06-10T01:13:23.858 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:23 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:23.413+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:23.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:23 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:23.413+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:23.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:23 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:23.413+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:23.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:23 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:23.413+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:23.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:23 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:23.413+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.389036+0000 front 2024-06-10T01:12:49.389016+0000 (oldest deadline 2024-06-10T01:13:12.289127+0000) 2024-06-10T01:13:23.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:23 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:23.413+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:56.390413+0000 front 2024-06-10T01:12:56.390285+0000 (oldest deadline 2024-06-10T01:13:22.290215+0000) 2024-06-10T01:13:23.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:23 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:23.564+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:23.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:23 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:23.564+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:23.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:23 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:23.564+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:23.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:23 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:23.564+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:41.102653+0000 front 2024-06-10T01:12:41.102799+0000 (oldest deadline 2024-06-10T01:13:05.802845+0000) 2024-06-10T01:13:23.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:23 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:23.564+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:48.604216+0000 front 2024-06-10T01:12:48.604270+0000 (oldest deadline 2024-06-10T01:13:13.904217+0000) 2024-06-10T01:13:23.974 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:23 smithi167 podman[74618]: 2024-06-10 01:13:23.674650605 +0000 UTC m=+5.668483292 container died acefb6e84b079a795d2d5f5c1abf544dee2c2d9bb43a1a5ecf6d5ef0a5ae8b88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-06-10T01:13:24.333 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:24 smithi167 podman[74618]: 2024-06-10 01:13:24.079103414 +0000 UTC m=+6.072936095 container cleanup acefb6e84b079a795d2d5f5c1abf544dee2c2d9bb43a1a5ecf6d5ef0a5ae8b88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T01:13:24.333 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:24 smithi167 bash[74618]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8 2024-06-10T01:13:24.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:24 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:24.076+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:24.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:24 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:24.076+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:24.334 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:24 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:24.076+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:24.334 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:24 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:24.076+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:24.334 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:24 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:24.076+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:57.307803+0000 front 2024-06-10T01:12:57.307782+0000 (oldest deadline 2024-06-10T01:13:20.207892+0000) 2024-06-10T01:13:24.738 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:24 smithi167 podman[74634]: 2024-06-10 01:13:24.38106655 +0000 UTC m=+0.698230175 container remove acefb6e84b079a795d2d5f5c1abf544dee2c2d9bb43a1a5ecf6d5ef0a5ae8b88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3d89acc) 2024-06-10T01:13:24.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:24 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:24.386+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:26.009 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:24 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:24.386+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:26.010 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:24 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:24.386+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:26.010 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:24 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:24.386+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:26.010 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:24 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:24.386+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.389036+0000 front 2024-06-10T01:12:49.389016+0000 (oldest deadline 2024-06-10T01:13:12.289127+0000) 2024-06-10T01:13:26.010 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:24 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:24.386+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:56.390413+0000 front 2024-06-10T01:12:56.390285+0000 (oldest deadline 2024-06-10T01:13:22.290215+0000) 2024-06-10T01:13:26.011 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:25 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:25.403+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:26.011 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:25 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:25.403+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:26.012 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:25 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:25.403+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:26.012 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:25 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:25.403+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:26.012 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:25 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:25.403+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.389036+0000 front 2024-06-10T01:12:49.389016+0000 (oldest deadline 2024-06-10T01:13:12.289127+0000) 2024-06-10T01:13:26.012 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:25 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:25.403+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:56.390413+0000 front 2024-06-10T01:12:56.390285+0000 (oldest deadline 2024-06-10T01:13:22.290215+0000) 2024-06-10T01:13:26.012 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:24 smithi167 podman[74726]: 2024-06-10 01:13:24.647255801 +0000 UTC m=+0.017050907 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:13:26.012 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:24 smithi167 podman[74726]: 2024-06-10 01:13:24.765356505 +0000 UTC m=+0.135151615 container create 5c4f6db9d28e9a00d76f62b91f414cc6817e3a072512979d3b1f12e28b4e5ad8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-06-10T01:13:26.012 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:24 smithi167 podman[74726]: 2024-06-10 01:13:24.985155014 +0000 UTC m=+0.354950122 container init 5c4f6db9d28e9a00d76f62b91f414cc6817e3a072512979d3b1f12e28b4e5ad8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8-deactivate, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.schema-version=1.0) 2024-06-10T01:13:26.013 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:24 smithi167 podman[74726]: 2024-06-10 01:13:24.988428362 +0000 UTC m=+0.358223473 container start 5c4f6db9d28e9a00d76f62b91f414cc6817e3a072512979d3b1f12e28b4e5ad8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240528, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-10T01:13:26.013 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:25 smithi167 podman[74726]: 2024-06-10 01:13:25.109789112 +0000 UTC m=+0.479584243 container attach 5c4f6db9d28e9a00d76f62b91f414cc6817e3a072512979d3b1f12e28b4e5ad8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True) 2024-06-10T01:13:26.013 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:24 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:24.590+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:26.013 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:24 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:24.590+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:26.013 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:24 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:24.590+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:26.013 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:24 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:24.590+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:41.102653+0000 front 2024-06-10T01:12:41.102799+0000 (oldest deadline 2024-06-10T01:13:05.802845+0000) 2024-06-10T01:13:26.014 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:24 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:24.590+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:48.604216+0000 front 2024-06-10T01:12:48.604270+0000 (oldest deadline 2024-06-10T01:13:13.904217+0000) 2024-06-10T01:13:26.014 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:25 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:25.569+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:26.014 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:25 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:25.569+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:26.014 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:25 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:25.569+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:26.014 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:25 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:25.569+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:41.102653+0000 front 2024-06-10T01:12:41.102799+0000 (oldest deadline 2024-06-10T01:13:05.802845+0000) 2024-06-10T01:13:26.014 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:25 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:25.569+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:48.604216+0000 front 2024-06-10T01:12:48.604270+0000 (oldest deadline 2024-06-10T01:13:13.904217+0000) 2024-06-10T01:13:26.015 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:25 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:25.046+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:26.015 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:25 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:25.046+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:26.015 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:25 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:25.046+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:26.015 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:25 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:25.046+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:26.015 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:25 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:25.046+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:57.307803+0000 front 2024-06-10T01:12:57.307782+0000 (oldest deadline 2024-06-10T01:13:20.207892+0000) 2024-06-10T01:13:26.015 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jun 10 01:13:25 smithi167 podman[74726]: 2024-06-10 01:13:25.299560335 +0000 UTC m=+0.669355444 container died 5c4f6db9d28e9a00d76f62b91f414cc6817e3a072512979d3b1f12e28b4e5ad8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T01:13:26.016 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.8.service' 2024-06-10T01:13:26.056 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:13:26.057 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-06-10T01:13:26.057 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-06-10T01:13:26.057 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.9 2024-06-10T01:13:26.262 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:26 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:26.059+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:26.262 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:26 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:26.059+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:26.262 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:26 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:26.059+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:26.262 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:26 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:26.059+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:26.263 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:26 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:26.059+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:57.307803+0000 front 2024-06-10T01:12:57.307782+0000 (oldest deadline 2024-06-10T01:13:20.207892+0000) 2024-06-10T01:13:26.385 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:26 smithi037 systemd[1]: Stopping Ceph osd.9 for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:13:26.640 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:26 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:26.390+0000 7f107bb54700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:13:26.640 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:26 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:26.390+0000 7f107bb54700 -1 osd.9 109 *** Got signal Terminated *** 2024-06-10T01:13:26.640 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:26 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:26.390+0000 7f107bb54700 -1 osd.9 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T01:13:26.641 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:26 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:26.594+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:26.641 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:26 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:26.594+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:26.641 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:26 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:26.594+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:26.641 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:26 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:26.594+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:41.102653+0000 front 2024-06-10T01:12:41.102799+0000 (oldest deadline 2024-06-10T01:13:05.802845+0000) 2024-06-10T01:13:26.641 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:26 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:26.594+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:48.604216+0000 front 2024-06-10T01:12:48.604270+0000 (oldest deadline 2024-06-10T01:13:13.904217+0000) 2024-06-10T01:13:26.858 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:26 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:26.435+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:26.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:26 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:26.435+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:26.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:26 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:26.435+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:26.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:26 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:26.435+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:26.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:26 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:26.435+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.389036+0000 front 2024-06-10T01:12:49.389016+0000 (oldest deadline 2024-06-10T01:13:12.289127+0000) 2024-06-10T01:13:26.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:26 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:26.435+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:56.390413+0000 front 2024-06-10T01:12:56.390285+0000 (oldest deadline 2024-06-10T01:13:22.290215+0000) 2024-06-10T01:13:27.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:27 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:27.084+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:27.334 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:27 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:27.084+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:27.334 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:27 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:27.084+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:27.334 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:27 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:27.084+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:27.334 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:27 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:27.084+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:57.307803+0000 front 2024-06-10T01:12:57.307782+0000 (oldest deadline 2024-06-10T01:13:20.207892+0000) 2024-06-10T01:13:27.843 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:27 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:27.565+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:27.843 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:27 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:27.565+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:27.844 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:27 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:27.565+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:27.844 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:27 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:27.565+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:41.102653+0000 front 2024-06-10T01:12:41.102799+0000 (oldest deadline 2024-06-10T01:13:05.802845+0000) 2024-06-10T01:13:27.844 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:27 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:27.565+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:48.604216+0000 front 2024-06-10T01:12:48.604270+0000 (oldest deadline 2024-06-10T01:13:13.904217+0000) 2024-06-10T01:13:27.858 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:27 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:27.471+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:27.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:27 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:27.471+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:27.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:27 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:27.471+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:27.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:27 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:27.471+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:27.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:27 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:27.471+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.389036+0000 front 2024-06-10T01:12:49.389016+0000 (oldest deadline 2024-06-10T01:13:12.289127+0000) 2024-06-10T01:13:27.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:27 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:27.471+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:56.390413+0000 front 2024-06-10T01:12:56.390285+0000 (oldest deadline 2024-06-10T01:13:22.290215+0000) 2024-06-10T01:13:28.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:28 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:28.071+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:28.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:28 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:28.071+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:28.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:28 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:28.071+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:28.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:28 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:28.071+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:28.334 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:28 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:28.071+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:57.307803+0000 front 2024-06-10T01:12:57.307782+0000 (oldest deadline 2024-06-10T01:13:20.207892+0000) 2024-06-10T01:13:28.858 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:28 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:28.447+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:28.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:28 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:28.447+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:28.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:28 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:28.447+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:28.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:28 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:28.447+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:28.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:28 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:28.447+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.389036+0000 front 2024-06-10T01:12:49.389016+0000 (oldest deadline 2024-06-10T01:13:12.289127+0000) 2024-06-10T01:13:28.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:28 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:28.447+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:56.390413+0000 front 2024-06-10T01:12:56.390285+0000 (oldest deadline 2024-06-10T01:13:22.290215+0000) 2024-06-10T01:13:28.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:28 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:28.447+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-06-10T01:13:02.290894+0000 front 2024-06-10T01:13:02.290912+0000 (oldest deadline 2024-06-10T01:13:28.190609+0000) 2024-06-10T01:13:28.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:28 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:28.568+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:28.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:28 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:28.568+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:28.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:28 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:28.568+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:28.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:28 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:28.568+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:41.102653+0000 front 2024-06-10T01:12:41.102799+0000 (oldest deadline 2024-06-10T01:13:05.802845+0000) 2024-06-10T01:13:28.891 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:28 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:28.568+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:48.604216+0000 front 2024-06-10T01:12:48.604270+0000 (oldest deadline 2024-06-10T01:13:13.904217+0000) 2024-06-10T01:13:29.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:29 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:29.060+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:29.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:29 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:29.060+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:29.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:29 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:29.060+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:29.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:29 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:29.060+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:29.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:29 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:29.060+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:57.307803+0000 front 2024-06-10T01:12:57.307782+0000 (oldest deadline 2024-06-10T01:13:20.207892+0000) 2024-06-10T01:13:29.858 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:29 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:29.461+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:29.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:29 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:29.461+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:29.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:29 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:29.461+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:29.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:29 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:29.461+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:29.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:29 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:29.461+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.389036+0000 front 2024-06-10T01:12:49.389016+0000 (oldest deadline 2024-06-10T01:13:12.289127+0000) 2024-06-10T01:13:29.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:29 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:29.461+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:56.390413+0000 front 2024-06-10T01:12:56.390285+0000 (oldest deadline 2024-06-10T01:13:22.290215+0000) 2024-06-10T01:13:29.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:29 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:29.461+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-06-10T01:13:02.290894+0000 front 2024-06-10T01:13:02.290912+0000 (oldest deadline 2024-06-10T01:13:28.190609+0000) 2024-06-10T01:13:29.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:29 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:29.527+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:29.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:29 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:29.527+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:29.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:29 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:29.527+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:29.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:29 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:29.527+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:41.102653+0000 front 2024-06-10T01:12:41.102799+0000 (oldest deadline 2024-06-10T01:13:05.802845+0000) 2024-06-10T01:13:29.891 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:29 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:29.527+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:48.604216+0000 front 2024-06-10T01:12:48.604270+0000 (oldest deadline 2024-06-10T01:13:13.904217+0000) 2024-06-10T01:13:30.332 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:30 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:30.082+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:30.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:30 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:30.082+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:30.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:30 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:30.082+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:30.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:30 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:30.082+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:30.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:30 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:30.082+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:57.307803+0000 front 2024-06-10T01:12:57.307782+0000 (oldest deadline 2024-06-10T01:13:20.207892+0000) 2024-06-10T01:13:30.858 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:30 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:30.495+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:30.861 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:30 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:30.495+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:30.861 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:30 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:30.495+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:30.862 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:30 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:30.495+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:30.862 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:30 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:30.495+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.389036+0000 front 2024-06-10T01:12:49.389016+0000 (oldest deadline 2024-06-10T01:13:12.289127+0000) 2024-06-10T01:13:30.862 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:30 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:30.495+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:56.390413+0000 front 2024-06-10T01:12:56.390285+0000 (oldest deadline 2024-06-10T01:13:22.290215+0000) 2024-06-10T01:13:30.862 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:30 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:30.495+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-06-10T01:13:02.290894+0000 front 2024-06-10T01:13:02.290912+0000 (oldest deadline 2024-06-10T01:13:28.190609+0000) 2024-06-10T01:13:30.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:30 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:30.532+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:15.998239+0000 front 2024-06-10T01:12:15.998211+0000 (oldest deadline 2024-06-10T01:12:40.098352+0000) 2024-06-10T01:13:30.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:30 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:30.532+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.599370+0000 front 2024-06-10T01:12:23.599253+0000 (oldest deadline 2024-06-10T01:12:48.899455+0000) 2024-06-10T01:13:30.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:30 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:30.532+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:28.900227+0000 front 2024-06-10T01:12:28.900027+0000 (oldest deadline 2024-06-10T01:12:54.800242+0000) 2024-06-10T01:13:30.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:30 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:30.532+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:41.102653+0000 front 2024-06-10T01:12:41.102799+0000 (oldest deadline 2024-06-10T01:13:05.802845+0000) 2024-06-10T01:13:30.890 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:30 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9[65060]: 2024-06-10T01:13:30.532+0000 7f1074140700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:48.604216+0000 front 2024-06-10T01:12:48.604270+0000 (oldest deadline 2024-06-10T01:13:13.904217+0000) 2024-06-10T01:13:31.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:31 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:31.126+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:31.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:31 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:31.126+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:31.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:31 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:31.126+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:31.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:31 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:31.126+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:31.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:31 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:31.126+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:57.307803+0000 front 2024-06-10T01:12:57.307782+0000 (oldest deadline 2024-06-10T01:13:20.207892+0000) 2024-06-10T01:13:31.858 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:31 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:31.493+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:31.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:31 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:31.493+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:31.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:31 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:31.493+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:31.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:31 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:31.493+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:31.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:31 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:31.493+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.389036+0000 front 2024-06-10T01:12:49.389016+0000 (oldest deadline 2024-06-10T01:13:12.289127+0000) 2024-06-10T01:13:31.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:31 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:31.493+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:56.390413+0000 front 2024-06-10T01:12:56.390285+0000 (oldest deadline 2024-06-10T01:13:22.290215+0000) 2024-06-10T01:13:31.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:31 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:31.493+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-06-10T01:13:02.290894+0000 front 2024-06-10T01:13:02.290912+0000 (oldest deadline 2024-06-10T01:13:28.190609+0000) 2024-06-10T01:13:32.044 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:31 smithi037 podman[222824]: 2024-06-10 01:13:31.77896037 +0000 UTC m=+5.500633006 container died 1a37515c235693eebbedafd616c151c2cb333075f8383f717cc0538e8f18ce7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, ceph=True) 2024-06-10T01:13:32.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:32 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:32.086+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:32.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:32 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:32.086+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:32.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:32 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:32.086+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:32.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:32 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:32.086+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:32.333 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:32 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:32.086+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:57.307803+0000 front 2024-06-10T01:12:57.307782+0000 (oldest deadline 2024-06-10T01:13:20.207892+0000) 2024-06-10T01:13:32.334 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:32 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:32.086+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-06-10T01:13:06.009381+0000 front 2024-06-10T01:13:06.009442+0000 (oldest deadline 2024-06-10T01:13:31.909504+0000) 2024-06-10T01:13:32.390 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:32 smithi037 podman[222824]: 2024-06-10 01:13:32.207810823 +0000 UTC m=+5.929483498 container cleanup 1a37515c235693eebbedafd616c151c2cb333075f8383f717cc0538e8f18ce7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-06-10T01:13:32.390 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:32 smithi037 bash[222824]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9 2024-06-10T01:13:32.836 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:32 smithi037 podman[222841]: 2024-06-10 01:13:32.400692964 +0000 UTC m=+0.618993724 container remove 1a37515c235693eebbedafd616c151c2cb333075f8383f717cc0538e8f18ce7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=) 2024-06-10T01:13:32.858 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:32 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:32.527+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:32.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:32 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:32.527+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:32.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:32 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:32.527+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:32.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:32 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:32.527+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:32.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:32 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:32.527+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.389036+0000 front 2024-06-10T01:12:49.389016+0000 (oldest deadline 2024-06-10T01:13:12.289127+0000) 2024-06-10T01:13:32.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:32 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:32.527+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:56.390413+0000 front 2024-06-10T01:12:56.390285+0000 (oldest deadline 2024-06-10T01:13:22.290215+0000) 2024-06-10T01:13:32.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:32 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:32.527+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-06-10T01:13:02.290894+0000 front 2024-06-10T01:13:02.290912+0000 (oldest deadline 2024-06-10T01:13:28.190609+0000) 2024-06-10T01:13:33.090 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:32 smithi037 podman[222931]: 2024-06-10 01:13:32.745924592 +0000 UTC m=+0.017717081 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:13:33.090 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:32 smithi037 podman[222931]: 2024-06-10 01:13:32.893434404 +0000 UTC m=+0.165226930 container create 51b3945c4135d6c66c47910cea6e3d97aafbd7fe6e6a35b74eeed7ccfe0c0c2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T01:13:33.090 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:33 smithi037 podman[222931]: 2024-06-10 01:13:33.095940624 +0000 UTC m=+0.367733104 container init 51b3945c4135d6c66c47910cea6e3d97aafbd7fe6e6a35b74eeed7ccfe0c0c2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-06-10T01:13:33.390 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:33 smithi037 podman[222931]: 2024-06-10 01:13:33.099855943 +0000 UTC m=+0.371648426 container start 51b3945c4135d6c66c47910cea6e3d97aafbd7fe6e6a35b74eeed7ccfe0c0c2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-06-10T01:13:33.390 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:33 smithi037 podman[222931]: 2024-06-10 01:13:33.220825396 +0000 UTC m=+0.492617887 container attach 51b3945c4135d6c66c47910cea6e3d97aafbd7fe6e6a35b74eeed7ccfe0c0c2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-06-10T01:13:33.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:33 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:33.113+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:33.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:33 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:33.113+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:33.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:33 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:33.113+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:33.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:33 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:33.113+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:33.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:33 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:33.113+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:57.307803+0000 front 2024-06-10T01:12:57.307782+0000 (oldest deadline 2024-06-10T01:13:20.207892+0000) 2024-06-10T01:13:33.584 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:33 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:33.113+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-06-10T01:13:06.009381+0000 front 2024-06-10T01:13:06.009442+0000 (oldest deadline 2024-06-10T01:13:31.909504+0000) 2024-06-10T01:13:33.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:33 smithi037 podman[222931]: 2024-06-10 01:13:33.411327545 +0000 UTC m=+0.683120028 container died 51b3945c4135d6c66c47910cea6e3d97aafbd7fe6e6a35b74eeed7ccfe0c0c2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T01:13:33.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:33 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:33.513+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:33.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:33 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:33.513+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:33.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:33 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:33.513+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:33.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:33 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:33.513+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:33.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:33 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:33.513+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.389036+0000 front 2024-06-10T01:12:49.389016+0000 (oldest deadline 2024-06-10T01:13:12.289127+0000) 2024-06-10T01:13:33.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:33 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:33.513+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:56.390413+0000 front 2024-06-10T01:12:56.390285+0000 (oldest deadline 2024-06-10T01:13:22.290215+0000) 2024-06-10T01:13:33.860 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:33 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:33.513+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-06-10T01:13:02.290894+0000 front 2024-06-10T01:13:02.290912+0000 (oldest deadline 2024-06-10T01:13:28.190609+0000) 2024-06-10T01:13:33.953 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.9.service' 2024-06-10T01:13:33.978 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:33 smithi037 podman[222957]: 2024-06-10 01:13:33.940912021 +0000 UTC m=+0.522197629 container remove 51b3945c4135d6c66c47910cea6e3d97aafbd7fe6e6a35b74eeed7ccfe0c0c2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True) 2024-06-10T01:13:33.979 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:33 smithi037 systemd[1]: ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.9.service: Deactivated successfully. 2024-06-10T01:13:33.979 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:33 smithi037 systemd[1]: Stopped Ceph osd.9 for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T01:13:33.979 INFO:journalctl@ceph.osd.9.smithi037.stdout:Jun 10 01:13:33 smithi037 systemd[1]: ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.9.service: Consumed 2min 57.784s CPU time. 2024-06-10T01:13:33.995 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:13:33.995 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-06-10T01:13:33.996 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-06-10T01:13:33.996 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.10 2024-06-10T01:13:34.296 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:34 smithi094 systemd[1]: Stopping Ceph osd.10 for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:13:34.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:34 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:34.139+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:34.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:34 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:34.139+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:34.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:34 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:34.139+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:34.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:34 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:34.139+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:34.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:34 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:34.139+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:57.307803+0000 front 2024-06-10T01:12:57.307782+0000 (oldest deadline 2024-06-10T01:13:20.207892+0000) 2024-06-10T01:13:34.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:34 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:34.139+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-06-10T01:13:06.009381+0000 front 2024-06-10T01:13:06.009442+0000 (oldest deadline 2024-06-10T01:13:31.909504+0000) 2024-06-10T01:13:34.608 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:34 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:34.302+0000 7fb1e3b64700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:13:34.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:34 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:34.302+0000 7fb1e3b64700 -1 osd.10 109 *** Got signal Terminated *** 2024-06-10T01:13:34.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:34 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:34.302+0000 7fb1e3b64700 -1 osd.10 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T01:13:34.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:34 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:34.490+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:34.609 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:34 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:34.490+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:34.610 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:34 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:34.490+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:34.610 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:34 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:34.490+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:34.610 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:34 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:34.490+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.389036+0000 front 2024-06-10T01:12:49.389016+0000 (oldest deadline 2024-06-10T01:13:12.289127+0000) 2024-06-10T01:13:34.610 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:34 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:34.490+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:56.390413+0000 front 2024-06-10T01:12:56.390285+0000 (oldest deadline 2024-06-10T01:13:22.290215+0000) 2024-06-10T01:13:34.610 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:34 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:34.490+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-06-10T01:13:02.290894+0000 front 2024-06-10T01:13:02.290912+0000 (oldest deadline 2024-06-10T01:13:28.190609+0000) 2024-06-10T01:13:35.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:35 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:35.125+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:35.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:35 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:35.125+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:35.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:35 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:35.125+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:35.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:35 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:35.125+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:35.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:35 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:35.125+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:57.307803+0000 front 2024-06-10T01:12:57.307782+0000 (oldest deadline 2024-06-10T01:13:20.207892+0000) 2024-06-10T01:13:35.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:35 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:35.125+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-06-10T01:13:06.009381+0000 front 2024-06-10T01:13:06.009442+0000 (oldest deadline 2024-06-10T01:13:31.909504+0000) 2024-06-10T01:13:35.858 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:35 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:35.447+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:35.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:35 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:35.447+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:35.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:35 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:35.447+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:35.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:35 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:35.447+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:35.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:35 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:35.447+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.389036+0000 front 2024-06-10T01:12:49.389016+0000 (oldest deadline 2024-06-10T01:13:12.289127+0000) 2024-06-10T01:13:35.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:35 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:35.447+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:56.390413+0000 front 2024-06-10T01:12:56.390285+0000 (oldest deadline 2024-06-10T01:13:22.290215+0000) 2024-06-10T01:13:35.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:35 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:35.447+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-06-10T01:13:02.290894+0000 front 2024-06-10T01:13:02.290912+0000 (oldest deadline 2024-06-10T01:13:28.190609+0000) 2024-06-10T01:13:36.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:36 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:36.142+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:36.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:36 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:36.142+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:36.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:36 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:36.142+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:36.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:36 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:36.142+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:36.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:36 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:36.142+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:57.307803+0000 front 2024-06-10T01:12:57.307782+0000 (oldest deadline 2024-06-10T01:13:20.207892+0000) 2024-06-10T01:13:36.584 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:36 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:36.142+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-06-10T01:13:06.009381+0000 front 2024-06-10T01:13:06.009442+0000 (oldest deadline 2024-06-10T01:13:31.909504+0000) 2024-06-10T01:13:36.584 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:36 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:36.142+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-06-10T01:13:11.910242+0000 front 2024-06-10T01:13:11.910311+0000 (oldest deadline 2024-06-10T01:13:36.010110+0000) 2024-06-10T01:13:36.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:36 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:36.417+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:36.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:36 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:36.417+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:36.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:36 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:36.417+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:36.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:36 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:36.417+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:36.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:36 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:36.417+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.389036+0000 front 2024-06-10T01:12:49.389016+0000 (oldest deadline 2024-06-10T01:13:12.289127+0000) 2024-06-10T01:13:36.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:36 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:36.417+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:56.390413+0000 front 2024-06-10T01:12:56.390285+0000 (oldest deadline 2024-06-10T01:13:22.290215+0000) 2024-06-10T01:13:36.860 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:36 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:36.417+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-06-10T01:13:02.290894+0000 front 2024-06-10T01:13:02.290912+0000 (oldest deadline 2024-06-10T01:13:28.190609+0000) 2024-06-10T01:13:37.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:37 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:37.118+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:37.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:37 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:37.118+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:37.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:37 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:37.118+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:37.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:37 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:37.118+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:37.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:37 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:37.118+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:57.307803+0000 front 2024-06-10T01:12:57.307782+0000 (oldest deadline 2024-06-10T01:13:20.207892+0000) 2024-06-10T01:13:37.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:37 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:37.118+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-06-10T01:13:06.009381+0000 front 2024-06-10T01:13:06.009442+0000 (oldest deadline 2024-06-10T01:13:31.909504+0000) 2024-06-10T01:13:37.584 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:37 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:37.118+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-06-10T01:13:11.910242+0000 front 2024-06-10T01:13:11.910311+0000 (oldest deadline 2024-06-10T01:13:36.010110+0000) 2024-06-10T01:13:37.858 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:37 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:37.419+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:37.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:37 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:37.419+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:37.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:37 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:37.419+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:37.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:37 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:37.419+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:37.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:37 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:37.419+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.389036+0000 front 2024-06-10T01:12:49.389016+0000 (oldest deadline 2024-06-10T01:13:12.289127+0000) 2024-06-10T01:13:37.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:37 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:37.419+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:56.390413+0000 front 2024-06-10T01:12:56.390285+0000 (oldest deadline 2024-06-10T01:13:22.290215+0000) 2024-06-10T01:13:37.859 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:37 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:37.419+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-06-10T01:13:02.290894+0000 front 2024-06-10T01:13:02.290912+0000 (oldest deadline 2024-06-10T01:13:28.190609+0000) 2024-06-10T01:13:38.582 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:38 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:38.151+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:38.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:38 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:38.151+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:38.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:38 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:38.151+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:38.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:38 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:38.151+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:38.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:38 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:38.151+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:57.307803+0000 front 2024-06-10T01:12:57.307782+0000 (oldest deadline 2024-06-10T01:13:20.207892+0000) 2024-06-10T01:13:38.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:38 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:38.151+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-06-10T01:13:06.009381+0000 front 2024-06-10T01:13:06.009442+0000 (oldest deadline 2024-06-10T01:13:31.909504+0000) 2024-06-10T01:13:38.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:38 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:38.151+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-06-10T01:13:11.910242+0000 front 2024-06-10T01:13:11.910311+0000 (oldest deadline 2024-06-10T01:13:36.010110+0000) 2024-06-10T01:13:38.760 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:38 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:38.447+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:13.183747+0000 front 2024-06-10T01:12:13.183769+0000 (oldest deadline 2024-06-10T01:12:37.283937+0000) 2024-06-10T01:13:38.760 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:38 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:38.447+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:25.986028+0000 front 2024-06-10T01:12:25.986052+0000 (oldest deadline 2024-06-10T01:12:50.686193+0000) 2024-06-10T01:13:38.760 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:38 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:38.447+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.687034+0000 front 2024-06-10T01:12:30.686930+0000 (oldest deadline 2024-06-10T01:13:00.087187+0000) 2024-06-10T01:13:38.760 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:38 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:38.447+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-06-10T01:12:40.087858+0000 front 2024-06-10T01:12:40.087707+0000 (oldest deadline 2024-06-10T01:13:03.587672+0000) 2024-06-10T01:13:38.760 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:38 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:38.447+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.389036+0000 front 2024-06-10T01:12:49.389016+0000 (oldest deadline 2024-06-10T01:13:12.289127+0000) 2024-06-10T01:13:38.760 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:38 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:38.447+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:56.390413+0000 front 2024-06-10T01:12:56.390285+0000 (oldest deadline 2024-06-10T01:13:22.290215+0000) 2024-06-10T01:13:38.761 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:38 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10[61742]: 2024-06-10T01:13:38.447+0000 7fb1e0158700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-06-10T01:13:02.290894+0000 front 2024-06-10T01:13:02.290912+0000 (oldest deadline 2024-06-10T01:13:28.190609+0000) 2024-06-10T01:13:39.484 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:39 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:39.200+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:39.484 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:39 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:39.200+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:39.485 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:39 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:39.200+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:39.485 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:39 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:39.200+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:39.485 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:39 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:39.200+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:57.307803+0000 front 2024-06-10T01:12:57.307782+0000 (oldest deadline 2024-06-10T01:13:20.207892+0000) 2024-06-10T01:13:39.485 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:39 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:39.200+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-06-10T01:13:06.009381+0000 front 2024-06-10T01:13:06.009442+0000 (oldest deadline 2024-06-10T01:13:31.909504+0000) 2024-06-10T01:13:39.485 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:39 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:39.200+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-06-10T01:13:11.910242+0000 front 2024-06-10T01:13:11.910311+0000 (oldest deadline 2024-06-10T01:13:36.010110+0000) 2024-06-10T01:13:40.064 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:39 smithi094 podman[73317]: 2024-06-10 01:13:39.819395879 +0000 UTC m=+5.623553706 container died d6e0886ceed64fa33b4238867fcbe0f176d4626128d4ba7e1d7cb6ac887ac024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T01:13:40.323 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:40 smithi094 podman[73317]: 2024-06-10 01:13:40.144567373 +0000 UTC m=+5.948725200 container cleanup d6e0886ceed64fa33b4238867fcbe0f176d4626128d4ba7e1d7cb6ac887ac024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-06-10T01:13:40.323 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:40 smithi094 bash[73317]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10 2024-06-10T01:13:40.501 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:40 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:40.165+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:40.501 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:40 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:40.165+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:40.501 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:40 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:40.165+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:40.501 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:40 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:40.165+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:40.501 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:40 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:40.165+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:57.307803+0000 front 2024-06-10T01:12:57.307782+0000 (oldest deadline 2024-06-10T01:13:20.207892+0000) 2024-06-10T01:13:40.501 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:40 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:40.165+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-06-10T01:13:06.009381+0000 front 2024-06-10T01:13:06.009442+0000 (oldest deadline 2024-06-10T01:13:31.909504+0000) 2024-06-10T01:13:40.502 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:40 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:40.165+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-06-10T01:13:11.910242+0000 front 2024-06-10T01:13:11.910311+0000 (oldest deadline 2024-06-10T01:13:36.010110+0000) 2024-06-10T01:13:40.608 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:40 smithi094 podman[73334]: 2024-06-10 01:13:40.329650175 +0000 UTC m=+0.513360314 container remove d6e0886ceed64fa33b4238867fcbe0f176d4626128d4ba7e1d7cb6ac887ac024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T01:13:40.993 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:40 smithi094 podman[73421]: 2024-06-10 01:13:40.747360848 +0000 UTC m=+0.093771470 container create 8b900b417f2d10fd732ed18f985c988b893483a89c37c4ae3628930a87e8ebef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-06-10T01:13:40.993 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:40 smithi094 podman[73421]: 2024-06-10 01:13:40.670874573 +0000 UTC m=+0.017285190 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:13:40.994 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:40 smithi094 podman[73421]: 2024-06-10 01:13:40.924563951 +0000 UTC m=+0.270974555 container init 8b900b417f2d10fd732ed18f985c988b893483a89c37c4ae3628930a87e8ebef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T01:13:40.994 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:40 smithi094 podman[73421]: 2024-06-10 01:13:40.927486568 +0000 UTC m=+0.273897172 container start 8b900b417f2d10fd732ed18f985c988b893483a89c37c4ae3628930a87e8ebef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-10T01:13:40.994 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:40 smithi094 podman[73421]: 2024-06-10 01:13:40.999527148 +0000 UTC m=+0.345937754 container attach 8b900b417f2d10fd732ed18f985c988b893483a89c37c4ae3628930a87e8ebef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-3d89acc) 2024-06-10T01:13:41.358 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:41 smithi094 podman[73421]: 2024-06-10 01:13:41.215053914 +0000 UTC m=+0.561464520 container died 8b900b417f2d10fd732ed18f985c988b893483a89c37c4ae3628930a87e8ebef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-06-10T01:13:41.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:41 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:41.163+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:41.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:41 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:41.163+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:41.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:41 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:41.163+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:41.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:41 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:41.163+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:41.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:41 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:41.163+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:57.307803+0000 front 2024-06-10T01:12:57.307782+0000 (oldest deadline 2024-06-10T01:13:20.207892+0000) 2024-06-10T01:13:41.584 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:41 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:41.163+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-06-10T01:13:06.009381+0000 front 2024-06-10T01:13:06.009442+0000 (oldest deadline 2024-06-10T01:13:31.909504+0000) 2024-06-10T01:13:41.584 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:41 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:41.163+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-06-10T01:13:11.910242+0000 front 2024-06-10T01:13:11.910311+0000 (oldest deadline 2024-06-10T01:13:36.010110+0000) 2024-06-10T01:13:41.761 INFO:journalctl@ceph.osd.10.smithi094.stdout:Jun 10 01:13:41 smithi094 podman[73447]: 2024-06-10 01:13:41.744757125 +0000 UTC m=+0.523887622 container remove 8b900b417f2d10fd732ed18f985c988b893483a89c37c4ae3628930a87e8ebef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc) 2024-06-10T01:13:41.767 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.10.service' 2024-06-10T01:13:41.805 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:13:41.805 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-06-10T01:13:41.805 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-06-10T01:13:41.806 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.11 2024-06-10T01:13:42.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:41 smithi167 systemd[1]: Stopping Ceph osd.11 for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:13:42.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:42 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:42.108+0000 7fe575453700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:13:42.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:42 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:42.108+0000 7fe575453700 -1 osd.11 109 *** Got signal Terminated *** 2024-06-10T01:13:42.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:42 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:42.108+0000 7fe575453700 -1 osd.11 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-10T01:13:42.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:42 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:42.168+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:42.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:42 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:42.168+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:42.584 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:42 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:42.168+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:42.584 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:42 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:42.168+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:42.584 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:42 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:42.168+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:57.307803+0000 front 2024-06-10T01:12:57.307782+0000 (oldest deadline 2024-06-10T01:13:20.207892+0000) 2024-06-10T01:13:42.584 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:42 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:42.168+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-06-10T01:13:06.009381+0000 front 2024-06-10T01:13:06.009442+0000 (oldest deadline 2024-06-10T01:13:31.909504+0000) 2024-06-10T01:13:42.584 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:42 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:42.168+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-06-10T01:13:11.910242+0000 front 2024-06-10T01:13:11.910311+0000 (oldest deadline 2024-06-10T01:13:36.010110+0000) 2024-06-10T01:13:43.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:43 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:43.166+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:43.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:43 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:43.166+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:43.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:43 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:43.166+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:43.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:43 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:43.166+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:43.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:43 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:43.166+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:57.307803+0000 front 2024-06-10T01:12:57.307782+0000 (oldest deadline 2024-06-10T01:13:20.207892+0000) 2024-06-10T01:13:43.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:43 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:43.166+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-06-10T01:13:06.009381+0000 front 2024-06-10T01:13:06.009442+0000 (oldest deadline 2024-06-10T01:13:31.909504+0000) 2024-06-10T01:13:43.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:43 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:43.166+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-06-10T01:13:11.910242+0000 front 2024-06-10T01:13:11.910311+0000 (oldest deadline 2024-06-10T01:13:36.010110+0000) 2024-06-10T01:13:44.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:44 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:44.201+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:44.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:44 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:44.201+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:44.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:44 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:44.201+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:44.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:44 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:44.201+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:44.584 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:44 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:44.201+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:57.307803+0000 front 2024-06-10T01:12:57.307782+0000 (oldest deadline 2024-06-10T01:13:20.207892+0000) 2024-06-10T01:13:44.584 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:44 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:44.201+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-06-10T01:13:06.009381+0000 front 2024-06-10T01:13:06.009442+0000 (oldest deadline 2024-06-10T01:13:31.909504+0000) 2024-06-10T01:13:44.584 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:44 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:44.201+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-06-10T01:13:11.910242+0000 front 2024-06-10T01:13:11.910311+0000 (oldest deadline 2024-06-10T01:13:36.010110+0000) 2024-06-10T01:13:45.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:45 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:45.211+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:45.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:45 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:45.211+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:45.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:45 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:45.211+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:45.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:45 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:45.211+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:45.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:45 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:45.211+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:57.307803+0000 front 2024-06-10T01:12:57.307782+0000 (oldest deadline 2024-06-10T01:13:20.207892+0000) 2024-06-10T01:13:45.584 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:45 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:45.211+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-06-10T01:13:06.009381+0000 front 2024-06-10T01:13:06.009442+0000 (oldest deadline 2024-06-10T01:13:31.909504+0000) 2024-06-10T01:13:45.584 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:45 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:45.211+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-06-10T01:13:11.910242+0000 front 2024-06-10T01:13:11.910311+0000 (oldest deadline 2024-06-10T01:13:36.010110+0000) 2024-06-10T01:13:46.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:46 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:46.261+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-06-10T01:12:14.299819+0000 front 2024-06-10T01:12:14.299915+0000 (oldest deadline 2024-06-10T01:12:37.199909+0000) 2024-06-10T01:13:46.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:46 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:46.261+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-06-10T01:12:23.601661+0000 front 2024-06-10T01:12:23.601538+0000 (oldest deadline 2024-06-10T01:12:50.602099+0000) 2024-06-10T01:13:46.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:46 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:46.261+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-06-10T01:12:30.602839+0000 front 2024-06-10T01:12:30.602859+0000 (oldest deadline 2024-06-10T01:12:55.902805+0000) 2024-06-10T01:13:46.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:46 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:46.261+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-06-10T01:12:49.806330+0000 front 2024-06-10T01:12:49.806347+0000 (oldest deadline 2024-06-10T01:13:12.106279+0000) 2024-06-10T01:13:46.583 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:46 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:46.261+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-06-10T01:12:57.307803+0000 front 2024-06-10T01:12:57.307782+0000 (oldest deadline 2024-06-10T01:13:20.207892+0000) 2024-06-10T01:13:46.584 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:46 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:46.261+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-06-10T01:13:06.009381+0000 front 2024-06-10T01:13:06.009442+0000 (oldest deadline 2024-06-10T01:13:31.909504+0000) 2024-06-10T01:13:46.584 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:46 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11[62526]: 2024-06-10T01:13:46.261+0000 7fe56da3f700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-06-10T01:13:11.910242+0000 front 2024-06-10T01:13:11.910311+0000 (oldest deadline 2024-06-10T01:13:36.010110+0000) 2024-06-10T01:13:47.777 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:47 smithi167 podman[74856]: 2024-06-10 01:13:47.511445569 +0000 UTC m=+5.516175069 container died dafbd393f1442817cf242ad470399856e2f10c181322eb81ec2486d8eae1ea71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T01:13:48.082 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:47 smithi167 podman[74856]: 2024-06-10 01:13:47.882982214 +0000 UTC m=+5.887711706 container cleanup dafbd393f1442817cf242ad470399856e2f10c181322eb81ec2486d8eae1ea71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-06-10T01:13:48.083 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:47 smithi167 bash[74856]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11 2024-06-10T01:13:48.495 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:48 smithi167 podman[74875]: 2024-06-10 01:13:48.142586818 +0000 UTC m=+0.626747234 container remove dafbd393f1442817cf242ad470399856e2f10c181322eb81ec2486d8eae1ea71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc) 2024-06-10T01:13:48.790 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:48 smithi167 podman[74959]: 2024-06-10 01:13:48.404113425 +0000 UTC m=+0.017118055 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-10T01:13:48.790 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:48 smithi167 podman[74959]: 2024-06-10 01:13:48.502003461 +0000 UTC m=+0.115008081 container create d67f5bfa01c0c2b15936427a843b50b21da145c109b24dba2789951982d99167 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, ceph=True) 2024-06-10T01:13:48.790 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:48 smithi167 podman[74959]: 2024-06-10 01:13:48.713113456 +0000 UTC m=+0.326118087 container init d67f5bfa01c0c2b15936427a843b50b21da145c109b24dba2789951982d99167 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-06-10T01:13:48.790 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:48 smithi167 podman[74959]: 2024-06-10 01:13:48.716703614 +0000 UTC m=+0.329708231 container start d67f5bfa01c0c2b15936427a843b50b21da145c109b24dba2789951982d99167 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-06-10T01:13:48.791 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:48 smithi167 podman[74959]: 2024-06-10 01:13:48.796078784 +0000 UTC m=+0.409083415 container attach d67f5bfa01c0c2b15936427a843b50b21da145c109b24dba2789951982d99167 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-06-10T01:13:49.086 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jun 10 01:13:49 smithi167 podman[74959]: 2024-06-10 01:13:49.012716535 +0000 UTC m=+0.625721154 container died d67f5bfa01c0c2b15936427a843b50b21da145c109b24dba2789951982d99167 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=) 2024-06-10T01:13:49.587 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@osd.11.service' 2024-06-10T01:13:49.626 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:13:49.626 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-06-10T01:13:49.627 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-06-10T01:13:49.627 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.a 2024-06-10T01:13:49.937 INFO:journalctl@ceph.mds.a.smithi037.stdout:Jun 10 01:13:49 smithi037 systemd[1]: Stopping Ceph mds.a for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:13:50.389 INFO:journalctl@ceph.mds.a.smithi037.stdout:Jun 10 01:13:49 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-a[72292]: 2024-06-10T01:13:49.942+0000 7f31037d7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:13:50.390 INFO:journalctl@ceph.mds.a.smithi037.stdout:Jun 10 01:13:49 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-a[72292]: 2024-06-10T01:13:49.942+0000 7f31037d7700 -1 mds.a *** got signal Terminated *** 2024-06-10T01:13:54.029 INFO:journalctl@ceph.mds.a.smithi037.stdout:Jun 10 01:13:53 smithi037 podman[223036]: 2024-06-10 01:13:53.781715903 +0000 UTC m=+3.935834779 container died e6d28d7dca9d73c0ee0ab1821b13faae41b78c69d81c98ef267083b85944e69d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-10T01:13:54.337 INFO:journalctl@ceph.mds.a.smithi037.stdout:Jun 10 01:13:54 smithi037 podman[223036]: 2024-06-10 01:13:54.108734755 +0000 UTC m=+4.262853631 container cleanup e6d28d7dca9d73c0ee0ab1821b13faae41b78c69d81c98ef267083b85944e69d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-06-10T01:13:54.337 INFO:journalctl@ceph.mds.a.smithi037.stdout:Jun 10 01:13:54 smithi037 bash[223036]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-a 2024-06-10T01:13:54.569 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.a.service' 2024-06-10T01:13:54.591 INFO:journalctl@ceph.mds.a.smithi037.stdout:Jun 10 01:13:54 smithi037 podman[223048]: 2024-06-10 01:13:54.34315027 +0000 UTC m=+0.548474053 container remove e6d28d7dca9d73c0ee0ab1821b13faae41b78c69d81c98ef267083b85944e69d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-06-10T01:13:54.591 INFO:journalctl@ceph.mds.a.smithi037.stdout:Jun 10 01:13:54 smithi037 systemd[1]: ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.a.service: Deactivated successfully. 2024-06-10T01:13:54.591 INFO:journalctl@ceph.mds.a.smithi037.stdout:Jun 10 01:13:54 smithi037 systemd[1]: Stopped Ceph mds.a for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T01:13:54.592 INFO:journalctl@ceph.mds.a.smithi037.stdout:Jun 10 01:13:54 smithi037 systemd[1]: ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.a.service: Consumed 1.178s CPU time. 2024-06-10T01:13:54.604 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:13:54.605 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-06-10T01:13:54.605 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-06-10T01:13:54.605 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.d 2024-06-10T01:13:54.890 INFO:journalctl@ceph.mds.d.smithi037.stdout:Jun 10 01:13:54 smithi037 systemd[1]: Stopping Ceph mds.d for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:13:55.390 INFO:journalctl@ceph.mds.d.smithi037.stdout:Jun 10 01:13:54 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-d[71025]: 2024-06-10T01:13:54.937+0000 7fdde9ef6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:13:55.390 INFO:journalctl@ceph.mds.d.smithi037.stdout:Jun 10 01:13:54 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-d[71025]: 2024-06-10T01:13:54.937+0000 7fdde9ef6700 -1 mds.d *** got signal Terminated *** 2024-06-10T01:13:59.268 INFO:journalctl@ceph.mds.d.smithi037.stdout:Jun 10 01:13:58 smithi037 podman[223156]: 2024-06-10 01:13:58.947507859 +0000 UTC m=+4.113563188 container died 2637085aa9ef443944fb215b4ff89b8d17f6867f4e34349ae6e19f4279eafcf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-d, RELEASE=squid-3d89acc, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-06-10T01:13:59.268 INFO:journalctl@ceph.mds.d.smithi037.stdout:Jun 10 01:13:59 smithi037 podman[223156]: 2024-06-10 01:13:59.274437954 +0000 UTC m=+4.440493279 container cleanup 2637085aa9ef443944fb215b4ff89b8d17f6867f4e34349ae6e19f4279eafcf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-d, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-10T01:13:59.557 INFO:journalctl@ceph.mds.d.smithi037.stdout:Jun 10 01:13:59 smithi037 bash[223156]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-d 2024-06-10T01:13:59.840 INFO:journalctl@ceph.mds.d.smithi037.stdout:Jun 10 01:13:59 smithi037 podman[223168]: 2024-06-10 01:13:59.563486808 +0000 UTC m=+0.612135412 container remove 2637085aa9ef443944fb215b4ff89b8d17f6867f4e34349ae6e19f4279eafcf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-d, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528) 2024-06-10T01:13:59.856 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.d.service' 2024-06-10T01:13:59.890 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:13:59.891 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-06-10T01:13:59.891 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-06-10T01:13:59.891 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.g 2024-06-10T01:14:00.139 INFO:journalctl@ceph.mds.g.smithi037.stdout:Jun 10 01:13:59 smithi037 systemd[1]: Stopping Ceph mds.g for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:14:00.640 INFO:journalctl@ceph.mds.g.smithi037.stdout:Jun 10 01:14:00 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-g[71848]: 2024-06-10T01:14:00.240+0000 7f235bee8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:14:00.640 INFO:journalctl@ceph.mds.g.smithi037.stdout:Jun 10 01:14:00 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-g[71848]: 2024-06-10T01:14:00.240+0000 7f235bee8700 -1 mds.g *** got signal Terminated *** 2024-06-10T01:14:01.576 INFO:journalctl@ceph.mds.g.smithi037.stdout:Jun 10 01:14:01 smithi037 podman[223276]: 2024-06-10 01:14:01.256391941 +0000 UTC m=+1.134715707 container died fd0e2ed2a2e400d6f379c5c380b1f5056746853c05057f41b222af481410949c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-g, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux ) 2024-06-10T01:14:01.576 INFO:journalctl@ceph.mds.g.smithi037.stdout:Jun 10 01:14:01 smithi037 podman[223276]: 2024-06-10 01:14:01.582677034 +0000 UTC m=+1.461000799 container cleanup fd0e2ed2a2e400d6f379c5c380b1f5056746853c05057f41b222af481410949c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-g, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T01:14:01.889 INFO:journalctl@ceph.mds.g.smithi037.stdout:Jun 10 01:14:01 smithi037 bash[223276]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-g 2024-06-10T01:14:01.890 INFO:journalctl@ceph.mds.g.smithi037.stdout:Jun 10 01:14:01 smithi037 podman[223288]: 2024-06-10 01:14:01.767271263 +0000 UTC m=+0.503155411 container remove fd0e2ed2a2e400d6f379c5c380b1f5056746853c05057f41b222af481410949c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-g, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-10T01:14:01.999 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.g.service' 2024-06-10T01:14:02.032 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:14:02.032 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-06-10T01:14:02.032 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-06-10T01:14:02.033 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.j 2024-06-10T01:14:02.372 INFO:journalctl@ceph.mds.j.smithi037.stdout:Jun 10 01:14:02 smithi037 systemd[1]: Stopping Ceph mds.j for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:14:02.639 INFO:journalctl@ceph.mds.j.smithi037.stdout:Jun 10 01:14:02 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-j[70587]: 2024-06-10T01:14:02.377+0000 7fcaceac4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:14:02.640 INFO:journalctl@ceph.mds.j.smithi037.stdout:Jun 10 01:14:02 smithi037 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-j[70587]: 2024-06-10T01:14:02.377+0000 7fcaceac4700 -1 mds.j *** got signal Terminated *** 2024-06-10T01:14:03.739 INFO:journalctl@ceph.mds.j.smithi037.stdout:Jun 10 01:14:03 smithi037 podman[223399]: 2024-06-10 01:14:03.493671151 +0000 UTC m=+1.232579034 container died c1217c6e90fe832be6c35f29ca5b54d2962071d3c3e46a92aa965538b9c04ea9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_CLEAN=True) 2024-06-10T01:14:04.081 INFO:journalctl@ceph.mds.j.smithi037.stdout:Jun 10 01:14:03 smithi037 podman[223399]: 2024-06-10 01:14:03.9198361 +0000 UTC m=+1.658743982 container cleanup c1217c6e90fe832be6c35f29ca5b54d2962071d3c3e46a92aa965538b9c04ea9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-j, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True) 2024-06-10T01:14:04.082 INFO:journalctl@ceph.mds.j.smithi037.stdout:Jun 10 01:14:03 smithi037 bash[223399]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-j 2024-06-10T01:14:04.305 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.j.service' 2024-06-10T01:14:04.338 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:14:04.338 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-06-10T01:14:04.338 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-06-10T01:14:04.338 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.b 2024-06-10T01:14:04.623 INFO:journalctl@ceph.mds.b.smithi094.stdout:Jun 10 01:14:04 smithi094 systemd[1]: Stopping Ceph mds.b for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:14:05.108 INFO:journalctl@ceph.mds.b.smithi094.stdout:Jun 10 01:14:04 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-b[67261]: 2024-06-10T01:14:04.628+0000 7f1babcb9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:14:05.108 INFO:journalctl@ceph.mds.b.smithi094.stdout:Jun 10 01:14:04 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-b[67261]: 2024-06-10T01:14:04.628+0000 7f1babcb9700 -1 mds.b *** got signal Terminated *** 2024-06-10T01:14:08.082 INFO:journalctl@ceph.mds.b.smithi094.stdout:Jun 10 01:14:07 smithi094 podman[73526]: 2024-06-10 01:14:07.812884281 +0000 UTC m=+3.281453204 container died 6afed0933a54dae4adba4d24cbeae32f71fc6f3cb5ee0e604c077bd61815e212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux ) 2024-06-10T01:14:08.358 INFO:journalctl@ceph.mds.b.smithi094.stdout:Jun 10 01:14:08 smithi094 podman[73526]: 2024-06-10 01:14:08.171011259 +0000 UTC m=+3.639580181 container cleanup 6afed0933a54dae4adba4d24cbeae32f71fc6f3cb5ee0e604c077bd61815e212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-b, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-06-10T01:14:08.358 INFO:journalctl@ceph.mds.b.smithi094.stdout:Jun 10 01:14:08 smithi094 bash[73526]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-b 2024-06-10T01:14:08.596 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.b.service' 2024-06-10T01:14:08.616 INFO:journalctl@ceph.mds.b.smithi094.stdout:Jun 10 01:14:08 smithi094 podman[73539]: 2024-06-10 01:14:08.472416424 +0000 UTC m=+0.649737004 container remove 6afed0933a54dae4adba4d24cbeae32f71fc6f3cb5ee0e604c077bd61815e212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, ceph=True, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-06-10T01:14:08.616 INFO:journalctl@ceph.mds.b.smithi094.stdout:Jun 10 01:14:08 smithi094 systemd[1]: ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.b.service: Deactivated successfully. 2024-06-10T01:14:08.616 INFO:journalctl@ceph.mds.b.smithi094.stdout:Jun 10 01:14:08 smithi094 systemd[1]: Stopped Ceph mds.b for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T01:14:08.616 INFO:journalctl@ceph.mds.b.smithi094.stdout:Jun 10 01:14:08 smithi094 systemd[1]: ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.b.service: Consumed 3.574s CPU time. 2024-06-10T01:14:08.628 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:14:08.629 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-06-10T01:14:08.629 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-06-10T01:14:08.629 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.e 2024-06-10T01:14:08.935 INFO:journalctl@ceph.mds.e.smithi094.stdout:Jun 10 01:14:08 smithi094 systemd[1]: Stopping Ceph mds.e for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:14:09.358 INFO:journalctl@ceph.mds.e.smithi094.stdout:Jun 10 01:14:08 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-e[66642]: 2024-06-10T01:14:08.940+0000 7f3105802700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:14:09.358 INFO:journalctl@ceph.mds.e.smithi094.stdout:Jun 10 01:14:08 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-e[66642]: 2024-06-10T01:14:08.940+0000 7f3105802700 -1 mds.e *** got signal Terminated *** 2024-06-10T01:14:12.089 INFO:journalctl@ceph.mds.e.smithi094.stdout:Jun 10 01:14:11 smithi094 podman[73646]: 2024-06-10 01:14:11.816707853 +0000 UTC m=+2.966690624 container died e7986ba36127b3a9a4e2d7cbbe74e4e866104e5c0e42c2963baac63e9f73f822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-e, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-06-10T01:14:12.089 INFO:journalctl@ceph.mds.e.smithi094.stdout:Jun 10 01:14:12 smithi094 podman[73646]: 2024-06-10 01:14:12.095450107 +0000 UTC m=+3.245432881 container cleanup e7986ba36127b3a9a4e2d7cbbe74e4e866104e5c0e42c2963baac63e9f73f822 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-e, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-10T01:14:12.340 INFO:journalctl@ceph.mds.e.smithi094.stdout:Jun 10 01:14:12 smithi094 bash[73646]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-e 2024-06-10T01:14:12.502 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.e.service' 2024-06-10T01:14:12.535 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:14:12.536 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-06-10T01:14:12.536 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-06-10T01:14:12.536 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.h 2024-06-10T01:14:12.858 INFO:journalctl@ceph.mds.h.smithi094.stdout:Jun 10 01:14:12 smithi094 systemd[1]: Stopping Ceph mds.h for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:14:13.358 INFO:journalctl@ceph.mds.h.smithi094.stdout:Jun 10 01:14:12 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-h[66959]: 2024-06-10T01:14:12.874+0000 7f088bc02700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:14:13.358 INFO:journalctl@ceph.mds.h.smithi094.stdout:Jun 10 01:14:12 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-h[66959]: 2024-06-10T01:14:12.874+0000 7f088bc02700 -1 mds.h *** got signal Terminated *** 2024-06-10T01:14:16.070 INFO:journalctl@ceph.mds.h.smithi094.stdout:Jun 10 01:14:15 smithi094 podman[73778]: 2024-06-10 01:14:15.814814661 +0000 UTC m=+3.055268178 container died d3cac041c120c76eaf9883d1718c0b12e643bf82faaa4931a2d0ebd0f2acad0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-06-10T01:14:16.358 INFO:journalctl@ceph.mds.h.smithi094.stdout:Jun 10 01:14:16 smithi094 podman[73778]: 2024-06-10 01:14:16.141041231 +0000 UTC m=+3.381494756 container cleanup d3cac041c120c76eaf9883d1718c0b12e643bf82faaa4931a2d0ebd0f2acad0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-h, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240528) 2024-06-10T01:14:16.359 INFO:journalctl@ceph.mds.h.smithi094.stdout:Jun 10 01:14:16 smithi094 bash[73778]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-h 2024-06-10T01:14:16.617 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.h.service' 2024-06-10T01:14:16.637 INFO:journalctl@ceph.mds.h.smithi094.stdout:Jun 10 01:14:16 smithi094 podman[73790]: 2024-06-10 01:14:16.376233149 +0000 UTC m=+0.553692200 container remove d3cac041c120c76eaf9883d1718c0b12e643bf82faaa4931a2d0ebd0f2acad0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux ) 2024-06-10T01:14:16.637 INFO:journalctl@ceph.mds.h.smithi094.stdout:Jun 10 01:14:16 smithi094 systemd[1]: ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.h.service: Deactivated successfully. 2024-06-10T01:14:16.637 INFO:journalctl@ceph.mds.h.smithi094.stdout:Jun 10 01:14:16 smithi094 systemd[1]: Stopped Ceph mds.h for 68793514-26c2-11ef-bc9b-c7b262605968. 2024-06-10T01:14:16.637 INFO:journalctl@ceph.mds.h.smithi094.stdout:Jun 10 01:14:16 smithi094 systemd[1]: ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.h.service: Consumed 2.477s CPU time. 2024-06-10T01:14:16.648 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:14:16.648 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-06-10T01:14:16.649 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-06-10T01:14:16.649 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.k 2024-06-10T01:14:17.006 INFO:journalctl@ceph.mds.k.smithi094.stdout:Jun 10 01:14:16 smithi094 systemd[1]: Stopping Ceph mds.k for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:14:17.358 INFO:journalctl@ceph.mds.k.smithi094.stdout:Jun 10 01:14:17 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-k[66328]: 2024-06-10T01:14:17.012+0000 7f8731fd9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:14:17.359 INFO:journalctl@ceph.mds.k.smithi094.stdout:Jun 10 01:14:17 smithi094 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-k[66328]: 2024-06-10T01:14:17.012+0000 7f8731fd9700 -1 mds.k *** got signal Terminated *** 2024-06-10T01:14:19.473 INFO:journalctl@ceph.mds.k.smithi094.stdout:Jun 10 01:14:19 smithi094 podman[73900]: 2024-06-10 01:14:19.212107265 +0000 UTC m=+2.332297152 container died 2b9d0a4608f39d83ee5f09b073a3697b09aecd60be9cb391c3dba1ab4c7e6c68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-k, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528) 2024-06-10T01:14:19.840 INFO:journalctl@ceph.mds.k.smithi094.stdout:Jun 10 01:14:19 smithi094 podman[73900]: 2024-06-10 01:14:19.554093955 +0000 UTC m=+2.674283845 container cleanup 2b9d0a4608f39d83ee5f09b073a3697b09aecd60be9cb391c3dba1ab4c7e6c68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-k, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, io.buildah.version=1.33.5) 2024-06-10T01:14:19.840 INFO:journalctl@ceph.mds.k.smithi094.stdout:Jun 10 01:14:19 smithi094 bash[73900]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-k 2024-06-10T01:14:19.979 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.k.service' 2024-06-10T01:14:20.009 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:14:20.010 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-06-10T01:14:20.010 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-06-10T01:14:20.010 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.c 2024-06-10T01:14:20.333 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jun 10 01:14:20 smithi167 systemd[1]: Stopping Ceph mds.c for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:14:20.597 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jun 10 01:14:20 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-c[67682]: 2024-06-10T01:14:20.508+0000 7fc689e7b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:14:20.597 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jun 10 01:14:20 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-c[67682]: 2024-06-10T01:14:20.508+0000 7fc689e7b700 -1 mds.c *** got signal Terminated *** 2024-06-10T01:14:24.295 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jun 10 01:14:23 smithi167 podman[75063]: 2024-06-10 01:14:23.923855898 +0000 UTC m=+3.714548223 container died 73324939eab34dccc1ac6a85e525934bddc3f3e4cc136a24a2d8632ededab4c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-10T01:14:24.582 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jun 10 01:14:24 smithi167 podman[75063]: 2024-06-10 01:14:24.399723607 +0000 UTC m=+4.190415923 container cleanup 73324939eab34dccc1ac6a85e525934bddc3f3e4cc136a24a2d8632ededab4c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-c, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-10T01:14:24.583 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jun 10 01:14:24 smithi167 bash[75063]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-c 2024-06-10T01:14:24.824 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.c.service' 2024-06-10T01:14:24.859 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:14:24.859 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-06-10T01:14:24.859 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-06-10T01:14:24.859 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.f 2024-06-10T01:14:25.181 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:14:24 smithi167 systemd[1]: Stopping Ceph mds.f for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:14:25.582 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:14:25 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-f[72143]: 2024-06-10T01:14:25.186+0000 7ff7bc315700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:14:25.583 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:14:25 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-f[72143]: 2024-06-10T01:14:25.186+0000 7ff7bc315700 -1 mds.f *** got signal Terminated *** 2024-06-10T01:14:28.473 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:14:28 smithi167 podman[75196]: 2024-06-10 01:14:28.20712818 +0000 UTC m=+3.125653405 container died b7fd438c57f54b9d65c7146eaae64f591f495d7c99208d55f34f7db38241db64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-f, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528) 2024-06-10T01:14:28.828 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:14:28 smithi167 podman[75196]: 2024-06-10 01:14:28.562035066 +0000 UTC m=+3.480560286 container cleanup b7fd438c57f54b9d65c7146eaae64f591f495d7c99208d55f34f7db38241db64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, ceph=True) 2024-06-10T01:14:28.828 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jun 10 01:14:28 smithi167 bash[75196]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-f 2024-06-10T01:14:28.977 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.f.service' 2024-06-10T01:14:29.011 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:14:29.012 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-06-10T01:14:29.012 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-06-10T01:14:29.012 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.i 2024-06-10T01:14:29.392 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jun 10 01:14:29 smithi167 systemd[1]: Stopping Ceph mds.i for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:14:29.833 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jun 10 01:14:29 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-i[67988]: 2024-06-10T01:14:29.397+0000 7f1f9954f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:14:29.833 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jun 10 01:14:29 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-i[67988]: 2024-06-10T01:14:29.397+0000 7f1f9954f700 -1 mds.i *** got signal Terminated *** 2024-06-10T01:14:32.215 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jun 10 01:14:31 smithi167 podman[75317]: 2024-06-10 01:14:31.940152037 +0000 UTC m=+2.707252761 container died 016d4410edc1fdb0573a1a1e7e6369c152269f74d2b1ac9a301a8728a02ed237 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-i, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=) 2024-06-10T01:14:32.566 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jun 10 01:14:32 smithi167 podman[75317]: 2024-06-10 01:14:32.312375086 +0000 UTC m=+3.079475807 container cleanup 016d4410edc1fdb0573a1a1e7e6369c152269f74d2b1ac9a301a8728a02ed237 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-i, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-06-10T01:14:32.566 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jun 10 01:14:32 smithi167 bash[75317]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-i 2024-06-10T01:14:32.724 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.i.service' 2024-06-10T01:14:32.756 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:14:32.756 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-06-10T01:14:32.756 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-06-10T01:14:32.756 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.l 2024-06-10T01:14:33.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jun 10 01:14:32 smithi167 systemd[1]: Stopping Ceph mds.l for 68793514-26c2-11ef-bc9b-c7b262605968... 2024-06-10T01:14:33.332 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jun 10 01:14:33 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-l[67064]: 2024-06-10T01:14:33.081+0000 7f92b6ce4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-10T01:14:33.333 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jun 10 01:14:33 smithi167 ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-l[67064]: 2024-06-10T01:14:33.081+0000 7f92b6ce4700 -1 mds.l *** got signal Terminated *** 2024-06-10T01:14:36.253 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jun 10 01:14:35 smithi167 podman[75438]: 2024-06-10 01:14:35.923464033 +0000 UTC m=+2.937363657 container died 035edf753fae6fde9869cd72f24fc7754dcb105de607f1de30c3ddffe6381785 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-l, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, ceph=True) 2024-06-10T01:14:36.582 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jun 10 01:14:36 smithi167 podman[75438]: 2024-06-10 01:14:36.341779653 +0000 UTC m=+3.355679299 container cleanup 035edf753fae6fde9869cd72f24fc7754dcb105de607f1de30c3ddffe6381785 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-l, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-3d89acc, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-06-10T01:14:36.583 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jun 10 01:14:36 smithi167 bash[75438]: ceph-68793514-26c2-11ef-bc9b-c7b262605968-mds-l 2024-06-10T01:14:36.816 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-68793514-26c2-11ef-bc9b-c7b262605968@mds.l.service' 2024-06-10T01:14:36.847 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-10T01:14:36.847 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-06-10T01:14:36.847 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 68793514-26c2-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-10T01:14:37.048 INFO:teuthology.orchestra.run.smithi037.stdout:Deleting cluster with fsid: 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T01:14:39.873 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 68793514-26c2-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-10T01:14:40.061 INFO:teuthology.orchestra.run.smithi094.stdout:Deleting cluster with fsid: 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T01:14:42.521 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 68793514-26c2-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-10T01:14:42.949 INFO:teuthology.orchestra.run.smithi167.stdout:Deleting cluster with fsid: 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T01:14:45.458 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-10T01:14:45.493 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-10T01:14:45.525 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-10T01:14:45.550 INFO:tasks.cephadm:Archiving crash dumps... 2024-06-10T01:14:45.552 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747344/remote/smithi037/crash 2024-06-10T01:14:45.553 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/crash -- . 2024-06-10T01:14:45.597 INFO:teuthology.orchestra.run.smithi037.stderr:tar: /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-10T01:14:45.597 INFO:teuthology.orchestra.run.smithi037.stderr:tar: Error is not recoverable: exiting now 2024-06-10T01:14:45.599 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747344/remote/smithi094/crash 2024-06-10T01:14:45.600 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/crash -- . 2024-06-10T01:14:45.653 INFO:teuthology.orchestra.run.smithi094.stderr:tar: /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-10T01:14:45.653 INFO:teuthology.orchestra.run.smithi094.stderr:tar: Error is not recoverable: exiting now 2024-06-10T01:14:45.656 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747344/remote/smithi167/crash 2024-06-10T01:14:45.657 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/crash -- . 2024-06-10T01:14:45.688 INFO:teuthology.orchestra.run.smithi167.stderr:tar: /var/lib/ceph/68793514-26c2-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-10T01:14:45.689 INFO:teuthology.orchestra.run.smithi167.stderr:tar: Error is not recoverable: exiting now 2024-06-10T01:14:45.690 INFO:tasks.cephadm:Checking cluster log for badness... 2024-06-10T01:14:45.690 DEBUG:teuthology.orchestra.run.smithi037:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-10T01:14:45.778 INFO:teuthology.orchestra.run.smithi037.stdout:2024-06-10T01:10:00.000100+0000 mon.a (mon.0) 1317 : cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-06-10T01:14:45.778 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-06-10T01:14:45.778 INFO:tasks.cephadm:Compressing logs... 2024-06-10T01:14:45.779 DEBUG:teuthology.orchestra.run.smithi037:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-10T01:14:45.781 DEBUG:teuthology.orchestra.run.smithi094:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-10T01:14:45.784 DEBUG:teuthology.orchestra.run.smithi167:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-10T01:14:45.812 INFO:teuthology.orchestra.run.smithi094.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-10T01:14:45.812 INFO:teuthology.orchestra.run.smithi094.stderr:: No such file or directory 2024-06-10T01:14:45.813 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph.log 2024-06-10T01:14:45.813 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-10T01:14:45.813 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.7.log 2024-06-10T01:14:45.813 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.b.log 2024-06-10T01:14:45.813 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.4.log 2024-06-10T01:14:45.813 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.7.log: /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.b.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-10T01:14:45.814 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.1.log 2024-06-10T01:14:45.814 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mgr.y.log 2024-06-10T01:14:45.814 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.h.log 2024-06-10T01:14:45.814 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.10.log 2024-06-10T01:14:45.817 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.h.log: 90.4% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-10T01:14:45.817 INFO:teuthology.orchestra.run.smithi094.stderr: 87.0% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-10T01:14:45.817 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-10T01:14:45.817 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.e.log 2024-06-10T01:14:45.818 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph.cephadm.log: 83.5% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-10T01:14:45.818 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-10T01:14:45.822 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.e.log: 93.2% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mgr.y.log.gz 2024-06-10T01:14:45.822 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mon.b.log 2024-06-10T01:14:45.827 INFO:teuthology.orchestra.run.smithi037.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose: No such file or directory -- 2024-06-10T01:14:45.827 INFO:teuthology.orchestra.run.smithi037.stderr: /var/log/ceph/cephadm.log 2024-06-10T01:14:45.833 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-06-10T01:14:45.834 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph.log 2024-06-10T01:14:45.834 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-10T01:14:45.834 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.g.log 2024-06-10T01:14:45.834 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.j.log 2024-06-10T01:14:45.838 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.3.log 2024-06-10T01:14:45.838 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.0.log 2024-06-10T01:14:45.839 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mgr.x.log 2024-06-10T01:14:45.839 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mon.a.log 2024-06-10T01:14:45.839 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.6.log 2024-06-10T01:14:45.839 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-10T01:14:45.839 INFO:teuthology.orchestra.run.smithi037.stderr: 86.9% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-10T01:14:45.840 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.a.log 2024-06-10T01:14:45.840 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-10T01:14:45.840 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.d.log 2024-06-10T01:14:45.840 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.9.log 2024-06-10T01:14:45.841 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.k.log 2024-06-10T01:14:45.842 INFO:teuthology.orchestra.run.smithi094.stderr: 91.0% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-10T01:14:45.844 INFO:teuthology.orchestra.run.smithi167.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-10T01:14:45.845 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-10T01:14:45.845 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph.log 2024-06-10T01:14:45.848 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.2.log 2024-06-10T01:14:45.849 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.f.log 2024-06-10T01:14:45.849 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-10T01:14:45.849 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.5.log 2024-06-10T01:14:45.849 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose --/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph.audit.log: /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.c.log 2024-06-10T01:14:45.849 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mon.c.log 2024-06-10T01:14:45.849 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mgr.z.log 2024-06-10T01:14:45.849 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.c.log: gzip -5 --verbose --/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mon.c.log: /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.i.log 2024-06-10T01:14:45.849 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-10T01:14:45.850 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.11.log 2024-06-10T01:14:45.850 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph.cephadm.log: -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.8.log 2024-06-10T01:14:45.850 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-10T01:14:45.850 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.8.log: -- /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.l.log 2024-06-10T01:14:45.862 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mon.b.log: /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.k.log: 94.5% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.k.log.gz 2024-06-10T01:14:45.897 INFO:teuthology.orchestra.run.smithi094.stderr: 94.7% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.h.log.gz 2024-06-10T01:14:45.902 INFO:teuthology.orchestra.run.smithi094.stderr: 94.6% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.e.log.gz 2024-06-10T01:14:45.906 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-volume.log: /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.l.log: 83.4% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-10T01:14:45.937 INFO:teuthology.orchestra.run.smithi167.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-10T01:14:45.953 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.d.log: /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.9.log: 90.2% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-10T01:14:45.985 INFO:teuthology.orchestra.run.smithi037.stderr: 83.4% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-10T01:14:46.014 INFO:teuthology.orchestra.run.smithi037.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-06-10T01:14:46.072 INFO:teuthology.orchestra.run.smithi037.stderr: 94.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-10T01:14:46.112 INFO:teuthology.orchestra.run.smithi167.stderr: 93.3% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mgr.z.log.gz 2024-06-10T01:14:46.130 INFO:teuthology.orchestra.run.smithi037.stderr: 94.7% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.j.log.gz 2024-06-10T01:14:46.179 INFO:teuthology.orchestra.run.smithi167.stderr: 91.0% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-10T01:14:46.197 INFO:teuthology.orchestra.run.smithi037.stderr: 94.6% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.g.log.gz 2024-06-10T01:14:46.199 INFO:teuthology.orchestra.run.smithi037.stderr: 94.7% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.a.log.gz 2024-06-10T01:14:46.214 INFO:teuthology.orchestra.run.smithi094.stderr: 94.2% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.b.log.gz 2024-06-10T01:14:46.233 INFO:teuthology.orchestra.run.smithi037.stderr: 94.7% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.d.log.gz 2024-06-10T01:14:46.342 INFO:teuthology.orchestra.run.smithi037.stderr: 91.1% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-10T01:14:46.343 INFO:teuthology.orchestra.run.smithi167.stderr: 90.4% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-10T01:14:46.373 INFO:teuthology.orchestra.run.smithi094.stderr: 91.8% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mon.b.log.gz 2024-06-10T01:14:46.397 INFO:teuthology.orchestra.run.smithi167.stderr: 87.0% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-10T01:14:47.325 INFO:teuthology.orchestra.run.smithi037.stderr: 90.1% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mgr.x.log.gz 2024-06-10T01:14:48.336 INFO:teuthology.orchestra.run.smithi167.stderr: 91.8% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mon.c.log.gz 2024-06-10T01:14:52.566 INFO:teuthology.orchestra.run.smithi037.stderr: 90.2% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mon.a.log.gz 2024-06-10T01:15:01.612 INFO:teuthology.orchestra.run.smithi094.stderr: 92.0% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.10.log.gz 2024-06-10T01:15:02.713 INFO:teuthology.orchestra.run.smithi167.stderr: 94.2% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.c.log.gz 2024-06-10T01:15:05.034 INFO:teuthology.orchestra.run.smithi094.stderr: 92.2% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.1.log.gz 2024-06-10T01:15:07.885 INFO:teuthology.orchestra.run.smithi094.stderr: 92.0% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.7.log.gz 2024-06-10T01:15:12.213 INFO:teuthology.orchestra.run.smithi037.stderr: 92.0% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.0.log.gz 2024-06-10T01:15:14.227 INFO:teuthology.orchestra.run.smithi037.stderr: 92.0% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.6.log.gz 2024-06-10T01:15:15.035 INFO:teuthology.orchestra.run.smithi094.stderr: 92.2% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.4.log.gz 2024-06-10T01:15:15.037 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-06-10T01:15:15.037 INFO:teuthology.orchestra.run.smithi094.stderr:real 0m29.239s 2024-06-10T01:15:15.037 INFO:teuthology.orchestra.run.smithi094.stderr:user 1m24.565s 2024-06-10T01:15:15.038 INFO:teuthology.orchestra.run.smithi094.stderr:sys 0m2.762s 2024-06-10T01:15:15.916 INFO:teuthology.orchestra.run.smithi037.stderr: 92.1% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.9.log.gz 2024-06-10T01:15:16.138 INFO:teuthology.orchestra.run.smithi037.stderr: 92.0% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.3.log.gz 2024-06-10T01:15:16.140 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-06-10T01:15:16.140 INFO:teuthology.orchestra.run.smithi037.stderr:real 0m30.344s 2024-06-10T01:15:16.140 INFO:teuthology.orchestra.run.smithi037.stderr:user 1m28.479s 2024-06-10T01:15:16.140 INFO:teuthology.orchestra.run.smithi037.stderr:sys 0m2.726s 2024-06-10T01:17:03.913 INFO:teuthology.orchestra.run.smithi167.stderr: 94.8% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.f.log.gz 2024-06-10T01:18:41.353 INFO:teuthology.orchestra.run.smithi167.stderr: 92.1% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.11.log.gz 2024-06-10T01:18:55.336 INFO:teuthology.orchestra.run.smithi167.stderr: 94.0% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.l.log.gz 2024-06-10T01:19:38.492 INFO:teuthology.orchestra.run.smithi167.stderr: 91.9% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.8.log.gz 2024-06-10T01:20:16.971 INFO:teuthology.orchestra.run.smithi167.stderr: 92.1% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.2.log.gz 2024-06-10T01:20:28.589 INFO:teuthology.orchestra.run.smithi167.stderr: 91.8% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-osd.5.log.gz 2024-06-10T01:24:16.053 INFO:teuthology.orchestra.run.smithi167.stderr: 93.3% -- replaced with /var/log/ceph/68793514-26c2-11ef-bc9b-c7b262605968/ceph-mds.i.log.gz 2024-06-10T01:24:16.055 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-06-10T01:24:16.055 INFO:teuthology.orchestra.run.smithi167.stderr:real 9m30.256s 2024-06-10T01:24:16.055 INFO:teuthology.orchestra.run.smithi167.stderr:user 5m47.328s 2024-06-10T01:24:16.056 INFO:teuthology.orchestra.run.smithi167.stderr:sys 0m13.451s 2024-06-10T01:24:16.056 INFO:tasks.cephadm:Archiving logs... 2024-06-10T01:24:16.057 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747344/remote/smithi037/log 2024-06-10T01:24:16.059 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-10T01:24:30.145 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747344/remote/smithi094/log 2024-06-10T01:24:30.146 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-10T01:24:43.901 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747344/remote/smithi167/log 2024-06-10T01:24:43.902 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-10T01:25:37.337 INFO:tasks.cephadm:Removing cluster... 2024-06-10T01:25:37.338 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 68793514-26c2-11ef-bc9b-c7b262605968 --force 2024-06-10T01:25:37.541 INFO:teuthology.orchestra.run.smithi037.stdout:Deleting cluster with fsid: 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T01:25:37.931 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 68793514-26c2-11ef-bc9b-c7b262605968 --force 2024-06-10T01:25:38.120 INFO:teuthology.orchestra.run.smithi094.stdout:Deleting cluster with fsid: 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T01:25:38.563 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 68793514-26c2-11ef-bc9b-c7b262605968 --force 2024-06-10T01:25:39.038 INFO:teuthology.orchestra.run.smithi167.stdout:Deleting cluster with fsid: 68793514-26c2-11ef-bc9b-c7b262605968 2024-06-10T01:25:39.648 INFO:tasks.cephadm:Removing cephadm ... 2024-06-10T01:25:39.649 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-10T01:25:39.669 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-10T01:25:39.687 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-10T01:25:39.703 INFO:tasks.cephadm:Teardown complete 2024-06-10T01:25:39.703 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-06-10T01:25:39.715 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-06-10T01:25:39.715 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-10T01:25:39.718 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-10T01:25:39.730 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-10T01:25:39.847 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-06-10T01:25:39.857 INFO:teuthology.task.clock:Checking final clock skew... 2024-06-10T01:25:39.857 DEBUG:teuthology.orchestra.run.smithi037:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-10T01:25:39.860 DEBUG:teuthology.orchestra.run.smithi094:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-10T01:25:39.863 DEBUG:teuthology.orchestra.run.smithi167:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-10T01:25:39.876 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-06-10T01:25:39.876 INFO:teuthology.orchestra.run.smithi167.stderr:bash: line 1: ntpq: command not found 2024-06-10T01:25:39.877 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-06-10T01:25:39.880 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-10T01:25:39.880 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-06-10T01:25:39.881 INFO:teuthology.orchestra.run.smithi094.stdout:^* hv01.front.sepia.ceph.com 3 7 377 6 +252us[ +276us] +/- 63ms 2024-06-10T01:25:39.881 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 6 -802us[ -802us] +/- 59ms 2024-06-10T01:25:39.881 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 8 +913us[ +937us] +/- 87ms 2024-06-10T01:25:39.881 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T01:25:39.882 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-10T01:25:39.883 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-06-10T01:25:39.883 INFO:teuthology.orchestra.run.smithi037.stdout:^* hv01.front.sepia.ceph.com 3 6 377 8 +224us[ +233us] +/- 63ms 2024-06-10T01:25:39.883 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 9 -814us[ -805us] +/- 59ms 2024-06-10T01:25:39.883 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 8 +916us[ +916us] +/- 87ms 2024-06-10T01:25:39.883 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T01:25:39.891 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-10T01:25:39.891 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2024-06-10T01:25:39.891 INFO:teuthology.orchestra.run.smithi167.stdout:^* hv01.front.sepia.ceph.com 3 6 377 11 +196us[ +175us] +/- 63ms 2024-06-10T01:25:39.891 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 5 -814us[ -814us] +/- 59ms 2024-06-10T01:25:39.891 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 6 +904us[ +904us] +/- 87ms 2024-06-10T01:25:39.891 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-10T01:25:39.892 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-06-10T01:25:39.903 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-06-10T01:25:39.903 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-06-10T01:25:39.925 DEBUG:teuthology.orchestra.run.smithi037:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-10T01:25:40.084 DEBUG:teuthology.orchestra.run.smithi094:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-10T01:25:40.160 DEBUG:teuthology.orchestra.run.smithi167:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-10T01:25:40.268 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-10T01:25:40.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T01:25:40.303 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-10T01:25:40.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T01:25:40.332 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-10T01:25:40.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T01:25:40.363 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-06-10T01:25:40.374 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-06-10T01:25:40.469 INFO:teuthology.task.internal:Duration was 2950.310811 seconds 2024-06-10T01:25:40.470 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-06-10T01:25:40.509 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-06-10T01:25:40.509 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-10T01:25:40.512 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-10T01:25:40.514 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-10T01:25:40.553 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-10T01:25:40.583 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-10T01:25:40.588 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-10T01:25:40.975 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-06-10T01:25:40.975 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi037.front.sepia.ceph.com 2024-06-10T01:25:40.975 DEBUG:teuthology.orchestra.run.smithi037:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-10T01:25:41.001 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi094.front.sepia.ceph.com 2024-06-10T01:25:41.001 DEBUG:teuthology.orchestra.run.smithi094:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-10T01:25:41.035 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi167.front.sepia.ceph.com 2024-06-10T01:25:41.035 DEBUG:teuthology.orchestra.run.smithi167:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-10T01:25:41.069 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-06-10T01:25:41.069 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-10T01:25:41.072 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-10T01:25:41.078 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-10T01:25:41.140 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-06-10T01:25:41.140 DEBUG:teuthology.orchestra.run.smithi037:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-10T01:25:41.143 DEBUG:teuthology.orchestra.run.smithi094:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-10T01:25:41.148 DEBUG:teuthology.orchestra.run.smithi167:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-10T01:25:42.374 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-06-10T01:25:42.386 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-06-10T01:25:42.386 DEBUG:teuthology.orchestra.run.smithi037:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-10T01:25:42.412 DEBUG:teuthology.orchestra.run.smithi094:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-10T01:25:42.440 DEBUG:teuthology.orchestra.run.smithi167:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-10T01:25:42.481 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-06-10T01:25:42.492 DEBUG:teuthology.orchestra.run.smithi037:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-10T01:25:42.494 DEBUG:teuthology.orchestra.run.smithi094:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-10T01:25:42.497 DEBUG:teuthology.orchestra.run.smithi167:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-10T01:25:42.520 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = core 2024-06-10T01:25:42.521 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = core 2024-06-10T01:25:42.552 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = core 2024-06-10T01:25:42.570 DEBUG:teuthology.orchestra.run.smithi037:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-10T01:25:42.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T01:25:42.601 DEBUG:teuthology.orchestra.run.smithi094:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-10T01:25:42.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T01:25:42.617 DEBUG:teuthology.orchestra.run.smithi167:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-10T01:25:42.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-10T01:25:42.633 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-06-10T01:25:42.644 INFO:teuthology.task.internal:Transferring archived files... 2024-06-10T01:25:42.646 DEBUG:teuthology.misc:Transferring archived files from smithi037:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747344/remote/smithi037 2024-06-10T01:25:42.646 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-10T01:25:42.710 DEBUG:teuthology.misc:Transferring archived files from smithi094:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747344/remote/smithi094 2024-06-10T01:25:42.710 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-10T01:25:42.748 DEBUG:teuthology.misc:Transferring archived files from smithi167:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-09_21:24:02-fs-squid-distro-default-smithi/7747344/remote/smithi167 2024-06-10T01:25:42.748 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-10T01:25:42.786 INFO:teuthology.task.internal:Removing archive directory... 2024-06-10T01:25:42.786 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-10T01:25:42.789 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-10T01:25:42.791 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-10T01:25:42.836 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-06-10T01:25:42.849 INFO:teuthology.task.internal:Not uploading archives. 2024-06-10T01:25:42.849 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-06-10T01:25:42.867 INFO:teuthology.task.internal:Tidying up after the test... 2024-06-10T01:25:42.867 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-10T01:25:42.870 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-10T01:25:42.872 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-10T01:25:42.883 INFO:teuthology.orchestra.run.smithi037.stdout: 265305 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 10 01:25 /home/ubuntu/cephtest 2024-06-10T01:25:42.886 INFO:teuthology.orchestra.run.smithi094.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 10 01:25 /home/ubuntu/cephtest 2024-06-10T01:25:42.891 INFO:teuthology.orchestra.run.smithi167.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 10 01:25 /home/ubuntu/cephtest 2024-06-10T01:25:42.896 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-06-10T01:25:42.908 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-06-10T01:25:42.973 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/misc}} duration: 2950.3108105659485 failure_reason: 'Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi037 with status 11: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b7a5d375cf4e4afab96b9d4d229b699ae8304bb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/multiple_rsync.sh''' flavor: default owner: scheduled_teuthology@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=20d7eaa86a65426c83a27ddfa1f5d0ab status: fail success: false 2024-06-10T01:25:42.973 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-10T01:25:43.066 INFO:teuthology.run:FAIL